2024-08-31T06:51:59.436 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-08-31T06:51:59.461 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-31T06:51:59.553 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448 branch: wip-mchangir-testing-20240828.085847-main-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: mchangir@redhat.com first_in_suite: false job_id: '7881448' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-20240828.085847-main-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-20240828.085847-main-debug sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 owner: scheduled_mchangir@teuthology priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5413 sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 sleep_before_teardown: 0 subset: 347/512 suite: fs suite_branch: wip-mchangir-testing-20240828.085847-main-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFVFIKESmF58AZqH1kSeTvUGEHzSlXUYyTOw4qkRYJUFdtrVjFpQ1ZvWhBlD4UUqJwOHsSOGghyeGpIXdYn6uLY= smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnBG2ZYJi4Q7yIdXwiidcBP2NKT2L1HWxiGEhOpvrIyNuaFWXKW/TWNtZp4wOvw0TxtVURWWvjOA/sGr3y5rhM= smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDiXb0kTISyt8l1yaF+VYjlABI3ZNEBYwK1ClNcY0Yhx9KfyPG0/8o0VDI3zpn1yzlUqBFU7lRkF5MD8nHUhHmI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-08-30_08:30:37 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-31T06:51:59.553 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa; will attempt to use it 2024-08-31T06:51:59.554 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks 2024-08-31T06:51:59.554 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-31T06:51:59.556 INFO:teuthology.task.internal:Checking packages... 2024-08-31T06:51:59.578 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd19e6118f7015f0047264006cb0c873528eb3ae6' 2024-08-31T06:51:59.578 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-31T06:51:59.578 INFO:teuthology.packaging:ref: None 2024-08-31T06:51:59.578 INFO:teuthology.packaging:tag: None 2024-08-31T06:51:59.578 INFO:teuthology.packaging:branch: wip-mchangir-testing-20240828.085847-main-debug 2024-08-31T06:51:59.578 INFO:teuthology.packaging:sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T06:51:59.578 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-20240828.085847-main-debug 2024-08-31T06:51:59.729 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4560.gd19e6118 2024-08-31T06:51:59.731 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-31T06:51:59.736 INFO:teuthology.task.internal:no buildpackages task found 2024-08-31T06:51:59.736 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-31T06:51:59.819 INFO:teuthology.task.internal:Saving configuration 2024-08-31T06:51:59.838 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-31T06:51:59.844 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-31T06:51:59.869 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-31 06:46:54.661851', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFVFIKESmF58AZqH1kSeTvUGEHzSlXUYyTOw4qkRYJUFdtrVjFpQ1ZvWhBlD4UUqJwOHsSOGghyeGpIXdYn6uLY='} 2024-08-31T06:51:59.896 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-31 06:46:54.663414', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnBG2ZYJi4Q7yIdXwiidcBP2NKT2L1HWxiGEhOpvrIyNuaFWXKW/TWNtZp4wOvw0TxtVURWWvjOA/sGr3y5rhM='} 2024-08-31T06:51:59.918 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-31 06:46:54.664774', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDiXb0kTISyt8l1yaF+VYjlABI3ZNEBYwK1ClNcY0Yhx9KfyPG0/8o0VDI3zpn1yzlUqBFU7lRkF5MD8nHUhHmI='} 2024-08-31T06:51:59.918 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-31T06:51:59.924 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-31T06:51:59.924 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-31T06:51:59.924 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-31T06:51:59.924 INFO:teuthology.run_tasks:Running task console_log... 2024-08-31T06:52:00.024 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f46af4dfac0>, signals=[15]) 2024-08-31T06:52:00.024 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-31T06:52:00.029 INFO:teuthology.task.internal:Opening connections... 2024-08-31T06:52:00.030 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-08-31T06:52:00.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T06:52:00.106 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-08-31T06:52:00.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T06:52:00.182 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2024-08-31T06:52:00.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T06:52:00.255 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-31T06:52:00.261 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-08-31T06:52:00.279 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-08-31T06:52:00.279 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-08-31T06:52:00.337 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="CentOS Stream" 2024-08-31T06:52:00.337 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="9" 2024-08-31T06:52:00.337 INFO:teuthology.orchestra.run.smithi032.stdout:ID="centos" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="rhel fedora" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="9" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el9" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:LOGO="fedora-logo-icon" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://centos.org/" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-31T06:52:00.338 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-31T06:52:00.339 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-08-31T06:52:00.362 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-08-31T06:52:00.380 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-08-31T06:52:00.380 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-08-31T06:52:00.438 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2024-08-31T06:52:00.438 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="9" 2024-08-31T06:52:00.438 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2024-08-31T06:52:00.438 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2024-08-31T06:52:00.438 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="9" 2024-08-31T06:52:00.439 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el9" 2024-08-31T06:52:00.439 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-31T06:52:00.439 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-08-31T06:52:00.439 INFO:teuthology.orchestra.run.smithi073.stdout:LOGO="fedora-logo-icon" 2024-08-31T06:52:00.439 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-31T06:52:00.439 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2024-08-31T06:52:00.439 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-31T06:52:00.439 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-31T06:52:00.439 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-31T06:52:00.439 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-08-31T06:52:00.461 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-08-31T06:52:00.480 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-08-31T06:52:00.480 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-08-31T06:52:00.537 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2024-08-31T06:52:00.537 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2024-08-31T06:52:00.538 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2024-08-31T06:52:00.538 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2024-08-31T06:52:00.538 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2024-08-31T06:52:00.538 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2024-08-31T06:52:00.538 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-31T06:52:00.538 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-08-31T06:52:00.538 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2024-08-31T06:52:00.538 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-31T06:52:00.538 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2024-08-31T06:52:00.538 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-31T06:52:00.539 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-31T06:52:00.539 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-31T06:52:00.539 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-08-31T06:52:00.562 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-31T06:52:00.571 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-31T06:52:00.606 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-31T06:52:00.606 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-08-31T06:52:00.609 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-08-31T06:52:00.611 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2024-08-31T06:52:00.627 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-31T06:52:00.633 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-31T06:52:00.633 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-08-31T06:52:00.666 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-08-31T06:52:00.669 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2024-08-31T06:52:00.761 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-31T06:52:00.872 INFO:teuthology.run_tasks:Running task kernel... 2024-08-31T06:52:01.003 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-31T06:52:01.003 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-08-31T06:52:01.003 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-08-31T06:52:01.004 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-31T06:52:01.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:52:01.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-08-31T06:52:01.202 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-08-31T06:52:01.202 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8ad8b8b160987a14eb199bb67cc00e23a08d43f6"... 2024-08-31T06:52:01.202 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-08-31T06:52:01.222 INFO:teuthology.orchestra.run.smithi032.stdout:5.14.0-503.el9.x86_64 2024-08-31T06:52:01.222 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-08-31T06:52:01.222 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-08-31T06:52:01.223 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum upgrade -y linux-firmware 2024-08-31T06:52:03.771 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-08-31T06:52:06.855 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-08-31T06:52:11.231 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.6 MB 00:00 2024-08-31T06:52:12.898 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 18 kB 00:00 2024-08-31T06:52:14.019 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 23 MB/s | 22 MB 00:00 2024-08-31T06:52:19.684 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-08-31T06:52:21.884 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-08-31T06:52:21.885 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-08-31T06:52:21.885 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-08-31T06:52:21.985 INFO:teuthology.task.kernel:Downloading kernel 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 on client.0... 2024-08-31T06:52:21.986 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-08-31T06:52:22.135 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/ 2024-08-31T06:52:22.136 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g8ad8b8b16098-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/ --input-file=- 2024-08-31T06:52:22.138 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi032.front.sepia.ceph.com to complete... 2024-08-31T06:52:28.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31 06:52:28 URL:https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g8ad8b8b16098-1.x86_64.rpm [359646365/359646365] -> "/tmp/kernel.x86_64.rpm" [1] 2024-08-31T06:52:28.455 INFO:teuthology.orchestra.run.smithi032.stderr:FINISHED --2024-08-31 06:52:28-- 2024-08-31T06:52:28.456 INFO:teuthology.orchestra.run.smithi032.stderr:Total wall clock time: 6.3s 2024-08-31T06:52:28.456 INFO:teuthology.orchestra.run.smithi032.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-08-31T06:52:28.459 INFO:teuthology.task.kernel:Installing kernel 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 on client.0... 2024-08-31T06:52:28.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-08-31T06:52:30.667 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying... ######################################## 2024-08-31T06:52:30.809 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing... ######################################## 2024-08-31T06:52:30.927 INFO:teuthology.orchestra.run.smithi032.stdout:Updating / installing... 2024-08-31T06:52:39.389 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-6.11.0_rc5_g8ad8b8b16098-1 ######################################## 2024-08-31T06:53:19.809 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-08-31T06:53:19.810 DEBUG:teuthology.orchestra.run.smithi032:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-08-31T06:53:19.891 INFO:teuthology.orchestra.run.smithi032.stdout:/boot/System.map-6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:53:19.891 INFO:teuthology.orchestra.run.smithi032.stdout:/boot/config-6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/boot/vmlinuz-6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/System.map 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/config 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-08-31T06:53:19.892 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crc32c-intel.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-avx2.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-08-31T06:53:19.893 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-08-31T06:53:19.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-08-31T06:53:19.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-08-31T06:53:19.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-08-31T06:53:19.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-08-31T06:53:19.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-08-31T06:53:19.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel 2024-08-31T06:53:19.894 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu/mce 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm-amd.ko 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm-intel.ko 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm.ko 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ansi_cprng.ko 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/anubis.ko 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/arc4.ko 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx 2024-08-31T06:53:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_memcpy.ko 2024-08-31T06:53:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_pq.ko 2024-08-31T06:53:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_raid6_recov.ko 2024-08-31T06:53:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_tx.ko 2024-08-31T06:53:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_xor.ko 2024-08-31T06:53:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/raid6test.ko 2024-08-31T06:53:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/authenc.ko 2024-08-31T06:53:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/authencesn.ko 2024-08-31T06:53:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blake2b_generic.ko 2024-08-31T06:53:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blowfish_common.ko 2024-08-31T06:53:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blowfish_generic.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/camellia_generic.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast5_generic.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast6_generic.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast_common.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ccm.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cmac.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crc32_generic.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crypto_engine.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crypto_user.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cts.ko 2024-08-31T06:53:19.897 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/des_generic.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/dh_generic.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ecc.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ecdh_generic.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/echainiv.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/essiv.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/fcrypt.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/khazad.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/lz4.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/lz4hc.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/md4.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/michael_mic.ko 2024-08-31T06:53:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/pcbc.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/pcrypt.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/rmd160.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/seed.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/serpent_generic.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/tcrypt.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/tea.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/twofish_common.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/twofish_generic.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/vmac.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/wp512.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xcbc.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xor.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xxhash_generic.ko 2024-08-31T06:53:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/acpi_ipmi.ko 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/acpi_pad.ko 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/ec_sys.ko 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/platform_profile.ko 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/sbs.ko 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/sbshc.ko 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/video.ko 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/acard-ahci.ko 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/ahci_platform.ko 2024-08-31T06:53:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/ata_generic.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/libahci_platform.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_acpi.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ali.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_amd.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_artop.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_atiixp.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_atp867x.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cmd640.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cmd64x.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cypress.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_efar.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt366.ko 2024-08-31T06:53:19.901 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt37x.ko 2024-08-31T06:53:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt3x2n.ko 2024-08-31T06:53:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt3x3.ko 2024-08-31T06:53:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_it8213.ko 2024-08-31T06:53:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_it821x.ko 2024-08-31T06:53:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_jmicron.ko 2024-08-31T06:53:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_marvell.ko 2024-08-31T06:53:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_mpiix.ko 2024-08-31T06:53:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_netcell.ko 2024-08-31T06:53:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ninja32.ko 2024-08-31T06:53:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ns87410.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ns87415.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_oldpiix.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_opti.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_optidma.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pcmcia.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pdc2027x.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_piccolo.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_rdc.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sch.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_serverworks.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sil680.ko 2024-08-31T06:53:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sis.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sl82c105.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_triflex.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_via.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pdc_adma.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_inic162x.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_mv.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_nv.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_promise.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_qstor.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sil.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sil24.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sis.ko 2024-08-31T06:53:19.904 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_svw.ko 2024-08-31T06:53:19.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sx4.ko 2024-08-31T06:53:19.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_uli.ko 2024-08-31T06:53:19.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_via.ko 2024-08-31T06:53:19.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_vsc.ko 2024-08-31T06:53:19.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/auxdisplay 2024-08-31T06:53:19.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/auxdisplay/ks0108.ko 2024-08-31T06:53:19.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base 2024-08-31T06:53:19.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap 2024-08-31T06:53:19.905 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap/regmap-i2c.ko 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap/regmap-mmio.ko 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/bcma 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/bcma/bcma.ko 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/aoe 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/aoe/aoe.ko 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/brd.ko 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/drbd 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/drbd/drbd.ko 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/floppy.ko 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/loop.ko 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/mtip32xx 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-08-31T06:53:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/nbd.ko 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/null_blk 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/null_blk/null_blk.ko 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/pktcdvd.ko 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/rbd.ko 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/virtio_blk.ko 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkback 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkfront.ko 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/zram 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/zram/zram.ko 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char 2024-08-31T06:53:19.907 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hangcheck-timer.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/amd-rng.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/intel-rng.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/via-rng.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/virtio-rng.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_si.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-08-31T06:53:19.908 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/lp.ko 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/mwave 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/mwave/mwave.ko 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ppdev.ko 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tlclk.ko 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm.ko 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_atmel.ko 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_infineon.ko 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_nsc.ko 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_tis.ko 2024-08-31T06:53:19.909 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/uv_mmtimer.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/virtio_console.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/p4-clockmod.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/powernow-k8.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/speedstep-lib.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp/ccp.ko 2024-08-31T06:53:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_common 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/padlock-aes.ko 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/padlock-sha.ko 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/virtio 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dca 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dca/dca.ko 2024-08-31T06:53:19.911 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/devfreq 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/dw 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/dw/dw_dmac.ko 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/ioat 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/ioat/ioatdma.ko 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/amd64_edac.ko 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/e752x_edac.ko 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/edac_mce_amd.ko 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i3000_edac.ko 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i3200_edac.ko 2024-08-31T06:53:19.912 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i5100_edac.ko 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i5400_edac.ko 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i7300_edac.ko 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i7core_edac.ko 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i82975x_edac.ko 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/ie31200_edac.ko 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/sb_edac.ko 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/x38_edac.ko 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware/edd.ko 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware/iscsi_ibft.ko 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio 2024-08-31T06:53:19.913 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio/gpio-ich.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio/gpio-viperboard.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-appleir.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-aureal.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-axff.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-dr.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-elecom.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-elo.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-emsff.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gaff.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gt683r.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gyration.ko 2024-08-31T06:53:19.914 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtek-kbd.ko 2024-08-31T06:53:19.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtek-mouse.ko 2024-08-31T06:53:19.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtekff.ko 2024-08-31T06:53:19.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-hyperv.ko 2024-08-31T06:53:19.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-icade.ko 2024-08-31T06:53:19.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-keytouch.ko 2024-08-31T06:53:19.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-kye.ko 2024-08-31T06:53:19.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-lcpower.ko 2024-08-31T06:53:19.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-led.ko 2024-08-31T06:53:19.915 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-lenovo.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-logitech-dj.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-multitouch.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-ortek.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-penmount.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-petalynx.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-picolcd.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-pl.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-plantronics.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-primax.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-rmi.ko 2024-08-31T06:53:19.916 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-arvo.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-common.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-isku.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-kone.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-konepure.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-lua.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-pyra.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-ryos.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-savu.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-saitek.ko 2024-08-31T06:53:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-samsung.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sensor-hub.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sjoy.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sony.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-speedlink.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-steelseries.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sunplus.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-thrustmaster.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-tivo.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-tmff.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-topseed.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-twinhan.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-uclogic.ko 2024-08-31T06:53:19.918 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-waltop.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-wiimote.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-xinmo.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-zpff.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-zydacron.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/uhid.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/wacom.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_balloon.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_utils.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_vmbus.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/abituguru.ko 2024-08-31T06:53:19.919 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/abituguru3.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/acpi_power_meter.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ad7414.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ad7418.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adc128d818.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1025.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1026.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1029.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1031.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm9240.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ads7828.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7410.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7411.ko 2024-08-31T06:53:19.920 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7462.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7470.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7475.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7x10.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/amc6821.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/applesmc.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asb100.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asc7621.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asus_atk0110.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/atxp1.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/coretemp.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-08-31T06:53:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/dme1737.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ds1621.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ds620.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/emc1403.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/emc6w201.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f71805f.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f71882fg.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f75375s.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/fam15h_power.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/fschmd.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/g760a.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/g762.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/gl518sm.ko 2024-08-31T06:53:19.922 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/gl520sm.ko 2024-08-31T06:53:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/hwmon-vid.ko 2024-08-31T06:53:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/i5500_temp.ko 2024-08-31T06:53:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/i5k_amb.ko 2024-08-31T06:53:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ibmaem.ko 2024-08-31T06:53:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ibmpex.ko 2024-08-31T06:53:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ina209.ko 2024-08-31T06:53:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ina2xx.ko 2024-08-31T06:53:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/it87.ko 2024-08-31T06:53:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/k10temp.ko 2024-08-31T06:53:19.923 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/k8temp.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lineage-pem.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm63.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm73.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm75.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm77.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm78.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm80.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm83.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm85.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm87.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm90.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm92.ko 2024-08-31T06:53:19.924 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm93.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95234.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95241.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95245.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc2945.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4151.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4215.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4222.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4245.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4260.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4261.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max16065.ko 2024-08-31T06:53:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max1619.ko 2024-08-31T06:53:19.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max1668.ko 2024-08-31T06:53:19.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max197.ko 2024-08-31T06:53:19.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6639.ko 2024-08-31T06:53:19.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6650.ko 2024-08-31T06:53:19.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6697.ko 2024-08-31T06:53:19.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/mcp3021.ko 2024-08-31T06:53:19.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6683.ko 2024-08-31T06:53:19.926 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6775-core.ko 2024-08-31T06:53:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6775.ko 2024-08-31T06:53:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct7802.ko 2024-08-31T06:53:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pc87360.ko 2024-08-31T06:53:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pc87427.ko 2024-08-31T06:53:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pcf8591.ko 2024-08-31T06:53:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus 2024-08-31T06:53:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-08-31T06:53:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-08-31T06:53:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max16064.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max34440.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max8688.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/powr1220.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch5627.ko 2024-08-31T06:53:19.928 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch5636.ko 2024-08-31T06:53:19.929 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch56xx-common.ko 2024-08-31T06:53:19.929 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sht15.ko 2024-08-31T06:53:19.929 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sht21.ko 2024-08-31T06:53:19.929 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/shtc1.ko 2024-08-31T06:53:19.929 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sis5595.ko 2024-08-31T06:53:19.929 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47b397.ko 2024-08-31T06:53:19.929 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47m1.ko 2024-08-31T06:53:19.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47m192.ko 2024-08-31T06:53:19.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/thmc50.ko 2024-08-31T06:53:19.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp102.ko 2024-08-31T06:53:19.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp103.ko 2024-08-31T06:53:19.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp401.ko 2024-08-31T06:53:19.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp421.ko 2024-08-31T06:53:19.930 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/via-cputemp.ko 2024-08-31T06:53:19.931 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/via686a.ko 2024-08-31T06:53:19.931 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/vt1211.ko 2024-08-31T06:53:19.931 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/vt8231.ko 2024-08-31T06:53:19.931 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83627ehf.ko 2024-08-31T06:53:19.931 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83627hf.ko 2024-08-31T06:53:19.931 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83781d.ko 2024-08-31T06:53:19.932 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83791d.ko 2024-08-31T06:53:19.932 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83792d.ko 2024-08-31T06:53:19.932 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83793.ko 2024-08-31T06:53:19.932 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83795.ko 2024-08-31T06:53:19.932 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83l785ts.ko 2024-08-31T06:53:19.932 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83l786ng.ko 2024-08-31T06:53:19.933 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c 2024-08-31T06:53:19.933 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos 2024-08-31T06:53:19.933 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-08-31T06:53:19.933 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-08-31T06:53:19.933 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses 2024-08-31T06:53:19.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-08-31T06:53:19.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-08-31T06:53:19.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-08-31T06:53:19.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-08-31T06:53:19.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-08-31T06:53:19.934 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-08-31T06:53:19.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-08-31T06:53:19.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-08-31T06:53:19.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-i801.ko 2024-08-31T06:53:19.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-isch.ko 2024-08-31T06:53:19.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-08-31T06:53:19.935 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-08-31T06:53:19.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-08-31T06:53:19.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-parport.ko 2024-08-31T06:53:19.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-08-31T06:53:19.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-08-31T06:53:19.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-08-31T06:53:19.936 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-08-31T06:53:19.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-08-31T06:53:19.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-08-31T06:53:19.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-via.ko 2024-08-31T06:53:19.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-08-31T06:53:19.937 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-08-31T06:53:19.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-dev.ko 2024-08-31T06:53:19.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-mux.ko 2024-08-31T06:53:19.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-08-31T06:53:19.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-smbus.ko 2024-08-31T06:53:19.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-stub.ko 2024-08-31T06:53:19.938 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband 2024-08-31T06:53:19.939 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core 2024-08-31T06:53:19.939 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_cm.ko 2024-08-31T06:53:19.939 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_core.ko 2024-08-31T06:53:19.939 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_umad.ko 2024-08-31T06:53:19.939 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-08-31T06:53:19.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/iw_cm.ko 2024-08-31T06:53:19.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/rdma_cm.ko 2024-08-31T06:53:19.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-08-31T06:53:19.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw 2024-08-31T06:53:19.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/cxgb4 2024-08-31T06:53:19.940 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-08-31T06:53:19.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx4 2024-08-31T06:53:19.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-08-31T06:53:19.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx5 2024-08-31T06:53:19.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-08-31T06:53:19.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mthca 2024-08-31T06:53:19.941 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-08-31T06:53:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp 2024-08-31T06:53:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/ipoib 2024-08-31T06:53:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-08-31T06:53:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/iser 2024-08-31T06:53:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-08-31T06:53:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/isert 2024-08-31T06:53:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-08-31T06:53:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srp 2024-08-31T06:53:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-08-31T06:53:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srpt 2024-08-31T06:53:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-08-31T06:53:19.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input 2024-08-31T06:53:19.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport 2024-08-31T06:53:19.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-08-31T06:53:19.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/fm801-gp.ko 2024-08-31T06:53:19.944 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/gameport.ko 2024-08-31T06:53:19.945 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/joydev.ko 2024-08-31T06:53:19.945 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/keyboard 2024-08-31T06:53:19.949 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/keyboard/gpio_keys.ko 2024-08-31T06:53:19.949 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse 2024-08-31T06:53:19.949 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/appletouch.ko 2024-08-31T06:53:19.949 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/bcm5974.ko 2024-08-31T06:53:19.950 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/cyapatp.ko 2024-08-31T06:53:19.950 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/elan_i2c.ko 2024-08-31T06:53:19.950 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/sermouse.ko 2024-08-31T06:53:19.950 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-08-31T06:53:19.950 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/synaptics_usb.ko 2024-08-31T06:53:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/vsxxxaa.ko 2024-08-31T06:53:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/rmi4 2024-08-31T06:53:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/rmi4/rmi_core.ko 2024-08-31T06:53:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio 2024-08-31T06:53:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/altera_ps2.ko 2024-08-31T06:53:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/arc_ps2.ko 2024-08-31T06:53:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-08-31T06:53:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/serio_raw.ko 2024-08-31T06:53:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/sparse-keymap.ko 2024-08-31T06:53:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds 2024-08-31T06:53:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-blinkm.ko 2024-08-31T06:53:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-lm3530.ko 2024-08-31T06:53:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-lp3944.ko 2024-08-31T06:53:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-ss4200.ko 2024-08-31T06:53:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger 2024-08-31T06:53:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-08-31T06:53:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-08-31T06:53:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-08-31T06:53:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-08-31T06:53:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-08-31T06:53:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-08-31T06:53:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-08-31T06:53:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-08-31T06:53:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md 2024-08-31T06:53:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/bcache 2024-08-31T06:53:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/bcache/bcache.ko 2024-08-31T06:53:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-bio-prison.ko 2024-08-31T06:53:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-cache-smq.ko 2024-08-31T06:53:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-cache.ko 2024-08-31T06:53:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-crypt.ko 2024-08-31T06:53:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-delay.ko 2024-08-31T06:53:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-flakey.ko 2024-08-31T06:53:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-log-userspace.ko 2024-08-31T06:53:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-multipath.ko 2024-08-31T06:53:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-queue-length.ko 2024-08-31T06:53:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-raid.ko 2024-08-31T06:53:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-round-robin.ko 2024-08-31T06:53:19.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-service-time.ko 2024-08-31T06:53:19.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-switch.ko 2024-08-31T06:53:19.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-thin-pool.ko 2024-08-31T06:53:19.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-verity.ko 2024-08-31T06:53:19.958 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/persistent-data 2024-08-31T06:53:19.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-08-31T06:53:19.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid0.ko 2024-08-31T06:53:19.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid1.ko 2024-08-31T06:53:19.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid10.ko 2024-08-31T06:53:19.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid456.ko 2024-08-31T06:53:19.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message 2024-08-31T06:53:19.959 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion 2024-08-31T06:53:19.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptbase.ko 2024-08-31T06:53:19.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptctl.ko 2024-08-31T06:53:19.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptfc.ko 2024-08-31T06:53:19.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptlan.ko 2024-08-31T06:53:19.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptsas.ko 2024-08-31T06:53:19.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptscsih.ko 2024-08-31T06:53:19.960 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptspi.ko 2024-08-31T06:53:19.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd 2024-08-31T06:53:19.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/lpc_ich.ko 2024-08-31T06:53:19.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/lpc_sch.ko 2024-08-31T06:53:19.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/mfd-core.ko 2024-08-31T06:53:19.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/sm501.ko 2024-08-31T06:53:19.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/viperboard.ko 2024-08-31T06:53:19.961 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/vx855.ko 2024-08-31T06:53:19.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/wl1273-core.ko 2024-08-31T06:53:19.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc 2024-08-31T06:53:19.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/altera-stapl 2024-08-31T06:53:19.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-08-31T06:53:19.962 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/apds9802als.ko 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/apds990x.ko 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/bh1770glc.ko 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/cb710 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/cb710/cb710.ko 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/at24.ko 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/max6875.ko 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/enclosure.ko 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/hpilo.ko 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/isl29003.ko 2024-08-31T06:53:19.963 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/isl29020.ko 2024-08-31T06:53:19.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d 2024-08-31T06:53:19.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-08-31T06:53:19.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-08-31T06:53:19.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei 2024-08-31T06:53:19.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei-me.ko 2024-08-31T06:53:19.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei-txe.ko 2024-08-31T06:53:19.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei.ko 2024-08-31T06:53:19.964 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-gru 2024-08-31T06:53:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-gru/gru.ko 2024-08-31T06:53:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp 2024-08-31T06:53:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xp.ko 2024-08-31T06:53:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xpc.ko 2024-08-31T06:53:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-08-31T06:53:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tifm_7xx1.ko 2024-08-31T06:53:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tifm_core.ko 2024-08-31T06:53:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tsl2550.ko 2024-08-31T06:53:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_balloon.ko 2024-08-31T06:53:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_vmci 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/bonding 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/bonding/bonding.ko 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/b53 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/b53/b53_common.ko 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/bcm-sf2.ko 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6060.ko 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6xxx 2024-08-31T06:53:19.966 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dummy.ko 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/eql.ko 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/8390.ko 2024-08-31T06:53:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-08-31T06:53:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-08-31T06:53:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-08-31T06:53:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/adaptec 2024-08-31T06:53:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-08-31T06:53:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/agere 2024-08-31T06:53:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/agere/et131x.ko 2024-08-31T06:53:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/alteon 2024-08-31T06:53:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-08-31T06:53:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/altera 2024-08-31T06:53:19.968 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/alx 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1c 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1e 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx 2024-08-31T06:53:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/genet 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade 2024-08-31T06:53:19.970 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade/bna 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-08-31T06:53:19.971 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco/enic 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-08-31T06:53:19.972 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dnet.ko 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex/benet 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ethoc.ko 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/fealnx.ko 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e100.ko 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000e 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-08-31T06:53:19.973 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/fm10k 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/i40e 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/iavf 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igb 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igbvf 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbe 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbevf 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-08-31T06:53:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libeth 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libie 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/jme.ko 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/skge.ko 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-08-31T06:53:19.975 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/micrel 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom/myri10ge 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/neterion 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/nvidia 2024-08-31T06:53:19.976 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-08-31T06:53:19.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines 2024-08-31T06:53:19.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-08-31T06:53:19.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-08-31T06:53:19.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic 2024-08-31T06:53:19.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/netxen 2024-08-31T06:53:19.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-08-31T06:53:19.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-08-31T06:53:19.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-08-31T06:53:19.977 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rdc 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/atp.ko 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rocker 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sfc 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/silan 2024-08-31T06:53:19.978 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis/sis190.ko 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis/sis900.ko 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun 2024-08-31T06:53:19.979 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/cassini.ko 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/niu.ko 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/sungem.ko 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/tehuti 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ti 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ti/tlan.ko 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-08-31T06:53:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/xircom 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/geneve.ko 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/hyperv 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ieee802154 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ieee802154/fakelb.ko 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ifb.ko 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ipvlan 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ipvlan/ipvlan.ko 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/macvlan.ko 2024-08-31T06:53:19.981 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/macvtap.ko 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio.ko 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-regmap.ko 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mii.ko 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/net_failover.ko 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/netconsole.ko 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/nlmon.ko 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs 2024-08-31T06:53:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs/pcs-lynx.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/amd.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm7xxx.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm87xx.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/broadcom.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/cicada.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/davicom.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/dp83640.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/et1011c.ko 2024-08-31T06:53:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/icplus.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/lxt.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/marvell.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/micrel.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/national.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/phylink.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/qsemi.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/realtek.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/smsc.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/ste10Xp.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/vitesse.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/bsd_comp.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_async.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_deflate.ko 2024-08-31T06:53:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_generic.ko 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_mppe.ko 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_synctty.ko 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pppoe.ko 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pppox.ko 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pptp.ko 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip/slhc.ko 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip/slip.ko 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/sungem_phy.ko 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/tap.ko 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team 2024-08-31T06:53:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_activebackup.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_broadcast.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_random.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/tun.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/veth.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/virtio_net.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vmxnet3 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vxlan 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vxlan/vxlan.ko 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netback 2024-08-31T06:53:19.986 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netback/xen-netback.ko 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netfront.ko 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ntb 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ntb/ntb.ko 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host/nvme-core.ko 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host/nvme.ko 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport.ko 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_cs.ko 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_pc.ko 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_serial.ko 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/hotplug 2024-08-31T06:53:19.987 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/pcie 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/pcie/aer_inject.ko 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/xen-pcifront.ko 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/i82092.ko 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/pd6729.ko 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/yenta_socket.ko 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl/intel 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/acer-wmi.ko 2024-08-31T06:53:19.988 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/acerhdf.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/apple-gmux.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-laptop.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-wmi.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/classmate-laptop.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/hdaps.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-08-31T06:53:19.989 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel_ips.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/msi-wmi.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/mxm-wmi.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/samsung-laptop.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/samsung-q10.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/topstar-laptop.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/toshiba_haps.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/wmi-bmof.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/wmi.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap/intel_rapl_common.ko 2024-08-31T06:53:19.990 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap/intel_rapl_msr.ko 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps-gpio.ko 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps-ldisc.ko 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps_parport.ko 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/pps_core.ko 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp/ptp.ko 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp/ptp_kvm.ko 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/atl 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-08-31T06:53:19.991 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/fmpm.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-bq32k.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1286.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1307.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1374.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1511.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1553.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1672.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1742.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds2404.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds3232.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-em3027.ko 2024-08-31T06:53:19.992 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-fm3130.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-isl12022.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-isl1208.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m41t80.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m48t35.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m48t59.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-max6900.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-msm6242.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf2127.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf85063.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8523.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8563.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8583.ko 2024-08-31T06:53:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rp5c01.ko 2024-08-31T06:53:19.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rs5c372.ko 2024-08-31T06:53:19.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-08-31T06:53:19.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rx8025.ko 2024-08-31T06:53:19.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rx8581.ko 2024-08-31T06:53:19.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-08-31T06:53:19.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-x1205.ko 2024-08-31T06:53:19.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi 2024-08-31T06:53:19.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-9xxx.ko 2024-08-31T06:53:19.994 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-sas.ko 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-xxxx.ko 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/BusLogic.ko 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/a100u2w.ko 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aacraid 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aacraid/aacraid.ko 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/advansys.ko 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/am53c974.ko 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/arcmsr 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-08-31T06:53:19.995 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/atp870u.ko 2024-08-31T06:53:19.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/be2iscsi 2024-08-31T06:53:19.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-08-31T06:53:19.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bfa 2024-08-31T06:53:19.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bfa/bfa.ko 2024-08-31T06:53:19.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2fc 2024-08-31T06:53:19.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-08-31T06:53:19.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2i 2024-08-31T06:53:19.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-08-31T06:53:19.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ch.ko 2024-08-31T06:53:19.996 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/csiostor 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/csiostor/csiostor.ko 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb3i 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb4i 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/dc395x.ko 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-08-31T06:53:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/dmx3191d.ko 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esas2r 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esas2r/esas2r.ko 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esp_scsi.ko 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe/fcoe.ko 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fnic 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fnic/fnic.ko 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hpsa.ko 2024-08-31T06:53:19.998 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hptiop.ko 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hv_storvsc.ko 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/initio.ko 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ipr.ko 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ips.ko 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/isci 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/isci/isci.ko 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/iscsi_tcp.ko 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libfc 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libfc/libfc.ko 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libiscsi.ko 2024-08-31T06:53:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libiscsi_tcp.ko 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libsas 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libsas/libsas.ko 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/lpfc 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/lpfc/lpfc.ko 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid.ko 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mpt3sas 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvsas 2024-08-31T06:53:20.000 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvsas/mvsas.ko 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvumi.ko 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pm8001 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pmcraid.ko 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla1280.ko 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla4xxx 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/raid_class.ko 2024-08-31T06:53:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_debug.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_fc.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_sas.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_spi.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_srp.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ses.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/st.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/stex.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/sym53c8xx_2 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/virtio_scsi.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/vmw_pvscsi.ko 2024-08-31T06:53:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/wd719x.ko 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ssb 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ssb/ssb.ko 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/iscsi 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/loopback 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/loopback/tcm_loop.ko 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_file.ko 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_iblock.ko 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_mod.ko 2024-08-31T06:53:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_pscsi.ko 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_user.ko 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/tcm_fc 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-08-31T06:53:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-08-31T06:53:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/ipwireless 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/n_gsm.ko 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/n_hdlc.ko 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/nozomi.ko 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/8250 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/arc_uart.ko 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/jsm 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/jsm/jsm.ko 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/core 2024-08-31T06:53:20.006 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/core/ufshcd-core.ko 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/host 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio.ko 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_aec.ko 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_cif.ko 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_pci_generic.ko 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_sercos3.ko 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/cdc-acm.ko 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/cdc-wdm.ko 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/usblp.ko 2024-08-31T06:53:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/usbtmc.ko 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/host 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/host/sl811-hcd.ko 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image/mdc800.ko 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image/microtek.ko 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/adutux.ko 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/appledisplay.ko 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/emi26.ko 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/emi62.ko 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/ezusb.ko 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/idmouse.ko 2024-08-31T06:53:20.008 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/iowarrior.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/isight_firmware.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/ldusb.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/legousbtower.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/sisusbvga 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/trancevibrator.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usb3503.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usblcd.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usbsevseg.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/uss720.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/yurex.ko 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/phy 2024-08-31T06:53:20.009 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/phy/phy-generic.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/aircable.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ark3116.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/belkin_sa.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ch341.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cp210x.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cyberjack.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cypress_m8.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/digi_acceleport.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/empeg.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ftdi_sio.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/garmin_gps.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/io_edgeport.ko 2024-08-31T06:53:20.010 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/io_ti.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ipaq.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ipw.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ir-usb.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/keyspan.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/keyspan_pda.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/kl5kusb105.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/kobil_sct.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mct_u232.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mos7720.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mos7840.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/navman.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/omninet.ko 2024-08-31T06:53:20.011 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/opticon.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/option.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/oti6858.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/pl2303.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/qcaux.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/qcserial.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/quatech2.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/safe_serial.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/sierra.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/spcp8x5.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ssu100.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/symbolserial.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-08-31T06:53:20.012 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb_debug.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb_wwan.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/visor.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/whiteheat.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/xsens_mt.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/uas.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-alauda.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-cypress.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-datafab.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-08-31T06:53:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-freecom.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-isd200.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-karma.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-onetouch.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-realtek.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-sddr09.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-sddr55.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-usbat.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/usb-storage.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/usbip-core.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/usbip-host.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-08-31T06:53:20.014 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci/vfio-pci.ko 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/vfio.ko 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost.ko 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_iotlb.ko 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_net.ko 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_scsi.ko 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight 2024-08-31T06:53:20.015 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/apple_bl.ko 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/lcd.ko 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/platform_lcd.ko 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio.ko 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_balloon.ko 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_mmio.ko 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci.ko 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_ring.ko 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters 2024-08-31T06:53:20.016 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters/ds2482.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters/ds2490.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_smem.ko 2024-08-31T06:53:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_therm.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/wire.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/alim1535_wdt.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/alim7101_wdt.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/f71808e_wdt.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/hpwdt.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/i6300esb.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/iTCO_wdt.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ib700wdt.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ibmasr.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-08-31T06:53:20.018 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/it8712f_wdt.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/it87_wdt.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/machzwd.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/nv_tco.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/pcwd_pci.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/pcwd_usb.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sch311x_wdt.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/softdog.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sp5100_tco.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/via_wdt.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83877f_wdt.ko 2024-08-31T06:53:20.019 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83977f_wdt.ko 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/wdt_pci.ko 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/xen_wdt.ko 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-acpi-processor.ko 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-evtchn.ko 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-gntalloc.ko 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-gntdev.ko 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-pciback 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-privcmd.ko 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xenfs 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xenfs/xenfs.ko 2024-08-31T06:53:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/9p 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/9p/9p.ko 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/affs 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/affs/affs.ko 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/befs 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/befs/befs.ko 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/binfmt_misc.ko 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/btrfs 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/btrfs/btrfs.ko 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cachefiles 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cachefiles/cachefiles.ko 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ceph 2024-08-31T06:53:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ceph/ceph.ko 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/coda 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/coda/coda.ko 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cramfs 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cramfs/cramfs.ko 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/dlm 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/dlm/dlm.ko 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ecryptfs 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ecryptfs/ecryptfs.ko 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/f2fs 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/f2fs/f2fs.ko 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/fat.ko 2024-08-31T06:53:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/msdos.ko 2024-08-31T06:53:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/vfat.ko 2024-08-31T06:53:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse 2024-08-31T06:53:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse/cuse.ko 2024-08-31T06:53:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse/fuse.ko 2024-08-31T06:53:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/gfs2 2024-08-31T06:53:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/gfs2/gfs2.ko 2024-08-31T06:53:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfs 2024-08-31T06:53:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfs/hfs.ko 2024-08-31T06:53:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfsplus 2024-08-31T06:53:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfsplus/hfsplus.ko 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/isofs 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/isofs/isofs.ko 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/jfs 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/jfs/jfs.ko 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/lockd 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/lockd/lockd.ko 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/minix 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/minix/minix.ko 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/netfs 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/netfs/netfs.ko 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/blocklayout 2024-08-31T06:53:20.024 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-08-31T06:53:20.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/filelayout 2024-08-31T06:53:20.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-08-31T06:53:20.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/flexfilelayout 2024-08-31T06:53:20.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-08-31T06:53:20.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfs.ko 2024-08-31T06:53:20.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfsv3.ko 2024-08-31T06:53:20.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfsv4.ko 2024-08-31T06:53:20.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common 2024-08-31T06:53:20.025 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common/grace.ko 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common/nfs_acl.ko 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfsd 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfsd/nfsd.ko 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nilfs2 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nilfs2/nilfs2.ko 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-celtic.ko 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-centeuro.ko 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-croatian.ko 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-cyrillic.ko 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-gaelic.ko 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-greek.ko 2024-08-31T06:53:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-iceland.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-inuit.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-roman.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-romanian.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-turkish.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1250.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1251.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1255.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp737.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp775.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp850.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp852.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp855.ko 2024-08-31T06:53:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp857.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp860.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp861.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp862.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp863.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp864.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp865.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp866.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp869.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp874.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp932.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp936.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp949.ko 2024-08-31T06:53:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp950.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_euc-jp.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-1.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-13.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-14.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-15.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-2.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-3.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-4.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-5.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-6.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-7.ko 2024-08-31T06:53:20.029 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-9.ko 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-r.ko 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-ru.ko 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-u.ko 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_ucs2_utils.ko 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_utf8.ko 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/cluster 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlm 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlmfs 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2.ko 2024-08-31T06:53:20.030 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/overlayfs 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/overlayfs/overlay.ko 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/pstore 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/pstore/ramoops.ko 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/reiserfs 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/reiserfs/reiserfs.ko 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/romfs 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/romfs/romfs.ko 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb 2024-08-31T06:53:20.031 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/client 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/client/cifs.ko 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common/cifs_arc4.ko 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common/cifs_md4.ko 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/squashfs 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/squashfs/squashfs.ko 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/sysv 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/sysv/sysv.ko 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/udf 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/udf/udf.ko 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ufs 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ufs/ufs.ko 2024-08-31T06:53:20.032 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/xfs 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/xfs/xfs.ko 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/rcu 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/rcu/rcutorture.ko 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/torture.ko 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/trace 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/asn1_encoder.ko 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crc-itu-t.ko 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crc8.ko 2024-08-31T06:53:20.033 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libaescfb.ko 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libarc4.ko 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libdes.ko 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/libcrc32c.ko 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lru_cache.ko 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4/lz4_compress.ko 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4/lz4hc_compress.ko 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/math 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/math/cordic.ko 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/raid6 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/raid6/raid6_pq.ko 2024-08-31T06:53:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/reed_solomon 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/reed_solomon/reed_solomon.ko 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_bm.ko 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_fsm.ko 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_kmp.ko 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/zstd 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/zstd/zstd_compress.ko 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/mm 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/mm/hwpoison-inject.ko 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan 2024-08-31T06:53:20.035 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/6lowpan.ko 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_dest.ko 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_fragment.ko 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_hop.ko 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_ipv6.ko 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_mobility.ko 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_routing.ko 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_udp.ko 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/garp.ko 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/mrp.ko 2024-08-31T06:53:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/p8022.ko 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/psnap.ko 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/stp.ko 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/8021q 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/8021q/8021q.ko 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet.ko 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_fd.ko 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_rdma.ko 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_virtio.ko 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/batman-adv 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/batman-adv/batman-adv.ko 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge 2024-08-31T06:53:20.037 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/br_netfilter.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/bridge.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_among.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_arp.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_ip.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_limit.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_log.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_mark.ko 2024-08-31T06:53:20.038 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_snat.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_stp.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtables.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-08-31T06:53:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ceph 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ceph/libceph.ko 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core/failover.ko 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core/pktgen.ko 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp.ko 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_diag.ko 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_ipv4.ko 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_ipv6.ko 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dns_resolver 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dns_resolver/dns_resolver.ko 2024-08-31T06:53:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/dsa_core.ko 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_brcm.ko 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_dsa.ko 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_none.ko 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_trailer.ko 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/6lowpan 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/ieee802154.ko 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/ieee802154_socket.ko 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ah4.ko 2024-08-31T06:53:20.041 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/esp4.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/fou.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/gre.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/inet_diag.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_gre.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_tunnel.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_vti.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ipcomp.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ipip.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arp_tables.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-08-31T06:53:20.042 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_security.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-08-31T06:53:20.043 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_bic.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_dctcp.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_diag.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_highspeed.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_htcp.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_hybla.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_illinois.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_lp.ko 2024-08-31T06:53:20.044 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_scalable.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_vegas.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_veno.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_westwood.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_yeah.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tunnel4.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/udp_diag.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/udp_tunnel.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/xfrm4_tunnel.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ah6.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/esp6.ko 2024-08-31T06:53:20.045 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/fou6.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_tunnel.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_vti.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ipcomp6.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-08-31T06:53:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-08-31T06:53:20.047 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/sit.ko 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/tunnel6.ko 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/xfrm6_tunnel.ko 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/key 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/key/af_key.ko 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_core.ko 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_debugfs.ko 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_eth.ko 2024-08-31T06:53:20.048 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ip.ko 2024-08-31T06:53:20.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ip6.ko 2024-08-31T06:53:20.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_netlink.ko 2024-08-31T06:53:20.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ppp.ko 2024-08-31T06:53:20.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/llc 2024-08-31T06:53:20.049 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/llc/llc.ko 2024-08-31T06:53:20.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mac802154 2024-08-31T06:53:20.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mac802154/mac802154.ko 2024-08-31T06:53:20.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mpls 2024-08-31T06:53:20.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mpls/mpls_gso.ko 2024-08-31T06:53:20.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter 2024-08-31T06:53:20.050 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset 2024-08-31T06:53:20.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set.ko 2024-08-31T06:53:20.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-08-31T06:53:20.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-08-31T06:53:20.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-08-31T06:53:20.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-08-31T06:53:20.051 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-08-31T06:53:20.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-08-31T06:53:20.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-08-31T06:53:20.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-08-31T06:53:20.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-08-31T06:53:20.052 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-08-31T06:53:20.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-08-31T06:53:20.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-08-31T06:53:20.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-08-31T06:53:20.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-08-31T06:53:20.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-08-31T06:53:20.053 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs 2024-08-31T06:53:20.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs.ko 2024-08-31T06:53:20.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-08-31T06:53:20.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-08-31T06:53:20.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-08-31T06:53:20.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-08-31T06:53:20.054 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-08-31T06:53:20.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-08-31T06:53:20.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-08-31T06:53:20.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-08-31T06:53:20.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-08-31T06:53:20.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-08-31T06:53:20.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-08-31T06:53:20.055 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-08-31T06:53:20.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-08-31T06:53:20.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conncount.ko 2024-08-31T06:53:20.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack.ko 2024-08-31T06:53:20.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-08-31T06:53:20.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-08-31T06:53:20.056 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-08-31T06:53:20.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_h323.ko 2024-08-31T06:53:20.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_irc.ko 2024-08-31T06:53:20.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-08-31T06:53:20.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-08-31T06:53:20.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-08-31T06:53:20.057 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_sane.ko 2024-08-31T06:53:20.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_sip.ko 2024-08-31T06:53:20.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-08-31T06:53:20.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-08-31T06:53:20.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_dup_netdev.ko 2024-08-31T06:53:20.058 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_flow_table.ko 2024-08-31T06:53:20.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_flow_table_inet.ko 2024-08-31T06:53:20.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_log_syslog.ko 2024-08-31T06:53:20.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat.ko 2024-08-31T06:53:20.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_amanda.ko 2024-08-31T06:53:20.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_ftp.ko 2024-08-31T06:53:20.059 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_irc.ko 2024-08-31T06:53:20.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_sip.ko 2024-08-31T06:53:20.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_tftp.ko 2024-08-31T06:53:20.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_synproxy_core.ko 2024-08-31T06:53:20.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_tables.ko 2024-08-31T06:53:20.060 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink.ko 2024-08-31T06:53:20.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_acct.ko 2024-08-31T06:53:20.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_log.ko 2024-08-31T06:53:20.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_osf.ko 2024-08-31T06:53:20.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_queue.ko 2024-08-31T06:53:20.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_chain_nat.ko 2024-08-31T06:53:20.061 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_compat.ko 2024-08-31T06:53:20.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_ct.ko 2024-08-31T06:53:20.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_dup_netdev.ko 2024-08-31T06:53:20.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_fib.ko 2024-08-31T06:53:20.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_fwd_netdev.ko 2024-08-31T06:53:20.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_hash.ko 2024-08-31T06:53:20.062 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_limit.ko 2024-08-31T06:53:20.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_log.ko 2024-08-31T06:53:20.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_masq.ko 2024-08-31T06:53:20.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_nat.ko 2024-08-31T06:53:20.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_numgen.ko 2024-08-31T06:53:20.063 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_queue.ko 2024-08-31T06:53:20.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_quota.ko 2024-08-31T06:53:20.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_redir.ko 2024-08-31T06:53:20.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_reject.ko 2024-08-31T06:53:20.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_reject_inet.ko 2024-08-31T06:53:20.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_socket.ko 2024-08-31T06:53:20.064 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_tunnel.ko 2024-08-31T06:53:20.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_AUDIT.ko 2024-08-31T06:53:20.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CHECKSUM.ko 2024-08-31T06:53:20.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CLASSIFY.ko 2024-08-31T06:53:20.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-08-31T06:53:20.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CT.ko 2024-08-31T06:53:20.065 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_DSCP.ko 2024-08-31T06:53:20.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_HL.ko 2024-08-31T06:53:20.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_HMARK.ko 2024-08-31T06:53:20.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_IDLETIMER.ko 2024-08-31T06:53:20.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_LED.ko 2024-08-31T06:53:20.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_LOG.ko 2024-08-31T06:53:20.066 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_MASQUERADE.ko 2024-08-31T06:53:20.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NETMAP.ko 2024-08-31T06:53:20.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NFLOG.ko 2024-08-31T06:53:20.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NFQUEUE.ko 2024-08-31T06:53:20.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_RATEEST.ko 2024-08-31T06:53:20.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_REDIRECT.ko 2024-08-31T06:53:20.067 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_SECMARK.ko 2024-08-31T06:53:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TCPMSS.ko 2024-08-31T06:53:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-08-31T06:53:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TEE.ko 2024-08-31T06:53:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TPROXY.ko 2024-08-31T06:53:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TRACE.ko 2024-08-31T06:53:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_addrtype.ko 2024-08-31T06:53:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_bpf.ko 2024-08-31T06:53:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cgroup.ko 2024-08-31T06:53:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cluster.ko 2024-08-31T06:53:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_comment.ko 2024-08-31T06:53:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connbytes.ko 2024-08-31T06:53:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connlabel.ko 2024-08-31T06:53:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connlimit.ko 2024-08-31T06:53:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connmark.ko 2024-08-31T06:53:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_conntrack.ko 2024-08-31T06:53:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cpu.ko 2024-08-31T06:53:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_dccp.ko 2024-08-31T06:53:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_devgroup.ko 2024-08-31T06:53:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_dscp.ko 2024-08-31T06:53:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ecn.ko 2024-08-31T06:53:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_esp.ko 2024-08-31T06:53:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_hashlimit.ko 2024-08-31T06:53:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_helper.ko 2024-08-31T06:53:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_hl.ko 2024-08-31T06:53:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ipcomp.ko 2024-08-31T06:53:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_iprange.ko 2024-08-31T06:53:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ipvs.ko 2024-08-31T06:53:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_l2tp.ko 2024-08-31T06:53:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_length.ko 2024-08-31T06:53:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_limit.ko 2024-08-31T06:53:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_mac.ko 2024-08-31T06:53:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_mark.ko 2024-08-31T06:53:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_multiport.ko 2024-08-31T06:53:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_nat.ko 2024-08-31T06:53:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_nfacct.ko 2024-08-31T06:53:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_osf.ko 2024-08-31T06:53:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_owner.ko 2024-08-31T06:53:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_physdev.ko 2024-08-31T06:53:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_pkttype.ko 2024-08-31T06:53:20.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_policy.ko 2024-08-31T06:53:20.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_quota.ko 2024-08-31T06:53:20.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_rateest.ko 2024-08-31T06:53:20.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_realm.ko 2024-08-31T06:53:20.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_recent.ko 2024-08-31T06:53:20.075 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_sctp.ko 2024-08-31T06:53:20.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_set.ko 2024-08-31T06:53:20.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_socket.ko 2024-08-31T06:53:20.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_state.ko 2024-08-31T06:53:20.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_statistic.ko 2024-08-31T06:53:20.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_string.ko 2024-08-31T06:53:20.076 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_tcpmss.ko 2024-08-31T06:53:20.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_time.ko 2024-08-31T06:53:20.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_u32.ko 2024-08-31T06:53:20.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netlink 2024-08-31T06:53:20.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netlink/netlink_diag.ko 2024-08-31T06:53:20.077 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/nsh 2024-08-31T06:53:20.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/nsh/nsh.ko 2024-08-31T06:53:20.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch 2024-08-31T06:53:20.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/openvswitch.ko 2024-08-31T06:53:20.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-geneve.ko 2024-08-31T06:53:20.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-gre.ko 2024-08-31T06:53:20.078 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-vxlan.ko 2024-08-31T06:53:20.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/packet 2024-08-31T06:53:20.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/packet/af_packet_diag.ko 2024-08-31T06:53:20.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds 2024-08-31T06:53:20.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds.ko 2024-08-31T06:53:20.079 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds_rdma.ko 2024-08-31T06:53:20.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds_tcp.ko 2024-08-31T06:53:20.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched 2024-08-31T06:53:20.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_csum.ko 2024-08-31T06:53:20.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_gact.ko 2024-08-31T06:53:20.080 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_mirred.ko 2024-08-31T06:53:20.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_nat.ko 2024-08-31T06:53:20.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_pedit.ko 2024-08-31T06:53:20.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_police.ko 2024-08-31T06:53:20.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_simple.ko 2024-08-31T06:53:20.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_skbedit.ko 2024-08-31T06:53:20.081 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_vlan.ko 2024-08-31T06:53:20.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_basic.ko 2024-08-31T06:53:20.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_bpf.ko 2024-08-31T06:53:20.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_flow.ko 2024-08-31T06:53:20.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_fw.ko 2024-08-31T06:53:20.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_route.ko 2024-08-31T06:53:20.082 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_u32.ko 2024-08-31T06:53:20.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_cmp.ko 2024-08-31T06:53:20.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_ipset.ko 2024-08-31T06:53:20.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_meta.ko 2024-08-31T06:53:20.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_nbyte.ko 2024-08-31T06:53:20.083 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_text.ko 2024-08-31T06:53:20.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_u32.ko 2024-08-31T06:53:20.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_choke.ko 2024-08-31T06:53:20.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_codel.ko 2024-08-31T06:53:20.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_drr.ko 2024-08-31T06:53:20.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_fq.ko 2024-08-31T06:53:20.084 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_gred.ko 2024-08-31T06:53:20.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_hfsc.ko 2024-08-31T06:53:20.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_hhf.ko 2024-08-31T06:53:20.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_htb.ko 2024-08-31T06:53:20.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_ingress.ko 2024-08-31T06:53:20.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_mqprio.ko 2024-08-31T06:53:20.085 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_mqprio_lib.ko 2024-08-31T06:53:20.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_multiq.ko 2024-08-31T06:53:20.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_netem.ko 2024-08-31T06:53:20.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_pie.ko 2024-08-31T06:53:20.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_plug.ko 2024-08-31T06:53:20.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_prio.ko 2024-08-31T06:53:20.086 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_qfq.ko 2024-08-31T06:53:20.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_red.ko 2024-08-31T06:53:20.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_sfb.ko 2024-08-31T06:53:20.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_sfq.ko 2024-08-31T06:53:20.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_tbf.ko 2024-08-31T06:53:20.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_teql.ko 2024-08-31T06:53:20.087 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp 2024-08-31T06:53:20.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp/sctp.ko 2024-08-31T06:53:20.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp/sctp_diag.ko 2024-08-31T06:53:20.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc 2024-08-31T06:53:20.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss 2024-08-31T06:53:20.088 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-08-31T06:53:20.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-08-31T06:53:20.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/sunrpc.ko 2024-08-31T06:53:20.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/xprtrdma 2024-08-31T06:53:20.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-08-31T06:53:20.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc 2024-08-31T06:53:20.089 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc/tipc.ko 2024-08-31T06:53:20.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc/tipc_diag.ko 2024-08-31T06:53:20.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/unix 2024-08-31T06:53:20.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/unix/unix_diag.ko 2024-08-31T06:53:20.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock 2024-08-31T06:53:20.090 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-08-31T06:53:20.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-08-31T06:53:20.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock.ko 2024-08-31T06:53:20.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock_diag.ko 2024-08-31T06:53:20.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock_loopback.ko 2024-08-31T06:53:20.091 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/xfrm 2024-08-31T06:53:20.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/xfrm/xfrm_ipcomp.ko 2024-08-31T06:53:20.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security 2024-08-31T06:53:20.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys 2024-08-31T06:53:20.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/encrypted-keys 2024-08-31T06:53:20.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-08-31T06:53:20.092 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/trusted-keys 2024-08-31T06:53:20.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/trusted-keys/trusted.ko 2024-08-31T06:53:20.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.alias 2024-08-31T06:53:20.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.alias.bin 2024-08-31T06:53:20.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin 2024-08-31T06:53:20.093 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.alias.bin 2024-08-31T06:53:20.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.bin 2024-08-31T06:53:20.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.modinfo 2024-08-31T06:53:20.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep 2024-08-31T06:53:20.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep.bin 2024-08-31T06:53:20.094 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.devname 2024-08-31T06:53:20.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.order 2024-08-31T06:53:20.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.softdep 2024-08-31T06:53:20.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.symbols 2024-08-31T06:53:20.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.symbols.bin 2024-08-31T06:53:20.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.weakdep 2024-08-31T06:53:20.095 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/vmlinuz 2024-08-31T06:53:20.101 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:53:20.101 DEBUG:teuthology.orchestra.run.smithi032:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-08-31T06:53:20.124 INFO:teuthology.orchestra.run.smithi032.stdout:postinstall scriptlet (using /bin/sh): 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g8ad8b8b16098 /lib/modules/6.11.0-rc5-g8ad8b8b16098/vmlinuz 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout:for file in vmlinuz System.map config; do 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g8ad8b8b16098/${file}" "/boot/${file}-6.11.0-rc5-g8ad8b8b16098"; then 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout: cp "/lib/modules/6.11.0-rc5-g8ad8b8b16098/${file}" "/boot/${file}-6.11.0-rc5-g8ad8b8b16098" 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout: fi 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout:done 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout:if [ -d "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" ] && \ 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" >/dev/null 2>&1; then 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" 2024-08-31T06:53:20.125 INFO:teuthology.orchestra.run.smithi032.stdout: cp -r "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" 2024-08-31T06:53:20.126 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2024-08-31T06:53:20.126 INFO:teuthology.orchestra.run.smithi032.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep" ]; then 2024-08-31T06:53:20.126 INFO:teuthology.orchestra.run.smithi032.stdout: /usr/sbin/depmod 6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:53:20.126 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2024-08-31T06:53:20.126 INFO:teuthology.orchestra.run.smithi032.stdout:preuninstall scriptlet (using /bin/sh): 2024-08-31T06:53:20.126 INFO:teuthology.orchestra.run.smithi032.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-31T06:53:20.126 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-install remove 6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:53:20.126 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2024-08-31T06:53:20.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo rpm -qi grub2-tools 2024-08-31T06:53:20.200 INFO:teuthology.orchestra.run.smithi032.stdout:Name : grub2-tools 2024-08-31T06:53:20.201 INFO:teuthology.orchestra.run.smithi032.stdout:Epoch : 1 2024-08-31T06:53:20.201 INFO:teuthology.orchestra.run.smithi032.stdout:Version : 2.06 2024-08-31T06:53:20.201 INFO:teuthology.orchestra.run.smithi032.stdout:Release : 82.el9 2024-08-31T06:53:20.201 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: x86_64 2024-08-31T06:53:20.201 INFO:teuthology.orchestra.run.smithi032.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-08-31T06:53:20.201 INFO:teuthology.orchestra.run.smithi032.stdout:Group : Unspecified 2024-08-31T06:53:20.201 INFO:teuthology.orchestra.run.smithi032.stdout:Size : 8274670 2024-08-31T06:53:20.201 INFO:teuthology.orchestra.run.smithi032.stdout:License : GPLv3+ 2024-08-31T06:53:20.201 INFO:teuthology.orchestra.run.smithi032.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:Packager : builder@centos.org 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:Vendor : CentOS 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:Summary : Support tools for GRUB. 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:Description : 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-31T06:53:20.202 INFO:teuthology.orchestra.run.smithi032.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-31T06:53:20.203 INFO:teuthology.orchestra.run.smithi032.stdout:hardware devices. 2024-08-31T06:53:20.203 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:53:20.203 INFO:teuthology.orchestra.run.smithi032.stdout:This subpackage provides tools for support of all platforms. 2024-08-31T06:53:20.204 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-31T06:53:20.204 INFO:teuthology.task.kernel:Updating grub on smithi032 to boot 6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:53:20.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-31T06:53:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2024-08-31T06:53:22.269 INFO:teuthology.orchestra.run.smithi032.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-31T06:53:22.295 INFO:teuthology.orchestra.run.smithi032.stderr:done 2024-08-31T06:53:22.297 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-08-31T06:53:22.312 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.cF7lsFwvZi 2024-08-31T06:53:22.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.cF7lsFwvZi 2024-08-31T06:53:22.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.cF7lsFwvZi 2024-08-31T06:53:22.529 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.cF7lsFwvZi is 6KB 2024-08-31T06:53:22.577 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.cF7lsFwvZi 2024-08-31T06:53:22.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo /bin/ls /boot/loader/entries || true 2024-08-31T06:53:22.654 INFO:teuthology.orchestra.run.smithi032.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-08-31T06:53:22.654 INFO:teuthology.orchestra.run.smithi032.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-08-31T06:53:22.654 INFO:teuthology.orchestra.run.smithi032.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-08-31T06:53:22.654 INFO:teuthology.orchestra.run.smithi032.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g8ad8b8b16098.conf 2024-08-31T06:53:22.656 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:53:22.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2024-08-31T06:53:52.953 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-08-31T06:53:52.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-08-31T06:53:52.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T06:54:11.517 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-08-31T06:54:20.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-08-31T06:54:20.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T06:54:38.909 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-08-31T06:54:50.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-08-31T06:54:50.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T06:54:53.985 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-08-31T06:55:08.985 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-08-31T06:55:08.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T06:55:09.350 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-08-31T06:55:10.110 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-08-31T06:55:10.110 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g8ad8b8b16098"... 2024-08-31T06:55:10.111 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-08-31T06:55:10.193 INFO:teuthology.orchestra.run.smithi032.stdout:6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:55:10.193 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g8ad8b8b16098 vs 6.11.0-rc5-g8ad8b8b16098 2024-08-31T06:55:10.193 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-31T06:55:10.193 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-31T06:55:11.194 DEBUG:teuthology.parallel:result is None 2024-08-31T06:55:11.194 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-31T06:55:11.204 INFO:teuthology.task.internal:Creating test directory... 2024-08-31T06:55:11.204 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-31T06:55:11.208 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-31T06:55:11.210 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-31T06:55:11.239 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-31T06:55:11.315 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-31T06:55:11.354 INFO:teuthology.task.internal:Creating archive directory... 2024-08-31T06:55:11.354 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-31T06:55:11.359 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-31T06:55:11.362 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-31T06:55:11.409 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-31T06:55:11.417 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-31T06:55:11.417 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-31T06:55:11.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:55:11.445 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-31T06:55:11.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:55:11.462 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-31T06:55:11.466 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-31T06:55:11.488 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-31T06:55:11.539 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T06:55:11.553 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T06:55:11.569 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T06:55:11.577 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T06:55:11.585 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T06:55:11.609 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T06:55:11.612 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-31T06:55:11.619 INFO:teuthology.task.internal:Configuring sudo... 2024-08-31T06:55:11.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-31T06:55:11.623 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-31T06:55:11.625 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-31T06:55:11.675 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-31T06:55:11.684 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-31T06:55:11.684 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-31T06:55:11.687 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-31T06:55:11.694 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-31T06:55:11.718 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T06:55:11.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T06:55:12.174 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T06:55:12.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T06:55:12.313 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T06:55:12.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-31T06:55:12.368 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T06:55:12.416 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T06:55:12.469 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T06:55:12.552 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T06:55:12.598 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T06:55:12.598 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-31T06:55:12.664 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T06:55:12.711 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T06:55:12.950 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T06:55:13.032 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T06:55:13.077 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T06:55:13.077 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-31T06:55:13.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-08-31T06:55:13.149 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-08-31T06:55:13.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2024-08-31T06:55:13.190 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T06:55:13.235 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T06:55:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T06:55:13.752 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-31T06:55:13.759 INFO:teuthology.task.internal:Starting timer... 2024-08-31T06:55:13.760 INFO:teuthology.run_tasks:Running task pcp... 2024-08-31T06:55:13.799 INFO:teuthology.run_tasks:Running task selinux... 2024-08-31T06:55:13.830 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-08-31T06:55:13.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2024-08-31T06:55:13.919 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2024-08-31T06:55:13.922 DEBUG:teuthology.orchestra.run.smithi073:> sudo service auditd rotate 2024-08-31T06:55:14.003 INFO:teuthology.orchestra.run.smithi073.stdout:Rotating logs: 2024-08-31T06:55:14.005 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2024-08-31T06:55:14.067 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2024-08-31T06:55:14.069 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-31T06:55:14.069 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2024-08-31T06:55:14.110 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2024-08-31T06:55:14.110 DEBUG:teuthology.orchestra.run.smithi073:> /usr/sbin/getenforce 2024-08-31T06:55:14.153 INFO:teuthology.orchestra.run.smithi073.stdout:Permissive 2024-08-31T06:55:14.153 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2024-08-31T06:55:14.183 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2024-08-31T06:55:14.183 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi073.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi084.front.sepia.ceph.com': 'permissive'} 2024-08-31T06:55:14.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T06:55:14.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:55:14.240 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T06:55:14.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:55:14.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T06:55:14.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:55:14.296 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-31T06:55:14.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2024-08-31T06:55:14.356 DEBUG:teuthology.orchestra.run.smithi073:> sudo /usr/sbin/setenforce permissive 2024-08-31T06:55:14.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2024-08-31T06:55:14.419 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-31T06:55:14.429 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-08-31T06:55:14.433 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-08-31T06:55:14.562 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-31T06:55:14.572 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-31T06:55:14.589 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi073.front.sepia.ceph.com,smithi084.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-31T06:57:23.262 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi084.front.sepia.ceph.com')] 2024-08-31T06:57:23.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-08-31T06:57:23.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T06:57:23.356 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-08-31T06:57:23.463 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-08-31T06:57:23.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-08-31T06:57:23.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T06:57:23.539 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-08-31T06:57:23.620 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-08-31T06:57:23.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-08-31T06:57:23.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T06:57:23.695 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-08-31T06:57:23.783 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-08-31T06:57:23.783 INFO:teuthology.run_tasks:Running task clock... 2024-08-31T06:57:23.792 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-31T06:57:23.793 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-31T06:57:23.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T06:57:23.797 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-31T06:57:23.797 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T06:57:23.800 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-31T06:57:23.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T06:57:23.828 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-31T06:57:23.843 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-31T06:57:23.856 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-31T06:57:23.858 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-31T06:57:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: ntpd: command not found 2024-08-31T06:57:23.872 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-31T06:57:23.878 INFO:teuthology.orchestra.run.smithi073.stdout:506 Cannot talk to daemon 2024-08-31T06:57:23.889 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-31T06:57:23.892 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-31T06:57:23.898 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2024-08-31T06:57:23.906 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-31T06:57:23.910 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2024-08-31T06:57:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-31T06:57:23.939 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-31T06:57:23.941 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2024-08-31T06:57:23.965 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2024-08-31T06:57:23.968 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-08-31T06:57:23.973 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T06:57:23.973 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-08-31T06:57:23.973 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:23.973 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:23.973 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:23.973 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:23.994 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-31T06:57:23.995 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-08-31T06:57:23.999 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T06:57:23.999 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-08-31T06:57:23.999 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:23.999 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:23.999 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:23.999 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-31T06:57:24.126 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-08-31T06:57:24.133 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T06:57:24.133 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-08-31T06:57:24.133 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:24.133 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:24.133 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:24.133 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:57:24.134 INFO:teuthology.run_tasks:Running task install... 2024-08-31T06:57:24.142 DEBUG:teuthology.task.install:project ceph 2024-08-31T06:57:24.142 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6'}} 2024-08-31T06:57:24.142 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6'} 2024-08-31T06:57:24.143 INFO:teuthology.task.install:Using flavor: default 2024-08-31T06:57:24.147 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-31T06:57:24.147 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-31T06:57:24.148 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'tag': None, 'wait_for_package': False} 2024-08-31T06:57:24.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T06:57:24.150 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'tag': None, 'wait_for_package': False} 2024-08-31T06:57:24.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T06:57:24.151 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'tag': None, 'wait_for_package': False} 2024-08-31T06:57:24.151 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T06:57:24.289 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/ 2024-08-31T06:57:24.289 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4560.gd19e6118 2024-08-31T06:57:24.306 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/ 2024-08-31T06:57:24.307 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4560.gd19e6118 2024-08-31T06:57:24.312 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/ 2024-08-31T06:57:24.312 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4560.gd19e6118 2024-08-31T06:57:24.399 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-31T06:57:24.399 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T06:57:24.399 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-31T06:57:24.415 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-31T06:57:24.416 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T06:57:24.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-31T06:57:24.419 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-31T06:57:24.420 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T06:57:24.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-31T06:57:24.425 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-31T06:57:24.425 DEBUG:teuthology.orchestra.run.smithi073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d19e6118f7015f0047264006cb0c873528eb3ae6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-31T06:57:24.446 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-31T06:57:24.446 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d19e6118f7015f0047264006cb0c873528eb3ae6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-31T06:57:24.472 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-31T06:57:24.472 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d19e6118f7015f0047264006cb0c873528eb3ae6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-31T06:57:24.491 DEBUG:teuthology.orchestra.run.smithi073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-31T06:57:24.512 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-31T06:57:24.567 DEBUG:teuthology.orchestra.run.smithi073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-31T06:57:24.571 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-31T06:57:24.587 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-31T06:57:24.631 INFO:teuthology.orchestra.run.smithi073.stdout:check_obsoletes = 1 2024-08-31T06:57:24.633 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum clean all 2024-08-31T06:57:24.649 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-31T06:57:24.651 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2024-08-31T06:57:24.654 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2024-08-31T06:57:24.704 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2024-08-31T06:57:24.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2024-08-31T06:57:24.856 INFO:teuthology.orchestra.run.smithi073.stdout:45 files removed 2024-08-31T06:57:24.878 INFO:teuthology.orchestra.run.smithi084.stdout:45 files removed 2024-08-31T06:57:24.883 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-31T06:57:24.904 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-31T06:57:24.981 INFO:teuthology.orchestra.run.smithi032.stdout:45 files removed 2024-08-31T06:57:25.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-31T06:57:25.510 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 255 kB/s | 84 kB 00:00 2024-08-31T06:57:25.514 INFO:teuthology.orchestra.run.smithi073.stdout:ceph packages for x86_64 234 kB/s | 84 kB 00:00 2024-08-31T06:57:25.697 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-08-31T06:57:25.799 INFO:teuthology.orchestra.run.smithi073.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-08-31T06:57:25.811 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-08-31T06:57:26.023 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 87 kB/s | 24 kB 00:00 2024-08-31T06:57:26.074 INFO:teuthology.orchestra.run.smithi073.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-08-31T06:57:26.078 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-08-31T06:57:26.315 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-08-31T06:57:26.695 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-08-31T06:57:27.539 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.2 MB 00:01 2024-08-31T06:57:27.583 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.2 MB 00:01 2024-08-31T06:57:29.913 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 20 MB 00:02 2024-08-31T06:57:30.460 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-08-31T06:57:31.908 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 20 MB 00:03 2024-08-31T06:57:34.605 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 6.6 MB 00:00 2024-08-31T06:57:36.164 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 18 kB 00:00 2024-08-31T06:57:36.452 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.6 MB 00:02 2024-08-31T06:57:37.293 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 22 MB/s | 22 MB 00:01 2024-08-31T06:57:38.144 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 18 kB 00:00 2024-08-31T06:57:38.915 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 35 MB/s | 22 MB 00:00 2024-08-31T06:57:44.190 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 1.0 kB/s | 1.7 kB 00:01 2024-08-31T06:57:44.904 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 2.9 kB/s | 1.7 kB 00:00 2024-08-31T06:57:46.494 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T06:57:46.495 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T06:57:46.508 INFO:teuthology.orchestra.run.smithi084.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-31T06:57:46.509 INFO:teuthology.orchestra.run.smithi084.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-31T06:57:46.510 INFO:teuthology.orchestra.run.smithi084.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-08-31T06:57:46.511 INFO:teuthology.orchestra.run.smithi084.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-31T06:57:46.513 INFO:teuthology.orchestra.run.smithi084.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-31T06:57:46.515 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-31T06:57:46.517 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-31T06:57:46.576 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 6.6 k 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.5 M 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 874 k 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 148 k 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 1.5 M 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 154 k 2024-08-31T06:57:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 14 M 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 7.4 M 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 49 k 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 10 M 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 44 M 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 285 k 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 781 k 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 226 k 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-shell x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 27 k 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 18 k 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-31T06:57:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 32 k 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 766 k 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 127 k 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 161 k 2024-08-31T06:57:46.588 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 321 k 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 305 k 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 99 k 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 86 k 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 170 k 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.4 M 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-31T06:57:46.589 INFO:teuthology.orchestra.run.smithi084.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 20 M 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 34 k 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 2.3 M 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 290 k 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 4.9 M 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 18 M 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 16 k 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 25 k 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-31T06:57:46.590 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 159 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 506 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.3 M 2024-08-31T06:57:46.591 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-31T06:57:46.592 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 45 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 141 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-31T06:57:46.593 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-31T06:57:46.594 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-31T06:57:46.595 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-31T06:57:46.596 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-31T06:57:46.597 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout:Installing weak dependencies: 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout:Install 148 Packages 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 2 Packages 2024-08-31T06:57:46.598 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:57:46.599 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 224 M 2024-08-31T06:57:46.599 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-08-31T06:57:47.008 INFO:teuthology.orchestra.run.smithi084.stdout:(1/150): ceph-19.3.0-4560.gd19e6118.el9.x86_64. 52 kB/s | 6.6 kB 00:00 2024-08-31T06:57:47.233 INFO:teuthology.orchestra.run.smithi084.stdout:(2/150): ceph-fuse-19.3.0-4560.gd19e6118.el9.x8 3.8 MB/s | 874 kB 00:00 2024-08-31T06:57:47.240 INFO:teuthology.orchestra.run.smithi073.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T06:57:47.241 INFO:teuthology.orchestra.run.smithi073.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T06:57:47.252 INFO:teuthology.orchestra.run.smithi073.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-31T06:57:47.253 INFO:teuthology.orchestra.run.smithi073.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-31T06:57:47.254 INFO:teuthology.orchestra.run.smithi073.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-08-31T06:57:47.254 INFO:teuthology.orchestra.run.smithi073.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-31T06:57:47.256 INFO:teuthology.orchestra.run.smithi073.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-31T06:57:47.258 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-31T06:57:47.259 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-31T06:57:47.321 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-08-31T06:57:47.330 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-08-31T06:57:47.330 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-08-31T06:57:47.330 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-08-31T06:57:47.330 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-08-31T06:57:47.330 INFO:teuthology.orchestra.run.smithi073.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-31T06:57:47.330 INFO:teuthology.orchestra.run.smithi073.stdout: ceph x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 6.6 k 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.5 M 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 874 k 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 148 k 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 1.5 M 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 154 k 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 14 M 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 7.4 M 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 49 k 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 10 M 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 44 M 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 285 k 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 781 k 2024-08-31T06:57:47.331 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 226 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-shell x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 27 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 18 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 32 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 766 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 127 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-31T06:57:47.332 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 161 k 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 321 k 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 305 k 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 99 k 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 86 k 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 170 k 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: librados2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.4 M 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-08-31T06:57:47.333 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 20 M 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 34 k 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 2.3 M 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 290 k 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 4.9 M 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 18 M 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 16 k 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 25 k 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-31T06:57:47.334 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 159 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 506 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-31T06:57:47.335 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.3 M 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-31T06:57:47.336 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 45 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 141 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-31T06:57:47.337 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-31T06:57:47.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-31T06:57:47.339 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-31T06:57:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-31T06:57:47.341 INFO:teuthology.orchestra.run.smithi073.stdout:Installing weak dependencies: 2024-08-31T06:57:47.342 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-31T06:57:47.342 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:57:47.342 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-08-31T06:57:47.342 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-08-31T06:57:47.342 INFO:teuthology.orchestra.run.smithi073.stdout:Install 148 Packages 2024-08-31T06:57:47.342 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 2 Packages 2024-08-31T06:57:47.342 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:57:47.342 INFO:teuthology.orchestra.run.smithi084.stdout:(3/150): ceph-base-19.3.0-4560.gd19e6118.el9.x8 12 MB/s | 5.5 MB 00:00 2024-08-31T06:57:47.343 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 224 M 2024-08-31T06:57:47.343 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-08-31T06:57:47.367 INFO:teuthology.orchestra.run.smithi084.stdout:(4/150): ceph-immutable-object-cache-19.3.0-456 1.1 MB/s | 148 kB 00:00 2024-08-31T06:57:47.601 INFO:teuthology.orchestra.run.smithi084.stdout:(5/150): ceph-mds-19.3.0-4560.gd19e6118.el9.x86 8.8 MB/s | 2.3 MB 00:00 2024-08-31T06:57:47.643 INFO:teuthology.orchestra.run.smithi084.stdout:(6/150): ceph-mgr-19.3.0-4560.gd19e6118.el9.x86 5.5 MB/s | 1.5 MB 00:00 2024-08-31T06:57:47.748 INFO:teuthology.orchestra.run.smithi073.stdout:(1/150): ceph-19.3.0-4560.gd19e6118.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-08-31T06:57:47.818 INFO:teuthology.orchestra.run.smithi084.stdout:(7/150): ceph-mon-19.3.0-4560.gd19e6118.el9.x86 23 MB/s | 4.9 MB 00:00 2024-08-31T06:57:48.198 INFO:teuthology.orchestra.run.smithi073.stdout:(2/150): ceph-fuse-19.3.0-4560.gd19e6118.el9.x8 1.9 MB/s | 874 kB 00:00 2024-08-31T06:57:48.251 INFO:teuthology.orchestra.run.smithi084.stdout:(8/150): ceph-common-19.3.0-4560.gd19e6118.el9. 15 MB/s | 20 MB 00:01 2024-08-31T06:57:48.264 INFO:teuthology.orchestra.run.smithi073.stdout:(3/150): ceph-immutable-object-cache-19.3.0-456 2.2 MB/s | 148 kB 00:00 2024-08-31T06:57:48.302 INFO:teuthology.orchestra.run.smithi084.stdout:(9/150): ceph-selinux-19.3.0-4560.gd19e6118.el9 505 kB/s | 25 kB 00:00 2024-08-31T06:57:48.365 INFO:teuthology.orchestra.run.smithi073.stdout:(4/150): ceph-base-19.3.0-4560.gd19e6118.el9.x8 6.7 MB/s | 5.5 MB 00:00 2024-08-31T06:57:48.494 INFO:teuthology.orchestra.run.smithi084.stdout:(10/150): ceph-radosgw-19.3.0-4560.gd19e6118.el 15 MB/s | 10 MB 00:00 2024-08-31T06:57:48.515 INFO:teuthology.orchestra.run.smithi073.stdout:(5/150): ceph-mgr-19.3.0-4560.gd19e6118.el9.x86 10 MB/s | 1.5 MB 00:00 2024-08-31T06:57:48.525 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 542 kB/s | 6.6 MB 00:12 2024-08-31T06:57:48.560 INFO:teuthology.orchestra.run.smithi084.stdout:(11/150): cephfs-mirror-19.3.0-4560.gd19e6118.e 3.3 MB/s | 226 kB 00:00 2024-08-31T06:57:48.627 INFO:teuthology.orchestra.run.smithi084.stdout:(12/150): cephfs-shell-19.3.0-4560.gd19e6118.el 407 kB/s | 27 kB 00:00 2024-08-31T06:57:48.694 INFO:teuthology.orchestra.run.smithi084.stdout:(13/150): libcephfs-devel-19.3.0-4560.gd19e6118 476 kB/s | 32 kB 00:00 2024-08-31T06:57:48.786 INFO:teuthology.orchestra.run.smithi084.stdout:(14/150): libcephfs2-19.3.0-4560.gd19e6118.el9. 8.2 MB/s | 766 kB 00:00 2024-08-31T06:57:48.816 INFO:teuthology.orchestra.run.smithi073.stdout:(6/150): ceph-mds-19.3.0-4560.gd19e6118.el9.x86 4.1 MB/s | 2.3 MB 00:00 2024-08-31T06:57:48.845 INFO:teuthology.orchestra.run.smithi084.stdout:(15/150): libcephsqlite-19.3.0-4560.gd19e6118.e 2.7 MB/s | 159 kB 00:00 2024-08-31T06:57:48.920 INFO:teuthology.orchestra.run.smithi084.stdout:(16/150): librados-devel-19.3.0-4560.gd19e6118. 1.7 MB/s | 127 kB 00:00 2024-08-31T06:57:49.004 INFO:teuthology.orchestra.run.smithi084.stdout:(17/150): libradosstriper1-19.3.0-4560.gd19e611 5.9 MB/s | 506 kB 00:00 2024-08-31T06:57:49.066 INFO:teuthology.orchestra.run.smithi073.stdout:(7/150): ceph-mon-19.3.0-4560.gd19e6118.el9.x86 8.9 MB/s | 4.9 MB 00:00 2024-08-31T06:57:49.437 INFO:teuthology.orchestra.run.smithi084.stdout:(18/150): librgw2-19.3.0-4560.gd19e6118.el9.x86 12 MB/s | 5.3 MB 00:00 2024-08-31T06:57:49.621 INFO:teuthology.orchestra.run.smithi084.stdout:(19/150): ceph-osd-19.3.0-4560.gd19e6118.el9.x8 9.0 MB/s | 18 MB 00:01 2024-08-31T06:57:49.646 INFO:teuthology.orchestra.run.smithi084.stdout:(20/150): python3-ceph-argparse-19.3.0-4560.gd1 216 kB/s | 45 kB 00:00 2024-08-31T06:57:49.680 INFO:teuthology.orchestra.run.smithi084.stdout:(21/150): python3-ceph-common-19.3.0-4560.gd19e 2.4 MB/s | 141 kB 00:00 2024-08-31T06:57:49.705 INFO:teuthology.orchestra.run.smithi084.stdout:(22/150): python3-cephfs-19.3.0-4560.gd19e6118. 2.7 MB/s | 161 kB 00:00 2024-08-31T06:57:49.755 INFO:teuthology.orchestra.run.smithi084.stdout:(23/150): python3-rados-19.3.0-4560.gd19e6118.e 4.2 MB/s | 321 kB 00:00 2024-08-31T06:57:49.780 INFO:teuthology.orchestra.run.smithi084.stdout:(24/150): python3-rbd-19.3.0-4560.gd19e6118.el9 4.0 MB/s | 305 kB 00:00 2024-08-31T06:57:49.831 INFO:teuthology.orchestra.run.smithi084.stdout:(25/150): python3-rgw-19.3.0-4560.gd19e6118.el9 1.3 MB/s | 99 kB 00:00 2024-08-31T06:57:49.856 INFO:teuthology.orchestra.run.smithi084.stdout:(26/150): rbd-fuse-19.3.0-4560.gd19e6118.el9.x8 1.1 MB/s | 86 kB 00:00 2024-08-31T06:57:49.940 INFO:teuthology.orchestra.run.smithi084.stdout:(27/150): rbd-nbd-19.3.0-4560.gd19e6118.el9.x86 2.0 MB/s | 170 kB 00:00 2024-08-31T06:57:50.015 INFO:teuthology.orchestra.run.smithi084.stdout:(28/150): ceph-grafana-dashboards-19.3.0-4560.g 457 kB/s | 34 kB 00:00 2024-08-31T06:57:50.083 INFO:teuthology.orchestra.run.smithi073.stdout:(8/150): ceph-radosgw-19.3.0-4560.gd19e6118.el9 10 MB/s | 10 MB 00:01 2024-08-31T06:57:50.091 INFO:teuthology.orchestra.run.smithi084.stdout:(29/150): ceph-mgr-cephadm-19.3.0-4560.gd19e611 2.0 MB/s | 154 kB 00:00 2024-08-31T06:57:50.133 INFO:teuthology.orchestra.run.smithi073.stdout:(9/150): ceph-selinux-19.3.0-4560.gd19e6118.el9 504 kB/s | 25 kB 00:00 2024-08-31T06:57:50.142 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 18 kB 00:00 2024-08-31T06:57:50.191 INFO:teuthology.orchestra.run.smithi084.stdout:(30/150): rbd-mirror-19.3.0-4560.gd19e6118.el9. 8.6 MB/s | 3.1 MB 00:00 2024-08-31T06:57:50.450 INFO:teuthology.orchestra.run.smithi073.stdout:(10/150): ceph-osd-19.3.0-4560.gd19e6118.el9.x8 11 MB/s | 18 MB 00:01 2024-08-31T06:57:50.500 INFO:teuthology.orchestra.run.smithi073.stdout:(11/150): cephfs-mirror-19.3.0-4560.gd19e6118.e 4.4 MB/s | 226 kB 00:00 2024-08-31T06:57:50.542 INFO:teuthology.orchestra.run.smithi073.stdout:(12/150): cephfs-shell-19.3.0-4560.gd19e6118.el 649 kB/s | 27 kB 00:00 2024-08-31T06:57:50.591 INFO:teuthology.orchestra.run.smithi084.stdout:(31/150): ceph-test-19.3.0-4560.gd19e6118.el9.x 19 MB/s | 44 MB 00:02 2024-08-31T06:57:50.592 INFO:teuthology.orchestra.run.smithi073.stdout:(13/150): libcephfs-devel-19.3.0-4560.gd19e6118 634 kB/s | 32 kB 00:00 2024-08-31T06:57:50.650 INFO:teuthology.orchestra.run.smithi084.stdout:(32/150): ceph-mgr-modules-core-19.3.0-4560.gd1 4.9 MB/s | 290 kB 00:00 2024-08-31T06:57:50.676 INFO:teuthology.orchestra.run.smithi073.stdout:(14/150): libcephfs2-19.3.0-4560.gd19e6118.el9. 9.0 MB/s | 766 kB 00:00 2024-08-31T06:57:50.717 INFO:teuthology.orchestra.run.smithi084.stdout:(33/150): ceph-mgr-rook-19.3.0-4560.gd19e6118.e 741 kB/s | 49 kB 00:00 2024-08-31T06:57:50.743 INFO:teuthology.orchestra.run.smithi073.stdout:(15/150): libcephsqlite-19.3.0-4560.gd19e6118.e 2.3 MB/s | 159 kB 00:00 2024-08-31T06:57:50.783 INFO:teuthology.orchestra.run.smithi084.stdout:(34/150): ceph-prometheus-alerts-19.3.0-4560.gd 247 kB/s | 16 kB 00:00 2024-08-31T06:57:50.810 INFO:teuthology.orchestra.run.smithi073.stdout:(16/150): librados-devel-19.3.0-4560.gd19e6118. 1.9 MB/s | 127 kB 00:00 2024-08-31T06:57:50.859 INFO:teuthology.orchestra.run.smithi084.stdout:(35/150): ceph-volume-19.3.0-4560.gd19e6118.el9 3.7 MB/s | 285 kB 00:00 2024-08-31T06:57:50.893 INFO:teuthology.orchestra.run.smithi073.stdout:(17/150): libradosstriper1-19.3.0-4560.gd19e611 6.0 MB/s | 506 kB 00:00 2024-08-31T06:57:51.059 INFO:teuthology.orchestra.run.smithi084.stdout:(36/150): ceph-mgr-diskprediction-local-19.3.0- 8.5 MB/s | 7.4 MB 00:00 2024-08-31T06:57:51.093 INFO:teuthology.orchestra.run.smithi084.stdout:(37/150): cephadm-19.3.0-4560.gd19e6118.el9.noa 3.3 MB/s | 781 kB 00:00 2024-08-31T06:57:51.118 INFO:teuthology.orchestra.run.smithi084.stdout:(38/150): cephfs-top-19.3.0-4560.gd19e6118.el9. 313 kB/s | 18 kB 00:00 2024-08-31T06:57:51.202 INFO:teuthology.orchestra.run.smithi073.stdout:(18/150): librgw2-19.3.0-4560.gd19e6118.el9.x86 17 MB/s | 5.3 MB 00:00 2024-08-31T06:57:51.221 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 24 MB/s | 22 MB 00:00 2024-08-31T06:57:51.252 INFO:teuthology.orchestra.run.smithi073.stdout:(19/150): python3-ceph-argparse-19.3.0-4560.gd1 907 kB/s | 45 kB 00:00 2024-08-31T06:57:51.260 INFO:teuthology.orchestra.run.smithi084.stdout:(39/150): ceph-mgr-dashboard-19.3.0-4560.gd19e6 12 MB/s | 14 MB 00:01 2024-08-31T06:57:51.302 INFO:teuthology.orchestra.run.smithi073.stdout:(20/150): python3-ceph-common-19.3.0-4560.gd19e 2.8 MB/s | 141 kB 00:00 2024-08-31T06:57:51.353 INFO:teuthology.orchestra.run.smithi073.stdout:(21/150): python3-cephfs-19.3.0-4560.gd19e6118. 3.2 MB/s | 161 kB 00:00 2024-08-31T06:57:51.369 INFO:teuthology.orchestra.run.smithi084.stdout:(40/150): fuse-2.9.9-16.el9.x86_64.rpm 290 kB/s | 80 kB 00:00 2024-08-31T06:57:51.403 INFO:teuthology.orchestra.run.smithi073.stdout:(22/150): python3-rados-19.3.0-4560.gd19e6118.e 6.3 MB/s | 321 kB 00:00 2024-08-31T06:57:51.419 INFO:teuthology.orchestra.run.smithi084.stdout:(41/150): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-08-31T06:57:51.462 INFO:teuthology.orchestra.run.smithi073.stdout:(23/150): python3-rbd-19.3.0-4560.gd19e6118.el9 5.1 MB/s | 305 kB 00:00 2024-08-31T06:57:51.485 INFO:teuthology.orchestra.run.smithi084.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-08-31T06:57:51.512 INFO:teuthology.orchestra.run.smithi073.stdout:(24/150): python3-rgw-19.3.0-4560.gd19e6118.el9 1.9 MB/s | 99 kB 00:00 2024-08-31T06:57:51.519 INFO:teuthology.orchestra.run.smithi084.stdout:(43/150): libgfortran-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 801 kB 00:00 2024-08-31T06:57:51.544 INFO:teuthology.orchestra.run.smithi084.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-08-31T06:57:51.554 INFO:teuthology.orchestra.run.smithi073.stdout:(25/150): rbd-fuse-19.3.0-4560.gd19e6118.el9.x8 2.0 MB/s | 86 kB 00:00 2024-08-31T06:57:51.570 INFO:teuthology.orchestra.run.smithi084.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-08-31T06:57:51.612 INFO:teuthology.orchestra.run.smithi084.stdout:(46/150): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-08-31T06:57:51.629 INFO:teuthology.orchestra.run.smithi084.stdout:(47/150): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-08-31T06:57:51.654 INFO:teuthology.orchestra.run.smithi084.stdout:(48/150): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-08-31T06:57:51.713 INFO:teuthology.orchestra.run.smithi073.stdout:(26/150): rbd-mirror-19.3.0-4560.gd19e6118.el9. 20 MB/s | 3.1 MB 00:00 2024-08-31T06:57:51.763 INFO:teuthology.orchestra.run.smithi073.stdout:(27/150): rbd-nbd-19.3.0-4560.gd19e6118.el9.x86 3.3 MB/s | 170 kB 00:00 2024-08-31T06:57:51.813 INFO:teuthology.orchestra.run.smithi073.stdout:(28/150): ceph-grafana-dashboards-19.3.0-4560.g 683 kB/s | 34 kB 00:00 2024-08-31T06:57:51.864 INFO:teuthology.orchestra.run.smithi073.stdout:(29/150): ceph-mgr-cephadm-19.3.0-4560.gd19e611 3.0 MB/s | 154 kB 00:00 2024-08-31T06:57:52.356 INFO:teuthology.orchestra.run.smithi073.stdout:(30/150): ceph-mgr-dashboard-19.3.0-4560.gd19e6 29 MB/s | 14 MB 00:00 2024-08-31T06:57:52.631 INFO:teuthology.orchestra.run.smithi073.stdout:(31/150): ceph-mgr-diskprediction-local-19.3.0- 27 MB/s | 7.4 MB 00:00 2024-08-31T06:57:52.690 INFO:teuthology.orchestra.run.smithi073.stdout:(32/150): ceph-mgr-modules-core-19.3.0-4560.gd1 4.9 MB/s | 290 kB 00:00 2024-08-31T06:57:52.740 INFO:teuthology.orchestra.run.smithi073.stdout:(33/150): ceph-mgr-rook-19.3.0-4560.gd19e6118.e 988 kB/s | 49 kB 00:00 2024-08-31T06:57:52.790 INFO:teuthology.orchestra.run.smithi073.stdout:(34/150): ceph-prometheus-alerts-19.3.0-4560.gd 328 kB/s | 16 kB 00:00 2024-08-31T06:57:52.841 INFO:teuthology.orchestra.run.smithi073.stdout:(35/150): ceph-volume-19.3.0-4560.gd19e6118.el9 5.6 MB/s | 285 kB 00:00 2024-08-31T06:57:52.908 INFO:teuthology.orchestra.run.smithi073.stdout:(36/150): cephadm-19.3.0-4560.gd19e6118.el9.noa 11 MB/s | 781 kB 00:00 2024-08-31T06:57:53.333 INFO:teuthology.orchestra.run.smithi073.stdout:(37/150): ceph-test-19.3.0-4560.gd19e6118.el9.x 14 MB/s | 44 MB 00:03 2024-08-31T06:57:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:(38/150): cephfs-top-19.3.0-4560.gd19e6118.el9. 40 kB/s | 18 kB 00:00 2024-08-31T06:57:53.867 INFO:teuthology.orchestra.run.smithi073.stdout:(39/150): ceph-common-19.3.0-4560.gd19e6118.el9 3.2 MB/s | 20 MB 00:06 2024-08-31T06:57:57.237 INFO:teuthology.orchestra.run.smithi084.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 19 kB/s | 107 kB 00:05 2024-08-31T06:57:57.303 INFO:teuthology.orchestra.run.smithi084.stdout:(50/150): automake-1.16.2-8.el9.noarch.rpm 121 kB/s | 689 kB 00:05 2024-08-31T06:57:57.329 INFO:teuthology.orchestra.run.smithi084.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 236 kB/s | 22 kB 00:00 2024-08-31T06:57:57.354 INFO:teuthology.orchestra.run.smithi084.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 165 kB/s | 943 kB 00:05 2024-08-31T06:57:57.836 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-08-31T06:57:59.529 INFO:teuthology.orchestra.run.smithi084.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 141 kB/s | 313 kB 00:02 2024-08-31T06:58:00.409 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T06:58:00.410 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T06:58:00.424 INFO:teuthology.orchestra.run.smithi032.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-31T06:58:00.425 INFO:teuthology.orchestra.run.smithi032.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-31T06:58:00.426 INFO:teuthology.orchestra.run.smithi032.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-08-31T06:58:00.427 INFO:teuthology.orchestra.run.smithi032.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-31T06:58:00.429 INFO:teuthology.orchestra.run.smithi032.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-31T06:58:00.432 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-31T06:58:00.433 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-31T06:58:00.500 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout:======================================================================================= 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout:======================================================================================= 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 6.6 k 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.5 M 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 874 k 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 148 k 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 1.5 M 2024-08-31T06:58:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 154 k 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 14 M 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 7.4 M 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 49 k 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 10 M 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 44 M 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 285 k 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 781 k 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 226 k 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-shell x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 27 k 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 18 k 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-31T06:58:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 32 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 766 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 127 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 161 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 321 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 305 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 99 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 86 k 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T06:58:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 170 k 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.4 M 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 20 M 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 34 k 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 2.3 M 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 290 k 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 4.9 M 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 18 M 2024-08-31T06:58:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 16 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 25 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 159 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-31T06:58:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 506 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.3 M 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-31T06:58:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 45 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 141 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-31T06:58:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-31T06:58:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-31T06:58:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-31T06:58:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-31T06:58:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-31T06:58:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-31T06:58:00.538 INFO:teuthology.orchestra.run.smithi032.stdout:Installing weak dependencies: 2024-08-31T06:58:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-31T06:58:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:58:00.538 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-08-31T06:58:00.538 INFO:teuthology.orchestra.run.smithi032.stdout:======================================================================================= 2024-08-31T06:58:00.538 INFO:teuthology.orchestra.run.smithi032.stdout:Install 148 Packages 2024-08-31T06:58:00.538 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 2 Packages 2024-08-31T06:58:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:58:00.539 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 224 M 2024-08-31T06:58:00.539 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-08-31T06:58:00.813 INFO:teuthology.orchestra.run.smithi084.stdout:(54/150): flexiblas-3.0.4-8.el9.x86_64.rpm 9.4 kB/s | 33 kB 00:03 2024-08-31T06:58:00.837 INFO:teuthology.orchestra.run.smithi084.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 14 kB/s | 18 kB 00:01 2024-08-31T06:58:00.971 INFO:teuthology.orchestra.run.smithi084.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 70 kB/s | 11 kB 00:00 2024-08-31T06:58:01.038 INFO:teuthology.orchestra.run.smithi084.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 799 kB/s | 160 kB 00:00 2024-08-31T06:58:01.055 INFO:teuthology.orchestra.run.smithi084.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-08-31T06:58:01.100 INFO:teuthology.orchestra.run.smithi073.stdout:(40/150): libquadmath-11.5.0-2.el9.x86_64.rpm 26 kB/s | 191 kB 00:07 2024-08-31T06:58:01.183 INFO:teuthology.orchestra.run.smithi073.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-08-31T06:58:01.205 INFO:teuthology.orchestra.run.smithi084.stdout:(59/150): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-08-31T06:58:01.297 INFO:teuthology.orchestra.run.smithi084.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-08-31T06:58:01.304 INFO:teuthology.orchestra.run.smithi032.stdout:(1/150): ceph-19.3.0-4560.gd19e6118.el9.x86_64. 49 kB/s | 6.6 kB 00:00 2024-08-31T06:58:01.308 INFO:teuthology.orchestra.run.smithi073.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-08-31T06:58:01.367 INFO:teuthology.orchestra.run.smithi073.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-08-31T06:58:01.426 INFO:teuthology.orchestra.run.smithi073.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-08-31T06:58:01.484 INFO:teuthology.orchestra.run.smithi073.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-08-31T06:58:01.487 INFO:teuthology.orchestra.run.smithi032.stdout:(2/150): ceph-fuse-19.3.0-4560.gd19e6118.el9.x8 4.7 MB/s | 874 kB 00:00 2024-08-31T06:58:01.563 INFO:teuthology.orchestra.run.smithi032.stdout:(3/150): ceph-immutable-object-cache-19.3.0-456 1.9 MB/s | 148 kB 00:00 2024-08-31T06:58:01.635 INFO:teuthology.orchestra.run.smithi073.stdout:(46/150): python3-urllib3-1.26.5-6.el9.noarch.r 1.4 MB/s | 215 kB 00:00 2024-08-31T06:58:01.655 INFO:teuthology.orchestra.run.smithi032.stdout:(4/150): ceph-base-19.3.0-4560.gd19e6118.el9.x8 11 MB/s | 5.5 MB 00:00 2024-08-31T06:58:01.738 INFO:teuthology.orchestra.run.smithi032.stdout:(5/150): ceph-mds-19.3.0-4560.gd19e6118.el9.x86 13 MB/s | 2.3 MB 00:00 2024-08-31T06:58:01.797 INFO:teuthology.orchestra.run.smithi032.stdout:(6/150): ceph-mgr-19.3.0-4560.gd19e6118.el9.x86 11 MB/s | 1.5 MB 00:00 2024-08-31T06:58:01.801 INFO:teuthology.orchestra.run.smithi073.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-08-31T06:58:01.868 INFO:teuthology.orchestra.run.smithi073.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-08-31T06:58:01.910 INFO:teuthology.orchestra.run.smithi073.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-08-31T06:58:01.952 INFO:teuthology.orchestra.run.smithi073.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 521 kB/s | 22 kB 00:00 2024-08-31T06:58:01.994 INFO:teuthology.orchestra.run.smithi073.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-08-31T06:58:02.045 INFO:teuthology.orchestra.run.smithi073.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-08-31T06:58:02.128 INFO:teuthology.orchestra.run.smithi073.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 36 MB/s | 3.0 MB 00:00 2024-08-31T06:58:02.170 INFO:teuthology.orchestra.run.smithi073.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-08-31T06:58:02.212 INFO:teuthology.orchestra.run.smithi073.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-08-31T06:58:02.222 INFO:teuthology.orchestra.run.smithi032.stdout:(7/150): ceph-mon-19.3.0-4560.gd19e6118.el9.x86 10 MB/s | 4.9 MB 00:00 2024-08-31T06:58:02.231 INFO:teuthology.orchestra.run.smithi084.stdout:(61/150): libuuid-devel-2.37.4-20.el9.x86_64.rp 24 kB/s | 24 kB 00:01 2024-08-31T06:58:02.248 INFO:teuthology.orchestra.run.smithi084.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 50 kB/s | 47 kB 00:00 2024-08-31T06:58:02.254 INFO:teuthology.orchestra.run.smithi073.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-08-31T06:58:02.296 INFO:teuthology.orchestra.run.smithi073.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-08-31T06:58:02.338 INFO:teuthology.orchestra.run.smithi073.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 662 kB 00:00 2024-08-31T06:58:02.343 INFO:teuthology.orchestra.run.smithi084.stdout:(63/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 613 kB/s | 3.0 MB 00:04 2024-08-31T06:58:02.380 INFO:teuthology.orchestra.run.smithi073.stdout:(59/150): libtool-2.4.6-46.el9.x86_64.rpm 14 MB/s | 581 kB 00:00 2024-08-31T06:58:02.422 INFO:teuthology.orchestra.run.smithi073.stdout:(60/150): libuuid-devel-2.37.4-20.el9.x86_64.rp 587 kB/s | 24 kB 00:00 2024-08-31T06:58:02.423 INFO:teuthology.orchestra.run.smithi084.stdout:(64/150): openblas-0.3.26-2.el9.x86_64.rpm 483 kB/s | 38 kB 00:00 2024-08-31T06:58:02.431 INFO:teuthology.orchestra.run.smithi032.stdout:(8/150): ceph-common-19.3.0-4560.gd19e6118.el9. 16 MB/s | 20 MB 00:01 2024-08-31T06:58:02.473 INFO:teuthology.orchestra.run.smithi032.stdout:(9/150): ceph-selinux-19.3.0-4560.gd19e6118.el9 609 kB/s | 25 kB 00:00 2024-08-31T06:58:02.482 INFO:teuthology.orchestra.run.smithi084.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 806 kB/s | 188 kB 00:00 2024-08-31T06:58:02.490 INFO:teuthology.orchestra.run.smithi073.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 701 kB/s | 47 kB 00:00 2024-08-31T06:58:02.515 INFO:teuthology.orchestra.run.smithi084.stdout:(66/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 292 kB 00:00 2024-08-31T06:58:02.539 INFO:teuthology.orchestra.run.smithi073.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.8 MB/s | 292 kB 00:00 2024-08-31T06:58:02.581 INFO:teuthology.orchestra.run.smithi073.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-08-31T06:58:02.623 INFO:teuthology.orchestra.run.smithi073.stdout:(64/150): openblas-0.3.26-2.el9.x86_64.rpm 923 kB/s | 38 kB 00:00 2024-08-31T06:58:02.724 INFO:teuthology.orchestra.run.smithi073.stdout:(65/150): openblas-openmp-0.3.26-2.el9.x86_64.r 49 MB/s | 4.9 MB 00:00 2024-08-31T06:58:02.798 INFO:teuthology.orchestra.run.smithi032.stdout:(10/150): ceph-osd-19.3.0-4560.gd19e6118.el9.x8 18 MB/s | 18 MB 00:01 2024-08-31T06:58:02.815 INFO:teuthology.orchestra.run.smithi073.stdout:(66/150): openssl-devel-3.2.2-4.el9.x86_64.rpm 48 MB/s | 4.4 MB 00:00 2024-08-31T06:58:02.857 INFO:teuthology.orchestra.run.smithi032.stdout:(11/150): cephfs-mirror-19.3.0-4560.gd19e6118.e 3.8 MB/s | 226 kB 00:00 2024-08-31T06:58:02.874 INFO:teuthology.orchestra.run.smithi073.stdout:(67/150): postgresql-13.16-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-08-31T06:58:02.924 INFO:teuthology.orchestra.run.smithi073.stdout:(68/150): postgresql-contrib-13.16-1.el9.x86_64 18 MB/s | 910 kB 00:00 2024-08-31T06:58:02.966 INFO:teuthology.orchestra.run.smithi073.stdout:(69/150): postgresql-private-libs-13.16-1.el9.x 3.2 MB/s | 137 kB 00:00 2024-08-31T06:58:02.998 INFO:teuthology.orchestra.run.smithi032.stdout:(12/150): ceph-radosgw-19.3.0-4560.gd19e6118.el 13 MB/s | 10 MB 00:00 2024-08-31T06:58:03.023 INFO:teuthology.orchestra.run.smithi032.stdout:(13/150): cephfs-shell-19.3.0-4560.gd19e6118.el 162 kB/s | 27 kB 00:00 2024-08-31T06:58:03.049 INFO:teuthology.orchestra.run.smithi032.stdout:(14/150): libcephfs-devel-19.3.0-4560.gd19e6118 634 kB/s | 32 kB 00:00 2024-08-31T06:58:03.083 INFO:teuthology.orchestra.run.smithi073.stdout:(70/150): postgresql-server-13.16-1.el9.x86_64. 50 MB/s | 5.8 MB 00:00 2024-08-31T06:58:03.091 INFO:teuthology.orchestra.run.smithi032.stdout:(15/150): libcephfs2-19.3.0-4560.gd19e6118.el9. 11 MB/s | 766 kB 00:00 2024-08-31T06:58:03.116 INFO:teuthology.orchestra.run.smithi032.stdout:(16/150): libcephsqlite-19.3.0-4560.gd19e6118.e 2.3 MB/s | 159 kB 00:00 2024-08-31T06:58:03.133 INFO:teuthology.orchestra.run.smithi073.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-08-31T06:58:03.142 INFO:teuthology.orchestra.run.smithi032.stdout:(17/150): librados-devel-19.3.0-4560.gd19e6118. 2.5 MB/s | 127 kB 00:00 2024-08-31T06:58:03.175 INFO:teuthology.orchestra.run.smithi073.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 2.0 MB/s | 87 kB 00:00 2024-08-31T06:58:03.175 INFO:teuthology.orchestra.run.smithi032.stdout:(18/150): libradosstriper1-19.3.0-4560.gd19e611 8.4 MB/s | 506 kB 00:00 2024-08-31T06:58:03.234 INFO:teuthology.orchestra.run.smithi032.stdout:(19/150): python3-ceph-argparse-19.3.0-4560.gd1 772 kB/s | 45 kB 00:00 2024-08-31T06:58:03.301 INFO:teuthology.orchestra.run.smithi073.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 48 MB/s | 6.0 MB 00:00 2024-08-31T06:58:03.342 INFO:teuthology.orchestra.run.smithi073.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-08-31T06:58:03.384 INFO:teuthology.orchestra.run.smithi073.stdout:(75/150): python3-devel-3.9.19-8.el9.x86_64.rpm 5.7 MB/s | 245 kB 00:00 2024-08-31T06:58:03.401 INFO:teuthology.orchestra.run.smithi032.stdout:(20/150): python3-ceph-common-19.3.0-4560.gd19e 847 kB/s | 141 kB 00:00 2024-08-31T06:58:03.426 INFO:teuthology.orchestra.run.smithi073.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-08-31T06:58:03.468 INFO:teuthology.orchestra.run.smithi073.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-08-31T06:58:03.476 INFO:teuthology.orchestra.run.smithi032.stdout:(21/150): librgw2-19.3.0-4560.gd19e6118.el9.x86 16 MB/s | 5.3 MB 00:00 2024-08-31T06:58:03.501 INFO:teuthology.orchestra.run.smithi032.stdout:(22/150): python3-cephfs-19.3.0-4560.gd19e6118. 1.6 MB/s | 161 kB 00:00 2024-08-31T06:58:03.510 INFO:teuthology.orchestra.run.smithi073.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-08-31T06:58:03.527 INFO:teuthology.orchestra.run.smithi032.stdout:(23/150): python3-rados-19.3.0-4560.gd19e6118.e 6.3 MB/s | 321 kB 00:00 2024-08-31T06:58:03.552 INFO:teuthology.orchestra.run.smithi032.stdout:(24/150): python3-rbd-19.3.0-4560.gd19e6118.el9 5.9 MB/s | 305 kB 00:00 2024-08-31T06:58:03.552 INFO:teuthology.orchestra.run.smithi073.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-08-31T06:58:03.586 INFO:teuthology.orchestra.run.smithi032.stdout:(25/150): python3-rgw-19.3.0-4560.gd19e6118.el9 1.7 MB/s | 99 kB 00:00 2024-08-31T06:58:03.594 INFO:teuthology.orchestra.run.smithi073.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 836 kB/s | 35 kB 00:00 2024-08-31T06:58:03.611 INFO:teuthology.orchestra.run.smithi032.stdout:(26/150): rbd-fuse-19.3.0-4560.gd19e6118.el9.x8 1.4 MB/s | 86 kB 00:00 2024-08-31T06:58:03.661 INFO:teuthology.orchestra.run.smithi032.stdout:(27/150): rbd-nbd-19.3.0-4560.gd19e6118.el9.x86 3.3 MB/s | 170 kB 00:00 2024-08-31T06:58:03.703 INFO:teuthology.orchestra.run.smithi073.stdout:(81/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 47 MB/s | 5.1 MB 00:00 2024-08-31T06:58:03.737 INFO:teuthology.orchestra.run.smithi032.stdout:(28/150): rbd-mirror-19.3.0-4560.gd19e6118.el9. 21 MB/s | 3.1 MB 00:00 2024-08-31T06:58:03.745 INFO:teuthology.orchestra.run.smithi073.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-08-31T06:58:03.761 INFO:teuthology.orchestra.run.smithi032.stdout:(29/150): ceph-grafana-dashboards-19.3.0-4560.g 343 kB/s | 34 kB 00:00 2024-08-31T06:58:03.787 INFO:teuthology.orchestra.run.smithi032.stdout:(30/150): ceph-mgr-cephadm-19.3.0-4560.gd19e611 3.0 MB/s | 154 kB 00:00 2024-08-31T06:58:03.787 INFO:teuthology.orchestra.run.smithi073.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-08-31T06:58:03.829 INFO:teuthology.orchestra.run.smithi073.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-08-31T06:58:03.871 INFO:teuthology.orchestra.run.smithi073.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-08-31T06:58:03.913 INFO:teuthology.orchestra.run.smithi073.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-08-31T06:58:03.955 INFO:teuthology.orchestra.run.smithi073.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-08-31T06:58:03.996 INFO:teuthology.orchestra.run.smithi073.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-08-31T06:58:04.038 INFO:teuthology.orchestra.run.smithi073.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-08-31T06:58:04.080 INFO:teuthology.orchestra.run.smithi073.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-08-31T06:58:04.187 INFO:teuthology.orchestra.run.smithi032.stdout:(31/150): ceph-mgr-diskprediction-local-19.3.0- 18 MB/s | 7.4 MB 00:00 2024-08-31T06:58:04.246 INFO:teuthology.orchestra.run.smithi032.stdout:(32/150): ceph-mgr-modules-core-19.3.0-4560.gd1 4.9 MB/s | 290 kB 00:00 2024-08-31T06:58:04.305 INFO:teuthology.orchestra.run.smithi032.stdout:(33/150): ceph-mgr-rook-19.3.0-4560.gd19e6118.e 845 kB/s | 49 kB 00:00 2024-08-31T06:58:04.356 INFO:teuthology.orchestra.run.smithi073.stdout:(91/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 59 MB/s | 16 MB 00:00 2024-08-31T06:58:04.363 INFO:teuthology.orchestra.run.smithi032.stdout:(34/150): ceph-prometheus-alerts-19.3.0-4560.gd 284 kB/s | 16 kB 00:00 2024-08-31T06:58:04.398 INFO:teuthology.orchestra.run.smithi073.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-08-31T06:58:04.430 INFO:teuthology.orchestra.run.smithi032.stdout:(35/150): ceph-volume-19.3.0-4560.gd19e6118.el9 4.2 MB/s | 285 kB 00:00 2024-08-31T06:58:04.439 INFO:teuthology.orchestra.run.smithi073.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-08-31T06:58:04.481 INFO:teuthology.orchestra.run.smithi073.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-08-31T06:58:04.523 INFO:teuthology.orchestra.run.smithi073.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 862 kB/s | 36 kB 00:00 2024-08-31T06:58:04.574 INFO:teuthology.orchestra.run.smithi073.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-08-31T06:58:04.614 INFO:teuthology.orchestra.run.smithi032.stdout:(36/150): ceph-mgr-dashboard-19.3.0-4560.gd19e6 17 MB/s | 14 MB 00:00 2024-08-31T06:58:04.616 INFO:teuthology.orchestra.run.smithi073.stdout:(97/150): socat-1.7.4.1-6.el9.x86_64.rpm 7.1 MB/s | 304 kB 00:00 2024-08-31T06:58:04.656 INFO:teuthology.orchestra.run.smithi032.stdout:(37/150): cephadm-19.3.0-4560.gd19e6118.el9.noa 3.4 MB/s | 781 kB 00:00 2024-08-31T06:58:04.658 INFO:teuthology.orchestra.run.smithi073.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-08-31T06:58:04.681 INFO:teuthology.orchestra.run.smithi032.stdout:(38/150): cephfs-top-19.3.0-4560.gd19e6118.el9. 273 kB/s | 18 kB 00:00 2024-08-31T06:58:04.700 INFO:teuthology.orchestra.run.smithi073.stdout:(99/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 2.2 MB/s | 96 kB 00:00 2024-08-31T06:58:04.742 INFO:teuthology.orchestra.run.smithi073.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-08-31T06:58:04.867 INFO:teuthology.orchestra.run.smithi073.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 358 kB/s | 45 kB 00:00 2024-08-31T06:58:05.365 INFO:teuthology.orchestra.run.smithi032.stdout:(39/150): ceph-test-19.3.0-4560.gd19e6118.el9.x 15 MB/s | 44 MB 00:02 2024-08-31T06:58:05.390 INFO:teuthology.orchestra.run.smithi032.stdout:(40/150): fuse-2.9.9-16.el9.x86_64.rpm 109 kB/s | 80 kB 00:00 2024-08-31T06:58:05.400 INFO:teuthology.orchestra.run.smithi073.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 42 kB/s | 22 kB 00:00 2024-08-31T06:58:05.765 INFO:teuthology.orchestra.run.smithi032.stdout:(41/150): libquadmath-11.5.0-2.el9.x86_64.rpm 478 kB/s | 191 kB 00:00 2024-08-31T06:58:05.790 INFO:teuthology.orchestra.run.smithi032.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 633 kB/s | 253 kB 00:00 2024-08-31T06:58:05.916 INFO:teuthology.orchestra.run.smithi032.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 853 kB/s | 106 kB 00:00 2024-08-31T06:58:05.958 INFO:teuthology.orchestra.run.smithi032.stdout:(44/150): libgfortran-11.5.0-2.el9.x86_64.rpm 628 kB/s | 801 kB 00:01 2024-08-31T06:58:06.041 INFO:teuthology.orchestra.run.smithi032.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-08-31T06:58:06.083 INFO:teuthology.orchestra.run.smithi032.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 994 kB/s | 125 kB 00:00 2024-08-31T06:58:06.209 INFO:teuthology.orchestra.run.smithi032.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-08-31T06:58:06.284 INFO:teuthology.orchestra.run.smithi073.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 978 kB/s | 863 kB 00:00 2024-08-31T06:58:06.334 INFO:teuthology.orchestra.run.smithi073.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-31T06:58:06.359 INFO:teuthology.orchestra.run.smithi073.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-08-31T06:58:06.382 INFO:teuthology.orchestra.run.smithi073.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 887 kB/s | 19 kB 00:00 2024-08-31T06:58:06.417 INFO:teuthology.orchestra.run.smithi032.stdout:(48/150): python3-cryptography-36.0.1-4.el9.x86 1.9 MB/s | 1.2 MB 00:00 2024-08-31T06:58:06.482 INFO:teuthology.orchestra.run.smithi073.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-08-31T06:58:06.507 INFO:teuthology.orchestra.run.smithi073.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-31T06:58:06.533 INFO:teuthology.orchestra.run.smithi073.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-31T06:58:06.558 INFO:teuthology.orchestra.run.smithi073.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-31T06:58:06.583 INFO:teuthology.orchestra.run.smithi073.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-08-31T06:58:06.625 INFO:teuthology.orchestra.run.smithi073.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-08-31T06:58:06.663 INFO:teuthology.orchestra.run.smithi073.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 14 MB/s | 548 kB 00:00 2024-08-31T06:58:06.688 INFO:teuthology.orchestra.run.smithi073.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-31T06:58:06.713 INFO:teuthology.orchestra.run.smithi073.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-31T06:58:06.755 INFO:teuthology.orchestra.run.smithi073.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 339 kB/s | 14 kB 00:00 2024-08-31T06:58:06.781 INFO:teuthology.orchestra.run.smithi073.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-08-31T06:58:06.814 INFO:teuthology.orchestra.run.smithi073.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-08-31T06:58:06.840 INFO:teuthology.orchestra.run.smithi073.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-08-31T06:58:06.865 INFO:teuthology.orchestra.run.smithi073.stdout:(120/150): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-08-31T06:58:06.932 INFO:teuthology.orchestra.run.smithi073.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 31 MB/s | 2.0 MB 00:00 2024-08-31T06:58:06.943 INFO:teuthology.orchestra.run.smithi032.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 802 kB/s | 689 kB 00:00 2024-08-31T06:58:06.957 INFO:teuthology.orchestra.run.smithi073.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-08-31T06:58:06.982 INFO:teuthology.orchestra.run.smithi073.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-08-31T06:58:07.008 INFO:teuthology.orchestra.run.smithi073.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 704 kB/s | 18 kB 00:00 2024-08-31T06:58:07.033 INFO:teuthology.orchestra.run.smithi073.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-08-31T06:58:07.058 INFO:teuthology.orchestra.run.smithi073.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-08-31T06:58:07.084 INFO:teuthology.orchestra.run.smithi073.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-08-31T06:58:07.084 INFO:teuthology.orchestra.run.smithi032.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 160 kB/s | 107 kB 00:00 2024-08-31T06:58:07.125 INFO:teuthology.orchestra.run.smithi073.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 25 MB/s | 1.0 MB 00:00 2024-08-31T06:58:07.135 INFO:teuthology.orchestra.run.smithi032.stdout:(51/150): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-08-31T06:58:07.151 INFO:teuthology.orchestra.run.smithi073.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-31T06:58:07.168 INFO:teuthology.orchestra.run.smithi073.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 4.6 MB/s | 79 kB 00:00 2024-08-31T06:58:07.193 INFO:teuthology.orchestra.run.smithi073.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-31T06:58:07.218 INFO:teuthology.orchestra.run.smithi073.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-08-31T06:58:07.243 INFO:teuthology.orchestra.run.smithi073.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 655 kB/s | 16 kB 00:00 2024-08-31T06:58:07.269 INFO:teuthology.orchestra.run.smithi073.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-08-31T06:58:07.294 INFO:teuthology.orchestra.run.smithi073.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-08-31T06:58:07.319 INFO:teuthology.orchestra.run.smithi073.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-08-31T06:58:07.345 INFO:teuthology.orchestra.run.smithi073.stdout:(137/150): python3-routes-2.5.1-5.el9.noarch.rp 7.4 MB/s | 188 kB 00:00 2024-08-31T06:58:07.370 INFO:teuthology.orchestra.run.smithi073.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-31T06:58:07.395 INFO:teuthology.orchestra.run.smithi073.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-08-31T06:58:07.420 INFO:teuthology.orchestra.run.smithi073.stdout:(140/150): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-08-31T06:58:07.446 INFO:teuthology.orchestra.run.smithi073.stdout:(141/150): python3-webob-1.8.8-2.el9.noarch.rpm 9.0 MB/s | 230 kB 00:00 2024-08-31T06:58:07.465 INFO:teuthology.orchestra.run.smithi084.stdout:(67/150): postgresql-13.16-1.el9.x86_64.rpm 328 kB/s | 1.6 MB 00:04 2024-08-31T06:58:07.471 INFO:teuthology.orchestra.run.smithi073.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-31T06:58:07.493 INFO:teuthology.orchestra.run.smithi032.stdout:(52/150): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-08-31T06:58:07.505 INFO:teuthology.orchestra.run.smithi073.stdout:(143/150): python3-werkzeug-2.0.3-3.el9.1.noarc 13 MB/s | 427 kB 00:00 2024-08-31T06:58:07.530 INFO:teuthology.orchestra.run.smithi073.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-08-31T06:58:07.555 INFO:teuthology.orchestra.run.smithi073.stdout:(145/150): re2-20211101-20.el9.x86_64.rpm 7.6 MB/s | 191 kB 00:00 2024-08-31T06:58:07.605 INFO:teuthology.orchestra.run.smithi073.stdout:(146/150): thrift-0.15.0-2.el9.x86_64.rpm 32 MB/s | 1.6 MB 00:00 2024-08-31T06:58:07.694 INFO:teuthology.orchestra.run.smithi032.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-08-31T06:58:07.773 INFO:teuthology.orchestra.run.smithi084.stdout:(68/150): postgresql-contrib-13.16-1.el9.x86_64 2.9 MB/s | 910 kB 00:00 2024-08-31T06:58:07.847 INFO:teuthology.orchestra.run.smithi073.stdout:(147/150): librados2-19.3.0-4560.gd19e6118.el9. 14 MB/s | 3.4 MB 00:00 2024-08-31T06:58:07.860 INFO:teuthology.orchestra.run.smithi032.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 404 kB/s | 313 kB 00:00 2024-08-31T06:58:07.965 INFO:teuthology.orchestra.run.smithi073.stdout:(148/150): librbd1-19.3.0-4560.gd19e6118.el9.x8 27 MB/s | 3.1 MB 00:00 2024-08-31T06:58:08.177 INFO:teuthology.orchestra.run.smithi032.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-08-31T06:58:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-08-31T06:58:08.528 INFO:teuthology.orchestra.run.smithi032.stdout:(57/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-08-31T06:58:08.936 INFO:teuthology.orchestra.run.smithi032.stdout:(58/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 211 kB/s | 160 kB 00:00 2024-08-31T06:58:09.012 INFO:teuthology.orchestra.run.smithi032.stdout:(59/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-08-31T06:58:09.421 INFO:teuthology.orchestra.run.smithi032.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 744 kB/s | 662 kB 00:00 2024-08-31T06:58:09.554 INFO:teuthology.orchestra.run.smithi032.stdout:(61/150): libuuid-devel-2.37.4-20.el9.x86_64.rp 45 kB/s | 24 kB 00:00 2024-08-31T06:58:09.846 INFO:teuthology.orchestra.run.smithi032.stdout:(62/150): libtool-2.4.6-46.el9.x86_64.rpm 639 kB/s | 581 kB 00:00 2024-08-31T06:58:10.013 INFO:teuthology.orchestra.run.smithi032.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 80 kB/s | 47 kB 00:00 2024-08-31T06:58:10.023 INFO:teuthology.orchestra.run.smithi084.stdout:(69/150): openssl-devel-3.2.2-4.el9.x86_64.rpm 602 kB/s | 4.4 MB 00:07 2024-08-31T06:58:10.396 INFO:teuthology.orchestra.run.smithi032.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 347 kB/s | 292 kB 00:00 2024-08-31T06:58:10.597 INFO:teuthology.orchestra.run.smithi032.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 251 kB/s | 188 kB 00:00 2024-08-31T06:58:10.622 INFO:teuthology.orchestra.run.smithi032.stdout:(66/150): openblas-0.3.26-2.el9.x86_64.rpm 63 kB/s | 38 kB 00:00 2024-08-31T06:58:10.748 INFO:teuthology.orchestra.run.smithi084.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 46 kB/s | 137 kB 00:02 2024-08-31T06:58:10.907 INFO:teuthology.orchestra.run.smithi084.stdout:(71/150): openblas-openmp-0.3.26-2.el9.x86_64.r 596 kB/s | 4.9 MB 00:08 2024-08-31T06:58:12.114 INFO:teuthology.orchestra.run.smithi032.stdout:(67/150): openblas-openmp-0.3.26-2.el9.x86_64.r 2.9 MB/s | 4.9 MB 00:01 2024-08-31T06:58:12.239 INFO:teuthology.orchestra.run.smithi032.stdout:(68/150): openssl-devel-3.2.2-4.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-08-31T06:58:12.531 INFO:teuthology.orchestra.run.smithi032.stdout:(69/150): postgresql-13.16-1.el9.x86_64.rpm 856 kB/s | 1.6 MB 00:01 2024-08-31T06:58:13.014 INFO:teuthology.orchestra.run.smithi032.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 177 kB/s | 137 kB 00:00 2024-08-31T06:58:13.423 INFO:teuthology.orchestra.run.smithi084.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 35 kB/s | 87 kB 00:02 2024-08-31T06:58:13.456 INFO:teuthology.orchestra.run.smithi032.stdout:(71/150): postgresql-contrib-13.16-1.el9.x86_64 678 kB/s | 910 kB 00:01 2024-08-31T06:58:13.540 INFO:teuthology.orchestra.run.smithi084.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 370 kB/s | 1.0 MB 00:02 2024-08-31T06:58:13.822 INFO:teuthology.orchestra.run.smithi032.stdout:(72/150): postgresql-server-13.16-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-08-31T06:58:13.905 INFO:teuthology.orchestra.run.smithi073.stdout:(149/150): fuse-2.9.9-16.el9.x86_64.rpm 3.9 kB/s | 80 kB 00:20 2024-08-31T06:58:13.965 INFO:teuthology.orchestra.run.smithi032.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-08-31T06:58:14.064 INFO:teuthology.orchestra.run.smithi073.stdout:(150/150): libgfortran-11.5.0-2.el9.x86_64.rpm 39 kB/s | 801 kB 00:20 2024-08-31T06:58:14.068 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-08-31T06:58:14.069 INFO:teuthology.orchestra.run.smithi073.stdout:Total 8.4 MB/s | 224 MB 00:26 2024-08-31T06:58:14.168 INFO:teuthology.orchestra.run.smithi032.stdout:(74/150): python3-attrs-20.3.0-7.el9.noarch.rpm 123 kB/s | 87 kB 00:00 2024-08-31T06:58:14.601 INFO:teuthology.orchestra.run.smithi032.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 96 kB/s | 61 kB 00:00 2024-08-31T06:58:15.024 INFO:teuthology.orchestra.run.smithi032.stdout:(76/150): python3-devel-3.9.19-8.el9.x86_64.rpm 285 kB/s | 245 kB 00:00 2024-08-31T06:58:15.177 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-08-31T06:58:15.301 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-08-31T06:58:15.302 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-08-31T06:58:15.400 INFO:teuthology.orchestra.run.smithi032.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 311 kB/s | 248 kB 00:00 2024-08-31T06:58:15.515 INFO:teuthology.orchestra.run.smithi084.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 31 kB/s | 61 kB 00:01 2024-08-31T06:58:15.725 INFO:teuthology.orchestra.run.smithi032.stdout:(78/150): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-08-31T06:58:15.783 INFO:teuthology.orchestra.run.smithi032.stdout:(79/150): python3-jsonpatch-1.21-16.el9.noarch. 34 kB/s | 26 kB 00:00 2024-08-31T06:58:15.925 INFO:teuthology.orchestra.run.smithi032.stdout:(80/150): python3-jsonpointer-2.0-4.el9.noarch. 37 kB/s | 19 kB 00:00 2024-08-31T06:58:16.467 INFO:teuthology.orchestra.run.smithi032.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 51 kB/s | 35 kB 00:00 2024-08-31T06:58:16.492 INFO:teuthology.orchestra.run.smithi032.stdout:(82/150): python3-mako-1.1.4-6.el9.noarch.rpm 225 kB/s | 172 kB 00:00 2024-08-31T06:58:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-08-31T06:58:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-08-31T06:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout:(83/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-08-31T06:58:17.330 INFO:teuthology.orchestra.run.smithi032.stdout:(84/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 291 kB/s | 250 kB 00:00 2024-08-31T06:58:17.419 INFO:teuthology.orchestra.run.smithi032.stdout:(85/150): python3-oauthlib-3.1.1-5.el9.noarch.r 256 kB/s | 222 kB 00:00 2024-08-31T06:58:17.423 INFO:teuthology.orchestra.run.smithi084.stdout:(75/150): python3-devel-3.9.19-8.el9.x86_64.rpm 128 kB/s | 245 kB 00:01 2024-08-31T06:58:17.852 INFO:teuthology.orchestra.run.smithi032.stdout:(86/150): python3-packaging-20.9-5.el9.noarch.r 114 kB/s | 77 kB 00:00 2024-08-31T06:58:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:(87/150): python3-prettytable-0.7.2-27.el9.noar 69 kB/s | 42 kB 00:00 2024-08-31T06:58:18.127 INFO:teuthology.orchestra.run.smithi032.stdout:(88/150): python3-protobuf-3.14.0-13.el9.noarch 351 kB/s | 269 kB 00:00 2024-08-31T06:58:18.553 INFO:teuthology.orchestra.run.smithi032.stdout:(89/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 227 kB/s | 159 kB 00:00 2024-08-31T06:58:18.719 INFO:teuthology.orchestra.run.smithi032.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 86 kB/s | 51 kB 00:00 2024-08-31T06:58:18.753 INFO:teuthology.orchestra.run.smithi032.stdout:(91/150): python3-pyasn1-modules-0.4.8-6.el9.no 356 kB/s | 279 kB 00:00 2024-08-31T06:58:19.316 INFO:teuthology.orchestra.run.smithi032.stdout:(92/150): python3-requests-oauthlib-1.3.0-12.el 70 kB/s | 54 kB 00:00 2024-08-31T06:58:19.440 INFO:teuthology.orchestra.run.smithi084.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 123 kB/s | 248 kB 00:02 2024-08-31T06:58:19.545 INFO:teuthology.orchestra.run.smithi032.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 53 kB/s | 42 kB 00:00 2024-08-31T06:58:19.590 INFO:teuthology.orchestra.run.smithi084.stdout:(77/150): postgresql-server-13.16-1.el9.x86_64. 618 kB/s | 5.8 MB 00:09 2024-08-31T06:58:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 63 kB/s | 44 kB 00:00 2024-08-31T06:58:20.387 INFO:teuthology.orchestra.run.smithi032.stdout:(95/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.7 MB/s | 16 MB 00:01 2024-08-31T06:58:20.579 INFO:teuthology.orchestra.run.smithi032.stdout:(96/150): qatlib-24.02.0-1.el9.x86_64.rpm 216 kB/s | 221 kB 00:01 2024-08-31T06:58:20.813 INFO:teuthology.orchestra.run.smithi032.stdout:(97/150): qatlib-service-24.02.0-1.el9.x86_64.r 45 kB/s | 36 kB 00:00 2024-08-31T06:58:21.021 INFO:teuthology.orchestra.run.smithi032.stdout:(98/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-08-31T06:58:21.439 INFO:teuthology.orchestra.run.smithi032.stdout:(99/150): socat-1.7.4.1-6.el9.x86_64.rpm 354 kB/s | 304 kB 00:00 2024-08-31T06:58:21.473 INFO:teuthology.orchestra.run.smithi084.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 13 kB/s | 26 kB 00:02 2024-08-31T06:58:21.488 INFO:teuthology.orchestra.run.smithi032.stdout:(100/150): uuid-1.6.2-55.el9.x86_64.rpm 87 kB/s | 58 kB 00:00 2024-08-31T06:58:21.498 INFO:teuthology.orchestra.run.smithi084.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 10 kB/s | 19 kB 00:01 2024-08-31T06:58:21.689 INFO:teuthology.orchestra.run.smithi032.stdout:(101/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 144 kB/s | 96 kB 00:00 2024-08-31T06:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout:(102/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2024-08-31T06:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout:(103/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 76 kB/s | 45 kB 00:00 2024-08-31T06:58:22.140 INFO:teuthology.orchestra.run.smithi032.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 9.1 MB/s | 548 kB 00:00 2024-08-31T06:58:22.290 INFO:teuthology.orchestra.run.smithi032.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-08-31T06:58:22.307 INFO:teuthology.orchestra.run.smithi032.stdout:(106/150): lua-devel-5.4.4-4.el9.x86_64.rpm 36 kB/s | 22 kB 00:00 2024-08-31T06:58:22.324 INFO:teuthology.orchestra.run.smithi032.stdout:(107/150): grpc-data-1.46.7-10.el9.noarch.rpm 582 kB/s | 19 kB 00:00 2024-08-31T06:58:22.349 INFO:teuthology.orchestra.run.smithi032.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.9 MB/s | 26 kB 00:00 2024-08-31T06:58:22.404 INFO:teuthology.orchestra.run.smithi032.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 919 kB/s | 49 kB 00:00 2024-08-31T06:58:22.450 INFO:teuthology.orchestra.run.smithi032.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.9 MB/s | 67 kB 00:00 2024-08-31T06:58:22.521 INFO:teuthology.orchestra.run.smithi032.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 2.1 MB/s | 151 kB 00:00 2024-08-31T06:58:22.575 INFO:teuthology.orchestra.run.smithi032.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 15 MB/s | 839 kB 00:00 2024-08-31T06:58:22.617 INFO:teuthology.orchestra.run.smithi032.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-08-31T06:58:22.643 INFO:teuthology.orchestra.run.smithi032.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-31T06:58:22.668 INFO:teuthology.orchestra.run.smithi032.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-31T06:58:22.693 INFO:teuthology.orchestra.run.smithi032.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 570 kB/s | 14 kB 00:00 2024-08-31T06:58:22.727 INFO:teuthology.orchestra.run.smithi032.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-08-31T06:58:22.790 INFO:teuthology.orchestra.run.smithi032.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 5.6 MB/s | 358 kB 00:00 2024-08-31T06:58:22.827 INFO:teuthology.orchestra.run.smithi032.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.1 MB/s | 303 kB 00:00 2024-08-31T06:58:22.861 INFO:teuthology.orchestra.run.smithi032.stdout:(120/150): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-08-31T06:58:22.917 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-08-31T06:58:22.953 INFO:teuthology.orchestra.run.smithi032.stdout:(121/150): libarrow-9.0.0-11.el9.x86_64.rpm 6.8 MB/s | 4.4 MB 00:00 2024-08-31T06:58:22.978 INFO:teuthology.orchestra.run.smithi032.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-08-31T06:58:22.992 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-31T06:58:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 435 kB/s | 11 kB 00:00 2024-08-31T06:58:23.026 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-31T06:58:23.029 INFO:teuthology.orchestra.run.smithi032.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-08-31T06:58:23.032 INFO:teuthology.orchestra.run.smithi084.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 23 kB/s | 35 kB 00:01 2024-08-31T06:58:23.054 INFO:teuthology.orchestra.run.smithi032.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 937 kB/s | 23 kB 00:00 2024-08-31T06:58:23.057 INFO:teuthology.orchestra.run.smithi084.stdout:(81/150): python3-mako-1.1.4-6.el9.noarch.rpm 110 kB/s | 172 kB 00:01 2024-08-31T06:58:23.069 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-31T06:58:23.079 INFO:teuthology.orchestra.run.smithi032.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 788 kB/s | 19 kB 00:00 2024-08-31T06:58:23.105 INFO:teuthology.orchestra.run.smithi032.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-08-31T06:58:23.168 INFO:teuthology.orchestra.run.smithi084.stdout:(82/150): python3-babel-2.9.1-2.el9.noarch.rpm 627 kB/s | 6.0 MB 00:09 2024-08-31T06:58:23.176 INFO:teuthology.orchestra.run.smithi032.stdout:(128/150): python3-grpcio-1.46.7-10.el9.x86_64. 6.5 MB/s | 2.0 MB 00:00 2024-08-31T06:58:23.205 INFO:teuthology.orchestra.run.smithi032.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.6 MB/s | 46 kB 00:00 2024-08-31T06:58:23.247 INFO:teuthology.orchestra.run.smithi032.stdout:(130/150): protobuf-compiler-3.14.0-13.el9.x86_ 726 kB/s | 863 kB 00:01 2024-08-31T06:58:23.280 INFO:teuthology.orchestra.run.smithi032.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-08-31T06:58:23.299 INFO:teuthology.orchestra.run.smithi084.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-08-31T06:58:23.314 INFO:teuthology.orchestra.run.smithi032.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-08-31T06:58:23.340 INFO:teuthology.orchestra.run.smithi032.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-08-31T06:58:23.350 INFO:teuthology.orchestra.run.smithi084.stdout:(84/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 856 kB/s | 250 kB 00:00 2024-08-31T06:58:23.365 INFO:teuthology.orchestra.run.smithi032.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-08-31T06:58:23.375 INFO:teuthology.orchestra.run.smithi084.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-08-31T06:58:23.390 INFO:teuthology.orchestra.run.smithi032.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-08-31T06:58:23.400 INFO:teuthology.orchestra.run.smithi084.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-08-31T06:58:23.411 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-31T06:58:23.416 INFO:teuthology.orchestra.run.smithi032.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-08-31T06:58:23.441 INFO:teuthology.orchestra.run.smithi032.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-08-31T06:58:23.459 INFO:teuthology.orchestra.run.smithi084.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-08-31T06:58:23.466 INFO:teuthology.orchestra.run.smithi032.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-31T06:58:23.484 INFO:teuthology.orchestra.run.smithi084.stdout:(88/150): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-08-31T06:58:23.500 INFO:teuthology.orchestra.run.smithi032.stdout:(139/150): python3-kubernetes-26.1.0-2.el9.noar 2.6 MB/s | 1.0 MB 00:00 2024-08-31T06:58:23.518 INFO:teuthology.orchestra.run.smithi084.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-08-31T06:58:23.525 INFO:teuthology.orchestra.run.smithi032.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-08-31T06:58:23.535 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T06:58:23.543 INFO:teuthology.orchestra.run.smithi084.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-08-31T06:58:23.577 INFO:teuthology.orchestra.run.smithi084.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 920 kB/s | 54 kB 00:00 2024-08-31T06:58:23.584 INFO:teuthology.orchestra.run.smithi032.stdout:(141/150): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-08-31T06:58:23.609 INFO:teuthology.orchestra.run.smithi032.stdout:(142/150): python3-typing-extensions-4.12.2-2.e 686 kB/s | 74 kB 00:00 2024-08-31T06:58:23.634 INFO:teuthology.orchestra.run.smithi032.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-08-31T06:58:23.635 INFO:teuthology.orchestra.run.smithi084.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-08-31T06:58:23.659 INFO:teuthology.orchestra.run.smithi032.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 811 kB/s | 20 kB 00:00 2024-08-31T06:58:23.677 INFO:teuthology.orchestra.run.smithi084.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-08-31T06:58:23.683 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T06:58:23.685 INFO:teuthology.orchestra.run.smithi032.stdout:(145/150): python3-webob-1.8.8-2.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-08-31T06:58:23.711 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T06:58:23.718 INFO:teuthology.orchestra.run.smithi032.stdout:(146/150): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-08-31T06:58:23.736 INFO:teuthology.orchestra.run.smithi084.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-08-31T06:58:23.744 INFO:teuthology.orchestra.run.smithi032.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-08-31T06:58:23.778 INFO:teuthology.orchestra.run.smithi084.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 863 kB/s | 36 kB 00:00 2024-08-31T06:58:23.802 INFO:teuthology.orchestra.run.smithi032.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-08-31T06:58:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T06:58:23.828 INFO:teuthology.orchestra.run.smithi084.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-08-31T06:58:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T06:58:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-31T06:58:23.886 INFO:teuthology.orchestra.run.smithi084.stdout:(97/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-08-31T06:58:23.899 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-31T06:58:23.937 INFO:teuthology.orchestra.run.smithi084.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-08-31T06:58:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:(149/150): librados2-19.3.0-4560.gd19e6118.el9. 13 MB/s | 3.4 MB 00:00 2024-08-31T06:58:23.996 INFO:teuthology.orchestra.run.smithi084.stdout:(99/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.6 MB/s | 96 kB 00:00 2024-08-31T06:58:24.036 INFO:teuthology.orchestra.run.smithi032.stdout:(150/150): librbd1-19.3.0-4560.gd19e6118.el9.x8 11 MB/s | 3.1 MB 00:00 2024-08-31T06:58:24.043 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-08-31T06:58:24.043 INFO:teuthology.orchestra.run.smithi032.stdout:Total 9.5 MB/s | 224 MB 00:23 2024-08-31T06:58:24.055 INFO:teuthology.orchestra.run.smithi084.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-08-31T06:58:24.057 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-31T06:58:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-31T06:58:24.113 INFO:teuthology.orchestra.run.smithi084.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-08-31T06:58:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T06:58:24.214 INFO:teuthology.orchestra.run.smithi084.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-08-31T06:58:24.256 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T06:58:24.284 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T06:58:24.347 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T06:58:24.377 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T06:58:24.398 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T06:58:24.456 INFO:teuthology.orchestra.run.smithi084.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-08-31T06:58:24.523 INFO:teuthology.orchestra.run.smithi084.stdout:(104/150): python3-numpy-1.20.1-5.el9.x86_64.rp 3.4 MB/s | 5.1 MB 00:01 2024-08-31T06:58:24.573 INFO:teuthology.orchestra.run.smithi084.stdout:(105/150): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-08-31T06:58:24.602 INFO:teuthology.orchestra.run.smithi084.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 686 kB/s | 19 kB 00:00 2024-08-31T06:58:24.635 INFO:teuthology.orchestra.run.smithi084.stdout:(107/150): gperftools-libs-2.9.1-2.el9.x86_64.r 2.7 MB/s | 309 kB 00:00 2024-08-31T06:58:24.661 INFO:teuthology.orchestra.run.smithi084.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-31T06:58:24.686 INFO:teuthology.orchestra.run.smithi084.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-31T06:58:24.691 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T06:58:24.711 INFO:teuthology.orchestra.run.smithi084.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-31T06:58:24.726 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T06:58:24.736 INFO:teuthology.orchestra.run.smithi084.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-08-31T06:58:24.835 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/152 2024-08-31T06:58:24.875 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-31T06:58:24.895 INFO:teuthology.orchestra.run.smithi084.stdout:(112/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-08-31T06:58:24.910 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-31T06:58:24.937 INFO:teuthology.orchestra.run.smithi084.stdout:(113/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-08-31T06:58:24.962 INFO:teuthology.orchestra.run.smithi084.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-08-31T06:58:24.985 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-31T06:58:24.987 INFO:teuthology.orchestra.run.smithi084.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-31T06:58:25.005 INFO:teuthology.orchestra.run.smithi084.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 834 kB/s | 14 kB 00:00 2024-08-31T06:58:25.030 INFO:teuthology.orchestra.run.smithi084.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-08-31T06:58:25.036 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-31T06:58:25.063 INFO:teuthology.orchestra.run.smithi084.stdout:(118/150): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-08-31T06:58:25.076 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-31T06:58:25.089 INFO:teuthology.orchestra.run.smithi084.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-08-31T06:58:25.114 INFO:teuthology.orchestra.run.smithi084.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-08-31T06:58:25.119 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-31T06:58:25.139 INFO:teuthology.orchestra.run.smithi084.stdout:(121/150): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-08-31T06:58:25.152 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-31T06:58:25.164 INFO:teuthology.orchestra.run.smithi084.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-08-31T06:58:25.180 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-31T06:58:25.190 INFO:teuthology.orchestra.run.smithi084.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-08-31T06:58:25.248 INFO:teuthology.orchestra.run.smithi084.stdout:(124/150): libarrow-9.0.0-11.el9.x86_64.rpm 6.8 MB/s | 4.4 MB 00:00 2024-08-31T06:58:25.251 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-31T06:58:25.273 INFO:teuthology.orchestra.run.smithi084.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-08-31T06:58:25.299 INFO:teuthology.orchestra.run.smithi084.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-08-31T06:58:25.312 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-31T06:58:25.324 INFO:teuthology.orchestra.run.smithi084.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-08-31T06:58:25.341 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-31T06:58:25.349 INFO:teuthology.orchestra.run.smithi084.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-08-31T06:58:25.375 INFO:teuthology.orchestra.run.smithi084.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-31T06:58:25.376 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-31T06:58:25.400 INFO:teuthology.orchestra.run.smithi084.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-31T06:58:25.425 INFO:teuthology.orchestra.run.smithi084.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-31T06:58:25.456 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-31T06:58:25.459 INFO:teuthology.orchestra.run.smithi084.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-08-31T06:58:25.484 INFO:teuthology.orchestra.run.smithi084.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 653 kB/s | 16 kB 00:00 2024-08-31T06:58:25.487 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 30/152 2024-08-31T06:58:25.509 INFO:teuthology.orchestra.run.smithi084.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-08-31T06:58:25.531 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 31/152 2024-08-31T06:58:25.534 INFO:teuthology.orchestra.run.smithi084.stdout:(135/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-08-31T06:58:25.559 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 32/152 2024-08-31T06:58:25.560 INFO:teuthology.orchestra.run.smithi084.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-08-31T06:58:25.585 INFO:teuthology.orchestra.run.smithi084.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-08-31T06:58:25.627 INFO:teuthology.orchestra.run.smithi084.stdout:(138/150): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-08-31T06:58:25.633 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-31T06:58:25.652 INFO:teuthology.orchestra.run.smithi084.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 884 kB/s | 59 kB 00:00 2024-08-31T06:58:25.678 INFO:teuthology.orchestra.run.smithi084.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-08-31T06:58:25.691 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-31T06:58:25.703 INFO:teuthology.orchestra.run.smithi084.stdout:(141/150): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-08-31T06:58:25.728 INFO:teuthology.orchestra.run.smithi084.stdout:(142/150): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-08-31T06:58:25.753 INFO:teuthology.orchestra.run.smithi084.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-31T06:58:25.767 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-31T06:58:25.779 INFO:teuthology.orchestra.run.smithi084.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-08-31T06:58:25.804 INFO:teuthology.orchestra.run.smithi084.stdout:(145/150): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-08-31T06:58:25.829 INFO:teuthology.orchestra.run.smithi084.stdout:(146/150): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-08-31T06:58:25.856 INFO:teuthology.orchestra.run.smithi084.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-31T06:58:25.913 INFO:teuthology.orchestra.run.smithi084.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-08-31T06:58:25.920 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-31T06:58:25.978 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-31T06:58:25.978 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-08-31T06:58:26.105 INFO:teuthology.orchestra.run.smithi084.stdout:(149/150): librbd1-19.3.0-4560.gd19e6118.el9.x8 13 MB/s | 3.1 MB 00:00 2024-08-31T06:58:26.146 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-08-31T06:58:26.146 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-08-31T06:58:26.155 INFO:teuthology.orchestra.run.smithi084.stdout:(150/150): librados2-19.3.0-4560.gd19e6118.el9. 10 MB/s | 3.4 MB 00:00 2024-08-31T06:58:26.160 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-08-31T06:58:26.160 INFO:teuthology.orchestra.run.smithi084.stdout:Total 5.6 MB/s | 224 MB 00:39 2024-08-31T06:58:26.526 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-31T06:58:26.624 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-31T06:58:26.660 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-31T06:58:26.703 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-31T06:58:26.808 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 42/152 2024-08-31T06:58:26.841 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-31T06:58:26.935 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-31T06:58:26.998 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-31T06:58:27.071 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-31T06:58:27.120 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 47/152 2024-08-31T06:58:27.170 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-31T06:58:27.218 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-31T06:58:27.251 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-08-31T06:58:27.253 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-31T06:58:27.299 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-31T06:58:27.347 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-31T06:58:27.380 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-08-31T06:58:27.380 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-08-31T06:58:27.498 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-31T06:58:27.542 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-31T06:58:27.583 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-31T06:58:27.688 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-31T06:58:27.725 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 57/152 2024-08-31T06:58:27.765 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-31T06:58:27.791 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-31T06:58:27.815 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-31T06:58:27.891 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-31T06:58:28.020 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-31T06:58:28.056 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-31T06:58:28.084 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-31T06:58:28.118 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-31T06:58:28.162 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-31T06:58:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-31T06:58:28.239 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-31T06:58:28.269 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-31T06:58:28.304 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-31T06:58:28.345 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-31T06:58:28.387 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-31T06:58:28.422 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-31T06:58:28.510 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-31T06:58:28.561 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-31T06:58:28.591 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-31T06:58:28.624 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-08-31T06:58:28.624 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-08-31T06:58:28.697 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-31T06:58:28.760 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-31T06:58:29.133 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-08-31T06:58:29.133 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-08-31T06:58:29.509 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-31T06:58:29.556 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-31T06:58:29.590 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-31T06:58:29.693 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-31T06:58:29.754 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 83/152 2024-08-31T06:58:29.799 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-31T06:58:29.887 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-31T06:58:29.951 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-31T06:58:30.134 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-0.3.26-2.el9.x86_64 87/152 2024-08-31T06:58:30.164 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 88/152 2024-08-31T06:58:30.241 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-31T06:58:30.897 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-31T06:58:31.025 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-31T06:58:32.293 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-31T06:58:32.379 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T06:58:32.406 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-31T06:58:32.439 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 95/152 2024-08-31T06:58:32.768 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-31T06:58:32.819 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T06:58:32.910 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T06:58:32.940 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 98/152 2024-08-31T06:58:32.976 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-31T06:58:33.474 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-31T06:58:33.496 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T06:58:33.568 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T06:58:33.598 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 102/152 2024-08-31T06:58:34.993 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-08-31T06:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-08-31T06:58:35.057 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-31T06:58:35.082 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-31T06:58:35.117 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-31T06:58:35.124 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-31T06:58:35.173 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-31T06:58:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-31T06:58:35.549 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-31T06:58:35.591 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T06:58:35.670 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-31T06:58:35.717 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T06:58:35.742 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T06:58:35.774 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T06:58:35.832 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T06:58:35.875 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T06:58:35.904 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-31T06:58:35.910 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T06:58:35.912 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T06:58:35.930 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-31T06:58:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T06:58:35.945 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T06:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T06:58:36.084 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-31T06:58:36.105 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T06:58:36.139 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-31T06:58:36.173 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-31T06:58:36.272 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-31T06:58:36.297 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T06:58:36.403 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T06:58:36.431 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T06:58:36.448 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-31T06:58:36.477 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T06:58:36.499 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T06:58:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-31T06:58:36.515 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T06:58:36.520 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T06:58:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T06:58:36.556 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T06:58:36.655 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T06:58:36.656 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-31T06:58:36.656 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-31T06:58:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:58:36.670 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T06:58:36.682 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T06:58:36.706 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T06:58:36.792 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T06:58:36.816 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T06:58:36.817 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T06:58:36.848 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T06:58:36.859 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T06:58:36.957 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/152 2024-08-31T06:58:36.988 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-31T06:58:37.025 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-31T06:58:37.097 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-31T06:58:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-31T06:58:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-31T06:58:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-31T06:58:37.247 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T06:58:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-31T06:58:37.292 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T06:58:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-31T06:58:37.363 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-31T06:58:37.416 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-31T06:58:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/152 2024-08-31T06:58:37.454 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-31T06:58:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-31T06:58:37.489 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-31T06:58:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-31T06:58:37.567 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-31T06:58:37.600 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 30/152 2024-08-31T06:58:37.613 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-31T06:58:37.644 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 31/152 2024-08-31T06:58:37.672 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 32/152 2024-08-31T06:58:37.674 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-31T06:58:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-31T06:58:37.743 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-31T06:58:37.783 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-31T06:58:37.804 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-31T06:58:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-31T06:58:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-31T06:58:37.879 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-31T06:58:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-31T06:58:38.021 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-31T06:58:38.026 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-31T06:58:38.059 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-31T06:58:38.072 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-31T06:58:38.106 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-31T06:58:38.200 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-31T06:58:38.262 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 30/152 2024-08-31T06:58:38.311 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 31/152 2024-08-31T06:58:38.342 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 32/152 2024-08-31T06:58:38.442 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-31T06:58:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-31T06:58:38.591 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-31T06:58:38.607 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-31T06:58:38.693 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-31T06:58:38.721 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-31T06:58:38.764 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-31T06:58:38.775 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-31T06:58:38.802 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 42/152 2024-08-31T06:58:38.835 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-31T06:58:38.843 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-31T06:58:38.928 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-31T06:58:38.992 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-31T06:58:39.065 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-31T06:58:39.104 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 47/152 2024-08-31T06:58:39.140 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-31T06:58:39.187 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-31T06:58:39.222 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-31T06:58:39.274 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-31T06:58:39.316 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-31T06:58:39.454 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-31T06:58:39.501 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-31T06:58:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-31T06:58:39.543 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-31T06:58:39.637 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-31T06:58:39.647 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-31T06:58:39.677 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-31T06:58:39.693 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 57/152 2024-08-31T06:58:39.721 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-31T06:58:39.732 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-31T06:58:39.758 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-31T06:58:39.782 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-31T06:58:39.787 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 42/152 2024-08-31T06:58:39.833 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-31T06:58:39.857 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-31T06:58:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-31T06:58:39.976 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-31T06:58:40.006 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-31T06:58:40.034 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-31T06:58:40.039 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-31T06:58:40.069 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-31T06:58:40.112 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-31T06:58:40.126 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-31T06:58:40.150 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-31T06:58:40.182 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-31T06:58:40.185 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 47/152 2024-08-31T06:58:40.211 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-31T06:58:40.222 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-31T06:58:40.246 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-31T06:58:40.279 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-31T06:58:40.290 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-31T06:58:40.321 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-31T06:58:40.338 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-31T06:58:40.355 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-31T06:58:40.385 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-31T06:58:40.443 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-31T06:58:40.494 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-31T06:58:40.525 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-31T06:58:40.629 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-31T06:58:40.685 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-31T06:58:40.685 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-31T06:58:40.868 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-31T06:58:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-31T06:58:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-31T06:58:41.094 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-31T06:58:41.142 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 57/152 2024-08-31T06:58:41.188 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-31T06:58:41.224 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-31T06:58:41.248 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-31T06:58:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-31T06:58:41.424 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-31T06:58:41.493 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-31T06:58:41.531 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-31T06:58:41.536 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-31T06:58:41.581 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-31T06:58:41.618 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-31T06:58:41.632 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-31T06:58:41.669 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-31T06:58:41.677 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 83/152 2024-08-31T06:58:41.723 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-31T06:58:41.726 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-31T06:58:41.769 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-31T06:58:41.809 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-31T06:58:41.854 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-31T06:58:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-31T06:58:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-31T06:58:41.959 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-31T06:58:41.966 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-31T06:58:41.997 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-31T06:58:42.017 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-31T06:58:42.103 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-31T06:58:42.158 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-31T06:58:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.26-2.el9.x86_64 87/152 2024-08-31T06:58:42.223 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-31T06:58:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 88/152 2024-08-31T06:58:42.316 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-31T06:58:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-31T06:58:42.424 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-31T06:58:42.982 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-31T06:58:43.113 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-31T06:58:43.381 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-31T06:58:43.441 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-31T06:58:43.474 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-31T06:58:43.596 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-31T06:58:43.667 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 83/152 2024-08-31T06:58:43.720 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-31T06:58:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-31T06:58:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-31T06:58:44.141 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.26-2.el9.x86_64 87/152 2024-08-31T06:58:44.172 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 88/152 2024-08-31T06:58:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-31T06:58:44.364 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-31T06:58:44.449 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T06:58:44.478 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-31T06:58:44.511 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 95/152 2024-08-31T06:58:44.836 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-31T06:58:44.882 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T06:58:44.956 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T06:58:44.986 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 98/152 2024-08-31T06:58:45.022 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-31T06:58:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-31T06:58:45.312 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-31T06:58:45.517 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-31T06:58:45.542 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T06:58:45.614 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T06:58:45.644 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 102/152 2024-08-31T06:58:46.986 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-31T06:58:47.094 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T06:58:47.141 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-31T06:58:47.179 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 95/152 2024-08-31T06:58:47.578 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-31T06:58:47.649 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T06:58:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T06:58:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 98/152 2024-08-31T06:58:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-31T06:58:47.920 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T06:58:47.956 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T06:58:48.422 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-31T06:58:48.451 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T06:58:48.521 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T06:58:48.526 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T06:58:48.552 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 102/152 2024-08-31T06:58:48.559 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T06:58:48.651 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T06:58:48.652 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-31T06:58:48.652 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-31T06:58:48.652 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:58:48.686 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T06:58:48.754 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /sys 2024-08-31T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /proc 2024-08-31T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /mnt 2024-08-31T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /var/tmp 2024-08-31T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /home 2024-08-31T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /root 2024-08-31T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /tmp 2024-08-31T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:58:48.940 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T06:58:48.958 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T06:58:48.958 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:58:48.958 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-31T06:58:48.958 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:58:50.534 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T06:58:50.561 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T06:58:50.561 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:58:50.561 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-31T06:58:50.561 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T06:58:50.562 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T06:58:50.562 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:58:50.785 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T06:58:50.809 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T06:58:50.809 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:58:50.809 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-31T06:58:50.809 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T06:58:50.809 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T06:58:50.809 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:58:51.228 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T06:58:51.269 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T06:58:51.293 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T06:58:51.293 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:58:51.293 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-31T06:58:51.293 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T06:58:51.293 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T06:58:51.293 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:58:51.399 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-31T06:58:51.423 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T06:58:51.457 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-31T06:58:51.520 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-31T06:58:51.705 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-31T06:58:51.753 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-31T06:58:51.831 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-31T06:58:51.920 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-31T06:58:52.064 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-31T06:58:52.097 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-31T06:58:52.129 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-31T06:58:52.130 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T06:58:52.164 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-31T06:58:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T06:58:52.332 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T06:58:52.332 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-31T06:58:52.332 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-31T06:58:52.332 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:58:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T06:58:52.628 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T06:58:52.655 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T06:58:52.688 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 122/152 2024-08-31T06:58:52.753 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 123/152 2024-08-31T06:58:52.778 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T06:58:55.982 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T06:58:56.034 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T06:58:57.004 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T06:58:57.043 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T06:58:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T06:58:57.329 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 127/152 2024-08-31T06:58:57.362 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T06:58:57.387 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T06:58:57.387 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:58:57.387 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-31T06:58:57.387 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T06:58:57.387 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T06:58:57.387 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:58:57.437 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T06:58:57.449 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T06:58:57.550 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 130/152 2024-08-31T06:58:57.602 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 131/152 2024-08-31T06:58:58.572 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-31T06:58:58.598 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T06:58:58.622 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T06:58:58.622 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:58:58.622 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-31T06:58:58.622 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T06:58:58.622 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T06:58:58.622 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:58:58.666 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T06:58:58.688 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T06:58:58.688 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:58:58.688 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-31T06:58:58.688 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:58:58.741 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T06:58:58.764 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T06:58:58.765 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:58:58.765 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-31T06:58:58.765 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T06:58:58.765 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T06:58:58.765 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:58:59.083 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T06:58:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T06:58:59.107 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:58:59.107 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-31T06:58:59.107 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T06:58:59.107 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T06:58:59.107 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:59:00.459 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T06:59:00.459 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2024-08-31T06:59:00.459 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2024-08-31T06:59:00.459 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2024-08-31T06:59:00.460 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2024-08-31T06:59:00.460 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2024-08-31T06:59:00.460 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /root 2024-08-31T06:59:00.460 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2024-08-31T06:59:00.460 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:00.639 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T06:59:00.654 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T06:59:00.654 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:00.654 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-31T06:59:00.655 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:02.242 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T06:59:02.269 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T06:59:02.269 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:02.269 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-31T06:59:02.269 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T06:59:02.270 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T06:59:02.270 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:02.493 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T06:59:02.517 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T06:59:02.517 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:02.517 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-31T06:59:02.517 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T06:59:02.517 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T06:59:02.517 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:02.977 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T06:59:03.000 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T06:59:03.000 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:03.000 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-31T06:59:03.000 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T06:59:03.000 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T06:59:03.000 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:03.107 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-31T06:59:03.164 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-31T06:59:03.219 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-31T06:59:03.403 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-31T06:59:03.454 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 137/152 2024-08-31T06:59:03.459 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-31T06:59:03.496 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 138/152 2024-08-31T06:59:03.528 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-31T06:59:03.537 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 139/152 2024-08-31T06:59:03.568 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-31T06:59:03.595 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-31T06:59:03.617 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-31T06:59:03.748 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-31T06:59:03.780 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 142/152 2024-08-31T06:59:03.793 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-31T06:59:03.825 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-31T06:59:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-31T06:59:03.852 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-31T06:59:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 144/152 2024-08-31T06:59:04.241 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T06:59:04.278 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T06:59:04.310 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 122/152 2024-08-31T06:59:04.375 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 123/152 2024-08-31T06:59:04.409 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T06:59:04.455 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T06:59:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T06:59:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T06:59:05.361 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 146/152 2024-08-31T06:59:05.450 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 147/152 2024-08-31T06:59:05.491 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-31T06:59:05.575 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-31T06:59:05.606 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-31T06:59:05.606 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T06:59:05.685 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T06:59:05.685 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2024-08-31T06:59:05.685 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2024-08-31T06:59:05.685 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2024-08-31T06:59:05.685 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2024-08-31T06:59:05.685 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2024-08-31T06:59:05.685 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /root 2024-08-31T06:59:05.686 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2024-08-31T06:59:05.686 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:05.698 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T06:59:05.698 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T06:59:05.905 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T06:59:05.941 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T06:59:05.941 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:05.941 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-31T06:59:05.941 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:07.566 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T06:59:07.614 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T06:59:07.724 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T06:59:07.781 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T06:59:07.781 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:07.782 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-31T06:59:07.782 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T06:59:07.782 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T06:59:07.782 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:08.073 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T06:59:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T06:59:08.125 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:08.125 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-31T06:59:08.125 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T06:59:08.125 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T06:59:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:08.575 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T06:59:08.606 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T06:59:08.658 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T06:59:08.710 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T06:59:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-31T06:59:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T06:59:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T06:59:08.710 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:08.719 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T06:59:08.862 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-31T06:59:08.889 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 127/152 2024-08-31T06:59:08.925 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T06:59:08.928 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-31T06:59:08.949 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T06:59:08.949 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:08.949 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-31T06:59:08.949 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T06:59:08.949 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T06:59:08.949 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:09.000 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T06:59:09.006 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-31T06:59:09.012 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T06:59:09.110 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 130/152 2024-08-31T06:59:09.157 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 131/152 2024-08-31T06:59:09.248 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-31T06:59:09.313 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-31T06:59:09.402 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-31T06:59:09.511 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-31T06:59:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-31T06:59:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-31T06:59:09.777 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-31T06:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-31T06:59:10.112 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-31T06:59:10.144 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T06:59:10.168 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T06:59:10.168 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:10.168 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-31T06:59:10.168 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T06:59:10.168 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T06:59:10.169 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:10.211 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T06:59:10.234 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T06:59:10.234 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:10.234 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-31T06:59:10.234 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:10.287 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T06:59:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T06:59:10.310 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:10.310 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-31T06:59:10.310 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T06:59:10.310 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T06:59:10.310 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:10.382 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T06:59:10.421 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T06:59:10.464 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 122/152 2024-08-31T06:59:10.548 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 123/152 2024-08-31T06:59:10.585 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T06:59:10.629 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T06:59:10.653 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T06:59:10.653 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:10.653 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-31T06:59:10.653 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T06:59:10.653 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T06:59:10.653 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:12.356 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T06:59:12.356 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 1/152 2024-08-31T06:59:12.356 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2/152 2024-08-31T06:59:12.356 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 3/152 2024-08-31T06:59:12.356 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 4/152 2024-08-31T06:59:12.356 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 5/152 2024-08-31T06:59:12.356 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T06:59:12.356 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T06:59:12.356 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 8/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 9/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 10/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 11/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 13/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 14/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 15/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 16/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 17/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 18/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 19/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 20/152 2024-08-31T06:59:12.357 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 21/152 2024-08-31T06:59:12.358 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 22/152 2024-08-31T06:59:12.358 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 23/152 2024-08-31T06:59:12.358 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 24/152 2024-08-31T06:59:12.358 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 25/152 2024-08-31T06:59:12.358 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 26/152 2024-08-31T06:59:12.358 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 27/152 2024-08-31T06:59:12.368 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 28/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 29/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 30/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 31/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 32/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 33/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 34/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 35/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 36/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 37/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 38/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 39/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-31T06:59:12.369 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-31T06:59:12.370 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/152 2024-08-31T06:59:12.371 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-31T06:59:12.372 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-31T06:59:12.373 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-31T06:59:12.374 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/152 2024-08-31T06:59:12.375 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-31T06:59:12.376 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 149/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-31T06:59:12.377 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 151/152 2024-08-31T06:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T06:59:14.532 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T06:59:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T06:59:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:59:14.629 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2024-08-31T06:59:14.629 INFO:teuthology.orchestra.run.smithi073.stdout: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.630 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-08-31T06:59:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-31T06:59:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: automake-1.16.2-8.el9.noarch 2024-08-31T06:59:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: bison-3.7.4-5.el9.x86_64 2024-08-31T06:59:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-31T06:59:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.630 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:14.631 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.632 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: flex-2.6.4-9.el9.x86_64 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-31T06:59:14.633 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-31T06:59:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-31T06:59:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-31T06:59:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-31T06:59:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-31T06:59:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.634 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: lua-5.4.4-4.el9.x86_64 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-0.3.26-2.el9.x86_64 2024-08-31T06:59:14.635 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-08-31T06:59:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-08-31T06:59:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-31T06:59:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-13.16-1.el9.x86_64 2024-08-31T06:59:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-31T06:59:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-31T06:59:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-31T06:59:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-31T06:59:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-31T06:59:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-31T06:59:14.636 INFO:teuthology.orchestra.run.smithi073.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-31T06:59:14.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-31T06:59:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-31T06:59:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-31T06:59:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-08-31T06:59:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-31T06:59:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-31T06:59:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-31T06:59:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-31T06:59:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-31T06:59:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-31T06:59:14.638 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-31T06:59:14.639 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply-3.11-14.el9.noarch 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-31T06:59:14.640 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-31T06:59:14.641 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-31T06:59:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-31T06:59:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-31T06:59:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-31T06:59:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-31T06:59:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-31T06:59:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-31T06:59:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob-1.8.8-2.el9.noarch 2024-08-31T06:59:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-31T06:59:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-31T06:59:14.642 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: re2-1:20211101-20.el9.x86_64 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-31T06:59:14.643 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-31T06:59:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-31T06:59:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-31T06:59:14.644 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T06:59:14.644 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-08-31T06:59:14.975 DEBUG:teuthology.parallel:result is None 2024-08-31T06:59:14.999 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 137/152 2024-08-31T06:59:15.040 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 138/152 2024-08-31T06:59:15.081 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 139/152 2024-08-31T06:59:15.113 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-31T06:59:15.141 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-31T06:59:15.322 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 142/152 2024-08-31T06:59:15.361 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-31T06:59:15.391 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 144/152 2024-08-31T06:59:15.682 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T06:59:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T06:59:15.875 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T06:59:15.953 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T06:59:15.989 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T06:59:16.009 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T06:59:16.062 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 127/152 2024-08-31T06:59:16.093 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T06:59:16.144 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T06:59:16.145 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:16.145 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-31T06:59:16.145 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T06:59:16.145 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T06:59:16.145 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:16.202 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T06:59:16.228 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T06:59:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 130/152 2024-08-31T06:59:16.404 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 131/152 2024-08-31T06:59:16.862 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 146/152 2024-08-31T06:59:16.944 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 147/152 2024-08-31T06:59:16.983 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-31T06:59:17.064 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-31T06:59:17.096 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-31T06:59:17.096 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T06:59:17.180 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T06:59:17.180 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T06:59:17.509 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-31T06:59:17.538 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T06:59:17.588 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T06:59:17.589 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:17.589 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-31T06:59:17.589 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T06:59:17.589 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T06:59:17.589 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T06:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T06:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-31T06:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:17.747 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T06:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T06:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-31T06:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T06:59:17.798 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T06:59:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T06:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T06:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T06:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-31T06:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T06:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T06:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:23.336 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 137/152 2024-08-31T06:59:23.378 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 138/152 2024-08-31T06:59:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 139/152 2024-08-31T06:59:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-31T06:59:23.494 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-31T06:59:23.766 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 142/152 2024-08-31T06:59:23.810 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T06:59:23.810 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 1/152 2024-08-31T06:59:23.810 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 3/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 4/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 5/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 8/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 9/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 10/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 11/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 13/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 14/152 2024-08-31T06:59:23.811 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 15/152 2024-08-31T06:59:23.812 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 16/152 2024-08-31T06:59:23.812 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 17/152 2024-08-31T06:59:23.812 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 18/152 2024-08-31T06:59:23.812 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 19/152 2024-08-31T06:59:23.812 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 20/152 2024-08-31T06:59:23.812 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 21/152 2024-08-31T06:59:23.812 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 22/152 2024-08-31T06:59:23.812 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 23/152 2024-08-31T06:59:23.813 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 24/152 2024-08-31T06:59:23.813 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 25/152 2024-08-31T06:59:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-31T06:59:23.824 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 26/152 2024-08-31T06:59:23.824 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 27/152 2024-08-31T06:59:23.824 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 28/152 2024-08-31T06:59:23.824 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 29/152 2024-08-31T06:59:23.824 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 30/152 2024-08-31T06:59:23.824 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 31/152 2024-08-31T06:59:23.824 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 32/152 2024-08-31T06:59:23.824 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 33/152 2024-08-31T06:59:23.824 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 34/152 2024-08-31T06:59:23.824 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 35/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 36/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 37/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 38/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 39/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-31T06:59:23.825 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-31T06:59:23.826 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-31T06:59:23.827 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-31T06:59:23.828 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-31T06:59:23.829 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-31T06:59:23.830 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-31T06:59:23.831 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-31T06:59:23.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 149/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-31T06:59:23.833 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 151/152 2024-08-31T06:59:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 144/152 2024-08-31T06:59:24.569 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T06:59:24.633 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T06:59:24.674 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T06:59:26.031 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 146/152 2024-08-31T06:59:26.062 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T06:59:26.062 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:26.062 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2024-08-31T06:59:26.062 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.062 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.062 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-08-31T06:59:26.062 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: automake-1.16.2-8.el9.noarch 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: bison-3.7.4-5.el9.x86_64 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:26.063 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:26.064 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: flex-2.6.4-9.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-31T06:59:26.065 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: lua-5.4.4-4.el9.x86_64 2024-08-31T06:59:26.066 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.26-2.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-13.16-1.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-31T06:59:26.067 INFO:teuthology.orchestra.run.smithi084.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-08-31T06:59:26.068 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-31T06:59:26.069 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply-3.11-14.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-31T06:59:26.070 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-31T06:59:26.071 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.8-2.el9.noarch 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:26.072 INFO:teuthology.orchestra.run.smithi084.stdout: re2-1:20211101-20.el9.x86_64 2024-08-31T06:59:26.073 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-31T06:59:26.073 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-31T06:59:26.073 INFO:teuthology.orchestra.run.smithi084.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-31T06:59:26.073 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-31T06:59:26.073 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-31T06:59:26.073 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-31T06:59:26.073 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T06:59:26.073 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-08-31T06:59:26.146 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 147/152 2024-08-31T06:59:26.203 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-31T06:59:26.325 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-31T06:59:26.368 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-31T06:59:26.368 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T06:59:26.417 DEBUG:teuthology.parallel:result is None 2024-08-31T06:59:26.475 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T06:59:26.475 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T06:59:27.014 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T06:59:27.014 INFO:teuthology.orchestra.run.smithi032.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-08-31T06:59:27.014 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 1/152 2024-08-31T06:59:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2/152 2024-08-31T06:59:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 3/152 2024-08-31T06:59:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 4/152 2024-08-31T06:59:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 5/152 2024-08-31T06:59:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T06:59:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T06:59:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 8/152 2024-08-31T06:59:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 9/152 2024-08-31T06:59:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 10/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 11/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 13/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 14/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 15/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 16/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 17/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 18/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 19/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 20/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 21/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 22/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 23/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 24/152 2024-08-31T06:59:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 25/152 2024-08-31T06:59:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 26/152 2024-08-31T06:59:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 27/152 2024-08-31T06:59:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 28/152 2024-08-31T06:59:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 29/152 2024-08-31T06:59:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 30/152 2024-08-31T06:59:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 31/152 2024-08-31T06:59:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 32/152 2024-08-31T06:59:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 33/152 2024-08-31T06:59:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 34/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 35/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 36/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 37/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 38/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 39/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-31T06:59:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-31T06:59:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-31T06:59:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-31T06:59:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-31T06:59:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-31T06:59:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-31T06:59:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/152 2024-08-31T06:59:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-31T06:59:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 149/152 2024-08-31T06:59:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-31T06:59:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 151/152 2024-08-31T06:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T06:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2024-08-31T06:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-08-31T06:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-31T06:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: automake-1.16.2-8.el9.noarch 2024-08-31T06:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: bison-3.7.4-5.el9.x86_64 2024-08-31T06:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-31T06:59:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: flex-2.6.4-9.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-31T06:59:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: lua-5.4.4-4.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.26-2.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-13.16-1.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-31T06:59:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-31T06:59:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-31T06:59:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply-3.11-14.el9.noarch 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-31T06:59:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.8-2.el9.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-31T06:59:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-31T06:59:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T06:59:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: re2-1:20211101-20.el9.x86_64 2024-08-31T06:59:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-31T06:59:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-31T06:59:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-31T06:59:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-31T06:59:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-31T06:59:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-31T06:59:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T06:59:35.592 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-08-31T06:59:36.098 DEBUG:teuthology.parallel:result is None 2024-08-31T06:59:36.098 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T06:59:36.238 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-31T06:59:36.282 INFO:teuthology.orchestra.run.smithi032.stdout:19.3.0-4560.gd19e6118.el9 2024-08-31T06:59:36.283 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4560.gd19e6118.el9 2024-08-31T06:59:36.283 INFO:teuthology.task.install:The correct ceph version 19.3.0-4560.gd19e6118 is installed. 2024-08-31T06:59:36.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T06:59:36.429 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-31T06:59:36.455 INFO:teuthology.orchestra.run.smithi073.stdout:19.3.0-4560.gd19e6118.el9 2024-08-31T06:59:36.455 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4560.gd19e6118.el9 2024-08-31T06:59:36.456 INFO:teuthology.task.install:The correct ceph version 19.3.0-4560.gd19e6118 is installed. 2024-08-31T06:59:36.457 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T06:59:36.596 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-31T06:59:36.623 INFO:teuthology.orchestra.run.smithi084.stdout:19.3.0-4560.gd19e6118.el9 2024-08-31T06:59:36.624 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4560.gd19e6118.el9 2024-08-31T06:59:36.624 INFO:teuthology.task.install:The correct ceph version 19.3.0-4560.gd19e6118 is installed. 2024-08-31T06:59:36.626 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-31T06:59:36.626 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T06:59:36.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-31T06:59:36.687 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T06:59:36.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-31T06:59:36.720 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T06:59:36.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-31T06:59:36.752 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-31T06:59:36.753 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T06:59:36.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-08-31T06:59:36.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-31T06:59:36.908 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T06:59:36.908 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-08-31T06:59:36.937 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-31T06:59:37.006 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T06:59:37.006 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2024-08-31T06:59:37.036 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-31T06:59:37.106 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-31T06:59:37.107 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T06:59:37.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-31T06:59:37.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-31T06:59:37.266 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T06:59:37.266 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-31T06:59:37.292 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-31T06:59:37.356 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T06:59:37.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-31T06:59:37.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-31T06:59:37.454 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-31T06:59:37.454 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T06:59:37.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-08-31T06:59:37.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-31T06:59:37.610 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T06:59:37.611 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-08-31T06:59:37.636 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-31T06:59:37.702 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T06:59:37.702 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2024-08-31T06:59:37.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-31T06:59:37.800 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-31T06:59:37.894 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-08-31T06:59:37.895 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T06:59:37.895 INFO:tasks.cephadm:Cluster fsid is 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T06:59:37.895 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-31T06:59:37.896 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.32', 'mon.b': '172.21.15.73', 'mon.c': '172.21.15.84'} 2024-08-31T06:59:37.896 INFO:tasks.cephadm:First mon is mon.a on smithi032 2024-08-31T06:59:37.896 INFO:tasks.cephadm:First mgr is x 2024-08-31T06:59:37.896 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-31T06:59:37.896 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname $(hostname -s) 2024-08-31T06:59:37.957 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname $(hostname -s) 2024-08-31T06:59:37.982 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname $(hostname -s) 2024-08-31T06:59:38.013 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-31T06:59:38.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T06:59:38.152 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82586/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4560-gd19e6118', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4560.gd19e6118'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-28 11:15:32.801765', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-20240828.085847-main-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-08-31T06:59:38.286 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-mchangir-testing-20240828.085847-main-debug, sha1 d19e6118f7015f0047264006cb0c873528eb3ae6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T06:59:38.288 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm 2024-08-31T06:59:38.288 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm 2024-08-31T06:59:38.288 DEBUG:teuthology.orchestra.run.smithi032:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-31T06:59:38.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r--. 1 ubuntu ubuntu 801010 Aug 31 06:59 /home/ubuntu/cephtest/cephadm 2024-08-31T06:59:38.891 DEBUG:teuthology.orchestra.run.smithi073:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-31T06:59:39.494 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r--. 1 ubuntu ubuntu 801010 Aug 31 06:59 /home/ubuntu/cephtest/cephadm 2024-08-31T06:59:39.494 DEBUG:teuthology.orchestra.run.smithi084:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-31T06:59:40.064 INFO:teuthology.orchestra.run.smithi084.stdout:-rw-r--r--. 1 ubuntu ubuntu 801010 Aug 31 06:59 /home/ubuntu/cephtest/cephadm 2024-08-31T06:59:40.064 DEBUG:teuthology.orchestra.run.smithi032:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-31T06:59:40.102 DEBUG:teuthology.orchestra.run.smithi073:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-31T06:59:40.122 DEBUG:teuthology.orchestra.run.smithi084:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-31T06:59:40.166 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 on all hosts... 2024-08-31T06:59:40.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 pull 2024-08-31T06:59:40.170 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 pull 2024-08-31T06:59:40.175 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 pull 2024-08-31T06:59:41.659 INFO:teuthology.orchestra.run.smithi084.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-31T06:59:41.966 INFO:teuthology.orchestra.run.smithi073.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-31T06:59:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-31T07:00:14.240 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-08-31T07:00:14.240 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev)", 2024-08-31T07:00:14.240 INFO:teuthology.orchestra.run.smithi084.stdout: "image_id": "79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0", 2024-08-31T07:00:14.240 INFO:teuthology.orchestra.run.smithi084.stdout: "repo_digests": [ 2024-08-31T07:00:14.240 INFO:teuthology.orchestra.run.smithi084.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61" 2024-08-31T07:00:14.241 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-08-31T07:00:14.241 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-08-31T07:00:15.549 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:00:15.550 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev)", 2024-08-31T07:00:15.550 INFO:teuthology.orchestra.run.smithi032.stdout: "image_id": "79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0", 2024-08-31T07:00:15.550 INFO:teuthology.orchestra.run.smithi032.stdout: "repo_digests": [ 2024-08-31T07:00:15.550 INFO:teuthology.orchestra.run.smithi032.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61" 2024-08-31T07:00:15.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:00:15.550 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:00:19.231 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-08-31T07:00:19.231 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_version": "ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev)", 2024-08-31T07:00:19.231 INFO:teuthology.orchestra.run.smithi073.stdout: "image_id": "79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0", 2024-08-31T07:00:19.232 INFO:teuthology.orchestra.run.smithi073.stdout: "repo_digests": [ 2024-08-31T07:00:19.232 INFO:teuthology.orchestra.run.smithi073.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61" 2024-08-31T07:00:19.232 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-08-31T07:00:19.232 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-08-31T07:00:19.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph 2024-08-31T07:00:19.327 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph 2024-08-31T07:00:19.357 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph 2024-08-31T07:00:19.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /etc/ceph 2024-08-31T07:00:19.449 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /etc/ceph 2024-08-31T07:00:19.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /etc/ceph 2024-08-31T07:00:19.504 INFO:tasks.cephadm:Writing seed config... 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-08-31T07:00:19.506 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-08-31T07:00:19.507 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-31T07:00:19.507 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-31T07:00:19.507 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-08-31T07:00:19.507 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-31T07:00:19.507 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-31T07:00:19.507 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-31T07:00:19.507 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-31T07:00:19.507 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-31T07:00:19.507 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-31T07:00:19.507 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-08-31T07:00:19.508 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:00:19.508 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-31T07:00:19.544 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 965fc21a-6766-11ef-bcd4-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-08-31T07:00:19.544 DEBUG:teuthology.orchestra.run.smithi032:mon.a> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.a.service 2024-08-31T07:00:19.588 DEBUG:teuthology.orchestra.run.smithi032:mgr.x> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.x.service 2024-08-31T07:00:19.633 INFO:tasks.cephadm:Bootstrapping... 2024-08-31T07:00:19.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 -v bootstrap --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.32 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-31T07:00:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-08-31T07:00:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', '-v', 'bootstrap', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.32', '--skip-admin-label'] 2024-08-31T07:00:19.903 INFO:teuthology.orchestra.run.smithi032.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-31T07:00:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying podman|docker is present... 2024-08-31T07:00:20.527 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 5.2.2 2024-08-31T07:00:20.528 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying lvm2 is present... 2024-08-31T07:00:20.528 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying time synchronization is in place... 2024-08-31T07:00:20.542 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-31T07:00:20.542 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-31T07:00:20.554 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-31T07:00:20.555 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-08-31T07:00:20.568 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2024-08-31T07:00:20.581 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2024-08-31T07:00:20.581 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2024-08-31T07:00:20.581 INFO:teuthology.orchestra.run.smithi032.stdout:Repeating the final host check... 2024-08-31T07:00:20.636 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 5.2.2 2024-08-31T07:00:20.636 INFO:teuthology.orchestra.run.smithi032.stdout:podman (/bin/podman) version 5.2.2 is present 2024-08-31T07:00:20.636 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl is present 2024-08-31T07:00:20.636 INFO:teuthology.orchestra.run.smithi032.stdout:lvcreate is present 2024-08-31T07:00:20.649 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-31T07:00:20.649 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-31T07:00:20.662 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-31T07:00:20.662 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-08-31T07:00:20.675 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2024-08-31T07:00:20.688 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2024-08-31T07:00:20.688 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2024-08-31T07:00:20.689 INFO:teuthology.orchestra.run.smithi032.stdout:Host looks OK 2024-08-31T07:00:20.689 INFO:teuthology.orchestra.run.smithi032.stdout:Cluster fsid: 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:20.689 INFO:teuthology.orchestra.run.smithi032.stdout:Acquiring lock 139644541998128 on /run/cephadm/965fc21a-6766-11ef-bcd4-c7b262605968.lock 2024-08-31T07:00:20.689 INFO:teuthology.orchestra.run.smithi032.stdout:Lock 139644541998128 acquired on /run/cephadm/965fc21a-6766-11ef-bcd4-c7b262605968.lock 2024-08-31T07:00:20.689 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 3300 ... 2024-08-31T07:00:20.689 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 6789 ... 2024-08-31T07:00:20.690 INFO:teuthology.orchestra.run.smithi032.stdout:Base mon IP(s) is [172.21.15.32:3300, 172.21.15.32:6789], mon addrv is [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-08-31T07:00:20.697 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.32 metric 100 2024-08-31T07:00:20.697 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.32 metric 100 2024-08-31T07:00:20.703 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-31T07:00:20.703 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-08-31T07:00:20.703 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-08-31T07:00:20.709 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-31T07:00:20.709 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-31T07:00:20.709 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-31T07:00:20.710 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-08-31T07:00:20.710 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-08-31T07:00:20.710 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-31T07:00:20.710 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2024-08-31T07:00:20.710 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2024-08-31T07:00:20.710 INFO:teuthology.orchestra.run.smithi032.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-31T07:00:20.711 INFO:teuthology.orchestra.run.smithi032.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-31T07:00:20.711 INFO:teuthology.orchestra.run.smithi032.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-31T07:00:30.076 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 2024-08-31T07:00:30.076 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-31T07:00:30.076 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Getting image source signatures 2024-08-31T07:00:30.076 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-31T07:00:30.076 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:1c7f48e91f1eb9e77df623f13bab8a42aaf12d95954655c07fb62a1a85ca19b2 2024-08-31T07:00:30.076 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying config sha256:79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 2024-08-31T07:00:30.076 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-31T07:00:32.131 INFO:teuthology.orchestra.run.smithi032.stdout:ceph: stdout ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev) 2024-08-31T07:00:32.132 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph version: ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev) 2024-08-31T07:00:32.132 INFO:teuthology.orchestra.run.smithi032.stdout:Extracting ceph user uid/gid from container image... 2024-08-31T07:00:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:stat: stdout 167 167 2024-08-31T07:00:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial keys... 2024-08-31T07:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQCRv9Jm2leZJRAAkYCjBI8i8TIcojV3UsMj4A== 2024-08-31T07:00:35.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQCSv9Jmiuu6KBAAsJP7IDoDr+OJD37464p3jQ== 2024-08-31T07:00:36.354 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQCTv9JmGXKuLRAAOVQgY1h2eAcxQTK73CjHiw== 2024-08-31T07:00:36.355 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial monmap... 2024-08-31T07:00:37.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-31T07:00:37.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-31T07:00:37.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:37.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-31T07:00:37.415 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool for a [v2:172.21.15.32:3300,v1:172.21.15.32:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-31T07:00:37.415 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = quincy 2024-08-31T07:00:37.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: set fsid to 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:37.416 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-31T07:00:37.416 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:00:37.416 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mon... 2024-08-31T07:00:38.555 INFO:teuthology.orchestra.run.smithi032.stdout:create mon.a on 2024-08-31T07:00:38.834 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-31T07:00:39.057 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-31T07:00:39.337 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-965fc21a-6766-11ef-bcd4-c7b262605968.target → /etc/systemd/system/ceph-965fc21a-6766-11ef-bcd4-c7b262605968.target. 2024-08-31T07:00:39.337 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-965fc21a-6766-11ef-bcd4-c7b262605968.target → /etc/systemd/system/ceph-965fc21a-6766-11ef-bcd4-c7b262605968.target. 2024-08-31T07:00:39.679 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.a 2024-08-31T07:00:39.679 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.a.service: Unit ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.a.service not loaded. 2024-08-31T07:00:39.953 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-965fc21a-6766-11ef-bcd4-c7b262605968.target.wants/ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.a.service → /etc/systemd/system/ceph-965fc21a-6766-11ef-bcd4-c7b262605968@.service. 2024-08-31T07:00:40.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:40 smithi032 podman[16863]: 2024-08-31 07:00:40.516080361 +0000 UTC m=+0.328018619 container start 36a3936ff735860eb137a738423581445c4ac43165c042f399d27a710b02e863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:00:40.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:40 smithi032 bash[16863]: 36a3936ff735860eb137a738423581445c4ac43165c042f399d27a710b02e863 2024-08-31T07:00:40.615 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-31T07:00:40.615 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-08-31T07:00:40.629 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-31T07:00:40.629 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-08-31T07:00:40.629 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-08-31T07:00:40.629 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2024-08-31T07:00:40.629 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon to start... 2024-08-31T07:00:40.630 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon... 2024-08-31T07:00:40.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:40 smithi032 systemd[1]: Started Ceph mon.a for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:00:40.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:40 smithi032 ceph-mon[16882]: mkfs 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:40.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:40 smithi032 ceph-mon[16882]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-31T07:00:41.974 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-31T07:00:41.974 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: monmap epoch 1 2024-08-31T07:00:41.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:41.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: last_changed 2024-08-31T07:00:36.785570+0000 2024-08-31T07:00:41.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: created 2024-08-31T07:00:36.785570+0000 2024-08-31T07:00:41.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: min_mon_release 19 (squid) 2024-08-31T07:00:41.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: election_strategy: 1 2024-08-31T07:00:41.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-08-31T07:00:41.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: fsmap 2024-08-31T07:00:41.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: osdmap e1: 0 total, 0 up, 0 in 2024-08-31T07:00:41.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: mgrmap e1: no daemons active 2024-08-31T07:00:41.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:41 smithi032 ceph-mon[16882]: from='client.? 172.21.15.32:0/136551623' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout cluster: 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout id: 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout services: 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.67999s) 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout data: 2024-08-31T07:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-31T07:00:41.984 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-31T07:00:41.984 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-31T07:00:41.984 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pgs: 2024-08-31T07:00:41.984 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:41.984 INFO:teuthology.orchestra.run.smithi032.stdout:mon is available 2024-08-31T07:00:41.984 INFO:teuthology.orchestra.run.smithi032.stdout:Assimilating anything we can from ceph.conf... 2024-08-31T07:00:43.003 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:42 smithi032 ceph-mon[16882]: from='client.? 172.21.15.32:0/3771693185' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-31T07:00:43.003 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:42 smithi032 ceph-mon[16882]: from='client.? 172.21.15.32:0/3771693185' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-31T07:00:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2024-08-31T07:00:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-31T07:00:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-08-31T07:00:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-31T07:00:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-31T07:00:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-31T07:00:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-31T07:00:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-31T07:00:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-31T07:00:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2024-08-31T07:00:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-31T07:00:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-31T07:00:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-31T07:00:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-31T07:00:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-31T07:00:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:Generating new minimal ceph.conf... 2024-08-31T07:00:44.178 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:43 smithi032 ceph-mon[16882]: from='client.? 172.21.15.32:0/1962744062' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:00:44.460 INFO:teuthology.orchestra.run.smithi032.stdout:Restarting the monitor... 2024-08-31T07:00:44.843 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:44 smithi032 systemd[1]: Stopping Ceph mon.a for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:00:45.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:44 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a[16878]: 2024-08-31T07:00:44.842+0000 7f2caac00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:00:45.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:44 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a[16878]: 2024-08-31T07:00:44.842+0000 7f2caac00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-31T07:00:45.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:44 smithi032 podman[17125]: 2024-08-31 07:00:44.863781012 +0000 UTC m=+0.135821157 container died 36a3936ff735860eb137a738423581445c4ac43165c042f399d27a710b02e863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-08-31T07:00:45.364 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:45 smithi032 podman[17125]: 2024-08-31 07:00:45.364373915 +0000 UTC m=+0.636414056 container cleanup 36a3936ff735860eb137a738423581445c4ac43165c042f399d27a710b02e863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:00:45.760 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:45 smithi032 bash[17125]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a 2024-08-31T07:00:45.903 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:45 smithi032 podman[17136]: 2024-08-31 07:00:45.523679934 +0000 UTC m=+0.644923585 container remove 36a3936ff735860eb137a738423581445c4ac43165c042f399d27a710b02e863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True) 2024-08-31T07:00:46.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:45 smithi032 systemd[1]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-31T07:00:46.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:45 smithi032 systemd[1]: Stopped Ceph mon.a for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:00:46.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:45 smithi032 systemd[1]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.a.service: Consumed 1.313s CPU time. 2024-08-31T07:00:46.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:45 smithi032 systemd[1]: Starting Ceph mon.a for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:00:46.434 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 podman[17231]: 2024-08-31 07:00:46.034014139 +0000 UTC m=+0.028057052 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T07:00:46.434 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 podman[17231]: 2024-08-31 07:00:46.132867057 +0000 UTC m=+0.126909968 container create b31472b21c6499fe68fb2fe01e124d4798873f526931f3f50a4fca1ba1a80989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T07:00:46.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 podman[17231]: 2024-08-31 07:00:46.327209384 +0000 UTC m=+0.321252297 container init b31472b21c6499fe68fb2fe01e124d4798873f526931f3f50a4fca1ba1a80989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-31T07:00:46.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 podman[17231]: 2024-08-31 07:00:46.33415829 +0000 UTC m=+0.328201217 container start b31472b21c6499fe68fb2fe01e124d4798873f526931f3f50a4fca1ba1a80989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:00:46.446 INFO:teuthology.orchestra.run.smithi032.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-31T07:00:46.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: set uid:gid to 167:167 (ceph:ceph) 2024-08-31T07:00:46.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev), process ceph-mon, pid 2 2024-08-31T07:00:46.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: pidfile_write: ignore empty --pid-file 2024-08-31T07:00:46.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 bash[17231]: b31472b21c6499fe68fb2fe01e124d4798873f526931f3f50a4fca1ba1a80989 2024-08-31T07:00:46.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: load: jerasure load: lrc 2024-08-31T07:00:46.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: RocksDB version: 7.9.2 2024-08-31T07:00:46.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Git sha 0 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Compile date 2024-08-28 10:19:06 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: DB SUMMARY 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: DB Session ID: 3G07U8UQQOMGICJXQTW9 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: CURRENT file: CURRENT 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: IDENTITY file: IDENTITY 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94411 ; 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.error_if_exists: 0 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.create_if_missing: 0 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.paranoid_checks: 1 2024-08-31T07:00:46.750 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.env: 0x55db5f0b15a0 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.fs: PosixFileSystem 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.info_log: 0x55db8b7934a0 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_file_opening_threads: 16 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.statistics: (nil) 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.use_fsync: 0 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_log_file_size: 0 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-31T07:00:46.751 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.keep_log_file_num: 1000 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.recycle_log_file_num: 0 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.allow_fallocate: 1 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.allow_mmap_reads: 0 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.allow_mmap_writes: 0 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.use_direct_reads: 0 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.create_missing_column_families: 0 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.db_log_dir: 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.wal_dir: 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-31T07:00:46.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.advise_random_on_open: 1 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.db_write_buffer_size: 0 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.write_buffer_manager: 0x55db8b7975e0 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.rate_limiter: (nil) 2024-08-31T07:00:46.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.wal_recovery_mode: 2 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.enable_thread_tracking: 0 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.enable_pipelined_write: 0 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.unordered_write: 0 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.row_cache: None 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.wal_filter: None 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.allow_ingest_behind: 0 2024-08-31T07:00:46.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.two_write_queues: 0 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.manual_wal_flush: 0 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.wal_compression: 0 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.atomic_flush: 0 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.log_readahead_size: 0 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.best_efforts_recovery: 0 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.allow_data_in_errors: 0 2024-08-31T07:00:46.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.db_host_id: __hostname__ 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_background_jobs: 2 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_background_compactions: -1 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_subcompactions: 1 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_total_wal_size: 0 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-31T07:00:46.757 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_open_files: -1 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bytes_per_sync: 0 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_readahead_size: 0 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_background_flushes: -1 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Compression algorithms supported: 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: kZSTD supported: 0 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: kXpressCompression supported: 0 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: kBZip2Compression supported: 0 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: kLZ4Compression supported: 1 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: kZlibCompression supported: 1 2024-08-31T07:00:46.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: kLZ4HCCompression supported: 1 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: kSnappyCompression supported: 1 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 systemd[1]: Started Ceph mon.a for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.merge_operator: 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_filter: None 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_filter_factory: None 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.sst_partitioner_factory: None 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-31T07:00:46.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55db8b7930e0) 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: cache_index_and_filter_blocks: 1 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: pin_top_level_index_and_filter: 1 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_type: 0 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: data_block_index_type: 0 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_shortening: 1 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: checksum: 4 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: no_block_cache: 0 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache: 0x55db87756850 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_name: BinnedLRUCache 2024-08-31T07:00:46.760 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_options: 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: capacity : 536870912 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: num_shard_bits : 4 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: strict_capacity_limit : 0 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: high_pri_pool_ratio: 0.000 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_compressed: (nil) 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: persistent_cache: (nil) 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_size: 4096 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_size_deviation: 10 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_restart_interval: 16 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_block_restart_interval: 1 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: metadata_block_size: 4096 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: partition_filters: 0 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: use_delta_encoding: 1 2024-08-31T07:00:46.761 INFO:journalctl@ceph.mon.a.smithi032.stdout: filter_policy: bloomfilter 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: whole_key_filtering: 1 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: verify_compression: 0 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: read_amp_bytes_per_bit: 0 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: format_version: 5 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: enable_index_compression: 1 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_align: 0 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: max_auto_readahead_size: 262144 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: prepopulate_block_cache: 0 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: initial_auto_readahead_size: 8192 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout: num_file_reads_for_auto_readahead: 2 2024-08-31T07:00:46.762 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.write_buffer_size: 33554432 2024-08-31T07:00:46.763 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_write_buffer_number: 2 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compression: NoCompression 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bottommost_compression: Disabled 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.prefix_extractor: nullptr 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.num_levels: 7 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-31T07:00:46.764 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compression_opts.level: 32767 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compression_opts.strategy: 0 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compression_opts.enabled: false 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-31T07:00:46.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.target_file_size_base: 67108864 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-31T07:00:46.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.arena_block_size: 1048576 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.disable_auto_compactions: 0 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-31T07:00:46.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.inplace_update_support: 0 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.bloom_locality: 0 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.max_successive_merges: 0 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.paranoid_file_checks: 0 2024-08-31T07:00:46.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.force_consistency_checks: 1 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.report_bg_io_stats: 0 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.ttl: 2592000 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.enable_blob_files: false 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.min_blob_size: 0 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.blob_file_size: 268435456 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-31T07:00:46.769 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.blob_file_starting_level: 0 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bed84b3e-32a6-4e57-bbb6-19e29a44be34 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725087646444129, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725087646445974, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90641, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 280, "table_properties": {"data_size": 88682, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11486, "raw_average_key_size": 48, "raw_value_size": 82207, "raw_average_value_size": 343, "num_data_blocks": 12, "num_entries": 239, "num_filter_entries": 239, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725087646, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bed84b3e-32a6-4e57-bbb6-19e29a44be34", "db_session_id": "3G07U8UQQOMGICJXQTW9", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725087646446184, "job": 1, "event": "recovery_finished"} 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55db8b7b8a80 2024-08-31T07:00:46.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: DB pointer 0x55db8c55a000 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** DB Stats ** 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** Compaction Stats [default] ** 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-31T07:00:46.771 INFO:journalctl@ceph.mon.a.smithi032.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: L0 2/0 90.38 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: Sum 2/0 90.38 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** Compaction Stats [default] ** 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 75.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-31T07:00:46.772 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative compaction: 0.00 GB write, 11.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval compaction: 0.00 GB write, 11.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout: Block cache BinnedLRUCache@0x55db87756850#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: starting mon.a rank 0 at public addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] at bind addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: mon.a@-1(???) e1 preinit fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: mon.a@-1(???).mds e1 new map 2024-08-31T07:00:46.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: mon.a@-1(???).mds e1 print_map 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout: e1 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout: btime 2024-08-31T07:00:40:646567+0000 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout: legacy client fscid: -1 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout: No filesystems configured 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: monmap epoch 1 2024-08-31T07:00:46.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:46.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: last_changed 2024-08-31T07:00:36.785570+0000 2024-08-31T07:00:46.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: created 2024-08-31T07:00:36.785570+0000 2024-08-31T07:00:46.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: min_mon_release 19 (squid) 2024-08-31T07:00:46.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: election_strategy: 1 2024-08-31T07:00:46.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-08-31T07:00:46.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: fsmap 2024-08-31T07:00:46.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: osdmap e1: 0 total, 0 up, 0 in 2024-08-31T07:00:46.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:46 smithi032 ceph-mon[17252]: mgrmap e1: no daemons active 2024-08-31T07:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-31T07:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-31T07:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mgr... 2024-08-31T07:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-31T07:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-31T07:00:48.158 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.x 2024-08-31T07:00:48.158 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.x.service: Unit ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.x.service not loaded. 2024-08-31T07:00:48.392 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-965fc21a-6766-11ef-bcd4-c7b262605968.target.wants/ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-965fc21a-6766-11ef-bcd4-c7b262605968@.service. 2024-08-31T07:00:48.419 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:48 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2980295500' entity='client.admin' 2024-08-31T07:00:48.742 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:00:48 smithi032 systemd[1]: Starting Ceph mgr.x for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:00:48.996 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:00:48 smithi032 podman[17462]: 2024-08-31 07:00:48.645954033 +0000 UTC m=+0.027000791 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T07:00:48.996 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:00:48 smithi032 podman[17462]: 2024-08-31 07:00:48.775386587 +0000 UTC m=+0.156433338 container create 74ab0db0eca7851fa7969fecea00a799ca4bf05321c076ea4c006619337a6109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T07:00:48.996 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:00:48 smithi032 podman[17462]: 2024-08-31 07:00:48.99597079 +0000 UTC m=+0.377017543 container init 74ab0db0eca7851fa7969fecea00a799ca4bf05321c076ea4c006619337a6109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-31T07:00:49.113 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-31T07:00:49.114 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-08-31T07:00:49.127 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-31T07:00:49.127 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-08-31T07:00:49.128 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-08-31T07:00:49.128 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2024-08-31T07:00:49.145 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-31T07:00:49.145 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-08-31T07:00:49.158 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-31T07:00:49.159 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-08-31T07:00:49.159 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-08-31T07:00:49.159 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-31T07:00:49.159 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr to start... 2024-08-31T07:00:49.159 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr... 2024-08-31T07:00:49.300 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:00:49 smithi032 podman[17462]: 2024-08-31 07:00:49.002123603 +0000 UTC m=+0.383170356 container start 74ab0db0eca7851fa7969fecea00a799ca4bf05321c076ea4c006619337a6109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T07:00:49.301 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:00:49 smithi032 bash[17462]: 74ab0db0eca7851fa7969fecea00a799ca4bf05321c076ea4c006619337a6109 2024-08-31T07:00:49.301 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:00:49 smithi032 systemd[1]: Started Ceph mgr.x for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:00:50.194 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:49 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1459028665' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-31T07:00:50.554 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:50.554 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-31T07:00:50.555 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-31T07:00:50.556 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-31T07:00:50.557 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-31T07:00:50.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:50.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-31T07:00:50.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-31T07:00:50.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-31T07:00:50.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-31T07:00:50.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-31T07:00:50.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-31T07:00:50.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-31T07:00:50.560 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "btime": "2024-08-31T07:00:40:646567+0000", 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-31T07:00:50.561 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-08-31T07:00:40.647570+0000", 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-08-31T07:00:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (1/15)... 2024-08-31T07:00:53.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:53 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/852808934' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-31T07:00:53.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:53.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-08-31T07:00:53.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:00:53.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-08-31T07:00:53.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-31T07:00:53.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-31T07:00:53.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-31T07:00:53.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:53.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-31T07:00:53.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "btime": "2024-08-31T07:00:40:646567+0000", 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-31T07:00:53.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-08-31T07:00:40.647570+0000", 2024-08-31T07:00:53.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T07:00:53.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:53.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-31T07:00:53.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-08-31T07:00:53.841 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (2/15)... 2024-08-31T07:00:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: Activating manager daemon x 2024-08-31T07:00:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: mgrmap e2: x(active, starting, since 0.00481541s) 2024-08-31T07:00:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: from='mgr.14100 172.21.15.32:0/1149628396' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T07:00:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: from='mgr.14100 172.21.15.32:0/1149628396' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T07:00:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: from='mgr.14100 172.21.15.32:0/1149628396' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T07:00:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: from='mgr.14100 172.21.15.32:0/1149628396' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:00:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: from='mgr.14100 172.21.15.32:0/1149628396' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T07:00:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: Manager daemon x is now available 2024-08-31T07:00:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: from='mgr.14100 172.21.15.32:0/1149628396' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T07:00:55.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: from='mgr.14100 172.21.15.32:0/1149628396' entity='mgr.x' 2024-08-31T07:00:55.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: from='mgr.14100 172.21.15.32:0/1149628396' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T07:00:55.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: from='mgr.14100 172.21.15.32:0/1149628396' entity='mgr.x' 2024-08-31T07:00:55.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:55 smithi032 ceph-mon[17252]: from='mgr.14100 172.21.15.32:0/1149628396' entity='mgr.x' 2024-08-31T07:00:57.406 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:57.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-08-31T07:00:57.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:00:57.408 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-08-31T07:00:57.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-31T07:00:57.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:57.414 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "btime": "2024-08-31T07:00:40:646567+0000", 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T07:00:57.415 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:57.416 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-31T07:00:57.416 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T07:00:57.416 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-08-31T07:00:40.647570+0000", 2024-08-31T07:00:57.416 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T07:00:57.416 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-08-31T07:00:57.416 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-31T07:00:57.416 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-08-31T07:00:57.416 INFO:teuthology.orchestra.run.smithi032.stdout:mgr is available 2024-08-31T07:00:57.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:57 smithi032 ceph-mon[17252]: mgrmap e3: x(active, since 1.00995s) 2024-08-31T07:00:57.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:57 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3600098255' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-31T07:00:58.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:58 smithi032 ceph-mon[17252]: mgrmap e4: x(active, since 2s) 2024-08-31T07:00:58.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:58 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2079325896' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-31T07:00:59.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:59.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2024-08-31T07:00:59.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:00:59.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-31T07:00:59.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-08-31T07:00:59.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-31T07:00:59.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-31T07:00:59.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-31T07:00:59.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-31T07:00:59.029 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-31T07:00:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-31T07:00:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-08-31T07:00:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2024-08-31T07:00:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-31T07:00:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-31T07:00:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-31T07:00:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-31T07:00:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-31T07:00:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling cephadm module... 2024-08-31T07:01:00.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:00:59 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/595641889' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-08-31T07:01:00.358 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: ignoring --setuser ceph since I am not root 2024-08-31T07:01:00.358 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: ignoring --setgroup ceph since I am not root 2024-08-31T07:01:01.163 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:00 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/595641889' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-31T07:01:01.163 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:00 smithi032 ceph-mon[17252]: mgrmap e5: x(active, since 4s) 2024-08-31T07:01:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:01 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2765303423' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-31T07:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-08-31T07:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-31T07:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-31T07:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-31T07:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-31T07:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-08-31T07:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2024-08-31T07:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 5... 2024-08-31T07:01:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:06 smithi032 ceph-mon[17252]: Active manager daemon x restarted 2024-08-31T07:01:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:06 smithi032 ceph-mon[17252]: Activating manager daemon x 2024-08-31T07:01:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:06 smithi032 ceph-mon[17252]: osdmap e2: 0 total, 0 up, 0 in 2024-08-31T07:01:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:06 smithi032 ceph-mon[17252]: mgrmap e6: x(active, starting, since 0.00428466s) 2024-08-31T07:01:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:06 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:01:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:06 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T07:01:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:06 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T07:01:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:06 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T07:01:06.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:06 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T07:01:06.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:06 smithi032 ceph-mon[17252]: Manager daemon x is now available 2024-08-31T07:01:08.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-08-31T07:01:08.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-31T07:01:08.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-31T07:01:08.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-08-31T07:01:08.014 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 5 is available 2024-08-31T07:01:08.014 INFO:teuthology.orchestra.run.smithi032.stdout:Setting orchestrator backend to cephadm... 2024-08-31T07:01:08.327 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:08.327 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:08.327 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-mon[17252]: Found migration_current of "None". Setting to last migration. 2024-08-31T07:01:08.327 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:08.327 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:08.328 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:01:08.328 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:01:08.328 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T07:01:08.328 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T07:01:08.328 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-mon[17252]: mgrmap e7: x(active, since 1.00788s) 2024-08-31T07:01:08.328 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:01:08.220+0000 7f40c9200640 -1 log_channel(cephadm) log [ERR] : [31/Aug/2024:07:01:08] ENGINE Error in HTTPServer.serve 2024-08-31T07:01:08.328 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: Traceback (most recent call last): 2024-08-31T07:01:08.328 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self._connections.run(self.expiration_interval) 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self._run(expiration_interval) 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: s = self.context.wrap_socket( 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: return self.sslsocket_class._create( 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self.do_handshake() 2024-08-31T07:01:08.329 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T07:01:08.330 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self._sslobj.do_handshake() 2024-08-31T07:01:08.330 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T07:01:08.330 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:01:09.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:09 smithi032 ceph-mon[17252]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-31T07:01:09.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:09 smithi032 ceph-mon[17252]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-31T07:01:09.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:09 smithi032 ceph-mon[17252]: [31/Aug/2024:07:01:08] ENGINE Bus STARTING 2024-08-31T07:01:09.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:09 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:01:09.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:09 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:09.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:09 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:01:10.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:10 smithi032 ceph-mon[17252]: [31/Aug/2024:07:01:08] ENGINE Serving on https://172.21.15.32:7150 2024-08-31T07:01:10.232 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:10 smithi032 ceph-mon[17252]: [31/Aug/2024:07:01:08] ENGINE Error in HTTPServer.serve 2024-08-31T07:01:10.232 INFO:journalctl@ceph.mon.a.smithi032.stdout: Traceback (most recent call last): 2024-08-31T07:01:10.233 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T07:01:10.233 INFO:journalctl@ceph.mon.a.smithi032.stdout: self._connections.run(self.expiration_interval) 2024-08-31T07:01:10.233 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T07:01:10.233 INFO:journalctl@ceph.mon.a.smithi032.stdout: self._run(expiration_interval) 2024-08-31T07:01:10.233 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T07:01:10.234 INFO:journalctl@ceph.mon.a.smithi032.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T07:01:10.234 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T07:01:10.234 INFO:journalctl@ceph.mon.a.smithi032.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T07:01:10.234 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T07:01:10.234 INFO:journalctl@ceph.mon.a.smithi032.stdout: s = self.context.wrap_socket( 2024-08-31T07:01:10.235 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T07:01:10.235 INFO:journalctl@ceph.mon.a.smithi032.stdout: return self.sslsocket_class._create( 2024-08-31T07:01:10.235 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T07:01:10.235 INFO:journalctl@ceph.mon.a.smithi032.stdout: self.do_handshake() 2024-08-31T07:01:10.235 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T07:01:10.235 INFO:journalctl@ceph.mon.a.smithi032.stdout: self._sslobj.do_handshake() 2024-08-31T07:01:10.236 INFO:journalctl@ceph.mon.a.smithi032.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T07:01:10.236 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:10 smithi032 ceph-mon[17252]: [31/Aug/2024:07:01:08] ENGINE Serving on http://172.21.15.32:8765 2024-08-31T07:01:10.236 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:10 smithi032 ceph-mon[17252]: [31/Aug/2024:07:01:08] ENGINE Bus STARTED 2024-08-31T07:01:10.236 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:10 smithi032 ceph-mon[17252]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:10.236 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:10 smithi032 ceph-mon[17252]: mgrmap e8: x(active, since 2s) 2024-08-31T07:01:11.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-31T07:01:11.127 INFO:teuthology.orchestra.run.smithi032.stdout:Generating ssh key... 2024-08-31T07:01:11.431 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:11 smithi032 ceph-mon[17252]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:12.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-mon[17252]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:12.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-mon[17252]: Generating ssh key... 2024-08-31T07:01:12.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:12.583 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:12.584 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: Generating public/private rsa key pair. 2024-08-31T07:01:12.584 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: Your identification has been saved in /tmp/tmpru75esyi/key 2024-08-31T07:01:12.584 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: Your public key has been saved in /tmp/tmpru75esyi/key.pub 2024-08-31T07:01:12.584 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: The key fingerprint is: 2024-08-31T07:01:12.584 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: SHA256:8BbDnnU2UyK4xMCKgUrHkrYSTA1lYB+w7aWNrHDU6y4 ceph-965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:01:12.584 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: The key's randomart image is: 2024-08-31T07:01:12.584 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: +---[RSA 3072]----+ 2024-08-31T07:01:12.584 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: |oBO+ ..o .. . . | 2024-08-31T07:01:12.584 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: |+**=. ..+ . o | 2024-08-31T07:01:12.584 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: |+o==.o..+.. = | 2024-08-31T07:01:12.585 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: |oo+ *. +.= o o | 2024-08-31T07:01:12.585 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: |o .=.. S | 2024-08-31T07:01:12.585 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: |.... . | 2024-08-31T07:01:12.585 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: | . . | 2024-08-31T07:01:12.585 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: | E. | 2024-08-31T07:01:12.585 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: | .. | 2024-08-31T07:01:12.585 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: +----[SHA256]-----+ 2024-08-31T07:01:14.273 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:14 smithi032 ceph-mon[17252]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:14.280 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfqbAmmvCdA/isadVM9EzhIcxuCgkCq1SNklmcIcAtsb//y+vFRcJOjDqSFVIibEGkLThYfGnuCS62WFl2C1YAuJ96LlhHF8aAODTpqWfKy1flnsAvAl8FsToGpUoQQFQOkeJlj21Kkb0/pXvnTml6urO/wq9qaxWgwso1eKyoSX09Q8HReHcQbUip8TRV6888056IdI1oED0h9o38TPP+vLctSVcbTRe3Xgw3580njS5S7iPXGxG3ba/FiPnPsz8UBtV01DRJAZHpHBwj35tROG9hAao9EbtyNwien3WtBugpkPqzBXM65S2wdYw+pzZBYV9AIP/AZDQI5FiEvPcHm3/igielQR2cOWaBWxqPhxYZRxXVWVoXFVFVIc7xRE1pcc6l9nn2BN5gWYgksm4i/UpVAF+ENltjR5cb61LrZgCheP81piem+djiuInGmR8TrbllGn2SHqbPNP46RaXTU7i1wW4d5BTf9lqN4gJtuNI46l4QlHlsVr1OWEpa4kk= ceph-965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:01:14.280 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-31T07:01:14.280 INFO:teuthology.orchestra.run.smithi032.stdout:Adding key to root@localhost authorized_keys... 2024-08-31T07:01:14.299 INFO:teuthology.orchestra.run.smithi032.stdout:Adding host smithi032... 2024-08-31T07:01:15.532 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:15 smithi032 ceph-mon[17252]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:17.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:17 smithi032 ceph-mon[17252]: Deploying cephadm binary to smithi032 2024-08-31T07:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Added host 'smithi032' with addr '172.21.15.32' 2024-08-31T07:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying unmanaged mon service... 2024-08-31T07:01:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:20 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:20 smithi032 ceph-mon[17252]: Added host smithi032 2024-08-31T07:01:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:20 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:01:22.827 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:22 smithi032 ceph-mon[17252]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:22.828 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:22 smithi032 ceph-mon[17252]: Saving service mon spec with placement count:5 2024-08-31T07:01:22.828 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:22 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:22.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-31T07:01:22.833 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying unmanaged mgr service... 2024-08-31T07:01:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:24 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:24 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:24 smithi032 ceph-mon[17252]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:24 smithi032 ceph-mon[17252]: Saving service mgr spec with placement count:2 2024-08-31T07:01:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:24 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:24.692 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-31T07:01:26.797 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:26 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1993433505' entity='client.admin' 2024-08-31T07:01:28.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:27 smithi032 ceph-mon[17252]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:01:28.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:27 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:28.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:27 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2322338899' entity='client.admin' 2024-08-31T07:01:28.058 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling the dashboard module... 2024-08-31T07:01:29.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:29 smithi032 ceph-mon[17252]: from='mgr.14118 172.21.15.32:0/2938682827' entity='mgr.x' 2024-08-31T07:01:29.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:29 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2954353166' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-08-31T07:01:29.491 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:29 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: ignoring --setuser ceph since I am not root 2024-08-31T07:01:29.492 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:29 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: ignoring --setgroup ceph since I am not root 2024-08-31T07:01:30.221 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:30 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2954353166' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-31T07:01:30.221 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:30 smithi032 ceph-mon[17252]: mgrmap e9: x(active, since 22s) 2024-08-31T07:01:31.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:31 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1614756530' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-31T07:01:31.852 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-08-31T07:01:31.852 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-31T07:01:31.852 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-31T07:01:31.852 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-31T07:01:31.852 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-31T07:01:31.852 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-08-31T07:01:31.853 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2024-08-31T07:01:31.853 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 9... 2024-08-31T07:01:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:35 smithi032 ceph-mon[17252]: Active manager daemon x restarted 2024-08-31T07:01:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:35 smithi032 ceph-mon[17252]: Activating manager daemon x 2024-08-31T07:01:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:35 smithi032 ceph-mon[17252]: osdmap e3: 0 total, 0 up, 0 in 2024-08-31T07:01:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:35 smithi032 ceph-mon[17252]: mgrmap e10: x(active, starting, since 0.00417353s) 2024-08-31T07:01:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:01:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T07:01:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T07:01:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T07:01:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T07:01:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:35 smithi032 ceph-mon[17252]: Manager daemon x is now available 2024-08-31T07:01:36.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:36 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:01:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:36 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T07:01:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:36 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T07:01:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:36 smithi032 ceph-mon[17252]: mgrmap e11: x(active, since 1.0081s) 2024-08-31T07:01:37.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-08-31T07:01:37.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-31T07:01:37.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-31T07:01:37.293 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-08-31T07:01:37.293 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 9 is available 2024-08-31T07:01:37.293 INFO:teuthology.orchestra.run.smithi032.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-08-31T07:01:37.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-mon[17252]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-31T07:01:37.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-mon[17252]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-31T07:01:38.244 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:01:37.874+0000 7f4a32200640 -1 log_channel(cephadm) log [ERR] : [31/Aug/2024:07:01:37] ENGINE Error in HTTPServer.serve 2024-08-31T07:01:38.244 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: Traceback (most recent call last): 2024-08-31T07:01:38.244 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T07:01:38.244 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self._connections.run(self.expiration_interval) 2024-08-31T07:01:38.244 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T07:01:38.244 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self._run(expiration_interval) 2024-08-31T07:01:38.244 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T07:01:38.244 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T07:01:38.244 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: s = self.context.wrap_socket( 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: return self.sslsocket_class._create( 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self.do_handshake() 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self._sslobj.do_handshake() 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T07:01:38.245 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:01:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:01:38.975 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:38 smithi032 ceph-mon[17252]: mgrmap e12: x(active, since 2s) 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:38 smithi032 ceph-mon[17252]: [31/Aug/2024:07:01:37] ENGINE Bus STARTING 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:38 smithi032 ceph-mon[17252]: [31/Aug/2024:07:01:37] ENGINE Serving on https://172.21.15.32:7150 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:38 smithi032 ceph-mon[17252]: [31/Aug/2024:07:01:37] ENGINE Error in HTTPServer.serve 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout: Traceback (most recent call last): 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout: self._connections.run(self.expiration_interval) 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout: self._run(expiration_interval) 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T07:01:38.976 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout: s = self.context.wrap_socket( 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout: return self.sslsocket_class._create( 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout: self.do_handshake() 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout: self._sslobj.do_handshake() 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:38 smithi032 ceph-mon[17252]: [31/Aug/2024:07:01:37] ENGINE Serving on http://172.21.15.32:8765 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:38 smithi032 ceph-mon[17252]: [31/Aug/2024:07:01:37] ENGINE Bus STARTED 2024-08-31T07:01:38.977 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:38 smithi032 ceph-mon[17252]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:39.591 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUKa5D407Az3hOssC6lp+477ScTYowDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDgzMTA3MDEzOFoXDTM0\nMDkwMTA3MDEzOFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjMyMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAymh8gF+rN9EdwJzx4asyRGKHeITl2VH8QNu4\nrE4SAVttRtWvwlc3RBt75WeEqnan1gLGNpfA4Np5cqYoiOlE0ICYZPQfCcz2bcr5\nhR5OR+lnWJtsSgHERFCTofYU4gtqcUWN0O4vLMprK0JZ1ALGO5Uvsgx0rueLPCeQ\nJ4WqaQxzakKbLPx+ntm8pxb43Gv1YO99q468KnN7PycJOLdg6YCYvx+jHvCTdDUK\ni8KeQo0nXIeEpiOsov3HU9FGzOLgTnSfTywnD1jsHEDoCDz/vIW+JVFDtQn0ao9w\nNcVvigWbXA3BYZjCQN3W5TLQWDfYQxUIYoluqnW6d+YsIi89WHOKei4Rjw8e/hdn\n9156h0UxqWkTt5sK614rY+/GmRVqC7fi1EXIWLCHOLSUusXs2NeINv+J36mQSPKV\nCjkCj0+r2V5ZzlVriPEP/pTJjozGfs/gyqbi6GEm8i4Ap9J7wcdHHw++fUlDhOml\npwW3rc3jGJvty2uuA6Mpq1LjxaJU+D6nZSOsYyT30wozFRV6ph4x5pvD9d3IoW1o\nxwCa1juFqMYisnzwX9ClPFWhABPrMdfKp/y323m5mQdlGRIN+uB5YszQ4DFNjq69\nlRAHKNzCC9Y2C3wUHPn5isgfAhzYlHRobM8uypHoj/11hbxq52HuY9lBJ+8B64eQ\naHFp13cCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDMyghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPIDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQBC\nUt7RNGA0nbtWDZdnmo8/8lkYqwy7Tjo5TNjbIuJ9RsKPsxN6MYAXni2BQwXPrbmn\nVKbO5pBrE09hnDpTVv60hrKFoP6+ZrYZAb9gDyOsWZSkcicYASiBP2F2mZCMNv0t\nLvagYr/G8ZHuTrDc89X7IM6/ZF/gnKEkHl+sDbIQmp4z5VkwsGTrkp52oSsHWhmy\n1Vo2Rb+EdlNEUqnZ1l1qsGKdsQcojpyCuib1I85XsQROs9n3DoiHZXcquyYdaG1/\nPrPGahNB/kbFQ+nBiaW+Az/Tifq/wuSyIPpAaUiFSG7SxyuAEVdLFs6d4WtLfmFv\ndiqBw2k5J/B2B/83BSp+YSMSlO2idFHsXsKIaHyn/Flpc52BBjmTxckqC0Xrenrt\n1O2w2kRZXtzKjQpCsRqxipH0aZv+28cBjSUP074g/9pgvWxJ6nfuY2l65LvoSGQ8\n9qxbiF9NVO+u9Vsnn8J1ealleobYS8xnbX+CbWjRlKCK3Nevf7KxawyablhcZgIx\n8u23tTY4qa2UT6pKp85ypvwjTwh/nb3vbkrDqG/v16NtGEcU9cHiD30OJrtvJlw8\nZzNHO0ajzF566goq+7yat+sgCH3IRubJ/mAmne/KskmmHRjfTe0Dh54oOSCU1Gaj\nPzGDV5NJsWnclLhhhwJf7u7bp7sOSe+A3ZH1b1tHtw==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJJwIBAAKCAgEAymh8gF+rN9EdwJzx4asyRGKHeITl2VH8QNu4rE4SAVttRtWv\nwlc3RBt75WeEqnan1gLGNpfA4Np5cqYoiOlE0ICYZPQfCcz2bcr5hR5OR+lnWJts\nSgHERFCTofYU4gtqcUWN0O4vLMprK0JZ1ALGO5Uvsgx0rueLPCeQJ4WqaQxzakKb\nLPx+ntm8pxb43Gv1YO99q468KnN7PycJOLdg6YCYvx+jHvCTdDUKi8KeQo0nXIeE\npiOsov3HU9FGzOLgTnSfTywnD1jsHEDoCDz/vIW+JVFDtQn0ao9wNcVvigWbXA3B\nYZjCQN3W5TLQWDfYQxUIYoluqnW6d+YsIi89WHOKei4Rjw8e/hdn9156h0UxqWkT\nt5sK614rY+/GmRVqC7fi1EXIWLCHOLSUusXs2NeINv+J36mQSPKVCjkCj0+r2V5Z\nzlVriPEP/pTJjozGfs/gyqbi6GEm8i4Ap9J7wcdHHw++fUlDhOmlpwW3rc3jGJvt\ny2uuA6Mpq1LjxaJU+D6nZSOsYyT30wozFRV6ph4x5pvD9d3IoW1oxwCa1juFqMYi\nsnzwX9ClPFWhABPrMdfKp/y323m5mQdlGRIN+uB5YszQ4DFNjq69lRAHKNzCC9Y2\nC3wUHPn5isgfAhzYlHRobM8uypHoj/11hbxq52HuY9lBJ+8B64eQaHFp13cCAwEA\nAQKCAgAdBhWQPOeq1dAb3gVrNCa9mplNZVUmL/Tpgfg9sOYxDqChTyPoTKROnlhK\nQWlLHj+b1wMAMguE0p8mmv/l/fkRotb3ZvAKcjsF7FySQ5G7CxN4J7VD+XPgzeMV\njtlYvoVRX8Vhp73ZAN+UMOv7lDhDs/VGZTXYHLig9BVeLXMNpCzqmElzDnPDqHo+\nRXOC+/IQJ9dP4eRGfA+4swrF+LmrUfbVNR6pYZrrR23PM6WI6n+x63bQcf4f0x8h\nfDlYSNIcPenYKv8g2sHknUQIZmLPcCgfZsM8MwS6wP24oCqJ0pr//LL1oTd9oLHf\nZMVuWPeoStxMcOOPCtmltrouXrK/ejK9WNOeTgmzOXkao2re5Olrj6rdPfMOnMYl\nkfmjXh1UnkHFPCxKhb4CTW+CtSdlOkZJKPcyRkMll2NVQ3A8za4m7YrNFilxmOOH\ntEEnzzrauLd5MS7jZvwAGwXYLfwTwoiyc5dKTOnSWgjm7xXo9fGgFZjSGqp4iGPl\n906kpN+i3gu9AOFYZ4SYJGxXszoHT6vl/k8eduAckDAYtgss9s8VeiNQZm6+njtX\n8kGlPnSMhw1hLIAQ2e9cwHWYiEC2XiDtjrPyQsbMdEX2WR7/L+qux5U3Fn9AJ27L\njIp73SIvCJDhtzJQIQr1uJkcZmB0wYrsQ8W+y3nJrFnCUhsebQKCAQEA946daQFa\nydQ2k4CmIeBPP1hqcE7k68dR3nALlL8ClzC5Y8MCQb+WH4XXUbibXzZkjdjOuemY\nXB91KzONiQx3mKp+23Ect8ACBDUAWiYBsFTwozSa8wKhFJ4zZPxnxTdGZHvxS4ZY\nTQwcbYNCtzkFr12HeYMAoxL1kw+BBubRUUvYUE6zsq4ZXykpWJQKljPpy1MYaLe1\nOpGtqZb53w9XxsN69/Olya9mhKJrRuTK697yrJRvAXXrlZGowHx98+k5Hzq/ikOe\nJmEGbkxdu/umTMPc5DY+ADWcvvDb5eZdo5ChInDJCIGFGCmEW7mDRmvByS3XXUjG\nwa1Y62niPKCzmwKCAQEA0U+uvmrYkn5odza8fC7vJQuBw9dPBNkZHYiV0n2Z9Z34\nyLvRXQ7bU+34Q3goJ6H41tzf8yE3938iB0p6SGBTussEksxKkChTcZlI2UvDcTCu\nexFV8fTwTrBaBOF1ItlnR2BhNCymz8hWk+G/xQMhPOuxmZmyNlvZiCgMb02vbEF0\nAZ3l9ZgqnMSwLUGZm333YJucPzHFGTVOR5FS7Dyd2iK6VlBuyhP7w004qsVR4S9t\n6X6mqH30JA7pkR43jhJFNkdgiuCXDoErBDbLO4uvfFeEDbCs+q6GXlcLDx0vlARf\nuFeoWbl68MZPybOzeaMmQCGg2vzRbrGOiJtRDGpvVQKCAQApEj8pYbEnVQ7s7+Yg\nLq/XKZsXIO+38myxw9MYDYynesz3MS5ZrCiC+Fid1UvpwwNxySubCpddiVaOIVJ1\nD110Na0wSRbjoXZ0LjrtueqDz8PQv/SJJsGEwy5xrbKltKL4HtwMlLBnr/+zZ5iU\n/1/jFVhZn5NmesnFaUgV1guaOkQcJsKHfXz2HcUfTghnnRtHwXSGQxxNKEgwn7y6\n9jew4VOweXri+DfA/5VewKyt72JBoaq9CkxYlz7xGHZpFiOE++E1VjyRz30apY7T\nlk+j4sbuEdW9gSeNQjapcMvOTDt/lEi2IY7sUE8GTMPr1grjjrhxYJahIVpkTD/d\noSfLAoIBADaBU3XOiPsM49M3c7m61xeEeZQv+ym+BuZD7Q/VGUR2kJRlJE3hUof2\njhGztnEroosBFWhQnCxc9NWFNXanbuZTe6tme3wypBgO3IOJUSmmjakKiKDrMbCs\nB6UBuOkMR/K0TFuzzPSKrNDxlmiNOhiNH8Bp35h8LpdgJmzNhALiwHyzS3C64U3V\ngjMmxANTuBl2LCfgo/y+ftXwJ/HQjyKArJlRdKg2r/bZowoFPB8xpKOLEYcK1YqW\nP4hEoB7k7xUFXUSM24kxZaxBQIcPBvGQMJkICuN0kxT6Aa074mnfAbYVTYef3z3X\nxms12mSwcYN9vk+ISllS71TvbRmYlckCggEAMb/xbrx8PwOnFJhBQtCOcUQGzoPS\nhUR05EhxeIdFfgcoh9XcaH21w5MEJUBldnf8FNZmSf9Pgeodjns1dSXvCONkDioX\nTQJtkXmlcfdzai5unbbIMTvSwfqLP7sDNy5/ifuQIPxbq5jP65Ql5RtFzc+1BELa\nopyYqBN622lRaebifCH6nsnQ6fAggegp6L7jAteLTFH/ui4k3mjoRiMwAo0g/7iL\nVxTOCSysS1Ad4/w/9v+irDK5C+0RMBsTvZHs+DzSUUVGZ5P9WYbje3QDH6ORDjR2\n0Tp7cebr+cdJ2u7+vj9cKyWwvkDZ/nQT1tPpdaAJw4W2lLR7RB03so2y4A==\n-----END RSA PRIVATE KEY-----\n"} 2024-08-31T07:01:39.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:39 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:41.075 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:40 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:41.076 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:40 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:41.945 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-08-31T07:01:42.341 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:41 smithi032 ceph-mon[17252]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-08-31T07:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial admin user... 2024-08-31T07:01:44.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:44 smithi032 ceph-mon[17252]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:44.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:44 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$5q4aNNfuW3dh1YtInshrkeJ7K0/DSECz05tRhLd4UlPRYnj9VJNgO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725087704, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-31T07:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout:Fetching dashboard port number... 2024-08-31T07:01:46.245 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:45 smithi032 ceph-mon[17252]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:46.245 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:45 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:47.191 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:46 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4118708079' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-08-31T07:01:47.200 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 8443 2024-08-31T07:01:47.219 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-31T07:01:47.219 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-08-31T07:01:47.232 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-31T07:01:47.232 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-08-31T07:01:47.233 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-08-31T07:01:47.233 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-31T07:01:47.235 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph Dashboard is now available at: 2024-08-31T07:01:47.235 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:01:47.235 INFO:teuthology.orchestra.run.smithi032.stdout: URL: https://smithi032.front.sepia.ceph.com:8443/ 2024-08-31T07:01:47.235 INFO:teuthology.orchestra.run.smithi032.stdout: User: admin 2024-08-31T07:01:47.235 INFO:teuthology.orchestra.run.smithi032.stdout: Password: zvdf1ow2d9 2024-08-31T07:01:47.235 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:01:47.236 INFO:teuthology.orchestra.run.smithi032.stdout:Saving cluster configuration to /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config directory 2024-08-31T07:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-31T07:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-31T07:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-31T07:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:Or, if you are only running a single cluster on this host: 2024-08-31T07:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: ceph telemetry on 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:For more information see: 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:01:48.832 INFO:teuthology.orchestra.run.smithi032.stdout:Bootstrap complete. 2024-08-31T07:01:48.894 INFO:tasks.cephadm:Fetching config... 2024-08-31T07:01:48.894 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:01:48.894 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-31T07:01:48.932 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-31T07:01:48.932 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:01:48.932 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-31T07:01:49.009 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-31T07:01:49.010 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:01:49.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-31T07:01:49.098 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:48 smithi032 ceph-mon[17252]: mgrmap e13: x(active, since 12s) 2024-08-31T07:01:49.098 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:48 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2656078166' entity='client.admin' 2024-08-31T07:01:49.106 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-31T07:01:49.107 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:01:49.107 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-31T07:01:49.182 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-31T07:01:49.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfqbAmmvCdA/isadVM9EzhIcxuCgkCq1SNklmcIcAtsb//y+vFRcJOjDqSFVIibEGkLThYfGnuCS62WFl2C1YAuJ96LlhHF8aAODTpqWfKy1flnsAvAl8FsToGpUoQQFQOkeJlj21Kkb0/pXvnTml6urO/wq9qaxWgwso1eKyoSX09Q8HReHcQbUip8TRV6888056IdI1oED0h9o38TPP+vLctSVcbTRe3Xgw3580njS5S7iPXGxG3ba/FiPnPsz8UBtV01DRJAZHpHBwj35tROG9hAao9EbtyNwien3WtBugpkPqzBXM65S2wdYw+pzZBYV9AIP/AZDQI5FiEvPcHm3/igielQR2cOWaBWxqPhxYZRxXVWVoXFVFVIc7xRE1pcc6l9nn2BN5gWYgksm4i/UpVAF+ENltjR5cb61LrZgCheP81piem+djiuInGmR8TrbllGn2SHqbPNP46RaXTU7i1wW4d5BTf9lqN4gJtuNI46l4QlHlsVr1OWEpa4kk= ceph-965fc21a-6766-11ef-bcd4-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-31T07:01:49.302 INFO:teuthology.orchestra.run.smithi032.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfqbAmmvCdA/isadVM9EzhIcxuCgkCq1SNklmcIcAtsb//y+vFRcJOjDqSFVIibEGkLThYfGnuCS62WFl2C1YAuJ96LlhHF8aAODTpqWfKy1flnsAvAl8FsToGpUoQQFQOkeJlj21Kkb0/pXvnTml6urO/wq9qaxWgwso1eKyoSX09Q8HReHcQbUip8TRV6888056IdI1oED0h9o38TPP+vLctSVcbTRe3Xgw3580njS5S7iPXGxG3ba/FiPnPsz8UBtV01DRJAZHpHBwj35tROG9hAao9EbtyNwien3WtBugpkPqzBXM65S2wdYw+pzZBYV9AIP/AZDQI5FiEvPcHm3/igielQR2cOWaBWxqPhxYZRxXVWVoXFVFVIc7xRE1pcc6l9nn2BN5gWYgksm4i/UpVAF+ENltjR5cb61LrZgCheP81piem+djiuInGmR8TrbllGn2SHqbPNP46RaXTU7i1wW4d5BTf9lqN4gJtuNI46l4QlHlsVr1OWEpa4kk= ceph-965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:01:49.327 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfqbAmmvCdA/isadVM9EzhIcxuCgkCq1SNklmcIcAtsb//y+vFRcJOjDqSFVIibEGkLThYfGnuCS62WFl2C1YAuJ96LlhHF8aAODTpqWfKy1flnsAvAl8FsToGpUoQQFQOkeJlj21Kkb0/pXvnTml6urO/wq9qaxWgwso1eKyoSX09Q8HReHcQbUip8TRV6888056IdI1oED0h9o38TPP+vLctSVcbTRe3Xgw3580njS5S7iPXGxG3ba/FiPnPsz8UBtV01DRJAZHpHBwj35tROG9hAao9EbtyNwien3WtBugpkPqzBXM65S2wdYw+pzZBYV9AIP/AZDQI5FiEvPcHm3/igielQR2cOWaBWxqPhxYZRxXVWVoXFVFVIc7xRE1pcc6l9nn2BN5gWYgksm4i/UpVAF+ENltjR5cb61LrZgCheP81piem+djiuInGmR8TrbllGn2SHqbPNP46RaXTU7i1wW4d5BTf9lqN4gJtuNI46l4QlHlsVr1OWEpa4kk= ceph-965fc21a-6766-11ef-bcd4-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-31T07:01:49.371 INFO:teuthology.orchestra.run.smithi073.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfqbAmmvCdA/isadVM9EzhIcxuCgkCq1SNklmcIcAtsb//y+vFRcJOjDqSFVIibEGkLThYfGnuCS62WFl2C1YAuJ96LlhHF8aAODTpqWfKy1flnsAvAl8FsToGpUoQQFQOkeJlj21Kkb0/pXvnTml6urO/wq9qaxWgwso1eKyoSX09Q8HReHcQbUip8TRV6888056IdI1oED0h9o38TPP+vLctSVcbTRe3Xgw3580njS5S7iPXGxG3ba/FiPnPsz8UBtV01DRJAZHpHBwj35tROG9hAao9EbtyNwien3WtBugpkPqzBXM65S2wdYw+pzZBYV9AIP/AZDQI5FiEvPcHm3/igielQR2cOWaBWxqPhxYZRxXVWVoXFVFVIc7xRE1pcc6l9nn2BN5gWYgksm4i/UpVAF+ENltjR5cb61LrZgCheP81piem+djiuInGmR8TrbllGn2SHqbPNP46RaXTU7i1wW4d5BTf9lqN4gJtuNI46l4QlHlsVr1OWEpa4kk= ceph-965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:01:49.384 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfqbAmmvCdA/isadVM9EzhIcxuCgkCq1SNklmcIcAtsb//y+vFRcJOjDqSFVIibEGkLThYfGnuCS62WFl2C1YAuJ96LlhHF8aAODTpqWfKy1flnsAvAl8FsToGpUoQQFQOkeJlj21Kkb0/pXvnTml6urO/wq9qaxWgwso1eKyoSX09Q8HReHcQbUip8TRV6888056IdI1oED0h9o38TPP+vLctSVcbTRe3Xgw3580njS5S7iPXGxG3ba/FiPnPsz8UBtV01DRJAZHpHBwj35tROG9hAao9EbtyNwien3WtBugpkPqzBXM65S2wdYw+pzZBYV9AIP/AZDQI5FiEvPcHm3/igielQR2cOWaBWxqPhxYZRxXVWVoXFVFVIc7xRE1pcc6l9nn2BN5gWYgksm4i/UpVAF+ENltjR5cb61LrZgCheP81piem+djiuInGmR8TrbllGn2SHqbPNP46RaXTU7i1wW4d5BTf9lqN4gJtuNI46l4QlHlsVr1OWEpa4kk= ceph-965fc21a-6766-11ef-bcd4-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-31T07:01:49.427 INFO:teuthology.orchestra.run.smithi084.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfqbAmmvCdA/isadVM9EzhIcxuCgkCq1SNklmcIcAtsb//y+vFRcJOjDqSFVIibEGkLThYfGnuCS62WFl2C1YAuJ96LlhHF8aAODTpqWfKy1flnsAvAl8FsToGpUoQQFQOkeJlj21Kkb0/pXvnTml6urO/wq9qaxWgwso1eKyoSX09Q8HReHcQbUip8TRV6888056IdI1oED0h9o38TPP+vLctSVcbTRe3Xgw3580njS5S7iPXGxG3ba/FiPnPsz8UBtV01DRJAZHpHBwj35tROG9hAao9EbtyNwien3WtBugpkPqzBXM65S2wdYw+pzZBYV9AIP/AZDQI5FiEvPcHm3/igielQR2cOWaBWxqPhxYZRxXVWVoXFVFVIc7xRE1pcc6l9nn2BN5gWYgksm4i/UpVAF+ENltjR5cb61LrZgCheP81piem+djiuInGmR8TrbllGn2SHqbPNP46RaXTU7i1wW4d5BTf9lqN4gJtuNI46l4QlHlsVr1OWEpa4kk= ceph-965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:01:49.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-31T07:01:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:01:51.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:51 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/177154866' entity='client.admin' 2024-08-31T07:01:51.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:51 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:51.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:51 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:51.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:51 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"} : dispatch 2024-08-31T07:01:51.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:51 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:51.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:51 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:01:51.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:51 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:51.818 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-31T07:01:51.818 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-31T07:01:52.347 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:01:54.154 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi073 2024-08-31T07:01:54.154 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T07:01:54.154 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/etc/ceph/ceph.conf 2024-08-31T07:01:54.173 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T07:01:54.173 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:01:54.230 INFO:tasks.cephadm:Adding host smithi073 to orchestrator... 2024-08-31T07:01:54.231 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch host add smithi073 2024-08-31T07:01:54.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:54 smithi032 ceph-mon[17252]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:54.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:54 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:54.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:54 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:01:54.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:54 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:01:54.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:54 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:01:54.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:54 smithi032 ceph-mon[17252]: Updating smithi032:/etc/ceph/ceph.conf 2024-08-31T07:01:54.799 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:01:55.321 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:55 smithi032 ceph-mon[17252]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:01:55.322 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:55 smithi032 ceph-mon[17252]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:01:55.322 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:55 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:55.322 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:55 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:55.322 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:55 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:01:56.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:56 smithi032 ceph-mon[17252]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:01:57.545 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:57 smithi032 ceph-mon[17252]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:01:57.546 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:57 smithi032 ceph-mon[17252]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi073", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:01:58.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:58 smithi032 ceph-mon[17252]: Deploying cephadm binary to smithi073 2024-08-31T07:01:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:01:59 smithi032 ceph-mon[17252]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:01:59.707 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi073' with addr '172.21.15.73' 2024-08-31T07:02:00.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch host ls --format=json 2024-08-31T07:02:00.704 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:01.000 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:00 smithi032 ceph-mon[17252]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:01.000 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:00 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:01.000 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:00 smithi032 ceph-mon[17252]: Added host smithi073 2024-08-31T07:02:01.000 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:00 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:01.001 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:00 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:01.001 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:00 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:01.688 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:02:01.688 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.73", "hostname": "smithi073", "labels": [], "status": ""}] 2024-08-31T07:02:02.266 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:02 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:02.305 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi084 2024-08-31T07:02:02.305 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T07:02:02.306 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/etc/ceph/ceph.conf 2024-08-31T07:02:02.325 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T07:02:02.325 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:02.383 INFO:tasks.cephadm:Adding host smithi084 to orchestrator... 2024-08-31T07:02:02.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch host add smithi084 2024-08-31T07:02:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:03.136 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:03 smithi032 ceph-mon[17252]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:03.136 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:03 smithi032 ceph-mon[17252]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: Updating smithi073:/etc/ceph/ceph.conf 2024-08-31T07:02:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:05.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:05.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:05.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:05.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:04 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:06.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:05 smithi032 ceph-mon[17252]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:06.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:05 smithi032 ceph-mon[17252]: Deploying cephadm binary to smithi084 2024-08-31T07:02:07.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:06 smithi032 ceph-mon[17252]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:07.414 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi084' with addr '172.21.15.84' 2024-08-31T07:02:08.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch host ls --format=json 2024-08-31T07:02:08.486 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:08.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:08 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:08.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:08 smithi032 ceph-mon[17252]: Added host smithi084 2024-08-31T07:02:08.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:08 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:08.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:08 smithi032 ceph-mon[17252]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:08.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:08 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:08.691 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:08 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:09.333 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:02:09.333 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.73", "hostname": "smithi073", "labels": [], "status": ""}, {"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}] 2024-08-31T07:02:10.005 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:09 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:10.006 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:09 smithi032 ceph-mon[17252]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:02:10.046 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-31T07:02:10.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd crush tunables default 2024-08-31T07:02:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:10.816 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:10 smithi032 ceph-mon[17252]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-08-31T07:02:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:11 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2948515097' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-31T07:02:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:11 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:11 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:11 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:11 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:11 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:11 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:11 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:12.515 INFO:tasks.cephadm:Adding mon.a on smithi032 2024-08-31T07:02:12.515 INFO:tasks.cephadm:Adding mon.b on smithi073 2024-08-31T07:02:12.515 INFO:tasks.cephadm:Adding mon.c on smithi084 2024-08-31T07:02:12.516 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch apply mon '3;smithi032:172.21.15.32=a;smithi073:172.21.15.73=b;smithi084:172.21.15.84=c' 2024-08-31T07:02:12.752 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:12 smithi032 ceph-mon[17252]: Updating smithi084:/etc/ceph/ceph.conf 2024-08-31T07:02:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:12 smithi032 ceph-mon[17252]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:12 smithi032 ceph-mon[17252]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:12 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2948515097' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-31T07:02:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:12 smithi032 ceph-mon[17252]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T07:02:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:12 smithi032 ceph-mon[17252]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:12 smithi032 ceph-mon[17252]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:13.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:12 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:13.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:12 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:13.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:12 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:13.352 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled mon update... 2024-08-31T07:02:13.974 DEBUG:teuthology.orchestra.run.smithi073:mon.b> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.b.service 2024-08-31T07:02:13.977 DEBUG:teuthology.orchestra.run.smithi084:mon.c> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.c.service 2024-08-31T07:02:13.980 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-31T07:02:13.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-31T07:02:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi073:172.21.15.73=b;smithi084:172.21.15.84=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:02:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi073:172.21.15.73=b;smithi084:172.21.15.84=c;count:3 2024-08-31T07:02:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: Deploying daemon mon.c on smithi084 2024-08-31T07:02:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:14 smithi032 ceph-mon[17252]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:15.357 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T07:02:15.357 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","modified":"2024-08-31T07:00:36.785570Z","created":"2024-08-31T07:00:36.785570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-31T07:02:15.358 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-08-31T07:02:15.629 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:15 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/3133879749' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:16.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:16 smithi032 ceph-mon[17252]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:18.301 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-31T07:02:18.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-31T07:02:18.553 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:02:18.674 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:18 smithi084 systemd[1]: Started Ceph mon.c for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:02:23.745 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:23.745 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:02:23.745 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: mon.a calling monitor election 2024-08-31T07:02:23.745 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.745 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.745 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: mon.c calling monitor election 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: monmap epoch 2 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: last_changed 2024-08-31T07:02:18.329790+0000 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: created 2024-08-31T07:00:36.785570+0000 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: min_mon_release 19 (squid) 2024-08-31T07:02:23.746 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: election_strategy: 1 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.c 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: fsmap 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: mgrmap e13: x(active, since 47s) 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: overall HEALTH_OK 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:23.747 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:23 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:23.748 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T07:02:23.748 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":2,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","modified":"2024-08-31T07:02:18.329790Z","created":"2024-08-31T07:00:36.785570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-31T07:02:23.748 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 2 2024-08-31T07:02:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:02:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: mon.a calling monitor election 2024-08-31T07:02:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: mon.c calling monitor election 2024-08-31T07:02:23.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:23.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:23.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-31T07:02:23.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: monmap epoch 2 2024-08-31T07:02:23.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:02:23.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: last_changed 2024-08-31T07:02:18.329790+0000 2024-08-31T07:02:23.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: created 2024-08-31T07:00:36.785570+0000 2024-08-31T07:02:23.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: min_mon_release 19 (squid) 2024-08-31T07:02:23.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: election_strategy: 1 2024-08-31T07:02:23.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-08-31T07:02:23.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.c 2024-08-31T07:02:23.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: fsmap 2024-08-31T07:02:23.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T07:02:23.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: mgrmap e13: x(active, since 47s) 2024-08-31T07:02:23.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: overall HEALTH_OK 2024-08-31T07:02:23.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:23.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:23.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:23.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:23.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:23 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:24 smithi084 ceph-mon[22387]: Deploying daemon mon.b on smithi073 2024-08-31T07:02:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:24 smithi084 ceph-mon[22387]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:24 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/1695271264' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:24 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:24.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:24 smithi032 ceph-mon[17252]: Deploying daemon mon.b on smithi073 2024-08-31T07:02:24.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:24 smithi032 ceph-mon[17252]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:24.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:24 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/1695271264' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:24.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:24 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:25.327 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-31T07:02:25.327 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-31T07:02:25.579 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:02:26.259 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T07:02:26.259 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":2,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","modified":"2024-08-31T07:02:18.329790Z","created":"2024-08-31T07:00:36.785570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-31T07:02:26.259 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 2 2024-08-31T07:02:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:26 smithi084 ceph-mon[22387]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:26 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/3332893201' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:27.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-31T07:02:27.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-31T07:02:27.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-31T07:02:27.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-31T07:02:27.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-31T07:02:27.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-31T07:02:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:26 smithi032 ceph-mon[17252]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:26 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/3332893201' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:27.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-31T07:02:27.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T07:02:27.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T07:02:27.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T07:02:27.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:02:27.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:02:27.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Updating smithi073:/etc/ceph/ceph.conf 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Deploying cephadm binary to smithi084 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Added host smithi084 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2948515097' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Updating smithi084:/etc/ceph/ceph.conf 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2948515097' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T07:02:27.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi073:172.21.15.73=b;smithi084:172.21.15.84=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi073:172.21.15.73=b;smithi084:172.21.15.84=c;count:3 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:27.434 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Deploying daemon mon.c on smithi084 2024-08-31T07:02:27.435 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.435 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/3133879749' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:27.435 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.435 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.435 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:02:27.435 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.a calling monitor election 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.c calling monitor election 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: monmap epoch 2 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:02:27.436 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: last_changed 2024-08-31T07:02:18.329790+0000 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: created 2024-08-31T07:00:36.785570+0000 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: min_mon_release 19 (squid) 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: election_strategy: 1 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.c 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: fsmap 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mgrmap e13: x(active, since 47s) 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: overall HEALTH_OK 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:27.437 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:27.438 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: Deploying daemon mon.b on smithi073 2024-08-31T07:02:27.438 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.438 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/1695271264' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:27.438 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:27.438 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:27.438 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/3332893201' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:27.438 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 ceph-mon[21995]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-31T07:02:27.438 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 bash[21977]: 6548fef870e27d3b2b7b8a2a862ba7b533661c7ae2beff6312937c9965856047 2024-08-31T07:02:27.438 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:27 smithi073 systemd[1]: Started Ceph mon.b for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:02:27.857 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-31T07:02:27.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-31T07:02:28.109 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:02:32.390 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T07:02:32.390 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":3,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","modified":"2024-08-31T07:02:27.071714Z","created":"2024-08-31T07:00:36.785570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:3300","nonce":0},{"type":"v1","addr":"172.21.15.73:6789","nonce":0}]},"addr":"172.21.15.73:6789/0","public_addr":"172.21.15.73:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-08-31T07:02:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 3 2024-08-31T07:02:32.403 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:02:32.403 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: mon.a calling monitor election 2024-08-31T07:02:32.403 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.403 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:32.403 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: mon.c calling monitor election 2024-08-31T07:02:32.403 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:32.403 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.403 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.403 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: mon.b calling monitor election 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: monmap epoch 3 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: last_changed 2024-08-31T07:02:27.071714+0000 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: created 2024-08-31T07:00:36.785570+0000 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: min_mon_release 19 (squid) 2024-08-31T07:02:32.404 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: election_strategy: 1 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.c 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: fsmap 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: mgrmap e13: x(active, since 56s) 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: overall HEALTH_OK 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.405 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:32 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: mon.a calling monitor election 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: mon.c calling monitor election 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: mon.b calling monitor election 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: monmap epoch 3 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: last_changed 2024-08-31T07:02:27.071714+0000 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: created 2024-08-31T07:00:36.785570+0000 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: min_mon_release 19 (squid) 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: election_strategy: 1 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.c 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: fsmap 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T07:02:32.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: mgrmap e13: x(active, since 56s) 2024-08-31T07:02:32.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: overall HEALTH_OK 2024-08-31T07:02:32.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:32 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:02:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: mon.a calling monitor election 2024-08-31T07:02:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: mon.c calling monitor election 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: mon.b calling monitor election 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: monmap epoch 3 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:02:32.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: last_changed 2024-08-31T07:02:27.071714+0000 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: created 2024-08-31T07:00:36.785570+0000 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: min_mon_release 19 (squid) 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: election_strategy: 1 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.c 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: fsmap 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: mgrmap e13: x(active, since 56s) 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: overall HEALTH_OK 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.611 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:32.611 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:32 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:33.137 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-31T07:02:33.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config generate-minimal-conf 2024-08-31T07:02:33.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:33 smithi073 ceph-mon[21995]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:33.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:33 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:33.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:33 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/6692704' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:33.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:33 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:33.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:33 smithi084 ceph-mon[22387]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:33.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:33 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:33.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:33 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/6692704' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:33 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:33.519 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:33.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:33 smithi032 ceph-mon[17252]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:33 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:33 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/6692704' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T07:02:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:33 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:34.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:34 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:34 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:34 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:34 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:34 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:34.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:34 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:34.381 INFO:teuthology.orchestra.run.smithi032.stdout:# minimal ceph.conf for 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:02:34.381 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-08-31T07:02:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:02:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: mon_host = [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] 2024-08-31T07:02:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:34 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:34 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:34 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:34 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:34 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:34 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:34 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:34 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:34 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:34 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:34 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:34 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:35.105 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-31T07:02:35.105 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:02:35.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.conf 2024-08-31T07:02:35.165 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:02:35.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:35.263 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T07:02:35.264 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.conf 2024-08-31T07:02:35.295 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T07:02:35.296 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:35.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:35.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: Updating smithi032:/etc/ceph/ceph.conf 2024-08-31T07:02:35.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: Updating smithi073:/etc/ceph/ceph.conf 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: Updating smithi084:/etc/ceph/ceph.conf 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1734007758' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:35.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:02:35.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:35 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:35.369 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T07:02:35.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.conf 2024-08-31T07:02:35.403 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T07:02:35.403 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:35.481 INFO:tasks.cephadm:enabling module snap_schedule 2024-08-31T07:02:35.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-08-31T07:02:35.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:35.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: Updating smithi032:/etc/ceph/ceph.conf 2024-08-31T07:02:35.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: Updating smithi073:/etc/ceph/ceph.conf 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: Updating smithi084:/etc/ceph/ceph.conf 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1734007758' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:35.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:02:35.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:35 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:35.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: Updating smithi032:/etc/ceph/ceph.conf 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: Updating smithi073:/etc/ceph/ceph.conf 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: Updating smithi084:/etc/ceph/ceph.conf 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1734007758' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:35.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:35.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:02:35.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:35 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:36 smithi032 ceph-mon[17252]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:36 smithi032 ceph-mon[17252]: Reconfiguring mon.a (unknown last config time)... 2024-08-31T07:02:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:36 smithi032 ceph-mon[17252]: Reconfiguring daemon mon.a on smithi032 2024-08-31T07:02:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:36 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:36 smithi073 ceph-mon[21995]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:36 smithi073 ceph-mon[21995]: Reconfiguring mon.a (unknown last config time)... 2024-08-31T07:02:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:36 smithi073 ceph-mon[21995]: Reconfiguring daemon mon.a on smithi032 2024-08-31T07:02:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:36 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:36 smithi084 ceph-mon[22387]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:36 smithi084 ceph-mon[22387]: Reconfiguring mon.a (unknown last config time)... 2024-08-31T07:02:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:36 smithi084 ceph-mon[22387]: Reconfiguring daemon mon.a on smithi032 2024-08-31T07:02:36.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:36 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:37.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:37 smithi073 ceph-mon[21995]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:37.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:37 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:37.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:37 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:37.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:37 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:37.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:37 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:02:37.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:37 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:37.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:37 smithi032 ceph-mon[17252]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:37.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:37 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:37.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:37 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:37.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:37 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:37.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:37 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:02:37.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:37 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:37 smithi084 ceph-mon[22387]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:02:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:37 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:37 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:37 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:37 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:02:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:37 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:38.599 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:38 smithi084 ceph-mon[22387]: Reconfiguring mon.b (monmap changed)... 2024-08-31T07:02:38.599 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:38 smithi084 ceph-mon[22387]: Reconfiguring daemon mon.b on smithi073 2024-08-31T07:02:38.599 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:38 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3158256382' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-31T07:02:38.599 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:38 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:38.599 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:38 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:38.600 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:38 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:38.600 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:38 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:02:38.600 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:38 smithi084 ceph-mon[22387]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:38.607 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:38 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: ignoring --setuser ceph since I am not root 2024-08-31T07:02:38.608 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:38 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: ignoring --setgroup ceph since I am not root 2024-08-31T07:02:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:38 smithi032 ceph-mon[17252]: Reconfiguring mon.b (monmap changed)... 2024-08-31T07:02:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:38 smithi032 ceph-mon[17252]: Reconfiguring daemon mon.b on smithi073 2024-08-31T07:02:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:38 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3158256382' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-31T07:02:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:38 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:38 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:38 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:38 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:02:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:38 smithi032 ceph-mon[17252]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:38 smithi073 ceph-mon[21995]: Reconfiguring mon.b (monmap changed)... 2024-08-31T07:02:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:38 smithi073 ceph-mon[21995]: Reconfiguring daemon mon.b on smithi073 2024-08-31T07:02:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:38 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3158256382' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-31T07:02:38.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:38 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:38.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:38 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' 2024-08-31T07:02:38.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:38 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:02:38.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:38 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:02:38.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:38 smithi073 ceph-mon[21995]: from='mgr.14150 172.21.15.32:0/1637890449' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:39.067 INFO:tasks.cephadm:Adding mgr.x on smithi032 2024-08-31T07:02:39.067 INFO:tasks.cephadm:Adding mgr.y on smithi073 2024-08-31T07:02:39.067 INFO:tasks.cephadm:Adding mgr.z on smithi084 2024-08-31T07:02:39.068 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch apply mgr '3;smithi032=x;smithi073=y;smithi084=z' 2024-08-31T07:02:39.360 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:39 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3158256382' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-31T07:02:39.360 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:39 smithi084 ceph-mon[22387]: mgrmap e14: x(active, since 62s) 2024-08-31T07:02:39.439 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:02:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:39 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3158256382' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-31T07:02:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:39 smithi032 ceph-mon[17252]: mgrmap e14: x(active, since 62s) 2024-08-31T07:02:39.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:39 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3158256382' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-31T07:02:39.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:39 smithi073 ceph-mon[21995]: mgrmap e14: x(active, since 62s) 2024-08-31T07:02:44.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: Active manager daemon x restarted 2024-08-31T07:02:44.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: Activating manager daemon x 2024-08-31T07:02:44.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: osdmap e5: 0 total, 0 up, 0 in 2024-08-31T07:02:44.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: mgrmap e15: x(active, starting, since 0.00570764s) 2024-08-31T07:02:44.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:02:44.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:44.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:44.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T07:02:44.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T07:02:44.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T07:02:44.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T07:02:44.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:44 smithi084 ceph-mon[22387]: Manager daemon x is now available 2024-08-31T07:02:45.014 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: Active manager daemon x restarted 2024-08-31T07:02:45.014 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: Activating manager daemon x 2024-08-31T07:02:45.014 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: osdmap e5: 0 total, 0 up, 0 in 2024-08-31T07:02:45.014 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: mgrmap e15: x(active, starting, since 0.00570764s) 2024-08-31T07:02:45.014 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:02:45.014 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:45.014 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:45.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T07:02:45.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T07:02:45.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T07:02:45.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T07:02:45.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:44 smithi032 ceph-mon[17252]: Manager daemon x is now available 2024-08-31T07:02:45.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: Active manager daemon x restarted 2024-08-31T07:02:45.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: Activating manager daemon x 2024-08-31T07:02:45.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: osdmap e5: 0 total, 0 up, 0 in 2024-08-31T07:02:45.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: mgrmap e15: x(active, starting, since 0.00570764s) 2024-08-31T07:02:45.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:02:45.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:02:45.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:02:45.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T07:02:45.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T07:02:45.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T07:02:45.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T07:02:45.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:44 smithi073 ceph-mon[21995]: Manager daemon x is now available 2024-08-31T07:02:45.691 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled mgr update... 2024-08-31T07:02:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T07:02:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T07:02:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:45 smithi084 ceph-mon[22387]: mgrmap e16: x(active, since 1.01311s) 2024-08-31T07:02:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:46.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:46.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T07:02:46.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T07:02:46.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:45 smithi032 ceph-mon[17252]: mgrmap e16: x(active, since 1.01311s) 2024-08-31T07:02:46.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:46.113 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:46.113 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T07:02:46.113 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T07:02:46.113 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:45 smithi073 ceph-mon[21995]: mgrmap e16: x(active, since 1.01311s) 2024-08-31T07:02:46.113 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:46.285 DEBUG:teuthology.orchestra.run.smithi073:mgr.y> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.y.service 2024-08-31T07:02:46.288 DEBUG:teuthology.orchestra.run.smithi084:mgr.z> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.z.service 2024-08-31T07:02:46.291 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-08-31T07:02:46.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-08-31T07:02:46.295 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-08-31T07:02:46.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set client debug_client 20 2024-08-31T07:02:46.340 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-08-31T07:02:46.340 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set client debug_ms 1 2024-08-31T07:02:46.387 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-08-31T07:02:46.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-08-31T07:02:46.434 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-08-31T07:02:46.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-08-31T07:02:46.485 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-08-31T07:02:46.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds debug_mds 20 2024-08-31T07:02:46.531 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-08-31T07:02:46.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-08-31T07:02:46.581 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-08-31T07:02:46.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds debug_ms 2 2024-08-31T07:02:46.609 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:02:46.349+0000 7f5f24c00640 -1 log_channel(cephadm) log [ERR] : [31/Aug/2024:07:02:46] ENGINE Error in HTTPServer.serve 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: Traceback (most recent call last): 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self._connections.run(self.expiration_interval) 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self._run(expiration_interval) 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: s = self.context.wrap_socket( 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T07:02:46.610 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: return self.sslsocket_class._create( 2024-08-31T07:02:46.611 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T07:02:46.611 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self.do_handshake() 2024-08-31T07:02:46.611 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T07:02:46.611 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: self._sslobj.do_handshake() 2024-08-31T07:02:46.611 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T07:02:46.611 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:02:46.619 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-08-31T07:02:46.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-08-31T07:02:46.672 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-08-31T07:02:46.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-08-31T07:02:46.726 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-08-31T07:02:46.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-08-31T07:02:46.781 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-08-31T07:02:46.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-08-31T07:02:46.836 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-08-31T07:02:46.836 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-08-31T07:02:46.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-mon[17252]: Saving service mgr spec with placement smithi032=x;smithi073=y;smithi084=z;count:3 2024-08-31T07:02:46.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-mon[17252]: [31/Aug/2024:07:02:46] ENGINE Bus STARTING 2024-08-31T07:02:46.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-mon[17252]: [31/Aug/2024:07:02:46] ENGINE Serving on https://172.21.15.32:7150 2024-08-31T07:02:46.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-mon[17252]: [31/Aug/2024:07:02:46] ENGINE Error in HTTPServer.serve 2024-08-31T07:02:46.860 INFO:journalctl@ceph.mon.a.smithi032.stdout: Traceback (most recent call last): 2024-08-31T07:02:46.860 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: self._connections.run(self.expiration_interval) 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: self._run(expiration_interval) 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: s = self.context.wrap_socket( 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: return self.sslsocket_class._create( 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: self.do_handshake() 2024-08-31T07:02:46.861 INFO:journalctl@ceph.mon.a.smithi032.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T07:02:46.862 INFO:journalctl@ceph.mon.a.smithi032.stdout: self._sslobj.do_handshake() 2024-08-31T07:02:46.862 INFO:journalctl@ceph.mon.a.smithi032.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T07:02:46.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-mon[17252]: [31/Aug/2024:07:02:46] ENGINE Serving on http://172.21.15.32:8765 2024-08-31T07:02:46.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-mon[17252]: [31/Aug/2024:07:02:46] ENGINE Bus STARTED 2024-08-31T07:02:46.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:46.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:46.878 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-08-31T07:02:46.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-08-31T07:02:46.930 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-08-31T07:02:46.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-08-31T07:02:46.984 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-08-31T07:02:46.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-08-31T07:02:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:46 smithi084 ceph-mon[22387]: Saving service mgr spec with placement smithi032=x;smithi073=y;smithi084=z;count:3 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:46 smithi084 ceph-mon[22387]: [31/Aug/2024:07:02:46] ENGINE Bus STARTING 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:46 smithi084 ceph-mon[22387]: [31/Aug/2024:07:02:46] ENGINE Serving on https://172.21.15.32:7150 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:46 smithi084 ceph-mon[22387]: [31/Aug/2024:07:02:46] ENGINE Error in HTTPServer.serve 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout: Traceback (most recent call last): 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout: self._connections.run(self.expiration_interval) 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout: self._run(expiration_interval) 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T07:02:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout: s = self.context.wrap_socket( 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout: return self.sslsocket_class._create( 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout: self.do_handshake() 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout: self._sslobj.do_handshake() 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:46 smithi084 ceph-mon[22387]: [31/Aug/2024:07:02:46] ENGINE Serving on http://172.21.15.32:8765 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:46 smithi084 ceph-mon[22387]: [31/Aug/2024:07:02:46] ENGINE Bus STARTED 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:46.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:47.036 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-08-31T07:02:47.036 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-08-31T07:02:47.096 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-08-31T07:02:47.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-08-31T07:02:47.154 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-08-31T07:02:47.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-08-31T07:02:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:46 smithi073 ceph-mon[21995]: Saving service mgr spec with placement smithi032=x;smithi073=y;smithi084=z;count:3 2024-08-31T07:02:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:46 smithi073 ceph-mon[21995]: [31/Aug/2024:07:02:46] ENGINE Bus STARTING 2024-08-31T07:02:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:46 smithi073 ceph-mon[21995]: [31/Aug/2024:07:02:46] ENGINE Serving on https://172.21.15.32:7150 2024-08-31T07:02:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:46 smithi073 ceph-mon[21995]: [31/Aug/2024:07:02:46] ENGINE Error in HTTPServer.serve 2024-08-31T07:02:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout: Traceback (most recent call last): 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: self._connections.run(self.expiration_interval) 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: self._run(expiration_interval) 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: s = self.context.wrap_socket( 2024-08-31T07:02:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T07:02:47.181 INFO:journalctl@ceph.mon.b.smithi073.stdout: return self.sslsocket_class._create( 2024-08-31T07:02:47.181 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T07:02:47.181 INFO:journalctl@ceph.mon.b.smithi073.stdout: self.do_handshake() 2024-08-31T07:02:47.181 INFO:journalctl@ceph.mon.b.smithi073.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T07:02:47.181 INFO:journalctl@ceph.mon.b.smithi073.stdout: self._sslobj.do_handshake() 2024-08-31T07:02:47.181 INFO:journalctl@ceph.mon.b.smithi073.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T07:02:47.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:46 smithi073 ceph-mon[21995]: [31/Aug/2024:07:02:46] ENGINE Serving on http://172.21.15.32:8765 2024-08-31T07:02:47.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:46 smithi073 ceph-mon[21995]: [31/Aug/2024:07:02:46] ENGINE Bus STARTED 2024-08-31T07:02:47.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:47.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:47.208 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-08-31T07:02:47.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-08-31T07:02:47.209 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:47.263 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-08-31T07:02:47.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-08-31T07:02:47.335 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-08-31T07:02:47.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-08-31T07:02:47.401 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-08-31T07:02:47.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-08-31T07:02:47.460 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-08-31T07:02:47.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-08-31T07:02:47.460 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:47.522 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-08-31T07:02:47.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-08-31T07:02:47.587 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-08-31T07:02:47.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-08-31T07:02:47.662 DEBUG:tasks.cephadm:set 26 configs 2024-08-31T07:02:47.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi032') 2024-08-31T07:02:48.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:48.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: mgrmap e17: x(active, since 2s) 2024-08-31T07:02:48.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:48.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:48.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: mgrmap e17: x(active, since 2s) 2024-08-31T07:02:48.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:48.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:48.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: mgrmap e17: x(active, since 2s) 2024-08-31T07:02:48.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:48.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:50.051 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:49 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1539935945' entity='client.admin' 2024-08-31T07:02:50.051 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:49 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3060960947' entity='client.admin' 2024-08-31T07:02:50.054 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi032') 2024-08-31T07:02:50.110 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:50.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:49 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1539935945' entity='client.admin' 2024-08-31T07:02:50.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:49 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3060960947' entity='client.admin' 2024-08-31T07:02:50.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:49 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1539935945' entity='client.admin' 2024-08-31T07:02:50.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:49 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3060960947' entity='client.admin' 2024-08-31T07:02:51.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:51.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:51.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:51.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: Updating smithi032:/etc/ceph/ceph.conf 2024-08-31T07:02:51.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: Updating smithi073:/etc/ceph/ceph.conf 2024-08-31T07:02:51.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: Updating smithi084:/etc/ceph/ceph.conf 2024-08-31T07:02:51.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:51.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:51.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:51 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:51.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:51.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:51.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: Updating smithi032:/etc/ceph/ceph.conf 2024-08-31T07:02:51.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: Updating smithi073:/etc/ceph/ceph.conf 2024-08-31T07:02:51.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: Updating smithi084:/etc/ceph/ceph.conf 2024-08-31T07:02:51.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:51.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:51.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:51 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"} : dispatch 2024-08-31T07:02:51.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:51.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:51.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: Updating smithi032:/etc/ceph/ceph.conf 2024-08-31T07:02:51.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: Updating smithi073:/etc/ceph/ceph.conf 2024-08-31T07:02:51.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: Updating smithi084:/etc/ceph/ceph.conf 2024-08-31T07:02:51.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:51.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:51.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.493 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:51 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 -- 172.21.15.32:0/145593996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9d0c03f2d0 msgr2=0x7f9d0c0417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 --2- 172.21.15.32:0/145593996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9d0c03f2d0 0x7f9d0c0417e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009c90 tx=0x7f9d14005f90 comp rx=0 tx=0).stop 2024-08-31T07:02:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 -- 172.21.15.32:0/145593996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d2806c340 msgr2=0x7f9d2819be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 --2- 172.21.15.32:0/145593996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d2806c340 0x7f9d2819be00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9d1802a930 tx=0x7f9d18003b20 comp rx=0 tx=0).stop 2024-08-31T07:02:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 -- 172.21.15.32:0/145593996 shutdown_connections 2024-08-31T07:02:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 --2- 172.21.15.32:0/145593996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9d0c03f2d0 0x7f9d0c0417e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 --2- 172.21.15.32:0/145593996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d28109240 0x7f9d2819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 --2- 172.21.15.32:0/145593996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d28106700 0x7f9d2819c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 --2- 172.21.15.32:0/145593996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d2806c340 0x7f9d2819be00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 -- 172.21.15.32:0/145593996 >> 172.21.15.32:0/145593996 conn(0x7f9d280fdda0 msgr2=0x7f9d28104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 -- 172.21.15.32:0/145593996 shutdown_connections 2024-08-31T07:02:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:51.914+0000 7f9d2f000640 1 -- 172.21.15.32:0/145593996 wait complete. 2024-08-31T07:02:52.367 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:52.381 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:52 smithi032 ceph-mon[17252]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:52.381 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:52 smithi032 ceph-mon[17252]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:52.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:52 smithi032 ceph-mon[17252]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:52.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:52 smithi032 ceph-mon[17252]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:52.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:52 smithi032 ceph-mon[17252]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:52.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:52 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:52.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:52 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:02:52.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:52 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:52.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:52 smithi032 ceph-mon[17252]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:52.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:52 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:02:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:52 smithi073 ceph-mon[21995]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:52 smithi073 ceph-mon[21995]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:52.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:52 smithi073 ceph-mon[21995]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:52.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:52 smithi073 ceph-mon[21995]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:52.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:52 smithi073 ceph-mon[21995]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:52.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:52 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:52.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:52 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:02:52.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:52 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:52.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:52 smithi073 ceph-mon[21995]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:52.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:52 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:02:52.482 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi032') 2024-08-31T07:02:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:52 smithi084 ceph-mon[22387]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:52 smithi084 ceph-mon[22387]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:52 smithi084 ceph-mon[22387]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T07:02:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:52 smithi084 ceph-mon[22387]: Updating smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:52.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:52 smithi084 ceph-mon[22387]: Updating smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:52.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:52 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:52.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:52 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:02:52.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:52 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:52.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:52 smithi084 ceph-mon[22387]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-08-31T07:02:52.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:52 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:02:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.129+0000 7faefc600640 1 -- 172.21.15.32:0/2448700851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef4104580 msgr2=0x7faef4104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.129+0000 7faefc600640 1 --2- 172.21.15.32:0/2448700851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef4104580 0x7faef4104990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faee0009860 tx=0x7faee002a3d0 comp rx=0 tx=0).stop 2024-08-31T07:02:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.129+0000 7faefc600640 1 -- 172.21.15.32:0/2448700851 shutdown_connections 2024-08-31T07:02:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.129+0000 7faefc600640 1 --2- 172.21.15.32:0/2448700851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef4104f20 0x7faef410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.129+0000 7faefc600640 1 --2- 172.21.15.32:0/2448700851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef4104580 0x7faef4104990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.129+0000 7faefc600640 1 --2- 172.21.15.32:0/2448700851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef41034f0 0x7faef4103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.129+0000 7faefc600640 1 -- 172.21.15.32:0/2448700851 >> 172.21.15.32:0/2448700851 conn(0x7faef40fee80 msgr2=0x7faef4101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.129+0000 7faefc600640 1 -- 172.21.15.32:0/2448700851 shutdown_connections 2024-08-31T07:02:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.129+0000 7faefc600640 1 -- 172.21.15.32:0/2448700851 wait complete. 2024-08-31T07:02:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.130+0000 7faefc600640 1 Processor -- start 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.130+0000 7faefc600640 1 -- start start 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faefc600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef41034f0 0x7faef419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faefc600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4104580 0x7faef419c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faefc600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef4104f20 0x7faef419d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faefc600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faef410f030 con 0x7faef4104f20 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faefc600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faef410eeb0 con 0x7faef41034f0 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faefc600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faef410f1b0 con 0x7faef4104580 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faef9200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4104580 0x7faef419c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faef9200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4104580 0x7faef419c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52614/0 (socket says 172.21.15.32:52614) 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faef9200640 1 -- 172.21.15.32:0/2603375283 learned_addr learned my addr 172.21.15.32:0/2603375283 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faef9c00640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef41034f0 0x7faef419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faefa600640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef4104f20 0x7faef419d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faef9200640 1 -- 172.21.15.32:0/2603375283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef41034f0 msgr2=0x7faef419bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:52.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.131+0000 7faef9200640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef41034f0 0x7faef419bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:52.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.132+0000 7faef9200640 1 -- 172.21.15.32:0/2603375283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef4104f20 msgr2=0x7faef419d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:52.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.132+0000 7faef9200640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef4104f20 0x7faef419d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:52.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.132+0000 7faef9200640 1 -- 172.21.15.32:0/2603375283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef419d930 con 0x7faef4104580 2024-08-31T07:02:52.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.132+0000 7faef9c00640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef41034f0 0x7faef419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:02:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.132+0000 7faef9200640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4104580 0x7faef419c570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faee0000c00 tx=0x7faee0032f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.132+0000 7faef2000640 1 -- 172.21.15.32:0/2603375283 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0040070 con 0x7faef4104580 2024-08-31T07:02:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.132+0000 7faef2000640 1 -- 172.21.15.32:0/2603375283 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faee0031400 con 0x7faef4104580 2024-08-31T07:02:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.132+0000 7faefc600640 1 -- 172.21.15.32:0/2603375283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef419dbc0 con 0x7faef4104580 2024-08-31T07:02:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.132+0000 7faef2000640 1 -- 172.21.15.32:0/2603375283 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0004920 con 0x7faef4104580 2024-08-31T07:02:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.132+0000 7faefc600640 1 -- 172.21.15.32:0/2603375283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef41a4320 con 0x7faef4104580 2024-08-31T07:02:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.134+0000 7faef2000640 1 -- 172.21.15.32:0/2603375283 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7faee0043070 con 0x7faef4104580 2024-08-31T07:02:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.133+0000 7faefc600640 1 -- 172.21.15.32:0/2603375283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef4103960 con 0x7faef4104580 2024-08-31T07:02:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.134+0000 7faef2000640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faedc03f2d0 0x7faedc0417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.134+0000 7faef2000640 1 -- 172.21.15.32:0/2603375283 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7faee0074ab0 con 0x7faef4104580 2024-08-31T07:02:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.134+0000 7faef9c00640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faedc03f2d0 0x7faedc0417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.135+0000 7faef9c00640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faedc03f2d0 0x7faedc0417e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faee80096c0 tx=0x7faee8009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.139+0000 7faef2000640 1 -- 172.21.15.32:0/2603375283 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faee004de10 con 0x7faef4104580 2024-08-31T07:02:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.354+0000 7faefc600640 1 -- 172.21.15.32:0/2603375283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7faef41a48c0 con 0x7faef4104580 2024-08-31T07:02:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.370+0000 7faef2000640 1 -- 172.21.15.32:0/2603375283 <== mon.1 v2:172.21.15.84:3300/0 7 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faee007a080 con 0x7faef4104580 2024-08-31T07:02:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.370+0000 7faef2000640 1 -- 172.21.15.32:0/2603375283 <== mon.1 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7faee00315b0 con 0x7faef4104580 2024-08-31T07:02:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.379+0000 7faefc600640 1 -- 172.21.15.32:0/2603375283 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faedc03f2d0 msgr2=0x7faedc0417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.379+0000 7faefc600640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faedc03f2d0 0x7faedc0417e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faee80096c0 tx=0x7faee8009260 comp rx=0 tx=0).stop 2024-08-31T07:02:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.379+0000 7faefc600640 1 -- 172.21.15.32:0/2603375283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4104580 msgr2=0x7faef419c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.379+0000 7faefc600640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4104580 0x7faef419c570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faee0000c00 tx=0x7faee0032f90 comp rx=0 tx=0).stop 2024-08-31T07:02:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.382+0000 7faefc600640 1 -- 172.21.15.32:0/2603375283 shutdown_connections 2024-08-31T07:02:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.383+0000 7faefc600640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faedc03f2d0 0x7faedc0417e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.383+0000 7faefc600640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faef4104f20 0x7faef419d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.383+0000 7faefc600640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faef4104580 0x7faef419c570 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.383+0000 7faefc600640 1 --2- 172.21.15.32:0/2603375283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef41034f0 0x7faef419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.384+0000 7faefc600640 1 -- 172.21.15.32:0/2603375283 >> 172.21.15.32:0/2603375283 conn(0x7faef40fee80 msgr2=0x7faef4075390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.391+0000 7faefc600640 1 -- 172.21.15.32:0/2603375283 shutdown_connections 2024-08-31T07:02:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:52.393+0000 7faefc600640 1 -- 172.21.15.32:0/2603375283 wait complete. 2024-08-31T07:02:53.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:53 smithi032 ceph-mon[17252]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:53.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:53 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:53.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:53 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:53.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:53 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:53.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:53 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T07:02:53.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:53 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T07:02:53.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:53 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T07:02:53.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:53 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:53.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:53 smithi032 ceph-mon[17252]: Deploying daemon mgr.z on smithi084 2024-08-31T07:02:53.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:53 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:02:53.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:53 smithi084 ceph-mon[22387]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:53.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:53 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:53.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:53 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:53.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:53 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:53.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:53 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T07:02:53.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:53 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T07:02:53.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:53 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T07:02:53.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:53 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:53.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:53 smithi084 ceph-mon[22387]: Deploying daemon mgr.z on smithi084 2024-08-31T07:02:53.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:53 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:02:53.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:53 smithi073 ceph-mon[21995]: Updating smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T07:02:53.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:53 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:53.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:53 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:53.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:53 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:53.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:53 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T07:02:53.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:53 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T07:02:53.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:53 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T07:02:53.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:53 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:53.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:53 smithi073 ceph-mon[21995]: Deploying daemon mgr.z on smithi084 2024-08-31T07:02:53.601 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:53 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:02:54.004 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:54.005 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:54.006 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.203+0000 7fb00fa00640 1 -- 172.21.15.32:0/1069831839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb008109b90 msgr2=0x7fb0081103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.203+0000 7fb00fa00640 1 --2- 172.21.15.32:0/1069831839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb008109b90 0x7fb0081103c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb000009ca0 tx=0x7fb00002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.203+0000 7fb00fa00640 1 -- 172.21.15.32:0/1069831839 shutdown_connections 2024-08-31T07:02:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.203+0000 7fb00fa00640 1 --2- 172.21.15.32:0/1069831839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb008109b90 0x7fb0081103c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.203+0000 7fb00fa00640 1 --2- 172.21.15.32:0/1069831839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb008109240 0x7fb008109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.203+0000 7fb00fa00640 1 --2- 172.21.15.32:0/1069831839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb008103900 0x7fb008103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.203+0000 7fb00fa00640 1 -- 172.21.15.32:0/1069831839 >> 172.21.15.32:0/1069831839 conn(0x7fb00807b5f0 msgr2=0x7fb00807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.203+0000 7fb00fa00640 1 -- 172.21.15.32:0/1069831839 shutdown_connections 2024-08-31T07:02:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.203+0000 7fb00fa00640 1 -- 172.21.15.32:0/1069831839 wait complete. 2024-08-31T07:02:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.204+0000 7fb00fa00640 1 Processor -- start 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.204+0000 7fb00fa00640 1 -- start start 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00fa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb008103900 0x7fb00819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00fa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb008109240 0x7fb00819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00fa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb008109b90 0x7fb00819fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00fa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb0081138a0 con 0x7fb008109b90 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00fa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb008113720 con 0x7fb008103900 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00fa00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb008113a20 con 0x7fb008109240 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb008103900 0x7fb00819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb008109b90 0x7fb00819fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb008103900 0x7fb00819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:43658/0 (socket says 172.21.15.32:43658) 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb007e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb008109240 0x7fb00819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00d000640 1 -- 172.21.15.32:0/3916205739 learned_addr learned my addr 172.21.15.32:0/3916205739 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:02:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00d000640 1 -- 172.21.15.32:0/3916205739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb008109240 msgr2=0x7fb00819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00d000640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb008109240 0x7fb00819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00d000640 1 -- 172.21.15.32:0/3916205739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb008109b90 msgr2=0x7fb00819fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00d000640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb008109b90 0x7fb00819fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb00d000640 1 -- 172.21.15.32:0/3916205739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0081a0160 con 0x7fb008103900 2024-08-31T07:02:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.205+0000 7fb007e00640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb008109240 0x7fb00819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:02:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.206+0000 7fb00d000640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb008103900 0x7fb00819a1f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e9f0 tx=0x7faffc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.206+0000 7fb005600640 1 -- 172.21.15.32:0/3916205739 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc00cd80 con 0x7fb008103900 2024-08-31T07:02:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.206+0000 7fb00fa00640 1 -- 172.21.15.32:0/3916205739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0081a0450 con 0x7fb008103900 2024-08-31T07:02:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.206+0000 7fb005600640 1 -- 172.21.15.32:0/3916205739 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faffc0054d0 con 0x7fb008103900 2024-08-31T07:02:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.206+0000 7fb005600640 1 -- 172.21.15.32:0/3916205739 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc010690 con 0x7fb008103900 2024-08-31T07:02:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.206+0000 7fb00fa00640 1 -- 172.21.15.32:0/3916205739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0081a6bb0 con 0x7fb008103900 2024-08-31T07:02:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.207+0000 7fb00fa00640 1 -- 172.21.15.32:0/3916205739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb008103d10 con 0x7fb008103900 2024-08-31T07:02:54.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.208+0000 7fb005600640 1 -- 172.21.15.32:0/3916205739 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7faffc010830 con 0x7fb008103900 2024-08-31T07:02:54.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.208+0000 7fb005600640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faff003f2d0 0x7faff00417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:54.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.208+0000 7fb005600640 1 -- 172.21.15.32:0/3916205739 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7faffc053f80 con 0x7fb008103900 2024-08-31T07:02:54.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.208+0000 7fb007e00640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faff003f2d0 0x7faff00417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:54.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.209+0000 7fb007e00640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faff003f2d0 0x7faff00417e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faff40095f0 tx=0x7faff4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:54.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.213+0000 7fb005600640 1 -- 172.21.15.32:0/3916205739 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faffc016330 con 0x7fb008103900 2024-08-31T07:02:54.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.411+0000 7fb00fa00640 1 -- 172.21.15.32:0/3916205739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fb00810b3f0 con 0x7fb008103900 2024-08-31T07:02:54.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.416+0000 7fb005600640 1 -- 172.21.15.32:0/3916205739 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v15)=0 v15) ==== 117+0+0 (secure 0 0 0) 0x7faffc0046c0 con 0x7fb008103900 2024-08-31T07:02:54.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.419+0000 7fb00fa00640 1 -- 172.21.15.32:0/3916205739 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faff003f2d0 msgr2=0x7faff00417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:54.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.419+0000 7fb00fa00640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faff003f2d0 0x7faff00417e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faff40095f0 tx=0x7faff4005f90 comp rx=0 tx=0).stop 2024-08-31T07:02:54.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.419+0000 7fb00fa00640 1 -- 172.21.15.32:0/3916205739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb008103900 msgr2=0x7fb00819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:54.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.419+0000 7fb00fa00640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb008103900 0x7fb00819a1f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e9f0 tx=0x7faffc00edf0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.419+0000 7fb00fa00640 1 -- 172.21.15.32:0/3916205739 shutdown_connections 2024-08-31T07:02:54.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.419+0000 7fb00fa00640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faff003f2d0 0x7faff00417e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.419+0000 7fb00fa00640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb008109b90 0x7fb00819fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.419+0000 7fb00fa00640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb008109240 0x7fb00819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.419+0000 7fb00fa00640 1 --2- 172.21.15.32:0/3916205739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb008103900 0x7fb00819a1f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:54.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.419+0000 7fb00fa00640 1 -- 172.21.15.32:0/3916205739 >> 172.21.15.32:0/3916205739 conn(0x7fb00807b5f0 msgr2=0x7fb008105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:54.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.420+0000 7fb00fa00640 1 -- 172.21.15.32:0/3916205739 shutdown_connections 2024-08-31T07:02:54.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:54.420+0000 7fb00fa00640 1 -- 172.21.15.32:0/3916205739 wait complete. 2024-08-31T07:02:54.452 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi032') 2024-08-31T07:02:54.740 INFO:journalctl@ceph.mgr.z.smithi084.stdout:Aug 31 07:02:54 smithi084 systemd[1]: Started Ceph mgr.z for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:02:55.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:55 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:02:55.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:55.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:55.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:55.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T07:02:55.485 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T07:02:55.485 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T07:02:55.485 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.524+0000 7f5c20e00640 1 -- 172.21.15.32:0/2825593880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1c075b70 msgr2=0x7f5c1c075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.524+0000 7f5c20e00640 1 --2- 172.21.15.32:0/2825593880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1c075b70 0x7f5c1c075f80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009900 tx=0x7f5c0802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.524+0000 7f5c20e00640 1 -- 172.21.15.32:0/2825593880 shutdown_connections 2024-08-31T07:02:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.524+0000 7f5c20e00640 1 --2- 172.21.15.32:0/2825593880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c1141e0 0x7f5c1c1166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.524+0000 7f5c20e00640 1 --2- 172.21.15.32:0/2825593880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1c075b70 0x7f5c1c075f80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.524+0000 7f5c20e00640 1 --2- 172.21.15.32:0/2825593880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c1c077120 0x7f5c1c075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.524+0000 7f5c20e00640 1 -- 172.21.15.32:0/2825593880 >> 172.21.15.32:0/2825593880 conn(0x7f5c1c0ff380 msgr2=0x7f5c1c1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.524+0000 7f5c20e00640 1 -- 172.21.15.32:0/2825593880 shutdown_connections 2024-08-31T07:02:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.525+0000 7f5c20e00640 1 -- 172.21.15.32:0/2825593880 wait complete. 2024-08-31T07:02:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.526+0000 7f5c20e00640 1 Processor -- start 2024-08-31T07:02:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.526+0000 7f5c20e00640 1 -- start start 2024-08-31T07:02:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c20e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c1c075b70 0x7f5c1c1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c20e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c077120 0x7f5c1c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c20e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1c1141e0 0x7f5c1c1a6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c20e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c119e70 con 0x7f5c1c077120 2024-08-31T07:02:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c20e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c119cf0 con 0x7f5c1c1141e0 2024-08-31T07:02:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c20e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c119ff0 con 0x7f5c1c075b70 2024-08-31T07:02:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c19e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c1c075b70 0x7f5c1c1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c19e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c1c075b70 0x7f5c1c1a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52646/0 (socket says 172.21.15.32:52646) 2024-08-31T07:02:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c19e00640 1 -- 172.21.15.32:0/3245362207 learned_addr learned my addr 172.21.15.32:0/3245362207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:02:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c19400640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c077120 0x7f5c1c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c1a800640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1c1141e0 0x7f5c1c1a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c19e00640 1 -- 172.21.15.32:0/3245362207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1c1141e0 msgr2=0x7f5c1c1a6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c19e00640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1c1141e0 0x7f5c1c1a6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c19e00640 1 -- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c077120 msgr2=0x7f5c1c1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c19e00640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c077120 0x7f5c1c1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c19e00640 1 -- 172.21.15.32:0/3245362207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1c1a6770 con 0x7f5c1c075b70 2024-08-31T07:02:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c19400640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c077120 0x7f5c1c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:02:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.527+0000 7f5c1a800640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1c1141e0 0x7f5c1c1a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:02:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.528+0000 7f5c19e00640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c1c075b70 0x7f5c1c1a0800 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00ea60 tx=0x7f5c0c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.528+0000 7f5c12a00640 1 -- 172.21.15.32:0/3245362207 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c00cde0 con 0x7f5c1c075b70 2024-08-31T07:02:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.528+0000 7f5c12a00640 1 -- 172.21.15.32:0/3245362207 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c0c0035f0 con 0x7f5c1c075b70 2024-08-31T07:02:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.528+0000 7f5c20e00640 1 -- 172.21.15.32:0/3245362207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1c1a6a60 con 0x7f5c1c075b70 2024-08-31T07:02:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.528+0000 7f5c12a00640 1 -- 172.21.15.32:0/3245362207 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c010690 con 0x7f5c1c075b70 2024-08-31T07:02:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.528+0000 7f5c20e00640 1 -- 172.21.15.32:0/3245362207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c1c1ad1c0 con 0x7f5c1c075b70 2024-08-31T07:02:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.529+0000 7f5c12a00640 1 -- 172.21.15.32:0/3245362207 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7f5c0c010830 con 0x7f5c1c075b70 2024-08-31T07:02:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.529+0000 7f5c20e00640 1 -- 172.21.15.32:0/3245362207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c1c075f80 con 0x7f5c1c075b70 2024-08-31T07:02:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.530+0000 7f5c12a00640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c0003f2d0 0x7f5c000417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.530+0000 7f5c12a00640 1 -- 172.21.15.32:0/3245362207 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5c0c053f40 con 0x7f5c1c075b70 2024-08-31T07:02:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.530+0000 7f5c19400640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c0003f2d0 0x7f5c000417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.531+0000 7f5c19400640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c0003f2d0 0x7f5c000417e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c08006010 tx=0x7f5c08000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.535+0000 7f5c12a00640 1 -- 172.21.15.32:0/3245362207 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5c0c01f330 con 0x7f5c1c075b70 2024-08-31T07:02:55.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:55 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:02:55.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:55.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:55.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:55.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T07:02:55.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T07:02:55.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T07:02:55.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.733+0000 7f5c20e00640 1 -- 172.21.15.32:0/3245362207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f5c1c02e430 con 0x7f5c1c075b70 2024-08-31T07:02:55.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:55 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:02:55.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:55.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:55.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:55.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T07:02:55.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T07:02:55.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T07:02:55.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.739+0000 7f5c12a00640 1 -- 172.21.15.32:0/3245362207 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v16)=0 v16) ==== 106+0+0 (secure 0 0 0) 0x7f5c0c01cd60 con 0x7f5c1c075b70 2024-08-31T07:02:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 -- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c0003f2d0 msgr2=0x7f5c000417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c0003f2d0 0x7f5c000417e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c08006010 tx=0x7f5c08000bc0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 -- 172.21.15.32:0/3245362207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c1c075b70 msgr2=0x7f5c1c1a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c1c075b70 0x7f5c1c1a0800 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00ea60 tx=0x7f5c0c00ef20 comp rx=0 tx=0).stop 2024-08-31T07:02:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 -- 172.21.15.32:0/3245362207 shutdown_connections 2024-08-31T07:02:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c0003f2d0 0x7f5c000417e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c1c1141e0 0x7f5c1c1a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c077120 0x7f5c1c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 --2- 172.21.15.32:0/3245362207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c1c075b70 0x7f5c1c1a0800 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 -- 172.21.15.32:0/3245362207 >> 172.21.15.32:0/3245362207 conn(0x7f5c1c0ff380 msgr2=0x7f5c1c1100e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 -- 172.21.15.32:0/3245362207 shutdown_connections 2024-08-31T07:02:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:55.742+0000 7f5c20e00640 1 -- 172.21.15.32:0/3245362207 wait complete. 2024-08-31T07:02:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:56.674 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:56 smithi032 ceph-mon[17252]: Deploying daemon mgr.y on smithi073 2024-08-31T07:02:56.674 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:56 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:02:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:56 smithi073 ceph-mon[21995]: Deploying daemon mgr.y on smithi073 2024-08-31T07:02:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:56 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:02:56.679 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:02:56 smithi073 podman[24312]: 2024-08-31 07:02:56.573817839 +0000 UTC m=+0.115181185 container create 9bcb6b0713010539911125fe0aa0a1085240c678019c72a85220fde04965a586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-31T07:02:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:56 smithi084 ceph-mon[22387]: Deploying daemon mgr.y on smithi073 2024-08-31T07:02:56.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:56 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:02:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.967+0000 7efee3000640 1 -- 172.21.15.32:0/1281098648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efedc103910 msgr2=0x7efedc103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.967+0000 7efee3000640 1 --2- 172.21.15.32:0/1281098648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efedc103910 0x7efedc103d20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efecc009900 tx=0x7efecc02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:02:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.967+0000 7efee3000640 1 -- 172.21.15.32:0/1281098648 shutdown_connections 2024-08-31T07:02:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.967+0000 7efee3000640 1 --2- 172.21.15.32:0/1281098648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efedc109ba0 0x7efedc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.967+0000 7efee3000640 1 --2- 172.21.15.32:0/1281098648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc109250 0x7efedc109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.967+0000 7efee3000640 1 --2- 172.21.15.32:0/1281098648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efedc103910 0x7efedc103d20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.967+0000 7efee3000640 1 -- 172.21.15.32:0/1281098648 >> 172.21.15.32:0/1281098648 conn(0x7efedc0ff380 msgr2=0x7efedc1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.967+0000 7efee3000640 1 -- 172.21.15.32:0/1281098648 shutdown_connections 2024-08-31T07:02:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.967+0000 7efee3000640 1 -- 172.21.15.32:0/1281098648 wait complete. 2024-08-31T07:02:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.969+0000 7efee3000640 1 Processor -- start 2024-08-31T07:02:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.969+0000 7efee3000640 1 -- start start 2024-08-31T07:02:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efee3000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efedc103910 0x7efedc0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efee3000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efedc109250 0x7efedc0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efee3000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc109ba0 0x7efedc075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efee3000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1138b0 con 0x7efedc109250 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efee3000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efedc113730 con 0x7efedc103910 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efee3000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efedc113a30 con 0x7efedc109ba0 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efedbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efedc103910 0x7efedc0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efedbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efedc103910 0x7efedc0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59710/0 (socket says 172.21.15.32:59710) 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efedbe00640 1 -- 172.21.15.32:0/2337930413 learned_addr learned my addr 172.21.15.32:0/2337930413 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efee1000640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc109ba0 0x7efedc075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efedb400640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efedc109250 0x7efedc0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efedbe00640 1 -- 172.21.15.32:0/2337930413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc109ba0 msgr2=0x7efedc075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efedbe00640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc109ba0 0x7efedc075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efedbe00640 1 -- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efedc109250 msgr2=0x7efedc0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efedbe00640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efedc109250 0x7efedc0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efedbe00640 1 -- 172.21.15.32:0/2337930413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc1a8540 con 0x7efedc103910 2024-08-31T07:02:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.970+0000 7efedb400640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efedc109250 0x7efedc0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:02:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.971+0000 7efee1000640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc109ba0 0x7efedc075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:02:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.971+0000 7efedbe00640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efedc103910 0x7efedc0754b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efecc005ab0 tx=0x7efecc003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.971+0000 7efed8c00640 1 -- 172.21.15.32:0/2337930413 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc03f070 con 0x7efedc103910 2024-08-31T07:02:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.971+0000 7efed8c00640 1 -- 172.21.15.32:0/2337930413 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efecc031d60 con 0x7efedc103910 2024-08-31T07:02:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.971+0000 7efee3000640 1 -- 172.21.15.32:0/2337930413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc1a87d0 con 0x7efedc103910 2024-08-31T07:02:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.971+0000 7efed8c00640 1 -- 172.21.15.32:0/2337930413 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc035030 con 0x7efedc103910 2024-08-31T07:02:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.971+0000 7efee3000640 1 -- 172.21.15.32:0/2337930413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc1a8d00 con 0x7efedc103910 2024-08-31T07:02:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.973+0000 7efee3000640 1 -- 172.21.15.32:0/2337930413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efedc103d20 con 0x7efedc103910 2024-08-31T07:02:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.973+0000 7efed8c00640 1 -- 172.21.15.32:0/2337930413 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7efecc042070 con 0x7efedc103910 2024-08-31T07:02:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.973+0000 7efed8c00640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efec003f2d0 0x7efec00417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.974+0000 7efed8c00640 1 -- 172.21.15.32:0/2337930413 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7efecc074290 con 0x7efedc103910 2024-08-31T07:02:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.974+0000 7efedb400640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efec003f2d0 0x7efec00417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.975+0000 7efedb400640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efec003f2d0 0x7efec00417e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efedc108d40 tx=0x7efec8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.978+0000 7efed8c00640 1 -- 172.21.15.32:0/2337930413 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efecc049030 con 0x7efedc103910 2024-08-31T07:02:57.179 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:02:56 smithi073 podman[24312]: 2024-08-31 07:02:56.784741865 +0000 UTC m=+0.326105209 container init 9bcb6b0713010539911125fe0aa0a1085240c678019c72a85220fde04965a586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-31T07:02:57.179 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:02:56 smithi073 podman[24312]: 2024-08-31 07:02:56.787757913 +0000 UTC m=+0.329121258 container start 9bcb6b0713010539911125fe0aa0a1085240c678019c72a85220fde04965a586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-y, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:02:57.179 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:02:56 smithi073 ceph-mgr[24331]: -- 172.21.15.73:0/2972835008 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563698c9b0e0 con 0x56369da3a480 2024-08-31T07:02:57.180 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:02:56 smithi073 bash[24312]: 9bcb6b0713010539911125fe0aa0a1085240c678019c72a85220fde04965a586 2024-08-31T07:02:57.180 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:02:56 smithi073 systemd[1]: Started Ceph mgr.y for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:02:57.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.186+0000 7efee3000640 1 -- 172.21.15.32:0/2337930413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7efedc02e430 con 0x7efedc103910 2024-08-31T07:02:57.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.192+0000 7efed8c00640 1 -- 172.21.15.32:0/2337930413 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v18)=0 v18) ==== 116+0+0 (secure 0 0 0) 0x7efecc0303a0 con 0x7efedc103910 2024-08-31T07:02:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 -- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efec003f2d0 msgr2=0x7efec00417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efec003f2d0 0x7efec00417e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efedc108d40 tx=0x7efec8005f90 comp rx=0 tx=0).stop 2024-08-31T07:02:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 -- 172.21.15.32:0/2337930413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efedc103910 msgr2=0x7efedc0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efedc103910 0x7efedc0754b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efecc005ab0 tx=0x7efecc003c80 comp rx=0 tx=0).stop 2024-08-31T07:02:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 -- 172.21.15.32:0/2337930413 shutdown_connections 2024-08-31T07:02:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efec003f2d0 0x7efec00417e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc109ba0 0x7efedc075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efedc109250 0x7efedc0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:57.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 --2- 172.21.15.32:0/2337930413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efedc103910 0x7efedc0754b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:57.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 -- 172.21.15.32:0/2337930413 >> 172.21.15.32:0/2337930413 conn(0x7efedc0ff380 msgr2=0x7efedc1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:57.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 -- 172.21.15.32:0/2337930413 shutdown_connections 2024-08-31T07:02:57.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.194+0000 7efee3000640 1 -- 172.21.15.32:0/2337930413 wait complete. 2024-08-31T07:02:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:57 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:02:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:57 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:02:58.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:57 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:02:58.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:58.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:57 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:02:58.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:58.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:57 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:02:58.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:02:58.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:57 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:02:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:58.352 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:58.352 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:58.352 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:58.358 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:02:58.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi032') 2024-08-31T07:02:58.433 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi032') 2024-08-31T07:02:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.638+0000 7f36afe00640 1 -- 172.21.15.32:0/3396333080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b0102b50 msgr2=0x7f36b010f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.638+0000 7f36afe00640 1 --2- 172.21.15.32:0/3396333080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b0102b50 0x7f36b010f870 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3698009900 tx=0x7f369802a480 comp rx=0 tx=0).stop 2024-08-31T07:02:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.638+0000 7f36afe00640 1 -- 172.21.15.32:0/3396333080 shutdown_connections 2024-08-31T07:02:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.638+0000 7f36afe00640 1 --2- 172.21.15.32:0/3396333080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36b010fdb0 0x7f36b01121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.638+0000 7f36afe00640 1 --2- 172.21.15.32:0/3396333080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b0102b50 0x7f36b010f870 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.638+0000 7f36afe00640 1 --2- 172.21.15.32:0/3396333080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36b0102200 0x7f36b0102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.638+0000 7f36afe00640 1 -- 172.21.15.32:0/3396333080 >> 172.21.15.32:0/3396333080 conn(0x7f36b00fe060 msgr2=0x7f36b01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.638+0000 7f36afe00640 1 -- 172.21.15.32:0/3396333080 shutdown_connections 2024-08-31T07:02:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.639+0000 7f36afe00640 1 -- 172.21.15.32:0/3396333080 wait complete. 2024-08-31T07:02:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.640+0000 7f36afe00640 1 Processor -- start 2024-08-31T07:02:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.640+0000 7f36afe00640 1 -- start start 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.640+0000 7f36afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b0102200 0x7f36b019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.640+0000 7f36afe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36b0102b50 0x7f36b019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.640+0000 7f36afe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36b010fdb0 0x7f36b01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.640+0000 7f36afe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f36b0115800 con 0x7f36b010fdb0 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.640+0000 7f36afe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f36b0115680 con 0x7f36b0102200 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.640+0000 7f36afe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36b0115980 con 0x7f36b0102b50 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36a7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36b0102b50 0x7f36b019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36aea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b0102200 0x7f36b019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36a7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36b0102b50 0x7f36b019c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57168/0 (socket says 172.21.15.32:57168) 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36a7e00640 1 -- 172.21.15.32:0/2945607460 learned_addr learned my addr 172.21.15.32:0/2945607460 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36aea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b0102200 0x7f36b019c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59728/0 (socket says 172.21.15.32:59728) 2024-08-31T07:02:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36af400640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36b010fdb0 0x7f36b01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36a7e00640 1 -- 172.21.15.32:0/2945607460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b0102200 msgr2=0x7f36b019c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36a7e00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b0102200 0x7f36b019c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36a7e00640 1 -- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36b010fdb0 msgr2=0x7f36b01a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36a7e00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36b010fdb0 0x7f36b01a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36a7e00640 1 -- 172.21.15.32:0/2945607460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b01a21a0 con 0x7f36b0102b50 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36af400640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36b010fdb0 0x7f36b01a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36aea00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b0102200 0x7f36b019c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.641+0000 7f36a7e00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36b0102b50 0x7f36b019c7d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3698005940 tx=0x7f36980040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.642+0000 7f36a7400640 1 -- 172.21.15.32:0/2945607460 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369803f030 con 0x7f36b0102b50 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.642+0000 7f36afe00640 1 -- 172.21.15.32:0/2945607460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b01a2430 con 0x7f36b0102b50 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.642+0000 7f36a7400640 1 -- 172.21.15.32:0/2945607460 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36980044c0 con 0x7f36b0102b50 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.642+0000 7f36a7400640 1 -- 172.21.15.32:0/2945607460 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698035070 con 0x7f36b0102b50 2024-08-31T07:02:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.642+0000 7f36afe00640 1 -- 172.21.15.32:0/2945607460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b01a8bf0 con 0x7f36b0102b50 2024-08-31T07:02:58.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.643+0000 7f36afe00640 1 -- 172.21.15.32:0/2945607460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b0102610 con 0x7f36b0102b50 2024-08-31T07:02:58.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.643+0000 7f36a7400640 1 -- 172.21.15.32:0/2945607460 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7f3698042030 con 0x7f36b0102b50 2024-08-31T07:02:58.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.644+0000 7f36a7400640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f369403f2d0 0x7f36940417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:58.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.644+0000 7f36a7400640 1 -- 172.21.15.32:0/2945607460 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3698073f80 con 0x7f36b0102b50 2024-08-31T07:02:58.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.644+0000 7f36aea00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f369403f2d0 0x7f36940417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:58.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.645+0000 7f36aea00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f369403f2d0 0x7f36940417e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009fc0 tx=0x7f36a0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.654+0000 7f36a7400640 1 -- 172.21.15.32:0/2945607460 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3698038350 con 0x7f36b0102b50 2024-08-31T07:02:58.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.856+0000 7f36afe00640 1 -- 172.21.15.32:0/2945607460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f36b010af40 con 0x7f36b0102b50 2024-08-31T07:02:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.863+0000 7f36a7400640 1 -- 172.21.15.32:0/2945607460 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v19)=0 v19) ==== 108+0+0 (secure 0 0 0) 0x7f3698003cf0 con 0x7f36b0102b50 2024-08-31T07:02:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 -- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f369403f2d0 msgr2=0x7f36940417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f369403f2d0 0x7f36940417e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009fc0 tx=0x7f36a0005f90 comp rx=0 tx=0).stop 2024-08-31T07:02:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 -- 172.21.15.32:0/2945607460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36b0102b50 msgr2=0x7f36b019c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36b0102b50 0x7f36b019c7d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3698005940 tx=0x7f36980040c0 comp rx=0 tx=0).stop 2024-08-31T07:02:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 -- 172.21.15.32:0/2945607460 shutdown_connections 2024-08-31T07:02:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f369403f2d0 0x7f36940417e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36b010fdb0 0x7f36b01a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36b0102b50 0x7f36b019c7d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 --2- 172.21.15.32:0/2945607460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b0102200 0x7f36b019c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:58.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 -- 172.21.15.32:0/2945607460 >> 172.21.15.32:0/2945607460 conn(0x7f36b00fe060 msgr2=0x7f36b0111f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:58.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 -- 172.21.15.32:0/2945607460 shutdown_connections 2024-08-31T07:02:58.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:58.866+0000 7f36afe00640 1 -- 172.21.15.32:0/2945607460 wait complete. 2024-08-31T07:02:59.217 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.217 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: Reconfiguring mgr.x (unknown last config time)... 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: Reconfiguring daemon mgr.x on smithi032 2024-08-31T07:02:59.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:02:59 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:02:59.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:59.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:59.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: Reconfiguring mgr.x (unknown last config time)... 2024-08-31T07:02:59.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T07:02:59.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T07:02:59.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:59.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: Reconfiguring daemon mgr.x on smithi032 2024-08-31T07:02:59.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:02:59 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:02:59.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:02:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:02:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: Reconfiguring mgr.x (unknown last config time)... 2024-08-31T07:02:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T07:02:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T07:02:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:02:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: Reconfiguring daemon mgr.x on smithi032 2024-08-31T07:02:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:02:59 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:02:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.395+0000 7f066cc00640 1 -- 172.21.15.32:0/2147369111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f066810dbc0 msgr2=0x7f06681100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.395+0000 7f066cc00640 1 --2- 172.21.15.32:0/2147369111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f066810dbc0 0x7f06681100b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0660009ca0 tx=0x7f066002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:02:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.395+0000 7f066cc00640 1 -- 172.21.15.32:0/2147369111 shutdown_connections 2024-08-31T07:02:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.395+0000 7f066cc00640 1 --2- 172.21.15.32:0/2147369111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f066810dbc0 0x7f06681100b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.395+0000 7f066cc00640 1 --2- 172.21.15.32:0/2147369111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06681050b0 0x7f066810d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.395+0000 7f066cc00640 1 --2- 172.21.15.32:0/2147369111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0668104760 0x7f0668104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.395+0000 7f066cc00640 1 -- 172.21.15.32:0/2147369111 >> 172.21.15.32:0/2147369111 conn(0x7f0668079480 msgr2=0x7f06680798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.395+0000 7f066cc00640 1 -- 172.21.15.32:0/2147369111 shutdown_connections 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.395+0000 7f066cc00640 1 -- 172.21.15.32:0/2147369111 wait complete. 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.396+0000 7f066cc00640 1 Processor -- start 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.396+0000 7f066cc00640 1 -- start start 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f066cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0668104760 0x7f066819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f066cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06681050b0 0x7f066819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f066cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f066810dbc0 0x7f06681a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f066cc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0668113850 con 0x7f0668104760 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f066cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f06681136d0 con 0x7f066810dbc0 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f066cc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06681139d0 con 0x7f06681050b0 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f0665e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0668104760 0x7f066819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f065d400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06681050b0 0x7f066819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f0665e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0668104760 0x7f066819c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43958/0 (socket says 172.21.15.32:43958) 2024-08-31T07:02:59.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f0666800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f066810dbc0 0x7f06681a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.397+0000 7f0665e00640 1 -- 172.21.15.32:0/1502336213 learned_addr learned my addr 172.21.15.32:0/1502336213 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f0666800640 1 -- 172.21.15.32:0/1502336213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06681050b0 msgr2=0x7f066819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f0666800640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06681050b0 0x7f066819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f0666800640 1 -- 172.21.15.32:0/1502336213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0668104760 msgr2=0x7f066819c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f0666800640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0668104760 0x7f066819c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f0666800640 1 -- 172.21.15.32:0/1502336213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06681a23f0 con 0x7f066810dbc0 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f0666800640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f066810dbc0 0x7f06681a1cc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f066002ade0 tx=0x7f0660031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f065f400640 1 -- 172.21.15.32:0/1502336213 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660003b70 con 0x7f066810dbc0 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f065f400640 1 -- 172.21.15.32:0/1502336213 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0660031910 con 0x7f066810dbc0 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f065f400640 1 -- 172.21.15.32:0/1502336213 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066002f950 con 0x7f066810dbc0 2024-08-31T07:02:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f066cc00640 1 -- 172.21.15.32:0/1502336213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06681a2680 con 0x7f066810dbc0 2024-08-31T07:02:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.398+0000 7f066cc00640 1 -- 172.21.15.32:0/1502336213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681a8e40 con 0x7f066810dbc0 2024-08-31T07:02:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.399+0000 7f066cc00640 1 -- 172.21.15.32:0/1502336213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066806bfa0 con 0x7f066810dbc0 2024-08-31T07:02:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.399+0000 7f065f400640 1 -- 172.21.15.32:0/1502336213 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7f0660004040 con 0x7f066810dbc0 2024-08-31T07:02:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.400+0000 7f065f400640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f064c03f2d0 0x7f064c0417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:02:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.400+0000 7f065f400640 1 -- 172.21.15.32:0/1502336213 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0660074cf0 con 0x7f066810dbc0 2024-08-31T07:02:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.400+0000 7f0665e00640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f064c03f2d0 0x7f064c0417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:02:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.401+0000 7f0665e00640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f064c03f2d0 0x7f064c0417e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f06580095f0 tx=0x7f0658005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:02:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.405+0000 7f065f400640 1 -- 172.21.15.32:0/1502336213 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f066002faf0 con 0x7f066810dbc0 2024-08-31T07:02:59.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.600+0000 7f066cc00640 1 -- 172.21.15.32:0/1502336213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f06681a9040 con 0x7f066810dbc0 2024-08-31T07:02:59.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.608+0000 7f065f400640 1 -- 172.21.15.32:0/1502336213 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v20)=0 v20) ==== 119+0+0 (secure 0 0 0) 0x7f066003b410 con 0x7f066810dbc0 2024-08-31T07:02:59.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 -- 172.21.15.32:0/1502336213 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f064c03f2d0 msgr2=0x7f064c0417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:59.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f064c03f2d0 0x7f064c0417e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f06580095f0 tx=0x7f0658005f90 comp rx=0 tx=0).stop 2024-08-31T07:02:59.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 -- 172.21.15.32:0/1502336213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f066810dbc0 msgr2=0x7f06681a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:02:59.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f066810dbc0 0x7f06681a1cc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f066002ade0 tx=0x7f0660031660 comp rx=0 tx=0).stop 2024-08-31T07:02:59.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 -- 172.21.15.32:0/1502336213 shutdown_connections 2024-08-31T07:02:59.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f064c03f2d0 0x7f064c0417e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:59.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f066810dbc0 0x7f06681a1cc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:59.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06681050b0 0x7f066819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:59.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 --2- 172.21.15.32:0/1502336213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0668104760 0x7f066819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:02:59.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 -- 172.21.15.32:0/1502336213 >> 172.21.15.32:0/1502336213 conn(0x7f0668079480 msgr2=0x7f066810cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:02:59.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 -- 172.21.15.32:0/1502336213 shutdown_connections 2024-08-31T07:02:59.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.611+0000 7f066cc00640 1 -- 172.21.15.32:0/1502336213 wait complete. 2024-08-31T07:03:00.249 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:00 smithi032 ceph-mon[17252]: Standby manager daemon z started 2024-08-31T07:03:00.249 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:00 smithi032 ceph-mon[17252]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-31T07:03:00.249 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:00 smithi032 ceph-mon[17252]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T07:03:00.249 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:00 smithi032 ceph-mon[17252]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-31T07:03:00.249 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:00 smithi032 ceph-mon[17252]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T07:03:00.249 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:00 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:00.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.328+0000 7efc27c00640 1 -- 172.21.15.32:0/170549323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc20109b90 msgr2=0x7efc201103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:00.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.328+0000 7efc27c00640 1 --2- 172.21.15.32:0/170549323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc20109b90 0x7efc201103c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc18009ca0 tx=0x7efc1802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.328+0000 7efc27c00640 1 -- 172.21.15.32:0/170549323 shutdown_connections 2024-08-31T07:03:00.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.328+0000 7efc27c00640 1 --2- 172.21.15.32:0/170549323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc20109b90 0x7efc201103c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.328+0000 7efc27c00640 1 --2- 172.21.15.32:0/170549323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc20109240 0x7efc20109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.328+0000 7efc27c00640 1 --2- 172.21.15.32:0/170549323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc20103900 0x7efc20103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.328+0000 7efc27c00640 1 -- 172.21.15.32:0/170549323 >> 172.21.15.32:0/170549323 conn(0x7efc2007b5f0 msgr2=0x7efc2007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:00.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.328+0000 7efc27c00640 1 -- 172.21.15.32:0/170549323 shutdown_connections 2024-08-31T07:03:00.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.328+0000 7efc27c00640 1 -- 172.21.15.32:0/170549323 wait complete. 2024-08-31T07:03:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.329+0000 7efc27c00640 1 Processor -- start 2024-08-31T07:03:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.329+0000 7efc27c00640 1 -- start start 2024-08-31T07:03:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc27c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc20103900 0x7efc2019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc27c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc20109240 0x7efc2019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc27c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc20109b90 0x7efc201a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc27c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc201138a0 con 0x7efc20109b90 2024-08-31T07:03:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc27c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc20113720 con 0x7efc20109240 2024-08-31T07:03:00.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc27c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc20113a20 con 0x7efc20103900 2024-08-31T07:03:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc25200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc20103900 0x7efc2019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc25200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc20103900 0x7efc2019c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57224/0 (socket says 172.21.15.32:57224) 2024-08-31T07:03:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc17e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc20109240 0x7efc2019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc25200640 1 -- 172.21.15.32:0/762596993 learned_addr learned my addr 172.21.15.32:0/762596993 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc25c00640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc20109b90 0x7efc201a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:00.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc25200640 1 -- 172.21.15.32:0/762596993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc20109240 msgr2=0x7efc2019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc25200640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc20109240 0x7efc2019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc25200640 1 -- 172.21.15.32:0/762596993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc20109b90 msgr2=0x7efc201a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc25200640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc20109b90 0x7efc201a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.330+0000 7efc25200640 1 -- 172.21.15.32:0/762596993 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc201a2360 con 0x7efc20103900 2024-08-31T07:03:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.331+0000 7efc25200640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc20103900 0x7efc2019c3f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efc1000e9f0 tx=0x7efc1000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.331+0000 7efc1e000640 1 -- 172.21.15.32:0/762596993 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1000cd80 con 0x7efc20103900 2024-08-31T07:03:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.331+0000 7efc1e000640 1 -- 172.21.15.32:0/762596993 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc100054d0 con 0x7efc20103900 2024-08-31T07:03:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.331+0000 7efc1e000640 1 -- 172.21.15.32:0/762596993 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10010690 con 0x7efc20103900 2024-08-31T07:03:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.331+0000 7efc27c00640 1 -- 172.21.15.32:0/762596993 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc201a2650 con 0x7efc20103900 2024-08-31T07:03:00.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.331+0000 7efc27c00640 1 -- 172.21.15.32:0/762596993 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc201a8db0 con 0x7efc20103900 2024-08-31T07:03:00.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.332+0000 7efc27c00640 1 -- 172.21.15.32:0/762596993 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc20103d10 con 0x7efc20103900 2024-08-31T07:03:00.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.333+0000 7efc1e000640 1 -- 172.21.15.32:0/762596993 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 101904+0+0 (secure 0 0 0) 0x7efc10004e90 con 0x7efc20103900 2024-08-31T07:03:00.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.334+0000 7efc1e000640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc04079ee0 0x7efc0407c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:00.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.334+0000 7efc1e000640 1 -- 172.21.15.32:0/762596993 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7efc1009bbc0 con 0x7efc20103900 2024-08-31T07:03:00.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.334+0000 7efc17e00640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc04079ee0 0x7efc0407c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:00.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.335+0000 7efc17e00640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc04079ee0 0x7efc0407c3f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efc20106c70 tx=0x7efc08005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:00.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.339+0000 7efc1e000640 1 -- 172.21.15.32:0/762596993 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efc10064460 con 0x7efc20103900 2024-08-31T07:03:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:00 smithi073 ceph-mon[21995]: Standby manager daemon z started 2024-08-31T07:03:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:00 smithi073 ceph-mon[21995]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-31T07:03:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:00 smithi073 ceph-mon[21995]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T07:03:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:00 smithi073 ceph-mon[21995]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-31T07:03:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:00 smithi073 ceph-mon[21995]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T07:03:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:00 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:00.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:00 smithi084 ceph-mon[22387]: Standby manager daemon z started 2024-08-31T07:03:00.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:00 smithi084 ceph-mon[22387]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-31T07:03:00.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:00 smithi084 ceph-mon[22387]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T07:03:00.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:00 smithi084 ceph-mon[22387]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-31T07:03:00.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:00 smithi084 ceph-mon[22387]: from='mgr.? 172.21.15.84:0/1665941885' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T07:03:00.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:00 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:00.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.539+0000 7efc27c00640 1 -- 172.21.15.32:0/762596993 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7efc201a9040 con 0x7efc20103900 2024-08-31T07:03:00.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.545+0000 7efc1e000640 1 -- 172.21.15.32:0/762596993 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v22)=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7efc10068c20 con 0x7efc20103900 2024-08-31T07:03:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 -- 172.21.15.32:0/762596993 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc04079ee0 msgr2=0x7efc0407c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc04079ee0 0x7efc0407c3f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efc20106c70 tx=0x7efc08005f90 comp rx=0 tx=0).stop 2024-08-31T07:03:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 -- 172.21.15.32:0/762596993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc20103900 msgr2=0x7efc2019c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc20103900 0x7efc2019c3f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efc1000e9f0 tx=0x7efc1000edf0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 -- 172.21.15.32:0/762596993 shutdown_connections 2024-08-31T07:03:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc04079ee0 0x7efc0407c3f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc20109b90 0x7efc201a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc20109240 0x7efc2019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 --2- 172.21.15.32:0/762596993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc20103900 0x7efc2019c3f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 -- 172.21.15.32:0/762596993 >> 172.21.15.32:0/762596993 conn(0x7efc2007b5f0 msgr2=0x7efc20104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 -- 172.21.15.32:0/762596993 shutdown_connections 2024-08-31T07:03:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7efc27c00640 1 -- 172.21.15.32:0/762596993 wait complete. 2024-08-31T07:03:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.540+0000 7f42fde00640 1 -- 172.21.15.32:0/163457074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42f8109bb0 msgr2=0x7f42f81103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.540+0000 7f42fde00640 1 --2- 172.21.15.32:0/163457074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42f8109bb0 0x7f42f81103e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009ca0 tx=0x7f42f002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.540+0000 7f42fde00640 1 -- 172.21.15.32:0/163457074 shutdown_connections 2024-08-31T07:03:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.540+0000 7f42fde00640 1 --2- 172.21.15.32:0/163457074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42f8109bb0 0x7f42f81103e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.540+0000 7f42fde00640 1 --2- 172.21.15.32:0/163457074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8109260 0x7f42f8109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.540+0000 7f42fde00640 1 --2- 172.21.15.32:0/163457074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f8103920 0x7f42f8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.540+0000 7f42fde00640 1 -- 172.21.15.32:0/163457074 >> 172.21.15.32:0/163457074 conn(0x7f42f80ff330 msgr2=0x7f42f8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.541+0000 7f42fde00640 1 -- 172.21.15.32:0/163457074 shutdown_connections 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.541+0000 7f42fde00640 1 -- 172.21.15.32:0/163457074 wait complete. 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.542+0000 7f42fde00640 1 Processor -- start 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.542+0000 7f42fde00640 1 -- start start 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.542+0000 7f42fde00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f8103920 0x7f42f80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.542+0000 7f42fde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42f8109260 0x7f42f80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42fde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8109bb0 0x7f42f8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42fde00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f42f81138c0 con 0x7f42f8109260 2024-08-31T07:03:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42fde00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8113740 con 0x7f42f8109bb0 2024-08-31T07:03:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42fde00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8113a40 con 0x7f42f8103920 2024-08-31T07:03:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42f6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f8103920 0x7f42f80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42f6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f8103920 0x7f42f80754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57246/0 (socket says 172.21.15.32:57246) 2024-08-31T07:03:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42f6a00640 1 -- 172.21.15.32:0/3288217211 learned_addr learned my addr 172.21.15.32:0/3288217211 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42f7400640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8109bb0 0x7f42f8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42f6000640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42f8109260 0x7f42f80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42f7400640 1 -- 172.21.15.32:0/3288217211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f8103920 msgr2=0x7f42f80754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42f7400640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f8103920 0x7f42f80754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42f7400640 1 -- 172.21.15.32:0/3288217211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42f8109260 msgr2=0x7f42f80759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42f7400640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42f8109260 0x7f42f80759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.543+0000 7f42f7400640 1 -- 172.21.15.32:0/3288217211 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f81a84b0 con 0x7f42f8109bb0 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.544+0000 7f42f7400640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8109bb0 0x7f42f8075f30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f42f0005e30 tx=0x7f42f00317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.544+0000 7f42fca00640 1 -- 172.21.15.32:0/3288217211 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0003b70 con 0x7f42f8109bb0 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.545+0000 7f42fca00640 1 -- 172.21.15.32:0/3288217211 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f0003d10 con 0x7f42f8109bb0 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.545+0000 7f42fca00640 1 -- 172.21.15.32:0/3288217211 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f002f970 con 0x7f42f8109bb0 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.545+0000 7f42fde00640 1 -- 172.21.15.32:0/3288217211 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f81a8740 con 0x7f42f8109bb0 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.545+0000 7f42fde00640 1 -- 172.21.15.32:0/3288217211 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f81a8c30 con 0x7f42f8109bb0 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.546+0000 7f42fde00640 1 -- 172.21.15.32:0/3288217211 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f8103d30 con 0x7f42f8109bb0 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.548+0000 7f42fca00640 1 -- 172.21.15.32:0/3288217211 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 101904+0+0 (secure 0 0 0) 0x7f42f002fb10 con 0x7f42f8109bb0 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.549+0000 7f42fca00640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42dc079ee0 0x7f42dc07c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.549+0000 7f42fca00640 1 -- 172.21.15.32:0/3288217211 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f42f00bc710 con 0x7f42f8109bb0 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.549+0000 7f42f6a00640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42dc079ee0 0x7f42dc07c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.550+0000 7f42f6a00640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42dc079ee0 0x7f42dc07c3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f42f8076f00 tx=0x7f42e8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.553+0000 7f42fca00640 1 -- 172.21.15.32:0/3288217211 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f42f0089840 con 0x7f42f8109bb0 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.754+0000 7f42fde00640 1 -- 172.21.15.32:0/3288217211 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f42f802e430 con 0x7f42f8109bb0 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.760+0000 7f42fca00640 1 -- 172.21.15.32:0/3288217211 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v23)=0 v23) ==== 129+0+0 (secure 0 0 0) 0x7f42f008e260 con 0x7f42f8109bb0 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 -- 172.21.15.32:0/3288217211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42dc079ee0 msgr2=0x7f42dc07c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42dc079ee0 0x7f42dc07c3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f42f8076f00 tx=0x7f42e8005f90 comp rx=0 tx=0).stop 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 -- 172.21.15.32:0/3288217211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8109bb0 msgr2=0x7f42f8075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8109bb0 0x7f42f8075f30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f42f0005e30 tx=0x7f42f00317c0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 -- 172.21.15.32:0/3288217211 shutdown_connections 2024-08-31T07:03:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42dc079ee0 0x7f42dc07c3f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8109bb0 0x7f42f8075f30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42f8109260 0x7f42f80759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 --2- 172.21.15.32:0/3288217211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f8103920 0x7f42f80754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 -- 172.21.15.32:0/3288217211 >> 172.21.15.32:0/3288217211 conn(0x7f42f80ff330 msgr2=0x7f42f8106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:01.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.762+0000 7f42fde00640 1 -- 172.21.15.32:0/3288217211 shutdown_connections 2024-08-31T07:03:01.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.763+0000 7f42fde00640 1 -- 172.21.15.32:0/3288217211 wait complete. 2024-08-31T07:03:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:01 smithi032 ceph-mon[17252]: mgrmap e18: x(active, since 15s), standbys: z 2024-08-31T07:03:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-31T07:03:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:01 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:01 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:01.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:01 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:01 smithi073 ceph-mon[21995]: mgrmap e18: x(active, since 15s), standbys: z 2024-08-31T07:03:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-31T07:03:01.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:01.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:01 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:01.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:01 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:01.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:01 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:01.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.847+0000 7fe30e200640 1 -- 172.21.15.32:0/2118452018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe308104770 msgr2=0x7fe308104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:01.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.847+0000 7fe30e200640 1 --2- 172.21.15.32:0/2118452018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe308104770 0x7fe308104b80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009900 tx=0x7fe2f802a400 comp rx=0 tx=0).stop 2024-08-31T07:03:01.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.847+0000 7fe30e200640 1 -- 172.21.15.32:0/2118452018 shutdown_connections 2024-08-31T07:03:01.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.847+0000 7fe30e200640 1 --2- 172.21.15.32:0/2118452018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe30810dbd0 0x7fe3081100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.847+0000 7fe30e200640 1 --2- 172.21.15.32:0/2118452018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3081050c0 0x7fe30810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.847+0000 7fe30e200640 1 --2- 172.21.15.32:0/2118452018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe308104770 0x7fe308104b80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.847+0000 7fe30e200640 1 -- 172.21.15.32:0/2118452018 >> 172.21.15.32:0/2118452018 conn(0x7fe3080794e0 msgr2=0x7fe3080798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:01.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.847+0000 7fe30e200640 1 -- 172.21.15.32:0/2118452018 shutdown_connections 2024-08-31T07:03:01.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.847+0000 7fe30e200640 1 -- 172.21.15.32:0/2118452018 wait complete. 2024-08-31T07:03:01.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.849+0000 7fe30e200640 1 Processor -- start 2024-08-31T07:03:01.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.849+0000 7fe30e200640 1 -- start start 2024-08-31T07:03:01.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.849+0000 7fe30e200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe308104770 0x7fe30819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:01.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.849+0000 7fe30e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3081050c0 0x7fe30819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:01.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.849+0000 7fe30e200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe30810dbd0 0x7fe30819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:01.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe30e200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe308113880 con 0x7fe30810dbd0 2024-08-31T07:03:01.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe30e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe308113700 con 0x7fe3081050c0 2024-08-31T07:03:01.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe30e200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe308113a00 con 0x7fe308104770 2024-08-31T07:03:01.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe307400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe308104770 0x7fe30819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:01.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe306a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3081050c0 0x7fe30819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe307400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe308104770 0x7fe30819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57130/0 (socket says 172.21.15.32:57130) 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe307400640 1 -- 172.21.15.32:0/749025903 learned_addr learned my addr 172.21.15.32:0/749025903 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe306a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3081050c0 0x7fe30819a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59692/0 (socket says 172.21.15.32:59692) 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe307e00640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe30810dbd0 0x7fe30819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe307400640 1 -- 172.21.15.32:0/749025903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3081050c0 msgr2=0x7fe30819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe307400640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3081050c0 0x7fe30819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe307400640 1 -- 172.21.15.32:0/749025903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe30810dbd0 msgr2=0x7fe30819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe307400640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe30810dbd0 0x7fe30819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe307400640 1 -- 172.21.15.32:0/749025903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3081a0240 con 0x7fe308104770 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.850+0000 7fe306a00640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3081050c0 0x7fe30819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.851+0000 7fe307400640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe308104770 0x7fe30819a2d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8005ab0 tx=0x7fe2f8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:01.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.851+0000 7fe2ffe00640 1 -- 172.21.15.32:0/749025903 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f803f070 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.851+0000 7fe2ffe00640 1 -- 172.21.15.32:0/749025903 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2f8031b50 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.851+0000 7fe30e200640 1 -- 172.21.15.32:0/749025903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3081a04d0 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.851+0000 7fe2ffe00640 1 -- 172.21.15.32:0/749025903 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8035030 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.851+0000 7fe30e200640 1 -- 172.21.15.32:0/749025903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3081a6c90 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.852+0000 7fe30e200640 1 -- 172.21.15.32:0/749025903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe30806bfe0 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.853+0000 7fe2ffe00640 1 -- 172.21.15.32:0/749025903 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7fe2f8042070 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.853+0000 7fe2ffe00640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2ec03f2d0 0x7fe2ec0417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.853+0000 7fe2ffe00640 1 -- 172.21.15.32:0/749025903 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe2f8074090 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.854+0000 7fe306a00640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2ec03f2d0 0x7fe2ec0417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.854+0000 7fe306a00640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2ec03f2d0 0x7fe2ec0417e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40095f0 tx=0x7fe2f4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:56.858+0000 7fe2ffe00640 1 -- 172.21.15.32:0/749025903 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe2f8031d00 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.082+0000 7fe30e200640 1 -- 172.21.15.32:0/749025903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fe308108740 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.091+0000 7fe2ffe00640 1 -- 172.21.15.32:0/749025903 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v17)=0 v17) ==== 120+0+0 (secure 0 0 0) 0x7fe2f804cd00 con 0x7fe308104770 2024-08-31T07:03:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 -- 172.21.15.32:0/749025903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2ec03f2d0 msgr2=0x7fe2ec0417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2ec03f2d0 0x7fe2ec0417e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40095f0 tx=0x7fe2f4005f90 comp rx=0 tx=0).stop 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 -- 172.21.15.32:0/749025903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe308104770 msgr2=0x7fe30819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe308104770 0x7fe30819a2d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8005ab0 tx=0x7fe2f8003b90 comp rx=0 tx=0).stop 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 -- 172.21.15.32:0/749025903 shutdown_connections 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2ec03f2d0 0x7fe2ec0417e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe30810dbd0 0x7fe30819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3081050c0 0x7fe30819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 --2- 172.21.15.32:0/749025903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe308104770 0x7fe30819a2d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 -- 172.21.15.32:0/749025903 >> 172.21.15.32:0/749025903 conn(0x7fe3080794e0 msgr2=0x7fe3080ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 -- 172.21.15.32:0/749025903 shutdown_connections 2024-08-31T07:03:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:57.094+0000 7fe30e200640 1 -- 172.21.15.32:0/749025903 wait complete. 2024-08-31T07:03:01.495 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:01 smithi084 ceph-mon[22387]: mgrmap e18: x(active, since 15s), standbys: z 2024-08-31T07:03:01.495 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-31T07:03:01.495 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:01.495 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:01 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:01.495 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:01 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:01.495 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:01 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:02.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:02 smithi032 ceph-mon[17252]: Standby manager daemon y started 2024-08-31T07:03:02.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:02 smithi032 ceph-mon[17252]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-31T07:03:02.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:02 smithi032 ceph-mon[17252]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T07:03:02.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:02 smithi032 ceph-mon[17252]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-31T07:03:02.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:02 smithi032 ceph-mon[17252]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T07:03:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.085+0000 7f1262c00640 1 -- 172.21.15.32:0/2786478949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f125c105b20 msgr2=0x7f125c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.085+0000 7f1262c00640 1 --2- 172.21.15.32:0/2786478949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f125c105b20 0x7f125c105f30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f124c009900 tx=0x7f124c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.085+0000 7f1262c00640 1 -- 172.21.15.32:0/2786478949 shutdown_connections 2024-08-31T07:03:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.085+0000 7f1262c00640 1 --2- 172.21.15.32:0/2786478949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f125c10bdb0 0x7f125c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.085+0000 7f1262c00640 1 --2- 172.21.15.32:0/2786478949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c10b460 0x7f125c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.085+0000 7f1262c00640 1 --2- 172.21.15.32:0/2786478949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f125c105b20 0x7f125c105f30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.085+0000 7f1262c00640 1 -- 172.21.15.32:0/2786478949 >> 172.21.15.32:0/2786478949 conn(0x7f125c1015b0 msgr2=0x7f125c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.085+0000 7f1262c00640 1 -- 172.21.15.32:0/2786478949 shutdown_connections 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.085+0000 7f1262c00640 1 -- 172.21.15.32:0/2786478949 wait complete. 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.086+0000 7f1262c00640 1 Processor -- start 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.086+0000 7f1262c00640 1 -- start start 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f1262c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c105b20 0x7f125c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f1262c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f125c10b460 0x7f125c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f1262c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f125c10bdb0 0x7f125c1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f1262c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f125c115ac0 con 0x7f125c105b20 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f1262c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f125c115940 con 0x7f125c10bdb0 2024-08-31T07:03:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f1262c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f125c115c40 con 0x7f125c10b460 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f125be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c105b20 0x7f125c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f125be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c105b20 0x7f125c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44024/0 (socket says 172.21.15.32:44024) 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f125be00640 1 -- 172.21.15.32:0/1534898324 learned_addr learned my addr 172.21.15.32:0/1534898324 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f1260c00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f125c10bdb0 0x7f125c1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.087+0000 7f1253e00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f125c10b460 0x7f125c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f125be00640 1 -- 172.21.15.32:0/1534898324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f125c10b460 msgr2=0x7f125c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f125be00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f125c10b460 0x7f125c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f125be00640 1 -- 172.21.15.32:0/1534898324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f125c10bdb0 msgr2=0x7f125c1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f125be00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f125c10bdb0 0x7f125c1a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f125be00640 1 -- 172.21.15.32:0/1534898324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c1a23c0 con 0x7f125c105b20 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f1260c00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f125c10bdb0 0x7f125c1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:03:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f125be00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c105b20 0x7f125c19c450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f124c005ab0 tx=0x7f124c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f1259600640 1 -- 172.21.15.32:0/1534898324 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c03f070 con 0x7f125c105b20 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f1259600640 1 -- 172.21.15.32:0/1534898324 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f124c031d60 con 0x7f125c105b20 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f1262c00640 1 -- 172.21.15.32:0/1534898324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c1a2650 con 0x7f125c105b20 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f1259600640 1 -- 172.21.15.32:0/1534898324 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c035030 con 0x7f125c105b20 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.088+0000 7f1262c00640 1 -- 172.21.15.32:0/1534898324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1a8e10 con 0x7f125c105b20 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.090+0000 7f1262c00640 1 -- 172.21.15.32:0/1534898324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125c105f30 con 0x7f125c105b20 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.091+0000 7f1259600640 1 -- 172.21.15.32:0/1534898324 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) ==== 101904+0+0 (secure 0 0 0) 0x7f124c042070 con 0x7f125c105b20 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.092+0000 7f1259600640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1240079ee0 0x7f124007c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.092+0000 7f1259600640 1 -- 172.21.15.32:0/1534898324 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f124c0bc030 con 0x7f125c105b20 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.092+0000 7f1253e00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1240079ee0 0x7f124007c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.092+0000 7f1253e00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1240079ee0 0x7f124007c3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f125c10af30 tx=0x7f1244005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.097+0000 7f1259600640 1 -- 172.21.15.32:0/1534898324 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f124c084790 con 0x7f125c105b20 2024-08-31T07:03:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.122+0000 7f1259600640 1 -- 172.21.15.32:0/1534898324 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f124c080a20 con 0x7f125c105b20 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.306+0000 7f1262c00640 1 -- 172.21.15.32:0/1534898324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f125c10db80 con 0x7f125c105b20 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.310+0000 7f1259600640 1 -- 172.21.15.32:0/1534898324 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v24)=0 v24) ==== 125+0+0 (secure 0 0 0) 0x7f124c0b83e0 con 0x7f125c105b20 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 -- 172.21.15.32:0/1534898324 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1240079ee0 msgr2=0x7f124007c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1240079ee0 0x7f124007c3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f125c10af30 tx=0x7f1244005f90 comp rx=0 tx=0).stop 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 -- 172.21.15.32:0/1534898324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c105b20 msgr2=0x7f125c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c105b20 0x7f125c19c450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f124c005ab0 tx=0x7f124c003c80 comp rx=0 tx=0).stop 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 -- 172.21.15.32:0/1534898324 shutdown_connections 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1240079ee0 0x7f124007c3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f125c10bdb0 0x7f125c1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f125c10b460 0x7f125c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 --2- 172.21.15.32:0/1534898324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f125c105b20 0x7f125c19c450 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 -- 172.21.15.32:0/1534898324 >> 172.21.15.32:0/1534898324 conn(0x7f125c1015b0 msgr2=0x7f125c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:02.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 -- 172.21.15.32:0/1534898324 shutdown_connections 2024-08-31T07:03:02.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:02.313+0000 7f1262c00640 1 -- 172.21.15.32:0/1534898324 wait complete. 2024-08-31T07:03:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:02 smithi073 ceph-mon[21995]: Standby manager daemon y started 2024-08-31T07:03:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:02 smithi073 ceph-mon[21995]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-31T07:03:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:02 smithi073 ceph-mon[21995]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T07:03:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:02 smithi073 ceph-mon[21995]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-31T07:03:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:02 smithi073 ceph-mon[21995]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T07:03:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:02 smithi084 ceph-mon[22387]: Standby manager daemon y started 2024-08-31T07:03:02.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:02 smithi084 ceph-mon[22387]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-31T07:03:02.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:02 smithi084 ceph-mon[22387]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T07:03:02.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:02 smithi084 ceph-mon[22387]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-31T07:03:02.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:02 smithi084 ceph-mon[22387]: from='mgr.? 172.21.15.73:0/3738459016' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T07:03:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.907+0000 7fc477200640 1 -- 172.21.15.32:0/752616761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4781099e0 msgr2=0x7fc478110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.907+0000 7fc477200640 1 --2- 172.21.15.32:0/752616761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4781099e0 0x7fc478110210 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc470009ca0 tx=0x7fc47002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.907+0000 7fc477200640 1 -- 172.21.15.32:0/752616761 shutdown_connections 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.907+0000 7fc477200640 1 --2- 172.21.15.32:0/752616761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4781099e0 0x7fc478110210 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.907+0000 7fc477200640 1 --2- 172.21.15.32:0/752616761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc478109090 0x7fc4781094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.907+0000 7fc477200640 1 --2- 172.21.15.32:0/752616761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc478103750 0x7fc478103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.907+0000 7fc477200640 1 -- 172.21.15.32:0/752616761 >> 172.21.15.32:0/752616761 conn(0x7fc4780ff1e0 msgr2=0x7fc478101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.907+0000 7fc477200640 1 -- 172.21.15.32:0/752616761 shutdown_connections 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.907+0000 7fc477200640 1 -- 172.21.15.32:0/752616761 wait complete. 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.908+0000 7fc477200640 1 Processor -- start 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.908+0000 7fc477200640 1 -- start start 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc477200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc478103750 0x7fc47819c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc477200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc478109090 0x7fc47819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc477200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4781099e0 0x7fc4781a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc477200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781136b0 con 0x7fc478109090 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc477200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc478113530 con 0x7fc478103750 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc477200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc478113830 con 0x7fc4781099e0 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc475e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc478103750 0x7fc47819c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc475e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc478103750 0x7fc47819c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59776/0 (socket says 172.21.15.32:59776) 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc475e00640 1 -- 172.21.15.32:0/1398345555 learned_addr learned my addr 172.21.15.32:0/1398345555 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc475400640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc478109090 0x7fc47819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc476800640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4781099e0 0x7fc4781a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc475e00640 1 -- 172.21.15.32:0/1398345555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4781099e0 msgr2=0x7fc4781a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc475e00640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4781099e0 0x7fc4781a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc475e00640 1 -- 172.21.15.32:0/1398345555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc478109090 msgr2=0x7fc47819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc475e00640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc478109090 0x7fc47819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc475e00640 1 -- 172.21.15.32:0/1398345555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4781a21b0 con 0x7fc478103750 2024-08-31T07:03:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.909+0000 7fc476800640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4781099e0 0x7fc4781a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.910+0000 7fc475e00640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc478103750 0x7fc47819c240 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc46800e9f0 tx=0x7fc46800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.910+0000 7fc46ea00640 1 -- 172.21.15.32:0/1398345555 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46800cd80 con 0x7fc478103750 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.910+0000 7fc46ea00640 1 -- 172.21.15.32:0/1398345555 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4680054d0 con 0x7fc478103750 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.910+0000 7fc477200640 1 -- 172.21.15.32:0/1398345555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4781a24a0 con 0x7fc478103750 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.910+0000 7fc46ea00640 1 -- 172.21.15.32:0/1398345555 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468010690 con 0x7fc478103750 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.910+0000 7fc477200640 1 -- 172.21.15.32:0/1398345555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4781a8c00 con 0x7fc478103750 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.911+0000 7fc46ea00640 1 -- 172.21.15.32:0/1398345555 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7fc468010830 con 0x7fc478103750 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.911+0000 7fc477200640 1 -- 172.21.15.32:0/1398345555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc478103b60 con 0x7fc478103750 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.912+0000 7fc46ea00640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc45c03f2d0 0x7fc45c0417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.912+0000 7fc46ea00640 1 -- 172.21.15.32:0/1398345555 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc4680540f0 con 0x7fc478103750 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.912+0000 7fc475400640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc45c03f2d0 0x7fc45c0417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.913+0000 7fc475400640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc45c03f2d0 0x7fc45c0417e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc4640095f0 tx=0x7fc464005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:02:59.917+0000 7fc46ea00640 1 -- 172.21.15.32:0/1398345555 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc468019de0 con 0x7fc478103750 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.111+0000 7fc46ea00640 1 -- 172.21.15.32:0/1398345555 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 18) ==== 101904+0+0 (secure 0 0 0) 0x7fc4680199c0 con 0x7fc478103750 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.124+0000 7fc477200640 1 -- 172.21.15.32:0/1398345555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fc4781a8f40 con 0x7fc478103750 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.129+0000 7fc46ea00640 1 -- 172.21.15.32:0/1398345555 <== mon.2 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v21)=0 v21) ==== 120+0+0 (secure 0 0 0) 0x7fc468067e50 con 0x7fc478103750 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 -- 172.21.15.32:0/1398345555 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc45c03f2d0 msgr2=0x7fc45c0417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc45c03f2d0 0x7fc45c0417e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc4640095f0 tx=0x7fc464005f90 comp rx=0 tx=0).stop 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 -- 172.21.15.32:0/1398345555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc478103750 msgr2=0x7fc47819c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc478103750 0x7fc47819c240 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc46800e9f0 tx=0x7fc46800edf0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 -- 172.21.15.32:0/1398345555 shutdown_connections 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc45c03f2d0 0x7fc45c0417e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4781099e0 0x7fc4781a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc478109090 0x7fc47819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 --2- 172.21.15.32:0/1398345555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc478103750 0x7fc47819c240 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:02.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 -- 172.21.15.32:0/1398345555 >> 172.21.15.32:0/1398345555 conn(0x7fc4780ff1e0 msgr2=0x7fc478106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:02.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 -- 172.21.15.32:0/1398345555 shutdown_connections 2024-08-31T07:03:02.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:00.132+0000 7fc477200640 1 -- 172.21.15.32:0/1398345555 wait complete. 2024-08-31T07:03:02.671 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:03:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:03:02.904 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:03:02.905 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:03:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:03:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.192+0000 7fa7bb200640 1 -- 172.21.15.32:0/3618447066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b410fec0 msgr2=0x7fa7b41122e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.192+0000 7fa7bb200640 1 --2- 172.21.15.32:0/3618447066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b410fec0 0x7fa7b41122e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009ca0 tx=0x7fa7ac02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:03:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.192+0000 7fa7bb200640 1 -- 172.21.15.32:0/3618447066 shutdown_connections 2024-08-31T07:03:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.192+0000 7fa7bb200640 1 --2- 172.21.15.32:0/3618447066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b410fec0 0x7fa7b41122e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.192+0000 7fa7bb200640 1 --2- 172.21.15.32:0/3618447066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b4075e40 0x7fa7b410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.192+0000 7fa7bb200640 1 --2- 172.21.15.32:0/3618447066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7b40754f0 0x7fa7b4075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.192+0000 7fa7bb200640 1 -- 172.21.15.32:0/3618447066 >> 172.21.15.32:0/3618447066 conn(0x7fa7b407b5f0 msgr2=0x7fa7b407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:03.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.192+0000 7fa7bb200640 1 -- 172.21.15.32:0/3618447066 shutdown_connections 2024-08-31T07:03:03.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.192+0000 7fa7bb200640 1 -- 172.21.15.32:0/3618447066 wait complete. 2024-08-31T07:03:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.193+0000 7fa7bb200640 1 Processor -- start 2024-08-31T07:03:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7bb200640 1 -- start start 2024-08-31T07:03:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7bb200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b40754f0 0x7fa7b419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7bb200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7b4075e40 0x7fa7b419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7bb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b410fec0 0x7fa7b41a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7bb200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4115a80 con 0x7fa7b40754f0 2024-08-31T07:03:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7bb200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4115900 con 0x7fa7b410fec0 2024-08-31T07:03:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7bb200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4115c00 con 0x7fa7b4075e40 2024-08-31T07:03:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b40754f0 0x7fa7b419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b40754f0 0x7fa7b419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44040/0 (socket says 172.21.15.32:44040) 2024-08-31T07:03:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7b3e00640 1 -- 172.21.15.32:0/1753469431 learned_addr learned my addr 172.21.15.32:0/1753469431 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7b3400640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7b4075e40 0x7fa7b419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.194+0000 7fa7b9200640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b410fec0 0x7fa7b41a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7b3e00640 1 -- 172.21.15.32:0/1753469431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7b4075e40 msgr2=0x7fa7b419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7b3e00640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7b4075e40 0x7fa7b419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7b3e00640 1 -- 172.21.15.32:0/1753469431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b410fec0 msgr2=0x7fa7b41a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7b3e00640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b410fec0 0x7fa7b41a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7b3e00640 1 -- 172.21.15.32:0/1753469431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b41a2380 con 0x7fa7b40754f0 2024-08-31T07:03:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7b9200640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b410fec0 0x7fa7b41a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:03:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7b3e00640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b40754f0 0x7fa7b419c410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400d950 tx=0x7fa7a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7b0c00640 1 -- 172.21.15.32:0/1753469431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4017070 con 0x7fa7b40754f0 2024-08-31T07:03:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7b0c00640 1 -- 172.21.15.32:0/1753469431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a4009c30 con 0x7fa7b40754f0 2024-08-31T07:03:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7b0c00640 1 -- 172.21.15.32:0/1753469431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400f620 con 0x7fa7b40754f0 2024-08-31T07:03:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.195+0000 7fa7bb200640 1 -- 172.21.15.32:0/1753469431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b41a2670 con 0x7fa7b40754f0 2024-08-31T07:03:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.196+0000 7fa7bb200640 1 -- 172.21.15.32:0/1753469431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b41a8dd0 con 0x7fa7b40754f0 2024-08-31T07:03:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.197+0000 7fa7bb200640 1 -- 172.21.15.32:0/1753469431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b4075900 con 0x7fa7b40754f0 2024-08-31T07:03:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.200+0000 7fa7b0c00640 1 -- 172.21.15.32:0/1753469431 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fa7a4005030 con 0x7fa7b40754f0 2024-08-31T07:03:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.201+0000 7fa7b0c00640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7980b4b70 0x7fa7980b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.201+0000 7fa7b0c00640 1 -- 172.21.15.32:0/1753469431 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa7a40e3650 con 0x7fa7b40754f0 2024-08-31T07:03:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.201+0000 7fa7b3400640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7980b4b70 0x7fa7980b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.202+0000 7fa7b3400640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7980b4b70 0x7fa7980b7080 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4111c30 tx=0x7fa7a000b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.202+0000 7fa7b0c00640 1 -- 172.21.15.32:0/1753469431 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa7a4020070 con 0x7fa7b40754f0 2024-08-31T07:03:03.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:03 smithi032 ceph-mon[17252]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-08-31T07:03:03.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:03 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-31T07:03:03.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:03 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1534898324' entity='client.admin' 2024-08-31T07:03:03.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.402+0000 7fa7bb200640 1 -- 172.21.15.32:0/1753469431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fa7b41a90c0 con 0x7fa7b40754f0 2024-08-31T07:03:03.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.407+0000 7fa7b0c00640 1 -- 172.21.15.32:0/1753469431 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7fa7a40ac360 con 0x7fa7b40754f0 2024-08-31T07:03:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.409+0000 7fa7bb200640 1 -- 172.21.15.32:0/1753469431 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7980b4b70 msgr2=0x7fa7980b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.409+0000 7fa7bb200640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7980b4b70 0x7fa7980b7080 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4111c30 tx=0x7fa7a000b410 comp rx=0 tx=0).stop 2024-08-31T07:03:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.409+0000 7fa7bb200640 1 -- 172.21.15.32:0/1753469431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b40754f0 msgr2=0x7fa7b419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.409+0000 7fa7bb200640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b40754f0 0x7fa7b419c410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400d950 tx=0x7fa7a400de10 comp rx=0 tx=0).stop 2024-08-31T07:03:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.410+0000 7fa7bb200640 1 -- 172.21.15.32:0/1753469431 shutdown_connections 2024-08-31T07:03:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.410+0000 7fa7bb200640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7980b4b70 0x7fa7980b7080 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.410+0000 7fa7bb200640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b410fec0 0x7fa7b41a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.410+0000 7fa7bb200640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7b4075e40 0x7fa7b419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.410+0000 7fa7bb200640 1 --2- 172.21.15.32:0/1753469431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b40754f0 0x7fa7b419c410 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.410+0000 7fa7bb200640 1 -- 172.21.15.32:0/1753469431 >> 172.21.15.32:0/1753469431 conn(0x7fa7b407b5f0 msgr2=0x7fa7b41099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.410+0000 7fa7bb200640 1 -- 172.21.15.32:0/1753469431 shutdown_connections 2024-08-31T07:03:03.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:03.410+0000 7fa7bb200640 1 -- 172.21.15.32:0/1753469431 wait complete. 2024-08-31T07:03:03.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:03 smithi073 ceph-mon[21995]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-08-31T07:03:03.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:03 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-31T07:03:03.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:03 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1534898324' entity='client.admin' 2024-08-31T07:03:03.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:03 smithi084 ceph-mon[22387]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-08-31T07:03:03.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:03 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-31T07:03:03.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:03 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1534898324' entity='client.admin' 2024-08-31T07:03:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.084+0000 7f8d8e800640 1 -- 172.21.15.32:0/2360245728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d88109240 msgr2=0x7f8d88109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.084+0000 7f8d8e800640 1 --2- 172.21.15.32:0/2360245728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d88109240 0x7f8d88109650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009900 tx=0x7f8d7402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.084+0000 7f8d8e800640 1 -- 172.21.15.32:0/2360245728 shutdown_connections 2024-08-31T07:03:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.084+0000 7f8d8e800640 1 --2- 172.21.15.32:0/2360245728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d88109b90 0x7f8d881103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.084+0000 7f8d8e800640 1 --2- 172.21.15.32:0/2360245728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d88109240 0x7f8d88109650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.084+0000 7f8d8e800640 1 --2- 172.21.15.32:0/2360245728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d88103900 0x7f8d88103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.084+0000 7f8d8e800640 1 -- 172.21.15.32:0/2360245728 >> 172.21.15.32:0/2360245728 conn(0x7f8d8807b5f0 msgr2=0x7f8d8807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.084+0000 7f8d8e800640 1 -- 172.21.15.32:0/2360245728 shutdown_connections 2024-08-31T07:03:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.084+0000 7f8d8e800640 1 -- 172.21.15.32:0/2360245728 wait complete. 2024-08-31T07:03:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.085+0000 7f8d8e800640 1 Processor -- start 2024-08-31T07:03:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d8e800640 1 -- start start 2024-08-31T07:03:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d8e800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d88103900 0x7f8d8819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d8e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d88109240 0x7f8d8819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d8e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d88109b90 0x7f8d881a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d8e800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8d88113850 con 0x7f8d88109b90 2024-08-31T07:03:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d8e800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881136d0 con 0x7f8d88109240 2024-08-31T07:03:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d8e800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881139d0 con 0x7f8d88103900 2024-08-31T07:03:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d88109b90 0x7f8d881a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d88109b90 0x7f8d881a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44056/0 (socket says 172.21.15.32:44056) 2024-08-31T07:03:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d87e00640 1 -- 172.21.15.32:0/1021458132 learned_addr learned my addr 172.21.15.32:0/1021458132 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d86a00640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d88109240 0x7f8d8819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.086+0000 7f8d87400640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d88103900 0x7f8d8819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.087+0000 7f8d87e00640 1 -- 172.21.15.32:0/1021458132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d88103900 msgr2=0x7f8d8819c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.087+0000 7f8d87e00640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d88103900 0x7f8d8819c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.087+0000 7f8d87e00640 1 -- 172.21.15.32:0/1021458132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d88109240 msgr2=0x7f8d8819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.087+0000 7f8d87e00640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d88109240 0x7f8d8819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.087+0000 7f8d87e00640 1 -- 172.21.15.32:0/1021458132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d881a2390 con 0x7f8d88109b90 2024-08-31T07:03:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.087+0000 7f8d87e00640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d88109b90 0x7f8d881a1c60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000eaa0 tx=0x7f8d8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.087+0000 7f8d7fe00640 1 -- 172.21.15.32:0/1021458132 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8000cdb0 con 0x7f8d88109b90 2024-08-31T07:03:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.088+0000 7f8d7fe00640 1 -- 172.21.15.32:0/1021458132 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d80005020 con 0x7f8d88109b90 2024-08-31T07:03:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.088+0000 7f8d8e800640 1 -- 172.21.15.32:0/1021458132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d881a2680 con 0x7f8d88109b90 2024-08-31T07:03:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.088+0000 7f8d7fe00640 1 -- 172.21.15.32:0/1021458132 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d800116d0 con 0x7f8d88109b90 2024-08-31T07:03:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.088+0000 7f8d8e800640 1 -- 172.21.15.32:0/1021458132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d881a8de0 con 0x7f8d88109b90 2024-08-31T07:03:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.089+0000 7f8d8e800640 1 -- 172.21.15.32:0/1021458132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d88103d10 con 0x7f8d88109b90 2024-08-31T07:03:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.091+0000 7f8d7fe00640 1 -- 172.21.15.32:0/1021458132 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f8d800051d0 con 0x7f8d88109b90 2024-08-31T07:03:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.092+0000 7f8d7fe00640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d6c0b4b70 0x7f8d6c0b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.093+0000 7f8d7fe00640 1 -- 172.21.15.32:0/1021458132 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8d800e3640 con 0x7f8d88109b90 2024-08-31T07:03:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.093+0000 7f8d87400640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d6c0b4b70 0x7f8d6c0b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:04.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.093+0000 7f8d87400640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d6c0b4b70 0x7f8d6c0b7080 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8d78005d00 tx=0x7f8d7800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:04.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.097+0000 7f8d7fe00640 1 -- 172.21.15.32:0/1021458132 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8d80016030 con 0x7f8d88109b90 2024-08-31T07:03:04.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.301+0000 7f8d8e800640 1 -- 172.21.15.32:0/1021458132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f8d881a9080 con 0x7f8d88109b90 2024-08-31T07:03:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.305+0000 7f8d7fe00640 1 -- 172.21.15.32:0/1021458132 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v26)=0 v26) ==== 133+0+0 (secure 0 0 0) 0x7f8d800abc80 con 0x7f8d88109b90 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f8d8e800640 1 -- 172.21.15.32:0/1021458132 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d6c0b4b70 msgr2=0x7f8d6c0b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f8d8e800640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d6c0b4b70 0x7f8d6c0b7080 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8d78005d00 tx=0x7f8d7800b410 comp rx=0 tx=0).stop 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f8d8e800640 1 -- 172.21.15.32:0/1021458132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d88109b90 msgr2=0x7f8d881a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f8d8e800640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d88109b90 0x7f8d881a1c60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000eaa0 tx=0x7f8d8000ef60 comp rx=0 tx=0).stop 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.309+0000 7f8d8e800640 1 -- 172.21.15.32:0/1021458132 shutdown_connections 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.309+0000 7f8d8e800640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d6c0b4b70 0x7f8d6c0b7080 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.309+0000 7f8d8e800640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d88109b90 0x7f8d881a1c60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.309+0000 7f8d8e800640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d88109240 0x7f8d8819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.309+0000 7f8d8e800640 1 --2- 172.21.15.32:0/1021458132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d88103900 0x7f8d8819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.309+0000 7f8d8e800640 1 -- 172.21.15.32:0/1021458132 >> 172.21.15.32:0/1021458132 conn(0x7f8d8807b5f0 msgr2=0x7f8d88105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.309+0000 7f8d8e800640 1 -- 172.21.15.32:0/1021458132 shutdown_connections 2024-08-31T07:03:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.309+0000 7f8d8e800640 1 -- 172.21.15.32:0/1021458132 wait complete. 2024-08-31T07:03:04.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:04 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1753469431' entity='client.admin' 2024-08-31T07:03:04.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:04 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1021458132' entity='client.admin' 2024-08-31T07:03:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:04 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1753469431' entity='client.admin' 2024-08-31T07:03:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:04 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1021458132' entity='client.admin' 2024-08-31T07:03:04.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:04 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1753469431' entity='client.admin' 2024-08-31T07:03:04.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:04 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1021458132' entity='client.admin' 2024-08-31T07:03:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f95f5a00640 1 -- 172.21.15.32:0/1066543507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95f0075550 msgr2=0x7f95f0075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f95f5a00640 1 --2- 172.21.15.32:0/1066543507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95f0075550 0x7f95f0075960 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009900 tx=0x7f95e002a410 comp rx=0 tx=0).stop 2024-08-31T07:03:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f95f5a00640 1 -- 172.21.15.32:0/1066543507 shutdown_connections 2024-08-31T07:03:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f95f5a00640 1 --2- 172.21.15.32:0/1066543507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f010fcb0 0x7f95f01120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f95f5a00640 1 --2- 172.21.15.32:0/1066543507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f0075ea0 0x7f95f010f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f95f5a00640 1 --2- 172.21.15.32:0/1066543507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95f0075550 0x7f95f0075960 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f95f5a00640 1 -- 172.21.15.32:0/1066543507 >> 172.21.15.32:0/1066543507 conn(0x7f95f00ff1e0 msgr2=0x7f95f0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f95f5a00640 1 -- 172.21.15.32:0/1066543507 shutdown_connections 2024-08-31T07:03:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.308+0000 7f95f5a00640 1 -- 172.21.15.32:0/1066543507 wait complete. 2024-08-31T07:03:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.310+0000 7f95f5a00640 1 Processor -- start 2024-08-31T07:03:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.310+0000 7f95f5a00640 1 -- start start 2024-08-31T07:03:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.310+0000 7f95f5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f0075550 0x7f95f019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.310+0000 7f95f5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f0075ea0 0x7f95f019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.310+0000 7f95f5a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95f010fcb0 0x7f95f01a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.310+0000 7f95f5a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01158b0 con 0x7f95f0075ea0 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.310+0000 7f95f5a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0115730 con 0x7f95f0075550 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.310+0000 7f95f5a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0115a30 con 0x7f95f010fcb0 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.311+0000 7f95ef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f0075ea0 0x7f95f019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.311+0000 7f95ef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f0075ea0 0x7f95f019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44078/0 (socket says 172.21.15.32:44078) 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.311+0000 7f95ef400640 1 -- 172.21.15.32:0/3571869106 learned_addr learned my addr 172.21.15.32:0/3571869106 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.311+0000 7f95efe00640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f0075550 0x7f95f019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.311+0000 7f95f5000640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95f010fcb0 0x7f95f01a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.311+0000 7f95f5000640 1 -- 172.21.15.32:0/3571869106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f0075550 msgr2=0x7f95f019c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.311+0000 7f95f5000640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f0075550 0x7f95f019c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.311+0000 7f95f5000640 1 -- 172.21.15.32:0/3571869106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f0075ea0 msgr2=0x7f95f019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.311+0000 7f95f5000640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f0075ea0 0x7f95f019c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.311+0000 7f95f5000640 1 -- 172.21.15.32:0/3571869106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f01a21d0 con 0x7f95f010fcb0 2024-08-31T07:03:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.312+0000 7f95f5000640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95f010fcb0 0x7f95f01a1aa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009fd0 tx=0x7f95e800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.312+0000 7f95ecc00640 1 -- 172.21.15.32:0/3571869106 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e800ce20 con 0x7f95f010fcb0 2024-08-31T07:03:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.312+0000 7f95f5a00640 1 -- 172.21.15.32:0/3571869106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f01a24c0 con 0x7f95f010fcb0 2024-08-31T07:03:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.312+0000 7f95ecc00640 1 -- 172.21.15.32:0/3571869106 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e8005020 con 0x7f95f010fcb0 2024-08-31T07:03:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.312+0000 7f95ecc00640 1 -- 172.21.15.32:0/3571869106 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8010690 con 0x7f95f010fcb0 2024-08-31T07:03:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.312+0000 7f95f5a00640 1 -- 172.21.15.32:0/3571869106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f01a8c20 con 0x7f95f010fcb0 2024-08-31T07:03:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.314+0000 7f95f5a00640 1 -- 172.21.15.32:0/3571869106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f0075960 con 0x7f95f010fcb0 2024-08-31T07:03:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.316+0000 7f95ecc00640 1 -- 172.21.15.32:0/3571869106 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f95e8010830 con 0x7f95f010fcb0 2024-08-31T07:03:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.318+0000 7f95ecc00640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f95d40b4b70 0x7f95d40b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.318+0000 7f95ecc00640 1 -- 172.21.15.32:0/3571869106 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f95e80e3c40 con 0x7f95f010fcb0 2024-08-31T07:03:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.318+0000 7f95efe00640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f95d40b4b70 0x7f95d40b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.319+0000 7f95efe00640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f95d40b4b70 0x7f95d40b7080 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f95e0005ab0 tx=0x7f95e0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.321+0000 7f95ecc00640 1 -- 172.21.15.32:0/3571869106 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f95e80ac8d0 con 0x7f95f010fcb0 2024-08-31T07:03:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.526+0000 7f95f5a00640 1 -- 172.21.15.32:0/3571869106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f95f01a8e50 con 0x7f95f010fcb0 2024-08-31T07:03:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.532+0000 7f95ecc00640 1 -- 172.21.15.32:0/3571869106 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v27)=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7f95e80ac1d0 con 0x7f95f010fcb0 2024-08-31T07:03:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 -- 172.21.15.32:0/3571869106 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f95d40b4b70 msgr2=0x7f95d40b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f95d40b4b70 0x7f95d40b7080 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f95e0005ab0 tx=0x7f95e0005a20 comp rx=0 tx=0).stop 2024-08-31T07:03:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 -- 172.21.15.32:0/3571869106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95f010fcb0 msgr2=0x7f95f01a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95f010fcb0 0x7f95f01a1aa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009fd0 tx=0x7f95e800ef10 comp rx=0 tx=0).stop 2024-08-31T07:03:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 -- 172.21.15.32:0/3571869106 shutdown_connections 2024-08-31T07:03:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f95d40b4b70 0x7f95d40b7080 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95f010fcb0 0x7f95f01a1aa0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f0075ea0 0x7f95f019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 --2- 172.21.15.32:0/3571869106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95f0075550 0x7f95f019c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 -- 172.21.15.32:0/3571869106 >> 172.21.15.32:0/3571869106 conn(0x7f95f00ff1e0 msgr2=0x7f95f01097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 -- 172.21.15.32:0/3571869106 shutdown_connections 2024-08-31T07:03:04.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.535+0000 7f95f5a00640 1 -- 172.21.15.32:0/3571869106 wait complete. 2024-08-31T07:03:05.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.615+0000 7f5aa5400640 1 -- 172.21.15.32:0/2570976389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa010b470 msgr2=0x7f5aa010b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:05.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.615+0000 7f5aa5400640 1 --2- 172.21.15.32:0/2570976389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa010b470 0x7f5aa010b880 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009900 tx=0x7f5a8802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.615+0000 7f5aa5400640 1 -- 172.21.15.32:0/2570976389 shutdown_connections 2024-08-31T07:03:05.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.615+0000 7f5aa5400640 1 --2- 172.21.15.32:0/2570976389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa010bdc0 0x7f5aa01125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.615+0000 7f5aa5400640 1 --2- 172.21.15.32:0/2570976389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa010b470 0x7f5aa010b880 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.615+0000 7f5aa5400640 1 --2- 172.21.15.32:0/2570976389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aa0105b30 0x7f5aa0105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.615+0000 7f5aa5400640 1 -- 172.21.15.32:0/2570976389 >> 172.21.15.32:0/2570976389 conn(0x7f5aa0101600 msgr2=0x7f5aa0103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:05.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.615+0000 7f5aa5400640 1 -- 172.21.15.32:0/2570976389 shutdown_connections 2024-08-31T07:03:05.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.615+0000 7f5aa5400640 1 -- 172.21.15.32:0/2570976389 wait complete. 2024-08-31T07:03:05.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.616+0000 7f5aa5400640 1 Processor -- start 2024-08-31T07:03:05.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.616+0000 7f5aa5400640 1 -- start start 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5aa5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aa0105b30 0x7f5aa019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5aa5400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa010b470 0x7f5aa019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5aa5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa010bdc0 0x7f5aa01a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5aa5400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0115ad0 con 0x7f5aa010bdc0 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5aa5400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0115950 con 0x7f5aa0105b30 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5aa5400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0115c50 con 0x7f5aa010b470 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5a9f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa010bdc0 0x7f5aa01a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5a9e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aa0105b30 0x7f5aa019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5a9f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa010bdc0 0x7f5aa01a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44114/0 (socket says 172.21.15.32:44114) 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5a9f200640 1 -- 172.21.15.32:0/976051798 learned_addr learned my addr 172.21.15.32:0/976051798 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5a9e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aa0105b30 0x7f5aa019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59926/0 (socket says 172.21.15.32:59926) 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.617+0000 7f5a95e00640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa010b470 0x7f5aa019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.618+0000 7f5a9e800640 1 -- 172.21.15.32:0/976051798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa010b470 msgr2=0x7f5aa019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.618+0000 7f5a9e800640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa010b470 0x7f5aa019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.618+0000 7f5a9e800640 1 -- 172.21.15.32:0/976051798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa010bdc0 msgr2=0x7f5aa01a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.618+0000 7f5a9e800640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa010bdc0 0x7f5aa01a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.618+0000 7f5a9e800640 1 -- 172.21.15.32:0/976051798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa01a23d0 con 0x7f5aa0105b30 2024-08-31T07:03:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.618+0000 7f5a9f200640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa010bdc0 0x7f5aa01a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:03:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.618+0000 7f5a9e800640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aa0105b30 0x7f5aa019c460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000eb00 tx=0x7f5a9000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.618+0000 7f5a97e00640 1 -- 172.21.15.32:0/976051798 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90010430 con 0x7f5aa0105b30 2024-08-31T07:03:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.619+0000 7f5a97e00640 1 -- 172.21.15.32:0/976051798 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a90010a80 con 0x7f5aa0105b30 2024-08-31T07:03:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.619+0000 7f5aa5400640 1 -- 172.21.15.32:0/976051798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa01a26c0 con 0x7f5aa0105b30 2024-08-31T07:03:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.619+0000 7f5a97e00640 1 -- 172.21.15.32:0/976051798 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90017a90 con 0x7f5aa0105b30 2024-08-31T07:03:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.619+0000 7f5aa5400640 1 -- 172.21.15.32:0/976051798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa01a8e20 con 0x7f5aa0105b30 2024-08-31T07:03:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.620+0000 7f5aa5400640 1 -- 172.21.15.32:0/976051798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa0105f40 con 0x7f5aa0105b30 2024-08-31T07:03:05.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.622+0000 7f5a97e00640 1 -- 172.21.15.32:0/976051798 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f5a90017c70 con 0x7f5aa0105b30 2024-08-31T07:03:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.623+0000 7f5a97e00640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a840b4af0 0x7f5a840b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.623+0000 7f5a97e00640 1 -- 172.21.15.32:0/976051798 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5a900e4090 con 0x7f5aa0105b30 2024-08-31T07:03:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.623+0000 7f5a95e00640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a840b4af0 0x7f5a840b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.624+0000 7f5a95e00640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a840b4af0 0x7f5a840b7000 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5a8802a9e0 tx=0x7f5a88005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.627+0000 7f5a97e00640 1 -- 172.21.15.32:0/976051798 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5a900acd20 con 0x7f5aa0105b30 2024-08-31T07:03:05.702 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:05 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:05.702 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:05 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.829+0000 7f5aa5400640 1 -- 172.21.15.32:0/976051798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f5aa002e430 con 0x7f5aa0105b30 2024-08-31T07:03:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.835+0000 7f5a97e00640 1 -- 172.21.15.32:0/976051798 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f5a900b5260 con 0x7f5aa0105b30 2024-08-31T07:03:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 -- 172.21.15.32:0/976051798 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a840b4af0 msgr2=0x7f5a840b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a840b4af0 0x7f5a840b7000 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5a8802a9e0 tx=0x7f5a88005a20 comp rx=0 tx=0).stop 2024-08-31T07:03:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 -- 172.21.15.32:0/976051798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aa0105b30 msgr2=0x7f5aa019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aa0105b30 0x7f5aa019c460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000eb00 tx=0x7f5a9000c2d0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 -- 172.21.15.32:0/976051798 shutdown_connections 2024-08-31T07:03:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a840b4af0 0x7f5a840b7000 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa010bdc0 0x7f5aa01a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5aa010b470 0x7f5aa019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 --2- 172.21.15.32:0/976051798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5aa0105b30 0x7f5aa019c460 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 -- 172.21.15.32:0/976051798 >> 172.21.15.32:0/976051798 conn(0x7f5aa0101600 msgr2=0x7f5aa0108a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 -- 172.21.15.32:0/976051798 shutdown_connections 2024-08-31T07:03:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.838+0000 7f5aa5400640 1 -- 172.21.15.32:0/976051798 wait complete. 2024-08-31T07:03:05.933 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:05 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:05.933 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:05 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:05.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:05 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:05.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:05 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.088+0000 7f3422600640 1 -- 172.21.15.32:0/4031846518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f341c10b290 msgr2=0x7f341c10b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.088+0000 7f3422600640 1 --2- 172.21.15.32:0/4031846518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f341c10b290 0x7f341c10b6a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3408009900 tx=0x7f340802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.088+0000 7f3422600640 1 -- 172.21.15.32:0/4031846518 shutdown_connections 2024-08-31T07:03:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.088+0000 7f3422600640 1 --2- 172.21.15.32:0/4031846518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c10bbe0 0x7f341c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.088+0000 7f3422600640 1 --2- 172.21.15.32:0/4031846518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f341c10b290 0x7f341c10b6a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.088+0000 7f3422600640 1 --2- 172.21.15.32:0/4031846518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f341c105950 0x7f341c105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.088+0000 7f3422600640 1 -- 172.21.15.32:0/4031846518 >> 172.21.15.32:0/4031846518 conn(0x7f341c101420 msgr2=0x7f341c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.088+0000 7f3422600640 1 -- 172.21.15.32:0/4031846518 shutdown_connections 2024-08-31T07:03:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.088+0000 7f3422600640 1 -- 172.21.15.32:0/4031846518 wait complete. 2024-08-31T07:03:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3422600640 1 Processor -- start 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3422600640 1 -- start start 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3422600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c105950 0x7f341c19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3422600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f341c10b290 0x7f341c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3422600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f341c10bbe0 0x7f341c1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3422600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f341c1158f0 con 0x7f341c10bbe0 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3422600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f341c115770 con 0x7f341c10b290 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3422600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f341c115a70 con 0x7f341c105950 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3421c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f341c10bbe0 0x7f341c1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3421200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c105950 0x7f341c19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3421200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c105950 0x7f341c19c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58800/0 (socket says 172.21.15.32:58800) 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f341be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f341c10b290 0x7f341c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3421c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f341c10bbe0 0x7f341c1a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36284/0 (socket says 172.21.15.32:36284) 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.090+0000 7f3421200640 1 -- 172.21.15.32:0/4214767996 learned_addr learned my addr 172.21.15.32:0/4214767996 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.091+0000 7f3421200640 1 -- 172.21.15.32:0/4214767996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f341c10b290 msgr2=0x7f341c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.091+0000 7f3421200640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f341c10b290 0x7f341c19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.091+0000 7f3421200640 1 -- 172.21.15.32:0/4214767996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f341c10bbe0 msgr2=0x7f341c1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.091+0000 7f3421200640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f341c10bbe0 0x7f341c1a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.091+0000 7f3421200640 1 -- 172.21.15.32:0/4214767996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341c1a2210 con 0x7f341c105950 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.091+0000 7f3421c00640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f341c10bbe0 0x7f341c1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.091+0000 7f3421200640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c105950 0x7f341c19c2a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f341000e9f0 tx=0x7f341000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.091+0000 7f3419600640 1 -- 172.21.15.32:0/4214767996 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341000cd80 con 0x7f341c105950 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.092+0000 7f3422600640 1 -- 172.21.15.32:0/4214767996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341c1a2500 con 0x7f341c105950 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.092+0000 7f3419600640 1 -- 172.21.15.32:0/4214767996 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34100108e0 con 0x7f341c105950 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.092+0000 7f3419600640 1 -- 172.21.15.32:0/4214767996 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410018980 con 0x7f341c105950 2024-08-31T07:03:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.092+0000 7f3422600640 1 -- 172.21.15.32:0/4214767996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341c1a8c60 con 0x7f341c105950 2024-08-31T07:03:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.093+0000 7f3422600640 1 -- 172.21.15.32:0/4214767996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341c105d60 con 0x7f341c105950 2024-08-31T07:03:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.095+0000 7f3419600640 1 -- 172.21.15.32:0/4214767996 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f3410010430 con 0x7f341c105950 2024-08-31T07:03:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.097+0000 7f3419600640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34040b4b70 0x7f34040b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.097+0000 7f3419600640 1 -- 172.21.15.32:0/4214767996 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f34100e3b50 con 0x7f341c105950 2024-08-31T07:03:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.097+0000 7f341be00640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34040b4b70 0x7f34040b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.098+0000 7f341be00640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34040b4b70 0x7f34040b7080 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f340802a9e0 tx=0x7f3408005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.098+0000 7f3419600640 1 -- 172.21.15.32:0/4214767996 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f34100ac190 con 0x7f341c105950 2024-08-31T07:03:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.304+0000 7f3422600640 1 -- 172.21.15.32:0/4214767996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f341c02e430 con 0x7f341c105950 2024-08-31T07:03:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.313+0000 7f3419600640 1 -- 172.21.15.32:0/4214767996 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7f34100b0950 con 0x7f341c105950 2024-08-31T07:03:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 -- 172.21.15.32:0/4214767996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34040b4b70 msgr2=0x7f34040b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34040b4b70 0x7f34040b7080 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f340802a9e0 tx=0x7f3408005a20 comp rx=0 tx=0).stop 2024-08-31T07:03:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 -- 172.21.15.32:0/4214767996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c105950 msgr2=0x7f341c19c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c105950 0x7f341c19c2a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f341000e9f0 tx=0x7f341000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 -- 172.21.15.32:0/4214767996 shutdown_connections 2024-08-31T07:03:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34040b4b70 0x7f34040b7080 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f341c10bbe0 0x7f341c1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f341c10b290 0x7f341c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 --2- 172.21.15.32:0/4214767996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f341c105950 0x7f341c19c2a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 -- 172.21.15.32:0/4214767996 >> 172.21.15.32:0/4214767996 conn(0x7f341c101420 msgr2=0x7f341c10a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 -- 172.21.15.32:0/4214767996 shutdown_connections 2024-08-31T07:03:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:06.316+0000 7f3422600640 1 -- 172.21.15.32:0/4214767996 wait complete. 2024-08-31T07:03:06.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:06 smithi032 ceph-mon[17252]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:06.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:06 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:06.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:06 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:06 smithi073 ceph-mon[21995]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:06 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:06 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:06 smithi084 ceph-mon[22387]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:06 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:06.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:06 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.331+0000 7ff5b5200640 1 -- 172.21.15.32:0/3325746094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b010b470 msgr2=0x7ff5b010b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.331+0000 7ff5b5200640 1 --2- 172.21.15.32:0/3325746094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b010b470 0x7ff5b010b880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009860 tx=0x7ff59c02a340 comp rx=0 tx=0).stop 2024-08-31T07:03:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.331+0000 7ff5b5200640 1 -- 172.21.15.32:0/3325746094 shutdown_connections 2024-08-31T07:03:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.331+0000 7ff5b5200640 1 --2- 172.21.15.32:0/3325746094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b010bdc0 0x7ff5b01125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.331+0000 7ff5b5200640 1 --2- 172.21.15.32:0/3325746094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b010b470 0x7ff5b010b880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.331+0000 7ff5b5200640 1 --2- 172.21.15.32:0/3325746094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5b0105b30 0x7ff5b0105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.331+0000 7ff5b5200640 1 -- 172.21.15.32:0/3325746094 >> 172.21.15.32:0/3325746094 conn(0x7ff5b0101600 msgr2=0x7ff5b0103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.331+0000 7ff5b5200640 1 -- 172.21.15.32:0/3325746094 shutdown_connections 2024-08-31T07:03:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.331+0000 7ff5b5200640 1 -- 172.21.15.32:0/3325746094 wait complete. 2024-08-31T07:03:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.332+0000 7ff5b5200640 1 Processor -- start 2024-08-31T07:03:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5b5200640 1 -- start start 2024-08-31T07:03:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5b5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b0105b30 0x7ff5b019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5b5200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b010b470 0x7ff5b019c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5b5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5b010bdc0 0x7ff5b01a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5b5200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0115a80 con 0x7ff5b010bdc0 2024-08-31T07:03:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5b5200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0115900 con 0x7ff5b0105b30 2024-08-31T07:03:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5b5200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0115c00 con 0x7ff5b010b470 2024-08-31T07:03:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5ade00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b0105b30 0x7ff5b019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5ade00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b0105b30 0x7ff5b019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52788/0 (socket says 172.21.15.32:52788) 2024-08-31T07:03:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5ade00640 1 -- 172.21.15.32:0/1730996609 learned_addr learned my addr 172.21.15.32:0/1730996609 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.333+0000 7ff5ae800640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5b010bdc0 0x7ff5b01a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5ad400640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b010b470 0x7ff5b019c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5ade00640 1 -- 172.21.15.32:0/1730996609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b010b470 msgr2=0x7ff5b019c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5ade00640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b010b470 0x7ff5b019c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5ade00640 1 -- 172.21.15.32:0/1730996609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5b010bdc0 msgr2=0x7ff5b01a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5ade00640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5b010bdc0 0x7ff5b01a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5ade00640 1 -- 172.21.15.32:0/1730996609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b01a23a0 con 0x7ff5b0105b30 2024-08-31T07:03:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5ad400640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b010b470 0x7ff5b019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:03:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5ade00640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b0105b30 0x7ff5b019c430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000e9c0 tx=0x7ff5a000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5a6a00640 1 -- 172.21.15.32:0/1730996609 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000cde0 con 0x7ff5b0105b30 2024-08-31T07:03:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5a6a00640 1 -- 172.21.15.32:0/1730996609 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a00035f0 con 0x7ff5b0105b30 2024-08-31T07:03:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.334+0000 7ff5a6a00640 1 -- 172.21.15.32:0/1730996609 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0010690 con 0x7ff5b0105b30 2024-08-31T07:03:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.335+0000 7ff5b5200640 1 -- 172.21.15.32:0/1730996609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b01a2690 con 0x7ff5b0105b30 2024-08-31T07:03:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.335+0000 7ff5b5200640 1 -- 172.21.15.32:0/1730996609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b01a8df0 con 0x7ff5b0105b30 2024-08-31T07:03:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.336+0000 7ff5b5200640 1 -- 172.21.15.32:0/1730996609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b0106db0 con 0x7ff5b0105b30 2024-08-31T07:03:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.338+0000 7ff5a6a00640 1 -- 172.21.15.32:0/1730996609 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7ff5a00037a0 con 0x7ff5b0105b30 2024-08-31T07:03:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.340+0000 7ff5a6a00640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5940b4b70 0x7ff5940b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.340+0000 7ff5a6a00640 1 -- 172.21.15.32:0/1730996609 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff5a00e3a30 con 0x7ff5b0105b30 2024-08-31T07:03:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.340+0000 7ff5ad400640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5940b4b70 0x7ff5940b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.341+0000 7ff5ad400640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5940b4b70 0x7ff5940b7080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff59c035040 tx=0x7ff59c030d10 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.341+0000 7ff5a6a00640 1 -- 172.21.15.32:0/1730996609 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff5a00ac750 con 0x7ff5b0105b30 2024-08-31T07:03:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.467+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2342745474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf0109240 msgr2=0x7f2bf0109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.467+0000 7f2bf6c00640 1 --2- 172.21.15.32:0/2342745474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf0109240 0x7f2bf0109650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009900 tx=0x7f2bdc02a410 comp rx=0 tx=0).stop 2024-08-31T07:03:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.467+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2342745474 shutdown_connections 2024-08-31T07:03:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.467+0000 7f2bf6c00640 1 --2- 172.21.15.32:0/2342745474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bf0109b90 0x7f2bf01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.467+0000 7f2bf6c00640 1 --2- 172.21.15.32:0/2342745474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf0109240 0x7f2bf0109650 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.467+0000 7f2bf6c00640 1 --2- 172.21.15.32:0/2342745474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf0103900 0x7f2bf0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.467+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2342745474 >> 172.21.15.32:0/2342745474 conn(0x7f2bf007b5f0 msgr2=0x7f2bf007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.467+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2342745474 shutdown_connections 2024-08-31T07:03:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.467+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2342745474 wait complete. 2024-08-31T07:03:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.468+0000 7f2bf6c00640 1 Processor -- start 2024-08-31T07:03:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2bf6c00640 1 -- start start 2024-08-31T07:03:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2bf6c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf0103900 0x7f2bf019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2bf6c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bf0109240 0x7f2bf019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2bf6c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf0109b90 0x7f2bf01a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2bf6c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0113850 con 0x7f2bf0109b90 2024-08-31T07:03:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2bf6c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01136d0 con 0x7f2bf0103900 2024-08-31T07:03:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2bf6c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01139d0 con 0x7f2bf0109240 2024-08-31T07:03:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2befe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf0103900 0x7f2bf019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2befe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf0103900 0x7f2bf019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52810/0 (socket says 172.21.15.32:52810) 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2befe00640 1 -- 172.21.15.32:0/2558759397 learned_addr learned my addr 172.21.15.32:0/2558759397 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.469+0000 7f2bf4c00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf0109b90 0x7f2bf01a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.470+0000 7f2bef400640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bf0109240 0x7f2bf019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.470+0000 7f2befe00640 1 -- 172.21.15.32:0/2558759397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bf0109240 msgr2=0x7f2bf019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.470+0000 7f2befe00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bf0109240 0x7f2bf019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.470+0000 7f2befe00640 1 -- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf0109b90 msgr2=0x7f2bf01a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.470+0000 7f2befe00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf0109b90 0x7f2bf01a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.470+0000 7f2befe00640 1 -- 172.21.15.32:0/2558759397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf01a23b0 con 0x7f2bf0103900 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.470+0000 7f2bf4c00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf0109b90 0x7f2bf01a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.470+0000 7f2befe00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf0103900 0x7f2bf019c440 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2be000e9f0 tx=0x7f2be000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.470+0000 7f2becc00640 1 -- 172.21.15.32:0/2558759397 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be000cd80 con 0x7f2bf0103900 2024-08-31T07:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.471+0000 7f2becc00640 1 -- 172.21.15.32:0/2558759397 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be00108e0 con 0x7f2bf0103900 2024-08-31T07:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.471+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2558759397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf01a26a0 con 0x7f2bf0103900 2024-08-31T07:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.471+0000 7f2becc00640 1 -- 172.21.15.32:0/2558759397 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0018980 con 0x7f2bf0103900 2024-08-31T07:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.471+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2558759397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf01a8e00 con 0x7f2bf0103900 2024-08-31T07:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.472+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2558759397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf0103d10 con 0x7f2bf0103900 2024-08-31T07:03:08.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.476+0000 7f2becc00640 1 -- 172.21.15.32:0/2558759397 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f2be0010430 con 0x7f2bf0103900 2024-08-31T07:03:08.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.478+0000 7f2becc00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bd40b4b70 0x7f2bd40b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:08.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.478+0000 7f2becc00640 1 -- 172.21.15.32:0/2558759397 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2be00e3e20 con 0x7f2bf0103900 2024-08-31T07:03:08.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.478+0000 7f2bef400640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bd40b4b70 0x7f2bd40b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:08.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.478+0000 7f2becc00640 1 -- 172.21.15.32:0/2558759397 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2be0115130 con 0x7f2bf0103900 2024-08-31T07:03:08.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.479+0000 7f2bef400640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bd40b4b70 0x7f2bd40b7080 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009230 tx=0x7f2bdc031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.551+0000 7ff5b5200640 1 -- 172.21.15.32:0/1730996609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7ff5b002e430 con 0x7ff5b0105b30 2024-08-31T07:03:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.557+0000 7ff5a6a00640 1 -- 172.21.15.32:0/1730996609 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v32)=0 v32) ==== 116+0+0 (secure 0 0 0) 0x7ff5a00ac050 con 0x7ff5b0105b30 2024-08-31T07:03:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 -- 172.21.15.32:0/1730996609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5940b4b70 msgr2=0x7ff5940b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5940b4b70 0x7ff5940b7080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff59c035040 tx=0x7ff59c030d10 comp rx=0 tx=0).stop 2024-08-31T07:03:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 -- 172.21.15.32:0/1730996609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b0105b30 msgr2=0x7ff5b019c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b0105b30 0x7ff5b019c430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000e9c0 tx=0x7ff5a000ee80 comp rx=0 tx=0).stop 2024-08-31T07:03:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 -- 172.21.15.32:0/1730996609 shutdown_connections 2024-08-31T07:03:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5940b4b70 0x7ff5940b7080 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5b010bdc0 0x7ff5b01a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b010b470 0x7ff5b019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 --2- 172.21.15.32:0/1730996609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5b0105b30 0x7ff5b019c430 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 -- 172.21.15.32:0/1730996609 >> 172.21.15.32:0/1730996609 conn(0x7ff5b0101600 msgr2=0x7ff5b0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 -- 172.21.15.32:0/1730996609 shutdown_connections 2024-08-31T07:03:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.560+0000 7ff5b5200640 1 -- 172.21.15.32:0/1730996609 wait complete. 2024-08-31T07:03:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.690+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2558759397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f2bf01a90a0 con 0x7f2bf0103900 2024-08-31T07:03:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.696+0000 7f2becc00640 1 -- 172.21.15.32:0/2558759397 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v33)=0 v33) ==== 127+0+0 (secure 0 0 0) 0x7f2be00acab0 con 0x7f2bf0103900 2024-08-31T07:03:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bd40b4b70 msgr2=0x7f2bd40b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bd40b4b70 0x7f2bd40b7080 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009230 tx=0x7f2bdc031400 comp rx=0 tx=0).stop 2024-08-31T07:03:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2558759397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf0103900 msgr2=0x7f2bf019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf0103900 0x7f2bf019c440 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2be000e9f0 tx=0x7f2be000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2558759397 shutdown_connections 2024-08-31T07:03:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bd40b4b70 0x7f2bd40b7080 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf0109b90 0x7f2bf01a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:08 smithi032 ceph-mon[17252]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:08.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:08 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bf0109240 0x7f2bf019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 --2- 172.21.15.32:0/2558759397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf0103900 0x7f2bf019c440 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2558759397 >> 172.21.15.32:0/2558759397 conn(0x7f2bf007b5f0 msgr2=0x7f2bf0104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:08.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.698+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2558759397 shutdown_connections 2024-08-31T07:03:08.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:08.699+0000 7f2bf6c00640 1 -- 172.21.15.32:0/2558759397 wait complete. 2024-08-31T07:03:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:08 smithi073 ceph-mon[21995]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:08 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:08.993 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:08 smithi084 ceph-mon[22387]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:08.993 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:08 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:09.204 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi032') 2024-08-31T07:03:09.204 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi032') 2024-08-31T07:03:09.205 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi032') 2024-08-31T07:03:09.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:09 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:09.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:09 smithi032 ceph-mon[17252]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:09.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:09 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' 2024-08-31T07:03:09.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:09 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:09.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:09 smithi073 ceph-mon[21995]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:09.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:09 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' 2024-08-31T07:03:09.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:09 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:09.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:09 smithi084 ceph-mon[22387]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:09.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:09 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' 2024-08-31T07:03:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.226+0000 7f0aa1000640 1 -- 172.21.15.32:0/2983426029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a9c109b90 msgr2=0x7f0a9c1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.226+0000 7f0aa1000640 1 --2- 172.21.15.32:0/2983426029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a9c109b90 0x7f0a9c1103c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009ca0 tx=0x7f0a9402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.226+0000 7f0aa1000640 1 -- 172.21.15.32:0/2983426029 shutdown_connections 2024-08-31T07:03:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.226+0000 7f0aa1000640 1 --2- 172.21.15.32:0/2983426029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a9c109b90 0x7f0a9c1103c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.226+0000 7f0aa1000640 1 --2- 172.21.15.32:0/2983426029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a9c109240 0x7f0a9c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.226+0000 7f0aa1000640 1 --2- 172.21.15.32:0/2983426029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a9c103900 0x7f0a9c103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.226+0000 7f0aa1000640 1 -- 172.21.15.32:0/2983426029 >> 172.21.15.32:0/2983426029 conn(0x7f0a9c07b5f0 msgr2=0x7f0a9c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.226+0000 7f0aa1000640 1 -- 172.21.15.32:0/2983426029 shutdown_connections 2024-08-31T07:03:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.226+0000 7f0aa1000640 1 -- 172.21.15.32:0/2983426029 wait complete. 2024-08-31T07:03:10.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.227+0000 7f0aa1000640 1 Processor -- start 2024-08-31T07:03:10.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.227+0000 7f0aa1000640 1 -- start start 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.227+0000 7f0aa1000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a9c103900 0x7f0a9c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.227+0000 7f0aa1000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a9c109240 0x7f0a9c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.227+0000 7f0aa1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a9c109b90 0x7f0a9c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.227+0000 7f0aa1000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c113850 con 0x7f0a9c109240 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.227+0000 7f0aa1000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1136d0 con 0x7f0a9c109b90 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.227+0000 7f0aa1000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1139d0 con 0x7f0a9c103900 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a91400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a9c109240 0x7f0a9c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a91400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a9c109240 0x7f0a9c0759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36364/0 (socket says 172.21.15.32:36364) 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a91400640 1 -- 172.21.15.32:0/2907610426 learned_addr learned my addr 172.21.15.32:0/2907610426 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a99e00640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a9c103900 0x7f0a9c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a9a800640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a9c109b90 0x7f0a9c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a91400640 1 -- 172.21.15.32:0/2907610426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a9c103900 msgr2=0x7f0a9c0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a91400640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a9c103900 0x7f0a9c0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a91400640 1 -- 172.21.15.32:0/2907610426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a9c109b90 msgr2=0x7f0a9c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a91400640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a9c109b90 0x7f0a9c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a91400640 1 -- 172.21.15.32:0/2907610426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c1a8420 con 0x7f0a9c109240 2024-08-31T07:03:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.228+0000 7f0a9a800640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a9c109b90 0x7f0a9c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:03:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.229+0000 7f0a91400640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a9c109240 0x7f0a9c0759f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400e9f0 tx=0x7f0a8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.229+0000 7f0a93400640 1 -- 172.21.15.32:0/2907610426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8400ccc0 con 0x7f0a9c109240 2024-08-31T07:03:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.229+0000 7f0a93400640 1 -- 172.21.15.32:0/2907610426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a840108e0 con 0x7f0a9c109240 2024-08-31T07:03:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.229+0000 7f0a93400640 1 -- 172.21.15.32:0/2907610426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84018860 con 0x7f0a9c109240 2024-08-31T07:03:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.229+0000 7f0aa1000640 1 -- 172.21.15.32:0/2907610426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9c1a8710 con 0x7f0a9c109240 2024-08-31T07:03:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.229+0000 7f0aa1000640 1 -- 172.21.15.32:0/2907610426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9c1a8ca0 con 0x7f0a9c109240 2024-08-31T07:03:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.230+0000 7f0aa1000640 1 -- 172.21.15.32:0/2907610426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a9c103d10 con 0x7f0a9c109240 2024-08-31T07:03:10.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.233+0000 7f0a93400640 1 -- 172.21.15.32:0/2907610426 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f0a84010430 con 0x7f0a9c109240 2024-08-31T07:03:10.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.234+0000 7f0a93400640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0a800b4b70 0x7f0a800b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:10.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.234+0000 7f0a93400640 1 -- 172.21.15.32:0/2907610426 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0a840e3cf0 con 0x7f0a9c109240 2024-08-31T07:03:10.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.234+0000 7f0a99e00640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0a800b4b70 0x7f0a800b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:10.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.235+0000 7f0a99e00640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0a800b4b70 0x7f0a800b7080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c076f00 tx=0x7f0a8c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:10.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.236+0000 7f0a93400640 1 -- 172.21.15.32:0/2907610426 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0a840ac330 con 0x7f0a9c109240 2024-08-31T07:03:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.437+0000 7f0aa1000640 1 -- 172.21.15.32:0/2907610426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f0a9c02e430 con 0x7f0a9c109240 2024-08-31T07:03:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.442+0000 7f0a93400640 1 -- 172.21.15.32:0/2907610426 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v34)=0 v34) ==== 115+0+0 (secure 0 0 0) 0x7f0a840b0af0 con 0x7f0a9c109240 2024-08-31T07:03:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 -- 172.21.15.32:0/2907610426 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0a800b4b70 msgr2=0x7f0a800b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0a800b4b70 0x7f0a800b7080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c076f00 tx=0x7f0a8c005f90 comp rx=0 tx=0).stop 2024-08-31T07:03:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 -- 172.21.15.32:0/2907610426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a9c109240 msgr2=0x7f0a9c0759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a9c109240 0x7f0a9c0759f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400e9f0 tx=0x7f0a8400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 -- 172.21.15.32:0/2907610426 shutdown_connections 2024-08-31T07:03:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0a800b4b70 0x7f0a800b7080 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a9c109b90 0x7f0a9c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a9c109240 0x7f0a9c0759f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 --2- 172.21.15.32:0/2907610426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a9c103900 0x7f0a9c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 -- 172.21.15.32:0/2907610426 >> 172.21.15.32:0/2907610426 conn(0x7f0a9c07b5f0 msgr2=0x7f0a9c1051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.444+0000 7f0aa1000640 1 -- 172.21.15.32:0/2907610426 shutdown_connections 2024-08-31T07:03:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.445+0000 7f0aa1000640 1 -- 172.21.15.32:0/2907610426 wait complete. 2024-08-31T07:03:10.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.828+0000 7fc73c400640 1 -- 172.21.15.32:0/1853751136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc73410b460 msgr2=0x7fc73410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:10.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.828+0000 7fc73c400640 1 --2- 172.21.15.32:0/1853751136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc73410b460 0x7fc73410b870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc720009860 tx=0x7fc72002a340 comp rx=0 tx=0).stop 2024-08-31T07:03:10.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.828+0000 7fc73c400640 1 -- 172.21.15.32:0/1853751136 shutdown_connections 2024-08-31T07:03:10.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.828+0000 7fc73c400640 1 --2- 172.21.15.32:0/1853751136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc73410bdb0 0x7fc7341125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.828+0000 7fc73c400640 1 --2- 172.21.15.32:0/1853751136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc73410b460 0x7fc73410b870 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.828+0000 7fc73c400640 1 --2- 172.21.15.32:0/1853751136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc734105b20 0x7fc734105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.828+0000 7fc73c400640 1 -- 172.21.15.32:0/1853751136 >> 172.21.15.32:0/1853751136 conn(0x7fc7341015b0 msgr2=0x7fc734103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:10.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.828+0000 7fc73c400640 1 -- 172.21.15.32:0/1853751136 shutdown_connections 2024-08-31T07:03:10.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.828+0000 7fc73c400640 1 -- 172.21.15.32:0/1853751136 wait complete. 2024-08-31T07:03:10.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.829+0000 7fc73c400640 1 Processor -- start 2024-08-31T07:03:10.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.829+0000 7fc73c400640 1 -- start start 2024-08-31T07:03:10.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc73c400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc734105b20 0x7fc73419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:10.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc73c400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73410b460 0x7fc73419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:10.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc73c400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc73410bdb0 0x7fc7341a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:10.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc73c400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc734115a70 con 0x7fc734105b20 2024-08-31T07:03:10.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc73c400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341158f0 con 0x7fc73410bdb0 2024-08-31T07:03:10.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc73c400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc734115bf0 con 0x7fc73410b460 2024-08-31T07:03:10.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc739a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc734105b20 0x7fc73419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc739a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc734105b20 0x7fc73419c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36370/0 (socket says 172.21.15.32:36370) 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc739a00640 1 -- 172.21.15.32:0/2260460679 learned_addr learned my addr 172.21.15.32:0/2260460679 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc739000640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73410b460 0x7fc73419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.830+0000 7fc73a400640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc73410bdb0 0x7fc7341a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc739a00640 1 -- 172.21.15.32:0/2260460679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73410b460 msgr2=0x7fc73419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc739a00640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73410b460 0x7fc73419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc739a00640 1 -- 172.21.15.32:0/2260460679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc73410bdb0 msgr2=0x7fc7341a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc739a00640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc73410bdb0 0x7fc7341a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc739a00640 1 -- 172.21.15.32:0/2260460679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7341a2410 con 0x7fc734105b20 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc739000640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73410b460 0x7fc73419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc739a00640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc734105b20 0x7fc73419c4a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc72800d950 tx=0x7fc72800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc732000640 1 -- 172.21.15.32:0/2260460679 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728017030 con 0x7fc734105b20 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc732000640 1 -- 172.21.15.32:0/2260460679 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc728004020 con 0x7fc734105b20 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc73c400640 1 -- 172.21.15.32:0/2260460679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7341a2700 con 0x7fc734105b20 2024-08-31T07:03:10.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc732000640 1 -- 172.21.15.32:0/2260460679 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7280047a0 con 0x7fc734105b20 2024-08-31T07:03:10.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.831+0000 7fc73c400640 1 -- 172.21.15.32:0/2260460679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a8e60 con 0x7fc734105b20 2024-08-31T07:03:10.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.832+0000 7fc73c400640 1 -- 172.21.15.32:0/2260460679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc734105f30 con 0x7fc734105b20 2024-08-31T07:03:10.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.835+0000 7fc732000640 1 -- 172.21.15.32:0/2260460679 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fc728009a30 con 0x7fc734105b20 2024-08-31T07:03:10.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.837+0000 7fc732000640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc71c0b4af0 0x7fc71c0b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:10.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.838+0000 7fc732000640 1 -- 172.21.15.32:0/2260460679 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc7280e3610 con 0x7fc734105b20 2024-08-31T07:03:10.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.838+0000 7fc739000640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc71c0b4af0 0x7fc71c0b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:10.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.838+0000 7fc739000640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc71c0b4af0 0x7fc71c0b7000 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc720035040 tx=0x7fc720030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:10.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:10.839+0000 7fc732000640 1 -- 172.21.15.32:0/2260460679 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc7280ac2a0 con 0x7fc734105b20 2024-08-31T07:03:11.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.040+0000 7fc73c400640 1 -- 172.21.15.32:0/2260460679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fc73419d5d0 con 0x7fc734105b20 2024-08-31T07:03:11.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.044+0000 7fc732000640 1 -- 172.21.15.32:0/2260460679 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v35)=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7fc73419d5d0 con 0x7fc734105b20 2024-08-31T07:03:11.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 -- 172.21.15.32:0/2260460679 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc71c0b4af0 msgr2=0x7fc71c0b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc71c0b4af0 0x7fc71c0b7000 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc720035040 tx=0x7fc720030340 comp rx=0 tx=0).stop 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 -- 172.21.15.32:0/2260460679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc734105b20 msgr2=0x7fc73419c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc734105b20 0x7fc73419c4a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc72800d950 tx=0x7fc72800de10 comp rx=0 tx=0).stop 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 -- 172.21.15.32:0/2260460679 shutdown_connections 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc71c0b4af0 0x7fc71c0b7000 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc73410bdb0 0x7fc7341a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc73410b460 0x7fc73419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 --2- 172.21.15.32:0/2260460679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc734105b20 0x7fc73419c4a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 -- 172.21.15.32:0/2260460679 >> 172.21.15.32:0/2260460679 conn(0x7fc7341015b0 msgr2=0x7fc7341039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 -- 172.21.15.32:0/2260460679 shutdown_connections 2024-08-31T07:03:11.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:11.047+0000 7fc73c400640 1 -- 172.21.15.32:0/2260460679 wait complete. 2024-08-31T07:03:11.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:11 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2907610426' entity='client.admin' 2024-08-31T07:03:11.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:11 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2260460679' entity='client.admin' 2024-08-31T07:03:11.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:11 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2907610426' entity='client.admin' 2024-08-31T07:03:11.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:11 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2260460679' entity='client.admin' 2024-08-31T07:03:11.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:11 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2907610426' entity='client.admin' 2024-08-31T07:03:11.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:11 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2260460679' entity='client.admin' 2024-08-31T07:03:12.597 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:12 smithi032 ceph-mon[17252]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:12 smithi084 ceph-mon[22387]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:12 smithi073 ceph-mon[21995]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:13.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.842+0000 7f69c0e00640 1 -- 172.21.15.32:0/3159039530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69bc10d9c0 msgr2=0x7f69bc10feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:13.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.842+0000 7f69c0e00640 1 --2- 172.21.15.32:0/3159039530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69bc10d9c0 0x7f69bc10feb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009ca0 tx=0x7f69b402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.842+0000 7f69c0e00640 1 -- 172.21.15.32:0/3159039530 shutdown_connections 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.842+0000 7f69c0e00640 1 --2- 172.21.15.32:0/3159039530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69bc10d9c0 0x7f69bc10feb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.842+0000 7f69c0e00640 1 --2- 172.21.15.32:0/3159039530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc104eb0 0x7f69bc10d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.842+0000 7f69c0e00640 1 --2- 172.21.15.32:0/3159039530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69bc104560 0x7f69bc104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.842+0000 7f69c0e00640 1 -- 172.21.15.32:0/3159039530 >> 172.21.15.32:0/3159039530 conn(0x7f69bc0fe060 msgr2=0x7f69bc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.842+0000 7f69c0e00640 1 -- 172.21.15.32:0/3159039530 shutdown_connections 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.842+0000 7f69c0e00640 1 -- 172.21.15.32:0/3159039530 wait complete. 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.843+0000 7f69c0e00640 1 Processor -- start 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.843+0000 7f69c0e00640 1 -- start start 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69c0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69bc104560 0x7f69bc19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69c0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69bc104eb0 0x7f69bc19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69c0e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc10d9c0 0x7f69bc1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69c0e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc113650 con 0x7f69bc104eb0 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69c0e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc1134d0 con 0x7f69bc104560 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69c0e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc1137d0 con 0x7f69bc10d9c0 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69baa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69bc104eb0 0x7f69bc19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69bb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69bc104560 0x7f69bc19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69baa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69bc104eb0 0x7f69bc19c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44096/0 (socket says 172.21.15.32:44096) 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69baa00640 1 -- 172.21.15.32:0/4015955587 learned_addr learned my addr 172.21.15.32:0/4015955587 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.844+0000 7f69bbe00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc10d9c0 0x7f69bc1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69bb400640 1 -- 172.21.15.32:0/4015955587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc10d9c0 msgr2=0x7f69bc1a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69bb400640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc10d9c0 0x7f69bc1a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69bb400640 1 -- 172.21.15.32:0/4015955587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69bc104eb0 msgr2=0x7f69bc19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69bb400640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69bc104eb0 0x7f69bc19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69bb400640 1 -- 172.21.15.32:0/4015955587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc1a21d0 con 0x7f69bc104560 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69bbe00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc10d9c0 0x7f69bc1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69baa00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69bc104eb0 0x7f69bc19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69bb400640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69bc104560 0x7f69bc19c260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00e9f0 tx=0x7f69ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69b3e00640 1 -- 172.21.15.32:0/4015955587 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00cd80 con 0x7f69bc104560 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69c0e00640 1 -- 172.21.15.32:0/4015955587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc1a24c0 con 0x7f69bc104560 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69b3e00640 1 -- 172.21.15.32:0/4015955587 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69ac0035f0 con 0x7f69bc104560 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.845+0000 7f69b3e00640 1 -- 172.21.15.32:0/4015955587 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac010690 con 0x7f69bc104560 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.846+0000 7f69c0e00640 1 -- 172.21.15.32:0/4015955587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc1a8c20 con 0x7f69bc104560 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.847+0000 7f69c0e00640 1 -- 172.21.15.32:0/4015955587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69bc104970 con 0x7f69bc104560 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.849+0000 7f69b3e00640 1 -- 172.21.15.32:0/4015955587 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f69ac0037a0 con 0x7f69bc104560 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.850+0000 7f69b3e00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69a00b4b70 0x7f69a00b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.850+0000 7f69b3e00640 1 -- 172.21.15.32:0/4015955587 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f69ac0e3970 con 0x7f69bc104560 2024-08-31T07:03:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.850+0000 7f69baa00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69a00b4b70 0x7f69a00b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.851+0000 7f69baa00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69a00b4b70 0x7f69a00b7080 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f69bc101840 tx=0x7f69a8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:04.856+0000 7f69b3e00640 1 -- 172.21.15.32:0/4015955587 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f69ac0ac600 con 0x7f69bc104560 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.052+0000 7f69c0e00640 1 -- 172.21.15.32:0/4015955587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f69bc1a8f60 con 0x7f69bc104560 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.058+0000 7f69b3e00640 1 -- 172.21.15.32:0/4015955587 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7f69ac0abf00 con 0x7f69bc104560 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 -- 172.21.15.32:0/4015955587 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69a00b4b70 msgr2=0x7f69a00b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69a00b4b70 0x7f69a00b7080 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f69bc101840 tx=0x7f69a8005f90 comp rx=0 tx=0).stop 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 -- 172.21.15.32:0/4015955587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69bc104560 msgr2=0x7f69bc19c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69bc104560 0x7f69bc19c260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00e9f0 tx=0x7f69ac00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 -- 172.21.15.32:0/4015955587 shutdown_connections 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69a00b4b70 0x7f69a00b7080 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc10d9c0 0x7f69bc1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69bc104eb0 0x7f69bc19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 --2- 172.21.15.32:0/4015955587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69bc104560 0x7f69bc19c260 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:13.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 -- 172.21.15.32:0/4015955587 >> 172.21.15.32:0/4015955587 conn(0x7f69bc0fe060 msgr2=0x7f69bc1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:13.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 -- 172.21.15.32:0/4015955587 shutdown_connections 2024-08-31T07:03:13.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:05.061+0000 7f69c0e00640 1 -- 172.21.15.32:0/4015955587 wait complete. 2024-08-31T07:03:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:14 smithi032 ceph-mon[17252]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:14 smithi084 ceph-mon[22387]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:14.749 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi032') 2024-08-31T07:03:14.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.264+0000 7fd168c00640 1 -- 172.21.15.32:0/441716076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd164109070 msgr2=0x7fd164109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:14.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.264+0000 7fd168c00640 1 --2- 172.21.15.32:0/441716076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd164109070 0x7fd164109480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009900 tx=0x7fd14c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:03:14.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.264+0000 7fd168c00640 1 -- 172.21.15.32:0/441716076 shutdown_connections 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.264+0000 7fd168c00640 1 --2- 172.21.15.32:0/441716076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1641099c0 0x7fd1641101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.264+0000 7fd168c00640 1 --2- 172.21.15.32:0/441716076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd164109070 0x7fd164109480 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.264+0000 7fd168c00640 1 --2- 172.21.15.32:0/441716076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd164103730 0x7fd164103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.264+0000 7fd168c00640 1 -- 172.21.15.32:0/441716076 >> 172.21.15.32:0/441716076 conn(0x7fd1640ff180 msgr2=0x7fd1641015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.264+0000 7fd168c00640 1 -- 172.21.15.32:0/441716076 shutdown_connections 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.264+0000 7fd168c00640 1 -- 172.21.15.32:0/441716076 wait complete. 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.265+0000 7fd168c00640 1 Processor -- start 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.265+0000 7fd168c00640 1 -- start start 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd168c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd164103730 0x7fd16419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd168c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd164109070 0x7fd16419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd168c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1641099c0 0x7fd16419f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd168c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd164113630 con 0x7fd164109070 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd168c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641134b0 con 0x7fd1641099c0 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd168c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641137b0 con 0x7fd164103730 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd15aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd164109070 0x7fd16419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd163400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd164103730 0x7fd16419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd163400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd164103730 0x7fd16419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58810/0 (socket says 172.21.15.32:58810) 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd15aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd164109070 0x7fd16419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36308/0 (socket says 172.21.15.32:36308) 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd163400640 1 -- 172.21.15.32:0/2366176084 learned_addr learned my addr 172.21.15.32:0/2366176084 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.266+0000 7fd163e00640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1641099c0 0x7fd16419f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd163400640 1 -- 172.21.15.32:0/2366176084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1641099c0 msgr2=0x7fd16419f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd163400640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1641099c0 0x7fd16419f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd163400640 1 -- 172.21.15.32:0/2366176084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd164109070 msgr2=0x7fd16419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd163400640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd164109070 0x7fd16419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd163400640 1 -- 172.21.15.32:0/2366176084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1641a0070 con 0x7fd164103730 2024-08-31T07:03:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd163400640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd164103730 0x7fd16419a100 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd15400d950 tx=0x7fd15400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd160c00640 1 -- 172.21.15.32:0/2366176084 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154017030 con 0x7fd164103730 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd160c00640 1 -- 172.21.15.32:0/2366176084 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd154004020 con 0x7fd164103730 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd168c00640 1 -- 172.21.15.32:0/2366176084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1641a0360 con 0x7fd164103730 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd160c00640 1 -- 172.21.15.32:0/2366176084 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1540047a0 con 0x7fd164103730 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.267+0000 7fd168c00640 1 -- 172.21.15.32:0/2366176084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1641a6ac0 con 0x7fd164103730 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.268+0000 7fd168c00640 1 -- 172.21.15.32:0/2366176084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd164103b40 con 0x7fd164103730 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.271+0000 7fd160c00640 1 -- 172.21.15.32:0/2366176084 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fd154009a30 con 0x7fd164103730 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.272+0000 7fd160c00640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1480b4b70 0x7fd1480b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.272+0000 7fd160c00640 1 -- 172.21.15.32:0/2366176084 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd1540e2fc0 con 0x7fd164103730 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.272+0000 7fd15aa00640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1480b4b70 0x7fd1480b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.273+0000 7fd15aa00640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1480b4b70 0x7fd1480b7080 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009230 tx=0x7fd14c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.276+0000 7fd160c00640 1 -- 172.21.15.32:0/2366176084 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd1540abc50 con 0x7fd164103730 2024-08-31T07:03:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.505+0000 7fd168c00640 1 -- 172.21.15.32:0/2366176084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fd16402e430 con 0x7fd164103730 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.512+0000 7fd160c00640 1 -- 172.21.15.32:0/2366176084 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v31)=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7fd1540ab550 con 0x7fd164103730 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 -- 172.21.15.32:0/2366176084 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1480b4b70 msgr2=0x7fd1480b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1480b4b70 0x7fd1480b7080 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009230 tx=0x7fd14c031400 comp rx=0 tx=0).stop 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 -- 172.21.15.32:0/2366176084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd164103730 msgr2=0x7fd16419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd164103730 0x7fd16419a100 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd15400d950 tx=0x7fd15400de10 comp rx=0 tx=0).stop 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 -- 172.21.15.32:0/2366176084 shutdown_connections 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1480b4b70 0x7fd1480b7080 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1641099c0 0x7fd16419f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd164109070 0x7fd16419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 --2- 172.21.15.32:0/2366176084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd164103730 0x7fd16419a100 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 -- 172.21.15.32:0/2366176084 >> 172.21.15.32:0/2366176084 conn(0x7fd1640ff180 msgr2=0x7fd1641015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 -- 172.21.15.32:0/2366176084 shutdown_connections 2024-08-31T07:03:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:07.514+0000 7fd168c00640 1 -- 172.21.15.32:0/2366176084 wait complete. 2024-08-31T07:03:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:14 smithi073 ceph-mon[21995]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:16.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:16 smithi084 ceph-mon[22387]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:16.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:16 smithi032 ceph-mon[17252]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:16 smithi073 ceph-mon[21995]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:18.183 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.183 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.183 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.184 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.679 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.679 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.679 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.679 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.679 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.679 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '965fc21a-6766-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi032') 2024-08-31T07:03:18.680 INFO:tasks.cephadm:Deploying OSDs... 2024-08-31T07:03:18.680 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:03:18.680 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2024-08-31T07:03:18.718 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-31T07:03:18.718 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2024-08-31T07:03:18.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:18 smithi084 ceph-mon[22387]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:18.788 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:18 smithi032 ceph-mon[17252]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:18.797 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-31T07:03:18.797 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:18.797 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-08-31T07:03:18.797 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:18.797 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:18.797 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-08-31 07:01:52.883263028 +0000 2024-08-31T07:03:18.797 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-08-31 06:56:57.314956314 +0000 2024-08-31T07:03:18.797 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-08-31 06:56:57.314956314 +0000 2024-08-31T07:03:18.797 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-08-31 06:56:57.314956314 +0000 2024-08-31T07:03:18.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-31T07:03:18.892 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-08-31T07:03:18.892 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-08-31T07:03:18.892 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000180347 s, 2.8 MB/s 2024-08-31T07:03:18.895 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-31T07:03:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:18 smithi073 ceph-mon[21995]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:18.931 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2024-08-31T07:03:19.007 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-31T07:03:19.008 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:19.008 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-08-31T07:03:19.008 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:19.008 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:19.008 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-08-31 07:01:52.923262239 +0000 2024-08-31T07:03:19.008 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-08-31 06:56:57.789946794 +0000 2024-08-31T07:03:19.008 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-08-31 06:56:57.789946794 +0000 2024-08-31T07:03:19.008 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-08-31 06:56:57.789946794 +0000 2024-08-31T07:03:19.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-31T07:03:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-08-31T07:03:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-08-31T07:03:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000173705 s, 2.9 MB/s 2024-08-31T07:03:19.103 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-31T07:03:19.139 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2024-08-31T07:03:19.216 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-31T07:03:19.216 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:19.216 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-08-31T07:03:19.216 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:19.217 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:19.217 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-08-31 07:01:52.963261449 +0000 2024-08-31T07:03:19.217 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-08-31 06:56:58.233937897 +0000 2024-08-31T07:03:19.217 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-08-31 06:56:58.233937897 +0000 2024-08-31T07:03:19.217 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-08-31 06:56:58.233937897 +0000 2024-08-31T07:03:19.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-31T07:03:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-08-31T07:03:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-08-31T07:03:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.00606934 s, 84.4 kB/s 2024-08-31T07:03:19.318 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-31T07:03:19.355 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2024-08-31T07:03:19.430 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-31T07:03:19.430 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:19.430 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-08-31T07:03:19.430 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:19.430 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:19.430 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-08-31 07:01:53.005260621 +0000 2024-08-31T07:03:19.430 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-08-31 06:56:58.718928177 +0000 2024-08-31T07:03:19.430 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-08-31 06:56:58.718928177 +0000 2024-08-31T07:03:19.430 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-08-31 06:56:58.718928177 +0000 2024-08-31T07:03:19.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-31T07:03:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-08-31T07:03:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-08-31T07:03:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000179651 s, 2.8 MB/s 2024-08-31T07:03:19.526 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-31T07:03:19.562 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-08-31T07:03:19.562 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-08-31T07:03:19.582 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-31T07:03:19.582 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-08-31T07:03:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-31T07:03:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:19.642 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-08-31T07:03:19.642 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:19.642 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:19.642 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-08-31 07:02:56.301077723 +0000 2024-08-31T07:03:19.642 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-08-31 06:57:03.480464962 +0000 2024-08-31T07:03:19.642 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-08-31 06:57:03.480464962 +0000 2024-08-31T07:03:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-08-31 06:57:03.480464962 +0000 2024-08-31T07:03:19.642 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-31T07:03:19.712 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-08-31T07:03:19.712 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-08-31T07:03:19.712 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000290945 s, 1.8 MB/s 2024-08-31T07:03:19.713 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-31T07:03:19.772 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-08-31T07:03:19.828 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-31T07:03:19.828 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:19.828 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-31T07:03:19.828 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:19.829 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:19.829 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-08-31 07:02:56.301077723 +0000 2024-08-31T07:03:19.829 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-08-31 06:57:03.809456858 +0000 2024-08-31T07:03:19.829 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-08-31 06:57:03.809456858 +0000 2024-08-31T07:03:19.829 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-08-31 06:57:03.809456858 +0000 2024-08-31T07:03:19.830 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-31T07:03:19.894 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-08-31T07:03:19.894 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-08-31T07:03:19.894 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000209215 s, 2.4 MB/s 2024-08-31T07:03:19.895 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-31T07:03:19.953 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-08-31T07:03:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-31T07:03:20.010 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:20.010 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-08-31T07:03:20.011 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:20.011 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:20.011 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-08-31 07:02:56.302077699 +0000 2024-08-31T07:03:20.011 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-08-31 06:57:04.119449222 +0000 2024-08-31T07:03:20.011 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-08-31 06:57:04.119449222 +0000 2024-08-31T07:03:20.011 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-08-31 06:57:04.119449222 +0000 2024-08-31T07:03:20.012 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-31T07:03:20.075 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-08-31T07:03:20.075 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-08-31T07:03:20.075 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000203159 s, 2.5 MB/s 2024-08-31T07:03:20.076 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-31T07:03:20.133 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-08-31T07:03:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-31T07:03:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:20.188 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-08-31T07:03:20.188 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:20.188 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:20.189 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-08-31 07:02:56.302077699 +0000 2024-08-31T07:03:20.189 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-08-31 06:57:04.456440920 +0000 2024-08-31T07:03:20.189 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-08-31 06:57:04.456440920 +0000 2024-08-31T07:03:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-08-31 06:57:04.456440920 +0000 2024-08-31T07:03:20.189 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-31T07:03:20.251 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-08-31T07:03:20.252 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-08-31T07:03:20.252 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000175073 s, 2.9 MB/s 2024-08-31T07:03:20.253 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-31T07:03:20.309 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-08-31T07:03:20.309 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2024-08-31T07:03:20.326 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-31T07:03:20.326 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2024-08-31T07:03:20.381 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-31T07:03:20.382 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:20.382 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-08-31T07:03:20.382 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:20.382 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:20.382 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-08-31 07:02:53.958193095 +0000 2024-08-31T07:03:20.382 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-08-31 06:56:58.415720223 +0000 2024-08-31T07:03:20.382 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-08-31 06:56:58.415720223 +0000 2024-08-31T07:03:20.382 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-08-31 06:56:58.415720223 +0000 2024-08-31T07:03:20.382 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-31T07:03:20.449 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-08-31T07:03:20.449 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-08-31T07:03:20.449 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000269838 s, 1.9 MB/s 2024-08-31T07:03:20.450 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-31T07:03:20.505 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2024-08-31T07:03:20.561 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-31T07:03:20.562 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:20.562 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-31T07:03:20.562 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:20.562 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:20.562 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-08-31 07:02:53.959193078 +0000 2024-08-31T07:03:20.563 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-08-31 06:56:58.733714509 +0000 2024-08-31T07:03:20.563 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-08-31 06:56:58.733714509 +0000 2024-08-31T07:03:20.563 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-08-31 06:56:58.733714509 +0000 2024-08-31T07:03:20.563 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-31T07:03:20.625 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-08-31T07:03:20.625 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-08-31T07:03:20.625 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.00023589 s, 2.2 MB/s 2024-08-31T07:03:20.626 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:20 smithi084 ceph-mon[22387]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:20.626 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:20.626 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:20.626 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:03:20.626 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:03:20.626 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:20.627 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-31T07:03:20.684 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2024-08-31T07:03:20.739 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-31T07:03:20.739 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:20.739 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-08-31T07:03:20.739 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:20.739 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:20.739 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-08-31 07:02:53.959193078 +0000 2024-08-31T07:03:20.739 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-08-31 06:56:59.054708741 +0000 2024-08-31T07:03:20.739 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-08-31 06:56:59.054708741 +0000 2024-08-31T07:03:20.739 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-08-31 06:56:59.054708741 +0000 2024-08-31T07:03:20.740 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-31T07:03:20.803 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-08-31T07:03:20.803 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-08-31T07:03:20.803 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000174363 s, 2.9 MB/s 2024-08-31T07:03:20.804 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-31T07:03:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:20 smithi032 ceph-mon[17252]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:03:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:03:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:20.860 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2024-08-31T07:03:20.915 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-31T07:03:20.916 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T07:03:20.916 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-08-31T07:03:20.916 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:03:20.916 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T07:03:20.916 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-08-31 07:02:53.959193078 +0000 2024-08-31T07:03:20.916 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-08-31 06:56:59.380702883 +0000 2024-08-31T07:03:20.916 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-08-31 06:56:59.380702883 +0000 2024-08-31T07:03:20.917 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-08-31 06:56:59.380702883 +0000 2024-08-31T07:03:20.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-31T07:03:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:20 smithi073 ceph-mon[21995]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:20.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:20.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T07:03:20.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T07:03:20.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-08-31T07:03:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-08-31T07:03:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000211058 s, 2.4 MB/s 2024-08-31T07:03:20.981 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-31T07:03:21.038 INFO:tasks.cephadm:Deploying osd.0 on smithi032 with /dev/vg_nvme/lv_4... 2024-08-31T07:03:21.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-31T07:03:21.432 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:03:21.728 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:21 smithi032 ceph-mon[17252]: Reconfiguring mon.c (monmap changed)... 2024-08-31T07:03:21.728 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:21 smithi032 ceph-mon[17252]: Reconfiguring daemon mon.c on smithi084 2024-08-31T07:03:21.728 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:21.728 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:21.728 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:03:21.729 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:21.729 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:03:21.729 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:21.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:21 smithi084 ceph-mon[22387]: Reconfiguring mon.c (monmap changed)... 2024-08-31T07:03:21.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:21 smithi084 ceph-mon[22387]: Reconfiguring daemon mon.c on smithi084 2024-08-31T07:03:21.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:21.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:21.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:03:21.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:21.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:03:21.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:21.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:21 smithi073 ceph-mon[21995]: Reconfiguring mon.c (monmap changed)... 2024-08-31T07:03:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:21 smithi073 ceph-mon[21995]: Reconfiguring daemon mon.c on smithi084 2024-08-31T07:03:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:03:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:03:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:22.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:22 smithi032 ceph-mon[17252]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:22.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:22 smithi084 ceph-mon[22387]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:22.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:22 smithi073 ceph-mon[21995]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:03:24.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_4 2024-08-31T07:03:24.731 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:24 smithi073 ceph-mon[21995]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:24 smithi084 ceph-mon[22387]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:24.812 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:03:24.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:24 smithi032 ceph-mon[17252]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.405+0000 7f56ad200640 1 -- 172.21.15.32:0/2629010872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a810bbe0 msgr2=0x7f56a8112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.405+0000 7f56ad200640 1 --2- 172.21.15.32:0/2629010872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a810bbe0 0x7f56a8112410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009ca0 tx=0x7f56a002a8b0 comp rx=0 tx=0).stop 2024-08-31T07:03:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.405+0000 7f56ad200640 1 -- 172.21.15.32:0/2629010872 shutdown_connections 2024-08-31T07:03:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.405+0000 7f56ad200640 1 --2- 172.21.15.32:0/2629010872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a810bbe0 0x7f56a8112410 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.405+0000 7f56ad200640 1 --2- 172.21.15.32:0/2629010872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56a810b290 0x7f56a810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.405+0000 7f56ad200640 1 --2- 172.21.15.32:0/2629010872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a8105950 0x7f56a8105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.405+0000 7f56ad200640 1 -- 172.21.15.32:0/2629010872 >> 172.21.15.32:0/2629010872 conn(0x7f56a8101420 msgr2=0x7f56a8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.405+0000 7f56ad200640 1 -- 172.21.15.32:0/2629010872 shutdown_connections 2024-08-31T07:03:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.405+0000 7f56ad200640 1 -- 172.21.15.32:0/2629010872 wait complete. 2024-08-31T07:03:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.406+0000 7f56ad200640 1 Processor -- start 2024-08-31T07:03:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.406+0000 7f56ad200640 1 -- start start 2024-08-31T07:03:25.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56ad200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56a8105950 0x7f56a819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:25.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56ad200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a810b290 0x7f56a819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:25.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56ad200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a810bbe0 0x7f56a81a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:25.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56ad200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56a81158f0 con 0x7f56a810b290 2024-08-31T07:03:25.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56ad200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8115770 con 0x7f56a8105950 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56ad200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8115a70 con 0x7f56a810bbe0 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56a6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a810b290 0x7f56a819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56a7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56a8105950 0x7f56a819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56a6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a810b290 0x7f56a819c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60810/0 (socket says 172.21.15.32:60810) 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56a6a00640 1 -- 172.21.15.32:0/935770379 learned_addr learned my addr 172.21.15.32:0/935770379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.407+0000 7f56a7e00640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a810bbe0 0x7f56a81a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.408+0000 7f56a7400640 1 -- 172.21.15.32:0/935770379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a810bbe0 msgr2=0x7f56a81a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.408+0000 7f56a7400640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a810bbe0 0x7f56a81a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.408+0000 7f56a7400640 1 -- 172.21.15.32:0/935770379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a810b290 msgr2=0x7f56a819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.408+0000 7f56a7400640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a810b290 0x7f56a819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.408+0000 7f56a7400640 1 -- 172.21.15.32:0/935770379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a81a2110 con 0x7f56a8105950 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.408+0000 7f56a7400640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56a8105950 0x7f56a819c260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f569800c970 tx=0x7f569800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.408+0000 7f569fe00640 1 -- 172.21.15.32:0/935770379 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698007bb0 con 0x7f56a8105950 2024-08-31T07:03:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.408+0000 7f56ad200640 1 -- 172.21.15.32:0/935770379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a81a2400 con 0x7f56a8105950 2024-08-31T07:03:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.409+0000 7f569fe00640 1 -- 172.21.15.32:0/935770379 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5698007d50 con 0x7f56a8105950 2024-08-31T07:03:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.409+0000 7f56ad200640 1 -- 172.21.15.32:0/935770379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a81a8c20 con 0x7f56a8105950 2024-08-31T07:03:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.409+0000 7f569fe00640 1 -- 172.21.15.32:0/935770379 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698018780 con 0x7f56a8105950 2024-08-31T07:03:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.410+0000 7f56ad200640 1 -- 172.21.15.32:0/935770379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a8105d60 con 0x7f56a8105950 2024-08-31T07:03:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.412+0000 7f569fe00640 1 -- 172.21.15.32:0/935770379 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f569800f450 con 0x7f56a8105950 2024-08-31T07:03:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.414+0000 7f569fe00640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f568c0b4c20 0x7f568c0b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.414+0000 7f569fe00640 1 -- 172.21.15.32:0/935770379 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f56980e3800 con 0x7f56a8105950 2024-08-31T07:03:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.414+0000 7f56a6a00640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f568c0b4c20 0x7f568c0b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.415+0000 7f56a6a00640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f568c0b4c20 0x7f568c0b7130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f56a8104c70 tx=0x7f5694005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:25.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.415+0000 7f569fe00640 1 -- 172.21.15.32:0/935770379 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5698014030 con 0x7f56a8105950 2024-08-31T07:03:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:25.635+0000 7f56ad200640 1 -- 172.21.15.32:0/935770379 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f56a80635a0 con 0x7f568c0b4c20 2024-08-31T07:03:26.502 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:26 smithi032 ceph-mon[17252]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:26.502 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:03:26.502 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:03:26.503 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:26 smithi084 ceph-mon[22387]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:26.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:03:26.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:03:26.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:26 smithi073 ceph-mon[21995]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:03:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:03:26.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:27 smithi032 ceph-mon[17252]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:03:27.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:27 smithi084 ceph-mon[22387]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:03:27.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:27 smithi073 ceph-mon[21995]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:03:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:28 smithi084 ceph-mon[22387]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:28.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:28 smithi032 ceph-mon[17252]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:28 smithi073 ceph-mon[21995]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:29.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:29 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2598932222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "91a7f222-d5e9-423d-ba87-fa1278455da4"} : dispatch 2024-08-31T07:03:29.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:29 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2598932222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91a7f222-d5e9-423d-ba87-fa1278455da4"}]': finished 2024-08-31T07:03:29.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:29 smithi084 ceph-mon[22387]: osdmap e6: 1 total, 0 up, 1 in 2024-08-31T07:03:29.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:29 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:29.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:29 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2598932222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "91a7f222-d5e9-423d-ba87-fa1278455da4"} : dispatch 2024-08-31T07:03:29.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:29 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2598932222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91a7f222-d5e9-423d-ba87-fa1278455da4"}]': finished 2024-08-31T07:03:29.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:29 smithi032 ceph-mon[17252]: osdmap e6: 1 total, 0 up, 1 in 2024-08-31T07:03:29.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:29 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:29.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:29 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2598932222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "91a7f222-d5e9-423d-ba87-fa1278455da4"} : dispatch 2024-08-31T07:03:29.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:29 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2598932222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91a7f222-d5e9-423d-ba87-fa1278455da4"}]': finished 2024-08-31T07:03:29.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:29 smithi073 ceph-mon[21995]: osdmap e6: 1 total, 0 up, 1 in 2024-08-31T07:03:29.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:29 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:30.749 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:30 smithi084 ceph-mon[22387]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:30.749 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:30 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/642475074' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:03:30.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:30 smithi032 ceph-mon[17252]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:30.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:30 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/642475074' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:03:30.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:30 smithi073 ceph-mon[21995]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:30.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:30 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/642475074' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:03:32.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:32 smithi032 ceph-mon[17252]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:32 smithi073 ceph-mon[21995]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:32.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:32 smithi084 ceph-mon[22387]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:34.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:34 smithi032 ceph-mon[17252]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:34 smithi073 ceph-mon[21995]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:34 smithi084 ceph-mon[22387]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:36.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:36 smithi032 ceph-mon[17252]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:36 smithi073 ceph-mon[21995]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:36 smithi084 ceph-mon[22387]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:38.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:38 smithi032 ceph-mon[17252]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:38.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-31T07:03:38.666 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:38 smithi073 ceph-mon[21995]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-31T07:03:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:38 smithi084 ceph-mon[22387]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-31T07:03:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:39.802 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:39 smithi032 ceph-mon[17252]: Deploying daemon osd.0 on smithi032 2024-08-31T07:03:39.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:39 smithi073 ceph-mon[21995]: Deploying daemon osd.0 on smithi032 2024-08-31T07:03:39.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:39 smithi084 ceph-mon[22387]: Deploying daemon osd.0 on smithi032 2024-08-31T07:03:40.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:40 smithi032 ceph-mon[17252]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:40 smithi073 ceph-mon[21995]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:40 smithi084 ceph-mon[22387]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:42.538 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:42 smithi032 ceph-mon[17252]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:42 smithi073 ceph-mon[21995]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:42 smithi084 ceph-mon[22387]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:44.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:44 smithi032 ceph-mon[17252]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:44 smithi073 ceph-mon[21995]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:44 smithi084 ceph-mon[22387]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:45.658 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:03:45.658 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:45.658 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:45.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:03:45.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:45.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:03:45.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:45.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:46.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:46 smithi032 ceph-mon[17252]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:46 smithi073 ceph-mon[21995]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:46 smithi084 ceph-mon[22387]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:48.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:48 smithi032 ceph-mon[17252]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:48.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:48.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:48.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:48.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:03:48.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:48 smithi073 ceph-mon[21995]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:48.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:03:48.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:48 smithi084 ceph-mon[22387]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:03:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:50.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.466+0000 7f569fe00640 1 -- 172.21.15.32:0/935770379 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f56a80635a0 con 0x7f568c0b4c20 2024-08-31T07:03:50.469 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 0 on host 'smithi032' 2024-08-31T07:03:50.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.468+0000 7f56ad200640 1 -- 172.21.15.32:0/935770379 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f568c0b4c20 msgr2=0x7f568c0b7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:50.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.468+0000 7f56ad200640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f568c0b4c20 0x7f568c0b7130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f56a8104c70 tx=0x7f5694005f90 comp rx=0 tx=0).stop 2024-08-31T07:03:50.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.468+0000 7f56ad200640 1 -- 172.21.15.32:0/935770379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56a8105950 msgr2=0x7f56a819c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:50.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.468+0000 7f56ad200640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56a8105950 0x7f56a819c260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f569800c970 tx=0x7f569800ce30 comp rx=0 tx=0).stop 2024-08-31T07:03:50.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.468+0000 7f56ad200640 1 -- 172.21.15.32:0/935770379 shutdown_connections 2024-08-31T07:03:50.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.468+0000 7f56ad200640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f568c0b4c20 0x7f568c0b7130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:50.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.468+0000 7f56ad200640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56a810bbe0 0x7f56a81a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:50.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.469+0000 7f56ad200640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56a810b290 0x7f56a819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:50.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.469+0000 7f56ad200640 1 --2- 172.21.15.32:0/935770379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56a8105950 0x7f56a819c260 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:50.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.469+0000 7f56ad200640 1 -- 172.21.15.32:0/935770379 >> 172.21.15.32:0/935770379 conn(0x7f56a8101420 msgr2=0x7f56a810a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:50.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.469+0000 7f56ad200640 1 -- 172.21.15.32:0/935770379 shutdown_connections 2024-08-31T07:03:50.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:03:50.469+0000 7f56ad200640 1 -- 172.21.15.32:0/935770379 wait complete. 2024-08-31T07:03:50.792 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:50 smithi084 ceph-mon[22387]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:50.792 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:03:50.792 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:50.792 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:50.843 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:50 smithi032 ceph-mon[17252]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:50.843 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:03:50.843 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:50.843 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:50 smithi073 ceph-mon[21995]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:03:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:50.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:51.407 DEBUG:teuthology.orchestra.run.smithi032:osd.0> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.0.service 2024-08-31T07:03:51.411 INFO:tasks.cephadm:Deploying osd.1 on smithi073 with /dev/vg_nvme/lv_4... 2024-08-31T07:03:51.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-31T07:03:51.649 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:03:51.771 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:51 smithi073 ceph-mon[21995]: from='osd.0 [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T07:03:51.771 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:51 smithi073 ceph-mon[21995]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T07:03:51.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:51 smithi032 ceph-mon[17252]: from='osd.0 [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T07:03:51.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:51 smithi032 ceph-mon[17252]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T07:03:51.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:51 smithi084 ceph-mon[22387]: from='osd.0 [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T07:03:51.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:51 smithi084 ceph-mon[22387]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T07:03:52.845 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:52 smithi073 ceph-mon[21995]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:52.845 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:52 smithi073 ceph-mon[21995]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-31T07:03:52.845 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:52 smithi073 ceph-mon[21995]: osdmap e7: 1 total, 0 up, 1 in 2024-08-31T07:03:52.846 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:52 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:52.846 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:52 smithi073 ceph-mon[21995]: from='osd.0 [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:03:52.846 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:52 smithi073 ceph-mon[21995]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:03:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:52 smithi032 ceph-mon[17252]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:52 smithi032 ceph-mon[17252]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-31T07:03:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:52 smithi032 ceph-mon[17252]: osdmap e7: 1 total, 0 up, 1 in 2024-08-31T07:03:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:52 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:52 smithi032 ceph-mon[17252]: from='osd.0 [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:03:52.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:52 smithi032 ceph-mon[17252]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:03:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:52 smithi084 ceph-mon[22387]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:52.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:52 smithi084 ceph-mon[22387]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-31T07:03:52.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:52 smithi084 ceph-mon[22387]: osdmap e7: 1 total, 0 up, 1 in 2024-08-31T07:03:52.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:52 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:52.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:52 smithi084 ceph-mon[22387]: from='osd.0 [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:03:52.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:52 smithi084 ceph-mon[22387]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:03:53.358 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:03:53 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0[34910]: 2024-08-31T07:03:53.075+0000 7f00ca800640 -1 osd.0 0 waiting for initial osdmap 2024-08-31T07:03:53.358 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:03:53 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0[34910]: 2024-08-31T07:03:53.087+0000 7f00c0800640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:03:53.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:53 smithi073 ceph-mon[21995]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:03:53.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:53 smithi073 ceph-mon[21995]: osdmap e8: 1 total, 0 up, 1 in 2024-08-31T07:03:53.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:53 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:53.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:53 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:53.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:53 smithi073 ceph-mon[21995]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:53.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:53 smithi073 ceph-mon[21995]: from='osd.0 ' entity='osd.0' 2024-08-31T07:03:53.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:53 smithi032 ceph-mon[17252]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:03:53.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:53 smithi032 ceph-mon[17252]: osdmap e8: 1 total, 0 up, 1 in 2024-08-31T07:03:53.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:53 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:53.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:53 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:53.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:53 smithi032 ceph-mon[17252]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:53.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:53 smithi032 ceph-mon[17252]: from='osd.0 ' entity='osd.0' 2024-08-31T07:03:53.994 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:53 smithi084 ceph-mon[22387]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:03:53.994 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:53 smithi084 ceph-mon[22387]: osdmap e8: 1 total, 0 up, 1 in 2024-08-31T07:03:53.994 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:53 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:53.994 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:53 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:53.994 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:53 smithi084 ceph-mon[22387]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T07:03:53.995 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:53 smithi084 ceph-mon[22387]: from='osd.0 ' entity='osd.0' 2024-08-31T07:03:53.995 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T07:03:54.023 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_4 2024-08-31T07:03:54.274 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:54 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:54 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:54 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:54 smithi073 ceph-mon[21995]: osd.0 [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] boot 2024-08-31T07:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:54 smithi073 ceph-mon[21995]: osdmap e9: 1 total, 1 up, 1 in 2024-08-31T07:03:54.644 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:54 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.735+0000 7f976414d640 1 -- 172.21.15.73:0/3888082775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f975c108a90 msgr2=0x7f975c10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.735+0000 7f976414d640 1 --2- 172.21.15.73:0/3888082775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f975c108a90 0x7f975c10f2c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9754009900 tx=0x7f975402a410 comp rx=0 tx=0).stop 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.735+0000 7f976414d640 1 -- 172.21.15.73:0/3888082775 shutdown_connections 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.735+0000 7f976414d640 1 --2- 172.21.15.73:0/3888082775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f975c108a90 0x7f975c10f2c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.735+0000 7f976414d640 1 --2- 172.21.15.73:0/3888082775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f975c108140 0x7f975c108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.735+0000 7f976414d640 1 --2- 172.21.15.73:0/3888082775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f975c102800 0x7f975c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.735+0000 7f976414d640 1 -- 172.21.15.73:0/3888082775 >> 172.21.15.73:0/3888082775 conn(0x7f975c0794d0 msgr2=0x7f975c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.735+0000 7f976414d640 1 -- 172.21.15.73:0/3888082775 shutdown_connections 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.735+0000 7f976414d640 1 -- 172.21.15.73:0/3888082775 wait complete. 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f976414d640 1 Processor -- start 2024-08-31T07:03:54.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f976414d640 1 -- start start 2024-08-31T07:03:54.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f976414d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f975c102800 0x7f975c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:54.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f976414d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f975c108140 0x7f975c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:54.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f976414d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f975c108a90 0x7f975c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:54.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f976414d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f975c113860 con 0x7f975c108a90 2024-08-31T07:03:54.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f976414d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1136e0 con 0x7f975c102800 2024-08-31T07:03:54.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f976414d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1139e0 con 0x7f975c108140 2024-08-31T07:03:54.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f9761ee9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f975c102800 0x7f975c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:54.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f9761ee9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f975c102800 0x7f975c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58962/0 (socket says 172.21.15.73:58962) 2024-08-31T07:03:54.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f9761ee9640 1 -- 172.21.15.73:0/3727004627 learned_addr learned my addr 172.21.15.73:0/3727004627 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-08-31T07:03:54.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.736+0000 7f97626ea640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f975c108a90 0x7f975c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:54.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f9761ee9640 1 -- 172.21.15.73:0/3727004627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f975c108140 msgr2=0x7f975c19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:54.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f9761ee9640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f975c108140 0x7f975c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:54.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f9761ee9640 1 -- 172.21.15.73:0/3727004627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f975c108a90 msgr2=0x7f975c19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:03:54.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f9761ee9640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f975c108a90 0x7f975c19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:03:54.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f9761ee9640 1 -- 172.21.15.73:0/3727004627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c1a0260 con 0x7f975c102800 2024-08-31T07:03:54.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f9761ee9640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f975c102800 0x7f975c19a2f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f975800ba50 tx=0x7f975800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:54.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f9752ffd640 1 -- 172.21.15.73:0/3727004627 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758019070 con 0x7f975c102800 2024-08-31T07:03:54.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f976414d640 1 -- 172.21.15.73:0/3727004627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c1a0550 con 0x7f975c102800 2024-08-31T07:03:54.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f9752ffd640 1 -- 172.21.15.73:0/3727004627 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975800cbe0 con 0x7f975c102800 2024-08-31T07:03:54.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f9752ffd640 1 -- 172.21.15.73:0/3727004627 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580075d0 con 0x7f975c102800 2024-08-31T07:03:54.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.737+0000 7f976414d640 1 -- 172.21.15.73:0/3727004627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c1a6cb0 con 0x7f975c102800 2024-08-31T07:03:54.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.738+0000 7f976414d640 1 -- 172.21.15.73:0/3727004627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975c102c10 con 0x7f975c102800 2024-08-31T07:03:54.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.740+0000 7f9752ffd640 1 -- 172.21.15.73:0/3727004627 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f975801c070 con 0x7f975c102800 2024-08-31T07:03:54.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.740+0000 7f9752ffd640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97480b4b70 0x7f97480b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:03:54.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.740+0000 7f9752ffd640 1 -- 172.21.15.73:0/3727004627 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1982+0+0 (secure 0 0 0) 0x7f97580e3180 con 0x7f975c102800 2024-08-31T07:03:54.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.740+0000 7f97616e8640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97480b4b70 0x7f97480b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:03:54.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.741+0000 7f97616e8640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97480b4b70 0x7f97480b7080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f974c005fd0 tx=0x7f974c005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:03:54.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.744+0000 7f9752ffd640 1 -- 172.21.15.73:0/3727004627 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f97580abc50 con 0x7f975c102800 2024-08-31T07:03:54.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:54 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:03:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:54 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:03:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:54 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:54 smithi032 ceph-mon[17252]: osd.0 [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] boot 2024-08-31T07:03:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:54 smithi032 ceph-mon[17252]: osdmap e9: 1 total, 1 up, 1 in 2024-08-31T07:03:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:54 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:54.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:03:54.927+0000 7f976414d640 1 -- 172.21.15.73:0/3727004627 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f975c0635a0 con 0x7f97480b4b70 2024-08-31T07:03:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:54 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:03:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:54 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:03:54.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:54 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:54.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:54 smithi084 ceph-mon[22387]: osd.0 [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] boot 2024-08-31T07:03:54.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:54 smithi084 ceph-mon[22387]: osdmap e9: 1 total, 1 up, 1 in 2024-08-31T07:03:54.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:54 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T07:03:55.605 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:55 smithi073 ceph-mon[21995]: pgmap v32: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:03:55.605 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:55 smithi073 ceph-mon[21995]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:03:55.605 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:03:55.605 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:03:55.605 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:55.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:55 smithi032 ceph-mon[17252]: pgmap v32: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:03:55.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:55 smithi032 ceph-mon[17252]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:03:55.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:03:55.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:03:55.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:55.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:55 smithi084 ceph-mon[22387]: pgmap v32: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:03:55.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:55 smithi084 ceph-mon[22387]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:03:55.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:03:55.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:03:55.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:56.739 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:56 smithi073 ceph-mon[21995]: osdmap e10: 1 total, 1 up, 1 in 2024-08-31T07:03:56.739 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:56 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi032 2024-08-31T07:03:56.739 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:56 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.739 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:56 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.739 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:56 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:03:56.739 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:56 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi032 to 17094M 2024-08-31T07:03:56.740 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:56 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.740 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:56 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:56.740 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:56 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:03:56.740 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:56 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:56 smithi032 ceph-mon[17252]: osdmap e10: 1 total, 1 up, 1 in 2024-08-31T07:03:56.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:56 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi032 2024-08-31T07:03:56.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:56 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:56 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:56 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:03:56.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:56 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi032 to 17094M 2024-08-31T07:03:56.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:56 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:56 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:56.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:56 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:03:56.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:56 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:56 smithi084 ceph-mon[22387]: osdmap e10: 1 total, 1 up, 1 in 2024-08-31T07:03:56.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:56 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi032 2024-08-31T07:03:56.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:56 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:56 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:56 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:03:56.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:56 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi032 to 17094M 2024-08-31T07:03:56.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:56 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:56.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:56 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:03:56.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:56 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:03:56.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:56 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:03:57.848 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:57 smithi073 ceph-mon[21995]: pgmap v34: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:03:57.848 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:57 smithi073 ceph-mon[21995]: from='client.? 172.21.15.73:0/1090448067' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0035b2ca-ad88-42e8-a46b-da03efcc7c04"} : dispatch 2024-08-31T07:03:57.848 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:57 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0035b2ca-ad88-42e8-a46b-da03efcc7c04"} : dispatch 2024-08-31T07:03:57.848 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:57 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0035b2ca-ad88-42e8-a46b-da03efcc7c04"}]': finished 2024-08-31T07:03:57.848 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:57 smithi073 ceph-mon[21995]: osdmap e11: 2 total, 1 up, 2 in 2024-08-31T07:03:57.849 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:03:57.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:57 smithi032 ceph-mon[17252]: pgmap v34: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:03:57.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:57 smithi032 ceph-mon[17252]: from='client.? 172.21.15.73:0/1090448067' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0035b2ca-ad88-42e8-a46b-da03efcc7c04"} : dispatch 2024-08-31T07:03:57.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:57 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0035b2ca-ad88-42e8-a46b-da03efcc7c04"} : dispatch 2024-08-31T07:03:57.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:57 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0035b2ca-ad88-42e8-a46b-da03efcc7c04"}]': finished 2024-08-31T07:03:57.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:57 smithi032 ceph-mon[17252]: osdmap e11: 2 total, 1 up, 2 in 2024-08-31T07:03:57.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:03:57.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:57 smithi084 ceph-mon[22387]: pgmap v34: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:03:57.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:57 smithi084 ceph-mon[22387]: from='client.? 172.21.15.73:0/1090448067' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0035b2ca-ad88-42e8-a46b-da03efcc7c04"} : dispatch 2024-08-31T07:03:57.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:57 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0035b2ca-ad88-42e8-a46b-da03efcc7c04"} : dispatch 2024-08-31T07:03:57.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:57 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0035b2ca-ad88-42e8-a46b-da03efcc7c04"}]': finished 2024-08-31T07:03:57.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:57 smithi084 ceph-mon[22387]: osdmap e11: 2 total, 1 up, 2 in 2024-08-31T07:03:57.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:03:58.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:58 smithi032 ceph-mon[17252]: from='client.? 172.21.15.73:0/2642495669' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:03:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:58 smithi073 ceph-mon[21995]: from='client.? 172.21.15.73:0/2642495669' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:03:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:58 smithi084 ceph-mon[22387]: from='client.? 172.21.15.73:0/2642495669' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:03:59.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:03:59 smithi032 ceph-mon[17252]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:03:59.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:03:59 smithi073 ceph-mon[21995]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:03:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:03:59 smithi084 ceph-mon[22387]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:01 smithi084 ceph-mon[22387]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:01 smithi032 ceph-mon[17252]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:02.109 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:01 smithi073 ceph-mon[21995]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:03.781 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:03 smithi073 ceph-mon[21995]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:03.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:03 smithi084 ceph-mon[22387]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:03 smithi032 ceph-mon[17252]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:05.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:05 smithi073 ceph-mon[21995]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:05.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-31T07:04:05.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:06.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:05 smithi032 ceph-mon[17252]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:06.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-31T07:04:06.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:05 smithi084 ceph-mon[22387]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-31T07:04:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:06 smithi073 ceph-mon[21995]: Deploying daemon osd.1 on smithi073 2024-08-31T07:04:07.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:06 smithi032 ceph-mon[17252]: Deploying daemon osd.1 on smithi073 2024-08-31T07:04:07.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:06 smithi084 ceph-mon[22387]: Deploying daemon osd.1 on smithi073 2024-08-31T07:04:08.013 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:07 smithi073 ceph-mon[21995]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:08.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:07 smithi032 ceph-mon[17252]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:08.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:07 smithi084 ceph-mon[22387]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:09.855 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:09 smithi073 ceph-mon[21995]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:10.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:09 smithi032 ceph-mon[17252]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:10.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:09 smithi084 ceph-mon[22387]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:11.085 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:10 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:11.085 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:10 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:11.085 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:10 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:11.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:10 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:11.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:10 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:11.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:10 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:11.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:10 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:11.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:10 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:11.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:10 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:12.087 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:11 smithi073 ceph-mon[21995]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:12.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:11 smithi032 ceph-mon[17252]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:12.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:11 smithi084 ceph-mon[22387]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:14.333 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:14 smithi073 ceph-mon[21995]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:14.334 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:14.334 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:14.334 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:14.334 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:14.334 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:14.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:14 smithi032 ceph-mon[17252]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:14.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:14.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:14.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:14.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:14.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:14 smithi084 ceph-mon[22387]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:14.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:14.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:14.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:14.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.618+0000 7f9752ffd640 1 -- 172.21.15.73:0/3727004627 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f975c0635a0 con 0x7f97480b4b70 2024-08-31T07:04:14.619 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 1 on host 'smithi073' 2024-08-31T07:04:14.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 -- 172.21.15.73:0/3727004627 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97480b4b70 msgr2=0x7f97480b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:04:14.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97480b4b70 0x7f97480b7080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f974c005fd0 tx=0x7f974c005850 comp rx=0 tx=0).stop 2024-08-31T07:04:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 -- 172.21.15.73:0/3727004627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f975c102800 msgr2=0x7f975c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:04:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f975c102800 0x7f975c19a2f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f975800ba50 tx=0x7f975800bf10 comp rx=0 tx=0).stop 2024-08-31T07:04:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 -- 172.21.15.73:0/3727004627 shutdown_connections 2024-08-31T07:04:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97480b4b70 0x7f97480b7080 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f975c108a90 0x7f975c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f975c108140 0x7f975c19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 --2- 172.21.15.73:0/3727004627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f975c102800 0x7f975c19a2f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 -- 172.21.15.73:0/3727004627 >> 172.21.15.73:0/3727004627 conn(0x7f975c0794d0 msgr2=0x7f975c1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:04:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 -- 172.21.15.73:0/3727004627 shutdown_connections 2024-08-31T07:04:14.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:04:14.620+0000 7f976414d640 1 -- 172.21.15.73:0/3727004627 wait complete. 2024-08-31T07:04:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:15.248 DEBUG:teuthology.orchestra.run.smithi073:osd.1> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.1.service 2024-08-31T07:04:15.250 INFO:tasks.cephadm:Deploying osd.2 on smithi084 with /dev/vg_nvme/lv_4... 2024-08-31T07:04:15.250 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-31T07:04:15.274 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:15.274 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:15.274 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:15.277 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:15.277 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:15.277 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:15.501 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:04:16.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:16 smithi084 ceph-mon[22387]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:16.345 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:16 smithi073 ceph-mon[21995]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:16.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:16 smithi032 ceph-mon[17252]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:16.634 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:16 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1[27600]: 2024-08-31T07:04:16.483+0000 7ff1f5602640 -1 osd.1 0 log_to_monitors true 2024-08-31T07:04:17.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:17 smithi032 ceph-mon[17252]: from='osd.1 [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T07:04:17.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:17 smithi032 ceph-mon[17252]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T07:04:17.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:17 smithi073 ceph-mon[21995]: from='osd.1 [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T07:04:17.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:17 smithi073 ceph-mon[21995]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T07:04:17.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:17 smithi084 ceph-mon[22387]: from='osd.1 [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T07:04:17.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:17 smithi084 ceph-mon[22387]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T07:04:18.054 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T07:04:18.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_4 2024-08-31T07:04:18.162 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:18.162 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-31T07:04:18.162 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: osdmap e12: 2 total, 1 up, 2 in 2024-08-31T07:04:18.162 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:18.163 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='osd.1 [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:04:18.163 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:04:18.163 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.163 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.163 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:04:18.163 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.163 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:18.163 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:18.163 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.180 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1[27600]: 2024-08-31T07:04:18.028+0000 7ff1f1584640 -1 osd.1 0 waiting for initial osdmap 2024-08-31T07:04:18.180 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1[27600]: 2024-08-31T07:04:18.034+0000 7ff1ed382640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:04:18.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:18.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-31T07:04:18.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: osdmap e12: 2 total, 1 up, 2 in 2024-08-31T07:04:18.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:18.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='osd.1 [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:04:18.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:04:18.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.182 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:04:18.182 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.182 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:18.182 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:18.182 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.384 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:04:18.573 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:18.573 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-31T07:04:18.573 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: osdmap e12: 2 total, 1 up, 2 in 2024-08-31T07:04:18.573 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:18.573 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='osd.1 [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:04:18.573 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:04:18.573 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.573 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:04:18.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:18.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:18.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:18.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.824+0000 7fa048b43640 1 -- 172.21.15.84:0/2933661538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa044109250 msgr2=0x7fa044109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:04:18.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.824+0000 7fa048b43640 1 --2- 172.21.15.84:0/2933661538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa044109250 0x7fa044109660 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa038009900 tx=0x7fa03802a410 comp rx=0 tx=0).stop 2024-08-31T07:04:18.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.824+0000 7fa048b43640 1 -- 172.21.15.84:0/2933661538 shutdown_connections 2024-08-31T07:04:18.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.824+0000 7fa048b43640 1 --2- 172.21.15.84:0/2933661538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa044109ba0 0x7fa0441103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:18.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.824+0000 7fa048b43640 1 --2- 172.21.15.84:0/2933661538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa044109250 0x7fa044109660 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:18.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.824+0000 7fa048b43640 1 --2- 172.21.15.84:0/2933661538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa044103960 0x7fa044103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:18.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.824+0000 7fa048b43640 1 -- 172.21.15.84:0/2933661538 >> 172.21.15.84:0/2933661538 conn(0x7fa04407b6b0 msgr2=0x7fa04407bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:04:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.824+0000 7fa048b43640 1 -- 172.21.15.84:0/2933661538 shutdown_connections 2024-08-31T07:04:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.824+0000 7fa048b43640 1 -- 172.21.15.84:0/2933661538 wait complete. 2024-08-31T07:04:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa048b43640 1 Processor -- start 2024-08-31T07:04:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa048b43640 1 -- start start 2024-08-31T07:04:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa048b43640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa044103960 0x7fa04419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:04:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa048b43640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa044109250 0x7fa04419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:04:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa048b43640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa044109ba0 0x7fa04419fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:04:18.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa048b43640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441138b0 con 0x7fa044109250 2024-08-31T07:04:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa048b43640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa044113730 con 0x7fa044103960 2024-08-31T07:04:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa048b43640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa044113a30 con 0x7fa044109ba0 2024-08-31T07:04:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa042d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa044109ba0 0x7fa04419fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:04:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa042d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa044109ba0 0x7fa04419fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34282/0 (socket says 172.21.15.84:34282) 2024-08-31T07:04:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.825+0000 7fa042d9d640 1 -- 172.21.15.84:0/2358011741 learned_addr learned my addr 172.21.15.84:0/2358011741 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-08-31T07:04:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa04259c640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa044103960 0x7fa04419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:04:18.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa041d9b640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa044109250 0x7fa04419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:04:18.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa042d9d640 1 -- 172.21.15.84:0/2358011741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa044103960 msgr2=0x7fa04419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:04:18.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa042d9d640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa044103960 0x7fa04419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:18.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa042d9d640 1 -- 172.21.15.84:0/2358011741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa044109250 msgr2=0x7fa04419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:04:18.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa042d9d640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa044109250 0x7fa04419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:18.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa042d9d640 1 -- 172.21.15.84:0/2358011741 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0441a0170 con 0x7fa044109ba0 2024-08-31T07:04:18.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa04259c640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa044103960 0x7fa04419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:04:18.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa042d9d640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa044109ba0 0x7fa04419fa40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00eaa0 tx=0x7fa03c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:04:18.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa0337fe640 1 -- 172.21.15.84:0/2358011741 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00cdb0 con 0x7fa044109ba0 2024-08-31T07:04:18.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa0337fe640 1 -- 172.21.15.84:0/2358011741 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa03c005020 con 0x7fa044109ba0 2024-08-31T07:04:18.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa048b43640 1 -- 172.21.15.84:0/2358011741 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0441a0460 con 0x7fa044109ba0 2024-08-31T07:04:18.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa0337fe640 1 -- 172.21.15.84:0/2358011741 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0116d0 con 0x7fa044109ba0 2024-08-31T07:04:18.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.826+0000 7fa048b43640 1 -- 172.21.15.84:0/2358011741 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0441a6bc0 con 0x7fa044109ba0 2024-08-31T07:04:18.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.827+0000 7fa048b43640 1 -- 172.21.15.84:0/2358011741 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa044103d70 con 0x7fa044109ba0 2024-08-31T07:04:18.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.830+0000 7fa0337fe640 1 -- 172.21.15.84:0/2358011741 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fa03c0051d0 con 0x7fa044109ba0 2024-08-31T07:04:18.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.831+0000 7fa0337fe640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa02c0b4b70 0x7fa02c0b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:04:18.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.831+0000 7fa0337fe640 1 -- 172.21.15.84:0/2358011741 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7fa03c0e3e60 con 0x7fa044109ba0 2024-08-31T07:04:18.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.831+0000 7fa04259c640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa02c0b4b70 0x7fa02c0b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:04:18.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.832+0000 7fa04259c640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa02c0b4b70 0x7fa02c0b7080 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa034005d00 tx=0x7fa034005c70 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:04:18.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:18.833+0000 7fa0337fe640 1 -- 172.21.15.84:0/2358011741 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa03c016030 con 0x7fa044109ba0 2024-08-31T07:04:19.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:19.025+0000 7fa048b43640 1 -- 172.21.15.84:0/2358011741 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa0440635a0 con 0x7fa02c0b4b70 2024-08-31T07:04:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:19 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi073 2024-08-31T07:04:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:19 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi073 to 17088M 2024-08-31T07:04:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:19 smithi084 ceph-mon[22387]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:04:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:19 smithi084 ceph-mon[22387]: osdmap e13: 2 total, 1 up, 2 in 2024-08-31T07:04:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:19 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:19 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:19 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi073 2024-08-31T07:04:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:19 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi073 to 17088M 2024-08-31T07:04:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:19 smithi032 ceph-mon[17252]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:04:19.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:19 smithi032 ceph-mon[17252]: osdmap e13: 2 total, 1 up, 2 in 2024-08-31T07:04:19.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:19 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:19.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:19 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:19 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi073 2024-08-31T07:04:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:19 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi073 to 17088M 2024-08-31T07:04:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:19 smithi073 ceph-mon[21995]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:04:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:19 smithi073 ceph-mon[21995]: osdmap e13: 2 total, 1 up, 2 in 2024-08-31T07:04:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:19 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:19 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:20.340 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:20 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:04:20.341 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:20 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:04:20.341 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:20 smithi073 ceph-mon[21995]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:20.341 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:20 smithi073 ceph-mon[21995]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:04:20.341 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:04:20.341 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:20 smithi073 ceph-mon[21995]: osd.1 [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] boot 2024-08-31T07:04:20.341 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:20 smithi073 ceph-mon[21995]: osdmap e14: 2 total, 2 up, 2 in 2024-08-31T07:04:20.341 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:20.341 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:04:20.341 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:20 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:04:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:20 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:04:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:20 smithi032 ceph-mon[17252]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:20 smithi032 ceph-mon[17252]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:04:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:04:20.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:20 smithi032 ceph-mon[17252]: osd.1 [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] boot 2024-08-31T07:04:20.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:20 smithi032 ceph-mon[17252]: osdmap e14: 2 total, 2 up, 2 in 2024-08-31T07:04:20.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:20.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:04:20.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:20.410 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:20 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:04:20.410 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:20 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:04:20.410 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:20 smithi084 ceph-mon[22387]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T07:04:20.410 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:20 smithi084 ceph-mon[22387]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:04:20.411 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:04:20.411 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:20 smithi084 ceph-mon[22387]: osd.1 [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] boot 2024-08-31T07:04:20.411 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:20 smithi084 ceph-mon[22387]: osdmap e14: 2 total, 2 up, 2 in 2024-08-31T07:04:20.411 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T07:04:20.411 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:04:20.411 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:21.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:21 smithi032 ceph-mon[17252]: osdmap e15: 2 total, 2 up, 2 in 2024-08-31T07:04:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:21 smithi073 ceph-mon[21995]: osdmap e15: 2 total, 2 up, 2 in 2024-08-31T07:04:21.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:21 smithi084 ceph-mon[22387]: osdmap e15: 2 total, 2 up, 2 in 2024-08-31T07:04:22.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:22 smithi084 ceph-mon[22387]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:22.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:22 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/3962727595' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "395e3fdd-1a48-43cc-afad-7af3af791e14"} : dispatch 2024-08-31T07:04:22.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:22 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "395e3fdd-1a48-43cc-afad-7af3af791e14"} : dispatch 2024-08-31T07:04:22.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:22 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "395e3fdd-1a48-43cc-afad-7af3af791e14"}]': finished 2024-08-31T07:04:22.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:22 smithi084 ceph-mon[22387]: osdmap e16: 3 total, 2 up, 3 in 2024-08-31T07:04:22.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:22.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:22 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/3222498277' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:04:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:22 smithi032 ceph-mon[17252]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:22 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/3962727595' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "395e3fdd-1a48-43cc-afad-7af3af791e14"} : dispatch 2024-08-31T07:04:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:22 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "395e3fdd-1a48-43cc-afad-7af3af791e14"} : dispatch 2024-08-31T07:04:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:22 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "395e3fdd-1a48-43cc-afad-7af3af791e14"}]': finished 2024-08-31T07:04:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:22 smithi032 ceph-mon[17252]: osdmap e16: 3 total, 2 up, 3 in 2024-08-31T07:04:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:22 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/3222498277' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:04:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:22 smithi073 ceph-mon[21995]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:22 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/3962727595' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "395e3fdd-1a48-43cc-afad-7af3af791e14"} : dispatch 2024-08-31T07:04:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:22 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "395e3fdd-1a48-43cc-afad-7af3af791e14"} : dispatch 2024-08-31T07:04:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:22 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "395e3fdd-1a48-43cc-afad-7af3af791e14"}]': finished 2024-08-31T07:04:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:22 smithi073 ceph-mon[21995]: osdmap e16: 3 total, 2 up, 3 in 2024-08-31T07:04:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:22 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/3222498277' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:04:24.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:24 smithi032 ceph-mon[17252]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:24 smithi073 ceph-mon[21995]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:24 smithi084 ceph-mon[22387]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:26.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:26 smithi032 ceph-mon[17252]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:26 smithi073 ceph-mon[21995]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:26 smithi084 ceph-mon[22387]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:28.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:28 smithi084 ceph-mon[22387]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:28.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:28 smithi032 ceph-mon[17252]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:28 smithi073 ceph-mon[21995]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:30 smithi032 ceph-mon[17252]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-31T07:04:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:30.374 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:30 smithi084 ceph-mon[22387]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:30.374 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-31T07:04:30.374 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:30 smithi073 ceph-mon[21995]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-31T07:04:30.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:31.343 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:31 smithi084 ceph-mon[22387]: Deploying daemon osd.2 on smithi084 2024-08-31T07:04:31.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:31 smithi032 ceph-mon[17252]: Deploying daemon osd.2 on smithi084 2024-08-31T07:04:31.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:31 smithi073 ceph-mon[21995]: Deploying daemon osd.2 on smithi084 2024-08-31T07:04:32.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:32 smithi032 ceph-mon[17252]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:32.374 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:32 smithi084 ceph-mon[22387]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:32 smithi073 ceph-mon[21995]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:34.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:34 smithi032 ceph-mon[17252]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:34 smithi073 ceph-mon[21995]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:34 smithi084 ceph-mon[22387]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:35.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:35 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:35.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:35 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:35.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:35 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:35.367 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:35 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:35.367 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:35 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:35.367 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:35 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:35.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:35 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:35.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:35 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:35.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:35 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:36.354 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:36 smithi084 ceph-mon[22387]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:36.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:36 smithi032 ceph-mon[17252]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:36.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:36 smithi073 ceph-mon[21995]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:38 smithi032 ceph-mon[17252]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:38 smithi073 ceph-mon[21995]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:38.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:38.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:38.718 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:38 smithi084 ceph-mon[22387]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:38.718 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:38.718 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:38.718 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:38.718 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:38.718 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:39.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.019+0000 7fa0337fe640 1 -- 172.21.15.84:0/2358011741 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa0440635a0 con 0x7fa02c0b4b70 2024-08-31T07:04:39.021 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 2 on host 'smithi084' 2024-08-31T07:04:39.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 -- 172.21.15.84:0/2358011741 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa02c0b4b70 msgr2=0x7fa02c0b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:04:39.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa02c0b4b70 0x7fa02c0b7080 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa034005d00 tx=0x7fa034005c70 comp rx=0 tx=0).stop 2024-08-31T07:04:39.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 -- 172.21.15.84:0/2358011741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa044109ba0 msgr2=0x7fa04419fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:04:39.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa044109ba0 0x7fa04419fa40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00eaa0 tx=0x7fa03c00ef60 comp rx=0 tx=0).stop 2024-08-31T07:04:39.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 -- 172.21.15.84:0/2358011741 shutdown_connections 2024-08-31T07:04:39.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa02c0b4b70 0x7fa02c0b7080 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:39.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa044109ba0 0x7fa04419fa40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:39.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa044109250 0x7fa04419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:39.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 --2- 172.21.15.84:0/2358011741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa044103960 0x7fa04419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:39.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 -- 172.21.15.84:0/2358011741 >> 172.21.15.84:0/2358011741 conn(0x7fa04407b6b0 msgr2=0x7fa044107260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:04:39.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 -- 172.21.15.84:0/2358011741 shutdown_connections 2024-08-31T07:04:39.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:04:39.022+0000 7fa048b43640 1 -- 172.21.15.84:0/2358011741 wait complete. 2024-08-31T07:04:39.596 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:39.596 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:39.596 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:39.623 DEBUG:teuthology.orchestra.run.smithi084:osd.2> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.2.service 2024-08-31T07:04:39.665 INFO:tasks.cephadm:Deploying osd.3 on smithi032 with /dev/vg_nvme/lv_3... 2024-08-31T07:04:39.666 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-31T07:04:39.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:04:39.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:39.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:04:40.466 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:40 smithi032 ceph-mon[17252]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:40.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:40 smithi084 ceph-mon[22387]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:40 smithi073 ceph-mon[21995]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:41.121 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:40 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2[28669]: 2024-08-31T07:04:40.901+0000 7f2b7a40a640 -1 osd.2 0 log_to_monitors true 2024-08-31T07:04:41.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:41 smithi084 ceph-mon[22387]: from='osd.2 [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T07:04:41.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:41 smithi084 ceph-mon[22387]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T07:04:41.595 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:41 smithi032 ceph-mon[17252]: from='osd.2 [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T07:04:41.595 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:41 smithi032 ceph-mon[17252]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T07:04:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:41 smithi073 ceph-mon[21995]: from='osd.2 [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T07:04:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:41 smithi073 ceph-mon[21995]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T07:04:42.453 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:42 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2[28669]: 2024-08-31T07:04:42.364+0000 7f2b7638c640 -1 osd.2 0 waiting for initial osdmap 2024-08-31T07:04:42.454 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:42 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2[28669]: 2024-08-31T07:04:42.372+0000 7f2b7218a640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:04:42.454 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:42 smithi084 ceph-mon[22387]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:42.454 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:42 smithi084 ceph-mon[22387]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-31T07:04:42.454 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:42 smithi084 ceph-mon[22387]: from='osd.2 [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:04:42.454 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:42 smithi084 ceph-mon[22387]: osdmap e17: 3 total, 2 up, 3 in 2024-08-31T07:04:42.454 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:42 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:42.454 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:42 smithi084 ceph-mon[22387]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:04:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:42 smithi032 ceph-mon[17252]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:42 smithi032 ceph-mon[17252]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-31T07:04:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:42 smithi032 ceph-mon[17252]: from='osd.2 [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:04:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:42 smithi032 ceph-mon[17252]: osdmap e17: 3 total, 2 up, 3 in 2024-08-31T07:04:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:42 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:42.597 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:42 smithi032 ceph-mon[17252]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:04:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:42 smithi073 ceph-mon[21995]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:42 smithi073 ceph-mon[21995]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-31T07:04:42.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:42 smithi073 ceph-mon[21995]: from='osd.2 [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:04:42.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:42 smithi073 ceph-mon[21995]: osdmap e17: 3 total, 2 up, 3 in 2024-08-31T07:04:42.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:42 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:42.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:42 smithi073 ceph-mon[21995]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:04:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:04:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: osdmap e18: 3 total, 2 up, 3 in 2024-08-31T07:04:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi084 2024-08-31T07:04:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:04:43.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi084 to 17088M 2024-08-31T07:04:43.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:43.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:43.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:04:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: osdmap e18: 3 total, 2 up, 3 in 2024-08-31T07:04:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:43.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi084 2024-08-31T07:04:43.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:04:43.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi084 to 17088M 2024-08-31T07:04:43.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:43.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:43.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.692 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:04:43.724 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_3 2024-08-31T07:04:43.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:04:43.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: osdmap e18: 3 total, 2 up, 3 in 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi084 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi084 to 17088M 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:04:43.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:04:44.565 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:44 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:04:44.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:44 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:04:44.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:44 smithi032 ceph-mon[17252]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:44.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:44.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:44 smithi032 ceph-mon[17252]: osd.2 [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] boot 2024-08-31T07:04:44.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:44 smithi032 ceph-mon[17252]: osdmap e19: 3 total, 3 up, 3 in 2024-08-31T07:04:44.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:44 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:04:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:44 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:04:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:44 smithi073 ceph-mon[21995]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:44.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:44.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:44 smithi073 ceph-mon[21995]: osd.2 [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] boot 2024-08-31T07:04:44.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:44 smithi073 ceph-mon[21995]: osdmap e19: 3 total, 3 up, 3 in 2024-08-31T07:04:44.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:44 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:04:44.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:44 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:04:44.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:44 smithi084 ceph-mon[22387]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-31T07:04:44.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:44.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:44 smithi084 ceph-mon[22387]: osd.2 [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] boot 2024-08-31T07:04:44.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:44 smithi084 ceph-mon[22387]: osdmap e19: 3 total, 3 up, 3 in 2024-08-31T07:04:44.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T07:04:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.747+0000 7f8538a00640 1 -- 172.21.15.32:0/761394644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853410b290 msgr2=0x7f853410b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:04:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.747+0000 7f8538a00640 1 --2- 172.21.15.32:0/761394644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853410b290 0x7f853410b6a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8520009900 tx=0x7f852002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:04:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.747+0000 7f8538a00640 1 -- 172.21.15.32:0/761394644 shutdown_connections 2024-08-31T07:04:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.747+0000 7f8538a00640 1 --2- 172.21.15.32:0/761394644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f853410bbe0 0x7f8534112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.747+0000 7f8538a00640 1 --2- 172.21.15.32:0/761394644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853410b290 0x7f853410b6a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.747+0000 7f8538a00640 1 --2- 172.21.15.32:0/761394644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8534105950 0x7f8534105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.747+0000 7f8538a00640 1 -- 172.21.15.32:0/761394644 >> 172.21.15.32:0/761394644 conn(0x7f8534101420 msgr2=0x7f8534103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:04:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.747+0000 7f8538a00640 1 -- 172.21.15.32:0/761394644 shutdown_connections 2024-08-31T07:04:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.747+0000 7f8538a00640 1 -- 172.21.15.32:0/761394644 wait complete. 2024-08-31T07:04:44.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.748+0000 7f8538a00640 1 Processor -- start 2024-08-31T07:04:44.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.748+0000 7f8538a00640 1 -- start start 2024-08-31T07:04:44.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.748+0000 7f8538a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8534105950 0x7f853419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:04:44.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.748+0000 7f8538a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853410b290 0x7f853419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.748+0000 7f8538a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f853410bbe0 0x7f85341a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.748+0000 7f8538a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85341158a0 con 0x7f853410b290 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.748+0000 7f8538a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8534115720 con 0x7f8534105950 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.748+0000 7f8538a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8534115a20 con 0x7f853410bbe0 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8532a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853410b290 0x7f853419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8533400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8534105950 0x7f853419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8533400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8534105950 0x7f853419c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37956/0 (socket says 172.21.15.32:37956) 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8533400640 1 -- 172.21.15.32:0/1863368651 learned_addr learned my addr 172.21.15.32:0/1863368651 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8533e00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f853410bbe0 0x7f85341a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8533400640 1 -- 172.21.15.32:0/1863368651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f853410bbe0 msgr2=0x7f85341a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:04:44.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8533400640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f853410bbe0 0x7f85341a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8533400640 1 -- 172.21.15.32:0/1863368651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853410b290 msgr2=0x7f853419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8533400640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853410b290 0x7f853419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8533400640 1 -- 172.21.15.32:0/1863368651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85341a22a0 con 0x7f8534105950 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.749+0000 7f8532a00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853410b290 0x7f853419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.750+0000 7f8533400640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8534105950 0x7f853419c330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f852400ea60 tx=0x7f852400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.750+0000 7f852be00640 1 -- 172.21.15.32:0/1863368651 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852400cde0 con 0x7f8534105950 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.750+0000 7f8538a00640 1 -- 172.21.15.32:0/1863368651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85341a2590 con 0x7f8534105950 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.750+0000 7f852be00640 1 -- 172.21.15.32:0/1863368651 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85240035f0 con 0x7f8534105950 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.750+0000 7f852be00640 1 -- 172.21.15.32:0/1863368651 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524010690 con 0x7f8534105950 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.750+0000 7f8538a00640 1 -- 172.21.15.32:0/1863368651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85341a8cf0 con 0x7f8534105950 2024-08-31T07:04:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.751+0000 7f8538a00640 1 -- 172.21.15.32:0/1863368651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8534105d60 con 0x7f8534105950 2024-08-31T07:04:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.754+0000 7f852be00640 1 -- 172.21.15.32:0/1863368651 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f85240037a0 con 0x7f8534105950 2024-08-31T07:04:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.755+0000 7f852be00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85180b4af0 0x7f85180b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:04:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.755+0000 7f852be00640 1 -- 172.21.15.32:0/1863368651 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3066+0+0 (secure 0 0 0) 0x7f85240e40e0 con 0x7f8534105950 2024-08-31T07:04:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.755+0000 7f8532a00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85180b4af0 0x7f85180b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:04:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.756+0000 7f8532a00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85180b4af0 0x7f85180b7000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8520009230 tx=0x7f8520031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:04:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.757+0000 7f852be00640 1 -- 172.21.15.32:0/1863368651 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8524056130 con 0x7f8534105950 2024-08-31T07:04:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:44.965+0000 7f8538a00640 1 -- 172.21.15.32:0/1863368651 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f85340635a0 con 0x7f85180b4af0 2024-08-31T07:04:45.741 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:45 smithi032 ceph-mon[17252]: osdmap e20: 3 total, 3 up, 3 in 2024-08-31T07:04:45.742 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:04:45.742 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:04:45.742 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:45.742 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-31T07:04:45.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:45 smithi073 ceph-mon[21995]: osdmap e20: 3 total, 3 up, 3 in 2024-08-31T07:04:45.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:04:45.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:04:45.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:45.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-31T07:04:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:45 smithi084 ceph-mon[22387]: osdmap e20: 3 total, 3 up, 3 in 2024-08-31T07:04:45.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:04:45.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:04:45.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:45.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-31T07:04:46.777 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39241]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T07:04:46.777 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39241]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:46.778 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39241]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:46.778 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39241]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:46.778 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:46 smithi032 ceph-mon[17252]: pgmap v68: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:46.778 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:46 smithi032 ceph-mon[17252]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:04:46.778 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-31T07:04:46.778 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:46 smithi032 ceph-mon[17252]: osdmap e21: 3 total, 3 up, 3 in 2024-08-31T07:04:46.778 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-31T07:04:46.836 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:46 smithi073 ceph-mon[21995]: pgmap v68: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:46.836 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:46 smithi073 ceph-mon[21995]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:04:46.836 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-31T07:04:46.836 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:46 smithi073 ceph-mon[21995]: osdmap e21: 3 total, 3 up, 3 in 2024-08-31T07:04:46.836 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-31T07:04:46.912 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:46 smithi084 ceph-mon[22387]: pgmap v68: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:46.912 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:46 smithi084 ceph-mon[22387]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:04:46.912 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-31T07:04:46.912 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:46 smithi084 ceph-mon[22387]: osdmap e21: 3 total, 3 up, 3 in 2024-08-31T07:04:46.913 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-31T07:04:47.036 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39268]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T07:04:47.036 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39268]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.036 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39268]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.036 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 sudo[39268]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.036 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39246]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T07:04:47.036 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39246]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.036 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39246]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.036 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:04:46 smithi032 sudo[39246]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.101 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:46 smithi073 sudo[31319]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T07:04:47.101 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:46 smithi073 sudo[31319]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.101 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:46 smithi073 sudo[31319]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.101 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:46 smithi073 sudo[31319]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.101 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:46 smithi073 sudo[31322]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T07:04:47.101 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:46 smithi073 sudo[31322]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.101 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:46 smithi073 sudo[31322]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.101 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:04:46 smithi073 sudo[31322]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 sudo[32322]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T07:04:47.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 sudo[32322]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 sudo[32322]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 sudo[32322]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 sudo[32325]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T07:04:47.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 sudo[32325]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 sudo[32325]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 sudo[32325]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.241 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:46 smithi084 sudo[32316]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T07:04:47.241 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:46 smithi084 sudo[32316]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.241 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:46 smithi084 sudo[32316]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.241 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:46 smithi084 sudo[32316]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.241 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:46 smithi084 sudo[32319]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T07:04:47.241 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:46 smithi084 sudo[32319]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.242 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:46 smithi084 sudo[32319]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.242 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:04:46 smithi084 sudo[32319]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.302 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 sudo[39271]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T07:04:47.302 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 sudo[39271]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.302 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 sudo[39271]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.302 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 sudo[39271]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 sudo[31325]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T07:04:47.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 sudo[31325]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 sudo[31325]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 sudo[31325]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 sudo[31328]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T07:04:47.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 sudo[31328]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T07:04:47.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 sudo[31328]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T07:04:47.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 sudo[31328]: pam_unix(sudo:session): session closed for user root 2024-08-31T07:04:47.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:04:47.540+0000 7f852be00640 1 -- 172.21.15.32:0/1863368651 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f8524004000 con 0x7f8534105950 2024-08-31T07:04:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-31T07:04:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: osdmap e22: 3 total, 3 up, 3 in 2024-08-31T07:04:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T07:04:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T07:04:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:04:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:47 smithi032 ceph-mon[17252]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T07:04:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-31T07:04:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: osdmap e22: 3 total, 3 up, 3 in 2024-08-31T07:04:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T07:04:47.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:04:47.931 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:04:47.931 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:04:47.931 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:47 smithi073 ceph-mon[21995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T07:04:47.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-31T07:04:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: osdmap e22: 3 total, 3 up, 3 in 2024-08-31T07:04:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:04:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T07:04:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T07:04:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:04:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:04:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:04:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T07:04:47.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:04:47.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:04:47.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:04:47.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T07:04:47.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T07:04:47.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T07:04:47.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T07:04:47.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T07:04:47.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:47 smithi084 ceph-mon[22387]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T07:04:48.837 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:48 smithi032 ceph-mon[17252]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:48.838 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:48 smithi032 ceph-mon[17252]: osdmap e23: 3 total, 3 up, 3 in 2024-08-31T07:04:48.838 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:48 smithi032 ceph-mon[17252]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-08-31T07:04:48.838 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:48 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3009296302' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bfaeb828-d33c-4791-95ec-79d1669b7316"} : dispatch 2024-08-31T07:04:48.838 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:48 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3009296302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfaeb828-d33c-4791-95ec-79d1669b7316"}]': finished 2024-08-31T07:04:48.838 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:48 smithi032 ceph-mon[17252]: osdmap e24: 4 total, 3 up, 4 in 2024-08-31T07:04:48.838 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:04:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:48 smithi073 ceph-mon[21995]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:48 smithi073 ceph-mon[21995]: osdmap e23: 3 total, 3 up, 3 in 2024-08-31T07:04:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:48 smithi073 ceph-mon[21995]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-08-31T07:04:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:48 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3009296302' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bfaeb828-d33c-4791-95ec-79d1669b7316"} : dispatch 2024-08-31T07:04:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:48 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3009296302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfaeb828-d33c-4791-95ec-79d1669b7316"}]': finished 2024-08-31T07:04:48.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:48 smithi073 ceph-mon[21995]: osdmap e24: 4 total, 3 up, 4 in 2024-08-31T07:04:48.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:04:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:48 smithi084 ceph-mon[22387]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:48 smithi084 ceph-mon[22387]: osdmap e23: 3 total, 3 up, 3 in 2024-08-31T07:04:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:48 smithi084 ceph-mon[22387]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-08-31T07:04:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:48 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3009296302' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bfaeb828-d33c-4791-95ec-79d1669b7316"} : dispatch 2024-08-31T07:04:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:48 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3009296302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfaeb828-d33c-4791-95ec-79d1669b7316"}]': finished 2024-08-31T07:04:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:48 smithi084 ceph-mon[22387]: osdmap e24: 4 total, 3 up, 4 in 2024-08-31T07:04:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:04:49.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:49 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2606583116' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:04:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:49 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2606583116' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:04:49.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:49 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2606583116' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:04:50.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:50 smithi032 ceph-mon[17252]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:50.894 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:50 smithi084 ceph-mon[22387]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:50 smithi073 ceph-mon[21995]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:52 smithi032 ceph-mon[17252]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:52 smithi073 ceph-mon[21995]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:52 smithi084 ceph-mon[22387]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:54 smithi032 ceph-mon[17252]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:54 smithi073 ceph-mon[21995]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:54 smithi084 ceph-mon[22387]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:56.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:56 smithi032 ceph-mon[17252]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:56 smithi073 ceph-mon[21995]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:56 smithi084 ceph-mon[22387]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:58 smithi032 ceph-mon[17252]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-31T07:04:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:58 smithi073 ceph-mon[21995]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-31T07:04:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:58 smithi084 ceph-mon[22387]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:04:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-31T07:04:58.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:04:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:04:59 smithi032 ceph-mon[17252]: Deploying daemon osd.3 on smithi032 2024-08-31T07:04:59.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:04:59 smithi073 ceph-mon[21995]: Deploying daemon osd.3 on smithi032 2024-08-31T07:04:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:04:59 smithi084 ceph-mon[22387]: Deploying daemon osd.3 on smithi032 2024-08-31T07:05:00.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:00 smithi032 ceph-mon[17252]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:00 smithi073 ceph-mon[21995]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:00 smithi084 ceph-mon[22387]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:01.786 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:01 smithi032 ceph-mon[17252]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:01 smithi073 ceph-mon[21995]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:01 smithi084 ceph-mon[22387]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:03.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:03 smithi032 ceph-mon[17252]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:03 smithi073 ceph-mon[21995]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:04.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:03 smithi084 ceph-mon[22387]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:06.045 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:05 smithi032 ceph-mon[17252]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:06.045 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:06.045 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:06.045 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:06.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:05 smithi073 ceph-mon[21995]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:06.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:06.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:06.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:05 smithi084 ceph-mon[22387]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:07.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:07 smithi032 ceph-mon[17252]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:08.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:07 smithi073 ceph-mon[21995]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:08.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:07 smithi084 ceph-mon[22387]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:09.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:09 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:09.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:09 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:09.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:09 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:09.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:09 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:09.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:09 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:09 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:09 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:09 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:09 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:09 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:09.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:09 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:09.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:09 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:09.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:09 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:09.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:09 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:09.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:09 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:10.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:10 smithi032 ceph-mon[17252]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:10.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:10 smithi073 ceph-mon[21995]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:10.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:10 smithi084 ceph-mon[22387]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:10.709 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 3 on host 'smithi032' 2024-08-31T07:05:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.707+0000 7f852be00640 1 -- 172.21.15.32:0/1863368651 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f85340635a0 con 0x7f85180b4af0 2024-08-31T07:05:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 -- 172.21.15.32:0/1863368651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85180b4af0 msgr2=0x7f85180b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:05:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85180b4af0 0x7f85180b7000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8520009230 tx=0x7f8520031400 comp rx=0 tx=0).stop 2024-08-31T07:05:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 -- 172.21.15.32:0/1863368651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8534105950 msgr2=0x7f853419c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:05:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8534105950 0x7f853419c330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f852400ea60 tx=0x7f852400ef20 comp rx=0 tx=0).stop 2024-08-31T07:05:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 -- 172.21.15.32:0/1863368651 shutdown_connections 2024-08-31T07:05:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85180b4af0 0x7f85180b7000 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f853410bbe0 0x7f85341a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853410b290 0x7f853419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 --2- 172.21.15.32:0/1863368651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8534105950 0x7f853419c330 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 -- 172.21.15.32:0/1863368651 >> 172.21.15.32:0/1863368651 conn(0x7f8534101420 msgr2=0x7f8534103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:05:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.710+0000 7f8538a00640 1 -- 172.21.15.32:0/1863368651 shutdown_connections 2024-08-31T07:05:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:05:10.711+0000 7f8538a00640 1 -- 172.21.15.32:0/1863368651 wait complete. 2024-08-31T07:05:11.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:11 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:11.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:11 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:11.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:11 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:11 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:11 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:11 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:11.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:11 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:11.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:11 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:11.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:11 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:11.568 DEBUG:teuthology.orchestra.run.smithi032:osd.3> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.3.service 2024-08-31T07:05:11.572 INFO:tasks.cephadm:Deploying osd.4 on smithi073 with /dev/vg_nvme/lv_3... 2024-08-31T07:05:11.572 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-31T07:05:11.825 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:05:12.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:12 smithi032 ceph-mon[17252]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:12.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:12 smithi032 ceph-mon[17252]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-31T07:05:12.253 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:12 smithi073 ceph-mon[21995]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:12.253 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:12 smithi073 ceph-mon[21995]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-31T07:05:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:12 smithi084 ceph-mon[22387]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:12 smithi084 ceph-mon[22387]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-31T07:05:13.277 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:13 smithi073 ceph-mon[21995]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-31T07:05:13.277 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:13 smithi073 ceph-mon[21995]: osdmap e25: 4 total, 3 up, 4 in 2024-08-31T07:05:13.277 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:13 smithi073 ceph-mon[21995]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:05:13.277 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:13 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:13.315 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:13 smithi032 ceph-mon[17252]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-31T07:05:13.315 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:13 smithi032 ceph-mon[17252]: osdmap e25: 4 total, 3 up, 4 in 2024-08-31T07:05:13.315 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:13 smithi032 ceph-mon[17252]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:05:13.315 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:13 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:13.315 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:05:13 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3[41331]: 2024-08-31T07:05:13.126+0000 7f3aec600640 -1 osd.3 0 waiting for initial osdmap 2024-08-31T07:05:13.316 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:05:13 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3[41331]: 2024-08-31T07:05:13.133+0000 7f3ae7600640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:05:13.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:13 smithi084 ceph-mon[22387]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-31T07:05:13.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:13 smithi084 ceph-mon[22387]: osdmap e25: 4 total, 3 up, 4 in 2024-08-31T07:05:13.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:13 smithi084 ceph-mon[22387]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:05:13.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:13 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:14.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:14 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:05:14.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:14 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:05:14.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:14 smithi073 ceph-mon[21995]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:14.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:14 smithi073 ceph-mon[21995]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:05:14.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:14 smithi073 ceph-mon[21995]: osdmap e26: 4 total, 3 up, 4 in 2024-08-31T07:05:14.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:14.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:14.401 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T07:05:14.428 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_3 2024-08-31T07:05:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:14 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:05:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:14 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:05:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:14 smithi084 ceph-mon[22387]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:14 smithi084 ceph-mon[22387]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:05:14.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:14 smithi084 ceph-mon[22387]: osdmap e26: 4 total, 3 up, 4 in 2024-08-31T07:05:14.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:14.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:14 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:05:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:14 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:05:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:14 smithi032 ceph-mon[17252]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-31T07:05:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:14 smithi032 ceph-mon[17252]: from='osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:05:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:14 smithi032 ceph-mon[17252]: osdmap e26: 4 total, 3 up, 4 in 2024-08-31T07:05:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:14.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:05:15.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.140+0000 7f67610e2640 1 -- 172.21.15.73:0/4021807579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f675c104620 msgr2=0x7f675c104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:05:15.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.140+0000 7f67610e2640 1 --2- 172.21.15.73:0/4021807579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f675c104620 0x7f675c104a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f674c009900 tx=0x7f674c02a410 comp rx=0 tx=0).stop 2024-08-31T07:05:15.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.140+0000 7f67610e2640 1 -- 172.21.15.73:0/4021807579 shutdown_connections 2024-08-31T07:05:15.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.140+0000 7f67610e2640 1 --2- 172.21.15.73:0/4021807579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f675c10dbd0 0x7f675c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:15.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.140+0000 7f67610e2640 1 --2- 172.21.15.73:0/4021807579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675c105050 0x7f675c10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:15.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.140+0000 7f67610e2640 1 --2- 172.21.15.73:0/4021807579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f675c104620 0x7f675c104a30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:15.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.140+0000 7f67610e2640 1 -- 172.21.15.73:0/4021807579 >> 172.21.15.73:0/4021807579 conn(0x7f675c0794e0 msgr2=0x7f675c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:05:15.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.140+0000 7f67610e2640 1 -- 172.21.15.73:0/4021807579 shutdown_connections 2024-08-31T07:05:15.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.140+0000 7f67610e2640 1 -- 172.21.15.73:0/4021807579 wait complete. 2024-08-31T07:05:15.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.141+0000 7f67610e2640 1 Processor -- start 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.141+0000 7f67610e2640 1 -- start start 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.141+0000 7f67610e2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f675c104620 0x7f675c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.141+0000 7f67610e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f675c105050 0x7f675c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.141+0000 7f67610e2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675c10dbd0 0x7f675c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.141+0000 7f67610e2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f675c113880 con 0x7f675c10dbd0 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.141+0000 7f67610e2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f675c113700 con 0x7f675c104620 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.141+0000 7f67610e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f675c113a00 con 0x7f675c105050 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f675ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f675c104620 0x7f675c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f675ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f675c104620 0x7f675c19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60390/0 (socket says 172.21.15.73:60390) 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f675ad9d640 1 -- 172.21.15.73:0/2582653422 learned_addr learned my addr 172.21.15.73:0/2582653422 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-08-31T07:05:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f675b59e640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675c10dbd0 0x7f675c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:05:15.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f675ad9d640 1 -- 172.21.15.73:0/2582653422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f675c105050 msgr2=0x7f675c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f675ad9d640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f675c105050 0x7f675c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f675ad9d640 1 -- 172.21.15.73:0/2582653422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675c10dbd0 msgr2=0x7f675c19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f675ad9d640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675c10dbd0 0x7f675c19faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f675ad9d640 1 -- 172.21.15.73:0/2582653422 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c1a0220 con 0x7f675c104620 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f675ad9d640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f675c104620 0x7f675c19a2b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f674c006010 tx=0x7f674c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f6753fff640 1 -- 172.21.15.73:0/2582653422 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c03f030 con 0x7f675c104620 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f67610e2640 1 -- 172.21.15.73:0/2582653422 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c1a04b0 con 0x7f675c104620 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f6753fff640 1 -- 172.21.15.73:0/2582653422 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f674c003800 con 0x7f675c104620 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f6753fff640 1 -- 172.21.15.73:0/2582653422 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c035070 con 0x7f675c104620 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.142+0000 7f67610e2640 1 -- 172.21.15.73:0/2582653422 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c102410 con 0x7f675c104620 2024-08-31T07:05:15.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.144+0000 7f67610e2640 1 -- 172.21.15.73:0/2582653422 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675c104a30 con 0x7f675c104620 2024-08-31T07:05:15.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.145+0000 7f6753fff640 1 -- 172.21.15.73:0/2582653422 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f674c0039b0 con 0x7f675c104620 2024-08-31T07:05:15.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.146+0000 7f6753fff640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67480b4c00 0x7f67480b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:05:15.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.146+0000 7f6753fff640 1 -- 172.21.15.73:0/2582653422 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3884+0+0 (secure 0 0 0) 0x7f674c104b30 con 0x7f675c104620 2024-08-31T07:05:15.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.146+0000 7f675a59c640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67480b4c00 0x7f67480b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:05:15.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.147+0000 7f675a59c640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67480b4c00 0x7f67480b7110 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f675c079d80 tx=0x7f6744006c90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:05:15.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.150+0000 7f6753fff640 1 -- 172.21.15.73:0/2582653422 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f674c0ccdb0 con 0x7f675c104620 2024-08-31T07:05:15.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:15.342+0000 7f67610e2640 1 -- 172.21.15.73:0/2582653422 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f675c0635a0 con 0x7f67480b4c00 2024-08-31T07:05:15.343 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:15 smithi073 ceph-mon[21995]: osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] boot 2024-08-31T07:05:15.343 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:15 smithi073 ceph-mon[21995]: osdmap e27: 4 total, 4 up, 4 in 2024-08-31T07:05:15.343 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:15.343 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:15 smithi073 ceph-mon[21995]: osdmap e28: 4 total, 4 up, 4 in 2024-08-31T07:05:15.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:15 smithi032 ceph-mon[17252]: osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] boot 2024-08-31T07:05:15.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:15 smithi032 ceph-mon[17252]: osdmap e27: 4 total, 4 up, 4 in 2024-08-31T07:05:15.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:15.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:15 smithi032 ceph-mon[17252]: osdmap e28: 4 total, 4 up, 4 in 2024-08-31T07:05:15.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:15 smithi084 ceph-mon[22387]: osd.3 [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] boot 2024-08-31T07:05:15.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:15 smithi084 ceph-mon[22387]: osdmap e27: 4 total, 4 up, 4 in 2024-08-31T07:05:15.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T07:05:15.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:15 smithi084 ceph-mon[22387]: osdmap e28: 4 total, 4 up, 4 in 2024-08-31T07:05:16.425 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:16.425 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:05:16.425 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:05:16.425 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:05:16.425 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:16.426 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.426 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.426 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:16.426 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:16.426 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.426 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:16.426 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:16.426 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.426 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:16 smithi073 ceph-mon[21995]: osdmap e29: 4 total, 4 up, 4 in 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:16.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:16.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:16 smithi084 ceph-mon[22387]: osdmap e29: 4 total, 4 up, 4 in 2024-08-31T07:05:16.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:16.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:05:16.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:05:16.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:05:16.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:16.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:16.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:16.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:16.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:16.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:16.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:16 smithi032 ceph-mon[17252]: osdmap e29: 4 total, 4 up, 4 in 2024-08-31T07:05:17.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:17 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi032 2024-08-31T07:05:17.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:17 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi032 to 8547M 2024-08-31T07:05:17.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:17 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi032 2024-08-31T07:05:17.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:17 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi032 to 8547M 2024-08-31T07:05:17.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:17 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi032 2024-08-31T07:05:17.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:17 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi032 to 8547M 2024-08-31T07:05:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:18 smithi073 ceph-mon[21995]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:18 smithi073 ceph-mon[21995]: from='client.? 172.21.15.73:0/1580314344' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e286ae6e-6d64-452c-b26a-d2aebbfa8742"} : dispatch 2024-08-31T07:05:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:18 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e286ae6e-6d64-452c-b26a-d2aebbfa8742"} : dispatch 2024-08-31T07:05:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:18 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e286ae6e-6d64-452c-b26a-d2aebbfa8742"}]': finished 2024-08-31T07:05:18.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:18 smithi073 ceph-mon[21995]: osdmap e30: 5 total, 4 up, 5 in 2024-08-31T07:05:18.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:18 smithi084 ceph-mon[22387]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:18 smithi084 ceph-mon[22387]: from='client.? 172.21.15.73:0/1580314344' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e286ae6e-6d64-452c-b26a-d2aebbfa8742"} : dispatch 2024-08-31T07:05:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:18 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e286ae6e-6d64-452c-b26a-d2aebbfa8742"} : dispatch 2024-08-31T07:05:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:18 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e286ae6e-6d64-452c-b26a-d2aebbfa8742"}]': finished 2024-08-31T07:05:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:18 smithi084 ceph-mon[22387]: osdmap e30: 5 total, 4 up, 5 in 2024-08-31T07:05:18.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:18.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:18 smithi032 ceph-mon[17252]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:18.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:18 smithi032 ceph-mon[17252]: from='client.? 172.21.15.73:0/1580314344' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e286ae6e-6d64-452c-b26a-d2aebbfa8742"} : dispatch 2024-08-31T07:05:18.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:18 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e286ae6e-6d64-452c-b26a-d2aebbfa8742"} : dispatch 2024-08-31T07:05:18.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:18 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e286ae6e-6d64-452c-b26a-d2aebbfa8742"}]': finished 2024-08-31T07:05:18.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:18 smithi032 ceph-mon[17252]: osdmap e30: 5 total, 4 up, 5 in 2024-08-31T07:05:18.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:19 smithi073 ceph-mon[21995]: from='client.? 172.21.15.73:0/1686023295' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:05:19.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:19 smithi084 ceph-mon[22387]: from='client.? 172.21.15.73:0/1686023295' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:05:19.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:19 smithi032 ceph-mon[17252]: from='client.? 172.21.15.73:0/1686023295' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:05:20.391 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:20 smithi084 ceph-mon[22387]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:20 smithi073 ceph-mon[21995]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:20.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:20 smithi032 ceph-mon[17252]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:22 smithi073 ceph-mon[21995]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T07:05:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:22 smithi084 ceph-mon[22387]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T07:05:22.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:22 smithi032 ceph-mon[17252]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T07:05:24.435 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:24 smithi073 ceph-mon[21995]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-31T07:05:24.494 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:24 smithi084 ceph-mon[22387]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-31T07:05:24.612 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:24 smithi032 ceph-mon[17252]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-31T07:05:26.251 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:26 smithi073 ceph-mon[21995]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-08-31T07:05:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:26 smithi084 ceph-mon[22387]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-08-31T07:05:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:26 smithi032 ceph-mon[17252]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-08-31T07:05:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:27 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-31T07:05:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:27 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:27 smithi073 ceph-mon[21995]: Deploying daemon osd.4 on smithi073 2024-08-31T07:05:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:27 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-31T07:05:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:27 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:27 smithi084 ceph-mon[22387]: Deploying daemon osd.4 on smithi073 2024-08-31T07:05:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:27 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-31T07:05:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:27 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:27 smithi032 ceph-mon[17252]: Deploying daemon osd.4 on smithi073 2024-08-31T07:05:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:28 smithi073 ceph-mon[21995]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:05:28.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:28 smithi084 ceph-mon[22387]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:05:28.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:28 smithi032 ceph-mon[17252]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:05:30.354 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:30 smithi073 ceph-mon[21995]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-31T07:05:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:30 smithi084 ceph-mon[22387]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-31T07:05:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:30 smithi032 ceph-mon[17252]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-31T07:05:32.267 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:32 smithi073 ceph-mon[21995]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T07:05:32.267 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:32 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:32.267 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:32 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:32.267 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:32 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:32 smithi084 ceph-mon[22387]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T07:05:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:32 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:32.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:32 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:32.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:32 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:32 smithi032 ceph-mon[17252]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T07:05:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:32 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:32 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:32 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:34 smithi073 ceph-mon[21995]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:34.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:34 smithi084 ceph-mon[22387]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:34.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:34 smithi032 ceph-mon[17252]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:34.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:34.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:34.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:34.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:34.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:35.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.374+0000 7f6753fff640 1 -- 172.21.15.73:0/2582653422 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f675c0635a0 con 0x7f67480b4c00 2024-08-31T07:05:35.376 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 4 on host 'smithi073' 2024-08-31T07:05:35.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 -- 172.21.15.73:0/2582653422 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67480b4c00 msgr2=0x7f67480b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:05:35.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67480b4c00 0x7f67480b7110 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f675c079d80 tx=0x7f6744006c90 comp rx=0 tx=0).stop 2024-08-31T07:05:35.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 -- 172.21.15.73:0/2582653422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f675c104620 msgr2=0x7f675c19a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:05:35.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f675c104620 0x7f675c19a2b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f674c006010 tx=0x7f674c0042f0 comp rx=0 tx=0).stop 2024-08-31T07:05:35.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 -- 172.21.15.73:0/2582653422 shutdown_connections 2024-08-31T07:05:35.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67480b4c00 0x7f67480b7110 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:35.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f675c10dbd0 0x7f675c19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:35.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f675c105050 0x7f675c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:35.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 --2- 172.21.15.73:0/2582653422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f675c104620 0x7f675c19a2b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:35.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 -- 172.21.15.73:0/2582653422 >> 172.21.15.73:0/2582653422 conn(0x7f675c0794e0 msgr2=0x7f675c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:05:35.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 -- 172.21.15.73:0/2582653422 shutdown_connections 2024-08-31T07:05:35.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:05:35.377+0000 7f67610e2640 1 -- 172.21.15.73:0/2582653422 wait complete. 2024-08-31T07:05:36.062 DEBUG:teuthology.orchestra.run.smithi073:osd.4> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.4.service 2024-08-31T07:05:36.064 INFO:tasks.cephadm:Deploying osd.5 on smithi084 with /dev/vg_nvme/lv_3... 2024-08-31T07:05:36.064 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-31T07:05:36.318 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:05:36.364 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:36 smithi073 ceph-mon[21995]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:36.364 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:36 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:36.364 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:36 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:36.364 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:36 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:36.439 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:36 smithi084 ceph-mon[22387]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:36.439 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:36 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:36.439 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:36 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:36.439 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:36 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:36 smithi032 ceph-mon[17252]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:36 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:36 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:36 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:38.180 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:05:37 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4[34058]: 2024-08-31T07:05:37.843+0000 7fcfb6dbc640 -1 osd.4 0 log_to_monitors true 2024-08-31T07:05:38.454 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:38 smithi084 ceph-mon[22387]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:38.454 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:38 smithi084 ceph-mon[22387]: from='osd.4 [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T07:05:38.454 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:38 smithi084 ceph-mon[22387]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T07:05:38.573 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:38 smithi073 ceph-mon[21995]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:38.573 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:38 smithi073 ceph-mon[21995]: from='osd.4 [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T07:05:38.573 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:38 smithi073 ceph-mon[21995]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T07:05:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:38 smithi032 ceph-mon[17252]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:38 smithi032 ceph-mon[17252]: from='osd.4 [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T07:05:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:38 smithi032 ceph-mon[17252]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T07:05:39.014 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T07:05:39.043 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_3 2024-08-31T07:05:39.291 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:05:39.412 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-31T07:05:39.412 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: osdmap e31: 5 total, 4 up, 5 in 2024-08-31T07:05:39.412 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:39.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='osd.4 [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:05:39.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:05:39.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:39.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:39.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:39.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:39.413 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-31T07:05:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: osdmap e31: 5 total, 4 up, 5 in 2024-08-31T07:05:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:39.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='osd.4 [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:05:39.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:05:39.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:39.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:39.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:39.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:39.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.620 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-31T07:05:39.620 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: osdmap e31: 5 total, 4 up, 5 in 2024-08-31T07:05:39.620 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:39.620 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='osd.4 [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:05:39.620 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:05:39.620 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:39.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:05:39.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:39.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:05:39.621 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:39.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.798+0000 7fcef2fe4640 1 -- 172.21.15.84:0/3798029817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceec108a90 msgr2=0x7fceec10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:05:39.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.798+0000 7fcef2fe4640 1 --2- 172.21.15.84:0/3798029817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceec108a90 0x7fceec10f2c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009ca0 tx=0x7fcee402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.798+0000 7fcef2fe4640 1 -- 172.21.15.84:0/3798029817 shutdown_connections 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.798+0000 7fcef2fe4640 1 --2- 172.21.15.84:0/3798029817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceec108a90 0x7fceec10f2c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.798+0000 7fcef2fe4640 1 --2- 172.21.15.84:0/3798029817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec108140 0x7fceec108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.798+0000 7fcef2fe4640 1 --2- 172.21.15.84:0/3798029817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceec102800 0x7fceec102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.798+0000 7fcef2fe4640 1 -- 172.21.15.84:0/3798029817 >> 172.21.15.84:0/3798029817 conn(0x7fceec0794d0 msgr2=0x7fceec0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.798+0000 7fcef2fe4640 1 -- 172.21.15.84:0/3798029817 shutdown_connections 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.798+0000 7fcef2fe4640 1 -- 172.21.15.84:0/3798029817 wait complete. 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef2fe4640 1 Processor -- start 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef2fe4640 1 -- start start 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef2fe4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec102800 0x7fceec19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:05:39.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef2fe4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceec108140 0x7fceec19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef2fe4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceec108a90 0x7fceec19fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef2fe4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fceec113880 con 0x7fceec102800 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef2fe4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fceec113700 con 0x7fceec108140 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef2fe4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fceec113a00 con 0x7fceec108a90 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef1581640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceec108a90 0x7fceec19fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef1581640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceec108a90 0x7fceec19fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59378/0 (socket says 172.21.15.84:59378) 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef1581640 1 -- 172.21.15.84:0/529458356 learned_addr learned my addr 172.21.15.84:0/529458356 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fceebfff640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceec108140 0x7fceec19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.799+0000 7fcef0d80640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec102800 0x7fceec19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcef1581640 1 -- 172.21.15.84:0/529458356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceec108140 msgr2=0x7fceec19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcef1581640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceec108140 0x7fceec19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:39.803 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcef1581640 1 -- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec102800 msgr2=0x7fceec19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:05:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcef1581640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec102800 0x7fceec19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:05:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcef1581640 1 -- 172.21.15.84:0/529458356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec1a01c0 con 0x7fceec108a90 2024-08-31T07:05:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcef0d80640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec102800 0x7fceec19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:05:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcef1581640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceec108a90 0x7fceec19fa90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcee4005cc0 tx=0x7fcee402f470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:05:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcee9ffb640 1 -- 172.21.15.84:0/529458356 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4003cf0 con 0x7fceec108a90 2024-08-31T07:05:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcef2fe4640 1 -- 172.21.15.84:0/529458356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceec1a0450 con 0x7fceec108a90 2024-08-31T07:05:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcee9ffb640 1 -- 172.21.15.84:0/529458356 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee40044f0 con 0x7fceec108a90 2024-08-31T07:05:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcee9ffb640 1 -- 172.21.15.84:0/529458356 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee402fc30 con 0x7fceec108a90 2024-08-31T07:05:39.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.800+0000 7fcef2fe4640 1 -- 172.21.15.84:0/529458356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceec1a6c10 con 0x7fceec108a90 2024-08-31T07:05:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.801+0000 7fcef2fe4640 1 -- 172.21.15.84:0/529458356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceec102c10 con 0x7fceec108a90 2024-08-31T07:05:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.803+0000 7fcee9ffb640 1 -- 172.21.15.84:0/529458356 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fcee40046a0 con 0x7fceec108a90 2024-08-31T07:05:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.804+0000 7fcee9ffb640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcedc0b4b80 0x7fcedc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:05:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.804+0000 7fcee9ffb640 1 -- 172.21.15.84:0/529458356 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4036+0+0 (secure 0 0 0) 0x7fcee4105840 con 0x7fceec108a90 2024-08-31T07:05:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.804+0000 7fcef0d80640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcedc0b4b80 0x7fcedc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:05:39.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.805+0000 7fcef0d80640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcedc0b4b80 0x7fcedc0b7090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcee0005fd0 tx=0x7fcee0005ea0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:05:39.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.808+0000 7fcee9ffb640 1 -- 172.21.15.84:0/529458356 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcee40cd980 con 0x7fceec108a90 2024-08-31T07:05:39.929 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4[34058]: 2024-08-31T07:05:39.619+0000 7fcfb2d3e640 -1 osd.4 0 waiting for initial osdmap 2024-08-31T07:05:39.929 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:05:39 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4[34058]: 2024-08-31T07:05:39.625+0000 7fcfaab34640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:05:39.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:05:39.988+0000 7fcef2fe4640 1 -- 172.21.15.84:0/529458356 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fceec0635a0 con 0x7fcedc0b4b80 2024-08-31T07:05:40.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:40.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi073 2024-08-31T07:05:40.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi073 to 8544M 2024-08-31T07:05:40.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:05:40.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: osdmap e32: 5 total, 4 up, 5 in 2024-08-31T07:05:40.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:40.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:40.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: from='osd.4 ' entity='osd.4' 2024-08-31T07:05:40.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:05:40.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:05:40.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:40 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:40.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:40.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi073 2024-08-31T07:05:40.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi073 to 8544M 2024-08-31T07:05:40.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:05:40.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: osdmap e32: 5 total, 4 up, 5 in 2024-08-31T07:05:40.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:40.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:40.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: from='osd.4 ' entity='osd.4' 2024-08-31T07:05:40.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:05:40.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:05:40.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:40 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi073 2024-08-31T07:05:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi073 to 8544M 2024-08-31T07:05:40.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:05:40.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: osdmap e32: 5 total, 4 up, 5 in 2024-08-31T07:05:40.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:40.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:40.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: from='osd.4 ' entity='osd.4' 2024-08-31T07:05:40.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:05:40.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:05:40.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:40 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:41.312 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:41 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:05:41.312 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:41 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:05:41.313 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:41 smithi084 ceph-mon[22387]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:05:41.313 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:41.313 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:41 smithi084 ceph-mon[22387]: osd.4 [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] boot 2024-08-31T07:05:41.313 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:41 smithi084 ceph-mon[22387]: osdmap e33: 5 total, 5 up, 5 in 2024-08-31T07:05:41.313 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:41.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:41 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:05:41.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:41 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:05:41.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:41 smithi032 ceph-mon[17252]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:05:41.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:41.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:41 smithi032 ceph-mon[17252]: osd.4 [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] boot 2024-08-31T07:05:41.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:41 smithi032 ceph-mon[17252]: osdmap e33: 5 total, 5 up, 5 in 2024-08-31T07:05:41.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:41 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:05:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:41 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:05:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:41 smithi073 ceph-mon[21995]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:05:41.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:41.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:41 smithi073 ceph-mon[21995]: osd.4 [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] boot 2024-08-31T07:05:41.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:41 smithi073 ceph-mon[21995]: osdmap e33: 5 total, 5 up, 5 in 2024-08-31T07:05:41.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T07:05:42.493 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:42 smithi084 ceph-mon[22387]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:42.493 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:42 smithi084 ceph-mon[22387]: osdmap e34: 5 total, 5 up, 5 in 2024-08-31T07:05:42.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:42 smithi032 ceph-mon[17252]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:42.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:42 smithi032 ceph-mon[17252]: osdmap e34: 5 total, 5 up, 5 in 2024-08-31T07:05:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:42 smithi073 ceph-mon[21995]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-31T07:05:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:42 smithi073 ceph-mon[21995]: osdmap e34: 5 total, 5 up, 5 in 2024-08-31T07:05:43.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:43 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/1543435157' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "98d5b70c-95dd-4bbb-b715-9872c4a4102e"} : dispatch 2024-08-31T07:05:43.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:43 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "98d5b70c-95dd-4bbb-b715-9872c4a4102e"} : dispatch 2024-08-31T07:05:43.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:43 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98d5b70c-95dd-4bbb-b715-9872c4a4102e"}]': finished 2024-08-31T07:05:43.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:43 smithi084 ceph-mon[22387]: osdmap e35: 6 total, 5 up, 6 in 2024-08-31T07:05:43.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:05:43.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:43 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/643374071' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:05:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:43 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/1543435157' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "98d5b70c-95dd-4bbb-b715-9872c4a4102e"} : dispatch 2024-08-31T07:05:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:43 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "98d5b70c-95dd-4bbb-b715-9872c4a4102e"} : dispatch 2024-08-31T07:05:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:43 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98d5b70c-95dd-4bbb-b715-9872c4a4102e"}]': finished 2024-08-31T07:05:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:43 smithi032 ceph-mon[17252]: osdmap e35: 6 total, 5 up, 6 in 2024-08-31T07:05:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:05:43.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:43 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/643374071' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:05:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:43 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/1543435157' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "98d5b70c-95dd-4bbb-b715-9872c4a4102e"} : dispatch 2024-08-31T07:05:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:43 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "98d5b70c-95dd-4bbb-b715-9872c4a4102e"} : dispatch 2024-08-31T07:05:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:43 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98d5b70c-95dd-4bbb-b715-9872c4a4102e"}]': finished 2024-08-31T07:05:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:43 smithi073 ceph-mon[21995]: osdmap e35: 6 total, 5 up, 6 in 2024-08-31T07:05:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:05:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:43 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/643374071' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:05:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:44 smithi073 ceph-mon[21995]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:44 smithi084 ceph-mon[22387]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:44.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:44 smithi032 ceph-mon[17252]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:46.659 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:46 smithi032 ceph-mon[17252]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:46 smithi073 ceph-mon[21995]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:46 smithi084 ceph-mon[22387]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:48 smithi073 ceph-mon[21995]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:48.713 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:48 smithi084 ceph-mon[22387]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:48.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:48 smithi032 ceph-mon[17252]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:50.604 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:50 smithi084 ceph-mon[22387]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:50 smithi073 ceph-mon[21995]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:50.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:50 smithi032 ceph-mon[17252]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:51.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:51 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-31T07:05:51.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:51 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:51.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:51 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-31T07:05:51.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:51 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:51.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:51 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-31T07:05:51.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:51 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:05:52.483 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:52 smithi084 ceph-mon[22387]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:52.483 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:52 smithi084 ceph-mon[22387]: Deploying daemon osd.5 on smithi084 2024-08-31T07:05:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:52 smithi073 ceph-mon[21995]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:52 smithi073 ceph-mon[21995]: Deploying daemon osd.5 on smithi084 2024-08-31T07:05:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:52 smithi032 ceph-mon[17252]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:52 smithi032 ceph-mon[17252]: Deploying daemon osd.5 on smithi084 2024-08-31T07:05:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:54 smithi073 ceph-mon[21995]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:54.694 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:54 smithi084 ceph-mon[22387]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:54 smithi032 ceph-mon[17252]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:56 smithi073 ceph-mon[21995]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:56.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:56 smithi084 ceph-mon[22387]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:56.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:56 smithi032 ceph-mon[17252]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:57.676 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:57.676 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:57.676 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:57.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:57.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:57.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:57.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:05:57.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:57.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:05:58.837 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:05:58 smithi084 ceph-mon[22387]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:58.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:05:58 smithi032 ceph-mon[17252]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:05:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:05:58 smithi073 ceph-mon[21995]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:00.319 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:00 smithi084 ceph-mon[22387]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:00.319 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:00.319 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:00.319 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:00.319 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:00.319 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:00.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:00 smithi032 ceph-mon[17252]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:00.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:00.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:00.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:00.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:00.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:00.646 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.645+0000 7fcee9ffb640 1 -- 172.21.15.84:0/529458356 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fceec0635a0 con 0x7fcedc0b4b80 2024-08-31T07:06:00.647 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 5 on host 'smithi084' 2024-08-31T07:06:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 -- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcedc0b4b80 msgr2=0x7fcedc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcedc0b4b80 0x7fcedc0b7090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcee0005fd0 tx=0x7fcee0005ea0 comp rx=0 tx=0).stop 2024-08-31T07:06:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 -- 172.21.15.84:0/529458356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceec108a90 msgr2=0x7fceec19fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceec108a90 0x7fceec19fa90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcee4005cc0 tx=0x7fcee402f470 comp rx=0 tx=0).stop 2024-08-31T07:06:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 -- 172.21.15.84:0/529458356 shutdown_connections 2024-08-31T07:06:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcedc0b4b80 0x7fcedc0b7090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceec108a90 0x7fceec19fa90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:00.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceec108140 0x7fceec19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:00.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 --2- 172.21.15.84:0/529458356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceec102800 0x7fceec19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:00.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 -- 172.21.15.84:0/529458356 >> 172.21.15.84:0/529458356 conn(0x7fceec0794d0 msgr2=0x7fceec105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:06:00.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 -- 172.21.15.84:0/529458356 shutdown_connections 2024-08-31T07:06:00.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:06:00.648+0000 7fcef2fe4640 1 -- 172.21.15.84:0/529458356 wait complete. 2024-08-31T07:06:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:00 smithi073 ceph-mon[21995]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:00.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:00.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:00.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:01.302 DEBUG:teuthology.orchestra.run.smithi084:osd.5> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.5.service 2024-08-31T07:06:01.304 INFO:tasks.cephadm:Deploying osd.6 on smithi032 with /dev/vg_nvme/lv_2... 2024-08-31T07:06:01.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-31T07:06:01.607 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:01.607 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:01.607 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:01.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:01.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:01.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:01.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:01.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:01.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:01.691 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:06:02.502 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:02 smithi084 ceph-mon[22387]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:02.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:02 smithi032 ceph-mon[17252]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:02 smithi073 ceph-mon[21995]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:03.241 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:06:02 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:06:02.907+0000 7ff9c5301640 -1 osd.5 0 log_to_monitors true 2024-08-31T07:06:03.436 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:03 smithi032 ceph-mon[17252]: from='osd.5 [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T07:06:03.437 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:03 smithi032 ceph-mon[17252]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T07:06:03.567 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:03 smithi084 ceph-mon[22387]: from='osd.5 [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T07:06:03.567 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:03 smithi084 ceph-mon[22387]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T07:06:03.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:03 smithi073 ceph-mon[21995]: from='osd.5 [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T07:06:03.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:03 smithi073 ceph-mon[21995]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T07:06:04.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:04 smithi032 ceph-mon[17252]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:04.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:04 smithi032 ceph-mon[17252]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-31T07:06:04.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:04 smithi032 ceph-mon[17252]: osdmap e36: 6 total, 5 up, 6 in 2024-08-31T07:06:04.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:04 smithi032 ceph-mon[17252]: from='osd.5 [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:06:04.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:04 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:04.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:04 smithi032 ceph-mon[17252]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:06:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:04 smithi073 ceph-mon[21995]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:04 smithi073 ceph-mon[21995]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-31T07:06:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:04 smithi073 ceph-mon[21995]: osdmap e36: 6 total, 5 up, 6 in 2024-08-31T07:06:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:04 smithi073 ceph-mon[21995]: from='osd.5 [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:06:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:04 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:04 smithi073 ceph-mon[21995]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:06:04.740 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:06:04 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:06:04.267+0000 7ff9c1283640 -1 osd.5 0 waiting for initial osdmap 2024-08-31T07:06:04.741 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:06:04 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:06:04.274+0000 7ff9bd081640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:06:04.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:04 smithi084 ceph-mon[22387]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:04.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:04 smithi084 ceph-mon[22387]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-31T07:06:04.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:04 smithi084 ceph-mon[22387]: osdmap e36: 6 total, 5 up, 6 in 2024-08-31T07:06:04.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:04 smithi084 ceph-mon[22387]: from='osd.5 [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:06:04.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:04 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:04.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:04 smithi084 ceph-mon[22387]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:06:05.211 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:06:05.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_2 2024-08-31T07:06:05.295 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:06:05.295 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: osdmap e37: 6 total, 5 up, 6 in 2024-08-31T07:06:05.295 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:05.295 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:05.295 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi084 2024-08-31T07:06:05.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:05.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:05.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi084 to 8544M 2024-08-31T07:06:05.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:05.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:05.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:06:05.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: osdmap e37: 6 total, 5 up, 6 in 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi084 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi084 to 8544M 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:05.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:05.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: osdmap e37: 6 total, 5 up, 6 in 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi084 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi084 to 8544M 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:05.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:05.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.320+0000 7f2e83200640 1 -- 172.21.15.32:0/4106515438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84109070 msgr2=0x7f2e84109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.320+0000 7f2e83200640 1 --2- 172.21.15.32:0/4106515438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84109070 0x7f2e84109480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009900 tx=0x7f2e6c02a410 comp rx=0 tx=0).stop 2024-08-31T07:06:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.320+0000 7f2e83200640 1 -- 172.21.15.32:0/4106515438 shutdown_connections 2024-08-31T07:06:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.320+0000 7f2e83200640 1 --2- 172.21.15.32:0/4106515438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e841099c0 0x7f2e841101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.320+0000 7f2e83200640 1 --2- 172.21.15.32:0/4106515438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84109070 0x7f2e84109480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.320+0000 7f2e83200640 1 --2- 172.21.15.32:0/4106515438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e84103730 0x7f2e84103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.320+0000 7f2e83200640 1 -- 172.21.15.32:0/4106515438 >> 172.21.15.32:0/4106515438 conn(0x7f2e840ff180 msgr2=0x7f2e841015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:06:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.320+0000 7f2e83200640 1 -- 172.21.15.32:0/4106515438 shutdown_connections 2024-08-31T07:06:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.320+0000 7f2e83200640 1 -- 172.21.15.32:0/4106515438 wait complete. 2024-08-31T07:06:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.321+0000 7f2e83200640 1 Processor -- start 2024-08-31T07:06:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.321+0000 7f2e83200640 1 -- start start 2024-08-31T07:06:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e83200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84103730 0x7f2e8419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:06:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e83200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e84109070 0x7f2e8419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:06:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e83200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e841099c0 0x7f2e8419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:06:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e83200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e84113650 con 0x7f2e84103730 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e83200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e841134d0 con 0x7f2e84109070 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e83200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2e841137d0 con 0x7f2e841099c0 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e81e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84103730 0x7f2e8419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e81e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84103730 0x7f2e8419a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41288/0 (socket says 172.21.15.32:41288) 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e81e00640 1 -- 172.21.15.32:0/681202114 learned_addr learned my addr 172.21.15.32:0/681202114 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e7be00640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e84109070 0x7f2e8419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e82800640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e841099c0 0x7f2e8419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.322+0000 7f2e81e00640 1 -- 172.21.15.32:0/681202114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e841099c0 msgr2=0x7f2e8419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e81e00640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e841099c0 0x7f2e8419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e81e00640 1 -- 172.21.15.32:0/681202114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e84109070 msgr2=0x7f2e8419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e81e00640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e84109070 0x7f2e8419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e81e00640 1 -- 172.21.15.32:0/681202114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e841a0010 con 0x7f2e84103730 2024-08-31T07:06:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e82800640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e841099c0 0x7f2e8419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:06:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e81e00640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84103730 0x7f2e8419a0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400e9f0 tx=0x7f2e7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:06:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e7aa00640 1 -- 172.21.15.32:0/681202114 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7400cd80 con 0x7f2e84103730 2024-08-31T07:06:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e83200640 1 -- 172.21.15.32:0/681202114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e841a0300 con 0x7f2e84103730 2024-08-31T07:06:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e7aa00640 1 -- 172.21.15.32:0/681202114 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e740108e0 con 0x7f2e84103730 2024-08-31T07:06:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e7aa00640 1 -- 172.21.15.32:0/681202114 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74018980 con 0x7f2e84103730 2024-08-31T07:06:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.323+0000 7f2e83200640 1 -- 172.21.15.32:0/681202114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e841a6a60 con 0x7f2e84103730 2024-08-31T07:06:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.324+0000 7f2e83200640 1 -- 172.21.15.32:0/681202114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e84103b40 con 0x7f2e84103730 2024-08-31T07:06:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.327+0000 7f2e7aa00640 1 -- 172.21.15.32:0/681202114 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f2e74010430 con 0x7f2e84103730 2024-08-31T07:06:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.329+0000 7f2e7aa00640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e680b4b80 0x7f2e680b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:06:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.330+0000 7f2e7aa00640 1 -- 172.21.15.32:0/681202114 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4748+0+0 (secure 0 0 0) 0x7f2e740e4cf0 con 0x7f2e84103730 2024-08-31T07:06:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.330+0000 7f2e7be00640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e680b4b80 0x7f2e680b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:06:06.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.330+0000 7f2e7be00640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e680b4b80 0x7f2e680b7090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009230 tx=0x7f2e6c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:06:06.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.332+0000 7f2e7aa00640 1 -- 172.21.15.32:0/681202114 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2e740acb70 con 0x7f2e84103730 2024-08-31T07:06:06.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:06.539+0000 7f2e83200640 1 -- 172.21.15.32:0/681202114 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2e840635a0 con 0x7f2e680b4b80 2024-08-31T07:06:06.541 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:06 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:06:06.541 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:06 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:06:06.541 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:06 smithi032 ceph-mon[17252]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:06.541 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:06 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:06.541 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:06 smithi032 ceph-mon[17252]: osd.5 [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] boot 2024-08-31T07:06:06.541 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:06 smithi032 ceph-mon[17252]: osdmap e38: 6 total, 6 up, 6 in 2024-08-31T07:06:06.541 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:06 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:06 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:06:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:06 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:06:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:06 smithi073 ceph-mon[21995]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:06 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:06 smithi073 ceph-mon[21995]: osd.5 [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] boot 2024-08-31T07:06:06.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:06 smithi073 ceph-mon[21995]: osdmap e38: 6 total, 6 up, 6 in 2024-08-31T07:06:06.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:06 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:06 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:06:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:06 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:06:06.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:06 smithi084 ceph-mon[22387]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-08-31T07:06:06.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:06 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:06.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:06 smithi084 ceph-mon[22387]: osd.5 [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] boot 2024-08-31T07:06:06.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:06 smithi084 ceph-mon[22387]: osdmap e38: 6 total, 6 up, 6 in 2024-08-31T07:06:06.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:06 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T07:06:07.452 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:07 smithi032 ceph-mon[17252]: osdmap e39: 6 total, 6 up, 6 in 2024-08-31T07:06:07.453 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:07 smithi032 ceph-mon[17252]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:06:07.453 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:07 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:06:07.453 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:07 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:06:07.453 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:07 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:07.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:07 smithi073 ceph-mon[21995]: osdmap e39: 6 total, 6 up, 6 in 2024-08-31T07:06:07.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:07 smithi073 ceph-mon[21995]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:06:07.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:07 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:06:07.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:07 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:06:07.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:07 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:07.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:07 smithi084 ceph-mon[22387]: osdmap e39: 6 total, 6 up, 6 in 2024-08-31T07:06:07.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:07 smithi084 ceph-mon[22387]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:06:07.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:07 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:06:07.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:07 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:06:07.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:07 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:08.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:08 smithi032 ceph-mon[17252]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:08.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:08 smithi032 ceph-mon[17252]: osdmap e40: 6 total, 6 up, 6 in 2024-08-31T07:06:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:08 smithi073 ceph-mon[21995]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:08 smithi073 ceph-mon[21995]: osdmap e40: 6 total, 6 up, 6 in 2024-08-31T07:06:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:08 smithi084 ceph-mon[22387]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:08 smithi084 ceph-mon[22387]: osdmap e40: 6 total, 6 up, 6 in 2024-08-31T07:06:10.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:10 smithi032 ceph-mon[17252]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:10.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:10 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/449680858' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5e79d62-80ff-4e1b-b031-2ce622f81c33"} : dispatch 2024-08-31T07:06:10.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:10 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/449680858' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5e79d62-80ff-4e1b-b031-2ce622f81c33"}]': finished 2024-08-31T07:06:10.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:10 smithi032 ceph-mon[17252]: osdmap e41: 7 total, 6 up, 7 in 2024-08-31T07:06:10.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:10 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:10.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:10 smithi073 ceph-mon[21995]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:10.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:10 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/449680858' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5e79d62-80ff-4e1b-b031-2ce622f81c33"} : dispatch 2024-08-31T07:06:10.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:10 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/449680858' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5e79d62-80ff-4e1b-b031-2ce622f81c33"}]': finished 2024-08-31T07:06:10.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:10 smithi073 ceph-mon[21995]: osdmap e41: 7 total, 6 up, 7 in 2024-08-31T07:06:10.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:10 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:10.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:10 smithi084 ceph-mon[22387]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:10.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:10 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/449680858' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5e79d62-80ff-4e1b-b031-2ce622f81c33"} : dispatch 2024-08-31T07:06:10.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:10 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/449680858' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5e79d62-80ff-4e1b-b031-2ce622f81c33"}]': finished 2024-08-31T07:06:10.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:10 smithi084 ceph-mon[22387]: osdmap e41: 7 total, 6 up, 7 in 2024-08-31T07:06:10.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:10 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:11.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:11 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/324370859' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:06:11.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:11 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/324370859' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:06:11.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:11 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/324370859' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:06:12.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:12 smithi032 ceph-mon[17252]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-08-31T07:06:12.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:12 smithi073 ceph-mon[21995]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-08-31T07:06:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:12 smithi084 ceph-mon[22387]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-08-31T07:06:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:14 smithi032 ceph-mon[17252]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-08-31T07:06:14.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:14 smithi073 ceph-mon[21995]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-08-31T07:06:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:14 smithi084 ceph-mon[22387]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-08-31T07:06:16.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:16 smithi032 ceph-mon[17252]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T07:06:16.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:16 smithi073 ceph-mon[21995]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T07:06:16.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:16 smithi084 ceph-mon[22387]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T07:06:18.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:18 smithi032 ceph-mon[17252]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-08-31T07:06:18.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:18 smithi073 ceph-mon[21995]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-08-31T07:06:18.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:18 smithi084 ceph-mon[22387]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-08-31T07:06:20.564 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:20 smithi084 ceph-mon[22387]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:06:20.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:20 smithi032 ceph-mon[17252]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:06:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:20 smithi073 ceph-mon[21995]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:06:21.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-31T07:06:21.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:21.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-31T07:06:21.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:21.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-31T07:06:21.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:22.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:22 smithi032 ceph-mon[17252]: Deploying daemon osd.6 on smithi032 2024-08-31T07:06:22.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:22 smithi032 ceph-mon[17252]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:22 smithi073 ceph-mon[21995]: Deploying daemon osd.6 on smithi032 2024-08-31T07:06:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:22 smithi073 ceph-mon[21995]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:22.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:22 smithi084 ceph-mon[22387]: Deploying daemon osd.6 on smithi032 2024-08-31T07:06:22.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:22 smithi084 ceph-mon[22387]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:24.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:24 smithi032 ceph-mon[17252]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:24 smithi073 ceph-mon[21995]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:24 smithi084 ceph-mon[22387]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:26 smithi032 ceph-mon[17252]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:26 smithi073 ceph-mon[21995]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:26 smithi084 ceph-mon[22387]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:28.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:28 smithi032 ceph-mon[17252]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:28 smithi073 ceph-mon[21995]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:28 smithi084 ceph-mon[22387]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:29.474 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:29 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:29.475 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:29 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:29.475 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:29 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:29.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:29 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:29.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:29 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:29.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:29 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:29.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:29 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:29.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:29 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:29.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:29 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:30 smithi032 ceph-mon[17252]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:30.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:30 smithi073 ceph-mon[21995]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:30.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:30 smithi084 ceph-mon[22387]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:32.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:32 smithi032 ceph-mon[17252]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:32.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:32 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:32.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:32 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:32 smithi073 ceph-mon[21995]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:32 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:32 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:32.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:32 smithi084 ceph-mon[22387]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:32.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:32 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:32.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:32 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:33.430 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 6 on host 'smithi032' 2024-08-31T07:06:33.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.429+0000 7f2e7aa00640 1 -- 172.21.15.32:0/681202114 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2e840635a0 con 0x7f2e680b4b80 2024-08-31T07:06:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 -- 172.21.15.32:0/681202114 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e680b4b80 msgr2=0x7f2e680b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e680b4b80 0x7f2e680b7090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009230 tx=0x7f2e6c031400 comp rx=0 tx=0).stop 2024-08-31T07:06:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 -- 172.21.15.32:0/681202114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84103730 msgr2=0x7f2e8419a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84103730 0x7f2e8419a0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400e9f0 tx=0x7f2e7400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:06:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 -- 172.21.15.32:0/681202114 shutdown_connections 2024-08-31T07:06:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e680b4b80 0x7f2e680b7090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e841099c0 0x7f2e8419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e84109070 0x7f2e8419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 --2- 172.21.15.32:0/681202114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84103730 0x7f2e8419a0a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 -- 172.21.15.32:0/681202114 >> 172.21.15.32:0/681202114 conn(0x7f2e840ff180 msgr2=0x7f2e841015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:06:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 -- 172.21.15.32:0/681202114 shutdown_connections 2024-08-31T07:06:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:06:33.432+0000 7f2e83200640 1 -- 172.21.15.32:0/681202114 wait complete. 2024-08-31T07:06:33.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:33 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:33.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:33 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:33.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:33 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:33.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:33 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:33.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:33 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:33.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:33 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:33.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:33 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:33.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:33 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:33.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:33 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:33.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:33 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:33.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:33 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:33.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:33 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:33.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:33 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:33.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:33 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:33.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:33 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:33.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:33 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:33.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:33 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:33.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:33 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:34.447 DEBUG:teuthology.orchestra.run.smithi032:osd.6> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.6.service 2024-08-31T07:06:34.452 INFO:tasks.cephadm:Deploying osd.7 on smithi073 with /dev/vg_nvme/lv_2... 2024-08-31T07:06:34.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-31T07:06:34.708 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:06:34.763 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:34 smithi032 ceph-mon[17252]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:34.763 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:06:34 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:06:34.563+0000 7f3ca8822640 -1 osd.6 0 log_to_monitors true 2024-08-31T07:06:34.828 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:34 smithi073 ceph-mon[21995]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:34 smithi084 ceph-mon[22387]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:35.778 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:35 smithi032 ceph-mon[17252]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-31T07:06:35.840 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:35 smithi073 ceph-mon[21995]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-31T07:06:35.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:35 smithi084 ceph-mon[22387]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-31T07:06:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:36 smithi032 ceph-mon[17252]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:36 smithi032 ceph-mon[17252]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-31T07:06:36.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:36 smithi032 ceph-mon[17252]: osdmap e42: 7 total, 6 up, 7 in 2024-08-31T07:06:36.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:36 smithi032 ceph-mon[17252]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:06:36.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:36 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:36 smithi073 ceph-mon[21995]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:36 smithi073 ceph-mon[21995]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-31T07:06:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:36 smithi073 ceph-mon[21995]: osdmap e42: 7 total, 6 up, 7 in 2024-08-31T07:06:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:36 smithi073 ceph-mon[21995]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:06:36.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:36 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:36 smithi084 ceph-mon[22387]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:36 smithi084 ceph-mon[22387]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-31T07:06:36.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:36 smithi084 ceph-mon[22387]: osdmap e42: 7 total, 6 up, 7 in 2024-08-31T07:06:36.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:36 smithi084 ceph-mon[22387]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:06:36.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:36 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:37.264 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T07:06:37.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_2 2024-08-31T07:06:37.403 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:06:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:06:37.114+0000 7f3ca3400640 -1 osd.6 0 waiting for initial osdmap 2024-08-31T07:06:37.403 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:06:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:06:37.147+0000 7f3c99400640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:06:37.541 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:06:37.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:37 smithi073 ceph-mon[21995]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:06:37.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:37 smithi073 ceph-mon[21995]: osdmap e43: 7 total, 6 up, 7 in 2024-08-31T07:06:37.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:37 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:37.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:37 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:37.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:37 smithi073 ceph-mon[21995]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' 2024-08-31T07:06:37.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:37 smithi032 ceph-mon[17252]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:06:37.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:37 smithi032 ceph-mon[17252]: osdmap e43: 7 total, 6 up, 7 in 2024-08-31T07:06:37.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:37 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:37.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:37 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:37.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:37 smithi032 ceph-mon[17252]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' 2024-08-31T07:06:37.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:37 smithi084 ceph-mon[22387]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:06:37.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:37 smithi084 ceph-mon[22387]: osdmap e43: 7 total, 6 up, 7 in 2024-08-31T07:06:37.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:37 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:37.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:37 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:37.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:37 smithi084 ceph-mon[22387]: from='osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887]' entity='osd.6' 2024-08-31T07:06:38.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.009+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1741747844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1ec102830 msgr2=0x7fd1ec102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:38.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.009+0000 7fd1f3f0e640 1 --2- 172.21.15.73:0/1741747844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1ec102830 0x7fd1ec102c40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009900 tx=0x7fd1e802a410 comp rx=0 tx=0).stop 2024-08-31T07:06:38.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.009+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1741747844 shutdown_connections 2024-08-31T07:06:38.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.009+0000 7fd1f3f0e640 1 --2- 172.21.15.73:0/1741747844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec108ac0 0x7fd1ec10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:38.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.009+0000 7fd1f3f0e640 1 --2- 172.21.15.73:0/1741747844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec108170 0x7fd1ec108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:38.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.009+0000 7fd1f3f0e640 1 --2- 172.21.15.73:0/1741747844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1ec102830 0x7fd1ec102c40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:38.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.009+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1741747844 >> 172.21.15.73:0/1741747844 conn(0x7fd1ec0794d0 msgr2=0x7fd1ec0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:06:38.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.009+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1741747844 shutdown_connections 2024-08-31T07:06:38.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.009+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1741747844 wait complete. 2024-08-31T07:06:38.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.009+0000 7fd1f3f0e640 1 Processor -- start 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f3f0e640 1 -- start start 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f3f0e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec102830 0x7fd1ec19b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f3f0e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1ec108170 0x7fd1ec19b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f3f0e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec108ac0 0x7fd1ec19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f3f0e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec112770 con 0x7fd1ec102830 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f3f0e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1125f0 con 0x7fd1ec108170 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f3f0e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1128f0 con 0x7fd1ec108ac0 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f14a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1ec108170 0x7fd1ec19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f14a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1ec108170 0x7fd1ec19b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52010/0 (socket says 172.21.15.73:52010) 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f14a9640 1 -- 172.21.15.73:0/1248350517 learned_addr learned my addr 172.21.15.73:0/1248350517 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-08-31T07:06:38.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f1caa640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec102830 0x7fd1ec19b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f24ab640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec108ac0 0x7fd1ec19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f14a9640 1 -- 172.21.15.73:0/1248350517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec108ac0 msgr2=0x7fd1ec19bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f14a9640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec108ac0 0x7fd1ec19bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f14a9640 1 -- 172.21.15.73:0/1248350517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec102830 msgr2=0x7fd1ec19b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f14a9640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec102830 0x7fd1ec19b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.010+0000 7fd1f14a9640 1 -- 172.21.15.73:0/1248350517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec1a6390 con 0x7fd1ec108170 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.011+0000 7fd1f14a9640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1ec108170 0x7fd1ec19b640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00e9f0 tx=0x7fd1dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.011+0000 7fd1e2ffd640 1 -- 172.21.15.73:0/1248350517 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00ccc0 con 0x7fd1ec108170 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.011+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1248350517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ec198f50 con 0x7fd1ec108170 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.011+0000 7fd1e2ffd640 1 -- 172.21.15.73:0/1248350517 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1dc0108e0 con 0x7fd1ec108170 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.011+0000 7fd1e2ffd640 1 -- 172.21.15.73:0/1248350517 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc018880 con 0x7fd1ec108170 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.011+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1248350517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ec1994e0 con 0x7fd1ec108170 2024-08-31T07:06:38.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.012+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1248350517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1ec19af10 con 0x7fd1ec108170 2024-08-31T07:06:38.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.014+0000 7fd1e2ffd640 1 -- 172.21.15.73:0/1248350517 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fd1dc010430 con 0x7fd1ec108170 2024-08-31T07:06:38.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.015+0000 7fd1e2ffd640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1d80b4c30 0x7fd1d80b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:06:38.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.015+0000 7fd1e2ffd640 1 -- 172.21.15.73:0/1248350517 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4900+0+0 (secure 0 0 0) 0x7fd1dc0e4bd0 con 0x7fd1ec108170 2024-08-31T07:06:38.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.015+0000 7fd1f1caa640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1d80b4c30 0x7fd1d80b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:06:38.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.016+0000 7fd1f1caa640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1d80b4c30 0x7fd1d80b7140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000c00 tx=0x7fd1e8031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:06:38.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.018+0000 7fd1e2ffd640 1 -- 172.21.15.73:0/1248350517 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd1dc0aca50 con 0x7fd1ec108170 2024-08-31T07:06:38.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:38.214+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1248350517 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd1ec0635a0 con 0x7fd1d80b4c30 2024-08-31T07:06:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:06:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:06:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] boot 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: osdmap e44: 7 total, 7 up, 7 in 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:38.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:38.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:38.610 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.876 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:06:38.876 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:06:38.876 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:38.876 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:38.876 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] boot 2024-08-31T07:06:38.876 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: osdmap e44: 7 total, 7 up, 7 in 2024-08-31T07:06:38.876 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:38.876 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:06:38.876 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:06:38.876 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:38.877 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.877 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.877 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:38.877 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:38.877 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:38.877 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.877 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:38.877 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:38.877 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:06:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:06:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: osd.6 [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] boot 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: osdmap e44: 7 total, 7 up, 7 in 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:38.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:38.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:39.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:39 smithi073 ceph-mon[21995]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:06:39.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:39 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi032 2024-08-31T07:06:39.625 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:39 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi032 to 5698M 2024-08-31T07:06:39.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:39 smithi032 ceph-mon[17252]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:06:39.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:39 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi032 2024-08-31T07:06:39.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:39 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi032 to 5698M 2024-08-31T07:06:39.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:39 smithi084 ceph-mon[22387]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:06:39.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:39 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi032 2024-08-31T07:06:39.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:39 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi032 to 5698M 2024-08-31T07:06:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:40 smithi073 ceph-mon[21995]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:40 smithi073 ceph-mon[21995]: osdmap e45: 7 total, 7 up, 7 in 2024-08-31T07:06:40.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:40 smithi032 ceph-mon[17252]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:40.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:40 smithi032 ceph-mon[17252]: osdmap e45: 7 total, 7 up, 7 in 2024-08-31T07:06:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:40 smithi084 ceph-mon[22387]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:40 smithi084 ceph-mon[22387]: osdmap e45: 7 total, 7 up, 7 in 2024-08-31T07:06:41.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:41 smithi032 ceph-mon[17252]: from='client.? 172.21.15.73:0/2534866257' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afca1d86-d56c-498f-8086-0114cd88e0c4"} : dispatch 2024-08-31T07:06:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:41 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afca1d86-d56c-498f-8086-0114cd88e0c4"} : dispatch 2024-08-31T07:06:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:41 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afca1d86-d56c-498f-8086-0114cd88e0c4"}]': finished 2024-08-31T07:06:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:41 smithi032 ceph-mon[17252]: osdmap e46: 8 total, 7 up, 8 in 2024-08-31T07:06:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:06:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:41 smithi032 ceph-mon[17252]: from='client.? 172.21.15.73:0/1095591174' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:06:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:41 smithi073 ceph-mon[21995]: from='client.? 172.21.15.73:0/2534866257' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afca1d86-d56c-498f-8086-0114cd88e0c4"} : dispatch 2024-08-31T07:06:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:41 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afca1d86-d56c-498f-8086-0114cd88e0c4"} : dispatch 2024-08-31T07:06:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:41 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afca1d86-d56c-498f-8086-0114cd88e0c4"}]': finished 2024-08-31T07:06:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:41 smithi073 ceph-mon[21995]: osdmap e46: 8 total, 7 up, 8 in 2024-08-31T07:06:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:06:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:41 smithi073 ceph-mon[21995]: from='client.? 172.21.15.73:0/1095591174' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:06:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:41 smithi084 ceph-mon[22387]: from='client.? 172.21.15.73:0/2534866257' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afca1d86-d56c-498f-8086-0114cd88e0c4"} : dispatch 2024-08-31T07:06:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:41 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afca1d86-d56c-498f-8086-0114cd88e0c4"} : dispatch 2024-08-31T07:06:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:41 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afca1d86-d56c-498f-8086-0114cd88e0c4"}]': finished 2024-08-31T07:06:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:41 smithi084 ceph-mon[22387]: osdmap e46: 8 total, 7 up, 8 in 2024-08-31T07:06:41.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:06:41.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:41 smithi084 ceph-mon[22387]: from='client.? 172.21.15.73:0/1095591174' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:06:42.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:42 smithi032 ceph-mon[17252]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:42 smithi073 ceph-mon[21995]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:42 smithi084 ceph-mon[22387]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:44.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:44 smithi032 ceph-mon[17252]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:44 smithi073 ceph-mon[21995]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:44 smithi084 ceph-mon[22387]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:46.711 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:46 smithi032 ceph-mon[17252]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:46.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:46 smithi073 ceph-mon[21995]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:46 smithi084 ceph-mon[22387]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:48.640 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:48 smithi073 ceph-mon[21995]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:48.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:48 smithi032 ceph-mon[17252]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:48 smithi084 ceph-mon[22387]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:49.832 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:49 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-31T07:06:49.832 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:49 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:49.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:49 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-31T07:06:49.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:49 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:49.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:49 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-31T07:06:49.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:49 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:50 smithi073 ceph-mon[21995]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:50 smithi073 ceph-mon[21995]: Deploying daemon osd.7 on smithi073 2024-08-31T07:06:50.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:50 smithi032 ceph-mon[17252]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:50.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:50 smithi032 ceph-mon[17252]: Deploying daemon osd.7 on smithi073 2024-08-31T07:06:50.891 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:50 smithi084 ceph-mon[22387]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:50.892 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:50 smithi084 ceph-mon[22387]: Deploying daemon osd.7 on smithi073 2024-08-31T07:06:52.674 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:52 smithi073 ceph-mon[21995]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:52 smithi032 ceph-mon[17252]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:52 smithi084 ceph-mon[22387]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:54 smithi032 ceph-mon[17252]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:54 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:54 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:54 smithi073 ceph-mon[21995]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:54 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:54 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:54 smithi084 ceph-mon[22387]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:54 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:54 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:55.607 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:55.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:56.660 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:56 smithi073 ceph-mon[21995]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:56.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:56 smithi032 ceph-mon[17252]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:56 smithi084 ceph-mon[22387]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:58.468 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 7 on host 'smithi073' 2024-08-31T07:06:58.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.467+0000 7fd1e2ffd640 1 -- 172.21.15.73:0/1248350517 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd1ec0635a0 con 0x7fd1d80b4c30 2024-08-31T07:06:58.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:58 smithi073 ceph-mon[21995]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:58.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:58.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:58.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:58.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:58.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:58.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1248350517 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1d80b4c30 msgr2=0x7fd1d80b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:58.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1d80b4c30 0x7fd1d80b7140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000c00 tx=0x7fd1e8031400 comp rx=0 tx=0).stop 2024-08-31T07:06:58.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1248350517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1ec108170 msgr2=0x7fd1ec19b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:06:58.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1ec108170 0x7fd1ec19b640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00e9f0 tx=0x7fd1dc00edf0 comp rx=0 tx=0).stop 2024-08-31T07:06:58.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1248350517 shutdown_connections 2024-08-31T07:06:58.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1d80b4c30 0x7fd1d80b7140 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:58.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec108ac0 0x7fd1ec19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:58.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1ec108170 0x7fd1ec19b640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:58.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 --2- 172.21.15.73:0/1248350517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec102830 0x7fd1ec19b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:06:58.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1248350517 >> 172.21.15.73:0/1248350517 conn(0x7fd1ec0794d0 msgr2=0x7fd1ec106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:06:58.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1248350517 shutdown_connections 2024-08-31T07:06:58.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:06:58.470+0000 7fd1f3f0e640 1 -- 172.21.15.73:0/1248350517 wait complete. 2024-08-31T07:06:58.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:58 smithi084 ceph-mon[22387]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:58.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:58.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:58.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:58.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:58.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:58 smithi032 ceph-mon[17252]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:06:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:06:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:06:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:59.146 DEBUG:teuthology.orchestra.run.smithi073:osd.7> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.7.service 2024-08-31T07:06:59.148 INFO:tasks.cephadm:Deploying osd.8 on smithi084 with /dev/vg_nvme/lv_2... 2024-08-31T07:06:59.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-31T07:06:59.402 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:06:59.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:59.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:59.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:06:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:59.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:59.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:59.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:06:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:59.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:06:59.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:06:59.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:06:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:00.573 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:00 smithi073 ceph-mon[21995]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:07:00.729 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:00 smithi084 ceph-mon[22387]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:07:00.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:00 smithi032 ceph-mon[17252]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:07:00.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:07:00 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:07:00.641+0000 7fb15137e640 -1 osd.7 0 log_to_monitors true 2024-08-31T07:07:01.661 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:01 smithi084 ceph-mon[22387]: from='osd.7 [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T07:07:01.661 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:01 smithi084 ceph-mon[22387]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T07:07:01.670 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:01 smithi073 ceph-mon[21995]: from='osd.7 [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T07:07:01.670 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:01 smithi073 ceph-mon[21995]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T07:07:01.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:01 smithi032 ceph-mon[17252]: from='osd.7 [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T07:07:01.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:01 smithi032 ceph-mon[17252]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T07:07:01.973 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T07:07:02.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_2 2024-08-31T07:07:02.251 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:07:02.657 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:07:02.657 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: osdmap e47: 8 total, 7 up, 8 in 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='osd.7 [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.658 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:02.659 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:02.659 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.748+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3580052256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f54108140 msgr2=0x7f5f54108550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:07:02.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.748+0000 7f5f59fa6640 1 --2- 172.21.15.84:0/3580052256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f54108140 0x7f5f54108550 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009900 tx=0x7f5f3c02a410 comp rx=0 tx=0).stop 2024-08-31T07:07:02.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.748+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3580052256 shutdown_connections 2024-08-31T07:07:02.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.748+0000 7f5f59fa6640 1 --2- 172.21.15.84:0/3580052256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f54108a90 0x7f5f5410f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:02.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.748+0000 7f5f59fa6640 1 --2- 172.21.15.84:0/3580052256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f54108140 0x7f5f54108550 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:02.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.748+0000 7f5f59fa6640 1 --2- 172.21.15.84:0/3580052256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f54102800 0x7f5f54102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:02.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.748+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3580052256 >> 172.21.15.84:0/3580052256 conn(0x7f5f540794d0 msgr2=0x7f5f540798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:07:02.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.748+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3580052256 shutdown_connections 2024-08-31T07:07:02.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.748+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3580052256 wait complete. 2024-08-31T07:07:02.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.749+0000 7f5f59fa6640 1 Processor -- start 2024-08-31T07:07:02.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.749+0000 7f5f59fa6640 1 -- start start 2024-08-31T07:07:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.749+0000 7f5f59fa6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f54102800 0x7f5f5406aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:07:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.749+0000 7f5f59fa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f54108140 0x7f5f5406b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:07:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.749+0000 7f5f59fa6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f54108a90 0x7f5f5419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:07:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.749+0000 7f5f59fa6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5f54112760 con 0x7f5f54108140 2024-08-31T07:07:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.749+0000 7f5f59fa6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5f541125e0 con 0x7f5f54102800 2024-08-31T07:07:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.749+0000 7f5f59fa6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5f541128e0 con 0x7f5f54108a90 2024-08-31T07:07:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f53fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f54108a90 0x7f5f5419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:07:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f53fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f54108a90 0x7f5f5419fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39630/0 (socket says 172.21.15.84:39630) 2024-08-31T07:07:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f53fff640 1 -- 172.21.15.84:0/3943432057 learned_addr learned my addr 172.21.15.84:0/3943432057 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-08-31T07:07:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f537fe640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f54102800 0x7f5f5406aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:07:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f52ffd640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f54108140 0x7f5f5406b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:07:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f53fff640 1 -- 172.21.15.84:0/3943432057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f54102800 msgr2=0x7f5f5406aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:07:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f53fff640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f54102800 0x7f5f5406aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f53fff640 1 -- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f54108140 msgr2=0x7f5f5406b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:07:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f53fff640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f54108140 0x7f5f5406b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f53fff640 1 -- 172.21.15.84:0/3943432057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f541a01e0 con 0x7f5f54108a90 2024-08-31T07:07:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f537fe640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f54102800 0x7f5f5406aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:07:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f52ffd640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f54108140 0x7f5f5406b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:07:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f53fff640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f54108a90 0x7f5f5419fab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00eaa0 tx=0x7f5f4c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:07:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f50ff9640 1 -- 172.21.15.84:0/3943432057 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00cdb0 con 0x7f5f54108a90 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f50ff9640 1 -- 172.21.15.84:0/3943432057 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f4c005020 con 0x7f5f54108a90 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3943432057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f541a04d0 con 0x7f5f54108a90 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f50ff9640 1 -- 172.21.15.84:0/3943432057 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c0116d0 con 0x7f5f54108a90 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.750+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3943432057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f541a6c30 con 0x7f5f54108a90 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.751+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3943432057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f54102c10 con 0x7f5f54108a90 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.753+0000 7f5f50ff9640 1 -- 172.21.15.84:0/3943432057 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f5f4c0051d0 con 0x7f5f54108a90 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.754+0000 7f5f50ff9640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5f400b4b80 0x7f5f400b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.754+0000 7f5f50ff9640 1 -- 172.21.15.84:0/3943432057 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5332+0+0 (secure 0 0 0) 0x7f5f4c0e4c40 con 0x7f5f54108a90 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.754+0000 7f5f537fe640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5f400b4b80 0x7f5f400b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.755+0000 7f5f537fe640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5f400b4b80 0x7f5f400b7090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009930 tx=0x7f5f44008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:07:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.756+0000 7f5f50ff9640 1 -- 172.21.15.84:0/3943432057 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5f4c016030 con 0x7f5f54108a90 2024-08-31T07:07:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:07:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-31T07:07:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: osdmap e47: 8 total, 7 up, 8 in 2024-08-31T07:07:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='osd.7 [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:07:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:07:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:02.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:02.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:02.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:02.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:02.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:07:02.481+0000 7fb14d300640 -1 osd.7 0 waiting for initial osdmap 2024-08-31T07:07:02.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:07:02.491+0000 7fb1490fe640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:07:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:07:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-31T07:07:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: osdmap e47: 8 total, 7 up, 8 in 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='osd.7 [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:02.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:02.942+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3943432057 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5f540635a0 con 0x7f5f400b4b80 2024-08-31T07:07:03.677 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:03 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi073 2024-08-31T07:07:03.678 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:03 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi073 to 5696M 2024-08-31T07:07:03.678 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:03 smithi084 ceph-mon[22387]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:07:03.678 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:03 smithi084 ceph-mon[22387]: osdmap e48: 8 total, 7 up, 8 in 2024-08-31T07:07:03.678 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:03 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:03.678 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:03 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:03.678 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:03 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:07:03.678 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:03 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:07:03.678 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:03 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:03.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:03 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi073 2024-08-31T07:07:03.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:03 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi073 to 5696M 2024-08-31T07:07:03.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:03 smithi032 ceph-mon[17252]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:07:03.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:03 smithi032 ceph-mon[17252]: osdmap e48: 8 total, 7 up, 8 in 2024-08-31T07:07:03.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:03 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:03.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:03 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:03.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:03 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:07:03.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:03 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:07:03.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:03 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:03.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:03 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi073 2024-08-31T07:07:03.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:03 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi073 to 5696M 2024-08-31T07:07:03.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:03 smithi073 ceph-mon[21995]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:07:03.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:03 smithi073 ceph-mon[21995]: osdmap e48: 8 total, 7 up, 8 in 2024-08-31T07:07:03.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:03 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:03.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:03 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:03.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:03 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:07:03.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:03 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:07:03.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:03 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:04.629 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:04 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:07:04.629 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:04 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:07:04.629 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:04 smithi084 ceph-mon[22387]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:07:04.629 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:04 smithi084 ceph-mon[22387]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:07:04.630 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:04 smithi084 ceph-mon[22387]: osd.7 [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] boot 2024-08-31T07:07:04.630 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:04 smithi084 ceph-mon[22387]: osdmap e49: 8 total, 8 up, 8 in 2024-08-31T07:07:04.630 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:04 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:04.630 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:04 smithi084 ceph-mon[22387]: osdmap e50: 8 total, 8 up, 8 in 2024-08-31T07:07:04.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:04 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:07:04.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:04 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:07:04.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:04 smithi032 ceph-mon[17252]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:07:04.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:04 smithi032 ceph-mon[17252]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:07:04.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:04 smithi032 ceph-mon[17252]: osd.7 [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] boot 2024-08-31T07:07:04.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:04 smithi032 ceph-mon[17252]: osdmap e49: 8 total, 8 up, 8 in 2024-08-31T07:07:04.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:04 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:04.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:04 smithi032 ceph-mon[17252]: osdmap e50: 8 total, 8 up, 8 in 2024-08-31T07:07:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:04 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:07:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:04 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:07:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:04 smithi073 ceph-mon[21995]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T07:07:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:04 smithi073 ceph-mon[21995]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:07:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:04 smithi073 ceph-mon[21995]: osd.7 [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] boot 2024-08-31T07:07:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:04 smithi073 ceph-mon[21995]: osdmap e49: 8 total, 8 up, 8 in 2024-08-31T07:07:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:04 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T07:07:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:04 smithi073 ceph-mon[21995]: osdmap e50: 8 total, 8 up, 8 in 2024-08-31T07:07:05.751 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:05 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/974730089' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7"} : dispatch 2024-08-31T07:07:05.751 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:05 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7"} : dispatch 2024-08-31T07:07:05.752 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:05 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7"}]': finished 2024-08-31T07:07:05.752 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:05 smithi084 ceph-mon[22387]: osdmap e51: 9 total, 8 up, 9 in 2024-08-31T07:07:05.752 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:05.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:05 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/974730089' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7"} : dispatch 2024-08-31T07:07:05.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:05 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7"} : dispatch 2024-08-31T07:07:05.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:05 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7"}]': finished 2024-08-31T07:07:05.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:05 smithi032 ceph-mon[17252]: osdmap e51: 9 total, 8 up, 9 in 2024-08-31T07:07:05.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:05.873 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:05 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/974730089' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7"} : dispatch 2024-08-31T07:07:05.873 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:05 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7"} : dispatch 2024-08-31T07:07:05.873 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:05 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7"}]': finished 2024-08-31T07:07:05.873 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:05 smithi073 ceph-mon[21995]: osdmap e51: 9 total, 8 up, 9 in 2024-08-31T07:07:05.873 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:06 smithi032 ceph-mon[17252]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:06 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/3994503894' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:07:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:06 smithi073 ceph-mon[21995]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:06 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/3994503894' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:07:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:06 smithi084 ceph-mon[22387]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:06 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/3994503894' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:07:08.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:08 smithi032 ceph-mon[17252]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:08 smithi073 ceph-mon[21995]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:08.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:08 smithi084 ceph-mon[22387]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:10.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:10 smithi032 ceph-mon[17252]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T07:07:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:10 smithi073 ceph-mon[21995]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T07:07:10.989 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:10 smithi084 ceph-mon[22387]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T07:07:12.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:12 smithi032 ceph-mon[17252]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-08-31T07:07:12.893 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:12 smithi084 ceph-mon[22387]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-08-31T07:07:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:12 smithi073 ceph-mon[21995]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-08-31T07:07:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:14 smithi084 ceph-mon[22387]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-08-31T07:07:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-31T07:07:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:14 smithi032 ceph-mon[17252]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-08-31T07:07:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-31T07:07:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:14 smithi073 ceph-mon[21995]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-08-31T07:07:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-31T07:07:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:15.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:15 smithi084 ceph-mon[22387]: Deploying daemon osd.8 on smithi084 2024-08-31T07:07:15.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:15 smithi032 ceph-mon[17252]: Deploying daemon osd.8 on smithi084 2024-08-31T07:07:15.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:15 smithi073 ceph-mon[21995]: Deploying daemon osd.8 on smithi084 2024-08-31T07:07:16.590 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:16 smithi084 ceph-mon[22387]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:07:16.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:16 smithi032 ceph-mon[17252]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:07:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:16 smithi073 ceph-mon[21995]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:07:18.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:18 smithi032 ceph-mon[17252]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-31T07:07:18.795 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:18 smithi084 ceph-mon[22387]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-31T07:07:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:18 smithi073 ceph-mon[21995]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-31T07:07:20.752 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:20 smithi084 ceph-mon[22387]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T07:07:20.753 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:20.753 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:20.753 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:20 smithi032 ceph-mon[17252]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T07:07:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:20 smithi073 ceph-mon[21995]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T07:07:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:22.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:22 smithi084 ceph-mon[22387]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:22.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:22 smithi032 ceph-mon[17252]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:22.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:22 smithi073 ceph-mon[21995]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:23.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.688+0000 7f5f50ff9640 1 -- 172.21.15.84:0/3943432057 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5f540635a0 con 0x7f5f400b4b80 2024-08-31T07:07:23.689 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 8 on host 'smithi084' 2024-08-31T07:07:23.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.690+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5f400b4b80 msgr2=0x7f5f400b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.690+0000 7f5f59fa6640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5f400b4b80 0x7f5f400b7090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009930 tx=0x7f5f44008040 comp rx=0 tx=0).stop 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.691+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3943432057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f54108a90 msgr2=0x7f5f5419fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.691+0000 7f5f59fa6640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f54108a90 0x7f5f5419fab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00eaa0 tx=0x7f5f4c00ef60 comp rx=0 tx=0).stop 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.691+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3943432057 shutdown_connections 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.691+0000 7f5f59fa6640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5f400b4b80 0x7f5f400b7090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.691+0000 7f5f59fa6640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f54108a90 0x7f5f5419fab0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.691+0000 7f5f59fa6640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f54108140 0x7f5f5406b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.691+0000 7f5f59fa6640 1 --2- 172.21.15.84:0/3943432057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f54102800 0x7f5f5406aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.691+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3943432057 >> 172.21.15.84:0/3943432057 conn(0x7f5f540794d0 msgr2=0x7f5f54105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.691+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3943432057 shutdown_connections 2024-08-31T07:07:23.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:07:23.691+0000 7f5f59fa6640 1 -- 172.21.15.84:0/3943432057 wait complete. 2024-08-31T07:07:24.299 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:24 smithi084 ceph-mon[22387]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:24.299 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.299 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.299 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:24.299 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:24.299 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.299 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:24.299 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.300 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.325 DEBUG:teuthology.orchestra.run.smithi084:osd.8> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.8.service 2024-08-31T07:07:24.327 INFO:tasks.cephadm:Deploying osd.9 on smithi032 with /dev/vg_nvme/lv_1... 2024-08-31T07:07:24.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-31T07:07:24.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:24 smithi032 ceph-mon[17252]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:24 smithi073 ceph-mon[21995]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:24.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:07:26.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:26 smithi032 ceph-mon[17252]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:26.118 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:26 smithi084 ceph-mon[22387]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:26 smithi073 ceph-mon[21995]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:26.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:07:26 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:07:26.116+0000 7fcb642a5640 -1 osd.8 0 log_to_monitors true 2024-08-31T07:07:27.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:27 smithi084 ceph-mon[22387]: from='osd.8 [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T07:07:27.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:27 smithi084 ceph-mon[22387]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T07:07:27.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:27 smithi032 ceph-mon[17252]: from='osd.8 [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T07:07:27.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:27 smithi032 ceph-mon[17252]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T07:07:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:27 smithi073 ceph-mon[21995]: from='osd.8 [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T07:07:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:27 smithi073 ceph-mon[21995]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T07:07:28.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:28.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-31T07:07:28.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: osdmap e52: 9 total, 8 up, 9 in 2024-08-31T07:07:28.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:28.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='osd.8 [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:07:28.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi084 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi084 to 5696M 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:28.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-31T07:07:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: osdmap e52: 9 total, 8 up, 9 in 2024-08-31T07:07:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='osd.8 [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:07:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi084 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi084 to 5696M 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:28.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:07:28.032+0000 7fcb60227640 -1 osd.8 0 waiting for initial osdmap 2024-08-31T07:07:28.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:07:28.048+0000 7fcb5c025640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: osdmap e52: 9 total, 8 up, 9 in 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='osd.8 [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi084 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:28.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:07:28.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi084 to 5696M 2024-08-31T07:07:28.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:28.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:28.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:28.779 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:07:28.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_1 2024-08-31T07:07:29.227 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:07:29.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:29 smithi032 ceph-mon[17252]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:07:29.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:29 smithi032 ceph-mon[17252]: osdmap e53: 9 total, 8 up, 9 in 2024-08-31T07:07:29.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:29 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:29.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:29 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:29.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:29 smithi073 ceph-mon[21995]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:07:29.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:29 smithi073 ceph-mon[21995]: osdmap e53: 9 total, 8 up, 9 in 2024-08-31T07:07:29.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:29 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:29.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:29 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:29.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:29 smithi084 ceph-mon[22387]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:07:29.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:29 smithi084 ceph-mon[22387]: osdmap e53: 9 total, 8 up, 9 in 2024-08-31T07:07:29.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:29 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:29.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:29 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:29.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.872+0000 7fa1e3000640 1 -- 172.21.15.32:0/1589252543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1dc102230 msgr2=0x7fa1dc10f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:07:29.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.872+0000 7fa1e3000640 1 --2- 172.21.15.32:0/1589252543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1dc102230 0x7fa1dc10f710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009900 tx=0x7fa1c802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:07:29.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.872+0000 7fa1e3000640 1 -- 172.21.15.32:0/1589252543 shutdown_connections 2024-08-31T07:07:29.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.872+0000 7fa1e3000640 1 --2- 172.21.15.32:0/1589252543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1dc10fdb0 0x7fa1dc1122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:29.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.872+0000 7fa1e3000640 1 --2- 172.21.15.32:0/1589252543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1dc102230 0x7fa1dc10f710 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:29.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.872+0000 7fa1e3000640 1 --2- 172.21.15.32:0/1589252543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1dc1018e0 0x7fa1dc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.872+0000 7fa1e3000640 1 -- 172.21.15.32:0/1589252543 >> 172.21.15.32:0/1589252543 conn(0x7fa1dc0775a0 msgr2=0x7fa1dc0779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.872+0000 7fa1e3000640 1 -- 172.21.15.32:0/1589252543 shutdown_connections 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.872+0000 7fa1e3000640 1 -- 172.21.15.32:0/1589252543 wait complete. 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.873+0000 7fa1e3000640 1 Processor -- start 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1e3000640 1 -- start start 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1e3000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1dc1018e0 0x7fa1dc19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1e3000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1dc102230 0x7fa1dc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1e3000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1dc10fdb0 0x7fa1dc1a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1e3000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc115a10 con 0x7fa1dc102230 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1e3000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc115890 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1e3000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc115b90 con 0x7fa1dc1018e0 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1dc102230 0x7fa1dc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1dc102230 0x7fa1dc19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44312/0 (socket says 172.21.15.32:44312) 2024-08-31T07:07:29.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1db400640 1 -- 172.21.15.32:0/4093719073 learned_addr learned my addr 172.21.15.32:0/4093719073 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1e1000640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1dc10fdb0 0x7fa1dc1a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.874+0000 7fa1dbe00640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1dc1018e0 0x7fa1dc19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.875+0000 7fa1e1000640 1 -- 172.21.15.32:0/4093719073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1dc1018e0 msgr2=0x7fa1dc19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.875+0000 7fa1e1000640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1dc1018e0 0x7fa1dc19c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.875+0000 7fa1e1000640 1 -- 172.21.15.32:0/4093719073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1dc102230 msgr2=0x7fa1dc19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.875+0000 7fa1e1000640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1dc102230 0x7fa1dc19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.875+0000 7fa1e1000640 1 -- 172.21.15.32:0/4093719073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1dc1a2350 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.875+0000 7fa1db400640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1dc102230 0x7fa1dc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.875+0000 7fa1e1000640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1dc10fdb0 0x7fa1dc1a1c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4009fd0 tx=0x7fa1d400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.875+0000 7fa1d8c00640 1 -- 172.21.15.32:0/4093719073 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d400ce20 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.875+0000 7fa1e3000640 1 -- 172.21.15.32:0/4093719073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1dc1a2640 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.876+0000 7fa1d8c00640 1 -- 172.21.15.32:0/4093719073 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1d4005020 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.876+0000 7fa1e3000640 1 -- 172.21.15.32:0/4093719073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1dc1a8da0 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.876+0000 7fa1d8c00640 1 -- 172.21.15.32:0/4093719073 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4010690 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.877+0000 7fa1e3000640 1 -- 172.21.15.32:0/4093719073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1dc101cf0 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.880+0000 7fa1d8c00640 1 -- 172.21.15.32:0/4093719073 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fa1d40054d0 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.882+0000 7fa1d8c00640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1c00b4b80 0x7fa1c00b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:07:29.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.882+0000 7fa1d8c00640 1 -- 172.21.15.32:0/4093719073 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6077+0+0 (secure 0 0 0) 0x7fa1d40e4ff0 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.882+0000 7fa1dbe00640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1c00b4b80 0x7fa1c00b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:07:29.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.882+0000 7fa1d8c00640 1 -- 172.21.15.32:0/4093719073 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa1d40ac940 con 0x7fa1dc10fdb0 2024-08-31T07:07:29.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:29.883+0000 7fa1dbe00640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1c00b4b80 0x7fa1c00b7090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009bd0 tx=0x7fa1cc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:07:30.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:30.090+0000 7fa1e3000640 1 -- 172.21.15.32:0/4093719073 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa1dc0635a0 con 0x7fa1c00b4b80 2024-08-31T07:07:30.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:30 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:07:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:30 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:07:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:30 smithi032 ceph-mon[17252]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:30 smithi032 ceph-mon[17252]: osd.8 [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] boot 2024-08-31T07:07:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:30 smithi032 ceph-mon[17252]: osdmap e54: 9 total, 9 up, 9 in 2024-08-31T07:07:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:30 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:07:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:30 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:07:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:30 smithi073 ceph-mon[21995]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:30 smithi073 ceph-mon[21995]: osd.8 [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] boot 2024-08-31T07:07:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:30 smithi073 ceph-mon[21995]: osdmap e54: 9 total, 9 up, 9 in 2024-08-31T07:07:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:30.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:30 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:07:30.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:30 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:07:30.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:30 smithi084 ceph-mon[22387]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-08-31T07:07:30.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:30.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:30 smithi084 ceph-mon[22387]: osd.8 [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] boot 2024-08-31T07:07:30.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:30 smithi084 ceph-mon[22387]: osdmap e54: 9 total, 9 up, 9 in 2024-08-31T07:07:30.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T07:07:31.354 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:31 smithi084 ceph-mon[22387]: osdmap e55: 9 total, 9 up, 9 in 2024-08-31T07:07:31.355 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:31 smithi084 ceph-mon[22387]: from='client.24527 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:07:31.355 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:31 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:07:31.355 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:31 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:07:31.355 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:31 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:31.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:31 smithi032 ceph-mon[17252]: osdmap e55: 9 total, 9 up, 9 in 2024-08-31T07:07:31.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:31 smithi032 ceph-mon[17252]: from='client.24527 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:07:31.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:31 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:07:31.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:31 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:07:31.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:31 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:31.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:31 smithi073 ceph-mon[21995]: osdmap e55: 9 total, 9 up, 9 in 2024-08-31T07:07:31.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:31 smithi073 ceph-mon[21995]: from='client.24527 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:07:31.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:31 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:07:31.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:31 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:07:31.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:31 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:32.099 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:32 smithi032 ceph-mon[17252]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:32.099 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:32 smithi032 ceph-mon[17252]: osdmap e56: 9 total, 9 up, 9 in 2024-08-31T07:07:32.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:32 smithi084 ceph-mon[22387]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:32.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:32 smithi084 ceph-mon[22387]: osdmap e56: 9 total, 9 up, 9 in 2024-08-31T07:07:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:32 smithi073 ceph-mon[21995]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:32 smithi073 ceph-mon[21995]: osdmap e56: 9 total, 9 up, 9 in 2024-08-31T07:07:34.312 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:34 smithi032 ceph-mon[17252]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:34.312 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:34 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/448909349' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1d428503-9203-4ddd-b618-114e17fe9f3a"} : dispatch 2024-08-31T07:07:34.312 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:34 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/448909349' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d428503-9203-4ddd-b618-114e17fe9f3a"}]': finished 2024-08-31T07:07:34.312 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:34 smithi032 ceph-mon[17252]: osdmap e57: 10 total, 9 up, 10 in 2024-08-31T07:07:34.312 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:34 smithi073 ceph-mon[21995]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:34 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/448909349' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1d428503-9203-4ddd-b618-114e17fe9f3a"} : dispatch 2024-08-31T07:07:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:34 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/448909349' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d428503-9203-4ddd-b618-114e17fe9f3a"}]': finished 2024-08-31T07:07:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:34 smithi073 ceph-mon[21995]: osdmap e57: 10 total, 9 up, 10 in 2024-08-31T07:07:34.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:34 smithi084 ceph-mon[22387]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:34 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/448909349' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1d428503-9203-4ddd-b618-114e17fe9f3a"} : dispatch 2024-08-31T07:07:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:34 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/448909349' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d428503-9203-4ddd-b618-114e17fe9f3a"}]': finished 2024-08-31T07:07:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:34 smithi084 ceph-mon[22387]: osdmap e57: 10 total, 9 up, 10 in 2024-08-31T07:07:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:35.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:35 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/744894362' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:07:35.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:35 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/744894362' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:07:35.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:35 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/744894362' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:07:36.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:36 smithi032 ceph-mon[17252]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-08-31T07:07:36.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:36 smithi073 ceph-mon[21995]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-08-31T07:07:36.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:36 smithi084 ceph-mon[22387]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-08-31T07:07:38.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:38 smithi032 ceph-mon[17252]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-08-31T07:07:38.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:38 smithi073 ceph-mon[21995]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-08-31T07:07:38.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:38 smithi084 ceph-mon[22387]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-08-31T07:07:40.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:40 smithi032 ceph-mon[17252]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T07:07:40.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:40 smithi073 ceph-mon[21995]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T07:07:40.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:40 smithi084 ceph-mon[22387]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T07:07:42.118 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:42 smithi032 ceph-mon[17252]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-31T07:07:42.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:42 smithi073 ceph-mon[21995]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-31T07:07:42.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:42 smithi084 ceph-mon[22387]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-31T07:07:44.336 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:44 smithi032 ceph-mon[17252]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:07:44.336 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-31T07:07:44.336 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:44.336 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:44 smithi032 ceph-mon[17252]: Deploying daemon osd.9 on smithi032 2024-08-31T07:07:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:44 smithi073 ceph-mon[21995]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:07:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-31T07:07:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:44 smithi073 ceph-mon[21995]: Deploying daemon osd.9 on smithi032 2024-08-31T07:07:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:44 smithi084 ceph-mon[22387]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T07:07:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-31T07:07:44.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:44.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:44 smithi084 ceph-mon[22387]: Deploying daemon osd.9 on smithi032 2024-08-31T07:07:46.346 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:46 smithi032 ceph-mon[17252]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-31T07:07:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:46 smithi073 ceph-mon[21995]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-31T07:07:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:46 smithi084 ceph-mon[22387]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-31T07:07:48.198 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:48 smithi032 ceph-mon[17252]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:48.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:48 smithi073 ceph-mon[21995]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:48.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:48 smithi084 ceph-mon[22387]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:50.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:50 smithi032 ceph-mon[17252]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:50.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:50 smithi073 ceph-mon[21995]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:50.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:50 smithi084 ceph-mon[22387]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:52.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:52 smithi032 ceph-mon[17252]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:52.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:52 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:52.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:52 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:52.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:52 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:52.410 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:52 smithi084 ceph-mon[22387]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:52.410 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:52 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:52.411 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:52 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:52.411 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:52 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:52 smithi073 ceph-mon[21995]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:52 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:52 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:52 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:54.153 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:54 smithi032 ceph-mon[17252]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:54.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:54 smithi073 ceph-mon[21995]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:54.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:54 smithi084 ceph-mon[22387]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:55 smithi032 ceph-mon[17252]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:55.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:55 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:55.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:55.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:55.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:55 smithi073 ceph-mon[21995]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:55.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:55.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:55.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:55 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:55.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:55.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:55.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:55 smithi084 ceph-mon[22387]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:55.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:07:55.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:07:55.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:55 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.544+0000 7fa1d8c00640 1 -- 172.21.15.32:0/4093719073 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa1dc0635a0 con 0x7fa1c00b4b80 2024-08-31T07:07:56.545 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 9 on host 'smithi032' 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.546+0000 7fa1e3000640 1 -- 172.21.15.32:0/4093719073 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1c00b4b80 msgr2=0x7fa1c00b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.546+0000 7fa1e3000640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1c00b4b80 0x7fa1c00b7090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009bd0 tx=0x7fa1cc005f90 comp rx=0 tx=0).stop 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.546+0000 7fa1e3000640 1 -- 172.21.15.32:0/4093719073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1dc10fdb0 msgr2=0x7fa1dc1a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.546+0000 7fa1e3000640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1dc10fdb0 0x7fa1dc1a1c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4009fd0 tx=0x7fa1d400ef10 comp rx=0 tx=0).stop 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.547+0000 7fa1e3000640 1 -- 172.21.15.32:0/4093719073 shutdown_connections 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.547+0000 7fa1e3000640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1c00b4b80 0x7fa1c00b7090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.547+0000 7fa1e3000640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1dc10fdb0 0x7fa1dc1a1c20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.547+0000 7fa1e3000640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1dc102230 0x7fa1dc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.547+0000 7fa1e3000640 1 --2- 172.21.15.32:0/4093719073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1dc1018e0 0x7fa1dc19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.547+0000 7fa1e3000640 1 -- 172.21.15.32:0/4093719073 >> 172.21.15.32:0/4093719073 conn(0x7fa1dc0775a0 msgr2=0x7fa1dc110430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.547+0000 7fa1e3000640 1 -- 172.21.15.32:0/4093719073 shutdown_connections 2024-08-31T07:07:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:07:56.547+0000 7fa1e3000640 1 -- 172.21.15.32:0/4093719073 wait complete. 2024-08-31T07:07:56.893 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:56 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:56.893 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:56 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:56.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:56 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:56 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:56 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:56 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:56 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:07:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:56 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:56 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:07:57.429 DEBUG:teuthology.orchestra.run.smithi032:osd.9> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.9.service 2024-08-31T07:07:57.433 INFO:tasks.cephadm:Deploying osd.10 on smithi073 with /dev/vg_nvme/lv_1... 2024-08-31T07:07:57.433 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-31T07:07:57.660 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:57 smithi032 ceph-mon[17252]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:57.661 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:57 smithi032 ceph-mon[17252]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-31T07:07:57.687 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:07:57.808 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:57 smithi073 ceph-mon[21995]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:57.808 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:57 smithi073 ceph-mon[21995]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-31T07:07:57.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:57 smithi084 ceph-mon[22387]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:57.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:57 smithi084 ceph-mon[22387]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-31T07:07:58.928 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:58 smithi073 ceph-mon[21995]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-31T07:07:58.928 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:58 smithi073 ceph-mon[21995]: osdmap e58: 10 total, 9 up, 10 in 2024-08-31T07:07:58.928 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:58 smithi073 ceph-mon[21995]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:07:58.928 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:58 smithi084 ceph-mon[22387]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-31T07:07:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:58 smithi084 ceph-mon[22387]: osdmap e58: 10 total, 9 up, 10 in 2024-08-31T07:07:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:58 smithi084 ceph-mon[22387]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:07:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:59.091 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:58 smithi032 ceph-mon[17252]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-31T07:07:59.091 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:58 smithi032 ceph-mon[17252]: osdmap e58: 10 total, 9 up, 10 in 2024-08-31T07:07:59.091 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:58 smithi032 ceph-mon[17252]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]} : dispatch 2024-08-31T07:07:59.091 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:59.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:07:59 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:07:59.171+0000 7fd2ee000640 -1 osd.9 0 waiting for initial osdmap 2024-08-31T07:07:59.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:07:59 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:07:59.204+0000 7fd2e3600640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:07:59.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:59 smithi032 ceph-mon[17252]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:07:59.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:59 smithi032 ceph-mon[17252]: osdmap e59: 10 total, 9 up, 10 in 2024-08-31T07:07:59.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:59.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:59.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:59 smithi032 ceph-mon[17252]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:59.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:07:59 smithi032 ceph-mon[17252]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' 2024-08-31T07:07:59.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:59 smithi073 ceph-mon[21995]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:07:59.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:59 smithi073 ceph-mon[21995]: osdmap e59: 10 total, 9 up, 10 in 2024-08-31T07:07:59.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:59.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:59.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:59 smithi073 ceph-mon[21995]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:59.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:07:59 smithi073 ceph-mon[21995]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' 2024-08-31T07:07:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:59 smithi084 ceph-mon[22387]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-08-31T07:07:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:59 smithi084 ceph-mon[22387]: osdmap e59: 10 total, 9 up, 10 in 2024-08-31T07:07:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:07:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:59 smithi084 ceph-mon[22387]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-31T07:07:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:07:59 smithi084 ceph-mon[22387]: from='osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479]' entity='osd.9' 2024-08-31T07:08:00.361 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-08-31T07:08:00.386 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_1 2024-08-31T07:08:00.630 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:08:00.916 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:00 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:08:00.916 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:00 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:08:00.917 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:08:00.917 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:00 smithi073 ceph-mon[21995]: osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] boot 2024-08-31T07:08:00.917 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:00 smithi073 ceph-mon[21995]: osdmap e60: 10 total, 10 up, 10 in 2024-08-31T07:08:00.917 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:08:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:00 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:08:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:00 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:08:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:08:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:00 smithi084 ceph-mon[22387]: osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] boot 2024-08-31T07:08:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:00 smithi084 ceph-mon[22387]: osdmap e60: 10 total, 10 up, 10 in 2024-08-31T07:08:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:08:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:00 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:08:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:00 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:08:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:08:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:00 smithi032 ceph-mon[17252]: osd.9 [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] boot 2024-08-31T07:08:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:00 smithi032 ceph-mon[17252]: osdmap e60: 10 total, 10 up, 10 in 2024-08-31T07:08:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T07:08:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.123+0000 7f5a98146640 1 -- 172.21.15.73:0/2670368193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a901081f0 msgr2=0x7f5a90108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.123+0000 7f5a98146640 1 --2- 172.21.15.73:0/2670368193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a901081f0 0x7f5a90108600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009900 tx=0x7f5a8002a410 comp rx=0 tx=0).stop 2024-08-31T07:08:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.123+0000 7f5a98146640 1 -- 172.21.15.73:0/2670368193 shutdown_connections 2024-08-31T07:08:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.123+0000 7f5a98146640 1 --2- 172.21.15.73:0/2670368193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a90108b40 0x7f5a9010f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.123+0000 7f5a98146640 1 --2- 172.21.15.73:0/2670368193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a901081f0 0x7f5a90108600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.123+0000 7f5a98146640 1 --2- 172.21.15.73:0/2670368193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a901028b0 0x7f5a90102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.123+0000 7f5a98146640 1 -- 172.21.15.73:0/2670368193 >> 172.21.15.73:0/2670368193 conn(0x7f5a90079470 msgr2=0x7f5a90079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.123+0000 7f5a98146640 1 -- 172.21.15.73:0/2670368193 shutdown_connections 2024-08-31T07:08:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.123+0000 7f5a98146640 1 -- 172.21.15.73:0/2670368193 wait complete. 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.123+0000 7f5a98146640 1 Processor -- start 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a98146640 1 -- start start 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a98146640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a901028b0 0x7f5a9019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a98146640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a901081f0 0x7f5a9019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a98146640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a90108b40 0x7f5a9019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a98146640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5a901139b0 con 0x7f5a901028b0 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a98146640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5a90113830 con 0x7f5a90108b40 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a98146640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5a90113b30 con 0x7f5a901081f0 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a966e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a90108b40 0x7f5a9019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a966e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a90108b40 0x7f5a9019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37452/0 (socket says 172.21.15.73:37452) 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a966e3640 1 -- 172.21.15.73:0/1110364675 learned_addr learned my addr 172.21.15.73:0/1110364675 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a95ee2640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a901028b0 0x7f5a9019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:01.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a956e1640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a901081f0 0x7f5a9019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a966e3640 1 -- 172.21.15.73:0/1110364675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a901081f0 msgr2=0x7f5a9019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a966e3640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a901081f0 0x7f5a9019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a966e3640 1 -- 172.21.15.73:0/1110364675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a901028b0 msgr2=0x7f5a9019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a966e3640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a901028b0 0x7f5a9019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.124+0000 7f5a966e3640 1 -- 172.21.15.73:0/1110364675 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a901a0330 con 0x7f5a90108b40 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.125+0000 7f5a966e3640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a90108b40 0x7f5a9019fc00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800b970 tx=0x7f5a8800bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.125+0000 7f5a86ffd640 1 -- 172.21.15.73:0/1110364675 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88019070 con 0x7f5a90108b40 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.125+0000 7f5a86ffd640 1 -- 172.21.15.73:0/1110364675 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a88014440 con 0x7f5a90108b40 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.125+0000 7f5a98146640 1 -- 172.21.15.73:0/1110364675 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a901a0620 con 0x7f5a90108b40 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.125+0000 7f5a86ffd640 1 -- 172.21.15.73:0/1110364675 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a880075d0 con 0x7f5a90108b40 2024-08-31T07:08:01.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.125+0000 7f5a98146640 1 -- 172.21.15.73:0/1110364675 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a901a6d80 con 0x7f5a90108b40 2024-08-31T07:08:01.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.126+0000 7f5a98146640 1 -- 172.21.15.73:0/1110364675 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a90102cc0 con 0x7f5a90108b40 2024-08-31T07:08:01.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.127+0000 7f5a86ffd640 1 -- 172.21.15.73:0/1110364675 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f5a8801c070 con 0x7f5a90108b40 2024-08-31T07:08:01.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.128+0000 7f5a86ffd640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a7c0b4c00 0x7f5a7c0b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:01.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.128+0000 7f5a86ffd640 1 -- 172.21.15.73:0/1110364675 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6476+0+0 (secure 0 0 0) 0x7f5a880e4330 con 0x7f5a90108b40 2024-08-31T07:08:01.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.129+0000 7f5a95ee2640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a7c0b4c00 0x7f5a7c0b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:01.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.129+0000 7f5a95ee2640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a7c0b4c00 0x7f5a7c0b7110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009990 tx=0x7f5a8c009110 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:01.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.131+0000 7f5a86ffd640 1 -- 172.21.15.73:0/1110364675 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5a880abb90 con 0x7f5a90108b40 2024-08-31T07:08:01.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:01.315+0000 7f5a98146640 1 -- 172.21.15.73:0/1110364675 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5a900635a0 con 0x7f5a7c0b4c00 2024-08-31T07:08:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='client.24554 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi032 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi032 to 4273M 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:01.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:01.931 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:01.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:01.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='client.24554 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:08:01.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:08:01.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:08:01.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:01.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi032 2024-08-31T07:08:01.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:01.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:01.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:01.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:01.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:01.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:01.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi032 to 4273M 2024-08-31T07:08:01.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:01.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:01.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:01.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='client.24554 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:08:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:08:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:08:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi032 2024-08-31T07:08:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:02.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:02.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:02.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:02.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:02.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi032 to 4273M 2024-08-31T07:08:02.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:02.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:02.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:02.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:02.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:02 smithi073 ceph-mon[21995]: osdmap e61: 10 total, 10 up, 10 in 2024-08-31T07:08:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:02 smithi084 ceph-mon[22387]: osdmap e61: 10 total, 10 up, 10 in 2024-08-31T07:08:03.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:02 smithi032 ceph-mon[17252]: osdmap e61: 10 total, 10 up, 10 in 2024-08-31T07:08:03.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:03 smithi073 ceph-mon[21995]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:03.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:03 smithi084 ceph-mon[22387]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:03 smithi032 ceph-mon[17252]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:04 smithi073 ceph-mon[21995]: from='client.? 172.21.15.73:0/2105028362' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3e92a61c-4465-44db-813e-d5b731ab2947"} : dispatch 2024-08-31T07:08:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:04 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3e92a61c-4465-44db-813e-d5b731ab2947"} : dispatch 2024-08-31T07:08:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:04 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e92a61c-4465-44db-813e-d5b731ab2947"}]': finished 2024-08-31T07:08:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:04 smithi073 ceph-mon[21995]: osdmap e62: 11 total, 10 up, 11 in 2024-08-31T07:08:04.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:04 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:04.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:04 smithi073 ceph-mon[21995]: from='client.? 172.21.15.73:0/1414466239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:08:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:04 smithi084 ceph-mon[22387]: from='client.? 172.21.15.73:0/2105028362' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3e92a61c-4465-44db-813e-d5b731ab2947"} : dispatch 2024-08-31T07:08:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:04 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3e92a61c-4465-44db-813e-d5b731ab2947"} : dispatch 2024-08-31T07:08:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:04 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e92a61c-4465-44db-813e-d5b731ab2947"}]': finished 2024-08-31T07:08:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:04 smithi084 ceph-mon[22387]: osdmap e62: 11 total, 10 up, 11 in 2024-08-31T07:08:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:04 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:04.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:04 smithi084 ceph-mon[22387]: from='client.? 172.21.15.73:0/1414466239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:08:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:04 smithi032 ceph-mon[17252]: from='client.? 172.21.15.73:0/2105028362' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3e92a61c-4465-44db-813e-d5b731ab2947"} : dispatch 2024-08-31T07:08:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:04 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3e92a61c-4465-44db-813e-d5b731ab2947"} : dispatch 2024-08-31T07:08:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:04 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e92a61c-4465-44db-813e-d5b731ab2947"}]': finished 2024-08-31T07:08:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:04 smithi032 ceph-mon[17252]: osdmap e62: 11 total, 10 up, 11 in 2024-08-31T07:08:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:04 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:04 smithi032 ceph-mon[17252]: from='client.? 172.21.15.73:0/1414466239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:08:05.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:05 smithi073 ceph-mon[21995]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:05.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:05 smithi084 ceph-mon[22387]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:06.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:05 smithi032 ceph-mon[17252]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:08.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:07 smithi032 ceph-mon[17252]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:08.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:07 smithi073 ceph-mon[21995]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:08.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:07 smithi084 ceph-mon[22387]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:09.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:09 smithi073 ceph-mon[21995]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:10.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:09 smithi032 ceph-mon[17252]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:10.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:09 smithi084 ceph-mon[22387]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:11 smithi032 ceph-mon[17252]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:12.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:11 smithi073 ceph-mon[21995]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:12.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:11 smithi084 ceph-mon[22387]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-31T07:08:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:12 smithi032 ceph-mon[17252]: Deploying daemon osd.10 on smithi073 2024-08-31T07:08:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-31T07:08:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:13.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:12 smithi073 ceph-mon[21995]: Deploying daemon osd.10 on smithi073 2024-08-31T07:08:13.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-31T07:08:13.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:13.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:12 smithi084 ceph-mon[22387]: Deploying daemon osd.10 on smithi073 2024-08-31T07:08:14.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:13 smithi032 ceph-mon[17252]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:14.225 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:13 smithi073 ceph-mon[21995]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:14.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:13 smithi084 ceph-mon[22387]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:16.051 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:15 smithi073 ceph-mon[21995]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:16.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:15 smithi032 ceph-mon[17252]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:16.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:15 smithi084 ceph-mon[22387]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:18.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:17 smithi032 ceph-mon[17252]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:18.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:17 smithi073 ceph-mon[21995]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:17 smithi084 ceph-mon[22387]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:19.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:19.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:19.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:20.141 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:19 smithi084 ceph-mon[22387]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:20.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:19 smithi073 ceph-mon[21995]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:20.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:19 smithi032 ceph-mon[17252]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.473+0000 7f5a86ffd640 1 -- 172.21.15.73:0/1110364675 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5a900635a0 con 0x7f5a7c0b4c00 2024-08-31T07:08:21.475 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 10 on host 'smithi073' 2024-08-31T07:08:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 -- 172.21.15.73:0/1110364675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a7c0b4c00 msgr2=0x7f5a7c0b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a7c0b4c00 0x7f5a7c0b7110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009990 tx=0x7f5a8c009110 comp rx=0 tx=0).stop 2024-08-31T07:08:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 -- 172.21.15.73:0/1110364675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a90108b40 msgr2=0x7f5a9019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a90108b40 0x7f5a9019fc00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800b970 tx=0x7f5a8800bd70 comp rx=0 tx=0).stop 2024-08-31T07:08:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 -- 172.21.15.73:0/1110364675 shutdown_connections 2024-08-31T07:08:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a7c0b4c00 0x7f5a7c0b7110 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a90108b40 0x7f5a9019fc00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a901081f0 0x7f5a9019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 --2- 172.21.15.73:0/1110364675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a901028b0 0x7f5a9019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:21.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 -- 172.21.15.73:0/1110364675 >> 172.21.15.73:0/1110364675 conn(0x7f5a90079470 msgr2=0x7f5a901062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:21.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 -- 172.21.15.73:0/1110364675 shutdown_connections 2024-08-31T07:08:21.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-08-31T07:08:21.476+0000 7f5a98146640 1 -- 172.21.15.73:0/1110364675 wait complete. 2024-08-31T07:08:21.843 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:21.843 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:21.843 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:21.843 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:21.843 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:21.843 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:21 smithi073 ceph-mon[21995]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:21.844 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:21.844 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:21.844 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:21.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:21.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:21.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:21.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:21.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:21.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:21 smithi084 ceph-mon[22387]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:21.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:21.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:21.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:22.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:22.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:22.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:22.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:22.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:22.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:21 smithi032 ceph-mon[17252]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:22.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:22.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:22.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:22.179 DEBUG:teuthology.orchestra.run.smithi073:osd.10> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.10.service 2024-08-31T07:08:22.181 INFO:tasks.cephadm:Deploying osd.11 on smithi084 with /dev/vg_nvme/lv_1... 2024-08-31T07:08:22.182 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-31T07:08:22.424 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:08:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:23 smithi073 ceph-mon[21995]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:23.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:23 smithi073 ceph-mon[21995]: from='osd.10 [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T07:08:23.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:23 smithi073 ceph-mon[21995]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T07:08:23.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:08:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:08:23.685+0000 7f199efef640 -1 osd.10 0 log_to_monitors true 2024-08-31T07:08:23.979 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:23 smithi084 ceph-mon[22387]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:23.979 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:23 smithi084 ceph-mon[22387]: from='osd.10 [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T07:08:23.979 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:23 smithi084 ceph-mon[22387]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T07:08:24.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:23 smithi032 ceph-mon[17252]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:24.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:23 smithi032 ceph-mon[17252]: from='osd.10 [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T07:08:24.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:23 smithi032 ceph-mon[17252]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T07:08:24.932 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:24 smithi073 ceph-mon[21995]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-31T07:08:24.933 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:24 smithi073 ceph-mon[21995]: osdmap e63: 11 total, 10 up, 11 in 2024-08-31T07:08:24.933 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:24.933 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:24 smithi073 ceph-mon[21995]: from='osd.10 [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:08:24.933 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:24 smithi073 ceph-mon[21995]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:08:25.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:24 smithi032 ceph-mon[17252]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-31T07:08:25.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:24 smithi032 ceph-mon[17252]: osdmap e63: 11 total, 10 up, 11 in 2024-08-31T07:08:25.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:25.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:24 smithi032 ceph-mon[17252]: from='osd.10 [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:08:25.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:24 smithi032 ceph-mon[17252]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:08:25.143 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:24 smithi084 ceph-mon[22387]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-31T07:08:25.143 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:24 smithi084 ceph-mon[22387]: osdmap e63: 11 total, 10 up, 11 in 2024-08-31T07:08:25.143 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:25.143 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:24 smithi084 ceph-mon[22387]: from='osd.10 [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:08:25.143 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:24 smithi084 ceph-mon[22387]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]} : dispatch 2024-08-31T07:08:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-08-31T07:08:25.173 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_1 2024-08-31T07:08:25.417 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:08:25.679 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:08:25.235+0000 7f199af71640 -1 osd.10 0 waiting for initial osdmap 2024-08-31T07:08:25.679 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:08:25.249+0000 7f1992d67640 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:08:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.957+0000 7faf44859640 1 -- 172.21.15.84:0/900255679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf40103900 msgr2=0x7faf40103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.957+0000 7faf44859640 1 --2- 172.21.15.84:0/900255679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf40103900 0x7faf40103d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf30009900 tx=0x7faf3002a410 comp rx=0 tx=0).stop 2024-08-31T07:08:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.957+0000 7faf44859640 1 -- 172.21.15.84:0/900255679 shutdown_connections 2024-08-31T07:08:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.957+0000 7faf44859640 1 --2- 172.21.15.84:0/900255679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf40109b90 0x7faf401103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.957+0000 7faf44859640 1 --2- 172.21.15.84:0/900255679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf40109240 0x7faf40109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.957+0000 7faf44859640 1 --2- 172.21.15.84:0/900255679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf40103900 0x7faf40103d10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.957+0000 7faf44859640 1 -- 172.21.15.84:0/900255679 >> 172.21.15.84:0/900255679 conn(0x7faf4007b5f0 msgr2=0x7faf4007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:25.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.957+0000 7faf44859640 1 -- 172.21.15.84:0/900255679 shutdown_connections 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.957+0000 7faf44859640 1 -- 172.21.15.84:0/900255679 wait complete. 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf44859640 1 Processor -- start 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf44859640 1 -- start start 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf44859640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf40103900 0x7faf4019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf44859640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf40109240 0x7faf4019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf44859640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf40109b90 0x7faf4019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf44859640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf40113890 con 0x7faf40109b90 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf44859640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf40113710 con 0x7faf40109240 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf44859640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf40113a10 con 0x7faf40103900 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf3e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf40103900 0x7faf4019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf3e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf40103900 0x7faf4019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48074/0 (socket says 172.21.15.84:48074) 2024-08-31T07:08:25.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.958+0000 7faf3e59c640 1 -- 172.21.15.84:0/4031885024 learned_addr learned my addr 172.21.15.84:0/4031885024 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf3e59c640 1 -- 172.21.15.84:0/4031885024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf40109240 msgr2=0x7faf4019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf3e59c640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf40109240 0x7faf4019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf3ed9d640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf40109b90 0x7faf4019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf3e59c640 1 -- 172.21.15.84:0/4031885024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf40109b90 msgr2=0x7faf4019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf3e59c640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf40109b90 0x7faf4019fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf3e59c640 1 -- 172.21.15.84:0/4031885024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf401a0270 con 0x7faf40103900 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf3e59c640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf40103900 0x7faf4019a300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faf30006010 tx=0x7faf300042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf2f7fe640 1 -- 172.21.15.84:0/4031885024 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3003f030 con 0x7faf40103900 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf44859640 1 -- 172.21.15.84:0/4031885024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf401a0500 con 0x7faf40103900 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf2f7fe640 1 -- 172.21.15.84:0/4031885024 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf30003800 con 0x7faf40103900 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf2f7fe640 1 -- 172.21.15.84:0/4031885024 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30035070 con 0x7faf40103900 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.959+0000 7faf44859640 1 -- 172.21.15.84:0/4031885024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf401a6cc0 con 0x7faf40103900 2024-08-31T07:08:25.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.960+0000 7faf44859640 1 -- 172.21.15.84:0/4031885024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf40103d10 con 0x7faf40103900 2024-08-31T07:08:25.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.962+0000 7faf2f7fe640 1 -- 172.21.15.84:0/4031885024 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7faf300039b0 con 0x7faf40103900 2024-08-31T07:08:25.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.962+0000 7faf2f7fe640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf280b4b80 0x7faf280b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:25.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.963+0000 7faf2f7fe640 1 -- 172.21.15.84:0/4031885024 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6629+0+0 (secure 0 0 0) 0x7faf30105720 con 0x7faf40103900 2024-08-31T07:08:25.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.963+0000 7faf3dd9b640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf280b4b80 0x7faf280b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:25.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.963+0000 7faf3dd9b640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf280b4b80 0x7faf280b7090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faf40106260 tx=0x7faf34006c90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:25.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:25.965+0000 7faf2f7fe640 1 -- 172.21.15.84:0/4031885024 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faf300ccee0 con 0x7faf40103900 2024-08-31T07:08:26.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:26.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:08:26.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: osdmap e64: 11 total, 10 up, 11 in 2024-08-31T07:08:26.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:26.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:26.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi073 2024-08-31T07:08:26.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi073 to 4272M 2024-08-31T07:08:26.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:26.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:26.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:25 smithi032 ceph-mon[17252]: from='osd.10 ' entity='osd.10' 2024-08-31T07:08:26.140 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:26.140 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:08:26.140 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: osdmap e64: 11 total, 10 up, 11 in 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi073 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi073 to 4272M 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:26.141 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:26.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.142 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:25 smithi073 ceph-mon[21995]: from='osd.10 ' entity='osd.10' 2024-08-31T07:08:26.150 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:26.150+0000 7faf44859640 1 -- 172.21.15.84:0/4031885024 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7faf400635a0 con 0x7faf280b4b80 2024-08-31T07:08:26.151 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:26.151 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-08-31T07:08:26.151 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: osdmap e64: 11 total, 10 up, 11 in 2024-08-31T07:08:26.151 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:26.151 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:26.151 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi073 2024-08-31T07:08:26.151 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.152 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.152 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.152 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.152 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.152 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:26.152 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi073 to 4272M 2024-08-31T07:08:26.152 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.152 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:26.152 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:26.152 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:26.153 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:25 smithi084 ceph-mon[22387]: from='osd.10 ' entity='osd.10' 2024-08-31T07:08:26.825 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:08:26.826 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:08:26.826 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:26.826 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: from='client.24562 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:08:26.826 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:08:26.826 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:08:26.826 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:26.826 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: osd.10 [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] boot 2024-08-31T07:08:26.826 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: osdmap e65: 11 total, 11 up, 11 in 2024-08-31T07:08:26.826 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:26.826 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:26 smithi084 ceph-mon[22387]: osdmap e66: 11 total, 11 up, 11 in 2024-08-31T07:08:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:08:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:08:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: from='client.24562 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:08:27.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:08:27.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:08:27.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:27.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: osd.10 [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] boot 2024-08-31T07:08:27.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: osdmap e65: 11 total, 11 up, 11 in 2024-08-31T07:08:27.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:27.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:26 smithi032 ceph-mon[17252]: osdmap e66: 11 total, 11 up, 11 in 2024-08-31T07:08:27.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:08:27.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:08:27.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:27.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: from='client.24562 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:08:27.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T07:08:27.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T07:08:27.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:27.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: osd.10 [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] boot 2024-08-31T07:08:27.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: osdmap e65: 11 total, 11 up, 11 in 2024-08-31T07:08:27.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T07:08:27.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:26 smithi073 ceph-mon[21995]: osdmap e66: 11 total, 11 up, 11 in 2024-08-31T07:08:28.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:28 smithi032 ceph-mon[17252]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:28.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:28 smithi032 ceph-mon[17252]: osdmap e67: 11 total, 11 up, 11 in 2024-08-31T07:08:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:28 smithi073 ceph-mon[21995]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:28 smithi073 ceph-mon[21995]: osdmap e67: 11 total, 11 up, 11 in 2024-08-31T07:08:28.971 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:28 smithi084 ceph-mon[22387]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T07:08:28.971 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:28 smithi084 ceph-mon[22387]: osdmap e67: 11 total, 11 up, 11 in 2024-08-31T07:08:29.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:29 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/1406812091' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2163952e-3884-434b-8d0b-d489ea25c1cf"} : dispatch 2024-08-31T07:08:29.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:29 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2163952e-3884-434b-8d0b-d489ea25c1cf"} : dispatch 2024-08-31T07:08:29.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:29 smithi084 ceph-mon[22387]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2163952e-3884-434b-8d0b-d489ea25c1cf"}]': finished 2024-08-31T07:08:29.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:29 smithi084 ceph-mon[22387]: osdmap e68: 12 total, 11 up, 12 in 2024-08-31T07:08:29.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:29 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:29.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:29 smithi084 ceph-mon[22387]: from='client.? 172.21.15.84:0/2735210960' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:08:29.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:29 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/1406812091' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2163952e-3884-434b-8d0b-d489ea25c1cf"} : dispatch 2024-08-31T07:08:29.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:29 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2163952e-3884-434b-8d0b-d489ea25c1cf"} : dispatch 2024-08-31T07:08:29.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:29 smithi032 ceph-mon[17252]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2163952e-3884-434b-8d0b-d489ea25c1cf"}]': finished 2024-08-31T07:08:29.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:29 smithi032 ceph-mon[17252]: osdmap e68: 12 total, 11 up, 12 in 2024-08-31T07:08:29.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:29 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:29.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:29 smithi032 ceph-mon[17252]: from='client.? 172.21.15.84:0/2735210960' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:08:29.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:29 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/1406812091' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2163952e-3884-434b-8d0b-d489ea25c1cf"} : dispatch 2024-08-31T07:08:29.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:29 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2163952e-3884-434b-8d0b-d489ea25c1cf"} : dispatch 2024-08-31T07:08:29.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:29 smithi073 ceph-mon[21995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2163952e-3884-434b-8d0b-d489ea25c1cf"}]': finished 2024-08-31T07:08:29.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:29 smithi073 ceph-mon[21995]: osdmap e68: 12 total, 11 up, 12 in 2024-08-31T07:08:29.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:29 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:29.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:29 smithi073 ceph-mon[21995]: from='client.? 172.21.15.84:0/2735210960' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T07:08:30.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:30 smithi032 ceph-mon[17252]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-31T07:08:30.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:30 smithi073 ceph-mon[21995]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-31T07:08:30.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:30 smithi084 ceph-mon[22387]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-31T07:08:32.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:32 smithi032 ceph-mon[17252]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-08-31T07:08:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:32 smithi073 ceph-mon[21995]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-08-31T07:08:32.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:32 smithi084 ceph-mon[22387]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-08-31T07:08:34.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:34 smithi032 ceph-mon[17252]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-08-31T07:08:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:34 smithi073 ceph-mon[21995]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-08-31T07:08:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:34 smithi084 ceph-mon[22387]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-08-31T07:08:36.679 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:36 smithi084 ceph-mon[22387]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T07:08:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:36 smithi032 ceph-mon[17252]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T07:08:36.864 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:36 smithi073 ceph-mon[21995]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T07:08:37.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:37 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-31T07:08:37.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:37 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:37.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:37 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-31T07:08:37.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:37 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:37.916 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:37 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-31T07:08:37.916 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:37 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:38 smithi084 ceph-mon[22387]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-31T07:08:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:38 smithi084 ceph-mon[22387]: Deploying daemon osd.11 on smithi084 2024-08-31T07:08:38.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:38 smithi032 ceph-mon[17252]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-31T07:08:38.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:38 smithi032 ceph-mon[17252]: Deploying daemon osd.11 on smithi084 2024-08-31T07:08:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:38 smithi073 ceph-mon[21995]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-31T07:08:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:38 smithi073 ceph-mon[21995]: Deploying daemon osd.11 on smithi084 2024-08-31T07:08:40.845 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:40 smithi084 ceph-mon[22387]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:40.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:40 smithi032 ceph-mon[17252]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:40 smithi073 ceph-mon[21995]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:42.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:42 smithi032 ceph-mon[17252]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:42 smithi073 ceph-mon[21995]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:42 smithi084 ceph-mon[22387]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:43.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:43.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:43.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:43 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:43.899 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:43.899 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:43.899 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:43.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:43.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:43.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:44.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:44 smithi032 ceph-mon[17252]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:44 smithi073 ceph-mon[21995]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:44 smithi084 ceph-mon[22387]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:45.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:45 smithi084 ceph-mon[22387]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:45.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:45 smithi073 ceph-mon[21995]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:46.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:45 smithi032 ceph-mon[17252]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:46.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.556+0000 7faf2f7fe640 1 -- 172.21.15.84:0/4031885024 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7faf400635a0 con 0x7faf280b4b80 2024-08-31T07:08:46.557 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 11 on host 'smithi084' 2024-08-31T07:08:46.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.558+0000 7faf44859640 1 -- 172.21.15.84:0/4031885024 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf280b4b80 msgr2=0x7faf280b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:46.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.558+0000 7faf44859640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf280b4b80 0x7faf280b7090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faf40106260 tx=0x7faf34006c90 comp rx=0 tx=0).stop 2024-08-31T07:08:46.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.558+0000 7faf44859640 1 -- 172.21.15.84:0/4031885024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf40103900 msgr2=0x7faf4019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:46.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.558+0000 7faf44859640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf40103900 0x7faf4019a300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faf30006010 tx=0x7faf300042f0 comp rx=0 tx=0).stop 2024-08-31T07:08:46.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.558+0000 7faf44859640 1 -- 172.21.15.84:0/4031885024 shutdown_connections 2024-08-31T07:08:46.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.558+0000 7faf44859640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf280b4b80 0x7faf280b7090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:46.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.558+0000 7faf44859640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf40109b90 0x7faf4019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:46.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.558+0000 7faf44859640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf40109240 0x7faf4019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:46.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.559+0000 7faf44859640 1 --2- 172.21.15.84:0/4031885024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf40103900 0x7faf4019a300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:46.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.559+0000 7faf44859640 1 -- 172.21.15.84:0/4031885024 >> 172.21.15.84:0/4031885024 conn(0x7faf4007b5f0 msgr2=0x7faf40107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:46.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.559+0000 7faf44859640 1 -- 172.21.15.84:0/4031885024 shutdown_connections 2024-08-31T07:08:46.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:46.559+0000 7faf44859640 1 -- 172.21.15.84:0/4031885024 wait complete. 2024-08-31T07:08:46.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:46.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:46.931 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:46.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:46.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:46.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:46.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:46.932 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.009 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.009 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.009 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:47.009 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:47.010 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.010 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:47.010 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.010 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:47.290 DEBUG:teuthology.orchestra.run.smithi084:osd.11> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.11.service 2024-08-31T07:08:47.293 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-31T07:08:47.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd stat -f json 2024-08-31T07:08:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:08:47.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:47 smithi084 ceph-mon[22387]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:48.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:47 smithi032 ceph-mon[17252]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:48.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:47 smithi073 ceph-mon[21995]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:48.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.317+0000 7f1e52200640 1 -- 172.21.15.32:0/3714440509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4c103920 msgr2=0x7f1e4c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:48.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.317+0000 7f1e52200640 1 --2- 172.21.15.32:0/3714440509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4c103920 0x7f1e4c103d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009900 tx=0x7f1e3c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.317+0000 7f1e52200640 1 -- 172.21.15.32:0/3714440509 shutdown_connections 2024-08-31T07:08:48.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.317+0000 7f1e52200640 1 --2- 172.21.15.32:0/3714440509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109bb0 0x7f1e4c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.317+0000 7f1e52200640 1 --2- 172.21.15.32:0/3714440509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c109260 0x7f1e4c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.317+0000 7f1e52200640 1 --2- 172.21.15.32:0/3714440509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4c103920 0x7f1e4c103d30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.317+0000 7f1e52200640 1 -- 172.21.15.32:0/3714440509 >> 172.21.15.32:0/3714440509 conn(0x7f1e4c0ff370 msgr2=0x7f1e4c1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:48.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.317+0000 7f1e52200640 1 -- 172.21.15.32:0/3714440509 shutdown_connections 2024-08-31T07:08:48.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.317+0000 7f1e52200640 1 -- 172.21.15.32:0/3714440509 wait complete. 2024-08-31T07:08:48.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.318+0000 7f1e52200640 1 Processor -- start 2024-08-31T07:08:48.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e52200640 1 -- start start 2024-08-31T07:08:48.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e52200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c103920 0x7f1e4c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e52200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4c109260 0x7f1e4c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e52200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109bb0 0x7f1e4c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e52200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c113870 con 0x7f1e4c103920 2024-08-31T07:08:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e52200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1136f0 con 0x7f1e4c109bb0 2024-08-31T07:08:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e52200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1139f0 con 0x7f1e4c109260 2024-08-31T07:08:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e4b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c103920 0x7f1e4c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e43e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4c109260 0x7f1e4c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e43e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4c109260 0x7f1e4c19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:56124/0 (socket says 172.21.15.32:56124) 2024-08-31T07:08:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.319+0000 7f1e43e00640 1 -- 172.21.15.32:0/618455892 learned_addr learned my addr 172.21.15.32:0/618455892 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:08:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.320+0000 7f1e4be00640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109bb0 0x7f1e4c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.320+0000 7f1e43e00640 1 -- 172.21.15.32:0/618455892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109bb0 msgr2=0x7f1e4c19fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.320+0000 7f1e43e00640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109bb0 0x7f1e4c19fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.320+0000 7f1e43e00640 1 -- 172.21.15.32:0/618455892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c103920 msgr2=0x7f1e4c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.320+0000 7f1e43e00640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c103920 0x7f1e4c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.320+0000 7f1e43e00640 1 -- 172.21.15.32:0/618455892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c1a01f0 con 0x7f1e4c109260 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.320+0000 7f1e43e00640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4c109260 0x7f1e4c19a820 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400e9f0 tx=0x7f1e3400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.320+0000 7f1e48c00640 1 -- 172.21.15.32:0/618455892 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3400ccc0 con 0x7f1e4c109260 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.321+0000 7f1e52200640 1 -- 172.21.15.32:0/618455892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c1a04e0 con 0x7f1e4c109260 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.321+0000 7f1e48c00640 1 -- 172.21.15.32:0/618455892 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e340108e0 con 0x7f1e4c109260 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.321+0000 7f1e48c00640 1 -- 172.21.15.32:0/618455892 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34018860 con 0x7f1e4c109260 2024-08-31T07:08:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.321+0000 7f1e52200640 1 -- 172.21.15.32:0/618455892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c1a6c40 con 0x7f1e4c109260 2024-08-31T07:08:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.322+0000 7f1e52200640 1 -- 172.21.15.32:0/618455892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c103d30 con 0x7f1e4c109260 2024-08-31T07:08:48.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.324+0000 7f1e48c00640 1 -- 172.21.15.32:0/618455892 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f1e34010430 con 0x7f1e4c109260 2024-08-31T07:08:48.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.326+0000 7f1e48c00640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e300b4b80 0x7f1e300b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:48.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.327+0000 7f1e48c00640 1 -- 172.21.15.32:0/618455892 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7024+0+0 (secure 0 0 0) 0x7f1e340e5460 con 0x7f1e4c109260 2024-08-31T07:08:48.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.327+0000 7f1e4b400640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e300b4b80 0x7f1e300b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:48.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.327+0000 7f1e4b400640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e300b4b80 0x7f1e300b7090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c031440 tx=0x7f1e3c009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:48.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.329+0000 7f1e48c00640 1 -- 172.21.15.32:0/618455892 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1e340ac3d0 con 0x7f1e4c109260 2024-08-31T07:08:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.524+0000 7f1e52200640 1 -- 172.21.15.32:0/618455892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1e4c19b3b0 con 0x7f1e4c109260 2024-08-31T07:08:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.524+0000 7f1e48c00640 1 -- 172.21.15.32:0/618455892 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f1e4c19b3b0 con 0x7f1e4c109260 2024-08-31T07:08:48.526 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:08:48.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 -- 172.21.15.32:0/618455892 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e300b4b80 msgr2=0x7f1e300b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:48.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e300b4b80 0x7f1e300b7090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c031440 tx=0x7f1e3c009560 comp rx=0 tx=0).stop 2024-08-31T07:08:48.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 -- 172.21.15.32:0/618455892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4c109260 msgr2=0x7f1e4c19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:48.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4c109260 0x7f1e4c19a820 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400e9f0 tx=0x7f1e3400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 -- 172.21.15.32:0/618455892 shutdown_connections 2024-08-31T07:08:48.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e300b4b80 0x7f1e300b7090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e4c109bb0 0x7f1e4c19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e4c109260 0x7f1e4c19a820 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 --2- 172.21.15.32:0/618455892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c103920 0x7f1e4c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 -- 172.21.15.32:0/618455892 >> 172.21.15.32:0/618455892 conn(0x7f1e4c0ff370 msgr2=0x7f1e4c101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 -- 172.21.15.32:0/618455892 shutdown_connections 2024-08-31T07:08:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:48.527+0000 7f1e52200640 1 -- 172.21.15.32:0/618455892 wait complete. 2024-08-31T07:08:48.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:08:48 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:08:48.600+0000 7f579dc49640 -1 osd.11 0 log_to_monitors true 2024-08-31T07:08:48.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:48 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/618455892' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T07:08:48.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:48 smithi032 ceph-mon[17252]: from='osd.11 [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T07:08:48.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:48 smithi032 ceph-mon[17252]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T07:08:49.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:48 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/618455892' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T07:08:49.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:48 smithi073 ceph-mon[21995]: from='osd.11 [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T07:08:49.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:48 smithi073 ceph-mon[21995]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T07:08:49.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:48 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/618455892' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T07:08:49.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:48 smithi084 ceph-mon[22387]: from='osd.11 [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T07:08:49.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:48 smithi084 ceph-mon[22387]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T07:08:49.514 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1725088106,"num_in_osds":12,"osd_in_since":1725088108,"num_remapped_pgs":0} 2024-08-31T07:08:49.890 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:08:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:08:49.751+0000 7f5799bcb640 -1 osd.11 0 waiting for initial osdmap 2024-08-31T07:08:49.890 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:08:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:08:49.765+0000 7f57959c9640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T07:08:49.890 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:49 smithi084 ceph-mon[22387]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-31T07:08:49.890 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:49 smithi084 ceph-mon[22387]: osdmap e69: 12 total, 11 up, 12 in 2024-08-31T07:08:49.891 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:49 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:49.891 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:49 smithi084 ceph-mon[22387]: from='osd.11 [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:08:49.891 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:49 smithi084 ceph-mon[22387]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:49.891 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:49 smithi084 ceph-mon[22387]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:08:50.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:49 smithi032 ceph-mon[17252]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-31T07:08:50.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:49 smithi032 ceph-mon[17252]: osdmap e69: 12 total, 11 up, 12 in 2024-08-31T07:08:50.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:49 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:50.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:49 smithi032 ceph-mon[17252]: from='osd.11 [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:08:50.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:49 smithi032 ceph-mon[17252]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:50.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:49 smithi032 ceph-mon[17252]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:08:50.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:49 smithi073 ceph-mon[21995]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-31T07:08:50.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:49 smithi073 ceph-mon[21995]: osdmap e69: 12 total, 11 up, 12 in 2024-08-31T07:08:50.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:49 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:50.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:49 smithi073 ceph-mon[21995]: from='osd.11 [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:08:50.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:49 smithi073 ceph-mon[21995]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:50.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:49 smithi073 ceph-mon[21995]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-08-31T07:08:50.516 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd stat -f json 2024-08-31T07:08:50.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:08:50.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: osdmap e70: 12 total, 11 up, 12 in 2024-08-31T07:08:50.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:50.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: Detected new or changed devices on smithi084 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi084 to 4272M 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:50.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:50.896 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:08:51.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:08:51.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: osdmap e70: 12 total, 11 up, 12 in 2024-08-31T07:08:51.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: Detected new or changed devices on smithi084 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi084 to 4272M 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:51.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:51.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:51.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-08-31T07:08:51.242 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: osdmap e70: 12 total, 11 up, 12 in 2024-08-31T07:08:51.242 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:51.242 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:51.243 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: Detected new or changed devices on smithi084 2024-08-31T07:08:51.243 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:51.243 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:51.243 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:51.243 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:51.243 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:51.244 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T07:08:51.244 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi084 to 4272M 2024-08-31T07:08:51.244 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:51.244 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:51.244 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:51.244 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.517+0000 7f6d8ac00640 1 -- 172.21.15.32:0/1267084511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84109090 msgr2=0x7f6d841094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.517+0000 7f6d8ac00640 1 --2- 172.21.15.32:0/1267084511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84109090 0x7f6d841094a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009900 tx=0x7f6d7002a410 comp rx=0 tx=0).stop 2024-08-31T07:08:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.517+0000 7f6d8ac00640 1 -- 172.21.15.32:0/1267084511 shutdown_connections 2024-08-31T07:08:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.517+0000 7f6d8ac00640 1 --2- 172.21.15.32:0/1267084511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d841099e0 0x7f6d84110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.517+0000 7f6d8ac00640 1 --2- 172.21.15.32:0/1267084511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84109090 0x7f6d841094a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.517+0000 7f6d8ac00640 1 --2- 172.21.15.32:0/1267084511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d84103750 0x7f6d84103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.517+0000 7f6d8ac00640 1 -- 172.21.15.32:0/1267084511 >> 172.21.15.32:0/1267084511 conn(0x7f6d840ff1a0 msgr2=0x7f6d841015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.517+0000 7f6d8ac00640 1 -- 172.21.15.32:0/1267084511 shutdown_connections 2024-08-31T07:08:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.517+0000 7f6d8ac00640 1 -- 172.21.15.32:0/1267084511 wait complete. 2024-08-31T07:08:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.518+0000 7f6d8ac00640 1 Processor -- start 2024-08-31T07:08:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.518+0000 7f6d8ac00640 1 -- start start 2024-08-31T07:08:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d8ac00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84103750 0x7f6d840754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d8ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d84109090 0x7f6d840759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d8ac00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d841099e0 0x7f6d84075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d8ac00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6d841136a0 con 0x7f6d841099e0 2024-08-31T07:08:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d8ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6d84113520 con 0x7f6d84109090 2024-08-31T07:08:51.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d8ac00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6d84113820 con 0x7f6d84103750 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d89800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84103750 0x7f6d840754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d89800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84103750 0x7f6d840754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:56144/0 (socket says 172.21.15.32:56144) 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d89800640 1 -- 172.21.15.32:0/74040991 learned_addr learned my addr 172.21.15.32:0/74040991 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d88e00640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d84109090 0x7f6d840759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.519+0000 7f6d8a200640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d841099e0 0x7f6d84075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d89800640 1 -- 172.21.15.32:0/74040991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d84109090 msgr2=0x7f6d840759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d89800640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d84109090 0x7f6d840759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d89800640 1 -- 172.21.15.32:0/74040991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d841099e0 msgr2=0x7f6d84075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d89800640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d841099e0 0x7f6d84075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d89800640 1 -- 172.21.15.32:0/74040991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d841a8350 con 0x7f6d84103750 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d88e00640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d84109090 0x7f6d840759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d89800640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84103750 0x7f6d840754b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800e9f0 tx=0x7f6d7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:51.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d82000640 1 -- 172.21.15.32:0/74040991 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800cd80 con 0x7f6d84103750 2024-08-31T07:08:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d82000640 1 -- 172.21.15.32:0/74040991 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d780108e0 con 0x7f6d84103750 2024-08-31T07:08:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d8ac00640 1 -- 172.21.15.32:0/74040991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d841a8640 con 0x7f6d84103750 2024-08-31T07:08:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d82000640 1 -- 172.21.15.32:0/74040991 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d78018980 con 0x7f6d84103750 2024-08-31T07:08:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.520+0000 7f6d8ac00640 1 -- 172.21.15.32:0/74040991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d841a8ba0 con 0x7f6d84103750 2024-08-31T07:08:51.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.521+0000 7f6d8ac00640 1 -- 172.21.15.32:0/74040991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d84103b60 con 0x7f6d84103750 2024-08-31T07:08:51.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.524+0000 7f6d82000640 1 -- 172.21.15.32:0/74040991 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f6d78010430 con 0x7f6d84103750 2024-08-31T07:08:51.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.525+0000 7f6d82000640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d6c0b4c00 0x7f6d6c0b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:51.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.525+0000 7f6d82000640 1 -- 172.21.15.32:0/74040991 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f6d780e5400 con 0x7f6d84103750 2024-08-31T07:08:51.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.525+0000 7f6d88e00640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d6c0b4c00 0x7f6d6c0b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:51.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.526+0000 7f6d88e00640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d6c0b4c00 0x7f6d6c0b7110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6d7002aa00 tx=0x7f6d70031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:51.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.527+0000 7f6d82000640 1 -- 172.21.15.32:0/74040991 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6d780ac860 con 0x7f6d84103750 2024-08-31T07:08:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.725+0000 7f6d8ac00640 1 -- 172.21.15.32:0/74040991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6d8402e430 con 0x7f6d84103750 2024-08-31T07:08:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.726+0000 7f6d82000640 1 -- 172.21.15.32:0/74040991 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f6d780ac160 con 0x7f6d84103750 2024-08-31T07:08:51.727 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:08:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.728+0000 7f6d8ac00640 1 -- 172.21.15.32:0/74040991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d6c0b4c00 msgr2=0x7f6d6c0b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.728+0000 7f6d8ac00640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d6c0b4c00 0x7f6d6c0b7110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6d7002aa00 tx=0x7f6d70031400 comp rx=0 tx=0).stop 2024-08-31T07:08:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.728+0000 7f6d8ac00640 1 -- 172.21.15.32:0/74040991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84103750 msgr2=0x7f6d840754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.728+0000 7f6d8ac00640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84103750 0x7f6d840754b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800e9f0 tx=0x7f6d7800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:08:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.728+0000 7f6d8ac00640 1 -- 172.21.15.32:0/74040991 shutdown_connections 2024-08-31T07:08:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.728+0000 7f6d8ac00640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d6c0b4c00 0x7f6d6c0b7110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.728+0000 7f6d8ac00640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d841099e0 0x7f6d84075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.728+0000 7f6d8ac00640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d84109090 0x7f6d840759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.728+0000 7f6d8ac00640 1 --2- 172.21.15.32:0/74040991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84103750 0x7f6d840754b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.728+0000 7f6d8ac00640 1 -- 172.21.15.32:0/74040991 >> 172.21.15.32:0/74040991 conn(0x7f6d840ff1a0 msgr2=0x7f6d841015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.729+0000 7f6d8ac00640 1 -- 172.21.15.32:0/74040991 shutdown_connections 2024-08-31T07:08:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:51.729+0000 7f6d8ac00640 1 -- 172.21.15.32:0/74040991 wait complete. 2024-08-31T07:08:52.009 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:51 smithi084 ceph-mon[22387]: purged_snaps scrub starts 2024-08-31T07:08:52.009 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:51 smithi084 ceph-mon[22387]: purged_snaps scrub ok 2024-08-31T07:08:52.010 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:51 smithi084 ceph-mon[22387]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:52.010 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:51 smithi084 ceph-mon[22387]: osd.11 [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] boot 2024-08-31T07:08:52.010 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:51 smithi084 ceph-mon[22387]: osdmap e71: 12 total, 12 up, 12 in 2024-08-31T07:08:52.010 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:51 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:52.010 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:51 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/74040991' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T07:08:52.074 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:51 smithi032 ceph-mon[17252]: purged_snaps scrub starts 2024-08-31T07:08:52.074 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:51 smithi032 ceph-mon[17252]: purged_snaps scrub ok 2024-08-31T07:08:52.074 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:51 smithi032 ceph-mon[17252]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:52.074 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:51 smithi032 ceph-mon[17252]: osd.11 [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] boot 2024-08-31T07:08:52.074 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:51 smithi032 ceph-mon[17252]: osdmap e71: 12 total, 12 up, 12 in 2024-08-31T07:08:52.074 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:51 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:52.075 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:51 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/74040991' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T07:08:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:51 smithi073 ceph-mon[21995]: purged_snaps scrub starts 2024-08-31T07:08:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:51 smithi073 ceph-mon[21995]: purged_snaps scrub ok 2024-08-31T07:08:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:51 smithi073 ceph-mon[21995]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-31T07:08:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:51 smithi073 ceph-mon[21995]: osd.11 [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] boot 2024-08-31T07:08:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:51 smithi073 ceph-mon[21995]: osdmap e71: 12 total, 12 up, 12 in 2024-08-31T07:08:52.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:51 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T07:08:52.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:51 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/74040991' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T07:08:52.583 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1725088130,"num_in_osds":12,"osd_in_since":1725088108,"num_remapped_pgs":0} 2024-08-31T07:08:52.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:08:52.955 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:08:53.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:53 smithi032 ceph-mon[17252]: osdmap e72: 12 total, 12 up, 12 in 2024-08-31T07:08:53.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:53 smithi073 ceph-mon[21995]: osdmap e72: 12 total, 12 up, 12 in 2024-08-31T07:08:53.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:53 smithi084 ceph-mon[22387]: osdmap e72: 12 total, 12 up, 12 in 2024-08-31T07:08:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.752+0000 7f6c2d600640 1 -- 172.21.15.32:0/3887882702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c281099c0 msgr2=0x7f6c281101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.752+0000 7f6c2d600640 1 --2- 172.21.15.32:0/3887882702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c281099c0 0x7f6c281101f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009ca0 tx=0x7f6c2002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.752+0000 7f6c2d600640 1 -- 172.21.15.32:0/3887882702 shutdown_connections 2024-08-31T07:08:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.752+0000 7f6c2d600640 1 --2- 172.21.15.32:0/3887882702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c281099c0 0x7f6c281101f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.752+0000 7f6c2d600640 1 --2- 172.21.15.32:0/3887882702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c28109070 0x7f6c28109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.752+0000 7f6c2d600640 1 --2- 172.21.15.32:0/3887882702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c28103730 0x7f6c28103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.752+0000 7f6c2d600640 1 -- 172.21.15.32:0/3887882702 >> 172.21.15.32:0/3887882702 conn(0x7f6c280ff180 msgr2=0x7f6c281015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.753+0000 7f6c2d600640 1 -- 172.21.15.32:0/3887882702 shutdown_connections 2024-08-31T07:08:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.753+0000 7f6c2d600640 1 -- 172.21.15.32:0/3887882702 wait complete. 2024-08-31T07:08:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.754+0000 7f6c2d600640 1 Processor -- start 2024-08-31T07:08:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.754+0000 7f6c2d600640 1 -- start start 2024-08-31T07:08:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.754+0000 7f6c2d600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c28103730 0x7f6c2819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.754+0000 7f6c2d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28109070 0x7f6c2819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.754+0000 7f6c2d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c281099c0 0x7f6c281a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.754+0000 7f6c2d600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c281136c0 con 0x7f6c28109070 2024-08-31T07:08:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.754+0000 7f6c2d600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28113540 con 0x7f6c281099c0 2024-08-31T07:08:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.754+0000 7f6c2d600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28113840 con 0x7f6c28103730 2024-08-31T07:08:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.754+0000 7f6c27400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28109070 0x7f6c2819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c27400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28109070 0x7f6c2819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46238/0 (socket says 172.21.15.32:46238) 2024-08-31T07:08:53.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c27400640 1 -- 172.21.15.32:0/2857219552 learned_addr learned my addr 172.21.15.32:0/2857219552 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:08:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c27e00640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c28103730 0x7f6c2819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c2cc00640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c281099c0 0x7f6c281a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c27400640 1 -- 172.21.15.32:0/2857219552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c28103730 msgr2=0x7f6c2819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c27400640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c28103730 0x7f6c2819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c27400640 1 -- 172.21.15.32:0/2857219552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c281099c0 msgr2=0x7f6c281a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c27400640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c281099c0 0x7f6c281a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c27400640 1 -- 172.21.15.32:0/2857219552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c281a21e0 con 0x7f6c28109070 2024-08-31T07:08:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c27e00640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c28103730 0x7f6c2819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:08:53.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.755+0000 7f6c27400640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28109070 0x7f6c2819c810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400e9f0 tx=0x7f6c1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.756+0000 7f6c24c00640 1 -- 172.21.15.32:0/2857219552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400ccc0 con 0x7f6c28109070 2024-08-31T07:08:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.756+0000 7f6c2d600640 1 -- 172.21.15.32:0/2857219552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c281a24d0 con 0x7f6c28109070 2024-08-31T07:08:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.756+0000 7f6c24c00640 1 -- 172.21.15.32:0/2857219552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c140108e0 con 0x7f6c28109070 2024-08-31T07:08:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.756+0000 7f6c24c00640 1 -- 172.21.15.32:0/2857219552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14018860 con 0x7f6c28109070 2024-08-31T07:08:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.756+0000 7f6c2d600640 1 -- 172.21.15.32:0/2857219552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c281a8c30 con 0x7f6c28109070 2024-08-31T07:08:53.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.757+0000 7f6c2d600640 1 -- 172.21.15.32:0/2857219552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c28103b40 con 0x7f6c28109070 2024-08-31T07:08:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.761+0000 7f6c24c00640 1 -- 172.21.15.32:0/2857219552 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f6c14010430 con 0x7f6c28109070 2024-08-31T07:08:53.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.762+0000 7f6c24c00640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6c0c0b4b80 0x7f6c0c0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:53.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.762+0000 7f6c24c00640 1 -- 172.21.15.32:0/2857219552 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f6c140e54b0 con 0x7f6c28109070 2024-08-31T07:08:53.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.762+0000 7f6c27e00640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6c0c0b4b80 0x7f6c0c0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.763+0000 7f6c27e00640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6c0c0b4b80 0x7f6c0c0b7090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c18005d00 tx=0x7f6c1800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.764+0000 7f6c24c00640 1 -- 172.21.15.32:0/2857219552 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6c140ac360 con 0x7f6c28109070 2024-08-31T07:08:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.962+0000 7f6c2d600640 1 -- 172.21.15.32:0/2857219552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c2819d3a0 con 0x7f6c28109070 2024-08-31T07:08:53.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.963+0000 7f6c24c00640 1 -- 172.21.15.32:0/2857219552 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18721 (secure 0 0 0) 0x7f6c2819d3a0 con 0x7f6c28109070 2024-08-31T07:08:53.966 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:08:53.966 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:08:52.326813+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:08:53.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 -- 172.21.15.32:0/2857219552 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6c0c0b4b80 msgr2=0x7f6c0c0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:53.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6c0c0b4b80 0x7f6c0c0b7090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c18005d00 tx=0x7f6c1800b410 comp rx=0 tx=0).stop 2024-08-31T07:08:53.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 -- 172.21.15.32:0/2857219552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28109070 msgr2=0x7f6c2819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:53.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28109070 0x7f6c2819c810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400e9f0 tx=0x7f6c1400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 -- 172.21.15.32:0/2857219552 shutdown_connections 2024-08-31T07:08:53.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6c0c0b4b80 0x7f6c0c0b7090 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c281099c0 0x7f6c281a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28109070 0x7f6c2819c810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 --2- 172.21.15.32:0/2857219552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c28103730 0x7f6c2819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:53.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 -- 172.21.15.32:0/2857219552 >> 172.21.15.32:0/2857219552 conn(0x7f6c280ff180 msgr2=0x7f6c2810e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:53.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 -- 172.21.15.32:0/2857219552 shutdown_connections 2024-08-31T07:08:53.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:53.966+0000 7f6c2d600640 1 -- 172.21.15.32:0/2857219552 wait complete. 2024-08-31T07:08:54.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:54 smithi032 ceph-mon[17252]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:08:54.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:54 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2857219552' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:08:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:54 smithi073 ceph-mon[21995]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:08:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:54 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2857219552' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:08:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:54 smithi084 ceph-mon[22387]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:08:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:54 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2857219552' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:08:54.784 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-31T07:04:45.144128+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-31T07:08:54.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-31T07:08:55.154 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:08:55.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.919+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2695451325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0103730 msgr2=0x7f8eb0103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:55.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.919+0000 7f8eb6a00640 1 --2- 172.21.15.32:0/2695451325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0103730 0x7f8eb0103b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009900 tx=0x7f8ea402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:08:55.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.920+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2695451325 shutdown_connections 2024-08-31T07:08:55.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.920+0000 7f8eb6a00640 1 --2- 172.21.15.32:0/2695451325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb01099c0 0x7f8eb01101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:55.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.920+0000 7f8eb6a00640 1 --2- 172.21.15.32:0/2695451325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eb0109070 0x7f8eb0109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:55.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.920+0000 7f8eb6a00640 1 --2- 172.21.15.32:0/2695451325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0103730 0x7f8eb0103b40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:55.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.920+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2695451325 >> 172.21.15.32:0/2695451325 conn(0x7f8eb00ff180 msgr2=0x7f8eb01015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:55.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.920+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2695451325 shutdown_connections 2024-08-31T07:08:55.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.920+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2695451325 wait complete. 2024-08-31T07:08:55.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.921+0000 7f8eb6a00640 1 Processor -- start 2024-08-31T07:08:55.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.921+0000 7f8eb6a00640 1 -- start start 2024-08-31T07:08:55.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.921+0000 7f8eb6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0103730 0x7f8eb00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:55.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.921+0000 7f8eb6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb0109070 0x7f8eb00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:55.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.921+0000 7f8eb6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eb01099c0 0x7f8eb0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:55.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.921+0000 7f8eb6a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb0113670 con 0x7f8eb0103730 2024-08-31T07:08:55.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.921+0000 7f8eb6a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb01134f0 con 0x7f8eb01099c0 2024-08-31T07:08:55.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.921+0000 7f8eb6a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb01137f0 con 0x7f8eb0109070 2024-08-31T07:08:55.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.922+0000 7f8eb5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0103730 0x7f8eb00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:55.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.922+0000 7f8eb4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb0109070 0x7f8eb00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:55.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.922+0000 7f8eb5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0103730 0x7f8eb00754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40142/0 (socket says 172.21.15.32:40142) 2024-08-31T07:08:55.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.922+0000 7f8eb5600640 1 -- 172.21.15.32:0/2486277542 learned_addr learned my addr 172.21.15.32:0/2486277542 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:08:55.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.922+0000 7f8eb6000640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eb01099c0 0x7f8eb0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:55.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.922+0000 7f8eb4c00640 1 -- 172.21.15.32:0/2486277542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eb01099c0 msgr2=0x7f8eb0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:55.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.922+0000 7f8eb4c00640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eb01099c0 0x7f8eb0075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:55.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.922+0000 7f8eb4c00640 1 -- 172.21.15.32:0/2486277542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0103730 msgr2=0x7f8eb00754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:55.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.922+0000 7f8eb4c00640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0103730 0x7f8eb00754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:55.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.922+0000 7f8eb4c00640 1 -- 172.21.15.32:0/2486277542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb01a84c0 con 0x7f8eb0109070 2024-08-31T07:08:55.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.923+0000 7f8eb4c00640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb0109070 0x7f8eb00759f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8eb006c5d0 tx=0x7f8e9c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:55.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.923+0000 7f8eae000640 1 -- 172.21.15.32:0/2486277542 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c00cc40 con 0x7f8eb0109070 2024-08-31T07:08:55.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.923+0000 7f8eae000640 1 -- 172.21.15.32:0/2486277542 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e9c00cde0 con 0x7f8eb0109070 2024-08-31T07:08:55.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.923+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2486277542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb01a87b0 con 0x7f8eb0109070 2024-08-31T07:08:55.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.923+0000 7f8eae000640 1 -- 172.21.15.32:0/2486277542 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c018780 con 0x7f8eb0109070 2024-08-31T07:08:55.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.923+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2486277542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb01a8d40 con 0x7f8eb0109070 2024-08-31T07:08:55.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.924+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2486277542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb0103b40 con 0x7f8eb0109070 2024-08-31T07:08:55.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.927+0000 7f8eae000640 1 -- 172.21.15.32:0/2486277542 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f8e9c010a90 con 0x7f8eb0109070 2024-08-31T07:08:55.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.928+0000 7f8eae000640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e980b4c00 0x7f8e980b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:55.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.928+0000 7f8eae000640 1 -- 172.21.15.32:0/2486277542 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f8e9c0e5590 con 0x7f8eb0109070 2024-08-31T07:08:55.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.928+0000 7f8eb5600640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e980b4c00 0x7f8e980b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:55.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.929+0000 7f8eb5600640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e980b4c00 0x7f8e980b7110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009230 tx=0x7f8ea40058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:55.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:55.930+0000 7f8eae000640 1 -- 172.21.15.32:0/2486277542 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8e9c0ac3a0 con 0x7f8eb0109070 2024-08-31T07:08:56.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.122+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2486277542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8eb010b740 con 0x7f8eb0109070 2024-08-31T07:08:56.125 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-08-31T07:08:56.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.123+0000 7f8eae000640 1 -- 172.21.15.32:0/2486277542 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f8e9c0b0b60 con 0x7f8eb0109070 2024-08-31T07:08:56.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.125+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2486277542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e980b4c00 msgr2=0x7f8e980b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.125+0000 7f8eb6a00640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e980b4c00 0x7f8e980b7110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009230 tx=0x7f8ea40058d0 comp rx=0 tx=0).stop 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.125+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2486277542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb0109070 msgr2=0x7f8eb00759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.125+0000 7f8eb6a00640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb0109070 0x7f8eb00759f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8eb006c5d0 tx=0x7f8e9c00ee50 comp rx=0 tx=0).stop 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.126+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2486277542 shutdown_connections 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.126+0000 7f8eb6a00640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e980b4c00 0x7f8e980b7110 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.126+0000 7f8eb6a00640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eb01099c0 0x7f8eb0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.126+0000 7f8eb6a00640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eb0109070 0x7f8eb00759f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.126+0000 7f8eb6a00640 1 --2- 172.21.15.32:0/2486277542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0103730 0x7f8eb00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.126+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2486277542 >> 172.21.15.32:0/2486277542 conn(0x7f8eb00ff180 msgr2=0x7f8eb01015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.126+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2486277542 shutdown_connections 2024-08-31T07:08:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:56.126+0000 7f8eb6a00640 1 -- 172.21.15.32:0/2486277542 wait complete. 2024-08-31T07:08:56.478 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:56 smithi032 ceph-mon[17252]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:08:56.478 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:56 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2486277542' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T07:08:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:56 smithi073 ceph-mon[21995]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:08:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:56 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2486277542' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T07:08:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:56 smithi084 ceph-mon[22387]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:08:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:56 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2486277542' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T07:08:56.869 INFO:tasks.cephadm:Adding mds.a on smithi032 2024-08-31T07:08:56.870 INFO:tasks.cephadm:Adding mds.d on smithi032 2024-08-31T07:08:56.870 INFO:tasks.cephadm:Adding mds.g on smithi032 2024-08-31T07:08:56.870 INFO:tasks.cephadm:Adding mds.j on smithi032 2024-08-31T07:08:56.870 INFO:tasks.cephadm:Adding mds.b on smithi073 2024-08-31T07:08:56.870 INFO:tasks.cephadm:Adding mds.e on smithi073 2024-08-31T07:08:56.870 INFO:tasks.cephadm:Adding mds.h on smithi073 2024-08-31T07:08:56.870 INFO:tasks.cephadm:Adding mds.k on smithi073 2024-08-31T07:08:56.871 INFO:tasks.cephadm:Adding mds.c on smithi084 2024-08-31T07:08:56.871 INFO:tasks.cephadm:Adding mds.f on smithi084 2024-08-31T07:08:56.871 INFO:tasks.cephadm:Adding mds.i on smithi084 2024-08-31T07:08:56.871 INFO:tasks.cephadm:Adding mds.l on smithi084 2024-08-31T07:08:56.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph orch apply mds all '12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi084=c;smithi084=f;smithi084=i;smithi084=l' 2024-08-31T07:08:57.131 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:08:57.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.655+0000 7f53c4029640 1 -- 172.21.15.84:0/1113405443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53bc075b70 msgr2=0x7f53bc075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:57.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.655+0000 7f53c4029640 1 --2- 172.21.15.84:0/1113405443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53bc075b70 0x7f53bc075f80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009900 tx=0x7f53ac02a410 comp rx=0 tx=0).stop 2024-08-31T07:08:57.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.655+0000 7f53c4029640 1 -- 172.21.15.84:0/1113405443 shutdown_connections 2024-08-31T07:08:57.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.655+0000 7f53c4029640 1 --2- 172.21.15.84:0/1113405443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53bc1141d0 0x7f53bc1166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:57.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.655+0000 7f53c4029640 1 --2- 172.21.15.84:0/1113405443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53bc075b70 0x7f53bc075f80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:57.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.655+0000 7f53c4029640 1 --2- 172.21.15.84:0/1113405443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53bc077120 0x7f53bc075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.655+0000 7f53c4029640 1 -- 172.21.15.84:0/1113405443 >> 172.21.15.84:0/1113405443 conn(0x7f53bc07b5f0 msgr2=0x7f53bc07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.655+0000 7f53c4029640 1 -- 172.21.15.84:0/1113405443 shutdown_connections 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.655+0000 7f53c4029640 1 -- 172.21.15.84:0/1113405443 wait complete. 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c4029640 1 Processor -- start 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c4029640 1 -- start start 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c4029640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53bc075b70 0x7f53bc1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c4029640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53bc077120 0x7f53bc1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c4029640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53bc1141d0 0x7f53bc1a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c4029640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc119e70 con 0x7f53bc075b70 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c4029640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc119cf0 con 0x7f53bc1141d0 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c4029640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc119ff0 con 0x7f53bc077120 2024-08-31T07:08:57.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c15c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53bc077120 0x7f53bc1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c15c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53bc077120 0x7f53bc1a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57562/0 (socket says 172.21.15.84:57562) 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c15c4640 1 -- 172.21.15.84:0/2630458609 learned_addr learned my addr 172.21.15.84:0/2630458609 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c25c6640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53bc1141d0 0x7f53bc1a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c1dc5640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53bc075b70 0x7f53bc1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c15c4640 1 -- 172.21.15.84:0/2630458609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53bc1141d0 msgr2=0x7f53bc1a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c15c4640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53bc1141d0 0x7f53bc1a60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c15c4640 1 -- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53bc075b70 msgr2=0x7f53bc1a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c15c4640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53bc075b70 0x7f53bc1a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c15c4640 1 -- 172.21.15.84:0/2630458609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc1a67d0 con 0x7f53bc077120 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.656+0000 7f53c1dc5640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53bc075b70 0x7f53bc1a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.657+0000 7f53c15c4640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53bc077120 0x7f53bc1a0e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53ac032fd0 tx=0x7f53ac004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.657+0000 7f53b2ffd640 1 -- 172.21.15.84:0/2630458609 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac03f030 con 0x7f53bc077120 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.657+0000 7f53b2ffd640 1 -- 172.21.15.84:0/2630458609 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53ac004490 con 0x7f53bc077120 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.657+0000 7f53c4029640 1 -- 172.21.15.84:0/2630458609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc1a6a60 con 0x7f53bc077120 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.657+0000 7f53b2ffd640 1 -- 172.21.15.84:0/2630458609 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac035070 con 0x7f53bc077120 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.657+0000 7f53c4029640 1 -- 172.21.15.84:0/2630458609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc1ad220 con 0x7f53bc077120 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.658+0000 7f53c4029640 1 -- 172.21.15.84:0/2630458609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53bc075f80 con 0x7f53bc077120 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.659+0000 7f53b2ffd640 1 -- 172.21.15.84:0/2630458609 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f53ac042030 con 0x7f53bc077120 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.660+0000 7f53b2ffd640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53a80b4b80 0x7f53a80b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.660+0000 7f53b2ffd640 1 -- 172.21.15.84:0/2630458609 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f53ac1057c0 con 0x7f53bc077120 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.660+0000 7f53c1dc5640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53a80b4b80 0x7f53a80b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.661+0000 7f53c1dc5640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53a80b4b80 0x7f53a80b7090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005fd0 tx=0x7f53b80093e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.663+0000 7f53b2ffd640 1 -- 172.21.15.84:0/2630458609 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f53ac0cccc0 con 0x7f53bc077120 2024-08-31T07:08:57.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.845+0000 7f53c4029640 1 -- 172.21.15.84:0/2630458609 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi084=c;smithi084=f;smithi084=i;smithi084=l", "target": ["mon-mgr", ""]}) -- 0x7f53bc113640 con 0x7f53a80b4b80 2024-08-31T07:08:57.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.855+0000 7f53b2ffd640 1 -- 172.21.15.84:0/2630458609 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f53bc113640 con 0x7f53a80b4b80 2024-08-31T07:08:57.856 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled mds.all update... 2024-08-31T07:08:57.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 -- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53a80b4b80 msgr2=0x7f53a80b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:57.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53a80b4b80 0x7f53a80b7090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005fd0 tx=0x7f53b80093e0 comp rx=0 tx=0).stop 2024-08-31T07:08:57.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 -- 172.21.15.84:0/2630458609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53bc077120 msgr2=0x7f53bc1a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:57.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53bc077120 0x7f53bc1a0e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53ac032fd0 tx=0x7f53ac004270 comp rx=0 tx=0).stop 2024-08-31T07:08:57.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 -- 172.21.15.84:0/2630458609 shutdown_connections 2024-08-31T07:08:57.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53a80b4b80 0x7f53a80b7090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:57.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53bc1141d0 0x7f53bc1a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:57.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53bc077120 0x7f53bc1a0e00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:57.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 --2- 172.21.15.84:0/2630458609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53bc075b70 0x7f53bc1a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:57.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 -- 172.21.15.84:0/2630458609 >> 172.21.15.84:0/2630458609 conn(0x7f53bc07b5f0 msgr2=0x7f53bc112e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:57.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 -- 172.21.15.84:0/2630458609 shutdown_connections 2024-08-31T07:08:57.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-08-31T07:08:57.857+0000 7f53c4029640 1 -- 172.21.15.84:0/2630458609 wait complete. 2024-08-31T07:08:58.566 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:58 smithi084 ceph-mon[22387]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:08:58.566 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:58.567 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:58.568 DEBUG:teuthology.orchestra.run.smithi032:mds.a> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.a.service 2024-08-31T07:08:58.572 DEBUG:teuthology.orchestra.run.smithi032:mds.d> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.d.service 2024-08-31T07:08:58.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:58 smithi032 ceph-mon[17252]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:08:58.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:58.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:58.610 DEBUG:teuthology.orchestra.run.smithi032:mds.g> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.g.service 2024-08-31T07:08:58.657 DEBUG:teuthology.orchestra.run.smithi032:mds.j> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.j.service 2024-08-31T07:08:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:58 smithi073 ceph-mon[21995]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:08:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:08:58.702 DEBUG:teuthology.orchestra.run.smithi073:mds.b> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.b.service 2024-08-31T07:08:58.705 DEBUG:teuthology.orchestra.run.smithi073:mds.e> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.e.service 2024-08-31T07:08:58.750 DEBUG:teuthology.orchestra.run.smithi073:mds.h> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.h.service 2024-08-31T07:08:58.793 DEBUG:teuthology.orchestra.run.smithi073:mds.k> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.k.service 2024-08-31T07:08:58.836 DEBUG:teuthology.orchestra.run.smithi084:mds.c> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.c.service 2024-08-31T07:08:58.839 DEBUG:teuthology.orchestra.run.smithi084:mds.f> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.f.service 2024-08-31T07:08:58.883 DEBUG:teuthology.orchestra.run.smithi084:mds.i> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.i.service 2024-08-31T07:08:58.925 DEBUG:teuthology.orchestra.run.smithi084:mds.l> sudo journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.l.service 2024-08-31T07:08:58.968 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-31T07:08:58.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:08:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:08:59.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:59 smithi084 ceph-mon[22387]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi084=c;smithi084=f;smithi084=i;smithi084=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:08:59.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:59 smithi084 ceph-mon[22387]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi084=c;smithi084=f;smithi084=i;smithi084=l;count:12 2024-08-31T07:08:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:08:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:08:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:59.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:08:59 smithi084 ceph-mon[22387]: Deploying daemon mds.l on smithi084 2024-08-31T07:08:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:59 smithi032 ceph-mon[17252]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi084=c;smithi084=f;smithi084=i;smithi084=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:08:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:59 smithi032 ceph-mon[17252]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi084=c;smithi084=f;smithi084=i;smithi084=l;count:12 2024-08-31T07:08:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:08:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:08:59.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:59.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:08:59 smithi032 ceph-mon[17252]: Deploying daemon mds.l on smithi084 2024-08-31T07:08:59.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:59 smithi073 ceph-mon[21995]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi084=c;smithi084=f;smithi084=i;smithi084=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:08:59.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:59 smithi073 ceph-mon[21995]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi084=c;smithi084=f;smithi084=i;smithi084=l;count:12 2024-08-31T07:08:59.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:59.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:08:59.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:08:59.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:08:59.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:08:59.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:08:59.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:08:59 smithi073 ceph-mon[21995]: Deploying daemon mds.l on smithi084 2024-08-31T07:08:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.983+0000 7f5dd1e00640 1 -- 172.21.15.32:0/3934248579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dcc103750 msgr2=0x7f5dcc103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.983+0000 7f5dd1e00640 1 --2- 172.21.15.32:0/3934248579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dcc103750 0x7f5dcc103b60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009900 tx=0x7f5dc002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:08:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.983+0000 7f5dd1e00640 1 -- 172.21.15.32:0/3934248579 shutdown_connections 2024-08-31T07:08:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.983+0000 7f5dd1e00640 1 --2- 172.21.15.32:0/3934248579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dcc1099e0 0x7f5dcc110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.983+0000 7f5dd1e00640 1 --2- 172.21.15.32:0/3934248579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dcc109090 0x7f5dcc1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.983+0000 7f5dd1e00640 1 --2- 172.21.15.32:0/3934248579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dcc103750 0x7f5dcc103b60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.983+0000 7f5dd1e00640 1 -- 172.21.15.32:0/3934248579 >> 172.21.15.32:0/3934248579 conn(0x7f5dcc0ff1a0 msgr2=0x7f5dcc1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:08:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.983+0000 7f5dd1e00640 1 -- 172.21.15.32:0/3934248579 shutdown_connections 2024-08-31T07:08:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.983+0000 7f5dd1e00640 1 -- 172.21.15.32:0/3934248579 wait complete. 2024-08-31T07:08:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dd1e00640 1 Processor -- start 2024-08-31T07:08:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dd1e00640 1 -- start start 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dd1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dcc103750 0x7f5dcc0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dd1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dcc109090 0x7f5dcc0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dd1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dcc1099e0 0x7f5dcc075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dd1e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc1136b0 con 0x7f5dcc109090 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dd1e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc113530 con 0x7f5dcc103750 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dd1e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc113830 con 0x7f5dcc1099e0 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dcc109090 0x7f5dcc0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dcc109090 0x7f5dcc0759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40154/0 (socket says 172.21.15.32:40154) 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.985+0000 7f5dcbe00640 1 -- 172.21.15.32:0/232412166 learned_addr learned my addr 172.21.15.32:0/232412166 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.986+0000 7f5dd0a00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dcc103750 0x7f5dcc0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.986+0000 7f5dd1400640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dcc1099e0 0x7f5dcc075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.986+0000 7f5dcbe00640 1 -- 172.21.15.32:0/232412166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dcc1099e0 msgr2=0x7f5dcc075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:59.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.986+0000 7f5dcbe00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dcc1099e0 0x7f5dcc075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.986+0000 7f5dcbe00640 1 -- 172.21.15.32:0/232412166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dcc103750 msgr2=0x7f5dcc0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:08:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.986+0000 7f5dcbe00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dcc103750 0x7f5dcc0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:08:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.986+0000 7f5dcbe00640 1 -- 172.21.15.32:0/232412166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc1a8360 con 0x7f5dcc109090 2024-08-31T07:08:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.986+0000 7f5dcbe00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dcc109090 0x7f5dcc0759f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5db800e9c0 tx=0x7f5db800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.986+0000 7f5dc9600640 1 -- 172.21.15.32:0/232412166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db800ccc0 con 0x7f5dcc109090 2024-08-31T07:08:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.987+0000 7f5dc9600640 1 -- 172.21.15.32:0/232412166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5db80054d0 con 0x7f5dcc109090 2024-08-31T07:08:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.987+0000 7f5dd1e00640 1 -- 172.21.15.32:0/232412166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dcc1a8650 con 0x7f5dcc109090 2024-08-31T07:08:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.987+0000 7f5dc9600640 1 -- 172.21.15.32:0/232412166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8010430 con 0x7f5dcc109090 2024-08-31T07:08:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.987+0000 7f5dd1e00640 1 -- 172.21.15.32:0/232412166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dcc1a8b40 con 0x7f5dcc109090 2024-08-31T07:08:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.988+0000 7f5dd1e00640 1 -- 172.21.15.32:0/232412166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dcc103b60 con 0x7f5dcc109090 2024-08-31T07:08:59.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.990+0000 7f5dc9600640 1 -- 172.21.15.32:0/232412166 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f5db8005020 con 0x7f5dcc109090 2024-08-31T07:08:59.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.992+0000 7f5dc9600640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5db40b4b80 0x7f5db40b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:08:59.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.992+0000 7f5dc9600640 1 -- 172.21.15.32:0/232412166 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f5db80e51c0 con 0x7f5dcc109090 2024-08-31T07:08:59.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.992+0000 7f5dd0a00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5db40b4b80 0x7f5db40b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:08:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.993+0000 7f5dd0a00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5db40b4b80 0x7f5db40b7090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0000c00 tx=0x7f5dc0031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:08:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:08:59.996+0000 7f5dc9600640 1 -- 172.21.15.32:0/232412166 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5db80ac620 con 0x7f5dcc109090 2024-08-31T07:09:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.227+0000 7f5dd1e00640 1 -- 172.21.15.32:0/232412166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dcc0767b0 con 0x7f5dcc109090 2024-08-31T07:09:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.228+0000 7f5dc9600640 1 -- 172.21.15.32:0/232412166 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f5db80abf20 con 0x7f5dcc109090 2024-08-31T07:09:00.230 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:00.230 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":1,"btime":"2024-08-31T07:00:40:646567+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-31T07:09:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 1 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.230+0000 7f5dd1e00640 1 -- 172.21.15.32:0/232412166 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5db40b4b80 msgr2=0x7f5db40b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5db40b4b80 0x7f5db40b7090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0000c00 tx=0x7f5dc0031400 comp rx=0 tx=0).stop 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 -- 172.21.15.32:0/232412166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dcc109090 msgr2=0x7f5dcc0759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dcc109090 0x7f5dcc0759f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5db800e9c0 tx=0x7f5db800ee80 comp rx=0 tx=0).stop 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 -- 172.21.15.32:0/232412166 shutdown_connections 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5db40b4b80 0x7f5db40b7090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5dcc1099e0 0x7f5dcc075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dcc109090 0x7f5dcc0759f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 --2- 172.21.15.32:0/232412166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dcc103750 0x7f5dcc0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 -- 172.21.15.32:0/232412166 >> 172.21.15.32:0/232412166 conn(0x7f5dcc0ff1a0 msgr2=0x7f5dcc1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 -- 172.21.15.32:0/232412166 shutdown_connections 2024-08-31T07:09:00.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:00.231+0000 7f5dd1e00640 1 -- 172.21.15.32:0/232412166 wait complete. 2024-08-31T07:09:00.577 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:09:00 smithi084 systemd[1]: Starting Ceph mds.l for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:09:00.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:00 smithi032 ceph-mon[17252]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:00.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:00 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/232412166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:00.874 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:00 smithi084 ceph-mon[22387]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:00.874 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:00 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/232412166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:00.874 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:09:00 smithi084 podman[53579]: 2024-08-31 07:09:00.62148461 +0000 UTC m=+0.015983827 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:09:00.874 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:09:00 smithi084 podman[53579]: 2024-08-31 07:09:00.75188937 +0000 UTC m=+0.146388577 container create 6508136a91b3c1a6091a7db1974af867a8f6f7aca29ec67411b64a23c3f94339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-08-31T07:09:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:00 smithi073 ceph-mon[21995]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:00 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/232412166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:01.490 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:09:00 smithi084 podman[53579]: 2024-08-31 07:09:00.996343862 +0000 UTC m=+0.390843071 container init 6508136a91b3c1a6091a7db1974af867a8f6f7aca29ec67411b64a23c3f94339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:09:01.491 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:09:00 smithi084 podman[53579]: 2024-08-31 07:09:00.999035337 +0000 UTC m=+0.393534546 container start 6508136a91b3c1a6091a7db1974af867a8f6f7aca29ec67411b64a23c3f94339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-08-31T07:09:01.491 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:09:01 smithi084 ceph-mds[53599]: -- 172.21.15.84:0/1987900097 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e117e35860 con 0x55e11bfdb600 2024-08-31T07:09:01.491 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:09:01 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l[53595]: starting mds.l at 2024-08-31T07:09:01.491 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:09:01 smithi084 bash[53579]: 6508136a91b3c1a6091a7db1974af867a8f6f7aca29ec67411b64a23c3f94339 2024-08-31T07:09:01.491 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:09:01 smithi084 systemd[1]: Started Ceph mds.l for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.84:6832/2138862695,v1:172.21.15.84:6833/2138862695] up:boot 2024-08-31T07:09:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: fsmap 1 up:standby 2024-08-31T07:09:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-31T07:09:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:02.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:02.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:02.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:02.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:02.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:02.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:02 smithi073 ceph-mon[21995]: Deploying daemon mds.k on smithi073 2024-08-31T07:09:02.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:02.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.84:6832/2138862695,v1:172.21.15.84:6833/2138862695] up:boot 2024-08-31T07:09:02.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: fsmap 1 up:standby 2024-08-31T07:09:02.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-31T07:09:02.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:02.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:02.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:02.361 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:02.361 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:02.361 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:02.361 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:02 smithi032 ceph-mon[17252]: Deploying daemon mds.k on smithi073 2024-08-31T07:09:02.474 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:02.475 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.84:6832/2138862695,v1:172.21.15.84:6833/2138862695] up:boot 2024-08-31T07:09:02.475 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: fsmap 1 up:standby 2024-08-31T07:09:02.475 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-31T07:09:02.475 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:02.475 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:02.475 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:02.475 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:02.475 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:02.475 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:02.475 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:02 smithi084 ceph-mon[22387]: Deploying daemon mds.k on smithi073 2024-08-31T07:09:03.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:09:03.355 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:09:03 smithi073 systemd[1]: Starting Ceph mds.k for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:09:03.412 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:03.679 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:09:03 smithi073 podman[52230]: 2024-08-31 07:09:03.25669817 +0000 UTC m=+0.015973895 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:09:03.679 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:09:03 smithi073 podman[52230]: 2024-08-31 07:09:03.436566193 +0000 UTC m=+0.195841903 container create b6c895bcc623f3e873a9251933d47964942f257e4752acd5935bdc474de51de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T07:09:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.029+0000 7f3a69200640 1 -- 172.21.15.32:0/574834708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a64077120 msgr2=0x7f3a64075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.029+0000 7f3a69200640 1 --2- 172.21.15.32:0/574834708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a64077120 0x7f3a64075550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009900 tx=0x7f3a5402a410 comp rx=0 tx=0).stop 2024-08-31T07:09:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.030+0000 7f3a69200640 1 -- 172.21.15.32:0/574834708 shutdown_connections 2024-08-31T07:09:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.030+0000 7f3a69200640 1 --2- 172.21.15.32:0/574834708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a64114020 0x7f3a641164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.030+0000 7f3a69200640 1 --2- 172.21.15.32:0/574834708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a64075b70 0x7f3a64075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.030+0000 7f3a69200640 1 --2- 172.21.15.32:0/574834708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a64077120 0x7f3a64075550 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.030+0000 7f3a69200640 1 -- 172.21.15.32:0/574834708 >> 172.21.15.32:0/574834708 conn(0x7f3a640ff1e0 msgr2=0x7f3a64101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.030+0000 7f3a69200640 1 -- 172.21.15.32:0/574834708 shutdown_connections 2024-08-31T07:09:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.030+0000 7f3a69200640 1 -- 172.21.15.32:0/574834708 wait complete. 2024-08-31T07:09:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.031+0000 7f3a69200640 1 Processor -- start 2024-08-31T07:09:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.031+0000 7f3a69200640 1 -- start start 2024-08-31T07:09:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.031+0000 7f3a69200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a64075b70 0x7f3a641a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.031+0000 7f3a69200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a64077120 0x7f3a641a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.031+0000 7f3a69200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a64114020 0x7f3a641a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.031+0000 7f3a69200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64119c60 con 0x7f3a64075b70 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.031+0000 7f3a69200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64119ae0 con 0x7f3a64114020 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.031+0000 7f3a69200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64119de0 con 0x7f3a64077120 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.032+0000 7f3a62a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a64077120 0x7f3a641a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.032+0000 7f3a62a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a64077120 0x7f3a641a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:39270/0 (socket says 172.21.15.32:39270) 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.032+0000 7f3a62a00640 1 -- 172.21.15.32:0/4132528747 learned_addr learned my addr 172.21.15.32:0/4132528747 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.032+0000 7f3a63400640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a64075b70 0x7f3a641a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.032+0000 7f3a63e00640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a64114020 0x7f3a641a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.032+0000 7f3a62a00640 1 -- 172.21.15.32:0/4132528747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a64114020 msgr2=0x7f3a641a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.032+0000 7f3a62a00640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a64114020 0x7f3a641a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.032+0000 7f3a62a00640 1 -- 172.21.15.32:0/4132528747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a64075b70 msgr2=0x7f3a641a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.032+0000 7f3a62a00640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a64075b70 0x7f3a641a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.032+0000 7f3a62a00640 1 -- 172.21.15.32:0/4132528747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a641a65e0 con 0x7f3a64077120 2024-08-31T07:09:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.033+0000 7f3a62a00640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a64077120 0x7f3a641a0c10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000e9c0 tx=0x7f3a5000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.033+0000 7f3a5be00640 1 -- 172.21.15.32:0/4132528747 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000cd80 con 0x7f3a64077120 2024-08-31T07:09:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.033+0000 7f3a69200640 1 -- 172.21.15.32:0/4132528747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a641a68d0 con 0x7f3a64077120 2024-08-31T07:09:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.033+0000 7f3a5be00640 1 -- 172.21.15.32:0/4132528747 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a500054d0 con 0x7f3a64077120 2024-08-31T07:09:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.033+0000 7f3a5be00640 1 -- 172.21.15.32:0/4132528747 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50010430 con 0x7f3a64077120 2024-08-31T07:09:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.033+0000 7f3a69200640 1 -- 172.21.15.32:0/4132528747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a641ad030 con 0x7f3a64077120 2024-08-31T07:09:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.034+0000 7f3a69200640 1 -- 172.21.15.32:0/4132528747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a64075f80 con 0x7f3a64077120 2024-08-31T07:09:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.037+0000 7f3a5be00640 1 -- 172.21.15.32:0/4132528747 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f3a50004fd0 con 0x7f3a64077120 2024-08-31T07:09:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.038+0000 7f3a5be00640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3a480b4c00 0x7f3a480b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.038+0000 7f3a5be00640 1 -- 172.21.15.32:0/4132528747 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f3a500e5200 con 0x7f3a64077120 2024-08-31T07:09:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.038+0000 7f3a63400640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3a480b4c00 0x7f3a480b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.039+0000 7f3a63400640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3a480b4c00 0x7f3a480b7110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a54031440 tx=0x7f3a54009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.040+0000 7f3a5be00640 1 -- 172.21.15.32:0/4132528747 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3a500ac660 con 0x7f3a64077120 2024-08-31T07:09:04.044 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:09:03 smithi073 podman[52230]: 2024-08-31 07:09:03.681158457 +0000 UTC m=+0.440434168 container init b6c895bcc623f3e873a9251933d47964942f257e4752acd5935bdc474de51de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T07:09:04.044 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:09:03 smithi073 podman[52230]: 2024-08-31 07:09:03.68385172 +0000 UTC m=+0.443127431 container start b6c895bcc623f3e873a9251933d47964942f257e4752acd5935bdc474de51de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, ceph=True) 2024-08-31T07:09:04.044 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:09:03 smithi073 ceph-mds[52250]: -- 172.21.15.73:0/368353364 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564cdb25b860 con 0x564cdf401a80 2024-08-31T07:09:04.044 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:09:03 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k[52246]: starting mds.k at 2024-08-31T07:09:04.044 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:09:03 smithi073 bash[52230]: b6c895bcc623f3e873a9251933d47964942f257e4752acd5935bdc474de51de2 2024-08-31T07:09:04.044 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:09:03 smithi073 systemd[1]: Started Ceph mds.k for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:04 smithi032 ceph-mon[17252]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:04 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:04 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:04 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:04 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:04 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:04 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:04.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.295+0000 7f3a69200640 1 -- 172.21.15.32:0/4132528747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a641a17a0 con 0x7f3a64077120 2024-08-31T07:09:04.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.296+0000 7f3a5be00640 1 -- 172.21.15.32:0/4132528747 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2260 (secure 0 0 0) 0x7f3a500abf60 con 0x7f3a64077120 2024-08-31T07:09:04.297 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:04.297 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"btime":"2024-08-31T07:09:04:042738+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-31T07:09:04.297 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.298+0000 7f3a69200640 1 -- 172.21.15.32:0/4132528747 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3a480b4c00 msgr2=0x7f3a480b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.298+0000 7f3a69200640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3a480b4c00 0x7f3a480b7110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a54031440 tx=0x7f3a54009560 comp rx=0 tx=0).stop 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.298+0000 7f3a69200640 1 -- 172.21.15.32:0/4132528747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a64077120 msgr2=0x7f3a641a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.298+0000 7f3a69200640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a64077120 0x7f3a641a0c10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000e9c0 tx=0x7f3a5000ee80 comp rx=0 tx=0).stop 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.298+0000 7f3a69200640 1 -- 172.21.15.32:0/4132528747 shutdown_connections 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.298+0000 7f3a69200640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3a480b4c00 0x7f3a480b7110 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.298+0000 7f3a69200640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a64114020 0x7f3a641a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.298+0000 7f3a69200640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a64077120 0x7f3a641a0c10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.298+0000 7f3a69200640 1 --2- 172.21.15.32:0/4132528747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a64075b70 0x7f3a641a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.299+0000 7f3a69200640 1 -- 172.21.15.32:0/4132528747 >> 172.21.15.32:0/4132528747 conn(0x7f3a640ff1e0 msgr2=0x7f3a6410ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.299+0000 7f3a69200640 1 -- 172.21.15.32:0/4132528747 shutdown_connections 2024-08-31T07:09:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:04.299+0000 7f3a69200640 1 -- 172.21.15.32:0/4132528747 wait complete. 2024-08-31T07:09:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:04 smithi073 ceph-mon[21995]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:04 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:04 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:04 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:04 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:04 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:04.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:04 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:04.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:04 smithi084 ceph-mon[22387]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:04.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:04 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:04.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:04 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:04.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:04 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:04.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:04 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:04.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:04 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:04.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:04 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:05.323 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:05 smithi032 ceph-mon[17252]: Deploying daemon mds.j on smithi032 2024-08-31T07:09:05.323 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:05 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.73:6832/2003936745,v1:172.21.15.73:6833/2003936745] up:boot 2024-08-31T07:09:05.323 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:05 smithi032 ceph-mon[17252]: fsmap 2 up:standby 2024-08-31T07:09:05.323 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:05 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-31T07:09:05.323 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:05 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4132528747' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:05.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:05 smithi073 ceph-mon[21995]: Deploying daemon mds.j on smithi032 2024-08-31T07:09:05.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:05 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.73:6832/2003936745,v1:172.21.15.73:6833/2003936745] up:boot 2024-08-31T07:09:05.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:05 smithi073 ceph-mon[21995]: fsmap 2 up:standby 2024-08-31T07:09:05.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:05 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-31T07:09:05.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:05 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4132528747' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:05.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:05 smithi084 ceph-mon[22387]: Deploying daemon mds.j on smithi032 2024-08-31T07:09:05.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:05 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.73:6832/2003936745,v1:172.21.15.73:6833/2003936745] up:boot 2024-08-31T07:09:05.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:05 smithi084 ceph-mon[22387]: fsmap 2 up:standby 2024-08-31T07:09:05.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:05 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-31T07:09:05.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:05 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4132528747' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:06.091 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:06 smithi073 ceph-mon[21995]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:06.337 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:06 smithi032 ceph-mon[17252]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:06.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:06 smithi084 ceph-mon[22387]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:07.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:09:07.602 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:09:07 smithi032 podman[60339]: 2024-08-31 07:09:07.373164081 +0000 UTC m=+0.160431712 container create 85284c2b24cc997496b80bc5f22607c5f0b6e288d6b206a408cce54c3bd0c57c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-08-31T07:09:07.602 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:09:07 smithi032 podman[60339]: 2024-08-31 07:09:07.600807768 +0000 UTC m=+0.388075420 container init 85284c2b24cc997496b80bc5f22607c5f0b6e288d6b206a408cce54c3bd0c57c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-08-31T07:09:07.807 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:07.858 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:09:07 smithi032 podman[60339]: 2024-08-31 07:09:07.607545118 +0000 UTC m=+0.394812748 container start 85284c2b24cc997496b80bc5f22607c5f0b6e288d6b206a408cce54c3bd0c57c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-31T07:09:07.858 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:09:07 smithi032 bash[60339]: 85284c2b24cc997496b80bc5f22607c5f0b6e288d6b206a408cce54c3bd0c57c 2024-08-31T07:09:07.858 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:09:07 smithi032 systemd[1]: Started Ceph mds.j for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:07.858 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:09:07 smithi032 ceph-mds[60384]: -- 172.21.15.32:0/632188884 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557af0985860 con 0x557af4b2ba80 2024-08-31T07:09:07.858 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:09:07 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j[60367]: starting mds.j at 2024-08-31T07:09:08.263 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:08 smithi032 ceph-mon[17252]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:08.263 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:08.263 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:08.263 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:08.264 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:08.264 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:08.264 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:08 smithi073 ceph-mon[21995]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:08.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.448+0000 7fea17000640 1 -- 172.21.15.32:0/2039924110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10075b70 msgr2=0x7fea10075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.448+0000 7fea17000640 1 --2- 172.21.15.32:0/2039924110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10075b70 0x7fea10075f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009900 tx=0x7fe9fc02a410 comp rx=0 tx=0).stop 2024-08-31T07:09:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.448+0000 7fea17000640 1 -- 172.21.15.32:0/2039924110 shutdown_connections 2024-08-31T07:09:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.448+0000 7fea17000640 1 --2- 172.21.15.32:0/2039924110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea10114020 0x7fea101164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.448+0000 7fea17000640 1 --2- 172.21.15.32:0/2039924110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10075b70 0x7fea10075f80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.448+0000 7fea17000640 1 --2- 172.21.15.32:0/2039924110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea10077120 0x7fea10075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.448+0000 7fea17000640 1 -- 172.21.15.32:0/2039924110 >> 172.21.15.32:0/2039924110 conn(0x7fea100ff1e0 msgr2=0x7fea10101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.448+0000 7fea17000640 1 -- 172.21.15.32:0/2039924110 shutdown_connections 2024-08-31T07:09:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.448+0000 7fea17000640 1 -- 172.21.15.32:0/2039924110 wait complete. 2024-08-31T07:09:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.449+0000 7fea17000640 1 Processor -- start 2024-08-31T07:09:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.449+0000 7fea17000640 1 -- start start 2024-08-31T07:09:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea17000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea10075b70 0x7fea101a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea17000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea10077120 0x7fea101a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea17000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10114020 0x7fea101a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea17000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fea10119c60 con 0x7fea10114020 2024-08-31T07:09:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea17000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea10119ae0 con 0x7fea10077120 2024-08-31T07:09:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea17000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fea10119de0 con 0x7fea10075b70 2024-08-31T07:09:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea15c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea10075b70 0x7fea101a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea15c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea10075b70 0x7fea101a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59514/0 (socket says 172.21.15.32:59514) 2024-08-31T07:09:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea15c00640 1 -- 172.21.15.32:0/120015503 learned_addr learned my addr 172.21.15.32:0/120015503 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea15200640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea10077120 0x7fea101a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.450+0000 7fea16600640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10114020 0x7fea101a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.451+0000 7fea15c00640 1 -- 172.21.15.32:0/120015503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea10077120 msgr2=0x7fea101a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.451+0000 7fea15c00640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea10077120 0x7fea101a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.451+0000 7fea15c00640 1 -- 172.21.15.32:0/120015503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10114020 msgr2=0x7fea101a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.451+0000 7fea15c00640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10114020 0x7fea101a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.451+0000 7fea15c00640 1 -- 172.21.15.32:0/120015503 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea101a65c0 con 0x7fea10075b70 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.451+0000 7fea15c00640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea10075b70 0x7fea101a0650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea0400d950 tx=0x7fea0400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.451+0000 7fea0e000640 1 -- 172.21.15.32:0/120015503 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04017030 con 0x7fea10075b70 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.451+0000 7fea17000640 1 -- 172.21.15.32:0/120015503 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea101a68b0 con 0x7fea10075b70 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.451+0000 7fea17000640 1 -- 172.21.15.32:0/120015503 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea101ad010 con 0x7fea10075b70 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.452+0000 7fea0e000640 1 -- 172.21.15.32:0/120015503 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea04004020 con 0x7fea10075b70 2024-08-31T07:09:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.452+0000 7fea0e000640 1 -- 172.21.15.32:0/120015503 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea040047a0 con 0x7fea10075b70 2024-08-31T07:09:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.453+0000 7fea17000640 1 -- 172.21.15.32:0/120015503 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f00052c0 con 0x7fea10075b70 2024-08-31T07:09:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.455+0000 7fea0e000640 1 -- 172.21.15.32:0/120015503 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fea040041d0 con 0x7fea10075b70 2024-08-31T07:09:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.457+0000 7fea0e000640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9f80b4c00 0x7fe9f80b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.457+0000 7fea0e000640 1 -- 172.21.15.32:0/120015503 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fea040e4660 con 0x7fea10075b70 2024-08-31T07:09:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.457+0000 7fea15200640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9f80b4c00 0x7fe9f80b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.458+0000 7fea15200640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9f80b4c00 0x7fe9f80b7110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc000c00 tx=0x7fe9fc031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.461+0000 7fea0e000640 1 -- 172.21.15.32:0/120015503 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fea040abac0 con 0x7fea10075b70 2024-08-31T07:09:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:08 smithi084 ceph-mon[22387]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:08.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:08.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:08.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:08.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:08.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.710+0000 7fea17000640 1 -- 172.21.15.32:0/120015503 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9f00055b0 con 0x7fea10075b70 2024-08-31T07:09:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.710+0000 7fea0e000640 1 -- 172.21.15.32:0/120015503 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7fea040ab3c0 con 0x7fea10075b70 2024-08-31T07:09:08.711 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:08.711 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":4,"btime":"2024-08-31T07:09:08:059667+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-08-31T07:09:08.712 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 4 2024-08-31T07:09:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.713+0000 7fea17000640 1 -- 172.21.15.32:0/120015503 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9f80b4c00 msgr2=0x7fe9f80b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.713+0000 7fea17000640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9f80b4c00 0x7fe9f80b7110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc000c00 tx=0x7fe9fc031400 comp rx=0 tx=0).stop 2024-08-31T07:09:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.713+0000 7fea17000640 1 -- 172.21.15.32:0/120015503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea10075b70 msgr2=0x7fea101a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.713+0000 7fea17000640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea10075b70 0x7fea101a0650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea0400d950 tx=0x7fea0400de10 comp rx=0 tx=0).stop 2024-08-31T07:09:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.713+0000 7fea17000640 1 -- 172.21.15.32:0/120015503 shutdown_connections 2024-08-31T07:09:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.713+0000 7fea17000640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9f80b4c00 0x7fe9f80b7110 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.713+0000 7fea17000640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10114020 0x7fea101a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.713+0000 7fea17000640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea10077120 0x7fea101a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.713+0000 7fea17000640 1 --2- 172.21.15.32:0/120015503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea10075b70 0x7fea101a0650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.713+0000 7fea17000640 1 -- 172.21.15.32:0/120015503 >> 172.21.15.32:0/120015503 conn(0x7fea100ff1e0 msgr2=0x7fea1010ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.714+0000 7fea17000640 1 -- 172.21.15.32:0/120015503 shutdown_connections 2024-08-31T07:09:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:08.714+0000 7fea17000640 1 -- 172.21.15.32:0/120015503 wait complete. 2024-08-31T07:09:09.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:09 smithi032 ceph-mon[17252]: Deploying daemon mds.d on smithi032 2024-08-31T07:09:09.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:09 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.32:6834/1250889023,v1:172.21.15.32:6835/1250889023] up:boot 2024-08-31T07:09:09.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:09 smithi032 ceph-mon[17252]: fsmap 3 up:standby 2024-08-31T07:09:09.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:09 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-31T07:09:09.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:09 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/120015503' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:09 smithi073 ceph-mon[21995]: Deploying daemon mds.d on smithi032 2024-08-31T07:09:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:09 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.32:6834/1250889023,v1:172.21.15.32:6835/1250889023] up:boot 2024-08-31T07:09:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:09 smithi073 ceph-mon[21995]: fsmap 3 up:standby 2024-08-31T07:09:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:09 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-31T07:09:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:09 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/120015503' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:09.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:09 smithi084 ceph-mon[22387]: Deploying daemon mds.d on smithi032 2024-08-31T07:09:09.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:09 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.32:6834/1250889023,v1:172.21.15.32:6835/1250889023] up:boot 2024-08-31T07:09:09.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:09 smithi084 ceph-mon[22387]: fsmap 3 up:standby 2024-08-31T07:09:09.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:09 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-31T07:09:09.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:09 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/120015503' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:10.226 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:10 smithi032 ceph-mon[17252]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:10.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:10 smithi073 ceph-mon[21995]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:10.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:10 smithi084 ceph-mon[22387]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:11.608 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:09:11 smithi032 systemd[1]: Starting Ceph mds.d for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:09:11.949 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:09:11 smithi032 podman[60732]: 2024-08-31 07:09:11.518437441 +0000 UTC m=+0.028054082 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:09:11.949 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:09:11 smithi032 podman[60732]: 2024-08-31 07:09:11.69011704 +0000 UTC m=+0.199733650 container create c677467ca775a1c944396c7a77cc5a8289a4d18a2399aa8053786152788fcd63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-31T07:09:11.949 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:09:11 smithi032 podman[60732]: 2024-08-31 07:09:11.948890089 +0000 UTC m=+0.458506724 container init c677467ca775a1c944396c7a77cc5a8289a4d18a2399aa8053786152788fcd63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-31T07:09:11.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:09:12.358 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:09:11 smithi032 podman[60732]: 2024-08-31 07:09:11.955321294 +0000 UTC m=+0.464937906 container start c677467ca775a1c944396c7a77cc5a8289a4d18a2399aa8053786152788fcd63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-d, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-31T07:09:12.358 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:09:12 smithi032 ceph-mds[60751]: -- 172.21.15.32:0/229647440 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55686884f860 con 0x55686c9f5600 2024-08-31T07:09:12.358 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:09:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-d[60747]: starting mds.d at 2024-08-31T07:09:12.358 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:09:12 smithi032 bash[60732]: c677467ca775a1c944396c7a77cc5a8289a4d18a2399aa8053786152788fcd63 2024-08-31T07:09:12.358 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:09:12 smithi032 systemd[1]: Started Ceph mds.d for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:12.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:12 smithi032 ceph-mon[17252]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:12.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:12.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:12.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:12.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:12.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:12.854 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:12 smithi073 ceph-mon[21995]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:12.893 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:12 smithi084 ceph-mon[22387]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:12.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:12.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:12.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:12.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:13.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.032+0000 7f2fa0e00640 1 -- 172.21.15.32:0/4028903577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c1144e0 msgr2=0x7f2f9c116900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:13.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.032+0000 7f2fa0e00640 1 --2- 172.21.15.32:0/4028903577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c1144e0 0x7f2f9c116900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009ca0 tx=0x7f2f9402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.033+0000 7f2fa0e00640 1 -- 172.21.15.32:0/4028903577 shutdown_connections 2024-08-31T07:09:13.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.033+0000 7f2fa0e00640 1 --2- 172.21.15.32:0/4028903577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c1144e0 0x7f2f9c116900 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.033+0000 7f2fa0e00640 1 --2- 172.21.15.32:0/4028903577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c078030 0x7f2f9c113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.033+0000 7f2fa0e00640 1 --2- 172.21.15.32:0/4028903577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c0776e0 0x7f2f9c077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.033+0000 7f2fa0e00640 1 -- 172.21.15.32:0/4028903577 >> 172.21.15.32:0/4028903577 conn(0x7f2f9c101570 msgr2=0x7f2f9c1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.033+0000 7f2fa0e00640 1 -- 172.21.15.32:0/4028903577 shutdown_connections 2024-08-31T07:09:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.033+0000 7f2fa0e00640 1 -- 172.21.15.32:0/4028903577 wait complete. 2024-08-31T07:09:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.034+0000 7f2fa0e00640 1 Processor -- start 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.034+0000 7f2fa0e00640 1 -- start start 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.034+0000 7f2fa0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c0776e0 0x7f2f9c1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.034+0000 7f2fa0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c078030 0x7f2f9c1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.034+0000 7f2fa0e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c1144e0 0x7f2f9c1a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.034+0000 7f2fa0e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c119e90 con 0x7f2f9c078030 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.034+0000 7f2fa0e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c119d10 con 0x7f2f9c0776e0 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.034+0000 7f2fa0e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c11a010 con 0x7f2f9c1144e0 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c078030 0x7f2f9c1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c0776e0 0x7f2f9c1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c078030 0x7f2f9c1a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51914/0 (socket says 172.21.15.32:51914) 2024-08-31T07:09:13.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99400640 1 -- 172.21.15.32:0/550097052 learned_addr learned my addr 172.21.15.32:0/550097052 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f9a800640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c1144e0 0x7f2f9c1a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99e00640 1 -- 172.21.15.32:0/550097052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c1144e0 msgr2=0x7f2f9c1a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99e00640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c1144e0 0x7f2f9c1a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99e00640 1 -- 172.21.15.32:0/550097052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c078030 msgr2=0x7f2f9c1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99e00640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c078030 0x7f2f9c1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99e00640 1 -- 172.21.15.32:0/550097052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9c1a6790 con 0x7f2f9c0776e0 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99400640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c078030 0x7f2f9c1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.035+0000 7f2f99e00640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c0776e0 0x7f2f9c1a0820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c005fd0 tx=0x7f2f8c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.036+0000 7f2f92a00640 1 -- 172.21.15.32:0/550097052 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c009c30 con 0x7f2f9c0776e0 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.036+0000 7f2f92a00640 1 -- 172.21.15.32:0/550097052 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f8c009dd0 con 0x7f2f9c0776e0 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.036+0000 7f2fa0e00640 1 -- 172.21.15.32:0/550097052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9c1a6a80 con 0x7f2f9c0776e0 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.036+0000 7f2f92a00640 1 -- 172.21.15.32:0/550097052 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c010460 con 0x7f2f9c0776e0 2024-08-31T07:09:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.036+0000 7f2fa0e00640 1 -- 172.21.15.32:0/550097052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9c1ad1e0 con 0x7f2f9c0776e0 2024-08-31T07:09:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.037+0000 7f2fa0e00640 1 -- 172.21.15.32:0/550097052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9c077af0 con 0x7f2f9c0776e0 2024-08-31T07:09:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.039+0000 7f2f92a00640 1 -- 172.21.15.32:0/550097052 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f2f8c005030 con 0x7f2f9c0776e0 2024-08-31T07:09:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.041+0000 7f2f92a00640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f800b4b80 0x7f2f800b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.041+0000 7f2f92a00640 1 -- 172.21.15.32:0/550097052 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f2f8c0e4da0 con 0x7f2f9c0776e0 2024-08-31T07:09:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.041+0000 7f2f99400640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f800b4b80 0x7f2f800b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.042+0000 7f2f99400640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f800b4b80 0x7f2f800b7090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c07b050 tx=0x7f2f88005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:13.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.042+0000 7f2f92a00640 1 -- 172.21.15.32:0/550097052 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2f8c017020 con 0x7f2f9c0776e0 2024-08-31T07:09:13.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.279+0000 7f2fa0e00640 1 -- 172.21.15.32:0/550097052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f9c1a1950 con 0x7f2f9c0776e0 2024-08-31T07:09:13.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.280+0000 7f2f92a00640 1 -- 172.21.15.32:0/550097052 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f2f8c0abbb0 con 0x7f2f9c0776e0 2024-08-31T07:09:13.281 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:13.282 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"btime":"2024-08-31T07:09:12:569888+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-08-31T07:09:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2024-08-31T07:09:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 -- 172.21.15.32:0/550097052 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f800b4b80 msgr2=0x7f2f800b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f800b4b80 0x7f2f800b7090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c07b050 tx=0x7f2f88005f90 comp rx=0 tx=0).stop 2024-08-31T07:09:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 -- 172.21.15.32:0/550097052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c0776e0 msgr2=0x7f2f9c1a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c0776e0 0x7f2f9c1a0820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c005fd0 tx=0x7f2f8c00dde0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 -- 172.21.15.32:0/550097052 shutdown_connections 2024-08-31T07:09:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f800b4b80 0x7f2f800b7090 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c1144e0 0x7f2f9c1a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c078030 0x7f2f9c1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 --2- 172.21.15.32:0/550097052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c0776e0 0x7f2f9c1a0820 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:13.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 -- 172.21.15.32:0/550097052 >> 172.21.15.32:0/550097052 conn(0x7f2f9c101570 msgr2=0x7f2f9c1164f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:13.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 -- 172.21.15.32:0/550097052 shutdown_connections 2024-08-31T07:09:13.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:13.283+0000 7f2fa0e00640 1 -- 172.21.15.32:0/550097052 wait complete. 2024-08-31T07:09:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:13 smithi032 ceph-mon[17252]: Deploying daemon mds.e on smithi073 2024-08-31T07:09:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:13 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.32:6836/2918070851,v1:172.21.15.32:6837/2918070851] up:boot 2024-08-31T07:09:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:13 smithi032 ceph-mon[17252]: fsmap 4 up:standby 2024-08-31T07:09:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:13 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-31T07:09:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:13 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/550097052' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:13.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:13 smithi073 ceph-mon[21995]: Deploying daemon mds.e on smithi073 2024-08-31T07:09:13.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:13 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.32:6836/2918070851,v1:172.21.15.32:6837/2918070851] up:boot 2024-08-31T07:09:13.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:13 smithi073 ceph-mon[21995]: fsmap 4 up:standby 2024-08-31T07:09:13.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:13 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-31T07:09:13.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:13 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/550097052' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:13.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:13 smithi084 ceph-mon[22387]: Deploying daemon mds.e on smithi073 2024-08-31T07:09:13.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:13 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.32:6836/2918070851,v1:172.21.15.32:6837/2918070851] up:boot 2024-08-31T07:09:13.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:13 smithi084 ceph-mon[22387]: fsmap 4 up:standby 2024-08-31T07:09:13.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:13 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-31T07:09:13.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:13 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/550097052' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:14.402 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:09:14 smithi073 systemd[1]: Starting Ceph mds.e for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:09:14.670 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:14 smithi073 ceph-mon[21995]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:14.670 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:09:14 smithi073 podman[52545]: 2024-08-31 07:09:14.304633013 +0000 UTC m=+0.015823301 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:09:14.670 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:09:14 smithi073 podman[52545]: 2024-08-31 07:09:14.417172324 +0000 UTC m=+0.128362596 container create ca5c809c4cb553da515871d5033bbdf07dc13f28380b6d8b1130631723e7b909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-08-31T07:09:14.671 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:09:14 smithi073 podman[52545]: 2024-08-31 07:09:14.670279251 +0000 UTC m=+0.381469530 container init ca5c809c4cb553da515871d5033bbdf07dc13f28380b6d8b1130631723e7b909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:09:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:14 smithi032 ceph-mon[17252]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:14.929 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:09:14 smithi073 podman[52545]: 2024-08-31 07:09:14.673261592 +0000 UTC m=+0.384451865 container start ca5c809c4cb553da515871d5033bbdf07dc13f28380b6d8b1130631723e7b909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:09:14.929 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:09:14 smithi073 ceph-mds[52564]: -- 172.21.15.73:0/286212888 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ed1d89860 con 0x561ed5f2f180 2024-08-31T07:09:14.929 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:09:14 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e[52560]: starting mds.e at 2024-08-31T07:09:14.929 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:09:14 smithi073 bash[52545]: ca5c809c4cb553da515871d5033bbdf07dc13f28380b6d8b1130631723e7b909 2024-08-31T07:09:14.930 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:09:14 smithi073 systemd[1]: Started Ceph mds.e for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:14.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:14 smithi084 ceph-mon[22387]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:15.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:15.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:15.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:15.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:15.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:15.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:15.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:15.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: Deploying daemon mds.f on smithi084 2024-08-31T07:09:15.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.73:6834/1927490574,v1:172.21.15.73:6835/1927490574] up:boot 2024-08-31T07:09:15.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: fsmap 5 up:standby 2024-08-31T07:09:15.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-31T07:09:16.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:16.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:16.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:16.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:16.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:16.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:16.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:16.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: Deploying daemon mds.f on smithi084 2024-08-31T07:09:16.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.73:6834/1927490574,v1:172.21.15.73:6835/1927490574] up:boot 2024-08-31T07:09:16.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: fsmap 5 up:standby 2024-08-31T07:09:16.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-31T07:09:16.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:16.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:16.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:16.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:16.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:16.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:16.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:16.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: Deploying daemon mds.f on smithi084 2024-08-31T07:09:16.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.73:6834/1927490574,v1:172.21.15.73:6835/1927490574] up:boot 2024-08-31T07:09:16.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: fsmap 5 up:standby 2024-08-31T07:09:16.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-31T07:09:16.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:09:16.597 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:16.990 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:09:16 smithi084 systemd[1]: Starting Ceph mds.f for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:09:17.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.251+0000 7f72a4200640 1 -- 172.21.15.32:0/1754794186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f729c109240 msgr2=0x7f729c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:17.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.251+0000 7f72a4200640 1 --2- 172.21.15.32:0/1754794186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f729c109240 0x7f729c109650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7288009900 tx=0x7f728802a410 comp rx=0 tx=0).stop 2024-08-31T07:09:17.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.251+0000 7f72a4200640 1 -- 172.21.15.32:0/1754794186 shutdown_connections 2024-08-31T07:09:17.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.251+0000 7f72a4200640 1 --2- 172.21.15.32:0/1754794186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c109b90 0x7f729c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:17.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.251+0000 7f72a4200640 1 --2- 172.21.15.32:0/1754794186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f729c109240 0x7f729c109650 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.251+0000 7f72a4200640 1 --2- 172.21.15.32:0/1754794186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f729c103900 0x7f729c103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.251+0000 7f72a4200640 1 -- 172.21.15.32:0/1754794186 >> 172.21.15.32:0/1754794186 conn(0x7f729c07b5f0 msgr2=0x7f729c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.251+0000 7f72a4200640 1 -- 172.21.15.32:0/1754794186 shutdown_connections 2024-08-31T07:09:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.251+0000 7f72a4200640 1 -- 172.21.15.32:0/1754794186 wait complete. 2024-08-31T07:09:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.252+0000 7f72a4200640 1 Processor -- start 2024-08-31T07:09:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.252+0000 7f72a4200640 1 -- start start 2024-08-31T07:09:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a4200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f729c103900 0x7f729c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a4200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c109240 0x7f729c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a4200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f729c109b90 0x7f729c1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a4200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f729c113850 con 0x7f729c103900 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a4200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1136d0 con 0x7f729c109240 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a4200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1139d0 con 0x7f729c109b90 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a1800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f729c103900 0x7f729c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a1800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f729c103900 0x7f729c19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37354/0 (socket says 172.21.15.32:37354) 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a1800640 1 -- 172.21.15.32:0/68043237 learned_addr learned my addr 172.21.15.32:0/68043237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a2200640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f729c109b90 0x7f729c1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.253+0000 7f72a0e00640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c109240 0x7f729c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.254+0000 7f72a1800640 1 -- 172.21.15.32:0/68043237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f729c109b90 msgr2=0x7f729c1a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.254+0000 7f72a1800640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f729c109b90 0x7f729c1a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.254+0000 7f72a1800640 1 -- 172.21.15.32:0/68043237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c109240 msgr2=0x7f729c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.254+0000 7f72a1800640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c109240 0x7f729c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.254+0000 7f72a1800640 1 -- 172.21.15.32:0/68043237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c1a2410 con 0x7f729c103900 2024-08-31T07:09:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.254+0000 7f72a2200640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f729c109b90 0x7f729c1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.254+0000 7f72a1800640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f729c103900 0x7f729c19c4a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f729000e9f0 tx=0x7f729000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.254+0000 7f729a000640 1 -- 172.21.15.32:0/68043237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729000cd80 con 0x7f729c103900 2024-08-31T07:09:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.254+0000 7f729a000640 1 -- 172.21.15.32:0/68043237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72900108e0 con 0x7f729c103900 2024-08-31T07:09:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.254+0000 7f729a000640 1 -- 172.21.15.32:0/68043237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290018980 con 0x7f729c103900 2024-08-31T07:09:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.255+0000 7f72a4200640 1 -- 172.21.15.32:0/68043237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729c1a2700 con 0x7f729c103900 2024-08-31T07:09:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.255+0000 7f72a4200640 1 -- 172.21.15.32:0/68043237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729c1a8e60 con 0x7f729c103900 2024-08-31T07:09:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.256+0000 7f72a4200640 1 -- 172.21.15.32:0/68043237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f729c104b80 con 0x7f729c103900 2024-08-31T07:09:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.258+0000 7f729a000640 1 -- 172.21.15.32:0/68043237 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f7290010430 con 0x7f729c103900 2024-08-31T07:09:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.261+0000 7f729a000640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f72840b4b80 0x7f72840b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.261+0000 7f729a000640 1 -- 172.21.15.32:0/68043237 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f72900e5620 con 0x7f729c103900 2024-08-31T07:09:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.261+0000 7f72a0e00640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f72840b4b80 0x7f72840b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.262+0000 7f729a000640 1 -- 172.21.15.32:0/68043237 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f72900ac430 con 0x7f729c103900 2024-08-31T07:09:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.262+0000 7f72a0e00640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f72840b4b80 0x7f72840b7090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7288009230 tx=0x7f7288031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:17.342 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:09:17 smithi084 podman[53881]: 2024-08-31 07:09:16.943568255 +0000 UTC m=+0.015795521 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:09:17.342 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:09:17 smithi084 podman[53881]: 2024-08-31 07:09:17.081231924 +0000 UTC m=+0.153459189 container create e0369cd25dcbb57d0f0194cada8d1bc984157d0730f172b3fe9f5d19412e2e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-31T07:09:17.342 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:09:17 smithi084 podman[53881]: 2024-08-31 07:09:17.342157829 +0000 UTC m=+0.414385096 container init e0369cd25dcbb57d0f0194cada8d1bc984157d0730f172b3fe9f5d19412e2e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-31T07:09:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.497+0000 7f72a4200640 1 -- 172.21.15.32:0/68043237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f729c103d10 con 0x7f729c103900 2024-08-31T07:09:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.498+0000 7f729a000640 1 -- 172.21.15.32:0/68043237 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f729c103d10 con 0x7f729c103900 2024-08-31T07:09:17.499 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:17.499 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"btime":"2024-08-31T07:09:15:584870+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-31T07:09:17.499 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-08-31T07:09:17.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 -- 172.21.15.32:0/68043237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f72840b4b80 msgr2=0x7f72840b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:17.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f72840b4b80 0x7f72840b7090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7288009230 tx=0x7f7288031400 comp rx=0 tx=0).stop 2024-08-31T07:09:17.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 -- 172.21.15.32:0/68043237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f729c103900 msgr2=0x7f729c19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:17.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f729c103900 0x7f729c19c4a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f729000e9f0 tx=0x7f729000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:09:17.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 -- 172.21.15.32:0/68043237 shutdown_connections 2024-08-31T07:09:17.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f72840b4b80 0x7f72840b7090 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:17.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f729c109b90 0x7f729c1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:17.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c109240 0x7f729c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:17.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 --2- 172.21.15.32:0/68043237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f729c103900 0x7f729c19c4a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:17.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 -- 172.21.15.32:0/68043237 >> 172.21.15.32:0/68043237 conn(0x7f729c07b5f0 msgr2=0x7f729c107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:17.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.500+0000 7f72a4200640 1 -- 172.21.15.32:0/68043237 shutdown_connections 2024-08-31T07:09:17.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:17.501+0000 7f72a4200640 1 -- 172.21.15.32:0/68043237 wait complete. 2024-08-31T07:09:17.740 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:09:17 smithi084 podman[53881]: 2024-08-31 07:09:17.345018633 +0000 UTC m=+0.417245900 container start e0369cd25dcbb57d0f0194cada8d1bc984157d0730f172b3fe9f5d19412e2e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731) 2024-08-31T07:09:17.740 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-mds[53900]: -- 172.21.15.84:0/3854318380 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b665e81860 con 0x55b66a027a80 2024-08-31T07:09:17.741 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f[53896]: starting mds.f at 2024-08-31T07:09:17.741 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:09:17 smithi084 bash[53881]: e0369cd25dcbb57d0f0194cada8d1bc984157d0730f172b3fe9f5d19412e2e39 2024-08-31T07:09:17.741 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:09:17 smithi084 systemd[1]: Started Ceph mds.f for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:18.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:17 smithi032 ceph-mon[17252]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:18.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:17 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/68043237' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:18.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:17 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:18.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:17 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:18.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:17 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:18.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:17 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:18.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:17 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:18.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:17 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:18.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:17 smithi032 ceph-mon[17252]: Deploying daemon mds.c on smithi084 2024-08-31T07:09:18.121 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-mon[22387]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:18.121 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/68043237' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:18.121 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:18.121 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:18.121 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:18.121 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:18.121 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:18.121 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:18.121 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:17 smithi084 ceph-mon[22387]: Deploying daemon mds.c on smithi084 2024-08-31T07:09:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:17 smithi073 ceph-mon[21995]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:17 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/68043237' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:17 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:17 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:18.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:17 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:18.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:17 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:18.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:17 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:18.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:17 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:18.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:17 smithi073 ceph-mon[21995]: Deploying daemon mds.c on smithi084 2024-08-31T07:09:19.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:18 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.84:6834/204703622,v1:172.21.15.84:6835/204703622] up:boot 2024-08-31T07:09:19.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:18 smithi032 ceph-mon[17252]: fsmap 6 up:standby 2024-08-31T07:09:19.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T07:09:19.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:18 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.84:6834/204703622,v1:172.21.15.84:6835/204703622] up:boot 2024-08-31T07:09:19.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:18 smithi073 ceph-mon[21995]: fsmap 6 up:standby 2024-08-31T07:09:19.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T07:09:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:18 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.84:6834/204703622,v1:172.21.15.84:6835/204703622] up:boot 2024-08-31T07:09:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:18 smithi084 ceph-mon[22387]: fsmap 6 up:standby 2024-08-31T07:09:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T07:09:19.650 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:09:19 smithi084 systemd[1]: Starting Ceph mds.c for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:09:19.900 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:19 smithi084 ceph-mon[22387]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:19.901 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:09:19 smithi084 podman[54178]: 2024-08-31 07:09:19.552393986 +0000 UTC m=+0.015778265 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:09:19.901 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:09:19 smithi084 podman[54178]: 2024-08-31 07:09:19.695046568 +0000 UTC m=+0.158430841 container create 4ee2e5b1123e71e4f81ecef74dc9a885d2e449179eaed5fd417f5f10e8b8b542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T07:09:19.901 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:09:19 smithi084 podman[54178]: 2024-08-31 07:09:19.897864308 +0000 UTC m=+0.361248581 container init 4ee2e5b1123e71e4f81ecef74dc9a885d2e449179eaed5fd417f5f10e8b8b542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-31T07:09:19.901 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:09:19 smithi084 podman[54178]: 2024-08-31 07:09:19.900865444 +0000 UTC m=+0.364249713 container start 4ee2e5b1123e71e4f81ecef74dc9a885d2e449179eaed5fd417f5f10e8b8b542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c, io.buildah.version=1.37.1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:09:20.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:19 smithi032 ceph-mon[17252]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:20.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:19 smithi073 ceph-mon[21995]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:20.240 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:09:19 smithi084 ceph-mds[54197]: -- 172.21.15.84:0/2910228074 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557159cb5860 con 0x55715de5ba80 2024-08-31T07:09:20.240 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:09:19 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c[54193]: starting mds.c at 2024-08-31T07:09:20.241 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:09:19 smithi084 bash[54178]: 4ee2e5b1123e71e4f81ecef74dc9a885d2e449179eaed5fd417f5f10e8b8b542 2024-08-31T07:09:20.241 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:09:19 smithi084 systemd[1]: Started Ceph mds.c for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:20.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:09:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:21.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:21.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:21.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:21.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:21.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:21.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:21.096 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:21 smithi032 ceph-mon[17252]: Deploying daemon mds.h on smithi073 2024-08-31T07:09:21.096 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:21 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] up:boot 2024-08-31T07:09:21.096 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:21 smithi032 ceph-mon[17252]: fsmap 7 up:standby 2024-08-31T07:09:21.096 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T07:09:21.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.390+0000 7f3f3ca00640 1 -- 172.21.15.32:0/4206006129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f38109090 msgr2=0x7f3f381094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:21.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.390+0000 7f3f3ca00640 1 --2- 172.21.15.32:0/4206006129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f38109090 0x7f3f381094a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009900 tx=0x7f3f2402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:09:21.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.390+0000 7f3f3ca00640 1 -- 172.21.15.32:0/4206006129 shutdown_connections 2024-08-31T07:09:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.390+0000 7f3f3ca00640 1 --2- 172.21.15.32:0/4206006129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f381099e0 0x7f3f38110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.390+0000 7f3f3ca00640 1 --2- 172.21.15.32:0/4206006129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f38109090 0x7f3f381094a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.390+0000 7f3f3ca00640 1 --2- 172.21.15.32:0/4206006129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f38103750 0x7f3f38103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.390+0000 7f3f3ca00640 1 -- 172.21.15.32:0/4206006129 >> 172.21.15.32:0/4206006129 conn(0x7f3f380ff1e0 msgr2=0x7f3f38101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.390+0000 7f3f3ca00640 1 -- 172.21.15.32:0/4206006129 shutdown_connections 2024-08-31T07:09:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.390+0000 7f3f3ca00640 1 -- 172.21.15.32:0/4206006129 wait complete. 2024-08-31T07:09:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.391+0000 7f3f3ca00640 1 Processor -- start 2024-08-31T07:09:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.391+0000 7f3f3ca00640 1 -- start start 2024-08-31T07:09:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f3ca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f38103750 0x7f3f3819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f3ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f38109090 0x7f3f3819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f3ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f381099e0 0x7f3f381a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f3ca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3f38113650 con 0x7f3f38103750 2024-08-31T07:09:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f3ca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3f381134d0 con 0x7f3f38109090 2024-08-31T07:09:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f3ca00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3f381137d0 con 0x7f3f381099e0 2024-08-31T07:09:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f37400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f38103750 0x7f3f3819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:21.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f36a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f38109090 0x7f3f3819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:21.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f37400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f38103750 0x7f3f3819c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37368/0 (socket says 172.21.15.32:37368) 2024-08-31T07:09:21.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f37e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f381099e0 0x7f3f381a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:21.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.392+0000 7f3f37400640 1 -- 172.21.15.32:0/2651945167 learned_addr learned my addr 172.21.15.32:0/2651945167 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:21.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f36a00640 1 -- 172.21.15.32:0/2651945167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f381099e0 msgr2=0x7f3f381a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:21.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f36a00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f381099e0 0x7f3f381a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:21.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f36a00640 1 -- 172.21.15.32:0/2651945167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f38103750 msgr2=0x7f3f3819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:21.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f36a00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f38103750 0x7f3f3819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:21.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f36a00640 1 -- 172.21.15.32:0/2651945167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f381a2250 con 0x7f3f38109090 2024-08-31T07:09:21.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f37400640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f38103750 0x7f3f3819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:21.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f37e00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f381099e0 0x7f3f381a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:21.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f36a00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f38109090 0x7f3f3819c880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f2402a910 tx=0x7f3f24004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:21.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f2fe00640 1 -- 172.21.15.32:0/2651945167 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2403f070 con 0x7f3f38109090 2024-08-31T07:09:21.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f3ca00640 1 -- 172.21.15.32:0/2651945167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f381a24e0 con 0x7f3f38109090 2024-08-31T07:09:21.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f2fe00640 1 -- 172.21.15.32:0/2651945167 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f24003ca0 con 0x7f3f38109090 2024-08-31T07:09:21.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f2fe00640 1 -- 172.21.15.32:0/2651945167 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24035070 con 0x7f3f38109090 2024-08-31T07:09:21.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.393+0000 7f3f3ca00640 1 -- 172.21.15.32:0/2651945167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f381a8ca0 con 0x7f3f38109090 2024-08-31T07:09:21.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.394+0000 7f3f3ca00640 1 -- 172.21.15.32:0/2651945167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f38103b60 con 0x7f3f38109090 2024-08-31T07:09:21.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.397+0000 7f3f2fe00640 1 -- 172.21.15.32:0/2651945167 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f3f2403b460 con 0x7f3f38109090 2024-08-31T07:09:21.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.398+0000 7f3f2fe00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3f1c0b4b80 0x7f3f1c0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:21.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.398+0000 7f3f2fe00640 1 -- 172.21.15.32:0/2651945167 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f3f24105170 con 0x7f3f38109090 2024-08-31T07:09:21.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.398+0000 7f3f37400640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3f1c0b4b80 0x7f3f1c0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:21.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.399+0000 7f3f37400640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3f1c0b4b80 0x7f3f1c0b7090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3f28005fd0 tx=0x7f3f2800b480 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:21.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.400+0000 7f3f2fe00640 1 -- 172.21.15.32:0/2651945167 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3f240cbf80 con 0x7f3f38109090 2024-08-31T07:09:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:21 smithi073 ceph-mon[21995]: Deploying daemon mds.h on smithi073 2024-08-31T07:09:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:21 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] up:boot 2024-08-31T07:09:21.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:21 smithi073 ceph-mon[21995]: fsmap 7 up:standby 2024-08-31T07:09:21.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T07:09:21.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:21.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:21.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:21.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:21.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:21.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:21.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:21 smithi084 ceph-mon[22387]: Deploying daemon mds.h on smithi073 2024-08-31T07:09:21.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:21 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] up:boot 2024-08-31T07:09:21.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:21 smithi084 ceph-mon[22387]: fsmap 7 up:standby 2024-08-31T07:09:21.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T07:09:21.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.632+0000 7f3f3ca00640 1 -- 172.21.15.32:0/2651945167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f381a8e70 con 0x7f3f38109090 2024-08-31T07:09:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.633+0000 7f3f2fe00640 1 -- 172.21.15.32:0/2651945167 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6382 (secure 0 0 0) 0x7f3f240d0740 con 0x7f3f38109090 2024-08-31T07:09:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:21.634 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"btime":"2024-08-31T07:09:20:839214+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-08-31T07:09:21.634 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-08-31T07:09:21.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.635+0000 7f3f3ca00640 1 -- 172.21.15.32:0/2651945167 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3f1c0b4b80 msgr2=0x7f3f1c0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:21.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.635+0000 7f3f3ca00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3f1c0b4b80 0x7f3f1c0b7090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3f28005fd0 tx=0x7f3f2800b480 comp rx=0 tx=0).stop 2024-08-31T07:09:21.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.635+0000 7f3f3ca00640 1 -- 172.21.15.32:0/2651945167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f38109090 msgr2=0x7f3f3819c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:21.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.635+0000 7f3f3ca00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f38109090 0x7f3f3819c880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f2402a910 tx=0x7f3f24004060 comp rx=0 tx=0).stop 2024-08-31T07:09:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.635+0000 7f3f3ca00640 1 -- 172.21.15.32:0/2651945167 shutdown_connections 2024-08-31T07:09:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.635+0000 7f3f3ca00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3f1c0b4b80 0x7f3f1c0b7090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.635+0000 7f3f3ca00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f381099e0 0x7f3f381a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.635+0000 7f3f3ca00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f38109090 0x7f3f3819c880 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.635+0000 7f3f3ca00640 1 --2- 172.21.15.32:0/2651945167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f38103750 0x7f3f3819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.635+0000 7f3f3ca00640 1 -- 172.21.15.32:0/2651945167 >> 172.21.15.32:0/2651945167 conn(0x7f3f380ff1e0 msgr2=0x7f3f38108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.636+0000 7f3f3ca00640 1 -- 172.21.15.32:0/2651945167 shutdown_connections 2024-08-31T07:09:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:21.636+0000 7f3f3ca00640 1 -- 172.21.15.32:0/2651945167 wait complete. 2024-08-31T07:09:22.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:22 smithi073 ceph-mon[21995]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:22.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:22 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2651945167' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:22.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:22 smithi032 ceph-mon[17252]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:22 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2651945167' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:22.473 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:22 smithi084 ceph-mon[22387]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:22.473 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:22 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2651945167' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:22.929 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:09:22 smithi073 systemd[1]: Started Ceph mds.h for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: Deploying daemon mds.i on smithi084 2024-08-31T07:09:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] up:boot 2024-08-31T07:09:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: fsmap 8 up:standby 2024-08-31T07:09:23.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T07:09:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:23.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: Deploying daemon mds.i on smithi084 2024-08-31T07:09:23.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:23.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] up:boot 2024-08-31T07:09:23.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: fsmap 8 up:standby 2024-08-31T07:09:23.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T07:09:23.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:23.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:23.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: Deploying daemon mds.i on smithi084 2024-08-31T07:09:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] up:boot 2024-08-31T07:09:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: fsmap 8 up:standby 2024-08-31T07:09:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T07:09:24.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:09:24.740 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:09:24 smithi084 systemd[1]: Starting Ceph mds.i for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:09:24.869 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:25.053 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:09:24 smithi084 podman[54489]: 2024-08-31 07:09:24.686461262 +0000 UTC m=+0.016400884 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:09:25.053 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:09:24 smithi084 podman[54489]: 2024-08-31 07:09:24.817213687 +0000 UTC m=+0.147153337 container create 6e86158acbed2772ef857f643e384d3a76bb837367c66292d98d37d7f93c7fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-31T07:09:25.053 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:09:25 smithi084 podman[54489]: 2024-08-31 07:09:25.053380194 +0000 UTC m=+0.383319811 container init 6e86158acbed2772ef857f643e384d3a76bb837367c66292d98d37d7f93c7fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:09:25.490 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:09:25 smithi084 podman[54489]: 2024-08-31 07:09:25.056250235 +0000 UTC m=+0.386189846 container start 6e86158acbed2772ef857f643e384d3a76bb837367c66292d98d37d7f93c7fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-31T07:09:25.491 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:09:25 smithi084 ceph-mds[54508]: -- 172.21.15.84:0/2309216121 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563afa223860 con 0x563afe3c9a80 2024-08-31T07:09:25.491 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:09:25 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i[54504]: starting mds.i at 2024-08-31T07:09:25.491 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:09:25 smithi084 bash[54489]: 6e86158acbed2772ef857f643e384d3a76bb837367c66292d98d37d7f93c7fce 2024-08-31T07:09:25.491 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:09:25 smithi084 systemd[1]: Started Ceph mds.i for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.516+0000 7fef5f000640 1 -- 172.21.15.32:0/4153768880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef58075ea0 msgr2=0x7fef5810f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.516+0000 7fef5f000640 1 --2- 172.21.15.32:0/4153768880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef58075ea0 0x7fef5810f770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fef44009900 tx=0x7fef4402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:09:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.516+0000 7fef5f000640 1 -- 172.21.15.32:0/4153768880 shutdown_connections 2024-08-31T07:09:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.516+0000 7fef5f000640 1 --2- 172.21.15.32:0/4153768880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5810fcb0 0x7fef581120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.516+0000 7fef5f000640 1 --2- 172.21.15.32:0/4153768880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef58075ea0 0x7fef5810f770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.516+0000 7fef5f000640 1 --2- 172.21.15.32:0/4153768880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef58075550 0x7fef58075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.516+0000 7fef5f000640 1 -- 172.21.15.32:0/4153768880 >> 172.21.15.32:0/4153768880 conn(0x7fef580ff1e0 msgr2=0x7fef58101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.516+0000 7fef5f000640 1 -- 172.21.15.32:0/4153768880 shutdown_connections 2024-08-31T07:09:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.516+0000 7fef5f000640 1 -- 172.21.15.32:0/4153768880 wait complete. 2024-08-31T07:09:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.517+0000 7fef5f000640 1 Processor -- start 2024-08-31T07:09:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.517+0000 7fef5f000640 1 -- start start 2024-08-31T07:09:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5f000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef58075550 0x7fef5819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef58075ea0 0x7fef5819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5f000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5810fcb0 0x7fef581a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5f000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef581158b0 con 0x7fef58075550 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fef58115730 con 0x7fef58075ea0 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5f000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fef58115a30 con 0x7fef5810fcb0 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef58075550 0x7fef5819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef58075550 0x7fef5819c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37384/0 (socket says 172.21.15.32:37384) 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5dc00640 1 -- 172.21.15.32:0/3498865784 learned_addr learned my addr 172.21.15.32:0/3498865784 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5e600640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5810fcb0 0x7fef581a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5d200640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef58075ea0 0x7fef5819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5e600640 1 -- 172.21.15.32:0/3498865784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef58075ea0 msgr2=0x7fef5819c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5e600640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef58075ea0 0x7fef5819c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5e600640 1 -- 172.21.15.32:0/3498865784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef58075550 msgr2=0x7fef5819c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:25.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5e600640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef58075550 0x7fef5819c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:25.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.518+0000 7fef5e600640 1 -- 172.21.15.32:0/3498865784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef581a2210 con 0x7fef5810fcb0 2024-08-31T07:09:25.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.519+0000 7fef5dc00640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef58075550 0x7fef5819c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:25.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.519+0000 7fef5e600640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5810fcb0 0x7fef581a1ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef50009fd0 tx=0x7fef5000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.519+0000 7fef56000640 1 -- 172.21.15.32:0/3498865784 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5000ce20 con 0x7fef5810fcb0 2024-08-31T07:09:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.519+0000 7fef56000640 1 -- 172.21.15.32:0/3498865784 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef50005020 con 0x7fef5810fcb0 2024-08-31T07:09:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.519+0000 7fef56000640 1 -- 172.21.15.32:0/3498865784 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50010690 con 0x7fef5810fcb0 2024-08-31T07:09:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.519+0000 7fef5f000640 1 -- 172.21.15.32:0/3498865784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef581a2500 con 0x7fef5810fcb0 2024-08-31T07:09:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.519+0000 7fef5f000640 1 -- 172.21.15.32:0/3498865784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef581a8c60 con 0x7fef5810fcb0 2024-08-31T07:09:25.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.520+0000 7fef5f000640 1 -- 172.21.15.32:0/3498865784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef58075960 con 0x7fef5810fcb0 2024-08-31T07:09:25.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.523+0000 7fef56000640 1 -- 172.21.15.32:0/3498865784 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fef500054d0 con 0x7fef5810fcb0 2024-08-31T07:09:25.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.524+0000 7fef56000640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef400b4c00 0x7fef400b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:25.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.524+0000 7fef56000640 1 -- 172.21.15.32:0/3498865784 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fef500e5480 con 0x7fef5810fcb0 2024-08-31T07:09:25.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.524+0000 7fef5dc00640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef400b4c00 0x7fef400b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:25.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.525+0000 7fef5dc00640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef400b4c00 0x7fef400b7110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0095f0 tx=0x7fef4c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:25.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.528+0000 7fef56000640 1 -- 172.21.15.32:0/3498865784 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fef500ac8e0 con 0x7fef5810fcb0 2024-08-31T07:09:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.771+0000 7fef5f000640 1 -- 172.21.15.32:0/3498865784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef5819d3d0 con 0x7fef5810fcb0 2024-08-31T07:09:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.772+0000 7fef56000640 1 -- 172.21.15.32:0/3498865784 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7207 (secure 0 0 0) 0x7fef500ac1e0 con 0x7fef5810fcb0 2024-08-31T07:09:25.773 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:25.773 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"btime":"2024-08-31T07:09:23:063922+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-31T07:09:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2024-08-31T07:09:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 -- 172.21.15.32:0/3498865784 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef400b4c00 msgr2=0x7fef400b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef400b4c00 0x7fef400b7110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0095f0 tx=0x7fef4c005f90 comp rx=0 tx=0).stop 2024-08-31T07:09:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 -- 172.21.15.32:0/3498865784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5810fcb0 msgr2=0x7fef581a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5810fcb0 0x7fef581a1ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef50009fd0 tx=0x7fef5000ef10 comp rx=0 tx=0).stop 2024-08-31T07:09:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 -- 172.21.15.32:0/3498865784 shutdown_connections 2024-08-31T07:09:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef400b4c00 0x7fef400b7110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5810fcb0 0x7fef581a1ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef58075ea0 0x7fef5819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 --2- 172.21.15.32:0/3498865784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef58075550 0x7fef5819c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 -- 172.21.15.32:0/3498865784 >> 172.21.15.32:0/3498865784 conn(0x7fef580ff1e0 msgr2=0x7fef58109830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 -- 172.21.15.32:0/3498865784 shutdown_connections 2024-08-31T07:09:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:25.775+0000 7fef5f000640 1 -- 172.21.15.32:0/3498865784 wait complete. 2024-08-31T07:09:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: Deploying daemon mds.g on smithi032 2024-08-31T07:09:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3498865784' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] up:boot 2024-08-31T07:09:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: fsmap 9 up:standby 2024-08-31T07:09:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T07:09:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:26.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:26.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:26.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: Deploying daemon mds.g on smithi032 2024-08-31T07:09:26.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3498865784' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:26.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] up:boot 2024-08-31T07:09:26.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: fsmap 9 up:standby 2024-08-31T07:09:26.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T07:09:26.632 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:26.632 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:26.632 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:26.632 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:26.632 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:26.632 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:26.632 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:26.633 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: Deploying daemon mds.g on smithi032 2024-08-31T07:09:26.633 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3498865784' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:26.633 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] up:boot 2024-08-31T07:09:26.633 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: fsmap 9 up:standby 2024-08-31T07:09:26.633 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T07:09:28.346 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:28 smithi032 ceph-mon[17252]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:28.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:28 smithi084 ceph-mon[22387]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:28.608 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:09:28 smithi032 systemd[1]: Starting Ceph mds.g for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:09:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:28 smithi073 ceph-mon[21995]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:28.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:09:28.981 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:09:28 smithi032 podman[61490]: 2024-08-31 07:09:28.565106914 +0000 UTC m=+0.028494503 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:09:28.981 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:09:28 smithi032 podman[61490]: 2024-08-31 07:09:28.727674225 +0000 UTC m=+0.191061813 container create 972d2b5c098a89d64f3a3d2f2e782399fadf7b6cfb5b90bc3caee0f27f1e4c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:09:28.981 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:09:28 smithi032 podman[61490]: 2024-08-31 07:09:28.98060585 +0000 UTC m=+0.443993439 container init 972d2b5c098a89d64f3a3d2f2e782399fadf7b6cfb5b90bc3caee0f27f1e4c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:09:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:29.358 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:09:28 smithi032 podman[61490]: 2024-08-31 07:09:28.987085691 +0000 UTC m=+0.450473279 container start 972d2b5c098a89d64f3a3d2f2e782399fadf7b6cfb5b90bc3caee0f27f1e4c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-31T07:09:29.358 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:09:29 smithi032 ceph-mds[61529]: -- 172.21.15.32:0/145861789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c3772ed860 con 0x55c37b493600 2024-08-31T07:09:29.358 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:09:29 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-g[61523]: starting mds.g at 2024-08-31T07:09:29.358 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:09:29 smithi032 bash[61490]: 972d2b5c098a89d64f3a3d2f2e782399fadf7b6cfb5b90bc3caee0f27f1e4c6d 2024-08-31T07:09:29.359 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:09:29 smithi032 systemd[1]: Started Ceph mds.g for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:29.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.967+0000 7f8a43000640 1 -- 172.21.15.32:0/3391074192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3c105b20 msgr2=0x7f8a3c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:29.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.967+0000 7f8a43000640 1 --2- 172.21.15.32:0/3391074192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3c105b20 0x7f8a3c105f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009900 tx=0x7f8a2c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:09:29.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.968+0000 7f8a43000640 1 -- 172.21.15.32:0/3391074192 shutdown_connections 2024-08-31T07:09:29.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.968+0000 7f8a43000640 1 --2- 172.21.15.32:0/3391074192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c10bdb0 0x7f8a3c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:29.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.968+0000 7f8a43000640 1 --2- 172.21.15.32:0/3391074192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a3c10b460 0x7f8a3c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:29.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.968+0000 7f8a43000640 1 --2- 172.21.15.32:0/3391074192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3c105b20 0x7f8a3c105f30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:29.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.968+0000 7f8a43000640 1 -- 172.21.15.32:0/3391074192 >> 172.21.15.32:0/3391074192 conn(0x7f8a3c1015b0 msgr2=0x7f8a3c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:29.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.968+0000 7f8a43000640 1 -- 172.21.15.32:0/3391074192 shutdown_connections 2024-08-31T07:09:29.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.968+0000 7f8a43000640 1 -- 172.21.15.32:0/3391074192 wait complete. 2024-08-31T07:09:29.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.969+0000 7f8a43000640 1 Processor -- start 2024-08-31T07:09:29.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.969+0000 7f8a43000640 1 -- start start 2024-08-31T07:09:29.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a43000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3c105b20 0x7f8a3c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:29.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a43000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a3c10b460 0x7f8a3c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:29.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a43000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c10bdb0 0x7f8a3c1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:29.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a43000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c115ac0 con 0x7f8a3c105b20 2024-08-31T07:09:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a43000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c115940 con 0x7f8a3c10bdb0 2024-08-31T07:09:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a43000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c115c40 con 0x7f8a3c10b460 2024-08-31T07:09:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3c105b20 0x7f8a3c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3c105b20 0x7f8a3c19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42384/0 (socket says 172.21.15.32:42384) 2024-08-31T07:09:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a3be00640 1 -- 172.21.15.32:0/2505267316 learned_addr learned my addr 172.21.15.32:0/2505267316 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a33e00640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a3c10b460 0x7f8a3c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.970+0000 7f8a41000640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c10bdb0 0x7f8a3c1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.971+0000 7f8a41000640 1 -- 172.21.15.32:0/2505267316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a3c10b460 msgr2=0x7f8a3c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.971+0000 7f8a41000640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a3c10b460 0x7f8a3c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.971+0000 7f8a41000640 1 -- 172.21.15.32:0/2505267316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3c105b20 msgr2=0x7f8a3c19c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.971+0000 7f8a41000640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3c105b20 0x7f8a3c19c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.971+0000 7f8a41000640 1 -- 172.21.15.32:0/2505267316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c1a2400 con 0x7f8a3c10bdb0 2024-08-31T07:09:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.971+0000 7f8a3be00640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3c105b20 0x7f8a3c19c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.971+0000 7f8a33e00640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a3c10b460 0x7f8a3c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:29.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.972+0000 7f8a41000640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c10bdb0 0x7f8a3c1a1cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009fd0 tx=0x7f8a3400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:29.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.972+0000 7f8a39600640 1 -- 172.21.15.32:0/2505267316 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400ce20 con 0x7f8a3c10bdb0 2024-08-31T07:09:29.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.972+0000 7f8a39600640 1 -- 172.21.15.32:0/2505267316 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a34005020 con 0x7f8a3c10bdb0 2024-08-31T07:09:29.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.972+0000 7f8a39600640 1 -- 172.21.15.32:0/2505267316 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34010690 con 0x7f8a3c10bdb0 2024-08-31T07:09:29.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.972+0000 7f8a43000640 1 -- 172.21.15.32:0/2505267316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c1a26f0 con 0x7f8a3c10bdb0 2024-08-31T07:09:29.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.972+0000 7f8a43000640 1 -- 172.21.15.32:0/2505267316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c1a8e50 con 0x7f8a3c10bdb0 2024-08-31T07:09:29.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.973+0000 7f8a43000640 1 -- 172.21.15.32:0/2505267316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3c105f30 con 0x7f8a3c10bdb0 2024-08-31T07:09:29.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.976+0000 7f8a39600640 1 -- 172.21.15.32:0/2505267316 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f8a340054d0 con 0x7f8a3c10bdb0 2024-08-31T07:09:29.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.977+0000 7f8a39600640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8a200b4c00 0x7f8a200b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:29.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.977+0000 7f8a39600640 1 -- 172.21.15.32:0/2505267316 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f8a340e5460 con 0x7f8a3c10bdb0 2024-08-31T07:09:29.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.977+0000 7f8a3be00640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8a200b4c00 0x7f8a200b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:29.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.978+0000 7f8a3be00640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8a200b4c00 0x7f8a200b7110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0096c0 tx=0x7f8a2c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:29.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:29.979+0000 7f8a39600640 1 -- 172.21.15.32:0/2505267316 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8a340ac8c0 con 0x7f8a3c10bdb0 2024-08-31T07:09:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.221+0000 7f8a43000640 1 -- 172.21.15.32:0/2505267316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a3c19d5c0 con 0x7f8a3c10bdb0 2024-08-31T07:09:30.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.222+0000 7f8a39600640 1 -- 172.21.15.32:0/2505267316 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8860 (secure 0 0 0) 0x7f8a3c19d5c0 con 0x7f8a3c10bdb0 2024-08-31T07:09:30.223 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:30.223 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":11,"btime":"2024-08-31T07:09:29:188653+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24694,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-31T07:09:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 11 2024-08-31T07:09:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 -- 172.21.15.32:0/2505267316 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8a200b4c00 msgr2=0x7f8a200b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8a200b4c00 0x7f8a200b7110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0096c0 tx=0x7f8a2c005a20 comp rx=0 tx=0).stop 2024-08-31T07:09:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 -- 172.21.15.32:0/2505267316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c10bdb0 msgr2=0x7f8a3c1a1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c10bdb0 0x7f8a3c1a1cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009fd0 tx=0x7f8a3400ef10 comp rx=0 tx=0).stop 2024-08-31T07:09:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 -- 172.21.15.32:0/2505267316 shutdown_connections 2024-08-31T07:09:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8a200b4c00 0x7f8a200b7110 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a3c10bdb0 0x7f8a3c1a1cd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a3c10b460 0x7f8a3c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 --2- 172.21.15.32:0/2505267316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a3c105b20 0x7f8a3c19c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 -- 172.21.15.32:0/2505267316 >> 172.21.15.32:0/2505267316 conn(0x7f8a3c1015b0 msgr2=0x7f8a3c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 -- 172.21.15.32:0/2505267316 shutdown_connections 2024-08-31T07:09:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:30.225+0000 7f8a43000640 1 -- 172.21.15.32:0/2505267316 wait complete. 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.32:6838/1062605926,v1:172.21.15.32:6839/1062605926] up:boot 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: fsmap 10 up:standby 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:30 smithi032 ceph-mon[17252]: Deploying daemon mds.a on smithi032 2024-08-31T07:09:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.32:6838/1062605926,v1:172.21.15.32:6839/1062605926] up:boot 2024-08-31T07:09:30.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: fsmap 10 up:standby 2024-08-31T07:09:30.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-31T07:09:30.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:30.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:30.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:30.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:30.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:30.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:30.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:30 smithi084 ceph-mon[22387]: Deploying daemon mds.a on smithi032 2024-08-31T07:09:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.32:6838/1062605926,v1:172.21.15.32:6839/1062605926] up:boot 2024-08-31T07:09:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: fsmap 10 up:standby 2024-08-31T07:09:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-31T07:09:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:30.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:30.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:30.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:30.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:30.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:30.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:30 smithi073 ceph-mon[21995]: Deploying daemon mds.a on smithi032 2024-08-31T07:09:31.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:31 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2505267316' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:31.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:31 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2505267316' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:31.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:31 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2505267316' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:32.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:32 smithi084 ceph-mon[22387]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:32.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:32 smithi032 ceph-mon[17252]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:32 smithi073 ceph-mon[21995]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:32.858 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:09:32 smithi032 systemd[1]: Starting Ceph mds.a for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:09:33.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:09:33.221 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:09:32 smithi032 podman[61902]: 2024-08-31 07:09:32.824293123 +0000 UTC m=+0.028097169 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:09:33.221 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:09:32 smithi032 podman[61902]: 2024-08-31 07:09:32.965178715 +0000 UTC m=+0.168982737 container create 080c3be7d34ff010755491a9f969acf132a074746824f9d5941b71cc10ad7bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-31T07:09:33.222 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:09:33 smithi032 podman[61902]: 2024-08-31 07:09:33.220895334 +0000 UTC m=+0.424699388 container init 080c3be7d34ff010755491a9f969acf132a074746824f9d5941b71cc10ad7bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-31T07:09:33.521 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:33.536 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:09:33 smithi032 podman[61902]: 2024-08-31 07:09:33.228170704 +0000 UTC m=+0.431974730 container start 080c3be7d34ff010755491a9f969acf132a074746824f9d5941b71cc10ad7bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD) 2024-08-31T07:09:33.536 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:09:33 smithi032 ceph-mds[61941]: -- 172.21.15.32:0/3756585249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab8d1b3860 con 0x55ab91359180 2024-08-31T07:09:33.536 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:09:33 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a[61919]: starting mds.a at 2024-08-31T07:09:33.536 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:09:33 smithi032 bash[61902]: 080c3be7d34ff010755491a9f969acf132a074746824f9d5941b71cc10ad7bde 2024-08-31T07:09:33.536 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:09:33 smithi032 systemd[1]: Started Ceph mds.a for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.170+0000 7f0dfa800640 1 -- 172.21.15.32:0/2086556927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df410bbe0 msgr2=0x7f0df4112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.170+0000 7f0dfa800640 1 --2- 172.21.15.32:0/2086556927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df410bbe0 0x7f0df4112410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009ca0 tx=0x7f0dec02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:09:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.171+0000 7f0dfa800640 1 -- 172.21.15.32:0/2086556927 shutdown_connections 2024-08-31T07:09:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.171+0000 7f0dfa800640 1 --2- 172.21.15.32:0/2086556927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df410bbe0 0x7f0df4112410 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.171+0000 7f0dfa800640 1 --2- 172.21.15.32:0/2086556927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df410b290 0x7f0df410b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.171+0000 7f0dfa800640 1 --2- 172.21.15.32:0/2086556927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df4105950 0x7f0df4105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.171+0000 7f0dfa800640 1 -- 172.21.15.32:0/2086556927 >> 172.21.15.32:0/2086556927 conn(0x7f0df4101420 msgr2=0x7f0df4103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.171+0000 7f0dfa800640 1 -- 172.21.15.32:0/2086556927 shutdown_connections 2024-08-31T07:09:34.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.171+0000 7f0dfa800640 1 -- 172.21.15.32:0/2086556927 wait complete. 2024-08-31T07:09:34.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.172+0000 7f0dfa800640 1 Processor -- start 2024-08-31T07:09:34.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.172+0000 7f0dfa800640 1 -- start start 2024-08-31T07:09:34.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.172+0000 7f0dfa800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4105950 0x7f0df419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:34.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.172+0000 7f0dfa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df410b290 0x7f0df419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:34.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.172+0000 7f0dfa800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df410bbe0 0x7f0df41a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:34.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.172+0000 7f0dfa800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41158a0 con 0x7f0df410bbe0 2024-08-31T07:09:34.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.172+0000 7f0dfa800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4115720 con 0x7f0df410b290 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.172+0000 7f0dfa800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4115a20 con 0x7f0df4105950 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df9400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4105950 0x7f0df419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df9400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4105950 0x7f0df419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52882/0 (socket says 172.21.15.32:52882) 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df9400640 1 -- 172.21.15.32:0/1273249039 learned_addr learned my addr 172.21.15.32:0/1273249039 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df8a00640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df410b290 0x7f0df419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df9e00640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df410bbe0 0x7f0df41a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df9400640 1 -- 172.21.15.32:0/1273249039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df410b290 msgr2=0x7f0df419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df9400640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df410b290 0x7f0df419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df9400640 1 -- 172.21.15.32:0/1273249039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df410bbe0 msgr2=0x7f0df41a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df9400640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df410bbe0 0x7f0df41a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df9400640 1 -- 172.21.15.32:0/1273249039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df41a21c0 con 0x7f0df4105950 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.173+0000 7f0df8a00640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df410b290 0x7f0df419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.174+0000 7f0df9400640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4105950 0x7f0df419c250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0de800ea60 tx=0x7f0de800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:34.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.174+0000 7f0df2000640 1 -- 172.21.15.32:0/1273249039 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de800cde0 con 0x7f0df4105950 2024-08-31T07:09:34.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.174+0000 7f0df2000640 1 -- 172.21.15.32:0/1273249039 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de80035f0 con 0x7f0df4105950 2024-08-31T07:09:34.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.174+0000 7f0df2000640 1 -- 172.21.15.32:0/1273249039 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8010690 con 0x7f0df4105950 2024-08-31T07:09:34.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.174+0000 7f0dfa800640 1 -- 172.21.15.32:0/1273249039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df41a24b0 con 0x7f0df4105950 2024-08-31T07:09:34.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.174+0000 7f0dfa800640 1 -- 172.21.15.32:0/1273249039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df41a8c10 con 0x7f0df4105950 2024-08-31T07:09:34.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.175+0000 7f0dfa800640 1 -- 172.21.15.32:0/1273249039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df4105d60 con 0x7f0df4105950 2024-08-31T07:09:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.178+0000 7f0df2000640 1 -- 172.21.15.32:0/1273249039 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f0de80037a0 con 0x7f0df4105950 2024-08-31T07:09:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.179+0000 7f0df2000640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ddc0b4b80 0x7f0ddc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.179+0000 7f0df2000640 1 -- 172.21.15.32:0/1273249039 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f0de80e5350 con 0x7f0df4105950 2024-08-31T07:09:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.180+0000 7f0df8a00640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ddc0b4b80 0x7f0ddc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.180+0000 7f0df8a00640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ddc0b4b80 0x7f0ddc0b7090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0df4104d00 tx=0x7f0de0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.181+0000 7f0df2000640 1 -- 172.21.15.32:0/1273249039 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0de8056130 con 0x7f0df4105950 2024-08-31T07:09:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.415+0000 7f0dfa800640 1 -- 172.21.15.32:0/1273249039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0df419d380 con 0x7f0df4105950 2024-08-31T07:09:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.416+0000 7f0df2000640 1 -- 172.21.15.32:0/1273249039 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9686 (secure 0 0 0) 0x7f0de80ac7b0 con 0x7f0df4105950 2024-08-31T07:09:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:34.418 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"btime":"2024-08-31T07:09:34:212195+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24694,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-08-31T07:09:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2024-08-31T07:09:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 -- 172.21.15.32:0/1273249039 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ddc0b4b80 msgr2=0x7f0ddc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ddc0b4b80 0x7f0ddc0b7090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0df4104d00 tx=0x7f0de0005f90 comp rx=0 tx=0).stop 2024-08-31T07:09:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 -- 172.21.15.32:0/1273249039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4105950 msgr2=0x7f0df419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4105950 0x7f0df419c250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0de800ea60 tx=0x7f0de800ef20 comp rx=0 tx=0).stop 2024-08-31T07:09:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 -- 172.21.15.32:0/1273249039 shutdown_connections 2024-08-31T07:09:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ddc0b4b80 0x7f0ddc0b7090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0df410bbe0 0x7f0df41a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df410b290 0x7f0df419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 --2- 172.21.15.32:0/1273249039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4105950 0x7f0df419c250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 -- 172.21.15.32:0/1273249039 >> 172.21.15.32:0/1273249039 conn(0x7f0df4101420 msgr2=0x7f0df410a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:34.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 -- 172.21.15.32:0/1273249039 shutdown_connections 2024-08-31T07:09:34.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:34.419+0000 7f0dfa800640 1 -- 172.21.15.32:0/1273249039 wait complete. 2024-08-31T07:09:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:34.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:34.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:34.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:34.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:34.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: Deploying daemon mds.b on smithi073 2024-08-31T07:09:34.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] up:boot 2024-08-31T07:09:34.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: fsmap 11 up:standby 2024-08-31T07:09:34.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-31T07:09:34.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:34 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1273249039' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: Deploying daemon mds.b on smithi073 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] up:boot 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: fsmap 11 up:standby 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-31T07:09:34.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:34 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1273249039' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:34.833 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: Deploying daemon mds.b on smithi073 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] up:boot 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: fsmap 11 up:standby 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-31T07:09:34.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:34 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1273249039' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:35.635 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:09:35 smithi073 systemd[1]: Starting Ceph mds.b for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:09:35.890 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:09:35 smithi073 podman[53154]: 2024-08-31 07:09:35.537387724 +0000 UTC m=+0.015826458 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:09:35.890 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:09:35 smithi073 podman[53154]: 2024-08-31 07:09:35.662509833 +0000 UTC m=+0.140948560 container create 44ccc555afa0678607b339df34128c3335005fc079c1b497c550483f953f8034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-31T07:09:35.890 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:09:35 smithi073 podman[53154]: 2024-08-31 07:09:35.890117484 +0000 UTC m=+0.368556211 container init 44ccc555afa0678607b339df34128c3335005fc079c1b497c550483f953f8034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-31T07:09:36.179 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:09:35 smithi073 podman[53154]: 2024-08-31 07:09:35.89315311 +0000 UTC m=+0.371591845 container start 44ccc555afa0678607b339df34128c3335005fc079c1b497c550483f953f8034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1) 2024-08-31T07:09:36.179 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:09:35 smithi073 ceph-mds[53173]: -- 172.21.15.73:0/2286337342 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55993f3b9860 con 0x55994355fa80 2024-08-31T07:09:36.179 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:09:35 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b[53169]: starting mds.b at 2024-08-31T07:09:36.179 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:09:35 smithi073 bash[53154]: 44ccc555afa0678607b339df34128c3335005fc079c1b497c550483f953f8034 2024-08-31T07:09:36.179 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:09:35 smithi073 systemd[1]: Started Ceph mds.b for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:09:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:36 smithi084 ceph-mon[22387]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:36.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:36 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:36 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:36 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:36 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.743 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:36 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.743 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:36 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:09:36.812 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:36 smithi073 ceph-mon[21995]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:36.812 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:36 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.812 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:36 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.813 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:36 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.813 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:36 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.813 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:36 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.813 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:36 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:09:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:36 smithi032 ceph-mon[17252]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:36 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:36 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:36 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:36 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:36 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:36 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:09:37.192 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:09:37.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:37 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] up:boot 2024-08-31T07:09:37.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:37 smithi073 ceph-mon[21995]: fsmap 12 up:standby 2024-08-31T07:09:37.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:37 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T07:09:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:37 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] up:boot 2024-08-31T07:09:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:37 smithi084 ceph-mon[22387]: fsmap 12 up:standby 2024-08-31T07:09:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:37 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T07:09:37.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:37 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] up:boot 2024-08-31T07:09:37.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:37 smithi032 ceph-mon[17252]: fsmap 12 up:standby 2024-08-31T07:09:37.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:37 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T07:09:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.375+0000 7fd65f400640 1 -- 172.21.15.32:0/1353687676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65810fe70 msgr2=0x7fd6581122b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.375+0000 7fd65f400640 1 --2- 172.21.15.32:0/1353687676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65810fe70 0x7fd6581122b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd650009ca0 tx=0x7fd65002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.375+0000 7fd65f400640 1 -- 172.21.15.32:0/1353687676 shutdown_connections 2024-08-31T07:09:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.375+0000 7fd65f400640 1 --2- 172.21.15.32:0/1353687676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65810fe70 0x7fd6581122b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.375+0000 7fd65f400640 1 --2- 172.21.15.32:0/1353687676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd658075ea0 0x7fd65810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.375+0000 7fd65f400640 1 --2- 172.21.15.32:0/1353687676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd658075550 0x7fd658075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.375+0000 7fd65f400640 1 -- 172.21.15.32:0/1353687676 >> 172.21.15.32:0/1353687676 conn(0x7fd6580ff380 msgr2=0x7fd6581017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.375+0000 7fd65f400640 1 -- 172.21.15.32:0/1353687676 shutdown_connections 2024-08-31T07:09:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.375+0000 7fd65f400640 1 -- 172.21.15.32:0/1353687676 wait complete. 2024-08-31T07:09:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.376+0000 7fd65f400640 1 Processor -- start 2024-08-31T07:09:38.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.376+0000 7fd65f400640 1 -- start start 2024-08-31T07:09:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd65f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd658075550 0x7fd65819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd65f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd658075ea0 0x7fd65819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd65f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65810fe70 0x7fd6581a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd65f400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd658115a70 con 0x7fd65810fe70 2024-08-31T07:09:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd65f400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581158f0 con 0x7fd658075550 2024-08-31T07:09:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd65f400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd658115bf0 con 0x7fd658075ea0 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd65ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd658075550 0x7fd65819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd65d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65810fe70 0x7fd6581a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd657e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd658075ea0 0x7fd65819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd657e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd658075ea0 0x7fd65819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57488/0 (socket says 172.21.15.32:57488) 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd65ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd658075550 0x7fd65819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47184/0 (socket says 172.21.15.32:47184) 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.377+0000 7fd657e00640 1 -- 172.21.15.32:0/89751547 learned_addr learned my addr 172.21.15.32:0/89751547 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.378+0000 7fd65ca00640 1 -- 172.21.15.32:0/89751547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd658075ea0 msgr2=0x7fd65819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.378+0000 7fd65ca00640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd658075ea0 0x7fd65819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.378+0000 7fd65ca00640 1 -- 172.21.15.32:0/89751547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65810fe70 msgr2=0x7fd6581a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.378+0000 7fd65ca00640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65810fe70 0x7fd6581a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.378+0000 7fd65ca00640 1 -- 172.21.15.32:0/89751547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6581a2370 con 0x7fd658075550 2024-08-31T07:09:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.378+0000 7fd65d400640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65810fe70 0x7fd6581a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:09:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.378+0000 7fd657e00640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd658075ea0 0x7fd65819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.378+0000 7fd65ca00640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd658075550 0x7fd65819c400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00e9f0 tx=0x7fd64c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.379+0000 7fd655600640 1 -- 172.21.15.32:0/89751547 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00ccc0 con 0x7fd658075550 2024-08-31T07:09:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.379+0000 7fd65f400640 1 -- 172.21.15.32:0/89751547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6581a2660 con 0x7fd658075550 2024-08-31T07:09:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.379+0000 7fd655600640 1 -- 172.21.15.32:0/89751547 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd64c005020 con 0x7fd658075550 2024-08-31T07:09:38.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.379+0000 7fd655600640 1 -- 172.21.15.32:0/89751547 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c004a00 con 0x7fd658075550 2024-08-31T07:09:38.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.379+0000 7fd65f400640 1 -- 172.21.15.32:0/89751547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6581a8dc0 con 0x7fd658075550 2024-08-31T07:09:38.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.380+0000 7fd65f400640 1 -- 172.21.15.32:0/89751547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd658075960 con 0x7fd658075550 2024-08-31T07:09:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.383+0000 7fd655600640 1 -- 172.21.15.32:0/89751547 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fd64c004ba0 con 0x7fd658075550 2024-08-31T07:09:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.384+0000 7fd655600640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd6400b4c00 0x7fd6400b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.384+0000 7fd655600640 1 -- 172.21.15.32:0/89751547 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fd64c0e4d70 con 0x7fd658075550 2024-08-31T07:09:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.384+0000 7fd657e00640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd6400b4c00 0x7fd6400b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:38.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.385+0000 7fd657e00640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd6400b4c00 0x7fd6400b7110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd658112070 tx=0x7fd64400b4a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.386+0000 7fd655600640 1 -- 172.21.15.32:0/89751547 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd64c0ac1d0 con 0x7fd658075550 2024-08-31T07:09:38.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.631+0000 7fd65f400640 1 -- 172.21.15.32:0/89751547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd65819d530 con 0x7fd658075550 2024-08-31T07:09:38.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.632+0000 7fd655600640 1 -- 172.21.15.32:0/89751547 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10509 (secure 0 0 0) 0x7fd64c0abad0 con 0x7fd658075550 2024-08-31T07:09:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:38.634 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"btime":"2024-08-31T07:09:36:446392+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24632,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24668,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24670,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24676,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24694,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24734,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-31T07:09:38.634 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-08-31T07:09:38.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.635+0000 7fd65f400640 1 -- 172.21.15.32:0/89751547 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd6400b4c00 msgr2=0x7fd6400b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:38.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.635+0000 7fd65f400640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd6400b4c00 0x7fd6400b7110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd658112070 tx=0x7fd64400b4a0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.635+0000 7fd65f400640 1 -- 172.21.15.32:0/89751547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd658075550 msgr2=0x7fd65819c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:38.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.635+0000 7fd65f400640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd658075550 0x7fd65819c400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00e9f0 tx=0x7fd64c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.635+0000 7fd65f400640 1 -- 172.21.15.32:0/89751547 shutdown_connections 2024-08-31T07:09:38.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.635+0000 7fd65f400640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd6400b4c00 0x7fd6400b7110 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.635+0000 7fd65f400640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65810fe70 0x7fd6581a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.635+0000 7fd65f400640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd658075ea0 0x7fd65819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.635+0000 7fd65f400640 1 --2- 172.21.15.32:0/89751547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd658075550 0x7fd65819c400 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:38.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.635+0000 7fd65f400640 1 -- 172.21.15.32:0/89751547 >> 172.21.15.32:0/89751547 conn(0x7fd6580ff380 msgr2=0x7fd658109980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:38.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.636+0000 7fd65f400640 1 -- 172.21.15.32:0/89751547 shutdown_connections 2024-08-31T07:09:38.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:38.636+0000 7fd65f400640 1 -- 172.21.15.32:0/89751547 wait complete. 2024-08-31T07:09:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:38 smithi084 ceph-mon[22387]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:38 smithi032 ceph-mon[17252]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:38.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:38 smithi073 ceph-mon[21995]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:39.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-31T07:09:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:39 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/89751547' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:09:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:09:39.747 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:39 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/89751547' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:39.748 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:39.748 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:39.748 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:39.748 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:09:39.748 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:39.748 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:09:39.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:39 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/89751547' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:09:39.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:39.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:39.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:39.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:09:39.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:39.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:09:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:40 smithi084 ceph-mon[22387]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:40 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:40.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:40 smithi032 ceph-mon[17252]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:40.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:40 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:40 smithi073 ceph-mon[21995]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:40 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.396+0000 7ff523a00640 1 -- 172.21.15.32:0/103509911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff51c077120 msgr2=0x7ff51c075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.396+0000 7ff523a00640 1 --2- 172.21.15.32:0/103509911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff51c077120 0x7ff51c075550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff51002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:09:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.396+0000 7ff523a00640 1 -- 172.21.15.32:0/103509911 shutdown_connections 2024-08-31T07:09:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.396+0000 7ff523a00640 1 --2- 172.21.15.32:0/103509911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff51c1141f0 0x7ff51c1166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.396+0000 7ff523a00640 1 --2- 172.21.15.32:0/103509911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c075b70 0x7ff51c075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.396+0000 7ff523a00640 1 --2- 172.21.15.32:0/103509911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff51c077120 0x7ff51c075550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.396+0000 7ff523a00640 1 -- 172.21.15.32:0/103509911 >> 172.21.15.32:0/103509911 conn(0x7ff51c0ff330 msgr2=0x7ff51c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.396+0000 7ff523a00640 1 -- 172.21.15.32:0/103509911 shutdown_connections 2024-08-31T07:09:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.396+0000 7ff523a00640 1 -- 172.21.15.32:0/103509911 wait complete. 2024-08-31T07:09:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.398+0000 7ff523a00640 1 Processor -- start 2024-08-31T07:09:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.398+0000 7ff523a00640 1 -- start start 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.398+0000 7ff523a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c075b70 0x7ff51c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.398+0000 7ff523a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff51c077120 0x7ff51c1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.398+0000 7ff523a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff51c1141f0 0x7ff51c1a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.398+0000 7ff523a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c119e30 con 0x7ff51c1141f0 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.398+0000 7ff523a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c119cb0 con 0x7ff51c077120 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.398+0000 7ff523a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c119fb0 con 0x7ff51c075b70 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff51be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff51c077120 0x7ff51c1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff521a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff51c1141f0 0x7ff51c1a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff51be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff51c077120 0x7ff51c1a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47212/0 (socket says 172.21.15.32:47212) 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff51be00640 1 -- 172.21.15.32:0/4071476298 learned_addr learned my addr 172.21.15.32:0/4071476298 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:41.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff521000640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c075b70 0x7ff51c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:41.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff51be00640 1 -- 172.21.15.32:0/4071476298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c075b70 msgr2=0x7ff51c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:41.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff51be00640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c075b70 0x7ff51c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:41.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff51be00640 1 -- 172.21.15.32:0/4071476298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff51c1141f0 msgr2=0x7ff51c1a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:41.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff51be00640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff51c1141f0 0x7ff51c1a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:41.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff51be00640 1 -- 172.21.15.32:0/4071476298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c1a67f0 con 0x7ff51c077120 2024-08-31T07:09:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.399+0000 7ff51be00640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff51c077120 0x7ff51c1a0e20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e9c0 tx=0x7ff50800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.400+0000 7ff519600640 1 -- 172.21.15.32:0/4071476298 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50800cd80 con 0x7ff51c077120 2024-08-31T07:09:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.400+0000 7ff519600640 1 -- 172.21.15.32:0/4071476298 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5080054d0 con 0x7ff51c077120 2024-08-31T07:09:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.400+0000 7ff519600640 1 -- 172.21.15.32:0/4071476298 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508010430 con 0x7ff51c077120 2024-08-31T07:09:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.400+0000 7ff523a00640 1 -- 172.21.15.32:0/4071476298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c1a6ae0 con 0x7ff51c077120 2024-08-31T07:09:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.400+0000 7ff523a00640 1 -- 172.21.15.32:0/4071476298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c1ad240 con 0x7ff51c077120 2024-08-31T07:09:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.401+0000 7ff523a00640 1 -- 172.21.15.32:0/4071476298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51c075f80 con 0x7ff51c077120 2024-08-31T07:09:41.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.404+0000 7ff519600640 1 -- 172.21.15.32:0/4071476298 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7ff508004fb0 con 0x7ff51c077120 2024-08-31T07:09:41.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.405+0000 7ff519600640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5040b4c00 0x7ff5040b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:41.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.406+0000 7ff519600640 1 -- 172.21.15.32:0/4071476298 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7ff5080e53f0 con 0x7ff51c077120 2024-08-31T07:09:41.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.406+0000 7ff521000640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5040b4c00 0x7ff5040b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:41.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.406+0000 7ff519600640 1 -- 172.21.15.32:0/4071476298 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff508056130 con 0x7ff51c077120 2024-08-31T07:09:41.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.407+0000 7ff521000640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5040b4c00 0x7ff5040b7110 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff510031440 tx=0x7ff510009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:41.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.626+0000 7ff523a00640 1 -- 172.21.15.32:0/4071476298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff51c112f80 con 0x7ff51c077120 2024-08-31T07:09:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.627+0000 7ff519600640 1 -- 172.21.15.32:0/4071476298 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7ff5080ac850 con 0x7ff51c077120 2024-08-31T07:09:41.628 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:41.628 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-08-31T07:09:41.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:09:41.628 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-08-31T07:09:41.628 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-08-31T07:09:41.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:09:41.629 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-08-31T07:09:41.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.629+0000 7ff523a00640 1 -- 172.21.15.32:0/4071476298 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5040b4c00 msgr2=0x7ff5040b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:41.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.629+0000 7ff523a00640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5040b4c00 0x7ff5040b7110 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff510031440 tx=0x7ff510009560 comp rx=0 tx=0).stop 2024-08-31T07:09:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.630+0000 7ff523a00640 1 -- 172.21.15.32:0/4071476298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff51c077120 msgr2=0x7ff51c1a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.630+0000 7ff523a00640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff51c077120 0x7ff51c1a0e20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e9c0 tx=0x7ff50800ee80 comp rx=0 tx=0).stop 2024-08-31T07:09:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.630+0000 7ff523a00640 1 -- 172.21.15.32:0/4071476298 shutdown_connections 2024-08-31T07:09:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.630+0000 7ff523a00640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5040b4c00 0x7ff5040b7110 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.630+0000 7ff523a00640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff51c1141f0 0x7ff51c1a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.630+0000 7ff523a00640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff51c077120 0x7ff51c1a0e20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.630+0000 7ff523a00640 1 --2- 172.21.15.32:0/4071476298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff51c075b70 0x7ff51c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.630+0000 7ff523a00640 1 -- 172.21.15.32:0/4071476298 >> 172.21.15.32:0/4071476298 conn(0x7ff51c0ff330 msgr2=0x7ff51c1100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.630+0000 7ff523a00640 1 -- 172.21.15.32:0/4071476298 shutdown_connections 2024-08-31T07:09:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:41.630+0000 7ff523a00640 1 -- 172.21.15.32:0/4071476298 wait complete. 2024-08-31T07:09:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:41.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:42.582 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-31T07:09:42.582 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-31T07:09:42.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:42 smithi032 ceph-mon[17252]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:42.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:42 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4071476298' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-31T07:09:42.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:42 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:42.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:42 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:42.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:42 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:42.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:42 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:09:42.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:42 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:42 smithi073 ceph-mon[21995]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:42 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4071476298' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-31T07:09:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:42 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:42 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:42 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:42 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:09:42.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:42 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:42 smithi084 ceph-mon[22387]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:42 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4071476298' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-31T07:09:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:42 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:42 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:42 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:09:42.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:42 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:09:42.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:42 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:09:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.575+0000 7fd908a00640 1 -- 172.21.15.32:0/941773930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd904109b90 msgr2=0x7fd9041103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.575+0000 7fd908a00640 1 --2- 172.21.15.32:0/941773930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd904109b90 0x7fd9041103c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009ca0 tx=0x7fd8fc02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:09:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.575+0000 7fd908a00640 1 -- 172.21.15.32:0/941773930 shutdown_connections 2024-08-31T07:09:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.575+0000 7fd908a00640 1 --2- 172.21.15.32:0/941773930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd904109b90 0x7fd9041103c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.575+0000 7fd908a00640 1 --2- 172.21.15.32:0/941773930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd904109240 0x7fd904109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.575+0000 7fd908a00640 1 --2- 172.21.15.32:0/941773930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd904103900 0x7fd904103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.575+0000 7fd908a00640 1 -- 172.21.15.32:0/941773930 >> 172.21.15.32:0/941773930 conn(0x7fd90407b5f0 msgr2=0x7fd90407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.575+0000 7fd908a00640 1 -- 172.21.15.32:0/941773930 shutdown_connections 2024-08-31T07:09:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.575+0000 7fd908a00640 1 -- 172.21.15.32:0/941773930 wait complete. 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.577+0000 7fd908a00640 1 Processor -- start 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.577+0000 7fd908a00640 1 -- start start 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.577+0000 7fd908a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd904103900 0x7fd90419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.577+0000 7fd908a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd904109240 0x7fd90419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.577+0000 7fd908a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd904109b90 0x7fd9041a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.577+0000 7fd908a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd904113850 con 0x7fd904109b90 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.577+0000 7fd908a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041136d0 con 0x7fd904103900 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.577+0000 7fd908a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041139d0 con 0x7fd904109240 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd901e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd904103900 0x7fd90419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd902800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd904109b90 0x7fd9041a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd901400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd904109240 0x7fd90419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd901e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd904103900 0x7fd90419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47230/0 (socket says 172.21.15.32:47230) 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd901e00640 1 -- 172.21.15.32:0/122250551 learned_addr learned my addr 172.21.15.32:0/122250551 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd901400640 1 -- 172.21.15.32:0/122250551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd904103900 msgr2=0x7fd90419c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd901400640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd904103900 0x7fd90419c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd901400640 1 -- 172.21.15.32:0/122250551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd904109b90 msgr2=0x7fd9041a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd901400640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd904109b90 0x7fd9041a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd901400640 1 -- 172.21.15.32:0/122250551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9041a23d0 con 0x7fd904109240 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.578+0000 7fd901400640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd904109240 0x7fd90419ca00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0005fd0 tx=0x7fd8f000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.579+0000 7fd8faa00640 1 -- 172.21.15.32:0/122250551 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0009c00 con 0x7fd904109240 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.579+0000 7fd8faa00640 1 -- 172.21.15.32:0/122250551 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8f0009da0 con 0x7fd904109240 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.579+0000 7fd908a00640 1 -- 172.21.15.32:0/122250551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9041a26c0 con 0x7fd904109240 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.579+0000 7fd8faa00640 1 -- 172.21.15.32:0/122250551 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0003db0 con 0x7fd904109240 2024-08-31T07:09:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.579+0000 7fd908a00640 1 -- 172.21.15.32:0/122250551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9041a8e20 con 0x7fd904109240 2024-08-31T07:09:43.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.580+0000 7fd908a00640 1 -- 172.21.15.32:0/122250551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd904103d10 con 0x7fd904109240 2024-08-31T07:09:43.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.583+0000 7fd8faa00640 1 -- 172.21.15.32:0/122250551 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fd8f0003f50 con 0x7fd904109240 2024-08-31T07:09:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.584+0000 7fd8faa00640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8e80b4b80 0x7fd8e80b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.584+0000 7fd8faa00640 1 -- 172.21.15.32:0/122250551 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fd8f00e47d0 con 0x7fd904109240 2024-08-31T07:09:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.584+0000 7fd901e00640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8e80b4b80 0x7fd8e80b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.585+0000 7fd901e00640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8e80b4b80 0x7fd8e80b7090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009c90 tx=0x7fd8f4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.586+0000 7fd8faa00640 1 -- 172.21.15.32:0/122250551 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd8f0017020 con 0x7fd904109240 2024-08-31T07:09:43.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:43.783+0000 7fd908a00640 1 -- 172.21.15.32:0/122250551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd9041a90c0 con 0x7fd904109240 2024-08-31T07:09:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.623+0000 7fd8faa00640 1 -- 172.21.15.32:0/122250551 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7fd8f00ab5e0 con 0x7fd904109240 2024-08-31T07:09:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-08-31T07:09:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 -- 172.21.15.32:0/122250551 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8e80b4b80 msgr2=0x7fd8e80b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8e80b4b80 0x7fd8e80b7090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009c90 tx=0x7fd8f4005f90 comp rx=0 tx=0).stop 2024-08-31T07:09:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 -- 172.21.15.32:0/122250551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd904109240 msgr2=0x7fd90419ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd904109240 0x7fd90419ca00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0005fd0 tx=0x7fd8f000dde0 comp rx=0 tx=0).stop 2024-08-31T07:09:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 -- 172.21.15.32:0/122250551 shutdown_connections 2024-08-31T07:09:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8e80b4b80 0x7fd8e80b7090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd904109b90 0x7fd9041a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd904109240 0x7fd90419ca00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 --2- 172.21.15.32:0/122250551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd904103900 0x7fd90419c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 -- 172.21.15.32:0/122250551 >> 172.21.15.32:0/122250551 conn(0x7fd90407b5f0 msgr2=0x7fd904105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 -- 172.21.15.32:0/122250551 shutdown_connections 2024-08-31T07:09:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:44.627+0000 7fd908a00640 1 -- 172.21.15.32:0/122250551 wait complete. 2024-08-31T07:09:44.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:44 smithi032 ceph-mon[17252]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:44.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:44 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/122250551' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T07:09:44.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:44 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T07:09:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:44 smithi073 ceph-mon[21995]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:44 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/122250551' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T07:09:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:44 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T07:09:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:44 smithi084 ceph-mon[22387]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:44 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/122250551' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T07:09:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:44 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T07:09:45.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-31T07:09:45.804 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:45.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:45 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-31T07:09:45.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:45 smithi032 ceph-mon[17252]: osdmap e73: 12 total, 12 up, 12 in 2024-08-31T07:09:45.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:45 smithi032 ceph-mon[17252]: pgmap v271: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:45.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:45 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-31T07:09:45.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:45 smithi073 ceph-mon[21995]: osdmap e73: 12 total, 12 up, 12 in 2024-08-31T07:09:45.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:45 smithi073 ceph-mon[21995]: pgmap v271: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:45 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-31T07:09:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:45 smithi084 ceph-mon[22387]: osdmap e73: 12 total, 12 up, 12 in 2024-08-31T07:09:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:45 smithi084 ceph-mon[22387]: pgmap v271: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.386+0000 7efec1200640 1 -- 172.21.15.32:0/2338752063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc075ea0 msgr2=0x7efebc10f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.386+0000 7efec1200640 1 --2- 172.21.15.32:0/2338752063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc075ea0 0x7efebc10f920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efea4009900 tx=0x7efea402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:09:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.386+0000 7efec1200640 1 -- 172.21.15.32:0/2338752063 shutdown_connections 2024-08-31T07:09:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.386+0000 7efec1200640 1 --2- 172.21.15.32:0/2338752063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efebc10fe60 0x7efebc1122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.386+0000 7efec1200640 1 --2- 172.21.15.32:0/2338752063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc075ea0 0x7efebc10f920 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.386+0000 7efec1200640 1 --2- 172.21.15.32:0/2338752063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc075550 0x7efebc075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.386+0000 7efec1200640 1 -- 172.21.15.32:0/2338752063 >> 172.21.15.32:0/2338752063 conn(0x7efebc07b5f0 msgr2=0x7efebc07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.387+0000 7efec1200640 1 -- 172.21.15.32:0/2338752063 shutdown_connections 2024-08-31T07:09:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.387+0000 7efec1200640 1 -- 172.21.15.32:0/2338752063 wait complete. 2024-08-31T07:09:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efec1200640 1 Processor -- start 2024-08-31T07:09:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efec1200640 1 -- start start 2024-08-31T07:09:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efec1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efebc075550 0x7efebc19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efec1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc075ea0 0x7efebc19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efec1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc10fe60 0x7efebc1a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efec1200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efebc115a10 con 0x7efebc10fe60 2024-08-31T07:09:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efec1200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efebc115890 con 0x7efebc075ea0 2024-08-31T07:09:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efec1200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efebc115b90 con 0x7efebc075550 2024-08-31T07:09:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efeb9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efebc075550 0x7efebc19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efeba800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc10fe60 0x7efebc1a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efeb9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efebc075550 0x7efebc19c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:56006/0 (socket says 172.21.15.32:56006) 2024-08-31T07:09:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.388+0000 7efeb9e00640 1 -- 172.21.15.32:0/3358195673 learned_addr learned my addr 172.21.15.32:0/3358195673 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.389+0000 7efeb3e00640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc075ea0 0x7efebc19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.389+0000 7efeb9e00640 1 -- 172.21.15.32:0/3358195673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc075ea0 msgr2=0x7efebc19ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.389+0000 7efeb9e00640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc075ea0 0x7efebc19ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.389+0000 7efeb9e00640 1 -- 172.21.15.32:0/3358195673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc10fe60 msgr2=0x7efebc1a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.389+0000 7efeb9e00640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc10fe60 0x7efebc1a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.389+0000 7efeb9e00640 1 -- 172.21.15.32:0/3358195673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc1a23d0 con 0x7efebc075550 2024-08-31T07:09:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.389+0000 7efeb9e00640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efebc075550 0x7efebc19c460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efeac00ea60 tx=0x7efeac00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.390+0000 7efeb2a00640 1 -- 172.21.15.32:0/3358195673 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac00cde0 con 0x7efebc075550 2024-08-31T07:09:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.390+0000 7efec1200640 1 -- 172.21.15.32:0/3358195673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc1a26c0 con 0x7efebc075550 2024-08-31T07:09:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.390+0000 7efec1200640 1 -- 172.21.15.32:0/3358195673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc1a8e20 con 0x7efebc075550 2024-08-31T07:09:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.390+0000 7efeb2a00640 1 -- 172.21.15.32:0/3358195673 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeac0035f0 con 0x7efebc075550 2024-08-31T07:09:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.390+0000 7efeb2a00640 1 -- 172.21.15.32:0/3358195673 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac018690 con 0x7efebc075550 2024-08-31T07:09:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.391+0000 7efec1200640 1 -- 172.21.15.32:0/3358195673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc075960 con 0x7efebc075550 2024-08-31T07:09:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.393+0000 7efeb2a00640 1 -- 172.21.15.32:0/3358195673 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7efeac003a10 con 0x7efebc075550 2024-08-31T07:09:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.395+0000 7efeb2a00640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efea00b4c00 0x7efea00b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.395+0000 7efeb2a00640 1 -- 172.21.15.32:0/3358195673 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7712+0+0 (secure 0 0 0) 0x7efeac0e5250 con 0x7efebc075550 2024-08-31T07:09:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.395+0000 7efeb3e00640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efea00b4c00 0x7efea00b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.396+0000 7efeb3e00640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efea00b4c00 0x7efea00b7110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efea4000c00 tx=0x7efea4031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:46.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.397+0000 7efeb2a00640 1 -- 172.21.15.32:0/3358195673 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efeac0ac5f0 con 0x7efebc075550 2024-08-31T07:09:46.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.595+0000 7efec1200640 1 -- 172.21.15.32:0/3358195673 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efebc19d590 con 0x7efebc075550 2024-08-31T07:09:46.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.634+0000 7efeb2a00640 1 -- 172.21.15.32:0/3358195673 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7efeac0abef0 con 0x7efebc075550 2024-08-31T07:09:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-08-31T07:09:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.639+0000 7efec1200640 1 -- 172.21.15.32:0/3358195673 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efea00b4c00 msgr2=0x7efea00b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.639+0000 7efec1200640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efea00b4c00 0x7efea00b7110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efea4000c00 tx=0x7efea4031400 comp rx=0 tx=0).stop 2024-08-31T07:09:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.639+0000 7efec1200640 1 -- 172.21.15.32:0/3358195673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efebc075550 msgr2=0x7efebc19c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.639+0000 7efec1200640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efebc075550 0x7efebc19c460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efeac00ea60 tx=0x7efeac00ef20 comp rx=0 tx=0).stop 2024-08-31T07:09:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.640+0000 7efec1200640 1 -- 172.21.15.32:0/3358195673 shutdown_connections 2024-08-31T07:09:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.640+0000 7efec1200640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efea00b4c00 0x7efea00b7110 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.640+0000 7efec1200640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc10fe60 0x7efebc1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.640+0000 7efec1200640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc075ea0 0x7efebc19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.640+0000 7efec1200640 1 --2- 172.21.15.32:0/3358195673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efebc075550 0x7efebc19c460 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.640+0000 7efec1200640 1 -- 172.21.15.32:0/3358195673 >> 172.21.15.32:0/3358195673 conn(0x7efebc07b5f0 msgr2=0x7efebc101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.641+0000 7efec1200640 1 -- 172.21.15.32:0/3358195673 shutdown_connections 2024-08-31T07:09:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:46.641+0000 7efec1200640 1 -- 172.21.15.32:0/3358195673 wait complete. 2024-08-31T07:09:46.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:46 smithi032 ceph-mon[17252]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T07:09:46.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:46 smithi032 ceph-mon[17252]: osdmap e74: 12 total, 12 up, 12 in 2024-08-31T07:09:46.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:46 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3358195673' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-31T07:09:46.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:46 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-31T07:09:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:46 smithi073 ceph-mon[21995]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T07:09:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:46 smithi073 ceph-mon[21995]: osdmap e74: 12 total, 12 up, 12 in 2024-08-31T07:09:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:46 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3358195673' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-31T07:09:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:46 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-31T07:09:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:46 smithi084 ceph-mon[22387]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T07:09:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:46 smithi084 ceph-mon[22387]: osdmap e74: 12 total, 12 up, 12 in 2024-08-31T07:09:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:46 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3358195673' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-31T07:09:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:46 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-31T07:09:47.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-31T07:09:47.921 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:47 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-31T07:09:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:47 smithi073 ceph-mon[21995]: osdmap e75: 12 total, 12 up, 12 in 2024-08-31T07:09:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:47 smithi073 ceph-mon[21995]: pgmap v274: 97 pgs: 7 creating+peering, 78 unknown, 12 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:47.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:47 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-31T07:09:47.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:47 smithi084 ceph-mon[22387]: osdmap e75: 12 total, 12 up, 12 in 2024-08-31T07:09:47.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:47 smithi084 ceph-mon[22387]: pgmap v274: 97 pgs: 7 creating+peering, 78 unknown, 12 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:48.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:47 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-31T07:09:48.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:47 smithi032 ceph-mon[17252]: osdmap e75: 12 total, 12 up, 12 in 2024-08-31T07:09:48.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:47 smithi032 ceph-mon[17252]: pgmap v274: 97 pgs: 7 creating+peering, 78 unknown, 12 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.544+0000 7fb855e00640 1 -- 172.21.15.32:0/23849196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb850075b70 msgr2=0x7fb850075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.544+0000 7fb855e00640 1 --2- 172.21.15.32:0/23849196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb850075b70 0x7fb850075f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009900 tx=0x7fb83c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:09:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.544+0000 7fb855e00640 1 -- 172.21.15.32:0/23849196 shutdown_connections 2024-08-31T07:09:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.544+0000 7fb855e00640 1 --2- 172.21.15.32:0/23849196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8501141e0 0x7fb8501166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.544+0000 7fb855e00640 1 --2- 172.21.15.32:0/23849196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb850075b70 0x7fb850075f80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.544+0000 7fb855e00640 1 --2- 172.21.15.32:0/23849196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850077120 0x7fb850075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.544+0000 7fb855e00640 1 -- 172.21.15.32:0/23849196 >> 172.21.15.32:0/23849196 conn(0x7fb8500ff380 msgr2=0x7fb8501017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.544+0000 7fb855e00640 1 -- 172.21.15.32:0/23849196 shutdown_connections 2024-08-31T07:09:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.544+0000 7fb855e00640 1 -- 172.21.15.32:0/23849196 wait complete. 2024-08-31T07:09:48.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.545+0000 7fb855e00640 1 Processor -- start 2024-08-31T07:09:48.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.545+0000 7fb855e00640 1 -- start start 2024-08-31T07:09:48.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb855e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850075b70 0x7fb8501a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:48.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb855e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb850077120 0x7fb8501a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:48.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb855e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8501141e0 0x7fb8501a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:48.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb855e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb850119e70 con 0x7fb8501141e0 2024-08-31T07:09:48.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb855e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb850119cf0 con 0x7fb850077120 2024-08-31T07:09:48.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb855e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb850119ff0 con 0x7fb850075b70 2024-08-31T07:09:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb84ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850075b70 0x7fb8501a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb84ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850075b70 0x7fb8501a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:56024/0 (socket says 172.21.15.32:56024) 2024-08-31T07:09:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb84f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8501141e0 0x7fb8501a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb84ea00640 1 -- 172.21.15.32:0/3971647561 learned_addr learned my addr 172.21.15.32:0/3971647561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.546+0000 7fb84e000640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb850077120 0x7fb8501a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.547+0000 7fb84f400640 1 -- 172.21.15.32:0/3971647561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850075b70 msgr2=0x7fb8501a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.547+0000 7fb84f400640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850075b70 0x7fb8501a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.547+0000 7fb84f400640 1 -- 172.21.15.32:0/3971647561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb850077120 msgr2=0x7fb8501a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.547+0000 7fb84f400640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb850077120 0x7fb8501a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.547+0000 7fb84f400640 1 -- 172.21.15.32:0/3971647561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8501a67f0 con 0x7fb8501141e0 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.547+0000 7fb84f400640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8501141e0 0x7fb8501a60c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb848009fd0 tx=0x7fb84800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.547+0000 7fb854a00640 1 -- 172.21.15.32:0/3971647561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84800ce20 con 0x7fb8501141e0 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.548+0000 7fb854a00640 1 -- 172.21.15.32:0/3971647561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb848005020 con 0x7fb8501141e0 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.548+0000 7fb855e00640 1 -- 172.21.15.32:0/3971647561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8501a6ae0 con 0x7fb8501141e0 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.548+0000 7fb854a00640 1 -- 172.21.15.32:0/3971647561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848010690 con 0x7fb8501141e0 2024-08-31T07:09:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.548+0000 7fb855e00640 1 -- 172.21.15.32:0/3971647561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8501ad240 con 0x7fb8501141e0 2024-08-31T07:09:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.549+0000 7fb855e00640 1 -- 172.21.15.32:0/3971647561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb850075f80 con 0x7fb8501141e0 2024-08-31T07:09:48.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.553+0000 7fb854a00640 1 -- 172.21.15.32:0/3971647561 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fb8480054d0 con 0x7fb8501141e0 2024-08-31T07:09:48.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.555+0000 7fb854a00640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb8340b4c00 0x7fb8340b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:48.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.555+0000 7fb854a00640 1 -- 172.21.15.32:0/3971647561 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8094+0+0 (secure 0 0 0) 0x7fb8480e5680 con 0x7fb8501141e0 2024-08-31T07:09:48.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.555+0000 7fb84ea00640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb8340b4c00 0x7fb8340b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:48.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.556+0000 7fb854a00640 1 -- 172.21.15.32:0/3971647561 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb8480ac7f0 con 0x7fb8501141e0 2024-08-31T07:09:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.556+0000 7fb84ea00640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb8340b4c00 0x7fb8340b7110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb840009cd0 tx=0x7fb840005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:48.795+0000 7fb855e00640 1 -- 172.21.15.32:0/3971647561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb8501a19b0 con 0x7fb8501141e0 2024-08-31T07:09:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:48 smithi073 ceph-mon[21995]: osdmap e76: 12 total, 12 up, 12 in 2024-08-31T07:09:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:48 smithi084 ceph-mon[22387]: osdmap e76: 12 total, 12 up, 12 in 2024-08-31T07:09:49.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:48 smithi032 ceph-mon[17252]: osdmap e76: 12 total, 12 up, 12 in 2024-08-31T07:09:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.646+0000 7fb854a00640 1 -- 172.21.15.32:0/3971647561 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb8501a19b0 con 0x7fb8501141e0 2024-08-31T07:09:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2024-08-31T07:09:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.650+0000 7fb855e00640 1 -- 172.21.15.32:0/3971647561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb8340b4c00 msgr2=0x7fb8340b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.650+0000 7fb855e00640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb8340b4c00 0x7fb8340b7110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb840009cd0 tx=0x7fb840005f90 comp rx=0 tx=0).stop 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.650+0000 7fb855e00640 1 -- 172.21.15.32:0/3971647561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8501141e0 msgr2=0x7fb8501a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.650+0000 7fb855e00640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8501141e0 0x7fb8501a60c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb848009fd0 tx=0x7fb84800ef10 comp rx=0 tx=0).stop 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.650+0000 7fb855e00640 1 -- 172.21.15.32:0/3971647561 shutdown_connections 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.650+0000 7fb855e00640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb8340b4c00 0x7fb8340b7110 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.650+0000 7fb855e00640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8501141e0 0x7fb8501a60c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.650+0000 7fb855e00640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb850077120 0x7fb8501a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.650+0000 7fb855e00640 1 --2- 172.21.15.32:0/3971647561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb850075b70 0x7fb8501a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.650+0000 7fb855e00640 1 -- 172.21.15.32:0/3971647561 >> 172.21.15.32:0/3971647561 conn(0x7fb8500ff380 msgr2=0x7fb8501100e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.651+0000 7fb855e00640 1 -- 172.21.15.32:0/3971647561 shutdown_connections 2024-08-31T07:09:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:49.651+0000 7fb855e00640 1 -- 172.21.15.32:0/3971647561 wait complete. 2024-08-31T07:09:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:49 smithi073 ceph-mon[21995]: osdmap e77: 12 total, 12 up, 12 in 2024-08-31T07:09:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:49 smithi073 ceph-mon[21995]: pgmap v277: 97 pgs: 18 creating+peering, 44 unknown, 35 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:49 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3971647561' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-31T07:09:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:49 smithi073 ceph-mon[21995]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-31T07:09:49.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:49 smithi084 ceph-mon[22387]: osdmap e77: 12 total, 12 up, 12 in 2024-08-31T07:09:49.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:49 smithi084 ceph-mon[22387]: pgmap v277: 97 pgs: 18 creating+peering, 44 unknown, 35 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:49.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:49 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3971647561' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-31T07:09:49.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:49 smithi084 ceph-mon[22387]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-31T07:09:50.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:49 smithi032 ceph-mon[17252]: osdmap e77: 12 total, 12 up, 12 in 2024-08-31T07:09:50.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:49 smithi032 ceph-mon[17252]: pgmap v277: 97 pgs: 18 creating+peering, 44 unknown, 35 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:09:50.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:49 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3971647561' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-31T07:09:50.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:49 smithi032 ceph-mon[17252]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-31T07:09:50.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:09:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:50 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3971647561' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-31T07:09:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:50 smithi073 ceph-mon[21995]: osdmap e78: 12 total, 12 up, 12 in 2024-08-31T07:09:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:50 smithi073 ceph-mon[21995]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-31T07:09:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:50 smithi073 ceph-mon[21995]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-31T07:09:50.942 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:50 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3971647561' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-31T07:09:50.942 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:50 smithi032 ceph-mon[17252]: osdmap e78: 12 total, 12 up, 12 in 2024-08-31T07:09:50.942 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:50 smithi032 ceph-mon[17252]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-31T07:09:50.942 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:50 smithi032 ceph-mon[17252]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-31T07:09:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:50 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3971647561' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-31T07:09:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:50 smithi084 ceph-mon[22387]: osdmap e78: 12 total, 12 up, 12 in 2024-08-31T07:09:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:50 smithi084 ceph-mon[22387]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-31T07:09:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:50 smithi084 ceph-mon[22387]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-31T07:09:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.396+0000 7f2622c00640 1 -- 172.21.15.32:0/1572134811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f261c10b290 msgr2=0x7f261c10b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.396+0000 7f2622c00640 1 --2- 172.21.15.32:0/1572134811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f261c10b290 0x7f261c10b6a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2608009900 tx=0x7f260802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:09:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.396+0000 7f2622c00640 1 -- 172.21.15.32:0/1572134811 shutdown_connections 2024-08-31T07:09:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.396+0000 7f2622c00640 1 --2- 172.21.15.32:0/1572134811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f261c10bbe0 0x7f261c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.396+0000 7f2622c00640 1 --2- 172.21.15.32:0/1572134811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f261c10b290 0x7f261c10b6a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.396+0000 7f2622c00640 1 --2- 172.21.15.32:0/1572134811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c105950 0x7f261c105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.396+0000 7f2622c00640 1 -- 172.21.15.32:0/1572134811 >> 172.21.15.32:0/1572134811 conn(0x7f261c101420 msgr2=0x7f261c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.396+0000 7f2622c00640 1 -- 172.21.15.32:0/1572134811 shutdown_connections 2024-08-31T07:09:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.396+0000 7f2622c00640 1 -- 172.21.15.32:0/1572134811 wait complete. 2024-08-31T07:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.397+0000 7f2622c00640 1 Processor -- start 2024-08-31T07:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2622c00640 1 -- start start 2024-08-31T07:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2622c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c105950 0x7f261c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2622c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f261c10b290 0x7f261c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2622c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f261c10bbe0 0x7f261c1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2622c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f261c1158a0 con 0x7f261c10bbe0 2024-08-31T07:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2622c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f261c115720 con 0x7f261c10b290 2024-08-31T07:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2622c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f261c115a20 con 0x7f261c105950 2024-08-31T07:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2620e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f261c10b290 0x7f261c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2621800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c105950 0x7f261c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2622200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f261c10bbe0 0x7f261c1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2620e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f261c10b290 0x7f261c19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50770/0 (socket says 172.21.15.32:50770) 2024-08-31T07:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2621800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c105950 0x7f261c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:56052/0 (socket says 172.21.15.32:56052) 2024-08-31T07:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.398+0000 7f2620e00640 1 -- 172.21.15.32:0/2371818866 learned_addr learned my addr 172.21.15.32:0/2371818866 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.399+0000 7f2620e00640 1 -- 172.21.15.32:0/2371818866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c105950 msgr2=0x7f261c19c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.399+0000 7f2620e00640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c105950 0x7f261c19c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.399+0000 7f2620e00640 1 -- 172.21.15.32:0/2371818866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f261c10bbe0 msgr2=0x7f261c1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.399+0000 7f2620e00640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f261c10bbe0 0x7f261c1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.399+0000 7f2620e00640 1 -- 172.21.15.32:0/2371818866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c1a2220 con 0x7f261c10b290 2024-08-31T07:09:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.399+0000 7f2622200640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f261c10bbe0 0x7f261c1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.399+0000 7f2620e00640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f261c10b290 0x7f261c19c850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f260802a910 tx=0x7f2608031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.400+0000 7f261a000640 1 -- 172.21.15.32:0/2371818866 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260803f030 con 0x7f261c10b290 2024-08-31T07:09:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.400+0000 7f2622c00640 1 -- 172.21.15.32:0/2371818866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261c1a24b0 con 0x7f261c10b290 2024-08-31T07:09:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.400+0000 7f2622c00640 1 -- 172.21.15.32:0/2371818866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261c1a8c70 con 0x7f261c10b290 2024-08-31T07:09:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.400+0000 7f261a000640 1 -- 172.21.15.32:0/2371818866 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2608004490 con 0x7f261c10b290 2024-08-31T07:09:51.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.400+0000 7f261a000640 1 -- 172.21.15.32:0/2371818866 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608049060 con 0x7f261c10b290 2024-08-31T07:09:51.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.401+0000 7f2622c00640 1 -- 172.21.15.32:0/2371818866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc0052c0 con 0x7f261c10b290 2024-08-31T07:09:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.403+0000 7f261a000640 1 -- 172.21.15.32:0/2371818866 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f2608042030 con 0x7f261c10b290 2024-08-31T07:09:51.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.405+0000 7f261a000640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26040b4c00 0x7f26040b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:51.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.405+0000 7f261a000640 1 -- 172.21.15.32:0/2371818866 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f2608105190 con 0x7f261c10b290 2024-08-31T07:09:51.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.405+0000 7f2621800640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26040b4c00 0x7f26040b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:51.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.406+0000 7f2621800640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26040b4c00 0x7f26040b7110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2610009bd0 tx=0x7f2610005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:51.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.407+0000 7f261a000640 1 -- 172.21.15.32:0/2371818866 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f26080cbc30 con 0x7f261c10b290 2024-08-31T07:09:51.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.604+0000 7f2622c00640 1 -- 172.21.15.32:0/2371818866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25fc005880 con 0x7f261c10b290 2024-08-31T07:09:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.606+0000 7f261a000640 1 -- 172.21.15.32:0/2371818866 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22532 (secure 0 0 0) 0x7f26080d03f0 con 0x7f261c10b290 2024-08-31T07:09:51.608 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:51.609 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:09:48.797927+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:09:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 -- 172.21.15.32:0/2371818866 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26040b4c00 msgr2=0x7f26040b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26040b4c00 0x7f26040b7110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2610009bd0 tx=0x7f2610005f90 comp rx=0 tx=0).stop 2024-08-31T07:09:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 -- 172.21.15.32:0/2371818866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f261c10b290 msgr2=0x7f261c19c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f261c10b290 0x7f261c19c850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f260802a910 tx=0x7f2608031a20 comp rx=0 tx=0).stop 2024-08-31T07:09:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 -- 172.21.15.32:0/2371818866 shutdown_connections 2024-08-31T07:09:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26040b4c00 0x7f26040b7110 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f261c10bbe0 0x7f261c1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f261c10b290 0x7f261c19c850 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:51.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 --2- 172.21.15.32:0/2371818866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c105950 0x7f261c19c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 -- 172.21.15.32:0/2371818866 >> 172.21.15.32:0/2371818866 conn(0x7f261c101420 msgr2=0x7f261c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 -- 172.21.15.32:0/2371818866 shutdown_connections 2024-08-31T07:09:51.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:51.609+0000 7f2622c00640 1 -- 172.21.15.32:0/2371818866 wait complete. 2024-08-31T07:09:51.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:51 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] up:active 2024-08-31T07:09:51.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:51 smithi032 ceph-mon[17252]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:09:51.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:51 smithi032 ceph-mon[17252]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:09:51.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:51 smithi032 ceph-mon[17252]: pgmap v279: 97 pgs: 11 creating+peering, 7 unknown, 79 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 496 B/s wr, 5 op/s 2024-08-31T07:09:51.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:51 smithi032 ceph-mon[17252]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T07:09:51.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:51 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2371818866' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:09:51.929 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:09:51 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b[53169]: 2024-08-31T07:09:51.460+0000 7f3fdb3dc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-31T07:09:51.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:51 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] up:active 2024-08-31T07:09:51.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:51 smithi073 ceph-mon[21995]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:09:51.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:51 smithi073 ceph-mon[21995]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:09:51.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:51 smithi073 ceph-mon[21995]: pgmap v279: 97 pgs: 11 creating+peering, 7 unknown, 79 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 496 B/s wr, 5 op/s 2024-08-31T07:09:51.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:51 smithi073 ceph-mon[21995]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T07:09:51.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:51 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2371818866' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:09:51.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:51 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] up:active 2024-08-31T07:09:51.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:51 smithi084 ceph-mon[22387]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:09:51.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:51 smithi084 ceph-mon[22387]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:09:51.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:51 smithi084 ceph-mon[22387]: pgmap v279: 97 pgs: 11 creating+peering, 7 unknown, 79 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 496 B/s wr, 5 op/s 2024-08-31T07:09:51.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:51 smithi084 ceph-mon[22387]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T07:09:51.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:51 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2371818866' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:09:52.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:09:52.803 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:52 smithi073 ceph-mon[21995]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-31T07:09:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:52 smithi073 ceph-mon[21995]: Cluster is now healthy 2024-08-31T07:09:52.942 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:52 smithi032 ceph-mon[17252]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-31T07:09:52.942 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:52 smithi032 ceph-mon[17252]: Cluster is now healthy 2024-08-31T07:09:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:52 smithi084 ceph-mon[22387]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-31T07:09:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:52 smithi084 ceph-mon[22387]: Cluster is now healthy 2024-08-31T07:09:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.435+0000 7ff8c0600640 1 -- 172.21.15.32:0/2157733931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b8109b90 msgr2=0x7ff8b81103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.435+0000 7ff8c0600640 1 --2- 172.21.15.32:0/2157733931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b8109b90 0x7ff8b81103c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009ca0 tx=0x7ff8b002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.435+0000 7ff8c0600640 1 -- 172.21.15.32:0/2157733931 shutdown_connections 2024-08-31T07:09:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.435+0000 7ff8c0600640 1 --2- 172.21.15.32:0/2157733931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b8109b90 0x7ff8b81103c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.435+0000 7ff8c0600640 1 --2- 172.21.15.32:0/2157733931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b8109240 0x7ff8b8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.435+0000 7ff8c0600640 1 --2- 172.21.15.32:0/2157733931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b8103900 0x7ff8b8103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.435+0000 7ff8c0600640 1 -- 172.21.15.32:0/2157733931 >> 172.21.15.32:0/2157733931 conn(0x7ff8b807b5f0 msgr2=0x7ff8b807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.435+0000 7ff8c0600640 1 -- 172.21.15.32:0/2157733931 shutdown_connections 2024-08-31T07:09:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.435+0000 7ff8c0600640 1 -- 172.21.15.32:0/2157733931 wait complete. 2024-08-31T07:09:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.436+0000 7ff8c0600640 1 Processor -- start 2024-08-31T07:09:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8c0600640 1 -- start start 2024-08-31T07:09:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8c0600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b8103900 0x7ff8b819c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8c0600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b8109240 0x7ff8b819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8c0600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b8109b90 0x7ff8b81a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8c0600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b81138a0 con 0x7ff8b8109240 2024-08-31T07:09:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8c0600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8113720 con 0x7ff8b8103900 2024-08-31T07:09:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8c0600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8113a20 con 0x7ff8b8109b90 2024-08-31T07:09:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8bd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b8109240 0x7ff8b819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8bd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b8109240 0x7ff8b819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37662/0 (socket says 172.21.15.32:37662) 2024-08-31T07:09:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8bd200640 1 -- 172.21.15.32:0/1740172072 learned_addr learned my addr 172.21.15.32:0/1740172072 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8be600640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b8109b90 0x7ff8b81a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:53.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.437+0000 7ff8bdc00640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b8103900 0x7ff8b819c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.438+0000 7ff8bd200640 1 -- 172.21.15.32:0/1740172072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b8109b90 msgr2=0x7ff8b81a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.438+0000 7ff8bd200640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b8109b90 0x7ff8b81a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.438+0000 7ff8bd200640 1 -- 172.21.15.32:0/1740172072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b8103900 msgr2=0x7ff8b819c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.438+0000 7ff8bd200640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b8103900 0x7ff8b819c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.438+0000 7ff8bd200640 1 -- 172.21.15.32:0/1740172072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b81a2380 con 0x7ff8b8109240 2024-08-31T07:09:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.438+0000 7ff8bdc00640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b8103900 0x7ff8b819c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:53.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.438+0000 7ff8bd200640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b8109240 0x7ff8b819c9b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400e9f0 tx=0x7ff8a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:53.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.439+0000 7ff8b6000640 1 -- 172.21.15.32:0/1740172072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a400ccc0 con 0x7ff8b8109240 2024-08-31T07:09:53.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.439+0000 7ff8b6000640 1 -- 172.21.15.32:0/1740172072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a40108e0 con 0x7ff8b8109240 2024-08-31T07:09:53.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.439+0000 7ff8c0600640 1 -- 172.21.15.32:0/1740172072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b81a2670 con 0x7ff8b8109240 2024-08-31T07:09:53.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.439+0000 7ff8b6000640 1 -- 172.21.15.32:0/1740172072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4018860 con 0x7ff8b8109240 2024-08-31T07:09:53.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.439+0000 7ff8c0600640 1 -- 172.21.15.32:0/1740172072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b81a8dd0 con 0x7ff8b8109240 2024-08-31T07:09:53.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.440+0000 7ff8c0600640 1 -- 172.21.15.32:0/1740172072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b8103d10 con 0x7ff8b8109240 2024-08-31T07:09:53.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.442+0000 7ff8b6000640 1 -- 172.21.15.32:0/1740172072 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7ff8a4010430 con 0x7ff8b8109240 2024-08-31T07:09:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.444+0000 7ff8b6000640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8a00b4b80 0x7ff8a00b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.444+0000 7ff8b6000640 1 -- 172.21.15.32:0/1740172072 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7ff8a40e5470 con 0x7ff8b8109240 2024-08-31T07:09:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.445+0000 7ff8bdc00640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8a00b4b80 0x7ff8a00b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:53.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.445+0000 7ff8bdc00640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8a00b4b80 0x7ff8a00b7090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0095f0 tx=0x7ff8ac005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:53.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.448+0000 7ff8b6000640 1 -- 172.21.15.32:0/1740172072 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff8a40abf10 con 0x7ff8b8109240 2024-08-31T07:09:53.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.651+0000 7ff8c0600640 1 -- 172.21.15.32:0/1740172072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8b81a90b0 con 0x7ff8b8109240 2024-08-31T07:09:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.653+0000 7ff8b6000640 1 -- 172.21.15.32:0/1740172072 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22532 (secure 0 0 0) 0x7ff8a40b06d0 con 0x7ff8b8109240 2024-08-31T07:09:53.654 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:09:53.654 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:09:48.797927+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:09:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 -- 172.21.15.32:0/1740172072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8a00b4b80 msgr2=0x7ff8a00b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8a00b4b80 0x7ff8a00b7090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0095f0 tx=0x7ff8ac005f90 comp rx=0 tx=0).stop 2024-08-31T07:09:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 -- 172.21.15.32:0/1740172072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b8109240 msgr2=0x7ff8b819c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b8109240 0x7ff8b819c9b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400e9f0 tx=0x7ff8a400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 -- 172.21.15.32:0/1740172072 shutdown_connections 2024-08-31T07:09:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8a00b4b80 0x7ff8a00b7090 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b8109b90 0x7ff8b81a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b8109240 0x7ff8b819c9b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 --2- 172.21.15.32:0/1740172072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b8103900 0x7ff8b819c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 -- 172.21.15.32:0/1740172072 >> 172.21.15.32:0/1740172072 conn(0x7ff8b807b5f0 msgr2=0x7ff8b8107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 -- 172.21.15.32:0/1740172072 shutdown_connections 2024-08-31T07:09:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:53.656+0000 7ff8c0600640 1 -- 172.21.15.32:0/1740172072 wait complete. 2024-08-31T07:09:53.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:53 smithi073 ceph-mon[21995]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-08-31T07:09:53.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:53 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1740172072' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:09:53.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:53 smithi084 ceph-mon[22387]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-08-31T07:09:53.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:53 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1740172072' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:09:54.045 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:53 smithi032 ceph-mon[17252]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-08-31T07:09:54.045 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:53 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1740172072' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:09:54.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-31T07:09:54.863 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:55.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.600+0000 7f1cc0c00640 1 -- 172.21.15.32:0/1288804228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc10bbe0 msgr2=0x7f1cbc112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.600+0000 7f1cc0c00640 1 --2- 172.21.15.32:0/1288804228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc10bbe0 0x7f1cbc112410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009ca0 tx=0x7f1cb402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:09:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.600+0000 7f1cc0c00640 1 -- 172.21.15.32:0/1288804228 shutdown_connections 2024-08-31T07:09:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.600+0000 7f1cc0c00640 1 --2- 172.21.15.32:0/1288804228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc10bbe0 0x7f1cbc112410 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.600+0000 7f1cc0c00640 1 --2- 172.21.15.32:0/1288804228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc10b290 0x7f1cbc10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.600+0000 7f1cc0c00640 1 --2- 172.21.15.32:0/1288804228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc105950 0x7f1cbc105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.600+0000 7f1cc0c00640 1 -- 172.21.15.32:0/1288804228 >> 172.21.15.32:0/1288804228 conn(0x7f1cbc101420 msgr2=0x7f1cbc103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.600+0000 7f1cc0c00640 1 -- 172.21.15.32:0/1288804228 shutdown_connections 2024-08-31T07:09:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.600+0000 7f1cc0c00640 1 -- 172.21.15.32:0/1288804228 wait complete. 2024-08-31T07:09:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.602+0000 7f1cc0c00640 1 Processor -- start 2024-08-31T07:09:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.602+0000 7f1cc0c00640 1 -- start start 2024-08-31T07:09:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.602+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc105950 0x7f1cbc19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc10b290 0x7f1cbc19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc10bbe0 0x7f1cbc1a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cc0c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1158f0 con 0x7f1cbc105950 2024-08-31T07:09:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cc0c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc115770 con 0x7f1cbc10b290 2024-08-31T07:09:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cc0c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc115a70 con 0x7f1cbc10bbe0 2024-08-31T07:09:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cbb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc105950 0x7f1cbc19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cbaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc10b290 0x7f1cbc19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cbb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc105950 0x7f1cbc19c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37684/0 (socket says 172.21.15.32:37684) 2024-08-31T07:09:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cbb400640 1 -- 172.21.15.32:0/3077452344 learned_addr learned my addr 172.21.15.32:0/3077452344 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cbbe00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc10bbe0 0x7f1cbc1a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cbaa00640 1 -- 172.21.15.32:0/3077452344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc10bbe0 msgr2=0x7f1cbc1a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cbaa00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc10bbe0 0x7f1cbc1a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cbaa00640 1 -- 172.21.15.32:0/3077452344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc105950 msgr2=0x7f1cbc19c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cbaa00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc105950 0x7f1cbc19c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.603+0000 7f1cbaa00640 1 -- 172.21.15.32:0/3077452344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc1a21b0 con 0x7f1cbc10b290 2024-08-31T07:09:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.604+0000 7f1cbaa00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc10b290 0x7f1cbc19c7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8005fd0 tx=0x7f1ca800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.604+0000 7f1cb3e00640 1 -- 172.21.15.32:0/3077452344 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8009c00 con 0x7f1cbc10b290 2024-08-31T07:09:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.604+0000 7f1cb3e00640 1 -- 172.21.15.32:0/3077452344 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca8009da0 con 0x7f1cbc10b290 2024-08-31T07:09:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.604+0000 7f1cc0c00640 1 -- 172.21.15.32:0/3077452344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc1a24a0 con 0x7f1cbc10b290 2024-08-31T07:09:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.604+0000 7f1cb3e00640 1 -- 172.21.15.32:0/3077452344 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8003db0 con 0x7f1cbc10b290 2024-08-31T07:09:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.604+0000 7f1cc0c00640 1 -- 172.21.15.32:0/3077452344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc1a8c00 con 0x7f1cbc10b290 2024-08-31T07:09:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.605+0000 7f1cc0c00640 1 -- 172.21.15.32:0/3077452344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cbc105d60 con 0x7f1cbc10b290 2024-08-31T07:09:55.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.608+0000 7f1cb3e00640 1 -- 172.21.15.32:0/3077452344 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f1ca8004000 con 0x7f1cbc10b290 2024-08-31T07:09:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.610+0000 7f1cb3e00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca00b4c00 0x7f1ca00b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.610+0000 7f1cb3e00640 1 -- 172.21.15.32:0/3077452344 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f1ca80e4b60 con 0x7f1cbc10b290 2024-08-31T07:09:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.610+0000 7f1cbb400640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca00b4c00 0x7f1ca00b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.611+0000 7f1cbb400640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca00b4c00 0x7f1ca00b7110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009c90 tx=0x7f1cac005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:55.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.611+0000 7f1cb3e00640 1 -- 172.21.15.32:0/3077452344 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1ca8017020 con 0x7f1cbc10b290 2024-08-31T07:09:55.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:55.856+0000 7f1cc0c00640 1 -- 172.21.15.32:0/3077452344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1cbc19d370 con 0x7f1cbc10b290 2024-08-31T07:09:56.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:55 smithi032 ceph-mon[17252]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-31T07:09:56.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:55 smithi073 ceph-mon[21995]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-31T07:09:56.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:55 smithi084 ceph-mon[22387]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-31T07:09:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.821+0000 7f1cb3e00640 1 -- 172.21.15.32:0/3077452344 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f1ca80ab680 con 0x7f1cbc10b290 2024-08-31T07:09:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 -- 172.21.15.32:0/3077452344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca00b4c00 msgr2=0x7f1ca00b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca00b4c00 0x7f1ca00b7110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009c90 tx=0x7f1cac005f90 comp rx=0 tx=0).stop 2024-08-31T07:09:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 -- 172.21.15.32:0/3077452344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc10b290 msgr2=0x7f1cbc19c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc10b290 0x7f1cbc19c7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8005fd0 tx=0x7f1ca800dde0 comp rx=0 tx=0).stop 2024-08-31T07:09:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 -- 172.21.15.32:0/3077452344 shutdown_connections 2024-08-31T07:09:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca00b4c00 0x7f1ca00b7110 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc10bbe0 0x7f1cbc1a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cbc10b290 0x7f1cbc19c7e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 --2- 172.21.15.32:0/3077452344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc105950 0x7f1cbc19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 -- 172.21.15.32:0/3077452344 >> 172.21.15.32:0/3077452344 conn(0x7f1cbc101420 msgr2=0x7f1cbc108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 -- 172.21.15.32:0/3077452344 shutdown_connections 2024-08-31T07:09:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:56.824+0000 7f1cc0c00640 1 -- 172.21.15.32:0/3077452344 wait complete. 2024-08-31T07:09:57.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:56 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3077452344' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-31T07:09:57.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:56 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-31T07:09:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:56 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3077452344' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-31T07:09:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:56 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-31T07:09:57.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:56 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3077452344' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-31T07:09:57.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:56 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-31T07:09:57.735 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-08-31T07:09:57.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-08-31T07:09:58.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:57 smithi032 ceph-mon[17252]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-31T07:09:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:57 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-31T07:09:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:57 smithi032 ceph-mon[17252]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:09:58.119 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:09:58.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:57 smithi073 ceph-mon[21995]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-31T07:09:58.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:57 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-31T07:09:58.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:57 smithi073 ceph-mon[21995]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:09:58.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:57 smithi084 ceph-mon[22387]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-31T07:09:58.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:57 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-31T07:09:58.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:57 smithi084 ceph-mon[22387]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:09:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.818+0000 7feea6400640 1 -- 172.21.15.32:0/2238611149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea010bbe0 msgr2=0x7feea0112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.818+0000 7feea6400640 1 --2- 172.21.15.32:0/2238611149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea010bbe0 0x7feea0112410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee98009be0 tx=0x7fee9802a6c0 comp rx=0 tx=0).stop 2024-08-31T07:09:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.818+0000 7feea6400640 1 -- 172.21.15.32:0/2238611149 shutdown_connections 2024-08-31T07:09:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.818+0000 7feea6400640 1 --2- 172.21.15.32:0/2238611149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea010bbe0 0x7feea0112410 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.818+0000 7feea6400640 1 --2- 172.21.15.32:0/2238611149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea010b290 0x7feea010b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.818+0000 7feea6400640 1 --2- 172.21.15.32:0/2238611149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feea0105950 0x7feea0105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.818+0000 7feea6400640 1 -- 172.21.15.32:0/2238611149 >> 172.21.15.32:0/2238611149 conn(0x7feea0101420 msgr2=0x7feea0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.818+0000 7feea6400640 1 -- 172.21.15.32:0/2238611149 shutdown_connections 2024-08-31T07:09:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.818+0000 7feea6400640 1 -- 172.21.15.32:0/2238611149 wait complete. 2024-08-31T07:09:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.819+0000 7feea6400640 1 Processor -- start 2024-08-31T07:09:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea6400640 1 -- start start 2024-08-31T07:09:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea6400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feea0105950 0x7feea019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea6400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea010b290 0x7feea019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea010bbe0 0x7feea01a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea6400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feea01158a0 con 0x7feea010bbe0 2024-08-31T07:09:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea6400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feea0115720 con 0x7feea010b290 2024-08-31T07:09:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea6400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feea0115a20 con 0x7feea0105950 2024-08-31T07:09:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea5000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feea0105950 0x7feea019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea5000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feea0105950 0x7feea019c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:48944/0 (socket says 172.21.15.32:48944) 2024-08-31T07:09:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea5000640 1 -- 172.21.15.32:0/2047594128 learned_addr learned my addr 172.21.15.32:0/2047594128 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:09:58.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7fee9fe00640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea010b290 0x7feea019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:58.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.820+0000 7feea5a00640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea010bbe0 0x7feea01a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:58.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.821+0000 7feea5000640 1 -- 172.21.15.32:0/2047594128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea010b290 msgr2=0x7feea019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:58.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.821+0000 7feea5000640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea010b290 0x7feea019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:58.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.821+0000 7feea5000640 1 -- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea010bbe0 msgr2=0x7feea01a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:58.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.821+0000 7feea5000640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea010bbe0 0x7feea01a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:58.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.821+0000 7feea5000640 1 -- 172.21.15.32:0/2047594128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea01a21e0 con 0x7feea0105950 2024-08-31T07:09:58.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.821+0000 7feea5a00640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea010bbe0 0x7feea01a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:09:58.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.821+0000 7feea5000640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feea0105950 0x7feea019c270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee9400e9f0 tx=0x7fee9400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:58.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.827+0000 7fee9d600640 1 -- 172.21.15.32:0/2047594128 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9400cd80 con 0x7feea0105950 2024-08-31T07:09:58.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.827+0000 7fee9d600640 1 -- 172.21.15.32:0/2047594128 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee940054d0 con 0x7feea0105950 2024-08-31T07:09:58.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.827+0000 7feea6400640 1 -- 172.21.15.32:0/2047594128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea01a24d0 con 0x7feea0105950 2024-08-31T07:09:58.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.827+0000 7fee9d600640 1 -- 172.21.15.32:0/2047594128 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94010690 con 0x7feea0105950 2024-08-31T07:09:58.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.827+0000 7feea6400640 1 -- 172.21.15.32:0/2047594128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea01a8c30 con 0x7feea0105950 2024-08-31T07:09:58.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.828+0000 7feea6400640 1 -- 172.21.15.32:0/2047594128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea0105d60 con 0x7feea0105950 2024-08-31T07:09:58.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.830+0000 7fee9d600640 1 -- 172.21.15.32:0/2047594128 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fee94004e90 con 0x7feea0105950 2024-08-31T07:09:58.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.831+0000 7fee9d600640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fee880b4c00 0x7fee880b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:09:58.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.832+0000 7fee9d600640 1 -- 172.21.15.32:0/2047594128 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fee940e5560 con 0x7feea0105950 2024-08-31T07:09:58.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.832+0000 7fee9fe00640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fee880b4c00 0x7fee880b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:09:58.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.832+0000 7fee9fe00640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fee880b4c00 0x7fee880b7110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feea0104c70 tx=0x7fee8c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:09:58.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:58.835+0000 7fee9d600640 1 -- 172.21.15.32:0/2047594128 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fee940ac650 con 0x7feea0105950 2024-08-31T07:09:59.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.072+0000 7feea6400640 1 -- 172.21.15.32:0/2047594128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7feea010da30 con 0x7feea0105950 2024-08-31T07:09:59.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.835+0000 7fee9d600640 1 -- 172.21.15.32:0/2047594128 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fee940abf50 con 0x7feea0105950 2024-08-31T07:09:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.838+0000 7feea6400640 1 -- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fee880b4c00 msgr2=0x7fee880b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.838+0000 7feea6400640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fee880b4c00 0x7fee880b7110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feea0104c70 tx=0x7fee8c009260 comp rx=0 tx=0).stop 2024-08-31T07:09:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.839+0000 7feea6400640 1 -- 172.21.15.32:0/2047594128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feea0105950 msgr2=0x7feea019c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:09:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.839+0000 7feea6400640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feea0105950 0x7feea019c270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee9400e9f0 tx=0x7fee9400edf0 comp rx=0 tx=0).stop 2024-08-31T07:09:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.839+0000 7feea6400640 1 -- 172.21.15.32:0/2047594128 shutdown_connections 2024-08-31T07:09:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.839+0000 7feea6400640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fee880b4c00 0x7fee880b7110 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.839+0000 7feea6400640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea010bbe0 0x7feea01a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.839+0000 7feea6400640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea010b290 0x7feea019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.839+0000 7feea6400640 1 --2- 172.21.15.32:0/2047594128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feea0105950 0x7feea019c270 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:09:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.839+0000 7feea6400640 1 -- 172.21.15.32:0/2047594128 >> 172.21.15.32:0/2047594128 conn(0x7feea0101420 msgr2=0x7feea0108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:09:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.839+0000 7feea6400640 1 -- 172.21.15.32:0/2047594128 shutdown_connections 2024-08-31T07:09:59.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:09:59.839+0000 7feea6400640 1 -- 172.21.15.32:0/2047594128 wait complete. 2024-08-31T07:10:00.078 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:59 smithi032 ceph-mon[17252]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-08-31T07:10:00.079 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:59 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2047594128' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-08-31T07:10:00.079 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:09:59 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-08-31T07:10:00.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:59 smithi073 ceph-mon[21995]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-08-31T07:10:00.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:59 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2047594128' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-08-31T07:10:00.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:09:59 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-08-31T07:10:00.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:59 smithi084 ceph-mon[22387]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-08-31T07:10:00.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:59 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2047594128' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-08-31T07:10:00.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:09:59 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-08-31T07:10:00.745 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-31T07:10:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:00 smithi032 ceph-mon[17252]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-31T07:10:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:00 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-31T07:10:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:00 smithi032 ceph-mon[17252]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-31T07:10:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:00 smithi032 ceph-mon[17252]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:10:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:00 smithi032 ceph-mon[17252]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-08-31T07:10:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:00 smithi032 ceph-mon[17252]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-08-31T07:10:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:00 smithi032 ceph-mon[17252]: Health detail: HEALTH_WARN 1 filesystem is online with fewer MDS than max_mds 2024-08-31T07:10:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:00 smithi032 ceph-mon[17252]: [WRN] MDS_UP_LESS_THAN_MAX: 1 filesystem is online with fewer MDS than max_mds 2024-08-31T07:10:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:00 smithi032 ceph-mon[17252]: fs cephfs has 2 MDS online, but wants 5 2024-08-31T07:10:01.134 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:00 smithi073 ceph-mon[21995]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-31T07:10:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:00 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-31T07:10:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:00 smithi073 ceph-mon[21995]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-31T07:10:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:00 smithi073 ceph-mon[21995]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:10:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:00 smithi073 ceph-mon[21995]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-08-31T07:10:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:00 smithi073 ceph-mon[21995]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-08-31T07:10:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:00 smithi073 ceph-mon[21995]: Health detail: HEALTH_WARN 1 filesystem is online with fewer MDS than max_mds 2024-08-31T07:10:01.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:00 smithi073 ceph-mon[21995]: [WRN] MDS_UP_LESS_THAN_MAX: 1 filesystem is online with fewer MDS than max_mds 2024-08-31T07:10:01.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:00 smithi073 ceph-mon[21995]: fs cephfs has 2 MDS online, but wants 5 2024-08-31T07:10:01.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:00 smithi084 ceph-mon[22387]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-31T07:10:01.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:00 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-31T07:10:01.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:00 smithi084 ceph-mon[22387]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-31T07:10:01.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:00 smithi084 ceph-mon[22387]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T07:10:01.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:00 smithi084 ceph-mon[22387]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-08-31T07:10:01.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:00 smithi084 ceph-mon[22387]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-08-31T07:10:01.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:00 smithi084 ceph-mon[22387]: Health detail: HEALTH_WARN 1 filesystem is online with fewer MDS than max_mds 2024-08-31T07:10:01.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:00 smithi084 ceph-mon[22387]: [WRN] MDS_UP_LESS_THAN_MAX: 1 filesystem is online with fewer MDS than max_mds 2024-08-31T07:10:01.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:00 smithi084 ceph-mon[22387]: fs cephfs has 2 MDS online, but wants 5 2024-08-31T07:10:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.771+0000 7f282b200640 1 -- 172.21.15.32:0/269599975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c10fc90 msgr2=0x7f282c1120d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.771+0000 7f282b200640 1 --2- 172.21.15.32:0/269599975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c10fc90 0x7f282c1120d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2824009be0 tx=0x7f282402a6c0 comp rx=0 tx=0).stop 2024-08-31T07:10:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.771+0000 7f282b200640 1 -- 172.21.15.32:0/269599975 shutdown_connections 2024-08-31T07:10:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.771+0000 7f282b200640 1 --2- 172.21.15.32:0/269599975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c10fc90 0x7f282c1120d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.771+0000 7f282b200640 1 --2- 172.21.15.32:0/269599975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c075ea0 0x7f282c10f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.771+0000 7f282b200640 1 --2- 172.21.15.32:0/269599975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c075550 0x7f282c075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.771+0000 7f282b200640 1 -- 172.21.15.32:0/269599975 >> 172.21.15.32:0/269599975 conn(0x7f282c0ff180 msgr2=0x7f282c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.771+0000 7f282b200640 1 -- 172.21.15.32:0/269599975 shutdown_connections 2024-08-31T07:10:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.771+0000 7f282b200640 1 -- 172.21.15.32:0/269599975 wait complete. 2024-08-31T07:10:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.772+0000 7f282b200640 1 Processor -- start 2024-08-31T07:10:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.772+0000 7f282b200640 1 -- start start 2024-08-31T07:10:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f282b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c075550 0x7f282c19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f282b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c075ea0 0x7f282c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f282b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c10fc90 0x7f282c1a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f282b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f282c115800 con 0x7f282c075550 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f282b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f282c115680 con 0x7f282c10fc90 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f282b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f282c115980 con 0x7f282c075ea0 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f2829e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c075550 0x7f282c19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f2829e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c075550 0x7f282c19c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37874/0 (socket says 172.21.15.32:37874) 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f2829e00640 1 -- 172.21.15.32:0/1764384200 learned_addr learned my addr 172.21.15.32:0/1764384200 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f282a800640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c10fc90 0x7f282c1a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.773+0000 7f2823e00640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c075ea0 0x7f282c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.774+0000 7f2829e00640 1 -- 172.21.15.32:0/1764384200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c075ea0 msgr2=0x7f282c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.774+0000 7f2829e00640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c075ea0 0x7f282c19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.774+0000 7f2829e00640 1 -- 172.21.15.32:0/1764384200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c10fc90 msgr2=0x7f282c1a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.774+0000 7f2829e00640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c10fc90 0x7f282c1a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.774+0000 7f2829e00640 1 -- 172.21.15.32:0/1764384200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c1a21c0 con 0x7f282c075550 2024-08-31T07:10:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.774+0000 7f2829e00640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c075550 0x7f282c19c250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f281c00d950 tx=0x7f281c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.774+0000 7f2822a00640 1 -- 172.21.15.32:0/1764384200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c017030 con 0x7f282c075550 2024-08-31T07:10:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.774+0000 7f282b200640 1 -- 172.21.15.32:0/1764384200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c1a24b0 con 0x7f282c075550 2024-08-31T07:10:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.775+0000 7f282b200640 1 -- 172.21.15.32:0/1764384200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c1a8c10 con 0x7f282c075550 2024-08-31T07:10:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.775+0000 7f2822a00640 1 -- 172.21.15.32:0/1764384200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f281c004020 con 0x7f282c075550 2024-08-31T07:10:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.775+0000 7f2822a00640 1 -- 172.21.15.32:0/1764384200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c0047a0 con 0x7f282c075550 2024-08-31T07:10:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.776+0000 7f282b200640 1 -- 172.21.15.32:0/1764384200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28080052c0 con 0x7f282c075550 2024-08-31T07:10:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.778+0000 7f2822a00640 1 -- 172.21.15.32:0/1764384200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f281c0041d0 con 0x7f282c075550 2024-08-31T07:10:01.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.779+0000 7f2822a00640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4c00 0x7f28100b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:01.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.780+0000 7f2822a00640 1 -- 172.21.15.32:0/1764384200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f281c0e4960 con 0x7f282c075550 2024-08-31T07:10:01.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.780+0000 7f2823e00640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4c00 0x7f28100b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:01.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.780+0000 7f2823e00640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4c00 0x7f28100b7110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28140096c0 tx=0x7f2814009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:01.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:01.783+0000 7f2822a00640 1 -- 172.21.15.32:0/1764384200 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f281c0aba50 con 0x7f282c075550 2024-08-31T07:10:01.845 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:10:01 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b[53169]: 2024-08-31T07:10:01.460+0000 7f3fdb3dc640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-08-31T07:10:02.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.027+0000 7f282b200640 1 -- 172.21.15.32:0/1764384200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f28080055b0 con 0x7f282c075550 2024-08-31T07:10:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:01 smithi032 ceph-mon[17252]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-08-31T07:10:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:01 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] up:active 2024-08-31T07:10:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:01 smithi032 ceph-mon[17252]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-31T07:10:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:01 smithi032 ceph-mon[17252]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-08-31T07:10:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:01 smithi032 ceph-mon[17252]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-08-31T07:10:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:01 smithi032 ceph-mon[17252]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-31T07:10:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:01 smithi073 ceph-mon[21995]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-08-31T07:10:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:01 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] up:active 2024-08-31T07:10:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:01 smithi073 ceph-mon[21995]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-31T07:10:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:01 smithi073 ceph-mon[21995]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-08-31T07:10:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:01 smithi073 ceph-mon[21995]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-08-31T07:10:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:01 smithi073 ceph-mon[21995]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-31T07:10:02.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:01 smithi084 ceph-mon[22387]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-08-31T07:10:02.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:01 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] up:active 2024-08-31T07:10:02.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:01 smithi084 ceph-mon[22387]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-31T07:10:02.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:01 smithi084 ceph-mon[22387]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-08-31T07:10:02.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:01 smithi084 ceph-mon[22387]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-08-31T07:10:02.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:01 smithi084 ceph-mon[22387]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-31T07:10:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.861+0000 7f2822a00640 1 -- 172.21.15.32:0/1764384200 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f281c0ab350 con 0x7f282c075550 2024-08-31T07:10:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.864+0000 7f282b200640 1 -- 172.21.15.32:0/1764384200 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4c00 msgr2=0x7f28100b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.864+0000 7f282b200640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4c00 0x7f28100b7110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28140096c0 tx=0x7f2814009260 comp rx=0 tx=0).stop 2024-08-31T07:10:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.864+0000 7f282b200640 1 -- 172.21.15.32:0/1764384200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c075550 msgr2=0x7f282c19c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.864+0000 7f282b200640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c075550 0x7f282c19c250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f281c00d950 tx=0x7f281c00de10 comp rx=0 tx=0).stop 2024-08-31T07:10:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.864+0000 7f282b200640 1 -- 172.21.15.32:0/1764384200 shutdown_connections 2024-08-31T07:10:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.864+0000 7f282b200640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4c00 0x7f28100b7110 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.864+0000 7f282b200640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c10fc90 0x7f282c1a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.864+0000 7f282b200640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c075ea0 0x7f282c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.864+0000 7f282b200640 1 --2- 172.21.15.32:0/1764384200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c075550 0x7f282c19c250 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.864+0000 7f282b200640 1 -- 172.21.15.32:0/1764384200 >> 172.21.15.32:0/1764384200 conn(0x7f282c0ff180 msgr2=0x7f282c101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.865+0000 7f282b200640 1 -- 172.21.15.32:0/1764384200 shutdown_connections 2024-08-31T07:10:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:02.865+0000 7f282b200640 1 -- 172.21.15.32:0/1764384200 wait complete. 2024-08-31T07:10:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:02 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] up:active 2024-08-31T07:10:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:02 smithi084 ceph-mon[22387]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-31T07:10:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:02 smithi084 ceph-mon[22387]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-08-31T07:10:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:02 smithi084 ceph-mon[22387]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-08-31T07:10:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:02 smithi084 ceph-mon[22387]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-08-31T07:10:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:02 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1764384200' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-31T07:10:03.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:02 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] up:active 2024-08-31T07:10:03.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:02 smithi032 ceph-mon[17252]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-31T07:10:03.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:02 smithi032 ceph-mon[17252]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-08-31T07:10:03.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:02 smithi032 ceph-mon[17252]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-08-31T07:10:03.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:02 smithi032 ceph-mon[17252]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-08-31T07:10:03.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:02 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1764384200' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-31T07:10:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:02 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] up:active 2024-08-31T07:10:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:02 smithi073 ceph-mon[21995]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-31T07:10:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:02 smithi073 ceph-mon[21995]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-08-31T07:10:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:02 smithi073 ceph-mon[21995]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-08-31T07:10:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:02 smithi073 ceph-mon[21995]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-08-31T07:10:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:02 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1764384200' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-31T07:10:03.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-31T07:10:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:03 smithi032 ceph-mon[17252]: pgmap v285: 97 pgs: 97 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 4 op/s 2024-08-31T07:10:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:03 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1764384200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-31T07:10:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:03 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] up:active 2024-08-31T07:10:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:03 smithi032 ceph-mon[17252]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-31T07:10:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:03 smithi032 ceph-mon[17252]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-31T07:10:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:03 smithi032 ceph-mon[17252]: Cluster is now healthy 2024-08-31T07:10:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:03 smithi032 ceph-mon[17252]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-08-31T07:10:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:03 smithi032 ceph-mon[17252]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-08-31T07:10:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:03 smithi032 ceph-mon[17252]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-08-31T07:10:04.114 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:03 smithi073 ceph-mon[21995]: pgmap v285: 97 pgs: 97 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 4 op/s 2024-08-31T07:10:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:03 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1764384200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-31T07:10:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:03 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] up:active 2024-08-31T07:10:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:03 smithi073 ceph-mon[21995]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-31T07:10:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:03 smithi073 ceph-mon[21995]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-31T07:10:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:03 smithi073 ceph-mon[21995]: Cluster is now healthy 2024-08-31T07:10:04.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:03 smithi073 ceph-mon[21995]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-08-31T07:10:04.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:03 smithi073 ceph-mon[21995]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-08-31T07:10:04.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:03 smithi073 ceph-mon[21995]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-08-31T07:10:04.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:03 smithi084 ceph-mon[22387]: pgmap v285: 97 pgs: 97 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 4 op/s 2024-08-31T07:10:04.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:03 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1764384200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-31T07:10:04.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:03 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] up:active 2024-08-31T07:10:04.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:03 smithi084 ceph-mon[22387]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-31T07:10:04.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:03 smithi084 ceph-mon[22387]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-31T07:10:04.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:03 smithi084 ceph-mon[22387]: Cluster is now healthy 2024-08-31T07:10:04.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:03 smithi084 ceph-mon[22387]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-08-31T07:10:04.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:03 smithi084 ceph-mon[22387]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-08-31T07:10:04.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:03 smithi084 ceph-mon[22387]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-08-31T07:10:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.711+0000 7fd47f200640 1 -- 172.21.15.32:0/4097598410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd480103730 msgr2=0x7fd480103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.711+0000 7fd47f200640 1 --2- 172.21.15.32:0/4097598410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd480103730 0x7fd480103b40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd470009900 tx=0x7fd47002a410 comp rx=0 tx=0).stop 2024-08-31T07:10:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.712+0000 7fd47f200640 1 -- 172.21.15.32:0/4097598410 shutdown_connections 2024-08-31T07:10:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.712+0000 7fd47f200640 1 --2- 172.21.15.32:0/4097598410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4801099c0 0x7fd4801101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.712+0000 7fd47f200640 1 --2- 172.21.15.32:0/4097598410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd480109070 0x7fd480109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.712+0000 7fd47f200640 1 --2- 172.21.15.32:0/4097598410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd480103730 0x7fd480103b40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.712+0000 7fd47f200640 1 -- 172.21.15.32:0/4097598410 >> 172.21.15.32:0/4097598410 conn(0x7fd4800ff180 msgr2=0x7fd4801015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.712+0000 7fd47f200640 1 -- 172.21.15.32:0/4097598410 shutdown_connections 2024-08-31T07:10:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.712+0000 7fd47f200640 1 -- 172.21.15.32:0/4097598410 wait complete. 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.713+0000 7fd47f200640 1 Processor -- start 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.713+0000 7fd47f200640 1 -- start start 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.713+0000 7fd47f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd480103730 0x7fd48019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.713+0000 7fd47f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd480109070 0x7fd48019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.713+0000 7fd47f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801099c0 0x7fd4801a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.713+0000 7fd47f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113690 con 0x7fd4801099c0 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.713+0000 7fd47f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113510 con 0x7fd480109070 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.713+0000 7fd47f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113810 con 0x7fd480103730 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801099c0 0x7fd4801a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801099c0 0x7fd4801a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37882/0 (socket says 172.21.15.32:37882) 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47e800640 1 -- 172.21.15.32:0/4067053420 learned_addr learned my addr 172.21.15.32:0/4067053420 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47d400640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd480109070 0x7fd48019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47de00640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd480103730 0x7fd48019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:04.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47e800640 1 -- 172.21.15.32:0/4067053420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd480103730 msgr2=0x7fd48019c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:04.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47e800640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd480103730 0x7fd48019c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:04.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47e800640 1 -- 172.21.15.32:0/4067053420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd480109070 msgr2=0x7fd48019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:04.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47e800640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd480109070 0x7fd48019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:04.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47e800640 1 -- 172.21.15.32:0/4067053420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4801a2270 con 0x7fd4801099c0 2024-08-31T07:10:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.714+0000 7fd47e800640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801099c0 0x7fd4801a1b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd47800eb10 tx=0x7fd47800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.715+0000 7fd476a00640 1 -- 172.21.15.32:0/4067053420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478004b20 con 0x7fd4801099c0 2024-08-31T07:10:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.715+0000 7fd47f200640 1 -- 172.21.15.32:0/4067053420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4801a2560 con 0x7fd4801099c0 2024-08-31T07:10:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.715+0000 7fd476a00640 1 -- 172.21.15.32:0/4067053420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd478004cc0 con 0x7fd4801099c0 2024-08-31T07:10:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.715+0000 7fd476a00640 1 -- 172.21.15.32:0/4067053420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478010670 con 0x7fd4801099c0 2024-08-31T07:10:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.715+0000 7fd47f200640 1 -- 172.21.15.32:0/4067053420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a8cc0 con 0x7fd4801099c0 2024-08-31T07:10:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.716+0000 7fd47f200640 1 -- 172.21.15.32:0/4067053420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd480103b40 con 0x7fd4801099c0 2024-08-31T07:10:04.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.718+0000 7fd476a00640 1 -- 172.21.15.32:0/4067053420 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fd478005000 con 0x7fd4801099c0 2024-08-31T07:10:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.721+0000 7fd476a00640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd4640b4b80 0x7fd4640b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.721+0000 7fd476a00640 1 -- 172.21.15.32:0/4067053420 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fd4780e5880 con 0x7fd4801099c0 2024-08-31T07:10:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.721+0000 7fd47de00640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd4640b4b80 0x7fd4640b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.722+0000 7fd47de00640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd4640b4b80 0x7fd4640b7090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd470031440 tx=0x7fd470009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.723+0000 7fd476a00640 1 -- 172.21.15.32:0/4067053420 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd478056130 con 0x7fd4801099c0 2024-08-31T07:10:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:04.961+0000 7fd47f200640 1 -- 172.21.15.32:0/4067053420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fd4801a8fa0 con 0x7fd4801099c0 2024-08-31T07:10:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:04 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] up:active 2024-08-31T07:10:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:04 smithi073 ceph-mon[21995]: fsmap cephfs:5 7 up:standby 5 up:active 2024-08-31T07:10:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:04 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:10:05.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:04 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] up:active 2024-08-31T07:10:05.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:04 smithi084 ceph-mon[22387]: fsmap cephfs:5 7 up:standby 5 up:active 2024-08-31T07:10:05.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:04 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:10:05.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:04 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] up:active 2024-08-31T07:10:05.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:04 smithi032 ceph-mon[17252]: fsmap cephfs:5 7 up:standby 5 up:active 2024-08-31T07:10:05.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:04 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:10:05.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.885+0000 7fd476a00640 1 -- 172.21.15.32:0/4067053420 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7fd4780ac9d0 con 0x7fd4801099c0 2024-08-31T07:10:05.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.888+0000 7fd47f200640 1 -- 172.21.15.32:0/4067053420 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd4640b4b80 msgr2=0x7fd4640b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:05.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.888+0000 7fd47f200640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd4640b4b80 0x7fd4640b7090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd470031440 tx=0x7fd470009560 comp rx=0 tx=0).stop 2024-08-31T07:10:05.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.888+0000 7fd47f200640 1 -- 172.21.15.32:0/4067053420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801099c0 msgr2=0x7fd4801a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.888+0000 7fd47f200640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801099c0 0x7fd4801a1b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd47800eb10 tx=0x7fd47800c2d0 comp rx=0 tx=0).stop 2024-08-31T07:10:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.888+0000 7fd47f200640 1 -- 172.21.15.32:0/4067053420 shutdown_connections 2024-08-31T07:10:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.889+0000 7fd47f200640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd4640b4b80 0x7fd4640b7090 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.889+0000 7fd47f200640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801099c0 0x7fd4801a1b40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.889+0000 7fd47f200640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd480109070 0x7fd48019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.889+0000 7fd47f200640 1 --2- 172.21.15.32:0/4067053420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd480103730 0x7fd48019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.889+0000 7fd47f200640 1 -- 172.21.15.32:0/4067053420 >> 172.21.15.32:0/4067053420 conn(0x7fd4800ff180 msgr2=0x7fd48010e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:05.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.889+0000 7fd47f200640 1 -- 172.21.15.32:0/4067053420 shutdown_connections 2024-08-31T07:10:05.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:05.889+0000 7fd47f200640 1 -- 172.21.15.32:0/4067053420 wait complete. 2024-08-31T07:10:06.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:05 smithi073 ceph-mon[21995]: pgmap v286: 97 pgs: 97 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.7 KiB/s wr, 8 op/s 2024-08-31T07:10:06.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:05 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4067053420' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-31T07:10:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:05 smithi084 ceph-mon[22387]: pgmap v286: 97 pgs: 97 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.7 KiB/s wr, 8 op/s 2024-08-31T07:10:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:05 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4067053420' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-31T07:10:06.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:05 smithi032 ceph-mon[17252]: pgmap v286: 97 pgs: 97 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.7 KiB/s wr, 8 op/s 2024-08-31T07:10:06.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:05 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4067053420' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-31T07:10:06.936 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-08-31T07:10:06.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-31T07:10:07.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:06 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4067053420' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-31T07:10:07.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:06 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:10:07.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:06 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4067053420' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-31T07:10:07.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:06 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:10:07.320 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:07.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:06 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4067053420' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-31T07:10:07.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:06 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:10:07.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.982+0000 7f8562200640 1 -- 172.21.15.32:0/122066774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855c109090 msgr2=0x7f855c1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:07.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.982+0000 7f8562200640 1 --2- 172.21.15.32:0/122066774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855c109090 0x7f855c1094a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8548009900 tx=0x7f854802a410 comp rx=0 tx=0).stop 2024-08-31T07:10:07.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.983+0000 7f8562200640 1 -- 172.21.15.32:0/122066774 shutdown_connections 2024-08-31T07:10:07.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.983+0000 7f8562200640 1 --2- 172.21.15.32:0/122066774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f855c1099e0 0x7f855c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:07.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.983+0000 7f8562200640 1 --2- 172.21.15.32:0/122066774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855c109090 0x7f855c1094a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:07.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.983+0000 7f8562200640 1 --2- 172.21.15.32:0/122066774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f855c103750 0x7f855c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:07.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.983+0000 7f8562200640 1 -- 172.21.15.32:0/122066774 >> 172.21.15.32:0/122066774 conn(0x7f855c0ff1a0 msgr2=0x7f855c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:07.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.983+0000 7f8562200640 1 -- 172.21.15.32:0/122066774 shutdown_connections 2024-08-31T07:10:07.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.983+0000 7f8562200640 1 -- 172.21.15.32:0/122066774 wait complete. 2024-08-31T07:10:07.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.984+0000 7f8562200640 1 Processor -- start 2024-08-31T07:10:07.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.984+0000 7f8562200640 1 -- start start 2024-08-31T07:10:07.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.984+0000 7f8562200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f855c103750 0x7f855c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.984+0000 7f8562200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855c109090 0x7f855c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8562200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f855c1099e0 0x7f855c1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8562200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f855c1136c0 con 0x7f855c109090 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8562200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f855c113540 con 0x7f855c1099e0 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8562200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f855c113840 con 0x7f855c103750 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f855be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855c109090 0x7f855c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8560e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f855c103750 0x7f855c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f855be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855c109090 0x7f855c19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39220/0 (socket says 172.21.15.32:39220) 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8560e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f855c103750 0x7f855c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41720/0 (socket says 172.21.15.32:41720) 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f855be00640 1 -- 172.21.15.32:0/3931532191 learned_addr learned my addr 172.21.15.32:0/3931532191 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8561800640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f855c1099e0 0x7f855c1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8560e00640 1 -- 172.21.15.32:0/3931532191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f855c1099e0 msgr2=0x7f855c1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8560e00640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f855c1099e0 0x7f855c1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8560e00640 1 -- 172.21.15.32:0/3931532191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855c109090 msgr2=0x7f855c19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8560e00640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855c109090 0x7f855c19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f8560e00640 1 -- 172.21.15.32:0/3931532191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c1a2220 con 0x7f855c103750 2024-08-31T07:10:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.985+0000 7f855be00640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855c109090 0x7f855c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.986+0000 7f8560e00640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f855c103750 0x7f855c19c2b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f855000e9f0 tx=0x7f855000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.986+0000 7f8559600640 1 -- 172.21.15.32:0/3931532191 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855000ccc0 con 0x7f855c103750 2024-08-31T07:10:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.986+0000 7f8559600640 1 -- 172.21.15.32:0/3931532191 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85500108e0 con 0x7f855c103750 2024-08-31T07:10:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.986+0000 7f8559600640 1 -- 172.21.15.32:0/3931532191 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550018880 con 0x7f855c103750 2024-08-31T07:10:07.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.986+0000 7f8562200640 1 -- 172.21.15.32:0/3931532191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855c1a2510 con 0x7f855c103750 2024-08-31T07:10:07.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.986+0000 7f8562200640 1 -- 172.21.15.32:0/3931532191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855c1a8c70 con 0x7f855c103750 2024-08-31T07:10:07.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.987+0000 7f8562200640 1 -- 172.21.15.32:0/3931532191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f855c103b60 con 0x7f855c103750 2024-08-31T07:10:07.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.990+0000 7f8559600640 1 -- 172.21.15.32:0/3931532191 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f8550010430 con 0x7f855c103750 2024-08-31T07:10:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.991+0000 7f8559600640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85440b4c00 0x7f85440b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.991+0000 7f8559600640 1 -- 172.21.15.32:0/3931532191 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f85500e5460 con 0x7f855c103750 2024-08-31T07:10:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.991+0000 7f855be00640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85440b4c00 0x7f85440b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.992+0000 7f855be00640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85440b4c00 0x7f85440b7110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8548006010 tx=0x7f8548005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:07.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:07.993+0000 7f8559600640 1 -- 172.21.15.32:0/3931532191 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f85500abf00 con 0x7f855c103750 2024-08-31T07:10:08.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:07 smithi032 ceph-mon[17252]: pgmap v287: 97 pgs: 97 active+clean; 584 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-08-31T07:10:08.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:07 smithi073 ceph-mon[21995]: pgmap v287: 97 pgs: 97 active+clean; 584 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-08-31T07:10:08.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.215+0000 7f8562200640 1 -- 172.21.15.32:0/3931532191 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f855c10fbf0 con 0x7f85440b4c00 2024-08-31T07:10:08.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:07 smithi084 ceph-mon[22387]: pgmap v287: 97 pgs: 97 active+clean; 584 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-08-31T07:10:08.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.256+0000 7f8559600640 1 -- 172.21.15.32:0/3931532191 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f855c10fbf0 con 0x7f85440b4c00 2024-08-31T07:10:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.258+0000 7f8562200640 1 -- 172.21.15.32:0/3931532191 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85440b4c00 msgr2=0x7f85440b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.258+0000 7f8562200640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85440b4c00 0x7f85440b7110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8548006010 tx=0x7f8548005a20 comp rx=0 tx=0).stop 2024-08-31T07:10:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.258+0000 7f8562200640 1 -- 172.21.15.32:0/3931532191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f855c103750 msgr2=0x7f855c19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.258+0000 7f8562200640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f855c103750 0x7f855c19c2b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f855000e9f0 tx=0x7f855000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:10:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.258+0000 7f8562200640 1 -- 172.21.15.32:0/3931532191 shutdown_connections 2024-08-31T07:10:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.258+0000 7f8562200640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85440b4c00 0x7f85440b7110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.259+0000 7f8562200640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f855c1099e0 0x7f855c1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.259+0000 7f8562200640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855c109090 0x7f855c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.259+0000 7f8562200640 1 --2- 172.21.15.32:0/3931532191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f855c103750 0x7f855c19c2b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.259+0000 7f8562200640 1 -- 172.21.15.32:0/3931532191 >> 172.21.15.32:0/3931532191 conn(0x7f855c0ff1a0 msgr2=0x7f855c10e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:08.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.259+0000 7f8562200640 1 -- 172.21.15.32:0/3931532191 shutdown_connections 2024-08-31T07:10:08.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:08.259+0000 7f8562200640 1 -- 172.21.15.32:0/3931532191 wait complete. 2024-08-31T07:10:08.596 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:10:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:10:08.223+0000 7f5f2d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T07:10:08.597 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:10:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:10:08.223+0000 7f5f2d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T07:10:08.597 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:10:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:10:08.223+0000 7f5f2d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T07:10:08.597 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:10:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:10:08.223+0000 7f5f2d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T07:10:08.597 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:10:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:10:08.223+0000 7f5f2d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T07:10:08.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-31T07:10:09.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:08 smithi073 ceph-mon[21995]: from='client.24802 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:09.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:08 smithi084 ceph-mon[22387]: from='client.24802 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:09.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:08 smithi032 ceph-mon[17252]: from='client.24802 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.002+0000 7f7e21200640 1 -- 172.21.15.32:0/2878164226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e1c105b20 msgr2=0x7f7e1c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.002+0000 7f7e21200640 1 --2- 172.21.15.32:0/2878164226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e1c105b20 0x7f7e1c105f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009900 tx=0x7f7e0c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.002+0000 7f7e21200640 1 -- 172.21.15.32:0/2878164226 shutdown_connections 2024-08-31T07:10:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.002+0000 7f7e21200640 1 --2- 172.21.15.32:0/2878164226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c10bdb0 0x7f7e1c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.002+0000 7f7e21200640 1 --2- 172.21.15.32:0/2878164226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e1c10b460 0x7f7e1c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.002+0000 7f7e21200640 1 --2- 172.21.15.32:0/2878164226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e1c105b20 0x7f7e1c105f30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.002+0000 7f7e21200640 1 -- 172.21.15.32:0/2878164226 >> 172.21.15.32:0/2878164226 conn(0x7f7e1c1015b0 msgr2=0x7f7e1c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.002+0000 7f7e21200640 1 -- 172.21.15.32:0/2878164226 shutdown_connections 2024-08-31T07:10:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.002+0000 7f7e21200640 1 -- 172.21.15.32:0/2878164226 wait complete. 2024-08-31T07:10:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.003+0000 7f7e21200640 1 Processor -- start 2024-08-31T07:10:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.003+0000 7f7e21200640 1 -- start start 2024-08-31T07:10:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.003+0000 7f7e21200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e1c105b20 0x7f7e1c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e21200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c10b460 0x7f7e1c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e21200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e1c10bdb0 0x7f7e1c1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e21200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c115ac0 con 0x7f7e1c105b20 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e21200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c115940 con 0x7f7e1c10bdb0 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e21200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c115c40 con 0x7f7e1c10b460 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e19e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e1c105b20 0x7f7e1c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e19e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e1c105b20 0x7f7e1c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39244/0 (socket says 172.21.15.32:39244) 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e19e00640 1 -- 172.21.15.32:0/59509299 learned_addr learned my addr 172.21.15.32:0/59509299 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e13e00640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c10b460 0x7f7e1c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e1a800640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e1c10bdb0 0x7f7e1c1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e19e00640 1 -- 172.21.15.32:0/59509299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c10b460 msgr2=0x7f7e1c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e19e00640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c10b460 0x7f7e1c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e19e00640 1 -- 172.21.15.32:0/59509299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e1c10bdb0 msgr2=0x7f7e1c1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e19e00640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e1c10bdb0 0x7f7e1c1a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e19e00640 1 -- 172.21.15.32:0/59509299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c1a23c0 con 0x7f7e1c105b20 2024-08-31T07:10:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.004+0000 7f7e13e00640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c10b460 0x7f7e1c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.005+0000 7f7e19e00640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e1c105b20 0x7f7e1c19c450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c005ab0 tx=0x7f7e0c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.005+0000 7f7e12a00640 1 -- 172.21.15.32:0/59509299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c03f070 con 0x7f7e1c105b20 2024-08-31T07:10:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.005+0000 7f7e12a00640 1 -- 172.21.15.32:0/59509299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e0c031d60 con 0x7f7e1c105b20 2024-08-31T07:10:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.005+0000 7f7e21200640 1 -- 172.21.15.32:0/59509299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1c1a2650 con 0x7f7e1c105b20 2024-08-31T07:10:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.005+0000 7f7e12a00640 1 -- 172.21.15.32:0/59509299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c035030 con 0x7f7e1c105b20 2024-08-31T07:10:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.005+0000 7f7e21200640 1 -- 172.21.15.32:0/59509299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1c1a8e10 con 0x7f7e1c105b20 2024-08-31T07:10:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.007+0000 7f7e21200640 1 -- 172.21.15.32:0/59509299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e1c105f30 con 0x7f7e1c105b20 2024-08-31T07:10:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.011+0000 7f7e12a00640 1 -- 172.21.15.32:0/59509299 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f7e0c042070 con 0x7f7e1c105b20 2024-08-31T07:10:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.012+0000 7f7e12a00640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7e000b4c10 0x7f7e000b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.012+0000 7f7e12a00640 1 -- 172.21.15.32:0/59509299 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f7e0c106240 con 0x7f7e1c105b20 2024-08-31T07:10:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.012+0000 7f7e13e00640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7e000b4c10 0x7f7e000b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.013+0000 7f7e13e00640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7e000b4c10 0x7f7e000b7120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c10af30 tx=0x7f7e0400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.014+0000 7f7e12a00640 1 -- 172.21.15.32:0/59509299 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7e0c0cd230 con 0x7f7e1c105b20 2024-08-31T07:10:10.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:09 smithi032 ceph-mon[17252]: pgmap v288: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-08-31T07:10:10.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:09 smithi032 ceph-mon[17252]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-08-31T07:10:10.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:09 smithi073 ceph-mon[21995]: pgmap v288: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-08-31T07:10:10.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:09 smithi073 ceph-mon[21995]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-08-31T07:10:10.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.239+0000 7f7e21200640 1 -- 172.21.15.32:0/59509299 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7e1c10dbe0 con 0x7f7e000b4c10 2024-08-31T07:10:10.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:09 smithi084 ceph-mon[22387]: pgmap v288: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-08-31T07:10:10.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:09 smithi084 ceph-mon[22387]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-08-31T07:10:10.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.266+0000 7f7e12a00640 1 -- 172.21.15.32:0/59509299 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e1c10dbe0 con 0x7f7e000b4c10 2024-08-31T07:10:10.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 -- 172.21.15.32:0/59509299 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7e000b4c10 msgr2=0x7f7e000b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:10.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7e000b4c10 0x7f7e000b7120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c10af30 tx=0x7f7e0400b390 comp rx=0 tx=0).stop 2024-08-31T07:10:10.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 -- 172.21.15.32:0/59509299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e1c105b20 msgr2=0x7f7e1c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:10.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e1c105b20 0x7f7e1c19c450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c005ab0 tx=0x7f7e0c003cd0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 -- 172.21.15.32:0/59509299 shutdown_connections 2024-08-31T07:10:10.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7e000b4c10 0x7f7e000b7120 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e1c10bdb0 0x7f7e1c1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e1c10b460 0x7f7e1c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 --2- 172.21.15.32:0/59509299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e1c105b20 0x7f7e1c19c450 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 -- 172.21.15.32:0/59509299 >> 172.21.15.32:0/59509299 conn(0x7f7e1c1015b0 msgr2=0x7f7e1c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 -- 172.21.15.32:0/59509299 shutdown_connections 2024-08-31T07:10:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:10.269+0000 7f7e21200640 1 -- 172.21.15.32:0/59509299 wait complete. 2024-08-31T07:10:11.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-31T07:10:11.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:10 smithi073 ceph-mon[21995]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:11.184 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:10 smithi032 ceph-mon[17252]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:11.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:10 smithi084 ceph-mon[22387]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.145+0000 7facb3000640 1 -- 172.21.15.32:0/2243886268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac1099e0 msgr2=0x7facac110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.145+0000 7facb3000640 1 --2- 172.21.15.32:0/2243886268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac1099e0 0x7facac110210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faca4009ca0 tx=0x7faca402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:10:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.145+0000 7facb3000640 1 -- 172.21.15.32:0/2243886268 shutdown_connections 2024-08-31T07:10:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.145+0000 7facb3000640 1 --2- 172.21.15.32:0/2243886268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac1099e0 0x7facac110210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.145+0000 7facb3000640 1 --2- 172.21.15.32:0/2243886268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7facac109090 0x7facac1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.145+0000 7facb3000640 1 --2- 172.21.15.32:0/2243886268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac103750 0x7facac103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.145+0000 7facb3000640 1 -- 172.21.15.32:0/2243886268 >> 172.21.15.32:0/2243886268 conn(0x7facac0ff1e0 msgr2=0x7facac101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.145+0000 7facb3000640 1 -- 172.21.15.32:0/2243886268 shutdown_connections 2024-08-31T07:10:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.145+0000 7facb3000640 1 -- 172.21.15.32:0/2243886268 wait complete. 2024-08-31T07:10:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.146+0000 7facb3000640 1 Processor -- start 2024-08-31T07:10:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.146+0000 7facb3000640 1 -- start start 2024-08-31T07:10:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb3000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7facac103750 0x7facac19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb3000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac109090 0x7facac19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb3000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac1099e0 0x7facac19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb3000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7facac1136b0 con 0x7facac1099e0 2024-08-31T07:10:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb3000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7facac113530 con 0x7facac103750 2024-08-31T07:10:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb3000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7facac113830 con 0x7facac109090 2024-08-31T07:10:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb2600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac1099e0 0x7facac19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac109090 0x7facac19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb2600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac1099e0 0x7facac19f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39258/0 (socket says 172.21.15.32:39258) 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb2600640 1 -- 172.21.15.32:0/2877825035 learned_addr learned my addr 172.21.15.32:0/2877825035 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb1c00640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7facac103750 0x7facac19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.147+0000 7facb2600640 1 -- 172.21.15.32:0/2877825035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac109090 msgr2=0x7facac19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.148+0000 7facb2600640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac109090 0x7facac19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.148+0000 7facb2600640 1 -- 172.21.15.32:0/2877825035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7facac103750 msgr2=0x7facac19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.148+0000 7facb2600640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7facac103750 0x7facac19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.148+0000 7facb2600640 1 -- 172.21.15.32:0/2877825035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac1a0010 con 0x7facac1099e0 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.148+0000 7facb2600640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac1099e0 0x7facac19f8e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faca4005cc0 tx=0x7faca4031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.148+0000 7facaa000640 1 -- 172.21.15.32:0/2877825035 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4003b70 con 0x7facac1099e0 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.148+0000 7facaa000640 1 -- 172.21.15.32:0/2877825035 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca40318f0 con 0x7facac1099e0 2024-08-31T07:10:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.148+0000 7facaa000640 1 -- 172.21.15.32:0/2877825035 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca402f950 con 0x7facac1099e0 2024-08-31T07:10:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.148+0000 7facb3000640 1 -- 172.21.15.32:0/2877825035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac1a02a0 con 0x7facac1099e0 2024-08-31T07:10:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.148+0000 7facb3000640 1 -- 172.21.15.32:0/2877825035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac1a6a60 con 0x7facac1099e0 2024-08-31T07:10:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.150+0000 7facb3000640 1 -- 172.21.15.32:0/2877825035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facac103b60 con 0x7facac1099e0 2024-08-31T07:10:12.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.152+0000 7facaa000640 1 -- 172.21.15.32:0/2877825035 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7faca4004b40 con 0x7facac1099e0 2024-08-31T07:10:12.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.153+0000 7facaa000640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fac940b4c10 0x7fac940b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:12.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.154+0000 7facaa000640 1 -- 172.21.15.32:0/2877825035 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7faca4106660 con 0x7facac1099e0 2024-08-31T07:10:12.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.154+0000 7facb1c00640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fac940b4c10 0x7fac940b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:12.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.154+0000 7facb1c00640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fac940b4c10 0x7fac940b7120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faca0009bd0 tx=0x7faca0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:12.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.156+0000 7facaa000640 1 -- 172.21.15.32:0/2877825035 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faca40cd5f0 con 0x7facac1099e0 2024-08-31T07:10:12.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:11 smithi073 ceph-mon[21995]: pgmap v289: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-08-31T07:10:12.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:11 smithi084 ceph-mon[22387]: pgmap v289: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-08-31T07:10:12.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:11 smithi032 ceph-mon[17252]: pgmap v289: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-08-31T07:10:12.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.379+0000 7facb3000640 1 -- 172.21.15.32:0/2877825035 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7facac0635a0 con 0x7fac940b4c10 2024-08-31T07:10:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.385+0000 7facaa000640 1 -- 172.21.15.32:0/2877825035 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7facac0635a0 con 0x7fac940b4c10 2024-08-31T07:10:12.386 INFO:teuthology.orchestra.run.smithi032.stdout:/volumes/qa/sv_0/94c8df5e-e5c9-4c53-8aa9-89d822d362df 2024-08-31T07:10:12.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 -- 172.21.15.32:0/2877825035 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fac940b4c10 msgr2=0x7fac940b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:12.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fac940b4c10 0x7fac940b7120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faca0009bd0 tx=0x7faca0005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:12.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 -- 172.21.15.32:0/2877825035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac1099e0 msgr2=0x7facac19f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:12.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac1099e0 0x7facac19f8e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faca4005cc0 tx=0x7faca4031700 comp rx=0 tx=0).stop 2024-08-31T07:10:12.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 -- 172.21.15.32:0/2877825035 shutdown_connections 2024-08-31T07:10:12.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fac940b4c10 0x7fac940b7120 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:12.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac1099e0 0x7facac19f8e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:12.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facac109090 0x7facac19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:12.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 --2- 172.21.15.32:0/2877825035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7facac103750 0x7facac19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:12.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 -- 172.21.15.32:0/2877825035 >> 172.21.15.32:0/2877825035 conn(0x7facac0ff1e0 msgr2=0x7facac101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:12.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.387+0000 7facb3000640 1 -- 172.21.15.32:0/2877825035 shutdown_connections 2024-08-31T07:10:12.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:12.388+0000 7facb3000640 1 -- 172.21.15.32:0/2877825035 wait complete. 2024-08-31T07:10:13.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:12 smithi032 ceph-mon[17252]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:13.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:12 smithi073 ceph-mon[21995]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:13.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:12 smithi084 ceph-mon[22387]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:13.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-31T07:10:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:14.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:13 smithi032 ceph-mon[17252]: pgmap v290: 97 pgs: 97 active+clean; 611 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-08-31T07:10:14.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:13 smithi073 ceph-mon[21995]: pgmap v290: 97 pgs: 97 active+clean; 611 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-08-31T07:10:14.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:13 smithi084 ceph-mon[22387]: pgmap v290: 97 pgs: 97 active+clean; 611 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-08-31T07:10:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.248+0000 7f51d6200640 1 -- 172.21.15.32:0/1967779102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d010b290 msgr2=0x7f51d010b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.248+0000 7f51d6200640 1 --2- 172.21.15.32:0/1967779102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d010b290 0x7f51d010b6a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009900 tx=0x7f51bc02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:10:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.249+0000 7f51d6200640 1 -- 172.21.15.32:0/1967779102 shutdown_connections 2024-08-31T07:10:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.249+0000 7f51d6200640 1 --2- 172.21.15.32:0/1967779102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d010bbe0 0x7f51d0112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.249+0000 7f51d6200640 1 --2- 172.21.15.32:0/1967779102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d010b290 0x7f51d010b6a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.249+0000 7f51d6200640 1 --2- 172.21.15.32:0/1967779102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0105950 0x7f51d0105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.249+0000 7f51d6200640 1 -- 172.21.15.32:0/1967779102 >> 172.21.15.32:0/1967779102 conn(0x7f51d0101420 msgr2=0x7f51d0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:14.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.249+0000 7f51d6200640 1 -- 172.21.15.32:0/1967779102 shutdown_connections 2024-08-31T07:10:14.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.249+0000 7f51d6200640 1 -- 172.21.15.32:0/1967779102 wait complete. 2024-08-31T07:10:14.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.250+0000 7f51d6200640 1 Processor -- start 2024-08-31T07:10:14.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.250+0000 7f51d6200640 1 -- start start 2024-08-31T07:10:14.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.250+0000 7f51d6200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0105950 0x7f51d019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:14.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.250+0000 7f51d6200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d010b290 0x7f51d019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:14.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.250+0000 7f51d6200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d010bbe0 0x7f51d01a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:14.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.250+0000 7f51d6200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f51d01158a0 con 0x7f51d0105950 2024-08-31T07:10:14.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.250+0000 7f51d6200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0115720 con 0x7f51d010bbe0 2024-08-31T07:10:14.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.250+0000 7f51d6200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0115a20 con 0x7f51d010b290 2024-08-31T07:10:14.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0105950 0x7f51d019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0105950 0x7f51d019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39284/0 (socket says 172.21.15.32:39284) 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d4e00640 1 -- 172.21.15.32:0/184953958 learned_addr learned my addr 172.21.15.32:0/184953958 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d5800640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d010bbe0 0x7f51d01a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51cfe00640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d010b290 0x7f51d019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d4e00640 1 -- 172.21.15.32:0/184953958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d010b290 msgr2=0x7f51d019c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d4e00640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d010b290 0x7f51d019c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d4e00640 1 -- 172.21.15.32:0/184953958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d010bbe0 msgr2=0x7f51d01a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d4e00640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d010bbe0 0x7f51d01a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d4e00640 1 -- 172.21.15.32:0/184953958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d01a2220 con 0x7f51d0105950 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d5800640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d010bbe0 0x7f51d01a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:14.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.251+0000 7f51d4e00640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0105950 0x7f51d019c2b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f51c400ea60 tx=0x7f51c400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.252+0000 7f51cd600640 1 -- 172.21.15.32:0/184953958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c400cde0 con 0x7f51d0105950 2024-08-31T07:10:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.252+0000 7f51d6200640 1 -- 172.21.15.32:0/184953958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d01a2510 con 0x7f51d0105950 2024-08-31T07:10:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.252+0000 7f51cd600640 1 -- 172.21.15.32:0/184953958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51c40035f0 con 0x7f51d0105950 2024-08-31T07:10:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.252+0000 7f51cd600640 1 -- 172.21.15.32:0/184953958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4010690 con 0x7f51d0105950 2024-08-31T07:10:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.252+0000 7f51d6200640 1 -- 172.21.15.32:0/184953958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d01a8c70 con 0x7f51d0105950 2024-08-31T07:10:14.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.253+0000 7f51d6200640 1 -- 172.21.15.32:0/184953958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d0105d60 con 0x7f51d0105950 2024-08-31T07:10:14.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.255+0000 7f51cd600640 1 -- 172.21.15.32:0/184953958 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f51c40037a0 con 0x7f51d0105950 2024-08-31T07:10:14.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.257+0000 7f51cd600640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51b80b4c10 0x7f51b80b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:14.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.257+0000 7f51cd600640 1 -- 172.21.15.32:0/184953958 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f51c40e5a40 con 0x7f51d0105950 2024-08-31T07:10:14.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.257+0000 7f51cfe00640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51b80b4c10 0x7f51b80b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:14.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.258+0000 7f51cfe00640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51b80b4c10 0x7f51b80b7120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009230 tx=0x7f51bc031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:14.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.259+0000 7f51cd600640 1 -- 172.21.15.32:0/184953958 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f51c40ac9d0 con 0x7f51d0105950 2024-08-31T07:10:14.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.484+0000 7f51d6200640 1 -- 172.21.15.32:0/184953958 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f51d00635a0 con 0x7f51b80b4c10 2024-08-31T07:10:14.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.512+0000 7f51cd600640 1 -- 172.21.15.32:0/184953958 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f51d00635a0 con 0x7f51b80b4c10 2024-08-31T07:10:14.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.514+0000 7f51d6200640 1 -- 172.21.15.32:0/184953958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51b80b4c10 msgr2=0x7f51b80b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:14.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.514+0000 7f51d6200640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51b80b4c10 0x7f51b80b7120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009230 tx=0x7f51bc031400 comp rx=0 tx=0).stop 2024-08-31T07:10:14.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.514+0000 7f51d6200640 1 -- 172.21.15.32:0/184953958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0105950 msgr2=0x7f51d019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:14.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.514+0000 7f51d6200640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0105950 0x7f51d019c2b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f51c400ea60 tx=0x7f51c400ef20 comp rx=0 tx=0).stop 2024-08-31T07:10:14.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.514+0000 7f51d6200640 1 -- 172.21.15.32:0/184953958 shutdown_connections 2024-08-31T07:10:14.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.514+0000 7f51d6200640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51b80b4c10 0x7f51b80b7120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.514+0000 7f51d6200640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d010bbe0 0x7f51d01a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.514+0000 7f51d6200640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d010b290 0x7f51d019c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.514+0000 7f51d6200640 1 --2- 172.21.15.32:0/184953958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0105950 0x7f51d019c2b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.514+0000 7f51d6200640 1 -- 172.21.15.32:0/184953958 >> 172.21.15.32:0/184953958 conn(0x7f51d0101420 msgr2=0x7f51d0103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.515+0000 7f51d6200640 1 -- 172.21.15.32:0/184953958 shutdown_connections 2024-08-31T07:10:14.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:14.515+0000 7f51d6200640 1 -- 172.21.15.32:0/184953958 wait complete. 2024-08-31T07:10:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:14 smithi073 ceph-mon[21995]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:15.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:14 smithi084 ceph-mon[22387]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:15.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:14 smithi032 ceph-mon[17252]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:15.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-31T07:10:15.831 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:16.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:15 smithi073 ceph-mon[21995]: pgmap v291: 97 pgs: 97 active+clean; 611 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 22 op/s 2024-08-31T07:10:16.186 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:15 smithi032 ceph-mon[17252]: pgmap v291: 97 pgs: 97 active+clean; 611 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 22 op/s 2024-08-31T07:10:16.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:15 smithi084 ceph-mon[22387]: pgmap v291: 97 pgs: 97 active+clean; 611 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 22 op/s 2024-08-31T07:10:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.490+0000 7f3d10a00640 1 -- 172.21.15.32:0/3081405051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d0c078030 msgr2=0x7f3d0c113dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.490+0000 7f3d10a00640 1 --2- 172.21.15.32:0/3081405051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d0c078030 0x7f3d0c113dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009900 tx=0x7f3cf402a410 comp rx=0 tx=0).stop 2024-08-31T07:10:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.490+0000 7f3d10a00640 1 -- 172.21.15.32:0/3081405051 shutdown_connections 2024-08-31T07:10:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.490+0000 7f3d10a00640 1 --2- 172.21.15.32:0/3081405051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d0c114310 0x7f3d0c116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.490+0000 7f3d10a00640 1 --2- 172.21.15.32:0/3081405051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d0c078030 0x7f3d0c113dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.490+0000 7f3d10a00640 1 --2- 172.21.15.32:0/3081405051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d0c0776e0 0x7f3d0c077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.490+0000 7f3d10a00640 1 -- 172.21.15.32:0/3081405051 >> 172.21.15.32:0/3081405051 conn(0x7f3d0c101420 msgr2=0x7f3d0c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.490+0000 7f3d10a00640 1 -- 172.21.15.32:0/3081405051 shutdown_connections 2024-08-31T07:10:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.490+0000 7f3d10a00640 1 -- 172.21.15.32:0/3081405051 wait complete. 2024-08-31T07:10:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.491+0000 7f3d10a00640 1 Processor -- start 2024-08-31T07:10:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.491+0000 7f3d10a00640 1 -- start start 2024-08-31T07:10:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.491+0000 7f3d10a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d0c0776e0 0x7f3d0c1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d10a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d0c078030 0x7f3d0c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d10a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d0c114310 0x7f3d0c1a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d10a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c119c70 con 0x7f3d0c0776e0 2024-08-31T07:10:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d10a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c119af0 con 0x7f3d0c114310 2024-08-31T07:10:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d10a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c119df0 con 0x7f3d0c078030 2024-08-31T07:10:16.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d03e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d0c078030 0x7f3d0c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:16.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d0b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d0c0776e0 0x7f3d0c1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:16.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d03e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d0c078030 0x7f3d0c1a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:39312/0 (socket says 172.21.15.32:39312) 2024-08-31T07:10:16.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d03e00640 1 -- 172.21.15.32:0/1853379642 learned_addr learned my addr 172.21.15.32:0/1853379642 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d0be00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d0c114310 0x7f3d0c1a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d03e00640 1 -- 172.21.15.32:0/1853379642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d0c114310 msgr2=0x7f3d0c1a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d03e00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d0c114310 0x7f3d0c1a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d03e00640 1 -- 172.21.15.32:0/1853379642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d0c0776e0 msgr2=0x7f3d0c1a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d03e00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d0c0776e0 0x7f3d0c1a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d03e00640 1 -- 172.21.15.32:0/1853379642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c1a6690 con 0x7f3d0c078030 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.492+0000 7f3d0b400640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d0c0776e0 0x7f3d0c1a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.493+0000 7f3d03e00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d0c078030 0x7f3d0c1a0cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4006010 tx=0x7f3cf4004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.493+0000 7f3d08c00640 1 -- 172.21.15.32:0/1853379642 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf403f030 con 0x7f3d0c078030 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.493+0000 7f3d10a00640 1 -- 172.21.15.32:0/1853379642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c1a6920 con 0x7f3d0c078030 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.493+0000 7f3d08c00640 1 -- 172.21.15.32:0/1853379642 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf4003aa0 con 0x7f3d0c078030 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.493+0000 7f3d08c00640 1 -- 172.21.15.32:0/1853379642 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4035070 con 0x7f3d0c078030 2024-08-31T07:10:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.493+0000 7f3d10a00640 1 -- 172.21.15.32:0/1853379642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1ad0e0 con 0x7f3d0c078030 2024-08-31T07:10:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.494+0000 7f3d10a00640 1 -- 172.21.15.32:0/1853379642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c078bf0 con 0x7f3d0c078030 2024-08-31T07:10:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.497+0000 7f3d08c00640 1 -- 172.21.15.32:0/1853379642 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f3cf4042030 con 0x7f3d0c078030 2024-08-31T07:10:16.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.498+0000 7f3d08c00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cf00b4c10 0x7f3cf00b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:16.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.499+0000 7f3d08c00640 1 -- 172.21.15.32:0/1853379642 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f3cf4105b90 con 0x7f3d0c078030 2024-08-31T07:10:16.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.499+0000 7f3d0b400640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cf00b4c10 0x7f3cf00b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:16.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.499+0000 7f3d0b400640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cf00b4c10 0x7f3cf00b7120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009fc0 tx=0x7f3cfc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:16.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.501+0000 7f3d08c00640 1 -- 172.21.15.32:0/1853379642 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3cf40cc4d0 con 0x7f3d0c078030 2024-08-31T07:10:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.726+0000 7f3d10a00640 1 -- 172.21.15.32:0/1853379642 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f3d0c0635a0 con 0x7f3cf00b4c10 2024-08-31T07:10:16.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.732+0000 7f3d08c00640 1 -- 172.21.15.32:0/1853379642 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3d0c0635a0 con 0x7f3cf00b4c10 2024-08-31T07:10:16.733 INFO:teuthology.orchestra.run.smithi032.stdout:/volumes/qa/sv_1/aeeafabb-ed2a-477b-bc60-8858e680a053 2024-08-31T07:10:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 -- 172.21.15.32:0/1853379642 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cf00b4c10 msgr2=0x7f3cf00b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cf00b4c10 0x7f3cf00b7120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009fc0 tx=0x7f3cfc005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 -- 172.21.15.32:0/1853379642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d0c078030 msgr2=0x7f3d0c1a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d0c078030 0x7f3d0c1a0cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4006010 tx=0x7f3cf4004060 comp rx=0 tx=0).stop 2024-08-31T07:10:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 -- 172.21.15.32:0/1853379642 shutdown_connections 2024-08-31T07:10:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cf00b4c10 0x7f3cf00b7120 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d0c114310 0x7f3d0c1a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d0c078030 0x7f3d0c1a0cc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 --2- 172.21.15.32:0/1853379642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d0c0776e0 0x7f3d0c1a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 -- 172.21.15.32:0/1853379642 >> 172.21.15.32:0/1853379642 conn(0x7f3d0c101420 msgr2=0x7f3d0c116470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 -- 172.21.15.32:0/1853379642 shutdown_connections 2024-08-31T07:10:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:16.734+0000 7f3d10a00640 1 -- 172.21.15.32:0/1853379642 wait complete. 2024-08-31T07:10:17.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:10:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:17 smithi084 ceph-mon[22387]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:17 smithi084 ceph-mon[22387]: pgmap v292: 97 pgs: 97 active+clean; 611 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.9 KiB/s wr, 18 op/s 2024-08-31T07:10:18.255 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:17 smithi032 ceph-mon[17252]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:18.255 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:17 smithi032 ceph-mon[17252]: pgmap v292: 97 pgs: 97 active+clean; 611 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.9 KiB/s wr, 18 op/s 2024-08-31T07:10:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:17 smithi073 ceph-mon[21995]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:10:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:17 smithi073 ceph-mon[21995]: pgmap v292: 97 pgs: 97 active+clean; 611 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.9 KiB/s wr, 18 op/s 2024-08-31T07:10:18.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.588+0000 7f66edc00640 1 -- 172.21.15.32:0/366909963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e8109090 msgr2=0x7f66e81094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:18.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.588+0000 7f66edc00640 1 --2- 172.21.15.32:0/366909963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e8109090 0x7f66e81094a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009900 tx=0x7f66d402a410 comp rx=0 tx=0).stop 2024-08-31T07:10:18.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.588+0000 7f66edc00640 1 -- 172.21.15.32:0/366909963 shutdown_connections 2024-08-31T07:10:18.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.588+0000 7f66edc00640 1 --2- 172.21.15.32:0/366909963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66e81099e0 0x7f66e8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:18.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.588+0000 7f66edc00640 1 --2- 172.21.15.32:0/366909963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e8109090 0x7f66e81094a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:18.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.588+0000 7f66edc00640 1 --2- 172.21.15.32:0/366909963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e8103750 0x7f66e8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:18.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.588+0000 7f66edc00640 1 -- 172.21.15.32:0/366909963 >> 172.21.15.32:0/366909963 conn(0x7f66e80ff1e0 msgr2=0x7f66e8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:18.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.588+0000 7f66edc00640 1 -- 172.21.15.32:0/366909963 shutdown_connections 2024-08-31T07:10:18.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.588+0000 7f66edc00640 1 -- 172.21.15.32:0/366909963 wait complete. 2024-08-31T07:10:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.590+0000 7f66edc00640 1 Processor -- start 2024-08-31T07:10:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.590+0000 7f66edc00640 1 -- start start 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.590+0000 7f66edc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66e8103750 0x7f66e819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.590+0000 7f66edc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e8109090 0x7f66e819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.590+0000 7f66edc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e81099e0 0x7f66e81a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.590+0000 7f66edc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8113650 con 0x7f66e81099e0 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.590+0000 7f66edc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f66e81134d0 con 0x7f66e8109090 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.590+0000 7f66edc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f66e81137d0 con 0x7f66e8103750 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.590+0000 7f66ed200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e81099e0 0x7f66e81a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.590+0000 7f66ed200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e81099e0 0x7f66e81a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39358/0 (socket says 172.21.15.32:39358) 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.591+0000 7f66ed200640 1 -- 172.21.15.32:0/1508983157 learned_addr learned my addr 172.21.15.32:0/1508983157 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.591+0000 7f66e7e00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66e8103750 0x7f66e819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.591+0000 7f66e7400640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e8109090 0x7f66e819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.591+0000 7f66e7e00640 1 -- 172.21.15.32:0/1508983157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e8109090 msgr2=0x7f66e819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.591+0000 7f66e7e00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e8109090 0x7f66e819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.591+0000 7f66e7e00640 1 -- 172.21.15.32:0/1508983157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e81099e0 msgr2=0x7f66e81a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.591+0000 7f66e7e00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e81099e0 0x7f66e81a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.591+0000 7f66e7e00640 1 -- 172.21.15.32:0/1508983157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e81a2250 con 0x7f66e8103750 2024-08-31T07:10:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.591+0000 7f66ed200640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e81099e0 0x7f66e81a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.591+0000 7f66e7e00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66e8103750 0x7f66e819c2e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f66d8005fd0 tx=0x7f66d800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.592+0000 7f66e4c00640 1 -- 172.21.15.32:0/1508983157 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8009c30 con 0x7f66e8103750 2024-08-31T07:10:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.592+0000 7f66edc00640 1 -- 172.21.15.32:0/1508983157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e81a2540 con 0x7f66e8103750 2024-08-31T07:10:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.592+0000 7f66edc00640 1 -- 172.21.15.32:0/1508983157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e81a8ca0 con 0x7f66e8103750 2024-08-31T07:10:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.592+0000 7f66e4c00640 1 -- 172.21.15.32:0/1508983157 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d8009dd0 con 0x7f66e8103750 2024-08-31T07:10:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.592+0000 7f66e4c00640 1 -- 172.21.15.32:0/1508983157 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8020060 con 0x7f66e8103750 2024-08-31T07:10:18.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.593+0000 7f66edc00640 1 -- 172.21.15.32:0/1508983157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c80052c0 con 0x7f66e8103750 2024-08-31T07:10:18.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.598+0000 7f66e4c00640 1 -- 172.21.15.32:0/1508983157 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f66d80035f0 con 0x7f66e8103750 2024-08-31T07:10:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.599+0000 7f66e4c00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66cc0b4c10 0x7f66cc0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.600+0000 7f66e4c00640 1 -- 172.21.15.32:0/1508983157 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f66d80e4d20 con 0x7f66e8103750 2024-08-31T07:10:18.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.600+0000 7f66e7400640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66cc0b4c10 0x7f66cc0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:18.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.600+0000 7f66e4c00640 1 -- 172.21.15.32:0/1508983157 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f66d8017020 con 0x7f66e8103750 2024-08-31T07:10:18.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.600+0000 7f66e7400640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66cc0b4c10 0x7f66cc0b7120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009230 tx=0x7f66d4031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.835+0000 7f66edc00640 1 -- 172.21.15.32:0/1508983157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66c80055b0 con 0x7f66e8103750 2024-08-31T07:10:18.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.836+0000 7f66e4c00640 1 -- 172.21.15.32:0/1508983157 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7f66d80ab660 con 0x7f66e8103750 2024-08-31T07:10:18.837 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:10:18.837 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"btime":"2024-08-31T07:10:05:878594+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:10:04.962915+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:10:18.837 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2024-08-31T07:10:18.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 -- 172.21.15.32:0/1508983157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66cc0b4c10 msgr2=0x7f66cc0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66cc0b4c10 0x7f66cc0b7120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009230 tx=0x7f66d4031400 comp rx=0 tx=0).stop 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 -- 172.21.15.32:0/1508983157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66e8103750 msgr2=0x7f66e819c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66e8103750 0x7f66e819c2e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f66d8005fd0 tx=0x7f66d800dde0 comp rx=0 tx=0).stop 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 -- 172.21.15.32:0/1508983157 shutdown_connections 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66cc0b4c10 0x7f66cc0b7120 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66e81099e0 0x7f66e81a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66e8109090 0x7f66e819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 --2- 172.21.15.32:0/1508983157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66e8103750 0x7f66e819c2e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 -- 172.21.15.32:0/1508983157 >> 172.21.15.32:0/1508983157 conn(0x7f66e80ff1e0 msgr2=0x7f66e8108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.840+0000 7f66edc00640 1 -- 172.21.15.32:0/1508983157 shutdown_connections 2024-08-31T07:10:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:18.841+0000 7f66edc00640 1 -- 172.21.15.32:0/1508983157 wait complete. 2024-08-31T07:10:19.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:18 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1508983157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:10:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:18 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1508983157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:10:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:18 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1508983157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:10:19.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:10:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:20.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:19 smithi032 ceph-mon[17252]: pgmap v293: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-08-31T07:10:20.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:19 smithi084 ceph-mon[22387]: pgmap v293: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-08-31T07:10:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:19 smithi073 ceph-mon[21995]: pgmap v293: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-08-31T07:10:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.650+0000 7fc0ae200640 1 -- 172.21.15.32:0/4113475995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a8103750 msgr2=0x7fc0a8103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.650+0000 7fc0ae200640 1 --2- 172.21.15.32:0/4113475995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a8103750 0x7fc0a8103b60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009900 tx=0x7fc09c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:10:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.650+0000 7fc0ae200640 1 -- 172.21.15.32:0/4113475995 shutdown_connections 2024-08-31T07:10:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.650+0000 7fc0ae200640 1 --2- 172.21.15.32:0/4113475995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0a81099e0 0x7fc0a8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.650+0000 7fc0ae200640 1 --2- 172.21.15.32:0/4113475995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a8109090 0x7fc0a81094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.650+0000 7fc0ae200640 1 --2- 172.21.15.32:0/4113475995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a8103750 0x7fc0a8103b60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:20.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.650+0000 7fc0ae200640 1 -- 172.21.15.32:0/4113475995 >> 172.21.15.32:0/4113475995 conn(0x7fc0a80ff1a0 msgr2=0x7fc0a81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:20.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.650+0000 7fc0ae200640 1 -- 172.21.15.32:0/4113475995 shutdown_connections 2024-08-31T07:10:20.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.650+0000 7fc0ae200640 1 -- 172.21.15.32:0/4113475995 wait complete. 2024-08-31T07:10:20.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.651+0000 7fc0ae200640 1 Processor -- start 2024-08-31T07:10:20.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.651+0000 7fc0ae200640 1 -- start start 2024-08-31T07:10:20.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.651+0000 7fc0ae200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a8103750 0x7fc0a80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:20.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0ae200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0a8109090 0x7fc0a80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:20.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0ae200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a81099e0 0x7fc0a8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0ae200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a8113700 con 0x7fc0a8103750 2024-08-31T07:10:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0ae200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a8113580 con 0x7fc0a8109090 2024-08-31T07:10:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0ae200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a8113880 con 0x7fc0a81099e0 2024-08-31T07:10:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0ace00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a8103750 0x7fc0a80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0a8109090 0x7fc0a80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0ace00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a8103750 0x7fc0a80754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39380/0 (socket says 172.21.15.32:39380) 2024-08-31T07:10:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0ace00640 1 -- 172.21.15.32:0/3218363544 learned_addr learned my addr 172.21.15.32:0/3218363544 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:20.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0a8109090 0x7fc0a80759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:43802/0 (socket says 172.21.15.32:43802) 2024-08-31T07:10:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0ad800640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a81099e0 0x7fc0a8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0a7e00640 1 -- 172.21.15.32:0/3218363544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a81099e0 msgr2=0x7fc0a8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0a7e00640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a81099e0 0x7fc0a8075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0a7e00640 1 -- 172.21.15.32:0/3218363544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a8103750 msgr2=0x7fc0a80754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0a7e00640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a8103750 0x7fc0a80754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.652+0000 7fc0a7e00640 1 -- 172.21.15.32:0/3218363544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a81a8370 con 0x7fc0a8109090 2024-08-31T07:10:20.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.653+0000 7fc0a7e00640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0a8109090 0x7fc0a80759f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc0a806b6d0 tx=0x7fc09400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.653+0000 7fc0a5600640 1 -- 172.21.15.32:0/3218363544 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09400cc40 con 0x7fc0a8109090 2024-08-31T07:10:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.653+0000 7fc0ae200640 1 -- 172.21.15.32:0/3218363544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a81a8660 con 0x7fc0a8109090 2024-08-31T07:10:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.653+0000 7fc0a5600640 1 -- 172.21.15.32:0/3218363544 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc09400cde0 con 0x7fc0a8109090 2024-08-31T07:10:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.653+0000 7fc0a5600640 1 -- 172.21.15.32:0/3218363544 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094004a00 con 0x7fc0a8109090 2024-08-31T07:10:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.653+0000 7fc0ae200640 1 -- 172.21.15.32:0/3218363544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a81a8ba0 con 0x7fc0a8109090 2024-08-31T07:10:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.654+0000 7fc0ae200640 1 -- 172.21.15.32:0/3218363544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a8103b60 con 0x7fc0a8109090 2024-08-31T07:10:20.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.658+0000 7fc0a5600640 1 -- 172.21.15.32:0/3218363544 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fc094004ba0 con 0x7fc0a8109090 2024-08-31T07:10:20.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.659+0000 7fc0a5600640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0900b4c10 0x7fc0900b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:20.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.659+0000 7fc0a5600640 1 -- 172.21.15.32:0/3218363544 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fc0940e53a0 con 0x7fc0a8109090 2024-08-31T07:10:20.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.660+0000 7fc0ace00640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0900b4c10 0x7fc0900b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:20.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.660+0000 7fc0a5600640 1 -- 172.21.15.32:0/3218363544 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc0940ac3c0 con 0x7fc0a8109090 2024-08-31T07:10:20.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.660+0000 7fc0ace00640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0900b4c10 0x7fc0900b7120 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005ec0 tx=0x7fc09c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.856+0000 7fc0ae200640 1 -- 172.21.15.32:0/3218363544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0a80767b0 con 0x7fc0a8109090 2024-08-31T07:10:20.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.858+0000 7fc0a5600640 1 -- 172.21.15.32:0/3218363544 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22532 (secure 0 0 0) 0x7fc0940abcc0 con 0x7fc0a8109090 2024-08-31T07:10:20.859 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:10:20.859 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:09:48.797927+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 -- 172.21.15.32:0/3218363544 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0900b4c10 msgr2=0x7fc0900b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0900b4c10 0x7fc0900b7120 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005ec0 tx=0x7fc09c005a20 comp rx=0 tx=0).stop 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 -- 172.21.15.32:0/3218363544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0a8109090 msgr2=0x7fc0a80759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0a8109090 0x7fc0a80759f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc0a806b6d0 tx=0x7fc09400ee50 comp rx=0 tx=0).stop 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 -- 172.21.15.32:0/3218363544 shutdown_connections 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0900b4c10 0x7fc0900b7120 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a81099e0 0x7fc0a8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0a8109090 0x7fc0a80759f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 --2- 172.21.15.32:0/3218363544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a8103750 0x7fc0a80754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 -- 172.21.15.32:0/3218363544 >> 172.21.15.32:0/3218363544 conn(0x7fc0a80ff1a0 msgr2=0x7fc0a81015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 -- 172.21.15.32:0/3218363544 shutdown_connections 2024-08-31T07:10:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:20.861+0000 7fc0ae200640 1 -- 172.21.15.32:0/3218363544 wait complete. 2024-08-31T07:10:21.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:20 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3218363544' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:10:21.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:20 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3218363544' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:10:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:20 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3218363544' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:10:21.647 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-31T07:10:21.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T07:10:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:22.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:21 smithi084 ceph-mon[22387]: pgmap v294: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-08-31T07:10:22.288 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:21 smithi032 ceph-mon[17252]: pgmap v294: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-08-31T07:10:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:21 smithi073 ceph-mon[21995]: pgmap v294: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-08-31T07:10:22.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.607+0000 7f4911800640 1 -- 172.21.15.32:0/760150999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f490c105960 msgr2=0x7f490c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:22.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.607+0000 7f4911800640 1 --2- 172.21.15.32:0/760150999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f490c105960 0x7f490c105d70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009900 tx=0x7f48fc02a410 comp rx=0 tx=0).stop 2024-08-31T07:10:22.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.607+0000 7f4911800640 1 -- 172.21.15.32:0/760150999 shutdown_connections 2024-08-31T07:10:22.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.607+0000 7f4911800640 1 --2- 172.21.15.32:0/760150999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490c10bbf0 0x7f490c112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:22.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.607+0000 7f4911800640 1 --2- 172.21.15.32:0/760150999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c10b2a0 0x7f490c10b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:22.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.607+0000 7f4911800640 1 --2- 172.21.15.32:0/760150999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f490c105960 0x7f490c105d70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:22.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.607+0000 7f4911800640 1 -- 172.21.15.32:0/760150999 >> 172.21.15.32:0/760150999 conn(0x7f490c101410 msgr2=0x7f490c103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:22.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.607+0000 7f4911800640 1 -- 172.21.15.32:0/760150999 shutdown_connections 2024-08-31T07:10:22.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.607+0000 7f4911800640 1 -- 172.21.15.32:0/760150999 wait complete. 2024-08-31T07:10:22.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.608+0000 7f4911800640 1 Processor -- start 2024-08-31T07:10:22.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.608+0000 7f4911800640 1 -- start start 2024-08-31T07:10:22.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f4911800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490c105960 0x7f490c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:22.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f4911800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c10b2a0 0x7f490c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:22.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f4911800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f490c10bbf0 0x7f490c1a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:22.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f4911800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f490c115900 con 0x7f490c10bbf0 2024-08-31T07:10:22.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f4911800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f490c115780 con 0x7f490c10b2a0 2024-08-31T07:10:22.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f4911800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f490c115a80 con 0x7f490c105960 2024-08-31T07:10:22.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f4910e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f490c10bbf0 0x7f490c1a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:22.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f4910e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f490c10bbf0 0x7f490c1a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39400/0 (socket says 172.21.15.32:39400) 2024-08-31T07:10:22.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f4910e00640 1 -- 172.21.15.32:0/3088149651 learned_addr learned my addr 172.21.15.32:0/3088149651 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:22.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f490be00640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490c105960 0x7f490c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:22.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.609+0000 7f490b400640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c10b2a0 0x7f490c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4910e00640 1 -- 172.21.15.32:0/3088149651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490c105960 msgr2=0x7f490c19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4910e00640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490c105960 0x7f490c19c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4910e00640 1 -- 172.21.15.32:0/3088149651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c10b2a0 msgr2=0x7f490c19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4910e00640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c10b2a0 0x7f490c19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4910e00640 1 -- 172.21.15.32:0/3088149651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c1a2320 con 0x7f490c10bbf0 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4910e00640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f490c10bbf0 0x7f490c1a1bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f490400eb10 tx=0x7f490400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4908c00640 1 -- 172.21.15.32:0/3088149651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904004b20 con 0x7f490c10bbf0 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4908c00640 1 -- 172.21.15.32:0/3088149651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4904004cc0 con 0x7f490c10bbf0 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4911800640 1 -- 172.21.15.32:0/3088149651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c1a2610 con 0x7f490c10bbf0 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4908c00640 1 -- 172.21.15.32:0/3088149651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904010670 con 0x7f490c10bbf0 2024-08-31T07:10:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.610+0000 7f4911800640 1 -- 172.21.15.32:0/3088149651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1a8d70 con 0x7f490c10bbf0 2024-08-31T07:10:22.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.612+0000 7f4911800640 1 -- 172.21.15.32:0/3088149651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490c105d70 con 0x7f490c10bbf0 2024-08-31T07:10:22.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.614+0000 7f4908c00640 1 -- 172.21.15.32:0/3088149651 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f4904005000 con 0x7f490c10bbf0 2024-08-31T07:10:22.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.615+0000 7f4908c00640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f00b4c10 0x7f48f00b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:22.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.615+0000 7f4908c00640 1 -- 172.21.15.32:0/3088149651 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f49040e5a30 con 0x7f490c10bbf0 2024-08-31T07:10:22.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.616+0000 7f490be00640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f00b4c10 0x7f48f00b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:22.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.616+0000 7f490be00640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f00b4c10 0x7f48f00b7120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005ab0 tx=0x7f48fc005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:22.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.617+0000 7f4908c00640 1 -- 172.21.15.32:0/3088149651 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4904056130 con 0x7f490c10bbf0 2024-08-31T07:10:22.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.810+0000 7f4911800640 1 -- 172.21.15.32:0/3088149651 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f490c0635a0 con 0x7f48f00b4c10 2024-08-31T07:10:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.816+0000 7f4908c00640 1 -- 172.21.15.32:0/3088149651 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258903 (secure 0 0 0) 0x7f490c0635a0 con 0x7f48f00b4c10 2024-08-31T07:10:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:10:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-08-31T07:10:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.819+0000 7f4911800640 1 -- 172.21.15.32:0/3088149651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f00b4c10 msgr2=0x7f48f00b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.819+0000 7f4911800640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f00b4c10 0x7f48f00b7120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005ab0 tx=0x7f48fc005a20 comp rx=0 tx=0).stop 2024-08-31T07:10:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.819+0000 7f4911800640 1 -- 172.21.15.32:0/3088149651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f490c10bbf0 msgr2=0x7f490c1a1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.819+0000 7f4911800640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f490c10bbf0 0x7f490c1a1bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f490400eb10 tx=0x7f490400c2d0 comp rx=0 tx=0).stop 2024-08-31T07:10:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.820+0000 7f4911800640 1 -- 172.21.15.32:0/3088149651 shutdown_connections 2024-08-31T07:10:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.820+0000 7f4911800640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f00b4c10 0x7f48f00b7120 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.820+0000 7f4911800640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f490c10bbf0 0x7f490c1a1bf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.820+0000 7f4911800640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c10b2a0 0x7f490c19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.820+0000 7f4911800640 1 --2- 172.21.15.32:0/3088149651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490c105960 0x7f490c19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.820+0000 7f4911800640 1 -- 172.21.15.32:0/3088149651 >> 172.21.15.32:0/3088149651 conn(0x7f490c101410 msgr2=0x7f490c10a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.820+0000 7f4911800640 1 -- 172.21.15.32:0/3088149651 shutdown_connections 2024-08-31T07:10:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:22.820+0000 7f4911800640 1 -- 172.21.15.32:0/3088149651 wait complete. 2024-08-31T07:10:23.638 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-08-31T07:10:22.765365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652960,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":293,"num_read_kb":240,"num_write":365,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363900,"kb_used_data":10492,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577924,"statfs":{"total":1151940427776,"available":1151567794176,"internally_reserved":0,"allocated":10743808,"data_stored":6926562,"data_compressed":108822,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53776,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":117,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003251"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661879+0000","last_change":"2024-08-31T07:09:47.665725+0000","last_active":"2024-08-31T07:09:49.661879+0000","last_peered":"2024-08-31T07:09:49.661879+0000","last_clean":"2024-08-31T07:09:49.661879+0000","last_became_active":"2024-08-31T07:09:47.665368+0000","last_became_peered":"2024-08-31T07:09:47.665368+0000","last_unstale":"2024-08-31T07:09:49.661879+0000","last_undegraded":"2024-08-31T07:09:49.661879+0000","last_fullsized":"2024-08-31T07:09:49.661879+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:27:01.636099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.652993+0000","last_change":"2024-08-31T07:09:47.676882+0000","last_active":"2024-08-31T07:09:49.652993+0000","last_peered":"2024-08-31T07:09:49.652993+0000","last_clean":"2024-08-31T07:09:49.652993+0000","last_became_active":"2024-08-31T07:09:47.674825+0000","last_became_peered":"2024-08-31T07:09:47.674825+0000","last_unstale":"2024-08-31T07:09:49.652993+0000","last_undegraded":"2024-08-31T07:09:49.652993+0000","last_fullsized":"2024-08-31T07:09:49.652993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:09.148398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.652831+0000","last_change":"2024-08-31T07:09:47.674480+0000","last_active":"2024-08-31T07:09:49.652831+0000","last_peered":"2024-08-31T07:09:49.652831+0000","last_clean":"2024-08-31T07:09:49.652831+0000","last_became_active":"2024-08-31T07:09:47.672316+0000","last_became_peered":"2024-08-31T07:09:47.672316+0000","last_unstale":"2024-08-31T07:09:49.652831+0000","last_undegraded":"2024-08-31T07:09:49.652831+0000","last_fullsized":"2024-08-31T07:09:49.652831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:28:13.231039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.654048+0000","last_change":"2024-08-31T07:09:47.659276+0000","last_active":"2024-08-31T07:09:49.654048+0000","last_peered":"2024-08-31T07:09:49.654048+0000","last_clean":"2024-08-31T07:09:49.654048+0000","last_became_active":"2024-08-31T07:09:47.658924+0000","last_became_peered":"2024-08-31T07:09:47.658924+0000","last_unstale":"2024-08-31T07:09:49.654048+0000","last_undegraded":"2024-08-31T07:09:49.654048+0000","last_fullsized":"2024-08-31T07:09:49.654048+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:01:08.134245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.668175+0000","last_change":"2024-08-31T07:09:47.667260+0000","last_active":"2024-08-31T07:09:49.668175+0000","last_peered":"2024-08-31T07:09:49.668175+0000","last_clean":"2024-08-31T07:09:49.668175+0000","last_became_active":"2024-08-31T07:09:47.664572+0000","last_became_peered":"2024-08-31T07:09:47.664572+0000","last_unstale":"2024-08-31T07:09:49.668175+0000","last_undegraded":"2024-08-31T07:09:49.668175+0000","last_fullsized":"2024-08-31T07:09:49.668175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:11.336985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.667372+0000","last_change":"2024-08-31T07:09:47.661181+0000","last_active":"2024-08-31T07:09:49.667372+0000","last_peered":"2024-08-31T07:09:49.667372+0000","last_clean":"2024-08-31T07:09:49.667372+0000","last_became_active":"2024-08-31T07:09:47.660761+0000","last_became_peered":"2024-08-31T07:09:47.660761+0000","last_unstale":"2024-08-31T07:09:49.667372+0000","last_undegraded":"2024-08-31T07:09:49.667372+0000","last_fullsized":"2024-08-31T07:09:49.667372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:50:54.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665627+0000","last_change":"2024-08-31T07:09:47.668762+0000","last_active":"2024-08-31T07:09:49.665627+0000","last_peered":"2024-08-31T07:09:49.665627+0000","last_clean":"2024-08-31T07:09:49.665627+0000","last_became_active":"2024-08-31T07:09:47.668406+0000","last_became_peered":"2024-08-31T07:09:47.668406+0000","last_unstale":"2024-08-31T07:09:49.665627+0000","last_undegraded":"2024-08-31T07:09:49.665627+0000","last_fullsized":"2024-08-31T07:09:49.665627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:43:39.868144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.650611+0000","last_change":"2024-08-31T07:09:47.669455+0000","last_active":"2024-08-31T07:09:49.650611+0000","last_peered":"2024-08-31T07:09:49.650611+0000","last_clean":"2024-08-31T07:09:49.650611+0000","last_became_active":"2024-08-31T07:09:47.663669+0000","last_became_peered":"2024-08-31T07:09:47.663669+0000","last_unstale":"2024-08-31T07:09:49.650611+0000","last_undegraded":"2024-08-31T07:09:49.650611+0000","last_fullsized":"2024-08-31T07:09:49.650611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:22:59.685012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.671722+0000","last_change":"2024-08-31T07:09:47.664214+0000","last_active":"2024-08-31T07:09:49.671722+0000","last_peered":"2024-08-31T07:09:49.671722+0000","last_clean":"2024-08-31T07:09:49.671722+0000","last_became_active":"2024-08-31T07:09:47.663832+0000","last_became_peered":"2024-08-31T07:09:47.663832+0000","last_unstale":"2024-08-31T07:09:49.671722+0000","last_undegraded":"2024-08-31T07:09:49.671722+0000","last_fullsized":"2024-08-31T07:09:49.671722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:29:14.107212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.667863+0000","last_change":"2024-08-31T07:09:47.658369+0000","last_active":"2024-08-31T07:09:49.667863+0000","last_peered":"2024-08-31T07:09:49.667863+0000","last_clean":"2024-08-31T07:09:49.667863+0000","last_became_active":"2024-08-31T07:09:47.656749+0000","last_became_peered":"2024-08-31T07:09:47.656749+0000","last_unstale":"2024-08-31T07:09:49.667863+0000","last_undegraded":"2024-08-31T07:09:49.667863+0000","last_fullsized":"2024-08-31T07:09:49.667863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:16:46.243603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665972+0000","last_change":"2024-08-31T07:09:47.666951+0000","last_active":"2024-08-31T07:09:49.665972+0000","last_peered":"2024-08-31T07:09:49.665972+0000","last_clean":"2024-08-31T07:09:49.665972+0000","last_became_active":"2024-08-31T07:09:47.666475+0000","last_became_peered":"2024-08-31T07:09:47.666475+0000","last_unstale":"2024-08-31T07:09:49.665972+0000","last_undegraded":"2024-08-31T07:09:49.665972+0000","last_fullsized":"2024-08-31T07:09:49.665972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:58:42.242086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.654190+0000","last_change":"2024-08-31T07:09:47.658344+0000","last_active":"2024-08-31T07:09:49.654190+0000","last_peered":"2024-08-31T07:09:49.654190+0000","last_clean":"2024-08-31T07:09:49.654190+0000","last_became_active":"2024-08-31T07:09:47.657991+0000","last_became_peered":"2024-08-31T07:09:47.657991+0000","last_unstale":"2024-08-31T07:09:49.654190+0000","last_undegraded":"2024-08-31T07:09:49.654190+0000","last_fullsized":"2024-08-31T07:09:49.654190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:25:59.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661053+0000","last_change":"2024-08-31T07:09:47.661644+0000","last_active":"2024-08-31T07:09:49.661053+0000","last_peered":"2024-08-31T07:09:49.661053+0000","last_clean":"2024-08-31T07:09:49.661053+0000","last_became_active":"2024-08-31T07:09:47.661248+0000","last_became_peered":"2024-08-31T07:09:47.661248+0000","last_unstale":"2024-08-31T07:09:49.661053+0000","last_undegraded":"2024-08-31T07:09:49.661053+0000","last_fullsized":"2024-08-31T07:09:49.661053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:42.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.652301+0000","last_change":"2024-08-31T07:09:47.680052+0000","last_active":"2024-08-31T07:09:49.652301+0000","last_peered":"2024-08-31T07:09:49.652301+0000","last_clean":"2024-08-31T07:09:49.652301+0000","last_became_active":"2024-08-31T07:09:47.679604+0000","last_became_peered":"2024-08-31T07:09:47.679604+0000","last_unstale":"2024-08-31T07:09:49.652301+0000","last_undegraded":"2024-08-31T07:09:49.652301+0000","last_fullsized":"2024-08-31T07:09:49.652301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:23:18.937470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.651734+0000","last_change":"2024-08-31T07:09:47.677984+0000","last_active":"2024-08-31T07:09:49.651734+0000","last_peered":"2024-08-31T07:09:49.651734+0000","last_clean":"2024-08-31T07:09:49.651734+0000","last_became_active":"2024-08-31T07:09:47.674957+0000","last_became_peered":"2024-08-31T07:09:47.674957+0000","last_unstale":"2024-08-31T07:09:49.651734+0000","last_undegraded":"2024-08-31T07:09:49.651734+0000","last_fullsized":"2024-08-31T07:09:49.651734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:08:26.503746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.670145+0000","last_change":"2024-08-31T07:09:47.672625+0000","last_active":"2024-08-31T07:09:49.670145+0000","last_peered":"2024-08-31T07:09:49.670145+0000","last_clean":"2024-08-31T07:09:49.670145+0000","last_became_active":"2024-08-31T07:09:47.672184+0000","last_became_peered":"2024-08-31T07:09:47.672184+0000","last_unstale":"2024-08-31T07:09:49.670145+0000","last_undegraded":"2024-08-31T07:09:49.670145+0000","last_fullsized":"2024-08-31T07:09:49.670145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:17.091659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.662237+0000","last_change":"2024-08-31T07:09:47.672597+0000","last_active":"2024-08-31T07:09:49.662237+0000","last_peered":"2024-08-31T07:09:49.662237+0000","last_clean":"2024-08-31T07:09:49.662237+0000","last_became_active":"2024-08-31T07:09:47.672153+0000","last_became_peered":"2024-08-31T07:09:47.672153+0000","last_unstale":"2024-08-31T07:09:49.662237+0000","last_undegraded":"2024-08-31T07:09:49.662237+0000","last_fullsized":"2024-08-31T07:09:49.662237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:26:41.388526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664954+0000","last_change":"2024-08-31T07:09:47.666253+0000","last_active":"2024-08-31T07:09:49.664954+0000","last_peered":"2024-08-31T07:09:49.664954+0000","last_clean":"2024-08-31T07:09:49.664954+0000","last_became_active":"2024-08-31T07:09:47.663732+0000","last_became_peered":"2024-08-31T07:09:47.663732+0000","last_unstale":"2024-08-31T07:09:49.664954+0000","last_undegraded":"2024-08-31T07:09:49.664954+0000","last_fullsized":"2024-08-31T07:09:49.664954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:10:59.887765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664632+0000","last_change":"2024-08-31T07:09:47.668502+0000","last_active":"2024-08-31T07:09:49.664632+0000","last_peered":"2024-08-31T07:09:49.664632+0000","last_clean":"2024-08-31T07:09:49.664632+0000","last_became_active":"2024-08-31T07:09:47.666487+0000","last_became_peered":"2024-08-31T07:09:47.666487+0000","last_unstale":"2024-08-31T07:09:49.664632+0000","last_undegraded":"2024-08-31T07:09:49.664632+0000","last_fullsized":"2024-08-31T07:09:49.664632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:49:25.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661288+0000","last_change":"2024-08-31T07:09:47.659454+0000","last_active":"2024-08-31T07:09:49.661288+0000","last_peered":"2024-08-31T07:09:49.661288+0000","last_clean":"2024-08-31T07:09:49.661288+0000","last_became_active":"2024-08-31T07:09:47.658976+0000","last_became_peered":"2024-08-31T07:09:47.658976+0000","last_unstale":"2024-08-31T07:09:49.661288+0000","last_undegraded":"2024-08-31T07:09:49.661288+0000","last_fullsized":"2024-08-31T07:09:49.661288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:31.374905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661088+0000","last_change":"2024-08-31T07:09:47.673612+0000","last_active":"2024-08-31T07:09:49.661088+0000","last_peered":"2024-08-31T07:09:49.661088+0000","last_clean":"2024-08-31T07:09:49.661088+0000","last_became_active":"2024-08-31T07:09:47.673137+0000","last_became_peered":"2024-08-31T07:09:47.673137+0000","last_unstale":"2024-08-31T07:09:49.661088+0000","last_undegraded":"2024-08-31T07:09:49.661088+0000","last_fullsized":"2024-08-31T07:09:49.661088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:46:11.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.670995+0000","last_change":"2024-08-31T07:09:47.669195+0000","last_active":"2024-08-31T07:09:49.670995+0000","last_peered":"2024-08-31T07:09:49.670995+0000","last_clean":"2024-08-31T07:09:49.670995+0000","last_became_active":"2024-08-31T07:09:47.668790+0000","last_became_peered":"2024-08-31T07:09:47.668790+0000","last_unstale":"2024-08-31T07:09:49.670995+0000","last_undegraded":"2024-08-31T07:09:49.670995+0000","last_fullsized":"2024-08-31T07:09:49.670995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:27:14.393931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.650976+0000","last_change":"2024-08-31T07:09:47.667566+0000","last_active":"2024-08-31T07:09:49.650976+0000","last_peered":"2024-08-31T07:09:49.650976+0000","last_clean":"2024-08-31T07:09:49.650976+0000","last_became_active":"2024-08-31T07:09:47.666605+0000","last_became_peered":"2024-08-31T07:09:47.666605+0000","last_unstale":"2024-08-31T07:09:49.650976+0000","last_undegraded":"2024-08-31T07:09:49.650976+0000","last_fullsized":"2024-08-31T07:09:49.650976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:05:44.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661572+0000","last_change":"2024-08-31T07:09:47.660478+0000","last_active":"2024-08-31T07:09:49.661572+0000","last_peered":"2024-08-31T07:09:49.661572+0000","last_clean":"2024-08-31T07:09:49.661572+0000","last_became_active":"2024-08-31T07:09:47.660107+0000","last_became_peered":"2024-08-31T07:09:47.660107+0000","last_unstale":"2024-08-31T07:09:49.661572+0000","last_undegraded":"2024-08-31T07:09:49.661572+0000","last_fullsized":"2024-08-31T07:09:49.661572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:29.133370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.651507+0000","last_change":"2024-08-31T07:09:47.671967+0000","last_active":"2024-08-31T07:09:49.651507+0000","last_peered":"2024-08-31T07:09:49.651507+0000","last_clean":"2024-08-31T07:09:49.651507+0000","last_became_active":"2024-08-31T07:09:47.671569+0000","last_became_peered":"2024-08-31T07:09:47.671569+0000","last_unstale":"2024-08-31T07:09:49.651507+0000","last_undegraded":"2024-08-31T07:09:49.651507+0000","last_fullsized":"2024-08-31T07:09:49.651507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:19:15.107958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665166+0000","last_change":"2024-08-31T07:09:47.651055+0000","last_active":"2024-08-31T07:09:49.665166+0000","last_peered":"2024-08-31T07:09:49.665166+0000","last_clean":"2024-08-31T07:09:49.665166+0000","last_became_active":"2024-08-31T07:09:47.650674+0000","last_became_peered":"2024-08-31T07:09:47.650674+0000","last_unstale":"2024-08-31T07:09:49.665166+0000","last_undegraded":"2024-08-31T07:09:49.665166+0000","last_fullsized":"2024-08-31T07:09:49.665166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:35:54.647379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661824+0000","last_change":"2024-08-31T07:09:47.663231+0000","last_active":"2024-08-31T07:09:49.661824+0000","last_peered":"2024-08-31T07:09:49.661824+0000","last_clean":"2024-08-31T07:09:49.661824+0000","last_became_active":"2024-08-31T07:09:47.662854+0000","last_became_peered":"2024-08-31T07:09:47.662854+0000","last_unstale":"2024-08-31T07:09:49.661824+0000","last_undegraded":"2024-08-31T07:09:49.661824+0000","last_fullsized":"2024-08-31T07:09:49.661824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:51.358557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661452+0000","last_change":"2024-08-31T07:09:47.665096+0000","last_active":"2024-08-31T07:09:49.661452+0000","last_peered":"2024-08-31T07:09:49.661452+0000","last_clean":"2024-08-31T07:09:49.661452+0000","last_became_active":"2024-08-31T07:09:47.664698+0000","last_became_peered":"2024-08-31T07:09:47.664698+0000","last_unstale":"2024-08-31T07:09:49.661452+0000","last_undegraded":"2024-08-31T07:09:49.661452+0000","last_fullsized":"2024-08-31T07:09:49.661452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:44:35.391844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.652735+0000","last_change":"2024-08-31T07:09:47.672108+0000","last_active":"2024-08-31T07:09:49.652735+0000","last_peered":"2024-08-31T07:09:49.652735+0000","last_clean":"2024-08-31T07:09:49.652735+0000","last_became_active":"2024-08-31T07:09:47.670998+0000","last_became_peered":"2024-08-31T07:09:47.670998+0000","last_unstale":"2024-08-31T07:09:49.652735+0000","last_undegraded":"2024-08-31T07:09:49.652735+0000","last_fullsized":"2024-08-31T07:09:49.652735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:26:49.319861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664334+0000","last_change":"2024-08-31T07:09:47.665548+0000","last_active":"2024-08-31T07:09:49.664334+0000","last_peered":"2024-08-31T07:09:49.664334+0000","last_clean":"2024-08-31T07:09:49.664334+0000","last_became_active":"2024-08-31T07:09:47.665177+0000","last_became_peered":"2024-08-31T07:09:47.665177+0000","last_unstale":"2024-08-31T07:09:49.664334+0000","last_undegraded":"2024-08-31T07:09:49.664334+0000","last_fullsized":"2024-08-31T07:09:49.664334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:03:15.892224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664509+0000","last_change":"2024-08-31T07:09:47.659514+0000","last_active":"2024-08-31T07:09:49.664509+0000","last_peered":"2024-08-31T07:09:49.664509+0000","last_clean":"2024-08-31T07:09:49.664509+0000","last_became_active":"2024-08-31T07:09:47.659178+0000","last_became_peered":"2024-08-31T07:09:47.659178+0000","last_unstale":"2024-08-31T07:09:49.664509+0000","last_undegraded":"2024-08-31T07:09:49.664509+0000","last_fullsized":"2024-08-31T07:09:49.664509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:58:04.215441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661046+0000","last_change":"2024-08-31T07:09:47.667456+0000","last_active":"2024-08-31T07:09:49.661046+0000","last_peered":"2024-08-31T07:09:49.661046+0000","last_clean":"2024-08-31T07:09:49.661046+0000","last_became_active":"2024-08-31T07:09:47.667102+0000","last_became_peered":"2024-08-31T07:09:47.667102+0000","last_unstale":"2024-08-31T07:09:49.661046+0000","last_undegraded":"2024-08-31T07:09:49.661046+0000","last_fullsized":"2024-08-31T07:09:49.661046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:44:39.747729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.668077+0000","last_change":"2024-08-31T07:09:47.661099+0000","last_active":"2024-08-31T07:09:49.668077+0000","last_peered":"2024-08-31T07:09:49.668077+0000","last_clean":"2024-08-31T07:09:49.668077+0000","last_became_active":"2024-08-31T07:09:47.660709+0000","last_became_peered":"2024-08-31T07:09:47.660709+0000","last_unstale":"2024-08-31T07:09:49.668077+0000","last_undegraded":"2024-08-31T07:09:49.668077+0000","last_fullsized":"2024-08-31T07:09:49.668077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:03:59.071779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:19.911394+0000","last_change":"2024-08-31T07:09:45.641088+0000","last_active":"2024-08-31T07:10:19.911394+0000","last_peered":"2024-08-31T07:10:19.911394+0000","last_clean":"2024-08-31T07:10:19.911394+0000","last_became_active":"2024-08-31T07:09:45.640673+0000","last_became_peered":"2024-08-31T07:09:45.640673+0000","last_unstale":"2024-08-31T07:10:19.911394+0000","last_undegraded":"2024-08-31T07:10:19.911394+0000","last_fullsized":"2024-08-31T07:10:19.911394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:30:17.593494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.662758+0000","last_change":"2024-08-31T07:09:47.666703+0000","last_active":"2024-08-31T07:09:49.662758+0000","last_peered":"2024-08-31T07:09:49.662758+0000","last_clean":"2024-08-31T07:09:49.662758+0000","last_became_active":"2024-08-31T07:09:47.666348+0000","last_became_peered":"2024-08-31T07:09:47.666348+0000","last_unstale":"2024-08-31T07:09:49.662758+0000","last_undegraded":"2024-08-31T07:09:49.662758+0000","last_fullsized":"2024-08-31T07:09:49.662758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:39.688810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:01.867844+0000","last_change":"2024-08-31T07:09:45.642431+0000","last_active":"2024-08-31T07:10:01.867844+0000","last_peered":"2024-08-31T07:10:01.867844+0000","last_clean":"2024-08-31T07:10:01.867844+0000","last_became_active":"2024-08-31T07:09:45.642048+0000","last_became_peered":"2024-08-31T07:09:45.642048+0000","last_unstale":"2024-08-31T07:10:01.867844+0000","last_undegraded":"2024-08-31T07:10:01.867844+0000","last_fullsized":"2024-08-31T07:10:01.867844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:47:42.574442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:00.863011+0000","last_change":"2024-08-31T07:09:45.632162+0000","last_active":"2024-08-31T07:10:00.863011+0000","last_peered":"2024-08-31T07:10:00.863011+0000","last_clean":"2024-08-31T07:10:00.863011+0000","last_became_active":"2024-08-31T07:09:45.631749+0000","last_became_peered":"2024-08-31T07:09:45.631749+0000","last_unstale":"2024-08-31T07:10:00.863011+0000","last_undegraded":"2024-08-31T07:10:00.863011+0000","last_fullsized":"2024-08-31T07:10:00.863011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:16:29.967823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.650897+0000","last_change":"2024-08-31T07:09:47.666288+0000","last_active":"2024-08-31T07:09:49.650897+0000","last_peered":"2024-08-31T07:09:49.650897+0000","last_clean":"2024-08-31T07:09:49.650897+0000","last_became_active":"2024-08-31T07:09:47.665723+0000","last_became_peered":"2024-08-31T07:09:47.665723+0000","last_unstale":"2024-08-31T07:09:49.650897+0000","last_undegraded":"2024-08-31T07:09:49.650897+0000","last_fullsized":"2024-08-31T07:09:49.650897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:38:28.122720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.881727+0000","last_change":"2024-08-31T07:09:45.636508+0000","last_active":"2024-08-31T07:10:02.881727+0000","last_peered":"2024-08-31T07:10:02.881727+0000","last_clean":"2024-08-31T07:10:02.881727+0000","last_became_active":"2024-08-31T07:09:45.636073+0000","last_became_peered":"2024-08-31T07:09:45.636073+0000","last_unstale":"2024-08-31T07:10:02.881727+0000","last_undegraded":"2024-08-31T07:10:02.881727+0000","last_fullsized":"2024-08-31T07:10:02.881727+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:55:45.063517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664618+0000","last_change":"2024-08-31T07:09:47.661306+0000","last_active":"2024-08-31T07:09:49.664618+0000","last_peered":"2024-08-31T07:09:49.664618+0000","last_clean":"2024-08-31T07:09:49.664618+0000","last_became_active":"2024-08-31T07:09:47.660957+0000","last_became_peered":"2024-08-31T07:09:47.660957+0000","last_unstale":"2024-08-31T07:09:49.664618+0000","last_undegraded":"2024-08-31T07:09:49.664618+0000","last_fullsized":"2024-08-31T07:09:49.664618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:18:23.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:20.912431+0000","last_change":"2024-08-31T07:09:45.640947+0000","last_active":"2024-08-31T07:10:20.912431+0000","last_peered":"2024-08-31T07:10:20.912431+0000","last_clean":"2024-08-31T07:10:20.912431+0000","last_became_active":"2024-08-31T07:09:45.640532+0000","last_became_peered":"2024-08-31T07:09:45.640532+0000","last_unstale":"2024-08-31T07:10:20.912431+0000","last_undegraded":"2024-08-31T07:10:20.912431+0000","last_fullsized":"2024-08-31T07:10:20.912431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:28.177795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.671873+0000","last_change":"2024-08-31T07:09:47.661060+0000","last_active":"2024-08-31T07:09:49.671873+0000","last_peered":"2024-08-31T07:09:49.671873+0000","last_clean":"2024-08-31T07:09:49.671873+0000","last_became_active":"2024-08-31T07:09:47.660692+0000","last_became_peered":"2024-08-31T07:09:47.660692+0000","last_unstale":"2024-08-31T07:09:49.671873+0000","last_undegraded":"2024-08-31T07:09:49.671873+0000","last_fullsized":"2024-08-31T07:09:49.671873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:48:38.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.882243+0000","last_change":"2024-08-31T07:09:45.641155+0000","last_active":"2024-08-31T07:10:02.882243+0000","last_peered":"2024-08-31T07:10:02.882243+0000","last_clean":"2024-08-31T07:10:02.882243+0000","last_became_active":"2024-08-31T07:09:45.640680+0000","last_became_peered":"2024-08-31T07:09:45.640680+0000","last_unstale":"2024-08-31T07:10:02.882243+0000","last_undegraded":"2024-08-31T07:10:02.882243+0000","last_fullsized":"2024-08-31T07:10:02.882243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:29:03.287570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.653822+0000","last_change":"2024-08-31T07:09:47.675582+0000","last_active":"2024-08-31T07:09:49.653822+0000","last_peered":"2024-08-31T07:09:49.653822+0000","last_clean":"2024-08-31T07:09:49.653822+0000","last_became_active":"2024-08-31T07:09:47.675266+0000","last_became_peered":"2024-08-31T07:09:47.675266+0000","last_unstale":"2024-08-31T07:09:49.653822+0000","last_undegraded":"2024-08-31T07:09:49.653822+0000","last_fullsized":"2024-08-31T07:09:49.653822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:36:33.122531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:01.865666+0000","last_change":"2024-08-31T07:09:45.651006+0000","last_active":"2024-08-31T07:10:01.865666+0000","last_peered":"2024-08-31T07:10:01.865666+0000","last_clean":"2024-08-31T07:10:01.865666+0000","last_became_active":"2024-08-31T07:09:45.650588+0000","last_became_peered":"2024-08-31T07:09:45.650588+0000","last_unstale":"2024-08-31T07:10:01.865666+0000","last_undegraded":"2024-08-31T07:10:01.865666+0000","last_fullsized":"2024-08-31T07:10:01.865666+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:34:16.015892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.671047+0000","last_change":"2024-08-31T07:09:47.662289+0000","last_active":"2024-08-31T07:09:49.671047+0000","last_peered":"2024-08-31T07:09:49.671047+0000","last_clean":"2024-08-31T07:09:49.671047+0000","last_became_active":"2024-08-31T07:09:47.661932+0000","last_became_peered":"2024-08-31T07:09:47.661932+0000","last_unstale":"2024-08-31T07:09:49.671047+0000","last_undegraded":"2024-08-31T07:09:49.671047+0000","last_fullsized":"2024-08-31T07:09:49.671047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:50.440504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'28","reported_seq":92,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:20.913308+0000","last_change":"2024-08-31T07:09:45.642196+0000","last_active":"2024-08-31T07:10:20.913308+0000","last_peered":"2024-08-31T07:10:20.913308+0000","last_clean":"2024-08-31T07:10:20.913308+0000","last_became_active":"2024-08-31T07:09:45.641797+0000","last_became_peered":"2024-08-31T07:09:45.641797+0000","last_unstale":"2024-08-31T07:10:20.913308+0000","last_undegraded":"2024-08-31T07:10:20.913308+0000","last_fullsized":"2024-08-31T07:10:20.913308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:51:05.953335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54372,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":69,"num_write":30,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.668763+0000","last_change":"2024-08-31T07:09:47.660184+0000","last_active":"2024-08-31T07:09:49.668763+0000","last_peered":"2024-08-31T07:09:49.668763+0000","last_clean":"2024-08-31T07:09:49.668763+0000","last_became_active":"2024-08-31T07:09:47.659571+0000","last_became_peered":"2024-08-31T07:09:47.659571+0000","last_unstale":"2024-08-31T07:09:49.668763+0000","last_undegraded":"2024-08-31T07:09:49.668763+0000","last_fullsized":"2024-08-31T07:09:49.668763+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:41:13.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:03.881055+0000","last_change":"2024-08-31T07:09:45.639938+0000","last_active":"2024-08-31T07:10:03.881055+0000","last_peered":"2024-08-31T07:10:03.881055+0000","last_clean":"2024-08-31T07:10:03.881055+0000","last_became_active":"2024-08-31T07:09:45.639569+0000","last_became_peered":"2024-08-31T07:09:45.639569+0000","last_unstale":"2024-08-31T07:10:03.881055+0000","last_undegraded":"2024-08-31T07:10:03.881055+0000","last_fullsized":"2024-08-31T07:10:03.881055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:16:36.589872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.660768+0000","last_change":"2024-08-31T07:09:47.662648+0000","last_active":"2024-08-31T07:09:49.660768+0000","last_peered":"2024-08-31T07:09:49.660768+0000","last_clean":"2024-08-31T07:09:49.660768+0000","last_became_active":"2024-08-31T07:09:47.661167+0000","last_became_peered":"2024-08-31T07:09:47.661167+0000","last_unstale":"2024-08-31T07:09:49.660768+0000","last_undegraded":"2024-08-31T07:09:49.660768+0000","last_fullsized":"2024-08-31T07:09:49.660768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:33:42.127895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.883459+0000","last_change":"2024-08-31T07:09:45.642187+0000","last_active":"2024-08-31T07:10:02.883459+0000","last_peered":"2024-08-31T07:10:02.883459+0000","last_clean":"2024-08-31T07:10:02.883459+0000","last_became_active":"2024-08-31T07:09:45.641721+0000","last_became_peered":"2024-08-31T07:09:45.641721+0000","last_unstale":"2024-08-31T07:10:02.883459+0000","last_undegraded":"2024-08-31T07:10:02.883459+0000","last_fullsized":"2024-08-31T07:10:02.883459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:00:45.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665867+0000","last_change":"2024-08-31T07:09:47.665956+0000","last_active":"2024-08-31T07:09:49.665867+0000","last_peered":"2024-08-31T07:09:49.665867+0000","last_clean":"2024-08-31T07:09:49.665867+0000","last_became_active":"2024-08-31T07:09:47.664367+0000","last_became_peered":"2024-08-31T07:09:47.664367+0000","last_unstale":"2024-08-31T07:09:49.665867+0000","last_undegraded":"2024-08-31T07:09:49.665867+0000","last_fullsized":"2024-08-31T07:09:49.665867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:34.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:19.911195+0000","last_change":"2024-08-31T07:09:45.636440+0000","last_active":"2024-08-31T07:10:19.911195+0000","last_peered":"2024-08-31T07:10:19.911195+0000","last_clean":"2024-08-31T07:10:19.911195+0000","last_became_active":"2024-08-31T07:09:45.636058+0000","last_became_peered":"2024-08-31T07:09:45.636058+0000","last_unstale":"2024-08-31T07:10:19.911195+0000","last_undegraded":"2024-08-31T07:10:19.911195+0000","last_fullsized":"2024-08-31T07:10:19.911195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:38.929227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.666093+0000","last_change":"2024-08-31T07:09:47.660564+0000","last_active":"2024-08-31T07:09:49.666093+0000","last_peered":"2024-08-31T07:09:49.666093+0000","last_clean":"2024-08-31T07:09:49.666093+0000","last_became_active":"2024-08-31T07:09:47.660157+0000","last_became_peered":"2024-08-31T07:09:47.660157+0000","last_unstale":"2024-08-31T07:09:49.666093+0000","last_undegraded":"2024-08-31T07:09:49.666093+0000","last_fullsized":"2024-08-31T07:09:49.666093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:49:01.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":39,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.882782+0000","last_change":"2024-08-31T07:09:45.649140+0000","last_active":"2024-08-31T07:10:02.882782+0000","last_peered":"2024-08-31T07:10:02.882782+0000","last_clean":"2024-08-31T07:10:02.882782+0000","last_became_active":"2024-08-31T07:09:45.648796+0000","last_became_peered":"2024-08-31T07:09:45.648796+0000","last_unstale":"2024-08-31T07:10:02.882782+0000","last_undegraded":"2024-08-31T07:10:02.882782+0000","last_fullsized":"2024-08-31T07:10:02.882782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:33:37.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661343+0000","last_change":"2024-08-31T07:09:47.663833+0000","last_active":"2024-08-31T07:09:49.661343+0000","last_peered":"2024-08-31T07:09:49.661343+0000","last_clean":"2024-08-31T07:09:49.661343+0000","last_became_active":"2024-08-31T07:09:47.663451+0000","last_became_peered":"2024-08-31T07:09:47.663451+0000","last_unstale":"2024-08-31T07:09:49.661343+0000","last_undegraded":"2024-08-31T07:09:49.661343+0000","last_fullsized":"2024-08-31T07:09:49.661343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:27:22.245254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:01.854727+0000","last_change":"2024-08-31T07:09:45.636593+0000","last_active":"2024-08-31T07:10:01.854727+0000","last_peered":"2024-08-31T07:10:01.854727+0000","last_clean":"2024-08-31T07:10:01.854727+0000","last_became_active":"2024-08-31T07:09:45.636166+0000","last_became_peered":"2024-08-31T07:09:45.636166+0000","last_unstale":"2024-08-31T07:10:01.854727+0000","last_undegraded":"2024-08-31T07:10:01.854727+0000","last_fullsized":"2024-08-31T07:10:01.854727+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:28.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661580+0000","last_change":"2024-08-31T07:09:47.660825+0000","last_active":"2024-08-31T07:09:49.661580+0000","last_peered":"2024-08-31T07:09:49.661580+0000","last_clean":"2024-08-31T07:09:49.661580+0000","last_became_active":"2024-08-31T07:09:47.660474+0000","last_became_peered":"2024-08-31T07:09:47.660474+0000","last_unstale":"2024-08-31T07:09:49.661580+0000","last_undegraded":"2024-08-31T07:09:49.661580+0000","last_fullsized":"2024-08-31T07:09:49.661580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:42:20.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.881356+0000","last_change":"2024-08-31T07:09:45.632727+0000","last_active":"2024-08-31T07:10:02.881356+0000","last_peered":"2024-08-31T07:10:02.881356+0000","last_clean":"2024-08-31T07:10:02.881356+0000","last_became_active":"2024-08-31T07:09:45.632251+0000","last_became_peered":"2024-08-31T07:09:45.632251+0000","last_unstale":"2024-08-31T07:10:02.881356+0000","last_undegraded":"2024-08-31T07:10:02.881356+0000","last_fullsized":"2024-08-31T07:10:02.881356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:09:30.197409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665265+0000","last_change":"2024-08-31T07:09:47.665501+0000","last_active":"2024-08-31T07:09:49.665265+0000","last_peered":"2024-08-31T07:09:49.665265+0000","last_clean":"2024-08-31T07:09:49.665265+0000","last_became_active":"2024-08-31T07:09:47.665162+0000","last_became_peered":"2024-08-31T07:09:47.665162+0000","last_unstale":"2024-08-31T07:09:49.665265+0000","last_undegraded":"2024-08-31T07:09:49.665265+0000","last_fullsized":"2024-08-31T07:09:49.665265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:34:29.605836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.883385+0000","last_change":"2024-08-31T07:09:45.636580+0000","last_active":"2024-08-31T07:10:02.883385+0000","last_peered":"2024-08-31T07:10:02.883385+0000","last_clean":"2024-08-31T07:10:02.883385+0000","last_became_active":"2024-08-31T07:09:45.636163+0000","last_became_peered":"2024-08-31T07:09:45.636163+0000","last_unstale":"2024-08-31T07:10:02.883385+0000","last_undegraded":"2024-08-31T07:10:02.883385+0000","last_fullsized":"2024-08-31T07:10:02.883385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:18:07.567858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.666257+0000","last_change":"2024-08-31T07:09:47.671865+0000","last_active":"2024-08-31T07:09:49.666257+0000","last_peered":"2024-08-31T07:09:49.666257+0000","last_clean":"2024-08-31T07:09:49.666257+0000","last_became_active":"2024-08-31T07:09:47.671454+0000","last_became_peered":"2024-08-31T07:09:47.671454+0000","last_unstale":"2024-08-31T07:09:49.666257+0000","last_undegraded":"2024-08-31T07:09:49.666257+0000","last_fullsized":"2024-08-31T07:09:49.666257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:47.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'6","reported_seq":46,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:17.908227+0000","last_change":"2024-08-31T07:09:45.653217+0000","last_active":"2024-08-31T07:10:17.908227+0000","last_peered":"2024-08-31T07:10:17.908227+0000","last_clean":"2024-08-31T07:10:17.908227+0000","last_became_active":"2024-08-31T07:09:45.652819+0000","last_became_peered":"2024-08-31T07:09:45.652819+0000","last_unstale":"2024-08-31T07:10:17.908227+0000","last_undegraded":"2024-08-31T07:10:17.908227+0000","last_fullsized":"2024-08-31T07:10:17.908227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:25:11.454068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.666257+0000","last_change":"2024-08-31T07:09:47.663652+0000","last_active":"2024-08-31T07:09:49.666257+0000","last_peered":"2024-08-31T07:09:49.666257+0000","last_clean":"2024-08-31T07:09:49.666257+0000","last_became_active":"2024-08-31T07:09:47.663274+0000","last_became_peered":"2024-08-31T07:09:47.663274+0000","last_unstale":"2024-08-31T07:09:49.666257+0000","last_undegraded":"2024-08-31T07:09:49.666257+0000","last_fullsized":"2024-08-31T07:09:49.666257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:31:59.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665834+0000","last_change":"2024-08-31T07:08:27.623298+0000","last_active":"2024-08-31T07:09:49.665834+0000","last_peered":"2024-08-31T07:09:49.665834+0000","last_clean":"2024-08-31T07:09:49.665834+0000","last_became_active":"2024-08-31T07:08:27.619072+0000","last_became_peered":"2024-08-31T07:08:27.619072+0000","last_unstale":"2024-08-31T07:09:49.665834+0000","last_undegraded":"2024-08-31T07:09:49.665834+0000","last_fullsized":"2024-08-31T07:09:49.665834+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_clean_scrub_stamp":"2024-08-31T07:04:45.509398+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:07.967421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:03.881045+0000","last_change":"2024-08-31T07:09:45.638104+0000","last_active":"2024-08-31T07:10:03.881045+0000","last_peered":"2024-08-31T07:10:03.881045+0000","last_clean":"2024-08-31T07:10:03.881045+0000","last_became_active":"2024-08-31T07:09:45.637693+0000","last_became_peered":"2024-08-31T07:09:45.637693+0000","last_unstale":"2024-08-31T07:10:03.881045+0000","last_undegraded":"2024-08-31T07:10:03.881045+0000","last_fullsized":"2024-08-31T07:10:03.881045+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:50:15.135061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.650982+0000","last_change":"2024-08-31T07:09:47.664958+0000","last_active":"2024-08-31T07:09:49.650982+0000","last_peered":"2024-08-31T07:09:49.650982+0000","last_clean":"2024-08-31T07:09:49.650982+0000","last_became_active":"2024-08-31T07:09:47.661166+0000","last_became_peered":"2024-08-31T07:09:47.661166+0000","last_unstale":"2024-08-31T07:09:49.650982+0000","last_undegraded":"2024-08-31T07:09:49.650982+0000","last_fullsized":"2024-08-31T07:09:49.650982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:45:58.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:00.859595+0000","last_change":"2024-08-31T07:09:45.640881+0000","last_active":"2024-08-31T07:10:00.859595+0000","last_peered":"2024-08-31T07:10:00.859595+0000","last_clean":"2024-08-31T07:10:00.859595+0000","last_became_active":"2024-08-31T07:09:45.640487+0000","last_became_peered":"2024-08-31T07:09:45.640487+0000","last_unstale":"2024-08-31T07:10:00.859595+0000","last_undegraded":"2024-08-31T07:10:00.859595+0000","last_fullsized":"2024-08-31T07:10:00.859595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:06.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.660262+0000","last_change":"2024-08-31T07:09:47.655008+0000","last_active":"2024-08-31T07:09:49.660262+0000","last_peered":"2024-08-31T07:09:49.660262+0000","last_clean":"2024-08-31T07:09:49.660262+0000","last_became_active":"2024-08-31T07:09:47.654645+0000","last_became_peered":"2024-08-31T07:09:47.654645+0000","last_unstale":"2024-08-31T07:09:49.660262+0000","last_undegraded":"2024-08-31T07:09:49.660262+0000","last_fullsized":"2024-08-31T07:09:49.660262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:23:55.916053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:16.905891+0000","last_change":"2024-08-31T07:09:45.649324+0000","last_active":"2024-08-31T07:10:16.905891+0000","last_peered":"2024-08-31T07:10:16.905891+0000","last_clean":"2024-08-31T07:10:16.905891+0000","last_became_active":"2024-08-31T07:09:45.648915+0000","last_became_peered":"2024-08-31T07:09:45.648915+0000","last_unstale":"2024-08-31T07:10:16.905891+0000","last_undegraded":"2024-08-31T07:10:16.905891+0000","last_fullsized":"2024-08-31T07:10:16.905891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:31:57.853654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.662189+0000","last_change":"2024-08-31T07:09:47.661328+0000","last_active":"2024-08-31T07:09:49.662189+0000","last_peered":"2024-08-31T07:09:49.662189+0000","last_clean":"2024-08-31T07:09:49.662189+0000","last_became_active":"2024-08-31T07:09:47.660927+0000","last_became_peered":"2024-08-31T07:09:47.660927+0000","last_unstale":"2024-08-31T07:09:49.662189+0000","last_undegraded":"2024-08-31T07:09:49.662189+0000","last_fullsized":"2024-08-31T07:09:49.662189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:40:46.876894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'6","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:16.906910+0000","last_change":"2024-08-31T07:09:45.637747+0000","last_active":"2024-08-31T07:10:16.906910+0000","last_peered":"2024-08-31T07:10:16.906910+0000","last_clean":"2024-08-31T07:10:16.906910+0000","last_became_active":"2024-08-31T07:09:45.637368+0000","last_became_peered":"2024-08-31T07:09:45.637368+0000","last_unstale":"2024-08-31T07:10:16.906910+0000","last_undegraded":"2024-08-31T07:10:16.906910+0000","last_fullsized":"2024-08-31T07:10:16.906910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:01.174109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.666099+0000","last_change":"2024-08-31T07:09:47.680802+0000","last_active":"2024-08-31T07:09:49.666099+0000","last_peered":"2024-08-31T07:09:49.666099+0000","last_clean":"2024-08-31T07:09:49.666099+0000","last_became_active":"2024-08-31T07:09:47.680363+0000","last_became_peered":"2024-08-31T07:09:47.680363+0000","last_unstale":"2024-08-31T07:09:49.666099+0000","last_undegraded":"2024-08-31T07:09:49.666099+0000","last_fullsized":"2024-08-31T07:09:49.666099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:46:48.744071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665680+0000","last_change":"2024-08-31T07:09:45.651222+0000","last_active":"2024-08-31T07:09:49.665680+0000","last_peered":"2024-08-31T07:09:49.665680+0000","last_clean":"2024-08-31T07:09:49.665680+0000","last_became_active":"2024-08-31T07:09:45.650760+0000","last_became_peered":"2024-08-31T07:09:45.650760+0000","last_unstale":"2024-08-31T07:09:49.665680+0000","last_undegraded":"2024-08-31T07:09:49.665680+0000","last_fullsized":"2024-08-31T07:09:49.665680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T19:03:33.942165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.667511+0000","last_change":"2024-08-31T07:09:47.665881+0000","last_active":"2024-08-31T07:09:49.667511+0000","last_peered":"2024-08-31T07:09:49.667511+0000","last_clean":"2024-08-31T07:09:49.667511+0000","last_became_active":"2024-08-31T07:09:47.665485+0000","last_became_peered":"2024-08-31T07:09:47.665485+0000","last_unstale":"2024-08-31T07:09:49.667511+0000","last_undegraded":"2024-08-31T07:09:49.667511+0000","last_fullsized":"2024-08-31T07:09:49.667511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:30.820714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.882688+0000","last_change":"2024-08-31T07:09:45.651893+0000","last_active":"2024-08-31T07:10:02.882688+0000","last_peered":"2024-08-31T07:10:02.882688+0000","last_clean":"2024-08-31T07:10:02.882688+0000","last_became_active":"2024-08-31T07:09:45.651547+0000","last_became_peered":"2024-08-31T07:09:45.651547+0000","last_unstale":"2024-08-31T07:10:02.882688+0000","last_undegraded":"2024-08-31T07:10:02.882688+0000","last_fullsized":"2024-08-31T07:10:02.882688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:11:30.708018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661581+0000","last_change":"2024-08-31T07:09:47.655551+0000","last_active":"2024-08-31T07:09:49.661581+0000","last_peered":"2024-08-31T07:09:49.661581+0000","last_clean":"2024-08-31T07:09:49.661581+0000","last_became_active":"2024-08-31T07:09:47.655188+0000","last_became_peered":"2024-08-31T07:09:47.655188+0000","last_unstale":"2024-08-31T07:09:49.661581+0000","last_undegraded":"2024-08-31T07:09:49.661581+0000","last_fullsized":"2024-08-31T07:09:49.661581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:43:51.419599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'13","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:16.659560+0000","last_change":"2024-08-31T07:09:45.636212+0000","last_active":"2024-08-31T07:10:16.659560+0000","last_peered":"2024-08-31T07:10:16.659560+0000","last_clean":"2024-08-31T07:10:16.659560+0000","last_became_active":"2024-08-31T07:09:45.635678+0000","last_became_peered":"2024-08-31T07:09:45.635678+0000","last_unstale":"2024-08-31T07:10:16.659560+0000","last_undegraded":"2024-08-31T07:10:16.659560+0000","last_fullsized":"2024-08-31T07:10:16.659560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:13:17.803459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.651577+0000","last_change":"2024-08-31T07:09:47.666476+0000","last_active":"2024-08-31T07:09:49.651577+0000","last_peered":"2024-08-31T07:09:49.651577+0000","last_clean":"2024-08-31T07:09:49.651577+0000","last_became_active":"2024-08-31T07:09:47.666111+0000","last_became_peered":"2024-08-31T07:09:47.666111+0000","last_unstale":"2024-08-31T07:09:49.651577+0000","last_undegraded":"2024-08-31T07:09:49.651577+0000","last_fullsized":"2024-08-31T07:09:49.651577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:08:53.459549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.881628+0000","last_change":"2024-08-31T07:09:45.631571+0000","last_active":"2024-08-31T07:10:02.881628+0000","last_peered":"2024-08-31T07:10:02.881628+0000","last_clean":"2024-08-31T07:10:02.881628+0000","last_became_active":"2024-08-31T07:09:45.630981+0000","last_became_peered":"2024-08-31T07:09:45.630981+0000","last_unstale":"2024-08-31T07:10:02.881628+0000","last_undegraded":"2024-08-31T07:10:02.881628+0000","last_fullsized":"2024-08-31T07:10:02.881628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:36:53.482955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.653216+0000","last_change":"2024-08-31T07:09:47.675054+0000","last_active":"2024-08-31T07:09:49.653216+0000","last_peered":"2024-08-31T07:09:49.653216+0000","last_clean":"2024-08-31T07:09:49.653216+0000","last_became_active":"2024-08-31T07:09:47.672652+0000","last_became_peered":"2024-08-31T07:09:47.672652+0000","last_unstale":"2024-08-31T07:09:49.653216+0000","last_undegraded":"2024-08-31T07:09:49.653216+0000","last_fullsized":"2024-08-31T07:09:49.653216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:32:05.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:59.855841+0000","last_change":"2024-08-31T07:09:45.648746+0000","last_active":"2024-08-31T07:09:59.855841+0000","last_peered":"2024-08-31T07:09:59.855841+0000","last_clean":"2024-08-31T07:09:59.855841+0000","last_became_active":"2024-08-31T07:09:45.648424+0000","last_became_peered":"2024-08-31T07:09:45.648424+0000","last_unstale":"2024-08-31T07:09:59.855841+0000","last_undegraded":"2024-08-31T07:09:59.855841+0000","last_fullsized":"2024-08-31T07:09:59.855841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:40:32.132963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.667942+0000","last_change":"2024-08-31T07:09:47.663213+0000","last_active":"2024-08-31T07:09:49.667942+0000","last_peered":"2024-08-31T07:09:49.667942+0000","last_clean":"2024-08-31T07:09:49.667942+0000","last_became_active":"2024-08-31T07:09:47.662457+0000","last_became_peered":"2024-08-31T07:09:47.662457+0000","last_unstale":"2024-08-31T07:09:49.667942+0000","last_undegraded":"2024-08-31T07:09:49.667942+0000","last_fullsized":"2024-08-31T07:09:49.667942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:54:27.069200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.882950+0000","last_change":"2024-08-31T07:09:45.640149+0000","last_active":"2024-08-31T07:10:02.882950+0000","last_peered":"2024-08-31T07:10:02.882950+0000","last_clean":"2024-08-31T07:10:02.882950+0000","last_became_active":"2024-08-31T07:09:45.639581+0000","last_became_peered":"2024-08-31T07:09:45.639581+0000","last_unstale":"2024-08-31T07:10:02.882950+0000","last_undegraded":"2024-08-31T07:10:02.882950+0000","last_fullsized":"2024-08-31T07:10:02.882950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:27:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661706+0000","last_change":"2024-08-31T07:09:47.672613+0000","last_active":"2024-08-31T07:09:49.661706+0000","last_peered":"2024-08-31T07:09:49.661706+0000","last_clean":"2024-08-31T07:09:49.661706+0000","last_became_active":"2024-08-31T07:09:47.672173+0000","last_became_peered":"2024-08-31T07:09:47.672173+0000","last_unstale":"2024-08-31T07:09:49.661706+0000","last_undegraded":"2024-08-31T07:09:49.661706+0000","last_fullsized":"2024-08-31T07:09:49.661706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:49.706113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:01.868486+0000","last_change":"2024-08-31T07:09:45.637567+0000","last_active":"2024-08-31T07:10:01.868486+0000","last_peered":"2024-08-31T07:10:01.868486+0000","last_clean":"2024-08-31T07:10:01.868486+0000","last_became_active":"2024-08-31T07:09:45.637180+0000","last_became_peered":"2024-08-31T07:09:45.637180+0000","last_unstale":"2024-08-31T07:10:01.868486+0000","last_undegraded":"2024-08-31T07:10:01.868486+0000","last_fullsized":"2024-08-31T07:10:01.868486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:48:07.190829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.651923+0000","last_change":"2024-08-31T07:09:47.674980+0000","last_active":"2024-08-31T07:09:49.651923+0000","last_peered":"2024-08-31T07:09:49.651923+0000","last_clean":"2024-08-31T07:09:49.651923+0000","last_became_active":"2024-08-31T07:09:47.674528+0000","last_became_peered":"2024-08-31T07:09:47.674528+0000","last_unstale":"2024-08-31T07:09:49.651923+0000","last_undegraded":"2024-08-31T07:09:49.651923+0000","last_fullsized":"2024-08-31T07:09:49.651923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:15:09.464221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:59.854478+0000","last_change":"2024-08-31T07:09:45.641167+0000","last_active":"2024-08-31T07:09:59.854478+0000","last_peered":"2024-08-31T07:09:59.854478+0000","last_clean":"2024-08-31T07:09:59.854478+0000","last_became_active":"2024-08-31T07:09:45.640761+0000","last_became_peered":"2024-08-31T07:09:45.640761+0000","last_unstale":"2024-08-31T07:09:59.854478+0000","last_undegraded":"2024-08-31T07:09:59.854478+0000","last_fullsized":"2024-08-31T07:09:59.854478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:28:59.963145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.666431+0000","last_change":"2024-08-31T07:09:47.668808+0000","last_active":"2024-08-31T07:09:49.666431+0000","last_peered":"2024-08-31T07:09:49.666431+0000","last_clean":"2024-08-31T07:09:49.666431+0000","last_became_active":"2024-08-31T07:09:47.668291+0000","last_became_peered":"2024-08-31T07:09:47.668291+0000","last_unstale":"2024-08-31T07:09:49.666431+0000","last_undegraded":"2024-08-31T07:09:49.666431+0000","last_fullsized":"2024-08-31T07:09:49.666431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:23.433511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.662765+0000","last_change":"2024-08-31T07:09:47.668039+0000","last_active":"2024-08-31T07:09:49.662765+0000","last_peered":"2024-08-31T07:09:49.662765+0000","last_clean":"2024-08-31T07:09:49.662765+0000","last_became_active":"2024-08-31T07:09:47.667567+0000","last_became_peered":"2024-08-31T07:09:47.667567+0000","last_unstale":"2024-08-31T07:09:49.662765+0000","last_undegraded":"2024-08-31T07:09:49.662765+0000","last_fullsized":"2024-08-31T07:09:49.662765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:33:50.192864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.882828+0000","last_change":"2024-08-31T07:09:45.636229+0000","last_active":"2024-08-31T07:10:02.882828+0000","last_peered":"2024-08-31T07:10:02.882828+0000","last_clean":"2024-08-31T07:10:02.882828+0000","last_became_active":"2024-08-31T07:09:45.635767+0000","last_became_peered":"2024-08-31T07:09:45.635767+0000","last_unstale":"2024-08-31T07:10:02.882828+0000","last_undegraded":"2024-08-31T07:10:02.882828+0000","last_fullsized":"2024-08-31T07:10:02.882828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:55:11.092133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.670756+0000","last_change":"2024-08-31T07:09:47.660615+0000","last_active":"2024-08-31T07:09:49.670756+0000","last_peered":"2024-08-31T07:09:49.670756+0000","last_clean":"2024-08-31T07:09:49.670756+0000","last_became_active":"2024-08-31T07:09:47.660269+0000","last_became_peered":"2024-08-31T07:09:47.660269+0000","last_unstale":"2024-08-31T07:09:49.670756+0000","last_undegraded":"2024-08-31T07:09:49.670756+0000","last_fullsized":"2024-08-31T07:09:49.670756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:42:12.618039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:03.875405+0000","last_change":"2024-08-31T07:09:45.641699+0000","last_active":"2024-08-31T07:10:03.875405+0000","last_peered":"2024-08-31T07:10:03.875405+0000","last_clean":"2024-08-31T07:10:03.875405+0000","last_became_active":"2024-08-31T07:09:45.641122+0000","last_became_peered":"2024-08-31T07:09:45.641122+0000","last_unstale":"2024-08-31T07:10:03.875405+0000","last_undegraded":"2024-08-31T07:10:03.875405+0000","last_fullsized":"2024-08-31T07:10:03.875405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:26:07.130266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.653822+0000","last_change":"2024-08-31T07:09:47.675459+0000","last_active":"2024-08-31T07:09:49.653822+0000","last_peered":"2024-08-31T07:09:49.653822+0000","last_clean":"2024-08-31T07:09:49.653822+0000","last_became_active":"2024-08-31T07:09:47.672246+0000","last_became_peered":"2024-08-31T07:09:47.672246+0000","last_unstale":"2024-08-31T07:09:49.653822+0000","last_undegraded":"2024-08-31T07:09:49.653822+0000","last_fullsized":"2024-08-31T07:09:49.653822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:49:23.547352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664498+0000","last_change":"2024-08-31T07:09:45.649522+0000","last_active":"2024-08-31T07:09:49.664498+0000","last_peered":"2024-08-31T07:09:49.664498+0000","last_clean":"2024-08-31T07:09:49.664498+0000","last_became_active":"2024-08-31T07:09:45.649109+0000","last_became_peered":"2024-08-31T07:09:45.649109+0000","last_unstale":"2024-08-31T07:09:49.664498+0000","last_undegraded":"2024-08-31T07:09:49.664498+0000","last_fullsized":"2024-08-31T07:09:49.664498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:52:52.432605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.651317+0000","last_change":"2024-08-31T07:09:47.670583+0000","last_active":"2024-08-31T07:09:49.651317+0000","last_peered":"2024-08-31T07:09:49.651317+0000","last_clean":"2024-08-31T07:09:49.651317+0000","last_became_active":"2024-08-31T07:09:47.670162+0000","last_became_peered":"2024-08-31T07:09:47.670162+0000","last_unstale":"2024-08-31T07:09:49.651317+0000","last_undegraded":"2024-08-31T07:09:49.651317+0000","last_fullsized":"2024-08-31T07:09:49.651317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:35:35.135678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":52,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:18.909212+0000","last_change":"2024-08-31T07:09:45.639636+0000","last_active":"2024-08-31T07:10:18.909212+0000","last_peered":"2024-08-31T07:10:18.909212+0000","last_clean":"2024-08-31T07:10:18.909212+0000","last_became_active":"2024-08-31T07:09:45.639029+0000","last_became_peered":"2024-08-31T07:09:45.639029+0000","last_unstale":"2024-08-31T07:10:18.909212+0000","last_undegraded":"2024-08-31T07:10:18.909212+0000","last_fullsized":"2024-08-31T07:10:18.909212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:54.110149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62358,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":212,"num_read_kb":171,"num_write":240,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":178560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285320,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678036,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":416266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874265,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":1061667,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037790,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":415805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234021,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":416183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397545,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":1009483,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561069,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":415913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":1008379,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920826,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":415774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117055,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":415347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378693,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":467552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542219,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":466941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705743,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":417252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":52258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T07:10:23.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T07:10:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:24.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:23 smithi084 ceph-mon[22387]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 KiB/s wr, 14 op/s 2024-08-31T07:10:24.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:23 smithi084 ceph-mon[22387]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:10:24.356 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:23 smithi032 ceph-mon[17252]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 KiB/s wr, 14 op/s 2024-08-31T07:10:24.356 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:23 smithi032 ceph-mon[17252]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:10:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:23 smithi073 ceph-mon[21995]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.4 KiB/s wr, 14 op/s 2024-08-31T07:10:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:23 smithi073 ceph-mon[21995]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:10:24.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.691+0000 7f7d07200640 1 -- 172.21.15.32:0/2454786889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d00103920 msgr2=0x7f7d00103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:24.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.691+0000 7f7d07200640 1 --2- 172.21.15.32:0/2454786889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d00103920 0x7f7d00103d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009900 tx=0x7f7cf002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:10:24.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.691+0000 7f7d07200640 1 -- 172.21.15.32:0/2454786889 shutdown_connections 2024-08-31T07:10:24.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.691+0000 7f7d07200640 1 --2- 172.21.15.32:0/2454786889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d00109bb0 0x7f7d001103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:24.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.691+0000 7f7d07200640 1 --2- 172.21.15.32:0/2454786889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d00109260 0x7f7d00109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:24.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.691+0000 7f7d07200640 1 --2- 172.21.15.32:0/2454786889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d00103920 0x7f7d00103d30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:24.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.691+0000 7f7d07200640 1 -- 172.21.15.32:0/2454786889 >> 172.21.15.32:0/2454786889 conn(0x7f7d000ff3b0 msgr2=0x7f7d00101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:24.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.692+0000 7f7d07200640 1 -- 172.21.15.32:0/2454786889 shutdown_connections 2024-08-31T07:10:24.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.692+0000 7f7d07200640 1 -- 172.21.15.32:0/2454786889 wait complete. 2024-08-31T07:10:24.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.693+0000 7f7d07200640 1 Processor -- start 2024-08-31T07:10:24.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.693+0000 7f7d07200640 1 -- start start 2024-08-31T07:10:24.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.693+0000 7f7d07200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d00103920 0x7f7d0019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:24.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.693+0000 7f7d07200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d00109260 0x7f7d0019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:24.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.693+0000 7f7d07200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d00109bb0 0x7f7d001a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:24.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.693+0000 7f7d07200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00113870 con 0x7f7d00109260 2024-08-31T07:10:24.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.693+0000 7f7d07200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001136f0 con 0x7f7d00103920 2024-08-31T07:10:24.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.693+0000 7f7d07200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001139f0 con 0x7f7d00109bb0 2024-08-31T07:10:24.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7cf7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d00109260 0x7f7d0019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:24.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7cf7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d00109260 0x7f7d0019ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39418/0 (socket says 172.21.15.32:39418) 2024-08-31T07:10:24.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7cf7e00640 1 -- 172.21.15.32:0/1919315693 learned_addr learned my addr 172.21.15.32:0/1919315693 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:24.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7d05200640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d00109bb0 0x7f7d001a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:24.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7cffe00640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d00103920 0x7f7d0019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:24.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7d05200640 1 -- 172.21.15.32:0/1919315693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d00103920 msgr2=0x7f7d0019c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:24.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7d05200640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d00103920 0x7f7d0019c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:24.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7d05200640 1 -- 172.21.15.32:0/1919315693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d00109260 msgr2=0x7f7d0019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:24.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7d05200640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d00109260 0x7f7d0019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:24.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7d05200640 1 -- 172.21.15.32:0/1919315693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d001a23d0 con 0x7f7d00109bb0 2024-08-31T07:10:24.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.694+0000 7f7cf7e00640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d00109260 0x7f7d0019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:24.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.695+0000 7f7d05200640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d00109bb0 0x7f7d001a1ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009fd0 tx=0x7f7cf800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:24.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.695+0000 7f7cfd600640 1 -- 172.21.15.32:0/1919315693 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf800ce20 con 0x7f7d00109bb0 2024-08-31T07:10:24.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.695+0000 7f7d07200640 1 -- 172.21.15.32:0/1919315693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d001a26c0 con 0x7f7d00109bb0 2024-08-31T07:10:24.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.695+0000 7f7cfd600640 1 -- 172.21.15.32:0/1919315693 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf8005020 con 0x7f7d00109bb0 2024-08-31T07:10:24.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.695+0000 7f7cfd600640 1 -- 172.21.15.32:0/1919315693 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8010690 con 0x7f7d00109bb0 2024-08-31T07:10:24.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.695+0000 7f7d07200640 1 -- 172.21.15.32:0/1919315693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d001a8e20 con 0x7f7d00109bb0 2024-08-31T07:10:24.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.696+0000 7f7d07200640 1 -- 172.21.15.32:0/1919315693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d00104ba0 con 0x7f7d00109bb0 2024-08-31T07:10:24.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.699+0000 7f7cfd600640 1 -- 172.21.15.32:0/1919315693 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f7cf80054d0 con 0x7f7d00109bb0 2024-08-31T07:10:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.700+0000 7f7cfd600640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ce40b4c10 0x7f7ce40b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.700+0000 7f7cfd600640 1 -- 172.21.15.32:0/1919315693 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f7cf80e5a10 con 0x7f7d00109bb0 2024-08-31T07:10:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.700+0000 7f7cffe00640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ce40b4c10 0x7f7ce40b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.701+0000 7f7cffe00640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ce40b4c10 0x7f7ce40b7120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0031440 tx=0x7f7cf0009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:24.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.702+0000 7f7cfd600640 1 -- 172.21.15.32:0/1919315693 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7cf80ac9a0 con 0x7f7d00109bb0 2024-08-31T07:10:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.894+0000 7f7d07200640 1 -- 172.21.15.32:0/1919315693 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d0010f690 con 0x7f7ce40b4c10 2024-08-31T07:10:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.900+0000 7f7cfd600640 1 -- 172.21.15.32:0/1919315693 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258902 (secure 0 0 0) 0x7f7d0010f690 con 0x7f7ce40b4c10 2024-08-31T07:10:24.904 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:10:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-08-31T07:10:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.904+0000 7f7d07200640 1 -- 172.21.15.32:0/1919315693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ce40b4c10 msgr2=0x7f7ce40b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.904+0000 7f7d07200640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ce40b4c10 0x7f7ce40b7120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0031440 tx=0x7f7cf0009560 comp rx=0 tx=0).stop 2024-08-31T07:10:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.904+0000 7f7d07200640 1 -- 172.21.15.32:0/1919315693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d00109bb0 msgr2=0x7f7d001a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.904+0000 7f7d07200640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d00109bb0 0x7f7d001a1ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009fd0 tx=0x7f7cf800ef10 comp rx=0 tx=0).stop 2024-08-31T07:10:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.904+0000 7f7d07200640 1 -- 172.21.15.32:0/1919315693 shutdown_connections 2024-08-31T07:10:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.904+0000 7f7d07200640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ce40b4c10 0x7f7ce40b7120 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.904+0000 7f7d07200640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d00109bb0 0x7f7d001a1ca0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.904+0000 7f7d07200640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d00109260 0x7f7d0019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.904+0000 7f7d07200640 1 --2- 172.21.15.32:0/1919315693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d00103920 0x7f7d0019c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.904+0000 7f7d07200640 1 -- 172.21.15.32:0/1919315693 >> 172.21.15.32:0/1919315693 conn(0x7f7d000ff3b0 msgr2=0x7f7d001088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:24.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.905+0000 7f7d07200640 1 -- 172.21.15.32:0/1919315693 shutdown_connections 2024-08-31T07:10:24.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:24.905+0000 7f7d07200640 1 -- 172.21.15.32:0/1919315693 wait complete. 2024-08-31T07:10:25.667 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-08-31T07:10:24.765808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652960,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":308,"num_read_kb":245,"num_write":365,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363904,"kb_used_data":10496,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577920,"statfs":{"total":1151940427776,"available":1151567790080,"internally_reserved":0,"allocated":10747904,"data_stored":6930819,"data_compressed":108822,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27544,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":119,"num_read_kb":84,"num_write":19,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002996"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661879+0000","last_change":"2024-08-31T07:09:47.665725+0000","last_active":"2024-08-31T07:09:49.661879+0000","last_peered":"2024-08-31T07:09:49.661879+0000","last_clean":"2024-08-31T07:09:49.661879+0000","last_became_active":"2024-08-31T07:09:47.665368+0000","last_became_peered":"2024-08-31T07:09:47.665368+0000","last_unstale":"2024-08-31T07:09:49.661879+0000","last_undegraded":"2024-08-31T07:09:49.661879+0000","last_fullsized":"2024-08-31T07:09:49.661879+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:27:01.636099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.652993+0000","last_change":"2024-08-31T07:09:47.676882+0000","last_active":"2024-08-31T07:09:49.652993+0000","last_peered":"2024-08-31T07:09:49.652993+0000","last_clean":"2024-08-31T07:09:49.652993+0000","last_became_active":"2024-08-31T07:09:47.674825+0000","last_became_peered":"2024-08-31T07:09:47.674825+0000","last_unstale":"2024-08-31T07:09:49.652993+0000","last_undegraded":"2024-08-31T07:09:49.652993+0000","last_fullsized":"2024-08-31T07:09:49.652993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:09.148398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.652831+0000","last_change":"2024-08-31T07:09:47.674480+0000","last_active":"2024-08-31T07:09:49.652831+0000","last_peered":"2024-08-31T07:09:49.652831+0000","last_clean":"2024-08-31T07:09:49.652831+0000","last_became_active":"2024-08-31T07:09:47.672316+0000","last_became_peered":"2024-08-31T07:09:47.672316+0000","last_unstale":"2024-08-31T07:09:49.652831+0000","last_undegraded":"2024-08-31T07:09:49.652831+0000","last_fullsized":"2024-08-31T07:09:49.652831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:28:13.231039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.654048+0000","last_change":"2024-08-31T07:09:47.659276+0000","last_active":"2024-08-31T07:09:49.654048+0000","last_peered":"2024-08-31T07:09:49.654048+0000","last_clean":"2024-08-31T07:09:49.654048+0000","last_became_active":"2024-08-31T07:09:47.658924+0000","last_became_peered":"2024-08-31T07:09:47.658924+0000","last_unstale":"2024-08-31T07:09:49.654048+0000","last_undegraded":"2024-08-31T07:09:49.654048+0000","last_fullsized":"2024-08-31T07:09:49.654048+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:01:08.134245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.668175+0000","last_change":"2024-08-31T07:09:47.667260+0000","last_active":"2024-08-31T07:09:49.668175+0000","last_peered":"2024-08-31T07:09:49.668175+0000","last_clean":"2024-08-31T07:09:49.668175+0000","last_became_active":"2024-08-31T07:09:47.664572+0000","last_became_peered":"2024-08-31T07:09:47.664572+0000","last_unstale":"2024-08-31T07:09:49.668175+0000","last_undegraded":"2024-08-31T07:09:49.668175+0000","last_fullsized":"2024-08-31T07:09:49.668175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:11.336985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.667372+0000","last_change":"2024-08-31T07:09:47.661181+0000","last_active":"2024-08-31T07:09:49.667372+0000","last_peered":"2024-08-31T07:09:49.667372+0000","last_clean":"2024-08-31T07:09:49.667372+0000","last_became_active":"2024-08-31T07:09:47.660761+0000","last_became_peered":"2024-08-31T07:09:47.660761+0000","last_unstale":"2024-08-31T07:09:49.667372+0000","last_undegraded":"2024-08-31T07:09:49.667372+0000","last_fullsized":"2024-08-31T07:09:49.667372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:50:54.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665627+0000","last_change":"2024-08-31T07:09:47.668762+0000","last_active":"2024-08-31T07:09:49.665627+0000","last_peered":"2024-08-31T07:09:49.665627+0000","last_clean":"2024-08-31T07:09:49.665627+0000","last_became_active":"2024-08-31T07:09:47.668406+0000","last_became_peered":"2024-08-31T07:09:47.668406+0000","last_unstale":"2024-08-31T07:09:49.665627+0000","last_undegraded":"2024-08-31T07:09:49.665627+0000","last_fullsized":"2024-08-31T07:09:49.665627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:43:39.868144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.650611+0000","last_change":"2024-08-31T07:09:47.669455+0000","last_active":"2024-08-31T07:09:49.650611+0000","last_peered":"2024-08-31T07:09:49.650611+0000","last_clean":"2024-08-31T07:09:49.650611+0000","last_became_active":"2024-08-31T07:09:47.663669+0000","last_became_peered":"2024-08-31T07:09:47.663669+0000","last_unstale":"2024-08-31T07:09:49.650611+0000","last_undegraded":"2024-08-31T07:09:49.650611+0000","last_fullsized":"2024-08-31T07:09:49.650611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:22:59.685012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.671722+0000","last_change":"2024-08-31T07:09:47.664214+0000","last_active":"2024-08-31T07:09:49.671722+0000","last_peered":"2024-08-31T07:09:49.671722+0000","last_clean":"2024-08-31T07:09:49.671722+0000","last_became_active":"2024-08-31T07:09:47.663832+0000","last_became_peered":"2024-08-31T07:09:47.663832+0000","last_unstale":"2024-08-31T07:09:49.671722+0000","last_undegraded":"2024-08-31T07:09:49.671722+0000","last_fullsized":"2024-08-31T07:09:49.671722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:29:14.107212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.667863+0000","last_change":"2024-08-31T07:09:47.658369+0000","last_active":"2024-08-31T07:09:49.667863+0000","last_peered":"2024-08-31T07:09:49.667863+0000","last_clean":"2024-08-31T07:09:49.667863+0000","last_became_active":"2024-08-31T07:09:47.656749+0000","last_became_peered":"2024-08-31T07:09:47.656749+0000","last_unstale":"2024-08-31T07:09:49.667863+0000","last_undegraded":"2024-08-31T07:09:49.667863+0000","last_fullsized":"2024-08-31T07:09:49.667863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:16:46.243603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665972+0000","last_change":"2024-08-31T07:09:47.666951+0000","last_active":"2024-08-31T07:09:49.665972+0000","last_peered":"2024-08-31T07:09:49.665972+0000","last_clean":"2024-08-31T07:09:49.665972+0000","last_became_active":"2024-08-31T07:09:47.666475+0000","last_became_peered":"2024-08-31T07:09:47.666475+0000","last_unstale":"2024-08-31T07:09:49.665972+0000","last_undegraded":"2024-08-31T07:09:49.665972+0000","last_fullsized":"2024-08-31T07:09:49.665972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:58:42.242086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.654190+0000","last_change":"2024-08-31T07:09:47.658344+0000","last_active":"2024-08-31T07:09:49.654190+0000","last_peered":"2024-08-31T07:09:49.654190+0000","last_clean":"2024-08-31T07:09:49.654190+0000","last_became_active":"2024-08-31T07:09:47.657991+0000","last_became_peered":"2024-08-31T07:09:47.657991+0000","last_unstale":"2024-08-31T07:09:49.654190+0000","last_undegraded":"2024-08-31T07:09:49.654190+0000","last_fullsized":"2024-08-31T07:09:49.654190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:25:59.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661053+0000","last_change":"2024-08-31T07:09:47.661644+0000","last_active":"2024-08-31T07:09:49.661053+0000","last_peered":"2024-08-31T07:09:49.661053+0000","last_clean":"2024-08-31T07:09:49.661053+0000","last_became_active":"2024-08-31T07:09:47.661248+0000","last_became_peered":"2024-08-31T07:09:47.661248+0000","last_unstale":"2024-08-31T07:09:49.661053+0000","last_undegraded":"2024-08-31T07:09:49.661053+0000","last_fullsized":"2024-08-31T07:09:49.661053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:42.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.652301+0000","last_change":"2024-08-31T07:09:47.680052+0000","last_active":"2024-08-31T07:09:49.652301+0000","last_peered":"2024-08-31T07:09:49.652301+0000","last_clean":"2024-08-31T07:09:49.652301+0000","last_became_active":"2024-08-31T07:09:47.679604+0000","last_became_peered":"2024-08-31T07:09:47.679604+0000","last_unstale":"2024-08-31T07:09:49.652301+0000","last_undegraded":"2024-08-31T07:09:49.652301+0000","last_fullsized":"2024-08-31T07:09:49.652301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:23:18.937470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.651734+0000","last_change":"2024-08-31T07:09:47.677984+0000","last_active":"2024-08-31T07:09:49.651734+0000","last_peered":"2024-08-31T07:09:49.651734+0000","last_clean":"2024-08-31T07:09:49.651734+0000","last_became_active":"2024-08-31T07:09:47.674957+0000","last_became_peered":"2024-08-31T07:09:47.674957+0000","last_unstale":"2024-08-31T07:09:49.651734+0000","last_undegraded":"2024-08-31T07:09:49.651734+0000","last_fullsized":"2024-08-31T07:09:49.651734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:08:26.503746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.670145+0000","last_change":"2024-08-31T07:09:47.672625+0000","last_active":"2024-08-31T07:09:49.670145+0000","last_peered":"2024-08-31T07:09:49.670145+0000","last_clean":"2024-08-31T07:09:49.670145+0000","last_became_active":"2024-08-31T07:09:47.672184+0000","last_became_peered":"2024-08-31T07:09:47.672184+0000","last_unstale":"2024-08-31T07:09:49.670145+0000","last_undegraded":"2024-08-31T07:09:49.670145+0000","last_fullsized":"2024-08-31T07:09:49.670145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:17.091659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.662237+0000","last_change":"2024-08-31T07:09:47.672597+0000","last_active":"2024-08-31T07:09:49.662237+0000","last_peered":"2024-08-31T07:09:49.662237+0000","last_clean":"2024-08-31T07:09:49.662237+0000","last_became_active":"2024-08-31T07:09:47.672153+0000","last_became_peered":"2024-08-31T07:09:47.672153+0000","last_unstale":"2024-08-31T07:09:49.662237+0000","last_undegraded":"2024-08-31T07:09:49.662237+0000","last_fullsized":"2024-08-31T07:09:49.662237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:26:41.388526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664954+0000","last_change":"2024-08-31T07:09:47.666253+0000","last_active":"2024-08-31T07:09:49.664954+0000","last_peered":"2024-08-31T07:09:49.664954+0000","last_clean":"2024-08-31T07:09:49.664954+0000","last_became_active":"2024-08-31T07:09:47.663732+0000","last_became_peered":"2024-08-31T07:09:47.663732+0000","last_unstale":"2024-08-31T07:09:49.664954+0000","last_undegraded":"2024-08-31T07:09:49.664954+0000","last_fullsized":"2024-08-31T07:09:49.664954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:10:59.887765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664632+0000","last_change":"2024-08-31T07:09:47.668502+0000","last_active":"2024-08-31T07:09:49.664632+0000","last_peered":"2024-08-31T07:09:49.664632+0000","last_clean":"2024-08-31T07:09:49.664632+0000","last_became_active":"2024-08-31T07:09:47.666487+0000","last_became_peered":"2024-08-31T07:09:47.666487+0000","last_unstale":"2024-08-31T07:09:49.664632+0000","last_undegraded":"2024-08-31T07:09:49.664632+0000","last_fullsized":"2024-08-31T07:09:49.664632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:49:25.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661288+0000","last_change":"2024-08-31T07:09:47.659454+0000","last_active":"2024-08-31T07:09:49.661288+0000","last_peered":"2024-08-31T07:09:49.661288+0000","last_clean":"2024-08-31T07:09:49.661288+0000","last_became_active":"2024-08-31T07:09:47.658976+0000","last_became_peered":"2024-08-31T07:09:47.658976+0000","last_unstale":"2024-08-31T07:09:49.661288+0000","last_undegraded":"2024-08-31T07:09:49.661288+0000","last_fullsized":"2024-08-31T07:09:49.661288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:31.374905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661088+0000","last_change":"2024-08-31T07:09:47.673612+0000","last_active":"2024-08-31T07:09:49.661088+0000","last_peered":"2024-08-31T07:09:49.661088+0000","last_clean":"2024-08-31T07:09:49.661088+0000","last_became_active":"2024-08-31T07:09:47.673137+0000","last_became_peered":"2024-08-31T07:09:47.673137+0000","last_unstale":"2024-08-31T07:09:49.661088+0000","last_undegraded":"2024-08-31T07:09:49.661088+0000","last_fullsized":"2024-08-31T07:09:49.661088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:46:11.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.670995+0000","last_change":"2024-08-31T07:09:47.669195+0000","last_active":"2024-08-31T07:09:49.670995+0000","last_peered":"2024-08-31T07:09:49.670995+0000","last_clean":"2024-08-31T07:09:49.670995+0000","last_became_active":"2024-08-31T07:09:47.668790+0000","last_became_peered":"2024-08-31T07:09:47.668790+0000","last_unstale":"2024-08-31T07:09:49.670995+0000","last_undegraded":"2024-08-31T07:09:49.670995+0000","last_fullsized":"2024-08-31T07:09:49.670995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:27:14.393931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.650976+0000","last_change":"2024-08-31T07:09:47.667566+0000","last_active":"2024-08-31T07:09:49.650976+0000","last_peered":"2024-08-31T07:09:49.650976+0000","last_clean":"2024-08-31T07:09:49.650976+0000","last_became_active":"2024-08-31T07:09:47.666605+0000","last_became_peered":"2024-08-31T07:09:47.666605+0000","last_unstale":"2024-08-31T07:09:49.650976+0000","last_undegraded":"2024-08-31T07:09:49.650976+0000","last_fullsized":"2024-08-31T07:09:49.650976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:05:44.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661572+0000","last_change":"2024-08-31T07:09:47.660478+0000","last_active":"2024-08-31T07:09:49.661572+0000","last_peered":"2024-08-31T07:09:49.661572+0000","last_clean":"2024-08-31T07:09:49.661572+0000","last_became_active":"2024-08-31T07:09:47.660107+0000","last_became_peered":"2024-08-31T07:09:47.660107+0000","last_unstale":"2024-08-31T07:09:49.661572+0000","last_undegraded":"2024-08-31T07:09:49.661572+0000","last_fullsized":"2024-08-31T07:09:49.661572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:29.133370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.651507+0000","last_change":"2024-08-31T07:09:47.671967+0000","last_active":"2024-08-31T07:09:49.651507+0000","last_peered":"2024-08-31T07:09:49.651507+0000","last_clean":"2024-08-31T07:09:49.651507+0000","last_became_active":"2024-08-31T07:09:47.671569+0000","last_became_peered":"2024-08-31T07:09:47.671569+0000","last_unstale":"2024-08-31T07:09:49.651507+0000","last_undegraded":"2024-08-31T07:09:49.651507+0000","last_fullsized":"2024-08-31T07:09:49.651507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:19:15.107958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665166+0000","last_change":"2024-08-31T07:09:47.651055+0000","last_active":"2024-08-31T07:09:49.665166+0000","last_peered":"2024-08-31T07:09:49.665166+0000","last_clean":"2024-08-31T07:09:49.665166+0000","last_became_active":"2024-08-31T07:09:47.650674+0000","last_became_peered":"2024-08-31T07:09:47.650674+0000","last_unstale":"2024-08-31T07:09:49.665166+0000","last_undegraded":"2024-08-31T07:09:49.665166+0000","last_fullsized":"2024-08-31T07:09:49.665166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:35:54.647379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661824+0000","last_change":"2024-08-31T07:09:47.663231+0000","last_active":"2024-08-31T07:09:49.661824+0000","last_peered":"2024-08-31T07:09:49.661824+0000","last_clean":"2024-08-31T07:09:49.661824+0000","last_became_active":"2024-08-31T07:09:47.662854+0000","last_became_peered":"2024-08-31T07:09:47.662854+0000","last_unstale":"2024-08-31T07:09:49.661824+0000","last_undegraded":"2024-08-31T07:09:49.661824+0000","last_fullsized":"2024-08-31T07:09:49.661824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:51.358557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661452+0000","last_change":"2024-08-31T07:09:47.665096+0000","last_active":"2024-08-31T07:09:49.661452+0000","last_peered":"2024-08-31T07:09:49.661452+0000","last_clean":"2024-08-31T07:09:49.661452+0000","last_became_active":"2024-08-31T07:09:47.664698+0000","last_became_peered":"2024-08-31T07:09:47.664698+0000","last_unstale":"2024-08-31T07:09:49.661452+0000","last_undegraded":"2024-08-31T07:09:49.661452+0000","last_fullsized":"2024-08-31T07:09:49.661452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:44:35.391844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.652735+0000","last_change":"2024-08-31T07:09:47.672108+0000","last_active":"2024-08-31T07:09:49.652735+0000","last_peered":"2024-08-31T07:09:49.652735+0000","last_clean":"2024-08-31T07:09:49.652735+0000","last_became_active":"2024-08-31T07:09:47.670998+0000","last_became_peered":"2024-08-31T07:09:47.670998+0000","last_unstale":"2024-08-31T07:09:49.652735+0000","last_undegraded":"2024-08-31T07:09:49.652735+0000","last_fullsized":"2024-08-31T07:09:49.652735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:26:49.319861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664334+0000","last_change":"2024-08-31T07:09:47.665548+0000","last_active":"2024-08-31T07:09:49.664334+0000","last_peered":"2024-08-31T07:09:49.664334+0000","last_clean":"2024-08-31T07:09:49.664334+0000","last_became_active":"2024-08-31T07:09:47.665177+0000","last_became_peered":"2024-08-31T07:09:47.665177+0000","last_unstale":"2024-08-31T07:09:49.664334+0000","last_undegraded":"2024-08-31T07:09:49.664334+0000","last_fullsized":"2024-08-31T07:09:49.664334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:03:15.892224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664509+0000","last_change":"2024-08-31T07:09:47.659514+0000","last_active":"2024-08-31T07:09:49.664509+0000","last_peered":"2024-08-31T07:09:49.664509+0000","last_clean":"2024-08-31T07:09:49.664509+0000","last_became_active":"2024-08-31T07:09:47.659178+0000","last_became_peered":"2024-08-31T07:09:47.659178+0000","last_unstale":"2024-08-31T07:09:49.664509+0000","last_undegraded":"2024-08-31T07:09:49.664509+0000","last_fullsized":"2024-08-31T07:09:49.664509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:58:04.215441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661046+0000","last_change":"2024-08-31T07:09:47.667456+0000","last_active":"2024-08-31T07:09:49.661046+0000","last_peered":"2024-08-31T07:09:49.661046+0000","last_clean":"2024-08-31T07:09:49.661046+0000","last_became_active":"2024-08-31T07:09:47.667102+0000","last_became_peered":"2024-08-31T07:09:47.667102+0000","last_unstale":"2024-08-31T07:09:49.661046+0000","last_undegraded":"2024-08-31T07:09:49.661046+0000","last_fullsized":"2024-08-31T07:09:49.661046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:44:39.747729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.668077+0000","last_change":"2024-08-31T07:09:47.661099+0000","last_active":"2024-08-31T07:09:49.668077+0000","last_peered":"2024-08-31T07:09:49.668077+0000","last_clean":"2024-08-31T07:09:49.668077+0000","last_became_active":"2024-08-31T07:09:47.660709+0000","last_became_peered":"2024-08-31T07:09:47.660709+0000","last_unstale":"2024-08-31T07:09:49.668077+0000","last_undegraded":"2024-08-31T07:09:49.668077+0000","last_fullsized":"2024-08-31T07:09:49.668077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:03:59.071779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:19.911394+0000","last_change":"2024-08-31T07:09:45.641088+0000","last_active":"2024-08-31T07:10:19.911394+0000","last_peered":"2024-08-31T07:10:19.911394+0000","last_clean":"2024-08-31T07:10:19.911394+0000","last_became_active":"2024-08-31T07:09:45.640673+0000","last_became_peered":"2024-08-31T07:09:45.640673+0000","last_unstale":"2024-08-31T07:10:19.911394+0000","last_undegraded":"2024-08-31T07:10:19.911394+0000","last_fullsized":"2024-08-31T07:10:19.911394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:30:17.593494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.662758+0000","last_change":"2024-08-31T07:09:47.666703+0000","last_active":"2024-08-31T07:09:49.662758+0000","last_peered":"2024-08-31T07:09:49.662758+0000","last_clean":"2024-08-31T07:09:49.662758+0000","last_became_active":"2024-08-31T07:09:47.666348+0000","last_became_peered":"2024-08-31T07:09:47.666348+0000","last_unstale":"2024-08-31T07:09:49.662758+0000","last_undegraded":"2024-08-31T07:09:49.662758+0000","last_fullsized":"2024-08-31T07:09:49.662758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:39.688810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:01.867844+0000","last_change":"2024-08-31T07:09:45.642431+0000","last_active":"2024-08-31T07:10:01.867844+0000","last_peered":"2024-08-31T07:10:01.867844+0000","last_clean":"2024-08-31T07:10:01.867844+0000","last_became_active":"2024-08-31T07:09:45.642048+0000","last_became_peered":"2024-08-31T07:09:45.642048+0000","last_unstale":"2024-08-31T07:10:01.867844+0000","last_undegraded":"2024-08-31T07:10:01.867844+0000","last_fullsized":"2024-08-31T07:10:01.867844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:47:42.574442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:00.863011+0000","last_change":"2024-08-31T07:09:45.632162+0000","last_active":"2024-08-31T07:10:00.863011+0000","last_peered":"2024-08-31T07:10:00.863011+0000","last_clean":"2024-08-31T07:10:00.863011+0000","last_became_active":"2024-08-31T07:09:45.631749+0000","last_became_peered":"2024-08-31T07:09:45.631749+0000","last_unstale":"2024-08-31T07:10:00.863011+0000","last_undegraded":"2024-08-31T07:10:00.863011+0000","last_fullsized":"2024-08-31T07:10:00.863011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:16:29.967823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.650897+0000","last_change":"2024-08-31T07:09:47.666288+0000","last_active":"2024-08-31T07:09:49.650897+0000","last_peered":"2024-08-31T07:09:49.650897+0000","last_clean":"2024-08-31T07:09:49.650897+0000","last_became_active":"2024-08-31T07:09:47.665723+0000","last_became_peered":"2024-08-31T07:09:47.665723+0000","last_unstale":"2024-08-31T07:09:49.650897+0000","last_undegraded":"2024-08-31T07:09:49.650897+0000","last_fullsized":"2024-08-31T07:09:49.650897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:38:28.122720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.881727+0000","last_change":"2024-08-31T07:09:45.636508+0000","last_active":"2024-08-31T07:10:02.881727+0000","last_peered":"2024-08-31T07:10:02.881727+0000","last_clean":"2024-08-31T07:10:02.881727+0000","last_became_active":"2024-08-31T07:09:45.636073+0000","last_became_peered":"2024-08-31T07:09:45.636073+0000","last_unstale":"2024-08-31T07:10:02.881727+0000","last_undegraded":"2024-08-31T07:10:02.881727+0000","last_fullsized":"2024-08-31T07:10:02.881727+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:55:45.063517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664618+0000","last_change":"2024-08-31T07:09:47.661306+0000","last_active":"2024-08-31T07:09:49.664618+0000","last_peered":"2024-08-31T07:09:49.664618+0000","last_clean":"2024-08-31T07:09:49.664618+0000","last_became_active":"2024-08-31T07:09:47.660957+0000","last_became_peered":"2024-08-31T07:09:47.660957+0000","last_unstale":"2024-08-31T07:09:49.664618+0000","last_undegraded":"2024-08-31T07:09:49.664618+0000","last_fullsized":"2024-08-31T07:09:49.664618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:18:23.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:20.912431+0000","last_change":"2024-08-31T07:09:45.640947+0000","last_active":"2024-08-31T07:10:20.912431+0000","last_peered":"2024-08-31T07:10:20.912431+0000","last_clean":"2024-08-31T07:10:20.912431+0000","last_became_active":"2024-08-31T07:09:45.640532+0000","last_became_peered":"2024-08-31T07:09:45.640532+0000","last_unstale":"2024-08-31T07:10:20.912431+0000","last_undegraded":"2024-08-31T07:10:20.912431+0000","last_fullsized":"2024-08-31T07:10:20.912431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:28.177795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.671873+0000","last_change":"2024-08-31T07:09:47.661060+0000","last_active":"2024-08-31T07:09:49.671873+0000","last_peered":"2024-08-31T07:09:49.671873+0000","last_clean":"2024-08-31T07:09:49.671873+0000","last_became_active":"2024-08-31T07:09:47.660692+0000","last_became_peered":"2024-08-31T07:09:47.660692+0000","last_unstale":"2024-08-31T07:09:49.671873+0000","last_undegraded":"2024-08-31T07:09:49.671873+0000","last_fullsized":"2024-08-31T07:09:49.671873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:48:38.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.882243+0000","last_change":"2024-08-31T07:09:45.641155+0000","last_active":"2024-08-31T07:10:02.882243+0000","last_peered":"2024-08-31T07:10:02.882243+0000","last_clean":"2024-08-31T07:10:02.882243+0000","last_became_active":"2024-08-31T07:09:45.640680+0000","last_became_peered":"2024-08-31T07:09:45.640680+0000","last_unstale":"2024-08-31T07:10:02.882243+0000","last_undegraded":"2024-08-31T07:10:02.882243+0000","last_fullsized":"2024-08-31T07:10:02.882243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:29:03.287570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.653822+0000","last_change":"2024-08-31T07:09:47.675582+0000","last_active":"2024-08-31T07:09:49.653822+0000","last_peered":"2024-08-31T07:09:49.653822+0000","last_clean":"2024-08-31T07:09:49.653822+0000","last_became_active":"2024-08-31T07:09:47.675266+0000","last_became_peered":"2024-08-31T07:09:47.675266+0000","last_unstale":"2024-08-31T07:09:49.653822+0000","last_undegraded":"2024-08-31T07:09:49.653822+0000","last_fullsized":"2024-08-31T07:09:49.653822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:36:33.122531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:01.865666+0000","last_change":"2024-08-31T07:09:45.651006+0000","last_active":"2024-08-31T07:10:01.865666+0000","last_peered":"2024-08-31T07:10:01.865666+0000","last_clean":"2024-08-31T07:10:01.865666+0000","last_became_active":"2024-08-31T07:09:45.650588+0000","last_became_peered":"2024-08-31T07:09:45.650588+0000","last_unstale":"2024-08-31T07:10:01.865666+0000","last_undegraded":"2024-08-31T07:10:01.865666+0000","last_fullsized":"2024-08-31T07:10:01.865666+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:34:16.015892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.671047+0000","last_change":"2024-08-31T07:09:47.662289+0000","last_active":"2024-08-31T07:09:49.671047+0000","last_peered":"2024-08-31T07:09:49.671047+0000","last_clean":"2024-08-31T07:09:49.671047+0000","last_became_active":"2024-08-31T07:09:47.661932+0000","last_became_peered":"2024-08-31T07:09:47.661932+0000","last_unstale":"2024-08-31T07:09:49.671047+0000","last_undegraded":"2024-08-31T07:09:49.671047+0000","last_fullsized":"2024-08-31T07:09:49.671047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:50.440504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'28","reported_seq":92,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:20.913308+0000","last_change":"2024-08-31T07:09:45.642196+0000","last_active":"2024-08-31T07:10:20.913308+0000","last_peered":"2024-08-31T07:10:20.913308+0000","last_clean":"2024-08-31T07:10:20.913308+0000","last_became_active":"2024-08-31T07:09:45.641797+0000","last_became_peered":"2024-08-31T07:09:45.641797+0000","last_unstale":"2024-08-31T07:10:20.913308+0000","last_undegraded":"2024-08-31T07:10:20.913308+0000","last_fullsized":"2024-08-31T07:10:20.913308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:51:05.953335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54372,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":69,"num_write":30,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.668763+0000","last_change":"2024-08-31T07:09:47.660184+0000","last_active":"2024-08-31T07:09:49.668763+0000","last_peered":"2024-08-31T07:09:49.668763+0000","last_clean":"2024-08-31T07:09:49.668763+0000","last_became_active":"2024-08-31T07:09:47.659571+0000","last_became_peered":"2024-08-31T07:09:47.659571+0000","last_unstale":"2024-08-31T07:09:49.668763+0000","last_undegraded":"2024-08-31T07:09:49.668763+0000","last_fullsized":"2024-08-31T07:09:49.668763+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:41:13.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:03.881055+0000","last_change":"2024-08-31T07:09:45.639938+0000","last_active":"2024-08-31T07:10:03.881055+0000","last_peered":"2024-08-31T07:10:03.881055+0000","last_clean":"2024-08-31T07:10:03.881055+0000","last_became_active":"2024-08-31T07:09:45.639569+0000","last_became_peered":"2024-08-31T07:09:45.639569+0000","last_unstale":"2024-08-31T07:10:03.881055+0000","last_undegraded":"2024-08-31T07:10:03.881055+0000","last_fullsized":"2024-08-31T07:10:03.881055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:16:36.589872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.660768+0000","last_change":"2024-08-31T07:09:47.662648+0000","last_active":"2024-08-31T07:09:49.660768+0000","last_peered":"2024-08-31T07:09:49.660768+0000","last_clean":"2024-08-31T07:09:49.660768+0000","last_became_active":"2024-08-31T07:09:47.661167+0000","last_became_peered":"2024-08-31T07:09:47.661167+0000","last_unstale":"2024-08-31T07:09:49.660768+0000","last_undegraded":"2024-08-31T07:09:49.660768+0000","last_fullsized":"2024-08-31T07:09:49.660768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:33:42.127895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.883459+0000","last_change":"2024-08-31T07:09:45.642187+0000","last_active":"2024-08-31T07:10:02.883459+0000","last_peered":"2024-08-31T07:10:02.883459+0000","last_clean":"2024-08-31T07:10:02.883459+0000","last_became_active":"2024-08-31T07:09:45.641721+0000","last_became_peered":"2024-08-31T07:09:45.641721+0000","last_unstale":"2024-08-31T07:10:02.883459+0000","last_undegraded":"2024-08-31T07:10:02.883459+0000","last_fullsized":"2024-08-31T07:10:02.883459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:00:45.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665867+0000","last_change":"2024-08-31T07:09:47.665956+0000","last_active":"2024-08-31T07:09:49.665867+0000","last_peered":"2024-08-31T07:09:49.665867+0000","last_clean":"2024-08-31T07:09:49.665867+0000","last_became_active":"2024-08-31T07:09:47.664367+0000","last_became_peered":"2024-08-31T07:09:47.664367+0000","last_unstale":"2024-08-31T07:09:49.665867+0000","last_undegraded":"2024-08-31T07:09:49.665867+0000","last_fullsized":"2024-08-31T07:09:49.665867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:34.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:19.911195+0000","last_change":"2024-08-31T07:09:45.636440+0000","last_active":"2024-08-31T07:10:19.911195+0000","last_peered":"2024-08-31T07:10:19.911195+0000","last_clean":"2024-08-31T07:10:19.911195+0000","last_became_active":"2024-08-31T07:09:45.636058+0000","last_became_peered":"2024-08-31T07:09:45.636058+0000","last_unstale":"2024-08-31T07:10:19.911195+0000","last_undegraded":"2024-08-31T07:10:19.911195+0000","last_fullsized":"2024-08-31T07:10:19.911195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:38.929227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.666093+0000","last_change":"2024-08-31T07:09:47.660564+0000","last_active":"2024-08-31T07:09:49.666093+0000","last_peered":"2024-08-31T07:09:49.666093+0000","last_clean":"2024-08-31T07:09:49.666093+0000","last_became_active":"2024-08-31T07:09:47.660157+0000","last_became_peered":"2024-08-31T07:09:47.660157+0000","last_unstale":"2024-08-31T07:09:49.666093+0000","last_undegraded":"2024-08-31T07:09:49.666093+0000","last_fullsized":"2024-08-31T07:09:49.666093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:49:01.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":39,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.882782+0000","last_change":"2024-08-31T07:09:45.649140+0000","last_active":"2024-08-31T07:10:02.882782+0000","last_peered":"2024-08-31T07:10:02.882782+0000","last_clean":"2024-08-31T07:10:02.882782+0000","last_became_active":"2024-08-31T07:09:45.648796+0000","last_became_peered":"2024-08-31T07:09:45.648796+0000","last_unstale":"2024-08-31T07:10:02.882782+0000","last_undegraded":"2024-08-31T07:10:02.882782+0000","last_fullsized":"2024-08-31T07:10:02.882782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:33:37.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661343+0000","last_change":"2024-08-31T07:09:47.663833+0000","last_active":"2024-08-31T07:09:49.661343+0000","last_peered":"2024-08-31T07:09:49.661343+0000","last_clean":"2024-08-31T07:09:49.661343+0000","last_became_active":"2024-08-31T07:09:47.663451+0000","last_became_peered":"2024-08-31T07:09:47.663451+0000","last_unstale":"2024-08-31T07:09:49.661343+0000","last_undegraded":"2024-08-31T07:09:49.661343+0000","last_fullsized":"2024-08-31T07:09:49.661343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:27:22.245254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:01.854727+0000","last_change":"2024-08-31T07:09:45.636593+0000","last_active":"2024-08-31T07:10:01.854727+0000","last_peered":"2024-08-31T07:10:01.854727+0000","last_clean":"2024-08-31T07:10:01.854727+0000","last_became_active":"2024-08-31T07:09:45.636166+0000","last_became_peered":"2024-08-31T07:09:45.636166+0000","last_unstale":"2024-08-31T07:10:01.854727+0000","last_undegraded":"2024-08-31T07:10:01.854727+0000","last_fullsized":"2024-08-31T07:10:01.854727+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:28.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661580+0000","last_change":"2024-08-31T07:09:47.660825+0000","last_active":"2024-08-31T07:09:49.661580+0000","last_peered":"2024-08-31T07:09:49.661580+0000","last_clean":"2024-08-31T07:09:49.661580+0000","last_became_active":"2024-08-31T07:09:47.660474+0000","last_became_peered":"2024-08-31T07:09:47.660474+0000","last_unstale":"2024-08-31T07:09:49.661580+0000","last_undegraded":"2024-08-31T07:09:49.661580+0000","last_fullsized":"2024-08-31T07:09:49.661580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:42:20.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.881356+0000","last_change":"2024-08-31T07:09:45.632727+0000","last_active":"2024-08-31T07:10:02.881356+0000","last_peered":"2024-08-31T07:10:02.881356+0000","last_clean":"2024-08-31T07:10:02.881356+0000","last_became_active":"2024-08-31T07:09:45.632251+0000","last_became_peered":"2024-08-31T07:09:45.632251+0000","last_unstale":"2024-08-31T07:10:02.881356+0000","last_undegraded":"2024-08-31T07:10:02.881356+0000","last_fullsized":"2024-08-31T07:10:02.881356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:09:30.197409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665265+0000","last_change":"2024-08-31T07:09:47.665501+0000","last_active":"2024-08-31T07:09:49.665265+0000","last_peered":"2024-08-31T07:09:49.665265+0000","last_clean":"2024-08-31T07:09:49.665265+0000","last_became_active":"2024-08-31T07:09:47.665162+0000","last_became_peered":"2024-08-31T07:09:47.665162+0000","last_unstale":"2024-08-31T07:09:49.665265+0000","last_undegraded":"2024-08-31T07:09:49.665265+0000","last_fullsized":"2024-08-31T07:09:49.665265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:34:29.605836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.883385+0000","last_change":"2024-08-31T07:09:45.636580+0000","last_active":"2024-08-31T07:10:02.883385+0000","last_peered":"2024-08-31T07:10:02.883385+0000","last_clean":"2024-08-31T07:10:02.883385+0000","last_became_active":"2024-08-31T07:09:45.636163+0000","last_became_peered":"2024-08-31T07:09:45.636163+0000","last_unstale":"2024-08-31T07:10:02.883385+0000","last_undegraded":"2024-08-31T07:10:02.883385+0000","last_fullsized":"2024-08-31T07:10:02.883385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:18:07.567858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.666257+0000","last_change":"2024-08-31T07:09:47.671865+0000","last_active":"2024-08-31T07:09:49.666257+0000","last_peered":"2024-08-31T07:09:49.666257+0000","last_clean":"2024-08-31T07:09:49.666257+0000","last_became_active":"2024-08-31T07:09:47.671454+0000","last_became_peered":"2024-08-31T07:09:47.671454+0000","last_unstale":"2024-08-31T07:09:49.666257+0000","last_undegraded":"2024-08-31T07:09:49.666257+0000","last_fullsized":"2024-08-31T07:09:49.666257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:47.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'6","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:22.916156+0000","last_change":"2024-08-31T07:09:45.653217+0000","last_active":"2024-08-31T07:10:22.916156+0000","last_peered":"2024-08-31T07:10:22.916156+0000","last_clean":"2024-08-31T07:10:22.916156+0000","last_became_active":"2024-08-31T07:09:45.652819+0000","last_became_peered":"2024-08-31T07:09:45.652819+0000","last_unstale":"2024-08-31T07:10:22.916156+0000","last_undegraded":"2024-08-31T07:10:22.916156+0000","last_fullsized":"2024-08-31T07:10:22.916156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:25:11.454068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.666257+0000","last_change":"2024-08-31T07:09:47.663652+0000","last_active":"2024-08-31T07:09:49.666257+0000","last_peered":"2024-08-31T07:09:49.666257+0000","last_clean":"2024-08-31T07:09:49.666257+0000","last_became_active":"2024-08-31T07:09:47.663274+0000","last_became_peered":"2024-08-31T07:09:47.663274+0000","last_unstale":"2024-08-31T07:09:49.666257+0000","last_undegraded":"2024-08-31T07:09:49.666257+0000","last_fullsized":"2024-08-31T07:09:49.666257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:31:59.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665834+0000","last_change":"2024-08-31T07:08:27.623298+0000","last_active":"2024-08-31T07:09:49.665834+0000","last_peered":"2024-08-31T07:09:49.665834+0000","last_clean":"2024-08-31T07:09:49.665834+0000","last_became_active":"2024-08-31T07:08:27.619072+0000","last_became_peered":"2024-08-31T07:08:27.619072+0000","last_unstale":"2024-08-31T07:09:49.665834+0000","last_undegraded":"2024-08-31T07:09:49.665834+0000","last_fullsized":"2024-08-31T07:09:49.665834+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_clean_scrub_stamp":"2024-08-31T07:04:45.509398+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:07.967421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:03.881045+0000","last_change":"2024-08-31T07:09:45.638104+0000","last_active":"2024-08-31T07:10:03.881045+0000","last_peered":"2024-08-31T07:10:03.881045+0000","last_clean":"2024-08-31T07:10:03.881045+0000","last_became_active":"2024-08-31T07:09:45.637693+0000","last_became_peered":"2024-08-31T07:09:45.637693+0000","last_unstale":"2024-08-31T07:10:03.881045+0000","last_undegraded":"2024-08-31T07:10:03.881045+0000","last_fullsized":"2024-08-31T07:10:03.881045+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:50:15.135061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.650982+0000","last_change":"2024-08-31T07:09:47.664958+0000","last_active":"2024-08-31T07:09:49.650982+0000","last_peered":"2024-08-31T07:09:49.650982+0000","last_clean":"2024-08-31T07:09:49.650982+0000","last_became_active":"2024-08-31T07:09:47.661166+0000","last_became_peered":"2024-08-31T07:09:47.661166+0000","last_unstale":"2024-08-31T07:09:49.650982+0000","last_undegraded":"2024-08-31T07:09:49.650982+0000","last_fullsized":"2024-08-31T07:09:49.650982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:45:58.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:00.859595+0000","last_change":"2024-08-31T07:09:45.640881+0000","last_active":"2024-08-31T07:10:00.859595+0000","last_peered":"2024-08-31T07:10:00.859595+0000","last_clean":"2024-08-31T07:10:00.859595+0000","last_became_active":"2024-08-31T07:09:45.640487+0000","last_became_peered":"2024-08-31T07:09:45.640487+0000","last_unstale":"2024-08-31T07:10:00.859595+0000","last_undegraded":"2024-08-31T07:10:00.859595+0000","last_fullsized":"2024-08-31T07:10:00.859595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:06.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.660262+0000","last_change":"2024-08-31T07:09:47.655008+0000","last_active":"2024-08-31T07:09:49.660262+0000","last_peered":"2024-08-31T07:09:49.660262+0000","last_clean":"2024-08-31T07:09:49.660262+0000","last_became_active":"2024-08-31T07:09:47.654645+0000","last_became_peered":"2024-08-31T07:09:47.654645+0000","last_unstale":"2024-08-31T07:09:49.660262+0000","last_undegraded":"2024-08-31T07:09:49.660262+0000","last_fullsized":"2024-08-31T07:09:49.660262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:23:55.916053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":47,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:21.913369+0000","last_change":"2024-08-31T07:09:45.649324+0000","last_active":"2024-08-31T07:10:21.913369+0000","last_peered":"2024-08-31T07:10:21.913369+0000","last_clean":"2024-08-31T07:10:21.913369+0000","last_became_active":"2024-08-31T07:09:45.648915+0000","last_became_peered":"2024-08-31T07:09:45.648915+0000","last_unstale":"2024-08-31T07:10:21.913369+0000","last_undegraded":"2024-08-31T07:10:21.913369+0000","last_fullsized":"2024-08-31T07:10:21.913369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:31:57.853654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.662189+0000","last_change":"2024-08-31T07:09:47.661328+0000","last_active":"2024-08-31T07:09:49.662189+0000","last_peered":"2024-08-31T07:09:49.662189+0000","last_clean":"2024-08-31T07:09:49.662189+0000","last_became_active":"2024-08-31T07:09:47.660927+0000","last_became_peered":"2024-08-31T07:09:47.660927+0000","last_unstale":"2024-08-31T07:09:49.662189+0000","last_undegraded":"2024-08-31T07:09:49.662189+0000","last_fullsized":"2024-08-31T07:09:49.662189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:40:46.876894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'6","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:21.914392+0000","last_change":"2024-08-31T07:09:45.637747+0000","last_active":"2024-08-31T07:10:21.914392+0000","last_peered":"2024-08-31T07:10:21.914392+0000","last_clean":"2024-08-31T07:10:21.914392+0000","last_became_active":"2024-08-31T07:09:45.637368+0000","last_became_peered":"2024-08-31T07:09:45.637368+0000","last_unstale":"2024-08-31T07:10:21.914392+0000","last_undegraded":"2024-08-31T07:10:21.914392+0000","last_fullsized":"2024-08-31T07:10:21.914392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:01.174109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.666099+0000","last_change":"2024-08-31T07:09:47.680802+0000","last_active":"2024-08-31T07:09:49.666099+0000","last_peered":"2024-08-31T07:09:49.666099+0000","last_clean":"2024-08-31T07:09:49.666099+0000","last_became_active":"2024-08-31T07:09:47.680363+0000","last_became_peered":"2024-08-31T07:09:47.680363+0000","last_unstale":"2024-08-31T07:09:49.666099+0000","last_undegraded":"2024-08-31T07:09:49.666099+0000","last_fullsized":"2024-08-31T07:09:49.666099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:46:48.744071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.665680+0000","last_change":"2024-08-31T07:09:45.651222+0000","last_active":"2024-08-31T07:09:49.665680+0000","last_peered":"2024-08-31T07:09:49.665680+0000","last_clean":"2024-08-31T07:09:49.665680+0000","last_became_active":"2024-08-31T07:09:45.650760+0000","last_became_peered":"2024-08-31T07:09:45.650760+0000","last_unstale":"2024-08-31T07:09:49.665680+0000","last_undegraded":"2024-08-31T07:09:49.665680+0000","last_fullsized":"2024-08-31T07:09:49.665680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T19:03:33.942165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.667511+0000","last_change":"2024-08-31T07:09:47.665881+0000","last_active":"2024-08-31T07:09:49.667511+0000","last_peered":"2024-08-31T07:09:49.667511+0000","last_clean":"2024-08-31T07:09:49.667511+0000","last_became_active":"2024-08-31T07:09:47.665485+0000","last_became_peered":"2024-08-31T07:09:47.665485+0000","last_unstale":"2024-08-31T07:09:49.667511+0000","last_undegraded":"2024-08-31T07:09:49.667511+0000","last_fullsized":"2024-08-31T07:09:49.667511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:30.820714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.882688+0000","last_change":"2024-08-31T07:09:45.651893+0000","last_active":"2024-08-31T07:10:02.882688+0000","last_peered":"2024-08-31T07:10:02.882688+0000","last_clean":"2024-08-31T07:10:02.882688+0000","last_became_active":"2024-08-31T07:09:45.651547+0000","last_became_peered":"2024-08-31T07:09:45.651547+0000","last_unstale":"2024-08-31T07:10:02.882688+0000","last_undegraded":"2024-08-31T07:10:02.882688+0000","last_fullsized":"2024-08-31T07:10:02.882688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:11:30.708018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661581+0000","last_change":"2024-08-31T07:09:47.655551+0000","last_active":"2024-08-31T07:09:49.661581+0000","last_peered":"2024-08-31T07:09:49.661581+0000","last_clean":"2024-08-31T07:09:49.661581+0000","last_became_active":"2024-08-31T07:09:47.655188+0000","last_became_peered":"2024-08-31T07:09:47.655188+0000","last_unstale":"2024-08-31T07:09:49.661581+0000","last_undegraded":"2024-08-31T07:09:49.661581+0000","last_fullsized":"2024-08-31T07:09:49.661581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:43:51.419599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'13","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:16.659560+0000","last_change":"2024-08-31T07:09:45.636212+0000","last_active":"2024-08-31T07:10:16.659560+0000","last_peered":"2024-08-31T07:10:16.659560+0000","last_clean":"2024-08-31T07:10:16.659560+0000","last_became_active":"2024-08-31T07:09:45.635678+0000","last_became_peered":"2024-08-31T07:09:45.635678+0000","last_unstale":"2024-08-31T07:10:16.659560+0000","last_undegraded":"2024-08-31T07:10:16.659560+0000","last_fullsized":"2024-08-31T07:10:16.659560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:13:17.803459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.651577+0000","last_change":"2024-08-31T07:09:47.666476+0000","last_active":"2024-08-31T07:09:49.651577+0000","last_peered":"2024-08-31T07:09:49.651577+0000","last_clean":"2024-08-31T07:09:49.651577+0000","last_became_active":"2024-08-31T07:09:47.666111+0000","last_became_peered":"2024-08-31T07:09:47.666111+0000","last_unstale":"2024-08-31T07:09:49.651577+0000","last_undegraded":"2024-08-31T07:09:49.651577+0000","last_fullsized":"2024-08-31T07:09:49.651577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:08:53.459549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.881628+0000","last_change":"2024-08-31T07:09:45.631571+0000","last_active":"2024-08-31T07:10:02.881628+0000","last_peered":"2024-08-31T07:10:02.881628+0000","last_clean":"2024-08-31T07:10:02.881628+0000","last_became_active":"2024-08-31T07:09:45.630981+0000","last_became_peered":"2024-08-31T07:09:45.630981+0000","last_unstale":"2024-08-31T07:10:02.881628+0000","last_undegraded":"2024-08-31T07:10:02.881628+0000","last_fullsized":"2024-08-31T07:10:02.881628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:36:53.482955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.653216+0000","last_change":"2024-08-31T07:09:47.675054+0000","last_active":"2024-08-31T07:09:49.653216+0000","last_peered":"2024-08-31T07:09:49.653216+0000","last_clean":"2024-08-31T07:09:49.653216+0000","last_became_active":"2024-08-31T07:09:47.672652+0000","last_became_peered":"2024-08-31T07:09:47.672652+0000","last_unstale":"2024-08-31T07:09:49.653216+0000","last_undegraded":"2024-08-31T07:09:49.653216+0000","last_fullsized":"2024-08-31T07:09:49.653216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:32:05.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:59.855841+0000","last_change":"2024-08-31T07:09:45.648746+0000","last_active":"2024-08-31T07:09:59.855841+0000","last_peered":"2024-08-31T07:09:59.855841+0000","last_clean":"2024-08-31T07:09:59.855841+0000","last_became_active":"2024-08-31T07:09:45.648424+0000","last_became_peered":"2024-08-31T07:09:45.648424+0000","last_unstale":"2024-08-31T07:09:59.855841+0000","last_undegraded":"2024-08-31T07:09:59.855841+0000","last_fullsized":"2024-08-31T07:09:59.855841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:40:32.132963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.667942+0000","last_change":"2024-08-31T07:09:47.663213+0000","last_active":"2024-08-31T07:09:49.667942+0000","last_peered":"2024-08-31T07:09:49.667942+0000","last_clean":"2024-08-31T07:09:49.667942+0000","last_became_active":"2024-08-31T07:09:47.662457+0000","last_became_peered":"2024-08-31T07:09:47.662457+0000","last_unstale":"2024-08-31T07:09:49.667942+0000","last_undegraded":"2024-08-31T07:09:49.667942+0000","last_fullsized":"2024-08-31T07:09:49.667942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:54:27.069200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.882950+0000","last_change":"2024-08-31T07:09:45.640149+0000","last_active":"2024-08-31T07:10:02.882950+0000","last_peered":"2024-08-31T07:10:02.882950+0000","last_clean":"2024-08-31T07:10:02.882950+0000","last_became_active":"2024-08-31T07:09:45.639581+0000","last_became_peered":"2024-08-31T07:09:45.639581+0000","last_unstale":"2024-08-31T07:10:02.882950+0000","last_undegraded":"2024-08-31T07:10:02.882950+0000","last_fullsized":"2024-08-31T07:10:02.882950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:27:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.661706+0000","last_change":"2024-08-31T07:09:47.672613+0000","last_active":"2024-08-31T07:09:49.661706+0000","last_peered":"2024-08-31T07:09:49.661706+0000","last_clean":"2024-08-31T07:09:49.661706+0000","last_became_active":"2024-08-31T07:09:47.672173+0000","last_became_peered":"2024-08-31T07:09:47.672173+0000","last_unstale":"2024-08-31T07:09:49.661706+0000","last_undegraded":"2024-08-31T07:09:49.661706+0000","last_fullsized":"2024-08-31T07:09:49.661706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:49.706113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:01.868486+0000","last_change":"2024-08-31T07:09:45.637567+0000","last_active":"2024-08-31T07:10:01.868486+0000","last_peered":"2024-08-31T07:10:01.868486+0000","last_clean":"2024-08-31T07:10:01.868486+0000","last_became_active":"2024-08-31T07:09:45.637180+0000","last_became_peered":"2024-08-31T07:09:45.637180+0000","last_unstale":"2024-08-31T07:10:01.868486+0000","last_undegraded":"2024-08-31T07:10:01.868486+0000","last_fullsized":"2024-08-31T07:10:01.868486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:48:07.190829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.651923+0000","last_change":"2024-08-31T07:09:47.674980+0000","last_active":"2024-08-31T07:09:49.651923+0000","last_peered":"2024-08-31T07:09:49.651923+0000","last_clean":"2024-08-31T07:09:49.651923+0000","last_became_active":"2024-08-31T07:09:47.674528+0000","last_became_peered":"2024-08-31T07:09:47.674528+0000","last_unstale":"2024-08-31T07:09:49.651923+0000","last_undegraded":"2024-08-31T07:09:49.651923+0000","last_fullsized":"2024-08-31T07:09:49.651923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:15:09.464221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:59.854478+0000","last_change":"2024-08-31T07:09:45.641167+0000","last_active":"2024-08-31T07:09:59.854478+0000","last_peered":"2024-08-31T07:09:59.854478+0000","last_clean":"2024-08-31T07:09:59.854478+0000","last_became_active":"2024-08-31T07:09:45.640761+0000","last_became_peered":"2024-08-31T07:09:45.640761+0000","last_unstale":"2024-08-31T07:09:59.854478+0000","last_undegraded":"2024-08-31T07:09:59.854478+0000","last_fullsized":"2024-08-31T07:09:59.854478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:28:59.963145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.666431+0000","last_change":"2024-08-31T07:09:47.668808+0000","last_active":"2024-08-31T07:09:49.666431+0000","last_peered":"2024-08-31T07:09:49.666431+0000","last_clean":"2024-08-31T07:09:49.666431+0000","last_became_active":"2024-08-31T07:09:47.668291+0000","last_became_peered":"2024-08-31T07:09:47.668291+0000","last_unstale":"2024-08-31T07:09:49.666431+0000","last_undegraded":"2024-08-31T07:09:49.666431+0000","last_fullsized":"2024-08-31T07:09:49.666431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:23.433511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.662765+0000","last_change":"2024-08-31T07:09:47.668039+0000","last_active":"2024-08-31T07:09:49.662765+0000","last_peered":"2024-08-31T07:09:49.662765+0000","last_clean":"2024-08-31T07:09:49.662765+0000","last_became_active":"2024-08-31T07:09:47.667567+0000","last_became_peered":"2024-08-31T07:09:47.667567+0000","last_unstale":"2024-08-31T07:09:49.662765+0000","last_undegraded":"2024-08-31T07:09:49.662765+0000","last_fullsized":"2024-08-31T07:09:49.662765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:33:50.192864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:02.882828+0000","last_change":"2024-08-31T07:09:45.636229+0000","last_active":"2024-08-31T07:10:02.882828+0000","last_peered":"2024-08-31T07:10:02.882828+0000","last_clean":"2024-08-31T07:10:02.882828+0000","last_became_active":"2024-08-31T07:09:45.635767+0000","last_became_peered":"2024-08-31T07:09:45.635767+0000","last_unstale":"2024-08-31T07:10:02.882828+0000","last_undegraded":"2024-08-31T07:10:02.882828+0000","last_fullsized":"2024-08-31T07:10:02.882828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:55:11.092133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.670756+0000","last_change":"2024-08-31T07:09:47.660615+0000","last_active":"2024-08-31T07:09:49.670756+0000","last_peered":"2024-08-31T07:09:49.670756+0000","last_clean":"2024-08-31T07:09:49.670756+0000","last_became_active":"2024-08-31T07:09:47.660269+0000","last_became_peered":"2024-08-31T07:09:47.660269+0000","last_unstale":"2024-08-31T07:09:49.670756+0000","last_undegraded":"2024-08-31T07:09:49.670756+0000","last_fullsized":"2024-08-31T07:09:49.670756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:42:12.618039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:03.875405+0000","last_change":"2024-08-31T07:09:45.641699+0000","last_active":"2024-08-31T07:10:03.875405+0000","last_peered":"2024-08-31T07:10:03.875405+0000","last_clean":"2024-08-31T07:10:03.875405+0000","last_became_active":"2024-08-31T07:09:45.641122+0000","last_became_peered":"2024-08-31T07:09:45.641122+0000","last_unstale":"2024-08-31T07:10:03.875405+0000","last_undegraded":"2024-08-31T07:10:03.875405+0000","last_fullsized":"2024-08-31T07:10:03.875405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:26:07.130266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.653822+0000","last_change":"2024-08-31T07:09:47.675459+0000","last_active":"2024-08-31T07:09:49.653822+0000","last_peered":"2024-08-31T07:09:49.653822+0000","last_clean":"2024-08-31T07:09:49.653822+0000","last_became_active":"2024-08-31T07:09:47.672246+0000","last_became_peered":"2024-08-31T07:09:47.672246+0000","last_unstale":"2024-08-31T07:09:49.653822+0000","last_undegraded":"2024-08-31T07:09:49.653822+0000","last_fullsized":"2024-08-31T07:09:49.653822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:49:23.547352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.664498+0000","last_change":"2024-08-31T07:09:45.649522+0000","last_active":"2024-08-31T07:09:49.664498+0000","last_peered":"2024-08-31T07:09:49.664498+0000","last_clean":"2024-08-31T07:09:49.664498+0000","last_became_active":"2024-08-31T07:09:45.649109+0000","last_became_peered":"2024-08-31T07:09:45.649109+0000","last_unstale":"2024-08-31T07:09:49.664498+0000","last_undegraded":"2024-08-31T07:09:49.664498+0000","last_fullsized":"2024-08-31T07:09:49.664498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:52:52.432605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:09:49.651317+0000","last_change":"2024-08-31T07:09:47.670583+0000","last_active":"2024-08-31T07:09:49.651317+0000","last_peered":"2024-08-31T07:09:49.651317+0000","last_clean":"2024-08-31T07:09:49.651317+0000","last_became_active":"2024-08-31T07:09:47.670162+0000","last_became_peered":"2024-08-31T07:09:47.670162+0000","last_unstale":"2024-08-31T07:09:49.651317+0000","last_undegraded":"2024-08-31T07:09:49.651317+0000","last_fullsized":"2024-08-31T07:09:49.651317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:35:35.135678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":52,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T07:10:18.909212+0000","last_change":"2024-08-31T07:09:45.639636+0000","last_active":"2024-08-31T07:10:18.909212+0000","last_peered":"2024-08-31T07:10:18.909212+0000","last_clean":"2024-08-31T07:10:18.909212+0000","last_became_active":"2024-08-31T07:09:45.639029+0000","last_became_peered":"2024-08-31T07:09:45.639029+0000","last_unstale":"2024-08-31T07:10:18.909212+0000","last_undegraded":"2024-08-31T07:10:18.909212+0000","last_fullsized":"2024-08-31T07:10:18.909212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:54.110149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62358,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":227,"num_read_kb":176,"num_write":240,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":182817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285320,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678036,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":416266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874265,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":1061667,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037790,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":415805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234021,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":416183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":1009483,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561070,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":415913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":1008379,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920826,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":415774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117056,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":415347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378694,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":471809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542219,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":466941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":417252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":52258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T07:10:25.672 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-31T07:10:25.672 INFO:tasks.cephadm:Setting up client nodes... 2024-08-31T07:10:25.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-31T07:10:26.090 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:26.229 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:25 smithi032 ceph-mon[17252]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-08-31T07:10:26.229 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:25 smithi032 ceph-mon[17252]: from='client.24856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:10:26.231 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:25 smithi073 ceph-mon[21995]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-08-31T07:10:26.231 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:25 smithi073 ceph-mon[21995]: from='client.24856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:10:26.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:25 smithi084 ceph-mon[22387]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-08-31T07:10:26.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:25 smithi084 ceph-mon[22387]: from='client.24856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:10:26.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.743+0000 7fedc1a00640 1 -- 172.21.15.32:0/2708484129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc109ba0 msgr2=0x7fedbc1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.743+0000 7fedc1a00640 1 --2- 172.21.15.32:0/2708484129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc109ba0 0x7fedbc1103d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009ca0 tx=0x7fedb402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:10:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.743+0000 7fedc1a00640 1 -- 172.21.15.32:0/2708484129 shutdown_connections 2024-08-31T07:10:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.743+0000 7fedc1a00640 1 --2- 172.21.15.32:0/2708484129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc109ba0 0x7fedbc1103d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.743+0000 7fedc1a00640 1 --2- 172.21.15.32:0/2708484129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedbc109250 0x7fedbc109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.743+0000 7fedc1a00640 1 --2- 172.21.15.32:0/2708484129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc103910 0x7fedbc103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.743+0000 7fedc1a00640 1 -- 172.21.15.32:0/2708484129 >> 172.21.15.32:0/2708484129 conn(0x7fedbc0ff340 msgr2=0x7fedbc101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.743+0000 7fedc1a00640 1 -- 172.21.15.32:0/2708484129 shutdown_connections 2024-08-31T07:10:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.743+0000 7fedc1a00640 1 -- 172.21.15.32:0/2708484129 wait complete. 2024-08-31T07:10:26.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.744+0000 7fedc1a00640 1 Processor -- start 2024-08-31T07:10:26.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.744+0000 7fedc1a00640 1 -- start start 2024-08-31T07:10:26.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedc1a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedbc103910 0x7fedbc19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:26.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedc1a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109250 0x7fedbc19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:26.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedc1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc109ba0 0x7fedbc19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:26.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedc1a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc113860 con 0x7fedbc109ba0 2024-08-31T07:10:26.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedc1a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc1136e0 con 0x7fedbc103910 2024-08-31T07:10:26.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedc1a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc1139e0 con 0x7fedbc109250 2024-08-31T07:10:26.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedbb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc109ba0 0x7fedbc19fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:26.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedbb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc109ba0 0x7fedbc19fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57054/0 (socket says 172.21.15.32:57054) 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedbb400640 1 -- 172.21.15.32:0/3573287561 learned_addr learned my addr 172.21.15.32:0/3573287561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedb3e00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109250 0x7fedbc19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.745+0000 7fedbaa00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedbc103910 0x7fedbc19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedb3e00640 1 -- 172.21.15.32:0/3573287561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedbc103910 msgr2=0x7fedbc19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedb3e00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedbc103910 0x7fedbc19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedb3e00640 1 -- 172.21.15.32:0/3573287561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc109ba0 msgr2=0x7fedbc19fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedb3e00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc109ba0 0x7fedbc19fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedb3e00640 1 -- 172.21.15.32:0/3573287561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc1a0180 con 0x7fedbc109250 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedbb400640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc109ba0 0x7fedbc19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedb3e00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109250 0x7fedbc19a7b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fedbc06c710 tx=0x7feda400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedb3400640 1 -- 172.21.15.32:0/3573287561 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda400cc40 con 0x7fedbc109250 2024-08-31T07:10:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedc1a00640 1 -- 172.21.15.32:0/3573287561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedbc1a0470 con 0x7fedbc109250 2024-08-31T07:10:26.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedb3400640 1 -- 172.21.15.32:0/3573287561 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feda400cde0 con 0x7fedbc109250 2024-08-31T07:10:26.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedc1a00640 1 -- 172.21.15.32:0/3573287561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedbc1a6bd0 con 0x7fedbc109250 2024-08-31T07:10:26.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.746+0000 7fedb3400640 1 -- 172.21.15.32:0/3573287561 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4018760 con 0x7fedbc109250 2024-08-31T07:10:26.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.747+0000 7fedc1a00640 1 -- 172.21.15.32:0/3573287561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed980052c0 con 0x7fedbc109250 2024-08-31T07:10:26.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.751+0000 7fedb3400640 1 -- 172.21.15.32:0/3573287561 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7feda4018900 con 0x7fedbc109250 2024-08-31T07:10:26.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.752+0000 7fedb3400640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feda00b4c90 0x7feda00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:26.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.752+0000 7fedb3400640 1 -- 172.21.15.32:0/3573287561 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7feda40e57d0 con 0x7fedbc109250 2024-08-31T07:10:26.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.752+0000 7fedbaa00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feda00b4c90 0x7feda00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:26.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.753+0000 7fedbaa00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feda00b4c90 0x7feda00b71a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fedac005d00 tx=0x7fedac00b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:26.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:26.753+0000 7fedb3400640 1 -- 172.21.15.32:0/3573287561 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7feda40ac110 con 0x7fedbc109250 2024-08-31T07:10:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.031+0000 7fedc1a00640 1 -- 172.21.15.32:0/3573287561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fed980055b0 con 0x7fedbc109250 2024-08-31T07:10:27.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.040+0000 7fedb3400640 1 -- 172.21.15.32:0/3573287561 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7feda40b08d0 con 0x7fedbc109250 2024-08-31T07:10:27.041 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-08-31T07:10:27.041 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDjwdJmKecCAhAAUu1PXAhWnq24Cz4R67+3bw== 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 -- 172.21.15.32:0/3573287561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feda00b4c90 msgr2=0x7feda00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feda00b4c90 0x7feda00b71a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fedac005d00 tx=0x7fedac00b410 comp rx=0 tx=0).stop 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 -- 172.21.15.32:0/3573287561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109250 msgr2=0x7fedbc19a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109250 0x7fedbc19a7b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fedbc06c710 tx=0x7feda400ee50 comp rx=0 tx=0).stop 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 -- 172.21.15.32:0/3573287561 shutdown_connections 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feda00b4c90 0x7feda00b71a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc109ba0 0x7fedbc19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109250 0x7fedbc19a7b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 --2- 172.21.15.32:0/3573287561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedbc103910 0x7fedbc19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 -- 172.21.15.32:0/3573287561 >> 172.21.15.32:0/3573287561 conn(0x7fedbc0ff340 msgr2=0x7fedbc108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:27.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 -- 172.21.15.32:0/3573287561 shutdown_connections 2024-08-31T07:10:27.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:27.043+0000 7fedc1a00640 1 -- 172.21.15.32:0/3573287561 wait complete. 2024-08-31T07:10:27.802 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:10:27.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-31T07:10:27.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-31T07:10:27.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph config log 1 --format=json 2024-08-31T07:10:28.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:27 smithi032 ceph-mon[17252]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-08-31T07:10:28.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:27 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3573287561' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-31T07:10:28.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:27 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-31T07:10:28.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:27 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-31T07:10:28.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:27 smithi084 ceph-mon[22387]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-08-31T07:10:28.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:27 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3573287561' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-31T07:10:28.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:27 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-31T07:10:28.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:27 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-31T07:10:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:27 smithi073 ceph-mon[21995]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-08-31T07:10:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:27 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3573287561' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-31T07:10:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:27 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-31T07:10:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:27 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-31T07:10:29.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.229+0000 7fc833800640 1 -- 172.21.15.32:0/2661667168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c1144e0 msgr2=0x7fc82c116900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:29.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.229+0000 7fc833800640 1 --2- 172.21.15.32:0/2661667168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c1144e0 0x7fc82c116900 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc824009ca0 tx=0x7fc82402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:10:29.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.229+0000 7fc833800640 1 -- 172.21.15.32:0/2661667168 shutdown_connections 2024-08-31T07:10:29.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.229+0000 7fc833800640 1 --2- 172.21.15.32:0/2661667168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c1144e0 0x7fc82c116900 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:29.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.229+0000 7fc833800640 1 --2- 172.21.15.32:0/2661667168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc82c078030 0x7fc82c113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:29.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.229+0000 7fc833800640 1 --2- 172.21.15.32:0/2661667168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82c0776e0 0x7fc82c077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:29.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.229+0000 7fc833800640 1 -- 172.21.15.32:0/2661667168 >> 172.21.15.32:0/2661667168 conn(0x7fc82c101590 msgr2=0x7fc82c1039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:29.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.229+0000 7fc833800640 1 -- 172.21.15.32:0/2661667168 shutdown_connections 2024-08-31T07:10:29.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.229+0000 7fc833800640 1 -- 172.21.15.32:0/2661667168 wait complete. 2024-08-31T07:10:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.231+0000 7fc833800640 1 Processor -- start 2024-08-31T07:10:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.231+0000 7fc833800640 1 -- start start 2024-08-31T07:10:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.231+0000 7fc833800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c0776e0 0x7fc82c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.231+0000 7fc833800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc82c078030 0x7fc82c1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.231+0000 7fc833800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82c1144e0 0x7fc82c1a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.231+0000 7fc833800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c119e90 con 0x7fc82c0776e0 2024-08-31T07:10:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.231+0000 7fc833800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c119d10 con 0x7fc82c078030 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.231+0000 7fc833800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c11a010 con 0x7fc82c1144e0 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.232+0000 7fc830e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c0776e0 0x7fc82c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.232+0000 7fc830e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c0776e0 0x7fc82c1a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57068/0 (socket says 172.21.15.32:57068) 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.232+0000 7fc830e00640 1 -- 172.21.15.32:0/1018137302 learned_addr learned my addr 172.21.15.32:0/1018137302 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.232+0000 7fc831800640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82c1144e0 0x7fc82c1a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.232+0000 7fc82be00640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc82c078030 0x7fc82c1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.232+0000 7fc830e00640 1 -- 172.21.15.32:0/1018137302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82c1144e0 msgr2=0x7fc82c1a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.232+0000 7fc830e00640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82c1144e0 0x7fc82c1a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.232+0000 7fc830e00640 1 -- 172.21.15.32:0/1018137302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc82c078030 msgr2=0x7fc82c1a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.232+0000 7fc830e00640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc82c078030 0x7fc82c1a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.232+0000 7fc830e00640 1 -- 172.21.15.32:0/1018137302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c1a6750 con 0x7fc82c0776e0 2024-08-31T07:10:29.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.233+0000 7fc830e00640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c0776e0 0x7fc82c1a07e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc82000d950 tx=0x7fc82000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:29.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.233+0000 7fc829600640 1 -- 172.21.15.32:0/1018137302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820017030 con 0x7fc82c0776e0 2024-08-31T07:10:29.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.233+0000 7fc829600640 1 -- 172.21.15.32:0/1018137302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc820004020 con 0x7fc82c0776e0 2024-08-31T07:10:29.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.233+0000 7fc833800640 1 -- 172.21.15.32:0/1018137302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82c1a6a40 con 0x7fc82c0776e0 2024-08-31T07:10:29.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.233+0000 7fc829600640 1 -- 172.21.15.32:0/1018137302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200047a0 con 0x7fc82c0776e0 2024-08-31T07:10:29.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.233+0000 7fc833800640 1 -- 172.21.15.32:0/1018137302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82c1ad1a0 con 0x7fc82c0776e0 2024-08-31T07:10:29.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.234+0000 7fc833800640 1 -- 172.21.15.32:0/1018137302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82c077af0 con 0x7fc82c0776e0 2024-08-31T07:10:29.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.237+0000 7fc829600640 1 -- 172.21.15.32:0/1018137302 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fc820009a30 con 0x7fc82c0776e0 2024-08-31T07:10:29.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.238+0000 7fc829600640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8140b4c90 0x7fc8140b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:29.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.238+0000 7fc829600640 1 -- 172.21.15.32:0/1018137302 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fc8200e5360 con 0x7fc82c0776e0 2024-08-31T07:10:29.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.238+0000 7fc82be00640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8140b4c90 0x7fc8140b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:29.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.239+0000 7fc82be00640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8140b4c90 0x7fc8140b71a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc82c07a900 tx=0x7fc818005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:29.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.240+0000 7fc829600640 1 -- 172.21.15.32:0/1018137302 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc8200ac370 con 0x7fc82c0776e0 2024-08-31T07:10:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.436+0000 7fc833800640 1 -- 172.21.15.32:0/1018137302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fc82c1a1910 con 0x7fc82c0776e0 2024-08-31T07:10:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.437+0000 7fc829600640 1 -- 172.21.15.32:0/1018137302 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v53) ==== 86+0+133 (secure 0 0 0) 0x7fc8200abc70 con 0x7fc82c0776e0 2024-08-31T07:10:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 -- 172.21.15.32:0/1018137302 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8140b4c90 msgr2=0x7fc8140b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8140b4c90 0x7fc8140b71a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc82c07a900 tx=0x7fc818005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 -- 172.21.15.32:0/1018137302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c0776e0 msgr2=0x7fc82c1a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c0776e0 0x7fc82c1a07e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc82000d950 tx=0x7fc82000de10 comp rx=0 tx=0).stop 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 -- 172.21.15.32:0/1018137302 shutdown_connections 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8140b4c90 0x7fc8140b71a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc82c1144e0 0x7fc82c1a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc82c078030 0x7fc82c1a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 --2- 172.21.15.32:0/1018137302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc82c0776e0 0x7fc82c1a07e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 -- 172.21.15.32:0/1018137302 >> 172.21.15.32:0/1018137302 conn(0x7fc82c101590 msgr2=0x7fc82c116530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 -- 172.21.15.32:0/1018137302 shutdown_connections 2024-08-31T07:10:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:29.440+0000 7fc833800640 1 -- 172.21.15.32:0/1018137302 wait complete. 2024-08-31T07:10:30.211 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:29 smithi032 ceph-mon[17252]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-08-31T07:10:30.211 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:29 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1018137302' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-31T07:10:30.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:29 smithi084 ceph-mon[22387]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-08-31T07:10:30.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:29 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1018137302' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-31T07:10:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:29 smithi073 ceph-mon[21995]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-08-31T07:10:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:29 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1018137302' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-31T07:10:30.659 INFO:teuthology.orchestra.run.smithi032.stdout:[{"version":53,"timestamp":"2024-08-31T07:09:36.055471+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-08-31T07:10:30.659 INFO:tasks.ceph_manager:config epoch is 53 2024-08-31T07:10:30.659 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-31T07:10:30.659 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-31T07:10:30.660 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph mgr dump --format=json 2024-08-31T07:10:31.042 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.858+0000 7f0cbce00640 1 -- 172.21.15.32:0/215110546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb8109090 msgr2=0x7f0cb81094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.858+0000 7f0cbce00640 1 --2- 172.21.15.32:0/215110546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb8109090 0x7f0cb81094a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009900 tx=0x7f0ca002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:10:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.859+0000 7f0cbce00640 1 -- 172.21.15.32:0/215110546 shutdown_connections 2024-08-31T07:10:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.859+0000 7f0cbce00640 1 --2- 172.21.15.32:0/215110546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb81099e0 0x7f0cb8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.859+0000 7f0cbce00640 1 --2- 172.21.15.32:0/215110546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb8109090 0x7f0cb81094a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.859+0000 7f0cbce00640 1 --2- 172.21.15.32:0/215110546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb8103750 0x7f0cb8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.859+0000 7f0cbce00640 1 -- 172.21.15.32:0/215110546 >> 172.21.15.32:0/215110546 conn(0x7f0cb80ff1e0 msgr2=0x7f0cb8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:31.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.859+0000 7f0cbce00640 1 -- 172.21.15.32:0/215110546 shutdown_connections 2024-08-31T07:10:31.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.859+0000 7f0cbce00640 1 -- 172.21.15.32:0/215110546 wait complete. 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.860+0000 7f0cbce00640 1 Processor -- start 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.860+0000 7f0cbce00640 1 -- start start 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.860+0000 7f0cbce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb8103750 0x7f0cb819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.860+0000 7f0cbce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb8109090 0x7f0cb819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.860+0000 7f0cbce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81099e0 0x7f0cb81a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.860+0000 7f0cbce00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8113650 con 0x7f0cb81099e0 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.860+0000 7f0cbce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81134d0 con 0x7f0cb8103750 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.860+0000 7f0cbce00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81137d0 con 0x7f0cb8109090 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cb7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb8103750 0x7f0cb819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cb7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb8103750 0x7f0cb819c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58214/0 (socket says 172.21.15.32:58214) 2024-08-31T07:10:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cb7400640 1 -- 172.21.15.32:0/1898907235 learned_addr learned my addr 172.21.15.32:0/1898907235 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:31.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cb7e00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81099e0 0x7f0cb81a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:31.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cafe00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb8109090 0x7f0cb819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:31.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cb7400640 1 -- 172.21.15.32:0/1898907235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb8109090 msgr2=0x7f0cb819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:31.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cb7400640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb8109090 0x7f0cb819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:31.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cb7400640 1 -- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81099e0 msgr2=0x7f0cb81a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:31.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cb7400640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81099e0 0x7f0cb81a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:31.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cb7400640 1 -- 172.21.15.32:0/1898907235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb81a2230 con 0x7f0cb8103750 2024-08-31T07:10:31.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.861+0000 7f0cb7e00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81099e0 0x7f0cb81a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:31.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.862+0000 7f0cb7400640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb8103750 0x7f0cb819c2c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800d950 tx=0x7f0ca800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:31.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.862+0000 7f0cb4c00640 1 -- 172.21.15.32:0/1898907235 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8017070 con 0x7f0cb8103750 2024-08-31T07:10:31.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.862+0000 7f0cbce00640 1 -- 172.21.15.32:0/1898907235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb81a2520 con 0x7f0cb8103750 2024-08-31T07:10:31.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.862+0000 7f0cb4c00640 1 -- 172.21.15.32:0/1898907235 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca8009c30 con 0x7f0cb8103750 2024-08-31T07:10:31.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.862+0000 7f0cb4c00640 1 -- 172.21.15.32:0/1898907235 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800f620 con 0x7f0cb8103750 2024-08-31T07:10:31.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.862+0000 7f0cbce00640 1 -- 172.21.15.32:0/1898907235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb81a8c80 con 0x7f0cb8103750 2024-08-31T07:10:31.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.863+0000 7f0cbce00640 1 -- 172.21.15.32:0/1898907235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb8103b60 con 0x7f0cb8103750 2024-08-31T07:10:31.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.866+0000 7f0cb4c00640 1 -- 172.21.15.32:0/1898907235 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f0ca8005030 con 0x7f0cb8103750 2024-08-31T07:10:31.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.867+0000 7f0cb4c00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0c9c0b4c10 0x7f0c9c0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:31.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.867+0000 7f0cb4c00640 1 -- 172.21.15.32:0/1898907235 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f0ca80e4f00 con 0x7f0cb8103750 2024-08-31T07:10:31.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.867+0000 7f0cafe00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0c9c0b4c10 0x7f0c9c0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:31.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.868+0000 7f0cafe00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0c9c0b4c10 0x7f0c9c0b7120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009230 tx=0x7f0ca0031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:31.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:31.869+0000 7f0cb4c00640 1 -- 172.21.15.32:0/1898907235 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0ca8009de0 con 0x7f0cb8103750 2024-08-31T07:10:32.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:31 smithi032 ceph-mon[17252]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-08-31T07:10:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.116+0000 7f0cbce00640 1 -- 172.21.15.32:0/1898907235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0cb81a8f20 con 0x7f0cb8103750 2024-08-31T07:10:32.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.121+0000 7f0cb4c00640 1 -- 172.21.15.32:0/1898907235 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+293207 (secure 0 0 0) 0x7f0ca80ab840 con 0x7f0cb8103750 2024-08-31T07:10:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:10:32.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 -- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0c9c0b4c10 msgr2=0x7f0c9c0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:32.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0c9c0b4c10 0x7f0c9c0b7120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009230 tx=0x7f0ca0031400 comp rx=0 tx=0).stop 2024-08-31T07:10:32.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 -- 172.21.15.32:0/1898907235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb8103750 msgr2=0x7f0cb819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:32.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb8103750 0x7f0cb819c2c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800d950 tx=0x7f0ca800de10 comp rx=0 tx=0).stop 2024-08-31T07:10:32.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 -- 172.21.15.32:0/1898907235 shutdown_connections 2024-08-31T07:10:32.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0c9c0b4c10 0x7f0c9c0b7120 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:32.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cb81099e0 0x7f0cb81a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:32.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cb8109090 0x7f0cb819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:32.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 --2- 172.21.15.32:0/1898907235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb8103750 0x7f0cb819c2c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:32.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 -- 172.21.15.32:0/1898907235 >> 172.21.15.32:0/1898907235 conn(0x7f0cb80ff1e0 msgr2=0x7f0cb8108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:32.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.125+0000 7f0cbce00640 1 -- 172.21.15.32:0/1898907235 shutdown_connections 2024-08-31T07:10:32.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:32.126+0000 7f0cbce00640 1 -- 172.21.15.32:0/1898907235 wait complete. 2024-08-31T07:10:32.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:31 smithi084 ceph-mon[22387]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-08-31T07:10:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:31 smithi073 ceph-mon[21995]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-08-31T07:10:32.963 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"flags":0,"active_gid":14220,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4041113440},{"type":"v1","addr":"172.21.15.32:6801","nonce":4041113440}]},"active_addr":"172.21.15.32:6801/4041113440","active_change":"2024-08-31T07:02:44.661936+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24172,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24179,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.32:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":505518887}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2201240747}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1206032550}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3733177002}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1040188956}]}]} 2024-08-31T07:10:32.971 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-31T07:10:32.971 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-31T07:10:32.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:10:33.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:32 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1898907235' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-31T07:10:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:33.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:32 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1898907235' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-31T07:10:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:32 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1898907235' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-31T07:10:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.987+0000 7f3353e00640 1 -- 172.21.15.32:0/92569374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f335410bbe0 msgr2=0x7f3354112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.987+0000 7f3353e00640 1 --2- 172.21.15.32:0/92569374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f335410bbe0 0x7f3354112410 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f334c009ca0 tx=0x7f334c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:10:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.987+0000 7f3353e00640 1 -- 172.21.15.32:0/92569374 shutdown_connections 2024-08-31T07:10:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.987+0000 7f3353e00640 1 --2- 172.21.15.32:0/92569374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f335410bbe0 0x7f3354112410 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.987+0000 7f3353e00640 1 --2- 172.21.15.32:0/92569374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f335410b290 0x7f335410b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.987+0000 7f3353e00640 1 --2- 172.21.15.32:0/92569374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3354105950 0x7f3354105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.987+0000 7f3353e00640 1 -- 172.21.15.32:0/92569374 >> 172.21.15.32:0/92569374 conn(0x7f3354101420 msgr2=0x7f3354103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.988+0000 7f3353e00640 1 -- 172.21.15.32:0/92569374 shutdown_connections 2024-08-31T07:10:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.988+0000 7f3353e00640 1 -- 172.21.15.32:0/92569374 wait complete. 2024-08-31T07:10:33.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.989+0000 7f3353e00640 1 Processor -- start 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.989+0000 7f3353e00640 1 -- start start 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.989+0000 7f3353e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3354105950 0x7f335419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.989+0000 7f3353e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f335410b290 0x7f335419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.989+0000 7f3353e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f335410bbe0 0x7f33541a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.989+0000 7f3353e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f33541158a0 con 0x7f335410b290 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.989+0000 7f3353e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3354115720 con 0x7f3354105950 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.989+0000 7f3353e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3354115a20 con 0x7f335410bbe0 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3352000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f335410b290 0x7f335419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3352a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3354105950 0x7f335419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3352000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f335410b290 0x7f335419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57096/0 (socket says 172.21.15.32:57096) 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3352000640 1 -- 172.21.15.32:0/4055814758 learned_addr learned my addr 172.21.15.32:0/4055814758 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3353400640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f335410bbe0 0x7f33541a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3352a00640 1 -- 172.21.15.32:0/4055814758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f335410bbe0 msgr2=0x7f33541a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3352a00640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f335410bbe0 0x7f33541a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3352a00640 1 -- 172.21.15.32:0/4055814758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f335410b290 msgr2=0x7f335419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3352a00640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f335410b290 0x7f335419c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3352a00640 1 -- 172.21.15.32:0/4055814758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33541a2180 con 0x7f3354105950 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3353400640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f335410bbe0 0x7f33541a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.990+0000 7f3352000640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f335410b290 0x7f335419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.991+0000 7f3352a00640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3354105950 0x7f335419c210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f334400e9f0 tx=0x7f334400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.991+0000 7f334b400640 1 -- 172.21.15.32:0/4055814758 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334400ccc0 con 0x7f3354105950 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.991+0000 7f334b400640 1 -- 172.21.15.32:0/4055814758 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33440108e0 con 0x7f3354105950 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.991+0000 7f334b400640 1 -- 172.21.15.32:0/4055814758 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344018880 con 0x7f3354105950 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.991+0000 7f3353e00640 1 -- 172.21.15.32:0/4055814758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33541a2470 con 0x7f3354105950 2024-08-31T07:10:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.991+0000 7f3353e00640 1 -- 172.21.15.32:0/4055814758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33541a8bd0 con 0x7f3354105950 2024-08-31T07:10:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.992+0000 7f3353e00640 1 -- 172.21.15.32:0/4055814758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3354105d60 con 0x7f3354105950 2024-08-31T07:10:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.994+0000 7f334b400640 1 -- 172.21.15.32:0/4055814758 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f3344010430 con 0x7f3354105950 2024-08-31T07:10:33.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.996+0000 7f334b400640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f33380b4c10 0x7f33380b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.996+0000 7f334b400640 1 -- 172.21.15.32:0/4055814758 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f33440e56c0 con 0x7f3354105950 2024-08-31T07:10:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.996+0000 7f3352000640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f33380b4c10 0x7f33380b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.997+0000 7f3352000640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f33380b4c10 0x7f33380b7120 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3354104c00 tx=0x7f3340005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:33.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:33.998+0000 7f334b400640 1 -- 172.21.15.32:0/4055814758 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f33440ac000 con 0x7f3354105950 2024-08-31T07:10:34.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:33 smithi032 ceph-mon[17252]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-08-31T07:10:34.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.197+0000 7f3353e00640 1 -- 172.21.15.32:0/4055814758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f335402e430 con 0x7f3354105950 2024-08-31T07:10:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.199+0000 7f334b400640 1 -- 172.21.15.32:0/4055814758 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22532 (secure 0 0 0) 0x7f33440b07c0 con 0x7f3354105950 2024-08-31T07:10:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:10:34.200 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:09:48.797927+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:10:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 -- 172.21.15.32:0/4055814758 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f33380b4c10 msgr2=0x7f33380b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f33380b4c10 0x7f33380b7120 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3354104c00 tx=0x7f3340005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 -- 172.21.15.32:0/4055814758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3354105950 msgr2=0x7f335419c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3354105950 0x7f335419c210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f334400e9f0 tx=0x7f334400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 -- 172.21.15.32:0/4055814758 shutdown_connections 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f33380b4c10 0x7f33380b7120 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f335410bbe0 0x7f33541a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f335410b290 0x7f335419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 --2- 172.21.15.32:0/4055814758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3354105950 0x7f335419c210 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 -- 172.21.15.32:0/4055814758 >> 172.21.15.32:0/4055814758 conn(0x7f3354101420 msgr2=0x7f3354108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 -- 172.21.15.32:0/4055814758 shutdown_connections 2024-08-31T07:10:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:34.202+0000 7f3353e00640 1 -- 172.21.15.32:0/4055814758 wait complete. 2024-08-31T07:10:34.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:33 smithi084 ceph-mon[22387]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-08-31T07:10:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:33 smithi073 ceph-mon[21995]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-08-31T07:10:34.990 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-31T07:10:34.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:10:35.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:34 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4055814758' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:10:35.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:34 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4055814758' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:10:35.367 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:35.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:34 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4055814758' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:10:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.988+0000 7fcfc5e00640 1 -- 172.21.15.32:0/627421910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc0103750 msgr2=0x7fcfc0103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.988+0000 7fcfc5e00640 1 --2- 172.21.15.32:0/627421910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc0103750 0x7fcfc0103b60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009860 tx=0x7fcfb402a340 comp rx=0 tx=0).stop 2024-08-31T07:10:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.988+0000 7fcfc5e00640 1 -- 172.21.15.32:0/627421910 shutdown_connections 2024-08-31T07:10:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.988+0000 7fcfc5e00640 1 --2- 172.21.15.32:0/627421910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc01099e0 0x7fcfc0110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.988+0000 7fcfc5e00640 1 --2- 172.21.15.32:0/627421910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc0109090 0x7fcfc01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.988+0000 7fcfc5e00640 1 --2- 172.21.15.32:0/627421910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc0103750 0x7fcfc0103b60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.988+0000 7fcfc5e00640 1 -- 172.21.15.32:0/627421910 >> 172.21.15.32:0/627421910 conn(0x7fcfc00ff1e0 msgr2=0x7fcfc0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.988+0000 7fcfc5e00640 1 -- 172.21.15.32:0/627421910 shutdown_connections 2024-08-31T07:10:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.988+0000 7fcfc5e00640 1 -- 172.21.15.32:0/627421910 wait complete. 2024-08-31T07:10:35.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.989+0000 7fcfc5e00640 1 Processor -- start 2024-08-31T07:10:35.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.990+0000 7fcfc5e00640 1 -- start start 2024-08-31T07:10:35.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.990+0000 7fcfc5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc0103750 0x7fcfc00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:35.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.990+0000 7fcfc5e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc0109090 0x7fcfc00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:35.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.990+0000 7fcfc5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc01099e0 0x7fcfc0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:35.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.990+0000 7fcfc5e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0113690 con 0x7fcfc0103750 2024-08-31T07:10:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.990+0000 7fcfc5e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0113510 con 0x7fcfc01099e0 2024-08-31T07:10:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.990+0000 7fcfc5e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0113810 con 0x7fcfc0109090 2024-08-31T07:10:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.990+0000 7fcfc4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc0103750 0x7fcfc00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.990+0000 7fcfc4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc0103750 0x7fcfc00754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34360/0 (socket says 172.21.15.32:34360) 2024-08-31T07:10:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.990+0000 7fcfc4a00640 1 -- 172.21.15.32:0/3655377556 learned_addr learned my addr 172.21.15.32:0/3655377556 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.991+0000 7fcfbfe00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc0109090 0x7fcfc00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.991+0000 7fcfc5400640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc01099e0 0x7fcfc0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.991+0000 7fcfc4a00640 1 -- 172.21.15.32:0/3655377556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc0109090 msgr2=0x7fcfc00759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.991+0000 7fcfc4a00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc0109090 0x7fcfc00759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.991+0000 7fcfc4a00640 1 -- 172.21.15.32:0/3655377556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc01099e0 msgr2=0x7fcfc0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.991+0000 7fcfc4a00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc01099e0 0x7fcfc0075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.991+0000 7fcfc4a00640 1 -- 172.21.15.32:0/3655377556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc01a8380 con 0x7fcfc0103750 2024-08-31T07:10:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.991+0000 7fcfc4a00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc0103750 0x7fcfc00754b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4035040 tx=0x7fcfb4032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.992+0000 7fcfbd600640 1 -- 172.21.15.32:0/3655377556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4037040 con 0x7fcfc0103750 2024-08-31T07:10:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.992+0000 7fcfc5e00640 1 -- 172.21.15.32:0/3655377556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc01a8610 con 0x7fcfc0103750 2024-08-31T07:10:35.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.992+0000 7fcfbd600640 1 -- 172.21.15.32:0/3655377556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb4041070 con 0x7fcfc0103750 2024-08-31T07:10:35.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.992+0000 7fcfbd600640 1 -- 172.21.15.32:0/3655377556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4003850 con 0x7fcfc0103750 2024-08-31T07:10:35.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.992+0000 7fcfc5e00640 1 -- 172.21.15.32:0/3655377556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc01a8a90 con 0x7fcfc0103750 2024-08-31T07:10:35.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.993+0000 7fcfc5e00640 1 -- 172.21.15.32:0/3655377556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc0103b60 con 0x7fcfc0103750 2024-08-31T07:10:35.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.995+0000 7fcfbd600640 1 -- 172.21.15.32:0/3655377556 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fcfb40039f0 con 0x7fcfc0103750 2024-08-31T07:10:35.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.996+0000 7fcfbd600640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfa80b4c10 0x7fcfa80b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:35.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.997+0000 7fcfbd600640 1 -- 172.21.15.32:0/3655377556 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fcfb41056e0 con 0x7fcfc0103750 2024-08-31T07:10:35.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.997+0000 7fcfbfe00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfa80b4c10 0x7fcfa80b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:35.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.997+0000 7fcfbfe00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfa80b4c10 0x7fcfa80b7120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0108c40 tx=0x7fcfac009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:36.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:35.999+0000 7fcfbd600640 1 -- 172.21.15.32:0/3655377556 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcfb40cc670 con 0x7fcfc0103750 2024-08-31T07:10:36.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:35 smithi032 ceph-mon[17252]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-31T07:10:36.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.196+0000 7fcfc5e00640 1 -- 172.21.15.32:0/3655377556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfc002e430 con 0x7fcfc0103750 2024-08-31T07:10:36.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.198+0000 7fcfbd600640 1 -- 172.21.15.32:0/3655377556 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22532 (secure 0 0 0) 0x7fcfb40cbf70 con 0x7fcfc0103750 2024-08-31T07:10:36.200 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:10:36.200 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:09:48.797927+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:10:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.200+0000 7fcfc5e00640 1 -- 172.21.15.32:0/3655377556 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfa80b4c10 msgr2=0x7fcfa80b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.200+0000 7fcfc5e00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfa80b4c10 0x7fcfa80b7120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0108c40 tx=0x7fcfac009260 comp rx=0 tx=0).stop 2024-08-31T07:10:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.200+0000 7fcfc5e00640 1 -- 172.21.15.32:0/3655377556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc0103750 msgr2=0x7fcfc00754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.200+0000 7fcfc5e00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc0103750 0x7fcfc00754b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4035040 tx=0x7fcfb4032d50 comp rx=0 tx=0).stop 2024-08-31T07:10:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.200+0000 7fcfc5e00640 1 -- 172.21.15.32:0/3655377556 shutdown_connections 2024-08-31T07:10:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.200+0000 7fcfc5e00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfa80b4c10 0x7fcfa80b7120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:36.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.200+0000 7fcfc5e00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfc01099e0 0x7fcfc0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:36.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.200+0000 7fcfc5e00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc0109090 0x7fcfc00759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:36.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.200+0000 7fcfc5e00640 1 --2- 172.21.15.32:0/3655377556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc0103750 0x7fcfc00754b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:36.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.200+0000 7fcfc5e00640 1 -- 172.21.15.32:0/3655377556 >> 172.21.15.32:0/3655377556 conn(0x7fcfc00ff1e0 msgr2=0x7fcfc0101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:36.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.201+0000 7fcfc5e00640 1 -- 172.21.15.32:0/3655377556 shutdown_connections 2024-08-31T07:10:36.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:36.201+0000 7fcfc5e00640 1 -- 172.21.15.32:0/3655377556 wait complete. 2024-08-31T07:10:36.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:35 smithi084 ceph-mon[22387]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-31T07:10:36.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:35 smithi073 ceph-mon[21995]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-31T07:10:37.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-31T07:10:37.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-31T07:10:37.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-31T07:10:37.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-31T07:10:37.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-31T07:10:37.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-31T07:10:37.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-31T07:10:37.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-31T07:10:37.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-31T07:10:37.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-31T07:10:37.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-31T07:10:37.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-31T07:10:37.225 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:36 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3655377556' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:10:37.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:36 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3655377556' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:10:37.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:36 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3655377556' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:10:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:38.093 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:38.122 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:38.145 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:38.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:37 smithi084 ceph-mon[22387]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-31T07:10:38.270 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:37 smithi032 ceph-mon[17252]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-31T07:10:38.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:37 smithi073 ceph-mon[21995]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-31T07:10:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:39.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.832+0000 7f45b2400640 1 -- 172.21.15.32:0/3556582405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac109090 msgr2=0x7f45ac1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.832+0000 7f45b2400640 1 --2- 172.21.15.32:0/3556582405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac109090 0x7f45ac1094a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4594009900 tx=0x7f459402a410 comp rx=0 tx=0).stop 2024-08-31T07:10:39.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.832+0000 7f45b2400640 1 -- 172.21.15.32:0/3556582405 shutdown_connections 2024-08-31T07:10:39.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.832+0000 7f45b2400640 1 --2- 172.21.15.32:0/3556582405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45ac1099e0 0x7f45ac110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.832+0000 7f45b2400640 1 --2- 172.21.15.32:0/3556582405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac109090 0x7f45ac1094a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.832+0000 7f45b2400640 1 --2- 172.21.15.32:0/3556582405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ac103750 0x7f45ac103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.832+0000 7f45b2400640 1 -- 172.21.15.32:0/3556582405 >> 172.21.15.32:0/3556582405 conn(0x7f45ac0ff1a0 msgr2=0x7f45ac1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:39.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.832+0000 7f45b2400640 1 -- 172.21.15.32:0/3556582405 shutdown_connections 2024-08-31T07:10:39.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.832+0000 7f45b2400640 1 -- 172.21.15.32:0/3556582405 wait complete. 2024-08-31T07:10:39.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.834+0000 7f45b2400640 1 Processor -- start 2024-08-31T07:10:39.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.834+0000 7f45b2400640 1 -- start start 2024-08-31T07:10:39.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45b2400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45ac103750 0x7f45ac0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:39.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45b2400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ac109090 0x7f45ac0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:39.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45b2400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac1099e0 0x7f45ac075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:39.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45b2400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac113650 con 0x7f45ac1099e0 2024-08-31T07:10:39.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45b2400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1134d0 con 0x7f45ac109090 2024-08-31T07:10:39.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45b2400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1137d0 con 0x7f45ac103750 2024-08-31T07:10:39.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45b1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac1099e0 0x7f45ac075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:39.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45b1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac1099e0 0x7f45ac075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34400/0 (socket says 172.21.15.32:34400) 2024-08-31T07:10:39.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45b1000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45ac103750 0x7f45ac0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:39.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45b1a00640 1 -- 172.21.15.32:0/3145497991 learned_addr learned my addr 172.21.15.32:0/3145497991 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:39.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.835+0000 7f45a3e00640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ac109090 0x7f45ac0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45b1000640 1 -- 172.21.15.32:0/3145497991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ac109090 msgr2=0x7f45ac0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45b1000640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ac109090 0x7f45ac0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45b1000640 1 -- 172.21.15.32:0/3145497991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac1099e0 msgr2=0x7f45ac075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45b1000640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac1099e0 0x7f45ac075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45b1000640 1 -- 172.21.15.32:0/3145497991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac1a83c0 con 0x7f45ac103750 2024-08-31T07:10:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45b1a00640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac1099e0 0x7f45ac075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:39.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45b1000640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45ac103750 0x7f45ac0754b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e9f0 tx=0x7f459c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:39.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45aa000640 1 -- 172.21.15.32:0/3145497991 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c00ccc0 con 0x7f45ac103750 2024-08-31T07:10:39.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac1a86b0 con 0x7f45ac103750 2024-08-31T07:10:39.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45aa000640 1 -- 172.21.15.32:0/3145497991 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f459c0108e0 con 0x7f45ac103750 2024-08-31T07:10:39.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.836+0000 7f45aa000640 1 -- 172.21.15.32:0/3145497991 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c018880 con 0x7f45ac103750 2024-08-31T07:10:39.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.837+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac1a8c10 con 0x7f45ac103750 2024-08-31T07:10:39.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.838+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45ac0638b0 con 0x7f45ac103750 2024-08-31T07:10:39.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.728+0000 7f26c3e00640 1 -- 172.21.15.32:0/277060191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc109bb0 msgr2=0x7f26bc1103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.728+0000 7f26c3e00640 1 --2- 172.21.15.32:0/277060191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc109bb0 0x7f26bc1103e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009ca0 tx=0x7f26b402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.728+0000 7f26c3e00640 1 -- 172.21.15.32:0/277060191 shutdown_connections 2024-08-31T07:10:39.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.728+0000 7f26c3e00640 1 --2- 172.21.15.32:0/277060191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc109bb0 0x7f26bc1103e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.728+0000 7f26c3e00640 1 --2- 172.21.15.32:0/277060191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26bc109260 0x7f26bc109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.728+0000 7f26c3e00640 1 --2- 172.21.15.32:0/277060191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26bc103920 0x7f26bc103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.728+0000 7f26c3e00640 1 -- 172.21.15.32:0/277060191 >> 172.21.15.32:0/277060191 conn(0x7f26bc0ff330 msgr2=0x7f26bc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.728+0000 7f26c3e00640 1 -- 172.21.15.32:0/277060191 shutdown_connections 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.728+0000 7f26c3e00640 1 -- 172.21.15.32:0/277060191 wait complete. 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.730+0000 7f26c3e00640 1 Processor -- start 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.730+0000 7f26c3e00640 1 -- start start 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc103920 0x7f26bc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26bc109260 0x7f26bc19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26bc109bb0 0x7f26bc19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1138c0 con 0x7f26bc103920 2024-08-31T07:10:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc113740 con 0x7f26bc109bb0 2024-08-31T07:10:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c3e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc113a40 con 0x7f26bc109260 2024-08-31T07:10:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc103920 0x7f26bc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc103920 0x7f26bc19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34382/0 (socket says 172.21.15.32:34382) 2024-08-31T07:10:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c1400640 1 -- 172.21.15.32:0/2356254769 learned_addr learned my addr 172.21.15.32:0/2356254769 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c1e00640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26bc109bb0 0x7f26bc19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.731+0000 7f26c0a00640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26bc109260 0x7f26bc19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.732+0000 7f26c1400640 1 -- 172.21.15.32:0/2356254769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26bc109260 msgr2=0x7f26bc19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.732+0000 7f26c1400640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26bc109260 0x7f26bc19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.732+0000 7f26c1400640 1 -- 172.21.15.32:0/2356254769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26bc109bb0 msgr2=0x7f26bc19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.732+0000 7f26c1400640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26bc109bb0 0x7f26bc19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.732+0000 7f26c1400640 1 -- 172.21.15.32:0/2356254769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc1a01e0 con 0x7f26bc103920 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.732+0000 7f26c1400640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc103920 0x7f26bc19a270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26b000e9f0 tx=0x7f26b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.732+0000 7f26ba000640 1 -- 172.21.15.32:0/2356254769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b000cd80 con 0x7f26bc103920 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.732+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26bc1a04d0 con 0x7f26bc103920 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.732+0000 7f26ba000640 1 -- 172.21.15.32:0/2356254769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26b00054d0 con 0x7f26bc103920 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.733+0000 7f26ba000640 1 -- 172.21.15.32:0/2356254769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0010690 con 0x7f26bc103920 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.733+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26bc1a6c30 con 0x7f26bc103920 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.734+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f26bc0638b0 con 0x7f26bc103920 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.736+0000 7f26ba000640 1 -- 172.21.15.32:0/2356254769 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f26b0004e30 con 0x7f26bc103920 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.739+0000 7f26ba000640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26a40b4c90 0x7f26a40b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.739+0000 7f26ba000640 1 -- 172.21.15.32:0/2356254769 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f26b00e5af0 con 0x7f26bc103920 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.739+0000 7f26c0a00640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26a40b4c90 0x7f26a40b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.739+0000 7f26ba000640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] conn(0x7f26a40be6a0 0x7f26a40c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:39.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.739+0000 7f26ba000640 1 -- 172.21.15.32:0/2356254769 --> [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26b0025ea0 con 0x7f26a40be6a0 2024-08-31T07:10:39.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.739+0000 7f26ba000640 1 -- 172.21.15.32:0/2356254769 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f26b00e5ea0 con 0x7f26bc103920 2024-08-31T07:10:39.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.740+0000 7f26c1e00640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] conn(0x7f26a40be6a0 0x7f26a40c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:39.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.740+0000 7f26c0a00640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26a40b4c90 0x7f26a40b71a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26bc101a20 tx=0x7f26a8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:39.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.740+0000 7f26c1e00640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] conn(0x7f26a40be6a0 0x7f26a40c0af0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:39.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.741+0000 7f26ba000640 1 -- 172.21.15.32:0/2356254769 <== osd.4 v2:172.21.15.73:6808/3774619077 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f26b0025ea0 con 0x7f26a40be6a0 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.756+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 --> [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f26bc063cb0 con 0x7f26a40be6a0 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.757+0000 7f26ba000640 1 -- 172.21.15.32:0/2356254769 <== osd.4 v2:172.21.15.73:6808/3774619077 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f26bc063cb0 con 0x7f26a40be6a0 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.757+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 >> [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] conn(0x7f26a40be6a0 msgr2=0x7f26a40c0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.757+0000 7f26c3e00640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] conn(0x7f26a40be6a0 0x7f26a40c0af0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.757+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26a40b4c90 msgr2=0x7f26a40b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.757+0000 7f26c3e00640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26a40b4c90 0x7f26a40b71a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26bc101a20 tx=0x7f26a8005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.757+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc103920 msgr2=0x7f26bc19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.757+0000 7f26c3e00640 1 --2- 172.21.15.32:0/2356254769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc103920 0x7f26bc19a270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26b000e9f0 tx=0x7f26b000edf0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.757+0000 7f26c1e00640 1 -- 172.21.15.32:0/2356254769 reap_dead start 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.758+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 shutdown_connections 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.758+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 >> 172.21.15.32:0/2356254769 conn(0x7f26bc0ff330 msgr2=0x7f26bc1087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.758+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 shutdown_connections 2024-08-31T07:10:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.758+0000 7f26c3e00640 1 -- 172.21.15.32:0/2356254769 wait complete. 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.840+0000 7f45aa000640 1 -- 172.21.15.32:0/3145497991 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f459c010430 con 0x7f45ac103750 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.842+0000 7f45aa000640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f45900b4c90 0x7f45900b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.842+0000 7f45aa000640 1 -- 172.21.15.32:0/3145497991 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f459c0e5e30 con 0x7f45ac103750 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.842+0000 7f45a3e00640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f45900b4c90 0x7f45900b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.842+0000 7f45aa000640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] conn(0x7f45900be6a0 0x7f45900c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.842+0000 7f45aa000640 1 -- 172.21.15.32:0/3145497991 --> [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f459c015e80 con 0x7f45900be6a0 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.842+0000 7f45aa000640 1 -- 172.21.15.32:0/3145497991 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f459c0e61e0 con 0x7f45ac103750 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.843+0000 7f45b1a00640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] conn(0x7f45900be6a0 0x7f45900c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.843+0000 7f45a3e00640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f45900b4c90 0x7f45900b71a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4594009230 tx=0x7f4594031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.843+0000 7f45b1a00640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] conn(0x7f45900be6a0 0x7f45900c0af0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:39.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.844+0000 7f45aa000640 1 -- 172.21.15.32:0/3145497991 <== osd.5 v2:172.21.15.84:6808/3489144690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f459c015e80 con 0x7f45900be6a0 2024-08-31T07:10:39.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.859+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 --> [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f45ac063d60 con 0x7f45900be6a0 2024-08-31T07:10:39.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45aa000640 1 -- 172.21.15.32:0/3145497991 <== osd.5 v2:172.21.15.84:6808/3489144690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f45ac063d60 con 0x7f45900be6a0 2024-08-31T07:10:39.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 >> [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] conn(0x7f45900be6a0 msgr2=0x7f45900c0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b2400640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] conn(0x7f45900be6a0 0x7f45900c0af0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f45900b4c90 msgr2=0x7f45900b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b2400640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f45900b4c90 0x7f45900b71a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4594009230 tx=0x7f4594031400 comp rx=0 tx=0).stop 2024-08-31T07:10:39.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45ac103750 msgr2=0x7f45ac0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:39.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b2400640 1 --2- 172.21.15.32:0/3145497991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45ac103750 0x7f45ac0754b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e9f0 tx=0x7f459c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:10:39.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b1a00640 1 -- 172.21.15.32:0/3145497991 reap_dead start 2024-08-31T07:10:39.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 shutdown_connections 2024-08-31T07:10:39.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 >> 172.21.15.32:0/3145497991 conn(0x7f45ac0ff1a0 msgr2=0x7f45ac1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:39.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 shutdown_connections 2024-08-31T07:10:39.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:39.860+0000 7f45b2400640 1 -- 172.21.15.32:0/3145497991 wait complete. 2024-08-31T07:10:40.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:39 smithi032 ceph-mon[17252]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:10:40.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:39 smithi073 ceph-mon[21995]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:10:40.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:39 smithi084 ceph-mon[22387]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:10:41.632 INFO:teuthology.orchestra.run.smithi032.stdout:163208757306 2024-08-31T07:10:41.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-31T07:10:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.438+0000 7f86ab000640 1 -- 172.21.15.32:0/3677051893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a41141d0 msgr2=0x7f86a41166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.438+0000 7f86ab000640 1 --2- 172.21.15.32:0/3677051893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a41141d0 0x7f86a41166a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f869c009ca0 tx=0x7f869c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:10:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.438+0000 7f86ab000640 1 -- 172.21.15.32:0/3677051893 shutdown_connections 2024-08-31T07:10:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.438+0000 7f86ab000640 1 --2- 172.21.15.32:0/3677051893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a41141d0 0x7f86a41166a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.438+0000 7f86ab000640 1 --2- 172.21.15.32:0/3677051893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a4075b70 0x7f86a4075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.438+0000 7f86ab000640 1 --2- 172.21.15.32:0/3677051893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a4077120 0x7f86a4075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.438+0000 7f86ab000640 1 -- 172.21.15.32:0/3677051893 >> 172.21.15.32:0/3677051893 conn(0x7f86a407b5f0 msgr2=0x7f86a407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.438+0000 7f86ab000640 1 -- 172.21.15.32:0/3677051893 shutdown_connections 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.438+0000 7f86ab000640 1 -- 172.21.15.32:0/3677051893 wait complete. 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.439+0000 7f86ab000640 1 Processor -- start 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.439+0000 7f86ab000640 1 -- start start 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86ab000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a4075b70 0x7f86a41a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86ab000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a4077120 0x7f86a41a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86ab000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a41141d0 0x7f86a41a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86ab000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4119e10 con 0x7f86a4075b70 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86ab000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4119c90 con 0x7f86a41141d0 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86ab000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4119f90 con 0x7f86a4077120 2024-08-31T07:10:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a4075b70 0x7f86a41a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a4075b70 0x7f86a41a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34436/0 (socket says 172.21.15.32:34436) 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86a3e00640 1 -- 172.21.15.32:0/533513735 learned_addr learned my addr 172.21.15.32:0/533513735 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86a9000640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a41141d0 0x7f86a41a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f869b400640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a4077120 0x7f86a41a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86a3e00640 1 -- 172.21.15.32:0/533513735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a4077120 msgr2=0x7f86a41a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86a3e00640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86a4077120 0x7f86a41a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86a3e00640 1 -- 172.21.15.32:0/533513735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a41141d0 msgr2=0x7f86a41a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86a3e00640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a41141d0 0x7f86a41a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.440+0000 7f86a3e00640 1 -- 172.21.15.32:0/533513735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a41a6710 con 0x7f86a4075b70 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.441+0000 7f86a3e00640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a4075b70 0x7f86a41a07a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f869400d950 tx=0x7f869400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.441+0000 7f86a1600640 1 -- 172.21.15.32:0/533513735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694017030 con 0x7f86a4075b70 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.441+0000 7f86a1600640 1 -- 172.21.15.32:0/533513735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8694004020 con 0x7f86a4075b70 2024-08-31T07:10:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.441+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a41a6a00 con 0x7f86a4075b70 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.441+0000 7f86a1600640 1 -- 172.21.15.32:0/533513735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86940047a0 con 0x7f86a4075b70 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.441+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a41ad160 con 0x7f86a4075b70 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.442+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86a4113840 con 0x7f86a4075b70 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.445+0000 7f86a1600640 1 -- 172.21.15.32:0/533513735 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f8694009a30 con 0x7f86a4075b70 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.446+0000 7f86a1600640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86880b4c90 0x7f86880b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.446+0000 7f86a1600640 1 -- 172.21.15.32:0/533513735 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f86940e5320 con 0x7f86a4075b70 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.446+0000 7f869b400640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86880b4c90 0x7f86880b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.446+0000 7f86a1600640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] conn(0x7f86880be6a0 0x7f86880c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.446+0000 7f86a1600640 1 -- 172.21.15.32:0/533513735 --> [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86880c11c0 con 0x7f86880be6a0 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.446+0000 7f86a1600640 1 -- 172.21.15.32:0/533513735 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f86940e56d0 con 0x7f86a4075b70 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.447+0000 7f86a9000640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] conn(0x7f86880be6a0 0x7f86880c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.447+0000 7f869b400640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86880b4c90 0x7f86880b71a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f86a4116b50 tx=0x7f868c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.447+0000 7f86a9000640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] conn(0x7f86880be6a0 0x7f86880c0af0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.448+0000 7f86a1600640 1 -- 172.21.15.32:0/533513735 <== osd.6 v2:172.21.15.32:6818/1095521887 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86880c11c0 con 0x7f86880be6a0 2024-08-31T07:10:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.463+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 --> [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86a4076a60 con 0x7f86880be6a0 2024-08-31T07:10:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.463+0000 7f86a1600640 1 -- 172.21.15.32:0/533513735 <== osd.6 v2:172.21.15.32:6818/1095521887 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f86a4076a60 con 0x7f86880be6a0 2024-08-31T07:10:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.463+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] conn(0x7f86880be6a0 msgr2=0x7f86880c0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.463+0000 7f86ab000640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] conn(0x7f86880be6a0 0x7f86880c0af0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.464+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86880b4c90 msgr2=0x7f86880b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.464+0000 7f86ab000640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86880b4c90 0x7f86880b71a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f86a4116b50 tx=0x7f868c005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.464+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a4075b70 msgr2=0x7f86a41a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.464+0000 7f86ab000640 1 --2- 172.21.15.32:0/533513735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86a4075b70 0x7f86a41a07a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f869400d950 tx=0x7f869400de10 comp rx=0 tx=0).stop 2024-08-31T07:10:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.464+0000 7f86a9000640 1 -- 172.21.15.32:0/533513735 reap_dead start 2024-08-31T07:10:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.464+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 shutdown_connections 2024-08-31T07:10:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.464+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 >> 172.21.15.32:0/533513735 conn(0x7f86a407b5f0 msgr2=0x7f86a4112de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.464+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 shutdown_connections 2024-08-31T07:10:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:41.464+0000 7f86ab000640 1 -- 172.21.15.32:0/533513735 wait complete. 2024-08-31T07:10:42.276 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:41 smithi032 ceph-mon[17252]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-08-31T07:10:42.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:41 smithi073 ceph-mon[21995]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-08-31T07:10:42.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:41 smithi084 ceph-mon[22387]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-08-31T07:10:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.685+0000 7fbd20c00640 1 -- 172.21.15.32:0/3474459006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd1c1050a0 msgr2=0x7fbd1c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.685+0000 7fbd20c00640 1 --2- 172.21.15.32:0/3474459006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd1c1050a0 0x7fbd1c10d5f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009860 tx=0x7fbd0802a340 comp rx=0 tx=0).stop 2024-08-31T07:10:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.685+0000 7fbd20c00640 1 -- 172.21.15.32:0/3474459006 shutdown_connections 2024-08-31T07:10:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.685+0000 7fbd20c00640 1 --2- 172.21.15.32:0/3474459006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd1c10dbb0 0x7fbd1c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.685+0000 7fbd20c00640 1 --2- 172.21.15.32:0/3474459006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd1c1050a0 0x7fbd1c10d5f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.685+0000 7fbd20c00640 1 --2- 172.21.15.32:0/3474459006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd1c104750 0x7fbd1c104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.685+0000 7fbd20c00640 1 -- 172.21.15.32:0/3474459006 >> 172.21.15.32:0/3474459006 conn(0x7fbd1c0794d0 msgr2=0x7fbd1c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.686+0000 7fbd20c00640 1 -- 172.21.15.32:0/3474459006 shutdown_connections 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.686+0000 7fbd20c00640 1 -- 172.21.15.32:0/3474459006 wait complete. 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.687+0000 7fbd20c00640 1 Processor -- start 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.687+0000 7fbd20c00640 1 -- start start 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd20c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd1c104750 0x7fbd1c198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd20c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd1c1050a0 0x7fbd1c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd20c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd1c10dbb0 0x7fbd1c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd20c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c111620 con 0x7fbd1c104750 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd20c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1114a0 con 0x7fbd1c1050a0 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd20c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1117a0 con 0x7fbd1c10dbb0 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd19e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd1c104750 0x7fbd1c198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd19e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd1c104750 0x7fbd1c198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34460/0 (socket says 172.21.15.32:34460) 2024-08-31T07:10:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd19e00640 1 -- 172.21.15.32:0/2942925221 learned_addr learned my addr 172.21.15.32:0/2942925221 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd19400640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd1c1050a0 0x7fbd1c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.688+0000 7fbd1a800640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd1c10dbb0 0x7fbd1c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd19400640 1 -- 172.21.15.32:0/2942925221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd1c10dbb0 msgr2=0x7fbd1c19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd19400640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd1c10dbb0 0x7fbd1c19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd19400640 1 -- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd1c104750 msgr2=0x7fbd1c198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd19400640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd1c104750 0x7fbd1c198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd19400640 1 -- 172.21.15.32:0/2942925221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c19e0c0 con 0x7fbd1c1050a0 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd19e00640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd1c104750 0x7fbd1c198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd19400640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd1c1050a0 0x7fbd1c1986f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009780 tx=0x7fbd08032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd12a00640 1 -- 172.21.15.32:0/2942925221 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08037040 con 0x7fbd1c1050a0 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd12a00640 1 -- 172.21.15.32:0/2942925221 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd08030c10 con 0x7fbd1c1050a0 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c19e350 con 0x7fbd1c1050a0 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.689+0000 7fbd12a00640 1 -- 172.21.15.32:0/2942925221 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd080035f0 con 0x7fbd1c1050a0 2024-08-31T07:10:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.691+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c1a4b10 con 0x7fbd1c1050a0 2024-08-31T07:10:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.692+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd1c063870 con 0x7fbd1c1050a0 2024-08-31T07:10:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.693+0000 7fbd12a00640 1 -- 172.21.15.32:0/2942925221 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fbd08043020 con 0x7fbd1c1050a0 2024-08-31T07:10:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.696+0000 7fbd12a00640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbd000b90a0 0x7fbd000bb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.696+0000 7fbd12a00640 1 -- 172.21.15.32:0/2942925221 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fbd08104ce0 con 0x7fbd1c1050a0 2024-08-31T07:10:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.696+0000 7fbd19e00640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbd000b90a0 0x7fbd000bb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.696+0000 7fbd12a00640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] conn(0x7fbd000c2ab0 0x7fbd000c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.696+0000 7fbd12a00640 1 -- 172.21.15.32:0/2942925221 --> [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd08042e70 con 0x7fbd000c2ab0 2024-08-31T07:10:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.696+0000 7fbd12a00640 1 -- 172.21.15.32:0/2942925221 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fbd08105090 con 0x7fbd1c1050a0 2024-08-31T07:10:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.697+0000 7fbd19e00640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbd000b90a0 0x7fbd000bb5b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c011490 tx=0x7fbd0c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.697+0000 7fbd1a800640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] conn(0x7fbd000c2ab0 0x7fbd000c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.697+0000 7fbd1a800640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] conn(0x7fbd000c2ab0 0x7fbd000c4f00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.698+0000 7fbd12a00640 1 -- 172.21.15.32:0/2942925221 <== osd.3 v2:172.21.15.32:6810/1969453562 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd08042e70 con 0x7fbd000c2ab0 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.713+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 --> [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbd1c063cf0 con 0x7fbd000c2ab0 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.713+0000 7fbd12a00640 1 -- 172.21.15.32:0/2942925221 <== osd.3 v2:172.21.15.32:6810/1969453562 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbd1c063cf0 con 0x7fbd000c2ab0 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] conn(0x7fbd000c2ab0 msgr2=0x7fbd000c4f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd20c00640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] conn(0x7fbd000c2ab0 0x7fbd000c4f00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbd000b90a0 msgr2=0x7fbd000bb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd20c00640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbd000b90a0 0x7fbd000bb5b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c011490 tx=0x7fbd0c009260 comp rx=0 tx=0).stop 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd1c1050a0 msgr2=0x7fbd1c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd20c00640 1 --2- 172.21.15.32:0/2942925221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd1c1050a0 0x7fbd1c1986f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009780 tx=0x7fbd08032d50 comp rx=0 tx=0).stop 2024-08-31T07:10:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd1a800640 1 -- 172.21.15.32:0/2942925221 reap_dead start 2024-08-31T07:10:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 shutdown_connections 2024-08-31T07:10:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 >> 172.21.15.32:0/2942925221 conn(0x7fbd1c0794d0 msgr2=0x7fbd1c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 shutdown_connections 2024-08-31T07:10:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.714+0000 7fbd20c00640 1 -- 172.21.15.32:0/2942925221 wait complete. 2024-08-31T07:10:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.836+0000 7f4862000640 1 -- 172.21.15.32:0/1239039780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c10b290 msgr2=0x7f485c10b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.836+0000 7f4862000640 1 --2- 172.21.15.32:0/1239039780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c10b290 0x7f485c10b6a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4848009900 tx=0x7f484802a410 comp rx=0 tx=0).stop 2024-08-31T07:10:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.836+0000 7f4862000640 1 -- 172.21.15.32:0/1239039780 shutdown_connections 2024-08-31T07:10:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.836+0000 7f4862000640 1 --2- 172.21.15.32:0/1239039780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f485c10bbe0 0x7f485c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.836+0000 7f4862000640 1 --2- 172.21.15.32:0/1239039780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c10b290 0x7f485c10b6a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.836+0000 7f4862000640 1 --2- 172.21.15.32:0/1239039780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f485c105950 0x7f485c105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.836+0000 7f4862000640 1 -- 172.21.15.32:0/1239039780 >> 172.21.15.32:0/1239039780 conn(0x7f485c101420 msgr2=0x7f485c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.836+0000 7f4862000640 1 -- 172.21.15.32:0/1239039780 shutdown_connections 2024-08-31T07:10:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.836+0000 7f4862000640 1 -- 172.21.15.32:0/1239039780 wait complete. 2024-08-31T07:10:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.837+0000 7f4862000640 1 Processor -- start 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.837+0000 7f4862000640 1 -- start start 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4862000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c105950 0x7f485c19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4862000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f485c10b290 0x7f485c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4862000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f485c10bbe0 0x7f485c1a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4862000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1158a0 con 0x7f485c105950 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4862000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f485c115720 con 0x7f485c10bbe0 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4862000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f485c115a20 con 0x7f485c10b290 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4860c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c105950 0x7f485c19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4860c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c105950 0x7f485c19c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34480/0 (socket says 172.21.15.32:34480) 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4860c00640 1 -- 172.21.15.32:0/1386697103 learned_addr learned my addr 172.21.15.32:0/1386697103 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4861600640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f485c10bbe0 0x7f485c1a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f485be00640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f485c10b290 0x7f485c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4860c00640 1 -- 172.21.15.32:0/1386697103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f485c10b290 msgr2=0x7f485c19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4860c00640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f485c10b290 0x7f485c19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4860c00640 1 -- 172.21.15.32:0/1386697103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f485c10bbe0 msgr2=0x7f485c1a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4860c00640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f485c10bbe0 0x7f485c1a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.838+0000 7f4860c00640 1 -- 172.21.15.32:0/1386697103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c1a2240 con 0x7f485c105950 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.839+0000 7f4860c00640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c105950 0x7f485c19c2d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f485000e9f0 tx=0x7f485000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.839+0000 7f4859600640 1 -- 172.21.15.32:0/1386697103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485000cd80 con 0x7f485c105950 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.839+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c1a2530 con 0x7f485c105950 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.839+0000 7f4859600640 1 -- 172.21.15.32:0/1386697103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48500054d0 con 0x7f485c105950 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.839+0000 7f4859600640 1 -- 172.21.15.32:0/1386697103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850010690 con 0x7f485c105950 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.839+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c1a8c90 con 0x7f485c105950 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.840+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f485c0638f0 con 0x7f485c105950 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.844+0000 7f4859600640 1 -- 172.21.15.32:0/1386697103 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f4850004e30 con 0x7f485c105950 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.844+0000 7f4859600640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48440b4c10 0x7f48440b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:42.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.844+0000 7f4859600640 1 -- 172.21.15.32:0/1386697103 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f48500e5b70 con 0x7f485c105950 2024-08-31T07:10:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.844+0000 7f485be00640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48440b4c10 0x7f48440b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.845+0000 7f4859600640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] conn(0x7f48440be620 0x7f48440c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.845+0000 7f4859600640 1 -- 172.21.15.32:0/1386697103 --> [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4850025ea0 con 0x7f48440be620 2024-08-31T07:10:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.845+0000 7f4859600640 1 -- 172.21.15.32:0/1386697103 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f48500e5f20 con 0x7f485c105950 2024-08-31T07:10:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.845+0000 7f4861600640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] conn(0x7f48440be620 0x7f48440c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.845+0000 7f485be00640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48440b4c10 0x7f48440b7120 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4848009230 tx=0x7f4848031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.845+0000 7f4861600640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] conn(0x7f48440be620 0x7f48440c0a70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:42.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.846+0000 7f4859600640 1 -- 172.21.15.32:0/1386697103 <== osd.11 v2:172.21.15.84:6824/3154011828 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4850025ea0 con 0x7f48440be620 2024-08-31T07:10:42.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.862+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 --> [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f485c063d70 con 0x7f48440be620 2024-08-31T07:10:42.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.862+0000 7f4859600640 1 -- 172.21.15.32:0/1386697103 <== osd.11 v2:172.21.15.84:6824/3154011828 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f485c063d70 con 0x7f48440be620 2024-08-31T07:10:42.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.863+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 >> [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] conn(0x7f48440be620 msgr2=0x7f48440c0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.863+0000 7f4862000640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] conn(0x7f48440be620 0x7f48440c0a70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.863+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48440b4c10 msgr2=0x7f48440b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.863+0000 7f4862000640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48440b4c10 0x7f48440b7120 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4848009230 tx=0x7f4848031400 comp rx=0 tx=0).stop 2024-08-31T07:10:42.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.863+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c105950 msgr2=0x7f485c19c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:42.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.863+0000 7f4862000640 1 --2- 172.21.15.32:0/1386697103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c105950 0x7f485c19c2d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f485000e9f0 tx=0x7f485000edf0 comp rx=0 tx=0).stop 2024-08-31T07:10:42.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.863+0000 7f4861600640 1 -- 172.21.15.32:0/1386697103 reap_dead start 2024-08-31T07:10:42.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.863+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 shutdown_connections 2024-08-31T07:10:42.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.863+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 >> 172.21.15.32:0/1386697103 conn(0x7f485c101420 msgr2=0x7f485c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:42.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.864+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 shutdown_connections 2024-08-31T07:10:42.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:42.864+0000 7f4862000640 1 -- 172.21.15.32:0/1386697103 wait complete. 2024-08-31T07:10:43.192 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:43 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:10:43.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:43 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:10:43.335 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:42 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:10:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.512+0000 7f39c1c00640 1 -- 172.21.15.32:0/338838490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc075e40 msgr2=0x7f39bc10f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.512+0000 7f39c1c00640 1 --2- 172.21.15.32:0/338838490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc075e40 0x7f39bc10f980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009860 tx=0x7f39a402a340 comp rx=0 tx=0).stop 2024-08-31T07:10:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.512+0000 7f39c1c00640 1 -- 172.21.15.32:0/338838490 shutdown_connections 2024-08-31T07:10:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.512+0000 7f39c1c00640 1 --2- 172.21.15.32:0/338838490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39bc10fec0 0x7f39bc1122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.512+0000 7f39c1c00640 1 --2- 172.21.15.32:0/338838490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc075e40 0x7f39bc10f980 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.512+0000 7f39c1c00640 1 --2- 172.21.15.32:0/338838490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39bc0754f0 0x7f39bc075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.512+0000 7f39c1c00640 1 -- 172.21.15.32:0/338838490 >> 172.21.15.32:0/338838490 conn(0x7f39bc07b5f0 msgr2=0x7f39bc07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:43.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.512+0000 7f39c1c00640 1 -- 172.21.15.32:0/338838490 shutdown_connections 2024-08-31T07:10:43.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.512+0000 7f39c1c00640 1 -- 172.21.15.32:0/338838490 wait complete. 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.513+0000 7f39c1c00640 1 Processor -- start 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.513+0000 7f39c1c00640 1 -- start start 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39c1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39bc0754f0 0x7f39bc19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39c1c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39bc075e40 0x7f39bc19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39c1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc10fec0 0x7f39bc1a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39c1c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc1159f0 con 0x7f39bc0754f0 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39c1c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc115870 con 0x7f39bc10fec0 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39c1c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc115b70 con 0x7f39bc075e40 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39baa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39bc0754f0 0x7f39bc19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39b2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39bc075e40 0x7f39bc19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39b2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39bc075e40 0x7f39bc19c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44768/0 (socket says 172.21.15.32:44768) 2024-08-31T07:10:43.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39b2000640 1 -- 172.21.15.32:0/1305843247 learned_addr learned my addr 172.21.15.32:0/1305843247 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39baa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39bc0754f0 0x7f39bc19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34498/0 (socket says 172.21.15.32:34498) 2024-08-31T07:10:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39bb400640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc10fec0 0x7f39bc1a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39b2000640 1 -- 172.21.15.32:0/1305843247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc10fec0 msgr2=0x7f39bc1a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39b2000640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc10fec0 0x7f39bc1a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39b2000640 1 -- 172.21.15.32:0/1305843247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39bc0754f0 msgr2=0x7f39bc19c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39b2000640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39bc0754f0 0x7f39bc19c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39b2000640 1 -- 172.21.15.32:0/1305843247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc1a2390 con 0x7f39bc075e40 2024-08-31T07:10:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.514+0000 7f39bb400640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc10fec0 0x7f39bc1a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.515+0000 7f39b2000640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39bc075e40 0x7f39bc19c9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f39a4035040 tx=0x7f39a4032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.515+0000 7f39b3e00640 1 -- 172.21.15.32:0/1305843247 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4040020 con 0x7f39bc075e40 2024-08-31T07:10:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.515+0000 7f39b3e00640 1 -- 172.21.15.32:0/1305843247 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a4030d50 con 0x7f39bc075e40 2024-08-31T07:10:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.515+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc1a2620 con 0x7f39bc075e40 2024-08-31T07:10:43.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.515+0000 7f39b3e00640 1 -- 172.21.15.32:0/1305843247 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4037040 con 0x7f39bc075e40 2024-08-31T07:10:43.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.515+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc1a8de0 con 0x7f39bc075e40 2024-08-31T07:10:43.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.516+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39bc0638f0 con 0x7f39bc075e40 2024-08-31T07:10:43.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.519+0000 7f39b3e00640 1 -- 172.21.15.32:0/1305843247 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f39a4043020 con 0x7f39bc075e40 2024-08-31T07:10:43.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.520+0000 7f39b3e00640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39a00b4c10 0x7f39a00b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.520+0000 7f39b3e00640 1 -- 172.21.15.32:0/1305843247 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f39a4105250 con 0x7f39bc075e40 2024-08-31T07:10:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.520+0000 7f39baa00640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39a00b4c10 0x7f39a00b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.520+0000 7f39b3e00640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] conn(0x7f39a00be620 0x7f39a00c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.521+0000 7f39b3e00640 1 -- 172.21.15.32:0/1305843247 --> [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39a4009030 con 0x7f39a00be620 2024-08-31T07:10:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.521+0000 7f39b3e00640 1 -- 172.21.15.32:0/1305843247 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f39a4105600 con 0x7f39bc075e40 2024-08-31T07:10:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.521+0000 7f39bb400640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] conn(0x7f39a00be620 0x7f39a00c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.521+0000 7f39baa00640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39a00b4c10 0x7f39a00b7120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f39ac005fd0 tx=0x7f39ac009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.521+0000 7f39bb400640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] conn(0x7f39a00be620 0x7f39a00c0a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:43.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.522+0000 7f39b3e00640 1 -- 172.21.15.32:0/1305843247 <== osd.1 v2:172.21.15.73:6800/4226535850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f39a4009030 con 0x7f39a00be620 2024-08-31T07:10:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.537+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 --> [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f39bc063d70 con 0x7f39a00be620 2024-08-31T07:10:43.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.537+0000 7f39b3e00640 1 -- 172.21.15.32:0/1305843247 <== osd.1 v2:172.21.15.73:6800/4226535850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f39bc063d70 con 0x7f39a00be620 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.537+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 >> [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] conn(0x7f39a00be620 msgr2=0x7f39a00c0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.537+0000 7f39c1c00640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] conn(0x7f39a00be620 0x7f39a00c0a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.537+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39a00b4c10 msgr2=0x7f39a00b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.538+0000 7f39c1c00640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39a00b4c10 0x7f39a00b7120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f39ac005fd0 tx=0x7f39ac009260 comp rx=0 tx=0).stop 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.538+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39bc075e40 msgr2=0x7f39bc19c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.538+0000 7f39c1c00640 1 --2- 172.21.15.32:0/1305843247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39bc075e40 0x7f39bc19c9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f39a4035040 tx=0x7f39a4032d50 comp rx=0 tx=0).stop 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.538+0000 7f39bb400640 1 -- 172.21.15.32:0/1305843247 reap_dead start 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.538+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 shutdown_connections 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.538+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 >> 172.21.15.32:0/1305843247 conn(0x7f39bc07b5f0 msgr2=0x7f39bc101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.538+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 shutdown_connections 2024-08-31T07:10:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:43.538+0000 7f39c1c00640 1 -- 172.21.15.32:0/1305843247 wait complete. 2024-08-31T07:10:44.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.665+0000 7f880f600640 1 -- 172.21.15.32:0/2673840116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880810bdb0 msgr2=0x7f88081125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:44.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.665+0000 7f880f600640 1 --2- 172.21.15.32:0/2673840116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880810bdb0 0x7f88081125e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8800009ca0 tx=0x7f880002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:10:44.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.665+0000 7f880f600640 1 -- 172.21.15.32:0/2673840116 shutdown_connections 2024-08-31T07:10:44.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.665+0000 7f880f600640 1 --2- 172.21.15.32:0/2673840116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880810bdb0 0x7f88081125e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:44.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.665+0000 7f880f600640 1 --2- 172.21.15.32:0/2673840116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f880810b460 0x7f880810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.665+0000 7f880f600640 1 --2- 172.21.15.32:0/2673840116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8808105b20 0x7f8808105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.665+0000 7f880f600640 1 -- 172.21.15.32:0/2673840116 >> 172.21.15.32:0/2673840116 conn(0x7f88081015b0 msgr2=0x7f8808103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.665+0000 7f880f600640 1 -- 172.21.15.32:0/2673840116 shutdown_connections 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.665+0000 7f880f600640 1 -- 172.21.15.32:0/2673840116 wait complete. 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.666+0000 7f880f600640 1 Processor -- start 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.666+0000 7f880f600640 1 -- start start 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880f600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8808105b20 0x7f880819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880f600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f880810b460 0x7f880819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880f600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880810bdb0 0x7f88081a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880f600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8808115a70 con 0x7f880810bdb0 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880f600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f88081158f0 con 0x7f880810b460 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880f600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8808115bf0 con 0x7f8808105b20 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880810bdb0 0x7f88081a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:44.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8808105b20 0x7f880819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8808105b20 0x7f880819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44706/0 (socket says 172.21.15.32:44706) 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880810bdb0 0x7f88081a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34418/0 (socket says 172.21.15.32:34418) 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880cc00640 1 -- 172.21.15.32:0/1701090083 learned_addr learned my addr 172.21.15.32:0/1701090083 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f87ffe00640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f880810b460 0x7f880819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880cc00640 1 -- 172.21.15.32:0/1701090083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f880810b460 msgr2=0x7f880819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880cc00640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f880810b460 0x7f880819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880cc00640 1 -- 172.21.15.32:0/1701090083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880810bdb0 msgr2=0x7f88081a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880cc00640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880810bdb0 0x7f88081a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.667+0000 7f880cc00640 1 -- 172.21.15.32:0/1701090083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88081a2410 con 0x7f8808105b20 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.668+0000 7f880cc00640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8808105b20 0x7f880819c4a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f87f800ea60 tx=0x7f87f800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.668+0000 7f8806000640 1 -- 172.21.15.32:0/1701090083 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f800cde0 con 0x7f8808105b20 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.668+0000 7f8806000640 1 -- 172.21.15.32:0/1701090083 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f80035f0 con 0x7f8808105b20 2024-08-31T07:10:44.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.668+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88081a2700 con 0x7f8808105b20 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.668+0000 7f8806000640 1 -- 172.21.15.32:0/1701090083 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8010690 con 0x7f8808105b20 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.668+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88081a8e60 con 0x7f8808105b20 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.669+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88080638b0 con 0x7f8808105b20 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.671+0000 7f8806000640 1 -- 172.21.15.32:0/1701090083 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f87f80037a0 con 0x7f8808105b20 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.673+0000 7f8806000640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87ec0b4c10 0x7f87ec0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.673+0000 7f8806000640 1 -- 172.21.15.32:0/1701090083 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f87f80e5a20 con 0x7f8808105b20 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.673+0000 7f87ffe00640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87ec0b4c10 0x7f87ec0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.673+0000 7f8806000640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] conn(0x7f87ec0be620 0x7f87ec0c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.673+0000 7f8806000640 1 -- 172.21.15.32:0/1701090083 --> [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87f8022e50 con 0x7f87ec0be620 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.673+0000 7f8806000640 1 -- 172.21.15.32:0/1701090083 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f87f80e5dd0 con 0x7f8808105b20 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.674+0000 7f880d600640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] conn(0x7f87ec0be620 0x7f87ec0c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.674+0000 7f87ffe00640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87ec0b4c10 0x7f87ec0b7120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8808104cb0 tx=0x7f87f0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.674+0000 7f880d600640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] conn(0x7f87ec0be620 0x7f87ec0c0a70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:44.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.675+0000 7f8806000640 1 -- 172.21.15.32:0/1701090083 <== osd.2 v2:172.21.15.84:6800/904624859 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87f8022e50 con 0x7f87ec0be620 2024-08-31T07:10:44.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.689+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 --> [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8808063d30 con 0x7f87ec0be620 2024-08-31T07:10:44.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.690+0000 7f8806000640 1 -- 172.21.15.32:0/1701090083 <== osd.2 v2:172.21.15.84:6800/904624859 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8808063d30 con 0x7f87ec0be620 2024-08-31T07:10:44.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.690+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 >> [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] conn(0x7f87ec0be620 msgr2=0x7f87ec0c0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:44.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.690+0000 7f880f600640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] conn(0x7f87ec0be620 0x7f87ec0c0a70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:44.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.691+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87ec0b4c10 msgr2=0x7f87ec0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:44.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.691+0000 7f880f600640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87ec0b4c10 0x7f87ec0b7120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8808104cb0 tx=0x7f87f0005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:44.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.691+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8808105b20 msgr2=0x7f880819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.691+0000 7f880f600640 1 --2- 172.21.15.32:0/1701090083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8808105b20 0x7f880819c4a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f87f800ea60 tx=0x7f87f800ef20 comp rx=0 tx=0).stop 2024-08-31T07:10:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.691+0000 7f880d600640 1 -- 172.21.15.32:0/1701090083 reap_dead start 2024-08-31T07:10:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.691+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 shutdown_connections 2024-08-31T07:10:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.691+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 >> 172.21.15.32:0/1701090083 conn(0x7f88081015b0 msgr2=0x7f880810aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.691+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 shutdown_connections 2024-08-31T07:10:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:40.691+0000 7f880f600640 1 -- 172.21.15.32:0/1701090083 wait complete. 2024-08-31T07:10:44.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:44 smithi032 ceph-mon[17252]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-31T07:10:44.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:44 smithi073 ceph-mon[21995]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-31T07:10:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:44 smithi084 ceph-mon[22387]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-31T07:10:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:10:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:141733920830 2024-08-31T07:10:44.774 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-31T07:10:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.044+0000 7f1eaa600640 1 -- 172.21.15.32:0/3117380333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea4103920 msgr2=0x7f1ea4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.044+0000 7f1eaa600640 1 --2- 172.21.15.32:0/3117380333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea4103920 0x7f1ea4103d30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009900 tx=0x7f1e9402a410 comp rx=0 tx=0).stop 2024-08-31T07:10:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.045+0000 7f1eaa600640 1 -- 172.21.15.32:0/3117380333 shutdown_connections 2024-08-31T07:10:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.045+0000 7f1eaa600640 1 --2- 172.21.15.32:0/3117380333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ea4109bb0 0x7f1ea41103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.045+0000 7f1eaa600640 1 --2- 172.21.15.32:0/3117380333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4109260 0x7f1ea4109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.045+0000 7f1eaa600640 1 --2- 172.21.15.32:0/3117380333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea4103920 0x7f1ea4103d30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.045+0000 7f1eaa600640 1 -- 172.21.15.32:0/3117380333 >> 172.21.15.32:0/3117380333 conn(0x7f1ea40ff3b0 msgr2=0x7f1ea4101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.045+0000 7f1eaa600640 1 -- 172.21.15.32:0/3117380333 shutdown_connections 2024-08-31T07:10:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.045+0000 7f1eaa600640 1 -- 172.21.15.32:0/3117380333 wait complete. 2024-08-31T07:10:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.046+0000 7f1eaa600640 1 Processor -- start 2024-08-31T07:10:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.046+0000 7f1eaa600640 1 -- start start 2024-08-31T07:10:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.046+0000 7f1eaa600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ea4103920 0x7f1ea419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.047+0000 7f1eaa600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4109260 0x7f1ea419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.047+0000 7f1eaa600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea4109bb0 0x7f1ea41a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.047+0000 7f1eaa600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea4113870 con 0x7f1ea4109bb0 2024-08-31T07:10:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.047+0000 7f1eaa600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea41136f0 con 0x7f1ea4109260 2024-08-31T07:10:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.047+0000 7f1eaa600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea41139f0 con 0x7f1ea4103920 2024-08-31T07:10:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.047+0000 7f1ea3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ea4103920 0x7f1ea419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.047+0000 7f1ea3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ea4103920 0x7f1ea419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44794/0 (socket says 172.21.15.32:44794) 2024-08-31T07:10:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.047+0000 7f1ea3400640 1 -- 172.21.15.32:0/168157130 learned_addr learned my addr 172.21.15.32:0/168157130 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.047+0000 7f1ea2a00640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4109260 0x7f1ea419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.047+0000 7f1ea3e00640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea4109bb0 0x7f1ea41a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.048+0000 7f1ea3400640 1 -- 172.21.15.32:0/168157130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4109260 msgr2=0x7f1ea419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.048+0000 7f1ea3400640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4109260 0x7f1ea419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.048+0000 7f1ea3400640 1 -- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea4109bb0 msgr2=0x7f1ea41a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.048+0000 7f1ea3400640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea4109bb0 0x7f1ea41a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.048+0000 7f1ea3400640 1 -- 172.21.15.32:0/168157130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea41a2390 con 0x7f1ea4103920 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.048+0000 7f1ea3e00640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea4109bb0 0x7f1ea41a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.048+0000 7f1ea3400640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ea4103920 0x7f1ea419c420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e94005940 tx=0x7f1e940040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.048+0000 7f1e9be00640 1 -- 172.21.15.32:0/168157130 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9403f030 con 0x7f1ea4103920 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.049+0000 7f1e9be00640 1 -- 172.21.15.32:0/168157130 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e940044c0 con 0x7f1ea4103920 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.049+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea41a2620 con 0x7f1ea4103920 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.049+0000 7f1e9be00640 1 -- 172.21.15.32:0/168157130 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94035070 con 0x7f1ea4103920 2024-08-31T07:10:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.049+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea41a8de0 con 0x7f1ea4103920 2024-08-31T07:10:45.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.050+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1ea410f9e0 con 0x7f1ea4103920 2024-08-31T07:10:45.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.052+0000 7f1e9be00640 1 -- 172.21.15.32:0/168157130 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f1e94042030 con 0x7f1ea4103920 2024-08-31T07:10:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.054+0000 7f1e9be00640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e880b4c90 0x7f1e880b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.054+0000 7f1e9be00640 1 -- 172.21.15.32:0/168157130 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8397+0+0 (secure 0 0 0) 0x7f1e94105e60 con 0x7f1ea4103920 2024-08-31T07:10:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.054+0000 7f1ea2a00640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e880b4c90 0x7f1e880b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.055+0000 7f1e9be00640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] conn(0x7f1e880be980 0x7f1e880c0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.055+0000 7f1e9be00640 1 -- 172.21.15.32:0/168157130 --> [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e9403ce50 con 0x7f1e880be980 2024-08-31T07:10:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.055+0000 7f1e9be00640 1 -- 172.21.15.32:0/168157130 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f1e94106210 con 0x7f1ea4103920 2024-08-31T07:10:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.055+0000 7f1ea3e00640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] conn(0x7f1e880be980 0x7f1e880c0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.055+0000 7f1ea2a00640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e880b4c90 0x7f1e880b71a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1ea406b0c0 tx=0x7f1e90005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.056+0000 7f1ea3e00640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] conn(0x7f1e880be980 0x7f1e880c0e70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:45.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.057+0000 7f1e9be00640 1 -- 172.21.15.32:0/168157130 <== osd.0 v2:172.21.15.32:6802/1563132016 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e9403ce50 con 0x7f1e880be980 2024-08-31T07:10:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.071+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 --> [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1ea4104ba0 con 0x7f1e880be980 2024-08-31T07:10:45.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.072+0000 7f1e9be00640 1 -- 172.21.15.32:0/168157130 <== osd.0 v2:172.21.15.32:6802/1563132016 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1ea4104ba0 con 0x7f1e880be980 2024-08-31T07:10:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.072+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] conn(0x7f1e880be980 msgr2=0x7f1e880c0e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:45.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.072+0000 7f1eaa600640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] conn(0x7f1e880be980 0x7f1e880c0e70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.072+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e880b4c90 msgr2=0x7f1e880b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.072+0000 7f1eaa600640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e880b4c90 0x7f1e880b71a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1ea406b0c0 tx=0x7f1e90005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.073+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ea4103920 msgr2=0x7f1ea419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.073+0000 7f1eaa600640 1 --2- 172.21.15.32:0/168157130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ea4103920 0x7f1ea419c420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e94005940 tx=0x7f1e940040c0 comp rx=0 tx=0).stop 2024-08-31T07:10:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.073+0000 7f1ea3e00640 1 -- 172.21.15.32:0/168157130 reap_dead start 2024-08-31T07:10:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.073+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 shutdown_connections 2024-08-31T07:10:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.073+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 >> 172.21.15.32:0/168157130 conn(0x7f1ea40ff3b0 msgr2=0x7f1ea4108890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.073+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 shutdown_connections 2024-08-31T07:10:45.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.073+0000 7f1eaa600640 1 -- 172.21.15.32:0/168157130 wait complete. 2024-08-31T07:10:45.254 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-08-31T07:10:45.254 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-31T07:10:45.254 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]} : dispatch 2024-08-31T07:10:45.254 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-08-31T07:10:45.255 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]} : dispatch 2024-08-31T07:10:45.255 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-08-31T07:10:45.255 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-31T07:10:45.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-08-31T07:10:45.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-31T07:10:45.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]} : dispatch 2024-08-31T07:10:45.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-08-31T07:10:45.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]} : dispatch 2024-08-31T07:10:45.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-08-31T07:10:45.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-31T07:10:45.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-08-31T07:10:45.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-31T07:10:45.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]} : dispatch 2024-08-31T07:10:45.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-08-31T07:10:45.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]} : dispatch 2024-08-31T07:10:45.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-08-31T07:10:45.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-31T07:10:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:46.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:46 smithi032 ceph-mon[17252]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:10:46.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-08-31T07:10:46.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-31T07:10:46.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-08-31T07:10:46.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-08-31T07:10:46.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-08-31T07:10:46.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-08-31T07:10:46.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-31T07:10:46.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:46 smithi032 ceph-mon[17252]: osdmap e79: 12 total, 12 up, 12 in 2024-08-31T07:10:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:46 smithi073 ceph-mon[21995]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:10:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-08-31T07:10:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-31T07:10:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-08-31T07:10:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-08-31T07:10:46.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-08-31T07:10:46.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-08-31T07:10:46.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-31T07:10:46.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:46 smithi073 ceph-mon[21995]: osdmap e79: 12 total, 12 up, 12 in 2024-08-31T07:10:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:46 smithi084 ceph-mon[22387]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:10:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-08-31T07:10:46.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-31T07:10:46.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-08-31T07:10:46.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-08-31T07:10:46.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-08-31T07:10:46.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-08-31T07:10:46.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-31T07:10:46.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:46 smithi084 ceph-mon[22387]: osdmap e79: 12 total, 12 up, 12 in 2024-08-31T07:10:46.986 INFO:teuthology.orchestra.run.smithi032.stdout:38654705749 2024-08-31T07:10:46.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-31T07:10:47.199 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:47 smithi032 ceph-mon[17252]: osdmap e80: 12 total, 12 up, 12 in 2024-08-31T07:10:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.373+0000 7f6a63e00640 1 -- 172.21.15.32:0/2263430888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6410c450 msgr2=0x7f6a6410c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.373+0000 7f6a63e00640 1 --2- 172.21.15.32:0/2263430888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6410c450 0x7f6a6410c860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009900 tx=0x7f6a4c02a410 comp rx=0 tx=0).stop 2024-08-31T07:10:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.373+0000 7f6a63e00640 1 -- 172.21.15.32:0/2263430888 shutdown_connections 2024-08-31T07:10:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.373+0000 7f6a63e00640 1 --2- 172.21.15.32:0/2263430888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6410cda0 0x7f6a64116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.373+0000 7f6a63e00640 1 --2- 172.21.15.32:0/2263430888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6410c450 0x7f6a6410c860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.373+0000 7f6a63e00640 1 --2- 172.21.15.32:0/2263430888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6410a230 0x7f6a6410a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.373+0000 7f6a63e00640 1 -- 172.21.15.32:0/2263430888 >> 172.21.15.32:0/2263430888 conn(0x7f6a640791c0 msgr2=0x7f6a6407b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.373+0000 7f6a63e00640 1 -- 172.21.15.32:0/2263430888 shutdown_connections 2024-08-31T07:10:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.374+0000 7f6a63e00640 1 -- 172.21.15.32:0/2263430888 wait complete. 2024-08-31T07:10:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.375+0000 7f6a63e00640 1 Processor -- start 2024-08-31T07:10:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.375+0000 7f6a63e00640 1 -- start start 2024-08-31T07:10:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a63e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6410a230 0x7f6a641a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a63e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6410c450 0x7f6a641a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a63e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6410cda0 0x7f6a641a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a63e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64119cc0 con 0x7f6a6410c450 2024-08-31T07:10:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a63e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64119b40 con 0x7f6a6410a230 2024-08-31T07:10:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a63e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64119e40 con 0x7f6a6410cda0 2024-08-31T07:10:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a5be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6410c450 0x7f6a641a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a5be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6410c450 0x7f6a641a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42522/0 (socket says 172.21.15.32:42522) 2024-08-31T07:10:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a62a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6410a230 0x7f6a641a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a5be00640 1 -- 172.21.15.32:0/2323086884 learned_addr learned my addr 172.21.15.32:0/2323086884 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.376+0000 7f6a63400640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6410cda0 0x7f6a641a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.377+0000 7f6a62a00640 1 -- 172.21.15.32:0/2323086884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6410cda0 msgr2=0x7f6a641a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.377+0000 7f6a62a00640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6410cda0 0x7f6a641a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.377+0000 7f6a62a00640 1 -- 172.21.15.32:0/2323086884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6410c450 msgr2=0x7f6a641a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.377+0000 7f6a62a00640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6410c450 0x7f6a641a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.377+0000 7f6a62a00640 1 -- 172.21.15.32:0/2323086884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a641a65a0 con 0x7f6a6410a230 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.377+0000 7f6a62a00640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6410a230 0x7f6a641a0630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400e9c0 tx=0x7f6a5400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.377+0000 7f6a5b400640 1 -- 172.21.15.32:0/2323086884 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400ccc0 con 0x7f6a6410a230 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.378+0000 7f6a5b400640 1 -- 172.21.15.32:0/2323086884 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a540035f0 con 0x7f6a6410a230 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.378+0000 7f6a63e00640 1 -- 172.21.15.32:0/2323086884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a641a6890 con 0x7f6a6410a230 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.378+0000 7f6a63e00640 1 -- 172.21.15.32:0/2323086884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a641acff0 con 0x7f6a6410a230 2024-08-31T07:10:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.378+0000 7f6a5b400640 1 -- 172.21.15.32:0/2323086884 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400ce60 con 0x7f6a6410a230 2024-08-31T07:10:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.379+0000 7f6a58c00640 1 -- 172.21.15.32:0/2323086884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a640638b0 con 0x7f6a6410a230 2024-08-31T07:10:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.381+0000 7f6a5b400640 1 -- 172.21.15.32:0/2323086884 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f6a54004000 con 0x7f6a6410a230 2024-08-31T07:10:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.383+0000 7f6a5b400640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6a480b4c10 0x7f6a480b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.383+0000 7f6a5b400640 1 -- 172.21.15.32:0/2323086884 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6a540e5ae0 con 0x7f6a6410a230 2024-08-31T07:10:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.383+0000 7f6a5be00640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6a480b4c10 0x7f6a480b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.383+0000 7f6a5b400640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] conn(0x7f6a480be900 0x7f6a480c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.383+0000 7f6a5b400640 1 -- 172.21.15.32:0/2323086884 --> [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a5401ee50 con 0x7f6a480be900 2024-08-31T07:10:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.383+0000 7f6a5b400640 1 -- 172.21.15.32:0/2323086884 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f6a540e5e90 con 0x7f6a6410a230 2024-08-31T07:10:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.384+0000 7f6a63400640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] conn(0x7f6a480be900 0x7f6a480c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.384+0000 7f6a5be00640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6a480b4c10 0x7f6a480b7120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005f70 tx=0x7f6a4c005e80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.384+0000 7f6a63400640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] conn(0x7f6a480be900 0x7f6a480c0df0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.385+0000 7f6a5b400640 1 -- 172.21.15.32:0/2323086884 <== osd.10 v2:172.21.15.73:6824/2996715410 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a5401ee50 con 0x7f6a480be900 2024-08-31T07:10:47.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.401+0000 7f6a58c00640 1 -- 172.21.15.32:0/2323086884 --> [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6a64063d30 con 0x7f6a480be900 2024-08-31T07:10:47.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.402+0000 7f6a5b400640 1 -- 172.21.15.32:0/2323086884 <== osd.10 v2:172.21.15.73:6824/2996715410 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6a64063d30 con 0x7f6a480be900 2024-08-31T07:10:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.402+0000 7f6a58c00640 1 -- 172.21.15.32:0/2323086884 >> [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] conn(0x7f6a480be900 msgr2=0x7f6a480c0df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.402+0000 7f6a58c00640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] conn(0x7f6a480be900 0x7f6a480c0df0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.403+0000 7f6a58c00640 1 -- 172.21.15.32:0/2323086884 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6a480b4c10 msgr2=0x7f6a480b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.403+0000 7f6a58c00640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6a480b4c10 0x7f6a480b7120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005f70 tx=0x7f6a4c005e80 comp rx=0 tx=0).stop 2024-08-31T07:10:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.403+0000 7f6a58c00640 1 -- 172.21.15.32:0/2323086884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6410a230 msgr2=0x7f6a641a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.403+0000 7f6a58c00640 1 --2- 172.21.15.32:0/2323086884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6410a230 0x7f6a641a0630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400e9c0 tx=0x7f6a5400ee80 comp rx=0 tx=0).stop 2024-08-31T07:10:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.403+0000 7f6a63400640 1 -- 172.21.15.32:0/2323086884 reap_dead start 2024-08-31T07:10:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.403+0000 7f6a58c00640 1 -- 172.21.15.32:0/2323086884 shutdown_connections 2024-08-31T07:10:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.403+0000 7f6a58c00640 1 -- 172.21.15.32:0/2323086884 >> 172.21.15.32:0/2323086884 conn(0x7f6a640791c0 msgr2=0x7f6a6410dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.403+0000 7f6a58c00640 1 -- 172.21.15.32:0/2323086884 shutdown_connections 2024-08-31T07:10:47.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:47.403+0000 7f6a58c00640 1 -- 172.21.15.32:0/2323086884 wait complete. 2024-08-31T07:10:47.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:47 smithi073 ceph-mon[21995]: osdmap e80: 12 total, 12 up, 12 in 2024-08-31T07:10:47.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:47 smithi084 ceph-mon[22387]: osdmap e80: 12 total, 12 up, 12 in 2024-08-31T07:10:47.908 INFO:teuthology.orchestra.run.smithi032.stdout:188978561074 2024-08-31T07:10:47.908 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-31T07:10:48.263 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:48 smithi032 ceph-mon[17252]: pgmap v309: 97 pgs: 1 peering, 96 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:48.303 INFO:teuthology.orchestra.run.smithi032.stdout:81604378698 2024-08-31T07:10:48.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-31T07:10:48.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:48 smithi073 ceph-mon[21995]: pgmap v309: 97 pgs: 1 peering, 96 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:48.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:48 smithi084 ceph-mon[22387]: pgmap v309: 97 pgs: 1 peering, 96 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.966+0000 7f4375c00640 1 -- 172.21.15.32:0/743409860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4370075ea0 msgr2=0x7f437010f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.966+0000 7f4375c00640 1 --2- 172.21.15.32:0/743409860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4370075ea0 0x7f437010f770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f435c009900 tx=0x7f435c02a410 comp rx=0 tx=0).stop 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.966+0000 7f4375c00640 1 -- 172.21.15.32:0/743409860 shutdown_connections 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.966+0000 7f4375c00640 1 --2- 172.21.15.32:0/743409860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f437010fcb0 0x7f43701120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.966+0000 7f4375c00640 1 --2- 172.21.15.32:0/743409860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4370075ea0 0x7f437010f770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.966+0000 7f4375c00640 1 --2- 172.21.15.32:0/743409860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4370075550 0x7f4370075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.966+0000 7f4375c00640 1 -- 172.21.15.32:0/743409860 >> 172.21.15.32:0/743409860 conn(0x7f43700ff1e0 msgr2=0x7f4370101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.966+0000 7f4375c00640 1 -- 172.21.15.32:0/743409860 shutdown_connections 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.966+0000 7f4375c00640 1 -- 172.21.15.32:0/743409860 wait complete. 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.968+0000 7f4375c00640 1 Processor -- start 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.968+0000 7f4375c00640 1 -- start start 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.968+0000 7f4375c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4370075550 0x7f437019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.968+0000 7f4375c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4370075ea0 0x7f437019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.968+0000 7f4375c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f437010fcb0 0x7f43701a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.968+0000 7f4375c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4370115860 con 0x7f437010fcb0 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.968+0000 7f4375c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f43701156e0 con 0x7f4370075ea0 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.968+0000 7f4375c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f43701159e0 con 0x7f4370075550 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f4375200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f437010fcb0 0x7f43701a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f4375200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f437010fcb0 0x7f43701a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42502/0 (socket says 172.21.15.32:42502) 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f4375200640 1 -- 172.21.15.32:0/2742452381 learned_addr learned my addr 172.21.15.32:0/2742452381 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f436fe00640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4370075550 0x7f437019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f436f400640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4370075ea0 0x7f437019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f4375200640 1 -- 172.21.15.32:0/2742452381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4370075550 msgr2=0x7f437019c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f4375200640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4370075550 0x7f437019c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f4375200640 1 -- 172.21.15.32:0/2742452381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4370075ea0 msgr2=0x7f437019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f4375200640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4370075ea0 0x7f437019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f4375200640 1 -- 172.21.15.32:0/2742452381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43701a2200 con 0x7f437010fcb0 2024-08-31T07:10:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.969+0000 7f4375200640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f437010fcb0 0x7f43701a1ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f436800eaa0 tx=0x7f436800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.970+0000 7f436cc00640 1 -- 172.21.15.32:0/2742452381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436800cdb0 con 0x7f437010fcb0 2024-08-31T07:10:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.970+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43701a24f0 con 0x7f437010fcb0 2024-08-31T07:10:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.970+0000 7f436cc00640 1 -- 172.21.15.32:0/2742452381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4368005020 con 0x7f437010fcb0 2024-08-31T07:10:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.970+0000 7f436cc00640 1 -- 172.21.15.32:0/2742452381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680116d0 con 0x7f437010fcb0 2024-08-31T07:10:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.970+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43701a8c50 con 0x7f437010fcb0 2024-08-31T07:10:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.971+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43701091a0 con 0x7f437010fcb0 2024-08-31T07:10:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.974+0000 7f436cc00640 1 -- 172.21.15.32:0/2742452381 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f43680051d0 con 0x7f437010fcb0 2024-08-31T07:10:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.975+0000 7f436cc00640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43540b4c10 0x7f43540b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.975+0000 7f436cc00640 1 -- 172.21.15.32:0/2742452381 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8397+0+0 (secure 0 0 0) 0x7f43680e5970 con 0x7f437010fcb0 2024-08-31T07:10:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.975+0000 7f436fe00640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43540b4c10 0x7f43540b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.975+0000 7f436cc00640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] conn(0x7f43540be900 0x7f43540c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.975+0000 7f436cc00640 1 -- 172.21.15.32:0/2742452381 --> [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4368014ec0 con 0x7f43540be900 2024-08-31T07:10:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.975+0000 7f436cc00640 1 -- 172.21.15.32:0/2742452381 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f43680e5d20 con 0x7f437010fcb0 2024-08-31T07:10:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.976+0000 7f436f400640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] conn(0x7f43540be900 0x7f43540c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.976+0000 7f436fe00640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43540b4c10 0x7f43540b7120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f436000bbe0 tx=0x7f436000b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.976+0000 7f436f400640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] conn(0x7f43540be900 0x7f43540c0df0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.978+0000 7f436cc00640 1 -- 172.21.15.32:0/2742452381 <== osd.7 v2:172.21.15.73:6816/2091730330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4368014ec0 con 0x7f43540be900 2024-08-31T07:10:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.992+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 --> [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4370076a60 con 0x7f43540be900 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.993+0000 7f436cc00640 1 -- 172.21.15.32:0/2742452381 <== osd.7 v2:172.21.15.73:6816/2091730330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4370076a60 con 0x7f43540be900 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.993+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 >> [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] conn(0x7f43540be900 msgr2=0x7f43540c0df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.993+0000 7f4375c00640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] conn(0x7f43540be900 0x7f43540c0df0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.993+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43540b4c10 msgr2=0x7f43540b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.993+0000 7f4375c00640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43540b4c10 0x7f43540b7120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f436000bbe0 tx=0x7f436000b390 comp rx=0 tx=0).stop 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.993+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f437010fcb0 msgr2=0x7f43701a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.993+0000 7f4375c00640 1 --2- 172.21.15.32:0/2742452381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f437010fcb0 0x7f43701a1ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f436800eaa0 tx=0x7f436800ef60 comp rx=0 tx=0).stop 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.993+0000 7f4375200640 1 -- 172.21.15.32:0/2742452381 reap_dead start 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.994+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 shutdown_connections 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.994+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 >> 172.21.15.32:0/2742452381 conn(0x7f43700ff1e0 msgr2=0x7f4370108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.994+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 shutdown_connections 2024-08-31T07:10:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:45.994+0000 7f4375c00640 1 -- 172.21.15.32:0/2742452381 wait complete. 2024-08-31T07:10:50.293 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:50 smithi084 ceph-mon[22387]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:50.299 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:50 smithi032 ceph-mon[17252]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:50.302 INFO:teuthology.orchestra.run.smithi032.stdout:115964117060 2024-08-31T07:10:50.302 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-31T07:10:50.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:50 smithi073 ceph-mon[21995]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:50.749 INFO:teuthology.orchestra.run.smithi032.stdout:60129542224 2024-08-31T07:10:50.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-31T07:10:51.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.020+0000 7fe33ea00640 1 -- 172.21.15.32:0/1680594081 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3300318a0 con 0x7fe340077120 2024-08-31T07:10:51.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.021+0000 7fe346e00640 1 -- 172.21.15.32:0/1680594081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe340077120 msgr2=0x7fe340075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:51.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.021+0000 7fe346e00640 1 --2- 172.21.15.32:0/1680594081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe340077120 0x7fe340075550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe330009900 tx=0x7fe33002a410 comp rx=0 tx=0).stop 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.022+0000 7fe346e00640 1 -- 172.21.15.32:0/1680594081 shutdown_connections 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.022+0000 7fe346e00640 1 --2- 172.21.15.32:0/1680594081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3401141e0 0x7fe3401166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.022+0000 7fe346e00640 1 --2- 172.21.15.32:0/1680594081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe340075b70 0x7fe340075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.022+0000 7fe346e00640 1 --2- 172.21.15.32:0/1680594081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe340077120 0x7fe340075550 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.022+0000 7fe346e00640 1 -- 172.21.15.32:0/1680594081 >> 172.21.15.32:0/1680594081 conn(0x7fe3400ff380 msgr2=0x7fe3401017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.022+0000 7fe346e00640 1 -- 172.21.15.32:0/1680594081 shutdown_connections 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.022+0000 7fe346e00640 1 -- 172.21.15.32:0/1680594081 wait complete. 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.023+0000 7fe346e00640 1 Processor -- start 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.023+0000 7fe346e00640 1 -- start start 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.023+0000 7fe346e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe340075b70 0x7fe3401a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.023+0000 7fe346e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe340077120 0x7fe3401a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.023+0000 7fe346e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3401141e0 0x7fe3401a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe346e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe340119e70 con 0x7fe340075b70 2024-08-31T07:10:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe346e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe340119cf0 con 0x7fe340077120 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe346e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe340119ff0 con 0x7fe3401141e0 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe33fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe340075b70 0x7fe3401a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe344e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3401141e0 0x7fe3401a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe33fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe340075b70 0x7fe3401a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42550/0 (socket says 172.21.15.32:42550) 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe33f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe340077120 0x7fe3401a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe33fe00640 1 -- 172.21.15.32:0/3739654280 learned_addr learned my addr 172.21.15.32:0/3739654280 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe33fe00640 1 -- 172.21.15.32:0/3739654280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3401141e0 msgr2=0x7fe3401a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe33fe00640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3401141e0 0x7fe3401a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe33fe00640 1 -- 172.21.15.32:0/3739654280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe340077120 msgr2=0x7fe3401a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe33fe00640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe340077120 0x7fe3401a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.024+0000 7fe33fe00640 1 -- 172.21.15.32:0/3739654280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3401a6790 con 0x7fe340075b70 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.025+0000 7fe33fe00640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe340075b70 0x7fe3401a0820 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe330005940 tx=0x7fe330004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.025+0000 7fe33cc00640 1 -- 172.21.15.32:0/3739654280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33003f030 con 0x7fe340075b70 2024-08-31T07:10:51.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.025+0000 7fe33cc00640 1 -- 172.21.15.32:0/3739654280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe330003aa0 con 0x7fe340075b70 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.025+0000 7fe33cc00640 1 -- 172.21.15.32:0/3739654280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330035070 con 0x7fe340075b70 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.025+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3401a6a20 con 0x7fe340075b70 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.025+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3401ad1e0 con 0x7fe340075b70 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.026+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe3400638f0 con 0x7fe340075b70 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.029+0000 7fe33cc00640 1 -- 172.21.15.32:0/3739654280 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fe330042030 con 0x7fe340075b70 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.030+0000 7fe33cc00640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3240b4c90 0x7fe3240b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.030+0000 7fe33cc00640 1 -- 172.21.15.32:0/3739654280 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe330105ff0 con 0x7fe340075b70 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.030+0000 7fe33f400640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3240b4c90 0x7fe3240b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.031+0000 7fe33cc00640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] conn(0x7fe3240be970 0x7fe3240c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.031+0000 7fe33cc00640 1 -- 172.21.15.32:0/3739654280 --> [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3240c1530 con 0x7fe3240be970 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.031+0000 7fe33cc00640 1 -- 172.21.15.32:0/3739654280 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fe330106420 con 0x7fe340075b70 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.031+0000 7fe344e00640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] conn(0x7fe3240be970 0x7fe3240c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:51.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.031+0000 7fe33f400640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3240b4c90 0x7fe3240b71a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe340116110 tx=0x7fe32c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:51.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.031+0000 7fe344e00640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] conn(0x7fe3240be970 0x7fe3240c0e60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:51.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.032+0000 7fe33cc00640 1 -- 172.21.15.32:0/3739654280 <== osd.8 v2:172.21.15.84:6816/2957423402 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3240c1530 con 0x7fe3240be970 2024-08-31T07:10:51.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.050+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 --> [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe340063d70 con 0x7fe3240be970 2024-08-31T07:10:51.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.050+0000 7fe33cc00640 1 -- 172.21.15.32:0/3739654280 <== osd.8 v2:172.21.15.84:6816/2957423402 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe340063d70 con 0x7fe3240be970 2024-08-31T07:10:51.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.051+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 >> [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] conn(0x7fe3240be970 msgr2=0x7fe3240c0e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:51.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.051+0000 7fe346e00640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] conn(0x7fe3240be970 0x7fe3240c0e60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:51.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.052+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3240b4c90 msgr2=0x7fe3240b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:51.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.052+0000 7fe346e00640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3240b4c90 0x7fe3240b71a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe340116110 tx=0x7fe32c005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:51.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.052+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe340075b70 msgr2=0x7fe3401a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:51.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.052+0000 7fe346e00640 1 --2- 172.21.15.32:0/3739654280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe340075b70 0x7fe3401a0820 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe330005940 tx=0x7fe330004270 comp rx=0 tx=0).stop 2024-08-31T07:10:51.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.052+0000 7fe344e00640 1 -- 172.21.15.32:0/3739654280 reap_dead start 2024-08-31T07:10:51.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.053+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 shutdown_connections 2024-08-31T07:10:51.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.053+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 >> 172.21.15.32:0/3739654280 conn(0x7fe3400ff380 msgr2=0x7fe3401100e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:51.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.053+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 shutdown_connections 2024-08-31T07:10:51.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:51.053+0000 7fe346e00640 1 -- 172.21.15.32:0/3739654280 wait complete. 2024-08-31T07:10:51.518 INFO:teuthology.orchestra.run.smithi032.stdout:210453397551 2024-08-31T07:10:51.518 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-31T07:10:51.873 INFO:teuthology.orchestra.run.smithi032.stdout:304942678041 2024-08-31T07:10:51.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-31T07:10:52.207 INFO:teuthology.orchestra.run.smithi032.stdout:279172874271 2024-08-31T07:10:52.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-31T07:10:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.499+0000 7f3170e00640 1 -- 172.21.15.32:0/3001608536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f316c103920 msgr2=0x7f316c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.499+0000 7f3170e00640 1 --2- 172.21.15.32:0/3001608536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f316c103920 0x7f316c103d30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f315c009900 tx=0x7f315c02a410 comp rx=0 tx=0).stop 2024-08-31T07:10:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.499+0000 7f3170e00640 1 -- 172.21.15.32:0/3001608536 shutdown_connections 2024-08-31T07:10:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.499+0000 7f3170e00640 1 --2- 172.21.15.32:0/3001608536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f316c109bb0 0x7f316c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.499+0000 7f3170e00640 1 --2- 172.21.15.32:0/3001608536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f316c109260 0x7f316c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.499+0000 7f3170e00640 1 --2- 172.21.15.32:0/3001608536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f316c103920 0x7f316c103d30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.499+0000 7f3170e00640 1 -- 172.21.15.32:0/3001608536 >> 172.21.15.32:0/3001608536 conn(0x7f316c0ff330 msgr2=0x7f316c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.499+0000 7f3170e00640 1 -- 172.21.15.32:0/3001608536 shutdown_connections 2024-08-31T07:10:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.499+0000 7f3170e00640 1 -- 172.21.15.32:0/3001608536 wait complete. 2024-08-31T07:10:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.501+0000 7f3170e00640 1 Processor -- start 2024-08-31T07:10:52.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.501+0000 7f3170e00640 1 -- start start 2024-08-31T07:10:52.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.501+0000 7f3170e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f316c103920 0x7f316c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:52.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.501+0000 7f3170e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f316c109260 0x7f316c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:52.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.501+0000 7f3170e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f316c109bb0 0x7f316c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:52.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.501+0000 7f3170e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f316c113870 con 0x7f316c109bb0 2024-08-31T07:10:52.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.501+0000 7f3170e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f316c1136f0 con 0x7f316c103920 2024-08-31T07:10:52.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.501+0000 7f3170e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f316c1139f0 con 0x7f316c109260 2024-08-31T07:10:52.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.502+0000 7f3169e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f316c103920 0x7f316c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.502+0000 7f3169e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f316c103920 0x7f316c0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34486/0 (socket says 172.21.15.32:34486) 2024-08-31T07:10:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.502+0000 7f3169e00640 1 -- 172.21.15.32:0/3609546417 learned_addr learned my addr 172.21.15.32:0/3609546417 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.502+0000 7f3169400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f316c109260 0x7f316c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.502+0000 7f316a800640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f316c109bb0 0x7f316c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.502+0000 7f3169e00640 1 -- 172.21.15.32:0/3609546417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f316c109260 msgr2=0x7f316c0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.502+0000 7f3169e00640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f316c109260 0x7f316c0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.502+0000 7f3169e00640 1 -- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f316c109bb0 msgr2=0x7f316c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.502+0000 7f3169e00640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f316c109bb0 0x7f316c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.502+0000 7f3169e00640 1 -- 172.21.15.32:0/3609546417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c1a8500 con 0x7f316c103920 2024-08-31T07:10:52.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.503+0000 7f3169e00640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f316c103920 0x7f316c0754b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f315c005940 tx=0x7f315c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:52.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.503+0000 7f3162a00640 1 -- 172.21.15.32:0/3609546417 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c03f030 con 0x7f316c103920 2024-08-31T07:10:52.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.503+0000 7f3162a00640 1 -- 172.21.15.32:0/3609546417 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f315c0044c0 con 0x7f316c103920 2024-08-31T07:10:52.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.503+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c1a8790 con 0x7f316c103920 2024-08-31T07:10:52.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.503+0000 7f3162a00640 1 -- 172.21.15.32:0/3609546417 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c035070 con 0x7f316c103920 2024-08-31T07:10:52.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.503+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c1a8c70 con 0x7f316c103920 2024-08-31T07:10:52.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.504+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f316c0638b0 con 0x7f316c103920 2024-08-31T07:10:52.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.507+0000 7f3162a00640 1 -- 172.21.15.32:0/3609546417 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f315c042030 con 0x7f316c103920 2024-08-31T07:10:52.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.509+0000 7f3162a00640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f31500b4c10 0x7f31500b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:52.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.509+0000 7f3162a00640 1 -- 172.21.15.32:0/3609546417 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f315c105fc0 con 0x7f316c103920 2024-08-31T07:10:52.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.509+0000 7f3169400640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f31500b4c10 0x7f31500b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:52.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.509+0000 7f3162a00640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] conn(0x7f31500be900 0x7f31500c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:52.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.509+0000 7f3162a00640 1 -- 172.21.15.32:0/3609546417 --> [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f315c03ce50 con 0x7f31500be900 2024-08-31T07:10:52.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.509+0000 7f3162a00640 1 -- 172.21.15.32:0/3609546417 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f315c106370 con 0x7f316c103920 2024-08-31T07:10:52.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.510+0000 7f316a800640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] conn(0x7f31500be900 0x7f31500c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:52.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.510+0000 7f3169400640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f31500b4c10 0x7f31500b7120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f316c101b00 tx=0x7f3158005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:52.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.511+0000 7f316a800640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] conn(0x7f31500be900 0x7f31500c0df0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.512+0000 7f3162a00640 1 -- 172.21.15.32:0/3609546417 <== osd.9 v2:172.21.15.32:6826/2828083479 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f315c03ce50 con 0x7f31500be900 2024-08-31T07:10:52.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.527+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 --> [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f316c063d30 con 0x7f31500be900 2024-08-31T07:10:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.527+0000 7f3162a00640 1 -- 172.21.15.32:0/3609546417 <== osd.9 v2:172.21.15.32:6826/2828083479 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f316c063d30 con 0x7f31500be900 2024-08-31T07:10:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] conn(0x7f31500be900 msgr2=0x7f31500c0df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f3170e00640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] conn(0x7f31500be900 0x7f31500c0df0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f31500b4c10 msgr2=0x7f31500b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f3170e00640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f31500b4c10 0x7f31500b7120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f316c101b00 tx=0x7f3158005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f316c103920 msgr2=0x7f316c0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f3170e00640 1 --2- 172.21.15.32:0/3609546417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f316c103920 0x7f316c0754b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f315c005940 tx=0x7f315c0040c0 comp rx=0 tx=0).stop 2024-08-31T07:10:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f316a800640 1 -- 172.21.15.32:0/3609546417 reap_dead start 2024-08-31T07:10:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 shutdown_connections 2024-08-31T07:10:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 >> 172.21.15.32:0/3609546417 conn(0x7f316c0ff330 msgr2=0x7f316c1027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 shutdown_connections 2024-08-31T07:10:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:52.528+0000 7f3170e00640 1 -- 172.21.15.32:0/3609546417 wait complete. 2024-08-31T07:10:52.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:52 smithi032 ceph-mon[17252]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:52 smithi073 ceph-mon[21995]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:52 smithi084 ceph-mon[22387]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:53.322 INFO:teuthology.orchestra.run.smithi032.stdout:257698037797 2024-08-31T07:10:53.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-31T07:10:53.824 INFO:teuthology.orchestra.run.smithi032.stdout:231928234028 2024-08-31T07:10:53.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-31T07:10:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:53.927 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:53.928 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:53.929 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:53.942 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:54.080 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:54.083 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:54.109 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:10:54.695 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:54 smithi032 ceph-mon[17252]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:54.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.878+0000 7f4197200640 1 -- 172.21.15.32:0/2637699504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f419810b2a0 msgr2=0x7f419810b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.878+0000 7f4197200640 1 --2- 172.21.15.32:0/2637699504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f419810b2a0 0x7f419810b6b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4184009860 tx=0x7f418402a340 comp rx=0 tx=0).stop 2024-08-31T07:10:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.878+0000 7f4197200640 1 -- 172.21.15.32:0/2637699504 shutdown_connections 2024-08-31T07:10:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.878+0000 7f4197200640 1 --2- 172.21.15.32:0/2637699504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419810bbf0 0x7f4198112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.878+0000 7f4197200640 1 --2- 172.21.15.32:0/2637699504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f419810b2a0 0x7f419810b6b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.878+0000 7f4197200640 1 --2- 172.21.15.32:0/2637699504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4198105960 0x7f4198105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.878+0000 7f4197200640 1 -- 172.21.15.32:0/2637699504 >> 172.21.15.32:0/2637699504 conn(0x7f4198101410 msgr2=0x7f4198103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.878+0000 7f4197200640 1 -- 172.21.15.32:0/2637699504 shutdown_connections 2024-08-31T07:10:54.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.878+0000 7f4197200640 1 -- 172.21.15.32:0/2637699504 wait complete. 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.879+0000 7f4197200640 1 Processor -- start 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4197200640 1 -- start start 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4197200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198105960 0x7f419819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4197200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419810b2a0 0x7f419819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4197200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419810bbf0 0x7f41981a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4197200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f41981158b0 con 0x7f419810b2a0 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4197200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4198115730 con 0x7f4198105960 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4197200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4198115a30 con 0x7f419810bbf0 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4195400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419810b2a0 0x7f419819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4195400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419810b2a0 0x7f419819c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42580/0 (socket says 172.21.15.32:42580) 2024-08-31T07:10:54.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4195400640 1 -- 172.21.15.32:0/658812267 learned_addr learned my addr 172.21.15.32:0/658812267 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.880+0000 7f4195e00640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198105960 0x7f419819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.881+0000 7f4196800640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419810bbf0 0x7f41981a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.881+0000 7f4195e00640 1 -- 172.21.15.32:0/658812267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419810bbf0 msgr2=0x7f41981a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.881+0000 7f4195e00640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419810bbf0 0x7f41981a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.881+0000 7f4195e00640 1 -- 172.21.15.32:0/658812267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419810b2a0 msgr2=0x7f419819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.881+0000 7f4195e00640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419810b2a0 0x7f419819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.881+0000 7f4195e00640 1 -- 172.21.15.32:0/658812267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41981a21f0 con 0x7f4198105960 2024-08-31T07:10:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.881+0000 7f4195400640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419810b2a0 0x7f419819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.881+0000 7f4196800640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419810bbf0 0x7f41981a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:10:54.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.881+0000 7f4195e00640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198105960 0x7f419819c280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f418800e9f0 tx=0x7f418800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.881+0000 7f418ea00640 1 -- 172.21.15.32:0/658812267 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418800ccc0 con 0x7f4198105960 2024-08-31T07:10:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.882+0000 7f4197200640 1 -- 172.21.15.32:0/658812267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41981a24e0 con 0x7f4198105960 2024-08-31T07:10:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.882+0000 7f418ea00640 1 -- 172.21.15.32:0/658812267 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41880108e0 con 0x7f4198105960 2024-08-31T07:10:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.882+0000 7f418ea00640 1 -- 172.21.15.32:0/658812267 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188018880 con 0x7f4198105960 2024-08-31T07:10:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.882+0000 7f4197200640 1 -- 172.21.15.32:0/658812267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41981a8c40 con 0x7f4198105960 2024-08-31T07:10:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.883+0000 7f4197200640 1 -- 172.21.15.32:0/658812267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4198106be0 con 0x7f4198105960 2024-08-31T07:10:54.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.885+0000 7f418ea00640 1 -- 172.21.15.32:0/658812267 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f4188010430 con 0x7f4198105960 2024-08-31T07:10:54.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.887+0000 7f418ea00640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f417c0b4c10 0x7f417c0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:54.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.887+0000 7f418ea00640 1 -- 172.21.15.32:0/658812267 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f41880e5800 con 0x7f4198105960 2024-08-31T07:10:54.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.887+0000 7f4195400640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f417c0b4c10 0x7f417c0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:54.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.888+0000 7f4195400640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f417c0b4c10 0x7f417c0b7120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4184000c00 tx=0x7f4184030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:54.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:54.889+0000 7f418ea00640 1 -- 172.21.15.32:0/658812267 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f41880ac6d0 con 0x7f4198105960 2024-08-31T07:10:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:54 smithi073 ceph-mon[21995]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:54 smithi084 ceph-mon[22387]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-08-31T07:10:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.104+0000 7f4197200640 1 -- 172.21.15.32:0/658812267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f419810d9c0 con 0x7f4198105960 2024-08-31T07:10:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.105+0000 7f418ea00640 1 -- 172.21.15.32:0/658812267 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f41880abfd0 con 0x7f4198105960 2024-08-31T07:10:55.106 INFO:teuthology.orchestra.run.smithi032.stdout:115964117062 2024-08-31T07:10:55.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.107+0000 7f4197200640 1 -- 172.21.15.32:0/658812267 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f417c0b4c10 msgr2=0x7f417c0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:55.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.107+0000 7f4197200640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f417c0b4c10 0x7f417c0b7120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4184000c00 tx=0x7f4184030340 comp rx=0 tx=0).stop 2024-08-31T07:10:55.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.107+0000 7f4197200640 1 -- 172.21.15.32:0/658812267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198105960 msgr2=0x7f419819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.107+0000 7f4197200640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198105960 0x7f419819c280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f418800e9f0 tx=0x7f418800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.107+0000 7f4197200640 1 -- 172.21.15.32:0/658812267 shutdown_connections 2024-08-31T07:10:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.108+0000 7f4197200640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f417c0b4c10 0x7f417c0b7120 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.108+0000 7f4197200640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f419810bbf0 0x7f41981a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.108+0000 7f4197200640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419810b2a0 0x7f419819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.108+0000 7f4197200640 1 --2- 172.21.15.32:0/658812267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198105960 0x7f419819c280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.108+0000 7f4197200640 1 -- 172.21.15.32:0/658812267 >> 172.21.15.32:0/658812267 conn(0x7f4198101410 msgr2=0x7f4198103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.108+0000 7f4197200640 1 -- 172.21.15.32:0/658812267 shutdown_connections 2024-08-31T07:10:55.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.108+0000 7f4197200640 1 -- 172.21.15.32:0/658812267 wait complete. 2024-08-31T07:10:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.501+0000 7effd7200640 1 -- 172.21.15.32:0/2970487901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effd8105960 msgr2=0x7effd8105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.501+0000 7effd7200640 1 --2- 172.21.15.32:0/2970487901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effd8105960 0x7effd8105d70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7effc8009900 tx=0x7effc802a410 comp rx=0 tx=0).stop 2024-08-31T07:10:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.502+0000 7effd7200640 1 -- 172.21.15.32:0/2970487901 shutdown_connections 2024-08-31T07:10:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.502+0000 7effd7200640 1 --2- 172.21.15.32:0/2970487901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd810bbf0 0x7effd8112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.502+0000 7effd7200640 1 --2- 172.21.15.32:0/2970487901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effd810b2a0 0x7effd810b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.502+0000 7effd7200640 1 --2- 172.21.15.32:0/2970487901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effd8105960 0x7effd8105d70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.502+0000 7effd7200640 1 -- 172.21.15.32:0/2970487901 >> 172.21.15.32:0/2970487901 conn(0x7effd8101410 msgr2=0x7effd8103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.502+0000 7effd7200640 1 -- 172.21.15.32:0/2970487901 shutdown_connections 2024-08-31T07:10:55.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.502+0000 7effd7200640 1 -- 172.21.15.32:0/2970487901 wait complete. 2024-08-31T07:10:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.503+0000 7effd7200640 1 Processor -- start 2024-08-31T07:10:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.503+0000 7effd7200640 1 -- start start 2024-08-31T07:10:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effd8105960 0x7effd819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd810b2a0 0x7effd819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd7200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effd810bbf0 0x7effd81a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7effd8115900 con 0x7effd810b2a0 2024-08-31T07:10:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7effd8115780 con 0x7effd8105960 2024-08-31T07:10:55.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd7200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7effd8115a80 con 0x7effd810bbf0 2024-08-31T07:10:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effd8105960 0x7effd819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effd8105960 0x7effd819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34524/0 (socket says 172.21.15.32:34524) 2024-08-31T07:10:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd5e00640 1 -- 172.21.15.32:0/3149736577 learned_addr learned my addr 172.21.15.32:0/3149736577 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd5400640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd810b2a0 0x7effd819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.504+0000 7effd6800640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effd810bbf0 0x7effd81a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.505+0000 7effd5e00640 1 -- 172.21.15.32:0/3149736577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effd810bbf0 msgr2=0x7effd81a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.505+0000 7effd5e00640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effd810bbf0 0x7effd81a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.505+0000 7effd5e00640 1 -- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd810b2a0 msgr2=0x7effd819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.505+0000 7effd5e00640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd810b2a0 0x7effd819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.505+0000 7effd5e00640 1 -- 172.21.15.32:0/3149736577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd81a2220 con 0x7effd8105960 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.505+0000 7effd5400640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd810b2a0 0x7effd819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.505+0000 7effd5e00640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effd8105960 0x7effd819c2b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7effc8005ab0 tx=0x7effc8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.505+0000 7effcea00640 1 -- 172.21.15.32:0/3149736577 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc803f070 con 0x7effd8105960 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.506+0000 7effd7200640 1 -- 172.21.15.32:0/3149736577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd81a24b0 con 0x7effd8105960 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.506+0000 7effcea00640 1 -- 172.21.15.32:0/3149736577 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc8031b60 con 0x7effd8105960 2024-08-31T07:10:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.506+0000 7effcea00640 1 -- 172.21.15.32:0/3149736577 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8035030 con 0x7effd8105960 2024-08-31T07:10:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.506+0000 7effd7200640 1 -- 172.21.15.32:0/3149736577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd81a8c70 con 0x7effd8105960 2024-08-31T07:10:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.507+0000 7effd7200640 1 -- 172.21.15.32:0/3149736577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd8105d70 con 0x7effd8105960 2024-08-31T07:10:55.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.509+0000 7effcea00640 1 -- 172.21.15.32:0/3149736577 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7effc8042070 con 0x7effd8105960 2024-08-31T07:10:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.511+0000 7effcea00640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effbc0b4c90 0x7effbc0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.511+0000 7effcea00640 1 -- 172.21.15.32:0/3149736577 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7effc8106100 con 0x7effd8105960 2024-08-31T07:10:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.511+0000 7effd5400640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effbc0b4c90 0x7effbc0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.512+0000 7effd5400640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effbc0b4c90 0x7effbc0b71a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7effd8104cc0 tx=0x7effc4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:55.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.513+0000 7effcea00640 1 -- 172.21.15.32:0/3149736577 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7effc80cc980 con 0x7effd8105960 2024-08-31T07:10:55.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.714+0000 7effd7200640 1 -- 172.21.15.32:0/3149736577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7effd819d3e0 con 0x7effd8105960 2024-08-31T07:10:55.716 INFO:teuthology.orchestra.run.smithi032.stdout:81604378701 2024-08-31T07:10:55.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.715+0000 7effcea00640 1 -- 172.21.15.32:0/3149736577 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7effd819d3e0 con 0x7effd8105960 2024-08-31T07:10:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 -- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effbc0b4c90 msgr2=0x7effbc0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effbc0b4c90 0x7effbc0b71a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7effd8104cc0 tx=0x7effc4005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 -- 172.21.15.32:0/3149736577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effd8105960 msgr2=0x7effd819c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effd8105960 0x7effd819c2b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7effc8005ab0 tx=0x7effc8003c10 comp rx=0 tx=0).stop 2024-08-31T07:10:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 -- 172.21.15.32:0/3149736577 shutdown_connections 2024-08-31T07:10:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effbc0b4c90 0x7effbc0b71a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effd810bbf0 0x7effd81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd810b2a0 0x7effd819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 --2- 172.21.15.32:0/3149736577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effd8105960 0x7effd819c2b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:55.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 -- 172.21.15.32:0/3149736577 >> 172.21.15.32:0/3149736577 conn(0x7effd8101410 msgr2=0x7effd810a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:55.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 -- 172.21.15.32:0/3149736577 shutdown_connections 2024-08-31T07:10:55.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.718+0000 7effd7200640 1 -- 172.21.15.32:0/3149736577 wait complete. 2024-08-31T07:10:55.795 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:55 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/658812267' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-31T07:10:55.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:55 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/658812267' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-31T07:10:55.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:55 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/658812267' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-31T07:10:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.794+0000 7f67cf000640 1 -- 172.21.15.32:0/135670803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67c8109070 msgr2=0x7f67c8109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.794+0000 7f67cf000640 1 --2- 172.21.15.32:0/135670803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67c8109070 0x7f67c8109480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009900 tx=0x7f67b402a410 comp rx=0 tx=0).stop 2024-08-31T07:10:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.794+0000 7f67cf000640 1 -- 172.21.15.32:0/135670803 shutdown_connections 2024-08-31T07:10:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.794+0000 7f67cf000640 1 --2- 172.21.15.32:0/135670803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67c81099c0 0x7f67c81101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.794+0000 7f67cf000640 1 --2- 172.21.15.32:0/135670803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67c8109070 0x7f67c8109480 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.794+0000 7f67cf000640 1 --2- 172.21.15.32:0/135670803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67c8103730 0x7f67c8103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.794+0000 7f67cf000640 1 -- 172.21.15.32:0/135670803 >> 172.21.15.32:0/135670803 conn(0x7f67c80ff180 msgr2=0x7f67c81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.794+0000 7f67cf000640 1 -- 172.21.15.32:0/135670803 shutdown_connections 2024-08-31T07:10:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.794+0000 7f67cf000640 1 -- 172.21.15.32:0/135670803 wait complete. 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.795+0000 7f67cf000640 1 Processor -- start 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cf000640 1 -- start start 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cf000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67c8103730 0x7f67c819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cf000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67c8109070 0x7f67c819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cf000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67c81099c0 0x7f67c81a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cf000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67c8113680 con 0x7f67c8109070 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cf000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f67c8113500 con 0x7f67c8103730 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cf000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67c8113800 con 0x7f67c81099c0 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67c8109070 0x7f67c819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cdc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67c8103730 0x7f67c819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67c8109070 0x7f67c819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42600/0 (socket says 172.21.15.32:42600) 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67cd200640 1 -- 172.21.15.32:0/2699860808 learned_addr learned my addr 172.21.15.32:0/2699860808 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:56.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.796+0000 7f67ce600640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67c81099c0 0x7f67c81a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67cd200640 1 -- 172.21.15.32:0/2699860808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67c81099c0 msgr2=0x7f67c81a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67cd200640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67c81099c0 0x7f67c81a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67cd200640 1 -- 172.21.15.32:0/2699860808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67c8103730 msgr2=0x7f67c819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67cd200640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67c8103730 0x7f67c819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67cd200640 1 -- 172.21.15.32:0/2699860808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c81a2200 con 0x7f67c8109070 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67cdc00640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67c8103730 0x7f67c819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67cd200640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67c8109070 0x7f67c819c830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67b4005ab0 tx=0x7f67b4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67c6000640 1 -- 172.21.15.32:0/2699860808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b403f070 con 0x7f67c8109070 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67c6000640 1 -- 172.21.15.32:0/2699860808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b4031b50 con 0x7f67c8109070 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67c6000640 1 -- 172.21.15.32:0/2699860808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4035030 con 0x7f67c8109070 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.797+0000 7f67cf000640 1 -- 172.21.15.32:0/2699860808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c81a2490 con 0x7f67c8109070 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.798+0000 7f67cf000640 1 -- 172.21.15.32:0/2699860808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c81a8c50 con 0x7f67c8109070 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.799+0000 7f67cf000640 1 -- 172.21.15.32:0/2699860808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c8103b40 con 0x7f67c8109070 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.801+0000 7f67c6000640 1 -- 172.21.15.32:0/2699860808 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f67b4042070 con 0x7f67c8109070 2024-08-31T07:10:56.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.803+0000 7f67c6000640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67b00b4c90 0x7f67b00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.803+0000 7f67c6000640 1 -- 172.21.15.32:0/2699860808 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f67b41055c0 con 0x7f67c8109070 2024-08-31T07:10:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.804+0000 7f67cdc00640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67b00b4c90 0x7f67b00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.804+0000 7f67cdc00640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67b00b4c90 0x7f67b00b71a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0095f0 tx=0x7f67bc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:55.808+0000 7f67c6000640 1 -- 172.21.15.32:0/2699860808 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f67b40cbe40 con 0x7f67c8109070 2024-08-31T07:10:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.011+0000 7f67cf000640 1 -- 172.21.15.32:0/2699860808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f67c81a8e20 con 0x7f67c8109070 2024-08-31T07:10:56.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.011+0000 7f67c6000640 1 -- 172.21.15.32:0/2699860808 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f67b40d0600 con 0x7f67c8109070 2024-08-31T07:10:56.271 INFO:teuthology.orchestra.run.smithi032.stdout:210453397553 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 -- 172.21.15.32:0/2699860808 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67b00b4c90 msgr2=0x7f67b00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67b00b4c90 0x7f67b00b71a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0095f0 tx=0x7f67bc005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 -- 172.21.15.32:0/2699860808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67c8109070 msgr2=0x7f67c819c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67c8109070 0x7f67c819c830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67b4005ab0 tx=0x7f67b4003b90 comp rx=0 tx=0).stop 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 -- 172.21.15.32:0/2699860808 shutdown_connections 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67b00b4c90 0x7f67b00b71a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67c81099c0 0x7f67c81a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67c8109070 0x7f67c819c830 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 --2- 172.21.15.32:0/2699860808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67c8103730 0x7f67c819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 -- 172.21.15.32:0/2699860808 >> 172.21.15.32:0/2699860808 conn(0x7f67c80ff180 msgr2=0x7f67c810e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 -- 172.21.15.32:0/2699860808 shutdown_connections 2024-08-31T07:10:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:56.014+0000 7f67cf000640 1 -- 172.21.15.32:0/2699860808 wait complete. 2024-08-31T07:10:56.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:56 smithi032 ceph-mon[17252]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 66 B/s, 1 keys/s, 0 objects/s recovering 2024-08-31T07:10:56.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:56 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3149736577' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-31T07:10:56.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:56 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2699860808' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-31T07:10:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:56 smithi073 ceph-mon[21995]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 66 B/s, 1 keys/s, 0 objects/s recovering 2024-08-31T07:10:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:56 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3149736577' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-31T07:10:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:56 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2699860808' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-31T07:10:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:56 smithi084 ceph-mon[22387]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 66 B/s, 1 keys/s, 0 objects/s recovering 2024-08-31T07:10:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:56 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3149736577' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-31T07:10:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:56 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2699860808' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-31T07:10:57.761 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:57 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3752925504' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-31T07:10:57.761 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:57 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/919754778' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-31T07:10:57.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:57 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3752925504' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-31T07:10:57.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:57 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/919754778' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-31T07:10:57.995 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:57 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3752925504' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-31T07:10:57.995 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:57 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/919754778' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-31T07:10:58.294 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117060 got 115964117062 for osd.3 2024-08-31T07:10:58.294 DEBUG:teuthology.parallel:result is None 2024-08-31T07:10:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.248+0000 7f3be3e00640 1 -- 172.21.15.32:0/1749247345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3be4077120 msgr2=0x7f3be4075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.248+0000 7f3be3e00640 1 --2- 172.21.15.32:0/1749247345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3be4077120 0x7f3be4075550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009900 tx=0x7f3bd402a410 comp rx=0 tx=0).stop 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.248+0000 7f3be3e00640 1 -- 172.21.15.32:0/1749247345 shutdown_connections 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.248+0000 7f3be3e00640 1 --2- 172.21.15.32:0/1749247345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be4114020 0x7f3be41164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.248+0000 7f3be3e00640 1 --2- 172.21.15.32:0/1749247345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be4075b70 0x7f3be4075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.248+0000 7f3be3e00640 1 --2- 172.21.15.32:0/1749247345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3be4077120 0x7f3be4075550 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.248+0000 7f3be3e00640 1 -- 172.21.15.32:0/1749247345 >> 172.21.15.32:0/1749247345 conn(0x7f3be40ff1e0 msgr2=0x7f3be4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.248+0000 7f3be3e00640 1 -- 172.21.15.32:0/1749247345 shutdown_connections 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.248+0000 7f3be3e00640 1 -- 172.21.15.32:0/1749247345 wait complete. 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.249+0000 7f3be3e00640 1 Processor -- start 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.249+0000 7f3be3e00640 1 -- start start 2024-08-31T07:10:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.250+0000 7f3be3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be4075b70 0x7f3be41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.250+0000 7f3be3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be4077120 0x7f3be41a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.250+0000 7f3be3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3be4114020 0x7f3be41a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.250+0000 7f3be3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4119c60 con 0x7f3be4075b70 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.250+0000 7f3be3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4119ae0 con 0x7f3be4114020 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.250+0000 7f3be3e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4119de0 con 0x7f3be4077120 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.250+0000 7f3be2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be4075b70 0x7f3be41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.250+0000 7f3be2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be4075b70 0x7f3be41a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58346/0 (socket says 172.21.15.32:58346) 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.250+0000 7f3be2a00640 1 -- 172.21.15.32:0/919754778 learned_addr learned my addr 172.21.15.32:0/919754778 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.250+0000 7f3bdbe00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be4077120 0x7f3be41a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3be2a00640 1 -- 172.21.15.32:0/919754778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be4077120 msgr2=0x7f3be41a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3be2a00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be4077120 0x7f3be41a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3be2a00640 1 -- 172.21.15.32:0/919754778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3be4114020 msgr2=0x7f3be41a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3be3400640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3be4114020 0x7f3be41a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3be2a00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3be4114020 0x7f3be41a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3be2a00640 1 -- 172.21.15.32:0/919754778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be41a6600 con 0x7f3be4075b70 2024-08-31T07:10:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3be3400640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3be4114020 0x7f3be41a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:10:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3be2a00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be4075b70 0x7f3be41a0690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4005940 tx=0x7f3bd4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3bdb400640 1 -- 172.21.15.32:0/919754778 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd403f030 con 0x7f3be4075b70 2024-08-31T07:10:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3bdb400640 1 -- 172.21.15.32:0/919754778 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd4003aa0 con 0x7f3be4075b70 2024-08-31T07:10:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3be3e00640 1 -- 172.21.15.32:0/919754778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be41a6890 con 0x7f3be4075b70 2024-08-31T07:10:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.251+0000 7f3bdb400640 1 -- 172.21.15.32:0/919754778 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4035070 con 0x7f3be4075b70 2024-08-31T07:10:58.617 INFO:teuthology.orchestra.run.smithi032.stdout:38654705751 2024-08-31T07:10:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.252+0000 7f3be3e00640 1 -- 172.21.15.32:0/919754778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be41ad050 con 0x7f3be4075b70 2024-08-31T07:10:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.253+0000 7f3be3e00640 1 -- 172.21.15.32:0/919754778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be4075f80 con 0x7f3be4075b70 2024-08-31T07:10:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.255+0000 7f3bdb400640 1 -- 172.21.15.32:0/919754778 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f3bd4042030 con 0x7f3be4075b70 2024-08-31T07:10:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.258+0000 7f3bdb400640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3bc80b4c90 0x7f3bc80b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.258+0000 7f3bdb400640 1 -- 172.21.15.32:0/919754778 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f3bd4105b10 con 0x7f3be4075b70 2024-08-31T07:10:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.258+0000 7f3bdbe00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3bc80b4c90 0x7f3bc80b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:58.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.259+0000 7f3bdbe00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3bc80b4c90 0x7f3bc80b71a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc005fd0 tx=0x7f3bcc005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.262+0000 7f3bdb400640 1 -- 172.21.15.32:0/919754778 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3bd40cc9e0 con 0x7f3be4075b70 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.468+0000 7f3be3e00640 1 -- 172.21.15.32:0/919754778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3be41a17c0 con 0x7f3be4075b70 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.468+0000 7f3bdb400640 1 -- 172.21.15.32:0/919754778 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3bd40cc2e0 con 0x7f3be4075b70 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.470+0000 7f3be3e00640 1 -- 172.21.15.32:0/919754778 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3bc80b4c90 msgr2=0x7f3bc80b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.470+0000 7f3be3e00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3bc80b4c90 0x7f3bc80b71a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc005fd0 tx=0x7f3bcc005850 comp rx=0 tx=0).stop 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.470+0000 7f3be3e00640 1 -- 172.21.15.32:0/919754778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be4075b70 msgr2=0x7f3be41a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.470+0000 7f3be3e00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be4075b70 0x7f3be41a0690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4005940 tx=0x7f3bd4004270 comp rx=0 tx=0).stop 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.470+0000 7f3be3e00640 1 -- 172.21.15.32:0/919754778 shutdown_connections 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.470+0000 7f3be3e00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3bc80b4c90 0x7f3bc80b71a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.470+0000 7f3be3e00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3be4114020 0x7f3be41a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.470+0000 7f3be3e00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be4077120 0x7f3be41a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.470+0000 7f3be3e00640 1 --2- 172.21.15.32:0/919754778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be4075b70 0x7f3be41a0690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.470+0000 7f3be3e00640 1 -- 172.21.15.32:0/919754778 >> 172.21.15.32:0/919754778 conn(0x7f3be40ff1e0 msgr2=0x7f3be410ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:58.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.471+0000 7f3be3e00640 1 -- 172.21.15.32:0/919754778 shutdown_connections 2024-08-31T07:10:58.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.471+0000 7f3be3e00640 1 -- 172.21.15.32:0/919754778 wait complete. 2024-08-31T07:10:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.060+0000 7f5ee2000640 1 -- 172.21.15.32:0/3513616969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5edc0776e0 msgr2=0x7f5edc077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.060+0000 7f5ee2000640 1 --2- 172.21.15.32:0/3513616969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5edc0776e0 0x7f5edc077af0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009900 tx=0x7f5ecc02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.060+0000 7f5ee2000640 1 -- 172.21.15.32:0/3513616969 shutdown_connections 2024-08-31T07:10:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.060+0000 7f5ee2000640 1 --2- 172.21.15.32:0/3513616969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5edc1144e0 0x7f5edc116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.060+0000 7f5ee2000640 1 --2- 172.21.15.32:0/3513616969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5edc078030 0x7f5edc113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.060+0000 7f5ee2000640 1 --2- 172.21.15.32:0/3513616969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5edc0776e0 0x7f5edc077af0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.060+0000 7f5ee2000640 1 -- 172.21.15.32:0/3513616969 >> 172.21.15.32:0/3513616969 conn(0x7f5edc1015b0 msgr2=0x7f5edc103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.060+0000 7f5ee2000640 1 -- 172.21.15.32:0/3513616969 shutdown_connections 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.060+0000 7f5ee2000640 1 -- 172.21.15.32:0/3513616969 wait complete. 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.061+0000 7f5ee2000640 1 Processor -- start 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.061+0000 7f5ee2000640 1 -- start start 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.061+0000 7f5ee2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5edc0776e0 0x7f5edc1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.061+0000 7f5ee2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5edc078030 0x7f5edc1a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.061+0000 7f5ee2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5edc1144e0 0x7f5edc1a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.061+0000 7f5ee2000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc119e40 con 0x7f5edc0776e0 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.061+0000 7f5ee2000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc119cc0 con 0x7f5edc1144e0 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.061+0000 7f5ee2000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc119fc0 con 0x7f5edc078030 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5edc0776e0 0x7f5edc1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5edc1144e0 0x7f5edc1a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5edc1144e0 0x7f5edc1a60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53832/0 (socket says 172.21.15.32:53832) 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edbe00640 1 -- 172.21.15.32:0/1951546780 learned_addr learned my addr 172.21.15.32:0/1951546780 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5edc078030 0x7f5edc1a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edbe00640 1 -- 172.21.15.32:0/1951546780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5edc078030 msgr2=0x7f5edc1a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edbe00640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5edc078030 0x7f5edc1a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edbe00640 1 -- 172.21.15.32:0/1951546780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5edc0776e0 msgr2=0x7f5edc1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edbe00640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5edc0776e0 0x7f5edc1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edbe00640 1 -- 172.21.15.32:0/1951546780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc1a6800 con 0x7f5edc1144e0 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.062+0000 7f5edbe00640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5edc1144e0 0x7f5edc1a60d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400efd0 tx=0x7f5ed400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.063+0000 7f5ed3e00640 1 -- 172.21.15.32:0/1951546780 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4003b70 con 0x7f5edc1144e0 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.063+0000 7f5ed3e00640 1 -- 172.21.15.32:0/1951546780 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed4003d10 con 0x7f5edc1144e0 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.063+0000 7f5ee2000640 1 -- 172.21.15.32:0/1951546780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc1a6af0 con 0x7f5edc1144e0 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.063+0000 7f5ed3e00640 1 -- 172.21.15.32:0/1951546780 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40108a0 con 0x7f5edc1144e0 2024-08-31T07:10:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.063+0000 7f5ee2000640 1 -- 172.21.15.32:0/1951546780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc1ad250 con 0x7f5edc1144e0 2024-08-31T07:10:58.799 INFO:teuthology.orchestra.run.smithi032.stdout:141733920834 2024-08-31T07:10:58.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.064+0000 7f5ee2000640 1 -- 172.21.15.32:0/1951546780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc077af0 con 0x7f5edc1144e0 2024-08-31T07:10:58.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.066+0000 7f5ed3e00640 1 -- 172.21.15.32:0/1951546780 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f5ed4004020 con 0x7f5edc1144e0 2024-08-31T07:10:58.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.068+0000 7f5ed3e00640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ec00b4c10 0x7f5ec00b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:58.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.068+0000 7f5ed3e00640 1 -- 172.21.15.32:0/1951546780 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5ed40e57d0 con 0x7f5edc1144e0 2024-08-31T07:10:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.068+0000 7f5edb400640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ec00b4c10 0x7f5ec00b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.069+0000 7f5edb400640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ec00b4c10 0x7f5ec00b7120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc031440 tx=0x7f5ecc009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.072+0000 7f5ed3e00640 1 -- 172.21.15.32:0/1951546780 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5ed40ac6a0 con 0x7f5edc1144e0 2024-08-31T07:10:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.273+0000 7f5ee2000640 1 -- 172.21.15.32:0/1951546780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f5edc02e430 con 0x7f5edc1144e0 2024-08-31T07:10:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.274+0000 7f5ed3e00640 1 -- 172.21.15.32:0/1951546780 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5ed40abfa0 con 0x7f5edc1144e0 2024-08-31T07:10:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.276+0000 7f5ee2000640 1 -- 172.21.15.32:0/1951546780 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ec00b4c10 msgr2=0x7f5ec00b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.276+0000 7f5ee2000640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ec00b4c10 0x7f5ec00b7120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc031440 tx=0x7f5ecc009560 comp rx=0 tx=0).stop 2024-08-31T07:10:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.276+0000 7f5ee2000640 1 -- 172.21.15.32:0/1951546780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5edc1144e0 msgr2=0x7f5edc1a60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.276+0000 7f5ee2000640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5edc1144e0 0x7f5edc1a60d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400efd0 tx=0x7f5ed400c6a0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.276+0000 7f5ee2000640 1 -- 172.21.15.32:0/1951546780 shutdown_connections 2024-08-31T07:10:58.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.276+0000 7f5ee2000640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ec00b4c10 0x7f5ec00b7120 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.276+0000 7f5ee2000640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5edc1144e0 0x7f5edc1a60d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.276+0000 7f5ee2000640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5edc078030 0x7f5edc1a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.276+0000 7f5ee2000640 1 --2- 172.21.15.32:0/1951546780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5edc0776e0 0x7f5edc1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:58.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.276+0000 7f5ee2000640 1 -- 172.21.15.32:0/1951546780 >> 172.21.15.32:0/1951546780 conn(0x7f5edc1015b0 msgr2=0x7f5edc116620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:58.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.277+0000 7f5ee2000640 1 -- 172.21.15.32:0/1951546780 shutdown_connections 2024-08-31T07:10:58.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:58.277+0000 7f5ee2000640 1 -- 172.21.15.32:0/1951546780 wait complete. 2024-08-31T07:10:58.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:58 smithi032 ceph-mon[17252]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 60 B/s, 0 keys/s, 0 objects/s recovering 2024-08-31T07:10:58.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:10:58 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1951546780' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-31T07:10:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:58 smithi073 ceph-mon[21995]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 60 B/s, 0 keys/s, 0 objects/s recovering 2024-08-31T07:10:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:10:58 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1951546780' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-31T07:10:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:58 smithi084 ceph-mon[22387]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 60 B/s, 0 keys/s, 0 objects/s recovering 2024-08-31T07:10:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:10:58 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1951546780' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-31T07:10:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.087+0000 7f1709c00640 1 -- 172.21.15.32:0/3680488123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f170410bdc0 msgr2=0x7f17041125f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.087+0000 7f1709c00640 1 --2- 172.21.15.32:0/3680488123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f170410bdc0 0x7f17041125f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009ca0 tx=0x7f16fc02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.087+0000 7f1709c00640 1 -- 172.21.15.32:0/3680488123 shutdown_connections 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.088+0000 7f1709c00640 1 --2- 172.21.15.32:0/3680488123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f170410bdc0 0x7f17041125f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.088+0000 7f1709c00640 1 --2- 172.21.15.32:0/3680488123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f170410b470 0x7f170410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.088+0000 7f1709c00640 1 --2- 172.21.15.32:0/3680488123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1704105b30 0x7f1704105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.088+0000 7f1709c00640 1 -- 172.21.15.32:0/3680488123 >> 172.21.15.32:0/3680488123 conn(0x7f1704101600 msgr2=0x7f1704103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.088+0000 7f1709c00640 1 -- 172.21.15.32:0/3680488123 shutdown_connections 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.088+0000 7f1709c00640 1 -- 172.21.15.32:0/3680488123 wait complete. 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.089+0000 7f1709c00640 1 Processor -- start 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.089+0000 7f1709c00640 1 -- start start 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.089+0000 7f1709c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1704105b30 0x7f170419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.089+0000 7f1709c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f170410b470 0x7f170419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.089+0000 7f1709c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f170410bdc0 0x7f17041a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.089+0000 7f1709c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1704115a80 con 0x7f1704105b30 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.089+0000 7f1709c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1704115900 con 0x7f170410bdc0 2024-08-31T07:10:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.089+0000 7f1709c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1704115c00 con 0x7f170410b470 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.089+0000 7f1702a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1704105b30 0x7f170419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.090+0000 7f1702a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1704105b30 0x7f170419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58320/0 (socket says 172.21.15.32:58320) 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.090+0000 7f1702a00640 1 -- 172.21.15.32:0/3752925504 learned_addr learned my addr 172.21.15.32:0/3752925504 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.090+0000 7f1702000640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f170410b470 0x7f170419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.090+0000 7f1703400640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f170410bdc0 0x7f17041a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.090+0000 7f1702a00640 1 -- 172.21.15.32:0/3752925504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f170410b470 msgr2=0x7f170419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.090+0000 7f1702a00640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f170410b470 0x7f170419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.090+0000 7f1702a00640 1 -- 172.21.15.32:0/3752925504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f170410bdc0 msgr2=0x7f17041a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.090+0000 7f1702a00640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f170410bdc0 0x7f17041a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.090+0000 7f1702a00640 1 -- 172.21.15.32:0/3752925504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17041a2380 con 0x7f1704105b30 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.090+0000 7f1702000640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f170410b470 0x7f170419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.091+0000 7f1702a00640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1704105b30 0x7f170419c410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16f400e9f0 tx=0x7f16f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.091+0000 7f16fb400640 1 -- 172.21.15.32:0/3752925504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400cd80 con 0x7f1704105b30 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.091+0000 7f16fb400640 1 -- 172.21.15.32:0/3752925504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f40108e0 con 0x7f1704105b30 2024-08-31T07:10:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.091+0000 7f16fb400640 1 -- 172.21.15.32:0/3752925504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4018980 con 0x7f1704105b30 2024-08-31T07:10:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.091+0000 7f1709c00640 1 -- 172.21.15.32:0/3752925504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17041a2670 con 0x7f1704105b30 2024-08-31T07:10:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.091+0000 7f1709c00640 1 -- 172.21.15.32:0/3752925504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17041a8dd0 con 0x7f1704105b30 2024-08-31T07:10:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.092+0000 7f1709c00640 1 -- 172.21.15.32:0/3752925504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1704105f40 con 0x7f1704105b30 2024-08-31T07:10:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.094+0000 7f16fb400640 1 -- 172.21.15.32:0/3752925504 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f16f4010430 con 0x7f1704105b30 2024-08-31T07:10:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.096+0000 7f16fb400640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16e80b4c10 0x7f16e80b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:10:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.096+0000 7f16fb400640 1 -- 172.21.15.32:0/3752925504 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f16f40e5c20 con 0x7f1704105b30 2024-08-31T07:10:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.096+0000 7f1702000640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16e80b4c10 0x7f16e80b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:10:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.097+0000 7f1702000640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16e80b4c10 0x7f16e80b7120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1704104e50 tx=0x7f16f0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:10:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.098+0000 7f16fb400640 1 -- 172.21.15.32:0/3752925504 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f16f40ac4a0 con 0x7f1704105b30 2024-08-31T07:10:59.244 INFO:teuthology.orchestra.run.smithi032.stdout:163208757309 2024-08-31T07:10:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.316+0000 7f1709c00640 1 -- 172.21.15.32:0/3752925504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f170410d620 con 0x7f1704105b30 2024-08-31T07:10:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.317+0000 7f16fb400640 1 -- 172.21.15.32:0/3752925504 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f16f40b0c60 con 0x7f1704105b30 2024-08-31T07:10:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 -- 172.21.15.32:0/3752925504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16e80b4c10 msgr2=0x7f16e80b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16e80b4c10 0x7f16e80b7120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1704104e50 tx=0x7f16f0005f90 comp rx=0 tx=0).stop 2024-08-31T07:10:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 -- 172.21.15.32:0/3752925504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1704105b30 msgr2=0x7f170419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:10:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1704105b30 0x7f170419c410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16f400e9f0 tx=0x7f16f400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 -- 172.21.15.32:0/3752925504 shutdown_connections 2024-08-31T07:10:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16e80b4c10 0x7f16e80b7120 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f170410bdc0 0x7f17041a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f170410b470 0x7f170419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 --2- 172.21.15.32:0/3752925504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1704105b30 0x7f170419c410 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:10:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 -- 172.21.15.32:0/3752925504 >> 172.21.15.32:0/3752925504 conn(0x7f1704101600 msgr2=0x7f1704108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:10:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 -- 172.21.15.32:0/3752925504 shutdown_connections 2024-08-31T07:10:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:10:57.320+0000 7f1709c00640 1 -- 172.21.15.32:0/3752925504 wait complete. 2024-08-31T07:11:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.038+0000 7f1e1f800640 1 -- 172.21.15.32:0/1223856550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e18109bb0 msgr2=0x7f1e181103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.038+0000 7f1e1f800640 1 --2- 172.21.15.32:0/1223856550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e18109bb0 0x7f1e181103e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009ca0 tx=0x7f1e1002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.038+0000 7f1e1f800640 1 -- 172.21.15.32:0/1223856550 shutdown_connections 2024-08-31T07:11:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.038+0000 7f1e1f800640 1 --2- 172.21.15.32:0/1223856550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e18109bb0 0x7f1e181103e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.038+0000 7f1e1f800640 1 --2- 172.21.15.32:0/1223856550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18109260 0x7f1e18109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.038+0000 7f1e1f800640 1 --2- 172.21.15.32:0/1223856550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e18103920 0x7f1e18103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.038+0000 7f1e1f800640 1 -- 172.21.15.32:0/1223856550 >> 172.21.15.32:0/1223856550 conn(0x7f1e180ff330 msgr2=0x7f1e18101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.039+0000 7f1e1f800640 1 -- 172.21.15.32:0/1223856550 shutdown_connections 2024-08-31T07:11:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.039+0000 7f1e1f800640 1 -- 172.21.15.32:0/1223856550 wait complete. 2024-08-31T07:11:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.040+0000 7f1e1f800640 1 Processor -- start 2024-08-31T07:11:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.040+0000 7f1e1f800640 1 -- start start 2024-08-31T07:11:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.040+0000 7f1e1f800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18103920 0x7f1e1819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.040+0000 7f1e1f800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e18109260 0x7f1e1819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.040+0000 7f1e1f800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e18109bb0 0x7f1e181a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.040+0000 7f1e1f800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e18113870 con 0x7f1e18109260 2024-08-31T07:11:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.040+0000 7f1e1f800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181136f0 con 0x7f1e18109bb0 2024-08-31T07:11:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.040+0000 7f1e1f800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181139f0 con 0x7f1e18103920 2024-08-31T07:11:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.041+0000 7f1e17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e18109260 0x7f1e1819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.041+0000 7f1e1ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18103920 0x7f1e1819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.041+0000 7f1e1ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18103920 0x7f1e1819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51086/0 (socket says 172.21.15.32:51086) 2024-08-31T07:11:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.041+0000 7f1e1ce00640 1 -- 172.21.15.32:0/2110555294 learned_addr learned my addr 172.21.15.32:0/2110555294 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.041+0000 7f1e1d800640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e18109bb0 0x7f1e181a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.041+0000 7f1e1ce00640 1 -- 172.21.15.32:0/2110555294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e18109bb0 msgr2=0x7f1e181a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.041+0000 7f1e1ce00640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e18109bb0 0x7f1e181a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.041+0000 7f1e1ce00640 1 -- 172.21.15.32:0/2110555294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e18109260 msgr2=0x7f1e1819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.041+0000 7f1e1ce00640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e18109260 0x7f1e1819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.041+0000 7f1e1ce00640 1 -- 172.21.15.32:0/2110555294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e181a2370 con 0x7f1e18103920 2024-08-31T07:11:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.042+0000 7f1e1ce00640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18103920 0x7f1e1819c400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00e9f0 tx=0x7f1e0c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.042+0000 7f1e15600640 1 -- 172.21.15.32:0/2110555294 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00cd80 con 0x7f1e18103920 2024-08-31T07:11:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.042+0000 7f1e15600640 1 -- 172.21.15.32:0/2110555294 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e0c0108e0 con 0x7f1e18103920 2024-08-31T07:11:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.042+0000 7f1e1f800640 1 -- 172.21.15.32:0/2110555294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e181a2660 con 0x7f1e18103920 2024-08-31T07:11:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.042+0000 7f1e15600640 1 -- 172.21.15.32:0/2110555294 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c018980 con 0x7f1e18103920 2024-08-31T07:11:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.042+0000 7f1e1f800640 1 -- 172.21.15.32:0/2110555294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e181a8dc0 con 0x7f1e18103920 2024-08-31T07:11:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.043+0000 7f1e1f800640 1 -- 172.21.15.32:0/2110555294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e18103d30 con 0x7f1e18103920 2024-08-31T07:11:00.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.049+0000 7f1e15600640 1 -- 172.21.15.32:0/2110555294 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f1e0c010430 con 0x7f1e18103920 2024-08-31T07:11:00.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.051+0000 7f1e15600640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e000b4c10 0x7f1e000b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.051+0000 7f1e15600640 1 -- 172.21.15.32:0/2110555294 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1e0c0e5c20 con 0x7f1e18103920 2024-08-31T07:11:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.051+0000 7f1e17e00640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e000b4c10 0x7f1e000b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.051+0000 7f1e15600640 1 -- 172.21.15.32:0/2110555294 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1e0c1161d0 con 0x7f1e18103920 2024-08-31T07:11:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.052+0000 7f1e17e00640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e000b4c10 0x7f1e000b7120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1e18109ae0 tx=0x7f1e04005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.259+0000 7f1e1f800640 1 -- 172.21.15.32:0/2110555294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f1e181a90b0 con 0x7f1e18103920 2024-08-31T07:11:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.259+0000 7f1e15600640 1 -- 172.21.15.32:0/2110555294 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1e0c0ac4a0 con 0x7f1e18103920 2024-08-31T07:11:00.260 INFO:teuthology.orchestra.run.smithi032.stdout:304942678044 2024-08-31T07:11:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.261+0000 7f1e1f800640 1 -- 172.21.15.32:0/2110555294 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e000b4c10 msgr2=0x7f1e000b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.261+0000 7f1e1f800640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e000b4c10 0x7f1e000b7120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1e18109ae0 tx=0x7f1e04005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.261+0000 7f1e1f800640 1 -- 172.21.15.32:0/2110555294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18103920 msgr2=0x7f1e1819c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.261+0000 7f1e1f800640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18103920 0x7f1e1819c400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00e9f0 tx=0x7f1e0c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.262+0000 7f1e1f800640 1 -- 172.21.15.32:0/2110555294 shutdown_connections 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.262+0000 7f1e1f800640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e000b4c10 0x7f1e000b7120 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.262+0000 7f1e1f800640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e18109bb0 0x7f1e181a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.262+0000 7f1e1f800640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e18109260 0x7f1e1819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.262+0000 7f1e1f800640 1 --2- 172.21.15.32:0/2110555294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18103920 0x7f1e1819c400 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.262+0000 7f1e1f800640 1 -- 172.21.15.32:0/2110555294 >> 172.21.15.32:0/2110555294 conn(0x7f1e180ff330 msgr2=0x7f1e181059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.262+0000 7f1e1f800640 1 -- 172.21.15.32:0/2110555294 shutdown_connections 2024-08-31T07:11:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.262+0000 7f1e1f800640 1 -- 172.21.15.32:0/2110555294 wait complete. 2024-08-31T07:11:00.697 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:00 smithi032 ceph-mon[17252]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:00.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:00 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2110555294' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-31T07:11:00.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:00 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/464732858' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-31T07:11:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:00 smithi073 ceph-mon[21995]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:00 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2110555294' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-31T07:11:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:00 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/464732858' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-31T07:11:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:00 smithi084 ceph-mon[22387]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:00 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2110555294' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-31T07:11:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:00 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/464732858' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-31T07:11:01.522 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678041 got 304942678044 for osd.11 2024-08-31T07:11:01.522 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:01.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:01 smithi032 ceph-mon[17252]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-08-31T07:11:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:01 smithi073 ceph-mon[21995]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-08-31T07:11:01.976 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705749 got 38654705751 for osd.0 2024-08-31T07:11:01.977 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:01 smithi084 ceph-mon[22387]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-08-31T07:11:02.073 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920830 got 141733920834 for osd.4 2024-08-31T07:11:02.073 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:02.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.661+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2331626682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d8010bbe0 msgr2=0x7f1d80112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:02.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.661+0000 7f1d7fe00640 1 --2- 172.21.15.32:0/2331626682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d8010bbe0 0x7f1d80112410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009ca0 tx=0x7f1d7802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.661+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2331626682 shutdown_connections 2024-08-31T07:11:02.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.661+0000 7f1d7fe00640 1 --2- 172.21.15.32:0/2331626682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d8010bbe0 0x7f1d80112410 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.661+0000 7f1d7fe00640 1 --2- 172.21.15.32:0/2331626682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d8010b290 0x7f1d8010b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.661+0000 7f1d7fe00640 1 --2- 172.21.15.32:0/2331626682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d80105950 0x7f1d80105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.661+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2331626682 >> 172.21.15.32:0/2331626682 conn(0x7f1d80101420 msgr2=0x7f1d80103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.661+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2331626682 shutdown_connections 2024-08-31T07:11:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.661+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2331626682 wait complete. 2024-08-31T07:11:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.663+0000 7f1d7fe00640 1 Processor -- start 2024-08-31T07:11:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.663+0000 7f1d7fe00640 1 -- start start 2024-08-31T07:11:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.663+0000 7f1d7fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80105950 0x7f1d8019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.663+0000 7f1d7fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d8010b290 0x7f1d8019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.663+0000 7f1d7fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d8010bbe0 0x7f1d801a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.663+0000 7f1d7fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d801158a0 con 0x7f1d8010b290 2024-08-31T07:11:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.663+0000 7f1d7fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d80115720 con 0x7f1d80105950 2024-08-31T07:11:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.663+0000 7f1d7fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1d80115a20 con 0x7f1d8010bbe0 2024-08-31T07:11:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.664+0000 7f1d7e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d8010b290 0x7f1d8019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.664+0000 7f1d7e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d8010b290 0x7f1d8019c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58406/0 (socket says 172.21.15.32:58406) 2024-08-31T07:11:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.664+0000 7f1d7e000640 1 -- 172.21.15.32:0/2199732612 learned_addr learned my addr 172.21.15.32:0/2199732612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.664+0000 7f1d7ea00640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80105950 0x7f1d8019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:02.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.664+0000 7f1d7f400640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d8010bbe0 0x7f1d801a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.664+0000 7f1d7e000640 1 -- 172.21.15.32:0/2199732612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d8010bbe0 msgr2=0x7f1d801a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.664+0000 7f1d7e000640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d8010bbe0 0x7f1d801a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.664+0000 7f1d7e000640 1 -- 172.21.15.32:0/2199732612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80105950 msgr2=0x7f1d8019c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.664+0000 7f1d7e000640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80105950 0x7f1d8019c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.664+0000 7f1d7e000640 1 -- 172.21.15.32:0/2199732612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d801a21a0 con 0x7f1d8010b290 2024-08-31T07:11:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.665+0000 7f1d7e000640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d8010b290 0x7f1d8019c7d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00d950 tx=0x7f1d6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.665+0000 7f1d84a00640 1 -- 172.21.15.32:0/2199732612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c017070 con 0x7f1d8010b290 2024-08-31T07:11:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.665+0000 7f1d84a00640 1 -- 172.21.15.32:0/2199732612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d6c009c30 con 0x7f1d8010b290 2024-08-31T07:11:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.665+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2199732612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d801a2490 con 0x7f1d8010b290 2024-08-31T07:11:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.665+0000 7f1d84a00640 1 -- 172.21.15.32:0/2199732612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0047a0 con 0x7f1d8010b290 2024-08-31T07:11:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.665+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2199732612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d801a8bf0 con 0x7f1d8010b290 2024-08-31T07:11:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.666+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2199732612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d80105d60 con 0x7f1d8010b290 2024-08-31T07:11:02.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.668+0000 7f1d84a00640 1 -- 172.21.15.32:0/2199732612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f1d6c004020 con 0x7f1d8010b290 2024-08-31T07:11:02.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.670+0000 7f1d84a00640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d640b4c10 0x7f1d640b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:02.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.670+0000 7f1d84a00640 1 -- 172.21.15.32:0/2199732612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1d6c0e4fe0 con 0x7f1d8010b290 2024-08-31T07:11:02.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.670+0000 7f1d7ea00640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d640b4c10 0x7f1d640b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:02.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.671+0000 7f1d7ea00640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d640b4c10 0x7f1d640b7120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009bd0 tx=0x7f1d70005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:02.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.674+0000 7f1d84a00640 1 -- 172.21.15.32:0/2199732612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1d6c020020 con 0x7f1d8010b290 2024-08-31T07:11:02.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.125+0000 7f6d73200640 1 -- 172.21.15.32:0/4184016355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d6c109260 msgr2=0x7f6d6c109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:02.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.125+0000 7f6d73200640 1 --2- 172.21.15.32:0/4184016355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d6c109260 0x7f6d6c109670 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009860 tx=0x7f6d5802a340 comp rx=0 tx=0).stop 2024-08-31T07:11:02.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.125+0000 7f6d73200640 1 -- 172.21.15.32:0/4184016355 shutdown_connections 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.125+0000 7f6d73200640 1 --2- 172.21.15.32:0/4184016355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d6c109bb0 0x7f6d6c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.125+0000 7f6d73200640 1 --2- 172.21.15.32:0/4184016355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d6c109260 0x7f6d6c109670 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.125+0000 7f6d73200640 1 --2- 172.21.15.32:0/4184016355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d6c103920 0x7f6d6c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.125+0000 7f6d73200640 1 -- 172.21.15.32:0/4184016355 >> 172.21.15.32:0/4184016355 conn(0x7f6d6c0ff3b0 msgr2=0x7f6d6c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.125+0000 7f6d73200640 1 -- 172.21.15.32:0/4184016355 shutdown_connections 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.125+0000 7f6d73200640 1 -- 172.21.15.32:0/4184016355 wait complete. 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.127+0000 7f6d73200640 1 Processor -- start 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.127+0000 7f6d73200640 1 -- start start 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.127+0000 7f6d73200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d6c103920 0x7f6d6c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.127+0000 7f6d73200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d6c109260 0x7f6d6c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.127+0000 7f6d73200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d6c109bb0 0x7f6d6c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.127+0000 7f6d73200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c113870 con 0x7f6d6c103920 2024-08-31T07:11:02.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.127+0000 7f6d73200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c1136f0 con 0x7f6d6c109260 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d73200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c1139f0 con 0x7f6d6c109bb0 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d6c103920 0x7f6d6c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d6c103920 0x7f6d6c0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58388/0 (socket says 172.21.15.32:58388) 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d6be00640 1 -- 172.21.15.32:0/464732858 learned_addr learned my addr 172.21.15.32:0/464732858 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d71200640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d6c109bb0 0x7f6d6c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d6b400640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d6c109260 0x7f6d6c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d6be00640 1 -- 172.21.15.32:0/464732858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d6c109bb0 msgr2=0x7f6d6c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d6be00640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d6c109bb0 0x7f6d6c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d6be00640 1 -- 172.21.15.32:0/464732858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d6c109260 msgr2=0x7f6d6c0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d6be00640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d6c109260 0x7f6d6c0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d6be00640 1 -- 172.21.15.32:0/464732858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c1a8520 con 0x7f6d6c103920 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.128+0000 7f6d71200640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d6c109bb0 0x7f6d6c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.129+0000 7f6d6be00640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d6c103920 0x7f6d6c0754b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00e9c0 tx=0x7f6d5c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.129+0000 7f6d68c00640 1 -- 172.21.15.32:0/464732858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00cd80 con 0x7f6d6c103920 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.129+0000 7f6d73200640 1 -- 172.21.15.32:0/464732858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6c1a8810 con 0x7f6d6c103920 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.129+0000 7f6d68c00640 1 -- 172.21.15.32:0/464732858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d5c0035f0 con 0x7f6d6c103920 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.129+0000 7f6d68c00640 1 -- 172.21.15.32:0/464732858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c010690 con 0x7f6d6c103920 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.129+0000 7f6d73200640 1 -- 172.21.15.32:0/464732858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6c1a8d20 con 0x7f6d6c103920 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.130+0000 7f6d73200640 1 -- 172.21.15.32:0/464732858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6c104ba0 con 0x7f6d6c103920 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.132+0000 7f6d68c00640 1 -- 172.21.15.32:0/464732858 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f6d5c003fb0 con 0x7f6d6c103920 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.135+0000 7f6d68c00640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d500b4c10 0x7f6d500b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.135+0000 7f6d68c00640 1 -- 172.21.15.32:0/464732858 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6d5c0e56e0 con 0x7f6d6c103920 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.135+0000 7f6d6b400640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d500b4c10 0x7f6d500b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.136+0000 7f6d6b400640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d500b4c10 0x7f6d500b7120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d58005940 tx=0x7f6d580058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.139+0000 7f6d68c00640 1 -- 172.21.15.32:0/464732858 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6d5c0ac5b0 con 0x7f6d6c103920 2024-08-31T07:11:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.349+0000 7f6d73200640 1 -- 172.21.15.32:0/464732858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f6d6c10b410 con 0x7f6d6c103920 2024-08-31T07:11:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.349+0000 7f6d68c00640 1 -- 172.21.15.32:0/464732858 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6d5c0abeb0 con 0x7f6d6c103920 2024-08-31T07:11:02.780 INFO:teuthology.orchestra.run.smithi032.stdout:279172874273 2024-08-31T07:11:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 -- 172.21.15.32:0/464732858 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d500b4c10 msgr2=0x7f6d500b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d500b4c10 0x7f6d500b7120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d58005940 tx=0x7f6d580058d0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 -- 172.21.15.32:0/464732858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d6c103920 msgr2=0x7f6d6c0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d6c103920 0x7f6d6c0754b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00e9c0 tx=0x7f6d5c00edc0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 -- 172.21.15.32:0/464732858 shutdown_connections 2024-08-31T07:11:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6d500b4c10 0x7f6d500b7120 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d6c109bb0 0x7f6d6c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d6c109260 0x7f6d6c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 --2- 172.21.15.32:0/464732858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d6c103920 0x7f6d6c0754b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 -- 172.21.15.32:0/464732858 >> 172.21.15.32:0/464732858 conn(0x7f6d6c0ff3b0 msgr2=0x7f6d6c1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 -- 172.21.15.32:0/464732858 shutdown_connections 2024-08-31T07:11:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:00.352+0000 7f6d73200640 1 -- 172.21.15.32:0/464732858 wait complete. 2024-08-31T07:11:02.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.873+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2199732612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1d8010d440 con 0x7f1d8010b290 2024-08-31T07:11:02.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.873+0000 7f1d84a00640 1 -- 172.21.15.32:0/2199732612 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1d6c0abeb0 con 0x7f1d8010b290 2024-08-31T07:11:02.875 INFO:teuthology.orchestra.run.smithi032.stdout:188978561078 2024-08-31T07:11:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.876+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2199732612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d640b4c10 msgr2=0x7f1d640b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.876+0000 7f1d7fe00640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d640b4c10 0x7f1d640b7120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009bd0 tx=0x7f1d70005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.876+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2199732612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d8010b290 msgr2=0x7f1d8019c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.876+0000 7f1d7fe00640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d8010b290 0x7f1d8019c7d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00d950 tx=0x7f1d6c00de10 comp rx=0 tx=0).stop 2024-08-31T07:11:02.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.876+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2199732612 shutdown_connections 2024-08-31T07:11:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.876+0000 7f1d7fe00640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d640b4c10 0x7f1d640b7120 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.876+0000 7f1d7fe00640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d8010bbe0 0x7f1d801a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.876+0000 7f1d7fe00640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d8010b290 0x7f1d8019c7d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.876+0000 7f1d7fe00640 1 --2- 172.21.15.32:0/2199732612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80105950 0x7f1d8019c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.876+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2199732612 >> 172.21.15.32:0/2199732612 conn(0x7f1d80101420 msgr2=0x7f1d8010a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.877+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2199732612 shutdown_connections 2024-08-31T07:11:02.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:02.877+0000 7f1d7fe00640 1 -- 172.21.15.32:0/2199732612 wait complete. 2024-08-31T07:11:03.713 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874271 got 279172874273 for osd.10 2024-08-31T07:11:03.713 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:03.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:03 smithi032 ceph-mon[17252]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-08-31T07:11:03.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:03 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2199732612' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-31T07:11:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.015+0000 7f9f9f000640 1 -- 172.21.15.32:0/520406990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f98109bb0 msgr2=0x7f9f981103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.015+0000 7f9f9f000640 1 --2- 172.21.15.32:0/520406990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f98109bb0 0x7f9f981103e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009ca0 tx=0x7f9f9002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.015+0000 7f9f9f000640 1 -- 172.21.15.32:0/520406990 shutdown_connections 2024-08-31T07:11:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.015+0000 7f9f9f000640 1 --2- 172.21.15.32:0/520406990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f98109bb0 0x7f9f981103e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.015+0000 7f9f9f000640 1 --2- 172.21.15.32:0/520406990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f98109260 0x7f9f98109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.015+0000 7f9f9f000640 1 --2- 172.21.15.32:0/520406990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f98103920 0x7f9f98103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.015+0000 7f9f9f000640 1 -- 172.21.15.32:0/520406990 >> 172.21.15.32:0/520406990 conn(0x7f9f980ff3b0 msgr2=0x7f9f98101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.015+0000 7f9f9f000640 1 -- 172.21.15.32:0/520406990 shutdown_connections 2024-08-31T07:11:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.015+0000 7f9f9f000640 1 -- 172.21.15.32:0/520406990 wait complete. 2024-08-31T07:11:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.016+0000 7f9f9f000640 1 Processor -- start 2024-08-31T07:11:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.016+0000 7f9f9f000640 1 -- start start 2024-08-31T07:11:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.016+0000 7f9f9f000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f98103920 0x7f9f9819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.016+0000 7f9f9f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f98109260 0x7f9f9819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.016+0000 7f9f9f000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f98109bb0 0x7f9f9819fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.016+0000 7f9f9f000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9f981138c0 con 0x7f9f98103920 2024-08-31T07:11:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.016+0000 7f9f9f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98113740 con 0x7f9f98109260 2024-08-31T07:11:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.016+0000 7f9f9f000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98113a40 con 0x7f9f98109bb0 2024-08-31T07:11:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f98103920 0x7f9f9819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f98103920 0x7f9f9819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58418/0 (socket says 172.21.15.32:58418) 2024-08-31T07:11:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f97e00640 1 -- 172.21.15.32:0/1756352796 learned_addr learned my addr 172.21.15.32:0/1756352796 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f9d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f98109bb0 0x7f9f9819fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f97400640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f98109260 0x7f9f9819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f97e00640 1 -- 172.21.15.32:0/1756352796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f98109bb0 msgr2=0x7f9f9819fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f97e00640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f98109bb0 0x7f9f9819fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f97e00640 1 -- 172.21.15.32:0/1756352796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f98109260 msgr2=0x7f9f9819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f97e00640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f98109260 0x7f9f9819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f97e00640 1 -- 172.21.15.32:0/1756352796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f981a01c0 con 0x7f9f98103920 2024-08-31T07:11:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.017+0000 7f9f97400640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f98109260 0x7f9f9819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.018+0000 7f9f97e00640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f98103920 0x7f9f9819a250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800e9f0 tx=0x7f9f8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.018+0000 7f9f94c00640 1 -- 172.21.15.32:0/1756352796 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800cd80 con 0x7f9f98103920 2024-08-31T07:11:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.018+0000 7f9f94c00640 1 -- 172.21.15.32:0/1756352796 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f880108e0 con 0x7f9f98103920 2024-08-31T07:11:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.018+0000 7f9f94c00640 1 -- 172.21.15.32:0/1756352796 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88018980 con 0x7f9f98103920 2024-08-31T07:11:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.018+0000 7f9f9f000640 1 -- 172.21.15.32:0/1756352796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f981a04b0 con 0x7f9f98103920 2024-08-31T07:11:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.018+0000 7f9f9f000640 1 -- 172.21.15.32:0/1756352796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f981a6c10 con 0x7f9f98103920 2024-08-31T07:11:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.020+0000 7f9f9f000640 1 -- 172.21.15.32:0/1756352796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f98104ba0 con 0x7f9f98103920 2024-08-31T07:11:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.022+0000 7f9f94c00640 1 -- 172.21.15.32:0/1756352796 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f9f88010430 con 0x7f9f98103920 2024-08-31T07:11:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.023+0000 7f9f94c00640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9f7c0b4c90 0x7f9f7c0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.023+0000 7f9f94c00640 1 -- 172.21.15.32:0/1756352796 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f9f880e56b0 con 0x7f9f98103920 2024-08-31T07:11:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.023+0000 7f9f97400640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9f7c0b4c90 0x7f9f7c0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.024+0000 7f9f97400640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9f7c0b4c90 0x7f9f7c0b71a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9f98102ab0 tx=0x7f9f8400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.027+0000 7f9f94c00640 1 -- 172.21.15.32:0/1756352796 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9f880abf30 con 0x7f9f98103920 2024-08-31T07:11:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:03 smithi073 ceph-mon[21995]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-08-31T07:11:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:03 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2199732612' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-31T07:11:04.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.225+0000 7f9f9f000640 1 -- 172.21.15.32:0/1756352796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9f9819b380 con 0x7f9f98103920 2024-08-31T07:11:04.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.226+0000 7f9f94c00640 1 -- 172.21.15.32:0/1756352796 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9f880b06f0 con 0x7f9f98103920 2024-08-31T07:11:04.226 INFO:teuthology.orchestra.run.smithi032.stdout:60129542228 2024-08-31T07:11:04.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 -- 172.21.15.32:0/1756352796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9f7c0b4c90 msgr2=0x7f9f7c0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:04.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9f7c0b4c90 0x7f9f7c0b71a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9f98102ab0 tx=0x7f9f8400b390 comp rx=0 tx=0).stop 2024-08-31T07:11:04.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 -- 172.21.15.32:0/1756352796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f98103920 msgr2=0x7f9f9819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:04.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f98103920 0x7f9f9819a250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800e9f0 tx=0x7f9f8800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 -- 172.21.15.32:0/1756352796 shutdown_connections 2024-08-31T07:11:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9f7c0b4c90 0x7f9f7c0b71a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f98109bb0 0x7f9f9819fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f98109260 0x7f9f9819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 --2- 172.21.15.32:0/1756352796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f98103920 0x7f9f9819a250 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 -- 172.21.15.32:0/1756352796 >> 172.21.15.32:0/1756352796 conn(0x7f9f980ff3b0 msgr2=0x7f9f981067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 -- 172.21.15.32:0/1756352796 shutdown_connections 2024-08-31T07:11:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:04.228+0000 7f9f9f000640 1 -- 172.21.15.32:0/1756352796 wait complete. 2024-08-31T07:11:04.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:03 smithi084 ceph-mon[22387]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-08-31T07:11:04.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:03 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2199732612' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-31T07:11:04.499 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378698 got 81604378701 for osd.2 2024-08-31T07:11:04.499 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:05.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:04 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1756352796' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-31T07:11:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:04 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1756352796' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-31T07:11:05.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:04 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1756352796' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-31T07:11:05.551 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542224 got 60129542228 for osd.1 2024-08-31T07:11:05.551 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:06.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:05 smithi032 ceph-mon[17252]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-08-31T07:11:06.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:05 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3774572193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-31T07:11:06.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:05 smithi073 ceph-mon[21995]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-08-31T07:11:06.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:05 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3774572193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-31T07:11:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:05 smithi084 ceph-mon[22387]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-08-31T07:11:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:05 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3774572193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-31T07:11:06.272 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561074 got 188978561078 for osd.6 2024-08-31T07:11:06.272 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:06.608 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757306 got 163208757309 for osd.5 2024-08-31T07:11:06.609 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.247+0000 7f9402800640 1 -- 172.21.15.32:0/2780837414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93fc10bde0 msgr2=0x7f93fc112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.247+0000 7f9402800640 1 --2- 172.21.15.32:0/2780837414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93fc10bde0 0x7f93fc112610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009ca0 tx=0x7f93f402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.247+0000 7f9402800640 1 -- 172.21.15.32:0/2780837414 shutdown_connections 2024-08-31T07:11:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.247+0000 7f9402800640 1 --2- 172.21.15.32:0/2780837414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93fc10bde0 0x7f93fc112610 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.247+0000 7f9402800640 1 --2- 172.21.15.32:0/2780837414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93fc10b490 0x7f93fc10b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.247+0000 7f9402800640 1 --2- 172.21.15.32:0/2780837414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc105b50 0x7f93fc105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.247+0000 7f9402800640 1 -- 172.21.15.32:0/2780837414 >> 172.21.15.32:0/2780837414 conn(0x7f93fc101580 msgr2=0x7f93fc1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.247+0000 7f9402800640 1 -- 172.21.15.32:0/2780837414 shutdown_connections 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.247+0000 7f9402800640 1 -- 172.21.15.32:0/2780837414 wait complete. 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.248+0000 7f9402800640 1 Processor -- start 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.248+0000 7f9402800640 1 -- start start 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.248+0000 7f9402800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93fc105b50 0x7f93fc19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.248+0000 7f9402800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93fc10b490 0x7f93fc19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f9402800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc10bde0 0x7f93fc1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f9402800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc115af0 con 0x7f93fc10b490 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f9402800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc115970 con 0x7f93fc10bde0 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f9402800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc115c70 con 0x7f93fc105b50 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f93f2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93fc10b490 0x7f93fc19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f93f2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93fc10b490 0x7f93fc19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58432/0 (socket says 172.21.15.32:58432) 2024-08-31T07:11:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f93f2a00640 1 -- 172.21.15.32:0/3774572193 learned_addr learned my addr 172.21.15.32:0/3774572193 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f93fbe00640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc10bde0 0x7f93fc1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f93fb400640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93fc105b50 0x7f93fc19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f93f2a00640 1 -- 172.21.15.32:0/3774572193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93fc105b50 msgr2=0x7f93fc19c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f93f2a00640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93fc105b50 0x7f93fc19c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f93f2a00640 1 -- 172.21.15.32:0/3774572193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc10bde0 msgr2=0x7f93fc1a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f93f2a00640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc10bde0 0x7f93fc1a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.249+0000 7f93f2a00640 1 -- 172.21.15.32:0/3774572193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc1a23b0 con 0x7f93fc10b490 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.250+0000 7f93f2a00640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93fc10b490 0x7f93fc19c9e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9f0 tx=0x7f93e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.250+0000 7f93f8c00640 1 -- 172.21.15.32:0/3774572193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e400ccc0 con 0x7f93fc10b490 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.250+0000 7f93f8c00640 1 -- 172.21.15.32:0/3774572193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e40108e0 con 0x7f93fc10b490 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.250+0000 7f93f8c00640 1 -- 172.21.15.32:0/3774572193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4018860 con 0x7f93fc10b490 2024-08-31T07:11:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.250+0000 7f9402800640 1 -- 172.21.15.32:0/3774572193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc1a26a0 con 0x7f93fc10b490 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.250+0000 7f9402800640 1 -- 172.21.15.32:0/3774572193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc1a8e00 con 0x7f93fc10b490 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.251+0000 7f9402800640 1 -- 172.21.15.32:0/3774572193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93fc105f60 con 0x7f93fc10b490 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.254+0000 7f93f8c00640 1 -- 172.21.15.32:0/3774572193 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f93e4010430 con 0x7f93fc10b490 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.256+0000 7f93f8c00640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93e00b4c90 0x7f93e00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.256+0000 7f93f8c00640 1 -- 172.21.15.32:0/3774572193 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f93e40e5ad0 con 0x7f93fc10b490 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.256+0000 7f93fb400640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93e00b4c90 0x7f93e00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.257+0000 7f93fb400640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93e00b4c90 0x7f93e00b71a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93ec005d00 tx=0x7f93ec00b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.257+0000 7f93f8c00640 1 -- 172.21.15.32:0/3774572193 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f93e40ac350 con 0x7f93fc10b490 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.460+0000 7f9402800640 1 -- 172.21.15.32:0/3774572193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f93fc10dc30 con 0x7f93fc10b490 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.460+0000 7f93f8c00640 1 -- 172.21.15.32:0/3774572193 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f93e40b0b10 con 0x7f93fc10b490 2024-08-31T07:11:06.708 INFO:teuthology.orchestra.run.smithi032.stdout:257698037799 2024-08-31T07:11:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 -- 172.21.15.32:0/3774572193 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93e00b4c90 msgr2=0x7f93e00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93e00b4c90 0x7f93e00b71a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93ec005d00 tx=0x7f93ec00b410 comp rx=0 tx=0).stop 2024-08-31T07:11:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 -- 172.21.15.32:0/3774572193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93fc10b490 msgr2=0x7f93fc19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93fc10b490 0x7f93fc19c9e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9f0 tx=0x7f93e400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 -- 172.21.15.32:0/3774572193 shutdown_connections 2024-08-31T07:11:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93e00b4c90 0x7f93e00b71a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc10bde0 0x7f93fc1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93fc10b490 0x7f93fc19c9e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 --2- 172.21.15.32:0/3774572193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93fc105b50 0x7f93fc19c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 -- 172.21.15.32:0/3774572193 >> 172.21.15.32:0/3774572193 conn(0x7f93fc101580 msgr2=0x7f93fc10aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 -- 172.21.15.32:0/3774572193 shutdown_connections 2024-08-31T07:11:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:05.463+0000 7f9402800640 1 -- 172.21.15.32:0/3774572193 wait complete. 2024-08-31T07:11:07.612 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397551 got 210453397553 for osd.7 2024-08-31T07:11:07.613 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:08.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:07 smithi032 ceph-mon[17252]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-08-31T07:11:08.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:07 smithi073 ceph-mon[21995]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-08-31T07:11:08.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:07 smithi084 ceph-mon[22387]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-08-31T07:11:08.798 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:08.829 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037797 got 257698037799 for osd.9 2024-08-31T07:11:08.829 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:09.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.493+0000 7fba81600640 1 -- 172.21.15.32:0/1254382544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c075b70 msgr2=0x7fba7c075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:09.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.493+0000 7fba81600640 1 --2- 172.21.15.32:0/1254382544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c075b70 0x7fba7c075f80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba64009900 tx=0x7fba6402a410 comp rx=0 tx=0).stop 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.493+0000 7fba81600640 1 -- 172.21.15.32:0/1254382544 shutdown_connections 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.493+0000 7fba81600640 1 --2- 172.21.15.32:0/1254382544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba7c1141f0 0x7fba7c1166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.493+0000 7fba81600640 1 --2- 172.21.15.32:0/1254382544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c075b70 0x7fba7c075f80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.493+0000 7fba81600640 1 --2- 172.21.15.32:0/1254382544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba7c077120 0x7fba7c075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.493+0000 7fba81600640 1 -- 172.21.15.32:0/1254382544 >> 172.21.15.32:0/1254382544 conn(0x7fba7c0ff330 msgr2=0x7fba7c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.493+0000 7fba81600640 1 -- 172.21.15.32:0/1254382544 shutdown_connections 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.493+0000 7fba81600640 1 -- 172.21.15.32:0/1254382544 wait complete. 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.494+0000 7fba81600640 1 Processor -- start 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.494+0000 7fba81600640 1 -- start start 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba81600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba7c075b70 0x7fba7c1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba81600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c077120 0x7fba7c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba81600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba7c1141f0 0x7fba7c1a6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba81600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c119e30 con 0x7fba7c077120 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba81600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c119cb0 con 0x7fba7c1141f0 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba81600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c119fb0 con 0x7fba7c075b70 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba73e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c077120 0x7fba7c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba73e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c077120 0x7fba7c1a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36320/0 (socket says 172.21.15.32:36320) 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba73e00640 1 -- 172.21.15.32:0/1677858002 learned_addr learned my addr 172.21.15.32:0/1677858002 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba7aa00640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba7c075b70 0x7fba7c1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.495+0000 7fba7b400640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba7c1141f0 0x7fba7c1a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.496+0000 7fba73e00640 1 -- 172.21.15.32:0/1677858002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba7c075b70 msgr2=0x7fba7c1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.496+0000 7fba73e00640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba7c075b70 0x7fba7c1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.496+0000 7fba73e00640 1 -- 172.21.15.32:0/1677858002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba7c1141f0 msgr2=0x7fba7c1a6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.496+0000 7fba73e00640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba7c1141f0 0x7fba7c1a6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.496+0000 7fba73e00640 1 -- 172.21.15.32:0/1677858002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c1a6770 con 0x7fba7c077120 2024-08-31T07:11:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.496+0000 7fba73e00640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c077120 0x7fba7c1a0da0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fba64032fd0 tx=0x7fba64031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.497+0000 7fba73400640 1 -- 172.21.15.32:0/1677858002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6403f030 con 0x7fba7c077120 2024-08-31T07:11:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.497+0000 7fba81600640 1 -- 172.21.15.32:0/1677858002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c1a6a00 con 0x7fba7c077120 2024-08-31T07:11:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.497+0000 7fba73400640 1 -- 172.21.15.32:0/1677858002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba64004490 con 0x7fba7c077120 2024-08-31T07:11:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.497+0000 7fba73400640 1 -- 172.21.15.32:0/1677858002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64035070 con 0x7fba7c077120 2024-08-31T07:11:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.497+0000 7fba81600640 1 -- 172.21.15.32:0/1677858002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c1ad1c0 con 0x7fba7c077120 2024-08-31T07:11:09.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.498+0000 7fba81600640 1 -- 172.21.15.32:0/1677858002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c075f80 con 0x7fba7c077120 2024-08-31T07:11:09.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.501+0000 7fba73400640 1 -- 172.21.15.32:0/1677858002 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fba64042030 con 0x7fba7c077120 2024-08-31T07:11:09.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.502+0000 7fba73400640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fba600b4c10 0x7fba600b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:09.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.503+0000 7fba73400640 1 -- 172.21.15.32:0/1677858002 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fba64105760 con 0x7fba7c077120 2024-08-31T07:11:09.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.503+0000 7fba7aa00640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fba600b4c10 0x7fba600b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:09.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.504+0000 7fba7aa00640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fba600b4c10 0x7fba600b7120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009bd0 tx=0x7fba6c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.507+0000 7fba73400640 1 -- 172.21.15.32:0/1677858002 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fba640cbfe0 con 0x7fba7c077120 2024-08-31T07:11:09.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.720+0000 7fba81600640 1 -- 172.21.15.32:0/1677858002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fba7c112f80 con 0x7fba7c077120 2024-08-31T07:11:09.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.720+0000 7fba73400640 1 -- 172.21.15.32:0/1677858002 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fba640d07a0 con 0x7fba7c077120 2024-08-31T07:11:09.721 INFO:teuthology.orchestra.run.smithi032.stdout:231928234031 2024-08-31T07:11:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.722+0000 7fba81600640 1 -- 172.21.15.32:0/1677858002 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fba600b4c10 msgr2=0x7fba600b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.722+0000 7fba81600640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fba600b4c10 0x7fba600b7120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009bd0 tx=0x7fba6c005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.722+0000 7fba81600640 1 -- 172.21.15.32:0/1677858002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c077120 msgr2=0x7fba7c1a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.722+0000 7fba81600640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c077120 0x7fba7c1a0da0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fba64032fd0 tx=0x7fba64031a20 comp rx=0 tx=0).stop 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.723+0000 7fba81600640 1 -- 172.21.15.32:0/1677858002 shutdown_connections 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.723+0000 7fba81600640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fba600b4c10 0x7fba600b7120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.723+0000 7fba81600640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba7c1141f0 0x7fba7c1a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.723+0000 7fba81600640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7c077120 0x7fba7c1a0da0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.723+0000 7fba81600640 1 --2- 172.21.15.32:0/1677858002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba7c075b70 0x7fba7c1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.723+0000 7fba81600640 1 -- 172.21.15.32:0/1677858002 >> 172.21.15.32:0/1677858002 conn(0x7fba7c0ff330 msgr2=0x7fba7c1100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.723+0000 7fba81600640 1 -- 172.21.15.32:0/1677858002 shutdown_connections 2024-08-31T07:11:09.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:09.723+0000 7fba81600640 1 -- 172.21.15.32:0/1677858002 wait complete. 2024-08-31T07:11:10.088 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:09 smithi032 ceph-mon[17252]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:10.088 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:09 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1677858002' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-31T07:11:10.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:09 smithi073 ceph-mon[21995]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:10.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:09 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1677858002' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-31T07:11:10.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:09 smithi084 ceph-mon[22387]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:10.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:09 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1677858002' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-31T07:11:10.496 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234028 got 231928234031 for osd.8 2024-08-31T07:11:10.496 DEBUG:teuthology.parallel:result is None 2024-08-31T07:11:10.496 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-31T07:11:10.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T07:11:10.921 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:11.316 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:11 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:11:11.316 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:11 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:11:11.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:11 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:11:11.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:11 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:11:11.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:11 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:11:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:11 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:11:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:11 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:11:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:11 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:11:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:11 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:11:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:11 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:11:11.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:11 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:11:11.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:11 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:11:11.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:11 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:11:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:11 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:11:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:11 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:11:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.604+0000 7fc71f200640 1 -- 172.21.15.32:0/2243591585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72010fd10 msgr2=0x7fc720112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.604+0000 7fc71f200640 1 --2- 172.21.15.32:0/2243591585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72010fd10 0x7fc720112130 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc718009ca0 tx=0x7fc71802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.604+0000 7fc71f200640 1 -- 172.21.15.32:0/2243591585 shutdown_connections 2024-08-31T07:11:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.604+0000 7fc71f200640 1 --2- 172.21.15.32:0/2243591585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc72010fd10 0x7fc720112130 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.604+0000 7fc71f200640 1 --2- 172.21.15.32:0/2243591585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc720075e40 0x7fc72010f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.604+0000 7fc71f200640 1 --2- 172.21.15.32:0/2243591585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7200754f0 0x7fc720075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.604+0000 7fc71f200640 1 -- 172.21.15.32:0/2243591585 >> 172.21.15.32:0/2243591585 conn(0x7fc7200ff1a0 msgr2=0x7fc7201015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.604+0000 7fc71f200640 1 -- 172.21.15.32:0/2243591585 shutdown_connections 2024-08-31T07:11:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.604+0000 7fc71f200640 1 -- 172.21.15.32:0/2243591585 wait complete. 2024-08-31T07:11:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.605+0000 7fc71f200640 1 Processor -- start 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71f200640 1 -- start start 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7200754f0 0x7fc72019c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc720075e40 0x7fc72019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc72010fd10 0x7fc7201a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc7201158d0 con 0x7fc7200754f0 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc720115750 con 0x7fc72010fd10 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc720115a50 con 0x7fc720075e40 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7200754f0 0x7fc72019c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7200754f0 0x7fc72019c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36340/0 (socket says 172.21.15.32:36340) 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71de00640 1 -- 172.21.15.32:0/4180677442 learned_addr learned my addr 172.21.15.32:0/4180677442 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc71e800640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc72010fd10 0x7fc7201a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.606+0000 7fc717e00640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc720075e40 0x7fc72019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:11.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.607+0000 7fc71de00640 1 -- 172.21.15.32:0/4180677442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc720075e40 msgr2=0x7fc72019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:11.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.607+0000 7fc71de00640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc720075e40 0x7fc72019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.607+0000 7fc71de00640 1 -- 172.21.15.32:0/4180677442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc72010fd10 msgr2=0x7fc7201a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:11.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.607+0000 7fc71de00640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc72010fd10 0x7fc7201a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.607+0000 7fc71de00640 1 -- 172.21.15.32:0/4180677442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7201a2190 con 0x7fc7200754f0 2024-08-31T07:11:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.607+0000 7fc71de00640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7200754f0 0x7fc72019c220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc71000e9f0 tx=0x7fc71000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.608+0000 7fc716a00640 1 -- 172.21.15.32:0/4180677442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71000cd80 con 0x7fc7200754f0 2024-08-31T07:11:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.608+0000 7fc71f200640 1 -- 172.21.15.32:0/4180677442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7201a2480 con 0x7fc7200754f0 2024-08-31T07:11:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.608+0000 7fc716a00640 1 -- 172.21.15.32:0/4180677442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7100054d0 con 0x7fc7200754f0 2024-08-31T07:11:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.608+0000 7fc716a00640 1 -- 172.21.15.32:0/4180677442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710010690 con 0x7fc7200754f0 2024-08-31T07:11:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.608+0000 7fc71f200640 1 -- 172.21.15.32:0/4180677442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7201a8be0 con 0x7fc7200754f0 2024-08-31T07:11:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.609+0000 7fc71f200640 1 -- 172.21.15.32:0/4180677442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc720075900 con 0x7fc7200754f0 2024-08-31T07:11:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.611+0000 7fc716a00640 1 -- 172.21.15.32:0/4180677442 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fc710004e30 con 0x7fc7200754f0 2024-08-31T07:11:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.613+0000 7fc716a00640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7040b4c10 0x7fc7040b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:11.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.613+0000 7fc716a00640 1 -- 172.21.15.32:0/4180677442 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fc7100e59f0 con 0x7fc7200754f0 2024-08-31T07:11:11.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.613+0000 7fc717e00640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7040b4c10 0x7fc7040b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:11.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.614+0000 7fc717e00640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7040b4c10 0x7fc7040b7120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc720112290 tx=0x7fc70800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:11.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.615+0000 7fc716a00640 1 -- 172.21.15.32:0/4180677442 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc7100ac8c0 con 0x7fc7200754f0 2024-08-31T07:11:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.809+0000 7fc71f200640 1 -- 172.21.15.32:0/4180677442 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc720101210 con 0x7fc7040b4c10 2024-08-31T07:11:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.817+0000 7fc716a00640 1 -- 172.21.15.32:0/4180677442 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259011 (secure 0 0 0) 0x7fc720101210 con 0x7fc7040b4c10 2024-08-31T07:11:11.821 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:11.822 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-08-31T07:11:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.821+0000 7fc71f200640 1 -- 172.21.15.32:0/4180677442 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7040b4c10 msgr2=0x7fc7040b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.821+0000 7fc71f200640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7040b4c10 0x7fc7040b7120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc720112290 tx=0x7fc70800b390 comp rx=0 tx=0).stop 2024-08-31T07:11:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.821+0000 7fc71f200640 1 -- 172.21.15.32:0/4180677442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7200754f0 msgr2=0x7fc72019c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.821+0000 7fc71f200640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7200754f0 0x7fc72019c220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc71000e9f0 tx=0x7fc71000edf0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.822+0000 7fc71f200640 1 -- 172.21.15.32:0/4180677442 shutdown_connections 2024-08-31T07:11:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.822+0000 7fc71f200640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7040b4c10 0x7fc7040b7120 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.822+0000 7fc71f200640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc72010fd10 0x7fc7201a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.822+0000 7fc71f200640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc720075e40 0x7fc72019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.822+0000 7fc71f200640 1 --2- 172.21.15.32:0/4180677442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7200754f0 0x7fc72019c220 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.822+0000 7fc71f200640 1 -- 172.21.15.32:0/4180677442 >> 172.21.15.32:0/4180677442 conn(0x7fc7200ff1a0 msgr2=0x7fc720100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.822+0000 7fc71f200640 1 -- 172.21.15.32:0/4180677442 shutdown_connections 2024-08-31T07:11:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:11.822+0000 7fc71f200640 1 -- 172.21.15.32:0/4180677442 wait complete. 2024-08-31T07:11:12.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:12 smithi073 ceph-mon[21995]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:12 smithi084 ceph-mon[22387]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:12.549 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:12 smithi032 ceph-mon[17252]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:12.588 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-08-31T07:11:10.779365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659205,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":764,"num_read_kb":486,"num_write":372,"num_write_kb":2289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2952490,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364276,"kb_used_data":10800,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577548,"statfs":{"total":1151940427776,"available":1151567409152,"internally_reserved":0,"allocated":11059200,"data_stored":7163055,"data_compressed":177834,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003475"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018337+0000","last_change":"2024-08-31T07:09:47.665725+0000","last_active":"2024-08-31T07:10:46.018337+0000","last_peered":"2024-08-31T07:10:46.018337+0000","last_clean":"2024-08-31T07:10:46.018337+0000","last_became_active":"2024-08-31T07:09:47.665368+0000","last_became_peered":"2024-08-31T07:09:47.665368+0000","last_unstale":"2024-08-31T07:10:46.018337+0000","last_undegraded":"2024-08-31T07:10:46.018337+0000","last_fullsized":"2024-08-31T07:10:46.018337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:27:01.636099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324615+0000","last_change":"2024-08-31T07:09:47.676882+0000","last_active":"2024-08-31T07:10:46.324615+0000","last_peered":"2024-08-31T07:10:46.324615+0000","last_clean":"2024-08-31T07:10:46.324615+0000","last_became_active":"2024-08-31T07:09:47.674825+0000","last_became_peered":"2024-08-31T07:09:47.674825+0000","last_unstale":"2024-08-31T07:10:46.324615+0000","last_undegraded":"2024-08-31T07:10:46.324615+0000","last_fullsized":"2024-08-31T07:10:46.324615+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:09.148398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.323881+0000","last_change":"2024-08-31T07:09:47.674480+0000","last_active":"2024-08-31T07:10:46.323881+0000","last_peered":"2024-08-31T07:10:46.323881+0000","last_clean":"2024-08-31T07:10:46.323881+0000","last_became_active":"2024-08-31T07:09:47.672316+0000","last_became_peered":"2024-08-31T07:09:47.672316+0000","last_unstale":"2024-08-31T07:10:46.323881+0000","last_undegraded":"2024-08-31T07:10:46.323881+0000","last_fullsized":"2024-08-31T07:10:46.323881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:28:13.231039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.153980+0000","last_change":"2024-08-31T07:09:47.659276+0000","last_active":"2024-08-31T07:10:46.153980+0000","last_peered":"2024-08-31T07:10:46.153980+0000","last_clean":"2024-08-31T07:10:46.153980+0000","last_became_active":"2024-08-31T07:09:47.658924+0000","last_became_peered":"2024-08-31T07:09:47.658924+0000","last_unstale":"2024-08-31T07:10:46.153980+0000","last_undegraded":"2024-08-31T07:10:46.153980+0000","last_fullsized":"2024-08-31T07:10:46.153980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:01:08.134245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021006+0000","last_change":"2024-08-31T07:09:47.667260+0000","last_active":"2024-08-31T07:10:46.021006+0000","last_peered":"2024-08-31T07:10:46.021006+0000","last_clean":"2024-08-31T07:10:46.021006+0000","last_became_active":"2024-08-31T07:09:47.664572+0000","last_became_peered":"2024-08-31T07:09:47.664572+0000","last_unstale":"2024-08-31T07:10:46.021006+0000","last_undegraded":"2024-08-31T07:10:46.021006+0000","last_fullsized":"2024-08-31T07:10:46.021006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:11.336985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019758+0000","last_change":"2024-08-31T07:09:47.661181+0000","last_active":"2024-08-31T07:10:46.019758+0000","last_peered":"2024-08-31T07:10:46.019758+0000","last_clean":"2024-08-31T07:10:46.019758+0000","last_became_active":"2024-08-31T07:09:47.660761+0000","last_became_peered":"2024-08-31T07:09:47.660761+0000","last_unstale":"2024-08-31T07:10:46.019758+0000","last_undegraded":"2024-08-31T07:10:46.019758+0000","last_fullsized":"2024-08-31T07:10:46.019758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:50:54.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257002+0000","last_change":"2024-08-31T07:09:47.668762+0000","last_active":"2024-08-31T07:10:46.257002+0000","last_peered":"2024-08-31T07:10:46.257002+0000","last_clean":"2024-08-31T07:10:46.257002+0000","last_became_active":"2024-08-31T07:09:47.668406+0000","last_became_peered":"2024-08-31T07:09:47.668406+0000","last_unstale":"2024-08-31T07:10:46.257002+0000","last_undegraded":"2024-08-31T07:10:46.257002+0000","last_fullsized":"2024-08-31T07:10:46.257002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:43:39.868144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015452+0000","last_change":"2024-08-31T07:09:47.669455+0000","last_active":"2024-08-31T07:10:46.015452+0000","last_peered":"2024-08-31T07:10:46.015452+0000","last_clean":"2024-08-31T07:10:46.015452+0000","last_became_active":"2024-08-31T07:09:47.663669+0000","last_became_peered":"2024-08-31T07:09:47.663669+0000","last_unstale":"2024-08-31T07:10:46.015452+0000","last_undegraded":"2024-08-31T07:10:46.015452+0000","last_fullsized":"2024-08-31T07:10:46.015452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:22:59.685012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016753+0000","last_change":"2024-08-31T07:09:47.664214+0000","last_active":"2024-08-31T07:10:46.016753+0000","last_peered":"2024-08-31T07:10:46.016753+0000","last_clean":"2024-08-31T07:10:46.016753+0000","last_became_active":"2024-08-31T07:09:47.663832+0000","last_became_peered":"2024-08-31T07:09:47.663832+0000","last_unstale":"2024-08-31T07:10:46.016753+0000","last_undegraded":"2024-08-31T07:10:46.016753+0000","last_fullsized":"2024-08-31T07:10:46.016753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:29:14.107212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.326515+0000","last_change":"2024-08-31T07:10:46.326410+0000","last_active":"2024-08-31T07:10:46.326515+0000","last_peered":"2024-08-31T07:10:46.326515+0000","last_clean":"2024-08-31T07:10:46.326515+0000","last_became_active":"2024-08-31T07:10:46.325958+0000","last_became_peered":"2024-08-31T07:10:46.325958+0000","last_unstale":"2024-08-31T07:10:46.326515+0000","last_undegraded":"2024-08-31T07:10:46.326515+0000","last_fullsized":"2024-08-31T07:10:46.326515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:19.230393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257100+0000","last_change":"2024-08-31T07:09:47.666951+0000","last_active":"2024-08-31T07:10:46.257100+0000","last_peered":"2024-08-31T07:10:46.257100+0000","last_clean":"2024-08-31T07:10:46.257100+0000","last_became_active":"2024-08-31T07:09:47.666475+0000","last_became_peered":"2024-08-31T07:09:47.666475+0000","last_unstale":"2024-08-31T07:10:46.257100+0000","last_undegraded":"2024-08-31T07:10:46.257100+0000","last_fullsized":"2024-08-31T07:10:46.257100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:58:42.242086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154138+0000","last_change":"2024-08-31T07:09:47.658344+0000","last_active":"2024-08-31T07:10:46.154138+0000","last_peered":"2024-08-31T07:10:46.154138+0000","last_clean":"2024-08-31T07:10:46.154138+0000","last_became_active":"2024-08-31T07:09:47.657991+0000","last_became_peered":"2024-08-31T07:09:47.657991+0000","last_unstale":"2024-08-31T07:10:46.154138+0000","last_undegraded":"2024-08-31T07:10:46.154138+0000","last_fullsized":"2024-08-31T07:10:46.154138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:25:59.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018102+0000","last_change":"2024-08-31T07:09:47.661644+0000","last_active":"2024-08-31T07:10:46.018102+0000","last_peered":"2024-08-31T07:10:46.018102+0000","last_clean":"2024-08-31T07:10:46.018102+0000","last_became_active":"2024-08-31T07:09:47.661248+0000","last_became_peered":"2024-08-31T07:09:47.661248+0000","last_unstale":"2024-08-31T07:10:46.018102+0000","last_undegraded":"2024-08-31T07:10:46.018102+0000","last_fullsized":"2024-08-31T07:10:46.018102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:42.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.323555+0000","last_change":"2024-08-31T07:09:47.680052+0000","last_active":"2024-08-31T07:10:46.323555+0000","last_peered":"2024-08-31T07:10:46.323555+0000","last_clean":"2024-08-31T07:10:46.323555+0000","last_became_active":"2024-08-31T07:09:47.679604+0000","last_became_peered":"2024-08-31T07:09:47.679604+0000","last_unstale":"2024-08-31T07:10:46.323555+0000","last_undegraded":"2024-08-31T07:10:46.323555+0000","last_fullsized":"2024-08-31T07:10:46.323555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:23:18.937470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.323337+0000","last_change":"2024-08-31T07:09:47.677984+0000","last_active":"2024-08-31T07:10:46.323337+0000","last_peered":"2024-08-31T07:10:46.323337+0000","last_clean":"2024-08-31T07:10:46.323337+0000","last_became_active":"2024-08-31T07:09:47.674957+0000","last_became_peered":"2024-08-31T07:09:47.674957+0000","last_unstale":"2024-08-31T07:10:46.323337+0000","last_undegraded":"2024-08-31T07:10:46.323337+0000","last_fullsized":"2024-08-31T07:10:46.323337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:08:26.503746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016112+0000","last_change":"2024-08-31T07:09:47.672625+0000","last_active":"2024-08-31T07:10:46.016112+0000","last_peered":"2024-08-31T07:10:46.016112+0000","last_clean":"2024-08-31T07:10:46.016112+0000","last_became_active":"2024-08-31T07:09:47.672184+0000","last_became_peered":"2024-08-31T07:09:47.672184+0000","last_unstale":"2024-08-31T07:10:46.016112+0000","last_undegraded":"2024-08-31T07:10:46.016112+0000","last_fullsized":"2024-08-31T07:10:46.016112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:17.091659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018341+0000","last_change":"2024-08-31T07:09:47.672597+0000","last_active":"2024-08-31T07:10:46.018341+0000","last_peered":"2024-08-31T07:10:46.018341+0000","last_clean":"2024-08-31T07:10:46.018341+0000","last_became_active":"2024-08-31T07:09:47.672153+0000","last_became_peered":"2024-08-31T07:09:47.672153+0000","last_unstale":"2024-08-31T07:10:46.018341+0000","last_undegraded":"2024-08-31T07:10:46.018341+0000","last_fullsized":"2024-08-31T07:10:46.018341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:26:41.388526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324552+0000","last_change":"2024-08-31T07:09:47.666253+0000","last_active":"2024-08-31T07:10:46.324552+0000","last_peered":"2024-08-31T07:10:46.324552+0000","last_clean":"2024-08-31T07:10:46.324552+0000","last_became_active":"2024-08-31T07:09:47.663732+0000","last_became_peered":"2024-08-31T07:09:47.663732+0000","last_unstale":"2024-08-31T07:10:46.324552+0000","last_undegraded":"2024-08-31T07:10:46.324552+0000","last_fullsized":"2024-08-31T07:10:46.324552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:10:59.887765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023639+0000","last_change":"2024-08-31T07:09:47.668502+0000","last_active":"2024-08-31T07:10:46.023639+0000","last_peered":"2024-08-31T07:10:46.023639+0000","last_clean":"2024-08-31T07:10:46.023639+0000","last_became_active":"2024-08-31T07:09:47.666487+0000","last_became_peered":"2024-08-31T07:09:47.666487+0000","last_unstale":"2024-08-31T07:10:46.023639+0000","last_undegraded":"2024-08-31T07:10:46.023639+0000","last_fullsized":"2024-08-31T07:10:46.023639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:49:25.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018869+0000","last_change":"2024-08-31T07:09:47.659454+0000","last_active":"2024-08-31T07:10:46.018869+0000","last_peered":"2024-08-31T07:10:46.018869+0000","last_clean":"2024-08-31T07:10:46.018869+0000","last_became_active":"2024-08-31T07:09:47.658976+0000","last_became_peered":"2024-08-31T07:09:47.658976+0000","last_unstale":"2024-08-31T07:10:46.018869+0000","last_undegraded":"2024-08-31T07:10:46.018869+0000","last_fullsized":"2024-08-31T07:10:46.018869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:31.374905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018210+0000","last_change":"2024-08-31T07:09:47.673612+0000","last_active":"2024-08-31T07:10:46.018210+0000","last_peered":"2024-08-31T07:10:46.018210+0000","last_clean":"2024-08-31T07:10:46.018210+0000","last_became_active":"2024-08-31T07:09:47.673137+0000","last_became_peered":"2024-08-31T07:09:47.673137+0000","last_unstale":"2024-08-31T07:10:46.018210+0000","last_undegraded":"2024-08-31T07:10:46.018210+0000","last_fullsized":"2024-08-31T07:10:46.018210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:46:11.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016333+0000","last_change":"2024-08-31T07:09:47.669195+0000","last_active":"2024-08-31T07:10:46.016333+0000","last_peered":"2024-08-31T07:10:46.016333+0000","last_clean":"2024-08-31T07:10:46.016333+0000","last_became_active":"2024-08-31T07:09:47.668790+0000","last_became_peered":"2024-08-31T07:09:47.668790+0000","last_unstale":"2024-08-31T07:10:46.016333+0000","last_undegraded":"2024-08-31T07:10:46.016333+0000","last_fullsized":"2024-08-31T07:10:46.016333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:27:14.393931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015464+0000","last_change":"2024-08-31T07:09:47.667566+0000","last_active":"2024-08-31T07:10:46.015464+0000","last_peered":"2024-08-31T07:10:46.015464+0000","last_clean":"2024-08-31T07:10:46.015464+0000","last_became_active":"2024-08-31T07:09:47.666605+0000","last_became_peered":"2024-08-31T07:09:47.666605+0000","last_unstale":"2024-08-31T07:10:46.015464+0000","last_undegraded":"2024-08-31T07:10:46.015464+0000","last_fullsized":"2024-08-31T07:10:46.015464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:05:44.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022143+0000","last_change":"2024-08-31T07:09:47.660478+0000","last_active":"2024-08-31T07:10:46.022143+0000","last_peered":"2024-08-31T07:10:46.022143+0000","last_clean":"2024-08-31T07:10:46.022143+0000","last_became_active":"2024-08-31T07:09:47.660107+0000","last_became_peered":"2024-08-31T07:09:47.660107+0000","last_unstale":"2024-08-31T07:10:46.022143+0000","last_undegraded":"2024-08-31T07:10:46.022143+0000","last_fullsized":"2024-08-31T07:10:46.022143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:29.133370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016418+0000","last_change":"2024-08-31T07:09:47.671967+0000","last_active":"2024-08-31T07:10:46.016418+0000","last_peered":"2024-08-31T07:10:46.016418+0000","last_clean":"2024-08-31T07:10:46.016418+0000","last_became_active":"2024-08-31T07:09:47.671569+0000","last_became_peered":"2024-08-31T07:09:47.671569+0000","last_unstale":"2024-08-31T07:10:46.016418+0000","last_undegraded":"2024-08-31T07:10:46.016418+0000","last_fullsized":"2024-08-31T07:10:46.016418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:19:15.107958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023641+0000","last_change":"2024-08-31T07:09:47.651055+0000","last_active":"2024-08-31T07:10:46.023641+0000","last_peered":"2024-08-31T07:10:46.023641+0000","last_clean":"2024-08-31T07:10:46.023641+0000","last_became_active":"2024-08-31T07:09:47.650674+0000","last_became_peered":"2024-08-31T07:09:47.650674+0000","last_unstale":"2024-08-31T07:10:46.023641+0000","last_undegraded":"2024-08-31T07:10:46.023641+0000","last_fullsized":"2024-08-31T07:10:46.023641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:35:54.647379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018236+0000","last_change":"2024-08-31T07:09:47.663231+0000","last_active":"2024-08-31T07:10:46.018236+0000","last_peered":"2024-08-31T07:10:46.018236+0000","last_clean":"2024-08-31T07:10:46.018236+0000","last_became_active":"2024-08-31T07:09:47.662854+0000","last_became_peered":"2024-08-31T07:09:47.662854+0000","last_unstale":"2024-08-31T07:10:46.018236+0000","last_undegraded":"2024-08-31T07:10:46.018236+0000","last_fullsized":"2024-08-31T07:10:46.018236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:51.358557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.326495+0000","last_change":"2024-08-31T07:10:46.326380+0000","last_active":"2024-08-31T07:10:46.326495+0000","last_peered":"2024-08-31T07:10:46.326495+0000","last_clean":"2024-08-31T07:10:46.326495+0000","last_became_active":"2024-08-31T07:10:46.325784+0000","last_became_peered":"2024-08-31T07:10:46.325784+0000","last_unstale":"2024-08-31T07:10:46.326495+0000","last_undegraded":"2024-08-31T07:10:46.326495+0000","last_fullsized":"2024-08-31T07:10:46.326495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:16:56.396024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.323995+0000","last_change":"2024-08-31T07:09:47.672108+0000","last_active":"2024-08-31T07:10:46.323995+0000","last_peered":"2024-08-31T07:10:46.323995+0000","last_clean":"2024-08-31T07:10:46.323995+0000","last_became_active":"2024-08-31T07:09:47.670998+0000","last_became_peered":"2024-08-31T07:09:47.670998+0000","last_unstale":"2024-08-31T07:10:46.323995+0000","last_undegraded":"2024-08-31T07:10:46.323995+0000","last_fullsized":"2024-08-31T07:10:46.323995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:26:49.319861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022405+0000","last_change":"2024-08-31T07:09:47.665548+0000","last_active":"2024-08-31T07:10:46.022405+0000","last_peered":"2024-08-31T07:10:46.022405+0000","last_clean":"2024-08-31T07:10:46.022405+0000","last_became_active":"2024-08-31T07:09:47.665177+0000","last_became_peered":"2024-08-31T07:09:47.665177+0000","last_unstale":"2024-08-31T07:10:46.022405+0000","last_undegraded":"2024-08-31T07:10:46.022405+0000","last_fullsized":"2024-08-31T07:10:46.022405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:03:15.892224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.026863+0000","last_change":"2024-08-31T07:10:46.026746+0000","last_active":"2024-08-31T07:10:46.026863+0000","last_peered":"2024-08-31T07:10:46.026863+0000","last_clean":"2024-08-31T07:10:46.026863+0000","last_became_active":"2024-08-31T07:10:46.026286+0000","last_became_peered":"2024-08-31T07:10:46.026286+0000","last_unstale":"2024-08-31T07:10:46.026863+0000","last_undegraded":"2024-08-31T07:10:46.026863+0000","last_fullsized":"2024-08-31T07:10:46.026863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:34:17.120906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017936+0000","last_change":"2024-08-31T07:09:47.667456+0000","last_active":"2024-08-31T07:10:46.017936+0000","last_peered":"2024-08-31T07:10:46.017936+0000","last_clean":"2024-08-31T07:10:46.017936+0000","last_became_active":"2024-08-31T07:09:47.667102+0000","last_became_peered":"2024-08-31T07:09:47.667102+0000","last_unstale":"2024-08-31T07:10:46.017936+0000","last_undegraded":"2024-08-31T07:10:46.017936+0000","last_fullsized":"2024-08-31T07:10:46.017936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:44:39.747729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.326260+0000","last_change":"2024-08-31T07:10:46.326139+0000","last_active":"2024-08-31T07:10:46.326260+0000","last_peered":"2024-08-31T07:10:46.326260+0000","last_clean":"2024-08-31T07:10:46.326260+0000","last_became_active":"2024-08-31T07:10:46.325785+0000","last_became_peered":"2024-08-31T07:10:46.325785+0000","last_unstale":"2024-08-31T07:10:46.326260+0000","last_undegraded":"2024-08-31T07:10:46.326260+0000","last_fullsized":"2024-08-31T07:10:46.326260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:13.495835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":97,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:04.980876+0000","last_change":"2024-08-31T07:09:45.641088+0000","last_active":"2024-08-31T07:11:04.980876+0000","last_peered":"2024-08-31T07:11:04.980876+0000","last_clean":"2024-08-31T07:11:04.980876+0000","last_became_active":"2024-08-31T07:09:45.640673+0000","last_became_peered":"2024-08-31T07:09:45.640673+0000","last_unstale":"2024-08-31T07:11:04.980876+0000","last_undegraded":"2024-08-31T07:11:04.980876+0000","last_fullsized":"2024-08-31T07:11:04.980876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:30:17.593494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022664+0000","last_change":"2024-08-31T07:09:47.666703+0000","last_active":"2024-08-31T07:10:46.022664+0000","last_peered":"2024-08-31T07:10:46.022664+0000","last_clean":"2024-08-31T07:10:46.022664+0000","last_became_active":"2024-08-31T07:09:47.666348+0000","last_became_peered":"2024-08-31T07:09:47.666348+0000","last_unstale":"2024-08-31T07:10:46.022664+0000","last_undegraded":"2024-08-31T07:10:46.022664+0000","last_fullsized":"2024-08-31T07:10:46.022664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:39.688810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019815+0000","last_change":"2024-08-31T07:09:45.642431+0000","last_active":"2024-08-31T07:10:46.019815+0000","last_peered":"2024-08-31T07:10:46.019815+0000","last_clean":"2024-08-31T07:10:46.019815+0000","last_became_active":"2024-08-31T07:09:45.642048+0000","last_became_peered":"2024-08-31T07:09:45.642048+0000","last_unstale":"2024-08-31T07:10:46.019815+0000","last_undegraded":"2024-08-31T07:10:46.019815+0000","last_fullsized":"2024-08-31T07:10:46.019815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:47:42.574442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017223+0000","last_change":"2024-08-31T07:09:45.632162+0000","last_active":"2024-08-31T07:10:46.017223+0000","last_peered":"2024-08-31T07:10:46.017223+0000","last_clean":"2024-08-31T07:10:46.017223+0000","last_became_active":"2024-08-31T07:09:45.631749+0000","last_became_peered":"2024-08-31T07:09:45.631749+0000","last_unstale":"2024-08-31T07:10:46.017223+0000","last_undegraded":"2024-08-31T07:10:46.017223+0000","last_fullsized":"2024-08-31T07:10:46.017223+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:16:29.967823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015529+0000","last_change":"2024-08-31T07:09:47.666288+0000","last_active":"2024-08-31T07:10:46.015529+0000","last_peered":"2024-08-31T07:10:46.015529+0000","last_clean":"2024-08-31T07:10:46.015529+0000","last_became_active":"2024-08-31T07:09:47.665723+0000","last_became_peered":"2024-08-31T07:09:47.665723+0000","last_unstale":"2024-08-31T07:10:46.015529+0000","last_undegraded":"2024-08-31T07:10:46.015529+0000","last_fullsized":"2024-08-31T07:10:46.015529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:38:28.122720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023048+0000","last_change":"2024-08-31T07:09:45.636508+0000","last_active":"2024-08-31T07:10:46.023048+0000","last_peered":"2024-08-31T07:10:46.023048+0000","last_clean":"2024-08-31T07:10:46.023048+0000","last_became_active":"2024-08-31T07:09:45.636073+0000","last_became_peered":"2024-08-31T07:09:45.636073+0000","last_unstale":"2024-08-31T07:10:46.023048+0000","last_undegraded":"2024-08-31T07:10:46.023048+0000","last_fullsized":"2024-08-31T07:10:46.023048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:55:45.063517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.323509+0000","last_change":"2024-08-31T07:09:47.661306+0000","last_active":"2024-08-31T07:10:46.323509+0000","last_peered":"2024-08-31T07:10:46.323509+0000","last_clean":"2024-08-31T07:10:46.323509+0000","last_became_active":"2024-08-31T07:09:47.660957+0000","last_became_peered":"2024-08-31T07:09:47.660957+0000","last_unstale":"2024-08-31T07:10:46.323509+0000","last_undegraded":"2024-08-31T07:10:46.323509+0000","last_fullsized":"2024-08-31T07:10:46.323509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:18:23.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":155,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:05.981678+0000","last_change":"2024-08-31T07:09:45.640947+0000","last_active":"2024-08-31T07:11:05.981678+0000","last_peered":"2024-08-31T07:11:05.981678+0000","last_clean":"2024-08-31T07:11:05.981678+0000","last_became_active":"2024-08-31T07:09:45.640532+0000","last_became_peered":"2024-08-31T07:09:45.640532+0000","last_unstale":"2024-08-31T07:11:05.981678+0000","last_undegraded":"2024-08-31T07:11:05.981678+0000","last_fullsized":"2024-08-31T07:11:05.981678+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:28.177795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016603+0000","last_change":"2024-08-31T07:09:47.661060+0000","last_active":"2024-08-31T07:10:46.016603+0000","last_peered":"2024-08-31T07:10:46.016603+0000","last_clean":"2024-08-31T07:10:46.016603+0000","last_became_active":"2024-08-31T07:09:47.660692+0000","last_became_peered":"2024-08-31T07:09:47.660692+0000","last_unstale":"2024-08-31T07:10:46.016603+0000","last_undegraded":"2024-08-31T07:10:46.016603+0000","last_fullsized":"2024-08-31T07:10:46.016603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:48:38.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257131+0000","last_change":"2024-08-31T07:09:45.641155+0000","last_active":"2024-08-31T07:10:46.257131+0000","last_peered":"2024-08-31T07:10:46.257131+0000","last_clean":"2024-08-31T07:10:46.257131+0000","last_became_active":"2024-08-31T07:09:45.640680+0000","last_became_peered":"2024-08-31T07:09:45.640680+0000","last_unstale":"2024-08-31T07:10:46.257131+0000","last_undegraded":"2024-08-31T07:10:46.257131+0000","last_fullsized":"2024-08-31T07:10:46.257131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:29:03.287570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324342+0000","last_change":"2024-08-31T07:09:47.675582+0000","last_active":"2024-08-31T07:10:46.324342+0000","last_peered":"2024-08-31T07:10:46.324342+0000","last_clean":"2024-08-31T07:10:46.324342+0000","last_became_active":"2024-08-31T07:09:47.675266+0000","last_became_peered":"2024-08-31T07:09:47.675266+0000","last_unstale":"2024-08-31T07:10:46.324342+0000","last_undegraded":"2024-08-31T07:10:46.324342+0000","last_fullsized":"2024-08-31T07:10:46.324342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:36:33.122531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021129+0000","last_change":"2024-08-31T07:09:45.651006+0000","last_active":"2024-08-31T07:10:46.021129+0000","last_peered":"2024-08-31T07:10:46.021129+0000","last_clean":"2024-08-31T07:10:46.021129+0000","last_became_active":"2024-08-31T07:09:45.650588+0000","last_became_peered":"2024-08-31T07:09:45.650588+0000","last_unstale":"2024-08-31T07:10:46.021129+0000","last_undegraded":"2024-08-31T07:10:46.021129+0000","last_fullsized":"2024-08-31T07:10:46.021129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:34:16.015892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016246+0000","last_change":"2024-08-31T07:09:47.662289+0000","last_active":"2024-08-31T07:10:46.016246+0000","last_peered":"2024-08-31T07:10:46.016246+0000","last_clean":"2024-08-31T07:10:46.016246+0000","last_became_active":"2024-08-31T07:09:47.661932+0000","last_became_peered":"2024-08-31T07:09:47.661932+0000","last_unstale":"2024-08-31T07:10:46.016246+0000","last_undegraded":"2024-08-31T07:10:46.016246+0000","last_fullsized":"2024-08-31T07:10:46.016246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:50.440504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'31","reported_seq":191,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:05.981679+0000","last_change":"2024-08-31T07:09:45.642196+0000","last_active":"2024-08-31T07:11:05.981679+0000","last_peered":"2024-08-31T07:11:05.981679+0000","last_clean":"2024-08-31T07:11:05.981679+0000","last_became_active":"2024-08-31T07:09:45.641797+0000","last_became_peered":"2024-08-31T07:09:45.641797+0000","last_unstale":"2024-08-31T07:11:05.981679+0000","last_undegraded":"2024-08-31T07:11:05.981679+0000","last_fullsized":"2024-08-31T07:11:05.981679+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:51:05.953335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60617,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":135,"num_read_kb":126,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020491+0000","last_change":"2024-08-31T07:09:47.660184+0000","last_active":"2024-08-31T07:10:46.020491+0000","last_peered":"2024-08-31T07:10:46.020491+0000","last_clean":"2024-08-31T07:10:46.020491+0000","last_became_active":"2024-08-31T07:09:47.659571+0000","last_became_peered":"2024-08-31T07:09:47.659571+0000","last_unstale":"2024-08-31T07:10:46.020491+0000","last_undegraded":"2024-08-31T07:10:46.020491+0000","last_fullsized":"2024-08-31T07:10:46.020491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:41:13.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.035782+0000","last_change":"2024-08-31T07:10:46.035336+0000","last_active":"2024-08-31T07:10:46.035782+0000","last_peered":"2024-08-31T07:10:46.035782+0000","last_clean":"2024-08-31T07:10:46.035782+0000","last_became_active":"2024-08-31T07:10:46.021299+0000","last_became_peered":"2024-08-31T07:10:46.021299+0000","last_unstale":"2024-08-31T07:10:46.035782+0000","last_undegraded":"2024-08-31T07:10:46.035782+0000","last_fullsized":"2024-08-31T07:10:46.035782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:06:15.931855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.326482+0000","last_change":"2024-08-31T07:10:46.326354+0000","last_active":"2024-08-31T07:10:46.326482+0000","last_peered":"2024-08-31T07:10:46.326482+0000","last_clean":"2024-08-31T07:10:46.326482+0000","last_became_active":"2024-08-31T07:10:46.325869+0000","last_became_peered":"2024-08-31T07:10:46.325869+0000","last_unstale":"2024-08-31T07:10:46.326482+0000","last_undegraded":"2024-08-31T07:10:46.326482+0000","last_fullsized":"2024-08-31T07:10:46.326482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:21.832088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022856+0000","last_change":"2024-08-31T07:09:45.642187+0000","last_active":"2024-08-31T07:10:46.022856+0000","last_peered":"2024-08-31T07:10:46.022856+0000","last_clean":"2024-08-31T07:10:46.022856+0000","last_became_active":"2024-08-31T07:09:45.641721+0000","last_became_peered":"2024-08-31T07:09:45.641721+0000","last_unstale":"2024-08-31T07:10:46.022856+0000","last_undegraded":"2024-08-31T07:10:46.022856+0000","last_fullsized":"2024-08-31T07:10:46.022856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:00:45.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257256+0000","last_change":"2024-08-31T07:09:47.665956+0000","last_active":"2024-08-31T07:10:46.257256+0000","last_peered":"2024-08-31T07:10:46.257256+0000","last_clean":"2024-08-31T07:10:46.257256+0000","last_became_active":"2024-08-31T07:09:47.664367+0000","last_became_peered":"2024-08-31T07:09:47.664367+0000","last_unstale":"2024-08-31T07:10:46.257256+0000","last_undegraded":"2024-08-31T07:10:46.257256+0000","last_fullsized":"2024-08-31T07:10:46.257256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:34.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":97,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:04.983351+0000","last_change":"2024-08-31T07:09:45.636440+0000","last_active":"2024-08-31T07:11:04.983351+0000","last_peered":"2024-08-31T07:11:04.983351+0000","last_clean":"2024-08-31T07:11:04.983351+0000","last_became_active":"2024-08-31T07:09:45.636058+0000","last_became_peered":"2024-08-31T07:09:45.636058+0000","last_unstale":"2024-08-31T07:11:04.983351+0000","last_undegraded":"2024-08-31T07:11:04.983351+0000","last_fullsized":"2024-08-31T07:11:04.983351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:38.929227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.256870+0000","last_change":"2024-08-31T07:09:47.660564+0000","last_active":"2024-08-31T07:10:46.256870+0000","last_peered":"2024-08-31T07:10:46.256870+0000","last_clean":"2024-08-31T07:10:46.256870+0000","last_became_active":"2024-08-31T07:09:47.660157+0000","last_became_peered":"2024-08-31T07:09:47.660157+0000","last_unstale":"2024-08-31T07:10:46.256870+0000","last_undegraded":"2024-08-31T07:10:46.256870+0000","last_fullsized":"2024-08-31T07:10:46.256870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:49:01.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020455+0000","last_change":"2024-08-31T07:09:45.649140+0000","last_active":"2024-08-31T07:10:46.020455+0000","last_peered":"2024-08-31T07:10:46.020455+0000","last_clean":"2024-08-31T07:10:46.020455+0000","last_became_active":"2024-08-31T07:09:45.648796+0000","last_became_peered":"2024-08-31T07:09:45.648796+0000","last_unstale":"2024-08-31T07:10:46.020455+0000","last_undegraded":"2024-08-31T07:10:46.020455+0000","last_fullsized":"2024-08-31T07:10:46.020455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:33:37.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023205+0000","last_change":"2024-08-31T07:09:47.663833+0000","last_active":"2024-08-31T07:10:46.023205+0000","last_peered":"2024-08-31T07:10:46.023205+0000","last_clean":"2024-08-31T07:10:46.023205+0000","last_became_active":"2024-08-31T07:09:47.663451+0000","last_became_peered":"2024-08-31T07:09:47.663451+0000","last_unstale":"2024-08-31T07:10:46.023205+0000","last_undegraded":"2024-08-31T07:10:46.023205+0000","last_fullsized":"2024-08-31T07:10:46.023205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:27:22.245254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021332+0000","last_change":"2024-08-31T07:09:45.636593+0000","last_active":"2024-08-31T07:10:46.021332+0000","last_peered":"2024-08-31T07:10:46.021332+0000","last_clean":"2024-08-31T07:10:46.021332+0000","last_became_active":"2024-08-31T07:09:45.636166+0000","last_became_peered":"2024-08-31T07:09:45.636166+0000","last_unstale":"2024-08-31T07:10:46.021332+0000","last_undegraded":"2024-08-31T07:10:46.021332+0000","last_fullsized":"2024-08-31T07:10:46.021332+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:28.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018126+0000","last_change":"2024-08-31T07:09:47.660825+0000","last_active":"2024-08-31T07:10:46.018126+0000","last_peered":"2024-08-31T07:10:46.018126+0000","last_clean":"2024-08-31T07:10:46.018126+0000","last_became_active":"2024-08-31T07:09:47.660474+0000","last_became_peered":"2024-08-31T07:09:47.660474+0000","last_unstale":"2024-08-31T07:10:46.018126+0000","last_undegraded":"2024-08-31T07:10:46.018126+0000","last_fullsized":"2024-08-31T07:10:46.018126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:42:20.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017406+0000","last_change":"2024-08-31T07:09:45.632727+0000","last_active":"2024-08-31T07:10:46.017406+0000","last_peered":"2024-08-31T07:10:46.017406+0000","last_clean":"2024-08-31T07:10:46.017406+0000","last_became_active":"2024-08-31T07:09:45.632251+0000","last_became_peered":"2024-08-31T07:09:45.632251+0000","last_unstale":"2024-08-31T07:10:46.017406+0000","last_undegraded":"2024-08-31T07:10:46.017406+0000","last_fullsized":"2024-08-31T07:10:46.017406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:09:30.197409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257404+0000","last_change":"2024-08-31T07:09:47.665501+0000","last_active":"2024-08-31T07:10:46.257404+0000","last_peered":"2024-08-31T07:10:46.257404+0000","last_clean":"2024-08-31T07:10:46.257404+0000","last_became_active":"2024-08-31T07:09:47.665162+0000","last_became_peered":"2024-08-31T07:09:47.665162+0000","last_unstale":"2024-08-31T07:10:46.257404+0000","last_undegraded":"2024-08-31T07:10:46.257404+0000","last_fullsized":"2024-08-31T07:10:46.257404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:34:29.605836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021262+0000","last_change":"2024-08-31T07:09:45.636580+0000","last_active":"2024-08-31T07:10:46.021262+0000","last_peered":"2024-08-31T07:10:46.021262+0000","last_clean":"2024-08-31T07:10:46.021262+0000","last_became_active":"2024-08-31T07:09:45.636163+0000","last_became_peered":"2024-08-31T07:09:45.636163+0000","last_unstale":"2024-08-31T07:10:46.021262+0000","last_undegraded":"2024-08-31T07:10:46.021262+0000","last_fullsized":"2024-08-31T07:10:46.021262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:18:07.567858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257075+0000","last_change":"2024-08-31T07:09:47.671865+0000","last_active":"2024-08-31T07:10:46.257075+0000","last_peered":"2024-08-31T07:10:46.257075+0000","last_clean":"2024-08-31T07:10:46.257075+0000","last_became_active":"2024-08-31T07:09:47.671454+0000","last_became_peered":"2024-08-31T07:09:47.671454+0000","last_unstale":"2024-08-31T07:10:46.257075+0000","last_undegraded":"2024-08-31T07:10:46.257075+0000","last_fullsized":"2024-08-31T07:10:46.257075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:47.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'6","reported_seq":100,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:07.985597+0000","last_change":"2024-08-31T07:09:45.653217+0000","last_active":"2024-08-31T07:11:07.985597+0000","last_peered":"2024-08-31T07:11:07.985597+0000","last_clean":"2024-08-31T07:11:07.985597+0000","last_became_active":"2024-08-31T07:09:45.652819+0000","last_became_peered":"2024-08-31T07:09:45.652819+0000","last_unstale":"2024-08-31T07:11:07.985597+0000","last_undegraded":"2024-08-31T07:11:07.985597+0000","last_fullsized":"2024-08-31T07:11:07.985597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:25:11.454068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257248+0000","last_change":"2024-08-31T07:09:47.663652+0000","last_active":"2024-08-31T07:10:46.257248+0000","last_peered":"2024-08-31T07:10:46.257248+0000","last_clean":"2024-08-31T07:10:46.257248+0000","last_became_active":"2024-08-31T07:09:47.663274+0000","last_became_peered":"2024-08-31T07:09:47.663274+0000","last_unstale":"2024-08-31T07:10:46.257248+0000","last_undegraded":"2024-08-31T07:10:46.257248+0000","last_fullsized":"2024-08-31T07:10:46.257248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:31:59.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257370+0000","last_change":"2024-08-31T07:08:27.623298+0000","last_active":"2024-08-31T07:10:46.257370+0000","last_peered":"2024-08-31T07:10:46.257370+0000","last_clean":"2024-08-31T07:10:46.257370+0000","last_became_active":"2024-08-31T07:08:27.619072+0000","last_became_peered":"2024-08-31T07:08:27.619072+0000","last_unstale":"2024-08-31T07:10:46.257370+0000","last_undegraded":"2024-08-31T07:10:46.257370+0000","last_fullsized":"2024-08-31T07:10:46.257370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_clean_scrub_stamp":"2024-08-31T07:04:45.509398+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:07.967421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154268+0000","last_change":"2024-08-31T07:09:45.638104+0000","last_active":"2024-08-31T07:10:46.154268+0000","last_peered":"2024-08-31T07:10:46.154268+0000","last_clean":"2024-08-31T07:10:46.154268+0000","last_became_active":"2024-08-31T07:09:45.637693+0000","last_became_peered":"2024-08-31T07:09:45.637693+0000","last_unstale":"2024-08-31T07:10:46.154268+0000","last_undegraded":"2024-08-31T07:10:46.154268+0000","last_fullsized":"2024-08-31T07:10:46.154268+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:50:15.135061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015717+0000","last_change":"2024-08-31T07:09:47.664958+0000","last_active":"2024-08-31T07:10:46.015717+0000","last_peered":"2024-08-31T07:10:46.015717+0000","last_clean":"2024-08-31T07:10:46.015717+0000","last_became_active":"2024-08-31T07:09:47.661166+0000","last_became_peered":"2024-08-31T07:09:47.661166+0000","last_unstale":"2024-08-31T07:10:46.015717+0000","last_undegraded":"2024-08-31T07:10:46.015717+0000","last_fullsized":"2024-08-31T07:10:46.015717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:45:58.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257126+0000","last_change":"2024-08-31T07:09:45.640881+0000","last_active":"2024-08-31T07:10:46.257126+0000","last_peered":"2024-08-31T07:10:46.257126+0000","last_clean":"2024-08-31T07:10:46.257126+0000","last_became_active":"2024-08-31T07:09:45.640487+0000","last_became_peered":"2024-08-31T07:09:45.640487+0000","last_unstale":"2024-08-31T07:10:46.257126+0000","last_undegraded":"2024-08-31T07:10:46.257126+0000","last_fullsized":"2024-08-31T07:10:46.257126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:06.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017222+0000","last_change":"2024-08-31T07:09:47.655008+0000","last_active":"2024-08-31T07:10:46.017222+0000","last_peered":"2024-08-31T07:10:46.017222+0000","last_clean":"2024-08-31T07:10:46.017222+0000","last_became_active":"2024-08-31T07:09:47.654645+0000","last_became_peered":"2024-08-31T07:09:47.654645+0000","last_unstale":"2024-08-31T07:10:46.017222+0000","last_undegraded":"2024-08-31T07:10:46.017222+0000","last_fullsized":"2024-08-31T07:10:46.017222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:23:55.916053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":96,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:06.985608+0000","last_change":"2024-08-31T07:09:45.649324+0000","last_active":"2024-08-31T07:11:06.985608+0000","last_peered":"2024-08-31T07:11:06.985608+0000","last_clean":"2024-08-31T07:11:06.985608+0000","last_became_active":"2024-08-31T07:09:45.648915+0000","last_became_peered":"2024-08-31T07:09:45.648915+0000","last_unstale":"2024-08-31T07:11:06.985608+0000","last_undegraded":"2024-08-31T07:11:06.985608+0000","last_fullsized":"2024-08-31T07:11:06.985608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:31:57.853654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022278+0000","last_change":"2024-08-31T07:09:47.661328+0000","last_active":"2024-08-31T07:10:46.022278+0000","last_peered":"2024-08-31T07:10:46.022278+0000","last_clean":"2024-08-31T07:10:46.022278+0000","last_became_active":"2024-08-31T07:09:47.660927+0000","last_became_peered":"2024-08-31T07:09:47.660927+0000","last_unstale":"2024-08-31T07:10:46.022278+0000","last_undegraded":"2024-08-31T07:10:46.022278+0000","last_fullsized":"2024-08-31T07:10:46.022278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:40:46.876894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'6","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:06.986330+0000","last_change":"2024-08-31T07:09:45.637747+0000","last_active":"2024-08-31T07:11:06.986330+0000","last_peered":"2024-08-31T07:11:06.986330+0000","last_clean":"2024-08-31T07:11:06.986330+0000","last_became_active":"2024-08-31T07:09:45.637368+0000","last_became_peered":"2024-08-31T07:09:45.637368+0000","last_unstale":"2024-08-31T07:11:06.986330+0000","last_undegraded":"2024-08-31T07:11:06.986330+0000","last_fullsized":"2024-08-31T07:11:06.986330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:01.174109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019918+0000","last_change":"2024-08-31T07:09:47.680802+0000","last_active":"2024-08-31T07:10:46.019918+0000","last_peered":"2024-08-31T07:10:46.019918+0000","last_clean":"2024-08-31T07:10:46.019918+0000","last_became_active":"2024-08-31T07:09:47.680363+0000","last_became_peered":"2024-08-31T07:09:47.680363+0000","last_unstale":"2024-08-31T07:10:46.019918+0000","last_undegraded":"2024-08-31T07:10:46.019918+0000","last_fullsized":"2024-08-31T07:10:46.019918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:46:48.744071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324292+0000","last_change":"2024-08-31T07:09:45.651222+0000","last_active":"2024-08-31T07:10:46.324292+0000","last_peered":"2024-08-31T07:10:46.324292+0000","last_clean":"2024-08-31T07:10:46.324292+0000","last_became_active":"2024-08-31T07:09:45.650760+0000","last_became_peered":"2024-08-31T07:09:45.650760+0000","last_unstale":"2024-08-31T07:10:46.324292+0000","last_undegraded":"2024-08-31T07:10:46.324292+0000","last_fullsized":"2024-08-31T07:10:46.324292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T19:03:33.942165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020011+0000","last_change":"2024-08-31T07:09:47.665881+0000","last_active":"2024-08-31T07:10:46.020011+0000","last_peered":"2024-08-31T07:10:46.020011+0000","last_clean":"2024-08-31T07:10:46.020011+0000","last_became_active":"2024-08-31T07:09:47.665485+0000","last_became_peered":"2024-08-31T07:09:47.665485+0000","last_unstale":"2024-08-31T07:10:46.020011+0000","last_undegraded":"2024-08-31T07:10:46.020011+0000","last_fullsized":"2024-08-31T07:10:46.020011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:30.820714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020786+0000","last_change":"2024-08-31T07:09:45.651893+0000","last_active":"2024-08-31T07:10:46.020786+0000","last_peered":"2024-08-31T07:10:46.020786+0000","last_clean":"2024-08-31T07:10:46.020786+0000","last_became_active":"2024-08-31T07:09:45.651547+0000","last_became_peered":"2024-08-31T07:09:45.651547+0000","last_unstale":"2024-08-31T07:10:46.020786+0000","last_undegraded":"2024-08-31T07:10:46.020786+0000","last_fullsized":"2024-08-31T07:10:46.020786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:11:30.708018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017674+0000","last_change":"2024-08-31T07:09:47.655551+0000","last_active":"2024-08-31T07:10:46.017674+0000","last_peered":"2024-08-31T07:10:46.017674+0000","last_clean":"2024-08-31T07:10:46.017674+0000","last_became_active":"2024-08-31T07:09:47.655188+0000","last_became_peered":"2024-08-31T07:09:47.655188+0000","last_unstale":"2024-08-31T07:10:46.017674+0000","last_undegraded":"2024-08-31T07:10:46.017674+0000","last_fullsized":"2024-08-31T07:10:46.017674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:43:51.419599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'15","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324668+0000","last_change":"2024-08-31T07:09:45.636212+0000","last_active":"2024-08-31T07:10:46.324668+0000","last_peered":"2024-08-31T07:10:46.324668+0000","last_clean":"2024-08-31T07:10:46.324668+0000","last_became_active":"2024-08-31T07:09:45.635678+0000","last_became_peered":"2024-08-31T07:09:45.635678+0000","last_unstale":"2024-08-31T07:10:46.324668+0000","last_undegraded":"2024-08-31T07:10:46.324668+0000","last_fullsized":"2024-08-31T07:10:46.324668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:13:17.803459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016490+0000","last_change":"2024-08-31T07:09:47.666476+0000","last_active":"2024-08-31T07:10:46.016490+0000","last_peered":"2024-08-31T07:10:46.016490+0000","last_clean":"2024-08-31T07:10:46.016490+0000","last_became_active":"2024-08-31T07:09:47.666111+0000","last_became_peered":"2024-08-31T07:09:47.666111+0000","last_unstale":"2024-08-31T07:10:46.016490+0000","last_undegraded":"2024-08-31T07:10:46.016490+0000","last_fullsized":"2024-08-31T07:10:46.016490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:08:53.459549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154374+0000","last_change":"2024-08-31T07:09:45.631571+0000","last_active":"2024-08-31T07:10:46.154374+0000","last_peered":"2024-08-31T07:10:46.154374+0000","last_clean":"2024-08-31T07:10:46.154374+0000","last_became_active":"2024-08-31T07:09:45.630981+0000","last_became_peered":"2024-08-31T07:09:45.630981+0000","last_unstale":"2024-08-31T07:10:46.154374+0000","last_undegraded":"2024-08-31T07:10:46.154374+0000","last_fullsized":"2024-08-31T07:10:46.154374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:36:53.482955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324734+0000","last_change":"2024-08-31T07:09:47.675054+0000","last_active":"2024-08-31T07:10:46.324734+0000","last_peered":"2024-08-31T07:10:46.324734+0000","last_clean":"2024-08-31T07:10:46.324734+0000","last_became_active":"2024-08-31T07:09:47.672652+0000","last_became_peered":"2024-08-31T07:09:47.672652+0000","last_unstale":"2024-08-31T07:10:46.324734+0000","last_undegraded":"2024-08-31T07:10:46.324734+0000","last_fullsized":"2024-08-31T07:10:46.324734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:32:05.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016218+0000","last_change":"2024-08-31T07:09:45.648746+0000","last_active":"2024-08-31T07:10:46.016218+0000","last_peered":"2024-08-31T07:10:46.016218+0000","last_clean":"2024-08-31T07:10:46.016218+0000","last_became_active":"2024-08-31T07:09:45.648424+0000","last_became_peered":"2024-08-31T07:09:45.648424+0000","last_unstale":"2024-08-31T07:10:46.016218+0000","last_undegraded":"2024-08-31T07:10:46.016218+0000","last_fullsized":"2024-08-31T07:10:46.016218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:40:32.132963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020225+0000","last_change":"2024-08-31T07:09:47.663213+0000","last_active":"2024-08-31T07:10:46.020225+0000","last_peered":"2024-08-31T07:10:46.020225+0000","last_clean":"2024-08-31T07:10:46.020225+0000","last_became_active":"2024-08-31T07:09:47.662457+0000","last_became_peered":"2024-08-31T07:09:47.662457+0000","last_unstale":"2024-08-31T07:10:46.020225+0000","last_undegraded":"2024-08-31T07:10:46.020225+0000","last_fullsized":"2024-08-31T07:10:46.020225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:54:27.069200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015595+0000","last_change":"2024-08-31T07:09:45.640149+0000","last_active":"2024-08-31T07:10:46.015595+0000","last_peered":"2024-08-31T07:10:46.015595+0000","last_clean":"2024-08-31T07:10:46.015595+0000","last_became_active":"2024-08-31T07:09:45.639581+0000","last_became_peered":"2024-08-31T07:09:45.639581+0000","last_unstale":"2024-08-31T07:10:46.015595+0000","last_undegraded":"2024-08-31T07:10:46.015595+0000","last_fullsized":"2024-08-31T07:10:46.015595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:27:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018016+0000","last_change":"2024-08-31T07:09:47.672613+0000","last_active":"2024-08-31T07:10:46.018016+0000","last_peered":"2024-08-31T07:10:46.018016+0000","last_clean":"2024-08-31T07:10:46.018016+0000","last_became_active":"2024-08-31T07:09:47.672173+0000","last_became_peered":"2024-08-31T07:09:47.672173+0000","last_unstale":"2024-08-31T07:10:46.018016+0000","last_undegraded":"2024-08-31T07:10:46.018016+0000","last_fullsized":"2024-08-31T07:10:46.018016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:49.706113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022533+0000","last_change":"2024-08-31T07:09:45.637567+0000","last_active":"2024-08-31T07:10:46.022533+0000","last_peered":"2024-08-31T07:10:46.022533+0000","last_clean":"2024-08-31T07:10:46.022533+0000","last_became_active":"2024-08-31T07:09:45.637180+0000","last_became_peered":"2024-08-31T07:09:45.637180+0000","last_unstale":"2024-08-31T07:10:46.022533+0000","last_undegraded":"2024-08-31T07:10:46.022533+0000","last_fullsized":"2024-08-31T07:10:46.022533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:48:07.190829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016578+0000","last_change":"2024-08-31T07:09:47.674980+0000","last_active":"2024-08-31T07:10:46.016578+0000","last_peered":"2024-08-31T07:10:46.016578+0000","last_clean":"2024-08-31T07:10:46.016578+0000","last_became_active":"2024-08-31T07:09:47.674528+0000","last_became_peered":"2024-08-31T07:09:47.674528+0000","last_unstale":"2024-08-31T07:10:46.016578+0000","last_undegraded":"2024-08-31T07:10:46.016578+0000","last_fullsized":"2024-08-31T07:10:46.016578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:15:09.464221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021024+0000","last_change":"2024-08-31T07:09:45.641167+0000","last_active":"2024-08-31T07:10:46.021024+0000","last_peered":"2024-08-31T07:10:46.021024+0000","last_clean":"2024-08-31T07:10:46.021024+0000","last_became_active":"2024-08-31T07:09:45.640761+0000","last_became_peered":"2024-08-31T07:09:45.640761+0000","last_unstale":"2024-08-31T07:10:46.021024+0000","last_undegraded":"2024-08-31T07:10:46.021024+0000","last_fullsized":"2024-08-31T07:10:46.021024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:28:59.963145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257534+0000","last_change":"2024-08-31T07:09:47.668808+0000","last_active":"2024-08-31T07:10:46.257534+0000","last_peered":"2024-08-31T07:10:46.257534+0000","last_clean":"2024-08-31T07:10:46.257534+0000","last_became_active":"2024-08-31T07:09:47.668291+0000","last_became_peered":"2024-08-31T07:09:47.668291+0000","last_unstale":"2024-08-31T07:10:46.257534+0000","last_undegraded":"2024-08-31T07:10:46.257534+0000","last_fullsized":"2024-08-31T07:10:46.257534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:23.433511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020994+0000","last_change":"2024-08-31T07:09:47.668039+0000","last_active":"2024-08-31T07:10:46.020994+0000","last_peered":"2024-08-31T07:10:46.020994+0000","last_clean":"2024-08-31T07:10:46.020994+0000","last_became_active":"2024-08-31T07:09:47.667567+0000","last_became_peered":"2024-08-31T07:09:47.667567+0000","last_unstale":"2024-08-31T07:10:46.020994+0000","last_undegraded":"2024-08-31T07:10:46.020994+0000","last_fullsized":"2024-08-31T07:10:46.020994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:33:50.192864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.041708+0000","last_change":"2024-08-31T07:10:46.041197+0000","last_active":"2024-08-31T07:10:46.041708+0000","last_peered":"2024-08-31T07:10:46.041708+0000","last_clean":"2024-08-31T07:10:46.041708+0000","last_became_active":"2024-08-31T07:10:46.023966+0000","last_became_peered":"2024-08-31T07:10:46.023966+0000","last_unstale":"2024-08-31T07:10:46.041708+0000","last_undegraded":"2024-08-31T07:10:46.041708+0000","last_fullsized":"2024-08-31T07:10:46.041708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:34.302665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016265+0000","last_change":"2024-08-31T07:09:47.660615+0000","last_active":"2024-08-31T07:10:46.016265+0000","last_peered":"2024-08-31T07:10:46.016265+0000","last_clean":"2024-08-31T07:10:46.016265+0000","last_became_active":"2024-08-31T07:09:47.660269+0000","last_became_peered":"2024-08-31T07:09:47.660269+0000","last_unstale":"2024-08-31T07:10:46.016265+0000","last_undegraded":"2024-08-31T07:10:46.016265+0000","last_fullsized":"2024-08-31T07:10:46.016265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:42:12.618039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020632+0000","last_change":"2024-08-31T07:09:45.641699+0000","last_active":"2024-08-31T07:10:46.020632+0000","last_peered":"2024-08-31T07:10:46.020632+0000","last_clean":"2024-08-31T07:10:46.020632+0000","last_became_active":"2024-08-31T07:09:45.641122+0000","last_became_peered":"2024-08-31T07:09:45.641122+0000","last_unstale":"2024-08-31T07:10:46.020632+0000","last_undegraded":"2024-08-31T07:10:46.020632+0000","last_fullsized":"2024-08-31T07:10:46.020632+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:26:07.130266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324235+0000","last_change":"2024-08-31T07:09:47.675459+0000","last_active":"2024-08-31T07:10:46.324235+0000","last_peered":"2024-08-31T07:10:46.324235+0000","last_clean":"2024-08-31T07:10:46.324235+0000","last_became_active":"2024-08-31T07:09:47.672246+0000","last_became_peered":"2024-08-31T07:09:47.672246+0000","last_unstale":"2024-08-31T07:10:46.324235+0000","last_undegraded":"2024-08-31T07:10:46.324235+0000","last_fullsized":"2024-08-31T07:10:46.324235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:49:23.547352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016380+0000","last_change":"2024-08-31T07:09:45.649522+0000","last_active":"2024-08-31T07:10:46.016380+0000","last_peered":"2024-08-31T07:10:46.016380+0000","last_clean":"2024-08-31T07:10:46.016380+0000","last_became_active":"2024-08-31T07:09:45.649109+0000","last_became_peered":"2024-08-31T07:09:45.649109+0000","last_unstale":"2024-08-31T07:10:46.016380+0000","last_undegraded":"2024-08-31T07:10:46.016380+0000","last_fullsized":"2024-08-31T07:10:46.016380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:52:52.432605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016283+0000","last_change":"2024-08-31T07:09:47.670583+0000","last_active":"2024-08-31T07:10:46.016283+0000","last_peered":"2024-08-31T07:10:46.016283+0000","last_clean":"2024-08-31T07:10:46.016283+0000","last_became_active":"2024-08-31T07:09:47.670162+0000","last_became_peered":"2024-08-31T07:09:47.670162+0000","last_unstale":"2024-08-31T07:10:46.016283+0000","last_undegraded":"2024-08-31T07:10:46.016283+0000","last_fullsized":"2024-08-31T07:10:46.016283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:35:35.135678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":106,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:08.988655+0000","last_change":"2024-08-31T07:09:45.639636+0000","last_active":"2024-08-31T07:11:08.988655+0000","last_peered":"2024-08-31T07:11:08.988655+0000","last_clean":"2024-08-31T07:11:08.988655+0000","last_became_active":"2024-08-31T07:09:45.639029+0000","last_became_peered":"2024-08-31T07:09:45.639029+0000","last_unstale":"2024-08-31T07:11:08.988655+0000","last_undegraded":"2024-08-31T07:11:08.988655+0000","last_fullsized":"2024-08-31T07:11:08.988655+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:54.110149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68603,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":683,"num_read_kb":417,"num_write":247,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":650,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285320,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678046,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":884736,"data_stored":434353,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874275,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1028096,"data_stored":1085349,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037801,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":433242,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234032,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":888832,"data_stored":433038,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397556,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":1026920,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561080,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":880640,"data_stored":433350,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":1025816,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":880640,"data_stored":433211,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117066,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":432784,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378704,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":495423,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":494880,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":901120,"data_stored":434689,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T07:11:12.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T07:11:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:13.454 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:13 smithi032 ceph-mon[17252]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:11:13.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:13 smithi084 ceph-mon[22387]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:11:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.632+0000 7f475be00640 1 -- 172.21.15.32:0/2784779866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475410b460 msgr2=0x7f475410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.632+0000 7f475be00640 1 --2- 172.21.15.32:0/2784779866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475410b460 0x7f475410b870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4740009900 tx=0x7f474002a410 comp rx=0 tx=0).stop 2024-08-31T07:11:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.632+0000 7f475be00640 1 -- 172.21.15.32:0/2784779866 shutdown_connections 2024-08-31T07:11:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.632+0000 7f475be00640 1 --2- 172.21.15.32:0/2784779866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f475410bdb0 0x7f47541125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.632+0000 7f475be00640 1 --2- 172.21.15.32:0/2784779866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475410b460 0x7f475410b870 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.632+0000 7f475be00640 1 --2- 172.21.15.32:0/2784779866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4754105b20 0x7f4754105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.632+0000 7f475be00640 1 -- 172.21.15.32:0/2784779866 >> 172.21.15.32:0/2784779866 conn(0x7f4754101570 msgr2=0x7f47541039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.632+0000 7f475be00640 1 -- 172.21.15.32:0/2784779866 shutdown_connections 2024-08-31T07:11:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.632+0000 7f475be00640 1 -- 172.21.15.32:0/2784779866 wait complete. 2024-08-31T07:11:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.634+0000 7f475be00640 1 Processor -- start 2024-08-31T07:11:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.634+0000 7f475be00640 1 -- start start 2024-08-31T07:11:13.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.634+0000 7f475be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754105b20 0x7f475419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.634+0000 7f475be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f475410b460 0x7f475419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.634+0000 7f475be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f475410bdb0 0x7f47541a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.634+0000 7f475be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4754115a70 con 0x7f4754105b20 2024-08-31T07:11:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.634+0000 7f475be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47541158f0 con 0x7f475410b460 2024-08-31T07:11:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.634+0000 7f475be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4754115bf0 con 0x7f475410bdb0 2024-08-31T07:11:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.635+0000 7f4759400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754105b20 0x7f475419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.635+0000 7f4759400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754105b20 0x7f475419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36350/0 (socket says 172.21.15.32:36350) 2024-08-31T07:11:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.635+0000 7f4759400640 1 -- 172.21.15.32:0/3440127508 learned_addr learned my addr 172.21.15.32:0/3440127508 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.635+0000 7f4758a00640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f475410b460 0x7f475419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.635+0000 7f4759e00640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f475410bdb0 0x7f47541a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:13.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.635+0000 7f4759400640 1 -- 172.21.15.32:0/3440127508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f475410bdb0 msgr2=0x7f47541a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:13.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.635+0000 7f4759400640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f475410bdb0 0x7f47541a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.635+0000 7f4759400640 1 -- 172.21.15.32:0/3440127508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f475410b460 msgr2=0x7f475419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.635+0000 7f4759400640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f475410b460 0x7f475419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.635+0000 7f4759400640 1 -- 172.21.15.32:0/3440127508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47541a23b0 con 0x7f4754105b20 2024-08-31T07:11:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.636+0000 7f4759400640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754105b20 0x7f475419c440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f474800e9f0 tx=0x7f474800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.636+0000 7f4752000640 1 -- 172.21.15.32:0/3440127508 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474800cd80 con 0x7f4754105b20 2024-08-31T07:11:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.636+0000 7f475be00640 1 -- 172.21.15.32:0/3440127508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47541a26a0 con 0x7f4754105b20 2024-08-31T07:11:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.636+0000 7f475be00640 1 -- 172.21.15.32:0/3440127508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47541a8e00 con 0x7f4754105b20 2024-08-31T07:11:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.636+0000 7f4752000640 1 -- 172.21.15.32:0/3440127508 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47480108e0 con 0x7f4754105b20 2024-08-31T07:11:13.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.636+0000 7f4752000640 1 -- 172.21.15.32:0/3440127508 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748018980 con 0x7f4754105b20 2024-08-31T07:11:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.637+0000 7f475be00640 1 -- 172.21.15.32:0/3440127508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47340052c0 con 0x7f4754105b20 2024-08-31T07:11:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.640+0000 7f4752000640 1 -- 172.21.15.32:0/3440127508 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f4748010430 con 0x7f4754105b20 2024-08-31T07:11:13.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.642+0000 7f4752000640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f473c0b4c90 0x7f473c0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:13.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.642+0000 7f4752000640 1 -- 172.21.15.32:0/3440127508 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f47480e5770 con 0x7f4754105b20 2024-08-31T07:11:13.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.642+0000 7f4758a00640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f473c0b4c90 0x7f473c0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:13.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.643+0000 7f4758a00640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f473c0b4c90 0x7f473c0b71a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4740009230 tx=0x7f4740031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:13.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.644+0000 7f4752000640 1 -- 172.21.15.32:0/3440127508 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f47480abff0 con 0x7f4754105b20 2024-08-31T07:11:13.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:13 smithi073 ceph-mon[21995]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:11:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.839+0000 7f475be00640 1 -- 172.21.15.32:0/3440127508 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4734000bf0 con 0x7f473c0b4c90 2024-08-31T07:11:13.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.844+0000 7f4752000640 1 -- 172.21.15.32:0/3440127508 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259013 (secure 0 0 0) 0x7f4734000bf0 con 0x7f473c0b4c90 2024-08-31T07:11:13.848 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-08-31T07:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.849+0000 7f475be00640 1 -- 172.21.15.32:0/3440127508 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f473c0b4c90 msgr2=0x7f473c0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.849+0000 7f475be00640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f473c0b4c90 0x7f473c0b71a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4740009230 tx=0x7f4740031400 comp rx=0 tx=0).stop 2024-08-31T07:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.849+0000 7f475be00640 1 -- 172.21.15.32:0/3440127508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754105b20 msgr2=0x7f475419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.849+0000 7f475be00640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754105b20 0x7f475419c440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f474800e9f0 tx=0x7f474800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.849+0000 7f475be00640 1 -- 172.21.15.32:0/3440127508 shutdown_connections 2024-08-31T07:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.849+0000 7f475be00640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f473c0b4c90 0x7f473c0b71a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.849+0000 7f475be00640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f475410bdb0 0x7f47541a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.849+0000 7f475be00640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f475410b460 0x7f475419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.849+0000 7f475be00640 1 --2- 172.21.15.32:0/3440127508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754105b20 0x7f475419c440 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:13.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.850+0000 7f475be00640 1 -- 172.21.15.32:0/3440127508 >> 172.21.15.32:0/3440127508 conn(0x7f4754101570 msgr2=0x7f4754103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:13.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.850+0000 7f475be00640 1 -- 172.21.15.32:0/3440127508 shutdown_connections 2024-08-31T07:11:13.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:13.850+0000 7f475be00640 1 -- 172.21.15.32:0/3440127508 wait complete. 2024-08-31T07:11:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:14 smithi084 ceph-mon[22387]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:14 smithi032 ceph-mon[17252]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:14.623 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-08-31T07:11:12.779895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659205,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":794,"num_read_kb":501,"num_write":372,"num_write_kb":2289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2952490,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364276,"kb_used_data":10800,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577548,"statfs":{"total":1151940427776,"available":1151567409152,"internally_reserved":0,"allocated":11059200,"data_stored":7163055,"data_compressed":177834,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003515"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018337+0000","last_change":"2024-08-31T07:09:47.665725+0000","last_active":"2024-08-31T07:10:46.018337+0000","last_peered":"2024-08-31T07:10:46.018337+0000","last_clean":"2024-08-31T07:10:46.018337+0000","last_became_active":"2024-08-31T07:09:47.665368+0000","last_became_peered":"2024-08-31T07:09:47.665368+0000","last_unstale":"2024-08-31T07:10:46.018337+0000","last_undegraded":"2024-08-31T07:10:46.018337+0000","last_fullsized":"2024-08-31T07:10:46.018337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:27:01.636099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324615+0000","last_change":"2024-08-31T07:09:47.676882+0000","last_active":"2024-08-31T07:10:46.324615+0000","last_peered":"2024-08-31T07:10:46.324615+0000","last_clean":"2024-08-31T07:10:46.324615+0000","last_became_active":"2024-08-31T07:09:47.674825+0000","last_became_peered":"2024-08-31T07:09:47.674825+0000","last_unstale":"2024-08-31T07:10:46.324615+0000","last_undegraded":"2024-08-31T07:10:46.324615+0000","last_fullsized":"2024-08-31T07:10:46.324615+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:09.148398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.323881+0000","last_change":"2024-08-31T07:09:47.674480+0000","last_active":"2024-08-31T07:10:46.323881+0000","last_peered":"2024-08-31T07:10:46.323881+0000","last_clean":"2024-08-31T07:10:46.323881+0000","last_became_active":"2024-08-31T07:09:47.672316+0000","last_became_peered":"2024-08-31T07:09:47.672316+0000","last_unstale":"2024-08-31T07:10:46.323881+0000","last_undegraded":"2024-08-31T07:10:46.323881+0000","last_fullsized":"2024-08-31T07:10:46.323881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:28:13.231039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.153980+0000","last_change":"2024-08-31T07:09:47.659276+0000","last_active":"2024-08-31T07:10:46.153980+0000","last_peered":"2024-08-31T07:10:46.153980+0000","last_clean":"2024-08-31T07:10:46.153980+0000","last_became_active":"2024-08-31T07:09:47.658924+0000","last_became_peered":"2024-08-31T07:09:47.658924+0000","last_unstale":"2024-08-31T07:10:46.153980+0000","last_undegraded":"2024-08-31T07:10:46.153980+0000","last_fullsized":"2024-08-31T07:10:46.153980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:01:08.134245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021006+0000","last_change":"2024-08-31T07:09:47.667260+0000","last_active":"2024-08-31T07:10:46.021006+0000","last_peered":"2024-08-31T07:10:46.021006+0000","last_clean":"2024-08-31T07:10:46.021006+0000","last_became_active":"2024-08-31T07:09:47.664572+0000","last_became_peered":"2024-08-31T07:09:47.664572+0000","last_unstale":"2024-08-31T07:10:46.021006+0000","last_undegraded":"2024-08-31T07:10:46.021006+0000","last_fullsized":"2024-08-31T07:10:46.021006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:11.336985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019758+0000","last_change":"2024-08-31T07:09:47.661181+0000","last_active":"2024-08-31T07:10:46.019758+0000","last_peered":"2024-08-31T07:10:46.019758+0000","last_clean":"2024-08-31T07:10:46.019758+0000","last_became_active":"2024-08-31T07:09:47.660761+0000","last_became_peered":"2024-08-31T07:09:47.660761+0000","last_unstale":"2024-08-31T07:10:46.019758+0000","last_undegraded":"2024-08-31T07:10:46.019758+0000","last_fullsized":"2024-08-31T07:10:46.019758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:50:54.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257002+0000","last_change":"2024-08-31T07:09:47.668762+0000","last_active":"2024-08-31T07:10:46.257002+0000","last_peered":"2024-08-31T07:10:46.257002+0000","last_clean":"2024-08-31T07:10:46.257002+0000","last_became_active":"2024-08-31T07:09:47.668406+0000","last_became_peered":"2024-08-31T07:09:47.668406+0000","last_unstale":"2024-08-31T07:10:46.257002+0000","last_undegraded":"2024-08-31T07:10:46.257002+0000","last_fullsized":"2024-08-31T07:10:46.257002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:43:39.868144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015452+0000","last_change":"2024-08-31T07:09:47.669455+0000","last_active":"2024-08-31T07:10:46.015452+0000","last_peered":"2024-08-31T07:10:46.015452+0000","last_clean":"2024-08-31T07:10:46.015452+0000","last_became_active":"2024-08-31T07:09:47.663669+0000","last_became_peered":"2024-08-31T07:09:47.663669+0000","last_unstale":"2024-08-31T07:10:46.015452+0000","last_undegraded":"2024-08-31T07:10:46.015452+0000","last_fullsized":"2024-08-31T07:10:46.015452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:22:59.685012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016753+0000","last_change":"2024-08-31T07:09:47.664214+0000","last_active":"2024-08-31T07:10:46.016753+0000","last_peered":"2024-08-31T07:10:46.016753+0000","last_clean":"2024-08-31T07:10:46.016753+0000","last_became_active":"2024-08-31T07:09:47.663832+0000","last_became_peered":"2024-08-31T07:09:47.663832+0000","last_unstale":"2024-08-31T07:10:46.016753+0000","last_undegraded":"2024-08-31T07:10:46.016753+0000","last_fullsized":"2024-08-31T07:10:46.016753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:29:14.107212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.326515+0000","last_change":"2024-08-31T07:10:46.326410+0000","last_active":"2024-08-31T07:10:46.326515+0000","last_peered":"2024-08-31T07:10:46.326515+0000","last_clean":"2024-08-31T07:10:46.326515+0000","last_became_active":"2024-08-31T07:10:46.325958+0000","last_became_peered":"2024-08-31T07:10:46.325958+0000","last_unstale":"2024-08-31T07:10:46.326515+0000","last_undegraded":"2024-08-31T07:10:46.326515+0000","last_fullsized":"2024-08-31T07:10:46.326515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:19.230393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257100+0000","last_change":"2024-08-31T07:09:47.666951+0000","last_active":"2024-08-31T07:10:46.257100+0000","last_peered":"2024-08-31T07:10:46.257100+0000","last_clean":"2024-08-31T07:10:46.257100+0000","last_became_active":"2024-08-31T07:09:47.666475+0000","last_became_peered":"2024-08-31T07:09:47.666475+0000","last_unstale":"2024-08-31T07:10:46.257100+0000","last_undegraded":"2024-08-31T07:10:46.257100+0000","last_fullsized":"2024-08-31T07:10:46.257100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:58:42.242086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154138+0000","last_change":"2024-08-31T07:09:47.658344+0000","last_active":"2024-08-31T07:10:46.154138+0000","last_peered":"2024-08-31T07:10:46.154138+0000","last_clean":"2024-08-31T07:10:46.154138+0000","last_became_active":"2024-08-31T07:09:47.657991+0000","last_became_peered":"2024-08-31T07:09:47.657991+0000","last_unstale":"2024-08-31T07:10:46.154138+0000","last_undegraded":"2024-08-31T07:10:46.154138+0000","last_fullsized":"2024-08-31T07:10:46.154138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:25:59.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018102+0000","last_change":"2024-08-31T07:09:47.661644+0000","last_active":"2024-08-31T07:10:46.018102+0000","last_peered":"2024-08-31T07:10:46.018102+0000","last_clean":"2024-08-31T07:10:46.018102+0000","last_became_active":"2024-08-31T07:09:47.661248+0000","last_became_peered":"2024-08-31T07:09:47.661248+0000","last_unstale":"2024-08-31T07:10:46.018102+0000","last_undegraded":"2024-08-31T07:10:46.018102+0000","last_fullsized":"2024-08-31T07:10:46.018102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:42.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.323555+0000","last_change":"2024-08-31T07:09:47.680052+0000","last_active":"2024-08-31T07:10:46.323555+0000","last_peered":"2024-08-31T07:10:46.323555+0000","last_clean":"2024-08-31T07:10:46.323555+0000","last_became_active":"2024-08-31T07:09:47.679604+0000","last_became_peered":"2024-08-31T07:09:47.679604+0000","last_unstale":"2024-08-31T07:10:46.323555+0000","last_undegraded":"2024-08-31T07:10:46.323555+0000","last_fullsized":"2024-08-31T07:10:46.323555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:23:18.937470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.323337+0000","last_change":"2024-08-31T07:09:47.677984+0000","last_active":"2024-08-31T07:10:46.323337+0000","last_peered":"2024-08-31T07:10:46.323337+0000","last_clean":"2024-08-31T07:10:46.323337+0000","last_became_active":"2024-08-31T07:09:47.674957+0000","last_became_peered":"2024-08-31T07:09:47.674957+0000","last_unstale":"2024-08-31T07:10:46.323337+0000","last_undegraded":"2024-08-31T07:10:46.323337+0000","last_fullsized":"2024-08-31T07:10:46.323337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:08:26.503746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016112+0000","last_change":"2024-08-31T07:09:47.672625+0000","last_active":"2024-08-31T07:10:46.016112+0000","last_peered":"2024-08-31T07:10:46.016112+0000","last_clean":"2024-08-31T07:10:46.016112+0000","last_became_active":"2024-08-31T07:09:47.672184+0000","last_became_peered":"2024-08-31T07:09:47.672184+0000","last_unstale":"2024-08-31T07:10:46.016112+0000","last_undegraded":"2024-08-31T07:10:46.016112+0000","last_fullsized":"2024-08-31T07:10:46.016112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:17.091659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018341+0000","last_change":"2024-08-31T07:09:47.672597+0000","last_active":"2024-08-31T07:10:46.018341+0000","last_peered":"2024-08-31T07:10:46.018341+0000","last_clean":"2024-08-31T07:10:46.018341+0000","last_became_active":"2024-08-31T07:09:47.672153+0000","last_became_peered":"2024-08-31T07:09:47.672153+0000","last_unstale":"2024-08-31T07:10:46.018341+0000","last_undegraded":"2024-08-31T07:10:46.018341+0000","last_fullsized":"2024-08-31T07:10:46.018341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:26:41.388526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324552+0000","last_change":"2024-08-31T07:09:47.666253+0000","last_active":"2024-08-31T07:10:46.324552+0000","last_peered":"2024-08-31T07:10:46.324552+0000","last_clean":"2024-08-31T07:10:46.324552+0000","last_became_active":"2024-08-31T07:09:47.663732+0000","last_became_peered":"2024-08-31T07:09:47.663732+0000","last_unstale":"2024-08-31T07:10:46.324552+0000","last_undegraded":"2024-08-31T07:10:46.324552+0000","last_fullsized":"2024-08-31T07:10:46.324552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:10:59.887765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023639+0000","last_change":"2024-08-31T07:09:47.668502+0000","last_active":"2024-08-31T07:10:46.023639+0000","last_peered":"2024-08-31T07:10:46.023639+0000","last_clean":"2024-08-31T07:10:46.023639+0000","last_became_active":"2024-08-31T07:09:47.666487+0000","last_became_peered":"2024-08-31T07:09:47.666487+0000","last_unstale":"2024-08-31T07:10:46.023639+0000","last_undegraded":"2024-08-31T07:10:46.023639+0000","last_fullsized":"2024-08-31T07:10:46.023639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:49:25.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018869+0000","last_change":"2024-08-31T07:09:47.659454+0000","last_active":"2024-08-31T07:10:46.018869+0000","last_peered":"2024-08-31T07:10:46.018869+0000","last_clean":"2024-08-31T07:10:46.018869+0000","last_became_active":"2024-08-31T07:09:47.658976+0000","last_became_peered":"2024-08-31T07:09:47.658976+0000","last_unstale":"2024-08-31T07:10:46.018869+0000","last_undegraded":"2024-08-31T07:10:46.018869+0000","last_fullsized":"2024-08-31T07:10:46.018869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:31.374905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018210+0000","last_change":"2024-08-31T07:09:47.673612+0000","last_active":"2024-08-31T07:10:46.018210+0000","last_peered":"2024-08-31T07:10:46.018210+0000","last_clean":"2024-08-31T07:10:46.018210+0000","last_became_active":"2024-08-31T07:09:47.673137+0000","last_became_peered":"2024-08-31T07:09:47.673137+0000","last_unstale":"2024-08-31T07:10:46.018210+0000","last_undegraded":"2024-08-31T07:10:46.018210+0000","last_fullsized":"2024-08-31T07:10:46.018210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:46:11.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016333+0000","last_change":"2024-08-31T07:09:47.669195+0000","last_active":"2024-08-31T07:10:46.016333+0000","last_peered":"2024-08-31T07:10:46.016333+0000","last_clean":"2024-08-31T07:10:46.016333+0000","last_became_active":"2024-08-31T07:09:47.668790+0000","last_became_peered":"2024-08-31T07:09:47.668790+0000","last_unstale":"2024-08-31T07:10:46.016333+0000","last_undegraded":"2024-08-31T07:10:46.016333+0000","last_fullsized":"2024-08-31T07:10:46.016333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:27:14.393931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015464+0000","last_change":"2024-08-31T07:09:47.667566+0000","last_active":"2024-08-31T07:10:46.015464+0000","last_peered":"2024-08-31T07:10:46.015464+0000","last_clean":"2024-08-31T07:10:46.015464+0000","last_became_active":"2024-08-31T07:09:47.666605+0000","last_became_peered":"2024-08-31T07:09:47.666605+0000","last_unstale":"2024-08-31T07:10:46.015464+0000","last_undegraded":"2024-08-31T07:10:46.015464+0000","last_fullsized":"2024-08-31T07:10:46.015464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:05:44.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022143+0000","last_change":"2024-08-31T07:09:47.660478+0000","last_active":"2024-08-31T07:10:46.022143+0000","last_peered":"2024-08-31T07:10:46.022143+0000","last_clean":"2024-08-31T07:10:46.022143+0000","last_became_active":"2024-08-31T07:09:47.660107+0000","last_became_peered":"2024-08-31T07:09:47.660107+0000","last_unstale":"2024-08-31T07:10:46.022143+0000","last_undegraded":"2024-08-31T07:10:46.022143+0000","last_fullsized":"2024-08-31T07:10:46.022143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:29.133370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016418+0000","last_change":"2024-08-31T07:09:47.671967+0000","last_active":"2024-08-31T07:10:46.016418+0000","last_peered":"2024-08-31T07:10:46.016418+0000","last_clean":"2024-08-31T07:10:46.016418+0000","last_became_active":"2024-08-31T07:09:47.671569+0000","last_became_peered":"2024-08-31T07:09:47.671569+0000","last_unstale":"2024-08-31T07:10:46.016418+0000","last_undegraded":"2024-08-31T07:10:46.016418+0000","last_fullsized":"2024-08-31T07:10:46.016418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:19:15.107958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023641+0000","last_change":"2024-08-31T07:09:47.651055+0000","last_active":"2024-08-31T07:10:46.023641+0000","last_peered":"2024-08-31T07:10:46.023641+0000","last_clean":"2024-08-31T07:10:46.023641+0000","last_became_active":"2024-08-31T07:09:47.650674+0000","last_became_peered":"2024-08-31T07:09:47.650674+0000","last_unstale":"2024-08-31T07:10:46.023641+0000","last_undegraded":"2024-08-31T07:10:46.023641+0000","last_fullsized":"2024-08-31T07:10:46.023641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:35:54.647379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018236+0000","last_change":"2024-08-31T07:09:47.663231+0000","last_active":"2024-08-31T07:10:46.018236+0000","last_peered":"2024-08-31T07:10:46.018236+0000","last_clean":"2024-08-31T07:10:46.018236+0000","last_became_active":"2024-08-31T07:09:47.662854+0000","last_became_peered":"2024-08-31T07:09:47.662854+0000","last_unstale":"2024-08-31T07:10:46.018236+0000","last_undegraded":"2024-08-31T07:10:46.018236+0000","last_fullsized":"2024-08-31T07:10:46.018236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:51.358557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.326495+0000","last_change":"2024-08-31T07:10:46.326380+0000","last_active":"2024-08-31T07:10:46.326495+0000","last_peered":"2024-08-31T07:10:46.326495+0000","last_clean":"2024-08-31T07:10:46.326495+0000","last_became_active":"2024-08-31T07:10:46.325784+0000","last_became_peered":"2024-08-31T07:10:46.325784+0000","last_unstale":"2024-08-31T07:10:46.326495+0000","last_undegraded":"2024-08-31T07:10:46.326495+0000","last_fullsized":"2024-08-31T07:10:46.326495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:16:56.396024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.323995+0000","last_change":"2024-08-31T07:09:47.672108+0000","last_active":"2024-08-31T07:10:46.323995+0000","last_peered":"2024-08-31T07:10:46.323995+0000","last_clean":"2024-08-31T07:10:46.323995+0000","last_became_active":"2024-08-31T07:09:47.670998+0000","last_became_peered":"2024-08-31T07:09:47.670998+0000","last_unstale":"2024-08-31T07:10:46.323995+0000","last_undegraded":"2024-08-31T07:10:46.323995+0000","last_fullsized":"2024-08-31T07:10:46.323995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:26:49.319861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022405+0000","last_change":"2024-08-31T07:09:47.665548+0000","last_active":"2024-08-31T07:10:46.022405+0000","last_peered":"2024-08-31T07:10:46.022405+0000","last_clean":"2024-08-31T07:10:46.022405+0000","last_became_active":"2024-08-31T07:09:47.665177+0000","last_became_peered":"2024-08-31T07:09:47.665177+0000","last_unstale":"2024-08-31T07:10:46.022405+0000","last_undegraded":"2024-08-31T07:10:46.022405+0000","last_fullsized":"2024-08-31T07:10:46.022405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:03:15.892224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.026863+0000","last_change":"2024-08-31T07:10:46.026746+0000","last_active":"2024-08-31T07:10:46.026863+0000","last_peered":"2024-08-31T07:10:46.026863+0000","last_clean":"2024-08-31T07:10:46.026863+0000","last_became_active":"2024-08-31T07:10:46.026286+0000","last_became_peered":"2024-08-31T07:10:46.026286+0000","last_unstale":"2024-08-31T07:10:46.026863+0000","last_undegraded":"2024-08-31T07:10:46.026863+0000","last_fullsized":"2024-08-31T07:10:46.026863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:34:17.120906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017936+0000","last_change":"2024-08-31T07:09:47.667456+0000","last_active":"2024-08-31T07:10:46.017936+0000","last_peered":"2024-08-31T07:10:46.017936+0000","last_clean":"2024-08-31T07:10:46.017936+0000","last_became_active":"2024-08-31T07:09:47.667102+0000","last_became_peered":"2024-08-31T07:09:47.667102+0000","last_unstale":"2024-08-31T07:10:46.017936+0000","last_undegraded":"2024-08-31T07:10:46.017936+0000","last_fullsized":"2024-08-31T07:10:46.017936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:44:39.747729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.326260+0000","last_change":"2024-08-31T07:10:46.326139+0000","last_active":"2024-08-31T07:10:46.326260+0000","last_peered":"2024-08-31T07:10:46.326260+0000","last_clean":"2024-08-31T07:10:46.326260+0000","last_became_active":"2024-08-31T07:10:46.325785+0000","last_became_peered":"2024-08-31T07:10:46.325785+0000","last_unstale":"2024-08-31T07:10:46.326260+0000","last_undegraded":"2024-08-31T07:10:46.326260+0000","last_fullsized":"2024-08-31T07:10:46.326260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:13.495835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":102,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:09.988720+0000","last_change":"2024-08-31T07:09:45.641088+0000","last_active":"2024-08-31T07:11:09.988720+0000","last_peered":"2024-08-31T07:11:09.988720+0000","last_clean":"2024-08-31T07:11:09.988720+0000","last_became_active":"2024-08-31T07:09:45.640673+0000","last_became_peered":"2024-08-31T07:09:45.640673+0000","last_unstale":"2024-08-31T07:11:09.988720+0000","last_undegraded":"2024-08-31T07:11:09.988720+0000","last_fullsized":"2024-08-31T07:11:09.988720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:30:17.593494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022664+0000","last_change":"2024-08-31T07:09:47.666703+0000","last_active":"2024-08-31T07:10:46.022664+0000","last_peered":"2024-08-31T07:10:46.022664+0000","last_clean":"2024-08-31T07:10:46.022664+0000","last_became_active":"2024-08-31T07:09:47.666348+0000","last_became_peered":"2024-08-31T07:09:47.666348+0000","last_unstale":"2024-08-31T07:10:46.022664+0000","last_undegraded":"2024-08-31T07:10:46.022664+0000","last_fullsized":"2024-08-31T07:10:46.022664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:39.688810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019815+0000","last_change":"2024-08-31T07:09:45.642431+0000","last_active":"2024-08-31T07:10:46.019815+0000","last_peered":"2024-08-31T07:10:46.019815+0000","last_clean":"2024-08-31T07:10:46.019815+0000","last_became_active":"2024-08-31T07:09:45.642048+0000","last_became_peered":"2024-08-31T07:09:45.642048+0000","last_unstale":"2024-08-31T07:10:46.019815+0000","last_undegraded":"2024-08-31T07:10:46.019815+0000","last_fullsized":"2024-08-31T07:10:46.019815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:47:42.574442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017223+0000","last_change":"2024-08-31T07:09:45.632162+0000","last_active":"2024-08-31T07:10:46.017223+0000","last_peered":"2024-08-31T07:10:46.017223+0000","last_clean":"2024-08-31T07:10:46.017223+0000","last_became_active":"2024-08-31T07:09:45.631749+0000","last_became_peered":"2024-08-31T07:09:45.631749+0000","last_unstale":"2024-08-31T07:10:46.017223+0000","last_undegraded":"2024-08-31T07:10:46.017223+0000","last_fullsized":"2024-08-31T07:10:46.017223+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:16:29.967823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015529+0000","last_change":"2024-08-31T07:09:47.666288+0000","last_active":"2024-08-31T07:10:46.015529+0000","last_peered":"2024-08-31T07:10:46.015529+0000","last_clean":"2024-08-31T07:10:46.015529+0000","last_became_active":"2024-08-31T07:09:47.665723+0000","last_became_peered":"2024-08-31T07:09:47.665723+0000","last_unstale":"2024-08-31T07:10:46.015529+0000","last_undegraded":"2024-08-31T07:10:46.015529+0000","last_fullsized":"2024-08-31T07:10:46.015529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:38:28.122720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023048+0000","last_change":"2024-08-31T07:09:45.636508+0000","last_active":"2024-08-31T07:10:46.023048+0000","last_peered":"2024-08-31T07:10:46.023048+0000","last_clean":"2024-08-31T07:10:46.023048+0000","last_became_active":"2024-08-31T07:09:45.636073+0000","last_became_peered":"2024-08-31T07:09:45.636073+0000","last_unstale":"2024-08-31T07:10:46.023048+0000","last_undegraded":"2024-08-31T07:10:46.023048+0000","last_fullsized":"2024-08-31T07:10:46.023048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:55:45.063517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.323509+0000","last_change":"2024-08-31T07:09:47.661306+0000","last_active":"2024-08-31T07:10:46.323509+0000","last_peered":"2024-08-31T07:10:46.323509+0000","last_clean":"2024-08-31T07:10:46.323509+0000","last_became_active":"2024-08-31T07:09:47.660957+0000","last_became_peered":"2024-08-31T07:09:47.660957+0000","last_unstale":"2024-08-31T07:10:46.323509+0000","last_undegraded":"2024-08-31T07:10:46.323509+0000","last_fullsized":"2024-08-31T07:10:46.323509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:18:23.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":165,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:10.989341+0000","last_change":"2024-08-31T07:09:45.640947+0000","last_active":"2024-08-31T07:11:10.989341+0000","last_peered":"2024-08-31T07:11:10.989341+0000","last_clean":"2024-08-31T07:11:10.989341+0000","last_became_active":"2024-08-31T07:09:45.640532+0000","last_became_peered":"2024-08-31T07:09:45.640532+0000","last_unstale":"2024-08-31T07:11:10.989341+0000","last_undegraded":"2024-08-31T07:11:10.989341+0000","last_fullsized":"2024-08-31T07:11:10.989341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:28.177795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016603+0000","last_change":"2024-08-31T07:09:47.661060+0000","last_active":"2024-08-31T07:10:46.016603+0000","last_peered":"2024-08-31T07:10:46.016603+0000","last_clean":"2024-08-31T07:10:46.016603+0000","last_became_active":"2024-08-31T07:09:47.660692+0000","last_became_peered":"2024-08-31T07:09:47.660692+0000","last_unstale":"2024-08-31T07:10:46.016603+0000","last_undegraded":"2024-08-31T07:10:46.016603+0000","last_fullsized":"2024-08-31T07:10:46.016603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:48:38.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257131+0000","last_change":"2024-08-31T07:09:45.641155+0000","last_active":"2024-08-31T07:10:46.257131+0000","last_peered":"2024-08-31T07:10:46.257131+0000","last_clean":"2024-08-31T07:10:46.257131+0000","last_became_active":"2024-08-31T07:09:45.640680+0000","last_became_peered":"2024-08-31T07:09:45.640680+0000","last_unstale":"2024-08-31T07:10:46.257131+0000","last_undegraded":"2024-08-31T07:10:46.257131+0000","last_fullsized":"2024-08-31T07:10:46.257131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:29:03.287570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324342+0000","last_change":"2024-08-31T07:09:47.675582+0000","last_active":"2024-08-31T07:10:46.324342+0000","last_peered":"2024-08-31T07:10:46.324342+0000","last_clean":"2024-08-31T07:10:46.324342+0000","last_became_active":"2024-08-31T07:09:47.675266+0000","last_became_peered":"2024-08-31T07:09:47.675266+0000","last_unstale":"2024-08-31T07:10:46.324342+0000","last_undegraded":"2024-08-31T07:10:46.324342+0000","last_fullsized":"2024-08-31T07:10:46.324342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:36:33.122531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021129+0000","last_change":"2024-08-31T07:09:45.651006+0000","last_active":"2024-08-31T07:10:46.021129+0000","last_peered":"2024-08-31T07:10:46.021129+0000","last_clean":"2024-08-31T07:10:46.021129+0000","last_became_active":"2024-08-31T07:09:45.650588+0000","last_became_peered":"2024-08-31T07:09:45.650588+0000","last_unstale":"2024-08-31T07:10:46.021129+0000","last_undegraded":"2024-08-31T07:10:46.021129+0000","last_fullsized":"2024-08-31T07:10:46.021129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:34:16.015892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016246+0000","last_change":"2024-08-31T07:09:47.662289+0000","last_active":"2024-08-31T07:10:46.016246+0000","last_peered":"2024-08-31T07:10:46.016246+0000","last_clean":"2024-08-31T07:10:46.016246+0000","last_became_active":"2024-08-31T07:09:47.661932+0000","last_became_peered":"2024-08-31T07:09:47.661932+0000","last_unstale":"2024-08-31T07:10:46.016246+0000","last_undegraded":"2024-08-31T07:10:46.016246+0000","last_fullsized":"2024-08-31T07:10:46.016246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:50.440504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'31","reported_seq":201,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:10.989429+0000","last_change":"2024-08-31T07:09:45.642196+0000","last_active":"2024-08-31T07:11:10.989429+0000","last_peered":"2024-08-31T07:11:10.989429+0000","last_clean":"2024-08-31T07:11:10.989429+0000","last_became_active":"2024-08-31T07:09:45.641797+0000","last_became_peered":"2024-08-31T07:09:45.641797+0000","last_unstale":"2024-08-31T07:11:10.989429+0000","last_undegraded":"2024-08-31T07:11:10.989429+0000","last_fullsized":"2024-08-31T07:11:10.989429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:51:05.953335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60617,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":145,"num_read_kb":131,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020491+0000","last_change":"2024-08-31T07:09:47.660184+0000","last_active":"2024-08-31T07:10:46.020491+0000","last_peered":"2024-08-31T07:10:46.020491+0000","last_clean":"2024-08-31T07:10:46.020491+0000","last_became_active":"2024-08-31T07:09:47.659571+0000","last_became_peered":"2024-08-31T07:09:47.659571+0000","last_unstale":"2024-08-31T07:10:46.020491+0000","last_undegraded":"2024-08-31T07:10:46.020491+0000","last_fullsized":"2024-08-31T07:10:46.020491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:41:13.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.035782+0000","last_change":"2024-08-31T07:10:46.035336+0000","last_active":"2024-08-31T07:10:46.035782+0000","last_peered":"2024-08-31T07:10:46.035782+0000","last_clean":"2024-08-31T07:10:46.035782+0000","last_became_active":"2024-08-31T07:10:46.021299+0000","last_became_peered":"2024-08-31T07:10:46.021299+0000","last_unstale":"2024-08-31T07:10:46.035782+0000","last_undegraded":"2024-08-31T07:10:46.035782+0000","last_fullsized":"2024-08-31T07:10:46.035782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:06:15.931855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.326482+0000","last_change":"2024-08-31T07:10:46.326354+0000","last_active":"2024-08-31T07:10:46.326482+0000","last_peered":"2024-08-31T07:10:46.326482+0000","last_clean":"2024-08-31T07:10:46.326482+0000","last_became_active":"2024-08-31T07:10:46.325869+0000","last_became_peered":"2024-08-31T07:10:46.325869+0000","last_unstale":"2024-08-31T07:10:46.326482+0000","last_undegraded":"2024-08-31T07:10:46.326482+0000","last_fullsized":"2024-08-31T07:10:46.326482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:21.832088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022856+0000","last_change":"2024-08-31T07:09:45.642187+0000","last_active":"2024-08-31T07:10:46.022856+0000","last_peered":"2024-08-31T07:10:46.022856+0000","last_clean":"2024-08-31T07:10:46.022856+0000","last_became_active":"2024-08-31T07:09:45.641721+0000","last_became_peered":"2024-08-31T07:09:45.641721+0000","last_unstale":"2024-08-31T07:10:46.022856+0000","last_undegraded":"2024-08-31T07:10:46.022856+0000","last_fullsized":"2024-08-31T07:10:46.022856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:00:45.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257256+0000","last_change":"2024-08-31T07:09:47.665956+0000","last_active":"2024-08-31T07:10:46.257256+0000","last_peered":"2024-08-31T07:10:46.257256+0000","last_clean":"2024-08-31T07:10:46.257256+0000","last_became_active":"2024-08-31T07:09:47.664367+0000","last_became_peered":"2024-08-31T07:09:47.664367+0000","last_unstale":"2024-08-31T07:10:46.257256+0000","last_undegraded":"2024-08-31T07:10:46.257256+0000","last_fullsized":"2024-08-31T07:10:46.257256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:34.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":102,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:09.991027+0000","last_change":"2024-08-31T07:09:45.636440+0000","last_active":"2024-08-31T07:11:09.991027+0000","last_peered":"2024-08-31T07:11:09.991027+0000","last_clean":"2024-08-31T07:11:09.991027+0000","last_became_active":"2024-08-31T07:09:45.636058+0000","last_became_peered":"2024-08-31T07:09:45.636058+0000","last_unstale":"2024-08-31T07:11:09.991027+0000","last_undegraded":"2024-08-31T07:11:09.991027+0000","last_fullsized":"2024-08-31T07:11:09.991027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:38.929227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.256870+0000","last_change":"2024-08-31T07:09:47.660564+0000","last_active":"2024-08-31T07:10:46.256870+0000","last_peered":"2024-08-31T07:10:46.256870+0000","last_clean":"2024-08-31T07:10:46.256870+0000","last_became_active":"2024-08-31T07:09:47.660157+0000","last_became_peered":"2024-08-31T07:09:47.660157+0000","last_unstale":"2024-08-31T07:10:46.256870+0000","last_undegraded":"2024-08-31T07:10:46.256870+0000","last_fullsized":"2024-08-31T07:10:46.256870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:49:01.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020455+0000","last_change":"2024-08-31T07:09:45.649140+0000","last_active":"2024-08-31T07:10:46.020455+0000","last_peered":"2024-08-31T07:10:46.020455+0000","last_clean":"2024-08-31T07:10:46.020455+0000","last_became_active":"2024-08-31T07:09:45.648796+0000","last_became_peered":"2024-08-31T07:09:45.648796+0000","last_unstale":"2024-08-31T07:10:46.020455+0000","last_undegraded":"2024-08-31T07:10:46.020455+0000","last_fullsized":"2024-08-31T07:10:46.020455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:33:37.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023205+0000","last_change":"2024-08-31T07:09:47.663833+0000","last_active":"2024-08-31T07:10:46.023205+0000","last_peered":"2024-08-31T07:10:46.023205+0000","last_clean":"2024-08-31T07:10:46.023205+0000","last_became_active":"2024-08-31T07:09:47.663451+0000","last_became_peered":"2024-08-31T07:09:47.663451+0000","last_unstale":"2024-08-31T07:10:46.023205+0000","last_undegraded":"2024-08-31T07:10:46.023205+0000","last_fullsized":"2024-08-31T07:10:46.023205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:27:22.245254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021332+0000","last_change":"2024-08-31T07:09:45.636593+0000","last_active":"2024-08-31T07:10:46.021332+0000","last_peered":"2024-08-31T07:10:46.021332+0000","last_clean":"2024-08-31T07:10:46.021332+0000","last_became_active":"2024-08-31T07:09:45.636166+0000","last_became_peered":"2024-08-31T07:09:45.636166+0000","last_unstale":"2024-08-31T07:10:46.021332+0000","last_undegraded":"2024-08-31T07:10:46.021332+0000","last_fullsized":"2024-08-31T07:10:46.021332+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:28.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018126+0000","last_change":"2024-08-31T07:09:47.660825+0000","last_active":"2024-08-31T07:10:46.018126+0000","last_peered":"2024-08-31T07:10:46.018126+0000","last_clean":"2024-08-31T07:10:46.018126+0000","last_became_active":"2024-08-31T07:09:47.660474+0000","last_became_peered":"2024-08-31T07:09:47.660474+0000","last_unstale":"2024-08-31T07:10:46.018126+0000","last_undegraded":"2024-08-31T07:10:46.018126+0000","last_fullsized":"2024-08-31T07:10:46.018126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:42:20.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017406+0000","last_change":"2024-08-31T07:09:45.632727+0000","last_active":"2024-08-31T07:10:46.017406+0000","last_peered":"2024-08-31T07:10:46.017406+0000","last_clean":"2024-08-31T07:10:46.017406+0000","last_became_active":"2024-08-31T07:09:45.632251+0000","last_became_peered":"2024-08-31T07:09:45.632251+0000","last_unstale":"2024-08-31T07:10:46.017406+0000","last_undegraded":"2024-08-31T07:10:46.017406+0000","last_fullsized":"2024-08-31T07:10:46.017406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:09:30.197409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257404+0000","last_change":"2024-08-31T07:09:47.665501+0000","last_active":"2024-08-31T07:10:46.257404+0000","last_peered":"2024-08-31T07:10:46.257404+0000","last_clean":"2024-08-31T07:10:46.257404+0000","last_became_active":"2024-08-31T07:09:47.665162+0000","last_became_peered":"2024-08-31T07:09:47.665162+0000","last_unstale":"2024-08-31T07:10:46.257404+0000","last_undegraded":"2024-08-31T07:10:46.257404+0000","last_fullsized":"2024-08-31T07:10:46.257404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:34:29.605836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021262+0000","last_change":"2024-08-31T07:09:45.636580+0000","last_active":"2024-08-31T07:10:46.021262+0000","last_peered":"2024-08-31T07:10:46.021262+0000","last_clean":"2024-08-31T07:10:46.021262+0000","last_became_active":"2024-08-31T07:09:45.636163+0000","last_became_peered":"2024-08-31T07:09:45.636163+0000","last_unstale":"2024-08-31T07:10:46.021262+0000","last_undegraded":"2024-08-31T07:10:46.021262+0000","last_fullsized":"2024-08-31T07:10:46.021262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:18:07.567858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257075+0000","last_change":"2024-08-31T07:09:47.671865+0000","last_active":"2024-08-31T07:10:46.257075+0000","last_peered":"2024-08-31T07:10:46.257075+0000","last_clean":"2024-08-31T07:10:46.257075+0000","last_became_active":"2024-08-31T07:09:47.671454+0000","last_became_peered":"2024-08-31T07:09:47.671454+0000","last_unstale":"2024-08-31T07:10:46.257075+0000","last_undegraded":"2024-08-31T07:10:46.257075+0000","last_fullsized":"2024-08-31T07:10:46.257075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:47.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'6","reported_seq":100,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:07.985597+0000","last_change":"2024-08-31T07:09:45.653217+0000","last_active":"2024-08-31T07:11:07.985597+0000","last_peered":"2024-08-31T07:11:07.985597+0000","last_clean":"2024-08-31T07:11:07.985597+0000","last_became_active":"2024-08-31T07:09:45.652819+0000","last_became_peered":"2024-08-31T07:09:45.652819+0000","last_unstale":"2024-08-31T07:11:07.985597+0000","last_undegraded":"2024-08-31T07:11:07.985597+0000","last_fullsized":"2024-08-31T07:11:07.985597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:25:11.454068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257248+0000","last_change":"2024-08-31T07:09:47.663652+0000","last_active":"2024-08-31T07:10:46.257248+0000","last_peered":"2024-08-31T07:10:46.257248+0000","last_clean":"2024-08-31T07:10:46.257248+0000","last_became_active":"2024-08-31T07:09:47.663274+0000","last_became_peered":"2024-08-31T07:09:47.663274+0000","last_unstale":"2024-08-31T07:10:46.257248+0000","last_undegraded":"2024-08-31T07:10:46.257248+0000","last_fullsized":"2024-08-31T07:10:46.257248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:31:59.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257370+0000","last_change":"2024-08-31T07:08:27.623298+0000","last_active":"2024-08-31T07:10:46.257370+0000","last_peered":"2024-08-31T07:10:46.257370+0000","last_clean":"2024-08-31T07:10:46.257370+0000","last_became_active":"2024-08-31T07:08:27.619072+0000","last_became_peered":"2024-08-31T07:08:27.619072+0000","last_unstale":"2024-08-31T07:10:46.257370+0000","last_undegraded":"2024-08-31T07:10:46.257370+0000","last_fullsized":"2024-08-31T07:10:46.257370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_clean_scrub_stamp":"2024-08-31T07:04:45.509398+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:07.967421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154268+0000","last_change":"2024-08-31T07:09:45.638104+0000","last_active":"2024-08-31T07:10:46.154268+0000","last_peered":"2024-08-31T07:10:46.154268+0000","last_clean":"2024-08-31T07:10:46.154268+0000","last_became_active":"2024-08-31T07:09:45.637693+0000","last_became_peered":"2024-08-31T07:09:45.637693+0000","last_unstale":"2024-08-31T07:10:46.154268+0000","last_undegraded":"2024-08-31T07:10:46.154268+0000","last_fullsized":"2024-08-31T07:10:46.154268+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:50:15.135061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015717+0000","last_change":"2024-08-31T07:09:47.664958+0000","last_active":"2024-08-31T07:10:46.015717+0000","last_peered":"2024-08-31T07:10:46.015717+0000","last_clean":"2024-08-31T07:10:46.015717+0000","last_became_active":"2024-08-31T07:09:47.661166+0000","last_became_peered":"2024-08-31T07:09:47.661166+0000","last_unstale":"2024-08-31T07:10:46.015717+0000","last_undegraded":"2024-08-31T07:10:46.015717+0000","last_fullsized":"2024-08-31T07:10:46.015717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:45:58.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257126+0000","last_change":"2024-08-31T07:09:45.640881+0000","last_active":"2024-08-31T07:10:46.257126+0000","last_peered":"2024-08-31T07:10:46.257126+0000","last_clean":"2024-08-31T07:10:46.257126+0000","last_became_active":"2024-08-31T07:09:45.640487+0000","last_became_peered":"2024-08-31T07:09:45.640487+0000","last_unstale":"2024-08-31T07:10:46.257126+0000","last_undegraded":"2024-08-31T07:10:46.257126+0000","last_fullsized":"2024-08-31T07:10:46.257126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:06.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017222+0000","last_change":"2024-08-31T07:09:47.655008+0000","last_active":"2024-08-31T07:10:46.017222+0000","last_peered":"2024-08-31T07:10:46.017222+0000","last_clean":"2024-08-31T07:10:46.017222+0000","last_became_active":"2024-08-31T07:09:47.654645+0000","last_became_peered":"2024-08-31T07:09:47.654645+0000","last_unstale":"2024-08-31T07:10:46.017222+0000","last_undegraded":"2024-08-31T07:10:46.017222+0000","last_fullsized":"2024-08-31T07:10:46.017222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:23:55.916053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":96,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:06.985608+0000","last_change":"2024-08-31T07:09:45.649324+0000","last_active":"2024-08-31T07:11:06.985608+0000","last_peered":"2024-08-31T07:11:06.985608+0000","last_clean":"2024-08-31T07:11:06.985608+0000","last_became_active":"2024-08-31T07:09:45.648915+0000","last_became_peered":"2024-08-31T07:09:45.648915+0000","last_unstale":"2024-08-31T07:11:06.985608+0000","last_undegraded":"2024-08-31T07:11:06.985608+0000","last_fullsized":"2024-08-31T07:11:06.985608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:31:57.853654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022278+0000","last_change":"2024-08-31T07:09:47.661328+0000","last_active":"2024-08-31T07:10:46.022278+0000","last_peered":"2024-08-31T07:10:46.022278+0000","last_clean":"2024-08-31T07:10:46.022278+0000","last_became_active":"2024-08-31T07:09:47.660927+0000","last_became_peered":"2024-08-31T07:09:47.660927+0000","last_unstale":"2024-08-31T07:10:46.022278+0000","last_undegraded":"2024-08-31T07:10:46.022278+0000","last_fullsized":"2024-08-31T07:10:46.022278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:40:46.876894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'6","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:06.986330+0000","last_change":"2024-08-31T07:09:45.637747+0000","last_active":"2024-08-31T07:11:06.986330+0000","last_peered":"2024-08-31T07:11:06.986330+0000","last_clean":"2024-08-31T07:11:06.986330+0000","last_became_active":"2024-08-31T07:09:45.637368+0000","last_became_peered":"2024-08-31T07:09:45.637368+0000","last_unstale":"2024-08-31T07:11:06.986330+0000","last_undegraded":"2024-08-31T07:11:06.986330+0000","last_fullsized":"2024-08-31T07:11:06.986330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:01.174109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019918+0000","last_change":"2024-08-31T07:09:47.680802+0000","last_active":"2024-08-31T07:10:46.019918+0000","last_peered":"2024-08-31T07:10:46.019918+0000","last_clean":"2024-08-31T07:10:46.019918+0000","last_became_active":"2024-08-31T07:09:47.680363+0000","last_became_peered":"2024-08-31T07:09:47.680363+0000","last_unstale":"2024-08-31T07:10:46.019918+0000","last_undegraded":"2024-08-31T07:10:46.019918+0000","last_fullsized":"2024-08-31T07:10:46.019918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:46:48.744071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324292+0000","last_change":"2024-08-31T07:09:45.651222+0000","last_active":"2024-08-31T07:10:46.324292+0000","last_peered":"2024-08-31T07:10:46.324292+0000","last_clean":"2024-08-31T07:10:46.324292+0000","last_became_active":"2024-08-31T07:09:45.650760+0000","last_became_peered":"2024-08-31T07:09:45.650760+0000","last_unstale":"2024-08-31T07:10:46.324292+0000","last_undegraded":"2024-08-31T07:10:46.324292+0000","last_fullsized":"2024-08-31T07:10:46.324292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T19:03:33.942165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020011+0000","last_change":"2024-08-31T07:09:47.665881+0000","last_active":"2024-08-31T07:10:46.020011+0000","last_peered":"2024-08-31T07:10:46.020011+0000","last_clean":"2024-08-31T07:10:46.020011+0000","last_became_active":"2024-08-31T07:09:47.665485+0000","last_became_peered":"2024-08-31T07:09:47.665485+0000","last_unstale":"2024-08-31T07:10:46.020011+0000","last_undegraded":"2024-08-31T07:10:46.020011+0000","last_fullsized":"2024-08-31T07:10:46.020011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:30.820714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020786+0000","last_change":"2024-08-31T07:09:45.651893+0000","last_active":"2024-08-31T07:10:46.020786+0000","last_peered":"2024-08-31T07:10:46.020786+0000","last_clean":"2024-08-31T07:10:46.020786+0000","last_became_active":"2024-08-31T07:09:45.651547+0000","last_became_peered":"2024-08-31T07:09:45.651547+0000","last_unstale":"2024-08-31T07:10:46.020786+0000","last_undegraded":"2024-08-31T07:10:46.020786+0000","last_fullsized":"2024-08-31T07:10:46.020786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:11:30.708018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017674+0000","last_change":"2024-08-31T07:09:47.655551+0000","last_active":"2024-08-31T07:10:46.017674+0000","last_peered":"2024-08-31T07:10:46.017674+0000","last_clean":"2024-08-31T07:10:46.017674+0000","last_became_active":"2024-08-31T07:09:47.655188+0000","last_became_peered":"2024-08-31T07:09:47.655188+0000","last_unstale":"2024-08-31T07:10:46.017674+0000","last_undegraded":"2024-08-31T07:10:46.017674+0000","last_fullsized":"2024-08-31T07:10:46.017674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:43:51.419599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'15","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324668+0000","last_change":"2024-08-31T07:09:45.636212+0000","last_active":"2024-08-31T07:10:46.324668+0000","last_peered":"2024-08-31T07:10:46.324668+0000","last_clean":"2024-08-31T07:10:46.324668+0000","last_became_active":"2024-08-31T07:09:45.635678+0000","last_became_peered":"2024-08-31T07:09:45.635678+0000","last_unstale":"2024-08-31T07:10:46.324668+0000","last_undegraded":"2024-08-31T07:10:46.324668+0000","last_fullsized":"2024-08-31T07:10:46.324668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:13:17.803459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016490+0000","last_change":"2024-08-31T07:09:47.666476+0000","last_active":"2024-08-31T07:10:46.016490+0000","last_peered":"2024-08-31T07:10:46.016490+0000","last_clean":"2024-08-31T07:10:46.016490+0000","last_became_active":"2024-08-31T07:09:47.666111+0000","last_became_peered":"2024-08-31T07:09:47.666111+0000","last_unstale":"2024-08-31T07:10:46.016490+0000","last_undegraded":"2024-08-31T07:10:46.016490+0000","last_fullsized":"2024-08-31T07:10:46.016490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:08:53.459549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154374+0000","last_change":"2024-08-31T07:09:45.631571+0000","last_active":"2024-08-31T07:10:46.154374+0000","last_peered":"2024-08-31T07:10:46.154374+0000","last_clean":"2024-08-31T07:10:46.154374+0000","last_became_active":"2024-08-31T07:09:45.630981+0000","last_became_peered":"2024-08-31T07:09:45.630981+0000","last_unstale":"2024-08-31T07:10:46.154374+0000","last_undegraded":"2024-08-31T07:10:46.154374+0000","last_fullsized":"2024-08-31T07:10:46.154374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:36:53.482955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324734+0000","last_change":"2024-08-31T07:09:47.675054+0000","last_active":"2024-08-31T07:10:46.324734+0000","last_peered":"2024-08-31T07:10:46.324734+0000","last_clean":"2024-08-31T07:10:46.324734+0000","last_became_active":"2024-08-31T07:09:47.672652+0000","last_became_peered":"2024-08-31T07:09:47.672652+0000","last_unstale":"2024-08-31T07:10:46.324734+0000","last_undegraded":"2024-08-31T07:10:46.324734+0000","last_fullsized":"2024-08-31T07:10:46.324734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:32:05.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016218+0000","last_change":"2024-08-31T07:09:45.648746+0000","last_active":"2024-08-31T07:10:46.016218+0000","last_peered":"2024-08-31T07:10:46.016218+0000","last_clean":"2024-08-31T07:10:46.016218+0000","last_became_active":"2024-08-31T07:09:45.648424+0000","last_became_peered":"2024-08-31T07:09:45.648424+0000","last_unstale":"2024-08-31T07:10:46.016218+0000","last_undegraded":"2024-08-31T07:10:46.016218+0000","last_fullsized":"2024-08-31T07:10:46.016218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:40:32.132963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020225+0000","last_change":"2024-08-31T07:09:47.663213+0000","last_active":"2024-08-31T07:10:46.020225+0000","last_peered":"2024-08-31T07:10:46.020225+0000","last_clean":"2024-08-31T07:10:46.020225+0000","last_became_active":"2024-08-31T07:09:47.662457+0000","last_became_peered":"2024-08-31T07:09:47.662457+0000","last_unstale":"2024-08-31T07:10:46.020225+0000","last_undegraded":"2024-08-31T07:10:46.020225+0000","last_fullsized":"2024-08-31T07:10:46.020225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:54:27.069200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015595+0000","last_change":"2024-08-31T07:09:45.640149+0000","last_active":"2024-08-31T07:10:46.015595+0000","last_peered":"2024-08-31T07:10:46.015595+0000","last_clean":"2024-08-31T07:10:46.015595+0000","last_became_active":"2024-08-31T07:09:45.639581+0000","last_became_peered":"2024-08-31T07:09:45.639581+0000","last_unstale":"2024-08-31T07:10:46.015595+0000","last_undegraded":"2024-08-31T07:10:46.015595+0000","last_fullsized":"2024-08-31T07:10:46.015595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:27:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.018016+0000","last_change":"2024-08-31T07:09:47.672613+0000","last_active":"2024-08-31T07:10:46.018016+0000","last_peered":"2024-08-31T07:10:46.018016+0000","last_clean":"2024-08-31T07:10:46.018016+0000","last_became_active":"2024-08-31T07:09:47.672173+0000","last_became_peered":"2024-08-31T07:09:47.672173+0000","last_unstale":"2024-08-31T07:10:46.018016+0000","last_undegraded":"2024-08-31T07:10:46.018016+0000","last_fullsized":"2024-08-31T07:10:46.018016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:49.706113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022533+0000","last_change":"2024-08-31T07:09:45.637567+0000","last_active":"2024-08-31T07:10:46.022533+0000","last_peered":"2024-08-31T07:10:46.022533+0000","last_clean":"2024-08-31T07:10:46.022533+0000","last_became_active":"2024-08-31T07:09:45.637180+0000","last_became_peered":"2024-08-31T07:09:45.637180+0000","last_unstale":"2024-08-31T07:10:46.022533+0000","last_undegraded":"2024-08-31T07:10:46.022533+0000","last_fullsized":"2024-08-31T07:10:46.022533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:48:07.190829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016578+0000","last_change":"2024-08-31T07:09:47.674980+0000","last_active":"2024-08-31T07:10:46.016578+0000","last_peered":"2024-08-31T07:10:46.016578+0000","last_clean":"2024-08-31T07:10:46.016578+0000","last_became_active":"2024-08-31T07:09:47.674528+0000","last_became_peered":"2024-08-31T07:09:47.674528+0000","last_unstale":"2024-08-31T07:10:46.016578+0000","last_undegraded":"2024-08-31T07:10:46.016578+0000","last_fullsized":"2024-08-31T07:10:46.016578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:15:09.464221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021024+0000","last_change":"2024-08-31T07:09:45.641167+0000","last_active":"2024-08-31T07:10:46.021024+0000","last_peered":"2024-08-31T07:10:46.021024+0000","last_clean":"2024-08-31T07:10:46.021024+0000","last_became_active":"2024-08-31T07:09:45.640761+0000","last_became_peered":"2024-08-31T07:09:45.640761+0000","last_unstale":"2024-08-31T07:10:46.021024+0000","last_undegraded":"2024-08-31T07:10:46.021024+0000","last_fullsized":"2024-08-31T07:10:46.021024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:28:59.963145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257534+0000","last_change":"2024-08-31T07:09:47.668808+0000","last_active":"2024-08-31T07:10:46.257534+0000","last_peered":"2024-08-31T07:10:46.257534+0000","last_clean":"2024-08-31T07:10:46.257534+0000","last_became_active":"2024-08-31T07:09:47.668291+0000","last_became_peered":"2024-08-31T07:09:47.668291+0000","last_unstale":"2024-08-31T07:10:46.257534+0000","last_undegraded":"2024-08-31T07:10:46.257534+0000","last_fullsized":"2024-08-31T07:10:46.257534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:23.433511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020994+0000","last_change":"2024-08-31T07:09:47.668039+0000","last_active":"2024-08-31T07:10:46.020994+0000","last_peered":"2024-08-31T07:10:46.020994+0000","last_clean":"2024-08-31T07:10:46.020994+0000","last_became_active":"2024-08-31T07:09:47.667567+0000","last_became_peered":"2024-08-31T07:09:47.667567+0000","last_unstale":"2024-08-31T07:10:46.020994+0000","last_undegraded":"2024-08-31T07:10:46.020994+0000","last_fullsized":"2024-08-31T07:10:46.020994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:33:50.192864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.041708+0000","last_change":"2024-08-31T07:10:46.041197+0000","last_active":"2024-08-31T07:10:46.041708+0000","last_peered":"2024-08-31T07:10:46.041708+0000","last_clean":"2024-08-31T07:10:46.041708+0000","last_became_active":"2024-08-31T07:10:46.023966+0000","last_became_peered":"2024-08-31T07:10:46.023966+0000","last_unstale":"2024-08-31T07:10:46.041708+0000","last_undegraded":"2024-08-31T07:10:46.041708+0000","last_fullsized":"2024-08-31T07:10:46.041708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:34.302665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016265+0000","last_change":"2024-08-31T07:09:47.660615+0000","last_active":"2024-08-31T07:10:46.016265+0000","last_peered":"2024-08-31T07:10:46.016265+0000","last_clean":"2024-08-31T07:10:46.016265+0000","last_became_active":"2024-08-31T07:09:47.660269+0000","last_became_peered":"2024-08-31T07:09:47.660269+0000","last_unstale":"2024-08-31T07:10:46.016265+0000","last_undegraded":"2024-08-31T07:10:46.016265+0000","last_fullsized":"2024-08-31T07:10:46.016265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:42:12.618039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020632+0000","last_change":"2024-08-31T07:09:45.641699+0000","last_active":"2024-08-31T07:10:46.020632+0000","last_peered":"2024-08-31T07:10:46.020632+0000","last_clean":"2024-08-31T07:10:46.020632+0000","last_became_active":"2024-08-31T07:09:45.641122+0000","last_became_peered":"2024-08-31T07:09:45.641122+0000","last_unstale":"2024-08-31T07:10:46.020632+0000","last_undegraded":"2024-08-31T07:10:46.020632+0000","last_fullsized":"2024-08-31T07:10:46.020632+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:26:07.130266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324235+0000","last_change":"2024-08-31T07:09:47.675459+0000","last_active":"2024-08-31T07:10:46.324235+0000","last_peered":"2024-08-31T07:10:46.324235+0000","last_clean":"2024-08-31T07:10:46.324235+0000","last_became_active":"2024-08-31T07:09:47.672246+0000","last_became_peered":"2024-08-31T07:09:47.672246+0000","last_unstale":"2024-08-31T07:10:46.324235+0000","last_undegraded":"2024-08-31T07:10:46.324235+0000","last_fullsized":"2024-08-31T07:10:46.324235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:49:23.547352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016380+0000","last_change":"2024-08-31T07:09:45.649522+0000","last_active":"2024-08-31T07:10:46.016380+0000","last_peered":"2024-08-31T07:10:46.016380+0000","last_clean":"2024-08-31T07:10:46.016380+0000","last_became_active":"2024-08-31T07:09:45.649109+0000","last_became_peered":"2024-08-31T07:09:45.649109+0000","last_unstale":"2024-08-31T07:10:46.016380+0000","last_undegraded":"2024-08-31T07:10:46.016380+0000","last_fullsized":"2024-08-31T07:10:46.016380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:52:52.432605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016283+0000","last_change":"2024-08-31T07:09:47.670583+0000","last_active":"2024-08-31T07:10:46.016283+0000","last_peered":"2024-08-31T07:10:46.016283+0000","last_clean":"2024-08-31T07:10:46.016283+0000","last_became_active":"2024-08-31T07:09:47.670162+0000","last_became_peered":"2024-08-31T07:09:47.670162+0000","last_unstale":"2024-08-31T07:10:46.016283+0000","last_undegraded":"2024-08-31T07:10:46.016283+0000","last_fullsized":"2024-08-31T07:10:46.016283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:35:35.135678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":106,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:08.988655+0000","last_change":"2024-08-31T07:09:45.639636+0000","last_active":"2024-08-31T07:11:08.988655+0000","last_peered":"2024-08-31T07:11:08.988655+0000","last_clean":"2024-08-31T07:11:08.988655+0000","last_became_active":"2024-08-31T07:09:45.639029+0000","last_became_peered":"2024-08-31T07:09:45.639029+0000","last_unstale":"2024-08-31T07:11:08.988655+0000","last_undegraded":"2024-08-31T07:11:08.988655+0000","last_fullsized":"2024-08-31T07:11:08.988655+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:54.110149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68603,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":713,"num_read_kb":432,"num_write":247,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":650,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285320,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678047,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":884736,"data_stored":434353,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874276,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1028096,"data_stored":1085349,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037801,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":897024,"data_stored":433242,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234032,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":888832,"data_stored":433038,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397556,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":1026920,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561080,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":880640,"data_stored":433350,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":1025816,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":880640,"data_stored":433211,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117066,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":432784,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378704,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":495423,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":494880,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":901120,"data_stored":434689,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T07:11:14.627 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-31T07:11:14.628 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-31T07:11:14.628 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-31T07:11:14.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph health --format=json 2024-08-31T07:11:14.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:14 smithi073 ceph-mon[21995]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:15.006 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:15.460 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:15 smithi032 ceph-mon[17252]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:11:15.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:15 smithi084 ceph-mon[22387]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:11:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.637+0000 7f233e400640 1 -- 172.21.15.32:0/1072569867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233810b290 msgr2=0x7f233810b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.637+0000 7f233e400640 1 --2- 172.21.15.32:0/1072569867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233810b290 0x7f233810b6a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2324009900 tx=0x7f232402a410 comp rx=0 tx=0).stop 2024-08-31T07:11:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.637+0000 7f233e400640 1 -- 172.21.15.32:0/1072569867 shutdown_connections 2024-08-31T07:11:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.637+0000 7f233e400640 1 --2- 172.21.15.32:0/1072569867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f233810bbe0 0x7f2338112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:15.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.637+0000 7f233e400640 1 --2- 172.21.15.32:0/1072569867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233810b290 0x7f233810b6a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:15.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.637+0000 7f233e400640 1 --2- 172.21.15.32:0/1072569867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2338105950 0x7f2338105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:15.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.637+0000 7f233e400640 1 -- 172.21.15.32:0/1072569867 >> 172.21.15.32:0/1072569867 conn(0x7f2338101420 msgr2=0x7f2338103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:15.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.637+0000 7f233e400640 1 -- 172.21.15.32:0/1072569867 shutdown_connections 2024-08-31T07:11:15.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.637+0000 7f233e400640 1 -- 172.21.15.32:0/1072569867 wait complete. 2024-08-31T07:11:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.638+0000 7f233e400640 1 Processor -- start 2024-08-31T07:11:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.638+0000 7f233e400640 1 -- start start 2024-08-31T07:11:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.638+0000 7f233e400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2338105950 0x7f233819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.638+0000 7f233e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233810b290 0x7f233819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.638+0000 7f233e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f233810bbe0 0x7f23381a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f233e400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f23381158a0 con 0x7f233810b290 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f233e400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2338115720 con 0x7f233810bbe0 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f233e400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2338115a20 con 0x7f2338105950 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f2337e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233810b290 0x7f233819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f233d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2338105950 0x7f233819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f2337e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233810b290 0x7f233819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36370/0 (socket says 172.21.15.32:36370) 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f233d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2338105950 0x7f233819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37962/0 (socket says 172.21.15.32:37962) 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f2337e00640 1 -- 172.21.15.32:0/901960944 learned_addr learned my addr 172.21.15.32:0/901960944 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f233da00640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f233810bbe0 0x7f23381a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f2337e00640 1 -- 172.21.15.32:0/901960944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2338105950 msgr2=0x7f233819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f2337e00640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2338105950 0x7f233819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f2337e00640 1 -- 172.21.15.32:0/901960944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f233810bbe0 msgr2=0x7f23381a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f2337e00640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f233810bbe0 0x7f23381a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.639+0000 7f2337e00640 1 -- 172.21.15.32:0/901960944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23381a2220 con 0x7f233810b290 2024-08-31T07:11:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.640+0000 7f2337e00640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233810b290 0x7f233819c850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2324006010 tx=0x7f2324004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.640+0000 7f2335600640 1 -- 172.21.15.32:0/901960944 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232403f030 con 0x7f233810b290 2024-08-31T07:11:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.640+0000 7f2335600640 1 -- 172.21.15.32:0/901960944 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2324003aa0 con 0x7f233810b290 2024-08-31T07:11:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.640+0000 7f233e400640 1 -- 172.21.15.32:0/901960944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23381a24b0 con 0x7f233810b290 2024-08-31T07:11:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.640+0000 7f2335600640 1 -- 172.21.15.32:0/901960944 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324035070 con 0x7f233810b290 2024-08-31T07:11:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.640+0000 7f233e400640 1 -- 172.21.15.32:0/901960944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23381a8c70 con 0x7f233810b290 2024-08-31T07:11:15.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.641+0000 7f233e400640 1 -- 172.21.15.32:0/901960944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2338105d60 con 0x7f233810b290 2024-08-31T07:11:15.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.644+0000 7f2335600640 1 -- 172.21.15.32:0/901960944 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f2324042030 con 0x7f233810b290 2024-08-31T07:11:15.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.647+0000 7f2335600640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f23200b4c90 0x7f23200b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:15.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.648+0000 7f2335600640 1 -- 172.21.15.32:0/901960944 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2324105bd0 con 0x7f233810b290 2024-08-31T07:11:15.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.648+0000 7f233d000640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f23200b4c90 0x7f23200b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:15.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.648+0000 7f233d000640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f23200b4c90 0x7f23200b71a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f232c009bd0 tx=0x7f232c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:15.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.649+0000 7f2335600640 1 -- 172.21.15.32:0/901960944 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f23240cc450 con 0x7f233810b290 2024-08-31T07:11:15.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:15 smithi073 ceph-mon[21995]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:11:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.890+0000 7f233e400640 1 -- 172.21.15.32:0/901960944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f233819d3e0 con 0x7f233810b290 2024-08-31T07:11:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.890+0000 7f2335600640 1 -- 172.21.15.32:0/901960944 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f23240d0c10 con 0x7f233810b290 2024-08-31T07:11:15.891 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:15.891 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 -- 172.21.15.32:0/901960944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f23200b4c90 msgr2=0x7f23200b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f23200b4c90 0x7f23200b71a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f232c009bd0 tx=0x7f232c005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 -- 172.21.15.32:0/901960944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233810b290 msgr2=0x7f233819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233810b290 0x7f233819c850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2324006010 tx=0x7f2324004060 comp rx=0 tx=0).stop 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 -- 172.21.15.32:0/901960944 shutdown_connections 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f23200b4c90 0x7f23200b71a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f233810bbe0 0x7f23381a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f233810b290 0x7f233819c850 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 --2- 172.21.15.32:0/901960944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2338105950 0x7f233819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 -- 172.21.15.32:0/901960944 >> 172.21.15.32:0/901960944 conn(0x7f2338101420 msgr2=0x7f2338103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 -- 172.21.15.32:0/901960944 shutdown_connections 2024-08-31T07:11:15.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:15.893+0000 7f233e400640 1 -- 172.21.15.32:0/901960944 wait complete. 2024-08-31T07:11:16.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:16 smithi084 ceph-mon[22387]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:16.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:16 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/901960944' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-31T07:11:16.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:16 smithi032 ceph-mon[17252]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:16.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:16 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/901960944' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-31T07:11:16.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:16 smithi073 ceph-mon[21995]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:16.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:16 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/901960944' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-31T07:11:16.768 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-31T07:11:16.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:11:17.146 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.806+0000 7f22f6c00640 1 -- 172.21.15.32:0/3778987518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22f010fe70 msgr2=0x7f22f01122b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.806+0000 7f22f6c00640 1 --2- 172.21.15.32:0/3778987518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22f010fe70 0x7f22f01122b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009ca0 tx=0x7f22e802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:11:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.806+0000 7f22f6c00640 1 -- 172.21.15.32:0/3778987518 shutdown_connections 2024-08-31T07:11:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.806+0000 7f22f6c00640 1 --2- 172.21.15.32:0/3778987518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22f010fe70 0x7f22f01122b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.806+0000 7f22f6c00640 1 --2- 172.21.15.32:0/3778987518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f0075ea0 0x7f22f010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.806+0000 7f22f6c00640 1 --2- 172.21.15.32:0/3778987518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22f0075550 0x7f22f0075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:17.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.806+0000 7f22f6c00640 1 -- 172.21.15.32:0/3778987518 >> 172.21.15.32:0/3778987518 conn(0x7f22f00ff380 msgr2=0x7f22f01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.806+0000 7f22f6c00640 1 -- 172.21.15.32:0/3778987518 shutdown_connections 2024-08-31T07:11:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.806+0000 7f22f6c00640 1 -- 172.21.15.32:0/3778987518 wait complete. 2024-08-31T07:11:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.807+0000 7f22f6c00640 1 Processor -- start 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.807+0000 7f22f6c00640 1 -- start start 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.807+0000 7f22f6c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f0075550 0x7f22f019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.807+0000 7f22f6c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22f0075ea0 0x7f22f019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f6c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22f010fe70 0x7f22f01a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f6c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0115a70 con 0x7f22f010fe70 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f6c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01158f0 con 0x7f22f0075550 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f6c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0115bf0 con 0x7f22f0075ea0 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f0075550 0x7f22f019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22f010fe70 0x7f22f01a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f0075550 0x7f22f019c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36162/0 (socket says 172.21.15.32:36162) 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22efe00640 1 -- 172.21.15.32:0/3860618090 learned_addr learned my addr 172.21.15.32:0/3860618090 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22f010fe70 0x7f22f01a1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54672/0 (socket says 172.21.15.32:54672) 2024-08-31T07:11:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22ef400640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22f0075ea0 0x7f22f019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f4c00640 1 -- 172.21.15.32:0/3860618090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22f0075ea0 msgr2=0x7f22f019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f4c00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22f0075ea0 0x7f22f019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:17.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f4c00640 1 -- 172.21.15.32:0/3860618090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f0075550 msgr2=0x7f22f019c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f4c00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f0075550 0x7f22f019c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22f4c00640 1 -- 172.21.15.32:0/3860618090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f01a2310 con 0x7f22f010fe70 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22ef400640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22f0075ea0 0x7f22f019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.808+0000 7f22efe00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f0075550 0x7f22f019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.809+0000 7f22f4c00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22f010fe70 0x7f22f01a1be0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f22e8005e30 tx=0x7f22e80317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.809+0000 7f22ecc00640 1 -- 172.21.15.32:0/3860618090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8003b70 con 0x7f22f010fe70 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.809+0000 7f22f6c00640 1 -- 172.21.15.32:0/3860618090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f01a25a0 con 0x7f22f010fe70 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.809+0000 7f22ecc00640 1 -- 172.21.15.32:0/3860618090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e8003d10 con 0x7f22f010fe70 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.809+0000 7f22ecc00640 1 -- 172.21.15.32:0/3860618090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e802f970 con 0x7f22f010fe70 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.809+0000 7f22f6c00640 1 -- 172.21.15.32:0/3860618090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f01a8d60 con 0x7f22f010fe70 2024-08-31T07:11:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.810+0000 7f22f6c00640 1 -- 172.21.15.32:0/3860618090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f0075960 con 0x7f22f010fe70 2024-08-31T07:11:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.813+0000 7f22ecc00640 1 -- 172.21.15.32:0/3860618090 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f22e8004b40 con 0x7f22f010fe70 2024-08-31T07:11:17.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.815+0000 7f22ecc00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22d40b4c10 0x7f22d40b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:17.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.816+0000 7f22ecc00640 1 -- 172.21.15.32:0/3860618090 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f22e8106540 con 0x7f22f010fe70 2024-08-31T07:11:17.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.816+0000 7f22efe00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22d40b4c10 0x7f22d40b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:17.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.816+0000 7f22efe00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22d40b4c10 0x7f22d40b7120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22e0011500 tx=0x7f22e0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:17.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:17.817+0000 7f22ecc00640 1 -- 172.21.15.32:0/3860618090 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f22e80cd410 con 0x7f22f010fe70 2024-08-31T07:11:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.051+0000 7f22f6c00640 1 -- 172.21.15.32:0/3860618090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22f01a8ff0 con 0x7f22f010fe70 2024-08-31T07:11:18.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.052+0000 7f22ecc00640 1 -- 172.21.15.32:0/3860618090 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7f22e80ccd10 con 0x7f22f010fe70 2024-08-31T07:11:18.053 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2024-08-31T07:11:18.054 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"btime":"2024-08-31T07:10:05:878594+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:10:04.962915+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:11:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.056+0000 7f22f6c00640 1 -- 172.21.15.32:0/3860618090 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22d40b4c10 msgr2=0x7f22d40b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.056+0000 7f22f6c00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22d40b4c10 0x7f22d40b7120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22e0011500 tx=0x7f22e0005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.056+0000 7f22f6c00640 1 -- 172.21.15.32:0/3860618090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22f010fe70 msgr2=0x7f22f01a1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.056+0000 7f22f6c00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22f010fe70 0x7f22f01a1be0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f22e8005e30 tx=0x7f22e80317c0 comp rx=0 tx=0).stop 2024-08-31T07:11:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.057+0000 7f22f6c00640 1 -- 172.21.15.32:0/3860618090 shutdown_connections 2024-08-31T07:11:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.057+0000 7f22f6c00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22d40b4c10 0x7f22d40b7120 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.057+0000 7f22f6c00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22f010fe70 0x7f22f01a1be0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.057+0000 7f22f6c00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22f0075ea0 0x7f22f019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.057+0000 7f22f6c00640 1 --2- 172.21.15.32:0/3860618090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f0075550 0x7f22f019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.057+0000 7f22f6c00640 1 -- 172.21.15.32:0/3860618090 >> 172.21.15.32:0/3860618090 conn(0x7f22f00ff380 msgr2=0x7f22f0109980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.057+0000 7f22f6c00640 1 -- 172.21.15.32:0/3860618090 shutdown_connections 2024-08-31T07:11:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:18.057+0000 7f22f6c00640 1 -- 172.21.15.32:0/3860618090 wait complete. 2024-08-31T07:11:18.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:18 smithi032 ceph-mon[17252]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:18.373 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:18 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3860618090' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:18.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:18 smithi084 ceph-mon[22387]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:18.493 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:18 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3860618090' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:18.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:18 smithi073 ceph-mon[21995]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:18.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:18 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3860618090' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:19.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:11:19.386 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:20.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.039+0000 7fd97b200640 1 -- 172.21.15.32:0/1094831171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c105960 msgr2=0x7fd97c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.039+0000 7fd97b200640 1 --2- 172.21.15.32:0/1094831171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c105960 0x7fd97c105d70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009900 tx=0x7fd96c02a410 comp rx=0 tx=0).stop 2024-08-31T07:11:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.039+0000 7fd97b200640 1 -- 172.21.15.32:0/1094831171 shutdown_connections 2024-08-31T07:11:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.039+0000 7fd97b200640 1 --2- 172.21.15.32:0/1094831171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd97c10bbf0 0x7fd97c112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.039+0000 7fd97b200640 1 --2- 172.21.15.32:0/1094831171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd97c10b2a0 0x7fd97c10b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.039+0000 7fd97b200640 1 --2- 172.21.15.32:0/1094831171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c105960 0x7fd97c105d70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.039+0000 7fd97b200640 1 -- 172.21.15.32:0/1094831171 >> 172.21.15.32:0/1094831171 conn(0x7fd97c101410 msgr2=0x7fd97c103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.039+0000 7fd97b200640 1 -- 172.21.15.32:0/1094831171 shutdown_connections 2024-08-31T07:11:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.039+0000 7fd97b200640 1 -- 172.21.15.32:0/1094831171 wait complete. 2024-08-31T07:11:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.040+0000 7fd97b200640 1 Processor -- start 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd97b200640 1 -- start start 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd97b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c105960 0x7fd97c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd97b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd97c10b2a0 0x7fd97c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd97b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd97c10bbf0 0x7fd97c1a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd97b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c115900 con 0x7fd97c105960 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd97b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c115780 con 0x7fd97c10bbf0 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd97b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c115a80 con 0x7fd97c10b2a0 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd979e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c105960 0x7fd97c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd979e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c105960 0x7fd97c19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54680/0 (socket says 172.21.15.32:54680) 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd979e00640 1 -- 172.21.15.32:0/732003660 learned_addr learned my addr 172.21.15.32:0/732003660 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.041+0000 7fd979400640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd97c10b2a0 0x7fd97c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.042+0000 7fd97a800640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd97c10bbf0 0x7fd97c1a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.042+0000 7fd979e00640 1 -- 172.21.15.32:0/732003660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd97c10b2a0 msgr2=0x7fd97c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.042+0000 7fd979e00640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd97c10b2a0 0x7fd97c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.042+0000 7fd979e00640 1 -- 172.21.15.32:0/732003660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd97c10bbf0 msgr2=0x7fd97c1a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.042+0000 7fd979e00640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd97c10bbf0 0x7fd97c1a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.042+0000 7fd979e00640 1 -- 172.21.15.32:0/732003660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c1a2340 con 0x7fd97c105960 2024-08-31T07:11:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.042+0000 7fd979400640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd97c10b2a0 0x7fd97c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.042+0000 7fd979e00640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c105960 0x7fd97c19c3d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd96c005ab0 tx=0x7fd96c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.043+0000 7fd972a00640 1 -- 172.21.15.32:0/732003660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c03f070 con 0x7fd97c105960 2024-08-31T07:11:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.043+0000 7fd97b200640 1 -- 172.21.15.32:0/732003660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c1a25d0 con 0x7fd97c105960 2024-08-31T07:11:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.043+0000 7fd972a00640 1 -- 172.21.15.32:0/732003660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd96c031b60 con 0x7fd97c105960 2024-08-31T07:11:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.043+0000 7fd972a00640 1 -- 172.21.15.32:0/732003660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c035030 con 0x7fd97c105960 2024-08-31T07:11:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.043+0000 7fd97b200640 1 -- 172.21.15.32:0/732003660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c1a8d90 con 0x7fd97c105960 2024-08-31T07:11:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.044+0000 7fd97b200640 1 -- 172.21.15.32:0/732003660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c105d70 con 0x7fd97c105960 2024-08-31T07:11:20.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.046+0000 7fd972a00640 1 -- 172.21.15.32:0/732003660 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fd96c042070 con 0x7fd97c105960 2024-08-31T07:11:20.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.048+0000 7fd972a00640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd9600b4c10 0x7fd9600b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:20.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.048+0000 7fd972a00640 1 -- 172.21.15.32:0/732003660 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fd96c105b10 con 0x7fd97c105960 2024-08-31T07:11:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.048+0000 7fd979400640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd9600b4c10 0x7fd9600b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.049+0000 7fd979400640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd9600b4c10 0x7fd9600b7120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd97c104b10 tx=0x7fd968005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:20.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.052+0000 7fd972a00640 1 -- 172.21.15.32:0/732003660 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd96c0cc390 con 0x7fd97c105960 2024-08-31T07:11:20.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.288+0000 7fd97b200640 1 -- 172.21.15.32:0/732003660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd97c10d9f0 con 0x7fd97c105960 2024-08-31T07:11:20.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.288+0000 7fd972a00640 1 -- 172.21.15.32:0/732003660 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7fd96c0d0b50 con 0x7fd97c105960 2024-08-31T07:11:20.289 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:20.290 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"btime":"2024-08-31T07:10:05:878594+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:10:04.962915+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:11:20.290 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2024-08-31T07:11:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 -- 172.21.15.32:0/732003660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd9600b4c10 msgr2=0x7fd9600b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd9600b4c10 0x7fd9600b7120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd97c104b10 tx=0x7fd968005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 -- 172.21.15.32:0/732003660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c105960 msgr2=0x7fd97c19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c105960 0x7fd97c19c3d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd96c005ab0 tx=0x7fd96c003c10 comp rx=0 tx=0).stop 2024-08-31T07:11:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 -- 172.21.15.32:0/732003660 shutdown_connections 2024-08-31T07:11:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd9600b4c10 0x7fd9600b7120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd97c10bbf0 0x7fd97c1a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd97c10b2a0 0x7fd97c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 --2- 172.21.15.32:0/732003660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd97c105960 0x7fd97c19c3d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 -- 172.21.15.32:0/732003660 >> 172.21.15.32:0/732003660 conn(0x7fd97c101410 msgr2=0x7fd97c10a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 -- 172.21.15.32:0/732003660 shutdown_connections 2024-08-31T07:11:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:20.291+0000 7fd97b200640 1 -- 172.21.15.32:0/732003660 wait complete. 2024-08-31T07:11:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:20 smithi084 ceph-mon[22387]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:20.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:20 smithi032 ceph-mon[17252]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:20 smithi073 ceph-mon[21995]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:21.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:11:21.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:21 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/732003660' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:21.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:21 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/732003660' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:21.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:21 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/732003660' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.255+0000 7f1459a00640 1 -- 172.21.15.32:0/2676859027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145410b290 msgr2=0x7f145410b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.255+0000 7f1459a00640 1 --2- 172.21.15.32:0/2676859027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145410b290 0x7f145410b6a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1440009900 tx=0x7f144002a410 comp rx=0 tx=0).stop 2024-08-31T07:11:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.255+0000 7f1459a00640 1 -- 172.21.15.32:0/2676859027 shutdown_connections 2024-08-31T07:11:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.255+0000 7f1459a00640 1 --2- 172.21.15.32:0/2676859027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145410bbe0 0x7f1454112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.255+0000 7f1459a00640 1 --2- 172.21.15.32:0/2676859027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145410b290 0x7f145410b6a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.255+0000 7f1459a00640 1 --2- 172.21.15.32:0/2676859027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1454105950 0x7f1454105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.255+0000 7f1459a00640 1 -- 172.21.15.32:0/2676859027 >> 172.21.15.32:0/2676859027 conn(0x7f1454101420 msgr2=0x7f1454103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.255+0000 7f1459a00640 1 -- 172.21.15.32:0/2676859027 shutdown_connections 2024-08-31T07:11:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.255+0000 7f1459a00640 1 -- 172.21.15.32:0/2676859027 wait complete. 2024-08-31T07:11:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.256+0000 7f1459a00640 1 Processor -- start 2024-08-31T07:11:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.256+0000 7f1459a00640 1 -- start start 2024-08-31T07:11:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1459a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1454105950 0x7f145419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1459a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145410b290 0x7f145419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1459a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145410bbe0 0x7f14541a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1459a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14541158f0 con 0x7f145410b290 2024-08-31T07:11:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1459a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1454115770 con 0x7f1454105950 2024-08-31T07:11:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1459a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1454115a70 con 0x7f145410bbe0 2024-08-31T07:11:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1453400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145410b290 0x7f145419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1453400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145410b290 0x7f145419c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54694/0 (socket says 172.21.15.32:54694) 2024-08-31T07:11:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1453400640 1 -- 172.21.15.32:0/4270364951 learned_addr learned my addr 172.21.15.32:0/4270364951 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1453e00640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1454105950 0x7f145419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.257+0000 7f1459000640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145410bbe0 0x7f14541a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1453400640 1 -- 172.21.15.32:0/4270364951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145410bbe0 msgr2=0x7f14541a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1453400640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145410bbe0 0x7f14541a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1453400640 1 -- 172.21.15.32:0/4270364951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1454105950 msgr2=0x7f145419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1453400640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1454105950 0x7f145419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1453400640 1 -- 172.21.15.32:0/4270364951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14541a2250 con 0x7f145410b290 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1459000640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145410bbe0 0x7f14541a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1453400640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145410b290 0x7f145419c880 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1440005ab0 tx=0x7f1440003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1450c00640 1 -- 172.21.15.32:0/4270364951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144003f070 con 0x7f145410b290 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1450c00640 1 -- 172.21.15.32:0/4270364951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1440031b50 con 0x7f145410b290 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1459a00640 1 -- 172.21.15.32:0/4270364951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14541a24e0 con 0x7f145410b290 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.258+0000 7f1450c00640 1 -- 172.21.15.32:0/4270364951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440035030 con 0x7f145410b290 2024-08-31T07:11:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.259+0000 7f1459a00640 1 -- 172.21.15.32:0/4270364951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14541a8ca0 con 0x7f145410b290 2024-08-31T07:11:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.260+0000 7f1459a00640 1 -- 172.21.15.32:0/4270364951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1454105d60 con 0x7f145410b290 2024-08-31T07:11:22.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.264+0000 7f1450c00640 1 -- 172.21.15.32:0/4270364951 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f1440042070 con 0x7f145410b290 2024-08-31T07:11:22.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.265+0000 7f1450c00640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14380b4c10 0x7f14380b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:22.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.265+0000 7f1450c00640 1 -- 172.21.15.32:0/4270364951 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1440105930 con 0x7f145410b290 2024-08-31T07:11:22.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.265+0000 7f1453e00640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14380b4c10 0x7f14380b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:22.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.266+0000 7f1453e00640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14380b4c10 0x7f14380b7120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1444009bd0 tx=0x7f1444005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:22.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.268+0000 7f1450c00640 1 -- 172.21.15.32:0/4270364951 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f14400cc210 con 0x7f145410b290 2024-08-31T07:11:22.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:22 smithi032 ceph-mon[17252]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:22.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.463+0000 7f1459a00640 1 -- 172.21.15.32:0/4270364951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f145402e430 con 0x7f145410b290 2024-08-31T07:11:22.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.464+0000 7f1450c00640 1 -- 172.21.15.32:0/4270364951 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22921 (secure 0 0 0) 0x7f14400d09d0 con 0x7f145410b290 2024-08-31T07:11:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:22.465 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:10:46.011475+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:11:22.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.466+0000 7f1459a00640 1 -- 172.21.15.32:0/4270364951 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14380b4c10 msgr2=0x7f14380b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:22.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.466+0000 7f1459a00640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14380b4c10 0x7f14380b7120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1444009bd0 tx=0x7f1444005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:22.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.466+0000 7f1459a00640 1 -- 172.21.15.32:0/4270364951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145410b290 msgr2=0x7f145419c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:22.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.466+0000 7f1459a00640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145410b290 0x7f145419c880 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1440005ab0 tx=0x7f1440003b90 comp rx=0 tx=0).stop 2024-08-31T07:11:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.466+0000 7f1459a00640 1 -- 172.21.15.32:0/4270364951 shutdown_connections 2024-08-31T07:11:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.467+0000 7f1459a00640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14380b4c10 0x7f14380b7120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.467+0000 7f1459a00640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145410bbe0 0x7f14541a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.467+0000 7f1459a00640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145410b290 0x7f145419c880 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.467+0000 7f1459a00640 1 --2- 172.21.15.32:0/4270364951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1454105950 0x7f145419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.467+0000 7f1459a00640 1 -- 172.21.15.32:0/4270364951 >> 172.21.15.32:0/4270364951 conn(0x7f1454101420 msgr2=0x7f1454103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.467+0000 7f1459a00640 1 -- 172.21.15.32:0/4270364951 shutdown_connections 2024-08-31T07:11:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:22.467+0000 7f1459a00640 1 -- 172.21.15.32:0/4270364951 wait complete. 2024-08-31T07:11:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:22 smithi084 ceph-mon[22387]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:22 smithi073 ceph-mon[21995]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:23.364 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:23 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4270364951' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:11:23.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:11:23.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:23 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4270364951' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:11:23.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:23 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4270364951' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:11:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:24.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:24 smithi032 ceph-mon[17252]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.440+0000 7f86f0c00640 1 -- 172.21.15.32:0/1411966471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec109070 msgr2=0x7f86ec109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.440+0000 7f86f0c00640 1 --2- 172.21.15.32:0/1411966471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec109070 0x7f86ec109480 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009900 tx=0x7f86d802a410 comp rx=0 tx=0).stop 2024-08-31T07:11:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.440+0000 7f86f0c00640 1 -- 172.21.15.32:0/1411966471 shutdown_connections 2024-08-31T07:11:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.440+0000 7f86f0c00640 1 --2- 172.21.15.32:0/1411966471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86ec1099c0 0x7f86ec1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.440+0000 7f86f0c00640 1 --2- 172.21.15.32:0/1411966471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec109070 0x7f86ec109480 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.440+0000 7f86f0c00640 1 --2- 172.21.15.32:0/1411966471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86ec103730 0x7f86ec103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.440+0000 7f86f0c00640 1 -- 172.21.15.32:0/1411966471 >> 172.21.15.32:0/1411966471 conn(0x7f86ec0ff180 msgr2=0x7f86ec1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.440+0000 7f86f0c00640 1 -- 172.21.15.32:0/1411966471 shutdown_connections 2024-08-31T07:11:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.440+0000 7f86f0c00640 1 -- 172.21.15.32:0/1411966471 wait complete. 2024-08-31T07:11:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.442+0000 7f86f0c00640 1 Processor -- start 2024-08-31T07:11:24.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.442+0000 7f86f0c00640 1 -- start start 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86f0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86ec103730 0x7f86ec19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86f0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec109070 0x7f86ec19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86f0c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86ec1099c0 0x7f86ec1a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86f0c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec1136a0 con 0x7f86ec109070 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86f0c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec113520 con 0x7f86ec103730 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86f0c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec113820 con 0x7f86ec1099c0 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86eaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec109070 0x7f86ec19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86eaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec109070 0x7f86ec19c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54714/0 (socket says 172.21.15.32:54714) 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86eaa00640 1 -- 172.21.15.32:0/2060084900 learned_addr learned my addr 172.21.15.32:0/2060084900 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86eb400640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86ec103730 0x7f86ec19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86ebe00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86ec1099c0 0x7f86ec1a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:24.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86eaa00640 1 -- 172.21.15.32:0/2060084900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86ec1099c0 msgr2=0x7f86ec1a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:24.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86eaa00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86ec1099c0 0x7f86ec1a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86eaa00640 1 -- 172.21.15.32:0/2060084900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86ec103730 msgr2=0x7f86ec19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86eaa00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86ec103730 0x7f86ec19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.443+0000 7f86eaa00640 1 -- 172.21.15.32:0/2060084900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec1a2240 con 0x7f86ec109070 2024-08-31T07:11:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.444+0000 7f86eaa00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec109070 0x7f86ec19c870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f86d8005ab0 tx=0x7f86d8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.444+0000 7f86e3e00640 1 -- 172.21.15.32:0/2060084900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d803f070 con 0x7f86ec109070 2024-08-31T07:11:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.444+0000 7f86f0c00640 1 -- 172.21.15.32:0/2060084900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ec1a24d0 con 0x7f86ec109070 2024-08-31T07:11:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.444+0000 7f86e3e00640 1 -- 172.21.15.32:0/2060084900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86d8031b50 con 0x7f86ec109070 2024-08-31T07:11:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.444+0000 7f86e3e00640 1 -- 172.21.15.32:0/2060084900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8035030 con 0x7f86ec109070 2024-08-31T07:11:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.444+0000 7f86f0c00640 1 -- 172.21.15.32:0/2060084900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ec1a8c90 con 0x7f86ec109070 2024-08-31T07:11:24.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.445+0000 7f86f0c00640 1 -- 172.21.15.32:0/2060084900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ec103b40 con 0x7f86ec109070 2024-08-31T07:11:24.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.448+0000 7f86e3e00640 1 -- 172.21.15.32:0/2060084900 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f86d8042070 con 0x7f86ec109070 2024-08-31T07:11:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.449+0000 7f86e3e00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86d00b4c90 0x7f86d00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.449+0000 7f86e3e00640 1 -- 172.21.15.32:0/2060084900 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f86d8105cc0 con 0x7f86ec109070 2024-08-31T07:11:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.449+0000 7f86eb400640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86d00b4c90 0x7f86d00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:24.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.450+0000 7f86eb400640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86d00b4c90 0x7f86d00b71a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009bd0 tx=0x7f86dc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:24.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.451+0000 7f86e3e00640 1 -- 172.21.15.32:0/2060084900 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f86d80cc540 con 0x7f86ec109070 2024-08-31T07:11:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:24 smithi084 ceph-mon[22387]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:24 smithi073 ceph-mon[21995]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:24.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.683+0000 7f86f0c00640 1 -- 172.21.15.32:0/2060084900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86ec19d400 con 0x7f86ec109070 2024-08-31T07:11:24.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.684+0000 7f86e3e00640 1 -- 172.21.15.32:0/2060084900 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7f86d80d0d00 con 0x7f86ec109070 2024-08-31T07:11:24.685 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:24.685 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"btime":"2024-08-31T07:10:05:878594+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:10:04.962915+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:11:24.686 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 -- 172.21.15.32:0/2060084900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86d00b4c90 msgr2=0x7f86d00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86d00b4c90 0x7f86d00b71a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009bd0 tx=0x7f86dc005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 -- 172.21.15.32:0/2060084900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec109070 msgr2=0x7f86ec19c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec109070 0x7f86ec19c870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f86d8005ab0 tx=0x7f86d8003b90 comp rx=0 tx=0).stop 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 -- 172.21.15.32:0/2060084900 shutdown_connections 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86d00b4c90 0x7f86d00b71a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86ec1099c0 0x7f86ec1a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec109070 0x7f86ec19c870 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 --2- 172.21.15.32:0/2060084900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86ec103730 0x7f86ec19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 -- 172.21.15.32:0/2060084900 >> 172.21.15.32:0/2060084900 conn(0x7f86ec0ff180 msgr2=0x7f86ec108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:24.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 -- 172.21.15.32:0/2060084900 shutdown_connections 2024-08-31T07:11:24.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:24.687+0000 7f86f0c00640 1 -- 172.21.15.32:0/2060084900 wait complete. 2024-08-31T07:11:25.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:25 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2060084900' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:25.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:25 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2060084900' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:25.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-31T07:09:48.797887+0000', 'modified': '2024-08-31T07:10:04.962915+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24734, 'mds_1': 24718, 'mds_2': 24694, 'mds_3': 24692, 'mds_4': 24676}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24625': {'gid': 24625, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6833/2138862695', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6832', 'nonce': 2138862695}, {'type': 'v1', 'addr': '172.21.15.84:6833', 'nonce': 2138862695}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24632': {'gid': 24632, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6833/2003936745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 2003936745}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 2003936745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24644': {'gid': 24644, 'name': 'j', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.32:6835/1250889023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1250889023}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1250889023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6835/1927490574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 1927490574}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 1927490574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24670': {'gid': 24670, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6835/204703622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 204703622}, {'type': 'v1', 'addr': '172.21.15.84:6835', 'nonce': 204703622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24676': {'gid': 24676, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.84:6837/4208526312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 4208526312}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 4208526312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24692': {'gid': 24692, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.73:6837/2735170682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 2735170682}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2735170682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24694': {'gid': 24694, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.84:6839/2807364599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6838', 'nonce': 2807364599}, {'type': 'v1', 'addr': '172.21.15.84:6839', 'nonce': 2807364599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24718': {'gid': 24718, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.32:6841/2484505785', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6840', 'nonce': 2484505785}, {'type': 'v1', 'addr': '172.21.15.32:6841', 'nonce': 2484505785}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24734': {'gid': 24734, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.73:6839/794507464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 794507464}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 794507464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24734, 'qdb_cluster': [24676, 24692, 24718, 24694, 24734]} 2024-08-31T07:11:25.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-08-31T07:11:25.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.c status 2024-08-31T07:11:25.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:25 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2060084900' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:26.027 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:26 smithi084 ceph-mon[22387]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:26.500 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:26 smithi032 ceph-mon[17252]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:26.580 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:26 smithi073 ceph-mon[21995]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:26.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.676+0000 7fc5f7200640 1 -- 172.21.15.32:0/1439108398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f8109090 msgr2=0x7fc5f81094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:26.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.676+0000 7fc5f7200640 1 --2- 172.21.15.32:0/1439108398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f8109090 0x7fc5f81094a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009900 tx=0x7fc5e002a410 comp rx=0 tx=0).stop 2024-08-31T07:11:26.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.676+0000 7fc5f7200640 1 -- 172.21.15.32:0/1439108398 shutdown_connections 2024-08-31T07:11:26.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.676+0000 7fc5f7200640 1 --2- 172.21.15.32:0/1439108398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f81099e0 0x7fc5f8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.676+0000 7fc5f7200640 1 --2- 172.21.15.32:0/1439108398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f8109090 0x7fc5f81094a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.676+0000 7fc5f7200640 1 --2- 172.21.15.32:0/1439108398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f8103750 0x7fc5f8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.676+0000 7fc5f7200640 1 -- 172.21.15.32:0/1439108398 >> 172.21.15.32:0/1439108398 conn(0x7fc5f80ff1a0 msgr2=0x7fc5f81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:26.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.676+0000 7fc5f7200640 1 -- 172.21.15.32:0/1439108398 shutdown_connections 2024-08-31T07:11:26.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.676+0000 7fc5f7200640 1 -- 172.21.15.32:0/1439108398 wait complete. 2024-08-31T07:11:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.677+0000 7fc5f7200640 1 Processor -- start 2024-08-31T07:11:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f7200640 1 -- start start 2024-08-31T07:11:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8103750 0x7fc5f819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f7200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f8109090 0x7fc5f819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f81099e0 0x7fc5f81a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81136c0 con 0x7fc5f81099e0 2024-08-31T07:11:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8113540 con 0x7fc5f8103750 2024-08-31T07:11:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f7200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8113840 con 0x7fc5f8109090 2024-08-31T07:11:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f81099e0 0x7fc5f81a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5ed400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f8109090 0x7fc5f819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8103750 0x7fc5f819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f81099e0 0x7fc5f81a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56200/0 (socket says 172.21.15.32:56200) 2024-08-31T07:11:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.678+0000 7fc5f6800640 1 -- 172.21.15.32:0/2123351935 learned_addr learned my addr 172.21.15.32:0/2123351935 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.679+0000 7fc5f6800640 1 -- 172.21.15.32:0/2123351935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f8109090 msgr2=0x7fc5f819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.679+0000 7fc5f6800640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f8109090 0x7fc5f819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.679+0000 7fc5f6800640 1 -- 172.21.15.32:0/2123351935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8103750 msgr2=0x7fc5f819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.679+0000 7fc5f6800640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8103750 0x7fc5f819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.679+0000 7fc5f6800640 1 -- 172.21.15.32:0/2123351935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f81a2220 con 0x7fc5f81099e0 2024-08-31T07:11:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.679+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8103750 0x7fc5f819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.679+0000 7fc5f6800640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f81099e0 0x7fc5f81a1af0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000efd0 tx=0x7fc5f000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.679+0000 7fc5ef400640 1 -- 172.21.15.32:0/2123351935 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0003b70 con 0x7fc5f81099e0 2024-08-31T07:11:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.680+0000 7fc5ef400640 1 -- 172.21.15.32:0/2123351935 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5f0003d10 con 0x7fc5f81099e0 2024-08-31T07:11:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.680+0000 7fc5ef400640 1 -- 172.21.15.32:0/2123351935 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00108a0 con 0x7fc5f81099e0 2024-08-31T07:11:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.680+0000 7fc5f7200640 1 -- 172.21.15.32:0/2123351935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f81a2510 con 0x7fc5f81099e0 2024-08-31T07:11:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.680+0000 7fc5f7200640 1 -- 172.21.15.32:0/2123351935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f81a8c70 con 0x7fc5f81099e0 2024-08-31T07:11:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.683+0000 7fc603e17740 1 Processor -- start 2024-08-31T07:11:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.683+0000 7fc5ef400640 1 -- 172.21.15.32:0/2123351935 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fc5f0004020 con 0x7fc5f81099e0 2024-08-31T07:11:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc603e17740 1 -- start start 2024-08-31T07:11:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc603e17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b72408aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc603e17740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b7203a0a0 0x555b72408fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc603e17740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b72042280 0x555b72409520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc603e17740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555b7237f550 con 0x555b7203a0a0 2024-08-31T07:11:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc603e17740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555b7236eae0 con 0x555b72042280 2024-08-31T07:11:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc603e17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555b72372a90 con 0x555b72062f60 2024-08-31T07:11:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc5f5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b7203a0a0 0x555b72408fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc5ed400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b72408aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc5ed400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b72408aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37906/0 (socket says 172.21.15.32:37906) 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.684+0000 7fc5ed400640 1 -- 172.21.15.32:0/2085469203 learned_addr learned my addr 172.21.15.32:0/2085469203 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.685+0000 7fc5ed400640 1 --2- 172.21.15.32:0/2085469203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b72042280 0x555b72409520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.685+0000 7fc5f5e00640 1 -- 172.21.15.32:0/2085469203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 msgr2=0x555b72408aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.685+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/2085469203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b72408aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.685+0000 7fc5f5e00640 1 -- 172.21.15.32:0/2085469203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b72042280 msgr2=0x555b72409520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.685+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/2085469203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b72042280 0x555b72409520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.685+0000 7fc5f5e00640 1 -- 172.21.15.32:0/2085469203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b720ce540 con 0x555b7203a0a0 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.685+0000 7fc5ef400640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc5dc0b4c90 0x7fc5dc0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.685+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/2085469203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b7203a0a0 0x555b72408fe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009fd0 tx=0x7fc5e80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c116b2086dd0486 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.685+0000 7fc5ef400640 1 -- 172.21.15.32:0/2123351935 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fc5f00e5770 con 0x7fc5f81099e0 2024-08-31T07:11:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.686+0000 7fc5ed400640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc5dc0b4c90 0x7fc5dc0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.686+0000 7fc5e7400640 1 -- 172.21.15.32:0/2085469203 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8004310 con 0x555b7203a0a0 2024-08-31T07:11:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.686+0000 7fc5e7400640 1 -- 172.21.15.32:0/2085469203 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5e80044b0 con 0x555b7203a0a0 2024-08-31T07:11:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.686+0000 7fc5e7400640 1 -- 172.21.15.32:0/2085469203 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8004e50 con 0x555b7203a0a0 2024-08-31T07:11:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.687+0000 7fc603e17740 1 -- 172.21.15.32:0/2085469203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b7203a0a0 msgr2=0x555b72408fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.687+0000 7fc603e17740 1 --2- 172.21.15.32:0/2085469203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b7203a0a0 0x555b72408fe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009fd0 tx=0x7fc5e80118d0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.687+0000 7fc5ed400640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc5dc0b4c90 0x7fc5dc0b71a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0031440 tx=0x7fc5e0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.687+0000 7fc603e17740 1 -- 172.21.15.32:0/2085469203 shutdown_connections 2024-08-31T07:11:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.687+0000 7fc603e17740 1 --2- 172.21.15.32:0/2085469203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b72042280 0x555b72409520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.687+0000 7fc603e17740 1 --2- 172.21.15.32:0/2085469203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b7203a0a0 0x555b72408fe0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.687+0000 7fc603e17740 1 --2- 172.21.15.32:0/2085469203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b72408aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.687+0000 7fc603e17740 1 -- 172.21.15.32:0/2085469203 >> 172.21.15.32:0/2085469203 conn(0x555b72057350 msgr2=0x555b72406990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.687+0000 7fc603e17740 1 -- 172.21.15.32:0/2085469203 shutdown_connections 2024-08-31T07:11:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.687+0000 7fc603e17740 1 -- 172.21.15.32:0/2085469203 wait complete. 2024-08-31T07:11:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.688+0000 7fc603e17740 1 -- start start 2024-08-31T07:11:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.688+0000 7fc603e17740 1 Processor -- start 2024-08-31T07:11:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.688+0000 7fc603e17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b7203a0a0 0x555b72414250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.688+0000 7fc603e17740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b72042280 0x555b72414790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.688+0000 7fc603e17740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72062f60 0x555b724205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.688+0000 7fc603e17740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555b721d15f0 con 0x555b72062f60 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.688+0000 7fc603e17740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555b722f5bd0 con 0x555b72042280 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.688+0000 7fc603e17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555b72417280 con 0x555b7203a0a0 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc603e17740 10 client.0 _ll_register_callbacks cb 0x555b7231b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc5f6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72062f60 0x555b724205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc5f5e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b7203a0a0 0x555b72414250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc5f6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72062f60 0x555b724205d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56212/0 (socket says 172.21.15.32:56212) 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc5f6800640 1 -- 172.21.15.32:0/3279212304 learned_addr learned my addr 172.21.15.32:0/3279212304 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc5ed400640 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b72042280 0x555b72414790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc5f6800640 1 -- 172.21.15.32:0/3279212304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b7203a0a0 msgr2=0x555b72414250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc5f6800640 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b7203a0a0 0x555b72414250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc5f6800640 1 -- 172.21.15.32:0/3279212304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b72042280 msgr2=0x555b72414790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc5f6800640 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b72042280 0x555b72414790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.689+0000 7fc5f6800640 1 -- 172.21.15.32:0/3279212304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b72336650 con 0x555b72062f60 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.690+0000 7fc5f6800640 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72062f60 0x555b724205d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00abf30 tx=0x7fc5f00098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9e32dd05c31e05 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.690+0000 7fc5dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.690+0000 7fc5dbe00640 1 -- 172.21.15.32:0/3279212304 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00d2030 con 0x555b72062f60 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.690+0000 7fc603e17740 1 -- 172.21.15.32:0/3279212304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e80044b0 con 0x555b72062f60 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.690+0000 7fc5dbe00640 1 -- 172.21.15.32:0/3279212304 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fc5f0004560 con 0x555b72062f60 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.690+0000 7fc5dbe00640 1 -- 172.21.15.32:0/3279212304 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5f00abfa0 con 0x555b72062f60 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.690+0000 7fc603e17740 10 client.15069 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.690+0000 7fc603e17740 1 -- 172.21.15.32:0/3279212304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b724230e0 con 0x555b72062f60 2024-08-31T07:11:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.690+0000 7fc5dbe00640 1 -- 172.21.15.32:0/3279212304 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5e80044b0 con 0x555b72062f60 2024-08-31T07:11:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.691+0000 7fc5dbe00640 1 -- 172.21.15.32:0/3279212304 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc5f001f020 con 0x555b72062f60 2024-08-31T07:11:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.691+0000 7fc603e17740 10 client.15069 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.691+0000 7fc603e17740 10 client.15069 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-31T07:11:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.691+0000 7fc603e17740 20 client.15069 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.691+0000 7fc603e17740 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555b720218e0 0x555b72426b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.691+0000 7fc603e17740 4 client.15069 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:11:26.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.691+0000 7fc603e17740 1 -- 172.21.15.32:0/3279212304 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b723814a0 con 0x555b720218e0 2024-08-31T07:11:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.691+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555b720218e0 0x555b72426b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.692+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555b720218e0 0x555b72426b90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=57dc871fbee5cff9 server_cookie=483e069f1f5297a9 in_seq=0 out_seq=0 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.692+0000 7fc5dbe00640 10 client.15069 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.693+0000 7fc5dbe00640 1 -- 172.21.15.32:0/3279212304 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b723814a0 con 0x555b720218e0 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.693+0000 7fc5dbe00640 10 client.15069 handle_command_reply: tid=0 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.693+0000 7fc603e17740 1 client.15069 shutdown 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.693+0000 7fc603e17740 1 -- 172.21.15.32:0/3279212304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72062f60 msgr2=0x555b724205d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.693+0000 7fc603e17740 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72062f60 0x555b724205d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00abf30 tx=0x7fc5f00098a0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.694+0000 7fc603e17740 1 -- 172.21.15.32:0/3279212304 shutdown_connections 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.694+0000 7fc603e17740 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555b720218e0 0x555b72426b90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.694+0000 7fc603e17740 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72062f60 0x555b724205d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.694+0000 7fc603e17740 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b72042280 0x555b72414790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.694+0000 7fc603e17740 1 --2- 172.21.15.32:0/3279212304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b7203a0a0 0x555b72414250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.694+0000 7fc5dbe00640 0 client.15069 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:11:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.694+0000 7fc603e17740 1 -- 172.21.15.32:0/3279212304 >> 172.21.15.32:0/3279212304 conn(0x555b72057350 msgr2=0x555b7240bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.694+0000 7fc603e17740 1 -- 172.21.15.32:0/3279212304 shutdown_connections 2024-08-31T07:11:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.694+0000 7fc603e17740 1 -- 172.21.15.32:0/3279212304 wait complete. 2024-08-31T07:11:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.694+0000 7fc603e17740 20 client.15069 trim_cache size 0 max 16384 2024-08-31T07:11:26.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.713+0000 7fc603e17740 1 Processor -- start 2024-08-31T07:11:26.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.713+0000 7fc603e17740 1 -- start start 2024-08-31T07:11:26.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.713+0000 7fc603e17740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b72414000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.713+0000 7fc603e17740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 0x555b72414540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.713+0000 7fc603e17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b7240bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.713+0000 7fc603e17740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555b72424ed0 con 0x555b72042280 2024-08-31T07:11:26.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.713+0000 7fc603e17740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555b724255a0 con 0x555b7203a0a0 2024-08-31T07:11:26.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.713+0000 7fc603e17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555b72425c70 con 0x555b72062f60 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5ed400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 0x555b72414540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5ed400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 0x555b72414540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56216/0 (socket says 172.21.15.32:56216) 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5f5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b72414000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5ed400640 1 -- 172.21.15.32:0/3194201452 learned_addr learned my addr 172.21.15.32:0/3194201452 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5f6800640 1 --2- 172.21.15.32:0/3194201452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b7240bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5f5e00640 1 -- 172.21.15.32:0/3194201452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 msgr2=0x555b7240bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/3194201452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b7240bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5f5e00640 1 -- 172.21.15.32:0/3194201452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 msgr2=0x555b72414540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/3194201452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 0x555b72414540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5f5e00640 1 -- 172.21.15.32:0/3194201452 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b7240ad90 con 0x555b7203a0a0 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5ed400640 1 --2- 172.21.15.32:0/3194201452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 0x555b72414540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.714+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/3194201452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b72414000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc5e801b040 tx=0x7fc5e8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6565789fbc102cdb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.715+0000 7fc5e6a00640 1 -- 172.21.15.32:0/3194201452 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8032030 con 0x555b7203a0a0 2024-08-31T07:11:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.715+0000 7fc5e6a00640 1 -- 172.21.15.32:0/3194201452 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5e80035f0 con 0x555b7203a0a0 2024-08-31T07:11:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.715+0000 7fc5e6a00640 1 -- 172.21.15.32:0/3194201452 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800f720 con 0x555b7203a0a0 2024-08-31T07:11:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.715+0000 7fc603e17740 1 -- 172.21.15.32:0/3194201452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 msgr2=0x555b72414000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.715+0000 7fc603e17740 1 --2- 172.21.15.32:0/3194201452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b72414000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc5e801b040 tx=0x7fc5e8004ab0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.716+0000 7fc603e17740 1 -- 172.21.15.32:0/3194201452 shutdown_connections 2024-08-31T07:11:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.716+0000 7fc603e17740 1 --2- 172.21.15.32:0/3194201452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b7240bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.716+0000 7fc603e17740 1 --2- 172.21.15.32:0/3194201452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 0x555b72414540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.716+0000 7fc603e17740 1 --2- 172.21.15.32:0/3194201452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b72414000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.716+0000 7fc603e17740 1 -- 172.21.15.32:0/3194201452 >> 172.21.15.32:0/3194201452 conn(0x555b72057350 msgr2=0x555b72424550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.716+0000 7fc603e17740 1 -- 172.21.15.32:0/3194201452 shutdown_connections 2024-08-31T07:11:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.716+0000 7fc603e17740 1 -- 172.21.15.32:0/3194201452 wait complete. 2024-08-31T07:11:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.716+0000 7fc603e17740 1 -- start start 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc603e17740 1 Processor -- start 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc603e17740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b7242adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc603e17740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 0x555b7242b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc603e17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b7242b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc603e17740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555b72426a10 con 0x555b72042280 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc603e17740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555b72426340 con 0x555b7203a0a0 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc603e17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555b724270e0 con 0x555b72062f60 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc603e17740 10 client.0 _ll_register_callbacks cb 0x7fc5f00ab710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc5f5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b7242adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc5ed400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 0x555b7242b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc5f5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b7242adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58946/0 (socket says 172.21.15.32:58946) 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc5f5e00640 1 -- 172.21.15.32:0/10878527 learned_addr learned my addr 172.21.15.32:0/10878527 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:26.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.717+0000 7fc5f6800640 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b7242b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.718+0000 7fc5f5e00640 1 -- 172.21.15.32:0/10878527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 msgr2=0x555b7242b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.718+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b7242b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.718+0000 7fc5f5e00640 1 -- 172.21.15.32:0/10878527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 msgr2=0x555b7242b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.718+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 0x555b7242b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.718+0000 7fc5f5e00640 1 -- 172.21.15.32:0/10878527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b72491060 con 0x555b7203a0a0 2024-08-31T07:11:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.718+0000 7fc5f5e00640 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b7242adc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009d60 tx=0x7fc5e8004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aaff3d240a4619 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.718+0000 7fc5dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:11:26.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.718+0000 7fc5dbe00640 1 -- 172.21.15.32:0/10878527 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8003bb0 con 0x555b7203a0a0 2024-08-31T07:11:26.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.719+0000 7fc5dbe00640 1 -- 172.21.15.32:0/10878527 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fc5e800f4e0 con 0x555b7203a0a0 2024-08-31T07:11:26.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.719+0000 7fc603e17740 1 -- 172.21.15.32:0/10878527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e80035f0 con 0x555b7203a0a0 2024-08-31T07:11:26.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.719+0000 7fc5dbe00640 1 -- 172.21.15.32:0/10878527 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5e8003d50 con 0x555b7203a0a0 2024-08-31T07:11:26.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.719+0000 7fc603e17740 10 client.25091 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:26.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.719+0000 7fc603e17740 1 -- 172.21.15.32:0/10878527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b72492270 con 0x555b7203a0a0 2024-08-31T07:11:26.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.719+0000 7fc5dbe00640 1 -- 172.21.15.32:0/10878527 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5e80035f0 con 0x555b7203a0a0 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.719+0000 7fc5dbe00640 1 -- 172.21.15.32:0/10878527 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc5e802e680 con 0x555b7203a0a0 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.719+0000 7fc603e17740 10 client.25091 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.720+0000 7fc603e17740 10 client.25091 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.720+0000 7fc603e17740 20 client.25091 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.720+0000 7fc603e17740 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555b71ff6c90 0x555b724953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.720+0000 7fc603e17740 4 client.25091 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.720+0000 7fc603e17740 1 -- 172.21.15.32:0/10878527 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "status"}) -- 0x555b7237cf40 con 0x555b71ff6c90 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.720+0000 7fc5ed400640 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555b71ff6c90 0x555b724953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.720+0000 7fc5ed400640 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555b71ff6c90 0x555b724953e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a0e168d059cd6e56 server_cookie=758a7794d53990e2 in_seq=0 out_seq=0 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.720+0000 7fc5dbe00640 10 client.25091 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc5dbe00640 1 -- 172.21.15.32:0/10878527 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x555b7237cf40 con 0x555b71ff6c90 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc5dbe00640 10 client.25091 handle_command_reply: tid=0 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 client.25091 shutdown 2024-08-31T07:11:26.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 -- 172.21.15.32:0/10878527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 msgr2=0x555b7242adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b7242adc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009d60 tx=0x7fc5e8004760 comp rx=0 tx=0).stop 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 -- 172.21.15.32:0/10878527 shutdown_connections 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555b71ff6c90 0x555b724953e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b72062f60 0x555b7242b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b72042280 0x555b7242b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 --2- 172.21.15.32:0/10878527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b7203a0a0 0x555b7242adc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc5dbe00640 0 client.25091 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 -- 172.21.15.32:0/10878527 >> 172.21.15.32:0/10878527 conn(0x555b72057350 msgr2=0x555b7240bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 -- 172.21.15.32:0/10878527 shutdown_connections 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 1 -- 172.21.15.32:0/10878527 wait complete. 2024-08-31T07:11:26.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.721+0000 7fc603e17740 20 client.25091 trim_cache size 0 max 16384 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 4, 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24676, 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 83.851408487, 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 126.785606699 2024-08-31T07:11:26.727 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:11:26.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 -- 172.21.15.32:0/2123351935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc5dc0b4c90 msgr2=0x7fc5dc0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:26.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc5dc0b4c90 0x7fc5dc0b71a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0031440 tx=0x7fc5e0005a20 comp rx=0 tx=0).stop 2024-08-31T07:11:26.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 -- 172.21.15.32:0/2123351935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f81099e0 msgr2=0x7fc5f81a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:26.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f81099e0 0x7fc5f81a1af0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000efd0 tx=0x7fc5f000c6a0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 -- 172.21.15.32:0/2123351935 shutdown_connections 2024-08-31T07:11:26.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc5dc0b4c90 0x7fc5dc0b71a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5f81099e0 0x7fc5f81a1af0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5f8109090 0x7fc5f819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 --2- 172.21.15.32:0/2123351935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8103750 0x7fc5f819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:26.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 -- 172.21.15.32:0/2123351935 >> 172.21.15.32:0/2123351935 conn(0x7fc5f80ff1a0 msgr2=0x7fc5f810e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:26.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 -- 172.21.15.32:0/2123351935 shutdown_connections 2024-08-31T07:11:26.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:26.722+0000 7fc5f7200640 1 -- 172.21.15.32:0/2123351935 wait complete. 2024-08-31T07:11:27.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.h status 2024-08-31T07:11:27.990 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:28.477 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:28 smithi032 ceph-mon[17252]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:28.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:28 smithi084 ceph-mon[22387]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:28.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.654+0000 7fcb06c00640 1 -- 172.21.15.32:0/4092678651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0010b2a0 msgr2=0x7fcb0010b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:28.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.654+0000 7fcb06c00640 1 --2- 172.21.15.32:0/4092678651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0010b2a0 0x7fcb0010b6b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009900 tx=0x7fcaec02a410 comp rx=0 tx=0).stop 2024-08-31T07:11:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.654+0000 7fcb06c00640 1 -- 172.21.15.32:0/4092678651 shutdown_connections 2024-08-31T07:11:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.654+0000 7fcb06c00640 1 --2- 172.21.15.32:0/4092678651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb0010bbf0 0x7fcb00112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.654+0000 7fcb06c00640 1 --2- 172.21.15.32:0/4092678651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0010b2a0 0x7fcb0010b6b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.654+0000 7fcb06c00640 1 --2- 172.21.15.32:0/4092678651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb00105960 0x7fcb00105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.654+0000 7fcb06c00640 1 -- 172.21.15.32:0/4092678651 >> 172.21.15.32:0/4092678651 conn(0x7fcb00101410 msgr2=0x7fcb00103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.654+0000 7fcb06c00640 1 -- 172.21.15.32:0/4092678651 shutdown_connections 2024-08-31T07:11:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.654+0000 7fcb06c00640 1 -- 172.21.15.32:0/4092678651 wait complete. 2024-08-31T07:11:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.655+0000 7fcb06c00640 1 Processor -- start 2024-08-31T07:11:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.656+0000 7fcb06c00640 1 -- start start 2024-08-31T07:11:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.656+0000 7fcb06c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb00105960 0x7fcb0019c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.656+0000 7fcb06c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0010b2a0 0x7fcb0019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.656+0000 7fcb06c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb0010bbf0 0x7fcb001a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.656+0000 7fcb06c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001158b0 con 0x7fcb0010b2a0 2024-08-31T07:11:28.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.656+0000 7fcb06c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00115730 con 0x7fcb00105960 2024-08-31T07:11:28.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.656+0000 7fcb06c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00115a30 con 0x7fcb0010bbf0 2024-08-31T07:11:28.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.656+0000 7fcb04e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0010b2a0 0x7fcb0019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.656+0000 7fcb04e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0010b2a0 0x7fcb0019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56234/0 (socket says 172.21.15.32:56234) 2024-08-31T07:11:28.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.656+0000 7fcb04e00640 1 -- 172.21.15.32:0/849349369 learned_addr learned my addr 172.21.15.32:0/849349369 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:28.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.657+0000 7fcb06200640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb0010bbf0 0x7fcb001a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.657+0000 7fcb05800640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb00105960 0x7fcb0019c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.657+0000 7fcb04e00640 1 -- 172.21.15.32:0/849349369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb0010bbf0 msgr2=0x7fcb001a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.657+0000 7fcb04e00640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb0010bbf0 0x7fcb001a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.657+0000 7fcb04e00640 1 -- 172.21.15.32:0/849349369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb00105960 msgr2=0x7fcb0019c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.657+0000 7fcb04e00640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb00105960 0x7fcb0019c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.657+0000 7fcb04e00640 1 -- 172.21.15.32:0/849349369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb001a21f0 con 0x7fcb0010b2a0 2024-08-31T07:11:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.657+0000 7fcb06200640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb0010bbf0 0x7fcb001a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.657+0000 7fcb04e00640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0010b2a0 0x7fcb0019c820 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcaec006010 tx=0x7fcaec004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.658+0000 7fcafe000640 1 -- 172.21.15.32:0/849349369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec03f030 con 0x7fcb0010b2a0 2024-08-31T07:11:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.658+0000 7fcb06c00640 1 -- 172.21.15.32:0/849349369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb001a2480 con 0x7fcb0010b2a0 2024-08-31T07:11:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.658+0000 7fcafe000640 1 -- 172.21.15.32:0/849349369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaec003aa0 con 0x7fcb0010b2a0 2024-08-31T07:11:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.658+0000 7fcafe000640 1 -- 172.21.15.32:0/849349369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec035070 con 0x7fcb0010b2a0 2024-08-31T07:11:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.658+0000 7fcb06c00640 1 -- 172.21.15.32:0/849349369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb001a8c40 con 0x7fcb0010b2a0 2024-08-31T07:11:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.661+0000 7fcb0f013740 1 Processor -- start 2024-08-31T07:11:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.661+0000 7fcb0f013740 1 -- start start 2024-08-31T07:11:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.661+0000 7fcafe000640 1 -- 172.21.15.32:0/849349369 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fcaec042030 con 0x7fcb0010b2a0 2024-08-31T07:11:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x5588885f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x5588885f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x5588885fa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb0f013740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558888570550 con 0x558888233280 2024-08-31T07:11:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb0f013740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55888855fae0 con 0x55888822b0a0 2024-08-31T07:11:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb0f013740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558888563a90 con 0x558888253f60 2024-08-31T07:11:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb04e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x5588885fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb05800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x5588885f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb06200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x5588885f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb06200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x5588885f9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37970/0 (socket says 172.21.15.32:37970) 2024-08-31T07:11:28.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb04e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x5588885fa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56246/0 (socket says 172.21.15.32:56246) 2024-08-31T07:11:28.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb05800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x5588885f9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58974/0 (socket says 172.21.15.32:58974) 2024-08-31T07:11:28.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb06200640 1 -- 172.21.15.32:0/55730743 learned_addr learned my addr 172.21.15.32:0/55730743 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:28.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.662+0000 7fcb06200640 1 -- 172.21.15.32:0/55730743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 msgr2=0x5588885f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:28.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.663+0000 7fcb06200640 1 --2- 172.21.15.32:0/55730743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x5588885f9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.663+0000 7fcb06200640 1 -- 172.21.15.32:0/55730743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 msgr2=0x5588885fa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:28.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.663+0000 7fcb06200640 1 --2- 172.21.15.32:0/55730743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x5588885fa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.663+0000 7fcb06200640 1 -- 172.21.15.32:0/55730743 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588882bf540 con 0x558888253f60 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.663+0000 7fcb06200640 1 --2- 172.21.15.32:0/55730743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x5588885f9aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80117e0 tx=0x7fcaf8011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2cd10129eb4e2f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.663+0000 7fcaf2a00640 1 -- 172.21.15.32:0/55730743 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8011310 con 0x558888253f60 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.663+0000 7fcaf2a00640 1 -- 172.21.15.32:0/55730743 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf8003e20 con 0x558888253f60 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.663+0000 7fcaf2a00640 1 -- 172.21.15.32:0/55730743 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80044f0 con 0x558888253f60 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.663+0000 7fcafe000640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcae80b4c10 0x7fcae80b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcafe000640 1 -- 172.21.15.32:0/849349369 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fcaec105440 con 0x7fcb0010b2a0 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcb05800640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcae80b4c10 0x7fcae80b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcb0f013740 1 -- 172.21.15.32:0/55730743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 msgr2=0x5588885f9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcb0f013740 1 --2- 172.21.15.32:0/55730743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x5588885f9aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80117e0 tx=0x7fcaf8011ca0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcb05800640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcae80b4c10 0x7fcae80b7120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5588885f8720 tx=0x7fcaf400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcb0f013740 1 -- 172.21.15.32:0/55730743 shutdown_connections 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcb0f013740 1 --2- 172.21.15.32:0/55730743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x5588885fa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcb0f013740 1 --2- 172.21.15.32:0/55730743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x5588885f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcb0f013740 1 --2- 172.21.15.32:0/55730743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x5588885f9aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcb0f013740 1 -- 172.21.15.32:0/55730743 >> 172.21.15.32:0/55730743 conn(0x558888248350 msgr2=0x5588885f7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:28.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.664+0000 7fcb0f013740 1 -- 172.21.15.32:0/55730743 shutdown_connections 2024-08-31T07:11:28.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.665+0000 7fcb0f013740 1 -- 172.21.15.32:0/55730743 wait complete. 2024-08-31T07:11:28.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.665+0000 7fcb0f013740 1 -- start start 2024-08-31T07:11:28.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.665+0000 7fcb0f013740 1 Processor -- start 2024-08-31T07:11:28.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55888822b0a0 0x558888605290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888605810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558888253f60 0x558888611570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb0f013740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5588883c25f0 con 0x558888233280 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb0f013740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5588884e6bd0 con 0x558888253f60 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb0f013740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558888608280 con 0x55888822b0a0 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb0f013740 10 client.0 _ll_register_callbacks cb 0x55888850c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb04e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888605810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb05800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55888822b0a0 0x558888605290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb06200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558888253f60 0x558888611570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb05800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55888822b0a0 0x558888605290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37986/0 (socket says 172.21.15.32:37986) 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb05800640 1 -- 172.21.15.32:0/2278616529 learned_addr learned my addr 172.21.15.32:0/2278616529 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.666+0000 7fcb04e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888605810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56260/0 (socket says 172.21.15.32:56260) 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcb06200640 1 -- 172.21.15.32:0/2278616529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55888822b0a0 msgr2=0x558888605290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:28.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcb06200640 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55888822b0a0 0x558888605290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcb06200640 1 -- 172.21.15.32:0/2278616529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 msgr2=0x558888605810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcb06200640 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888605810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcb06200640 1 -- 172.21.15.32:0/2278616529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558888527650 con 0x558888253f60 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcb05800640 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55888822b0a0 0x558888605290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcb06200640 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558888253f60 0x558888611570 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8019000 tx=0x7fcaf8011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=321c446df330c87f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcae7400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcae7400640 1 -- 172.21.15.32:0/2278616529 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf800fdb0 con 0x558888253f60 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcb0f013740 1 -- 172.21.15.32:0/2278616529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf8003e20 con 0x558888253f60 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.667+0000 7fcae7400640 1 -- 172.21.15.32:0/2278616529 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fcaf8004a40 con 0x558888253f60 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcae7400640 1 -- 172.21.15.32:0/2278616529 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcaf8004c40 con 0x558888253f60 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcb0f013740 10 client.25103 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcb0f013740 1 -- 172.21.15.32:0/2278616529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588886140e0 con 0x558888253f60 2024-08-31T07:11:28.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcae7400640 1 -- 172.21.15.32:0/2278616529 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf8003e20 con 0x558888253f60 2024-08-31T07:11:28.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcae7400640 1 -- 172.21.15.32:0/2278616529 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcaf8018020 con 0x558888253f60 2024-08-31T07:11:28.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcb0f013740 10 client.25103 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:28.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcb0f013740 10 client.25103 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-31T07:11:28.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcb0f013740 20 client.25103 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:28.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcb0f013740 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5588882128e0 0x5588886178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcb0f013740 4 client.25103 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:11:28.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.668+0000 7fcb0f013740 1 -- 172.21.15.32:0/2278616529 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588885724a0 con 0x5588882128e0 2024-08-31T07:11:28.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.669+0000 7fcb05800640 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5588882128e0 0x5588886178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.669+0000 7fcb05800640 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5588882128e0 0x5588886178e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46d0df2cda959799 server_cookie=2b5eacdf7bd506e4 in_seq=0 out_seq=0 2024-08-31T07:11:28.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.669+0000 7fcae7400640 10 client.25103 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:11:28.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.670+0000 7fcae7400640 1 -- 172.21.15.32:0/2278616529 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588885724a0 con 0x5588882128e0 2024-08-31T07:11:28.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.670+0000 7fcae7400640 10 client.25103 handle_command_reply: tid=0 2024-08-31T07:11:28.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.670+0000 7fcb0f013740 1 client.25103 shutdown 2024-08-31T07:11:28.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 1 -- 172.21.15.32:0/2278616529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558888253f60 msgr2=0x558888611570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558888253f60 0x558888611570 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8019000 tx=0x7fcaf8011440 comp rx=0 tx=0).stop 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 1 -- 172.21.15.32:0/2278616529 shutdown_connections 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5588882128e0 0x5588886178e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558888253f60 0x558888611570 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888605810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 1 --2- 172.21.15.32:0/2278616529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55888822b0a0 0x558888605290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 1 -- 172.21.15.32:0/2278616529 >> 172.21.15.32:0/2278616529 conn(0x558888248350 msgr2=0x5588885fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcae7400640 0 client.25103 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 1 -- 172.21.15.32:0/2278616529 shutdown_connections 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 1 -- 172.21.15.32:0/2278616529 wait complete. 2024-08-31T07:11:28.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.671+0000 7fcb0f013740 20 client.25103 trim_cache size 0 max 16384 2024-08-31T07:11:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:28 smithi073 ceph-mon[21995]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:28.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.690+0000 7fcb0f013740 1 Processor -- start 2024-08-31T07:11:28.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.690+0000 7fcb0f013740 1 -- start start 2024-08-31T07:11:28.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.690+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x558888604780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.690+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888233280 0x558888600440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888253f60 0x5588886114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb0f013740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558888615c20 con 0x558888253f60 2024-08-31T07:11:28.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb0f013740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5588886162f0 con 0x55888822b0a0 2024-08-31T07:11:28.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb0f013740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5588886169c0 con 0x558888233280 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb05800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x558888604780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb05800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x558888604780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58988/0 (socket says 172.21.15.32:58988) 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb06200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888253f60 0x5588886114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb05800640 1 -- 172.21.15.32:0/520366919 learned_addr learned my addr 172.21.15.32:0/520366919 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb04e00640 1 --2- 172.21.15.32:0/520366919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888233280 0x558888600440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb05800640 1 -- 172.21.15.32:0/520366919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888233280 msgr2=0x558888600440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb05800640 1 --2- 172.21.15.32:0/520366919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888233280 0x558888600440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb05800640 1 -- 172.21.15.32:0/520366919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888253f60 msgr2=0x5588886114e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb05800640 1 --2- 172.21.15.32:0/520366919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888253f60 0x5588886114e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.691+0000 7fcb05800640 1 -- 172.21.15.32:0/520366919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588885fbd90 con 0x55888822b0a0 2024-08-31T07:11:28.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.692+0000 7fcb05800640 1 --2- 172.21.15.32:0/520366919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x558888604780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x558888618200 tx=0x7fcaf4004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ec19fca58be276b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:28.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.692+0000 7fcaf2000640 1 -- 172.21.15.32:0/520366919 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf401e970 con 0x55888822b0a0 2024-08-31T07:11:28.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.692+0000 7fcaf2000640 1 -- 172.21.15.32:0/520366919 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf402a470 con 0x55888822b0a0 2024-08-31T07:11:28.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.692+0000 7fcaf2000640 1 -- 172.21.15.32:0/520366919 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf401d670 con 0x55888822b0a0 2024-08-31T07:11:28.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 -- 172.21.15.32:0/520366919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 msgr2=0x558888604780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:28.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 --2- 172.21.15.32:0/520366919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x558888604780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x558888618200 tx=0x7fcaf4004610 comp rx=0 tx=0).stop 2024-08-31T07:11:28.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 -- 172.21.15.32:0/520366919 shutdown_connections 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 --2- 172.21.15.32:0/520366919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888253f60 0x5588886114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 --2- 172.21.15.32:0/520366919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888233280 0x558888600440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 --2- 172.21.15.32:0/520366919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x558888604780 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 -- 172.21.15.32:0/520366919 >> 172.21.15.32:0/520366919 conn(0x558888248350 msgr2=0x5588885f2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 -- 172.21.15.32:0/520366919 shutdown_connections 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 -- 172.21.15.32:0/520366919 wait complete. 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 -- start start 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.693+0000 7fcb0f013740 1 Processor -- start 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x558888619180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888619740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb0f013740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x55888861e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb0f013740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558888617760 con 0x558888233280 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb0f013740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558888617090 con 0x55888822b0a0 2024-08-31T07:11:28.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb0f013740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558888617e30 con 0x558888253f60 2024-08-31T07:11:28.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb0f013740 10 client.0 _ll_register_callbacks cb 0x7fcaf80116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:28.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb04e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888619740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb04e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888619740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56280/0 (socket says 172.21.15.32:56280) 2024-08-31T07:11:28.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb04e00640 1 -- 172.21.15.32:0/3725940913 learned_addr learned my addr 172.21.15.32:0/3725940913 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:28.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb06200640 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x55888861e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.694+0000 7fcb05800640 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x558888619180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcb04e00640 1 -- 172.21.15.32:0/3725940913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 msgr2=0x55888861e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:28.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcb04e00640 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x55888861e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcb04e00640 1 -- 172.21.15.32:0/3725940913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 msgr2=0x558888619180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:28.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcb04e00640 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x558888619180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcb04e00640 1 -- 172.21.15.32:0/3725940913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55888861eda0 con 0x558888233280 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcb06200640 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x55888861e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcb04e00640 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888619740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0cba20 tx=0x7fcaec03b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d11ca9461dbef9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcae7400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcae7400640 1 -- 172.21.15.32:0/3725940913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec049040 con 0x558888233280 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcae7400640 1 -- 172.21.15.32:0/3725940913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fcaec0d01b0 con 0x558888233280 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.695+0000 7fcb0f013740 1 -- 172.21.15.32:0/3725940913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf402a470 con 0x558888233280 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcae7400640 1 -- 172.21.15.32:0/3725940913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcaec0cbaf0 con 0x558888233280 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcb0f013740 10 client.15093 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcb0f013740 1 -- 172.21.15.32:0/3725940913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558888620160 con 0x558888233280 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcae7400640 1 -- 172.21.15.32:0/3725940913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf402a470 con 0x558888233280 2024-08-31T07:11:28.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcae7400640 1 -- 172.21.15.32:0/3725940913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcaec0c7fb0 con 0x558888233280 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcb0f013740 10 client.15093 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcb0f013740 10 client.15093 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcb0f013740 20 client.15093 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcb0f013740 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5588881e7c90 0x558888685d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcb0f013740 4 client.15093 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.696+0000 7fcb0f013740 1 -- 172.21.15.32:0/3725940913 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "status"}) -- 0x55888856df40 con 0x5588881e7c90 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.697+0000 7fcb05800640 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5588881e7c90 0x558888685d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.697+0000 7fcb05800640 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5588881e7c90 0x558888685d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d204151ccc026d2 server_cookie=44a81ddedfe2cad6 in_seq=0 out_seq=0 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.697+0000 7fcae7400640 10 client.15093 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.697+0000 7fcae7400640 1 -- 172.21.15.32:0/3725940913 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55888856df40 con 0x5588881e7c90 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.697+0000 7fcae7400640 10 client.15093 handle_command_reply: tid=0 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.697+0000 7fcb0f013740 1 client.15093 shutdown 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 1 -- 172.21.15.32:0/3725940913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 msgr2=0x558888619740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888619740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0cba20 tx=0x7fcaec03b5d0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 1 -- 172.21.15.32:0/3725940913 shutdown_connections 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5588881e7c90 0x558888685d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558888253f60 0x55888861e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558888233280 0x558888619740 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 1 --2- 172.21.15.32:0/3725940913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55888822b0a0 0x558888619180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 1 -- 172.21.15.32:0/3725940913 >> 172.21.15.32:0/3725940913 conn(0x558888248350 msgr2=0x5588885f2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcae7400640 0 client.15093 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 1 -- 172.21.15.32:0/3725940913 shutdown_connections 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 1 -- 172.21.15.32:0/3725940913 wait complete. 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.698+0000 7fcb0f013740 20 client.15093 trim_cache size 0 max 16384 2024-08-31T07:11:28.703 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 3, 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24692, 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 86.843358062999997, 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 126.134372167 2024-08-31T07:11:28.704 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:11:28.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 -- 172.21.15.32:0/849349369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcae80b4c10 msgr2=0x7fcae80b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:28.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcae80b4c10 0x7fcae80b7120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5588885f8720 tx=0x7fcaf400b390 comp rx=0 tx=0).stop 2024-08-31T07:11:28.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 -- 172.21.15.32:0/849349369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0010b2a0 msgr2=0x7fcb0019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:28.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0010b2a0 0x7fcb0019c820 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcaec006010 tx=0x7fcaec004060 comp rx=0 tx=0).stop 2024-08-31T07:11:28.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 -- 172.21.15.32:0/849349369 shutdown_connections 2024-08-31T07:11:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcae80b4c10 0x7fcae80b7120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb0010bbf0 0x7fcb001a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb0010b2a0 0x7fcb0019c820 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 --2- 172.21.15.32:0/849349369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb00105960 0x7fcb0019c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 -- 172.21.15.32:0/849349369 >> 172.21.15.32:0/849349369 conn(0x7fcb00101410 msgr2=0x7fcb0010a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 -- 172.21.15.32:0/849349369 shutdown_connections 2024-08-31T07:11:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:28.699+0000 7fcb06c00640 1 -- 172.21.15.32:0/849349369 wait complete. 2024-08-31T07:11:29.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.i status 2024-08-31T07:11:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:30 smithi084 ceph-mon[22387]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:30.514 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:30 smithi032 ceph-mon[17252]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:30 smithi073 ceph-mon[21995]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.691+0000 7f3488a00640 1 -- 172.21.15.32:0/1774547979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348410b290 msgr2=0x7f348410b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.691+0000 7f3488a00640 1 --2- 172.21.15.32:0/1774547979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348410b290 0x7f348410b6a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3470009900 tx=0x7f347002a410 comp rx=0 tx=0).stop 2024-08-31T07:11:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.691+0000 7f3488a00640 1 -- 172.21.15.32:0/1774547979 shutdown_connections 2024-08-31T07:11:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.691+0000 7f3488a00640 1 --2- 172.21.15.32:0/1774547979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f348410bbe0 0x7f3484112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.691+0000 7f3488a00640 1 --2- 172.21.15.32:0/1774547979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348410b290 0x7f348410b6a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.691+0000 7f3488a00640 1 --2- 172.21.15.32:0/1774547979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3484105950 0x7f3484105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.691+0000 7f3488a00640 1 -- 172.21.15.32:0/1774547979 >> 172.21.15.32:0/1774547979 conn(0x7f3484101420 msgr2=0x7f3484103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.691+0000 7f3488a00640 1 -- 172.21.15.32:0/1774547979 shutdown_connections 2024-08-31T07:11:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.691+0000 7f3488a00640 1 -- 172.21.15.32:0/1774547979 wait complete. 2024-08-31T07:11:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.692+0000 7f3488a00640 1 Processor -- start 2024-08-31T07:11:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.692+0000 7f3488a00640 1 -- start start 2024-08-31T07:11:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3488a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484105950 0x7f348419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3488a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f348410b290 0x7f348419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3488a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348410bbe0 0x7f34841a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3488a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f34841158a0 con 0x7f348410bbe0 2024-08-31T07:11:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3488a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3484115720 con 0x7f3484105950 2024-08-31T07:11:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3488a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3484115a20 con 0x7f348410b290 2024-08-31T07:11:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3483e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348410bbe0 0x7f34841a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3483400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484105950 0x7f348419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3482a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f348410b290 0x7f348419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3483400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484105950 0x7f348419c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59014/0 (socket says 172.21.15.32:59014) 2024-08-31T07:11:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3482a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f348410b290 0x7f348419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38036/0 (socket says 172.21.15.32:38036) 2024-08-31T07:11:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.693+0000 7f3483400640 1 -- 172.21.15.32:0/2338094792 learned_addr learned my addr 172.21.15.32:0/2338094792 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f3482a00640 1 -- 172.21.15.32:0/2338094792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484105950 msgr2=0x7f348419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f3482a00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484105950 0x7f348419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f3482a00640 1 -- 172.21.15.32:0/2338094792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348410bbe0 msgr2=0x7f34841a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f3482a00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348410bbe0 0x7f34841a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f3482a00640 1 -- 172.21.15.32:0/2338094792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34841a2240 con 0x7f348410b290 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f3483400640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484105950 0x7f348419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f3483e00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348410bbe0 0x7f34841a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f3482a00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f348410b290 0x7f348419c870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f347002a930 tx=0x7f3470004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f347be00640 1 -- 172.21.15.32:0/2338094792 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347003f070 con 0x7f348410b290 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f347be00640 1 -- 172.21.15.32:0/2338094792 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3470003ca0 con 0x7f348410b290 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f3488a00640 1 -- 172.21.15.32:0/2338094792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34841a24d0 con 0x7f348410b290 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f347be00640 1 -- 172.21.15.32:0/2338094792 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470035070 con 0x7f348410b290 2024-08-31T07:11:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.694+0000 7f3488a00640 1 -- 172.21.15.32:0/2338094792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34841a8c90 con 0x7f348410b290 2024-08-31T07:11:30.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.697+0000 7f3490dfc740 1 Processor -- start 2024-08-31T07:11:30.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3490dfc740 1 -- start start 2024-08-31T07:11:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c49f60 0x556682fefaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556682feffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 0x556682ff0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3490dfc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556682f66550 con 0x556682c29280 2024-08-31T07:11:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3490dfc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556682f55ae0 con 0x556682c49f60 2024-08-31T07:11:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3490dfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556682f59a90 con 0x556682c210a0 2024-08-31T07:11:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f347be00640 1 -- 172.21.15.32:0/2338094792 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f347003b460 con 0x7f348410b290 2024-08-31T07:11:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3483e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c49f60 0x556682fefaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3483e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c49f60 0x556682fefaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59030/0 (socket says 172.21.15.32:59030) 2024-08-31T07:11:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3483400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556682feffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3483e00640 1 -- 172.21.15.32:0/2448628476 learned_addr learned my addr 172.21.15.32:0/2448628476 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.698+0000 7f3482a00640 1 --2- 172.21.15.32:0/2448628476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 0x556682ff0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.699+0000 7f3483400640 1 -- 172.21.15.32:0/2448628476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c49f60 msgr2=0x556682fefaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.699+0000 7f3483400640 1 --2- 172.21.15.32:0/2448628476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c49f60 0x556682fefaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.699+0000 7f3483400640 1 -- 172.21.15.32:0/2448628476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 msgr2=0x556682ff0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.699+0000 7f3483400640 1 --2- 172.21.15.32:0/2448628476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 0x556682ff0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.699+0000 7f3483400640 1 -- 172.21.15.32:0/2448628476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556682cb5540 con 0x556682c210a0 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.699+0000 7f3483400640 1 --2- 172.21.15.32:0/2448628476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556682feffe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3474011510 tx=0x7f34740119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90a1abad8997c88d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.699+0000 7f346fe00640 1 -- 172.21.15.32:0/2448628476 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347401e070 con 0x556682c210a0 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.699+0000 7f346fe00640 1 -- 172.21.15.32:0/2448628476 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f347400fa70 con 0x556682c210a0 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.700+0000 7f346fe00640 1 -- 172.21.15.32:0/2448628476 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474019b60 con 0x556682c210a0 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.700+0000 7f347be00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34680b4c10 0x7f34680b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.700+0000 7f347be00640 1 -- 172.21.15.32:0/2338094792 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f34701058a0 con 0x7f348410b290 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.700+0000 7f3483e00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34680b4c10 0x7f34680b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.700+0000 7f3490dfc740 1 -- 172.21.15.32:0/2448628476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 msgr2=0x556682feffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.700+0000 7f3490dfc740 1 --2- 172.21.15.32:0/2448628476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556682feffe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3474011510 tx=0x7f34740119d0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.701+0000 7f3490dfc740 1 -- 172.21.15.32:0/2448628476 shutdown_connections 2024-08-31T07:11:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.701+0000 7f3490dfc740 1 --2- 172.21.15.32:0/2448628476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 0x556682ff0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.701+0000 7f3490dfc740 1 --2- 172.21.15.32:0/2448628476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556682feffe0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.701+0000 7f3490dfc740 1 --2- 172.21.15.32:0/2448628476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c49f60 0x556682fefaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.701+0000 7f3490dfc740 1 -- 172.21.15.32:0/2448628476 >> 172.21.15.32:0/2448628476 conn(0x556682c3e350 msgr2=0x556682fed990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.701+0000 7f3490dfc740 1 -- 172.21.15.32:0/2448628476 shutdown_connections 2024-08-31T07:11:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.701+0000 7f3490dfc740 1 -- 172.21.15.32:0/2448628476 wait complete. 2024-08-31T07:11:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.701+0000 7f3483e00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34680b4c10 0x7f34680b7120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f347c011990 tx=0x7f347c0096a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:30.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.702+0000 7f3490dfc740 1 -- start start 2024-08-31T07:11:30.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.702+0000 7f3490dfc740 1 Processor -- start 2024-08-31T07:11:30.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.702+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c210a0 0x556682ffb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.702+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 0x556682ffb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.702+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c49f60 0x556683007570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.702+0000 7f3490dfc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556682db85f0 con 0x556682c29280 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.702+0000 7f3490dfc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556682edcbd0 con 0x556682c210a0 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.702+0000 7f3490dfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556682ffe280 con 0x556682c49f60 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.702+0000 7f3490dfc740 10 client.0 _ll_register_callbacks cb 0x556682f02210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3482a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 0x556682ffb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3482a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 0x556682ffb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56290/0 (socket says 172.21.15.32:56290) 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3483400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c210a0 0x556682ffb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3482a00640 1 -- 172.21.15.32:0/595061756 learned_addr learned my addr 172.21.15.32:0/595061756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3483e00640 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c49f60 0x556683007570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3482a00640 1 -- 172.21.15.32:0/595061756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c49f60 msgr2=0x556683007570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3482a00640 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c49f60 0x556683007570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3482a00640 1 -- 172.21.15.32:0/595061756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c210a0 msgr2=0x556682ffb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3482a00640 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c210a0 0x556682ffb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3482a00640 1 -- 172.21.15.32:0/595061756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556682f1d650 con 0x556682c29280 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3483400640 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c210a0 0x556682ffb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f3482a00640 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 0x556682ffb770 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f34700cc110 tx=0x7f3470109040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd2b45de7360574 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.703+0000 7f346cc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.704+0000 7f346cc00640 1 -- 172.21.15.32:0/595061756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470048070 con 0x556682c29280 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.704+0000 7f346cc00640 1 -- 172.21.15.32:0/595061756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f34700cc3e0 con 0x556682c29280 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.704+0000 7f3490dfc740 1 -- 172.21.15.32:0/595061756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f347400fa70 con 0x556682c29280 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.704+0000 7f346cc00640 1 -- 172.21.15.32:0/595061756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f34700cbf50 con 0x556682c29280 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.704+0000 7f3490dfc740 10 client.15105 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.704+0000 7f3490dfc740 1 -- 172.21.15.32:0/595061756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55668300a060 con 0x556682c29280 2024-08-31T07:11:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.704+0000 7f346cc00640 1 -- 172.21.15.32:0/595061756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f347400fa70 con 0x556682c29280 2024-08-31T07:11:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.704+0000 7f346cc00640 1 -- 172.21.15.32:0/595061756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f347004b060 con 0x556682c29280 2024-08-31T07:11:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.705+0000 7f3490dfc740 10 client.15105 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.705+0000 7f3490dfc740 10 client.15105 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T07:11:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.705+0000 7f3490dfc740 20 client.15105 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.705+0000 7f3490dfc740 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x556682c088e0 0x55668300db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.705+0000 7f3490dfc740 4 client.15105 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:11:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.705+0000 7f3490dfc740 1 -- 172.21.15.32:0/595061756 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556682f684a0 con 0x556682c088e0 2024-08-31T07:11:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.705+0000 7f3483400640 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x556682c088e0 0x55668300db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.706+0000 7f3483400640 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x556682c088e0 0x55668300db50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec53f899487febc6 server_cookie=ef367ae606a6fffe in_seq=0 out_seq=0 2024-08-31T07:11:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.706+0000 7f346cc00640 10 client.15105 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:11:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.706+0000 7f346cc00640 1 -- 172.21.15.32:0/595061756 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556682f684a0 con 0x556682c088e0 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.706+0000 7f346cc00640 10 client.15105 handle_command_reply: tid=0 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 client.15105 shutdown 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 -- 172.21.15.32:0/595061756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 msgr2=0x556682ffb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 0x556682ffb770 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f34700cc110 tx=0x7f3470109040 comp rx=0 tx=0).stop 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 -- 172.21.15.32:0/595061756 shutdown_connections 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x556682c088e0 0x55668300db50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c49f60 0x556683007570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c29280 0x556682ffb770 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 --2- 172.21.15.32:0/595061756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c210a0 0x556682ffb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f346cc00640 0 client.15105 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 -- 172.21.15.32:0/595061756 >> 172.21.15.32:0/595061756 conn(0x556682c3e350 msgr2=0x556682ff2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 -- 172.21.15.32:0/595061756 shutdown_connections 2024-08-31T07:11:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 1 -- 172.21.15.32:0/595061756 wait complete. 2024-08-31T07:11:30.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.707+0000 7f3490dfc740 20 client.15105 trim_cache size 0 max 16384 2024-08-31T07:11:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.726+0000 7f3490dfc740 1 Processor -- start 2024-08-31T07:11:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.726+0000 7f3490dfc740 1 -- start start 2024-08-31T07:11:30.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c210a0 0x55668300db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c29280 0x55668300b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c49f60 0x5566830074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3490dfc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55668300be90 con 0x556682c210a0 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3490dfc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55668300c560 con 0x556682c29280 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3490dfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55668300cc30 con 0x556682c49f60 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3483400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c210a0 0x55668300db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3483400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c210a0 0x55668300db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56300/0 (socket says 172.21.15.32:56300) 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3483400640 1 -- 172.21.15.32:0/2293242704 learned_addr learned my addr 172.21.15.32:0/2293242704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3483e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c49f60 0x5566830074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.727+0000 7f3482a00640 1 --2- 172.21.15.32:0/2293242704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c29280 0x55668300b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.728+0000 7f3483400640 1 -- 172.21.15.32:0/2293242704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c49f60 msgr2=0x5566830074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.728+0000 7f3483400640 1 --2- 172.21.15.32:0/2293242704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c49f60 0x5566830074e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.728+0000 7f3483400640 1 -- 172.21.15.32:0/2293242704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c29280 msgr2=0x55668300b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.728+0000 7f3483400640 1 --2- 172.21.15.32:0/2293242704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c29280 0x55668300b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.728+0000 7f3483400640 1 -- 172.21.15.32:0/2293242704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556682ff1d90 con 0x556682c210a0 2024-08-31T07:11:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.728+0000 7f3483400640 1 --2- 172.21.15.32:0/2293242704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c210a0 0x55668300db50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3474011ed0 tx=0x7f3474005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74460b78f41ffb7d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:30.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.728+0000 7f346f400640 1 -- 172.21.15.32:0/2293242704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347401e040 con 0x556682c210a0 2024-08-31T07:11:30.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.728+0000 7f346f400640 1 -- 172.21.15.32:0/2293242704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f347401f040 con 0x556682c210a0 2024-08-31T07:11:30.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.728+0000 7f346f400640 1 -- 172.21.15.32:0/2293242704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474005510 con 0x556682c210a0 2024-08-31T07:11:30.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.729+0000 7f3490dfc740 1 -- 172.21.15.32:0/2293242704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c210a0 msgr2=0x55668300db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:30.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.729+0000 7f3490dfc740 1 --2- 172.21.15.32:0/2293242704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c210a0 0x55668300db50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3474011ed0 tx=0x7f3474005db0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.729+0000 7f3490dfc740 1 -- 172.21.15.32:0/2293242704 shutdown_connections 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.729+0000 7f3490dfc740 1 --2- 172.21.15.32:0/2293242704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c49f60 0x5566830074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.729+0000 7f3490dfc740 1 --2- 172.21.15.32:0/2293242704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c29280 0x55668300b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.729+0000 7f3490dfc740 1 --2- 172.21.15.32:0/2293242704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c210a0 0x55668300db50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.729+0000 7f3490dfc740 1 -- 172.21.15.32:0/2293242704 >> 172.21.15.32:0/2293242704 conn(0x556682c3e350 msgr2=0x556682ff9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.729+0000 7f3490dfc740 1 -- 172.21.15.32:0/2293242704 shutdown_connections 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.729+0000 7f3490dfc740 1 -- 172.21.15.32:0/2293242704 wait complete. 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.730+0000 7f3490dfc740 1 -- start start 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.730+0000 7f3490dfc740 1 Processor -- start 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.730+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556683016550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.730+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c29280 0x55668300fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.730+0000 7f3490dfc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c49f60 0x55668300ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.730+0000 7f3490dfc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55668300d9d0 con 0x556682c49f60 2024-08-31T07:11:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.730+0000 7f3490dfc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55668300d300 con 0x556682c29280 2024-08-31T07:11:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.730+0000 7f3490dfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55668300e0a0 con 0x556682c210a0 2024-08-31T07:11:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.730+0000 7f3490dfc740 10 client.0 _ll_register_callbacks cb 0x7f3470004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3483400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556683016550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3483e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c49f60 0x55668300ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3483400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556683016550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38084/0 (socket says 172.21.15.32:38084) 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3483400640 1 -- 172.21.15.32:0/4162020789 learned_addr learned my addr 172.21.15.32:0/4162020789 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3482a00640 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c29280 0x55668300fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3483400640 1 -- 172.21.15.32:0/4162020789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c29280 msgr2=0x55668300fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3483400640 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c29280 0x55668300fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3483400640 1 -- 172.21.15.32:0/4162020789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c49f60 msgr2=0x55668300ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3483400640 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c49f60 0x55668300ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3483400640 1 -- 172.21.15.32:0/4162020789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556683010680 con 0x556682c210a0 2024-08-31T07:11:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f3483400640 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556683016550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3474013040 tx=0x7f3474011270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=654f5a21e410e258 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.731+0000 7f346cc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.732+0000 7f346cc00640 1 -- 172.21.15.32:0/4162020789 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347403a060 con 0x556682c210a0 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.732+0000 7f346cc00640 1 -- 172.21.15.32:0/4162020789 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f347400f850 con 0x556682c210a0 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.732+0000 7f3490dfc740 1 -- 172.21.15.32:0/4162020789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f347401f040 con 0x556682c210a0 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.732+0000 7f346cc00640 1 -- 172.21.15.32:0/4162020789 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f347401e040 con 0x556682c210a0 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.732+0000 7f3490dfc740 10 client.25126 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.732+0000 7f3490dfc740 1 -- 172.21.15.32:0/4162020789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556683079020 con 0x556682c210a0 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.733+0000 7f346cc00640 1 -- 172.21.15.32:0/4162020789 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f347401f040 con 0x556682c210a0 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.733+0000 7f346cc00640 1 -- 172.21.15.32:0/4162020789 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3474019a40 con 0x556682c210a0 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.733+0000 7f3490dfc740 10 client.25126 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.733+0000 7f3490dfc740 10 client.25126 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.733+0000 7f3490dfc740 20 client.25126 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.733+0000 7f3490dfc740 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x556682bddc90 0x55668307c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.733+0000 7f3490dfc740 4 client.25126 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.733+0000 7f3490dfc740 1 -- 172.21.15.32:0/4162020789 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "status"}) -- 0x556682f63f40 con 0x556682bddc90 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.733+0000 7f3482a00640 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x556682bddc90 0x55668307c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.734+0000 7f3482a00640 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x556682bddc90 0x55668307c0c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91a72407a1c2a19d server_cookie=82dd0ab7f0635151 in_seq=0 out_seq=0 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.734+0000 7f346cc00640 10 client.25126 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.734+0000 7f346cc00640 1 -- 172.21.15.32:0/4162020789 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556682f63f40 con 0x556682bddc90 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.734+0000 7f346cc00640 10 client.25126 handle_command_reply: tid=0 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.734+0000 7f3490dfc740 1 client.25126 shutdown 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.734+0000 7f3490dfc740 1 -- 172.21.15.32:0/4162020789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 msgr2=0x556683016550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.734+0000 7f3490dfc740 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556683016550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3474013040 tx=0x7f3474011270 comp rx=0 tx=0).stop 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.734+0000 7f3490dfc740 1 -- 172.21.15.32:0/4162020789 shutdown_connections 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3490dfc740 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x556682bddc90 0x55668307c0c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3490dfc740 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556682c49f60 0x55668300ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3490dfc740 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556682c29280 0x55668300fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3490dfc740 1 --2- 172.21.15.32:0/4162020789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556682c210a0 0x556683016550 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f346cc00640 0 client.25126 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:11:30.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3490dfc740 1 -- 172.21.15.32:0/4162020789 >> 172.21.15.32:0/4162020789 conn(0x556682c3e350 msgr2=0x556682ff98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:30.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3490dfc740 1 -- 172.21.15.32:0/4162020789 shutdown_connections 2024-08-31T07:11:30.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3490dfc740 1 -- 172.21.15.32:0/4162020789 wait complete. 2024-08-31T07:11:30.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3490dfc740 20 client.25126 trim_cache size 0 max 16384 2024-08-31T07:11:30.740 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:11:30.740 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:11:30.740 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 2, 2024-08-31T07:11:30.740 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24694, 2024-08-31T07:11:30.740 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:11:30.741 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:11:30.741 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:11:30.741 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 89.886138412999998, 2024-08-31T07:11:30.741 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2024-08-31T07:11:30.741 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:11:30.741 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T07:11:30.741 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 125.643846615 2024-08-31T07:11:30.741 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:11:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3488a00640 1 -- 172.21.15.32:0/2338094792 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34680b4c10 msgr2=0x7f34680b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3488a00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34680b4c10 0x7f34680b7120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f347c011990 tx=0x7f347c0096a0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3488a00640 1 -- 172.21.15.32:0/2338094792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f348410b290 msgr2=0x7f348419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.735+0000 7f3488a00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f348410b290 0x7f348419c870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f347002a930 tx=0x7f3470004060 comp rx=0 tx=0).stop 2024-08-31T07:11:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.736+0000 7f3488a00640 1 -- 172.21.15.32:0/2338094792 shutdown_connections 2024-08-31T07:11:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.736+0000 7f3488a00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34680b4c10 0x7f34680b7120 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.736+0000 7f3488a00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348410bbe0 0x7f34841a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.736+0000 7f3488a00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f348410b290 0x7f348419c870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.736+0000 7f3488a00640 1 --2- 172.21.15.32:0/2338094792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3484105950 0x7f348419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:30.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.736+0000 7f3488a00640 1 -- 172.21.15.32:0/2338094792 >> 172.21.15.32:0/2338094792 conn(0x7f3484101420 msgr2=0x7f3484103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:30.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.736+0000 7f3488a00640 1 -- 172.21.15.32:0/2338094792 shutdown_connections 2024-08-31T07:11:30.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:30.736+0000 7f3488a00640 1 -- 172.21.15.32:0/2338094792 wait complete. 2024-08-31T07:11:31.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.a status 2024-08-31T07:11:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:32.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:32 smithi032 ceph-mon[17252]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:32 smithi084 ceph-mon[22387]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.617+0000 7f4f94800640 1 -- 172.21.15.32:0/3430507116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8c10b470 msgr2=0x7f4f8c10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.617+0000 7f4f94800640 1 --2- 172.21.15.32:0/3430507116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8c10b470 0x7f4f8c10b880 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009900 tx=0x7f4f7c02a410 comp rx=0 tx=0).stop 2024-08-31T07:11:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.617+0000 7f4f94800640 1 -- 172.21.15.32:0/3430507116 shutdown_connections 2024-08-31T07:11:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.617+0000 7f4f94800640 1 --2- 172.21.15.32:0/3430507116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f8c10bdc0 0x7f4f8c1125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.617+0000 7f4f94800640 1 --2- 172.21.15.32:0/3430507116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8c10b470 0x7f4f8c10b880 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.617+0000 7f4f94800640 1 --2- 172.21.15.32:0/3430507116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8c105b30 0x7f4f8c105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.617+0000 7f4f94800640 1 -- 172.21.15.32:0/3430507116 >> 172.21.15.32:0/3430507116 conn(0x7f4f8c101600 msgr2=0x7f4f8c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.617+0000 7f4f94800640 1 -- 172.21.15.32:0/3430507116 shutdown_connections 2024-08-31T07:11:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.617+0000 7f4f94800640 1 -- 172.21.15.32:0/3430507116 wait complete. 2024-08-31T07:11:32.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.618+0000 7f4f94800640 1 Processor -- start 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.618+0000 7f4f94800640 1 -- start start 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f94800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f8c105b30 0x7f4f8c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f94800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8c10b470 0x7f4f8c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f94800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8c10bdc0 0x7f4f8c1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f94800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c115a80 con 0x7f4f8c10bdc0 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f94800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c115900 con 0x7f4f8c10b470 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f94800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c115c00 con 0x7f4f8c105b30 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f92800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8c10bdc0 0x7f4f8c1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f92800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8c10bdc0 0x7f4f8c1a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56324/0 (socket says 172.21.15.32:56324) 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f92800640 1 -- 172.21.15.32:0/3409879928 learned_addr learned my addr 172.21.15.32:0/3409879928 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f91400640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8c10b470 0x7f4f8c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.619+0000 7f4f91e00640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f8c105b30 0x7f4f8c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f92800640 1 -- 172.21.15.32:0/3409879928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f8c105b30 msgr2=0x7f4f8c19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f92800640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f8c105b30 0x7f4f8c19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f92800640 1 -- 172.21.15.32:0/3409879928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8c10b470 msgr2=0x7f4f8c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f92800640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8c10b470 0x7f4f8c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f92800640 1 -- 172.21.15.32:0/3409879928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c1a23c0 con 0x7f4f8c10bdc0 2024-08-31T07:11:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f91400640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8c10b470 0x7f4f8c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f92800640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8c10bdc0 0x7f4f8c1a1c90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800eaa0 tx=0x7f4f8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f86a00640 1 -- 172.21.15.32:0/3409879928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8800cdb0 con 0x7f4f8c10bdc0 2024-08-31T07:11:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f86a00640 1 -- 172.21.15.32:0/3409879928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f88005020 con 0x7f4f8c10bdc0 2024-08-31T07:11:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f94800640 1 -- 172.21.15.32:0/3409879928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8c1a26b0 con 0x7f4f8c10bdc0 2024-08-31T07:11:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f86a00640 1 -- 172.21.15.32:0/3409879928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f880116d0 con 0x7f4f8c10bdc0 2024-08-31T07:11:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.620+0000 7f4f94800640 1 -- 172.21.15.32:0/3409879928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8c1a8e10 con 0x7f4f8c10bdc0 2024-08-31T07:11:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.623+0000 7f4f9b3ed740 1 Processor -- start 2024-08-31T07:11:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.624+0000 7f4f86a00640 1 -- 172.21.15.32:0/3409879928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f4f880051d0 con 0x7f4f8c10bdc0 2024-08-31T07:11:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.624+0000 7f4f9b3ed740 1 -- start start 2024-08-31T07:11:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.624+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 0x55ba37ee48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.624+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 0x55ba37ee4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.624+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37ee5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.624+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ba37e5b550 con 0x55ba37b1e280 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.624+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ba37e4aae0 con 0x55ba37b3ef60 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.624+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ba37e4ea90 con 0x55ba37b160a0 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 0x55ba37ee48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 0x55ba37ee4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 0x55ba37ee48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59064/0 (socket says 172.21.15.32:59064) 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 0x55ba37ee4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38100/0 (socket says 172.21.15.32:38100) 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91400640 1 -- 172.21.15.32:0/1035069715 learned_addr learned my addr 172.21.15.32:0/1035069715 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:32.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91400640 1 --2- 172.21.15.32:0/1035069715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37ee5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91e00640 1 -- 172.21.15.32:0/1035069715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 msgr2=0x55ba37ee48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:32.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91e00640 1 --2- 172.21.15.32:0/1035069715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 0x55ba37ee48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91e00640 1 -- 172.21.15.32:0/1035069715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 msgr2=0x55ba37ee5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:32.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91e00640 1 --2- 172.21.15.32:0/1035069715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37ee5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.625+0000 7f4f91e00640 1 -- 172.21.15.32:0/1035069715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba37baa540 con 0x55ba37b160a0 2024-08-31T07:11:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.626+0000 7f4f86a00640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4f740b4c10 0x7f4f740b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.626+0000 7f4f91e00640 1 --2- 172.21.15.32:0/1035069715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 0x55ba37ee4e30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009c20 tx=0x7f4f80011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ddd1c296755196f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.626+0000 7f4f86a00640 1 -- 172.21.15.32:0/3409879928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f4f880e5520 con 0x7f4f8c10bdc0 2024-08-31T07:11:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.626+0000 7f4f91400640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4f740b4c10 0x7f4f740b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.626+0000 7f4f7b000640 1 -- 172.21.15.32:0/1035069715 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80004050 con 0x55ba37b160a0 2024-08-31T07:11:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.626+0000 7f4f7b000640 1 -- 172.21.15.32:0/1035069715 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f800054d0 con 0x55ba37b160a0 2024-08-31T07:11:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.626+0000 7f4f7b000640 1 -- 172.21.15.32:0/1035069715 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80004c80 con 0x55ba37b160a0 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.627+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1035069715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 msgr2=0x55ba37ee4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.627+0000 7f4f91400640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4f740b4c10 0x7f4f740b7120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c005f70 tx=0x7f4f7c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.627+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/1035069715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 0x55ba37ee4e30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009c20 tx=0x7f4f80011810 comp rx=0 tx=0).stop 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.627+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1035069715 shutdown_connections 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.627+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/1035069715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37ee5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.627+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/1035069715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 0x55ba37ee4e30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.627+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/1035069715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 0x55ba37ee48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.627+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1035069715 >> 172.21.15.32:0/1035069715 conn(0x55ba37b33350 msgr2=0x55ba37ee27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.627+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1035069715 shutdown_connections 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.627+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1035069715 wait complete. 2024-08-31T07:11:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.628+0000 7f4f9b3ed740 1 -- start start 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.628+0000 7f4f9b3ed740 1 Processor -- start 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.628+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b160a0 0x55ba37eefff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37ef0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b3ef60 0x55ba37efc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ba37cad5f0 con 0x55ba37b1e280 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ba37dd1bd0 con 0x55ba37b160a0 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ba37ef30d0 con 0x55ba37b3ef60 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f9b3ed740 10 client.0 _ll_register_callbacks cb 0x55ba37df7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f91400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37ef0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f91e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b160a0 0x55ba37eefff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f91400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37ef0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56334/0 (socket says 172.21.15.32:56334) 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f91400640 1 -- 172.21.15.32:0/280912366 learned_addr learned my addr 172.21.15.32:0/280912366 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f92800640 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b3ef60 0x55ba37efc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f92800640 1 -- 172.21.15.32:0/280912366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b160a0 msgr2=0x55ba37eefff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:32.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f92800640 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b160a0 0x55ba37eefff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f92800640 1 -- 172.21.15.32:0/280912366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 msgr2=0x55ba37ef0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f92800640 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37ef0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f92800640 1 -- 172.21.15.32:0/280912366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba37e12650 con 0x55ba37b3ef60 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.629+0000 7f4f91400640 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37ef0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.630+0000 7f4f92800640 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b3ef60 0x55ba37efc3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4f880e2dd0 tx=0x7f4f880abe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63dca399c580858c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.630+0000 7f4f6f400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.630+0000 7f4f6f400640 1 -- 172.21.15.32:0/280912366 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8801f040 con 0x55ba37b3ef60 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.630+0000 7f4f6f400640 1 -- 172.21.15.32:0/280912366 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f4f880b01b0 con 0x55ba37b3ef60 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.630+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/280912366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f800054d0 con 0x55ba37b3ef60 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.630+0000 7f4f6f400640 1 -- 172.21.15.32:0/280912366 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f880054f0 con 0x55ba37b3ef60 2024-08-31T07:11:32.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.630+0000 7f4f9b3ed740 10 client.25138 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.630+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/280912366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba37efee40 con 0x55ba37b3ef60 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.631+0000 7f4f6f400640 1 -- 172.21.15.32:0/280912366 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f800054d0 con 0x55ba37b3ef60 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.631+0000 7f4f6f400640 1 -- 172.21.15.32:0/280912366 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4f8800f040 con 0x55ba37b3ef60 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.631+0000 7f4f9b3ed740 10 client.25138 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.631+0000 7f4f9b3ed740 10 client.25138 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.631+0000 7f4f9b3ed740 20 client.25138 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.631+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ba37afd8e0 0x55ba37f02840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.631+0000 7f4f9b3ed740 4 client.25138 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.631+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/280912366 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba37e5d4a0 con 0x55ba37afd8e0 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.631+0000 7f4f91e00640 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ba37afd8e0 0x55ba37f02840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.632+0000 7f4f91e00640 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ba37afd8e0 0x55ba37f02840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efd1baa82527b4b1 server_cookie=117d0c4c0b70c58f in_seq=0 out_seq=0 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.632+0000 7f4f6f400640 10 client.25138 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:11:32.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f6f400640 1 -- 172.21.15.32:0/280912366 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba37e5d4a0 con 0x55ba37afd8e0 2024-08-31T07:11:32.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f6f400640 10 client.25138 handle_command_reply: tid=0 2024-08-31T07:11:32.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f9b3ed740 1 client.25138 shutdown 2024-08-31T07:11:32.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/280912366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b3ef60 msgr2=0x55ba37efc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b3ef60 0x55ba37efc3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4f880e2dd0 tx=0x7f4f880abe60 comp rx=0 tx=0).stop 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/280912366 shutdown_connections 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b3ef60 0x55ba37efc3c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37ef0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ba37afd8e0 0x55ba37f02840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/280912366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b160a0 0x55ba37eefff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/280912366 >> 172.21.15.32:0/280912366 conn(0x55ba37b33350 msgr2=0x55ba37ee7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.633+0000 7f4f6f400640 0 client.25138 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.634+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/280912366 shutdown_connections 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.634+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/280912366 wait complete. 2024-08-31T07:11:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.634+0000 7f4f9b3ed740 20 client.25138 trim_cache size 0 max 16384 2024-08-31T07:11:32.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.653+0000 7f4f9b3ed740 1 Processor -- start 2024-08-31T07:11:32.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.653+0000 7f4f9b3ed740 1 -- start start 2024-08-31T07:11:32.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.653+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 0x55ba37edd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.653+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37eee870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.653+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 0x55ba37eeedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.653+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ba37f00b80 con 0x55ba37b1e280 2024-08-31T07:11:32.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.653+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ba37f01250 con 0x55ba37b3ef60 2024-08-31T07:11:32.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.653+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ba37f01920 con 0x55ba37b160a0 2024-08-31T07:11:32.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f91400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37eee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f91400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37eee870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56344/0 (socket says 172.21.15.32:56344) 2024-08-31T07:11:32.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f91400640 1 -- 172.21.15.32:0/3239136686 learned_addr learned my addr 172.21.15.32:0/3239136686 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:32.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f91e00640 1 --2- 172.21.15.32:0/3239136686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 0x55ba37edd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f92800640 1 --2- 172.21.15.32:0/3239136686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 0x55ba37eeedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f91400640 1 -- 172.21.15.32:0/3239136686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 msgr2=0x55ba37edd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f91400640 1 --2- 172.21.15.32:0/3239136686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 0x55ba37edd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f91400640 1 -- 172.21.15.32:0/3239136686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 msgr2=0x55ba37eeedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f91400640 1 --2- 172.21.15.32:0/3239136686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 0x55ba37eeedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f91400640 1 -- 172.21.15.32:0/3239136686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba37eebb60 con 0x55ba37b1e280 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.654+0000 7f4f91400640 1 --2- 172.21.15.32:0/3239136686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37eee870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c004350 tx=0x7f4f7c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f957ce9a03b1ba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.655+0000 7f4f7a600640 1 -- 172.21.15.32:0/3239136686 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c035040 con 0x55ba37b1e280 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.655+0000 7f4f7a600640 1 -- 172.21.15.32:0/3239136686 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f7c004770 con 0x55ba37b1e280 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.655+0000 7f4f7a600640 1 -- 172.21.15.32:0/3239136686 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c003850 con 0x55ba37b1e280 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.655+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/3239136686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 msgr2=0x55ba37eee870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.655+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/3239136686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37eee870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c004350 tx=0x7f4f7c009260 comp rx=0 tx=0).stop 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.656+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/3239136686 shutdown_connections 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.656+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/3239136686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b3ef60 0x55ba37eeedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.656+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/3239136686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b1e280 0x55ba37eee870 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.656+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/3239136686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b160a0 0x55ba37edd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.656+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/3239136686 >> 172.21.15.32:0/3239136686 conn(0x55ba37b33350 msgr2=0x55ba37efc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:32.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.656+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/3239136686 shutdown_connections 2024-08-31T07:11:32.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.656+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/3239136686 wait complete. 2024-08-31T07:11:32.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.656+0000 7f4f9b3ed740 1 -- start start 2024-08-31T07:11:32.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f9b3ed740 1 Processor -- start 2024-08-31T07:11:32.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b160a0 0x55ba37f06a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b1e280 0x55ba37f06f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f9b3ed740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b3ef60 0x55ba37f07490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ba37f026c0 con 0x55ba37b3ef60 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ba37f01ff0 con 0x55ba37b160a0 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f9b3ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ba37f02d90 con 0x55ba37b1e280 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f9b3ed740 10 client.0 _ll_register_callbacks cb 0x7f4f880ab8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f92800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b3ef60 0x55ba37f07490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f92800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b3ef60 0x55ba37f07490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56348/0 (socket says 172.21.15.32:56348) 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f92800640 1 -- 172.21.15.32:0/1460949590 learned_addr learned my addr 172.21.15.32:0/1460949590 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.657+0000 7f4f91e00640 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b160a0 0x55ba37f06a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f91400640 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b1e280 0x55ba37f06f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f92800640 1 -- 172.21.15.32:0/1460949590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b1e280 msgr2=0x55ba37f06f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f92800640 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b1e280 0x55ba37f06f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f92800640 1 -- 172.21.15.32:0/1460949590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b160a0 msgr2=0x55ba37f06a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:32.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f92800640 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b160a0 0x55ba37f06a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f92800640 1 -- 172.21.15.32:0/1460949590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba37f6cd10 con 0x55ba37b3ef60 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f91400640 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b1e280 0x55ba37f06f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f92800640 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b3ef60 0x55ba37f07490 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800cfd0 tx=0x7f4f880a7940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db26d7a639bc236 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f6f400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f6f400640 1 -- 172.21.15.32:0/1460949590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8801f070 con 0x55ba37b3ef60 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f6f400640 1 -- 172.21.15.32:0/1460949590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f4f88091b90 con 0x55ba37b3ef60 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.658+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1460949590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f7c004770 con 0x55ba37b3ef60 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f6f400640 1 -- 172.21.15.32:0/1460949590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f88005690 con 0x55ba37b3ef60 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f9b3ed740 10 client.15129 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1460949590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba37f6dec0 con 0x55ba37b3ef60 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f6f400640 1 -- 172.21.15.32:0/1460949590 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f7c004770 con 0x55ba37b3ef60 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f6f400640 1 -- 172.21.15.32:0/1460949590 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4f880b16f0 con 0x55ba37b3ef60 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f9b3ed740 10 client.15129 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:32.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f9b3ed740 10 client.15129 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-31T07:11:32.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f9b3ed740 20 client.15129 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:32.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ba37ad2c90 0x55ba37f71810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:32.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f9b3ed740 4 client.15129 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:11:32.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.659+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1460949590 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "status"}) -- 0x55ba37e58f40 con 0x55ba37ad2c90 2024-08-31T07:11:32.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.660+0000 7f4f91e00640 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ba37ad2c90 0x55ba37f71810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.660+0000 7f4f91e00640 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ba37ad2c90 0x55ba37f71810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7bf7a6cb8ff3276 server_cookie=9f29347217c22f67 in_seq=0 out_seq=0 2024-08-31T07:11:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.660+0000 7f4f6f400640 10 client.15129 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:11:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.660+0000 7f4f6f400640 1 -- 172.21.15.32:0/1460949590 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ba37e58f40 con 0x55ba37ad2c90 2024-08-31T07:11:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.660+0000 7f4f6f400640 10 client.15129 handle_command_reply: tid=0 2024-08-31T07:11:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.660+0000 7f4f9b3ed740 1 client.15129 shutdown 2024-08-31T07:11:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1460949590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b3ef60 msgr2=0x55ba37f07490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:32.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b3ef60 0x55ba37f07490 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800cfd0 tx=0x7f4f880a7940 comp rx=0 tx=0).stop 2024-08-31T07:11:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1460949590 shutdown_connections 2024-08-31T07:11:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba37b3ef60 0x55ba37f07490 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ba37b1e280 0x55ba37f06f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ba37ad2c90 0x55ba37f71810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 1 --2- 172.21.15.32:0/1460949590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba37b160a0 0x55ba37f06a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f6f400640 0 client.15129 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:11:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1460949590 >> 172.21.15.32:0/1460949590 conn(0x55ba37b33350 msgr2=0x55ba37edd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1460949590 shutdown_connections 2024-08-31T07:11:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 1 -- 172.21.15.32:0/1460949590 wait complete. 2024-08-31T07:11:32.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.661+0000 7f4f9b3ed740 20 client.15129 trim_cache size 0 max 16384 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 1, 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24718, 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 92.821600298999996, 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 119.324331453 2024-08-31T07:11:32.667 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:11:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 -- 172.21.15.32:0/3409879928 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4f740b4c10 msgr2=0x7f4f740b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4f740b4c10 0x7f4f740b7120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c005f70 tx=0x7f4f7c031400 comp rx=0 tx=0).stop 2024-08-31T07:11:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 -- 172.21.15.32:0/3409879928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8c10bdc0 msgr2=0x7f4f8c1a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8c10bdc0 0x7f4f8c1a1c90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800eaa0 tx=0x7f4f8800ef60 comp rx=0 tx=0).stop 2024-08-31T07:11:32.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 -- 172.21.15.32:0/3409879928 shutdown_connections 2024-08-31T07:11:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4f740b4c10 0x7f4f740b7120 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8c10bdc0 0x7f4f8c1a1c90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8c10b470 0x7f4f8c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 --2- 172.21.15.32:0/3409879928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f8c105b30 0x7f4f8c19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 -- 172.21.15.32:0/3409879928 >> 172.21.15.32:0/3409879928 conn(0x7f4f8c101600 msgr2=0x7f4f8c108ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 -- 172.21.15.32:0/3409879928 shutdown_connections 2024-08-31T07:11:32.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:32.662+0000 7f4f94800640 1 -- 172.21.15.32:0/3409879928 wait complete. 2024-08-31T07:11:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:32 smithi073 ceph-mon[21995]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:33.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.b status 2024-08-31T07:11:34.159 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:34.474 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:34 smithi032 ceph-mon[17252]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:34 smithi084 ceph-mon[22387]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:34 smithi073 ceph-mon[21995]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.828+0000 7f68cca00640 1 -- 172.21.15.32:0/3298465530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c8105950 msgr2=0x7f68c8105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.828+0000 7f68cca00640 1 --2- 172.21.15.32:0/3298465530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c8105950 0x7f68c8105d60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009900 tx=0x7f68b802a400 comp rx=0 tx=0).stop 2024-08-31T07:11:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.828+0000 7f68cca00640 1 -- 172.21.15.32:0/3298465530 shutdown_connections 2024-08-31T07:11:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.828+0000 7f68cca00640 1 --2- 172.21.15.32:0/3298465530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c810bbe0 0x7f68c8112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.828+0000 7f68cca00640 1 --2- 172.21.15.32:0/3298465530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c810b290 0x7f68c810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.828+0000 7f68cca00640 1 --2- 172.21.15.32:0/3298465530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c8105950 0x7f68c8105d60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.828+0000 7f68cca00640 1 -- 172.21.15.32:0/3298465530 >> 172.21.15.32:0/3298465530 conn(0x7f68c8101420 msgr2=0x7f68c8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.828+0000 7f68cca00640 1 -- 172.21.15.32:0/3298465530 shutdown_connections 2024-08-31T07:11:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.828+0000 7f68cca00640 1 -- 172.21.15.32:0/3298465530 wait complete. 2024-08-31T07:11:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.829+0000 7f68cca00640 1 Processor -- start 2024-08-31T07:11:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.829+0000 7f68cca00640 1 -- start start 2024-08-31T07:11:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68cca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c8105950 0x7f68c819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68cca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c810b290 0x7f68c819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68cca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c810bbe0 0x7f68c81a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68cca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81158a0 con 0x7f68c8105950 2024-08-31T07:11:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68cca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8115720 con 0x7f68c810bbe0 2024-08-31T07:11:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68cca00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8115a20 con 0x7f68c810b290 2024-08-31T07:11:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68c6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c8105950 0x7f68c819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68c5e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c810b290 0x7f68c819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68c6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c8105950 0x7f68c819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56366/0 (socket says 172.21.15.32:56366) 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68c6800640 1 -- 172.21.15.32:0/2699230113 learned_addr learned my addr 172.21.15.32:0/2699230113 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68c5e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c810b290 0x7f68c819c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38128/0 (socket says 172.21.15.32:38128) 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.830+0000 7f68c7200640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c810bbe0 0x7f68c81a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68c5e00640 1 -- 172.21.15.32:0/2699230113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c810bbe0 msgr2=0x7f68c81a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68c5e00640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c810bbe0 0x7f68c81a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68c5e00640 1 -- 172.21.15.32:0/2699230113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c8105950 msgr2=0x7f68c819c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68c5e00640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c8105950 0x7f68c819c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68c5e00640 1 -- 172.21.15.32:0/2699230113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c81a22a0 con 0x7f68c810b290 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68c6800640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c8105950 0x7f68c819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68c5e00640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c810b290 0x7f68c819c8d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f68b4005fd0 tx=0x7f68b400dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68bf400640 1 -- 172.21.15.32:0/2699230113 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4009c00 con 0x7f68c810b290 2024-08-31T07:11:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68bf400640 1 -- 172.21.15.32:0/2699230113 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b400be30 con 0x7f68c810b290 2024-08-31T07:11:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68cca00640 1 -- 172.21.15.32:0/2699230113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c81a2590 con 0x7f68c810b290 2024-08-31T07:11:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68bf400640 1 -- 172.21.15.32:0/2699230113 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4003d40 con 0x7f68c810b290 2024-08-31T07:11:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.831+0000 7f68cca00640 1 -- 172.21.15.32:0/2699230113 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c81a8cf0 con 0x7f68c810b290 2024-08-31T07:11:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.834+0000 7f68d4077740 1 Processor -- start 2024-08-31T07:11:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68d4077740 1 -- start start 2024-08-31T07:11:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68d4077740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf6af60 0x55e06d310aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68d4077740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d310fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68d4077740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf4a280 0x55e06d311520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68d4077740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e06d287550 con 0x55e06cf420a0 2024-08-31T07:11:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68d4077740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e06d276ae0 con 0x55e06cf4a280 2024-08-31T07:11:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68d4077740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e06d27aa90 con 0x55e06cf6af60 2024-08-31T07:11:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68c6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d310fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68c6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d310fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56378/0 (socket says 172.21.15.32:56378) 2024-08-31T07:11:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68c6800640 1 -- 172.21.15.32:0/4202100892 learned_addr learned my addr 172.21.15.32:0/4202100892 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.835+0000 7f68c7200640 1 --2- 172.21.15.32:0/4202100892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf6af60 0x55e06d310aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.836+0000 7f68bf400640 1 -- 172.21.15.32:0/2699230113 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f68b4003ee0 con 0x7f68c810b290 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.836+0000 7f68c7200640 1 -- 172.21.15.32:0/4202100892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf4a280 msgr2=0x55e06d311520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.836+0000 7f68c7200640 1 --2- 172.21.15.32:0/4202100892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf4a280 0x55e06d311520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.836+0000 7f68c7200640 1 -- 172.21.15.32:0/4202100892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 msgr2=0x55e06d310fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.836+0000 7f68c7200640 1 --2- 172.21.15.32:0/4202100892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d310fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.836+0000 7f68c7200640 1 -- 172.21.15.32:0/4202100892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e06cfd6540 con 0x55e06cf6af60 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.836+0000 7f68c6800640 1 --2- 172.21.15.32:0/4202100892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d310fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.836+0000 7f68c7200640 1 --2- 172.21.15.32:0/4202100892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf6af60 0x55e06d310aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009f10 tx=0x7f68c0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb3321ab472b80f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.836+0000 7f68b3400640 1 -- 172.21.15.32:0/4202100892 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0004d20 con 0x55e06cf6af60 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.837+0000 7f68b3400640 1 -- 172.21.15.32:0/4202100892 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c0004050 con 0x55e06cf6af60 2024-08-31T07:11:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.837+0000 7f68b3400640 1 -- 172.21.15.32:0/4202100892 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0005220 con 0x55e06cf6af60 2024-08-31T07:11:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68d4077740 1 -- 172.21.15.32:0/4202100892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf6af60 msgr2=0x55e06d310aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68d4077740 1 --2- 172.21.15.32:0/4202100892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf6af60 0x55e06d310aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009f10 tx=0x7f68c0011810 comp rx=0 tx=0).stop 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68d4077740 1 -- 172.21.15.32:0/4202100892 shutdown_connections 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68d4077740 1 --2- 172.21.15.32:0/4202100892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf4a280 0x55e06d311520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68d4077740 1 --2- 172.21.15.32:0/4202100892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d310fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68d4077740 1 --2- 172.21.15.32:0/4202100892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf6af60 0x55e06d310aa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68d4077740 1 -- 172.21.15.32:0/4202100892 >> 172.21.15.32:0/4202100892 conn(0x55e06cf5f350 msgr2=0x55e06d30e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68d4077740 1 -- 172.21.15.32:0/4202100892 shutdown_connections 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68d4077740 1 -- 172.21.15.32:0/4202100892 wait complete. 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68bf400640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68ac0b4b20 0x7f68ac0b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68bf400640 1 -- 172.21.15.32:0/2699230113 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f68b40e5160 con 0x7f68c810b290 2024-08-31T07:11:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68c6800640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68ac0b4b20 0x7f68ac0b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.838+0000 7f68d4077740 1 -- start start 2024-08-31T07:11:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.839+0000 7f68d4077740 1 Processor -- start 2024-08-31T07:11:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.839+0000 7f68c6800640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68ac0b4b20 0x7f68ac0b7030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009230 tx=0x7f68b80058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.839+0000 7f68d4077740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d31c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.839+0000 7f68d4077740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 0x55e06d31c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.839+0000 7f68d4077740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 0x55e06d328570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68d4077740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e06d0d95f0 con 0x55e06cf420a0 2024-08-31T07:11:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68d4077740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e06d1fdbd0 con 0x55e06cf6af60 2024-08-31T07:11:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68d4077740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e06d31f280 con 0x55e06cf4a280 2024-08-31T07:11:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68d4077740 10 client.0 _ll_register_callbacks cb 0x55e06d223210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d31c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d31c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56388/0 (socket says 172.21.15.32:56388) 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c6800640 1 -- 172.21.15.32:0/2804638060 learned_addr learned my addr 172.21.15.32:0/2804638060 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c7200640 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 0x55e06d328570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c5e00640 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 0x55e06d31c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c6800640 1 -- 172.21.15.32:0/2804638060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 msgr2=0x55e06d31c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c6800640 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 0x55e06d31c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c6800640 1 -- 172.21.15.32:0/2804638060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 msgr2=0x55e06d328570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c6800640 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 0x55e06d328570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c6800640 1 -- 172.21.15.32:0/2804638060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e06d23e650 con 0x55e06cf420a0 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.840+0000 7f68c7200640 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 0x55e06d328570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.841+0000 7f68c6800640 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d31c260 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f68b8000c00 tx=0x7f68b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7039fb469c59713d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.841+0000 7f68a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:11:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.841+0000 7f68a7e00640 1 -- 172.21.15.32:0/2804638060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b804e070 con 0x55e06cf420a0 2024-08-31T07:11:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.841+0000 7f68a7e00640 1 -- 172.21.15.32:0/2804638060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f68b8004770 con 0x55e06cf420a0 2024-08-31T07:11:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.841+0000 7f68d4077740 1 -- 172.21.15.32:0/2804638060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c0004050 con 0x55e06cf420a0 2024-08-31T07:11:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.841+0000 7f68a7e00640 1 -- 172.21.15.32:0/2804638060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68b805a050 con 0x55e06cf420a0 2024-08-31T07:11:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.841+0000 7f68d4077740 10 client.15141 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.841+0000 7f68d4077740 1 -- 172.21.15.32:0/2804638060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e06d32b0b0 con 0x55e06cf420a0 2024-08-31T07:11:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.842+0000 7f68a7e00640 1 -- 172.21.15.32:0/2804638060 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c0004050 con 0x55e06cf420a0 2024-08-31T07:11:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.842+0000 7f68a7e00640 1 -- 172.21.15.32:0/2804638060 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f68b8003b50 con 0x55e06cf420a0 2024-08-31T07:11:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.842+0000 7f68d4077740 10 client.15141 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.842+0000 7f68d4077740 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T07:11:34.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.842+0000 7f68d4077740 20 client.15141 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:34.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.842+0000 7f68d4077740 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e06cf298e0 0x55e06d32eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.842+0000 7f68d4077740 4 client.15141 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:11:34.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.842+0000 7f68d4077740 1 -- 172.21.15.32:0/2804638060 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e06d2894a0 con 0x55e06cf298e0 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.843+0000 7f68c5e00640 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e06cf298e0 0x55e06d32eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.843+0000 7f68c5e00640 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e06cf298e0 0x55e06d32eb10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d536bbc023e8f0b1 server_cookie=e3685c78237fde16 in_seq=0 out_seq=0 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.843+0000 7f68a7e00640 10 client.15141 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68a7e00640 1 -- 172.21.15.32:0/2804638060 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e06d2894a0 con 0x55e06cf298e0 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68a7e00640 10 client.15141 handle_command_reply: tid=0 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68d4077740 1 client.15141 shutdown 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68d4077740 1 -- 172.21.15.32:0/2804638060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 msgr2=0x55e06d31c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68d4077740 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d31c260 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f68b8000c00 tx=0x7f68b8009260 comp rx=0 tx=0).stop 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68d4077740 1 -- 172.21.15.32:0/2804638060 shutdown_connections 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68d4077740 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e06cf298e0 0x55e06d32eb10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68d4077740 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 0x55e06d328570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68d4077740 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 0x55e06d31c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68d4077740 1 --2- 172.21.15.32:0/2804638060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d31c260 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68a7e00640 0 client.15141 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:11:34.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.844+0000 7f68d4077740 1 -- 172.21.15.32:0/2804638060 >> 172.21.15.32:0/2804638060 conn(0x55e06cf5f350 msgr2=0x55e06d313c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:34.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.845+0000 7f68d4077740 1 -- 172.21.15.32:0/2804638060 shutdown_connections 2024-08-31T07:11:34.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.845+0000 7f68d4077740 1 -- 172.21.15.32:0/2804638060 wait complete. 2024-08-31T07:11:34.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.845+0000 7f68d4077740 20 client.15141 trim_cache size 0 max 16384 2024-08-31T07:11:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.864+0000 7f68d4077740 1 Processor -- start 2024-08-31T07:11:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.864+0000 7f68d4077740 1 -- start start 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.864+0000 7f68d4077740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf420a0 0x55e06d32c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.864+0000 7f68d4077740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf4a280 0x55e06d31aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.864+0000 7f68d4077740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf6af60 0x55e06d31b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.864+0000 7f68d4077740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e06d32ce50 con 0x55e06cf6af60 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.864+0000 7f68d4077740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e06d32d520 con 0x55e06cf4a280 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.864+0000 7f68d4077740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e06d32dbf0 con 0x55e06cf420a0 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf6af60 0x55e06d31b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf420a0 0x55e06d32c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf6af60 0x55e06d31b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56398/0 (socket says 172.21.15.32:56398) 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c7200640 1 -- 172.21.15.32:0/1454953847 learned_addr learned my addr 172.21.15.32:0/1454953847 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c5e00640 1 --2- 172.21.15.32:0/1454953847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf4a280 0x55e06d31aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c6800640 1 -- 172.21.15.32:0/1454953847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf4a280 msgr2=0x55e06d31aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c6800640 1 --2- 172.21.15.32:0/1454953847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf4a280 0x55e06d31aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c6800640 1 -- 172.21.15.32:0/1454953847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf6af60 msgr2=0x55e06d31b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c6800640 1 --2- 172.21.15.32:0/1454953847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf6af60 0x55e06d31b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c6800640 1 -- 172.21.15.32:0/1454953847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e06d31d520 con 0x55e06cf420a0 2024-08-31T07:11:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.865+0000 7f68c6800640 1 --2- 172.21.15.32:0/1454953847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf420a0 0x55e06d32c520 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f68b804a4f0 tx=0x7f68b802a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc076e9d27c36017 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.866+0000 7f68b2a00640 1 -- 172.21.15.32:0/1454953847 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8062070 con 0x55e06cf420a0 2024-08-31T07:11:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.866+0000 7f68b2a00640 1 -- 172.21.15.32:0/1454953847 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b80035f0 con 0x55e06cf420a0 2024-08-31T07:11:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.866+0000 7f68b2a00640 1 -- 172.21.15.32:0/1454953847 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b804e050 con 0x55e06cf420a0 2024-08-31T07:11:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 -- 172.21.15.32:0/1454953847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf420a0 msgr2=0x55e06d32c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 --2- 172.21.15.32:0/1454953847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf420a0 0x55e06d32c520 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f68b804a4f0 tx=0x7f68b802a400 comp rx=0 tx=0).stop 2024-08-31T07:11:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 -- 172.21.15.32:0/1454953847 shutdown_connections 2024-08-31T07:11:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 --2- 172.21.15.32:0/1454953847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf6af60 0x55e06d31b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 --2- 172.21.15.32:0/1454953847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf4a280 0x55e06d31aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 --2- 172.21.15.32:0/1454953847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf420a0 0x55e06d32c520 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 -- 172.21.15.32:0/1454953847 >> 172.21.15.32:0/1454953847 conn(0x55e06cf5f350 msgr2=0x55e06d309710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 -- 172.21.15.32:0/1454953847 shutdown_connections 2024-08-31T07:11:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 -- 172.21.15.32:0/1454953847 wait complete. 2024-08-31T07:11:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 -- start start 2024-08-31T07:11:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.867+0000 7f68d4077740 1 Processor -- start 2024-08-31T07:11:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68d4077740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d337230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68d4077740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 0x55e06d3377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68d4077740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 0x55e06d332f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68d4077740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e06d32e990 con 0x55e06cf420a0 2024-08-31T07:11:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68d4077740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e06d32e2c0 con 0x55e06cf6af60 2024-08-31T07:11:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68d4077740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e06d32f060 con 0x55e06cf4a280 2024-08-31T07:11:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68d4077740 10 client.0 _ll_register_callbacks cb 0x7f68b8031810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68c5e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 0x55e06d3377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68c5e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 0x55e06d3377f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38170/0 (socket says 172.21.15.32:38170) 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68c5e00640 1 -- 172.21.15.32:0/2211219964 learned_addr learned my addr 172.21.15.32:0/2211219964 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.868+0000 7f68c6800640 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d337230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68c7200640 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 0x55e06d332f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68c5e00640 1 -- 172.21.15.32:0/2211219964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 msgr2=0x55e06d332f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68c5e00640 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 0x55e06d332f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68c5e00640 1 -- 172.21.15.32:0/2211219964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 msgr2=0x55e06d337230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68c5e00640 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d337230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68c5e00640 1 -- 172.21.15.32:0/2211219964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e06d3336b0 con 0x55e06cf4a280 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68c7200640 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 0x55e06d332f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68c6800640 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d337230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68c5e00640 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 0x55e06d3377f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f68b40a79c0 tx=0x7f68b40ab3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4a12a53ef42a2e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68a7e00640 1 -- 172.21.15.32:0/2211219964 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b405b070 con 0x55e06cf4a280 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68a7e00640 1 -- 172.21.15.32:0/2211219964 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f68b40a7f80 con 0x55e06cf4a280 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68d4077740 1 -- 172.21.15.32:0/2211219964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b80035f0 con 0x55e06cf4a280 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68a7e00640 1 -- 172.21.15.32:0/2211219964 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68b4020040 con 0x55e06cf4a280 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68d4077740 10 client.25162 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.869+0000 7f68d4077740 1 -- 172.21.15.32:0/2211219964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e06d399c00 con 0x55e06cf4a280 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.870+0000 7f68a7e00640 1 -- 172.21.15.32:0/2211219964 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b80035f0 con 0x55e06cf4a280 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.870+0000 7f68a7e00640 1 -- 172.21.15.32:0/2211219964 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f68b40c3060 con 0x55e06cf4a280 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.870+0000 7f68d4077740 10 client.25162 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.870+0000 7f68d4077740 10 client.25162 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.870+0000 7f68d4077740 20 client.25162 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.870+0000 7f68d4077740 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e06cefec90 0x55e06d39d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.870+0000 7f68d4077740 4 client.25162 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:11:34.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.870+0000 7f68d4077740 1 -- 172.21.15.32:0/2211219964 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "status"}) -- 0x55e06d284f40 con 0x55e06cefec90 2024-08-31T07:11:34.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.870+0000 7f68c6800640 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e06cefec90 0x55e06d39d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.871+0000 7f68a7e00640 10 client.25162 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:11:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.871+0000 7f68c6800640 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e06cefec90 0x55e06d39d560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f8a719ed6344bd server_cookie=3b25d0097263bf37 in_seq=0 out_seq=0 2024-08-31T07:11:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.871+0000 7f68a7e00640 1 -- 172.21.15.32:0/2211219964 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e06d284f40 con 0x55e06cefec90 2024-08-31T07:11:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.871+0000 7f68a7e00640 10 client.25162 handle_command_reply: tid=0 2024-08-31T07:11:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.871+0000 7f68d4077740 1 client.25162 shutdown 2024-08-31T07:11:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.871+0000 7f68d4077740 1 -- 172.21.15.32:0/2211219964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 msgr2=0x55e06d3377f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.871+0000 7f68d4077740 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 0x55e06d3377f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f68b40a79c0 tx=0x7f68b40ab3a0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68d4077740 1 -- 172.21.15.32:0/2211219964 shutdown_connections 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68d4077740 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e06cefec90 0x55e06d39d560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68a7e00640 0 client.25162 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68d4077740 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e06cf6af60 0x55e06d332f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68d4077740 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e06cf4a280 0x55e06d3377f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68d4077740 1 --2- 172.21.15.32:0/2211219964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e06cf420a0 0x55e06d337230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68d4077740 1 -- 172.21.15.32:0/2211219964 >> 172.21.15.32:0/2211219964 conn(0x55e06cf5f350 msgr2=0x55e06d309710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68d4077740 1 -- 172.21.15.32:0/2211219964 shutdown_connections 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68d4077740 1 -- 172.21.15.32:0/2211219964 wait complete. 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68d4077740 20 client.25162 trim_cache size 0 max 16384 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:11:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24734, 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 105.222614717, 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 118.943280063 2024-08-31T07:11:34.877 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:11:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68cca00640 1 -- 172.21.15.32:0/2699230113 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68ac0b4b20 msgr2=0x7f68ac0b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.872+0000 7f68cca00640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68ac0b4b20 0x7f68ac0b7030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009230 tx=0x7f68b80058d0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.873+0000 7f68cca00640 1 -- 172.21.15.32:0/2699230113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c810b290 msgr2=0x7f68c819c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.873+0000 7f68cca00640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c810b290 0x7f68c819c8d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f68b4005fd0 tx=0x7f68b400dd20 comp rx=0 tx=0).stop 2024-08-31T07:11:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.873+0000 7f68cca00640 1 -- 172.21.15.32:0/2699230113 shutdown_connections 2024-08-31T07:11:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.873+0000 7f68cca00640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68ac0b4b20 0x7f68ac0b7030 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.873+0000 7f68cca00640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c810bbe0 0x7f68c81a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.873+0000 7f68cca00640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c810b290 0x7f68c819c8d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.873+0000 7f68cca00640 1 --2- 172.21.15.32:0/2699230113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c8105950 0x7f68c819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.873+0000 7f68cca00640 1 -- 172.21.15.32:0/2699230113 >> 172.21.15.32:0/2699230113 conn(0x7f68c8101420 msgr2=0x7f68c8103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.873+0000 7f68cca00640 1 -- 172.21.15.32:0/2699230113 shutdown_connections 2024-08-31T07:11:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:34.873+0000 7f68cca00640 1 -- 172.21.15.32:0/2699230113 wait complete. 2024-08-31T07:11:35.773 INFO:tasks.cephadm:Setup complete, yielding 2024-08-31T07:11:35.774 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-31T07:11:35.791 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-08-31T07:11:35.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch status' 2024-08-31T07:11:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:36.500 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:36 smithi032 ceph-mon[17252]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:36 smithi073 ceph-mon[21995]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:36 smithi084 ceph-mon[22387]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:36.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.815+0000 7fb72b200640 1 -- 172.21.15.32:0/3602916947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c109090 msgr2=0x7fb72c1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.815+0000 7fb72b200640 1 --2- 172.21.15.32:0/3602916947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c109090 0x7fb72c1094a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb718009900 tx=0x7fb71802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:11:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.815+0000 7fb72b200640 1 -- 172.21.15.32:0/3602916947 shutdown_connections 2024-08-31T07:11:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.815+0000 7fb72b200640 1 --2- 172.21.15.32:0/3602916947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb72c1099e0 0x7fb72c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.815+0000 7fb72b200640 1 --2- 172.21.15.32:0/3602916947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c109090 0x7fb72c1094a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.815+0000 7fb72b200640 1 --2- 172.21.15.32:0/3602916947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb72c103750 0x7fb72c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.815+0000 7fb72b200640 1 -- 172.21.15.32:0/3602916947 >> 172.21.15.32:0/3602916947 conn(0x7fb72c0ff1e0 msgr2=0x7fb72c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.815+0000 7fb72b200640 1 -- 172.21.15.32:0/3602916947 shutdown_connections 2024-08-31T07:11:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.815+0000 7fb72b200640 1 -- 172.21.15.32:0/3602916947 wait complete. 2024-08-31T07:11:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.816+0000 7fb72b200640 1 Processor -- start 2024-08-31T07:11:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb72b200640 1 -- start start 2024-08-31T07:11:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb72b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb72c103750 0x7fb72c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb72b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb72c109090 0x7fb72c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb72b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c1099e0 0x7fb72c1a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb72b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c113650 con 0x7fb72c1099e0 2024-08-31T07:11:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb72b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1134d0 con 0x7fb72c109090 2024-08-31T07:11:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb72b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1137d0 con 0x7fb72c103750 2024-08-31T07:11:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb72a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c1099e0 0x7fb72c1a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb72a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c1099e0 0x7fb72c1a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43868/0 (socket says 172.21.15.32:43868) 2024-08-31T07:11:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb72a800640 1 -- 172.21.15.32:0/527852819 learned_addr learned my addr 172.21.15.32:0/527852819 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.817+0000 7fb729e00640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb72c103750 0x7fb72c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.818+0000 7fb729400640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb72c109090 0x7fb72c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.818+0000 7fb72a800640 1 -- 172.21.15.32:0/527852819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb72c103750 msgr2=0x7fb72c19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.818+0000 7fb72a800640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb72c103750 0x7fb72c19c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.818+0000 7fb72a800640 1 -- 172.21.15.32:0/527852819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb72c109090 msgr2=0x7fb72c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.818+0000 7fb72a800640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb72c109090 0x7fb72c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.818+0000 7fb72a800640 1 -- 172.21.15.32:0/527852819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c1a2230 con 0x7fb72c1099e0 2024-08-31T07:11:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.818+0000 7fb72a800640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c1099e0 0x7fb72c1a1b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb72400eb10 tx=0x7fb72400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.818+0000 7fb722a00640 1 -- 172.21.15.32:0/527852819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724004b20 con 0x7fb72c1099e0 2024-08-31T07:11:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.819+0000 7fb722a00640 1 -- 172.21.15.32:0/527852819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb724004cc0 con 0x7fb72c1099e0 2024-08-31T07:11:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.819+0000 7fb722a00640 1 -- 172.21.15.32:0/527852819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724010670 con 0x7fb72c1099e0 2024-08-31T07:11:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.819+0000 7fb72b200640 1 -- 172.21.15.32:0/527852819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c1a2520 con 0x7fb72c1099e0 2024-08-31T07:11:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.819+0000 7fb72b200640 1 -- 172.21.15.32:0/527852819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c1a8c80 con 0x7fb72c1099e0 2024-08-31T07:11:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.820+0000 7fb72b200640 1 -- 172.21.15.32:0/527852819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c103b60 con 0x7fb72c1099e0 2024-08-31T07:11:36.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.822+0000 7fb722a00640 1 -- 172.21.15.32:0/527852819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fb724005000 con 0x7fb72c1099e0 2024-08-31T07:11:36.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.824+0000 7fb722a00640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7100b4c90 0x7fb7100b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:36.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.824+0000 7fb722a00640 1 -- 172.21.15.32:0/527852819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb7240e5b30 con 0x7fb72c1099e0 2024-08-31T07:11:36.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.824+0000 7fb729e00640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7100b4c90 0x7fb7100b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:36.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.825+0000 7fb729e00640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7100b4c90 0x7fb7100b71a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009bd0 tx=0x7fb71c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:36.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:36.826+0000 7fb722a00640 1 -- 172.21.15.32:0/527852819 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb724056130 con 0x7fb72c1099e0 2024-08-31T07:11:37.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.040+0000 7fb72b200640 1 -- 172.21.15.32:0/527852819 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fb72c10fbf0 con 0x7fb7100b4c90 2024-08-31T07:11:37.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.041+0000 7fb722a00640 1 -- 172.21.15.32:0/527852819 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fb72c10fbf0 con 0x7fb7100b4c90 2024-08-31T07:11:37.042 INFO:teuthology.orchestra.run.smithi032.stdout:Backend: cephadm 2024-08-31T07:11:37.042 INFO:teuthology.orchestra.run.smithi032.stdout:Available: Yes 2024-08-31T07:11:37.042 INFO:teuthology.orchestra.run.smithi032.stdout:Paused: No 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 -- 172.21.15.32:0/527852819 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7100b4c90 msgr2=0x7fb7100b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7100b4c90 0x7fb7100b71a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009bd0 tx=0x7fb71c005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 -- 172.21.15.32:0/527852819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c1099e0 msgr2=0x7fb72c1a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c1099e0 0x7fb72c1a1b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb72400eb10 tx=0x7fb72400c2d0 comp rx=0 tx=0).stop 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 -- 172.21.15.32:0/527852819 shutdown_connections 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7100b4c90 0x7fb7100b71a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72c1099e0 0x7fb72c1a1b00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb72c109090 0x7fb72c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 --2- 172.21.15.32:0/527852819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb72c103750 0x7fb72c19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 -- 172.21.15.32:0/527852819 >> 172.21.15.32:0/527852819 conn(0x7fb72c0ff1e0 msgr2=0x7fb72c10e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:37.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 -- 172.21.15.32:0/527852819 shutdown_connections 2024-08-31T07:11:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:37.044+0000 7fb72b200640 1 -- 172.21.15.32:0/527852819 wait complete. 2024-08-31T07:11:37.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-31T07:11:38.322 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:38.518 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:38 smithi032 ceph-mon[17252]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:38.519 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:38 smithi032 ceph-mon[17252]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:38 smithi073 ceph-mon[21995]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:38 smithi073 ceph-mon[21995]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:38 smithi084 ceph-mon[22387]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:38 smithi084 ceph-mon[22387]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.026+0000 7fd0b3000640 1 -- 172.21.15.32:0/3447651574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0ac109070 msgr2=0x7fd0ac109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.026+0000 7fd0b3000640 1 --2- 172.21.15.32:0/3447651574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0ac109070 0x7fd0ac109480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd098009900 tx=0x7fd09802a410 comp rx=0 tx=0).stop 2024-08-31T07:11:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.026+0000 7fd0b3000640 1 -- 172.21.15.32:0/3447651574 shutdown_connections 2024-08-31T07:11:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.026+0000 7fd0b3000640 1 --2- 172.21.15.32:0/3447651574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0ac1099c0 0x7fd0ac1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.026+0000 7fd0b3000640 1 --2- 172.21.15.32:0/3447651574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0ac109070 0x7fd0ac109480 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.026+0000 7fd0b3000640 1 --2- 172.21.15.32:0/3447651574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac103730 0x7fd0ac103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.026+0000 7fd0b3000640 1 -- 172.21.15.32:0/3447651574 >> 172.21.15.32:0/3447651574 conn(0x7fd0ac0ff180 msgr2=0x7fd0ac1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.026+0000 7fd0b3000640 1 -- 172.21.15.32:0/3447651574 shutdown_connections 2024-08-31T07:11:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.026+0000 7fd0b3000640 1 -- 172.21.15.32:0/3447651574 wait complete. 2024-08-31T07:11:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.027+0000 7fd0b3000640 1 Processor -- start 2024-08-31T07:11:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.027+0000 7fd0b3000640 1 -- start start 2024-08-31T07:11:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.028+0000 7fd0b3000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0ac103730 0x7fd0ac19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.028+0000 7fd0b3000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac109070 0x7fd0ac19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.028+0000 7fd0b3000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0ac1099c0 0x7fd0ac19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.028+0000 7fd0b3000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac113630 con 0x7fd0ac103730 2024-08-31T07:11:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.028+0000 7fd0b3000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac1134b0 con 0x7fd0ac109070 2024-08-31T07:11:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.028+0000 7fd0b3000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac1137b0 con 0x7fd0ac1099c0 2024-08-31T07:11:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.028+0000 7fd0b1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0ac103730 0x7fd0ac19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.028+0000 7fd0b1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0ac103730 0x7fd0ac19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43890/0 (socket says 172.21.15.32:43890) 2024-08-31T07:11:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.028+0000 7fd0b1c00640 1 -- 172.21.15.32:0/737963148 learned_addr learned my addr 172.21.15.32:0/737963148 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.029+0000 7fd0b1200640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac109070 0x7fd0ac19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.029+0000 7fd0b2600640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0ac1099c0 0x7fd0ac19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.029+0000 7fd0b1c00640 1 -- 172.21.15.32:0/737963148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0ac1099c0 msgr2=0x7fd0ac19f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.029+0000 7fd0b1c00640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0ac1099c0 0x7fd0ac19f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.029+0000 7fd0b1c00640 1 -- 172.21.15.32:0/737963148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac109070 msgr2=0x7fd0ac19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.029+0000 7fd0b1c00640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac109070 0x7fd0ac19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.029+0000 7fd0b1c00640 1 -- 172.21.15.32:0/737963148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ac1a0030 con 0x7fd0ac103730 2024-08-31T07:11:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.029+0000 7fd0b1c00640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0ac103730 0x7fd0ac19a0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000e9f0 tx=0x7fd0a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.029+0000 7fd0aa000640 1 -- 172.21.15.32:0/737963148 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a000cd80 con 0x7fd0ac103730 2024-08-31T07:11:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.030+0000 7fd0b3000640 1 -- 172.21.15.32:0/737963148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ac1a0320 con 0x7fd0ac103730 2024-08-31T07:11:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.030+0000 7fd0aa000640 1 -- 172.21.15.32:0/737963148 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a00108e0 con 0x7fd0ac103730 2024-08-31T07:11:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.030+0000 7fd0aa000640 1 -- 172.21.15.32:0/737963148 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0018980 con 0x7fd0ac103730 2024-08-31T07:11:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.030+0000 7fd0b3000640 1 -- 172.21.15.32:0/737963148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ac1a6a80 con 0x7fd0ac103730 2024-08-31T07:11:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.031+0000 7fd0b3000640 1 -- 172.21.15.32:0/737963148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0ac103b40 con 0x7fd0ac103730 2024-08-31T07:11:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.033+0000 7fd0aa000640 1 -- 172.21.15.32:0/737963148 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fd0a0010430 con 0x7fd0ac103730 2024-08-31T07:11:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.035+0000 7fd0aa000640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0940b4c10 0x7fd0940b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.035+0000 7fd0aa000640 1 -- 172.21.15.32:0/737963148 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fd0a00e5c20 con 0x7fd0ac103730 2024-08-31T07:11:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.035+0000 7fd0b1200640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0940b4c10 0x7fd0940b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.036+0000 7fd0b1200640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0940b4c10 0x7fd0940b7120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd098009230 tx=0x7fd098031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.037+0000 7fd0aa000640 1 -- 172.21.15.32:0/737963148 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd0a00ac4a0 con 0x7fd0ac103730 2024-08-31T07:11:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.238+0000 7fd0b3000640 1 -- 172.21.15.32:0/737963148 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd0ac10b800 con 0x7fd0940b4c10 2024-08-31T07:11:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.251+0000 7fd0aa000640 1 -- 172.21.15.32:0/737963148 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fd0ac10b800 con 0x7fd0940b4c10 2024-08-31T07:11:39.252 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.a smithi032 running (2m) 29s ago 2m 84.2M - 19.3.0-4560-gd19e6118 79f4cde5e369 080c3be7d34f 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.b smithi073 running (2m) 55s ago 2m 94.1M - 19.3.0-4560-gd19e6118 79f4cde5e369 44ccc555afa0 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.c smithi084 running (2m) 55s ago 2m 85.0M - 19.3.0-4560-gd19e6118 79f4cde5e369 4ee2e5b1123e 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.d smithi032 running (2m) 29s ago 2m 80.5M - 19.3.0-4560-gd19e6118 79f4cde5e369 c677467ca775 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.e smithi073 running (2m) 55s ago 2m 84.5M - 19.3.0-4560-gd19e6118 79f4cde5e369 ca5c809c4cb5 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.f smithi084 running (2m) 55s ago 2m 85.1M - 19.3.0-4560-gd19e6118 79f4cde5e369 e0369cd25dcb 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.g smithi032 running (2m) 29s ago 2m 80.3M - 19.3.0-4560-gd19e6118 79f4cde5e369 972d2b5c098a 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.h smithi073 running (2m) 55s ago 2m 84.8M - 19.3.0-4560-gd19e6118 79f4cde5e369 6a87ec5165bc 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.i smithi084 running (2m) 55s ago 2m 87.8M - 19.3.0-4560-gd19e6118 79f4cde5e369 6e86158acbed 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.j smithi032 running (2m) 29s ago 2m 83.2M - 19.3.0-4560-gd19e6118 79f4cde5e369 85284c2b24cc 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.k smithi073 running (2m) 55s ago 2m 84.2M - 19.3.0-4560-gd19e6118 79f4cde5e369 b6c895bcc623 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mds.l smithi084 running (2m) 55s ago 2m 84.5M - 19.3.0-4560-gd19e6118 79f4cde5e369 6508136a91b3 2024-08-31T07:11:39.253 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.x smithi032 *:9283,8765 running (10m) 29s ago 10m 584M - 19.3.0-4560-gd19e6118 79f4cde5e369 74ab0db0eca7 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.y smithi073 *:8443,8765 running (8m) 55s ago 8m 523M - 19.3.0-4560-gd19e6118 79f4cde5e369 9bcb6b071301 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.z smithi084 *:8443,8765 running (8m) 55s ago 8m 523M - 19.3.0-4560-gd19e6118 79f4cde5e369 a92d031f0259 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a smithi032 running (10m) 29s ago 10m 119M 2048M 19.3.0-4560-gd19e6118 79f4cde5e369 b31472b21c64 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b smithi073 running (9m) 55s ago 9m 111M 2048M 19.3.0-4560-gd19e6118 79f4cde5e369 6548fef870e2 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c smithi084 running (9m) 55s ago 9m 112M 2048M 19.3.0-4560-gd19e6118 79f4cde5e369 6c50cf5198a6 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (7m) 29s ago 7m 121M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 52e95a3065b3 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi073 running (7m) 55s ago 7m 114M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 90e5cb4190b8 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi084 running (7m) 55s ago 7m 113M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 c8e82bab121e 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi032 running (6m) 29s ago 6m 109M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 b04ee35543bc 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi073 running (6m) 55s ago 6m 120M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 7eecb0f6f05a 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi084 running (5m) 55s ago 5m 110M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 2ef863cf2744 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 smithi032 running (5m) 29s ago 5m 121M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 09f9fbbd1e9a 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 smithi073 running (4m) 55s ago 4m 111M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 981c6be7c501 2024-08-31T07:11:39.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.8 smithi084 running (4m) 55s ago 4m 112M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 239da4646f7a 2024-08-31T07:11:39.255 INFO:teuthology.orchestra.run.smithi032.stdout:osd.9 smithi032 running (3m) 29s ago 3m 117M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 2aa17f86b696 2024-08-31T07:11:39.255 INFO:teuthology.orchestra.run.smithi032.stdout:osd.10 smithi073 running (3m) 55s ago 3m 121M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 62b1a507befa 2024-08-31T07:11:39.255 INFO:teuthology.orchestra.run.smithi032.stdout:osd.11 smithi084 running (2m) 55s ago 2m 108M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 69cb6cbec855 2024-08-31T07:11:39.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 -- 172.21.15.32:0/737963148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0940b4c10 msgr2=0x7fd0940b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0940b4c10 0x7fd0940b7120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd098009230 tx=0x7fd098031400 comp rx=0 tx=0).stop 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 -- 172.21.15.32:0/737963148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0ac103730 msgr2=0x7fd0ac19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0ac103730 0x7fd0ac19a0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000e9f0 tx=0x7fd0a000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 -- 172.21.15.32:0/737963148 shutdown_connections 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0940b4c10 0x7fd0940b7120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0ac1099c0 0x7fd0ac19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ac109070 0x7fd0ac19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 --2- 172.21.15.32:0/737963148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0ac103730 0x7fd0ac19a0c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 -- 172.21.15.32:0/737963148 >> 172.21.15.32:0/737963148 conn(0x7fd0ac0ff180 msgr2=0x7fd0ac1015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 -- 172.21.15.32:0/737963148 shutdown_connections 2024-08-31T07:11:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:39.254+0000 7fd0b3000640 1 -- 172.21.15.32:0/737963148 wait complete. 2024-08-31T07:11:40.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-31T07:11:40.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:40 smithi032 ceph-mon[17252]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:40.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:40 smithi032 ceph-mon[17252]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:40.442 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:40 smithi073 ceph-mon[21995]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:40 smithi073 ceph-mon[21995]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:40 smithi084 ceph-mon[22387]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:11:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:40 smithi084 ceph-mon[22387]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.054+0000 7f5948c00640 1 -- 172.21.15.32:0/2015081121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5944109f50 msgr2=0x7f594410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.054+0000 7f5948c00640 1 --2- 172.21.15.32:0/2015081121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5944109f50 0x7f594410a360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5934009900 tx=0x7f593402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.055+0000 7f5948c00640 1 -- 172.21.15.32:0/2015081121 shutdown_connections 2024-08-31T07:11:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.055+0000 7f5948c00640 1 --2- 172.21.15.32:0/2015081121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59441101e0 0x7f5944116a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.055+0000 7f5948c00640 1 --2- 172.21.15.32:0/2015081121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594410f890 0x7f594410fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.055+0000 7f5948c00640 1 --2- 172.21.15.32:0/2015081121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5944109f50 0x7f594410a360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.055+0000 7f5948c00640 1 -- 172.21.15.32:0/2015081121 >> 172.21.15.32:0/2015081121 conn(0x7f5944077e80 msgr2=0x7f5944107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.055+0000 7f5948c00640 1 -- 172.21.15.32:0/2015081121 shutdown_connections 2024-08-31T07:11:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.055+0000 7f5948c00640 1 -- 172.21.15.32:0/2015081121 wait complete. 2024-08-31T07:11:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.056+0000 7f5948c00640 1 Processor -- start 2024-08-31T07:11:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.056+0000 7f5948c00640 1 -- start start 2024-08-31T07:11:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5948c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5944109f50 0x7f59441a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5948c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f594410f890 0x7f59441a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5948c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59441101e0 0x7f59441a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5948c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5944119ea0 con 0x7f59441101e0 2024-08-31T07:11:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5948c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5944119d20 con 0x7f5944109f50 2024-08-31T07:11:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5948c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f594411a020 con 0x7f594410f890 2024-08-31T07:11:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5941e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5944109f50 0x7f59441a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5942800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59441101e0 0x7f59441a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5941e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5944109f50 0x7f59441a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34250/0 (socket says 172.21.15.32:34250) 2024-08-31T07:11:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5941e00640 1 -- 172.21.15.32:0/3039801265 learned_addr learned my addr 172.21.15.32:0/3039801265 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.057+0000 7f5941400640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f594410f890 0x7f59441a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f5942800640 1 -- 172.21.15.32:0/3039801265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f594410f890 msgr2=0x7f59441a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f5942800640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f594410f890 0x7f59441a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f5942800640 1 -- 172.21.15.32:0/3039801265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5944109f50 msgr2=0x7f59441a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f5942800640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5944109f50 0x7f59441a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f5942800640 1 -- 172.21.15.32:0/3039801265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59441a67e0 con 0x7f59441101e0 2024-08-31T07:11:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f5942800640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59441101e0 0x7f59441a60b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f593c009fd0 tx=0x7f593c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f593aa00640 1 -- 172.21.15.32:0/3039801265 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c00ce20 con 0x7f59441101e0 2024-08-31T07:11:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f593aa00640 1 -- 172.21.15.32:0/3039801265 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f593c005020 con 0x7f59441101e0 2024-08-31T07:11:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f5948c00640 1 -- 172.21.15.32:0/3039801265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59441a6ad0 con 0x7f59441101e0 2024-08-31T07:11:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f593aa00640 1 -- 172.21.15.32:0/3039801265 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c010690 con 0x7f59441101e0 2024-08-31T07:11:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.058+0000 7f5948c00640 1 -- 172.21.15.32:0/3039801265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59441ad230 con 0x7f59441101e0 2024-08-31T07:11:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.060+0000 7f5948c00640 1 -- 172.21.15.32:0/3039801265 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f594410a360 con 0x7f59441101e0 2024-08-31T07:11:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.062+0000 7f593aa00640 1 -- 172.21.15.32:0/3039801265 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f593c0054d0 con 0x7f59441101e0 2024-08-31T07:11:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.063+0000 7f593aa00640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f59280b4c10 0x7f59280b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.063+0000 7f593aa00640 1 -- 172.21.15.32:0/3039801265 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f593c0e5a60 con 0x7f59441101e0 2024-08-31T07:11:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.064+0000 7f5941e00640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f59280b4c10 0x7f59280b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.064+0000 7f5941e00640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f59280b4c10 0x7f59280b7120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5934031440 tx=0x7f59340058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.065+0000 7f593aa00640 1 -- 172.21.15.32:0/3039801265 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f593c0ac930 con 0x7f59441101e0 2024-08-31T07:11:41.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.271+0000 7f5948c00640 1 -- 172.21.15.32:0/3039801265 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f59440635a0 con 0x7f59280b4c10 2024-08-31T07:11:41.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.278+0000 7f593aa00640 1 -- 172.21.15.32:0/3039801265 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f59440635a0 con 0x7f59280b4c10 2024-08-31T07:11:41.279 INFO:teuthology.orchestra.run.smithi032.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-31T07:11:41.279 INFO:teuthology.orchestra.run.smithi032.stdout:mds.all 12/12 57s ago 2m smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi084=c;smithi084=f;smithi084=i;smithi084=l;count:12 2024-08-31T07:11:41.279 INFO:teuthology.orchestra.run.smithi032.stdout:mgr 3/3 57s ago 8m smithi032=x;smithi073=y;smithi084=z;count:3 2024-08-31T07:11:41.279 INFO:teuthology.orchestra.run.smithi032.stdout:mon 3/3 57s ago 9m smithi032:172.21.15.32=a;smithi073:172.21.15.73=b;smithi084:172.21.15.84=c;count:3 2024-08-31T07:11:41.279 INFO:teuthology.orchestra.run.smithi032.stdout:osd 12 57s ago - 2024-08-31T07:11:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 -- 172.21.15.32:0/3039801265 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f59280b4c10 msgr2=0x7f59280b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f59280b4c10 0x7f59280b7120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5934031440 tx=0x7f59340058d0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 -- 172.21.15.32:0/3039801265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59441101e0 msgr2=0x7f59441a60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59441101e0 0x7f59441a60b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f593c009fd0 tx=0x7f593c00ef10 comp rx=0 tx=0).stop 2024-08-31T07:11:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 -- 172.21.15.32:0/3039801265 shutdown_connections 2024-08-31T07:11:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f59280b4c10 0x7f59280b7120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59441101e0 0x7f59441a60b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f594410f890 0x7f59441a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 --2- 172.21.15.32:0/3039801265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5944109f50 0x7f59441a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 -- 172.21.15.32:0/3039801265 >> 172.21.15.32:0/3039801265 conn(0x7f5944077e80 msgr2=0x7f59440785b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 -- 172.21.15.32:0/3039801265 shutdown_connections 2024-08-31T07:11:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:41.280+0000 7f5948c00640 1 -- 172.21.15.32:0/3039801265 wait complete. 2024-08-31T07:11:42.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-31T07:11:42.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:42 smithi032 ceph-mon[17252]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:42.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:42 smithi032 ceph-mon[17252]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:42.464 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:42 smithi073 ceph-mon[21995]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:42 smithi073 ceph-mon[21995]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:42 smithi084 ceph-mon[22387]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:42 smithi084 ceph-mon[22387]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.136+0000 7fe7c5600640 1 -- 172.21.15.32:0/1347704369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7c0105b20 msgr2=0x7fe7c0105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.136+0000 7fe7c5600640 1 --2- 172.21.15.32:0/1347704369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7c0105b20 0x7fe7c0105f30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009900 tx=0x7fe7b002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.137+0000 7fe7c5600640 1 -- 172.21.15.32:0/1347704369 shutdown_connections 2024-08-31T07:11:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.137+0000 7fe7c5600640 1 --2- 172.21.15.32:0/1347704369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c010bdb0 0x7fe7c01125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.137+0000 7fe7c5600640 1 --2- 172.21.15.32:0/1347704369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c010b460 0x7fe7c010b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.137+0000 7fe7c5600640 1 --2- 172.21.15.32:0/1347704369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7c0105b20 0x7fe7c0105f30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.137+0000 7fe7c5600640 1 -- 172.21.15.32:0/1347704369 >> 172.21.15.32:0/1347704369 conn(0x7fe7c01015b0 msgr2=0x7fe7c0103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.137+0000 7fe7c5600640 1 -- 172.21.15.32:0/1347704369 shutdown_connections 2024-08-31T07:11:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.137+0000 7fe7c5600640 1 -- 172.21.15.32:0/1347704369 wait complete. 2024-08-31T07:11:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.138+0000 7fe7c5600640 1 Processor -- start 2024-08-31T07:11:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.138+0000 7fe7c5600640 1 -- start start 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.138+0000 7fe7c5600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c0105b20 0x7fe7c019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.138+0000 7fe7c5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7c010b460 0x7fe7c019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.138+0000 7fe7c5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c010bdb0 0x7fe7c01a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.138+0000 7fe7c5600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0115a70 con 0x7fe7c010b460 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.138+0000 7fe7c5600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01158f0 con 0x7fe7c010bdb0 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.138+0000 7fe7c5600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0115bf0 con 0x7fe7c0105b20 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7be000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7c010b460 0x7fe7c019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7be000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7c010b460 0x7fe7c019c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43932/0 (socket says 172.21.15.32:43932) 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7be000640 1 -- 172.21.15.32:0/1396380644 learned_addr learned my addr 172.21.15.32:0/1396380644 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7bea00640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c0105b20 0x7fe7c019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7bf400640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c010bdb0 0x7fe7c01a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7be000640 1 -- 172.21.15.32:0/1396380644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c0105b20 msgr2=0x7fe7c019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7be000640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c0105b20 0x7fe7c019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7be000640 1 -- 172.21.15.32:0/1396380644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c010bdb0 msgr2=0x7fe7c01a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7be000640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c010bdb0 0x7fe7c01a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7be000640 1 -- 172.21.15.32:0/1396380644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c01a23b0 con 0x7fe7c010b460 2024-08-31T07:11:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7bea00640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c0105b20 0x7fe7c019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.139+0000 7fe7be000640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7c010b460 0x7fe7c019c9e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00e9f0 tx=0x7fe7ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.140+0000 7fe7b7400640 1 -- 172.21.15.32:0/1396380644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac00ccc0 con 0x7fe7c010b460 2024-08-31T07:11:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.140+0000 7fe7b7400640 1 -- 172.21.15.32:0/1396380644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ac0108e0 con 0x7fe7c010b460 2024-08-31T07:11:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.140+0000 7fe7b7400640 1 -- 172.21.15.32:0/1396380644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac018860 con 0x7fe7c010b460 2024-08-31T07:11:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.140+0000 7fe7c5600640 1 -- 172.21.15.32:0/1396380644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c01a26a0 con 0x7fe7c010b460 2024-08-31T07:11:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.140+0000 7fe7c5600640 1 -- 172.21.15.32:0/1396380644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c01a8e00 con 0x7fe7c010b460 2024-08-31T07:11:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.141+0000 7fe7c5600640 1 -- 172.21.15.32:0/1396380644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c0105f30 con 0x7fe7c010b460 2024-08-31T07:11:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.143+0000 7fe7b7400640 1 -- 172.21.15.32:0/1396380644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fe7ac010430 con 0x7fe7c010b460 2024-08-31T07:11:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.145+0000 7fe7b7400640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7a40b4c90 0x7fe7a40b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.145+0000 7fe7b7400640 1 -- 172.21.15.32:0/1396380644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe7ac0e5b50 con 0x7fe7c010b460 2024-08-31T07:11:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.145+0000 7fe7bea00640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7a40b4c90 0x7fe7a40b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.146+0000 7fe7bea00640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7a40b4c90 0x7fe7a40b71a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0031440 tx=0x7fe7b0032ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.147+0000 7fe7b7400640 1 -- 172.21.15.32:0/1396380644 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe7ac0ac3d0 con 0x7fe7c010b460 2024-08-31T07:11:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.352+0000 7fe7c5600640 1 -- 172.21.15.32:0/1396380644 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fe7c00635a0 con 0x7fe7a40b4c90 2024-08-31T07:11:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.354+0000 7fe7b7400640 1 -- 172.21.15.32:0/1396380644 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7fe7c00635a0 con 0x7fe7a40b4c90 2024-08-31T07:11:43.355 INFO:teuthology.orchestra.run.smithi032.stdout:HOST ADDR LABELS STATUS 2024-08-31T07:11:43.355 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 172.21.15.32 2024-08-31T07:11:43.355 INFO:teuthology.orchestra.run.smithi032.stdout:smithi073 172.21.15.73 2024-08-31T07:11:43.356 INFO:teuthology.orchestra.run.smithi032.stdout:smithi084 172.21.15.84 2024-08-31T07:11:43.356 INFO:teuthology.orchestra.run.smithi032.stdout:3 hosts in cluster 2024-08-31T07:11:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 -- 172.21.15.32:0/1396380644 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7a40b4c90 msgr2=0x7fe7a40b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7a40b4c90 0x7fe7a40b71a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0031440 tx=0x7fe7b0032ed0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 -- 172.21.15.32:0/1396380644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7c010b460 msgr2=0x7fe7c019c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7c010b460 0x7fe7c019c9e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00e9f0 tx=0x7fe7ac00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 -- 172.21.15.32:0/1396380644 shutdown_connections 2024-08-31T07:11:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7a40b4c90 0x7fe7a40b71a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7c010bdb0 0x7fe7c01a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7c010b460 0x7fe7c019c9e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 --2- 172.21.15.32:0/1396380644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c0105b20 0x7fe7c019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 -- 172.21.15.32:0/1396380644 >> 172.21.15.32:0/1396380644 conn(0x7fe7c01015b0 msgr2=0x7fe7c01039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 -- 172.21.15.32:0/1396380644 shutdown_connections 2024-08-31T07:11:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:43.357+0000 7fe7c5600640 1 -- 172.21.15.32:0/1396380644 wait complete. 2024-08-31T07:11:44.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-31T07:11:44.577 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:44.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:44 smithi032 ceph-mon[17252]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:44.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:44 smithi032 ceph-mon[17252]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:44 smithi073 ceph-mon[21995]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:44 smithi073 ceph-mon[21995]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:44 smithi084 ceph-mon[22387]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:11:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:44 smithi084 ceph-mon[22387]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:45.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.285+0000 7f310fe00640 1 -- 172.21.15.32:0/3200768937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3110105960 msgr2=0x7f3110105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.285+0000 7f310fe00640 1 --2- 172.21.15.32:0/3200768937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3110105960 0x7f3110105d70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3100009900 tx=0x7f310002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.285+0000 7f310fe00640 1 -- 172.21.15.32:0/3200768937 shutdown_connections 2024-08-31T07:11:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.285+0000 7f310fe00640 1 --2- 172.21.15.32:0/3200768937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f311010bbf0 0x7f3110112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.285+0000 7f310fe00640 1 --2- 172.21.15.32:0/3200768937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f311010b2a0 0x7f311010b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.285+0000 7f310fe00640 1 --2- 172.21.15.32:0/3200768937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3110105960 0x7f3110105d70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.285+0000 7f310fe00640 1 -- 172.21.15.32:0/3200768937 >> 172.21.15.32:0/3200768937 conn(0x7f31101013d0 msgr2=0x7f3110103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.285+0000 7f310fe00640 1 -- 172.21.15.32:0/3200768937 shutdown_connections 2024-08-31T07:11:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.285+0000 7f310fe00640 1 -- 172.21.15.32:0/3200768937 wait complete. 2024-08-31T07:11:45.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.286+0000 7f310fe00640 1 Processor -- start 2024-08-31T07:11:45.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.286+0000 7f310fe00640 1 -- start start 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110105960 0x7f311019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311010b2a0 0x7f311019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f311010bbf0 0x7f31101a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f31101158b0 con 0x7f311010b2a0 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3110115730 con 0x7f311010bbf0 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3110115a30 con 0x7f3110105960 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311010b2a0 0x7f311019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311010b2a0 0x7f311019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43962/0 (socket says 172.21.15.32:43962) 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310e000640 1 -- 172.21.15.32:0/1244808607 learned_addr learned my addr 172.21.15.32:0/1244808607 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310ea00640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110105960 0x7f311019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310f400640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f311010bbf0 0x7f31101a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310e000640 1 -- 172.21.15.32:0/1244808607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110105960 msgr2=0x7f311019c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:45.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.287+0000 7f310e000640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110105960 0x7f311019c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.288+0000 7f310e000640 1 -- 172.21.15.32:0/1244808607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f311010bbf0 msgr2=0x7f31101a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:45.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.288+0000 7f310e000640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f311010bbf0 0x7f31101a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.288+0000 7f310e000640 1 -- 172.21.15.32:0/1244808607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31101a21d0 con 0x7f311010b2a0 2024-08-31T07:11:45.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.288+0000 7f310ea00640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110105960 0x7f311019c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:45.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.288+0000 7f310e000640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311010b2a0 0x7f311019c800 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00e9f0 tx=0x7f30fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.288+0000 7f3107400640 1 -- 172.21.15.32:0/1244808607 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc00ccc0 con 0x7f311010b2a0 2024-08-31T07:11:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.288+0000 7f3107400640 1 -- 172.21.15.32:0/1244808607 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30fc0108e0 con 0x7f311010b2a0 2024-08-31T07:11:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.288+0000 7f310fe00640 1 -- 172.21.15.32:0/1244808607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31101a24c0 con 0x7f311010b2a0 2024-08-31T07:11:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.288+0000 7f3107400640 1 -- 172.21.15.32:0/1244808607 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc018860 con 0x7f311010b2a0 2024-08-31T07:11:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.288+0000 7f310fe00640 1 -- 172.21.15.32:0/1244808607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31101a8c20 con 0x7f311010b2a0 2024-08-31T07:11:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.290+0000 7f310fe00640 1 -- 172.21.15.32:0/1244808607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3110105d70 con 0x7f311010b2a0 2024-08-31T07:11:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.294+0000 7f3107400640 1 -- 172.21.15.32:0/1244808607 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f30fc010430 con 0x7f311010b2a0 2024-08-31T07:11:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.295+0000 7f3107400640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30f40b4c10 0x7f30f40b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.295+0000 7f3107400640 1 -- 172.21.15.32:0/1244808607 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f30fc0e5bd0 con 0x7f311010b2a0 2024-08-31T07:11:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.296+0000 7f310ea00640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30f40b4c10 0x7f30f40b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.296+0000 7f310ea00640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30f40b4c10 0x7f30f40b7120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3100031440 tx=0x7f3100032ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.298+0000 7f3107400640 1 -- 172.21.15.32:0/1244808607 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f30fc0ac450 con 0x7f311010b2a0 2024-08-31T07:11:45.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.502+0000 7f310fe00640 1 -- 172.21.15.32:0/1244808607 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f31100635a0 con 0x7f30f40b4c10 2024-08-31T07:11:45.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.504+0000 7f3107400640 1 -- 172.21.15.32:0/1244808607 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f31100635a0 con 0x7f30f40b4c10 2024-08-31T07:11:45.505 INFO:teuthology.orchestra.run.smithi032.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-31T07:11:45.505 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402DN480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-08-31T07:11:45.505 INFO:teuthology.orchestra.run.smithi032.stdout:smithi073 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040115400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-31T07:11:45.505 INFO:teuthology.orchestra.run.smithi032.stdout:smithi084 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WF400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.506+0000 7f310fe00640 1 -- 172.21.15.32:0/1244808607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30f40b4c10 msgr2=0x7f30f40b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30f40b4c10 0x7f30f40b7120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3100031440 tx=0x7f3100032ed0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 -- 172.21.15.32:0/1244808607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311010b2a0 msgr2=0x7f311019c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311010b2a0 0x7f311019c800 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00e9f0 tx=0x7f30fc00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 -- 172.21.15.32:0/1244808607 shutdown_connections 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30f40b4c10 0x7f30f40b7120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f311010bbf0 0x7f31101a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f311010b2a0 0x7f311019c800 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 --2- 172.21.15.32:0/1244808607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3110105960 0x7f311019c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 -- 172.21.15.32:0/1244808607 >> 172.21.15.32:0/1244808607 conn(0x7f31101013d0 msgr2=0x7f31101037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:45.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 -- 172.21.15.32:0/1244808607 shutdown_connections 2024-08-31T07:11:45.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:45.507+0000 7f310fe00640 1 -- 172.21.15.32:0/1244808607 wait complete. 2024-08-31T07:11:46.285 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-31T07:11:46.294 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-08-31T07:11:46.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-31T07:11:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:46 smithi032 ceph-mon[17252]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:46 smithi032 ceph-mon[17252]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:46 smithi073 ceph-mon[21995]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:46 smithi073 ceph-mon[21995]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:46 smithi084 ceph-mon[22387]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:11:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:46 smithi084 ceph-mon[22387]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:11:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.367+0000 7fb781200640 1 -- 172.21.15.32:0/439408954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77c103750 msgr2=0x7fb77c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.367+0000 7fb781200640 1 --2- 172.21.15.32:0/439408954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77c103750 0x7fb77c103b60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009900 tx=0x7fb76c02a410 comp rx=0 tx=0).stop 2024-08-31T07:11:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.368+0000 7fb781200640 1 -- 172.21.15.32:0/439408954 shutdown_connections 2024-08-31T07:11:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.368+0000 7fb781200640 1 --2- 172.21.15.32:0/439408954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c1099e0 0x7fb77c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.368+0000 7fb781200640 1 --2- 172.21.15.32:0/439408954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb77c109090 0x7fb77c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.368+0000 7fb781200640 1 --2- 172.21.15.32:0/439408954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77c103750 0x7fb77c103b60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.368+0000 7fb781200640 1 -- 172.21.15.32:0/439408954 >> 172.21.15.32:0/439408954 conn(0x7fb77c0ff1e0 msgr2=0x7fb77c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.368+0000 7fb781200640 1 -- 172.21.15.32:0/439408954 shutdown_connections 2024-08-31T07:11:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.368+0000 7fb781200640 1 -- 172.21.15.32:0/439408954 wait complete. 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.369+0000 7fb781200640 1 Processor -- start 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.369+0000 7fb781200640 1 -- start start 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.369+0000 7fb781200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb77c103750 0x7fb77c19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.369+0000 7fb781200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77c109090 0x7fb77c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.369+0000 7fb781200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c1099e0 0x7fb77c1a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.369+0000 7fb781200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1136b0 con 0x7fb77c1099e0 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.369+0000 7fb781200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c113530 con 0x7fb77c103750 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.369+0000 7fb781200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c113830 con 0x7fb77c109090 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb77c103750 0x7fb77c19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb77c103750 0x7fb77c19c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53470/0 (socket says 172.21.15.32:53470) 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77b400640 1 -- 172.21.15.32:0/1565345085 learned_addr learned my addr 172.21.15.32:0/1565345085 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77aa00640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77c109090 0x7fb77c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77be00640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c1099e0 0x7fb77c1a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77b400640 1 -- 172.21.15.32:0/1565345085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77c109090 msgr2=0x7fb77c19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77b400640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77c109090 0x7fb77c19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77b400640 1 -- 172.21.15.32:0/1565345085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c1099e0 msgr2=0x7fb77c1a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77b400640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c1099e0 0x7fb77c1a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77b400640 1 -- 172.21.15.32:0/1565345085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c1a2290 con 0x7fb77c103750 2024-08-31T07:11:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.370+0000 7fb77b400640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb77c103750 0x7fb77c19c320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb76c005940 tx=0x7fb76c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.371+0000 7fb773e00640 1 -- 172.21.15.32:0/1565345085 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c03f030 con 0x7fb77c103750 2024-08-31T07:11:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.371+0000 7fb773e00640 1 -- 172.21.15.32:0/1565345085 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb76c0044c0 con 0x7fb77c103750 2024-08-31T07:11:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.371+0000 7fb781200640 1 -- 172.21.15.32:0/1565345085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c1a2520 con 0x7fb77c103750 2024-08-31T07:11:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.371+0000 7fb773e00640 1 -- 172.21.15.32:0/1565345085 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c035070 con 0x7fb77c103750 2024-08-31T07:11:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.371+0000 7fb781200640 1 -- 172.21.15.32:0/1565345085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c1a8ce0 con 0x7fb77c103750 2024-08-31T07:11:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.372+0000 7fb781200640 1 -- 172.21.15.32:0/1565345085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77c103b60 con 0x7fb77c103750 2024-08-31T07:11:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.374+0000 7fb773e00640 1 -- 172.21.15.32:0/1565345085 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7fb76c042030 con 0x7fb77c103750 2024-08-31T07:11:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.375+0000 7fb773e00640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7600b4b80 0x7fb7600b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.376+0000 7fb773e00640 1 -- 172.21.15.32:0/1565345085 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb76c105600 con 0x7fb77c103750 2024-08-31T07:11:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.376+0000 7fb77aa00640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7600b4b80 0x7fb7600b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.376+0000 7fb77aa00640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7600b4b80 0x7fb7600b7090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb77c1106c0 tx=0x7fb768005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.379+0000 7fb773e00640 1 -- 172.21.15.32:0/1565345085 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb76c0cbfe0 con 0x7fb77c103750 2024-08-31T07:11:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.626+0000 7fb781200640 1 -- 172.21.15.32:0/1565345085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb77c1a8f40 con 0x7fb77c103750 2024-08-31T07:11:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.627+0000 7fb773e00640 1 -- 172.21.15.32:0/1565345085 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3119 (secure 0 0 0) 0x7fb76c0d07a0 con 0x7fb77c103750 2024-08-31T07:11:47.628 INFO:teuthology.orchestra.run.smithi032.stdout:e28 2024-08-31T07:11:47.628 INFO:teuthology.orchestra.run.smithi032.stdout:btime 2024-08-31T07:10:05:878594+0000 2024-08-31T07:11:47.628 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 28 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-08-31T07:09:48.797887+0000 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-08-31T07:10:04.962915+0000 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-08-31T07:11:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 300 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 0 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 5 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1,2,3,4 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=24734,1=24718,2=24694,3=24692,4=24676} 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-08-31T07:11:47.630 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data disabled 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 0 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 24734 members: 24676,24692,24718,24694,24734 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b{0:24734} state up:active seq 5 addr [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.f{0:24670} state up:standby-replay seq 1 addr [v2:172.21.15.84:6834/204703622,v1:172.21.15.84:6835/204703622] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a{1:24718} state up:active seq 8 addr [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.e{1:24668} state up:standby-replay seq 1 addr [v2:172.21.15.73:6834/1927490574,v1:172.21.15.73:6835/1927490574] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.i{2:24694} state up:active seq 10 addr [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.j{2:24644} state up:standby-replay seq 1 addr [v2:172.21.15.32:6834/1250889023,v1:172.21.15.32:6835/1250889023] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.h{3:24692} state up:active seq 11 addr [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.631 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.k{3:24632} state up:standby-replay seq 1 addr [v2:172.21.15.73:6832/2003936745,v1:172.21.15.73:6833/2003936745] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.632 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c{4:24676} state up:active seq 12 addr [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.632 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.l{4:24625} state up:standby-replay seq 1 addr [v2:172.21.15.84:6832/2138862695,v1:172.21.15.84:6833/2138862695] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.632 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:47.632 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:47.632 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-08-31T07:11:47.632 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:47.632 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d{-1:14652} state up:standby seq 1 addr [v2:172.21.15.32:6836/2918070851,v1:172.21.15.32:6837/2918070851] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.632 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.g{-1:14697} state up:standby seq 1 addr [v2:172.21.15.32:6838/1062605926,v1:172.21.15.32:6839/1062605926] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T07:11:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2024-08-31T07:11:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.629+0000 7fb781200640 1 -- 172.21.15.32:0/1565345085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7600b4b80 msgr2=0x7fb7600b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.629+0000 7fb781200640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7600b4b80 0x7fb7600b7090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb77c1106c0 tx=0x7fb768005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.629+0000 7fb781200640 1 -- 172.21.15.32:0/1565345085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb77c103750 msgr2=0x7fb77c19c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.629+0000 7fb781200640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb77c103750 0x7fb77c19c320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb76c005940 tx=0x7fb76c0040c0 comp rx=0 tx=0).stop 2024-08-31T07:11:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.629+0000 7fb781200640 1 -- 172.21.15.32:0/1565345085 shutdown_connections 2024-08-31T07:11:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.629+0000 7fb781200640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7600b4b80 0x7fb7600b7090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.629+0000 7fb781200640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c1099e0 0x7fb77c1a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:47.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.629+0000 7fb781200640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77c109090 0x7fb77c19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:47.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.629+0000 7fb781200640 1 --2- 172.21.15.32:0/1565345085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb77c103750 0x7fb77c19c320 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:47.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.629+0000 7fb781200640 1 -- 172.21.15.32:0/1565345085 >> 172.21.15.32:0/1565345085 conn(0x7fb77c0ff1e0 msgr2=0x7fb77c10e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:47.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.630+0000 7fb781200640 1 -- 172.21.15.32:0/1565345085 shutdown_connections 2024-08-31T07:11:47.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:47.630+0000 7fb781200640 1 -- 172.21.15.32:0/1565345085 wait complete. 2024-08-31T07:11:48.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:48 smithi032 ceph-mon[17252]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-31T07:11:48.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:48 smithi032 ceph-mon[17252]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-08-31T07:11:48.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:48 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1565345085' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-31T07:11:48.483 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-31T07:11:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:48 smithi073 ceph-mon[21995]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-31T07:11:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:48 smithi073 ceph-mon[21995]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-08-31T07:11:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:48 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1565345085' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-31T07:11:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:48 smithi084 ceph-mon[22387]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-31T07:11:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:48 smithi084 ceph-mon[22387]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-08-31T07:11:48.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:48 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1565345085' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-31T07:11:48.866 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.662+0000 7f32f2000640 1 -- 172.21.15.32:0/1087862215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ec10fe80 msgr2=0x7f32ec1122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.662+0000 7f32f2000640 1 --2- 172.21.15.32:0/1087862215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ec10fe80 0x7f32ec1122c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009ca0 tx=0x7f32e402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:11:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.662+0000 7f32f2000640 1 -- 172.21.15.32:0/1087862215 shutdown_connections 2024-08-31T07:11:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.662+0000 7f32f2000640 1 --2- 172.21.15.32:0/1087862215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ec10fe80 0x7f32ec1122c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.662+0000 7f32f2000640 1 --2- 172.21.15.32:0/1087862215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ec075ea0 0x7f32ec10f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.662+0000 7f32f2000640 1 --2- 172.21.15.32:0/1087862215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32ec075550 0x7f32ec075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.662+0000 7f32f2000640 1 -- 172.21.15.32:0/1087862215 >> 172.21.15.32:0/1087862215 conn(0x7f32ec0ff370 msgr2=0x7f32ec1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.662+0000 7f32f2000640 1 -- 172.21.15.32:0/1087862215 shutdown_connections 2024-08-31T07:11:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.662+0000 7f32f2000640 1 -- 172.21.15.32:0/1087862215 wait complete. 2024-08-31T07:11:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.663+0000 7f32f2000640 1 Processor -- start 2024-08-31T07:11:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.663+0000 7f32f2000640 1 -- start start 2024-08-31T07:11:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32f2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32ec075550 0x7f32ec19c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32f2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ec075ea0 0x7f32ec19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32f2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ec10fe80 0x7f32ec1a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32f2000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1159f0 con 0x7f32ec075ea0 2024-08-31T07:11:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32f2000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec115870 con 0x7f32ec075550 2024-08-31T07:11:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32f2000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec115b70 con 0x7f32ec10fe80 2024-08-31T07:11:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32e3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ec075ea0 0x7f32ec19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32e3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ec075ea0 0x7f32ec19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33242/0 (socket says 172.21.15.32:33242) 2024-08-31T07:11:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32e3e00640 1 -- 172.21.15.32:0/377373312 learned_addr learned my addr 172.21.15.32:0/377373312 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32ebe00640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ec10fe80 0x7f32ec1a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.664+0000 7f32eb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32ec075550 0x7f32ec19c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.665+0000 7f32e3e00640 1 -- 172.21.15.32:0/377373312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ec10fe80 msgr2=0x7f32ec1a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.665+0000 7f32e3e00640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ec10fe80 0x7f32ec1a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.665+0000 7f32e3e00640 1 -- 172.21.15.32:0/377373312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32ec075550 msgr2=0x7f32ec19c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.665+0000 7f32e3e00640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32ec075550 0x7f32ec19c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.665+0000 7f32e3e00640 1 -- 172.21.15.32:0/377373312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec1a22f0 con 0x7f32ec075ea0 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.665+0000 7f32e3e00640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ec075ea0 0x7f32ec19c920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32d400e9c0 tx=0x7f32d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.665+0000 7f32e8c00640 1 -- 172.21.15.32:0/377373312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d400cd80 con 0x7f32ec075ea0 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.666+0000 7f32e8c00640 1 -- 172.21.15.32:0/377373312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32d40035f0 con 0x7f32ec075ea0 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.666+0000 7f32f2000640 1 -- 172.21.15.32:0/377373312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ec1a25e0 con 0x7f32ec075ea0 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.666+0000 7f32e8c00640 1 -- 172.21.15.32:0/377373312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4010690 con 0x7f32ec075ea0 2024-08-31T07:11:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.666+0000 7f32f2000640 1 -- 172.21.15.32:0/377373312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ec1a8d40 con 0x7f32ec075ea0 2024-08-31T07:11:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.667+0000 7f32f2000640 1 -- 172.21.15.32:0/377373312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ec075960 con 0x7f32ec075ea0 2024-08-31T07:11:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.669+0000 7f32e8c00640 1 -- 172.21.15.32:0/377373312 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f32d40037a0 con 0x7f32ec075ea0 2024-08-31T07:11:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.670+0000 7f32e8c00640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32d00b4c00 0x7f32d00b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.671+0000 7f32e8c00640 1 -- 172.21.15.32:0/377373312 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f32d40e55f0 con 0x7f32ec075ea0 2024-08-31T07:11:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.671+0000 7f32eb400640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32d00b4c00 0x7f32d00b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:49.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.671+0000 7f32eb400640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32d00b4c00 0x7f32d00b7110 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009bd0 tx=0x7f32dc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.674+0000 7f32e8c00640 1 -- 172.21.15.32:0/377373312 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f32d40ac620 con 0x7f32ec075ea0 2024-08-31T07:11:49.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.874+0000 7f32f2000640 1 -- 172.21.15.32:0/377373312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f32ec1a90d0 con 0x7f32ec075ea0 2024-08-31T07:11:49.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.875+0000 7f32e8c00640 1 -- 172.21.15.32:0/377373312 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v80) ==== 56+0+5619 (secure 0 0 0) 0x7f32d40abf20 con 0x7f32ec075ea0 2024-08-31T07:11:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 80 2024-08-31T07:11:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:fsid 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:11:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-08-31T07:00:40.646815+0000 2024-08-31T07:11:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-08-31T07:10:46.011475+0000 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:crush_version 26 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:full_ratio 0.95 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:backfillfull_ratio 0.9 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:nearfull_ratio 0.85 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:require_min_compat_client luminous 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:min_compat_client luminous 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:require_osd_release squid 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:stretch_mode_enabled false 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:max_osd 12 2024-08-31T07:11:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 up in weight 1 up_from 9 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6802/1563132016,v1:172.21.15.32:6803/1563132016] [v2:172.21.15.32:6804/1563132016,v1:172.21.15.32:6805/1563132016] exists,up 91a7f222-d5e9-423d-ba87-fa1278455da4 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 up in weight 1 up_from 14 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6800/4226535850,v1:172.21.15.73:6801/4226535850] [v2:172.21.15.73:6802/4226535850,v1:172.21.15.73:6803/4226535850] exists,up 0035b2ca-ad88-42e8-a46b-da03efcc7c04 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 up in weight 1 up_from 19 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6800/904624859,v1:172.21.15.84:6801/904624859] [v2:172.21.15.84:6802/904624859,v1:172.21.15.84:6803/904624859] exists,up 395e3fdd-1a48-43cc-afad-7af3af791e14 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 up in weight 1 up_from 27 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6810/1969453562,v1:172.21.15.32:6811/1969453562] [v2:172.21.15.32:6812/1969453562,v1:172.21.15.32:6813/1969453562] exists,up bfaeb828-d33c-4791-95ec-79d1669b7316 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 up in weight 1 up_from 33 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6808/3774619077,v1:172.21.15.73:6809/3774619077] [v2:172.21.15.73:6810/3774619077,v1:172.21.15.73:6811/3774619077] exists,up e286ae6e-6d64-452c-b26a-d2aebbfa8742 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 up in weight 1 up_from 38 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6808/3489144690,v1:172.21.15.84:6809/3489144690] [v2:172.21.15.84:6810/3489144690,v1:172.21.15.84:6811/3489144690] exists,up 98d5b70c-95dd-4bbb-b715-9872c4a4102e 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 up in weight 1 up_from 44 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6818/1095521887,v1:172.21.15.32:6819/1095521887] [v2:172.21.15.32:6820/1095521887,v1:172.21.15.32:6821/1095521887] exists,up a5e79d62-80ff-4e1b-b031-2ce622f81c33 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 up in weight 1 up_from 49 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6816/2091730330,v1:172.21.15.73:6817/2091730330] [v2:172.21.15.73:6818/2091730330,v1:172.21.15.73:6819/2091730330] exists,up afca1d86-d56c-498f-8086-0114cd88e0c4 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.8 up in weight 1 up_from 54 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6816/2957423402,v1:172.21.15.84:6817/2957423402] [v2:172.21.15.84:6818/2957423402,v1:172.21.15.84:6819/2957423402] exists,up c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.9 up in weight 1 up_from 60 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6826/2828083479,v1:172.21.15.32:6827/2828083479] [v2:172.21.15.32:6828/2828083479,v1:172.21.15.32:6829/2828083479] exists,up 1d428503-9203-4ddd-b618-114e17fe9f3a 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.10 up in weight 1 up_from 65 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6824/2996715410,v1:172.21.15.73:6825/2996715410] [v2:172.21.15.73:6826/2996715410,v1:172.21.15.73:6827/2996715410] exists,up 3e92a61c-4465-44db-813e-d5b731ab2947 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.11 up in weight 1 up_from 71 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6824/3154011828,v1:172.21.15.84:6825/3154011828] [v2:172.21.15.84:6826/3154011828,v1:172.21.15.84:6827/3154011828] exists,up 2163952e-3884-434b-8d0b-d489ea25c1cf 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.13 [2,11] 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.1c [8,11] 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.12 [3,9] 2024-08-31T07:11:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.1a [5,6] 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.21 [4,8] 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.24 [4,9] 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.36 [4,9] 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/973985469 expires 2024-09-01T07:02:44.661783+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/1291326212 expires 2024-09-01T07:01:06.343703+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/717875748 expires 2024-09-01T07:01:06.343703+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/4001745995 expires 2024-09-01T07:02:44.661783+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/717875748 expires 2024-09-01T07:01:06.343703+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/2312402120 expires 2024-09-01T07:01:06.343703+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/2898888137 expires 2024-09-01T07:01:06.343703+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/115361289 expires 2024-09-01T07:01:35.585062+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/2767703591 expires 2024-09-01T07:02:44.661783+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3767202779 expires 2024-09-01T07:01:35.585062+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/376966273 expires 2024-09-01T07:02:44.661783+0000 2024-08-31T07:11:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/4009368030 expires 2024-09-01T07:01:35.585062+0000 2024-08-31T07:11:49.880 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3157328338 expires 2024-09-01T07:01:35.585062+0000 2024-08-31T07:11:49.880 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/4009368030 expires 2024-09-01T07:01:35.585062+0000 2024-08-31T07:11:49.880 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/4001745995 expires 2024-09-01T07:02:44.661783+0000 2024-08-31T07:11:49.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 -- 172.21.15.32:0/377373312 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32d00b4c00 msgr2=0x7f32d00b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:49.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32d00b4c00 0x7f32d00b7110 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009bd0 tx=0x7f32dc005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:49.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 -- 172.21.15.32:0/377373312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ec075ea0 msgr2=0x7f32ec19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:49.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ec075ea0 0x7f32ec19c920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32d400e9c0 tx=0x7f32d400ee80 comp rx=0 tx=0).stop 2024-08-31T07:11:49.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 -- 172.21.15.32:0/377373312 shutdown_connections 2024-08-31T07:11:49.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32d00b4c00 0x7f32d00b7110 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:49.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ec10fe80 0x7f32ec1a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:49.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ec075ea0 0x7f32ec19c920 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:49.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 --2- 172.21.15.32:0/377373312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32ec075550 0x7f32ec19c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:49.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 -- 172.21.15.32:0/377373312 >> 172.21.15.32:0/377373312 conn(0x7f32ec0ff370 msgr2=0x7f32ec109990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:49.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 -- 172.21.15.32:0/377373312 shutdown_connections 2024-08-31T07:11:49.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:49.878+0000 7f32f2000640 1 -- 172.21.15.32:0/377373312 wait complete. 2024-08-31T07:11:50.542 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:50 smithi084 ceph-mon[22387]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-08-31T07:11:50.543 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:50 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/377373312' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-31T07:11:50.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:50 smithi032 ceph-mon[17252]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-08-31T07:11:50.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:50 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/377373312' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-31T07:11:50.676 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-31T07:11:50.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:11:50.688 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:50 smithi073 ceph-mon[21995]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-08-31T07:11:50.688 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:50 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/377373312' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-31T07:11:51.063 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.739+0000 7fdb30600640 1 -- 172.21.15.32:0/2793518185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb28109240 msgr2=0x7fdb28109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.739+0000 7fdb30600640 1 --2- 172.21.15.32:0/2793518185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb28109240 0x7fdb28109650 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009900 tx=0x7fdb1402a410 comp rx=0 tx=0).stop 2024-08-31T07:11:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.739+0000 7fdb30600640 1 -- 172.21.15.32:0/2793518185 shutdown_connections 2024-08-31T07:11:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.739+0000 7fdb30600640 1 --2- 172.21.15.32:0/2793518185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb28109b90 0x7fdb281103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.739+0000 7fdb30600640 1 --2- 172.21.15.32:0/2793518185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb28109240 0x7fdb28109650 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.739+0000 7fdb30600640 1 --2- 172.21.15.32:0/2793518185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb28103900 0x7fdb28103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.739+0000 7fdb30600640 1 -- 172.21.15.32:0/2793518185 >> 172.21.15.32:0/2793518185 conn(0x7fdb2807b5f0 msgr2=0x7fdb2807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.739+0000 7fdb30600640 1 -- 172.21.15.32:0/2793518185 shutdown_connections 2024-08-31T07:11:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.739+0000 7fdb30600640 1 -- 172.21.15.32:0/2793518185 wait complete. 2024-08-31T07:11:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.740+0000 7fdb30600640 1 Processor -- start 2024-08-31T07:11:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.740+0000 7fdb30600640 1 -- start start 2024-08-31T07:11:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb30600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb28103900 0x7fdb2819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb30600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb28109240 0x7fdb2819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb30600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb28109b90 0x7fdb2819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb30600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb28113850 con 0x7fdb28103900 2024-08-31T07:11:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb30600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281136d0 con 0x7fdb28109b90 2024-08-31T07:11:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb30600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281139d0 con 0x7fdb28109240 2024-08-31T07:11:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb2dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb28103900 0x7fdb2819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb2d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb28109240 0x7fdb2819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb2d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb28109240 0x7fdb2819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38016/0 (socket says 172.21.15.32:38016) 2024-08-31T07:11:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb2d200640 1 -- 172.21.15.32:0/1982021955 learned_addr learned my addr 172.21.15.32:0/1982021955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb2dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb28103900 0x7fdb2819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33264/0 (socket says 172.21.15.32:33264) 2024-08-31T07:11:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.741+0000 7fdb2e600640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb28109b90 0x7fdb2819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.742+0000 7fdb2dc00640 1 -- 172.21.15.32:0/1982021955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb28109240 msgr2=0x7fdb2819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.742+0000 7fdb2dc00640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb28109240 0x7fdb2819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.742+0000 7fdb2dc00640 1 -- 172.21.15.32:0/1982021955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb28109b90 msgr2=0x7fdb2819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.742+0000 7fdb2dc00640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb28109b90 0x7fdb2819fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.742+0000 7fdb2dc00640 1 -- 172.21.15.32:0/1982021955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb281a02b0 con 0x7fdb28103900 2024-08-31T07:11:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.742+0000 7fdb2dc00640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb28103900 0x7fdb2819a340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00e9f0 tx=0x7fdb1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.742+0000 7fdb26000640 1 -- 172.21.15.32:0/1982021955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00cd80 con 0x7fdb28103900 2024-08-31T07:11:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.743+0000 7fdb26000640 1 -- 172.21.15.32:0/1982021955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb1c0035f0 con 0x7fdb28103900 2024-08-31T07:11:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.743+0000 7fdb26000640 1 -- 172.21.15.32:0/1982021955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c010670 con 0x7fdb28103900 2024-08-31T07:11:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.743+0000 7fdb30600640 1 -- 172.21.15.32:0/1982021955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb281a05a0 con 0x7fdb28103900 2024-08-31T07:11:51.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.743+0000 7fdb30600640 1 -- 172.21.15.32:0/1982021955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb281a6d00 con 0x7fdb28103900 2024-08-31T07:11:51.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.744+0000 7fdb30600640 1 -- 172.21.15.32:0/1982021955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb28103d10 con 0x7fdb28103900 2024-08-31T07:11:51.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.746+0000 7fdb26000640 1 -- 172.21.15.32:0/1982021955 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7fdb1c0037a0 con 0x7fdb28103900 2024-08-31T07:11:51.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.748+0000 7fdb26000640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb100b4c00 0x7fdb100b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:51.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.748+0000 7fdb26000640 1 -- 172.21.15.32:0/1982021955 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fdb1c0e5690 con 0x7fdb28103900 2024-08-31T07:11:51.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.748+0000 7fdb2d200640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb100b4c00 0x7fdb100b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:51.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.749+0000 7fdb2d200640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb100b4c00 0x7fdb100b7110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdb281082b0 tx=0x7fdb14009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:51.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.749+0000 7fdb26000640 1 -- 172.21.15.32:0/1982021955 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fdb1c056130 con 0x7fdb28103900 2024-08-31T07:11:51.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.983+0000 7fdb30600640 1 -- 172.21.15.32:0/1982021955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb2819b470 con 0x7fdb28103900 2024-08-31T07:11:51.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.983+0000 7fdb26000640 1 -- 172.21.15.32:0/1982021955 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7fdb1c0ac6c0 con 0x7fdb28103900 2024-08-31T07:11:51.985 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:51.985 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"btime":"2024-08-31T07:10:05:878594+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:10:04.962915+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:11:51.985 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2024-08-31T07:11:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 -- 172.21.15.32:0/1982021955 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb100b4c00 msgr2=0x7fdb100b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb100b4c00 0x7fdb100b7110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdb281082b0 tx=0x7fdb14009560 comp rx=0 tx=0).stop 2024-08-31T07:11:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 -- 172.21.15.32:0/1982021955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb28103900 msgr2=0x7fdb2819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb28103900 0x7fdb2819a340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00e9f0 tx=0x7fdb1c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:11:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 -- 172.21.15.32:0/1982021955 shutdown_connections 2024-08-31T07:11:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb100b4c00 0x7fdb100b7110 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb28109b90 0x7fdb2819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb28109240 0x7fdb2819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 --2- 172.21.15.32:0/1982021955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb28103900 0x7fdb2819a340 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:51.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 -- 172.21.15.32:0/1982021955 >> 172.21.15.32:0/1982021955 conn(0x7fdb2807b5f0 msgr2=0x7fdb28105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:51.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 -- 172.21.15.32:0/1982021955 shutdown_connections 2024-08-31T07:11:51.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:51.986+0000 7fdb30600640 1 -- 172.21.15.32:0/1982021955 wait complete. 2024-08-31T07:11:52.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:52 smithi032 ceph-mon[17252]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-08-31T07:11:52.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:52 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1982021955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:52 smithi073 ceph-mon[21995]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-08-31T07:11:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:52 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1982021955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:52 smithi084 ceph-mon[22387]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-08-31T07:11:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:52 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1982021955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:52.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:11:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.814+0000 7f4778600640 1 -- 172.21.15.32:0/750956367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4770103920 msgr2=0x7f4770103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.814+0000 7f4778600640 1 --2- 172.21.15.32:0/750956367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4770103920 0x7f4770103d30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4764009860 tx=0x7f476402a340 comp rx=0 tx=0).stop 2024-08-31T07:11:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.814+0000 7f4778600640 1 -- 172.21.15.32:0/750956367 shutdown_connections 2024-08-31T07:11:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.814+0000 7f4778600640 1 --2- 172.21.15.32:0/750956367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4770109bb0 0x7f47701103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:53.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.814+0000 7f4778600640 1 --2- 172.21.15.32:0/750956367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4770109260 0x7f4770109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:53.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.814+0000 7f4778600640 1 --2- 172.21.15.32:0/750956367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4770103920 0x7f4770103d30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:53.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.814+0000 7f4778600640 1 -- 172.21.15.32:0/750956367 >> 172.21.15.32:0/750956367 conn(0x7f47700ff370 msgr2=0x7f47701017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:53.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.814+0000 7f4778600640 1 -- 172.21.15.32:0/750956367 shutdown_connections 2024-08-31T07:11:53.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.814+0000 7f4778600640 1 -- 172.21.15.32:0/750956367 wait complete. 2024-08-31T07:11:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.815+0000 7f4778600640 1 Processor -- start 2024-08-31T07:11:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.815+0000 7f4778600640 1 -- start start 2024-08-31T07:11:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4778600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4770103920 0x7f47700754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4778600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4770109260 0x7f47700759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4778600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4770109bb0 0x7f4770075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4778600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4770113870 con 0x7f4770103920 2024-08-31T07:11:53.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4778600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47701136f0 con 0x7f4770109260 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4778600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47701139f0 con 0x7f4770109bb0 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4775c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4770103920 0x7f47700754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4775c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4770103920 0x7f47700754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33292/0 (socket says 172.21.15.32:33292) 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4775c00640 1 -- 172.21.15.32:0/1757881646 learned_addr learned my addr 172.21.15.32:0/1757881646 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4775200640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4770109260 0x7f47700759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4776600640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4770109bb0 0x7f4770075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4776600640 1 -- 172.21.15.32:0/1757881646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4770109260 msgr2=0x7f47700759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4776600640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4770109260 0x7f47700759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4776600640 1 -- 172.21.15.32:0/1757881646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4770103920 msgr2=0x7f47700754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4776600640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4770103920 0x7f47700754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.816+0000 7f4776600640 1 -- 172.21.15.32:0/1757881646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47701a8500 con 0x7f4770109bb0 2024-08-31T07:11:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.817+0000 7f4776600640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4770109bb0 0x7f4770075f30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f476800ef00 tx=0x7f476800c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.817+0000 7f476e000640 1 -- 172.21.15.32:0/1757881646 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768004b20 con 0x7f4770109bb0 2024-08-31T07:11:53.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.817+0000 7f476e000640 1 -- 172.21.15.32:0/1757881646 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4768003e20 con 0x7f4770109bb0 2024-08-31T07:11:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.817+0000 7f4778600640 1 -- 172.21.15.32:0/1757881646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47701a87f0 con 0x7f4770109bb0 2024-08-31T07:11:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.817+0000 7f476e000640 1 -- 172.21.15.32:0/1757881646 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476800b570 con 0x7f4770109bb0 2024-08-31T07:11:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.817+0000 7f4778600640 1 -- 172.21.15.32:0/1757881646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47701a8ce0 con 0x7f4770109bb0 2024-08-31T07:11:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.819+0000 7f4778600640 1 -- 172.21.15.32:0/1757881646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4770103d30 con 0x7f4770109bb0 2024-08-31T07:11:53.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.821+0000 7f476e000640 1 -- 172.21.15.32:0/1757881646 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f476801d020 con 0x7f4770109bb0 2024-08-31T07:11:53.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.822+0000 7f476e000640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f47580b4b80 0x7f47580b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:53.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.822+0000 7f476e000640 1 -- 172.21.15.32:0/1757881646 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f47680e5130 con 0x7f4770109bb0 2024-08-31T07:11:53.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.823+0000 7f4775c00640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f47580b4b80 0x7f47580b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:53.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.823+0000 7f4775c00640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f47580b4b80 0x7f47580b7090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f476402a930 tx=0x7f4764030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:53.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:53.827+0000 7f476e000640 1 -- 172.21.15.32:0/1757881646 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4768054130 con 0x7f4770109bb0 2024-08-31T07:11:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.059+0000 7f4778600640 1 -- 172.21.15.32:0/1757881646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f477002e430 con 0x7f4770109bb0 2024-08-31T07:11:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.060+0000 7f476e000640 1 -- 172.21.15.32:0/1757881646 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7f4768004cc0 con 0x7f4770109bb0 2024-08-31T07:11:54.061 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:54.061 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"btime":"2024-08-31T07:10:05:878594+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:10:04.962915+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:11:54.062 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 -- 172.21.15.32:0/1757881646 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f47580b4b80 msgr2=0x7f47580b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f47580b4b80 0x7f47580b7090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f476402a930 tx=0x7f4764030340 comp rx=0 tx=0).stop 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 -- 172.21.15.32:0/1757881646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4770109bb0 msgr2=0x7f4770075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4770109bb0 0x7f4770075f30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f476800ef00 tx=0x7f476800c740 comp rx=0 tx=0).stop 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 -- 172.21.15.32:0/1757881646 shutdown_connections 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f47580b4b80 0x7f47580b7090 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4770109bb0 0x7f4770075f30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4770109260 0x7f47700759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 --2- 172.21.15.32:0/1757881646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4770103920 0x7f47700754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 -- 172.21.15.32:0/1757881646 >> 172.21.15.32:0/1757881646 conn(0x7f47700ff370 msgr2=0x7f4770101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:54.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 -- 172.21.15.32:0/1757881646 shutdown_connections 2024-08-31T07:11:54.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:54.063+0000 7f4778600640 1 -- 172.21.15.32:0/1757881646 wait complete. 2024-08-31T07:11:54.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:54 smithi032 ceph-mon[17252]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-08-31T07:11:54.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:54 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1757881646' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:54 smithi073 ceph-mon[21995]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-08-31T07:11:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:54 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1757881646' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:54 smithi084 ceph-mon[22387]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-08-31T07:11:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:54 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1757881646' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:11:54.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:11:55.181 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:55.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.863+0000 7fb9e1c00640 1 -- 172.21.15.32:0/2510202819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc075b70 msgr2=0x7fb9dc075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.863+0000 7fb9e1c00640 1 --2- 172.21.15.32:0/2510202819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc075b70 0x7fb9dc075f80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009900 tx=0x7fb9c802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:11:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.863+0000 7fb9e1c00640 1 -- 172.21.15.32:0/2510202819 shutdown_connections 2024-08-31T07:11:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.863+0000 7fb9e1c00640 1 --2- 172.21.15.32:0/2510202819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9dc1141f0 0x7fb9dc1166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.863+0000 7fb9e1c00640 1 --2- 172.21.15.32:0/2510202819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc075b70 0x7fb9dc075f80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.863+0000 7fb9e1c00640 1 --2- 172.21.15.32:0/2510202819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9dc077120 0x7fb9dc075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.863+0000 7fb9e1c00640 1 -- 172.21.15.32:0/2510202819 >> 172.21.15.32:0/2510202819 conn(0x7fb9dc0ff370 msgr2=0x7fb9dc1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.863+0000 7fb9e1c00640 1 -- 172.21.15.32:0/2510202819 shutdown_connections 2024-08-31T07:11:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.863+0000 7fb9e1c00640 1 -- 172.21.15.32:0/2510202819 wait complete. 2024-08-31T07:11:55.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.864+0000 7fb9e1c00640 1 Processor -- start 2024-08-31T07:11:55.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.864+0000 7fb9e1c00640 1 -- start start 2024-08-31T07:11:55.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9e1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9dc075b70 0x7fb9dc1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:55.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9e1c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9dc077120 0x7fb9dc1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9e1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc1141f0 0x7fb9dc1a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9e1c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc119e30 con 0x7fb9dc1141f0 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9e1c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc119cb0 con 0x7fb9dc075b70 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9e1c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc119fb0 con 0x7fb9dc077120 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9da000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9dc077120 0x7fb9dc1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9daa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9dc075b70 0x7fb9dc1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9da000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9dc077120 0x7fb9dc1a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36218/0 (socket says 172.21.15.32:36218) 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9daa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9dc075b70 0x7fb9dc1a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52698/0 (socket says 172.21.15.32:52698) 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9da000640 1 -- 172.21.15.32:0/197140063 learned_addr learned my addr 172.21.15.32:0/197140063 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9db400640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc1141f0 0x7fb9dc1a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9daa00640 1 -- 172.21.15.32:0/197140063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9dc077120 msgr2=0x7fb9dc1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9daa00640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9dc077120 0x7fb9dc1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:55.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9daa00640 1 -- 172.21.15.32:0/197140063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc1141f0 msgr2=0x7fb9dc1a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:55.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9daa00640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc1141f0 0x7fb9dc1a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:55.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.865+0000 7fb9daa00640 1 -- 172.21.15.32:0/197140063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc1a67b0 con 0x7fb9dc075b70 2024-08-31T07:11:55.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.866+0000 7fb9daa00640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9dc075b70 0x7fb9dc1a0840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00e9f0 tx=0x7fb9cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:55.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.866+0000 7fb9d3400640 1 -- 172.21.15.32:0/197140063 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc00cd80 con 0x7fb9dc075b70 2024-08-31T07:11:55.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.866+0000 7fb9d3400640 1 -- 172.21.15.32:0/197140063 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9cc0035f0 con 0x7fb9dc075b70 2024-08-31T07:11:55.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.866+0000 7fb9e1c00640 1 -- 172.21.15.32:0/197140063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc1a6aa0 con 0x7fb9dc075b70 2024-08-31T07:11:55.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.866+0000 7fb9d3400640 1 -- 172.21.15.32:0/197140063 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc010670 con 0x7fb9dc075b70 2024-08-31T07:11:55.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.866+0000 7fb9e1c00640 1 -- 172.21.15.32:0/197140063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc1ad200 con 0x7fb9dc075b70 2024-08-31T07:11:55.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.867+0000 7fb9e1c00640 1 -- 172.21.15.32:0/197140063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc075f80 con 0x7fb9dc075b70 2024-08-31T07:11:55.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.870+0000 7fb9d3400640 1 -- 172.21.15.32:0/197140063 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7fb9cc0037a0 con 0x7fb9dc075b70 2024-08-31T07:11:55.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.871+0000 7fb9d3400640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9c00b4b80 0x7fb9c00b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:55.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.871+0000 7fb9d3400640 1 -- 172.21.15.32:0/197140063 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb9cc0e56d0 con 0x7fb9dc075b70 2024-08-31T07:11:55.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.872+0000 7fb9da000640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9c00b4b80 0x7fb9c00b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:55.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.872+0000 7fb9da000640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9c00b4b80 0x7fb9c00b7090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8005f70 tx=0x7fb9c8009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:55.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:55.873+0000 7fb9d3400640 1 -- 172.21.15.32:0/197140063 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb9cc056130 con 0x7fb9dc075b70 2024-08-31T07:11:56.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.069+0000 7fb9e1c00640 1 -- 172.21.15.32:0/197140063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9dc1a1970 con 0x7fb9dc075b70 2024-08-31T07:11:56.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.071+0000 7fb9d3400640 1 -- 172.21.15.32:0/197140063 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22921 (secure 0 0 0) 0x7fb9cc0ac700 con 0x7fb9dc075b70 2024-08-31T07:11:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:11:56.072 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:10:46.011475+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:11:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 -- 172.21.15.32:0/197140063 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9c00b4b80 msgr2=0x7fb9c00b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9c00b4b80 0x7fb9c00b7090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8005f70 tx=0x7fb9c8009560 comp rx=0 tx=0).stop 2024-08-31T07:11:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 -- 172.21.15.32:0/197140063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9dc075b70 msgr2=0x7fb9dc1a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9dc075b70 0x7fb9dc1a0840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00e9f0 tx=0x7fb9cc00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:11:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 -- 172.21.15.32:0/197140063 shutdown_connections 2024-08-31T07:11:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9c00b4b80 0x7fb9c00b7090 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9dc1141f0 0x7fb9dc1a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9dc077120 0x7fb9dc1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 --2- 172.21.15.32:0/197140063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9dc075b70 0x7fb9dc1a0840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:56.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 -- 172.21.15.32:0/197140063 >> 172.21.15.32:0/197140063 conn(0x7fb9dc0ff370 msgr2=0x7fb9dc1100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:56.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 -- 172.21.15.32:0/197140063 shutdown_connections 2024-08-31T07:11:56.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:56.074+0000 7fb9e1c00640 1 -- 172.21.15.32:0/197140063 wait complete. 2024-08-31T07:11:56.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:56 smithi032 ceph-mon[17252]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-08-31T07:11:56.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:56 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/197140063' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:11:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:56 smithi073 ceph-mon[21995]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-08-31T07:11:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:56 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/197140063' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:11:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:56 smithi084 ceph-mon[22387]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-08-31T07:11:56.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:56 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/197140063' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:11:56.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-31T07:09:48.797887+0000', 'modified': '2024-08-31T07:10:04.962915+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24734, 'mds_1': 24718, 'mds_2': 24694, 'mds_3': 24692, 'mds_4': 24676}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24625': {'gid': 24625, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6833/2138862695', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6832', 'nonce': 2138862695}, {'type': 'v1', 'addr': '172.21.15.84:6833', 'nonce': 2138862695}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24632': {'gid': 24632, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6833/2003936745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 2003936745}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 2003936745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24644': {'gid': 24644, 'name': 'j', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.32:6835/1250889023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1250889023}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1250889023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6835/1927490574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 1927490574}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 1927490574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24670': {'gid': 24670, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6835/204703622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 204703622}, {'type': 'v1', 'addr': '172.21.15.84:6835', 'nonce': 204703622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24676': {'gid': 24676, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.84:6837/4208526312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 4208526312}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 4208526312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24692': {'gid': 24692, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.73:6837/2735170682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 2735170682}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2735170682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24694': {'gid': 24694, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.84:6839/2807364599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6838', 'nonce': 2807364599}, {'type': 'v1', 'addr': '172.21.15.84:6839', 'nonce': 2807364599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24718': {'gid': 24718, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.32:6841/2484505785', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6840', 'nonce': 2484505785}, {'type': 'v1', 'addr': '172.21.15.32:6841', 'nonce': 2484505785}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24734': {'gid': 24734, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.73:6839/794507464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 794507464}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 794507464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24734, 'qdb_cluster': [24676, 24692, 24718, 24694, 24734]} 2024-08-31T07:11:56.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-08-31T07:11:56.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.c status 2024-08-31T07:11:57.260 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.917+0000 7f816b200640 1 -- 172.21.15.32:0/4061669755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8164103730 msgr2=0x7f8164103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.917+0000 7f816b200640 1 --2- 172.21.15.32:0/4061669755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8164103730 0x7f8164103b40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8158009900 tx=0x7f815802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.917+0000 7f816b200640 1 -- 172.21.15.32:0/4061669755 shutdown_connections 2024-08-31T07:11:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.917+0000 7f816b200640 1 --2- 172.21.15.32:0/4061669755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81641099c0 0x7f81641101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.917+0000 7f816b200640 1 --2- 172.21.15.32:0/4061669755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8164109070 0x7f8164109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.917+0000 7f816b200640 1 --2- 172.21.15.32:0/4061669755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8164103730 0x7f8164103b40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.917+0000 7f816b200640 1 -- 172.21.15.32:0/4061669755 >> 172.21.15.32:0/4061669755 conn(0x7f81640ff180 msgr2=0x7f81641015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.917+0000 7f816b200640 1 -- 172.21.15.32:0/4061669755 shutdown_connections 2024-08-31T07:11:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.917+0000 7f816b200640 1 -- 172.21.15.32:0/4061669755 wait complete. 2024-08-31T07:11:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.918+0000 7f816b200640 1 Processor -- start 2024-08-31T07:11:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.918+0000 7f816b200640 1 -- start start 2024-08-31T07:11:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f816b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8164103730 0x7f816419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f816b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8164109070 0x7f816419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f816b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81641099c0 0x7f816419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f816b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8164113690 con 0x7f8164103730 2024-08-31T07:11:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f816b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8164113510 con 0x7f81641099c0 2024-08-31T07:11:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f816b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8164113810 con 0x7f8164109070 2024-08-31T07:11:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f8169e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8164103730 0x7f816419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f8169e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8164103730 0x7f816419a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40868/0 (socket says 172.21.15.32:40868) 2024-08-31T07:11:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f8169e00640 1 -- 172.21.15.32:0/3885512618 learned_addr learned my addr 172.21.15.32:0/3885512618 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f8169400640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8164109070 0x7f816419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.919+0000 7f816a800640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81641099c0 0x7f816419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f8169e00640 1 -- 172.21.15.32:0/3885512618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8164109070 msgr2=0x7f816419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f8169e00640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8164109070 0x7f816419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f8169e00640 1 -- 172.21.15.32:0/3885512618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81641099c0 msgr2=0x7f816419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f8169e00640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81641099c0 0x7f816419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f8169e00640 1 -- 172.21.15.32:0/3885512618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81641a0010 con 0x7f8164103730 2024-08-31T07:11:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f8169e00640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8164103730 0x7f816419a0a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8158005940 tx=0x7f8158004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f815ea00640 1 -- 172.21.15.32:0/3885512618 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815803f030 con 0x7f8164103730 2024-08-31T07:11:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f816b200640 1 -- 172.21.15.32:0/3885512618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81641a02a0 con 0x7f8164103730 2024-08-31T07:11:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f815ea00640 1 -- 172.21.15.32:0/3885512618 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8158003aa0 con 0x7f8164103730 2024-08-31T07:11:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f815ea00640 1 -- 172.21.15.32:0/3885512618 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158035070 con 0x7f8164103730 2024-08-31T07:11:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.920+0000 7f816b200640 1 -- 172.21.15.32:0/3885512618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81641a6a60 con 0x7f8164103730 2024-08-31T07:11:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.924+0000 7f8173451740 1 Processor -- start 2024-08-31T07:11:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.924+0000 7f8173451740 1 -- start start 2024-08-31T07:11:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.924+0000 7f815ea00640 1 -- 172.21.15.32:0/3885512618 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f8158042030 con 0x7f8164103730 2024-08-31T07:11:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.924+0000 7f8173451740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d9bf60 0x557f82141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.924+0000 7f8173451740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d730a0 0x557f82141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.924+0000 7f8173451740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f82142520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.924+0000 7f8173451740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557f820b8550 con 0x557f81d730a0 2024-08-31T07:11:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.924+0000 7f8173451740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557f820a7ae0 con 0x557f81d7b280 2024-08-31T07:11:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.924+0000 7f8173451740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557f820aba90 con 0x557f81d9bf60 2024-08-31T07:11:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f8169e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d730a0 0x557f82141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f816a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d9bf60 0x557f82141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f8169e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d730a0 0x557f82141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40880/0 (socket says 172.21.15.32:40880) 2024-08-31T07:11:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f816a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d9bf60 0x557f82141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36246/0 (socket says 172.21.15.32:36246) 2024-08-31T07:11:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f8169e00640 1 -- 172.21.15.32:0/530531737 learned_addr learned my addr 172.21.15.32:0/530531737 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f8169400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f82142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f816a800640 1 -- 172.21.15.32:0/530531737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 msgr2=0x557f82142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f816a800640 1 --2- 172.21.15.32:0/530531737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f82142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f816a800640 1 -- 172.21.15.32:0/530531737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d730a0 msgr2=0x557f82141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f816a800640 1 --2- 172.21.15.32:0/530531737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d730a0 0x557f82141fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.925+0000 7f816a800640 1 -- 172.21.15.32:0/530531737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f81e07540 con 0x557f81d9bf60 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.926+0000 7f816a800640 1 --2- 172.21.15.32:0/530531737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d9bf60 0x557f82141aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8160009f30 tx=0x7f8160010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b20804f790ded9d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.926+0000 7f8153000640 1 -- 172.21.15.32:0/530531737 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160016070 con 0x557f81d9bf60 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.926+0000 7f8153000640 1 -- 172.21.15.32:0/530531737 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160004d20 con 0x557f81d9bf60 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.926+0000 7f8153000640 1 -- 172.21.15.32:0/530531737 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600127d0 con 0x557f81d9bf60 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.926+0000 7f815ea00640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f814c0b4c00 0x7f814c0b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f815ea00640 1 -- 172.21.15.32:0/3885512618 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f8158105860 con 0x7f8164103730 2024-08-31T07:11:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8169e00640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f814c0b4c00 0x7f814c0b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8173451740 1 -- 172.21.15.32:0/530531737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d9bf60 msgr2=0x557f82141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8173451740 1 --2- 172.21.15.32:0/530531737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d9bf60 0x557f82141aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8160009f30 tx=0x7f8160010800 comp rx=0 tx=0).stop 2024-08-31T07:11:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8173451740 1 -- 172.21.15.32:0/530531737 shutdown_connections 2024-08-31T07:11:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8173451740 1 --2- 172.21.15.32:0/530531737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f82142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8173451740 1 --2- 172.21.15.32:0/530531737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d730a0 0x557f82141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8173451740 1 --2- 172.21.15.32:0/530531737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d9bf60 0x557f82141aa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8173451740 1 -- 172.21.15.32:0/530531737 >> 172.21.15.32:0/530531737 conn(0x557f81d90350 msgr2=0x557f8213f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8173451740 1 -- 172.21.15.32:0/530531737 shutdown_connections 2024-08-31T07:11:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8173451740 1 -- 172.21.15.32:0/530531737 wait complete. 2024-08-31T07:11:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.927+0000 7f8169e00640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f814c0b4c00 0x7f814c0b7110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8158032af0 tx=0x7f815810b530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.928+0000 7f8173451740 1 -- start start 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.928+0000 7f8173451740 1 Processor -- start 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.928+0000 7f8173451740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f8214d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.928+0000 7f8173451740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f8214d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.928+0000 7f8173451740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f821595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.928+0000 7f8173451740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557f81f0a5f0 con 0x557f81d9bf60 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.928+0000 7f8173451740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557f8202ebd0 con 0x557f81d7b280 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.928+0000 7f8173451740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557f82150280 con 0x557f81d730a0 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f8173451740 10 client.0 _ll_register_callbacks cb 0x557f82054210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f8169400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f8214d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f816a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f821595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f8169400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f8214d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52726/0 (socket says 172.21.15.32:52726) 2024-08-31T07:11:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f8169400640 1 -- 172.21.15.32:0/3054211801 learned_addr learned my addr 172.21.15.32:0/3054211801 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f816a800640 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f8214d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f816a800640 1 -- 172.21.15.32:0/3054211801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 msgr2=0x557f8214d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f816a800640 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f8214d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f816a800640 1 -- 172.21.15.32:0/3054211801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 msgr2=0x557f8214d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f816a800640 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f8214d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.929+0000 7f816a800640 1 -- 172.21.15.32:0/3054211801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f8206f650 con 0x557f81d9bf60 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.930+0000 7f816a800640 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f821595d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8160005970 tx=0x7f8160005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ad9a9fd24f9a92 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.930+0000 7f8147400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.930+0000 7f8147400640 1 -- 172.21.15.32:0/3054211801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160016030 con 0x557f81d9bf60 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.930+0000 7f8147400640 1 -- 172.21.15.32:0/3054211801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f8160003b70 con 0x557f81d9bf60 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.930+0000 7f8173451740 1 -- 172.21.15.32:0/3054211801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8160004d20 con 0x557f81d9bf60 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.930+0000 7f8147400640 1 -- 172.21.15.32:0/3054211801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8160023040 con 0x557f81d9bf60 2024-08-31T07:11:57.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.930+0000 7f8173451740 10 client.15219 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.930+0000 7f8173451740 1 -- 172.21.15.32:0/3054211801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f8215c0c0 con 0x557f81d9bf60 2024-08-31T07:11:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.930+0000 7f8147400640 1 -- 172.21.15.32:0/3054211801 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160004d20 con 0x557f81d9bf60 2024-08-31T07:11:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.931+0000 7f8147400640 1 -- 172.21.15.32:0/3054211801 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f816000f040 con 0x557f81d9bf60 2024-08-31T07:11:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.931+0000 7f8173451740 10 client.15219 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.931+0000 7f8173451740 10 client.15219 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-31T07:11:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.931+0000 7f8173451740 20 client.15219 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.931+0000 7f8173451740 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557f81d5a8e0 0x557f8215f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.931+0000 7f8173451740 4 client.15219 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:11:57.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.931+0000 7f8173451740 1 -- 172.21.15.32:0/3054211801 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f820ba4a0 con 0x557f81d5a8e0 2024-08-31T07:11:57.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.932+0000 7f8169400640 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557f81d5a8e0 0x557f8215f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.932+0000 7f8169400640 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557f81d5a8e0 0x557f8215f8f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7543ebe1a7585436 server_cookie=2204a4d2b5b0028a in_seq=0 out_seq=0 2024-08-31T07:11:57.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.932+0000 7f8147400640 10 client.15219 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:11:57.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.933+0000 7f8147400640 1 -- 172.21.15.32:0/3054211801 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f820ba4a0 con 0x557f81d5a8e0 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.933+0000 7f8147400640 10 client.15219 handle_command_reply: tid=0 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.933+0000 7f8173451740 1 client.15219 shutdown 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 1 -- 172.21.15.32:0/3054211801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 msgr2=0x557f821595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f821595d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8160005970 tx=0x7f8160005a70 comp rx=0 tx=0).stop 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 1 -- 172.21.15.32:0/3054211801 shutdown_connections 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557f81d5a8e0 0x557f8215f8f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f821595d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f8214d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 1 --2- 172.21.15.32:0/3054211801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f8214d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 1 -- 172.21.15.32:0/3054211801 >> 172.21.15.32:0/3054211801 conn(0x557f81d90350 msgr2=0x557f82144c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8147400640 0 client.15219 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 1 -- 172.21.15.32:0/3054211801 shutdown_connections 2024-08-31T07:11:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 1 -- 172.21.15.32:0/3054211801 wait complete. 2024-08-31T07:11:57.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.934+0000 7f8173451740 20 client.15219 trim_cache size 0 max 16384 2024-08-31T07:11:57.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.953+0000 7f8173451740 1 Processor -- start 2024-08-31T07:11:57.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.953+0000 7f8173451740 1 -- start start 2024-08-31T07:11:57.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8173451740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f8214d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8173451740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d7b280 0x557f8214d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8173451740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d9bf60 0x557f82144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8173451740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557f8215dc30 con 0x557f81d7b280 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8173451740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557f8215e300 con 0x557f81d9bf60 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8173451740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557f8215e9d0 con 0x557f81d730a0 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8169400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d7b280 0x557f8214d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f816a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f8214d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8169400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d7b280 0x557f8214d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40888/0 (socket says 172.21.15.32:40888) 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8169400640 1 -- 172.21.15.32:0/4131879708 learned_addr learned my addr 172.21.15.32:0/4131879708 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f816a800640 1 --2- 172.21.15.32:0/4131879708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d9bf60 0x557f82144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8169400640 1 -- 172.21.15.32:0/4131879708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 msgr2=0x557f8214d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8169400640 1 --2- 172.21.15.32:0/4131879708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f8214d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8169400640 1 -- 172.21.15.32:0/4131879708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d9bf60 msgr2=0x557f82144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8169400640 1 --2- 172.21.15.32:0/4131879708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d9bf60 0x557f82144ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f8169400640 1 -- 172.21.15.32:0/4131879708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f8214e550 con 0x557f81d7b280 2024-08-31T07:11:57.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.954+0000 7f816a800640 1 --2- 172.21.15.32:0/4131879708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f8214d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:57.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.955+0000 7f8169400640 1 --2- 172.21.15.32:0/4131879708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d7b280 0x557f8214d580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8154009e20 tx=0x7f8154005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f595559a69388c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:57.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.955+0000 7f8152600640 1 -- 172.21.15.32:0/4131879708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154004020 con 0x557f81d7b280 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.955+0000 7f8152600640 1 -- 172.21.15.32:0/4131879708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81540054d0 con 0x557f81d7b280 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.955+0000 7f8152600640 1 -- 172.21.15.32:0/4131879708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154004c60 con 0x557f81d7b280 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.956+0000 7f8173451740 1 -- 172.21.15.32:0/4131879708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d7b280 msgr2=0x557f8214d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.956+0000 7f8173451740 1 --2- 172.21.15.32:0/4131879708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d7b280 0x557f8214d580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8154009e20 tx=0x7f8154005f90 comp rx=0 tx=0).stop 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.956+0000 7f8173451740 1 -- 172.21.15.32:0/4131879708 shutdown_connections 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.956+0000 7f8173451740 1 --2- 172.21.15.32:0/4131879708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d9bf60 0x557f82144ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.956+0000 7f8173451740 1 --2- 172.21.15.32:0/4131879708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d7b280 0x557f8214d580 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.956+0000 7f8173451740 1 --2- 172.21.15.32:0/4131879708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f8214d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.956+0000 7f8173451740 1 -- 172.21.15.32:0/4131879708 >> 172.21.15.32:0/4131879708 conn(0x557f81d90350 msgr2=0x557f82148440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.956+0000 7f8173451740 1 -- 172.21.15.32:0/4131879708 shutdown_connections 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.956+0000 7f8173451740 1 -- 172.21.15.32:0/4131879708 wait complete. 2024-08-31T07:11:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.956+0000 7f8173451740 1 -- start start 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.957+0000 7f8173451740 1 Processor -- start 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.957+0000 7f8173451740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f821684b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.957+0000 7f8173451740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f82163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.957+0000 7f8173451740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f82163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.957+0000 7f8173451740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557f8215f770 con 0x557f81d9bf60 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.957+0000 7f8173451740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557f8215f0a0 con 0x557f81d7b280 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.957+0000 7f8173451740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557f8215fe40 con 0x557f81d730a0 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.957+0000 7f8173451740 10 client.0 _ll_register_callbacks cb 0x7f8160012550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.957+0000 7f816a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f82163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f816a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f82163fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40900/0 (socket says 172.21.15.32:40900) 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f816a800640 1 -- 172.21.15.32:0/2908915084 learned_addr learned my addr 172.21.15.32:0/2908915084 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f8169400640 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f82163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f816a800640 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f821684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f816a800640 1 -- 172.21.15.32:0/2908915084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 msgr2=0x557f821684b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f816a800640 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f821684b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f816a800640 1 -- 172.21.15.32:0/2908915084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 msgr2=0x557f82163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f816a800640 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f82163960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f816a800640 1 -- 172.21.15.32:0/2908915084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f82164740 con 0x557f81d9bf60 2024-08-31T07:11:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f816a800640 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f82163fb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8160011010 tx=0x7f8160009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3974789b381021 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.958+0000 7f8147400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:11:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.959+0000 7f8147400640 1 -- 172.21.15.32:0/2908915084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160016070 con 0x557f81d9bf60 2024-08-31T07:11:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.959+0000 7f8147400640 1 -- 172.21.15.32:0/2908915084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f8160003c30 con 0x557f81d9bf60 2024-08-31T07:11:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.959+0000 7f8173451740 1 -- 172.21.15.32:0/2908915084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81540054d0 con 0x557f81d9bf60 2024-08-31T07:11:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.959+0000 7f8147400640 1 -- 172.21.15.32:0/2908915084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8160033040 con 0x557f81d9bf60 2024-08-31T07:11:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.959+0000 7f8173451740 10 client.15225 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.959+0000 7f8173451740 1 -- 172.21.15.32:0/2908915084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f821cae10 con 0x557f81d9bf60 2024-08-31T07:11:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.959+0000 7f8147400640 1 -- 172.21.15.32:0/2908915084 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81540054d0 con 0x557f81d9bf60 2024-08-31T07:11:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.959+0000 7f8147400640 1 -- 172.21.15.32:0/2908915084 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f816001a580 con 0x557f81d9bf60 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.960+0000 7f8173451740 10 client.15225 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.960+0000 7f8173451740 10 client.15225 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.960+0000 7f8173451740 20 client.15225 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.960+0000 7f8173451740 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557f81d2fc90 0x557f821ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.960+0000 7f8173451740 4 client.15225 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.960+0000 7f8173451740 1 -- 172.21.15.32:0/2908915084 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "status"}) -- 0x557f820b5f40 con 0x557f81d2fc90 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.960+0000 7f8169400640 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557f81d2fc90 0x557f821ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8169400640 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557f81d2fc90 0x557f821ce630 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=222f99673d42b323 server_cookie=71929d765af6e47f in_seq=0 out_seq=0 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8147400640 10 client.15225 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8147400640 1 -- 172.21.15.32:0/2908915084 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557f820b5f40 con 0x557f81d2fc90 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8147400640 10 client.15225 handle_command_reply: tid=0 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8173451740 1 client.15225 shutdown 2024-08-31T07:11:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8173451740 1 -- 172.21.15.32:0/2908915084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 msgr2=0x557f82163fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8173451740 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f82163fb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8160011010 tx=0x7f8160009600 comp rx=0 tx=0).stop 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8173451740 1 -- 172.21.15.32:0/2908915084 shutdown_connections 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8173451740 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557f81d2fc90 0x557f821ce630 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8173451740 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f81d9bf60 0x557f82163fb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8173451740 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f81d7b280 0x557f82163960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8173451740 1 --2- 172.21.15.32:0/2908915084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f81d730a0 0x557f821684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8173451740 1 -- 172.21.15.32:0/2908915084 >> 172.21.15.32:0/2908915084 conn(0x557f81d90350 msgr2=0x557f82148440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8147400640 0 client.15225 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.961+0000 7f8173451740 1 -- 172.21.15.32:0/2908915084 shutdown_connections 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f8173451740 1 -- 172.21.15.32:0/2908915084 wait complete. 2024-08-31T07:11:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f8173451740 20 client.15225 trim_cache size 0 max 16384 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 4, 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24676, 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 115.090640967, 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T07:11:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 158.024836779 2024-08-31T07:11:57.967 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:11:57.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f816b200640 1 -- 172.21.15.32:0/3885512618 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f814c0b4c00 msgr2=0x7f814c0b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:57.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f816b200640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f814c0b4c00 0x7f814c0b7110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8158032af0 tx=0x7f815810b530 comp rx=0 tx=0).stop 2024-08-31T07:11:57.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f816b200640 1 -- 172.21.15.32:0/3885512618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8164103730 msgr2=0x7f816419a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:57.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f816b200640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8164103730 0x7f816419a0a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8158005940 tx=0x7f8158004270 comp rx=0 tx=0).stop 2024-08-31T07:11:57.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f816b200640 1 -- 172.21.15.32:0/3885512618 shutdown_connections 2024-08-31T07:11:57.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f816b200640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f814c0b4c00 0x7f814c0b7110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f816b200640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81641099c0 0x7f816419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f816b200640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8164109070 0x7f816419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f816b200640 1 --2- 172.21.15.32:0/3885512618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8164103730 0x7f816419a0a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.962+0000 7f816b200640 1 -- 172.21.15.32:0/3885512618 >> 172.21.15.32:0/3885512618 conn(0x7f81640ff180 msgr2=0x7f81641015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.963+0000 7f816b200640 1 -- 172.21.15.32:0/3885512618 shutdown_connections 2024-08-31T07:11:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:57.963+0000 7f816b200640 1 -- 172.21.15.32:0/3885512618 wait complete. 2024-08-31T07:11:58.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:11:58 smithi032 ceph-mon[17252]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-08-31T07:11:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:11:58 smithi073 ceph-mon[21995]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-08-31T07:11:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:11:58 smithi084 ceph-mon[22387]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-08-31T07:11:58.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.h status 2024-08-31T07:11:59.175 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:11:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.818+0000 7efff7e00640 1 -- 172.21.15.32:0/1647812650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 msgr2=0x7efff8112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.818+0000 7efff7e00640 1 --2- 172.21.15.32:0/1647812650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 0x7efff8112410 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efff0009ca0 tx=0x7efff002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.818+0000 7efff7e00640 1 -- 172.21.15.32:0/1647812650 shutdown_connections 2024-08-31T07:11:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.818+0000 7efff7e00640 1 --2- 172.21.15.32:0/1647812650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 0x7efff8112410 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.818+0000 7efff7e00640 1 --2- 172.21.15.32:0/1647812650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810b290 0x7efff810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.818+0000 7efff7e00640 1 --2- 172.21.15.32:0/1647812650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff8105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.818+0000 7efff7e00640 1 -- 172.21.15.32:0/1647812650 >> 172.21.15.32:0/1647812650 conn(0x7efff8101420 msgr2=0x7efff8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.818+0000 7efff7e00640 1 -- 172.21.15.32:0/1647812650 shutdown_connections 2024-08-31T07:11:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.818+0000 7efff7e00640 1 -- 172.21.15.32:0/1647812650 wait complete. 2024-08-31T07:11:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.819+0000 7efff7e00640 1 Processor -- start 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.819+0000 7efff7e00640 1 -- start start 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810b290 0x7efff819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 0x7efff81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efff81158a0 con 0x7efff810bbe0 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efff8115720 con 0x7efff810b290 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff7e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efff8115a20 con 0x7efff8105950 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 0x7efff81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 0x7efff81a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40918/0 (socket says 172.21.15.32:40918) 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff7400640 1 -- 172.21.15.32:0/3884993072 learned_addr learned my addr 172.21.15.32:0/3884993072 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.820+0000 7efff6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810b290 0x7efff819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7efff6000640 1 -- 172.21.15.32:0/3884993072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 msgr2=0x7efff819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7efff6000640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7efff6000640 1 -- 172.21.15.32:0/3884993072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 msgr2=0x7efff81a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7efff6000640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 0x7efff81a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7efff6000640 1 -- 172.21.15.32:0/3884993072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff81a2180 con 0x7efff810b290 2024-08-31T07:11:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7efff7400640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 0x7efff81a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7efff6000640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810b290 0x7efff819c7b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efff806c660 tx=0x7effe400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7effef400640 1 -- 172.21.15.32:0/3884993072 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe400cc40 con 0x7efff810b290 2024-08-31T07:11:59.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7efff7e00640 1 -- 172.21.15.32:0/3884993072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff81a2470 con 0x7efff810b290 2024-08-31T07:11:59.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7effef400640 1 -- 172.21.15.32:0/3884993072 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe400cde0 con 0x7efff810b290 2024-08-31T07:11:59.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7effef400640 1 -- 172.21.15.32:0/3884993072 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe4018760 con 0x7efff810b290 2024-08-31T07:11:59.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.821+0000 7efff7e00640 1 -- 172.21.15.32:0/3884993072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff81a8bd0 con 0x7efff810b290 2024-08-31T07:11:59.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.825+0000 7f00045eb740 1 Processor -- start 2024-08-31T07:11:59.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.826+0000 7f00045eb740 1 -- start start 2024-08-31T07:11:59.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.826+0000 7f00045eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7c0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.826+0000 7f00045eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7c0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.826+0000 7f00045eb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7c1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.826+0000 7f00045eb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561f7d737550 con 0x561f7d3fa280 2024-08-31T07:11:59.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.826+0000 7f00045eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f7d726ae0 con 0x561f7d41af60 2024-08-31T07:11:59.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.826+0000 7f00045eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561f7d72aa90 con 0x561f7d3f20a0 2024-08-31T07:11:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7c0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7c0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7c0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36290/0 (socket says 172.21.15.32:36290) 2024-08-31T07:11:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7c0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52786/0 (socket says 172.21.15.32:52786) 2024-08-31T07:11:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff6a00640 1 -- 172.21.15.32:0/2186547447 learned_addr learned my addr 172.21.15.32:0/2186547447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7effef400640 1 -- 172.21.15.32:0/3884993072 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7effe4010a90 con 0x7efff810b290 2024-08-31T07:11:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff6a00640 1 -- 172.21.15.32:0/2186547447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 msgr2=0x561f7d7c0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff6a00640 1 --2- 172.21.15.32:0/2186547447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7c0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff6a00640 1 -- 172.21.15.32:0/2186547447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 msgr2=0x561f7d7c1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff6a00640 1 --2- 172.21.15.32:0/2186547447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff6a00640 1 -- 172.21.15.32:0/2186547447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7d486540 con 0x561f7d3f20a0 2024-08-31T07:11:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.827+0000 7efff6a00640 1 --2- 172.21.15.32:0/2186547447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7c0fe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7effe800bd50 tx=0x7effe80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f8f121315bed8e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.828+0000 7effe3e00640 1 -- 172.21.15.32:0/2186547447 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8005220 con 0x561f7d3f20a0 2024-08-31T07:11:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.828+0000 7effe3e00640 1 -- 172.21.15.32:0/2186547447 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe800be40 con 0x561f7d3f20a0 2024-08-31T07:11:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.828+0000 7effe3e00640 1 -- 172.21.15.32:0/2186547447 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe80117c0 con 0x561f7d3f20a0 2024-08-31T07:11:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.828+0000 7effef400640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effdc0b4b80 0x7effdc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7f00045eb740 1 -- 172.21.15.32:0/2186547447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 msgr2=0x561f7d7c0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7f00045eb740 1 --2- 172.21.15.32:0/2186547447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7c0fe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7effe800bd50 tx=0x7effe80107a0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7f00045eb740 1 -- 172.21.15.32:0/2186547447 shutdown_connections 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7f00045eb740 1 --2- 172.21.15.32:0/2186547447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7c1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7f00045eb740 1 --2- 172.21.15.32:0/2186547447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7c0fe0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7f00045eb740 1 --2- 172.21.15.32:0/2186547447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7c0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7f00045eb740 1 -- 172.21.15.32:0/2186547447 >> 172.21.15.32:0/2186547447 conn(0x561f7d40f350 msgr2=0x561f7d7be990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7effef400640 1 -- 172.21.15.32:0/3884993072 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7effe40e5480 con 0x7efff810b290 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7efff7400640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effdc0b4b80 0x7effdc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7f00045eb740 1 -- 172.21.15.32:0/2186547447 shutdown_connections 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.829+0000 7f00045eb740 1 -- 172.21.15.32:0/2186547447 wait complete. 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.830+0000 7efff7400640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effdc0b4b80 0x7effdc0b7090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efff0005e30 tx=0x7efff0005c50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.830+0000 7f00045eb740 1 -- start start 2024-08-31T07:11:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.830+0000 7f00045eb740 1 Processor -- start 2024-08-31T07:11:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.830+0000 7f00045eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.830+0000 7f00045eb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.830+0000 7f00045eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7d8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.830+0000 7f00045eb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561f7d5895f0 con 0x561f7d3fa280 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.830+0000 7f00045eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f7d6adbd0 con 0x561f7d41af60 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.830+0000 7f00045eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561f7d7cf280 con 0x561f7d3f20a0 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7efff6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7efff6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7efff6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36300/0 (socket says 172.21.15.32:36300) 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7f00045eb740 10 client.0 _ll_register_callbacks cb 0x561f7d6d3210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7efff6a00640 1 -- 172.21.15.32:0/2599571446 learned_addr learned my addr 172.21.15.32:0/2599571446 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7efff7400640 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7d8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7efff6a00640 1 -- 172.21.15.32:0/2599571446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 msgr2=0x561f7d7d8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7efff6a00640 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7d8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7efff6a00640 1 -- 172.21.15.32:0/2599571446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 msgr2=0x561f7d7cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7efff6a00640 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.831+0000 7efff6a00640 1 -- 172.21.15.32:0/2599571446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7d6ee650 con 0x561f7d3f20a0 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.832+0000 7efff6a00640 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7cc2c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7effe8011010 tx=0x7effe8011770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba76365263fab7cd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.832+0000 7effe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.832+0000 7effe0c00640 1 -- 172.21.15.32:0/2599571446 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8011d70 con 0x561f7d3f20a0 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.832+0000 7effe0c00640 1 -- 172.21.15.32:0/2599571446 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7effe8004020 con 0x561f7d3f20a0 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.832+0000 7f00045eb740 1 -- 172.21.15.32:0/2599571446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561f7d7394a0 con 0x561f7d3f20a0 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.832+0000 7effe0c00640 1 -- 172.21.15.32:0/2599571446 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effe8004220 con 0x561f7d3f20a0 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.832+0000 7f00045eb740 10 client.25252 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.832+0000 7f00045eb740 1 -- 172.21.15.32:0/2599571446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561f7d7db110 con 0x561f7d3f20a0 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.832+0000 7effe0c00640 1 -- 172.21.15.32:0/2599571446 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561f7d7394a0 con 0x561f7d3f20a0 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.833+0000 7effe0c00640 1 -- 172.21.15.32:0/2599571446 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effe802d070 con 0x561f7d3f20a0 2024-08-31T07:11:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.833+0000 7f00045eb740 10 client.25252 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:59.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.833+0000 7f00045eb740 10 client.25252 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-31T07:11:59.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.833+0000 7f00045eb740 20 client.25252 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:59.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.833+0000 7f00045eb740 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x561f7d3d98e0 0x561f7d7dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.833+0000 7f00045eb740 4 client.25252 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:11:59.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.833+0000 7f00045eb740 1 -- 172.21.15.32:0/2599571446 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f7d734f40 con 0x561f7d3d98e0 2024-08-31T07:11:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.833+0000 7efff6000640 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x561f7d3d98e0 0x561f7d7dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.834+0000 7efff6000640 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x561f7d3d98e0 0x561f7d7dc510 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52acb535c6a85f03 server_cookie=bc37d8d5fc1cfe1d in_seq=0 out_seq=0 2024-08-31T07:11:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.834+0000 7effe0c00640 10 client.25252 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:11:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7effe0c00640 1 -- 172.21.15.32:0/2599571446 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7effe4085e50 con 0x561f7d3d98e0 2024-08-31T07:11:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7effe0c00640 10 client.25252 handle_command_reply: tid=0 2024-08-31T07:11:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 client.25252 shutdown 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 -- 172.21.15.32:0/2599571446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 msgr2=0x561f7d7cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7cc2c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7effe8011010 tx=0x7effe8011770 comp rx=0 tx=0).stop 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 -- 172.21.15.32:0/2599571446 shutdown_connections 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x561f7d3d98e0 0x561f7d7dc510 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7d8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 --2- 172.21.15.32:0/2599571446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7cc2c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7effe0c00640 0 client.25252 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 -- 172.21.15.32:0/2599571446 >> 172.21.15.32:0/2599571446 conn(0x561f7d40f350 msgr2=0x561f7d7c3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 -- 172.21.15.32:0/2599571446 shutdown_connections 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 1 -- 172.21.15.32:0/2599571446 wait complete. 2024-08-31T07:11:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.835+0000 7f00045eb740 20 client.25252 trim_cache size 0 max 16384 2024-08-31T07:11:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.854+0000 7f00045eb740 1 Processor -- start 2024-08-31T07:11:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.854+0000 7f00045eb740 1 -- start start 2024-08-31T07:11:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7f00045eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7d84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7f00045eb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7d8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7f00045eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7f00045eb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561f7d7dca50 con 0x561f7d3fa280 2024-08-31T07:11:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7f00045eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f7d7dd120 con 0x561f7d41af60 2024-08-31T07:11:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7f00045eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561f7d7dd7f0 con 0x561f7d3f20a0 2024-08-31T07:11:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7efff6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7d8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7efff6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7d8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40948/0 (socket says 172.21.15.32:40948) 2024-08-31T07:11:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7efff6000640 1 -- 172.21.15.32:0/1052710422 learned_addr learned my addr 172.21.15.32:0/1052710422 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7efff7400640 1 --2- 172.21.15.32:0/1052710422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.855+0000 7efff6a00640 1 --2- 172.21.15.32:0/1052710422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7d84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.856+0000 7efff6000640 1 -- 172.21.15.32:0/1052710422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 msgr2=0x561f7d7d84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.856+0000 7efff6000640 1 --2- 172.21.15.32:0/1052710422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7d84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.856+0000 7efff6000640 1 -- 172.21.15.32:0/1052710422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 msgr2=0x561f7d7c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.856+0000 7efff6000640 1 --2- 172.21.15.32:0/1052710422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.856+0000 7efff6000640 1 -- 172.21.15.32:0/1052710422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7d7c2d90 con 0x561f7d3fa280 2024-08-31T07:11:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.856+0000 7efff6000640 1 --2- 172.21.15.32:0/1052710422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7d8a20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7effe40abbc0 tx=0x7effe4093040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf2bebad5c8a8a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.856+0000 7effe3400640 1 -- 172.21.15.32:0/1052710422 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe40e2ff0 con 0x561f7d3fa280 2024-08-31T07:11:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.856+0000 7effe3400640 1 -- 172.21.15.32:0/1052710422 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561f7d734f40 con 0x561f7d3fa280 2024-08-31T07:11:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.856+0000 7effe3400640 1 -- 172.21.15.32:0/1052710422 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe4093310 con 0x561f7d3fa280 2024-08-31T07:11:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.857+0000 7f00045eb740 1 -- 172.21.15.32:0/1052710422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 msgr2=0x561f7d7d8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.857+0000 7f00045eb740 1 --2- 172.21.15.32:0/1052710422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7d8a20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7effe40abbc0 tx=0x7effe4093040 comp rx=0 tx=0).stop 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.857+0000 7f00045eb740 1 -- 172.21.15.32:0/1052710422 shutdown_connections 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.857+0000 7f00045eb740 1 --2- 172.21.15.32:0/1052710422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.857+0000 7f00045eb740 1 --2- 172.21.15.32:0/1052710422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3fa280 0x561f7d7d8a20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.857+0000 7f00045eb740 1 --2- 172.21.15.32:0/1052710422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3f20a0 0x561f7d7d84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.857+0000 7f00045eb740 1 -- 172.21.15.32:0/1052710422 >> 172.21.15.32:0/1052710422 conn(0x561f7d40f350 msgr2=0x561f7d7b9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.857+0000 7f00045eb740 1 -- 172.21.15.32:0/1052710422 shutdown_connections 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.857+0000 7f00045eb740 1 -- 172.21.15.32:0/1052710422 wait complete. 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.858+0000 7f00045eb740 1 -- start start 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.858+0000 7f00045eb740 1 Processor -- start 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.858+0000 7f00045eb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3f20a0 0x561f7d7e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.858+0000 7f00045eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3fa280 0x561f7d7e2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.858+0000 7f00045eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.858+0000 7f00045eb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561f7d7de590 con 0x561f7d3f20a0 2024-08-31T07:11:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.858+0000 7f00045eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f7d7ddec0 con 0x561f7d41af60 2024-08-31T07:11:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.858+0000 7f00045eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561f7d7dec60 con 0x561f7d3fa280 2024-08-31T07:11:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.858+0000 7f00045eb740 10 client.0 _ll_register_callbacks cb 0x7effe80054d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:11:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3f20a0 0x561f7d7e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3f20a0 0x561f7d7e71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40962/0 (socket says 172.21.15.32:40962) 2024-08-31T07:11:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff6a00640 1 -- 172.21.15.32:0/3424309118 learned_addr learned my addr 172.21.15.32:0/3424309118 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:11:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff6000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3fa280 0x561f7d7e2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff7400640 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff6a00640 1 -- 172.21.15.32:0/3424309118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3fa280 msgr2=0x561f7d7e2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff6a00640 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3fa280 0x561f7d7e2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff6a00640 1 -- 172.21.15.32:0/3424309118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 msgr2=0x561f7d7e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:11:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff6a00640 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7e2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff6a00640 1 -- 172.21.15.32:0/3424309118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7d7e3450 con 0x561f7d3f20a0 2024-08-31T07:11:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.859+0000 7efff6000640 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3fa280 0x561f7d7e2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:11:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.860+0000 7efff6a00640 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3f20a0 0x561f7d7e71f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7effe80255f0 tx=0x7effe8011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3945136de6e9a5d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:11:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.860+0000 7effe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.860+0000 7effe0c00640 1 -- 172.21.15.32:0/3424309118 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe803d060 con 0x561f7d3f20a0 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.860+0000 7effe0c00640 1 -- 172.21.15.32:0/3424309118 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7effe8011d70 con 0x561f7d3f20a0 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.860+0000 7f00045eb740 1 -- 172.21.15.32:0/3424309118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561f7d734f40 con 0x561f7d3f20a0 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.860+0000 7effe0c00640 1 -- 172.21.15.32:0/3424309118 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effe8032070 con 0x561f7d3f20a0 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.860+0000 7f00045eb740 10 client.15240 fetch_fsmap learned FSMap version 28 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.860+0000 7f00045eb740 1 -- 172.21.15.32:0/3424309118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561f7d849b60 con 0x561f7d3f20a0 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.861+0000 7effe0c00640 1 -- 172.21.15.32:0/3424309118 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561f7d734f40 con 0x561f7d3f20a0 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.861+0000 7effe0c00640 1 -- 172.21.15.32:0/3424309118 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effe802d040 con 0x561f7d3f20a0 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.861+0000 7f00045eb740 10 client.15240 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.861+0000 7f00045eb740 10 client.15240 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.861+0000 7f00045eb740 20 client.15240 populate_metadata read hostname 'smithi032' 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.861+0000 7f00045eb740 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x561f7d3aec90 0x561f7d84d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:11:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.861+0000 7f00045eb740 4 client.15240 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:11:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.861+0000 7f00045eb740 1 -- 172.21.15.32:0/3424309118 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "status"}) -- 0x561f7d732680 con 0x561f7d3aec90 2024-08-31T07:11:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.861+0000 7efff6000640 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x561f7d3aec90 0x561f7d84d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:11:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.862+0000 7efff6000640 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x561f7d3aec90 0x561f7d84d360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba783dac6e53e5d3 server_cookie=6317416d8b35a00a in_seq=0 out_seq=0 2024-08-31T07:11:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.862+0000 7effe0c00640 10 client.15240 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:11:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.862+0000 7effe0c00640 1 -- 172.21.15.32:0/3424309118 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561f7d732680 con 0x561f7d3aec90 2024-08-31T07:11:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.862+0000 7effe0c00640 10 client.15240 handle_command_reply: tid=0 2024-08-31T07:11:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.862+0000 7f00045eb740 1 client.15240 shutdown 2024-08-31T07:11:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.862+0000 7f00045eb740 1 -- 172.21.15.32:0/3424309118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3f20a0 msgr2=0x561f7d7e71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.862+0000 7f00045eb740 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3f20a0 0x561f7d7e71f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7effe80255f0 tx=0x7effe8011440 comp rx=0 tx=0).stop 2024-08-31T07:11:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.863+0000 7f00045eb740 1 -- 172.21.15.32:0/3424309118 shutdown_connections 2024-08-31T07:11:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.863+0000 7f00045eb740 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x561f7d3aec90 0x561f7d84d360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.863+0000 7f00045eb740 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7d41af60 0x561f7d7e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.863+0000 7f00045eb740 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f7d3fa280 0x561f7d7e2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.863+0000 7effe0c00640 0 client.15240 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.863+0000 7f00045eb740 1 --2- 172.21.15.32:0/3424309118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f7d3f20a0 0x561f7d7e71f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.863+0000 7f00045eb740 1 -- 172.21.15.32:0/3424309118 >> 172.21.15.32:0/3424309118 conn(0x561f7d40f350 msgr2=0x561f7d7b9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.863+0000 7f00045eb740 1 -- 172.21.15.32:0/3424309118 shutdown_connections 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.863+0000 7f00045eb740 1 -- 172.21.15.32:0/3424309118 wait complete. 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.863+0000 7f00045eb740 20 client.15240 trim_cache size 0 max 16384 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 3, 2024-08-31T07:11:59.867 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24692, 2024-08-31T07:11:59.868 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:11:59.868 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:11:59.868 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:11:59.868 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 118.007174569, 2024-08-31T07:11:59.868 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2024-08-31T07:11:59.868 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:11:59.868 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T07:11:59.868 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 157.29818852299999 2024-08-31T07:11:59.868 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:11:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 -- 172.21.15.32:0/3884993072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effdc0b4b80 msgr2=0x7effdc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effdc0b4b80 0x7effdc0b7090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efff0005e30 tx=0x7efff0005c50 comp rx=0 tx=0).stop 2024-08-31T07:11:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 -- 172.21.15.32:0/3884993072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810b290 msgr2=0x7efff819c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:11:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810b290 0x7efff819c7b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efff806c660 tx=0x7effe400ee50 comp rx=0 tx=0).stop 2024-08-31T07:11:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 -- 172.21.15.32:0/3884993072 shutdown_connections 2024-08-31T07:11:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effdc0b4b80 0x7effdc0b7090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 0x7efff81a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810b290 0x7efff819c7b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 --2- 172.21.15.32:0/3884993072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:11:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 -- 172.21.15.32:0/3884993072 >> 172.21.15.32:0/3884993072 conn(0x7efff8101420 msgr2=0x7efff810a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:11:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 -- 172.21.15.32:0/3884993072 shutdown_connections 2024-08-31T07:11:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:11:59.864+0000 7efff7e00640 1 -- 172.21.15.32:0/3884993072 wait complete. 2024-08-31T07:12:00.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:00 smithi032 ceph-mon[17252]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:12:00.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.i status 2024-08-31T07:12:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:00 smithi073 ceph-mon[21995]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:12:00.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:00 smithi084 ceph-mon[22387]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:12:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:01.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.729+0000 7f527b000640 1 -- 172.21.15.32:0/2566844847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274109b90 msgr2=0x7f52741103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:01.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.729+0000 7f527b000640 1 --2- 172.21.15.32:0/2566844847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274109b90 0x7f52741103c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f526c009ca0 tx=0x7f526c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.729+0000 7f527b000640 1 -- 172.21.15.32:0/2566844847 shutdown_connections 2024-08-31T07:12:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.729+0000 7f527b000640 1 --2- 172.21.15.32:0/2566844847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274109b90 0x7f52741103c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.729+0000 7f527b000640 1 --2- 172.21.15.32:0/2566844847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5274109240 0x7f5274109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.729+0000 7f527b000640 1 --2- 172.21.15.32:0/2566844847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5274103900 0x7f5274103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.729+0000 7f527b000640 1 -- 172.21.15.32:0/2566844847 >> 172.21.15.32:0/2566844847 conn(0x7f527407b5f0 msgr2=0x7f527407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.729+0000 7f527b000640 1 -- 172.21.15.32:0/2566844847 shutdown_connections 2024-08-31T07:12:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.729+0000 7f527b000640 1 -- 172.21.15.32:0/2566844847 wait complete. 2024-08-31T07:12:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.730+0000 7f527b000640 1 Processor -- start 2024-08-31T07:12:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.730+0000 7f527b000640 1 -- start start 2024-08-31T07:12:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f527b000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5274103900 0x7f527419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f527b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5274109240 0x7f527419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f527b000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274109b90 0x7f527419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f527b000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5274113850 con 0x7f5274109b90 2024-08-31T07:12:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f527b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f52741136d0 con 0x7f5274109240 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f527b000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f52741139d0 con 0x7f5274103900 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f5273e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5274103900 0x7f527419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f5273e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5274103900 0x7f527419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36338/0 (socket says 172.21.15.32:36338) 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f5273e00640 1 -- 172.21.15.32:0/2380407974 learned_addr learned my addr 172.21.15.32:0/2380407974 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f526b400640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5274109240 0x7f527419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f5279000640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274109b90 0x7f527419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f5273e00640 1 -- 172.21.15.32:0/2380407974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5274109240 msgr2=0x7f527419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f5273e00640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5274109240 0x7f527419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f5273e00640 1 -- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274109b90 msgr2=0x7f527419f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f5273e00640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274109b90 0x7f527419f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f5273e00640 1 -- 172.21.15.32:0/2380407974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52741a00f0 con 0x7f5274103900 2024-08-31T07:12:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f526b400640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5274109240 0x7f527419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.731+0000 7f5279000640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274109b90 0x7f527419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.732+0000 7f5273e00640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5274103900 0x7f527419a180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f526400e9f0 tx=0x7f526400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.732+0000 7f5271600640 1 -- 172.21.15.32:0/2380407974 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526400cd80 con 0x7f5274103900 2024-08-31T07:12:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.732+0000 7f527b000640 1 -- 172.21.15.32:0/2380407974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52741a03e0 con 0x7f5274103900 2024-08-31T07:12:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.732+0000 7f5271600640 1 -- 172.21.15.32:0/2380407974 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52640108e0 con 0x7f5274103900 2024-08-31T07:12:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.732+0000 7f5271600640 1 -- 172.21.15.32:0/2380407974 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264018980 con 0x7f5274103900 2024-08-31T07:12:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.732+0000 7f527b000640 1 -- 172.21.15.32:0/2380407974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52741a6b40 con 0x7f5274103900 2024-08-31T07:12:01.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.735+0000 7f5281c13740 1 Processor -- start 2024-08-31T07:12:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.735+0000 7f5281c13740 1 -- start start 2024-08-31T07:12:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5271600640 1 -- 172.21.15.32:0/2380407974 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f5264010430 con 0x7f5274103900 2024-08-31T07:12:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5281c13740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 0x557caf407d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5281c13740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf408240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5281c13740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 0x557caf408780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5281c13740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557caf37a550 con 0x557caf05df60 2024-08-31T07:12:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5281c13740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557caf369ae0 con 0x557caf03d280 2024-08-31T07:12:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5281c13740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557caf36da90 con 0x557caf0350a0 2024-08-31T07:12:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5279000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 0x557caf407d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5273e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf408240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5273e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf408240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36354/0 (socket says 172.21.15.32:36354) 2024-08-31T07:12:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5279000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 0x557caf407d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40990/0 (socket says 172.21.15.32:40990) 2024-08-31T07:12:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.736+0000 7f5273e00640 1 -- 172.21.15.32:0/3975108799 learned_addr learned my addr 172.21.15.32:0/3975108799 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.737+0000 7f526b400640 1 --2- 172.21.15.32:0/3975108799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 0x557caf408780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.737+0000 7f5273e00640 1 -- 172.21.15.32:0/3975108799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 msgr2=0x557caf408780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.737+0000 7f5273e00640 1 --2- 172.21.15.32:0/3975108799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 0x557caf408780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.737+0000 7f5273e00640 1 -- 172.21.15.32:0/3975108799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 msgr2=0x557caf407d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.737+0000 7f5273e00640 1 --2- 172.21.15.32:0/3975108799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 0x557caf407d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.737+0000 7f5273e00640 1 -- 172.21.15.32:0/3975108799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557caf0c9540 con 0x557caf0350a0 2024-08-31T07:12:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.737+0000 7f526b400640 1 --2- 172.21.15.32:0/3975108799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 0x557caf408780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.737+0000 7f5279000640 1 --2- 172.21.15.32:0/3975108799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 0x557caf407d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.737+0000 7f5273e00640 1 --2- 172.21.15.32:0/3975108799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf408240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f52640eb6d0 tx=0x7f52640ebb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a835442e614c3aac server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:01.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.738+0000 7f5269600640 1 -- 172.21.15.32:0/3975108799 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52640f8070 con 0x557caf0350a0 2024-08-31T07:12:01.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.738+0000 7f5271600640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52580b4c00 0x7f52580b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.738+0000 7f5269600640 1 -- 172.21.15.32:0/3975108799 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52640aba10 con 0x557caf0350a0 2024-08-31T07:12:01.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.738+0000 7f5269600640 1 -- 172.21.15.32:0/3975108799 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264073140 con 0x557caf0350a0 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.738+0000 7f5271600640 1 -- 172.21.15.32:0/2380407974 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f52640e55d0 con 0x7f5274103900 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.738+0000 7f5279000640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52580b4c00 0x7f52580b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.739+0000 7f5281c13740 1 -- 172.21.15.32:0/3975108799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 msgr2=0x557caf408240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.739+0000 7f5281c13740 1 --2- 172.21.15.32:0/3975108799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf408240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f52640eb6d0 tx=0x7f52640ebb90 comp rx=0 tx=0).stop 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.739+0000 7f5279000640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52580b4c00 0x7f52580b7110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f526c004af0 tx=0x7f526c005c30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.739+0000 7f5281c13740 1 -- 172.21.15.32:0/3975108799 shutdown_connections 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.739+0000 7f5281c13740 1 --2- 172.21.15.32:0/3975108799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 0x557caf408780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.739+0000 7f5281c13740 1 --2- 172.21.15.32:0/3975108799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf408240 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.739+0000 7f5281c13740 1 --2- 172.21.15.32:0/3975108799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 0x557caf407d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.739+0000 7f5281c13740 1 -- 172.21.15.32:0/3975108799 >> 172.21.15.32:0/3975108799 conn(0x557caf052350 msgr2=0x557caf405bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.739+0000 7f5281c13740 1 -- 172.21.15.32:0/3975108799 shutdown_connections 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.739+0000 7f5281c13740 1 -- 172.21.15.32:0/3975108799 wait complete. 2024-08-31T07:12:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.740+0000 7f5281c13740 1 -- start start 2024-08-31T07:12:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.740+0000 7f5281c13740 1 Processor -- start 2024-08-31T07:12:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.740+0000 7f5281c13740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf4134d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.740+0000 7f5281c13740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 0x557caf413a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5281c13740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 0x557caf41f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5281c13740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557caf1cc5f0 con 0x557caf05df60 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5281c13740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557caf2f0bd0 con 0x557caf03d280 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5281c13740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557caf4164e0 con 0x557caf0350a0 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5281c13740 10 client.0 _ll_register_callbacks cb 0x557caf316210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5273e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf4134d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5273e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf4134d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36360/0 (socket says 172.21.15.32:36360) 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5279000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 0x557caf41f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5273e00640 1 -- 172.21.15.32:0/229926805 learned_addr learned my addr 172.21.15.32:0/229926805 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f526b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 0x557caf413a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5273e00640 1 -- 172.21.15.32:0/229926805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 msgr2=0x557caf413a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5273e00640 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 0x557caf413a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5273e00640 1 -- 172.21.15.32:0/229926805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 msgr2=0x557caf41f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5273e00640 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 0x557caf41f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.741+0000 7f5273e00640 1 -- 172.21.15.32:0/229926805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557caf331650 con 0x557caf0350a0 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.742+0000 7f5273e00640 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf4134d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52640abe90 tx=0x7f5264072710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49b3f15d6287efba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.742+0000 7f5262000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.742+0000 7f5262000640 1 -- 172.21.15.32:0/229926805 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52640720c0 con 0x557caf0350a0 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.742+0000 7f5262000640 1 -- 172.21.15.32:0/229926805 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5264070970 con 0x557caf0350a0 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.742+0000 7f5281c13740 1 -- 172.21.15.32:0/229926805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52640aba10 con 0x557caf0350a0 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.742+0000 7f5262000640 1 -- 172.21.15.32:0/229926805 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5264072310 con 0x557caf0350a0 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.742+0000 7f5281c13740 10 client.25270 fetch_fsmap learned FSMap version 28 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.742+0000 7f5281c13740 1 -- 172.21.15.32:0/229926805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557caf422310 con 0x557caf0350a0 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.742+0000 7f5262000640 1 -- 172.21.15.32:0/229926805 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52640aba10 con 0x557caf0350a0 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.743+0000 7f5262000640 1 -- 172.21.15.32:0/229926805 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f52640eb160 con 0x557caf0350a0 2024-08-31T07:12:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.743+0000 7f5281c13740 10 client.25270 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:12:01.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.743+0000 7f5281c13740 10 client.25270 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T07:12:01.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.743+0000 7f5281c13740 20 client.25270 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:01.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.743+0000 7f5281c13740 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557caf01c8e0 0x557caf4257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.743+0000 7f5281c13740 4 client.25270 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:01.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.743+0000 7f5281c13740 1 -- 172.21.15.32:0/229926805 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557caf37c4a0 con 0x557caf01c8e0 2024-08-31T07:12:01.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.743+0000 7f526b400640 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557caf01c8e0 0x557caf4257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.744+0000 7f526b400640 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557caf01c8e0 0x557caf4257c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed7536c69eed2e96 server_cookie=e50fd4dcd0841a55 in_seq=0 out_seq=0 2024-08-31T07:12:01.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.744+0000 7f5262000640 10 client.25270 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:12:01.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.744+0000 7f5262000640 1 -- 172.21.15.32:0/229926805 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557caf37c4a0 con 0x557caf01c8e0 2024-08-31T07:12:01.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.744+0000 7f5262000640 10 client.25270 handle_command_reply: tid=0 2024-08-31T07:12:01.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.744+0000 7f5281c13740 1 client.25270 shutdown 2024-08-31T07:12:01.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 1 -- 172.21.15.32:0/229926805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 msgr2=0x557caf4134d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:01.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf4134d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52640abe90 tx=0x7f5264072710 comp rx=0 tx=0).stop 2024-08-31T07:12:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 1 -- 172.21.15.32:0/229926805 shutdown_connections 2024-08-31T07:12:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557caf01c8e0 0x557caf4257c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf05df60 0x557caf41f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf03d280 0x557caf413a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 1 --2- 172.21.15.32:0/229926805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf4134d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5262000640 0 client.25270 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:12:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 1 -- 172.21.15.32:0/229926805 >> 172.21.15.32:0/229926805 conn(0x557caf052350 msgr2=0x557caf40aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 1 -- 172.21.15.32:0/229926805 shutdown_connections 2024-08-31T07:12:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 1 -- 172.21.15.32:0/229926805 wait complete. 2024-08-31T07:12:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.745+0000 7f5281c13740 20 client.25270 trim_cache size 0 max 16384 2024-08-31T07:12:01.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.764+0000 7f5281c13740 1 Processor -- start 2024-08-31T07:12:01.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.764+0000 7f5281c13740 1 -- start start 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5281c13740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf405c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5281c13740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 0x557caf4129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5281c13740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf05df60 0x557caf41f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5281c13740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557caf423b00 con 0x557caf03d280 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5281c13740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557caf4241d0 con 0x557caf05df60 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5281c13740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557caf4248a0 con 0x557caf0350a0 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5273e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf405c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f526b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 0x557caf4129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5273e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf405c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36362/0 (socket says 172.21.15.32:36362) 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5279000640 1 --2- 172.21.15.32:0/2280818616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf05df60 0x557caf41f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5273e00640 1 -- 172.21.15.32:0/2280818616 learned_addr learned my addr 172.21.15.32:0/2280818616 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5273e00640 1 -- 172.21.15.32:0/2280818616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf05df60 msgr2=0x557caf41f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5273e00640 1 --2- 172.21.15.32:0/2280818616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf05df60 0x557caf41f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5273e00640 1 -- 172.21.15.32:0/2280818616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 msgr2=0x557caf4129e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5273e00640 1 --2- 172.21.15.32:0/2280818616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 0x557caf4129e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f5273e00640 1 -- 172.21.15.32:0/2280818616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557caf40ab60 con 0x557caf0350a0 2024-08-31T07:12:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.765+0000 7f526b400640 1 --2- 172.21.15.32:0/2280818616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 0x557caf4129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.766+0000 7f5273e00640 1 --2- 172.21.15.32:0/2280818616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf405c60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52640ab670 tx=0x7f52640716a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a8034d08416b513 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.766+0000 7f5268c00640 1 -- 172.21.15.32:0/2280818616 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264072570 con 0x557caf0350a0 2024-08-31T07:12:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.766+0000 7f5268c00640 1 -- 172.21.15.32:0/2280818616 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52640e8b40 con 0x557caf0350a0 2024-08-31T07:12:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.766+0000 7f5268c00640 1 -- 172.21.15.32:0/2280818616 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52640727c0 con 0x557caf0350a0 2024-08-31T07:12:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.767+0000 7f5281c13740 1 -- 172.21.15.32:0/2280818616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 msgr2=0x557caf405c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.767+0000 7f5281c13740 1 --2- 172.21.15.32:0/2280818616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf405c60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52640ab670 tx=0x7f52640716a0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.767+0000 7f5281c13740 1 -- 172.21.15.32:0/2280818616 shutdown_connections 2024-08-31T07:12:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.767+0000 7f5281c13740 1 --2- 172.21.15.32:0/2280818616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf05df60 0x557caf41f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.767+0000 7f5281c13740 1 --2- 172.21.15.32:0/2280818616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 0x557caf4129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.767+0000 7f5281c13740 1 --2- 172.21.15.32:0/2280818616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf405c60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.767+0000 7f5281c13740 1 -- 172.21.15.32:0/2280818616 >> 172.21.15.32:0/2280818616 conn(0x557caf052350 msgr2=0x557caf405550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.767+0000 7f5281c13740 1 -- 172.21.15.32:0/2280818616 shutdown_connections 2024-08-31T07:12:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.767+0000 7f5281c13740 1 -- 172.21.15.32:0/2280818616 wait complete. 2024-08-31T07:12:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.768+0000 7f5281c13740 1 -- start start 2024-08-31T07:12:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.768+0000 7f5281c13740 1 Processor -- start 2024-08-31T07:12:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.768+0000 7f5281c13740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf42e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.768+0000 7f5281c13740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 0x557caf4276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.768+0000 7f5281c13740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf05df60 0x557caf427c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.768+0000 7f5281c13740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557caf425640 con 0x557caf03d280 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.768+0000 7f5281c13740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557caf424f70 con 0x557caf05df60 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.768+0000 7f5281c13740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557caf425d10 con 0x557caf0350a0 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.768+0000 7f5281c13740 10 client.0 _ll_register_callbacks cb 0x7f52640eb520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.768+0000 7f5273e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf42e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5273e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf42e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36376/0 (socket says 172.21.15.32:36376) 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5273e00640 1 -- 172.21.15.32:0/1503021934 learned_addr learned my addr 172.21.15.32:0/1503021934 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f526b400640 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 0x557caf4276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5279000640 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf05df60 0x557caf427c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5273e00640 1 -- 172.21.15.32:0/1503021934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf05df60 msgr2=0x557caf427c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5273e00640 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf05df60 0x557caf427c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5273e00640 1 -- 172.21.15.32:0/1503021934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 msgr2=0x557caf4276d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5273e00640 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 0x557caf4276d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5273e00640 1 -- 172.21.15.32:0/1503021934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557caf428380 con 0x557caf0350a0 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f526b400640 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 0x557caf4276d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5273e00640 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf42e220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5264072eb0 tx=0x7f52640e3050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d52e16c2b4fbc887 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5262000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.769+0000 7f5262000640 1 -- 172.21.15.32:0/1503021934 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264072a10 con 0x557caf0350a0 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5281c13740 1 -- 172.21.15.32:0/1503021934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557caf377f40 con 0x557caf0350a0 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5262000640 1 -- 172.21.15.32:0/1503021934 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f52640a84f0 con 0x557caf0350a0 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5262000640 1 -- 172.21.15.32:0/1503021934 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5264072c70 con 0x557caf0350a0 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5281c13740 10 client.25276 fetch_fsmap learned FSMap version 28 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5281c13740 1 -- 172.21.15.32:0/1503021934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557caf428380 con 0x557caf0350a0 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5262000640 1 -- 172.21.15.32:0/1503021934 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557caf377f40 con 0x557caf0350a0 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5262000640 1 -- 172.21.15.32:0/1503021934 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f52640a8190 con 0x557caf0350a0 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5281c13740 10 client.25276 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:12:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5281c13740 10 client.25276 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T07:12:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5281c13740 20 client.25276 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5281c13740 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557caeff1c90 0x557caf493e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5281c13740 4 client.25276 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:12:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.770+0000 7f5281c13740 1 -- 172.21.15.32:0/1503021934 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "status"}) -- 0x557caf375680 con 0x557caeff1c90 2024-08-31T07:12:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.771+0000 7f526b400640 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557caeff1c90 0x557caf493e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.771+0000 7f526b400640 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557caeff1c90 0x557caf493e10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7aeb062bcc51358 server_cookie=e815dd29c681eab1 in_seq=0 out_seq=0 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.771+0000 7f5262000640 10 client.25276 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.771+0000 7f5262000640 1 -- 172.21.15.32:0/1503021934 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557caf375680 con 0x557caeff1c90 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.771+0000 7f5262000640 10 client.25276 handle_command_reply: tid=0 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.771+0000 7f5281c13740 1 client.25276 shutdown 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 1 -- 172.21.15.32:0/1503021934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 msgr2=0x557caf42e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf42e220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5264072eb0 tx=0x7f52640e3050 comp rx=0 tx=0).stop 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 1 -- 172.21.15.32:0/1503021934 shutdown_connections 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557caeff1c90 0x557caf493e10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557caf05df60 0x557caf427c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557caf03d280 0x557caf4276d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5262000640 0 client.25276 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:12:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 1 --2- 172.21.15.32:0/1503021934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557caf0350a0 0x557caf42e220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 1 -- 172.21.15.32:0/1503021934 >> 172.21.15.32:0/1503021934 conn(0x557caf052350 msgr2=0x557caf413280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 1 -- 172.21.15.32:0/1503021934 shutdown_connections 2024-08-31T07:12:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 1 -- 172.21.15.32:0/1503021934 wait complete. 2024-08-31T07:12:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.772+0000 7f5281c13740 20 client.25276 trim_cache size 0 max 16384 2024-08-31T07:12:01.776 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:01.776 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:12:01.776 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 2, 2024-08-31T07:12:01.777 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24694, 2024-08-31T07:12:01.777 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:12:01.777 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:12:01.777 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:12:01.777 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 120.922380586, 2024-08-31T07:12:01.777 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2024-08-31T07:12:01.777 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:12:01.777 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T07:12:01.777 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 156.68008989800001 2024-08-31T07:12:01.777 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 -- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52580b4c00 msgr2=0x7f52580b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52580b4c00 0x7f52580b7110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f526c004af0 tx=0x7f526c005c30 comp rx=0 tx=0).stop 2024-08-31T07:12:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 -- 172.21.15.32:0/2380407974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5274103900 msgr2=0x7f527419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5274103900 0x7f527419a180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f526400e9f0 tx=0x7f526400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 -- 172.21.15.32:0/2380407974 shutdown_connections 2024-08-31T07:12:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52580b4c00 0x7f52580b7110 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5274109b90 0x7f527419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5274109240 0x7f527419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 --2- 172.21.15.32:0/2380407974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5274103900 0x7f527419a180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 -- 172.21.15.32:0/2380407974 >> 172.21.15.32:0/2380407974 conn(0x7f527407b5f0 msgr2=0x7f5274107290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 -- 172.21.15.32:0/2380407974 shutdown_connections 2024-08-31T07:12:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:01.773+0000 7f527b000640 1 -- 172.21.15.32:0/2380407974 wait complete. 2024-08-31T07:12:02.522 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:02 smithi032 ceph-mon[17252]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:12:02.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.a status 2024-08-31T07:12:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:02 smithi073 ceph-mon[21995]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:12:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:02 smithi084 ceph-mon[22387]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:12:02.934 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.563+0000 7ff8ca200640 1 -- 172.21.15.32:0/3671444527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c4103750 msgr2=0x7ff8c4103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.563+0000 7ff8ca200640 1 --2- 172.21.15.32:0/3671444527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c4103750 0x7ff8c4103b60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009900 tx=0x7ff8b802a410 comp rx=0 tx=0).stop 2024-08-31T07:12:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.563+0000 7ff8ca200640 1 -- 172.21.15.32:0/3671444527 shutdown_connections 2024-08-31T07:12:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.563+0000 7ff8ca200640 1 --2- 172.21.15.32:0/3671444527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8c41099e0 0x7ff8c4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.563+0000 7ff8ca200640 1 --2- 172.21.15.32:0/3671444527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8c4109090 0x7ff8c41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.563+0000 7ff8ca200640 1 --2- 172.21.15.32:0/3671444527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c4103750 0x7ff8c4103b60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.563+0000 7ff8ca200640 1 -- 172.21.15.32:0/3671444527 >> 172.21.15.32:0/3671444527 conn(0x7ff8c40ff1a0 msgr2=0x7ff8c41015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.563+0000 7ff8ca200640 1 -- 172.21.15.32:0/3671444527 shutdown_connections 2024-08-31T07:12:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.563+0000 7ff8ca200640 1 -- 172.21.15.32:0/3671444527 wait complete. 2024-08-31T07:12:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.564+0000 7ff8ca200640 1 Processor -- start 2024-08-31T07:12:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8ca200640 1 -- start start 2024-08-31T07:12:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8ca200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8c4103750 0x7ff8c419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8ca200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c4109090 0x7ff8c419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8ca200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8c41099e0 0x7ff8c419f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8ca200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c41136e0 con 0x7ff8c4109090 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8ca200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c4113560 con 0x7ff8c41099e0 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8ca200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c4113860 con 0x7ff8c4103750 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8c3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c4109090 0x7ff8c419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8c3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c4109090 0x7ff8c419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41046/0 (socket says 172.21.15.32:41046) 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8c3e00640 1 -- 172.21.15.32:0/3651116903 learned_addr learned my addr 172.21.15.32:0/3651116903 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.565+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8c4103750 0x7ff8c419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.566+0000 7ff8c9800640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8c41099e0 0x7ff8c419f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.566+0000 7ff8c3e00640 1 -- 172.21.15.32:0/3651116903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8c4103750 msgr2=0x7ff8c419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.566+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8c4103750 0x7ff8c419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.566+0000 7ff8c3e00640 1 -- 172.21.15.32:0/3651116903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8c41099e0 msgr2=0x7ff8c419f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.566+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8c41099e0 0x7ff8c419f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.566+0000 7ff8c3e00640 1 -- 172.21.15.32:0/3651116903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c41a0080 con 0x7ff8c4109090 2024-08-31T07:12:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.566+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c4109090 0x7ff8c419a6b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000d740 tx=0x7ff8b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.566+0000 7ff8c1600640 1 -- 172.21.15.32:0/3651116903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0017070 con 0x7ff8c4109090 2024-08-31T07:12:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.566+0000 7ff8ca200640 1 -- 172.21.15.32:0/3651116903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c41a0370 con 0x7ff8c4109090 2024-08-31T07:12:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.567+0000 7ff8c1600640 1 -- 172.21.15.32:0/3651116903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8b000de50 con 0x7ff8c4109090 2024-08-31T07:12:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.567+0000 7ff8c1600640 1 -- 172.21.15.32:0/3651116903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000f620 con 0x7ff8c4109090 2024-08-31T07:12:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.567+0000 7ff8ca200640 1 -- 172.21.15.32:0/3651116903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c41a6ad0 con 0x7ff8c4109090 2024-08-31T07:12:03.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.570+0000 7ff8d247d740 1 Processor -- start 2024-08-31T07:12:03.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.570+0000 7ff8d247d740 1 -- start start 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.570+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3fbf60 0x55a24d7a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.570+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.570+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 0x55a24d7a2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.570+0000 7ff8d247d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a24d718550 con 0x55a24d3d30a0 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.570+0000 7ff8d247d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a24d707ae0 con 0x55a24d3db280 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.570+0000 7ff8d247d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a24d70ba90 con 0x55a24d3fbf60 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.570+0000 7ff8c1600640 1 -- 172.21.15.32:0/3651116903 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7ff8b001a070 con 0x7ff8c4109090 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 0x55a24d7a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7a1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41056/0 (socket says 172.21.15.32:41056) 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c8e00640 1 -- 172.21.15.32:0/2470657893 learned_addr learned my addr 172.21.15.32:0/2470657893 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c9800640 1 --2- 172.21.15.32:0/2470657893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3fbf60 0x55a24d7a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c9800640 1 -- 172.21.15.32:0/2470657893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 msgr2=0x55a24d7a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c9800640 1 --2- 172.21.15.32:0/2470657893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 0x55a24d7a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c9800640 1 -- 172.21.15.32:0/2470657893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 msgr2=0x55a24d7a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c9800640 1 --2- 172.21.15.32:0/2470657893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c9800640 1 -- 172.21.15.32:0/2470657893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24d467540 con 0x55a24d3fbf60 2024-08-31T07:12:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.571+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/2470657893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 0x55a24d7a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.572+0000 7ff8c9800640 1 --2- 172.21.15.32:0/2470657893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3fbf60 0x55a24d7a1aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009eb0 tx=0x7ff8bc0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef320067bac4f6e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.572+0000 7ff8b6000640 1 -- 172.21.15.32:0/2470657893 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc003b70 con 0x55a24d3fbf60 2024-08-31T07:12:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.572+0000 7ff8b6000640 1 -- 172.21.15.32:0/2470657893 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8bc015070 con 0x55a24d3fbf60 2024-08-31T07:12:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.572+0000 7ff8b6000640 1 -- 172.21.15.32:0/2470657893 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc01a810 con 0x55a24d3fbf60 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8c1600640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8ac0b4b80 0x7ff8ac0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8c1600640 1 -- 172.21.15.32:0/3651116903 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff8b00e4c00 con 0x7ff8c4109090 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8d247d740 1 -- 172.21.15.32:0/2470657893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3fbf60 msgr2=0x55a24d7a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8d247d740 1 --2- 172.21.15.32:0/2470657893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3fbf60 0x55a24d7a1aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009eb0 tx=0x7ff8bc0107a0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8ac0b4b80 0x7ff8ac0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8d247d740 1 -- 172.21.15.32:0/2470657893 shutdown_connections 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8d247d740 1 --2- 172.21.15.32:0/2470657893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 0x55a24d7a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8d247d740 1 --2- 172.21.15.32:0/2470657893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8d247d740 1 --2- 172.21.15.32:0/2470657893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3fbf60 0x55a24d7a1aa0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8d247d740 1 -- 172.21.15.32:0/2470657893 >> 172.21.15.32:0/2470657893 conn(0x55a24d3f0350 msgr2=0x55a24d79f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8d247d740 1 -- 172.21.15.32:0/2470657893 shutdown_connections 2024-08-31T07:12:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.573+0000 7ff8d247d740 1 -- 172.21.15.32:0/2470657893 wait complete. 2024-08-31T07:12:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.574+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8ac0b4b80 0x7ff8ac0b7090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80096c0 tx=0x7ff8b8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.574+0000 7ff8d247d740 1 -- start start 2024-08-31T07:12:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.574+0000 7ff8d247d740 1 Processor -- start 2024-08-31T07:12:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3d30a0 0x55a24d7ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3db280 0x55a24d7ad810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3fbf60 0x55a24d7bd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8d247d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a24d56a5f0 con 0x55a24d3db280 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8d247d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a24d68ebd0 con 0x55a24d3fbf60 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8d247d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a24d7b4690 con 0x55a24d3d30a0 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8d247d740 10 client.0 _ll_register_callbacks cb 0x55a24d6b4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c8e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3d30a0 0x55a24d7ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c8e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3d30a0 0x55a24d7ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36420/0 (socket says 172.21.15.32:36420) 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c8e00640 1 -- 172.21.15.32:0/15356579 learned_addr learned my addr 172.21.15.32:0/15356579 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3db280 0x55a24d7ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c9800640 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3fbf60 0x55a24d7bd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c8e00640 1 -- 172.21.15.32:0/15356579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3fbf60 msgr2=0x55a24d7bd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3fbf60 0x55a24d7bd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c8e00640 1 -- 172.21.15.32:0/15356579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3db280 msgr2=0x55a24d7ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3db280 0x55a24d7ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c8e00640 1 -- 172.21.15.32:0/15356579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24d6cf650 con 0x55a24d3d30a0 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.575+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3db280 0x55a24d7ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.576+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3d30a0 0x55a24d7ad2d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8005ab0 tx=0x7ff8b8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6912d18f00e92de server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.576+0000 7ff8aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.576+0000 7ff8aaa00640 1 -- 172.21.15.32:0/15356579 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b804e070 con 0x55a24d3d30a0 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.576+0000 7ff8aaa00640 1 -- 172.21.15.32:0/15356579 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff8b8004b80 con 0x55a24d3d30a0 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.576+0000 7ff8d247d740 1 -- 172.21.15.32:0/15356579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8bc015070 con 0x55a24d3d30a0 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.576+0000 7ff8aaa00640 1 -- 172.21.15.32:0/15356579 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8b805a050 con 0x55a24d3d30a0 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.576+0000 7ff8d247d740 10 client.25288 fetch_fsmap learned FSMap version 28 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.576+0000 7ff8d247d740 1 -- 172.21.15.32:0/15356579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a24d7c04f0 con 0x55a24d3d30a0 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.577+0000 7ff8aaa00640 1 -- 172.21.15.32:0/15356579 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8bc015070 con 0x55a24d3d30a0 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.577+0000 7ff8aaa00640 1 -- 172.21.15.32:0/15356579 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff8b80324b0 con 0x55a24d3d30a0 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.577+0000 7ff8d247d740 10 client.25288 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:12:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.577+0000 7ff8d247d740 10 client.25288 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-31T07:12:03.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.577+0000 7ff8d247d740 20 client.25288 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:03.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.577+0000 7ff8d247d740 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a24d3ba8e0 0x55a24d7c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.577+0000 7ff8d247d740 4 client.25288 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:03.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.577+0000 7ff8d247d740 1 -- 172.21.15.32:0/15356579 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a24d71a4a0 con 0x55a24d3ba8e0 2024-08-31T07:12:03.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.577+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a24d3ba8e0 0x55a24d7c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.578+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a24d3ba8e0 0x55a24d7c3fd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3e0ef8c399d3be1 server_cookie=4ea45dd0d78668ae in_seq=0 out_seq=0 2024-08-31T07:12:03.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.578+0000 7ff8aaa00640 10 client.25288 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:12:03.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8aaa00640 1 -- 172.21.15.32:0/15356579 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a24d71a4a0 con 0x55a24d3ba8e0 2024-08-31T07:12:03.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8aaa00640 10 client.25288 handle_command_reply: tid=0 2024-08-31T07:12:03.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 client.25288 shutdown 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 -- 172.21.15.32:0/15356579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3d30a0 msgr2=0x55a24d7ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3d30a0 0x55a24d7ad2d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8005ab0 tx=0x7ff8b8009260 comp rx=0 tx=0).stop 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 -- 172.21.15.32:0/15356579 shutdown_connections 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a24d3ba8e0 0x55a24d7c3fd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3fbf60 0x55a24d7bd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3db280 0x55a24d7ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 --2- 172.21.15.32:0/15356579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3d30a0 0x55a24d7ad2d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8aaa00640 0 client.25288 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 -- 172.21.15.32:0/15356579 >> 172.21.15.32:0/15356579 conn(0x55a24d3f0350 msgr2=0x55a24d7a4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 -- 172.21.15.32:0/15356579 shutdown_connections 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.579+0000 7ff8d247d740 1 -- 172.21.15.32:0/15356579 wait complete. 2024-08-31T07:12:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.580+0000 7ff8d247d740 20 client.25288 trim_cache size 0 max 16384 2024-08-31T07:12:03.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.599+0000 7ff8d247d740 1 Processor -- start 2024-08-31T07:12:03.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.599+0000 7ff8d247d740 1 -- start start 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.599+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3f0350 0x55a24d7ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.599+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.599+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 0x55a24d7a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.599+0000 7ff8d247d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a24d7c2310 con 0x55a24d3d30a0 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.599+0000 7ff8d247d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a24d7c29e0 con 0x55a24d3db280 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.599+0000 7ff8d247d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a24d7c30b0 con 0x55a24d3f0350 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c8e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3f0350 0x55a24d7ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c8e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3f0350 0x55a24d7ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36434/0 (socket says 172.21.15.32:36434) 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c8e00640 1 -- 172.21.15.32:0/2769129811 learned_addr learned my addr 172.21.15.32:0/2769129811 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c9800640 1 --2- 172.21.15.32:0/2769129811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 0x55a24d7a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c8e00640 1 -- 172.21.15.32:0/2769129811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 msgr2=0x55a24d7a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/2769129811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 0x55a24d7a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c8e00640 1 -- 172.21.15.32:0/2769129811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 msgr2=0x55a24d7ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/2769129811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c8e00640 1 -- 172.21.15.32:0/2769129811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24d7a28b0 con 0x55a24d3f0350 2024-08-31T07:12:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c9800640 1 --2- 172.21.15.32:0/2769129811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 0x55a24d7a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.600+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/2769129811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3f0350 0x55a24d7ad080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8032880 tx=0x7ff8b8032660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=590a539605c34d4a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.601+0000 7ff8b5600640 1 -- 172.21.15.32:0/2769129811 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b804e070 con 0x55a24d3f0350 2024-08-31T07:12:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.601+0000 7ff8b5600640 1 -- 172.21.15.32:0/2769129811 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8b8063070 con 0x55a24d3f0350 2024-08-31T07:12:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.601+0000 7ff8b5600640 1 -- 172.21.15.32:0/2769129811 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8035040 con 0x55a24d3f0350 2024-08-31T07:12:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.601+0000 7ff8d247d740 1 -- 172.21.15.32:0/2769129811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3f0350 msgr2=0x55a24d7ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.601+0000 7ff8d247d740 1 --2- 172.21.15.32:0/2769129811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3f0350 0x55a24d7ad080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8032880 tx=0x7ff8b8032660 comp rx=0 tx=0).stop 2024-08-31T07:12:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.601+0000 7ff8d247d740 1 -- 172.21.15.32:0/2769129811 shutdown_connections 2024-08-31T07:12:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.601+0000 7ff8d247d740 1 --2- 172.21.15.32:0/2769129811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3db280 0x55a24d7a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.601+0000 7ff8d247d740 1 --2- 172.21.15.32:0/2769129811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.601+0000 7ff8d247d740 1 --2- 172.21.15.32:0/2769129811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3f0350 0x55a24d7ad080 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.601+0000 7ff8d247d740 1 -- 172.21.15.32:0/2769129811 >> 172.21.15.32:0/2769129811 conn(0x55a24d79ab80 msgr2=0x55a24d7c1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.602+0000 7ff8d247d740 1 -- 172.21.15.32:0/2769129811 shutdown_connections 2024-08-31T07:12:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.602+0000 7ff8d247d740 1 -- 172.21.15.32:0/2769129811 wait complete. 2024-08-31T07:12:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.602+0000 7ff8d247d740 1 -- start start 2024-08-31T07:12:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.602+0000 7ff8d247d740 1 Processor -- start 2024-08-31T07:12:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3db280 0x55a24d7c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8d247d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3f0350 0x55a24d82e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8d247d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a24d7c3e50 con 0x55a24d3d30a0 2024-08-31T07:12:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8d247d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a24d7c3780 con 0x55a24d3f0350 2024-08-31T07:12:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8d247d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a24d7c4520 con 0x55a24d3db280 2024-08-31T07:12:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8d247d740 10 client.0 _ll_register_callbacks cb 0x7ff8b8004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8c8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8c8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41074/0 (socket says 172.21.15.32:41074) 2024-08-31T07:12:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8c8e00640 1 -- 172.21.15.32:0/3712715592 learned_addr learned my addr 172.21.15.32:0/3712715592 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3db280 0x55a24d7c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8c9800640 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3f0350 0x55a24d82e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8c8e00640 1 -- 172.21.15.32:0/3712715592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3db280 msgr2=0x55a24d7c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:03.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3db280 0x55a24d7c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8c8e00640 1 -- 172.21.15.32:0/3712715592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3f0350 msgr2=0x55a24d82e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:03.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3f0350 0x55a24d82e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.603+0000 7ff8c8e00640 1 -- 172.21.15.32:0/3712715592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24d7c9300 con 0x55a24d3d30a0 2024-08-31T07:12:03.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.604+0000 7ff8c8e00640 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7c8540 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff8b802a930 tx=0x7ff8b80046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44fc5af913b4d16 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:03.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.604+0000 7ff8aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:12:03.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.604+0000 7ff8aaa00640 1 -- 172.21.15.32:0/3712715592 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b806b070 con 0x55a24d3d30a0 2024-08-31T07:12:03.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.604+0000 7ff8aaa00640 1 -- 172.21.15.32:0/3712715592 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff8b8004b80 con 0x55a24d3d30a0 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.604+0000 7ff8d247d740 1 -- 172.21.15.32:0/3712715592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b8063070 con 0x55a24d3d30a0 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.604+0000 7ff8aaa00640 1 -- 172.21.15.32:0/3712715592 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8b8077050 con 0x55a24d3d30a0 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.605+0000 7ff8d247d740 10 client.15276 fetch_fsmap learned FSMap version 28 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.605+0000 7ff8d247d740 1 -- 172.21.15.32:0/3712715592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a24d82f9f0 con 0x55a24d3d30a0 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.605+0000 7ff8aaa00640 1 -- 172.21.15.32:0/3712715592 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8b8063070 con 0x55a24d3d30a0 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.605+0000 7ff8aaa00640 1 -- 172.21.15.32:0/3712715592 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff8b804a450 con 0x55a24d3d30a0 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.605+0000 7ff8d247d740 10 client.15276 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.605+0000 7ff8d247d740 10 client.15276 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.605+0000 7ff8d247d740 20 client.15276 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.605+0000 7ff8d247d740 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a24d3fbf60 0x55a24d832f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.605+0000 7ff8d247d740 4 client.15276 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.605+0000 7ff8d247d740 1 -- 172.21.15.32:0/3712715592 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "status"}) -- 0x55a24d715f40 con 0x55a24d3fbf60 2024-08-31T07:12:03.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.606+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a24d3fbf60 0x55a24d832f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.606+0000 7ff8c3e00640 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a24d3fbf60 0x55a24d832f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=318c9841c48e7f34 server_cookie=c59e6b63b0c25075 in_seq=0 out_seq=0 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.606+0000 7ff8aaa00640 10 client.15276 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.606+0000 7ff8aaa00640 1 -- 172.21.15.32:0/3712715592 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55a24d715f40 con 0x55a24d3fbf60 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.606+0000 7ff8aaa00640 10 client.15276 handle_command_reply: tid=0 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.606+0000 7ff8d247d740 1 client.15276 shutdown 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.606+0000 7ff8d247d740 1 -- 172.21.15.32:0/3712715592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 msgr2=0x55a24d7c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.606+0000 7ff8d247d740 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7c8540 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff8b802a930 tx=0x7ff8b80046e0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.607+0000 7ff8d247d740 1 -- 172.21.15.32:0/3712715592 shutdown_connections 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.607+0000 7ff8d247d740 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a24d3fbf60 0x55a24d832f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.607+0000 7ff8d247d740 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a24d3f0350 0x55a24d82e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.607+0000 7ff8d247d740 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a24d3db280 0x55a24d7c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.607+0000 7ff8d247d740 1 --2- 172.21.15.32:0/3712715592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a24d3d30a0 0x55a24d7c8540 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.607+0000 7ff8aaa00640 0 client.15276 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:12:03.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.607+0000 7ff8d247d740 1 -- 172.21.15.32:0/3712715592 >> 172.21.15.32:0/3712715592 conn(0x55a24d79ab80 msgr2=0x55a24d7a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:03.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.607+0000 7ff8d247d740 1 -- 172.21.15.32:0/3712715592 shutdown_connections 2024-08-31T07:12:03.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.607+0000 7ff8d247d740 1 -- 172.21.15.32:0/3712715592 wait complete. 2024-08-31T07:12:03.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.607+0000 7ff8d247d740 20 client.15276 trim_cache size 0 max 16384 2024-08-31T07:12:03.612 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:03.612 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:12:03.612 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 1, 2024-08-31T07:12:03.612 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24718, 2024-08-31T07:12:03.612 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:12:03.613 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:12:03.613 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:12:03.613 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 123.766440475, 2024-08-31T07:12:03.613 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2024-08-31T07:12:03.613 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:12:03.613 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T07:12:03.613 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 150.26917232 2024-08-31T07:12:03.613 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 -- 172.21.15.32:0/3651116903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8ac0b4b80 msgr2=0x7ff8ac0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8ac0b4b80 0x7ff8ac0b7090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80096c0 tx=0x7ff8b8005a20 comp rx=0 tx=0).stop 2024-08-31T07:12:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 -- 172.21.15.32:0/3651116903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c4109090 msgr2=0x7ff8c419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c4109090 0x7ff8c419a6b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000d740 tx=0x7ff8b000dc00 comp rx=0 tx=0).stop 2024-08-31T07:12:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 -- 172.21.15.32:0/3651116903 shutdown_connections 2024-08-31T07:12:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff8ac0b4b80 0x7ff8ac0b7090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8c41099e0 0x7ff8c419f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8c4109090 0x7ff8c419a6b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 --2- 172.21.15.32:0/3651116903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8c4103750 0x7ff8c419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 -- 172.21.15.32:0/3651116903 >> 172.21.15.32:0/3651116903 conn(0x7ff8c40ff1a0 msgr2=0x7ff8c41015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:03.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 -- 172.21.15.32:0/3651116903 shutdown_connections 2024-08-31T07:12:03.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:03.608+0000 7ff8ca200640 1 -- 172.21.15.32:0/3651116903 wait complete. 2024-08-31T07:12:04.546 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:04 smithi032 ceph-mon[17252]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:12:04.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.b status 2024-08-31T07:12:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:04 smithi073 ceph-mon[21995]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:12:04.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:04 smithi084 ceph-mon[22387]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:12:04.925 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.582+0000 7f1972600640 1 -- 172.21.15.32:0/1598274548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f196c103900 msgr2=0x7f196c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.582+0000 7f1972600640 1 --2- 172.21.15.32:0/1598274548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f196c103900 0x7f196c103d10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f195c009900 tx=0x7f195c02a410 comp rx=0 tx=0).stop 2024-08-31T07:12:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.582+0000 7f1972600640 1 -- 172.21.15.32:0/1598274548 shutdown_connections 2024-08-31T07:12:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.582+0000 7f1972600640 1 --2- 172.21.15.32:0/1598274548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f196c109b90 0x7f196c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.582+0000 7f1972600640 1 --2- 172.21.15.32:0/1598274548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196c109240 0x7f196c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.582+0000 7f1972600640 1 --2- 172.21.15.32:0/1598274548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f196c103900 0x7f196c103d10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.582+0000 7f1972600640 1 -- 172.21.15.32:0/1598274548 >> 172.21.15.32:0/1598274548 conn(0x7f196c07b5f0 msgr2=0x7f196c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.582+0000 7f1972600640 1 -- 172.21.15.32:0/1598274548 shutdown_connections 2024-08-31T07:12:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.582+0000 7f1972600640 1 -- 172.21.15.32:0/1598274548 wait complete. 2024-08-31T07:12:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.583+0000 7f1972600640 1 Processor -- start 2024-08-31T07:12:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.583+0000 7f1972600640 1 -- start start 2024-08-31T07:12:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f1972600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f196c103900 0x7f196c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f1972600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196c109240 0x7f196c19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f1972600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f196c109b90 0x7f196c1a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f1972600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1138a0 con 0x7f196c109b90 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f1972600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f196c113720 con 0x7f196c109240 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f1972600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f196c113a20 con 0x7f196c103900 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f196be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f196c109b90 0x7f196c1a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f196b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f196c103900 0x7f196c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f196be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f196c109b90 0x7f196c1a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41080/0 (socket says 172.21.15.32:41080) 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f196be00640 1 -- 172.21.15.32:0/1091831891 learned_addr learned my addr 172.21.15.32:0/1091831891 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.584+0000 7f1963e00640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196c109240 0x7f196c19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f196b400640 1 -- 172.21.15.32:0/1091831891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196c109240 msgr2=0x7f196c19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f196b400640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196c109240 0x7f196c19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f196b400640 1 -- 172.21.15.32:0/1091831891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f196c109b90 msgr2=0x7f196c1a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f196b400640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f196c109b90 0x7f196c1a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f196b400640 1 -- 172.21.15.32:0/1091831891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c1a23a0 con 0x7f196c103900 2024-08-31T07:12:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f196be00640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f196c109b90 0x7f196c1a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f196b400640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f196c103900 0x7f196c19c430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f195c02aa00 tx=0x7f195c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f1968c00640 1 -- 172.21.15.32:0/1091831891 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c03f070 con 0x7f196c103900 2024-08-31T07:12:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f1972600640 1 -- 172.21.15.32:0/1091831891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c1a2630 con 0x7f196c103900 2024-08-31T07:12:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f1968c00640 1 -- 172.21.15.32:0/1091831891 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c031b50 con 0x7f196c103900 2024-08-31T07:12:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f1968c00640 1 -- 172.21.15.32:0/1091831891 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c035030 con 0x7f196c103900 2024-08-31T07:12:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.585+0000 7f1972600640 1 -- 172.21.15.32:0/1091831891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c1a8df0 con 0x7f196c103900 2024-08-31T07:12:05.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.588+0000 7f19791d2740 1 Processor -- start 2024-08-31T07:12:05.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.588+0000 7f19791d2740 1 -- start start 2024-08-31T07:12:05.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f19791d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d42e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f19791d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 0x555a4d42ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f19791d2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d42f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f19791d2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555a4d3a5550 con 0x555a4d068280 2024-08-31T07:12:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f19791d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555a4d394ae0 con 0x555a4d0600a0 2024-08-31T07:12:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f19791d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555a4d398a90 con 0x555a4d088f60 2024-08-31T07:12:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f1968c00640 1 -- 172.21.15.32:0/1091831891 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f195c042070 con 0x7f196c103900 2024-08-31T07:12:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f1963e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d42f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f1963e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d42f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41082/0 (socket says 172.21.15.32:41082) 2024-08-31T07:12:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f1963e00640 1 -- 172.21.15.32:0/3993128661 learned_addr learned my addr 172.21.15.32:0/3993128661 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f196be00640 1 --2- 172.21.15.32:0/3993128661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d42e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.589+0000 7f196b400640 1 --2- 172.21.15.32:0/3993128661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 0x555a4d42ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.590+0000 7f196be00640 1 -- 172.21.15.32:0/3993128661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 msgr2=0x555a4d42ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.590+0000 7f196be00640 1 --2- 172.21.15.32:0/3993128661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 0x555a4d42ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.590+0000 7f196be00640 1 -- 172.21.15.32:0/3993128661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 msgr2=0x555a4d42f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.590+0000 7f196be00640 1 --2- 172.21.15.32:0/3993128661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d42f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.590+0000 7f196be00640 1 -- 172.21.15.32:0/3993128661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4d0f4540 con 0x555a4d088f60 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.590+0000 7f196b400640 1 --2- 172.21.15.32:0/3993128661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 0x555a4d42ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.590+0000 7f196be00640 1 --2- 172.21.15.32:0/3993128661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d42e8f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f19640117e0 tx=0x7f1964011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17b429fd0b90f419 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.590+0000 7f1960c00640 1 -- 172.21.15.32:0/3993128661 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964011310 con 0x555a4d088f60 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.591+0000 7f1960c00640 1 -- 172.21.15.32:0/3993128661 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196400f9a0 con 0x555a4d088f60 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.591+0000 7f1960c00640 1 -- 172.21.15.32:0/3993128661 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640055f0 con 0x555a4d088f60 2024-08-31T07:12:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.591+0000 7f1968c00640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19500b4b80 0x7f19500b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.591+0000 7f1968c00640 1 -- 172.21.15.32:0/1091831891 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f195c1057a0 con 0x7f196c103900 2024-08-31T07:12:05.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.591+0000 7f196b400640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19500b4b80 0x7f19500b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.591+0000 7f19791d2740 1 -- 172.21.15.32:0/3993128661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 msgr2=0x555a4d42e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.591+0000 7f19791d2740 1 --2- 172.21.15.32:0/3993128661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d42e8f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f19640117e0 tx=0x7f1964011ca0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.592+0000 7f19791d2740 1 -- 172.21.15.32:0/3993128661 shutdown_connections 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.592+0000 7f19791d2740 1 --2- 172.21.15.32:0/3993128661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d42f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.592+0000 7f19791d2740 1 --2- 172.21.15.32:0/3993128661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 0x555a4d42ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.592+0000 7f19791d2740 1 --2- 172.21.15.32:0/3993128661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d42e8f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.592+0000 7f19791d2740 1 -- 172.21.15.32:0/3993128661 >> 172.21.15.32:0/3993128661 conn(0x555a4d07d350 msgr2=0x555a4d42c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.592+0000 7f19791d2740 1 -- 172.21.15.32:0/3993128661 shutdown_connections 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.592+0000 7f19791d2740 1 -- 172.21.15.32:0/3993128661 wait complete. 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.592+0000 7f196b400640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19500b4b80 0x7f19500b7090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f195c0cbe80 tx=0x7f195c003f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.592+0000 7f19791d2740 1 -- start start 2024-08-31T07:12:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.593+0000 7f19791d2740 1 Processor -- start 2024-08-31T07:12:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.593+0000 7f19791d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d0600a0 0x555a4d439f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.593+0000 7f19791d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 0x555a4d43a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.593+0000 7f19791d2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d088f60 0x555a4d4463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.593+0000 7f19791d2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555a4d1f75f0 con 0x555a4d088f60 2024-08-31T07:12:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.593+0000 7f19791d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555a4d31bbd0 con 0x555a4d068280 2024-08-31T07:12:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.593+0000 7f19791d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555a4d43d0d0 con 0x555a4d0600a0 2024-08-31T07:12:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.593+0000 7f19791d2740 10 client.0 _ll_register_callbacks cb 0x555a4d341210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.593+0000 7f196be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d0600a0 0x555a4d439f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f196be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d088f60 0x555a4d4463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f196be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d088f60 0x555a4d4463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41084/0 (socket says 172.21.15.32:41084) 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f1963e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 0x555a4d43a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f196be00640 1 -- 172.21.15.32:0/320980239 learned_addr learned my addr 172.21.15.32:0/320980239 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f1963e00640 1 -- 172.21.15.32:0/320980239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d0600a0 msgr2=0x555a4d439f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f1963e00640 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d0600a0 0x555a4d439f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f1963e00640 1 -- 172.21.15.32:0/320980239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d088f60 msgr2=0x555a4d4463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f1963e00640 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d088f60 0x555a4d4463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f1963e00640 1 -- 172.21.15.32:0/320980239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4d35c650 con 0x555a4d068280 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f1963e00640 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 0x555a4d43a490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1954012700 tx=0x7f1954012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f634724e057a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.594+0000 7f1959600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.595+0000 7f1959600640 1 -- 172.21.15.32:0/320980239 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954004030 con 0x555a4d068280 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.595+0000 7f1959600640 1 -- 172.21.15.32:0/320980239 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f19540041d0 con 0x555a4d068280 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.595+0000 7f19791d2740 1 -- 172.21.15.32:0/320980239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f196400f9a0 con 0x555a4d068280 2024-08-31T07:12:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.595+0000 7f1959600640 1 -- 172.21.15.32:0/320980239 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1954004410 con 0x555a4d068280 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.595+0000 7f19791d2740 10 client.25301 fetch_fsmap learned FSMap version 28 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.595+0000 7f19791d2740 1 -- 172.21.15.32:0/320980239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a4d448e10 con 0x555a4d068280 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.595+0000 7f1959600640 1 -- 172.21.15.32:0/320980239 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196400f9a0 con 0x555a4d068280 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.595+0000 7f1959600640 1 -- 172.21.15.32:0/320980239 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1954003b50 con 0x555a4d068280 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.596+0000 7f19791d2740 10 client.25301 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.596+0000 7f19791d2740 10 client.25301 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.596+0000 7f19791d2740 20 client.25301 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.596+0000 7f19791d2740 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a4d0478e0 0x555a4d44c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.596+0000 7f19791d2740 4 client.25301 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.596+0000 7f19791d2740 1 -- 172.21.15.32:0/320980239 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a4d3a74a0 con 0x555a4d0478e0 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.596+0000 7f196be00640 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a4d0478e0 0x555a4d44c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.596+0000 7f196be00640 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a4d0478e0 0x555a4d44c860 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d44b8bac0668dbc server_cookie=caa84174ae24e98a in_seq=0 out_seq=0 2024-08-31T07:12:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.596+0000 7f1959600640 10 client.25301 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.597+0000 7f1959600640 1 -- 172.21.15.32:0/320980239 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a4d3a74a0 con 0x555a4d0478e0 2024-08-31T07:12:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.597+0000 7f1959600640 10 client.25301 handle_command_reply: tid=0 2024-08-31T07:12:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.597+0000 7f19791d2740 1 client.25301 shutdown 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 1 -- 172.21.15.32:0/320980239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 msgr2=0x555a4d43a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 0x555a4d43a490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1954012700 tx=0x7f1954012bc0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 1 -- 172.21.15.32:0/320980239 shutdown_connections 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a4d0478e0 0x555a4d44c860 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d088f60 0x555a4d4463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 0x555a4d43a490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f1959600640 0 client.25301 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 1 --2- 172.21.15.32:0/320980239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d0600a0 0x555a4d439f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 1 -- 172.21.15.32:0/320980239 >> 172.21.15.32:0/320980239 conn(0x555a4d07d350 msgr2=0x555a4d431a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 1 -- 172.21.15.32:0/320980239 shutdown_connections 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 1 -- 172.21.15.32:0/320980239 wait complete. 2024-08-31T07:12:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.598+0000 7f19791d2740 20 client.25301 trim_cache size 0 max 16384 2024-08-31T07:12:05.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.617+0000 7f19791d2740 1 Processor -- start 2024-08-31T07:12:05.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.617+0000 7f19791d2740 1 -- start start 2024-08-31T07:12:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f19791d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 0x555a4d439440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f19791d2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d446360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f19791d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d4387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f19791d2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555a4d44aba0 con 0x555a4d068280 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f19791d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555a4d44b270 con 0x555a4d0600a0 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f19791d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555a4d44b940 con 0x555a4d088f60 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f1963e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d446360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f1963e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d446360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41094/0 (socket says 172.21.15.32:41094) 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f1963e00640 1 -- 172.21.15.32:0/3008673308 learned_addr learned my addr 172.21.15.32:0/3008673308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f196be00640 1 --2- 172.21.15.32:0/3008673308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d4387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f196be00640 1 --2- 172.21.15.32:0/3008673308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 0x555a4d439440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f1963e00640 1 -- 172.21.15.32:0/3008673308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 msgr2=0x555a4d4387d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f1963e00640 1 --2- 172.21.15.32:0/3008673308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d4387d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f1963e00640 1 -- 172.21.15.32:0/3008673308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 msgr2=0x555a4d439440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f1963e00640 1 --2- 172.21.15.32:0/3008673308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 0x555a4d439440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.618+0000 7f1963e00640 1 -- 172.21.15.32:0/3008673308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4d431750 con 0x555a4d068280 2024-08-31T07:12:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.619+0000 7f196be00640 1 --2- 172.21.15.32:0/3008673308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d4387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.619+0000 7f1963e00640 1 --2- 172.21.15.32:0/3008673308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d446360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f19540126d0 tx=0x7f195400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd5122ab3e5bd3b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.619+0000 7f195be00640 1 -- 172.21.15.32:0/3008673308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954026690 con 0x555a4d068280 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.619+0000 7f195be00640 1 -- 172.21.15.32:0/3008673308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1954003fa0 con 0x555a4d068280 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.619+0000 7f195be00640 1 -- 172.21.15.32:0/3008673308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540035f0 con 0x555a4d068280 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.620+0000 7f19791d2740 1 -- 172.21.15.32:0/3008673308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 msgr2=0x555a4d446360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.620+0000 7f19791d2740 1 --2- 172.21.15.32:0/3008673308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d446360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f19540126d0 tx=0x7f195400f790 comp rx=0 tx=0).stop 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.620+0000 7f19791d2740 1 -- 172.21.15.32:0/3008673308 shutdown_connections 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.620+0000 7f19791d2740 1 --2- 172.21.15.32:0/3008673308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d4387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.620+0000 7f19791d2740 1 --2- 172.21.15.32:0/3008673308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d068280 0x555a4d446360 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.620+0000 7f19791d2740 1 --2- 172.21.15.32:0/3008673308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d0600a0 0x555a4d439440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.620+0000 7f19791d2740 1 -- 172.21.15.32:0/3008673308 >> 172.21.15.32:0/3008673308 conn(0x555a4d07d350 msgr2=0x555a4d4274c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.620+0000 7f19791d2740 1 -- 172.21.15.32:0/3008673308 shutdown_connections 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.620+0000 7f19791d2740 1 -- 172.21.15.32:0/3008673308 wait complete. 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.621+0000 7f19791d2740 1 -- start start 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.621+0000 7f19791d2740 1 Processor -- start 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.621+0000 7f19791d2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d0600a0 0x555a4d4550c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.621+0000 7f19791d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 0x555a4d44e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.621+0000 7f19791d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d44eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.621+0000 7f19791d2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555a4d44c6e0 con 0x555a4d0600a0 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.621+0000 7f19791d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555a4d44c010 con 0x555a4d068280 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.621+0000 7f19791d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555a4d44cdb0 con 0x555a4d088f60 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.621+0000 7f19791d2740 10 client.0 _ll_register_callbacks cb 0x7f1954009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.621+0000 7f196be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d0600a0 0x555a4d4550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f196be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d0600a0 0x555a4d4550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41108/0 (socket says 172.21.15.32:41108) 2024-08-31T07:12:05.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f196be00640 1 -- 172.21.15.32:0/3690349544 learned_addr learned my addr 172.21.15.32:0/3690349544 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f1963e00640 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 0x555a4d44e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f196be00640 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d44eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f196be00640 1 -- 172.21.15.32:0/3690349544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 msgr2=0x555a4d44eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f196be00640 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d44eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f196be00640 1 -- 172.21.15.32:0/3690349544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 msgr2=0x555a4d44e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f196be00640 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 0x555a4d44e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f196be00640 1 -- 172.21.15.32:0/3690349544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4d44f020 con 0x555a4d0600a0 2024-08-31T07:12:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f196be00640 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d0600a0 0x555a4d4550c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f196400f790 tx=0x7f1964009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9deca21abd5d1a57 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.622+0000 7f1959600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:12:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f1959600640 1 -- 172.21.15.32:0/3690349544 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196401e040 con 0x555a4d0600a0 2024-08-31T07:12:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f1959600640 1 -- 172.21.15.32:0/3690349544 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1964011210 con 0x555a4d0600a0 2024-08-31T07:12:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f19791d2740 1 -- 172.21.15.32:0/3690349544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1954003fa0 con 0x555a4d0600a0 2024-08-31T07:12:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f1959600640 1 -- 172.21.15.32:0/3690349544 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1964011410 con 0x555a4d0600a0 2024-08-31T07:12:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f19791d2740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f19791d2740 1 -- 172.21.15.32:0/3690349544 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a4d454a70 con 0x555a4d0600a0 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f1959600640 1 -- 172.21.15.32:0/3690349544 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1954003fa0 con 0x555a4d0600a0 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f1959600640 1 -- 172.21.15.32:0/3690349544 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1964034070 con 0x555a4d0600a0 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f19791d2740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f19791d2740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f19791d2740 20 client.15294 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.623+0000 7f19791d2740 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a4d01cc90 0x555a4d4bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.624+0000 7f19791d2740 4 client.15294 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.624+0000 7f19791d2740 1 -- 172.21.15.32:0/3690349544 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "status"}) -- 0x555a4d3a2f40 con 0x555a4d01cc90 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.624+0000 7f1963e00640 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a4d01cc90 0x555a4d4bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.624+0000 7f1963e00640 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a4d01cc90 0x555a4d4bb240 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=924acd7310c39dd8 server_cookie=4b98a14e74ab2055 in_seq=0 out_seq=0 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.624+0000 7f1959600640 10 client.15294 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f1959600640 1 -- 172.21.15.32:0/3690349544 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555a4d3a2f40 con 0x555a4d01cc90 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f1959600640 10 client.15294 handle_command_reply: tid=0 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 client.15294 shutdown 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 -- 172.21.15.32:0/3690349544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d0600a0 msgr2=0x555a4d4550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d0600a0 0x555a4d4550c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f196400f790 tx=0x7f1964009a30 comp rx=0 tx=0).stop 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 -- 172.21.15.32:0/3690349544 shutdown_connections 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a4d01cc90 0x555a4d4bb240 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a4d088f60 0x555a4d44eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f1959600640 0 client.15294 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a4d068280 0x555a4d44e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 --2- 172.21.15.32:0/3690349544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a4d0600a0 0x555a4d4550c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 -- 172.21.15.32:0/3690349544 >> 172.21.15.32:0/3690349544 conn(0x555a4d07d350 msgr2=0x555a4d439d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 -- 172.21.15.32:0/3690349544 shutdown_connections 2024-08-31T07:12:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 1 -- 172.21.15.32:0/3690349544 wait complete. 2024-08-31T07:12:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.625+0000 7f19791d2740 20 client.15294 trim_cache size 0 max 16384 2024-08-31T07:12:05.630 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:12:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-08-31T07:12:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24734, 2024-08-31T07:12:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:12:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:12:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:12:05.630 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 135.97594699000001, 2024-08-31T07:12:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2024-08-31T07:12:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:12:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T07:12:05.631 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 149.69661195200001 2024-08-31T07:12:05.631 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 -- 172.21.15.32:0/1091831891 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19500b4b80 msgr2=0x7f19500b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19500b4b80 0x7f19500b7090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f195c0cbe80 tx=0x7f195c003f90 comp rx=0 tx=0).stop 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 -- 172.21.15.32:0/1091831891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f196c103900 msgr2=0x7f196c19c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f196c103900 0x7f196c19c430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f195c02aa00 tx=0x7f195c003b90 comp rx=0 tx=0).stop 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 -- 172.21.15.32:0/1091831891 shutdown_connections 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19500b4b80 0x7f19500b7090 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f196c109b90 0x7f196c1a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196c109240 0x7f196c19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 --2- 172.21.15.32:0/1091831891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f196c103900 0x7f196c19c430 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 -- 172.21.15.32:0/1091831891 >> 172.21.15.32:0/1091831891 conn(0x7f196c07b5f0 msgr2=0x7f196c107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 -- 172.21.15.32:0/1091831891 shutdown_connections 2024-08-31T07:12:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:05.626+0000 7f1972600640 1 -- 172.21.15.32:0/1091831891 wait complete. 2024-08-31T07:12:06.373 INFO:teuthology.run_tasks:Running task kclient... 2024-08-31T07:12:06.400 INFO:tasks.kclient:Mounting kernel clients... 2024-08-31T07:12:06.400 INFO:tasks.kclient:config is {'client.0': {}} 2024-08-31T07:12:06.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:12:06.401 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-08-31T07:12:06.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:12:06.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-08-31T07:12:06.536 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-08-31T07:12:06.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T07:12:06.539 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-08-31T07:12:06.539 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-31T07:12:06.539 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-31T07:12:06.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs ls 2024-08-31T07:12:06.587 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:06 smithi032 ceph-mon[17252]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:06 smithi073 ceph-mon[21995]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:06 smithi084 ceph-mon[22387]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:06.918 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.608+0000 7f87fd800640 1 -- 172.21.15.32:0/1727263902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87f8105b20 msgr2=0x7f87f8105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.608+0000 7f87fd800640 1 --2- 172.21.15.32:0/1727263902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87f8105b20 0x7f87f8105f30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009900 tx=0x7f87e802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.608+0000 7f87fd800640 1 -- 172.21.15.32:0/1727263902 shutdown_connections 2024-08-31T07:12:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.608+0000 7f87fd800640 1 --2- 172.21.15.32:0/1727263902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87f810bdb0 0x7f87f81125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.608+0000 7f87fd800640 1 --2- 172.21.15.32:0/1727263902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f810b460 0x7f87f810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.608+0000 7f87fd800640 1 --2- 172.21.15.32:0/1727263902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87f8105b20 0x7f87f8105f30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.608+0000 7f87fd800640 1 -- 172.21.15.32:0/1727263902 >> 172.21.15.32:0/1727263902 conn(0x7f87f8101570 msgr2=0x7f87f81039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.608+0000 7f87fd800640 1 -- 172.21.15.32:0/1727263902 shutdown_connections 2024-08-31T07:12:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.608+0000 7f87fd800640 1 -- 172.21.15.32:0/1727263902 wait complete. 2024-08-31T07:12:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.609+0000 7f87fd800640 1 Processor -- start 2024-08-31T07:12:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.609+0000 7f87fd800640 1 -- start start 2024-08-31T07:12:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87fd800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87f8105b20 0x7f87f819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87fd800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f810b460 0x7f87f819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87fd800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87f810bdb0 0x7f87f81a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87fd800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8115a70 con 0x7f87f8105b20 2024-08-31T07:12:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87fd800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81158f0 con 0x7f87f810b460 2024-08-31T07:12:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87fd800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8115bf0 con 0x7f87f810bdb0 2024-08-31T07:12:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87f6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87f8105b20 0x7f87f819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87f7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87f810bdb0 0x7f87f81a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87f6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87f8105b20 0x7f87f819c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35296/0 (socket says 172.21.15.32:35296) 2024-08-31T07:12:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87f6a00640 1 -- 172.21.15.32:0/3549636639 learned_addr learned my addr 172.21.15.32:0/3549636639 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:07.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.610+0000 7f87efe00640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f810b460 0x7f87f819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87f6a00640 1 -- 172.21.15.32:0/3549636639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87f810bdb0 msgr2=0x7f87f81a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87f6a00640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87f810bdb0 0x7f87f81a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87f6a00640 1 -- 172.21.15.32:0/3549636639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f810b460 msgr2=0x7f87f819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87f6a00640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f810b460 0x7f87f819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87f6a00640 1 -- 172.21.15.32:0/3549636639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f81a23f0 con 0x7f87f8105b20 2024-08-31T07:12:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87efe00640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f810b460 0x7f87f819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87f6a00640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87f8105b20 0x7f87f819c480 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f87e8005940 tx=0x7f87e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87ef400640 1 -- 172.21.15.32:0/3549636639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e803f030 con 0x7f87f8105b20 2024-08-31T07:12:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87ef400640 1 -- 172.21.15.32:0/3549636639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e8003aa0 con 0x7f87f8105b20 2024-08-31T07:12:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87fd800640 1 -- 172.21.15.32:0/3549636639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f81a2680 con 0x7f87f8105b20 2024-08-31T07:12:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.611+0000 7f87ef400640 1 -- 172.21.15.32:0/3549636639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8035070 con 0x7f87f8105b20 2024-08-31T07:12:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.612+0000 7f87fd800640 1 -- 172.21.15.32:0/3549636639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f81a8e40 con 0x7f87f8105b20 2024-08-31T07:12:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.613+0000 7f87fd800640 1 -- 172.21.15.32:0/3549636639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f8105f30 con 0x7f87f8105b20 2024-08-31T07:12:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.615+0000 7f87ef400640 1 -- 172.21.15.32:0/3549636639 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f87e8042030 con 0x7f87f8105b20 2024-08-31T07:12:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.616+0000 7f87ef400640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87dc0b4c00 0x7f87dc0b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.616+0000 7f87ef400640 1 -- 172.21.15.32:0/3549636639 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f87e81056a0 con 0x7f87f8105b20 2024-08-31T07:12:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.617+0000 7f87efe00640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87dc0b4c00 0x7f87dc0b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.617+0000 7f87efe00640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87dc0b4c00 0x7f87dc0b7110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f87f810af60 tx=0x7f87e0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:07.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.621+0000 7f87ef400640 1 -- 172.21.15.32:0/3549636639 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f87e80cc080 con 0x7f87f8105b20 2024-08-31T07:12:07.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.860+0000 7f87fd800640 1 -- 172.21.15.32:0/3549636639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f87f802e430 con 0x7f87f8105b20 2024-08-31T07:12:07.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.861+0000 7f87ef400640 1 -- 172.21.15.32:0/3549636639 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f87e80d0840 con 0x7f87f8105b20 2024-08-31T07:12:07.862 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-31T07:12:07.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 -- 172.21.15.32:0/3549636639 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87dc0b4c00 msgr2=0x7f87dc0b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:07.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87dc0b4c00 0x7f87dc0b7110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f87f810af60 tx=0x7f87e0005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:07.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 -- 172.21.15.32:0/3549636639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87f8105b20 msgr2=0x7f87f819c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:07.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87f8105b20 0x7f87f819c480 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f87e8005940 tx=0x7f87e8004270 comp rx=0 tx=0).stop 2024-08-31T07:12:07.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 -- 172.21.15.32:0/3549636639 shutdown_connections 2024-08-31T07:12:07.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87dc0b4c00 0x7f87dc0b7110 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:07.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87f810bdb0 0x7f87f81a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:07.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87f810b460 0x7f87f819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:07.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 --2- 172.21.15.32:0/3549636639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87f8105b20 0x7f87f819c480 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:07.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 -- 172.21.15.32:0/3549636639 >> 172.21.15.32:0/3549636639 conn(0x7f87f8101570 msgr2=0x7f87f8103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:07.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.863+0000 7f87fd800640 1 -- 172.21.15.32:0/3549636639 shutdown_connections 2024-08-31T07:12:07.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:07.864+0000 7f87fd800640 1 -- 172.21.15.32:0/3549636639 wait complete. 2024-08-31T07:12:08.587 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:08 smithi032 ceph-mon[17252]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:08.587 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:08 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3549636639' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-31T07:12:08.624 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-31T07:12:08.625 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-31T07:12:08.625 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-08-31T07:12:08.625 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-31T07:12:08.625 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-31T07:12:08.625 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-31T07:12:08.625 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-31T07:12:08.625 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-31T07:12:08.625 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-31T07:12:08.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:12:08.625 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-08-31T07:12:08.662 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-31T07:12:08.662 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-31T07:12:08.662 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-08-31T07:12:08.662 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-08-31T07:12:08.662 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-08-31T07:12:08.662 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-08-31T07:12:08.662 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42168sec preferred_lft 42168sec 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-08-31T07:12:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-31T07:12:08.664 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-08-31T07:12:08.664 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-08-31T07:12:08.664 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-31T07:12:08.664 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-31T07:12:08.664 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-08-31T07:12:08.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2024-08-31T07:12:08.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2024-08-31T07:12:08.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2024-08-31T07:12:08.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-31T07:12:08.665 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-08-31T07:12:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:08 smithi073 ceph-mon[21995]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:08 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3549636639' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-31T07:12:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:08 smithi084 ceph-mon[22387]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:08 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3549636639' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-31T07:12:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T07:12:09.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T07:12:09.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:12:09.077 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-31T07:12:09.133 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-08-31T07:12:09.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:12:09.136 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-08-31T07:12:09.182 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-08-31T07:12:09.182 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-31T07:12:09.182 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-08-31T07:12:09.182 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-08-31T07:12:09.182 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-31T07:12:09.182 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-31T07:12:09.183 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-08-31T07:12:09.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-31T07:12:09.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-31T07:12:09.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-31T07:12:09.183 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-08-31T07:12:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T07:12:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T07:12:09.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:12:09.782 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-08-31T07:12:09.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:12:09.818 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-08-31T07:12:09.893 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-31T07:12:09.893 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-08-31T07:12:09.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-31T07:12:09.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-31T07:12:09.893 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-08-31T07:12:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T07:12:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T07:12:10.072 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-31T07:12:10.073 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-31T07:12:10.073 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-08-31T07:12:10.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-31T07:12:10.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-31T07:12:10.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-31T07:12:10.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-31T07:12:10.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-31T07:12:10.073 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-08-31T07:12:10.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T07:12:10.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T07:12:10.333 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-31T07:12:10.333 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-08-31T07:12:10.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-08-31T07:12:10.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2024-08-31T07:12:10.333 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-08-31T07:12:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T07:12:10.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T07:12:10.500 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-31T07:12:10.501 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-31T07:12:10.536 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-31T07:12:10.536 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-31T07:12:10.536 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-31T07:12:10.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:10 smithi032 ceph-mon[17252]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:12:10.611 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-08-31T07:12:10.611 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-08-31T07:12:10.611 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/aeeafabb-ed2a-477b-bc60-8858e680a053 2024-08-31T07:12:10.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:12:10.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/aeeafabb-ed2a-477b-bc60-8858e680a053 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-08-31T07:12:10.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:10 smithi073 ceph-mon[21995]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:12:10.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:10 smithi084 ceph-mon[22387]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:12:10.867 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-08-31T07:12:10.867 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-08-31T07:12:10.867 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-08-31T07:12:11.549 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-08-31T07:12:11.549 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-08-31T07:12:11.549 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-08-31T07:12:11.549 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.73:6789,172.21.15.84:6789" 2024-08-31T07:12:11.549 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.73:6789,172.21.15.84:6789:/volumes/qa/sv_1/aeeafabb-ed2a-477b-bc60-8858e680a053 2024-08-31T07:12:11.549 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=965fc21a-6766-11ef-bcd4-c7b262605968" will pass to kernel 2024-08-31T07:12:11.552 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-08-31T07:12:11.552 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-08-31T07:12:11.552 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-31T07:12:11.552 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-08-31T07:12:11.552 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-31T07:12:11.555 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-31T07:12:11.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:12:11.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-31T07:12:11.606 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:11 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:12:11.617 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-31T07:12:11.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:11 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:12:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-31T07:12:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T07:12:11.721 INFO:teuthology.orchestra.run.smithi032.stdout:client25316 2024-08-31T07:12:11.721 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:12:11.721 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-31T07:12:11.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:11 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:12:11.757 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-31T07:12:11.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-31T07:12:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T07:12:11.890 INFO:teuthology.orchestra.run.smithi032.stdout:client25316 2024-08-31T07:12:11.890 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-31T07:12:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-31T07:12:11.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T07:12:11.984 INFO:teuthology.orchestra.run.smithi032.stdout:client25316 2024-08-31T07:12:11.984 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/965fc21a-6766-11ef-bcd4-c7b262605968.client25316/status') 2024-08-31T07:12:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T07:12:12.102 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.25316 (3)172.21.15.32:0/3986931927 2024-08-31T07:12:12.102 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2024-08-31T07:12:12.103 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2024-08-31T07:12:12.103 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2024-08-31T07:12:12.103 INFO:teuthology.orchestra.run.smithi032.stderr:71 bytes copied, 5.8626e-05 s, 1.2 MB/s 2024-08-31T07:12:12.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T07:12:12.114 INFO:teuthology.run_tasks:Running task exec... 2024-08-31T07:12:12.130 INFO:teuthology.task.exec:Executing custom commands... 2024-08-31T07:12:12.131 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-08-31T07:12:12.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-08-31T07:12:12.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.371+0000 7f0cc2800640 1 -- 172.21.15.32:0/2487029040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cbc109af0 msgr2=0x7f0cbc110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:12.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.371+0000 7f0cc2800640 1 --2- 172.21.15.32:0/2487029040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cbc109af0 0x7f0cbc110320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009ca0 tx=0x7f0cb402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:12:12.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.371+0000 7f0cc2800640 1 -- 172.21.15.32:0/2487029040 shutdown_connections 2024-08-31T07:12:12.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.371+0000 7f0cc2800640 1 --2- 172.21.15.32:0/2487029040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cbc109af0 0x7f0cbc110320 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:12.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.371+0000 7f0cc2800640 1 --2- 172.21.15.32:0/2487029040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cbc1091a0 0x7f0cbc1095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.371+0000 7f0cc2800640 1 --2- 172.21.15.32:0/2487029040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc103870 0x7f0cbc103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.371+0000 7f0cc2800640 1 -- 172.21.15.32:0/2487029040 >> 172.21.15.32:0/2487029040 conn(0x7f0cbc0ff270 msgr2=0x7f0cbc1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.371+0000 7f0cc2800640 1 -- 172.21.15.32:0/2487029040 shutdown_connections 2024-08-31T07:12:12.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.371+0000 7f0cc2800640 1 -- 172.21.15.32:0/2487029040 wait complete. 2024-08-31T07:12:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.372+0000 7f0cc2800640 1 Processor -- start 2024-08-31T07:12:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.372+0000 7f0cc2800640 1 -- start start 2024-08-31T07:12:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cc2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cbc103870 0x7f0cbc19c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:12.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cc2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cbc1091a0 0x7f0cbc19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cc2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc109af0 0x7f0cbc1a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cc2800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1137a0 con 0x7f0cbc103870 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cc2800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc113620 con 0x7f0cbc1091a0 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cc2800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc113920 con 0x7f0cbc109af0 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cbc103870 0x7f0cbc19c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cbc1091a0 0x7f0cbc19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cbc1091a0 0x7f0cbc19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46092/0 (socket says 172.21.15.32:46092) 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbaa00640 1 -- 172.21.15.32:0/1693005092 learned_addr learned my addr 172.21.15.32:0/1693005092 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbbe00640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc109af0 0x7f0cbc1a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbaa00640 1 -- 172.21.15.32:0/1693005092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc109af0 msgr2=0x7f0cbc1a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbaa00640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc109af0 0x7f0cbc1a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:12.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbaa00640 1 -- 172.21.15.32:0/1693005092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cbc103870 msgr2=0x7f0cbc19c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbaa00640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cbc103870 0x7f0cbc19c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbaa00640 1 -- 172.21.15.32:0/1693005092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc1a22b0 con 0x7f0cbc1091a0 2024-08-31T07:12:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.373+0000 7f0cbb400640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cbc103870 0x7f0cbc19c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.374+0000 7f0cbaa00640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cbc1091a0 0x7f0cbc19c920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800e9f0 tx=0x7f0ca800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:12.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.374+0000 7f0cb3e00640 1 -- 172.21.15.32:0/1693005092 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800ccc0 con 0x7f0cbc1091a0 2024-08-31T07:12:12.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.374+0000 7f0cb3e00640 1 -- 172.21.15.32:0/1693005092 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca80108e0 con 0x7f0cbc1091a0 2024-08-31T07:12:12.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.374+0000 7f0cc2800640 1 -- 172.21.15.32:0/1693005092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cbc1a25a0 con 0x7f0cbc1091a0 2024-08-31T07:12:12.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.374+0000 7f0cb3e00640 1 -- 172.21.15.32:0/1693005092 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8018860 con 0x7f0cbc1091a0 2024-08-31T07:12:12.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.374+0000 7f0cc2800640 1 -- 172.21.15.32:0/1693005092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cbc1a8d30 con 0x7f0cbc1091a0 2024-08-31T07:12:12.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.375+0000 7f0cc2800640 1 -- 172.21.15.32:0/1693005092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cbc104ae0 con 0x7f0cbc1091a0 2024-08-31T07:12:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.378+0000 7f0cb3e00640 1 -- 172.21.15.32:0/1693005092 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f0ca8010430 con 0x7f0cbc1091a0 2024-08-31T07:12:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.380+0000 7f0cb3e00640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ca00b4c00 0x7f0ca00b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.380+0000 7f0cb3e00640 1 -- 172.21.15.32:0/1693005092 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f0ca80e54b0 con 0x7f0cbc1091a0 2024-08-31T07:12:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.380+0000 7f0cbb400640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ca00b4c00 0x7f0ca00b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.381+0000 7f0cbb400640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ca00b4c00 0x7f0ca00b7110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009fc0 tx=0x7f0cac005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.384+0000 7f0cb3e00640 1 -- 172.21.15.32:0/1693005092 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0ca80abe90 con 0x7f0cbc1091a0 2024-08-31T07:12:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:12.628+0000 7f0cc2800640 1 -- 172.21.15.32:0/1693005092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f0cbc103c80 con 0x7f0cbc1091a0 2024-08-31T07:12:12.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:12 smithi073 ceph-mon[21995]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:12.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:12 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:12 smithi084 ceph-mon[22387]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:12 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:12 smithi032 ceph-mon[17252]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:12 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.422+0000 7f0cb3e00640 1 -- 172.21.15.32:0/1693005092 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f0ca80b0650 con 0x7f0cbc1091a0 2024-08-31T07:12:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.424+0000 7f0cc2800640 1 -- 172.21.15.32:0/1693005092 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ca00b4c00 msgr2=0x7f0ca00b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.424+0000 7f0cc2800640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ca00b4c00 0x7f0ca00b7110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009fc0 tx=0x7f0cac005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.424+0000 7f0cc2800640 1 -- 172.21.15.32:0/1693005092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cbc1091a0 msgr2=0x7f0cbc19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.424+0000 7f0cc2800640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cbc1091a0 0x7f0cbc19c920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800e9f0 tx=0x7f0ca800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.424+0000 7f0cc2800640 1 -- 172.21.15.32:0/1693005092 shutdown_connections 2024-08-31T07:12:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.424+0000 7f0cc2800640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ca00b4c00 0x7f0ca00b7110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.424+0000 7f0cc2800640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cbc109af0 0x7f0cbc1a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.424+0000 7f0cc2800640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cbc1091a0 0x7f0cbc19c920 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.424+0000 7f0cc2800640 1 --2- 172.21.15.32:0/1693005092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cbc103870 0x7f0cbc19c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.424+0000 7f0cc2800640 1 -- 172.21.15.32:0/1693005092 >> 172.21.15.32:0/1693005092 conn(0x7f0cbc0ff270 msgr2=0x7f0cbc108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.425+0000 7f0cc2800640 1 -- 172.21.15.32:0/1693005092 shutdown_connections 2024-08-31T07:12:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.425+0000 7f0cc2800640 1 -- 172.21.15.32:0/1693005092 wait complete. 2024-08-31T07:12:13.451 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-31T07:12:13.460 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-08-31T07:12:13.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-31T07:12:13.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:13 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1693005092' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-31T07:12:13.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:13 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-31T07:12:13.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:13 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:13 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:13 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:13 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.734+0000 7f8fb7400640 1 -- 172.21.15.32:0/1802524291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb010b390 msgr2=0x7f8fb010b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:13.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.734+0000 7f8fb7400640 1 --2- 172.21.15.32:0/1802524291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb010b390 0x7f8fb010b7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009900 tx=0x7f8f9c02a410 comp rx=0 tx=0).stop 2024-08-31T07:12:13.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.734+0000 7f8fb7400640 1 -- 172.21.15.32:0/1802524291 shutdown_connections 2024-08-31T07:12:13.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.734+0000 7f8fb7400640 1 --2- 172.21.15.32:0/1802524291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb010bce0 0x7f8fb0112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.734+0000 7f8fb7400640 1 --2- 172.21.15.32:0/1802524291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb010b390 0x7f8fb010b7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.734+0000 7f8fb7400640 1 --2- 172.21.15.32:0/1802524291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb0105a60 0x7f8fb0105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.734+0000 7f8fb7400640 1 -- 172.21.15.32:0/1802524291 >> 172.21.15.32:0/1802524291 conn(0x7f8fb0101480 msgr2=0x7f8fb01038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:13.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.734+0000 7f8fb7400640 1 -- 172.21.15.32:0/1802524291 shutdown_connections 2024-08-31T07:12:13.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.734+0000 7f8fb7400640 1 -- 172.21.15.32:0/1802524291 wait complete. 2024-08-31T07:12:13.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.735+0000 7f8fb7400640 1 Processor -- start 2024-08-31T07:12:13.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.735+0000 7f8fb7400640 1 -- start start 2024-08-31T07:12:13.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.735+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb0105a60 0x7f8fb019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:13.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.735+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb010b390 0x7f8fb019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:13.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.735+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb010bce0 0x7f8fb01a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:13.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.735+0000 7f8fb7400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01159a0 con 0x7f8fb010b390 2024-08-31T07:12:13.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.735+0000 7f8fb7400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0115820 con 0x7f8fb0105a60 2024-08-31T07:12:13.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.735+0000 7f8fb7400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0115b20 con 0x7f8fb010bce0 2024-08-31T07:12:13.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fb4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb0105a60 0x7f8fb019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:13.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fafe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb010b390 0x7f8fb019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:13.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fb4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb0105a60 0x7f8fb019c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46108/0 (socket says 172.21.15.32:46108) 2024-08-31T07:12:13.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fb4a00640 1 -- 172.21.15.32:0/711566558 learned_addr learned my addr 172.21.15.32:0/711566558 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:13.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fb5400640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb010bce0 0x7f8fb01a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:13.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fafe00640 1 -- 172.21.15.32:0/711566558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb010bce0 msgr2=0x7f8fb01a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:13.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fafe00640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb010bce0 0x7f8fb01a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fafe00640 1 -- 172.21.15.32:0/711566558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb0105a60 msgr2=0x7f8fb019c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:13.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fafe00640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb0105a60 0x7f8fb019c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fafe00640 1 -- 172.21.15.32:0/711566558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb01a22d0 con 0x7f8fb010b390 2024-08-31T07:12:13.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fb4a00640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb0105a60 0x7f8fb019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:13.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.736+0000 7f8fb5400640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb010bce0 0x7f8fb01a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:13.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.737+0000 7f8fafe00640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb010b390 0x7f8fb019c940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c02a930 tx=0x7f8f9c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:13.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.737+0000 7f8fad600640 1 -- 172.21.15.32:0/711566558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c03f070 con 0x7f8fb010b390 2024-08-31T07:12:13.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.737+0000 7f8fad600640 1 -- 172.21.15.32:0/711566558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f9c004490 con 0x7f8fb010b390 2024-08-31T07:12:13.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.737+0000 7f8fb7400640 1 -- 172.21.15.32:0/711566558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb01a2560 con 0x7f8fb010b390 2024-08-31T07:12:13.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.737+0000 7f8fad600640 1 -- 172.21.15.32:0/711566558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c035070 con 0x7f8fb010b390 2024-08-31T07:12:13.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.737+0000 7f8fb7400640 1 -- 172.21.15.32:0/711566558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb01a8d50 con 0x7f8fb010b390 2024-08-31T07:12:13.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.739+0000 7f8fb7400640 1 -- 172.21.15.32:0/711566558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb0106cd0 con 0x7f8fb010b390 2024-08-31T07:12:13.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:13 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1693005092' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-31T07:12:13.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:13 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-31T07:12:13.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:13 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:13 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:13 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:13 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.741+0000 7f8fad600640 1 -- 172.21.15.32:0/711566558 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f8f9c03b460 con 0x7f8fb010b390 2024-08-31T07:12:13.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.742+0000 7f8fad600640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8f980b4b80 0x7f8f980b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:13.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.743+0000 7f8fad600640 1 -- 172.21.15.32:0/711566558 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f8f9c1056f0 con 0x7f8fb010b390 2024-08-31T07:12:13.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.743+0000 7f8fb4a00640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8f980b4b80 0x7f8f980b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:13.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.744+0000 7f8fb4a00640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8f980b4b80 0x7f8f980b7090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40095f0 tx=0x7f8fa4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:13.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.747+0000 7f8fad600640 1 -- 172.21.15.32:0/711566558 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8f9c0cc0d0 con 0x7f8fb010b390 2024-08-31T07:12:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:13 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1693005092' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-31T07:12:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:13 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-31T07:12:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:13 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:13 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:13 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:13 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.946+0000 7f8fb7400640 1 -- 172.21.15.32:0/711566558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fb010dad0 con 0x7f8fb010b390 2024-08-31T07:12:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.947+0000 7f8fad600640 1 -- 172.21.15.32:0/711566558 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22921 (secure 0 0 0) 0x7f8f9c0d0890 con 0x7f8fb010b390 2024-08-31T07:12:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:13.949 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:10:46.011475+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 -- 172.21.15.32:0/711566558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8f980b4b80 msgr2=0x7f8f980b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8f980b4b80 0x7f8f980b7090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40095f0 tx=0x7f8fa4005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 -- 172.21.15.32:0/711566558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb010b390 msgr2=0x7f8fb019c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb010b390 0x7f8fb019c940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c02a930 tx=0x7f8f9c031a20 comp rx=0 tx=0).stop 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 -- 172.21.15.32:0/711566558 shutdown_connections 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8f980b4b80 0x7f8f980b7090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb010bce0 0x7f8fb01a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb010b390 0x7f8fb019c940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 --2- 172.21.15.32:0/711566558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb0105a60 0x7f8fb019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 -- 172.21.15.32:0/711566558 >> 172.21.15.32:0/711566558 conn(0x7f8fb0101480 msgr2=0x7f8fb01038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 -- 172.21.15.32:0/711566558 shutdown_connections 2024-08-31T07:12:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:13.950+0000 7f8fb7400640 1 -- 172.21.15.32:0/711566558 wait complete. 2024-08-31T07:12:13.974 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-31T07:04:45.144128+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-31T07:09:43.795937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-31T07:09:46.609242+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-31T07:12:13.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-31T07:12:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.205+0000 7f79cce00640 1 -- 172.21.15.32:0/526275334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c81022b0 msgr2=0x7f79c81026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.205+0000 7f79cce00640 1 --2- 172.21.15.32:0/526275334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c81022b0 0x7f79c81026c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009900 tx=0x7f79b802a480 comp rx=0 tx=0).stop 2024-08-31T07:12:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.205+0000 7f79cce00640 1 -- 172.21.15.32:0/526275334 shutdown_connections 2024-08-31T07:12:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.205+0000 7f79cce00640 1 --2- 172.21.15.32:0/526275334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c810fe60 0x7f79c8112280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.205+0000 7f79cce00640 1 --2- 172.21.15.32:0/526275334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c8102c00 0x7f79c810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.205+0000 7f79cce00640 1 --2- 172.21.15.32:0/526275334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c81022b0 0x7f79c81026c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.205+0000 7f79cce00640 1 -- 172.21.15.32:0/526275334 >> 172.21.15.32:0/526275334 conn(0x7f79c80fe130 msgr2=0x7f79c8100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.205+0000 7f79cce00640 1 -- 172.21.15.32:0/526275334 shutdown_connections 2024-08-31T07:12:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.205+0000 7f79cce00640 1 -- 172.21.15.32:0/526275334 wait complete. 2024-08-31T07:12:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.206+0000 7f79cce00640 1 Processor -- start 2024-08-31T07:12:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.206+0000 7f79cce00640 1 -- start start 2024-08-31T07:12:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79cce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c81022b0 0x7f79c819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79cce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c8102c00 0x7f79c819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79cce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c810fe60 0x7f79c81a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79cce00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79c81159a0 con 0x7f79c810fe60 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79cce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f79c8115820 con 0x7f79c81022b0 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79cce00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f79c8115b20 con 0x7f79c8102c00 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79c6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c810fe60 0x7f79c81a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79c6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c810fe60 0x7f79c81a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35358/0 (socket says 172.21.15.32:35358) 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79c5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c81022b0 0x7f79c819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79c6800640 1 -- 172.21.15.32:0/1191345344 learned_addr learned my addr 172.21.15.32:0/1191345344 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.207+0000 7f79c5400640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c8102c00 0x7f79c819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79c5400640 1 -- 172.21.15.32:0/1191345344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c81022b0 msgr2=0x7f79c819c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79c5400640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c81022b0 0x7f79c819c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79c5400640 1 -- 172.21.15.32:0/1191345344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c810fe60 msgr2=0x7f79c81a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79c5400640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c810fe60 0x7f79c81a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79c5400640 1 -- 172.21.15.32:0/1191345344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c81a2310 con 0x7f79c8102c00 2024-08-31T07:12:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79c5e00640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c81022b0 0x7f79c819c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79c6800640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c810fe60 0x7f79c81a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79c5400640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c8102c00 0x7f79c819c980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f79b400e9f0 tx=0x7f79b400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79bea00640 1 -- 172.21.15.32:0/1191345344 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b400ccc0 con 0x7f79c8102c00 2024-08-31T07:12:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79bea00640 1 -- 172.21.15.32:0/1191345344 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4005020 con 0x7f79c8102c00 2024-08-31T07:12:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79cce00640 1 -- 172.21.15.32:0/1191345344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c81a2600 con 0x7f79c8102c00 2024-08-31T07:12:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79bea00640 1 -- 172.21.15.32:0/1191345344 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4004a00 con 0x7f79c8102c00 2024-08-31T07:12:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.208+0000 7f79cce00640 1 -- 172.21.15.32:0/1191345344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c81a8d90 con 0x7f79c8102c00 2024-08-31T07:12:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.209+0000 7f79cce00640 1 -- 172.21.15.32:0/1191345344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c81026c0 con 0x7f79c8102c00 2024-08-31T07:12:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.212+0000 7f79bea00640 1 -- 172.21.15.32:0/1191345344 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f79b4004ba0 con 0x7f79c8102c00 2024-08-31T07:12:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.214+0000 7f79bea00640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79ac0b4b80 0x7f79ac0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.214+0000 7f79bea00640 1 -- 172.21.15.32:0/1191345344 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f79b40e5290 con 0x7f79c8102c00 2024-08-31T07:12:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.215+0000 7f79c5e00640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79ac0b4b80 0x7f79ac0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.215+0000 7f79c5e00640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79ac0b4b80 0x7f79ac0b7090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79b8005ab0 tx=0x7f79b8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.215+0000 7f79bea00640 1 -- 172.21.15.32:0/1191345344 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f79b40ac350 con 0x7f79c8102c00 2024-08-31T07:12:14.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.471+0000 7f79cce00640 1 -- 172.21.15.32:0/1191345344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f79c802e430 con 0x7f79c8102c00 2024-08-31T07:12:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.472+0000 7f79bea00640 1 -- 172.21.15.32:0/1191345344 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f79b40abc50 con 0x7f79c8102c00 2024-08-31T07:12:14.472 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-08-31T07:12:14.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.474+0000 7f79cce00640 1 -- 172.21.15.32:0/1191345344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79ac0b4b80 msgr2=0x7f79ac0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.474+0000 7f79cce00640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79ac0b4b80 0x7f79ac0b7090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79b8005ab0 tx=0x7f79b8005a20 comp rx=0 tx=0).stop 2024-08-31T07:12:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.474+0000 7f79cce00640 1 -- 172.21.15.32:0/1191345344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c8102c00 msgr2=0x7f79c819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.474+0000 7f79cce00640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c8102c00 0x7f79c819c980 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f79b400e9f0 tx=0x7f79b400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.474+0000 7f79cce00640 1 -- 172.21.15.32:0/1191345344 shutdown_connections 2024-08-31T07:12:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.474+0000 7f79cce00640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79ac0b4b80 0x7f79ac0b7090 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.474+0000 7f79cce00640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c810fe60 0x7f79c81a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.474+0000 7f79cce00640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c8102c00 0x7f79c819c980 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.474+0000 7f79cce00640 1 --2- 172.21.15.32:0/1191345344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c81022b0 0x7f79c819c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.475+0000 7f79cce00640 1 -- 172.21.15.32:0/1191345344 >> 172.21.15.32:0/1191345344 conn(0x7f79c80fe130 msgr2=0x7f79c8111e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.475+0000 7f79cce00640 1 -- 172.21.15.32:0/1191345344 shutdown_connections 2024-08-31T07:12:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.475+0000 7f79cce00640 1 -- 172.21.15.32:0/1191345344 wait complete. 2024-08-31T07:12:14.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-31T07:12:14.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:14 smithi073 ceph-mon[21995]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-08-31T07:12:14.747 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:14 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-31T07:12:14.748 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:14 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:12:14.748 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:14.748 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:14.748 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:14 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/711566558' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:12:14.749 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:14 smithi084 ceph-mon[22387]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-08-31T07:12:14.749 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:14 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-31T07:12:14.749 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:14 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:12:14.749 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:14.749 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:14.749 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:14 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/711566558' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:12:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.734+0000 7f55c1800640 1 -- 172.21.15.32:0/799342983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc10b1e0 msgr2=0x7f55bc10b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.734+0000 7f55c1800640 1 --2- 172.21.15.32:0/799342983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc10b1e0 0x7f55bc10b5f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009900 tx=0x7f55a802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.734+0000 7f55c1800640 1 -- 172.21.15.32:0/799342983 shutdown_connections 2024-08-31T07:12:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.734+0000 7f55c1800640 1 --2- 172.21.15.32:0/799342983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55bc10bb30 0x7f55bc112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.734+0000 7f55c1800640 1 --2- 172.21.15.32:0/799342983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc10b1e0 0x7f55bc10b5f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.734+0000 7f55c1800640 1 --2- 172.21.15.32:0/799342983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55bc1058b0 0x7f55bc105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.734+0000 7f55c1800640 1 -- 172.21.15.32:0/799342983 >> 172.21.15.32:0/799342983 conn(0x7f55bc1012b0 msgr2=0x7f55bc103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.735+0000 7f55c1800640 1 -- 172.21.15.32:0/799342983 shutdown_connections 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.735+0000 7f55c1800640 1 -- 172.21.15.32:0/799342983 wait complete. 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.736+0000 7f55c1800640 1 Processor -- start 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.736+0000 7f55c1800640 1 -- start start 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.736+0000 7f55c1800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc1058b0 0x7f55bc19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.736+0000 7f55c1800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55bc10b1e0 0x7f55bc19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.736+0000 7f55c1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55bc10bb30 0x7f55bc1a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.736+0000 7f55c1800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc115790 con 0x7f55bc1058b0 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.736+0000 7f55c1800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc115610 con 0x7f55bc10bb30 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.736+0000 7f55c1800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc115910 con 0x7f55bc10b1e0 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc1058b0 0x7f55bc19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bb400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55bc10b1e0 0x7f55bc19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc1058b0 0x7f55bc19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35370/0 (socket says 172.21.15.32:35370) 2024-08-31T07:12:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bb400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55bc10b1e0 0x7f55bc19c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34926/0 (socket says 172.21.15.32:34926) 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bbe00640 1 -- 172.21.15.32:0/1009403590 learned_addr learned my addr 172.21.15.32:0/1009403590 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55c0e00640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55bc10bb30 0x7f55bc1a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bbe00640 1 -- 172.21.15.32:0/1009403590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55bc10b1e0 msgr2=0x7f55bc19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bbe00640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55bc10b1e0 0x7f55bc19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bbe00640 1 -- 172.21.15.32:0/1009403590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55bc10bb30 msgr2=0x7f55bc1a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bbe00640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55bc10bb30 0x7f55bc1a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bbe00640 1 -- 172.21.15.32:0/1009403590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc1a20e0 con 0x7f55bc1058b0 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bb400640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55bc10b1e0 0x7f55bc19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.737+0000 7f55bbe00640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc1058b0 0x7f55bc19c1b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00d950 tx=0x7f55ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.738+0000 7f55b8c00640 1 -- 172.21.15.32:0/1009403590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac017070 con 0x7f55bc1058b0 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.738+0000 7f55b8c00640 1 -- 172.21.15.32:0/1009403590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55ac009c30 con 0x7f55bc1058b0 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.738+0000 7f55c1800640 1 -- 172.21.15.32:0/1009403590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc1a23d0 con 0x7f55bc1058b0 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.738+0000 7f55b8c00640 1 -- 172.21.15.32:0/1009403590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00f620 con 0x7f55bc1058b0 2024-08-31T07:12:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.738+0000 7f55c1800640 1 -- 172.21.15.32:0/1009403590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc1a8b60 con 0x7f55bc1058b0 2024-08-31T07:12:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.739+0000 7f55c1800640 1 -- 172.21.15.32:0/1009403590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc105cc0 con 0x7f55bc1058b0 2024-08-31T07:12:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.741+0000 7f55b8c00640 1 -- 172.21.15.32:0/1009403590 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f55ac010460 con 0x7f55bc1058b0 2024-08-31T07:12:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.743+0000 7f55b8c00640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f55a00b4b80 0x7f55a00b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.743+0000 7f55b8c00640 1 -- 172.21.15.32:0/1009403590 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f55ac0e4e40 con 0x7f55bc1058b0 2024-08-31T07:12:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.743+0000 7f55bb400640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f55a00b4b80 0x7f55a00b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.744+0000 7f55bb400640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f55a00b4b80 0x7f55a00b7090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009230 tx=0x7f55a8031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.746+0000 7f55b8c00640 1 -- 172.21.15.32:0/1009403590 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f55ac009de0 con 0x7f55bc1058b0 2024-08-31T07:12:14.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:14 smithi032 ceph-mon[17252]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-08-31T07:12:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:14 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-31T07:12:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:14 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:12:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:14 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/711566558' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:12:14.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.945+0000 7f55c1800640 1 -- 172.21.15.32:0/1009403590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55bc02e430 con 0x7f55bc1058b0 2024-08-31T07:12:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.946+0000 7f55b8c00640 1 -- 172.21.15.32:0/1009403590 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f55ac0ab820 con 0x7f55bc1058b0 2024-08-31T07:12:14.947 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-08-31T07:12:14.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.948+0000 7f55c1800640 1 -- 172.21.15.32:0/1009403590 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f55a00b4b80 msgr2=0x7f55a00b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f55a00b4b80 0x7f55a00b7090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009230 tx=0x7f55a8031400 comp rx=0 tx=0).stop 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 -- 172.21.15.32:0/1009403590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc1058b0 msgr2=0x7f55bc19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc1058b0 0x7f55bc19c1b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00d950 tx=0x7f55ac00de10 comp rx=0 tx=0).stop 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 -- 172.21.15.32:0/1009403590 shutdown_connections 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f55a00b4b80 0x7f55a00b7090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55bc10bb30 0x7f55bc1a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55bc10b1e0 0x7f55bc19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 --2- 172.21.15.32:0/1009403590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc1058b0 0x7f55bc19c1b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 -- 172.21.15.32:0/1009403590 >> 172.21.15.32:0/1009403590 conn(0x7f55bc1012b0 msgr2=0x7f55bc108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 -- 172.21.15.32:0/1009403590 shutdown_connections 2024-08-31T07:12:14.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:14.949+0000 7f55c1800640 1 -- 172.21.15.32:0/1009403590 wait complete. 2024-08-31T07:12:14.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-31T07:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.200+0000 7f40fca00640 1 -- 172.21.15.32:0/352090851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f8109950 msgr2=0x7f40f8110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.200+0000 7f40fca00640 1 --2- 172.21.15.32:0/352090851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f8109950 0x7f40f8110180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009ca0 tx=0x7f40f002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.200+0000 7f40fca00640 1 -- 172.21.15.32:0/352090851 shutdown_connections 2024-08-31T07:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.200+0000 7f40fca00640 1 --2- 172.21.15.32:0/352090851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f8109950 0x7f40f8110180 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.200+0000 7f40fca00640 1 --2- 172.21.15.32:0/352090851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f8109000 0x7f40f8109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.200+0000 7f40fca00640 1 --2- 172.21.15.32:0/352090851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f81036d0 0x7f40f8103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.200+0000 7f40fca00640 1 -- 172.21.15.32:0/352090851 >> 172.21.15.32:0/352090851 conn(0x7f40f80ff130 msgr2=0x7f40f8101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.201+0000 7f40fca00640 1 -- 172.21.15.32:0/352090851 shutdown_connections 2024-08-31T07:12:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.201+0000 7f40fca00640 1 -- 172.21.15.32:0/352090851 wait complete. 2024-08-31T07:12:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40fca00640 1 Processor -- start 2024-08-31T07:12:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40fca00640 1 -- start start 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40fca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f81036d0 0x7f40f819c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40fca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f8109000 0x7f40f819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40fca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f8109950 0x7f40f81a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40fca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8113600 con 0x7f40f81036d0 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40fca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8113480 con 0x7f40f8109000 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40fca00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8113780 con 0x7f40f8109950 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40f6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f81036d0 0x7f40f819c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40f5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f8109000 0x7f40f819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40f6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f81036d0 0x7f40f819c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35384/0 (socket says 172.21.15.32:35384) 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.202+0000 7f40f6800640 1 -- 172.21.15.32:0/335379220 learned_addr learned my addr 172.21.15.32:0/335379220 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40f7200640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f8109950 0x7f40f81a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40f6800640 1 -- 172.21.15.32:0/335379220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f8109950 msgr2=0x7f40f81a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40f6800640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f8109950 0x7f40f81a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40f6800640 1 -- 172.21.15.32:0/335379220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f8109000 msgr2=0x7f40f819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40f6800640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f8109000 0x7f40f819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40f6800640 1 -- 172.21.15.32:0/335379220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f81a20d0 con 0x7f40f81036d0 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40f7200640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f8109950 0x7f40f81a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40f6800640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f81036d0 0x7f40f819c1a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f40e800ea60 tx=0x7f40e800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40ef400640 1 -- 172.21.15.32:0/335379220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e800cde0 con 0x7f40f81036d0 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40ef400640 1 -- 172.21.15.32:0/335379220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e80035f0 con 0x7f40f81036d0 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.203+0000 7f40fca00640 1 -- 172.21.15.32:0/335379220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f81a23c0 con 0x7f40f81036d0 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.204+0000 7f40ef400640 1 -- 172.21.15.32:0/335379220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8010690 con 0x7f40f81036d0 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.204+0000 7f40fca00640 1 -- 172.21.15.32:0/335379220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f81a8b50 con 0x7f40f81036d0 2024-08-31T07:12:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.205+0000 7f40fca00640 1 -- 172.21.15.32:0/335379220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f8103ae0 con 0x7f40f81036d0 2024-08-31T07:12:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.207+0000 7f40ef400640 1 -- 172.21.15.32:0/335379220 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f40e80037a0 con 0x7f40f81036d0 2024-08-31T07:12:15.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.208+0000 7f40ef400640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40dc0b4b80 0x7f40dc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:15.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.209+0000 7f40ef400640 1 -- 172.21.15.32:0/335379220 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f40e80e5640 con 0x7f40f81036d0 2024-08-31T07:12:15.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.209+0000 7f40f5e00640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40dc0b4b80 0x7f40dc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:15.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.210+0000 7f40f5e00640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40dc0b4b80 0x7f40dc0b7090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40f8102980 tx=0x7f40e4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:15.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.213+0000 7f40ef400640 1 -- 172.21.15.32:0/335379220 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f40e80ac670 con 0x7f40f81036d0 2024-08-31T07:12:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.411+0000 7f40fca00640 1 -- 172.21.15.32:0/335379220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40f81a8ee0 con 0x7f40f81036d0 2024-08-31T07:12:15.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.411+0000 7f40ef400640 1 -- 172.21.15.32:0/335379220 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f40e80abf70 con 0x7f40f81036d0 2024-08-31T07:12:15.412 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-08-31T07:12:15.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.413+0000 7f40fca00640 1 -- 172.21.15.32:0/335379220 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40dc0b4b80 msgr2=0x7f40dc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:15.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.413+0000 7f40fca00640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40dc0b4b80 0x7f40dc0b7090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40f8102980 tx=0x7f40e4005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.413+0000 7f40fca00640 1 -- 172.21.15.32:0/335379220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f81036d0 msgr2=0x7f40f819c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.413+0000 7f40fca00640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f81036d0 0x7f40f819c1a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f40e800ea60 tx=0x7f40e800ef20 comp rx=0 tx=0).stop 2024-08-31T07:12:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.413+0000 7f40fca00640 1 -- 172.21.15.32:0/335379220 shutdown_connections 2024-08-31T07:12:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.413+0000 7f40fca00640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40dc0b4b80 0x7f40dc0b7090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.413+0000 7f40fca00640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f8109950 0x7f40f81a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.413+0000 7f40fca00640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f8109000 0x7f40f819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.413+0000 7f40fca00640 1 --2- 172.21.15.32:0/335379220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f81036d0 0x7f40f819c1a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.414+0000 7f40fca00640 1 -- 172.21.15.32:0/335379220 >> 172.21.15.32:0/335379220 conn(0x7f40f80ff130 msgr2=0x7f40f8108600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.414+0000 7f40fca00640 1 -- 172.21.15.32:0/335379220 shutdown_connections 2024-08-31T07:12:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:15.414+0000 7f40fca00640 1 -- 172.21.15.32:0/335379220 wait complete. 2024-08-31T07:12:15.440 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-08-31T07:12:15.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:12:15.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:15 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1191345344' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T07:12:15.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:12:15.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:12:15.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:15.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:15 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1009403590' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T07:12:15.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:15 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/335379220' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T07:12:15.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:15 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1191345344' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T07:12:15.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:12:15.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:12:15.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:15.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:15 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1009403590' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T07:12:15.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:15 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/335379220' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T07:12:15.818 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:15.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:15 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1191345344' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T07:12:15.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:12:15.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:12:15.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:12:15.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:15 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1009403590' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T07:12:15.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:15 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/335379220' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T07:12:16.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.511+0000 7fcbe3200640 1 -- 172.21.15.32:0/1060964402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc103750 msgr2=0x7fcbdc103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:16.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.511+0000 7fcbe3200640 1 --2- 172.21.15.32:0/1060964402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc103750 0x7fcbdc103b60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009900 tx=0x7fcbd002a410 comp rx=0 tx=0).stop 2024-08-31T07:12:16.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.511+0000 7fcbe3200640 1 -- 172.21.15.32:0/1060964402 shutdown_connections 2024-08-31T07:12:16.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.511+0000 7fcbe3200640 1 --2- 172.21.15.32:0/1060964402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbdc1099e0 0x7fcbdc110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:16.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.511+0000 7fcbe3200640 1 --2- 172.21.15.32:0/1060964402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbdc109090 0x7fcbdc1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:16.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.511+0000 7fcbe3200640 1 --2- 172.21.15.32:0/1060964402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc103750 0x7fcbdc103b60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:16.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.511+0000 7fcbe3200640 1 -- 172.21.15.32:0/1060964402 >> 172.21.15.32:0/1060964402 conn(0x7fcbdc0ff1a0 msgr2=0x7fcbdc1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:16.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.512+0000 7fcbe3200640 1 -- 172.21.15.32:0/1060964402 shutdown_connections 2024-08-31T07:12:16.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.512+0000 7fcbe3200640 1 -- 172.21.15.32:0/1060964402 wait complete. 2024-08-31T07:12:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.513+0000 7fcbe3200640 1 Processor -- start 2024-08-31T07:12:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.513+0000 7fcbe3200640 1 -- start start 2024-08-31T07:12:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.513+0000 7fcbe3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc103750 0x7fcbdc19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.513+0000 7fcbe3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbdc109090 0x7fcbdc19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.513+0000 7fcbe3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbdc1099e0 0x7fcbdc1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.513+0000 7fcbe3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc1136e0 con 0x7fcbdc103750 2024-08-31T07:12:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.513+0000 7fcbe3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc113560 con 0x7fcbdc109090 2024-08-31T07:12:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.513+0000 7fcbe3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc113860 con 0x7fcbdc1099e0 2024-08-31T07:12:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc103750 0x7fcbdc19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc103750 0x7fcbdc19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58426/0 (socket says 172.21.15.32:58426) 2024-08-31T07:12:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe1e00640 1 -- 172.21.15.32:0/643085693 learned_addr learned my addr 172.21.15.32:0/643085693 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe2800640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbdc1099e0 0x7fcbdc1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe1400640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbdc109090 0x7fcbdc19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe1e00640 1 -- 172.21.15.32:0/643085693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbdc1099e0 msgr2=0x7fcbdc1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe1e00640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbdc1099e0 0x7fcbdc1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe1e00640 1 -- 172.21.15.32:0/643085693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbdc109090 msgr2=0x7fcbdc19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe1e00640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbdc109090 0x7fcbdc19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe1e00640 1 -- 172.21.15.32:0/643085693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbdc1a2220 con 0x7fcbdc103750 2024-08-31T07:12:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.514+0000 7fcbe1e00640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc103750 0x7fcbdc19c2b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0005ab0 tx=0x7fcbd0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.515+0000 7fcbd6a00640 1 -- 172.21.15.32:0/643085693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd003f070 con 0x7fcbdc103750 2024-08-31T07:12:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.515+0000 7fcbd6a00640 1 -- 172.21.15.32:0/643085693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbd0031d60 con 0x7fcbdc103750 2024-08-31T07:12:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.515+0000 7fcbd6a00640 1 -- 172.21.15.32:0/643085693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0035030 con 0x7fcbdc103750 2024-08-31T07:12:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.515+0000 7fcbe3200640 1 -- 172.21.15.32:0/643085693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbdc1a24b0 con 0x7fcbdc103750 2024-08-31T07:12:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.515+0000 7fcbe3200640 1 -- 172.21.15.32:0/643085693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbdc1a8c70 con 0x7fcbdc103750 2024-08-31T07:12:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.516+0000 7fcbe3200640 1 -- 172.21.15.32:0/643085693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbdc103b60 con 0x7fcbdc103750 2024-08-31T07:12:16.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.518+0000 7fcbd6a00640 1 -- 172.21.15.32:0/643085693 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7fcbd0042070 con 0x7fcbdc103750 2024-08-31T07:12:16.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.520+0000 7fcbd6a00640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcbc40b4b80 0x7fcbc40b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:16.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.520+0000 7fcbd6a00640 1 -- 172.21.15.32:0/643085693 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fcbd01060a0 con 0x7fcbdc103750 2024-08-31T07:12:16.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.520+0000 7fcbe1400640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcbc40b4b80 0x7fcbc40b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:16.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.521+0000 7fcbe1400640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcbc40b4b80 0x7fcbc40b7090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc1057e0 tx=0x7fcbcc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.522+0000 7fcbd6a00640 1 -- 172.21.15.32:0/643085693 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcbd00cca80 con 0x7fcbdc103750 2024-08-31T07:12:16.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:16 smithi032 ceph-mon[17252]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-08-31T07:12:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.759+0000 7fcbe3200640 1 -- 172.21.15.32:0/643085693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbdc1a8fa0 con 0x7fcbdc103750 2024-08-31T07:12:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.760+0000 7fcbd6a00640 1 -- 172.21.15.32:0/643085693 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7fcbd00d1240 con 0x7fcbdc103750 2024-08-31T07:12:16.762 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:12:16.763 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:12:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 -- 172.21.15.32:0/643085693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcbc40b4b80 msgr2=0x7fcbc40b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:16.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcbc40b4b80 0x7fcbc40b7090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc1057e0 tx=0x7fcbcc005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 -- 172.21.15.32:0/643085693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc103750 msgr2=0x7fcbdc19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc103750 0x7fcbdc19c2b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0005ab0 tx=0x7fcbd0003c80 comp rx=0 tx=0).stop 2024-08-31T07:12:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 -- 172.21.15.32:0/643085693 shutdown_connections 2024-08-31T07:12:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcbc40b4b80 0x7fcbc40b7090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbdc1099e0 0x7fcbdc1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbdc109090 0x7fcbdc19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 --2- 172.21.15.32:0/643085693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbdc103750 0x7fcbdc19c2b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 -- 172.21.15.32:0/643085693 >> 172.21.15.32:0/643085693 conn(0x7fcbdc0ff1a0 msgr2=0x7fcbdc108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 -- 172.21.15.32:0/643085693 shutdown_connections 2024-08-31T07:12:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:16.763+0000 7fcbe3200640 1 -- 172.21.15.32:0/643085693 wait complete. 2024-08-31T07:12:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:16 smithi073 ceph-mon[21995]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-08-31T07:12:16.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:16 smithi084 ceph-mon[22387]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-08-31T07:12:17.603 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-08-31T07:12:17.603 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-08-31T07:12:17.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-31T07:12:17.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.830+0000 7f79bfe00640 1 -- 172.21.15.32:0/2643899846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c0103660 msgr2=0x7f79c0103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:17.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.830+0000 7f79bfe00640 1 --2- 172.21.15.32:0/2643899846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c0103660 0x7f79c0103a70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009860 tx=0x7f79b002a340 comp rx=0 tx=0).stop 2024-08-31T07:12:17.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.830+0000 7f79bfe00640 1 -- 172.21.15.32:0/2643899846 shutdown_connections 2024-08-31T07:12:17.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.830+0000 7f79bfe00640 1 --2- 172.21.15.32:0/2643899846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c01098e0 0x7f79c0110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:17.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.830+0000 7f79bfe00640 1 --2- 172.21.15.32:0/2643899846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c0108f90 0x7f79c01093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:17.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.830+0000 7f79bfe00640 1 --2- 172.21.15.32:0/2643899846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c0103660 0x7f79c0103a70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:17.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.830+0000 7f79bfe00640 1 -- 172.21.15.32:0/2643899846 >> 172.21.15.32:0/2643899846 conn(0x7f79c00ff0a0 msgr2=0x7f79c01014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:17.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.830+0000 7f79bfe00640 1 -- 172.21.15.32:0/2643899846 shutdown_connections 2024-08-31T07:12:17.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.830+0000 7f79bfe00640 1 -- 172.21.15.32:0/2643899846 wait complete. 2024-08-31T07:12:17.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.831+0000 7f79bfe00640 1 Processor -- start 2024-08-31T07:12:17.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.831+0000 7f79bfe00640 1 -- start start 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c0103660 0x7f79c00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c0108f90 0x7f79c00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c01098e0 0x7f79c0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79bfe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79c01135b0 con 0x7f79c01098e0 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79bfe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f79c0113430 con 0x7f79c0103660 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79bfe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f79c0113730 con 0x7f79c0108f90 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79bea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c0103660 0x7f79c00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79bf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c01098e0 0x7f79c0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79bea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c0103660 0x7f79c00754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57078/0 (socket says 172.21.15.32:57078) 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79bea00640 1 -- 172.21.15.32:0/3027424147 learned_addr learned my addr 172.21.15.32:0/3027424147 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.832+0000 7f79b7e00640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c0108f90 0x7f79c00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:17.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79bf400640 1 -- 172.21.15.32:0/3027424147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c0108f90 msgr2=0x7f79c00759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79bf400640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c0108f90 0x7f79c00759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79bf400640 1 -- 172.21.15.32:0/3027424147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c0103660 msgr2=0x7f79c00754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79bf400640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c0103660 0x7f79c00754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79bf400640 1 -- 172.21.15.32:0/3027424147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c01a82a0 con 0x7f79c01098e0 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79bea00640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c0103660 0x7f79c00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79bf400640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c01098e0 0x7f79c0075f30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009fd0 tx=0x7f79b800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79b7400640 1 -- 172.21.15.32:0/3027424147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b800ce20 con 0x7f79c01098e0 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79bfe00640 1 -- 172.21.15.32:0/3027424147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c01a8590 con 0x7f79c01098e0 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79b7400640 1 -- 172.21.15.32:0/3027424147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b8005020 con 0x7f79c01098e0 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79b7400640 1 -- 172.21.15.32:0/3027424147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8010690 con 0x7f79c01098e0 2024-08-31T07:12:17.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.833+0000 7f79bfe00640 1 -- 172.21.15.32:0/3027424147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c01a8ae0 con 0x7f79c01098e0 2024-08-31T07:12:17.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.834+0000 7f79bfe00640 1 -- 172.21.15.32:0/3027424147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c0103a70 con 0x7f79c01098e0 2024-08-31T07:12:17.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.837+0000 7f79b7400640 1 -- 172.21.15.32:0/3027424147 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f79b80054d0 con 0x7f79c01098e0 2024-08-31T07:12:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.838+0000 7f79b7400640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79a40b4bb0 0x7f79a40b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.839+0000 7f79b7400640 1 -- 172.21.15.32:0/3027424147 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f79b80e5880 con 0x7f79c01098e0 2024-08-31T07:12:17.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.839+0000 7f79bea00640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79a40b4bb0 0x7f79a40b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:17.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.839+0000 7f79bea00640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79a40b4bb0 0x7f79a40b70c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f79b002a930 tx=0x7f79b0030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:17.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:17.840+0000 7f79b7400640 1 -- 172.21.15.32:0/3027424147 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f79b80ac8b0 con 0x7f79c01098e0 2024-08-31T07:12:17.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:17 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/643085693' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:17.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:17 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/643085693' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:17.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:17 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/643085693' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.035+0000 7f79bfe00640 1 -- 172.21.15.32:0/3027424147 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79c00635a0 con 0x7f79a40b4bb0 2024-08-31T07:12:18.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.040+0000 7f79b7400640 1 -- 172.21.15.32:0/3027424147 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259134 (secure 0 0 0) 0x7f79c00635a0 con 0x7f79a40b4bb0 2024-08-31T07:12:18.045 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-08-31T07:12:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 -- 172.21.15.32:0/3027424147 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79a40b4bb0 msgr2=0x7f79a40b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79a40b4bb0 0x7f79a40b70c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f79b002a930 tx=0x7f79b0030340 comp rx=0 tx=0).stop 2024-08-31T07:12:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 -- 172.21.15.32:0/3027424147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c01098e0 msgr2=0x7f79c0075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c01098e0 0x7f79c0075f30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009fd0 tx=0x7f79b800ef10 comp rx=0 tx=0).stop 2024-08-31T07:12:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 -- 172.21.15.32:0/3027424147 shutdown_connections 2024-08-31T07:12:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79a40b4bb0 0x7f79a40b70c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79c01098e0 0x7f79c0075f30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79c0108f90 0x7f79c00759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 --2- 172.21.15.32:0/3027424147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79c0103660 0x7f79c00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 -- 172.21.15.32:0/3027424147 >> 172.21.15.32:0/3027424147 conn(0x7f79c00ff0a0 msgr2=0x7f79c01014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 -- 172.21.15.32:0/3027424147 shutdown_connections 2024-08-31T07:12:18.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.044+0000 7f79bfe00640 1 -- 172.21.15.32:0/3027424147 wait complete. 2024-08-31T07:12:18.069 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-08-31T07:12:16.797561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660329,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1429,"num_read_kb":816,"num_write":389,"num_write_kb":2303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2952490,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":821,"ondisk_log_size":821,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364208,"kb_used_data":10800,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577616,"statfs":{"total":1151940427776,"available":1151567478784,"internally_reserved":0,"allocated":11059200,"data_stored":7166427,"data_compressed":177834,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":311,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002907"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198399+0000","last_change":"2024-08-31T07:09:47.665725+0000","last_active":"2024-08-31T07:11:45.198399+0000","last_peered":"2024-08-31T07:11:45.198399+0000","last_clean":"2024-08-31T07:11:45.198399+0000","last_became_active":"2024-08-31T07:09:47.665368+0000","last_became_peered":"2024-08-31T07:09:47.665368+0000","last_unstale":"2024-08-31T07:11:45.198399+0000","last_undegraded":"2024-08-31T07:11:45.198399+0000","last_fullsized":"2024-08-31T07:11:45.198399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:27:01.636099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192497+0000","last_change":"2024-08-31T07:09:47.676882+0000","last_active":"2024-08-31T07:11:45.192497+0000","last_peered":"2024-08-31T07:11:45.192497+0000","last_clean":"2024-08-31T07:11:45.192497+0000","last_became_active":"2024-08-31T07:09:47.674825+0000","last_became_peered":"2024-08-31T07:09:47.674825+0000","last_unstale":"2024-08-31T07:11:45.192497+0000","last_undegraded":"2024-08-31T07:11:45.192497+0000","last_fullsized":"2024-08-31T07:11:45.192497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:09.148398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199170+0000","last_change":"2024-08-31T07:09:47.674480+0000","last_active":"2024-08-31T07:11:45.199170+0000","last_peered":"2024-08-31T07:11:45.199170+0000","last_clean":"2024-08-31T07:11:45.199170+0000","last_became_active":"2024-08-31T07:09:47.672316+0000","last_became_peered":"2024-08-31T07:09:47.672316+0000","last_unstale":"2024-08-31T07:11:45.199170+0000","last_undegraded":"2024-08-31T07:11:45.199170+0000","last_fullsized":"2024-08-31T07:11:45.199170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:28:13.231039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.172744+0000","last_change":"2024-08-31T07:09:47.659276+0000","last_active":"2024-08-31T07:11:45.172744+0000","last_peered":"2024-08-31T07:11:45.172744+0000","last_clean":"2024-08-31T07:11:45.172744+0000","last_became_active":"2024-08-31T07:09:47.658924+0000","last_became_peered":"2024-08-31T07:09:47.658924+0000","last_unstale":"2024-08-31T07:11:45.172744+0000","last_undegraded":"2024-08-31T07:11:45.172744+0000","last_fullsized":"2024-08-31T07:11:45.172744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:01:08.134245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.174698+0000","last_change":"2024-08-31T07:09:47.667260+0000","last_active":"2024-08-31T07:11:45.174698+0000","last_peered":"2024-08-31T07:11:45.174698+0000","last_clean":"2024-08-31T07:11:45.174698+0000","last_became_active":"2024-08-31T07:09:47.664572+0000","last_became_peered":"2024-08-31T07:09:47.664572+0000","last_unstale":"2024-08-31T07:11:45.174698+0000","last_undegraded":"2024-08-31T07:11:45.174698+0000","last_fullsized":"2024-08-31T07:11:45.174698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:11.336985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188974+0000","last_change":"2024-08-31T07:09:47.661181+0000","last_active":"2024-08-31T07:11:45.188974+0000","last_peered":"2024-08-31T07:11:45.188974+0000","last_clean":"2024-08-31T07:11:45.188974+0000","last_became_active":"2024-08-31T07:09:47.660761+0000","last_became_peered":"2024-08-31T07:09:47.660761+0000","last_unstale":"2024-08-31T07:11:45.188974+0000","last_undegraded":"2024-08-31T07:11:45.188974+0000","last_fullsized":"2024-08-31T07:11:45.188974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:50:54.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196463+0000","last_change":"2024-08-31T07:09:47.668762+0000","last_active":"2024-08-31T07:11:45.196463+0000","last_peered":"2024-08-31T07:11:45.196463+0000","last_clean":"2024-08-31T07:11:45.196463+0000","last_became_active":"2024-08-31T07:09:47.668406+0000","last_became_peered":"2024-08-31T07:09:47.668406+0000","last_unstale":"2024-08-31T07:11:45.196463+0000","last_undegraded":"2024-08-31T07:11:45.196463+0000","last_fullsized":"2024-08-31T07:11:45.196463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:43:39.868144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.190957+0000","last_change":"2024-08-31T07:09:47.669455+0000","last_active":"2024-08-31T07:11:45.190957+0000","last_peered":"2024-08-31T07:11:45.190957+0000","last_clean":"2024-08-31T07:11:45.190957+0000","last_became_active":"2024-08-31T07:09:47.663669+0000","last_became_peered":"2024-08-31T07:09:47.663669+0000","last_unstale":"2024-08-31T07:11:45.190957+0000","last_undegraded":"2024-08-31T07:11:45.190957+0000","last_fullsized":"2024-08-31T07:11:45.190957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:22:59.685012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194827+0000","last_change":"2024-08-31T07:09:47.664214+0000","last_active":"2024-08-31T07:11:45.194827+0000","last_peered":"2024-08-31T07:11:45.194827+0000","last_clean":"2024-08-31T07:11:45.194827+0000","last_became_active":"2024-08-31T07:09:47.663832+0000","last_became_peered":"2024-08-31T07:09:47.663832+0000","last_unstale":"2024-08-31T07:11:45.194827+0000","last_undegraded":"2024-08-31T07:11:45.194827+0000","last_fullsized":"2024-08-31T07:11:45.194827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:29:14.107212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194866+0000","last_change":"2024-08-31T07:10:46.326410+0000","last_active":"2024-08-31T07:11:45.194866+0000","last_peered":"2024-08-31T07:11:45.194866+0000","last_clean":"2024-08-31T07:11:45.194866+0000","last_became_active":"2024-08-31T07:10:46.325958+0000","last_became_peered":"2024-08-31T07:10:46.325958+0000","last_unstale":"2024-08-31T07:11:45.194866+0000","last_undegraded":"2024-08-31T07:11:45.194866+0000","last_fullsized":"2024-08-31T07:11:45.194866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:19.230393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196579+0000","last_change":"2024-08-31T07:09:47.666951+0000","last_active":"2024-08-31T07:11:45.196579+0000","last_peered":"2024-08-31T07:11:45.196579+0000","last_clean":"2024-08-31T07:11:45.196579+0000","last_became_active":"2024-08-31T07:09:47.666475+0000","last_became_peered":"2024-08-31T07:09:47.666475+0000","last_unstale":"2024-08-31T07:11:45.196579+0000","last_undegraded":"2024-08-31T07:11:45.196579+0000","last_fullsized":"2024-08-31T07:11:45.196579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:58:42.242086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.169998+0000","last_change":"2024-08-31T07:09:47.658344+0000","last_active":"2024-08-31T07:11:45.169998+0000","last_peered":"2024-08-31T07:11:45.169998+0000","last_clean":"2024-08-31T07:11:45.169998+0000","last_became_active":"2024-08-31T07:09:47.657991+0000","last_became_peered":"2024-08-31T07:09:47.657991+0000","last_unstale":"2024-08-31T07:11:45.169998+0000","last_undegraded":"2024-08-31T07:11:45.169998+0000","last_fullsized":"2024-08-31T07:11:45.169998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:25:59.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196676+0000","last_change":"2024-08-31T07:09:47.661644+0000","last_active":"2024-08-31T07:11:45.196676+0000","last_peered":"2024-08-31T07:11:45.196676+0000","last_clean":"2024-08-31T07:11:45.196676+0000","last_became_active":"2024-08-31T07:09:47.661248+0000","last_became_peered":"2024-08-31T07:09:47.661248+0000","last_unstale":"2024-08-31T07:11:45.196676+0000","last_undegraded":"2024-08-31T07:11:45.196676+0000","last_fullsized":"2024-08-31T07:11:45.196676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:42.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195194+0000","last_change":"2024-08-31T07:09:47.680052+0000","last_active":"2024-08-31T07:11:45.195194+0000","last_peered":"2024-08-31T07:11:45.195194+0000","last_clean":"2024-08-31T07:11:45.195194+0000","last_became_active":"2024-08-31T07:09:47.679604+0000","last_became_peered":"2024-08-31T07:09:47.679604+0000","last_unstale":"2024-08-31T07:11:45.195194+0000","last_undegraded":"2024-08-31T07:11:45.195194+0000","last_fullsized":"2024-08-31T07:11:45.195194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:23:18.937470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195477+0000","last_change":"2024-08-31T07:09:47.677984+0000","last_active":"2024-08-31T07:11:45.195477+0000","last_peered":"2024-08-31T07:11:45.195477+0000","last_clean":"2024-08-31T07:11:45.195477+0000","last_became_active":"2024-08-31T07:09:47.674957+0000","last_became_peered":"2024-08-31T07:09:47.674957+0000","last_unstale":"2024-08-31T07:11:45.195477+0000","last_undegraded":"2024-08-31T07:11:45.195477+0000","last_fullsized":"2024-08-31T07:11:45.195477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:08:26.503746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198878+0000","last_change":"2024-08-31T07:09:47.672625+0000","last_active":"2024-08-31T07:11:45.198878+0000","last_peered":"2024-08-31T07:11:45.198878+0000","last_clean":"2024-08-31T07:11:45.198878+0000","last_became_active":"2024-08-31T07:09:47.672184+0000","last_became_peered":"2024-08-31T07:09:47.672184+0000","last_unstale":"2024-08-31T07:11:45.198878+0000","last_undegraded":"2024-08-31T07:11:45.198878+0000","last_fullsized":"2024-08-31T07:11:45.198878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:17.091659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198670+0000","last_change":"2024-08-31T07:09:47.672597+0000","last_active":"2024-08-31T07:11:45.198670+0000","last_peered":"2024-08-31T07:11:45.198670+0000","last_clean":"2024-08-31T07:11:45.198670+0000","last_became_active":"2024-08-31T07:09:47.672153+0000","last_became_peered":"2024-08-31T07:09:47.672153+0000","last_unstale":"2024-08-31T07:11:45.198670+0000","last_undegraded":"2024-08-31T07:11:45.198670+0000","last_fullsized":"2024-08-31T07:11:45.198670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:26:41.388526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193223+0000","last_change":"2024-08-31T07:09:47.666253+0000","last_active":"2024-08-31T07:11:45.193223+0000","last_peered":"2024-08-31T07:11:45.193223+0000","last_clean":"2024-08-31T07:11:45.193223+0000","last_became_active":"2024-08-31T07:09:47.663732+0000","last_became_peered":"2024-08-31T07:09:47.663732+0000","last_unstale":"2024-08-31T07:11:45.193223+0000","last_undegraded":"2024-08-31T07:11:45.193223+0000","last_fullsized":"2024-08-31T07:11:45.193223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:10:59.887765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188202+0000","last_change":"2024-08-31T07:09:47.668502+0000","last_active":"2024-08-31T07:11:45.188202+0000","last_peered":"2024-08-31T07:11:45.188202+0000","last_clean":"2024-08-31T07:11:45.188202+0000","last_became_active":"2024-08-31T07:09:47.666487+0000","last_became_peered":"2024-08-31T07:09:47.666487+0000","last_unstale":"2024-08-31T07:11:45.188202+0000","last_undegraded":"2024-08-31T07:11:45.188202+0000","last_fullsized":"2024-08-31T07:11:45.188202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:49:25.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.184945+0000","last_change":"2024-08-31T07:09:47.659454+0000","last_active":"2024-08-31T07:11:45.184945+0000","last_peered":"2024-08-31T07:11:45.184945+0000","last_clean":"2024-08-31T07:11:45.184945+0000","last_became_active":"2024-08-31T07:09:47.658976+0000","last_became_peered":"2024-08-31T07:09:47.658976+0000","last_unstale":"2024-08-31T07:11:45.184945+0000","last_undegraded":"2024-08-31T07:11:45.184945+0000","last_fullsized":"2024-08-31T07:11:45.184945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:31.374905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199707+0000","last_change":"2024-08-31T07:09:47.673612+0000","last_active":"2024-08-31T07:11:45.199707+0000","last_peered":"2024-08-31T07:11:45.199707+0000","last_clean":"2024-08-31T07:11:45.199707+0000","last_became_active":"2024-08-31T07:09:47.673137+0000","last_became_peered":"2024-08-31T07:09:47.673137+0000","last_unstale":"2024-08-31T07:11:45.199707+0000","last_undegraded":"2024-08-31T07:11:45.199707+0000","last_fullsized":"2024-08-31T07:11:45.199707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:46:11.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194525+0000","last_change":"2024-08-31T07:09:47.669195+0000","last_active":"2024-08-31T07:11:45.194525+0000","last_peered":"2024-08-31T07:11:45.194525+0000","last_clean":"2024-08-31T07:11:45.194525+0000","last_became_active":"2024-08-31T07:09:47.668790+0000","last_became_peered":"2024-08-31T07:09:47.668790+0000","last_unstale":"2024-08-31T07:11:45.194525+0000","last_undegraded":"2024-08-31T07:11:45.194525+0000","last_fullsized":"2024-08-31T07:11:45.194525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:27:14.393931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.181333+0000","last_change":"2024-08-31T07:09:47.667566+0000","last_active":"2024-08-31T07:11:45.181333+0000","last_peered":"2024-08-31T07:11:45.181333+0000","last_clean":"2024-08-31T07:11:45.181333+0000","last_became_active":"2024-08-31T07:09:47.666605+0000","last_became_peered":"2024-08-31T07:09:47.666605+0000","last_unstale":"2024-08-31T07:11:45.181333+0000","last_undegraded":"2024-08-31T07:11:45.181333+0000","last_fullsized":"2024-08-31T07:11:45.181333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:05:44.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192240+0000","last_change":"2024-08-31T07:09:47.660478+0000","last_active":"2024-08-31T07:11:45.192240+0000","last_peered":"2024-08-31T07:11:45.192240+0000","last_clean":"2024-08-31T07:11:45.192240+0000","last_became_active":"2024-08-31T07:09:47.660107+0000","last_became_peered":"2024-08-31T07:09:47.660107+0000","last_unstale":"2024-08-31T07:11:45.192240+0000","last_undegraded":"2024-08-31T07:11:45.192240+0000","last_fullsized":"2024-08-31T07:11:45.192240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:29.133370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188131+0000","last_change":"2024-08-31T07:09:47.671967+0000","last_active":"2024-08-31T07:11:45.188131+0000","last_peered":"2024-08-31T07:11:45.188131+0000","last_clean":"2024-08-31T07:11:45.188131+0000","last_became_active":"2024-08-31T07:09:47.671569+0000","last_became_peered":"2024-08-31T07:09:47.671569+0000","last_unstale":"2024-08-31T07:11:45.188131+0000","last_undegraded":"2024-08-31T07:11:45.188131+0000","last_fullsized":"2024-08-31T07:11:45.188131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:19:15.107958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185867+0000","last_change":"2024-08-31T07:09:47.651055+0000","last_active":"2024-08-31T07:11:45.185867+0000","last_peered":"2024-08-31T07:11:45.185867+0000","last_clean":"2024-08-31T07:11:45.185867+0000","last_became_active":"2024-08-31T07:09:47.650674+0000","last_became_peered":"2024-08-31T07:09:47.650674+0000","last_unstale":"2024-08-31T07:11:45.185867+0000","last_undegraded":"2024-08-31T07:11:45.185867+0000","last_fullsized":"2024-08-31T07:11:45.185867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:35:54.647379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180140+0000","last_change":"2024-08-31T07:09:47.663231+0000","last_active":"2024-08-31T07:11:45.180140+0000","last_peered":"2024-08-31T07:11:45.180140+0000","last_clean":"2024-08-31T07:11:45.180140+0000","last_became_active":"2024-08-31T07:09:47.662854+0000","last_became_peered":"2024-08-31T07:09:47.662854+0000","last_unstale":"2024-08-31T07:11:45.180140+0000","last_undegraded":"2024-08-31T07:11:45.180140+0000","last_fullsized":"2024-08-31T07:11:45.180140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:51.358557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199287+0000","last_change":"2024-08-31T07:10:46.326380+0000","last_active":"2024-08-31T07:11:45.199287+0000","last_peered":"2024-08-31T07:11:45.199287+0000","last_clean":"2024-08-31T07:11:45.199287+0000","last_became_active":"2024-08-31T07:10:46.325784+0000","last_became_peered":"2024-08-31T07:10:46.325784+0000","last_unstale":"2024-08-31T07:11:45.199287+0000","last_undegraded":"2024-08-31T07:11:45.199287+0000","last_fullsized":"2024-08-31T07:11:45.199287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:16:56.396024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200247+0000","last_change":"2024-08-31T07:09:47.672108+0000","last_active":"2024-08-31T07:11:45.200247+0000","last_peered":"2024-08-31T07:11:45.200247+0000","last_clean":"2024-08-31T07:11:45.200247+0000","last_became_active":"2024-08-31T07:09:47.670998+0000","last_became_peered":"2024-08-31T07:09:47.670998+0000","last_unstale":"2024-08-31T07:11:45.200247+0000","last_undegraded":"2024-08-31T07:11:45.200247+0000","last_fullsized":"2024-08-31T07:11:45.200247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:26:49.319861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198226+0000","last_change":"2024-08-31T07:09:47.665548+0000","last_active":"2024-08-31T07:11:45.198226+0000","last_peered":"2024-08-31T07:11:45.198226+0000","last_clean":"2024-08-31T07:11:45.198226+0000","last_became_active":"2024-08-31T07:09:47.665177+0000","last_became_peered":"2024-08-31T07:09:47.665177+0000","last_unstale":"2024-08-31T07:11:45.198226+0000","last_undegraded":"2024-08-31T07:11:45.198226+0000","last_fullsized":"2024-08-31T07:11:45.198226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:03:15.892224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.177785+0000","last_change":"2024-08-31T07:10:46.026746+0000","last_active":"2024-08-31T07:11:45.177785+0000","last_peered":"2024-08-31T07:11:45.177785+0000","last_clean":"2024-08-31T07:11:45.177785+0000","last_became_active":"2024-08-31T07:10:46.026286+0000","last_became_peered":"2024-08-31T07:10:46.026286+0000","last_unstale":"2024-08-31T07:11:45.177785+0000","last_undegraded":"2024-08-31T07:11:45.177785+0000","last_fullsized":"2024-08-31T07:11:45.177785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:34:17.120906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180769+0000","last_change":"2024-08-31T07:09:47.667456+0000","last_active":"2024-08-31T07:11:45.180769+0000","last_peered":"2024-08-31T07:11:45.180769+0000","last_clean":"2024-08-31T07:11:45.180769+0000","last_became_active":"2024-08-31T07:09:47.667102+0000","last_became_peered":"2024-08-31T07:09:47.667102+0000","last_unstale":"2024-08-31T07:11:45.180769+0000","last_undegraded":"2024-08-31T07:11:45.180769+0000","last_fullsized":"2024-08-31T07:11:45.180769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:44:39.747729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.181991+0000","last_change":"2024-08-31T07:10:46.326139+0000","last_active":"2024-08-31T07:11:45.181991+0000","last_peered":"2024-08-31T07:11:45.181991+0000","last_clean":"2024-08-31T07:11:45.181991+0000","last_became_active":"2024-08-31T07:10:46.325785+0000","last_became_peered":"2024-08-31T07:10:46.325785+0000","last_unstale":"2024-08-31T07:11:45.181991+0000","last_undegraded":"2024-08-31T07:11:45.181991+0000","last_fullsized":"2024-08-31T07:11:45.181991+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:13.495835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":170,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:16.094074+0000","last_change":"2024-08-31T07:09:45.641088+0000","last_active":"2024-08-31T07:12:16.094074+0000","last_peered":"2024-08-31T07:12:16.094074+0000","last_clean":"2024-08-31T07:12:16.094074+0000","last_became_active":"2024-08-31T07:09:45.640673+0000","last_became_peered":"2024-08-31T07:09:45.640673+0000","last_unstale":"2024-08-31T07:12:16.094074+0000","last_undegraded":"2024-08-31T07:12:16.094074+0000","last_fullsized":"2024-08-31T07:12:16.094074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:30:17.593494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192254+0000","last_change":"2024-08-31T07:09:47.666703+0000","last_active":"2024-08-31T07:11:45.192254+0000","last_peered":"2024-08-31T07:11:45.192254+0000","last_clean":"2024-08-31T07:11:45.192254+0000","last_became_active":"2024-08-31T07:09:47.666348+0000","last_became_peered":"2024-08-31T07:09:47.666348+0000","last_unstale":"2024-08-31T07:11:45.192254+0000","last_undegraded":"2024-08-31T07:11:45.192254+0000","last_fullsized":"2024-08-31T07:11:45.192254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:39.688810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019815+0000","last_change":"2024-08-31T07:09:45.642431+0000","last_active":"2024-08-31T07:10:46.019815+0000","last_peered":"2024-08-31T07:10:46.019815+0000","last_clean":"2024-08-31T07:10:46.019815+0000","last_became_active":"2024-08-31T07:09:45.642048+0000","last_became_peered":"2024-08-31T07:09:45.642048+0000","last_unstale":"2024-08-31T07:10:46.019815+0000","last_undegraded":"2024-08-31T07:10:46.019815+0000","last_fullsized":"2024-08-31T07:10:46.019815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:47:42.574442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017286+0000","last_change":"2024-08-31T07:09:45.632162+0000","last_active":"2024-08-31T07:10:46.017286+0000","last_peered":"2024-08-31T07:10:46.017286+0000","last_clean":"2024-08-31T07:10:46.017286+0000","last_became_active":"2024-08-31T07:09:45.631749+0000","last_became_peered":"2024-08-31T07:09:45.631749+0000","last_unstale":"2024-08-31T07:10:46.017286+0000","last_undegraded":"2024-08-31T07:10:46.017286+0000","last_fullsized":"2024-08-31T07:10:46.017286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:16:29.967823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185787+0000","last_change":"2024-08-31T07:09:47.666288+0000","last_active":"2024-08-31T07:11:45.185787+0000","last_peered":"2024-08-31T07:11:45.185787+0000","last_clean":"2024-08-31T07:11:45.185787+0000","last_became_active":"2024-08-31T07:09:47.665723+0000","last_became_peered":"2024-08-31T07:09:47.665723+0000","last_unstale":"2024-08-31T07:11:45.185787+0000","last_undegraded":"2024-08-31T07:11:45.185787+0000","last_fullsized":"2024-08-31T07:11:45.185787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:38:28.122720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023120+0000","last_change":"2024-08-31T07:09:45.636508+0000","last_active":"2024-08-31T07:10:46.023120+0000","last_peered":"2024-08-31T07:10:46.023120+0000","last_clean":"2024-08-31T07:10:46.023120+0000","last_became_active":"2024-08-31T07:09:45.636073+0000","last_became_peered":"2024-08-31T07:09:45.636073+0000","last_unstale":"2024-08-31T07:10:46.023120+0000","last_undegraded":"2024-08-31T07:10:46.023120+0000","last_fullsized":"2024-08-31T07:10:46.023120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:55:45.063517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187943+0000","last_change":"2024-08-31T07:09:47.661306+0000","last_active":"2024-08-31T07:11:45.187943+0000","last_peered":"2024-08-31T07:11:45.187943+0000","last_clean":"2024-08-31T07:11:45.187943+0000","last_became_active":"2024-08-31T07:09:47.660957+0000","last_became_peered":"2024-08-31T07:09:47.660957+0000","last_unstale":"2024-08-31T07:11:45.187943+0000","last_undegraded":"2024-08-31T07:11:45.187943+0000","last_fullsized":"2024-08-31T07:11:45.187943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:18:23.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'6","reported_seq":288,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:11.085576+0000","last_change":"2024-08-31T07:09:45.640947+0000","last_active":"2024-08-31T07:12:11.085576+0000","last_peered":"2024-08-31T07:12:11.085576+0000","last_clean":"2024-08-31T07:12:11.085576+0000","last_became_active":"2024-08-31T07:09:45.640532+0000","last_became_peered":"2024-08-31T07:09:45.640532+0000","last_unstale":"2024-08-31T07:12:11.085576+0000","last_undegraded":"2024-08-31T07:12:11.085576+0000","last_fullsized":"2024-08-31T07:12:11.085576+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:28.177795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.191708+0000","last_change":"2024-08-31T07:09:47.661060+0000","last_active":"2024-08-31T07:11:45.191708+0000","last_peered":"2024-08-31T07:11:45.191708+0000","last_clean":"2024-08-31T07:11:45.191708+0000","last_became_active":"2024-08-31T07:09:47.660692+0000","last_became_peered":"2024-08-31T07:09:47.660692+0000","last_unstale":"2024-08-31T07:11:45.191708+0000","last_undegraded":"2024-08-31T07:11:45.191708+0000","last_fullsized":"2024-08-31T07:11:45.191708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:48:38.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257194+0000","last_change":"2024-08-31T07:09:45.641155+0000","last_active":"2024-08-31T07:10:46.257194+0000","last_peered":"2024-08-31T07:10:46.257194+0000","last_clean":"2024-08-31T07:10:46.257194+0000","last_became_active":"2024-08-31T07:09:45.640680+0000","last_became_peered":"2024-08-31T07:09:45.640680+0000","last_unstale":"2024-08-31T07:10:46.257194+0000","last_undegraded":"2024-08-31T07:10:46.257194+0000","last_fullsized":"2024-08-31T07:10:46.257194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:29:03.287570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199892+0000","last_change":"2024-08-31T07:09:47.675582+0000","last_active":"2024-08-31T07:11:45.199892+0000","last_peered":"2024-08-31T07:11:45.199892+0000","last_clean":"2024-08-31T07:11:45.199892+0000","last_became_active":"2024-08-31T07:09:47.675266+0000","last_became_peered":"2024-08-31T07:09:47.675266+0000","last_unstale":"2024-08-31T07:11:45.199892+0000","last_undegraded":"2024-08-31T07:11:45.199892+0000","last_fullsized":"2024-08-31T07:11:45.199892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:36:33.122531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021194+0000","last_change":"2024-08-31T07:09:45.651006+0000","last_active":"2024-08-31T07:10:46.021194+0000","last_peered":"2024-08-31T07:10:46.021194+0000","last_clean":"2024-08-31T07:10:46.021194+0000","last_became_active":"2024-08-31T07:09:45.650588+0000","last_became_peered":"2024-08-31T07:09:45.650588+0000","last_unstale":"2024-08-31T07:10:46.021194+0000","last_undegraded":"2024-08-31T07:10:46.021194+0000","last_fullsized":"2024-08-31T07:10:46.021194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:34:16.015892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.190030+0000","last_change":"2024-08-31T07:09:47.662289+0000","last_active":"2024-08-31T07:11:45.190030+0000","last_peered":"2024-08-31T07:11:45.190030+0000","last_clean":"2024-08-31T07:11:45.190030+0000","last_became_active":"2024-08-31T07:09:47.661932+0000","last_became_peered":"2024-08-31T07:09:47.661932+0000","last_unstale":"2024-08-31T07:11:45.190030+0000","last_undegraded":"2024-08-31T07:11:45.190030+0000","last_fullsized":"2024-08-31T07:11:45.190030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:50.440504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":328,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:11.537240+0000","last_change":"2024-08-31T07:09:45.642196+0000","last_active":"2024-08-31T07:12:11.537240+0000","last_peered":"2024-08-31T07:12:11.537240+0000","last_clean":"2024-08-31T07:12:11.537240+0000","last_became_active":"2024-08-31T07:09:45.641797+0000","last_became_peered":"2024-08-31T07:09:45.641797+0000","last_unstale":"2024-08-31T07:12:11.537240+0000","last_undegraded":"2024-08-31T07:12:11.537240+0000","last_fullsized":"2024-08-31T07:12:11.537240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:51:05.953335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61177,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":266,"num_read_kb":192,"num_write":38,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196022+0000","last_change":"2024-08-31T07:09:47.660184+0000","last_active":"2024-08-31T07:11:45.196022+0000","last_peered":"2024-08-31T07:11:45.196022+0000","last_clean":"2024-08-31T07:11:45.196022+0000","last_became_active":"2024-08-31T07:09:47.659571+0000","last_became_peered":"2024-08-31T07:09:47.659571+0000","last_unstale":"2024-08-31T07:11:45.196022+0000","last_undegraded":"2024-08-31T07:11:45.196022+0000","last_fullsized":"2024-08-31T07:11:45.196022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:41:13.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.035782+0000","last_change":"2024-08-31T07:10:46.035336+0000","last_active":"2024-08-31T07:10:46.035782+0000","last_peered":"2024-08-31T07:10:46.035782+0000","last_clean":"2024-08-31T07:10:46.035782+0000","last_became_active":"2024-08-31T07:10:46.021299+0000","last_became_peered":"2024-08-31T07:10:46.021299+0000","last_unstale":"2024-08-31T07:10:46.035782+0000","last_undegraded":"2024-08-31T07:10:46.035782+0000","last_fullsized":"2024-08-31T07:10:46.035782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:06:15.931855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194680+0000","last_change":"2024-08-31T07:10:46.326354+0000","last_active":"2024-08-31T07:11:45.194680+0000","last_peered":"2024-08-31T07:11:45.194680+0000","last_clean":"2024-08-31T07:11:45.194680+0000","last_became_active":"2024-08-31T07:10:46.325869+0000","last_became_peered":"2024-08-31T07:10:46.325869+0000","last_unstale":"2024-08-31T07:11:45.194680+0000","last_undegraded":"2024-08-31T07:11:45.194680+0000","last_fullsized":"2024-08-31T07:11:45.194680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:21.832088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022972+0000","last_change":"2024-08-31T07:09:45.642187+0000","last_active":"2024-08-31T07:10:46.022972+0000","last_peered":"2024-08-31T07:10:46.022972+0000","last_clean":"2024-08-31T07:10:46.022972+0000","last_became_active":"2024-08-31T07:09:45.641721+0000","last_became_peered":"2024-08-31T07:09:45.641721+0000","last_unstale":"2024-08-31T07:10:46.022972+0000","last_undegraded":"2024-08-31T07:10:46.022972+0000","last_fullsized":"2024-08-31T07:10:46.022972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:00:45.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196019+0000","last_change":"2024-08-31T07:09:47.665956+0000","last_active":"2024-08-31T07:11:45.196019+0000","last_peered":"2024-08-31T07:11:45.196019+0000","last_clean":"2024-08-31T07:11:45.196019+0000","last_became_active":"2024-08-31T07:09:47.664367+0000","last_became_peered":"2024-08-31T07:09:47.664367+0000","last_unstale":"2024-08-31T07:11:45.196019+0000","last_undegraded":"2024-08-31T07:11:45.196019+0000","last_fullsized":"2024-08-31T07:11:45.196019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:34.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'7","reported_seq":170,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:15.091359+0000","last_change":"2024-08-31T07:09:45.636440+0000","last_active":"2024-08-31T07:12:15.091359+0000","last_peered":"2024-08-31T07:12:15.091359+0000","last_clean":"2024-08-31T07:12:15.091359+0000","last_became_active":"2024-08-31T07:09:45.636058+0000","last_became_peered":"2024-08-31T07:09:45.636058+0000","last_unstale":"2024-08-31T07:12:15.091359+0000","last_undegraded":"2024-08-31T07:12:15.091359+0000","last_fullsized":"2024-08-31T07:12:15.091359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:38.929227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194840+0000","last_change":"2024-08-31T07:09:47.660564+0000","last_active":"2024-08-31T07:11:45.194840+0000","last_peered":"2024-08-31T07:11:45.194840+0000","last_clean":"2024-08-31T07:11:45.194840+0000","last_became_active":"2024-08-31T07:09:47.660157+0000","last_became_peered":"2024-08-31T07:09:47.660157+0000","last_unstale":"2024-08-31T07:11:45.194840+0000","last_undegraded":"2024-08-31T07:11:45.194840+0000","last_fullsized":"2024-08-31T07:11:45.194840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:49:01.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020515+0000","last_change":"2024-08-31T07:09:45.649140+0000","last_active":"2024-08-31T07:10:46.020515+0000","last_peered":"2024-08-31T07:10:46.020515+0000","last_clean":"2024-08-31T07:10:46.020515+0000","last_became_active":"2024-08-31T07:09:45.648796+0000","last_became_peered":"2024-08-31T07:09:45.648796+0000","last_unstale":"2024-08-31T07:10:46.020515+0000","last_undegraded":"2024-08-31T07:10:46.020515+0000","last_fullsized":"2024-08-31T07:10:46.020515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:33:37.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192390+0000","last_change":"2024-08-31T07:09:47.663833+0000","last_active":"2024-08-31T07:11:45.192390+0000","last_peered":"2024-08-31T07:11:45.192390+0000","last_clean":"2024-08-31T07:11:45.192390+0000","last_became_active":"2024-08-31T07:09:47.663451+0000","last_became_peered":"2024-08-31T07:09:47.663451+0000","last_unstale":"2024-08-31T07:11:45.192390+0000","last_undegraded":"2024-08-31T07:11:45.192390+0000","last_fullsized":"2024-08-31T07:11:45.192390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:27:22.245254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021407+0000","last_change":"2024-08-31T07:09:45.636593+0000","last_active":"2024-08-31T07:10:46.021407+0000","last_peered":"2024-08-31T07:10:46.021407+0000","last_clean":"2024-08-31T07:10:46.021407+0000","last_became_active":"2024-08-31T07:09:45.636166+0000","last_became_peered":"2024-08-31T07:09:45.636166+0000","last_unstale":"2024-08-31T07:10:46.021407+0000","last_undegraded":"2024-08-31T07:10:46.021407+0000","last_fullsized":"2024-08-31T07:10:46.021407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:28.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183385+0000","last_change":"2024-08-31T07:09:47.660825+0000","last_active":"2024-08-31T07:11:45.183385+0000","last_peered":"2024-08-31T07:11:45.183385+0000","last_clean":"2024-08-31T07:11:45.183385+0000","last_became_active":"2024-08-31T07:09:47.660474+0000","last_became_peered":"2024-08-31T07:09:47.660474+0000","last_unstale":"2024-08-31T07:11:45.183385+0000","last_undegraded":"2024-08-31T07:11:45.183385+0000","last_fullsized":"2024-08-31T07:11:45.183385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:42:20.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017462+0000","last_change":"2024-08-31T07:09:45.632727+0000","last_active":"2024-08-31T07:10:46.017462+0000","last_peered":"2024-08-31T07:10:46.017462+0000","last_clean":"2024-08-31T07:10:46.017462+0000","last_became_active":"2024-08-31T07:09:45.632251+0000","last_became_peered":"2024-08-31T07:09:45.632251+0000","last_unstale":"2024-08-31T07:10:46.017462+0000","last_undegraded":"2024-08-31T07:10:46.017462+0000","last_fullsized":"2024-08-31T07:10:46.017462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:09:30.197409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196430+0000","last_change":"2024-08-31T07:09:47.665501+0000","last_active":"2024-08-31T07:11:45.196430+0000","last_peered":"2024-08-31T07:11:45.196430+0000","last_clean":"2024-08-31T07:11:45.196430+0000","last_became_active":"2024-08-31T07:09:47.665162+0000","last_became_peered":"2024-08-31T07:09:47.665162+0000","last_unstale":"2024-08-31T07:11:45.196430+0000","last_undegraded":"2024-08-31T07:11:45.196430+0000","last_fullsized":"2024-08-31T07:11:45.196430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:34:29.605836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021320+0000","last_change":"2024-08-31T07:09:45.636580+0000","last_active":"2024-08-31T07:10:46.021320+0000","last_peered":"2024-08-31T07:10:46.021320+0000","last_clean":"2024-08-31T07:10:46.021320+0000","last_became_active":"2024-08-31T07:09:45.636163+0000","last_became_peered":"2024-08-31T07:09:45.636163+0000","last_unstale":"2024-08-31T07:10:46.021320+0000","last_undegraded":"2024-08-31T07:10:46.021320+0000","last_fullsized":"2024-08-31T07:10:46.021320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:18:07.567858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195120+0000","last_change":"2024-08-31T07:09:47.671865+0000","last_active":"2024-08-31T07:11:45.195120+0000","last_peered":"2024-08-31T07:11:45.195120+0000","last_clean":"2024-08-31T07:11:45.195120+0000","last_became_active":"2024-08-31T07:09:47.671454+0000","last_became_peered":"2024-08-31T07:09:47.671454+0000","last_unstale":"2024-08-31T07:11:45.195120+0000","last_undegraded":"2024-08-31T07:11:45.195120+0000","last_fullsized":"2024-08-31T07:11:45.195120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:47.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'7","reported_seq":168,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:13.088891+0000","last_change":"2024-08-31T07:09:45.653217+0000","last_active":"2024-08-31T07:12:13.088891+0000","last_peered":"2024-08-31T07:12:13.088891+0000","last_clean":"2024-08-31T07:12:13.088891+0000","last_became_active":"2024-08-31T07:09:45.652819+0000","last_became_peered":"2024-08-31T07:09:45.652819+0000","last_unstale":"2024-08-31T07:12:13.088891+0000","last_undegraded":"2024-08-31T07:12:13.088891+0000","last_fullsized":"2024-08-31T07:12:13.088891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:25:11.454068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196166+0000","last_change":"2024-08-31T07:09:47.663652+0000","last_active":"2024-08-31T07:11:45.196166+0000","last_peered":"2024-08-31T07:11:45.196166+0000","last_clean":"2024-08-31T07:11:45.196166+0000","last_became_active":"2024-08-31T07:09:47.663274+0000","last_became_peered":"2024-08-31T07:09:47.663274+0000","last_unstale":"2024-08-31T07:11:45.196166+0000","last_undegraded":"2024-08-31T07:11:45.196166+0000","last_fullsized":"2024-08-31T07:11:45.196166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:31:59.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257370+0000","last_change":"2024-08-31T07:08:27.623298+0000","last_active":"2024-08-31T07:10:46.257370+0000","last_peered":"2024-08-31T07:10:46.257370+0000","last_clean":"2024-08-31T07:10:46.257370+0000","last_became_active":"2024-08-31T07:08:27.619072+0000","last_became_peered":"2024-08-31T07:08:27.619072+0000","last_unstale":"2024-08-31T07:10:46.257370+0000","last_undegraded":"2024-08-31T07:10:46.257370+0000","last_fullsized":"2024-08-31T07:10:46.257370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_clean_scrub_stamp":"2024-08-31T07:04:45.509398+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:07.967421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154319+0000","last_change":"2024-08-31T07:09:45.638104+0000","last_active":"2024-08-31T07:10:46.154319+0000","last_peered":"2024-08-31T07:10:46.154319+0000","last_clean":"2024-08-31T07:10:46.154319+0000","last_became_active":"2024-08-31T07:09:45.637693+0000","last_became_peered":"2024-08-31T07:09:45.637693+0000","last_unstale":"2024-08-31T07:10:46.154319+0000","last_undegraded":"2024-08-31T07:10:46.154319+0000","last_fullsized":"2024-08-31T07:10:46.154319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:50:15.135061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.189353+0000","last_change":"2024-08-31T07:09:47.664958+0000","last_active":"2024-08-31T07:11:45.189353+0000","last_peered":"2024-08-31T07:11:45.189353+0000","last_clean":"2024-08-31T07:11:45.189353+0000","last_became_active":"2024-08-31T07:09:47.661166+0000","last_became_peered":"2024-08-31T07:09:47.661166+0000","last_unstale":"2024-08-31T07:11:45.189353+0000","last_undegraded":"2024-08-31T07:11:45.189353+0000","last_fullsized":"2024-08-31T07:11:45.189353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:45:58.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257184+0000","last_change":"2024-08-31T07:09:45.640881+0000","last_active":"2024-08-31T07:10:46.257184+0000","last_peered":"2024-08-31T07:10:46.257184+0000","last_clean":"2024-08-31T07:10:46.257184+0000","last_became_active":"2024-08-31T07:09:45.640487+0000","last_became_peered":"2024-08-31T07:09:45.640487+0000","last_unstale":"2024-08-31T07:10:46.257184+0000","last_undegraded":"2024-08-31T07:10:46.257184+0000","last_fullsized":"2024-08-31T07:10:46.257184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:06.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.171981+0000","last_change":"2024-08-31T07:09:47.655008+0000","last_active":"2024-08-31T07:11:45.171981+0000","last_peered":"2024-08-31T07:11:45.171981+0000","last_clean":"2024-08-31T07:11:45.171981+0000","last_became_active":"2024-08-31T07:09:47.654645+0000","last_became_peered":"2024-08-31T07:09:47.654645+0000","last_unstale":"2024-08-31T07:11:45.171981+0000","last_undegraded":"2024-08-31T07:11:45.171981+0000","last_fullsized":"2024-08-31T07:11:45.171981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:23:55.916053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'6","reported_seq":163,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:12.086211+0000","last_change":"2024-08-31T07:09:45.649324+0000","last_active":"2024-08-31T07:12:12.086211+0000","last_peered":"2024-08-31T07:12:12.086211+0000","last_clean":"2024-08-31T07:12:12.086211+0000","last_became_active":"2024-08-31T07:09:45.648915+0000","last_became_peered":"2024-08-31T07:09:45.648915+0000","last_unstale":"2024-08-31T07:12:12.086211+0000","last_undegraded":"2024-08-31T07:12:12.086211+0000","last_fullsized":"2024-08-31T07:12:12.086211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:31:57.853654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187160+0000","last_change":"2024-08-31T07:09:47.661328+0000","last_active":"2024-08-31T07:11:45.187160+0000","last_peered":"2024-08-31T07:11:45.187160+0000","last_clean":"2024-08-31T07:11:45.187160+0000","last_became_active":"2024-08-31T07:09:47.660927+0000","last_became_peered":"2024-08-31T07:09:47.660927+0000","last_unstale":"2024-08-31T07:11:45.187160+0000","last_undegraded":"2024-08-31T07:11:45.187160+0000","last_fullsized":"2024-08-31T07:11:45.187160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:40:46.876894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'7","reported_seq":166,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:12.087548+0000","last_change":"2024-08-31T07:09:45.637747+0000","last_active":"2024-08-31T07:12:12.087548+0000","last_peered":"2024-08-31T07:12:12.087548+0000","last_clean":"2024-08-31T07:12:12.087548+0000","last_became_active":"2024-08-31T07:09:45.637368+0000","last_became_peered":"2024-08-31T07:09:45.637368+0000","last_unstale":"2024-08-31T07:12:12.087548+0000","last_undegraded":"2024-08-31T07:12:12.087548+0000","last_fullsized":"2024-08-31T07:12:12.087548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:01.174109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193217+0000","last_change":"2024-08-31T07:09:47.680802+0000","last_active":"2024-08-31T07:11:45.193217+0000","last_peered":"2024-08-31T07:11:45.193217+0000","last_clean":"2024-08-31T07:11:45.193217+0000","last_became_active":"2024-08-31T07:09:47.680363+0000","last_became_peered":"2024-08-31T07:09:47.680363+0000","last_unstale":"2024-08-31T07:11:45.193217+0000","last_undegraded":"2024-08-31T07:11:45.193217+0000","last_fullsized":"2024-08-31T07:11:45.193217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:46:48.744071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324353+0000","last_change":"2024-08-31T07:09:45.651222+0000","last_active":"2024-08-31T07:10:46.324353+0000","last_peered":"2024-08-31T07:10:46.324353+0000","last_clean":"2024-08-31T07:10:46.324353+0000","last_became_active":"2024-08-31T07:09:45.650760+0000","last_became_peered":"2024-08-31T07:09:45.650760+0000","last_unstale":"2024-08-31T07:10:46.324353+0000","last_undegraded":"2024-08-31T07:10:46.324353+0000","last_fullsized":"2024-08-31T07:10:46.324353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T19:03:33.942165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183968+0000","last_change":"2024-08-31T07:09:47.665881+0000","last_active":"2024-08-31T07:11:45.183968+0000","last_peered":"2024-08-31T07:11:45.183968+0000","last_clean":"2024-08-31T07:11:45.183968+0000","last_became_active":"2024-08-31T07:09:47.665485+0000","last_became_peered":"2024-08-31T07:09:47.665485+0000","last_unstale":"2024-08-31T07:11:45.183968+0000","last_undegraded":"2024-08-31T07:11:45.183968+0000","last_fullsized":"2024-08-31T07:11:45.183968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:30.820714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020851+0000","last_change":"2024-08-31T07:09:45.651893+0000","last_active":"2024-08-31T07:10:46.020851+0000","last_peered":"2024-08-31T07:10:46.020851+0000","last_clean":"2024-08-31T07:10:46.020851+0000","last_became_active":"2024-08-31T07:09:45.651547+0000","last_became_peered":"2024-08-31T07:09:45.651547+0000","last_unstale":"2024-08-31T07:10:46.020851+0000","last_undegraded":"2024-08-31T07:10:46.020851+0000","last_fullsized":"2024-08-31T07:10:46.020851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:11:30.708018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.175560+0000","last_change":"2024-08-31T07:09:47.655551+0000","last_active":"2024-08-31T07:11:45.175560+0000","last_peered":"2024-08-31T07:11:45.175560+0000","last_clean":"2024-08-31T07:11:45.175560+0000","last_became_active":"2024-08-31T07:09:47.655188+0000","last_became_peered":"2024-08-31T07:09:47.655188+0000","last_unstale":"2024-08-31T07:11:45.175560+0000","last_undegraded":"2024-08-31T07:11:45.175560+0000","last_fullsized":"2024-08-31T07:11:45.175560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:43:51.419599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'17","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:55.669546+0000","last_change":"2024-08-31T07:09:45.636212+0000","last_active":"2024-08-31T07:11:55.669546+0000","last_peered":"2024-08-31T07:11:55.669546+0000","last_clean":"2024-08-31T07:11:55.669546+0000","last_became_active":"2024-08-31T07:09:45.635678+0000","last_became_peered":"2024-08-31T07:09:45.635678+0000","last_unstale":"2024-08-31T07:11:55.669546+0000","last_undegraded":"2024-08-31T07:11:55.669546+0000","last_fullsized":"2024-08-31T07:11:55.669546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:13:17.803459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185987+0000","last_change":"2024-08-31T07:09:47.666476+0000","last_active":"2024-08-31T07:11:45.185987+0000","last_peered":"2024-08-31T07:11:45.185987+0000","last_clean":"2024-08-31T07:11:45.185987+0000","last_became_active":"2024-08-31T07:09:47.666111+0000","last_became_peered":"2024-08-31T07:09:47.666111+0000","last_unstale":"2024-08-31T07:11:45.185987+0000","last_undegraded":"2024-08-31T07:11:45.185987+0000","last_fullsized":"2024-08-31T07:11:45.185987+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:08:53.459549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154441+0000","last_change":"2024-08-31T07:09:45.631571+0000","last_active":"2024-08-31T07:10:46.154441+0000","last_peered":"2024-08-31T07:10:46.154441+0000","last_clean":"2024-08-31T07:10:46.154441+0000","last_became_active":"2024-08-31T07:09:45.630981+0000","last_became_peered":"2024-08-31T07:09:45.630981+0000","last_unstale":"2024-08-31T07:10:46.154441+0000","last_undegraded":"2024-08-31T07:10:46.154441+0000","last_fullsized":"2024-08-31T07:10:46.154441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:36:53.482955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193936+0000","last_change":"2024-08-31T07:09:47.675054+0000","last_active":"2024-08-31T07:11:45.193936+0000","last_peered":"2024-08-31T07:11:45.193936+0000","last_clean":"2024-08-31T07:11:45.193936+0000","last_became_active":"2024-08-31T07:09:47.672652+0000","last_became_peered":"2024-08-31T07:09:47.672652+0000","last_unstale":"2024-08-31T07:11:45.193936+0000","last_undegraded":"2024-08-31T07:11:45.193936+0000","last_fullsized":"2024-08-31T07:11:45.193936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:32:05.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016284+0000","last_change":"2024-08-31T07:09:45.648746+0000","last_active":"2024-08-31T07:10:46.016284+0000","last_peered":"2024-08-31T07:10:46.016284+0000","last_clean":"2024-08-31T07:10:46.016284+0000","last_became_active":"2024-08-31T07:09:45.648424+0000","last_became_peered":"2024-08-31T07:09:45.648424+0000","last_unstale":"2024-08-31T07:10:46.016284+0000","last_undegraded":"2024-08-31T07:10:46.016284+0000","last_fullsized":"2024-08-31T07:10:46.016284+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:40:32.132963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.165179+0000","last_change":"2024-08-31T07:09:47.663213+0000","last_active":"2024-08-31T07:11:45.165179+0000","last_peered":"2024-08-31T07:11:45.165179+0000","last_clean":"2024-08-31T07:11:45.165179+0000","last_became_active":"2024-08-31T07:09:47.662457+0000","last_became_peered":"2024-08-31T07:09:47.662457+0000","last_unstale":"2024-08-31T07:11:45.165179+0000","last_undegraded":"2024-08-31T07:11:45.165179+0000","last_fullsized":"2024-08-31T07:11:45.165179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:54:27.069200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015653+0000","last_change":"2024-08-31T07:09:45.640149+0000","last_active":"2024-08-31T07:10:46.015653+0000","last_peered":"2024-08-31T07:10:46.015653+0000","last_clean":"2024-08-31T07:10:46.015653+0000","last_became_active":"2024-08-31T07:09:45.639581+0000","last_became_peered":"2024-08-31T07:09:45.639581+0000","last_unstale":"2024-08-31T07:10:46.015653+0000","last_undegraded":"2024-08-31T07:10:46.015653+0000","last_fullsized":"2024-08-31T07:10:46.015653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:27:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198479+0000","last_change":"2024-08-31T07:09:47.672613+0000","last_active":"2024-08-31T07:11:45.198479+0000","last_peered":"2024-08-31T07:11:45.198479+0000","last_clean":"2024-08-31T07:11:45.198479+0000","last_became_active":"2024-08-31T07:09:47.672173+0000","last_became_peered":"2024-08-31T07:09:47.672173+0000","last_unstale":"2024-08-31T07:11:45.198479+0000","last_undegraded":"2024-08-31T07:11:45.198479+0000","last_fullsized":"2024-08-31T07:11:45.198479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:49.706113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022590+0000","last_change":"2024-08-31T07:09:45.637567+0000","last_active":"2024-08-31T07:10:46.022590+0000","last_peered":"2024-08-31T07:10:46.022590+0000","last_clean":"2024-08-31T07:10:46.022590+0000","last_became_active":"2024-08-31T07:09:45.637180+0000","last_became_peered":"2024-08-31T07:09:45.637180+0000","last_unstale":"2024-08-31T07:10:46.022590+0000","last_undegraded":"2024-08-31T07:10:46.022590+0000","last_fullsized":"2024-08-31T07:10:46.022590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:48:07.190829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194363+0000","last_change":"2024-08-31T07:09:47.674980+0000","last_active":"2024-08-31T07:11:45.194363+0000","last_peered":"2024-08-31T07:11:45.194363+0000","last_clean":"2024-08-31T07:11:45.194363+0000","last_became_active":"2024-08-31T07:09:47.674528+0000","last_became_peered":"2024-08-31T07:09:47.674528+0000","last_unstale":"2024-08-31T07:11:45.194363+0000","last_undegraded":"2024-08-31T07:11:45.194363+0000","last_fullsized":"2024-08-31T07:11:45.194363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:15:09.464221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021096+0000","last_change":"2024-08-31T07:09:45.641167+0000","last_active":"2024-08-31T07:10:46.021096+0000","last_peered":"2024-08-31T07:10:46.021096+0000","last_clean":"2024-08-31T07:10:46.021096+0000","last_became_active":"2024-08-31T07:09:45.640761+0000","last_became_peered":"2024-08-31T07:09:45.640761+0000","last_unstale":"2024-08-31T07:10:46.021096+0000","last_undegraded":"2024-08-31T07:10:46.021096+0000","last_fullsized":"2024-08-31T07:10:46.021096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:28:59.963145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195596+0000","last_change":"2024-08-31T07:09:47.668808+0000","last_active":"2024-08-31T07:11:45.195596+0000","last_peered":"2024-08-31T07:11:45.195596+0000","last_clean":"2024-08-31T07:11:45.195596+0000","last_became_active":"2024-08-31T07:09:47.668291+0000","last_became_peered":"2024-08-31T07:09:47.668291+0000","last_unstale":"2024-08-31T07:11:45.195596+0000","last_undegraded":"2024-08-31T07:11:45.195596+0000","last_fullsized":"2024-08-31T07:11:45.195596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:23.433511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192982+0000","last_change":"2024-08-31T07:09:47.668039+0000","last_active":"2024-08-31T07:11:45.192982+0000","last_peered":"2024-08-31T07:11:45.192982+0000","last_clean":"2024-08-31T07:11:45.192982+0000","last_became_active":"2024-08-31T07:09:47.667567+0000","last_became_peered":"2024-08-31T07:09:47.667567+0000","last_unstale":"2024-08-31T07:11:45.192982+0000","last_undegraded":"2024-08-31T07:11:45.192982+0000","last_fullsized":"2024-08-31T07:11:45.192982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:33:50.192864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.041708+0000","last_change":"2024-08-31T07:10:46.041197+0000","last_active":"2024-08-31T07:10:46.041708+0000","last_peered":"2024-08-31T07:10:46.041708+0000","last_clean":"2024-08-31T07:10:46.041708+0000","last_became_active":"2024-08-31T07:10:46.023966+0000","last_became_peered":"2024-08-31T07:10:46.023966+0000","last_unstale":"2024-08-31T07:10:46.041708+0000","last_undegraded":"2024-08-31T07:10:46.041708+0000","last_fullsized":"2024-08-31T07:10:46.041708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:34.302665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192551+0000","last_change":"2024-08-31T07:09:47.660615+0000","last_active":"2024-08-31T07:11:45.192551+0000","last_peered":"2024-08-31T07:11:45.192551+0000","last_clean":"2024-08-31T07:11:45.192551+0000","last_became_active":"2024-08-31T07:09:47.660269+0000","last_became_peered":"2024-08-31T07:09:47.660269+0000","last_unstale":"2024-08-31T07:11:45.192551+0000","last_undegraded":"2024-08-31T07:11:45.192551+0000","last_fullsized":"2024-08-31T07:11:45.192551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:42:12.618039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020701+0000","last_change":"2024-08-31T07:09:45.641699+0000","last_active":"2024-08-31T07:10:46.020701+0000","last_peered":"2024-08-31T07:10:46.020701+0000","last_clean":"2024-08-31T07:10:46.020701+0000","last_became_active":"2024-08-31T07:09:45.641122+0000","last_became_peered":"2024-08-31T07:09:45.641122+0000","last_unstale":"2024-08-31T07:10:46.020701+0000","last_undegraded":"2024-08-31T07:10:46.020701+0000","last_fullsized":"2024-08-31T07:10:46.020701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:26:07.130266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200241+0000","last_change":"2024-08-31T07:09:47.675459+0000","last_active":"2024-08-31T07:11:45.200241+0000","last_peered":"2024-08-31T07:11:45.200241+0000","last_clean":"2024-08-31T07:11:45.200241+0000","last_became_active":"2024-08-31T07:09:47.672246+0000","last_became_peered":"2024-08-31T07:09:47.672246+0000","last_unstale":"2024-08-31T07:11:45.200241+0000","last_undegraded":"2024-08-31T07:11:45.200241+0000","last_fullsized":"2024-08-31T07:11:45.200241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:49:23.547352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016428+0000","last_change":"2024-08-31T07:09:45.649522+0000","last_active":"2024-08-31T07:10:46.016428+0000","last_peered":"2024-08-31T07:10:46.016428+0000","last_clean":"2024-08-31T07:10:46.016428+0000","last_became_active":"2024-08-31T07:09:45.649109+0000","last_became_peered":"2024-08-31T07:09:45.649109+0000","last_unstale":"2024-08-31T07:10:46.016428+0000","last_undegraded":"2024-08-31T07:10:46.016428+0000","last_fullsized":"2024-08-31T07:10:46.016428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:52:52.432605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.179375+0000","last_change":"2024-08-31T07:09:47.670583+0000","last_active":"2024-08-31T07:11:45.179375+0000","last_peered":"2024-08-31T07:11:45.179375+0000","last_clean":"2024-08-31T07:11:45.179375+0000","last_became_active":"2024-08-31T07:09:47.670162+0000","last_became_peered":"2024-08-31T07:09:47.670162+0000","last_unstale":"2024-08-31T07:11:45.179375+0000","last_undegraded":"2024-08-31T07:11:45.179375+0000","last_fullsized":"2024-08-31T07:11:45.179375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:35:35.135678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'14","reported_seq":173,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:14.089952+0000","last_change":"2024-08-31T07:09:45.639636+0000","last_active":"2024-08-31T07:12:14.089952+0000","last_peered":"2024-08-31T07:12:14.089952+0000","last_clean":"2024-08-31T07:12:14.089952+0000","last_became_active":"2024-08-31T07:09:45.639029+0000","last_became_peered":"2024-08-31T07:09:45.639029+0000","last_unstale":"2024-08-31T07:12:14.089952+0000","last_undegraded":"2024-08-31T07:12:14.089952+0000","last_fullsized":"2024-08-31T07:12:14.089952+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:54.110149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69727,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1348,"num_read_kb":747,"num_write":264,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":650,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":209181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285320,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678060,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":434494,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874288,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":1086050,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037814,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":433242,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234045,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":433038,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397569,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1027343,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561093,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":433491,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757326,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1025957,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":433352,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117079,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432784,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378717,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":496265,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542243,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":495581,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705767,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":434830,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T07:12:18.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-31T07:12:18.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.335+0000 7f8613e00640 1 -- 172.21.15.32:0/1464079994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f860c10db10 msgr2=0x7f860c110020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.335+0000 7f8613e00640 1 --2- 172.21.15.32:0/1464079994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f860c10db10 0x7f860c110020 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8604009be0 tx=0x7f860402a6c0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.335+0000 7f8613e00640 1 -- 172.21.15.32:0/1464079994 shutdown_connections 2024-08-31T07:12:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.335+0000 7f8613e00640 1 --2- 172.21.15.32:0/1464079994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f860c10db10 0x7f860c110020 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.335+0000 7f8613e00640 1 --2- 172.21.15.32:0/1464079994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f860c104f90 0x7f860c10d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.335+0000 7f8613e00640 1 --2- 172.21.15.32:0/1464079994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f860c104560 0x7f860c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.335+0000 7f8613e00640 1 -- 172.21.15.32:0/1464079994 >> 172.21.15.32:0/1464079994 conn(0x7f860c0fe150 msgr2=0x7f860c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.335+0000 7f8613e00640 1 -- 172.21.15.32:0/1464079994 shutdown_connections 2024-08-31T07:12:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.335+0000 7f8613e00640 1 -- 172.21.15.32:0/1464079994 wait complete. 2024-08-31T07:12:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8613e00640 1 Processor -- start 2024-08-31T07:12:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8613e00640 1 -- start start 2024-08-31T07:12:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8613e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f860c104560 0x7f860c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8613e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f860c104f90 0x7f860c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8613e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f860c10db10 0x7f860c1a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8613e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f860c1137a0 con 0x7f860c104560 2024-08-31T07:12:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8613e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f860c113620 con 0x7f860c10db10 2024-08-31T07:12:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8613e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f860c113920 con 0x7f860c104f90 2024-08-31T07:12:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8611400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f860c104560 0x7f860c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8611400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f860c104560 0x7f860c19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58472/0 (socket says 172.21.15.32:58472) 2024-08-31T07:12:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.337+0000 7f8611400640 1 -- 172.21.15.32:0/2211090174 learned_addr learned my addr 172.21.15.32:0/2211090174 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.338+0000 7f8610a00640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f860c104f90 0x7f860c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.338+0000 7f8611e00640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f860c10db10 0x7f860c1a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.338+0000 7f8611400640 1 -- 172.21.15.32:0/2211090174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f860c104f90 msgr2=0x7f860c19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.338+0000 7f8611400640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f860c104f90 0x7f860c19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.338+0000 7f8611400640 1 -- 172.21.15.32:0/2211090174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f860c10db10 msgr2=0x7f860c1a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.338+0000 7f8611400640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f860c10db10 0x7f860c1a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.338+0000 7f8611400640 1 -- 172.21.15.32:0/2211090174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860c1a2270 con 0x7f860c104560 2024-08-31T07:12:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.338+0000 7f8611400640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f860c104560 0x7f860c19c340 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f860000e9f0 tx=0x7f860000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.338+0000 7f860a000640 1 -- 172.21.15.32:0/2211090174 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860000cd80 con 0x7f860c104560 2024-08-31T07:12:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.339+0000 7f8613e00640 1 -- 172.21.15.32:0/2211090174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860c1a2560 con 0x7f860c104560 2024-08-31T07:12:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.339+0000 7f860a000640 1 -- 172.21.15.32:0/2211090174 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86000108e0 con 0x7f860c104560 2024-08-31T07:12:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.339+0000 7f860a000640 1 -- 172.21.15.32:0/2211090174 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600018980 con 0x7f860c104560 2024-08-31T07:12:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.339+0000 7f8613e00640 1 -- 172.21.15.32:0/2211090174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f860c1a8cf0 con 0x7f860c104560 2024-08-31T07:12:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.340+0000 7f8613e00640 1 -- 172.21.15.32:0/2211090174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f860c104970 con 0x7f860c104560 2024-08-31T07:12:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.342+0000 7f860a000640 1 -- 172.21.15.32:0/2211090174 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f8600010430 con 0x7f860c104560 2024-08-31T07:12:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.344+0000 7f860a000640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85f40b4b80 0x7f85f40b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.344+0000 7f860a000640 1 -- 172.21.15.32:0/2211090174 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f86000e5a70 con 0x7f860c104560 2024-08-31T07:12:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.344+0000 7f8610a00640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85f40b4b80 0x7f85f40b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.345+0000 7f8610a00640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85f40b4b80 0x7f85f40b7090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f860c10e9f0 tx=0x7f85f8009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:18.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.346+0000 7f860a000640 1 -- 172.21.15.32:0/2211090174 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f86000ac450 con 0x7f860c104560 2024-08-31T07:12:18.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.540+0000 7f8613e00640 1 -- 172.21.15.32:0/2211090174 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f860c102fe0 con 0x7f85f40b4b80 2024-08-31T07:12:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.545+0000 7f860a000640 1 -- 172.21.15.32:0/2211090174 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259134 (secure 0 0 0) 0x7f860c102fe0 con 0x7f85f40b4b80 2024-08-31T07:12:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:18.551 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-08-31T07:12:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 -- 172.21.15.32:0/2211090174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85f40b4b80 msgr2=0x7f85f40b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85f40b4b80 0x7f85f40b7090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f860c10e9f0 tx=0x7f85f8009260 comp rx=0 tx=0).stop 2024-08-31T07:12:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 -- 172.21.15.32:0/2211090174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f860c104560 msgr2=0x7f860c19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f860c104560 0x7f860c19c340 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f860000e9f0 tx=0x7f860000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 -- 172.21.15.32:0/2211090174 shutdown_connections 2024-08-31T07:12:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f85f40b4b80 0x7f85f40b7090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f860c10db10 0x7f860c1a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f860c104f90 0x7f860c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 --2- 172.21.15.32:0/2211090174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f860c104560 0x7f860c19c340 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 -- 172.21.15.32:0/2211090174 >> 172.21.15.32:0/2211090174 conn(0x7f860c0fe150 msgr2=0x7f860c1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 -- 172.21.15.32:0/2211090174 shutdown_connections 2024-08-31T07:12:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.549+0000 7f8613e00640 1 -- 172.21.15.32:0/2211090174 wait complete. 2024-08-31T07:12:18.575 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-08-31T07:12:16.797561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660329,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1429,"num_read_kb":816,"num_write":389,"num_write_kb":2303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2952490,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":821,"ondisk_log_size":821,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364208,"kb_used_data":10800,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577616,"statfs":{"total":1151940427776,"available":1151567478784,"internally_reserved":0,"allocated":11059200,"data_stored":7166427,"data_compressed":177834,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":311,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002907"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198399+0000","last_change":"2024-08-31T07:09:47.665725+0000","last_active":"2024-08-31T07:11:45.198399+0000","last_peered":"2024-08-31T07:11:45.198399+0000","last_clean":"2024-08-31T07:11:45.198399+0000","last_became_active":"2024-08-31T07:09:47.665368+0000","last_became_peered":"2024-08-31T07:09:47.665368+0000","last_unstale":"2024-08-31T07:11:45.198399+0000","last_undegraded":"2024-08-31T07:11:45.198399+0000","last_fullsized":"2024-08-31T07:11:45.198399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:27:01.636099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192497+0000","last_change":"2024-08-31T07:09:47.676882+0000","last_active":"2024-08-31T07:11:45.192497+0000","last_peered":"2024-08-31T07:11:45.192497+0000","last_clean":"2024-08-31T07:11:45.192497+0000","last_became_active":"2024-08-31T07:09:47.674825+0000","last_became_peered":"2024-08-31T07:09:47.674825+0000","last_unstale":"2024-08-31T07:11:45.192497+0000","last_undegraded":"2024-08-31T07:11:45.192497+0000","last_fullsized":"2024-08-31T07:11:45.192497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:09.148398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199170+0000","last_change":"2024-08-31T07:09:47.674480+0000","last_active":"2024-08-31T07:11:45.199170+0000","last_peered":"2024-08-31T07:11:45.199170+0000","last_clean":"2024-08-31T07:11:45.199170+0000","last_became_active":"2024-08-31T07:09:47.672316+0000","last_became_peered":"2024-08-31T07:09:47.672316+0000","last_unstale":"2024-08-31T07:11:45.199170+0000","last_undegraded":"2024-08-31T07:11:45.199170+0000","last_fullsized":"2024-08-31T07:11:45.199170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:28:13.231039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.172744+0000","last_change":"2024-08-31T07:09:47.659276+0000","last_active":"2024-08-31T07:11:45.172744+0000","last_peered":"2024-08-31T07:11:45.172744+0000","last_clean":"2024-08-31T07:11:45.172744+0000","last_became_active":"2024-08-31T07:09:47.658924+0000","last_became_peered":"2024-08-31T07:09:47.658924+0000","last_unstale":"2024-08-31T07:11:45.172744+0000","last_undegraded":"2024-08-31T07:11:45.172744+0000","last_fullsized":"2024-08-31T07:11:45.172744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:01:08.134245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.174698+0000","last_change":"2024-08-31T07:09:47.667260+0000","last_active":"2024-08-31T07:11:45.174698+0000","last_peered":"2024-08-31T07:11:45.174698+0000","last_clean":"2024-08-31T07:11:45.174698+0000","last_became_active":"2024-08-31T07:09:47.664572+0000","last_became_peered":"2024-08-31T07:09:47.664572+0000","last_unstale":"2024-08-31T07:11:45.174698+0000","last_undegraded":"2024-08-31T07:11:45.174698+0000","last_fullsized":"2024-08-31T07:11:45.174698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:11.336985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188974+0000","last_change":"2024-08-31T07:09:47.661181+0000","last_active":"2024-08-31T07:11:45.188974+0000","last_peered":"2024-08-31T07:11:45.188974+0000","last_clean":"2024-08-31T07:11:45.188974+0000","last_became_active":"2024-08-31T07:09:47.660761+0000","last_became_peered":"2024-08-31T07:09:47.660761+0000","last_unstale":"2024-08-31T07:11:45.188974+0000","last_undegraded":"2024-08-31T07:11:45.188974+0000","last_fullsized":"2024-08-31T07:11:45.188974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:50:54.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196463+0000","last_change":"2024-08-31T07:09:47.668762+0000","last_active":"2024-08-31T07:11:45.196463+0000","last_peered":"2024-08-31T07:11:45.196463+0000","last_clean":"2024-08-31T07:11:45.196463+0000","last_became_active":"2024-08-31T07:09:47.668406+0000","last_became_peered":"2024-08-31T07:09:47.668406+0000","last_unstale":"2024-08-31T07:11:45.196463+0000","last_undegraded":"2024-08-31T07:11:45.196463+0000","last_fullsized":"2024-08-31T07:11:45.196463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:43:39.868144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.190957+0000","last_change":"2024-08-31T07:09:47.669455+0000","last_active":"2024-08-31T07:11:45.190957+0000","last_peered":"2024-08-31T07:11:45.190957+0000","last_clean":"2024-08-31T07:11:45.190957+0000","last_became_active":"2024-08-31T07:09:47.663669+0000","last_became_peered":"2024-08-31T07:09:47.663669+0000","last_unstale":"2024-08-31T07:11:45.190957+0000","last_undegraded":"2024-08-31T07:11:45.190957+0000","last_fullsized":"2024-08-31T07:11:45.190957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:22:59.685012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194827+0000","last_change":"2024-08-31T07:09:47.664214+0000","last_active":"2024-08-31T07:11:45.194827+0000","last_peered":"2024-08-31T07:11:45.194827+0000","last_clean":"2024-08-31T07:11:45.194827+0000","last_became_active":"2024-08-31T07:09:47.663832+0000","last_became_peered":"2024-08-31T07:09:47.663832+0000","last_unstale":"2024-08-31T07:11:45.194827+0000","last_undegraded":"2024-08-31T07:11:45.194827+0000","last_fullsized":"2024-08-31T07:11:45.194827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:29:14.107212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194866+0000","last_change":"2024-08-31T07:10:46.326410+0000","last_active":"2024-08-31T07:11:45.194866+0000","last_peered":"2024-08-31T07:11:45.194866+0000","last_clean":"2024-08-31T07:11:45.194866+0000","last_became_active":"2024-08-31T07:10:46.325958+0000","last_became_peered":"2024-08-31T07:10:46.325958+0000","last_unstale":"2024-08-31T07:11:45.194866+0000","last_undegraded":"2024-08-31T07:11:45.194866+0000","last_fullsized":"2024-08-31T07:11:45.194866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:19.230393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196579+0000","last_change":"2024-08-31T07:09:47.666951+0000","last_active":"2024-08-31T07:11:45.196579+0000","last_peered":"2024-08-31T07:11:45.196579+0000","last_clean":"2024-08-31T07:11:45.196579+0000","last_became_active":"2024-08-31T07:09:47.666475+0000","last_became_peered":"2024-08-31T07:09:47.666475+0000","last_unstale":"2024-08-31T07:11:45.196579+0000","last_undegraded":"2024-08-31T07:11:45.196579+0000","last_fullsized":"2024-08-31T07:11:45.196579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:58:42.242086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.169998+0000","last_change":"2024-08-31T07:09:47.658344+0000","last_active":"2024-08-31T07:11:45.169998+0000","last_peered":"2024-08-31T07:11:45.169998+0000","last_clean":"2024-08-31T07:11:45.169998+0000","last_became_active":"2024-08-31T07:09:47.657991+0000","last_became_peered":"2024-08-31T07:09:47.657991+0000","last_unstale":"2024-08-31T07:11:45.169998+0000","last_undegraded":"2024-08-31T07:11:45.169998+0000","last_fullsized":"2024-08-31T07:11:45.169998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:25:59.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196676+0000","last_change":"2024-08-31T07:09:47.661644+0000","last_active":"2024-08-31T07:11:45.196676+0000","last_peered":"2024-08-31T07:11:45.196676+0000","last_clean":"2024-08-31T07:11:45.196676+0000","last_became_active":"2024-08-31T07:09:47.661248+0000","last_became_peered":"2024-08-31T07:09:47.661248+0000","last_unstale":"2024-08-31T07:11:45.196676+0000","last_undegraded":"2024-08-31T07:11:45.196676+0000","last_fullsized":"2024-08-31T07:11:45.196676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:42.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195194+0000","last_change":"2024-08-31T07:09:47.680052+0000","last_active":"2024-08-31T07:11:45.195194+0000","last_peered":"2024-08-31T07:11:45.195194+0000","last_clean":"2024-08-31T07:11:45.195194+0000","last_became_active":"2024-08-31T07:09:47.679604+0000","last_became_peered":"2024-08-31T07:09:47.679604+0000","last_unstale":"2024-08-31T07:11:45.195194+0000","last_undegraded":"2024-08-31T07:11:45.195194+0000","last_fullsized":"2024-08-31T07:11:45.195194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:23:18.937470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195477+0000","last_change":"2024-08-31T07:09:47.677984+0000","last_active":"2024-08-31T07:11:45.195477+0000","last_peered":"2024-08-31T07:11:45.195477+0000","last_clean":"2024-08-31T07:11:45.195477+0000","last_became_active":"2024-08-31T07:09:47.674957+0000","last_became_peered":"2024-08-31T07:09:47.674957+0000","last_unstale":"2024-08-31T07:11:45.195477+0000","last_undegraded":"2024-08-31T07:11:45.195477+0000","last_fullsized":"2024-08-31T07:11:45.195477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:08:26.503746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198878+0000","last_change":"2024-08-31T07:09:47.672625+0000","last_active":"2024-08-31T07:11:45.198878+0000","last_peered":"2024-08-31T07:11:45.198878+0000","last_clean":"2024-08-31T07:11:45.198878+0000","last_became_active":"2024-08-31T07:09:47.672184+0000","last_became_peered":"2024-08-31T07:09:47.672184+0000","last_unstale":"2024-08-31T07:11:45.198878+0000","last_undegraded":"2024-08-31T07:11:45.198878+0000","last_fullsized":"2024-08-31T07:11:45.198878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:17.091659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198670+0000","last_change":"2024-08-31T07:09:47.672597+0000","last_active":"2024-08-31T07:11:45.198670+0000","last_peered":"2024-08-31T07:11:45.198670+0000","last_clean":"2024-08-31T07:11:45.198670+0000","last_became_active":"2024-08-31T07:09:47.672153+0000","last_became_peered":"2024-08-31T07:09:47.672153+0000","last_unstale":"2024-08-31T07:11:45.198670+0000","last_undegraded":"2024-08-31T07:11:45.198670+0000","last_fullsized":"2024-08-31T07:11:45.198670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:26:41.388526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193223+0000","last_change":"2024-08-31T07:09:47.666253+0000","last_active":"2024-08-31T07:11:45.193223+0000","last_peered":"2024-08-31T07:11:45.193223+0000","last_clean":"2024-08-31T07:11:45.193223+0000","last_became_active":"2024-08-31T07:09:47.663732+0000","last_became_peered":"2024-08-31T07:09:47.663732+0000","last_unstale":"2024-08-31T07:11:45.193223+0000","last_undegraded":"2024-08-31T07:11:45.193223+0000","last_fullsized":"2024-08-31T07:11:45.193223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:10:59.887765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188202+0000","last_change":"2024-08-31T07:09:47.668502+0000","last_active":"2024-08-31T07:11:45.188202+0000","last_peered":"2024-08-31T07:11:45.188202+0000","last_clean":"2024-08-31T07:11:45.188202+0000","last_became_active":"2024-08-31T07:09:47.666487+0000","last_became_peered":"2024-08-31T07:09:47.666487+0000","last_unstale":"2024-08-31T07:11:45.188202+0000","last_undegraded":"2024-08-31T07:11:45.188202+0000","last_fullsized":"2024-08-31T07:11:45.188202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:49:25.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.184945+0000","last_change":"2024-08-31T07:09:47.659454+0000","last_active":"2024-08-31T07:11:45.184945+0000","last_peered":"2024-08-31T07:11:45.184945+0000","last_clean":"2024-08-31T07:11:45.184945+0000","last_became_active":"2024-08-31T07:09:47.658976+0000","last_became_peered":"2024-08-31T07:09:47.658976+0000","last_unstale":"2024-08-31T07:11:45.184945+0000","last_undegraded":"2024-08-31T07:11:45.184945+0000","last_fullsized":"2024-08-31T07:11:45.184945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:31.374905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199707+0000","last_change":"2024-08-31T07:09:47.673612+0000","last_active":"2024-08-31T07:11:45.199707+0000","last_peered":"2024-08-31T07:11:45.199707+0000","last_clean":"2024-08-31T07:11:45.199707+0000","last_became_active":"2024-08-31T07:09:47.673137+0000","last_became_peered":"2024-08-31T07:09:47.673137+0000","last_unstale":"2024-08-31T07:11:45.199707+0000","last_undegraded":"2024-08-31T07:11:45.199707+0000","last_fullsized":"2024-08-31T07:11:45.199707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:46:11.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194525+0000","last_change":"2024-08-31T07:09:47.669195+0000","last_active":"2024-08-31T07:11:45.194525+0000","last_peered":"2024-08-31T07:11:45.194525+0000","last_clean":"2024-08-31T07:11:45.194525+0000","last_became_active":"2024-08-31T07:09:47.668790+0000","last_became_peered":"2024-08-31T07:09:47.668790+0000","last_unstale":"2024-08-31T07:11:45.194525+0000","last_undegraded":"2024-08-31T07:11:45.194525+0000","last_fullsized":"2024-08-31T07:11:45.194525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:27:14.393931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.181333+0000","last_change":"2024-08-31T07:09:47.667566+0000","last_active":"2024-08-31T07:11:45.181333+0000","last_peered":"2024-08-31T07:11:45.181333+0000","last_clean":"2024-08-31T07:11:45.181333+0000","last_became_active":"2024-08-31T07:09:47.666605+0000","last_became_peered":"2024-08-31T07:09:47.666605+0000","last_unstale":"2024-08-31T07:11:45.181333+0000","last_undegraded":"2024-08-31T07:11:45.181333+0000","last_fullsized":"2024-08-31T07:11:45.181333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:05:44.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192240+0000","last_change":"2024-08-31T07:09:47.660478+0000","last_active":"2024-08-31T07:11:45.192240+0000","last_peered":"2024-08-31T07:11:45.192240+0000","last_clean":"2024-08-31T07:11:45.192240+0000","last_became_active":"2024-08-31T07:09:47.660107+0000","last_became_peered":"2024-08-31T07:09:47.660107+0000","last_unstale":"2024-08-31T07:11:45.192240+0000","last_undegraded":"2024-08-31T07:11:45.192240+0000","last_fullsized":"2024-08-31T07:11:45.192240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:29.133370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188131+0000","last_change":"2024-08-31T07:09:47.671967+0000","last_active":"2024-08-31T07:11:45.188131+0000","last_peered":"2024-08-31T07:11:45.188131+0000","last_clean":"2024-08-31T07:11:45.188131+0000","last_became_active":"2024-08-31T07:09:47.671569+0000","last_became_peered":"2024-08-31T07:09:47.671569+0000","last_unstale":"2024-08-31T07:11:45.188131+0000","last_undegraded":"2024-08-31T07:11:45.188131+0000","last_fullsized":"2024-08-31T07:11:45.188131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:19:15.107958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185867+0000","last_change":"2024-08-31T07:09:47.651055+0000","last_active":"2024-08-31T07:11:45.185867+0000","last_peered":"2024-08-31T07:11:45.185867+0000","last_clean":"2024-08-31T07:11:45.185867+0000","last_became_active":"2024-08-31T07:09:47.650674+0000","last_became_peered":"2024-08-31T07:09:47.650674+0000","last_unstale":"2024-08-31T07:11:45.185867+0000","last_undegraded":"2024-08-31T07:11:45.185867+0000","last_fullsized":"2024-08-31T07:11:45.185867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:35:54.647379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180140+0000","last_change":"2024-08-31T07:09:47.663231+0000","last_active":"2024-08-31T07:11:45.180140+0000","last_peered":"2024-08-31T07:11:45.180140+0000","last_clean":"2024-08-31T07:11:45.180140+0000","last_became_active":"2024-08-31T07:09:47.662854+0000","last_became_peered":"2024-08-31T07:09:47.662854+0000","last_unstale":"2024-08-31T07:11:45.180140+0000","last_undegraded":"2024-08-31T07:11:45.180140+0000","last_fullsized":"2024-08-31T07:11:45.180140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:51.358557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199287+0000","last_change":"2024-08-31T07:10:46.326380+0000","last_active":"2024-08-31T07:11:45.199287+0000","last_peered":"2024-08-31T07:11:45.199287+0000","last_clean":"2024-08-31T07:11:45.199287+0000","last_became_active":"2024-08-31T07:10:46.325784+0000","last_became_peered":"2024-08-31T07:10:46.325784+0000","last_unstale":"2024-08-31T07:11:45.199287+0000","last_undegraded":"2024-08-31T07:11:45.199287+0000","last_fullsized":"2024-08-31T07:11:45.199287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:16:56.396024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200247+0000","last_change":"2024-08-31T07:09:47.672108+0000","last_active":"2024-08-31T07:11:45.200247+0000","last_peered":"2024-08-31T07:11:45.200247+0000","last_clean":"2024-08-31T07:11:45.200247+0000","last_became_active":"2024-08-31T07:09:47.670998+0000","last_became_peered":"2024-08-31T07:09:47.670998+0000","last_unstale":"2024-08-31T07:11:45.200247+0000","last_undegraded":"2024-08-31T07:11:45.200247+0000","last_fullsized":"2024-08-31T07:11:45.200247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:26:49.319861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198226+0000","last_change":"2024-08-31T07:09:47.665548+0000","last_active":"2024-08-31T07:11:45.198226+0000","last_peered":"2024-08-31T07:11:45.198226+0000","last_clean":"2024-08-31T07:11:45.198226+0000","last_became_active":"2024-08-31T07:09:47.665177+0000","last_became_peered":"2024-08-31T07:09:47.665177+0000","last_unstale":"2024-08-31T07:11:45.198226+0000","last_undegraded":"2024-08-31T07:11:45.198226+0000","last_fullsized":"2024-08-31T07:11:45.198226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:03:15.892224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.177785+0000","last_change":"2024-08-31T07:10:46.026746+0000","last_active":"2024-08-31T07:11:45.177785+0000","last_peered":"2024-08-31T07:11:45.177785+0000","last_clean":"2024-08-31T07:11:45.177785+0000","last_became_active":"2024-08-31T07:10:46.026286+0000","last_became_peered":"2024-08-31T07:10:46.026286+0000","last_unstale":"2024-08-31T07:11:45.177785+0000","last_undegraded":"2024-08-31T07:11:45.177785+0000","last_fullsized":"2024-08-31T07:11:45.177785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:34:17.120906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180769+0000","last_change":"2024-08-31T07:09:47.667456+0000","last_active":"2024-08-31T07:11:45.180769+0000","last_peered":"2024-08-31T07:11:45.180769+0000","last_clean":"2024-08-31T07:11:45.180769+0000","last_became_active":"2024-08-31T07:09:47.667102+0000","last_became_peered":"2024-08-31T07:09:47.667102+0000","last_unstale":"2024-08-31T07:11:45.180769+0000","last_undegraded":"2024-08-31T07:11:45.180769+0000","last_fullsized":"2024-08-31T07:11:45.180769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:44:39.747729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.181991+0000","last_change":"2024-08-31T07:10:46.326139+0000","last_active":"2024-08-31T07:11:45.181991+0000","last_peered":"2024-08-31T07:11:45.181991+0000","last_clean":"2024-08-31T07:11:45.181991+0000","last_became_active":"2024-08-31T07:10:46.325785+0000","last_became_peered":"2024-08-31T07:10:46.325785+0000","last_unstale":"2024-08-31T07:11:45.181991+0000","last_undegraded":"2024-08-31T07:11:45.181991+0000","last_fullsized":"2024-08-31T07:11:45.181991+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:13.495835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":170,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:16.094074+0000","last_change":"2024-08-31T07:09:45.641088+0000","last_active":"2024-08-31T07:12:16.094074+0000","last_peered":"2024-08-31T07:12:16.094074+0000","last_clean":"2024-08-31T07:12:16.094074+0000","last_became_active":"2024-08-31T07:09:45.640673+0000","last_became_peered":"2024-08-31T07:09:45.640673+0000","last_unstale":"2024-08-31T07:12:16.094074+0000","last_undegraded":"2024-08-31T07:12:16.094074+0000","last_fullsized":"2024-08-31T07:12:16.094074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:30:17.593494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192254+0000","last_change":"2024-08-31T07:09:47.666703+0000","last_active":"2024-08-31T07:11:45.192254+0000","last_peered":"2024-08-31T07:11:45.192254+0000","last_clean":"2024-08-31T07:11:45.192254+0000","last_became_active":"2024-08-31T07:09:47.666348+0000","last_became_peered":"2024-08-31T07:09:47.666348+0000","last_unstale":"2024-08-31T07:11:45.192254+0000","last_undegraded":"2024-08-31T07:11:45.192254+0000","last_fullsized":"2024-08-31T07:11:45.192254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:39.688810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019815+0000","last_change":"2024-08-31T07:09:45.642431+0000","last_active":"2024-08-31T07:10:46.019815+0000","last_peered":"2024-08-31T07:10:46.019815+0000","last_clean":"2024-08-31T07:10:46.019815+0000","last_became_active":"2024-08-31T07:09:45.642048+0000","last_became_peered":"2024-08-31T07:09:45.642048+0000","last_unstale":"2024-08-31T07:10:46.019815+0000","last_undegraded":"2024-08-31T07:10:46.019815+0000","last_fullsized":"2024-08-31T07:10:46.019815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:47:42.574442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017286+0000","last_change":"2024-08-31T07:09:45.632162+0000","last_active":"2024-08-31T07:10:46.017286+0000","last_peered":"2024-08-31T07:10:46.017286+0000","last_clean":"2024-08-31T07:10:46.017286+0000","last_became_active":"2024-08-31T07:09:45.631749+0000","last_became_peered":"2024-08-31T07:09:45.631749+0000","last_unstale":"2024-08-31T07:10:46.017286+0000","last_undegraded":"2024-08-31T07:10:46.017286+0000","last_fullsized":"2024-08-31T07:10:46.017286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:16:29.967823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185787+0000","last_change":"2024-08-31T07:09:47.666288+0000","last_active":"2024-08-31T07:11:45.185787+0000","last_peered":"2024-08-31T07:11:45.185787+0000","last_clean":"2024-08-31T07:11:45.185787+0000","last_became_active":"2024-08-31T07:09:47.665723+0000","last_became_peered":"2024-08-31T07:09:47.665723+0000","last_unstale":"2024-08-31T07:11:45.185787+0000","last_undegraded":"2024-08-31T07:11:45.185787+0000","last_fullsized":"2024-08-31T07:11:45.185787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:38:28.122720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023120+0000","last_change":"2024-08-31T07:09:45.636508+0000","last_active":"2024-08-31T07:10:46.023120+0000","last_peered":"2024-08-31T07:10:46.023120+0000","last_clean":"2024-08-31T07:10:46.023120+0000","last_became_active":"2024-08-31T07:09:45.636073+0000","last_became_peered":"2024-08-31T07:09:45.636073+0000","last_unstale":"2024-08-31T07:10:46.023120+0000","last_undegraded":"2024-08-31T07:10:46.023120+0000","last_fullsized":"2024-08-31T07:10:46.023120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:55:45.063517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187943+0000","last_change":"2024-08-31T07:09:47.661306+0000","last_active":"2024-08-31T07:11:45.187943+0000","last_peered":"2024-08-31T07:11:45.187943+0000","last_clean":"2024-08-31T07:11:45.187943+0000","last_became_active":"2024-08-31T07:09:47.660957+0000","last_became_peered":"2024-08-31T07:09:47.660957+0000","last_unstale":"2024-08-31T07:11:45.187943+0000","last_undegraded":"2024-08-31T07:11:45.187943+0000","last_fullsized":"2024-08-31T07:11:45.187943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:18:23.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'6","reported_seq":288,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:11.085576+0000","last_change":"2024-08-31T07:09:45.640947+0000","last_active":"2024-08-31T07:12:11.085576+0000","last_peered":"2024-08-31T07:12:11.085576+0000","last_clean":"2024-08-31T07:12:11.085576+0000","last_became_active":"2024-08-31T07:09:45.640532+0000","last_became_peered":"2024-08-31T07:09:45.640532+0000","last_unstale":"2024-08-31T07:12:11.085576+0000","last_undegraded":"2024-08-31T07:12:11.085576+0000","last_fullsized":"2024-08-31T07:12:11.085576+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:28.177795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.191708+0000","last_change":"2024-08-31T07:09:47.661060+0000","last_active":"2024-08-31T07:11:45.191708+0000","last_peered":"2024-08-31T07:11:45.191708+0000","last_clean":"2024-08-31T07:11:45.191708+0000","last_became_active":"2024-08-31T07:09:47.660692+0000","last_became_peered":"2024-08-31T07:09:47.660692+0000","last_unstale":"2024-08-31T07:11:45.191708+0000","last_undegraded":"2024-08-31T07:11:45.191708+0000","last_fullsized":"2024-08-31T07:11:45.191708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:48:38.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257194+0000","last_change":"2024-08-31T07:09:45.641155+0000","last_active":"2024-08-31T07:10:46.257194+0000","last_peered":"2024-08-31T07:10:46.257194+0000","last_clean":"2024-08-31T07:10:46.257194+0000","last_became_active":"2024-08-31T07:09:45.640680+0000","last_became_peered":"2024-08-31T07:09:45.640680+0000","last_unstale":"2024-08-31T07:10:46.257194+0000","last_undegraded":"2024-08-31T07:10:46.257194+0000","last_fullsized":"2024-08-31T07:10:46.257194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:29:03.287570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199892+0000","last_change":"2024-08-31T07:09:47.675582+0000","last_active":"2024-08-31T07:11:45.199892+0000","last_peered":"2024-08-31T07:11:45.199892+0000","last_clean":"2024-08-31T07:11:45.199892+0000","last_became_active":"2024-08-31T07:09:47.675266+0000","last_became_peered":"2024-08-31T07:09:47.675266+0000","last_unstale":"2024-08-31T07:11:45.199892+0000","last_undegraded":"2024-08-31T07:11:45.199892+0000","last_fullsized":"2024-08-31T07:11:45.199892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:36:33.122531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021194+0000","last_change":"2024-08-31T07:09:45.651006+0000","last_active":"2024-08-31T07:10:46.021194+0000","last_peered":"2024-08-31T07:10:46.021194+0000","last_clean":"2024-08-31T07:10:46.021194+0000","last_became_active":"2024-08-31T07:09:45.650588+0000","last_became_peered":"2024-08-31T07:09:45.650588+0000","last_unstale":"2024-08-31T07:10:46.021194+0000","last_undegraded":"2024-08-31T07:10:46.021194+0000","last_fullsized":"2024-08-31T07:10:46.021194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:34:16.015892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.190030+0000","last_change":"2024-08-31T07:09:47.662289+0000","last_active":"2024-08-31T07:11:45.190030+0000","last_peered":"2024-08-31T07:11:45.190030+0000","last_clean":"2024-08-31T07:11:45.190030+0000","last_became_active":"2024-08-31T07:09:47.661932+0000","last_became_peered":"2024-08-31T07:09:47.661932+0000","last_unstale":"2024-08-31T07:11:45.190030+0000","last_undegraded":"2024-08-31T07:11:45.190030+0000","last_fullsized":"2024-08-31T07:11:45.190030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:50.440504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":328,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:11.537240+0000","last_change":"2024-08-31T07:09:45.642196+0000","last_active":"2024-08-31T07:12:11.537240+0000","last_peered":"2024-08-31T07:12:11.537240+0000","last_clean":"2024-08-31T07:12:11.537240+0000","last_became_active":"2024-08-31T07:09:45.641797+0000","last_became_peered":"2024-08-31T07:09:45.641797+0000","last_unstale":"2024-08-31T07:12:11.537240+0000","last_undegraded":"2024-08-31T07:12:11.537240+0000","last_fullsized":"2024-08-31T07:12:11.537240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:51:05.953335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61177,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":266,"num_read_kb":192,"num_write":38,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196022+0000","last_change":"2024-08-31T07:09:47.660184+0000","last_active":"2024-08-31T07:11:45.196022+0000","last_peered":"2024-08-31T07:11:45.196022+0000","last_clean":"2024-08-31T07:11:45.196022+0000","last_became_active":"2024-08-31T07:09:47.659571+0000","last_became_peered":"2024-08-31T07:09:47.659571+0000","last_unstale":"2024-08-31T07:11:45.196022+0000","last_undegraded":"2024-08-31T07:11:45.196022+0000","last_fullsized":"2024-08-31T07:11:45.196022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:41:13.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.035782+0000","last_change":"2024-08-31T07:10:46.035336+0000","last_active":"2024-08-31T07:10:46.035782+0000","last_peered":"2024-08-31T07:10:46.035782+0000","last_clean":"2024-08-31T07:10:46.035782+0000","last_became_active":"2024-08-31T07:10:46.021299+0000","last_became_peered":"2024-08-31T07:10:46.021299+0000","last_unstale":"2024-08-31T07:10:46.035782+0000","last_undegraded":"2024-08-31T07:10:46.035782+0000","last_fullsized":"2024-08-31T07:10:46.035782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:06:15.931855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194680+0000","last_change":"2024-08-31T07:10:46.326354+0000","last_active":"2024-08-31T07:11:45.194680+0000","last_peered":"2024-08-31T07:11:45.194680+0000","last_clean":"2024-08-31T07:11:45.194680+0000","last_became_active":"2024-08-31T07:10:46.325869+0000","last_became_peered":"2024-08-31T07:10:46.325869+0000","last_unstale":"2024-08-31T07:11:45.194680+0000","last_undegraded":"2024-08-31T07:11:45.194680+0000","last_fullsized":"2024-08-31T07:11:45.194680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:21.832088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022972+0000","last_change":"2024-08-31T07:09:45.642187+0000","last_active":"2024-08-31T07:10:46.022972+0000","last_peered":"2024-08-31T07:10:46.022972+0000","last_clean":"2024-08-31T07:10:46.022972+0000","last_became_active":"2024-08-31T07:09:45.641721+0000","last_became_peered":"2024-08-31T07:09:45.641721+0000","last_unstale":"2024-08-31T07:10:46.022972+0000","last_undegraded":"2024-08-31T07:10:46.022972+0000","last_fullsized":"2024-08-31T07:10:46.022972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:00:45.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196019+0000","last_change":"2024-08-31T07:09:47.665956+0000","last_active":"2024-08-31T07:11:45.196019+0000","last_peered":"2024-08-31T07:11:45.196019+0000","last_clean":"2024-08-31T07:11:45.196019+0000","last_became_active":"2024-08-31T07:09:47.664367+0000","last_became_peered":"2024-08-31T07:09:47.664367+0000","last_unstale":"2024-08-31T07:11:45.196019+0000","last_undegraded":"2024-08-31T07:11:45.196019+0000","last_fullsized":"2024-08-31T07:11:45.196019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:34.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'7","reported_seq":170,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:15.091359+0000","last_change":"2024-08-31T07:09:45.636440+0000","last_active":"2024-08-31T07:12:15.091359+0000","last_peered":"2024-08-31T07:12:15.091359+0000","last_clean":"2024-08-31T07:12:15.091359+0000","last_became_active":"2024-08-31T07:09:45.636058+0000","last_became_peered":"2024-08-31T07:09:45.636058+0000","last_unstale":"2024-08-31T07:12:15.091359+0000","last_undegraded":"2024-08-31T07:12:15.091359+0000","last_fullsized":"2024-08-31T07:12:15.091359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:38.929227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194840+0000","last_change":"2024-08-31T07:09:47.660564+0000","last_active":"2024-08-31T07:11:45.194840+0000","last_peered":"2024-08-31T07:11:45.194840+0000","last_clean":"2024-08-31T07:11:45.194840+0000","last_became_active":"2024-08-31T07:09:47.660157+0000","last_became_peered":"2024-08-31T07:09:47.660157+0000","last_unstale":"2024-08-31T07:11:45.194840+0000","last_undegraded":"2024-08-31T07:11:45.194840+0000","last_fullsized":"2024-08-31T07:11:45.194840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:49:01.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020515+0000","last_change":"2024-08-31T07:09:45.649140+0000","last_active":"2024-08-31T07:10:46.020515+0000","last_peered":"2024-08-31T07:10:46.020515+0000","last_clean":"2024-08-31T07:10:46.020515+0000","last_became_active":"2024-08-31T07:09:45.648796+0000","last_became_peered":"2024-08-31T07:09:45.648796+0000","last_unstale":"2024-08-31T07:10:46.020515+0000","last_undegraded":"2024-08-31T07:10:46.020515+0000","last_fullsized":"2024-08-31T07:10:46.020515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:33:37.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192390+0000","last_change":"2024-08-31T07:09:47.663833+0000","last_active":"2024-08-31T07:11:45.192390+0000","last_peered":"2024-08-31T07:11:45.192390+0000","last_clean":"2024-08-31T07:11:45.192390+0000","last_became_active":"2024-08-31T07:09:47.663451+0000","last_became_peered":"2024-08-31T07:09:47.663451+0000","last_unstale":"2024-08-31T07:11:45.192390+0000","last_undegraded":"2024-08-31T07:11:45.192390+0000","last_fullsized":"2024-08-31T07:11:45.192390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:27:22.245254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021407+0000","last_change":"2024-08-31T07:09:45.636593+0000","last_active":"2024-08-31T07:10:46.021407+0000","last_peered":"2024-08-31T07:10:46.021407+0000","last_clean":"2024-08-31T07:10:46.021407+0000","last_became_active":"2024-08-31T07:09:45.636166+0000","last_became_peered":"2024-08-31T07:09:45.636166+0000","last_unstale":"2024-08-31T07:10:46.021407+0000","last_undegraded":"2024-08-31T07:10:46.021407+0000","last_fullsized":"2024-08-31T07:10:46.021407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:28.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183385+0000","last_change":"2024-08-31T07:09:47.660825+0000","last_active":"2024-08-31T07:11:45.183385+0000","last_peered":"2024-08-31T07:11:45.183385+0000","last_clean":"2024-08-31T07:11:45.183385+0000","last_became_active":"2024-08-31T07:09:47.660474+0000","last_became_peered":"2024-08-31T07:09:47.660474+0000","last_unstale":"2024-08-31T07:11:45.183385+0000","last_undegraded":"2024-08-31T07:11:45.183385+0000","last_fullsized":"2024-08-31T07:11:45.183385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:42:20.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017462+0000","last_change":"2024-08-31T07:09:45.632727+0000","last_active":"2024-08-31T07:10:46.017462+0000","last_peered":"2024-08-31T07:10:46.017462+0000","last_clean":"2024-08-31T07:10:46.017462+0000","last_became_active":"2024-08-31T07:09:45.632251+0000","last_became_peered":"2024-08-31T07:09:45.632251+0000","last_unstale":"2024-08-31T07:10:46.017462+0000","last_undegraded":"2024-08-31T07:10:46.017462+0000","last_fullsized":"2024-08-31T07:10:46.017462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:09:30.197409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196430+0000","last_change":"2024-08-31T07:09:47.665501+0000","last_active":"2024-08-31T07:11:45.196430+0000","last_peered":"2024-08-31T07:11:45.196430+0000","last_clean":"2024-08-31T07:11:45.196430+0000","last_became_active":"2024-08-31T07:09:47.665162+0000","last_became_peered":"2024-08-31T07:09:47.665162+0000","last_unstale":"2024-08-31T07:11:45.196430+0000","last_undegraded":"2024-08-31T07:11:45.196430+0000","last_fullsized":"2024-08-31T07:11:45.196430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:34:29.605836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021320+0000","last_change":"2024-08-31T07:09:45.636580+0000","last_active":"2024-08-31T07:10:46.021320+0000","last_peered":"2024-08-31T07:10:46.021320+0000","last_clean":"2024-08-31T07:10:46.021320+0000","last_became_active":"2024-08-31T07:09:45.636163+0000","last_became_peered":"2024-08-31T07:09:45.636163+0000","last_unstale":"2024-08-31T07:10:46.021320+0000","last_undegraded":"2024-08-31T07:10:46.021320+0000","last_fullsized":"2024-08-31T07:10:46.021320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:18:07.567858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195120+0000","last_change":"2024-08-31T07:09:47.671865+0000","last_active":"2024-08-31T07:11:45.195120+0000","last_peered":"2024-08-31T07:11:45.195120+0000","last_clean":"2024-08-31T07:11:45.195120+0000","last_became_active":"2024-08-31T07:09:47.671454+0000","last_became_peered":"2024-08-31T07:09:47.671454+0000","last_unstale":"2024-08-31T07:11:45.195120+0000","last_undegraded":"2024-08-31T07:11:45.195120+0000","last_fullsized":"2024-08-31T07:11:45.195120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:47.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'7","reported_seq":168,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:13.088891+0000","last_change":"2024-08-31T07:09:45.653217+0000","last_active":"2024-08-31T07:12:13.088891+0000","last_peered":"2024-08-31T07:12:13.088891+0000","last_clean":"2024-08-31T07:12:13.088891+0000","last_became_active":"2024-08-31T07:09:45.652819+0000","last_became_peered":"2024-08-31T07:09:45.652819+0000","last_unstale":"2024-08-31T07:12:13.088891+0000","last_undegraded":"2024-08-31T07:12:13.088891+0000","last_fullsized":"2024-08-31T07:12:13.088891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:25:11.454068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196166+0000","last_change":"2024-08-31T07:09:47.663652+0000","last_active":"2024-08-31T07:11:45.196166+0000","last_peered":"2024-08-31T07:11:45.196166+0000","last_clean":"2024-08-31T07:11:45.196166+0000","last_became_active":"2024-08-31T07:09:47.663274+0000","last_became_peered":"2024-08-31T07:09:47.663274+0000","last_unstale":"2024-08-31T07:11:45.196166+0000","last_undegraded":"2024-08-31T07:11:45.196166+0000","last_fullsized":"2024-08-31T07:11:45.196166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:31:59.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257370+0000","last_change":"2024-08-31T07:08:27.623298+0000","last_active":"2024-08-31T07:10:46.257370+0000","last_peered":"2024-08-31T07:10:46.257370+0000","last_clean":"2024-08-31T07:10:46.257370+0000","last_became_active":"2024-08-31T07:08:27.619072+0000","last_became_peered":"2024-08-31T07:08:27.619072+0000","last_unstale":"2024-08-31T07:10:46.257370+0000","last_undegraded":"2024-08-31T07:10:46.257370+0000","last_fullsized":"2024-08-31T07:10:46.257370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_clean_scrub_stamp":"2024-08-31T07:04:45.509398+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:07.967421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154319+0000","last_change":"2024-08-31T07:09:45.638104+0000","last_active":"2024-08-31T07:10:46.154319+0000","last_peered":"2024-08-31T07:10:46.154319+0000","last_clean":"2024-08-31T07:10:46.154319+0000","last_became_active":"2024-08-31T07:09:45.637693+0000","last_became_peered":"2024-08-31T07:09:45.637693+0000","last_unstale":"2024-08-31T07:10:46.154319+0000","last_undegraded":"2024-08-31T07:10:46.154319+0000","last_fullsized":"2024-08-31T07:10:46.154319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:50:15.135061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.189353+0000","last_change":"2024-08-31T07:09:47.664958+0000","last_active":"2024-08-31T07:11:45.189353+0000","last_peered":"2024-08-31T07:11:45.189353+0000","last_clean":"2024-08-31T07:11:45.189353+0000","last_became_active":"2024-08-31T07:09:47.661166+0000","last_became_peered":"2024-08-31T07:09:47.661166+0000","last_unstale":"2024-08-31T07:11:45.189353+0000","last_undegraded":"2024-08-31T07:11:45.189353+0000","last_fullsized":"2024-08-31T07:11:45.189353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:45:58.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257184+0000","last_change":"2024-08-31T07:09:45.640881+0000","last_active":"2024-08-31T07:10:46.257184+0000","last_peered":"2024-08-31T07:10:46.257184+0000","last_clean":"2024-08-31T07:10:46.257184+0000","last_became_active":"2024-08-31T07:09:45.640487+0000","last_became_peered":"2024-08-31T07:09:45.640487+0000","last_unstale":"2024-08-31T07:10:46.257184+0000","last_undegraded":"2024-08-31T07:10:46.257184+0000","last_fullsized":"2024-08-31T07:10:46.257184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:06.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.171981+0000","last_change":"2024-08-31T07:09:47.655008+0000","last_active":"2024-08-31T07:11:45.171981+0000","last_peered":"2024-08-31T07:11:45.171981+0000","last_clean":"2024-08-31T07:11:45.171981+0000","last_became_active":"2024-08-31T07:09:47.654645+0000","last_became_peered":"2024-08-31T07:09:47.654645+0000","last_unstale":"2024-08-31T07:11:45.171981+0000","last_undegraded":"2024-08-31T07:11:45.171981+0000","last_fullsized":"2024-08-31T07:11:45.171981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:23:55.916053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'6","reported_seq":163,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:12.086211+0000","last_change":"2024-08-31T07:09:45.649324+0000","last_active":"2024-08-31T07:12:12.086211+0000","last_peered":"2024-08-31T07:12:12.086211+0000","last_clean":"2024-08-31T07:12:12.086211+0000","last_became_active":"2024-08-31T07:09:45.648915+0000","last_became_peered":"2024-08-31T07:09:45.648915+0000","last_unstale":"2024-08-31T07:12:12.086211+0000","last_undegraded":"2024-08-31T07:12:12.086211+0000","last_fullsized":"2024-08-31T07:12:12.086211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:31:57.853654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187160+0000","last_change":"2024-08-31T07:09:47.661328+0000","last_active":"2024-08-31T07:11:45.187160+0000","last_peered":"2024-08-31T07:11:45.187160+0000","last_clean":"2024-08-31T07:11:45.187160+0000","last_became_active":"2024-08-31T07:09:47.660927+0000","last_became_peered":"2024-08-31T07:09:47.660927+0000","last_unstale":"2024-08-31T07:11:45.187160+0000","last_undegraded":"2024-08-31T07:11:45.187160+0000","last_fullsized":"2024-08-31T07:11:45.187160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:40:46.876894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'7","reported_seq":166,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:12.087548+0000","last_change":"2024-08-31T07:09:45.637747+0000","last_active":"2024-08-31T07:12:12.087548+0000","last_peered":"2024-08-31T07:12:12.087548+0000","last_clean":"2024-08-31T07:12:12.087548+0000","last_became_active":"2024-08-31T07:09:45.637368+0000","last_became_peered":"2024-08-31T07:09:45.637368+0000","last_unstale":"2024-08-31T07:12:12.087548+0000","last_undegraded":"2024-08-31T07:12:12.087548+0000","last_fullsized":"2024-08-31T07:12:12.087548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:01.174109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193217+0000","last_change":"2024-08-31T07:09:47.680802+0000","last_active":"2024-08-31T07:11:45.193217+0000","last_peered":"2024-08-31T07:11:45.193217+0000","last_clean":"2024-08-31T07:11:45.193217+0000","last_became_active":"2024-08-31T07:09:47.680363+0000","last_became_peered":"2024-08-31T07:09:47.680363+0000","last_unstale":"2024-08-31T07:11:45.193217+0000","last_undegraded":"2024-08-31T07:11:45.193217+0000","last_fullsized":"2024-08-31T07:11:45.193217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:46:48.744071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324353+0000","last_change":"2024-08-31T07:09:45.651222+0000","last_active":"2024-08-31T07:10:46.324353+0000","last_peered":"2024-08-31T07:10:46.324353+0000","last_clean":"2024-08-31T07:10:46.324353+0000","last_became_active":"2024-08-31T07:09:45.650760+0000","last_became_peered":"2024-08-31T07:09:45.650760+0000","last_unstale":"2024-08-31T07:10:46.324353+0000","last_undegraded":"2024-08-31T07:10:46.324353+0000","last_fullsized":"2024-08-31T07:10:46.324353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T19:03:33.942165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183968+0000","last_change":"2024-08-31T07:09:47.665881+0000","last_active":"2024-08-31T07:11:45.183968+0000","last_peered":"2024-08-31T07:11:45.183968+0000","last_clean":"2024-08-31T07:11:45.183968+0000","last_became_active":"2024-08-31T07:09:47.665485+0000","last_became_peered":"2024-08-31T07:09:47.665485+0000","last_unstale":"2024-08-31T07:11:45.183968+0000","last_undegraded":"2024-08-31T07:11:45.183968+0000","last_fullsized":"2024-08-31T07:11:45.183968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:30.820714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020851+0000","last_change":"2024-08-31T07:09:45.651893+0000","last_active":"2024-08-31T07:10:46.020851+0000","last_peered":"2024-08-31T07:10:46.020851+0000","last_clean":"2024-08-31T07:10:46.020851+0000","last_became_active":"2024-08-31T07:09:45.651547+0000","last_became_peered":"2024-08-31T07:09:45.651547+0000","last_unstale":"2024-08-31T07:10:46.020851+0000","last_undegraded":"2024-08-31T07:10:46.020851+0000","last_fullsized":"2024-08-31T07:10:46.020851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:11:30.708018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.175560+0000","last_change":"2024-08-31T07:09:47.655551+0000","last_active":"2024-08-31T07:11:45.175560+0000","last_peered":"2024-08-31T07:11:45.175560+0000","last_clean":"2024-08-31T07:11:45.175560+0000","last_became_active":"2024-08-31T07:09:47.655188+0000","last_became_peered":"2024-08-31T07:09:47.655188+0000","last_unstale":"2024-08-31T07:11:45.175560+0000","last_undegraded":"2024-08-31T07:11:45.175560+0000","last_fullsized":"2024-08-31T07:11:45.175560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:43:51.419599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'17","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:55.669546+0000","last_change":"2024-08-31T07:09:45.636212+0000","last_active":"2024-08-31T07:11:55.669546+0000","last_peered":"2024-08-31T07:11:55.669546+0000","last_clean":"2024-08-31T07:11:55.669546+0000","last_became_active":"2024-08-31T07:09:45.635678+0000","last_became_peered":"2024-08-31T07:09:45.635678+0000","last_unstale":"2024-08-31T07:11:55.669546+0000","last_undegraded":"2024-08-31T07:11:55.669546+0000","last_fullsized":"2024-08-31T07:11:55.669546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:13:17.803459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185987+0000","last_change":"2024-08-31T07:09:47.666476+0000","last_active":"2024-08-31T07:11:45.185987+0000","last_peered":"2024-08-31T07:11:45.185987+0000","last_clean":"2024-08-31T07:11:45.185987+0000","last_became_active":"2024-08-31T07:09:47.666111+0000","last_became_peered":"2024-08-31T07:09:47.666111+0000","last_unstale":"2024-08-31T07:11:45.185987+0000","last_undegraded":"2024-08-31T07:11:45.185987+0000","last_fullsized":"2024-08-31T07:11:45.185987+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:08:53.459549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154441+0000","last_change":"2024-08-31T07:09:45.631571+0000","last_active":"2024-08-31T07:10:46.154441+0000","last_peered":"2024-08-31T07:10:46.154441+0000","last_clean":"2024-08-31T07:10:46.154441+0000","last_became_active":"2024-08-31T07:09:45.630981+0000","last_became_peered":"2024-08-31T07:09:45.630981+0000","last_unstale":"2024-08-31T07:10:46.154441+0000","last_undegraded":"2024-08-31T07:10:46.154441+0000","last_fullsized":"2024-08-31T07:10:46.154441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:36:53.482955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193936+0000","last_change":"2024-08-31T07:09:47.675054+0000","last_active":"2024-08-31T07:11:45.193936+0000","last_peered":"2024-08-31T07:11:45.193936+0000","last_clean":"2024-08-31T07:11:45.193936+0000","last_became_active":"2024-08-31T07:09:47.672652+0000","last_became_peered":"2024-08-31T07:09:47.672652+0000","last_unstale":"2024-08-31T07:11:45.193936+0000","last_undegraded":"2024-08-31T07:11:45.193936+0000","last_fullsized":"2024-08-31T07:11:45.193936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:32:05.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016284+0000","last_change":"2024-08-31T07:09:45.648746+0000","last_active":"2024-08-31T07:10:46.016284+0000","last_peered":"2024-08-31T07:10:46.016284+0000","last_clean":"2024-08-31T07:10:46.016284+0000","last_became_active":"2024-08-31T07:09:45.648424+0000","last_became_peered":"2024-08-31T07:09:45.648424+0000","last_unstale":"2024-08-31T07:10:46.016284+0000","last_undegraded":"2024-08-31T07:10:46.016284+0000","last_fullsized":"2024-08-31T07:10:46.016284+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:40:32.132963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.165179+0000","last_change":"2024-08-31T07:09:47.663213+0000","last_active":"2024-08-31T07:11:45.165179+0000","last_peered":"2024-08-31T07:11:45.165179+0000","last_clean":"2024-08-31T07:11:45.165179+0000","last_became_active":"2024-08-31T07:09:47.662457+0000","last_became_peered":"2024-08-31T07:09:47.662457+0000","last_unstale":"2024-08-31T07:11:45.165179+0000","last_undegraded":"2024-08-31T07:11:45.165179+0000","last_fullsized":"2024-08-31T07:11:45.165179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:54:27.069200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015653+0000","last_change":"2024-08-31T07:09:45.640149+0000","last_active":"2024-08-31T07:10:46.015653+0000","last_peered":"2024-08-31T07:10:46.015653+0000","last_clean":"2024-08-31T07:10:46.015653+0000","last_became_active":"2024-08-31T07:09:45.639581+0000","last_became_peered":"2024-08-31T07:09:45.639581+0000","last_unstale":"2024-08-31T07:10:46.015653+0000","last_undegraded":"2024-08-31T07:10:46.015653+0000","last_fullsized":"2024-08-31T07:10:46.015653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:27:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198479+0000","last_change":"2024-08-31T07:09:47.672613+0000","last_active":"2024-08-31T07:11:45.198479+0000","last_peered":"2024-08-31T07:11:45.198479+0000","last_clean":"2024-08-31T07:11:45.198479+0000","last_became_active":"2024-08-31T07:09:47.672173+0000","last_became_peered":"2024-08-31T07:09:47.672173+0000","last_unstale":"2024-08-31T07:11:45.198479+0000","last_undegraded":"2024-08-31T07:11:45.198479+0000","last_fullsized":"2024-08-31T07:11:45.198479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:49.706113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022590+0000","last_change":"2024-08-31T07:09:45.637567+0000","last_active":"2024-08-31T07:10:46.022590+0000","last_peered":"2024-08-31T07:10:46.022590+0000","last_clean":"2024-08-31T07:10:46.022590+0000","last_became_active":"2024-08-31T07:09:45.637180+0000","last_became_peered":"2024-08-31T07:09:45.637180+0000","last_unstale":"2024-08-31T07:10:46.022590+0000","last_undegraded":"2024-08-31T07:10:46.022590+0000","last_fullsized":"2024-08-31T07:10:46.022590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:48:07.190829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194363+0000","last_change":"2024-08-31T07:09:47.674980+0000","last_active":"2024-08-31T07:11:45.194363+0000","last_peered":"2024-08-31T07:11:45.194363+0000","last_clean":"2024-08-31T07:11:45.194363+0000","last_became_active":"2024-08-31T07:09:47.674528+0000","last_became_peered":"2024-08-31T07:09:47.674528+0000","last_unstale":"2024-08-31T07:11:45.194363+0000","last_undegraded":"2024-08-31T07:11:45.194363+0000","last_fullsized":"2024-08-31T07:11:45.194363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:15:09.464221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021096+0000","last_change":"2024-08-31T07:09:45.641167+0000","last_active":"2024-08-31T07:10:46.021096+0000","last_peered":"2024-08-31T07:10:46.021096+0000","last_clean":"2024-08-31T07:10:46.021096+0000","last_became_active":"2024-08-31T07:09:45.640761+0000","last_became_peered":"2024-08-31T07:09:45.640761+0000","last_unstale":"2024-08-31T07:10:46.021096+0000","last_undegraded":"2024-08-31T07:10:46.021096+0000","last_fullsized":"2024-08-31T07:10:46.021096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:28:59.963145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195596+0000","last_change":"2024-08-31T07:09:47.668808+0000","last_active":"2024-08-31T07:11:45.195596+0000","last_peered":"2024-08-31T07:11:45.195596+0000","last_clean":"2024-08-31T07:11:45.195596+0000","last_became_active":"2024-08-31T07:09:47.668291+0000","last_became_peered":"2024-08-31T07:09:47.668291+0000","last_unstale":"2024-08-31T07:11:45.195596+0000","last_undegraded":"2024-08-31T07:11:45.195596+0000","last_fullsized":"2024-08-31T07:11:45.195596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:23.433511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192982+0000","last_change":"2024-08-31T07:09:47.668039+0000","last_active":"2024-08-31T07:11:45.192982+0000","last_peered":"2024-08-31T07:11:45.192982+0000","last_clean":"2024-08-31T07:11:45.192982+0000","last_became_active":"2024-08-31T07:09:47.667567+0000","last_became_peered":"2024-08-31T07:09:47.667567+0000","last_unstale":"2024-08-31T07:11:45.192982+0000","last_undegraded":"2024-08-31T07:11:45.192982+0000","last_fullsized":"2024-08-31T07:11:45.192982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:33:50.192864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.041708+0000","last_change":"2024-08-31T07:10:46.041197+0000","last_active":"2024-08-31T07:10:46.041708+0000","last_peered":"2024-08-31T07:10:46.041708+0000","last_clean":"2024-08-31T07:10:46.041708+0000","last_became_active":"2024-08-31T07:10:46.023966+0000","last_became_peered":"2024-08-31T07:10:46.023966+0000","last_unstale":"2024-08-31T07:10:46.041708+0000","last_undegraded":"2024-08-31T07:10:46.041708+0000","last_fullsized":"2024-08-31T07:10:46.041708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:34.302665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192551+0000","last_change":"2024-08-31T07:09:47.660615+0000","last_active":"2024-08-31T07:11:45.192551+0000","last_peered":"2024-08-31T07:11:45.192551+0000","last_clean":"2024-08-31T07:11:45.192551+0000","last_became_active":"2024-08-31T07:09:47.660269+0000","last_became_peered":"2024-08-31T07:09:47.660269+0000","last_unstale":"2024-08-31T07:11:45.192551+0000","last_undegraded":"2024-08-31T07:11:45.192551+0000","last_fullsized":"2024-08-31T07:11:45.192551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:42:12.618039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020701+0000","last_change":"2024-08-31T07:09:45.641699+0000","last_active":"2024-08-31T07:10:46.020701+0000","last_peered":"2024-08-31T07:10:46.020701+0000","last_clean":"2024-08-31T07:10:46.020701+0000","last_became_active":"2024-08-31T07:09:45.641122+0000","last_became_peered":"2024-08-31T07:09:45.641122+0000","last_unstale":"2024-08-31T07:10:46.020701+0000","last_undegraded":"2024-08-31T07:10:46.020701+0000","last_fullsized":"2024-08-31T07:10:46.020701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:26:07.130266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200241+0000","last_change":"2024-08-31T07:09:47.675459+0000","last_active":"2024-08-31T07:11:45.200241+0000","last_peered":"2024-08-31T07:11:45.200241+0000","last_clean":"2024-08-31T07:11:45.200241+0000","last_became_active":"2024-08-31T07:09:47.672246+0000","last_became_peered":"2024-08-31T07:09:47.672246+0000","last_unstale":"2024-08-31T07:11:45.200241+0000","last_undegraded":"2024-08-31T07:11:45.200241+0000","last_fullsized":"2024-08-31T07:11:45.200241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:49:23.547352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016428+0000","last_change":"2024-08-31T07:09:45.649522+0000","last_active":"2024-08-31T07:10:46.016428+0000","last_peered":"2024-08-31T07:10:46.016428+0000","last_clean":"2024-08-31T07:10:46.016428+0000","last_became_active":"2024-08-31T07:09:45.649109+0000","last_became_peered":"2024-08-31T07:09:45.649109+0000","last_unstale":"2024-08-31T07:10:46.016428+0000","last_undegraded":"2024-08-31T07:10:46.016428+0000","last_fullsized":"2024-08-31T07:10:46.016428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:52:52.432605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.179375+0000","last_change":"2024-08-31T07:09:47.670583+0000","last_active":"2024-08-31T07:11:45.179375+0000","last_peered":"2024-08-31T07:11:45.179375+0000","last_clean":"2024-08-31T07:11:45.179375+0000","last_became_active":"2024-08-31T07:09:47.670162+0000","last_became_peered":"2024-08-31T07:09:47.670162+0000","last_unstale":"2024-08-31T07:11:45.179375+0000","last_undegraded":"2024-08-31T07:11:45.179375+0000","last_fullsized":"2024-08-31T07:11:45.179375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:35:35.135678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'14","reported_seq":173,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:14.089952+0000","last_change":"2024-08-31T07:09:45.639636+0000","last_active":"2024-08-31T07:12:14.089952+0000","last_peered":"2024-08-31T07:12:14.089952+0000","last_clean":"2024-08-31T07:12:14.089952+0000","last_became_active":"2024-08-31T07:09:45.639029+0000","last_became_peered":"2024-08-31T07:09:45.639029+0000","last_unstale":"2024-08-31T07:12:14.089952+0000","last_undegraded":"2024-08-31T07:12:14.089952+0000","last_fullsized":"2024-08-31T07:12:14.089952+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:54.110149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69727,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1348,"num_read_kb":747,"num_write":264,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":650,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":209181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285320,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678060,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":434494,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874288,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":1086050,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037814,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":433242,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234045,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":433038,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397569,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1027343,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561093,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":433491,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757326,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1025957,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":433352,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117079,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432784,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378717,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":496265,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542243,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":495581,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705767,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":434830,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T07:12:18.579 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-08-31T07:12:18.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-31T07:12:18.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:18 smithi032 ceph-mon[17252]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-31T07:12:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.842+0000 7f306f000640 1 -- 172.21.15.32:0/2567857850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306810db20 msgr2=0x7f3068110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.842+0000 7f306f000640 1 --2- 172.21.15.32:0/2567857850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306810db20 0x7f3068110010 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3060009ca0 tx=0x7f306002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.842+0000 7f306f000640 1 -- 172.21.15.32:0/2567857850 shutdown_connections 2024-08-31T07:12:18.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.842+0000 7f306f000640 1 --2- 172.21.15.32:0/2567857850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306810db20 0x7f3068110010 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.842+0000 7f306f000640 1 --2- 172.21.15.32:0/2567857850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3068105010 0x7f306810d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.842+0000 7f306f000640 1 --2- 172.21.15.32:0/2567857850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30681046c0 0x7f3068104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.842+0000 7f306f000640 1 -- 172.21.15.32:0/2567857850 >> 172.21.15.32:0/2567857850 conn(0x7f30680fe150 msgr2=0x7f30681005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:18.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.842+0000 7f306f000640 1 -- 172.21.15.32:0/2567857850 shutdown_connections 2024-08-31T07:12:18.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.842+0000 7f306f000640 1 -- 172.21.15.32:0/2567857850 wait complete. 2024-08-31T07:12:18.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.843+0000 7f306f000640 1 Processor -- start 2024-08-31T07:12:18.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.843+0000 7f306f000640 1 -- start start 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f306f000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30681046c0 0x7f306819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f306f000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3068105010 0x7f306819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f306f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f306810db20 0x7f30681a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f306f000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3068113790 con 0x7f30681046c0 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f306f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3068113610 con 0x7f306810db20 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f306f000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3068113910 con 0x7f3068105010 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f3067e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30681046c0 0x7f306819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f3067e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30681046c0 0x7f306819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58478/0 (socket says 172.21.15.32:58478) 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f3067e00640 1 -- 172.21.15.32:0/3268085281 learned_addr learned my addr 172.21.15.32:0/3268085281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f305fe00640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3068105010 0x7f306819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.844+0000 7f306d000640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f306810db20 0x7f30681a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f3067e00640 1 -- 172.21.15.32:0/3268085281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3068105010 msgr2=0x7f306819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f3067e00640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3068105010 0x7f306819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f3067e00640 1 -- 172.21.15.32:0/3268085281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f306810db20 msgr2=0x7f30681a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f3067e00640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f306810db20 0x7f30681a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f3067e00640 1 -- 172.21.15.32:0/3268085281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30681a2260 con 0x7f30681046c0 2024-08-31T07:12:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f3067e00640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30681046c0 0x7f306819c330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f305800ea60 tx=0x7f305800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f3065600640 1 -- 172.21.15.32:0/3268085281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305800cde0 con 0x7f30681046c0 2024-08-31T07:12:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f306f000640 1 -- 172.21.15.32:0/3268085281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30681a2550 con 0x7f30681046c0 2024-08-31T07:12:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f3065600640 1 -- 172.21.15.32:0/3268085281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30580035f0 con 0x7f30681046c0 2024-08-31T07:12:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f306f000640 1 -- 172.21.15.32:0/3268085281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30681a8ce0 con 0x7f30681046c0 2024-08-31T07:12:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.845+0000 7f3065600640 1 -- 172.21.15.32:0/3268085281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058005220 con 0x7f30681046c0 2024-08-31T07:12:18.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.847+0000 7f306f000640 1 -- 172.21.15.32:0/3268085281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3068104ad0 con 0x7f30681046c0 2024-08-31T07:12:18.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.849+0000 7f3065600640 1 -- 172.21.15.32:0/3268085281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f30580037a0 con 0x7f30681046c0 2024-08-31T07:12:18.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.850+0000 7f3065600640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f304c0b4b80 0x7f304c0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:18.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.851+0000 7f3065600640 1 -- 172.21.15.32:0/3268085281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f30580e5970 con 0x7f30681046c0 2024-08-31T07:12:18.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.851+0000 7f305fe00640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f304c0b4b80 0x7f304c0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:18.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.852+0000 7f305fe00640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f304c0b4b80 0x7f304c0b7090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f306810ea00 tx=0x7f3050005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:18.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:18.852+0000 7f3065600640 1 -- 172.21.15.32:0/3268085281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f30580ac9a0 con 0x7f30681046c0 2024-08-31T07:12:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:18 smithi073 ceph-mon[21995]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-31T07:12:18.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:18 smithi084 ceph-mon[22387]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-31T07:12:19.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.048+0000 7f306f000640 1 -- 172.21.15.32:0/3268085281 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f306810fae0 con 0x7f304c0b4b80 2024-08-31T07:12:19.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.053+0000 7f3065600640 1 -- 172.21.15.32:0/3268085281 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259135 (secure 0 0 0) 0x7f306810fae0 con 0x7f304c0b4b80 2024-08-31T07:12:19.058 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:19.059 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 -- 172.21.15.32:0/3268085281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f304c0b4b80 msgr2=0x7f304c0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f304c0b4b80 0x7f304c0b7090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f306810ea00 tx=0x7f3050005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 -- 172.21.15.32:0/3268085281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30681046c0 msgr2=0x7f306819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30681046c0 0x7f306819c330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f305800ea60 tx=0x7f305800ef20 comp rx=0 tx=0).stop 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 -- 172.21.15.32:0/3268085281 shutdown_connections 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f304c0b4b80 0x7f304c0b7090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f306810db20 0x7f30681a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3068105010 0x7f306819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 --2- 172.21.15.32:0/3268085281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30681046c0 0x7f306819c330 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 -- 172.21.15.32:0/3268085281 >> 172.21.15.32:0/3268085281 conn(0x7f30680fe150 msgr2=0x7f306810f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:19.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 -- 172.21.15.32:0/3268085281 shutdown_connections 2024-08-31T07:12:19.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:19.058+0000 7f306f000640 1 -- 172.21.15.32:0/3268085281 wait complete. 2024-08-31T07:12:19.085 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-08-31T07:12:18.798073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665702,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1461,"num_read_kb":843,"num_write":390,"num_write_kb":2309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2952490,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":822,"ondisk_log_size":822,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364216,"kb_used_data":10808,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577608,"statfs":{"total":1151940427776,"available":1151567470592,"internally_reserved":0,"allocated":11067392,"data_stored":7171800,"data_compressed":177834,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":77,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002920"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198399+0000","last_change":"2024-08-31T07:09:47.665725+0000","last_active":"2024-08-31T07:11:45.198399+0000","last_peered":"2024-08-31T07:11:45.198399+0000","last_clean":"2024-08-31T07:11:45.198399+0000","last_became_active":"2024-08-31T07:09:47.665368+0000","last_became_peered":"2024-08-31T07:09:47.665368+0000","last_unstale":"2024-08-31T07:11:45.198399+0000","last_undegraded":"2024-08-31T07:11:45.198399+0000","last_fullsized":"2024-08-31T07:11:45.198399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:27:01.636099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192497+0000","last_change":"2024-08-31T07:09:47.676882+0000","last_active":"2024-08-31T07:11:45.192497+0000","last_peered":"2024-08-31T07:11:45.192497+0000","last_clean":"2024-08-31T07:11:45.192497+0000","last_became_active":"2024-08-31T07:09:47.674825+0000","last_became_peered":"2024-08-31T07:09:47.674825+0000","last_unstale":"2024-08-31T07:11:45.192497+0000","last_undegraded":"2024-08-31T07:11:45.192497+0000","last_fullsized":"2024-08-31T07:11:45.192497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:09.148398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199170+0000","last_change":"2024-08-31T07:09:47.674480+0000","last_active":"2024-08-31T07:11:45.199170+0000","last_peered":"2024-08-31T07:11:45.199170+0000","last_clean":"2024-08-31T07:11:45.199170+0000","last_became_active":"2024-08-31T07:09:47.672316+0000","last_became_peered":"2024-08-31T07:09:47.672316+0000","last_unstale":"2024-08-31T07:11:45.199170+0000","last_undegraded":"2024-08-31T07:11:45.199170+0000","last_fullsized":"2024-08-31T07:11:45.199170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:28:13.231039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.172744+0000","last_change":"2024-08-31T07:09:47.659276+0000","last_active":"2024-08-31T07:11:45.172744+0000","last_peered":"2024-08-31T07:11:45.172744+0000","last_clean":"2024-08-31T07:11:45.172744+0000","last_became_active":"2024-08-31T07:09:47.658924+0000","last_became_peered":"2024-08-31T07:09:47.658924+0000","last_unstale":"2024-08-31T07:11:45.172744+0000","last_undegraded":"2024-08-31T07:11:45.172744+0000","last_fullsized":"2024-08-31T07:11:45.172744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:01:08.134245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.174698+0000","last_change":"2024-08-31T07:09:47.667260+0000","last_active":"2024-08-31T07:11:45.174698+0000","last_peered":"2024-08-31T07:11:45.174698+0000","last_clean":"2024-08-31T07:11:45.174698+0000","last_became_active":"2024-08-31T07:09:47.664572+0000","last_became_peered":"2024-08-31T07:09:47.664572+0000","last_unstale":"2024-08-31T07:11:45.174698+0000","last_undegraded":"2024-08-31T07:11:45.174698+0000","last_fullsized":"2024-08-31T07:11:45.174698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:11.336985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193916+0000","last_change":"2024-08-31T07:09:47.661181+0000","last_active":"2024-08-31T07:11:45.193916+0000","last_peered":"2024-08-31T07:11:45.193916+0000","last_clean":"2024-08-31T07:11:45.193916+0000","last_became_active":"2024-08-31T07:09:47.660761+0000","last_became_peered":"2024-08-31T07:09:47.660761+0000","last_unstale":"2024-08-31T07:11:45.193916+0000","last_undegraded":"2024-08-31T07:11:45.193916+0000","last_fullsized":"2024-08-31T07:11:45.193916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:50:54.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196590+0000","last_change":"2024-08-31T07:09:47.668762+0000","last_active":"2024-08-31T07:11:45.196590+0000","last_peered":"2024-08-31T07:11:45.196590+0000","last_clean":"2024-08-31T07:11:45.196590+0000","last_became_active":"2024-08-31T07:09:47.668406+0000","last_became_peered":"2024-08-31T07:09:47.668406+0000","last_unstale":"2024-08-31T07:11:45.196590+0000","last_undegraded":"2024-08-31T07:11:45.196590+0000","last_fullsized":"2024-08-31T07:11:45.196590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:43:39.868144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.190957+0000","last_change":"2024-08-31T07:09:47.669455+0000","last_active":"2024-08-31T07:11:45.190957+0000","last_peered":"2024-08-31T07:11:45.190957+0000","last_clean":"2024-08-31T07:11:45.190957+0000","last_became_active":"2024-08-31T07:09:47.663669+0000","last_became_peered":"2024-08-31T07:09:47.663669+0000","last_unstale":"2024-08-31T07:11:45.190957+0000","last_undegraded":"2024-08-31T07:11:45.190957+0000","last_fullsized":"2024-08-31T07:11:45.190957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:22:59.685012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194827+0000","last_change":"2024-08-31T07:09:47.664214+0000","last_active":"2024-08-31T07:11:45.194827+0000","last_peered":"2024-08-31T07:11:45.194827+0000","last_clean":"2024-08-31T07:11:45.194827+0000","last_became_active":"2024-08-31T07:09:47.663832+0000","last_became_peered":"2024-08-31T07:09:47.663832+0000","last_unstale":"2024-08-31T07:11:45.194827+0000","last_undegraded":"2024-08-31T07:11:45.194827+0000","last_fullsized":"2024-08-31T07:11:45.194827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:29:14.107212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194866+0000","last_change":"2024-08-31T07:10:46.326410+0000","last_active":"2024-08-31T07:11:45.194866+0000","last_peered":"2024-08-31T07:11:45.194866+0000","last_clean":"2024-08-31T07:11:45.194866+0000","last_became_active":"2024-08-31T07:10:46.325958+0000","last_became_peered":"2024-08-31T07:10:46.325958+0000","last_unstale":"2024-08-31T07:11:45.194866+0000","last_undegraded":"2024-08-31T07:11:45.194866+0000","last_fullsized":"2024-08-31T07:11:45.194866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:19.230393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196579+0000","last_change":"2024-08-31T07:09:47.666951+0000","last_active":"2024-08-31T07:11:45.196579+0000","last_peered":"2024-08-31T07:11:45.196579+0000","last_clean":"2024-08-31T07:11:45.196579+0000","last_became_active":"2024-08-31T07:09:47.666475+0000","last_became_peered":"2024-08-31T07:09:47.666475+0000","last_unstale":"2024-08-31T07:11:45.196579+0000","last_undegraded":"2024-08-31T07:11:45.196579+0000","last_fullsized":"2024-08-31T07:11:45.196579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:58:42.242086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.169998+0000","last_change":"2024-08-31T07:09:47.658344+0000","last_active":"2024-08-31T07:11:45.169998+0000","last_peered":"2024-08-31T07:11:45.169998+0000","last_clean":"2024-08-31T07:11:45.169998+0000","last_became_active":"2024-08-31T07:09:47.657991+0000","last_became_peered":"2024-08-31T07:09:47.657991+0000","last_unstale":"2024-08-31T07:11:45.169998+0000","last_undegraded":"2024-08-31T07:11:45.169998+0000","last_fullsized":"2024-08-31T07:11:45.169998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:25:59.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196676+0000","last_change":"2024-08-31T07:09:47.661644+0000","last_active":"2024-08-31T07:11:45.196676+0000","last_peered":"2024-08-31T07:11:45.196676+0000","last_clean":"2024-08-31T07:11:45.196676+0000","last_became_active":"2024-08-31T07:09:47.661248+0000","last_became_peered":"2024-08-31T07:09:47.661248+0000","last_unstale":"2024-08-31T07:11:45.196676+0000","last_undegraded":"2024-08-31T07:11:45.196676+0000","last_fullsized":"2024-08-31T07:11:45.196676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:42.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195194+0000","last_change":"2024-08-31T07:09:47.680052+0000","last_active":"2024-08-31T07:11:45.195194+0000","last_peered":"2024-08-31T07:11:45.195194+0000","last_clean":"2024-08-31T07:11:45.195194+0000","last_became_active":"2024-08-31T07:09:47.679604+0000","last_became_peered":"2024-08-31T07:09:47.679604+0000","last_unstale":"2024-08-31T07:11:45.195194+0000","last_undegraded":"2024-08-31T07:11:45.195194+0000","last_fullsized":"2024-08-31T07:11:45.195194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:23:18.937470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195477+0000","last_change":"2024-08-31T07:09:47.677984+0000","last_active":"2024-08-31T07:11:45.195477+0000","last_peered":"2024-08-31T07:11:45.195477+0000","last_clean":"2024-08-31T07:11:45.195477+0000","last_became_active":"2024-08-31T07:09:47.674957+0000","last_became_peered":"2024-08-31T07:09:47.674957+0000","last_unstale":"2024-08-31T07:11:45.195477+0000","last_undegraded":"2024-08-31T07:11:45.195477+0000","last_fullsized":"2024-08-31T07:11:45.195477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:08:26.503746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198878+0000","last_change":"2024-08-31T07:09:47.672625+0000","last_active":"2024-08-31T07:11:45.198878+0000","last_peered":"2024-08-31T07:11:45.198878+0000","last_clean":"2024-08-31T07:11:45.198878+0000","last_became_active":"2024-08-31T07:09:47.672184+0000","last_became_peered":"2024-08-31T07:09:47.672184+0000","last_unstale":"2024-08-31T07:11:45.198878+0000","last_undegraded":"2024-08-31T07:11:45.198878+0000","last_fullsized":"2024-08-31T07:11:45.198878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:17.091659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198670+0000","last_change":"2024-08-31T07:09:47.672597+0000","last_active":"2024-08-31T07:11:45.198670+0000","last_peered":"2024-08-31T07:11:45.198670+0000","last_clean":"2024-08-31T07:11:45.198670+0000","last_became_active":"2024-08-31T07:09:47.672153+0000","last_became_peered":"2024-08-31T07:09:47.672153+0000","last_unstale":"2024-08-31T07:11:45.198670+0000","last_undegraded":"2024-08-31T07:11:45.198670+0000","last_fullsized":"2024-08-31T07:11:45.198670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:26:41.388526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193223+0000","last_change":"2024-08-31T07:09:47.666253+0000","last_active":"2024-08-31T07:11:45.193223+0000","last_peered":"2024-08-31T07:11:45.193223+0000","last_clean":"2024-08-31T07:11:45.193223+0000","last_became_active":"2024-08-31T07:09:47.663732+0000","last_became_peered":"2024-08-31T07:09:47.663732+0000","last_unstale":"2024-08-31T07:11:45.193223+0000","last_undegraded":"2024-08-31T07:11:45.193223+0000","last_fullsized":"2024-08-31T07:11:45.193223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:10:59.887765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188202+0000","last_change":"2024-08-31T07:09:47.668502+0000","last_active":"2024-08-31T07:11:45.188202+0000","last_peered":"2024-08-31T07:11:45.188202+0000","last_clean":"2024-08-31T07:11:45.188202+0000","last_became_active":"2024-08-31T07:09:47.666487+0000","last_became_peered":"2024-08-31T07:09:47.666487+0000","last_unstale":"2024-08-31T07:11:45.188202+0000","last_undegraded":"2024-08-31T07:11:45.188202+0000","last_fullsized":"2024-08-31T07:11:45.188202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:49:25.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.184945+0000","last_change":"2024-08-31T07:09:47.659454+0000","last_active":"2024-08-31T07:11:45.184945+0000","last_peered":"2024-08-31T07:11:45.184945+0000","last_clean":"2024-08-31T07:11:45.184945+0000","last_became_active":"2024-08-31T07:09:47.658976+0000","last_became_peered":"2024-08-31T07:09:47.658976+0000","last_unstale":"2024-08-31T07:11:45.184945+0000","last_undegraded":"2024-08-31T07:11:45.184945+0000","last_fullsized":"2024-08-31T07:11:45.184945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:31.374905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199707+0000","last_change":"2024-08-31T07:09:47.673612+0000","last_active":"2024-08-31T07:11:45.199707+0000","last_peered":"2024-08-31T07:11:45.199707+0000","last_clean":"2024-08-31T07:11:45.199707+0000","last_became_active":"2024-08-31T07:09:47.673137+0000","last_became_peered":"2024-08-31T07:09:47.673137+0000","last_unstale":"2024-08-31T07:11:45.199707+0000","last_undegraded":"2024-08-31T07:11:45.199707+0000","last_fullsized":"2024-08-31T07:11:45.199707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:46:11.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194525+0000","last_change":"2024-08-31T07:09:47.669195+0000","last_active":"2024-08-31T07:11:45.194525+0000","last_peered":"2024-08-31T07:11:45.194525+0000","last_clean":"2024-08-31T07:11:45.194525+0000","last_became_active":"2024-08-31T07:09:47.668790+0000","last_became_peered":"2024-08-31T07:09:47.668790+0000","last_unstale":"2024-08-31T07:11:45.194525+0000","last_undegraded":"2024-08-31T07:11:45.194525+0000","last_fullsized":"2024-08-31T07:11:45.194525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:27:14.393931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.181333+0000","last_change":"2024-08-31T07:09:47.667566+0000","last_active":"2024-08-31T07:11:45.181333+0000","last_peered":"2024-08-31T07:11:45.181333+0000","last_clean":"2024-08-31T07:11:45.181333+0000","last_became_active":"2024-08-31T07:09:47.666605+0000","last_became_peered":"2024-08-31T07:09:47.666605+0000","last_unstale":"2024-08-31T07:11:45.181333+0000","last_undegraded":"2024-08-31T07:11:45.181333+0000","last_fullsized":"2024-08-31T07:11:45.181333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:05:44.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192240+0000","last_change":"2024-08-31T07:09:47.660478+0000","last_active":"2024-08-31T07:11:45.192240+0000","last_peered":"2024-08-31T07:11:45.192240+0000","last_clean":"2024-08-31T07:11:45.192240+0000","last_became_active":"2024-08-31T07:09:47.660107+0000","last_became_peered":"2024-08-31T07:09:47.660107+0000","last_unstale":"2024-08-31T07:11:45.192240+0000","last_undegraded":"2024-08-31T07:11:45.192240+0000","last_fullsized":"2024-08-31T07:11:45.192240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:29.133370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188131+0000","last_change":"2024-08-31T07:09:47.671967+0000","last_active":"2024-08-31T07:11:45.188131+0000","last_peered":"2024-08-31T07:11:45.188131+0000","last_clean":"2024-08-31T07:11:45.188131+0000","last_became_active":"2024-08-31T07:09:47.671569+0000","last_became_peered":"2024-08-31T07:09:47.671569+0000","last_unstale":"2024-08-31T07:11:45.188131+0000","last_undegraded":"2024-08-31T07:11:45.188131+0000","last_fullsized":"2024-08-31T07:11:45.188131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:19:15.107958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185867+0000","last_change":"2024-08-31T07:09:47.651055+0000","last_active":"2024-08-31T07:11:45.185867+0000","last_peered":"2024-08-31T07:11:45.185867+0000","last_clean":"2024-08-31T07:11:45.185867+0000","last_became_active":"2024-08-31T07:09:47.650674+0000","last_became_peered":"2024-08-31T07:09:47.650674+0000","last_unstale":"2024-08-31T07:11:45.185867+0000","last_undegraded":"2024-08-31T07:11:45.185867+0000","last_fullsized":"2024-08-31T07:11:45.185867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:35:54.647379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180140+0000","last_change":"2024-08-31T07:09:47.663231+0000","last_active":"2024-08-31T07:11:45.180140+0000","last_peered":"2024-08-31T07:11:45.180140+0000","last_clean":"2024-08-31T07:11:45.180140+0000","last_became_active":"2024-08-31T07:09:47.662854+0000","last_became_peered":"2024-08-31T07:09:47.662854+0000","last_unstale":"2024-08-31T07:11:45.180140+0000","last_undegraded":"2024-08-31T07:11:45.180140+0000","last_fullsized":"2024-08-31T07:11:45.180140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:51.358557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199287+0000","last_change":"2024-08-31T07:10:46.326380+0000","last_active":"2024-08-31T07:11:45.199287+0000","last_peered":"2024-08-31T07:11:45.199287+0000","last_clean":"2024-08-31T07:11:45.199287+0000","last_became_active":"2024-08-31T07:10:46.325784+0000","last_became_peered":"2024-08-31T07:10:46.325784+0000","last_unstale":"2024-08-31T07:11:45.199287+0000","last_undegraded":"2024-08-31T07:11:45.199287+0000","last_fullsized":"2024-08-31T07:11:45.199287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:16:56.396024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200247+0000","last_change":"2024-08-31T07:09:47.672108+0000","last_active":"2024-08-31T07:11:45.200247+0000","last_peered":"2024-08-31T07:11:45.200247+0000","last_clean":"2024-08-31T07:11:45.200247+0000","last_became_active":"2024-08-31T07:09:47.670998+0000","last_became_peered":"2024-08-31T07:09:47.670998+0000","last_unstale":"2024-08-31T07:11:45.200247+0000","last_undegraded":"2024-08-31T07:11:45.200247+0000","last_fullsized":"2024-08-31T07:11:45.200247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:26:49.319861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198226+0000","last_change":"2024-08-31T07:09:47.665548+0000","last_active":"2024-08-31T07:11:45.198226+0000","last_peered":"2024-08-31T07:11:45.198226+0000","last_clean":"2024-08-31T07:11:45.198226+0000","last_became_active":"2024-08-31T07:09:47.665177+0000","last_became_peered":"2024-08-31T07:09:47.665177+0000","last_unstale":"2024-08-31T07:11:45.198226+0000","last_undegraded":"2024-08-31T07:11:45.198226+0000","last_fullsized":"2024-08-31T07:11:45.198226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:03:15.892224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.177785+0000","last_change":"2024-08-31T07:10:46.026746+0000","last_active":"2024-08-31T07:11:45.177785+0000","last_peered":"2024-08-31T07:11:45.177785+0000","last_clean":"2024-08-31T07:11:45.177785+0000","last_became_active":"2024-08-31T07:10:46.026286+0000","last_became_peered":"2024-08-31T07:10:46.026286+0000","last_unstale":"2024-08-31T07:11:45.177785+0000","last_undegraded":"2024-08-31T07:11:45.177785+0000","last_fullsized":"2024-08-31T07:11:45.177785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:34:17.120906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180769+0000","last_change":"2024-08-31T07:09:47.667456+0000","last_active":"2024-08-31T07:11:45.180769+0000","last_peered":"2024-08-31T07:11:45.180769+0000","last_clean":"2024-08-31T07:11:45.180769+0000","last_became_active":"2024-08-31T07:09:47.667102+0000","last_became_peered":"2024-08-31T07:09:47.667102+0000","last_unstale":"2024-08-31T07:11:45.180769+0000","last_undegraded":"2024-08-31T07:11:45.180769+0000","last_fullsized":"2024-08-31T07:11:45.180769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:44:39.747729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.181991+0000","last_change":"2024-08-31T07:10:46.326139+0000","last_active":"2024-08-31T07:11:45.181991+0000","last_peered":"2024-08-31T07:11:45.181991+0000","last_clean":"2024-08-31T07:11:45.181991+0000","last_became_active":"2024-08-31T07:10:46.325785+0000","last_became_peered":"2024-08-31T07:10:46.325785+0000","last_unstale":"2024-08-31T07:11:45.181991+0000","last_undegraded":"2024-08-31T07:11:45.181991+0000","last_fullsized":"2024-08-31T07:11:45.181991+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:13.495835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":170,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:16.094074+0000","last_change":"2024-08-31T07:09:45.641088+0000","last_active":"2024-08-31T07:12:16.094074+0000","last_peered":"2024-08-31T07:12:16.094074+0000","last_clean":"2024-08-31T07:12:16.094074+0000","last_became_active":"2024-08-31T07:09:45.640673+0000","last_became_peered":"2024-08-31T07:09:45.640673+0000","last_unstale":"2024-08-31T07:12:16.094074+0000","last_undegraded":"2024-08-31T07:12:16.094074+0000","last_fullsized":"2024-08-31T07:12:16.094074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:30:17.593494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192254+0000","last_change":"2024-08-31T07:09:47.666703+0000","last_active":"2024-08-31T07:11:45.192254+0000","last_peered":"2024-08-31T07:11:45.192254+0000","last_clean":"2024-08-31T07:11:45.192254+0000","last_became_active":"2024-08-31T07:09:47.666348+0000","last_became_peered":"2024-08-31T07:09:47.666348+0000","last_unstale":"2024-08-31T07:11:45.192254+0000","last_undegraded":"2024-08-31T07:11:45.192254+0000","last_fullsized":"2024-08-31T07:11:45.192254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:39.688810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019885+0000","last_change":"2024-08-31T07:09:45.642431+0000","last_active":"2024-08-31T07:10:46.019885+0000","last_peered":"2024-08-31T07:10:46.019885+0000","last_clean":"2024-08-31T07:10:46.019885+0000","last_became_active":"2024-08-31T07:09:45.642048+0000","last_became_peered":"2024-08-31T07:09:45.642048+0000","last_unstale":"2024-08-31T07:10:46.019885+0000","last_undegraded":"2024-08-31T07:10:46.019885+0000","last_fullsized":"2024-08-31T07:10:46.019885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:47:42.574442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017286+0000","last_change":"2024-08-31T07:09:45.632162+0000","last_active":"2024-08-31T07:10:46.017286+0000","last_peered":"2024-08-31T07:10:46.017286+0000","last_clean":"2024-08-31T07:10:46.017286+0000","last_became_active":"2024-08-31T07:09:45.631749+0000","last_became_peered":"2024-08-31T07:09:45.631749+0000","last_unstale":"2024-08-31T07:10:46.017286+0000","last_undegraded":"2024-08-31T07:10:46.017286+0000","last_fullsized":"2024-08-31T07:10:46.017286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:16:29.967823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185787+0000","last_change":"2024-08-31T07:09:47.666288+0000","last_active":"2024-08-31T07:11:45.185787+0000","last_peered":"2024-08-31T07:11:45.185787+0000","last_clean":"2024-08-31T07:11:45.185787+0000","last_became_active":"2024-08-31T07:09:47.665723+0000","last_became_peered":"2024-08-31T07:09:47.665723+0000","last_unstale":"2024-08-31T07:11:45.185787+0000","last_undegraded":"2024-08-31T07:11:45.185787+0000","last_fullsized":"2024-08-31T07:11:45.185787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:38:28.122720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023120+0000","last_change":"2024-08-31T07:09:45.636508+0000","last_active":"2024-08-31T07:10:46.023120+0000","last_peered":"2024-08-31T07:10:46.023120+0000","last_clean":"2024-08-31T07:10:46.023120+0000","last_became_active":"2024-08-31T07:09:45.636073+0000","last_became_peered":"2024-08-31T07:09:45.636073+0000","last_unstale":"2024-08-31T07:10:46.023120+0000","last_undegraded":"2024-08-31T07:10:46.023120+0000","last_fullsized":"2024-08-31T07:10:46.023120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:55:45.063517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187943+0000","last_change":"2024-08-31T07:09:47.661306+0000","last_active":"2024-08-31T07:11:45.187943+0000","last_peered":"2024-08-31T07:11:45.187943+0000","last_clean":"2024-08-31T07:11:45.187943+0000","last_became_active":"2024-08-31T07:09:47.660957+0000","last_became_peered":"2024-08-31T07:09:47.660957+0000","last_unstale":"2024-08-31T07:11:45.187943+0000","last_undegraded":"2024-08-31T07:11:45.187943+0000","last_fullsized":"2024-08-31T07:11:45.187943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:18:23.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'6","reported_seq":298,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:16.093149+0000","last_change":"2024-08-31T07:09:45.640947+0000","last_active":"2024-08-31T07:12:16.093149+0000","last_peered":"2024-08-31T07:12:16.093149+0000","last_clean":"2024-08-31T07:12:16.093149+0000","last_became_active":"2024-08-31T07:09:45.640532+0000","last_became_peered":"2024-08-31T07:09:45.640532+0000","last_unstale":"2024-08-31T07:12:16.093149+0000","last_undegraded":"2024-08-31T07:12:16.093149+0000","last_fullsized":"2024-08-31T07:12:16.093149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:28.177795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":266,"num_read_kb":266,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.191708+0000","last_change":"2024-08-31T07:09:47.661060+0000","last_active":"2024-08-31T07:11:45.191708+0000","last_peered":"2024-08-31T07:11:45.191708+0000","last_clean":"2024-08-31T07:11:45.191708+0000","last_became_active":"2024-08-31T07:09:47.660692+0000","last_became_peered":"2024-08-31T07:09:47.660692+0000","last_unstale":"2024-08-31T07:11:45.191708+0000","last_undegraded":"2024-08-31T07:11:45.191708+0000","last_fullsized":"2024-08-31T07:11:45.191708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:48:38.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257194+0000","last_change":"2024-08-31T07:09:45.641155+0000","last_active":"2024-08-31T07:10:46.257194+0000","last_peered":"2024-08-31T07:10:46.257194+0000","last_clean":"2024-08-31T07:10:46.257194+0000","last_became_active":"2024-08-31T07:09:45.640680+0000","last_became_peered":"2024-08-31T07:09:45.640680+0000","last_unstale":"2024-08-31T07:10:46.257194+0000","last_undegraded":"2024-08-31T07:10:46.257194+0000","last_fullsized":"2024-08-31T07:10:46.257194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:29:03.287570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199892+0000","last_change":"2024-08-31T07:09:47.675582+0000","last_active":"2024-08-31T07:11:45.199892+0000","last_peered":"2024-08-31T07:11:45.199892+0000","last_clean":"2024-08-31T07:11:45.199892+0000","last_became_active":"2024-08-31T07:09:47.675266+0000","last_became_peered":"2024-08-31T07:09:47.675266+0000","last_unstale":"2024-08-31T07:11:45.199892+0000","last_undegraded":"2024-08-31T07:11:45.199892+0000","last_fullsized":"2024-08-31T07:11:45.199892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:36:33.122531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021194+0000","last_change":"2024-08-31T07:09:45.651006+0000","last_active":"2024-08-31T07:10:46.021194+0000","last_peered":"2024-08-31T07:10:46.021194+0000","last_clean":"2024-08-31T07:10:46.021194+0000","last_became_active":"2024-08-31T07:09:45.650588+0000","last_became_peered":"2024-08-31T07:09:45.650588+0000","last_unstale":"2024-08-31T07:10:46.021194+0000","last_undegraded":"2024-08-31T07:10:46.021194+0000","last_fullsized":"2024-08-31T07:10:46.021194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:34:16.015892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.190030+0000","last_change":"2024-08-31T07:09:47.662289+0000","last_active":"2024-08-31T07:11:45.190030+0000","last_peered":"2024-08-31T07:11:45.190030+0000","last_clean":"2024-08-31T07:11:45.190030+0000","last_became_active":"2024-08-31T07:09:47.661932+0000","last_became_peered":"2024-08-31T07:09:47.661932+0000","last_unstale":"2024-08-31T07:11:45.190030+0000","last_undegraded":"2024-08-31T07:11:45.190030+0000","last_fullsized":"2024-08-31T07:11:45.190030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:50.440504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'37","reported_seq":341,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:16.094196+0000","last_change":"2024-08-31T07:09:45.642196+0000","last_active":"2024-08-31T07:12:16.094196+0000","last_peered":"2024-08-31T07:12:16.094196+0000","last_clean":"2024-08-31T07:12:16.094196+0000","last_became_active":"2024-08-31T07:09:45.641797+0000","last_became_peered":"2024-08-31T07:09:45.641797+0000","last_unstale":"2024-08-31T07:12:16.094196+0000","last_undegraded":"2024-08-31T07:12:16.094196+0000","last_fullsized":"2024-08-31T07:12:16.094196+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:51:05.953335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66550,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":278,"num_read_kb":204,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196022+0000","last_change":"2024-08-31T07:09:47.660184+0000","last_active":"2024-08-31T07:11:45.196022+0000","last_peered":"2024-08-31T07:11:45.196022+0000","last_clean":"2024-08-31T07:11:45.196022+0000","last_became_active":"2024-08-31T07:09:47.659571+0000","last_became_peered":"2024-08-31T07:09:47.659571+0000","last_unstale":"2024-08-31T07:11:45.196022+0000","last_undegraded":"2024-08-31T07:11:45.196022+0000","last_fullsized":"2024-08-31T07:11:45.196022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:41:13.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.035782+0000","last_change":"2024-08-31T07:10:46.035336+0000","last_active":"2024-08-31T07:10:46.035782+0000","last_peered":"2024-08-31T07:10:46.035782+0000","last_clean":"2024-08-31T07:10:46.035782+0000","last_became_active":"2024-08-31T07:10:46.021299+0000","last_became_peered":"2024-08-31T07:10:46.021299+0000","last_unstale":"2024-08-31T07:10:46.035782+0000","last_undegraded":"2024-08-31T07:10:46.035782+0000","last_fullsized":"2024-08-31T07:10:46.035782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:06:15.931855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194680+0000","last_change":"2024-08-31T07:10:46.326354+0000","last_active":"2024-08-31T07:11:45.194680+0000","last_peered":"2024-08-31T07:11:45.194680+0000","last_clean":"2024-08-31T07:11:45.194680+0000","last_became_active":"2024-08-31T07:10:46.325869+0000","last_became_peered":"2024-08-31T07:10:46.325869+0000","last_unstale":"2024-08-31T07:11:45.194680+0000","last_undegraded":"2024-08-31T07:11:45.194680+0000","last_fullsized":"2024-08-31T07:11:45.194680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:21.832088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022972+0000","last_change":"2024-08-31T07:09:45.642187+0000","last_active":"2024-08-31T07:10:46.022972+0000","last_peered":"2024-08-31T07:10:46.022972+0000","last_clean":"2024-08-31T07:10:46.022972+0000","last_became_active":"2024-08-31T07:09:45.641721+0000","last_became_peered":"2024-08-31T07:09:45.641721+0000","last_unstale":"2024-08-31T07:10:46.022972+0000","last_undegraded":"2024-08-31T07:10:46.022972+0000","last_fullsized":"2024-08-31T07:10:46.022972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:00:45.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196019+0000","last_change":"2024-08-31T07:09:47.665956+0000","last_active":"2024-08-31T07:11:45.196019+0000","last_peered":"2024-08-31T07:11:45.196019+0000","last_clean":"2024-08-31T07:11:45.196019+0000","last_became_active":"2024-08-31T07:09:47.664367+0000","last_became_peered":"2024-08-31T07:09:47.664367+0000","last_unstale":"2024-08-31T07:11:45.196019+0000","last_undegraded":"2024-08-31T07:11:45.196019+0000","last_fullsized":"2024-08-31T07:11:45.196019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:34.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'7","reported_seq":170,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:15.091359+0000","last_change":"2024-08-31T07:09:45.636440+0000","last_active":"2024-08-31T07:12:15.091359+0000","last_peered":"2024-08-31T07:12:15.091359+0000","last_clean":"2024-08-31T07:12:15.091359+0000","last_became_active":"2024-08-31T07:09:45.636058+0000","last_became_peered":"2024-08-31T07:09:45.636058+0000","last_unstale":"2024-08-31T07:12:15.091359+0000","last_undegraded":"2024-08-31T07:12:15.091359+0000","last_fullsized":"2024-08-31T07:12:15.091359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:38.929227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194840+0000","last_change":"2024-08-31T07:09:47.660564+0000","last_active":"2024-08-31T07:11:45.194840+0000","last_peered":"2024-08-31T07:11:45.194840+0000","last_clean":"2024-08-31T07:11:45.194840+0000","last_became_active":"2024-08-31T07:09:47.660157+0000","last_became_peered":"2024-08-31T07:09:47.660157+0000","last_unstale":"2024-08-31T07:11:45.194840+0000","last_undegraded":"2024-08-31T07:11:45.194840+0000","last_fullsized":"2024-08-31T07:11:45.194840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:49:01.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020515+0000","last_change":"2024-08-31T07:09:45.649140+0000","last_active":"2024-08-31T07:10:46.020515+0000","last_peered":"2024-08-31T07:10:46.020515+0000","last_clean":"2024-08-31T07:10:46.020515+0000","last_became_active":"2024-08-31T07:09:45.648796+0000","last_became_peered":"2024-08-31T07:09:45.648796+0000","last_unstale":"2024-08-31T07:10:46.020515+0000","last_undegraded":"2024-08-31T07:10:46.020515+0000","last_fullsized":"2024-08-31T07:10:46.020515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:33:37.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192390+0000","last_change":"2024-08-31T07:09:47.663833+0000","last_active":"2024-08-31T07:11:45.192390+0000","last_peered":"2024-08-31T07:11:45.192390+0000","last_clean":"2024-08-31T07:11:45.192390+0000","last_became_active":"2024-08-31T07:09:47.663451+0000","last_became_peered":"2024-08-31T07:09:47.663451+0000","last_unstale":"2024-08-31T07:11:45.192390+0000","last_undegraded":"2024-08-31T07:11:45.192390+0000","last_fullsized":"2024-08-31T07:11:45.192390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:27:22.245254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021407+0000","last_change":"2024-08-31T07:09:45.636593+0000","last_active":"2024-08-31T07:10:46.021407+0000","last_peered":"2024-08-31T07:10:46.021407+0000","last_clean":"2024-08-31T07:10:46.021407+0000","last_became_active":"2024-08-31T07:09:45.636166+0000","last_became_peered":"2024-08-31T07:09:45.636166+0000","last_unstale":"2024-08-31T07:10:46.021407+0000","last_undegraded":"2024-08-31T07:10:46.021407+0000","last_fullsized":"2024-08-31T07:10:46.021407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:28.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183385+0000","last_change":"2024-08-31T07:09:47.660825+0000","last_active":"2024-08-31T07:11:45.183385+0000","last_peered":"2024-08-31T07:11:45.183385+0000","last_clean":"2024-08-31T07:11:45.183385+0000","last_became_active":"2024-08-31T07:09:47.660474+0000","last_became_peered":"2024-08-31T07:09:47.660474+0000","last_unstale":"2024-08-31T07:11:45.183385+0000","last_undegraded":"2024-08-31T07:11:45.183385+0000","last_fullsized":"2024-08-31T07:11:45.183385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:42:20.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017462+0000","last_change":"2024-08-31T07:09:45.632727+0000","last_active":"2024-08-31T07:10:46.017462+0000","last_peered":"2024-08-31T07:10:46.017462+0000","last_clean":"2024-08-31T07:10:46.017462+0000","last_became_active":"2024-08-31T07:09:45.632251+0000","last_became_peered":"2024-08-31T07:09:45.632251+0000","last_unstale":"2024-08-31T07:10:46.017462+0000","last_undegraded":"2024-08-31T07:10:46.017462+0000","last_fullsized":"2024-08-31T07:10:46.017462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:09:30.197409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196430+0000","last_change":"2024-08-31T07:09:47.665501+0000","last_active":"2024-08-31T07:11:45.196430+0000","last_peered":"2024-08-31T07:11:45.196430+0000","last_clean":"2024-08-31T07:11:45.196430+0000","last_became_active":"2024-08-31T07:09:47.665162+0000","last_became_peered":"2024-08-31T07:09:47.665162+0000","last_unstale":"2024-08-31T07:11:45.196430+0000","last_undegraded":"2024-08-31T07:11:45.196430+0000","last_fullsized":"2024-08-31T07:11:45.196430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:34:29.605836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021320+0000","last_change":"2024-08-31T07:09:45.636580+0000","last_active":"2024-08-31T07:10:46.021320+0000","last_peered":"2024-08-31T07:10:46.021320+0000","last_clean":"2024-08-31T07:10:46.021320+0000","last_became_active":"2024-08-31T07:09:45.636163+0000","last_became_peered":"2024-08-31T07:09:45.636163+0000","last_unstale":"2024-08-31T07:10:46.021320+0000","last_undegraded":"2024-08-31T07:10:46.021320+0000","last_fullsized":"2024-08-31T07:10:46.021320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:18:07.567858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195120+0000","last_change":"2024-08-31T07:09:47.671865+0000","last_active":"2024-08-31T07:11:45.195120+0000","last_peered":"2024-08-31T07:11:45.195120+0000","last_clean":"2024-08-31T07:11:45.195120+0000","last_became_active":"2024-08-31T07:09:47.671454+0000","last_became_peered":"2024-08-31T07:09:47.671454+0000","last_unstale":"2024-08-31T07:11:45.195120+0000","last_undegraded":"2024-08-31T07:11:45.195120+0000","last_fullsized":"2024-08-31T07:11:45.195120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:47.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'7","reported_seq":168,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:13.088891+0000","last_change":"2024-08-31T07:09:45.653217+0000","last_active":"2024-08-31T07:12:13.088891+0000","last_peered":"2024-08-31T07:12:13.088891+0000","last_clean":"2024-08-31T07:12:13.088891+0000","last_became_active":"2024-08-31T07:09:45.652819+0000","last_became_peered":"2024-08-31T07:09:45.652819+0000","last_unstale":"2024-08-31T07:12:13.088891+0000","last_undegraded":"2024-08-31T07:12:13.088891+0000","last_fullsized":"2024-08-31T07:12:13.088891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:25:11.454068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196166+0000","last_change":"2024-08-31T07:09:47.663652+0000","last_active":"2024-08-31T07:11:45.196166+0000","last_peered":"2024-08-31T07:11:45.196166+0000","last_clean":"2024-08-31T07:11:45.196166+0000","last_became_active":"2024-08-31T07:09:47.663274+0000","last_became_peered":"2024-08-31T07:09:47.663274+0000","last_unstale":"2024-08-31T07:11:45.196166+0000","last_undegraded":"2024-08-31T07:11:45.196166+0000","last_fullsized":"2024-08-31T07:11:45.196166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:31:59.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257370+0000","last_change":"2024-08-31T07:08:27.623298+0000","last_active":"2024-08-31T07:10:46.257370+0000","last_peered":"2024-08-31T07:10:46.257370+0000","last_clean":"2024-08-31T07:10:46.257370+0000","last_became_active":"2024-08-31T07:08:27.619072+0000","last_became_peered":"2024-08-31T07:08:27.619072+0000","last_unstale":"2024-08-31T07:10:46.257370+0000","last_undegraded":"2024-08-31T07:10:46.257370+0000","last_fullsized":"2024-08-31T07:10:46.257370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_clean_scrub_stamp":"2024-08-31T07:04:45.509398+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:07.967421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154319+0000","last_change":"2024-08-31T07:09:45.638104+0000","last_active":"2024-08-31T07:10:46.154319+0000","last_peered":"2024-08-31T07:10:46.154319+0000","last_clean":"2024-08-31T07:10:46.154319+0000","last_became_active":"2024-08-31T07:09:45.637693+0000","last_became_peered":"2024-08-31T07:09:45.637693+0000","last_unstale":"2024-08-31T07:10:46.154319+0000","last_undegraded":"2024-08-31T07:10:46.154319+0000","last_fullsized":"2024-08-31T07:10:46.154319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:50:15.135061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.189353+0000","last_change":"2024-08-31T07:09:47.664958+0000","last_active":"2024-08-31T07:11:45.189353+0000","last_peered":"2024-08-31T07:11:45.189353+0000","last_clean":"2024-08-31T07:11:45.189353+0000","last_became_active":"2024-08-31T07:09:47.661166+0000","last_became_peered":"2024-08-31T07:09:47.661166+0000","last_unstale":"2024-08-31T07:11:45.189353+0000","last_undegraded":"2024-08-31T07:11:45.189353+0000","last_fullsized":"2024-08-31T07:11:45.189353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:45:58.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257184+0000","last_change":"2024-08-31T07:09:45.640881+0000","last_active":"2024-08-31T07:10:46.257184+0000","last_peered":"2024-08-31T07:10:46.257184+0000","last_clean":"2024-08-31T07:10:46.257184+0000","last_became_active":"2024-08-31T07:09:45.640487+0000","last_became_peered":"2024-08-31T07:09:45.640487+0000","last_unstale":"2024-08-31T07:10:46.257184+0000","last_undegraded":"2024-08-31T07:10:46.257184+0000","last_fullsized":"2024-08-31T07:10:46.257184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:06.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.171981+0000","last_change":"2024-08-31T07:09:47.655008+0000","last_active":"2024-08-31T07:11:45.171981+0000","last_peered":"2024-08-31T07:11:45.171981+0000","last_clean":"2024-08-31T07:11:45.171981+0000","last_became_active":"2024-08-31T07:09:47.654645+0000","last_became_peered":"2024-08-31T07:09:47.654645+0000","last_unstale":"2024-08-31T07:11:45.171981+0000","last_undegraded":"2024-08-31T07:11:45.171981+0000","last_fullsized":"2024-08-31T07:11:45.171981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:23:55.916053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'6","reported_seq":168,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:17.093741+0000","last_change":"2024-08-31T07:09:45.649324+0000","last_active":"2024-08-31T07:12:17.093741+0000","last_peered":"2024-08-31T07:12:17.093741+0000","last_clean":"2024-08-31T07:12:17.093741+0000","last_became_active":"2024-08-31T07:09:45.648915+0000","last_became_peered":"2024-08-31T07:09:45.648915+0000","last_unstale":"2024-08-31T07:12:17.093741+0000","last_undegraded":"2024-08-31T07:12:17.093741+0000","last_fullsized":"2024-08-31T07:12:17.093741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:31:57.853654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187160+0000","last_change":"2024-08-31T07:09:47.661328+0000","last_active":"2024-08-31T07:11:45.187160+0000","last_peered":"2024-08-31T07:11:45.187160+0000","last_clean":"2024-08-31T07:11:45.187160+0000","last_became_active":"2024-08-31T07:09:47.660927+0000","last_became_peered":"2024-08-31T07:09:47.660927+0000","last_unstale":"2024-08-31T07:11:45.187160+0000","last_undegraded":"2024-08-31T07:11:45.187160+0000","last_fullsized":"2024-08-31T07:11:45.187160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:40:46.876894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'7","reported_seq":171,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:17.095410+0000","last_change":"2024-08-31T07:09:45.637747+0000","last_active":"2024-08-31T07:12:17.095410+0000","last_peered":"2024-08-31T07:12:17.095410+0000","last_clean":"2024-08-31T07:12:17.095410+0000","last_became_active":"2024-08-31T07:09:45.637368+0000","last_became_peered":"2024-08-31T07:09:45.637368+0000","last_unstale":"2024-08-31T07:12:17.095410+0000","last_undegraded":"2024-08-31T07:12:17.095410+0000","last_fullsized":"2024-08-31T07:12:17.095410+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:01.174109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193217+0000","last_change":"2024-08-31T07:09:47.680802+0000","last_active":"2024-08-31T07:11:45.193217+0000","last_peered":"2024-08-31T07:11:45.193217+0000","last_clean":"2024-08-31T07:11:45.193217+0000","last_became_active":"2024-08-31T07:09:47.680363+0000","last_became_peered":"2024-08-31T07:09:47.680363+0000","last_unstale":"2024-08-31T07:11:45.193217+0000","last_undegraded":"2024-08-31T07:11:45.193217+0000","last_fullsized":"2024-08-31T07:11:45.193217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:46:48.744071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324353+0000","last_change":"2024-08-31T07:09:45.651222+0000","last_active":"2024-08-31T07:10:46.324353+0000","last_peered":"2024-08-31T07:10:46.324353+0000","last_clean":"2024-08-31T07:10:46.324353+0000","last_became_active":"2024-08-31T07:09:45.650760+0000","last_became_peered":"2024-08-31T07:09:45.650760+0000","last_unstale":"2024-08-31T07:10:46.324353+0000","last_undegraded":"2024-08-31T07:10:46.324353+0000","last_fullsized":"2024-08-31T07:10:46.324353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T19:03:33.942165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183968+0000","last_change":"2024-08-31T07:09:47.665881+0000","last_active":"2024-08-31T07:11:45.183968+0000","last_peered":"2024-08-31T07:11:45.183968+0000","last_clean":"2024-08-31T07:11:45.183968+0000","last_became_active":"2024-08-31T07:09:47.665485+0000","last_became_peered":"2024-08-31T07:09:47.665485+0000","last_unstale":"2024-08-31T07:11:45.183968+0000","last_undegraded":"2024-08-31T07:11:45.183968+0000","last_fullsized":"2024-08-31T07:11:45.183968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:30.820714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020851+0000","last_change":"2024-08-31T07:09:45.651893+0000","last_active":"2024-08-31T07:10:46.020851+0000","last_peered":"2024-08-31T07:10:46.020851+0000","last_clean":"2024-08-31T07:10:46.020851+0000","last_became_active":"2024-08-31T07:09:45.651547+0000","last_became_peered":"2024-08-31T07:09:45.651547+0000","last_unstale":"2024-08-31T07:10:46.020851+0000","last_undegraded":"2024-08-31T07:10:46.020851+0000","last_fullsized":"2024-08-31T07:10:46.020851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:11:30.708018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.175560+0000","last_change":"2024-08-31T07:09:47.655551+0000","last_active":"2024-08-31T07:11:45.175560+0000","last_peered":"2024-08-31T07:11:45.175560+0000","last_clean":"2024-08-31T07:11:45.175560+0000","last_became_active":"2024-08-31T07:09:47.655188+0000","last_became_peered":"2024-08-31T07:09:47.655188+0000","last_unstale":"2024-08-31T07:11:45.175560+0000","last_undegraded":"2024-08-31T07:11:45.175560+0000","last_fullsized":"2024-08-31T07:11:45.175560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:43:51.419599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'17","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:55.669546+0000","last_change":"2024-08-31T07:09:45.636212+0000","last_active":"2024-08-31T07:11:55.669546+0000","last_peered":"2024-08-31T07:11:55.669546+0000","last_clean":"2024-08-31T07:11:55.669546+0000","last_became_active":"2024-08-31T07:09:45.635678+0000","last_became_peered":"2024-08-31T07:09:45.635678+0000","last_unstale":"2024-08-31T07:11:55.669546+0000","last_undegraded":"2024-08-31T07:11:55.669546+0000","last_fullsized":"2024-08-31T07:11:55.669546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:13:17.803459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185987+0000","last_change":"2024-08-31T07:09:47.666476+0000","last_active":"2024-08-31T07:11:45.185987+0000","last_peered":"2024-08-31T07:11:45.185987+0000","last_clean":"2024-08-31T07:11:45.185987+0000","last_became_active":"2024-08-31T07:09:47.666111+0000","last_became_peered":"2024-08-31T07:09:47.666111+0000","last_unstale":"2024-08-31T07:11:45.185987+0000","last_undegraded":"2024-08-31T07:11:45.185987+0000","last_fullsized":"2024-08-31T07:11:45.185987+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:08:53.459549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154441+0000","last_change":"2024-08-31T07:09:45.631571+0000","last_active":"2024-08-31T07:10:46.154441+0000","last_peered":"2024-08-31T07:10:46.154441+0000","last_clean":"2024-08-31T07:10:46.154441+0000","last_became_active":"2024-08-31T07:09:45.630981+0000","last_became_peered":"2024-08-31T07:09:45.630981+0000","last_unstale":"2024-08-31T07:10:46.154441+0000","last_undegraded":"2024-08-31T07:10:46.154441+0000","last_fullsized":"2024-08-31T07:10:46.154441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:36:53.482955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193936+0000","last_change":"2024-08-31T07:09:47.675054+0000","last_active":"2024-08-31T07:11:45.193936+0000","last_peered":"2024-08-31T07:11:45.193936+0000","last_clean":"2024-08-31T07:11:45.193936+0000","last_became_active":"2024-08-31T07:09:47.672652+0000","last_became_peered":"2024-08-31T07:09:47.672652+0000","last_unstale":"2024-08-31T07:11:45.193936+0000","last_undegraded":"2024-08-31T07:11:45.193936+0000","last_fullsized":"2024-08-31T07:11:45.193936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:32:05.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016284+0000","last_change":"2024-08-31T07:09:45.648746+0000","last_active":"2024-08-31T07:10:46.016284+0000","last_peered":"2024-08-31T07:10:46.016284+0000","last_clean":"2024-08-31T07:10:46.016284+0000","last_became_active":"2024-08-31T07:09:45.648424+0000","last_became_peered":"2024-08-31T07:09:45.648424+0000","last_unstale":"2024-08-31T07:10:46.016284+0000","last_undegraded":"2024-08-31T07:10:46.016284+0000","last_fullsized":"2024-08-31T07:10:46.016284+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:40:32.132963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.165179+0000","last_change":"2024-08-31T07:09:47.663213+0000","last_active":"2024-08-31T07:11:45.165179+0000","last_peered":"2024-08-31T07:11:45.165179+0000","last_clean":"2024-08-31T07:11:45.165179+0000","last_became_active":"2024-08-31T07:09:47.662457+0000","last_became_peered":"2024-08-31T07:09:47.662457+0000","last_unstale":"2024-08-31T07:11:45.165179+0000","last_undegraded":"2024-08-31T07:11:45.165179+0000","last_fullsized":"2024-08-31T07:11:45.165179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:54:27.069200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015653+0000","last_change":"2024-08-31T07:09:45.640149+0000","last_active":"2024-08-31T07:10:46.015653+0000","last_peered":"2024-08-31T07:10:46.015653+0000","last_clean":"2024-08-31T07:10:46.015653+0000","last_became_active":"2024-08-31T07:09:45.639581+0000","last_became_peered":"2024-08-31T07:09:45.639581+0000","last_unstale":"2024-08-31T07:10:46.015653+0000","last_undegraded":"2024-08-31T07:10:46.015653+0000","last_fullsized":"2024-08-31T07:10:46.015653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:27:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198479+0000","last_change":"2024-08-31T07:09:47.672613+0000","last_active":"2024-08-31T07:11:45.198479+0000","last_peered":"2024-08-31T07:11:45.198479+0000","last_clean":"2024-08-31T07:11:45.198479+0000","last_became_active":"2024-08-31T07:09:47.672173+0000","last_became_peered":"2024-08-31T07:09:47.672173+0000","last_unstale":"2024-08-31T07:11:45.198479+0000","last_undegraded":"2024-08-31T07:11:45.198479+0000","last_fullsized":"2024-08-31T07:11:45.198479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:49.706113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022590+0000","last_change":"2024-08-31T07:09:45.637567+0000","last_active":"2024-08-31T07:10:46.022590+0000","last_peered":"2024-08-31T07:10:46.022590+0000","last_clean":"2024-08-31T07:10:46.022590+0000","last_became_active":"2024-08-31T07:09:45.637180+0000","last_became_peered":"2024-08-31T07:09:45.637180+0000","last_unstale":"2024-08-31T07:10:46.022590+0000","last_undegraded":"2024-08-31T07:10:46.022590+0000","last_fullsized":"2024-08-31T07:10:46.022590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:48:07.190829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194363+0000","last_change":"2024-08-31T07:09:47.674980+0000","last_active":"2024-08-31T07:11:45.194363+0000","last_peered":"2024-08-31T07:11:45.194363+0000","last_clean":"2024-08-31T07:11:45.194363+0000","last_became_active":"2024-08-31T07:09:47.674528+0000","last_became_peered":"2024-08-31T07:09:47.674528+0000","last_unstale":"2024-08-31T07:11:45.194363+0000","last_undegraded":"2024-08-31T07:11:45.194363+0000","last_fullsized":"2024-08-31T07:11:45.194363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:15:09.464221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021096+0000","last_change":"2024-08-31T07:09:45.641167+0000","last_active":"2024-08-31T07:10:46.021096+0000","last_peered":"2024-08-31T07:10:46.021096+0000","last_clean":"2024-08-31T07:10:46.021096+0000","last_became_active":"2024-08-31T07:09:45.640761+0000","last_became_peered":"2024-08-31T07:09:45.640761+0000","last_unstale":"2024-08-31T07:10:46.021096+0000","last_undegraded":"2024-08-31T07:10:46.021096+0000","last_fullsized":"2024-08-31T07:10:46.021096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:28:59.963145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195596+0000","last_change":"2024-08-31T07:09:47.668808+0000","last_active":"2024-08-31T07:11:45.195596+0000","last_peered":"2024-08-31T07:11:45.195596+0000","last_clean":"2024-08-31T07:11:45.195596+0000","last_became_active":"2024-08-31T07:09:47.668291+0000","last_became_peered":"2024-08-31T07:09:47.668291+0000","last_unstale":"2024-08-31T07:11:45.195596+0000","last_undegraded":"2024-08-31T07:11:45.195596+0000","last_fullsized":"2024-08-31T07:11:45.195596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:23.433511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192982+0000","last_change":"2024-08-31T07:09:47.668039+0000","last_active":"2024-08-31T07:11:45.192982+0000","last_peered":"2024-08-31T07:11:45.192982+0000","last_clean":"2024-08-31T07:11:45.192982+0000","last_became_active":"2024-08-31T07:09:47.667567+0000","last_became_peered":"2024-08-31T07:09:47.667567+0000","last_unstale":"2024-08-31T07:11:45.192982+0000","last_undegraded":"2024-08-31T07:11:45.192982+0000","last_fullsized":"2024-08-31T07:11:45.192982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:33:50.192864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.041708+0000","last_change":"2024-08-31T07:10:46.041197+0000","last_active":"2024-08-31T07:10:46.041708+0000","last_peered":"2024-08-31T07:10:46.041708+0000","last_clean":"2024-08-31T07:10:46.041708+0000","last_became_active":"2024-08-31T07:10:46.023966+0000","last_became_peered":"2024-08-31T07:10:46.023966+0000","last_unstale":"2024-08-31T07:10:46.041708+0000","last_undegraded":"2024-08-31T07:10:46.041708+0000","last_fullsized":"2024-08-31T07:10:46.041708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:34.302665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192551+0000","last_change":"2024-08-31T07:09:47.660615+0000","last_active":"2024-08-31T07:11:45.192551+0000","last_peered":"2024-08-31T07:11:45.192551+0000","last_clean":"2024-08-31T07:11:45.192551+0000","last_became_active":"2024-08-31T07:09:47.660269+0000","last_became_peered":"2024-08-31T07:09:47.660269+0000","last_unstale":"2024-08-31T07:11:45.192551+0000","last_undegraded":"2024-08-31T07:11:45.192551+0000","last_fullsized":"2024-08-31T07:11:45.192551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:42:12.618039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020701+0000","last_change":"2024-08-31T07:09:45.641699+0000","last_active":"2024-08-31T07:10:46.020701+0000","last_peered":"2024-08-31T07:10:46.020701+0000","last_clean":"2024-08-31T07:10:46.020701+0000","last_became_active":"2024-08-31T07:09:45.641122+0000","last_became_peered":"2024-08-31T07:09:45.641122+0000","last_unstale":"2024-08-31T07:10:46.020701+0000","last_undegraded":"2024-08-31T07:10:46.020701+0000","last_fullsized":"2024-08-31T07:10:46.020701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:26:07.130266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200241+0000","last_change":"2024-08-31T07:09:47.675459+0000","last_active":"2024-08-31T07:11:45.200241+0000","last_peered":"2024-08-31T07:11:45.200241+0000","last_clean":"2024-08-31T07:11:45.200241+0000","last_became_active":"2024-08-31T07:09:47.672246+0000","last_became_peered":"2024-08-31T07:09:47.672246+0000","last_unstale":"2024-08-31T07:11:45.200241+0000","last_undegraded":"2024-08-31T07:11:45.200241+0000","last_fullsized":"2024-08-31T07:11:45.200241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:49:23.547352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016428+0000","last_change":"2024-08-31T07:09:45.649522+0000","last_active":"2024-08-31T07:10:46.016428+0000","last_peered":"2024-08-31T07:10:46.016428+0000","last_clean":"2024-08-31T07:10:46.016428+0000","last_became_active":"2024-08-31T07:09:45.649109+0000","last_became_peered":"2024-08-31T07:09:45.649109+0000","last_unstale":"2024-08-31T07:10:46.016428+0000","last_undegraded":"2024-08-31T07:10:46.016428+0000","last_fullsized":"2024-08-31T07:10:46.016428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:52:52.432605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.179375+0000","last_change":"2024-08-31T07:09:47.670583+0000","last_active":"2024-08-31T07:11:45.179375+0000","last_peered":"2024-08-31T07:11:45.179375+0000","last_clean":"2024-08-31T07:11:45.179375+0000","last_became_active":"2024-08-31T07:09:47.670162+0000","last_became_peered":"2024-08-31T07:09:47.670162+0000","last_unstale":"2024-08-31T07:11:45.179375+0000","last_undegraded":"2024-08-31T07:11:45.179375+0000","last_fullsized":"2024-08-31T07:11:45.179375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:35:35.135678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'14","reported_seq":173,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:14.089952+0000","last_change":"2024-08-31T07:09:45.639636+0000","last_active":"2024-08-31T07:12:14.089952+0000","last_peered":"2024-08-31T07:12:14.089952+0000","last_clean":"2024-08-31T07:12:14.089952+0000","last_became_active":"2024-08-31T07:09:45.639029+0000","last_became_peered":"2024-08-31T07:09:45.639029+0000","last_unstale":"2024-08-31T07:12:14.089952+0000","last_undegraded":"2024-08-31T07:12:14.089952+0000","last_fullsized":"2024-08-31T07:12:14.089952+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:54.110149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75100,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1380,"num_read_kb":774,"num_write":265,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":650,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":214554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285320,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678060,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":434494,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874289,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1091423,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037814,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":433242,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234045,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":433038,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397570,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1027343,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561094,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":433491,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757326,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1025957,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":433352,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117079,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432784,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378717,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":496265,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542243,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":495581,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705767,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":434830,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T07:12:19.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:12:19.513 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:19.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:19 smithi032 ceph-mon[17252]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:19.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:19 smithi032 ceph-mon[17252]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:19.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:19 smithi073 ceph-mon[21995]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:19.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:19 smithi073 ceph-mon[21995]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:19.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:19 smithi084 ceph-mon[22387]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:19.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:19 smithi084 ceph-mon[22387]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:20.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.206+0000 7fa4e3200640 1 -- 172.21.15.32:0/3173016068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e41099e0 msgr2=0x7fa4e4110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:20.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.206+0000 7fa4e3200640 1 --2- 172.21.15.32:0/3173016068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e41099e0 0x7fa4e4110210 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009ca0 tx=0x7fa4dc02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.207+0000 7fa4e3200640 1 -- 172.21.15.32:0/3173016068 shutdown_connections 2024-08-31T07:12:20.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.207+0000 7fa4e3200640 1 --2- 172.21.15.32:0/3173016068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e41099e0 0x7fa4e4110210 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.207+0000 7fa4e3200640 1 --2- 172.21.15.32:0/3173016068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e4109090 0x7fa4e41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.207+0000 7fa4e3200640 1 --2- 172.21.15.32:0/3173016068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4e4103750 0x7fa4e4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.207+0000 7fa4e3200640 1 -- 172.21.15.32:0/3173016068 >> 172.21.15.32:0/3173016068 conn(0x7fa4e40ff1e0 msgr2=0x7fa4e4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:20.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.207+0000 7fa4e3200640 1 -- 172.21.15.32:0/3173016068 shutdown_connections 2024-08-31T07:12:20.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.207+0000 7fa4e3200640 1 -- 172.21.15.32:0/3173016068 wait complete. 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.208+0000 7fa4e3200640 1 Processor -- start 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.208+0000 7fa4e3200640 1 -- start start 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.208+0000 7fa4e3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e4103750 0x7fa4e419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.208+0000 7fa4e3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e4109090 0x7fa4e419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.208+0000 7fa4e3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4e41099e0 0x7fa4e419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.208+0000 7fa4e3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4113700 con 0x7fa4e4109090 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.208+0000 7fa4e3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4113580 con 0x7fa4e41099e0 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.208+0000 7fa4e3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4113880 con 0x7fa4e4103750 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.209+0000 7fa4e1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e4103750 0x7fa4e419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.209+0000 7fa4e1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e4103750 0x7fa4e419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38948/0 (socket says 172.21.15.32:38948) 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.209+0000 7fa4e1e00640 1 -- 172.21.15.32:0/2270056932 learned_addr learned my addr 172.21.15.32:0/2270056932 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:20.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.209+0000 7fa4e1400640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e4109090 0x7fa4e419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.209+0000 7fa4e2800640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4e41099e0 0x7fa4e419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.209+0000 7fa4e1e00640 1 -- 172.21.15.32:0/2270056932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4e41099e0 msgr2=0x7fa4e419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.209+0000 7fa4e1e00640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4e41099e0 0x7fa4e419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.209+0000 7fa4e1e00640 1 -- 172.21.15.32:0/2270056932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e4109090 msgr2=0x7fa4e419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.209+0000 7fa4e1e00640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e4109090 0x7fa4e419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.209+0000 7fa4e1e00640 1 -- 172.21.15.32:0/2270056932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e419ffe0 con 0x7fa4e4103750 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.210+0000 7fa4e1e00640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e4103750 0x7fa4e419a070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400e9f0 tx=0x7fa4d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.210+0000 7fa4daa00640 1 -- 172.21.15.32:0/2270056932 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400cd80 con 0x7fa4e4103750 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.210+0000 7fa4e3200640 1 -- 172.21.15.32:0/2270056932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e41a02d0 con 0x7fa4e4103750 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.210+0000 7fa4daa00640 1 -- 172.21.15.32:0/2270056932 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d40108e0 con 0x7fa4e4103750 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.210+0000 7fa4daa00640 1 -- 172.21.15.32:0/2270056932 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4018980 con 0x7fa4e4103750 2024-08-31T07:12:20.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.210+0000 7fa4e3200640 1 -- 172.21.15.32:0/2270056932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e41a6a30 con 0x7fa4e4103750 2024-08-31T07:12:20.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.211+0000 7fa4e3200640 1 -- 172.21.15.32:0/2270056932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e4103b60 con 0x7fa4e4103750 2024-08-31T07:12:20.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.214+0000 7fa4daa00640 1 -- 172.21.15.32:0/2270056932 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7fa4d4010430 con 0x7fa4e4103750 2024-08-31T07:12:20.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.215+0000 7fa4daa00640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4c80b4b80 0x7fa4c80b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:20.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.215+0000 7fa4daa00640 1 -- 172.21.15.32:0/2270056932 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fa4d40e5a90 con 0x7fa4e4103750 2024-08-31T07:12:20.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.215+0000 7fa4e1400640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4c80b4b80 0x7fa4c80b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:20.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.216+0000 7fa4e1400640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4c80b4b80 0x7fa4c80b7090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4102ac0 tx=0x7fa4d0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:20.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.217+0000 7fa4daa00640 1 -- 172.21.15.32:0/2270056932 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa4d40ac470 con 0x7fa4e4103750 2024-08-31T07:12:20.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.450+0000 7fa4e3200640 1 -- 172.21.15.32:0/2270056932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4e402e430 con 0x7fa4e4103750 2024-08-31T07:12:20.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.451+0000 7fa4daa00640 1 -- 172.21.15.32:0/2270056932 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7fa4d40b0c30 con 0x7fa4e4103750 2024-08-31T07:12:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:20.454 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:12:20.454 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:12:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 -- 172.21.15.32:0/2270056932 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4c80b4b80 msgr2=0x7fa4c80b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4c80b4b80 0x7fa4c80b7090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4102ac0 tx=0x7fa4d0005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 -- 172.21.15.32:0/2270056932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e4103750 msgr2=0x7fa4e419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e4103750 0x7fa4e419a070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400e9f0 tx=0x7fa4d400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 -- 172.21.15.32:0/2270056932 shutdown_connections 2024-08-31T07:12:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4c80b4b80 0x7fa4c80b7090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4e41099e0 0x7fa4e419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e4109090 0x7fa4e419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 --2- 172.21.15.32:0/2270056932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e4103750 0x7fa4e419a070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 -- 172.21.15.32:0/2270056932 >> 172.21.15.32:0/2270056932 conn(0x7fa4e40ff1e0 msgr2=0x7fa4e4108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 -- 172.21.15.32:0/2270056932 shutdown_connections 2024-08-31T07:12:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:20.454+0000 7fa4e3200640 1 -- 172.21.15.32:0/2270056932 wait complete. 2024-08-31T07:12:20.809 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:20 smithi032 ceph-mon[17252]: pgmap v355: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-31T07:12:20.809 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:20 smithi032 ceph-mon[17252]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:20.809 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:20 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2270056932' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:20.892 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:20 smithi084 ceph-mon[22387]: pgmap v355: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-31T07:12:20.892 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:20 smithi084 ceph-mon[22387]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:20.892 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:20 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2270056932' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:20 smithi073 ceph-mon[21995]: pgmap v355: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-31T07:12:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:20 smithi073 ceph-mon[21995]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:20 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2270056932' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:21.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:12:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.241+0000 7f5377000640 1 -- 172.21.15.32:0/1068287380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5370103900 msgr2=0x7f5370103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.241+0000 7f5377000640 1 --2- 172.21.15.32:0/1068287380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5370103900 0x7f5370103d10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5360009900 tx=0x7f536002a410 comp rx=0 tx=0).stop 2024-08-31T07:12:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.241+0000 7f5377000640 1 -- 172.21.15.32:0/1068287380 shutdown_connections 2024-08-31T07:12:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.241+0000 7f5377000640 1 --2- 172.21.15.32:0/1068287380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5370109b90 0x7f53701103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.241+0000 7f5377000640 1 --2- 172.21.15.32:0/1068287380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5370109240 0x7f5370109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.241+0000 7f5377000640 1 --2- 172.21.15.32:0/1068287380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5370103900 0x7f5370103d10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.241+0000 7f5377000640 1 -- 172.21.15.32:0/1068287380 >> 172.21.15.32:0/1068287380 conn(0x7f537007b5f0 msgr2=0x7f537007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.241+0000 7f5377000640 1 -- 172.21.15.32:0/1068287380 shutdown_connections 2024-08-31T07:12:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.241+0000 7f5377000640 1 -- 172.21.15.32:0/1068287380 wait complete. 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.242+0000 7f5377000640 1 Processor -- start 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.242+0000 7f5377000640 1 -- start start 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.242+0000 7f5377000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5370103900 0x7f537019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.242+0000 7f5377000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5370109240 0x7f537019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.242+0000 7f5377000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5370109b90 0x7f537019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.242+0000 7f5377000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5370113850 con 0x7f5370109b90 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.242+0000 7f5377000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f53701136d0 con 0x7f5370103900 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.242+0000 7f5377000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f53701139d0 con 0x7f5370109240 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.243+0000 7f536fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5370103900 0x7f537019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.243+0000 7f536fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5370103900 0x7f537019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57152/0 (socket says 172.21.15.32:57152) 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.243+0000 7f536fe00640 1 -- 172.21.15.32:0/823159239 learned_addr learned my addr 172.21.15.32:0/823159239 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.243+0000 7f5375000640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5370109b90 0x7f537019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.243+0000 7f536f400640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5370109240 0x7f537019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.243+0000 7f536fe00640 1 -- 172.21.15.32:0/823159239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5370109240 msgr2=0x7f537019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.243+0000 7f536fe00640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5370109240 0x7f537019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.243+0000 7f536fe00640 1 -- 172.21.15.32:0/823159239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5370109b90 msgr2=0x7f537019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.243+0000 7f536fe00640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5370109b90 0x7f537019fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.243+0000 7f536fe00640 1 -- 172.21.15.32:0/823159239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53701a01b0 con 0x7f5370103900 2024-08-31T07:12:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.244+0000 7f536fe00640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5370103900 0x7f537019a240 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5360005940 tx=0x7f53600042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.244+0000 7f536cc00640 1 -- 172.21.15.32:0/823159239 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536003f030 con 0x7f5370103900 2024-08-31T07:12:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.244+0000 7f5377000640 1 -- 172.21.15.32:0/823159239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53701a0440 con 0x7f5370103900 2024-08-31T07:12:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.244+0000 7f536cc00640 1 -- 172.21.15.32:0/823159239 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5360003cb0 con 0x7f5370103900 2024-08-31T07:12:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.244+0000 7f536cc00640 1 -- 172.21.15.32:0/823159239 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360035070 con 0x7f5370103900 2024-08-31T07:12:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.244+0000 7f5377000640 1 -- 172.21.15.32:0/823159239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53701a6c00 con 0x7f5370103900 2024-08-31T07:12:22.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.245+0000 7f5377000640 1 -- 172.21.15.32:0/823159239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5370103d10 con 0x7f5370103900 2024-08-31T07:12:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.248+0000 7f536cc00640 1 -- 172.21.15.32:0/823159239 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f5360042030 con 0x7f5370103900 2024-08-31T07:12:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.249+0000 7f536cc00640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53540b4b80 0x7f53540b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.250+0000 7f536cc00640 1 -- 172.21.15.32:0/823159239 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5360105ce0 con 0x7f5370103900 2024-08-31T07:12:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.250+0000 7f536f400640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53540b4b80 0x7f53540b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:22.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.251+0000 7f536f400640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53540b4b80 0x7f53540b7090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5370106040 tx=0x7f535c00b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:22.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.251+0000 7f536cc00640 1 -- 172.21.15.32:0/823159239 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f53600cc6c0 con 0x7f5370103900 2024-08-31T07:12:22.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.448+0000 7f5377000640 1 -- 172.21.15.32:0/823159239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f537019b370 con 0x7f5370103900 2024-08-31T07:12:22.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.450+0000 7f536cc00640 1 -- 172.21.15.32:0/823159239 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22921 (secure 0 0 0) 0x7f53600d0e80 con 0x7f5370103900 2024-08-31T07:12:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:22.452 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:10:46.011475+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:12:22.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.452+0000 7f5377000640 1 -- 172.21.15.32:0/823159239 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53540b4b80 msgr2=0x7f53540b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:22.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.452+0000 7f5377000640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53540b4b80 0x7f53540b7090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5370106040 tx=0x7f535c00b410 comp rx=0 tx=0).stop 2024-08-31T07:12:22.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.452+0000 7f5377000640 1 -- 172.21.15.32:0/823159239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5370103900 msgr2=0x7f537019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.452+0000 7f5377000640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5370103900 0x7f537019a240 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5360005940 tx=0x7f53600042f0 comp rx=0 tx=0).stop 2024-08-31T07:12:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.452+0000 7f5377000640 1 -- 172.21.15.32:0/823159239 shutdown_connections 2024-08-31T07:12:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.452+0000 7f5377000640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f53540b4b80 0x7f53540b7090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.452+0000 7f5377000640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5370109b90 0x7f537019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.452+0000 7f5377000640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5370109240 0x7f537019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.452+0000 7f5377000640 1 --2- 172.21.15.32:0/823159239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5370103900 0x7f537019a240 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.452+0000 7f5377000640 1 -- 172.21.15.32:0/823159239 >> 172.21.15.32:0/823159239 conn(0x7f537007b5f0 msgr2=0x7f53701072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.453+0000 7f5377000640 1 -- 172.21.15.32:0/823159239 shutdown_connections 2024-08-31T07:12:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:22.453+0000 7f5377000640 1 -- 172.21.15.32:0/823159239 wait complete. 2024-08-31T07:12:22.852 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:22 smithi032 ceph-mon[17252]: pgmap v356: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-31T07:12:22.852 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:22 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/823159239' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:12:22.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:22 smithi073 ceph-mon[21995]: pgmap v356: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-31T07:12:22.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:22 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/823159239' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:12:22.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:22 smithi084 ceph-mon[22387]: pgmap v356: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-31T07:12:22.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:22 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/823159239' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:12:23.251 DEBUG:tasks.fwd_scrub:Yielding 2024-08-31T07:12:23.251 INFO:teuthology.run_tasks:Running task exec... 2024-08-31T07:12:23.259 INFO:teuthology.task.exec:Executing custom commands... 2024-08-31T07:12:23.259 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-08-31T07:12:23.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-08-31T07:12:23.260 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-08-31T07:12:23.260 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:12:23.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:12:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.483+0000 7fbcfd400640 1 -- 172.21.15.32:0/1505090972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf81036b0 msgr2=0x7fbcf8103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.483+0000 7fbcfd400640 1 --2- 172.21.15.32:0/1505090972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf81036b0 0x7fbcf8103ac0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009900 tx=0x7fbce802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.483+0000 7fbcfd400640 1 -- 172.21.15.32:0/1505090972 shutdown_connections 2024-08-31T07:12:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.483+0000 7fbcfd400640 1 --2- 172.21.15.32:0/1505090972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf8109930 0x7fbcf8110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.483+0000 7fbcfd400640 1 --2- 172.21.15.32:0/1505090972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf8108fe0 0x7fbcf81093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.483+0000 7fbcfd400640 1 --2- 172.21.15.32:0/1505090972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf81036b0 0x7fbcf8103ac0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.483+0000 7fbcfd400640 1 -- 172.21.15.32:0/1505090972 >> 172.21.15.32:0/1505090972 conn(0x7fbcf80ff130 msgr2=0x7fbcf8101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.483+0000 7fbcfd400640 1 -- 172.21.15.32:0/1505090972 shutdown_connections 2024-08-31T07:12:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.483+0000 7fbcfd400640 1 -- 172.21.15.32:0/1505090972 wait complete. 2024-08-31T07:12:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.484+0000 7fbcfd400640 1 Processor -- start 2024-08-31T07:12:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.484+0000 7fbcfd400640 1 -- start start 2024-08-31T07:12:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcfd400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf81036b0 0x7fbcf819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcfd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf8108fe0 0x7fbcf819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcfd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf8109930 0x7fbcf81a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcfd400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81135e0 con 0x7fbcf8108fe0 2024-08-31T07:12:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcfd400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8113460 con 0x7fbcf8109930 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcfd400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8113760 con 0x7fbcf81036b0 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcf7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf8108fe0 0x7fbcf819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcf7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf8108fe0 0x7fbcf819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58544/0 (socket says 172.21.15.32:58544) 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcf7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf81036b0 0x7fbcf819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcf7400640 1 -- 172.21.15.32:0/590219805 learned_addr learned my addr 172.21.15.32:0/590219805 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcfca00640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf8109930 0x7fbcf81a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.485+0000 7fbcf7400640 1 -- 172.21.15.32:0/590219805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf81036b0 msgr2=0x7fbcf819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.486+0000 7fbcf7400640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf81036b0 0x7fbcf819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.486+0000 7fbcf7400640 1 -- 172.21.15.32:0/590219805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf8109930 msgr2=0x7fbcf81a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.486+0000 7fbcf7400640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf8109930 0x7fbcf81a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.486+0000 7fbcf7400640 1 -- 172.21.15.32:0/590219805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf81a21f0 con 0x7fbcf8108fe0 2024-08-31T07:12:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.486+0000 7fbcf7400640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf8108fe0 0x7fbcf819c860 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbce400e9f0 tx=0x7fbce400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.486+0000 7fbcf4c00640 1 -- 172.21.15.32:0/590219805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce400ccc0 con 0x7fbcf8108fe0 2024-08-31T07:12:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.486+0000 7fbcfd400640 1 -- 172.21.15.32:0/590219805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf81a24e0 con 0x7fbcf8108fe0 2024-08-31T07:12:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.486+0000 7fbcf4c00640 1 -- 172.21.15.32:0/590219805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce40108e0 con 0x7fbcf8108fe0 2024-08-31T07:12:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.486+0000 7fbcf4c00640 1 -- 172.21.15.32:0/590219805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce4018860 con 0x7fbcf8108fe0 2024-08-31T07:12:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.486+0000 7fbcfd400640 1 -- 172.21.15.32:0/590219805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf81a8c70 con 0x7fbcf8108fe0 2024-08-31T07:12:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.488+0000 7fbcfd400640 1 -- 172.21.15.32:0/590219805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf8103ac0 con 0x7fbcf8108fe0 2024-08-31T07:12:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.491+0000 7fbcf4c00640 1 -- 172.21.15.32:0/590219805 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7fbce4010430 con 0x7fbcf8108fe0 2024-08-31T07:12:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.492+0000 7fbcf4c00640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcdc0b4b70 0x7fbcdc0b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.492+0000 7fbcf4c00640 1 -- 172.21.15.32:0/590219805 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fbce40e59a0 con 0x7fbcf8108fe0 2024-08-31T07:12:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.493+0000 7fbcf7e00640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcdc0b4b70 0x7fbcdc0b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.493+0000 7fbcf7e00640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcdc0b4b70 0x7fbcdc0b7080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbce8031440 tx=0x7fbce8009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.497+0000 7fbcf4c00640 1 -- 172.21.15.32:0/590219805 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fbce40ac380 con 0x7fbcf8108fe0 2024-08-31T07:12:23.658 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.743+0000 7fbcfd400640 1 -- 172.21.15.32:0/590219805 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fbcf810f3c0 con 0x7fbcdc0b4b70 2024-08-31T07:12:23.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.920+0000 7fbcf4c00640 1 -- 172.21.15.32:0/590219805 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbcf810f3c0 con 0x7fbcdc0b4b70 2024-08-31T07:12:23.921 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path / 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 -- 172.21.15.32:0/590219805 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcdc0b4b70 msgr2=0x7fbcdc0b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcdc0b4b70 0x7fbcdc0b7080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbce8031440 tx=0x7fbce8009560 comp rx=0 tx=0).stop 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 -- 172.21.15.32:0/590219805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf8108fe0 msgr2=0x7fbcf819c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf8108fe0 0x7fbcf819c860 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbce400e9f0 tx=0x7fbce400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 -- 172.21.15.32:0/590219805 shutdown_connections 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcdc0b4b70 0x7fbcdc0b7080 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf8109930 0x7fbcf81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf8108fe0 0x7fbcf819c860 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 --2- 172.21.15.32:0/590219805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf81036b0 0x7fbcf819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 -- 172.21.15.32:0/590219805 >> 172.21.15.32:0/590219805 conn(0x7fbcf80ff130 msgr2=0x7fbcf8108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 -- 172.21.15.32:0/590219805 shutdown_connections 2024-08-31T07:12:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:23.922+0000 7fbcfd400640 1 -- 172.21.15.32:0/590219805 wait complete. 2024-08-31T07:12:23.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-08-31T07:12:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.176+0000 7f4edbc00640 1 -- 172.21.15.32:0/2729426157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4103880 msgr2=0x7f4ed4103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.176+0000 7f4edbc00640 1 --2- 172.21.15.32:0/2729426157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4103880 0x7f4ed4103c90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009900 tx=0x7f4ec802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.176+0000 7f4edbc00640 1 -- 172.21.15.32:0/2729426157 shutdown_connections 2024-08-31T07:12:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.176+0000 7f4edbc00640 1 --2- 172.21.15.32:0/2729426157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ed4109b00 0x7f4ed4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.176+0000 7f4edbc00640 1 --2- 172.21.15.32:0/2729426157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ed41091b0 0x7f4ed41095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.176+0000 7f4edbc00640 1 --2- 172.21.15.32:0/2729426157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4103880 0x7f4ed4103c90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.176+0000 7f4edbc00640 1 -- 172.21.15.32:0/2729426157 >> 172.21.15.32:0/2729426157 conn(0x7f4ed40ff2e0 msgr2=0x7f4ed4101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.176+0000 7f4edbc00640 1 -- 172.21.15.32:0/2729426157 shutdown_connections 2024-08-31T07:12:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.177+0000 7f4edbc00640 1 -- 172.21.15.32:0/2729426157 wait complete. 2024-08-31T07:12:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.178+0000 7f4edbc00640 1 Processor -- start 2024-08-31T07:12:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.178+0000 7f4edbc00640 1 -- start start 2024-08-31T07:12:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.178+0000 7f4edbc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4103880 0x7f4ed419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.178+0000 7f4edbc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ed41091b0 0x7f4ed419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.178+0000 7f4edbc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ed4109b00 0x7f4ed41a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.178+0000 7f4edbc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41137b0 con 0x7f4ed4109b00 2024-08-31T07:12:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.178+0000 7f4edbc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4113630 con 0x7f4ed41091b0 2024-08-31T07:12:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.178+0000 7f4edbc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4113930 con 0x7f4ed4103880 2024-08-31T07:12:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed9200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4103880 0x7f4ed419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ed41091b0 0x7f4ed419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ed4109b00 0x7f4ed41a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed9200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4103880 0x7f4ed419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:39022/0 (socket says 172.21.15.32:39022) 2024-08-31T07:12:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed9200640 1 -- 172.21.15.32:0/3222772480 learned_addr learned my addr 172.21.15.32:0/3222772480 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed9200640 1 -- 172.21.15.32:0/3222772480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ed41091b0 msgr2=0x7f4ed419c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed9200640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ed41091b0 0x7f4ed419c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed9200640 1 -- 172.21.15.32:0/3222772480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ed4109b00 msgr2=0x7f4ed41a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed9200640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ed4109b00 0x7f4ed41a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed9200640 1 -- 172.21.15.32:0/3222772480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed41a2380 con 0x7f4ed4103880 2024-08-31T07:12:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed3e00640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ed41091b0 0x7f4ed419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.179+0000 7f4ed9200640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4103880 0x7f4ed419c450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8005940 tx=0x7f4ec8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.180+0000 7f4ed1600640 1 -- 172.21.15.32:0/3222772480 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec803f030 con 0x7f4ed4103880 2024-08-31T07:12:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.180+0000 7f4edbc00640 1 -- 172.21.15.32:0/3222772480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed41a2610 con 0x7f4ed4103880 2024-08-31T07:12:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.180+0000 7f4ed1600640 1 -- 172.21.15.32:0/3222772480 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ec8003aa0 con 0x7f4ed4103880 2024-08-31T07:12:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.180+0000 7f4ed1600640 1 -- 172.21.15.32:0/3222772480 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8035070 con 0x7f4ed4103880 2024-08-31T07:12:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.180+0000 7f4edbc00640 1 -- 172.21.15.32:0/3222772480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed41a8e00 con 0x7f4ed4103880 2024-08-31T07:12:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.181+0000 7f4edbc00640 1 -- 172.21.15.32:0/3222772480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed4103c90 con 0x7f4ed4103880 2024-08-31T07:12:24.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.183+0000 7f4ed1600640 1 -- 172.21.15.32:0/3222772480 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f4ec8042030 con 0x7f4ed4103880 2024-08-31T07:12:24.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.184+0000 7f4ed1600640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4ebc0b4b80 0x7f4ebc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.185+0000 7f4ed1600640 1 -- 172.21.15.32:0/3222772480 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f4ec8105920 con 0x7f4ed4103880 2024-08-31T07:12:24.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.185+0000 7f4ed3e00640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4ebc0b4b80 0x7f4ebc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.186+0000 7f4ed3e00640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4ebc0b4b80 0x7f4ebc0b7090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00095f0 tx=0x7f4ec0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:24.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.189+0000 7f4ed1600640 1 -- 172.21.15.32:0/3222772480 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4ec80cc300 con 0x7f4ed4103880 2024-08-31T07:12:24.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.336+0000 7efc0ae00640 1 -- 172.21.15.32:0/2092872973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc04109090 msgr2=0x7efc041094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.336+0000 7efc0ae00640 1 --2- 172.21.15.32:0/2092872973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc04109090 0x7efc041094a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009900 tx=0x7efbf002a410 comp rx=0 tx=0).stop 2024-08-31T07:12:24.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.336+0000 7efc0ae00640 1 -- 172.21.15.32:0/2092872973 shutdown_connections 2024-08-31T07:12:24.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.336+0000 7efc0ae00640 1 --2- 172.21.15.32:0/2092872973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc041099e0 0x7efc04110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.336+0000 7efc0ae00640 1 --2- 172.21.15.32:0/2092872973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc04109090 0x7efc041094a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.336+0000 7efc0ae00640 1 --2- 172.21.15.32:0/2092872973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc04103750 0x7efc04103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.336+0000 7efc0ae00640 1 -- 172.21.15.32:0/2092872973 >> 172.21.15.32:0/2092872973 conn(0x7efc040ff1e0 msgr2=0x7efc04101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:24.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.336+0000 7efc0ae00640 1 -- 172.21.15.32:0/2092872973 shutdown_connections 2024-08-31T07:12:24.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.336+0000 7efc0ae00640 1 -- 172.21.15.32:0/2092872973 wait complete. 2024-08-31T07:12:24.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.337+0000 7efc0ae00640 1 Processor -- start 2024-08-31T07:12:24.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.338+0000 7efc0ae00640 1 -- start start 2024-08-31T07:12:24.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.338+0000 7efc0ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc04103750 0x7efc040754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.338+0000 7efc0ae00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc04109090 0x7efc040759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.338+0000 7efc0ae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc041099e0 0x7efc04075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.338+0000 7efc0ae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc04113670 con 0x7efc04103750 2024-08-31T07:12:24.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.338+0000 7efc0ae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc041134f0 con 0x7efc041099e0 2024-08-31T07:12:24.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.338+0000 7efc0ae00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc041137f0 con 0x7efc04109090 2024-08-31T07:12:24.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.338+0000 7efc09a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc04103750 0x7efc040754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.338+0000 7efc09a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc04103750 0x7efc040754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58590/0 (socket says 172.21.15.32:58590) 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.339+0000 7efc09a00640 1 -- 172.21.15.32:0/1333720716 learned_addr learned my addr 172.21.15.32:0/1333720716 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.339+0000 7efc09000640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc04109090 0x7efc040759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.339+0000 7efc0a400640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc041099e0 0x7efc04075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.339+0000 7efc09000640 1 -- 172.21.15.32:0/1333720716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc041099e0 msgr2=0x7efc04075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.339+0000 7efc09000640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc041099e0 0x7efc04075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.339+0000 7efc09000640 1 -- 172.21.15.32:0/1333720716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc04103750 msgr2=0x7efc040754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.339+0000 7efc09000640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc04103750 0x7efc040754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.339+0000 7efc09000640 1 -- 172.21.15.32:0/1333720716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc041a83e0 con 0x7efc04109090 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.339+0000 7efc09a00640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc04103750 0x7efc040754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.339+0000 7efc09000640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc04109090 0x7efc040759f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efbf002a930 tx=0x7efbf0031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.340+0000 7efc02000640 1 -- 172.21.15.32:0/1333720716 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf003f070 con 0x7efc04109090 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.340+0000 7efc02000640 1 -- 172.21.15.32:0/1333720716 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf0004490 con 0x7efc04109090 2024-08-31T07:12:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.340+0000 7efc0ae00640 1 -- 172.21.15.32:0/1333720716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc041a8670 con 0x7efc04109090 2024-08-31T07:12:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.340+0000 7efc02000640 1 -- 172.21.15.32:0/1333720716 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0035070 con 0x7efc04109090 2024-08-31T07:12:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.340+0000 7efc0ae00640 1 -- 172.21.15.32:0/1333720716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc041a8ba0 con 0x7efc04109090 2024-08-31T07:12:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.343+0000 7efc12fdd740 1 Processor -- start 2024-08-31T07:12:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.343+0000 7efc12fdd740 1 -- start start 2024-08-31T07:12:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349adb350 0x55a349e817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349ab2490 0x55a349e81d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e82250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc12fdd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a349df7750 con 0x55a349adb350 2024-08-31T07:12:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc12fdd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a349de6ce0 con 0x55a349aba670 2024-08-31T07:12:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc12fdd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a349deac90 con 0x55a349ab2490 2024-08-31T07:12:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc0a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349adb350 0x55a349e817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc09a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349ab2490 0x55a349e81d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc0a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349adb350 0x55a349e817d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58596/0 (socket says 172.21.15.32:58596) 2024-08-31T07:12:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc0a400640 1 -- 172.21.15.32:0/1972485425 learned_addr learned my addr 172.21.15.32:0/1972485425 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc09a00640 1 -- 172.21.15.32:0/1972485425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 msgr2=0x55a349e82250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:12:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc09a00640 1 --2- 172.21.15.32:0/1972485425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e82250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc09a00640 1 -- 172.21.15.32:0/1972485425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349adb350 msgr2=0x55a349e817d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc09a00640 1 --2- 172.21.15.32:0/1972485425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349adb350 0x55a349e817d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.344+0000 7efc09a00640 1 -- 172.21.15.32:0/1972485425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a349b46930 con 0x55a349ab2490 2024-08-31T07:12:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.345+0000 7efc09a00640 1 --2- 172.21.15.32:0/1972485425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349ab2490 0x55a349e81d10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009fd0 tx=0x7efbf80119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6922aa76a1419dd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.345+0000 7efbf6a00640 1 -- 172.21.15.32:0/1972485425 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8004020 con 0x55a349ab2490 2024-08-31T07:12:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.345+0000 7efbf6a00640 1 -- 172.21.15.32:0/1972485425 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf80054d0 con 0x55a349ab2490 2024-08-31T07:12:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.345+0000 7efbf6a00640 1 -- 172.21.15.32:0/1972485425 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8004ca0 con 0x55a349ab2490 2024-08-31T07:12:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.346+0000 7efc02000640 1 -- 172.21.15.32:0/1333720716 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7efbf003b460 con 0x7efc04109090 2024-08-31T07:12:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.346+0000 7efc12fdd740 1 -- 172.21.15.32:0/1972485425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349ab2490 msgr2=0x55a349e81d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.346+0000 7efc12fdd740 1 --2- 172.21.15.32:0/1972485425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349ab2490 0x55a349e81d10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009fd0 tx=0x7efbf80119a0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.346+0000 7efc12fdd740 1 -- 172.21.15.32:0/1972485425 shutdown_connections 2024-08-31T07:12:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.346+0000 7efc12fdd740 1 --2- 172.21.15.32:0/1972485425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e82250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.346+0000 7efc12fdd740 1 --2- 172.21.15.32:0/1972485425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349ab2490 0x55a349e81d10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.346+0000 7efc12fdd740 1 --2- 172.21.15.32:0/1972485425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349adb350 0x55a349e817d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.346+0000 7efc12fdd740 1 -- 172.21.15.32:0/1972485425 >> 172.21.15.32:0/1972485425 conn(0x55a349acf740 msgr2=0x55a349e7f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.346+0000 7efc12fdd740 1 -- 172.21.15.32:0/1972485425 shutdown_connections 2024-08-31T07:12:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.346+0000 7efc12fdd740 1 -- 172.21.15.32:0/1972485425 wait complete. 2024-08-31T07:12:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.347+0000 7efc12fdd740 1 -- start start 2024-08-31T07:12:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.348+0000 7efc12fdd740 1 Processor -- start 2024-08-31T07:12:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.348+0000 7efc02000640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbec0b4b20 0x7efbec0b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.348+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e8d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.348+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e8d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.348+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349adb350 0x55a349e992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.348+0000 7efc12fdd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a349c49890 con 0x55a349ab2490 2024-08-31T07:12:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.348+0000 7efc12fdd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a349d6dcd0 con 0x55a349aba670 2024-08-31T07:12:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.348+0000 7efc12fdd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a349e8ffb0 con 0x55a349adb350 2024-08-31T07:12:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.348+0000 7efc02000640 1 -- 172.21.15.32:0/1333720716 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7efbf0105690 con 0x7efc04109090 2024-08-31T07:12:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.349+0000 7efc12fdd740 10 client.0 _ll_register_callbacks cb 0x55a349de1ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.349+0000 7efc0a400640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbec0b4b20 0x7efbec0b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.349+0000 7efc0a400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349adb350 0x55a349e992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.349+0000 7efc09a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e8d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.349+0000 7efc09a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e8d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58602/0 (socket says 172.21.15.32:58602) 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.349+0000 7efc09a00640 1 -- 172.21.15.32:0/1226044379 learned_addr learned my addr 172.21.15.32:0/1226044379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.349+0000 7efc09000640 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e8d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.349+0000 7efc09a00640 1 -- 172.21.15.32:0/1226044379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349adb350 msgr2=0x55a349e992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.349+0000 7efc09a00640 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349adb350 0x55a349e992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efc09a00640 1 -- 172.21.15.32:0/1226044379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 msgr2=0x55a349e8d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efc09a00640 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e8d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efc09a00640 1 -- 172.21.15.32:0/1226044379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a349db71b0 con 0x55a349ab2490 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efc09000640 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e8d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efc0a400640 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349adb350 0x55a349e992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efc0a400640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbec0b4b20 0x7efbec0b7030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efbfc005fd0 tx=0x7efbfc011440 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efc09a00640 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e8d3b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efbf801b040 tx=0x7efbf800fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ffcfcc6f4a7c70 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efbeb400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efbeb400640 1 -- 172.21.15.32:0/1226044379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8027020 con 0x55a349ab2490 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efbeb400640 1 -- 172.21.15.32:0/1226044379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7efbf80049e0 con 0x55a349ab2490 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.350+0000 7efc12fdd740 1 -- 172.21.15.32:0/1226044379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf80054d0 con 0x55a349ab2490 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.351+0000 7efbeb400640 1 -- 172.21.15.32:0/1226044379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efbf8032050 con 0x55a349ab2490 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.351+0000 7efc12fdd740 10 client.15396 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.351+0000 7efc12fdd740 1 -- 172.21.15.32:0/1226044379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a349e9bff0 con 0x55a349ab2490 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.351+0000 7efbeb400640 1 -- 172.21.15.32:0/1226044379 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf80054d0 con 0x55a349ab2490 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.351+0000 7efbeb400640 1 -- 172.21.15.32:0/1226044379 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efbf8023470 con 0x55a349ab2490 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.351+0000 7efc12fdd740 10 client.15396 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.352+0000 7efc12fdd740 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.352+0000 7efc12fdd740 20 client.15396 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.352+0000 7efc12fdd740 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a349a99cd0 0x55a349e9f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.352+0000 7efc12fdd740 4 client.15396 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:24.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.352+0000 7efc12fdd740 1 -- 172.21.15.32:0/1226044379 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a349df96a0 con 0x55a349a99cd0 2024-08-31T07:12:24.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.352+0000 7efc09000640 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a349a99cd0 0x55a349e9f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.353+0000 7efc09000640 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a349a99cd0 0x55a349e9f460 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd27c372869134f server_cookie=3941b605ad246dc4 in_seq=0 out_seq=0 2024-08-31T07:12:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.353+0000 7efbeb400640 10 client.15396 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.353+0000 7efbeb400640 1 -- 172.21.15.32:0/1226044379 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a349df96a0 con 0x55a349a99cd0 2024-08-31T07:12:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.353+0000 7efbeb400640 10 client.15396 handle_command_reply: tid=0 2024-08-31T07:12:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.353+0000 7efc12fdd740 1 client.15396 shutdown 2024-08-31T07:12:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 1 -- 172.21.15.32:0/1226044379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 msgr2=0x55a349e8d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e8d3b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efbf801b040 tx=0x7efbf800fe20 comp rx=0 tx=0).stop 2024-08-31T07:12:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 1 -- 172.21.15.32:0/1226044379 shutdown_connections 2024-08-31T07:12:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a349a99cd0 0x55a349e9f460 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349adb350 0x55a349e992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e8d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 1 --2- 172.21.15.32:0/1226044379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e8d3b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efbeb400640 0 client.15396 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 1 -- 172.21.15.32:0/1226044379 >> 172.21.15.32:0/1226044379 conn(0x55a349acf740 msgr2=0x55a349e849b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 1 -- 172.21.15.32:0/1226044379 shutdown_connections 2024-08-31T07:12:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 1 -- 172.21.15.32:0/1226044379 wait complete. 2024-08-31T07:12:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.354+0000 7efc12fdd740 20 client.15396 trim_cache size 0 max 16384 2024-08-31T07:12:24.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.373+0000 7efc12fdd740 1 Processor -- start 2024-08-31T07:12:24.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.373+0000 7efc12fdd740 1 -- start start 2024-08-31T07:12:24.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.373+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349a99cd0 0x55a349e991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.373+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e84fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.373+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349aba670 0x55a349e7efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.373+0000 7efc12fdd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b13b0 con 0x55a349ab2490 2024-08-31T07:12:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc12fdd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b2ef0 con 0x55a349a99cd0 2024-08-31T07:12:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc12fdd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b1a80 con 0x55a349aba670 2024-08-31T07:12:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc09000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e84fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc09000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e84fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58604/0 (socket says 172.21.15.32:58604) 2024-08-31T07:12:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc09000640 1 -- 172.21.15.32:0/2714632178 learned_addr learned my addr 172.21.15.32:0/2714632178 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc09a00640 1 --2- 172.21.15.32:0/2714632178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349a99cd0 0x55a349e991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc0a400640 1 --2- 172.21.15.32:0/2714632178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349aba670 0x55a349e7efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc09a00640 1 -- 172.21.15.32:0/2714632178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349aba670 msgr2=0x55a349e7efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:24.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc09a00640 1 --2- 172.21.15.32:0/2714632178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349aba670 0x55a349e7efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc09a00640 1 -- 172.21.15.32:0/2714632178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 msgr2=0x55a349e84fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:24.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc09a00640 1 --2- 172.21.15.32:0/2714632178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e84fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc09a00640 1 -- 172.21.15.32:0/2714632178 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a349e8c110 con 0x55a349a99cd0 2024-08-31T07:12:24.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.374+0000 7efc09000640 1 --2- 172.21.15.32:0/2714632178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e84fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:24.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.375+0000 7efc09a00640 1 --2- 172.21.15.32:0/2714632178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349a99cd0 0x55a349e991a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efbf8023a30 tx=0x7efbf8011ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1e80fe52d49712 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:24.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.375+0000 7efbf6000640 1 -- 172.21.15.32:0/2714632178 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8027070 con 0x55a349a99cd0 2024-08-31T07:12:24.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.375+0000 7efbf6000640 1 -- 172.21.15.32:0/2714632178 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf800fe90 con 0x55a349a99cd0 2024-08-31T07:12:24.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.375+0000 7efbf6000640 1 -- 172.21.15.32:0/2714632178 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf801d040 con 0x55a349a99cd0 2024-08-31T07:12:24.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.376+0000 7efc12fdd740 1 -- 172.21.15.32:0/2714632178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349a99cd0 msgr2=0x55a349e991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.376+0000 7efc12fdd740 1 --2- 172.21.15.32:0/2714632178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349a99cd0 0x55a349e991a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efbf8023a30 tx=0x7efbf8011ee0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.376+0000 7efc12fdd740 1 -- 172.21.15.32:0/2714632178 shutdown_connections 2024-08-31T07:12:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.376+0000 7efc12fdd740 1 --2- 172.21.15.32:0/2714632178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349aba670 0x55a349e7efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.376+0000 7efc12fdd740 1 --2- 172.21.15.32:0/2714632178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e84fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.376+0000 7efc12fdd740 1 --2- 172.21.15.32:0/2714632178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349a99cd0 0x55a349e991a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.376+0000 7efc12fdd740 1 -- 172.21.15.32:0/2714632178 >> 172.21.15.32:0/2714632178 conn(0x55a349acf740 msgr2=0x55a349e848d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.376+0000 7efc12fdd740 1 -- 172.21.15.32:0/2714632178 shutdown_connections 2024-08-31T07:12:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.376+0000 7efc12fdd740 1 -- 172.21.15.32:0/2714632178 wait complete. 2024-08-31T07:12:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.377+0000 7efc12fdd740 1 -- start start 2024-08-31T07:12:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.377+0000 7efc12fdd740 1 Processor -- start 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.377+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349a99cd0 0x55a349e913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.377+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e91910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.377+0000 7efc12fdd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e91e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.377+0000 7efc12fdd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b2820 con 0x55a349ab2490 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.377+0000 7efc12fdd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b2150 con 0x55a349aba670 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.377+0000 7efc12fdd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc041af390 con 0x55a349a99cd0 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.377+0000 7efc12fdd740 10 client.0 _ll_register_callbacks cb 0x7efbfc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc09a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349a99cd0 0x55a349e913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc09000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e91910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc09a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349a99cd0 0x55a349e913d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:39062/0 (socket says 172.21.15.32:39062) 2024-08-31T07:12:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc09a00640 1 -- 172.21.15.32:0/3304900491 learned_addr learned my addr 172.21.15.32:0/3304900491 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc0a400640 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e91e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc09a00640 1 -- 172.21.15.32:0/3304900491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 msgr2=0x55a349e91e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc09a00640 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e91e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc09a00640 1 -- 172.21.15.32:0/3304900491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 msgr2=0x55a349e91910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc09a00640 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e91910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc09a00640 1 -- 172.21.15.32:0/3304900491 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a349f08e60 con 0x55a349a99cd0 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efc09a00640 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349a99cd0 0x55a349e913d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efbf8014040 tx=0x7efbf8023720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217a0b2b2129f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.378+0000 7efbeb400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.379+0000 7efbeb400640 1 -- 172.21.15.32:0/3304900491 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf801d030 con 0x55a349a99cd0 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.379+0000 7efc12fdd740 1 -- 172.21.15.32:0/3304900491 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf800fe90 con 0x55a349a99cd0 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.379+0000 7efbeb400640 1 -- 172.21.15.32:0/3304900491 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7efbf80054c0 con 0x55a349a99cd0 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.379+0000 7efbeb400640 1 -- 172.21.15.32:0/3304900491 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efbf8044050 con 0x55a349a99cd0 2024-08-31T07:12:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.379+0000 7efc12fdd740 10 client.25420 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.379+0000 7efc12fdd740 1 -- 172.21.15.32:0/3304900491 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a349f0a120 con 0x55a349a99cd0 2024-08-31T07:12:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.380+0000 7efbeb400640 1 -- 172.21.15.32:0/3304900491 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf800fe90 con 0x55a349a99cd0 2024-08-31T07:12:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.380+0000 7efbeb400640 1 -- 172.21.15.32:0/3304900491 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efbf8036430 con 0x55a349a99cd0 2024-08-31T07:12:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.380+0000 7efc12fdd740 10 client.25420 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.380+0000 7efc12fdd740 10 client.25420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:12:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.380+0000 7efc12fdd740 20 client.25420 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.380+0000 7efc12fdd740 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a349adb350 0x55a349f0c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.380+0000 7efc12fdd740 4 client.25420 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:12:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.380+0000 7efc12fdd740 1 -- 172.21.15.32:0/3304900491 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a349df5140 con 0x55a349adb350 2024-08-31T07:12:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.380+0000 7efc09000640 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a349adb350 0x55a349f0c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.381+0000 7efc09000640 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a349adb350 0x55a349f0c7c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7490255974c9582e server_cookie=d94471b153fa2129 in_seq=0 out_seq=0 2024-08-31T07:12:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.381+0000 7efbeb400640 10 client.25420 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.383+0000 7efbeb400640 1 -- 172.21.15.32:0/3304900491 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a349df5140 con 0x55a349adb350 2024-08-31T07:12:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.383+0000 7efbeb400640 10 client.25420 handle_command_reply: tid=0 2024-08-31T07:12:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.383+0000 7efc12fdd740 1 client.25420 shutdown 2024-08-31T07:12:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.383+0000 7efc12fdd740 1 -- 172.21.15.32:0/3304900491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349a99cd0 msgr2=0x55a349e913d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.383+0000 7efc12fdd740 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349a99cd0 0x55a349e913d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efbf8014040 tx=0x7efbf8023720 comp rx=0 tx=0).stop 2024-08-31T07:12:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.384+0000 7efc12fdd740 1 -- 172.21.15.32:0/3304900491 shutdown_connections 2024-08-31T07:12:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.384+0000 7efc12fdd740 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a349adb350 0x55a349f0c7c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.384+0000 7efc12fdd740 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a349aba670 0x55a349e91e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.384+0000 7efc12fdd740 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a349ab2490 0x55a349e91910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.384+0000 7efc12fdd740 1 --2- 172.21.15.32:0/3304900491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a349a99cd0 0x55a349e913d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.384+0000 7efc12fdd740 1 -- 172.21.15.32:0/3304900491 >> 172.21.15.32:0/3304900491 conn(0x55a349acf740 msgr2=0x55a349e7ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.384+0000 7efbeb400640 0 client.25420 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.384+0000 7efc12fdd740 1 -- 172.21.15.32:0/3304900491 shutdown_connections 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.384+0000 7efc12fdd740 1 -- 172.21.15.32:0/3304900491 wait complete. 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.384+0000 7efc12fdd740 20 client.25420 trim_cache size 0 max 16384 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "323cd93f-7bb8-4706-b7dc-0b247836a042", 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:12:24.386 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 -- 172.21.15.32:0/1333720716 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbec0b4b20 msgr2=0x7efbec0b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbec0b4b20 0x7efbec0b7030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efbfc005fd0 tx=0x7efbfc011440 comp rx=0 tx=0).stop 2024-08-31T07:12:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 -- 172.21.15.32:0/1333720716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc04109090 msgr2=0x7efc040759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc04109090 0x7efc040759f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efbf002a930 tx=0x7efbf0031a20 comp rx=0 tx=0).stop 2024-08-31T07:12:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 -- 172.21.15.32:0/1333720716 shutdown_connections 2024-08-31T07:12:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbec0b4b20 0x7efbec0b7030 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc041099e0 0x7efc04075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc04109090 0x7efc040759f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 --2- 172.21.15.32:0/1333720716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc04103750 0x7efc040754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 -- 172.21.15.32:0/1333720716 >> 172.21.15.32:0/1333720716 conn(0x7efc040ff1e0 msgr2=0x7efc04101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 -- 172.21.15.32:0/1333720716 shutdown_connections 2024-08-31T07:12:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.385+0000 7efc0ae00640 1 -- 172.21.15.32:0/1333720716 wait complete. 2024-08-31T07:12:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.426+0000 7f4edbc00640 1 -- 172.21.15.32:0/3222772480 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f4ed410fcc0 con 0x7f4ebc0b4b80 2024-08-31T07:12:24.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.502+0000 7f4ed1600640 1 -- 172.21.15.32:0/3222772480 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f4ed410fcc0 con 0x7f4ebc0b4b80 2024-08-31T07:12:24.503 INFO:teuthology.orchestra.run.smithi032.stdout:Retention added to path / 2024-08-31T07:12:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 -- 172.21.15.32:0/3222772480 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4ebc0b4b80 msgr2=0x7f4ebc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4ebc0b4b80 0x7f4ebc0b7090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00095f0 tx=0x7f4ec0005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 -- 172.21.15.32:0/3222772480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4103880 msgr2=0x7f4ed419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4103880 0x7f4ed419c450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8005940 tx=0x7f4ec8004270 comp rx=0 tx=0).stop 2024-08-31T07:12:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 -- 172.21.15.32:0/3222772480 shutdown_connections 2024-08-31T07:12:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4ebc0b4b80 0x7f4ebc0b7090 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ed4109b00 0x7f4ed41a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ed41091b0 0x7f4ed419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 --2- 172.21.15.32:0/3222772480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4103880 0x7f4ed419c450 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 -- 172.21.15.32:0/3222772480 >> 172.21.15.32:0/3222772480 conn(0x7f4ed40ff2e0 msgr2=0x7f4ed410e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 -- 172.21.15.32:0/3222772480 shutdown_connections 2024-08-31T07:12:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.504+0000 7f4edbc00640 1 -- 172.21.15.32:0/3222772480 wait complete. 2024-08-31T07:12:24.529 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-31T07:12:24.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.751+0000 7fda7de00640 1 -- 172.21.15.32:0/3747166809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda78108ff0 msgr2=0x7fda78109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.751+0000 7fda7de00640 1 --2- 172.21.15.32:0/3747166809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda78108ff0 0x7fda78109400 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fda64009900 tx=0x7fda6402a410 comp rx=0 tx=0).stop 2024-08-31T07:12:24.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.752+0000 7fda7de00640 1 -- 172.21.15.32:0/3747166809 shutdown_connections 2024-08-31T07:12:24.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.752+0000 7fda7de00640 1 --2- 172.21.15.32:0/3747166809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda78109940 0x7fda78110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.752+0000 7fda7de00640 1 --2- 172.21.15.32:0/3747166809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda78108ff0 0x7fda78109400 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.752+0000 7fda7de00640 1 --2- 172.21.15.32:0/3747166809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda781036c0 0x7fda78103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.752+0000 7fda7de00640 1 -- 172.21.15.32:0/3747166809 >> 172.21.15.32:0/3747166809 conn(0x7fda780ff0e0 msgr2=0x7fda78101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:24.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.752+0000 7fda7de00640 1 -- 172.21.15.32:0/3747166809 shutdown_connections 2024-08-31T07:12:24.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.752+0000 7fda7de00640 1 -- 172.21.15.32:0/3747166809 wait complete. 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7de00640 1 Processor -- start 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7de00640 1 -- start start 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7de00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda781036c0 0x7fda78199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda78108ff0 0x7fda7819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda78109940 0x7fda7819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7de00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda781135b0 con 0x7fda78109940 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7de00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda78113430 con 0x7fda78108ff0 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7de00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fda78113730 con 0x7fda781036c0 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda78109940 0x7fda7819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda78109940 0x7fda7819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58626/0 (socket says 172.21.15.32:58626) 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.753+0000 7fda7d400640 1 -- 172.21.15.32:0/1549025031 learned_addr learned my addr 172.21.15.32:0/1549025031 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:24.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda77e00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda78108ff0 0x7fda7819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda7ca00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda781036c0 0x7fda78199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda7d400640 1 -- 172.21.15.32:0/1549025031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda781036c0 msgr2=0x7fda78199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda7d400640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda781036c0 0x7fda78199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda7d400640 1 -- 172.21.15.32:0/1549025031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda78108ff0 msgr2=0x7fda7819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda7d400640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda78108ff0 0x7fda7819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda7d400640 1 -- 172.21.15.32:0/1549025031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7819ff20 con 0x7fda78109940 2024-08-31T07:12:24.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda77e00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda78108ff0 0x7fda7819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:24.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda7d400640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda78109940 0x7fda7819f820 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda7000eb10 tx=0x7fda7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:24.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda75600640 1 -- 172.21.15.32:0/1549025031 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70004b20 con 0x7fda78109940 2024-08-31T07:12:24.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.754+0000 7fda75600640 1 -- 172.21.15.32:0/1549025031 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda70004cc0 con 0x7fda78109940 2024-08-31T07:12:24.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.755+0000 7fda7de00640 1 -- 172.21.15.32:0/1549025031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda781a0210 con 0x7fda78109940 2024-08-31T07:12:24.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.755+0000 7fda75600640 1 -- 172.21.15.32:0/1549025031 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70010670 con 0x7fda78109940 2024-08-31T07:12:24.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.755+0000 7fda7de00640 1 -- 172.21.15.32:0/1549025031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda781a69a0 con 0x7fda78109940 2024-08-31T07:12:24.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.756+0000 7fda7de00640 1 -- 172.21.15.32:0/1549025031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda78103ad0 con 0x7fda78109940 2024-08-31T07:12:24.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.758+0000 7fda75600640 1 -- 172.21.15.32:0/1549025031 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7fda70005020 con 0x7fda78109940 2024-08-31T07:12:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.759+0000 7fda75600640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda600b4b70 0x7fda600b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.760+0000 7fda75600640 1 -- 172.21.15.32:0/1549025031 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fda700e5930 con 0x7fda78109940 2024-08-31T07:12:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.760+0000 7fda7ca00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda600b4b70 0x7fda600b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:24.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.760+0000 7fda7ca00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda600b4b70 0x7fda600b7080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009bd0 tx=0x7fda6c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:24.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.763+0000 7fda75600640 1 -- 172.21.15.32:0/1549025031 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fda700ac960 con 0x7fda78109940 2024-08-31T07:12:24.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:24 smithi032 ceph-mon[17252]: pgmap v357: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-31T07:12:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:24 smithi073 ceph-mon[21995]: pgmap v357: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-31T07:12:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.984+0000 7fda7de00640 1 -- 172.21.15.32:0/1549025031 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fda780635a0 con 0x7fda600b4b70 2024-08-31T07:12:24.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:24 smithi084 ceph-mon[22387]: pgmap v357: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-31T07:12:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.993+0000 7fda75600640 1 -- 172.21.15.32:0/1549025031 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fda780635a0 con 0x7fda600b4b70 2024-08-31T07:12:24.993 INFO:teuthology.orchestra.run.smithi032.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-31T00:00:00", "created": "2024-08-31T07:12:23", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-08-31T07:12:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 -- 172.21.15.32:0/1549025031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda600b4b70 msgr2=0x7fda600b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda600b4b70 0x7fda600b7080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009bd0 tx=0x7fda6c005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 -- 172.21.15.32:0/1549025031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda78109940 msgr2=0x7fda7819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda78109940 0x7fda7819f820 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda7000eb10 tx=0x7fda7000c2d0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 -- 172.21.15.32:0/1549025031 shutdown_connections 2024-08-31T07:12:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda600b4b70 0x7fda600b7080 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda78109940 0x7fda7819f820 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda78108ff0 0x7fda7819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 --2- 172.21.15.32:0/1549025031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda781036c0 0x7fda78199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 -- 172.21.15.32:0/1549025031 >> 172.21.15.32:0/1549025031 conn(0x7fda780ff0e0 msgr2=0x7fda78101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 -- 172.21.15.32:0/1549025031 shutdown_connections 2024-08-31T07:12:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:24.995+0000 7fda7de00640 1 -- 172.21.15.32:0/1549025031 wait complete. 2024-08-31T07:12:25.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-31T07:12:25.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.244+0000 7faab4e00640 1 -- 172.21.15.32:0/2672187850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab010fe20 msgr2=0x7faab0112240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:25.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.244+0000 7faab4e00640 1 --2- 172.21.15.32:0/2672187850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab010fe20 0x7faab0112240 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009ca0 tx=0x7faaa802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.244+0000 7faab4e00640 1 -- 172.21.15.32:0/2672187850 shutdown_connections 2024-08-31T07:12:25.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.244+0000 7faab4e00640 1 --2- 172.21.15.32:0/2672187850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab010fe20 0x7faab0112240 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.244+0000 7faab4e00640 1 --2- 172.21.15.32:0/2672187850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab0075e20 0x7faab010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.244+0000 7faab4e00640 1 --2- 172.21.15.32:0/2672187850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab00754d0 0x7faab00758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.244+0000 7faab4e00640 1 -- 172.21.15.32:0/2672187850 >> 172.21.15.32:0/2672187850 conn(0x7faab00ff290 msgr2=0x7faab01016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:25.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.244+0000 7faab4e00640 1 -- 172.21.15.32:0/2672187850 shutdown_connections 2024-08-31T07:12:25.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.244+0000 7faab4e00640 1 -- 172.21.15.32:0/2672187850 wait complete. 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faab4e00640 1 Processor -- start 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faab4e00640 1 -- start start 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faab4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab00754d0 0x7faab019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faab4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab0075e20 0x7faab019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faab4e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab010fe20 0x7faab01a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faab4e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faab01159b0 con 0x7faab00754d0 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faab4e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faab0115830 con 0x7faab0075e20 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faab4e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faab0115b30 con 0x7faab010fe20 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faaade00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab00754d0 0x7faab019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faaade00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab00754d0 0x7faab019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58630/0 (socket says 172.21.15.32:58630) 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.246+0000 7faaade00640 1 -- 172.21.15.32:0/1501854454 learned_addr learned my addr 172.21.15.32:0/1501854454 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:25.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaae800640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab010fe20 0x7faab01a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaad400640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab0075e20 0x7faab019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaade00640 1 -- 172.21.15.32:0/1501854454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab010fe20 msgr2=0x7faab01a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaade00640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab010fe20 0x7faab01a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaade00640 1 -- 172.21.15.32:0/1501854454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab0075e20 msgr2=0x7faab019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaade00640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab0075e20 0x7faab019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaade00640 1 -- 172.21.15.32:0/1501854454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab01a2210 con 0x7faab00754d0 2024-08-31T07:12:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaad400640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab0075e20 0x7faab019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaae800640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab010fe20 0x7faab01a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaade00640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab00754d0 0x7faab019c2e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faaa000e9f0 tx=0x7faaa000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.247+0000 7faaa6a00640 1 -- 172.21.15.32:0/1501854454 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa000cd80 con 0x7faab00754d0 2024-08-31T07:12:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.248+0000 7faaa6a00640 1 -- 172.21.15.32:0/1501854454 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa00108e0 con 0x7faab00754d0 2024-08-31T07:12:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.248+0000 7faaa6a00640 1 -- 172.21.15.32:0/1501854454 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0018980 con 0x7faab00754d0 2024-08-31T07:12:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.248+0000 7faab4e00640 1 -- 172.21.15.32:0/1501854454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab01a2500 con 0x7faab00754d0 2024-08-31T07:12:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.248+0000 7faab4e00640 1 -- 172.21.15.32:0/1501854454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab01a8c90 con 0x7faab00754d0 2024-08-31T07:12:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.249+0000 7faab4e00640 1 -- 172.21.15.32:0/1501854454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab00769e0 con 0x7faab00754d0 2024-08-31T07:12:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.253+0000 7faaa6a00640 1 -- 172.21.15.32:0/1501854454 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7faaa0010430 con 0x7faab00754d0 2024-08-31T07:12:25.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.254+0000 7faaa6a00640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa940b4c00 0x7faa940b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:25.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.254+0000 7faaa6a00640 1 -- 172.21.15.32:0/1501854454 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7faaa00e5680 con 0x7faab00754d0 2024-08-31T07:12:25.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.254+0000 7faaad400640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa940b4c00 0x7faa940b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:25.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.255+0000 7faaad400640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa940b4c00 0x7faa940b7110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faab01121c0 tx=0x7faa9c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.258+0000 7faaa6a00640 1 -- 172.21.15.32:0/1501854454 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faaa00ac6b0 con 0x7faab00754d0 2024-08-31T07:12:25.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:12:25.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.482+0000 7faab4e00640 1 -- 172.21.15.32:0/1501854454 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7faab010a040 con 0x7faa940b4c00 2024-08-31T07:12:25.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.491+0000 7faaa6a00640 1 -- 172.21.15.32:0/1501854454 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7faab010a040 con 0x7faa940b4c00 2024-08-31T07:12:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/ 1m 6m3h 2024-08-31T07:12:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 -- 172.21.15.32:0/1501854454 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa940b4c00 msgr2=0x7faa940b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa940b4c00 0x7faa940b7110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faab01121c0 tx=0x7faa9c005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 -- 172.21.15.32:0/1501854454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab00754d0 msgr2=0x7faab019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab00754d0 0x7faab019c2e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faaa000e9f0 tx=0x7faaa000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 -- 172.21.15.32:0/1501854454 shutdown_connections 2024-08-31T07:12:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa940b4c00 0x7faa940b7110 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab010fe20 0x7faab01a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab0075e20 0x7faab019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 --2- 172.21.15.32:0/1501854454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab00754d0 0x7faab019c2e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 -- 172.21.15.32:0/1501854454 >> 172.21.15.32:0/1501854454 conn(0x7faab00ff290 msgr2=0x7faab0109930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:25.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 -- 172.21.15.32:0/1501854454 shutdown_connections 2024-08-31T07:12:25.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:25.494+0000 7faab4e00640 1 -- 172.21.15.32:0/1501854454 wait complete. 2024-08-31T07:12:25.518 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-08-31T07:12:25.586 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-08-31T07:12:25.595 INFO:teuthology.run_tasks:Running task quiescer... 2024-08-31T07:12:25.643 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-31T07:12:25.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T07:12:25.693 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:25 smithi032 ceph-mon[17252]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:25.693 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:25 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:12:25.693 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:25 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:12:25.693 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:25 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:12:25.693 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:25 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:12:25.693 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:25 smithi032 ceph-mon[17252]: from='client.25402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:25.835 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:25.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:25 smithi073 ceph-mon[21995]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:25.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:25 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:12:25.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:25 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:12:25.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:25 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:12:25.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:25 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:12:25.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:25 smithi073 ceph-mon[21995]: from='client.25402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:25.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:25 smithi084 ceph-mon[22387]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:25.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:25 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:12:25.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:25 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:12:25.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:25 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:12:25.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:25 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:12:25.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:25 smithi084 ceph-mon[22387]: from='client.25402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:26.491 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.492+0000 7fbaeea00640 1 -- 172.21.15.32:0/3722764391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbae8103920 msgr2=0x7fbae8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.492+0000 7fbaeea00640 1 --2- 172.21.15.32:0/3722764391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbae8103920 0x7fbae8103d30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009900 tx=0x7fbad802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.492+0000 7fbaeea00640 1 -- 172.21.15.32:0/3722764391 shutdown_connections 2024-08-31T07:12:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.492+0000 7fbaeea00640 1 --2- 172.21.15.32:0/3722764391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae8109bb0 0x7fbae81103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.492+0000 7fbaeea00640 1 --2- 172.21.15.32:0/3722764391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae8109260 0x7fbae8109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.492+0000 7fbaeea00640 1 --2- 172.21.15.32:0/3722764391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbae8103920 0x7fbae8103d30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.492+0000 7fbaeea00640 1 -- 172.21.15.32:0/3722764391 >> 172.21.15.32:0/3722764391 conn(0x7fbae80ff330 msgr2=0x7fbae8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.492+0000 7fbaeea00640 1 -- 172.21.15.32:0/3722764391 shutdown_connections 2024-08-31T07:12:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.492+0000 7fbaeea00640 1 -- 172.21.15.32:0/3722764391 wait complete. 2024-08-31T07:12:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.493+0000 7fbaeea00640 1 Processor -- start 2024-08-31T07:12:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.493+0000 7fbaeea00640 1 -- start start 2024-08-31T07:12:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbaeea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae8103920 0x7fbae80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbaeea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae8109260 0x7fbae80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbaeea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbae8109bb0 0x7fbae8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbaeea00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbae8113870 con 0x7fbae8103920 2024-08-31T07:12:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbaeea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbae81136f0 con 0x7fbae8109260 2024-08-31T07:12:26.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbaeea00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbae81139f0 con 0x7fbae8109bb0 2024-08-31T07:12:26.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae8103920 0x7fbae80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae8103920 0x7fbae80754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51970/0 (socket says 172.21.15.32:51970) 2024-08-31T07:12:26.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbae7e00640 1 -- 172.21.15.32:0/1002906603 learned_addr learned my addr 172.21.15.32:0/1002906603 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:26.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbae7400640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae8109260 0x7fbae80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.494+0000 7fbaeca00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbae8109bb0 0x7fbae8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.495+0000 7fbae7e00640 1 -- 172.21.15.32:0/1002906603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbae8109bb0 msgr2=0x7fbae8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:26.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.495+0000 7fbae7e00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbae8109bb0 0x7fbae8075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.495+0000 7fbae7e00640 1 -- 172.21.15.32:0/1002906603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae8109260 msgr2=0x7fbae80759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:26.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.495+0000 7fbae7e00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae8109260 0x7fbae80759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.495+0000 7fbae7e00640 1 -- 172.21.15.32:0/1002906603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae81a8580 con 0x7fbae8103920 2024-08-31T07:12:26.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.495+0000 7fbae7e00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae8103920 0x7fbae80754b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbad8005940 tx=0x7fbad80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:26.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.496+0000 7fbae4c00640 1 -- 172.21.15.32:0/1002906603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad803f030 con 0x7fbae8103920 2024-08-31T07:12:26.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.496+0000 7fbaeea00640 1 -- 172.21.15.32:0/1002906603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae81a8810 con 0x7fbae8103920 2024-08-31T07:12:26.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.496+0000 7fbae4c00640 1 -- 172.21.15.32:0/1002906603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad80044c0 con 0x7fbae8103920 2024-08-31T07:12:26.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.496+0000 7fbaeea00640 1 -- 172.21.15.32:0/1002906603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae81a8d40 con 0x7fbae8103920 2024-08-31T07:12:26.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.496+0000 7fbae4c00640 1 -- 172.21.15.32:0/1002906603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8035070 con 0x7fbae8103920 2024-08-31T07:12:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.499+0000 7fbae4c00640 1 -- 172.21.15.32:0/1002906603 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7fbad8042030 con 0x7fbae8103920 2024-08-31T07:12:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.500+0000 7fbaf56d9740 1 Processor -- start 2024-08-31T07:12:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.501+0000 7fbaf56d9740 1 -- start start 2024-08-31T07:12:26.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.501+0000 7fbae4c00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbacc0b4c90 0x7fbacc0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.501+0000 7fbae4c00640 1 -- 172.21.15.32:0/1002906603 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fbad8105740 con 0x7fbae8103920 2024-08-31T07:12:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.501+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 0x563caf71f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.501+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 0x563caf71fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.501+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf7203d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.501+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563caf696770 con 0x563caf37a070 2024-08-31T07:12:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.501+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563caf685d00 con 0x563caf359390 2024-08-31T07:12:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.501+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563caf689cb0 con 0x563caf3511b0 2024-08-31T07:12:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.501+0000 7fbaeca00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbacc0b4c90 0x7fbacc0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 0x563caf71f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbae7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 0x563caf71fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 0x563caf71f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51978/0 (socket says 172.21.15.32:51978) 2024-08-31T07:12:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbae7e00640 1 -- 172.21.15.32:0/1773743844 learned_addr learned my addr 172.21.15.32:0/1773743844 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbaeca00640 1 --2- 172.21.15.32:0/1773743844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf7203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbae7e00640 1 -- 172.21.15.32:0/1773743844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 msgr2=0x563caf71fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbae7e00640 1 --2- 172.21.15.32:0/1773743844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 0x563caf71fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbae7e00640 1 -- 172.21.15.32:0/1773743844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 msgr2=0x563caf7203d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbae7e00640 1 --2- 172.21.15.32:0/1773743844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf7203d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbae7e00640 1 -- 172.21.15.32:0/1773743844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563caf3e5650 con 0x563caf37a070 2024-08-31T07:12:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.502+0000 7fbaeca00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbacc0b4c90 0x7fbacc0b71a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbae0005fd0 tx=0x7fbae0009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.503+0000 7fbae7e00640 1 --2- 172.21.15.32:0/1773743844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 0x563caf71f950 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbad80c8040 tx=0x7fbad80cbb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9674a1f12c99542 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.503+0000 7fbadd200640 1 -- 172.21.15.32:0/1773743844 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8049070 con 0x563caf37a070 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.503+0000 7fbadd200640 1 -- 172.21.15.32:0/1773743844 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad80d5230 con 0x563caf37a070 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.503+0000 7fbadd200640 1 -- 172.21.15.32:0/1773743844 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad80d4da0 con 0x563caf37a070 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.504+0000 7fbaf56d9740 1 -- 172.21.15.32:0/1773743844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 msgr2=0x563caf71f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.504+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/1773743844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 0x563caf71f950 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbad80c8040 tx=0x7fbad80cbb90 comp rx=0 tx=0).stop 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.504+0000 7fbaf56d9740 1 -- 172.21.15.32:0/1773743844 shutdown_connections 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.504+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/1773743844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf7203d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.504+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/1773743844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 0x563caf71fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.504+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/1773743844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 0x563caf71f950 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.504+0000 7fbaf56d9740 1 -- 172.21.15.32:0/1773743844 >> 172.21.15.32:0/1773743844 conn(0x563caf36e460 msgr2=0x563caf71d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.504+0000 7fbaf56d9740 1 -- 172.21.15.32:0/1773743844 shutdown_connections 2024-08-31T07:12:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.504+0000 7fbaf56d9740 1 -- 172.21.15.32:0/1773743844 wait complete. 2024-08-31T07:12:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.505+0000 7fbaf56d9740 1 -- start start 2024-08-31T07:12:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.505+0000 7fbaf56d9740 1 Processor -- start 2024-08-31T07:12:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 0x563caf72b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf37a070 0x563caf737110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563caf4e8700 con 0x563caf3511b0 2024-08-31T07:12:26.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563caf60ccb0 con 0x563caf359390 2024-08-31T07:12:26.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563caf72def0 con 0x563caf37a070 2024-08-31T07:12:26.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbaf56d9740 10 client.0 _ll_register_callbacks cb 0x563caf632430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:26.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 0x563caf72b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbae7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 0x563caf72b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51982/0 (socket says 172.21.15.32:51982) 2024-08-31T07:12:26.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbae7e00640 1 -- 172.21.15.32:0/485043787 learned_addr learned my addr 172.21.15.32:0/485043787 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:26.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbae7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54862/0 (socket says 172.21.15.32:54862) 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.506+0000 7fbaeca00640 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf37a070 0x563caf737110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.507+0000 7fbae7400640 1 -- 172.21.15.32:0/485043787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf37a070 msgr2=0x563caf737110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.507+0000 7fbae7400640 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf37a070 0x563caf737110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.507+0000 7fbae7400640 1 -- 172.21.15.32:0/485043787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 msgr2=0x563caf72b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.507+0000 7fbae7400640 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 0x563caf72b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.507+0000 7fbae7400640 1 -- 172.21.15.32:0/485043787 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563caf64d870 con 0x563caf359390 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.507+0000 7fbae7400640 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72bb00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbae8101a20 tx=0x7fbad4012b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ca2285541a080e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.507+0000 7fbad1600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.507+0000 7fbad1600640 1 -- 172.21.15.32:0/485043787 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4004020 con 0x563caf359390 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.507+0000 7fbaf56d9740 1 -- 172.21.15.32:0/485043787 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad80d5230 con 0x563caf359390 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.507+0000 7fbad1600640 1 -- 172.21.15.32:0/485043787 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fbad40041c0 con 0x563caf359390 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.508+0000 7fbad1600640 1 -- 172.21.15.32:0/485043787 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbad40043c0 con 0x563caf359390 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.508+0000 7fbaf56d9740 10 client.25436 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.508+0000 7fbaf56d9740 1 -- 172.21.15.32:0/485043787 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563caf739f30 con 0x563caf359390 2024-08-31T07:12:26.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.508+0000 7fbad1600640 1 -- 172.21.15.32:0/485043787 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad80d5230 con 0x563caf359390 2024-08-31T07:12:26.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.508+0000 7fbad1600640 1 -- 172.21.15.32:0/485043787 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbad4003b50 con 0x563caf359390 2024-08-31T07:12:26.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.508+0000 7fbaf56d9740 10 client.25436 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:26.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.508+0000 7fbaf56d9740 10 client.25436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:12:26.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.508+0000 7fbaf56d9740 20 client.25436 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:26.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.509+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563caf3389f0 0x563caf73d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.509+0000 7fbaf56d9740 4 client.25436 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.509+0000 7fbaf56d9740 1 -- 172.21.15.32:0/485043787 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563caf6986c0 con 0x563caf3389f0 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.509+0000 7fbae7e00640 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563caf3389f0 0x563caf73d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.509+0000 7fbae7e00640 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563caf3389f0 0x563caf73d370 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9cd4b55a323445 server_cookie=850b9c37664da0e9 in_seq=0 out_seq=0 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.509+0000 7fbad1600640 10 client.25436 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.510+0000 7fbad1600640 1 -- 172.21.15.32:0/485043787 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563caf6986c0 con 0x563caf3389f0 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.510+0000 7fbad1600640 10 client.25436 handle_command_reply: tid=0 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.510+0000 7fbaf56d9740 1 client.25436 shutdown 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 1 -- 172.21.15.32:0/485043787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 msgr2=0x563caf72bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72bb00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbae8101a20 tx=0x7fbad4012b40 comp rx=0 tx=0).stop 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 1 -- 172.21.15.32:0/485043787 shutdown_connections 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563caf3389f0 0x563caf73d370 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf37a070 0x563caf737110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72bb00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/485043787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 0x563caf72b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbad1600640 0 client.25436 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:26.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 1 -- 172.21.15.32:0/485043787 >> 172.21.15.32:0/485043787 conn(0x563caf36e460 msgr2=0x563caf720980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:26.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 1 -- 172.21.15.32:0/485043787 shutdown_connections 2024-08-31T07:12:26.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 1 -- 172.21.15.32:0/485043787 wait complete. 2024-08-31T07:12:26.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.511+0000 7fbaf56d9740 20 client.25436 trim_cache size 0 max 16384 2024-08-31T07:12:26.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.531+0000 7fbaf56d9740 1 Processor -- start 2024-08-31T07:12:26.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.531+0000 7fbaf56d9740 1 -- start start 2024-08-31T07:12:26.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.531+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 0x563caf72bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.531+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf725a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.531+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 0x563caf720980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.531+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbae81b15a0 con 0x563caf37a070 2024-08-31T07:12:26.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbae81b30e0 con 0x563caf359390 2024-08-31T07:12:26.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbae81b1c70 con 0x563caf3511b0 2024-08-31T07:12:26.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 0x563caf72bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 0x563caf72bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36202/0 (socket says 172.21.15.32:36202) 2024-08-31T07:12:26.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbae7e00640 1 -- 172.21.15.32:0/3003859006 learned_addr learned my addr 172.21.15.32:0/3003859006 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:26.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbaeca00640 1 --2- 172.21.15.32:0/3003859006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 0x563caf720980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbae7400640 1 --2- 172.21.15.32:0/3003859006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf725a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbae7e00640 1 -- 172.21.15.32:0/3003859006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 msgr2=0x563caf725a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:26.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbae7e00640 1 --2- 172.21.15.32:0/3003859006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf725a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbae7e00640 1 -- 172.21.15.32:0/3003859006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 msgr2=0x563caf720980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:26.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbae7e00640 1 --2- 172.21.15.32:0/3003859006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 0x563caf720980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.532+0000 7fbae7e00640 1 -- 172.21.15.32:0/3003859006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563caf725fb0 con 0x563caf3511b0 2024-08-31T07:12:26.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.533+0000 7fbae7e00640 1 --2- 172.21.15.32:0/3003859006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 0x563caf72bbf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbad80cbcc0 tx=0x7fbad8005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fcf612950abbb5f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:26.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.533+0000 7fbad3e00640 1 -- 172.21.15.32:0/3003859006 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad80b7030 con 0x563caf3511b0 2024-08-31T07:12:26.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.533+0000 7fbad3e00640 1 -- 172.21.15.32:0/3003859006 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad80d0550 con 0x563caf3511b0 2024-08-31T07:12:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.533+0000 7fbad3e00640 1 -- 172.21.15.32:0/3003859006 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8049040 con 0x563caf3511b0 2024-08-31T07:12:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.534+0000 7fbaf56d9740 1 -- 172.21.15.32:0/3003859006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 msgr2=0x563caf72bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.534+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/3003859006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 0x563caf72bbf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbad80cbcc0 tx=0x7fbad8005c90 comp rx=0 tx=0).stop 2024-08-31T07:12:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.534+0000 7fbaf56d9740 1 -- 172.21.15.32:0/3003859006 shutdown_connections 2024-08-31T07:12:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.534+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/3003859006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf37a070 0x563caf720980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.534+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/3003859006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf725a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.534+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/3003859006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf3511b0 0x563caf72bbf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.534+0000 7fbaf56d9740 1 -- 172.21.15.32:0/3003859006 >> 172.21.15.32:0/3003859006 conn(0x563caf36e460 msgr2=0x563caf729c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.534+0000 7fbaf56d9740 1 -- 172.21.15.32:0/3003859006 shutdown_connections 2024-08-31T07:12:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.534+0000 7fbaf56d9740 1 -- 172.21.15.32:0/3003859006 wait complete. 2024-08-31T07:12:26.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.535+0000 7fbaf56d9740 1 -- start start 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.535+0000 7fbaf56d9740 1 Processor -- start 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 0x563caf72eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbaf56d9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf37a070 0x563caf72f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbae81b2a10 con 0x563caf3511b0 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbae81b2340 con 0x563caf359390 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbaf56d9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbae81af580 con 0x563caf37a070 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbaf56d9740 10 client.0 _ll_register_callbacks cb 0x7fbad4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbae7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 0x563caf72eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbaeca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf37a070 0x563caf72f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbae7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54888/0 (socket says 172.21.15.32:54888) 2024-08-31T07:12:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbae7400640 1 -- 172.21.15.32:0/2881590282 learned_addr learned my addr 172.21.15.32:0/2881590282 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.536+0000 7fbae7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 0x563caf72eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52014/0 (socket says 172.21.15.32:52014) 2024-08-31T07:12:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.537+0000 7fbae7400640 1 -- 172.21.15.32:0/2881590282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf37a070 msgr2=0x563caf72f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.537+0000 7fbae7400640 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf37a070 0x563caf72f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.537+0000 7fbae7400640 1 -- 172.21.15.32:0/2881590282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 msgr2=0x563caf72eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.537+0000 7fbae7400640 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 0x563caf72eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.537+0000 7fbae7400640 1 -- 172.21.15.32:0/2881590282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563caf7a6d70 con 0x563caf359390 2024-08-31T07:12:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.537+0000 7fbae7400640 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72f410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbad40059f0 tx=0x7fbad4005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ac76a6c692267e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.537+0000 7fbad1600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:12:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.537+0000 7fbad1600640 1 -- 172.21.15.32:0/2881590282 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad400f8f0 con 0x563caf359390 2024-08-31T07:12:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.537+0000 7fbad1600640 1 -- 172.21.15.32:0/2881590282 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fbad400fa90 con 0x563caf359390 2024-08-31T07:12:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.537+0000 7fbaf56d9740 1 -- 172.21.15.32:0/2881590282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad80d0550 con 0x563caf359390 2024-08-31T07:12:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.538+0000 7fbad1600640 1 -- 172.21.15.32:0/2881590282 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbad400fc90 con 0x563caf359390 2024-08-31T07:12:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.538+0000 7fbaf56d9740 10 client.25442 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.538+0000 7fbaf56d9740 1 -- 172.21.15.32:0/2881590282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563caf7a7f80 con 0x563caf359390 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.538+0000 7fbad1600640 1 -- 172.21.15.32:0/2881590282 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad80d0550 con 0x563caf359390 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.538+0000 7fbad1600640 1 -- 172.21.15.32:0/2881590282 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbad402e030 con 0x563caf359390 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.539+0000 7fbaf56d9740 10 client.25442 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.539+0000 7fbaf56d9740 10 client.25442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.539+0000 7fbaf56d9740 20 client.25442 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.539+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563caf30dda0 0x563caf7aa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.539+0000 7fbaf56d9740 4 client.25442 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.539+0000 7fbaf56d9740 1 -- 172.21.15.32:0/2881590282 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563caf694160 con 0x563caf30dda0 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.539+0000 7fbae7e00640 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563caf30dda0 0x563caf7aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbae7e00640 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563caf30dda0 0x563caf7aa720 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c273b8596d2fad server_cookie=54a77f612e9b141e in_seq=0 out_seq=0 2024-08-31T07:12:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbad1600640 10 client.25442 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:26.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbad1600640 1 -- 172.21.15.32:0/2881590282 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x563caf694160 con 0x563caf30dda0 2024-08-31T07:12:26.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbad1600640 10 client.25442 handle_command_reply: tid=0 2024-08-31T07:12:26.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbaf56d9740 1 client.25442 shutdown 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbaf56d9740 1 -- 172.21.15.32:0/2881590282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 msgr2=0x563caf72f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72f410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbad40059f0 tx=0x7fbad4005600 comp rx=0 tx=0).stop 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbaf56d9740 1 -- 172.21.15.32:0/2881590282 shutdown_connections 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563caf30dda0 0x563caf7aa720 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563caf37a070 0x563caf72f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbad1600640 0 client.25442 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563caf359390 0x563caf72f410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbaf56d9740 1 --2- 172.21.15.32:0/2881590282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563caf3511b0 0x563caf72eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.540+0000 7fbaf56d9740 1 -- 172.21.15.32:0/2881590282 >> 172.21.15.32:0/2881590282 conn(0x563caf36e460 msgr2=0x563caf720980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.541+0000 7fbaf56d9740 1 -- 172.21.15.32:0/2881590282 shutdown_connections 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.541+0000 7fbaf56d9740 1 -- 172.21.15.32:0/2881590282 wait complete. 2024-08-31T07:12:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.541+0000 7fbaf56d9740 20 client.25442 trim_cache size 0 max 16384 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "323cd93f-7bb8-4706-b7dc-0b247836a042", 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:12:26.544 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:12:26.545 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:12:26.545 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:12:26.545 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:12:26.545 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:12:26.545 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:12:26.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:12:26.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:12:26.545 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 -- 172.21.15.32:0/1002906603 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbacc0b4c90 msgr2=0x7fbacc0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbacc0b4c90 0x7fbacc0b71a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbae0005fd0 tx=0x7fbae0009500 comp rx=0 tx=0).stop 2024-08-31T07:12:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 -- 172.21.15.32:0/1002906603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae8103920 msgr2=0x7fbae80754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae8103920 0x7fbae80754b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbad8005940 tx=0x7fbad80040c0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 -- 172.21.15.32:0/1002906603 shutdown_connections 2024-08-31T07:12:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbacc0b4c90 0x7fbacc0b71a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbae8109bb0 0x7fbae8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae8109260 0x7fbae80759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 --2- 172.21.15.32:0/1002906603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae8103920 0x7fbae80754b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 -- 172.21.15.32:0/1002906603 >> 172.21.15.32:0/1002906603 conn(0x7fbae80ff330 msgr2=0x7fbae8107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 -- 172.21.15.32:0/1002906603 shutdown_connections 2024-08-31T07:12:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:26.542+0000 7fbaeea00640 1 -- 172.21.15.32:0/1002906603 wait complete. 2024-08-31T07:12:26.565 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:26 smithi032 ceph-mon[17252]: pgmap v358: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-08-31T07:12:26.565 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:26 smithi032 ceph-mon[17252]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:26.565 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:26 smithi032 ceph-mon[17252]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:26.565 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:26 smithi032 ceph-mon[17252]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-08-31T07:12:26.830 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:26 smithi073 ceph-mon[21995]: pgmap v358: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-08-31T07:12:26.830 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:26 smithi073 ceph-mon[21995]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:26.830 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:26 smithi073 ceph-mon[21995]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:26.830 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:26 smithi073 ceph-mon[21995]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-08-31T07:12:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:26 smithi084 ceph-mon[22387]: pgmap v358: 97 pgs: 97 active+clean; 650 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-08-31T07:12:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:26 smithi084 ceph-mon[22387]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:26 smithi084 ceph-mon[22387]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:12:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:26 smithi084 ceph-mon[22387]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-08-31T07:12:27.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.530+0000 7f39ade00640 1 -- 172.21.15.32:0/1336819895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a8077120 msgr2=0x7f39a8075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:27.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.530+0000 7f39ade00640 1 --2- 172.21.15.32:0/1336819895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a8077120 0x7f39a8075550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f399c009900 tx=0x7f399c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:27.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.530+0000 7f39ade00640 1 -- 172.21.15.32:0/1336819895 shutdown_connections 2024-08-31T07:12:27.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.530+0000 7f39ade00640 1 --2- 172.21.15.32:0/1336819895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a8114020 0x7f39a81164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:27.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.530+0000 7f39ade00640 1 --2- 172.21.15.32:0/1336819895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39a8075b70 0x7f39a8075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:27.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.530+0000 7f39ade00640 1 --2- 172.21.15.32:0/1336819895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a8077120 0x7f39a8075550 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:27.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.530+0000 7f39ade00640 1 -- 172.21.15.32:0/1336819895 >> 172.21.15.32:0/1336819895 conn(0x7f39a80ff1e0 msgr2=0x7f39a8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:27.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.530+0000 7f39ade00640 1 -- 172.21.15.32:0/1336819895 shutdown_connections 2024-08-31T07:12:27.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.530+0000 7f39ade00640 1 -- 172.21.15.32:0/1336819895 wait complete. 2024-08-31T07:12:27.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.531+0000 7f39ade00640 1 Processor -- start 2024-08-31T07:12:27.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39ade00640 1 -- start start 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39ade00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39a8075b70 0x7f39a81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39ade00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a8077120 0x7f39a81a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39ade00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a8114020 0x7f39a81a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39ade00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8119cb0 con 0x7f39a8075b70 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39ade00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8119b30 con 0x7f39a8077120 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39ade00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8119e30 con 0x7f39a8114020 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39aca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39a8075b70 0x7f39a81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39aca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39a8075b70 0x7f39a81a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52034/0 (socket says 172.21.15.32:52034) 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39aca00640 1 -- 172.21.15.32:0/3256307883 learned_addr learned my addr 172.21.15.32:0/3256307883 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39ad400640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a8114020 0x7f39a81a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.532+0000 7f39a7e00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a8077120 0x7f39a81a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.533+0000 7f39aca00640 1 -- 172.21.15.32:0/3256307883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a8114020 msgr2=0x7f39a81a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:27.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.533+0000 7f39aca00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a8114020 0x7f39a81a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:27.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.533+0000 7f39aca00640 1 -- 172.21.15.32:0/3256307883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a8077120 msgr2=0x7f39a81a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:27.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.533+0000 7f39aca00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a8077120 0x7f39a81a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:27.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.533+0000 7f39aca00640 1 -- 172.21.15.32:0/3256307883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a81a6610 con 0x7f39a8075b70 2024-08-31T07:12:27.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.533+0000 7f39a7e00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a8077120 0x7f39a81a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:27.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.533+0000 7f39aca00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39a8075b70 0x7f39a81a06a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f399c005ab0 tx=0x7f399c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:27.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.534+0000 7f39a5600640 1 -- 172.21.15.32:0/3256307883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c03f070 con 0x7f39a8075b70 2024-08-31T07:12:27.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.534+0000 7f39ade00640 1 -- 172.21.15.32:0/3256307883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a81a68a0 con 0x7f39a8075b70 2024-08-31T07:12:27.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.534+0000 7f39a5600640 1 -- 172.21.15.32:0/3256307883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f399c031b60 con 0x7f39a8075b70 2024-08-31T07:12:27.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.534+0000 7f39a5600640 1 -- 172.21.15.32:0/3256307883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c035030 con 0x7f39a8075b70 2024-08-31T07:12:27.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.534+0000 7f39ade00640 1 -- 172.21.15.32:0/3256307883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a81ad060 con 0x7f39a8075b70 2024-08-31T07:12:27.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.535+0000 7f39ade00640 1 -- 172.21.15.32:0/3256307883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a8075f80 con 0x7f39a8075b70 2024-08-31T07:12:27.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.538+0000 7f39a5600640 1 -- 172.21.15.32:0/3256307883 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f399c042070 con 0x7f39a8075b70 2024-08-31T07:12:27.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.539+0000 7f39a5600640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39900b4c90 0x7f39900b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:27.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.540+0000 7f39a5600640 1 -- 172.21.15.32:0/3256307883 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f399c105a80 con 0x7f39a8075b70 2024-08-31T07:12:27.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.540+0000 7f39a7e00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39900b4c90 0x7f39900b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:27.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.541+0000 7f39a7e00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39900b4c90 0x7f39900b71a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f39940095f0 tx=0x7f3994005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:27.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.544+0000 7f39a5600640 1 -- 172.21.15.32:0/3256307883 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f399c107e20 con 0x7f39a8075b70 2024-08-31T07:12:27.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.741+0000 7f39ade00640 1 -- 172.21.15.32:0/3256307883 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39a80635a0 con 0x7f39900b4c90 2024-08-31T07:12:27.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.747+0000 7f39a5600640 1 -- 172.21.15.32:0/3256307883 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259267 (secure 0 0 0) 0x7f39a80635a0 con 0x7f39900b4c90 2024-08-31T07:12:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 -- 172.21.15.32:0/3256307883 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39900b4c90 msgr2=0x7f39900b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39900b4c90 0x7f39900b71a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f39940095f0 tx=0x7f3994005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 -- 172.21.15.32:0/3256307883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39a8075b70 msgr2=0x7f39a81a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39a8075b70 0x7f39a81a06a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f399c005ab0 tx=0x7f399c003c10 comp rx=0 tx=0).stop 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 -- 172.21.15.32:0/3256307883 shutdown_connections 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39900b4c90 0x7f39900b71a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39a8114020 0x7f39a81a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39a8077120 0x7f39a81a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 --2- 172.21.15.32:0/3256307883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39a8075b70 0x7f39a81a06a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 -- 172.21.15.32:0/3256307883 >> 172.21.15.32:0/3256307883 conn(0x7f39a80ff1e0 msgr2=0x7f39a810ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 -- 172.21.15.32:0/3256307883 shutdown_connections 2024-08-31T07:12:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:27.751+0000 7f39ade00640 1 -- 172.21.15.32:0/3256307883 wait complete. 2024-08-31T07:12:28.398 INFO:tasks.cephfs.filesystem:scrub status for tag:323cd93f-7bb8-4706-b7dc-0b247836a042 - {'path': '/', 'tag': '323cd93f-7bb8-4706-b7dc-0b247836a042', 'options': 'recursive,force'} 2024-08-31T07:12:28.750 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-08-31T07:12:26.800628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1452694,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1866,"num_read_kb":1173,"num_write":614,"num_write_kb":4397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2952490,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":965,"ondisk_log_size":965,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364336,"kb_used_data":10924,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577488,"statfs":{"total":1151940427776,"available":1151567347712,"internally_reserved":0,"allocated":11186176,"data_stored":8363892,"data_compressed":234486,"data_compressed_allocated":307200,"data_compressed_original":3121152,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":792365,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":452,"num_read_kb":362,"num_write":227,"num_write_kb":2095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198399+0000","last_change":"2024-08-31T07:09:47.665725+0000","last_active":"2024-08-31T07:11:45.198399+0000","last_peered":"2024-08-31T07:11:45.198399+0000","last_clean":"2024-08-31T07:11:45.198399+0000","last_became_active":"2024-08-31T07:09:47.665368+0000","last_became_peered":"2024-08-31T07:09:47.665368+0000","last_unstale":"2024-08-31T07:11:45.198399+0000","last_undegraded":"2024-08-31T07:11:45.198399+0000","last_fullsized":"2024-08-31T07:11:45.198399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:27:01.636099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192497+0000","last_change":"2024-08-31T07:09:47.676882+0000","last_active":"2024-08-31T07:11:45.192497+0000","last_peered":"2024-08-31T07:11:45.192497+0000","last_clean":"2024-08-31T07:11:45.192497+0000","last_became_active":"2024-08-31T07:09:47.674825+0000","last_became_peered":"2024-08-31T07:09:47.674825+0000","last_unstale":"2024-08-31T07:11:45.192497+0000","last_undegraded":"2024-08-31T07:11:45.192497+0000","last_fullsized":"2024-08-31T07:11:45.192497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:09.148398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199170+0000","last_change":"2024-08-31T07:09:47.674480+0000","last_active":"2024-08-31T07:11:45.199170+0000","last_peered":"2024-08-31T07:11:45.199170+0000","last_clean":"2024-08-31T07:11:45.199170+0000","last_became_active":"2024-08-31T07:09:47.672316+0000","last_became_peered":"2024-08-31T07:09:47.672316+0000","last_unstale":"2024-08-31T07:11:45.199170+0000","last_undegraded":"2024-08-31T07:11:45.199170+0000","last_fullsized":"2024-08-31T07:11:45.199170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:28:13.231039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.172744+0000","last_change":"2024-08-31T07:09:47.659276+0000","last_active":"2024-08-31T07:11:45.172744+0000","last_peered":"2024-08-31T07:11:45.172744+0000","last_clean":"2024-08-31T07:11:45.172744+0000","last_became_active":"2024-08-31T07:09:47.658924+0000","last_became_peered":"2024-08-31T07:09:47.658924+0000","last_unstale":"2024-08-31T07:11:45.172744+0000","last_undegraded":"2024-08-31T07:11:45.172744+0000","last_fullsized":"2024-08-31T07:11:45.172744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:01:08.134245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.174959+0000","last_change":"2024-08-31T07:09:47.667260+0000","last_active":"2024-08-31T07:11:45.174959+0000","last_peered":"2024-08-31T07:11:45.174959+0000","last_clean":"2024-08-31T07:11:45.174959+0000","last_became_active":"2024-08-31T07:09:47.664572+0000","last_became_peered":"2024-08-31T07:09:47.664572+0000","last_unstale":"2024-08-31T07:11:45.174959+0000","last_undegraded":"2024-08-31T07:11:45.174959+0000","last_fullsized":"2024-08-31T07:11:45.174959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:11.336985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193916+0000","last_change":"2024-08-31T07:09:47.661181+0000","last_active":"2024-08-31T07:11:45.193916+0000","last_peered":"2024-08-31T07:11:45.193916+0000","last_clean":"2024-08-31T07:11:45.193916+0000","last_became_active":"2024-08-31T07:09:47.660761+0000","last_became_peered":"2024-08-31T07:09:47.660761+0000","last_unstale":"2024-08-31T07:11:45.193916+0000","last_undegraded":"2024-08-31T07:11:45.193916+0000","last_fullsized":"2024-08-31T07:11:45.193916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:50:54.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196590+0000","last_change":"2024-08-31T07:09:47.668762+0000","last_active":"2024-08-31T07:11:45.196590+0000","last_peered":"2024-08-31T07:11:45.196590+0000","last_clean":"2024-08-31T07:11:45.196590+0000","last_became_active":"2024-08-31T07:09:47.668406+0000","last_became_peered":"2024-08-31T07:09:47.668406+0000","last_unstale":"2024-08-31T07:11:45.196590+0000","last_undegraded":"2024-08-31T07:11:45.196590+0000","last_fullsized":"2024-08-31T07:11:45.196590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:43:39.868144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.191067+0000","last_change":"2024-08-31T07:09:47.669455+0000","last_active":"2024-08-31T07:11:45.191067+0000","last_peered":"2024-08-31T07:11:45.191067+0000","last_clean":"2024-08-31T07:11:45.191067+0000","last_became_active":"2024-08-31T07:09:47.663669+0000","last_became_peered":"2024-08-31T07:09:47.663669+0000","last_unstale":"2024-08-31T07:11:45.191067+0000","last_undegraded":"2024-08-31T07:11:45.191067+0000","last_fullsized":"2024-08-31T07:11:45.191067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:22:59.685012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194827+0000","last_change":"2024-08-31T07:09:47.664214+0000","last_active":"2024-08-31T07:11:45.194827+0000","last_peered":"2024-08-31T07:11:45.194827+0000","last_clean":"2024-08-31T07:11:45.194827+0000","last_became_active":"2024-08-31T07:09:47.663832+0000","last_became_peered":"2024-08-31T07:09:47.663832+0000","last_unstale":"2024-08-31T07:11:45.194827+0000","last_undegraded":"2024-08-31T07:11:45.194827+0000","last_fullsized":"2024-08-31T07:11:45.194827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:29:14.107212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194866+0000","last_change":"2024-08-31T07:10:46.326410+0000","last_active":"2024-08-31T07:11:45.194866+0000","last_peered":"2024-08-31T07:11:45.194866+0000","last_clean":"2024-08-31T07:11:45.194866+0000","last_became_active":"2024-08-31T07:10:46.325958+0000","last_became_peered":"2024-08-31T07:10:46.325958+0000","last_unstale":"2024-08-31T07:11:45.194866+0000","last_undegraded":"2024-08-31T07:11:45.194866+0000","last_fullsized":"2024-08-31T07:11:45.194866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:19.230393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196579+0000","last_change":"2024-08-31T07:09:47.666951+0000","last_active":"2024-08-31T07:11:45.196579+0000","last_peered":"2024-08-31T07:11:45.196579+0000","last_clean":"2024-08-31T07:11:45.196579+0000","last_became_active":"2024-08-31T07:09:47.666475+0000","last_became_peered":"2024-08-31T07:09:47.666475+0000","last_unstale":"2024-08-31T07:11:45.196579+0000","last_undegraded":"2024-08-31T07:11:45.196579+0000","last_fullsized":"2024-08-31T07:11:45.196579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:58:42.242086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.169998+0000","last_change":"2024-08-31T07:09:47.658344+0000","last_active":"2024-08-31T07:11:45.169998+0000","last_peered":"2024-08-31T07:11:45.169998+0000","last_clean":"2024-08-31T07:11:45.169998+0000","last_became_active":"2024-08-31T07:09:47.657991+0000","last_became_peered":"2024-08-31T07:09:47.657991+0000","last_unstale":"2024-08-31T07:11:45.169998+0000","last_undegraded":"2024-08-31T07:11:45.169998+0000","last_fullsized":"2024-08-31T07:11:45.169998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:25:59.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196676+0000","last_change":"2024-08-31T07:09:47.661644+0000","last_active":"2024-08-31T07:11:45.196676+0000","last_peered":"2024-08-31T07:11:45.196676+0000","last_clean":"2024-08-31T07:11:45.196676+0000","last_became_active":"2024-08-31T07:09:47.661248+0000","last_became_peered":"2024-08-31T07:09:47.661248+0000","last_unstale":"2024-08-31T07:11:45.196676+0000","last_undegraded":"2024-08-31T07:11:45.196676+0000","last_fullsized":"2024-08-31T07:11:45.196676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:42.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195194+0000","last_change":"2024-08-31T07:09:47.680052+0000","last_active":"2024-08-31T07:11:45.195194+0000","last_peered":"2024-08-31T07:11:45.195194+0000","last_clean":"2024-08-31T07:11:45.195194+0000","last_became_active":"2024-08-31T07:09:47.679604+0000","last_became_peered":"2024-08-31T07:09:47.679604+0000","last_unstale":"2024-08-31T07:11:45.195194+0000","last_undegraded":"2024-08-31T07:11:45.195194+0000","last_fullsized":"2024-08-31T07:11:45.195194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:23:18.937470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195477+0000","last_change":"2024-08-31T07:09:47.677984+0000","last_active":"2024-08-31T07:11:45.195477+0000","last_peered":"2024-08-31T07:11:45.195477+0000","last_clean":"2024-08-31T07:11:45.195477+0000","last_became_active":"2024-08-31T07:09:47.674957+0000","last_became_peered":"2024-08-31T07:09:47.674957+0000","last_unstale":"2024-08-31T07:11:45.195477+0000","last_undegraded":"2024-08-31T07:11:45.195477+0000","last_fullsized":"2024-08-31T07:11:45.195477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:08:26.503746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198878+0000","last_change":"2024-08-31T07:09:47.672625+0000","last_active":"2024-08-31T07:11:45.198878+0000","last_peered":"2024-08-31T07:11:45.198878+0000","last_clean":"2024-08-31T07:11:45.198878+0000","last_became_active":"2024-08-31T07:09:47.672184+0000","last_became_peered":"2024-08-31T07:09:47.672184+0000","last_unstale":"2024-08-31T07:11:45.198878+0000","last_undegraded":"2024-08-31T07:11:45.198878+0000","last_fullsized":"2024-08-31T07:11:45.198878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:17.091659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198670+0000","last_change":"2024-08-31T07:09:47.672597+0000","last_active":"2024-08-31T07:11:45.198670+0000","last_peered":"2024-08-31T07:11:45.198670+0000","last_clean":"2024-08-31T07:11:45.198670+0000","last_became_active":"2024-08-31T07:09:47.672153+0000","last_became_peered":"2024-08-31T07:09:47.672153+0000","last_unstale":"2024-08-31T07:11:45.198670+0000","last_undegraded":"2024-08-31T07:11:45.198670+0000","last_fullsized":"2024-08-31T07:11:45.198670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:26:41.388526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193223+0000","last_change":"2024-08-31T07:09:47.666253+0000","last_active":"2024-08-31T07:11:45.193223+0000","last_peered":"2024-08-31T07:11:45.193223+0000","last_clean":"2024-08-31T07:11:45.193223+0000","last_became_active":"2024-08-31T07:09:47.663732+0000","last_became_peered":"2024-08-31T07:09:47.663732+0000","last_unstale":"2024-08-31T07:11:45.193223+0000","last_undegraded":"2024-08-31T07:11:45.193223+0000","last_fullsized":"2024-08-31T07:11:45.193223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:10:59.887765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188202+0000","last_change":"2024-08-31T07:09:47.668502+0000","last_active":"2024-08-31T07:11:45.188202+0000","last_peered":"2024-08-31T07:11:45.188202+0000","last_clean":"2024-08-31T07:11:45.188202+0000","last_became_active":"2024-08-31T07:09:47.666487+0000","last_became_peered":"2024-08-31T07:09:47.666487+0000","last_unstale":"2024-08-31T07:11:45.188202+0000","last_undegraded":"2024-08-31T07:11:45.188202+0000","last_fullsized":"2024-08-31T07:11:45.188202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:49:25.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.184945+0000","last_change":"2024-08-31T07:09:47.659454+0000","last_active":"2024-08-31T07:11:45.184945+0000","last_peered":"2024-08-31T07:11:45.184945+0000","last_clean":"2024-08-31T07:11:45.184945+0000","last_became_active":"2024-08-31T07:09:47.658976+0000","last_became_peered":"2024-08-31T07:09:47.658976+0000","last_unstale":"2024-08-31T07:11:45.184945+0000","last_undegraded":"2024-08-31T07:11:45.184945+0000","last_fullsized":"2024-08-31T07:11:45.184945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:31.374905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199707+0000","last_change":"2024-08-31T07:09:47.673612+0000","last_active":"2024-08-31T07:11:45.199707+0000","last_peered":"2024-08-31T07:11:45.199707+0000","last_clean":"2024-08-31T07:11:45.199707+0000","last_became_active":"2024-08-31T07:09:47.673137+0000","last_became_peered":"2024-08-31T07:09:47.673137+0000","last_unstale":"2024-08-31T07:11:45.199707+0000","last_undegraded":"2024-08-31T07:11:45.199707+0000","last_fullsized":"2024-08-31T07:11:45.199707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:46:11.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194525+0000","last_change":"2024-08-31T07:09:47.669195+0000","last_active":"2024-08-31T07:11:45.194525+0000","last_peered":"2024-08-31T07:11:45.194525+0000","last_clean":"2024-08-31T07:11:45.194525+0000","last_became_active":"2024-08-31T07:09:47.668790+0000","last_became_peered":"2024-08-31T07:09:47.668790+0000","last_unstale":"2024-08-31T07:11:45.194525+0000","last_undegraded":"2024-08-31T07:11:45.194525+0000","last_fullsized":"2024-08-31T07:11:45.194525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:27:14.393931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.181333+0000","last_change":"2024-08-31T07:09:47.667566+0000","last_active":"2024-08-31T07:11:45.181333+0000","last_peered":"2024-08-31T07:11:45.181333+0000","last_clean":"2024-08-31T07:11:45.181333+0000","last_became_active":"2024-08-31T07:09:47.666605+0000","last_became_peered":"2024-08-31T07:09:47.666605+0000","last_unstale":"2024-08-31T07:11:45.181333+0000","last_undegraded":"2024-08-31T07:11:45.181333+0000","last_fullsized":"2024-08-31T07:11:45.181333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:05:44.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192240+0000","last_change":"2024-08-31T07:09:47.660478+0000","last_active":"2024-08-31T07:11:45.192240+0000","last_peered":"2024-08-31T07:11:45.192240+0000","last_clean":"2024-08-31T07:11:45.192240+0000","last_became_active":"2024-08-31T07:09:47.660107+0000","last_became_peered":"2024-08-31T07:09:47.660107+0000","last_unstale":"2024-08-31T07:11:45.192240+0000","last_undegraded":"2024-08-31T07:11:45.192240+0000","last_fullsized":"2024-08-31T07:11:45.192240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:29.133370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188131+0000","last_change":"2024-08-31T07:09:47.671967+0000","last_active":"2024-08-31T07:11:45.188131+0000","last_peered":"2024-08-31T07:11:45.188131+0000","last_clean":"2024-08-31T07:11:45.188131+0000","last_became_active":"2024-08-31T07:09:47.671569+0000","last_became_peered":"2024-08-31T07:09:47.671569+0000","last_unstale":"2024-08-31T07:11:45.188131+0000","last_undegraded":"2024-08-31T07:11:45.188131+0000","last_fullsized":"2024-08-31T07:11:45.188131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:19:15.107958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185867+0000","last_change":"2024-08-31T07:09:47.651055+0000","last_active":"2024-08-31T07:11:45.185867+0000","last_peered":"2024-08-31T07:11:45.185867+0000","last_clean":"2024-08-31T07:11:45.185867+0000","last_became_active":"2024-08-31T07:09:47.650674+0000","last_became_peered":"2024-08-31T07:09:47.650674+0000","last_unstale":"2024-08-31T07:11:45.185867+0000","last_undegraded":"2024-08-31T07:11:45.185867+0000","last_fullsized":"2024-08-31T07:11:45.185867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:35:54.647379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180140+0000","last_change":"2024-08-31T07:09:47.663231+0000","last_active":"2024-08-31T07:11:45.180140+0000","last_peered":"2024-08-31T07:11:45.180140+0000","last_clean":"2024-08-31T07:11:45.180140+0000","last_became_active":"2024-08-31T07:09:47.662854+0000","last_became_peered":"2024-08-31T07:09:47.662854+0000","last_unstale":"2024-08-31T07:11:45.180140+0000","last_undegraded":"2024-08-31T07:11:45.180140+0000","last_fullsized":"2024-08-31T07:11:45.180140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:51.358557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199287+0000","last_change":"2024-08-31T07:10:46.326380+0000","last_active":"2024-08-31T07:11:45.199287+0000","last_peered":"2024-08-31T07:11:45.199287+0000","last_clean":"2024-08-31T07:11:45.199287+0000","last_became_active":"2024-08-31T07:10:46.325784+0000","last_became_peered":"2024-08-31T07:10:46.325784+0000","last_unstale":"2024-08-31T07:11:45.199287+0000","last_undegraded":"2024-08-31T07:11:45.199287+0000","last_fullsized":"2024-08-31T07:11:45.199287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:16:56.396024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200247+0000","last_change":"2024-08-31T07:09:47.672108+0000","last_active":"2024-08-31T07:11:45.200247+0000","last_peered":"2024-08-31T07:11:45.200247+0000","last_clean":"2024-08-31T07:11:45.200247+0000","last_became_active":"2024-08-31T07:09:47.670998+0000","last_became_peered":"2024-08-31T07:09:47.670998+0000","last_unstale":"2024-08-31T07:11:45.200247+0000","last_undegraded":"2024-08-31T07:11:45.200247+0000","last_fullsized":"2024-08-31T07:11:45.200247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:26:49.319861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198226+0000","last_change":"2024-08-31T07:09:47.665548+0000","last_active":"2024-08-31T07:11:45.198226+0000","last_peered":"2024-08-31T07:11:45.198226+0000","last_clean":"2024-08-31T07:11:45.198226+0000","last_became_active":"2024-08-31T07:09:47.665177+0000","last_became_peered":"2024-08-31T07:09:47.665177+0000","last_unstale":"2024-08-31T07:11:45.198226+0000","last_undegraded":"2024-08-31T07:11:45.198226+0000","last_fullsized":"2024-08-31T07:11:45.198226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:03:15.892224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.177785+0000","last_change":"2024-08-31T07:10:46.026746+0000","last_active":"2024-08-31T07:11:45.177785+0000","last_peered":"2024-08-31T07:11:45.177785+0000","last_clean":"2024-08-31T07:11:45.177785+0000","last_became_active":"2024-08-31T07:10:46.026286+0000","last_became_peered":"2024-08-31T07:10:46.026286+0000","last_unstale":"2024-08-31T07:11:45.177785+0000","last_undegraded":"2024-08-31T07:11:45.177785+0000","last_fullsized":"2024-08-31T07:11:45.177785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:34:17.120906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180769+0000","last_change":"2024-08-31T07:09:47.667456+0000","last_active":"2024-08-31T07:11:45.180769+0000","last_peered":"2024-08-31T07:11:45.180769+0000","last_clean":"2024-08-31T07:11:45.180769+0000","last_became_active":"2024-08-31T07:09:47.667102+0000","last_became_peered":"2024-08-31T07:09:47.667102+0000","last_unstale":"2024-08-31T07:11:45.180769+0000","last_undegraded":"2024-08-31T07:11:45.180769+0000","last_fullsized":"2024-08-31T07:11:45.180769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:44:39.747729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.182194+0000","last_change":"2024-08-31T07:10:46.326139+0000","last_active":"2024-08-31T07:11:45.182194+0000","last_peered":"2024-08-31T07:11:45.182194+0000","last_clean":"2024-08-31T07:11:45.182194+0000","last_became_active":"2024-08-31T07:10:46.325785+0000","last_became_peered":"2024-08-31T07:10:46.325785+0000","last_unstale":"2024-08-31T07:11:45.182194+0000","last_undegraded":"2024-08-31T07:11:45.182194+0000","last_fullsized":"2024-08-31T07:11:45.182194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:13.495835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":180,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:25.108048+0000","last_change":"2024-08-31T07:09:45.641088+0000","last_active":"2024-08-31T07:12:25.108048+0000","last_peered":"2024-08-31T07:12:25.108048+0000","last_clean":"2024-08-31T07:12:25.108048+0000","last_became_active":"2024-08-31T07:09:45.640673+0000","last_became_peered":"2024-08-31T07:09:45.640673+0000","last_unstale":"2024-08-31T07:12:25.108048+0000","last_undegraded":"2024-08-31T07:12:25.108048+0000","last_fullsized":"2024-08-31T07:12:25.108048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:30:17.593494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192346+0000","last_change":"2024-08-31T07:09:47.666703+0000","last_active":"2024-08-31T07:11:45.192346+0000","last_peered":"2024-08-31T07:11:45.192346+0000","last_clean":"2024-08-31T07:11:45.192346+0000","last_became_active":"2024-08-31T07:09:47.666348+0000","last_became_peered":"2024-08-31T07:09:47.666348+0000","last_unstale":"2024-08-31T07:11:45.192346+0000","last_undegraded":"2024-08-31T07:11:45.192346+0000","last_fullsized":"2024-08-31T07:11:45.192346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:39.688810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019885+0000","last_change":"2024-08-31T07:09:45.642431+0000","last_active":"2024-08-31T07:10:46.019885+0000","last_peered":"2024-08-31T07:10:46.019885+0000","last_clean":"2024-08-31T07:10:46.019885+0000","last_became_active":"2024-08-31T07:09:45.642048+0000","last_became_peered":"2024-08-31T07:09:45.642048+0000","last_unstale":"2024-08-31T07:10:46.019885+0000","last_undegraded":"2024-08-31T07:10:46.019885+0000","last_fullsized":"2024-08-31T07:10:46.019885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:47:42.574442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017286+0000","last_change":"2024-08-31T07:09:45.632162+0000","last_active":"2024-08-31T07:10:46.017286+0000","last_peered":"2024-08-31T07:10:46.017286+0000","last_clean":"2024-08-31T07:10:46.017286+0000","last_became_active":"2024-08-31T07:09:45.631749+0000","last_became_peered":"2024-08-31T07:09:45.631749+0000","last_unstale":"2024-08-31T07:10:46.017286+0000","last_undegraded":"2024-08-31T07:10:46.017286+0000","last_fullsized":"2024-08-31T07:10:46.017286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:16:29.967823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185787+0000","last_change":"2024-08-31T07:09:47.666288+0000","last_active":"2024-08-31T07:11:45.185787+0000","last_peered":"2024-08-31T07:11:45.185787+0000","last_clean":"2024-08-31T07:11:45.185787+0000","last_became_active":"2024-08-31T07:09:47.665723+0000","last_became_peered":"2024-08-31T07:09:47.665723+0000","last_unstale":"2024-08-31T07:11:45.185787+0000","last_undegraded":"2024-08-31T07:11:45.185787+0000","last_fullsized":"2024-08-31T07:11:45.185787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:38:28.122720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023120+0000","last_change":"2024-08-31T07:09:45.636508+0000","last_active":"2024-08-31T07:10:46.023120+0000","last_peered":"2024-08-31T07:10:46.023120+0000","last_clean":"2024-08-31T07:10:46.023120+0000","last_became_active":"2024-08-31T07:09:45.636073+0000","last_became_peered":"2024-08-31T07:09:45.636073+0000","last_unstale":"2024-08-31T07:10:46.023120+0000","last_undegraded":"2024-08-31T07:10:46.023120+0000","last_fullsized":"2024-08-31T07:10:46.023120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:55:45.063517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187943+0000","last_change":"2024-08-31T07:09:47.661306+0000","last_active":"2024-08-31T07:11:45.187943+0000","last_peered":"2024-08-31T07:11:45.187943+0000","last_clean":"2024-08-31T07:11:45.187943+0000","last_became_active":"2024-08-31T07:09:47.660957+0000","last_became_peered":"2024-08-31T07:09:47.660957+0000","last_unstale":"2024-08-31T07:11:45.187943+0000","last_undegraded":"2024-08-31T07:11:45.187943+0000","last_fullsized":"2024-08-31T07:11:45.187943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:18:23.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'6","reported_seq":308,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:21.101061+0000","last_change":"2024-08-31T07:09:45.640947+0000","last_active":"2024-08-31T07:12:21.101061+0000","last_peered":"2024-08-31T07:12:21.101061+0000","last_clean":"2024-08-31T07:12:21.101061+0000","last_became_active":"2024-08-31T07:09:45.640532+0000","last_became_peered":"2024-08-31T07:09:45.640532+0000","last_unstale":"2024-08-31T07:12:21.101061+0000","last_undegraded":"2024-08-31T07:12:21.101061+0000","last_fullsized":"2024-08-31T07:12:21.101061+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:28.177795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":276,"num_read_kb":276,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.191708+0000","last_change":"2024-08-31T07:09:47.661060+0000","last_active":"2024-08-31T07:11:45.191708+0000","last_peered":"2024-08-31T07:11:45.191708+0000","last_clean":"2024-08-31T07:11:45.191708+0000","last_became_active":"2024-08-31T07:09:47.660692+0000","last_became_peered":"2024-08-31T07:09:47.660692+0000","last_unstale":"2024-08-31T07:11:45.191708+0000","last_undegraded":"2024-08-31T07:11:45.191708+0000","last_fullsized":"2024-08-31T07:11:45.191708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:48:38.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257194+0000","last_change":"2024-08-31T07:09:45.641155+0000","last_active":"2024-08-31T07:10:46.257194+0000","last_peered":"2024-08-31T07:10:46.257194+0000","last_clean":"2024-08-31T07:10:46.257194+0000","last_became_active":"2024-08-31T07:09:45.640680+0000","last_became_peered":"2024-08-31T07:09:45.640680+0000","last_unstale":"2024-08-31T07:10:46.257194+0000","last_undegraded":"2024-08-31T07:10:46.257194+0000","last_fullsized":"2024-08-31T07:10:46.257194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:29:03.287570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199892+0000","last_change":"2024-08-31T07:09:47.675582+0000","last_active":"2024-08-31T07:11:45.199892+0000","last_peered":"2024-08-31T07:11:45.199892+0000","last_clean":"2024-08-31T07:11:45.199892+0000","last_became_active":"2024-08-31T07:09:47.675266+0000","last_became_peered":"2024-08-31T07:09:47.675266+0000","last_unstale":"2024-08-31T07:11:45.199892+0000","last_undegraded":"2024-08-31T07:11:45.199892+0000","last_fullsized":"2024-08-31T07:11:45.199892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:36:33.122531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021194+0000","last_change":"2024-08-31T07:09:45.651006+0000","last_active":"2024-08-31T07:10:46.021194+0000","last_peered":"2024-08-31T07:10:46.021194+0000","last_clean":"2024-08-31T07:10:46.021194+0000","last_became_active":"2024-08-31T07:09:45.650588+0000","last_became_peered":"2024-08-31T07:09:45.650588+0000","last_unstale":"2024-08-31T07:10:46.021194+0000","last_undegraded":"2024-08-31T07:10:46.021194+0000","last_fullsized":"2024-08-31T07:10:46.021194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:34:16.015892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.190030+0000","last_change":"2024-08-31T07:09:47.662289+0000","last_active":"2024-08-31T07:11:45.190030+0000","last_peered":"2024-08-31T07:11:45.190030+0000","last_clean":"2024-08-31T07:11:45.190030+0000","last_became_active":"2024-08-31T07:09:47.661932+0000","last_became_peered":"2024-08-31T07:09:47.661932+0000","last_unstale":"2024-08-31T07:11:45.190030+0000","last_undegraded":"2024-08-31T07:11:45.190030+0000","last_fullsized":"2024-08-31T07:11:45.190030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:50.440504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'37","reported_seq":351,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:21.101077+0000","last_change":"2024-08-31T07:09:45.642196+0000","last_active":"2024-08-31T07:12:21.101077+0000","last_peered":"2024-08-31T07:12:21.101077+0000","last_clean":"2024-08-31T07:12:21.101077+0000","last_became_active":"2024-08-31T07:09:45.641797+0000","last_became_peered":"2024-08-31T07:09:45.641797+0000","last_unstale":"2024-08-31T07:12:21.101077+0000","last_undegraded":"2024-08-31T07:12:21.101077+0000","last_fullsized":"2024-08-31T07:12:21.101077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:51:05.953335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66550,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":288,"num_read_kb":209,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196022+0000","last_change":"2024-08-31T07:09:47.660184+0000","last_active":"2024-08-31T07:11:45.196022+0000","last_peered":"2024-08-31T07:11:45.196022+0000","last_clean":"2024-08-31T07:11:45.196022+0000","last_became_active":"2024-08-31T07:09:47.659571+0000","last_became_peered":"2024-08-31T07:09:47.659571+0000","last_unstale":"2024-08-31T07:11:45.196022+0000","last_undegraded":"2024-08-31T07:11:45.196022+0000","last_fullsized":"2024-08-31T07:11:45.196022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:41:13.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.035782+0000","last_change":"2024-08-31T07:10:46.035336+0000","last_active":"2024-08-31T07:10:46.035782+0000","last_peered":"2024-08-31T07:10:46.035782+0000","last_clean":"2024-08-31T07:10:46.035782+0000","last_became_active":"2024-08-31T07:10:46.021299+0000","last_became_peered":"2024-08-31T07:10:46.021299+0000","last_unstale":"2024-08-31T07:10:46.035782+0000","last_undegraded":"2024-08-31T07:10:46.035782+0000","last_fullsized":"2024-08-31T07:10:46.035782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:06:15.931855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194680+0000","last_change":"2024-08-31T07:10:46.326354+0000","last_active":"2024-08-31T07:11:45.194680+0000","last_peered":"2024-08-31T07:11:45.194680+0000","last_clean":"2024-08-31T07:11:45.194680+0000","last_became_active":"2024-08-31T07:10:46.325869+0000","last_became_peered":"2024-08-31T07:10:46.325869+0000","last_unstale":"2024-08-31T07:11:45.194680+0000","last_undegraded":"2024-08-31T07:11:45.194680+0000","last_fullsized":"2024-08-31T07:11:45.194680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:21.832088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022972+0000","last_change":"2024-08-31T07:09:45.642187+0000","last_active":"2024-08-31T07:10:46.022972+0000","last_peered":"2024-08-31T07:10:46.022972+0000","last_clean":"2024-08-31T07:10:46.022972+0000","last_became_active":"2024-08-31T07:09:45.641721+0000","last_became_peered":"2024-08-31T07:09:45.641721+0000","last_unstale":"2024-08-31T07:10:46.022972+0000","last_undegraded":"2024-08-31T07:10:46.022972+0000","last_fullsized":"2024-08-31T07:10:46.022972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:00:45.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196019+0000","last_change":"2024-08-31T07:09:47.665956+0000","last_active":"2024-08-31T07:11:45.196019+0000","last_peered":"2024-08-31T07:11:45.196019+0000","last_clean":"2024-08-31T07:11:45.196019+0000","last_became_active":"2024-08-31T07:09:47.664367+0000","last_became_peered":"2024-08-31T07:09:47.664367+0000","last_unstale":"2024-08-31T07:11:45.196019+0000","last_undegraded":"2024-08-31T07:11:45.196019+0000","last_fullsized":"2024-08-31T07:11:45.196019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:34.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'7","reported_seq":180,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:25.107094+0000","last_change":"2024-08-31T07:09:45.636440+0000","last_active":"2024-08-31T07:12:25.107094+0000","last_peered":"2024-08-31T07:12:25.107094+0000","last_clean":"2024-08-31T07:12:25.107094+0000","last_became_active":"2024-08-31T07:09:45.636058+0000","last_became_peered":"2024-08-31T07:09:45.636058+0000","last_unstale":"2024-08-31T07:12:25.107094+0000","last_undegraded":"2024-08-31T07:12:25.107094+0000","last_fullsized":"2024-08-31T07:12:25.107094+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:38.929227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194840+0000","last_change":"2024-08-31T07:09:47.660564+0000","last_active":"2024-08-31T07:11:45.194840+0000","last_peered":"2024-08-31T07:11:45.194840+0000","last_clean":"2024-08-31T07:11:45.194840+0000","last_became_active":"2024-08-31T07:09:47.660157+0000","last_became_peered":"2024-08-31T07:09:47.660157+0000","last_unstale":"2024-08-31T07:11:45.194840+0000","last_undegraded":"2024-08-31T07:11:45.194840+0000","last_fullsized":"2024-08-31T07:11:45.194840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:49:01.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020515+0000","last_change":"2024-08-31T07:09:45.649140+0000","last_active":"2024-08-31T07:10:46.020515+0000","last_peered":"2024-08-31T07:10:46.020515+0000","last_clean":"2024-08-31T07:10:46.020515+0000","last_became_active":"2024-08-31T07:09:45.648796+0000","last_became_peered":"2024-08-31T07:09:45.648796+0000","last_unstale":"2024-08-31T07:10:46.020515+0000","last_undegraded":"2024-08-31T07:10:46.020515+0000","last_fullsized":"2024-08-31T07:10:46.020515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:33:37.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192390+0000","last_change":"2024-08-31T07:09:47.663833+0000","last_active":"2024-08-31T07:11:45.192390+0000","last_peered":"2024-08-31T07:11:45.192390+0000","last_clean":"2024-08-31T07:11:45.192390+0000","last_became_active":"2024-08-31T07:09:47.663451+0000","last_became_peered":"2024-08-31T07:09:47.663451+0000","last_unstale":"2024-08-31T07:11:45.192390+0000","last_undegraded":"2024-08-31T07:11:45.192390+0000","last_fullsized":"2024-08-31T07:11:45.192390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:27:22.245254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021407+0000","last_change":"2024-08-31T07:09:45.636593+0000","last_active":"2024-08-31T07:10:46.021407+0000","last_peered":"2024-08-31T07:10:46.021407+0000","last_clean":"2024-08-31T07:10:46.021407+0000","last_became_active":"2024-08-31T07:09:45.636166+0000","last_became_peered":"2024-08-31T07:09:45.636166+0000","last_unstale":"2024-08-31T07:10:46.021407+0000","last_undegraded":"2024-08-31T07:10:46.021407+0000","last_fullsized":"2024-08-31T07:10:46.021407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:28.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183385+0000","last_change":"2024-08-31T07:09:47.660825+0000","last_active":"2024-08-31T07:11:45.183385+0000","last_peered":"2024-08-31T07:11:45.183385+0000","last_clean":"2024-08-31T07:11:45.183385+0000","last_became_active":"2024-08-31T07:09:47.660474+0000","last_became_peered":"2024-08-31T07:09:47.660474+0000","last_unstale":"2024-08-31T07:11:45.183385+0000","last_undegraded":"2024-08-31T07:11:45.183385+0000","last_fullsized":"2024-08-31T07:11:45.183385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:42:20.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017462+0000","last_change":"2024-08-31T07:09:45.632727+0000","last_active":"2024-08-31T07:10:46.017462+0000","last_peered":"2024-08-31T07:10:46.017462+0000","last_clean":"2024-08-31T07:10:46.017462+0000","last_became_active":"2024-08-31T07:09:45.632251+0000","last_became_peered":"2024-08-31T07:09:45.632251+0000","last_unstale":"2024-08-31T07:10:46.017462+0000","last_undegraded":"2024-08-31T07:10:46.017462+0000","last_fullsized":"2024-08-31T07:10:46.017462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:09:30.197409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196430+0000","last_change":"2024-08-31T07:09:47.665501+0000","last_active":"2024-08-31T07:11:45.196430+0000","last_peered":"2024-08-31T07:11:45.196430+0000","last_clean":"2024-08-31T07:11:45.196430+0000","last_became_active":"2024-08-31T07:09:47.665162+0000","last_became_peered":"2024-08-31T07:09:47.665162+0000","last_unstale":"2024-08-31T07:11:45.196430+0000","last_undegraded":"2024-08-31T07:11:45.196430+0000","last_fullsized":"2024-08-31T07:11:45.196430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:34:29.605836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'62","reported_seq":118,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:24.990505+0000","last_change":"2024-08-31T07:09:45.636580+0000","last_active":"2024-08-31T07:12:24.990505+0000","last_peered":"2024-08-31T07:12:24.990505+0000","last_clean":"2024-08-31T07:12:24.990505+0000","last_became_active":"2024-08-31T07:09:45.636163+0000","last_became_peered":"2024-08-31T07:09:45.636163+0000","last_unstale":"2024-08-31T07:12:24.990505+0000","last_undegraded":"2024-08-31T07:12:24.990505+0000","last_fullsized":"2024-08-31T07:12:24.990505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:18:07.567858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195120+0000","last_change":"2024-08-31T07:09:47.671865+0000","last_active":"2024-08-31T07:11:45.195120+0000","last_peered":"2024-08-31T07:11:45.195120+0000","last_clean":"2024-08-31T07:11:45.195120+0000","last_became_active":"2024-08-31T07:09:47.671454+0000","last_became_peered":"2024-08-31T07:09:47.671454+0000","last_unstale":"2024-08-31T07:11:45.195120+0000","last_undegraded":"2024-08-31T07:11:45.195120+0000","last_fullsized":"2024-08-31T07:11:45.195120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:47.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'7","reported_seq":178,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:23.104896+0000","last_change":"2024-08-31T07:09:45.653217+0000","last_active":"2024-08-31T07:12:23.104896+0000","last_peered":"2024-08-31T07:12:23.104896+0000","last_clean":"2024-08-31T07:12:23.104896+0000","last_became_active":"2024-08-31T07:09:45.652819+0000","last_became_peered":"2024-08-31T07:09:45.652819+0000","last_unstale":"2024-08-31T07:12:23.104896+0000","last_undegraded":"2024-08-31T07:12:23.104896+0000","last_fullsized":"2024-08-31T07:12:23.104896+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:25:11.454068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":145,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196166+0000","last_change":"2024-08-31T07:09:47.663652+0000","last_active":"2024-08-31T07:11:45.196166+0000","last_peered":"2024-08-31T07:11:45.196166+0000","last_clean":"2024-08-31T07:11:45.196166+0000","last_became_active":"2024-08-31T07:09:47.663274+0000","last_became_peered":"2024-08-31T07:09:47.663274+0000","last_unstale":"2024-08-31T07:11:45.196166+0000","last_undegraded":"2024-08-31T07:11:45.196166+0000","last_fullsized":"2024-08-31T07:11:45.196166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:31:59.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257370+0000","last_change":"2024-08-31T07:08:27.623298+0000","last_active":"2024-08-31T07:10:46.257370+0000","last_peered":"2024-08-31T07:10:46.257370+0000","last_clean":"2024-08-31T07:10:46.257370+0000","last_became_active":"2024-08-31T07:08:27.619072+0000","last_became_peered":"2024-08-31T07:08:27.619072+0000","last_unstale":"2024-08-31T07:10:46.257370+0000","last_undegraded":"2024-08-31T07:10:46.257370+0000","last_fullsized":"2024-08-31T07:10:46.257370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_clean_scrub_stamp":"2024-08-31T07:04:45.509398+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:07.967421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154319+0000","last_change":"2024-08-31T07:09:45.638104+0000","last_active":"2024-08-31T07:10:46.154319+0000","last_peered":"2024-08-31T07:10:46.154319+0000","last_clean":"2024-08-31T07:10:46.154319+0000","last_became_active":"2024-08-31T07:09:45.637693+0000","last_became_peered":"2024-08-31T07:09:45.637693+0000","last_unstale":"2024-08-31T07:10:46.154319+0000","last_undegraded":"2024-08-31T07:10:46.154319+0000","last_fullsized":"2024-08-31T07:10:46.154319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:50:15.135061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.189353+0000","last_change":"2024-08-31T07:09:47.664958+0000","last_active":"2024-08-31T07:11:45.189353+0000","last_peered":"2024-08-31T07:11:45.189353+0000","last_clean":"2024-08-31T07:11:45.189353+0000","last_became_active":"2024-08-31T07:09:47.661166+0000","last_became_peered":"2024-08-31T07:09:47.661166+0000","last_unstale":"2024-08-31T07:11:45.189353+0000","last_undegraded":"2024-08-31T07:11:45.189353+0000","last_fullsized":"2024-08-31T07:11:45.189353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:45:58.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257184+0000","last_change":"2024-08-31T07:09:45.640881+0000","last_active":"2024-08-31T07:10:46.257184+0000","last_peered":"2024-08-31T07:10:46.257184+0000","last_clean":"2024-08-31T07:10:46.257184+0000","last_became_active":"2024-08-31T07:09:45.640487+0000","last_became_peered":"2024-08-31T07:09:45.640487+0000","last_unstale":"2024-08-31T07:10:46.257184+0000","last_undegraded":"2024-08-31T07:10:46.257184+0000","last_fullsized":"2024-08-31T07:10:46.257184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:06.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.171981+0000","last_change":"2024-08-31T07:09:47.655008+0000","last_active":"2024-08-31T07:11:45.171981+0000","last_peered":"2024-08-31T07:11:45.171981+0000","last_clean":"2024-08-31T07:11:45.171981+0000","last_became_active":"2024-08-31T07:09:47.654645+0000","last_became_peered":"2024-08-31T07:09:47.654645+0000","last_unstale":"2024-08-31T07:11:45.171981+0000","last_undegraded":"2024-08-31T07:11:45.171981+0000","last_fullsized":"2024-08-31T07:11:45.171981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:23:55.916053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'6","reported_seq":173,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:22.101744+0000","last_change":"2024-08-31T07:09:45.649324+0000","last_active":"2024-08-31T07:12:22.101744+0000","last_peered":"2024-08-31T07:12:22.101744+0000","last_clean":"2024-08-31T07:12:22.101744+0000","last_became_active":"2024-08-31T07:09:45.648915+0000","last_became_peered":"2024-08-31T07:09:45.648915+0000","last_unstale":"2024-08-31T07:12:22.101744+0000","last_undegraded":"2024-08-31T07:12:22.101744+0000","last_fullsized":"2024-08-31T07:12:22.101744+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:31:57.853654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":141,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187160+0000","last_change":"2024-08-31T07:09:47.661328+0000","last_active":"2024-08-31T07:11:45.187160+0000","last_peered":"2024-08-31T07:11:45.187160+0000","last_clean":"2024-08-31T07:11:45.187160+0000","last_became_active":"2024-08-31T07:09:47.660927+0000","last_became_peered":"2024-08-31T07:09:47.660927+0000","last_unstale":"2024-08-31T07:11:45.187160+0000","last_undegraded":"2024-08-31T07:11:45.187160+0000","last_fullsized":"2024-08-31T07:11:45.187160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:40:46.876894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'7","reported_seq":176,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:22.103314+0000","last_change":"2024-08-31T07:09:45.637747+0000","last_active":"2024-08-31T07:12:22.103314+0000","last_peered":"2024-08-31T07:12:22.103314+0000","last_clean":"2024-08-31T07:12:22.103314+0000","last_became_active":"2024-08-31T07:09:45.637368+0000","last_became_peered":"2024-08-31T07:09:45.637368+0000","last_unstale":"2024-08-31T07:12:22.103314+0000","last_undegraded":"2024-08-31T07:12:22.103314+0000","last_fullsized":"2024-08-31T07:12:22.103314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:01.174109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193217+0000","last_change":"2024-08-31T07:09:47.680802+0000","last_active":"2024-08-31T07:11:45.193217+0000","last_peered":"2024-08-31T07:11:45.193217+0000","last_clean":"2024-08-31T07:11:45.193217+0000","last_became_active":"2024-08-31T07:09:47.680363+0000","last_became_peered":"2024-08-31T07:09:47.680363+0000","last_unstale":"2024-08-31T07:11:45.193217+0000","last_undegraded":"2024-08-31T07:11:45.193217+0000","last_fullsized":"2024-08-31T07:11:45.193217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:46:48.744071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324353+0000","last_change":"2024-08-31T07:09:45.651222+0000","last_active":"2024-08-31T07:10:46.324353+0000","last_peered":"2024-08-31T07:10:46.324353+0000","last_clean":"2024-08-31T07:10:46.324353+0000","last_became_active":"2024-08-31T07:09:45.650760+0000","last_became_peered":"2024-08-31T07:09:45.650760+0000","last_unstale":"2024-08-31T07:10:46.324353+0000","last_undegraded":"2024-08-31T07:10:46.324353+0000","last_fullsized":"2024-08-31T07:10:46.324353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T19:03:33.942165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183968+0000","last_change":"2024-08-31T07:09:47.665881+0000","last_active":"2024-08-31T07:11:45.183968+0000","last_peered":"2024-08-31T07:11:45.183968+0000","last_clean":"2024-08-31T07:11:45.183968+0000","last_became_active":"2024-08-31T07:09:47.665485+0000","last_became_peered":"2024-08-31T07:09:47.665485+0000","last_unstale":"2024-08-31T07:11:45.183968+0000","last_undegraded":"2024-08-31T07:11:45.183968+0000","last_fullsized":"2024-08-31T07:11:45.183968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:30.820714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020851+0000","last_change":"2024-08-31T07:09:45.651893+0000","last_active":"2024-08-31T07:10:46.020851+0000","last_peered":"2024-08-31T07:10:46.020851+0000","last_clean":"2024-08-31T07:10:46.020851+0000","last_became_active":"2024-08-31T07:09:45.651547+0000","last_became_peered":"2024-08-31T07:09:45.651547+0000","last_unstale":"2024-08-31T07:10:46.020851+0000","last_undegraded":"2024-08-31T07:10:46.020851+0000","last_fullsized":"2024-08-31T07:10:46.020851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:11:30.708018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.175560+0000","last_change":"2024-08-31T07:09:47.655551+0000","last_active":"2024-08-31T07:11:45.175560+0000","last_peered":"2024-08-31T07:11:45.175560+0000","last_clean":"2024-08-31T07:11:45.175560+0000","last_became_active":"2024-08-31T07:09:47.655188+0000","last_became_peered":"2024-08-31T07:09:47.655188+0000","last_unstale":"2024-08-31T07:11:45.175560+0000","last_undegraded":"2024-08-31T07:11:45.175560+0000","last_fullsized":"2024-08-31T07:11:45.175560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:43:51.419599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'101","reported_seq":216,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:25.489306+0000","last_change":"2024-08-31T07:09:45.636212+0000","last_active":"2024-08-31T07:12:25.489306+0000","last_peered":"2024-08-31T07:12:25.489306+0000","last_clean":"2024-08-31T07:12:25.489306+0000","last_became_active":"2024-08-31T07:09:45.635678+0000","last_became_peered":"2024-08-31T07:09:45.635678+0000","last_unstale":"2024-08-31T07:12:25.489306+0000","last_undegraded":"2024-08-31T07:12:25.489306+0000","last_fullsized":"2024-08-31T07:12:25.489306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:13:17.803459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":185,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185987+0000","last_change":"2024-08-31T07:09:47.666476+0000","last_active":"2024-08-31T07:11:45.185987+0000","last_peered":"2024-08-31T07:11:45.185987+0000","last_clean":"2024-08-31T07:11:45.185987+0000","last_became_active":"2024-08-31T07:09:47.666111+0000","last_became_peered":"2024-08-31T07:09:47.666111+0000","last_unstale":"2024-08-31T07:11:45.185987+0000","last_undegraded":"2024-08-31T07:11:45.185987+0000","last_fullsized":"2024-08-31T07:11:45.185987+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:08:53.459549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154441+0000","last_change":"2024-08-31T07:09:45.631571+0000","last_active":"2024-08-31T07:10:46.154441+0000","last_peered":"2024-08-31T07:10:46.154441+0000","last_clean":"2024-08-31T07:10:46.154441+0000","last_became_active":"2024-08-31T07:09:45.630981+0000","last_became_peered":"2024-08-31T07:09:45.630981+0000","last_unstale":"2024-08-31T07:10:46.154441+0000","last_undegraded":"2024-08-31T07:10:46.154441+0000","last_fullsized":"2024-08-31T07:10:46.154441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:36:53.482955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193936+0000","last_change":"2024-08-31T07:09:47.675054+0000","last_active":"2024-08-31T07:11:45.193936+0000","last_peered":"2024-08-31T07:11:45.193936+0000","last_clean":"2024-08-31T07:11:45.193936+0000","last_became_active":"2024-08-31T07:09:47.672652+0000","last_became_peered":"2024-08-31T07:09:47.672652+0000","last_unstale":"2024-08-31T07:11:45.193936+0000","last_undegraded":"2024-08-31T07:11:45.193936+0000","last_fullsized":"2024-08-31T07:11:45.193936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:32:05.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016284+0000","last_change":"2024-08-31T07:09:45.648746+0000","last_active":"2024-08-31T07:10:46.016284+0000","last_peered":"2024-08-31T07:10:46.016284+0000","last_clean":"2024-08-31T07:10:46.016284+0000","last_became_active":"2024-08-31T07:09:45.648424+0000","last_became_peered":"2024-08-31T07:09:45.648424+0000","last_unstale":"2024-08-31T07:10:46.016284+0000","last_undegraded":"2024-08-31T07:10:46.016284+0000","last_fullsized":"2024-08-31T07:10:46.016284+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:40:32.132963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.165179+0000","last_change":"2024-08-31T07:09:47.663213+0000","last_active":"2024-08-31T07:11:45.165179+0000","last_peered":"2024-08-31T07:11:45.165179+0000","last_clean":"2024-08-31T07:11:45.165179+0000","last_became_active":"2024-08-31T07:09:47.662457+0000","last_became_peered":"2024-08-31T07:09:47.662457+0000","last_unstale":"2024-08-31T07:11:45.165179+0000","last_undegraded":"2024-08-31T07:11:45.165179+0000","last_fullsized":"2024-08-31T07:11:45.165179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:54:27.069200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015653+0000","last_change":"2024-08-31T07:09:45.640149+0000","last_active":"2024-08-31T07:10:46.015653+0000","last_peered":"2024-08-31T07:10:46.015653+0000","last_clean":"2024-08-31T07:10:46.015653+0000","last_became_active":"2024-08-31T07:09:45.639581+0000","last_became_peered":"2024-08-31T07:09:45.639581+0000","last_unstale":"2024-08-31T07:10:46.015653+0000","last_undegraded":"2024-08-31T07:10:46.015653+0000","last_fullsized":"2024-08-31T07:10:46.015653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:27:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198479+0000","last_change":"2024-08-31T07:09:47.672613+0000","last_active":"2024-08-31T07:11:45.198479+0000","last_peered":"2024-08-31T07:11:45.198479+0000","last_clean":"2024-08-31T07:11:45.198479+0000","last_became_active":"2024-08-31T07:09:47.672173+0000","last_became_peered":"2024-08-31T07:09:47.672173+0000","last_unstale":"2024-08-31T07:11:45.198479+0000","last_undegraded":"2024-08-31T07:11:45.198479+0000","last_fullsized":"2024-08-31T07:11:45.198479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:49.706113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022590+0000","last_change":"2024-08-31T07:09:45.637567+0000","last_active":"2024-08-31T07:10:46.022590+0000","last_peered":"2024-08-31T07:10:46.022590+0000","last_clean":"2024-08-31T07:10:46.022590+0000","last_became_active":"2024-08-31T07:09:45.637180+0000","last_became_peered":"2024-08-31T07:09:45.637180+0000","last_unstale":"2024-08-31T07:10:46.022590+0000","last_undegraded":"2024-08-31T07:10:46.022590+0000","last_fullsized":"2024-08-31T07:10:46.022590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:48:07.190829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194363+0000","last_change":"2024-08-31T07:09:47.674980+0000","last_active":"2024-08-31T07:11:45.194363+0000","last_peered":"2024-08-31T07:11:45.194363+0000","last_clean":"2024-08-31T07:11:45.194363+0000","last_became_active":"2024-08-31T07:09:47.674528+0000","last_became_peered":"2024-08-31T07:09:47.674528+0000","last_unstale":"2024-08-31T07:11:45.194363+0000","last_undegraded":"2024-08-31T07:11:45.194363+0000","last_fullsized":"2024-08-31T07:11:45.194363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:15:09.464221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021096+0000","last_change":"2024-08-31T07:09:45.641167+0000","last_active":"2024-08-31T07:10:46.021096+0000","last_peered":"2024-08-31T07:10:46.021096+0000","last_clean":"2024-08-31T07:10:46.021096+0000","last_became_active":"2024-08-31T07:09:45.640761+0000","last_became_peered":"2024-08-31T07:09:45.640761+0000","last_unstale":"2024-08-31T07:10:46.021096+0000","last_undegraded":"2024-08-31T07:10:46.021096+0000","last_fullsized":"2024-08-31T07:10:46.021096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:28:59.963145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195596+0000","last_change":"2024-08-31T07:09:47.668808+0000","last_active":"2024-08-31T07:11:45.195596+0000","last_peered":"2024-08-31T07:11:45.195596+0000","last_clean":"2024-08-31T07:11:45.195596+0000","last_became_active":"2024-08-31T07:09:47.668291+0000","last_became_peered":"2024-08-31T07:09:47.668291+0000","last_unstale":"2024-08-31T07:11:45.195596+0000","last_undegraded":"2024-08-31T07:11:45.195596+0000","last_fullsized":"2024-08-31T07:11:45.195596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:23.433511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193074+0000","last_change":"2024-08-31T07:09:47.668039+0000","last_active":"2024-08-31T07:11:45.193074+0000","last_peered":"2024-08-31T07:11:45.193074+0000","last_clean":"2024-08-31T07:11:45.193074+0000","last_became_active":"2024-08-31T07:09:47.667567+0000","last_became_peered":"2024-08-31T07:09:47.667567+0000","last_unstale":"2024-08-31T07:11:45.193074+0000","last_undegraded":"2024-08-31T07:11:45.193074+0000","last_fullsized":"2024-08-31T07:11:45.193074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:33:50.192864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.041708+0000","last_change":"2024-08-31T07:10:46.041197+0000","last_active":"2024-08-31T07:10:46.041708+0000","last_peered":"2024-08-31T07:10:46.041708+0000","last_clean":"2024-08-31T07:10:46.041708+0000","last_became_active":"2024-08-31T07:10:46.023966+0000","last_became_peered":"2024-08-31T07:10:46.023966+0000","last_unstale":"2024-08-31T07:10:46.041708+0000","last_undegraded":"2024-08-31T07:10:46.041708+0000","last_fullsized":"2024-08-31T07:10:46.041708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:34.302665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192670+0000","last_change":"2024-08-31T07:09:47.660615+0000","last_active":"2024-08-31T07:11:45.192670+0000","last_peered":"2024-08-31T07:11:45.192670+0000","last_clean":"2024-08-31T07:11:45.192670+0000","last_became_active":"2024-08-31T07:09:47.660269+0000","last_became_peered":"2024-08-31T07:09:47.660269+0000","last_unstale":"2024-08-31T07:11:45.192670+0000","last_undegraded":"2024-08-31T07:11:45.192670+0000","last_fullsized":"2024-08-31T07:11:45.192670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:42:12.618039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020701+0000","last_change":"2024-08-31T07:09:45.641699+0000","last_active":"2024-08-31T07:10:46.020701+0000","last_peered":"2024-08-31T07:10:46.020701+0000","last_clean":"2024-08-31T07:10:46.020701+0000","last_became_active":"2024-08-31T07:09:45.641122+0000","last_became_peered":"2024-08-31T07:09:45.641122+0000","last_unstale":"2024-08-31T07:10:46.020701+0000","last_undegraded":"2024-08-31T07:10:46.020701+0000","last_fullsized":"2024-08-31T07:10:46.020701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:26:07.130266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200241+0000","last_change":"2024-08-31T07:09:47.675459+0000","last_active":"2024-08-31T07:11:45.200241+0000","last_peered":"2024-08-31T07:11:45.200241+0000","last_clean":"2024-08-31T07:11:45.200241+0000","last_became_active":"2024-08-31T07:09:47.672246+0000","last_became_peered":"2024-08-31T07:09:47.672246+0000","last_unstale":"2024-08-31T07:11:45.200241+0000","last_undegraded":"2024-08-31T07:11:45.200241+0000","last_fullsized":"2024-08-31T07:11:45.200241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:49:23.547352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016428+0000","last_change":"2024-08-31T07:09:45.649522+0000","last_active":"2024-08-31T07:10:46.016428+0000","last_peered":"2024-08-31T07:10:46.016428+0000","last_clean":"2024-08-31T07:10:46.016428+0000","last_became_active":"2024-08-31T07:09:45.649109+0000","last_became_peered":"2024-08-31T07:09:45.649109+0000","last_unstale":"2024-08-31T07:10:46.016428+0000","last_undegraded":"2024-08-31T07:10:46.016428+0000","last_fullsized":"2024-08-31T07:10:46.016428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:52:52.432605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.179375+0000","last_change":"2024-08-31T07:09:47.670583+0000","last_active":"2024-08-31T07:11:45.179375+0000","last_peered":"2024-08-31T07:11:45.179375+0000","last_clean":"2024-08-31T07:11:45.179375+0000","last_became_active":"2024-08-31T07:09:47.670162+0000","last_became_peered":"2024-08-31T07:09:47.670162+0000","last_unstale":"2024-08-31T07:11:45.179375+0000","last_undegraded":"2024-08-31T07:11:45.179375+0000","last_fullsized":"2024-08-31T07:11:45.179375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:35:35.135678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'14","reported_seq":184,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:24.383057+0000","last_change":"2024-08-31T07:09:45.639636+0000","last_active":"2024-08-31T07:12:24.383057+0000","last_peered":"2024-08-31T07:12:24.383057+0000","last_clean":"2024-08-31T07:12:24.383057+0000","last_became_active":"2024-08-31T07:09:45.639029+0000","last_became_peered":"2024-08-31T07:09:45.639029+0000","last_unstale":"2024-08-31T07:12:24.383057+0000","last_undegraded":"2024-08-31T07:12:24.383057+0000","last_fullsized":"2024-08-31T07:12:24.383057+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:54.110149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":862092,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1785,"num_read_kb":1104,"num_write":489,"num_write_kb":2245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":650,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":770048,"data_stored":2194776,"data_compressed":93670,"data_compressed_allocated":122880,"data_compressed_original":1916928,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285320,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":434494,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874290,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1091423,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037816,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":826458,"data_compressed":25040,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234047,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":826254,"data_compressed":25023,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397571,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1027343,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561095,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":433491,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":1419733,"data_compressed":60116,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":433352,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117081,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432784,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378719,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":501638,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542245,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":502092,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705769,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":434830,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":787422,"data_compressed":37001,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":397128,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394134,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T07:12:28.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T07:12:28.853 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:28 smithi032 ceph-mon[17252]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-08-31T07:12:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:28 smithi073 ceph-mon[21995]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-08-31T07:12:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:28 smithi084 ceph-mon[22387]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-08-31T07:12:29.184 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:29.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:29 smithi032 ceph-mon[17252]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:29.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.841+0000 7fc039600640 1 -- 172.21.15.32:0/1405061279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc034103750 msgr2=0x7fc034103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:29.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.841+0000 7fc039600640 1 --2- 172.21.15.32:0/1405061279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc034103750 0x7fc034103b60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc024009900 tx=0x7fc02402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:29.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.841+0000 7fc039600640 1 -- 172.21.15.32:0/1405061279 shutdown_connections 2024-08-31T07:12:29.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.841+0000 7fc039600640 1 --2- 172.21.15.32:0/1405061279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0341099e0 0x7fc034110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:29.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.841+0000 7fc039600640 1 --2- 172.21.15.32:0/1405061279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc034109090 0x7fc0341094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:29.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.841+0000 7fc039600640 1 --2- 172.21.15.32:0/1405061279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc034103750 0x7fc034103b60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:29.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.841+0000 7fc039600640 1 -- 172.21.15.32:0/1405061279 >> 172.21.15.32:0/1405061279 conn(0x7fc0340ff1a0 msgr2=0x7fc0341015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:29.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.841+0000 7fc039600640 1 -- 172.21.15.32:0/1405061279 shutdown_connections 2024-08-31T07:12:29.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.841+0000 7fc039600640 1 -- 172.21.15.32:0/1405061279 wait complete. 2024-08-31T07:12:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.842+0000 7fc039600640 1 Processor -- start 2024-08-31T07:12:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.842+0000 7fc039600640 1 -- start start 2024-08-31T07:12:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc039600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc034103750 0x7fc03419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc039600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc034109090 0x7fc03419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc039600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0341099e0 0x7fc03419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc039600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc034113700 con 0x7fc034103750 2024-08-31T07:12:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc039600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc034113580 con 0x7fc034109090 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc039600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc034113880 con 0x7fc0341099e0 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc033e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc034103750 0x7fc03419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc033400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc034109090 0x7fc03419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc033e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc034103750 0x7fc03419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52046/0 (socket says 172.21.15.32:52046) 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc033e00640 1 -- 172.21.15.32:0/286309444 learned_addr learned my addr 172.21.15.32:0/286309444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc038c00640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0341099e0 0x7fc03419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc033400640 1 -- 172.21.15.32:0/286309444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0341099e0 msgr2=0x7fc03419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc033400640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0341099e0 0x7fc03419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc033400640 1 -- 172.21.15.32:0/286309444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc034103750 msgr2=0x7fc03419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc033400640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc034103750 0x7fc03419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.843+0000 7fc033400640 1 -- 172.21.15.32:0/286309444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03419ffe0 con 0x7fc034109090 2024-08-31T07:12:29.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.844+0000 7fc033400640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc034109090 0x7fc03419a610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc03406b6d0 tx=0x7fc02000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:29.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.844+0000 7fc030c00640 1 -- 172.21.15.32:0/286309444 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02000cc40 con 0x7fc034109090 2024-08-31T07:12:29.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.844+0000 7fc030c00640 1 -- 172.21.15.32:0/286309444 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc02000cde0 con 0x7fc034109090 2024-08-31T07:12:29.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.844+0000 7fc039600640 1 -- 172.21.15.32:0/286309444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0341a02d0 con 0x7fc034109090 2024-08-31T07:12:29.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.844+0000 7fc030c00640 1 -- 172.21.15.32:0/286309444 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020018780 con 0x7fc034109090 2024-08-31T07:12:29.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.844+0000 7fc039600640 1 -- 172.21.15.32:0/286309444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc034075390 con 0x7fc034109090 2024-08-31T07:12:29.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.845+0000 7fc039600640 1 -- 172.21.15.32:0/286309444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc034103b60 con 0x7fc034109090 2024-08-31T07:12:29.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.848+0000 7fc030c00640 1 -- 172.21.15.32:0/286309444 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7fc020010a90 con 0x7fc034109090 2024-08-31T07:12:29.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.849+0000 7fc030c00640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0180b4c10 0x7fc0180b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:29.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.849+0000 7fc030c00640 1 -- 172.21.15.32:0/286309444 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fc0200e56d0 con 0x7fc034109090 2024-08-31T07:12:29.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.849+0000 7fc033e00640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0180b4c10 0x7fc0180b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:29.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.850+0000 7fc033e00640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0180b4c10 0x7fc0180b7120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc0240096c0 tx=0x7fc024005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:29.853+0000 7fc030c00640 1 -- 172.21.15.32:0/286309444 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc0200abf50 con 0x7fc034109090 2024-08-31T07:12:29.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:29 smithi073 ceph-mon[21995]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:29.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:29 smithi084 ceph-mon[22387]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.043+0000 7fc039600640 1 -- 172.21.15.32:0/286309444 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc03410b7c0 con 0x7fc0180b4c10 2024-08-31T07:12:30.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.048+0000 7fc030c00640 1 -- 172.21.15.32:0/286309444 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259282 (secure 0 0 0) 0x7fc03410b7c0 con 0x7fc0180b4c10 2024-08-31T07:12:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-08-31T07:12:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.051+0000 7fc039600640 1 -- 172.21.15.32:0/286309444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0180b4c10 msgr2=0x7fc0180b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.051+0000 7fc039600640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0180b4c10 0x7fc0180b7120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc0240096c0 tx=0x7fc024005a20 comp rx=0 tx=0).stop 2024-08-31T07:12:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.051+0000 7fc039600640 1 -- 172.21.15.32:0/286309444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc034109090 msgr2=0x7fc03419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.051+0000 7fc039600640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc034109090 0x7fc03419a610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc03406b6d0 tx=0x7fc02000ee50 comp rx=0 tx=0).stop 2024-08-31T07:12:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.051+0000 7fc039600640 1 -- 172.21.15.32:0/286309444 shutdown_connections 2024-08-31T07:12:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.051+0000 7fc039600640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0180b4c10 0x7fc0180b7120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.051+0000 7fc039600640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0341099e0 0x7fc03419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.051+0000 7fc039600640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc034109090 0x7fc03419a610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.051+0000 7fc039600640 1 --2- 172.21.15.32:0/286309444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc034103750 0x7fc03419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.051+0000 7fc039600640 1 -- 172.21.15.32:0/286309444 >> 172.21.15.32:0/286309444 conn(0x7fc0340ff1a0 msgr2=0x7fc0341015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.052+0000 7fc039600640 1 -- 172.21.15.32:0/286309444 shutdown_connections 2024-08-31T07:12:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:30.052+0000 7fc039600640 1 -- 172.21.15.32:0/286309444 wait complete. 2024-08-31T07:12:30.721 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:30 smithi032 ceph-mon[17252]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-08-31T07:12:30.760 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-08-31T07:12:28.801249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1453832,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1898,"num_read_kb":1196,"num_write":615,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2952490,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":966,"ondisk_log_size":966,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364392,"kb_used_data":10976,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577432,"statfs":{"total":1151940427776,"available":1151567290368,"internally_reserved":0,"allocated":11239424,"data_stored":8758806,"data_compressed":252577,"data_compressed_allocated":331776,"data_compressed_original":3489792,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":793503,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":469,"num_read_kb":380,"num_write":226,"num_write_kb":2096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003688"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198399+0000","last_change":"2024-08-31T07:09:47.665725+0000","last_active":"2024-08-31T07:11:45.198399+0000","last_peered":"2024-08-31T07:11:45.198399+0000","last_clean":"2024-08-31T07:11:45.198399+0000","last_became_active":"2024-08-31T07:09:47.665368+0000","last_became_peered":"2024-08-31T07:09:47.665368+0000","last_unstale":"2024-08-31T07:11:45.198399+0000","last_undegraded":"2024-08-31T07:11:45.198399+0000","last_fullsized":"2024-08-31T07:11:45.198399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:27:01.636099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192497+0000","last_change":"2024-08-31T07:09:47.676882+0000","last_active":"2024-08-31T07:11:45.192497+0000","last_peered":"2024-08-31T07:11:45.192497+0000","last_clean":"2024-08-31T07:11:45.192497+0000","last_became_active":"2024-08-31T07:09:47.674825+0000","last_became_peered":"2024-08-31T07:09:47.674825+0000","last_unstale":"2024-08-31T07:11:45.192497+0000","last_undegraded":"2024-08-31T07:11:45.192497+0000","last_fullsized":"2024-08-31T07:11:45.192497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:09.148398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199170+0000","last_change":"2024-08-31T07:09:47.674480+0000","last_active":"2024-08-31T07:11:45.199170+0000","last_peered":"2024-08-31T07:11:45.199170+0000","last_clean":"2024-08-31T07:11:45.199170+0000","last_became_active":"2024-08-31T07:09:47.672316+0000","last_became_peered":"2024-08-31T07:09:47.672316+0000","last_unstale":"2024-08-31T07:11:45.199170+0000","last_undegraded":"2024-08-31T07:11:45.199170+0000","last_fullsized":"2024-08-31T07:11:45.199170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:28:13.231039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.172744+0000","last_change":"2024-08-31T07:09:47.659276+0000","last_active":"2024-08-31T07:11:45.172744+0000","last_peered":"2024-08-31T07:11:45.172744+0000","last_clean":"2024-08-31T07:11:45.172744+0000","last_became_active":"2024-08-31T07:09:47.658924+0000","last_became_peered":"2024-08-31T07:09:47.658924+0000","last_unstale":"2024-08-31T07:11:45.172744+0000","last_undegraded":"2024-08-31T07:11:45.172744+0000","last_fullsized":"2024-08-31T07:11:45.172744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:01:08.134245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.174959+0000","last_change":"2024-08-31T07:09:47.667260+0000","last_active":"2024-08-31T07:11:45.174959+0000","last_peered":"2024-08-31T07:11:45.174959+0000","last_clean":"2024-08-31T07:11:45.174959+0000","last_became_active":"2024-08-31T07:09:47.664572+0000","last_became_peered":"2024-08-31T07:09:47.664572+0000","last_unstale":"2024-08-31T07:11:45.174959+0000","last_undegraded":"2024-08-31T07:11:45.174959+0000","last_fullsized":"2024-08-31T07:11:45.174959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:11.336985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193916+0000","last_change":"2024-08-31T07:09:47.661181+0000","last_active":"2024-08-31T07:11:45.193916+0000","last_peered":"2024-08-31T07:11:45.193916+0000","last_clean":"2024-08-31T07:11:45.193916+0000","last_became_active":"2024-08-31T07:09:47.660761+0000","last_became_peered":"2024-08-31T07:09:47.660761+0000","last_unstale":"2024-08-31T07:11:45.193916+0000","last_undegraded":"2024-08-31T07:11:45.193916+0000","last_fullsized":"2024-08-31T07:11:45.193916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:50:54.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196590+0000","last_change":"2024-08-31T07:09:47.668762+0000","last_active":"2024-08-31T07:11:45.196590+0000","last_peered":"2024-08-31T07:11:45.196590+0000","last_clean":"2024-08-31T07:11:45.196590+0000","last_became_active":"2024-08-31T07:09:47.668406+0000","last_became_peered":"2024-08-31T07:09:47.668406+0000","last_unstale":"2024-08-31T07:11:45.196590+0000","last_undegraded":"2024-08-31T07:11:45.196590+0000","last_fullsized":"2024-08-31T07:11:45.196590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:43:39.868144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.191067+0000","last_change":"2024-08-31T07:09:47.669455+0000","last_active":"2024-08-31T07:11:45.191067+0000","last_peered":"2024-08-31T07:11:45.191067+0000","last_clean":"2024-08-31T07:11:45.191067+0000","last_became_active":"2024-08-31T07:09:47.663669+0000","last_became_peered":"2024-08-31T07:09:47.663669+0000","last_unstale":"2024-08-31T07:11:45.191067+0000","last_undegraded":"2024-08-31T07:11:45.191067+0000","last_fullsized":"2024-08-31T07:11:45.191067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:22:59.685012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194827+0000","last_change":"2024-08-31T07:09:47.664214+0000","last_active":"2024-08-31T07:11:45.194827+0000","last_peered":"2024-08-31T07:11:45.194827+0000","last_clean":"2024-08-31T07:11:45.194827+0000","last_became_active":"2024-08-31T07:09:47.663832+0000","last_became_peered":"2024-08-31T07:09:47.663832+0000","last_unstale":"2024-08-31T07:11:45.194827+0000","last_undegraded":"2024-08-31T07:11:45.194827+0000","last_fullsized":"2024-08-31T07:11:45.194827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:29:14.107212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194866+0000","last_change":"2024-08-31T07:10:46.326410+0000","last_active":"2024-08-31T07:11:45.194866+0000","last_peered":"2024-08-31T07:11:45.194866+0000","last_clean":"2024-08-31T07:11:45.194866+0000","last_became_active":"2024-08-31T07:10:46.325958+0000","last_became_peered":"2024-08-31T07:10:46.325958+0000","last_unstale":"2024-08-31T07:11:45.194866+0000","last_undegraded":"2024-08-31T07:11:45.194866+0000","last_fullsized":"2024-08-31T07:11:45.194866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:19.230393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196579+0000","last_change":"2024-08-31T07:09:47.666951+0000","last_active":"2024-08-31T07:11:45.196579+0000","last_peered":"2024-08-31T07:11:45.196579+0000","last_clean":"2024-08-31T07:11:45.196579+0000","last_became_active":"2024-08-31T07:09:47.666475+0000","last_became_peered":"2024-08-31T07:09:47.666475+0000","last_unstale":"2024-08-31T07:11:45.196579+0000","last_undegraded":"2024-08-31T07:11:45.196579+0000","last_fullsized":"2024-08-31T07:11:45.196579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:58:42.242086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.169998+0000","last_change":"2024-08-31T07:09:47.658344+0000","last_active":"2024-08-31T07:11:45.169998+0000","last_peered":"2024-08-31T07:11:45.169998+0000","last_clean":"2024-08-31T07:11:45.169998+0000","last_became_active":"2024-08-31T07:09:47.657991+0000","last_became_peered":"2024-08-31T07:09:47.657991+0000","last_unstale":"2024-08-31T07:11:45.169998+0000","last_undegraded":"2024-08-31T07:11:45.169998+0000","last_fullsized":"2024-08-31T07:11:45.169998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:25:59.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196676+0000","last_change":"2024-08-31T07:09:47.661644+0000","last_active":"2024-08-31T07:11:45.196676+0000","last_peered":"2024-08-31T07:11:45.196676+0000","last_clean":"2024-08-31T07:11:45.196676+0000","last_became_active":"2024-08-31T07:09:47.661248+0000","last_became_peered":"2024-08-31T07:09:47.661248+0000","last_unstale":"2024-08-31T07:11:45.196676+0000","last_undegraded":"2024-08-31T07:11:45.196676+0000","last_fullsized":"2024-08-31T07:11:45.196676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:42.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195194+0000","last_change":"2024-08-31T07:09:47.680052+0000","last_active":"2024-08-31T07:11:45.195194+0000","last_peered":"2024-08-31T07:11:45.195194+0000","last_clean":"2024-08-31T07:11:45.195194+0000","last_became_active":"2024-08-31T07:09:47.679604+0000","last_became_peered":"2024-08-31T07:09:47.679604+0000","last_unstale":"2024-08-31T07:11:45.195194+0000","last_undegraded":"2024-08-31T07:11:45.195194+0000","last_fullsized":"2024-08-31T07:11:45.195194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:23:18.937470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195477+0000","last_change":"2024-08-31T07:09:47.677984+0000","last_active":"2024-08-31T07:11:45.195477+0000","last_peered":"2024-08-31T07:11:45.195477+0000","last_clean":"2024-08-31T07:11:45.195477+0000","last_became_active":"2024-08-31T07:09:47.674957+0000","last_became_peered":"2024-08-31T07:09:47.674957+0000","last_unstale":"2024-08-31T07:11:45.195477+0000","last_undegraded":"2024-08-31T07:11:45.195477+0000","last_fullsized":"2024-08-31T07:11:45.195477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:08:26.503746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198878+0000","last_change":"2024-08-31T07:09:47.672625+0000","last_active":"2024-08-31T07:11:45.198878+0000","last_peered":"2024-08-31T07:11:45.198878+0000","last_clean":"2024-08-31T07:11:45.198878+0000","last_became_active":"2024-08-31T07:09:47.672184+0000","last_became_peered":"2024-08-31T07:09:47.672184+0000","last_unstale":"2024-08-31T07:11:45.198878+0000","last_undegraded":"2024-08-31T07:11:45.198878+0000","last_fullsized":"2024-08-31T07:11:45.198878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:17.091659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198670+0000","last_change":"2024-08-31T07:09:47.672597+0000","last_active":"2024-08-31T07:11:45.198670+0000","last_peered":"2024-08-31T07:11:45.198670+0000","last_clean":"2024-08-31T07:11:45.198670+0000","last_became_active":"2024-08-31T07:09:47.672153+0000","last_became_peered":"2024-08-31T07:09:47.672153+0000","last_unstale":"2024-08-31T07:11:45.198670+0000","last_undegraded":"2024-08-31T07:11:45.198670+0000","last_fullsized":"2024-08-31T07:11:45.198670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:26:41.388526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193223+0000","last_change":"2024-08-31T07:09:47.666253+0000","last_active":"2024-08-31T07:11:45.193223+0000","last_peered":"2024-08-31T07:11:45.193223+0000","last_clean":"2024-08-31T07:11:45.193223+0000","last_became_active":"2024-08-31T07:09:47.663732+0000","last_became_peered":"2024-08-31T07:09:47.663732+0000","last_unstale":"2024-08-31T07:11:45.193223+0000","last_undegraded":"2024-08-31T07:11:45.193223+0000","last_fullsized":"2024-08-31T07:11:45.193223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:10:59.887765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188202+0000","last_change":"2024-08-31T07:09:47.668502+0000","last_active":"2024-08-31T07:11:45.188202+0000","last_peered":"2024-08-31T07:11:45.188202+0000","last_clean":"2024-08-31T07:11:45.188202+0000","last_became_active":"2024-08-31T07:09:47.666487+0000","last_became_peered":"2024-08-31T07:09:47.666487+0000","last_unstale":"2024-08-31T07:11:45.188202+0000","last_undegraded":"2024-08-31T07:11:45.188202+0000","last_fullsized":"2024-08-31T07:11:45.188202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:49:25.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.184945+0000","last_change":"2024-08-31T07:09:47.659454+0000","last_active":"2024-08-31T07:11:45.184945+0000","last_peered":"2024-08-31T07:11:45.184945+0000","last_clean":"2024-08-31T07:11:45.184945+0000","last_became_active":"2024-08-31T07:09:47.658976+0000","last_became_peered":"2024-08-31T07:09:47.658976+0000","last_unstale":"2024-08-31T07:11:45.184945+0000","last_undegraded":"2024-08-31T07:11:45.184945+0000","last_fullsized":"2024-08-31T07:11:45.184945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:31.374905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199707+0000","last_change":"2024-08-31T07:09:47.673612+0000","last_active":"2024-08-31T07:11:45.199707+0000","last_peered":"2024-08-31T07:11:45.199707+0000","last_clean":"2024-08-31T07:11:45.199707+0000","last_became_active":"2024-08-31T07:09:47.673137+0000","last_became_peered":"2024-08-31T07:09:47.673137+0000","last_unstale":"2024-08-31T07:11:45.199707+0000","last_undegraded":"2024-08-31T07:11:45.199707+0000","last_fullsized":"2024-08-31T07:11:45.199707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:46:11.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194525+0000","last_change":"2024-08-31T07:09:47.669195+0000","last_active":"2024-08-31T07:11:45.194525+0000","last_peered":"2024-08-31T07:11:45.194525+0000","last_clean":"2024-08-31T07:11:45.194525+0000","last_became_active":"2024-08-31T07:09:47.668790+0000","last_became_peered":"2024-08-31T07:09:47.668790+0000","last_unstale":"2024-08-31T07:11:45.194525+0000","last_undegraded":"2024-08-31T07:11:45.194525+0000","last_fullsized":"2024-08-31T07:11:45.194525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:27:14.393931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.181333+0000","last_change":"2024-08-31T07:09:47.667566+0000","last_active":"2024-08-31T07:11:45.181333+0000","last_peered":"2024-08-31T07:11:45.181333+0000","last_clean":"2024-08-31T07:11:45.181333+0000","last_became_active":"2024-08-31T07:09:47.666605+0000","last_became_peered":"2024-08-31T07:09:47.666605+0000","last_unstale":"2024-08-31T07:11:45.181333+0000","last_undegraded":"2024-08-31T07:11:45.181333+0000","last_fullsized":"2024-08-31T07:11:45.181333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:05:44.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192240+0000","last_change":"2024-08-31T07:09:47.660478+0000","last_active":"2024-08-31T07:11:45.192240+0000","last_peered":"2024-08-31T07:11:45.192240+0000","last_clean":"2024-08-31T07:11:45.192240+0000","last_became_active":"2024-08-31T07:09:47.660107+0000","last_became_peered":"2024-08-31T07:09:47.660107+0000","last_unstale":"2024-08-31T07:11:45.192240+0000","last_undegraded":"2024-08-31T07:11:45.192240+0000","last_fullsized":"2024-08-31T07:11:45.192240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:29.133370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188131+0000","last_change":"2024-08-31T07:09:47.671967+0000","last_active":"2024-08-31T07:11:45.188131+0000","last_peered":"2024-08-31T07:11:45.188131+0000","last_clean":"2024-08-31T07:11:45.188131+0000","last_became_active":"2024-08-31T07:09:47.671569+0000","last_became_peered":"2024-08-31T07:09:47.671569+0000","last_unstale":"2024-08-31T07:11:45.188131+0000","last_undegraded":"2024-08-31T07:11:45.188131+0000","last_fullsized":"2024-08-31T07:11:45.188131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:19:15.107958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185867+0000","last_change":"2024-08-31T07:09:47.651055+0000","last_active":"2024-08-31T07:11:45.185867+0000","last_peered":"2024-08-31T07:11:45.185867+0000","last_clean":"2024-08-31T07:11:45.185867+0000","last_became_active":"2024-08-31T07:09:47.650674+0000","last_became_peered":"2024-08-31T07:09:47.650674+0000","last_unstale":"2024-08-31T07:11:45.185867+0000","last_undegraded":"2024-08-31T07:11:45.185867+0000","last_fullsized":"2024-08-31T07:11:45.185867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:35:54.647379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180140+0000","last_change":"2024-08-31T07:09:47.663231+0000","last_active":"2024-08-31T07:11:45.180140+0000","last_peered":"2024-08-31T07:11:45.180140+0000","last_clean":"2024-08-31T07:11:45.180140+0000","last_became_active":"2024-08-31T07:09:47.662854+0000","last_became_peered":"2024-08-31T07:09:47.662854+0000","last_unstale":"2024-08-31T07:11:45.180140+0000","last_undegraded":"2024-08-31T07:11:45.180140+0000","last_fullsized":"2024-08-31T07:11:45.180140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:51.358557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199287+0000","last_change":"2024-08-31T07:10:46.326380+0000","last_active":"2024-08-31T07:11:45.199287+0000","last_peered":"2024-08-31T07:11:45.199287+0000","last_clean":"2024-08-31T07:11:45.199287+0000","last_became_active":"2024-08-31T07:10:46.325784+0000","last_became_peered":"2024-08-31T07:10:46.325784+0000","last_unstale":"2024-08-31T07:11:45.199287+0000","last_undegraded":"2024-08-31T07:11:45.199287+0000","last_fullsized":"2024-08-31T07:11:45.199287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:16:56.396024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200247+0000","last_change":"2024-08-31T07:09:47.672108+0000","last_active":"2024-08-31T07:11:45.200247+0000","last_peered":"2024-08-31T07:11:45.200247+0000","last_clean":"2024-08-31T07:11:45.200247+0000","last_became_active":"2024-08-31T07:09:47.670998+0000","last_became_peered":"2024-08-31T07:09:47.670998+0000","last_unstale":"2024-08-31T07:11:45.200247+0000","last_undegraded":"2024-08-31T07:11:45.200247+0000","last_fullsized":"2024-08-31T07:11:45.200247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:26:49.319861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198226+0000","last_change":"2024-08-31T07:09:47.665548+0000","last_active":"2024-08-31T07:11:45.198226+0000","last_peered":"2024-08-31T07:11:45.198226+0000","last_clean":"2024-08-31T07:11:45.198226+0000","last_became_active":"2024-08-31T07:09:47.665177+0000","last_became_peered":"2024-08-31T07:09:47.665177+0000","last_unstale":"2024-08-31T07:11:45.198226+0000","last_undegraded":"2024-08-31T07:11:45.198226+0000","last_fullsized":"2024-08-31T07:11:45.198226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:03:15.892224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.177785+0000","last_change":"2024-08-31T07:10:46.026746+0000","last_active":"2024-08-31T07:11:45.177785+0000","last_peered":"2024-08-31T07:11:45.177785+0000","last_clean":"2024-08-31T07:11:45.177785+0000","last_became_active":"2024-08-31T07:10:46.026286+0000","last_became_peered":"2024-08-31T07:10:46.026286+0000","last_unstale":"2024-08-31T07:11:45.177785+0000","last_undegraded":"2024-08-31T07:11:45.177785+0000","last_fullsized":"2024-08-31T07:11:45.177785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:34:17.120906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180769+0000","last_change":"2024-08-31T07:09:47.667456+0000","last_active":"2024-08-31T07:11:45.180769+0000","last_peered":"2024-08-31T07:11:45.180769+0000","last_clean":"2024-08-31T07:11:45.180769+0000","last_became_active":"2024-08-31T07:09:47.667102+0000","last_became_peered":"2024-08-31T07:09:47.667102+0000","last_unstale":"2024-08-31T07:11:45.180769+0000","last_undegraded":"2024-08-31T07:11:45.180769+0000","last_fullsized":"2024-08-31T07:11:45.180769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:44:39.747729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.182194+0000","last_change":"2024-08-31T07:10:46.326139+0000","last_active":"2024-08-31T07:11:45.182194+0000","last_peered":"2024-08-31T07:11:45.182194+0000","last_clean":"2024-08-31T07:11:45.182194+0000","last_became_active":"2024-08-31T07:10:46.325785+0000","last_became_peered":"2024-08-31T07:10:46.325785+0000","last_unstale":"2024-08-31T07:11:45.182194+0000","last_undegraded":"2024-08-31T07:11:45.182194+0000","last_fullsized":"2024-08-31T07:11:45.182194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:13.495835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":180,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:25.108048+0000","last_change":"2024-08-31T07:09:45.641088+0000","last_active":"2024-08-31T07:12:25.108048+0000","last_peered":"2024-08-31T07:12:25.108048+0000","last_clean":"2024-08-31T07:12:25.108048+0000","last_became_active":"2024-08-31T07:09:45.640673+0000","last_became_peered":"2024-08-31T07:09:45.640673+0000","last_unstale":"2024-08-31T07:12:25.108048+0000","last_undegraded":"2024-08-31T07:12:25.108048+0000","last_fullsized":"2024-08-31T07:12:25.108048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:30:17.593494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192346+0000","last_change":"2024-08-31T07:09:47.666703+0000","last_active":"2024-08-31T07:11:45.192346+0000","last_peered":"2024-08-31T07:11:45.192346+0000","last_clean":"2024-08-31T07:11:45.192346+0000","last_became_active":"2024-08-31T07:09:47.666348+0000","last_became_peered":"2024-08-31T07:09:47.666348+0000","last_unstale":"2024-08-31T07:11:45.192346+0000","last_undegraded":"2024-08-31T07:11:45.192346+0000","last_fullsized":"2024-08-31T07:11:45.192346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:39.688810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019885+0000","last_change":"2024-08-31T07:09:45.642431+0000","last_active":"2024-08-31T07:10:46.019885+0000","last_peered":"2024-08-31T07:10:46.019885+0000","last_clean":"2024-08-31T07:10:46.019885+0000","last_became_active":"2024-08-31T07:09:45.642048+0000","last_became_peered":"2024-08-31T07:09:45.642048+0000","last_unstale":"2024-08-31T07:10:46.019885+0000","last_undegraded":"2024-08-31T07:10:46.019885+0000","last_fullsized":"2024-08-31T07:10:46.019885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:47:42.574442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017286+0000","last_change":"2024-08-31T07:09:45.632162+0000","last_active":"2024-08-31T07:10:46.017286+0000","last_peered":"2024-08-31T07:10:46.017286+0000","last_clean":"2024-08-31T07:10:46.017286+0000","last_became_active":"2024-08-31T07:09:45.631749+0000","last_became_peered":"2024-08-31T07:09:45.631749+0000","last_unstale":"2024-08-31T07:10:46.017286+0000","last_undegraded":"2024-08-31T07:10:46.017286+0000","last_fullsized":"2024-08-31T07:10:46.017286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:16:29.967823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185787+0000","last_change":"2024-08-31T07:09:47.666288+0000","last_active":"2024-08-31T07:11:45.185787+0000","last_peered":"2024-08-31T07:11:45.185787+0000","last_clean":"2024-08-31T07:11:45.185787+0000","last_became_active":"2024-08-31T07:09:47.665723+0000","last_became_peered":"2024-08-31T07:09:47.665723+0000","last_unstale":"2024-08-31T07:11:45.185787+0000","last_undegraded":"2024-08-31T07:11:45.185787+0000","last_fullsized":"2024-08-31T07:11:45.185787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:38:28.122720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023120+0000","last_change":"2024-08-31T07:09:45.636508+0000","last_active":"2024-08-31T07:10:46.023120+0000","last_peered":"2024-08-31T07:10:46.023120+0000","last_clean":"2024-08-31T07:10:46.023120+0000","last_became_active":"2024-08-31T07:09:45.636073+0000","last_became_peered":"2024-08-31T07:09:45.636073+0000","last_unstale":"2024-08-31T07:10:46.023120+0000","last_undegraded":"2024-08-31T07:10:46.023120+0000","last_fullsized":"2024-08-31T07:10:46.023120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:55:45.063517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187943+0000","last_change":"2024-08-31T07:09:47.661306+0000","last_active":"2024-08-31T07:11:45.187943+0000","last_peered":"2024-08-31T07:11:45.187943+0000","last_clean":"2024-08-31T07:11:45.187943+0000","last_became_active":"2024-08-31T07:09:47.660957+0000","last_became_peered":"2024-08-31T07:09:47.660957+0000","last_unstale":"2024-08-31T07:11:45.187943+0000","last_undegraded":"2024-08-31T07:11:45.187943+0000","last_fullsized":"2024-08-31T07:11:45.187943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:18:23.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'6","reported_seq":318,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:26.108577+0000","last_change":"2024-08-31T07:09:45.640947+0000","last_active":"2024-08-31T07:12:26.108577+0000","last_peered":"2024-08-31T07:12:26.108577+0000","last_clean":"2024-08-31T07:12:26.108577+0000","last_became_active":"2024-08-31T07:09:45.640532+0000","last_became_peered":"2024-08-31T07:09:45.640532+0000","last_unstale":"2024-08-31T07:12:26.108577+0000","last_undegraded":"2024-08-31T07:12:26.108577+0000","last_fullsized":"2024-08-31T07:12:26.108577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:28.177795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":286,"num_read_kb":286,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.191708+0000","last_change":"2024-08-31T07:09:47.661060+0000","last_active":"2024-08-31T07:11:45.191708+0000","last_peered":"2024-08-31T07:11:45.191708+0000","last_clean":"2024-08-31T07:11:45.191708+0000","last_became_active":"2024-08-31T07:09:47.660692+0000","last_became_peered":"2024-08-31T07:09:47.660692+0000","last_unstale":"2024-08-31T07:11:45.191708+0000","last_undegraded":"2024-08-31T07:11:45.191708+0000","last_fullsized":"2024-08-31T07:11:45.191708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:48:38.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257194+0000","last_change":"2024-08-31T07:09:45.641155+0000","last_active":"2024-08-31T07:10:46.257194+0000","last_peered":"2024-08-31T07:10:46.257194+0000","last_clean":"2024-08-31T07:10:46.257194+0000","last_became_active":"2024-08-31T07:09:45.640680+0000","last_became_peered":"2024-08-31T07:09:45.640680+0000","last_unstale":"2024-08-31T07:10:46.257194+0000","last_undegraded":"2024-08-31T07:10:46.257194+0000","last_fullsized":"2024-08-31T07:10:46.257194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:29:03.287570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199892+0000","last_change":"2024-08-31T07:09:47.675582+0000","last_active":"2024-08-31T07:11:45.199892+0000","last_peered":"2024-08-31T07:11:45.199892+0000","last_clean":"2024-08-31T07:11:45.199892+0000","last_became_active":"2024-08-31T07:09:47.675266+0000","last_became_peered":"2024-08-31T07:09:47.675266+0000","last_unstale":"2024-08-31T07:11:45.199892+0000","last_undegraded":"2024-08-31T07:11:45.199892+0000","last_fullsized":"2024-08-31T07:11:45.199892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:36:33.122531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021194+0000","last_change":"2024-08-31T07:09:45.651006+0000","last_active":"2024-08-31T07:10:46.021194+0000","last_peered":"2024-08-31T07:10:46.021194+0000","last_clean":"2024-08-31T07:10:46.021194+0000","last_became_active":"2024-08-31T07:09:45.650588+0000","last_became_peered":"2024-08-31T07:09:45.650588+0000","last_unstale":"2024-08-31T07:10:46.021194+0000","last_undegraded":"2024-08-31T07:10:46.021194+0000","last_fullsized":"2024-08-31T07:10:46.021194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:34:16.015892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.190030+0000","last_change":"2024-08-31T07:09:47.662289+0000","last_active":"2024-08-31T07:11:45.190030+0000","last_peered":"2024-08-31T07:11:45.190030+0000","last_clean":"2024-08-31T07:11:45.190030+0000","last_became_active":"2024-08-31T07:09:47.661932+0000","last_became_peered":"2024-08-31T07:09:47.661932+0000","last_unstale":"2024-08-31T07:11:45.190030+0000","last_undegraded":"2024-08-31T07:11:45.190030+0000","last_fullsized":"2024-08-31T07:11:45.190030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:50.440504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'38","reported_seq":364,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:26.109365+0000","last_change":"2024-08-31T07:09:45.642196+0000","last_active":"2024-08-31T07:12:26.109365+0000","last_peered":"2024-08-31T07:12:26.109365+0000","last_clean":"2024-08-31T07:12:26.109365+0000","last_became_active":"2024-08-31T07:09:45.641797+0000","last_became_peered":"2024-08-31T07:09:45.641797+0000","last_unstale":"2024-08-31T07:12:26.109365+0000","last_undegraded":"2024-08-31T07:12:26.109365+0000","last_fullsized":"2024-08-31T07:12:26.109365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:51:05.953335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67688,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":300,"num_read_kb":217,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196022+0000","last_change":"2024-08-31T07:09:47.660184+0000","last_active":"2024-08-31T07:11:45.196022+0000","last_peered":"2024-08-31T07:11:45.196022+0000","last_clean":"2024-08-31T07:11:45.196022+0000","last_became_active":"2024-08-31T07:09:47.659571+0000","last_became_peered":"2024-08-31T07:09:47.659571+0000","last_unstale":"2024-08-31T07:11:45.196022+0000","last_undegraded":"2024-08-31T07:11:45.196022+0000","last_fullsized":"2024-08-31T07:11:45.196022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:41:13.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.035782+0000","last_change":"2024-08-31T07:10:46.035336+0000","last_active":"2024-08-31T07:10:46.035782+0000","last_peered":"2024-08-31T07:10:46.035782+0000","last_clean":"2024-08-31T07:10:46.035782+0000","last_became_active":"2024-08-31T07:10:46.021299+0000","last_became_peered":"2024-08-31T07:10:46.021299+0000","last_unstale":"2024-08-31T07:10:46.035782+0000","last_undegraded":"2024-08-31T07:10:46.035782+0000","last_fullsized":"2024-08-31T07:10:46.035782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:06:15.931855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194680+0000","last_change":"2024-08-31T07:10:46.326354+0000","last_active":"2024-08-31T07:11:45.194680+0000","last_peered":"2024-08-31T07:11:45.194680+0000","last_clean":"2024-08-31T07:11:45.194680+0000","last_became_active":"2024-08-31T07:10:46.325869+0000","last_became_peered":"2024-08-31T07:10:46.325869+0000","last_unstale":"2024-08-31T07:11:45.194680+0000","last_undegraded":"2024-08-31T07:11:45.194680+0000","last_fullsized":"2024-08-31T07:11:45.194680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:21.832088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022972+0000","last_change":"2024-08-31T07:09:45.642187+0000","last_active":"2024-08-31T07:10:46.022972+0000","last_peered":"2024-08-31T07:10:46.022972+0000","last_clean":"2024-08-31T07:10:46.022972+0000","last_became_active":"2024-08-31T07:09:45.641721+0000","last_became_peered":"2024-08-31T07:09:45.641721+0000","last_unstale":"2024-08-31T07:10:46.022972+0000","last_undegraded":"2024-08-31T07:10:46.022972+0000","last_fullsized":"2024-08-31T07:10:46.022972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:00:45.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196019+0000","last_change":"2024-08-31T07:09:47.665956+0000","last_active":"2024-08-31T07:11:45.196019+0000","last_peered":"2024-08-31T07:11:45.196019+0000","last_clean":"2024-08-31T07:11:45.196019+0000","last_became_active":"2024-08-31T07:09:47.664367+0000","last_became_peered":"2024-08-31T07:09:47.664367+0000","last_unstale":"2024-08-31T07:11:45.196019+0000","last_undegraded":"2024-08-31T07:11:45.196019+0000","last_fullsized":"2024-08-31T07:11:45.196019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:34.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'7","reported_seq":180,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:25.107094+0000","last_change":"2024-08-31T07:09:45.636440+0000","last_active":"2024-08-31T07:12:25.107094+0000","last_peered":"2024-08-31T07:12:25.107094+0000","last_clean":"2024-08-31T07:12:25.107094+0000","last_became_active":"2024-08-31T07:09:45.636058+0000","last_became_peered":"2024-08-31T07:09:45.636058+0000","last_unstale":"2024-08-31T07:12:25.107094+0000","last_undegraded":"2024-08-31T07:12:25.107094+0000","last_fullsized":"2024-08-31T07:12:25.107094+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:38.929227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194840+0000","last_change":"2024-08-31T07:09:47.660564+0000","last_active":"2024-08-31T07:11:45.194840+0000","last_peered":"2024-08-31T07:11:45.194840+0000","last_clean":"2024-08-31T07:11:45.194840+0000","last_became_active":"2024-08-31T07:09:47.660157+0000","last_became_peered":"2024-08-31T07:09:47.660157+0000","last_unstale":"2024-08-31T07:11:45.194840+0000","last_undegraded":"2024-08-31T07:11:45.194840+0000","last_fullsized":"2024-08-31T07:11:45.194840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:49:01.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020515+0000","last_change":"2024-08-31T07:09:45.649140+0000","last_active":"2024-08-31T07:10:46.020515+0000","last_peered":"2024-08-31T07:10:46.020515+0000","last_clean":"2024-08-31T07:10:46.020515+0000","last_became_active":"2024-08-31T07:09:45.648796+0000","last_became_peered":"2024-08-31T07:09:45.648796+0000","last_unstale":"2024-08-31T07:10:46.020515+0000","last_undegraded":"2024-08-31T07:10:46.020515+0000","last_fullsized":"2024-08-31T07:10:46.020515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:33:37.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192390+0000","last_change":"2024-08-31T07:09:47.663833+0000","last_active":"2024-08-31T07:11:45.192390+0000","last_peered":"2024-08-31T07:11:45.192390+0000","last_clean":"2024-08-31T07:11:45.192390+0000","last_became_active":"2024-08-31T07:09:47.663451+0000","last_became_peered":"2024-08-31T07:09:47.663451+0000","last_unstale":"2024-08-31T07:11:45.192390+0000","last_undegraded":"2024-08-31T07:11:45.192390+0000","last_fullsized":"2024-08-31T07:11:45.192390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:27:22.245254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021407+0000","last_change":"2024-08-31T07:09:45.636593+0000","last_active":"2024-08-31T07:10:46.021407+0000","last_peered":"2024-08-31T07:10:46.021407+0000","last_clean":"2024-08-31T07:10:46.021407+0000","last_became_active":"2024-08-31T07:09:45.636166+0000","last_became_peered":"2024-08-31T07:09:45.636166+0000","last_unstale":"2024-08-31T07:10:46.021407+0000","last_undegraded":"2024-08-31T07:10:46.021407+0000","last_fullsized":"2024-08-31T07:10:46.021407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:28.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183385+0000","last_change":"2024-08-31T07:09:47.660825+0000","last_active":"2024-08-31T07:11:45.183385+0000","last_peered":"2024-08-31T07:11:45.183385+0000","last_clean":"2024-08-31T07:11:45.183385+0000","last_became_active":"2024-08-31T07:09:47.660474+0000","last_became_peered":"2024-08-31T07:09:47.660474+0000","last_unstale":"2024-08-31T07:11:45.183385+0000","last_undegraded":"2024-08-31T07:11:45.183385+0000","last_fullsized":"2024-08-31T07:11:45.183385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:42:20.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017462+0000","last_change":"2024-08-31T07:09:45.632727+0000","last_active":"2024-08-31T07:10:46.017462+0000","last_peered":"2024-08-31T07:10:46.017462+0000","last_clean":"2024-08-31T07:10:46.017462+0000","last_became_active":"2024-08-31T07:09:45.632251+0000","last_became_peered":"2024-08-31T07:09:45.632251+0000","last_unstale":"2024-08-31T07:10:46.017462+0000","last_undegraded":"2024-08-31T07:10:46.017462+0000","last_fullsized":"2024-08-31T07:10:46.017462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:09:30.197409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196430+0000","last_change":"2024-08-31T07:09:47.665501+0000","last_active":"2024-08-31T07:11:45.196430+0000","last_peered":"2024-08-31T07:11:45.196430+0000","last_clean":"2024-08-31T07:11:45.196430+0000","last_became_active":"2024-08-31T07:09:47.665162+0000","last_became_peered":"2024-08-31T07:09:47.665162+0000","last_unstale":"2024-08-31T07:11:45.196430+0000","last_undegraded":"2024-08-31T07:11:45.196430+0000","last_fullsized":"2024-08-31T07:11:45.196430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:34:29.605836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'62","reported_seq":118,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:24.990505+0000","last_change":"2024-08-31T07:09:45.636580+0000","last_active":"2024-08-31T07:12:24.990505+0000","last_peered":"2024-08-31T07:12:24.990505+0000","last_clean":"2024-08-31T07:12:24.990505+0000","last_became_active":"2024-08-31T07:09:45.636163+0000","last_became_peered":"2024-08-31T07:09:45.636163+0000","last_unstale":"2024-08-31T07:12:24.990505+0000","last_undegraded":"2024-08-31T07:12:24.990505+0000","last_fullsized":"2024-08-31T07:12:24.990505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:18:07.567858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195120+0000","last_change":"2024-08-31T07:09:47.671865+0000","last_active":"2024-08-31T07:11:45.195120+0000","last_peered":"2024-08-31T07:11:45.195120+0000","last_clean":"2024-08-31T07:11:45.195120+0000","last_became_active":"2024-08-31T07:09:47.671454+0000","last_became_peered":"2024-08-31T07:09:47.671454+0000","last_unstale":"2024-08-31T07:11:45.195120+0000","last_undegraded":"2024-08-31T07:11:45.195120+0000","last_fullsized":"2024-08-31T07:11:45.195120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:47.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'7","reported_seq":178,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:23.104896+0000","last_change":"2024-08-31T07:09:45.653217+0000","last_active":"2024-08-31T07:12:23.104896+0000","last_peered":"2024-08-31T07:12:23.104896+0000","last_clean":"2024-08-31T07:12:23.104896+0000","last_became_active":"2024-08-31T07:09:45.652819+0000","last_became_peered":"2024-08-31T07:09:45.652819+0000","last_unstale":"2024-08-31T07:12:23.104896+0000","last_undegraded":"2024-08-31T07:12:23.104896+0000","last_fullsized":"2024-08-31T07:12:23.104896+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:25:11.454068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":145,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196166+0000","last_change":"2024-08-31T07:09:47.663652+0000","last_active":"2024-08-31T07:11:45.196166+0000","last_peered":"2024-08-31T07:11:45.196166+0000","last_clean":"2024-08-31T07:11:45.196166+0000","last_became_active":"2024-08-31T07:09:47.663274+0000","last_became_peered":"2024-08-31T07:09:47.663274+0000","last_unstale":"2024-08-31T07:11:45.196166+0000","last_undegraded":"2024-08-31T07:11:45.196166+0000","last_fullsized":"2024-08-31T07:11:45.196166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:31:59.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257370+0000","last_change":"2024-08-31T07:08:27.623298+0000","last_active":"2024-08-31T07:10:46.257370+0000","last_peered":"2024-08-31T07:10:46.257370+0000","last_clean":"2024-08-31T07:10:46.257370+0000","last_became_active":"2024-08-31T07:08:27.619072+0000","last_became_peered":"2024-08-31T07:08:27.619072+0000","last_unstale":"2024-08-31T07:10:46.257370+0000","last_undegraded":"2024-08-31T07:10:46.257370+0000","last_fullsized":"2024-08-31T07:10:46.257370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_clean_scrub_stamp":"2024-08-31T07:04:45.509398+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:07.967421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154319+0000","last_change":"2024-08-31T07:09:45.638104+0000","last_active":"2024-08-31T07:10:46.154319+0000","last_peered":"2024-08-31T07:10:46.154319+0000","last_clean":"2024-08-31T07:10:46.154319+0000","last_became_active":"2024-08-31T07:09:45.637693+0000","last_became_peered":"2024-08-31T07:09:45.637693+0000","last_unstale":"2024-08-31T07:10:46.154319+0000","last_undegraded":"2024-08-31T07:10:46.154319+0000","last_fullsized":"2024-08-31T07:10:46.154319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:50:15.135061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.189353+0000","last_change":"2024-08-31T07:09:47.664958+0000","last_active":"2024-08-31T07:11:45.189353+0000","last_peered":"2024-08-31T07:11:45.189353+0000","last_clean":"2024-08-31T07:11:45.189353+0000","last_became_active":"2024-08-31T07:09:47.661166+0000","last_became_peered":"2024-08-31T07:09:47.661166+0000","last_unstale":"2024-08-31T07:11:45.189353+0000","last_undegraded":"2024-08-31T07:11:45.189353+0000","last_fullsized":"2024-08-31T07:11:45.189353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:45:58.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257184+0000","last_change":"2024-08-31T07:09:45.640881+0000","last_active":"2024-08-31T07:10:46.257184+0000","last_peered":"2024-08-31T07:10:46.257184+0000","last_clean":"2024-08-31T07:10:46.257184+0000","last_became_active":"2024-08-31T07:09:45.640487+0000","last_became_peered":"2024-08-31T07:09:45.640487+0000","last_unstale":"2024-08-31T07:10:46.257184+0000","last_undegraded":"2024-08-31T07:10:46.257184+0000","last_fullsized":"2024-08-31T07:10:46.257184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:06.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.171981+0000","last_change":"2024-08-31T07:09:47.655008+0000","last_active":"2024-08-31T07:11:45.171981+0000","last_peered":"2024-08-31T07:11:45.171981+0000","last_clean":"2024-08-31T07:11:45.171981+0000","last_became_active":"2024-08-31T07:09:47.654645+0000","last_became_peered":"2024-08-31T07:09:47.654645+0000","last_unstale":"2024-08-31T07:11:45.171981+0000","last_undegraded":"2024-08-31T07:11:45.171981+0000","last_fullsized":"2024-08-31T07:11:45.171981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:23:55.916053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'6","reported_seq":178,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:27.109208+0000","last_change":"2024-08-31T07:09:45.649324+0000","last_active":"2024-08-31T07:12:27.109208+0000","last_peered":"2024-08-31T07:12:27.109208+0000","last_clean":"2024-08-31T07:12:27.109208+0000","last_became_active":"2024-08-31T07:09:45.648915+0000","last_became_peered":"2024-08-31T07:09:45.648915+0000","last_unstale":"2024-08-31T07:12:27.109208+0000","last_undegraded":"2024-08-31T07:12:27.109208+0000","last_fullsized":"2024-08-31T07:12:27.109208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:31:57.853654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187160+0000","last_change":"2024-08-31T07:09:47.661328+0000","last_active":"2024-08-31T07:11:45.187160+0000","last_peered":"2024-08-31T07:11:45.187160+0000","last_clean":"2024-08-31T07:11:45.187160+0000","last_became_active":"2024-08-31T07:09:47.660927+0000","last_became_peered":"2024-08-31T07:09:47.660927+0000","last_unstale":"2024-08-31T07:11:45.187160+0000","last_undegraded":"2024-08-31T07:11:45.187160+0000","last_fullsized":"2024-08-31T07:11:45.187160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:40:46.876894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'7","reported_seq":181,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:27.111174+0000","last_change":"2024-08-31T07:09:45.637747+0000","last_active":"2024-08-31T07:12:27.111174+0000","last_peered":"2024-08-31T07:12:27.111174+0000","last_clean":"2024-08-31T07:12:27.111174+0000","last_became_active":"2024-08-31T07:09:45.637368+0000","last_became_peered":"2024-08-31T07:09:45.637368+0000","last_unstale":"2024-08-31T07:12:27.111174+0000","last_undegraded":"2024-08-31T07:12:27.111174+0000","last_fullsized":"2024-08-31T07:12:27.111174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:01.174109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":148,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193217+0000","last_change":"2024-08-31T07:09:47.680802+0000","last_active":"2024-08-31T07:11:45.193217+0000","last_peered":"2024-08-31T07:11:45.193217+0000","last_clean":"2024-08-31T07:11:45.193217+0000","last_became_active":"2024-08-31T07:09:47.680363+0000","last_became_peered":"2024-08-31T07:09:47.680363+0000","last_unstale":"2024-08-31T07:11:45.193217+0000","last_undegraded":"2024-08-31T07:11:45.193217+0000","last_fullsized":"2024-08-31T07:11:45.193217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:46:48.744071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324353+0000","last_change":"2024-08-31T07:09:45.651222+0000","last_active":"2024-08-31T07:10:46.324353+0000","last_peered":"2024-08-31T07:10:46.324353+0000","last_clean":"2024-08-31T07:10:46.324353+0000","last_became_active":"2024-08-31T07:09:45.650760+0000","last_became_peered":"2024-08-31T07:09:45.650760+0000","last_unstale":"2024-08-31T07:10:46.324353+0000","last_undegraded":"2024-08-31T07:10:46.324353+0000","last_fullsized":"2024-08-31T07:10:46.324353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T19:03:33.942165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183968+0000","last_change":"2024-08-31T07:09:47.665881+0000","last_active":"2024-08-31T07:11:45.183968+0000","last_peered":"2024-08-31T07:11:45.183968+0000","last_clean":"2024-08-31T07:11:45.183968+0000","last_became_active":"2024-08-31T07:09:47.665485+0000","last_became_peered":"2024-08-31T07:09:47.665485+0000","last_unstale":"2024-08-31T07:11:45.183968+0000","last_undegraded":"2024-08-31T07:11:45.183968+0000","last_fullsized":"2024-08-31T07:11:45.183968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:30.820714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020851+0000","last_change":"2024-08-31T07:09:45.651893+0000","last_active":"2024-08-31T07:10:46.020851+0000","last_peered":"2024-08-31T07:10:46.020851+0000","last_clean":"2024-08-31T07:10:46.020851+0000","last_became_active":"2024-08-31T07:09:45.651547+0000","last_became_peered":"2024-08-31T07:09:45.651547+0000","last_unstale":"2024-08-31T07:10:46.020851+0000","last_undegraded":"2024-08-31T07:10:46.020851+0000","last_fullsized":"2024-08-31T07:10:46.020851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:11:30.708018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.175560+0000","last_change":"2024-08-31T07:09:47.655551+0000","last_active":"2024-08-31T07:11:45.175560+0000","last_peered":"2024-08-31T07:11:45.175560+0000","last_clean":"2024-08-31T07:11:45.175560+0000","last_became_active":"2024-08-31T07:09:47.655188+0000","last_became_peered":"2024-08-31T07:09:47.655188+0000","last_unstale":"2024-08-31T07:11:45.175560+0000","last_undegraded":"2024-08-31T07:11:45.175560+0000","last_fullsized":"2024-08-31T07:11:45.175560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:43:51.419599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'101","reported_seq":216,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:25.489306+0000","last_change":"2024-08-31T07:09:45.636212+0000","last_active":"2024-08-31T07:12:25.489306+0000","last_peered":"2024-08-31T07:12:25.489306+0000","last_clean":"2024-08-31T07:12:25.489306+0000","last_became_active":"2024-08-31T07:09:45.635678+0000","last_became_peered":"2024-08-31T07:09:45.635678+0000","last_unstale":"2024-08-31T07:12:25.489306+0000","last_undegraded":"2024-08-31T07:12:25.489306+0000","last_fullsized":"2024-08-31T07:12:25.489306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:13:17.803459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":185,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185987+0000","last_change":"2024-08-31T07:09:47.666476+0000","last_active":"2024-08-31T07:11:45.185987+0000","last_peered":"2024-08-31T07:11:45.185987+0000","last_clean":"2024-08-31T07:11:45.185987+0000","last_became_active":"2024-08-31T07:09:47.666111+0000","last_became_peered":"2024-08-31T07:09:47.666111+0000","last_unstale":"2024-08-31T07:11:45.185987+0000","last_undegraded":"2024-08-31T07:11:45.185987+0000","last_fullsized":"2024-08-31T07:11:45.185987+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:08:53.459549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154441+0000","last_change":"2024-08-31T07:09:45.631571+0000","last_active":"2024-08-31T07:10:46.154441+0000","last_peered":"2024-08-31T07:10:46.154441+0000","last_clean":"2024-08-31T07:10:46.154441+0000","last_became_active":"2024-08-31T07:09:45.630981+0000","last_became_peered":"2024-08-31T07:09:45.630981+0000","last_unstale":"2024-08-31T07:10:46.154441+0000","last_undegraded":"2024-08-31T07:10:46.154441+0000","last_fullsized":"2024-08-31T07:10:46.154441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:36:53.482955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193936+0000","last_change":"2024-08-31T07:09:47.675054+0000","last_active":"2024-08-31T07:11:45.193936+0000","last_peered":"2024-08-31T07:11:45.193936+0000","last_clean":"2024-08-31T07:11:45.193936+0000","last_became_active":"2024-08-31T07:09:47.672652+0000","last_became_peered":"2024-08-31T07:09:47.672652+0000","last_unstale":"2024-08-31T07:11:45.193936+0000","last_undegraded":"2024-08-31T07:11:45.193936+0000","last_fullsized":"2024-08-31T07:11:45.193936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:32:05.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016284+0000","last_change":"2024-08-31T07:09:45.648746+0000","last_active":"2024-08-31T07:10:46.016284+0000","last_peered":"2024-08-31T07:10:46.016284+0000","last_clean":"2024-08-31T07:10:46.016284+0000","last_became_active":"2024-08-31T07:09:45.648424+0000","last_became_peered":"2024-08-31T07:09:45.648424+0000","last_unstale":"2024-08-31T07:10:46.016284+0000","last_undegraded":"2024-08-31T07:10:46.016284+0000","last_fullsized":"2024-08-31T07:10:46.016284+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:40:32.132963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.165179+0000","last_change":"2024-08-31T07:09:47.663213+0000","last_active":"2024-08-31T07:11:45.165179+0000","last_peered":"2024-08-31T07:11:45.165179+0000","last_clean":"2024-08-31T07:11:45.165179+0000","last_became_active":"2024-08-31T07:09:47.662457+0000","last_became_peered":"2024-08-31T07:09:47.662457+0000","last_unstale":"2024-08-31T07:11:45.165179+0000","last_undegraded":"2024-08-31T07:11:45.165179+0000","last_fullsized":"2024-08-31T07:11:45.165179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:54:27.069200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015653+0000","last_change":"2024-08-31T07:09:45.640149+0000","last_active":"2024-08-31T07:10:46.015653+0000","last_peered":"2024-08-31T07:10:46.015653+0000","last_clean":"2024-08-31T07:10:46.015653+0000","last_became_active":"2024-08-31T07:09:45.639581+0000","last_became_peered":"2024-08-31T07:09:45.639581+0000","last_unstale":"2024-08-31T07:10:46.015653+0000","last_undegraded":"2024-08-31T07:10:46.015653+0000","last_fullsized":"2024-08-31T07:10:46.015653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:27:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198479+0000","last_change":"2024-08-31T07:09:47.672613+0000","last_active":"2024-08-31T07:11:45.198479+0000","last_peered":"2024-08-31T07:11:45.198479+0000","last_clean":"2024-08-31T07:11:45.198479+0000","last_became_active":"2024-08-31T07:09:47.672173+0000","last_became_peered":"2024-08-31T07:09:47.672173+0000","last_unstale":"2024-08-31T07:11:45.198479+0000","last_undegraded":"2024-08-31T07:11:45.198479+0000","last_fullsized":"2024-08-31T07:11:45.198479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:49.706113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022590+0000","last_change":"2024-08-31T07:09:45.637567+0000","last_active":"2024-08-31T07:10:46.022590+0000","last_peered":"2024-08-31T07:10:46.022590+0000","last_clean":"2024-08-31T07:10:46.022590+0000","last_became_active":"2024-08-31T07:09:45.637180+0000","last_became_peered":"2024-08-31T07:09:45.637180+0000","last_unstale":"2024-08-31T07:10:46.022590+0000","last_undegraded":"2024-08-31T07:10:46.022590+0000","last_fullsized":"2024-08-31T07:10:46.022590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:48:07.190829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194363+0000","last_change":"2024-08-31T07:09:47.674980+0000","last_active":"2024-08-31T07:11:45.194363+0000","last_peered":"2024-08-31T07:11:45.194363+0000","last_clean":"2024-08-31T07:11:45.194363+0000","last_became_active":"2024-08-31T07:09:47.674528+0000","last_became_peered":"2024-08-31T07:09:47.674528+0000","last_unstale":"2024-08-31T07:11:45.194363+0000","last_undegraded":"2024-08-31T07:11:45.194363+0000","last_fullsized":"2024-08-31T07:11:45.194363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:15:09.464221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021096+0000","last_change":"2024-08-31T07:09:45.641167+0000","last_active":"2024-08-31T07:10:46.021096+0000","last_peered":"2024-08-31T07:10:46.021096+0000","last_clean":"2024-08-31T07:10:46.021096+0000","last_became_active":"2024-08-31T07:09:45.640761+0000","last_became_peered":"2024-08-31T07:09:45.640761+0000","last_unstale":"2024-08-31T07:10:46.021096+0000","last_undegraded":"2024-08-31T07:10:46.021096+0000","last_fullsized":"2024-08-31T07:10:46.021096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:28:59.963145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195596+0000","last_change":"2024-08-31T07:09:47.668808+0000","last_active":"2024-08-31T07:11:45.195596+0000","last_peered":"2024-08-31T07:11:45.195596+0000","last_clean":"2024-08-31T07:11:45.195596+0000","last_became_active":"2024-08-31T07:09:47.668291+0000","last_became_peered":"2024-08-31T07:09:47.668291+0000","last_unstale":"2024-08-31T07:11:45.195596+0000","last_undegraded":"2024-08-31T07:11:45.195596+0000","last_fullsized":"2024-08-31T07:11:45.195596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:23.433511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193074+0000","last_change":"2024-08-31T07:09:47.668039+0000","last_active":"2024-08-31T07:11:45.193074+0000","last_peered":"2024-08-31T07:11:45.193074+0000","last_clean":"2024-08-31T07:11:45.193074+0000","last_became_active":"2024-08-31T07:09:47.667567+0000","last_became_peered":"2024-08-31T07:09:47.667567+0000","last_unstale":"2024-08-31T07:11:45.193074+0000","last_undegraded":"2024-08-31T07:11:45.193074+0000","last_fullsized":"2024-08-31T07:11:45.193074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:33:50.192864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.041708+0000","last_change":"2024-08-31T07:10:46.041197+0000","last_active":"2024-08-31T07:10:46.041708+0000","last_peered":"2024-08-31T07:10:46.041708+0000","last_clean":"2024-08-31T07:10:46.041708+0000","last_became_active":"2024-08-31T07:10:46.023966+0000","last_became_peered":"2024-08-31T07:10:46.023966+0000","last_unstale":"2024-08-31T07:10:46.041708+0000","last_undegraded":"2024-08-31T07:10:46.041708+0000","last_fullsized":"2024-08-31T07:10:46.041708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:34.302665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192670+0000","last_change":"2024-08-31T07:09:47.660615+0000","last_active":"2024-08-31T07:11:45.192670+0000","last_peered":"2024-08-31T07:11:45.192670+0000","last_clean":"2024-08-31T07:11:45.192670+0000","last_became_active":"2024-08-31T07:09:47.660269+0000","last_became_peered":"2024-08-31T07:09:47.660269+0000","last_unstale":"2024-08-31T07:11:45.192670+0000","last_undegraded":"2024-08-31T07:11:45.192670+0000","last_fullsized":"2024-08-31T07:11:45.192670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:42:12.618039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020701+0000","last_change":"2024-08-31T07:09:45.641699+0000","last_active":"2024-08-31T07:10:46.020701+0000","last_peered":"2024-08-31T07:10:46.020701+0000","last_clean":"2024-08-31T07:10:46.020701+0000","last_became_active":"2024-08-31T07:09:45.641122+0000","last_became_peered":"2024-08-31T07:09:45.641122+0000","last_unstale":"2024-08-31T07:10:46.020701+0000","last_undegraded":"2024-08-31T07:10:46.020701+0000","last_fullsized":"2024-08-31T07:10:46.020701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:26:07.130266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200241+0000","last_change":"2024-08-31T07:09:47.675459+0000","last_active":"2024-08-31T07:11:45.200241+0000","last_peered":"2024-08-31T07:11:45.200241+0000","last_clean":"2024-08-31T07:11:45.200241+0000","last_became_active":"2024-08-31T07:09:47.672246+0000","last_became_peered":"2024-08-31T07:09:47.672246+0000","last_unstale":"2024-08-31T07:11:45.200241+0000","last_undegraded":"2024-08-31T07:11:45.200241+0000","last_fullsized":"2024-08-31T07:11:45.200241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:49:23.547352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016428+0000","last_change":"2024-08-31T07:09:45.649522+0000","last_active":"2024-08-31T07:10:46.016428+0000","last_peered":"2024-08-31T07:10:46.016428+0000","last_clean":"2024-08-31T07:10:46.016428+0000","last_became_active":"2024-08-31T07:09:45.649109+0000","last_became_peered":"2024-08-31T07:09:45.649109+0000","last_unstale":"2024-08-31T07:10:46.016428+0000","last_undegraded":"2024-08-31T07:10:46.016428+0000","last_fullsized":"2024-08-31T07:10:46.016428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:52:52.432605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.179375+0000","last_change":"2024-08-31T07:09:47.670583+0000","last_active":"2024-08-31T07:11:45.179375+0000","last_peered":"2024-08-31T07:11:45.179375+0000","last_clean":"2024-08-31T07:11:45.179375+0000","last_became_active":"2024-08-31T07:09:47.670162+0000","last_became_peered":"2024-08-31T07:09:47.670162+0000","last_unstale":"2024-08-31T07:11:45.179375+0000","last_undegraded":"2024-08-31T07:11:45.179375+0000","last_fullsized":"2024-08-31T07:11:45.179375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:35:35.135678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'14","reported_seq":184,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:24.383057+0000","last_change":"2024-08-31T07:09:45.639636+0000","last_active":"2024-08-31T07:12:24.383057+0000","last_peered":"2024-08-31T07:12:24.383057+0000","last_clean":"2024-08-31T07:12:24.383057+0000","last_became_active":"2024-08-31T07:09:45.639029+0000","last_became_peered":"2024-08-31T07:09:45.639029+0000","last_unstale":"2024-08-31T07:12:24.383057+0000","last_undegraded":"2024-08-31T07:12:24.383057+0000","last_fullsized":"2024-08-31T07:12:24.383057+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:54.110149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":863230,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1817,"num_read_kb":1127,"num_write":490,"num_write_kb":2247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":650,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823296,"data_stored":2589690,"data_compressed":111761,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":372,"ondisk_log_size":372,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285320,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":434494,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874291,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1089536,"data_stored":1486337,"data_compressed":60116,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037816,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":826458,"data_compressed":25040,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234047,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":826254,"data_compressed":25023,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397572,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1027343,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561096,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":433491,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":1419733,"data_compressed":60116,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":433352,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117081,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432784,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378719,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":501638,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542245,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":502092,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705769,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":434830,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":787422,"data_compressed":37001,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":397128,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394134,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":463732,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T07:12:30.764 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-31T07:12:30.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T07:12:30.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:30 smithi073 ceph-mon[21995]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-08-31T07:12:30.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:30 smithi084 ceph-mon[22387]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-08-31T07:12:31.141 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:31.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.828+0000 7fd141200640 1 -- 172.21.15.32:0/3114085041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13c103900 msgr2=0x7fd13c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:31.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.828+0000 7fd141200640 1 --2- 172.21.15.32:0/3114085041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13c103900 0x7fd13c103d10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009900 tx=0x7fd12c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:31.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.828+0000 7fd141200640 1 -- 172.21.15.32:0/3114085041 shutdown_connections 2024-08-31T07:12:31.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.828+0000 7fd141200640 1 --2- 172.21.15.32:0/3114085041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd13c109b90 0x7fd13c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:31.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.828+0000 7fd141200640 1 --2- 172.21.15.32:0/3114085041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd13c109240 0x7fd13c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:31.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.828+0000 7fd141200640 1 --2- 172.21.15.32:0/3114085041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13c103900 0x7fd13c103d10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:31.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.828+0000 7fd141200640 1 -- 172.21.15.32:0/3114085041 >> 172.21.15.32:0/3114085041 conn(0x7fd13c07b5f0 msgr2=0x7fd13c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:31.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.828+0000 7fd141200640 1 -- 172.21.15.32:0/3114085041 shutdown_connections 2024-08-31T07:12:31.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.828+0000 7fd141200640 1 -- 172.21.15.32:0/3114085041 wait complete. 2024-08-31T07:12:31.831 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:31 smithi032 ceph-mon[17252]: from='client.25454 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:31.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.830+0000 7fd141200640 1 Processor -- start 2024-08-31T07:12:31.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.830+0000 7fd141200640 1 -- start start 2024-08-31T07:12:31.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.830+0000 7fd141200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd13c103900 0x7fd13c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.830+0000 7fd141200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd13c109240 0x7fd13c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.830+0000 7fd141200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13c109b90 0x7fd13c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.830+0000 7fd141200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1138a0 con 0x7fd13c109b90 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.830+0000 7fd141200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c113720 con 0x7fd13c109240 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.830+0000 7fd141200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c113a20 con 0x7fd13c103900 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd139e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd13c103900 0x7fd13c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd139e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd13c103900 0x7fd13c0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36252/0 (socket says 172.21.15.32:36252) 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd13a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13c109b90 0x7fd13c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd139e00640 1 -- 172.21.15.32:0/3131677557 learned_addr learned my addr 172.21.15.32:0/3131677557 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd133e00640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd13c109240 0x7fd13c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd139e00640 1 -- 172.21.15.32:0/3131677557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd13c109240 msgr2=0x7fd13c0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd139e00640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd13c109240 0x7fd13c0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd139e00640 1 -- 172.21.15.32:0/3131677557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13c109b90 msgr2=0x7fd13c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd139e00640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13c109b90 0x7fd13c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd139e00640 1 -- 172.21.15.32:0/3131677557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c1a85d0 con 0x7fd13c103900 2024-08-31T07:12:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.831+0000 7fd139e00640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd13c103900 0x7fd13c0754b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005ab0 tx=0x7fd12c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.832+0000 7fd132a00640 1 -- 172.21.15.32:0/3131677557 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c03f070 con 0x7fd13c103900 2024-08-31T07:12:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.832+0000 7fd132a00640 1 -- 172.21.15.32:0/3131677557 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd12c031b60 con 0x7fd13c103900 2024-08-31T07:12:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.832+0000 7fd141200640 1 -- 172.21.15.32:0/3131677557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c1a8860 con 0x7fd13c103900 2024-08-31T07:12:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.832+0000 7fd132a00640 1 -- 172.21.15.32:0/3131677557 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c035030 con 0x7fd13c103900 2024-08-31T07:12:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.832+0000 7fd141200640 1 -- 172.21.15.32:0/3131677557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c1a8d90 con 0x7fd13c103900 2024-08-31T07:12:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.833+0000 7fd141200640 1 -- 172.21.15.32:0/3131677557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13c103d10 con 0x7fd13c103900 2024-08-31T07:12:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.835+0000 7fd132a00640 1 -- 172.21.15.32:0/3131677557 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7fd12c042070 con 0x7fd13c103900 2024-08-31T07:12:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.837+0000 7fd132a00640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1200b4c90 0x7fd1200b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.837+0000 7fd132a00640 1 -- 172.21.15.32:0/3131677557 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fd12c1062a0 con 0x7fd13c103900 2024-08-31T07:12:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.837+0000 7fd133e00640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1200b4c90 0x7fd1200b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.838+0000 7fd133e00640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1200b4c90 0x7fd1200b71a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd13c108f60 tx=0x7fd124005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:31.839+0000 7fd132a00640 1 -- 172.21.15.32:0/3131677557 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd12c0ccb20 con 0x7fd13c103900 2024-08-31T07:12:31.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:31 smithi073 ceph-mon[21995]: from='client.25454 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:31.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:31 smithi084 ceph-mon[22387]: from='client.25454 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.030+0000 7fd141200640 1 -- 172.21.15.32:0/3131677557 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd13c0635a0 con 0x7fd1200b4c90 2024-08-31T07:12:32.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.035+0000 7fd132a00640 1 -- 172.21.15.32:0/3131677557 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259286 (secure 0 0 0) 0x7fd13c0635a0 con 0x7fd1200b4c90 2024-08-31T07:12:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:32.041 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-08-31T07:12:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.040+0000 7fd141200640 1 -- 172.21.15.32:0/3131677557 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1200b4c90 msgr2=0x7fd1200b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.040+0000 7fd141200640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1200b4c90 0x7fd1200b71a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd13c108f60 tx=0x7fd124005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.041+0000 7fd141200640 1 -- 172.21.15.32:0/3131677557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd13c103900 msgr2=0x7fd13c0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.041+0000 7fd141200640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd13c103900 0x7fd13c0754b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005ab0 tx=0x7fd12c003c10 comp rx=0 tx=0).stop 2024-08-31T07:12:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.041+0000 7fd141200640 1 -- 172.21.15.32:0/3131677557 shutdown_connections 2024-08-31T07:12:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.041+0000 7fd141200640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1200b4c90 0x7fd1200b71a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.041+0000 7fd141200640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13c109b90 0x7fd13c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.041+0000 7fd141200640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd13c109240 0x7fd13c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.041+0000 7fd141200640 1 --2- 172.21.15.32:0/3131677557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd13c103900 0x7fd13c0754b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.041+0000 7fd141200640 1 -- 172.21.15.32:0/3131677557 >> 172.21.15.32:0/3131677557 conn(0x7fd13c07b5f0 msgr2=0x7fd13c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.041+0000 7fd141200640 1 -- 172.21.15.32:0/3131677557 shutdown_connections 2024-08-31T07:12:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:32.041+0000 7fd141200640 1 -- 172.21.15.32:0/3131677557 wait complete. 2024-08-31T07:12:32.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:32 smithi032 ceph-mon[17252]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 55 op/s 2024-08-31T07:12:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:32 smithi073 ceph-mon[21995]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 55 op/s 2024-08-31T07:12:32.932 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-08-31T07:12:30.801730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1453832,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1908,"num_read_kb":1201,"num_write":615,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2952490,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":966,"ondisk_log_size":966,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364472,"kb_used_data":11052,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577352,"statfs":{"total":1151940427776,"available":1151567208448,"internally_reserved":0,"allocated":11317248,"data_stored":9546936,"data_compressed":289974,"data_compressed_allocated":380928,"data_compressed_original":4251648,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":788130,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":447,"num_read_kb":358,"num_write":225,"num_write_kb":2090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003656"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198399+0000","last_change":"2024-08-31T07:09:47.665725+0000","last_active":"2024-08-31T07:11:45.198399+0000","last_peered":"2024-08-31T07:11:45.198399+0000","last_clean":"2024-08-31T07:11:45.198399+0000","last_became_active":"2024-08-31T07:09:47.665368+0000","last_became_peered":"2024-08-31T07:09:47.665368+0000","last_unstale":"2024-08-31T07:11:45.198399+0000","last_undegraded":"2024-08-31T07:11:45.198399+0000","last_fullsized":"2024-08-31T07:11:45.198399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:27:01.636099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192497+0000","last_change":"2024-08-31T07:09:47.676882+0000","last_active":"2024-08-31T07:11:45.192497+0000","last_peered":"2024-08-31T07:11:45.192497+0000","last_clean":"2024-08-31T07:11:45.192497+0000","last_became_active":"2024-08-31T07:09:47.674825+0000","last_became_peered":"2024-08-31T07:09:47.674825+0000","last_unstale":"2024-08-31T07:11:45.192497+0000","last_undegraded":"2024-08-31T07:11:45.192497+0000","last_fullsized":"2024-08-31T07:11:45.192497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:09.148398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199170+0000","last_change":"2024-08-31T07:09:47.674480+0000","last_active":"2024-08-31T07:11:45.199170+0000","last_peered":"2024-08-31T07:11:45.199170+0000","last_clean":"2024-08-31T07:11:45.199170+0000","last_became_active":"2024-08-31T07:09:47.672316+0000","last_became_peered":"2024-08-31T07:09:47.672316+0000","last_unstale":"2024-08-31T07:11:45.199170+0000","last_undegraded":"2024-08-31T07:11:45.199170+0000","last_fullsized":"2024-08-31T07:11:45.199170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:28:13.231039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.172744+0000","last_change":"2024-08-31T07:09:47.659276+0000","last_active":"2024-08-31T07:11:45.172744+0000","last_peered":"2024-08-31T07:11:45.172744+0000","last_clean":"2024-08-31T07:11:45.172744+0000","last_became_active":"2024-08-31T07:09:47.658924+0000","last_became_peered":"2024-08-31T07:09:47.658924+0000","last_unstale":"2024-08-31T07:11:45.172744+0000","last_undegraded":"2024-08-31T07:11:45.172744+0000","last_fullsized":"2024-08-31T07:11:45.172744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:01:08.134245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.174959+0000","last_change":"2024-08-31T07:09:47.667260+0000","last_active":"2024-08-31T07:11:45.174959+0000","last_peered":"2024-08-31T07:11:45.174959+0000","last_clean":"2024-08-31T07:11:45.174959+0000","last_became_active":"2024-08-31T07:09:47.664572+0000","last_became_peered":"2024-08-31T07:09:47.664572+0000","last_unstale":"2024-08-31T07:11:45.174959+0000","last_undegraded":"2024-08-31T07:11:45.174959+0000","last_fullsized":"2024-08-31T07:11:45.174959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:11.336985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193916+0000","last_change":"2024-08-31T07:09:47.661181+0000","last_active":"2024-08-31T07:11:45.193916+0000","last_peered":"2024-08-31T07:11:45.193916+0000","last_clean":"2024-08-31T07:11:45.193916+0000","last_became_active":"2024-08-31T07:09:47.660761+0000","last_became_peered":"2024-08-31T07:09:47.660761+0000","last_unstale":"2024-08-31T07:11:45.193916+0000","last_undegraded":"2024-08-31T07:11:45.193916+0000","last_fullsized":"2024-08-31T07:11:45.193916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:50:54.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196590+0000","last_change":"2024-08-31T07:09:47.668762+0000","last_active":"2024-08-31T07:11:45.196590+0000","last_peered":"2024-08-31T07:11:45.196590+0000","last_clean":"2024-08-31T07:11:45.196590+0000","last_became_active":"2024-08-31T07:09:47.668406+0000","last_became_peered":"2024-08-31T07:09:47.668406+0000","last_unstale":"2024-08-31T07:11:45.196590+0000","last_undegraded":"2024-08-31T07:11:45.196590+0000","last_fullsized":"2024-08-31T07:11:45.196590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:43:39.868144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.191067+0000","last_change":"2024-08-31T07:09:47.669455+0000","last_active":"2024-08-31T07:11:45.191067+0000","last_peered":"2024-08-31T07:11:45.191067+0000","last_clean":"2024-08-31T07:11:45.191067+0000","last_became_active":"2024-08-31T07:09:47.663669+0000","last_became_peered":"2024-08-31T07:09:47.663669+0000","last_unstale":"2024-08-31T07:11:45.191067+0000","last_undegraded":"2024-08-31T07:11:45.191067+0000","last_fullsized":"2024-08-31T07:11:45.191067+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:22:59.685012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194827+0000","last_change":"2024-08-31T07:09:47.664214+0000","last_active":"2024-08-31T07:11:45.194827+0000","last_peered":"2024-08-31T07:11:45.194827+0000","last_clean":"2024-08-31T07:11:45.194827+0000","last_became_active":"2024-08-31T07:09:47.663832+0000","last_became_peered":"2024-08-31T07:09:47.663832+0000","last_unstale":"2024-08-31T07:11:45.194827+0000","last_undegraded":"2024-08-31T07:11:45.194827+0000","last_fullsized":"2024-08-31T07:11:45.194827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:29:14.107212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194866+0000","last_change":"2024-08-31T07:10:46.326410+0000","last_active":"2024-08-31T07:11:45.194866+0000","last_peered":"2024-08-31T07:11:45.194866+0000","last_clean":"2024-08-31T07:11:45.194866+0000","last_became_active":"2024-08-31T07:10:46.325958+0000","last_became_peered":"2024-08-31T07:10:46.325958+0000","last_unstale":"2024-08-31T07:11:45.194866+0000","last_undegraded":"2024-08-31T07:11:45.194866+0000","last_fullsized":"2024-08-31T07:11:45.194866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:19.230393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196579+0000","last_change":"2024-08-31T07:09:47.666951+0000","last_active":"2024-08-31T07:11:45.196579+0000","last_peered":"2024-08-31T07:11:45.196579+0000","last_clean":"2024-08-31T07:11:45.196579+0000","last_became_active":"2024-08-31T07:09:47.666475+0000","last_became_peered":"2024-08-31T07:09:47.666475+0000","last_unstale":"2024-08-31T07:11:45.196579+0000","last_undegraded":"2024-08-31T07:11:45.196579+0000","last_fullsized":"2024-08-31T07:11:45.196579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:58:42.242086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.169998+0000","last_change":"2024-08-31T07:09:47.658344+0000","last_active":"2024-08-31T07:11:45.169998+0000","last_peered":"2024-08-31T07:11:45.169998+0000","last_clean":"2024-08-31T07:11:45.169998+0000","last_became_active":"2024-08-31T07:09:47.657991+0000","last_became_peered":"2024-08-31T07:09:47.657991+0000","last_unstale":"2024-08-31T07:11:45.169998+0000","last_undegraded":"2024-08-31T07:11:45.169998+0000","last_fullsized":"2024-08-31T07:11:45.169998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:25:59.302936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196676+0000","last_change":"2024-08-31T07:09:47.661644+0000","last_active":"2024-08-31T07:11:45.196676+0000","last_peered":"2024-08-31T07:11:45.196676+0000","last_clean":"2024-08-31T07:11:45.196676+0000","last_became_active":"2024-08-31T07:09:47.661248+0000","last_became_peered":"2024-08-31T07:09:47.661248+0000","last_unstale":"2024-08-31T07:11:45.196676+0000","last_undegraded":"2024-08-31T07:11:45.196676+0000","last_fullsized":"2024-08-31T07:11:45.196676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:42.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195194+0000","last_change":"2024-08-31T07:09:47.680052+0000","last_active":"2024-08-31T07:11:45.195194+0000","last_peered":"2024-08-31T07:11:45.195194+0000","last_clean":"2024-08-31T07:11:45.195194+0000","last_became_active":"2024-08-31T07:09:47.679604+0000","last_became_peered":"2024-08-31T07:09:47.679604+0000","last_unstale":"2024-08-31T07:11:45.195194+0000","last_undegraded":"2024-08-31T07:11:45.195194+0000","last_fullsized":"2024-08-31T07:11:45.195194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:23:18.937470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195477+0000","last_change":"2024-08-31T07:09:47.677984+0000","last_active":"2024-08-31T07:11:45.195477+0000","last_peered":"2024-08-31T07:11:45.195477+0000","last_clean":"2024-08-31T07:11:45.195477+0000","last_became_active":"2024-08-31T07:09:47.674957+0000","last_became_peered":"2024-08-31T07:09:47.674957+0000","last_unstale":"2024-08-31T07:11:45.195477+0000","last_undegraded":"2024-08-31T07:11:45.195477+0000","last_fullsized":"2024-08-31T07:11:45.195477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:08:26.503746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198878+0000","last_change":"2024-08-31T07:09:47.672625+0000","last_active":"2024-08-31T07:11:45.198878+0000","last_peered":"2024-08-31T07:11:45.198878+0000","last_clean":"2024-08-31T07:11:45.198878+0000","last_became_active":"2024-08-31T07:09:47.672184+0000","last_became_peered":"2024-08-31T07:09:47.672184+0000","last_unstale":"2024-08-31T07:11:45.198878+0000","last_undegraded":"2024-08-31T07:11:45.198878+0000","last_fullsized":"2024-08-31T07:11:45.198878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:17.091659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198670+0000","last_change":"2024-08-31T07:09:47.672597+0000","last_active":"2024-08-31T07:11:45.198670+0000","last_peered":"2024-08-31T07:11:45.198670+0000","last_clean":"2024-08-31T07:11:45.198670+0000","last_became_active":"2024-08-31T07:09:47.672153+0000","last_became_peered":"2024-08-31T07:09:47.672153+0000","last_unstale":"2024-08-31T07:11:45.198670+0000","last_undegraded":"2024-08-31T07:11:45.198670+0000","last_fullsized":"2024-08-31T07:11:45.198670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:26:41.388526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193223+0000","last_change":"2024-08-31T07:09:47.666253+0000","last_active":"2024-08-31T07:11:45.193223+0000","last_peered":"2024-08-31T07:11:45.193223+0000","last_clean":"2024-08-31T07:11:45.193223+0000","last_became_active":"2024-08-31T07:09:47.663732+0000","last_became_peered":"2024-08-31T07:09:47.663732+0000","last_unstale":"2024-08-31T07:11:45.193223+0000","last_undegraded":"2024-08-31T07:11:45.193223+0000","last_fullsized":"2024-08-31T07:11:45.193223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:10:59.887765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188202+0000","last_change":"2024-08-31T07:09:47.668502+0000","last_active":"2024-08-31T07:11:45.188202+0000","last_peered":"2024-08-31T07:11:45.188202+0000","last_clean":"2024-08-31T07:11:45.188202+0000","last_became_active":"2024-08-31T07:09:47.666487+0000","last_became_peered":"2024-08-31T07:09:47.666487+0000","last_unstale":"2024-08-31T07:11:45.188202+0000","last_undegraded":"2024-08-31T07:11:45.188202+0000","last_fullsized":"2024-08-31T07:11:45.188202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:49:25.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.184945+0000","last_change":"2024-08-31T07:09:47.659454+0000","last_active":"2024-08-31T07:11:45.184945+0000","last_peered":"2024-08-31T07:11:45.184945+0000","last_clean":"2024-08-31T07:11:45.184945+0000","last_became_active":"2024-08-31T07:09:47.658976+0000","last_became_peered":"2024-08-31T07:09:47.658976+0000","last_unstale":"2024-08-31T07:11:45.184945+0000","last_undegraded":"2024-08-31T07:11:45.184945+0000","last_fullsized":"2024-08-31T07:11:45.184945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:31.374905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199707+0000","last_change":"2024-08-31T07:09:47.673612+0000","last_active":"2024-08-31T07:11:45.199707+0000","last_peered":"2024-08-31T07:11:45.199707+0000","last_clean":"2024-08-31T07:11:45.199707+0000","last_became_active":"2024-08-31T07:09:47.673137+0000","last_became_peered":"2024-08-31T07:09:47.673137+0000","last_unstale":"2024-08-31T07:11:45.199707+0000","last_undegraded":"2024-08-31T07:11:45.199707+0000","last_fullsized":"2024-08-31T07:11:45.199707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:46:11.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194525+0000","last_change":"2024-08-31T07:09:47.669195+0000","last_active":"2024-08-31T07:11:45.194525+0000","last_peered":"2024-08-31T07:11:45.194525+0000","last_clean":"2024-08-31T07:11:45.194525+0000","last_became_active":"2024-08-31T07:09:47.668790+0000","last_became_peered":"2024-08-31T07:09:47.668790+0000","last_unstale":"2024-08-31T07:11:45.194525+0000","last_undegraded":"2024-08-31T07:11:45.194525+0000","last_fullsized":"2024-08-31T07:11:45.194525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:27:14.393931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.181333+0000","last_change":"2024-08-31T07:09:47.667566+0000","last_active":"2024-08-31T07:11:45.181333+0000","last_peered":"2024-08-31T07:11:45.181333+0000","last_clean":"2024-08-31T07:11:45.181333+0000","last_became_active":"2024-08-31T07:09:47.666605+0000","last_became_peered":"2024-08-31T07:09:47.666605+0000","last_unstale":"2024-08-31T07:11:45.181333+0000","last_undegraded":"2024-08-31T07:11:45.181333+0000","last_fullsized":"2024-08-31T07:11:45.181333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:05:44.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192240+0000","last_change":"2024-08-31T07:09:47.660478+0000","last_active":"2024-08-31T07:11:45.192240+0000","last_peered":"2024-08-31T07:11:45.192240+0000","last_clean":"2024-08-31T07:11:45.192240+0000","last_became_active":"2024-08-31T07:09:47.660107+0000","last_became_peered":"2024-08-31T07:09:47.660107+0000","last_unstale":"2024-08-31T07:11:45.192240+0000","last_undegraded":"2024-08-31T07:11:45.192240+0000","last_fullsized":"2024-08-31T07:11:45.192240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:29.133370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.188131+0000","last_change":"2024-08-31T07:09:47.671967+0000","last_active":"2024-08-31T07:11:45.188131+0000","last_peered":"2024-08-31T07:11:45.188131+0000","last_clean":"2024-08-31T07:11:45.188131+0000","last_became_active":"2024-08-31T07:09:47.671569+0000","last_became_peered":"2024-08-31T07:09:47.671569+0000","last_unstale":"2024-08-31T07:11:45.188131+0000","last_undegraded":"2024-08-31T07:11:45.188131+0000","last_fullsized":"2024-08-31T07:11:45.188131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:19:15.107958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185867+0000","last_change":"2024-08-31T07:09:47.651055+0000","last_active":"2024-08-31T07:11:45.185867+0000","last_peered":"2024-08-31T07:11:45.185867+0000","last_clean":"2024-08-31T07:11:45.185867+0000","last_became_active":"2024-08-31T07:09:47.650674+0000","last_became_peered":"2024-08-31T07:09:47.650674+0000","last_unstale":"2024-08-31T07:11:45.185867+0000","last_undegraded":"2024-08-31T07:11:45.185867+0000","last_fullsized":"2024-08-31T07:11:45.185867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:35:54.647379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180140+0000","last_change":"2024-08-31T07:09:47.663231+0000","last_active":"2024-08-31T07:11:45.180140+0000","last_peered":"2024-08-31T07:11:45.180140+0000","last_clean":"2024-08-31T07:11:45.180140+0000","last_became_active":"2024-08-31T07:09:47.662854+0000","last_became_peered":"2024-08-31T07:09:47.662854+0000","last_unstale":"2024-08-31T07:11:45.180140+0000","last_undegraded":"2024-08-31T07:11:45.180140+0000","last_fullsized":"2024-08-31T07:11:45.180140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:51:51.358557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199287+0000","last_change":"2024-08-31T07:10:46.326380+0000","last_active":"2024-08-31T07:11:45.199287+0000","last_peered":"2024-08-31T07:11:45.199287+0000","last_clean":"2024-08-31T07:11:45.199287+0000","last_became_active":"2024-08-31T07:10:46.325784+0000","last_became_peered":"2024-08-31T07:10:46.325784+0000","last_unstale":"2024-08-31T07:11:45.199287+0000","last_undegraded":"2024-08-31T07:11:45.199287+0000","last_fullsized":"2024-08-31T07:11:45.199287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:16:56.396024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200247+0000","last_change":"2024-08-31T07:09:47.672108+0000","last_active":"2024-08-31T07:11:45.200247+0000","last_peered":"2024-08-31T07:11:45.200247+0000","last_clean":"2024-08-31T07:11:45.200247+0000","last_became_active":"2024-08-31T07:09:47.670998+0000","last_became_peered":"2024-08-31T07:09:47.670998+0000","last_unstale":"2024-08-31T07:11:45.200247+0000","last_undegraded":"2024-08-31T07:11:45.200247+0000","last_fullsized":"2024-08-31T07:11:45.200247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:26:49.319861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198226+0000","last_change":"2024-08-31T07:09:47.665548+0000","last_active":"2024-08-31T07:11:45.198226+0000","last_peered":"2024-08-31T07:11:45.198226+0000","last_clean":"2024-08-31T07:11:45.198226+0000","last_became_active":"2024-08-31T07:09:47.665177+0000","last_became_peered":"2024-08-31T07:09:47.665177+0000","last_unstale":"2024-08-31T07:11:45.198226+0000","last_undegraded":"2024-08-31T07:11:45.198226+0000","last_fullsized":"2024-08-31T07:11:45.198226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:03:15.892224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.177785+0000","last_change":"2024-08-31T07:10:46.026746+0000","last_active":"2024-08-31T07:11:45.177785+0000","last_peered":"2024-08-31T07:11:45.177785+0000","last_clean":"2024-08-31T07:11:45.177785+0000","last_became_active":"2024-08-31T07:10:46.026286+0000","last_became_peered":"2024-08-31T07:10:46.026286+0000","last_unstale":"2024-08-31T07:11:45.177785+0000","last_undegraded":"2024-08-31T07:11:45.177785+0000","last_fullsized":"2024-08-31T07:11:45.177785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:34:17.120906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.180769+0000","last_change":"2024-08-31T07:09:47.667456+0000","last_active":"2024-08-31T07:11:45.180769+0000","last_peered":"2024-08-31T07:11:45.180769+0000","last_clean":"2024-08-31T07:11:45.180769+0000","last_became_active":"2024-08-31T07:09:47.667102+0000","last_became_peered":"2024-08-31T07:09:47.667102+0000","last_unstale":"2024-08-31T07:11:45.180769+0000","last_undegraded":"2024-08-31T07:11:45.180769+0000","last_fullsized":"2024-08-31T07:11:45.180769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:44:39.747729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.182194+0000","last_change":"2024-08-31T07:10:46.326139+0000","last_active":"2024-08-31T07:11:45.182194+0000","last_peered":"2024-08-31T07:11:45.182194+0000","last_clean":"2024-08-31T07:11:45.182194+0000","last_became_active":"2024-08-31T07:10:46.325785+0000","last_became_peered":"2024-08-31T07:10:46.325785+0000","last_unstale":"2024-08-31T07:11:45.182194+0000","last_undegraded":"2024-08-31T07:11:45.182194+0000","last_fullsized":"2024-08-31T07:11:45.182194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:43:13.495835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":180,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:25.108048+0000","last_change":"2024-08-31T07:09:45.641088+0000","last_active":"2024-08-31T07:12:25.108048+0000","last_peered":"2024-08-31T07:12:25.108048+0000","last_clean":"2024-08-31T07:12:25.108048+0000","last_became_active":"2024-08-31T07:09:45.640673+0000","last_became_peered":"2024-08-31T07:09:45.640673+0000","last_unstale":"2024-08-31T07:12:25.108048+0000","last_undegraded":"2024-08-31T07:12:25.108048+0000","last_fullsized":"2024-08-31T07:12:25.108048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:30:17.593494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192346+0000","last_change":"2024-08-31T07:09:47.666703+0000","last_active":"2024-08-31T07:11:45.192346+0000","last_peered":"2024-08-31T07:11:45.192346+0000","last_clean":"2024-08-31T07:11:45.192346+0000","last_became_active":"2024-08-31T07:09:47.666348+0000","last_became_peered":"2024-08-31T07:09:47.666348+0000","last_unstale":"2024-08-31T07:11:45.192346+0000","last_undegraded":"2024-08-31T07:11:45.192346+0000","last_fullsized":"2024-08-31T07:11:45.192346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:39.688810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.019885+0000","last_change":"2024-08-31T07:09:45.642431+0000","last_active":"2024-08-31T07:10:46.019885+0000","last_peered":"2024-08-31T07:10:46.019885+0000","last_clean":"2024-08-31T07:10:46.019885+0000","last_became_active":"2024-08-31T07:09:45.642048+0000","last_became_peered":"2024-08-31T07:09:45.642048+0000","last_unstale":"2024-08-31T07:10:46.019885+0000","last_undegraded":"2024-08-31T07:10:46.019885+0000","last_fullsized":"2024-08-31T07:10:46.019885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:47:42.574442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017286+0000","last_change":"2024-08-31T07:09:45.632162+0000","last_active":"2024-08-31T07:10:46.017286+0000","last_peered":"2024-08-31T07:10:46.017286+0000","last_clean":"2024-08-31T07:10:46.017286+0000","last_became_active":"2024-08-31T07:09:45.631749+0000","last_became_peered":"2024-08-31T07:09:45.631749+0000","last_unstale":"2024-08-31T07:10:46.017286+0000","last_undegraded":"2024-08-31T07:10:46.017286+0000","last_fullsized":"2024-08-31T07:10:46.017286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:16:29.967823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185787+0000","last_change":"2024-08-31T07:09:47.666288+0000","last_active":"2024-08-31T07:11:45.185787+0000","last_peered":"2024-08-31T07:11:45.185787+0000","last_clean":"2024-08-31T07:11:45.185787+0000","last_became_active":"2024-08-31T07:09:47.665723+0000","last_became_peered":"2024-08-31T07:09:47.665723+0000","last_unstale":"2024-08-31T07:11:45.185787+0000","last_undegraded":"2024-08-31T07:11:45.185787+0000","last_fullsized":"2024-08-31T07:11:45.185787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:38:28.122720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.023120+0000","last_change":"2024-08-31T07:09:45.636508+0000","last_active":"2024-08-31T07:10:46.023120+0000","last_peered":"2024-08-31T07:10:46.023120+0000","last_clean":"2024-08-31T07:10:46.023120+0000","last_became_active":"2024-08-31T07:09:45.636073+0000","last_became_peered":"2024-08-31T07:09:45.636073+0000","last_unstale":"2024-08-31T07:10:46.023120+0000","last_undegraded":"2024-08-31T07:10:46.023120+0000","last_fullsized":"2024-08-31T07:10:46.023120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:55:45.063517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187943+0000","last_change":"2024-08-31T07:09:47.661306+0000","last_active":"2024-08-31T07:11:45.187943+0000","last_peered":"2024-08-31T07:11:45.187943+0000","last_clean":"2024-08-31T07:11:45.187943+0000","last_became_active":"2024-08-31T07:09:47.660957+0000","last_became_peered":"2024-08-31T07:09:47.660957+0000","last_unstale":"2024-08-31T07:11:45.187943+0000","last_undegraded":"2024-08-31T07:11:45.187943+0000","last_fullsized":"2024-08-31T07:11:45.187943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:18:23.740355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'6","reported_seq":318,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:26.108577+0000","last_change":"2024-08-31T07:09:45.640947+0000","last_active":"2024-08-31T07:12:26.108577+0000","last_peered":"2024-08-31T07:12:26.108577+0000","last_clean":"2024-08-31T07:12:26.108577+0000","last_became_active":"2024-08-31T07:09:45.640532+0000","last_became_peered":"2024-08-31T07:09:45.640532+0000","last_unstale":"2024-08-31T07:12:26.108577+0000","last_undegraded":"2024-08-31T07:12:26.108577+0000","last_fullsized":"2024-08-31T07:12:26.108577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:28.177795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":286,"num_read_kb":286,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.191708+0000","last_change":"2024-08-31T07:09:47.661060+0000","last_active":"2024-08-31T07:11:45.191708+0000","last_peered":"2024-08-31T07:11:45.191708+0000","last_clean":"2024-08-31T07:11:45.191708+0000","last_became_active":"2024-08-31T07:09:47.660692+0000","last_became_peered":"2024-08-31T07:09:47.660692+0000","last_unstale":"2024-08-31T07:11:45.191708+0000","last_undegraded":"2024-08-31T07:11:45.191708+0000","last_fullsized":"2024-08-31T07:11:45.191708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:48:38.242709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257194+0000","last_change":"2024-08-31T07:09:45.641155+0000","last_active":"2024-08-31T07:10:46.257194+0000","last_peered":"2024-08-31T07:10:46.257194+0000","last_clean":"2024-08-31T07:10:46.257194+0000","last_became_active":"2024-08-31T07:09:45.640680+0000","last_became_peered":"2024-08-31T07:09:45.640680+0000","last_unstale":"2024-08-31T07:10:46.257194+0000","last_undegraded":"2024-08-31T07:10:46.257194+0000","last_fullsized":"2024-08-31T07:10:46.257194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:29:03.287570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.199892+0000","last_change":"2024-08-31T07:09:47.675582+0000","last_active":"2024-08-31T07:11:45.199892+0000","last_peered":"2024-08-31T07:11:45.199892+0000","last_clean":"2024-08-31T07:11:45.199892+0000","last_became_active":"2024-08-31T07:09:47.675266+0000","last_became_peered":"2024-08-31T07:09:47.675266+0000","last_unstale":"2024-08-31T07:11:45.199892+0000","last_undegraded":"2024-08-31T07:11:45.199892+0000","last_fullsized":"2024-08-31T07:11:45.199892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:36:33.122531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021194+0000","last_change":"2024-08-31T07:09:45.651006+0000","last_active":"2024-08-31T07:10:46.021194+0000","last_peered":"2024-08-31T07:10:46.021194+0000","last_clean":"2024-08-31T07:10:46.021194+0000","last_became_active":"2024-08-31T07:09:45.650588+0000","last_became_peered":"2024-08-31T07:09:45.650588+0000","last_unstale":"2024-08-31T07:10:46.021194+0000","last_undegraded":"2024-08-31T07:10:46.021194+0000","last_fullsized":"2024-08-31T07:10:46.021194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:34:16.015892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.190030+0000","last_change":"2024-08-31T07:09:47.662289+0000","last_active":"2024-08-31T07:11:45.190030+0000","last_peered":"2024-08-31T07:11:45.190030+0000","last_clean":"2024-08-31T07:11:45.190030+0000","last_became_active":"2024-08-31T07:09:47.661932+0000","last_became_peered":"2024-08-31T07:09:47.661932+0000","last_unstale":"2024-08-31T07:11:45.190030+0000","last_undegraded":"2024-08-31T07:11:45.190030+0000","last_fullsized":"2024-08-31T07:11:45.190030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:50.440504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'38","reported_seq":364,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:26.109365+0000","last_change":"2024-08-31T07:09:45.642196+0000","last_active":"2024-08-31T07:12:26.109365+0000","last_peered":"2024-08-31T07:12:26.109365+0000","last_clean":"2024-08-31T07:12:26.109365+0000","last_became_active":"2024-08-31T07:09:45.641797+0000","last_became_peered":"2024-08-31T07:09:45.641797+0000","last_unstale":"2024-08-31T07:12:26.109365+0000","last_undegraded":"2024-08-31T07:12:26.109365+0000","last_fullsized":"2024-08-31T07:12:26.109365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:51:05.953335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67688,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":300,"num_read_kb":217,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196022+0000","last_change":"2024-08-31T07:09:47.660184+0000","last_active":"2024-08-31T07:11:45.196022+0000","last_peered":"2024-08-31T07:11:45.196022+0000","last_clean":"2024-08-31T07:11:45.196022+0000","last_became_active":"2024-08-31T07:09:47.659571+0000","last_became_peered":"2024-08-31T07:09:47.659571+0000","last_unstale":"2024-08-31T07:11:45.196022+0000","last_undegraded":"2024-08-31T07:11:45.196022+0000","last_fullsized":"2024-08-31T07:11:45.196022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:41:13.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.035782+0000","last_change":"2024-08-31T07:10:46.035336+0000","last_active":"2024-08-31T07:10:46.035782+0000","last_peered":"2024-08-31T07:10:46.035782+0000","last_clean":"2024-08-31T07:10:46.035782+0000","last_became_active":"2024-08-31T07:10:46.021299+0000","last_became_peered":"2024-08-31T07:10:46.021299+0000","last_unstale":"2024-08-31T07:10:46.035782+0000","last_undegraded":"2024-08-31T07:10:46.035782+0000","last_fullsized":"2024-08-31T07:10:46.035782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:06:15.931855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194680+0000","last_change":"2024-08-31T07:10:46.326354+0000","last_active":"2024-08-31T07:11:45.194680+0000","last_peered":"2024-08-31T07:11:45.194680+0000","last_clean":"2024-08-31T07:11:45.194680+0000","last_became_active":"2024-08-31T07:10:46.325869+0000","last_became_peered":"2024-08-31T07:10:46.325869+0000","last_unstale":"2024-08-31T07:11:45.194680+0000","last_undegraded":"2024-08-31T07:11:45.194680+0000","last_fullsized":"2024-08-31T07:11:45.194680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:31:21.832088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022972+0000","last_change":"2024-08-31T07:09:45.642187+0000","last_active":"2024-08-31T07:10:46.022972+0000","last_peered":"2024-08-31T07:10:46.022972+0000","last_clean":"2024-08-31T07:10:46.022972+0000","last_became_active":"2024-08-31T07:09:45.641721+0000","last_became_peered":"2024-08-31T07:09:45.641721+0000","last_unstale":"2024-08-31T07:10:46.022972+0000","last_undegraded":"2024-08-31T07:10:46.022972+0000","last_fullsized":"2024-08-31T07:10:46.022972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:00:45.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196019+0000","last_change":"2024-08-31T07:09:47.665956+0000","last_active":"2024-08-31T07:11:45.196019+0000","last_peered":"2024-08-31T07:11:45.196019+0000","last_clean":"2024-08-31T07:11:45.196019+0000","last_became_active":"2024-08-31T07:09:47.664367+0000","last_became_peered":"2024-08-31T07:09:47.664367+0000","last_unstale":"2024-08-31T07:11:45.196019+0000","last_undegraded":"2024-08-31T07:11:45.196019+0000","last_fullsized":"2024-08-31T07:11:45.196019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:34.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'7","reported_seq":180,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:25.107094+0000","last_change":"2024-08-31T07:09:45.636440+0000","last_active":"2024-08-31T07:12:25.107094+0000","last_peered":"2024-08-31T07:12:25.107094+0000","last_clean":"2024-08-31T07:12:25.107094+0000","last_became_active":"2024-08-31T07:09:45.636058+0000","last_became_peered":"2024-08-31T07:09:45.636058+0000","last_unstale":"2024-08-31T07:12:25.107094+0000","last_undegraded":"2024-08-31T07:12:25.107094+0000","last_fullsized":"2024-08-31T07:12:25.107094+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:38.929227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194840+0000","last_change":"2024-08-31T07:09:47.660564+0000","last_active":"2024-08-31T07:11:45.194840+0000","last_peered":"2024-08-31T07:11:45.194840+0000","last_clean":"2024-08-31T07:11:45.194840+0000","last_became_active":"2024-08-31T07:09:47.660157+0000","last_became_peered":"2024-08-31T07:09:47.660157+0000","last_unstale":"2024-08-31T07:11:45.194840+0000","last_undegraded":"2024-08-31T07:11:45.194840+0000","last_fullsized":"2024-08-31T07:11:45.194840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:49:01.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020515+0000","last_change":"2024-08-31T07:09:45.649140+0000","last_active":"2024-08-31T07:10:46.020515+0000","last_peered":"2024-08-31T07:10:46.020515+0000","last_clean":"2024-08-31T07:10:46.020515+0000","last_became_active":"2024-08-31T07:09:45.648796+0000","last_became_peered":"2024-08-31T07:09:45.648796+0000","last_unstale":"2024-08-31T07:10:46.020515+0000","last_undegraded":"2024-08-31T07:10:46.020515+0000","last_fullsized":"2024-08-31T07:10:46.020515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:33:37.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192390+0000","last_change":"2024-08-31T07:09:47.663833+0000","last_active":"2024-08-31T07:11:45.192390+0000","last_peered":"2024-08-31T07:11:45.192390+0000","last_clean":"2024-08-31T07:11:45.192390+0000","last_became_active":"2024-08-31T07:09:47.663451+0000","last_became_peered":"2024-08-31T07:09:47.663451+0000","last_unstale":"2024-08-31T07:11:45.192390+0000","last_undegraded":"2024-08-31T07:11:45.192390+0000","last_fullsized":"2024-08-31T07:11:45.192390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:27:22.245254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021407+0000","last_change":"2024-08-31T07:09:45.636593+0000","last_active":"2024-08-31T07:10:46.021407+0000","last_peered":"2024-08-31T07:10:46.021407+0000","last_clean":"2024-08-31T07:10:46.021407+0000","last_became_active":"2024-08-31T07:09:45.636166+0000","last_became_peered":"2024-08-31T07:09:45.636166+0000","last_unstale":"2024-08-31T07:10:46.021407+0000","last_undegraded":"2024-08-31T07:10:46.021407+0000","last_fullsized":"2024-08-31T07:10:46.021407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:28.422488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183385+0000","last_change":"2024-08-31T07:09:47.660825+0000","last_active":"2024-08-31T07:11:45.183385+0000","last_peered":"2024-08-31T07:11:45.183385+0000","last_clean":"2024-08-31T07:11:45.183385+0000","last_became_active":"2024-08-31T07:09:47.660474+0000","last_became_peered":"2024-08-31T07:09:47.660474+0000","last_unstale":"2024-08-31T07:11:45.183385+0000","last_undegraded":"2024-08-31T07:11:45.183385+0000","last_fullsized":"2024-08-31T07:11:45.183385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:42:20.822143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.017462+0000","last_change":"2024-08-31T07:09:45.632727+0000","last_active":"2024-08-31T07:10:46.017462+0000","last_peered":"2024-08-31T07:10:46.017462+0000","last_clean":"2024-08-31T07:10:46.017462+0000","last_became_active":"2024-08-31T07:09:45.632251+0000","last_became_peered":"2024-08-31T07:09:45.632251+0000","last_unstale":"2024-08-31T07:10:46.017462+0000","last_undegraded":"2024-08-31T07:10:46.017462+0000","last_fullsized":"2024-08-31T07:10:46.017462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:09:30.197409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196430+0000","last_change":"2024-08-31T07:09:47.665501+0000","last_active":"2024-08-31T07:11:45.196430+0000","last_peered":"2024-08-31T07:11:45.196430+0000","last_clean":"2024-08-31T07:11:45.196430+0000","last_became_active":"2024-08-31T07:09:47.665162+0000","last_became_peered":"2024-08-31T07:09:47.665162+0000","last_unstale":"2024-08-31T07:11:45.196430+0000","last_undegraded":"2024-08-31T07:11:45.196430+0000","last_fullsized":"2024-08-31T07:11:45.196430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:34:29.605836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'62","reported_seq":118,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:24.990505+0000","last_change":"2024-08-31T07:09:45.636580+0000","last_active":"2024-08-31T07:12:24.990505+0000","last_peered":"2024-08-31T07:12:24.990505+0000","last_clean":"2024-08-31T07:12:24.990505+0000","last_became_active":"2024-08-31T07:09:45.636163+0000","last_became_peered":"2024-08-31T07:09:45.636163+0000","last_unstale":"2024-08-31T07:12:24.990505+0000","last_undegraded":"2024-08-31T07:12:24.990505+0000","last_fullsized":"2024-08-31T07:12:24.990505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:18:07.567858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195120+0000","last_change":"2024-08-31T07:09:47.671865+0000","last_active":"2024-08-31T07:11:45.195120+0000","last_peered":"2024-08-31T07:11:45.195120+0000","last_clean":"2024-08-31T07:11:45.195120+0000","last_became_active":"2024-08-31T07:09:47.671454+0000","last_became_peered":"2024-08-31T07:09:47.671454+0000","last_unstale":"2024-08-31T07:11:45.195120+0000","last_undegraded":"2024-08-31T07:11:45.195120+0000","last_fullsized":"2024-08-31T07:11:45.195120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:47.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'7","reported_seq":183,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:28.112259+0000","last_change":"2024-08-31T07:09:45.653217+0000","last_active":"2024-08-31T07:12:28.112259+0000","last_peered":"2024-08-31T07:12:28.112259+0000","last_clean":"2024-08-31T07:12:28.112259+0000","last_became_active":"2024-08-31T07:09:45.652819+0000","last_became_peered":"2024-08-31T07:09:45.652819+0000","last_unstale":"2024-08-31T07:12:28.112259+0000","last_undegraded":"2024-08-31T07:12:28.112259+0000","last_fullsized":"2024-08-31T07:12:28.112259+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:25:11.454068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.196166+0000","last_change":"2024-08-31T07:09:47.663652+0000","last_active":"2024-08-31T07:11:45.196166+0000","last_peered":"2024-08-31T07:11:45.196166+0000","last_clean":"2024-08-31T07:11:45.196166+0000","last_became_active":"2024-08-31T07:09:47.663274+0000","last_became_peered":"2024-08-31T07:09:47.663274+0000","last_unstale":"2024-08-31T07:11:45.196166+0000","last_undegraded":"2024-08-31T07:11:45.196166+0000","last_fullsized":"2024-08-31T07:11:45.196166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:31:59.690365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257370+0000","last_change":"2024-08-31T07:08:27.623298+0000","last_active":"2024-08-31T07:10:46.257370+0000","last_peered":"2024-08-31T07:10:46.257370+0000","last_clean":"2024-08-31T07:10:46.257370+0000","last_became_active":"2024-08-31T07:08:27.619072+0000","last_became_peered":"2024-08-31T07:08:27.619072+0000","last_unstale":"2024-08-31T07:10:46.257370+0000","last_undegraded":"2024-08-31T07:10:46.257370+0000","last_fullsized":"2024-08-31T07:10:46.257370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:04:45.509398+0000","last_clean_scrub_stamp":"2024-08-31T07:04:45.509398+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:07.967421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154319+0000","last_change":"2024-08-31T07:09:45.638104+0000","last_active":"2024-08-31T07:10:46.154319+0000","last_peered":"2024-08-31T07:10:46.154319+0000","last_clean":"2024-08-31T07:10:46.154319+0000","last_became_active":"2024-08-31T07:09:45.637693+0000","last_became_peered":"2024-08-31T07:09:45.637693+0000","last_unstale":"2024-08-31T07:10:46.154319+0000","last_undegraded":"2024-08-31T07:10:46.154319+0000","last_fullsized":"2024-08-31T07:10:46.154319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:50:15.135061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.189353+0000","last_change":"2024-08-31T07:09:47.664958+0000","last_active":"2024-08-31T07:11:45.189353+0000","last_peered":"2024-08-31T07:11:45.189353+0000","last_clean":"2024-08-31T07:11:45.189353+0000","last_became_active":"2024-08-31T07:09:47.661166+0000","last_became_peered":"2024-08-31T07:09:47.661166+0000","last_unstale":"2024-08-31T07:11:45.189353+0000","last_undegraded":"2024-08-31T07:11:45.189353+0000","last_fullsized":"2024-08-31T07:11:45.189353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:45:58.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.257184+0000","last_change":"2024-08-31T07:09:45.640881+0000","last_active":"2024-08-31T07:10:46.257184+0000","last_peered":"2024-08-31T07:10:46.257184+0000","last_clean":"2024-08-31T07:10:46.257184+0000","last_became_active":"2024-08-31T07:09:45.640487+0000","last_became_peered":"2024-08-31T07:09:45.640487+0000","last_unstale":"2024-08-31T07:10:46.257184+0000","last_undegraded":"2024-08-31T07:10:46.257184+0000","last_fullsized":"2024-08-31T07:10:46.257184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:06.830407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.171981+0000","last_change":"2024-08-31T07:09:47.655008+0000","last_active":"2024-08-31T07:11:45.171981+0000","last_peered":"2024-08-31T07:11:45.171981+0000","last_clean":"2024-08-31T07:11:45.171981+0000","last_became_active":"2024-08-31T07:09:47.654645+0000","last_became_peered":"2024-08-31T07:09:47.654645+0000","last_unstale":"2024-08-31T07:11:45.171981+0000","last_undegraded":"2024-08-31T07:11:45.171981+0000","last_fullsized":"2024-08-31T07:11:45.171981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:23:55.916053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'6","reported_seq":178,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:27.109208+0000","last_change":"2024-08-31T07:09:45.649324+0000","last_active":"2024-08-31T07:12:27.109208+0000","last_peered":"2024-08-31T07:12:27.109208+0000","last_clean":"2024-08-31T07:12:27.109208+0000","last_became_active":"2024-08-31T07:09:45.648915+0000","last_became_peered":"2024-08-31T07:09:45.648915+0000","last_unstale":"2024-08-31T07:12:27.109208+0000","last_undegraded":"2024-08-31T07:12:27.109208+0000","last_fullsized":"2024-08-31T07:12:27.109208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:31:57.853654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.187160+0000","last_change":"2024-08-31T07:09:47.661328+0000","last_active":"2024-08-31T07:11:45.187160+0000","last_peered":"2024-08-31T07:11:45.187160+0000","last_clean":"2024-08-31T07:11:45.187160+0000","last_became_active":"2024-08-31T07:09:47.660927+0000","last_became_peered":"2024-08-31T07:09:47.660927+0000","last_unstale":"2024-08-31T07:11:45.187160+0000","last_undegraded":"2024-08-31T07:11:45.187160+0000","last_fullsized":"2024-08-31T07:11:45.187160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:40:46.876894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'7","reported_seq":181,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:27.111174+0000","last_change":"2024-08-31T07:09:45.637747+0000","last_active":"2024-08-31T07:12:27.111174+0000","last_peered":"2024-08-31T07:12:27.111174+0000","last_clean":"2024-08-31T07:12:27.111174+0000","last_became_active":"2024-08-31T07:09:45.637368+0000","last_became_peered":"2024-08-31T07:09:45.637368+0000","last_unstale":"2024-08-31T07:12:27.111174+0000","last_undegraded":"2024-08-31T07:12:27.111174+0000","last_fullsized":"2024-08-31T07:12:27.111174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:01.174109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":148,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193217+0000","last_change":"2024-08-31T07:09:47.680802+0000","last_active":"2024-08-31T07:11:45.193217+0000","last_peered":"2024-08-31T07:11:45.193217+0000","last_clean":"2024-08-31T07:11:45.193217+0000","last_became_active":"2024-08-31T07:09:47.680363+0000","last_became_peered":"2024-08-31T07:09:47.680363+0000","last_unstale":"2024-08-31T07:11:45.193217+0000","last_undegraded":"2024-08-31T07:11:45.193217+0000","last_fullsized":"2024-08-31T07:11:45.193217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:46:48.744071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.324353+0000","last_change":"2024-08-31T07:09:45.651222+0000","last_active":"2024-08-31T07:10:46.324353+0000","last_peered":"2024-08-31T07:10:46.324353+0000","last_clean":"2024-08-31T07:10:46.324353+0000","last_became_active":"2024-08-31T07:09:45.650760+0000","last_became_peered":"2024-08-31T07:09:45.650760+0000","last_unstale":"2024-08-31T07:10:46.324353+0000","last_undegraded":"2024-08-31T07:10:46.324353+0000","last_fullsized":"2024-08-31T07:10:46.324353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T19:03:33.942165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.183968+0000","last_change":"2024-08-31T07:09:47.665881+0000","last_active":"2024-08-31T07:11:45.183968+0000","last_peered":"2024-08-31T07:11:45.183968+0000","last_clean":"2024-08-31T07:11:45.183968+0000","last_became_active":"2024-08-31T07:09:47.665485+0000","last_became_peered":"2024-08-31T07:09:47.665485+0000","last_unstale":"2024-08-31T07:11:45.183968+0000","last_undegraded":"2024-08-31T07:11:45.183968+0000","last_fullsized":"2024-08-31T07:11:45.183968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:30.820714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020851+0000","last_change":"2024-08-31T07:09:45.651893+0000","last_active":"2024-08-31T07:10:46.020851+0000","last_peered":"2024-08-31T07:10:46.020851+0000","last_clean":"2024-08-31T07:10:46.020851+0000","last_became_active":"2024-08-31T07:09:45.651547+0000","last_became_peered":"2024-08-31T07:09:45.651547+0000","last_unstale":"2024-08-31T07:10:46.020851+0000","last_undegraded":"2024-08-31T07:10:46.020851+0000","last_fullsized":"2024-08-31T07:10:46.020851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:11:30.708018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.175560+0000","last_change":"2024-08-31T07:09:47.655551+0000","last_active":"2024-08-31T07:11:45.175560+0000","last_peered":"2024-08-31T07:11:45.175560+0000","last_clean":"2024-08-31T07:11:45.175560+0000","last_became_active":"2024-08-31T07:09:47.655188+0000","last_became_peered":"2024-08-31T07:09:47.655188+0000","last_unstale":"2024-08-31T07:11:45.175560+0000","last_undegraded":"2024-08-31T07:11:45.175560+0000","last_fullsized":"2024-08-31T07:11:45.175560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:43:51.419599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'101","reported_seq":216,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:25.489306+0000","last_change":"2024-08-31T07:09:45.636212+0000","last_active":"2024-08-31T07:12:25.489306+0000","last_peered":"2024-08-31T07:12:25.489306+0000","last_clean":"2024-08-31T07:12:25.489306+0000","last_became_active":"2024-08-31T07:09:45.635678+0000","last_became_peered":"2024-08-31T07:09:45.635678+0000","last_unstale":"2024-08-31T07:12:25.489306+0000","last_undegraded":"2024-08-31T07:12:25.489306+0000","last_fullsized":"2024-08-31T07:12:25.489306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:13:17.803459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":185,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.185987+0000","last_change":"2024-08-31T07:09:47.666476+0000","last_active":"2024-08-31T07:11:45.185987+0000","last_peered":"2024-08-31T07:11:45.185987+0000","last_clean":"2024-08-31T07:11:45.185987+0000","last_became_active":"2024-08-31T07:09:47.666111+0000","last_became_peered":"2024-08-31T07:09:47.666111+0000","last_unstale":"2024-08-31T07:11:45.185987+0000","last_undegraded":"2024-08-31T07:11:45.185987+0000","last_fullsized":"2024-08-31T07:11:45.185987+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:08:53.459549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.154441+0000","last_change":"2024-08-31T07:09:45.631571+0000","last_active":"2024-08-31T07:10:46.154441+0000","last_peered":"2024-08-31T07:10:46.154441+0000","last_clean":"2024-08-31T07:10:46.154441+0000","last_became_active":"2024-08-31T07:09:45.630981+0000","last_became_peered":"2024-08-31T07:09:45.630981+0000","last_unstale":"2024-08-31T07:10:46.154441+0000","last_undegraded":"2024-08-31T07:10:46.154441+0000","last_fullsized":"2024-08-31T07:10:46.154441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:36:53.482955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193936+0000","last_change":"2024-08-31T07:09:47.675054+0000","last_active":"2024-08-31T07:11:45.193936+0000","last_peered":"2024-08-31T07:11:45.193936+0000","last_clean":"2024-08-31T07:11:45.193936+0000","last_became_active":"2024-08-31T07:09:47.672652+0000","last_became_peered":"2024-08-31T07:09:47.672652+0000","last_unstale":"2024-08-31T07:11:45.193936+0000","last_undegraded":"2024-08-31T07:11:45.193936+0000","last_fullsized":"2024-08-31T07:11:45.193936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:32:05.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016284+0000","last_change":"2024-08-31T07:09:45.648746+0000","last_active":"2024-08-31T07:10:46.016284+0000","last_peered":"2024-08-31T07:10:46.016284+0000","last_clean":"2024-08-31T07:10:46.016284+0000","last_became_active":"2024-08-31T07:09:45.648424+0000","last_became_peered":"2024-08-31T07:09:45.648424+0000","last_unstale":"2024-08-31T07:10:46.016284+0000","last_undegraded":"2024-08-31T07:10:46.016284+0000","last_fullsized":"2024-08-31T07:10:46.016284+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:40:32.132963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.165179+0000","last_change":"2024-08-31T07:09:47.663213+0000","last_active":"2024-08-31T07:11:45.165179+0000","last_peered":"2024-08-31T07:11:45.165179+0000","last_clean":"2024-08-31T07:11:45.165179+0000","last_became_active":"2024-08-31T07:09:47.662457+0000","last_became_peered":"2024-08-31T07:09:47.662457+0000","last_unstale":"2024-08-31T07:11:45.165179+0000","last_undegraded":"2024-08-31T07:11:45.165179+0000","last_fullsized":"2024-08-31T07:11:45.165179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:54:27.069200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.015653+0000","last_change":"2024-08-31T07:09:45.640149+0000","last_active":"2024-08-31T07:10:46.015653+0000","last_peered":"2024-08-31T07:10:46.015653+0000","last_clean":"2024-08-31T07:10:46.015653+0000","last_became_active":"2024-08-31T07:09:45.639581+0000","last_became_peered":"2024-08-31T07:09:45.639581+0000","last_unstale":"2024-08-31T07:10:46.015653+0000","last_undegraded":"2024-08-31T07:10:46.015653+0000","last_fullsized":"2024-08-31T07:10:46.015653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:27:35.441445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.198479+0000","last_change":"2024-08-31T07:09:47.672613+0000","last_active":"2024-08-31T07:11:45.198479+0000","last_peered":"2024-08-31T07:11:45.198479+0000","last_clean":"2024-08-31T07:11:45.198479+0000","last_became_active":"2024-08-31T07:09:47.672173+0000","last_became_peered":"2024-08-31T07:09:47.672173+0000","last_unstale":"2024-08-31T07:11:45.198479+0000","last_undegraded":"2024-08-31T07:11:45.198479+0000","last_fullsized":"2024-08-31T07:11:45.198479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:50:49.706113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.022590+0000","last_change":"2024-08-31T07:09:45.637567+0000","last_active":"2024-08-31T07:10:46.022590+0000","last_peered":"2024-08-31T07:10:46.022590+0000","last_clean":"2024-08-31T07:10:46.022590+0000","last_became_active":"2024-08-31T07:09:45.637180+0000","last_became_peered":"2024-08-31T07:09:45.637180+0000","last_unstale":"2024-08-31T07:10:46.022590+0000","last_undegraded":"2024-08-31T07:10:46.022590+0000","last_fullsized":"2024-08-31T07:10:46.022590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:48:07.190829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.194363+0000","last_change":"2024-08-31T07:09:47.674980+0000","last_active":"2024-08-31T07:11:45.194363+0000","last_peered":"2024-08-31T07:11:45.194363+0000","last_clean":"2024-08-31T07:11:45.194363+0000","last_became_active":"2024-08-31T07:09:47.674528+0000","last_became_peered":"2024-08-31T07:09:47.674528+0000","last_unstale":"2024-08-31T07:11:45.194363+0000","last_undegraded":"2024-08-31T07:11:45.194363+0000","last_fullsized":"2024-08-31T07:11:45.194363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:15:09.464221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.021096+0000","last_change":"2024-08-31T07:09:45.641167+0000","last_active":"2024-08-31T07:10:46.021096+0000","last_peered":"2024-08-31T07:10:46.021096+0000","last_clean":"2024-08-31T07:10:46.021096+0000","last_became_active":"2024-08-31T07:09:45.640761+0000","last_became_peered":"2024-08-31T07:09:45.640761+0000","last_unstale":"2024-08-31T07:10:46.021096+0000","last_undegraded":"2024-08-31T07:10:46.021096+0000","last_fullsized":"2024-08-31T07:10:46.021096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:28:59.963145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.195596+0000","last_change":"2024-08-31T07:09:47.668808+0000","last_active":"2024-08-31T07:11:45.195596+0000","last_peered":"2024-08-31T07:11:45.195596+0000","last_clean":"2024-08-31T07:11:45.195596+0000","last_became_active":"2024-08-31T07:09:47.668291+0000","last_became_peered":"2024-08-31T07:09:47.668291+0000","last_unstale":"2024-08-31T07:11:45.195596+0000","last_undegraded":"2024-08-31T07:11:45.195596+0000","last_fullsized":"2024-08-31T07:11:45.195596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:23.433511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.193074+0000","last_change":"2024-08-31T07:09:47.668039+0000","last_active":"2024-08-31T07:11:45.193074+0000","last_peered":"2024-08-31T07:11:45.193074+0000","last_clean":"2024-08-31T07:11:45.193074+0000","last_became_active":"2024-08-31T07:09:47.667567+0000","last_became_peered":"2024-08-31T07:09:47.667567+0000","last_unstale":"2024-08-31T07:11:45.193074+0000","last_undegraded":"2024-08-31T07:11:45.193074+0000","last_fullsized":"2024-08-31T07:11:45.193074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:33:50.192864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.041708+0000","last_change":"2024-08-31T07:10:46.041197+0000","last_active":"2024-08-31T07:10:46.041708+0000","last_peered":"2024-08-31T07:10:46.041708+0000","last_clean":"2024-08-31T07:10:46.041708+0000","last_became_active":"2024-08-31T07:10:46.023966+0000","last_became_peered":"2024-08-31T07:10:46.023966+0000","last_unstale":"2024-08-31T07:10:46.041708+0000","last_undegraded":"2024-08-31T07:10:46.041708+0000","last_fullsized":"2024-08-31T07:10:46.041708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:34.302665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.192670+0000","last_change":"2024-08-31T07:09:47.660615+0000","last_active":"2024-08-31T07:11:45.192670+0000","last_peered":"2024-08-31T07:11:45.192670+0000","last_clean":"2024-08-31T07:11:45.192670+0000","last_became_active":"2024-08-31T07:09:47.660269+0000","last_became_peered":"2024-08-31T07:09:47.660269+0000","last_unstale":"2024-08-31T07:11:45.192670+0000","last_undegraded":"2024-08-31T07:11:45.192670+0000","last_fullsized":"2024-08-31T07:11:45.192670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:42:12.618039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.020701+0000","last_change":"2024-08-31T07:09:45.641699+0000","last_active":"2024-08-31T07:10:46.020701+0000","last_peered":"2024-08-31T07:10:46.020701+0000","last_clean":"2024-08-31T07:10:46.020701+0000","last_became_active":"2024-08-31T07:09:45.641122+0000","last_became_peered":"2024-08-31T07:09:45.641122+0000","last_unstale":"2024-08-31T07:10:46.020701+0000","last_undegraded":"2024-08-31T07:10:46.020701+0000","last_fullsized":"2024-08-31T07:10:46.020701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:26:07.130266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.200241+0000","last_change":"2024-08-31T07:09:47.675459+0000","last_active":"2024-08-31T07:11:45.200241+0000","last_peered":"2024-08-31T07:11:45.200241+0000","last_clean":"2024-08-31T07:11:45.200241+0000","last_became_active":"2024-08-31T07:09:47.672246+0000","last_became_peered":"2024-08-31T07:09:47.672246+0000","last_unstale":"2024-08-31T07:11:45.200241+0000","last_undegraded":"2024-08-31T07:11:45.200241+0000","last_fullsized":"2024-08-31T07:11:45.200241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:49:23.547352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:10:46.016428+0000","last_change":"2024-08-31T07:09:45.649522+0000","last_active":"2024-08-31T07:10:46.016428+0000","last_peered":"2024-08-31T07:10:46.016428+0000","last_clean":"2024-08-31T07:10:46.016428+0000","last_became_active":"2024-08-31T07:09:45.649109+0000","last_became_peered":"2024-08-31T07:09:45.649109+0000","last_unstale":"2024-08-31T07:10:46.016428+0000","last_undegraded":"2024-08-31T07:10:46.016428+0000","last_fullsized":"2024-08-31T07:10:46.016428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:52:52.432605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:11:45.179375+0000","last_change":"2024-08-31T07:09:47.670583+0000","last_active":"2024-08-31T07:11:45.179375+0000","last_peered":"2024-08-31T07:11:45.179375+0000","last_clean":"2024-08-31T07:11:45.179375+0000","last_became_active":"2024-08-31T07:09:47.670162+0000","last_became_peered":"2024-08-31T07:09:47.670162+0000","last_unstale":"2024-08-31T07:11:45.179375+0000","last_undegraded":"2024-08-31T07:11:45.179375+0000","last_fullsized":"2024-08-31T07:11:45.179375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:46.621510+0000","last_clean_scrub_stamp":"2024-08-31T07:09:46.621510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:35:35.135678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'14","reported_seq":189,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T07:12:29.113597+0000","last_change":"2024-08-31T07:09:45.639636+0000","last_active":"2024-08-31T07:12:29.113597+0000","last_peered":"2024-08-31T07:12:29.113597+0000","last_clean":"2024-08-31T07:12:29.113597+0000","last_became_active":"2024-08-31T07:09:45.639029+0000","last_became_peered":"2024-08-31T07:09:45.639029+0000","last_unstale":"2024-08-31T07:12:29.113597+0000","last_undegraded":"2024-08-31T07:12:29.113597+0000","last_fullsized":"2024-08-31T07:12:29.113597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T07:09:44.605574+0000","last_clean_scrub_stamp":"2024-08-31T07:09:44.605574+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:18:54.110149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":149,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":863230,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1827,"num_read_kb":1132,"num_write":490,"num_write_kb":2247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":650,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823296,"data_stored":2589690,"data_compressed":112140,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":372,"ondisk_log_size":372,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285320,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":434494,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874291,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1089536,"data_stored":1486337,"data_compressed":60116,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037817,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":826458,"data_compressed":25040,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234048,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":826254,"data_compressed":25040,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397572,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1027343,"data_compressed":42025,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561096,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":433491,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":1419733,"data_compressed":60116,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":433352,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117082,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1219776,"data_compressed":43131,"data_compressed_allocated":57344,"data_compressed_original":778240,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378720,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":502776,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":502092,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705770,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":434830,"data_compressed":5751,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36274,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":787422,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":397128,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394134,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":463732,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T07:12:32.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:12:32.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:32 smithi084 ceph-mon[22387]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 55 op/s 2024-08-31T07:12:33.350 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:33.812 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:33 smithi032 ceph-mon[17252]: from='client.25468 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:33.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:33 smithi073 ceph-mon[21995]: from='client.25468 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:33.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:33 smithi084 ceph-mon[22387]: from='client.25468 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T07:12:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.989+0000 7f0e31400640 1 -- 172.21.15.32:0/2186348836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e2c075b70 msgr2=0x7f0e2c075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.989+0000 7f0e31400640 1 --2- 172.21.15.32:0/2186348836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e2c075b70 0x7f0e2c075f80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009900 tx=0x7f0e1402a410 comp rx=0 tx=0).stop 2024-08-31T07:12:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.989+0000 7f0e31400640 1 -- 172.21.15.32:0/2186348836 shutdown_connections 2024-08-31T07:12:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.989+0000 7f0e31400640 1 --2- 172.21.15.32:0/2186348836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e2c1141f0 0x7f0e2c1166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.989+0000 7f0e31400640 1 --2- 172.21.15.32:0/2186348836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e2c075b70 0x7f0e2c075f80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.989+0000 7f0e31400640 1 --2- 172.21.15.32:0/2186348836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c077120 0x7f0e2c075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.989+0000 7f0e31400640 1 -- 172.21.15.32:0/2186348836 >> 172.21.15.32:0/2186348836 conn(0x7f0e2c0ff370 msgr2=0x7f0e2c1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.989+0000 7f0e31400640 1 -- 172.21.15.32:0/2186348836 shutdown_connections 2024-08-31T07:12:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.989+0000 7f0e31400640 1 -- 172.21.15.32:0/2186348836 wait complete. 2024-08-31T07:12:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.990+0000 7f0e31400640 1 Processor -- start 2024-08-31T07:12:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.990+0000 7f0e31400640 1 -- start start 2024-08-31T07:12:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e31400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e2c075b70 0x7f0e2c1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e31400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e2c077120 0x7f0e2c1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e31400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c1141f0 0x7f0e2c1a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e31400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c119df0 con 0x7f0e2c077120 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e31400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c119c70 con 0x7f0e2c1141f0 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e31400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c119f70 con 0x7f0e2c075b70 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e2c077120 0x7f0e2c1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e2a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e2c075b70 0x7f0e2c1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e2c077120 0x7f0e2c1a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52090/0 (socket says 172.21.15.32:52090) 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e23e00640 1 -- 172.21.15.32:0/263891227 learned_addr learned my addr 172.21.15.32:0/263891227 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e2a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e2c075b70 0x7f0e2c1a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36272/0 (socket says 172.21.15.32:36272) 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.991+0000 7f0e2b200640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c1141f0 0x7f0e2c1a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.992+0000 7f0e2a800640 1 -- 172.21.15.32:0/263891227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c1141f0 msgr2=0x7f0e2c1a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.992+0000 7f0e2a800640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c1141f0 0x7f0e2c1a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.992+0000 7f0e2a800640 1 -- 172.21.15.32:0/263891227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e2c077120 msgr2=0x7f0e2c1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.992+0000 7f0e2a800640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e2c077120 0x7f0e2c1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.992+0000 7f0e2a800640 1 -- 172.21.15.32:0/263891227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2c1a67b0 con 0x7f0e2c075b70 2024-08-31T07:12:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.992+0000 7f0e2a800640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e2c075b70 0x7f0e2c1a0840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00e9f0 tx=0x7f0e1c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.992+0000 7f0e23400640 1 -- 172.21.15.32:0/263891227 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c00ccc0 con 0x7f0e2c075b70 2024-08-31T07:12:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.993+0000 7f0e23400640 1 -- 172.21.15.32:0/263891227 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e1c0108e0 con 0x7f0e2c075b70 2024-08-31T07:12:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.993+0000 7f0e23400640 1 -- 172.21.15.32:0/263891227 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c018840 con 0x7f0e2c075b70 2024-08-31T07:12:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.993+0000 7f0e31400640 1 -- 172.21.15.32:0/263891227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2c1a6aa0 con 0x7f0e2c075b70 2024-08-31T07:12:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.993+0000 7f0e31400640 1 -- 172.21.15.32:0/263891227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2c1ad200 con 0x7f0e2c075b70 2024-08-31T07:12:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.994+0000 7f0e31400640 1 -- 172.21.15.32:0/263891227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e2c075f80 con 0x7f0e2c075b70 2024-08-31T07:12:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.996+0000 7f0e23400640 1 -- 172.21.15.32:0/263891227 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f0e1c010430 con 0x7f0e2c075b70 2024-08-31T07:12:33.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.998+0000 7f0e23400640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0e100b4c10 0x7f0e100b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:33.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.998+0000 7f0e23400640 1 -- 172.21.15.32:0/263891227 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f0e1c0e5b50 con 0x7f0e2c075b70 2024-08-31T07:12:33.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.998+0000 7f0e23e00640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0e100b4c10 0x7f0e100b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:33.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:33.999+0000 7f0e23e00640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0e100b4c10 0x7f0e100b7120 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005f70 tx=0x7f0e14009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:34.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.000+0000 7f0e23400640 1 -- 172.21.15.32:0/263891227 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0e1c0aca20 con 0x7f0e2c075b70 2024-08-31T07:12:34.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.233+0000 7f0e31400640 1 -- 172.21.15.32:0/263891227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e2c1a1970 con 0x7f0e2c075b70 2024-08-31T07:12:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.234+0000 7f0e23400640 1 -- 172.21.15.32:0/263891227 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f0e1c0ac320 con 0x7f0e2c075b70 2024-08-31T07:12:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:34.236 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:12:34.236 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:12:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 -- 172.21.15.32:0/263891227 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0e100b4c10 msgr2=0x7f0e100b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0e100b4c10 0x7f0e100b7120 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005f70 tx=0x7f0e14009560 comp rx=0 tx=0).stop 2024-08-31T07:12:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 -- 172.21.15.32:0/263891227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e2c075b70 msgr2=0x7f0e2c1a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e2c075b70 0x7f0e2c1a0840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00e9f0 tx=0x7f0e1c00edf0 comp rx=0 tx=0).stop 2024-08-31T07:12:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 -- 172.21.15.32:0/263891227 shutdown_connections 2024-08-31T07:12:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0e100b4c10 0x7f0e100b7120 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:34.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c1141f0 0x7f0e2c1a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:34.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e2c077120 0x7f0e2c1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:34.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 --2- 172.21.15.32:0/263891227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e2c075b70 0x7f0e2c1a0840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:34.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 -- 172.21.15.32:0/263891227 >> 172.21.15.32:0/263891227 conn(0x7f0e2c0ff370 msgr2=0x7f0e2c1100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:34.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 -- 172.21.15.32:0/263891227 shutdown_connections 2024-08-31T07:12:34.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:34.237+0000 7f0e31400640 1 -- 172.21.15.32:0/263891227 wait complete. 2024-08-31T07:12:34.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:34 smithi032 ceph-mon[17252]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-08-31T07:12:34.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:34 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/263891227' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:34 smithi073 ceph-mon[21995]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-08-31T07:12:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:34 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/263891227' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:34 smithi084 ceph-mon[22387]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-08-31T07:12:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:34 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/263891227' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:35.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:12:35.420 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:36.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.076+0000 7f86ff400640 1 -- 172.21.15.32:0/3367953158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f8075ea0 msgr2=0x7f86f810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:36.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.076+0000 7f86ff400640 1 --2- 172.21.15.32:0/3367953158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f8075ea0 0x7f86f810f920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009860 tx=0x7f86e402a340 comp rx=0 tx=0).stop 2024-08-31T07:12:36.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.076+0000 7f86ff400640 1 -- 172.21.15.32:0/3367953158 shutdown_connections 2024-08-31T07:12:36.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.076+0000 7f86ff400640 1 --2- 172.21.15.32:0/3367953158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f810fe60 0x7f86f81122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:36.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.076+0000 7f86ff400640 1 --2- 172.21.15.32:0/3367953158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f8075ea0 0x7f86f810f920 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:36.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.076+0000 7f86ff400640 1 --2- 172.21.15.32:0/3367953158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f8075550 0x7f86f8075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:36.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.076+0000 7f86ff400640 1 -- 172.21.15.32:0/3367953158 >> 172.21.15.32:0/3367953158 conn(0x7f86f807b5f0 msgr2=0x7f86f807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:36.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.076+0000 7f86ff400640 1 -- 172.21.15.32:0/3367953158 shutdown_connections 2024-08-31T07:12:36.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.076+0000 7f86ff400640 1 -- 172.21.15.32:0/3367953158 wait complete. 2024-08-31T07:12:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.077+0000 7f86ff400640 1 Processor -- start 2024-08-31T07:12:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.077+0000 7f86ff400640 1 -- start start 2024-08-31T07:12:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86ff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f8075550 0x7f86f819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86ff400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f8075ea0 0x7f86f819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86ff400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f810fe60 0x7f86f81a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86ff400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8115a10 con 0x7f86f8075550 2024-08-31T07:12:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86ff400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8115890 con 0x7f86f810fe60 2024-08-31T07:12:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86ff400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8115b90 con 0x7f86f8075ea0 2024-08-31T07:12:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86fca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f8075550 0x7f86f819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86fd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f810fe60 0x7f86f81a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86f7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f8075ea0 0x7f86f819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86fca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f8075550 0x7f86f819c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54096/0 (socket says 172.21.15.32:54096) 2024-08-31T07:12:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.078+0000 7f86fca00640 1 -- 172.21.15.32:0/93473681 learned_addr learned my addr 172.21.15.32:0/93473681 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:36.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.079+0000 7f86fd400640 1 -- 172.21.15.32:0/93473681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f8075ea0 msgr2=0x7f86f819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:36.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.079+0000 7f86fd400640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f8075ea0 0x7f86f819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:36.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.079+0000 7f86fd400640 1 -- 172.21.15.32:0/93473681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f8075550 msgr2=0x7f86f819c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:36.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.079+0000 7f86fd400640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f8075550 0x7f86f819c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:36.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.079+0000 7f86fd400640 1 -- 172.21.15.32:0/93473681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f81a2330 con 0x7f86f810fe60 2024-08-31T07:12:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.079+0000 7f86f7e00640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f8075ea0 0x7f86f819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.079+0000 7f86fca00640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f8075550 0x7f86f819c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.079+0000 7f86fd400640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f810fe60 0x7f86f81a1c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009fd0 tx=0x7f86f000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.079+0000 7f86f5600640 1 -- 172.21.15.32:0/93473681 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f000ce20 con 0x7f86f810fe60 2024-08-31T07:12:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.080+0000 7f86f5600640 1 -- 172.21.15.32:0/93473681 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f0005020 con 0x7f86f810fe60 2024-08-31T07:12:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.080+0000 7f86ff400640 1 -- 172.21.15.32:0/93473681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f81a2620 con 0x7f86f810fe60 2024-08-31T07:12:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.080+0000 7f86f5600640 1 -- 172.21.15.32:0/93473681 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0010690 con 0x7f86f810fe60 2024-08-31T07:12:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.080+0000 7f86ff400640 1 -- 172.21.15.32:0/93473681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81a8d80 con 0x7f86f810fe60 2024-08-31T07:12:36.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.081+0000 7f86ff400640 1 -- 172.21.15.32:0/93473681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f8075960 con 0x7f86f810fe60 2024-08-31T07:12:36.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.083+0000 7f86f5600640 1 -- 172.21.15.32:0/93473681 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f86f00054d0 con 0x7f86f810fe60 2024-08-31T07:12:36.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.085+0000 7f86f5600640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86e00b4c10 0x7f86e00b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:36.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.085+0000 7f86f5600640 1 -- 172.21.15.32:0/93473681 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f86f00e5a60 con 0x7f86f810fe60 2024-08-31T07:12:36.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.085+0000 7f86fca00640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86e00b4c10 0x7f86e00b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:36.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.086+0000 7f86fca00640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86e00b4c10 0x7f86e00b7120 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0096c0 tx=0x7f86ec009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:36.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.087+0000 7f86f5600640 1 -- 172.21.15.32:0/93473681 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f86f00ac930 con 0x7f86f810fe60 2024-08-31T07:12:36.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.328+0000 7f86ff400640 1 -- 172.21.15.32:0/93473681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86f819d4f0 con 0x7f86f810fe60 2024-08-31T07:12:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.329+0000 7f86f5600640 1 -- 172.21.15.32:0/93473681 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f86f00b3e20 con 0x7f86f810fe60 2024-08-31T07:12:36.330 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:36.330 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:12:36.330 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:12:36.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.331+0000 7f86ff400640 1 -- 172.21.15.32:0/93473681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86e00b4c10 msgr2=0x7f86e00b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:36.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.331+0000 7f86ff400640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86e00b4c10 0x7f86e00b7120 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0096c0 tx=0x7f86ec009260 comp rx=0 tx=0).stop 2024-08-31T07:12:36.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.331+0000 7f86ff400640 1 -- 172.21.15.32:0/93473681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f810fe60 msgr2=0x7f86f81a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:36.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.331+0000 7f86ff400640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f810fe60 0x7f86f81a1c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009fd0 tx=0x7f86f000ef10 comp rx=0 tx=0).stop 2024-08-31T07:12:36.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.332+0000 7f86ff400640 1 -- 172.21.15.32:0/93473681 shutdown_connections 2024-08-31T07:12:36.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.332+0000 7f86ff400640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86e00b4c10 0x7f86e00b7120 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:36.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.332+0000 7f86ff400640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86f810fe60 0x7f86f81a1c00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:36.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.332+0000 7f86ff400640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86f8075ea0 0x7f86f819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:36.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.332+0000 7f86ff400640 1 --2- 172.21.15.32:0/93473681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f8075550 0x7f86f819c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:36.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.332+0000 7f86ff400640 1 -- 172.21.15.32:0/93473681 >> 172.21.15.32:0/93473681 conn(0x7f86f807b5f0 msgr2=0x7f86f8101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:36.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.332+0000 7f86ff400640 1 -- 172.21.15.32:0/93473681 shutdown_connections 2024-08-31T07:12:36.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:36.332+0000 7f86ff400640 1 -- 172.21.15.32:0/93473681 wait complete. 2024-08-31T07:12:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:36 smithi032 ceph-mon[17252]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-08-31T07:12:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:36 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:12:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:36 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/93473681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:36 smithi073 ceph-mon[21995]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-08-31T07:12:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:36 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:12:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:36 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/93473681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:36 smithi084 ceph-mon[22387]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-08-31T07:12:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:36 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:12:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:36 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/93473681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:37.118 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:12:37.497 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.137+0000 7fa829600640 1 -- 172.21.15.32:0/1946692027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa824105b20 msgr2=0x7fa824105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.137+0000 7fa829600640 1 --2- 172.21.15.32:0/1946692027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa824105b20 0x7fa824105f30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa814009860 tx=0x7fa81402a340 comp rx=0 tx=0).stop 2024-08-31T07:12:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.137+0000 7fa829600640 1 -- 172.21.15.32:0/1946692027 shutdown_connections 2024-08-31T07:12:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.137+0000 7fa829600640 1 --2- 172.21.15.32:0/1946692027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa82410bdb0 0x7fa8241125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.137+0000 7fa829600640 1 --2- 172.21.15.32:0/1946692027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82410b460 0x7fa82410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:38.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.137+0000 7fa829600640 1 --2- 172.21.15.32:0/1946692027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa824105b20 0x7fa824105f30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:38.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.137+0000 7fa829600640 1 -- 172.21.15.32:0/1946692027 >> 172.21.15.32:0/1946692027 conn(0x7fa8241015b0 msgr2=0x7fa824103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:38.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.138+0000 7fa829600640 1 -- 172.21.15.32:0/1946692027 shutdown_connections 2024-08-31T07:12:38.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.138+0000 7fa829600640 1 -- 172.21.15.32:0/1946692027 wait complete. 2024-08-31T07:12:38.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.139+0000 7fa829600640 1 Processor -- start 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.139+0000 7fa829600640 1 -- start start 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.139+0000 7fa829600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa824105b20 0x7fa82419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.139+0000 7fa829600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa82410b460 0x7fa82419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.139+0000 7fa829600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa82410bdb0 0x7fa8241a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.139+0000 7fa829600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa824115a70 con 0x7fa82410bdb0 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.139+0000 7fa829600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241158f0 con 0x7fa82410b460 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.139+0000 7fa829600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa824115bf0 con 0x7fa824105b20 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa822a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa824105b20 0x7fa82419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa823400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa82410bdb0 0x7fa8241a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa822a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa824105b20 0x7fa82419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47564/0 (socket says 172.21.15.32:47564) 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa822a00640 1 -- 172.21.15.32:0/1497575245 learned_addr learned my addr 172.21.15.32:0/1497575245 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa822000640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa82410b460 0x7fa82419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:38.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa822a00640 1 -- 172.21.15.32:0/1497575245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa82410b460 msgr2=0x7fa82419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:38.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa822a00640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa82410b460 0x7fa82419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:38.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa822a00640 1 -- 172.21.15.32:0/1497575245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa82410bdb0 msgr2=0x7fa8241a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:38.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa822a00640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa82410bdb0 0x7fa8241a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:38.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa822a00640 1 -- 172.21.15.32:0/1497575245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8241a2390 con 0x7fa824105b20 2024-08-31T07:12:38.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa823400640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa82410bdb0 0x7fa8241a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:38.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.140+0000 7fa822a00640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa824105b20 0x7fa82419c420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8140303f0 tx=0x7fa814032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:38.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.141+0000 7fa81b400640 1 -- 172.21.15.32:0/1497575245 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814040020 con 0x7fa824105b20 2024-08-31T07:12:38.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.141+0000 7fa829600640 1 -- 172.21.15.32:0/1497575245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8241a2620 con 0x7fa824105b20 2024-08-31T07:12:38.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.141+0000 7fa81b400640 1 -- 172.21.15.32:0/1497575245 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa814030dd0 con 0x7fa824105b20 2024-08-31T07:12:38.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.141+0000 7fa81b400640 1 -- 172.21.15.32:0/1497575245 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814037040 con 0x7fa824105b20 2024-08-31T07:12:38.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.141+0000 7fa829600640 1 -- 172.21.15.32:0/1497575245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8241a8de0 con 0x7fa824105b20 2024-08-31T07:12:38.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.142+0000 7fa829600640 1 -- 172.21.15.32:0/1497575245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa824105f30 con 0x7fa824105b20 2024-08-31T07:12:38.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.145+0000 7fa81b400640 1 -- 172.21.15.32:0/1497575245 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7fa814043020 con 0x7fa824105b20 2024-08-31T07:12:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.146+0000 7fa81b400640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8080b4c10 0x7fa8080b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.146+0000 7fa81b400640 1 -- 172.21.15.32:0/1497575245 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fa814105780 con 0x7fa824105b20 2024-08-31T07:12:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.146+0000 7fa822000640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8080b4c10 0x7fa8080b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:38.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.147+0000 7fa822000640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8080b4c10 0x7fa8080b7120 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa82410af80 tx=0x7fa810009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:38.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.148+0000 7fa81b400640 1 -- 172.21.15.32:0/1497575245 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa81404a020 con 0x7fa824105b20 2024-08-31T07:12:38.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.344+0000 7fa829600640 1 -- 172.21.15.32:0/1497575245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa82402e430 con 0x7fa824105b20 2024-08-31T07:12:38.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.346+0000 7fa81b400640 1 -- 172.21.15.32:0/1497575245 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22921 (secure 0 0 0) 0x7fa8140328c0 con 0x7fa824105b20 2024-08-31T07:12:38.347 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:38.348 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:10:46.011475+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:12:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 -- 172.21.15.32:0/1497575245 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8080b4c10 msgr2=0x7fa8080b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8080b4c10 0x7fa8080b7120 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa82410af80 tx=0x7fa810009260 comp rx=0 tx=0).stop 2024-08-31T07:12:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 -- 172.21.15.32:0/1497575245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa824105b20 msgr2=0x7fa82419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa824105b20 0x7fa82419c420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8140303f0 tx=0x7fa814032da0 comp rx=0 tx=0).stop 2024-08-31T07:12:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 -- 172.21.15.32:0/1497575245 shutdown_connections 2024-08-31T07:12:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8080b4c10 0x7fa8080b7120 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa82410bdb0 0x7fa8241a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa82410b460 0x7fa82419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 --2- 172.21.15.32:0/1497575245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa824105b20 0x7fa82419c420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:38.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 -- 172.21.15.32:0/1497575245 >> 172.21.15.32:0/1497575245 conn(0x7fa8241015b0 msgr2=0x7fa8241039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:38.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 -- 172.21.15.32:0/1497575245 shutdown_connections 2024-08-31T07:12:38.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:38.349+0000 7fa829600640 1 -- 172.21.15.32:0/1497575245 wait complete. 2024-08-31T07:12:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:38 smithi032 ceph-mon[17252]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-08-31T07:12:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:38 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1497575245' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:12:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:38 smithi073 ceph-mon[21995]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-08-31T07:12:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:38 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1497575245' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:12:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:38 smithi084 ceph-mon[22387]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-08-31T07:12:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:38 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1497575245' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:12:39.392 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-08-31T07:12:39.774 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:40.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:40 smithi032 ceph-mon[17252]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 11 op/s 2024-08-31T07:12:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:40 smithi073 ceph-mon[21995]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 11 op/s 2024-08-31T07:12:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:40 smithi084 ceph-mon[22387]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 11 op/s 2024-08-31T07:12:41.361 INFO:teuthology.orchestra.run.smithi032.stdout:{"run_dir":"/var/run/ceph"} 2024-08-31T07:12:41.362 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-08-31T07:12:41.362 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 5413 2024-08-31T07:12:41.363 DEBUG:tasks.quiescer:Yielding 2024-08-31T07:12:41.363 INFO:teuthology.run_tasks:Running task workunit... 2024-08-31T07:12:41.386 INFO:tasks.workunit:Pulling workunits from ref d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T07:12:41.387 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-31T07:12:41.387 INFO:tasks.workunit:timeout=3h 2024-08-31T07:12:41.387 INFO:tasks.workunit:cleanup=True 2024-08-31T07:12:41.388 DEBUG:teuthology.orchestra.run.smithi032:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-31T07:12:41.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:12:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-08-31T07:12:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-08-31T07:12:41.426 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-08-31T07:12:41.426 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T07:12:41.426 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-08-31T07:12:41.427 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-08-31 07:10:14.491551605 +0000 2024-08-31T07:12:41.427 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-08-31 07:10:14.491551605 +0000 2024-08-31T07:12:41.427 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-08-31 07:12:11.611077394 +0000 2024-08-31T07:12:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-08-31 07:10:14.491551605 +0000 2024-08-31T07:12:41.427 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-08-31T07:12:41.427 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-08-31T07:12:41.533 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T07:12:41.571 INFO:tasks.workunit.client.0.smithi032.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-31T07:12:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:42.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.463+0000 7fb3d7600640 1 -- 172.21.15.32:0/2257841842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d0105b20 msgr2=0x7fb3d0105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:42.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.463+0000 7fb3d7600640 1 --2- 172.21.15.32:0/2257841842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d0105b20 0x7fb3d0105f30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009900 tx=0x7fb3c402a410 comp rx=0 tx=0).stop 2024-08-31T07:12:42.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.463+0000 7fb3d7600640 1 -- 172.21.15.32:0/2257841842 shutdown_connections 2024-08-31T07:12:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.463+0000 7fb3d7600640 1 --2- 172.21.15.32:0/2257841842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d010bdb0 0x7fb3d01125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.463+0000 7fb3d7600640 1 --2- 172.21.15.32:0/2257841842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d010b460 0x7fb3d010b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.463+0000 7fb3d7600640 1 --2- 172.21.15.32:0/2257841842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d0105b20 0x7fb3d0105f30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.463+0000 7fb3d7600640 1 -- 172.21.15.32:0/2257841842 >> 172.21.15.32:0/2257841842 conn(0x7fb3d01015b0 msgr2=0x7fb3d0103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.463+0000 7fb3d7600640 1 -- 172.21.15.32:0/2257841842 shutdown_connections 2024-08-31T07:12:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.463+0000 7fb3d7600640 1 -- 172.21.15.32:0/2257841842 wait complete. 2024-08-31T07:12:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.464+0000 7fb3d7600640 1 Processor -- start 2024-08-31T07:12:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.464+0000 7fb3d7600640 1 -- start start 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d7600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d0105b20 0x7fb3d019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d7600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d010b460 0x7fb3d019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d7600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d010bdb0 0x7fb3d01a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d7600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0115a70 con 0x7fb3d010b460 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d7600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d01158f0 con 0x7fb3d010bdb0 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d7600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0115bf0 con 0x7fb3d0105b20 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d0105b20 0x7fb3d019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d0105b20 0x7fb3d019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47582/0 (socket says 172.21.15.32:47582) 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d4c00640 1 -- 172.21.15.32:0/2481970285 learned_addr learned my addr 172.21.15.32:0/2481970285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3cfe00640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d010b460 0x7fb3d019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d5600640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d010bdb0 0x7fb3d01a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d4c00640 1 -- 172.21.15.32:0/2481970285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d010bdb0 msgr2=0x7fb3d01a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d4c00640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d010bdb0 0x7fb3d01a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d4c00640 1 -- 172.21.15.32:0/2481970285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d010b460 msgr2=0x7fb3d019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d4c00640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d010b460 0x7fb3d019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.465+0000 7fb3d4c00640 1 -- 172.21.15.32:0/2481970285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d01a23d0 con 0x7fb3d0105b20 2024-08-31T07:12:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.466+0000 7fb3d4c00640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d0105b20 0x7fb3d019c460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4005940 tx=0x7fb3c40040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.466+0000 7fb3cd600640 1 -- 172.21.15.32:0/2481970285 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c403f030 con 0x7fb3d0105b20 2024-08-31T07:12:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.466+0000 7fb3d7600640 1 -- 172.21.15.32:0/2481970285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d01a2660 con 0x7fb3d0105b20 2024-08-31T07:12:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.466+0000 7fb3d7600640 1 -- 172.21.15.32:0/2481970285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d01a8e20 con 0x7fb3d0105b20 2024-08-31T07:12:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.467+0000 7fb3cd600640 1 -- 172.21.15.32:0/2481970285 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c40044c0 con 0x7fb3d0105b20 2024-08-31T07:12:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.467+0000 7fb3cd600640 1 -- 172.21.15.32:0/2481970285 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4049060 con 0x7fb3d0105b20 2024-08-31T07:12:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.467+0000 7fb3d7600640 1 -- 172.21.15.32:0/2481970285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b00052c0 con 0x7fb3d0105b20 2024-08-31T07:12:42.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.470+0000 7fb3cd600640 1 -- 172.21.15.32:0/2481970285 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7fb3c4042030 con 0x7fb3d0105b20 2024-08-31T07:12:42.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.471+0000 7fb3cd600640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3b80b4c90 0x7fb3b80b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:42.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.472+0000 7fb3cd600640 1 -- 172.21.15.32:0/2481970285 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb3c4105790 con 0x7fb3d0105b20 2024-08-31T07:12:42.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.472+0000 7fb3cfe00640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3b80b4c90 0x7fb3b80b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.473+0000 7fb3cfe00640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3b80b4c90 0x7fb3b80b71a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb3d010afa0 tx=0x7fb3bc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:42.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.474+0000 7fb3cd600640 1 -- 172.21.15.32:0/2481970285 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb3c40cc010 con 0x7fb3d0105b20 2024-08-31T07:12:42.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.709+0000 7fb3d7600640 1 -- 172.21.15.32:0/2481970285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3b00055b0 con 0x7fb3d0105b20 2024-08-31T07:12:42.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.711+0000 7fb3cd600640 1 -- 172.21.15.32:0/2481970285 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7fb3c40d07d0 con 0x7fb3d0105b20 2024-08-31T07:12:42.712 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:12:42.712 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:12:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:12:42.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 -- 172.21.15.32:0/2481970285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3b80b4c90 msgr2=0x7fb3b80b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:42.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3b80b4c90 0x7fb3b80b71a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb3d010afa0 tx=0x7fb3bc005f90 comp rx=0 tx=0).stop 2024-08-31T07:12:42.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 -- 172.21.15.32:0/2481970285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d0105b20 msgr2=0x7fb3d019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:42.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d0105b20 0x7fb3d019c460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4005940 tx=0x7fb3c40040c0 comp rx=0 tx=0).stop 2024-08-31T07:12:42.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 -- 172.21.15.32:0/2481970285 shutdown_connections 2024-08-31T07:12:42.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3b80b4c90 0x7fb3b80b71a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:42.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3d010bdb0 0x7fb3d01a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d010b460 0x7fb3d019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 --2- 172.21.15.32:0/2481970285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3d0105b20 0x7fb3d019c460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 -- 172.21.15.32:0/2481970285 >> 172.21.15.32:0/2481970285 conn(0x7fb3d01015b0 msgr2=0x7fb3d01039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 -- 172.21.15.32:0/2481970285 shutdown_connections 2024-08-31T07:12:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:42.713+0000 7fb3d7600640 1 -- 172.21.15.32:0/2481970285 wait complete. 2024-08-31T07:12:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:42 smithi073 ceph-mon[21995]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:42 smithi084 ceph-mon[22387]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:43.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:42 smithi032 ceph-mon[17252]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:12:43.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-31T07:09:48.797887+0000', 'modified': '2024-08-31T07:12:12.630233+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24734, 'mds_1': 24718, 'mds_2': 24694, 'mds_3': 24692, 'mds_4': 24676}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24625': {'gid': 24625, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6833/2138862695', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6832', 'nonce': 2138862695}, {'type': 'v1', 'addr': '172.21.15.84:6833', 'nonce': 2138862695}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24632': {'gid': 24632, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6833/2003936745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 2003936745}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 2003936745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24644': {'gid': 24644, 'name': 'j', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.32:6835/1250889023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1250889023}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1250889023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6835/1927490574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 1927490574}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 1927490574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24670': {'gid': 24670, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6835/204703622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 204703622}, {'type': 'v1', 'addr': '172.21.15.84:6835', 'nonce': 204703622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24676': {'gid': 24676, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.84:6837/4208526312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 4208526312}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 4208526312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24692': {'gid': 24692, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.73:6837/2735170682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 2735170682}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2735170682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24694': {'gid': 24694, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.84:6839/2807364599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6838', 'nonce': 2807364599}, {'type': 'v1', 'addr': '172.21.15.84:6839', 'nonce': 2807364599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24718': {'gid': 24718, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.32:6841/2484505785', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6840', 'nonce': 2484505785}, {'type': 'v1', 'addr': '172.21.15.32:6841', 'nonce': 2484505785}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24734': {'gid': 24734, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.73:6839/794507464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 794507464}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 794507464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24734, 'qdb_cluster': [24734, 24694, 24718, 24692, 24676]} 2024-08-31T07:12:43.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-08-31T07:12:43.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.c status 2024-08-31T07:12:43.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:43 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2481970285' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:43.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:43 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2481970285' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:43.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:43 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2481970285' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:12:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.711+0000 7f01c3800640 1 -- 172.21.15.32:0/3563758095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01bc103900 msgr2=0x7f01bc103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.711+0000 7f01c3800640 1 --2- 172.21.15.32:0/3563758095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01bc103900 0x7f01bc103d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009900 tx=0x7f01ac02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.711+0000 7f01c3800640 1 -- 172.21.15.32:0/3563758095 shutdown_connections 2024-08-31T07:12:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.711+0000 7f01c3800640 1 --2- 172.21.15.32:0/3563758095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc109b90 0x7f01bc1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.711+0000 7f01c3800640 1 --2- 172.21.15.32:0/3563758095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01bc109240 0x7f01bc109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.711+0000 7f01c3800640 1 --2- 172.21.15.32:0/3563758095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01bc103900 0x7f01bc103d10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.711+0000 7f01c3800640 1 -- 172.21.15.32:0/3563758095 >> 172.21.15.32:0/3563758095 conn(0x7f01bc07b5f0 msgr2=0x7f01bc07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.711+0000 7f01c3800640 1 -- 172.21.15.32:0/3563758095 shutdown_connections 2024-08-31T07:12:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.711+0000 7f01c3800640 1 -- 172.21.15.32:0/3563758095 wait complete. 2024-08-31T07:12:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.712+0000 7f01c3800640 1 Processor -- start 2024-08-31T07:12:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.712+0000 7f01c3800640 1 -- start start 2024-08-31T07:12:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01c3800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01bc103900 0x7f01bc19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01c3800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc109240 0x7f01bc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01c3800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01bc109b90 0x7f01bc19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01c3800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc1138a0 con 0x7f01bc109240 2024-08-31T07:12:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01c3800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc113720 con 0x7f01bc103900 2024-08-31T07:12:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01c3800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc113a20 con 0x7f01bc109b90 2024-08-31T07:12:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc109240 0x7f01bc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01c0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01bc103900 0x7f01bc19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01c0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01bc103900 0x7f01bc19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60550/0 (socket says 172.21.15.32:60550) 2024-08-31T07:12:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01c0e00640 1 -- 172.21.15.32:0/3238292992 learned_addr learned my addr 172.21.15.32:0/3238292992 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.713+0000 7f01c1800640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01bc109b90 0x7f01bc19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.714+0000 7f01b3e00640 1 -- 172.21.15.32:0/3238292992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01bc109b90 msgr2=0x7f01bc19fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.714+0000 7f01b3e00640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01bc109b90 0x7f01bc19fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.714+0000 7f01b3e00640 1 -- 172.21.15.32:0/3238292992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01bc103900 msgr2=0x7f01bc19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.714+0000 7f01b3e00640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01bc103900 0x7f01bc19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.714+0000 7f01b3e00640 1 -- 172.21.15.32:0/3238292992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc1a02e0 con 0x7f01bc109240 2024-08-31T07:12:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.714+0000 7f01b3e00640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc109240 0x7f01bc19a910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01bc06c790 tx=0x7f01a400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.714+0000 7f01ba000640 1 -- 172.21.15.32:0/3238292992 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a400cc40 con 0x7f01bc109240 2024-08-31T07:12:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.714+0000 7f01ba000640 1 -- 172.21.15.32:0/3238292992 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a400cde0 con 0x7f01bc109240 2024-08-31T07:12:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.714+0000 7f01ba000640 1 -- 172.21.15.32:0/3238292992 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4018780 con 0x7f01bc109240 2024-08-31T07:12:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.714+0000 7f01c3800640 1 -- 172.21.15.32:0/3238292992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc1a05d0 con 0x7f01bc109240 2024-08-31T07:12:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.715+0000 7f01c3800640 1 -- 172.21.15.32:0/3238292992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc1a6d30 con 0x7f01bc109240 2024-08-31T07:12:44.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.718+0000 7f01ca3fa740 1 Processor -- start 2024-08-31T07:12:44.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.718+0000 7f01ca3fa740 1 -- start start 2024-08-31T07:12:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.718+0000 7f01ba000640 1 -- 172.21.15.32:0/3238292992 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f01a4010a90 con 0x7f01bc109240 2024-08-31T07:12:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.718+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f5a8f60 0x56211f94e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.718+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f94ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.718+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f588280 0x56211f94f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.718+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56211f8c5550 con 0x56211f5800a0 2024-08-31T07:12:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.718+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56211f8b4ae0 con 0x56211f5a8f60 2024-08-31T07:12:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.718+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56211f8b8a90 con 0x56211f588280 2024-08-31T07:12:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.719+0000 7f01c0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f94ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.719+0000 7f01c0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f94ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54162/0 (socket says 172.21.15.32:54162) 2024-08-31T07:12:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.719+0000 7f01c0e00640 1 -- 172.21.15.32:0/2634821380 learned_addr learned my addr 172.21.15.32:0/2634821380 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:44.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.719+0000 7f01b3e00640 1 --2- 172.21.15.32:0/2634821380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f588280 0x56211f94f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.719+0000 7f01c1800640 1 --2- 172.21.15.32:0/2634821380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f5a8f60 0x56211f94e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.719+0000 7f01c0e00640 1 -- 172.21.15.32:0/2634821380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f588280 msgr2=0x56211f94f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:44.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.719+0000 7f01c0e00640 1 --2- 172.21.15.32:0/2634821380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f588280 0x56211f94f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.719+0000 7f01c0e00640 1 -- 172.21.15.32:0/2634821380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f5a8f60 msgr2=0x56211f94e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:44.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.719+0000 7f01c0e00640 1 --2- 172.21.15.32:0/2634821380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f5a8f60 0x56211f94e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.719+0000 7f01c0e00640 1 -- 172.21.15.32:0/2634821380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56211f614540 con 0x56211f5800a0 2024-08-31T07:12:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.720+0000 7f01c0e00640 1 --2- 172.21.15.32:0/2634821380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f94ee30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f01ac006010 tx=0x7f01ac004100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32033e0c1deed90 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.720+0000 7f01b2000640 1 -- 172.21.15.32:0/2634821380 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac040070 con 0x56211f5800a0 2024-08-31T07:12:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.720+0000 7f01b2000640 1 -- 172.21.15.32:0/2634821380 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ac0054d0 con 0x56211f5800a0 2024-08-31T07:12:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.720+0000 7f01b2000640 1 -- 172.21.15.32:0/2634821380 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac035040 con 0x56211f5800a0 2024-08-31T07:12:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.720+0000 7f01ba000640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01a00b4c90 0x7f01a00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01ba000640 1 -- 172.21.15.32:0/3238292992 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f01a40e5600 con 0x7f01bc109240 2024-08-31T07:12:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01c1800640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01a00b4c90 0x7f01a00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01ca3fa740 1 -- 172.21.15.32:0/2634821380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 msgr2=0x56211f94ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/2634821380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f94ee30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f01ac006010 tx=0x7f01ac004100 comp rx=0 tx=0).stop 2024-08-31T07:12:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01ca3fa740 1 -- 172.21.15.32:0/2634821380 shutdown_connections 2024-08-31T07:12:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/2634821380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f588280 0x56211f94f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/2634821380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f94ee30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/2634821380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f5a8f60 0x56211f94e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01ca3fa740 1 -- 172.21.15.32:0/2634821380 >> 172.21.15.32:0/2634821380 conn(0x56211f59d350 msgr2=0x56211f94c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01ca3fa740 1 -- 172.21.15.32:0/2634821380 shutdown_connections 2024-08-31T07:12:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.721+0000 7f01c1800640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01a00b4c90 0x7f01a00b71a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f01b4005fd0 tx=0x7f01b4009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.722+0000 7f01ca3fa740 1 -- 172.21.15.32:0/2634821380 wait complete. 2024-08-31T07:12:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.722+0000 7f01ca3fa740 1 -- start start 2024-08-31T07:12:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.722+0000 7f01ca3fa740 1 Processor -- start 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5800a0 0x56211f95a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f95a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5a8f60 0x56211f966360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56211f7175f0 con 0x56211f5a8f60 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56211f83bbd0 con 0x56211f588280 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56211f95d070 con 0x56211f5800a0 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01ca3fa740 10 client.0 _ll_register_callbacks cb 0x56211f861210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01c1800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5a8f60 0x56211f966360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01c1800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5a8f60 0x56211f966360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54170/0 (socket says 172.21.15.32:54170) 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01c0e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5800a0 0x56211f95a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01c1800640 1 -- 172.21.15.32:0/3161149246 learned_addr learned my addr 172.21.15.32:0/3161149246 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.723+0000 7f01b3e00640 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f95a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.724+0000 7f01c0e00640 1 -- 172.21.15.32:0/3161149246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 msgr2=0x56211f95a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.724+0000 7f01c0e00640 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f95a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.724+0000 7f01c0e00640 1 -- 172.21.15.32:0/3161149246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5a8f60 msgr2=0x56211f966360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.724+0000 7f01c0e00640 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5a8f60 0x56211f966360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.724+0000 7f01c0e00640 1 -- 172.21.15.32:0/3161149246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56211f87c650 con 0x56211f5800a0 2024-08-31T07:12:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.724+0000 7f01c0e00640 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5800a0 0x56211f95a140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005ab0 tx=0x7f01ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bffde57a6bd4d709 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.724+0000 7f01aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:12:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.724+0000 7f01aaa00640 1 -- 172.21.15.32:0/3161149246 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac040070 con 0x56211f5800a0 2024-08-31T07:12:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.724+0000 7f01aaa00640 1 -- 172.21.15.32:0/3161149246 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f01ac005000 con 0x56211f5800a0 2024-08-31T07:12:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.724+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3161149246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ac0054d0 con 0x56211f5800a0 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.725+0000 7f01aaa00640 1 -- 172.21.15.32:0/3161149246 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01ac035040 con 0x56211f5800a0 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.725+0000 7f01ca3fa740 10 client.25504 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.725+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3161149246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56211f968ed0 con 0x56211f5800a0 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.725+0000 7f01aaa00640 1 -- 172.21.15.32:0/3161149246 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ac0054d0 con 0x56211f5800a0 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.725+0000 7f01aaa00640 1 -- 172.21.15.32:0/3161149246 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f01ac0314a0 con 0x56211f5800a0 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.725+0000 7f01ca3fa740 10 client.25504 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.725+0000 7f01ca3fa740 10 client.25504 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.725+0000 7f01ca3fa740 20 client.25504 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.726+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56211f5678e0 0x56211f96c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.726+0000 7f01ca3fa740 4 client.25504 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.726+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3161149246 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56211f8c74a0 con 0x56211f5678e0 2024-08-31T07:12:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.726+0000 7f01b3e00640 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56211f5678e0 0x56211f96c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.726+0000 7f01b3e00640 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56211f5678e0 0x56211f96c400 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f5b6e45b9dbc1fe server_cookie=2438ed5499e78061 in_seq=0 out_seq=0 2024-08-31T07:12:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.726+0000 7f01aaa00640 10 client.25504 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.727+0000 7f01aaa00640 1 -- 172.21.15.32:0/3161149246 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56211f8c74a0 con 0x56211f5678e0 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.727+0000 7f01aaa00640 10 client.25504 handle_command_reply: tid=0 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.727+0000 7f01ca3fa740 1 client.25504 shutdown 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3161149246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5800a0 msgr2=0x56211f95a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5800a0 0x56211f95a140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005ab0 tx=0x7f01ac009260 comp rx=0 tx=0).stop 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3161149246 shutdown_connections 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56211f5678e0 0x56211f96c400 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5a8f60 0x56211f966360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f95a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3161149246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5800a0 0x56211f95a140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3161149246 >> 172.21.15.32:0/3161149246 conn(0x56211f59d350 msgr2=0x56211f951ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01aaa00640 0 client.25504 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3161149246 shutdown_connections 2024-08-31T07:12:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3161149246 wait complete. 2024-08-31T07:12:44.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.728+0000 7f01ca3fa740 20 client.25504 trim_cache size 0 max 16384 2024-08-31T07:12:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.747+0000 7f01ca3fa740 1 Processor -- start 2024-08-31T07:12:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.747+0000 7f01ca3fa740 1 -- start start 2024-08-31T07:12:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.747+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f95a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f9662d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5a8f60 0x56211f966810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56211f96a740 con 0x56211f5800a0 2024-08-31T07:12:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56211f96ae10 con 0x56211f588280 2024-08-31T07:12:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56211f96b4e0 con 0x56211f5a8f60 2024-08-31T07:12:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01c0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f95a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01c0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f95a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54186/0 (socket says 172.21.15.32:54186) 2024-08-31T07:12:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01c0e00640 1 -- 172.21.15.32:0/4191763190 learned_addr learned my addr 172.21.15.32:0/4191763190 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01c1800640 1 --2- 172.21.15.32:0/4191763190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5a8f60 0x56211f966810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01b3e00640 1 --2- 172.21.15.32:0/4191763190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f9662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01c0e00640 1 -- 172.21.15.32:0/4191763190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5a8f60 msgr2=0x56211f966810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01c0e00640 1 --2- 172.21.15.32:0/4191763190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5a8f60 0x56211f966810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01c0e00640 1 -- 172.21.15.32:0/4191763190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 msgr2=0x56211f9662d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01c0e00640 1 --2- 172.21.15.32:0/4191763190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f9662d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.748+0000 7f01c0e00640 1 -- 172.21.15.32:0/4191763190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56211f95ab40 con 0x56211f5800a0 2024-08-31T07:12:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.749+0000 7f01c0e00640 1 --2- 172.21.15.32:0/4191763190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f95a600 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f01ac000c00 tx=0x7f01ac005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c9d6104ae5bd07 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:44.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.749+0000 7f01b1600640 1 -- 172.21.15.32:0/4191763190 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac040070 con 0x56211f5800a0 2024-08-31T07:12:44.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.749+0000 7f01b1600640 1 -- 172.21.15.32:0/4191763190 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ac05c020 con 0x56211f5800a0 2024-08-31T07:12:44.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.749+0000 7f01b1600640 1 -- 172.21.15.32:0/4191763190 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac003d00 con 0x56211f5800a0 2024-08-31T07:12:44.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.750+0000 7f01ca3fa740 1 -- 172.21.15.32:0/4191763190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 msgr2=0x56211f95a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:44.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.750+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/4191763190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f95a600 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f01ac000c00 tx=0x7f01ac005fd0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.750+0000 7f01ca3fa740 1 -- 172.21.15.32:0/4191763190 shutdown_connections 2024-08-31T07:12:44.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.750+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/4191763190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5a8f60 0x56211f966810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.750+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/4191763190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f9662d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.750+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/4191763190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f95a600 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.750+0000 7f01ca3fa740 1 -- 172.21.15.32:0/4191763190 >> 172.21.15.32:0/4191763190 conn(0x56211f59d350 msgr2=0x56211f959ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:44.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.750+0000 7f01ca3fa740 1 -- 172.21.15.32:0/4191763190 shutdown_connections 2024-08-31T07:12:44.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.750+0000 7f01ca3fa740 1 -- 172.21.15.32:0/4191763190 wait complete. 2024-08-31T07:12:44.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.750+0000 7f01ca3fa740 1 -- start start 2024-08-31T07:12:44.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.751+0000 7f01ca3fa740 1 Processor -- start 2024-08-31T07:12:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.751+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f975020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.751+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f9704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.751+0000 7f01ca3fa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5a8f60 0x56211f970bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.751+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56211f96c280 con 0x56211f5800a0 2024-08-31T07:12:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.751+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56211f96bbb0 con 0x56211f588280 2024-08-31T07:12:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.751+0000 7f01ca3fa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56211f96c950 con 0x56211f5a8f60 2024-08-31T07:12:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.751+0000 7f01ca3fa740 10 client.0 _ll_register_callbacks cb 0x7f01ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01c0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f975020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f9704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f9704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60576/0 (socket says 172.21.15.32:60576) 2024-08-31T07:12:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01c0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f975020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54188/0 (socket says 172.21.15.32:54188) 2024-08-31T07:12:44.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01c1800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5a8f60 0x56211f970bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01c0e00640 1 -- 172.21.15.32:0/3651554806 learned_addr learned my addr 172.21.15.32:0/3651554806 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01c0e00640 1 -- 172.21.15.32:0/3651554806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5a8f60 msgr2=0x56211f970bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01c0e00640 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5a8f60 0x56211f970bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01c0e00640 1 -- 172.21.15.32:0/3651554806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 msgr2=0x56211f9704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01c0e00640 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f9704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01c0e00640 1 -- 172.21.15.32:0/3651554806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56211f9d68d0 con 0x56211f5800a0 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.752+0000 7f01b3e00640 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f9704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.753+0000 7f01c0e00640 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f975020 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f01ac02abb0 tx=0x7f01ac031da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2121b959d59905a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.753+0000 7f01aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.753+0000 7f01aaa00640 1 -- 172.21.15.32:0/3651554806 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac040030 con 0x56211f5800a0 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.753+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3651554806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ac05c020 con 0x56211f5800a0 2024-08-31T07:12:44.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.753+0000 7f01aaa00640 1 -- 172.21.15.32:0/3651554806 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f01ac0037f0 con 0x56211f5800a0 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.753+0000 7f01aaa00640 1 -- 172.21.15.32:0/3651554806 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01ac071050 con 0x56211f5800a0 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.753+0000 7f01ca3fa740 10 client.15492 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.753+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3651554806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56211f9d7990 con 0x56211f5800a0 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.754+0000 7f01aaa00640 1 -- 172.21.15.32:0/3651554806 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ac05c020 con 0x56211f5800a0 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.754+0000 7f01aaa00640 1 -- 172.21.15.32:0/3651554806 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f01ac005310 con 0x56211f5800a0 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.754+0000 7f01ca3fa740 10 client.15492 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.754+0000 7f01ca3fa740 10 client.15492 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.754+0000 7f01ca3fa740 20 client.15492 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.754+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56211f53cc90 0x56211f9dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.754+0000 7f01ca3fa740 4 client.15492 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.754+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3651554806 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "status"}) -- 0x56211f8c2f40 con 0x56211f53cc90 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.754+0000 7f01b3e00640 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56211f53cc90 0x56211f9dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:44.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01b3e00640 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56211f53cc90 0x56211f9dacd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2bf21c409871dc85 server_cookie=733cbc6963fc3eb in_seq=0 out_seq=0 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01aaa00640 10 client.15492 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01aaa00640 1 -- 172.21.15.32:0/3651554806 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56211f8c2f40 con 0x56211f53cc90 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01aaa00640 10 client.15492 handle_command_reply: tid=0 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 client.15492 shutdown 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3651554806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 msgr2=0x56211f975020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f975020 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f01ac02abb0 tx=0x7f01ac031da0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3651554806 shutdown_connections 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56211f53cc90 0x56211f9dacd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56211f5a8f60 0x56211f970bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56211f588280 0x56211f9704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 --2- 172.21.15.32:0/3651554806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56211f5800a0 0x56211f975020 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3651554806 >> 172.21.15.32:0/3651554806 conn(0x56211f59d350 msgr2=0x56211f9519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01aaa00640 0 client.15492 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:12:44.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3651554806 shutdown_connections 2024-08-31T07:12:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.755+0000 7f01ca3fa740 1 -- 172.21.15.32:0/3651554806 wait complete. 2024-08-31T07:12:44.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01ca3fa740 20 client.15492 trim_cache size 0 max 16384 2024-08-31T07:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 4, 2024-08-31T07:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24676, 2024-08-31T07:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 161.884603688, 2024-08-31T07:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 29, 2024-08-31T07:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 80, 2024-08-31T07:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 204.81880115800001 2024-08-31T07:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:44.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01c3800640 1 -- 172.21.15.32:0/3238292992 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01a00b4c90 msgr2=0x7f01a00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:44.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01c3800640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01a00b4c90 0x7f01a00b71a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f01b4005fd0 tx=0x7f01b4009500 comp rx=0 tx=0).stop 2024-08-31T07:12:44.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01c3800640 1 -- 172.21.15.32:0/3238292992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc109240 msgr2=0x7f01bc19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:44.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01c3800640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc109240 0x7f01bc19a910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01bc06c790 tx=0x7f01a400ee50 comp rx=0 tx=0).stop 2024-08-31T07:12:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01c3800640 1 -- 172.21.15.32:0/3238292992 shutdown_connections 2024-08-31T07:12:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01c3800640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01a00b4c90 0x7f01a00b71a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01c3800640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01bc109b90 0x7f01bc19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01c3800640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01bc109240 0x7f01bc19a910 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01c3800640 1 --2- 172.21.15.32:0/3238292992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01bc103900 0x7f01bc19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.756+0000 7f01c3800640 1 -- 172.21.15.32:0/3238292992 >> 172.21.15.32:0/3238292992 conn(0x7f01bc07b5f0 msgr2=0x7f01bc107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.757+0000 7f01c3800640 1 -- 172.21.15.32:0/3238292992 shutdown_connections 2024-08-31T07:12:44.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:44.757+0000 7f01c3800640 1 -- 172.21.15.32:0/3238292992 wait complete. 2024-08-31T07:12:44.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:44 smithi032 ceph-mon[17252]: pgmap v367: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-31T07:12:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:44 smithi073 ceph-mon[21995]: pgmap v367: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-31T07:12:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:44 smithi084 ceph-mon[22387]: pgmap v367: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-31T07:12:45.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.h status 2024-08-31T07:12:45.965 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:45 smithi032 ceph-mon[17252]: pgmap v368: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-08-31T07:12:45.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:45 smithi084 ceph-mon[22387]: pgmap v368: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-08-31T07:12:46.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:45 smithi073 ceph-mon[21995]: pgmap v368: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-08-31T07:12:46.294 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.985+0000 7fb734a00640 1 -- 172.21.15.32:0/149926980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb730102190 msgr2=0x7fb7301025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.985+0000 7fb734a00640 1 --2- 172.21.15.32:0/149926980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb730102190 0x7fb7301025a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb720009900 tx=0x7fb72002a480 comp rx=0 tx=0).stop 2024-08-31T07:12:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.985+0000 7fb734a00640 1 -- 172.21.15.32:0/149926980 shutdown_connections 2024-08-31T07:12:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.985+0000 7fb734a00640 1 --2- 172.21.15.32:0/149926980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb73010fd40 0x7fb730112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.985+0000 7fb734a00640 1 --2- 172.21.15.32:0/149926980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb730102ae0 0x7fb73010f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.985+0000 7fb734a00640 1 --2- 172.21.15.32:0/149926980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb730102190 0x7fb7301025a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.985+0000 7fb734a00640 1 -- 172.21.15.32:0/149926980 >> 172.21.15.32:0/149926980 conn(0x7fb7300fe070 msgr2=0x7fb7301004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.985+0000 7fb734a00640 1 -- 172.21.15.32:0/149926980 shutdown_connections 2024-08-31T07:12:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.985+0000 7fb734a00640 1 -- 172.21.15.32:0/149926980 wait complete. 2024-08-31T07:12:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.986+0000 7fb734a00640 1 Processor -- start 2024-08-31T07:12:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.986+0000 7fb734a00640 1 -- start start 2024-08-31T07:12:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb734a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb730102190 0x7fb73019c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb734a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730102ae0 0x7fb73019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb734a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73010fd40 0x7fb7301a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb734a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7301157f0 con 0x7fb730102190 2024-08-31T07:12:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb734a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb730115670 con 0x7fb730102ae0 2024-08-31T07:12:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb734a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb730115970 con 0x7fb73010fd40 2024-08-31T07:12:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb72f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb730102190 0x7fb73019c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb72ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730102ae0 0x7fb73019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb72fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73010fd40 0x7fb7301a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb72f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb730102190 0x7fb73019c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57334/0 (socket says 172.21.15.32:57334) 2024-08-31T07:12:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.987+0000 7fb72f400640 1 -- 172.21.15.32:0/2637497338 learned_addr learned my addr 172.21.15.32:0/2637497338 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb72ea00640 1 -- 172.21.15.32:0/2637497338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73010fd40 msgr2=0x7fb7301a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb72ea00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73010fd40 0x7fb7301a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb72ea00640 1 -- 172.21.15.32:0/2637497338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb730102190 msgr2=0x7fb73019c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb72ea00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb730102190 0x7fb73019c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb72ea00640 1 -- 172.21.15.32:0/2637497338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7301a2190 con 0x7fb730102ae0 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb72fe00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73010fd40 0x7fb7301a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb72ea00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730102ae0 0x7fb73019c7c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00e9f0 tx=0x7fb71c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb727e00640 1 -- 172.21.15.32:0/2637497338 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00ccc0 con 0x7fb730102ae0 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb734a00640 1 -- 172.21.15.32:0/2637497338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7301a2480 con 0x7fb730102ae0 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb727e00640 1 -- 172.21.15.32:0/2637497338 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c0108e0 con 0x7fb730102ae0 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb727e00640 1 -- 172.21.15.32:0/2637497338 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c018880 con 0x7fb730102ae0 2024-08-31T07:12:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.988+0000 7fb734a00640 1 -- 172.21.15.32:0/2637497338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7301a8be0 con 0x7fb730102ae0 2024-08-31T07:12:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.991+0000 7fb73cc06740 1 Processor -- start 2024-08-31T07:12:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb73cc06740 1 -- start start 2024-08-31T07:12:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdd07f60 0x55a7be0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdce7280 0x55a7be0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:46.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb73cc06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a7be024550 con 0x55a7bdcdf0a0 2024-08-31T07:12:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb73cc06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a7be013ae0 con 0x55a7bdd07f60 2024-08-31T07:12:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb73cc06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a7be017a90 con 0x55a7bdce7280 2024-08-31T07:12:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb72f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb72f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57336/0 (socket says 172.21.15.32:57336) 2024-08-31T07:12:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb72f400640 1 -- 172.21.15.32:0/3530635313 learned_addr learned my addr 172.21.15.32:0/3530635313 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:46.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.992+0000 7fb72fe00640 1 --2- 172.21.15.32:0/3530635313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdd07f60 0x55a7be0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:46.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.993+0000 7fb727e00640 1 -- 172.21.15.32:0/2637497338 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7fb71c010430 con 0x7fb730102ae0 2024-08-31T07:12:46.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.993+0000 7fb72f400640 1 -- 172.21.15.32:0/3530635313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdce7280 msgr2=0x55a7be0ae520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:12:46.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.993+0000 7fb72f400640 1 --2- 172.21.15.32:0/3530635313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdce7280 0x55a7be0ae520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:46.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.993+0000 7fb72f400640 1 -- 172.21.15.32:0/3530635313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdd07f60 msgr2=0x55a7be0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:46.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.993+0000 7fb72f400640 1 --2- 172.21.15.32:0/3530635313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdd07f60 0x55a7be0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:46.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.993+0000 7fb72f400640 1 -- 172.21.15.32:0/3530635313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7bdd73540 con 0x55a7bdcdf0a0 2024-08-31T07:12:46.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.993+0000 7fb72fe00640 1 --2- 172.21.15.32:0/3530635313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdd07f60 0x55a7be0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:46.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.993+0000 7fb72f400640 1 --2- 172.21.15.32:0/3530635313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0adfe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb720000c00 tx=0x7fb720031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67080806e4a42af server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.994+0000 7fb71be00640 1 -- 172.21.15.32:0/3530635313 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720040070 con 0x55a7bdcdf0a0 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.994+0000 7fb71be00640 1 -- 172.21.15.32:0/3530635313 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb720004030 con 0x55a7bdcdf0a0 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.994+0000 7fb71be00640 1 -- 172.21.15.32:0/3530635313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720035040 con 0x55a7bdcdf0a0 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.994+0000 7fb73cc06740 1 -- 172.21.15.32:0/3530635313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 msgr2=0x55a7be0adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.994+0000 7fb73cc06740 1 --2- 172.21.15.32:0/3530635313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0adfe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb720000c00 tx=0x7fb720031b10 comp rx=0 tx=0).stop 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.994+0000 7fb727e00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7140b4c10 0x7fb7140b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.995+0000 7fb73cc06740 1 -- 172.21.15.32:0/3530635313 shutdown_connections 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.995+0000 7fb73cc06740 1 --2- 172.21.15.32:0/3530635313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdce7280 0x55a7be0ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.995+0000 7fb73cc06740 1 --2- 172.21.15.32:0/3530635313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0adfe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.995+0000 7fb73cc06740 1 --2- 172.21.15.32:0/3530635313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdd07f60 0x55a7be0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:46.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.995+0000 7fb73cc06740 1 -- 172.21.15.32:0/3530635313 >> 172.21.15.32:0/3530635313 conn(0x55a7bdcfc350 msgr2=0x55a7be0ab990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:46.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.995+0000 7fb727e00640 1 -- 172.21.15.32:0/2637497338 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb71c0e56d0 con 0x7fb730102ae0 2024-08-31T07:12:46.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.995+0000 7fb72fe00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7140b4c10 0x7fb7140b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:46.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.995+0000 7fb73cc06740 1 -- 172.21.15.32:0/3530635313 shutdown_connections 2024-08-31T07:12:46.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.995+0000 7fb73cc06740 1 -- 172.21.15.32:0/3530635313 wait complete. 2024-08-31T07:12:46.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.996+0000 7fb72fe00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7140b4c10 0x7fb7140b7120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb728009fd0 tx=0x7fb728009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:46.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.996+0000 7fb73cc06740 1 -- start start 2024-08-31T07:12:46.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.996+0000 7fb73cc06740 1 Processor -- start 2024-08-31T07:12:47.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.996+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0c5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb73cc06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a7bde765f0 con 0x55a7bdcdf0a0 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb73cc06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a7bdf9abd0 con 0x55a7bdce7280 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb73cc06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a7be0bc220 con 0x55a7bdd07f60 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb73cc06740 10 client.0 _ll_register_callbacks cb 0x55a7bdfc0210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb72f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb72ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb72f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57340/0 (socket says 172.21.15.32:57340) 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb72f400640 1 -- 172.21.15.32:0/2025699283 learned_addr learned my addr 172.21.15.32:0/2025699283 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb72fe00640 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb72ea00640 1 -- 172.21.15.32:0/2025699283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 msgr2=0x55a7be0c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb72ea00640 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0c5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb72ea00640 1 -- 172.21.15.32:0/2025699283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 msgr2=0x55a7be0b92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb72ea00640 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.997+0000 7fb72ea00640 1 -- 172.21.15.32:0/2025699283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7bdfdb650 con 0x55a7bdce7280 2024-08-31T07:12:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.998+0000 7fb72ea00640 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0b97f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0e2fe0 tx=0x7fb71c0107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9853365e9ce27a9b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.998+0000 7fb718c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.998+0000 7fb718c00640 1 -- 172.21.15.32:0/2025699283 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0b0770 con 0x55a7bdce7280 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.998+0000 7fb718c00640 1 -- 172.21.15.32:0/2025699283 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb71c0b03a0 con 0x55a7bdce7280 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.998+0000 7fb73cc06740 1 -- 172.21.15.32:0/2025699283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb720004030 con 0x55a7bdce7280 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.998+0000 7fb718c00640 1 -- 172.21.15.32:0/2025699283 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb71c0b5220 con 0x55a7bdce7280 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.998+0000 7fb73cc06740 10 client.25514 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.998+0000 7fb73cc06740 1 -- 172.21.15.32:0/2025699283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7be0c8080 con 0x55a7bdce7280 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.998+0000 7fb718c00640 1 -- 172.21.15.32:0/2025699283 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb720004030 con 0x55a7bdce7280 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.999+0000 7fb718c00640 1 -- 172.21.15.32:0/2025699283 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb71c097190 con 0x55a7bdce7280 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.999+0000 7fb73cc06740 10 client.25514 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.999+0000 7fb73cc06740 10 client.25514 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.999+0000 7fb73cc06740 20 client.25514 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.999+0000 7fb73cc06740 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7bdcc68e0 0x55a7be0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.999+0000 7fb73cc06740 4 client.25514 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.999+0000 7fb73cc06740 1 -- 172.21.15.32:0/2025699283 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7be0264a0 con 0x55a7bdcc68e0 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:46.999+0000 7fb72f400640 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7bdcc68e0 0x55a7be0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.000+0000 7fb72f400640 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7bdcc68e0 0x55a7be0cba50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ddea53d60ea673a0 server_cookie=d636e43f82ade513 in_seq=0 out_seq=0 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.000+0000 7fb718c00640 10 client.25514 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb718c00640 1 -- 172.21.15.32:0/2025699283 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7be0264a0 con 0x55a7bdcc68e0 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb718c00640 10 client.25514 handle_command_reply: tid=0 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 client.25514 shutdown 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 -- 172.21.15.32:0/2025699283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 msgr2=0x55a7be0b97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0b97f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0e2fe0 tx=0x7fb71c0107e0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 -- 172.21.15.32:0/2025699283 shutdown_connections 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7bdcc68e0 0x55a7be0cba50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0b97f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 --2- 172.21.15.32:0/2025699283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb718c00640 0 client.25514 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:12:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 -- 172.21.15.32:0/2025699283 >> 172.21.15.32:0/2025699283 conn(0x55a7bdcfc350 msgr2=0x55a7be0b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 -- 172.21.15.32:0/2025699283 shutdown_connections 2024-08-31T07:12:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.001+0000 7fb73cc06740 1 -- 172.21.15.32:0/2025699283 wait complete. 2024-08-31T07:12:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.002+0000 7fb73cc06740 20 client.25514 trim_cache size 0 max 16384 2024-08-31T07:12:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.020+0000 7fb73cc06740 1 Processor -- start 2024-08-31T07:12:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb73cc06740 1 -- start start 2024-08-31T07:12:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb73cc06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a7be0c9d90 con 0x55a7bdcdf0a0 2024-08-31T07:12:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb73cc06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a7be0ca460 con 0x55a7bdce7280 2024-08-31T07:12:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb73cc06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a7be0cab30 con 0x55a7bdd07f60 2024-08-31T07:12:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb72f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb72f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57344/0 (socket says 172.21.15.32:57344) 2024-08-31T07:12:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb72f400640 1 -- 172.21.15.32:0/1767353883 learned_addr learned my addr 172.21.15.32:0/1767353883 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb72ea00640 1 --2- 172.21.15.32:0/1767353883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.021+0000 7fb72fe00640 1 --2- 172.21.15.32:0/1767353883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.022+0000 7fb72f400640 1 -- 172.21.15.32:0/1767353883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 msgr2=0x55a7be0b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.022+0000 7fb72f400640 1 --2- 172.21.15.32:0/1767353883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.022+0000 7fb72f400640 1 -- 172.21.15.32:0/1767353883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 msgr2=0x55a7be0c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.022+0000 7fb72f400640 1 --2- 172.21.15.32:0/1767353883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.022+0000 7fb72f400640 1 -- 172.21.15.32:0/1767353883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7be0b4b40 con 0x55a7bdcdf0a0 2024-08-31T07:12:47.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.022+0000 7fb72fe00640 1 --2- 172.21.15.32:0/1767353883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.022+0000 7fb72f400640 1 --2- 172.21.15.32:0/1767353883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b12b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb7200051c0 tx=0x7fb720005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1436931d450ef56a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.022+0000 7fb71b400640 1 -- 172.21.15.32:0/1767353883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720040070 con 0x55a7bdcdf0a0 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.022+0000 7fb71b400640 1 -- 172.21.15.32:0/1767353883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb720004770 con 0x55a7bdcdf0a0 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.022+0000 7fb71b400640 1 -- 172.21.15.32:0/1767353883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720035040 con 0x55a7bdcdf0a0 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.023+0000 7fb73cc06740 1 -- 172.21.15.32:0/1767353883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 msgr2=0x55a7be0b12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.023+0000 7fb73cc06740 1 --2- 172.21.15.32:0/1767353883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b12b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb7200051c0 tx=0x7fb720005150 comp rx=0 tx=0).stop 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.023+0000 7fb73cc06740 1 -- 172.21.15.32:0/1767353883 shutdown_connections 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.023+0000 7fb73cc06740 1 --2- 172.21.15.32:0/1767353883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.023+0000 7fb73cc06740 1 --2- 172.21.15.32:0/1767353883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.023+0000 7fb73cc06740 1 --2- 172.21.15.32:0/1767353883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0b12b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.023+0000 7fb73cc06740 1 -- 172.21.15.32:0/1767353883 >> 172.21.15.32:0/1767353883 conn(0x55a7bdcfc350 msgr2=0x55a7be0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.023+0000 7fb73cc06740 1 -- 172.21.15.32:0/1767353883 shutdown_connections 2024-08-31T07:12:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.023+0000 7fb73cc06740 1 -- 172.21.15.32:0/1767353883 wait complete. 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.024+0000 7fb73cc06740 1 -- start start 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.024+0000 7fb73cc06740 1 Processor -- start 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb73cc06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb73cc06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a7be0cb8d0 con 0x55a7bdcdf0a0 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb73cc06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a7be0cb200 con 0x55a7bdce7280 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb73cc06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a7be0cbfa0 con 0x55a7bdd07f60 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb73cc06740 10 client.0 _ll_register_callbacks cb 0x7fb71c0ab720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb72ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb72f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb72ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38044/0 (socket says 172.21.15.32:38044) 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb72ea00640 1 -- 172.21.15.32:0/1472416614 learned_addr learned my addr 172.21.15.32:0/1472416614 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb72fe00640 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb72ea00640 1 -- 172.21.15.32:0/1472416614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 msgr2=0x55a7be0d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.025+0000 7fb72ea00640 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb72ea00640 1 -- 172.21.15.32:0/1472416614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 msgr2=0x55a7be0cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb72ea00640 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb72ea00640 1 -- 172.21.15.32:0/1472416614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7be135f20 con 0x55a7bdce7280 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb72f400640 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb72ea00640 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0d0230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0abcb0 tx=0x7fb71c0a7fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=174288674508bdee server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb718c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb718c00640 1 -- 172.21.15.32:0/1472416614 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0ab900 con 0x55a7bdce7280 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb718c00640 1 -- 172.21.15.32:0/1472416614 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb71c0ab420 con 0x55a7bdce7280 2024-08-31T07:12:47.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb73cc06740 1 -- 172.21.15.32:0/1472416614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb720004770 con 0x55a7bdce7280 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb718c00640 1 -- 172.21.15.32:0/1472416614 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb71c0b0770 con 0x55a7bdce7280 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb73cc06740 10 client.25520 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.026+0000 7fb73cc06740 1 -- 172.21.15.32:0/1472416614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7be1371a0 con 0x55a7bdce7280 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.027+0000 7fb718c00640 1 -- 172.21.15.32:0/1472416614 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb720004770 con 0x55a7bdce7280 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.027+0000 7fb718c00640 1 -- 172.21.15.32:0/1472416614 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb71c0cd290 con 0x55a7bdce7280 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.027+0000 7fb73cc06740 10 client.25520 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.027+0000 7fb73cc06740 10 client.25520 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.027+0000 7fb73cc06740 20 client.25520 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.027+0000 7fb73cc06740 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7bdc9bc90 0x55a7be13aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.027+0000 7fb73cc06740 4 client.25520 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.027+0000 7fb73cc06740 1 -- 172.21.15.32:0/1472416614 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "status"}) -- 0x55a7be021f40 con 0x55a7bdc9bc90 2024-08-31T07:12:47.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.027+0000 7fb72f400640 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7bdc9bc90 0x55a7be13aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:47.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb72f400640 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7bdc9bc90 0x55a7be13aa30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c30b80b3c43266f1 server_cookie=74b99300b3dfc144 in_seq=0 out_seq=0 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb718c00640 10 client.25520 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb718c00640 1 -- 172.21.15.32:0/1472416614 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a7be021f40 con 0x55a7bdc9bc90 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb718c00640 10 client.25520 handle_command_reply: tid=0 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb73cc06740 1 client.25520 shutdown 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb73cc06740 1 -- 172.21.15.32:0/1472416614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 msgr2=0x55a7be0d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb73cc06740 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0d0230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0abcb0 tx=0x7fb71c0a7fa0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb73cc06740 1 -- 172.21.15.32:0/1472416614 shutdown_connections 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb73cc06740 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7bdc9bc90 0x55a7be13aa30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb73cc06740 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7bdd07f60 0x55a7be0d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb73cc06740 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a7bdce7280 0x55a7be0d0230 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb718c00640 0 client.25520 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb73cc06740 1 --2- 172.21.15.32:0/1472416614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7bdcdf0a0 0x55a7be0cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.028+0000 7fb73cc06740 1 -- 172.21.15.32:0/1472416614 >> 172.21.15.32:0/1472416614 conn(0x55a7bdcfc350 msgr2=0x55a7be0b97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb73cc06740 1 -- 172.21.15.32:0/1472416614 shutdown_connections 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb73cc06740 1 -- 172.21.15.32:0/1472416614 wait complete. 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb73cc06740 20 client.25520 trim_cache size 0 max 16384 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 3, 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24692, 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:12:47.033 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 165.173071282, 2024-08-31T07:12:47.034 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 29, 2024-08-31T07:12:47.034 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:12:47.034 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 80, 2024-08-31T07:12:47.034 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 204.46408509299999 2024-08-31T07:12:47.034 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:47.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb734a00640 1 -- 172.21.15.32:0/2637497338 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7140b4c10 msgr2=0x7fb7140b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:47.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb734a00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7140b4c10 0x7fb7140b7120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb728009fd0 tx=0x7fb728009500 comp rx=0 tx=0).stop 2024-08-31T07:12:47.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb734a00640 1 -- 172.21.15.32:0/2637497338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730102ae0 msgr2=0x7fb73019c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb734a00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730102ae0 0x7fb73019c7c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00e9f0 tx=0x7fb71c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb734a00640 1 -- 172.21.15.32:0/2637497338 shutdown_connections 2024-08-31T07:12:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb734a00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7140b4c10 0x7fb7140b7120 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb734a00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73010fd40 0x7fb7301a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb734a00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730102ae0 0x7fb73019c7c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb734a00640 1 --2- 172.21.15.32:0/2637497338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb730102190 0x7fb73019c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.029+0000 7fb734a00640 1 -- 172.21.15.32:0/2637497338 >> 172.21.15.32:0/2637497338 conn(0x7fb7300fe070 msgr2=0x7fb730111f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.030+0000 7fb734a00640 1 -- 172.21.15.32:0/2637497338 shutdown_connections 2024-08-31T07:12:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:47.030+0000 7fb734a00640 1 -- 172.21.15.32:0/2637497338 wait complete. 2024-08-31T07:12:48.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.i status 2024-08-31T07:12:48.120 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:47 smithi032 ceph-mon[17252]: pgmap v369: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-08-31T07:12:48.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:47 smithi073 ceph-mon[21995]: pgmap v369: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-08-31T07:12:48.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:47 smithi084 ceph-mon[22387]: pgmap v369: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-08-31T07:12:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.088+0000 7f19a2a00640 1 -- 172.21.15.32:0/3114066467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c1050a0 msgr2=0x7f199c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.088+0000 7f19a2a00640 1 --2- 172.21.15.32:0/3114066467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c1050a0 0x7f199c10d5f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1988009900 tx=0x7f198802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.088+0000 7f19a2a00640 1 -- 172.21.15.32:0/3114066467 shutdown_connections 2024-08-31T07:12:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.088+0000 7f19a2a00640 1 --2- 172.21.15.32:0/3114066467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f199c10dbb0 0x7f199c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.088+0000 7f19a2a00640 1 --2- 172.21.15.32:0/3114066467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c1050a0 0x7f199c10d5f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.088+0000 7f19a2a00640 1 --2- 172.21.15.32:0/3114066467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f199c104750 0x7f199c104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.088+0000 7f19a2a00640 1 -- 172.21.15.32:0/3114066467 >> 172.21.15.32:0/3114066467 conn(0x7f199c079500 msgr2=0x7f199c079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.089+0000 7f19a2a00640 1 -- 172.21.15.32:0/3114066467 shutdown_connections 2024-08-31T07:12:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.089+0000 7f19a2a00640 1 -- 172.21.15.32:0/3114066467 wait complete. 2024-08-31T07:12:49.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.090+0000 7f19a2a00640 1 Processor -- start 2024-08-31T07:12:49.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.090+0000 7f19a2a00640 1 -- start start 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f19a2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f199c104750 0x7f199c102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f19a2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c1050a0 0x7f199c1029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f19a2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f199c10dbb0 0x7f199c102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f19a2a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f199c113840 con 0x7f199c1050a0 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f19a2a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1136c0 con 0x7f199c10dbb0 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f19a2a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1139c0 con 0x7f199c104750 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f199b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c1050a0 0x7f199c1029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f199be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f199c104750 0x7f199c102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f199be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f199c104750 0x7f199c102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55758/0 (socket says 172.21.15.32:55758) 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f199be00640 1 -- 172.21.15.32:0/95774881 learned_addr learned my addr 172.21.15.32:0/95774881 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f19a0a00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f199c10dbb0 0x7f199c102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f199be00640 1 -- 172.21.15.32:0/95774881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f199c10dbb0 msgr2=0x7f199c102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f199be00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f199c10dbb0 0x7f199c102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f199be00640 1 -- 172.21.15.32:0/95774881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c1050a0 msgr2=0x7f199c1029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f199be00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c1050a0 0x7f199c1029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.091+0000 7f199be00640 1 -- 172.21.15.32:0/95774881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c1a8530 con 0x7f199c104750 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.092+0000 7f19a0a00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f199c10dbb0 0x7f199c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.092+0000 7f199b400640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c1050a0 0x7f199c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.092+0000 7f199be00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f199c104750 0x7f199c102490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e9f0 tx=0x7f198c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.092+0000 7f1998c00640 1 -- 172.21.15.32:0/95774881 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c00cd80 con 0x7f199c104750 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.092+0000 7f1998c00640 1 -- 172.21.15.32:0/95774881 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f198c0054d0 con 0x7f199c104750 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.092+0000 7f19a2a00640 1 -- 172.21.15.32:0/95774881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199c1a8820 con 0x7f199c104750 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.092+0000 7f1998c00640 1 -- 172.21.15.32:0/95774881 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c010690 con 0x7f199c104750 2024-08-31T07:12:49.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.092+0000 7f19a2a00640 1 -- 172.21.15.32:0/95774881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199c1a8d30 con 0x7f199c104750 2024-08-31T07:12:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.095+0000 7f19a966e740 1 Processor -- start 2024-08-31T07:12:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f19a966e740 1 -- start start 2024-08-31T07:12:49.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f19a966e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f1998c00640 1 -- 172.21.15.32:0/95774881 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f198c004e90 con 0x7f199c104750 2024-08-31T07:12:49.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f19a966e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 0x557a2ac60e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f19a966e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 0x557a2ac61370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f19a966e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557a2abd7550 con 0x557a2a89a280 2024-08-31T07:12:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f19a966e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a2abc6ae0 con 0x557a2a8baf60 2024-08-31T07:12:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f19a966e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557a2abcaa90 con 0x557a2a8920a0 2024-08-31T07:12:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f199be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 0x557a2ac60e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f199be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 0x557a2ac60e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55770/0 (socket says 172.21.15.32:55770) 2024-08-31T07:12:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f199b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 0x557a2ac61370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f19a0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.096+0000 7f199be00640 1 -- 172.21.15.32:0/598126470 learned_addr learned my addr 172.21.15.32:0/598126470 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:49.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.097+0000 7f19a0a00640 1 -- 172.21.15.32:0/598126470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 msgr2=0x557a2ac60e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:49.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.097+0000 7f19a0a00640 1 --2- 172.21.15.32:0/598126470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 0x557a2ac60e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.097+0000 7f19a0a00640 1 -- 172.21.15.32:0/598126470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 msgr2=0x557a2ac61370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:49.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.097+0000 7f19a0a00640 1 --2- 172.21.15.32:0/598126470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 0x557a2ac61370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.097+0000 7f19a0a00640 1 -- 172.21.15.32:0/598126470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2a926540 con 0x557a2a8baf60 2024-08-31T07:12:49.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.097+0000 7f19a0a00640 1 --2- 172.21.15.32:0/598126470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac608f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1994009f90 tx=0x7f1994010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85715745126ddd5e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:49.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.098+0000 7f1991200640 1 -- 172.21.15.32:0/598126470 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994004d20 con 0x557a2a8baf60 2024-08-31T07:12:49.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.098+0000 7f1991200640 1 -- 172.21.15.32:0/598126470 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19940119b0 con 0x557a2a8baf60 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.098+0000 7f1991200640 1 -- 172.21.15.32:0/598126470 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19940054f0 con 0x557a2a8baf60 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.098+0000 7f1998c00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19800b4c90 0x7f19800b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.098+0000 7f1998c00640 1 -- 172.21.15.32:0/95774881 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f198c0e5830 con 0x7f199c104750 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.098+0000 7f199be00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19800b4c90 0x7f19800b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f19a966e740 1 -- 172.21.15.32:0/598126470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 msgr2=0x557a2ac608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f19a966e740 1 --2- 172.21.15.32:0/598126470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac608f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1994009f90 tx=0x7f1994010860 comp rx=0 tx=0).stop 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f19a966e740 1 -- 172.21.15.32:0/598126470 shutdown_connections 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f19a966e740 1 --2- 172.21.15.32:0/598126470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 0x557a2ac61370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f19a966e740 1 --2- 172.21.15.32:0/598126470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 0x557a2ac60e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f19a966e740 1 --2- 172.21.15.32:0/598126470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac608f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f19a966e740 1 -- 172.21.15.32:0/598126470 >> 172.21.15.32:0/598126470 conn(0x557a2a8af350 msgr2=0x557a2ac5e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f19a966e740 1 -- 172.21.15.32:0/598126470 shutdown_connections 2024-08-31T07:12:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f19a966e740 1 -- 172.21.15.32:0/598126470 wait complete. 2024-08-31T07:12:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f199be00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19800b4c90 0x7f19800b71a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f198c0abd60 tx=0x7f198c0eb550 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.099+0000 7f19a966e740 1 -- start start 2024-08-31T07:12:49.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.100+0000 7f19a966e740 1 Processor -- start 2024-08-31T07:12:49.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.100+0000 7f19a966e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8920a0 0x557a2ac6c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.100+0000 7f19a966e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a89a280 0x557a2ac6c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.100+0000 7f19a966e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.100+0000 7f19a966e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557a2aa295f0 con 0x557a2a8920a0 2024-08-31T07:12:49.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.100+0000 7f19a966e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a2ab4dbd0 con 0x557a2a8baf60 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.100+0000 7f19a966e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557a2ac6f0d0 con 0x557a2a89a280 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.100+0000 7f19a966e740 10 client.0 _ll_register_callbacks cb 0x557a2ab73210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.101+0000 7f19a0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8920a0 0x557a2ac6c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.101+0000 7f19a0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8920a0 0x557a2ac6c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57392/0 (socket says 172.21.15.32:57392) 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.101+0000 7f19a0a00640 1 -- 172.21.15.32:0/3744455673 learned_addr learned my addr 172.21.15.32:0/3744455673 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.101+0000 7f199b400640 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a89a280 0x557a2ac6c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.101+0000 7f19a0a00640 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.101+0000 7f19a0a00640 1 -- 172.21.15.32:0/3744455673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a89a280 msgr2=0x557a2ac6c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.101+0000 7f19a0a00640 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a89a280 0x557a2ac6c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.101+0000 7f19a0a00640 1 -- 172.21.15.32:0/3744455673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 msgr2=0x557a2ac783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.101+0000 7f19a0a00640 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.101+0000 7f19a0a00640 1 -- 172.21.15.32:0/3744455673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2ab8e650 con 0x557a2a8920a0 2024-08-31T07:12:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.102+0000 7f19a0a00640 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8920a0 0x557a2ac6c080 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1994005170 tx=0x7f1994013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb27548959a42d1f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.102+0000 7f1985600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.102+0000 7f1985600640 1 -- 172.21.15.32:0/3744455673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199401f070 con 0x557a2a8920a0 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.102+0000 7f19a966e740 1 -- 172.21.15.32:0/3744455673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19940119b0 con 0x557a2a8920a0 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.102+0000 7f1985600640 1 -- 172.21.15.32:0/3744455673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f19940052d0 con 0x557a2a8920a0 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.102+0000 7f1985600640 1 -- 172.21.15.32:0/3744455673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1994033050 con 0x557a2a8920a0 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.102+0000 7f19a966e740 10 client.15522 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.102+0000 7f19a966e740 1 -- 172.21.15.32:0/3744455673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a2ac7aeb0 con 0x557a2a8920a0 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.103+0000 7f1985600640 1 -- 172.21.15.32:0/3744455673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19940119b0 con 0x557a2a8920a0 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.103+0000 7f1985600640 1 -- 172.21.15.32:0/3744455673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f199401ba30 con 0x557a2a8920a0 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.103+0000 7f19a966e740 10 client.15522 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.103+0000 7f19a966e740 10 client.15522 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.103+0000 7f19a966e740 20 client.15522 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.103+0000 7f19a966e740 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557a2a8798e0 0x557a2ac7e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.103+0000 7f19a966e740 4 client.15522 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.103+0000 7f19a966e740 1 -- 172.21.15.32:0/3744455673 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a2abd94a0 con 0x557a2a8798e0 2024-08-31T07:12:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.103+0000 7f199b400640 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557a2a8798e0 0x557a2ac7e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.104+0000 7f199b400640 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557a2a8798e0 0x557a2ac7e4c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9d45606df6a456c server_cookie=120689400e1fc0a4 in_seq=0 out_seq=0 2024-08-31T07:12:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.104+0000 7f1985600640 10 client.15522 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:12:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f1985600640 1 -- 172.21.15.32:0/3744455673 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a2abd94a0 con 0x557a2a8798e0 2024-08-31T07:12:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f1985600640 10 client.15522 handle_command_reply: tid=0 2024-08-31T07:12:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f19a966e740 1 client.15522 shutdown 2024-08-31T07:12:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f19a966e740 1 -- 172.21.15.32:0/3744455673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8920a0 msgr2=0x557a2ac6c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f19a966e740 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8920a0 0x557a2ac6c080 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1994005170 tx=0x7f1994013040 comp rx=0 tx=0).stop 2024-08-31T07:12:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f19a966e740 1 -- 172.21.15.32:0/3744455673 shutdown_connections 2024-08-31T07:12:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f19a966e740 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557a2a8798e0 0x557a2ac7e4c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f19a966e740 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f19a966e740 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a89a280 0x557a2ac6c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f19a966e740 1 --2- 172.21.15.32:0/3744455673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8920a0 0x557a2ac6c080 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f19a966e740 1 -- 172.21.15.32:0/3744455673 >> 172.21.15.32:0/3744455673 conn(0x557a2a8af350 msgr2=0x557a2ac63ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.105+0000 7f1985600640 0 client.15522 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:12:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.106+0000 7f19a966e740 1 -- 172.21.15.32:0/3744455673 shutdown_connections 2024-08-31T07:12:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.106+0000 7f19a966e740 1 -- 172.21.15.32:0/3744455673 wait complete. 2024-08-31T07:12:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.106+0000 7f19a966e740 20 client.15522 trim_cache size 0 max 16384 2024-08-31T07:12:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.125+0000 7f19a966e740 1 Processor -- start 2024-08-31T07:12:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.125+0000 7f19a966e740 1 -- start start 2024-08-31T07:12:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f19a966e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 0x557a2ac7ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f19a966e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 0x557a2ac594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f19a966e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac6a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f19a966e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557a2ac7c800 con 0x557a2a89a280 2024-08-31T07:12:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f19a966e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a2ac7ced0 con 0x557a2a8baf60 2024-08-31T07:12:49.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f19a966e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557a2ac7d5a0 con 0x557a2a8920a0 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f199b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 0x557a2ac594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f19a0a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 0x557a2ac7ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f19a0a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 0x557a2ac7ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55792/0 (socket says 172.21.15.32:55792) 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f19a0a00640 1 -- 172.21.15.32:0/4097624603 learned_addr learned my addr 172.21.15.32:0/4097624603 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f199b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 0x557a2ac594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57404/0 (socket says 172.21.15.32:57404) 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f19a0a00640 1 --2- 172.21.15.32:0/4097624603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac6a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f199b400640 1 -- 172.21.15.32:0/4097624603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 msgr2=0x557a2ac7ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f199b400640 1 --2- 172.21.15.32:0/4097624603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 0x557a2ac7ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f199b400640 1 -- 172.21.15.32:0/4097624603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 msgr2=0x557a2ac6a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f199b400640 1 --2- 172.21.15.32:0/4097624603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac6a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.126+0000 7f199b400640 1 -- 172.21.15.32:0/4097624603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2ac63750 con 0x557a2a89a280 2024-08-31T07:12:49.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.127+0000 7f199b400640 1 --2- 172.21.15.32:0/4097624603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 0x557a2ac594c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1988009230 tx=0x7f1988005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7c04181bf32f7c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.127+0000 7f1987e00640 1 -- 172.21.15.32:0/4097624603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988035040 con 0x557a2a89a280 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.127+0000 7f1987e00640 1 -- 172.21.15.32:0/4097624603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1988004770 con 0x557a2a89a280 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.127+0000 7f1987e00640 1 -- 172.21.15.32:0/4097624603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19880038c0 con 0x557a2a89a280 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.128+0000 7f19a966e740 1 -- 172.21.15.32:0/4097624603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 msgr2=0x557a2ac594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.128+0000 7f19a966e740 1 --2- 172.21.15.32:0/4097624603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 0x557a2ac594c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1988009230 tx=0x7f1988005ee0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.128+0000 7f19a966e740 1 -- 172.21.15.32:0/4097624603 shutdown_connections 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.128+0000 7f19a966e740 1 --2- 172.21.15.32:0/4097624603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8baf60 0x557a2ac6a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.128+0000 7f19a966e740 1 --2- 172.21.15.32:0/4097624603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a89a280 0x557a2ac594c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.128+0000 7f19a966e740 1 --2- 172.21.15.32:0/4097624603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a8920a0 0x557a2ac7ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.128+0000 7f19a966e740 1 -- 172.21.15.32:0/4097624603 >> 172.21.15.32:0/4097624603 conn(0x557a2a8af350 msgr2=0x557a2ac78540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.128+0000 7f19a966e740 1 -- 172.21.15.32:0/4097624603 shutdown_connections 2024-08-31T07:12:49.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.128+0000 7f19a966e740 1 -- 172.21.15.32:0/4097624603 wait complete. 2024-08-31T07:12:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.128+0000 7f19a966e740 1 -- start start 2024-08-31T07:12:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.129+0000 7f19a966e740 1 Processor -- start 2024-08-31T07:12:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.129+0000 7f19a966e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8920a0 0x557a2ac86bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.129+0000 7f19a966e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a89a280 0x557a2ac87190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.129+0000 7f19a966e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8baf60 0x557a2ac82280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.129+0000 7f19a966e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557a2ac7e340 con 0x557a2a8baf60 2024-08-31T07:12:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.129+0000 7f19a966e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a2ac7dc70 con 0x557a2a8920a0 2024-08-31T07:12:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.129+0000 7f19a966e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557a2ac7ea10 con 0x557a2a89a280 2024-08-31T07:12:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.129+0000 7f19a966e740 10 client.0 _ll_register_callbacks cb 0x7f19940118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f19a0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8baf60 0x557a2ac82280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f19a0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8baf60 0x557a2ac82280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57414/0 (socket says 172.21.15.32:57414) 2024-08-31T07:12:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f19a0a00640 1 -- 172.21.15.32:0/251887273 learned_addr learned my addr 172.21.15.32:0/251887273 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f19a0a00640 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8920a0 0x557a2ac86bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f199b400640 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a89a280 0x557a2ac87190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f19a0a00640 1 -- 172.21.15.32:0/251887273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a89a280 msgr2=0x557a2ac87190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f19a0a00640 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a89a280 0x557a2ac87190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f19a0a00640 1 -- 172.21.15.32:0/251887273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8920a0 msgr2=0x557a2ac86bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f19a0a00640 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8920a0 0x557a2ac86bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f19a0a00640 1 -- 172.21.15.32:0/251887273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2ac82a10 con 0x557a2a8baf60 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f19a0a00640 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8baf60 0x557a2ac82280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1994009a60 tx=0x7f199401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311275e2a1a9b09b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.130+0000 7f1985600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.131+0000 7f1985600640 1 -- 172.21.15.32:0/251887273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199401f070 con 0x557a2a8baf60 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.131+0000 7f1985600640 1 -- 172.21.15.32:0/251887273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f199400e940 con 0x557a2a8baf60 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.131+0000 7f19a966e740 1 -- 172.21.15.32:0/251887273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1988004770 con 0x557a2a8baf60 2024-08-31T07:12:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.131+0000 7f1985600640 1 -- 172.21.15.32:0/251887273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f199404b050 con 0x557a2a8baf60 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.131+0000 7f19a966e740 10 client.15528 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.131+0000 7f19a966e740 1 -- 172.21.15.32:0/251887273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a2ace9550 con 0x557a2a8baf60 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.131+0000 7f1985600640 1 -- 172.21.15.32:0/251887273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1988004770 con 0x557a2a8baf60 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.131+0000 7f1985600640 1 -- 172.21.15.32:0/251887273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f19940055b0 con 0x557a2a8baf60 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.131+0000 7f19a966e740 10 client.15528 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.132+0000 7f19a966e740 10 client.15528 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.132+0000 7f19a966e740 20 client.15528 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.132+0000 7f19a966e740 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557a2a84ec90 0x557a2acecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.132+0000 7f19a966e740 4 client.15528 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.132+0000 7f19a966e740 1 -- 172.21.15.32:0/251887273 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "status"}) -- 0x557a2abd4f40 con 0x557a2a84ec90 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.132+0000 7f199b400640 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557a2a84ec90 0x557a2acecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f199b400640 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557a2a84ec90 0x557a2acecd10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7af9c388db4366fd server_cookie=c7c12c3397787ec6 in_seq=0 out_seq=0 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f1985600640 10 client.15528 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:12:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f1985600640 1 -- 172.21.15.32:0/251887273 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557a2abd4f40 con 0x557a2a84ec90 2024-08-31T07:12:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f1985600640 10 client.15528 handle_command_reply: tid=0 2024-08-31T07:12:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 client.15528 shutdown 2024-08-31T07:12:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 -- 172.21.15.32:0/251887273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8baf60 msgr2=0x557a2ac82280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8baf60 0x557a2ac82280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1994009a60 tx=0x7f199401d040 comp rx=0 tx=0).stop 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 -- 172.21.15.32:0/251887273 shutdown_connections 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557a2a84ec90 0x557a2acecd10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a2a8baf60 0x557a2ac82280 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a2a89a280 0x557a2ac87190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 --2- 172.21.15.32:0/251887273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a2a8920a0 0x557a2ac86bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 -- 172.21.15.32:0/251887273 >> 172.21.15.32:0/251887273 conn(0x557a2a8af350 msgr2=0x557a2ac594c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f1985600640 0 client.15528 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 -- 172.21.15.32:0/251887273 shutdown_connections 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 1 -- 172.21.15.32:0/251887273 wait complete. 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.133+0000 7f19a966e740 20 client.15528 trim_cache size 0 max 16384 2024-08-31T07:12:49.137 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 2, 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24694, 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 168.28377197899999, 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 29, 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 80, 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 204.04147976300001 2024-08-31T07:12:49.138 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 -- 172.21.15.32:0/95774881 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19800b4c90 msgr2=0x7f19800b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19800b4c90 0x7f19800b71a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f198c0abd60 tx=0x7f198c0eb550 comp rx=0 tx=0).stop 2024-08-31T07:12:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 -- 172.21.15.32:0/95774881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f199c104750 msgr2=0x7f199c102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f199c104750 0x7f199c102490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e9f0 tx=0x7f198c00edf0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 -- 172.21.15.32:0/95774881 shutdown_connections 2024-08-31T07:12:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f19800b4c90 0x7f19800b71a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f199c10dbb0 0x7f199c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f199c1050a0 0x7f199c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 --2- 172.21.15.32:0/95774881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f199c104750 0x7f199c102490 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 -- 172.21.15.32:0/95774881 >> 172.21.15.32:0/95774881 conn(0x7f199c079500 msgr2=0x7f199c1076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 -- 172.21.15.32:0/95774881 shutdown_connections 2024-08-31T07:12:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:49.134+0000 7f19a2a00640 1 -- 172.21.15.32:0/95774881 wait complete. 2024-08-31T07:12:50.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:49 smithi032 ceph-mon[17252]: pgmap v370: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:12:50.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.a status 2024-08-31T07:12:50.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:49 smithi073 ceph-mon[21995]: pgmap v370: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:12:50.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:49 smithi084 ceph-mon[22387]: pgmap v370: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:12:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.219+0000 7ff60a600640 1 -- 172.21.15.32:0/1878933232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff604078030 msgr2=0x7ff604113de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.219+0000 7ff60a600640 1 --2- 172.21.15.32:0/1878933232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff604078030 0x7ff604113de0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009900 tx=0x7ff5f002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.219+0000 7ff60a600640 1 -- 172.21.15.32:0/1878933232 shutdown_connections 2024-08-31T07:12:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.219+0000 7ff60a600640 1 --2- 172.21.15.32:0/1878933232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff604114320 0x7ff604116740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.219+0000 7ff60a600640 1 --2- 172.21.15.32:0/1878933232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff604078030 0x7ff604113de0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.219+0000 7ff60a600640 1 --2- 172.21.15.32:0/1878933232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6040776e0 0x7ff604077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.219+0000 7ff60a600640 1 -- 172.21.15.32:0/1878933232 >> 172.21.15.32:0/1878933232 conn(0x7ff604101410 msgr2=0x7ff604103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.219+0000 7ff60a600640 1 -- 172.21.15.32:0/1878933232 shutdown_connections 2024-08-31T07:12:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.219+0000 7ff60a600640 1 -- 172.21.15.32:0/1878933232 wait complete. 2024-08-31T07:12:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.220+0000 7ff60a600640 1 Processor -- start 2024-08-31T07:12:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.221+0000 7ff60a600640 1 -- start start 2024-08-31T07:12:51.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.221+0000 7ff60a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6040776e0 0x7ff6041a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.221+0000 7ff60a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff604078030 0x7ff6041a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.221+0000 7ff60a600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604114320 0x7ff6041a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.221+0000 7ff60a600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff604119cd0 con 0x7ff604078030 2024-08-31T07:12:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.221+0000 7ff60a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff604119b50 con 0x7ff6040776e0 2024-08-31T07:12:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.221+0000 7ff60a600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff604119e50 con 0x7ff604114320 2024-08-31T07:12:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff609200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6040776e0 0x7ff6041a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff603e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff604078030 0x7ff6041a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff609200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6040776e0 0x7ff6041a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38136/0 (socket says 172.21.15.32:38136) 2024-08-31T07:12:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff609200640 1 -- 172.21.15.32:0/1368171448 learned_addr learned my addr 172.21.15.32:0/1368171448 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff609c00640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604114320 0x7ff6041a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff609200640 1 -- 172.21.15.32:0/1368171448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604114320 msgr2=0x7ff6041a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff609200640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604114320 0x7ff6041a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff609200640 1 -- 172.21.15.32:0/1368171448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff604078030 msgr2=0x7ff6041a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff609200640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff604078030 0x7ff6041a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff609200640 1 -- 172.21.15.32:0/1368171448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6041a6610 con 0x7ff6040776e0 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.222+0000 7ff609c00640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604114320 0x7ff6041a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.223+0000 7ff609200640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6040776e0 0x7ff6041a06a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800e9f0 tx=0x7ff5f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.223+0000 7ff601600640 1 -- 172.21.15.32:0/1368171448 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800cd80 con 0x7ff6040776e0 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.223+0000 7ff60a600640 1 -- 172.21.15.32:0/1368171448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6041a6900 con 0x7ff6040776e0 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.223+0000 7ff601600640 1 -- 172.21.15.32:0/1368171448 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f80108e0 con 0x7ff6040776e0 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.223+0000 7ff601600640 1 -- 172.21.15.32:0/1368171448 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8018980 con 0x7ff6040776e0 2024-08-31T07:12:51.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.223+0000 7ff60a600640 1 -- 172.21.15.32:0/1368171448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6041ad060 con 0x7ff6040776e0 2024-08-31T07:12:51.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.226+0000 7ff612870740 1 Processor -- start 2024-08-31T07:12:51.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.226+0000 7ff612870740 1 -- start start 2024-08-31T07:12:51.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff612870740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddca1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff612870740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8d30a0 0x55f7ddca1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff612870740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8db280 0x55f7ddca2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff612870740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddc18550 con 0x55f7dd8fbf60 2024-08-31T07:12:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff612870740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddc07ae0 con 0x55f7dd8d30a0 2024-08-31T07:12:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff612870740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddc0ba90 con 0x55f7dd8db280 2024-08-31T07:12:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff609c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddca1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff609c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddca1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57440/0 (socket says 172.21.15.32:57440) 2024-08-31T07:12:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff609c00640 1 -- 172.21.15.32:0/3347672730 learned_addr learned my addr 172.21.15.32:0/3347672730 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:51.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff603e00640 1 --2- 172.21.15.32:0/3347672730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8db280 0x55f7ddca2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff601600640 1 -- 172.21.15.32:0/1368171448 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7ff5f8010430 con 0x7ff6040776e0 2024-08-31T07:12:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff603e00640 1 -- 172.21.15.32:0/3347672730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8d30a0 msgr2=0x55f7ddca1fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:12:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff603e00640 1 --2- 172.21.15.32:0/3347672730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8d30a0 0x55f7ddca1fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff603e00640 1 -- 172.21.15.32:0/3347672730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 msgr2=0x55f7ddca1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff603e00640 1 --2- 172.21.15.32:0/3347672730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddca1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.227+0000 7ff603e00640 1 -- 172.21.15.32:0/3347672730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7dd967540 con 0x55f7dd8db280 2024-08-31T07:12:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.228+0000 7ff603e00640 1 --2- 172.21.15.32:0/3347672730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8db280 0x55f7ddca2520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009230 tx=0x7ff5f0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69da8aea6e43dc1c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.228+0000 7ff5f6000640 1 -- 172.21.15.32:0/3347672730 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0035040 con 0x55f7dd8db280 2024-08-31T07:12:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.228+0000 7ff5f6000640 1 -- 172.21.15.32:0/3347672730 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f00054f0 con 0x55f7dd8db280 2024-08-31T07:12:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.228+0000 7ff5f6000640 1 -- 172.21.15.32:0/3347672730 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00035f0 con 0x55f7dd8db280 2024-08-31T07:12:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.229+0000 7ff601600640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5ec0b4c90 0x7ff5ec0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.229+0000 7ff612870740 1 -- 172.21.15.32:0/3347672730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8db280 msgr2=0x55f7ddca2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.229+0000 7ff612870740 1 --2- 172.21.15.32:0/3347672730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8db280 0x55f7ddca2520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009230 tx=0x7ff5f0004040 comp rx=0 tx=0).stop 2024-08-31T07:12:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.229+0000 7ff601600640 1 -- 172.21.15.32:0/1368171448 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff5f80e5650 con 0x7ff6040776e0 2024-08-31T07:12:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.229+0000 7ff612870740 1 -- 172.21.15.32:0/3347672730 shutdown_connections 2024-08-31T07:12:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.229+0000 7ff612870740 1 --2- 172.21.15.32:0/3347672730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8db280 0x55f7ddca2520 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.229+0000 7ff612870740 1 --2- 172.21.15.32:0/3347672730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8d30a0 0x55f7ddca1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.229+0000 7ff612870740 1 --2- 172.21.15.32:0/3347672730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddca1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.229+0000 7ff612870740 1 -- 172.21.15.32:0/3347672730 >> 172.21.15.32:0/3347672730 conn(0x55f7dd8f0350 msgr2=0x55f7ddc9f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:51.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.229+0000 7ff609c00640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5ec0b4c90 0x7ff5ec0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.230+0000 7ff612870740 1 -- 172.21.15.32:0/3347672730 shutdown_connections 2024-08-31T07:12:51.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.230+0000 7ff612870740 1 -- 172.21.15.32:0/3347672730 wait complete. 2024-08-31T07:12:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.230+0000 7ff609c00640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5ec0b4c90 0x7ff5ec0b71a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009fd0 tx=0x7ff5fc009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.230+0000 7ff612870740 1 -- start start 2024-08-31T07:12:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.231+0000 7ff612870740 1 Processor -- start 2024-08-31T07:12:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.231+0000 7ff612870740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 0x55f7ddcad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.231+0000 7ff612870740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8db280 0x55f7ddcada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.231+0000 7ff612870740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcb9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.231+0000 7ff612870740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f7dda6a5f0 con 0x55f7dd8db280 2024-08-31T07:12:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.231+0000 7ff612870740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddb8ebd0 con 0x55f7dd8fbf60 2024-08-31T07:12:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.231+0000 7ff612870740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddcb0220 con 0x55f7dd8d30a0 2024-08-31T07:12:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.231+0000 7ff612870740 10 client.0 _ll_register_callbacks cb 0x55f7ddbb4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff609c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcb9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff609c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcb9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38142/0 (socket says 172.21.15.32:38142) 2024-08-31T07:12:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff609c00640 1 -- 172.21.15.32:0/2671396283 learned_addr learned my addr 172.21.15.32:0/2671396283 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff609200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 0x55f7ddcad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff603e00640 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8db280 0x55f7ddcada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff609c00640 1 -- 172.21.15.32:0/2671396283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 msgr2=0x55f7ddcad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff609c00640 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 0x55f7ddcad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff609c00640 1 -- 172.21.15.32:0/2671396283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8db280 msgr2=0x55f7ddcada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff609c00640 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8db280 0x55f7ddcada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff609c00640 1 -- 172.21.15.32:0/2671396283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7ddbcf650 con 0x55f7dd8fbf60 2024-08-31T07:12:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff609c00640 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcb9510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc003d40 tx=0x7ff5fc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7161c0bacbe3239a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.232+0000 7ff5eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:12:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.233+0000 7ff5eaa00640 1 -- 172.21.15.32:0/2671396283 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc02c060 con 0x55f7dd8fbf60 2024-08-31T07:12:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.233+0000 7ff5eaa00640 1 -- 172.21.15.32:0/2671396283 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff5fc017030 con 0x55f7dd8fbf60 2024-08-31T07:12:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.233+0000 7ff612870740 1 -- 172.21.15.32:0/2671396283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f00054f0 con 0x55f7dd8fbf60 2024-08-31T07:12:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.233+0000 7ff5eaa00640 1 -- 172.21.15.32:0/2671396283 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5fc01f810 con 0x55f7dd8fbf60 2024-08-31T07:12:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.233+0000 7ff612870740 10 client.25547 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.233+0000 7ff612870740 1 -- 172.21.15.32:0/2671396283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7ddcbc170 con 0x55f7dd8fbf60 2024-08-31T07:12:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.233+0000 7ff5eaa00640 1 -- 172.21.15.32:0/2671396283 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f00054f0 con 0x55f7dd8fbf60 2024-08-31T07:12:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.234+0000 7ff5eaa00640 1 -- 172.21.15.32:0/2671396283 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff5fc031bb0 con 0x55f7dd8fbf60 2024-08-31T07:12:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.234+0000 7ff612870740 10 client.25547 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.234+0000 7ff612870740 10 client.25547 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.234+0000 7ff612870740 20 client.25547 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.234+0000 7ff612870740 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55f7dd8ba8e0 0x55f7ddcbfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.234+0000 7ff612870740 4 client.25547 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.234+0000 7ff612870740 1 -- 172.21.15.32:0/2671396283 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7ddc1a4a0 con 0x55f7dd8ba8e0 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.234+0000 7ff609200640 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55f7dd8ba8e0 0x55f7ddcbfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.235+0000 7ff609200640 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55f7dd8ba8e0 0x55f7ddcbfc00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ec9693ab713665a server_cookie=52a97eaca0d23ec0 in_seq=0 out_seq=0 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.235+0000 7ff5eaa00640 10 client.25547 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff5eaa00640 1 -- 172.21.15.32:0/2671396283 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f7ddc1a4a0 con 0x55f7dd8ba8e0 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff5eaa00640 10 client.25547 handle_command_reply: tid=0 2024-08-31T07:12:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 client.25547 shutdown 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 -- 172.21.15.32:0/2671396283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8fbf60 msgr2=0x55f7ddcb9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcb9510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc003d40 tx=0x7ff5fc004ab0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 -- 172.21.15.32:0/2671396283 shutdown_connections 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55f7dd8ba8e0 0x55f7ddcbfc00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcb9510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8db280 0x55f7ddcada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 --2- 172.21.15.32:0/2671396283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 0x55f7ddcad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff5eaa00640 0 client.25547 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 -- 172.21.15.32:0/2671396283 >> 172.21.15.32:0/2671396283 conn(0x55f7dd8f0350 msgr2=0x55f7ddca4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 -- 172.21.15.32:0/2671396283 shutdown_connections 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 1 -- 172.21.15.32:0/2671396283 wait complete. 2024-08-31T07:12:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.236+0000 7ff612870740 20 client.25547 trim_cache size 0 max 16384 2024-08-31T07:12:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.255+0000 7ff612870740 1 Processor -- start 2024-08-31T07:12:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.255+0000 7ff612870740 1 -- start start 2024-08-31T07:12:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff612870740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8d30a0 0x55f7ddca8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff612870740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 0x55f7ddca89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff612870740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcabbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff612870740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddcbdf40 con 0x55f7dd8d30a0 2024-08-31T07:12:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff612870740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddcbe610 con 0x55f7dd8db280 2024-08-31T07:12:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff612870740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddcbece0 con 0x55f7dd8fbf60 2024-08-31T07:12:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff609200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8d30a0 0x55f7ddca8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff609200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8d30a0 0x55f7ddca8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57452/0 (socket says 172.21.15.32:57452) 2024-08-31T07:12:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff609200640 1 -- 172.21.15.32:0/1322030718 learned_addr learned my addr 172.21.15.32:0/1322030718 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff603e00640 1 --2- 172.21.15.32:0/1322030718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 0x55f7ddca89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.256+0000 7ff609c00640 1 --2- 172.21.15.32:0/1322030718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcabbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.257+0000 7ff609200640 1 -- 172.21.15.32:0/1322030718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8fbf60 msgr2=0x55f7ddcabbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:51.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.257+0000 7ff609200640 1 --2- 172.21.15.32:0/1322030718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcabbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.257+0000 7ff609200640 1 -- 172.21.15.32:0/1322030718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 msgr2=0x55f7ddca89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:51.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.257+0000 7ff609200640 1 --2- 172.21.15.32:0/1322030718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 0x55f7ddca89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.257+0000 7ff609200640 1 -- 172.21.15.32:0/1322030718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7ddca28b0 con 0x55f7dd8d30a0 2024-08-31T07:12:51.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.257+0000 7ff609200640 1 --2- 172.21.15.32:0/1322030718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8d30a0 0x55f7ddca8480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c6a0 tx=0x7ff5f800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2c7032f644f588 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:51.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.257+0000 7ff5f5600640 1 -- 172.21.15.32:0/1322030718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80e31b0 con 0x55f7dd8d30a0 2024-08-31T07:12:51.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.257+0000 7ff5f5600640 1 -- 172.21.15.32:0/1322030718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f8010a90 con 0x55f7dd8d30a0 2024-08-31T07:12:51.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.257+0000 7ff5f5600640 1 -- 172.21.15.32:0/1322030718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8018d80 con 0x55f7dd8d30a0 2024-08-31T07:12:51.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.258+0000 7ff612870740 1 -- 172.21.15.32:0/1322030718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8d30a0 msgr2=0x55f7ddca8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.258+0000 7ff612870740 1 --2- 172.21.15.32:0/1322030718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8d30a0 0x55f7ddca8480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c6a0 tx=0x7ff5f800cf20 comp rx=0 tx=0).stop 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.258+0000 7ff612870740 1 -- 172.21.15.32:0/1322030718 shutdown_connections 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.258+0000 7ff612870740 1 --2- 172.21.15.32:0/1322030718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcabbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.258+0000 7ff612870740 1 --2- 172.21.15.32:0/1322030718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 0x55f7ddca89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.258+0000 7ff612870740 1 --2- 172.21.15.32:0/1322030718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8d30a0 0x55f7ddca8480 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.258+0000 7ff612870740 1 -- 172.21.15.32:0/1322030718 >> 172.21.15.32:0/1322030718 conn(0x55f7dd8f0350 msgr2=0x55f7ddcbd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.258+0000 7ff612870740 1 -- 172.21.15.32:0/1322030718 shutdown_connections 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.258+0000 7ff612870740 1 -- 172.21.15.32:0/1322030718 wait complete. 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.259+0000 7ff612870740 1 -- start start 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.259+0000 7ff612870740 1 Processor -- start 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff612870740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 0x55f7ddcc3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff612870740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 0x55f7ddcc4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff612870740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcc4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff612870740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddcbfa80 con 0x55f7dd8fbf60 2024-08-31T07:12:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff612870740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddcbf3b0 con 0x55f7dd8db280 2024-08-31T07:12:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff612870740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddcc0150 con 0x55f7dd8d30a0 2024-08-31T07:12:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff612870740 10 client.0 _ll_register_callbacks cb 0x7ff5fc011620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff609c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcc4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff609200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 0x55f7ddcc3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff603e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 0x55f7ddcc4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff609c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcc4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57462/0 (socket says 172.21.15.32:57462) 2024-08-31T07:12:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff609c00640 1 -- 172.21.15.32:0/4175020059 learned_addr learned my addr 172.21.15.32:0/4175020059 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:51.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff609200640 1 -- 172.21.15.32:0/4175020059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 msgr2=0x55f7ddcc4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:51.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff609200640 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 0x55f7ddcc4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff609200640 1 -- 172.21.15.32:0/4175020059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 msgr2=0x55f7ddcc4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:51.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff609200640 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcc4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff609200640 1 -- 172.21.15.32:0/4175020059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7ddd2a150 con 0x55f7dd8d30a0 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.260+0000 7ff603e00640 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 0x55f7ddcc4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.261+0000 7ff609c00640 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcc4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.261+0000 7ff609200640 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 0x55f7ddcc3f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c6a0 tx=0x7ff5f80abe30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a42f448b25bfa6c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.261+0000 7ff5eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.261+0000 7ff5eaa00640 1 -- 172.21.15.32:0/4175020059 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80c6060 con 0x55f7dd8d30a0 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.261+0000 7ff612870740 1 -- 172.21.15.32:0/4175020059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f8010a90 con 0x55f7dd8d30a0 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.261+0000 7ff5eaa00640 1 -- 172.21.15.32:0/4175020059 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff5f8097210 con 0x55f7dd8d30a0 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.261+0000 7ff5eaa00640 1 -- 172.21.15.32:0/4175020059 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5f80cd050 con 0x55f7dd8d30a0 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.261+0000 7ff612870740 10 client.25561 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.261+0000 7ff612870740 1 -- 172.21.15.32:0/4175020059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7ddd2b3b0 con 0x55f7dd8d30a0 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.262+0000 7ff5eaa00640 1 -- 172.21.15.32:0/4175020059 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f8010a90 con 0x55f7dd8d30a0 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.262+0000 7ff5eaa00640 1 -- 172.21.15.32:0/4175020059 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff5f80cd1f0 con 0x55f7dd8d30a0 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.262+0000 7ff612870740 10 client.25561 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.262+0000 7ff612870740 10 client.25561 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.262+0000 7ff612870740 20 client.25561 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.262+0000 7ff612870740 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55f7dd88fc90 0x55f7ddd2e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.262+0000 7ff612870740 4 client.25561 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.262+0000 7ff612870740 1 -- 172.21.15.32:0/4175020059 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "status"}) -- 0x55f7ddc15f40 con 0x55f7dd88fc90 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.262+0000 7ff603e00640 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55f7dd88fc90 0x55f7ddd2e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.263+0000 7ff603e00640 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55f7dd88fc90 0x55f7ddd2e840 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52358da5aaf7fd43 server_cookie=6ffcdcdd441eeba5 in_seq=0 out_seq=0 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.263+0000 7ff5eaa00640 10 client.25561 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.263+0000 7ff5eaa00640 1 -- 172.21.15.32:0/4175020059 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f7ddc15f40 con 0x55f7dd88fc90 2024-08-31T07:12:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.263+0000 7ff5eaa00640 10 client.25561 handle_command_reply: tid=0 2024-08-31T07:12:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.263+0000 7ff612870740 1 client.25561 shutdown 2024-08-31T07:12:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.263+0000 7ff612870740 1 -- 172.21.15.32:0/4175020059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 msgr2=0x55f7ddcc3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.263+0000 7ff612870740 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 0x55f7ddcc3f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c6a0 tx=0x7ff5f80abe30 comp rx=0 tx=0).stop 2024-08-31T07:12:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.264+0000 7ff612870740 1 -- 172.21.15.32:0/4175020059 shutdown_connections 2024-08-31T07:12:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.264+0000 7ff612870740 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f7dd8fbf60 0x55f7ddcc4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.264+0000 7ff612870740 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55f7dd88fc90 0x55f7ddd2e840 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.264+0000 7ff612870740 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f7dd8db280 0x55f7ddcc4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.264+0000 7ff612870740 1 --2- 172.21.15.32:0/4175020059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f7dd8d30a0 0x55f7ddcc3f00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.264+0000 7ff5eaa00640 0 client.25561 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:12:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.264+0000 7ff612870740 1 -- 172.21.15.32:0/4175020059 >> 172.21.15.32:0/4175020059 conn(0x55f7dd8f0350 msgr2=0x55f7ddca8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.264+0000 7ff612870740 1 -- 172.21.15.32:0/4175020059 shutdown_connections 2024-08-31T07:12:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.264+0000 7ff612870740 1 -- 172.21.15.32:0/4175020059 wait complete. 2024-08-31T07:12:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.264+0000 7ff612870740 20 client.25561 trim_cache size 0 max 16384 2024-08-31T07:12:51.269 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 1, 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24718, 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 171.42346247200001, 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 29, 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 80, 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 197.92619384299999 2024-08-31T07:12:51.270 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 -- 172.21.15.32:0/1368171448 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5ec0b4c90 msgr2=0x7ff5ec0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5ec0b4c90 0x7ff5ec0b71a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009fd0 tx=0x7ff5fc009500 comp rx=0 tx=0).stop 2024-08-31T07:12:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 -- 172.21.15.32:0/1368171448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6040776e0 msgr2=0x7ff6041a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6040776e0 0x7ff6041a06a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800e9f0 tx=0x7ff5f800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 -- 172.21.15.32:0/1368171448 shutdown_connections 2024-08-31T07:12:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff5ec0b4c90 0x7ff5ec0b71a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff604114320 0x7ff6041a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff604078030 0x7ff6041a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 --2- 172.21.15.32:0/1368171448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6040776e0 0x7ff6041a06a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 -- 172.21.15.32:0/1368171448 >> 172.21.15.32:0/1368171448 conn(0x7ff604101410 msgr2=0x7ff604116390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 -- 172.21.15.32:0/1368171448 shutdown_connections 2024-08-31T07:12:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:51.265+0000 7ff60a600640 1 -- 172.21.15.32:0/1368171448 wait complete. 2024-08-31T07:12:52.103 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:51 smithi032 ceph-mon[17252]: pgmap v371: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-08-31T07:12:52.142 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.b status 2024-08-31T07:12:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:51 smithi073 ceph-mon[21995]: pgmap v371: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-08-31T07:12:52.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:51 smithi084 ceph-mon[22387]: pgmap v371: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-08-31T07:12:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.178+0000 7f32a3200640 1 -- 172.21.15.32:0/2896822144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4103750 msgr2=0x7f32a4103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.178+0000 7f32a3200640 1 --2- 172.21.15.32:0/2896822144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4103750 0x7f32a4103b60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3294009900 tx=0x7f329402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.178+0000 7f32a3200640 1 -- 172.21.15.32:0/2896822144 shutdown_connections 2024-08-31T07:12:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.178+0000 7f32a3200640 1 --2- 172.21.15.32:0/2896822144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a41099e0 0x7f32a4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.178+0000 7f32a3200640 1 --2- 172.21.15.32:0/2896822144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a4109090 0x7f32a41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.178+0000 7f32a3200640 1 --2- 172.21.15.32:0/2896822144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4103750 0x7f32a4103b60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.178+0000 7f32a3200640 1 -- 172.21.15.32:0/2896822144 >> 172.21.15.32:0/2896822144 conn(0x7f32a40ff1e0 msgr2=0x7f32a4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.178+0000 7f32a3200640 1 -- 172.21.15.32:0/2896822144 shutdown_connections 2024-08-31T07:12:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.179+0000 7f32a3200640 1 -- 172.21.15.32:0/2896822144 wait complete. 2024-08-31T07:12:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.180+0000 7f32a3200640 1 Processor -- start 2024-08-31T07:12:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.180+0000 7f32a3200640 1 -- start start 2024-08-31T07:12:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.180+0000 7f32a3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4103750 0x7f32a419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.180+0000 7f32a3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a4109090 0x7f32a419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.180+0000 7f32a3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a41099e0 0x7f32a41a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.180+0000 7f32a3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41136b0 con 0x7f32a41099e0 2024-08-31T07:12:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.180+0000 7f32a3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32a4113530 con 0x7f32a4103750 2024-08-31T07:12:53.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.180+0000 7f32a3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32a4113830 con 0x7f32a4109090 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4103750 0x7f32a419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a41099e0 0x7f32a41a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4103750 0x7f32a419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38188/0 (socket says 172.21.15.32:38188) 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a1e00640 1 -- 172.21.15.32:0/3448009804 learned_addr learned my addr 172.21.15.32:0/3448009804 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a1400640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a4109090 0x7f32a419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a1e00640 1 -- 172.21.15.32:0/3448009804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a4109090 msgr2=0x7f32a419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a1e00640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a4109090 0x7f32a419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a1e00640 1 -- 172.21.15.32:0/3448009804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a41099e0 msgr2=0x7f32a41a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a1e00640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a41099e0 0x7f32a41a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a1e00640 1 -- 172.21.15.32:0/3448009804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a41a2230 con 0x7f32a4103750 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a2800640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a41099e0 0x7f32a41a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.181+0000 7f32a1e00640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4103750 0x7f32a419c2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3294005940 tx=0x7f32940040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.182+0000 7f329aa00640 1 -- 172.21.15.32:0/3448009804 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329403f030 con 0x7f32a4103750 2024-08-31T07:12:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.182+0000 7f329aa00640 1 -- 172.21.15.32:0/3448009804 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32940044c0 con 0x7f32a4103750 2024-08-31T07:12:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.182+0000 7f329aa00640 1 -- 172.21.15.32:0/3448009804 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294035070 con 0x7f32a4103750 2024-08-31T07:12:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.182+0000 7f32a3200640 1 -- 172.21.15.32:0/3448009804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a41a24c0 con 0x7f32a4103750 2024-08-31T07:12:53.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.182+0000 7f32a3200640 1 -- 172.21.15.32:0/3448009804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a41a8c80 con 0x7f32a4103750 2024-08-31T07:12:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.185+0000 7f32afc91740 1 Processor -- start 2024-08-31T07:12:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.185+0000 7f32afc91740 1 -- start start 2024-08-31T07:12:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.185+0000 7f32afc91740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f600022aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32afc91740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f600022fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32afc91740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f600023520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32afc91740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f5fff99550 con 0x55f5ffc7cf60 2024-08-31T07:12:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32afc91740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f5fff88ae0 con 0x55f5ffc5c280 2024-08-31T07:12:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32afc91740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f5fff8ca90 con 0x55f5ffc540a0 2024-08-31T07:12:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32a2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f600022aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32a2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f600022aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57484/0 (socket says 172.21.15.32:57484) 2024-08-31T07:12:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32a2800640 1 -- 172.21.15.32:0/903338544 learned_addr learned my addr 172.21.15.32:0/903338544 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:53.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32a1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f600023520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32a1400640 1 -- 172.21.15.32:0/903338544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 msgr2=0x55f600022fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f329aa00640 1 -- 172.21.15.32:0/3448009804 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f3294042030 con 0x7f32a4103750 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32a1400640 1 --2- 172.21.15.32:0/903338544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f600022fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32a1400640 1 -- 172.21.15.32:0/903338544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 msgr2=0x55f600022aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32a1400640 1 --2- 172.21.15.32:0/903338544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f600022aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32a1400640 1 -- 172.21.15.32:0/903338544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ffce8540 con 0x55f5ffc5c280 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.186+0000 7f32a2800640 1 --2- 172.21.15.32:0/903338544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f600022aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.187+0000 7f32a1400640 1 --2- 172.21.15.32:0/903338544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f600023520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f32a406b4a0 tx=0x7f3290011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=658a79ef366e3c7c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.187+0000 7f328f400640 1 -- 172.21.15.32:0/903338544 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290009d70 con 0x55f5ffc5c280 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.187+0000 7f328f400640 1 -- 172.21.15.32:0/903338544 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32900198c0 con 0x55f5ffc5c280 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.187+0000 7f328f400640 1 -- 172.21.15.32:0/903338544 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32900187d0 con 0x55f5ffc5c280 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f329aa00640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32880b4c90 0x7f32880b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f32afc91740 1 -- 172.21.15.32:0/903338544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 msgr2=0x55f600023520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:53.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f32afc91740 1 --2- 172.21.15.32:0/903338544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f600023520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f32a406b4a0 tx=0x7f3290011810 comp rx=0 tx=0).stop 2024-08-31T07:12:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f32afc91740 1 -- 172.21.15.32:0/903338544 shutdown_connections 2024-08-31T07:12:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f32afc91740 1 --2- 172.21.15.32:0/903338544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f600023520 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f329aa00640 1 -- 172.21.15.32:0/3448009804 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f32941056b0 con 0x7f32a4103750 2024-08-31T07:12:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f32afc91740 1 --2- 172.21.15.32:0/903338544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f600022fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f32afc91740 1 --2- 172.21.15.32:0/903338544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f600022aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f32afc91740 1 -- 172.21.15.32:0/903338544 >> 172.21.15.32:0/903338544 conn(0x55f5ffc71350 msgr2=0x55f600020990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f32a2800640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32880b4c90 0x7f32880b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f32afc91740 1 -- 172.21.15.32:0/903338544 shutdown_connections 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.188+0000 7f32afc91740 1 -- 172.21.15.32:0/903338544 wait complete. 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.189+0000 7f32afc91740 1 -- start start 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.189+0000 7f32a2800640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32880b4c90 0x7f32880b71a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f329c005fd0 tx=0x7f329c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.189+0000 7f32afc91740 1 Processor -- start 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32afc91740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc540a0 0x55f60002e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32afc91740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc5c280 0x55f60002e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32afc91740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc7cf60 0x55f60003a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32afc91740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f5ffdeb5f0 con 0x55f5ffc540a0 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32afc91740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f5fff0fbd0 con 0x55f5ffc7cf60 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32afc91740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f600031280 con 0x55f5ffc5c280 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32afc91740 10 client.0 _ll_register_callbacks cb 0x55f5fff35210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:53.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32a1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc540a0 0x55f60002e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32a1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc540a0 0x55f60002e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57494/0 (socket says 172.21.15.32:57494) 2024-08-31T07:12:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32a1e00640 1 -- 172.21.15.32:0/2592311048 learned_addr learned my addr 172.21.15.32:0/2592311048 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32a2800640 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc7cf60 0x55f60003a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.190+0000 7f32a1400640 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc5c280 0x55f60002e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.191+0000 7f32a1e00640 1 -- 172.21.15.32:0/2592311048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc5c280 msgr2=0x55f60002e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.191+0000 7f32a1e00640 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc5c280 0x55f60002e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.191+0000 7f32a1e00640 1 -- 172.21.15.32:0/2592311048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc7cf60 msgr2=0x55f60003a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.191+0000 7f32a1e00640 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc7cf60 0x55f60003a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.191+0000 7f32a1e00640 1 -- 172.21.15.32:0/2592311048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5fff50650 con 0x55f5ffc540a0 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.191+0000 7f32a1e00640 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc540a0 0x55f60002e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3294006010 tx=0x7f32940cb7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7cab8a87d4e830 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.191+0000 7f3283e00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.191+0000 7f3283e00640 1 -- 172.21.15.32:0/2592311048 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294049070 con 0x55f5ffc540a0 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.192+0000 7f32afc91740 1 -- 172.21.15.32:0/2592311048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32900198c0 con 0x55f5ffc540a0 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.192+0000 7f3283e00640 1 -- 172.21.15.32:0/2592311048 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f32940d4f80 con 0x55f5ffc540a0 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.192+0000 7f3283e00640 1 -- 172.21.15.32:0/2592311048 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32940d0750 con 0x55f5ffc540a0 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.192+0000 7f32afc91740 10 client.15558 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.192+0000 7f32afc91740 1 -- 172.21.15.32:0/2592311048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f60003d140 con 0x55f5ffc540a0 2024-08-31T07:12:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.192+0000 7f3283e00640 1 -- 172.21.15.32:0/2592311048 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32900198c0 con 0x55f5ffc540a0 2024-08-31T07:12:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.192+0000 7f3283e00640 1 -- 172.21.15.32:0/2592311048 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f32940b30b0 con 0x55f5ffc540a0 2024-08-31T07:12:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.192+0000 7f32afc91740 10 client.15558 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.192+0000 7f32afc91740 10 client.15558 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T07:12:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.193+0000 7f32afc91740 20 client.15558 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.193+0000 7f32afc91740 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f5ffc3b8e0 0x55f600040c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.193+0000 7f32afc91740 4 client.15558 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.193+0000 7f32afc91740 1 -- 172.21.15.32:0/2592311048 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5fff9b4a0 con 0x55f5ffc3b8e0 2024-08-31T07:12:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.193+0000 7f32a1400640 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f5ffc3b8e0 0x55f600040c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.193+0000 7f32a1400640 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f5ffc3b8e0 0x55f600040c50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1beba6642843e4d8 server_cookie=aba96ea5d29bab42 in_seq=0 out_seq=0 2024-08-31T07:12:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.193+0000 7f3283e00640 10 client.15558 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.194+0000 7f3283e00640 1 -- 172.21.15.32:0/2592311048 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5fff9b4a0 con 0x55f5ffc3b8e0 2024-08-31T07:12:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.194+0000 7f3283e00640 10 client.15558 handle_command_reply: tid=0 2024-08-31T07:12:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.194+0000 7f32afc91740 1 client.15558 shutdown 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 1 -- 172.21.15.32:0/2592311048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc540a0 msgr2=0x55f60002e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc540a0 0x55f60002e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3294006010 tx=0x7f32940cb7c0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 1 -- 172.21.15.32:0/2592311048 shutdown_connections 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f5ffc3b8e0 0x55f600040c50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc7cf60 0x55f60003a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc5c280 0x55f60002e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 1 --2- 172.21.15.32:0/2592311048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc540a0 0x55f60002e2f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 1 -- 172.21.15.32:0/2592311048 >> 172.21.15.32:0/2592311048 conn(0x55f5ffc71350 msgr2=0x55f600025c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f3283e00640 0 client.15558 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 1 -- 172.21.15.32:0/2592311048 shutdown_connections 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 1 -- 172.21.15.32:0/2592311048 wait complete. 2024-08-31T07:12:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.195+0000 7f32afc91740 20 client.15558 trim_cache size 0 max 16384 2024-08-31T07:12:53.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.214+0000 7f32afc91740 1 Processor -- start 2024-08-31T07:12:53.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32afc91740 1 -- start start 2024-08-31T07:12:53.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32afc91740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f60001b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32afc91740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f60003a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32afc91740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f60003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32afc91740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f60003ef90 con 0x55f5ffc7cf60 2024-08-31T07:12:53.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32afc91740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f60003f660 con 0x55f5ffc5c280 2024-08-31T07:12:53.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32afc91740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f60003fd30 con 0x55f5ffc540a0 2024-08-31T07:12:53.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32a2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f60003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32a2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f60003aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57498/0 (socket says 172.21.15.32:57498) 2024-08-31T07:12:53.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32a2800640 1 -- 172.21.15.32:0/2919293086 learned_addr learned my addr 172.21.15.32:0/2919293086 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:53.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32a1400640 1 --2- 172.21.15.32:0/2919293086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f60003a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.215+0000 7f32a1e00640 1 --2- 172.21.15.32:0/2919293086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f60001b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.216+0000 7f32a2800640 1 -- 172.21.15.32:0/2919293086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 msgr2=0x55f60001b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:53.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.216+0000 7f32a2800640 1 --2- 172.21.15.32:0/2919293086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f60001b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.216+0000 7f32a2800640 1 -- 172.21.15.32:0/2919293086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 msgr2=0x55f60003a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:53.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.216+0000 7f32a2800640 1 --2- 172.21.15.32:0/2919293086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f60003a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.216+0000 7f32a2800640 1 -- 172.21.15.32:0/2919293086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f60002f5b0 con 0x55f5ffc7cf60 2024-08-31T07:12:53.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.216+0000 7f32a1400640 1 --2- 172.21.15.32:0/2919293086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f60003a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:53.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.216+0000 7f32a2800640 1 --2- 172.21.15.32:0/2919293086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f60003aa80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f329c003cc0 tx=0x7f329c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97c4bd4f8bf5a25 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:53.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.216+0000 7f328ea00640 1 -- 172.21.15.32:0/2919293086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c003db0 con 0x55f5ffc7cf60 2024-08-31T07:12:53.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.216+0000 7f328ea00640 1 -- 172.21.15.32:0/2919293086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f329c00fd90 con 0x55f5ffc7cf60 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.216+0000 7f328ea00640 1 -- 172.21.15.32:0/2919293086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c011700 con 0x55f5ffc7cf60 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.217+0000 7f32afc91740 1 -- 172.21.15.32:0/2919293086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 msgr2=0x55f60003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.217+0000 7f32afc91740 1 --2- 172.21.15.32:0/2919293086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f60003aa80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f329c003cc0 tx=0x7f329c017000 comp rx=0 tx=0).stop 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.217+0000 7f32afc91740 1 -- 172.21.15.32:0/2919293086 shutdown_connections 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.217+0000 7f32afc91740 1 --2- 172.21.15.32:0/2919293086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f60003aa80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.217+0000 7f32afc91740 1 --2- 172.21.15.32:0/2919293086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f60003a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.217+0000 7f32afc91740 1 --2- 172.21.15.32:0/2919293086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f60001b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.217+0000 7f32afc91740 1 -- 172.21.15.32:0/2919293086 >> 172.21.15.32:0/2919293086 conn(0x55f5ffc71350 msgr2=0x55f60002cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.217+0000 7f32afc91740 1 -- 172.21.15.32:0/2919293086 shutdown_connections 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.217+0000 7f32afc91740 1 -- 172.21.15.32:0/2919293086 wait complete. 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.218+0000 7f32afc91740 1 -- start start 2024-08-31T07:12:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.218+0000 7f32afc91740 1 Processor -- start 2024-08-31T07:12:53.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.218+0000 7f32afc91740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f600049840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.218+0000 7f32afc91740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f600042cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.218+0000 7f32afc91740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f600043390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.218+0000 7f32afc91740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f600040ad0 con 0x55f5ffc7cf60 2024-08-31T07:12:53.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.218+0000 7f32afc91740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f600040400 con 0x55f5ffc5c280 2024-08-31T07:12:53.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.218+0000 7f32afc91740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f6000411a0 con 0x55f5ffc540a0 2024-08-31T07:12:53.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.218+0000 7f32afc91740 10 client.0 _ll_register_callbacks cb 0x7f32940cb950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:53.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f600043390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f600049840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f600042cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f600049840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55910/0 (socket says 172.21.15.32:55910) 2024-08-31T07:12:53.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a1e00640 1 -- 172.21.15.32:0/3354255384 learned_addr learned my addr 172.21.15.32:0/3354255384 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:53.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a1e00640 1 -- 172.21.15.32:0/3354255384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 msgr2=0x55f600042cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:53.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a1e00640 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f600042cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a1e00640 1 -- 172.21.15.32:0/3354255384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 msgr2=0x55f600043390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:53.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a1e00640 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f600043390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a1e00640 1 -- 172.21.15.32:0/3354255384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f600043b20 con 0x55f5ffc540a0 2024-08-31T07:12:53.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f32a1e00640 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f600049840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f32940b3500 tx=0x7f32940d0650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa46b721882ae70a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:53.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.219+0000 7f3283e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:12:53.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f3283e00640 1 -- 172.21.15.32:0/3354255384 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294049070 con 0x55f5ffc540a0 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f32afc91740 1 -- 172.21.15.32:0/3354255384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329c00fd90 con 0x55f5ffc540a0 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f3283e00640 1 -- 172.21.15.32:0/3354255384 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f32940b3070 con 0x55f5ffc540a0 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f3283e00640 1 -- 172.21.15.32:0/3354255384 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32940b3270 con 0x55f5ffc540a0 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f32afc91740 10 client.25576 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f32afc91740 1 -- 172.21.15.32:0/3354255384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6000ac1e0 con 0x55f5ffc540a0 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f3283e00640 1 -- 172.21.15.32:0/3354255384 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f329c00fd90 con 0x55f5ffc540a0 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f3283e00640 1 -- 172.21.15.32:0/3354255384 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f32940d3ad0 con 0x55f5ffc540a0 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f32afc91740 10 client.25576 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f32afc91740 10 client.25576 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.220+0000 7f32afc91740 20 client.25576 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.221+0000 7f32afc91740 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f5ffc10c90 0x55f6000af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.221+0000 7f32afc91740 4 client.25576 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T07:12:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.221+0000 7f32afc91740 1 -- 172.21.15.32:0/3354255384 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "status"}) -- 0x55f5fff96f40 con 0x55f5ffc10c90 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.221+0000 7f32a1400640 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f5ffc10c90 0x55f6000af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.221+0000 7f32a1400640 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f5ffc10c90 0x55f6000af8c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=450f76e5e0511c46 server_cookie=fc24680e83c95476 in_seq=0 out_seq=0 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.221+0000 7f3283e00640 10 client.25576 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f3283e00640 1 -- 172.21.15.32:0/3354255384 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f5fff96f40 con 0x55f5ffc10c90 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f3283e00640 10 client.25576 handle_command_reply: tid=0 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 client.25576 shutdown 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 -- 172.21.15.32:0/3354255384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 msgr2=0x55f600049840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f600049840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f32940b3500 tx=0x7f32940d0650 comp rx=0 tx=0).stop 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 -- 172.21.15.32:0/3354255384 shutdown_connections 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f5ffc10c90 0x55f6000af8c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f5ffc7cf60 0x55f600043390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5ffc5c280 0x55f600042cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 --2- 172.21.15.32:0/3354255384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f5ffc540a0 0x55f600049840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f3283e00640 0 client.25576 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:53.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 -- 172.21.15.32:0/3354255384 >> 172.21.15.32:0/3354255384 conn(0x55f5ffc71350 msgr2=0x55f600020210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:53.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 -- 172.21.15.32:0/3354255384 shutdown_connections 2024-08-31T07:12:53.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 1 -- 172.21.15.32:0/3354255384 wait complete. 2024-08-31T07:12:53.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.222+0000 7f32afc91740 20 client.25576 trim_cache size 0 max 16384 2024-08-31T07:12:53.227 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "965fc21a-6766-11ef-bcd4-c7b262605968", 2024-08-31T07:12:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-08-31T07:12:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24734, 2024-08-31T07:12:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-08-31T07:12:53.227 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-08-31T07:12:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-08-31T07:12:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 183.573066246, 2024-08-31T07:12:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 29, 2024-08-31T07:12:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 80, 2024-08-31T07:12:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 80, 2024-08-31T07:12:53.228 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 197.29374108799999 2024-08-31T07:12:53.228 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:53.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 -- 172.21.15.32:0/3448009804 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32880b4c90 msgr2=0x7f32880b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:53.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32880b4c90 0x7f32880b71a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f329c005fd0 tx=0x7f329c009500 comp rx=0 tx=0).stop 2024-08-31T07:12:53.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 -- 172.21.15.32:0/3448009804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4103750 msgr2=0x7f32a419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:53.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4103750 0x7f32a419c2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3294005940 tx=0x7f32940040c0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 -- 172.21.15.32:0/3448009804 shutdown_connections 2024-08-31T07:12:53.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32880b4c90 0x7f32880b71a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a41099e0 0x7f32a41a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32a4109090 0x7f32a419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 --2- 172.21.15.32:0/3448009804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4103750 0x7f32a419c2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:53.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 -- 172.21.15.32:0/3448009804 >> 172.21.15.32:0/3448009804 conn(0x7f32a40ff1e0 msgr2=0x7f32a410e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:53.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 -- 172.21.15.32:0/3448009804 shutdown_connections 2024-08-31T07:12:53.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:53.223+0000 7f32a3200640 1 -- 172.21.15.32:0/3448009804 wait complete. 2024-08-31T07:12:54.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:53 smithi032 ceph-mon[17252]: pgmap v372: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:12:54.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:53 smithi073 ceph-mon[21995]: pgmap v372: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:12:54.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:53 smithi084 ceph-mon[22387]: pgmap v372: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-31T07:12:54.268 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-08-31T07:12:56.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:55 smithi073 ceph-mon[21995]: pgmap v373: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:12:56.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:55 smithi084 ceph-mon[22387]: pgmap v373: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:12:56.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:55 smithi032 ceph-mon[17252]: pgmap v373: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:12:58.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:57 smithi073 ceph-mon[21995]: pgmap v374: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:12:58.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:57 smithi084 ceph-mon[22387]: pgmap v374: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:12:58.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:57 smithi032 ceph-mon[17252]: pgmap v374: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:12:58.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:12:58.831 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:12:59.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.536+0000 7f556d200640 1 -- 172.21.15.32:0/3221348777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5568075ea0 msgr2=0x7f556810f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:59.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.536+0000 7f556d200640 1 --2- 172.21.15.32:0/3221348777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5568075ea0 0x7f556810f770 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5554009900 tx=0x7f555402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.536+0000 7f556d200640 1 -- 172.21.15.32:0/3221348777 shutdown_connections 2024-08-31T07:12:59.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.536+0000 7f556d200640 1 --2- 172.21.15.32:0/3221348777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f556810fcb0 0x7f55681120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.536+0000 7f556d200640 1 --2- 172.21.15.32:0/3221348777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5568075ea0 0x7f556810f770 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.536+0000 7f556d200640 1 --2- 172.21.15.32:0/3221348777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5568075550 0x7f5568075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.536+0000 7f556d200640 1 -- 172.21.15.32:0/3221348777 >> 172.21.15.32:0/3221348777 conn(0x7f55680ff1e0 msgr2=0x7f5568101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.536+0000 7f556d200640 1 -- 172.21.15.32:0/3221348777 shutdown_connections 2024-08-31T07:12:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.536+0000 7f556d200640 1 -- 172.21.15.32:0/3221348777 wait complete. 2024-08-31T07:12:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.538+0000 7f556d200640 1 Processor -- start 2024-08-31T07:12:59.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.538+0000 7f556d200640 1 -- start start 2024-08-31T07:12:59.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.538+0000 7f556d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5568075550 0x7f556819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.538+0000 7f556d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5568075ea0 0x7f556819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f556d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f556810fcb0 0x7f55681a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f556d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55681158b0 con 0x7f5568075550 2024-08-31T07:12:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f556d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5568115730 con 0x7f5568075ea0 2024-08-31T07:12:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f556d200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5568115a30 con 0x7f556810fcb0 2024-08-31T07:12:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f5567400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5568075550 0x7f556819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f5566a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5568075ea0 0x7f556819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f5567400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5568075550 0x7f556819c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50676/0 (socket says 172.21.15.32:50676) 2024-08-31T07:12:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f5567400640 1 -- 172.21.15.32:0/3871741984 learned_addr learned my addr 172.21.15.32:0/3871741984 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:59.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f5567e00640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f556810fcb0 0x7f55681a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f5567400640 1 -- 172.21.15.32:0/3871741984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f556810fcb0 msgr2=0x7f55681a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f5567400640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f556810fcb0 0x7f55681a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f5567400640 1 -- 172.21.15.32:0/3871741984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5568075ea0 msgr2=0x7f556819c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f5567400640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5568075ea0 0x7f556819c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.539+0000 7f5567400640 1 -- 172.21.15.32:0/3871741984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55681a2210 con 0x7f5568075550 2024-08-31T07:12:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.540+0000 7f5567400640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5568075550 0x7f556819c2a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f555800e9f0 tx=0x7f555800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.540+0000 7f555fe00640 1 -- 172.21.15.32:0/3871741984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555800cd80 con 0x7f5568075550 2024-08-31T07:12:59.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.540+0000 7f555fe00640 1 -- 172.21.15.32:0/3871741984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55580108e0 con 0x7f5568075550 2024-08-31T07:12:59.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.540+0000 7f556d200640 1 -- 172.21.15.32:0/3871741984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55681a2500 con 0x7f5568075550 2024-08-31T07:12:59.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.540+0000 7f555fe00640 1 -- 172.21.15.32:0/3871741984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558018980 con 0x7f5568075550 2024-08-31T07:12:59.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.540+0000 7f556d200640 1 -- 172.21.15.32:0/3871741984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a8c60 con 0x7f5568075550 2024-08-31T07:12:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.544+0000 7f5575606740 1 Processor -- start 2024-08-31T07:12:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.544+0000 7f5575606740 1 -- start start 2024-08-31T07:12:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.544+0000 7f555fe00640 1 -- 172.21.15.32:0/3871741984 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f5558018b20 con 0x7f5568075550 2024-08-31T07:12:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.544+0000 7f5575606740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f157070 0x55af1f4fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.544+0000 7f5575606740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f4fd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.544+0000 7f5575606740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 0x55af1f4fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.544+0000 7f5575606740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55af1f473770 con 0x55af1f136390 2024-08-31T07:12:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.544+0000 7f5575606740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55af1f462d00 con 0x55af1f12e1b0 2024-08-31T07:12:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.544+0000 7f5575606740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55af1f466cb0 con 0x55af1f157070 2024-08-31T07:12:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5567400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f4fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5566a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 0x55af1f4fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5567400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f4fd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46686/0 (socket says 172.21.15.32:46686) 2024-08-31T07:12:59.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5567400640 1 -- 172.21.15.32:0/1821308195 learned_addr learned my addr 172.21.15.32:0/1821308195 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:59.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5567e00640 1 --2- 172.21.15.32:0/1821308195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f157070 0x55af1f4fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5566a00640 1 -- 172.21.15.32:0/1821308195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f157070 msgr2=0x55af1f4fcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:59.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5566a00640 1 --2- 172.21.15.32:0/1821308195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f157070 0x55af1f4fcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5566a00640 1 -- 172.21.15.32:0/1821308195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 msgr2=0x55af1f4fd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:59.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5566a00640 1 --2- 172.21.15.32:0/1821308195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f4fd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5566a00640 1 -- 172.21.15.32:0/1821308195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1f1c2650 con 0x55af1f136390 2024-08-31T07:12:59.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.545+0000 7f5567400640 1 --2- 172.21.15.32:0/1821308195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f4fd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:12:59.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.546+0000 7f5566a00640 1 --2- 172.21.15.32:0/1821308195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 0x55af1f4fd5a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f555402a910 tx=0x7f5554004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e388f7fa5c8057cb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.546+0000 7f555cc00640 1 -- 172.21.15.32:0/1821308195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554035040 con 0x55af1f136390 2024-08-31T07:12:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.546+0000 7f555cc00640 1 -- 172.21.15.32:0/1821308195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55540054f0 con 0x55af1f136390 2024-08-31T07:12:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.546+0000 7f555cc00640 1 -- 172.21.15.32:0/1821308195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55540035f0 con 0x55af1f136390 2024-08-31T07:12:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.547+0000 7f555fe00640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f554c0b4c10 0x7f554c0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.547+0000 7f555fe00640 1 -- 172.21.15.32:0/3871741984 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f55580e5730 con 0x7f5568075550 2024-08-31T07:12:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.547+0000 7f5567e00640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f554c0b4c10 0x7f554c0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.547+0000 7f5575606740 1 -- 172.21.15.32:0/1821308195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 msgr2=0x55af1f4fd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:59.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.547+0000 7f5575606740 1 --2- 172.21.15.32:0/1821308195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 0x55af1f4fd5a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f555402a910 tx=0x7f5554004040 comp rx=0 tx=0).stop 2024-08-31T07:12:59.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.548+0000 7f5575606740 1 -- 172.21.15.32:0/1821308195 shutdown_connections 2024-08-31T07:12:59.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.548+0000 7f5575606740 1 --2- 172.21.15.32:0/1821308195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 0x55af1f4fd5a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.548+0000 7f5575606740 1 --2- 172.21.15.32:0/1821308195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f4fd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.548+0000 7f5575606740 1 --2- 172.21.15.32:0/1821308195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f157070 0x55af1f4fcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.548+0000 7f5575606740 1 -- 172.21.15.32:0/1821308195 >> 172.21.15.32:0/1821308195 conn(0x55af1f14b460 msgr2=0x55af1f4faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:59.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.548+0000 7f5575606740 1 -- 172.21.15.32:0/1821308195 shutdown_connections 2024-08-31T07:12:59.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.548+0000 7f5575606740 1 -- 172.21.15.32:0/1821308195 wait complete. 2024-08-31T07:12:59.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.548+0000 7f5567e00640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f554c0b4c10 0x7f554c0b7120 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5560005fd0 tx=0x7f5560009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:59.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.548+0000 7f5575606740 1 -- start start 2024-08-31T07:12:59.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.549+0000 7f5575606740 1 Processor -- start 2024-08-31T07:12:59.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.549+0000 7f5575606740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f508730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.549+0000 7f5575606740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 0x55af1f508cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.549+0000 7f5575606740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 0x55af1f514590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.549+0000 7f5575606740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55af1f2c5700 con 0x55af1f157070 2024-08-31T07:12:59.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.549+0000 7f5575606740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55af1f3e9cb0 con 0x55af1f136390 2024-08-31T07:12:59.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.549+0000 7f5575606740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55af1f50b2a0 con 0x55af1f12e1b0 2024-08-31T07:12:59.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.549+0000 7f5575606740 10 client.0 _ll_register_callbacks cb 0x55af1f40f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:59.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5567400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f508730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5566a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 0x55af1f508cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5567e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 0x55af1f514590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5566a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 0x55af1f508cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46688/0 (socket says 172.21.15.32:46688) 2024-08-31T07:12:59.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5567400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f508730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:56244/0 (socket says 172.21.15.32:56244) 2024-08-31T07:12:59.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5566a00640 1 -- 172.21.15.32:0/355085515 learned_addr learned my addr 172.21.15.32:0/355085515 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:59.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5566a00640 1 -- 172.21.15.32:0/355085515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 msgr2=0x55af1f508730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:59.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5566a00640 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f508730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5566a00640 1 -- 172.21.15.32:0/355085515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 msgr2=0x55af1f514590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:59.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5566a00640 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 0x55af1f514590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.550+0000 7f5566a00640 1 -- 172.21.15.32:0/355085515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1f42a870 con 0x55af1f136390 2024-08-31T07:12:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.551+0000 7f5566a00640 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 0x55af1f508cb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5554031440 tx=0x7f5554004520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a00782fc1a3aa33 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.551+0000 7f5551600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:12:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.551+0000 7f5551600640 1 -- 172.21.15.32:0/355085515 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554035040 con 0x55af1f136390 2024-08-31T07:12:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.551+0000 7f5575606740 1 -- 172.21.15.32:0/355085515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55540054f0 con 0x55af1f136390 2024-08-31T07:12:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.551+0000 7f5551600640 1 -- 172.21.15.32:0/355085515 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5554004d00 con 0x55af1f136390 2024-08-31T07:12:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.551+0000 7f5551600640 1 -- 172.21.15.32:0/355085515 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5554009520 con 0x55af1f136390 2024-08-31T07:12:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.551+0000 7f5575606740 10 client.25583 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.551+0000 7f5575606740 1 -- 172.21.15.32:0/355085515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55af1f517300 con 0x55af1f136390 2024-08-31T07:12:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.552+0000 7f5551600640 1 -- 172.21.15.32:0/355085515 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55540054f0 con 0x55af1f136390 2024-08-31T07:12:59.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.552+0000 7f5551600640 1 -- 172.21.15.32:0/355085515 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f555404c070 con 0x55af1f136390 2024-08-31T07:12:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.552+0000 7f5575606740 10 client.25583 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.552+0000 7f5575606740 10 client.25583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:12:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.552+0000 7f5575606740 20 client.25583 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.552+0000 7f5575606740 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55af1f1159f0 0x55af1f51a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.552+0000 7f5575606740 4 client.25583 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:12:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.552+0000 7f5575606740 1 -- 172.21.15.32:0/355085515 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af1f4756c0 con 0x55af1f1159f0 2024-08-31T07:12:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.552+0000 7f5567400640 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55af1f1159f0 0x55af1f51a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.553+0000 7f5551600640 10 client.25583 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.553+0000 7f5567400640 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55af1f1159f0 0x55af1f51a660 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622e9e8b3c9ca222 server_cookie=930c6cdadd4deaaf in_seq=0 out_seq=0 2024-08-31T07:12:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5551600640 1 -- 172.21.15.32:0/355085515 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af1f4756c0 con 0x55af1f1159f0 2024-08-31T07:12:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5551600640 10 client.25583 handle_command_reply: tid=0 2024-08-31T07:12:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5575606740 1 client.25583 shutdown 2024-08-31T07:12:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5575606740 1 -- 172.21.15.32:0/355085515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 msgr2=0x55af1f508cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5575606740 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 0x55af1f508cb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5554031440 tx=0x7f5554004520 comp rx=0 tx=0).stop 2024-08-31T07:12:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5575606740 1 -- 172.21.15.32:0/355085515 shutdown_connections 2024-08-31T07:12:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5575606740 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55af1f1159f0 0x55af1f51a660 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5575606740 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 0x55af1f514590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5575606740 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 0x55af1f508cb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5575606740 1 --2- 172.21.15.32:0/355085515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f508730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5551600640 0 client.25583 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.554+0000 7f5575606740 1 -- 172.21.15.32:0/355085515 >> 172.21.15.32:0/355085515 conn(0x55af1f14b460 msgr2=0x55af1f4ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.555+0000 7f5575606740 1 -- 172.21.15.32:0/355085515 shutdown_connections 2024-08-31T07:12:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.555+0000 7f5575606740 1 -- 172.21.15.32:0/355085515 wait complete. 2024-08-31T07:12:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.555+0000 7f5575606740 20 client.25583 trim_cache size 0 max 16384 2024-08-31T07:12:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.578+0000 7f5575606740 1 Processor -- start 2024-08-31T07:12:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.578+0000 7f5575606740 1 -- start start 2024-08-31T07:12:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.578+0000 7f5575606740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f506db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.578+0000 7f5575606740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 0x55af1f5072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.578+0000 7f5575606740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 0x55af1f514500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.578+0000 7f5575606740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55681b1390 con 0x55af1f157070 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.578+0000 7f5575606740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55681b2ed0 con 0x55af1f136390 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.578+0000 7f5575606740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f55681b1a60 con 0x55af1f12e1b0 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5567400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f506db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5567400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f506db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:56252/0 (socket says 172.21.15.32:56252) 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5567400640 1 -- 172.21.15.32:0/2030774988 learned_addr learned my addr 172.21.15.32:0/2030774988 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5566a00640 1 --2- 172.21.15.32:0/2030774988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 0x55af1f5072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5567e00640 1 --2- 172.21.15.32:0/2030774988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 0x55af1f514500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5567400640 1 -- 172.21.15.32:0/2030774988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 msgr2=0x55af1f5072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5567400640 1 --2- 172.21.15.32:0/2030774988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 0x55af1f5072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5567400640 1 -- 172.21.15.32:0/2030774988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 msgr2=0x55af1f514500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:59.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5567400640 1 --2- 172.21.15.32:0/2030774988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 0x55af1f514500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5567400640 1 -- 172.21.15.32:0/2030774988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1f4fee10 con 0x55af1f12e1b0 2024-08-31T07:12:59.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.579+0000 7f5567e00640 1 --2- 172.21.15.32:0/2030774988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 0x55af1f514500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:12:59.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.580+0000 7f5567400640 1 --2- 172.21.15.32:0/2030774988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f506db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55580abd10 tx=0x7f55580eb530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e40d2380a512586 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:59.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.580+0000 7f5553e00640 1 -- 172.21.15.32:0/2030774988 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55580ac010 con 0x55af1f12e1b0 2024-08-31T07:12:59.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.580+0000 7f5553e00640 1 -- 172.21.15.32:0/2030774988 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55580b07d0 con 0x55af1f12e1b0 2024-08-31T07:12:59.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.580+0000 7f5553e00640 1 -- 172.21.15.32:0/2030774988 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55580a82a0 con 0x55af1f12e1b0 2024-08-31T07:12:59.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.581+0000 7f5575606740 1 -- 172.21.15.32:0/2030774988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 msgr2=0x55af1f506db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:59.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.581+0000 7f5575606740 1 --2- 172.21.15.32:0/2030774988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f506db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55580abd10 tx=0x7f55580eb530 comp rx=0 tx=0).stop 2024-08-31T07:12:59.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.581+0000 7f5575606740 1 -- 172.21.15.32:0/2030774988 shutdown_connections 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.581+0000 7f5575606740 1 --2- 172.21.15.32:0/2030774988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f157070 0x55af1f514500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.581+0000 7f5575606740 1 --2- 172.21.15.32:0/2030774988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f136390 0x55af1f5072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.581+0000 7f5575606740 1 --2- 172.21.15.32:0/2030774988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f12e1b0 0x55af1f506db0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.581+0000 7f5575606740 1 -- 172.21.15.32:0/2030774988 >> 172.21.15.32:0/2030774988 conn(0x55af1f14b460 msgr2=0x55af1f503670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.581+0000 7f5575606740 1 -- 172.21.15.32:0/2030774988 shutdown_connections 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.581+0000 7f5575606740 1 -- 172.21.15.32:0/2030774988 wait complete. 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.581+0000 7f5575606740 1 -- start start 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.582+0000 7f5575606740 1 Processor -- start 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.582+0000 7f5575606740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f519640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.582+0000 7f5575606740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 0x55af1f519b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.582+0000 7f5575606740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f157070 0x55af1f51a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.582+0000 7f5575606740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f55681b2800 con 0x55af1f136390 2024-08-31T07:12:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.582+0000 7f5575606740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55681b2130 con 0x55af1f12e1b0 2024-08-31T07:12:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.582+0000 7f5575606740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f55681af370 con 0x55af1f157070 2024-08-31T07:12:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.582+0000 7f5575606740 10 client.0 _ll_register_callbacks cb 0x7f5554009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:12:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5566a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 0x55af1f519b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5567400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f519640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5567400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f519640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46704/0 (socket says 172.21.15.32:46704) 2024-08-31T07:12:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5567400640 1 -- 172.21.15.32:0/1673480124 learned_addr learned my addr 172.21.15.32:0/1673480124 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:12:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5567e00640 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f157070 0x55af1f51a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5566a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 0x55af1f519b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50712/0 (socket says 172.21.15.32:50712) 2024-08-31T07:12:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5567400640 1 -- 172.21.15.32:0/1673480124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f157070 msgr2=0x55af1f51a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5567400640 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f157070 0x55af1f51a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5567400640 1 -- 172.21.15.32:0/1673480124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 msgr2=0x55af1f519b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5567400640 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 0x55af1f519b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5567400640 1 -- 172.21.15.32:0/1673480124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1f583dd0 con 0x55af1f12e1b0 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5567400640 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f519640 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f55580abfa0 tx=0x7f5558018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=412d242b708fbee1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.583+0000 7f5551600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.584+0000 7f5551600640 1 -- 172.21.15.32:0/1673480124 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55580ac010 con 0x55af1f12e1b0 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.584+0000 7f5551600640 1 -- 172.21.15.32:0/1673480124 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f55580a8600 con 0x55af1f12e1b0 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.584+0000 7f5575606740 1 -- 172.21.15.32:0/1673480124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55580b07d0 con 0x55af1f12e1b0 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.584+0000 7f5551600640 1 -- 172.21.15.32:0/1673480124 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55580a82a0 con 0x55af1f12e1b0 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.584+0000 7f5575606740 10 client.25589 fetch_fsmap learned FSMap version 29 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.584+0000 7f5575606740 1 -- 172.21.15.32:0/1673480124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55af1f585030 con 0x55af1f12e1b0 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.584+0000 7f5551600640 1 -- 172.21.15.32:0/1673480124 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55580b07d0 con 0x55af1f12e1b0 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.585+0000 7f5551600640 1 -- 172.21.15.32:0/1673480124 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f55580ab270 con 0x55af1f12e1b0 2024-08-31T07:12:59.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.585+0000 7f5575606740 10 client.25589 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.585+0000 7f5575606740 10 client.25589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.585+0000 7f5575606740 20 client.25589 populate_metadata read hostname 'smithi032' 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.585+0000 7f5575606740 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55af1f0eada0 0x55af1f587730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.585+0000 7f5575606740 4 client.25589 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.585+0000 7f5575606740 1 -- 172.21.15.32:0/1673480124 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af1f471160 con 0x55af1f0eada0 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.585+0000 7f5566a00640 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55af1f0eada0 0x55af1f587730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5551600640 10 client.25589 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5566a00640 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55af1f0eada0 0x55af1f587730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14eeef295173b59f server_cookie=8d108de23f660591 in_seq=0 out_seq=0 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5551600640 1 -- 172.21.15.32:0/1673480124 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55af1f471160 con 0x55af1f0eada0 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5551600640 10 client.25589 handle_command_reply: tid=0 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5575606740 1 client.25589 shutdown 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5575606740 1 -- 172.21.15.32:0/1673480124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 msgr2=0x55af1f519640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5575606740 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f519640 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f55580abfa0 tx=0x7f5558018420 comp rx=0 tx=0).stop 2024-08-31T07:12:59.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5575606740 1 -- 172.21.15.32:0/1673480124 shutdown_connections 2024-08-31T07:12:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5575606740 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55af1f0eada0 0x55af1f587730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5575606740 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55af1f157070 0x55af1f51a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5575606740 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55af1f136390 0x55af1f519b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5575606740 1 --2- 172.21.15.32:0/1673480124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af1f12e1b0 0x55af1f519640 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5551600640 0 client.25589 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:12:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.586+0000 7f5575606740 1 -- 172.21.15.32:0/1673480124 >> 172.21.15.32:0/1673480124 conn(0x55af1f14b460 msgr2=0x55af1f503670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.587+0000 7f5575606740 1 -- 172.21.15.32:0/1673480124 shutdown_connections 2024-08-31T07:12:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.587+0000 7f5575606740 1 -- 172.21.15.32:0/1673480124 wait complete. 2024-08-31T07:12:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.587+0000 7f5575606740 20 client.25589 trim_cache size 0 max 16384 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:12:59.591 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:12:59.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:12:59.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:12:59.592 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:12:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.587+0000 7f556d200640 1 -- 172.21.15.32:0/3871741984 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f554c0b4c10 msgr2=0x7f554c0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.587+0000 7f556d200640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f554c0b4c10 0x7f554c0b7120 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5560005fd0 tx=0x7f5560009500 comp rx=0 tx=0).stop 2024-08-31T07:12:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.587+0000 7f556d200640 1 -- 172.21.15.32:0/3871741984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5568075550 msgr2=0x7f556819c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:12:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.587+0000 7f556d200640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5568075550 0x7f556819c2a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f555800e9f0 tx=0x7f555800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.588+0000 7f556d200640 1 -- 172.21.15.32:0/3871741984 shutdown_connections 2024-08-31T07:12:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.588+0000 7f556d200640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f554c0b4c10 0x7f554c0b7120 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.588+0000 7f556d200640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f556810fcb0 0x7f55681a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.588+0000 7f556d200640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5568075ea0 0x7f556819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.588+0000 7f556d200640 1 --2- 172.21.15.32:0/3871741984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5568075550 0x7f556819c2a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:12:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.588+0000 7f556d200640 1 -- 172.21.15.32:0/3871741984 >> 172.21.15.32:0/3871741984 conn(0x7f55680ff1e0 msgr2=0x7f5568109830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:12:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.588+0000 7f556d200640 1 -- 172.21.15.32:0/3871741984 shutdown_connections 2024-08-31T07:12:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:12:59.588+0000 7f556d200640 1 -- 172.21.15.32:0/3871741984 wait complete. 2024-08-31T07:13:00.173 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:12:59 smithi032 ceph-mon[17252]: pgmap v375: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-08-31T07:13:00.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:12:59 smithi073 ceph-mon[21995]: pgmap v375: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-08-31T07:13:00.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:12:59 smithi084 ceph-mon[22387]: pgmap v375: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-08-31T07:13:00.579 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:13:00.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:13:00.840 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:13:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:13:00.516+0000 7f5f0d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T07:13:00.840 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:13:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:13:00.516+0000 7f5f0d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T07:13:00.840 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:13:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:13:00.516+0000 7f5f0d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T07:13:00.840 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:13:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:13:00.516+0000 7f5f0d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T07:13:00.840 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:13:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:13:00.516+0000 7f5f0d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T07:13:00.999 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.855+0000 7fe159800640 1 -- 172.21.15.32:0/2534458245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe154077120 msgr2=0x7fe154075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.855+0000 7fe159800640 1 --2- 172.21.15.32:0/2534458245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe154077120 0x7fe154075550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe144009900 tx=0x7fe14402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:13:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.855+0000 7fe159800640 1 -- 172.21.15.32:0/2534458245 shutdown_connections 2024-08-31T07:13:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.855+0000 7fe159800640 1 --2- 172.21.15.32:0/2534458245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe154114020 0x7fe1541164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.855+0000 7fe159800640 1 --2- 172.21.15.32:0/2534458245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe154075b70 0x7fe154075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.855+0000 7fe159800640 1 --2- 172.21.15.32:0/2534458245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe154077120 0x7fe154075550 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.855+0000 7fe159800640 1 -- 172.21.15.32:0/2534458245 >> 172.21.15.32:0/2534458245 conn(0x7fe1540ff1e0 msgr2=0x7fe154101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.855+0000 7fe159800640 1 -- 172.21.15.32:0/2534458245 shutdown_connections 2024-08-31T07:13:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.855+0000 7fe159800640 1 -- 172.21.15.32:0/2534458245 wait complete. 2024-08-31T07:13:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.857+0000 7fe159800640 1 Processor -- start 2024-08-31T07:13:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.857+0000 7fe159800640 1 -- start start 2024-08-31T07:13:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe159800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe154075b70 0x7fe1541a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe159800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe154077120 0x7fe1541a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe159800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe154114020 0x7fe1541a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe159800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe154119cb0 con 0x7fe154075b70 2024-08-31T07:13:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe159800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe154119b30 con 0x7fe154077120 2024-08-31T07:13:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe159800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe154119e30 con 0x7fe154114020 2024-08-31T07:13:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe153400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe154077120 0x7fe1541a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe153e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe154075b70 0x7fe1541a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe153400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe154077120 0x7fe1541a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46734/0 (socket says 172.21.15.32:46734) 2024-08-31T07:13:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe153400640 1 -- 172.21.15.32:0/1007177624 learned_addr learned my addr 172.21.15.32:0/1007177624 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.858+0000 7fe158e00640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe154114020 0x7fe1541a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.859+0000 7fe153400640 1 -- 172.21.15.32:0/1007177624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe154114020 msgr2=0x7fe1541a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.859+0000 7fe153400640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe154114020 0x7fe1541a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.859+0000 7fe153400640 1 -- 172.21.15.32:0/1007177624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe154075b70 msgr2=0x7fe1541a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.859+0000 7fe153400640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe154075b70 0x7fe1541a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.859+0000 7fe153400640 1 -- 172.21.15.32:0/1007177624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1541a6630 con 0x7fe154077120 2024-08-31T07:13:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.859+0000 7fe153400640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe154077120 0x7fe1541a0c60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe14000e9a0 tx=0x7fe14000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.859+0000 7fe158e00640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe154114020 0x7fe1541a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.860+0000 7fe150c00640 1 -- 172.21.15.32:0/1007177624 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14000cd80 con 0x7fe154077120 2024-08-31T07:13:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.860+0000 7fe159800640 1 -- 172.21.15.32:0/1007177624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1541a6920 con 0x7fe154077120 2024-08-31T07:13:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.860+0000 7fe150c00640 1 -- 172.21.15.32:0/1007177624 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1400035f0 con 0x7fe154077120 2024-08-31T07:13:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.860+0000 7fe150c00640 1 -- 172.21.15.32:0/1007177624 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140010690 con 0x7fe154077120 2024-08-31T07:13:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.860+0000 7fe159800640 1 -- 172.21.15.32:0/1007177624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541ad080 con 0x7fe154077120 2024-08-31T07:13:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.861+0000 7fe159800640 1 -- 172.21.15.32:0/1007177624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe154075f80 con 0x7fe154077120 2024-08-31T07:13:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.864+0000 7fe150c00640 1 -- 172.21.15.32:0/1007177624 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7fe140004020 con 0x7fe154077120 2024-08-31T07:13:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.866+0000 7fe150c00640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1380b4c10 0x7fe1380b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.867+0000 7fe150c00640 1 -- 172.21.15.32:0/1007177624 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe1400e5680 con 0x7fe154077120 2024-08-31T07:13:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.867+0000 7fe153e00640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1380b4c10 0x7fe1380b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.868+0000 7fe153e00640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1380b4c10 0x7fe1380b7120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe144005ab0 tx=0x7fe144005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.870+0000 7fe150c00640 1 -- 172.21.15.32:0/1007177624 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe1400ac550 con 0x7fe154077120 2024-08-31T07:13:01.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:01.907+0000 7fe150c00640 1 -- 172.21.15.32:0/1007177624 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe1400a80e0 con 0x7fe154077120 2024-08-31T07:13:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:01 smithi032 ceph-mon[17252]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-08-31T07:13:02.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.142+0000 7fe159800640 1 -- 172.21.15.32:0/1007177624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1541a17f0 con 0x7fe154077120 2024-08-31T07:13:02.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.143+0000 7fe150c00640 1 -- 172.21.15.32:0/1007177624 <== mon.2 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7fe1400203e0 con 0x7fe154077120 2024-08-31T07:13:02.145 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:13:02.145 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:13:02.145 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:13:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 -- 172.21.15.32:0/1007177624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1380b4c10 msgr2=0x7fe1380b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1380b4c10 0x7fe1380b7120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe144005ab0 tx=0x7fe144005a20 comp rx=0 tx=0).stop 2024-08-31T07:13:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 -- 172.21.15.32:0/1007177624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe154077120 msgr2=0x7fe1541a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe154077120 0x7fe1541a0c60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe14000e9a0 tx=0x7fe14000ee60 comp rx=0 tx=0).stop 2024-08-31T07:13:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 -- 172.21.15.32:0/1007177624 shutdown_connections 2024-08-31T07:13:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1380b4c10 0x7fe1380b7120 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe154114020 0x7fe1541a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe154077120 0x7fe1541a0c60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 --2- 172.21.15.32:0/1007177624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe154075b70 0x7fe1541a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 -- 172.21.15.32:0/1007177624 >> 172.21.15.32:0/1007177624 conn(0x7fe1540ff1e0 msgr2=0x7fe15410ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.147+0000 7fe159800640 1 -- 172.21.15.32:0/1007177624 shutdown_connections 2024-08-31T07:13:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:02.148+0000 7fe159800640 1 -- 172.21.15.32:0/1007177624 wait complete. 2024-08-31T07:13:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:01 smithi073 ceph-mon[21995]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-08-31T07:13:02.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:01 smithi084 ceph-mon[22387]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-08-31T07:13:02.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:13:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:02 smithi073 ceph-mon[21995]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-08-31T07:13:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:02 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1007177624' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:03.189 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:02 smithi032 ceph-mon[17252]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-08-31T07:13:03.189 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:02 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1007177624' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:03.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:02 smithi084 ceph-mon[22387]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-08-31T07:13:03.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:02 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1007177624' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.075+0000 7fe606600640 1 -- 172.21.15.32:0/1667325558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6000776e0 msgr2=0x7fe600077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.075+0000 7fe606600640 1 --2- 172.21.15.32:0/1667325558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6000776e0 0x7fe600077af0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009900 tx=0x7fe5f002a410 comp rx=0 tx=0).stop 2024-08-31T07:13:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.076+0000 7fe606600640 1 -- 172.21.15.32:0/1667325558 shutdown_connections 2024-08-31T07:13:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.076+0000 7fe606600640 1 --2- 172.21.15.32:0/1667325558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6001144e0 0x7fe600116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.076+0000 7fe606600640 1 --2- 172.21.15.32:0/1667325558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe600078030 0x7fe600113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.076+0000 7fe606600640 1 --2- 172.21.15.32:0/1667325558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6000776e0 0x7fe600077af0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.076+0000 7fe606600640 1 -- 172.21.15.32:0/1667325558 >> 172.21.15.32:0/1667325558 conn(0x7fe6001015b0 msgr2=0x7fe600103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.076+0000 7fe606600640 1 -- 172.21.15.32:0/1667325558 shutdown_connections 2024-08-31T07:13:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.076+0000 7fe606600640 1 -- 172.21.15.32:0/1667325558 wait complete. 2024-08-31T07:13:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.077+0000 7fe606600640 1 Processor -- start 2024-08-31T07:13:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.077+0000 7fe606600640 1 -- start start 2024-08-31T07:13:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.077+0000 7fe606600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6000776e0 0x7fe6001a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.077+0000 7fe606600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe600078030 0x7fe6001a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.077+0000 7fe606600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6001144e0 0x7fe6001a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe606600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe600119eb0 con 0x7fe6001144e0 2024-08-31T07:13:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe606600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe600119d30 con 0x7fe6000776e0 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe606600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe60011a030 con 0x7fe600078030 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe5fea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe600078030 0x7fe6001a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe5ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6001144e0 0x7fe6001a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe5fea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe600078030 0x7fe6001a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:56340/0 (socket says 172.21.15.32:56340) 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe5ff400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6000776e0 0x7fe6001a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe5fea00640 1 -- 172.21.15.32:0/4033251432 learned_addr learned my addr 172.21.15.32:0/4033251432 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe5fea00640 1 -- 172.21.15.32:0/4033251432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6000776e0 msgr2=0x7fe6001a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe5fea00640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6000776e0 0x7fe6001a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe5fea00640 1 -- 172.21.15.32:0/4033251432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6001144e0 msgr2=0x7fe6001a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe5fea00640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6001144e0 0x7fe6001a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.078+0000 7fe5fea00640 1 -- 172.21.15.32:0/4033251432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6001a67d0 con 0x7fe600078030 2024-08-31T07:13:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.079+0000 7fe5fea00640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe600078030 0x7fe6001a0e00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00e9f0 tx=0x7fe5ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.079+0000 7fe5f7e00640 1 -- 172.21.15.32:0/4033251432 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00ccc0 con 0x7fe600078030 2024-08-31T07:13:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.079+0000 7fe5f7e00640 1 -- 172.21.15.32:0/4033251432 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ec0108e0 con 0x7fe600078030 2024-08-31T07:13:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.079+0000 7fe606600640 1 -- 172.21.15.32:0/4033251432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6001a6ac0 con 0x7fe600078030 2024-08-31T07:13:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.079+0000 7fe5f7e00640 1 -- 172.21.15.32:0/4033251432 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec018860 con 0x7fe600078030 2024-08-31T07:13:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.079+0000 7fe606600640 1 -- 172.21.15.32:0/4033251432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6001ad220 con 0x7fe600078030 2024-08-31T07:13:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.082+0000 7fe60d1bd740 1 Processor -- start 2024-08-31T07:13:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.083+0000 7fe60d1bd740 1 -- start start 2024-08-31T07:13:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.083+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 0x55eea5315990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.083+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea5315ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.083+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5316410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.083+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55eea528c7b0 con 0x55eea4f471b0 2024-08-31T07:13:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.083+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55eea527bd40 con 0x55eea4f4f390 2024-08-31T07:13:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.083+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55eea527fcf0 con 0x55eea4f70070 2024-08-31T07:13:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5ff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea5315ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5ff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea5315ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50772/0 (socket says 172.21.15.32:50772) 2024-08-31T07:13:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5ff400640 1 -- 172.21.15.32:0/1611917398 learned_addr learned my addr 172.21.15.32:0/1611917398 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5ffe00640 1 --2- 172.21.15.32:0/1611917398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 0x55eea5315990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5f7e00640 1 -- 172.21.15.32:0/4033251432 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe5ec010430 con 0x7fe600078030 2024-08-31T07:13:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5ffe00640 1 -- 172.21.15.32:0/1611917398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 msgr2=0x55eea5316410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:13:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5ffe00640 1 --2- 172.21.15.32:0/1611917398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5316410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5ffe00640 1 -- 172.21.15.32:0/1611917398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 msgr2=0x55eea5315ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5ffe00640 1 --2- 172.21.15.32:0/1611917398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea5315ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5ffe00640 1 -- 172.21.15.32:0/1611917398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eea4fdb650 con 0x55eea4f70070 2024-08-31T07:13:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.084+0000 7fe5ffe00640 1 --2- 172.21.15.32:0/1611917398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 0x55eea5315990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009f00 tx=0x7fe5f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3aea6f14feda267 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.085+0000 7fe5f4c00640 1 -- 172.21.15.32:0/1611917398 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8004d20 con 0x55eea4f70070 2024-08-31T07:13:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.085+0000 7fe5f4c00640 1 -- 172.21.15.32:0/1611917398 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f8004030 con 0x55eea4f70070 2024-08-31T07:13:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.085+0000 7fe5f4c00640 1 -- 172.21.15.32:0/1611917398 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8005260 con 0x55eea4f70070 2024-08-31T07:13:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1611917398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 msgr2=0x55eea5315990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/1611917398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 0x55eea5315990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009f00 tx=0x7fe5f8011810 comp rx=0 tx=0).stop 2024-08-31T07:13:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1611917398 shutdown_connections 2024-08-31T07:13:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/1611917398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5316410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/1611917398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea5315ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/1611917398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 0x55eea5315990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1611917398 >> 172.21.15.32:0/1611917398 conn(0x55eea4f64460 msgr2=0x55eea5313880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe5f7e00640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5e40b4d20 0x7fe5e40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1611917398 shutdown_connections 2024-08-31T07:13:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe5f7e00640 1 -- 172.21.15.32:0/4033251432 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe5ec0e5740 con 0x7fe600078030 2024-08-31T07:13:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1611917398 wait complete. 2024-08-31T07:13:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.086+0000 7fe5ffe00640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5e40b4d20 0x7fe5e40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.087+0000 7fe60d1bd740 1 -- start start 2024-08-31T07:13:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.087+0000 7fe60d1bd740 1 Processor -- start 2024-08-31T07:13:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe5ffe00640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5e40b4d20 0x7fe5e40b7230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8011d10 tx=0x7fe5f8005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 0x55eea5321610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5321bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 0x55eea532d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55eea50de700 con 0x55eea4f70070 2024-08-31T07:13:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55eea5202cf0 con 0x55eea4f4f390 2024-08-31T07:13:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55eea5324170 con 0x55eea4f471b0 2024-08-31T07:13:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe60d1bd740 10 client.0 _ll_register_callbacks cb 0x55eea5228470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe5ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 0x55eea532d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe5fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5321bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.088+0000 7fe5ff400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 0x55eea5321610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.089+0000 7fe5fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5321bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46774/0 (socket says 172.21.15.32:46774) 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.089+0000 7fe5fea00640 1 -- 172.21.15.32:0/3437592280 learned_addr learned my addr 172.21.15.32:0/3437592280 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.089+0000 7fe5ff400640 1 -- 172.21.15.32:0/3437592280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 msgr2=0x55eea5321bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.089+0000 7fe5ff400640 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5321bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.089+0000 7fe5ff400640 1 -- 172.21.15.32:0/3437592280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 msgr2=0x55eea532d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.089+0000 7fe5ff400640 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 0x55eea532d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.089+0000 7fe5ff400640 1 -- 172.21.15.32:0/3437592280 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eea52438b0 con 0x55eea4f471b0 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.089+0000 7fe5fea00640 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5321bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.089+0000 7fe5ff400640 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 0x55eea5321610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0006010 tx=0x7fe5f00043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5474731b00917408 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.089+0000 7fe5e9600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.090+0000 7fe5e9600640 1 -- 172.21.15.32:0/3437592280 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0035040 con 0x55eea4f471b0 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.090+0000 7fe5e9600640 1 -- 172.21.15.32:0/3437592280 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe5f0005450 con 0x55eea4f471b0 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.090+0000 7fe60d1bd740 1 -- 172.21.15.32:0/3437592280 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f8004030 con 0x55eea4f471b0 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.090+0000 7fe5e9600640 1 -- 172.21.15.32:0/3437592280 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5f0005650 con 0x55eea4f471b0 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.090+0000 7fe60d1bd740 10 client.25618 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.090+0000 7fe60d1bd740 1 -- 172.21.15.32:0/3437592280 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eea5330230 con 0x55eea4f471b0 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.090+0000 7fe5e9600640 1 -- 172.21.15.32:0/3437592280 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f8004030 con 0x55eea4f471b0 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.091+0000 7fe5e9600640 1 -- 172.21.15.32:0/3437592280 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe5f0003ab0 con 0x55eea4f471b0 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.091+0000 7fe60d1bd740 10 client.25618 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.091+0000 7fe60d1bd740 10 client.25618 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.091+0000 7fe60d1bd740 20 client.25618 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.091+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eea4f2e9f0 0x55eea5333650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.091+0000 7fe60d1bd740 4 client.25618 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:04.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.091+0000 7fe60d1bd740 1 -- 172.21.15.32:0/3437592280 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eea528e700 con 0x55eea4f2e9f0 2024-08-31T07:13:04.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.091+0000 7fe5fea00640 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eea4f2e9f0 0x55eea5333650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.092+0000 7fe5fea00640 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eea4f2e9f0 0x55eea5333650 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95468d4fc0b72dca server_cookie=78839e47b00c5cad in_seq=0 out_seq=0 2024-08-31T07:13:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.092+0000 7fe5e9600640 10 client.25618 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:13:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe5e9600640 1 -- 172.21.15.32:0/3437592280 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eea528e700 con 0x55eea4f2e9f0 2024-08-31T07:13:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe5e9600640 10 client.25618 handle_command_reply: tid=0 2024-08-31T07:13:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 client.25618 shutdown 2024-08-31T07:13:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 -- 172.21.15.32:0/3437592280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 msgr2=0x55eea5321610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 0x55eea5321610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0006010 tx=0x7fe5f00043b0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 -- 172.21.15.32:0/3437592280 shutdown_connections 2024-08-31T07:13:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eea4f2e9f0 0x55eea5333650 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 0x55eea532d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5321bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/3437592280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 0x55eea5321610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe5e9600640 0 client.25618 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:13:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 -- 172.21.15.32:0/3437592280 >> 172.21.15.32:0/3437592280 conn(0x55eea4f64460 msgr2=0x55eea5318b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 -- 172.21.15.32:0/3437592280 shutdown_connections 2024-08-31T07:13:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 1 -- 172.21.15.32:0/3437592280 wait complete. 2024-08-31T07:13:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.093+0000 7fe60d1bd740 20 client.25618 trim_cache size 0 max 16384 2024-08-31T07:13:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.108+0000 7fe60d1bd740 1 Processor -- start 2024-08-31T07:13:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.108+0000 7fe60d1bd740 1 -- start start 2024-08-31T07:13:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.108+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea532d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.108+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea531abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.108+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 0x55eea531b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.108+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6001b58b0 con 0x55eea4f471b0 2024-08-31T07:13:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.108+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe6001b73f0 con 0x55eea4f4f390 2024-08-31T07:13:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.108+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe6001b5f80 con 0x55eea4f70070 2024-08-31T07:13:04.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5ff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea532d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5ff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea532d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50780/0 (socket says 172.21.15.32:50780) 2024-08-31T07:13:04.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5ff400640 1 -- 172.21.15.32:0/2086910105 learned_addr learned my addr 172.21.15.32:0/2086910105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:04.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5fea00640 1 --2- 172.21.15.32:0/2086910105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea531abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5ffe00640 1 --2- 172.21.15.32:0/2086910105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 0x55eea531b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5fea00640 1 -- 172.21.15.32:0/2086910105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 msgr2=0x55eea531b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5fea00640 1 --2- 172.21.15.32:0/2086910105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 0x55eea531b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5fea00640 1 -- 172.21.15.32:0/2086910105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 msgr2=0x55eea532d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5fea00640 1 --2- 172.21.15.32:0/2086910105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea532d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5fea00640 1 -- 172.21.15.32:0/2086910105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eea5313b60 con 0x55eea4f4f390 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5ff400640 1 --2- 172.21.15.32:0/2086910105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea532d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.109+0000 7fe5fea00640 1 --2- 172.21.15.32:0/2086910105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea531abe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0abc00 tx=0x7fe5ec009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61bf19a10b5ec6f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.110+0000 7fe5ebe00640 1 -- 172.21.15.32:0/2086910105 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec010a90 con 0x55eea4f4f390 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.110+0000 7fe5ebe00640 1 -- 172.21.15.32:0/2086910105 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ec0a8190 con 0x55eea4f4f390 2024-08-31T07:13:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.110+0000 7fe5ebe00640 1 -- 172.21.15.32:0/2086910105 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0930d0 con 0x55eea4f4f390 2024-08-31T07:13:04.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.111+0000 7fe60d1bd740 1 -- 172.21.15.32:0/2086910105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 msgr2=0x55eea531abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:04.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.111+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/2086910105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea531abe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0abc00 tx=0x7fe5ec009740 comp rx=0 tx=0).stop 2024-08-31T07:13:04.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.111+0000 7fe60d1bd740 1 -- 172.21.15.32:0/2086910105 shutdown_connections 2024-08-31T07:13:04.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.111+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/2086910105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f70070 0x55eea531b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.111+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/2086910105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea531abe0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.111+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/2086910105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f471b0 0x55eea532d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.111+0000 7fe60d1bd740 1 -- 172.21.15.32:0/2086910105 >> 172.21.15.32:0/2086910105 conn(0x55eea4f64460 msgr2=0x55eea531c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:04.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.111+0000 7fe60d1bd740 1 -- 172.21.15.32:0/2086910105 shutdown_connections 2024-08-31T07:13:04.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.111+0000 7fe60d1bd740 1 -- 172.21.15.32:0/2086910105 wait complete. 2024-08-31T07:13:04.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.111+0000 7fe60d1bd740 1 -- start start 2024-08-31T07:13:04.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe60d1bd740 1 Processor -- start 2024-08-31T07:13:04.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 0x55eea53293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5329910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe60d1bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 0x55eea5324380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6001b6d20 con 0x55eea4f70070 2024-08-31T07:13:04.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe6001b6650 con 0x55eea4f4f390 2024-08-31T07:13:04.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe60d1bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe6001b3890 con 0x55eea4f471b0 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe60d1bd740 10 client.0 _ll_register_callbacks cb 0x7fe5f0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe5fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5329910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe5fea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5329910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46794/0 (socket says 172.21.15.32:46794) 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe5fea00640 1 -- 172.21.15.32:0/1555942958 learned_addr learned my addr 172.21.15.32:0/1555942958 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.112+0000 7fe5ff400640 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 0x55eea53293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5ffe00640 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 0x55eea5324380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5fea00640 1 -- 172.21.15.32:0/1555942958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 msgr2=0x55eea53293d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5fea00640 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 0x55eea53293d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5fea00640 1 -- 172.21.15.32:0/1555942958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 msgr2=0x55eea5324380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5fea00640 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 0x55eea5324380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5fea00640 1 -- 172.21.15.32:0/1555942958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eea532a160 con 0x55eea4f4f390 2024-08-31T07:13:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5ffe00640 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 0x55eea5324380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5fea00640 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5329910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0abb00 tx=0x7fe5ec00cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e9c4cfd74b123e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5e9600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5e9600640 1 -- 172.21.15.32:0/1555942958 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec010720 con 0x55eea4f4f390 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe5e9600640 1 -- 172.21.15.32:0/1555942958 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe5ec0b0210 con 0x55eea4f4f390 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.113+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1555942958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ec0a8190 con 0x55eea4f4f390 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe5e9600640 1 -- 172.21.15.32:0/1555942958 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5ec0b0410 con 0x55eea4f4f390 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe60d1bd740 10 client.25616 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1555942958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eea539e5e0 con 0x55eea4f4f390 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe5e9600640 1 -- 172.21.15.32:0/1555942958 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ec0a8190 con 0x55eea4f4f390 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe5e9600640 1 -- 172.21.15.32:0/1555942958 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe5ec0a7870 con 0x55eea4f4f390 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe60d1bd740 10 client.25616 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe60d1bd740 10 client.25616 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:13:04.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe60d1bd740 20 client.25616 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:04.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eea4f03da0 0x55eea53a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:04.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe60d1bd740 4 client.25616 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:13:04.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.114+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1555942958 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eea528a1a0 con 0x55eea4f03da0 2024-08-31T07:13:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.115+0000 7fe5ff400640 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eea4f03da0 0x55eea53a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe5ff400640 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eea4f03da0 0x55eea53a0cd0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1b5f0dfb0da1d948 server_cookie=483c60c646fa9bed in_seq=0 out_seq=0 2024-08-31T07:13:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe5e9600640 10 client.25616 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:13:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe5e9600640 1 -- 172.21.15.32:0/1555942958 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eea528a1a0 con 0x55eea4f03da0 2024-08-31T07:13:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe5e9600640 10 client.25616 handle_command_reply: tid=0 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe60d1bd740 1 client.25616 shutdown 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1555942958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 msgr2=0x55eea5329910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5329910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0abb00 tx=0x7fe5ec00cef0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1555942958 shutdown_connections 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eea4f03da0 0x55eea53a0cd0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eea4f70070 0x55eea5324380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eea4f4f390 0x55eea5329910 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe60d1bd740 1 --2- 172.21.15.32:0/1555942958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eea4f471b0 0x55eea53293d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe5e9600640 0 client.25616 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.116+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1555942958 >> 172.21.15.32:0/1555942958 conn(0x55eea4f64460 msgr2=0x55eea531fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.117+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1555942958 shutdown_connections 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.117+0000 7fe60d1bd740 1 -- 172.21.15.32:0/1555942958 wait complete. 2024-08-31T07:13:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.117+0000 7fe60d1bd740 20 client.25616 trim_cache size 0 max 16384 2024-08-31T07:13:04.121 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:13:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 -- 172.21.15.32:0/4033251432 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5e40b4d20 msgr2=0x7fe5e40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5e40b4d20 0x7fe5e40b7230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8011d10 tx=0x7fe5f8005fd0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 -- 172.21.15.32:0/4033251432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe600078030 msgr2=0x7fe6001a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe600078030 0x7fe6001a0e00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00e9f0 tx=0x7fe5ec00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 -- 172.21.15.32:0/4033251432 shutdown_connections 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5e40b4d20 0x7fe5e40b7230 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6001144e0 0x7fe6001a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe600078030 0x7fe6001a0e00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 --2- 172.21.15.32:0/4033251432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6000776e0 0x7fe6001a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 -- 172.21.15.32:0/4033251432 >> 172.21.15.32:0/4033251432 conn(0x7fe6001015b0 msgr2=0x7fe6001164f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 -- 172.21.15.32:0/4033251432 shutdown_connections 2024-08-31T07:13:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:04.118+0000 7fe606600640 1 -- 172.21.15.32:0/4033251432 wait complete. 2024-08-31T07:13:04.149 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:03 smithi032 ceph-mon[17252]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-08-31T07:13:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:03 smithi073 ceph-mon[21995]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-08-31T07:13:04.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:03 smithi084 ceph-mon[22387]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-08-31T07:13:04.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:13:05.388 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:06.067 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:05 smithi032 ceph-mon[17252]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-08-31T07:13:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.152+0000 7f9a31600640 1 -- 172.21.15.32:0/3080592183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a2c109070 msgr2=0x7f9a2c109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.152+0000 7f9a31600640 1 --2- 172.21.15.32:0/3080592183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a2c109070 0x7f9a2c109480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009900 tx=0x7f9a1802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.152+0000 7f9a31600640 1 -- 172.21.15.32:0/3080592183 shutdown_connections 2024-08-31T07:13:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.152+0000 7f9a31600640 1 --2- 172.21.15.32:0/3080592183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a2c1099c0 0x7f9a2c1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.152+0000 7f9a31600640 1 --2- 172.21.15.32:0/3080592183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a2c109070 0x7f9a2c109480 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.152+0000 7f9a31600640 1 --2- 172.21.15.32:0/3080592183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c103730 0x7f9a2c103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.152+0000 7f9a31600640 1 -- 172.21.15.32:0/3080592183 >> 172.21.15.32:0/3080592183 conn(0x7f9a2c0ff180 msgr2=0x7f9a2c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.152+0000 7f9a31600640 1 -- 172.21.15.32:0/3080592183 shutdown_connections 2024-08-31T07:13:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.152+0000 7f9a31600640 1 -- 172.21.15.32:0/3080592183 wait complete. 2024-08-31T07:13:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.154+0000 7f9a31600640 1 Processor -- start 2024-08-31T07:13:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.154+0000 7f9a31600640 1 -- start start 2024-08-31T07:13:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.154+0000 7f9a31600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c103730 0x7f9a2c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.154+0000 7f9a31600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a2c109070 0x7f9a2c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.154+0000 7f9a31600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a2c1099c0 0x7f9a2c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.154+0000 7f9a31600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c113630 con 0x7f9a2c109070 2024-08-31T07:13:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.154+0000 7f9a31600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1134b0 con 0x7f9a2c1099c0 2024-08-31T07:13:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.154+0000 7f9a31600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1137b0 con 0x7f9a2c103730 2024-08-31T07:13:06.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.155+0000 7f9a2b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a2c109070 0x7f9a2c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.155+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c103730 0x7f9a2c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.155+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c103730 0x7f9a2c0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50430/0 (socket says 172.21.15.32:50430) 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.155+0000 7f9a2be00640 1 -- 172.21.15.32:0/1975581660 learned_addr learned my addr 172.21.15.32:0/1975581660 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.155+0000 7f9a30c00640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a2c1099c0 0x7f9a2c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.155+0000 7f9a30c00640 1 -- 172.21.15.32:0/1975581660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c103730 msgr2=0x7f9a2c0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.155+0000 7f9a30c00640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c103730 0x7f9a2c0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.155+0000 7f9a30c00640 1 -- 172.21.15.32:0/1975581660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a2c109070 msgr2=0x7f9a2c0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.155+0000 7f9a30c00640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a2c109070 0x7f9a2c0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.155+0000 7f9a30c00640 1 -- 172.21.15.32:0/1975581660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c1a83c0 con 0x7f9a2c1099c0 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.156+0000 7f9a2b400640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a2c109070 0x7f9a2c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:06.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.156+0000 7f9a30c00640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a2c1099c0 0x7f9a2c075f30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400ec50 tx=0x7f9a2400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.156+0000 7f9a28c00640 1 -- 172.21.15.32:0/1975581660 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24003b70 con 0x7f9a2c1099c0 2024-08-31T07:13:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.156+0000 7f9a31600640 1 -- 172.21.15.32:0/1975581660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c1a86b0 con 0x7f9a2c1099c0 2024-08-31T07:13:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.156+0000 7f9a28c00640 1 -- 172.21.15.32:0/1975581660 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a24003d10 con 0x7f9a2c1099c0 2024-08-31T07:13:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.156+0000 7f9a28c00640 1 -- 172.21.15.32:0/1975581660 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24010830 con 0x7f9a2c1099c0 2024-08-31T07:13:06.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.156+0000 7f9a31600640 1 -- 172.21.15.32:0/1975581660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c1a8bc0 con 0x7f9a2c1099c0 2024-08-31T07:13:06.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.160+0000 7f9a399bc740 1 Processor -- start 2024-08-31T07:13:06.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.160+0000 7f9a28c00640 1 -- 172.21.15.32:0/1975581660 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9a24003ec0 con 0x7f9a2c1099c0 2024-08-31T07:13:06.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.161+0000 7f9a399bc740 1 -- start start 2024-08-31T07:13:06.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.161+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b316070 0x56186b6bbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.161+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.161+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 0x56186b6bc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.161+0000 7f9a399bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56186b6327b0 con 0x56186b2f5390 2024-08-31T07:13:06.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.161+0000 7f9a399bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56186b621d40 con 0x56186b316070 2024-08-31T07:13:06.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.161+0000 7f9a399bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56186b625cf0 con 0x56186b2ed1b0 2024-08-31T07:13:06.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.161+0000 7f9a2b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b316070 0x56186b6bbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.162+0000 7f9a2b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b316070 0x56186b6bbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38536/0 (socket says 172.21.15.32:38536) 2024-08-31T07:13:06.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.162+0000 7f9a2b400640 1 -- 172.21.15.32:0/4228850697 learned_addr learned my addr 172.21.15.32:0/4228850697 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:06.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.162+0000 7f9a2be00640 1 --2- 172.21.15.32:0/4228850697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.162+0000 7f9a2b400640 1 --2- 172.21.15.32:0/4228850697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 0x56186b6bc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.162+0000 7f9a2be00640 1 -- 172.21.15.32:0/4228850697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b316070 msgr2=0x56186b6bbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:06.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.162+0000 7f9a2be00640 1 --2- 172.21.15.32:0/4228850697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b316070 0x56186b6bbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.162+0000 7f9a2be00640 1 -- 172.21.15.32:0/4228850697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 msgr2=0x56186b6bc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:06.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.162+0000 7f9a2be00640 1 --2- 172.21.15.32:0/4228850697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 0x56186b6bc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.162+0000 7f9a2be00640 1 -- 172.21.15.32:0/4228850697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56186b381650 con 0x56186b2ed1b0 2024-08-31T07:13:06.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.163+0000 7f9a2be00640 1 --2- 172.21.15.32:0/4228850697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6bc0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009c80 tx=0x7f9a1c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde633b9510643d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:06.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.163+0000 7f9a28c00640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a100b4d20 0x7f9a100b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.163+0000 7f9a21200640 1 -- 172.21.15.32:0/4228850697 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c004230 con 0x56186b2ed1b0 2024-08-31T07:13:06.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.163+0000 7f9a28c00640 1 -- 172.21.15.32:0/1975581660 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f9a240e5990 con 0x7f9a2c1099c0 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.163+0000 7f9a2b400640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a100b4d20 0x7f9a100b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.163+0000 7f9a21200640 1 -- 172.21.15.32:0/4228850697 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c0043d0 con 0x56186b2ed1b0 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.163+0000 7f9a21200640 1 -- 172.21.15.32:0/4228850697 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c019640 con 0x56186b2ed1b0 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.164+0000 7f9a399bc740 1 -- 172.21.15.32:0/4228850697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 msgr2=0x56186b6bc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.164+0000 7f9a2b400640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a100b4d20 0x7f9a100b7230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9a18005f70 tx=0x7f9a18031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.164+0000 7f9a399bc740 1 --2- 172.21.15.32:0/4228850697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6bc0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009c80 tx=0x7f9a1c011810 comp rx=0 tx=0).stop 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.164+0000 7f9a399bc740 1 -- 172.21.15.32:0/4228850697 shutdown_connections 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.164+0000 7f9a399bc740 1 --2- 172.21.15.32:0/4228850697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 0x56186b6bc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.164+0000 7f9a399bc740 1 --2- 172.21.15.32:0/4228850697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6bc0a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.164+0000 7f9a399bc740 1 --2- 172.21.15.32:0/4228850697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b316070 0x56186b6bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.164+0000 7f9a399bc740 1 -- 172.21.15.32:0/4228850697 >> 172.21.15.32:0/4228850697 conn(0x56186b30a460 msgr2=0x56186b6b9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.164+0000 7f9a399bc740 1 -- 172.21.15.32:0/4228850697 shutdown_connections 2024-08-31T07:13:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.164+0000 7f9a399bc740 1 -- 172.21.15.32:0/4228850697 wait complete. 2024-08-31T07:13:06.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.165+0000 7f9a399bc740 1 -- start start 2024-08-31T07:13:06.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.165+0000 7f9a399bc740 1 Processor -- start 2024-08-31T07:13:06.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 0x56186b6c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b316070 0x56186b6d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a399bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56186b484700 con 0x56186b2f5390 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a399bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56186b5a8cf0 con 0x56186b2ed1b0 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a399bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56186b6ca340 con 0x56186b316070 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a399bc740 10 client.0 _ll_register_callbacks cb 0x56186b5ce470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a2b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 0x56186b6c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38550/0 (socket says 172.21.15.32:38550) 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a2be00640 1 -- 172.21.15.32:0/2383099736 learned_addr learned my addr 172.21.15.32:0/2383099736 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a2b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 0x56186b6c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43468/0 (socket says 172.21.15.32:43468) 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.166+0000 7f9a30c00640 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b316070 0x56186b6d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.167+0000 7f9a2b400640 1 -- 172.21.15.32:0/2383099736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b316070 msgr2=0x56186b6d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:06.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.167+0000 7f9a2b400640 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b316070 0x56186b6d3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.167+0000 7f9a2b400640 1 -- 172.21.15.32:0/2383099736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 msgr2=0x56186b6c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.167+0000 7f9a2b400640 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.167+0000 7f9a2b400640 1 -- 172.21.15.32:0/2383099736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186b5e98b0 con 0x56186b2f5390 2024-08-31T07:13:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.167+0000 7f9a2be00640 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.167+0000 7f9a2b400640 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 0x56186b6c7be0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9a180096c0 tx=0x7f9a1802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d9137085b2da24 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.167+0000 7f9a15600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:13:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.168+0000 7f9a15600640 1 -- 172.21.15.32:0/2383099736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1804f030 con 0x56186b2f5390 2024-08-31T07:13:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.168+0000 7f9a15600640 1 -- 172.21.15.32:0/2383099736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f9a18003980 con 0x56186b2f5390 2024-08-31T07:13:06.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.168+0000 7f9a399bc740 1 -- 172.21.15.32:0/2383099736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c0043d0 con 0x56186b2f5390 2024-08-31T07:13:06.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.168+0000 7f9a15600640 1 -- 172.21.15.32:0/2383099736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a18035040 con 0x56186b2f5390 2024-08-31T07:13:06.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.168+0000 7f9a399bc740 10 client.15621 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.168+0000 7f9a399bc740 1 -- 172.21.15.32:0/2383099736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56186b6d6300 con 0x56186b2f5390 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.168+0000 7f9a15600640 1 -- 172.21.15.32:0/2383099736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c0043d0 con 0x56186b2f5390 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.169+0000 7f9a15600640 1 -- 172.21.15.32:0/2383099736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9a18048050 con 0x56186b2f5390 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.169+0000 7f9a399bc740 10 client.15621 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.169+0000 7f9a399bc740 10 client.15621 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.169+0000 7f9a399bc740 20 client.15621 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.169+0000 7f9a399bc740 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56186b2d49f0 0x56186b6d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.169+0000 7f9a399bc740 4 client.15621 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.169+0000 7f9a399bc740 1 -- 172.21.15.32:0/2383099736 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56186b634700 con 0x56186b2d49f0 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.169+0000 7f9a2be00640 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56186b2d49f0 0x56186b6d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.170+0000 7f9a2be00640 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56186b2d49f0 0x56186b6d97d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d7507db66dafe69 server_cookie=1d13f8271fc25166 in_seq=0 out_seq=0 2024-08-31T07:13:06.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.170+0000 7f9a15600640 10 client.15621 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:13:06.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.171+0000 7f9a15600640 1 -- 172.21.15.32:0/2383099736 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56186b634700 con 0x56186b2d49f0 2024-08-31T07:13:06.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.171+0000 7f9a15600640 10 client.15621 handle_command_reply: tid=0 2024-08-31T07:13:06.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.171+0000 7f9a399bc740 1 client.15621 shutdown 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 1 -- 172.21.15.32:0/2383099736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 msgr2=0x56186b6c7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 0x56186b6c7be0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9a180096c0 tx=0x7f9a1802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 1 -- 172.21.15.32:0/2383099736 shutdown_connections 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56186b2d49f0 0x56186b6d97d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b316070 0x56186b6d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b2f5390 0x56186b6c7be0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 1 --2- 172.21.15.32:0/2383099736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 1 -- 172.21.15.32:0/2383099736 >> 172.21.15.32:0/2383099736 conn(0x56186b30a460 msgr2=0x56186b6becf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a15600640 0 client.15621 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 1 -- 172.21.15.32:0/2383099736 shutdown_connections 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 1 -- 172.21.15.32:0/2383099736 wait complete. 2024-08-31T07:13:06.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.172+0000 7f9a399bc740 20 client.15621 trim_cache size 0 max 16384 2024-08-31T07:13:06.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:05 smithi073 ceph-mon[21995]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-08-31T07:13:06.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.190+0000 7f9a399bc740 1 Processor -- start 2024-08-31T07:13:06.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.190+0000 7f9a399bc740 1 -- start start 2024-08-31T07:13:06.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2f5390 0x56186b6c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 0x56186b6bd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a399bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1b1400 con 0x56186b316070 2024-08-31T07:13:06.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a399bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1b2f40 con 0x56186b2ed1b0 2024-08-31T07:13:06.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a399bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1b1ad0 con 0x56186b2f5390 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a30c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 0x56186b6bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6b8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38556/0 (socket says 172.21.15.32:38556) 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a2be00640 1 -- 172.21.15.32:0/664746471 learned_addr learned my addr 172.21.15.32:0/664746471 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a2b400640 1 --2- 172.21.15.32:0/664746471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2f5390 0x56186b6c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a2be00640 1 -- 172.21.15.32:0/664746471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2f5390 msgr2=0x56186b6c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a2be00640 1 --2- 172.21.15.32:0/664746471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2f5390 0x56186b6c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a2be00640 1 -- 172.21.15.32:0/664746471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 msgr2=0x56186b6bd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a2be00640 1 --2- 172.21.15.32:0/664746471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 0x56186b6bd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.191+0000 7f9a2be00640 1 -- 172.21.15.32:0/664746471 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56186b6bde50 con 0x56186b2ed1b0 2024-08-31T07:13:06.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.192+0000 7f9a30c00640 1 --2- 172.21.15.32:0/664746471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 0x56186b6bd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:06.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.192+0000 7f9a2be00640 1 --2- 172.21.15.32:0/664746471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6b8ac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c013010 tx=0x7f9a1c009c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48136203c0c3700a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:06.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.192+0000 7f9a17e00640 1 -- 172.21.15.32:0/664746471 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c031020 con 0x56186b2ed1b0 2024-08-31T07:13:06.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.192+0000 7f9a17e00640 1 -- 172.21.15.32:0/664746471 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c011d10 con 0x56186b2ed1b0 2024-08-31T07:13:06.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.192+0000 7f9a17e00640 1 -- 172.21.15.32:0/664746471 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c01f040 con 0x56186b2ed1b0 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.193+0000 7f9a399bc740 1 -- 172.21.15.32:0/664746471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 msgr2=0x56186b6b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.193+0000 7f9a399bc740 1 --2- 172.21.15.32:0/664746471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6b8ac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c013010 tx=0x7f9a1c009c10 comp rx=0 tx=0).stop 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.193+0000 7f9a399bc740 1 -- 172.21.15.32:0/664746471 shutdown_connections 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.193+0000 7f9a399bc740 1 --2- 172.21.15.32:0/664746471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 0x56186b6bd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.193+0000 7f9a399bc740 1 --2- 172.21.15.32:0/664746471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2f5390 0x56186b6c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.193+0000 7f9a399bc740 1 --2- 172.21.15.32:0/664746471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2ed1b0 0x56186b6b8ac0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.193+0000 7f9a399bc740 1 -- 172.21.15.32:0/664746471 >> 172.21.15.32:0/664746471 conn(0x56186b30a460 msgr2=0x56186b6c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.193+0000 7f9a399bc740 1 -- 172.21.15.32:0/664746471 shutdown_connections 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.193+0000 7f9a399bc740 1 -- 172.21.15.32:0/664746471 wait complete. 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.194+0000 7f9a399bc740 1 -- start start 2024-08-31T07:13:06.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.194+0000 7f9a399bc740 1 Processor -- start 2024-08-31T07:13:06.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2f5390 0x56186b6cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a399bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 0x56186b6cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a399bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1b2870 con 0x56186b316070 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a399bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1b21a0 con 0x56186b2f5390 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a399bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1af3e0 con 0x56186b2ed1b0 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a399bc740 10 client.0 _ll_register_callbacks cb 0x7f9a18009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a30c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 0x56186b6cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50466/0 (socket says 172.21.15.32:50466) 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a2be00640 1 -- 172.21.15.32:0/1759845244 learned_addr learned my addr 172.21.15.32:0/1759845244 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a2b400640 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2f5390 0x56186b6cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a2be00640 1 -- 172.21.15.32:0/1759845244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2f5390 msgr2=0x56186b6cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:06.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.195+0000 7f9a2be00640 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2f5390 0x56186b6cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.196+0000 7f9a2be00640 1 -- 172.21.15.32:0/1759845244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 msgr2=0x56186b6cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:06.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.196+0000 7f9a2be00640 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 0x56186b6cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.196+0000 7f9a2be00640 1 -- 172.21.15.32:0/1759845244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186b6ccaf0 con 0x56186b2ed1b0 2024-08-31T07:13:06.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.196+0000 7f9a2be00640 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6cfb80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005d40 tx=0x7f9a1c005190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46865cd524592382 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:06.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.196+0000 7f9a15600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.196+0000 7f9a15600640 1 -- 172.21.15.32:0/1759845244 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c03a020 con 0x56186b2ed1b0 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.196+0000 7f9a399bc740 1 -- 172.21.15.32:0/1759845244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c011d10 con 0x56186b2ed1b0 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.196+0000 7f9a15600640 1 -- 172.21.15.32:0/1759845244 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f9a1c0053b0 con 0x56186b2ed1b0 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.196+0000 7f9a15600640 1 -- 172.21.15.32:0/1759845244 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a1c045050 con 0x56186b2ed1b0 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.197+0000 7f9a399bc740 10 client.25642 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.197+0000 7f9a399bc740 1 -- 172.21.15.32:0/1759845244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56186b744ec0 con 0x56186b2ed1b0 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.197+0000 7f9a15600640 1 -- 172.21.15.32:0/1759845244 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c011d10 con 0x56186b2ed1b0 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.197+0000 7f9a15600640 1 -- 172.21.15.32:0/1759845244 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9a1c018070 con 0x56186b2ed1b0 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.197+0000 7f9a399bc740 10 client.25642 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.197+0000 7f9a399bc740 10 client.25642 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:13:06.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.197+0000 7f9a399bc740 20 client.25642 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.197+0000 7f9a399bc740 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56186b2a9da0 0x56186b7474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.197+0000 7f9a399bc740 4 client.25642 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.197+0000 7f9a399bc740 1 -- 172.21.15.32:0/1759845244 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56186b6301a0 con 0x56186b2a9da0 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.198+0000 7f9a2b400640 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56186b2a9da0 0x56186b7474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.198+0000 7f9a2b400640 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56186b2a9da0 0x56186b7474a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53185ac7fb7ff1a5 server_cookie=8b38edd26029d0b3 in_seq=0 out_seq=0 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.198+0000 7f9a15600640 10 client.25642 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.198+0000 7f9a15600640 1 -- 172.21.15.32:0/1759845244 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56186b6301a0 con 0x56186b2a9da0 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.198+0000 7f9a15600640 10 client.25642 handle_command_reply: tid=0 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.198+0000 7f9a399bc740 1 client.25642 shutdown 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 1 -- 172.21.15.32:0/1759845244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 msgr2=0x56186b6cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6cfb80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005d40 tx=0x7f9a1c005190 comp rx=0 tx=0).stop 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 1 -- 172.21.15.32:0/1759845244 shutdown_connections 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56186b2a9da0 0x56186b7474a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56186b316070 0x56186b6cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56186b2f5390 0x56186b6cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a15600640 0 client.25642 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:13:06.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 1 --2- 172.21.15.32:0/1759845244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56186b2ed1b0 0x56186b6cfb80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 1 -- 172.21.15.32:0/1759845244 >> 172.21.15.32:0/1759845244 conn(0x56186b30a460 msgr2=0x56186b6c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:06.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 1 -- 172.21.15.32:0/1759845244 shutdown_connections 2024-08-31T07:13:06.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 1 -- 172.21.15.32:0/1759845244 wait complete. 2024-08-31T07:13:06.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.199+0000 7f9a399bc740 20 client.25642 trim_cache size 0 max 16384 2024-08-31T07:13:06.203 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:13:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.200+0000 7f9a31600640 1 -- 172.21.15.32:0/1975581660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a100b4d20 msgr2=0x7f9a100b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.200+0000 7f9a31600640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a100b4d20 0x7f9a100b7230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9a18005f70 tx=0x7f9a18031400 comp rx=0 tx=0).stop 2024-08-31T07:13:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.200+0000 7f9a31600640 1 -- 172.21.15.32:0/1975581660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a2c1099c0 msgr2=0x7f9a2c075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:06.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.200+0000 7f9a31600640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a2c1099c0 0x7f9a2c075f30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400ec50 tx=0x7f9a2400c6a0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.200+0000 7f9a31600640 1 -- 172.21.15.32:0/1975581660 shutdown_connections 2024-08-31T07:13:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.200+0000 7f9a31600640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a100b4d20 0x7f9a100b7230 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.200+0000 7f9a31600640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a2c1099c0 0x7f9a2c075f30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.200+0000 7f9a31600640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a2c109070 0x7f9a2c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.200+0000 7f9a31600640 1 --2- 172.21.15.32:0/1975581660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c103730 0x7f9a2c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.200+0000 7f9a31600640 1 -- 172.21.15.32:0/1975581660 >> 172.21.15.32:0/1975581660 conn(0x7f9a2c0ff180 msgr2=0x7f9a2c1015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.201+0000 7f9a31600640 1 -- 172.21.15.32:0/1975581660 shutdown_connections 2024-08-31T07:13:06.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:06.201+0000 7f9a31600640 1 -- 172.21.15.32:0/1975581660 wait complete. 2024-08-31T07:13:06.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:05 smithi084 ceph-mon[22387]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-08-31T07:13:07.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:13:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:08.002 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:07 smithi032 ceph-mon[17252]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-08-31T07:13:08.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:07 smithi073 ceph-mon[21995]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-08-31T07:13:08.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.190+0000 7f6b48a00640 1 -- 172.21.15.32:0/623142474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b441043f0 msgr2=0x7f6b44104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:08.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.190+0000 7f6b48a00640 1 --2- 172.21.15.32:0/623142474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b441043f0 0x7f6b44104800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009900 tx=0x7f6b3402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.190+0000 7f6b48a00640 1 -- 172.21.15.32:0/623142474 shutdown_connections 2024-08-31T07:13:08.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.190+0000 7f6b48a00640 1 --2- 172.21.15.32:0/623142474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b4410d9a0 0x7f6b4410feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.190+0000 7f6b48a00640 1 --2- 172.21.15.32:0/623142474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b44104e20 0x7f6b4410d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.190+0000 7f6b48a00640 1 --2- 172.21.15.32:0/623142474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b441043f0 0x7f6b44104800 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.190+0000 7f6b48a00640 1 -- 172.21.15.32:0/623142474 >> 172.21.15.32:0/623142474 conn(0x7f6b440fe060 msgr2=0x7f6b441004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:08.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.190+0000 7f6b48a00640 1 -- 172.21.15.32:0/623142474 shutdown_connections 2024-08-31T07:13:08.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.191+0000 7f6b48a00640 1 -- 172.21.15.32:0/623142474 wait complete. 2024-08-31T07:13:08.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.192+0000 7f6b48a00640 1 Processor -- start 2024-08-31T07:13:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.192+0000 7f6b48a00640 1 -- start start 2024-08-31T07:13:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.192+0000 7f6b48a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b441043f0 0x7f6b4419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.192+0000 7f6b48a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b44104e20 0x7f6b4419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.192+0000 7f6b48a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b4410d9a0 0x7f6b4419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.192+0000 7f6b48a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6b44113680 con 0x7f6b4410d9a0 2024-08-31T07:13:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.192+0000 7f6b48a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b44113500 con 0x7f6b441043f0 2024-08-31T07:13:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.192+0000 7f6b48a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6b44113800 con 0x7f6b44104e20 2024-08-31T07:13:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b4410d9a0 0x7f6b4419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b43400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b441043f0 0x7f6b4419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b4410d9a0 0x7f6b4419f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43500/0 (socket says 172.21.15.32:43500) 2024-08-31T07:13:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b43400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b441043f0 0x7f6b4419a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38588/0 (socket says 172.21.15.32:38588) 2024-08-31T07:13:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b43e00640 1 -- 172.21.15.32:0/2280957292 learned_addr learned my addr 172.21.15.32:0/2280957292 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b42a00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b44104e20 0x7f6b4419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b43e00640 1 -- 172.21.15.32:0/2280957292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b44104e20 msgr2=0x7f6b4419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b43e00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b44104e20 0x7f6b4419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b43e00640 1 -- 172.21.15.32:0/2280957292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b441043f0 msgr2=0x7f6b4419a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b43e00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b441043f0 0x7f6b4419a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.193+0000 7f6b43e00640 1 -- 172.21.15.32:0/2280957292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b441a0040 con 0x7f6b4410d9a0 2024-08-31T07:13:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.194+0000 7f6b43e00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b4410d9a0 0x7f6b4419f910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00eb10 tx=0x7f6b3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.194+0000 7f6b3be00640 1 -- 172.21.15.32:0/2280957292 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c004b20 con 0x7f6b4410d9a0 2024-08-31T07:13:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.194+0000 7f6b3be00640 1 -- 172.21.15.32:0/2280957292 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b3c004cc0 con 0x7f6b4410d9a0 2024-08-31T07:13:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.194+0000 7f6b3be00640 1 -- 172.21.15.32:0/2280957292 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c010670 con 0x7f6b4410d9a0 2024-08-31T07:13:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.194+0000 7f6b48a00640 1 -- 172.21.15.32:0/2280957292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b441a0330 con 0x7f6b4410d9a0 2024-08-31T07:13:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.194+0000 7f6b48a00640 1 -- 172.21.15.32:0/2280957292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b441a6a90 con 0x7f6b4410d9a0 2024-08-31T07:13:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.198+0000 7f6b50c60740 1 Processor -- start 2024-08-31T07:13:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.199+0000 7f6b3be00640 1 -- 172.21.15.32:0/2280957292 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6b3c005020 con 0x7f6b4410d9a0 2024-08-31T07:13:08.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b50c60740 1 -- start start 2024-08-31T07:13:08.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0017eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0017f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 0x55dc0017f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b50c60740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dc000f57b0 con 0x55dbffdd9070 2024-08-31T07:13:08.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b50c60740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dc000e4d40 con 0x55dbffdb01b0 2024-08-31T07:13:08.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b50c60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dc000e8cf0 con 0x55dbffdb8390 2024-08-31T07:13:08.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0017eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0017eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43504/0 (socket says 172.21.15.32:43504) 2024-08-31T07:13:08.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b42a00640 1 -- 172.21.15.32:0/3181260511 learned_addr learned my addr 172.21.15.32:0/3181260511 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:08.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.200+0000 7f6b43400640 1 --2- 172.21.15.32:0/3181260511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0017f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.201+0000 7f6b42a00640 1 --2- 172.21.15.32:0/3181260511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 0x55dc0017f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.201+0000 7f6b42a00640 1 -- 172.21.15.32:0/3181260511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 msgr2=0x55dc0017f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:08.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.201+0000 7f6b42a00640 1 --2- 172.21.15.32:0/3181260511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 0x55dc0017f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.201+0000 7f6b42a00640 1 -- 172.21.15.32:0/3181260511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 msgr2=0x55dc0017f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:08.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.201+0000 7f6b42a00640 1 --2- 172.21.15.32:0/3181260511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0017f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.201+0000 7f6b42a00640 1 -- 172.21.15.32:0/3181260511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbffe44650 con 0x55dbffdd9070 2024-08-31T07:13:08.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.201+0000 7f6b42a00640 1 --2- 172.21.15.32:0/3181260511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0017eb60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6b30013710 tx=0x7f6b30005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46bda4c77e59995 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.202+0000 7f6b2fe00640 1 -- 172.21.15.32:0/3181260511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30013130 con 0x55dbffdd9070 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.202+0000 7f6b2fe00640 1 -- 172.21.15.32:0/3181260511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b30013c90 con 0x55dbffdd9070 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.202+0000 7f6b2fe00640 1 -- 172.21.15.32:0/3181260511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3000fda0 con 0x55dbffdd9070 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.202+0000 7f6b3be00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6b280b4ca0 0x7f6b280b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.202+0000 7f6b3be00640 1 -- 172.21.15.32:0/2280957292 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6b3c0e5a50 con 0x7f6b4410d9a0 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.202+0000 7f6b43400640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6b280b4ca0 0x7f6b280b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.203+0000 7f6b50c60740 1 -- 172.21.15.32:0/3181260511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 msgr2=0x55dc0017eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.203+0000 7f6b50c60740 1 --2- 172.21.15.32:0/3181260511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0017eb60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6b30013710 tx=0x7f6b30005850 comp rx=0 tx=0).stop 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.203+0000 7f6b50c60740 1 -- 172.21.15.32:0/3181260511 shutdown_connections 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.203+0000 7f6b50c60740 1 --2- 172.21.15.32:0/3181260511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 0x55dc0017f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.203+0000 7f6b50c60740 1 --2- 172.21.15.32:0/3181260511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0017f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.203+0000 7f6b50c60740 1 --2- 172.21.15.32:0/3181260511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0017eb60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.203+0000 7f6b50c60740 1 -- 172.21.15.32:0/3181260511 >> 172.21.15.32:0/3181260511 conn(0x55dbffdcd460 msgr2=0x55dc0017ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.203+0000 7f6b50c60740 1 -- 172.21.15.32:0/3181260511 shutdown_connections 2024-08-31T07:13:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.203+0000 7f6b43400640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6b280b4ca0 0x7f6b280b71b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6b34005ab0 tx=0x7f6b34005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.203+0000 7f6b50c60740 1 -- 172.21.15.32:0/3181260511 wait complete. 2024-08-31T07:13:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.204+0000 7f6b50c60740 1 -- start start 2024-08-31T07:13:08.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.205+0000 7f6b50c60740 1 Processor -- start 2024-08-31T07:13:08.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.205+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0018eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.205+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 0x55dc0018f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.205+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.205+0000 7f6b50c60740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dbfff47700 con 0x55dbffdd9070 2024-08-31T07:13:08.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.205+0000 7f6b50c60740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dc0006bcf0 con 0x55dbffdb01b0 2024-08-31T07:13:08.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.205+0000 7f6b50c60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dc00191640 con 0x55dbffdb8390 2024-08-31T07:13:08.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.205+0000 7f6b50c60740 10 client.0 _ll_register_callbacks cb 0x55dc00091470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:08.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.206+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 0x55dc0018f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.206+0000 7f6b43400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0018eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.206+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 0x55dc0018f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50502/0 (socket says 172.21.15.32:50502) 2024-08-31T07:13:08.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.206+0000 7f6b42a00640 1 -- 172.21.15.32:0/2386399345 learned_addr learned my addr 172.21.15.32:0/2386399345 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:08.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.206+0000 7f6b43e00640 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.206+0000 7f6b42a00640 1 -- 172.21.15.32:0/2386399345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 msgr2=0x55dc0018eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:08.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.206+0000 7f6b42a00640 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0018eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.206+0000 7f6b42a00640 1 -- 172.21.15.32:0/2386399345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 msgr2=0x55dc0019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:08.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.206+0000 7f6b42a00640 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.206+0000 7f6b42a00640 1 -- 172.21.15.32:0/2386399345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc000ac8b0 con 0x55dbffdb8390 2024-08-31T07:13:08.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.207+0000 7f6b42a00640 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 0x55dc0018f030 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000f920 tx=0x7f6b3000fdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff17132f0b80b207 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:08.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.207+0000 7f6b2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:08.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.207+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2386399345 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30023070 con 0x55dbffdb8390 2024-08-31T07:13:08.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.207+0000 7f6b50c60740 1 -- 172.21.15.32:0/2386399345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b30013c90 con 0x55dbffdb8390 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.207+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2386399345 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6b30005430 con 0x55dbffdb8390 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.207+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2386399345 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6b3002f050 con 0x55dbffdb8390 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.207+0000 7f6b50c60740 10 client.25654 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.207+0000 7f6b50c60740 1 -- 172.21.15.32:0/2386399345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc0019d670 con 0x55dbffdb8390 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.208+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2386399345 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b30013c90 con 0x55dbffdb8390 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.208+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2386399345 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6b300104c0 con 0x55dbffdb8390 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.208+0000 7f6b50c60740 10 client.25654 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.208+0000 7f6b50c60740 10 client.25654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.208+0000 7f6b50c60740 20 client.25654 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.208+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55dbffd979f0 0x55dc001a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.208+0000 7f6b50c60740 4 client.25654 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:08.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.208+0000 7f6b50c60740 1 -- 172.21.15.32:0/2386399345 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc000f7700 con 0x55dbffd979f0 2024-08-31T07:13:08.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.209+0000 7f6b43400640 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55dbffd979f0 0x55dc001a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.209+0000 7f6b43400640 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55dbffd979f0 0x55dc001a09d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae85fae682213a56 server_cookie=ffc776f3818b1709 in_seq=0 out_seq=0 2024-08-31T07:13:08.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.209+0000 7f6b2cc00640 10 client.25654 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:13:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.210+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2386399345 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc000f7700 con 0x55dbffd979f0 2024-08-31T07:13:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.210+0000 7f6b2cc00640 10 client.25654 handle_command_reply: tid=0 2024-08-31T07:13:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.210+0000 7f6b50c60740 1 client.25654 shutdown 2024-08-31T07:13:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 1 -- 172.21.15.32:0/2386399345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 msgr2=0x55dc0018f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 0x55dc0018f030 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000f920 tx=0x7f6b3000fdb0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 1 -- 172.21.15.32:0/2386399345 shutdown_connections 2024-08-31T07:13:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55dbffd979f0 0x55dc001a09d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffdb8390 0x55dc0018f030 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2386399345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0018eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 1 -- 172.21.15.32:0/2386399345 >> 172.21.15.32:0/2386399345 conn(0x55dbffdcd460 msgr2=0x55dc0017aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b2cc00640 0 client.25654 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:13:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 1 -- 172.21.15.32:0/2386399345 shutdown_connections 2024-08-31T07:13:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 1 -- 172.21.15.32:0/2386399345 wait complete. 2024-08-31T07:13:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.211+0000 7f6b50c60740 20 client.25654 trim_cache size 0 max 16384 2024-08-31T07:13:08.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.232+0000 7f6b50c60740 1 Processor -- start 2024-08-31T07:13:08.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.232+0000 7f6b50c60740 1 -- start start 2024-08-31T07:13:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0017b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 0x55dc0019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b50c60740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441acf40 con 0x55dbffdd9070 2024-08-31T07:13:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b50c60740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441aea80 con 0x55dbffdb01b0 2024-08-31T07:13:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b50c60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441ad610 con 0x55dbffd6cda0 2024-08-31T07:13:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b43400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0017b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 0x55dc0019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43524/0 (socket says 172.21.15.32:43524) 2024-08-31T07:13:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.233+0000 7f6b42a00640 1 -- 172.21.15.32:0/713663905 learned_addr learned my addr 172.21.15.32:0/713663905 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.234+0000 7f6b43400640 1 -- 172.21.15.32:0/713663905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 msgr2=0x55dc0019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.234+0000 7f6b43400640 1 --2- 172.21.15.32:0/713663905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 0x55dc0019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.234+0000 7f6b43400640 1 -- 172.21.15.32:0/713663905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 msgr2=0x55dc0019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.234+0000 7f6b43400640 1 --2- 172.21.15.32:0/713663905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.234+0000 7f6b43400640 1 -- 172.21.15.32:0/713663905 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc0019b320 con 0x55dbffdb01b0 2024-08-31T07:13:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.234+0000 7f6b43e00640 1 --2- 172.21.15.32:0/713663905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 0x55dc0019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.234+0000 7f6b43400640 1 --2- 172.21.15.32:0/713663905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0017b700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6b3402a9e0 tx=0x7f6b3403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e0d5fa907d2ca92 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.235+0000 7f6b2f400640 1 -- 172.21.15.32:0/713663905 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34035040 con 0x55dbffdb01b0 2024-08-31T07:13:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.235+0000 7f6b2f400640 1 -- 172.21.15.32:0/713663905 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b34004dc0 con 0x55dbffdb01b0 2024-08-31T07:13:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.235+0000 7f6b2f400640 1 -- 172.21.15.32:0/713663905 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34032710 con 0x55dbffdb01b0 2024-08-31T07:13:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.236+0000 7f6b50c60740 1 -- 172.21.15.32:0/713663905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 msgr2=0x55dc0017b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.236+0000 7f6b50c60740 1 --2- 172.21.15.32:0/713663905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0017b700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6b3402a9e0 tx=0x7f6b3403f000 comp rx=0 tx=0).stop 2024-08-31T07:13:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.236+0000 7f6b50c60740 1 -- 172.21.15.32:0/713663905 shutdown_connections 2024-08-31T07:13:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.236+0000 7f6b50c60740 1 --2- 172.21.15.32:0/713663905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 0x55dc0019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.236+0000 7f6b50c60740 1 --2- 172.21.15.32:0/713663905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdd9070 0x55dc0019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.236+0000 7f6b50c60740 1 --2- 172.21.15.32:0/713663905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdb01b0 0x55dc0017b700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.236+0000 7f6b50c60740 1 -- 172.21.15.32:0/713663905 >> 172.21.15.32:0/713663905 conn(0x55dbffdcd460 msgr2=0x55dc0017aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.236+0000 7f6b50c60740 1 -- 172.21.15.32:0/713663905 shutdown_connections 2024-08-31T07:13:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.236+0000 7f6b50c60740 1 -- 172.21.15.32:0/713663905 wait complete. 2024-08-31T07:13:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.236+0000 7f6b50c60740 1 -- start start 2024-08-31T07:13:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.237+0000 7f6b50c60740 1 Processor -- start 2024-08-31T07:13:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.237+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 0x55dc00197200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.237+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdb01b0 0x55dc001926b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.237+0000 7f6b50c60740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdd9070 0x55dc00192d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.237+0000 7f6b50c60740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441ae3b0 con 0x55dbffdb01b0 2024-08-31T07:13:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.237+0000 7f6b50c60740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441adce0 con 0x55dbffdd9070 2024-08-31T07:13:08.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.237+0000 7f6b50c60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441aaf20 con 0x55dbffd6cda0 2024-08-31T07:13:08.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.237+0000 7f6b50c60740 10 client.0 _ll_register_callbacks cb 0x7f6b30013610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:08.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:07 smithi084 ceph-mon[22387]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-08-31T07:13:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdb01b0 0x55dc001926b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdb01b0 0x55dc001926b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43534/0 (socket says 172.21.15.32:43534) 2024-08-31T07:13:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b42a00640 1 -- 172.21.15.32:0/2835617653 learned_addr learned my addr 172.21.15.32:0/2835617653 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b43400640 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 0x55dc00197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b43e00640 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdd9070 0x55dc00192d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b43400640 1 -- 172.21.15.32:0/2835617653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdd9070 msgr2=0x55dc00192d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b43400640 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdd9070 0x55dc00192d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b43400640 1 -- 172.21.15.32:0/2835617653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdb01b0 msgr2=0x55dc001926b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b43400640 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdb01b0 0x55dc001926b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b43400640 1 -- 172.21.15.32:0/2835617653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc00193490 con 0x55dbffd6cda0 2024-08-31T07:13:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b42a00640 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdb01b0 0x55dc001926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.238+0000 7f6b43e00640 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdd9070 0x55dc00192d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.239+0000 7f6b43400640 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 0x55dc00197200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b34005f70 tx=0x7f6b3402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b287bf4692212232 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.239+0000 7f6b2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.239+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2835617653 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3406c060 con 0x55dbffd6cda0 2024-08-31T07:13:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.239+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2835617653 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6b34004bc0 con 0x55dbffd6cda0 2024-08-31T07:13:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.239+0000 7f6b50c60740 1 -- 172.21.15.32:0/2835617653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b34004dc0 con 0x55dbffd6cda0 2024-08-31T07:13:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.239+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2835617653 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6b34003ea0 con 0x55dbffd6cda0 2024-08-31T07:13:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.239+0000 7f6b50c60740 10 client.25660 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.239+0000 7f6b50c60740 1 -- 172.21.15.32:0/2835617653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc0020b2b0 con 0x55dbffd6cda0 2024-08-31T07:13:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.240+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2835617653 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b34004dc0 con 0x55dbffd6cda0 2024-08-31T07:13:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.240+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2835617653 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6b3405a020 con 0x55dbffd6cda0 2024-08-31T07:13:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.240+0000 7f6b50c60740 10 client.25660 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.240+0000 7f6b50c60740 10 client.25660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:13:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.240+0000 7f6b50c60740 20 client.25660 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.240+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55dc0020b9b0 0x55dc0020ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.240+0000 7f6b50c60740 4 client.25660 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.240+0000 7f6b50c60740 1 -- 172.21.15.32:0/2835617653 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc000f31a0 con 0x55dc0020b9b0 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.241+0000 7f6b42a00640 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55dc0020b9b0 0x55dc0020ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.241+0000 7f6b2cc00640 10 client.25660 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.241+0000 7f6b42a00640 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55dc0020b9b0 0x55dc0020ddd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7627e6c312a03ecf server_cookie=f1a29d448f7f4d4b in_seq=0 out_seq=0 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.241+0000 7f6b2cc00640 1 -- 172.21.15.32:0/2835617653 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc000f31a0 con 0x55dc0020b9b0 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.241+0000 7f6b2cc00640 10 client.25660 handle_command_reply: tid=0 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.241+0000 7f6b50c60740 1 client.25660 shutdown 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 1 -- 172.21.15.32:0/2835617653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 msgr2=0x55dc00197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 0x55dc00197200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b34005f70 tx=0x7f6b3402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 1 -- 172.21.15.32:0/2835617653 shutdown_connections 2024-08-31T07:13:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55dc0020b9b0 0x55dc0020ddd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbffdd9070 0x55dc00192d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dbffdb01b0 0x55dc001926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 1 --2- 172.21.15.32:0/2835617653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dbffd6cda0 0x55dc00197200 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 1 -- 172.21.15.32:0/2835617653 >> 172.21.15.32:0/2835617653 conn(0x55dbffdcd460 msgr2=0x55dc0017fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:08.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b2cc00640 0 client.25660 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:13:08.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 1 -- 172.21.15.32:0/2835617653 shutdown_connections 2024-08-31T07:13:08.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 1 -- 172.21.15.32:0/2835617653 wait complete. 2024-08-31T07:13:08.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.242+0000 7f6b50c60740 20 client.25660 trim_cache size 0 max 16384 2024-08-31T07:13:08.249 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:13:08.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.243+0000 7f6b48a00640 1 -- 172.21.15.32:0/2280957292 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6b280b4ca0 msgr2=0x7f6b280b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:08.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.243+0000 7f6b48a00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6b280b4ca0 0x7f6b280b71b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6b34005ab0 tx=0x7f6b34005a20 comp rx=0 tx=0).stop 2024-08-31T07:13:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.243+0000 7f6b48a00640 1 -- 172.21.15.32:0/2280957292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b4410d9a0 msgr2=0x7f6b4419f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.243+0000 7f6b48a00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b4410d9a0 0x7f6b4419f910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00eb10 tx=0x7f6b3c00c2d0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.243+0000 7f6b48a00640 1 -- 172.21.15.32:0/2280957292 shutdown_connections 2024-08-31T07:13:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.243+0000 7f6b48a00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6b280b4ca0 0x7f6b280b71b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.243+0000 7f6b48a00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b4410d9a0 0x7f6b4419f910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.243+0000 7f6b48a00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b44104e20 0x7f6b4419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.243+0000 7f6b48a00640 1 --2- 172.21.15.32:0/2280957292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b441043f0 0x7f6b4419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.243+0000 7f6b48a00640 1 -- 172.21.15.32:0/2280957292 >> 172.21.15.32:0/2280957292 conn(0x7f6b440fe060 msgr2=0x7f6b44100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.244+0000 7f6b48a00640 1 -- 172.21.15.32:0/2280957292 shutdown_connections 2024-08-31T07:13:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:08.244+0000 7f6b48a00640 1 -- 172.21.15.32:0/2280957292 wait complete. 2024-08-31T07:13:09.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:13:09.269 INFO:tasks.quiescer.fs.[cephfs]:Total duration (52.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-31T07:13:09.269 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.4 2024-08-31T07:13:09.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:13:09.654 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:09.698 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:10.066 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:09 smithi032 ceph-mon[17252]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-08-31T07:13:10.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:09 smithi073 ceph-mon[21995]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-08-31T07:13:10.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:09 smithi084 ceph-mon[22387]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-08-31T07:13:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.396+0000 7ff7b8a00640 1 -- 172.21.15.32:0/1397658964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b4104770 msgr2=0x7ff7b4104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.396+0000 7ff7b8a00640 1 --2- 172.21.15.32:0/1397658964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b4104770 0x7ff7b4104b80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009900 tx=0x7ff7a402a410 comp rx=0 tx=0).stop 2024-08-31T07:13:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.396+0000 7ff7b8a00640 1 -- 172.21.15.32:0/1397658964 shutdown_connections 2024-08-31T07:13:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.396+0000 7ff7b8a00640 1 --2- 172.21.15.32:0/1397658964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7b410dbd0 0x7ff7b41100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.396+0000 7ff7b8a00640 1 --2- 172.21.15.32:0/1397658964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7b41050c0 0x7ff7b410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.396+0000 7ff7b8a00640 1 --2- 172.21.15.32:0/1397658964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b4104770 0x7ff7b4104b80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.396+0000 7ff7b8a00640 1 -- 172.21.15.32:0/1397658964 >> 172.21.15.32:0/1397658964 conn(0x7ff7b40794e0 msgr2=0x7ff7b40798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.396+0000 7ff7b8a00640 1 -- 172.21.15.32:0/1397658964 shutdown_connections 2024-08-31T07:13:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.396+0000 7ff7b8a00640 1 -- 172.21.15.32:0/1397658964 wait complete. 2024-08-31T07:13:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.397+0000 7ff7b8a00640 1 Processor -- start 2024-08-31T07:13:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b8a00640 1 -- start start 2024-08-31T07:13:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b8a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7b4104770 0x7ff7b419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7b41050c0 0x7ff7b419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b410dbd0 0x7ff7b41a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b8a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b4113880 con 0x7ff7b410dbd0 2024-08-31T07:13:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b8a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b4113700 con 0x7ff7b41050c0 2024-08-31T07:13:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b8a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b4113a00 con 0x7ff7b4104770 2024-08-31T07:13:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7b4104770 0x7ff7b419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b410dbd0 0x7ff7b41a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7b4104770 0x7ff7b419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50544/0 (socket says 172.21.15.32:50544) 2024-08-31T07:13:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.398+0000 7ff7b1e00640 1 -- 172.21.15.32:0/645932378 learned_addr learned my addr 172.21.15.32:0/645932378 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.399+0000 7ff7b1400640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7b41050c0 0x7ff7b419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.399+0000 7ff7b1e00640 1 -- 172.21.15.32:0/645932378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7b41050c0 msgr2=0x7ff7b419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.399+0000 7ff7b1e00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7b41050c0 0x7ff7b419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.399+0000 7ff7b1e00640 1 -- 172.21.15.32:0/645932378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b410dbd0 msgr2=0x7ff7b41a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.399+0000 7ff7b1e00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b410dbd0 0x7ff7b41a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.399+0000 7ff7b1e00640 1 -- 172.21.15.32:0/645932378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b41a2400 con 0x7ff7b4104770 2024-08-31T07:13:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.399+0000 7ff7b2800640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b410dbd0 0x7ff7b41a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.399+0000 7ff7b1e00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7b4104770 0x7ff7b419c490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4005ab0 tx=0x7ff7a4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.400+0000 7ff7aaa00640 1 -- 172.21.15.32:0/645932378 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a403f070 con 0x7ff7b4104770 2024-08-31T07:13:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.400+0000 7ff7b8a00640 1 -- 172.21.15.32:0/645932378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b41a2690 con 0x7ff7b4104770 2024-08-31T07:13:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.400+0000 7ff7aaa00640 1 -- 172.21.15.32:0/645932378 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a4031b50 con 0x7ff7b4104770 2024-08-31T07:13:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.400+0000 7ff7aaa00640 1 -- 172.21.15.32:0/645932378 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4035030 con 0x7ff7b4104770 2024-08-31T07:13:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.400+0000 7ff7b8a00640 1 -- 172.21.15.32:0/645932378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b41a8e50 con 0x7ff7b4104770 2024-08-31T07:13:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.403+0000 7ff7bf7e8740 1 Processor -- start 2024-08-31T07:13:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.403+0000 7ff7bf7e8740 1 -- start start 2024-08-31T07:13:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.404+0000 7ff7aaa00640 1 -- 172.21.15.32:0/645932378 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff7a4042070 con 0x7ff7b4104770 2024-08-31T07:13:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.404+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4d5070 0x55986c87a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.404+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c87aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.404+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4b4390 0x55986c87b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.404+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55986c7f17b0 con 0x55986c4ac1b0 2024-08-31T07:13:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.404+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55986c7e0d40 con 0x55986c4d5070 2024-08-31T07:13:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.404+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55986c7e4cf0 con 0x55986c4b4390 2024-08-31T07:13:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.404+0000 7ff7b2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4d5070 0x55986c87a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.404+0000 7ff7b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c87aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.404+0000 7ff7b1400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4b4390 0x55986c87b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.405+0000 7ff7b2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4d5070 0x55986c87a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38650/0 (socket says 172.21.15.32:38650) 2024-08-31T07:13:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.405+0000 7ff7b2800640 1 -- 172.21.15.32:0/2437372955 learned_addr learned my addr 172.21.15.32:0/2437372955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.405+0000 7ff7b2800640 1 -- 172.21.15.32:0/2437372955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4b4390 msgr2=0x55986c87b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.405+0000 7ff7b2800640 1 --2- 172.21.15.32:0/2437372955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4b4390 0x55986c87b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.405+0000 7ff7b2800640 1 -- 172.21.15.32:0/2437372955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 msgr2=0x55986c87aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:10.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.405+0000 7ff7b2800640 1 --2- 172.21.15.32:0/2437372955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c87aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.405+0000 7ff7b2800640 1 -- 172.21.15.32:0/2437372955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55986c540650 con 0x55986c4d5070 2024-08-31T07:13:10.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.405+0000 7ff7b2800640 1 --2- 172.21.15.32:0/2437372955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4d5070 0x55986c87a990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0107e0 tx=0x7ff7ac010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb990134a5db9801 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:10.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.406+0000 7ff79f000640 1 -- 172.21.15.32:0/2437372955 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac003b70 con 0x55986c4d5070 2024-08-31T07:13:10.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.406+0000 7ff79f000640 1 -- 172.21.15.32:0/2437372955 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7ac00ede0 con 0x55986c4d5070 2024-08-31T07:13:10.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.406+0000 7ff79f000640 1 -- 172.21.15.32:0/2437372955 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac004710 con 0x55986c4d5070 2024-08-31T07:13:10.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.406+0000 7ff7aaa00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7980b4d20 0x7ff7980b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.407+0000 7ff7b1e00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7980b4d20 0x7ff7980b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.407+0000 7ff7aaa00640 1 -- 172.21.15.32:0/645932378 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff7a4105b10 con 0x7ff7b4104770 2024-08-31T07:13:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.407+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2437372955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4d5070 msgr2=0x55986c87a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.407+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2437372955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4d5070 0x55986c87a990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0107e0 tx=0x7ff7ac010ca0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.408+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2437372955 shutdown_connections 2024-08-31T07:13:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.408+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2437372955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4b4390 0x55986c87b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.408+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2437372955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c87aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.408+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2437372955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4d5070 0x55986c87a990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.408+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2437372955 >> 172.21.15.32:0/2437372955 conn(0x55986c4c9460 msgr2=0x55986c878880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.408+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2437372955 shutdown_connections 2024-08-31T07:13:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.408+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2437372955 wait complete. 2024-08-31T07:13:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.408+0000 7ff7b1e00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7980b4d20 0x7ff7980b7230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55986c879610 tx=0x7ff7a41095c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.409+0000 7ff7bf7e8740 1 -- start start 2024-08-31T07:13:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.409+0000 7ff7bf7e8740 1 Processor -- start 2024-08-31T07:13:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.409+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c886650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c886c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c892460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55986c643700 con 0x55986c4ac1b0 2024-08-31T07:13:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55986c767cf0 con 0x55986c4b4390 2024-08-31T07:13:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55986c889170 con 0x55986c4d5070 2024-08-31T07:13:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7bf7e8740 10 client.0 _ll_register_callbacks cb 0x55986c78d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7b2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c886650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7b2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c886650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43588/0 (socket says 172.21.15.32:43588) 2024-08-31T07:13:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7b1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c886c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7b2800640 1 -- 172.21.15.32:0/1853677897 learned_addr learned my addr 172.21.15.32:0/1853677897 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.410+0000 7ff7b2800640 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c892460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.411+0000 7ff7b1400640 1 -- 172.21.15.32:0/1853677897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 msgr2=0x55986c892460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.411+0000 7ff7b1400640 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c892460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.411+0000 7ff7b1400640 1 -- 172.21.15.32:0/1853677897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 msgr2=0x55986c886650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.411+0000 7ff7b1400640 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c886650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.411+0000 7ff7b1400640 1 -- 172.21.15.32:0/1853677897 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55986c7a88b0 con 0x55986c4b4390 2024-08-31T07:13:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.411+0000 7ff7b1400640 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c886c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7b406b460 tx=0x7ff7a0012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f143214a17744700 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.411+0000 7ff793400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:13:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.411+0000 7ff793400640 1 -- 172.21.15.32:0/1853677897 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000f8f0 con 0x55986c4b4390 2024-08-31T07:13:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.411+0000 7ff793400640 1 -- 172.21.15.32:0/1853677897 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff7a000fa90 con 0x55986c4b4390 2024-08-31T07:13:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.411+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/1853677897 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7ac00ede0 con 0x55986c4b4390 2024-08-31T07:13:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff793400640 1 -- 172.21.15.32:0/1853677897 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7a000fc90 con 0x55986c4b4390 2024-08-31T07:13:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff7bf7e8740 10 client.25664 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/1853677897 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55986c895290 con 0x55986c4b4390 2024-08-31T07:13:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff793400640 1 -- 172.21.15.32:0/1853677897 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7ac00ede0 con 0x55986c4b4390 2024-08-31T07:13:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff793400640 1 -- 172.21.15.32:0/1853677897 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff7a001c420 con 0x55986c4b4390 2024-08-31T07:13:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff7bf7e8740 10 client.25664 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff7bf7e8740 10 client.25664 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:13:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff7bf7e8740 20 client.25664 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55986c4939f0 0x55986c898880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff7bf7e8740 4 client.25664 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:10.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.412+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/1853677897 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55986c7f3700 con 0x55986c4939f0 2024-08-31T07:13:10.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.413+0000 7ff7b2800640 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55986c4939f0 0x55986c898880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.414+0000 7ff793400640 10 client.25664 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:13:10.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.414+0000 7ff7b2800640 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55986c4939f0 0x55986c898880 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6526fd0230bed894 server_cookie=4e92e0665f4aa853 in_seq=0 out_seq=0 2024-08-31T07:13:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff793400640 1 -- 172.21.15.32:0/1853677897 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55986c7f3700 con 0x55986c4939f0 2024-08-31T07:13:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff793400640 10 client.25664 handle_command_reply: tid=0 2024-08-31T07:13:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff7bf7e8740 1 client.25664 shutdown 2024-08-31T07:13:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/1853677897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 msgr2=0x55986c886c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c886c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7b406b460 tx=0x7ff7a0012b60 comp rx=0 tx=0).stop 2024-08-31T07:13:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/1853677897 shutdown_connections 2024-08-31T07:13:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c892460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55986c4939f0 0x55986c898880 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c886c50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/1853677897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c886650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/1853677897 >> 172.21.15.32:0/1853677897 conn(0x55986c4c9460 msgr2=0x55986c87db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff793400640 0 client.25664 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:13:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.415+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/1853677897 shutdown_connections 2024-08-31T07:13:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.416+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/1853677897 wait complete. 2024-08-31T07:13:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.416+0000 7ff7bf7e8740 20 client.25664 trim_cache size 0 max 16384 2024-08-31T07:13:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.430+0000 7ff7bf7e8740 1 Processor -- start 2024-08-31T07:13:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.430+0000 7ff7bf7e8740 1 -- start start 2024-08-31T07:13:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.430+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c881c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.430+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c8801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.430+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c877e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.430+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b1550 con 0x55986c4ac1b0 2024-08-31T07:13:10.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.430+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b3090 con 0x55986c4b4390 2024-08-31T07:13:10.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.430+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b1c20 con 0x55986c4d5070 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c8801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c881c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c8801b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38658/0 (socket says 172.21.15.32:38658) 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b1400640 1 -- 172.21.15.32:0/2833087907 learned_addr learned my addr 172.21.15.32:0/2833087907 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b2800640 1 --2- 172.21.15.32:0/2833087907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c877e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b1400640 1 -- 172.21.15.32:0/2833087907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 msgr2=0x55986c877e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b1400640 1 --2- 172.21.15.32:0/2833087907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c877e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b1400640 1 -- 172.21.15.32:0/2833087907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 msgr2=0x55986c881c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b1400640 1 --2- 172.21.15.32:0/2833087907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c881c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b1400640 1 -- 172.21.15.32:0/2833087907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55986c887360 con 0x55986c4b4390 2024-08-31T07:13:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.431+0000 7ff7b2800640 1 --2- 172.21.15.32:0/2833087907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c881c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.432+0000 7ff7b1400640 1 --2- 172.21.15.32:0/2833087907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c8801b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7a001ca80 tx=0x7ff7a000f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ad27add75df3ae server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.432+0000 7ff79e600640 1 -- 172.21.15.32:0/2833087907 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0029030 con 0x55986c4b4390 2024-08-31T07:13:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.432+0000 7ff79e600640 1 -- 172.21.15.32:0/2833087907 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a000f9b0 con 0x55986c4b4390 2024-08-31T07:13:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.432+0000 7ff79e600640 1 -- 172.21.15.32:0/2833087907 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a001c420 con 0x55986c4b4390 2024-08-31T07:13:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.433+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2833087907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 msgr2=0x55986c8801b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.433+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2833087907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c8801b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7a001ca80 tx=0x7ff7a000f810 comp rx=0 tx=0).stop 2024-08-31T07:13:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.433+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2833087907 shutdown_connections 2024-08-31T07:13:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.433+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2833087907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c877e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.433+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2833087907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4b4390 0x55986c8801b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.433+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2833087907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4ac1b0 0x55986c881c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.433+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2833087907 >> 172.21.15.32:0/2833087907 conn(0x55986c4c9460 msgr2=0x55986c87faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.433+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2833087907 shutdown_connections 2024-08-31T07:13:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.433+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2833087907 wait complete. 2024-08-31T07:13:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.434+0000 7ff7bf7e8740 1 -- start start 2024-08-31T07:13:10.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.434+0000 7ff7bf7e8740 1 Processor -- start 2024-08-31T07:13:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4ac1b0 0x55986c88ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4b4390 0x55986c889f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7bf7e8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c88a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b29c0 con 0x55986c4b4390 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b22f0 con 0x55986c4ac1b0 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7bf7e8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41af530 con 0x55986c4d5070 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7bf7e8740 10 client.0 _ll_register_callbacks cb 0x7ff7a0009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4b4390 0x55986c889f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4b4390 0x55986c889f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43606/0 (socket says 172.21.15.32:43606) 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b1400640 1 -- 172.21.15.32:0/2460120587 learned_addr learned my addr 172.21.15.32:0/2460120587 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b2800640 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4ac1b0 0x55986c88ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b2800640 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c88a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b1400640 1 -- 172.21.15.32:0/2460120587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 msgr2=0x55986c88a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b1400640 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c88a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b1400640 1 -- 172.21.15.32:0/2460120587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4ac1b0 msgr2=0x55986c88ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b1400640 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4ac1b0 0x55986c88ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b1400640 1 -- 172.21.15.32:0/2460120587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55986c88aae0 con 0x55986c4b4390 2024-08-31T07:13:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.435+0000 7ff7b2800640 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c88a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.436+0000 7ff7b1400640 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4b4390 0x55986c889f30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff7a001c9b0 tx=0x7ff7a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bbc2aa6ae6eb62 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.436+0000 7ff793400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:13:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.436+0000 7ff793400640 1 -- 172.21.15.32:0/2460120587 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0005250 con 0x55986c4b4390 2024-08-31T07:13:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.436+0000 7ff793400640 1 -- 172.21.15.32:0/2460120587 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff7a00053f0 con 0x55986c4b4390 2024-08-31T07:13:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.436+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2460120587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a000f9b0 con 0x55986c4b4390 2024-08-31T07:13:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.436+0000 7ff793400640 1 -- 172.21.15.32:0/2460120587 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7a003d050 con 0x55986c4b4390 2024-08-31T07:13:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.436+0000 7ff7bf7e8740 10 client.15663 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.436+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2460120587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55986c9030b0 con 0x55986c4b4390 2024-08-31T07:13:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.437+0000 7ff793400640 1 -- 172.21.15.32:0/2460120587 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a000f9b0 con 0x55986c4b4390 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.437+0000 7ff793400640 1 -- 172.21.15.32:0/2460120587 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff7a0010430 con 0x55986c4b4390 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.437+0000 7ff7bf7e8740 10 client.15663 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.437+0000 7ff7bf7e8740 10 client.15663 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.437+0000 7ff7bf7e8740 20 client.15663 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.437+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55986c468da0 0x55986c905820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.437+0000 7ff7bf7e8740 4 client.15663 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.437+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2460120587 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55986c7ef1a0 con 0x55986c468da0 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.437+0000 7ff7b2800640 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55986c468da0 0x55986c905820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.438+0000 7ff7b2800640 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55986c468da0 0x55986c905820 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa4376025ce73711 server_cookie=df58d6f385b0741f in_seq=0 out_seq=0 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.438+0000 7ff793400640 10 client.15663 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:13:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.438+0000 7ff793400640 1 -- 172.21.15.32:0/2460120587 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55986c7ef1a0 con 0x55986c468da0 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.438+0000 7ff793400640 10 client.15663 handle_command_reply: tid=0 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.438+0000 7ff7bf7e8740 1 client.15663 shutdown 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2460120587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4b4390 msgr2=0x55986c889f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4b4390 0x55986c889f30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff7a001c9b0 tx=0x7ff7a0005f90 comp rx=0 tx=0).stop 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2460120587 shutdown_connections 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55986c4d5070 0x55986c88a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55986c4b4390 0x55986c889f30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55986c468da0 0x55986c905820 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 1 --2- 172.21.15.32:0/2460120587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55986c4ac1b0 0x55986c88ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2460120587 >> 172.21.15.32:0/2460120587 conn(0x55986c4c9460 msgr2=0x55986c884cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff793400640 0 client.15663 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2460120587 shutdown_connections 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 1 -- 172.21.15.32:0/2460120587 wait complete. 2024-08-31T07:13:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.439+0000 7ff7bf7e8740 20 client.15663 trim_cache size 0 max 16384 2024-08-31T07:13:10.444 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:13:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 -- 172.21.15.32:0/645932378 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7980b4d20 msgr2=0x7ff7980b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7980b4d20 0x7ff7980b7230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55986c879610 tx=0x7ff7a41095c0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 -- 172.21.15.32:0/645932378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7b4104770 msgr2=0x7ff7b419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7b4104770 0x7ff7b419c490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4005ab0 tx=0x7ff7a4003b90 comp rx=0 tx=0).stop 2024-08-31T07:13:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 -- 172.21.15.32:0/645932378 shutdown_connections 2024-08-31T07:13:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7980b4d20 0x7ff7980b7230 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b410dbd0 0x7ff7b41a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7b41050c0 0x7ff7b419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 --2- 172.21.15.32:0/645932378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7b4104770 0x7ff7b419c490 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 -- 172.21.15.32:0/645932378 >> 172.21.15.32:0/645932378 conn(0x7ff7b40794e0 msgr2=0x7ff7b41099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 -- 172.21.15.32:0/645932378 shutdown_connections 2024-08-31T07:13:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:10.440+0000 7ff7b8a00640 1 -- 172.21.15.32:0/645932378 wait complete. 2024-08-31T07:13:11.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:13:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.418+0000 7f54c5800640 1 -- 172.21.15.32:0/2301684953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54c0075ea0 msgr2=0x7f54c010f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.418+0000 7f54c5800640 1 --2- 172.21.15.32:0/2301684953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54c0075ea0 0x7f54c010f770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009900 tx=0x7f54ac02a400 comp rx=0 tx=0).stop 2024-08-31T07:13:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.418+0000 7f54c5800640 1 -- 172.21.15.32:0/2301684953 shutdown_connections 2024-08-31T07:13:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.418+0000 7f54c5800640 1 --2- 172.21.15.32:0/2301684953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54c010fcb0 0x7f54c01120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.418+0000 7f54c5800640 1 --2- 172.21.15.32:0/2301684953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54c0075ea0 0x7f54c010f770 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.418+0000 7f54c5800640 1 --2- 172.21.15.32:0/2301684953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c0075550 0x7f54c0075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.418+0000 7f54c5800640 1 -- 172.21.15.32:0/2301684953 >> 172.21.15.32:0/2301684953 conn(0x7f54c00ff1e0 msgr2=0x7f54c0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.418+0000 7f54c5800640 1 -- 172.21.15.32:0/2301684953 shutdown_connections 2024-08-31T07:13:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.418+0000 7f54c5800640 1 -- 172.21.15.32:0/2301684953 wait complete. 2024-08-31T07:13:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.420+0000 7f54c5800640 1 Processor -- start 2024-08-31T07:13:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.420+0000 7f54c5800640 1 -- start start 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54c5800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c0075550 0x7f54c019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54c5800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54c0075ea0 0x7f54c019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54c5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54c010fcb0 0x7f54c01a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54c5800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54c0115870 con 0x7f54c0075ea0 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54c5800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01156f0 con 0x7f54c010fcb0 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54c5800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01159f0 con 0x7f54c0075550 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54bfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c0075550 0x7f54c019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54bfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c0075550 0x7f54c019c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50608/0 (socket says 172.21.15.32:50608) 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54bfe00640 1 -- 172.21.15.32:0/3590377076 learned_addr learned my addr 172.21.15.32:0/3590377076 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54bf400640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54c0075ea0 0x7f54c019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54c4e00640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54c010fcb0 0x7f54c01a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54bfe00640 1 -- 172.21.15.32:0/3590377076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54c010fcb0 msgr2=0x7f54c01a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54bfe00640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54c010fcb0 0x7f54c01a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54bfe00640 1 -- 172.21.15.32:0/3590377076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54c0075ea0 msgr2=0x7f54c019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54bfe00640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54c0075ea0 0x7f54c019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.421+0000 7f54bfe00640 1 -- 172.21.15.32:0/3590377076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c01a2270 con 0x7f54c0075550 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.422+0000 7f54bfe00640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c0075550 0x7f54c019c300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f54b000ea60 tx=0x7f54b000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.422+0000 7f54bcc00640 1 -- 172.21.15.32:0/3590377076 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b000cd20 con 0x7f54c0075550 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.422+0000 7f54c5800640 1 -- 172.21.15.32:0/3590377076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c01a2560 con 0x7f54c0075550 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.422+0000 7f54bcc00640 1 -- 172.21.15.32:0/3590377076 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b00035f0 con 0x7f54c0075550 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.422+0000 7f54bcc00640 1 -- 172.21.15.32:0/3590377076 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0010690 con 0x7f54c0075550 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.422+0000 7f54c5800640 1 -- 172.21.15.32:0/3590377076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c01a8cc0 con 0x7f54c0075550 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.424+0000 7f54c5800640 1 -- 172.21.15.32:0/3590377076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c0075960 con 0x7f54c0075550 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.428+0000 7f54bcc00640 1 -- 172.21.15.32:0/3590377076 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f54b00037a0 con 0x7f54c0075550 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.429+0000 7f54bcc00640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f54a40b4ca0 0x7f54a40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.429+0000 7f54bcc00640 1 -- 172.21.15.32:0/3590377076 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f54b00e58a0 con 0x7f54c0075550 2024-08-31T07:13:11.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.430+0000 7f54bf400640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f54a40b4ca0 0x7f54a40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:11.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.431+0000 7f54bf400640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f54a40b4ca0 0x7f54a40b71b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f54ac006010 tx=0x7f54ac031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:11.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.433+0000 7f54bcc00640 1 -- 172.21.15.32:0/3590377076 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f54b00ac710 con 0x7f54c0075550 2024-08-31T07:13:11.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.807+0000 7f54c5800640 1 -- 172.21.15.32:0/3590377076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54c01a8fa0 con 0x7f54c0075550 2024-08-31T07:13:11.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.808+0000 7f54bcc00640 1 -- 172.21.15.32:0/3590377076 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f54b00ac010 con 0x7f54c0075550 2024-08-31T07:13:11.813 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:13:11.813 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 -- 172.21.15.32:0/3590377076 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f54a40b4ca0 msgr2=0x7f54a40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f54a40b4ca0 0x7f54a40b71b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f54ac006010 tx=0x7f54ac031400 comp rx=0 tx=0).stop 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 -- 172.21.15.32:0/3590377076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c0075550 msgr2=0x7f54c019c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c0075550 0x7f54c019c300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f54b000ea60 tx=0x7f54b000ee60 comp rx=0 tx=0).stop 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 -- 172.21.15.32:0/3590377076 shutdown_connections 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f54a40b4ca0 0x7f54a40b71b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54c010fcb0 0x7f54c01a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54c0075ea0 0x7f54c019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 --2- 172.21.15.32:0/3590377076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54c0075550 0x7f54c019c300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 -- 172.21.15.32:0/3590377076 >> 172.21.15.32:0/3590377076 conn(0x7f54c00ff1e0 msgr2=0x7f54c0109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.811+0000 7f54c5800640 1 -- 172.21.15.32:0/3590377076 shutdown_connections 2024-08-31T07:13:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:11.812+0000 7f54c5800640 1 -- 172.21.15.32:0/3590377076 wait complete. 2024-08-31T07:13:12.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:11 smithi032 ceph-mon[17252]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-08-31T07:13:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:11 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3590377076' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:12.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:11 smithi073 ceph-mon[21995]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-08-31T07:13:12.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:11 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3590377076' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:12.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:11 smithi084 ceph-mon[22387]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-08-31T07:13:12.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:11 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3590377076' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:12.652 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0' 2024-08-31T07:13:12.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0 2024-08-31T07:13:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.459+0000 7fce1fa00640 1 -- 172.21.15.32:0/4036243310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce18075e40 msgr2=0x7fce1810f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.459+0000 7fce1fa00640 1 --2- 172.21.15.32:0/4036243310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce18075e40 0x7fce1810f980 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce04009900 tx=0x7fce0402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.459+0000 7fce1fa00640 1 -- 172.21.15.32:0/4036243310 shutdown_connections 2024-08-31T07:13:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.459+0000 7fce1fa00640 1 --2- 172.21.15.32:0/4036243310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1810fec0 0x7fce181122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.459+0000 7fce1fa00640 1 --2- 172.21.15.32:0/4036243310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce18075e40 0x7fce1810f980 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.459+0000 7fce1fa00640 1 --2- 172.21.15.32:0/4036243310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce180754f0 0x7fce18075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.459+0000 7fce1fa00640 1 -- 172.21.15.32:0/4036243310 >> 172.21.15.32:0/4036243310 conn(0x7fce1807b5f0 msgr2=0x7fce1807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.459+0000 7fce1fa00640 1 -- 172.21.15.32:0/4036243310 shutdown_connections 2024-08-31T07:13:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.460+0000 7fce1fa00640 1 -- 172.21.15.32:0/4036243310 wait complete. 2024-08-31T07:13:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.461+0000 7fce1fa00640 1 Processor -- start 2024-08-31T07:13:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.461+0000 7fce1fa00640 1 -- start start 2024-08-31T07:13:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce1fa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce180754f0 0x7fce18074d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce1fa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce18075e40 0x7fce1806e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce1fa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce1810fec0 0x7fce1806e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce1fa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce18115a80 con 0x7fce18075e40 2024-08-31T07:13:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce1fa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce18115900 con 0x7fce180754f0 2024-08-31T07:13:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce1fa00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce18115c00 con 0x7fce1810fec0 2024-08-31T07:13:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce1d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce180754f0 0x7fce18074d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce1da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce1810fec0 0x7fce1806e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce1d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce180754f0 0x7fce18074d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38710/0 (socket says 172.21.15.32:38710) 2024-08-31T07:13:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce1d000640 1 -- 172.21.15.32:0/1380474461 learned_addr learned my addr 172.21.15.32:0/1380474461 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.462+0000 7fce17e00640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce18075e40 0x7fce1806e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.463+0000 7fce1d000640 1 -- 172.21.15.32:0/1380474461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce1810fec0 msgr2=0x7fce1806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.463+0000 7fce1d000640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce1810fec0 0x7fce1806e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.463+0000 7fce1d000640 1 -- 172.21.15.32:0/1380474461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce18075e40 msgr2=0x7fce1806e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.463+0000 7fce1d000640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce18075e40 0x7fce1806e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.463+0000 7fce1d000640 1 -- 172.21.15.32:0/1380474461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce180739e0 con 0x7fce180754f0 2024-08-31T07:13:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.463+0000 7fce1da00640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce1810fec0 0x7fce1806e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.464+0000 7fce1d000640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce180754f0 0x7fce18074d70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00ea60 tx=0x7fce0c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.464+0000 7fce15600640 1 -- 172.21.15.32:0/1380474461 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c00cde0 con 0x7fce180754f0 2024-08-31T07:13:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.464+0000 7fce15600640 1 -- 172.21.15.32:0/1380474461 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0c0035f0 con 0x7fce180754f0 2024-08-31T07:13:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.464+0000 7fce15600640 1 -- 172.21.15.32:0/1380474461 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c010690 con 0x7fce180754f0 2024-08-31T07:13:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.464+0000 7fce1fa00640 1 -- 172.21.15.32:0/1380474461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce18073cd0 con 0x7fce180754f0 2024-08-31T07:13:13.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.464+0000 7fce1fa00640 1 -- 172.21.15.32:0/1380474461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce18074190 con 0x7fce180754f0 2024-08-31T07:13:13.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.469+0000 7fce26804740 1 Processor -- start 2024-08-31T07:13:13.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.469+0000 7fce15600640 1 -- 172.21.15.32:0/1380474461 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fce0c0037a0 con 0x7fce180754f0 2024-08-31T07:13:13.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.469+0000 7fce26804740 1 -- start start 2024-08-31T07:13:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.470+0000 7fce26804740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e81766070 0x557e81b0b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.470+0000 7fce26804740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e8173d1b0 0x557e81b0bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.470+0000 7fce26804740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 0x557e81b0c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.470+0000 7fce26804740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557e81a827b0 con 0x557e81766070 2024-08-31T07:13:13.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.470+0000 7fce26804740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557e81a71d40 con 0x557e81745390 2024-08-31T07:13:13.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.470+0000 7fce26804740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557e81a75cf0 con 0x557e8173d1b0 2024-08-31T07:13:13.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.470+0000 7fce1d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e8173d1b0 0x557e81b0bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.471+0000 7fce1d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e8173d1b0 0x557e81b0bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50642/0 (socket says 172.21.15.32:50642) 2024-08-31T07:13:13.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.471+0000 7fce1d000640 1 -- 172.21.15.32:0/2198980417 learned_addr learned my addr 172.21.15.32:0/2198980417 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:13.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.471+0000 7fce17e00640 1 --2- 172.21.15.32:0/2198980417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 0x557e81b0c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.471+0000 7fce15600640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce000b4da0 0x7fce000b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.471+0000 7fce1d000640 1 -- 172.21.15.32:0/2198980417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 msgr2=0x557e81b0c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.471+0000 7fce1d000640 1 --2- 172.21.15.32:0/2198980417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 0x557e81b0c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.472+0000 7fce1d000640 1 -- 172.21.15.32:0/2198980417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e81766070 msgr2=0x557e81b0b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.472+0000 7fce1d000640 1 --2- 172.21.15.32:0/2198980417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e81766070 0x557e81b0b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.472+0000 7fce1d000640 1 -- 172.21.15.32:0/2198980417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e817d1650 con 0x557e8173d1b0 2024-08-31T07:13:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.472+0000 7fce17e00640 1 --2- 172.21.15.32:0/2198980417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 0x557e81b0c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.472+0000 7fce1da00640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce000b4da0 0x7fce000b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.472+0000 7fce15600640 1 -- 172.21.15.32:0/1380474461 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fce0c0e5730 con 0x7fce180754f0 2024-08-31T07:13:13.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.473+0000 7fce1d000640 1 --2- 172.21.15.32:0/2198980417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e8173d1b0 0x557e81b0bed0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce0c0e2de0 tx=0x7fce0c0707f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b93e9ade956da1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:13.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.473+0000 7fce0a000640 1 -- 172.21.15.32:0/2198980417 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c0abc10 con 0x557e8173d1b0 2024-08-31T07:13:13.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.473+0000 7fce0a000640 1 -- 172.21.15.32:0/2198980417 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0c0ab950 con 0x557e8173d1b0 2024-08-31T07:13:13.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.473+0000 7fce0a000640 1 -- 172.21.15.32:0/2198980417 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c0a1350 con 0x557e8173d1b0 2024-08-31T07:13:13.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.474+0000 7fce26804740 1 -- 172.21.15.32:0/2198980417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e8173d1b0 msgr2=0x557e81b0bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:13.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.474+0000 7fce1da00640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce000b4da0 0x7fce000b72b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce10009fd0 tx=0x7fce10009920 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:13.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.474+0000 7fce26804740 1 --2- 172.21.15.32:0/2198980417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e8173d1b0 0x557e81b0bed0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce0c0e2de0 tx=0x7fce0c0707f0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.475+0000 7fce26804740 1 -- 172.21.15.32:0/2198980417 shutdown_connections 2024-08-31T07:13:13.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.475+0000 7fce26804740 1 --2- 172.21.15.32:0/2198980417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 0x557e81b0c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.475+0000 7fce26804740 1 --2- 172.21.15.32:0/2198980417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e8173d1b0 0x557e81b0bed0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.475+0000 7fce26804740 1 --2- 172.21.15.32:0/2198980417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e81766070 0x557e81b0b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.475+0000 7fce26804740 1 -- 172.21.15.32:0/2198980417 >> 172.21.15.32:0/2198980417 conn(0x557e8175a460 msgr2=0x557e81b09880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:13.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.475+0000 7fce26804740 1 -- 172.21.15.32:0/2198980417 shutdown_connections 2024-08-31T07:13:13.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.475+0000 7fce26804740 1 -- 172.21.15.32:0/2198980417 wait complete. 2024-08-31T07:13:13.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.476+0000 7fce26804740 1 -- start start 2024-08-31T07:13:13.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.476+0000 7fce26804740 1 Processor -- start 2024-08-31T07:13:13.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.477+0000 7fce26804740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 0x557e81b17510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.477+0000 7fce26804740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81745390 0x557e81b17a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.477+0000 7fce26804740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81766070 0x557e81b23400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.477+0000 7fce26804740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557e818d4700 con 0x557e8173d1b0 2024-08-31T07:13:13.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.477+0000 7fce26804740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557e819f8cf0 con 0x557e81766070 2024-08-31T07:13:13.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.477+0000 7fce26804740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557e81b1a110 con 0x557e81745390 2024-08-31T07:13:13.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.477+0000 7fce26804740 10 client.0 _ll_register_callbacks cb 0x557e81a1e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:13.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.478+0000 7fce1da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81766070 0x557e81b23400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.478+0000 7fce1da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81766070 0x557e81b23400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38718/0 (socket says 172.21.15.32:38718) 2024-08-31T07:13:13.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.478+0000 7fce1da00640 1 -- 172.21.15.32:0/2525859490 learned_addr learned my addr 172.21.15.32:0/2525859490 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:13.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.478+0000 7fce17e00640 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81745390 0x557e81b17a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.478+0000 7fce1d000640 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 0x557e81b17510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.478+0000 7fce17e00640 1 -- 172.21.15.32:0/2525859490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81766070 msgr2=0x557e81b23400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:13.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.478+0000 7fce17e00640 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81766070 0x557e81b23400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.478+0000 7fce17e00640 1 -- 172.21.15.32:0/2525859490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 msgr2=0x557e81b17510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:13.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.478+0000 7fce17e00640 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 0x557e81b17510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.478+0000 7fce17e00640 1 -- 172.21.15.32:0/2525859490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e81a398b0 con 0x557e81745390 2024-08-31T07:13:13.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.479+0000 7fce17e00640 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81745390 0x557e81b17a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fce04009230 tx=0x7fce04004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10acad9f2ec2dfb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:13.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.479+0000 7fcdfea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:13.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.479+0000 7fcdfea00640 1 -- 172.21.15.32:0/2525859490 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04035040 con 0x557e81745390 2024-08-31T07:13:13.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.480+0000 7fcdfea00640 1 -- 172.21.15.32:0/2525859490 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fce04005510 con 0x557e81745390 2024-08-31T07:13:13.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.480+0000 7fce26804740 1 -- 172.21.15.32:0/2525859490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce0c0ab950 con 0x557e81745390 2024-08-31T07:13:13.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.480+0000 7fcdfea00640 1 -- 172.21.15.32:0/2525859490 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce04004750 con 0x557e81745390 2024-08-31T07:13:13.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.480+0000 7fce26804740 10 client.25696 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:13.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.480+0000 7fce26804740 1 -- 172.21.15.32:0/2525859490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e81b260f0 con 0x557e81745390 2024-08-31T07:13:13.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.481+0000 7fcdfea00640 1 -- 172.21.15.32:0/2525859490 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0c0ab950 con 0x557e81745390 2024-08-31T07:13:13.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.481+0000 7fcdfea00640 1 -- 172.21.15.32:0/2525859490 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce04003ae0 con 0x557e81745390 2024-08-31T07:13:13.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.481+0000 7fce26804740 10 client.25696 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:13.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.481+0000 7fce26804740 10 client.25696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:13:13.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.481+0000 7fce26804740 20 client.25696 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:13.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.481+0000 7fce26804740 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557e817249f0 0x557e81b29510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.481+0000 7fce26804740 4 client.25696 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:13.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.481+0000 7fce26804740 1 -- 172.21.15.32:0/2525859490 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e81a84700 con 0x557e817249f0 2024-08-31T07:13:13.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.482+0000 7fce1d000640 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557e817249f0 0x557e81b29510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.484+0000 7fcdfea00640 10 client.25696 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.484+0000 7fce1d000640 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557e817249f0 0x557e81b29510 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2db08e528dff23 server_cookie=da6d467e8eac8c9b in_seq=0 out_seq=0 2024-08-31T07:13:13.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.485+0000 7fcdfea00640 1 -- 172.21.15.32:0/2525859490 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e81a84700 con 0x557e817249f0 2024-08-31T07:13:13.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.485+0000 7fcdfea00640 10 client.25696 handle_command_reply: tid=0 2024-08-31T07:13:13.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.485+0000 7fce26804740 1 client.25696 shutdown 2024-08-31T07:13:13.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.486+0000 7fce26804740 1 -- 172.21.15.32:0/2525859490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81745390 msgr2=0x557e81b17a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:13.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.486+0000 7fce26804740 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81745390 0x557e81b17a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fce04009230 tx=0x7fce04004510 comp rx=0 tx=0).stop 2024-08-31T07:13:13.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.486+0000 7fce26804740 1 -- 172.21.15.32:0/2525859490 shutdown_connections 2024-08-31T07:13:13.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.486+0000 7fce26804740 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557e817249f0 0x557e81b29510 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.486+0000 7fce26804740 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81766070 0x557e81b23400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.486+0000 7fce26804740 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81745390 0x557e81b17a50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.486+0000 7fce26804740 1 --2- 172.21.15.32:0/2525859490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 0x557e81b17510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.486+0000 7fcdfea00640 0 client.25696 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:13.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.486+0000 7fce26804740 1 -- 172.21.15.32:0/2525859490 >> 172.21.15.32:0/2525859490 conn(0x557e8175a460 msgr2=0x557e81b0c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:13.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.486+0000 7fce26804740 1 -- 172.21.15.32:0/2525859490 shutdown_connections 2024-08-31T07:13:13.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.487+0000 7fce26804740 1 -- 172.21.15.32:0/2525859490 wait complete. 2024-08-31T07:13:13.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.487+0000 7fce26804740 20 client.25696 trim_cache size 0 max 16384 2024-08-31T07:13:13.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.502+0000 7fce26804740 1 Processor -- start 2024-08-31T07:13:13.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.502+0000 7fce26804740 1 -- start start 2024-08-31T07:13:13.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.502+0000 7fce26804740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8173d1b0 0x557e81b0d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.502+0000 7fce26804740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e81745390 0x557e81b18f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.502+0000 7fce26804740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 0x557e81b23300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.502+0000 7fce26804740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce181be290 con 0x557e81745390 2024-08-31T07:13:13.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.502+0000 7fce26804740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce181bfdd0 con 0x557e8173d1b0 2024-08-31T07:13:13.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.502+0000 7fce26804740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce181be960 con 0x557e81766070 2024-08-31T07:13:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.503+0000 7fce1d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8173d1b0 0x557e81b0d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.503+0000 7fce1d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8173d1b0 0x557e81b0d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38728/0 (socket says 172.21.15.32:38728) 2024-08-31T07:13:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.503+0000 7fce1d000640 1 -- 172.21.15.32:0/3112076974 learned_addr learned my addr 172.21.15.32:0/3112076974 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.503+0000 7fce17e00640 1 --2- 172.21.15.32:0/3112076974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e81745390 0x557e81b18f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.503+0000 7fce1da00640 1 --2- 172.21.15.32:0/3112076974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 0x557e81b23300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.504+0000 7fce17e00640 1 -- 172.21.15.32:0/3112076974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 msgr2=0x557e81b23300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.504+0000 7fce17e00640 1 --2- 172.21.15.32:0/3112076974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 0x557e81b23300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.504+0000 7fce17e00640 1 -- 172.21.15.32:0/3112076974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8173d1b0 msgr2=0x557e81b0d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.504+0000 7fce17e00640 1 --2- 172.21.15.32:0/3112076974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8173d1b0 0x557e81b0d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.504+0000 7fce17e00640 1 -- 172.21.15.32:0/3112076974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e81b0e7f0 con 0x557e81745390 2024-08-31T07:13:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.504+0000 7fce1da00640 1 --2- 172.21.15.32:0/3112076974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 0x557e81b23300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.504+0000 7fce17e00640 1 --2- 172.21.15.32:0/3112076974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e81745390 0x557e81b18f40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fce04031440 tx=0x7fce04005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a25222e8f3a0969 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.505+0000 7fce09600640 1 -- 172.21.15.32:0/3112076974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04035070 con 0x557e81745390 2024-08-31T07:13:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.505+0000 7fce09600640 1 -- 172.21.15.32:0/3112076974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce04041070 con 0x557e81745390 2024-08-31T07:13:13.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.505+0000 7fce09600640 1 -- 172.21.15.32:0/3112076974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04004b10 con 0x557e81745390 2024-08-31T07:13:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.506+0000 7fce26804740 1 -- 172.21.15.32:0/3112076974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e81745390 msgr2=0x557e81b18f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.506+0000 7fce26804740 1 --2- 172.21.15.32:0/3112076974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e81745390 0x557e81b18f40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fce04031440 tx=0x7fce04005d50 comp rx=0 tx=0).stop 2024-08-31T07:13:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.506+0000 7fce26804740 1 -- 172.21.15.32:0/3112076974 shutdown_connections 2024-08-31T07:13:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.506+0000 7fce26804740 1 --2- 172.21.15.32:0/3112076974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 0x557e81b23300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.506+0000 7fce26804740 1 --2- 172.21.15.32:0/3112076974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e81745390 0x557e81b18f40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.506+0000 7fce26804740 1 --2- 172.21.15.32:0/3112076974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e8173d1b0 0x557e81b0d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.506+0000 7fce26804740 1 -- 172.21.15.32:0/3112076974 >> 172.21.15.32:0/3112076974 conn(0x557e8175a460 msgr2=0x557e81b18830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:13.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.507+0000 7fce26804740 1 -- 172.21.15.32:0/3112076974 shutdown_connections 2024-08-31T07:13:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.507+0000 7fce26804740 1 -- 172.21.15.32:0/3112076974 wait complete. 2024-08-31T07:13:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.508+0000 7fce26804740 1 -- start start 2024-08-31T07:13:13.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.508+0000 7fce26804740 1 Processor -- start 2024-08-31T07:13:13.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.509+0000 7fce26804740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 0x557e81b1bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.510+0000 7fce26804740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 0x557e81b1c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.510+0000 7fce26804740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 0x557e81b92ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.510+0000 7fce26804740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce181bf700 con 0x557e8173d1b0 2024-08-31T07:13:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.510+0000 7fce1d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 0x557e81b1bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.511+0000 7fce1d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 0x557e81b1bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43644/0 (socket says 172.21.15.32:43644) 2024-08-31T07:13:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.511+0000 7fce1d000640 1 -- 172.21.15.32:0/646251301 learned_addr learned my addr 172.21.15.32:0/646251301 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.511+0000 7fce1da00640 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 0x557e81b92ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.511+0000 7fce26804740 1 -- 172.21.15.32:0/646251301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce181bf030 con 0x557e81745390 2024-08-31T07:13:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.511+0000 7fce17e00640 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 0x557e81b1c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.511+0000 7fce26804740 1 -- 172.21.15.32:0/646251301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce181bc2e0 con 0x557e81766070 2024-08-31T07:13:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.512+0000 7fce26804740 10 client.0 _ll_register_callbacks cb 0x7fce040041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.512+0000 7fce1da00640 1 -- 172.21.15.32:0/646251301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 msgr2=0x557e81b1c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.512+0000 7fce1da00640 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 0x557e81b1c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.512+0000 7fce1da00640 1 -- 172.21.15.32:0/646251301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 msgr2=0x557e81b1bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.512+0000 7fce1da00640 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 0x557e81b1bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.512+0000 7fce1da00640 1 -- 172.21.15.32:0/646251301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e81b93540 con 0x557e81766070 2024-08-31T07:13:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.512+0000 7fce1d000640 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 0x557e81b1bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:13.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.514+0000 7fce1da00640 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 0x557e81b92ed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce10005fd0 tx=0x7fce100096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30a40df002ca8e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:13.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.514+0000 7fcdfea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:13.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.514+0000 7fcdfea00640 1 -- 172.21.15.32:0/646251301 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10011da0 con 0x557e81766070 2024-08-31T07:13:13.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.514+0000 7fcdfea00640 1 -- 172.21.15.32:0/646251301 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fce10020410 con 0x557e81766070 2024-08-31T07:13:13.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.514+0000 7fce26804740 1 -- 172.21.15.32:0/646251301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce04041070 con 0x557e81766070 2024-08-31T07:13:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.515+0000 7fcdfea00640 1 -- 172.21.15.32:0/646251301 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce10020610 con 0x557e81766070 2024-08-31T07:13:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.515+0000 7fce26804740 10 client.25702 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.515+0000 7fce26804740 1 -- 172.21.15.32:0/646251301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e81b94810 con 0x557e81766070 2024-08-31T07:13:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.515+0000 7fcdfea00640 1 -- 172.21.15.32:0/646251301 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce04041070 con 0x557e81766070 2024-08-31T07:13:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.516+0000 7fcdfea00640 1 -- 172.21.15.32:0/646251301 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce10033850 con 0x557e81766070 2024-08-31T07:13:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.516+0000 7fce26804740 10 client.25702 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.516+0000 7fce26804740 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:13:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.516+0000 7fce26804740 20 client.25702 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.516+0000 7fce26804740 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557e816f9da0 0x557e81b96fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.516+0000 7fce26804740 4 client.25702 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:13:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.516+0000 7fce26804740 1 -- 172.21.15.32:0/646251301 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e81a801a0 con 0x557e816f9da0 2024-08-31T07:13:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.517+0000 7fce1d000640 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557e816f9da0 0x557e81b96fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.519+0000 7fce1d000640 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557e816f9da0 0x557e81b96fd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28cdd9e1a2000cd6 server_cookie=ef0d03844293577f in_seq=0 out_seq=0 2024-08-31T07:13:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.519+0000 7fcdfea00640 10 client.25702 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.519+0000 7fcdfea00640 1 -- 172.21.15.32:0/646251301 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e81a801a0 con 0x557e816f9da0 2024-08-31T07:13:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.519+0000 7fcdfea00640 10 client.25702 handle_command_reply: tid=0 2024-08-31T07:13:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.519+0000 7fce26804740 1 client.25702 shutdown 2024-08-31T07:13:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.520+0000 7fce26804740 1 -- 172.21.15.32:0/646251301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 msgr2=0x557e81b92ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.520+0000 7fce26804740 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 0x557e81b92ed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce10005fd0 tx=0x7fce100096a0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.520+0000 7fce26804740 1 -- 172.21.15.32:0/646251301 shutdown_connections 2024-08-31T07:13:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.520+0000 7fce26804740 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557e816f9da0 0x557e81b96fd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.520+0000 7fce26804740 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e81766070 0x557e81b92ed0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.520+0000 7fce26804740 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e81745390 0x557e81b1c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.520+0000 7fce26804740 1 --2- 172.21.15.32:0/646251301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557e8173d1b0 0x557e81b1bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.520+0000 7fce26804740 1 -- 172.21.15.32:0/646251301 >> 172.21.15.32:0/646251301 conn(0x557e8175a460 msgr2=0x557e81b19180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.520+0000 7fcdfea00640 0 client.25702 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.521+0000 7fce26804740 1 -- 172.21.15.32:0/646251301 shutdown_connections 2024-08-31T07:13:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.521+0000 7fce26804740 1 -- 172.21.15.32:0/646251301 wait complete. 2024-08-31T07:13:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.521+0000 7fce26804740 20 client.25702 trim_cache size 0 max 16384 2024-08-31T07:13:13.523 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:13:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.522+0000 7fce1fa00640 1 -- 172.21.15.32:0/1380474461 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce000b4da0 msgr2=0x7fce000b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.522+0000 7fce1fa00640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce000b4da0 0x7fce000b72b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce10009fd0 tx=0x7fce10009920 comp rx=0 tx=0).stop 2024-08-31T07:13:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.522+0000 7fce1fa00640 1 -- 172.21.15.32:0/1380474461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce180754f0 msgr2=0x7fce18074d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.522+0000 7fce1fa00640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce180754f0 0x7fce18074d70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00ea60 tx=0x7fce0c00ef20 comp rx=0 tx=0).stop 2024-08-31T07:13:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.522+0000 7fce1fa00640 1 -- 172.21.15.32:0/1380474461 shutdown_connections 2024-08-31T07:13:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.523+0000 7fce1fa00640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce000b4da0 0x7fce000b72b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.523+0000 7fce1fa00640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce1810fec0 0x7fce1806e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.523+0000 7fce1fa00640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce18075e40 0x7fce1806e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.523+0000 7fce1fa00640 1 --2- 172.21.15.32:0/1380474461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce180754f0 0x7fce18074d70 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00ea60 tx=0x7fce0c00ef20 comp rx=0 tx=0).stop 2024-08-31T07:13:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.523+0000 7fce1fa00640 1 -- 172.21.15.32:0/1380474461 >> 172.21.15.32:0/1380474461 conn(0x7fce1807b5f0 msgr2=0x7fce181089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.523+0000 7fce1fa00640 1 -- 172.21.15.32:0/1380474461 shutdown_connections 2024-08-31T07:13:13.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:13.523+0000 7fce1fa00640 1 -- 172.21.15.32:0/1380474461 wait complete. 2024-08-31T07:13:13.539 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:14.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:13 smithi073 ceph-mon[21995]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-08-31T07:13:14.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:13 smithi084 ceph-mon[22387]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-08-31T07:13:14.307 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:13 smithi032 ceph-mon[17252]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-08-31T07:13:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:14 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:13:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.183+0000 7f16f5a00640 1 -- 172.21.15.32:0/45241653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16f010fcc0 msgr2=0x7f16f0112100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.183+0000 7f16f5a00640 1 --2- 172.21.15.32:0/45241653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16f010fcc0 0x7f16f0112100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009ca0 tx=0x7f16e802a8e0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.183+0000 7f16f5a00640 1 -- 172.21.15.32:0/45241653 shutdown_connections 2024-08-31T07:13:15.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.183+0000 7f16f5a00640 1 --2- 172.21.15.32:0/45241653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16f010fcc0 0x7f16f0112100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.183+0000 7f16f5a00640 1 --2- 172.21.15.32:0/45241653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0075ea0 0x7f16f010f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.183+0000 7f16f5a00640 1 --2- 172.21.15.32:0/45241653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f0075550 0x7f16f0075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.183+0000 7f16f5a00640 1 -- 172.21.15.32:0/45241653 >> 172.21.15.32:0/45241653 conn(0x7f16f00ff1d0 msgr2=0x7f16f0101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:15.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.183+0000 7f16f5a00640 1 -- 172.21.15.32:0/45241653 shutdown_connections 2024-08-31T07:13:15.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.183+0000 7f16f5a00640 1 -- 172.21.15.32:0/45241653 wait complete. 2024-08-31T07:13:15.185 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:14 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:13:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.185+0000 7f16f5a00640 1 Processor -- start 2024-08-31T07:13:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.185+0000 7f16f5a00640 1 -- start start 2024-08-31T07:13:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.186+0000 7f16f5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f0075550 0x7f16f019c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.186+0000 7f16f5a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0075ea0 0x7f16f019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.186+0000 7f16f5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16f010fcc0 0x7f16f01a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.186+0000 7f16f5a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0115870 con 0x7f16f0075550 2024-08-31T07:13:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.186+0000 7f16f5a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f16f01156f0 con 0x7f16f010fcc0 2024-08-31T07:13:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.186+0000 7f16f5a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f16f01159f0 con 0x7f16f0075ea0 2024-08-31T07:13:15.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.186+0000 7f16efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f0075550 0x7f16f019c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.186+0000 7f16efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f0075550 0x7f16f019c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43660/0 (socket says 172.21.15.32:43660) 2024-08-31T07:13:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.186+0000 7f16efe00640 1 -- 172.21.15.32:0/2080550246 learned_addr learned my addr 172.21.15.32:0/2080550246 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.186+0000 7f16f5000640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16f010fcc0 0x7f16f01a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.187+0000 7f16ef400640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0075ea0 0x7f16f019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.187+0000 7f16f5000640 1 -- 172.21.15.32:0/2080550246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0075ea0 msgr2=0x7f16f019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.187+0000 7f16f5000640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0075ea0 0x7f16f019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.187+0000 7f16f5000640 1 -- 172.21.15.32:0/2080550246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f0075550 msgr2=0x7f16f019c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.187+0000 7f16f5000640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f0075550 0x7f16f019c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.187+0000 7f16f5000640 1 -- 172.21.15.32:0/2080550246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f01a21f0 con 0x7f16f010fcc0 2024-08-31T07:13:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.188+0000 7f16f5000640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16f010fcc0 0x7f16f01a1ac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16e8006010 tx=0x7f16e8030810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:15.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.188+0000 7f16ecc00640 1 -- 172.21.15.32:0/2080550246 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8030e10 con 0x7f16f010fcc0 2024-08-31T07:13:15.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.188+0000 7f16f5a00640 1 -- 172.21.15.32:0/2080550246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f01a2480 con 0x7f16f010fcc0 2024-08-31T07:13:15.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.188+0000 7f16f5a00640 1 -- 172.21.15.32:0/2080550246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f01a8c40 con 0x7f16f010fcc0 2024-08-31T07:13:15.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.189+0000 7f16ecc00640 1 -- 172.21.15.32:0/2080550246 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e8041070 con 0x7f16f010fcc0 2024-08-31T07:13:15.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.189+0000 7f16ecc00640 1 -- 172.21.15.32:0/2080550246 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e803a5f0 con 0x7f16f010fcc0 2024-08-31T07:13:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.194+0000 7f16ecc00640 1 -- 172.21.15.32:0/2080550246 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f16e803a790 con 0x7f16f010fcc0 2024-08-31T07:13:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.194+0000 7f16fdc57740 1 Processor -- start 2024-08-31T07:13:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.194+0000 7f16fdc57740 1 -- start start 2024-08-31T07:13:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.195+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2eb7840 0x55f6c3266ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.195+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c3267020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.195+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2e96b60 0x55f6c3267560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.195+0000 7f16fdc57740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f6c31d3530 con 0x55f6c2e96b60 2024-08-31T07:13:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.195+0000 7f16fdc57740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f6c31c2ac0 con 0x55f6c2e8e980 2024-08-31T07:13:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.195+0000 7f16fdc57740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f6c31c6a70 con 0x55f6c2eb7840 2024-08-31T07:13:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.195+0000 7f16ef400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2eb7840 0x55f6c3266ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.195+0000 7f16ef400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2eb7840 0x55f6c3266ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50692/0 (socket says 172.21.15.32:50692) 2024-08-31T07:13:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.195+0000 7f16ef400640 1 -- 172.21.15.32:0/2958196735 learned_addr learned my addr 172.21.15.32:0/2958196735 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.195+0000 7f16efe00640 1 --2- 172.21.15.32:0/2958196735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c3267020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.196+0000 7f16efe00640 1 -- 172.21.15.32:0/2958196735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2eb7840 msgr2=0x55f6c3266ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.196+0000 7f16efe00640 1 --2- 172.21.15.32:0/2958196735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2eb7840 0x55f6c3266ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.196+0000 7f16efe00640 1 -- 172.21.15.32:0/2958196735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2e96b60 msgr2=0x55f6c3267560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.196+0000 7f16efe00640 1 --2- 172.21.15.32:0/2958196735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2e96b60 0x55f6c3267560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.196+0000 7f16efe00640 1 -- 172.21.15.32:0/2958196735 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c3175880 con 0x55f6c2e8e980 2024-08-31T07:13:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.196+0000 7f16ecc00640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16d40b4d10 0x7f16d40b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.197+0000 7f16ecc00640 1 -- 172.21.15.32:0/2080550246 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f16e8106930 con 0x7f16f010fcc0 2024-08-31T07:13:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.197+0000 7f16ef400640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16d40b4d10 0x7f16d40b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.197+0000 7f16efe00640 1 --2- 172.21.15.32:0/2958196735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c3267020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16e00058c0 tx=0x7f16e000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ab834863c20c1e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.197+0000 7f16e5600640 1 -- 172.21.15.32:0/2958196735 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0015070 con 0x55f6c2e8e980 2024-08-31T07:13:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.198+0000 7f16e5600640 1 -- 172.21.15.32:0/2958196735 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e001a910 con 0x55f6c2e8e980 2024-08-31T07:13:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.198+0000 7f16e5600640 1 -- 172.21.15.32:0/2958196735 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0011830 con 0x55f6c2e8e980 2024-08-31T07:13:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.199+0000 7f16ef400640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16d40b4d10 0x7f16d40b7220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00ccb0 tx=0x7f16dc00c3b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.199+0000 7f16fdc57740 1 -- 172.21.15.32:0/2958196735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 msgr2=0x55f6c3267020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.199+0000 7f16fdc57740 1 --2- 172.21.15.32:0/2958196735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c3267020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16e00058c0 tx=0x7f16e000f730 comp rx=0 tx=0).stop 2024-08-31T07:13:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.199+0000 7f16fdc57740 1 -- 172.21.15.32:0/2958196735 shutdown_connections 2024-08-31T07:13:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.199+0000 7f16fdc57740 1 --2- 172.21.15.32:0/2958196735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2e96b60 0x55f6c3267560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.199+0000 7f16fdc57740 1 --2- 172.21.15.32:0/2958196735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c3267020 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.199+0000 7f16fdc57740 1 --2- 172.21.15.32:0/2958196735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2eb7840 0x55f6c3266ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.199+0000 7f16fdc57740 1 -- 172.21.15.32:0/2958196735 >> 172.21.15.32:0/2958196735 conn(0x55f6c2eab990 msgr2=0x55f6c32649d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:15.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.199+0000 7f16fdc57740 1 -- 172.21.15.32:0/2958196735 shutdown_connections 2024-08-31T07:13:15.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.199+0000 7f16fdc57740 1 -- 172.21.15.32:0/2958196735 wait complete. 2024-08-31T07:13:15.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.200+0000 7f16fdc57740 1 -- start start 2024-08-31T07:13:15.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.200+0000 7f16fdc57740 1 Processor -- start 2024-08-31T07:13:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.201+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c3272630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.201+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c3272b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.202+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 0x55f6c327e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.202+0000 7f16fdc57740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f6c30257a0 con 0x55f6c2eb7840 2024-08-31T07:13:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.202+0000 7f16ef400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c3272b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.202+0000 7f16ef400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c3272b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50704/0 (socket says 172.21.15.32:50704) 2024-08-31T07:13:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.202+0000 7f16ef400640 1 -- 172.21.15.32:0/507107392 learned_addr learned my addr 172.21.15.32:0/507107392 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.202+0000 7f16efe00640 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c3272630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.202+0000 7f16f5000640 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 0x55f6c327e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.202+0000 7f16fdc57740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f6c31497f0 con 0x55f6c2e8e980 2024-08-31T07:13:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.202+0000 7f16fdc57740 1 -- 172.21.15.32:0/507107392 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f6c3275310 con 0x55f6c2e96b60 2024-08-31T07:13:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.202+0000 7f16fdc57740 10 client.0 _ll_register_callbacks cb 0x55f6c316ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.203+0000 7f16f5000640 1 -- 172.21.15.32:0/507107392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 msgr2=0x55f6c3272b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.203+0000 7f16f5000640 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c3272b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.203+0000 7f16f5000640 1 -- 172.21.15.32:0/507107392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 msgr2=0x55f6c3272630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.203+0000 7f16f5000640 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c3272630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.203+0000 7f16f5000640 1 -- 172.21.15.32:0/507107392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c2f31b90 con 0x55f6c2eb7840 2024-08-31T07:13:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.203+0000 7f16ef400640 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c3272b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.204+0000 7f16f5000640 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 0x55f6c327e6e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f16e81040a0 tx=0x7f16e80ccac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6705eb83cf3c106e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.204+0000 7f16da000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:13:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.204+0000 7f16da000640 1 -- 172.21.15.32:0/507107392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e80cd040 con 0x55f6c2eb7840 2024-08-31T07:13:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.204+0000 7f16da000640 1 -- 172.21.15.32:0/507107392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f16e8004d50 con 0x55f6c2eb7840 2024-08-31T07:13:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.205+0000 7f16fdc57740 1 -- 172.21.15.32:0/507107392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e001a910 con 0x55f6c2eb7840 2024-08-31T07:13:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.205+0000 7f16da000640 1 -- 172.21.15.32:0/507107392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f16e80d18b0 con 0x55f6c2eb7840 2024-08-31T07:13:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.205+0000 7f16fdc57740 10 client.15693 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.205+0000 7f16fdc57740 1 -- 172.21.15.32:0/507107392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6c3281330 con 0x55f6c2eb7840 2024-08-31T07:13:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.205+0000 7f16da000640 1 -- 172.21.15.32:0/507107392 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e80f2070 con 0x55f6c2eb7840 2024-08-31T07:13:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.205+0000 7f16da000640 1 -- 172.21.15.32:0/507107392 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f16e80cbd10 con 0x55f6c2eb7840 2024-08-31T07:13:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.206+0000 7f16fdc57740 10 client.15693 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.206+0000 7f16fdc57740 10 client.15693 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:13:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.206+0000 7f16fdc57740 20 client.15693 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.206+0000 7f16fdc57740 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6c2e761c0 0x55f6c3284860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.206+0000 7f16fdc57740 4 client.15693 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.206+0000 7f16fdc57740 1 -- 172.21.15.32:0/507107392 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6c31d5480 con 0x55f6c2e761c0 2024-08-31T07:13:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.206+0000 7f16efe00640 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6c2e761c0 0x55f6c3284860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.207+0000 7f16efe00640 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6c2e761c0 0x55f6c3284860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6acdd53033cdfaf server_cookie=ec53ab98a1026821 in_seq=0 out_seq=0 2024-08-31T07:13:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.207+0000 7f16da000640 10 client.15693 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:15.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.208+0000 7f16da000640 1 -- 172.21.15.32:0/507107392 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f6c31d5480 con 0x55f6c2e761c0 2024-08-31T07:13:15.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.208+0000 7f16da000640 10 client.15693 handle_command_reply: tid=0 2024-08-31T07:13:15.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.209+0000 7f16fdc57740 1 client.15693 shutdown 2024-08-31T07:13:15.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.209+0000 7f16fdc57740 1 -- 172.21.15.32:0/507107392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 msgr2=0x55f6c327e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:15.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.209+0000 7f16fdc57740 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 0x55f6c327e6e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f16e81040a0 tx=0x7f16e80ccac0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.209+0000 7f16fdc57740 1 -- 172.21.15.32:0/507107392 shutdown_connections 2024-08-31T07:13:15.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.209+0000 7f16fdc57740 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6c2e761c0 0x55f6c3284860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.209+0000 7f16fdc57740 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 0x55f6c327e6e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.209+0000 7f16fdc57740 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c3272b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.209+0000 7f16da000640 0 client.15693 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:15.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.209+0000 7f16fdc57740 1 --2- 172.21.15.32:0/507107392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c3272630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.209+0000 7f16fdc57740 1 -- 172.21.15.32:0/507107392 >> 172.21.15.32:0/507107392 conn(0x55f6c2eab990 msgr2=0x55f6c325a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:15.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.210+0000 7f16fdc57740 1 -- 172.21.15.32:0/507107392 shutdown_connections 2024-08-31T07:13:15.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.210+0000 7f16fdc57740 1 -- 172.21.15.32:0/507107392 wait complete. 2024-08-31T07:13:15.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.210+0000 7f16fdc57740 20 client.15693 trim_cache size 0 max 16384 2024-08-31T07:13:15.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:14 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:13:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.247+0000 7f16fdc57740 1 Processor -- start 2024-08-31T07:13:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.247+0000 7f16fdc57740 1 -- start start 2024-08-31T07:13:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.247+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2e8e980 0x55f6c325a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.247+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c325ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.247+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2eb7840 0x55f6c326bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.248+0000 7f16fdc57740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16d000e650 con 0x55f6c2e8e980 2024-08-31T07:13:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.248+0000 7f16fdc57740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0010190 con 0x55f6c2eb7840 2024-08-31T07:13:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.248+0000 7f16fdc57740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f16d000ed20 con 0x55f6c2e96b60 2024-08-31T07:13:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.248+0000 7f16ef400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c325ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.248+0000 7f16ef400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c325ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50712/0 (socket says 172.21.15.32:50712) 2024-08-31T07:13:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.248+0000 7f16ef400640 1 -- 172.21.15.32:0/1639796608 learned_addr learned my addr 172.21.15.32:0/1639796608 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.248+0000 7f16f5000640 1 --2- 172.21.15.32:0/1639796608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2eb7840 0x55f6c326bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.248+0000 7f16efe00640 1 --2- 172.21.15.32:0/1639796608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2e8e980 0x55f6c325a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.249+0000 7f16ef400640 1 -- 172.21.15.32:0/1639796608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2eb7840 msgr2=0x55f6c326bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.249+0000 7f16ef400640 1 --2- 172.21.15.32:0/1639796608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2eb7840 0x55f6c326bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.249+0000 7f16ef400640 1 -- 172.21.15.32:0/1639796608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2e8e980 msgr2=0x55f6c325a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.249+0000 7f16ef400640 1 --2- 172.21.15.32:0/1639796608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2e8e980 0x55f6c325a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.249+0000 7f16ef400640 1 -- 172.21.15.32:0/1639796608 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c3272d10 con 0x55f6c2e96b60 2024-08-31T07:13:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.250+0000 7f16ef400640 1 --2- 172.21.15.32:0/1639796608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c325ad20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f16dc012600 tx=0x7f16dc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41cc53b213e5918b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.250+0000 7f16e4c00640 1 -- 172.21.15.32:0/1639796608 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc01f7c0 con 0x55f6c2e96b60 2024-08-31T07:13:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.251+0000 7f16e4c00640 1 -- 172.21.15.32:0/1639796608 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16dc01fe10 con 0x55f6c2e96b60 2024-08-31T07:13:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.251+0000 7f16e4c00640 1 -- 172.21.15.32:0/1639796608 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0283d0 con 0x55f6c2e96b60 2024-08-31T07:13:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.252+0000 7f16fdc57740 1 -- 172.21.15.32:0/1639796608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 msgr2=0x55f6c325ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.252+0000 7f16fdc57740 1 --2- 172.21.15.32:0/1639796608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c325ad20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f16dc012600 tx=0x7f16dc014000 comp rx=0 tx=0).stop 2024-08-31T07:13:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.252+0000 7f16fdc57740 1 -- 172.21.15.32:0/1639796608 shutdown_connections 2024-08-31T07:13:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.252+0000 7f16fdc57740 1 --2- 172.21.15.32:0/1639796608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2eb7840 0x55f6c326bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.252+0000 7f16fdc57740 1 --2- 172.21.15.32:0/1639796608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c325ad20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.252+0000 7f16fdc57740 1 --2- 172.21.15.32:0/1639796608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2e8e980 0x55f6c325a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.252+0000 7f16fdc57740 1 -- 172.21.15.32:0/1639796608 >> 172.21.15.32:0/1639796608 conn(0x55f6c2eab990 msgr2=0x55f6c326d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.252+0000 7f16fdc57740 1 -- 172.21.15.32:0/1639796608 shutdown_connections 2024-08-31T07:13:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.252+0000 7f16fdc57740 1 -- 172.21.15.32:0/1639796608 wait complete. 2024-08-31T07:13:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.253+0000 7f16fdc57740 1 -- start start 2024-08-31T07:13:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.253+0000 7f16fdc57740 1 Processor -- start 2024-08-31T07:13:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.254+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c327adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.254+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c32788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.254+0000 7f16fdc57740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 0x55f6c3278de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.254+0000 7f16fdc57740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16d000fac0 con 0x55f6c2eb7840 2024-08-31T07:13:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.254+0000 7f16fdc57740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f16d000f3f0 con 0x55f6c2e8e980 2024-08-31T07:13:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.254+0000 7f16fdc57740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f16d000c630 con 0x55f6c2e96b60 2024-08-31T07:13:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.254+0000 7f16fdc57740 10 client.0 _ll_register_callbacks cb 0x7f16e80cce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.254+0000 7f16efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c327adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.255+0000 7f16efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c327adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38778/0 (socket says 172.21.15.32:38778) 2024-08-31T07:13:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.255+0000 7f16efe00640 1 -- 172.21.15.32:0/1597058025 learned_addr learned my addr 172.21.15.32:0/1597058025 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.255+0000 7f16ef400640 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c32788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.255+0000 7f16f5000640 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 0x55f6c3278de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.256+0000 7f16efe00640 1 -- 172.21.15.32:0/1597058025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 msgr2=0x55f6c32788a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.256+0000 7f16efe00640 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c32788a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.256+0000 7f16efe00640 1 -- 172.21.15.32:0/1597058025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 msgr2=0x55f6c3278de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.256+0000 7f16efe00640 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 0x55f6c3278de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.256+0000 7f16efe00640 1 -- 172.21.15.32:0/1597058025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c3279550 con 0x55f6c2e8e980 2024-08-31T07:13:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.256+0000 7f16f5000640 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 0x55f6c3278de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.257+0000 7f16efe00640 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c327adf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f16e000bc30 tx=0x7f16e001a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a9b2c32b8794d19 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.257+0000 7f16da000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:13:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.257+0000 7f16da000640 1 -- 172.21.15.32:0/1597058025 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0015040 con 0x55f6c2e8e980 2024-08-31T07:13:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.257+0000 7f16da000640 1 -- 172.21.15.32:0/1597058025 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f16e0004040 con 0x55f6c2e8e980 2024-08-31T07:13:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.257+0000 7f16fdc57740 1 -- 172.21.15.32:0/1597058025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16dc01fe10 con 0x55f6c2e8e980 2024-08-31T07:13:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16da000640 1 -- 172.21.15.32:0/1597058025 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f16e0004240 con 0x55f6c2e8e980 2024-08-31T07:13:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16fdc57740 10 client.25712 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16fdc57740 1 -- 172.21.15.32:0/1597058025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6c32f0140 con 0x55f6c2e8e980 2024-08-31T07:13:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16da000640 1 -- 172.21.15.32:0/1597058025 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16dc01fe10 con 0x55f6c2e8e980 2024-08-31T07:13:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16da000640 1 -- 172.21.15.32:0/1597058025 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f16e002a070 con 0x55f6c2e8e980 2024-08-31T07:13:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16fdc57740 10 client.25712 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16fdc57740 10 client.25712 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:13:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16fdc57740 20 client.25712 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16fdc57740 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6c2e4b570 0x55f6c32f2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16fdc57740 4 client.25712 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}] 2024-08-31T07:13:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.258+0000 7f16fdc57740 1 -- 172.21.15.32:0/1597058025 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}) -- 0x55f6c31d0f20 con 0x55f6c2e4b570 2024-08-31T07:13:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.259+0000 7f16ef400640 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6c2e4b570 0x55f6c32f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.259+0000 7f16ef400640 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6c2e4b570 0x55f6c32f2600 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b870e73016f32e41 server_cookie=aabe8060d6c57b5f in_seq=0 out_seq=0 2024-08-31T07:13:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.259+0000 7f16da000640 10 client.25712 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16da000640 1 -- 172.21.15.32:0/1597058025 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f6c31d0f20 con 0x55f6c2e4b570 2024-08-31T07:13:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16da000640 10 client.25712 handle_command_reply: tid=0 2024-08-31T07:13:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16fdc57740 1 client.25712 shutdown 2024-08-31T07:13:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16fdc57740 1 -- 172.21.15.32:0/1597058025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 msgr2=0x55f6c327adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16fdc57740 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c327adf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f16e000bc30 tx=0x7f16e001a710 comp rx=0 tx=0).stop 2024-08-31T07:13:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16fdc57740 1 -- 172.21.15.32:0/1597058025 shutdown_connections 2024-08-31T07:13:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16fdc57740 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6c2e4b570 0x55f6c32f2600 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16fdc57740 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6c2eb7840 0x55f6c3278de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16fdc57740 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6c2e96b60 0x55f6c32788a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16fdc57740 1 --2- 172.21.15.32:0/1597058025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6c2e8e980 0x55f6c327adf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.269+0000 7f16fdc57740 1 -- 172.21.15.32:0/1597058025 >> 172.21.15.32:0/1597058025 conn(0x55f6c2eab990 msgr2=0x55f6c326bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.270+0000 7f16da000640 0 client.25712 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.270+0000 7f16fdc57740 1 -- 172.21.15.32:0/1597058025 shutdown_connections 2024-08-31T07:13:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.270+0000 7f16fdc57740 1 -- 172.21.15.32:0/1597058025 wait complete. 2024-08-31T07:13:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.270+0000 7f16fdc57740 20 client.25712 trim_cache size 0 max 16384 2024-08-31T07:13:15.272 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:13:15.272 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 29, 2024-08-31T07:13:15.272 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:13:15.272 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 2, 2024-08-31T07:13:15.272 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:13:15.272 INFO:teuthology.orchestra.run.smithi032.stdout: "7b061927": { 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 2, 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 146.4, 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:13:15.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:15.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:15.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:15.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:15.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:15.274 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:13:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.271+0000 7f16f5a00640 1 -- 172.21.15.32:0/2080550246 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16d40b4d10 msgr2=0x7f16d40b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.271+0000 7f16f5a00640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16d40b4d10 0x7f16d40b7220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00ccb0 tx=0x7f16dc00c3b0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.272+0000 7f16f5a00640 1 -- 172.21.15.32:0/2080550246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16f010fcc0 msgr2=0x7f16f01a1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.272+0000 7f16f5a00640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16f010fcc0 0x7f16f01a1ac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16e8006010 tx=0x7f16e8030810 comp rx=0 tx=0).stop 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.272+0000 7f16f5a00640 1 -- 172.21.15.32:0/2080550246 shutdown_connections 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.272+0000 7f16f5a00640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f16d40b4d10 0x7f16d40b7220 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.272+0000 7f16f5a00640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16f010fcc0 0x7f16f01a1ac0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.272+0000 7f16f5a00640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16f0075ea0 0x7f16f019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.272+0000 7f16f5a00640 1 --2- 172.21.15.32:0/2080550246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f0075550 0x7f16f019c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.272+0000 7f16f5a00640 1 -- 172.21.15.32:0/2080550246 >> 172.21.15.32:0/2080550246 conn(0x7f16f00ff1d0 msgr2=0x7f16f01097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.272+0000 7f16f5a00640 1 -- 172.21.15.32:0/2080550246 shutdown_connections 2024-08-31T07:13:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:15.272+0000 7f16f5a00640 1 -- 172.21.15.32:0/2080550246 wait complete. 2024-08-31T07:13:15.361 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:13:15.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:13:16.051 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:15 smithi032 ceph-mon[17252]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-08-31T07:13:16.051 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:13:16.051 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:13:16.051 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:15 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:13:16.108 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7b061927', quiesce took 6.8 seconds. Will release after: 19.599999999999998 2024-08-31T07:13:16.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:15 smithi073 ceph-mon[21995]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-08-31T07:13:16.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:13:16.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:13:16.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:15 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:13:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:16.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:15 smithi084 ceph-mon[22387]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-08-31T07:13:16.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:13:16.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:13:16.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:15 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:13:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.962+0000 7fe5efe00640 1 -- 172.21.15.32:0/3820758709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5f0105960 msgr2=0x7fe5f0105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.962+0000 7fe5efe00640 1 --2- 172.21.15.32:0/3820758709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5f0105960 0x7fe5f0105d70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009900 tx=0x7fe5e002a410 comp rx=0 tx=0).stop 2024-08-31T07:13:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.962+0000 7fe5efe00640 1 -- 172.21.15.32:0/3820758709 shutdown_connections 2024-08-31T07:13:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.962+0000 7fe5efe00640 1 --2- 172.21.15.32:0/3820758709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5f010bbf0 0x7fe5f0112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.962+0000 7fe5efe00640 1 --2- 172.21.15.32:0/3820758709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f010b2a0 0x7fe5f010b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.962+0000 7fe5efe00640 1 --2- 172.21.15.32:0/3820758709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5f0105960 0x7fe5f0105d70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.962+0000 7fe5efe00640 1 -- 172.21.15.32:0/3820758709 >> 172.21.15.32:0/3820758709 conn(0x7fe5f01013d0 msgr2=0x7fe5f0103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.962+0000 7fe5efe00640 1 -- 172.21.15.32:0/3820758709 shutdown_connections 2024-08-31T07:13:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.962+0000 7fe5efe00640 1 -- 172.21.15.32:0/3820758709 wait complete. 2024-08-31T07:13:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.964+0000 7fe5efe00640 1 Processor -- start 2024-08-31T07:13:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.964+0000 7fe5efe00640 1 -- start start 2024-08-31T07:13:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.964+0000 7fe5efe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f0105960 0x7fe5f019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.964+0000 7fe5efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5f010b2a0 0x7fe5f019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.965+0000 7fe5efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5f010bbf0 0x7fe5f01a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.965+0000 7fe5efe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01158b0 con 0x7fe5f010bbf0 2024-08-31T07:13:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.965+0000 7fe5efe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0115730 con 0x7fe5f010b2a0 2024-08-31T07:13:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.965+0000 7fe5efe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0115a30 con 0x7fe5f0105960 2024-08-31T07:13:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.965+0000 7fe5eea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f0105960 0x7fe5f019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.965+0000 7fe5eea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f0105960 0x7fe5f019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55150/0 (socket says 172.21.15.32:55150) 2024-08-31T07:13:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.965+0000 7fe5eea00640 1 -- 172.21.15.32:0/2167550524 learned_addr learned my addr 172.21.15.32:0/2167550524 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.965+0000 7fe5ef400640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5f010bbf0 0x7fe5f01a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.966+0000 7fe5eea00640 1 -- 172.21.15.32:0/2167550524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5f010b2a0 msgr2=0x7fe5f019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.966+0000 7fe5eea00640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5f010b2a0 0x7fe5f019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.966+0000 7fe5eea00640 1 -- 172.21.15.32:0/2167550524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5f010bbf0 msgr2=0x7fe5f01a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.966+0000 7fe5eea00640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5f010bbf0 0x7fe5f01a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.966+0000 7fe5eea00640 1 -- 172.21.15.32:0/2167550524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f01a2210 con 0x7fe5f0105960 2024-08-31T07:13:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.966+0000 7fe5eea00640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f0105960 0x7fe5f019c2a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0005940 tx=0x7fe5e00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.967+0000 7fe5f4a00640 1 -- 172.21.15.32:0/2167550524 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e003f030 con 0x7fe5f0105960 2024-08-31T07:13:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.967+0000 7fe5f4a00640 1 -- 172.21.15.32:0/2167550524 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e00044c0 con 0x7fe5f0105960 2024-08-31T07:13:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.967+0000 7fe5f4a00640 1 -- 172.21.15.32:0/2167550524 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0035070 con 0x7fe5f0105960 2024-08-31T07:13:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.967+0000 7fe5efe00640 1 -- 172.21.15.32:0/2167550524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f01a24a0 con 0x7fe5f0105960 2024-08-31T07:13:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.968+0000 7fe5efe00640 1 -- 172.21.15.32:0/2167550524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f01a2900 con 0x7fe5f0105960 2024-08-31T07:13:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.971+0000 7fe5f4a00640 1 -- 172.21.15.32:0/2167550524 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe5e0042030 con 0x7fe5f0105960 2024-08-31T07:13:16.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.972+0000 7fe5fca46740 1 Processor -- start 2024-08-31T07:13:16.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.972+0000 7fe5fca46740 1 -- start start 2024-08-31T07:13:16.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.973+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158654e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.973+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 0x56158654ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.973+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561586187670 0x56158654f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.973+0000 7fe5fca46740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5615864c4750 con 0x561586187670 2024-08-31T07:13:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.973+0000 7fe5fca46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5615864b3ce0 con 0x56158617f490 2024-08-31T07:13:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.973+0000 7fe5fca46740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615864b7c90 con 0x5615861a8350 2024-08-31T07:13:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5ef400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158654e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5ef400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158654e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55160/0 (socket says 172.21.15.32:55160) 2024-08-31T07:13:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5ef400640 1 -- 172.21.15.32:0/2643648196 learned_addr learned my addr 172.21.15.32:0/2643648196 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5eea00640 1 --2- 172.21.15.32:0/2643648196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 0x56158654ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5f4a00640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5d40b4d20 0x7fe5d40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5ee000640 1 --2- 172.21.15.32:0/2643648196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561586187670 0x56158654f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5ef400640 1 -- 172.21.15.32:0/2643648196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 msgr2=0x56158654ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5ef400640 1 --2- 172.21.15.32:0/2643648196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 0x56158654ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5ef400640 1 -- 172.21.15.32:0/2643648196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561586187670 msgr2=0x56158654f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5ef400640 1 --2- 172.21.15.32:0/2643648196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561586187670 0x56158654f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.974+0000 7fe5ef400640 1 -- 172.21.15.32:0/2643648196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561586213930 con 0x5615861a8350 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.975+0000 7fe5ef400640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5d40b4d20 0x7fe5d40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.975+0000 7fe5f4a00640 1 -- 172.21.15.32:0/2167550524 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe5e01055c0 con 0x7fe5f0105960 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.975+0000 7fe5eea00640 1 --2- 172.21.15.32:0/2643648196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 0x56158654ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.975+0000 7fe5ef400640 1 --2- 172.21.15.32:0/2643648196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158654e7d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80104f0 tx=0x7fe5e80109b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2fc94d2fd5f82a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.976+0000 7fe5dbe00640 1 -- 172.21.15.32:0/2643648196 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8019070 con 0x5615861a8350 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.976+0000 7fe5dbe00640 1 -- 172.21.15.32:0/2643648196 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e800edb0 con 0x5615861a8350 2024-08-31T07:13:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.976+0000 7fe5dbe00640 1 -- 172.21.15.32:0/2643648196 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e801dae0 con 0x5615861a8350 2024-08-31T07:13:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.977+0000 7fe5fca46740 1 -- 172.21.15.32:0/2643648196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 msgr2=0x56158654e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.977+0000 7fe5fca46740 1 --2- 172.21.15.32:0/2643648196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158654e7d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80104f0 tx=0x7fe5e80109b0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.977+0000 7fe5ef400640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5d40b4d20 0x7fe5d40b7230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8021000 tx=0x7fe5e801e000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.977+0000 7fe5fca46740 1 -- 172.21.15.32:0/2643648196 shutdown_connections 2024-08-31T07:13:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.977+0000 7fe5fca46740 1 --2- 172.21.15.32:0/2643648196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561586187670 0x56158654f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.977+0000 7fe5fca46740 1 --2- 172.21.15.32:0/2643648196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 0x56158654ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.977+0000 7fe5fca46740 1 --2- 172.21.15.32:0/2643648196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158654e7d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.977+0000 7fe5fca46740 1 -- 172.21.15.32:0/2643648196 >> 172.21.15.32:0/2643648196 conn(0x56158619c740 msgr2=0x56158654c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.977+0000 7fe5fca46740 1 -- 172.21.15.32:0/2643648196 shutdown_connections 2024-08-31T07:13:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.978+0000 7fe5fca46740 1 -- 172.21.15.32:0/2643648196 wait complete. 2024-08-31T07:13:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.978+0000 7fe5fca46740 1 -- start start 2024-08-31T07:13:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.979+0000 7fe5fca46740 1 Processor -- start 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 0x561586562b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561586187670 0x561586563070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158656ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5fca46740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561586316890 con 0x56158617f490 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5fca46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56158643acd0 con 0x561586187670 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5fca46740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615865656e0 con 0x5615861a8350 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5fca46740 10 client.0 _ll_register_callbacks cb 0x5615864aeef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5eea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 0x561586562b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5eea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 0x561586562b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59846/0 (socket says 172.21.15.32:59846) 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5eea00640 1 -- 172.21.15.32:0/3001511496 learned_addr learned my addr 172.21.15.32:0/3001511496 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.980+0000 7fe5ef400640 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158656ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.981+0000 7fe5ee000640 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561586187670 0x561586563070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.981+0000 7fe5eea00640 1 -- 172.21.15.32:0/3001511496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 msgr2=0x56158656ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.981+0000 7fe5eea00640 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158656ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.981+0000 7fe5eea00640 1 -- 172.21.15.32:0/3001511496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561586187670 msgr2=0x561586563070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.981+0000 7fe5eea00640 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561586187670 0x561586563070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.981+0000 7fe5eea00640 1 -- 172.21.15.32:0/3001511496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615864841b0 con 0x56158617f490 2024-08-31T07:13:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.981+0000 7fe5ef400640 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158656ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.981+0000 7fe5eea00640 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 0x561586562b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0102d90 tx=0x7fe5e00c8210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92340bffe531d983 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.981+0000 7fe5d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:13:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.982+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3001511496 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e01031b0 con 0x56158617f490 2024-08-31T07:13:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.982+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3001511496 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe5e00cbf30 con 0x56158617f490 2024-08-31T07:13:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.982+0000 7fe5fca46740 1 -- 172.21.15.32:0/3001511496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5e800edb0 con 0x56158617f490 2024-08-31T07:13:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.982+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3001511496 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5e00cb7e0 con 0x56158617f490 2024-08-31T07:13:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.982+0000 7fe5fca46740 10 client.15711 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.982+0000 7fe5fca46740 1 -- 172.21.15.32:0/3001511496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615865716e0 con 0x56158617f490 2024-08-31T07:13:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.983+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3001511496 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e800edb0 con 0x56158617f490 2024-08-31T07:13:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.984+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3001511496 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe5e00d4b60 con 0x56158617f490 2024-08-31T07:13:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.984+0000 7fe5fca46740 10 client.15711 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.984+0000 7fe5fca46740 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:13:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.984+0000 7fe5fca46740 20 client.15711 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.984+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561586166cd0 0x561586574b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.984+0000 7fe5fca46740 4 client.15711 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.984+0000 7fe5fca46740 1 -- 172.21.15.32:0/3001511496 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615864c66a0 con 0x561586166cd0 2024-08-31T07:13:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.984+0000 7fe5ee000640 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561586166cd0 0x561586574b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.985+0000 7fe5ee000640 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561586166cd0 0x561586574b30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c3743160f4f5d9b server_cookie=a2f594d1c626ed67 in_seq=0 out_seq=0 2024-08-31T07:13:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.985+0000 7fe5d8c00640 10 client.15711 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.986+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3001511496 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615864c66a0 con 0x561586166cd0 2024-08-31T07:13:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.986+0000 7fe5d8c00640 10 client.15711 handle_command_reply: tid=0 2024-08-31T07:13:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.986+0000 7fe5fca46740 1 client.15711 shutdown 2024-08-31T07:13:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.986+0000 7fe5fca46740 1 -- 172.21.15.32:0/3001511496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 msgr2=0x561586562b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.986+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 0x561586562b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0102d90 tx=0x7fe5e00c8210 comp rx=0 tx=0).stop 2024-08-31T07:13:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.986+0000 7fe5fca46740 1 -- 172.21.15.32:0/3001511496 shutdown_connections 2024-08-31T07:13:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.987+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561586166cd0 0x561586574b30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.987+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615861a8350 0x56158656ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.987+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561586187670 0x561586563070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.987+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3001511496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 0x561586562b30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.987+0000 7fe5d8c00640 0 client.15711 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.987+0000 7fe5fca46740 1 -- 172.21.15.32:0/3001511496 >> 172.21.15.32:0/3001511496 conn(0x56158619c740 msgr2=0x56158654ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.987+0000 7fe5fca46740 1 -- 172.21.15.32:0/3001511496 shutdown_connections 2024-08-31T07:13:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.987+0000 7fe5fca46740 1 -- 172.21.15.32:0/3001511496 wait complete. 2024-08-31T07:13:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:16.987+0000 7fe5fca46740 20 client.15711 trim_cache size 0 max 16384 2024-08-31T07:13:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.006+0000 7fe5fca46740 1 Processor -- start 2024-08-31T07:13:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.006+0000 7fe5fca46740 1 -- start start 2024-08-31T07:13:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.006+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 0x56158656e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.006+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 0x561586564810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.006+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615861a8350 0x56158655fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.006+0000 7fe5fca46740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01ad000 con 0x5615861a8350 2024-08-31T07:13:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.006+0000 7fe5fca46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01aeb40 con 0x56158617f490 2024-08-31T07:13:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.006+0000 7fe5fca46740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01ad6d0 con 0x561586187670 2024-08-31T07:13:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5ee000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 0x561586564810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5ee000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 0x561586564810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55176/0 (socket says 172.21.15.32:55176) 2024-08-31T07:13:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5ee000640 1 -- 172.21.15.32:0/3212561543 learned_addr learned my addr 172.21.15.32:0/3212561543 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5eea00640 1 --2- 172.21.15.32:0/3212561543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 0x56158656e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5ef400640 1 --2- 172.21.15.32:0/3212561543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615861a8350 0x56158655fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5ee000640 1 -- 172.21.15.32:0/3212561543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 msgr2=0x56158656e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5ee000640 1 --2- 172.21.15.32:0/3212561543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 0x56158656e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5ee000640 1 -- 172.21.15.32:0/3212561543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615861a8350 msgr2=0x56158655fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5ee000640 1 --2- 172.21.15.32:0/3212561543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615861a8350 0x56158655fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5ee000640 1 -- 172.21.15.32:0/3212561543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561586550ef0 con 0x561586187670 2024-08-31T07:13:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.007+0000 7fe5ee000640 1 --2- 172.21.15.32:0/3212561543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 0x561586564810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5615865752b0 tx=0x7fe5dc0097d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32942a5e708351ff server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.008+0000 7fe5db400640 1 -- 172.21.15.32:0/3212561543 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc023600 con 0x561586187670 2024-08-31T07:13:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.008+0000 7fe5db400640 1 -- 172.21.15.32:0/3212561543 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5dc023c50 con 0x561586187670 2024-08-31T07:13:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.008+0000 7fe5db400640 1 -- 172.21.15.32:0/3212561543 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc011c70 con 0x561586187670 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.009+0000 7fe5fca46740 1 -- 172.21.15.32:0/3212561543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 msgr2=0x561586564810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.009+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3212561543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 0x561586564810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5615865752b0 tx=0x7fe5dc0097d0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.009+0000 7fe5fca46740 1 -- 172.21.15.32:0/3212561543 shutdown_connections 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.009+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3212561543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615861a8350 0x56158655fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.009+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3212561543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 0x561586564810 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.009+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3212561543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56158617f490 0x56158656e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.009+0000 7fe5fca46740 1 -- 172.21.15.32:0/3212561543 >> 172.21.15.32:0/3212561543 conn(0x56158619c740 msgr2=0x56158654b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.009+0000 7fe5fca46740 1 -- 172.21.15.32:0/3212561543 shutdown_connections 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.009+0000 7fe5fca46740 1 -- 172.21.15.32:0/3212561543 wait complete. 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.010+0000 7fe5fca46740 1 -- start start 2024-08-31T07:13:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.010+0000 7fe5fca46740 1 Processor -- start 2024-08-31T07:13:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.010+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 0x561586567d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.010+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 0x5615865682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.010+0000 7fe5fca46740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615861a8350 0x5615865df520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.010+0000 7fe5fca46740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01ae470 con 0x56158617f490 2024-08-31T07:13:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.010+0000 7fe5fca46740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01adda0 con 0x5615861a8350 2024-08-31T07:13:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.010+0000 7fe5fca46740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01aafe0 con 0x561586187670 2024-08-31T07:13:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.011+0000 7fe5fca46740 10 client.0 _ll_register_callbacks cb 0x7fe5e0009280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.011+0000 7fe5ee000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 0x5615865682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.011+0000 7fe5ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615861a8350 0x5615865df520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.011+0000 7fe5eea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 0x561586567d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.011+0000 7fe5ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615861a8350 0x5615865df520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46350/0 (socket says 172.21.15.32:46350) 2024-08-31T07:13:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.011+0000 7fe5ef400640 1 -- 172.21.15.32:0/3179673464 learned_addr learned my addr 172.21.15.32:0/3179673464 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:17.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.012+0000 7fe5ef400640 1 -- 172.21.15.32:0/3179673464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 msgr2=0x5615865682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.012+0000 7fe5ef400640 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 0x5615865682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.012+0000 7fe5ef400640 1 -- 172.21.15.32:0/3179673464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 msgr2=0x561586567d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.012+0000 7fe5ef400640 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 0x561586567d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.012+0000 7fe5ef400640 1 -- 172.21.15.32:0/3179673464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615865dfa60 con 0x5615861a8350 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.012+0000 7fe5ef400640 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615861a8350 0x5615865df520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009fd0 tx=0x7fe5e801db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2e719fff930bf7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.012+0000 7fe5d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.012+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3179673464 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8019040 con 0x5615861a8350 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.013+0000 7fe5fca46740 1 -- 172.21.15.32:0/3179673464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5dc023c50 con 0x5615861a8350 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.013+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3179673464 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe5e8023040 con 0x5615861a8350 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.013+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3179673464 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5e8013070 con 0x5615861a8350 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.013+0000 7fe5fca46740 10 client.25727 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.013+0000 7fe5fca46740 1 -- 172.21.15.32:0/3179673464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615865e0bd0 con 0x5615861a8350 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.013+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3179673464 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5dc023c50 con 0x5615861a8350 2024-08-31T07:13:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.013+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3179673464 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe5e800e880 con 0x5615861a8350 2024-08-31T07:13:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.014+0000 7fe5fca46740 10 client.25727 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.014+0000 7fe5fca46740 10 client.25727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:13:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.014+0000 7fe5fca46740 20 client.25727 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.014+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56158613c080 0x5615865e3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.014+0000 7fe5fca46740 4 client.25727 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:13:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.014+0000 7fe5fca46740 1 -- 172.21.15.32:0/3179673464 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5615864c2140 con 0x56158613c080 2024-08-31T07:13:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.014+0000 7fe5eea00640 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56158613c080 0x5615865e3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.015+0000 7fe5d8c00640 10 client.25727 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.015+0000 7fe5eea00640 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56158613c080 0x5615865e3210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1e7f1e5fdd751c0 server_cookie=91eb21bf003edf6e in_seq=0 out_seq=0 2024-08-31T07:13:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.016+0000 7fe5d8c00640 1 -- 172.21.15.32:0/3179673464 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5615864c2140 con 0x56158613c080 2024-08-31T07:13:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.016+0000 7fe5d8c00640 10 client.25727 handle_command_reply: tid=0 2024-08-31T07:13:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.016+0000 7fe5fca46740 1 client.25727 shutdown 2024-08-31T07:13:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5fca46740 1 -- 172.21.15.32:0/3179673464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615861a8350 msgr2=0x5615865df520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615861a8350 0x5615865df520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009fd0 tx=0x7fe5e801db40 comp rx=0 tx=0).stop 2024-08-31T07:13:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5fca46740 1 -- 172.21.15.32:0/3179673464 shutdown_connections 2024-08-31T07:13:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56158613c080 0x5615865e3210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615861a8350 0x5615865df520 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561586187670 0x5615865682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5fca46740 1 --2- 172.21.15.32:0/3179673464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56158617f490 0x561586567d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5d8c00640 0 client.25727 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5fca46740 1 -- 172.21.15.32:0/3179673464 >> 172.21.15.32:0/3179673464 conn(0x56158619c740 msgr2=0x56158654b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5fca46740 1 -- 172.21.15.32:0/3179673464 shutdown_connections 2024-08-31T07:13:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.017+0000 7fe5fca46740 1 -- 172.21.15.32:0/3179673464 wait complete. 2024-08-31T07:13:17.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.018+0000 7fe5fca46740 20 client.25727 trim_cache size 0 max 16384 2024-08-31T07:13:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:13:17.020 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:13:17.020 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378", 2024-08-31T07:13:17.020 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:13:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:13:17.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.018+0000 7fe5efe00640 1 -- 172.21.15.32:0/2167550524 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5d40b4d20 msgr2=0x7fe5d40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.018+0000 7fe5efe00640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5d40b4d20 0x7fe5d40b7230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8021000 tx=0x7fe5e801e000 comp rx=0 tx=0).stop 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.018+0000 7fe5efe00640 1 -- 172.21.15.32:0/2167550524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f0105960 msgr2=0x7fe5f019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.018+0000 7fe5efe00640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f0105960 0x7fe5f019c2a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0005940 tx=0x7fe5e00040c0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.019+0000 7fe5efe00640 1 -- 172.21.15.32:0/2167550524 shutdown_connections 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.019+0000 7fe5efe00640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5d40b4d20 0x7fe5d40b7230 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.019+0000 7fe5efe00640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5f010bbf0 0x7fe5f01a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.019+0000 7fe5efe00640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5f010b2a0 0x7fe5f019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.019+0000 7fe5efe00640 1 --2- 172.21.15.32:0/2167550524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5f0105960 0x7fe5f019c2a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.019+0000 7fe5efe00640 1 -- 172.21.15.32:0/2167550524 >> 172.21.15.32:0/2167550524 conn(0x7fe5f01013d0 msgr2=0x7fe5f01037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.019+0000 7fe5efe00640 1 -- 172.21.15.32:0/2167550524 shutdown_connections 2024-08-31T07:13:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:17.020+0000 7fe5efe00640 1 -- 172.21.15.32:0/2167550524 wait complete. 2024-08-31T07:13:17.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:13:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:17 smithi073 ceph-mon[21995]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:17 smithi084 ceph-mon[22387]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:18.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:17 smithi032 ceph-mon[17252]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:18.401 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.095+0000 7f1473200640 1 -- 172.21.15.32:0/1985043871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146c075550 msgr2=0x7f146c075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.095+0000 7f1473200640 1 --2- 172.21.15.32:0/1985043871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146c075550 0x7f146c075960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1460009900 tx=0x7f146002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.095+0000 7f1473200640 1 -- 172.21.15.32:0/1985043871 shutdown_connections 2024-08-31T07:13:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.095+0000 7f1473200640 1 --2- 172.21.15.32:0/1985043871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f146c10fcb0 0x7f146c1120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.095+0000 7f1473200640 1 --2- 172.21.15.32:0/1985043871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146c075ea0 0x7f146c10f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.095+0000 7f1473200640 1 --2- 172.21.15.32:0/1985043871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146c075550 0x7f146c075960 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.095+0000 7f1473200640 1 -- 172.21.15.32:0/1985043871 >> 172.21.15.32:0/1985043871 conn(0x7f146c0ff1e0 msgr2=0x7f146c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.096+0000 7f1473200640 1 -- 172.21.15.32:0/1985043871 shutdown_connections 2024-08-31T07:13:19.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.096+0000 7f1473200640 1 -- 172.21.15.32:0/1985043871 wait complete. 2024-08-31T07:13:19.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.097+0000 7f1473200640 1 Processor -- start 2024-08-31T07:13:19.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.097+0000 7f1473200640 1 -- start start 2024-08-31T07:13:19.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.098+0000 7f1473200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146c075550 0x7f146c19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.098+0000 7f1473200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f146c075ea0 0x7f146c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.098+0000 7f1473200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146c10fcb0 0x7f146c1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.098+0000 7f1473200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1158b0 con 0x7f146c10fcb0 2024-08-31T07:13:19.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.098+0000 7f1473200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f146c115730 con 0x7f146c075550 2024-08-31T07:13:19.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.098+0000 7f1473200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f146c115a30 con 0x7f146c075ea0 2024-08-31T07:13:19.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.098+0000 7f1471e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146c075550 0x7f146c19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.098+0000 7f1471e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146c075550 0x7f146c19c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46362/0 (socket says 172.21.15.32:46362) 2024-08-31T07:13:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.098+0000 7f1471e00640 1 -- 172.21.15.32:0/921257006 learned_addr learned my addr 172.21.15.32:0/921257006 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.099+0000 7f1472800640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146c10fcb0 0x7f146c1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.099+0000 7f1471400640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f146c075ea0 0x7f146c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.099+0000 7f1471e00640 1 -- 172.21.15.32:0/921257006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f146c075ea0 msgr2=0x7f146c19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.099+0000 7f1471e00640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f146c075ea0 0x7f146c19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.099+0000 7f1471e00640 1 -- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146c10fcb0 msgr2=0x7f146c1a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:19.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.099+0000 7f1471e00640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146c10fcb0 0x7f146c1a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.099+0000 7f1471e00640 1 -- 172.21.15.32:0/921257006 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c1a21d0 con 0x7f146c075550 2024-08-31T07:13:19.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.099+0000 7f1472800640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146c10fcb0 0x7f146c1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:19.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.100+0000 7f1471e00640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146c075550 0x7f146c19c260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1460005ab0 tx=0x7f1460003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:19.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.100+0000 7f1466a00640 1 -- 172.21.15.32:0/921257006 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146003f070 con 0x7f146c075550 2024-08-31T07:13:19.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.100+0000 7f1466a00640 1 -- 172.21.15.32:0/921257006 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1460031b60 con 0x7f146c075550 2024-08-31T07:13:19.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.100+0000 7f1466a00640 1 -- 172.21.15.32:0/921257006 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460035030 con 0x7f146c075550 2024-08-31T07:13:19.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.101+0000 7f1473200640 1 -- 172.21.15.32:0/921257006 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c1a2460 con 0x7f146c075550 2024-08-31T07:13:19.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.101+0000 7f1473200640 1 -- 172.21.15.32:0/921257006 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c1a8c20 con 0x7f146c075550 2024-08-31T07:13:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.105+0000 7f1466a00640 1 -- 172.21.15.32:0/921257006 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1460042070 con 0x7f146c075550 2024-08-31T07:13:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.106+0000 7f147b5ba740 1 Processor -- start 2024-08-31T07:13:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.106+0000 7f147b5ba740 1 -- start start 2024-08-31T07:13:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.106+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b96e070 0x559e9bd13b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.106+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 0x559e9bd14060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.106+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b94d390 0x559e9bd145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.106+0000 7f147b5ba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559e9bc8a770 con 0x559e9b94d390 2024-08-31T07:13:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.106+0000 7f147b5ba740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559e9bc79d00 con 0x559e9b9451b0 2024-08-31T07:13:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.106+0000 7f147b5ba740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559e9bc7dcb0 con 0x559e9b96e070 2024-08-31T07:13:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.107+0000 7f1472800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b96e070 0x559e9bd13b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.107+0000 7f1472800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b96e070 0x559e9bd13b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55210/0 (socket says 172.21.15.32:55210) 2024-08-31T07:13:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.107+0000 7f1472800640 1 -- 172.21.15.32:0/1559187794 learned_addr learned my addr 172.21.15.32:0/1559187794 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.107+0000 7f1471400640 1 --2- 172.21.15.32:0/1559187794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b94d390 0x559e9bd145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.107+0000 7f1471e00640 1 --2- 172.21.15.32:0/1559187794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 0x559e9bd14060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.108+0000 7f1466a00640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14540b4ca0 0x7f14540b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.109+0000 7f1472800640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14540b4ca0 0x7f14540b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.109+0000 7f1471400640 1 -- 172.21.15.32:0/1559187794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b96e070 msgr2=0x559e9bd13b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.109+0000 7f1471400640 1 --2- 172.21.15.32:0/1559187794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b96e070 0x559e9bd13b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.109+0000 7f1471400640 1 -- 172.21.15.32:0/1559187794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 msgr2=0x559e9bd14060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.109+0000 7f1471400640 1 --2- 172.21.15.32:0/1559187794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 0x559e9bd14060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.109+0000 7f1471400640 1 -- 172.21.15.32:0/1559187794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9b9d9650 con 0x559e9b94d390 2024-08-31T07:13:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.109+0000 7f1466a00640 1 -- 172.21.15.32:0/921257006 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1460105960 con 0x7f146c075550 2024-08-31T07:13:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.109+0000 7f1471400640 1 --2- 172.21.15.32:0/1559187794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b94d390 0x559e9bd145a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f146c06b4a0 tx=0x7f145c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e44486ed0e43c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.110+0000 7f145b000640 1 -- 172.21.15.32:0/1559187794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c009d70 con 0x559e9b94d390 2024-08-31T07:13:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.110+0000 7f145b000640 1 -- 172.21.15.32:0/1559187794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f145c005020 con 0x559e9b94d390 2024-08-31T07:13:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.110+0000 7f145b000640 1 -- 172.21.15.32:0/1559187794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c0049e0 con 0x559e9b94d390 2024-08-31T07:13:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.110+0000 7f1472800640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14540b4ca0 0x7f14540b71b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1468009fd0 tx=0x7f1468004ab0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.111+0000 7f147b5ba740 1 -- 172.21.15.32:0/1559187794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b94d390 msgr2=0x559e9bd145a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.111+0000 7f147b5ba740 1 --2- 172.21.15.32:0/1559187794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b94d390 0x559e9bd145a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f146c06b4a0 tx=0x7f145c011810 comp rx=0 tx=0).stop 2024-08-31T07:13:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.111+0000 7f147b5ba740 1 -- 172.21.15.32:0/1559187794 shutdown_connections 2024-08-31T07:13:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.111+0000 7f147b5ba740 1 --2- 172.21.15.32:0/1559187794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b94d390 0x559e9bd145a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.111+0000 7f147b5ba740 1 --2- 172.21.15.32:0/1559187794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 0x559e9bd14060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.111+0000 7f147b5ba740 1 --2- 172.21.15.32:0/1559187794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b96e070 0x559e9bd13b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.111+0000 7f147b5ba740 1 -- 172.21.15.32:0/1559187794 >> 172.21.15.32:0/1559187794 conn(0x559e9b962460 msgr2=0x559e9bd11a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.111+0000 7f147b5ba740 1 -- 172.21.15.32:0/1559187794 shutdown_connections 2024-08-31T07:13:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.111+0000 7f147b5ba740 1 -- 172.21.15.32:0/1559187794 wait complete. 2024-08-31T07:13:19.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.112+0000 7f147b5ba740 1 -- start start 2024-08-31T07:13:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.112+0000 7f147b5ba740 1 Processor -- start 2024-08-31T07:13:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b9451b0 0x559e9bd28630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 0x559e9bd28c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 0x559e9bd2dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f147b5ba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559e9badc700 con 0x559e9b96e070 2024-08-31T07:13:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f147b5ba740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559e9bc00cb0 con 0x559e9b94d390 2024-08-31T07:13:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f147b5ba740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559e9bd22300 con 0x559e9b9451b0 2024-08-31T07:13:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f147b5ba740 10 client.0 _ll_register_callbacks cb 0x559e9bc26430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f1472800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 0x559e9bd2dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f1471400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 0x559e9bd28c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f1471400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 0x559e9bd28c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46378/0 (socket says 172.21.15.32:46378) 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f1471400640 1 -- 172.21.15.32:0/340796734 learned_addr learned my addr 172.21.15.32:0/340796734 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.113+0000 7f1471e00640 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b9451b0 0x559e9bd28630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.114+0000 7f1471e00640 1 -- 172.21.15.32:0/340796734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 msgr2=0x559e9bd28c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.114+0000 7f1471e00640 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 0x559e9bd28c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.114+0000 7f1471e00640 1 -- 172.21.15.32:0/340796734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 msgr2=0x559e9bd2dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.114+0000 7f1471e00640 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 0x559e9bd2dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.114+0000 7f1471e00640 1 -- 172.21.15.32:0/340796734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9bc41870 con 0x559e9b9451b0 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.114+0000 7f1472800640 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 0x559e9bd2dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.114+0000 7f1471e00640 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b9451b0 0x559e9bd28630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f14601030d0 tx=0x7f146002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2d0a9b5572c681c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:19.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.114+0000 7f144f400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:19.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.115+0000 7f144f400640 1 -- 172.21.15.32:0/340796734 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600cb890 con 0x559e9b9451b0 2024-08-31T07:13:19.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.115+0000 7f147b5ba740 1 -- 172.21.15.32:0/340796734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f145c005020 con 0x559e9b9451b0 2024-08-31T07:13:19.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.115+0000 7f144f400640 1 -- 172.21.15.32:0/340796734 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f14600d0510 con 0x559e9b9451b0 2024-08-31T07:13:19.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.115+0000 7f144f400640 1 -- 172.21.15.32:0/340796734 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1460103560 con 0x559e9b9451b0 2024-08-31T07:13:19.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.115+0000 7f147b5ba740 10 client.25750 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:19.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.115+0000 7f147b5ba740 1 -- 172.21.15.32:0/340796734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e9bd36b80 con 0x559e9b9451b0 2024-08-31T07:13:19.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.115+0000 7f144f400640 1 -- 172.21.15.32:0/340796734 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f145c005020 con 0x559e9b9451b0 2024-08-31T07:13:19.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.116+0000 7f144f400640 1 -- 172.21.15.32:0/340796734 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f14600d4f00 con 0x559e9b9451b0 2024-08-31T07:13:19.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.116+0000 7f147b5ba740 10 client.25750 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.116+0000 7f147b5ba740 10 client.25750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:13:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.116+0000 7f147b5ba740 20 client.25750 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.116+0000 7f147b5ba740 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559e9b92c9f0 0x559e9bd39f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.116+0000 7f147b5ba740 4 client.25750 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.116+0000 7f147b5ba740 1 -- 172.21.15.32:0/340796734 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e9bc8c6c0 con 0x559e9b92c9f0 2024-08-31T07:13:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.116+0000 7f1471400640 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559e9b92c9f0 0x559e9bd39f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.117+0000 7f1471400640 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559e9b92c9f0 0x559e9bd39f90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b58e6e6c5dd0c9b server_cookie=6b498863a40e0d8e in_seq=0 out_seq=0 2024-08-31T07:13:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.117+0000 7f144f400640 10 client.25750 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:19.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f144f400640 1 -- 172.21.15.32:0/340796734 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e9bc8c6c0 con 0x559e9b92c9f0 2024-08-31T07:13:19.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f144f400640 10 client.25750 handle_command_reply: tid=0 2024-08-31T07:13:19.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f147b5ba740 1 client.25750 shutdown 2024-08-31T07:13:19.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f147b5ba740 1 -- 172.21.15.32:0/340796734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b9451b0 msgr2=0x559e9bd28630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:19.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f147b5ba740 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b9451b0 0x559e9bd28630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f14601030d0 tx=0x7f146002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f147b5ba740 1 -- 172.21.15.32:0/340796734 shutdown_connections 2024-08-31T07:13:19.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f147b5ba740 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559e9b92c9f0 0x559e9bd39f90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f147b5ba740 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 0x559e9bd2dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f147b5ba740 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 0x559e9bd28c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f147b5ba740 1 --2- 172.21.15.32:0/340796734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b9451b0 0x559e9bd28630 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f144f400640 0 client.25750 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:19.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.118+0000 7f147b5ba740 1 -- 172.21.15.32:0/340796734 >> 172.21.15.32:0/340796734 conn(0x559e9b962460 msgr2=0x559e9bd16cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:19.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.119+0000 7f147b5ba740 1 -- 172.21.15.32:0/340796734 shutdown_connections 2024-08-31T07:13:19.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.119+0000 7f147b5ba740 1 -- 172.21.15.32:0/340796734 wait complete. 2024-08-31T07:13:19.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.119+0000 7f147b5ba740 20 client.25750 trim_cache size 0 max 16384 2024-08-31T07:13:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.155+0000 7f147b5ba740 1 Processor -- start 2024-08-31T07:13:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.156+0000 7f147b5ba740 1 -- start start 2024-08-31T07:13:19.157 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:18 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:13:19.157 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:18 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:13:19.157 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:18 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:13:19.157 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:18 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:13:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.156+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b9451b0 0x559e9bd2da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.156+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 0x559e9bd21620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.156+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b96e070 0x559e9bd16c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.156+0000 7f147b5ba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1acfb0 con 0x559e9b9451b0 2024-08-31T07:13:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.156+0000 7f147b5ba740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1aeaf0 con 0x559e9b94d390 2024-08-31T07:13:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.156+0000 7f147b5ba740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1ad680 con 0x559e9b96e070 2024-08-31T07:13:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.156+0000 7f1471e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b9451b0 0x559e9bd2da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.157+0000 7f1471e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b9451b0 0x559e9bd2da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59904/0 (socket says 172.21.15.32:59904) 2024-08-31T07:13:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.157+0000 7f1471e00640 1 -- 172.21.15.32:0/2521609330 learned_addr learned my addr 172.21.15.32:0/2521609330 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.157+0000 7f1471400640 1 --2- 172.21.15.32:0/2521609330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 0x559e9bd21620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.157+0000 7f1471e00640 1 -- 172.21.15.32:0/2521609330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b96e070 msgr2=0x559e9bd16c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:13:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.157+0000 7f1471e00640 1 --2- 172.21.15.32:0/2521609330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b96e070 0x559e9bd16c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.157+0000 7f1471e00640 1 -- 172.21.15.32:0/2521609330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 msgr2=0x559e9bd21620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.157+0000 7f1471e00640 1 --2- 172.21.15.32:0/2521609330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 0x559e9bd21620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.157+0000 7f1471e00640 1 -- 172.21.15.32:0/2521609330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9bd14960 con 0x559e9b9451b0 2024-08-31T07:13:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.158+0000 7f1471e00640 1 --2- 172.21.15.32:0/2521609330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b9451b0 0x559e9bd2da10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1460005f70 tx=0x7f146003b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ce82963a868417 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.158+0000 7f145a600640 1 -- 172.21.15.32:0/2521609330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600cb890 con 0x559e9b9451b0 2024-08-31T07:13:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.159+0000 7f145a600640 1 -- 172.21.15.32:0/2521609330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14600cbc30 con 0x559e9b9451b0 2024-08-31T07:13:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.159+0000 7f145a600640 1 -- 172.21.15.32:0/2521609330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600d5080 con 0x559e9b9451b0 2024-08-31T07:13:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.159+0000 7f147b5ba740 1 -- 172.21.15.32:0/2521609330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b9451b0 msgr2=0x559e9bd2da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.160+0000 7f147b5ba740 1 --2- 172.21.15.32:0/2521609330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b9451b0 0x559e9bd2da10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1460005f70 tx=0x7f146003b800 comp rx=0 tx=0).stop 2024-08-31T07:13:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.160+0000 7f147b5ba740 1 -- 172.21.15.32:0/2521609330 shutdown_connections 2024-08-31T07:13:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.160+0000 7f147b5ba740 1 --2- 172.21.15.32:0/2521609330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b96e070 0x559e9bd16c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.160+0000 7f147b5ba740 1 --2- 172.21.15.32:0/2521609330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b94d390 0x559e9bd21620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.160+0000 7f147b5ba740 1 --2- 172.21.15.32:0/2521609330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b9451b0 0x559e9bd2da10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.160+0000 7f147b5ba740 1 -- 172.21.15.32:0/2521609330 >> 172.21.15.32:0/2521609330 conn(0x559e9b962460 msgr2=0x559e9bd23570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.160+0000 7f147b5ba740 1 -- 172.21.15.32:0/2521609330 shutdown_connections 2024-08-31T07:13:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.160+0000 7f147b5ba740 1 -- 172.21.15.32:0/2521609330 wait complete. 2024-08-31T07:13:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.160+0000 7f147b5ba740 1 -- start start 2024-08-31T07:13:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.161+0000 7f147b5ba740 1 Processor -- start 2024-08-31T07:13:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.161+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 0x559e9bd2ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.161+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b94d390 0x559e9bd2f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.161+0000 7f147b5ba740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 0x559e9bd2f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.161+0000 7f147b5ba740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1ae420 con 0x559e9b96e070 2024-08-31T07:13:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.161+0000 7f147b5ba740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1add50 con 0x559e9b9451b0 2024-08-31T07:13:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.161+0000 7f147b5ba740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1aafb0 con 0x559e9b94d390 2024-08-31T07:13:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.161+0000 7f147b5ba740 10 client.0 _ll_register_callbacks cb 0x7f14600c7fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.162+0000 7f1471e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 0x559e9bd2ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.162+0000 7f1471e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 0x559e9bd2ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46400/0 (socket says 172.21.15.32:46400) 2024-08-31T07:13:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.162+0000 7f1471e00640 1 -- 172.21.15.32:0/3722536691 learned_addr learned my addr 172.21.15.32:0/3722536691 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.162+0000 7f1471e00640 1 -- 172.21.15.32:0/3722536691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b94d390 msgr2=0x559e9bd2f100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:13:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.162+0000 7f1471e00640 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b94d390 0x559e9bd2f100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.162+0000 7f1471e00640 1 -- 172.21.15.32:0/3722536691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 msgr2=0x559e9bd2f640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.162+0000 7f1472800640 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 0x559e9bd2f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.163+0000 7f1471e00640 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 0x559e9bd2f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.163+0000 7f1471e00640 1 -- 172.21.15.32:0/3722536691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9bd33dc0 con 0x559e9b9451b0 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.163+0000 7f1472800640 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 0x559e9bd2f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.163+0000 7f1471e00640 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 0x559e9bd2ebc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1460105d20 tx=0x7f14600d53b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0489df5fdd601a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.163+0000 7f144f400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.163+0000 7f144f400640 1 -- 172.21.15.32:0/3722536691 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600cc120 con 0x559e9b9451b0 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.163+0000 7f144f400640 1 -- 172.21.15.32:0/3722536691 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f14600d04d0 con 0x559e9b9451b0 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.163+0000 7f147b5ba740 1 -- 172.21.15.32:0/3722536691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14600cbc30 con 0x559e9b9451b0 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.164+0000 7f144f400640 1 -- 172.21.15.32:0/3722536691 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14600d06d0 con 0x559e9b9451b0 2024-08-31T07:13:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.164+0000 7f147b5ba740 10 client.25742 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.164+0000 7f147b5ba740 1 -- 172.21.15.32:0/3722536691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e9bd33dc0 con 0x559e9b9451b0 2024-08-31T07:13:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.164+0000 7f144f400640 1 -- 172.21.15.32:0/3722536691 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14600cbc30 con 0x559e9b9451b0 2024-08-31T07:13:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.164+0000 7f144f400640 1 -- 172.21.15.32:0/3722536691 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f14600c7a90 con 0x559e9b9451b0 2024-08-31T07:13:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.165+0000 7f147b5ba740 10 client.25742 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.165+0000 7f147b5ba740 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:13:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.165+0000 7f147b5ba740 20 client.25742 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.165+0000 7f147b5ba740 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559e9b901da0 0x559e9bda70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.165+0000 7f147b5ba740 4 client.25742 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:13:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.165+0000 7f147b5ba740 1 -- 172.21.15.32:0/3722536691 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e9bc88160 con 0x559e9b901da0 2024-08-31T07:13:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.167+0000 7f1471400640 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559e9b901da0 0x559e9bda70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:19.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.167+0000 7f1471400640 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559e9b901da0 0x559e9bda70b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541196cd4d421cbc server_cookie=aaf440bf67bf7267 in_seq=0 out_seq=0 2024-08-31T07:13:19.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.167+0000 7f144f400640 10 client.25742 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:19.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.168+0000 7f144f400640 1 -- 172.21.15.32:0/3722536691 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x559e9bc88160 con 0x559e9b901da0 2024-08-31T07:13:19.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.168+0000 7f144f400640 10 client.25742 handle_command_reply: tid=0 2024-08-31T07:13:19.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.168+0000 7f147b5ba740 1 client.25742 shutdown 2024-08-31T07:13:19.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.169+0000 7f147b5ba740 1 -- 172.21.15.32:0/3722536691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 msgr2=0x559e9bd2ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.169+0000 7f147b5ba740 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 0x559e9bd2ebc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1460105d20 tx=0x7f14600d53b0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.169+0000 7f147b5ba740 1 -- 172.21.15.32:0/3722536691 shutdown_connections 2024-08-31T07:13:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.169+0000 7f147b5ba740 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559e9b901da0 0x559e9bda70b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.169+0000 7f147b5ba740 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559e9b96e070 0x559e9bd2f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.169+0000 7f147b5ba740 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9b94d390 0x559e9bd2f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.169+0000 7f147b5ba740 1 --2- 172.21.15.32:0/3722536691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e9b9451b0 0x559e9bd2ebc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.169+0000 7f147b5ba740 1 -- 172.21.15.32:0/3722536691 >> 172.21.15.32:0/3722536691 conn(0x559e9b962460 msgr2=0x559e9bd16c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.169+0000 7f144f400640 0 client.25742 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.169+0000 7f147b5ba740 1 -- 172.21.15.32:0/3722536691 shutdown_connections 2024-08-31T07:13:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.170+0000 7f147b5ba740 1 -- 172.21.15.32:0/3722536691 wait complete. 2024-08-31T07:13:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.170+0000 7f147b5ba740 20 client.25742 trim_cache size 0 max 16384 2024-08-31T07:13:19.176 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:13:19.176 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:13:19.176 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:13:19.176 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:13:19.177 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:13:19.177 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378", 2024-08-31T07:13:19.177 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:13:19.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:19.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:13:19.177 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:13:19.178 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:13:19.178 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:13:19.178 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:13:19.178 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:13:19.178 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:13:19.178 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:13:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:13:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:13:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:13:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:13:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:13:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:13:19.179 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:13:19.180 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:13:19.180 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:13:19.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:19.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:19.180 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:13:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.171+0000 7f1473200640 1 -- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14540b4ca0 msgr2=0x7f14540b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.171+0000 7f1473200640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14540b4ca0 0x7f14540b71b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1468009fd0 tx=0x7f1468004ab0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.171+0000 7f1473200640 1 -- 172.21.15.32:0/921257006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146c075550 msgr2=0x7f146c19c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.171+0000 7f1473200640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146c075550 0x7f146c19c260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1460005ab0 tx=0x7f1460003c10 comp rx=0 tx=0).stop 2024-08-31T07:13:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.171+0000 7f1473200640 1 -- 172.21.15.32:0/921257006 shutdown_connections 2024-08-31T07:13:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.171+0000 7f1473200640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14540b4ca0 0x7f14540b71b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.171+0000 7f1473200640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146c10fcb0 0x7f146c1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.171+0000 7f1473200640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f146c075ea0 0x7f146c19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.171+0000 7f1473200640 1 --2- 172.21.15.32:0/921257006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146c075550 0x7f146c19c260 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:19.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.171+0000 7f1473200640 1 -- 172.21.15.32:0/921257006 >> 172.21.15.32:0/921257006 conn(0x7f146c0ff1e0 msgr2=0x7f146c1097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:19.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.172+0000 7f1473200640 1 -- 172.21.15.32:0/921257006 shutdown_connections 2024-08-31T07:13:19.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:19.173+0000 7f1473200640 1 -- 172.21.15.32:0/921257006 wait complete. 2024-08-31T07:13:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:18 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:13:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:18 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:13:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:18 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:13:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:18 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:13:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:18 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:13:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:18 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:13:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:18 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:13:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:18 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:13:20.088 INFO:tasks.cephfs.filesystem:scrub status for tag:2e220b7d-14bd-4c0f-9dab-7cb5e62ab378 - {'path': '/', 'tag': '2e220b7d-14bd-4c0f-9dab-7cb5e62ab378', 'options': 'recursive,force'} 2024-08-31T07:13:20.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:19 smithi084 ceph-mon[22387]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-08-31T07:13:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:19 smithi032 ceph-mon[17252]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-08-31T07:13:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:19 smithi073 ceph-mon[21995]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-08-31T07:13:22.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:21 smithi084 ceph-mon[22387]: pgmap v386: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-31T07:13:22.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:21 smithi032 ceph-mon[17252]: pgmap v386: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-31T07:13:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:21 smithi073 ceph-mon[21995]: pgmap v386: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-31T07:13:24.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:23 smithi084 ceph-mon[22387]: pgmap v387: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-31T07:13:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:23 smithi032 ceph-mon[17252]: pgmap v387: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-31T07:13:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:23 smithi073 ceph-mon[21995]: pgmap v387: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-31T07:13:26.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:25 smithi084 ceph-mon[22387]: pgmap v388: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-31T07:13:26.361 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:25 smithi032 ceph-mon[17252]: pgmap v388: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-31T07:13:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:25 smithi073 ceph-mon[21995]: pgmap v388: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-31T07:13:28.231 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:27 smithi073 ceph-mon[21995]: pgmap v389: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 853 B/s wr, 10 op/s 2024-08-31T07:13:28.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:27 smithi084 ceph-mon[22387]: pgmap v389: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 853 B/s wr, 10 op/s 2024-08-31T07:13:28.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:27 smithi032 ceph-mon[17252]: pgmap v389: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 853 B/s wr, 10 op/s 2024-08-31T07:13:30.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:29 smithi084 ceph-mon[22387]: pgmap v390: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-08-31T07:13:30.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:29 smithi032 ceph-mon[17252]: pgmap v390: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-08-31T07:13:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:29 smithi073 ceph-mon[21995]: pgmap v390: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-08-31T07:13:32.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:31 smithi084 ceph-mon[22387]: pgmap v391: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:32.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:31 smithi032 ceph-mon[17252]: pgmap v391: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:31 smithi073 ceph-mon[21995]: pgmap v391: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:34.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:33 smithi084 ceph-mon[22387]: pgmap v392: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 11 op/s 2024-08-31T07:13:34.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:33 smithi032 ceph-mon[17252]: pgmap v392: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 11 op/s 2024-08-31T07:13:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:33 smithi073 ceph-mon[21995]: pgmap v392: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 11 op/s 2024-08-31T07:13:35.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:13:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:35 smithi032 ceph-mon[17252]: pgmap v393: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:36.228 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:36.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:35 smithi084 ceph-mon[22387]: pgmap v393: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:36.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:35 smithi073 ceph-mon[21995]: pgmap v393: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.949+0000 7f6e55800640 1 -- 172.21.15.32:0/3373130678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e50107790 msgr2=0x7f6e50107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.949+0000 7f6e55800640 1 --2- 172.21.15.32:0/3373130678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e50107790 0x7f6e50107c70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009be0 tx=0x7f6e4802a6c0 comp rx=0 tx=0).stop 2024-08-31T07:13:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.950+0000 7f6e55800640 1 -- 172.21.15.32:0/3373130678 shutdown_connections 2024-08-31T07:13:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.950+0000 7f6e55800640 1 --2- 172.21.15.32:0/3373130678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e50107790 0x7f6e50107c70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.950+0000 7f6e55800640 1 --2- 172.21.15.32:0/3373130678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e50102ca0 0x7f6e501071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.950+0000 7f6e55800640 1 --2- 172.21.15.32:0/3373130678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e50102350 0x7f6e50102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.950+0000 7f6e55800640 1 -- 172.21.15.32:0/3373130678 >> 172.21.15.32:0/3373130678 conn(0x7f6e500fe060 msgr2=0x7f6e501004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.950+0000 7f6e55800640 1 -- 172.21.15.32:0/3373130678 shutdown_connections 2024-08-31T07:13:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.950+0000 7f6e55800640 1 -- 172.21.15.32:0/3373130678 wait complete. 2024-08-31T07:13:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.951+0000 7f6e55800640 1 Processor -- start 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.951+0000 7f6e55800640 1 -- start start 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.952+0000 7f6e55800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e50102350 0x7f6e501a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.952+0000 7f6e55800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e50102ca0 0x7f6e501a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.952+0000 7f6e55800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e50107790 0x7f6e501a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.952+0000 7f6e55800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50119c50 con 0x7f6e50102350 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.952+0000 7f6e55800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50119ad0 con 0x7f6e50102ca0 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.952+0000 7f6e55800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50119dd0 con 0x7f6e50107790 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.952+0000 7f6e4f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e50102ca0 0x7f6e501a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.952+0000 7f6e4f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e50102ca0 0x7f6e501a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59120/0 (socket says 172.21.15.32:59120) 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.952+0000 7f6e4f400640 1 -- 172.21.15.32:0/429684983 learned_addr learned my addr 172.21.15.32:0/429684983 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.952+0000 7f6e54e00640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e50107790 0x7f6e501a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.953+0000 7f6e4fe00640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e50102350 0x7f6e501a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.953+0000 7f6e4f400640 1 -- 172.21.15.32:0/429684983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e50107790 msgr2=0x7f6e501a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.953+0000 7f6e4f400640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e50107790 0x7f6e501a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.953+0000 7f6e4f400640 1 -- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e50102350 msgr2=0x7f6e501a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.953+0000 7f6e4f400640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e50102350 0x7f6e501a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.954+0000 7f6e4f400640 1 -- 172.21.15.32:0/429684983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e501a6550 con 0x7f6e50102ca0 2024-08-31T07:13:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.954+0000 7f6e4fe00640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e50102350 0x7f6e501a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.954+0000 7f6e4f400640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e50102ca0 0x7f6e501a0b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00e9c0 tx=0x7f6e3c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.955+0000 7f6e4cc00640 1 -- 172.21.15.32:0/429684983 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00ccc0 con 0x7f6e50102ca0 2024-08-31T07:13:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.955+0000 7f6e55800640 1 -- 172.21.15.32:0/429684983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e501a6840 con 0x7f6e50102ca0 2024-08-31T07:13:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.955+0000 7f6e55800640 1 -- 172.21.15.32:0/429684983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e501acfa0 con 0x7f6e50102ca0 2024-08-31T07:13:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.955+0000 7f6e4cc00640 1 -- 172.21.15.32:0/429684983 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e3c0054d0 con 0x7f6e50102ca0 2024-08-31T07:13:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.955+0000 7f6e4cc00640 1 -- 172.21.15.32:0/429684983 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c010430 con 0x7f6e50102ca0 2024-08-31T07:13:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.957+0000 7f6e55800640 1 -- 172.21.15.32:0/429684983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e300052c0 con 0x7f6e50102ca0 2024-08-31T07:13:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.962+0000 7f6e4cc00640 1 -- 172.21.15.32:0/429684983 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6e3c0046c0 con 0x7f6e50102ca0 2024-08-31T07:13:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.966+0000 7f6e4cc00640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6e340b4d20 0x7f6e340b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.966+0000 7f6e4cc00640 1 -- 172.21.15.32:0/429684983 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6e3c0e5820 con 0x7f6e50102ca0 2024-08-31T07:13:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.967+0000 7f6e4cc00640 1 -- 172.21.15.32:0/429684983 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6e3c1161d0 con 0x7f6e50102ca0 2024-08-31T07:13:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.967+0000 7f6e4fe00640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6e340b4d20 0x7f6e340b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:36.969+0000 7f6e4fe00640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6e340b4d20 0x7f6e340b7230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005fd0 tx=0x7f6e40009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.207+0000 7f6e55800640 1 -- 172.21.15.32:0/429684983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e300055b0 con 0x7f6e50102ca0 2024-08-31T07:13:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.208+0000 7f6e4cc00640 1 -- 172.21.15.32:0/429684983 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f6e3c0ac630 con 0x7f6e50102ca0 2024-08-31T07:13:37.209 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:13:37.210 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:13:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:13:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.211+0000 7f6e55800640 1 -- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6e340b4d20 msgr2=0x7f6e340b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.211+0000 7f6e55800640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6e340b4d20 0x7f6e340b7230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005fd0 tx=0x7f6e40009260 comp rx=0 tx=0).stop 2024-08-31T07:13:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.211+0000 7f6e55800640 1 -- 172.21.15.32:0/429684983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e50102ca0 msgr2=0x7f6e501a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.211+0000 7f6e55800640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e50102ca0 0x7f6e501a0b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00e9c0 tx=0x7f6e3c00ee80 comp rx=0 tx=0).stop 2024-08-31T07:13:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.212+0000 7f6e55800640 1 -- 172.21.15.32:0/429684983 shutdown_connections 2024-08-31T07:13:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.212+0000 7f6e55800640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6e340b4d20 0x7f6e340b7230 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.212+0000 7f6e55800640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e50107790 0x7f6e501a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.212+0000 7f6e55800640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e50102ca0 0x7f6e501a0b80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.212+0000 7f6e55800640 1 --2- 172.21.15.32:0/429684983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e50102350 0x7f6e501a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.212+0000 7f6e55800640 1 -- 172.21.15.32:0/429684983 >> 172.21.15.32:0/429684983 conn(0x7f6e500fe060 msgr2=0x7f6e501051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:37.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.212+0000 7f6e55800640 1 -- 172.21.15.32:0/429684983 shutdown_connections 2024-08-31T07:13:37.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:37.212+0000 7f6e55800640 1 -- 172.21.15.32:0/429684983 wait complete. 2024-08-31T07:13:38.138 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 7b061927 --release --await' 2024-08-31T07:13:38.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 7b061927 --release --await 2024-08-31T07:13:38.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:37 smithi084 ceph-mon[22387]: pgmap v394: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:38.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:37 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/429684983' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:38.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:37 smithi032 ceph-mon[17252]: pgmap v394: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:38.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:37 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/429684983' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:38.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:37 smithi073 ceph-mon[21995]: pgmap v394: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-31T07:13:38.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:37 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/429684983' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:39.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.286+0000 7f3e43200640 1 -- 172.21.15.32:0/385352181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4410d9c0 msgr2=0x7f3e4410feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:39.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.286+0000 7f3e43200640 1 --2- 172.21.15.32:0/385352181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4410d9c0 0x7f3e4410feb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009ca0 tx=0x7f3e3c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.286+0000 7f3e43200640 1 -- 172.21.15.32:0/385352181 shutdown_connections 2024-08-31T07:13:39.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.286+0000 7f3e43200640 1 --2- 172.21.15.32:0/385352181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4410d9c0 0x7f3e4410feb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.286+0000 7f3e43200640 1 --2- 172.21.15.32:0/385352181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e44104eb0 0x7f3e4410d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.286+0000 7f3e43200640 1 --2- 172.21.15.32:0/385352181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e44104560 0x7f3e44104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.286+0000 7f3e43200640 1 -- 172.21.15.32:0/385352181 >> 172.21.15.32:0/385352181 conn(0x7f3e440fe060 msgr2=0x7f3e441004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:39.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.286+0000 7f3e43200640 1 -- 172.21.15.32:0/385352181 shutdown_connections 2024-08-31T07:13:39.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.286+0000 7f3e43200640 1 -- 172.21.15.32:0/385352181 wait complete. 2024-08-31T07:13:39.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.287+0000 7f3e43200640 1 Processor -- start 2024-08-31T07:13:39.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.288+0000 7f3e43200640 1 -- start start 2024-08-31T07:13:39.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.288+0000 7f3e43200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e44104560 0x7f3e4410a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.288+0000 7f3e43200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e44104eb0 0x7f3e4410aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.288+0000 7f3e43200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e4410d9c0 0x7f3e4410b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.288+0000 7f3e43200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44103d30 con 0x7f3e44104560 2024-08-31T07:13:39.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.288+0000 7f3e43200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44103bb0 con 0x7f3e4410d9c0 2024-08-31T07:13:39.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.288+0000 7f3e43200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44103eb0 con 0x7f3e44104eb0 2024-08-31T07:13:39.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e41e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e44104560 0x7f3e4410a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e42800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e4410d9c0 0x7f3e4410b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e41e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e44104560 0x7f3e4410a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52120/0 (socket says 172.21.15.32:52120) 2024-08-31T07:13:39.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e41e00640 1 -- 172.21.15.32:0/2849733443 learned_addr learned my addr 172.21.15.32:0/2849733443 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:39.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e42800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e4410d9c0 0x7f3e4410b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59134/0 (socket says 172.21.15.32:59134) 2024-08-31T07:13:39.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e42800640 1 -- 172.21.15.32:0/2849733443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e44104eb0 msgr2=0x7f3e4410aee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:13:39.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e42800640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e44104eb0 0x7f3e4410aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e42800640 1 -- 172.21.15.32:0/2849733443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e44104560 msgr2=0x7f3e4410a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:39.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e42800640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e44104560 0x7f3e4410a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e42800640 1 -- 172.21.15.32:0/2849733443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441ac6a0 con 0x7f3e4410d9c0 2024-08-31T07:13:39.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.289+0000 7f3e41e00640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e44104560 0x7f3e4410a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:39.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.290+0000 7f3e42800640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e4410d9c0 0x7f3e4410b420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c005e30 tx=0x7f3e3c0317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:39.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.290+0000 7f3e3aa00640 1 -- 172.21.15.32:0/2849733443 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c003b70 con 0x7f3e4410d9c0 2024-08-31T07:13:39.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.290+0000 7f3e3aa00640 1 -- 172.21.15.32:0/2849733443 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3c003d10 con 0x7f3e4410d9c0 2024-08-31T07:13:39.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.290+0000 7f3e3aa00640 1 -- 172.21.15.32:0/2849733443 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c02f990 con 0x7f3e4410d9c0 2024-08-31T07:13:39.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.290+0000 7f3e43200640 1 -- 172.21.15.32:0/2849733443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441ac930 con 0x7f3e4410d9c0 2024-08-31T07:13:39.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.290+0000 7f3e43200640 1 -- 172.21.15.32:0/2849733443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441ace20 con 0x7f3e4410d9c0 2024-08-31T07:13:39.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.294+0000 7f3e3aa00640 1 -- 172.21.15.32:0/2849733443 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3e3c02fb30 con 0x7f3e4410d9c0 2024-08-31T07:13:39.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.295+0000 7f3e4fde0740 1 Processor -- start 2024-08-31T07:13:39.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.295+0000 7f3e4fde0740 1 -- start start 2024-08-31T07:13:39.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.296+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 0x5588533f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.296+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 0x5588533f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.296+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 0x5588533f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.296+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55885336aa90 con 0x558853025660 2024-08-31T07:13:39.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.296+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55885335a020 con 0x55885302d840 2024-08-31T07:13:39.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.296+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55885335dfd0 con 0x55885304e520 2024-08-31T07:13:39.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.296+0000 7f3e41400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 0x5588533f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.296+0000 7f3e41400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 0x5588533f4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41670/0 (socket says 172.21.15.32:41670) 2024-08-31T07:13:39.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.296+0000 7f3e41400640 1 -- 172.21.15.32:0/369942418 learned_addr learned my addr 172.21.15.32:0/369942418 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.297+0000 7f3e3aa00640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e280b4d20 0x7f3e280b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.297+0000 7f3e41400640 1 --2- 172.21.15.32:0/369942418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 0x5588533f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.297+0000 7f3e3aa00640 1 -- 172.21.15.32:0/2849733443 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f3e3c106570 con 0x7f3e4410d9c0 2024-08-31T07:13:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.297+0000 7f3e41400640 1 -- 172.21.15.32:0/369942418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 msgr2=0x5588533f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.297+0000 7f3e41400640 1 --2- 172.21.15.32:0/369942418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 0x5588533f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.297+0000 7f3e41400640 1 -- 172.21.15.32:0/369942418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 msgr2=0x5588533f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.297+0000 7f3e41400640 1 --2- 172.21.15.32:0/369942418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 0x5588533f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.297+0000 7f3e41400640 1 -- 172.21.15.32:0/369942418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588530b9b00 con 0x55885304e520 2024-08-31T07:13:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.298+0000 7f3e41400640 1 --2- 172.21.15.32:0/369942418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 0x5588533f4090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009fd0 tx=0x7f3e30009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d5494f9e2b2824 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:39.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.298+0000 7f3e42800640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e280b4d20 0x7f3e280b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.298+0000 7f3e2f400640 1 -- 172.21.15.32:0/369942418 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30013110 con 0x55885304e520 2024-08-31T07:13:39.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.298+0000 7f3e2f400640 1 -- 172.21.15.32:0/369942418 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e300132b0 con 0x55885304e520 2024-08-31T07:13:39.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.298+0000 7f3e2f400640 1 -- 172.21.15.32:0/369942418 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300047a0 con 0x55885304e520 2024-08-31T07:13:39.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.299+0000 7f3e4fde0740 1 -- 172.21.15.32:0/369942418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 msgr2=0x5588533f4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:39.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.299+0000 7f3e42800640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e280b4d20 0x7f3e280b7230 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c02ade0 tx=0x7f3e3c0095c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:39.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.299+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/369942418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 0x5588533f4090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009fd0 tx=0x7f3e30009660 comp rx=0 tx=0).stop 2024-08-31T07:13:39.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.299+0000 7f3e4fde0740 1 -- 172.21.15.32:0/369942418 shutdown_connections 2024-08-31T07:13:39.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.300+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/369942418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 0x5588533f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.300+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/369942418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 0x5588533f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.300+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/369942418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 0x5588533f4090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.300+0000 7f3e4fde0740 1 -- 172.21.15.32:0/369942418 >> 172.21.15.32:0/369942418 conn(0x558853042910 msgr2=0x5588533f1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:39.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.300+0000 7f3e4fde0740 1 -- 172.21.15.32:0/369942418 shutdown_connections 2024-08-31T07:13:39.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.300+0000 7f3e4fde0740 1 -- 172.21.15.32:0/369942418 wait complete. 2024-08-31T07:13:39.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.301+0000 7f3e4fde0740 1 -- start start 2024-08-31T07:13:39.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.301+0000 7f3e4fde0740 1 Processor -- start 2024-08-31T07:13:39.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 0x5588533ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 0x5588533ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 0x5588533fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5588531bcbb0 con 0x55885302d840 2024-08-31T07:13:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5588532e0f90 con 0x55885304e520 2024-08-31T07:13:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558853402870 con 0x558853025660 2024-08-31T07:13:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e4fde0740 10 client.0 _ll_register_callbacks cb 0x558853306710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:39.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e41400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 0x5588533ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e41400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 0x5588533ff4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41686/0 (socket says 172.21.15.32:41686) 2024-08-31T07:13:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e41400640 1 -- 172.21.15.32:0/1363310925 learned_addr learned my addr 172.21.15.32:0/1363310925 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.302+0000 7f3e41400640 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 0x5588533fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.303+0000 7f3e41e00640 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 0x5588533ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.303+0000 7f3e41400640 1 -- 172.21.15.32:0/1363310925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 msgr2=0x5588533ff4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.303+0000 7f3e41400640 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 0x5588533ff4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.303+0000 7f3e41400640 1 -- 172.21.15.32:0/1363310925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 msgr2=0x5588533ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.303+0000 7f3e41400640 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 0x5588533ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.303+0000 7f3e41400640 1 -- 172.21.15.32:0/1363310925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558853130960 con 0x55885304e520 2024-08-31T07:13:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.303+0000 7f3e41e00640 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 0x5588533ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.303+0000 7f3e41400640 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 0x5588533fff30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e30005380 tx=0x7f3e30005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a9e25c09f1dcf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.303+0000 7f3e23e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:13:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.304+0000 7f3e23e00640 1 -- 172.21.15.32:0/1363310925 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30013390 con 0x55885304e520 2024-08-31T07:13:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.304+0000 7f3e4fde0740 1 -- 172.21.15.32:0/1363310925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5588532f8e30 con 0x55885304e520 2024-08-31T07:13:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.304+0000 7f3e23e00640 1 -- 172.21.15.32:0/1363310925 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f3e30004020 con 0x55885304e520 2024-08-31T07:13:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.304+0000 7f3e23e00640 1 -- 172.21.15.32:0/1363310925 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e30004220 con 0x55885304e520 2024-08-31T07:13:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.304+0000 7f3e4fde0740 10 client.25760 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:39.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.304+0000 7f3e4fde0740 1 -- 172.21.15.32:0/1363310925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55885340e340 con 0x55885304e520 2024-08-31T07:13:39.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.304+0000 7f3e23e00640 1 -- 172.21.15.32:0/1363310925 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5588532f8e30 con 0x55885304e520 2024-08-31T07:13:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.305+0000 7f3e23e00640 1 -- 172.21.15.32:0/1363310925 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3e300046c0 con 0x55885304e520 2024-08-31T07:13:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.305+0000 7f3e4fde0740 10 client.25760 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.305+0000 7f3e4fde0740 10 client.25760 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:13:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.305+0000 7f3e4fde0740 20 client.25760 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.305+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55885300cea0 0x5588534116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.305+0000 7f3e4fde0740 4 client.25760 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.305+0000 7f3e4fde0740 1 -- 172.21.15.32:0/1363310925 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55885336c9e0 con 0x55885300cea0 2024-08-31T07:13:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.305+0000 7f3e41e00640 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55885300cea0 0x5588534116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.306+0000 7f3e41e00640 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55885300cea0 0x5588534116a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f310ff0bf586751 server_cookie=98d44cc250a82008 in_seq=0 out_seq=0 2024-08-31T07:13:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.306+0000 7f3e23e00640 10 client.25760 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e23e00640 1 -- 172.21.15.32:0/1363310925 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55885336c9e0 con 0x55885300cea0 2024-08-31T07:13:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e23e00640 10 client.25760 handle_command_reply: tid=0 2024-08-31T07:13:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 client.25760 shutdown 2024-08-31T07:13:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 -- 172.21.15.32:0/1363310925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 msgr2=0x5588533fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 0x5588533fff30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e30005380 tx=0x7f3e30005f90 comp rx=0 tx=0).stop 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 -- 172.21.15.32:0/1363310925 shutdown_connections 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55885300cea0 0x5588534116a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 0x5588533fff30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 0x5588533ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/1363310925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 0x5588533ff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 -- 172.21.15.32:0/1363310925 >> 172.21.15.32:0/1363310925 conn(0x558853042910 msgr2=0x5588533f7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e23e00640 0 client.25760 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 -- 172.21.15.32:0/1363310925 shutdown_connections 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.307+0000 7f3e4fde0740 1 -- 172.21.15.32:0/1363310925 wait complete. 2024-08-31T07:13:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.308+0000 7f3e4fde0740 20 client.25760 trim_cache size 0 max 16384 2024-08-31T07:13:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.326+0000 7f3e4fde0740 1 Processor -- start 2024-08-31T07:13:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.326+0000 7f3e4fde0740 1 -- start start 2024-08-31T07:13:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 0x55885340bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 0x5588533f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 0x5588533fc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b5700 con 0x55885302d840 2024-08-31T07:13:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b7240 con 0x55885304e520 2024-08-31T07:13:39.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b5dd0 con 0x558853025660 2024-08-31T07:13:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e41400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 0x55885340bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e41400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 0x55885340bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41688/0 (socket says 172.21.15.32:41688) 2024-08-31T07:13:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e41400640 1 -- 172.21.15.32:0/3060111030 learned_addr learned my addr 172.21.15.32:0/3060111030 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e41400640 1 --2- 172.21.15.32:0/3060111030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 0x5588533fc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.327+0000 7f3e41e00640 1 --2- 172.21.15.32:0/3060111030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 0x5588533f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.328+0000 7f3e41400640 1 -- 172.21.15.32:0/3060111030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 msgr2=0x55885340bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.328+0000 7f3e41400640 1 --2- 172.21.15.32:0/3060111030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 0x55885340bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.328+0000 7f3e41400640 1 -- 172.21.15.32:0/3060111030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 msgr2=0x5588533f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:39.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.328+0000 7f3e41400640 1 --2- 172.21.15.32:0/3060111030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 0x5588533f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.328+0000 7f3e41400640 1 -- 172.21.15.32:0/3060111030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558853402300 con 0x55885304e520 2024-08-31T07:13:39.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.328+0000 7f3e41400640 1 --2- 172.21.15.32:0/3060111030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 0x5588533fc6a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000f010 tx=0x7f3e30005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ab8cb5b097b18d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:39.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.328+0000 7f3e2ea00640 1 -- 172.21.15.32:0/3060111030 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300098d0 con 0x55885304e520 2024-08-31T07:13:39.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.328+0000 7f3e2ea00640 1 -- 172.21.15.32:0/3060111030 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e30009a70 con 0x55885304e520 2024-08-31T07:13:39.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.328+0000 7f3e2ea00640 1 -- 172.21.15.32:0/3060111030 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000e720 con 0x55885304e520 2024-08-31T07:13:39.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.329+0000 7f3e4fde0740 1 -- 172.21.15.32:0/3060111030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 msgr2=0x5588533fc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:39.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.329+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/3060111030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 0x5588533fc6a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000f010 tx=0x7f3e30005850 comp rx=0 tx=0).stop 2024-08-31T07:13:39.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.329+0000 7f3e4fde0740 1 -- 172.21.15.32:0/3060111030 shutdown_connections 2024-08-31T07:13:39.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.329+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/3060111030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885304e520 0x5588533fc6a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.329+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/3060111030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55885302d840 0x5588533f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.329+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/3060111030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558853025660 0x55885340bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.329+0000 7f3e4fde0740 1 -- 172.21.15.32:0/3060111030 >> 172.21.15.32:0/3060111030 conn(0x558853042910 msgr2=0x5588533f4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:39.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.330+0000 7f3e4fde0740 1 -- 172.21.15.32:0/3060111030 shutdown_connections 2024-08-31T07:13:39.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.330+0000 7f3e4fde0740 1 -- 172.21.15.32:0/3060111030 wait complete. 2024-08-31T07:13:39.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.331+0000 7f3e4fde0740 1 -- start start 2024-08-31T07:13:39.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.331+0000 7f3e4fde0740 1 Processor -- start 2024-08-31T07:13:39.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.331+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 0x5588534084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.331+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 0x558853403950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.331+0000 7f3e4fde0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 0x558853403fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.331+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b6b70 con 0x558853025660 2024-08-31T07:13:39.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.331+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b64a0 con 0x55885302d840 2024-08-31T07:13:39.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.331+0000 7f3e4fde0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b36e0 con 0x55885304e520 2024-08-31T07:13:39.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.332+0000 7f3e4fde0740 10 client.0 _ll_register_callbacks cb 0x7f3e30013f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:39.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.332+0000 7f3e41400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 0x5588534084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.332+0000 7f3e41400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 0x5588534084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52150/0 (socket says 172.21.15.32:52150) 2024-08-31T07:13:39.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.332+0000 7f3e41400640 1 -- 172.21.15.32:0/948425762 learned_addr learned my addr 172.21.15.32:0/948425762 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:39.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.332+0000 7f3e41e00640 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 0x558853403950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.332+0000 7f3e41400640 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 0x558853403fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.332+0000 7f3e41e00640 1 -- 172.21.15.32:0/948425762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 msgr2=0x558853403fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:39.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.332+0000 7f3e41e00640 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 0x558853403fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.332+0000 7f3e41e00640 1 -- 172.21.15.32:0/948425762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 msgr2=0x5588534084a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:39.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.333+0000 7f3e41e00640 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 0x5588534084a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.333+0000 7f3e41e00640 1 -- 172.21.15.32:0/948425762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558853404760 con 0x55885302d840 2024-08-31T07:13:39.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.333+0000 7f3e41400640 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 0x5588534084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.333+0000 7f3e41e00640 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 0x558853403950 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x558853411c40 tx=0x7f3e34013a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1f413a9d6a070de server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.333+0000 7f3e23e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.333+0000 7f3e23e00640 1 -- 172.21.15.32:0/948425762 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34021040 con 0x55885302d840 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.333+0000 7f3e23e00640 1 -- 172.21.15.32:0/948425762 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f3e34004310 con 0x55885302d840 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.333+0000 7f3e4fde0740 1 -- 172.21.15.32:0/948425762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e30009a70 con 0x55885302d840 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.334+0000 7f3e23e00640 1 -- 172.21.15.32:0/948425762 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e340035f0 con 0x55885302d840 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.334+0000 7f3e4fde0740 10 client.25766 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.334+0000 7f3e4fde0740 1 -- 172.21.15.32:0/948425762 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55885347c020 con 0x55885302d840 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.334+0000 7f3e23e00640 1 -- 172.21.15.32:0/948425762 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e30009a70 con 0x55885302d840 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.334+0000 7f3e23e00640 1 -- 172.21.15.32:0/948425762 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3e3401f8b0 con 0x55885302d840 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.335+0000 7f3e4fde0740 10 client.25766 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.335+0000 7f3e4fde0740 10 client.25766 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.335+0000 7f3e4fde0740 20 client.25766 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:39.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.335+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558852fe2250 0x55885347e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:39.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.335+0000 7f3e4fde0740 4 client.25766 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7b061927", "release": true, "await": true}] 2024-08-31T07:13:39.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.335+0000 7f3e4fde0740 1 -- 172.21.15.32:0/948425762 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7b061927", "release": true, "await": true}) -- 0x558853368480 con 0x558852fe2250 2024-08-31T07:13:39.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.335+0000 7f3e41400640 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558852fe2250 0x55885347e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:39.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.336+0000 7f3e23e00640 10 client.25766 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:39.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.336+0000 7f3e41400640 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558852fe2250 0x55885347e740 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8ec95d6654824e server_cookie=848ea34d4ecdeec5 in_seq=0 out_seq=0 2024-08-31T07:13:39.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.339+0000 7f3e23e00640 1 -- 172.21.15.32:0/948425762 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558853368480 con 0x558852fe2250 2024-08-31T07:13:39.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.339+0000 7f3e23e00640 10 client.25766 handle_command_reply: tid=0 2024-08-31T07:13:39.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.339+0000 7f3e4fde0740 1 client.25766 shutdown 2024-08-31T07:13:39.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.339+0000 7f3e4fde0740 1 -- 172.21.15.32:0/948425762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 msgr2=0x558853403950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:39.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.339+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 0x558853403950 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x558853411c40 tx=0x7f3e34013a50 comp rx=0 tx=0).stop 2024-08-31T07:13:39.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.339+0000 7f3e4fde0740 1 -- 172.21.15.32:0/948425762 shutdown_connections 2024-08-31T07:13:39.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.339+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558852fe2250 0x55885347e740 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.339+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55885304e520 0x558853403fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.340+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55885302d840 0x558853403950 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.340+0000 7f3e4fde0740 1 --2- 172.21.15.32:0/948425762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558853025660 0x5588534084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.340+0000 7f3e4fde0740 1 -- 172.21.15.32:0/948425762 >> 172.21.15.32:0/948425762 conn(0x558853042910 msgr2=0x5588533f59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:39.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.340+0000 7f3e23e00640 0 client.25766 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:39.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.340+0000 7f3e4fde0740 1 -- 172.21.15.32:0/948425762 shutdown_connections 2024-08-31T07:13:39.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.340+0000 7f3e4fde0740 1 -- 172.21.15.32:0/948425762 wait complete. 2024-08-31T07:13:39.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.340+0000 7f3e4fde0740 20 client.25766 trim_cache size 0 max 16384 2024-08-31T07:13:39.342 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:13:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 29, 2024-08-31T07:13:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:13:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 4, 2024-08-31T07:13:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:13:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "7b061927": { 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 4, 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 146.4, 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:13:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:13:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:13:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:39.344 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 -- 172.21.15.32:0/2849733443 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e280b4d20 msgr2=0x7f3e280b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e280b4d20 0x7f3e280b7230 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c02ade0 tx=0x7f3e3c0095c0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 -- 172.21.15.32:0/2849733443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e4410d9c0 msgr2=0x7f3e4410b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e4410d9c0 0x7f3e4410b420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c005e30 tx=0x7f3e3c0317c0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 -- 172.21.15.32:0/2849733443 shutdown_connections 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e280b4d20 0x7f3e280b7230 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e4410d9c0 0x7f3e4410b420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e44104eb0 0x7f3e4410aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 --2- 172.21.15.32:0/2849733443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e44104560 0x7f3e4410a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 -- 172.21.15.32:0/2849733443 >> 172.21.15.32:0/2849733443 conn(0x7f3e440fe060 msgr2=0x7f3e44113d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.341+0000 7f3e43200640 1 -- 172.21.15.32:0/2849733443 shutdown_connections 2024-08-31T07:13:39.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:39.342+0000 7f3e43200640 1 -- 172.21.15.32:0/2849733443 wait complete. 2024-08-31T07:13:40.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:39 smithi032 ceph-mon[17252]: pgmap v395: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:13:40.227 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7b061927', total seconds elapsed: 31.0 2024-08-31T07:13:40.227 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.4 2024-08-31T07:13:40.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:13:40.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:39 smithi084 ceph-mon[22387]: pgmap v395: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:13:40.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:39 smithi073 ceph-mon[21995]: pgmap v395: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:13:40.670 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:41.104 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:40 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:13:41.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:40 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:13:41.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:40 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:13:41.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.555+0000 7f295b200640 1 -- 172.21.15.32:0/190423725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295c075e40 msgr2=0x7f295c10f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.555+0000 7f295b200640 1 --2- 172.21.15.32:0/190423725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295c075e40 0x7f295c10f7d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2948009860 tx=0x7f294802a340 comp rx=0 tx=0).stop 2024-08-31T07:13:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.556+0000 7f295b200640 1 -- 172.21.15.32:0/190423725 shutdown_connections 2024-08-31T07:13:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.556+0000 7f295b200640 1 --2- 172.21.15.32:0/190423725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c10fd10 0x7f295c112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.556+0000 7f295b200640 1 --2- 172.21.15.32:0/190423725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295c075e40 0x7f295c10f7d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.556+0000 7f295b200640 1 --2- 172.21.15.32:0/190423725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c0754f0 0x7f295c075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.556+0000 7f295b200640 1 -- 172.21.15.32:0/190423725 >> 172.21.15.32:0/190423725 conn(0x7f295c0ff1a0 msgr2=0x7f295c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.556+0000 7f295b200640 1 -- 172.21.15.32:0/190423725 shutdown_connections 2024-08-31T07:13:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.556+0000 7f295b200640 1 -- 172.21.15.32:0/190423725 wait complete. 2024-08-31T07:13:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.557+0000 7f295b200640 1 Processor -- start 2024-08-31T07:13:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.558+0000 7f295b200640 1 -- start start 2024-08-31T07:13:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.558+0000 7f295b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295c0754f0 0x7f295c1071a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.558+0000 7f295b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c075e40 0x7f295c103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.558+0000 7f295b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c10fd10 0x7f295c103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.558+0000 7f295b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f295c115840 con 0x7f295c075e40 2024-08-31T07:13:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.558+0000 7f295b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1156c0 con 0x7f295c0754f0 2024-08-31T07:13:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.558+0000 7f295b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1159c0 con 0x7f295c10fd10 2024-08-31T07:13:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.559+0000 7f2959e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295c0754f0 0x7f295c1071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.559+0000 7f2959e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295c0754f0 0x7f295c1071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59192/0 (socket says 172.21.15.32:59192) 2024-08-31T07:13:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.559+0000 7f2959e00640 1 -- 172.21.15.32:0/3277382561 learned_addr learned my addr 172.21.15.32:0/3277382561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.559+0000 7f295a800640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c10fd10 0x7f295c103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:41.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.559+0000 7f2959e00640 1 -- 172.21.15.32:0/3277382561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c10fd10 msgr2=0x7f295c103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:41.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.559+0000 7f2959e00640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c10fd10 0x7f295c103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:41.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.559+0000 7f2959e00640 1 -- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c075e40 msgr2=0x7f295c103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:41.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.560+0000 7f2959400640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c075e40 0x7f295c103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:41.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.560+0000 7f2959e00640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c075e40 0x7f295c103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:41.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.560+0000 7f2959e00640 1 -- 172.21.15.32:0/3277382561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295c1ac7c0 con 0x7f295c0754f0 2024-08-31T07:13:41.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.560+0000 7f2959400640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c075e40 0x7f295c103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:13:41.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.560+0000 7f2959e00640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295c0754f0 0x7f295c1071a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f294c00e9c0 tx=0x7f294c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:41.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.561+0000 7f2952a00640 1 -- 172.21.15.32:0/3277382561 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c00cd80 con 0x7f295c0754f0 2024-08-31T07:13:41.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.561+0000 7f2952a00640 1 -- 172.21.15.32:0/3277382561 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f294c0035f0 con 0x7f295c0754f0 2024-08-31T07:13:41.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.561+0000 7f295b200640 1 -- 172.21.15.32:0/3277382561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295c1ac9f0 con 0x7f295c0754f0 2024-08-31T07:13:41.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.561+0000 7f2952a00640 1 -- 172.21.15.32:0/3277382561 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c010690 con 0x7f295c0754f0 2024-08-31T07:13:41.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.561+0000 7f295b200640 1 -- 172.21.15.32:0/3277382561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f295c1acf80 con 0x7f295c0754f0 2024-08-31T07:13:41.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.562+0000 7f295b200640 1 -- 172.21.15.32:0/3277382561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295c075900 con 0x7f295c0754f0 2024-08-31T07:13:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.567+0000 7f2952a00640 1 -- 172.21.15.32:0/3277382561 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f294c004000 con 0x7f295c0754f0 2024-08-31T07:13:41.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.570+0000 7f2952a00640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29400b4d20 0x7f29400b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.570+0000 7f2952a00640 1 -- 172.21.15.32:0/3277382561 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f294c0e5900 con 0x7f295c0754f0 2024-08-31T07:13:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.570+0000 7f2959400640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29400b4d20 0x7f29400b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.571+0000 7f2952a00640 1 -- 172.21.15.32:0/3277382561 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f294c1161d0 con 0x7f295c0754f0 2024-08-31T07:13:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:41.571+0000 7f2959400640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29400b4d20 0x7f29400b7230 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f295c112240 tx=0x7f2948000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.043+0000 7f295b200640 1 -- 172.21.15.32:0/3277382561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f295c104780 con 0x7f295c0754f0 2024-08-31T07:13:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.044+0000 7f2952a00640 1 -- 172.21.15.32:0/3277382561 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f295c104780 con 0x7f295c0754f0 2024-08-31T07:13:42.045 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:13:42.046 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:13:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:13:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 -- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29400b4d20 msgr2=0x7f29400b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29400b4d20 0x7f29400b7230 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f295c112240 tx=0x7f2948000bc0 comp rx=0 tx=0).stop 2024-08-31T07:13:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 -- 172.21.15.32:0/3277382561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295c0754f0 msgr2=0x7f295c1071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295c0754f0 0x7f295c1071a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f294c00e9c0 tx=0x7f294c00edc0 comp rx=0 tx=0).stop 2024-08-31T07:13:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 -- 172.21.15.32:0/3277382561 shutdown_connections 2024-08-31T07:13:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29400b4d20 0x7f29400b7230 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c10fd10 0x7f295c103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f295c075e40 0x7f295c103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 --2- 172.21.15.32:0/3277382561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295c0754f0 0x7f295c1071a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 -- 172.21.15.32:0/3277382561 >> 172.21.15.32:0/3277382561 conn(0x7f295c0ff1a0 msgr2=0x7f295c1087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 -- 172.21.15.32:0/3277382561 shutdown_connections 2024-08-31T07:13:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:42.048+0000 7f295b200640 1 -- 172.21.15.32:0/3277382561 wait complete. 2024-08-31T07:13:42.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:41 smithi084 ceph-mon[22387]: pgmap v396: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:13:42.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:41 smithi032 ceph-mon[17252]: pgmap v396: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:13:42.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:41 smithi073 ceph-mon[21995]: pgmap v396: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:13:42.873 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0' 2024-08-31T07:13:42.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0 2024-08-31T07:13:43.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:42 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3277382561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:43.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:42 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3277382561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:43.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:42 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3277382561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:43.992 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:43 smithi032 ceph-mon[17252]: pgmap v397: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:13:44.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.080+0000 7ff9cec00640 1 -- 172.21.15.32:0/1568534747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c8105b20 msgr2=0x7ff9c8105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:44.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.080+0000 7ff9cec00640 1 --2- 172.21.15.32:0/1568534747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c8105b20 0x7ff9c8105f30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009900 tx=0x7ff9b802a410 comp rx=0 tx=0).stop 2024-08-31T07:13:44.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.080+0000 7ff9cec00640 1 -- 172.21.15.32:0/1568534747 shutdown_connections 2024-08-31T07:13:44.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.080+0000 7ff9cec00640 1 --2- 172.21.15.32:0/1568534747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9c810bdb0 0x7ff9c81125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.080+0000 7ff9cec00640 1 --2- 172.21.15.32:0/1568534747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c810b460 0x7ff9c810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.080+0000 7ff9cec00640 1 --2- 172.21.15.32:0/1568534747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c8105b20 0x7ff9c8105f30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.080+0000 7ff9cec00640 1 -- 172.21.15.32:0/1568534747 >> 172.21.15.32:0/1568534747 conn(0x7ff9c8101570 msgr2=0x7ff9c81039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:44.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.081+0000 7ff9cec00640 1 -- 172.21.15.32:0/1568534747 shutdown_connections 2024-08-31T07:13:44.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.081+0000 7ff9cec00640 1 -- 172.21.15.32:0/1568534747 wait complete. 2024-08-31T07:13:44.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.082+0000 7ff9cec00640 1 Processor -- start 2024-08-31T07:13:44.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.082+0000 7ff9cec00640 1 -- start start 2024-08-31T07:13:44.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.082+0000 7ff9cec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9c8105b20 0x7ff9c807b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.083+0000 7ff9cec00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c810b460 0x7ff9c80776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.083+0000 7ff9cec00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c810bdb0 0x7ff9c8077de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.083+0000 7ff9cec00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8115ac0 con 0x7ff9c810bdb0 2024-08-31T07:13:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.083+0000 7ff9cec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8115940 con 0x7ff9c8105b20 2024-08-31T07:13:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.083+0000 7ff9cec00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8115c40 con 0x7ff9c810b460 2024-08-31T07:13:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.083+0000 7ff9ccc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c810bdb0 0x7ff9c8077de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.083+0000 7ff9ccc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c810bdb0 0x7ff9c8077de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52196/0 (socket says 172.21.15.32:52196) 2024-08-31T07:13:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.083+0000 7ff9ccc00640 1 -- 172.21.15.32:0/3370220869 learned_addr learned my addr 172.21.15.32:0/3370220869 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.083+0000 7ff9ccc00640 1 -- 172.21.15.32:0/3370220869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c810b460 msgr2=0x7ff9c80776c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:13:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.084+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c810b460 0x7ff9c80776c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.084+0000 7ff9ccc00640 1 -- 172.21.15.32:0/3370220869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9c8105b20 msgr2=0x7ff9c807b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:13:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.084+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9c8105b20 0x7ff9c807b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.084+0000 7ff9ccc00640 1 -- 172.21.15.32:0/3370220869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c81ac9a0 con 0x7ff9c810bdb0 2024-08-31T07:13:44.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.084+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c810bdb0 0x7ff9c8077de0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000eb10 tx=0x7ff9c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:44.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.084+0000 7ff9c4c00640 1 -- 172.21.15.32:0/3370220869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0004b20 con 0x7ff9c810bdb0 2024-08-31T07:13:44.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.085+0000 7ff9c4c00640 1 -- 172.21.15.32:0/3370220869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c0004cc0 con 0x7ff9c810bdb0 2024-08-31T07:13:44.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.085+0000 7ff9c4c00640 1 -- 172.21.15.32:0/3370220869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0010720 con 0x7ff9c810bdb0 2024-08-31T07:13:44.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.085+0000 7ff9cec00640 1 -- 172.21.15.32:0/3370220869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c81acbd0 con 0x7ff9c810bdb0 2024-08-31T07:13:44.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.085+0000 7ff9cec00640 1 -- 172.21.15.32:0/3370220869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c81ad0c0 con 0x7ff9c810bdb0 2024-08-31T07:13:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.089+0000 7ff9c4c00640 1 -- 172.21.15.32:0/3370220869 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff9c0005000 con 0x7ff9c810bdb0 2024-08-31T07:13:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.090+0000 7ff9d57c9740 1 Processor -- start 2024-08-31T07:13:44.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.090+0000 7ff9d57c9740 1 -- start start 2024-08-31T07:13:44.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.091+0000 7ff9c4c00640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff9ac0b4d20 0x7ff9ac0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.091+0000 7ff9c4c00640 1 -- 172.21.15.32:0/3370220869 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff9c00e5c00 con 0x7ff9c810bdb0 2024-08-31T07:13:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.091+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 0x55c20a239110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.091+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 0x55c20a239650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.091+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 0x55c20a239b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.091+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c20a1ae530 con 0x55c209e69980 2024-08-31T07:13:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.091+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c20a19dac0 con 0x55c209e71b60 2024-08-31T07:13:44.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.091+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c20a1a1a70 con 0x55c209e92840 2024-08-31T07:13:44.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.092+0000 7ff9c7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 0x55c20a239650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.092+0000 7ff9ccc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 0x55c20a239b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.092+0000 7ff9ccc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 0x55c20a239b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59212/0 (socket says 172.21.15.32:59212) 2024-08-31T07:13:44.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.092+0000 7ff9ccc00640 1 -- 172.21.15.32:0/73846208 learned_addr learned my addr 172.21.15.32:0/73846208 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:44.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.092+0000 7ff9c7e00640 1 --2- 172.21.15.32:0/73846208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 0x55c20a239110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.092+0000 7ff9ccc00640 1 -- 172.21.15.32:0/73846208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 msgr2=0x55c20a239110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.092+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/73846208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 0x55c20a239110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.092+0000 7ff9ccc00640 1 -- 172.21.15.32:0/73846208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 msgr2=0x55c20a239650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.092+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/73846208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 0x55c20a239650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.092+0000 7ff9ccc00640 1 -- 172.21.15.32:0/73846208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c20a150880 con 0x55c209e71b60 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.093+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/73846208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 0x55c20a239b90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00ac2a0 tx=0x7ff9c00ac210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb9051531157c893 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.093+0000 7ff9bd200640 1 -- 172.21.15.32:0/73846208 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00ac310 con 0x55c209e71b60 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.093+0000 7ff9bd200640 1 -- 172.21.15.32:0/73846208 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c00b0b80 con 0x55c209e71b60 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.093+0000 7ff9bd200640 1 -- 172.21.15.32:0/73846208 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00052f0 con 0x55c209e71b60 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.094+0000 7ff9c7400640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff9ac0b4d20 0x7ff9ac0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.094+0000 7ff9d57c9740 1 -- 172.21.15.32:0/73846208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 msgr2=0x55c20a239b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.094+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/73846208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 0x55c20a239b90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00ac2a0 tx=0x7ff9c00ac210 comp rx=0 tx=0).stop 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.094+0000 7ff9d57c9740 1 -- 172.21.15.32:0/73846208 shutdown_connections 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.094+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/73846208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 0x55c20a239b90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.094+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/73846208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 0x55c20a239650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.094+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/73846208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 0x55c20a239110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.094+0000 7ff9d57c9740 1 -- 172.21.15.32:0/73846208 >> 172.21.15.32:0/73846208 conn(0x55c209e86990 msgr2=0x55c20a237000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.095+0000 7ff9d57c9740 1 -- 172.21.15.32:0/73846208 shutdown_connections 2024-08-31T07:13:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.095+0000 7ff9c7400640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff9ac0b4d20 0x7ff9ac0b7230 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55c20a237d90 tx=0x7ff9b40046c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.095+0000 7ff9d57c9740 1 -- 172.21.15.32:0/73846208 wait complete. 2024-08-31T07:13:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.096+0000 7ff9d57c9740 1 -- start start 2024-08-31T07:13:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.096+0000 7ff9d57c9740 1 Processor -- start 2024-08-31T07:13:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 0x55c20a244d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 0x55c20a245280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 0x55c20a250be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c20a0007a0 con 0x55c209e69980 2024-08-31T07:13:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c20a1247f0 con 0x55c209e71b60 2024-08-31T07:13:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c20a2478f0 con 0x55c209e92840 2024-08-31T07:13:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9d57c9740 10 client.0 _ll_register_callbacks cb 0x55c20a149e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9c7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 0x55c20a244d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9c7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 0x55c20a244d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52224/0 (socket says 172.21.15.32:52224) 2024-08-31T07:13:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9c7e00640 1 -- 172.21.15.32:0/4269784921 learned_addr learned my addr 172.21.15.32:0/4269784921 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 0x55c20a250be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.097+0000 7ff9c7400640 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 0x55c20a245280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.098+0000 7ff9ccc00640 1 -- 172.21.15.32:0/4269784921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 msgr2=0x55c20a245280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.098+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 0x55c20a245280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.098+0000 7ff9ccc00640 1 -- 172.21.15.32:0/4269784921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 msgr2=0x55c20a244d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.098+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 0x55c20a244d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.098+0000 7ff9ccc00640 1 -- 172.21.15.32:0/4269784921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c209f0cb90 con 0x55c209e92840 2024-08-31T07:13:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.098+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 0x55c20a250be0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00ac2a0 tx=0x7ff9c00a8840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ba55ffd5df4667 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.098+0000 7ff9b1600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.099+0000 7ff9b1600640 1 -- 172.21.15.32:0/4269784921 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00d3040 con 0x55c209e92840 2024-08-31T07:13:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.099+0000 7ff9b1600640 1 -- 172.21.15.32:0/4269784921 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff9c0005500 con 0x55c209e92840 2024-08-31T07:13:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.099+0000 7ff9d57c9740 1 -- 172.21.15.32:0/4269784921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c00b0b80 con 0x55c209e92840 2024-08-31T07:13:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.099+0000 7ff9b1600640 1 -- 172.21.15.32:0/4269784921 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9c00ac310 con 0x55c209e92840 2024-08-31T07:13:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.099+0000 7ff9d57c9740 10 client.25786 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.099+0000 7ff9d57c9740 1 -- 172.21.15.32:0/4269784921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c20a253950 con 0x55c209e92840 2024-08-31T07:13:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.100+0000 7ff9b1600640 1 -- 172.21.15.32:0/4269784921 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c00b0b80 con 0x55c209e92840 2024-08-31T07:13:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.100+0000 7ff9b1600640 1 -- 172.21.15.32:0/4269784921 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff9c00e3cc0 con 0x55c209e92840 2024-08-31T07:13:44.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.100+0000 7ff9d57c9740 10 client.25786 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:44.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.100+0000 7ff9d57c9740 10 client.25786 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:13:44.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.100+0000 7ff9d57c9740 20 client.25786 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.100+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c209e511c0 0x55c20a256cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.101+0000 7ff9d57c9740 4 client.25786 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.101+0000 7ff9d57c9740 1 -- 172.21.15.32:0/4269784921 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c20a1b0480 con 0x55c209e511c0 2024-08-31T07:13:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.101+0000 7ff9c7e00640 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c209e511c0 0x55c20a256cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.102+0000 7ff9c7e00640 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c209e511c0 0x55c20a256cb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f60de6523d0a8de server_cookie=b2353dc3c201eaa5 in_seq=0 out_seq=0 2024-08-31T07:13:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.102+0000 7ff9b1600640 10 client.25786 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.103+0000 7ff9b1600640 1 -- 172.21.15.32:0/4269784921 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c20a1b0480 con 0x55c209e511c0 2024-08-31T07:13:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.103+0000 7ff9b1600640 10 client.25786 handle_command_reply: tid=0 2024-08-31T07:13:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.103+0000 7ff9d57c9740 1 client.25786 shutdown 2024-08-31T07:13:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 1 -- 172.21.15.32:0/4269784921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 msgr2=0x55c20a250be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 0x55c20a250be0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00ac2a0 tx=0x7ff9c00a8840 comp rx=0 tx=0).stop 2024-08-31T07:13:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 1 -- 172.21.15.32:0/4269784921 shutdown_connections 2024-08-31T07:13:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c209e511c0 0x55c20a256cb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e92840 0x55c20a250be0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e71b60 0x55c20a245280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/4269784921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e69980 0x55c20a244d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 1 -- 172.21.15.32:0/4269784921 >> 172.21.15.32:0/4269784921 conn(0x55c209e86990 msgr2=0x55c20a2357e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9b1600640 0 client.25786 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 1 -- 172.21.15.32:0/4269784921 shutdown_connections 2024-08-31T07:13:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 1 -- 172.21.15.32:0/4269784921 wait complete. 2024-08-31T07:13:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.104+0000 7ff9d57c9740 20 client.25786 trim_cache size 0 max 16384 2024-08-31T07:13:44.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.141+0000 7ff9d57c9740 1 Processor -- start 2024-08-31T07:13:44.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.141+0000 7ff9d57c9740 1 -- start start 2024-08-31T07:13:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.141+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e69980 0x55c20a250b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.142+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e71b60 0x55c20a23e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.142+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e92840 0x55c20a23e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.142+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8075150 con 0x55c209e92840 2024-08-31T07:13:44.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.142+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81aebf0 con 0x55c209e69980 2024-08-31T07:13:44.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.142+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81ad780 con 0x55c209e71b60 2024-08-31T07:13:44.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.142+0000 7ff9c7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e69980 0x55c20a250b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.142+0000 7ff9c7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e69980 0x55c20a250b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59232/0 (socket says 172.21.15.32:59232) 2024-08-31T07:13:44.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.142+0000 7ff9c7e00640 1 -- 172.21.15.32:0/1388267097 learned_addr learned my addr 172.21.15.32:0/1388267097 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:44.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.142+0000 7ff9c7400640 1 --2- 172.21.15.32:0/1388267097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e71b60 0x55c20a23e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.142+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/1388267097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e92840 0x55c20a23e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.143+0000 7ff9ccc00640 1 -- 172.21.15.32:0/1388267097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e71b60 msgr2=0x55c20a23e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:44.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.143+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/1388267097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e71b60 0x55c20a23e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.143+0000 7ff9ccc00640 1 -- 172.21.15.32:0/1388267097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e69980 msgr2=0x55c20a250b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:44.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.143+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/1388267097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e69980 0x55c20a250b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.143+0000 7ff9ccc00640 1 -- 172.21.15.32:0/1388267097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c20a2455f0 con 0x55c209e92840 2024-08-31T07:13:44.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.143+0000 7ff9ccc00640 1 --2- 172.21.15.32:0/1388267097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e92840 0x55c20a23e760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00058e0 tx=0x7ff9c00d3170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518580d4e18596ca server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:44.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.144+0000 7ff9b3e00640 1 -- 172.21.15.32:0/1388267097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00ab390 con 0x55c209e92840 2024-08-31T07:13:44.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.144+0000 7ff9b3e00640 1 -- 172.21.15.32:0/1388267097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c00b0b80 con 0x55c209e92840 2024-08-31T07:13:44.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.144+0000 7ff9b3e00640 1 -- 172.21.15.32:0/1388267097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00ab390 con 0x55c209e92840 2024-08-31T07:13:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.145+0000 7ff9d57c9740 1 -- 172.21.15.32:0/1388267097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e92840 msgr2=0x55c20a23e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.145+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/1388267097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e92840 0x55c20a23e760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00058e0 tx=0x7ff9c00d3170 comp rx=0 tx=0).stop 2024-08-31T07:13:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.146+0000 7ff9d57c9740 1 -- 172.21.15.32:0/1388267097 shutdown_connections 2024-08-31T07:13:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.146+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/1388267097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e92840 0x55c20a23e760 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.146+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/1388267097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e71b60 0x55c20a23e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.146+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/1388267097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e69980 0x55c20a250b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.146+0000 7ff9d57c9740 1 -- 172.21.15.32:0/1388267097 >> 172.21.15.32:0/1388267097 conn(0x55c209e86990 msgr2=0x55c20a2357e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.146+0000 7ff9d57c9740 1 -- 172.21.15.32:0/1388267097 shutdown_connections 2024-08-31T07:13:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.146+0000 7ff9d57c9740 1 -- 172.21.15.32:0/1388267097 wait complete. 2024-08-31T07:13:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.146+0000 7ff9d57c9740 1 -- start start 2024-08-31T07:13:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.147+0000 7ff9d57c9740 1 Processor -- start 2024-08-31T07:13:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e69980 0x55c20a257cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e71b60 0x55c20a2581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9d57c9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e92840 0x55c20a258730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81ae520 con 0x55c209e71b60 2024-08-31T07:13:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81ade50 con 0x55c209e92840 2024-08-31T07:13:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9d57c9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8073130 con 0x55c209e69980 2024-08-31T07:13:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9d57c9740 10 client.0 _ll_register_callbacks cb 0x7ff9c00abc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9ccc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e92840 0x55c20a258730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9c7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e71b60 0x55c20a2581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9ccc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e92840 0x55c20a258730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59242/0 (socket says 172.21.15.32:59242) 2024-08-31T07:13:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9c7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e71b60 0x55c20a2581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52236/0 (socket says 172.21.15.32:52236) 2024-08-31T07:13:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.148+0000 7ff9ccc00640 1 -- 172.21.15.32:0/3361660655 learned_addr learned my addr 172.21.15.32:0/3361660655 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.149+0000 7ff9c7e00640 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e69980 0x55c20a257cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.149+0000 7ff9c7400640 1 -- 172.21.15.32:0/3361660655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e69980 msgr2=0x55c20a257cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.149+0000 7ff9c7400640 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e69980 0x55c20a257cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.149+0000 7ff9c7400640 1 -- 172.21.15.32:0/3361660655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e92840 msgr2=0x55c20a258730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.149+0000 7ff9c7400640 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e92840 0x55c20a258730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.149+0000 7ff9c7400640 1 -- 172.21.15.32:0/3361660655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c20a2c0430 con 0x55c209e71b60 2024-08-31T07:13:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.150+0000 7ff9c7400640 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e71b60 0x55c20a2581f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40098e0 tx=0x7ff9b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d8d364d0dbba9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.150+0000 7ff9b1600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:13:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.151+0000 7ff9b1600640 1 -- 172.21.15.32:0/3361660655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b402c060 con 0x55c209e71b60 2024-08-31T07:13:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.151+0000 7ff9b1600640 1 -- 172.21.15.32:0/3361660655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff9b4010c10 con 0x55c209e71b60 2024-08-31T07:13:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.151+0000 7ff9d57c9740 1 -- 172.21.15.32:0/3361660655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c00b0b80 con 0x55c209e71b60 2024-08-31T07:13:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.151+0000 7ff9b1600640 1 -- 172.21.15.32:0/3361660655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9b4010e10 con 0x55c209e71b60 2024-08-31T07:13:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.151+0000 7ff9d57c9740 10 client.15774 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.151+0000 7ff9d57c9740 1 -- 172.21.15.32:0/3361660655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c20a2c1680 con 0x55c209e71b60 2024-08-31T07:13:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.152+0000 7ff9b1600640 1 -- 172.21.15.32:0/3361660655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b401b900 con 0x55c209e71b60 2024-08-31T07:13:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.152+0000 7ff9b1600640 1 -- 172.21.15.32:0/3361660655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff9b4026070 con 0x55c209e71b60 2024-08-31T07:13:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.152+0000 7ff9d57c9740 10 client.15774 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.152+0000 7ff9d57c9740 10 client.15774 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:13:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.152+0000 7ff9d57c9740 20 client.15774 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.152+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c209e26570 0x55c20a2c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.152+0000 7ff9d57c9740 4 client.15774 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}] 2024-08-31T07:13:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.152+0000 7ff9d57c9740 1 -- 172.21.15.32:0/3361660655 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}) -- 0x7ff9c00b0b80 con 0x55c209e26570 2024-08-31T07:13:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.153+0000 7ff9c7e00640 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c209e26570 0x55c20a2c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.154+0000 7ff9c7e00640 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c209e26570 0x55c20a2c3eb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b79e209affb87e server_cookie=454365a0827dcf8a in_seq=0 out_seq=0 2024-08-31T07:13:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.154+0000 7ff9b1600640 10 client.15774 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.159+0000 7ff9b1600640 1 -- 172.21.15.32:0/3361660655 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7ff9c00b0b80 con 0x55c209e26570 2024-08-31T07:13:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.159+0000 7ff9b1600640 10 client.15774 handle_command_reply: tid=0 2024-08-31T07:13:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.159+0000 7ff9d57c9740 1 client.15774 shutdown 2024-08-31T07:13:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 1 -- 172.21.15.32:0/3361660655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e71b60 msgr2=0x55c20a2581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e71b60 0x55c20a2581f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40098e0 tx=0x7ff9b4004730 comp rx=0 tx=0).stop 2024-08-31T07:13:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 1 -- 172.21.15.32:0/3361660655 shutdown_connections 2024-08-31T07:13:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c209e26570 0x55c20a2c3eb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c209e92840 0x55c20a258730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c209e71b60 0x55c20a2581f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 1 --2- 172.21.15.32:0/3361660655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c209e69980 0x55c20a257cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 1 -- 172.21.15.32:0/3361660655 >> 172.21.15.32:0/3361660655 conn(0x55c209e86990 msgr2=0x55c20a23e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9b1600640 0 client.15774 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 1 -- 172.21.15.32:0/3361660655 shutdown_connections 2024-08-31T07:13:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 1 -- 172.21.15.32:0/3361660655 wait complete. 2024-08-31T07:13:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.160+0000 7ff9d57c9740 20 client.15774 trim_cache size 0 max 16384 2024-08-31T07:13:44.162 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:13:44.162 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 29, 2024-08-31T07:13:44.162 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:13:44.162 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 6, 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "9cf8d043": { 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 6, 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 146.4, 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:13:44.163 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:13:44.164 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:13:44.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:44.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:44.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:44.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:44.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:44.164 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:13:44.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.161+0000 7ff9cec00640 1 -- 172.21.15.32:0/3370220869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff9ac0b4d20 msgr2=0x7ff9ac0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:44.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.161+0000 7ff9cec00640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff9ac0b4d20 0x7ff9ac0b7230 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55c20a237d90 tx=0x7ff9b40046c0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.161+0000 7ff9cec00640 1 -- 172.21.15.32:0/3370220869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c810bdb0 msgr2=0x7ff9c8077de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.161+0000 7ff9cec00640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c810bdb0 0x7ff9c8077de0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000eb10 tx=0x7ff9c000c2d0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.162+0000 7ff9cec00640 1 -- 172.21.15.32:0/3370220869 shutdown_connections 2024-08-31T07:13:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.162+0000 7ff9cec00640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff9ac0b4d20 0x7ff9ac0b7230 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.162+0000 7ff9cec00640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9c810bdb0 0x7ff9c8077de0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.162+0000 7ff9cec00640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c810b460 0x7ff9c80776c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.162+0000 7ff9cec00640 1 --2- 172.21.15.32:0/3370220869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9c8105b20 0x7ff9c807b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.162+0000 7ff9cec00640 1 -- 172.21.15.32:0/3370220869 >> 172.21.15.32:0/3370220869 conn(0x7ff9c8101570 msgr2=0x7ff9c8103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.162+0000 7ff9cec00640 1 -- 172.21.15.32:0/3370220869 shutdown_connections 2024-08-31T07:13:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:44.162+0000 7ff9cec00640 1 -- 172.21.15.32:0/3370220869 wait complete. 2024-08-31T07:13:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:43 smithi073 ceph-mon[21995]: pgmap v397: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:13:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:43 smithi084 ceph-mon[22387]: pgmap v397: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-31T07:13:45.021 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9cf8d043', quiesce took 4.8 seconds. Will release after: 21.599999999999998 2024-08-31T07:13:46.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:45 smithi032 ceph-mon[17252]: pgmap v398: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:13:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:45 smithi073 ceph-mon[21995]: pgmap v398: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:13:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:45 smithi084 ceph-mon[22387]: pgmap v398: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-31T07:13:48.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:47 smithi032 ceph-mon[17252]: pgmap v399: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:48.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:47 smithi073 ceph-mon[21995]: pgmap v399: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:48.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:47 smithi084 ceph-mon[22387]: pgmap v399: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:50.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:13:50.292 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:50 smithi084 ceph-mon[22387]: pgmap v400: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-08-31T07:13:50.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:50 smithi032 ceph-mon[17252]: pgmap v400: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-08-31T07:13:50.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:50 smithi073 ceph-mon[21995]: pgmap v400: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-08-31T07:13:50.485 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.234+0000 7f06d2a00640 1 -- 172.21.15.32:0/1244369773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06cc114020 msgr2=0x7f06cc1164f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.234+0000 7f06d2a00640 1 --2- 172.21.15.32:0/1244369773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06cc114020 0x7f06cc1164f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009ca0 tx=0x7f06c402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.234+0000 7f06d2a00640 1 -- 172.21.15.32:0/1244369773 shutdown_connections 2024-08-31T07:13:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.234+0000 7f06d2a00640 1 --2- 172.21.15.32:0/1244369773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06cc114020 0x7f06cc1164f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.234+0000 7f06d2a00640 1 --2- 172.21.15.32:0/1244369773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06cc075b70 0x7f06cc075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.234+0000 7f06d2a00640 1 --2- 172.21.15.32:0/1244369773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06cc077120 0x7f06cc075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.234+0000 7f06d2a00640 1 -- 172.21.15.32:0/1244369773 >> 172.21.15.32:0/1244369773 conn(0x7f06cc0ff1e0 msgr2=0x7f06cc101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.234+0000 7f06d2a00640 1 -- 172.21.15.32:0/1244369773 shutdown_connections 2024-08-31T07:13:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.234+0000 7f06d2a00640 1 -- 172.21.15.32:0/1244369773 wait complete. 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.235+0000 7f06d2a00640 1 Processor -- start 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.235+0000 7f06d2a00640 1 -- start start 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06cc075b70 0x7f06cc1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06cc077120 0x7f06cc1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06cc114020 0x7f06cc1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d2a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc119c60 con 0x7f06cc077120 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d2a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc119ae0 con 0x7f06cc114020 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d2a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc119de0 con 0x7f06cc075b70 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06cc077120 0x7f06cc1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d1600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06cc075b70 0x7f06cc1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06cc077120 0x7f06cc1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60636/0 (socket says 172.21.15.32:60636) 2024-08-31T07:13:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d0c00640 1 -- 172.21.15.32:0/3653668844 learned_addr learned my addr 172.21.15.32:0/3653668844 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.236+0000 7f06d2000640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06cc114020 0x7f06cc1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.237+0000 7f06d1600640 1 -- 172.21.15.32:0/3653668844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06cc114020 msgr2=0x7f06cc1a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.237+0000 7f06d1600640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06cc114020 0x7f06cc1a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.237+0000 7f06d1600640 1 -- 172.21.15.32:0/3653668844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06cc077120 msgr2=0x7f06cc1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.237+0000 7f06d1600640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06cc077120 0x7f06cc1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.237+0000 7f06d1600640 1 -- 172.21.15.32:0/3653668844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc1a6560 con 0x7f06cc075b70 2024-08-31T07:13:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.237+0000 7f06d0c00640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06cc077120 0x7f06cc1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.237+0000 7f06d1600640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06cc075b70 0x7f06cc1a05f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f06c000e9f0 tx=0x7f06c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.237+0000 7f06ca000640 1 -- 172.21.15.32:0/3653668844 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c000ccc0 con 0x7f06cc075b70 2024-08-31T07:13:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.237+0000 7f06ca000640 1 -- 172.21.15.32:0/3653668844 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c00108e0 con 0x7f06cc075b70 2024-08-31T07:13:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.237+0000 7f06ca000640 1 -- 172.21.15.32:0/3653668844 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0018880 con 0x7f06cc075b70 2024-08-31T07:13:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.238+0000 7f06d2a00640 1 -- 172.21.15.32:0/3653668844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc1a6850 con 0x7f06cc075b70 2024-08-31T07:13:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.238+0000 7f06d2a00640 1 -- 172.21.15.32:0/3653668844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc1acfb0 con 0x7f06cc075b70 2024-08-31T07:13:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.241+0000 7f06dae21740 1 Processor -- start 2024-08-31T07:13:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.241+0000 7f06dae21740 1 -- start start 2024-08-31T07:13:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.241+0000 7f06dae21740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f1eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.241+0000 7f06dae21740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f1f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.241+0000 7f06dae21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f1f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.241+0000 7f06dae21740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564ed0e95770 con 0x564ed0b79070 2024-08-31T07:13:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.241+0000 7f06dae21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564ed0e84d00 con 0x564ed0b58390 2024-08-31T07:13:51.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.241+0000 7f06dae21740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564ed0e88cb0 con 0x564ed0b501b0 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.242+0000 7f06d2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f1eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.242+0000 7f06d2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f1eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60642/0 (socket says 172.21.15.32:60642) 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.242+0000 7f06d0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f1f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.242+0000 7f06d2000640 1 -- 172.21.15.32:0/1675450638 learned_addr learned my addr 172.21.15.32:0/1675450638 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.242+0000 7f06d0c00640 1 -- 172.21.15.32:0/1675450638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 msgr2=0x564ed0f1f060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.242+0000 7f06d0c00640 1 --2- 172.21.15.32:0/1675450638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f1f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.242+0000 7f06d0c00640 1 -- 172.21.15.32:0/1675450638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 msgr2=0x564ed0f1eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.242+0000 7f06d0c00640 1 --2- 172.21.15.32:0/1675450638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f1eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.242+0000 7f06d0c00640 1 -- 172.21.15.32:0/1675450638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed0be4650 con 0x564ed0b58390 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.243+0000 7f06d0c00640 1 --2- 172.21.15.32:0/1675450638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f1f5a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06cc115f40 tx=0x7f06b80119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fdbf307c33de5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.243+0000 7f06be600640 1 -- 172.21.15.32:0/1675450638 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b801e070 con 0x564ed0b58390 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.243+0000 7f06be600640 1 -- 172.21.15.32:0/1675450638 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b80035f0 con 0x564ed0b58390 2024-08-31T07:13:51.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.243+0000 7f06be600640 1 -- 172.21.15.32:0/1675450638 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b800f7c0 con 0x564ed0b58390 2024-08-31T07:13:51.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.243+0000 7f06ca000640 1 -- 172.21.15.32:0/3653668844 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f06c0010430 con 0x7f06cc075b70 2024-08-31T07:13:51.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.244+0000 7f06dae21740 1 -- 172.21.15.32:0/1675450638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 msgr2=0x564ed0f1f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:51.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.244+0000 7f06dae21740 1 --2- 172.21.15.32:0/1675450638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f1f5a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06cc115f40 tx=0x7f06b80119a0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.244+0000 7f06dae21740 1 -- 172.21.15.32:0/1675450638 shutdown_connections 2024-08-31T07:13:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.244+0000 7f06dae21740 1 --2- 172.21.15.32:0/1675450638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f1f5a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.244+0000 7f06dae21740 1 --2- 172.21.15.32:0/1675450638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f1f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.244+0000 7f06dae21740 1 --2- 172.21.15.32:0/1675450638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f1eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.244+0000 7f06dae21740 1 -- 172.21.15.32:0/1675450638 >> 172.21.15.32:0/1675450638 conn(0x564ed0b6d460 msgr2=0x564ed0f1ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.244+0000 7f06dae21740 1 -- 172.21.15.32:0/1675450638 shutdown_connections 2024-08-31T07:13:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.244+0000 7f06dae21740 1 -- 172.21.15.32:0/1675450638 wait complete. 2024-08-31T07:13:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.245+0000 7f06dae21740 1 -- start start 2024-08-31T07:13:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.245+0000 7f06ca000640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06b40b4bb0 0x7f06b40b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.245+0000 7f06dae21740 1 Processor -- start 2024-08-31T07:13:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.245+0000 7f06ca000640 1 -- 172.21.15.32:0/3653668844 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f06c00e5830 con 0x7f06cc075b70 2024-08-31T07:13:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.245+0000 7f06d2000640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06b40b4bb0 0x7f06b40b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06dae21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b501b0 0x564ed0f2a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06dae21740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b58390 0x564ed0f2acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06dae21740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b79070 0x564ed0f365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06dae21740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564ed0ce7700 con 0x564ed0b58390 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06dae21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564ed0e0bcb0 con 0x564ed0b501b0 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06dae21740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564ed0f2d300 con 0x564ed0b79070 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06dae21740 10 client.0 _ll_register_callbacks cb 0x564ed0e31430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06d0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b58390 0x564ed0f2acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06d2000640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06b40b4bb0 0x7f06b40b70c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f06c40094e0 tx=0x7f06c4005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06d0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b58390 0x564ed0f2acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60658/0 (socket says 172.21.15.32:60658) 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.246+0000 7f06d0c00640 1 -- 172.21.15.32:0/1843886859 learned_addr learned my addr 172.21.15.32:0/1843886859 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.247+0000 7f06d1600640 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b501b0 0x564ed0f2a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.247+0000 7f06d2000640 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b79070 0x564ed0f365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.247+0000 7f06d1600640 1 -- 172.21.15.32:0/1843886859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b79070 msgr2=0x564ed0f365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.247+0000 7f06d1600640 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b79070 0x564ed0f365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.247+0000 7f06d1600640 1 -- 172.21.15.32:0/1843886859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b58390 msgr2=0x564ed0f2acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.247+0000 7f06d1600640 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b58390 0x564ed0f2acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.247+0000 7f06d1600640 1 -- 172.21.15.32:0/1843886859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed0e4c870 con 0x564ed0b501b0 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.247+0000 7f06d2000640 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b79070 0x564ed0f365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.247+0000 7f06d1600640 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b501b0 0x564ed0f2a750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f06c00e2fa0 tx=0x7f06c00abbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b899208eb7ef60e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.247+0000 7f06b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.248+0000 7f06b2a00640 1 -- 172.21.15.32:0/1843886859 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00b5260 con 0x564ed0b501b0 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.248+0000 7f06b2a00640 1 -- 172.21.15.32:0/1843886859 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f06c00a8280 con 0x564ed0b501b0 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.248+0000 7f06dae21740 1 -- 172.21.15.32:0/1843886859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b80035f0 con 0x564ed0b501b0 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.248+0000 7f06b2a00640 1 -- 172.21.15.32:0/1843886859 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06c00b03c0 con 0x564ed0b501b0 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06dae21740 10 client.25799 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06dae21740 1 -- 172.21.15.32:0/1843886859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ed0f393c0 con 0x564ed0b501b0 2024-08-31T07:13:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06b2a00640 1 -- 172.21.15.32:0/1843886859 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b80035f0 con 0x564ed0b501b0 2024-08-31T07:13:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06b2a00640 1 -- 172.21.15.32:0/1843886859 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f06c00a7ad0 con 0x564ed0b501b0 2024-08-31T07:13:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06dae21740 10 client.25799 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06dae21740 10 client.25799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:13:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06dae21740 20 client.25799 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06dae21740 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ed0b379f0 0x564ed0f3c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06dae21740 4 client.25799 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06dae21740 1 -- 172.21.15.32:0/1843886859 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ed0e976c0 con 0x564ed0b379f0 2024-08-31T07:13:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.249+0000 7f06d0c00640 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ed0b379f0 0x564ed0f3c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.250+0000 7f06d0c00640 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ed0b379f0 0x564ed0f3c830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b19f1d59c40c750 server_cookie=89e9c4d77d4f85cd in_seq=0 out_seq=0 2024-08-31T07:13:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.250+0000 7f06b2a00640 10 client.25799 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06b2a00640 1 -- 172.21.15.32:0/1843886859 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ed0e976c0 con 0x564ed0b379f0 2024-08-31T07:13:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06b2a00640 10 client.25799 handle_command_reply: tid=0 2024-08-31T07:13:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06dae21740 1 client.25799 shutdown 2024-08-31T07:13:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06dae21740 1 -- 172.21.15.32:0/1843886859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b501b0 msgr2=0x564ed0f2a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06dae21740 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b501b0 0x564ed0f2a750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f06c00e2fa0 tx=0x7f06c00abbc0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06dae21740 1 -- 172.21.15.32:0/1843886859 shutdown_connections 2024-08-31T07:13:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06dae21740 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ed0b379f0 0x564ed0f3c830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06dae21740 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b79070 0x564ed0f365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06dae21740 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b58390 0x564ed0f2acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06dae21740 1 --2- 172.21.15.32:0/1843886859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b501b0 0x564ed0f2a750 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06b2a00640 0 client.25799 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.251+0000 7f06dae21740 1 -- 172.21.15.32:0/1843886859 >> 172.21.15.32:0/1843886859 conn(0x564ed0b6d460 msgr2=0x564ed0f21d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.252+0000 7f06dae21740 1 -- 172.21.15.32:0/1843886859 shutdown_connections 2024-08-31T07:13:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.252+0000 7f06dae21740 1 -- 172.21.15.32:0/1843886859 wait complete. 2024-08-31T07:13:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.252+0000 7f06dae21740 20 client.25799 trim_cache size 0 max 16384 2024-08-31T07:13:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.270+0000 7f06dae21740 1 Processor -- start 2024-08-31T07:13:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06dae21740 1 -- start start 2024-08-31T07:13:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06dae21740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f25d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06dae21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06dae21740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06dae21740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1b5800 con 0x564ed0b79070 2024-08-31T07:13:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06dae21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1b7340 con 0x564ed0b58390 2024-08-31T07:13:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06dae21740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1b5ed0 con 0x564ed0b501b0 2024-08-31T07:13:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06d1600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f25d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06d2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06d0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06d1600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f25d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:49970/0 (socket says 172.21.15.32:49970) 2024-08-31T07:13:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.271+0000 7f06d1600640 1 -- 172.21.15.32:0/1925531652 learned_addr learned my addr 172.21.15.32:0/1925531652 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.272+0000 7f06d1600640 1 -- 172.21.15.32:0/1925531652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 msgr2=0x564ed0f364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.272+0000 7f06d1600640 1 --2- 172.21.15.32:0/1925531652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f364f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.272+0000 7f06d1600640 1 -- 172.21.15.32:0/1925531652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 msgr2=0x564ed0f200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.272+0000 7f06d1600640 1 --2- 172.21.15.32:0/1925531652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.272+0000 7f06d1600640 1 -- 172.21.15.32:0/1925531652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed0f2b810 con 0x564ed0b501b0 2024-08-31T07:13:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.272+0000 7f06d0c00640 1 --2- 172.21.15.32:0/1925531652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.272+0000 7f06d1600640 1 --2- 172.21.15.32:0/1925531652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f25d00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f06c00abf80 tx=0x7f06c00b0930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ce3af18bc6b7c0c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.273+0000 7f06bdc00640 1 -- 172.21.15.32:0/1925531652 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00b0560 con 0x564ed0b501b0 2024-08-31T07:13:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.273+0000 7f06bdc00640 1 -- 172.21.15.32:0/1925531652 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c00aafb0 con 0x564ed0b501b0 2024-08-31T07:13:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.273+0000 7f06bdc00640 1 -- 172.21.15.32:0/1925531652 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00a7960 con 0x564ed0b501b0 2024-08-31T07:13:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.274+0000 7f06dae21740 1 -- 172.21.15.32:0/1925531652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 msgr2=0x564ed0f25d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.274+0000 7f06dae21740 1 --2- 172.21.15.32:0/1925531652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f25d00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f06c00abf80 tx=0x7f06c00b0930 comp rx=0 tx=0).stop 2024-08-31T07:13:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.274+0000 7f06dae21740 1 -- 172.21.15.32:0/1925531652 shutdown_connections 2024-08-31T07:13:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.274+0000 7f06dae21740 1 --2- 172.21.15.32:0/1925531652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.274+0000 7f06dae21740 1 --2- 172.21.15.32:0/1925531652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.274+0000 7f06dae21740 1 --2- 172.21.15.32:0/1925531652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f25d00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.274+0000 7f06dae21740 1 -- 172.21.15.32:0/1925531652 >> 172.21.15.32:0/1925531652 conn(0x564ed0b6d460 msgr2=0x564ed0f1f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.274+0000 7f06dae21740 1 -- 172.21.15.32:0/1925531652 shutdown_connections 2024-08-31T07:13:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.274+0000 7f06dae21740 1 -- 172.21.15.32:0/1925531652 wait complete. 2024-08-31T07:13:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.275+0000 7f06dae21740 1 -- start start 2024-08-31T07:13:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.275+0000 7f06dae21740 1 Processor -- start 2024-08-31T07:13:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.275+0000 7f06dae21740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f2e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.275+0000 7f06dae21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f2ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.275+0000 7f06dae21740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f2f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.275+0000 7f06dae21740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1b6c70 con 0x564ed0b79070 2024-08-31T07:13:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.275+0000 7f06dae21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1b65a0 con 0x564ed0b58390 2024-08-31T07:13:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.275+0000 7f06dae21740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc1b37e0 con 0x564ed0b501b0 2024-08-31T07:13:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.275+0000 7f06dae21740 10 client.0 _ll_register_callbacks cb 0x7f06c0010720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.276+0000 7f06d0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f2ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.276+0000 7f06d1600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f2e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.276+0000 7f06d2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f2f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.276+0000 7f06d0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f2ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59564/0 (socket says 172.21.15.32:59564) 2024-08-31T07:13:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.276+0000 7f06d0c00640 1 -- 172.21.15.32:0/3854698694 learned_addr learned my addr 172.21.15.32:0/3854698694 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.276+0000 7f06d1600640 1 -- 172.21.15.32:0/3854698694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 msgr2=0x564ed0f2ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.276+0000 7f06d1600640 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f2ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.276+0000 7f06d1600640 1 -- 172.21.15.32:0/3854698694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 msgr2=0x564ed0f2f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.276+0000 7f06d1600640 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f2f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.276+0000 7f06d1600640 1 -- 172.21.15.32:0/3854698694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed0fa63f0 con 0x564ed0b501b0 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.277+0000 7f06d1600640 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f2e6f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f06c0000c00 tx=0x7f06c000cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6724a19d8f4c022 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.277+0000 7f06b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.277+0000 7f06b2a00640 1 -- 172.21.15.32:0/3854698694 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00b5260 con 0x564ed0b501b0 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.277+0000 7f06b2a00640 1 -- 172.21.15.32:0/3854698694 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f06c00ab460 con 0x564ed0b501b0 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.277+0000 7f06dae21740 1 -- 172.21.15.32:0/3854698694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c00aafb0 con 0x564ed0b501b0 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.277+0000 7f06b2a00640 1 -- 172.21.15.32:0/3854698694 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06c00b0560 con 0x564ed0b501b0 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.277+0000 7f06dae21740 10 client.25807 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.277+0000 7f06dae21740 1 -- 172.21.15.32:0/3854698694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ed0fa7640 con 0x564ed0b501b0 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.278+0000 7f06b2a00640 1 -- 172.21.15.32:0/3854698694 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c00aafb0 con 0x564ed0b501b0 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.278+0000 7f06b2a00640 1 -- 172.21.15.32:0/3854698694 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f06c0093260 con 0x564ed0b501b0 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.278+0000 7f06dae21740 10 client.25807 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.278+0000 7f06dae21740 10 client.25807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:13:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.278+0000 7f06dae21740 20 client.25807 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.278+0000 7f06dae21740 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ed0b0cda0 0x564ed0fa9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.278+0000 7f06dae21740 4 client.25807 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:13:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.278+0000 7f06dae21740 1 -- 172.21.15.32:0/3854698694 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ed0e93160 con 0x564ed0b0cda0 2024-08-31T07:13:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.279+0000 7f06d0c00640 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ed0b0cda0 0x564ed0fa9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.279+0000 7f06d0c00640 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ed0b0cda0 0x564ed0fa9b30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2052aab3a53fd6d7 server_cookie=b82d3bb5e2a42eb8 in_seq=0 out_seq=0 2024-08-31T07:13:51.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.279+0000 7f06b2a00640 10 client.25807 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.279+0000 7f06b2a00640 1 -- 172.21.15.32:0/3854698694 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x564ed0e93160 con 0x564ed0b0cda0 2024-08-31T07:13:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.279+0000 7f06b2a00640 10 client.25807 handle_command_reply: tid=0 2024-08-31T07:13:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.279+0000 7f06dae21740 1 client.25807 shutdown 2024-08-31T07:13:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 1 -- 172.21.15.32:0/3854698694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 msgr2=0x564ed0f2e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f2e6f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f06c0000c00 tx=0x7f06c000cef0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 1 -- 172.21.15.32:0/3854698694 shutdown_connections 2024-08-31T07:13:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ed0b0cda0 0x564ed0fa9b30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ed0b79070 0x564ed0f2f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ed0b58390 0x564ed0f2ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06b2a00640 0 client.25807 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:13:51.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 1 --2- 172.21.15.32:0/3854698694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ed0b501b0 0x564ed0f2e6f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 1 -- 172.21.15.32:0/3854698694 >> 172.21.15.32:0/3854698694 conn(0x564ed0b6d460 msgr2=0x564ed0f20350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:51.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 1 -- 172.21.15.32:0/3854698694 shutdown_connections 2024-08-31T07:13:51.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 1 -- 172.21.15.32:0/3854698694 wait complete. 2024-08-31T07:13:51.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.280+0000 7f06dae21740 20 client.25807 trim_cache size 0 max 16384 2024-08-31T07:13:51.283 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:13:51.283 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:13:51.283 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:13:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:13:51.285 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:13:51.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:51.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:13:51.285 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:13:51.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 -- 172.21.15.32:0/3653668844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06b40b4bb0 msgr2=0x7f06b40b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06b40b4bb0 0x7f06b40b70c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f06c40094e0 tx=0x7f06c4005fd0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 -- 172.21.15.32:0/3653668844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06cc075b70 msgr2=0x7f06cc1a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06cc075b70 0x7f06cc1a05f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f06c000e9f0 tx=0x7f06c000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 -- 172.21.15.32:0/3653668844 shutdown_connections 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06b40b4bb0 0x7f06b40b70c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06cc114020 0x7f06cc1a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06cc077120 0x7f06cc1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 --2- 172.21.15.32:0/3653668844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06cc075b70 0x7f06cc1a05f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 -- 172.21.15.32:0/3653668844 >> 172.21.15.32:0/3653668844 conn(0x7f06cc0ff1e0 msgr2=0x7f06cc10ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 -- 172.21.15.32:0/3653668844 shutdown_connections 2024-08-31T07:13:51.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:51.281+0000 7f06d2a00640 1 -- 172.21.15.32:0/3653668844 wait complete. 2024-08-31T07:13:52.245 INFO:tasks.workunit.client.0.smithi032.stderr:Updating files: 57% (7244/12697) Updating files: 58% (7365/12697) Updating files: 59% (7492/12697) Updating files: 60% (7619/12697) Updating files: 61% (7746/12697) Updating files: 62% (7873/12697) Updating files: 63% (8000/12697) Updating files: 64% (8127/12697) Updating files: 65% (8254/12697) Updating files: 66% (8381/12697) Updating files: 67% (8507/12697) Updating files: 68% (8634/12697) Updating files: 69% (8761/12697) Updating files: 70% (8888/12697) Updating files: 71% (9015/12697) Updating files: 72% (9142/12697) Updating files: 73% (9269/12697) Updating files: 74% (9396/12697) Updating files: 75% (9523/12697) Updating files: 76% (9650/12697) Updating files: 77% (9777/12697) Updating files: 78% (9904/12697) Updating files: 79% (10031/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 92% (11748/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-08-31T07:13:52.257 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:52 smithi032 ceph-mon[17252]: pgmap v401: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:52.292 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:13:52.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:13:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:52 smithi073 ceph-mon[21995]: pgmap v401: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:52 smithi084 ceph-mon[22387]: pgmap v401: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:52.917 INFO:tasks.workunit.client.0.smithi032.stderr:Note: switching to 'd19e6118f7015f0047264006cb0c873528eb3ae6'. 2024-08-31T07:13:52.917 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-08-31T07:13:52.917 INFO:tasks.workunit.client.0.smithi032.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-31T07:13:52.917 INFO:tasks.workunit.client.0.smithi032.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-31T07:13:52.917 INFO:tasks.workunit.client.0.smithi032.stderr:state without impacting any branches by switching back to a branch. 2024-08-31T07:13:52.917 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-08-31T07:13:52.917 INFO:tasks.workunit.client.0.smithi032.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr: git switch -c 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr:Or undo this operation with: 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr: git switch - 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-08-31T07:13:52.918 INFO:tasks.workunit.client.0.smithi032.stderr:HEAD is now at d19e6118f70 Merge PR #44359 into wip-mchangir-testing-20240828.085847-main-debug 2024-08-31T07:13:52.933 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-31T07:13:53.045 INFO:tasks.workunit.client.0.smithi032.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-31T07:13:53.051 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-31T07:13:53.051 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-31T07:13:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.488+0000 7f43c4800640 1 -- 172.21.15.32:0/2425137237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc10b460 msgr2=0x7f43bc10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.488+0000 7f43c4800640 1 --2- 172.21.15.32:0/2425137237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc10b460 0x7f43bc10b870 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009900 tx=0x7f43ac02a410 comp rx=0 tx=0).stop 2024-08-31T07:13:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.488+0000 7f43c4800640 1 -- 172.21.15.32:0/2425137237 shutdown_connections 2024-08-31T07:13:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.488+0000 7f43c4800640 1 --2- 172.21.15.32:0/2425137237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc10bdb0 0x7f43bc1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.488+0000 7f43c4800640 1 --2- 172.21.15.32:0/2425137237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc10b460 0x7f43bc10b870 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.488+0000 7f43c4800640 1 --2- 172.21.15.32:0/2425137237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43bc105b20 0x7f43bc105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.488+0000 7f43c4800640 1 -- 172.21.15.32:0/2425137237 >> 172.21.15.32:0/2425137237 conn(0x7f43bc1015b0 msgr2=0x7f43bc103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.488+0000 7f43c4800640 1 -- 172.21.15.32:0/2425137237 shutdown_connections 2024-08-31T07:13:53.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.489+0000 7f43c4800640 1 -- 172.21.15.32:0/2425137237 wait complete. 2024-08-31T07:13:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.490+0000 7f43c4800640 1 Processor -- start 2024-08-31T07:13:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.490+0000 7f43c4800640 1 -- start start 2024-08-31T07:13:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.490+0000 7f43c4800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43bc105b20 0x7f43bc10fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.490+0000 7f43c4800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc10b460 0x7f43bc10ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.490+0000 7f43c4800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc10bdb0 0x7f43bc1104c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.490+0000 7f43c4800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc115ac0 con 0x7f43bc10bdb0 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.490+0000 7f43c4800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc115940 con 0x7f43bc105b20 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.490+0000 7f43c4800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc115c40 con 0x7f43bc10b460 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc10bdb0 0x7f43bc1104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43bc105b20 0x7f43bc10fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43bc105b20 0x7f43bc10fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59580/0 (socket says 172.21.15.32:59580) 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c1e00640 1 -- 172.21.15.32:0/3762569908 learned_addr learned my addr 172.21.15.32:0/3762569908 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c1400640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc10b460 0x7f43bc10ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c1e00640 1 -- 172.21.15.32:0/3762569908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc10b460 msgr2=0x7f43bc10ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c1e00640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc10b460 0x7f43bc10ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c1e00640 1 -- 172.21.15.32:0/3762569908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc10bdb0 msgr2=0x7f43bc1104c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c1e00640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc10bdb0 0x7f43bc1104c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c1e00640 1 -- 172.21.15.32:0/3762569908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc1ac8f0 con 0x7f43bc105b20 2024-08-31T07:13:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.491+0000 7f43c2800640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc10bdb0 0x7f43bc1104c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.492+0000 7f43c1e00640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43bc105b20 0x7f43bc10fa40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43b000ea60 tx=0x7f43b000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.492+0000 7f43b6a00640 1 -- 172.21.15.32:0/3762569908 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b000cde0 con 0x7f43bc105b20 2024-08-31T07:13:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.492+0000 7f43c4800640 1 -- 172.21.15.32:0/3762569908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43bc1acbe0 con 0x7f43bc105b20 2024-08-31T07:13:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.492+0000 7f43b6a00640 1 -- 172.21.15.32:0/3762569908 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b00035f0 con 0x7f43bc105b20 2024-08-31T07:13:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.492+0000 7f43b6a00640 1 -- 172.21.15.32:0/3762569908 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0010690 con 0x7f43bc105b20 2024-08-31T07:13:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.492+0000 7f43c4800640 1 -- 172.21.15.32:0/3762569908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43bc1ad0f0 con 0x7f43bc105b20 2024-08-31T07:13:53.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.493+0000 7f43c4800640 1 -- 172.21.15.32:0/3762569908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43bc105f30 con 0x7f43bc105b20 2024-08-31T07:13:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.496+0000 7f43b6a00640 1 -- 172.21.15.32:0/3762569908 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f43b00037a0 con 0x7f43bc105b20 2024-08-31T07:13:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.497+0000 7f43b6a00640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43a40b4ca0 0x7f43a40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.497+0000 7f43b6a00640 1 -- 172.21.15.32:0/3762569908 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f43b00e5ac0 con 0x7f43bc105b20 2024-08-31T07:13:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.498+0000 7f43c1400640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43a40b4ca0 0x7f43a40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.498+0000 7f43c1400640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43a40b4ca0 0x7f43a40b71b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f43ac02aa00 tx=0x7f43ac031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:53.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.499+0000 7f43b6a00640 1 -- 172.21.15.32:0/3762569908 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f43b0056130 con 0x7f43bc105b20 2024-08-31T07:13:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.733+0000 7f43c4800640 1 -- 172.21.15.32:0/3762569908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43bc110d40 con 0x7f43bc105b20 2024-08-31T07:13:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.734+0000 7f43b6a00640 1 -- 172.21.15.32:0/3762569908 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f43bc110d40 con 0x7f43bc105b20 2024-08-31T07:13:53.736 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:13:53.736 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:13:53.736 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:13:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.737+0000 7f43c4800640 1 -- 172.21.15.32:0/3762569908 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43a40b4ca0 msgr2=0x7f43a40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.737+0000 7f43c4800640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43a40b4ca0 0x7f43a40b71b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f43ac02aa00 tx=0x7f43ac031400 comp rx=0 tx=0).stop 2024-08-31T07:13:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.737+0000 7f43c4800640 1 -- 172.21.15.32:0/3762569908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43bc105b20 msgr2=0x7f43bc10fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.737+0000 7f43c4800640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43bc105b20 0x7f43bc10fa40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43b000ea60 tx=0x7f43b000ef20 comp rx=0 tx=0).stop 2024-08-31T07:13:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.738+0000 7f43c4800640 1 -- 172.21.15.32:0/3762569908 shutdown_connections 2024-08-31T07:13:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.738+0000 7f43c4800640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43a40b4ca0 0x7f43a40b71b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.738+0000 7f43c4800640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43bc10bdb0 0x7f43bc1104c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.738+0000 7f43c4800640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc10b460 0x7f43bc10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.738+0000 7f43c4800640 1 --2- 172.21.15.32:0/3762569908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43bc105b20 0x7f43bc10fa40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.738+0000 7f43c4800640 1 -- 172.21.15.32:0/3762569908 >> 172.21.15.32:0/3762569908 conn(0x7f43bc1015b0 msgr2=0x7f43bc108a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.738+0000 7f43c4800640 1 -- 172.21.15.32:0/3762569908 shutdown_connections 2024-08-31T07:13:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:53.738+0000 7f43c4800640 1 -- 172.21.15.32:0/3762569908 wait complete. 2024-08-31T07:13:54.295 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:54 smithi032 ceph-mon[17252]: pgmap v402: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-08-31T07:13:54.295 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:54 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3762569908' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:54.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:54 smithi073 ceph-mon[21995]: pgmap v402: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-08-31T07:13:54.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:54 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3762569908' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:54.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:54 smithi084 ceph-mon[22387]: pgmap v402: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-08-31T07:13:54.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:54 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3762569908' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:13:54.815 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-31T07:13:54.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:13:54.935 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-31T07:13:55.003 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-31T07:13:55.007 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-31T07:13:55.007 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-31T07:13:55.074 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-31T07:13:55.082 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:13:55.082 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-31T07:13:55.159 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-08-31T07:13:55.160 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-08-31T07:13:55.160 DEBUG:teuthology.orchestra.run.smithi032:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d19e6118f7015f0047264006cb0c873528eb3ae6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-08-31T07:13:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.959+0000 7f9396400640 1 -- 172.21.15.32:0/3794276842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9390103730 msgr2=0x7f9390103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.959+0000 7f9396400640 1 --2- 172.21.15.32:0/3794276842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9390103730 0x7f9390103b40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9380009900 tx=0x7f938002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.959+0000 7f9396400640 1 -- 172.21.15.32:0/3794276842 shutdown_connections 2024-08-31T07:13:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.959+0000 7f9396400640 1 --2- 172.21.15.32:0/3794276842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93901099c0 0x7f93901101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.959+0000 7f9396400640 1 --2- 172.21.15.32:0/3794276842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9390109070 0x7f9390109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.959+0000 7f9396400640 1 --2- 172.21.15.32:0/3794276842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9390103730 0x7f9390103b40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.959+0000 7f9396400640 1 -- 172.21.15.32:0/3794276842 >> 172.21.15.32:0/3794276842 conn(0x7f93900ff180 msgr2=0x7f93901015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.959+0000 7f9396400640 1 -- 172.21.15.32:0/3794276842 shutdown_connections 2024-08-31T07:13:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.959+0000 7f9396400640 1 -- 172.21.15.32:0/3794276842 wait complete. 2024-08-31T07:13:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.960+0000 7f9396400640 1 Processor -- start 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.960+0000 7f9396400640 1 -- start start 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.960+0000 7f9396400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9390103730 0x7f93900754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.960+0000 7f9396400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9390109070 0x7f93900759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.960+0000 7f9396400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93901099c0 0x7f9390075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.960+0000 7f9396400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93901136c0 con 0x7f9390109070 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.960+0000 7f9396400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9390113540 con 0x7f9390103730 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.960+0000 7f9396400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9390113840 con 0x7f93901099c0 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9395000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9390103730 0x7f93900754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9395000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9390103730 0x7f93900754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57036/0 (socket says 172.21.15.32:57036) 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9395000640 1 -- 172.21.15.32:0/1303260647 learned_addr learned my addr 172.21.15.32:0/1303260647 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9387e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9390109070 0x7f93900759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9395a00640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93901099c0 0x7f9390075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9395000640 1 -- 172.21.15.32:0/1303260647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93901099c0 msgr2=0x7f9390075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9395000640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93901099c0 0x7f9390075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9395000640 1 -- 172.21.15.32:0/1303260647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9390109070 msgr2=0x7f93900759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9395000640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9390109070 0x7f93900759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9395000640 1 -- 172.21.15.32:0/1303260647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93901a8390 con 0x7f9390103730 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.961+0000 7f9387e00640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9390109070 0x7f93900759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.962+0000 7f9395000640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9390103730 0x7f93900754b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9380005ab0 tx=0x7f9380003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.962+0000 7f938e000640 1 -- 172.21.15.32:0/1303260647 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938003f070 con 0x7f9390103730 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.962+0000 7f938e000640 1 -- 172.21.15.32:0/1303260647 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9380031b60 con 0x7f9390103730 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.962+0000 7f938e000640 1 -- 172.21.15.32:0/1303260647 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380035030 con 0x7f9390103730 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.962+0000 7f9396400640 1 -- 172.21.15.32:0/1303260647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93901a8620 con 0x7f9390103730 2024-08-31T07:13:55.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.962+0000 7f9396400640 1 -- 172.21.15.32:0/1303260647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93901a8ab0 con 0x7f9390103730 2024-08-31T07:13:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.965+0000 7f939e6be740 1 Processor -- start 2024-08-31T07:13:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.965+0000 7f939e6be740 1 -- start start 2024-08-31T07:13:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f939e6be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f37c070 0x55d88f721b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f939e6be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 0x55d88f7220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f939e6be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f7225e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f939e6be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d88f6987b0 con 0x55d88f35b390 2024-08-31T07:13:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f939e6be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d88f687d40 con 0x55d88f37c070 2024-08-31T07:13:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f939e6be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d88f68bcf0 con 0x55d88f3531b0 2024-08-31T07:13:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f9395a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f37c070 0x55d88f721b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f9387e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f7225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f9395a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f37c070 0x55d88f721b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57048/0 (socket says 172.21.15.32:57048) 2024-08-31T07:13:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f9395a00640 1 -- 172.21.15.32:0/3750943482 learned_addr learned my addr 172.21.15.32:0/3750943482 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:55.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f9395a00640 1 -- 172.21.15.32:0/3750943482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 msgr2=0x55d88f7220a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:13:55.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f9395a00640 1 --2- 172.21.15.32:0/3750943482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 0x55d88f7220a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f9395a00640 1 -- 172.21.15.32:0/3750943482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 msgr2=0x55d88f7225e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f9395a00640 1 --2- 172.21.15.32:0/3750943482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f7225e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.966+0000 7f9395a00640 1 -- 172.21.15.32:0/3750943482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88f3e7650 con 0x55d88f37c070 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.967+0000 7f9395a00640 1 --2- 172.21.15.32:0/3750943482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f37c070 0x55d88f721b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9388011440 tx=0x7f9388011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e73533b1106a1b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.967+0000 7f938e000640 1 -- 172.21.15.32:0/1303260647 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9380042070 con 0x7f9390103730 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.967+0000 7f9386000640 1 -- 172.21.15.32:0/3750943482 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388003c30 con 0x55d88f37c070 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.967+0000 7f9386000640 1 -- 172.21.15.32:0/3750943482 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9388003dd0 con 0x55d88f37c070 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.967+0000 7f9386000640 1 -- 172.21.15.32:0/3750943482 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938800fae0 con 0x55d88f37c070 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.968+0000 7f939e6be740 1 -- 172.21.15.32:0/3750943482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f37c070 msgr2=0x55d88f721b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.968+0000 7f939e6be740 1 --2- 172.21.15.32:0/3750943482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f37c070 0x55d88f721b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9388011440 tx=0x7f9388011900 comp rx=0 tx=0).stop 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.968+0000 7f939e6be740 1 -- 172.21.15.32:0/3750943482 shutdown_connections 2024-08-31T07:13:55.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.968+0000 7f939e6be740 1 --2- 172.21.15.32:0/3750943482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f7225e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.968+0000 7f939e6be740 1 --2- 172.21.15.32:0/3750943482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 0x55d88f7220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.968+0000 7f939e6be740 1 --2- 172.21.15.32:0/3750943482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f37c070 0x55d88f721b60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.968+0000 7f939e6be740 1 -- 172.21.15.32:0/3750943482 >> 172.21.15.32:0/3750943482 conn(0x55d88f370460 msgr2=0x55d88f71fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.968+0000 7f939e6be740 1 -- 172.21.15.32:0/3750943482 shutdown_connections 2024-08-31T07:13:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.968+0000 7f939e6be740 1 -- 172.21.15.32:0/3750943482 wait complete. 2024-08-31T07:13:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.968+0000 7f938e000640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93740b4c40 0x7f93740b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.969+0000 7f939e6be740 1 -- start start 2024-08-31T07:13:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.969+0000 7f938e000640 1 -- 172.21.15.32:0/1303260647 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f9380105b20 con 0x7f9390103730 2024-08-31T07:13:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.969+0000 7f9395a00640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93740b4c40 0x7f93740b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.969+0000 7f939e6be740 1 Processor -- start 2024-08-31T07:13:55.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.969+0000 7f939e6be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 0x55d88f72d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f939e6be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f35b390 0x55d88f72dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f939e6be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f37c070 0x55d88f739690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9395a00640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93740b4c40 0x7f93740b7150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f938801b040 tx=0x7f938800f540 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f939e6be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d88f4ea700 con 0x55d88f37c070 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f939e6be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d88f60ecf0 con 0x55d88f35b390 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f939e6be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d88f730340 con 0x55d88f3531b0 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f939e6be740 10 client.0 _ll_register_callbacks cb 0x55d88f634470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9395000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 0x55d88f72d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9395000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 0x55d88f72d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46448/0 (socket says 172.21.15.32:46448) 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9395a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f37c070 0x55d88f739690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9395000640 1 -- 172.21.15.32:0/2138938755 learned_addr learned my addr 172.21.15.32:0/2138938755 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9387e00640 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f35b390 0x55d88f72dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9395000640 1 -- 172.21.15.32:0/2138938755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f35b390 msgr2=0x55d88f72dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9395000640 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f35b390 0x55d88f72dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9395000640 1 -- 172.21.15.32:0/2138938755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f37c070 msgr2=0x55d88f739690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9395000640 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f37c070 0x55d88f739690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.970+0000 7f9395000640 1 -- 172.21.15.32:0/2138938755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88f64f8b0 con 0x55d88f3531b0 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.971+0000 7f9395000640 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 0x55d88f72d730 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f93800c94a0 tx=0x7f93800cba60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac200b0ba6ecd192 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.971+0000 7f937ea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.971+0000 7f937ea00640 1 -- 172.21.15.32:0/2138938755 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93800cc230 con 0x55d88f3531b0 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.971+0000 7f937ea00640 1 -- 172.21.15.32:0/2138938755 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f93800c84c0 con 0x55d88f3531b0 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.971+0000 7f939e6be740 1 -- 172.21.15.32:0/2138938755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9388003dd0 con 0x55d88f3531b0 2024-08-31T07:13:55.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.971+0000 7f937ea00640 1 -- 172.21.15.32:0/2138938755 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93800d0aa0 con 0x55d88f3531b0 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.971+0000 7f939e6be740 10 client.25822 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.971+0000 7f939e6be740 1 -- 172.21.15.32:0/2138938755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d88f73c3e0 con 0x55d88f3531b0 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.972+0000 7f937ea00640 1 -- 172.21.15.32:0/2138938755 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9388003dd0 con 0x55d88f3531b0 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.972+0000 7f937ea00640 1 -- 172.21.15.32:0/2138938755 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f93800cb490 con 0x55d88f3531b0 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.972+0000 7f939e6be740 10 client.25822 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.972+0000 7f939e6be740 10 client.25822 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.972+0000 7f939e6be740 20 client.25822 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.972+0000 7f939e6be740 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d88f33a9f0 0x55d88f73f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.972+0000 7f939e6be740 4 client.25822 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.972+0000 7f939e6be740 1 -- 172.21.15.32:0/2138938755 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d88f69a700 con 0x55d88f33a9f0 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.972+0000 7f9387e00640 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d88f33a9f0 0x55d88f73f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.973+0000 7f9387e00640 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d88f33a9f0 0x55d88f73f810 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25590a195be85fd5 server_cookie=b1c159f6d2f029ec in_seq=0 out_seq=0 2024-08-31T07:13:55.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.973+0000 7f937ea00640 10 client.25822 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:13:55.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f937ea00640 1 -- 172.21.15.32:0/2138938755 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d88f69a700 con 0x55d88f33a9f0 2024-08-31T07:13:55.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f937ea00640 10 client.25822 handle_command_reply: tid=0 2024-08-31T07:13:55.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f939e6be740 1 client.25822 shutdown 2024-08-31T07:13:55.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f939e6be740 1 -- 172.21.15.32:0/2138938755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 msgr2=0x55d88f72d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:55.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f939e6be740 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 0x55d88f72d730 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f93800c94a0 tx=0x7f93800cba60 comp rx=0 tx=0).stop 2024-08-31T07:13:55.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f939e6be740 1 -- 172.21.15.32:0/2138938755 shutdown_connections 2024-08-31T07:13:55.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f939e6be740 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d88f33a9f0 0x55d88f73f810 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f939e6be740 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f37c070 0x55d88f739690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f939e6be740 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f35b390 0x55d88f72dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f939e6be740 1 --2- 172.21.15.32:0/2138938755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f3531b0 0x55d88f72d730 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f939e6be740 1 -- 172.21.15.32:0/2138938755 >> 172.21.15.32:0/2138938755 conn(0x55d88f370460 msgr2=0x55d88f724cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:55.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.974+0000 7f937ea00640 0 client.25822 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:13:55.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.975+0000 7f939e6be740 1 -- 172.21.15.32:0/2138938755 shutdown_connections 2024-08-31T07:13:55.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.975+0000 7f939e6be740 1 -- 172.21.15.32:0/2138938755 wait complete. 2024-08-31T07:13:55.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.975+0000 7f939e6be740 20 client.25822 trim_cache size 0 max 16384 2024-08-31T07:13:55.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.989+0000 7f939e6be740 1 Processor -- start 2024-08-31T07:13:55.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.989+0000 7f939e6be740 1 -- start start 2024-08-31T07:13:55.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.989+0000 7f939e6be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 0x55d88f728d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.989+0000 7f939e6be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f739590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.989+0000 7f939e6be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 0x55d88f72f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.989+0000 7f939e6be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93901b1230 con 0x55d88f35b390 2024-08-31T07:13:55.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.989+0000 7f939e6be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93901b2d70 con 0x55d88f3531b0 2024-08-31T07:13:55.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.989+0000 7f939e6be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93901b1900 con 0x55d88f37c070 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.990+0000 7f9395000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 0x55d88f728d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.990+0000 7f9395a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 0x55d88f72f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.990+0000 7f9395000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 0x55d88f728d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57066/0 (socket says 172.21.15.32:57066) 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.990+0000 7f9387e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f739590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.990+0000 7f9395000640 1 -- 172.21.15.32:0/3995183463 learned_addr learned my addr 172.21.15.32:0/3995183463 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.990+0000 7f9395a00640 1 -- 172.21.15.32:0/3995183463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 msgr2=0x55d88f728d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.990+0000 7f9395a00640 1 --2- 172.21.15.32:0/3995183463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 0x55d88f728d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.990+0000 7f9395a00640 1 -- 172.21.15.32:0/3995183463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 msgr2=0x55d88f739590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.990+0000 7f9395a00640 1 --2- 172.21.15.32:0/3995183463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f739590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.990+0000 7f9395a00640 1 -- 172.21.15.32:0/3995183463 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88f7243f0 con 0x55d88f37c070 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.991+0000 7f9395a00640 1 --2- 172.21.15.32:0/3995183463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 0x55d88f72f180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9388011e00 tx=0x7f9388009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d1020e7678afd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.991+0000 7f9385600640 1 -- 172.21.15.32:0/3995183463 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388004e20 con 0x55d88f37c070 2024-08-31T07:13:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.991+0000 7f9385600640 1 -- 172.21.15.32:0/3995183463 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938800fd00 con 0x55d88f37c070 2024-08-31T07:13:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.991+0000 7f9385600640 1 -- 172.21.15.32:0/3995183463 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388026670 con 0x55d88f37c070 2024-08-31T07:13:55.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 -- 172.21.15.32:0/3995183463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 msgr2=0x55d88f72f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:55.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 --2- 172.21.15.32:0/3995183463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 0x55d88f72f180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9388011e00 tx=0x7f9388009500 comp rx=0 tx=0).stop 2024-08-31T07:13:55.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 -- 172.21.15.32:0/3995183463 shutdown_connections 2024-08-31T07:13:55.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 --2- 172.21.15.32:0/3995183463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 0x55d88f72f180 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 --2- 172.21.15.32:0/3995183463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f739590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 --2- 172.21.15.32:0/3995183463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 0x55d88f728d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 -- 172.21.15.32:0/3995183463 >> 172.21.15.32:0/3995183463 conn(0x55d88f370460 msgr2=0x55d88f72ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 -- 172.21.15.32:0/3995183463 shutdown_connections 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 -- 172.21.15.32:0/3995183463 wait complete. 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 -- start start 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.992+0000 7f939e6be740 1 Processor -- start 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f939e6be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 0x55d88f735c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f939e6be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f7320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f939e6be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 0x55d88f732610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f939e6be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93901b26a0 con 0x55d88f35b390 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f939e6be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93901b1fd0 con 0x55d88f3531b0 2024-08-31T07:13:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f939e6be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93901af210 con 0x55d88f37c070 2024-08-31T07:13:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f939e6be740 10 client.0 _ll_register_callbacks cb 0x7f93800cbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f9387e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f7320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f9395000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 0x55d88f735c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f9387e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f7320d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60272/0 (socket says 172.21.15.32:60272) 2024-08-31T07:13:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f9387e00640 1 -- 172.21.15.32:0/3744225857 learned_addr learned my addr 172.21.15.32:0/3744225857 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.993+0000 7f9395a00640 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 0x55d88f732610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:55.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f9387e00640 1 -- 172.21.15.32:0/3744225857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 msgr2=0x55d88f732610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:55.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f9387e00640 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 0x55d88f732610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f9387e00640 1 -- 172.21.15.32:0/3744225857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 msgr2=0x55d88f735c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f9387e00640 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 0x55d88f735c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f9387e00640 1 -- 172.21.15.32:0/3744225857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88f732bd0 con 0x55d88f35b390 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f9395000640 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 0x55d88f735c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f9387e00640 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f7320d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9378009fd0 tx=0x7f9378012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b353268762d5f73 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f937ea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f937ea00640 1 -- 172.21.15.32:0/3744225857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378011b80 con 0x55d88f35b390 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f937ea00640 1 -- 172.21.15.32:0/3744225857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f9378011d20 con 0x55d88f35b390 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.994+0000 7f939e6be740 1 -- 172.21.15.32:0/3744225857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938800fd00 con 0x55d88f35b390 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f937ea00640 1 -- 172.21.15.32:0/3744225857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9378005020 con 0x55d88f35b390 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f939e6be740 10 client.15816 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f939e6be740 1 -- 172.21.15.32:0/3744225857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d88f7aaf90 con 0x55d88f35b390 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f937ea00640 1 -- 172.21.15.32:0/3744225857 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938800fd00 con 0x55d88f35b390 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f937ea00640 1 -- 172.21.15.32:0/3744225857 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9378004bb0 con 0x55d88f35b390 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f939e6be740 10 client.15816 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f939e6be740 10 client.15816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f939e6be740 20 client.15816 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f939e6be740 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d88f30fda0 0x55d88f7ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f939e6be740 4 client.15816 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.995+0000 7f939e6be740 1 -- 172.21.15.32:0/3744225857 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d88f6961a0 con 0x55d88f30fda0 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.996+0000 7f9395000640 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d88f30fda0 0x55d88f7ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.996+0000 7f9395000640 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d88f30fda0 0x55d88f7ad7a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e66f6c1da08c1788 server_cookie=8e16ffb3578121ae in_seq=0 out_seq=0 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.996+0000 7f937ea00640 10 client.15816 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f937ea00640 1 -- 172.21.15.32:0/3744225857 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d88f6961a0 con 0x55d88f30fda0 2024-08-31T07:13:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f937ea00640 10 client.15816 handle_command_reply: tid=0 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 client.15816 shutdown 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 -- 172.21.15.32:0/3744225857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 msgr2=0x55d88f7320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f7320d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9378009fd0 tx=0x7f9378012bc0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 -- 172.21.15.32:0/3744225857 shutdown_connections 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d88f30fda0 0x55d88f7ad7a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d88f37c070 0x55d88f732610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d88f35b390 0x55d88f7320d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 --2- 172.21.15.32:0/3744225857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d88f3531b0 0x55d88f735c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f937ea00640 0 client.15816 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 -- 172.21.15.32:0/3744225857 >> 172.21.15.32:0/3744225857 conn(0x55d88f370460 msgr2=0x55d88f723450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 -- 172.21.15.32:0/3744225857 shutdown_connections 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 1 -- 172.21.15.32:0/3744225857 wait complete. 2024-08-31T07:13:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.997+0000 7f939e6be740 20 client.15816 trim_cache size 0 max 16384 2024-08-31T07:13:56.003 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 -- 172.21.15.32:0/1303260647 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93740b4c40 msgr2=0x7f93740b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93740b4c40 0x7f93740b7150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f938801b040 tx=0x7f938800f540 comp rx=0 tx=0).stop 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 -- 172.21.15.32:0/1303260647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9390103730 msgr2=0x7f93900754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9390103730 0x7f93900754b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9380005ab0 tx=0x7f9380003c10 comp rx=0 tx=0).stop 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 -- 172.21.15.32:0/1303260647 shutdown_connections 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93740b4c40 0x7f93740b7150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93901099c0 0x7f9390075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9390109070 0x7f93900759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 --2- 172.21.15.32:0/1303260647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9390103730 0x7f93900754b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 -- 172.21.15.32:0/1303260647 >> 172.21.15.32:0/1303260647 conn(0x7f93900ff180 msgr2=0x7f93901015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.998+0000 7f9396400640 1 -- 172.21.15.32:0/1303260647 shutdown_connections 2024-08-31T07:13:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:55.999+0000 7f9396400640 1 -- 172.21.15.32:0/1303260647 wait complete. 2024-08-31T07:13:56.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:56 smithi032 ceph-mon[17252]: pgmap v403: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:56 smithi073 ceph-mon[21995]: pgmap v403: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:56.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:56 smithi084 ceph-mon[22387]: pgmap v403: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:57.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:13:57.419 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:13:58.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.023+0000 7f147bc00640 1 -- 172.21.15.32:0/3643085145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1474103920 msgr2=0x7f1474103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:58.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.023+0000 7f147bc00640 1 --2- 172.21.15.32:0/3643085145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1474103920 0x7f1474103d30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1468009900 tx=0x7f146802a410 comp rx=0 tx=0).stop 2024-08-31T07:13:58.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.023+0000 7f147bc00640 1 -- 172.21.15.32:0/3643085145 shutdown_connections 2024-08-31T07:13:58.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.023+0000 7f147bc00640 1 --2- 172.21.15.32:0/3643085145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1474109bb0 0x7f14741103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.023+0000 7f147bc00640 1 --2- 172.21.15.32:0/3643085145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1474109260 0x7f1474109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.023+0000 7f147bc00640 1 --2- 172.21.15.32:0/3643085145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1474103920 0x7f1474103d30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.023+0000 7f147bc00640 1 -- 172.21.15.32:0/3643085145 >> 172.21.15.32:0/3643085145 conn(0x7f14740ff370 msgr2=0x7f14741017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.023+0000 7f147bc00640 1 -- 172.21.15.32:0/3643085145 shutdown_connections 2024-08-31T07:13:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.023+0000 7f147bc00640 1 -- 172.21.15.32:0/3643085145 wait complete. 2024-08-31T07:13:58.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.024+0000 7f147bc00640 1 Processor -- start 2024-08-31T07:13:58.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f147bc00640 1 -- start start 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f147bc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1474103920 0x7f14740754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f147bc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1474109260 0x7f14740759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f147bc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1474109bb0 0x7f1474075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f147bc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14741138c0 con 0x7f1474103920 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f147bc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1474113740 con 0x7f1474109bb0 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f147bc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1474113a40 con 0x7f1474109260 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f1479200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1474103920 0x7f14740754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f1473e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1474109260 0x7f14740759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f1473e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1474109260 0x7f14740759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46490/0 (socket says 172.21.15.32:46490) 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f1473e00640 1 -- 172.21.15.32:0/1608987211 learned_addr learned my addr 172.21.15.32:0/1608987211 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.025+0000 7f1479c00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1474109bb0 0x7f1474075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.026+0000 7f1473e00640 1 -- 172.21.15.32:0/1608987211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1474109bb0 msgr2=0x7f1474075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.026+0000 7f1473e00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1474109bb0 0x7f1474075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.026+0000 7f1473e00640 1 -- 172.21.15.32:0/1608987211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1474103920 msgr2=0x7f14740754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.026+0000 7f1473e00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1474103920 0x7f14740754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.026+0000 7f1473e00640 1 -- 172.21.15.32:0/1608987211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14741a8530 con 0x7f1474109260 2024-08-31T07:13:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.026+0000 7f1473e00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1474109260 0x7f14740759f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f146000e9c0 tx=0x7f146000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.026+0000 7f1471600640 1 -- 172.21.15.32:0/1608987211 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146000ccc0 con 0x7f1474109260 2024-08-31T07:13:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.027+0000 7f1471600640 1 -- 172.21.15.32:0/1608987211 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14600054d0 con 0x7f1474109260 2024-08-31T07:13:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.027+0000 7f1471600640 1 -- 172.21.15.32:0/1608987211 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460010430 con 0x7f1474109260 2024-08-31T07:13:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.027+0000 7f147bc00640 1 -- 172.21.15.32:0/1608987211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14741a8820 con 0x7f1474109260 2024-08-31T07:13:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.027+0000 7f147bc00640 1 -- 172.21.15.32:0/1608987211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14741a8ce0 con 0x7f1474109260 2024-08-31T07:13:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.030+0000 7f14829d6740 1 Processor -- start 2024-08-31T07:13:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.030+0000 7f14829d6740 1 -- start start 2024-08-31T07:13:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.030+0000 7f14829d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 0x5594c5ff0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.030+0000 7f14829d6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c5ff0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.030+0000 7f14829d6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ff1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.030+0000 7f14829d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5594c5f677b0 con 0x5594c5c4b070 2024-08-31T07:13:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.030+0000 7f14829d6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5594c5f56d40 con 0x5594c5c2a390 2024-08-31T07:13:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.030+0000 7f14829d6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5594c5f5acf0 con 0x5594c5c221b0 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1479c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 0x5594c5ff0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1479200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c5ff0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1479c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 0x5594c5ff0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60300/0 (socket says 172.21.15.32:60300) 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1479c00640 1 -- 172.21.15.32:0/1162905209 learned_addr learned my addr 172.21.15.32:0/1162905209 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1471600640 1 -- 172.21.15.32:0/1608987211 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1460005020 con 0x7f1474109260 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1479200640 1 -- 172.21.15.32:0/1162905209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 msgr2=0x5594c5ff1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1473e00640 1 --2- 172.21.15.32:0/1162905209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ff1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1479200640 1 --2- 172.21.15.32:0/1162905209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ff1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1479200640 1 -- 172.21.15.32:0/1162905209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 msgr2=0x5594c5ff0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1479200640 1 --2- 172.21.15.32:0/1162905209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 0x5594c5ff0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1479200640 1 -- 172.21.15.32:0/1162905209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c5cb6650 con 0x5594c5c221b0 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1479c00640 1 --2- 172.21.15.32:0/1162905209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 0x5594c5ff0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:13:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.031+0000 7f1473e00640 1 --2- 172.21.15.32:0/1162905209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ff1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:13:58.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.032+0000 7f1479200640 1 --2- 172.21.15.32:0/1162905209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c5ff0ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f146802aa00 tx=0x7f1468004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db7db298c4bd5cdc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:58.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.032+0000 7f1466000640 1 -- 172.21.15.32:0/1162905209 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468035040 con 0x5594c5c221b0 2024-08-31T07:13:58.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.032+0000 7f1466000640 1 -- 172.21.15.32:0/1162905209 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14680054f0 con 0x5594c5c221b0 2024-08-31T07:13:58.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.032+0000 7f1466000640 1 -- 172.21.15.32:0/1162905209 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680035f0 con 0x5594c5c221b0 2024-08-31T07:13:58.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f1471600640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f145c0b4d20 0x7f145c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f14829d6740 1 -- 172.21.15.32:0/1162905209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 msgr2=0x5594c5ff0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f14829d6740 1 --2- 172.21.15.32:0/1162905209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c5ff0ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f146802aa00 tx=0x7f1468004040 comp rx=0 tx=0).stop 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f1471600640 1 -- 172.21.15.32:0/1608987211 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f14600e5880 con 0x7f1474109260 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f14829d6740 1 -- 172.21.15.32:0/1162905209 shutdown_connections 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f14829d6740 1 --2- 172.21.15.32:0/1162905209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ff1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f14829d6740 1 --2- 172.21.15.32:0/1162905209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c5ff0ed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f14829d6740 1 --2- 172.21.15.32:0/1162905209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 0x5594c5ff0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f14829d6740 1 -- 172.21.15.32:0/1162905209 >> 172.21.15.32:0/1162905209 conn(0x5594c5c3f460 msgr2=0x5594c5fee880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f1479c00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f145c0b4d20 0x7f145c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f14829d6740 1 -- 172.21.15.32:0/1162905209 shutdown_connections 2024-08-31T07:13:58.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.033+0000 7f14829d6740 1 -- 172.21.15.32:0/1162905209 wait complete. 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.034+0000 7f1479c00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f145c0b4d20 0x7f145c0b7230 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f146c005fd0 tx=0x7f146c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.034+0000 7f14829d6740 1 -- start start 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.034+0000 7f14829d6740 1 Processor -- start 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f14829d6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c5ffc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f14829d6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ffcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f14829d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 0x5594c6008400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f14829d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5594c5db9700 con 0x5594c5c4b070 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f14829d6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5594c5eddcf0 con 0x5594c5c2a390 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f14829d6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5594c5fff110 con 0x5594c5c221b0 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f14829d6740 10 client.0 _ll_register_callbacks cb 0x5594c5f03470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f1479c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 0x5594c6008400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f1473e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ffcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f1479200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c5ffc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f1473e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ffcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57100/0 (socket says 172.21.15.32:57100) 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.035+0000 7f1473e00640 1 -- 172.21.15.32:0/3377292199 learned_addr learned my addr 172.21.15.32:0/3377292199 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f1473e00640 1 -- 172.21.15.32:0/3377292199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 msgr2=0x5594c5ffc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f1473e00640 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c5ffc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f1473e00640 1 -- 172.21.15.32:0/3377292199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 msgr2=0x5594c6008400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f1473e00640 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 0x5594c6008400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f1473e00640 1 -- 172.21.15.32:0/3377292199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c5f1e8b0 con 0x5594c5c2a390 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f1473e00640 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ffcb60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f146000c6a0 tx=0x7f146000ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db5c873841f47155 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f145aa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f145aa00640 1 -- 172.21.15.32:0/3377292199 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600d2020 con 0x5594c5c2a390 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f145aa00640 1 -- 172.21.15.32:0/3377292199 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f14600a8220 con 0x5594c5c2a390 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f14829d6740 1 -- 172.21.15.32:0/3377292199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14680054f0 con 0x5594c5c2a390 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f145aa00640 1 -- 172.21.15.32:0/3377292199 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14600abaa0 con 0x5594c5c2a390 2024-08-31T07:13:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f14829d6740 10 client.25838 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.036+0000 7f14829d6740 1 -- 172.21.15.32:0/3377292199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5594c600b170 con 0x5594c5c2a390 2024-08-31T07:13:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.037+0000 7f145aa00640 1 -- 172.21.15.32:0/3377292199 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14680054f0 con 0x5594c5c2a390 2024-08-31T07:13:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.037+0000 7f145aa00640 1 -- 172.21.15.32:0/3377292199 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1460097190 con 0x5594c5c2a390 2024-08-31T07:13:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.037+0000 7f14829d6740 10 client.25838 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.037+0000 7f14829d6740 10 client.25838 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:13:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.037+0000 7f14829d6740 20 client.25838 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.037+0000 7f14829d6740 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5594c5c099f0 0x5594c600e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.037+0000 7f14829d6740 4 client.25838 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.037+0000 7f14829d6740 1 -- 172.21.15.32:0/3377292199 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594c5f69700 con 0x5594c5c099f0 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.037+0000 7f1479200640 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5594c5c099f0 0x5594c600e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.038+0000 7f1479200640 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5594c5c099f0 0x5594c600e660 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec1157891cd22ce2 server_cookie=cd0f12637cc79e17 in_seq=0 out_seq=0 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.038+0000 7f145aa00640 10 client.25838 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f145aa00640 1 -- 172.21.15.32:0/3377292199 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594c5f69700 con 0x5594c5c099f0 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f145aa00640 10 client.25838 handle_command_reply: tid=0 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f14829d6740 1 client.25838 shutdown 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f14829d6740 1 -- 172.21.15.32:0/3377292199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 msgr2=0x5594c5ffcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f14829d6740 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ffcb60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f146000c6a0 tx=0x7f146000ce60 comp rx=0 tx=0).stop 2024-08-31T07:13:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f14829d6740 1 -- 172.21.15.32:0/3377292199 shutdown_connections 2024-08-31T07:13:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f14829d6740 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5594c5c099f0 0x5594c600e660 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f14829d6740 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c4b070 0x5594c6008400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f14829d6740 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c2a390 0x5594c5ffcb60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f145aa00640 0 client.25838 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:13:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f14829d6740 1 --2- 172.21.15.32:0/3377292199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c5ffc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.039+0000 7f14829d6740 1 -- 172.21.15.32:0/3377292199 >> 172.21.15.32:0/3377292199 conn(0x5594c5c3f460 msgr2=0x5594c5ff3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.040+0000 7f14829d6740 1 -- 172.21.15.32:0/3377292199 shutdown_connections 2024-08-31T07:13:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.040+0000 7f14829d6740 1 -- 172.21.15.32:0/3377292199 wait complete. 2024-08-31T07:13:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.040+0000 7f14829d6740 20 client.25838 trim_cache size 0 max 16384 2024-08-31T07:13:58.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f14829d6740 1 Processor -- start 2024-08-31T07:13:58.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f14829d6740 1 -- start start 2024-08-31T07:13:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f14829d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c221b0 0x5594c5ff5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f14829d6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c2a390 0x5594c5ff6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f14829d6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c4b070 0x5594c5fede30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f14829d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14741b1540 con 0x5594c5c221b0 2024-08-31T07:13:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f14829d6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f14741b3080 con 0x5594c5c4b070 2024-08-31T07:13:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f14829d6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f14741b1c10 con 0x5594c5c2a390 2024-08-31T07:13:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f1479200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c221b0 0x5594c5ff5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f1479200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c221b0 0x5594c5ff5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60328/0 (socket says 172.21.15.32:60328) 2024-08-31T07:13:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.054+0000 7f1479200640 1 -- 172.21.15.32:0/321577065 learned_addr learned my addr 172.21.15.32:0/321577065 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:58.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.055+0000 7f1479c00640 1 --2- 172.21.15.32:0/321577065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c4b070 0x5594c5fede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.055+0000 7f1473e00640 1 --2- 172.21.15.32:0/321577065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c2a390 0x5594c5ff6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.055+0000 7f1479200640 1 -- 172.21.15.32:0/321577065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c2a390 msgr2=0x5594c5ff6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.055+0000 7f1479200640 1 --2- 172.21.15.32:0/321577065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c2a390 0x5594c5ff6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.055+0000 7f1479200640 1 -- 172.21.15.32:0/321577065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c4b070 msgr2=0x5594c5fede30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.055+0000 7f1479200640 1 --2- 172.21.15.32:0/321577065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c4b070 0x5594c5fede30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.055+0000 7f1479200640 1 -- 172.21.15.32:0/321577065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c5ffced0 con 0x5594c5c221b0 2024-08-31T07:13:58.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.055+0000 7f1479200640 1 --2- 172.21.15.32:0/321577065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c221b0 0x5594c5ff5c10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1468004b20 tx=0x7f1468004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89232fc5a5a037cd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.055+0000 7f1465600640 1 -- 172.21.15.32:0/321577065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146805c070 con 0x5594c5c221b0 2024-08-31T07:13:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.056+0000 7f1465600640 1 -- 172.21.15.32:0/321577065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1468054030 con 0x5594c5c221b0 2024-08-31T07:13:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.056+0000 7f1465600640 1 -- 172.21.15.32:0/321577065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468035040 con 0x5594c5c221b0 2024-08-31T07:13:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.056+0000 7f14829d6740 1 -- 172.21.15.32:0/321577065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c221b0 msgr2=0x5594c5ff5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.056+0000 7f14829d6740 1 --2- 172.21.15.32:0/321577065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c221b0 0x5594c5ff5c10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1468004b20 tx=0x7f1468004b50 comp rx=0 tx=0).stop 2024-08-31T07:13:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.056+0000 7f14829d6740 1 -- 172.21.15.32:0/321577065 shutdown_connections 2024-08-31T07:13:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.056+0000 7f14829d6740 1 --2- 172.21.15.32:0/321577065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c4b070 0x5594c5fede30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.056+0000 7f14829d6740 1 --2- 172.21.15.32:0/321577065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c2a390 0x5594c5ff6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.056+0000 7f14829d6740 1 --2- 172.21.15.32:0/321577065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c221b0 0x5594c5ff5c10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.056+0000 7f14829d6740 1 -- 172.21.15.32:0/321577065 >> 172.21.15.32:0/321577065 conn(0x5594c5c3f460 msgr2=0x5594c5ffac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.057+0000 7f14829d6740 1 -- 172.21.15.32:0/321577065 shutdown_connections 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.057+0000 7f14829d6740 1 -- 172.21.15.32:0/321577065 wait complete. 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.057+0000 7f14829d6740 1 -- start start 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.057+0000 7f14829d6740 1 Processor -- start 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f14829d6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c6004370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f14829d6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c2a390 0x5594c60048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f14829d6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c4b070 0x5594c5fff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f14829d6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14741b29b0 con 0x5594c5c2a390 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f14829d6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f14741b22e0 con 0x5594c5c4b070 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f14829d6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f14741af520 con 0x5594c5c221b0 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f14829d6740 10 client.0 _ll_register_callbacks cb 0x7f14600abd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f1473e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c2a390 0x5594c60048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f1479200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c6004370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f1473e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c2a390 0x5594c60048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60336/0 (socket says 172.21.15.32:60336) 2024-08-31T07:13:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f1473e00640 1 -- 172.21.15.32:0/20208345 learned_addr learned my addr 172.21.15.32:0/20208345 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:13:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.058+0000 7f1479c00640 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c4b070 0x5594c5fff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.059+0000 7f1479200640 1 -- 172.21.15.32:0/20208345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c4b070 msgr2=0x5594c5fff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:58.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.059+0000 7f1479200640 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c4b070 0x5594c5fff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.059+0000 7f1479200640 1 -- 172.21.15.32:0/20208345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c2a390 msgr2=0x5594c60048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:13:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.059+0000 7f1479200640 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c2a390 0x5594c60048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.059+0000 7f1479200640 1 -- 172.21.15.32:0/20208345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c5fff860 con 0x5594c5c221b0 2024-08-31T07:13:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.059+0000 7f1479200640 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c6004370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1468031440 tx=0x7f14680095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92eb8983ff76ae67 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:13:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.059+0000 7f145aa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:13:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.059+0000 7f145aa00640 1 -- 172.21.15.32:0/20208345 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468035070 con 0x5594c5c221b0 2024-08-31T07:13:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.059+0000 7f145aa00640 1 -- 172.21.15.32:0/20208345 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f14680035f0 con 0x5594c5c221b0 2024-08-31T07:13:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.059+0000 7f14829d6740 1 -- 172.21.15.32:0/20208345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1468054030 con 0x5594c5c221b0 2024-08-31T07:13:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f145aa00640 1 -- 172.21.15.32:0/20208345 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f146805c040 con 0x5594c5c221b0 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f14829d6740 10 client.25846 fetch_fsmap learned FSMap version 29 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f14829d6740 1 -- 172.21.15.32:0/20208345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5594c6079720 con 0x5594c5c221b0 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f145aa00640 1 -- 172.21.15.32:0/20208345 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1468054030 con 0x5594c5c221b0 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f145aa00640 1 -- 172.21.15.32:0/20208345 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1468004ec0 con 0x5594c5c221b0 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f14829d6740 10 client.25846 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f14829d6740 10 client.25846 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f14829d6740 20 client.25846 populate_metadata read hostname 'smithi032' 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f14829d6740 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5594c5bdeda0 0x5594c607b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f14829d6740 4 client.25846 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.060+0000 7f14829d6740 1 -- 172.21.15.32:0/20208345 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594c5f651a0 con 0x5594c5bdeda0 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.061+0000 7f1473e00640 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5594c5bdeda0 0x5594c607b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.061+0000 7f1473e00640 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5594c5bdeda0 0x5594c607b900 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46bbe9a31ae14145 server_cookie=dd10341331587b62 in_seq=0 out_seq=0 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.061+0000 7f145aa00640 10 client.25846 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:13:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f145aa00640 1 -- 172.21.15.32:0/20208345 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594c5f651a0 con 0x5594c5bdeda0 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f145aa00640 10 client.25846 handle_command_reply: tid=0 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 client.25846 shutdown 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 -- 172.21.15.32:0/20208345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 msgr2=0x5594c6004370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c6004370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1468031440 tx=0x7f14680095a0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 -- 172.21.15.32:0/20208345 shutdown_connections 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5594c5bdeda0 0x5594c607b900 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5594c5c4b070 0x5594c5fff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5594c5c2a390 0x5594c60048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 --2- 172.21.15.32:0/20208345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594c5c221b0 0x5594c6004370 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f145aa00640 0 client.25846 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 -- 172.21.15.32:0/20208345 >> 172.21.15.32:0/20208345 conn(0x5594c5c3f460 msgr2=0x5594c5ff1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 -- 172.21.15.32:0/20208345 shutdown_connections 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 1 -- 172.21.15.32:0/20208345 wait complete. 2024-08-31T07:13:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.062+0000 7f14829d6740 20 client.25846 trim_cache size 0 max 16384 2024-08-31T07:13:58.069 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:13:58.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 -- 172.21.15.32:0/1608987211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f145c0b4d20 msgr2=0x7f145c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f145c0b4d20 0x7f145c0b7230 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f146c005fd0 tx=0x7f146c009500 comp rx=0 tx=0).stop 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 -- 172.21.15.32:0/1608987211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1474109260 msgr2=0x7f14740759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1474109260 0x7f14740759f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f146000e9c0 tx=0x7f146000ee80 comp rx=0 tx=0).stop 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 -- 172.21.15.32:0/1608987211 shutdown_connections 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f145c0b4d20 0x7f145c0b7230 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1474109bb0 0x7f1474075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1474109260 0x7f14740759f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 --2- 172.21.15.32:0/1608987211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1474103920 0x7f14740754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 -- 172.21.15.32:0/1608987211 >> 172.21.15.32:0/1608987211 conn(0x7f14740ff370 msgr2=0x7f1474101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 -- 172.21.15.32:0/1608987211 shutdown_connections 2024-08-31T07:13:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:13:58.063+0000 7f147bc00640 1 -- 172.21.15.32:0/1608987211 wait complete. 2024-08-31T07:13:58.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:13:58 smithi032 ceph-mon[17252]: pgmap v404: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:58.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:13:58 smithi073 ceph-mon[21995]: pgmap v404: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:58.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:13:58 smithi084 ceph-mon[22387]: pgmap v404: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-31T07:13:58.963 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:13:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:00.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.035+0000 7faf39e00640 1 -- 172.21.15.32:0/1479835362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf34105b20 msgr2=0x7faf34105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:00.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.035+0000 7faf39e00640 1 --2- 172.21.15.32:0/1479835362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf34105b20 0x7faf34105f30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7faf24009900 tx=0x7faf2402a410 comp rx=0 tx=0).stop 2024-08-31T07:14:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.035+0000 7faf39e00640 1 -- 172.21.15.32:0/1479835362 shutdown_connections 2024-08-31T07:14:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.035+0000 7faf39e00640 1 --2- 172.21.15.32:0/1479835362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3410bdb0 0x7faf341125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.035+0000 7faf39e00640 1 --2- 172.21.15.32:0/1479835362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf3410b460 0x7faf3410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.035+0000 7faf39e00640 1 --2- 172.21.15.32:0/1479835362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf34105b20 0x7faf34105f30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.035+0000 7faf39e00640 1 -- 172.21.15.32:0/1479835362 >> 172.21.15.32:0/1479835362 conn(0x7faf341015b0 msgr2=0x7faf34103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.035+0000 7faf39e00640 1 -- 172.21.15.32:0/1479835362 shutdown_connections 2024-08-31T07:14:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.035+0000 7faf39e00640 1 -- 172.21.15.32:0/1479835362 wait complete. 2024-08-31T07:14:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.036+0000 7faf39e00640 1 Processor -- start 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.036+0000 7faf39e00640 1 -- start start 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf39e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf34105b20 0x7faf3419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf39e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3410b460 0x7faf3419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf39e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf3410bdb0 0x7faf341a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf39e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf34115ac0 con 0x7faf34105b20 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf39e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf34115940 con 0x7faf3410b460 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf39e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf34115c40 con 0x7faf3410bdb0 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf32a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf34105b20 0x7faf3419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf32a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf34105b20 0x7faf3419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60346/0 (socket says 172.21.15.32:60346) 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf32a00640 1 -- 172.21.15.32:0/2026280561 learned_addr learned my addr 172.21.15.32:0/2026280561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf2be00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3410b460 0x7faf3419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.037+0000 7faf33400640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf3410bdb0 0x7faf341a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.038+0000 7faf32a00640 1 -- 172.21.15.32:0/2026280561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf3410bdb0 msgr2=0x7faf341a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.038+0000 7faf32a00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf3410bdb0 0x7faf341a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.038+0000 7faf32a00640 1 -- 172.21.15.32:0/2026280561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3410b460 msgr2=0x7faf3419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.038+0000 7faf32a00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3410b460 0x7faf3419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.038+0000 7faf32a00640 1 -- 172.21.15.32:0/2026280561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf341a2440 con 0x7faf34105b20 2024-08-31T07:14:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.038+0000 7faf32a00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf34105b20 0x7faf3419c4d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faf24005ab0 tx=0x7faf24003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.038+0000 7faf38a00640 1 -- 172.21.15.32:0/2026280561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2403f070 con 0x7faf34105b20 2024-08-31T07:14:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.038+0000 7faf39e00640 1 -- 172.21.15.32:0/2026280561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf341a26d0 con 0x7faf34105b20 2024-08-31T07:14:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.038+0000 7faf38a00640 1 -- 172.21.15.32:0/2026280561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf24031d60 con 0x7faf34105b20 2024-08-31T07:14:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.039+0000 7faf38a00640 1 -- 172.21.15.32:0/2026280561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24049060 con 0x7faf34105b20 2024-08-31T07:14:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.039+0000 7faf39e00640 1 -- 172.21.15.32:0/2026280561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf341a8e90 con 0x7faf34105b20 2024-08-31T07:14:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.042+0000 7faf40a3c740 1 Processor -- start 2024-08-31T07:14:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.042+0000 7faf38a00640 1 -- 172.21.15.32:0/2026280561 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7faf24042070 con 0x7faf34105b20 2024-08-31T07:14:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.042+0000 7faf40a3c740 1 -- start start 2024-08-31T07:14:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.042+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b5f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.042+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.042+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b231390 0x55746b5f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.042+0000 7faf40a3c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55746b56e7b0 con 0x55746b2291b0 2024-08-31T07:14:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.042+0000 7faf40a3c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55746b55dd40 con 0x55746b231390 2024-08-31T07:14:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.042+0000 7faf40a3c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55746b561cf0 con 0x55746b252070 2024-08-31T07:14:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf32a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf33400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b5f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf33400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b5f7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46576/0 (socket says 172.21.15.32:46576) 2024-08-31T07:14:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf33400640 1 -- 172.21.15.32:0/1278897237 learned_addr learned my addr 172.21.15.32:0/1278897237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf2be00640 1 --2- 172.21.15.32:0/1278897237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b231390 0x55746b5f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf32a00640 1 -- 172.21.15.32:0/1278897237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 msgr2=0x55746b5f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf32a00640 1 --2- 172.21.15.32:0/1278897237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b5f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf32a00640 1 -- 172.21.15.32:0/1278897237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b231390 msgr2=0x55746b5f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf32a00640 1 --2- 172.21.15.32:0/1278897237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b231390 0x55746b5f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf32a00640 1 -- 172.21.15.32:0/1278897237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55746b2bd650 con 0x55746b2291b0 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf33400640 1 --2- 172.21.15.32:0/1278897237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b5f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.043+0000 7faf2be00640 1 --2- 172.21.15.32:0/1278897237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b231390 0x55746b5f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.044+0000 7faf38a00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf180b4ca0 0x7faf180b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.044+0000 7faf32a00640 1 --2- 172.21.15.32:0/1278897237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5f7ed0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faf241037b0 tx=0x7faf2410e320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee80223882a204c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.044+0000 7faf38a00640 1 -- 172.21.15.32:0/2026280561 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7faf24105fe0 con 0x7faf34105b20 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.044+0000 7faf33400640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf180b4ca0 0x7faf180b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.044+0000 7faf23e00640 1 -- 172.21.15.32:0/1278897237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240c86c0 con 0x55746b2291b0 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.044+0000 7faf23e00640 1 -- 172.21.15.32:0/1278897237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf24113030 con 0x55746b2291b0 2024-08-31T07:14:00.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.044+0000 7faf23e00640 1 -- 172.21.15.32:0/1278897237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240cbf70 con 0x55746b2291b0 2024-08-31T07:14:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.045+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278897237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 msgr2=0x55746b5f7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.045+0000 7faf33400640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf180b4ca0 0x7faf180b71b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faf2c005f40 tx=0x7faf2c0096a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.045+0000 7faf40a3c740 1 --2- 172.21.15.32:0/1278897237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5f7ed0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faf241037b0 tx=0x7faf2410e320 comp rx=0 tx=0).stop 2024-08-31T07:14:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.045+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278897237 shutdown_connections 2024-08-31T07:14:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.045+0000 7faf40a3c740 1 --2- 172.21.15.32:0/1278897237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b231390 0x55746b5f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.045+0000 7faf40a3c740 1 --2- 172.21.15.32:0/1278897237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5f7ed0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.045+0000 7faf40a3c740 1 --2- 172.21.15.32:0/1278897237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b5f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.045+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278897237 >> 172.21.15.32:0/1278897237 conn(0x55746b246460 msgr2=0x55746b5f5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.046+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278897237 shutdown_connections 2024-08-31T07:14:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.046+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278897237 wait complete. 2024-08-31T07:14:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.046+0000 7faf40a3c740 1 -- start start 2024-08-31T07:14:00.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.046+0000 7faf40a3c740 1 Processor -- start 2024-08-31T07:14:00.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 0x55746b6035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b603b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b60f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf40a3c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55746b3c0700 con 0x55746b231390 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf40a3c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55746b4e4cf0 con 0x55746b2291b0 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf40a3c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55746b605f30 con 0x55746b252070 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf40a3c740 10 client.0 _ll_register_callbacks cb 0x55746b50a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf2be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b603b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf2be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b603b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60352/0 (socket says 172.21.15.32:60352) 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf2be00640 1 -- 172.21.15.32:0/2661265381 learned_addr learned my addr 172.21.15.32:0/2661265381 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf32a00640 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 0x55746b6035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.047+0000 7faf33400640 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b60f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.048+0000 7faf2be00640 1 -- 172.21.15.32:0/2661265381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 msgr2=0x55746b60f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.048+0000 7faf2be00640 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b60f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.048+0000 7faf2be00640 1 -- 172.21.15.32:0/2661265381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 msgr2=0x55746b6035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.048+0000 7faf2be00640 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 0x55746b6035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.048+0000 7faf2be00640 1 -- 172.21.15.32:0/2661265381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55746b5258b0 con 0x55746b231390 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.048+0000 7faf32a00640 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 0x55746b6035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.048+0000 7faf2be00640 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b603b20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faf1c012700 tx=0x7faf1c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91768b3035802ca0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.048+0000 7faf20c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.048+0000 7faf20c00640 1 -- 172.21.15.32:0/2661265381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c004050 con 0x55746b231390 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.049+0000 7faf20c00640 1 -- 172.21.15.32:0/2661265381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7faf1c0041f0 con 0x55746b231390 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.049+0000 7faf40a3c740 1 -- 172.21.15.32:0/2661265381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf24113030 con 0x55746b231390 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.049+0000 7faf20c00640 1 -- 172.21.15.32:0/2661265381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faf1c004410 con 0x55746b231390 2024-08-31T07:14:00.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.049+0000 7faf40a3c740 10 client.15846 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.049+0000 7faf40a3c740 1 -- 172.21.15.32:0/2661265381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55746b611f60 con 0x55746b231390 2024-08-31T07:14:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.049+0000 7faf20c00640 1 -- 172.21.15.32:0/2661265381 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf24113030 con 0x55746b231390 2024-08-31T07:14:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.049+0000 7faf20c00640 1 -- 172.21.15.32:0/2661265381 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faf1c003c40 con 0x55746b231390 2024-08-31T07:14:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.049+0000 7faf40a3c740 10 client.15846 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.049+0000 7faf40a3c740 10 client.15846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:14:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.050+0000 7faf40a3c740 20 client.15846 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.050+0000 7faf40a3c740 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55746b2109f0 0x55746b6153f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.050+0000 7faf40a3c740 4 client.15846 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.050+0000 7faf40a3c740 1 -- 172.21.15.32:0/2661265381 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55746b570700 con 0x55746b2109f0 2024-08-31T07:14:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.050+0000 7faf32a00640 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55746b2109f0 0x55746b6153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.050+0000 7faf32a00640 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55746b2109f0 0x55746b6153f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7418092cdba66d56 server_cookie=7affbe786c667ebe in_seq=0 out_seq=0 2024-08-31T07:14:00.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.050+0000 7faf20c00640 10 client.15846 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:14:00.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.051+0000 7faf20c00640 1 -- 172.21.15.32:0/2661265381 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55746b570700 con 0x55746b2109f0 2024-08-31T07:14:00.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.051+0000 7faf20c00640 10 client.15846 handle_command_reply: tid=0 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.051+0000 7faf40a3c740 1 client.15846 shutdown 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.051+0000 7faf40a3c740 1 -- 172.21.15.32:0/2661265381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 msgr2=0x55746b603b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.051+0000 7faf40a3c740 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b603b20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faf1c012700 tx=0x7faf1c012bc0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.052+0000 7faf40a3c740 1 -- 172.21.15.32:0/2661265381 shutdown_connections 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.052+0000 7faf40a3c740 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55746b2109f0 0x55746b6153f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.052+0000 7faf40a3c740 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b60f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.052+0000 7faf40a3c740 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b603b20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.052+0000 7faf20c00640 0 client.15846 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.052+0000 7faf40a3c740 1 --2- 172.21.15.32:0/2661265381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 0x55746b6035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.052+0000 7faf40a3c740 1 -- 172.21.15.32:0/2661265381 >> 172.21.15.32:0/2661265381 conn(0x55746b246460 msgr2=0x55746b5fab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.052+0000 7faf40a3c740 1 -- 172.21.15.32:0/2661265381 shutdown_connections 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.052+0000 7faf40a3c740 1 -- 172.21.15.32:0/2661265381 wait complete. 2024-08-31T07:14:00.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.052+0000 7faf40a3c740 20 client.15846 trim_cache size 0 max 16384 2024-08-31T07:14:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.066+0000 7faf40a3c740 1 Processor -- start 2024-08-31T07:14:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.066+0000 7faf40a3c740 1 -- start start 2024-08-31T07:14:00.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b231390 0x55746b5fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b252070 0x55746b5f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf40a3c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b1550 con 0x55746b2291b0 2024-08-31T07:14:00.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf40a3c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b3090 con 0x55746b252070 2024-08-31T07:14:00.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf40a3c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b1c20 con 0x55746b231390 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf32a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf32a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5fcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60364/0 (socket says 172.21.15.32:60364) 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf32a00640 1 -- 172.21.15.32:0/3905074169 learned_addr learned my addr 172.21.15.32:0/3905074169 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf2be00640 1 --2- 172.21.15.32:0/3905074169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b231390 0x55746b5fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf33400640 1 --2- 172.21.15.32:0/3905074169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b252070 0x55746b5f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf32a00640 1 -- 172.21.15.32:0/3905074169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b231390 msgr2=0x55746b5fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf32a00640 1 --2- 172.21.15.32:0/3905074169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b231390 0x55746b5fcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf32a00640 1 -- 172.21.15.32:0/3905074169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b252070 msgr2=0x55746b5f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf32a00640 1 --2- 172.21.15.32:0/3905074169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b252070 0x55746b5f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf32a00640 1 -- 172.21.15.32:0/3905074169 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55746b5efa80 con 0x55746b2291b0 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.067+0000 7faf33400640 1 --2- 172.21.15.32:0/3905074169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b252070 0x55746b5f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:14:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.068+0000 7faf32a00640 1 --2- 172.21.15.32:0/3905074169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5fcaa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faf24103860 tx=0x7faf240c8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2219abfeff2150 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.068+0000 7faf23400640 1 -- 172.21.15.32:0/3905074169 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240cc150 con 0x55746b2291b0 2024-08-31T07:14:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.068+0000 7faf23400640 1 -- 172.21.15.32:0/3905074169 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf240cc310 con 0x55746b2291b0 2024-08-31T07:14:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.068+0000 7faf23400640 1 -- 172.21.15.32:0/3905074169 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240cc5d0 con 0x55746b2291b0 2024-08-31T07:14:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 -- 172.21.15.32:0/3905074169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 msgr2=0x55746b5fcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 --2- 172.21.15.32:0/3905074169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5fcaa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faf24103860 tx=0x7faf240c8040 comp rx=0 tx=0).stop 2024-08-31T07:14:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 -- 172.21.15.32:0/3905074169 shutdown_connections 2024-08-31T07:14:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 --2- 172.21.15.32:0/3905074169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b252070 0x55746b5f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 --2- 172.21.15.32:0/3905074169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b231390 0x55746b5fcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 --2- 172.21.15.32:0/3905074169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b2291b0 0x55746b5fcaa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 -- 172.21.15.32:0/3905074169 >> 172.21.15.32:0/3905074169 conn(0x55746b246460 msgr2=0x55746b601c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 -- 172.21.15.32:0/3905074169 shutdown_connections 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 -- 172.21.15.32:0/3905074169 wait complete. 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 -- start start 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.069+0000 7faf40a3c740 1 Processor -- start 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 0x55746b60b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b607b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf40a3c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b6080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf40a3c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b29c0 con 0x55746b231390 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf40a3c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b22f0 con 0x55746b2291b0 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf40a3c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf341af530 con 0x55746b252070 2024-08-31T07:14:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf40a3c740 10 client.0 _ll_register_callbacks cb 0x7faf1c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:00.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf2be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b607b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf32a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 0x55746b60b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf2be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b607b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60374/0 (socket says 172.21.15.32:60374) 2024-08-31T07:14:00.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf2be00640 1 -- 172.21.15.32:0/1278789515 learned_addr learned my addr 172.21.15.32:0/1278789515 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:00.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.070+0000 7faf33400640 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b6080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.071+0000 7faf32a00640 1 -- 172.21.15.32:0/1278789515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 msgr2=0x55746b6080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:00.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.071+0000 7faf32a00640 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b6080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.071+0000 7faf32a00640 1 -- 172.21.15.32:0/1278789515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 msgr2=0x55746b607b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:00.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.071+0000 7faf32a00640 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b607b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.071+0000 7faf32a00640 1 -- 172.21.15.32:0/1278789515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55746b6085f0 con 0x55746b2291b0 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.071+0000 7faf2be00640 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b607b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.071+0000 7faf32a00640 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 0x55746b60b610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faf24000c00 tx=0x7faf2403b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c714182ad4357b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.071+0000 7faf20c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.071+0000 7faf20c00640 1 -- 172.21.15.32:0/1278789515 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240cc5d0 con 0x55746b2291b0 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf20c00640 1 -- 172.21.15.32:0/1278789515 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7faf24003df0 con 0x55746b2291b0 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278789515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf240cc310 con 0x55746b2291b0 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf20c00640 1 -- 172.21.15.32:0/1278789515 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faf240c86c0 con 0x55746b2291b0 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf40a3c740 10 client.25862 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278789515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55746b680ab0 con 0x55746b2291b0 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf20c00640 1 -- 172.21.15.32:0/1278789515 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf240cc310 con 0x55746b2291b0 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf20c00640 1 -- 172.21.15.32:0/1278789515 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faf24103a30 con 0x55746b2291b0 2024-08-31T07:14:00.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf40a3c740 10 client.25862 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf40a3c740 10 client.25862 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf40a3c740 20 client.25862 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf40a3c740 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55746b1e5da0 0x55746b682f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf40a3c740 4 client.25862 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.072+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278789515 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55746b56c1a0 con 0x55746b1e5da0 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.073+0000 7faf2be00640 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55746b1e5da0 0x55746b682f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.073+0000 7faf2be00640 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55746b1e5da0 0x55746b682f00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abf49a6fdfc0bf3 server_cookie=ea1b4ade23370071 in_seq=0 out_seq=0 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.073+0000 7faf20c00640 10 client.25862 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.073+0000 7faf20c00640 1 -- 172.21.15.32:0/1278789515 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55746b56c1a0 con 0x55746b1e5da0 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf20c00640 10 client.25862 handle_command_reply: tid=0 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 client.25862 shutdown 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278789515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 msgr2=0x55746b60b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 0x55746b60b610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faf24000c00 tx=0x7faf2403b460 comp rx=0 tx=0).stop 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278789515 shutdown_connections 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55746b1e5da0 0x55746b682f00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746b252070 0x55746b6080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746b231390 0x55746b607b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 --2- 172.21.15.32:0/1278789515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55746b2291b0 0x55746b60b610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf20c00640 0 client.25862 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:14:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278789515 >> 172.21.15.32:0/1278789515 conn(0x55746b246460 msgr2=0x55746b601a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278789515 shutdown_connections 2024-08-31T07:14:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 1 -- 172.21.15.32:0/1278789515 wait complete. 2024-08-31T07:14:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.074+0000 7faf40a3c740 20 client.25862 trim_cache size 0 max 16384 2024-08-31T07:14:00.079 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:14:00.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 -- 172.21.15.32:0/2026280561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf180b4ca0 msgr2=0x7faf180b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:00.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf180b4ca0 0x7faf180b71b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faf2c005f40 tx=0x7faf2c0096a0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 -- 172.21.15.32:0/2026280561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf34105b20 msgr2=0x7faf3419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:00.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf34105b20 0x7faf3419c4d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faf24005ab0 tx=0x7faf24003c80 comp rx=0 tx=0).stop 2024-08-31T07:14:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 -- 172.21.15.32:0/2026280561 shutdown_connections 2024-08-31T07:14:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf180b4ca0 0x7faf180b71b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf3410bdb0 0x7faf341a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3410b460 0x7faf3419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 --2- 172.21.15.32:0/2026280561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf34105b20 0x7faf3419c4d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 -- 172.21.15.32:0/2026280561 >> 172.21.15.32:0/2026280561 conn(0x7faf341015b0 msgr2=0x7faf341039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 -- 172.21.15.32:0/2026280561 shutdown_connections 2024-08-31T07:14:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:00.075+0000 7faf39e00640 1 -- 172.21.15.32:0/2026280561 wait complete. 2024-08-31T07:14:00.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:00 smithi032 ceph-mon[17252]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:14:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:00 smithi073 ceph-mon[21995]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:14:00.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:00 smithi084 ceph-mon[22387]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-31T07:14:01.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:14:01.445 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:02.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.160+0000 7f9307200640 1 -- 172.21.15.32:0/2049798672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93081099c0 msgr2=0x7f93081101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.160+0000 7f9307200640 1 --2- 172.21.15.32:0/2049798672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93081099c0 0x7f93081101f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9300009ca0 tx=0x7f930002a8b0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.160+0000 7f9307200640 1 -- 172.21.15.32:0/2049798672 shutdown_connections 2024-08-31T07:14:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.160+0000 7f9307200640 1 --2- 172.21.15.32:0/2049798672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93081099c0 0x7f93081101f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.160+0000 7f9307200640 1 --2- 172.21.15.32:0/2049798672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9308109070 0x7f9308109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.160+0000 7f9307200640 1 --2- 172.21.15.32:0/2049798672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9308103730 0x7f9308103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.160+0000 7f9307200640 1 -- 172.21.15.32:0/2049798672 >> 172.21.15.32:0/2049798672 conn(0x7f93080ff180 msgr2=0x7f93081015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.160+0000 7f9307200640 1 -- 172.21.15.32:0/2049798672 shutdown_connections 2024-08-31T07:14:02.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.160+0000 7f9307200640 1 -- 172.21.15.32:0/2049798672 wait complete. 2024-08-31T07:14:02.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.161+0000 7f9307200640 1 Processor -- start 2024-08-31T07:14:02.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.161+0000 7f9307200640 1 -- start start 2024-08-31T07:14:02.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f9307200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9308103730 0x7f930819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f9307200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9308109070 0x7f930819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f9307200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93081099c0 0x7f930819f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f9307200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93081136c0 con 0x7f93081099c0 2024-08-31T07:14:02.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f9307200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9308113540 con 0x7f9308103730 2024-08-31T07:14:02.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f9307200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9308113840 con 0x7f9308109070 2024-08-31T07:14:02.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f9306800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93081099c0 0x7f930819f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f9306800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93081099c0 0x7f930819f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60396/0 (socket says 172.21.15.32:60396) 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f92fd400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9308109070 0x7f930819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f9306800640 1 -- 172.21.15.32:0/6877372 learned_addr learned my addr 172.21.15.32:0/6877372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.162+0000 7f9305e00640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9308103730 0x7f930819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.163+0000 7f9305e00640 1 -- 172.21.15.32:0/6877372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9308109070 msgr2=0x7f930819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.163+0000 7f9305e00640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9308109070 0x7f930819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.163+0000 7f9305e00640 1 -- 172.21.15.32:0/6877372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93081099c0 msgr2=0x7f930819f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.163+0000 7f9305e00640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93081099c0 0x7f930819f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.163+0000 7f9305e00640 1 -- 172.21.15.32:0/6877372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93081a0020 con 0x7f9308103730 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.163+0000 7f9305e00640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9308103730 0x7f930819a0b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f92f800d6d0 tx=0x7f92f800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:02.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.163+0000 7f92ff400640 1 -- 172.21.15.32:0/6877372 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8017070 con 0x7f9308103730 2024-08-31T07:14:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.164+0000 7f9307200640 1 -- 172.21.15.32:0/6877372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93081a0310 con 0x7f9308103730 2024-08-31T07:14:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.164+0000 7f92ff400640 1 -- 172.21.15.32:0/6877372 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f8009820 con 0x7f9308103730 2024-08-31T07:14:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.164+0000 7f92ff400640 1 -- 172.21.15.32:0/6877372 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8004bc0 con 0x7f9308103730 2024-08-31T07:14:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.164+0000 7f9307200640 1 -- 172.21.15.32:0/6877372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93081a6a70 con 0x7f9308103730 2024-08-31T07:14:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.167+0000 7f9313ab2740 1 Processor -- start 2024-08-31T07:14:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.167+0000 7f9313ab2740 1 -- start start 2024-08-31T07:14:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.167+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 0x557755465b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.167+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 0x5577554660a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.167+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 0x5577554665e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.167+0000 7f9313ab2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5577553dc7b0 con 0x5577550c0070 2024-08-31T07:14:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.167+0000 7f9313ab2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5577553cbd40 con 0x5577550971b0 2024-08-31T07:14:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.167+0000 7f9313ab2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5577553cfcf0 con 0x55775509f390 2024-08-31T07:14:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.167+0000 7f92ff400640 1 -- 172.21.15.32:0/6877372 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f92f8005220 con 0x7f9308103730 2024-08-31T07:14:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f9306800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 0x557755465b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f9306800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 0x557755465b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60404/0 (socket says 172.21.15.32:60404) 2024-08-31T07:14:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f9306800640 1 -- 172.21.15.32:0/720835763 learned_addr learned my addr 172.21.15.32:0/720835763 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f92fd400640 1 --2- 172.21.15.32:0/720835763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 0x5577554665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f9305e00640 1 --2- 172.21.15.32:0/720835763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 0x5577554660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f9306800640 1 -- 172.21.15.32:0/720835763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 msgr2=0x5577554665e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f9306800640 1 --2- 172.21.15.32:0/720835763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 0x5577554665e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f9306800640 1 -- 172.21.15.32:0/720835763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 msgr2=0x5577554660a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f9306800640 1 --2- 172.21.15.32:0/720835763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 0x5577554660a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f9306800640 1 -- 172.21.15.32:0/720835763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775512b650 con 0x5577550c0070 2024-08-31T07:14:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.168+0000 7f9305e00640 1 --2- 172.21.15.32:0/720835763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 0x5577554660a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:14:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.169+0000 7f9306800640 1 --2- 172.21.15.32:0/720835763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 0x557755465b60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f93000099f0 tx=0x7f9300009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25eec3692503e18f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.169+0000 7f92f7400640 1 -- 172.21.15.32:0/720835763 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300030800 con 0x5577550c0070 2024-08-31T07:14:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.169+0000 7f92f7400640 1 -- 172.21.15.32:0/720835763 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9300036030 con 0x5577550c0070 2024-08-31T07:14:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.169+0000 7f92f7400640 1 -- 172.21.15.32:0/720835763 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300004500 con 0x5577550c0070 2024-08-31T07:14:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f92ff400640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f92ec0b4ca0 0x7f92ec0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f92ff400640 1 -- 172.21.15.32:0/6877372 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f92f80e5090 con 0x7f9308103730 2024-08-31T07:14:02.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f9305e00640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f92ec0b4ca0 0x7f92ec0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f9313ab2740 1 -- 172.21.15.32:0/720835763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 msgr2=0x557755465b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:02.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f9313ab2740 1 --2- 172.21.15.32:0/720835763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 0x557755465b60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f93000099f0 tx=0x7f9300009a20 comp rx=0 tx=0).stop 2024-08-31T07:14:02.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f9313ab2740 1 -- 172.21.15.32:0/720835763 shutdown_connections 2024-08-31T07:14:02.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f9313ab2740 1 --2- 172.21.15.32:0/720835763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 0x5577554665e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f9313ab2740 1 --2- 172.21.15.32:0/720835763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 0x5577554660a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f9313ab2740 1 --2- 172.21.15.32:0/720835763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 0x557755465b60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f9313ab2740 1 -- 172.21.15.32:0/720835763 >> 172.21.15.32:0/720835763 conn(0x5577550b4460 msgr2=0x557755463a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:02.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f9313ab2740 1 -- 172.21.15.32:0/720835763 shutdown_connections 2024-08-31T07:14:02.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.170+0000 7f9313ab2740 1 -- 172.21.15.32:0/720835763 wait complete. 2024-08-31T07:14:02.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.171+0000 7f9305e00640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f92ec0b4ca0 0x7f92ec0b71b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5577554647e0 tx=0x7f92f80a7040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:02.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.171+0000 7f9313ab2740 1 -- start start 2024-08-31T07:14:02.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.171+0000 7f9313ab2740 1 Processor -- start 2024-08-31T07:14:02.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 0x557755475b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 0x5577554760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 0x557755481970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9313ab2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55775522e700 con 0x5577550c0070 2024-08-31T07:14:02.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9313ab2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557755352cf0 con 0x5577550971b0 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9313ab2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557755478640 con 0x55775509f390 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9313ab2740 10 client.0 _ll_register_callbacks cb 0x557755378470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9306800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 0x557755475b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9306800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 0x557755475b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57236/0 (socket says 172.21.15.32:57236) 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9306800640 1 -- 172.21.15.32:0/896973165 learned_addr learned my addr 172.21.15.32:0/896973165 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f92fd400640 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 0x5577554760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.172+0000 7f9306800640 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 0x557755481970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.173+0000 7f92fd400640 1 -- 172.21.15.32:0/896973165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 msgr2=0x557755475b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.173+0000 7f92fd400640 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 0x557755475b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.173+0000 7f92fd400640 1 -- 172.21.15.32:0/896973165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 msgr2=0x557755481970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:02.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.173+0000 7f92fd400640 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 0x557755481970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.173+0000 7f92fd400640 1 -- 172.21.15.32:0/896973165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577553938b0 con 0x55775509f390 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.173+0000 7f92fd400640 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 0x5577554760e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f930806c340 tx=0x7f92f000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75192a11b9cf1f90 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.173+0000 7f92ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.173+0000 7f92ebe00640 1 -- 172.21.15.32:0/896973165 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0003d70 con 0x55775509f390 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.173+0000 7f92ebe00640 1 -- 172.21.15.32:0/896973165 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f92f0005020 con 0x55775509f390 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.173+0000 7f9313ab2740 1 -- 172.21.15.32:0/896973165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9300036030 con 0x55775509f390 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.174+0000 7f92ebe00640 1 -- 172.21.15.32:0/896973165 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92f0005220 con 0x55775509f390 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.174+0000 7f9313ab2740 10 client.25876 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.174+0000 7f9313ab2740 1 -- 172.21.15.32:0/896973165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577554846e0 con 0x55775509f390 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.174+0000 7f92ebe00640 1 -- 172.21.15.32:0/896973165 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9300036030 con 0x55775509f390 2024-08-31T07:14:02.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.174+0000 7f92ebe00640 1 -- 172.21.15.32:0/896973165 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92f001b470 con 0x55775509f390 2024-08-31T07:14:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.175+0000 7f9313ab2740 10 client.25876 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.175+0000 7f9313ab2740 10 client.25876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:14:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.175+0000 7f9313ab2740 20 client.25876 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.175+0000 7f9313ab2740 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55775507e9f0 0x557755487bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.175+0000 7f9313ab2740 4 client.25876 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.175+0000 7f9313ab2740 1 -- 172.21.15.32:0/896973165 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577553de700 con 0x55775507e9f0 2024-08-31T07:14:02.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.175+0000 7f9306800640 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55775507e9f0 0x557755487bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.175+0000 7f9306800640 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55775507e9f0 0x557755487bc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36c017509f75aa3a server_cookie=377fa4e8dc49a294 in_seq=0 out_seq=0 2024-08-31T07:14:02.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.175+0000 7f92ebe00640 10 client.25876 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.176+0000 7f92ebe00640 1 -- 172.21.15.32:0/896973165 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577553de700 con 0x55775507e9f0 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.176+0000 7f92ebe00640 10 client.25876 handle_command_reply: tid=0 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.176+0000 7f9313ab2740 1 client.25876 shutdown 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.176+0000 7f9313ab2740 1 -- 172.21.15.32:0/896973165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 msgr2=0x5577554760e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f9313ab2740 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 0x5577554760e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f930806c340 tx=0x7f92f000be50 comp rx=0 tx=0).stop 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f9313ab2740 1 -- 172.21.15.32:0/896973165 shutdown_connections 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f9313ab2740 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550c0070 0x557755481970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f9313ab2740 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55775509f390 0x5577554760e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f9313ab2740 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55775507e9f0 0x557755487bc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f9313ab2740 1 --2- 172.21.15.32:0/896973165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550971b0 0x557755475b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f92ebe00640 0 client.25876 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:14:02.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f9313ab2740 1 -- 172.21.15.32:0/896973165 >> 172.21.15.32:0/896973165 conn(0x5577550b4460 msgr2=0x557755461ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:02.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f9313ab2740 1 -- 172.21.15.32:0/896973165 shutdown_connections 2024-08-31T07:14:02.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f9313ab2740 1 -- 172.21.15.32:0/896973165 wait complete. 2024-08-31T07:14:02.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.177+0000 7f9313ab2740 20 client.25876 trim_cache size 0 max 16384 2024-08-31T07:14:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.191+0000 7f9313ab2740 1 Processor -- start 2024-08-31T07:14:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.191+0000 7f9313ab2740 1 -- start start 2024-08-31T07:14:02.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550971b0 0x5577554672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775509f390 0x557755481870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550c0070 0x557755481db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9313ab2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93081aceb0 con 0x5577550971b0 2024-08-31T07:14:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9313ab2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93081ae9f0 con 0x55775509f390 2024-08-31T07:14:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9313ab2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93081ad580 con 0x5577550c0070 2024-08-31T07:14:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9306800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550971b0 0x5577554672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9306800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550971b0 0x5577554672a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60412/0 (socket says 172.21.15.32:60412) 2024-08-31T07:14:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9306800640 1 -- 172.21.15.32:0/3927560888 learned_addr learned my addr 172.21.15.32:0/3927560888 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f92fd400640 1 --2- 172.21.15.32:0/3927560888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775509f390 0x557755481870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9306800640 1 --2- 172.21.15.32:0/3927560888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550c0070 0x557755481db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9306800640 1 -- 172.21.15.32:0/3927560888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550c0070 msgr2=0x557755481db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9306800640 1 --2- 172.21.15.32:0/3927560888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550c0070 0x557755481db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9306800640 1 -- 172.21.15.32:0/3927560888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775509f390 msgr2=0x557755481870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9306800640 1 --2- 172.21.15.32:0/3927560888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775509f390 0x557755481870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.192+0000 7f9306800640 1 -- 172.21.15.32:0/3927560888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557755468470 con 0x5577550971b0 2024-08-31T07:14:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.193+0000 7f9306800640 1 --2- 172.21.15.32:0/3927560888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550971b0 0x5577554672a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9300009620 tx=0x7f930002a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd39258cf0d3d84 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.193+0000 7f92f6a00640 1 -- 172.21.15.32:0/3927560888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300060020 con 0x5577550971b0 2024-08-31T07:14:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.193+0000 7f92f6a00640 1 -- 172.21.15.32:0/3927560888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9300036040 con 0x5577550971b0 2024-08-31T07:14:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.193+0000 7f92f6a00640 1 -- 172.21.15.32:0/3927560888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300004530 con 0x5577550971b0 2024-08-31T07:14:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.194+0000 7f9313ab2740 1 -- 172.21.15.32:0/3927560888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550971b0 msgr2=0x5577554672a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.194+0000 7f9313ab2740 1 --2- 172.21.15.32:0/3927560888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550971b0 0x5577554672a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9300009620 tx=0x7f930002a8b0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.194+0000 7f9313ab2740 1 -- 172.21.15.32:0/3927560888 shutdown_connections 2024-08-31T07:14:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.194+0000 7f9313ab2740 1 --2- 172.21.15.32:0/3927560888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550c0070 0x557755481db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.194+0000 7f9313ab2740 1 --2- 172.21.15.32:0/3927560888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775509f390 0x557755481870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.194+0000 7f9313ab2740 1 --2- 172.21.15.32:0/3927560888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577550971b0 0x5577554672a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.194+0000 7f9313ab2740 1 -- 172.21.15.32:0/3927560888 >> 172.21.15.32:0/3927560888 conn(0x5577550b4460 msgr2=0x557755466b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.194+0000 7f9313ab2740 1 -- 172.21.15.32:0/3927560888 shutdown_connections 2024-08-31T07:14:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.194+0000 7f9313ab2740 1 -- 172.21.15.32:0/3927560888 wait complete. 2024-08-31T07:14:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.195+0000 7f9313ab2740 1 -- start start 2024-08-31T07:14:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.195+0000 7f9313ab2740 1 Processor -- start 2024-08-31T07:14:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.195+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550971b0 0x55775547a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.195+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55775509f390 0x55775547a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.195+0000 7f9313ab2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550c0070 0x5577554eb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.195+0000 7f9313ab2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93081ae320 con 0x55775509f390 2024-08-31T07:14:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.195+0000 7f9313ab2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93081adc50 con 0x5577550c0070 2024-08-31T07:14:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.195+0000 7f9313ab2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93081aae90 con 0x5577550971b0 2024-08-31T07:14:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.195+0000 7f9313ab2740 10 client.0 _ll_register_callbacks cb 0x7f92f0004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f9306800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550971b0 0x55775547a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f9306800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550971b0 0x55775547a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46650/0 (socket says 172.21.15.32:46650) 2024-08-31T07:14:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f9306800640 1 -- 172.21.15.32:0/4141666087 learned_addr learned my addr 172.21.15.32:0/4141666087 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f92fd400640 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55775509f390 0x55775547a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f9306800640 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550c0070 0x5577554eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f9306800640 1 -- 172.21.15.32:0/4141666087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550c0070 msgr2=0x5577554eb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f9306800640 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550c0070 0x5577554eb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f9306800640 1 -- 172.21.15.32:0/4141666087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55775509f390 msgr2=0x55775547a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f9306800640 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55775509f390 0x55775547a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f9306800640 1 -- 172.21.15.32:0/4141666087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577554ebc80 con 0x5577550971b0 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f9306800640 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550971b0 0x55775547a230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f93000039d0 tx=0x7f9300030610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=920ccd94c4e90d6d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.196+0000 7f92ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.197+0000 7f92ebe00640 1 -- 172.21.15.32:0/4141666087 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300060070 con 0x5577550971b0 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.197+0000 7f92ebe00640 1 -- 172.21.15.32:0/4141666087 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f9300030a10 con 0x5577550971b0 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.197+0000 7f9313ab2740 1 -- 172.21.15.32:0/4141666087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9300036040 con 0x5577550971b0 2024-08-31T07:14:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.197+0000 7f92ebe00640 1 -- 172.21.15.32:0/4141666087 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9300030c10 con 0x5577550971b0 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.197+0000 7f9313ab2740 10 client.25882 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.197+0000 7f9313ab2740 1 -- 172.21.15.32:0/4141666087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577554f2fc0 con 0x5577550971b0 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.198+0000 7f92ebe00640 1 -- 172.21.15.32:0/4141666087 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9300036040 con 0x5577550971b0 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.198+0000 7f92ebe00640 1 -- 172.21.15.32:0/4141666087 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9300043070 con 0x5577550971b0 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.198+0000 7f9313ab2740 10 client.25882 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.198+0000 7f9313ab2740 10 client.25882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.198+0000 7f9313ab2740 20 client.25882 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.198+0000 7f9313ab2740 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x557755053da0 0x5577554f5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.198+0000 7f9313ab2740 4 client.25882 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.198+0000 7f9313ab2740 1 -- 172.21.15.32:0/4141666087 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577553da1a0 con 0x557755053da0 2024-08-31T07:14:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.198+0000 7f92fd400640 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x557755053da0 0x5577554f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f92fd400640 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x557755053da0 0x5577554f5290 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df4c801931ad8969 server_cookie=db3561722e4f33f1 in_seq=0 out_seq=0 2024-08-31T07:14:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f92ebe00640 10 client.25882 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:14:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f92ebe00640 1 -- 172.21.15.32:0/4141666087 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577553da1a0 con 0x557755053da0 2024-08-31T07:14:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f92ebe00640 10 client.25882 handle_command_reply: tid=0 2024-08-31T07:14:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f9313ab2740 1 client.25882 shutdown 2024-08-31T07:14:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f9313ab2740 1 -- 172.21.15.32:0/4141666087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550971b0 msgr2=0x55775547a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f9313ab2740 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550971b0 0x55775547a230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f93000039d0 tx=0x7f9300030610 comp rx=0 tx=0).stop 2024-08-31T07:14:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f9313ab2740 1 -- 172.21.15.32:0/4141666087 shutdown_connections 2024-08-31T07:14:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f9313ab2740 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x557755053da0 0x5577554f5290 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f9313ab2740 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577550c0070 0x5577554eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f9313ab2740 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55775509f390 0x55775547a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f9313ab2740 1 --2- 172.21.15.32:0/4141666087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577550971b0 0x55775547a230 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f92ebe00640 0 client.25882 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:14:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.199+0000 7f9313ab2740 1 -- 172.21.15.32:0/4141666087 >> 172.21.15.32:0/4141666087 conn(0x5577550b4460 msgr2=0x55775546ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9313ab2740 1 -- 172.21.15.32:0/4141666087 shutdown_connections 2024-08-31T07:14:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9313ab2740 1 -- 172.21.15.32:0/4141666087 wait complete. 2024-08-31T07:14:02.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9313ab2740 20 client.25882 trim_cache size 0 max 16384 2024-08-31T07:14:02.208 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:14:02.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9307200640 1 -- 172.21.15.32:0/6877372 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f92ec0b4ca0 msgr2=0x7f92ec0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:02.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9307200640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f92ec0b4ca0 0x7f92ec0b71b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5577554647e0 tx=0x7f92f80a7040 comp rx=0 tx=0).stop 2024-08-31T07:14:02.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9307200640 1 -- 172.21.15.32:0/6877372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9308103730 msgr2=0x7f930819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:02.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9307200640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9308103730 0x7f930819a0b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f92f800d6d0 tx=0x7f92f800db90 comp rx=0 tx=0).stop 2024-08-31T07:14:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9307200640 1 -- 172.21.15.32:0/6877372 shutdown_connections 2024-08-31T07:14:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9307200640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f92ec0b4ca0 0x7f92ec0b71b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9307200640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93081099c0 0x7f930819f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9307200640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9308109070 0x7f930819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.200+0000 7f9307200640 1 --2- 172.21.15.32:0/6877372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9308103730 0x7f930819a0b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.201+0000 7f9307200640 1 -- 172.21.15.32:0/6877372 >> 172.21.15.32:0/6877372 conn(0x7f93080ff180 msgr2=0x7f93081086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.201+0000 7f9307200640 1 -- 172.21.15.32:0/6877372 shutdown_connections 2024-08-31T07:14:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:02.201+0000 7f9307200640 1 -- 172.21.15.32:0/6877372 wait complete. 2024-08-31T07:14:02.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:02 smithi032 ceph-mon[17252]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-08-31T07:14:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:02 smithi073 ceph-mon[21995]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-08-31T07:14:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:02 smithi084 ceph-mon[22387]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-08-31T07:14:03.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:14:03.664 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:04.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:04 smithi032 ceph-mon[17252]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-08-31T07:14:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.323+0000 7fa05fe00640 1 -- 172.21.15.32:0/87924089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa060103750 msgr2=0x7fa060103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.323+0000 7fa05fe00640 1 --2- 172.21.15.32:0/87924089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa060103750 0x7fa060103b60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa050009900 tx=0x7fa05002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.323+0000 7fa05fe00640 1 -- 172.21.15.32:0/87924089 shutdown_connections 2024-08-31T07:14:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.323+0000 7fa05fe00640 1 --2- 172.21.15.32:0/87924089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0601099e0 0x7fa060110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.323+0000 7fa05fe00640 1 --2- 172.21.15.32:0/87924089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060109090 0x7fa0601094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.323+0000 7fa05fe00640 1 --2- 172.21.15.32:0/87924089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa060103750 0x7fa060103b60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.324+0000 7fa05fe00640 1 -- 172.21.15.32:0/87924089 >> 172.21.15.32:0/87924089 conn(0x7fa0600ff1a0 msgr2=0x7fa0601015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.324+0000 7fa05fe00640 1 -- 172.21.15.32:0/87924089 shutdown_connections 2024-08-31T07:14:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.324+0000 7fa05fe00640 1 -- 172.21.15.32:0/87924089 wait complete. 2024-08-31T07:14:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.325+0000 7fa05fe00640 1 Processor -- start 2024-08-31T07:14:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.325+0000 7fa05fe00640 1 -- start start 2024-08-31T07:14:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.325+0000 7fa05fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa060103750 0x7fa06019c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.325+0000 7fa05fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060109090 0x7fa06019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.325+0000 7fa05fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0601099e0 0x7fa0601a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.325+0000 7fa05fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601136b0 con 0x7fa060109090 2024-08-31T07:14:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.325+0000 7fa05fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa060113530 con 0x7fa060103750 2024-08-31T07:14:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.325+0000 7fa05fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa060113830 con 0x7fa0601099e0 2024-08-31T07:14:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.325+0000 7fa057e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060109090 0x7fa06019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.326+0000 7fa05ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa060103750 0x7fa06019c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.326+0000 7fa057e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060109090 0x7fa06019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60434/0 (socket says 172.21.15.32:60434) 2024-08-31T07:14:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.326+0000 7fa057e00640 1 -- 172.21.15.32:0/3588100135 learned_addr learned my addr 172.21.15.32:0/3588100135 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.326+0000 7fa05f400640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0601099e0 0x7fa0601a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.326+0000 7fa05ea00640 1 -- 172.21.15.32:0/3588100135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0601099e0 msgr2=0x7fa0601a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.326+0000 7fa05ea00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0601099e0 0x7fa0601a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.326+0000 7fa05ea00640 1 -- 172.21.15.32:0/3588100135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060109090 msgr2=0x7fa06019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.326+0000 7fa05ea00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060109090 0x7fa06019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.326+0000 7fa05ea00640 1 -- 172.21.15.32:0/3588100135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0601a21f0 con 0x7fa060103750 2024-08-31T07:14:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.326+0000 7fa05ea00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa060103750 0x7fa06019c280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa05002a910 tx=0x7fa050031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:04.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.327+0000 7fa057400640 1 -- 172.21.15.32:0/3588100135 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05003f030 con 0x7fa060103750 2024-08-31T07:14:04.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.327+0000 7fa05fe00640 1 -- 172.21.15.32:0/3588100135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0601a2480 con 0x7fa060103750 2024-08-31T07:14:04.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.327+0000 7fa057400640 1 -- 172.21.15.32:0/3588100135 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa050004490 con 0x7fa060103750 2024-08-31T07:14:04.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.327+0000 7fa057400640 1 -- 172.21.15.32:0/3588100135 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050035070 con 0x7fa060103750 2024-08-31T07:14:04.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.327+0000 7fa05fe00640 1 -- 172.21.15.32:0/3588100135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0601a8c40 con 0x7fa060103750 2024-08-31T07:14:04.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.330+0000 7fa06c806740 1 Processor -- start 2024-08-31T07:14:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.330+0000 7fa06c806740 1 -- start start 2024-08-31T07:14:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.330+0000 7fa06c806740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ee4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.330+0000 7fa06c806740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b161b0 0x564938ee50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.330+0000 7fa06c806740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b1e390 0x564938ee55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.330+0000 7fa06c806740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564938e5b7b0 con 0x564938b3f070 2024-08-31T07:14:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.330+0000 7fa06c806740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564938e4ad40 con 0x564938b1e390 2024-08-31T07:14:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.330+0000 7fa06c806740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564938e4ecf0 con 0x564938b161b0 2024-08-31T07:14:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.331+0000 7fa05f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ee4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.331+0000 7fa05f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ee4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60444/0 (socket says 172.21.15.32:60444) 2024-08-31T07:14:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.331+0000 7fa05f400640 1 -- 172.21.15.32:0/1171917602 learned_addr learned my addr 172.21.15.32:0/1171917602 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.331+0000 7fa057e00640 1 --2- 172.21.15.32:0/1171917602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b1e390 0x564938ee55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.331+0000 7fa057400640 1 -- 172.21.15.32:0/3588100135 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa050042030 con 0x7fa060103750 2024-08-31T07:14:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.331+0000 7fa05f400640 1 -- 172.21.15.32:0/1171917602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b161b0 msgr2=0x564938ee50a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:14:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.331+0000 7fa05f400640 1 --2- 172.21.15.32:0/1171917602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b161b0 0x564938ee50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.331+0000 7fa05f400640 1 -- 172.21.15.32:0/1171917602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b1e390 msgr2=0x564938ee55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.331+0000 7fa05f400640 1 --2- 172.21.15.32:0/1171917602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b1e390 0x564938ee55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.331+0000 7fa05f400640 1 -- 172.21.15.32:0/1171917602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564938baa650 con 0x564938b3f070 2024-08-31T07:14:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.332+0000 7fa05f400640 1 --2- 172.21.15.32:0/1171917602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ee4b60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa058009fd0 tx=0x7fa0580118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd5b2e330539346 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.332+0000 7fa04fe00640 1 -- 172.21.15.32:0/1171917602 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058004d20 con 0x564938b3f070 2024-08-31T07:14:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.332+0000 7fa04fe00640 1 -- 172.21.15.32:0/1171917602 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa058004050 con 0x564938b3f070 2024-08-31T07:14:04.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.332+0000 7fa04fe00640 1 -- 172.21.15.32:0/1171917602 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058005220 con 0x564938b3f070 2024-08-31T07:14:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa057400640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0440b4ca0 0x7fa0440b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa06c806740 1 -- 172.21.15.32:0/1171917602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 msgr2=0x564938ee4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa06c806740 1 --2- 172.21.15.32:0/1171917602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ee4b60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa058009fd0 tx=0x7fa0580118d0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa057400640 1 -- 172.21.15.32:0/3588100135 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fa0501053d0 con 0x7fa060103750 2024-08-31T07:14:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa06c806740 1 -- 172.21.15.32:0/1171917602 shutdown_connections 2024-08-31T07:14:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa06c806740 1 --2- 172.21.15.32:0/1171917602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b1e390 0x564938ee55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa06c806740 1 --2- 172.21.15.32:0/1171917602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b161b0 0x564938ee50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa06c806740 1 --2- 172.21.15.32:0/1171917602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ee4b60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa06c806740 1 -- 172.21.15.32:0/1171917602 >> 172.21.15.32:0/1171917602 conn(0x564938b33460 msgr2=0x564938ee2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa05ea00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0440b4ca0 0x7fa0440b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa06c806740 1 -- 172.21.15.32:0/1171917602 shutdown_connections 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.333+0000 7fa06c806740 1 -- 172.21.15.32:0/1171917602 wait complete. 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.334+0000 7fa05ea00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0440b4ca0 0x7fa0440b71b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x564938ee37e0 tx=0x7fa05003b560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.334+0000 7fa06c806740 1 -- start start 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.334+0000 7fa06c806740 1 Processor -- start 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa06c806740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ef06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa06c806740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa06c806740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938efc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa06c806740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564938cad700 con 0x564938b3f070 2024-08-31T07:14:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa06c806740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564938dd1cf0 con 0x564938b161b0 2024-08-31T07:14:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa06c806740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564938ef3340 con 0x564938b1e390 2024-08-31T07:14:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa06c806740 10 client.0 _ll_register_callbacks cb 0x564938df7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa057e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa05f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938efc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa057e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46690/0 (socket says 172.21.15.32:46690) 2024-08-31T07:14:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa057e00640 1 -- 172.21.15.32:0/2934020912 learned_addr learned my addr 172.21.15.32:0/2934020912 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa05f400640 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ef06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa057e00640 1 -- 172.21.15.32:0/2934020912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 msgr2=0x564938ef06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.335+0000 7fa057e00640 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ef06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.336+0000 7fa057e00640 1 -- 172.21.15.32:0/2934020912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 msgr2=0x564938efc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.336+0000 7fa057e00640 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938efc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.336+0000 7fa057e00640 1 -- 172.21.15.32:0/2934020912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564938e128b0 con 0x564938b1e390 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.336+0000 7fa057e00640 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef0c20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa048012740 tx=0x7fa048012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912cc34bdee2b86d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.336+0000 7fa04cc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.336+0000 7fa04cc00640 1 -- 172.21.15.32:0/2934020912 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048004020 con 0x564938b1e390 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.336+0000 7fa06c806740 1 -- 172.21.15.32:0/2934020912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa058004050 con 0x564938b1e390 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.336+0000 7fa04cc00640 1 -- 172.21.15.32:0/2934020912 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fa0480041c0 con 0x564938b1e390 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa04cc00640 1 -- 172.21.15.32:0/2934020912 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0480043c0 con 0x564938b1e390 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa06c806740 10 client.25891 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa06c806740 1 -- 172.21.15.32:0/2934020912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564938eff3c0 con 0x564938b1e390 2024-08-31T07:14:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa04cc00640 1 -- 172.21.15.32:0/2934020912 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa058004050 con 0x564938b1e390 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa04cc00640 1 -- 172.21.15.32:0/2934020912 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa048003b50 con 0x564938b1e390 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa06c806740 10 client.25891 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa06c806740 10 client.25891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa06c806740 20 client.25891 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa06c806740 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564938afd9f0 0x564938f02880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa06c806740 4 client.25891 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.337+0000 7fa06c806740 1 -- 172.21.15.32:0/2934020912 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564938e5d700 con 0x564938afd9f0 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.338+0000 7fa05f400640 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564938afd9f0 0x564938f02880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.338+0000 7fa05f400640 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564938afd9f0 0x564938f02880 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=911b553bee86b7bb server_cookie=3f98ced51403cec0 in_seq=0 out_seq=0 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.338+0000 7fa04cc00640 10 client.25891 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.339+0000 7fa04cc00640 1 -- 172.21.15.32:0/2934020912 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564938e5d700 con 0x564938afd9f0 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.339+0000 7fa04cc00640 10 client.25891 handle_command_reply: tid=0 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.339+0000 7fa06c806740 1 client.25891 shutdown 2024-08-31T07:14:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.339+0000 7fa06c806740 1 -- 172.21.15.32:0/2934020912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 msgr2=0x564938ef0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.339+0000 7fa06c806740 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef0c20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa048012740 tx=0x7fa048012c00 comp rx=0 tx=0).stop 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.339+0000 7fa06c806740 1 -- 172.21.15.32:0/2934020912 shutdown_connections 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.340+0000 7fa06c806740 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564938afd9f0 0x564938f02880 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.340+0000 7fa06c806740 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938efc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.340+0000 7fa06c806740 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef0c20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.340+0000 7fa04cc00640 0 client.25891 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.340+0000 7fa06c806740 1 --2- 172.21.15.32:0/2934020912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ef06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.340+0000 7fa06c806740 1 -- 172.21.15.32:0/2934020912 >> 172.21.15.32:0/2934020912 conn(0x564938b33460 msgr2=0x564938ee7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.340+0000 7fa06c806740 1 -- 172.21.15.32:0/2934020912 shutdown_connections 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.340+0000 7fa06c806740 1 -- 172.21.15.32:0/2934020912 wait complete. 2024-08-31T07:14:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.340+0000 7fa06c806740 20 client.25891 trim_cache size 0 max 16384 2024-08-31T07:14:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.354+0000 7fa06c806740 1 Processor -- start 2024-08-31T07:14:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.354+0000 7fa06c806740 1 -- start start 2024-08-31T07:14:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.354+0000 7fa06c806740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ee9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.354+0000 7fa06c806740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938eea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa06c806740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938efc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa06c806740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b12d0 con 0x564938b3f070 2024-08-31T07:14:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa06c806740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b2e10 con 0x564938b161b0 2024-08-31T07:14:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa06c806740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b19a0 con 0x564938b1e390 2024-08-31T07:14:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa05f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938efc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa05f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938efc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60462/0 (socket says 172.21.15.32:60462) 2024-08-31T07:14:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa05f400640 1 -- 172.21.15.32:0/2683255002 learned_addr learned my addr 172.21.15.32:0/2683255002 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa057e00640 1 --2- 172.21.15.32:0/2683255002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938eea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa05f400640 1 --2- 172.21.15.32:0/2683255002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ee9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa05f400640 1 -- 172.21.15.32:0/2683255002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 msgr2=0x564938eea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa05f400640 1 --2- 172.21.15.32:0/2683255002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938eea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa05f400640 1 -- 172.21.15.32:0/2683255002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 msgr2=0x564938ee9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa05f400640 1 --2- 172.21.15.32:0/2683255002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ee9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.355+0000 7fa05f400640 1 -- 172.21.15.32:0/2683255002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564938ef1580 con 0x564938b3f070 2024-08-31T07:14:04.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.356+0000 7fa05f400640 1 --2- 172.21.15.32:0/2683255002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938efc590 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa058004940 tx=0x7fa0580048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296bb35e15ec245 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:04.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.356+0000 7fa04f400640 1 -- 172.21.15.32:0/2683255002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058004220 con 0x564938b3f070 2024-08-31T07:14:04.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.356+0000 7fa04f400640 1 -- 172.21.15.32:0/2683255002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05800f4e0 con 0x564938b3f070 2024-08-31T07:14:04.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.356+0000 7fa04f400640 1 -- 172.21.15.32:0/2683255002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0580286b0 con 0x564938b3f070 2024-08-31T07:14:04.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.357+0000 7fa06c806740 1 -- 172.21.15.32:0/2683255002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 msgr2=0x564938efc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:04.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.357+0000 7fa06c806740 1 --2- 172.21.15.32:0/2683255002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938efc590 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa058004940 tx=0x7fa0580048d0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.357+0000 7fa06c806740 1 -- 172.21.15.32:0/2683255002 shutdown_connections 2024-08-31T07:14:04.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.357+0000 7fa06c806740 1 --2- 172.21.15.32:0/2683255002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938efc590 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.357+0000 7fa06c806740 1 --2- 172.21.15.32:0/2683255002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938eea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.357+0000 7fa06c806740 1 --2- 172.21.15.32:0/2683255002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ee9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.357+0000 7fa06c806740 1 -- 172.21.15.32:0/2683255002 >> 172.21.15.32:0/2683255002 conn(0x564938b33460 msgr2=0x564938eeb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:04.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.357+0000 7fa06c806740 1 -- 172.21.15.32:0/2683255002 shutdown_connections 2024-08-31T07:14:04.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.357+0000 7fa06c806740 1 -- 172.21.15.32:0/2683255002 wait complete. 2024-08-31T07:14:04.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.357+0000 7fa06c806740 1 -- start start 2024-08-31T07:14:04.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.358+0000 7fa06c806740 1 Processor -- start 2024-08-31T07:14:04.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.358+0000 7fa06c806740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ef8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.358+0000 7fa06c806740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.358+0000 7fa06c806740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ef35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.358+0000 7fa06c806740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b2740 con 0x564938b3f070 2024-08-31T07:14:04.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.358+0000 7fa06c806740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b2070 con 0x564938b161b0 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.358+0000 7fa06c806740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601af2b0 con 0x564938b1e390 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.358+0000 7fa06c806740 10 client.0 _ll_register_callbacks cb 0x7fa048009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.358+0000 7fa05f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ef35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa05f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ef35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60466/0 (socket says 172.21.15.32:60466) 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa05f400640 1 -- 172.21.15.32:0/956441738 learned_addr learned my addr 172.21.15.32:0/956441738 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa057e00640 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa05f400640 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ef8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa057e00640 1 -- 172.21.15.32:0/956441738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 msgr2=0x564938ef8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa057e00640 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ef8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa057e00640 1 -- 172.21.15.32:0/956441738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 msgr2=0x564938ef35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa057e00640 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ef35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa057e00640 1 -- 172.21.15.32:0/956441738 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564938ef90b0 con 0x564938b1e390 2024-08-31T07:14:04.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa05f400640 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ef35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa057e00640 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef8b40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x564938ee78d0 tx=0x7fa048009dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a71a16a6aad90a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.359+0000 7fa04cc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.360+0000 7fa04cc00640 1 -- 172.21.15.32:0/956441738 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048003ff0 con 0x564938b1e390 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.360+0000 7fa06c806740 1 -- 172.21.15.32:0/956441738 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05800f4e0 con 0x564938b1e390 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.360+0000 7fa04cc00640 1 -- 172.21.15.32:0/956441738 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fa048018040 con 0x564938b1e390 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.360+0000 7fa04cc00640 1 -- 172.21.15.32:0/956441738 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa048004190 con 0x564938b1e390 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.360+0000 7fa06c806740 10 client.25897 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.360+0000 7fa06c806740 1 -- 172.21.15.32:0/956441738 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564938f6d5c0 con 0x564938b1e390 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.360+0000 7fa04cc00640 1 -- 172.21.15.32:0/956441738 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05800f4e0 con 0x564938b1e390 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.361+0000 7fa04cc00640 1 -- 172.21.15.32:0/956441738 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa04800fce0 con 0x564938b1e390 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.361+0000 7fa06c806740 10 client.25897 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.361+0000 7fa06c806740 10 client.25897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:14:04.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.361+0000 7fa06c806740 20 client.25897 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.361+0000 7fa06c806740 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564938ad2da0 0x564938f6fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.361+0000 7fa06c806740 4 client.25897 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.361+0000 7fa06c806740 1 -- 172.21.15.32:0/956441738 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564938e591a0 con 0x564938ad2da0 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.361+0000 7fa05f400640 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564938ad2da0 0x564938f6fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa05f400640 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564938ad2da0 0x564938f6fd60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3e6a27fe5d5f342 server_cookie=4ee2cf49dda420e8 in_seq=0 out_seq=0 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa04cc00640 10 client.25897 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa04cc00640 1 -- 172.21.15.32:0/956441738 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564938e591a0 con 0x564938ad2da0 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa04cc00640 10 client.25897 handle_command_reply: tid=0 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 client.25897 shutdown 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 -- 172.21.15.32:0/956441738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 msgr2=0x564938ef8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef8b40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x564938ee78d0 tx=0x7fa048009dc0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 -- 172.21.15.32:0/956441738 shutdown_connections 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564938ad2da0 0x564938f6fd60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564938b3f070 0x564938ef35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564938b1e390 0x564938ef8b40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa04cc00640 0 client.25897 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 --2- 172.21.15.32:0/956441738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564938b161b0 0x564938ef8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 -- 172.21.15.32:0/956441738 >> 172.21.15.32:0/956441738 conn(0x564938b33460 msgr2=0x564938eeed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 -- 172.21.15.32:0/956441738 shutdown_connections 2024-08-31T07:14:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 1 -- 172.21.15.32:0/956441738 wait complete. 2024-08-31T07:14:04.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.362+0000 7fa06c806740 20 client.25897 trim_cache size 0 max 16384 2024-08-31T07:14:04.367 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:14:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.363+0000 7fa05fe00640 1 -- 172.21.15.32:0/3588100135 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0440b4ca0 msgr2=0x7fa0440b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.363+0000 7fa05fe00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0440b4ca0 0x7fa0440b71b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x564938ee37e0 tx=0x7fa05003b560 comp rx=0 tx=0).stop 2024-08-31T07:14:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.363+0000 7fa05fe00640 1 -- 172.21.15.32:0/3588100135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa060103750 msgr2=0x7fa06019c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.363+0000 7fa05fe00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa060103750 0x7fa06019c280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa05002a910 tx=0x7fa050031a20 comp rx=0 tx=0).stop 2024-08-31T07:14:04.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.363+0000 7fa05fe00640 1 -- 172.21.15.32:0/3588100135 shutdown_connections 2024-08-31T07:14:04.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.363+0000 7fa05fe00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0440b4ca0 0x7fa0440b71b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.363+0000 7fa05fe00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0601099e0 0x7fa0601a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.363+0000 7fa05fe00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa060109090 0x7fa06019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.363+0000 7fa05fe00640 1 --2- 172.21.15.32:0/3588100135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa060103750 0x7fa06019c280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:04.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.363+0000 7fa05fe00640 1 -- 172.21.15.32:0/3588100135 >> 172.21.15.32:0/3588100135 conn(0x7fa0600ff1a0 msgr2=0x7fa0601087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:04.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.364+0000 7fa05fe00640 1 -- 172.21.15.32:0/3588100135 shutdown_connections 2024-08-31T07:14:04.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:04.364+0000 7fa05fe00640 1 -- 172.21.15.32:0/3588100135 wait complete. 2024-08-31T07:14:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:04 smithi073 ceph-mon[21995]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-08-31T07:14:04.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:04 smithi084 ceph-mon[22387]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-08-31T07:14:06.293 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:14:06.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:14:06.348 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:06 smithi032 ceph-mon[17252]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-08-31T07:14:06.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:06 smithi073 ceph-mon[21995]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-08-31T07:14:06.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:06 smithi084 ceph-mon[22387]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-08-31T07:14:06.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:14:06.684 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:07.417 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.422+0000 7fdb38a00640 1 -- 172.21.15.32:0/3242226897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb34109ba0 msgr2=0x7fdb341103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.422+0000 7fdb38a00640 1 --2- 172.21.15.32:0/3242226897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb34109ba0 0x7fdb341103d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009ca0 tx=0x7fdb2c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.422+0000 7fdb38a00640 1 -- 172.21.15.32:0/3242226897 shutdown_connections 2024-08-31T07:14:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.422+0000 7fdb38a00640 1 --2- 172.21.15.32:0/3242226897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb34109ba0 0x7fdb341103d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.422+0000 7fdb38a00640 1 --2- 172.21.15.32:0/3242226897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb34109250 0x7fdb34109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.422+0000 7fdb38a00640 1 --2- 172.21.15.32:0/3242226897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb34103910 0x7fdb34103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.422+0000 7fdb38a00640 1 -- 172.21.15.32:0/3242226897 >> 172.21.15.32:0/3242226897 conn(0x7fdb340ff380 msgr2=0x7fdb341017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.422+0000 7fdb38a00640 1 -- 172.21.15.32:0/3242226897 shutdown_connections 2024-08-31T07:14:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.422+0000 7fdb38a00640 1 -- 172.21.15.32:0/3242226897 wait complete. 2024-08-31T07:14:07.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.423+0000 7fdb38a00640 1 Processor -- start 2024-08-31T07:14:07.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.423+0000 7fdb38a00640 1 -- start start 2024-08-31T07:14:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb38a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb34103910 0x7fdb340754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb38a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb34109250 0x7fdb340759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb38a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb34109ba0 0x7fdb34075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb38a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34113860 con 0x7fdb34109250 2024-08-31T07:14:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb38a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341136e0 con 0x7fdb34103910 2024-08-31T07:14:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb38a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341139e0 con 0x7fdb34109ba0 2024-08-31T07:14:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb31400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb34109250 0x7fdb340759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb31e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb34103910 0x7fdb340754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb31400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb34109250 0x7fdb340759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52222/0 (socket says 172.21.15.32:52222) 2024-08-31T07:14:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb31400640 1 -- 172.21.15.32:0/534184540 learned_addr learned my addr 172.21.15.32:0/534184540 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.424+0000 7fdb32800640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb34109ba0 0x7fdb34075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb31400640 1 -- 172.21.15.32:0/534184540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb34109ba0 msgr2=0x7fdb34075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb31400640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb34109ba0 0x7fdb34075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb31400640 1 -- 172.21.15.32:0/534184540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb34103910 msgr2=0x7fdb340754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb31400640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb34103910 0x7fdb340754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb31400640 1 -- 172.21.15.32:0/534184540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb341a84d0 con 0x7fdb34109250 2024-08-31T07:14:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb31e00640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb34103910 0x7fdb340754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb31400640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb34109250 0x7fdb340759f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000e9c0 tx=0x7fdb2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb2aa00640 1 -- 172.21.15.32:0/534184540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000ccc0 con 0x7fdb34109250 2024-08-31T07:14:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb2aa00640 1 -- 172.21.15.32:0/534184540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb200054d0 con 0x7fdb34109250 2024-08-31T07:14:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb2aa00640 1 -- 172.21.15.32:0/534184540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20010430 con 0x7fdb34109250 2024-08-31T07:14:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb38a00640 1 -- 172.21.15.32:0/534184540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb341a87c0 con 0x7fdb34109250 2024-08-31T07:14:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.425+0000 7fdb38a00640 1 -- 172.21.15.32:0/534184540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb341a8cb0 con 0x7fdb34109250 2024-08-31T07:14:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.429+0000 7fdb3f65a740 1 Processor -- start 2024-08-31T07:14:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.429+0000 7fdb3f65a740 1 -- start start 2024-08-31T07:14:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.429+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe75e350 0x5597feb04600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.429+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb04b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.429+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe73d670 0x5597feb05080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.429+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5597fea7a750 con 0x5597fe73d670 2024-08-31T07:14:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.429+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5597fea69ce0 con 0x5597fe75e350 2024-08-31T07:14:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.429+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5597fea6dc90 con 0x5597fe735490 2024-08-31T07:14:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.430+0000 7fdb31e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb04b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.430+0000 7fdb32800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe75e350 0x5597feb04600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.430+0000 7fdb31e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb04b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35492/0 (socket says 172.21.15.32:35492) 2024-08-31T07:14:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.430+0000 7fdb32800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe75e350 0x5597feb04600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34326/0 (socket says 172.21.15.32:34326) 2024-08-31T07:14:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.430+0000 7fdb31e00640 1 -- 172.21.15.32:0/1059207477 learned_addr learned my addr 172.21.15.32:0/1059207477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.430+0000 7fdb2aa00640 1 -- 172.21.15.32:0/534184540 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fdb20005020 con 0x7fdb34109250 2024-08-31T07:14:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.430+0000 7fdb31e00640 1 -- 172.21.15.32:0/1059207477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe75e350 msgr2=0x5597feb04600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.430+0000 7fdb31e00640 1 --2- 172.21.15.32:0/1059207477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe75e350 0x5597feb04600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.430+0000 7fdb31e00640 1 -- 172.21.15.32:0/1059207477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe73d670 msgr2=0x5597feb05080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:14:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.430+0000 7fdb31400640 1 --2- 172.21.15.32:0/1059207477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe73d670 0x5597feb05080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.431+0000 7fdb31e00640 1 --2- 172.21.15.32:0/1059207477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe73d670 0x5597feb05080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.431+0000 7fdb31e00640 1 -- 172.21.15.32:0/1059207477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597fe7c9930 con 0x5597fe735490 2024-08-31T07:14:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.431+0000 7fdb31400640 1 --2- 172.21.15.32:0/1059207477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe73d670 0x5597feb05080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:14:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.431+0000 7fdb31e00640 1 --2- 172.21.15.32:0/1059207477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb04b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdb240114f0 tx=0x7fdb240119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1150bdf6d28ca1d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:07.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.431+0000 7fdb1f000640 1 -- 172.21.15.32:0/1059207477 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2401e070 con 0x5597fe735490 2024-08-31T07:14:07.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.431+0000 7fdb1f000640 1 -- 172.21.15.32:0/1059207477 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb240035f0 con 0x5597fe735490 2024-08-31T07:14:07.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.431+0000 7fdb1f000640 1 -- 172.21.15.32:0/1059207477 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400f820 con 0x5597fe735490 2024-08-31T07:14:07.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.432+0000 7fdb2aa00640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb180b4ca0 0x7fdb180b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.432+0000 7fdb2aa00640 1 -- 172.21.15.32:0/534184540 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fdb200e5820 con 0x7fdb34109250 2024-08-31T07:14:07.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.432+0000 7fdb32800640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb180b4ca0 0x7fdb180b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.432+0000 7fdb3f65a740 1 -- 172.21.15.32:0/1059207477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 msgr2=0x5597feb04b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.432+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/1059207477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb04b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdb240114f0 tx=0x7fdb240119b0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.433+0000 7fdb3f65a740 1 -- 172.21.15.32:0/1059207477 shutdown_connections 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.433+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/1059207477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe73d670 0x5597feb05080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.433+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/1059207477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb04b40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.433+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/1059207477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe75e350 0x5597feb04600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.433+0000 7fdb3f65a740 1 -- 172.21.15.32:0/1059207477 >> 172.21.15.32:0/1059207477 conn(0x5597fe752740 msgr2=0x5597feb024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.433+0000 7fdb3f65a740 1 -- 172.21.15.32:0/1059207477 shutdown_connections 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.433+0000 7fdb3f65a740 1 -- 172.21.15.32:0/1059207477 wait complete. 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.433+0000 7fdb32800640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb180b4ca0 0x7fdb180b71b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c02f500 tx=0x7fdb2c02f310 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.433+0000 7fdb3f65a740 1 -- start start 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.434+0000 7fdb3f65a740 1 Processor -- start 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.434+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe735490 0x5597feb10180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.434+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe73d670 0x5597feb106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.434+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb1c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.434+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5597fe8cc890 con 0x5597fe75e350 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.434+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5597fe9f0cd0 con 0x5597fe735490 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.434+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5597feb12de0 con 0x5597fe73d670 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.434+0000 7fdb3f65a740 10 client.0 _ll_register_callbacks cb 0x5597fea64ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb31400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe73d670 0x5597feb106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb32800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb1c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb31400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe73d670 0x5597feb106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35502/0 (socket says 172.21.15.32:35502) 2024-08-31T07:14:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb31e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe735490 0x5597feb10180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb31400640 1 -- 172.21.15.32:0/802987153 learned_addr learned my addr 172.21.15.32:0/802987153 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb31400640 1 -- 172.21.15.32:0/802987153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe735490 msgr2=0x5597feb10180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb31400640 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe735490 0x5597feb10180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb31400640 1 -- 172.21.15.32:0/802987153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 msgr2=0x5597feb1c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb31400640 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb1c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb31400640 1 -- 172.21.15.32:0/802987153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fea3a1b0 con 0x5597fe73d670 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb31400640 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe73d670 0x5597feb106c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000c6a0 tx=0x7fdb2000ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a99576f18114b173 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.435+0000 7fdb13400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.436+0000 7fdb13400640 1 -- 172.21.15.32:0/802987153 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb200d2020 con 0x5597fe73d670 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.436+0000 7fdb13400640 1 -- 172.21.15.32:0/802987153 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fdb200b0410 con 0x5597fe73d670 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.436+0000 7fdb3f65a740 1 -- 172.21.15.32:0/802987153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb240035f0 con 0x5597fe73d670 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.436+0000 7fdb13400640 1 -- 172.21.15.32:0/802987153 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb200abaa0 con 0x5597fe73d670 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.436+0000 7fdb3f65a740 10 client.25909 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.436+0000 7fdb3f65a740 1 -- 172.21.15.32:0/802987153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5597feb1edf0 con 0x5597fe73d670 2024-08-31T07:14:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.436+0000 7fdb13400640 1 -- 172.21.15.32:0/802987153 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb240035f0 con 0x5597fe73d670 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.437+0000 7fdb13400640 1 -- 172.21.15.32:0/802987153 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdb20097190 con 0x5597fe73d670 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.437+0000 7fdb3f65a740 10 client.25909 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.437+0000 7fdb3f65a740 10 client.25909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.437+0000 7fdb3f65a740 20 client.25909 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.437+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597fe71ccd0 0x5597feb22290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.437+0000 7fdb3f65a740 4 client.25909 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.437+0000 7fdb3f65a740 1 -- 172.21.15.32:0/802987153 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597fea7c6a0 con 0x5597fe71ccd0 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.437+0000 7fdb31e00640 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597fe71ccd0 0x5597feb22290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.438+0000 7fdb31e00640 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597fe71ccd0 0x5597feb22290 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9270c34708882947 server_cookie=368d1ebafb854e4f in_seq=0 out_seq=0 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.438+0000 7fdb13400640 10 client.25909 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:07.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.438+0000 7fdb13400640 1 -- 172.21.15.32:0/802987153 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597fea7c6a0 con 0x5597fe71ccd0 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.438+0000 7fdb13400640 10 client.25909 handle_command_reply: tid=0 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 client.25909 shutdown 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 -- 172.21.15.32:0/802987153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe73d670 msgr2=0x5597feb106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe73d670 0x5597feb106c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000c6a0 tx=0x7fdb2000ce60 comp rx=0 tx=0).stop 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 -- 172.21.15.32:0/802987153 shutdown_connections 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597fe71ccd0 0x5597feb22290 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb1c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe73d670 0x5597feb106c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/802987153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe735490 0x5597feb10180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb13400640 0 client.25909 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 -- 172.21.15.32:0/802987153 >> 172.21.15.32:0/802987153 conn(0x5597fe752740 msgr2=0x5597feb07790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:07.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 -- 172.21.15.32:0/802987153 shutdown_connections 2024-08-31T07:14:07.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 1 -- 172.21.15.32:0/802987153 wait complete. 2024-08-31T07:14:07.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.439+0000 7fdb3f65a740 20 client.25909 trim_cache size 0 max 16384 2024-08-31T07:14:07.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.459+0000 7fdb3f65a740 1 Processor -- start 2024-08-31T07:14:07.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.459+0000 7fdb3f65a740 1 -- start start 2024-08-31T07:14:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb05d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb1c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb1c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341b1480 con 0x5597fe75e350 2024-08-31T07:14:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341b2fc0 con 0x5597fe73d670 2024-08-31T07:14:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341b1b50 con 0x5597fe735490 2024-08-31T07:14:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb31400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb1c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb32800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb1c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb31e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb05d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb31400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb1c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34332/0 (socket says 172.21.15.32:34332) 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb31400640 1 -- 172.21.15.32:0/1572771090 learned_addr learned my addr 172.21.15.32:0/1572771090 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb31e00640 1 -- 172.21.15.32:0/1572771090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 msgr2=0x5597feb1c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb31e00640 1 --2- 172.21.15.32:0/1572771090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb1c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.460+0000 7fdb31e00640 1 -- 172.21.15.32:0/1572771090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 msgr2=0x5597feb1c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.461+0000 7fdb31e00640 1 --2- 172.21.15.32:0/1572771090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb1c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.461+0000 7fdb31e00640 1 -- 172.21.15.32:0/1572771090 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597feb0eee0 con 0x5597fe735490 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.461+0000 7fdb31e00640 1 --2- 172.21.15.32:0/1572771090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb05d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdb240058c0 tx=0x7fdb24009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c835857c42ca67b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.461+0000 7fdb1e600640 1 -- 172.21.15.32:0/1572771090 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24031070 con 0x5597fe735490 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.461+0000 7fdb1e600640 1 -- 172.21.15.32:0/1572771090 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2401f040 con 0x5597fe735490 2024-08-31T07:14:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.461+0000 7fdb1e600640 1 -- 172.21.15.32:0/1572771090 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2401e040 con 0x5597fe735490 2024-08-31T07:14:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.462+0000 7fdb3f65a740 1 -- 172.21.15.32:0/1572771090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 msgr2=0x5597feb05d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.462+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/1572771090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb05d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdb240058c0 tx=0x7fdb24009b50 comp rx=0 tx=0).stop 2024-08-31T07:14:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.462+0000 7fdb3f65a740 1 -- 172.21.15.32:0/1572771090 shutdown_connections 2024-08-31T07:14:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.462+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/1572771090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb1c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.462+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/1572771090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb1c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.462+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/1572771090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb05d40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.462+0000 7fdb3f65a740 1 -- 172.21.15.32:0/1572771090 >> 172.21.15.32:0/1572771090 conn(0x5597fe752740 msgr2=0x5597feb05630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.462+0000 7fdb3f65a740 1 -- 172.21.15.32:0/1572771090 shutdown_connections 2024-08-31T07:14:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.462+0000 7fdb3f65a740 1 -- 172.21.15.32:0/1572771090 wait complete. 2024-08-31T07:14:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.463+0000 7fdb3f65a740 1 -- start start 2024-08-31T07:14:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.463+0000 7fdb3f65a740 1 Processor -- start 2024-08-31T07:14:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb18ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb15010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb3f65a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb15720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341b28f0 con 0x5597fe75e350 2024-08-31T07:14:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341b2220 con 0x5597fe73d670 2024-08-31T07:14:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb3f65a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341af480 con 0x5597fe735490 2024-08-31T07:14:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb3f65a740 10 client.0 _ll_register_callbacks cb 0x7fdb20004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb31400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb15010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb31400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb15010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34338/0 (socket says 172.21.15.32:34338) 2024-08-31T07:14:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb31400640 1 -- 172.21.15.32:0/902924525 learned_addr learned my addr 172.21.15.32:0/902924525 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb32800640 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb15720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb31e00640 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb18ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb31400640 1 -- 172.21.15.32:0/902924525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 msgr2=0x5597feb18ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.464+0000 7fdb31400640 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb18ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb31400640 1 -- 172.21.15.32:0/902924525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 msgr2=0x5597feb15720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb31400640 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb15720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb31400640 1 -- 172.21.15.32:0/902924525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597feb8cc60 con 0x5597fe73d670 2024-08-31T07:14:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb31400640 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb15010 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb200049d0 tx=0x7fdb2000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3051086b707b20 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb13400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:14:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb13400640 1 -- 172.21.15.32:0/902924525 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb200d2070 con 0x5597fe73d670 2024-08-31T07:14:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb13400640 1 -- 172.21.15.32:0/902924525 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fdb200b5030 con 0x5597fe73d670 2024-08-31T07:14:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb3f65a740 1 -- 172.21.15.32:0/902924525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb2401f040 con 0x5597fe73d670 2024-08-31T07:14:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb13400640 1 -- 172.21.15.32:0/902924525 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb20005310 con 0x5597fe73d670 2024-08-31T07:14:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb3f65a740 10 client.25913 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.465+0000 7fdb3f65a740 1 -- 172.21.15.32:0/902924525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5597feb8de60 con 0x5597fe73d670 2024-08-31T07:14:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.466+0000 7fdb13400640 1 -- 172.21.15.32:0/902924525 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2401f040 con 0x5597fe73d670 2024-08-31T07:14:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.466+0000 7fdb13400640 1 -- 172.21.15.32:0/902924525 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdb200930b0 con 0x5597fe73d670 2024-08-31T07:14:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.466+0000 7fdb3f65a740 10 client.25913 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.466+0000 7fdb3f65a740 10 client.25913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:14:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.466+0000 7fdb3f65a740 20 client.25913 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.466+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597fe6f2080 0x5597feb904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.466+0000 7fdb3f65a740 4 client.25913 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:14:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.466+0000 7fdb3f65a740 1 -- 172.21.15.32:0/902924525 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5597fea78140 con 0x5597fe6f2080 2024-08-31T07:14:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.466+0000 7fdb31e00640 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597fe6f2080 0x5597feb904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.467+0000 7fdb31e00640 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597fe6f2080 0x5597feb904e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e65c94ab76da7a81 server_cookie=2fe17bd4398bfbb1 in_seq=0 out_seq=0 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.467+0000 7fdb13400640 10 client.25913 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.468+0000 7fdb13400640 1 -- 172.21.15.32:0/902924525 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5597fea78140 con 0x5597fe6f2080 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.468+0000 7fdb13400640 10 client.25913 handle_command_reply: tid=0 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 client.25913 shutdown 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 -- 172.21.15.32:0/902924525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 msgr2=0x5597feb15010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb15010 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb200049d0 tx=0x7fdb2000cf90 comp rx=0 tx=0).stop 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 -- 172.21.15.32:0/902924525 shutdown_connections 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597fe6f2080 0x5597feb904e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597fe75e350 0x5597feb15720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597fe73d670 0x5597feb15010 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 --2- 172.21.15.32:0/902924525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597fe735490 0x5597feb18ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb13400640 0 client.25913 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:07.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 -- 172.21.15.32:0/902924525 >> 172.21.15.32:0/902924525 conn(0x5597fe752740 msgr2=0x5597feb05ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:07.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 -- 172.21.15.32:0/902924525 shutdown_connections 2024-08-31T07:14:07.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 1 -- 172.21.15.32:0/902924525 wait complete. 2024-08-31T07:14:07.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.469+0000 7fdb3f65a740 20 client.25913 trim_cache size 0 max 16384 2024-08-31T07:14:07.474 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:14:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:14:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "9fcca2ec-b3db-4784-8740-c1e275303481", 2024-08-31T07:14:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:14:07.475 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:14:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.470+0000 7fdb38a00640 1 -- 172.21.15.32:0/534184540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb180b4ca0 msgr2=0x7fdb180b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.470+0000 7fdb38a00640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb180b4ca0 0x7fdb180b71b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c02f500 tx=0x7fdb2c02f310 comp rx=0 tx=0).stop 2024-08-31T07:14:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.470+0000 7fdb38a00640 1 -- 172.21.15.32:0/534184540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb34109250 msgr2=0x7fdb340759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.471+0000 7fdb38a00640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb34109250 0x7fdb340759f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000e9c0 tx=0x7fdb2000ee80 comp rx=0 tx=0).stop 2024-08-31T07:14:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.471+0000 7fdb38a00640 1 -- 172.21.15.32:0/534184540 shutdown_connections 2024-08-31T07:14:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.471+0000 7fdb38a00640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb180b4ca0 0x7fdb180b71b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.471+0000 7fdb38a00640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb34109ba0 0x7fdb34075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.471+0000 7fdb38a00640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb34109250 0x7fdb340759f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.471+0000 7fdb38a00640 1 --2- 172.21.15.32:0/534184540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb34103910 0x7fdb340754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.471+0000 7fdb38a00640 1 -- 172.21.15.32:0/534184540 >> 172.21.15.32:0/534184540 conn(0x7fdb340ff380 msgr2=0x7fdb341088e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.471+0000 7fdb38a00640 1 -- 172.21.15.32:0/534184540 shutdown_connections 2024-08-31T07:14:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:07.471+0000 7fdb38a00640 1 -- 172.21.15.32:0/534184540 wait complete. 2024-08-31T07:14:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.128+0000 7f9b0de00640 1 -- 172.21.15.32:0/710373711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b08103750 msgr2=0x7f9b08103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.128+0000 7f9b0de00640 1 --2- 172.21.15.32:0/710373711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b08103750 0x7f9b08103b60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009900 tx=0x7f9afc02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:14:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.129+0000 7f9b0de00640 1 -- 172.21.15.32:0/710373711 shutdown_connections 2024-08-31T07:14:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.129+0000 7f9b0de00640 1 --2- 172.21.15.32:0/710373711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b081099e0 0x7f9b08110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.129+0000 7f9b0de00640 1 --2- 172.21.15.32:0/710373711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b08109090 0x7f9b081094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.129+0000 7f9b0de00640 1 --2- 172.21.15.32:0/710373711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b08103750 0x7f9b08103b60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.129+0000 7f9b0de00640 1 -- 172.21.15.32:0/710373711 >> 172.21.15.32:0/710373711 conn(0x7f9b080ff1e0 msgr2=0x7f9b08101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.129+0000 7f9b0de00640 1 -- 172.21.15.32:0/710373711 shutdown_connections 2024-08-31T07:14:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.129+0000 7f9b0de00640 1 -- 172.21.15.32:0/710373711 wait complete. 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.130+0000 7f9b0de00640 1 Processor -- start 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.130+0000 7f9b0de00640 1 -- start start 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.130+0000 7f9b0de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08103750 0x7f9b0819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.130+0000 7f9b0de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b08109090 0x7f9b0819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.130+0000 7f9b0de00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b081099e0 0x7f9b081a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0de00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08113700 con 0x7f9b08109090 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0de00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08113580 con 0x7f9b08103750 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0de00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08113880 con 0x7f9b081099e0 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08103750 0x7f9b0819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b07e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b08109090 0x7f9b0819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08103750 0x7f9b0819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34358/0 (socket says 172.21.15.32:34358) 2024-08-31T07:14:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0ca00640 1 -- 172.21.15.32:0/2021638911 learned_addr learned my addr 172.21.15.32:0/2021638911 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:08.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0d400640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b081099e0 0x7f9b081a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:08.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0d400640 1 -- 172.21.15.32:0/2021638911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08103750 msgr2=0x7f9b0819c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:08.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0d400640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08103750 0x7f9b0819c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:08.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0d400640 1 -- 172.21.15.32:0/2021638911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b08109090 msgr2=0x7f9b0819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0d400640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b08109090 0x7f9b0819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b0d400640 1 -- 172.21.15.32:0/2021638911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b081a22a0 con 0x7f9b081099e0 2024-08-31T07:14:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.131+0000 7f9b07e00640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b08109090 0x7f9b0819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.132+0000 7f9b0d400640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b081099e0 0x7f9b081a1b70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009fd0 tx=0x7f9b0000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.132+0000 7f9b05600640 1 -- 172.21.15.32:0/2021638911 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000ce20 con 0x7f9b081099e0 2024-08-31T07:14:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.132+0000 7f9b05600640 1 -- 172.21.15.32:0/2021638911 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b00005020 con 0x7f9b081099e0 2024-08-31T07:14:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.132+0000 7f9b0de00640 1 -- 172.21.15.32:0/2021638911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b081a2590 con 0x7f9b081099e0 2024-08-31T07:14:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.132+0000 7f9b05600640 1 -- 172.21.15.32:0/2021638911 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00010690 con 0x7f9b081099e0 2024-08-31T07:14:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.132+0000 7f9b0de00640 1 -- 172.21.15.32:0/2021638911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b081a8cf0 con 0x7f9b081099e0 2024-08-31T07:14:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.133+0000 7f9b0de00640 1 -- 172.21.15.32:0/2021638911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b08103b60 con 0x7f9b081099e0 2024-08-31T07:14:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.136+0000 7f9b05600640 1 -- 172.21.15.32:0/2021638911 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9b000054d0 con 0x7f9b081099e0 2024-08-31T07:14:08.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.137+0000 7f9b05600640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9af00b4ca0 0x7f9af00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:08.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.137+0000 7f9b05600640 1 -- 172.21.15.32:0/2021638911 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f9b000e5bd0 con 0x7f9b081099e0 2024-08-31T07:14:08.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.137+0000 7f9b0ca00640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9af00b4ca0 0x7f9af00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:08.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.138+0000 7f9b0ca00640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9af00b4ca0 0x7f9af00b71b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0096c0 tx=0x7f9afc005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:08.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.141+0000 7f9b05600640 1 -- 172.21.15.32:0/2021638911 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9b000ac9e0 con 0x7f9b081099e0 2024-08-31T07:14:08.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:08 smithi032 ceph-mon[17252]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-08-31T07:14:08.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:08 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:14:08.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:08 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:14:08.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:08 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:14:08.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:08 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:14:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.380+0000 7f9b0de00640 1 -- 172.21.15.32:0/2021638911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b081a8fd0 con 0x7f9b081099e0 2024-08-31T07:14:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.381+0000 7f9b05600640 1 -- 172.21.15.32:0/2021638911 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f9b000ac2e0 con 0x7f9b081099e0 2024-08-31T07:14:08.382 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:14:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"btime":"2024-08-31T07:12:13:409703+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:12:12.630233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:14:08.382 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-08-31T07:14:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.383+0000 7f9b0de00640 1 -- 172.21.15.32:0/2021638911 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9af00b4ca0 msgr2=0x7f9af00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.383+0000 7f9b0de00640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9af00b4ca0 0x7f9af00b71b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0096c0 tx=0x7f9afc005a20 comp rx=0 tx=0).stop 2024-08-31T07:14:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.383+0000 7f9b0de00640 1 -- 172.21.15.32:0/2021638911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b081099e0 msgr2=0x7f9b081a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.383+0000 7f9b0de00640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b081099e0 0x7f9b081a1b70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009fd0 tx=0x7f9b0000ef10 comp rx=0 tx=0).stop 2024-08-31T07:14:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.383+0000 7f9b0de00640 1 -- 172.21.15.32:0/2021638911 shutdown_connections 2024-08-31T07:14:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.383+0000 7f9b0de00640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9af00b4ca0 0x7f9af00b71b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.383+0000 7f9b0de00640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b081099e0 0x7f9b081a1b70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.383+0000 7f9b0de00640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b08109090 0x7f9b0819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.383+0000 7f9b0de00640 1 --2- 172.21.15.32:0/2021638911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b08103750 0x7f9b0819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.383+0000 7f9b0de00640 1 -- 172.21.15.32:0/2021638911 >> 172.21.15.32:0/2021638911 conn(0x7f9b080ff1e0 msgr2=0x7f9b08108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.384+0000 7f9b0de00640 1 -- 172.21.15.32:0/2021638911 shutdown_connections 2024-08-31T07:14:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:08.384+0000 7f9b0de00640 1 -- 172.21.15.32:0/2021638911 wait complete. 2024-08-31T07:14:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:08 smithi073 ceph-mon[21995]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-08-31T07:14:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:08 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:14:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:08 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:14:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:08 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:14:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:08 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:14:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:08 smithi084 ceph-mon[22387]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-08-31T07:14:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:08 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:14:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:08 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:14:08.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:08 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:14:08.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:08 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:14:09.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:09 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2021638911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:14:09.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:14:09.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:09 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2021638911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:14:09.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:09 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2021638911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:14:09.824 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 9cf8d043 --release --await' 2024-08-31T07:14:09.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 9cf8d043 --release --await 2024-08-31T07:14:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:10.356 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:10 smithi032 ceph-mon[17252]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-08-31T07:14:10.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:10 smithi073 ceph-mon[21995]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-08-31T07:14:10.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:10 smithi084 ceph-mon[22387]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-08-31T07:14:10.523 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.544+0000 7f470bc00640 1 -- 172.21.15.32:0/1507282460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4704109250 msgr2=0x7f4704109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.544+0000 7f470bc00640 1 --2- 172.21.15.32:0/1507282460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4704109250 0x7f4704109660 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009900 tx=0x7f46f002a410 comp rx=0 tx=0).stop 2024-08-31T07:14:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.544+0000 7f470bc00640 1 -- 172.21.15.32:0/1507282460 shutdown_connections 2024-08-31T07:14:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.544+0000 7f470bc00640 1 --2- 172.21.15.32:0/1507282460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4704109ba0 0x7f47041103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.544+0000 7f470bc00640 1 --2- 172.21.15.32:0/1507282460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4704109250 0x7f4704109660 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.544+0000 7f470bc00640 1 --2- 172.21.15.32:0/1507282460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4704103910 0x7f4704103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.544+0000 7f470bc00640 1 -- 172.21.15.32:0/1507282460 >> 172.21.15.32:0/1507282460 conn(0x7f47040ff380 msgr2=0x7f47041017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.544+0000 7f470bc00640 1 -- 172.21.15.32:0/1507282460 shutdown_connections 2024-08-31T07:14:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.544+0000 7f470bc00640 1 -- 172.21.15.32:0/1507282460 wait complete. 2024-08-31T07:14:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.545+0000 7f470bc00640 1 Processor -- start 2024-08-31T07:14:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.546+0000 7f470bc00640 1 -- start start 2024-08-31T07:14:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.546+0000 7f470bc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4704103910 0x7f47040754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.546+0000 7f470bc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4704109250 0x7f47040759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.546+0000 7f470bc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4704109ba0 0x7f4704075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.546+0000 7f470bc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f47041138b0 con 0x7f4704103910 2024-08-31T07:14:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.546+0000 7f470bc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4704113730 con 0x7f4704109250 2024-08-31T07:14:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.546+0000 7f470bc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4704113a30 con 0x7f4704109ba0 2024-08-31T07:14:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.546+0000 7f4709200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4704103910 0x7f47040754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.546+0000 7f4709200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4704103910 0x7f47040754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52294/0 (socket says 172.21.15.32:52294) 2024-08-31T07:14:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.546+0000 7f4709200640 1 -- 172.21.15.32:0/4050897369 learned_addr learned my addr 172.21.15.32:0/4050897369 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.547+0000 7f4703e00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4704109250 0x7f47040759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.547+0000 7f4709c00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4704109ba0 0x7f4704075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.547+0000 7f4709200640 1 -- 172.21.15.32:0/4050897369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4704109ba0 msgr2=0x7f4704075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.547+0000 7f4709200640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4704109ba0 0x7f4704075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.547+0000 7f4709200640 1 -- 172.21.15.32:0/4050897369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4704109250 msgr2=0x7f47040759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.547+0000 7f4709200640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4704109250 0x7f47040759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.547+0000 7f4709200640 1 -- 172.21.15.32:0/4050897369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47041a84e0 con 0x7f4704103910 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.547+0000 7f4703e00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4704109250 0x7f47040759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.547+0000 7f4709200640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4704103910 0x7f47040754b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f46f800ea60 tx=0x7f46f800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.548+0000 7f4701600640 1 -- 172.21.15.32:0/4050897369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f800cde0 con 0x7f4704103910 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.548+0000 7f4701600640 1 -- 172.21.15.32:0/4050897369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f80035f0 con 0x7f4704103910 2024-08-31T07:14:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.548+0000 7f470bc00640 1 -- 172.21.15.32:0/4050897369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47041a87d0 con 0x7f4704103910 2024-08-31T07:14:10.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.548+0000 7f470bc00640 1 -- 172.21.15.32:0/4050897369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47041a8ce0 con 0x7f4704103910 2024-08-31T07:14:10.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.548+0000 7f4701600640 1 -- 172.21.15.32:0/4050897369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8010690 con 0x7f4704103910 2024-08-31T07:14:10.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.551+0000 7f4712811740 1 Processor -- start 2024-08-31T07:14:10.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.551+0000 7f4712811740 1 -- start start 2024-08-31T07:14:10.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4701600640 1 -- 172.21.15.32:0/4050897369 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f46f80037a0 con 0x7f4704103910 2024-08-31T07:14:10.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4712811740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 0x56466a569950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4712811740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a19b1b0 0x56466a569e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4712811740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1a3390 0x56466a56a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4712811740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56466a4e0770 con 0x56466a1c4070 2024-08-31T07:14:10.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4712811740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56466a4cfd00 con 0x56466a19b1b0 2024-08-31T07:14:10.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4712811740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56466a4d3cb0 con 0x56466a1a3390 2024-08-31T07:14:10.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4709200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a19b1b0 0x56466a569e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4709c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 0x56466a569950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4709200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a19b1b0 0x56466a569e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34368/0 (socket says 172.21.15.32:34368) 2024-08-31T07:14:10.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4709200640 1 -- 172.21.15.32:0/3049894680 learned_addr learned my addr 172.21.15.32:0/3049894680 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:10.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.552+0000 7f4703e00640 1 --2- 172.21.15.32:0/3049894680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1a3390 0x56466a56a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.553+0000 7f4703e00640 1 -- 172.21.15.32:0/3049894680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a19b1b0 msgr2=0x56466a569e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.553+0000 7f4703e00640 1 --2- 172.21.15.32:0/3049894680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a19b1b0 0x56466a569e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.553+0000 7f4703e00640 1 -- 172.21.15.32:0/3049894680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 msgr2=0x56466a569950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.553+0000 7f4703e00640 1 --2- 172.21.15.32:0/3049894680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 0x56466a569950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.553+0000 7f4703e00640 1 -- 172.21.15.32:0/3049894680 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56466a22f650 con 0x56466a1a3390 2024-08-31T07:14:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.553+0000 7f4709c00640 1 --2- 172.21.15.32:0/3049894680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 0x56466a569950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.553+0000 7f4703e00640 1 --2- 172.21.15.32:0/3049894680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1a3390 0x56466a56a3d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009230 tx=0x7f46f0004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6be7f053e1082b36 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.554+0000 7f46f6000640 1 -- 172.21.15.32:0/3049894680 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0035040 con 0x56466a1a3390 2024-08-31T07:14:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.554+0000 7f46f6000640 1 -- 172.21.15.32:0/3049894680 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f0004770 con 0x56466a1a3390 2024-08-31T07:14:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.554+0000 7f46f6000640 1 -- 172.21.15.32:0/3049894680 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f00038e0 con 0x56466a1a3390 2024-08-31T07:14:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.554+0000 7f4701600640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46ec0b4ca0 0x7f46ec0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.554+0000 7f4709c00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46ec0b4ca0 0x7f46ec0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.554+0000 7f4701600640 1 -- 172.21.15.32:0/4050897369 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f46f80e57a0 con 0x7f4704103910 2024-08-31T07:14:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.555+0000 7f4712811740 1 -- 172.21.15.32:0/3049894680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1a3390 msgr2=0x56466a56a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.555+0000 7f4712811740 1 --2- 172.21.15.32:0/3049894680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1a3390 0x56466a56a3d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009230 tx=0x7f46f0004470 comp rx=0 tx=0).stop 2024-08-31T07:14:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.556+0000 7f4712811740 1 -- 172.21.15.32:0/3049894680 shutdown_connections 2024-08-31T07:14:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.556+0000 7f4712811740 1 --2- 172.21.15.32:0/3049894680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1a3390 0x56466a56a3d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.556+0000 7f4712811740 1 --2- 172.21.15.32:0/3049894680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a19b1b0 0x56466a569e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.556+0000 7f4712811740 1 --2- 172.21.15.32:0/3049894680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 0x56466a569950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.556+0000 7f4712811740 1 -- 172.21.15.32:0/3049894680 >> 172.21.15.32:0/3049894680 conn(0x56466a1b8460 msgr2=0x56466a567840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.556+0000 7f4712811740 1 -- 172.21.15.32:0/3049894680 shutdown_connections 2024-08-31T07:14:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.556+0000 7f4712811740 1 -- 172.21.15.32:0/3049894680 wait complete. 2024-08-31T07:14:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.556+0000 7f4709c00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46ec0b4ca0 0x7f46ec0b71b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009fd0 tx=0x7f46fc009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.556+0000 7f4712811740 1 -- start start 2024-08-31T07:14:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.557+0000 7f4712811740 1 Processor -- start 2024-08-31T07:14:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.557+0000 7f4712811740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 0x56466a5755a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.557+0000 7f4712811740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a575b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.557+0000 7f4712811740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 0x56466a5813c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.557+0000 7f4712811740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56466a332700 con 0x56466a19b1b0 2024-08-31T07:14:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.557+0000 7f4712811740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56466a456cb0 con 0x56466a1a3390 2024-08-31T07:14:10.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.557+0000 7f4712811740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56466a5780d0 con 0x56466a1c4070 2024-08-31T07:14:10.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.557+0000 7f4712811740 10 client.0 _ll_register_callbacks cb 0x56466a47c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:10.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4709c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 0x56466a5813c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4709200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 0x56466a5755a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4709c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 0x56466a5813c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35600/0 (socket says 172.21.15.32:35600) 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4709c00640 1 -- 172.21.15.32:0/2213323822 learned_addr learned my addr 172.21.15.32:0/2213323822 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4703e00640 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a575b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4709c00640 1 -- 172.21.15.32:0/2213323822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 msgr2=0x56466a575b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4709c00640 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a575b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4709c00640 1 -- 172.21.15.32:0/2213323822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 msgr2=0x56466a5755a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4709c00640 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 0x56466a5755a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4709c00640 1 -- 172.21.15.32:0/2213323822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56466a497870 con 0x56466a1c4070 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f4709c00640 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 0x56466a5813c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00f7a0 tx=0x7f46fc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cfb0152d23c3a5f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.558+0000 7f46eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.559+0000 7f46eaa00640 1 -- 172.21.15.32:0/2213323822 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc01f9b0 con 0x56466a1c4070 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.559+0000 7f4712811740 1 -- 172.21.15.32:0/2213323822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f0004770 con 0x56466a1c4070 2024-08-31T07:14:10.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.559+0000 7f46eaa00640 1 -- 172.21.15.32:0/2213323822 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f46fc01fb50 con 0x56466a1c4070 2024-08-31T07:14:10.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.559+0000 7f46eaa00640 1 -- 172.21.15.32:0/2213323822 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f46fc01fd50 con 0x56466a1c4070 2024-08-31T07:14:10.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.559+0000 7f4712811740 10 client.25933 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.559+0000 7f4712811740 1 -- 172.21.15.32:0/2213323822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56466a5841e0 con 0x56466a1c4070 2024-08-31T07:14:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.560+0000 7f46eaa00640 1 -- 172.21.15.32:0/2213323822 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f0004770 con 0x56466a1c4070 2024-08-31T07:14:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.560+0000 7f46eaa00640 1 -- 172.21.15.32:0/2213323822 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f46fc028480 con 0x56466a1c4070 2024-08-31T07:14:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.560+0000 7f4712811740 10 client.25933 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.560+0000 7f4712811740 10 client.25933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:14:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.560+0000 7f4712811740 20 client.25933 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.560+0000 7f4712811740 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56466a1829f0 0x56466a587690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.560+0000 7f4712811740 4 client.25933 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.560+0000 7f4712811740 1 -- 172.21.15.32:0/2213323822 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56466a4e26c0 con 0x56466a1829f0 2024-08-31T07:14:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.561+0000 7f4709200640 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56466a1829f0 0x56466a587690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.561+0000 7f46eaa00640 10 client.25933 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:10.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.561+0000 7f4709200640 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56466a1829f0 0x56466a587690 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9970f1c1265e4f5f server_cookie=1ec5e573299ecf1b in_seq=0 out_seq=0 2024-08-31T07:14:10.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.562+0000 7f46eaa00640 1 -- 172.21.15.32:0/2213323822 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56466a4e26c0 con 0x56466a1829f0 2024-08-31T07:14:10.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.562+0000 7f46eaa00640 10 client.25933 handle_command_reply: tid=0 2024-08-31T07:14:10.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.562+0000 7f4712811740 1 client.25933 shutdown 2024-08-31T07:14:10.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 1 -- 172.21.15.32:0/2213323822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 msgr2=0x56466a5813c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:10.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 0x56466a5813c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00f7a0 tx=0x7f46fc004a40 comp rx=0 tx=0).stop 2024-08-31T07:14:10.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 1 -- 172.21.15.32:0/2213323822 shutdown_connections 2024-08-31T07:14:10.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56466a1829f0 0x56466a587690 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 0x56466a5813c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a575b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 1 --2- 172.21.15.32:0/2213323822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 0x56466a5755a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 1 -- 172.21.15.32:0/2213323822 >> 172.21.15.32:0/2213323822 conn(0x56466a1b8460 msgr2=0x56466a56cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f46eaa00640 0 client.25933 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 1 -- 172.21.15.32:0/2213323822 shutdown_connections 2024-08-31T07:14:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 1 -- 172.21.15.32:0/2213323822 wait complete. 2024-08-31T07:14:10.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.563+0000 7f4712811740 20 client.25933 trim_cache size 0 max 16384 2024-08-31T07:14:10.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.582+0000 7f4712811740 1 Processor -- start 2024-08-31T07:14:10.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.582+0000 7f4712811740 1 -- start start 2024-08-31T07:14:10.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4712811740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 0x56466a56ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4712811740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a56f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4712811740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 0x56466a566df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4712811740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f47041b1520 con 0x56466a19b1b0 2024-08-31T07:14:10.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4712811740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47041b3060 con 0x56466a1a3390 2024-08-31T07:14:10.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4712811740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47041b1bf0 con 0x56466a1c4070 2024-08-31T07:14:10.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4709200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 0x56466a56ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4709200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 0x56466a56ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52316/0 (socket says 172.21.15.32:52316) 2024-08-31T07:14:10.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4709200640 1 -- 172.21.15.32:0/2385039237 learned_addr learned my addr 172.21.15.32:0/2385039237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:10.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4703e00640 1 --2- 172.21.15.32:0/2385039237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a56f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4709c00640 1 --2- 172.21.15.32:0/2385039237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 0x56466a566df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4709200640 1 -- 172.21.15.32:0/2385039237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 msgr2=0x56466a566df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:10.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4709200640 1 --2- 172.21.15.32:0/2385039237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 0x56466a566df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4709200640 1 -- 172.21.15.32:0/2385039237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 msgr2=0x56466a56f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:10.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.583+0000 7f4709200640 1 --2- 172.21.15.32:0/2385039237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a56f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.584+0000 7f4709200640 1 -- 172.21.15.32:0/2385039237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56466a578010 con 0x56466a19b1b0 2024-08-31T07:14:10.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.584+0000 7f4709200640 1 --2- 172.21.15.32:0/2385039237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 0x56466a56ebd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f46f80e3020 tx=0x7f46f80eb550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66559057dd9a3101 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:10.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.584+0000 7f46f5600640 1 -- 172.21.15.32:0/2385039237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f80ab680 con 0x56466a19b1b0 2024-08-31T07:14:10.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.584+0000 7f46f5600640 1 -- 172.21.15.32:0/2385039237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f80aba20 con 0x56466a19b1b0 2024-08-31T07:14:10.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.584+0000 7f46f5600640 1 -- 172.21.15.32:0/2385039237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f80a81a0 con 0x56466a19b1b0 2024-08-31T07:14:10.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.585+0000 7f4712811740 1 -- 172.21.15.32:0/2385039237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 msgr2=0x56466a56ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:10.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.585+0000 7f4712811740 1 --2- 172.21.15.32:0/2385039237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 0x56466a56ebd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f46f80e3020 tx=0x7f46f80eb550 comp rx=0 tx=0).stop 2024-08-31T07:14:10.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.585+0000 7f4712811740 1 -- 172.21.15.32:0/2385039237 shutdown_connections 2024-08-31T07:14:10.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.585+0000 7f4712811740 1 --2- 172.21.15.32:0/2385039237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a1c4070 0x56466a566df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.585+0000 7f4712811740 1 --2- 172.21.15.32:0/2385039237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a56f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.585+0000 7f4712811740 1 --2- 172.21.15.32:0/2385039237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a19b1b0 0x56466a56ebd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.585+0000 7f4712811740 1 -- 172.21.15.32:0/2385039237 >> 172.21.15.32:0/2385039237 conn(0x56466a1b8460 msgr2=0x56466a573c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:10.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.586+0000 7f4712811740 1 -- 172.21.15.32:0/2385039237 shutdown_connections 2024-08-31T07:14:10.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.586+0000 7f4712811740 1 -- 172.21.15.32:0/2385039237 wait complete. 2024-08-31T07:14:10.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.587+0000 7f4712811740 1 -- start start 2024-08-31T07:14:10.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.587+0000 7f4712811740 1 Processor -- start 2024-08-31T07:14:10.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4712811740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a19b1b0 0x56466a57d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4712811740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a57d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4712811740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 0x56466a5782e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4712811740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f47041b2990 con 0x56466a1c4070 2024-08-31T07:14:10.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4712811740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47041b22c0 con 0x56466a1a3390 2024-08-31T07:14:10.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4712811740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47041af500 con 0x56466a19b1b0 2024-08-31T07:14:10.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4712811740 10 client.0 _ll_register_callbacks cb 0x7f46fc00f630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4703e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a57d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4709200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a19b1b0 0x56466a57d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4703e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a57d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34386/0 (socket says 172.21.15.32:34386) 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4703e00640 1 -- 172.21.15.32:0/3934069702 learned_addr learned my addr 172.21.15.32:0/3934069702 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4709200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a19b1b0 0x56466a57d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35620/0 (socket says 172.21.15.32:35620) 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.588+0000 7f4709c00640 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 0x56466a5782e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.589+0000 7f4703e00640 1 -- 172.21.15.32:0/3934069702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a19b1b0 msgr2=0x56466a57d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.589+0000 7f4703e00640 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a19b1b0 0x56466a57d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.589+0000 7f4703e00640 1 -- 172.21.15.32:0/3934069702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 msgr2=0x56466a5782e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.589+0000 7f4703e00640 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 0x56466a5782e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.589+0000 7f4703e00640 1 -- 172.21.15.32:0/3934069702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56466a578820 con 0x56466a1a3390 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.589+0000 7f4703e00640 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a57d8d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f46f002a930 tx=0x7f46f0005310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df49662b9a83823f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:10.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.589+0000 7f46eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:14:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.589+0000 7f46eaa00640 1 -- 172.21.15.32:0/3934069702 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0035040 con 0x56466a1a3390 2024-08-31T07:14:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.589+0000 7f4712811740 1 -- 172.21.15.32:0/3934069702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f80aba20 con 0x56466a1a3390 2024-08-31T07:14:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.589+0000 7f46eaa00640 1 -- 172.21.15.32:0/3934069702 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f46f0004970 con 0x56466a1a3390 2024-08-31T07:14:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f46eaa00640 1 -- 172.21.15.32:0/3934069702 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f46f0004b90 con 0x56466a1a3390 2024-08-31T07:14:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f4712811740 10 client.25937 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f4712811740 1 -- 172.21.15.32:0/3934069702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56466a5f27c0 con 0x56466a1a3390 2024-08-31T07:14:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f46eaa00640 1 -- 172.21.15.32:0/3934069702 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f80aba20 con 0x56466a1a3390 2024-08-31T07:14:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f46eaa00640 1 -- 172.21.15.32:0/3934069702 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f46f00314a0 con 0x56466a1a3390 2024-08-31T07:14:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f4712811740 10 client.25937 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:10.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f4712811740 10 client.25937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:14:10.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f4712811740 20 client.25937 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:10.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f4712811740 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56466a157da0 0x56466a5f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:10.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f4712811740 4 client.25937 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:14:10.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.590+0000 7f4712811740 1 -- 172.21.15.32:0/3934069702 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56466a4de160 con 0x56466a157da0 2024-08-31T07:14:10.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.591+0000 7f4709200640 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56466a157da0 0x56466a5f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:10.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.591+0000 7f4709200640 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56466a157da0 0x56466a5f4b50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0499ccb0085e549 server_cookie=a91603e2245c7ff5 in_seq=0 out_seq=0 2024-08-31T07:14:10.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.591+0000 7f46eaa00640 10 client.25937 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f46eaa00640 1 -- 172.21.15.32:0/3934069702 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x56466a4de160 con 0x56466a157da0 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f46eaa00640 10 client.25937 handle_command_reply: tid=0 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 client.25937 shutdown 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 -- 172.21.15.32:0/3934069702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 msgr2=0x56466a57d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a57d8d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f46f002a930 tx=0x7f46f0005310 comp rx=0 tx=0).stop 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 -- 172.21.15.32:0/3934069702 shutdown_connections 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56466a157da0 0x56466a5f4b50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56466a1c4070 0x56466a5782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56466a1a3390 0x56466a57d8d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 --2- 172.21.15.32:0/3934069702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56466a19b1b0 0x56466a57d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 -- 172.21.15.32:0/3934069702 >> 172.21.15.32:0/3934069702 conn(0x56466a1b8460 msgr2=0x56466a56a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:10.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f46eaa00640 0 client.25937 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:10.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 -- 172.21.15.32:0/3934069702 shutdown_connections 2024-08-31T07:14:10.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 1 -- 172.21.15.32:0/3934069702 wait complete. 2024-08-31T07:14:10.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.592+0000 7f4712811740 20 client.25937 trim_cache size 0 max 16384 2024-08-31T07:14:10.595 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:14:10.595 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:14:10.595 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:14:10.595 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:14:10.595 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "9fcca2ec-b3db-4784-8740-c1e275303481", 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:14:10.596 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:14:10.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:14:10.598 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:14:10.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.593+0000 7f470bc00640 1 -- 172.21.15.32:0/4050897369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46ec0b4ca0 msgr2=0x7f46ec0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:10.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.593+0000 7f470bc00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46ec0b4ca0 0x7f46ec0b71b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009fd0 tx=0x7f46fc009500 comp rx=0 tx=0).stop 2024-08-31T07:14:10.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.593+0000 7f470bc00640 1 -- 172.21.15.32:0/4050897369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4704103910 msgr2=0x7f47040754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:10.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.593+0000 7f470bc00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4704103910 0x7f47040754b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f46f800ea60 tx=0x7f46f800ef20 comp rx=0 tx=0).stop 2024-08-31T07:14:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.593+0000 7f470bc00640 1 -- 172.21.15.32:0/4050897369 shutdown_connections 2024-08-31T07:14:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.593+0000 7f470bc00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46ec0b4ca0 0x7f46ec0b71b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.593+0000 7f470bc00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4704109ba0 0x7f4704075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.593+0000 7f470bc00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4704109250 0x7f47040759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.593+0000 7f470bc00640 1 --2- 172.21.15.32:0/4050897369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4704103910 0x7f47040754b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.593+0000 7f470bc00640 1 -- 172.21.15.32:0/4050897369 >> 172.21.15.32:0/4050897369 conn(0x7f47040ff380 msgr2=0x7f47041017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.594+0000 7f470bc00640 1 -- 172.21.15.32:0/4050897369 shutdown_connections 2024-08-31T07:14:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:10.594+0000 7f470bc00640 1 -- 172.21.15.32:0/4050897369 wait complete. 2024-08-31T07:14:11.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.473+0000 7fe721800640 1 -- 172.21.15.32:0/1950554974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71c1047b0 msgr2=0x7fe71c104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:11.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.473+0000 7fe721800640 1 --2- 172.21.15.32:0/1950554974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71c1047b0 0x7fe71c104bc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009900 tx=0x7fe70c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.473+0000 7fe721800640 1 -- 172.21.15.32:0/1950554974 shutdown_connections 2024-08-31T07:14:11.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.473+0000 7fe721800640 1 --2- 172.21.15.32:0/1950554974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c10dbe0 0x7fe71c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.473+0000 7fe721800640 1 --2- 172.21.15.32:0/1950554974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe71c105100 0x7fe71c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.473+0000 7fe721800640 1 --2- 172.21.15.32:0/1950554974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71c1047b0 0x7fe71c104bc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.473+0000 7fe721800640 1 -- 172.21.15.32:0/1950554974 >> 172.21.15.32:0/1950554974 conn(0x7fe71c0794c0 msgr2=0x7fe71c0798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:11.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.473+0000 7fe721800640 1 -- 172.21.15.32:0/1950554974 shutdown_connections 2024-08-31T07:14:11.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.473+0000 7fe721800640 1 -- 172.21.15.32:0/1950554974 wait complete. 2024-08-31T07:14:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.474+0000 7fe721800640 1 Processor -- start 2024-08-31T07:14:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.474+0000 7fe721800640 1 -- start start 2024-08-31T07:14:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.474+0000 7fe721800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1047b0 0x7fe71c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.474+0000 7fe721800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71c105100 0x7fe71c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.474+0000 7fe721800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe71c10dbe0 0x7fe71c1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.474+0000 7fe721800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c113840 con 0x7fe71c105100 2024-08-31T07:14:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.474+0000 7fe721800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1136c0 con 0x7fe71c1047b0 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.474+0000 7fe721800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1139c0 con 0x7fe71c10dbe0 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1047b0 0x7fe71c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71c105100 0x7fe71c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1047b0 0x7fe71c19c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34408/0 (socket says 172.21.15.32:34408) 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71aa00640 1 -- 172.21.15.32:0/546136937 learned_addr learned my addr 172.21.15.32:0/546136937 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71b400640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe71c10dbe0 0x7fe71c1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71aa00640 1 -- 172.21.15.32:0/546136937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe71c10dbe0 msgr2=0x7fe71c1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71aa00640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe71c10dbe0 0x7fe71c1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71aa00640 1 -- 172.21.15.32:0/546136937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71c105100 msgr2=0x7fe71c19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71aa00640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71c105100 0x7fe71c19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71aa00640 1 -- 172.21.15.32:0/546136937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c1a2380 con 0x7fe71c1047b0 2024-08-31T07:14:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.475+0000 7fe71a000640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71c105100 0x7fe71c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.476+0000 7fe71aa00640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1047b0 0x7fe71c19c410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe70c005940 tx=0x7fe70c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.476+0000 7fe713400640 1 -- 172.21.15.32:0/546136937 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c03f030 con 0x7fe71c1047b0 2024-08-31T07:14:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.476+0000 7fe721800640 1 -- 172.21.15.32:0/546136937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c1a2610 con 0x7fe71c1047b0 2024-08-31T07:14:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.476+0000 7fe713400640 1 -- 172.21.15.32:0/546136937 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe70c0044c0 con 0x7fe71c1047b0 2024-08-31T07:14:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.476+0000 7fe713400640 1 -- 172.21.15.32:0/546136937 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c035070 con 0x7fe71c1047b0 2024-08-31T07:14:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.476+0000 7fe721800640 1 -- 172.21.15.32:0/546136937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c1a8dd0 con 0x7fe71c1047b0 2024-08-31T07:14:11.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.479+0000 7fe72860c740 1 Processor -- start 2024-08-31T07:14:11.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.479+0000 7fe72860c740 1 -- start start 2024-08-31T07:14:11.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe72860c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe72860c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0ddf660 0x55c5e11ae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe72860c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0de7840 0x55c5e11ae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe72860c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c5e1124a90 con 0x55c5e0de7840 2024-08-31T07:14:11.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe72860c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c5e1114020 con 0x55c5e0ddf660 2024-08-31T07:14:11.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe72860c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c5e1117fd0 con 0x55c5e0e08520 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe71a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0de7840 0x55c5e11ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe71b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe71a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0de7840 0x55c5e11ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52342/0 (socket says 172.21.15.32:52342) 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe71a000640 1 -- 172.21.15.32:0/2168875322 learned_addr learned my addr 172.21.15.32:0/2168875322 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.480+0000 7fe713400640 1 -- 172.21.15.32:0/546136937 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe70c042030 con 0x7fe71c1047b0 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.481+0000 7fe71a000640 1 -- 172.21.15.32:0/2168875322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 msgr2=0x55c5e11adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.481+0000 7fe71a000640 1 --2- 172.21.15.32:0/2168875322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.481+0000 7fe71a000640 1 -- 172.21.15.32:0/2168875322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0ddf660 msgr2=0x55c5e11ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.481+0000 7fe71a000640 1 --2- 172.21.15.32:0/2168875322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0ddf660 0x55c5e11ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.481+0000 7fe71a000640 1 -- 172.21.15.32:0/2168875322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e0e73b00 con 0x55c5e0de7840 2024-08-31T07:14:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.481+0000 7fe71a000640 1 --2- 172.21.15.32:0/2168875322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0de7840 0x55c5e11ae970 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe708011470 tx=0x7fe708011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4948313a20103a88 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:11.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.481+0000 7fe707e00640 1 -- 172.21.15.32:0/2168875322 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70801e070 con 0x55c5e0de7840 2024-08-31T07:14:11.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.481+0000 7fe707e00640 1 -- 172.21.15.32:0/2168875322 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe708004310 con 0x55c5e0de7840 2024-08-31T07:14:11.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.482+0000 7fe707e00640 1 -- 172.21.15.32:0/2168875322 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70800f9a0 con 0x55c5e0de7840 2024-08-31T07:14:11.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.482+0000 7fe713400640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7000b4d20 0x7fe7000b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.482+0000 7fe72860c740 1 -- 172.21.15.32:0/2168875322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0de7840 msgr2=0x55c5e11ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.482+0000 7fe72860c740 1 --2- 172.21.15.32:0/2168875322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0de7840 0x55c5e11ae970 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe708011470 tx=0x7fe708011930 comp rx=0 tx=0).stop 2024-08-31T07:14:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.483+0000 7fe72860c740 1 -- 172.21.15.32:0/2168875322 shutdown_connections 2024-08-31T07:14:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.483+0000 7fe72860c740 1 --2- 172.21.15.32:0/2168875322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0de7840 0x55c5e11ae970 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.483+0000 7fe72860c740 1 --2- 172.21.15.32:0/2168875322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0ddf660 0x55c5e11ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.483+0000 7fe713400640 1 -- 172.21.15.32:0/546136937 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe70c105890 con 0x7fe71c1047b0 2024-08-31T07:14:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.483+0000 7fe72860c740 1 --2- 172.21.15.32:0/2168875322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.483+0000 7fe72860c740 1 -- 172.21.15.32:0/2168875322 >> 172.21.15.32:0/2168875322 conn(0x55c5e0dfc910 msgr2=0x55c5e11abde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.483+0000 7fe72860c740 1 -- 172.21.15.32:0/2168875322 shutdown_connections 2024-08-31T07:14:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.483+0000 7fe71b400640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7000b4d20 0x7fe7000b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.483+0000 7fe72860c740 1 -- 172.21.15.32:0/2168875322 wait complete. 2024-08-31T07:14:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.484+0000 7fe72860c740 1 -- start start 2024-08-31T07:14:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.484+0000 7fe71b400640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7000b4d20 0x7fe7000b7230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe714005fd0 tx=0x7fe714009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.484+0000 7fe72860c740 1 Processor -- start 2024-08-31T07:14:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe72860c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe72860c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11b9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe72860c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe72860c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c5e0f76bb0 con 0x55c5e0ddf660 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe72860c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c5e109af90 con 0x55c5e0de7840 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe72860c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c5e11bc720 con 0x55c5e0e08520 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe72860c740 10 client.0 _ll_register_callbacks cb 0x55c5e10c0710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe71aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe71aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11b9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52356/0 (socket says 172.21.15.32:52356) 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe71aa00640 1 -- 172.21.15.32:0/2033741113 learned_addr learned my addr 172.21.15.32:0/2033741113 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe71a000640 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.485+0000 7fe71b400640 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe71aa00640 1 -- 172.21.15.32:0/2033741113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 msgr2=0x55c5e11b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe71aa00640 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe71aa00640 1 -- 172.21.15.32:0/2033741113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 msgr2=0x55c5e11b9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe71aa00640 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11b9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe71aa00640 1 -- 172.21.15.32:0/2033741113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e0eea960 con 0x55c5e0ddf660 2024-08-31T07:14:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe71aa00640 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11b9250 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0c92c0 tx=0x7fe70c0cba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756c746470ead33e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe704c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:14:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe704c00640 1 -- 172.21.15.32:0/2033741113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c049040 con 0x55c5e0ddf660 2024-08-31T07:14:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe704c00640 1 -- 172.21.15.32:0/2033741113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe70c0d0420 con 0x55c5e0ddf660 2024-08-31T07:14:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe72860c740 1 -- 172.21.15.32:0/2033741113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5e10b2e30 con 0x55c5e0ddf660 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe704c00640 1 -- 172.21.15.32:0/2033741113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe70c079020 con 0x55c5e0ddf660 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe72860c740 10 client.15939 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.486+0000 7fe72860c740 1 -- 172.21.15.32:0/2033741113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5e11c80d0 con 0x55c5e0ddf660 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.487+0000 7fe704c00640 1 -- 172.21.15.32:0/2033741113 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c5e10b2e30 con 0x55c5e0ddf660 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.487+0000 7fe704c00640 1 -- 172.21.15.32:0/2033741113 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe70c0d4e30 con 0x55c5e0ddf660 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.487+0000 7fe72860c740 10 client.15939 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.487+0000 7fe72860c740 10 client.15939 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.487+0000 7fe72860c740 20 client.15939 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.487+0000 7fe72860c740 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c5e0dc6ea0 0x55c5e11cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.487+0000 7fe72860c740 4 client.15939 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.487+0000 7fe72860c740 1 -- 172.21.15.32:0/2033741113 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5e11269e0 con 0x55c5e0dc6ea0 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.488+0000 7fe71a000640 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c5e0dc6ea0 0x55c5e11cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.488+0000 7fe71a000640 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c5e0dc6ea0 0x55c5e11cb4f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c746467c91adf3 server_cookie=a774faac441932ad in_seq=0 out_seq=0 2024-08-31T07:14:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.488+0000 7fe704c00640 10 client.15939 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.489+0000 7fe704c00640 1 -- 172.21.15.32:0/2033741113 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c5e11269e0 con 0x55c5e0dc6ea0 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.489+0000 7fe704c00640 10 client.15939 handle_command_reply: tid=0 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.489+0000 7fe72860c740 1 client.15939 shutdown 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.489+0000 7fe72860c740 1 -- 172.21.15.32:0/2033741113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 msgr2=0x55c5e11b9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.489+0000 7fe72860c740 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11b9250 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0c92c0 tx=0x7fe70c0cba60 comp rx=0 tx=0).stop 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.489+0000 7fe72860c740 1 -- 172.21.15.32:0/2033741113 shutdown_connections 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.490+0000 7fe72860c740 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c5e0dc6ea0 0x55c5e11cb4f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.490+0000 7fe72860c740 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.490+0000 7fe72860c740 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11b9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.490+0000 7fe72860c740 1 --2- 172.21.15.32:0/2033741113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11b9250 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.490+0000 7fe72860c740 1 -- 172.21.15.32:0/2033741113 >> 172.21.15.32:0/2033741113 conn(0x55c5e0dfc910 msgr2=0x55c5e11b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.490+0000 7fe704c00640 0 client.15939 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.490+0000 7fe72860c740 1 -- 172.21.15.32:0/2033741113 shutdown_connections 2024-08-31T07:14:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.490+0000 7fe72860c740 1 -- 172.21.15.32:0/2033741113 wait complete. 2024-08-31T07:14:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.490+0000 7fe72860c740 20 client.15939 trim_cache size 0 max 16384 2024-08-31T07:14:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.509+0000 7fe72860c740 1 Processor -- start 2024-08-31T07:14:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.509+0000 7fe72860c740 1 -- start start 2024-08-31T07:14:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.509+0000 7fe72860c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11bae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.509+0000 7fe72860c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11b6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.509+0000 7fe72860c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.509+0000 7fe72860c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b1400 con 0x55c5e0ddf660 2024-08-31T07:14:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.509+0000 7fe72860c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b2f40 con 0x55c5e0de7840 2024-08-31T07:14:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.509+0000 7fe72860c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b1ad0 con 0x55c5e0e08520 2024-08-31T07:14:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11bae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11bae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52366/0 (socket says 172.21.15.32:52366) 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71aa00640 1 -- 172.21.15.32:0/941542379 learned_addr learned my addr 172.21.15.32:0/941542379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71a000640 1 --2- 172.21.15.32:0/941542379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71b400640 1 --2- 172.21.15.32:0/941542379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71a000640 1 -- 172.21.15.32:0/941542379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 msgr2=0x55c5e11b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71a000640 1 --2- 172.21.15.32:0/941542379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71a000640 1 -- 172.21.15.32:0/941542379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 msgr2=0x55c5e11bae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71a000640 1 --2- 172.21.15.32:0/941542379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11bae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71a000640 1 -- 172.21.15.32:0/941542379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e11b01e0 con 0x55c5e0de7840 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.510+0000 7fe71a000640 1 --2- 172.21.15.32:0/941542379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11b6440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe708003fd0 tx=0x7fe708009ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb25f48e854a369 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.511+0000 7fe707400640 1 -- 172.21.15.32:0/941542379 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70801e040 con 0x55c5e0de7840 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.511+0000 7fe707400640 1 -- 172.21.15.32:0/941542379 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe70801f040 con 0x55c5e0de7840 2024-08-31T07:14:11.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.511+0000 7fe707400640 1 -- 172.21.15.32:0/941542379 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70800f920 con 0x55c5e0de7840 2024-08-31T07:14:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.512+0000 7fe72860c740 1 -- 172.21.15.32:0/941542379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 msgr2=0x55c5e11b6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.512+0000 7fe72860c740 1 --2- 172.21.15.32:0/941542379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11b6440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe708003fd0 tx=0x7fe708009ce0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.512+0000 7fe72860c740 1 -- 172.21.15.32:0/941542379 shutdown_connections 2024-08-31T07:14:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.512+0000 7fe72860c740 1 --2- 172.21.15.32:0/941542379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0e08520 0x55c5e11b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.512+0000 7fe72860c740 1 --2- 172.21.15.32:0/941542379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11b6440 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.512+0000 7fe72860c740 1 --2- 172.21.15.32:0/941542379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0ddf660 0x55c5e11bae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.512+0000 7fe72860c740 1 -- 172.21.15.32:0/941542379 >> 172.21.15.32:0/941542379 conn(0x55c5e0dfc910 msgr2=0x55c5e11ba770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.512+0000 7fe72860c740 1 -- 172.21.15.32:0/941542379 shutdown_connections 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.512+0000 7fe72860c740 1 -- 172.21.15.32:0/941542379 wait complete. 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.512+0000 7fe72860c740 1 -- start start 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.513+0000 7fe72860c740 1 Processor -- start 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.513+0000 7fe72860c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0ddf660 0x55c5e11be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.513+0000 7fe72860c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.513+0000 7fe72860c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0e08520 0x55c5e1234eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.513+0000 7fe72860c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b2870 con 0x55c5e0e08520 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.513+0000 7fe72860c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b21a0 con 0x55c5e0de7840 2024-08-31T07:14:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.513+0000 7fe72860c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1af3e0 con 0x55c5e0ddf660 2024-08-31T07:14:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.513+0000 7fe72860c740 10 client.0 _ll_register_callbacks cb 0x7fe70c0cbe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.513+0000 7fe71a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe71a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11be620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34448/0 (socket says 172.21.15.32:34448) 2024-08-31T07:14:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe71a000640 1 -- 172.21.15.32:0/1860894396 learned_addr learned my addr 172.21.15.32:0/1860894396 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe71b400640 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0e08520 0x55c5e1234eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe71aa00640 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0ddf660 0x55c5e11be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe71a000640 1 -- 172.21.15.32:0/1860894396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0ddf660 msgr2=0x55c5e11be0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe71a000640 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0ddf660 0x55c5e11be0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe71a000640 1 -- 172.21.15.32:0/1860894396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0e08520 msgr2=0x55c5e1234eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe71a000640 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0e08520 0x55c5e1234eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe71a000640 1 -- 172.21.15.32:0/1860894396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e12353f0 con 0x55c5e0de7840 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe71a000640 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11be620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe708004020 tx=0x7fe708005560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c455f324bce89d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.514+0000 7fe704c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.515+0000 7fe704c00640 1 -- 172.21.15.32:0/1860894396 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70801e040 con 0x55c5e0de7840 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.515+0000 7fe704c00640 1 -- 172.21.15.32:0/1860894396 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe708003a00 con 0x55c5e0de7840 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.515+0000 7fe72860c740 1 -- 172.21.15.32:0/1860894396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70801f040 con 0x55c5e0de7840 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.515+0000 7fe704c00640 1 -- 172.21.15.32:0/1860894396 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe708047050 con 0x55c5e0de7840 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.515+0000 7fe72860c740 10 client.25952 fetch_fsmap learned FSMap version 29 2024-08-31T07:14:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.515+0000 7fe72860c740 1 -- 172.21.15.32:0/1860894396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5e12366b0 con 0x55c5e0de7840 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.515+0000 7fe704c00640 1 -- 172.21.15.32:0/1860894396 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe70801f040 con 0x55c5e0de7840 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.516+0000 7fe704c00640 1 -- 172.21.15.32:0/1860894396 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe7080374f0 con 0x55c5e0de7840 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.516+0000 7fe72860c740 10 client.25952 fetch_fsmap finished waiting for FSMap version 29 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.516+0000 7fe72860c740 10 client.25952 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.516+0000 7fe72860c740 20 client.25952 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.516+0000 7fe72860c740 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c5e0d9c250 0x55c5e1238c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.516+0000 7fe72860c740 4 client.25952 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9cf8d043", "release": true, "await": true}] 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.516+0000 7fe72860c740 1 -- 172.21.15.32:0/1860894396 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9cf8d043", "release": true, "await": true}) -- 0x55c5e1122480 con 0x55c5e0d9c250 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.516+0000 7fe71aa00640 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c5e0d9c250 0x55c5e1238c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.517+0000 7fe71aa00640 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c5e0d9c250 0x55c5e1238c40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4a556697327e93 server_cookie=3ceec692cca3614b in_seq=0 out_seq=0 2024-08-31T07:14:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.517+0000 7fe704c00640 10 client.25952 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:11.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe704c00640 1 -- 172.21.15.32:0/1860894396 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55c5e1122480 con 0x55c5e0d9c250 2024-08-31T07:14:11.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe704c00640 10 client.25952 handle_command_reply: tid=0 2024-08-31T07:14:11.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe72860c740 1 client.25952 shutdown 2024-08-31T07:14:11.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe72860c740 1 -- 172.21.15.32:0/1860894396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 msgr2=0x55c5e11be620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:11.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe72860c740 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11be620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe708004020 tx=0x7fe708005560 comp rx=0 tx=0).stop 2024-08-31T07:14:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe72860c740 1 -- 172.21.15.32:0/1860894396 shutdown_connections 2024-08-31T07:14:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe72860c740 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c5e0d9c250 0x55c5e1238c40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe72860c740 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5e0e08520 0x55c5e1234eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe72860c740 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5e0de7840 0x55c5e11be620 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe72860c740 1 --2- 172.21.15.32:0/1860894396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c5e0ddf660 0x55c5e11be0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe72860c740 1 -- 172.21.15.32:0/1860894396 >> 172.21.15.32:0/1860894396 conn(0x55c5e0dfc910 msgr2=0x55c5e11ba700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe704c00640 0 client.25952 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.521+0000 7fe72860c740 1 -- 172.21.15.32:0/1860894396 shutdown_connections 2024-08-31T07:14:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.522+0000 7fe72860c740 1 -- 172.21.15.32:0/1860894396 wait complete. 2024-08-31T07:14:11.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.522+0000 7fe72860c740 20 client.25952 trim_cache size 0 max 16384 2024-08-31T07:14:11.524 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:14:11.524 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 29, 2024-08-31T07:14:11.524 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:14:11.524 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 8, 2024-08-31T07:14:11.524 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:14:11.524 INFO:teuthology.orchestra.run.smithi032.stdout: "9cf8d043": { 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 8, 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 146.4, 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:14:11.525 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:14:11.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:14:11.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:14:11.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:14:11.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:14:11.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:14:11.526 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.522+0000 7fe721800640 1 -- 172.21.15.32:0/546136937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7000b4d20 msgr2=0x7fe7000b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.522+0000 7fe721800640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7000b4d20 0x7fe7000b7230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe714005fd0 tx=0x7fe714009500 comp rx=0 tx=0).stop 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.522+0000 7fe721800640 1 -- 172.21.15.32:0/546136937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1047b0 msgr2=0x7fe71c19c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.522+0000 7fe721800640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1047b0 0x7fe71c19c410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe70c005940 tx=0x7fe70c0040c0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.523+0000 7fe721800640 1 -- 172.21.15.32:0/546136937 shutdown_connections 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.523+0000 7fe721800640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe7000b4d20 0x7fe7000b7230 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.523+0000 7fe721800640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe71c10dbe0 0x7fe71c1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.523+0000 7fe721800640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71c105100 0x7fe71c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.523+0000 7fe721800640 1 --2- 172.21.15.32:0/546136937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe71c1047b0 0x7fe71c19c410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.523+0000 7fe721800640 1 -- 172.21.15.32:0/546136937 >> 172.21.15.32:0/546136937 conn(0x7fe71c0794c0 msgr2=0x7fe71c10cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.523+0000 7fe721800640 1 -- 172.21.15.32:0/546136937 shutdown_connections 2024-08-31T07:14:11.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:11.523+0000 7fe721800640 1 -- 172.21.15.32:0/546136937 wait complete. 2024-08-31T07:14:11.534 INFO:tasks.workunit.client.0.smithi032.stderr:+ dbench 1 2024-08-31T07:14:11.555 INFO:tasks.workunit.client.0.smithi032.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-08-31T07:14:11.555 INFO:tasks.workunit.client.0.smithi032.stdout: 2024-08-31T07:14:11.555 INFO:tasks.workunit.client.0.smithi032.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-08-31T07:14:11.581 INFO:tasks.workunit.client.0.smithi032.stdout:0 of 1 processes prepared for launch 0 sec 2024-08-31T07:14:11.681 INFO:tasks.workunit.client.0.smithi032.stdout:1 of 1 processes prepared for launch 0 sec 2024-08-31T07:14:11.681 INFO:tasks.workunit.client.0.smithi032.stdout:releasing clients 2024-08-31T07:14:12.352 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:12 smithi032 ceph-mon[17252]: pgmap v411: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-08-31T07:14:12.388 INFO:tasks.cephfs.filesystem:scrub status for tag:9fcca2ec-b3db-4784-8740-c1e275303481 - {'path': '/', 'tag': '9fcca2ec-b3db-4784-8740-c1e275303481', 'options': 'recursive,force'} 2024-08-31T07:14:12.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:12 smithi073 ceph-mon[21995]: pgmap v411: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-08-31T07:14:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:12 smithi084 ceph-mon[22387]: pgmap v411: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-08-31T07:14:12.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 3533 38.38 MB/sec warmup 1 sec latency 6.666 ms 2024-08-31T07:14:12.753 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9cf8d043', total seconds elapsed: 32.5 2024-08-31T07:14:13.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 6545 26.32 MB/sec warmup 2 sec latency 19.645 ms 2024-08-31T07:14:14.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:14 smithi032 ceph-mon[17252]: pgmap v412: 97 pgs: 97 active+clean; 1.5 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-08-31T07:14:14.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:14 smithi073 ceph-mon[21995]: pgmap v412: 97 pgs: 97 active+clean; 1.5 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-08-31T07:14:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:14 smithi084 ceph-mon[22387]: pgmap v412: 97 pgs: 97 active+clean; 1.5 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-08-31T07:14:14.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 9480 22.07 MB/sec warmup 3 sec latency 12.133 ms 2024-08-31T07:14:15.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 12298 20.89 MB/sec warmup 4 sec latency 15.493 ms 2024-08-31T07:14:16.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:16 smithi032 ceph-mon[17252]: pgmap v413: 97 pgs: 97 active+clean; 2.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 95 KiB/s wr, 16 op/s 2024-08-31T07:14:16.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:16 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:14:16.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:16 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:14:16.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:16 smithi073 ceph-mon[21995]: pgmap v413: 97 pgs: 97 active+clean; 2.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 95 KiB/s wr, 16 op/s 2024-08-31T07:14:16.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:16 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:14:16.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:16 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:14:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:16 smithi084 ceph-mon[22387]: pgmap v413: 97 pgs: 97 active+clean; 2.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 95 KiB/s wr, 16 op/s 2024-08-31T07:14:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:16 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:14:16.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:16 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:14:16.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 15174 19.66 MB/sec warmup 5 sec latency 14.276 ms 2024-08-31T07:14:17.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:17 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:14:17.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:17 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:14:17.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:17 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:14:17.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:17 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:14:17.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:17 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:14:17.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:17 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:14:17.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:17 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:14:17.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:17 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:14:17.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:17 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:14:17.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 18217 19.42 MB/sec warmup 6 sec latency 30.878 ms 2024-08-31T07:14:18.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:18 smithi032 ceph-mon[17252]: pgmap v414: 97 pgs: 97 active+clean; 3.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 256 KiB/s wr, 24 op/s 2024-08-31T07:14:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:18 smithi073 ceph-mon[21995]: pgmap v414: 97 pgs: 97 active+clean; 3.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 256 KiB/s wr, 24 op/s 2024-08-31T07:14:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:18 smithi084 ceph-mon[22387]: pgmap v414: 97 pgs: 97 active+clean; 3.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 256 KiB/s wr, 24 op/s 2024-08-31T07:14:18.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 21597 19.34 MB/sec warmup 7 sec latency 12.134 ms 2024-08-31T07:14:19.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 24499 18.61 MB/sec warmup 8 sec latency 13.886 ms 2024-08-31T07:14:20.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:20 smithi032 ceph-mon[17252]: pgmap v415: 97 pgs: 97 active+clean; 8.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 703 KiB/s wr, 93 op/s 2024-08-31T07:14:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:20 smithi073 ceph-mon[21995]: pgmap v415: 97 pgs: 97 active+clean; 8.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 703 KiB/s wr, 93 op/s 2024-08-31T07:14:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:20 smithi084 ceph-mon[22387]: pgmap v415: 97 pgs: 97 active+clean; 8.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 703 KiB/s wr, 93 op/s 2024-08-31T07:14:20.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 27315 18.00 MB/sec warmup 9 sec latency 20.180 ms 2024-08-31T07:14:21.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 30121 17.92 MB/sec warmup 10 sec latency 11.350 ms 2024-08-31T07:14:22.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:22 smithi032 ceph-mon[17252]: pgmap v416: 97 pgs: 97 active+clean; 9.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 856 KiB/s wr, 101 op/s 2024-08-31T07:14:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:22 smithi073 ceph-mon[21995]: pgmap v416: 97 pgs: 97 active+clean; 9.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 856 KiB/s wr, 101 op/s 2024-08-31T07:14:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:22 smithi084 ceph-mon[22387]: pgmap v416: 97 pgs: 97 active+clean; 9.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 856 KiB/s wr, 101 op/s 2024-08-31T07:14:22.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 33068 17.73 MB/sec warmup 11 sec latency 10.973 ms 2024-08-31T07:14:23.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 36075 17.68 MB/sec warmup 12 sec latency 14.845 ms 2024-08-31T07:14:24.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:24 smithi032 ceph-mon[17252]: pgmap v417: 97 pgs: 97 active+clean; 14 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-08-31T07:14:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:24 smithi073 ceph-mon[21995]: pgmap v417: 97 pgs: 97 active+clean; 14 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-08-31T07:14:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:24 smithi084 ceph-mon[22387]: pgmap v417: 97 pgs: 97 active+clean; 14 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-08-31T07:14:24.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 39314 17.77 MB/sec warmup 13 sec latency 15.630 ms 2024-08-31T07:14:25.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 42231 17.46 MB/sec warmup 14 sec latency 13.926 ms 2024-08-31T07:14:26.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:26 smithi032 ceph-mon[17252]: pgmap v418: 97 pgs: 97 active+clean; 15 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-08-31T07:14:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:26 smithi073 ceph-mon[21995]: pgmap v418: 97 pgs: 97 active+clean; 15 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-08-31T07:14:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:26 smithi084 ceph-mon[22387]: pgmap v418: 97 pgs: 97 active+clean; 15 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-08-31T07:14:26.682 INFO:tasks.workunit.client.0.smithi032.stdout: 1 44591 17.06 MB/sec warmup 15 sec latency 11.865 ms 2024-08-31T07:14:27.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 47140 16.91 MB/sec warmup 16 sec latency 15.716 ms 2024-08-31T07:14:28.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:28 smithi032 ceph-mon[17252]: pgmap v419: 97 pgs: 97 active+clean; 21 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-08-31T07:14:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:28 smithi073 ceph-mon[21995]: pgmap v419: 97 pgs: 97 active+clean; 21 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-08-31T07:14:28.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:28 smithi084 ceph-mon[22387]: pgmap v419: 97 pgs: 97 active+clean; 21 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-08-31T07:14:28.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 50089 16.98 MB/sec warmup 17 sec latency 13.172 ms 2024-08-31T07:14:29.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 53297 17.02 MB/sec warmup 18 sec latency 12.121 ms 2024-08-31T07:14:30.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:30 smithi032 ceph-mon[17252]: pgmap v420: 97 pgs: 97 active+clean; 21 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.8 MiB/s wr, 239 op/s 2024-08-31T07:14:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:30 smithi073 ceph-mon[21995]: pgmap v420: 97 pgs: 97 active+clean; 21 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.8 MiB/s wr, 239 op/s 2024-08-31T07:14:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:30 smithi084 ceph-mon[22387]: pgmap v420: 97 pgs: 97 active+clean; 21 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.8 MiB/s wr, 239 op/s 2024-08-31T07:14:30.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 55722 16.61 MB/sec warmup 19 sec latency 12.783 ms 2024-08-31T07:14:31.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 58165 16.60 MB/sec warmup 20 sec latency 16.774 ms 2024-08-31T07:14:32.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:32 smithi032 ceph-mon[17252]: pgmap v421: 97 pgs: 97 active+clean; 26 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-08-31T07:14:32.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:32 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:14:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:32 smithi073 ceph-mon[21995]: pgmap v421: 97 pgs: 97 active+clean; 26 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-08-31T07:14:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:32 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:14:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:32 smithi084 ceph-mon[22387]: pgmap v421: 97 pgs: 97 active+clean; 26 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-08-31T07:14:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:32 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:14:32.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 60306 16.40 MB/sec warmup 21 sec latency 373.692 ms 2024-08-31T07:14:33.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 60707 15.73 MB/sec warmup 22 sec latency 1048.821 ms 2024-08-31T07:14:34.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:34 smithi032 ceph-mon[17252]: pgmap v422: 97 pgs: 97 active+clean; 28 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 254 op/s 2024-08-31T07:14:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:34 smithi073 ceph-mon[21995]: pgmap v422: 97 pgs: 97 active+clean; 28 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 254 op/s 2024-08-31T07:14:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:34 smithi084 ceph-mon[22387]: pgmap v422: 97 pgs: 97 active+clean; 28 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 254 op/s 2024-08-31T07:14:34.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 63816 15.80 MB/sec warmup 23 sec latency 15.232 ms 2024-08-31T07:14:35.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 66205 15.50 MB/sec warmup 24 sec latency 16.308 ms 2024-08-31T07:14:36.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:36 smithi032 ceph-mon[17252]: pgmap v423: 97 pgs: 97 active+clean; 29 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-08-31T07:14:36.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:36 smithi073 ceph-mon[21995]: pgmap v423: 97 pgs: 97 active+clean; 29 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-08-31T07:14:36.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:36 smithi084 ceph-mon[22387]: pgmap v423: 97 pgs: 97 active+clean; 29 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-08-31T07:14:36.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68087 15.38 MB/sec warmup 25 sec latency 51.690 ms 2024-08-31T07:14:37.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:37 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:14:37.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:37 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:14:37.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:37 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:14:37.683 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68087 14.78 MB/sec warmup 26 sec latency 1051.798 ms 2024-08-31T07:14:38.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:38 smithi032 ceph-mon[17252]: pgmap v424: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-08-31T07:14:38.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:38 smithi073 ceph-mon[21995]: pgmap v424: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-08-31T07:14:38.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:38 smithi084 ceph-mon[22387]: pgmap v424: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-08-31T07:14:38.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68087 14.24 MB/sec warmup 27 sec latency 2051.918 ms 2024-08-31T07:14:39.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68087 13.73 MB/sec warmup 28 sec latency 3052.027 ms 2024-08-31T07:14:40.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:40 smithi032 ceph-mon[17252]: pgmap v425: 97 pgs: 97 active+clean; 33 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-08-31T07:14:40.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:40 smithi073 ceph-mon[21995]: pgmap v425: 97 pgs: 97 active+clean; 33 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-08-31T07:14:40.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:40 smithi084 ceph-mon[22387]: pgmap v425: 97 pgs: 97 active+clean; 33 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-08-31T07:14:40.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68210 13.27 MB/sec warmup 29 sec latency 3479.151 ms 2024-08-31T07:14:41.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68210 12.82 MB/sec warmup 30 sec latency 1482.879 ms 2024-08-31T07:14:42.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:42 smithi032 ceph-mon[17252]: pgmap v426: 97 pgs: 97 active+clean; 33 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-08-31T07:14:42.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:14:42.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:42 smithi073 ceph-mon[21995]: pgmap v426: 97 pgs: 97 active+clean; 33 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-08-31T07:14:42.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:42 smithi084 ceph-mon[22387]: pgmap v426: 97 pgs: 97 active+clean; 33 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-08-31T07:14:42.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68210 12.41 MB/sec warmup 31 sec latency 2482.956 ms 2024-08-31T07:14:42.837 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.642+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3028297522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810b290 msgr2=0x7f4a9810b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.642+0000 7f4a9cc00640 1 --2- 172.21.15.32:0/3028297522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810b290 0x7f4a9810b6a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009900 tx=0x7f4a8402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.642+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3028297522 shutdown_connections 2024-08-31T07:14:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.642+0000 7f4a9cc00640 1 --2- 172.21.15.32:0/3028297522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a9810bbe0 0x7f4a98112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.642+0000 7f4a9cc00640 1 --2- 172.21.15.32:0/3028297522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810b290 0x7f4a9810b6a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.642+0000 7f4a9cc00640 1 --2- 172.21.15.32:0/3028297522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a98105950 0x7f4a98105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.642+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3028297522 >> 172.21.15.32:0/3028297522 conn(0x7f4a98101420 msgr2=0x7f4a98103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:43.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.642+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3028297522 shutdown_connections 2024-08-31T07:14:43.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.642+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3028297522 wait complete. 2024-08-31T07:14:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.643+0000 7f4a9cc00640 1 Processor -- start 2024-08-31T07:14:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.643+0000 7f4a9cc00640 1 -- start start 2024-08-31T07:14:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.643+0000 7f4a9cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a98105950 0x7f4a9819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.643+0000 7f4a9cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810b290 0x7f4a9819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.643+0000 7f4a9cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9810bbe0 0x7f4a981a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a9cc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981158f0 con 0x7f4a9810b290 2024-08-31T07:14:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a9cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a98115770 con 0x7f4a98105950 2024-08-31T07:14:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a9cc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a98115a70 con 0x7f4a9810bbe0 2024-08-31T07:14:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a96a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810b290 0x7f4a9819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a96a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810b290 0x7f4a9819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53958/0 (socket says 172.21.15.32:53958) 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a96a00640 1 -- 172.21.15.32:0/3272751064 learned_addr learned my addr 172.21.15.32:0/3272751064 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a97400640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a98105950 0x7f4a9819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a97e00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9810bbe0 0x7f4a981a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a96a00640 1 -- 172.21.15.32:0/3272751064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9810bbe0 msgr2=0x7f4a981a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a96a00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9810bbe0 0x7f4a981a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a96a00640 1 -- 172.21.15.32:0/3272751064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a98105950 msgr2=0x7f4a9819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a96a00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a98105950 0x7f4a9819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.644+0000 7f4a96a00640 1 -- 172.21.15.32:0/3272751064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a981a2230 con 0x7f4a9810b290 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.645+0000 7f4a96a00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810b290 0x7f4a9819c860 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4a84005ab0 tx=0x7f4a84003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.645+0000 7f4a8fe00640 1 -- 172.21.15.32:0/3272751064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8403f070 con 0x7f4a9810b290 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.645+0000 7f4a8fe00640 1 -- 172.21.15.32:0/3272751064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a84031b50 con 0x7f4a9810b290 2024-08-31T07:14:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.645+0000 7f4a8fe00640 1 -- 172.21.15.32:0/3272751064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84035030 con 0x7f4a9810b290 2024-08-31T07:14:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.645+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3272751064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a981a24c0 con 0x7f4a9810b290 2024-08-31T07:14:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.645+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3272751064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a981a8c80 con 0x7f4a9810b290 2024-08-31T07:14:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.648+0000 7f4aa500e740 1 Processor -- start 2024-08-31T07:14:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.648+0000 7f4aa500e740 1 -- start start 2024-08-31T07:14:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 0x558b004d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b001031b0 0x558b004d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0010b390 0x558b004d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4aa500e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558b00448770 con 0x558b001031b0 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4aa500e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558b00437d00 con 0x558b0010b390 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4aa500e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558b0043bcb0 con 0x558b0012c070 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4a8fe00640 1 -- 172.21.15.32:0/3272751064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4a84042070 con 0x7f4a9810b290 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 0x558b004d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 0x558b004d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55840/0 (socket says 172.21.15.32:55840) 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4a97e00640 1 -- 172.21.15.32:0/1203321224 learned_addr learned my addr 172.21.15.32:0/1203321224 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4a97400640 1 --2- 172.21.15.32:0/1203321224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b001031b0 0x558b004d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4a96a00640 1 --2- 172.21.15.32:0/1203321224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0010b390 0x558b004d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.649+0000 7f4a97e00640 1 -- 172.21.15.32:0/1203321224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0010b390 msgr2=0x558b004d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.650+0000 7f4a97e00640 1 --2- 172.21.15.32:0/1203321224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0010b390 0x558b004d24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.650+0000 7f4a97e00640 1 -- 172.21.15.32:0/1203321224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b001031b0 msgr2=0x558b004d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.650+0000 7f4a97e00640 1 --2- 172.21.15.32:0/1203321224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b001031b0 0x558b004d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.650+0000 7f4a97e00640 1 -- 172.21.15.32:0/1203321224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b00197650 con 0x558b0012c070 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.650+0000 7f4a96a00640 1 --2- 172.21.15.32:0/1203321224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0010b390 0x558b004d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.650+0000 7f4a97e00640 1 --2- 172.21.15.32:0/1203321224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 0x558b004d1a30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009fd0 tx=0x7f4a900118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c145b3a78917fba8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.650+0000 7f4a83e00640 1 -- 172.21.15.32:0/1203321224 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90004d20 con 0x558b0012c070 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.650+0000 7f4a83e00640 1 -- 172.21.15.32:0/1203321224 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a90004050 con 0x558b0012c070 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.650+0000 7f4a83e00640 1 -- 172.21.15.32:0/1203321224 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90005280 con 0x558b0012c070 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.651+0000 7f4a8fe00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 0x7f4a7c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.651+0000 7f4a8fe00640 1 -- 172.21.15.32:0/3272751064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f4a841059e0 con 0x7f4a9810b290 2024-08-31T07:14:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.651+0000 7f4a97400640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 0x7f4a7c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.651+0000 7f4aa500e740 1 -- 172.21.15.32:0/1203321224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 msgr2=0x558b004d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.651+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1203321224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 0x558b004d1a30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009fd0 tx=0x7f4a900118d0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.651+0000 7f4aa500e740 1 -- 172.21.15.32:0/1203321224 shutdown_connections 2024-08-31T07:14:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.651+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1203321224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0010b390 0x558b004d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.651+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1203321224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b001031b0 0x558b004d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.651+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1203321224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 0x558b004d1a30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.651+0000 7f4aa500e740 1 -- 172.21.15.32:0/1203321224 >> 172.21.15.32:0/1203321224 conn(0x558b00120460 msgr2=0x558b004cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.652+0000 7f4aa500e740 1 -- 172.21.15.32:0/1203321224 shutdown_connections 2024-08-31T07:14:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.652+0000 7f4aa500e740 1 -- 172.21.15.32:0/1203321224 wait complete. 2024-08-31T07:14:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.652+0000 7f4a97400640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 0x7f4a7c0b7230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x558b004d06b0 tx=0x7f4a88005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.652+0000 7f4aa500e740 1 -- start start 2024-08-31T07:14:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.652+0000 7f4aa500e740 1 Processor -- start 2024-08-31T07:14:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b001031b0 0x558b004dd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 0x558b004e9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4aa500e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558b0029a700 con 0x558b0010b390 2024-08-31T07:14:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4aa500e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558b003becb0 con 0x558b0012c070 2024-08-31T07:14:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4aa500e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558b004e0210 con 0x558b001031b0 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4aa500e740 10 client.0 _ll_register_callbacks cb 0x558b003e4430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4a96a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 0x558b004e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4a97400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b001031b0 0x558b004dd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4a96a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004ddae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53974/0 (socket says 172.21.15.32:53974) 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 0x558b004e9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:56758/0 (socket says 172.21.15.32:56758) 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.653+0000 7f4a96a00640 1 -- 172.21.15.32:0/1928066201 learned_addr learned my addr 172.21.15.32:0/1928066201 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.654+0000 7f4a96a00640 1 -- 172.21.15.32:0/1928066201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b001031b0 msgr2=0x558b004dd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.654+0000 7f4a96a00640 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b001031b0 0x558b004dd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.654+0000 7f4a96a00640 1 -- 172.21.15.32:0/1928066201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 msgr2=0x558b004e9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.654+0000 7f4a96a00640 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 0x558b004e9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.654+0000 7f4a96a00640 1 -- 172.21.15.32:0/1928066201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b003ff870 con 0x558b0010b390 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.654+0000 7f4a96a00640 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004ddae0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4a84006010 tx=0x7f4a840cc030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cf1876a5645fb0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.654+0000 7f4a80c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.654+0000 7f4a80c00640 1 -- 172.21.15.32:0/1928066201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84049070 con 0x558b0010b390 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.654+0000 7f4a80c00640 1 -- 172.21.15.32:0/1928066201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f4a841033d0 con 0x558b0010b390 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4aa500e740 1 -- 172.21.15.32:0/1928066201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a90004050 con 0x558b0010b390 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4a80c00640 1 -- 172.21.15.32:0/1928066201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4a841035d0 con 0x558b0010b390 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4aa500e740 10 client.15957 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4aa500e740 1 -- 172.21.15.32:0/1928066201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b004ec110 con 0x558b0010b390 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4a80c00640 1 -- 172.21.15.32:0/1928066201 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a90004050 con 0x558b0010b390 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4a80c00640 1 -- 172.21.15.32:0/1928066201 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f4a840d3ce0 con 0x558b0010b390 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4aa500e740 10 client.15957 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4aa500e740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:14:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4aa500e740 20 client.15957 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558b000ea9f0 0x558b004ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4aa500e740 4 client.15957 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.655+0000 7f4aa500e740 1 -- 172.21.15.32:0/1928066201 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b0044a6c0 con 0x558b000ea9f0 2024-08-31T07:14:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.656+0000 7f4a97400640 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558b000ea9f0 0x558b004ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.656+0000 7f4a97400640 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558b000ea9f0 0x558b004ef5e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2aa63341607b19c server_cookie=775f7b7eddcd7267 in_seq=0 out_seq=0 2024-08-31T07:14:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.656+0000 7f4a80c00640 10 client.15957 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.657+0000 7f4a80c00640 1 -- 172.21.15.32:0/1928066201 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b0044a6c0 con 0x558b000ea9f0 2024-08-31T07:14:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.657+0000 7f4a80c00640 10 client.15957 handle_command_reply: tid=0 2024-08-31T07:14:43.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.657+0000 7f4aa500e740 1 client.15957 shutdown 2024-08-31T07:14:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 1 -- 172.21.15.32:0/1928066201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 msgr2=0x558b004ddae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004ddae0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4a84006010 tx=0x7f4a840cc030 comp rx=0 tx=0).stop 2024-08-31T07:14:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 1 -- 172.21.15.32:0/1928066201 shutdown_connections 2024-08-31T07:14:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558b000ea9f0 0x558b004ef5e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 0x558b004e9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004ddae0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1928066201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b001031b0 0x558b004dd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 1 -- 172.21.15.32:0/1928066201 >> 172.21.15.32:0/1928066201 conn(0x558b00120460 msgr2=0x558b004d4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4a80c00640 0 client.15957 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 1 -- 172.21.15.32:0/1928066201 shutdown_connections 2024-08-31T07:14:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 1 -- 172.21.15.32:0/1928066201 wait complete. 2024-08-31T07:14:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.658+0000 7f4aa500e740 20 client.15957 trim_cache size 0 max 16384 2024-08-31T07:14:43.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.680+0000 7f4aa500e740 1 Processor -- start 2024-08-31T07:14:43.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.680+0000 7f4aa500e740 1 -- start start 2024-08-31T07:14:43.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.681+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b001031b0 0x558b004d8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.681+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.681+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 0x558b004cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.681+0000 7f4aa500e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b13e0 con 0x558b0010b390 2024-08-31T07:14:43.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.681+0000 7f4aa500e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b2fe0 con 0x558b0012c070 2024-08-31T07:14:43.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.681+0000 7f4aa500e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b1ab0 con 0x558b001031b0 2024-08-31T07:14:43.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.681+0000 7f4a96a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.681+0000 7f4a96a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53980/0 (socket says 172.21.15.32:53980) 2024-08-31T07:14:43.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.681+0000 7f4a96a00640 1 -- 172.21.15.32:0/3095563725 learned_addr learned my addr 172.21.15.32:0/3095563725 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:43.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.681+0000 7f4a97e00640 1 --2- 172.21.15.32:0/3095563725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 0x558b004cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.682+0000 7f4a97400640 1 --2- 172.21.15.32:0/3095563725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b001031b0 0x558b004d8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.682+0000 7f4a97e00640 1 -- 172.21.15.32:0/3095563725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b001031b0 msgr2=0x558b004d8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.682+0000 7f4a97e00640 1 --2- 172.21.15.32:0/3095563725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b001031b0 0x558b004d8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.682+0000 7f4a97e00640 1 -- 172.21.15.32:0/3095563725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 msgr2=0x558b004cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.682+0000 7f4a97e00640 1 --2- 172.21.15.32:0/3095563725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.682+0000 7f4a97e00640 1 -- 172.21.15.32:0/3095563725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b004de660 con 0x558b0012c070 2024-08-31T07:14:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.682+0000 7f4a97e00640 1 --2- 172.21.15.32:0/3095563725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 0x558b004cf450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4a90004990 tx=0x7f4a90003a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44238095dd38347 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.683+0000 7f4a83400640 1 -- 172.21.15.32:0/3095563725 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a900049c0 con 0x558b0012c070 2024-08-31T07:14:43.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.683+0000 7f4a83400640 1 -- 172.21.15.32:0/3095563725 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a90004500 con 0x558b0012c070 2024-08-31T07:14:43.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.683+0000 7f4a83400640 1 -- 172.21.15.32:0/3095563725 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a900052a0 con 0x558b0012c070 2024-08-31T07:14:43.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68210 12.02 MB/sec warmup 32 sec latency 3483.003 ms 2024-08-31T07:14:43.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.684+0000 7f4aa500e740 1 -- 172.21.15.32:0/3095563725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 msgr2=0x558b004cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:43.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.684+0000 7f4aa500e740 1 --2- 172.21.15.32:0/3095563725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 0x558b004cf450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4a90004990 tx=0x7f4a90003a30 comp rx=0 tx=0).stop 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.684+0000 7f4aa500e740 1 -- 172.21.15.32:0/3095563725 shutdown_connections 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.684+0000 7f4aa500e740 1 --2- 172.21.15.32:0/3095563725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0012c070 0x558b004cf450 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.684+0000 7f4aa500e740 1 --2- 172.21.15.32:0/3095563725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.684+0000 7f4aa500e740 1 --2- 172.21.15.32:0/3095563725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b001031b0 0x558b004d8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.684+0000 7f4aa500e740 1 -- 172.21.15.32:0/3095563725 >> 172.21.15.32:0/3095563725 conn(0x558b00120460 msgr2=0x558b004dbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.684+0000 7f4aa500e740 1 -- 172.21.15.32:0/3095563725 shutdown_connections 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.684+0000 7f4aa500e740 1 -- 172.21.15.32:0/3095563725 wait complete. 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.684+0000 7f4aa500e740 1 -- start start 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.685+0000 7f4aa500e740 1 Processor -- start 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.685+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b001031b0 0x558b004e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.685+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004e2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.685+0000 7f4aa500e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 0x558b004e2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.685+0000 7f4aa500e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b2850 con 0x558b0010b390 2024-08-31T07:14:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.685+0000 7f4aa500e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b2180 con 0x558b001031b0 2024-08-31T07:14:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.685+0000 7f4aa500e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981af3c0 con 0x558b0012c070 2024-08-31T07:14:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.685+0000 7f4aa500e740 10 client.0 _ll_register_callbacks cb 0x7f4a840c8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a97400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b001031b0 0x558b004e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a97400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b001031b0 0x558b004e5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:56776/0 (socket says 172.21.15.32:56776) 2024-08-31T07:14:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a97400640 1 -- 172.21.15.32:0/1057042329 learned_addr learned my addr 172.21.15.32:0/1057042329 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a96a00640 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a97e00640 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 0x558b004e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a97400640 1 -- 172.21.15.32:0/1057042329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 msgr2=0x558b004e2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a97400640 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 0x558b004e2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a97400640 1 -- 172.21.15.32:0/1057042329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 msgr2=0x558b004e2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a97400640 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004e2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a97400640 1 -- 172.21.15.32:0/1057042329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b004e2d60 con 0x558b001031b0 2024-08-31T07:14:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a97400640 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b001031b0 0x558b004e5c00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4a88012cb0 tx=0x7f4a880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b129c49795c7fd2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.686+0000 7f4a80c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.687+0000 7f4a80c00640 1 -- 172.21.15.32:0/1057042329 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8801db10 con 0x558b001031b0 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.687+0000 7f4a80c00640 1 -- 172.21.15.32:0/1057042329 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f4a8801dcb0 con 0x558b001031b0 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.687+0000 7f4aa500e740 1 -- 172.21.15.32:0/1057042329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a90004500 con 0x558b001031b0 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.687+0000 7f4a80c00640 1 -- 172.21.15.32:0/1057042329 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4a88031470 con 0x558b001031b0 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.687+0000 7f4aa500e740 10 client.25967 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.687+0000 7f4aa500e740 1 -- 172.21.15.32:0/1057042329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b005636f0 con 0x558b001031b0 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.687+0000 7f4a80c00640 1 -- 172.21.15.32:0/1057042329 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a90004500 con 0x558b001031b0 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.688+0000 7f4a80c00640 1 -- 172.21.15.32:0/1057042329 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f4a8803a5c0 con 0x558b001031b0 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.688+0000 7f4aa500e740 10 client.25967 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.688+0000 7f4aa500e740 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:14:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.688+0000 7f4aa500e740 20 client.25967 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.688+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558b000bfda0 0x558b00565f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.688+0000 7f4aa500e740 4 client.25967 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:14:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.688+0000 7f4aa500e740 1 -- 172.21.15.32:0/1057042329 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b00446160 con 0x558b000bfda0 2024-08-31T07:14:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.688+0000 7f4a96a00640 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558b000bfda0 0x558b00565f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.688+0000 7f4a96a00640 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558b000bfda0 0x558b00565f20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92085d927e7c875 server_cookie=cab01b1634c776c1 in_seq=0 out_seq=0 2024-08-31T07:14:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.688+0000 7f4a80c00640 10 client.25967 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4a80c00640 1 -- 172.21.15.32:0/1057042329 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x558b00446160 con 0x558b000bfda0 2024-08-31T07:14:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4a80c00640 10 client.25967 handle_command_reply: tid=0 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 client.25967 shutdown 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 -- 172.21.15.32:0/1057042329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b001031b0 msgr2=0x558b004e5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b001031b0 0x558b004e5c00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4a88012cb0 tx=0x7f4a880046c0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 -- 172.21.15.32:0/1057042329 shutdown_connections 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558b000bfda0 0x558b00565f20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0012c070 0x558b004e2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0010b390 0x558b004e2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 --2- 172.21.15.32:0/1057042329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b001031b0 0x558b004e5c00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4a80c00640 0 client.25967 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 -- 172.21.15.32:0/1057042329 >> 172.21.15.32:0/1057042329 conn(0x558b00120460 msgr2=0x558b004d8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 -- 172.21.15.32:0/1057042329 shutdown_connections 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.689+0000 7f4aa500e740 1 -- 172.21.15.32:0/1057042329 wait complete. 2024-08-31T07:14:43.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4aa500e740 20 client.25967 trim_cache size 0 max 16384 2024-08-31T07:14:43.694 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:14:43.694 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:14:43.694 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:14:43.694 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:14:43.694 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:14:43.694 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:14:43.694 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:14:43.694 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:14:43.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:14:43.702 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:14:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3272751064 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 msgr2=0x7f4a7c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4a9cc00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 0x7f4a7c0b7230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x558b004d06b0 tx=0x7f4a88005f90 comp rx=0 tx=0).stop 2024-08-31T07:14:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3272751064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810b290 msgr2=0x7f4a9819c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4a9cc00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810b290 0x7f4a9819c860 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4a84005ab0 tx=0x7f4a84003b90 comp rx=0 tx=0).stop 2024-08-31T07:14:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3272751064 shutdown_connections 2024-08-31T07:14:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4a9cc00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 0x7f4a7c0b7230 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4a9cc00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a9810bbe0 0x7f4a981a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4a9cc00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810b290 0x7f4a9819c860 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4a9cc00640 1 --2- 172.21.15.32:0/3272751064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a98105950 0x7f4a9819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.690+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3272751064 >> 172.21.15.32:0/3272751064 conn(0x7f4a98101420 msgr2=0x7f4a98103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.691+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3272751064 shutdown_connections 2024-08-31T07:14:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:43.691+0000 7f4a9cc00640 1 -- 172.21.15.32:0/3272751064 wait complete. 2024-08-31T07:14:44.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:44 smithi032 ceph-mon[17252]: pgmap v427: 97 pgs: 97 active+clean; 33 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 824 KiB/s wr, 111 op/s 2024-08-31T07:14:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:44 smithi073 ceph-mon[21995]: pgmap v427: 97 pgs: 97 active+clean; 33 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 824 KiB/s wr, 111 op/s 2024-08-31T07:14:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:44 smithi084 ceph-mon[22387]: pgmap v427: 97 pgs: 97 active+clean; 33 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 824 KiB/s wr, 111 op/s 2024-08-31T07:14:44.612 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:14:44.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:14:44.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68210 11.66 MB/sec warmup 33 sec latency 4483.058 ms 2024-08-31T07:14:44.991 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:45.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.647+0000 7f87c8800640 1 -- 172.21.15.32:0/10830802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c01050a0 msgr2=0x7f87c010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:45.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.647+0000 7f87c8800640 1 --2- 172.21.15.32:0/10830802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c01050a0 0x7f87c010d5f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009900 tx=0x7f87b002a410 comp rx=0 tx=0).stop 2024-08-31T07:14:45.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.648+0000 7f87c8800640 1 -- 172.21.15.32:0/10830802 shutdown_connections 2024-08-31T07:14:45.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.648+0000 7f87c8800640 1 --2- 172.21.15.32:0/10830802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87c010dbb0 0x7f87c01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:45.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.648+0000 7f87c8800640 1 --2- 172.21.15.32:0/10830802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c01050a0 0x7f87c010d5f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:45.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.648+0000 7f87c8800640 1 --2- 172.21.15.32:0/10830802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c0104750 0x7f87c0104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:45.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.648+0000 7f87c8800640 1 -- 172.21.15.32:0/10830802 >> 172.21.15.32:0/10830802 conn(0x7f87c0079500 msgr2=0x7f87c0079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:45.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.648+0000 7f87c8800640 1 -- 172.21.15.32:0/10830802 shutdown_connections 2024-08-31T07:14:45.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.648+0000 7f87c8800640 1 -- 172.21.15.32:0/10830802 wait complete. 2024-08-31T07:14:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.649+0000 7f87c8800640 1 Processor -- start 2024-08-31T07:14:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.649+0000 7f87c8800640 1 -- start start 2024-08-31T07:14:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.649+0000 7f87c8800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c0104750 0x7f87c0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.649+0000 7f87c8800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87c01050a0 0x7f87c01029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.649+0000 7f87c8800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c010dbb0 0x7f87c0102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.649+0000 7f87c8800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87c01137f0 con 0x7f87c0104750 2024-08-31T07:14:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.649+0000 7f87c8800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87c0113670 con 0x7f87c010dbb0 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c8800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f87c0113970 con 0x7f87c01050a0 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c5400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87c01050a0 0x7f87c01029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c5400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87c01050a0 0x7f87c01029d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55888/0 (socket says 172.21.15.32:55888) 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c0104750 0x7f87c0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c5400640 1 -- 172.21.15.32:0/2364827633 learned_addr learned my addr 172.21.15.32:0/2364827633 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c6800640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c010dbb0 0x7f87c0102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c5400640 1 -- 172.21.15.32:0/2364827633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c010dbb0 msgr2=0x7f87c0102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c5400640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c010dbb0 0x7f87c0102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c5400640 1 -- 172.21.15.32:0/2364827633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c0104750 msgr2=0x7f87c0102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c5400640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c0104750 0x7f87c0102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c5400640 1 -- 172.21.15.32:0/2364827633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c01a8520 con 0x7f87c01050a0 2024-08-31T07:14:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.650+0000 7f87c5e00640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c0104750 0x7f87c0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:45.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.651+0000 7f87c5400640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87c01050a0 0x7f87c01029d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f87b002a930 tx=0x7f87b0031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:45.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.651+0000 7f87baa00640 1 -- 172.21.15.32:0/2364827633 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b003f030 con 0x7f87c01050a0 2024-08-31T07:14:45.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.651+0000 7f87c8800640 1 -- 172.21.15.32:0/2364827633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c01a87b0 con 0x7f87c01050a0 2024-08-31T07:14:45.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.651+0000 7f87baa00640 1 -- 172.21.15.32:0/2364827633 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b0004490 con 0x7f87c01050a0 2024-08-31T07:14:45.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.651+0000 7f87baa00640 1 -- 172.21.15.32:0/2364827633 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0035070 con 0x7f87c01050a0 2024-08-31T07:14:45.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.651+0000 7f87c8800640 1 -- 172.21.15.32:0/2364827633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c01a8ce0 con 0x7f87c01050a0 2024-08-31T07:14:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.652+0000 7f87c8800640 1 -- 172.21.15.32:0/2364827633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c006c060 con 0x7f87c01050a0 2024-08-31T07:14:45.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.655+0000 7f87baa00640 1 -- 172.21.15.32:0/2364827633 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f87b0042030 con 0x7f87c01050a0 2024-08-31T07:14:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.657+0000 7f87baa00640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87a80b4d20 0x7f87a80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:45.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.657+0000 7f87baa00640 1 -- 172.21.15.32:0/2364827633 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f87b0105b70 con 0x7f87c01050a0 2024-08-31T07:14:45.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.657+0000 7f87c5e00640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87a80b4d20 0x7f87a80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:45.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.658+0000 7f87c5e00640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87a80b4d20 0x7f87a80b7230 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f87c0103ee0 tx=0x7f87b4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:45.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.658+0000 7f87baa00640 1 -- 172.21.15.32:0/2364827633 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f87b00cc330 con 0x7f87c01050a0 2024-08-31T07:14:45.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68333 11.32 MB/sec warmup 34 sec latency 4916.306 ms 2024-08-31T07:14:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.895+0000 7f87c8800640 1 -- 172.21.15.32:0/2364827633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87c0104b60 con 0x7f87c01050a0 2024-08-31T07:14:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.896+0000 7f87baa00640 1 -- 172.21.15.32:0/2364827633 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11967 (secure 0 0 0) 0x7f87c0104b60 con 0x7f87c01050a0 2024-08-31T07:14:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:14:45.898 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"btime":"2024-08-31T07:14:36:074596+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:14:35.939633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:14:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2024-08-31T07:14:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 -- 172.21.15.32:0/2364827633 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87a80b4d20 msgr2=0x7f87a80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87a80b4d20 0x7f87a80b7230 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f87c0103ee0 tx=0x7f87b4005f90 comp rx=0 tx=0).stop 2024-08-31T07:14:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 -- 172.21.15.32:0/2364827633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87c01050a0 msgr2=0x7f87c01029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87c01050a0 0x7f87c01029d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f87b002a930 tx=0x7f87b0031a20 comp rx=0 tx=0).stop 2024-08-31T07:14:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 -- 172.21.15.32:0/2364827633 shutdown_connections 2024-08-31T07:14:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f87a80b4d20 0x7f87a80b7230 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87c010dbb0 0x7f87c0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87c01050a0 0x7f87c01029d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 --2- 172.21.15.32:0/2364827633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c0104750 0x7f87c0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 -- 172.21.15.32:0/2364827633 >> 172.21.15.32:0/2364827633 conn(0x7f87c0079500 msgr2=0x7f87c01065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 -- 172.21.15.32:0/2364827633 shutdown_connections 2024-08-31T07:14:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:45.899+0000 7f87c8800640 1 -- 172.21.15.32:0/2364827633 wait complete. 2024-08-31T07:14:46.292 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:46 smithi032 ceph-mon[17252]: pgmap v428: 97 pgs: 97 active+clean; 44 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 1.5 MiB/s wr, 91 op/s 2024-08-31T07:14:46.292 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:46 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2364827633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:14:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:46 smithi073 ceph-mon[21995]: pgmap v428: 97 pgs: 97 active+clean; 44 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 1.5 MiB/s wr, 91 op/s 2024-08-31T07:14:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:46 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2364827633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:14:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:46 smithi084 ceph-mon[22387]: pgmap v428: 97 pgs: 97 active+clean; 44 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 1.5 MiB/s wr, 91 op/s 2024-08-31T07:14:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:46 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2364827633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:14:46.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68333 11.00 MB/sec warmup 35 sec latency 1489.943 ms 2024-08-31T07:14:46.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:14:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:47.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68333 10.69 MB/sec warmup 36 sec latency 2490.007 ms 2024-08-31T07:14:48.273 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:48 smithi032 ceph-mon[17252]: pgmap v429: 97 pgs: 97 active+clean; 52 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 2.1 MiB/s wr, 92 op/s 2024-08-31T07:14:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.335+0000 7f799b200640 1 -- 172.21.15.32:0/972162516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c075ea0 msgr2=0x7f799c10f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.335+0000 7f799b200640 1 --2- 172.21.15.32:0/972162516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c075ea0 0x7f799c10f770 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7988009900 tx=0x7f798802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.335+0000 7f799b200640 1 -- 172.21.15.32:0/972162516 shutdown_connections 2024-08-31T07:14:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.335+0000 7f799b200640 1 --2- 172.21.15.32:0/972162516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799c10fcb0 0x7f799c1120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.335+0000 7f799b200640 1 --2- 172.21.15.32:0/972162516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c075ea0 0x7f799c10f770 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.335+0000 7f799b200640 1 --2- 172.21.15.32:0/972162516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f799c075550 0x7f799c075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.335+0000 7f799b200640 1 -- 172.21.15.32:0/972162516 >> 172.21.15.32:0/972162516 conn(0x7f799c0ff1e0 msgr2=0x7f799c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.335+0000 7f799b200640 1 -- 172.21.15.32:0/972162516 shutdown_connections 2024-08-31T07:14:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.335+0000 7f799b200640 1 -- 172.21.15.32:0/972162516 wait complete. 2024-08-31T07:14:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799b200640 1 Processor -- start 2024-08-31T07:14:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799b200640 1 -- start start 2024-08-31T07:14:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799c075550 0x7f799c19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c075ea0 0x7f799c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f799c10fcb0 0x7f799c1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f799c1158b0 con 0x7f799c075ea0 2024-08-31T07:14:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f799c115730 con 0x7f799c10fcb0 2024-08-31T07:14:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f799c115a30 con 0x7f799c075550 2024-08-31T07:14:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799c075550 0x7f799c19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799c075550 0x7f799c19c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45512/0 (socket says 172.21.15.32:45512) 2024-08-31T07:14:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999e00640 1 -- 172.21.15.32:0/3234631009 learned_addr learned my addr 172.21.15.32:0/3234631009 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999400640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c075ea0 0x7f799c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799a800640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f799c10fcb0 0x7f799c1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999e00640 1 -- 172.21.15.32:0/3234631009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f799c10fcb0 msgr2=0x7f799c1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999e00640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f799c10fcb0 0x7f799c1a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999e00640 1 -- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c075ea0 msgr2=0x7f799c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999e00640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c075ea0 0x7f799c19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999e00640 1 -- 172.21.15.32:0/3234631009 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c1a2210 con 0x7f799c075550 2024-08-31T07:14:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999400640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c075ea0 0x7f799c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:14:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7999e00640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799c075550 0x7f799c19c2a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f798c00ea60 tx=0x7f798c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7992a00640 1 -- 172.21.15.32:0/3234631009 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c00cde0 con 0x7f799c075550 2024-08-31T07:14:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799b200640 1 -- 172.21.15.32:0/3234631009 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c1a2500 con 0x7f799c075550 2024-08-31T07:14:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7992a00640 1 -- 172.21.15.32:0/3234631009 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c0035f0 con 0x7f799c075550 2024-08-31T07:14:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f7992a00640 1 -- 172.21.15.32:0/3234631009 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c010690 con 0x7f799c075550 2024-08-31T07:14:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.340+0000 7f799b200640 1 -- 172.21.15.32:0/3234631009 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1a8c60 con 0x7f799c075550 2024-08-31T07:14:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f79a7a06740 1 Processor -- start 2024-08-31T07:14:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f79a7a06740 1 -- start start 2024-08-31T07:14:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f7992a00640 1 -- 172.21.15.32:0/3234631009 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f798c0037a0 con 0x7f799c075550 2024-08-31T07:14:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058edb070 0x561059280b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 0x5610592810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x5610592815e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f79a7a06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5610591f77b0 con 0x561058eb21b0 2024-08-31T07:14:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f79a7a06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5610591e6d40 con 0x561058edb070 2024-08-31T07:14:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f79a7a06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5610591eacf0 con 0x561058eba390 2024-08-31T07:14:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f799a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058edb070 0x561059280b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f799a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058edb070 0x561059280b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59646/0 (socket says 172.21.15.32:59646) 2024-08-31T07:14:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f799a800640 1 -- 172.21.15.32:0/2412044350 learned_addr learned my addr 172.21.15.32:0/2412044350 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f7999400640 1 --2- 172.21.15.32:0/2412044350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x5610592815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f799a800640 1 -- 172.21.15.32:0/2412044350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 msgr2=0x5610592815e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f799a800640 1 --2- 172.21.15.32:0/2412044350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x5610592815e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f799a800640 1 -- 172.21.15.32:0/2412044350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 msgr2=0x5610592810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f799a800640 1 --2- 172.21.15.32:0/2412044350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 0x5610592810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f799a800640 1 -- 172.21.15.32:0/2412044350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561058f46650 con 0x561058edb070 2024-08-31T07:14:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f7999400640 1 --2- 172.21.15.32:0/2412044350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x5610592815e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:48.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f799a800640 1 --2- 172.21.15.32:0/2412044350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058edb070 0x561059280b60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7994009fd0 tx=0x7f79940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b623885c550f8842 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f7987400640 1 -- 172.21.15.32:0/2412044350 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994004d20 con 0x561058edb070 2024-08-31T07:14:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f7987400640 1 -- 172.21.15.32:0/2412044350 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7994004050 con 0x561058edb070 2024-08-31T07:14:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f7987400640 1 -- 172.21.15.32:0/2412044350 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994005280 con 0x561058edb070 2024-08-31T07:14:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f7992a00640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79800b4ca0 0x7f79800b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.345+0000 7f7992a00640 1 -- 172.21.15.32:0/3234631009 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f798c0e5880 con 0x7f799c075550 2024-08-31T07:14:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.350+0000 7f7999e00640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79800b4ca0 0x7f79800b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.350+0000 7f79a7a06740 1 -- 172.21.15.32:0/2412044350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058edb070 msgr2=0x561059280b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.350+0000 7f79a7a06740 1 --2- 172.21.15.32:0/2412044350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058edb070 0x561059280b60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7994009fd0 tx=0x7f79940118d0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.350+0000 7f79a7a06740 1 -- 172.21.15.32:0/2412044350 shutdown_connections 2024-08-31T07:14:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.350+0000 7f79a7a06740 1 --2- 172.21.15.32:0/2412044350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x5610592815e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.350+0000 7f79a7a06740 1 --2- 172.21.15.32:0/2412044350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 0x5610592810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.350+0000 7f79a7a06740 1 --2- 172.21.15.32:0/2412044350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058edb070 0x561059280b60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.350+0000 7f79a7a06740 1 -- 172.21.15.32:0/2412044350 >> 172.21.15.32:0/2412044350 conn(0x561058ecf460 msgr2=0x56105927ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f7999e00640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79800b4ca0 0x7f79800b71b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x56105927f7e0 tx=0x7f798c009520 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 -- 172.21.15.32:0/2412044350 shutdown_connections 2024-08-31T07:14:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 -- 172.21.15.32:0/2412044350 wait complete. 2024-08-31T07:14:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 -- start start 2024-08-31T07:14:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 Processor -- start 2024-08-31T07:14:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eb21b0 0x56105928c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 0x56105928ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 0x561059298630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561059049700 con 0x561058edb070 2024-08-31T07:14:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56105916dcf0 con 0x561058eba390 2024-08-31T07:14:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56105928f340 con 0x561058eb21b0 2024-08-31T07:14:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 10 client.0 _ll_register_callbacks cb 0x561059193470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f7999400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 0x56105928ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f799a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eb21b0 0x56105928c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f799a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eb21b0 0x56105928c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45530/0 (socket says 172.21.15.32:45530) 2024-08-31T07:14:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f799a800640 1 -- 172.21.15.32:0/1973110029 learned_addr learned my addr 172.21.15.32:0/1973110029 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f799a800640 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 0x561059298630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f7999400640 1 -- 172.21.15.32:0/1973110029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eb21b0 msgr2=0x56105928c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f7999400640 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eb21b0 0x56105928c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f7999400640 1 -- 172.21.15.32:0/1973110029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 msgr2=0x561059298630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f7999400640 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 0x561059298630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f7999400640 1 -- 172.21.15.32:0/1973110029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610591ae8b0 con 0x561058eba390 2024-08-31T07:14:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f799a800640 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eb21b0 0x56105928c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f7999400640 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 0x56105928ccc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f798802a910 tx=0x7f7988005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1282da11cf6a93d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f797be00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:14:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f797be00640 1 -- 172.21.15.32:0/1973110029 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988035040 con 0x561058eba390 2024-08-31T07:14:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f797be00640 1 -- 172.21.15.32:0/1973110029 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f79880046c0 con 0x561058eba390 2024-08-31T07:14:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 -- 172.21.15.32:0/1973110029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7994004050 con 0x561058eba390 2024-08-31T07:14:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f797be00640 1 -- 172.21.15.32:0/1973110029 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7988004900 con 0x561058eba390 2024-08-31T07:14:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 10 client.25985 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.352+0000 7f79a7a06740 1 -- 172.21.15.32:0/1973110029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56105929b360 con 0x561058eba390 2024-08-31T07:14:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f797be00640 1 -- 172.21.15.32:0/1973110029 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7994004050 con 0x561058eba390 2024-08-31T07:14:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f797be00640 1 -- 172.21.15.32:0/1973110029 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f7988003c40 con 0x561058eba390 2024-08-31T07:14:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 10 client.25985 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 10 client.25985 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:14:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 20 client.25985 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x561058e999f0 0x56105929e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 4 client.25985 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 -- 172.21.15.32:0/1973110029 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610591f9700 con 0x561058e999f0 2024-08-31T07:14:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f799a800640 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x561058e999f0 0x56105929e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f799a800640 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x561058e999f0 0x56105929e7d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ba9d314461c9a95 server_cookie=374e06e8ea17f661 in_seq=0 out_seq=0 2024-08-31T07:14:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f797be00640 10 client.25985 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:14:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f797be00640 1 -- 172.21.15.32:0/1973110029 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5610591f9700 con 0x561058e999f0 2024-08-31T07:14:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f797be00640 10 client.25985 handle_command_reply: tid=0 2024-08-31T07:14:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 client.25985 shutdown 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 -- 172.21.15.32:0/1973110029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 msgr2=0x56105928ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 0x56105928ccc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f798802a910 tx=0x7f7988005270 comp rx=0 tx=0).stop 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 -- 172.21.15.32:0/1973110029 shutdown_connections 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x561058e999f0 0x56105929e7d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 0x561059298630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 0x56105928ccc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 --2- 172.21.15.32:0/1973110029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eb21b0 0x56105928c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f797be00640 0 client.25985 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 -- 172.21.15.32:0/1973110029 >> 172.21.15.32:0/1973110029 conn(0x561058ecf460 msgr2=0x561059283d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 -- 172.21.15.32:0/1973110029 shutdown_connections 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 1 -- 172.21.15.32:0/1973110029 wait complete. 2024-08-31T07:14:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.357+0000 7f79a7a06740 20 client.25985 trim_cache size 0 max 16384 2024-08-31T07:14:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f79a7a06740 1 Processor -- start 2024-08-31T07:14:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f79a7a06740 1 -- start start 2024-08-31T07:14:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 0x561059286380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 0x561059298530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058edb070 0x561059289770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f79a7a06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f799c1b13b0 con 0x561058eb21b0 2024-08-31T07:14:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f79a7a06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f799c1b2fb0 con 0x561058eba390 2024-08-31T07:14:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f79a7a06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f799c1b1a80 con 0x561058edb070 2024-08-31T07:14:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f799a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 0x561059286380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f7999400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 0x561059298530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f799a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 0x561059286380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43750/0 (socket says 172.21.15.32:43750) 2024-08-31T07:14:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f7999400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 0x561059298530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59660/0 (socket says 172.21.15.32:59660) 2024-08-31T07:14:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f799a800640 1 -- 172.21.15.32:0/1835489760 learned_addr learned my addr 172.21.15.32:0/1835489760 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.385+0000 7f799a800640 1 --2- 172.21.15.32:0/1835489760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058edb070 0x561059289770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f799a800640 1 -- 172.21.15.32:0/1835489760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058edb070 msgr2=0x561059289770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f799a800640 1 --2- 172.21.15.32:0/1835489760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058edb070 0x561059289770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f799a800640 1 -- 172.21.15.32:0/1835489760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 msgr2=0x561059298530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f799a800640 1 --2- 172.21.15.32:0/1835489760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 0x561059298530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f799a800640 1 -- 172.21.15.32:0/1835489760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561059282e50 con 0x561058eb21b0 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f799a800640 1 --2- 172.21.15.32:0/1835489760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 0x561059286380 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7994009f70 tx=0x7f7994009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9379b49568991688 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7986a00640 1 -- 172.21.15.32:0/1835489760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994004a00 con 0x561058eb21b0 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7986a00640 1 -- 172.21.15.32:0/1835489760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7994004ba0 con 0x561058eb21b0 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7986a00640 1 -- 172.21.15.32:0/1835489760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994028450 con 0x561058eb21b0 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 -- 172.21.15.32:0/1835489760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 msgr2=0x561059286380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 --2- 172.21.15.32:0/1835489760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 0x561059286380 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7994009f70 tx=0x7f7994009500 comp rx=0 tx=0).stop 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 -- 172.21.15.32:0/1835489760 shutdown_connections 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 --2- 172.21.15.32:0/1835489760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058edb070 0x561059289770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 --2- 172.21.15.32:0/1835489760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eba390 0x561059298530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 --2- 172.21.15.32:0/1835489760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058eb21b0 0x561059286380 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 -- 172.21.15.32:0/1835489760 >> 172.21.15.32:0/1835489760 conn(0x561058ecf460 msgr2=0x561059285c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 -- 172.21.15.32:0/1835489760 shutdown_connections 2024-08-31T07:14:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 -- 172.21.15.32:0/1835489760 wait complete. 2024-08-31T07:14:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 -- start start 2024-08-31T07:14:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 Processor -- start 2024-08-31T07:14:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eb21b0 0x561059290f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x561059291500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 0x561059302200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f799c1b2820 con 0x561058edb070 2024-08-31T07:14:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f799c1b2150 con 0x561058eb21b0 2024-08-31T07:14:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f799c1af390 con 0x561058eba390 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 10 client.0 _ll_register_callbacks cb 0x7f7988005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7999400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x561059291500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7999400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x561059291500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45550/0 (socket says 172.21.15.32:45550) 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7999400640 1 -- 172.21.15.32:0/2579658200 learned_addr learned my addr 172.21.15.32:0/2579658200 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f799a800640 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eb21b0 0x561059290f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f799a800640 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 0x561059302200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7999400640 1 -- 172.21.15.32:0/2579658200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eb21b0 msgr2=0x561059290f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7999400640 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eb21b0 0x561059290f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7999400640 1 -- 172.21.15.32:0/2579658200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 msgr2=0x561059302200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7999400640 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 0x561059302200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7999400640 1 -- 172.21.15.32:0/2579658200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561059302870 con 0x561058eba390 2024-08-31T07:14:48.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f799a800640 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 0x561059302200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:14:48.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f7999400640 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x561059291500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79880314f0 tx=0x7f79880055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616b0e18f12a48c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:48.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f797be00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:14:48.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f797be00640 1 -- 172.21.15.32:0/2579658200 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988035070 con 0x561058eba390 2024-08-31T07:14:48.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.387+0000 7f79a7a06740 1 -- 172.21.15.32:0/2579658200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7994004ba0 con 0x561058eba390 2024-08-31T07:14:48.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f797be00640 1 -- 172.21.15.32:0/2579658200 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f79880316e0 con 0x561058eba390 2024-08-31T07:14:48.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f797be00640 1 -- 172.21.15.32:0/2579658200 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f798805f050 con 0x561058eba390 2024-08-31T07:14:48.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 10 client.25999 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:48.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 -- 172.21.15.32:0/2579658200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561059309be0 con 0x561058eba390 2024-08-31T07:14:48.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f797be00640 1 -- 172.21.15.32:0/2579658200 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7994004ba0 con 0x561058eba390 2024-08-31T07:14:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f797be00640 1 -- 172.21.15.32:0/2579658200 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f79880035f0 con 0x561058eba390 2024-08-31T07:14:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 10 client.25999 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 10 client.25999 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:14:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 20 client.25999 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x561058e6eda0 0x56105930c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 4 client.25999 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:14:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 -- 172.21.15.32:0/2579658200 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610591f51a0 con 0x561058e6eda0 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799a800640 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x561058e6eda0 0x56105930c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799a800640 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x561058e6eda0 0x56105930c3d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=304ec848c743f4f4 server_cookie=e8d8f5bf547b8730 in_seq=0 out_seq=0 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f797be00640 10 client.25999 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f797be00640 1 -- 172.21.15.32:0/2579658200 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610591f51a0 con 0x561058e6eda0 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f797be00640 10 client.25999 handle_command_reply: tid=0 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 client.25999 shutdown 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 -- 172.21.15.32:0/2579658200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 msgr2=0x561059291500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x561059291500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79880314f0 tx=0x7f79880055f0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 -- 172.21.15.32:0/2579658200 shutdown_connections 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x561058e6eda0 0x56105930c3d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561058edb070 0x561059302200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561058eba390 0x561059291500 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 --2- 172.21.15.32:0/2579658200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561058eb21b0 0x561059290f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f797be00640 0 client.25999 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:14:48.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 -- 172.21.15.32:0/2579658200 >> 172.21.15.32:0/2579658200 conn(0x561058ecf460 msgr2=0x561059281b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:48.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 -- 172.21.15.32:0/2579658200 shutdown_connections 2024-08-31T07:14:48.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 1 -- 172.21.15.32:0/2579658200 wait complete. 2024-08-31T07:14:48.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f79a7a06740 20 client.25999 trim_cache size 0 max 16384 2024-08-31T07:14:48.400 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:14:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 -- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79800b4ca0 msgr2=0x7f79800b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79800b4ca0 0x7f79800b71b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x56105927f7e0 tx=0x7f798c009520 comp rx=0 tx=0).stop 2024-08-31T07:14:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 -- 172.21.15.32:0/3234631009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799c075550 msgr2=0x7f799c19c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799c075550 0x7f799c19c2a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f798c00ea60 tx=0x7f798c00ef20 comp rx=0 tx=0).stop 2024-08-31T07:14:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 -- 172.21.15.32:0/3234631009 shutdown_connections 2024-08-31T07:14:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f79800b4ca0 0x7f79800b71b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f799c10fcb0 0x7f799c1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c075ea0 0x7f799c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 --2- 172.21.15.32:0/3234631009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f799c075550 0x7f799c19c2a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 -- 172.21.15.32:0/3234631009 >> 172.21.15.32:0/3234631009 conn(0x7f799c0ff1e0 msgr2=0x7f799c109830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 -- 172.21.15.32:0/3234631009 shutdown_connections 2024-08-31T07:14:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:48.392+0000 7f799b200640 1 -- 172.21.15.32:0/3234631009 wait complete. 2024-08-31T07:14:48.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:48 smithi073 ceph-mon[21995]: pgmap v429: 97 pgs: 97 active+clean; 52 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 2.1 MiB/s wr, 92 op/s 2024-08-31T07:14:48.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:48 smithi084 ceph-mon[22387]: pgmap v429: 97 pgs: 97 active+clean; 52 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 2.1 MiB/s wr, 92 op/s 2024-08-31T07:14:48.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68333 10.41 MB/sec warmup 37 sec latency 3490.047 ms 2024-08-31T07:14:49.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:14:49.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68333 10.13 MB/sec warmup 38 sec latency 4490.096 ms 2024-08-31T07:14:49.978 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:50.310 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:50 smithi032 ceph-mon[17252]: pgmap v430: 97 pgs: 97 active+clean; 55 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 2.3 MiB/s wr, 89 op/s 2024-08-31T07:14:50.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:50 smithi073 ceph-mon[21995]: pgmap v430: 97 pgs: 97 active+clean; 55 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 2.3 MiB/s wr, 89 op/s 2024-08-31T07:14:50.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:50 smithi084 ceph-mon[22387]: pgmap v430: 97 pgs: 97 active+clean; 55 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 2.3 MiB/s wr, 89 op/s 2024-08-31T07:14:50.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.629+0000 7f5b22600640 1 -- 172.21.15.32:0/2883471394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c105b20 msgr2=0x7f5b1c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:50.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.629+0000 7f5b22600640 1 --2- 172.21.15.32:0/2883471394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c105b20 0x7f5b1c105f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009860 tx=0x7f5b0c02a340 comp rx=0 tx=0).stop 2024-08-31T07:14:50.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.629+0000 7f5b22600640 1 -- 172.21.15.32:0/2883471394 shutdown_connections 2024-08-31T07:14:50.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.629+0000 7f5b22600640 1 --2- 172.21.15.32:0/2883471394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1c10bdb0 0x7f5b1c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.629+0000 7f5b22600640 1 --2- 172.21.15.32:0/2883471394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b1c10b460 0x7f5b1c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.629+0000 7f5b22600640 1 --2- 172.21.15.32:0/2883471394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c105b20 0x7f5b1c105f30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.629+0000 7f5b22600640 1 -- 172.21.15.32:0/2883471394 >> 172.21.15.32:0/2883471394 conn(0x7f5b1c1015b0 msgr2=0x7f5b1c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:50.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.629+0000 7f5b22600640 1 -- 172.21.15.32:0/2883471394 shutdown_connections 2024-08-31T07:14:50.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.629+0000 7f5b22600640 1 -- 172.21.15.32:0/2883471394 wait complete. 2024-08-31T07:14:50.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.630+0000 7f5b22600640 1 Processor -- start 2024-08-31T07:14:50.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.630+0000 7f5b22600640 1 -- start start 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.630+0000 7f5b22600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c105b20 0x7f5b1c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.630+0000 7f5b22600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1c10b460 0x7f5b1c19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b22600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b1c10bdb0 0x7f5b1c1a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b22600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c115a70 con 0x7f5b1c10b460 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b22600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1158f0 con 0x7f5b1c10bdb0 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b22600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c115bf0 con 0x7f5b1c105b20 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1c10b460 0x7f5b1c19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1c10b460 0x7f5b1c19ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43780/0 (socket says 172.21.15.32:43780) 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b13e00640 1 -- 172.21.15.32:0/3029507080 learned_addr learned my addr 172.21.15.32:0/3029507080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b1b400640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c105b20 0x7f5b1c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b1be00640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b1c10bdb0 0x7f5b1c1a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b1b400640 1 -- 172.21.15.32:0/3029507080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b1c10bdb0 msgr2=0x7f5b1c1a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:50.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b1b400640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b1c10bdb0 0x7f5b1c1a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b1b400640 1 -- 172.21.15.32:0/3029507080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1c10b460 msgr2=0x7f5b1c19ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:50.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b1b400640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1c10b460 0x7f5b1c19ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b1b400640 1 -- 172.21.15.32:0/3029507080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c1a23d0 con 0x7f5b1c105b20 2024-08-31T07:14:50.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.631+0000 7f5b1be00640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b1c10bdb0 0x7f5b1c1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:50.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.632+0000 7f5b1b400640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c105b20 0x7f5b1c19c460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c035040 tx=0x7f5b0c032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:50.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.632+0000 7f5b18c00640 1 -- 172.21.15.32:0/3029507080 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c040020 con 0x7f5b1c105b20 2024-08-31T07:14:50.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.632+0000 7f5b22600640 1 -- 172.21.15.32:0/3029507080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1c1a2660 con 0x7f5b1c105b20 2024-08-31T07:14:50.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.632+0000 7f5b18c00640 1 -- 172.21.15.32:0/3029507080 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b0c030e10 con 0x7f5b1c105b20 2024-08-31T07:14:50.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.632+0000 7f5b18c00640 1 -- 172.21.15.32:0/3029507080 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c037040 con 0x7f5b1c105b20 2024-08-31T07:14:50.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.632+0000 7f5b22600640 1 -- 172.21.15.32:0/3029507080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1c1a8e20 con 0x7f5b1c105b20 2024-08-31T07:14:50.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.635+0000 7f5b29257740 1 Processor -- start 2024-08-31T07:14:50.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b29257740 1 -- start start 2024-08-31T07:14:50.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b29257740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8c3070 0x55dd7dc688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b29257740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc68df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b29257740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8a2390 0x55dd7dc69330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b29257740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dd7dbdf7b0 con 0x55dd7d89a1b0 2024-08-31T07:14:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b29257740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dd7dbced40 con 0x55dd7d8a2390 2024-08-31T07:14:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b29257740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dd7dbd2cf0 con 0x55dd7d8c3070 2024-08-31T07:14:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8c3070 0x55dd7dc688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8c3070 0x55dd7dc688b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45576/0 (socket says 172.21.15.32:45576) 2024-08-31T07:14:50.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b1be00640 1 -- 172.21.15.32:0/511868421 learned_addr learned my addr 172.21.15.32:0/511868421 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:50.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.636+0000 7f5b13e00640 1 --2- 172.21.15.32:0/511868421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8a2390 0x55dd7dc69330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.637+0000 7f5b18c00640 1 -- 172.21.15.32:0/3029507080 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5b0c043020 con 0x7f5b1c105b20 2024-08-31T07:14:50.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.637+0000 7f5b1be00640 1 -- 172.21.15.32:0/511868421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8a2390 msgr2=0x55dd7dc69330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:50.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.637+0000 7f5b1be00640 1 --2- 172.21.15.32:0/511868421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8a2390 0x55dd7dc69330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.637+0000 7f5b1be00640 1 -- 172.21.15.32:0/511868421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 msgr2=0x55dd7dc68df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:14:50.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.637+0000 7f5b1be00640 1 --2- 172.21.15.32:0/511868421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc68df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.637+0000 7f5b1be00640 1 -- 172.21.15.32:0/511868421 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd7d92e650 con 0x55dd7d8c3070 2024-08-31T07:14:50.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.637+0000 7f5b1be00640 1 --2- 172.21.15.32:0/511868421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8c3070 0x55dd7dc688b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009fd0 tx=0x7f5b1400ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8632a700b666bf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:50.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.638+0000 7f5b10c00640 1 -- 172.21.15.32:0/511868421 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1401d070 con 0x55dd7d8c3070 2024-08-31T07:14:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.638+0000 7f5b10c00640 1 -- 172.21.15.32:0/511868421 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b14003e20 con 0x55dd7d8c3070 2024-08-31T07:14:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.638+0000 7f5b10c00640 1 -- 172.21.15.32:0/511868421 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b140046a0 con 0x55dd7d8c3070 2024-08-31T07:14:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.638+0000 7f5b18c00640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5b000b4ca0 0x7f5b000b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.638+0000 7f5b29257740 1 -- 172.21.15.32:0/511868421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8c3070 msgr2=0x55dd7dc688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.638+0000 7f5b29257740 1 --2- 172.21.15.32:0/511868421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8c3070 0x55dd7dc688b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009fd0 tx=0x7f5b1400ebc0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b29257740 1 -- 172.21.15.32:0/511868421 shutdown_connections 2024-08-31T07:14:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b29257740 1 --2- 172.21.15.32:0/511868421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8a2390 0x55dd7dc69330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b29257740 1 --2- 172.21.15.32:0/511868421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc68df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b29257740 1 --2- 172.21.15.32:0/511868421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8c3070 0x55dd7dc688b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b18c00640 1 -- 172.21.15.32:0/3029507080 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5b0c104fa0 con 0x7f5b1c105b20 2024-08-31T07:14:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b29257740 1 -- 172.21.15.32:0/511868421 >> 172.21.15.32:0/511868421 conn(0x55dd7d8b7460 msgr2=0x55dd7dc667a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b29257740 1 -- 172.21.15.32:0/511868421 shutdown_connections 2024-08-31T07:14:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b1b400640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5b000b4ca0 0x7f5b000b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b29257740 1 -- 172.21.15.32:0/511868421 wait complete. 2024-08-31T07:14:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b29257740 1 -- start start 2024-08-31T07:14:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.639+0000 7f5b1b400640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5b000b4ca0 0x7f5b000b71b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55dd7dc67530 tx=0x7f5b0c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.640+0000 7f5b29257740 1 Processor -- start 2024-08-31T07:14:50.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.640+0000 7f5b29257740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc74510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.640+0000 7f5b29257740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d8a2390 0x55dd7dc74a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.640+0000 7f5b29257740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8c3070 0x55dd7dc803b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.640+0000 7f5b29257740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dd7da31700 con 0x55dd7d8a2390 2024-08-31T07:14:50.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.640+0000 7f5b29257740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dd7db55cf0 con 0x55dd7d8c3070 2024-08-31T07:14:50.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.640+0000 7f5b29257740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dd7dc77090 con 0x55dd7d89a1b0 2024-08-31T07:14:50.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.640+0000 7f5b29257740 10 client.0 _ll_register_callbacks cb 0x55dd7db7b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:50.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d8a2390 0x55dd7dc74a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc74510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d8a2390 0x55dd7dc74a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43810/0 (socket says 172.21.15.32:43810) 2024-08-31T07:14:50.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b13e00640 1 -- 172.21.15.32:0/1272799394 learned_addr learned my addr 172.21.15.32:0/1272799394 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b1be00640 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8c3070 0x55dd7dc803b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b13e00640 1 -- 172.21.15.32:0/1272799394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d89a1b0 msgr2=0x55dd7dc74510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b13e00640 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc74510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b13e00640 1 -- 172.21.15.32:0/1272799394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8c3070 msgr2=0x55dd7dc803b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b13e00640 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8c3070 0x55dd7dc803b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b13e00640 1 -- 172.21.15.32:0/1272799394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd7db968b0 con 0x55dd7d8a2390 2024-08-31T07:14:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b1be00640 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8c3070 0x55dd7dc803b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:14:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b13e00640 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d8a2390 0x55dd7dc74a90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5b040127d0 tx=0x7f5b04012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47d0b0708cc7af0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.641+0000 7f5b09600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:14:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.642+0000 7f5b09600640 1 -- 172.21.15.32:0/1272799394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04004020 con 0x55dd7d8a2390 2024-08-31T07:14:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.642+0000 7f5b09600640 1 -- 172.21.15.32:0/1272799394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5b040041c0 con 0x55dd7d8a2390 2024-08-31T07:14:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.642+0000 7f5b29257740 1 -- 172.21.15.32:0/1272799394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b14003e20 con 0x55dd7d8a2390 2024-08-31T07:14:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.642+0000 7f5b09600640 1 -- 172.21.15.32:0/1272799394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5b040043c0 con 0x55dd7d8a2390 2024-08-31T07:14:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.642+0000 7f5b29257740 10 client.15990 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.642+0000 7f5b29257740 1 -- 172.21.15.32:0/1272799394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dd7dc83180 con 0x55dd7d8a2390 2024-08-31T07:14:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.642+0000 7f5b09600640 1 -- 172.21.15.32:0/1272799394 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b14003e20 con 0x55dd7d8a2390 2024-08-31T07:14:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.643+0000 7f5b09600640 1 -- 172.21.15.32:0/1272799394 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f5b04003af0 con 0x55dd7d8a2390 2024-08-31T07:14:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.643+0000 7f5b29257740 10 client.15990 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.643+0000 7f5b29257740 10 client.15990 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:14:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.643+0000 7f5b29257740 20 client.15990 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.643+0000 7f5b29257740 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55dd7d8819f0 0x55dd7dc86670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.643+0000 7f5b29257740 4 client.15990 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.643+0000 7f5b29257740 1 -- 172.21.15.32:0/1272799394 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd7dbe1700 con 0x55dd7d8819f0 2024-08-31T07:14:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.643+0000 7f5b1be00640 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55dd7d8819f0 0x55dd7dc86670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.644+0000 7f5b1be00640 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55dd7d8819f0 0x55dd7dc86670 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=43966eb4042367f5 server_cookie=e2d9881ee38ebc6 in_seq=0 out_seq=0 2024-08-31T07:14:50.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.644+0000 7f5b09600640 10 client.15990 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:14:50.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b09600640 1 -- 172.21.15.32:0/1272799394 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dd7dbe1700 con 0x55dd7d8819f0 2024-08-31T07:14:50.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b09600640 10 client.15990 handle_command_reply: tid=0 2024-08-31T07:14:50.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 client.15990 shutdown 2024-08-31T07:14:50.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 -- 172.21.15.32:0/1272799394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d8a2390 msgr2=0x55dd7dc74a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:50.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d8a2390 0x55dd7dc74a90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5b040127d0 tx=0x7f5b04012c90 comp rx=0 tx=0).stop 2024-08-31T07:14:50.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 -- 172.21.15.32:0/1272799394 shutdown_connections 2024-08-31T07:14:50.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55dd7d8819f0 0x55dd7dc86670 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8c3070 0x55dd7dc803b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d8a2390 0x55dd7dc74a90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 --2- 172.21.15.32:0/1272799394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc74510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b09600640 0 client.15990 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:14:50.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 -- 172.21.15.32:0/1272799394 >> 172.21.15.32:0/1272799394 conn(0x55dd7d8b7460 msgr2=0x55dd7dc6ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:50.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 -- 172.21.15.32:0/1272799394 shutdown_connections 2024-08-31T07:14:50.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 1 -- 172.21.15.32:0/1272799394 wait complete. 2024-08-31T07:14:50.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.645+0000 7f5b29257740 20 client.15990 trim_cache size 0 max 16384 2024-08-31T07:14:50.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.659+0000 7f5b29257740 1 Processor -- start 2024-08-31T07:14:50.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.659+0000 7f5b29257740 1 -- start start 2024-08-31T07:14:50.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b29257740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8b7460 0x55dd7dc6d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b29257740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc6df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b29257740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8a2390 0x55dd7dc660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b29257740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1b1580 con 0x55dd7d89a1b0 2024-08-31T07:14:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b29257740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1b3180 con 0x55dd7d8a2390 2024-08-31T07:14:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b29257740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1b1c50 con 0x55dd7d8b7460 2024-08-31T07:14:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc6df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc6df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43816/0 (socket says 172.21.15.32:43816) 2024-08-31T07:14:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8b7460 0x55dd7dc6d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b13e00640 1 -- 172.21.15.32:0/2559569557 learned_addr learned my addr 172.21.15.32:0/2559569557 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b1be00640 1 --2- 172.21.15.32:0/2559569557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8a2390 0x55dd7dc660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b13e00640 1 -- 172.21.15.32:0/2559569557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8b7460 msgr2=0x55dd7dc6d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b13e00640 1 --2- 172.21.15.32:0/2559569557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8b7460 0x55dd7dc6d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b13e00640 1 -- 172.21.15.32:0/2559569557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8a2390 msgr2=0x55dd7dc660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.660+0000 7f5b13e00640 1 --2- 172.21.15.32:0/2559569557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8a2390 0x55dd7dc660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.661+0000 7f5b13e00640 1 -- 172.21.15.32:0/2559569557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd7dc755d0 con 0x55dd7d89a1b0 2024-08-31T07:14:50.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.661+0000 7f5b13e00640 1 --2- 172.21.15.32:0/2559569557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc6df00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009ed0 tx=0x7f5b0401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175e28046fb2386c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:50.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.661+0000 7f5b0be00640 1 -- 172.21.15.32:0/2559569557 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04003fe0 con 0x55dd7d89a1b0 2024-08-31T07:14:50.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.661+0000 7f5b0be00640 1 -- 172.21.15.32:0/2559569557 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b0401ce10 con 0x55dd7d89a1b0 2024-08-31T07:14:50.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.661+0000 7f5b0be00640 1 -- 172.21.15.32:0/2559569557 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04005070 con 0x55dd7d89a1b0 2024-08-31T07:14:50.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.662+0000 7f5b29257740 1 -- 172.21.15.32:0/2559569557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 msgr2=0x55dd7dc6df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:50.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.662+0000 7f5b29257740 1 --2- 172.21.15.32:0/2559569557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc6df00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009ed0 tx=0x7f5b0401c780 comp rx=0 tx=0).stop 2024-08-31T07:14:50.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.662+0000 7f5b29257740 1 -- 172.21.15.32:0/2559569557 shutdown_connections 2024-08-31T07:14:50.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.662+0000 7f5b29257740 1 --2- 172.21.15.32:0/2559569557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8a2390 0x55dd7dc660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.662+0000 7f5b29257740 1 --2- 172.21.15.32:0/2559569557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc6df00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.662+0000 7f5b29257740 1 --2- 172.21.15.32:0/2559569557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8b7460 0x55dd7dc6d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.662+0000 7f5b29257740 1 -- 172.21.15.32:0/2559569557 >> 172.21.15.32:0/2559569557 conn(0x55dd7dc80ba0 msgr2=0x55dd7dc72b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:50.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.662+0000 7f5b29257740 1 -- 172.21.15.32:0/2559569557 shutdown_connections 2024-08-31T07:14:50.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.662+0000 7f5b29257740 1 -- 172.21.15.32:0/2559569557 wait complete. 2024-08-31T07:14:50.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.663+0000 7f5b29257740 1 -- start start 2024-08-31T07:14:50.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.663+0000 7f5b29257740 1 Processor -- start 2024-08-31T07:14:50.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.663+0000 7f5b29257740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc7c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.663+0000 7f5b29257740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8a2390 0x55dd7dc772d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.663+0000 7f5b29257740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8b7460 0x55dd7dc77810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.663+0000 7f5b29257740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1b29f0 con 0x55dd7d89a1b0 2024-08-31T07:14:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.663+0000 7f5b29257740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1b2320 con 0x55dd7d8b7460 2024-08-31T07:14:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.663+0000 7f5b29257740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1af560 con 0x55dd7d8a2390 2024-08-31T07:14:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.663+0000 7f5b29257740 10 client.0 _ll_register_callbacks cb 0x7f5b04012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:50.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.664+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc7c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.664+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc7c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43824/0 (socket says 172.21.15.32:43824) 2024-08-31T07:14:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.664+0000 7f5b1be00640 1 -- 172.21.15.32:0/1043515832 learned_addr learned my addr 172.21.15.32:0/1043515832 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.664+0000 7f5b1be00640 1 -- 172.21.15.32:0/1043515832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8a2390 msgr2=0x55dd7dc772d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.664+0000 7f5b1be00640 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8a2390 0x55dd7dc772d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.664+0000 7f5b1be00640 1 -- 172.21.15.32:0/1043515832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8b7460 msgr2=0x55dd7dc77810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:14:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.664+0000 7f5b1be00640 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8b7460 0x55dd7dc77810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.664+0000 7f5b1be00640 1 -- 172.21.15.32:0/1043515832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd7dc77e80 con 0x55dd7d89a1b0 2024-08-31T07:14:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.664+0000 7f5b1be00640 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc7c9e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b14004340 tx=0x7f5b14003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0ce8870aba5736 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.664+0000 7f5b09600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:14:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.665+0000 7f5b09600640 1 -- 172.21.15.32:0/1043515832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1401d040 con 0x55dd7d89a1b0 2024-08-31T07:14:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.665+0000 7f5b09600640 1 -- 172.21.15.32:0/1043515832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5b1400cce0 con 0x55dd7d89a1b0 2024-08-31T07:14:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.665+0000 7f5b29257740 1 -- 172.21.15.32:0/1043515832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b0401ce10 con 0x55dd7d89a1b0 2024-08-31T07:14:50.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.665+0000 7f5b09600640 1 -- 172.21.15.32:0/1043515832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5b140053e0 con 0x55dd7d89a1b0 2024-08-31T07:14:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.665+0000 7f5b29257740 10 client.15996 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.665+0000 7f5b29257740 1 -- 172.21.15.32:0/1043515832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dd7dcf1e40 con 0x55dd7d89a1b0 2024-08-31T07:14:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.665+0000 7f5b09600640 1 -- 172.21.15.32:0/1043515832 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b0401ce10 con 0x55dd7d89a1b0 2024-08-31T07:14:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.666+0000 7f5b09600640 1 -- 172.21.15.32:0/1043515832 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f5b14023040 con 0x55dd7d89a1b0 2024-08-31T07:14:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.666+0000 7f5b29257740 10 client.15996 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:50.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.666+0000 7f5b29257740 10 client.15996 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:14:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.666+0000 7f5b29257740 20 client.15996 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.666+0000 7f5b29257740 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55dd7d8c3070 0x55dd7dcf4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.666+0000 7f5b29257740 4 client.15996 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:14:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.666+0000 7f5b29257740 1 -- 172.21.15.32:0/1043515832 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd7dbdd1a0 con 0x55dd7d8c3070 2024-08-31T07:14:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.666+0000 7f5b13e00640 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55dd7d8c3070 0x55dd7dcf4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:50.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b13e00640 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55dd7d8c3070 0x55dd7dcf4390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9eee6e67736d4efc server_cookie=671dff52a854b75d in_seq=0 out_seq=0 2024-08-31T07:14:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b09600640 10 client.15996 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:14:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b09600640 1 -- 172.21.15.32:0/1043515832 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd7dbdd1a0 con 0x55dd7d8c3070 2024-08-31T07:14:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b09600640 10 client.15996 handle_command_reply: tid=0 2024-08-31T07:14:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b29257740 1 client.15996 shutdown 2024-08-31T07:14:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b29257740 1 -- 172.21.15.32:0/1043515832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 msgr2=0x55dd7dc7c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b29257740 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc7c9e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b14004340 tx=0x7f5b14003e70 comp rx=0 tx=0).stop 2024-08-31T07:14:50.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b29257740 1 -- 172.21.15.32:0/1043515832 shutdown_connections 2024-08-31T07:14:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b29257740 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55dd7d8c3070 0x55dd7dcf4390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b29257740 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd7d8b7460 0x55dd7dc77810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b29257740 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd7d8a2390 0x55dd7dc772d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b29257740 1 --2- 172.21.15.32:0/1043515832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dd7d89a1b0 0x55dd7dc7c9e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b29257740 1 -- 172.21.15.32:0/1043515832 >> 172.21.15.32:0/1043515832 conn(0x55dd7dc80ba0 msgr2=0x55dd7dc6d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b09600640 0 client.15996 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:14:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.667+0000 7f5b29257740 1 -- 172.21.15.32:0/1043515832 shutdown_connections 2024-08-31T07:14:50.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b29257740 1 -- 172.21.15.32:0/1043515832 wait complete. 2024-08-31T07:14:50.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b29257740 20 client.15996 trim_cache size 0 max 16384 2024-08-31T07:14:50.685 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:14:50.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b22600640 1 -- 172.21.15.32:0/3029507080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5b000b4ca0 msgr2=0x7f5b000b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:50.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b22600640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5b000b4ca0 0x7f5b000b71b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55dd7dc67530 tx=0x7f5b0c005f90 comp rx=0 tx=0).stop 2024-08-31T07:14:50.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b22600640 1 -- 172.21.15.32:0/3029507080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c105b20 msgr2=0x7f5b1c19c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b22600640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c105b20 0x7f5b1c19c460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c035040 tx=0x7f5b0c032d50 comp rx=0 tx=0).stop 2024-08-31T07:14:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b22600640 1 -- 172.21.15.32:0/3029507080 shutdown_connections 2024-08-31T07:14:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b22600640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5b000b4ca0 0x7f5b000b71b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b22600640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b1c10bdb0 0x7f5b1c1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b22600640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b1c10b460 0x7f5b1c19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b22600640 1 --2- 172.21.15.32:0/3029507080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1c105b20 0x7f5b1c19c460 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:50.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.668+0000 7f5b22600640 1 -- 172.21.15.32:0/3029507080 >> 172.21.15.32:0/3029507080 conn(0x7f5b1c1015b0 msgr2=0x7f5b1c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:50.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.669+0000 7f5b22600640 1 -- 172.21.15.32:0/3029507080 shutdown_connections 2024-08-31T07:14:50.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:50.669+0000 7f5b22600640 1 -- 172.21.15.32:0/3029507080 wait complete. 2024-08-31T07:14:50.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68399 9.88 MB/sec warmup 39 sec latency 4942.314 ms 2024-08-31T07:14:51.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:14:51.684 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68399 9.63 MB/sec warmup 40 sec latency 1513.157 ms 2024-08-31T07:14:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:52.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:52 smithi032 ceph-mon[17252]: pgmap v431: 97 pgs: 97 active+clean; 55 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.8 MiB/s wr, 62 op/s 2024-08-31T07:14:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:52 smithi073 ceph-mon[21995]: pgmap v431: 97 pgs: 97 active+clean; 55 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.8 MiB/s wr, 62 op/s 2024-08-31T07:14:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:52 smithi084 ceph-mon[22387]: pgmap v431: 97 pgs: 97 active+clean; 55 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.8 MiB/s wr, 62 op/s 2024-08-31T07:14:52.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.571+0000 7f1c0a200640 1 -- 172.21.15.32:0/623920816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c0410b460 msgr2=0x7f1c0410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:52.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.571+0000 7f1c0a200640 1 --2- 172.21.15.32:0/623920816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c0410b460 0x7f1c0410b870 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009900 tx=0x7f1bec02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.573+0000 7f1c0a200640 1 -- 172.21.15.32:0/623920816 shutdown_connections 2024-08-31T07:14:52.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.573+0000 7f1c0a200640 1 --2- 172.21.15.32:0/623920816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c0410bdb0 0x7f1c041125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.573+0000 7f1c0a200640 1 --2- 172.21.15.32:0/623920816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c0410b460 0x7f1c0410b870 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.573+0000 7f1c0a200640 1 --2- 172.21.15.32:0/623920816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c04105b20 0x7f1c04105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.573+0000 7f1c0a200640 1 -- 172.21.15.32:0/623920816 >> 172.21.15.32:0/623920816 conn(0x7f1c041015b0 msgr2=0x7f1c04103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:52.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.573+0000 7f1c0a200640 1 -- 172.21.15.32:0/623920816 shutdown_connections 2024-08-31T07:14:52.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.573+0000 7f1c0a200640 1 -- 172.21.15.32:0/623920816 wait complete. 2024-08-31T07:14:52.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c0a200640 1 Processor -- start 2024-08-31T07:14:52.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c0a200640 1 -- start start 2024-08-31T07:14:52.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c0a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c04105b20 0x7f1c0419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c0a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c0410b460 0x7f1c0419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c0a200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c0410bdb0 0x7f1c041a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c0a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04115a70 con 0x7f1c0410b460 2024-08-31T07:14:52.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c0a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041158f0 con 0x7f1c04105b20 2024-08-31T07:14:52.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c0a200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04115bf0 con 0x7f1c0410bdb0 2024-08-31T07:14:52.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c03400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c04105b20 0x7f1c0419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c0410bdb0 0x7f1c041a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c03400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c04105b20 0x7f1c0419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59738/0 (socket says 172.21.15.32:59738) 2024-08-31T07:14:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c03400640 1 -- 172.21.15.32:0/4098753414 learned_addr learned my addr 172.21.15.32:0/4098753414 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1bfaa00640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c0410b460 0x7f1c0419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c03e00640 1 -- 172.21.15.32:0/4098753414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c04105b20 msgr2=0x7f1c0419c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c03e00640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c04105b20 0x7f1c0419c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c03e00640 1 -- 172.21.15.32:0/4098753414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c0410b460 msgr2=0x7f1c0419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c03e00640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c0410b460 0x7f1c0419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.574+0000 7f1c03e00640 1 -- 172.21.15.32:0/4098753414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c041a23f0 con 0x7f1c0410bdb0 2024-08-31T07:14:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.575+0000 7f1c03e00640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c0410bdb0 0x7f1c041a1cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009fd0 tx=0x7f1bfc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.575+0000 7f1c00c00640 1 -- 172.21.15.32:0/4098753414 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00ce20 con 0x7f1c0410bdb0 2024-08-31T07:14:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.575+0000 7f1c0a200640 1 -- 172.21.15.32:0/4098753414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c041a26e0 con 0x7f1c0410bdb0 2024-08-31T07:14:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.575+0000 7f1c00c00640 1 -- 172.21.15.32:0/4098753414 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bfc005020 con 0x7f1c0410bdb0 2024-08-31T07:14:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.575+0000 7f1c00c00640 1 -- 172.21.15.32:0/4098753414 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc010690 con 0x7f1c0410bdb0 2024-08-31T07:14:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.575+0000 7f1c0a200640 1 -- 172.21.15.32:0/4098753414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c041a8e40 con 0x7f1c0410bdb0 2024-08-31T07:14:52.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.579+0000 7f1c10fd3740 1 Processor -- start 2024-08-31T07:14:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1c00c00640 1 -- 172.21.15.32:0/4098753414 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1bfc0054d0 con 0x7f1c0410bdb0 2024-08-31T07:14:52.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1c10fd3740 1 -- start start 2024-08-31T07:14:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80455070 0x559d807fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d8042c1b0 0x559d807fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80434390 0x559d807fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559d807717b0 con 0x559d80434390 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559d80760d40 con 0x559d8042c1b0 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559d80764cf0 con 0x559d80455070 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1bfaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80455070 0x559d807fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1bfaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80434390 0x559d807fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1c03400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d8042c1b0 0x559d807fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1bfaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80434390 0x559d807fb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43850/0 (socket says 172.21.15.32:43850) 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1c03400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d8042c1b0 0x559d807fadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59742/0 (socket says 172.21.15.32:59742) 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.580+0000 7f1bfaa00640 1 -- 172.21.15.32:0/919215474 learned_addr learned my addr 172.21.15.32:0/919215474 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1bfaa00640 1 -- 172.21.15.32:0/919215474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80455070 msgr2=0x559d807fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1bfaa00640 1 --2- 172.21.15.32:0/919215474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80455070 0x559d807fa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1bfaa00640 1 -- 172.21.15.32:0/919215474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d8042c1b0 msgr2=0x559d807fadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1bfaa00640 1 --2- 172.21.15.32:0/919215474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d8042c1b0 0x559d807fadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1bfaa00640 1 -- 172.21.15.32:0/919215474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d804c0650 con 0x559d80434390 2024-08-31T07:14:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1c03400640 1 --2- 172.21.15.32:0/919215474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d8042c1b0 0x559d807fadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1bfaa00640 1 --2- 172.21.15.32:0/919215474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80434390 0x559d807fb330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1c0406c840 tx=0x7f1bec031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc28cac8439205a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1bf3e00640 1 -- 172.21.15.32:0/919215474 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec035070 con 0x559d80434390 2024-08-31T07:14:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1bf3e00640 1 -- 172.21.15.32:0/919215474 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bec0318a0 con 0x559d80434390 2024-08-31T07:14:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1bf3e00640 1 -- 172.21.15.32:0/919215474 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec004750 con 0x559d80434390 2024-08-31T07:14:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.581+0000 7f1c00c00640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1be80b4ca0 0x7f1be80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c00c00640 1 -- 172.21.15.32:0/4098753414 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1bfc0e5bd0 con 0x7f1c0410bdb0 2024-08-31T07:14:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c03400640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1be80b4ca0 0x7f1be80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 -- 172.21.15.32:0/919215474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80434390 msgr2=0x559d807fb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/919215474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80434390 0x559d807fb330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1c0406c840 tx=0x7f1bec031400 comp rx=0 tx=0).stop 2024-08-31T07:14:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 -- 172.21.15.32:0/919215474 shutdown_connections 2024-08-31T07:14:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/919215474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80434390 0x559d807fb330 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/919215474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d8042c1b0 0x559d807fadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/919215474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80455070 0x559d807fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 -- 172.21.15.32:0/919215474 >> 172.21.15.32:0/919215474 conn(0x559d80449460 msgr2=0x559d807f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:52.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c03400640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1be80b4ca0 0x7f1be80b71b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x559d807f9530 tx=0x7f1bf4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:52.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 -- 172.21.15.32:0/919215474 shutdown_connections 2024-08-31T07:14:52.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 -- 172.21.15.32:0/919215474 wait complete. 2024-08-31T07:14:52.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 -- start start 2024-08-31T07:14:52.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.583+0000 7f1c10fd3740 1 Processor -- start 2024-08-31T07:14:52.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d808064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80434390 0x559d80806a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d80812320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559d805c3700 con 0x559d8042c1b0 2024-08-31T07:14:52.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559d806e7cf0 con 0x559d80455070 2024-08-31T07:14:52.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559d80809030 con 0x559d80434390 2024-08-31T07:14:52.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c10fd3740 10 client.0 _ll_register_callbacks cb 0x559d8070d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:52.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d808064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d808064a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43866/0 (socket says 172.21.15.32:43866) 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d80812320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03400640 1 -- 172.21.15.32:0/1857140124 learned_addr learned my addr 172.21.15.32:0/1857140124 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1bfaa00640 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80434390 0x559d80806a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03e00640 1 -- 172.21.15.32:0/1857140124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80434390 msgr2=0x559d80806a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03e00640 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80434390 0x559d80806a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03e00640 1 -- 172.21.15.32:0/1857140124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 msgr2=0x559d808064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03e00640 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d808064a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03e00640 1 -- 172.21.15.32:0/1857140124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d807288b0 con 0x559d80455070 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03400640 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d808064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1c03e00640 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d80812320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0e3340 tx=0x7f1bfc0acb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3a6760c9cbea18 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.585+0000 7f1bf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.587+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1857140124 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0b5600 con 0x559d80455070 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.587+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1857140124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bec0318a0 con 0x559d80455070 2024-08-31T07:14:52.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.587+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1857140124 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1bfc0a8570 con 0x559d80455070 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.587+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1857140124 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc0b5170 con 0x559d80455070 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.587+0000 7f1c10fd3740 10 client.26015 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.587+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1857140124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d80815140 con 0x559d80455070 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.587+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1857140124 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bec0318a0 con 0x559d80455070 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1857140124 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1bfc0aafb0 con 0x559d80455070 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1c10fd3740 10 client.26015 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1c10fd3740 10 client.26015 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1c10fd3740 20 client.26015 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x559d804139f0 0x559d80818580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1c10fd3740 4 client.26015 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1857140124 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d80773700 con 0x559d804139f0 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1c03400640 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x559d804139f0 0x559d80818580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1c03400640 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x559d804139f0 0x559d80818580 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65ca72a2052fb97c server_cookie=5adf1f20f16e4659 in_seq=0 out_seq=0 2024-08-31T07:14:52.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1bf0c00640 10 client.26015 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.588+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1857140124 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d80773700 con 0x559d804139f0 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1bf0c00640 10 client.26015 handle_command_reply: tid=0 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 client.26015 shutdown 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1857140124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 msgr2=0x559d80812320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d80812320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0e3340 tx=0x7f1bfc0acb20 comp rx=0 tx=0).stop 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1857140124 shutdown_connections 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x559d804139f0 0x559d80818580 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d80812320 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80434390 0x559d80806a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1857140124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d808064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1bf0c00640 0 client.26015 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1857140124 >> 172.21.15.32:0/1857140124 conn(0x559d80449460 msgr2=0x559d807fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:52.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1857140124 shutdown_connections 2024-08-31T07:14:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1857140124 wait complete. 2024-08-31T07:14:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.589+0000 7f1c10fd3740 20 client.26015 trim_cache size 0 max 16384 2024-08-31T07:14:52.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.604+0000 7f1c10fd3740 1 Processor -- start 2024-08-31T07:14:52.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.604+0000 7f1c10fd3740 1 -- start start 2024-08-31T07:14:52.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d80812220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80434390 0x559d807f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d807ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b14f0 con 0x559d8042c1b0 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b30f0 con 0x559d80455070 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b1bc0 con 0x559d80434390 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c03400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d80812220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d807ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c03400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d80812220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43870/0 (socket says 172.21.15.32:43870) 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c03400640 1 -- 172.21.15.32:0/2004621816 learned_addr learned my addr 172.21.15.32:0/2004621816 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d807ffb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59760/0 (socket says 172.21.15.32:59760) 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.605+0000 7f1bfaa00640 1 --2- 172.21.15.32:0/2004621816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80434390 0x559d807f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c03e00640 1 -- 172.21.15.32:0/2004621816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80434390 msgr2=0x559d807f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:52.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c03e00640 1 --2- 172.21.15.32:0/2004621816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80434390 0x559d807f74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c03e00640 1 -- 172.21.15.32:0/2004621816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 msgr2=0x559d80812220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:52.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c03e00640 1 --2- 172.21.15.32:0/2004621816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d80812220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c03e00640 1 -- 172.21.15.32:0/2004621816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d807fcba0 con 0x559d80455070 2024-08-31T07:14:52.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c03400640 1 --2- 172.21.15.32:0/2004621816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d80812220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:52.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c03e00640 1 --2- 172.21.15.32:0/2004621816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d807ffb30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0abfe0 tx=0x7f1bfc005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f881395075b293cf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:52.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1bf3400640 1 -- 172.21.15.32:0/2004621816 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0b5310 con 0x559d80455070 2024-08-31T07:14:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1bf3400640 1 -- 172.21.15.32:0/2004621816 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bfc0aafb0 con 0x559d80455070 2024-08-31T07:14:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1bf3400640 1 -- 172.21.15.32:0/2004621816 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0ab850 con 0x559d80455070 2024-08-31T07:14:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c10fd3740 1 -- 172.21.15.32:0/2004621816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 msgr2=0x559d807ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/2004621816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d807ffb30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0abfe0 tx=0x7f1bfc005870 comp rx=0 tx=0).stop 2024-08-31T07:14:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c10fd3740 1 -- 172.21.15.32:0/2004621816 shutdown_connections 2024-08-31T07:14:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/2004621816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80455070 0x559d807ffb30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/2004621816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d80434390 0x559d807f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/2004621816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d8042c1b0 0x559d80812220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c10fd3740 1 -- 172.21.15.32:0/2004621816 >> 172.21.15.32:0/2004621816 conn(0x559d80449460 msgr2=0x559d807f6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c10fd3740 1 -- 172.21.15.32:0/2004621816 shutdown_connections 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.606+0000 7f1c10fd3740 1 -- 172.21.15.32:0/2004621816 wait complete. 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.608+0000 7f1c10fd3740 1 -- start start 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.608+0000 7f1c10fd3740 1 Processor -- start 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.608+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d8042c1b0 0x559d8080e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.608+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80434390 0x559d8080e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1c10fd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80455070 0x559d80809240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b2960 con 0x559d80455070 2024-08-31T07:14:52.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b2290 con 0x559d80434390 2024-08-31T07:14:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1c10fd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041af4d0 con 0x559d8042c1b0 2024-08-31T07:14:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1c10fd3740 10 client.0 _ll_register_callbacks cb 0x7f1bfc0abdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1bfaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80434390 0x559d8080e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1c03400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d8042c1b0 0x559d8080e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80455070 0x559d80809240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1bfaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80434390 0x559d8080e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59770/0 (socket says 172.21.15.32:59770) 2024-08-31T07:14:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1c03400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d8042c1b0 0x559d8080e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45642/0 (socket says 172.21.15.32:45642) 2024-08-31T07:14:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1bfaa00640 1 -- 172.21.15.32:0/1292397308 learned_addr learned my addr 172.21.15.32:0/1292397308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1bfaa00640 1 -- 172.21.15.32:0/1292397308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d8042c1b0 msgr2=0x559d8080e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1bfaa00640 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d8042c1b0 0x559d8080e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1bfaa00640 1 -- 172.21.15.32:0/1292397308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80455070 msgr2=0x559d80809240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1bfaa00640 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80455070 0x559d80809240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1bfaa00640 1 -- 172.21.15.32:0/1292397308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d80809880 con 0x559d80434390 2024-08-31T07:14:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1c03400640 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d8042c1b0 0x559d8080e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.609+0000 7f1c03e00640 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80455070 0x559d80809240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1bfaa00640 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80434390 0x559d8080e9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1bec033000 tx=0x7f1bec005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8addaa5d12432fde server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1bf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1292397308 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec035030 con 0x559d80434390 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1292397308 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1bec005670 con 0x559d80434390 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1292397308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bfc0aafb0 con 0x559d80434390 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1292397308 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1bec004950 con 0x559d80434390 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1c10fd3740 10 client.26021 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1292397308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d808837c0 con 0x559d80434390 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1292397308 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bfc0aafb0 con 0x559d80434390 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1292397308 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1bec03b460 con 0x559d80434390 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1c10fd3740 10 client.26021 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1c10fd3740 10 client.26021 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:14:52.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1c10fd3740 20 client.26021 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x559d803e8da0 0x559d80885a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1c10fd3740 4 client.26021 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1292397308 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d8076f1a0 con 0x559d803e8da0 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.610+0000 7f1c03400640 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x559d803e8da0 0x559d80885a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.612+0000 7f1c03400640 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x559d803e8da0 0x559d80885a50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9531d787a1eefcd server_cookie=96bba5b5980c2d1f in_seq=0 out_seq=0 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.612+0000 7f1bf0c00640 10 client.26021 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.612+0000 7f1bf0c00640 1 -- 172.21.15.32:0/1292397308 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d8076f1a0 con 0x559d803e8da0 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.612+0000 7f1bf0c00640 10 client.26021 handle_command_reply: tid=0 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.612+0000 7f1c10fd3740 1 client.26021 shutdown 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.612+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1292397308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80434390 msgr2=0x559d8080e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.612+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80434390 0x559d8080e9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1bec033000 tx=0x7f1bec005270 comp rx=0 tx=0).stop 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1292397308 shutdown_connections 2024-08-31T07:14:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x559d803e8da0 0x559d80885a50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d80455070 0x559d80809240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d80434390 0x559d8080e9d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1bf0c00640 0 client.26021 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:14:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c10fd3740 1 --2- 172.21.15.32:0/1292397308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d8042c1b0 0x559d8080e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1292397308 >> 172.21.15.32:0/1292397308 conn(0x559d80449460 msgr2=0x559d807fb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1292397308 shutdown_connections 2024-08-31T07:14:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c10fd3740 1 -- 172.21.15.32:0/1292397308 wait complete. 2024-08-31T07:14:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c10fd3740 20 client.26021 trim_cache size 0 max 16384 2024-08-31T07:14:52.618 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:14:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c0a200640 1 -- 172.21.15.32:0/4098753414 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1be80b4ca0 msgr2=0x7f1be80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c0a200640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1be80b4ca0 0x7f1be80b71b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x559d807f9530 tx=0x7f1bf4005f90 comp rx=0 tx=0).stop 2024-08-31T07:14:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c0a200640 1 -- 172.21.15.32:0/4098753414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c0410bdb0 msgr2=0x7f1c041a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.613+0000 7f1c0a200640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c0410bdb0 0x7f1c041a1cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009fd0 tx=0x7f1bfc00ef10 comp rx=0 tx=0).stop 2024-08-31T07:14:52.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.614+0000 7f1c0a200640 1 -- 172.21.15.32:0/4098753414 shutdown_connections 2024-08-31T07:14:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.614+0000 7f1c0a200640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1be80b4ca0 0x7f1be80b71b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.614+0000 7f1c0a200640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c0410bdb0 0x7f1c041a1cc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.614+0000 7f1c0a200640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c0410b460 0x7f1c0419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.614+0000 7f1c0a200640 1 --2- 172.21.15.32:0/4098753414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c04105b20 0x7f1c0419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.614+0000 7f1c0a200640 1 -- 172.21.15.32:0/4098753414 >> 172.21.15.32:0/4098753414 conn(0x7f1c041015b0 msgr2=0x7f1c041039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.614+0000 7f1c0a200640 1 -- 172.21.15.32:0/4098753414 shutdown_connections 2024-08-31T07:14:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:52.614+0000 7f1c0a200640 1 -- 172.21.15.32:0/4098753414 wait complete. 2024-08-31T07:14:52.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68399 9.39 MB/sec warmup 41 sec latency 2513.287 ms 2024-08-31T07:14:53.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68399 9.17 MB/sec warmup 42 sec latency 3513.353 ms 2024-08-31T07:14:53.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:14:54.124 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:54 smithi032 ceph-mon[17252]: pgmap v432: 97 pgs: 97 active+clean; 54 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.9 MiB/s wr, 68 op/s 2024-08-31T07:14:54.128 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:54.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:54 smithi073 ceph-mon[21995]: pgmap v432: 97 pgs: 97 active+clean; 54 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.9 MiB/s wr, 68 op/s 2024-08-31T07:14:54.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:54 smithi084 ceph-mon[22387]: pgmap v432: 97 pgs: 97 active+clean; 54 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.9 MiB/s wr, 68 op/s 2024-08-31T07:14:54.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 68399 8.96 MB/sec warmup 43 sec latency 4513.411 ms 2024-08-31T07:14:55.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.157+0000 7f5143200640 1 -- 172.21.15.32:0/2723423894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144103750 msgr2=0x7f5144103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:55.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.157+0000 7f5143200640 1 --2- 172.21.15.32:0/2723423894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144103750 0x7f5144103b60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5134009900 tx=0x7f513402a410 comp rx=0 tx=0).stop 2024-08-31T07:14:55.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.158+0000 7f5143200640 1 -- 172.21.15.32:0/2723423894 shutdown_connections 2024-08-31T07:14:55.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.158+0000 7f5143200640 1 --2- 172.21.15.32:0/2723423894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51441099e0 0x7f5144110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.158+0000 7f5143200640 1 --2- 172.21.15.32:0/2723423894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5144109090 0x7f51441094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.158+0000 7f5143200640 1 --2- 172.21.15.32:0/2723423894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144103750 0x7f5144103b60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.158+0000 7f5143200640 1 -- 172.21.15.32:0/2723423894 >> 172.21.15.32:0/2723423894 conn(0x7f51440ff1e0 msgr2=0x7f5144101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:55.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.158+0000 7f5143200640 1 -- 172.21.15.32:0/2723423894 shutdown_connections 2024-08-31T07:14:55.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.158+0000 7f5143200640 1 -- 172.21.15.32:0/2723423894 wait complete. 2024-08-31T07:14:55.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.159+0000 7f5143200640 1 Processor -- start 2024-08-31T07:14:55.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.159+0000 7f5143200640 1 -- start start 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.159+0000 7f5143200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5144103750 0x7f51440754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.159+0000 7f5143200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5144109090 0x7f51440759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.159+0000 7f5143200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51441099e0 0x7f5144075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5143200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f51441136b0 con 0x7f5144109090 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5143200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5144113530 con 0x7f51441099e0 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5143200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5144113830 con 0x7f5144103750 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5141e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5144103750 0x7f51440754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5141e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5144103750 0x7f51440754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45660/0 (socket says 172.21.15.32:45660) 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5141e00640 1 -- 172.21.15.32:0/2836927937 learned_addr learned my addr 172.21.15.32:0/2836927937 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5141400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5144109090 0x7f51440759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5142800640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51441099e0 0x7f5144075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5142800640 1 -- 172.21.15.32:0/2836927937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5144103750 msgr2=0x7f51440754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5142800640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5144103750 0x7f51440754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5142800640 1 -- 172.21.15.32:0/2836927937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5144109090 msgr2=0x7f51440759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5142800640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5144109090 0x7f51440759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5142800640 1 -- 172.21.15.32:0/2836927937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51441a83a0 con 0x7f51441099e0 2024-08-31T07:14:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.160+0000 7f5141400640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5144109090 0x7f51440759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.161+0000 7f5142800640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51441099e0 0x7f5144075f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f513c009fd0 tx=0x7f513c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.161+0000 7f513aa00640 1 -- 172.21.15.32:0/2836927937 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c00ce20 con 0x7f51441099e0 2024-08-31T07:14:55.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.161+0000 7f513aa00640 1 -- 172.21.15.32:0/2836927937 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513c005020 con 0x7f51441099e0 2024-08-31T07:14:55.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.161+0000 7f5143200640 1 -- 172.21.15.32:0/2836927937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51441a8690 con 0x7f51441099e0 2024-08-31T07:14:55.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.161+0000 7f5143200640 1 -- 172.21.15.32:0/2836927937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51441a8b80 con 0x7f51441099e0 2024-08-31T07:14:55.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.161+0000 7f513aa00640 1 -- 172.21.15.32:0/2836927937 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c010690 con 0x7f51441099e0 2024-08-31T07:14:55.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.164+0000 7f514f8b4740 1 Processor -- start 2024-08-31T07:14:55.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.165+0000 7f513aa00640 1 -- 172.21.15.32:0/2836927937 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f513c0054d0 con 0x7f51441099e0 2024-08-31T07:14:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.165+0000 7f514f8b4740 1 -- start start 2024-08-31T07:14:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.165+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x55569501db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.165+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 0x55569501e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.165+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 0x55569501e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.165+0000 7f514f8b4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555694f947b0 con 0x555694c57390 2024-08-31T07:14:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.165+0000 7f514f8b4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555694f83d40 con 0x555694c4f1b0 2024-08-31T07:14:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.165+0000 7f514f8b4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555694f87cf0 con 0x555694c78070 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.166+0000 7f5141400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x55569501db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.166+0000 7f5141400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x55569501db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45674/0 (socket says 172.21.15.32:45674) 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.166+0000 7f5141400640 1 -- 172.21.15.32:0/688646631 learned_addr learned my addr 172.21.15.32:0/688646631 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.166+0000 7f5141e00640 1 --2- 172.21.15.32:0/688646631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 0x55569501e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.166+0000 7f5141400640 1 --2- 172.21.15.32:0/688646631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 0x55569501e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.166+0000 7f5141e00640 1 -- 172.21.15.32:0/688646631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 msgr2=0x55569501db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.166+0000 7f5141e00640 1 --2- 172.21.15.32:0/688646631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x55569501db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.166+0000 7f5141e00640 1 -- 172.21.15.32:0/688646631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 msgr2=0x55569501e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.166+0000 7f5141e00640 1 --2- 172.21.15.32:0/688646631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 0x55569501e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.166+0000 7f5141e00640 1 -- 172.21.15.32:0/688646631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555694ce3650 con 0x555694c4f1b0 2024-08-31T07:14:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.167+0000 7f513aa00640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51280b4ca0 0x7f51280b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.167+0000 7f5141e00640 1 --2- 172.21.15.32:0/688646631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 0x55569501e0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5134004770 tx=0x7f5134031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9e9936403f436eb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.167+0000 7f513aa00640 1 -- 172.21.15.32:0/2836927937 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f513c0e5c20 con 0x7f51441099e0 2024-08-31T07:14:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.167+0000 7f5141400640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51280b4ca0 0x7f51280b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.167+0000 7f512f000640 1 -- 172.21.15.32:0/688646631 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134040070 con 0x555694c4f1b0 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.167+0000 7f512f000640 1 -- 172.21.15.32:0/688646631 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5134004030 con 0x555694c4f1b0 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.167+0000 7f512f000640 1 -- 172.21.15.32:0/688646631 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134035040 con 0x555694c4f1b0 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.168+0000 7f514f8b4740 1 -- 172.21.15.32:0/688646631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 msgr2=0x55569501e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.168+0000 7f514f8b4740 1 --2- 172.21.15.32:0/688646631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 0x55569501e0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5134004770 tx=0x7f5134031b10 comp rx=0 tx=0).stop 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.168+0000 7f5141400640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51280b4ca0 0x7f51280b71b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51300058c0 tx=0x7f5130005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.168+0000 7f514f8b4740 1 -- 172.21.15.32:0/688646631 shutdown_connections 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.168+0000 7f514f8b4740 1 --2- 172.21.15.32:0/688646631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 0x55569501e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.168+0000 7f514f8b4740 1 --2- 172.21.15.32:0/688646631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 0x55569501e0a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.168+0000 7f514f8b4740 1 --2- 172.21.15.32:0/688646631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x55569501db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.168+0000 7f514f8b4740 1 -- 172.21.15.32:0/688646631 >> 172.21.15.32:0/688646631 conn(0x555694c6c460 msgr2=0x55569501ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.168+0000 7f514f8b4740 1 -- 172.21.15.32:0/688646631 shutdown_connections 2024-08-31T07:14:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.168+0000 7f514f8b4740 1 -- 172.21.15.32:0/688646631 wait complete. 2024-08-31T07:14:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.169+0000 7f514f8b4740 1 -- start start 2024-08-31T07:14:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.169+0000 7f514f8b4740 1 Processor -- start 2024-08-31T07:14:55.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 0x5556950325f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 0x555695032bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x555695037bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f514f8b4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555694de6700 con 0x555694c57390 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f514f8b4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555694f0acf0 con 0x555694c4f1b0 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f514f8b4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55569502c2e0 con 0x555694c78070 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f514f8b4740 10 client.0 _ll_register_callbacks cb 0x555694f30470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f5141400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 0x555695032bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f5142800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x555695037bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f5141e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 0x5556950325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f5142800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x555695037bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45686/0 (socket says 172.21.15.32:45686) 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f5142800640 1 -- 172.21.15.32:0/2113271044 learned_addr learned my addr 172.21.15.32:0/2113271044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.170+0000 7f5141400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 0x555695032bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43930/0 (socket says 172.21.15.32:43930) 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f5142800640 1 -- 172.21.15.32:0/2113271044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 msgr2=0x5556950325f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f5142800640 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 0x5556950325f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f5142800640 1 -- 172.21.15.32:0/2113271044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 msgr2=0x555695032bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f5142800640 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 0x555695032bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f5142800640 1 -- 172.21.15.32:0/2113271044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555694f4b8b0 con 0x555694c78070 2024-08-31T07:14:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f5142800640 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x555695037bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f513c005fd0 tx=0x7f513c0acb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6ae21c769a1a2be server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f5123400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:14:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f5123400640 1 -- 172.21.15.32:0/2113271044 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c0b5650 con 0x555694c78070 2024-08-31T07:14:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f514f8b4740 1 -- 172.21.15.32:0/2113271044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5134004030 con 0x555694c78070 2024-08-31T07:14:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f5123400640 1 -- 172.21.15.32:0/2113271044 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f513c0a85c0 con 0x555694c78070 2024-08-31T07:14:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f5123400640 1 -- 172.21.15.32:0/2113271044 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f513c0c6050 con 0x555694c78070 2024-08-31T07:14:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f514f8b4740 10 client.26044 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.171+0000 7f514f8b4740 1 -- 172.21.15.32:0/2113271044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555695040ba0 con 0x555694c78070 2024-08-31T07:14:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.172+0000 7f5123400640 1 -- 172.21.15.32:0/2113271044 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513c0c9020 con 0x555694c78070 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.172+0000 7f5123400640 1 -- 172.21.15.32:0/2113271044 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f513c0a7ca0 con 0x555694c78070 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.172+0000 7f514f8b4740 10 client.26044 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.172+0000 7f514f8b4740 10 client.26044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.172+0000 7f514f8b4740 20 client.26044 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.172+0000 7f514f8b4740 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x555694c369f0 0x5556950440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.172+0000 7f514f8b4740 4 client.26044 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.172+0000 7f514f8b4740 1 -- 172.21.15.32:0/2113271044 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555694f96700 con 0x555694c369f0 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.172+0000 7f5141e00640 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x555694c369f0 0x5556950440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.173+0000 7f5141e00640 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x555694c369f0 0x5556950440e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62cacf1b569c5c62 server_cookie=fcaffba718aad43 in_seq=0 out_seq=0 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.173+0000 7f5123400640 10 client.26044 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f5123400640 1 -- 172.21.15.32:0/2113271044 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555694f96700 con 0x555694c369f0 2024-08-31T07:14:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f5123400640 10 client.26044 handle_command_reply: tid=0 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f514f8b4740 1 client.26044 shutdown 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f514f8b4740 1 -- 172.21.15.32:0/2113271044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 msgr2=0x555695037bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f514f8b4740 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x555695037bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f513c005fd0 tx=0x7f513c0acb70 comp rx=0 tx=0).stop 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f514f8b4740 1 -- 172.21.15.32:0/2113271044 shutdown_connections 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f514f8b4740 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c78070 0x555695037bf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f514f8b4740 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c57390 0x555695032bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f514f8b4740 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x555694c369f0 0x5556950440e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f514f8b4740 1 --2- 172.21.15.32:0/2113271044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c4f1b0 0x5556950325f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f5123400640 0 client.26044 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.174+0000 7f514f8b4740 1 -- 172.21.15.32:0/2113271044 >> 172.21.15.32:0/2113271044 conn(0x555694c6c460 msgr2=0x555695020cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.175+0000 7f514f8b4740 1 -- 172.21.15.32:0/2113271044 shutdown_connections 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.175+0000 7f514f8b4740 1 -- 172.21.15.32:0/2113271044 wait complete. 2024-08-31T07:14:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.175+0000 7f514f8b4740 20 client.26044 trim_cache size 0 max 16384 2024-08-31T07:14:55.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.200+0000 7f514f8b4740 1 Processor -- start 2024-08-31T07:14:55.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.200+0000 7f514f8b4740 1 -- start start 2024-08-31T07:14:55.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c4f1b0 0x55569502cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c57390 0x55569503e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c78070 0x555695020c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f514f8b4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ace90 con 0x555694c78070 2024-08-31T07:14:55.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f514f8b4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51441aea90 con 0x555694c57390 2024-08-31T07:14:55.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f514f8b4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ad560 con 0x555694c4f1b0 2024-08-31T07:14:55.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5141e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c4f1b0 0x55569502cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5141400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c57390 0x55569503e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5141e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c4f1b0 0x55569502cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45700/0 (socket says 172.21.15.32:45700) 2024-08-31T07:14:55.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5141e00640 1 -- 172.21.15.32:0/3335949008 learned_addr learned my addr 172.21.15.32:0/3335949008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:55.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5142800640 1 --2- 172.21.15.32:0/3335949008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c78070 0x555695020c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5141e00640 1 -- 172.21.15.32:0/3335949008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c57390 msgr2=0x55569503e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:55.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5141e00640 1 --2- 172.21.15.32:0/3335949008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c57390 0x55569503e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5141e00640 1 -- 172.21.15.32:0/3335949008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c78070 msgr2=0x555695020c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:55.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5141e00640 1 --2- 172.21.15.32:0/3335949008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c78070 0x555695020c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5141e00640 1 -- 172.21.15.32:0/3335949008 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556950170a0 con 0x555694c4f1b0 2024-08-31T07:14:55.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.201+0000 7f5142800640 1 --2- 172.21.15.32:0/3335949008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c78070 0x555695020c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:14:55.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.202+0000 7f5141e00640 1 --2- 172.21.15.32:0/3335949008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c4f1b0 0x55569502cc00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5134006010 tx=0x7f513402ab90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97f612755895dde2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:55.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.202+0000 7f512e600640 1 -- 172.21.15.32:0/3335949008 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134040040 con 0x555694c4f1b0 2024-08-31T07:14:55.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.202+0000 7f512e600640 1 -- 172.21.15.32:0/3335949008 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51340044c0 con 0x555694c4f1b0 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.202+0000 7f512e600640 1 -- 172.21.15.32:0/3335949008 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134035040 con 0x555694c4f1b0 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.203+0000 7f514f8b4740 1 -- 172.21.15.32:0/3335949008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c4f1b0 msgr2=0x55569502cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.203+0000 7f514f8b4740 1 --2- 172.21.15.32:0/3335949008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c4f1b0 0x55569502cc00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5134006010 tx=0x7f513402ab90 comp rx=0 tx=0).stop 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.203+0000 7f514f8b4740 1 -- 172.21.15.32:0/3335949008 shutdown_connections 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.203+0000 7f514f8b4740 1 --2- 172.21.15.32:0/3335949008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c78070 0x555695020c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.203+0000 7f514f8b4740 1 --2- 172.21.15.32:0/3335949008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c57390 0x55569503e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.203+0000 7f514f8b4740 1 --2- 172.21.15.32:0/3335949008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c4f1b0 0x55569502cc00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.203+0000 7f514f8b4740 1 -- 172.21.15.32:0/3335949008 >> 172.21.15.32:0/3335949008 conn(0x555694c6c460 msgr2=0x55569502c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.203+0000 7f514f8b4740 1 -- 172.21.15.32:0/3335949008 shutdown_connections 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.203+0000 7f514f8b4740 1 -- 172.21.15.32:0/3335949008 wait complete. 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.204+0000 7f514f8b4740 1 -- start start 2024-08-31T07:14:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.204+0000 7f514f8b4740 1 Processor -- start 2024-08-31T07:14:55.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.204+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c4f1b0 0x55569503e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.204+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c57390 0x55569503e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.204+0000 7f514f8b4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c78070 0x555695038a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.204+0000 7f514f8b4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ae300 con 0x555694c4f1b0 2024-08-31T07:14:55.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.204+0000 7f514f8b4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51441adc30 con 0x555694c78070 2024-08-31T07:14:55.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.204+0000 7f514f8b4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f51441aae70 con 0x555694c57390 2024-08-31T07:14:55.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.204+0000 7f514f8b4740 10 client.0 _ll_register_callbacks cb 0x7f513c0abeb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:55.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5141e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c4f1b0 0x55569503e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5142800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c78070 0x555695038a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5142800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c78070 0x555695038a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59832/0 (socket says 172.21.15.32:59832) 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5142800640 1 -- 172.21.15.32:0/1885251203 learned_addr learned my addr 172.21.15.32:0/1885251203 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5141400640 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c57390 0x55569503e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5142800640 1 -- 172.21.15.32:0/1885251203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c57390 msgr2=0x55569503e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5142800640 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c57390 0x55569503e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5142800640 1 -- 172.21.15.32:0/1885251203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c4f1b0 msgr2=0x55569503e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5142800640 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c4f1b0 0x55569503e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5142800640 1 -- 172.21.15.32:0/1885251203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556950391c0 con 0x555694c78070 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5141400640 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c57390 0x55569503e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5142800640 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c78070 0x555695038a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f513c0ac2c0 tx=0x7f513c0a8850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32cdb9f0f59eca13 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.205+0000 7f5123400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.206+0000 7f5123400640 1 -- 172.21.15.32:0/1885251203 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c0abc80 con 0x555694c78070 2024-08-31T07:14:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.206+0000 7f5123400640 1 -- 172.21.15.32:0/1885251203 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f513c0a8920 con 0x555694c78070 2024-08-31T07:14:55.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.206+0000 7f514f8b4740 1 -- 172.21.15.32:0/1885251203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555694f921a0 con 0x555694c78070 2024-08-31T07:14:55.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.206+0000 7f5123400640 1 -- 172.21.15.32:0/1885251203 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f513c0e3cf0 con 0x555694c78070 2024-08-31T07:14:55.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.206+0000 7f514f8b4740 10 client.26039 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:55.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.206+0000 7f514f8b4740 1 -- 172.21.15.32:0/1885251203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5556950af600 con 0x555694c78070 2024-08-31T07:14:55.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.206+0000 7f5123400640 1 -- 172.21.15.32:0/1885251203 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555694f921a0 con 0x555694c78070 2024-08-31T07:14:55.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.207+0000 7f5123400640 1 -- 172.21.15.32:0/1885251203 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f513c0cdaa0 con 0x555694c78070 2024-08-31T07:14:55.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.207+0000 7f514f8b4740 10 client.26039 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.207+0000 7f514f8b4740 10 client.26039 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.207+0000 7f514f8b4740 20 client.26039 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.207+0000 7f514f8b4740 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x555694c0bda0 0x5556950b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.207+0000 7f514f8b4740 4 client.26039 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.207+0000 7f514f8b4740 1 -- 172.21.15.32:0/1885251203 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555694f8f8e0 con 0x555694c0bda0 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.207+0000 7f5141e00640 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x555694c0bda0 0x5556950b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f5141e00640 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x555694c0bda0 0x5556950b1ba0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50156fd59e6decf5 server_cookie=418052da197ecacd in_seq=0 out_seq=0 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f5123400640 10 client.26039 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f5123400640 1 -- 172.21.15.32:0/1885251203 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555694f8f8e0 con 0x555694c0bda0 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f5123400640 10 client.26039 handle_command_reply: tid=0 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f514f8b4740 1 client.26039 shutdown 2024-08-31T07:14:55.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f514f8b4740 1 -- 172.21.15.32:0/1885251203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c78070 msgr2=0x555695038a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f514f8b4740 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c78070 0x555695038a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f513c0ac2c0 tx=0x7f513c0a8850 comp rx=0 tx=0).stop 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f514f8b4740 1 -- 172.21.15.32:0/1885251203 shutdown_connections 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f514f8b4740 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x555694c0bda0 0x5556950b1ba0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f514f8b4740 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555694c78070 0x555695038a30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f514f8b4740 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555694c57390 0x55569503e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f514f8b4740 1 --2- 172.21.15.32:0/1885251203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555694c4f1b0 0x55569503e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f5123400640 0 client.26039 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.208+0000 7f514f8b4740 1 -- 172.21.15.32:0/1885251203 >> 172.21.15.32:0/1885251203 conn(0x555694c6c460 msgr2=0x555695020c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.209+0000 7f514f8b4740 1 -- 172.21.15.32:0/1885251203 shutdown_connections 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.209+0000 7f514f8b4740 1 -- 172.21.15.32:0/1885251203 wait complete. 2024-08-31T07:14:55.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.209+0000 7f514f8b4740 20 client.26039 trim_cache size 0 max 16384 2024-08-31T07:14:55.213 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:14:55.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.209+0000 7f5143200640 1 -- 172.21.15.32:0/2836927937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51280b4ca0 msgr2=0x7f51280b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:55.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.209+0000 7f5143200640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51280b4ca0 0x7f51280b71b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51300058c0 tx=0x7f5130005f90 comp rx=0 tx=0).stop 2024-08-31T07:14:55.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.209+0000 7f5143200640 1 -- 172.21.15.32:0/2836927937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51441099e0 msgr2=0x7f5144075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:55.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.209+0000 7f5143200640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51441099e0 0x7f5144075f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f513c009fd0 tx=0x7f513c00ef10 comp rx=0 tx=0).stop 2024-08-31T07:14:55.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.210+0000 7f5143200640 1 -- 172.21.15.32:0/2836927937 shutdown_connections 2024-08-31T07:14:55.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.210+0000 7f5143200640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51280b4ca0 0x7f51280b71b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.210+0000 7f5143200640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51441099e0 0x7f5144075f30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.210+0000 7f5143200640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5144109090 0x7f51440759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.210+0000 7f5143200640 1 --2- 172.21.15.32:0/2836927937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5144103750 0x7f51440754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:55.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.210+0000 7f5143200640 1 -- 172.21.15.32:0/2836927937 >> 172.21.15.32:0/2836927937 conn(0x7f51440ff1e0 msgr2=0x7f5144101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:55.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.210+0000 7f5143200640 1 -- 172.21.15.32:0/2836927937 shutdown_connections 2024-08-31T07:14:55.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:55.210+0000 7f5143200640 1 -- 172.21.15.32:0/2836927937 wait complete. 2024-08-31T07:14:55.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 69809 8.89 MB/sec warmup 44 sec latency 5055.570 ms 2024-08-31T07:14:56.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:14:56.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:56 smithi032 ceph-mon[17252]: pgmap v433: 97 pgs: 97 active+clean; 57 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 3.4 MiB/s wr, 81 op/s 2024-08-31T07:14:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:56 smithi073 ceph-mon[21995]: pgmap v433: 97 pgs: 97 active+clean; 57 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 3.4 MiB/s wr, 81 op/s 2024-08-31T07:14:56.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:56 smithi084 ceph-mon[22387]: pgmap v433: 97 pgs: 97 active+clean; 57 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 3.4 MiB/s wr, 81 op/s 2024-08-31T07:14:56.532 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:56.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 71360 8.96 MB/sec warmup 45 sec latency 30.514 ms 2024-08-31T07:14:57.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.198+0000 7f1dff400640 1 -- 172.21.15.32:0/1169642215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df810fe60 msgr2=0x7f1df81122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:57.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.198+0000 7f1dff400640 1 --2- 172.21.15.32:0/1169642215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df810fe60 0x7f1df81122a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009ca0 tx=0x7f1df002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.198+0000 7f1dff400640 1 -- 172.21.15.32:0/1169642215 shutdown_connections 2024-08-31T07:14:57.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.198+0000 7f1dff400640 1 --2- 172.21.15.32:0/1169642215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df810fe60 0x7f1df81122a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.198+0000 7f1dff400640 1 --2- 172.21.15.32:0/1169642215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df8075ea0 0x7f1df810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.198+0000 7f1dff400640 1 --2- 172.21.15.32:0/1169642215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df8075550 0x7f1df8075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.198+0000 7f1dff400640 1 -- 172.21.15.32:0/1169642215 >> 172.21.15.32:0/1169642215 conn(0x7f1df807b5f0 msgr2=0x7f1df807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.198+0000 7f1dff400640 1 -- 172.21.15.32:0/1169642215 shutdown_connections 2024-08-31T07:14:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.198+0000 7f1dff400640 1 -- 172.21.15.32:0/1169642215 wait complete. 2024-08-31T07:14:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.199+0000 7f1dff400640 1 Processor -- start 2024-08-31T07:14:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.199+0000 7f1dff400640 1 -- start start 2024-08-31T07:14:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dff400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df8075550 0x7f1df81072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df8075ea0 0x7f1df81035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dff400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df810fe60 0x7f1df8103cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dff400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8115a10 con 0x7f1df8075ea0 2024-08-31T07:14:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dff400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8115890 con 0x7f1df8075550 2024-08-31T07:14:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dff400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8115b90 con 0x7f1df810fe60 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dfca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df8075550 0x7f1df81072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1defe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df8075ea0 0x7f1df81035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dfca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df8075550 0x7f1df81072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54904/0 (socket says 172.21.15.32:54904) 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dfca00640 1 -- 172.21.15.32:0/3834892268 learned_addr learned my addr 172.21.15.32:0/3834892268 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dfd400640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df810fe60 0x7f1df8103cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dfca00640 1 -- 172.21.15.32:0/3834892268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df810fe60 msgr2=0x7f1df8103cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dfca00640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df810fe60 0x7f1df8103cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dfca00640 1 -- 172.21.15.32:0/3834892268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df8075ea0 msgr2=0x7f1df81035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dfca00640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df8075ea0 0x7f1df81035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.200+0000 7f1dfca00640 1 -- 172.21.15.32:0/3834892268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df81ac970 con 0x7f1df8075550 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.201+0000 7f1dfca00640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df8075550 0x7f1df81072f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1de800ea60 tx=0x7f1de800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.201+0000 7f1df6000640 1 -- 172.21.15.32:0/3834892268 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de800cde0 con 0x7f1df8075550 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.201+0000 7f1df6000640 1 -- 172.21.15.32:0/3834892268 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1de80035f0 con 0x7f1df8075550 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.201+0000 7f1dff400640 1 -- 172.21.15.32:0/3834892268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df81acba0 con 0x7f1df8075550 2024-08-31T07:14:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.201+0000 7f1df6000640 1 -- 172.21.15.32:0/3834892268 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8010690 con 0x7f1df8075550 2024-08-31T07:14:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.201+0000 7f1dff400640 1 -- 172.21.15.32:0/3834892268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df81ad130 con 0x7f1df8075550 2024-08-31T07:14:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.205+0000 7f1e06088740 1 Processor -- start 2024-08-31T07:14:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.206+0000 7f1e06088740 1 -- start start 2024-08-31T07:14:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.206+0000 7f1e06088740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 0x557341c0c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.206+0000 7f1e06088740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c0cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.206+0000 7f1e06088740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 0x557341c0d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.206+0000 7f1e06088740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557341b837b0 con 0x557341846390 2024-08-31T07:14:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.206+0000 7f1e06088740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557341b72d40 con 0x557341867070 2024-08-31T07:14:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.206+0000 7f1e06088740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557341b76cf0 con 0x55734183e1b0 2024-08-31T07:14:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1df6000640 1 -- 172.21.15.32:0/3834892268 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1de80037a0 con 0x7f1df8075550 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1dfd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 0x557341c0c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1defe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 0x557341c0d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1dfd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 0x557341c0c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54920/0 (socket says 172.21.15.32:54920) 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1dfd400640 1 -- 172.21.15.32:0/2374382739 learned_addr learned my addr 172.21.15.32:0/2374382739 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1dfca00640 1 --2- 172.21.15.32:0/2374382739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c0cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1defe00640 1 -- 172.21.15.32:0/2374382739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 msgr2=0x557341c0cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1defe00640 1 --2- 172.21.15.32:0/2374382739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c0cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1defe00640 1 -- 172.21.15.32:0/2374382739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 msgr2=0x557341c0c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1defe00640 1 --2- 172.21.15.32:0/2374382739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 0x557341c0c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1defe00640 1 -- 172.21.15.32:0/2374382739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573418d2650 con 0x557341846390 2024-08-31T07:14:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1dfca00640 1 --2- 172.21.15.32:0/2374382739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c0cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:14:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.207+0000 7f1dfd400640 1 --2- 172.21.15.32:0/2374382739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 0x557341c0c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.208+0000 7f1defe00640 1 --2- 172.21.15.32:0/2374382739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 0x557341c0d330 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1df806c500 tx=0x7f1de0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba055356ff326826 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.208+0000 7f1dedc00640 1 -- 172.21.15.32:0/2374382739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de001e070 con 0x557341846390 2024-08-31T07:14:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.208+0000 7f1dedc00640 1 -- 172.21.15.32:0/2374382739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1de0009d50 con 0x557341846390 2024-08-31T07:14:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.208+0000 7f1dedc00640 1 -- 172.21.15.32:0/2374382739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0019450 con 0x557341846390 2024-08-31T07:14:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1df6000640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ddc0b4d20 0x7f1ddc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1e06088740 1 -- 172.21.15.32:0/2374382739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 msgr2=0x557341c0d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1e06088740 1 --2- 172.21.15.32:0/2374382739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 0x557341c0d330 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1df806c500 tx=0x7f1de0011810 comp rx=0 tx=0).stop 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1df6000640 1 -- 172.21.15.32:0/3834892268 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1de80e57b0 con 0x7f1df8075550 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1dfd400640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ddc0b4d20 0x7f1ddc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1e06088740 1 -- 172.21.15.32:0/2374382739 shutdown_connections 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1e06088740 1 --2- 172.21.15.32:0/2374382739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 0x557341c0d330 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1e06088740 1 --2- 172.21.15.32:0/2374382739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c0cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1e06088740 1 --2- 172.21.15.32:0/2374382739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 0x557341c0c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1e06088740 1 -- 172.21.15.32:0/2374382739 >> 172.21.15.32:0/2374382739 conn(0x55734185b460 msgr2=0x557341c0a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1e06088740 1 -- 172.21.15.32:0/2374382739 shutdown_connections 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.209+0000 7f1e06088740 1 -- 172.21.15.32:0/2374382739 wait complete. 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.210+0000 7f1dfd400640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ddc0b4d20 0x7f1ddc0b7230 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1df002f500 tx=0x7f1df002f490 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.210+0000 7f1e06088740 1 -- start start 2024-08-31T07:14:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.210+0000 7f1e06088740 1 Processor -- start 2024-08-31T07:14:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1e06088740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1e06088740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341846390 0x557341c18ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1e06088740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 0x557341c243b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1e06088740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5573419d5700 con 0x557341867070 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1e06088740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557341af9cf0 con 0x557341846390 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1e06088740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557341c1b090 con 0x55734183e1b0 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1e06088740 10 client.0 _ll_register_callbacks cb 0x557341b1f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1dfca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1dfca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c184f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59304/0 (socket says 172.21.15.32:59304) 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1dfd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 0x557341c243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1dfca00640 1 -- 172.21.15.32:0/439722928 learned_addr learned my addr 172.21.15.32:0/439722928 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.211+0000 7f1defe00640 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341846390 0x557341c18ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1defe00640 1 -- 172.21.15.32:0/439722928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 msgr2=0x557341c184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1defe00640 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1defe00640 1 -- 172.21.15.32:0/439722928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 msgr2=0x557341c243b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1defe00640 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 0x557341c243b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1defe00640 1 -- 172.21.15.32:0/439722928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557341b3a8b0 con 0x557341846390 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1dfca00640 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1dfd400640 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 0x557341c243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1defe00640 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341846390 0x557341c18ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1de0003ad0 tx=0x7f1de0005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e01bd86b07d4ede server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1de6000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1de6000640 1 -- 172.21.15.32:0/439722928 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de001e040 con 0x557341846390 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1de6000640 1 -- 172.21.15.32:0/439722928 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1de00035f0 con 0x557341846390 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.212+0000 7f1e06088740 1 -- 172.21.15.32:0/439722928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1de0009d50 con 0x557341846390 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1de6000640 1 -- 172.21.15.32:0/439722928 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1de00046c0 con 0x557341846390 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1e06088740 10 client.26051 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1e06088740 1 -- 172.21.15.32:0/439722928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557341c27160 con 0x557341846390 2024-08-31T07:14:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1de6000640 1 -- 172.21.15.32:0/439722928 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1de0009d50 con 0x557341846390 2024-08-31T07:14:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1de6000640 1 -- 172.21.15.32:0/439722928 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1de0021030 con 0x557341846390 2024-08-31T07:14:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1e06088740 10 client.26051 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1e06088740 10 client.26051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:14:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1e06088740 20 client.26051 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1e06088740 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5573418259f0 0x557341c2a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1e06088740 4 client.26051 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:14:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.213+0000 7f1e06088740 1 -- 172.21.15.32:0/439722928 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557341b85700 con 0x5573418259f0 2024-08-31T07:14:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.214+0000 7f1dfca00640 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5573418259f0 0x557341c2a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.214+0000 7f1dfca00640 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5573418259f0 0x557341c2a4c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4281b7aa9a7ca19a server_cookie=faf10babe2aa3742 in_seq=0 out_seq=0 2024-08-31T07:14:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.214+0000 7f1de6000640 10 client.26051 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.215+0000 7f1de6000640 1 -- 172.21.15.32:0/439722928 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1de8085e50 con 0x5573418259f0 2024-08-31T07:14:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.215+0000 7f1de6000640 10 client.26051 handle_command_reply: tid=0 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.215+0000 7f1e06088740 1 client.26051 shutdown 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 1 -- 172.21.15.32:0/439722928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341846390 msgr2=0x557341c18ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341846390 0x557341c18ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1de0003ad0 tx=0x7f1de0005730 comp rx=0 tx=0).stop 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 1 -- 172.21.15.32:0/439722928 shutdown_connections 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5573418259f0 0x557341c2a4c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 0x557341c243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341846390 0x557341c18ab0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 1 --2- 172.21.15.32:0/439722928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1de6000640 0 client.26051 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 1 -- 172.21.15.32:0/439722928 >> 172.21.15.32:0/439722928 conn(0x55734185b460 msgr2=0x557341c0fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 1 -- 172.21.15.32:0/439722928 shutdown_connections 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 1 -- 172.21.15.32:0/439722928 wait complete. 2024-08-31T07:14:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.216+0000 7f1e06088740 20 client.26051 trim_cache size 0 max 16384 2024-08-31T07:14:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.230+0000 7f1e06088740 1 Processor -- start 2024-08-31T07:14:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.230+0000 7f1e06088740 1 -- start start 2024-08-31T07:14:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.230+0000 7f1e06088740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c11b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.230+0000 7f1e06088740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 0x557341c12070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.230+0000 7f1e06088740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 0x557341c16b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.230+0000 7f1e06088740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81b5910 con 0x557341846390 2024-08-31T07:14:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.230+0000 7f1e06088740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81b7510 con 0x557341867070 2024-08-31T07:14:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.230+0000 7f1e06088740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81b5fe0 con 0x55734183e1b0 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1dfca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c11b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1defe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 0x557341c12070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1dfca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c11b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59318/0 (socket says 172.21.15.32:59318) 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1dfca00640 1 -- 172.21.15.32:0/2478950908 learned_addr learned my addr 172.21.15.32:0/2478950908 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1dfd400640 1 --2- 172.21.15.32:0/2478950908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 0x557341c16b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1dfca00640 1 -- 172.21.15.32:0/2478950908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 msgr2=0x557341c16b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1dfca00640 1 --2- 172.21.15.32:0/2478950908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 0x557341c16b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1dfca00640 1 -- 172.21.15.32:0/2478950908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 msgr2=0x557341c12070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1dfca00640 1 --2- 172.21.15.32:0/2478950908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 0x557341c12070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1dfca00640 1 -- 172.21.15.32:0/2478950908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557341c2a830 con 0x55734183e1b0 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.231+0000 7f1dfca00640 1 --2- 172.21.15.32:0/2478950908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c11b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009920 tx=0x7f1de8003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65b53ae523af220 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.232+0000 7f1ded200640 1 -- 172.21.15.32:0/2478950908 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de80ab9d0 con 0x55734183e1b0 2024-08-31T07:14:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.232+0000 7f1ded200640 1 -- 172.21.15.32:0/2478950908 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557341b85700 con 0x55734183e1b0 2024-08-31T07:14:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.232+0000 7f1ded200640 1 -- 172.21.15.32:0/2478950908 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de80e31a0 con 0x55734183e1b0 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.232+0000 7f1e06088740 1 -- 172.21.15.32:0/2478950908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 msgr2=0x557341c11b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.232+0000 7f1e06088740 1 --2- 172.21.15.32:0/2478950908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c11b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009920 tx=0x7f1de8003e90 comp rx=0 tx=0).stop 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.233+0000 7f1e06088740 1 -- 172.21.15.32:0/2478950908 shutdown_connections 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.233+0000 7f1e06088740 1 --2- 172.21.15.32:0/2478950908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557341867070 0x557341c16b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.233+0000 7f1e06088740 1 --2- 172.21.15.32:0/2478950908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341846390 0x557341c12070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.233+0000 7f1e06088740 1 --2- 172.21.15.32:0/2478950908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55734183e1b0 0x557341c11b30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.233+0000 7f1e06088740 1 -- 172.21.15.32:0/2478950908 >> 172.21.15.32:0/2478950908 conn(0x55734185b460 msgr2=0x557341c13430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.233+0000 7f1e06088740 1 -- 172.21.15.32:0/2478950908 shutdown_connections 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.233+0000 7f1e06088740 1 -- 172.21.15.32:0/2478950908 wait complete. 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.233+0000 7f1e06088740 1 -- start start 2024-08-31T07:14:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1e06088740 1 Processor -- start 2024-08-31T07:14:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1e06088740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55734183e1b0 0x557341c20c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1e06088740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557341846390 0x557341c2f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1e06088740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 0x557341c30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1e06088740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81b6d80 con 0x557341867070 2024-08-31T07:14:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1e06088740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81b66b0 con 0x55734183e1b0 2024-08-31T07:14:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1e06088740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81b38f0 con 0x557341846390 2024-08-31T07:14:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1e06088740 10 client.0 _ll_register_callbacks cb 0x7f1de0009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1defe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557341846390 0x557341c2f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1defe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557341846390 0x557341c2f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59326/0 (socket says 172.21.15.32:59326) 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.234+0000 7f1defe00640 1 -- 172.21.15.32:0/2600453115 learned_addr learned my addr 172.21.15.32:0/2600453115 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1dfca00640 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55734183e1b0 0x557341c20c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1dfd400640 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 0x557341c30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1defe00640 1 -- 172.21.15.32:0/2600453115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55734183e1b0 msgr2=0x557341c20c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1defe00640 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55734183e1b0 0x557341c20c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1defe00640 1 -- 172.21.15.32:0/2600453115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 msgr2=0x557341c30090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1defe00640 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 0x557341c30090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1defe00640 1 -- 172.21.15.32:0/2600453115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557341c30820 con 0x557341846390 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1defe00640 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557341846390 0x557341c2f9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009250 tx=0x7f1de0011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e45ac61ff5a3619f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:14:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1de6000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1de6000640 1 -- 172.21.15.32:0/2600453115 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de003b060 con 0x557341846390 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1e06088740 1 -- 172.21.15.32:0/2600453115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557341b85700 con 0x557341846390 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.235+0000 7f1de6000640 1 -- 172.21.15.32:0/2600453115 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1de0004310 con 0x557341846390 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1de6000640 1 -- 172.21.15.32:0/2600453115 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1de001e070 con 0x557341846390 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1e06088740 10 client.26068 fetch_fsmap learned FSMap version 31 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1e06088740 1 -- 172.21.15.32:0/2600453115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557341c94ef0 con 0x557341846390 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1de6000640 1 -- 172.21.15.32:0/2600453115 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557341b85700 con 0x557341846390 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1de6000640 1 -- 172.21.15.32:0/2600453115 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1de0035030 con 0x557341846390 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1e06088740 10 client.26068 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1e06088740 10 client.26068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1e06088740 20 client.26068 populate_metadata read hostname 'smithi032' 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1e06088740 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5573417fada0 0x557341c97680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1e06088740 4 client.26068 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:14:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.236+0000 7f1e06088740 1 -- 172.21.15.32:0/2600453115 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557341b811a0 con 0x5573417fada0 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.237+0000 7f1dfca00640 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5573417fada0 0x557341c97680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.237+0000 7f1dfca00640 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5573417fada0 0x557341c97680 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb75533fb9aac67 server_cookie=53ad8bb57ea99f65 in_seq=0 out_seq=0 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.237+0000 7f1de6000640 10 client.26068 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.237+0000 7f1de6000640 1 -- 172.21.15.32:0/2600453115 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557341b811a0 con 0x5573417fada0 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.237+0000 7f1de6000640 10 client.26068 handle_command_reply: tid=0 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.237+0000 7f1e06088740 1 client.26068 shutdown 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 1 -- 172.21.15.32:0/2600453115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557341846390 msgr2=0x557341c2f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557341846390 0x557341c2f9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009250 tx=0x7f1de0011d10 comp rx=0 tx=0).stop 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 1 -- 172.21.15.32:0/2600453115 shutdown_connections 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5573417fada0 0x557341c97680 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557341867070 0x557341c30090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557341846390 0x557341c2f9f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1de6000640 0 client.26068 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:14:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 1 --2- 172.21.15.32:0/2600453115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55734183e1b0 0x557341c20c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 1 -- 172.21.15.32:0/2600453115 >> 172.21.15.32:0/2600453115 conn(0x55734185b460 msgr2=0x557341c16b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 1 -- 172.21.15.32:0/2600453115 shutdown_connections 2024-08-31T07:14:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 1 -- 172.21.15.32:0/2600453115 wait complete. 2024-08-31T07:14:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.238+0000 7f1e06088740 20 client.26068 trim_cache size 0 max 16384 2024-08-31T07:14:57.242 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:14:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 -- 172.21.15.32:0/3834892268 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ddc0b4d20 msgr2=0x7f1ddc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ddc0b4d20 0x7f1ddc0b7230 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1df002f500 tx=0x7f1df002f490 comp rx=0 tx=0).stop 2024-08-31T07:14:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 -- 172.21.15.32:0/3834892268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df8075550 msgr2=0x7f1df81072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:14:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df8075550 0x7f1df81072f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1de800ea60 tx=0x7f1de800ef20 comp rx=0 tx=0).stop 2024-08-31T07:14:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 -- 172.21.15.32:0/3834892268 shutdown_connections 2024-08-31T07:14:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ddc0b4d20 0x7f1ddc0b7230 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1df810fe60 0x7f1df8103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df8075ea0 0x7f1df81035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 --2- 172.21.15.32:0/3834892268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df8075550 0x7f1df81072f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:14:57.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 -- 172.21.15.32:0/3834892268 >> 172.21.15.32:0/3834892268 conn(0x7f1df807b5f0 msgr2=0x7f1df81014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:14:57.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 -- 172.21.15.32:0/3834892268 shutdown_connections 2024-08-31T07:14:57.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:14:57.239+0000 7f1dff400640 1 -- 172.21.15.32:0/3834892268 wait complete. 2024-08-31T07:14:57.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 71629 8.77 MB/sec warmup 46 sec latency 758.133 ms 2024-08-31T07:14:58.282 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:14:58 smithi032 ceph-mon[17252]: pgmap v434: 97 pgs: 97 active+clean; 59 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.9 MiB/s wr, 87 op/s 2024-08-31T07:14:58.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:14:58 smithi073 ceph-mon[21995]: pgmap v434: 97 pgs: 97 active+clean; 59 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.9 MiB/s wr, 87 op/s 2024-08-31T07:14:58.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:14:58 smithi084 ceph-mon[22387]: pgmap v434: 97 pgs: 97 active+clean; 59 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.9 MiB/s wr, 87 op/s 2024-08-31T07:14:58.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 71629 8.59 MB/sec warmup 47 sec latency 1758.215 ms 2024-08-31T07:14:59.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:14:59.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:14:59.672 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:14:59.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 71629 8.41 MB/sec warmup 48 sec latency 2758.262 ms 2024-08-31T07:15:00.270 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:00 smithi032 ceph-mon[17252]: pgmap v435: 97 pgs: 97 active+clean; 61 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.6 MiB/s wr, 95 op/s 2024-08-31T07:15:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:00 smithi073 ceph-mon[21995]: pgmap v435: 97 pgs: 97 active+clean; 61 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.6 MiB/s wr, 95 op/s 2024-08-31T07:15:00.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:00 smithi084 ceph-mon[22387]: pgmap v435: 97 pgs: 97 active+clean; 61 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.6 MiB/s wr, 95 op/s 2024-08-31T07:15:00.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.615+0000 7f6f58600640 1 -- 172.21.15.32:0/251445880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f501144f0 msgr2=0x7f6f50116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:00.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.615+0000 7f6f58600640 1 --2- 172.21.15.32:0/251445880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f501144f0 0x7f6f50116910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009ca0 tx=0x7f6f4802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.615+0000 7f6f58600640 1 -- 172.21.15.32:0/251445880 shutdown_connections 2024-08-31T07:15:00.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.615+0000 7f6f58600640 1 --2- 172.21.15.32:0/251445880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f501144f0 0x7f6f50116910 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.615+0000 7f6f58600640 1 --2- 172.21.15.32:0/251445880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f50078030 0x7f6f50113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.615+0000 7f6f58600640 1 --2- 172.21.15.32:0/251445880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f500776e0 0x7f6f50077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.615+0000 7f6f58600640 1 -- 172.21.15.32:0/251445880 >> 172.21.15.32:0/251445880 conn(0x7f6f50101600 msgr2=0x7f6f50103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:00.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.615+0000 7f6f58600640 1 -- 172.21.15.32:0/251445880 shutdown_connections 2024-08-31T07:15:00.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.615+0000 7f6f58600640 1 -- 172.21.15.32:0/251445880 wait complete. 2024-08-31T07:15:00.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.616+0000 7f6f58600640 1 Processor -- start 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.616+0000 7f6f58600640 1 -- start start 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f58600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f500776e0 0x7f6f501a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f58600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f50078030 0x7f6f501a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f58600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f501144f0 0x7f6f501a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f58600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f50119ec0 con 0x7f6f50078030 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f58600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6f50119d40 con 0x7f6f501144f0 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f58600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5011a040 con 0x7f6f500776e0 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f55200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f50078030 0x7f6f501a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f55c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f500776e0 0x7f6f501a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f55200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f50078030 0x7f6f501a0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36964/0 (socket says 172.21.15.32:36964) 2024-08-31T07:15:00.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f55200640 1 -- 172.21.15.32:0/3470248621 learned_addr learned my addr 172.21.15.32:0/3470248621 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.617+0000 7f6f56600640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f501144f0 0x7f6f501a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.618+0000 7f6f55c00640 1 -- 172.21.15.32:0/3470248621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f501144f0 msgr2=0x7f6f501a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.618+0000 7f6f55c00640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f501144f0 0x7f6f501a6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.618+0000 7f6f55c00640 1 -- 172.21.15.32:0/3470248621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f50078030 msgr2=0x7f6f501a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.618+0000 7f6f55c00640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f50078030 0x7f6f501a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.618+0000 7f6f55c00640 1 -- 172.21.15.32:0/3470248621 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f501a6780 con 0x7f6f500776e0 2024-08-31T07:15:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.618+0000 7f6f56600640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f501144f0 0x7f6f501a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:00.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.618+0000 7f6f55c00640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f500776e0 0x7f6f501a0810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400e9f0 tx=0x7f6f4400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:00.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.618+0000 7f6f4e000640 1 -- 172.21.15.32:0/3470248621 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4400cd80 con 0x7f6f500776e0 2024-08-31T07:15:00.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.618+0000 7f6f58600640 1 -- 172.21.15.32:0/3470248621 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f501a6a70 con 0x7f6f500776e0 2024-08-31T07:15:00.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.618+0000 7f6f58600640 1 -- 172.21.15.32:0/3470248621 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f501ad1d0 con 0x7f6f500776e0 2024-08-31T07:15:00.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.619+0000 7f6f4e000640 1 -- 172.21.15.32:0/3470248621 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f440035f0 con 0x7f6f500776e0 2024-08-31T07:15:00.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.619+0000 7f6f4e000640 1 -- 172.21.15.32:0/3470248621 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44010690 con 0x7f6f500776e0 2024-08-31T07:15:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.622+0000 7f6f5f418740 1 Processor -- start 2024-08-31T07:15:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.622+0000 7f6f4e000640 1 -- 172.21.15.32:0/3470248621 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6f440108f0 con 0x7f6f500776e0 2024-08-31T07:15:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.622+0000 7f6f5f418740 1 -- start start 2024-08-31T07:15:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9aa0350 0x5643d9e46600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e46b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 0x5643d9e47080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f5f418740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5643d9dbc750 con 0x5643d9aa0350 2024-08-31T07:15:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f5f418740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5643d9dabce0 con 0x5643d9a77490 2024-08-31T07:15:00.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f5f418740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5643d9dafc90 con 0x5643d9a7f670 2024-08-31T07:15:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f55c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e46b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f55c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e46b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54980/0 (socket says 172.21.15.32:54980) 2024-08-31T07:15:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f55c00640 1 -- 172.21.15.32:0/1024538265 learned_addr learned my addr 172.21.15.32:0/1024538265 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f55200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 0x5643d9e47080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.623+0000 7f6f56600640 1 --2- 172.21.15.32:0/1024538265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9aa0350 0x5643d9e46600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f55c00640 1 -- 172.21.15.32:0/1024538265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 msgr2=0x5643d9e47080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f55c00640 1 --2- 172.21.15.32:0/1024538265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 0x5643d9e47080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f55c00640 1 -- 172.21.15.32:0/1024538265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9aa0350 msgr2=0x5643d9e46600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f55c00640 1 --2- 172.21.15.32:0/1024538265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9aa0350 0x5643d9e46600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f55c00640 1 -- 172.21.15.32:0/1024538265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d9b0b930 con 0x5643d9a77490 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f55c00640 1 --2- 172.21.15.32:0/1024538265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e46b40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f440e2d50 tx=0x7f6f440ebef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d85677e2c86ba89 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f4e000640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6f380b4d20 0x7f6f380b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f42a00640 1 -- 172.21.15.32:0/1024538265 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f440f4070 con 0x5643d9a77490 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f42a00640 1 -- 172.21.15.32:0/1024538265 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f440b6b60 con 0x5643d9a77490 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f42a00640 1 -- 172.21.15.32:0/1024538265 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f440c05e0 con 0x5643d9a77490 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f4e000640 1 -- 172.21.15.32:0/3470248621 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6f440e56a0 con 0x7f6f500776e0 2024-08-31T07:15:00.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.624+0000 7f6f56600640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6f380b4d20 0x7f6f380b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.625+0000 7f6f5f418740 1 -- 172.21.15.32:0/1024538265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 msgr2=0x5643d9e46b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.625+0000 7f6f5f418740 1 --2- 172.21.15.32:0/1024538265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e46b40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f440e2d50 tx=0x7f6f440ebef0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.625+0000 7f6f56600640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6f380b4d20 0x7f6f380b7230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6f48006010 tx=0x7f6f4802f3f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.625+0000 7f6f5f418740 1 -- 172.21.15.32:0/1024538265 shutdown_connections 2024-08-31T07:15:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.625+0000 7f6f5f418740 1 --2- 172.21.15.32:0/1024538265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 0x5643d9e47080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.625+0000 7f6f5f418740 1 --2- 172.21.15.32:0/1024538265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e46b40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.625+0000 7f6f5f418740 1 --2- 172.21.15.32:0/1024538265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9aa0350 0x5643d9e46600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.625+0000 7f6f5f418740 1 -- 172.21.15.32:0/1024538265 >> 172.21.15.32:0/1024538265 conn(0x5643d9a94740 msgr2=0x5643d9e444f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.626+0000 7f6f5f418740 1 -- 172.21.15.32:0/1024538265 shutdown_connections 2024-08-31T07:15:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.626+0000 7f6f5f418740 1 -- 172.21.15.32:0/1024538265 wait complete. 2024-08-31T07:15:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.626+0000 7f6f5f418740 1 -- start start 2024-08-31T07:15:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f5f418740 1 Processor -- start 2024-08-31T07:15:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 0x5643d9e52110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 0x5643d9e52650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9aa0350 0x5643d9e5e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f5f418740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5643d9c0e890 con 0x5643d9a77490 2024-08-31T07:15:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f5f418740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5643d9d32cd0 con 0x5643d9aa0350 2024-08-31T07:15:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f5f418740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5643d9e54de0 con 0x5643d9a7f670 2024-08-31T07:15:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f5f418740 10 client.0 _ll_register_callbacks cb 0x5643d9da6ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f55200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 0x5643d9e52650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f55200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 0x5643d9e52650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59366/0 (socket says 172.21.15.32:59366) 2024-08-31T07:15:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.627+0000 7f6f55200640 1 -- 172.21.15.32:0/55872025 learned_addr learned my addr 172.21.15.32:0/55872025 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.628+0000 7f6f55c00640 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 0x5643d9e52110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.628+0000 7f6f56600640 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9aa0350 0x5643d9e5e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.628+0000 7f6f55200640 1 -- 172.21.15.32:0/55872025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9aa0350 msgr2=0x5643d9e5e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.628+0000 7f6f55200640 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9aa0350 0x5643d9e5e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.628+0000 7f6f55200640 1 -- 172.21.15.32:0/55872025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 msgr2=0x5643d9e52110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.628+0000 7f6f55200640 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 0x5643d9e52110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.628+0000 7f6f55200640 1 -- 172.21.15.32:0/55872025 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d9d7c1b0 con 0x5643d9a7f670 2024-08-31T07:15:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.629+0000 7f6f55200640 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 0x5643d9e52650 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c012840 tx=0x7f6f3c012d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96cf6ccebca5c713 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.629+0000 7f6f37400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:15:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.629+0000 7f6f37400640 1 -- 172.21.15.32:0/55872025 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00f920 con 0x5643d9a7f670 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.629+0000 7f6f5f418740 1 -- 172.21.15.32:0/55872025 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f440b6b60 con 0x5643d9a7f670 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.629+0000 7f6f37400640 1 -- 172.21.15.32:0/55872025 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6f3c00fac0 con 0x5643d9a7f670 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f37400640 1 -- 172.21.15.32:0/55872025 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6f3c00fd00 con 0x5643d9a7f670 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f5f418740 10 client.26080 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f5f418740 1 -- 172.21.15.32:0/55872025 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5643d9e60e20 con 0x5643d9a7f670 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f37400640 1 -- 172.21.15.32:0/55872025 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f440b6b60 con 0x5643d9a7f670 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f37400640 1 -- 172.21.15.32:0/55872025 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f6f3c005300 con 0x5643d9a7f670 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f5f418740 10 client.26080 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f5f418740 10 client.26080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f5f418740 20 client.26080 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f5f418740 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5643d9a5ecd0 0x5643d9e643e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f5f418740 4 client.26080 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.630+0000 7f6f5f418740 1 -- 172.21.15.32:0/55872025 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643d9dbe6a0 con 0x5643d9a5ecd0 2024-08-31T07:15:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.631+0000 7f6f55c00640 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5643d9a5ecd0 0x5643d9e643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.631+0000 7f6f55c00640 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5643d9a5ecd0 0x5643d9e643e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e520893be1a4dadc server_cookie=f761746ae0c237d1 in_seq=0 out_seq=0 2024-08-31T07:15:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.631+0000 7f6f37400640 10 client.26080 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f37400640 1 -- 172.21.15.32:0/55872025 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643d9dbe6a0 con 0x5643d9a5ecd0 2024-08-31T07:15:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f37400640 10 client.26080 handle_command_reply: tid=0 2024-08-31T07:15:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 client.26080 shutdown 2024-08-31T07:15:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 -- 172.21.15.32:0/55872025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 msgr2=0x5643d9e52650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 0x5643d9e52650 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c012840 tx=0x7f6f3c012d00 comp rx=0 tx=0).stop 2024-08-31T07:15:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 -- 172.21.15.32:0/55872025 shutdown_connections 2024-08-31T07:15:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5643d9a5ecd0 0x5643d9e643e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9aa0350 0x5643d9e5e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9a7f670 0x5643d9e52650 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f37400640 0 client.26080 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 --2- 172.21.15.32:0/55872025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 0x5643d9e52110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 -- 172.21.15.32:0/55872025 >> 172.21.15.32:0/55872025 conn(0x5643d9a94740 msgr2=0x5643d9e49790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 -- 172.21.15.32:0/55872025 shutdown_connections 2024-08-31T07:15:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.632+0000 7f6f5f418740 1 -- 172.21.15.32:0/55872025 wait complete. 2024-08-31T07:15:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.633+0000 7f6f5f418740 20 client.26080 trim_cache size 0 max 16384 2024-08-31T07:15:00.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.653+0000 7f6f5f418740 1 Processor -- start 2024-08-31T07:15:00.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.653+0000 7f6f5f418740 1 -- start start 2024-08-31T07:15:00.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e47d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a7f670 0x5643d9e5e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 0x5643d9e5e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f5f418740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3000e620 con 0x5643d9a7f670 2024-08-31T07:15:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f5f418740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6f30010220 con 0x5643d9a77490 2024-08-31T07:15:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f5f418740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3000ecf0 con 0x5643d9aa0350 2024-08-31T07:15:00.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f55200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a7f670 0x5643d9e5e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f56600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 0x5643d9e5e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f56600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 0x5643d9e5e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59368/0 (socket says 172.21.15.32:59368) 2024-08-31T07:15:00.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f56600640 1 -- 172.21.15.32:0/4071115169 learned_addr learned my addr 172.21.15.32:0/4071115169 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:00.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f55200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a7f670 0x5643d9e5e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36990/0 (socket says 172.21.15.32:36990) 2024-08-31T07:15:00.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f55c00640 1 --2- 172.21.15.32:0/4071115169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e47d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f55c00640 1 -- 172.21.15.32:0/4071115169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 msgr2=0x5643d9e5e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:00.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f55c00640 1 --2- 172.21.15.32:0/4071115169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 0x5643d9e5e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f55c00640 1 -- 172.21.15.32:0/4071115169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a7f670 msgr2=0x5643d9e5e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:00.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f55c00640 1 --2- 172.21.15.32:0/4071115169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a7f670 0x5643d9e5e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.654+0000 7f6f55c00640 1 -- 172.21.15.32:0/4071115169 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d9e50e70 con 0x5643d9a77490 2024-08-31T07:15:00.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.655+0000 7f6f55c00640 1 --2- 172.21.15.32:0/4071115169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e47d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f440e2d50 tx=0x7f6f440ab520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3cc51ca06ecc802 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:00.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.655+0000 7f6f42000640 1 -- 172.21.15.32:0/4071115169 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f440c1a70 con 0x5643d9a77490 2024-08-31T07:15:00.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.655+0000 7f6f42000640 1 -- 172.21.15.32:0/4071115169 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f440eb380 con 0x5643d9a77490 2024-08-31T07:15:00.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.655+0000 7f6f42000640 1 -- 172.21.15.32:0/4071115169 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f440c1df0 con 0x5643d9a77490 2024-08-31T07:15:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.656+0000 7f6f5f418740 1 -- 172.21.15.32:0/4071115169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 msgr2=0x5643d9e47d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.656+0000 7f6f5f418740 1 --2- 172.21.15.32:0/4071115169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e47d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f440e2d50 tx=0x7f6f440ab520 comp rx=0 tx=0).stop 2024-08-31T07:15:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.656+0000 7f6f5f418740 1 -- 172.21.15.32:0/4071115169 shutdown_connections 2024-08-31T07:15:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.656+0000 7f6f5f418740 1 --2- 172.21.15.32:0/4071115169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 0x5643d9e5e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.656+0000 7f6f5f418740 1 --2- 172.21.15.32:0/4071115169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a7f670 0x5643d9e5e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.656+0000 7f6f5f418740 1 --2- 172.21.15.32:0/4071115169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a77490 0x5643d9e47d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.656+0000 7f6f5f418740 1 -- 172.21.15.32:0/4071115169 >> 172.21.15.32:0/4071115169 conn(0x5643d9a94740 msgr2=0x5643d9e47630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.656+0000 7f6f5f418740 1 -- 172.21.15.32:0/4071115169 shutdown_connections 2024-08-31T07:15:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.656+0000 7f6f5f418740 1 -- 172.21.15.32:0/4071115169 wait complete. 2024-08-31T07:15:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.657+0000 7f6f5f418740 1 -- start start 2024-08-31T07:15:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.657+0000 7f6f5f418740 1 Processor -- start 2024-08-31T07:15:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 0x5643d9e56c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a7f670 0x5643d9e57220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f5f418740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 0x5643d9ec7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f5f418740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3000fa90 con 0x5643d9a77490 2024-08-31T07:15:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f5f418740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3000f3c0 con 0x5643d9a7f670 2024-08-31T07:15:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f5f418740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3000c600 con 0x5643d9aa0350 2024-08-31T07:15:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f5f418740 10 client.0 _ll_register_callbacks cb 0x7f6f3c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f55c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 0x5643d9e56c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f55200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a7f670 0x5643d9e57220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f55200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a7f670 0x5643d9e57220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55000/0 (socket says 172.21.15.32:55000) 2024-08-31T07:15:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f55c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 0x5643d9e56c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36998/0 (socket says 172.21.15.32:36998) 2024-08-31T07:15:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f55200640 1 -- 172.21.15.32:0/4172369300 learned_addr learned my addr 172.21.15.32:0/4172369300 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f56600640 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 0x5643d9ec7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f55200640 1 -- 172.21.15.32:0/4172369300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 msgr2=0x5643d9ec7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:00.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f55200640 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 0x5643d9ec7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f55200640 1 -- 172.21.15.32:0/4172369300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 msgr2=0x5643d9e56c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f55200640 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 0x5643d9e56c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.658+0000 7f6f55200640 1 -- 172.21.15.32:0/4172369300 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d9ec85d0 con 0x5643d9a7f670 2024-08-31T07:15:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.659+0000 7f6f56600640 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 0x5643d9ec7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.659+0000 7f6f55200640 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a7f670 0x5643d9e57220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c012810 tx=0x7f6f3c00f800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b685832814b829f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.659+0000 7f6f37400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:15:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.659+0000 7f6f37400640 1 -- 172.21.15.32:0/4172369300 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c01c8e0 con 0x5643d9a7f670 2024-08-31T07:15:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.659+0000 7f6f37400640 1 -- 172.21.15.32:0/4172369300 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6f3c018040 con 0x5643d9a7f670 2024-08-31T07:15:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.659+0000 7f6f5f418740 1 -- 172.21.15.32:0/4172369300 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f440eb380 con 0x5643d9a7f670 2024-08-31T07:15:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.659+0000 7f6f37400640 1 -- 172.21.15.32:0/4172369300 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6f3c01ca80 con 0x5643d9a7f670 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.659+0000 7f6f5f418740 10 client.26075 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.659+0000 7f6f5f418740 1 -- 172.21.15.32:0/4172369300 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5643d9ecfa00 con 0x5643d9a7f670 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.660+0000 7f6f37400640 1 -- 172.21.15.32:0/4172369300 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f440eb380 con 0x5643d9a7f670 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.660+0000 7f6f37400640 1 -- 172.21.15.32:0/4172369300 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f6f3c010c10 con 0x5643d9a7f670 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.660+0000 7f6f5f418740 10 client.26075 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.660+0000 7f6f5f418740 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.660+0000 7f6f5f418740 20 client.26075 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.660+0000 7f6f5f418740 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5643d9a34080 0x5643d9ed2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.660+0000 7f6f5f418740 4 client.26075 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.660+0000 7f6f5f418740 1 -- 172.21.15.32:0/4172369300 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5643d9dba140 con 0x5643d9a34080 2024-08-31T07:15:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.660+0000 7f6f55c00640 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5643d9a34080 0x5643d9ed2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.661+0000 7f6f55c00640 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5643d9a34080 0x5643d9ed2300 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7ba7b5c092b9af server_cookie=f90de86d1bf909c2 in_seq=0 out_seq=0 2024-08-31T07:15:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.661+0000 7f6f37400640 10 client.26075 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f37400640 1 -- 172.21.15.32:0/4172369300 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5643d9dba140 con 0x5643d9a34080 2024-08-31T07:15:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f37400640 10 client.26075 handle_command_reply: tid=0 2024-08-31T07:15:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 client.26075 shutdown 2024-08-31T07:15:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 -- 172.21.15.32:0/4172369300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a7f670 msgr2=0x5643d9e57220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a7f670 0x5643d9e57220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c012810 tx=0x7f6f3c00f800 comp rx=0 tx=0).stop 2024-08-31T07:15:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 -- 172.21.15.32:0/4172369300 shutdown_connections 2024-08-31T07:15:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5643d9a34080 0x5643d9ed2300 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5643d9aa0350 0x5643d9ec7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643d9a7f670 0x5643d9e57220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 --2- 172.21.15.32:0/4172369300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643d9a77490 0x5643d9e56c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f37400640 0 client.26075 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 -- 172.21.15.32:0/4172369300 >> 172.21.15.32:0/4172369300 conn(0x5643d9a94740 msgr2=0x5643d9e47ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 -- 172.21.15.32:0/4172369300 shutdown_connections 2024-08-31T07:15:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.663+0000 7f6f5f418740 1 -- 172.21.15.32:0/4172369300 wait complete. 2024-08-31T07:15:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f5f418740 20 client.26075 trim_cache size 0 max 16384 2024-08-31T07:15:00.667 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:15:00.667 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:15:00.667 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "8efe0b72-f373-45ce-94be-5e685a09cf82", 2024-08-31T07:15:00.668 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:15:00.668 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:15:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f58600640 1 -- 172.21.15.32:0/3470248621 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6f380b4d20 msgr2=0x7f6f380b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f58600640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6f380b4d20 0x7f6f380b7230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6f48006010 tx=0x7f6f4802f3f0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f58600640 1 -- 172.21.15.32:0/3470248621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f500776e0 msgr2=0x7f6f501a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f58600640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f500776e0 0x7f6f501a0810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400e9f0 tx=0x7f6f4400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f58600640 1 -- 172.21.15.32:0/3470248621 shutdown_connections 2024-08-31T07:15:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f58600640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6f380b4d20 0x7f6f380b7230 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f58600640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f501144f0 0x7f6f501a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f58600640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f50078030 0x7f6f501a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f58600640 1 --2- 172.21.15.32:0/3470248621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f500776e0 0x7f6f501a0810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.664+0000 7f6f58600640 1 -- 172.21.15.32:0/3470248621 >> 172.21.15.32:0/3470248621 conn(0x7f6f50101600 msgr2=0x7f6f50114e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.665+0000 7f6f58600640 1 -- 172.21.15.32:0/3470248621 shutdown_connections 2024-08-31T07:15:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:00.665+0000 7f6f58600640 1 -- 172.21.15.32:0/3470248621 wait complete. 2024-08-31T07:15:00.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 72388 8.34 MB/sec warmup 49 sec latency 3342.062 ms 2024-08-31T07:15:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:01 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:15:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:01 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:01 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:01 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:01 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:01 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:01 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:15:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:01 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:01 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:01 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:01 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:01 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:01.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:01 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:15:01.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:01 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:01.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:01 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:01.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:01 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:01.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:01 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:01.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:01 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:01.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:15:01.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 73910 8.32 MB/sec warmup 50 sec latency 43.256 ms 2024-08-31T07:15:02.051 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:02.393 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:02.393 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:02.393 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:02.393 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.393 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.393 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.393 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:02.394 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.394 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:02.394 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.394 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:02 smithi032 ceph-mon[17252]: pgmap v436: 97 pgs: 97 active+clean; 60 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 2.8 MiB/s wr, 131 op/s 2024-08-31T07:15:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:02.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:02.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:02 smithi073 ceph-mon[21995]: pgmap v436: 97 pgs: 97 active+clean; 60 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 2.8 MiB/s wr, 131 op/s 2024-08-31T07:15:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:02.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:02.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:02.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:02 smithi084 ceph-mon[22387]: pgmap v436: 97 pgs: 97 active+clean; 60 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 2.8 MiB/s wr, 131 op/s 2024-08-31T07:15:02.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 75266 8.29 MB/sec warmup 51 sec latency 10.713 ms 2024-08-31T07:15:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.738+0000 7f466ac00640 1 -- 172.21.15.32:0/709970142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46640776e0 msgr2=0x7f4664077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.738+0000 7f466ac00640 1 --2- 172.21.15.32:0/709970142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46640776e0 0x7f4664077af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4658009860 tx=0x7f465802a340 comp rx=0 tx=0).stop 2024-08-31T07:15:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.738+0000 7f466ac00640 1 -- 172.21.15.32:0/709970142 shutdown_connections 2024-08-31T07:15:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.738+0000 7f466ac00640 1 --2- 172.21.15.32:0/709970142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4664114310 0x7f4664116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.738+0000 7f466ac00640 1 --2- 172.21.15.32:0/709970142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4664078030 0x7f4664113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.738+0000 7f466ac00640 1 --2- 172.21.15.32:0/709970142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46640776e0 0x7f4664077af0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.738+0000 7f466ac00640 1 -- 172.21.15.32:0/709970142 >> 172.21.15.32:0/709970142 conn(0x7f4664101420 msgr2=0x7f4664103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.738+0000 7f466ac00640 1 -- 172.21.15.32:0/709970142 shutdown_connections 2024-08-31T07:15:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.738+0000 7f466ac00640 1 -- 172.21.15.32:0/709970142 wait complete. 2024-08-31T07:15:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.739+0000 7f466ac00640 1 Processor -- start 2024-08-31T07:15:02.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.739+0000 7f466ac00640 1 -- start start 2024-08-31T07:15:02.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f466ac00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46640776e0 0x7f46641a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f466ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4664078030 0x7f46641a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f466ac00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4664114310 0x7f46641a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f466ac00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4664119c90 con 0x7f4664114310 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f466ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4664119b10 con 0x7f4664078030 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f466ac00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4664119e10 con 0x7f46640776e0 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f4669800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46640776e0 0x7f46641a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f4669800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46640776e0 0x7f46641a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59388/0 (socket says 172.21.15.32:59388) 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f4669800640 1 -- 172.21.15.32:0/298839905 learned_addr learned my addr 172.21.15.32:0/298839905 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f466a200640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4664114310 0x7f46641a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.740+0000 7f4668e00640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4664078030 0x7f46641a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.741+0000 7f4669800640 1 -- 172.21.15.32:0/298839905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4664078030 msgr2=0x7f46641a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.741+0000 7f4669800640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4664078030 0x7f46641a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.741+0000 7f4669800640 1 -- 172.21.15.32:0/298839905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4664114310 msgr2=0x7f46641a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.741+0000 7f4669800640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4664114310 0x7f46641a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.741+0000 7f4669800640 1 -- 172.21.15.32:0/298839905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46641a6690 con 0x7f46640776e0 2024-08-31T07:15:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.741+0000 7f4668e00640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4664078030 0x7f46641a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:15:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.741+0000 7f4669800640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46640776e0 0x7f46641a0720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4658035040 tx=0x7f4658032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.741+0000 7f4662000640 1 -- 172.21.15.32:0/298839905 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658037040 con 0x7f46640776e0 2024-08-31T07:15:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.741+0000 7f4662000640 1 -- 172.21.15.32:0/298839905 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4658041070 con 0x7f46640776e0 2024-08-31T07:15:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.741+0000 7f466ac00640 1 -- 172.21.15.32:0/298839905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46641a6920 con 0x7f46640776e0 2024-08-31T07:15:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.742+0000 7f466ac00640 1 -- 172.21.15.32:0/298839905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46641a6da0 con 0x7f46640776e0 2024-08-31T07:15:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.742+0000 7f4662000640 1 -- 172.21.15.32:0/298839905 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658003850 con 0x7f46640776e0 2024-08-31T07:15:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.745+0000 7f4672e19740 1 Processor -- start 2024-08-31T07:15:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.745+0000 7f4672e19740 1 -- start start 2024-08-31T07:15:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.745+0000 7f4662000640 1 -- 172.21.15.32:0/298839905 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4658004050 con 0x7f46640776e0 2024-08-31T07:15:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.746+0000 7f4672e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 0x559ee4ceab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.746+0000 7f4672e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee491c1b0 0x559ee4ceb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.746+0000 7f4672e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4924390 0x559ee4ceb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.746+0000 7f4672e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559ee4c61770 con 0x559ee491c1b0 2024-08-31T07:15:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.746+0000 7f4672e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559ee4c50d00 con 0x559ee4924390 2024-08-31T07:15:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.746+0000 7f4672e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559ee4c54cb0 con 0x559ee4945070 2024-08-31T07:15:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.746+0000 7f466a200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 0x559ee4ceab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.746+0000 7f4668e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4924390 0x559ee4ceb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.746+0000 7f466a200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 0x559ee4ceab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59392/0 (socket says 172.21.15.32:59392) 2024-08-31T07:15:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.746+0000 7f466a200640 1 -- 172.21.15.32:0/565057235 learned_addr learned my addr 172.21.15.32:0/565057235 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.747+0000 7f4669800640 1 --2- 172.21.15.32:0/565057235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee491c1b0 0x559ee4ceb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.747+0000 7f4668e00640 1 -- 172.21.15.32:0/565057235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 msgr2=0x559ee4ceab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.747+0000 7f4668e00640 1 --2- 172.21.15.32:0/565057235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 0x559ee4ceab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.747+0000 7f4668e00640 1 -- 172.21.15.32:0/565057235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee491c1b0 msgr2=0x559ee4ceb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.747+0000 7f4668e00640 1 --2- 172.21.15.32:0/565057235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee491c1b0 0x559ee4ceb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.747+0000 7f4668e00640 1 -- 172.21.15.32:0/565057235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee49b0650 con 0x559ee4924390 2024-08-31T07:15:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.747+0000 7f466a200640 1 --2- 172.21.15.32:0/565057235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 0x559ee4ceab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.747+0000 7f4668e00640 1 --2- 172.21.15.32:0/565057235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4924390 0x559ee4ceb5a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f466406c3d0 tx=0x7f4650010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec2d887392005048 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.747+0000 7f4656a00640 1 -- 172.21.15.32:0/565057235 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650010d70 con 0x559ee4924390 2024-08-31T07:15:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.748+0000 7f4656a00640 1 -- 172.21.15.32:0/565057235 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46500054d0 con 0x559ee4924390 2024-08-31T07:15:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.748+0000 7f4656a00640 1 -- 172.21.15.32:0/565057235 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46500126b0 con 0x559ee4924390 2024-08-31T07:15:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.748+0000 7f4662000640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f464c0b4e70 0x7f464c0b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.748+0000 7f4662000640 1 -- 172.21.15.32:0/298839905 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f4658104e40 con 0x7f46640776e0 2024-08-31T07:15:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.748+0000 7f466a200640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f464c0b4e70 0x7f464c0b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.749+0000 7f4672e19740 1 -- 172.21.15.32:0/565057235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4924390 msgr2=0x559ee4ceb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.749+0000 7f4672e19740 1 --2- 172.21.15.32:0/565057235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4924390 0x559ee4ceb5a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f466406c3d0 tx=0x7f4650010370 comp rx=0 tx=0).stop 2024-08-31T07:15:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.749+0000 7f4672e19740 1 -- 172.21.15.32:0/565057235 shutdown_connections 2024-08-31T07:15:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.749+0000 7f4672e19740 1 --2- 172.21.15.32:0/565057235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4924390 0x559ee4ceb5a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.749+0000 7f4672e19740 1 --2- 172.21.15.32:0/565057235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee491c1b0 0x559ee4ceb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.749+0000 7f4672e19740 1 --2- 172.21.15.32:0/565057235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 0x559ee4ceab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.749+0000 7f4672e19740 1 -- 172.21.15.32:0/565057235 >> 172.21.15.32:0/565057235 conn(0x559ee4939460 msgr2=0x559ee4ce8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.749+0000 7f466a200640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f464c0b4e70 0x7f464c0b7380 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f465c009fd0 tx=0x7f465c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.750+0000 7f4672e19740 1 -- 172.21.15.32:0/565057235 shutdown_connections 2024-08-31T07:15:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.750+0000 7f4672e19740 1 -- 172.21.15.32:0/565057235 wait complete. 2024-08-31T07:15:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.750+0000 7f4672e19740 1 -- start start 2024-08-31T07:15:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.750+0000 7f4672e19740 1 Processor -- start 2024-08-31T07:15:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.751+0000 7f4672e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.751+0000 7f4672e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4924390 0x559ee4cf6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.751+0000 7f4672e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4945070 0x559ee4d025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.751+0000 7f4672e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559ee4ab3700 con 0x559ee4945070 2024-08-31T07:15:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.751+0000 7f4672e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559ee4bd7cb0 con 0x559ee491c1b0 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.751+0000 7f4672e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559ee4cf9300 con 0x559ee4924390 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.751+0000 7f4669800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.751+0000 7f4672e19740 10 client.0 _ll_register_callbacks cb 0x559ee4bfd430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.751+0000 7f4669800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55024/0 (socket says 172.21.15.32:55024) 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f4668e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4924390 0x559ee4cf6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f4668e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4924390 0x559ee4cf6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59406/0 (socket says 172.21.15.32:59406) 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f4669800640 1 -- 172.21.15.32:0/3743142429 learned_addr learned my addr 172.21.15.32:0/3743142429 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f466a200640 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4945070 0x559ee4d025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f4669800640 1 -- 172.21.15.32:0/3743142429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4924390 msgr2=0x559ee4cf6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f4669800640 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4924390 0x559ee4cf6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f4669800640 1 -- 172.21.15.32:0/3743142429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4945070 msgr2=0x559ee4d025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f4669800640 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4945070 0x559ee4d025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f4669800640 1 -- 172.21.15.32:0/3743142429 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee4c18870 con 0x559ee491c1b0 2024-08-31T07:15:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f466a200640 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4945070 0x559ee4d025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:15:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.752+0000 7f4669800640 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf6760 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f46580c87c0 tx=0x7f46580c7540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7126b414835847bc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.753+0000 7f464b400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.753+0000 7f464b400640 1 -- 172.21.15.32:0/3743142429 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658030900 con 0x559ee491c1b0 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.753+0000 7f464b400640 1 -- 172.21.15.32:0/3743142429 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f46580cb9c0 con 0x559ee491c1b0 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.753+0000 7f4672e19740 1 -- 172.21.15.32:0/3743142429 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46500054d0 con 0x559ee491c1b0 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.753+0000 7f464b400640 1 -- 172.21.15.32:0/3743142429 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4658030e70 con 0x559ee491c1b0 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.753+0000 7f4672e19740 10 client.26084 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.753+0000 7f4672e19740 1 -- 172.21.15.32:0/3743142429 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559ee4d053a0 con 0x559ee491c1b0 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.754+0000 7f464b400640 1 -- 172.21.15.32:0/3743142429 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46500054d0 con 0x559ee491c1b0 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.754+0000 7f464b400640 1 -- 172.21.15.32:0/3743142429 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f46580d43e0 con 0x559ee491c1b0 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.754+0000 7f4672e19740 10 client.26084 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.754+0000 7f4672e19740 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.754+0000 7f4672e19740 20 client.26084 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.754+0000 7f4672e19740 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559ee49039f0 0x559ee4d08990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.754+0000 7f4672e19740 4 client.26084 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.754+0000 7f4672e19740 1 -- 172.21.15.32:0/3743142429 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ee4c636c0 con 0x559ee49039f0 2024-08-31T07:15:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.755+0000 7f4668e00640 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559ee49039f0 0x559ee4d08990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.755+0000 7f4668e00640 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559ee49039f0 0x559ee4d08990 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=498bdca454570571 server_cookie=b8ed5f406c5e5506 in_seq=0 out_seq=0 2024-08-31T07:15:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.755+0000 7f464b400640 10 client.26084 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.756+0000 7f464b400640 1 -- 172.21.15.32:0/3743142429 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559ee4c636c0 con 0x559ee49039f0 2024-08-31T07:15:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.756+0000 7f464b400640 10 client.26084 handle_command_reply: tid=0 2024-08-31T07:15:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.756+0000 7f4672e19740 1 client.26084 shutdown 2024-08-31T07:15:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 1 -- 172.21.15.32:0/3743142429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 msgr2=0x559ee4cf6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf6760 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f46580c87c0 tx=0x7f46580c7540 comp rx=0 tx=0).stop 2024-08-31T07:15:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 1 -- 172.21.15.32:0/3743142429 shutdown_connections 2024-08-31T07:15:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559ee49039f0 0x559ee4d08990 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4945070 0x559ee4d025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4924390 0x559ee4cf6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 1 --2- 172.21.15.32:0/3743142429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf6760 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f464b400640 0 client.26084 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:02.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 1 -- 172.21.15.32:0/3743142429 >> 172.21.15.32:0/3743142429 conn(0x559ee4939460 msgr2=0x559ee4cedcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:02.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 1 -- 172.21.15.32:0/3743142429 shutdown_connections 2024-08-31T07:15:02.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 1 -- 172.21.15.32:0/3743142429 wait complete. 2024-08-31T07:15:02.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.757+0000 7f4672e19740 20 client.26084 trim_cache size 0 max 16384 2024-08-31T07:15:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.777+0000 7f4672e19740 1 Processor -- start 2024-08-31T07:15:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.777+0000 7f4672e19740 1 -- start start 2024-08-31T07:15:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f4672e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f4672e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4924390 0x559ee4d024f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f4672e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 0x559ee4ce8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f4672e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46641b5730 con 0x559ee4924390 2024-08-31T07:15:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f4672e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46641b7330 con 0x559ee491c1b0 2024-08-31T07:15:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f4672e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f46641b5e00 con 0x559ee4945070 2024-08-31T07:15:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f4669800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f466a200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 0x559ee4ce8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f4669800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55030/0 (socket says 172.21.15.32:55030) 2024-08-31T07:15:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f4669800640 1 -- 172.21.15.32:0/2284971988 learned_addr learned my addr 172.21.15.32:0/2284971988 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.778+0000 7f4668e00640 1 --2- 172.21.15.32:0/2284971988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4924390 0x559ee4d024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.779+0000 7f4669800640 1 -- 172.21.15.32:0/2284971988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 msgr2=0x559ee4ce8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.779+0000 7f4669800640 1 --2- 172.21.15.32:0/2284971988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 0x559ee4ce8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.779+0000 7f4669800640 1 -- 172.21.15.32:0/2284971988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4924390 msgr2=0x559ee4d024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.779+0000 7f4669800640 1 --2- 172.21.15.32:0/2284971988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4924390 0x559ee4d024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.779+0000 7f4669800640 1 -- 172.21.15.32:0/2284971988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee4cece10 con 0x559ee491c1b0 2024-08-31T07:15:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.779+0000 7f4669800640 1 --2- 172.21.15.32:0/2284971988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf0340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f46580095f0 tx=0x7f46580303b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e8ce8cc501a502 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.779+0000 7f4656000640 1 -- 172.21.15.32:0/2284971988 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46580e4020 con 0x559ee491c1b0 2024-08-31T07:15:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.779+0000 7f4656000640 1 -- 172.21.15.32:0/2284971988 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4658030900 con 0x559ee491c1b0 2024-08-31T07:15:02.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.779+0000 7f4656000640 1 -- 172.21.15.32:0/2284971988 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46580e0e60 con 0x559ee491c1b0 2024-08-31T07:15:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.780+0000 7f4672e19740 1 -- 172.21.15.32:0/2284971988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 msgr2=0x559ee4cf0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.780+0000 7f4672e19740 1 --2- 172.21.15.32:0/2284971988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf0340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f46580095f0 tx=0x7f46580303b0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.780+0000 7f4672e19740 1 -- 172.21.15.32:0/2284971988 shutdown_connections 2024-08-31T07:15:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.780+0000 7f4672e19740 1 --2- 172.21.15.32:0/2284971988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee4945070 0x559ee4ce8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.780+0000 7f4672e19740 1 --2- 172.21.15.32:0/2284971988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4924390 0x559ee4d024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.780+0000 7f4672e19740 1 --2- 172.21.15.32:0/2284971988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee491c1b0 0x559ee4cf0340 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.780+0000 7f4672e19740 1 -- 172.21.15.32:0/2284971988 >> 172.21.15.32:0/2284971988 conn(0x559ee4939460 msgr2=0x559ee4cefc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.781+0000 7f4672e19740 1 -- 172.21.15.32:0/2284971988 shutdown_connections 2024-08-31T07:15:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.781+0000 7f4672e19740 1 -- 172.21.15.32:0/2284971988 wait complete. 2024-08-31T07:15:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.781+0000 7f4672e19740 1 -- start start 2024-08-31T07:15:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.781+0000 7f4672e19740 1 Processor -- start 2024-08-31T07:15:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4672e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee491c1b0 0x559ee4cfee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4672e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4924390 0x559ee4cf9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4672e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4945070 0x559ee4cf9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4672e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46641b6ba0 con 0x559ee4924390 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4672e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46641b64d0 con 0x559ee4945070 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4672e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f46641b3710 con 0x559ee491c1b0 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4672e19740 10 client.0 _ll_register_callbacks cb 0x7f4658004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4669800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee491c1b0 0x559ee4cfee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4669800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee491c1b0 0x559ee4cfee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59436/0 (socket says 172.21.15.32:59436) 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4669800640 1 -- 172.21.15.32:0/3318759470 learned_addr learned my addr 172.21.15.32:0/3318759470 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f4668e00640 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4924390 0x559ee4cf9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.782+0000 7f466a200640 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4945070 0x559ee4cf9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f4669800640 1 -- 172.21.15.32:0/3318759470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4945070 msgr2=0x559ee4cf9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f4669800640 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4945070 0x559ee4cf9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f4669800640 1 -- 172.21.15.32:0/3318759470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4924390 msgr2=0x559ee4cf9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f4669800640 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4924390 0x559ee4cf9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f4669800640 1 -- 172.21.15.32:0/3318759470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee4cfa320 con 0x559ee491c1b0 2024-08-31T07:15:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f466a200640 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4945070 0x559ee4cf9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f4669800640 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee491c1b0 0x559ee4cfee80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46580c75d0 tx=0x7f46580c7410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d819f00dc0bf51 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f464b400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:15:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f464b400640 1 -- 172.21.15.32:0/3318759470 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46580e4070 con 0x559ee491c1b0 2024-08-31T07:15:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f464b400640 1 -- 172.21.15.32:0/3318759470 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f46580ee2c0 con 0x559ee491c1b0 2024-08-31T07:15:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.783+0000 7f4672e19740 1 -- 172.21.15.32:0/3318759470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4658030900 con 0x559ee491c1b0 2024-08-31T07:15:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f464b400640 1 -- 172.21.15.32:0/3318759470 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f46580e0e60 con 0x559ee491c1b0 2024-08-31T07:15:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f4672e19740 10 client.26104 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f4672e19740 1 -- 172.21.15.32:0/3318759470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559ee4cfa320 con 0x559ee491c1b0 2024-08-31T07:15:02.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f464b400640 1 -- 172.21.15.32:0/3318759470 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4658030900 con 0x559ee491c1b0 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f464b400640 1 -- 172.21.15.32:0/3318759470 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f46580c6f50 con 0x559ee491c1b0 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f4672e19740 10 client.26104 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f4672e19740 10 client.26104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f4672e19740 20 client.26104 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f4672e19740 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559ee48d8da0 0x559ee4d768a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f4672e19740 4 client.26104 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.784+0000 7f4672e19740 1 -- 172.21.15.32:0/3318759470 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ee4c5f160 con 0x559ee48d8da0 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.785+0000 7f4668e00640 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559ee48d8da0 0x559ee4d768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.785+0000 7f4668e00640 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559ee48d8da0 0x559ee4d768a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e8618bc516cf322 server_cookie=92eaab970da640e4 in_seq=0 out_seq=0 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.785+0000 7f464b400640 10 client.26104 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.785+0000 7f464b400640 1 -- 172.21.15.32:0/3318759470 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x559ee4c5f160 con 0x559ee48d8da0 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.785+0000 7f464b400640 10 client.26104 handle_command_reply: tid=0 2024-08-31T07:15:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.785+0000 7f4672e19740 1 client.26104 shutdown 2024-08-31T07:15:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 1 -- 172.21.15.32:0/3318759470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee491c1b0 msgr2=0x559ee4cfee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee491c1b0 0x559ee4cfee80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46580c75d0 tx=0x7f46580c7410 comp rx=0 tx=0).stop 2024-08-31T07:15:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 1 -- 172.21.15.32:0/3318759470 shutdown_connections 2024-08-31T07:15:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559ee48d8da0 0x559ee4d768a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ee4945070 0x559ee4cf9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559ee4924390 0x559ee4cf9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 1 --2- 172.21.15.32:0/3318759470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ee491c1b0 0x559ee4cfee80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f464b400640 0 client.26104 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 1 -- 172.21.15.32:0/3318759470 >> 172.21.15.32:0/3318759470 conn(0x559ee4939460 msgr2=0x559ee4ce8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:02.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 1 -- 172.21.15.32:0/3318759470 shutdown_connections 2024-08-31T07:15:02.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 1 -- 172.21.15.32:0/3318759470 wait complete. 2024-08-31T07:15:02.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.786+0000 7f4672e19740 20 client.26104 trim_cache size 0 max 16384 2024-08-31T07:15:02.791 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:15:02.791 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:15:02.791 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:15:02.791 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:15:02.791 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:02.791 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "8efe0b72-f373-45ce-94be-5e685a09cf82", 2024-08-31T07:15:02.791 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:02.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:02.793 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:15:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:15:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:02.794 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:02.795 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:02.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:02.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:02.795 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:15:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 -- 172.21.15.32:0/298839905 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f464c0b4e70 msgr2=0x7f464c0b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f464c0b4e70 0x7f464c0b7380 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f465c009fd0 tx=0x7f465c009500 comp rx=0 tx=0).stop 2024-08-31T07:15:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 -- 172.21.15.32:0/298839905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46640776e0 msgr2=0x7f46641a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46640776e0 0x7f46641a0720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4658035040 tx=0x7f4658032d50 comp rx=0 tx=0).stop 2024-08-31T07:15:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 -- 172.21.15.32:0/298839905 shutdown_connections 2024-08-31T07:15:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f464c0b4e70 0x7f464c0b7380 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4664114310 0x7f46641a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4664078030 0x7f46641a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 --2- 172.21.15.32:0/298839905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46640776e0 0x7f46641a0720 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 -- 172.21.15.32:0/298839905 >> 172.21.15.32:0/298839905 conn(0x7f4664101420 msgr2=0x7f46641163b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:02.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 -- 172.21.15.32:0/298839905 shutdown_connections 2024-08-31T07:15:02.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:02.787+0000 7f466ac00640 1 -- 172.21.15.32:0/298839905 wait complete. 2024-08-31T07:15:03.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 75679 8.17 MB/sec warmup 52 sec latency 431.627 ms 2024-08-31T07:15:03.780 INFO:tasks.cephfs.filesystem:scrub status for tag:8efe0b72-f373-45ce-94be-5e685a09cf82 - {'path': '/', 'tag': '8efe0b72-f373-45ce-94be-5e685a09cf82', 'options': 'recursive,force'} 2024-08-31T07:15:04.323 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:04 smithi032 ceph-mon[17252]: pgmap v437: 97 pgs: 97 active+clean; 60 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-08-31T07:15:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:04 smithi073 ceph-mon[21995]: pgmap v437: 97 pgs: 97 active+clean; 60 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-08-31T07:15:04.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:04 smithi084 ceph-mon[22387]: pgmap v437: 97 pgs: 97 active+clean; 60 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-08-31T07:15:04.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 75679 8.01 MB/sec warmup 53 sec latency 1431.685 ms 2024-08-31T07:15:05.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 77151 7.97 MB/sec warmup 54 sec latency 1909.053 ms 2024-08-31T07:15:06.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:06 smithi073 ceph-mon[21995]: pgmap v438: 97 pgs: 97 active+clean; 56 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 2.8 MiB/s wr, 187 op/s 2024-08-31T07:15:06.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:06 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:15:06.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:06 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:15:06.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:06 smithi084 ceph-mon[22387]: pgmap v438: 97 pgs: 97 active+clean; 56 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 2.8 MiB/s wr, 187 op/s 2024-08-31T07:15:06.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:06 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:15:06.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:06 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:15:06.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:06 smithi032 ceph-mon[17252]: pgmap v438: 97 pgs: 97 active+clean; 56 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 2.8 MiB/s wr, 187 op/s 2024-08-31T07:15:06.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:06 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:15:06.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:06 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:15:06.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 78748 8.03 MB/sec warmup 55 sec latency 11.365 ms 2024-08-31T07:15:07.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:07 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:15:07.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:07 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:15:07.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:07 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:15:07.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:07 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:15:07.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:07 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:15:07.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:07 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:15:07.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:07 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:15:07.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:07 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:15:07.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:07 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:15:07.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 81098 8.11 MB/sec warmup 56 sec latency 11.018 ms 2024-08-31T07:15:08.302 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:08 smithi032 ceph-mon[17252]: pgmap v439: 97 pgs: 97 active+clean; 53 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-08-31T07:15:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:08 smithi073 ceph-mon[21995]: pgmap v439: 97 pgs: 97 active+clean; 53 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-08-31T07:15:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:08 smithi084 ceph-mon[22387]: pgmap v439: 97 pgs: 97 active+clean; 53 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-08-31T07:15:08.685 INFO:tasks.workunit.client.0.smithi032.stdout: 1 81866 8.08 MB/sec warmup 57 sec latency 466.077 ms 2024-08-31T07:15:09.686 INFO:tasks.workunit.client.0.smithi032.stdout: 1 82159 7.95 MB/sec warmup 58 sec latency 1077.659 ms 2024-08-31T07:15:10.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:10 smithi073 ceph-mon[21995]: pgmap v440: 97 pgs: 97 active+clean; 53 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-08-31T07:15:10.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:10 smithi084 ceph-mon[22387]: pgmap v440: 97 pgs: 97 active+clean; 53 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-08-31T07:15:10.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:10 smithi032 ceph-mon[17252]: pgmap v440: 97 pgs: 97 active+clean; 53 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-08-31T07:15:10.686 INFO:tasks.workunit.client.0.smithi032.stdout: 1 83990 7.93 MB/sec warmup 59 sec latency 274.877 ms 2024-08-31T07:15:11.686 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85504 7.99 MB/sec warmup 60 sec latency 62.612 ms 2024-08-31T07:15:12.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:12 smithi073 ceph-mon[21995]: pgmap v441: 97 pgs: 97 active+clean; 54 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.7 MiB/s wr, 259 op/s 2024-08-31T07:15:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:12 smithi084 ceph-mon[22387]: pgmap v441: 97 pgs: 97 active+clean; 54 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.7 MiB/s wr, 259 op/s 2024-08-31T07:15:12.591 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:12 smithi032 ceph-mon[17252]: pgmap v441: 97 pgs: 97 active+clean; 54 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.7 MiB/s wr, 259 op/s 2024-08-31T07:15:12.686 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85843 7.87 MB/sec warmup 61 sec latency 626.333 ms 2024-08-31T07:15:13.686 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85843 7.74 MB/sec warmup 62 sec latency 1626.442 ms 2024-08-31T07:15:14.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:14 smithi073 ceph-mon[21995]: pgmap v442: 97 pgs: 97 active+clean; 55 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-08-31T07:15:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:14 smithi084 ceph-mon[22387]: pgmap v442: 97 pgs: 97 active+clean; 55 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-08-31T07:15:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:14 smithi032 ceph-mon[17252]: pgmap v442: 97 pgs: 97 active+clean; 55 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-08-31T07:15:14.686 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85843 7.62 MB/sec warmup 63 sec latency 2626.610 ms 2024-08-31T07:15:15.686 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85967 7.51 MB/sec warmup 64 sec latency 3053.781 ms 2024-08-31T07:15:16.410 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:16 smithi032 ceph-mon[17252]: pgmap v443: 97 pgs: 97 active+clean; 55 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-08-31T07:15:16.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:16 smithi073 ceph-mon[21995]: pgmap v443: 97 pgs: 97 active+clean; 55 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-08-31T07:15:16.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:16 smithi084 ceph-mon[22387]: pgmap v443: 97 pgs: 97 active+clean; 55 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-08-31T07:15:16.687 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85967 7.39 MB/sec warmup 65 sec latency 1476.339 ms 2024-08-31T07:15:17.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:17 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:15:17.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:17 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:15:17.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:17 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:15:17.687 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85967 7.28 MB/sec warmup 66 sec latency 2476.405 ms 2024-08-31T07:15:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:18 smithi073 ceph-mon[21995]: pgmap v444: 97 pgs: 97 active+clean; 56 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-08-31T07:15:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:15:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:15:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:15:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:18 smithi084 ceph-mon[22387]: pgmap v444: 97 pgs: 97 active+clean; 56 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-08-31T07:15:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:15:18.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:15:18.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:15:18.513 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:18 smithi032 ceph-mon[17252]: pgmap v444: 97 pgs: 97 active+clean; 56 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-08-31T07:15:18.514 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:15:18.514 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:15:18.514 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:15:18.687 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85967 7.17 MB/sec warmup 67 sec latency 3476.528 ms 2024-08-31T07:15:19.687 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85967 7.06 MB/sec warmup 68 sec latency 4476.661 ms 2024-08-31T07:15:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:20 smithi073 ceph-mon[21995]: pgmap v445: 97 pgs: 97 active+clean; 56 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-08-31T07:15:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:20 smithi084 ceph-mon[22387]: pgmap v445: 97 pgs: 97 active+clean; 56 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-08-31T07:15:20.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:20 smithi032 ceph-mon[17252]: pgmap v445: 97 pgs: 97 active+clean; 56 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-08-31T07:15:20.687 INFO:tasks.workunit.client.0.smithi032.stdout: 1 87512 7.06 MB/sec warmup 69 sec latency 4902.212 ms 2024-08-31T07:15:21.687 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89141 7.13 MB/sec warmup 70 sec latency 14.390 ms 2024-08-31T07:15:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:22 smithi073 ceph-mon[21995]: pgmap v446: 97 pgs: 97 active+clean; 56 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-08-31T07:15:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:22 smithi084 ceph-mon[22387]: pgmap v446: 97 pgs: 97 active+clean; 56 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-08-31T07:15:22.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:22 smithi032 ceph-mon[17252]: pgmap v446: 97 pgs: 97 active+clean; 56 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-08-31T07:15:22.687 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89392 7.03 MB/sec warmup 71 sec latency 752.730 ms 2024-08-31T07:15:23.687 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89392 6.93 MB/sec warmup 72 sec latency 1752.815 ms 2024-08-31T07:15:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:24 smithi073 ceph-mon[21995]: pgmap v447: 97 pgs: 97 active+clean; 56 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 602 KiB/s wr, 87 op/s 2024-08-31T07:15:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:24 smithi084 ceph-mon[22387]: pgmap v447: 97 pgs: 97 active+clean; 56 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 602 KiB/s wr, 87 op/s 2024-08-31T07:15:24.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:24 smithi032 ceph-mon[17252]: pgmap v447: 97 pgs: 97 active+clean; 56 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 602 KiB/s wr, 87 op/s 2024-08-31T07:15:24.687 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89392 6.84 MB/sec warmup 73 sec latency 2752.951 ms 2024-08-31T07:15:25.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89448 6.75 MB/sec warmup 74 sec latency 3179.104 ms 2024-08-31T07:15:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:26 smithi084 ceph-mon[22387]: pgmap v448: 97 pgs: 97 active+clean; 57 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 372 KiB/s wr, 78 op/s 2024-08-31T07:15:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:26 smithi032 ceph-mon[17252]: pgmap v448: 97 pgs: 97 active+clean; 57 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 372 KiB/s wr, 78 op/s 2024-08-31T07:15:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:26 smithi073 ceph-mon[21995]: pgmap v448: 97 pgs: 97 active+clean; 57 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 372 KiB/s wr, 78 op/s 2024-08-31T07:15:26.687 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89448 6.66 MB/sec warmup 75 sec latency 1532.822 ms 2024-08-31T07:15:27.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89448 6.57 MB/sec warmup 76 sec latency 2532.982 ms 2024-08-31T07:15:28.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:28 smithi032 ceph-mon[17252]: pgmap v449: 97 pgs: 97 active+clean; 58 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 360 KiB/s wr, 82 op/s 2024-08-31T07:15:28.481 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:28 smithi073 ceph-mon[21995]: pgmap v449: 97 pgs: 97 active+clean; 58 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 360 KiB/s wr, 82 op/s 2024-08-31T07:15:28.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:28 smithi084 ceph-mon[22387]: pgmap v449: 97 pgs: 97 active+clean; 58 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 360 KiB/s wr, 82 op/s 2024-08-31T07:15:28.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89448 6.48 MB/sec warmup 77 sec latency 3533.062 ms 2024-08-31T07:15:29.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89764 6.42 MB/sec warmup 78 sec latency 4202.710 ms 2024-08-31T07:15:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:30 smithi084 ceph-mon[22387]: pgmap v450: 97 pgs: 97 active+clean; 58 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 324 KiB/s wr, 60 op/s 2024-08-31T07:15:30.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:30 smithi032 ceph-mon[17252]: pgmap v450: 97 pgs: 97 active+clean; 58 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 324 KiB/s wr, 60 op/s 2024-08-31T07:15:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:30 smithi073 ceph-mon[21995]: pgmap v450: 97 pgs: 97 active+clean; 58 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 324 KiB/s wr, 60 op/s 2024-08-31T07:15:30.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89764 6.34 MB/sec warmup 79 sec latency 1144.027 ms 2024-08-31T07:15:31.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 89764 6.26 MB/sec warmup 80 sec latency 2144.084 ms 2024-08-31T07:15:32.258 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:32 smithi032 ceph-mon[17252]: pgmap v451: 97 pgs: 97 active+clean; 64 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-08-31T07:15:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:32 smithi084 ceph-mon[22387]: pgmap v451: 97 pgs: 97 active+clean; 64 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-08-31T07:15:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:32 smithi073 ceph-mon[21995]: pgmap v451: 97 pgs: 97 active+clean; 64 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-08-31T07:15:32.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 91072 6.25 MB/sec warmup 81 sec latency 2716.800 ms 2024-08-31T07:15:33.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 92672 6.32 MB/sec warmup 82 sec latency 10.600 ms 2024-08-31T07:15:33.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:15:34.177 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:34 smithi084 ceph-mon[22387]: pgmap v452: 97 pgs: 97 active+clean; 65 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-08-31T07:15:34.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:34 smithi032 ceph-mon[17252]: pgmap v452: 97 pgs: 97 active+clean; 65 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-08-31T07:15:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:34 smithi073 ceph-mon[21995]: pgmap v452: 97 pgs: 97 active+clean; 65 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-08-31T07:15:34.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 92992 6.25 MB/sec warmup 83 sec latency 682.675 ms 2024-08-31T07:15:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.848+0000 7f5bf2e00640 1 -- 172.21.15.32:0/2780506879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec1141d0 msgr2=0x7f5bec1166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.848+0000 7f5bf2e00640 1 --2- 172.21.15.32:0/2780506879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec1141d0 0x7f5bec1166a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009ca0 tx=0x7f5be402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.848+0000 7f5bf2e00640 1 -- 172.21.15.32:0/2780506879 shutdown_connections 2024-08-31T07:15:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.848+0000 7f5bf2e00640 1 --2- 172.21.15.32:0/2780506879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec1141d0 0x7f5bec1166a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.848+0000 7f5bf2e00640 1 --2- 172.21.15.32:0/2780506879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bec075b70 0x7f5bec075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.848+0000 7f5bf2e00640 1 --2- 172.21.15.32:0/2780506879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bec077120 0x7f5bec075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.848+0000 7f5bf2e00640 1 -- 172.21.15.32:0/2780506879 >> 172.21.15.32:0/2780506879 conn(0x7f5bec07b5f0 msgr2=0x7f5bec07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.848+0000 7f5bf2e00640 1 -- 172.21.15.32:0/2780506879 shutdown_connections 2024-08-31T07:15:34.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.848+0000 7f5bf2e00640 1 -- 172.21.15.32:0/2780506879 wait complete. 2024-08-31T07:15:34.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.849+0000 7f5bf2e00640 1 Processor -- start 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.849+0000 7f5bf2e00640 1 -- start start 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5bf2e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bec075b70 0x7f5bec1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5bf2e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bec077120 0x7f5bec1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5bf2e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec1141d0 0x7f5bec1a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5bf2e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec119e60 con 0x7f5bec1141d0 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5bf2e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec119ce0 con 0x7f5bec075b70 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5bf2e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec119fe0 con 0x7f5bec077120 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec1141d0 0x7f5bec1a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bec077120 0x7f5bec1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec1141d0 0x7f5bec1a5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58826/0 (socket says 172.21.15.32:58826) 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5bf0e00640 1 -- 172.21.15.32:0/4043849185 learned_addr learned my addr 172.21.15.32:0/4043849185 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.850+0000 7f5bebe00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bec075b70 0x7f5bec1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5be3e00640 1 -- 172.21.15.32:0/4043849185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bec075b70 msgr2=0x7f5bec1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5be3e00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bec075b70 0x7f5bec1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5be3e00640 1 -- 172.21.15.32:0/4043849185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec1141d0 msgr2=0x7f5bec1a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5be3e00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec1141d0 0x7f5bec1a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5be3e00640 1 -- 172.21.15.32:0/4043849185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec1a6700 con 0x7f5bec077120 2024-08-31T07:15:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5be3e00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bec077120 0x7f5bec1a0d30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400e9c0 tx=0x7f5bd400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5be9600640 1 -- 172.21.15.32:0/4043849185 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd400ccc0 con 0x7f5bec077120 2024-08-31T07:15:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5be9600640 1 -- 172.21.15.32:0/4043849185 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bd4005020 con 0x7f5bec077120 2024-08-31T07:15:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5bf2e00640 1 -- 172.21.15.32:0/4043849185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec1a69f0 con 0x7f5bec077120 2024-08-31T07:15:34.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5be9600640 1 -- 172.21.15.32:0/4043849185 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4004a00 con 0x7f5bec077120 2024-08-31T07:15:34.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.851+0000 7f5bf2e00640 1 -- 172.21.15.32:0/4043849185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec1ad150 con 0x7f5bec077120 2024-08-31T07:15:34.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.854+0000 7f5bf9a31740 1 Processor -- start 2024-08-31T07:15:34.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bf9a31740 1 -- start start 2024-08-31T07:15:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5be9600640 1 -- 172.21.15.32:0/4043849185 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5bd40051d0 con 0x7f5bec077120 2024-08-31T07:15:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46e4070 0x556fc4a89950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a89e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a8a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556fc4a00770 con 0x556fc46c3390 2024-08-31T07:15:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556fc49efd00 con 0x556fc46e4070 2024-08-31T07:15:34.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556fc49f3cb0 con 0x556fc46bb1b0 2024-08-31T07:15:34.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46e4070 0x556fc4a89950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a8a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a89e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46e4070 0x556fc4a89950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37442/0 (socket says 172.21.15.32:37442) 2024-08-31T07:15:34.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.855+0000 7f5bf0e00640 1 -- 172.21.15.32:0/3512501865 learned_addr learned my addr 172.21.15.32:0/3512501865 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:34.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.856+0000 7f5bf0e00640 1 -- 172.21.15.32:0/3512501865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 msgr2=0x556fc4a89e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:34.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.856+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/3512501865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a89e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.856+0000 7f5bf0e00640 1 -- 172.21.15.32:0/3512501865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 msgr2=0x556fc4a8a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:34.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.856+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/3512501865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a8a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.856+0000 7f5bf0e00640 1 -- 172.21.15.32:0/3512501865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fc474f650 con 0x556fc46e4070 2024-08-31T07:15:34.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.856+0000 7f5bebe00640 1 --2- 172.21.15.32:0/3512501865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a89e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:34.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.856+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/3512501865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46e4070 0x556fc4a89950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5be40094c0 tx=0x7f5be4009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b722b10bb0a31a83 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:34.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.857+0000 7f5be1600640 1 -- 172.21.15.32:0/3512501865 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4031db0 con 0x556fc46e4070 2024-08-31T07:15:34.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.857+0000 7f5be1600640 1 -- 172.21.15.32:0/3512501865 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be4036030 con 0x556fc46e4070 2024-08-31T07:15:34.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.857+0000 7f5be1600640 1 -- 172.21.15.32:0/3512501865 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40042a0 con 0x556fc46e4070 2024-08-31T07:15:34.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.857+0000 7f5be9600640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 0x7f5bd00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.857+0000 7f5be9600640 1 -- 172.21.15.32:0/4043849185 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5bd40e54e0 con 0x7f5bec077120 2024-08-31T07:15:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.857+0000 7f5bebe00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 0x7f5bd00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.858+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3512501865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46e4070 msgr2=0x556fc4a89950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.858+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/3512501865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46e4070 0x556fc4a89950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5be40094c0 tx=0x7f5be4009a10 comp rx=0 tx=0).stop 2024-08-31T07:15:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.858+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3512501865 shutdown_connections 2024-08-31T07:15:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.858+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/3512501865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a8a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.858+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/3512501865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a89e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.858+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/3512501865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46e4070 0x556fc4a89950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.858+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3512501865 >> 172.21.15.32:0/3512501865 conn(0x556fc46d8460 msgr2=0x556fc4a87840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.858+0000 7f5bebe00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 0x7f5bd00b71b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x556fc4a885d0 tx=0x7f5bdc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.858+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3512501865 shutdown_connections 2024-08-31T07:15:34.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.858+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3512501865 wait complete. 2024-08-31T07:15:34.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.859+0000 7f5bf9a31740 1 -- start start 2024-08-31T07:15:34.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.859+0000 7f5bf9a31740 1 Processor -- start 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.859+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a95520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.859+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46c3390 0x556fc4a95a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.859+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46e4070 0x556fc4aa1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.859+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556fc4852700 con 0x556fc46e4070 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.859+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556fc4976cb0 con 0x556fc46c3390 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.859+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556fc4a97f50 con 0x556fc46bb1b0 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5bf9a31740 10 client.0 _ll_register_callbacks cb 0x556fc499c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46e4070 0x556fc4aa1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46c3390 0x556fc4a95a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46c3390 0x556fc4a95a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37452/0 (socket says 172.21.15.32:37452) 2024-08-31T07:15:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46e4070 0x556fc4aa1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58850/0 (socket says 172.21.15.32:58850) 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a95520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5be3e00640 1 -- 172.21.15.32:0/3941718249 learned_addr learned my addr 172.21.15.32:0/3941718249 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5be3e00640 1 -- 172.21.15.32:0/3941718249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 msgr2=0x556fc4a95520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5be3e00640 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a95520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5be3e00640 1 -- 172.21.15.32:0/3941718249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46e4070 msgr2=0x556fc4aa1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5be3e00640 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46e4070 0x556fc4aa1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.860+0000 7f5be3e00640 1 -- 172.21.15.32:0/3941718249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fc49b7870 con 0x556fc46c3390 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.861+0000 7f5be3e00640 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46c3390 0x556fc4a95a60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40a7c70 tx=0x7f5bd40aba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2fbcc6496644ded server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.861+0000 7f5bda000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.861+0000 7f5bda000640 1 -- 172.21.15.32:0/3941718249 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd401c070 con 0x556fc46c3390 2024-08-31T07:15:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.861+0000 7f5bda000640 1 -- 172.21.15.32:0/3941718249 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5bd40b01e0 con 0x556fc46c3390 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.861+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3941718249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be4036030 con 0x556fc46c3390 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.861+0000 7f5bda000640 1 -- 172.21.15.32:0/3941718249 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5bd40ab360 con 0x556fc46c3390 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.861+0000 7f5bf9a31740 10 client.26102 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.861+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3941718249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556fc4aa4020 con 0x556fc46c3390 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.861+0000 7f5bda000640 1 -- 172.21.15.32:0/3941718249 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be4036030 con 0x556fc46c3390 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.862+0000 7f5bda000640 1 -- 172.21.15.32:0/3941718249 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f5bd40b4a80 con 0x556fc46c3390 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.862+0000 7f5bf9a31740 10 client.26102 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.862+0000 7f5bf9a31740 10 client.26102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.862+0000 7f5bf9a31740 20 client.26102 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.862+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556fc46a29f0 0x556fc4aa7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.862+0000 7f5bf9a31740 4 client.26102 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.862+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3941718249 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fc4a026c0 con 0x556fc46a29f0 2024-08-31T07:15:34.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.862+0000 7f5bebe00640 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556fc46a29f0 0x556fc4aa7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.863+0000 7f5bebe00640 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556fc46a29f0 0x556fc4aa7590 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be11c79d0a4c1c2f server_cookie=feab96896ba5b79c in_seq=0 out_seq=0 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.863+0000 7f5bda000640 10 client.26102 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.863+0000 7f5bda000640 1 -- 172.21.15.32:0/3941718249 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556fc4a026c0 con 0x556fc46a29f0 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.863+0000 7f5bda000640 10 client.26102 handle_command_reply: tid=0 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 client.26102 shutdown 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3941718249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46c3390 msgr2=0x556fc4a95a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46c3390 0x556fc4a95a60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40a7c70 tx=0x7f5bd40aba80 comp rx=0 tx=0).stop 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3941718249 shutdown_connections 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556fc46a29f0 0x556fc4aa7590 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46e4070 0x556fc4aa1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bda000640 0 client.26102 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46c3390 0x556fc4a95a60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/3941718249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a95520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3941718249 >> 172.21.15.32:0/3941718249 conn(0x556fc46d8460 msgr2=0x556fc4a8cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3941718249 shutdown_connections 2024-08-31T07:15:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 1 -- 172.21.15.32:0/3941718249 wait complete. 2024-08-31T07:15:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.864+0000 7f5bf9a31740 20 client.26102 trim_cache size 0 max 16384 2024-08-31T07:15:34.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bf9a31740 1 Processor -- start 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bf9a31740 1 -- start start 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46a29f0 0x556fc4aa1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46bb1b0 0x556fc4a96f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a8ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b5720 con 0x556fc46c3390 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b7320 con 0x556fc46bb1b0 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b5df0 con 0x556fc46a29f0 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46a29f0 0x556fc4aa1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a8ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46a29f0 0x556fc4aa1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60302/0 (socket says 172.21.15.32:60302) 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.883+0000 7f5bebe00640 1 -- 172.21.15.32:0/1242450813 learned_addr learned my addr 172.21.15.32:0/1242450813 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:34.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.884+0000 7f5be3e00640 1 --2- 172.21.15.32:0/1242450813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46bb1b0 0x556fc4a96f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.884+0000 7f5bebe00640 1 -- 172.21.15.32:0/1242450813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46bb1b0 msgr2=0x556fc4a96f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:34.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.884+0000 7f5bebe00640 1 --2- 172.21.15.32:0/1242450813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46bb1b0 0x556fc4a96f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.884+0000 7f5bebe00640 1 -- 172.21.15.32:0/1242450813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 msgr2=0x556fc4a8ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:34.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.884+0000 7f5bebe00640 1 --2- 172.21.15.32:0/1242450813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a8ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.884+0000 7f5bebe00640 1 -- 172.21.15.32:0/1242450813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fc4a90e90 con 0x556fc46a29f0 2024-08-31T07:15:34.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.884+0000 7f5bebe00640 1 --2- 172.21.15.32:0/1242450813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46a29f0 0x556fc4aa1140 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc012cb0 tx=0x7f5bdc01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a15a6569e60f68f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:34.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.885+0000 7f5be0c00640 1 -- 172.21.15.32:0/1242450813 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc02b530 con 0x556fc46a29f0 2024-08-31T07:15:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.885+0000 7f5be0c00640 1 -- 172.21.15.32:0/1242450813 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bdc02bb80 con 0x556fc46a29f0 2024-08-31T07:15:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.885+0000 7f5be0c00640 1 -- 172.21.15.32:0/1242450813 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc01fcb0 con 0x556fc46a29f0 2024-08-31T07:15:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.885+0000 7f5bf9a31740 1 -- 172.21.15.32:0/1242450813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46a29f0 msgr2=0x556fc4aa1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.885+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/1242450813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46a29f0 0x556fc4aa1140 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc012cb0 tx=0x7f5bdc01b000 comp rx=0 tx=0).stop 2024-08-31T07:15:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.885+0000 7f5bf9a31740 1 -- 172.21.15.32:0/1242450813 shutdown_connections 2024-08-31T07:15:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.885+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/1242450813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a8ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.885+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/1242450813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46bb1b0 0x556fc4a96f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.885+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/1242450813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46a29f0 0x556fc4aa1140 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.885+0000 7f5bf9a31740 1 -- 172.21.15.32:0/1242450813 >> 172.21.15.32:0/1242450813 conn(0x556fc46d8460 msgr2=0x556fc4a96840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.886+0000 7f5bf9a31740 1 -- 172.21.15.32:0/1242450813 shutdown_connections 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.886+0000 7f5bf9a31740 1 -- 172.21.15.32:0/1242450813 wait complete. 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.886+0000 7f5bf9a31740 1 -- start start 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.886+0000 7f5bf9a31740 1 Processor -- start 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46a29f0 0x556fc4a9d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a9da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf9a31740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a98160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b6b90 con 0x556fc46c3390 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b64c0 con 0x556fc46a29f0 2024-08-31T07:15:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf9a31740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b3700 con 0x556fc46bb1b0 2024-08-31T07:15:34.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf9a31740 10 client.0 _ll_register_callbacks cb 0x7f5bd40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a98160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a9da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a98160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58858/0 (socket says 172.21.15.32:58858) 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf0e00640 1 -- 172.21.15.32:0/97660671 learned_addr learned my addr 172.21.15.32:0/97660671 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bebe00640 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46a29f0 0x556fc4a9d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.887+0000 7f5bf0e00640 1 -- 172.21.15.32:0/97660671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 msgr2=0x556fc4a9da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a9da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5bf0e00640 1 -- 172.21.15.32:0/97660671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46a29f0 msgr2=0x556fc4a9d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46a29f0 0x556fc4a9d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5bf0e00640 1 -- 172.21.15.32:0/97660671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fc4a988f0 con 0x556fc46c3390 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5be3e00640 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a9da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5bebe00640 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46a29f0 0x556fc4a9d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a98160 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009490 tx=0x7f5be402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e42d84f942fc22 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5bda000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5bda000640 1 -- 172.21.15.32:0/97660671 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be404a040 con 0x556fc46c3390 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5bda000640 1 -- 172.21.15.32:0/97660671 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5be4031450 con 0x556fc46c3390 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.888+0000 7f5bf9a31740 1 -- 172.21.15.32:0/97660671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bdc02bb80 con 0x556fc46c3390 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bda000640 1 -- 172.21.15.32:0/97660671 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5be4031650 con 0x556fc46c3390 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bf9a31740 10 client.16098 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bf9a31740 1 -- 172.21.15.32:0/97660671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556fc4b12980 con 0x556fc46c3390 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bda000640 1 -- 172.21.15.32:0/97660671 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bdc02bb80 con 0x556fc46c3390 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bda000640 1 -- 172.21.15.32:0/97660671 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f5be4043070 con 0x556fc46c3390 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bf9a31740 10 client.16098 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bf9a31740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bf9a31740 20 client.16098 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556fc46e4070 0x556fc4b14f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bf9a31740 4 client.16098 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.889+0000 7f5bf9a31740 1 -- 172.21.15.32:0/97660671 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556fc49fe160 con 0x556fc46e4070 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.890+0000 7f5bebe00640 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556fc46e4070 0x556fc4b14f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.890+0000 7f5bebe00640 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556fc46e4070 0x556fc4b14f00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=674ed6d5b4bb0c6d server_cookie=2941d55a1d3567a2 in_seq=0 out_seq=0 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.890+0000 7f5bda000640 10 client.16098 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.890+0000 7f5bda000640 1 -- 172.21.15.32:0/97660671 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x556fc49fe160 con 0x556fc46e4070 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.890+0000 7f5bda000640 10 client.16098 handle_command_reply: tid=0 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.890+0000 7f5bf9a31740 1 client.16098 shutdown 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 1 -- 172.21.15.32:0/97660671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 msgr2=0x556fc4a98160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a98160 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009490 tx=0x7f5be402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 1 -- 172.21.15.32:0/97660671 shutdown_connections 2024-08-31T07:15:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556fc46e4070 0x556fc4b14f00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556fc46c3390 0x556fc4a98160 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bda000640 0 client.16098 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fc46bb1b0 0x556fc4a9da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 1 --2- 172.21.15.32:0/97660671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fc46a29f0 0x556fc4a9d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 1 -- 172.21.15.32:0/97660671 >> 172.21.15.32:0/97660671 conn(0x556fc46d8460 msgr2=0x556fc4a8ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 1 -- 172.21.15.32:0/97660671 shutdown_connections 2024-08-31T07:15:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 1 -- 172.21.15.32:0/97660671 wait complete. 2024-08-31T07:15:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.891+0000 7f5bf9a31740 20 client.16098 trim_cache size 0 max 16384 2024-08-31T07:15:34.897 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:34.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:34.900 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:15:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 -- 172.21.15.32:0/4043849185 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 msgr2=0x7f5bd00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 0x7f5bd00b71b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x556fc4a885d0 tx=0x7f5bdc005f90 comp rx=0 tx=0).stop 2024-08-31T07:15:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 -- 172.21.15.32:0/4043849185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bec077120 msgr2=0x7f5bec1a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bec077120 0x7f5bec1a0d30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400e9c0 tx=0x7f5bd400ee80 comp rx=0 tx=0).stop 2024-08-31T07:15:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 -- 172.21.15.32:0/4043849185 shutdown_connections 2024-08-31T07:15:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 0x7f5bd00b71b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec1141d0 0x7f5bec1a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bec077120 0x7f5bec1a0d30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 --2- 172.21.15.32:0/4043849185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bec075b70 0x7f5bec1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 -- 172.21.15.32:0/4043849185 >> 172.21.15.32:0/4043849185 conn(0x7f5bec07b5f0 msgr2=0x7f5bec112aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 -- 172.21.15.32:0/4043849185 shutdown_connections 2024-08-31T07:15:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:34.892+0000 7f5bf2e00640 1 -- 172.21.15.32:0/4043849185 wait complete. 2024-08-31T07:15:35.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 94606 6.25 MB/sec warmup 84 sec latency 1108.864 ms 2024-08-31T07:15:36.062 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:15:36.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:15:36.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:36 smithi032 ceph-mon[17252]: pgmap v453: 97 pgs: 97 active+clean; 59 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.8 MiB/s wr, 113 op/s 2024-08-31T07:15:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:36.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:36 smithi084 ceph-mon[22387]: pgmap v453: 97 pgs: 97 active+clean; 59 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.8 MiB/s wr, 113 op/s 2024-08-31T07:15:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:36 smithi073 ceph-mon[21995]: pgmap v453: 97 pgs: 97 active+clean; 59 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.8 MiB/s wr, 113 op/s 2024-08-31T07:15:36.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 96185 6.32 MB/sec warmup 85 sec latency 12.920 ms 2024-08-31T07:15:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.135+0000 7fd6ab200640 1 -- 172.21.15.32:0/2909020120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a4109090 msgr2=0x7fd6a41094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:37.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.135+0000 7fd6ab200640 1 --2- 172.21.15.32:0/2909020120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a4109090 0x7fd6a41094a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd694009900 tx=0x7fd69402a410 comp rx=0 tx=0).stop 2024-08-31T07:15:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.135+0000 7fd6ab200640 1 -- 172.21.15.32:0/2909020120 shutdown_connections 2024-08-31T07:15:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.135+0000 7fd6ab200640 1 --2- 172.21.15.32:0/2909020120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a41099e0 0x7fd6a4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.135+0000 7fd6ab200640 1 --2- 172.21.15.32:0/2909020120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a4109090 0x7fd6a41094a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.135+0000 7fd6ab200640 1 --2- 172.21.15.32:0/2909020120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6a4103750 0x7fd6a4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.135+0000 7fd6ab200640 1 -- 172.21.15.32:0/2909020120 >> 172.21.15.32:0/2909020120 conn(0x7fd6a40ff1e0 msgr2=0x7fd6a4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.135+0000 7fd6ab200640 1 -- 172.21.15.32:0/2909020120 shutdown_connections 2024-08-31T07:15:37.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.135+0000 7fd6ab200640 1 -- 172.21.15.32:0/2909020120 wait complete. 2024-08-31T07:15:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.136+0000 7fd6ab200640 1 Processor -- start 2024-08-31T07:15:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6ab200640 1 -- start start 2024-08-31T07:15:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6ab200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6a4103750 0x7fd6a4076ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6ab200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a4109090 0x7fd6a40754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6ab200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a41099e0 0x7fd6a40759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6ab200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4113650 con 0x7fd6a41099e0 2024-08-31T07:15:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6ab200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a41134d0 con 0x7fd6a4109090 2024-08-31T07:15:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6ab200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a41137d0 con 0x7fd6a4103750 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6aa800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a41099e0 0x7fd6a40759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6aa800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a41099e0 0x7fd6a40759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39664/0 (socket says 172.21.15.32:39664) 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6aa800640 1 -- 172.21.15.32:0/1810935437 learned_addr learned my addr 172.21.15.32:0/1810935437 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.137+0000 7fd6a9e00640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6a4103750 0x7fd6a4076ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.138+0000 7fd6a9400640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a4109090 0x7fd6a40754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.138+0000 7fd6a9e00640 1 -- 172.21.15.32:0/1810935437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a4109090 msgr2=0x7fd6a40754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.138+0000 7fd6a9e00640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a4109090 0x7fd6a40754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.138+0000 7fd6a9e00640 1 -- 172.21.15.32:0/1810935437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a41099e0 msgr2=0x7fd6a40759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.138+0000 7fd6a9e00640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a41099e0 0x7fd6a40759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.138+0000 7fd6a9e00640 1 -- 172.21.15.32:0/1810935437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a4076210 con 0x7fd6a4103750 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.138+0000 7fd6a9400640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a4109090 0x7fd6a40754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:37.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.138+0000 7fd6a9e00640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6a4103750 0x7fd6a4076ec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd69800e9f0 tx=0x7fd69800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.138+0000 7fd69ea00640 1 -- 172.21.15.32:0/1810935437 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69800ccc0 con 0x7fd6a4103750 2024-08-31T07:15:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.139+0000 7fd6ab200640 1 -- 172.21.15.32:0/1810935437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a41aca60 con 0x7fd6a4103750 2024-08-31T07:15:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.139+0000 7fd6ab200640 1 -- 172.21.15.32:0/1810935437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a41acfc0 con 0x7fd6a4103750 2024-08-31T07:15:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.139+0000 7fd69ea00640 1 -- 172.21.15.32:0/1810935437 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6980108e0 con 0x7fd6a4103750 2024-08-31T07:15:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.139+0000 7fd69ea00640 1 -- 172.21.15.32:0/1810935437 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698018750 con 0x7fd6a4103750 2024-08-31T07:15:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.140+0000 7fd6ab200640 1 -- 172.21.15.32:0/1810935437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a41049d0 con 0x7fd6a4103750 2024-08-31T07:15:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.142+0000 7fd69ea00640 1 -- 172.21.15.32:0/1810935437 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd698010430 con 0x7fd6a4103750 2024-08-31T07:15:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.144+0000 7fd69ea00640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd68c0b4ca0 0x7fd68c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.144+0000 7fd69ea00640 1 -- 172.21.15.32:0/1810935437 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fd6980e5b30 con 0x7fd6a4103750 2024-08-31T07:15:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.144+0000 7fd6a9400640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd68c0b4ca0 0x7fd68c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.145+0000 7fd69ea00640 1 -- 172.21.15.32:0/1810935437 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd6980ac940 con 0x7fd6a4103750 2024-08-31T07:15:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.145+0000 7fd6a9400640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd68c0b4ca0 0x7fd68c0b71b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd694009230 tx=0x7fd694000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.378+0000 7fd6ab200640 1 -- 172.21.15.32:0/1810935437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6a410b7f0 con 0x7fd6a4103750 2024-08-31T07:15:37.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.379+0000 7fd69ea00640 1 -- 172.21.15.32:0/1810935437 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11967 (secure 0 0 0) 0x7fd6980ac240 con 0x7fd6a4103750 2024-08-31T07:15:37.381 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:15:37.381 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"btime":"2024-08-31T07:14:36:074596+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:14:35.939633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:15:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2024-08-31T07:15:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.381+0000 7fd6ab200640 1 -- 172.21.15.32:0/1810935437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd68c0b4ca0 msgr2=0x7fd68c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.381+0000 7fd6ab200640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd68c0b4ca0 0x7fd68c0b71b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd694009230 tx=0x7fd694000bc0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.381+0000 7fd6ab200640 1 -- 172.21.15.32:0/1810935437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6a4103750 msgr2=0x7fd6a4076ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.381+0000 7fd6ab200640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6a4103750 0x7fd6a4076ec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd69800e9f0 tx=0x7fd69800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.381+0000 7fd6ab200640 1 -- 172.21.15.32:0/1810935437 shutdown_connections 2024-08-31T07:15:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.381+0000 7fd6ab200640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd68c0b4ca0 0x7fd68c0b71b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.381+0000 7fd6ab200640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a41099e0 0x7fd6a40759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.381+0000 7fd6ab200640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6a4109090 0x7fd6a40754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.381+0000 7fd6ab200640 1 --2- 172.21.15.32:0/1810935437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6a4103750 0x7fd6a4076ec0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.381+0000 7fd6ab200640 1 -- 172.21.15.32:0/1810935437 >> 172.21.15.32:0/1810935437 conn(0x7fd6a40ff1e0 msgr2=0x7fd6a4101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.382+0000 7fd6ab200640 1 -- 172.21.15.32:0/1810935437 shutdown_connections 2024-08-31T07:15:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:37.382+0000 7fd6ab200640 1 -- 172.21.15.32:0/1810935437 wait complete. 2024-08-31T07:15:37.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 96483 6.25 MB/sec warmup 86 sec latency 651.504 ms 2024-08-31T07:15:38.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:15:38.481 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:38 smithi032 ceph-mon[17252]: pgmap v454: 97 pgs: 97 active+clean; 59 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.9 MiB/s wr, 136 op/s 2024-08-31T07:15:38.481 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:38 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1810935437' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:15:38.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:38 smithi084 ceph-mon[22387]: pgmap v454: 97 pgs: 97 active+clean; 59 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.9 MiB/s wr, 136 op/s 2024-08-31T07:15:38.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:38 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1810935437' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:15:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:38 smithi073 ceph-mon[21995]: pgmap v454: 97 pgs: 97 active+clean; 59 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.9 MiB/s wr, 136 op/s 2024-08-31T07:15:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:38 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1810935437' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:15:38.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 96483 6.18 MB/sec warmup 87 sec latency 1651.562 ms 2024-08-31T07:15:38.810 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.477+0000 7f877e400640 1 -- 172.21.15.32:0/1764725993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8778105b30 msgr2=0x7f8778105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:39.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.477+0000 7f877e400640 1 --2- 172.21.15.32:0/1764725993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8778105b30 0x7f8778105f40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8768009ca0 tx=0x7f876802a8b0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.477+0000 7f877e400640 1 -- 172.21.15.32:0/1764725993 shutdown_connections 2024-08-31T07:15:39.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.477+0000 7f877e400640 1 --2- 172.21.15.32:0/1764725993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f877810bdc0 0x7f87781125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.477+0000 7f877e400640 1 --2- 172.21.15.32:0/1764725993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877810b470 0x7f877810b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.477+0000 7f877e400640 1 --2- 172.21.15.32:0/1764725993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8778105b30 0x7f8778105f40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.477+0000 7f877e400640 1 -- 172.21.15.32:0/1764725993 >> 172.21.15.32:0/1764725993 conn(0x7f8778101600 msgr2=0x7f8778103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:39.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.477+0000 7f877e400640 1 -- 172.21.15.32:0/1764725993 shutdown_connections 2024-08-31T07:15:39.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.477+0000 7f877e400640 1 -- 172.21.15.32:0/1764725993 wait complete. 2024-08-31T07:15:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.478+0000 7f877e400640 1 Processor -- start 2024-08-31T07:15:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.478+0000 7f877e400640 1 -- start start 2024-08-31T07:15:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f877e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8778105b30 0x7f877819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f877e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877810b470 0x7f877819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f877e400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f877810bdc0 0x7f87781a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f877e400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8778115ad0 con 0x7f877810b470 2024-08-31T07:15:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f877e400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8778115950 con 0x7f8778105b30 2024-08-31T07:15:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f877e400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8778115c50 con 0x7f877810bdc0 2024-08-31T07:15:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f8777400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8778105b30 0x7f877819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f8777400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8778105b30 0x7f877819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53672/0 (socket says 172.21.15.32:53672) 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f8777400640 1 -- 172.21.15.32:0/1897202288 learned_addr learned my addr 172.21.15.32:0/1897202288 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f876fe00640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877810b470 0x7f877819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f8777e00640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f877810bdc0 0x7f87781a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f8777400640 1 -- 172.21.15.32:0/1897202288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f877810bdc0 msgr2=0x7f87781a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.479+0000 7f8777400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f877810bdc0 0x7f87781a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.480+0000 7f8777400640 1 -- 172.21.15.32:0/1897202288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877810b470 msgr2=0x7f877819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.480+0000 7f8777400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877810b470 0x7f877819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.480+0000 7f8777400640 1 -- 172.21.15.32:0/1897202288 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87781a23d0 con 0x7f8778105b30 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.480+0000 7f8777400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8778105b30 0x7f877819c460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8768005e30 tx=0x7f8768003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.480+0000 7f8774c00640 1 -- 172.21.15.32:0/1897202288 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768004040 con 0x7f8778105b30 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.480+0000 7f8774c00640 1 -- 172.21.15.32:0/1897202288 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8768040070 con 0x7f8778105b30 2024-08-31T07:15:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.480+0000 7f877e400640 1 -- 172.21.15.32:0/1897202288 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87781a2660 con 0x7f8778105b30 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.480+0000 7f8774c00640 1 -- 172.21.15.32:0/1897202288 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768031b30 con 0x7f8778105b30 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.480+0000 7f877e400640 1 -- 172.21.15.32:0/1897202288 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87781a8e20 con 0x7f8778105b30 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.483+0000 7f878501a740 1 Processor -- start 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f878501a740 1 -- start start 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f8774c00640 1 -- 172.21.15.32:0/1897202288 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8768031cf0 con 0x7f8778105b30 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f878501a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f878501a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf11b0 0x5571fc0bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f878501a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 0x5571fc0c0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f878501a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5571fc0367b0 con 0x5571fbd1a070 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f878501a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5571fc025d40 con 0x5571fbcf9390 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f878501a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5571fc029cf0 con 0x5571fbcf11b0 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f8777e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f8777e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39680/0 (socket says 172.21.15.32:39680) 2024-08-31T07:15:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f8777e00640 1 -- 172.21.15.32:0/1187334010 learned_addr learned my addr 172.21.15.32:0/1187334010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.484+0000 7f8777400640 1 --2- 172.21.15.32:0/1187334010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf11b0 0x5571fc0bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.485+0000 7f876fe00640 1 --2- 172.21.15.32:0/1187334010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 0x5571fc0c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.485+0000 7f8777e00640 1 -- 172.21.15.32:0/1187334010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf11b0 msgr2=0x5571fc0bfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.485+0000 7f8777e00640 1 --2- 172.21.15.32:0/1187334010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf11b0 0x5571fc0bfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.485+0000 7f8777e00640 1 -- 172.21.15.32:0/1187334010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 msgr2=0x5571fc0c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.485+0000 7f8777e00640 1 --2- 172.21.15.32:0/1187334010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 0x5571fc0c0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.485+0000 7f8777e00640 1 -- 172.21.15.32:0/1187334010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571fbd85650 con 0x5571fbd1a070 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.485+0000 7f876fe00640 1 --2- 172.21.15.32:0/1187334010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 0x5571fc0c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.485+0000 7f8777e00640 1 --2- 172.21.15.32:0/1187334010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bf990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f877000b6b0 tx=0x7f877000bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2eebb4922e3152 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.485+0000 7f876cc00640 1 -- 172.21.15.32:0/1187334010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770003bd0 con 0x5571fbd1a070 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.486+0000 7f876cc00640 1 -- 172.21.15.32:0/1187334010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8770015070 con 0x5571fbd1a070 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.486+0000 7f876cc00640 1 -- 172.21.15.32:0/1187334010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87700116a0 con 0x5571fbd1a070 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.486+0000 7f8774c00640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f875c0b4ca0 0x7f875c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.486+0000 7f8774c00640 1 -- 172.21.15.32:0/1897202288 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f87681065f0 con 0x7f8778105b30 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.486+0000 7f8777400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f875c0b4ca0 0x7f875c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.486+0000 7f878501a740 1 -- 172.21.15.32:0/1187334010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 msgr2=0x5571fc0bf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.486+0000 7f878501a740 1 --2- 172.21.15.32:0/1187334010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bf990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f877000b6b0 tx=0x7f877000bb70 comp rx=0 tx=0).stop 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.487+0000 7f878501a740 1 -- 172.21.15.32:0/1187334010 shutdown_connections 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.487+0000 7f878501a740 1 --2- 172.21.15.32:0/1187334010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 0x5571fc0c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.487+0000 7f878501a740 1 --2- 172.21.15.32:0/1187334010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf11b0 0x5571fc0bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.487+0000 7f878501a740 1 --2- 172.21.15.32:0/1187334010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bf990 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.487+0000 7f878501a740 1 -- 172.21.15.32:0/1187334010 >> 172.21.15.32:0/1187334010 conn(0x5571fbd0e460 msgr2=0x5571fc0bd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.487+0000 7f878501a740 1 -- 172.21.15.32:0/1187334010 shutdown_connections 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.487+0000 7f878501a740 1 -- 172.21.15.32:0/1187334010 wait complete. 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.487+0000 7f8777400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f875c0b4ca0 0x7f875c0b71b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5571fc0be610 tx=0x7f876810c110 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.487+0000 7f878501a740 1 -- start start 2024-08-31T07:15:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.488+0000 7f878501a740 1 Processor -- start 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.488+0000 7f878501a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf11b0 0x5571fc0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.488+0000 7f878501a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf9390 0x5571fc0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.488+0000 7f878501a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0d74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.488+0000 7f878501a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5571fbe88700 con 0x5571fbd1a070 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.488+0000 7f878501a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5571fbfaccf0 con 0x5571fbcf11b0 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.488+0000 7f878501a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5571fc0ce170 con 0x5571fbcf9390 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.488+0000 7f878501a740 10 client.0 _ll_register_callbacks cb 0x5571fbfd2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8777e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf11b0 0x5571fc0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8777e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf11b0 0x5571fc0cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53692/0 (socket says 172.21.15.32:53692) 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8777e00640 1 -- 172.21.15.32:0/140747225 learned_addr learned my addr 172.21.15.32:0/140747225 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8777e00640 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0d74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8777e00640 1 -- 172.21.15.32:0/140747225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf9390 msgr2=0x5571fc0cba90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8777e00640 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf9390 0x5571fc0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8777e00640 1 -- 172.21.15.32:0/140747225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 msgr2=0x5571fc0d74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8777e00640 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0d74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8777e00640 1 -- 172.21.15.32:0/140747225 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571fbfed8b0 con 0x5571fbcf11b0 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8777e00640 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf11b0 0x5571fc0cb550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f877001ae50 tx=0x7f877001ad50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c587e11aa17afc44 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.489+0000 7f8765600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.490+0000 7f8765600640 1 -- 172.21.15.32:0/140747225 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770023040 con 0x5571fbcf11b0 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.490+0000 7f8765600640 1 -- 172.21.15.32:0/140747225 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f877001a670 con 0x5571fbcf11b0 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.490+0000 7f878501a740 1 -- 172.21.15.32:0/140747225 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8770015070 con 0x5571fbcf11b0 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.490+0000 7f8765600640 1 -- 172.21.15.32:0/140747225 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8770031050 con 0x5571fbcf11b0 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.490+0000 7f878501a740 10 client.26126 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.490+0000 7f878501a740 1 -- 172.21.15.32:0/140747225 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5571fc0da190 con 0x5571fbcf11b0 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.490+0000 7f8765600640 1 -- 172.21.15.32:0/140747225 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87700054d0 con 0x5571fbcf11b0 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.490+0000 7f8765600640 1 -- 172.21.15.32:0/140747225 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f8770004b00 con 0x5571fbcf11b0 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.491+0000 7f878501a740 10 client.26126 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.491+0000 7f878501a740 10 client.26126 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.491+0000 7f878501a740 20 client.26126 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.491+0000 7f878501a740 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5571fbcd89f0 0x5571fc0dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.491+0000 7f878501a740 4 client.26126 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.491+0000 7f878501a740 1 -- 172.21.15.32:0/140747225 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8770015070 con 0x5571fbcd89f0 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.491+0000 7f876fe00640 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5571fbcd89f0 0x5571fc0dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.492+0000 7f876fe00640 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5571fbcd89f0 0x5571fc0dd4f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eca0fc8dc7c92e42 server_cookie=4e8678b882a6a5fa in_seq=0 out_seq=0 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.492+0000 7f8765600640 10 client.26126 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f8765600640 1 -- 172.21.15.32:0/140747225 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8770015070 con 0x5571fbcd89f0 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f8765600640 10 client.26126 handle_command_reply: tid=0 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 client.26126 shutdown 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 -- 172.21.15.32:0/140747225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf11b0 msgr2=0x5571fc0cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf11b0 0x5571fc0cb550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f877001ae50 tx=0x7f877001ad50 comp rx=0 tx=0).stop 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 -- 172.21.15.32:0/140747225 shutdown_connections 2024-08-31T07:15:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5571fbcd89f0 0x5571fc0dd4f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0d74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf9390 0x5571fc0cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f8765600640 0 client.26126 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:15:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 --2- 172.21.15.32:0/140747225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf11b0 0x5571fc0cb550 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 -- 172.21.15.32:0/140747225 >> 172.21.15.32:0/140747225 conn(0x5571fbd0e460 msgr2=0x5571fc0c2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 -- 172.21.15.32:0/140747225 shutdown_connections 2024-08-31T07:15:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 1 -- 172.21.15.32:0/140747225 wait complete. 2024-08-31T07:15:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.493+0000 7f878501a740 20 client.26126 trim_cache size 0 max 16384 2024-08-31T07:15:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.507+0000 7f878501a740 1 Processor -- start 2024-08-31T07:15:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.507+0000 7f878501a740 1 -- start start 2024-08-31T07:15:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f878501a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf11b0 0x5571fc0c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f878501a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 0x5571fc0bce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f878501a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f878501a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87781b14f0 con 0x5571fbd1a070 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f878501a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87781b30f0 con 0x5571fbcf9390 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f878501a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f87781b1bc0 con 0x5571fbcf11b0 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f8777e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f8777e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39700/0 (socket says 172.21.15.32:39700) 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f8777e00640 1 -- 172.21.15.32:0/2750954288 learned_addr learned my addr 172.21.15.32:0/2750954288 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f8777e00640 1 --2- 172.21.15.32:0/2750954288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf11b0 0x5571fc0c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.508+0000 7f876fe00640 1 --2- 172.21.15.32:0/2750954288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 0x5571fc0bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.509+0000 7f8777e00640 1 -- 172.21.15.32:0/2750954288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf11b0 msgr2=0x5571fc0c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.509+0000 7f8777e00640 1 --2- 172.21.15.32:0/2750954288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf11b0 0x5571fc0c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.509+0000 7f8777e00640 1 -- 172.21.15.32:0/2750954288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 msgr2=0x5571fc0bce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:39.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.509+0000 7f8777e00640 1 --2- 172.21.15.32:0/2750954288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 0x5571fc0bce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.509+0000 7f8777e00640 1 -- 172.21.15.32:0/2750954288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571fc0cc3f0 con 0x5571fbd1a070 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.509+0000 7f8777e00640 1 --2- 172.21.15.32:0/2750954288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bd370 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f877001af20 tx=0x7f8770001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f9c7e3f97e2971 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.509+0000 7f8767e00640 1 -- 172.21.15.32:0/2750954288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770005020 con 0x5571fbd1a070 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.509+0000 7f8767e00640 1 -- 172.21.15.32:0/2750954288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8770015040 con 0x5571fbd1a070 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.509+0000 7f8767e00640 1 -- 172.21.15.32:0/2750954288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87700048e0 con 0x5571fbd1a070 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.510+0000 7f878501a740 1 -- 172.21.15.32:0/2750954288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 msgr2=0x5571fc0bd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.510+0000 7f878501a740 1 --2- 172.21.15.32:0/2750954288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bd370 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f877001af20 tx=0x7f8770001e70 comp rx=0 tx=0).stop 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.510+0000 7f878501a740 1 -- 172.21.15.32:0/2750954288 shutdown_connections 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.510+0000 7f878501a740 1 --2- 172.21.15.32:0/2750954288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbd1a070 0x5571fc0bd370 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.510+0000 7f878501a740 1 --2- 172.21.15.32:0/2750954288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbcf9390 0x5571fc0bce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.510+0000 7f878501a740 1 --2- 172.21.15.32:0/2750954288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf11b0 0x5571fc0c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.510+0000 7f878501a740 1 -- 172.21.15.32:0/2750954288 >> 172.21.15.32:0/2750954288 conn(0x5571fbd0e460 msgr2=0x5571fc0c2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.510+0000 7f878501a740 1 -- 172.21.15.32:0/2750954288 shutdown_connections 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.510+0000 7f878501a740 1 -- 172.21.15.32:0/2750954288 wait complete. 2024-08-31T07:15:39.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.511+0000 7f878501a740 1 -- start start 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.511+0000 7f878501a740 1 Processor -- start 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.511+0000 7f878501a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbcf11b0 0x5571fc0d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.511+0000 7f878501a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf9390 0x5571fc0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.511+0000 7f878501a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbd1a070 0x5571fc0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.511+0000 7f878501a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87781b2960 con 0x5571fbcf11b0 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.511+0000 7f878501a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87781b2290 con 0x5571fbd1a070 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.511+0000 7f878501a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f87781af4d0 con 0x5571fbcf9390 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.511+0000 7f878501a740 10 client.0 _ll_register_callbacks cb 0x7f8770003c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f8777e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbcf11b0 0x5571fc0d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f8777e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbcf11b0 0x5571fc0d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39702/0 (socket says 172.21.15.32:39702) 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f8777e00640 1 -- 172.21.15.32:0/3852517809 learned_addr learned my addr 172.21.15.32:0/3852517809 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f876fe00640 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf9390 0x5571fc0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f8777e00640 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbd1a070 0x5571fc0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f8777e00640 1 -- 172.21.15.32:0/3852517809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf9390 msgr2=0x5571fc0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f8777e00640 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf9390 0x5571fc0cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f8777e00640 1 -- 172.21.15.32:0/3852517809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbd1a070 msgr2=0x5571fc0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f8777e00640 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbd1a070 0x5571fc0cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f8777e00640 1 -- 172.21.15.32:0/3852517809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571fc0cfed0 con 0x5571fbcf11b0 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.512+0000 7f8777e00640 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbcf11b0 0x5571fc0d3cb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8770004ed0 tx=0x7f8770005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd747007d8ff49d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.513+0000 7f8765600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.513+0000 7f8765600640 1 -- 172.21.15.32:0/3852517809 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770042060 con 0x5571fbcf11b0 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.513+0000 7f8765600640 1 -- 172.21.15.32:0/3852517809 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f877001a670 con 0x5571fbcf11b0 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.513+0000 7f878501a740 1 -- 172.21.15.32:0/3852517809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5571fc0c2220 con 0x5571fbcf11b0 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.513+0000 7f8765600640 1 -- 172.21.15.32:0/3852517809 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8770004000 con 0x5571fbcf11b0 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.513+0000 7f878501a740 10 client.16122 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.513+0000 7f878501a740 1 -- 172.21.15.32:0/3852517809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5571fc147e90 con 0x5571fbcf11b0 2024-08-31T07:15:39.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.514+0000 7f8765600640 1 -- 172.21.15.32:0/3852517809 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5571fc0c2220 con 0x5571fbcf11b0 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.514+0000 7f8765600640 1 -- 172.21.15.32:0/3852517809 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f8770011c80 con 0x5571fbcf11b0 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.514+0000 7f878501a740 10 client.16122 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.514+0000 7f878501a740 10 client.16122 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.514+0000 7f878501a740 20 client.16122 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.514+0000 7f878501a740 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5571fbcadda0 0x5571fc14a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.514+0000 7f878501a740 4 client.16122 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.514+0000 7f878501a740 1 -- 172.21.15.32:0/3852517809 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571fc038700 con 0x5571fbcadda0 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.514+0000 7f876fe00640 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5571fbcadda0 0x5571fc14a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f876fe00640 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5571fbcadda0 0x5571fc14a620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bec7d225e64f01de server_cookie=21c0acec5a2f6623 in_seq=0 out_seq=0 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f8765600640 10 client.16122 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f8765600640 1 -- 172.21.15.32:0/3852517809 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571fc038700 con 0x5571fbcadda0 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f8765600640 10 client.16122 handle_command_reply: tid=0 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f878501a740 1 client.16122 shutdown 2024-08-31T07:15:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f878501a740 1 -- 172.21.15.32:0/3852517809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbcf11b0 msgr2=0x5571fc0d3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f878501a740 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbcf11b0 0x5571fc0d3cb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8770004ed0 tx=0x7f8770005cc0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f878501a740 1 -- 172.21.15.32:0/3852517809 shutdown_connections 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f878501a740 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5571fbcadda0 0x5571fc14a620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f878501a740 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5571fbd1a070 0x5571fc0cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f878501a740 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5571fbcf9390 0x5571fc0cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f878501a740 1 --2- 172.21.15.32:0/3852517809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5571fbcf11b0 0x5571fc0d3cb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f8765600640 0 client.16122 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.515+0000 7f878501a740 1 -- 172.21.15.32:0/3852517809 >> 172.21.15.32:0/3852517809 conn(0x5571fbd0e460 msgr2=0x5571fc0cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.516+0000 7f878501a740 1 -- 172.21.15.32:0/3852517809 shutdown_connections 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.516+0000 7f878501a740 1 -- 172.21.15.32:0/3852517809 wait complete. 2024-08-31T07:15:39.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.516+0000 7f878501a740 20 client.16122 trim_cache size 0 max 16384 2024-08-31T07:15:39.528 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:15:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.516+0000 7f877e400640 1 -- 172.21.15.32:0/1897202288 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f875c0b4ca0 msgr2=0x7f875c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.516+0000 7f877e400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f875c0b4ca0 0x7f875c0b71b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5571fc0be610 tx=0x7f876810c110 comp rx=0 tx=0).stop 2024-08-31T07:15:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.516+0000 7f877e400640 1 -- 172.21.15.32:0/1897202288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8778105b30 msgr2=0x7f877819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.516+0000 7f877e400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8778105b30 0x7f877819c460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8768005e30 tx=0x7f8768003970 comp rx=0 tx=0).stop 2024-08-31T07:15:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.517+0000 7f877e400640 1 -- 172.21.15.32:0/1897202288 shutdown_connections 2024-08-31T07:15:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.517+0000 7f877e400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f875c0b4ca0 0x7f875c0b71b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.517+0000 7f877e400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f877810bdc0 0x7f87781a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.517+0000 7f877e400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877810b470 0x7f877819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.517+0000 7f877e400640 1 --2- 172.21.15.32:0/1897202288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8778105b30 0x7f877819c460 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.517+0000 7f877e400640 1 -- 172.21.15.32:0/1897202288 >> 172.21.15.32:0/1897202288 conn(0x7f8778101600 msgr2=0x7f877810aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.517+0000 7f877e400640 1 -- 172.21.15.32:0/1897202288 shutdown_connections 2024-08-31T07:15:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:39.517+0000 7f877e400640 1 -- 172.21.15.32:0/1897202288 wait complete. 2024-08-31T07:15:39.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 96483 6.11 MB/sec warmup 88 sec latency 2651.604 ms 2024-08-31T07:15:40.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:40 smithi084 ceph-mon[22387]: pgmap v455: 97 pgs: 97 active+clean; 59 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-08-31T07:15:40.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:40 smithi032 ceph-mon[17252]: pgmap v455: 97 pgs: 97 active+clean; 59 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-08-31T07:15:40.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:15:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:40 smithi073 ceph-mon[21995]: pgmap v455: 97 pgs: 97 active+clean; 59 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-08-31T07:15:40.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 97822 6.10 MB/sec warmup 89 sec latency 3075.728 ms 2024-08-31T07:15:41.048 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:41.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 99725 6.18 MB/sec warmup 90 sec latency 11.285 ms 2024-08-31T07:15:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.779+0000 7f3ff6600640 1 -- 172.21.15.32:0/2414181279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ff01099e0 msgr2=0x7f3ff0110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.779+0000 7f3ff6600640 1 --2- 172.21.15.32:0/2414181279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ff01099e0 0x7f3ff0110210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009ca0 tx=0x7f3fe802a8b0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.780+0000 7f3ff6600640 1 -- 172.21.15.32:0/2414181279 shutdown_connections 2024-08-31T07:15:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.780+0000 7f3ff6600640 1 --2- 172.21.15.32:0/2414181279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ff01099e0 0x7f3ff0110210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.780+0000 7f3ff6600640 1 --2- 172.21.15.32:0/2414181279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff0109090 0x7f3ff01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.780+0000 7f3ff6600640 1 --2- 172.21.15.32:0/2414181279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff0103750 0x7f3ff0103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.780+0000 7f3ff6600640 1 -- 172.21.15.32:0/2414181279 >> 172.21.15.32:0/2414181279 conn(0x7f3ff00ff1a0 msgr2=0x7f3ff01015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.780+0000 7f3ff6600640 1 -- 172.21.15.32:0/2414181279 shutdown_connections 2024-08-31T07:15:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.780+0000 7f3ff6600640 1 -- 172.21.15.32:0/2414181279 wait complete. 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.781+0000 7f3ff6600640 1 Processor -- start 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.781+0000 7f3ff6600640 1 -- start start 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.781+0000 7f3ff6600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff0103750 0x7f3ff00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3ff6600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff0109090 0x7f3ff00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3ff6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ff01099e0 0x7f3ff0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3ff6600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01136b0 con 0x7f3ff0103750 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3ff6600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0113530 con 0x7f3ff01099e0 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3ff6600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0113830 con 0x7f3ff0109090 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3fefe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff0109090 0x7f3ff00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3fefe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff0109090 0x7f3ff00759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45276/0 (socket says 172.21.15.32:45276) 2024-08-31T07:15:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3fefe00640 1 -- 172.21.15.32:0/1765086761 learned_addr learned my addr 172.21.15.32:0/1765086761 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3ff5c00640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ff01099e0 0x7f3ff0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3ff5200640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff0103750 0x7f3ff00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3fefe00640 1 -- 172.21.15.32:0/1765086761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ff01099e0 msgr2=0x7f3ff0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3fefe00640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ff01099e0 0x7f3ff0075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3fefe00640 1 -- 172.21.15.32:0/1765086761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff0103750 msgr2=0x7f3ff00754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3fefe00640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff0103750 0x7f3ff00754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.782+0000 7f3fefe00640 1 -- 172.21.15.32:0/1765086761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff01a8300 con 0x7f3ff0109090 2024-08-31T07:15:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.783+0000 7f3fefe00640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff0109090 0x7f3ff00759f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e9f0 tx=0x7f3fdc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.783+0000 7f3fed600640 1 -- 172.21.15.32:0/1765086761 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00ccc0 con 0x7f3ff0109090 2024-08-31T07:15:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.783+0000 7f3fed600640 1 -- 172.21.15.32:0/1765086761 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fdc0108e0 con 0x7f3ff0109090 2024-08-31T07:15:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.783+0000 7f3fed600640 1 -- 172.21.15.32:0/1765086761 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc018860 con 0x7f3ff0109090 2024-08-31T07:15:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.783+0000 7f3ff6600640 1 -- 172.21.15.32:0/1765086761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff01a85f0 con 0x7f3ff0109090 2024-08-31T07:15:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.783+0000 7f3ff6600640 1 -- 172.21.15.32:0/1765086761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff01a8b80 con 0x7f3ff0109090 2024-08-31T07:15:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.786+0000 7f3ffe83b740 1 Processor -- start 2024-08-31T07:15:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ffe83b740 1 -- start start 2024-08-31T07:15:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3fed600640 1 -- 172.21.15.32:0/1765086761 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3fdc010430 con 0x7f3ff0109090 2024-08-31T07:15:41.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564443818070 0x564443bbdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437ef1b0 0x564443bbe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437f7390 0x564443bbe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564443b347b0 con 0x564443818070 2024-08-31T07:15:41.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564443b23d40 con 0x5644437f7390 2024-08-31T07:15:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564443b27cf0 con 0x5644437ef1b0 2024-08-31T07:15:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ff5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564443818070 0x564443bbdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437ef1b0 0x564443bbe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437ef1b0 0x564443bbe0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45280/0 (socket says 172.21.15.32:45280) 2024-08-31T07:15:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ff5200640 1 -- 172.21.15.32:0/3728592074 learned_addr learned my addr 172.21.15.32:0/3728592074 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3ff5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564443818070 0x564443bbdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39724/0 (socket says 172.21.15.32:39724) 2024-08-31T07:15:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.787+0000 7f3fefe00640 1 --2- 172.21.15.32:0/3728592074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437f7390 0x564443bbe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.788+0000 7f3ff5c00640 1 -- 172.21.15.32:0/3728592074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437ef1b0 msgr2=0x564443bbe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.788+0000 7f3ff5c00640 1 --2- 172.21.15.32:0/3728592074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437ef1b0 0x564443bbe0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.788+0000 7f3ff5c00640 1 -- 172.21.15.32:0/3728592074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437f7390 msgr2=0x564443bbe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.788+0000 7f3ff5c00640 1 --2- 172.21.15.32:0/3728592074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437f7390 0x564443bbe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.788+0000 7f3ff5c00640 1 -- 172.21.15.32:0/3728592074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564443883650 con 0x564443818070 2024-08-31T07:15:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.788+0000 7f3ff5200640 1 --2- 172.21.15.32:0/3728592074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437ef1b0 0x564443bbe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.788+0000 7f3fefe00640 1 --2- 172.21.15.32:0/3728592074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437f7390 0x564443bbe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.789+0000 7f3ff5c00640 1 --2- 172.21.15.32:0/3728592074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564443818070 0x564443bbdb60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005e30 tx=0x7f3fe8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a0db17469fc5e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.789+0000 7f3fe2000640 1 -- 172.21.15.32:0/3728592074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80305a0 con 0x564443818070 2024-08-31T07:15:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.789+0000 7f3fe2000640 1 -- 172.21.15.32:0/3728592074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe8036030 con 0x564443818070 2024-08-31T07:15:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.789+0000 7f3fe2000640 1 -- 172.21.15.32:0/3728592074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80042a0 con 0x564443818070 2024-08-31T07:15:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.790+0000 7f3fed600640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3fd80b4d20 0x7f3fd80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.790+0000 7f3fed600640 1 -- 172.21.15.32:0/1765086761 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f3fdc0e5740 con 0x7f3ff0109090 2024-08-31T07:15:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.790+0000 7f3ff5200640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3fd80b4d20 0x7f3fd80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.790+0000 7f3ffe83b740 1 -- 172.21.15.32:0/3728592074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564443818070 msgr2=0x564443bbdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.790+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/3728592074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564443818070 0x564443bbdb60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005e30 tx=0x7f3fe8009800 comp rx=0 tx=0).stop 2024-08-31T07:15:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.790+0000 7f3ffe83b740 1 -- 172.21.15.32:0/3728592074 shutdown_connections 2024-08-31T07:15:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.790+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/3728592074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437f7390 0x564443bbe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.790+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/3728592074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437ef1b0 0x564443bbe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.790+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/3728592074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564443818070 0x564443bbdb60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.790+0000 7f3ffe83b740 1 -- 172.21.15.32:0/3728592074 >> 172.21.15.32:0/3728592074 conn(0x56444380c460 msgr2=0x564443bbba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.791+0000 7f3ffe83b740 1 -- 172.21.15.32:0/3728592074 shutdown_connections 2024-08-31T07:15:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.791+0000 7f3ffe83b740 1 -- 172.21.15.32:0/3728592074 wait complete. 2024-08-31T07:15:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.791+0000 7f3ff5200640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3fd80b4d20 0x7f3fd80b7230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x564443bbc7e0 tx=0x7f3fe4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.791+0000 7f3ffe83b740 1 -- start start 2024-08-31T07:15:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.791+0000 7f3ffe83b740 1 Processor -- start 2024-08-31T07:15:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437ef1b0 0x564443bc9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437f7390 0x564443bc9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564443818070 0x564443bd55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564443986700 con 0x5644437ef1b0 2024-08-31T07:15:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564443aaacf0 con 0x564443818070 2024-08-31T07:15:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564443bcc2e0 con 0x5644437f7390 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3ffe83b740 10 client.0 _ll_register_callbacks cb 0x564443ad0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3fefe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437f7390 0x564443bc9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3fefe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437f7390 0x564443bc9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45288/0 (socket says 172.21.15.32:45288) 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3fefe00640 1 -- 172.21.15.32:0/2355700719 learned_addr learned my addr 172.21.15.32:0/2355700719 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.792+0000 7f3ff5200640 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437ef1b0 0x564443bc9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3ff5c00640 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564443818070 0x564443bd55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3fefe00640 1 -- 172.21.15.32:0/2355700719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564443818070 msgr2=0x564443bd55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3fefe00640 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564443818070 0x564443bd55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3fefe00640 1 -- 172.21.15.32:0/2355700719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437ef1b0 msgr2=0x564443bc9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3fefe00640 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437ef1b0 0x564443bc9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3fefe00640 1 -- 172.21.15.32:0/2355700719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564443aeb8b0 con 0x5644437f7390 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3ff5200640 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437ef1b0 0x564443bc9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3fefe00640 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437f7390 0x564443bc9ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c6a0 tx=0x7f3fdc009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a875920ddbbbda63 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3fd6a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3fd6a00640 1 -- 172.21.15.32:0/2355700719 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0e3340 con 0x5644437f7390 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3fd6a00640 1 -- 172.21.15.32:0/2355700719 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f3fdc0930e0 con 0x5644437f7390 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.793+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2355700719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe8036030 con 0x5644437f7390 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3fd6a00640 1 -- 172.21.15.32:0/2355700719 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3fdc0aba10 con 0x5644437f7390 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3ffe83b740 10 client.26155 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2355700719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564443bd8370 con 0x5644437f7390 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3fd6a00640 1 -- 172.21.15.32:0/2355700719 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fdc0ae080 con 0x5644437f7390 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3fd6a00640 1 -- 172.21.15.32:0/2355700719 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f3fdc0b0300 con 0x5644437f7390 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3ffe83b740 10 client.26155 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3ffe83b740 10 client.26155 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3ffe83b740 20 client.26155 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5644437d69f0 0x564443bdb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3ffe83b740 4 client.26155 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.794+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2355700719 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564443b36700 con 0x5644437d69f0 2024-08-31T07:15:41.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.795+0000 7f3ff5200640 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5644437d69f0 0x564443bdb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.795+0000 7f3ff5200640 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5644437d69f0 0x564443bdb820 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93d6a20b90f91d03 server_cookie=94aa08d8fbaffdc3 in_seq=0 out_seq=0 2024-08-31T07:15:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.795+0000 7f3fd6a00640 10 client.26155 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:15:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3fd6a00640 1 -- 172.21.15.32:0/2355700719 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564443b36700 con 0x5644437d69f0 2024-08-31T07:15:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3fd6a00640 10 client.26155 handle_command_reply: tid=0 2024-08-31T07:15:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3ffe83b740 1 client.26155 shutdown 2024-08-31T07:15:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2355700719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437f7390 msgr2=0x564443bc9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437f7390 0x564443bc9ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c6a0 tx=0x7f3fdc009740 comp rx=0 tx=0).stop 2024-08-31T07:15:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2355700719 shutdown_connections 2024-08-31T07:15:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5644437d69f0 0x564443bdb820 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564443818070 0x564443bd55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644437f7390 0x564443bc9ca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/2355700719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437ef1b0 0x564443bc9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3fd6a00640 0 client.26155 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:15:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2355700719 >> 172.21.15.32:0/2355700719 conn(0x56444380c460 msgr2=0x564443bc0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.796+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2355700719 shutdown_connections 2024-08-31T07:15:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.797+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2355700719 wait complete. 2024-08-31T07:15:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.797+0000 7f3ffe83b740 20 client.26155 trim_cache size 0 max 16384 2024-08-31T07:15:41.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.811+0000 7f3ffe83b740 1 Processor -- start 2024-08-31T07:15:41.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.811+0000 7f3ffe83b740 1 -- start start 2024-08-31T07:15:41.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 0x564443bd54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564443818070 0x564443bbf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b1280 con 0x5644437f7390 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b2e80 con 0x5644437ef1b0 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b1950 con 0x564443818070 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53770/0 (socket says 172.21.15.32:53770) 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3ff5200640 1 -- 172.21.15.32:0/2714109649 learned_addr learned my addr 172.21.15.32:0/2714109649 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3ff5c00640 1 --2- 172.21.15.32:0/2714109649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564443818070 0x564443bbf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.812+0000 7f3fefe00640 1 --2- 172.21.15.32:0/2714109649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 0x564443bd54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.813+0000 7f3ff5200640 1 -- 172.21.15.32:0/2714109649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564443818070 msgr2=0x564443bbf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.813+0000 7f3ff5200640 1 --2- 172.21.15.32:0/2714109649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564443818070 0x564443bbf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.813+0000 7f3ff5200640 1 -- 172.21.15.32:0/2714109649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 msgr2=0x564443bd54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.813+0000 7f3ff5200640 1 --2- 172.21.15.32:0/2714109649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 0x564443bd54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.813+0000 7f3ff5200640 1 -- 172.21.15.32:0/2714109649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564443bbfe50 con 0x5644437ef1b0 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.813+0000 7f3ff5200640 1 --2- 172.21.15.32:0/2714109649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcb530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x564443bdbf50 tx=0x7f3fe4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56683fe8d2d81395 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.813+0000 7f3fe1600640 1 -- 172.21.15.32:0/2714109649 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4005020 con 0x5644437ef1b0 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.813+0000 7f3fe1600640 1 -- 172.21.15.32:0/2714109649 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe40047a0 con 0x5644437ef1b0 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.813+0000 7f3fe1600640 1 -- 172.21.15.32:0/2714109649 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4003820 con 0x5644437ef1b0 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.814+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2714109649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 msgr2=0x564443bcb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.814+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/2714109649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcb530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x564443bdbf50 tx=0x7f3fe4017000 comp rx=0 tx=0).stop 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.814+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2714109649 shutdown_connections 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.814+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/2714109649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564443818070 0x564443bbf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.814+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/2714109649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 0x564443bd54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.814+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/2714109649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcb530 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.814+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2714109649 >> 172.21.15.32:0/2714109649 conn(0x56444380c460 msgr2=0x564443bc7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.814+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2714109649 shutdown_connections 2024-08-31T07:15:41.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.814+0000 7f3ffe83b740 1 -- 172.21.15.32:0/2714109649 wait complete. 2024-08-31T07:15:41.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.815+0000 7f3ffe83b740 1 -- start start 2024-08-31T07:15:41.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.815+0000 7f3ffe83b740 1 Processor -- start 2024-08-31T07:15:41.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.815+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 0x564443bce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ffe83b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564443818070 0x564443c3f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b26f0 con 0x5644437f7390 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b2020 con 0x5644437ef1b0 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ffe83b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01af260 con 0x564443818070 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ffe83b740 10 client.0 _ll_register_callbacks cb 0x7f3fdc0eb220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53778/0 (socket says 172.21.15.32:53778) 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ff5c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564443818070 0x564443c3f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ff5200640 1 -- 172.21.15.32:0/487090167 learned_addr learned my addr 172.21.15.32:0/487090167 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3fefe00640 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 0x564443bce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ff5200640 1 -- 172.21.15.32:0/487090167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564443818070 msgr2=0x564443c3f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ff5200640 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564443818070 0x564443c3f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ff5200640 1 -- 172.21.15.32:0/487090167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 msgr2=0x564443bce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ff5200640 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 0x564443bce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3ff5200640 1 -- 172.21.15.32:0/487090167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564443c3f860 con 0x5644437ef1b0 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.816+0000 7f3fefe00640 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 0x564443bce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.817+0000 7f3ff5200640 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcdf20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005c20 tx=0x7f3fe4004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8e565bb556cf16 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.817+0000 7f3fd6a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.817+0000 7f3fd6a00640 1 -- 172.21.15.32:0/487090167 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe403f060 con 0x5644437ef1b0 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.817+0000 7f3ffe83b740 1 -- 172.21.15.32:0/487090167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe40047a0 con 0x5644437ef1b0 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.817+0000 7f3fd6a00640 1 -- 172.21.15.32:0/487090167 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f3fe4004f00 con 0x5644437ef1b0 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.817+0000 7f3fd6a00640 1 -- 172.21.15.32:0/487090167 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3fe4005100 con 0x5644437ef1b0 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.817+0000 7f3ffe83b740 10 client.26150 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.817+0000 7f3ffe83b740 1 -- 172.21.15.32:0/487090167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564443c46c80 con 0x5644437ef1b0 2024-08-31T07:15:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.818+0000 7f3fd6a00640 1 -- 172.21.15.32:0/487090167 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe40047a0 con 0x5644437ef1b0 2024-08-31T07:15:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.818+0000 7f3fd6a00640 1 -- 172.21.15.32:0/487090167 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f3fe4039030 con 0x5644437ef1b0 2024-08-31T07:15:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.818+0000 7f3ffe83b740 10 client.26150 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.818+0000 7f3ffe83b740 10 client.26150 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:15:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.818+0000 7f3ffe83b740 20 client.26150 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.818+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5644437abda0 0x564443c493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.818+0000 7f3ffe83b740 4 client.26150 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:15:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.818+0000 7f3ffe83b740 1 -- 172.21.15.32:0/487090167 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564443b321a0 con 0x5644437abda0 2024-08-31T07:15:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.818+0000 7f3fefe00640 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5644437abda0 0x564443c493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3fefe00640 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5644437abda0 0x564443c493c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=398fbf5d327bace server_cookie=5e16b2654fdae8f3 in_seq=0 out_seq=0 2024-08-31T07:15:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3fd6a00640 10 client.26150 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:15:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3fd6a00640 1 -- 172.21.15.32:0/487090167 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564443b321a0 con 0x5644437abda0 2024-08-31T07:15:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3fd6a00640 10 client.26150 handle_command_reply: tid=0 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3ffe83b740 1 client.26150 shutdown 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3ffe83b740 1 -- 172.21.15.32:0/487090167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 msgr2=0x564443bcdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcdf20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005c20 tx=0x7f3fe4004c70 comp rx=0 tx=0).stop 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3ffe83b740 1 -- 172.21.15.32:0/487090167 shutdown_connections 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5644437abda0 0x564443c493c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564443818070 0x564443c3f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644437f7390 0x564443bce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3ffe83b740 1 --2- 172.21.15.32:0/487090167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644437ef1b0 0x564443bcdf20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3fd6a00640 0 client.26150 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3ffe83b740 1 -- 172.21.15.32:0/487090167 >> 172.21.15.32:0/487090167 conn(0x56444380c460 msgr2=0x564443bc8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.819+0000 7f3ffe83b740 1 -- 172.21.15.32:0/487090167 shutdown_connections 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ffe83b740 1 -- 172.21.15.32:0/487090167 wait complete. 2024-08-31T07:15:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ffe83b740 20 client.26150 trim_cache size 0 max 16384 2024-08-31T07:15:41.825 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:15:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ff6600640 1 -- 172.21.15.32:0/1765086761 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3fd80b4d20 msgr2=0x7f3fd80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ff6600640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3fd80b4d20 0x7f3fd80b7230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x564443bbc7e0 tx=0x7f3fe4005f90 comp rx=0 tx=0).stop 2024-08-31T07:15:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ff6600640 1 -- 172.21.15.32:0/1765086761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff0109090 msgr2=0x7f3ff00759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ff6600640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff0109090 0x7f3ff00759f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e9f0 tx=0x7f3fdc00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ff6600640 1 -- 172.21.15.32:0/1765086761 shutdown_connections 2024-08-31T07:15:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ff6600640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3fd80b4d20 0x7f3fd80b7230 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ff6600640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ff01099e0 0x7f3ff0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ff6600640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ff0109090 0x7f3ff00759f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ff6600640 1 --2- 172.21.15.32:0/1765086761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff0103750 0x7f3ff00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.820+0000 7f3ff6600640 1 -- 172.21.15.32:0/1765086761 >> 172.21.15.32:0/1765086761 conn(0x7f3ff00ff1a0 msgr2=0x7f3ff01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.821+0000 7f3ff6600640 1 -- 172.21.15.32:0/1765086761 shutdown_connections 2024-08-31T07:15:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:41.821+0000 7f3ff6600640 1 -- 172.21.15.32:0/1765086761 wait complete. 2024-08-31T07:15:42.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:42 smithi084 ceph-mon[22387]: pgmap v456: 97 pgs: 97 active+clean; 60 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-08-31T07:15:42.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:42 smithi032 ceph-mon[17252]: pgmap v456: 97 pgs: 97 active+clean; 60 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-08-31T07:15:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:42 smithi073 ceph-mon[21995]: pgmap v456: 97 pgs: 97 active+clean; 60 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-08-31T07:15:42.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 100084 6.12 MB/sec warmup 91 sec latency 608.324 ms 2024-08-31T07:15:42.770 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:15:43.203 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:43.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 100084 6.05 MB/sec warmup 92 sec latency 1608.450 ms 2024-08-31T07:15:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.885+0000 7f6839e00640 1 -- 172.21.15.32:0/993999774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683410b2a0 msgr2=0x7f683410b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.885+0000 7f6839e00640 1 --2- 172.21.15.32:0/993999774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683410b2a0 0x7f683410b6b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6820009900 tx=0x7f682002a410 comp rx=0 tx=0).stop 2024-08-31T07:15:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.885+0000 7f6839e00640 1 -- 172.21.15.32:0/993999774 shutdown_connections 2024-08-31T07:15:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.885+0000 7f6839e00640 1 --2- 172.21.15.32:0/993999774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683410bbf0 0x7f6834112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.885+0000 7f6839e00640 1 --2- 172.21.15.32:0/993999774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683410b2a0 0x7f683410b6b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.885+0000 7f6839e00640 1 --2- 172.21.15.32:0/993999774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6834105960 0x7f6834105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.885+0000 7f6839e00640 1 -- 172.21.15.32:0/993999774 >> 172.21.15.32:0/993999774 conn(0x7f6834101410 msgr2=0x7f6834103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.886+0000 7f6839e00640 1 -- 172.21.15.32:0/993999774 shutdown_connections 2024-08-31T07:15:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.886+0000 7f6839e00640 1 -- 172.21.15.32:0/993999774 wait complete. 2024-08-31T07:15:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.887+0000 7f6839e00640 1 Processor -- start 2024-08-31T07:15:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.887+0000 7f6839e00640 1 -- start start 2024-08-31T07:15:43.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6839e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6834105960 0x7f683419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6839e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683410b2a0 0x7f683419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6839e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683410bbf0 0x7f68341a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6839e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68341158b0 con 0x7f6834105960 2024-08-31T07:15:43.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6839e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6834115730 con 0x7f683410bbf0 2024-08-31T07:15:43.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6839e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6834115a30 con 0x7f683410b2a0 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6838a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6834105960 0x7f683419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6838a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6834105960 0x7f683419c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39792/0 (socket says 172.21.15.32:39792) 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6838a00640 1 -- 172.21.15.32:0/2016502157 learned_addr learned my addr 172.21.15.32:0/2016502157 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6833e00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683410b2a0 0x7f683419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6839400640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683410bbf0 0x7f68341a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6838a00640 1 -- 172.21.15.32:0/2016502157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683410b2a0 msgr2=0x7f683419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6838a00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683410b2a0 0x7f683419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6838a00640 1 -- 172.21.15.32:0/2016502157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683410bbf0 msgr2=0x7f68341a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6838a00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683410bbf0 0x7f68341a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6838a00640 1 -- 172.21.15.32:0/2016502157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68341a2210 con 0x7f6834105960 2024-08-31T07:15:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.888+0000 7f6839400640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683410bbf0 0x7f68341a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:43.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.889+0000 7f6838a00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6834105960 0x7f683419c2a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f682800e9f0 tx=0x7f682800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.889+0000 7f6831600640 1 -- 172.21.15.32:0/2016502157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682800cd80 con 0x7f6834105960 2024-08-31T07:15:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.889+0000 7f6831600640 1 -- 172.21.15.32:0/2016502157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68280108e0 con 0x7f6834105960 2024-08-31T07:15:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.889+0000 7f6839e00640 1 -- 172.21.15.32:0/2016502157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68341a2500 con 0x7f6834105960 2024-08-31T07:15:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.889+0000 7f6831600640 1 -- 172.21.15.32:0/2016502157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828018980 con 0x7f6834105960 2024-08-31T07:15:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.889+0000 7f6839e00640 1 -- 172.21.15.32:0/2016502157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68341a8c60 con 0x7f6834105960 2024-08-31T07:15:43.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.893+0000 7f684204a740 1 Processor -- start 2024-08-31T07:15:43.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.893+0000 7f684204a740 1 -- start start 2024-08-31T07:15:43.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.893+0000 7f6831600640 1 -- 172.21.15.32:0/2016502157 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6828010430 con 0x7f6834105960 2024-08-31T07:15:43.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.893+0000 7f684204a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0af4070 0x5578a0e99b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.893+0000 7f684204a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0acb1b0 0x5578a0e9a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.893+0000 7f684204a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0e9a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.893+0000 7f684204a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5578a0e107b0 con 0x5578a0af4070 2024-08-31T07:15:43.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.893+0000 7f684204a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5578a0dffd40 con 0x5578a0ad3390 2024-08-31T07:15:43.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.893+0000 7f684204a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5578a0e03cf0 con 0x5578a0acb1b0 2024-08-31T07:15:43.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.894+0000 7f6839400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0af4070 0x5578a0e99b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.894+0000 7f6839400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0af4070 0x5578a0e99b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39800/0 (socket says 172.21.15.32:39800) 2024-08-31T07:15:43.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.894+0000 7f6839400640 1 -- 172.21.15.32:0/3823379468 learned_addr learned my addr 172.21.15.32:0/3823379468 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:43.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.894+0000 7f6838a00640 1 --2- 172.21.15.32:0/3823379468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0acb1b0 0x5578a0e9a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.894+0000 7f6833e00640 1 --2- 172.21.15.32:0/3823379468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0e9a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.894+0000 7f6839400640 1 -- 172.21.15.32:0/3823379468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0acb1b0 msgr2=0x5578a0e9a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:43.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.894+0000 7f6839400640 1 --2- 172.21.15.32:0/3823379468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0acb1b0 0x5578a0e9a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.894+0000 7f6839400640 1 -- 172.21.15.32:0/3823379468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 msgr2=0x5578a0e9a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.894+0000 7f6839400640 1 --2- 172.21.15.32:0/3823379468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0e9a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.894+0000 7f6839400640 1 -- 172.21.15.32:0/3823379468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a0b5f650 con 0x5578a0af4070 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.895+0000 7f6839400640 1 --2- 172.21.15.32:0/3823379468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0af4070 0x5578a0e99b60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f682c009fb0 tx=0x7f682c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755565e69dd628ba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.895+0000 7f6825c00640 1 -- 172.21.15.32:0/3823379468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c016070 con 0x5578a0af4070 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.895+0000 7f6825c00640 1 -- 172.21.15.32:0/3823379468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c012bc0 con 0x5578a0af4070 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.895+0000 7f6825c00640 1 -- 172.21.15.32:0/3823379468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c01bcb0 con 0x5578a0af4070 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.895+0000 7f6831600640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f681c0b4d20 0x7f681c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f6831600640 1 -- 172.21.15.32:0/2016502157 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f68280e5800 con 0x7f6834105960 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f6838a00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f681c0b4d20 0x7f681c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f684204a740 1 -- 172.21.15.32:0/3823379468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0af4070 msgr2=0x5578a0e99b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:43.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f684204a740 1 --2- 172.21.15.32:0/3823379468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0af4070 0x5578a0e99b60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f682c009fb0 tx=0x7f682c010800 comp rx=0 tx=0).stop 2024-08-31T07:15:43.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f684204a740 1 -- 172.21.15.32:0/3823379468 shutdown_connections 2024-08-31T07:15:43.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f684204a740 1 --2- 172.21.15.32:0/3823379468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0e9a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f684204a740 1 --2- 172.21.15.32:0/3823379468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0acb1b0 0x5578a0e9a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f684204a740 1 --2- 172.21.15.32:0/3823379468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0af4070 0x5578a0e99b60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f684204a740 1 -- 172.21.15.32:0/3823379468 >> 172.21.15.32:0/3823379468 conn(0x5578a0ae8460 msgr2=0x5578a0e97a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:43.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f684204a740 1 -- 172.21.15.32:0/3823379468 shutdown_connections 2024-08-31T07:15:43.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f684204a740 1 -- 172.21.15.32:0/3823379468 wait complete. 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.896+0000 7f6838a00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f681c0b4d20 0x7f681c0b7230 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f68280e3080 tx=0x7f68280eb550 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.897+0000 7f684204a740 1 -- start start 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.897+0000 7f684204a740 1 Processor -- start 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f684204a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f684204a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eaec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f684204a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0af4070 0x5578a0eb3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f684204a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5578a0c62700 con 0x5578a0acb1b0 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f684204a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5578a0d86cf0 con 0x5578a0ad3390 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f684204a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5578a0ea8340 con 0x5578a0af4070 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f684204a740 10 client.0 _ll_register_callbacks cb 0x5578a0dac470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f6839400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f6839400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39806/0 (socket says 172.21.15.32:39806) 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f6833e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eaec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f6839400640 1 -- 172.21.15.32:0/3179040718 learned_addr learned my addr 172.21.15.32:0/3179040718 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:43.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.898+0000 7f6839400640 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0af4070 0x5578a0eb3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f6833e00640 1 -- 172.21.15.32:0/3179040718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0af4070 msgr2=0x5578a0eb3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:43.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f6833e00640 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0af4070 0x5578a0eb3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f6833e00640 1 -- 172.21.15.32:0/3179040718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 msgr2=0x5578a0eae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:43.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f6833e00640 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f6833e00640 1 -- 172.21.15.32:0/3179040718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a0dc78b0 con 0x5578a0ad3390 2024-08-31T07:15:43.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f6839400640 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:43.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f6833e00640 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eaec90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f682002a930 tx=0x7f6820032f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde304cd4f17fffe server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:43.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f681a000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:15:43.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f681a000640 1 -- 172.21.15.32:0/3179040718 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820035040 con 0x5578a0ad3390 2024-08-31T07:15:43.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f684204a740 1 -- 172.21.15.32:0/3179040718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f682c012bc0 con 0x5578a0ad3390 2024-08-31T07:15:43.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.899+0000 7f681a000640 1 -- 172.21.15.32:0/3179040718 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f68200317a0 con 0x5578a0ad3390 2024-08-31T07:15:43.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.900+0000 7f681a000640 1 -- 172.21.15.32:0/3179040718 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6820004040 con 0x5578a0ad3390 2024-08-31T07:15:43.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.900+0000 7f684204a740 10 client.26162 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.900+0000 7f684204a740 1 -- 172.21.15.32:0/3179040718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5578a0ebcc50 con 0x5578a0ad3390 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.900+0000 7f681a000640 1 -- 172.21.15.32:0/3179040718 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c012bc0 con 0x5578a0ad3390 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.900+0000 7f681a000640 1 -- 172.21.15.32:0/3179040718 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f68200047e0 con 0x5578a0ad3390 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.901+0000 7f684204a740 10 client.26162 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.901+0000 7f684204a740 10 client.26162 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.901+0000 7f684204a740 20 client.26162 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.901+0000 7f684204a740 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5578a0ab29f0 0x5578a0ec0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.901+0000 7f684204a740 4 client.26162 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.901+0000 7f684204a740 1 -- 172.21.15.32:0/3179040718 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a0e12700 con 0x5578a0ab29f0 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.901+0000 7f6839400640 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5578a0ab29f0 0x5578a0ec0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.902+0000 7f6839400640 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5578a0ab29f0 0x5578a0ec0070 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ca6fedfbcb2abcb server_cookie=5c99241a21bf64be in_seq=0 out_seq=0 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.902+0000 7f681a000640 10 client.26162 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f681a000640 1 -- 172.21.15.32:0/3179040718 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5578a0e12700 con 0x5578a0ab29f0 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f681a000640 10 client.26162 handle_command_reply: tid=0 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 client.26162 shutdown 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 -- 172.21.15.32:0/3179040718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 msgr2=0x5578a0eaec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eaec90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f682002a930 tx=0x7f6820032f60 comp rx=0 tx=0).stop 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 -- 172.21.15.32:0/3179040718 shutdown_connections 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5578a0ab29f0 0x5578a0ec0070 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0af4070 0x5578a0eb3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eaec90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 --2- 172.21.15.32:0/3179040718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f681a000640 0 client.26162 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 -- 172.21.15.32:0/3179040718 >> 172.21.15.32:0/3179040718 conn(0x5578a0ae8460 msgr2=0x5578a0e9cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 -- 172.21.15.32:0/3179040718 shutdown_connections 2024-08-31T07:15:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 1 -- 172.21.15.32:0/3179040718 wait complete. 2024-08-31T07:15:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.903+0000 7f684204a740 20 client.26162 trim_cache size 0 max 16384 2024-08-31T07:15:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.917+0000 7f684204a740 1 Processor -- start 2024-08-31T07:15:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f684204a740 1 -- start start 2024-08-31T07:15:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f684204a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ae8460 0x5578a0ea8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f684204a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eb3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f684204a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0ad3390 0x5578a0e9cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f684204a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68341acff0 con 0x5578a0acb1b0 2024-08-31T07:15:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f684204a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68341aebf0 con 0x5578a0ae8460 2024-08-31T07:15:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f684204a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f68341ad6c0 con 0x5578a0ad3390 2024-08-31T07:15:43.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f6833e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eb3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f6839400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ae8460 0x5578a0ea8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f6839400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ae8460 0x5578a0ea8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53822/0 (socket says 172.21.15.32:53822) 2024-08-31T07:15:43.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f6839400640 1 -- 172.21.15.32:0/1993414376 learned_addr learned my addr 172.21.15.32:0/1993414376 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:43.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f6833e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eb3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39818/0 (socket says 172.21.15.32:39818) 2024-08-31T07:15:43.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.918+0000 7f6839400640 1 --2- 172.21.15.32:0/1993414376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0ad3390 0x5578a0e9cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.919+0000 7f6839400640 1 -- 172.21.15.32:0/1993414376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0ad3390 msgr2=0x5578a0e9cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.919+0000 7f6839400640 1 --2- 172.21.15.32:0/1993414376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0ad3390 0x5578a0e9cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.919+0000 7f6839400640 1 -- 172.21.15.32:0/1993414376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 msgr2=0x5578a0eb3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.919+0000 7f6839400640 1 --2- 172.21.15.32:0/1993414376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eb3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.919+0000 7f6839400640 1 -- 172.21.15.32:0/1993414376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a0e9c9c0 con 0x5578a0ae8460 2024-08-31T07:15:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.919+0000 7f6833e00640 1 --2- 172.21.15.32:0/1993414376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eb3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.919+0000 7f6839400640 1 --2- 172.21.15.32:0/1993414376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ae8460 0x5578a0ea8c60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f682c010d00 tx=0x7f682c01b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16100f1c18600a60 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.919+0000 7f6825200640 1 -- 172.21.15.32:0/1993414376 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c038070 con 0x5578a0ae8460 2024-08-31T07:15:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.920+0000 7f6825200640 1 -- 172.21.15.32:0/1993414376 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c012510 con 0x5578a0ae8460 2024-08-31T07:15:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.920+0000 7f6825200640 1 -- 172.21.15.32:0/1993414376 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c023040 con 0x5578a0ae8460 2024-08-31T07:15:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.920+0000 7f684204a740 1 -- 172.21.15.32:0/1993414376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ae8460 msgr2=0x5578a0ea8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.920+0000 7f684204a740 1 --2- 172.21.15.32:0/1993414376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ae8460 0x5578a0ea8c60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f682c010d00 tx=0x7f682c01b890 comp rx=0 tx=0).stop 2024-08-31T07:15:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.920+0000 7f684204a740 1 -- 172.21.15.32:0/1993414376 shutdown_connections 2024-08-31T07:15:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.920+0000 7f684204a740 1 --2- 172.21.15.32:0/1993414376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0ad3390 0x5578a0e9cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.920+0000 7f684204a740 1 --2- 172.21.15.32:0/1993414376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eb3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.920+0000 7f684204a740 1 --2- 172.21.15.32:0/1993414376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ae8460 0x5578a0ea8c60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.920+0000 7f684204a740 1 -- 172.21.15.32:0/1993414376 >> 172.21.15.32:0/1993414376 conn(0x5578a0eaee80 msgr2=0x5578a0ea8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.921+0000 7f684204a740 1 -- 172.21.15.32:0/1993414376 shutdown_connections 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.921+0000 7f684204a740 1 -- 172.21.15.32:0/1993414376 wait complete. 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.921+0000 7f684204a740 1 -- start start 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.921+0000 7f684204a740 1 Processor -- start 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f684204a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f684204a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f684204a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0ae8460 0x5578a0eb8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f684204a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68341ae460 con 0x5578a0acb1b0 2024-08-31T07:15:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f684204a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f68341add90 con 0x5578a0ad3390 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f684204a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f68341aaff0 con 0x5578a0ae8460 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f684204a740 10 client.0 _ll_register_callbacks cb 0x7f6820032bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f6833e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f6839400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f6833e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eba590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53826/0 (socket says 172.21.15.32:53826) 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f6833e00640 1 -- 172.21.15.32:0/2769476281 learned_addr learned my addr 172.21.15.32:0/2769476281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f6839400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eba050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39830/0 (socket says 172.21.15.32:39830) 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.922+0000 7f6839400640 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0ae8460 0x5578a0eb8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f6833e00640 1 -- 172.21.15.32:0/2769476281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0ae8460 msgr2=0x5578a0eb8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f6833e00640 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0ae8460 0x5578a0eb8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f6833e00640 1 -- 172.21.15.32:0/2769476281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 msgr2=0x5578a0eba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f6833e00640 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eba050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f6833e00640 1 -- 172.21.15.32:0/2769476281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a0eb92b0 con 0x5578a0ad3390 2024-08-31T07:15:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f6839400640 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f6833e00640 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eba590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6820009850 tx=0x7f6820032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21d67ebd8adc4149 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f681a000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:15:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f681a000640 1 -- 172.21.15.32:0/2769476281 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820035040 con 0x5578a0ad3390 2024-08-31T07:15:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f681a000640 1 -- 172.21.15.32:0/2769476281 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f68200048c0 con 0x5578a0ad3390 2024-08-31T07:15:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.923+0000 7f684204a740 1 -- 172.21.15.32:0/2769476281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f682c012510 con 0x5578a0ad3390 2024-08-31T07:15:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f681a000640 1 -- 172.21.15.32:0/2769476281 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f68200314a0 con 0x5578a0ad3390 2024-08-31T07:15:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f684204a740 10 client.26168 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f684204a740 1 -- 172.21.15.32:0/2769476281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5578a0f2b210 con 0x5578a0ad3390 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f681a000640 1 -- 172.21.15.32:0/2769476281 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c012510 con 0x5578a0ad3390 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f681a000640 1 -- 172.21.15.32:0/2769476281 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f6820003ce0 con 0x5578a0ad3390 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f684204a740 10 client.26168 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f684204a740 10 client.26168 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f684204a740 20 client.26168 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f684204a740 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5578a0af4070 0x5578a0f2d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f684204a740 4 client.26168 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.924+0000 7f684204a740 1 -- 172.21.15.32:0/2769476281 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578a0eaa060 con 0x5578a0af4070 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.925+0000 7f6839400640 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5578a0af4070 0x5578a0f2d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.925+0000 7f6839400640 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5578a0af4070 0x5578a0f2d9b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6e9c501df6675ed server_cookie=a869fd356b384be8 in_seq=0 out_seq=0 2024-08-31T07:15:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.925+0000 7f681a000640 10 client.26168 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f681a000640 1 -- 172.21.15.32:0/2769476281 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578a0eaa060 con 0x5578a0af4070 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f681a000640 10 client.26168 handle_command_reply: tid=0 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 client.26168 shutdown 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 -- 172.21.15.32:0/2769476281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 msgr2=0x5578a0eba590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eba590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6820009850 tx=0x7f6820032b20 comp rx=0 tx=0).stop 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 -- 172.21.15.32:0/2769476281 shutdown_connections 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5578a0af4070 0x5578a0f2d9b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578a0ae8460 0x5578a0eb8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578a0ad3390 0x5578a0eba590 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 --2- 172.21.15.32:0/2769476281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578a0acb1b0 0x5578a0eba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f681a000640 0 client.26168 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 -- 172.21.15.32:0/2769476281 >> 172.21.15.32:0/2769476281 conn(0x5578a0eaee80 msgr2=0x5578a0e9cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 -- 172.21.15.32:0/2769476281 shutdown_connections 2024-08-31T07:15:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 1 -- 172.21.15.32:0/2769476281 wait complete. 2024-08-31T07:15:43.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.926+0000 7f684204a740 20 client.26168 trim_cache size 0 max 16384 2024-08-31T07:15:43.931 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:15:43.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 -- 172.21.15.32:0/2016502157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f681c0b4d20 msgr2=0x7f681c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f681c0b4d20 0x7f681c0b7230 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f68280e3080 tx=0x7f68280eb550 comp rx=0 tx=0).stop 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 -- 172.21.15.32:0/2016502157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6834105960 msgr2=0x7f683419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6834105960 0x7f683419c2a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f682800e9f0 tx=0x7f682800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 -- 172.21.15.32:0/2016502157 shutdown_connections 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f681c0b4d20 0x7f681c0b7230 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f683410bbf0 0x7f68341a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f683410b2a0 0x7f683419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 --2- 172.21.15.32:0/2016502157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6834105960 0x7f683419c2a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 -- 172.21.15.32:0/2016502157 >> 172.21.15.32:0/2016502157 conn(0x7f6834101410 msgr2=0x7f6834103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 -- 172.21.15.32:0/2016502157 shutdown_connections 2024-08-31T07:15:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:43.927+0000 7f6839e00640 1 -- 172.21.15.32:0/2016502157 wait complete. 2024-08-31T07:15:44.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:44 smithi032 ceph-mon[17252]: pgmap v457: 97 pgs: 97 active+clean; 61 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-08-31T07:15:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:44 smithi073 ceph-mon[21995]: pgmap v457: 97 pgs: 97 active+clean; 61 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-08-31T07:15:44.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 100084 5.99 MB/sec warmup 93 sec latency 2608.563 ms 2024-08-31T07:15:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:44 smithi084 ceph-mon[22387]: pgmap v457: 97 pgs: 97 active+clean; 61 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-08-31T07:15:44.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:15:45.184 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:45.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 100152 5.92 MB/sec warmup 94 sec latency 3033.278 ms 2024-08-31T07:15:45.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.876+0000 7fb414e00640 1 -- 172.21.15.32:0/3749247951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb41010bbf0 msgr2=0x7fb410112420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:45.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.876+0000 7fb414e00640 1 --2- 172.21.15.32:0/3749247951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb41010bbf0 0x7fb410112420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb408009ca0 tx=0x7fb40802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.876+0000 7fb414e00640 1 -- 172.21.15.32:0/3749247951 shutdown_connections 2024-08-31T07:15:45.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.876+0000 7fb414e00640 1 --2- 172.21.15.32:0/3749247951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb41010bbf0 0x7fb410112420 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.876+0000 7fb414e00640 1 --2- 172.21.15.32:0/3749247951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41010b2a0 0x7fb41010b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.876+0000 7fb414e00640 1 --2- 172.21.15.32:0/3749247951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb410105960 0x7fb410105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.876+0000 7fb414e00640 1 -- 172.21.15.32:0/3749247951 >> 172.21.15.32:0/3749247951 conn(0x7fb410101410 msgr2=0x7fb410103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.876+0000 7fb414e00640 1 -- 172.21.15.32:0/3749247951 shutdown_connections 2024-08-31T07:15:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.876+0000 7fb414e00640 1 -- 172.21.15.32:0/3749247951 wait complete. 2024-08-31T07:15:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.877+0000 7fb414e00640 1 Processor -- start 2024-08-31T07:15:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.877+0000 7fb414e00640 1 -- start start 2024-08-31T07:15:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.877+0000 7fb414e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb410105960 0x7fb41019c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.877+0000 7fb414e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb41010b2a0 0x7fb41019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.877+0000 7fb414e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41010bbf0 0x7fb4101a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.877+0000 7fb414e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101158b0 con 0x7fb410105960 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb414e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb410115730 con 0x7fb41010bbf0 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb414e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb410115a30 con 0x7fb41010b2a0 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb407e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb41010b2a0 0x7fb41019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb40f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb410105960 0x7fb41019c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb407e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb41010b2a0 0x7fb41019c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42790/0 (socket says 172.21.15.32:42790) 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb407e00640 1 -- 172.21.15.32:0/3731483018 learned_addr learned my addr 172.21.15.32:0/3731483018 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb40fe00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41010bbf0 0x7fb4101a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb407e00640 1 -- 172.21.15.32:0/3731483018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41010bbf0 msgr2=0x7fb4101a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb407e00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41010bbf0 0x7fb4101a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb407e00640 1 -- 172.21.15.32:0/3731483018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb410105960 msgr2=0x7fb41019c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:45.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb407e00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb410105960 0x7fb41019c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb407e00640 1 -- 172.21.15.32:0/3731483018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4101a21b0 con 0x7fb41010b2a0 2024-08-31T07:15:45.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.878+0000 7fb40f400640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb410105960 0x7fb41019c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:45.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.879+0000 7fb407e00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb41010b2a0 0x7fb41019c7e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800e9c0 tx=0x7fb3f800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:45.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.879+0000 7fb40cc00640 1 -- 172.21.15.32:0/3731483018 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f800ccc0 con 0x7fb41010b2a0 2024-08-31T07:15:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.879+0000 7fb40cc00640 1 -- 172.21.15.32:0/3731483018 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f80054d0 con 0x7fb41010b2a0 2024-08-31T07:15:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.879+0000 7fb414e00640 1 -- 172.21.15.32:0/3731483018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4101a24a0 con 0x7fb41010b2a0 2024-08-31T07:15:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.879+0000 7fb40cc00640 1 -- 172.21.15.32:0/3731483018 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8010430 con 0x7fb41010b2a0 2024-08-31T07:15:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.879+0000 7fb414e00640 1 -- 172.21.15.32:0/3731483018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101a8c00 con 0x7fb41010b2a0 2024-08-31T07:15:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.882+0000 7fb41d07f740 1 Processor -- start 2024-08-31T07:15:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb41d07f740 1 -- start start 2024-08-31T07:15:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b011cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b011d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b011d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb41d07f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5627b00937b0 con 0x5627afd4e1b0 2024-08-31T07:15:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb41d07f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5627b0082d40 con 0x5627afd56390 2024-08-31T07:15:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb41d07f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5627b0086cf0 con 0x5627afd77070 2024-08-31T07:15:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb40cc00640 1 -- 172.21.15.32:0/3731483018 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb3f8005020 con 0x7fb41010b2a0 2024-08-31T07:15:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb40f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b011d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb40fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b011cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb40fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b011cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42802/0 (socket says 172.21.15.32:42802) 2024-08-31T07:15:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.883+0000 7fb40fe00640 1 -- 172.21.15.32:0/1621663985 learned_addr learned my addr 172.21.15.32:0/1621663985 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:45.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb407e00640 1 --2- 172.21.15.32:0/1621663985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b011d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb40fe00640 1 -- 172.21.15.32:0/1621663985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 msgr2=0x5627b011d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:45.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb40fe00640 1 --2- 172.21.15.32:0/1621663985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b011d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb40fe00640 1 -- 172.21.15.32:0/1621663985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 msgr2=0x5627b011d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:45.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb40fe00640 1 --2- 172.21.15.32:0/1621663985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b011d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb40fe00640 1 -- 172.21.15.32:0/1621663985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627afde2650 con 0x5627afd77070 2024-08-31T07:15:45.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb40f400640 1 --2- 172.21.15.32:0/1621663985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b011d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:45.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb407e00640 1 --2- 172.21.15.32:0/1621663985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b011d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb40fe00640 1 --2- 172.21.15.32:0/1621663985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b011cb60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb40802f410 tx=0x7fb40802f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50b150f30faf189c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb404c00640 1 -- 172.21.15.32:0/1621663985 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408031cf0 con 0x5627afd77070 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.884+0000 7fb404c00640 1 -- 172.21.15.32:0/1621663985 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb408036030 con 0x5627afd77070 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.885+0000 7fb404c00640 1 -- 172.21.15.32:0/1621663985 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408004240 con 0x5627afd77070 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.885+0000 7fb40cc00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3f40b4ca0 0x7fb3f40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb40cc00640 1 -- 172.21.15.32:0/3731483018 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb3f80e5850 con 0x7fb41010b2a0 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb41d07f740 1 -- 172.21.15.32:0/1621663985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 msgr2=0x5627b011cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb41d07f740 1 --2- 172.21.15.32:0/1621663985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b011cb60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb40802f410 tx=0x7fb40802f440 comp rx=0 tx=0).stop 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb40f400640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3f40b4ca0 0x7fb3f40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb41d07f740 1 -- 172.21.15.32:0/1621663985 shutdown_connections 2024-08-31T07:15:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb41d07f740 1 --2- 172.21.15.32:0/1621663985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b011d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb41d07f740 1 --2- 172.21.15.32:0/1621663985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b011d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb41d07f740 1 --2- 172.21.15.32:0/1621663985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b011cb60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb41d07f740 1 -- 172.21.15.32:0/1621663985 >> 172.21.15.32:0/1621663985 conn(0x5627afd6b460 msgr2=0x5627b011aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb41d07f740 1 -- 172.21.15.32:0/1621663985 shutdown_connections 2024-08-31T07:15:45.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.886+0000 7fb41d07f740 1 -- 172.21.15.32:0/1621663985 wait complete. 2024-08-31T07:15:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.887+0000 7fb40f400640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3f40b4ca0 0x7fb3f40b71b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb400009d60 tx=0x7fb400005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.887+0000 7fb41d07f740 1 -- start start 2024-08-31T07:15:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.887+0000 7fb41d07f740 1 Processor -- start 2024-08-31T07:15:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.887+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd4e1b0 0x5627b0128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.887+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd56390 0x5627b0128c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd77070 0x5627b0134450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb41d07f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5627afee5700 con 0x5627afd56390 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb41d07f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5627b0009cf0 con 0x5627afd77070 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb41d07f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5627b012b160 con 0x5627afd4e1b0 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb41d07f740 10 client.0 _ll_register_callbacks cb 0x5627b002f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb407e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd56390 0x5627b0128c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb40f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd4e1b0 0x5627b0128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb407e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd56390 0x5627b0128c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45628/0 (socket says 172.21.15.32:45628) 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb407e00640 1 -- 172.21.15.32:0/2887022478 learned_addr learned my addr 172.21.15.32:0/2887022478 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb40fe00640 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd77070 0x5627b0134450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb40f400640 1 -- 172.21.15.32:0/2887022478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd77070 msgr2=0x5627b0134450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb40f400640 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd77070 0x5627b0134450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb40f400640 1 -- 172.21.15.32:0/2887022478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd56390 msgr2=0x5627b0128c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb40f400640 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd56390 0x5627b0128c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb40f400640 1 -- 172.21.15.32:0/2887022478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b004a8b0 con 0x5627afd4e1b0 2024-08-31T07:15:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.888+0000 7fb407e00640 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd56390 0x5627b0128c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.889+0000 7fb40f400640 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd4e1b0 0x5627b0128700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb40000fb60 tx=0x7fb400004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8158fd0ced38db55 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.889+0000 7fb3fd600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:15:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.889+0000 7fb3fd600640 1 -- 172.21.15.32:0/2887022478 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40000fdd0 con 0x5627afd4e1b0 2024-08-31T07:15:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.889+0000 7fb3fd600640 1 -- 172.21.15.32:0/2887022478 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb400018030 con 0x5627afd4e1b0 2024-08-31T07:15:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.889+0000 7fb41d07f740 1 -- 172.21.15.32:0/2887022478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb408036030 con 0x5627afd4e1b0 2024-08-31T07:15:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.889+0000 7fb3fd600640 1 -- 172.21.15.32:0/2887022478 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb400004020 con 0x5627afd4e1b0 2024-08-31T07:15:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.889+0000 7fb41d07f740 10 client.26191 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.889+0000 7fb41d07f740 1 -- 172.21.15.32:0/2887022478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5627b0137140 con 0x5627afd4e1b0 2024-08-31T07:15:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.890+0000 7fb3fd600640 1 -- 172.21.15.32:0/2887022478 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb408036030 con 0x5627afd4e1b0 2024-08-31T07:15:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.890+0000 7fb3fd600640 1 -- 172.21.15.32:0/2887022478 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb4000050b0 con 0x5627afd4e1b0 2024-08-31T07:15:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.890+0000 7fb41d07f740 10 client.26191 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.890+0000 7fb41d07f740 10 client.26191 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:15:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.890+0000 7fb41d07f740 20 client.26191 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.890+0000 7fb41d07f740 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5627afd359f0 0x5627b013a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.890+0000 7fb41d07f740 4 client.26191 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.890+0000 7fb41d07f740 1 -- 172.21.15.32:0/2887022478 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627b0095700 con 0x5627afd359f0 2024-08-31T07:15:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.890+0000 7fb407e00640 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5627afd359f0 0x5627b013a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.891+0000 7fb407e00640 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5627afd359f0 0x5627b013a6f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30345655374f2850 server_cookie=97689e904d642e1f in_seq=0 out_seq=0 2024-08-31T07:15:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.891+0000 7fb3fd600640 10 client.26191 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:15:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb3fd600640 1 -- 172.21.15.32:0/2887022478 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627b0095700 con 0x5627afd359f0 2024-08-31T07:15:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb3fd600640 10 client.26191 handle_command_reply: tid=0 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 client.26191 shutdown 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 -- 172.21.15.32:0/2887022478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd4e1b0 msgr2=0x5627b0128700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd4e1b0 0x5627b0128700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb40000fb60 tx=0x7fb400004730 comp rx=0 tx=0).stop 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 -- 172.21.15.32:0/2887022478 shutdown_connections 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5627afd359f0 0x5627b013a6f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd77070 0x5627b0134450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd56390 0x5627b0128c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 --2- 172.21.15.32:0/2887022478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd4e1b0 0x5627b0128700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb3fd600640 0 client.26191 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 -- 172.21.15.32:0/2887022478 >> 172.21.15.32:0/2887022478 conn(0x5627afd6b460 msgr2=0x5627b011fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 -- 172.21.15.32:0/2887022478 shutdown_connections 2024-08-31T07:15:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 1 -- 172.21.15.32:0/2887022478 wait complete. 2024-08-31T07:15:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.892+0000 7fb41d07f740 20 client.26191 trim_cache size 0 max 16384 2024-08-31T07:15:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb41d07f740 1 Processor -- start 2024-08-31T07:15:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb41d07f740 1 -- start start 2024-08-31T07:15:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0120730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b011e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b012a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb41d07f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b12c0 con 0x5627afd4e1b0 2024-08-31T07:15:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb41d07f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b2ec0 con 0x5627afd56390 2024-08-31T07:15:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb41d07f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b1990 con 0x5627afd77070 2024-08-31T07:15:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb40f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0120730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb40f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0120730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45638/0 (socket says 172.21.15.32:45638) 2024-08-31T07:15:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb40f400640 1 -- 172.21.15.32:0/728884722 learned_addr learned my addr 172.21.15.32:0/728884722 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.907+0000 7fb40fe00640 1 --2- 172.21.15.32:0/728884722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b012a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.908+0000 7fb407e00640 1 --2- 172.21.15.32:0/728884722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b011e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.908+0000 7fb40f400640 1 -- 172.21.15.32:0/728884722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 msgr2=0x5627b012a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.908+0000 7fb40f400640 1 --2- 172.21.15.32:0/728884722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b012a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.908+0000 7fb40f400640 1 -- 172.21.15.32:0/728884722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 msgr2=0x5627b011e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.908+0000 7fb40f400640 1 --2- 172.21.15.32:0/728884722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b011e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.908+0000 7fb40f400640 1 -- 172.21.15.32:0/728884722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b01297c0 con 0x5627afd4e1b0 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.908+0000 7fb40fe00640 1 --2- 172.21.15.32:0/728884722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b012a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.908+0000 7fb40f400640 1 --2- 172.21.15.32:0/728884722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0120730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb40000f9f0 tx=0x7fb400012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb7b91c2ed579ed server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.908+0000 7fb3ffe00640 1 -- 172.21.15.32:0/728884722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400036070 con 0x5627afd4e1b0 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.909+0000 7fb3ffe00640 1 -- 172.21.15.32:0/728884722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4000047a0 con 0x5627afd4e1b0 2024-08-31T07:15:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.909+0000 7fb3ffe00640 1 -- 172.21.15.32:0/728884722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400003850 con 0x5627afd4e1b0 2024-08-31T07:15:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.909+0000 7fb41d07f740 1 -- 172.21.15.32:0/728884722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 msgr2=0x5627b0120730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.909+0000 7fb41d07f740 1 --2- 172.21.15.32:0/728884722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0120730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb40000f9f0 tx=0x7fb400012f70 comp rx=0 tx=0).stop 2024-08-31T07:15:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.909+0000 7fb41d07f740 1 -- 172.21.15.32:0/728884722 shutdown_connections 2024-08-31T07:15:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.909+0000 7fb41d07f740 1 --2- 172.21.15.32:0/728884722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b012a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.909+0000 7fb41d07f740 1 --2- 172.21.15.32:0/728884722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b011e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.909+0000 7fb41d07f740 1 --2- 172.21.15.32:0/728884722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0120730 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.909+0000 7fb41d07f740 1 -- 172.21.15.32:0/728884722 >> 172.21.15.32:0/728884722 conn(0x5627afd6b460 msgr2=0x5627b0129a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.910+0000 7fb41d07f740 1 -- 172.21.15.32:0/728884722 shutdown_connections 2024-08-31T07:15:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.910+0000 7fb41d07f740 1 -- 172.21.15.32:0/728884722 wait complete. 2024-08-31T07:15:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.910+0000 7fb41d07f740 1 -- start start 2024-08-31T07:15:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.910+0000 7fb41d07f740 1 Processor -- start 2024-08-31T07:15:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0130c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b012c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb41d07f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b012c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb41d07f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b2730 con 0x5627afd4e1b0 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb41d07f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b2060 con 0x5627afd56390 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb41d07f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101af2a0 con 0x5627afd77070 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb41d07f740 10 client.0 _ll_register_callbacks cb 0x7fb40000fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb40f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0130c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb40fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b012c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb40f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0130c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45640/0 (socket says 172.21.15.32:45640) 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb40f400640 1 -- 172.21.15.32:0/879178359 learned_addr learned my addr 172.21.15.32:0/879178359 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.911+0000 7fb407e00640 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b012c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb40fe00640 1 -- 172.21.15.32:0/879178359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 msgr2=0x5627b012c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb40fe00640 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b012c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb40fe00640 1 -- 172.21.15.32:0/879178359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 msgr2=0x5627b0130c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb40fe00640 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0130c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb40fe00640 1 -- 172.21.15.32:0/879178359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b012cf00 con 0x5627afd77070 2024-08-31T07:15:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb407e00640 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b012c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb40f400640 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0130c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb40fe00640 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b012c770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb40802f500 tx=0x7fb408009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=178146277244d7ae server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb3fd600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb3fd600640 1 -- 172.21.15.32:0/879178359 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40804a040 con 0x5627afd77070 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb41d07f740 1 -- 172.21.15.32:0/879178359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4000047a0 con 0x5627afd77070 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.912+0000 7fb3fd600640 1 -- 172.21.15.32:0/879178359 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb408031450 con 0x5627afd77070 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb3fd600640 1 -- 172.21.15.32:0/879178359 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb408031650 con 0x5627afd77070 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb41d07f740 10 client.26197 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb41d07f740 1 -- 172.21.15.32:0/879178359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5627b01a5140 con 0x5627afd77070 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb3fd600640 1 -- 172.21.15.32:0/879178359 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4000047a0 con 0x5627afd77070 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb3fd600640 1 -- 172.21.15.32:0/879178359 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb408043070 con 0x5627afd77070 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb41d07f740 10 client.26197 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb41d07f740 10 client.26197 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb41d07f740 20 client.26197 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb41d07f740 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5627afd0ada0 0x5627b01a7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb41d07f740 4 client.26197 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb41d07f740 1 -- 172.21.15.32:0/879178359 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627b00911a0 con 0x5627afd0ada0 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.913+0000 7fb40f400640 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5627afd0ada0 0x5627b01a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.914+0000 7fb40f400640 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5627afd0ada0 0x5627b01a7630 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84cb33f55e55b793 server_cookie=b06d966693a6af4a in_seq=0 out_seq=0 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.914+0000 7fb3fd600640 10 client.26197 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.914+0000 7fb3fd600640 1 -- 172.21.15.32:0/879178359 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627b00911a0 con 0x5627afd0ada0 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.914+0000 7fb3fd600640 10 client.26197 handle_command_reply: tid=0 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.914+0000 7fb41d07f740 1 client.26197 shutdown 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 1 -- 172.21.15.32:0/879178359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 msgr2=0x5627b012c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b012c770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb40802f500 tx=0x7fb408009450 comp rx=0 tx=0).stop 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 1 -- 172.21.15.32:0/879178359 shutdown_connections 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627afd77070 0x5627b012c770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5627afd0ada0 0x5627b01a7630 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627afd56390 0x5627b012c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 1 --2- 172.21.15.32:0/879178359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627afd4e1b0 0x5627b0130c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 1 -- 172.21.15.32:0/879178359 >> 172.21.15.32:0/879178359 conn(0x5627afd6b460 msgr2=0x5627b012a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb3fd600640 0 client.26197 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:15:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 1 -- 172.21.15.32:0/879178359 shutdown_connections 2024-08-31T07:15:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 1 -- 172.21.15.32:0/879178359 wait complete. 2024-08-31T07:15:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.915+0000 7fb41d07f740 20 client.26197 trim_cache size 0 max 16384 2024-08-31T07:15:45.920 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:15:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 -- 172.21.15.32:0/3731483018 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3f40b4ca0 msgr2=0x7fb3f40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3f40b4ca0 0x7fb3f40b71b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb400009d60 tx=0x7fb400005f90 comp rx=0 tx=0).stop 2024-08-31T07:15:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 -- 172.21.15.32:0/3731483018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb41010b2a0 msgr2=0x7fb41019c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb41010b2a0 0x7fb41019c7e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800e9c0 tx=0x7fb3f800ee80 comp rx=0 tx=0).stop 2024-08-31T07:15:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 -- 172.21.15.32:0/3731483018 shutdown_connections 2024-08-31T07:15:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3f40b4ca0 0x7fb3f40b71b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb41010bbf0 0x7fb4101a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb41010b2a0 0x7fb41019c7e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 --2- 172.21.15.32:0/3731483018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb410105960 0x7fb41019c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 -- 172.21.15.32:0/3731483018 >> 172.21.15.32:0/3731483018 conn(0x7fb410101410 msgr2=0x7fb41010a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 -- 172.21.15.32:0/3731483018 shutdown_connections 2024-08-31T07:15:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:45.916+0000 7fb414e00640 1 -- 172.21.15.32:0/3731483018 wait complete. 2024-08-31T07:15:46.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:46 smithi032 ceph-mon[17252]: pgmap v458: 97 pgs: 97 active+clean; 61 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 791 KiB/s wr, 122 op/s 2024-08-31T07:15:46.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:46 smithi073 ceph-mon[21995]: pgmap v458: 97 pgs: 97 active+clean; 61 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 791 KiB/s wr, 122 op/s 2024-08-31T07:15:46.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 100152 5.86 MB/sec warmup 95 sec latency 1527.690 ms 2024-08-31T07:15:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:46 smithi084 ceph-mon[22387]: pgmap v458: 97 pgs: 97 active+clean; 61 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 791 KiB/s wr, 122 op/s 2024-08-31T07:15:46.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:15:47.207 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:47.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 100152 5.80 MB/sec warmup 96 sec latency 2527.748 ms 2024-08-31T07:15:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.922+0000 7ffb26c00640 1 -- 172.21.15.32:0/3919205632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb20109bc0 msgr2=0x7ffb201103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.922+0000 7ffb26c00640 1 --2- 172.21.15.32:0/3919205632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb20109bc0 0x7ffb201103f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009ca0 tx=0x7ffb1802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.922+0000 7ffb26c00640 1 -- 172.21.15.32:0/3919205632 shutdown_connections 2024-08-31T07:15:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.922+0000 7ffb26c00640 1 --2- 172.21.15.32:0/3919205632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb20109bc0 0x7ffb201103f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.922+0000 7ffb26c00640 1 --2- 172.21.15.32:0/3919205632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb20109270 0x7ffb20109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.922+0000 7ffb26c00640 1 --2- 172.21.15.32:0/3919205632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20103930 0x7ffb20103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.922+0000 7ffb26c00640 1 -- 172.21.15.32:0/3919205632 >> 172.21.15.32:0/3919205632 conn(0x7ffb200ff380 msgr2=0x7ffb201017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.922+0000 7ffb26c00640 1 -- 172.21.15.32:0/3919205632 shutdown_connections 2024-08-31T07:15:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.922+0000 7ffb26c00640 1 -- 172.21.15.32:0/3919205632 wait complete. 2024-08-31T07:15:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.924+0000 7ffb26c00640 1 Processor -- start 2024-08-31T07:15:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.924+0000 7ffb26c00640 1 -- start start 2024-08-31T07:15:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb26c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb20103930 0x7ffb200754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb26c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb20109270 0x7ffb200759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb26c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20109bc0 0x7ffb20075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb26c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb20113880 con 0x7ffb20103930 2024-08-31T07:15:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb26c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffb20113700 con 0x7ffb20109270 2024-08-31T07:15:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb26c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffb20113a00 con 0x7ffb20109bc0 2024-08-31T07:15:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb1f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb20109270 0x7ffb200759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb20103930 0x7ffb200754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb1f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb20109270 0x7ffb200759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54628/0 (socket says 172.21.15.32:54628) 2024-08-31T07:15:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb1f400640 1 -- 172.21.15.32:0/2156696961 learned_addr learned my addr 172.21.15.32:0/2156696961 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb24c00640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20109bc0 0x7ffb20075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb1f400640 1 -- 172.21.15.32:0/2156696961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20109bc0 msgr2=0x7ffb20075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb1f400640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20109bc0 0x7ffb20075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb1f400640 1 -- 172.21.15.32:0/2156696961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb20103930 msgr2=0x7ffb200754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb1f400640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb20103930 0x7ffb200754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.925+0000 7ffb1f400640 1 -- 172.21.15.32:0/2156696961 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb201a8470 con 0x7ffb20109270 2024-08-31T07:15:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.926+0000 7ffb1f400640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb20109270 0x7ffb200759f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00e970 tx=0x7ffb0c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.926+0000 7ffb1cc00640 1 -- 172.21.15.32:0/2156696961 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00cd80 con 0x7ffb20109270 2024-08-31T07:15:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.926+0000 7ffb1cc00640 1 -- 172.21.15.32:0/2156696961 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0c0035f0 con 0x7ffb20109270 2024-08-31T07:15:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.926+0000 7ffb26c00640 1 -- 172.21.15.32:0/2156696961 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb201a8760 con 0x7ffb20109270 2024-08-31T07:15:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.926+0000 7ffb1cc00640 1 -- 172.21.15.32:0/2156696961 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c010690 con 0x7ffb20109270 2024-08-31T07:15:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.926+0000 7ffb26c00640 1 -- 172.21.15.32:0/2156696961 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb201a8ca0 con 0x7ffb20109270 2024-08-31T07:15:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.929+0000 7ffb2d9d7740 1 Processor -- start 2024-08-31T07:15:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb2d9d7740 1 -- start start 2024-08-31T07:15:47.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f0305390 0x55b4f06cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b4f06427b0 con 0x55b4f02fd1b0 2024-08-31T07:15:47.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4f0631d40 con 0x55b4f0305390 2024-08-31T07:15:47.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b4f0635cf0 con 0x55b4f0326070 2024-08-31T07:15:47.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb1cc00640 1 -- 172.21.15.32:0/2156696961 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ffb0c003aa0 con 0x7ffb20109270 2024-08-31T07:15:47.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45672/0 (socket says 172.21.15.32:45672) 2024-08-31T07:15:47.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb1fe00640 1 -- 172.21.15.32:0/813412310 learned_addr learned my addr 172.21.15.32:0/813412310 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.930+0000 7ffb24c00640 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.931+0000 7ffb1fe00640 1 -- 172.21.15.32:0/813412310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 msgr2=0x55b4f06cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.931+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.931+0000 7ffb1f400640 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f0305390 0x55b4f06cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.931+0000 7ffb1fe00640 1 -- 172.21.15.32:0/813412310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f0305390 msgr2=0x55b4f06cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.931+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f0305390 0x55b4f06cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.931+0000 7ffb1fe00640 1 -- 172.21.15.32:0/813412310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f0391650 con 0x55b4f02fd1b0 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.931+0000 7ffb24c00640 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.931+0000 7ffb1f400640 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f0305390 0x55b4f06cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.931+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cbdf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009c90 tx=0x7ffb10011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600facd3c9f9da6a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.932+0000 7ffb15200640 1 -- 172.21.15.32:0/813412310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10004230 con 0x55b4f02fd1b0 2024-08-31T07:15:47.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.932+0000 7ffb15200640 1 -- 172.21.15.32:0/813412310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb100043d0 con 0x55b4f02fd1b0 2024-08-31T07:15:47.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.932+0000 7ffb15200640 1 -- 172.21.15.32:0/813412310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10019620 con 0x55b4f02fd1b0 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.932+0000 7ffb1cc00640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb040b4ca0 0x7ffb040b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.933+0000 7ffb1cc00640 1 -- 172.21.15.32:0/2156696961 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ffb0c0e57e0 con 0x7ffb20109270 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.933+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/813412310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f02fd1b0 msgr2=0x55b4f06cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.933+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cbdf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009c90 tx=0x7ffb10011810 comp rx=0 tx=0).stop 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.933+0000 7ffb24c00640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb040b4ca0 0x7ffb040b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.933+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/813412310 shutdown_connections 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.933+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f0305390 0x55b4f06cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.933+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cbdf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.933+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/813412310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.933+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/813412310 >> 172.21.15.32:0/813412310 conn(0x55b4f031a460 msgr2=0x55b4f06c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.933+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/813412310 shutdown_connections 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.934+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/813412310 wait complete. 2024-08-31T07:15:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.934+0000 7ffb24c00640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb040b4ca0 0x7ffb040b71b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffb1802f500 tx=0x7ffb1802f490 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:47.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.934+0000 7ffb2d9d7740 1 -- start start 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.934+0000 7ffb2d9d7740 1 Processor -- start 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0305390 0x55b4f06d7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b4f0494700 con 0x55b4f0305390 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4f05b8cf0 con 0x55b4f02fd1b0 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b4f06da090 con 0x55b4f0326070 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb2d9d7740 10 client.0 _ll_register_callbacks cb 0x55b4f05de470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0305390 0x55b4f06d7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb24c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb1fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0305390 0x55b4f06d7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45678/0 (socket says 172.21.15.32:45678) 2024-08-31T07:15:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb24c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06e3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42862/0 (socket says 172.21.15.32:42862) 2024-08-31T07:15:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb1fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54644/0 (socket says 172.21.15.32:54644) 2024-08-31T07:15:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.935+0000 7ffb1f400640 1 -- 172.21.15.32:0/1663244015 learned_addr learned my addr 172.21.15.32:0/1663244015 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.936+0000 7ffb24c00640 1 -- 172.21.15.32:0/1663244015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 msgr2=0x55b4f06d74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.936+0000 7ffb24c00640 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.936+0000 7ffb24c00640 1 -- 172.21.15.32:0/1663244015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0305390 msgr2=0x55b4f06d7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.936+0000 7ffb24c00640 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0305390 0x55b4f06d7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.936+0000 7ffb24c00640 1 -- 172.21.15.32:0/1663244015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f05f98b0 con 0x55b4f0326070 2024-08-31T07:15:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.936+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.936+0000 7ffb1f400640 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0305390 0x55b4f06d7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:47.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.936+0000 7ffb24c00640 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06e3380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffb18006010 tx=0x7ffb1802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc9fb480dfad6c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:47.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.936+0000 7ffb09600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:15:47.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.936+0000 7ffb09600640 1 -- 172.21.15.32:0/1663244015 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18003c40 con 0x55b4f0326070 2024-08-31T07:15:47.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.937+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1663244015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb100043d0 con 0x55b4f0326070 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.937+0000 7ffb09600640 1 -- 172.21.15.32:0/1663244015 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ffb18003de0 con 0x55b4f0326070 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.937+0000 7ffb09600640 1 -- 172.21.15.32:0/1663244015 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffb18004040 con 0x55b4f0326070 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.937+0000 7ffb2d9d7740 10 client.26209 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.937+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1663244015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b4f06e60a0 con 0x55b4f0326070 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.937+0000 7ffb09600640 1 -- 172.21.15.32:0/1663244015 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb100043d0 con 0x55b4f0326070 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.937+0000 7ffb09600640 1 -- 172.21.15.32:0/1663244015 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ffb18051070 con 0x55b4f0326070 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.937+0000 7ffb2d9d7740 10 client.26209 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.937+0000 7ffb2d9d7740 10 client.26209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.938+0000 7ffb2d9d7740 20 client.26209 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.938+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b4f02e49f0 0x55b4f06e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.938+0000 7ffb2d9d7740 4 client.26209 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.938+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1663244015 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4f0644700 con 0x55b4f02e49f0 2024-08-31T07:15:47.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.938+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b4f02e49f0 0x55b4f06e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.939+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b4f02e49f0 0x55b4f06e9670 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af79d5e691af8950 server_cookie=776091f96c9ea151 in_seq=0 out_seq=0 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.939+0000 7ffb09600640 10 client.26209 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb09600640 1 -- 172.21.15.32:0/1663244015 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b4f0644700 con 0x55b4f02e49f0 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb09600640 10 client.26209 handle_command_reply: tid=0 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb2d9d7740 1 client.26209 shutdown 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1663244015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 msgr2=0x55b4f06e3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06e3380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffb18006010 tx=0x7ffb1802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1663244015 shutdown_connections 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b4f02e49f0 0x55b4f06e9670 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0326070 0x55b4f06e3380 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0305390 0x55b4f06d7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1663244015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1663244015 >> 172.21.15.32:0/1663244015 conn(0x55b4f031a460 msgr2=0x55b4f06cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.940+0000 7ffb09600640 0 client.26209 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.941+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1663244015 shutdown_connections 2024-08-31T07:15:47.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.941+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1663244015 wait complete. 2024-08-31T07:15:47.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.941+0000 7ffb2d9d7740 20 client.26209 trim_cache size 0 max 16384 2024-08-31T07:15:47.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.955+0000 7ffb2d9d7740 1 Processor -- start 2024-08-31T07:15:47.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.955+0000 7ffb2d9d7740 1 -- start start 2024-08-31T07:15:47.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.955+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.955+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 0x55b4f06d60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.955+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 0x55b4f06d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.955+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb201b14c0 con 0x55b4f0326070 2024-08-31T07:15:47.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffb201b30c0 con 0x55b4f02fd1b0 2024-08-31T07:15:47.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffb201b1b90 con 0x55b4f0305390 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb24c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 0x55b4f06d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb1fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb24c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 0x55b4f06d09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45682/0 (socket says 172.21.15.32:45682) 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb1fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54658/0 (socket says 172.21.15.32:54658) 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb24c00640 1 -- 172.21.15.32:0/3384906065 learned_addr learned my addr 172.21.15.32:0/3384906065 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb1f400640 1 --2- 172.21.15.32:0/3384906065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 0x55b4f06d60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb1fe00640 1 -- 172.21.15.32:0/3384906065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 msgr2=0x55b4f06d60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/3384906065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 0x55b4f06d60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb1fe00640 1 -- 172.21.15.32:0/3384906065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 msgr2=0x55b4f06d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/3384906065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 0x55b4f06d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.956+0000 7ffb1fe00640 1 -- 172.21.15.32:0/3384906065 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f06d8390 con 0x55b4f02fd1b0 2024-08-31T07:15:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.957+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/3384906065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d5b70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffb100093f0 tx=0x7ffb10005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6743dca01385586b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.957+0000 7ffb0be00640 1 -- 172.21.15.32:0/3384906065 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1001f040 con 0x55b4f02fd1b0 2024-08-31T07:15:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.957+0000 7ffb0be00640 1 -- 172.21.15.32:0/3384906065 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb1002b070 con 0x55b4f02fd1b0 2024-08-31T07:15:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.957+0000 7ffb0be00640 1 -- 172.21.15.32:0/3384906065 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1001a6a0 con 0x55b4f02fd1b0 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.958+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/3384906065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 msgr2=0x55b4f06d5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.958+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/3384906065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d5b70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffb100093f0 tx=0x7ffb10005f10 comp rx=0 tx=0).stop 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.958+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/3384906065 shutdown_connections 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.958+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/3384906065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 0x55b4f06d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.958+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/3384906065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 0x55b4f06d60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.958+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/3384906065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06d5b70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.958+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/3384906065 >> 172.21.15.32:0/3384906065 conn(0x55b4f031a460 msgr2=0x55b4f06d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.958+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/3384906065 shutdown_connections 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.958+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/3384906065 wait complete. 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.959+0000 7ffb2d9d7740 1 -- start start 2024-08-31T07:15:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.959+0000 7ffb2d9d7740 1 Processor -- start 2024-08-31T07:15:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 0x55b4f06cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb2d9d7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 0x55b4f06de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb201b2930 con 0x55b4f0326070 2024-08-31T07:15:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffb201b2260 con 0x55b4f02fd1b0 2024-08-31T07:15:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb2d9d7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffb201af4a0 con 0x55b4f0305390 2024-08-31T07:15:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb2d9d7740 10 client.0 _ll_register_callbacks cb 0x7ffb18031f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb1fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb1fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54664/0 (socket says 172.21.15.32:54664) 2024-08-31T07:15:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb1fe00640 1 -- 172.21.15.32:0/1143725186 learned_addr learned my addr 172.21.15.32:0/1143725186 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb1f400640 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 0x55b4f06cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.960+0000 7ffb24c00640 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 0x55b4f06de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb1f400640 1 -- 172.21.15.32:0/1143725186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 msgr2=0x55b4f06cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb1f400640 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb1f400640 1 -- 172.21.15.32:0/1143725186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 msgr2=0x55b4f06de2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb1f400640 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 0x55b4f06de2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb1f400640 1 -- 172.21.15.32:0/1143725186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f06de820 con 0x55b4f0305390 2024-08-31T07:15:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb24c00640 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 0x55b4f06de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb1f400640 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 0x55b4f06cb360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0098c0 tx=0x7ffb0c0ac490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff419bb45bc1dc8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb09600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb09600640 1 -- 172.21.15.32:0/1143725186 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0b0760 con 0x55b4f0305390 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb09600640 1 -- 172.21.15.32:0/1143725186 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ffb0c0e5de0 con 0x55b4f0305390 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1143725186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb1002b070 con 0x55b4f0305390 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb09600640 1 -- 172.21.15.32:0/1143725186 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffb0c0b5210 con 0x55b4f0305390 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb2d9d7740 10 client.26215 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.961+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1143725186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b4f06dfb00 con 0x55b4f0305390 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.962+0000 7ffb09600640 1 -- 172.21.15.32:0/1143725186 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb1002b070 con 0x55b4f0305390 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.962+0000 7ffb09600640 1 -- 172.21.15.32:0/1143725186 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ffb0c091ce0 con 0x55b4f0305390 2024-08-31T07:15:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.962+0000 7ffb2d9d7740 10 client.26215 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.962+0000 7ffb2d9d7740 10 client.26215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.962+0000 7ffb2d9d7740 20 client.26215 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.962+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b4f02b9da0 0x55b4f07561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.962+0000 7ffb2d9d7740 4 client.26215 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:15:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.962+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1143725186 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4f06401a0 con 0x55b4f02b9da0 2024-08-31T07:15:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.962+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b4f02b9da0 0x55b4f07561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb1fe00640 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b4f02b9da0 0x55b4f07561b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28d7f6a451dbcc0 server_cookie=8848c95a2415995a in_seq=0 out_seq=0 2024-08-31T07:15:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb09600640 10 client.26215 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb09600640 1 -- 172.21.15.32:0/1143725186 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4f06401a0 con 0x55b4f02b9da0 2024-08-31T07:15:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb09600640 10 client.26215 handle_command_reply: tid=0 2024-08-31T07:15:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb2d9d7740 1 client.26215 shutdown 2024-08-31T07:15:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1143725186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 msgr2=0x55b4f06cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 0x55b4f06cb360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0098c0 tx=0x7ffb0c0ac490 comp rx=0 tx=0).stop 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1143725186 shutdown_connections 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b4f02b9da0 0x55b4f07561b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b4f0326070 0x55b4f06de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b4f0305390 0x55b4f06cb360 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb2d9d7740 1 --2- 172.21.15.32:0/1143725186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4f02fd1b0 0x55b4f06cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1143725186 >> 172.21.15.32:0/1143725186 conn(0x55b4f031a460 msgr2=0x55b4f06d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.963+0000 7ffb09600640 0 client.26215 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.964+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1143725186 shutdown_connections 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.964+0000 7ffb2d9d7740 1 -- 172.21.15.32:0/1143725186 wait complete. 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.964+0000 7ffb2d9d7740 20 client.26215 trim_cache size 0 max 16384 2024-08-31T07:15:47.967 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:15:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 -- 172.21.15.32:0/2156696961 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb040b4ca0 msgr2=0x7ffb040b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb040b4ca0 0x7ffb040b71b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffb1802f500 tx=0x7ffb1802f490 comp rx=0 tx=0).stop 2024-08-31T07:15:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 -- 172.21.15.32:0/2156696961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb20109270 msgr2=0x7ffb200759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb20109270 0x7ffb200759f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00e970 tx=0x7ffb0c00ee30 comp rx=0 tx=0).stop 2024-08-31T07:15:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 -- 172.21.15.32:0/2156696961 shutdown_connections 2024-08-31T07:15:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb040b4ca0 0x7ffb040b71b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20109bc0 0x7ffb20075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb20109270 0x7ffb200759f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 --2- 172.21.15.32:0/2156696961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb20103930 0x7ffb200754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 -- 172.21.15.32:0/2156696961 >> 172.21.15.32:0/2156696961 conn(0x7ffb200ff380 msgr2=0x7ffb201017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 -- 172.21.15.32:0/2156696961 shutdown_connections 2024-08-31T07:15:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:47.965+0000 7ffb26c00640 1 -- 172.21.15.32:0/2156696961 wait complete. 2024-08-31T07:15:48.496 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:48 smithi032 ceph-mon[17252]: pgmap v459: 97 pgs: 97 active+clean; 62 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 561 KiB/s wr, 121 op/s 2024-08-31T07:15:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:48 smithi073 ceph-mon[21995]: pgmap v459: 97 pgs: 97 active+clean; 62 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 561 KiB/s wr, 121 op/s 2024-08-31T07:15:48.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 100152 5.74 MB/sec warmup 97 sec latency 3528.043 ms 2024-08-31T07:15:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:48 smithi084 ceph-mon[22387]: pgmap v459: 97 pgs: 97 active+clean; 62 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 561 KiB/s wr, 121 op/s 2024-08-31T07:15:49.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 100152 5.68 MB/sec warmup 98 sec latency 4528.174 ms 2024-08-31T07:15:50.004 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:15:50.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:15:50.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:50 smithi032 ceph-mon[17252]: pgmap v460: 97 pgs: 97 active+clean; 62 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 434 KiB/s wr, 86 op/s 2024-08-31T07:15:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:50.542 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:50 smithi084 ceph-mon[22387]: pgmap v460: 97 pgs: 97 active+clean; 62 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 434 KiB/s wr, 86 op/s 2024-08-31T07:15:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:50 smithi073 ceph-mon[21995]: pgmap v460: 97 pgs: 97 active+clean; 62 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 434 KiB/s wr, 86 op/s 2024-08-31T07:15:50.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 101768 5.69 MB/sec warmup 99 sec latency 4950.305 ms 2024-08-31T07:15:50.812 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-31T07:15:50.812 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.8 2024-08-31T07:15:50.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:15:51.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.157+0000 7f3927200640 1 -- 172.21.15.32:0/4288755144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3928104560 msgr2=0x7f3928104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:51.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.157+0000 7f3927200640 1 --2- 172.21.15.32:0/4288755144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3928104560 0x7f3928104970 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3918009900 tx=0x7f391802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.158+0000 7f3927200640 1 -- 172.21.15.32:0/4288755144 shutdown_connections 2024-08-31T07:15:51.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.158+0000 7f3927200640 1 --2- 172.21.15.32:0/4288755144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392810d9c0 0x7f392810feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.158+0000 7f3927200640 1 --2- 172.21.15.32:0/4288755144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3928104eb0 0x7f392810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.158+0000 7f3927200640 1 --2- 172.21.15.32:0/4288755144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3928104560 0x7f3928104970 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.158+0000 7f3927200640 1 -- 172.21.15.32:0/4288755144 >> 172.21.15.32:0/4288755144 conn(0x7f39280fe060 msgr2=0x7f39281004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:51.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.158+0000 7f3927200640 1 -- 172.21.15.32:0/4288755144 shutdown_connections 2024-08-31T07:15:51.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.158+0000 7f3927200640 1 -- 172.21.15.32:0/4288755144 wait complete. 2024-08-31T07:15:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.159+0000 7f3927200640 1 Processor -- start 2024-08-31T07:15:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.159+0000 7f3927200640 1 -- start start 2024-08-31T07:15:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.159+0000 7f3927200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3928104560 0x7f392819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.159+0000 7f3927200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3928104eb0 0x7f392819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.159+0000 7f3927200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392810d9c0 0x7f392819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.159+0000 7f3927200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3928113650 con 0x7f392810d9c0 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.159+0000 7f3927200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39281134d0 con 0x7f3928104560 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.159+0000 7f3927200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f39281137d0 con 0x7f3928104eb0 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3925e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3928104560 0x7f392819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3925400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3928104eb0 0x7f392819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3925e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3928104560 0x7f392819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54680/0 (socket says 172.21.15.32:54680) 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3925400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3928104eb0 0x7f392819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42916/0 (socket says 172.21.15.32:42916) 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3925e00640 1 -- 172.21.15.32:0/2808996601 learned_addr learned my addr 172.21.15.32:0/2808996601 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3926800640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392810d9c0 0x7f392819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3925400640 1 -- 172.21.15.32:0/2808996601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3928104560 msgr2=0x7f392819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3925400640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3928104560 0x7f392819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3925400640 1 -- 172.21.15.32:0/2808996601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392810d9c0 msgr2=0x7f392819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3925400640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392810d9c0 0x7f392819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.160+0000 7f3925400640 1 -- 172.21.15.32:0/2808996601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39281a0030 con 0x7f3928104eb0 2024-08-31T07:15:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.161+0000 7f3925400640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3928104eb0 0x7f392819a660 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f392806a830 tx=0x7f391400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.161+0000 7f391ea00640 1 -- 172.21.15.32:0/2808996601 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391400cc40 con 0x7f3928104eb0 2024-08-31T07:15:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.161+0000 7f391ea00640 1 -- 172.21.15.32:0/2808996601 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f391400cde0 con 0x7f3928104eb0 2024-08-31T07:15:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.161+0000 7f3927200640 1 -- 172.21.15.32:0/2808996601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39281a0320 con 0x7f3928104eb0 2024-08-31T07:15:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.161+0000 7f391ea00640 1 -- 172.21.15.32:0/2808996601 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914004a00 con 0x7f3928104eb0 2024-08-31T07:15:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.161+0000 7f3927200640 1 -- 172.21.15.32:0/2808996601 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3928102180 con 0x7f3928104eb0 2024-08-31T07:15:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.165+0000 7f3933e11740 1 Processor -- start 2024-08-31T07:15:51.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.165+0000 7f3933e11740 1 -- start start 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.165+0000 7f3933e11740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 0x55757050d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.165+0000 7f3933e11740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757013e490 0x55757050dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3933e11740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 0x55757050e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3933e11740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557570483750 con 0x557570167350 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3933e11740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557570472ce0 con 0x55757013e490 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3933e11740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557570476c90 con 0x557570146670 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3925e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757013e490 0x55757050dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3925400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 0x55757050e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3925e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757013e490 0x55757050dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54696/0 (socket says 172.21.15.32:54696) 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3925e00640 1 -- 172.21.15.32:0/758099691 learned_addr learned my addr 172.21.15.32:0/758099691 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:51.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3926800640 1 --2- 172.21.15.32:0/758099691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 0x55757050d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f391ea00640 1 -- 172.21.15.32:0/2808996601 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3914004ba0 con 0x7f3928104eb0 2024-08-31T07:15:51.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3926800640 1 -- 172.21.15.32:0/758099691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 msgr2=0x55757050e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:51.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3926800640 1 --2- 172.21.15.32:0/758099691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 0x55757050e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3926800640 1 -- 172.21.15.32:0/758099691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757013e490 msgr2=0x55757050dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:51.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.166+0000 7f3926800640 1 --2- 172.21.15.32:0/758099691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757013e490 0x55757050dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.167+0000 7f3926800640 1 -- 172.21.15.32:0/758099691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575701d2930 con 0x557570167350 2024-08-31T07:15:51.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.167+0000 7f3926800640 1 --2- 172.21.15.32:0/758099691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 0x55757050d7d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3920009f00 tx=0x7f3920011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7aca4a7502ccd0c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:51.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.167+0000 7f3913400640 1 -- 172.21.15.32:0/758099691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920004d20 con 0x557570167350 2024-08-31T07:15:51.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.167+0000 7f3913400640 1 -- 172.21.15.32:0/758099691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3920004030 con 0x557570167350 2024-08-31T07:15:51.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.167+0000 7f3913400640 1 -- 172.21.15.32:0/758099691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920005220 con 0x557570167350 2024-08-31T07:15:51.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.168+0000 7f391ea00640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f390c0b91b0 0x7f390c0bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.168+0000 7f391ea00640 1 -- 172.21.15.32:0/2808996601 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f39140e53f0 con 0x7f3928104eb0 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.168+0000 7f3925e00640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f390c0b91b0 0x7f390c0bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.168+0000 7f3933e11740 1 -- 172.21.15.32:0/758099691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 msgr2=0x55757050d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.168+0000 7f3933e11740 1 --2- 172.21.15.32:0/758099691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 0x55757050d7d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3920009f00 tx=0x7f3920011810 comp rx=0 tx=0).stop 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.169+0000 7f3933e11740 1 -- 172.21.15.32:0/758099691 shutdown_connections 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.169+0000 7f3933e11740 1 --2- 172.21.15.32:0/758099691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 0x55757050e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.169+0000 7f3933e11740 1 --2- 172.21.15.32:0/758099691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757013e490 0x55757050dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.169+0000 7f3933e11740 1 --2- 172.21.15.32:0/758099691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 0x55757050d7d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.169+0000 7f3933e11740 1 -- 172.21.15.32:0/758099691 >> 172.21.15.32:0/758099691 conn(0x55757015b740 msgr2=0x55757050b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.169+0000 7f3933e11740 1 -- 172.21.15.32:0/758099691 shutdown_connections 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.169+0000 7f3933e11740 1 -- 172.21.15.32:0/758099691 wait complete. 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.169+0000 7f3925e00640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f390c0b91b0 0x7f390c0bb6c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39180096c0 tx=0x7f39180058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.169+0000 7f3933e11740 1 -- start start 2024-08-31T07:15:51.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.170+0000 7f3933e11740 1 Processor -- start 2024-08-31T07:15:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.170+0000 7f3933e11740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757013e490 0x5575705193b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.170+0000 7f3933e11740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x5575705198f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.170+0000 7f3933e11740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 0x5575705252d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.170+0000 7f3933e11740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5575702d5890 con 0x557570167350 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.170+0000 7f3933e11740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5575703f9cd0 con 0x557570146670 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.170+0000 7f3933e11740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55757051bfb0 con 0x55757013e490 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.170+0000 7f3933e11740 10 client.0 _ll_register_callbacks cb 0x55757046def0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757013e490 0x5575705193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757013e490 0x5575705193b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42942/0 (socket says 172.21.15.32:42942) 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925e00640 1 -- 172.21.15.32:0/3737433724 learned_addr learned my addr 172.21.15.32:0/3737433724 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3926800640 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 0x5575705252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925400640 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x5575705198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925400640 1 -- 172.21.15.32:0/3737433724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757013e490 msgr2=0x5575705193b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925400640 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757013e490 0x5575705193b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925400640 1 -- 172.21.15.32:0/3737433724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 msgr2=0x5575705252d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925400640 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 0x5575705252d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925400640 1 -- 172.21.15.32:0/3737433724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575704431b0 con 0x557570146670 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925e00640 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757013e490 0x5575705193b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3925400640 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x5575705198f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f39140e2b60 tx=0x7f3914005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aede29203b4603cf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.171+0000 7f3907e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.172+0000 7f3907e00640 1 -- 172.21.15.32:0/3737433724 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39140ab8d0 con 0x557570146670 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.172+0000 7f3907e00640 1 -- 172.21.15.32:0/3737433724 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f39140b4990 con 0x557570146670 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.172+0000 7f3933e11740 1 -- 172.21.15.32:0/3737433724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3920004030 con 0x557570146670 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.172+0000 7f3907e00640 1 -- 172.21.15.32:0/3737433724 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f39140e2ff0 con 0x557570146670 2024-08-31T07:15:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.172+0000 7f3933e11740 10 client.26210 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.172+0000 7f3933e11740 1 -- 172.21.15.32:0/3737433724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557570528020 con 0x557570146670 2024-08-31T07:15:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.172+0000 7f3907e00640 1 -- 172.21.15.32:0/3737433724 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3920004030 con 0x557570146670 2024-08-31T07:15:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.173+0000 7f3907e00640 1 -- 172.21.15.32:0/3737433724 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f39140cebe0 con 0x557570146670 2024-08-31T07:15:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.173+0000 7f3933e11740 10 client.26210 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.173+0000 7f3933e11740 10 client.26210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.173+0000 7f3933e11740 20 client.26210 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.173+0000 7f3933e11740 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557570125cd0 0x55757052b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.173+0000 7f3933e11740 4 client.26210 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:51.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.173+0000 7f3933e11740 1 -- 172.21.15.32:0/3737433724 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575704856a0 con 0x557570125cd0 2024-08-31T07:15:51.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.173+0000 7f3925e00640 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557570125cd0 0x55757052b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.174+0000 7f3925e00640 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557570125cd0 0x55757052b460 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c392d65eac3b6ae server_cookie=15ce29604cc4af6a in_seq=0 out_seq=0 2024-08-31T07:15:51.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.174+0000 7f3907e00640 10 client.26210 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:51.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.174+0000 7f3907e00640 1 -- 172.21.15.32:0/3737433724 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5575704856a0 con 0x557570125cd0 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.174+0000 7f3907e00640 10 client.26210 handle_command_reply: tid=0 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.174+0000 7f3933e11740 1 client.26210 shutdown 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 1 -- 172.21.15.32:0/3737433724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 msgr2=0x5575705198f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x5575705198f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f39140e2b60 tx=0x7f3914005020 comp rx=0 tx=0).stop 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 1 -- 172.21.15.32:0/3737433724 shutdown_connections 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557570125cd0 0x55757052b460 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557570167350 0x5575705252d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x5575705198f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3907e00640 0 client.26210 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 1 --2- 172.21.15.32:0/3737433724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757013e490 0x5575705193b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 1 -- 172.21.15.32:0/3737433724 >> 172.21.15.32:0/3737433724 conn(0x55757015b740 msgr2=0x5575705109b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 1 -- 172.21.15.32:0/3737433724 shutdown_connections 2024-08-31T07:15:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 1 -- 172.21.15.32:0/3737433724 wait complete. 2024-08-31T07:15:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.175+0000 7f3933e11740 20 client.26210 trim_cache size 0 max 16384 2024-08-31T07:15:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.203+0000 7f3933e11740 1 Processor -- start 2024-08-31T07:15:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.203+0000 7f3933e11740 1 -- start start 2024-08-31T07:15:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.203+0000 7f3933e11740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757013e490 0x5575705251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.203+0000 7f3933e11740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 0x55757050af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.203+0000 7f3933e11740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570167350 0x55757050b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.203+0000 7f3933e11740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f39281acfb0 con 0x55757013e490 2024-08-31T07:15:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.203+0000 7f3933e11740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39281aebb0 con 0x557570167350 2024-08-31T07:15:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.203+0000 7f3933e11740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f39281ad680 con 0x557570146670 2024-08-31T07:15:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.203+0000 7f3925400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 0x55757050af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3925400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 0x55757050af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42950/0 (socket says 172.21.15.32:42950) 2024-08-31T07:15:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3925400640 1 -- 172.21.15.32:0/2540863148 learned_addr learned my addr 172.21.15.32:0/2540863148 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3925e00640 1 --2- 172.21.15.32:0/2540863148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757013e490 0x5575705251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3926800640 1 --2- 172.21.15.32:0/2540863148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570167350 0x55757050b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3925400640 1 -- 172.21.15.32:0/2540863148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570167350 msgr2=0x55757050b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3925400640 1 --2- 172.21.15.32:0/2540863148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570167350 0x55757050b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3925400640 1 -- 172.21.15.32:0/2540863148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757013e490 msgr2=0x5575705251d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3925400640 1 --2- 172.21.15.32:0/2540863148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757013e490 0x5575705251d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3925400640 1 -- 172.21.15.32:0/2540863148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557570518110 con 0x557570146670 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3926800640 1 --2- 172.21.15.32:0/2540863148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570167350 0x55757050b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.204+0000 7f3925400640 1 --2- 172.21.15.32:0/2540863148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 0x55757050af70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f39140a7b80 tx=0x7f3914005480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9bac267017a5699 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.205+0000 7f3912a00640 1 -- 172.21.15.32:0/2540863148 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39140a7290 con 0x557570146670 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.205+0000 7f3912a00640 1 -- 172.21.15.32:0/2540863148 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39140ab610 con 0x557570146670 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.205+0000 7f3912a00640 1 -- 172.21.15.32:0/2540863148 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39140ab270 con 0x557570146670 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.206+0000 7f3933e11740 1 -- 172.21.15.32:0/2540863148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 msgr2=0x55757050af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.206+0000 7f3933e11740 1 --2- 172.21.15.32:0/2540863148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 0x55757050af70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f39140a7b80 tx=0x7f3914005480 comp rx=0 tx=0).stop 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.206+0000 7f3933e11740 1 -- 172.21.15.32:0/2540863148 shutdown_connections 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.206+0000 7f3933e11740 1 --2- 172.21.15.32:0/2540863148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570167350 0x55757050b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.206+0000 7f3933e11740 1 --2- 172.21.15.32:0/2540863148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570146670 0x55757050af70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.206+0000 7f3933e11740 1 --2- 172.21.15.32:0/2540863148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757013e490 0x5575705251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.206+0000 7f3933e11740 1 -- 172.21.15.32:0/2540863148 >> 172.21.15.32:0/2540863148 conn(0x55757015b740 msgr2=0x557570514360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.206+0000 7f3933e11740 1 -- 172.21.15.32:0/2540863148 shutdown_connections 2024-08-31T07:15:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.206+0000 7f3933e11740 1 -- 172.21.15.32:0/2540863148 wait complete. 2024-08-31T07:15:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.207+0000 7f3933e11740 1 -- start start 2024-08-31T07:15:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.207+0000 7f3933e11740 1 Processor -- start 2024-08-31T07:15:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3933e11740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757013e490 0x557570591650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3933e11740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x55757058fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3933e11740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570167350 0x557570590070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3933e11740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f39281ae420 con 0x55757013e490 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3933e11740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39281add50 con 0x557570146670 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3933e11740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f39281aaf90 con 0x557570167350 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3933e11740 10 client.0 _ll_register_callbacks cb 0x7f39140e8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3925400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x55757058fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3925400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x55757058fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54722/0 (socket says 172.21.15.32:54722) 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3925400640 1 -- 172.21.15.32:0/2762453927 learned_addr learned my addr 172.21.15.32:0/2762453927 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3925e00640 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757013e490 0x557570591650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.208+0000 7f3926800640 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570167350 0x557570590070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.209+0000 7f3925400640 1 -- 172.21.15.32:0/2762453927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570167350 msgr2=0x557570590070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.209+0000 7f3925400640 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570167350 0x557570590070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.209+0000 7f3925400640 1 -- 172.21.15.32:0/2762453927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757013e490 msgr2=0x557570591650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.209+0000 7f3925400640 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757013e490 0x557570591650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.209+0000 7f3925400640 1 -- 172.21.15.32:0/2762453927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575705907e0 con 0x557570146670 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.209+0000 7f3925400640 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x55757058fb30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f39140a8d70 tx=0x7f3914005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ac5fc810dad1d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.209+0000 7f3907e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.209+0000 7f3907e00640 1 -- 172.21.15.32:0/2762453927 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39140ab450 con 0x557570146670 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.209+0000 7f3933e11740 1 -- 172.21.15.32:0/2762453927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39140ab610 con 0x557570146670 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.209+0000 7f3907e00640 1 -- 172.21.15.32:0/2762453927 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f39140aaf70 con 0x557570146670 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.210+0000 7f3907e00640 1 -- 172.21.15.32:0/2762453927 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f39140ab270 con 0x557570146670 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.210+0000 7f3933e11740 10 client.26216 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.210+0000 7f3933e11740 1 -- 172.21.15.32:0/2762453927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5575705907e0 con 0x557570146670 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.210+0000 7f3907e00640 1 -- 172.21.15.32:0/2762453927 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39140ab610 con 0x557570146670 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.210+0000 7f3907e00640 1 -- 172.21.15.32:0/2762453927 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f39140b1500 con 0x557570146670 2024-08-31T07:15:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.211+0000 7f3933e11740 10 client.26216 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.211+0000 7f3933e11740 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.211+0000 7f3933e11740 20 client.26216 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.211+0000 7f3933e11740 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5575700fb080 0x557570521580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.211+0000 7f3933e11740 4 client.26216 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:15:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.211+0000 7f3933e11740 1 -- 172.21.15.32:0/2762453927 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557570481140 con 0x5575700fb080 2024-08-31T07:15:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.211+0000 7f3925e00640 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5575700fb080 0x557570521580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.212+0000 7f3925e00640 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5575700fb080 0x557570521580 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e736e0dfeaf3600e server_cookie=c45f286c75348ec8 in_seq=0 out_seq=0 2024-08-31T07:15:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.212+0000 7f3907e00640 10 client.26216 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3907e00640 1 -- 172.21.15.32:0/2762453927 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557570481140 con 0x5575700fb080 2024-08-31T07:15:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3907e00640 10 client.26216 handle_command_reply: tid=0 2024-08-31T07:15:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3933e11740 1 client.26216 shutdown 2024-08-31T07:15:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3933e11740 1 -- 172.21.15.32:0/2762453927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 msgr2=0x55757058fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3933e11740 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x55757058fb30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f39140a8d70 tx=0x7f3914005020 comp rx=0 tx=0).stop 2024-08-31T07:15:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3933e11740 1 -- 172.21.15.32:0/2762453927 shutdown_connections 2024-08-31T07:15:51.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3933e11740 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5575700fb080 0x557570521580 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3907e00640 0 client.26216 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:51.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3933e11740 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557570167350 0x557570590070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3933e11740 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557570146670 0x55757058fb30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3933e11740 1 --2- 172.21.15.32:0/2762453927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757013e490 0x557570591650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.214+0000 7f3933e11740 1 -- 172.21.15.32:0/2762453927 >> 172.21.15.32:0/2762453927 conn(0x55757015b740 msgr2=0x55757050ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:51.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.215+0000 7f3933e11740 1 -- 172.21.15.32:0/2762453927 shutdown_connections 2024-08-31T07:15:51.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.215+0000 7f3933e11740 1 -- 172.21.15.32:0/2762453927 wait complete. 2024-08-31T07:15:51.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.215+0000 7f3933e11740 20 client.26216 trim_cache size 0 max 16384 2024-08-31T07:15:51.217 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:15:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:15:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "08644365-5854-418b-a4c9-1e76ad60211b", 2024-08-31T07:15:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:15:51.217 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:15:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.216+0000 7f3927200640 1 -- 172.21.15.32:0/2808996601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f390c0b91b0 msgr2=0x7f390c0bb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.216+0000 7f3927200640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f390c0b91b0 0x7f390c0bb6c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39180096c0 tx=0x7f39180058d0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.216+0000 7f3927200640 1 -- 172.21.15.32:0/2808996601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3928104eb0 msgr2=0x7f392819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.216+0000 7f3927200640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3928104eb0 0x7f392819a660 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f392806a830 tx=0x7f391400ee50 comp rx=0 tx=0).stop 2024-08-31T07:15:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.217+0000 7f3927200640 1 -- 172.21.15.32:0/2808996601 shutdown_connections 2024-08-31T07:15:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.217+0000 7f3927200640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f390c0b91b0 0x7f390c0bb6c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.217+0000 7f3927200640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392810d9c0 0x7f392819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.217+0000 7f3927200640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3928104eb0 0x7f392819a660 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.217+0000 7f3927200640 1 --2- 172.21.15.32:0/2808996601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3928104560 0x7f392819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.217+0000 7f3927200640 1 -- 172.21.15.32:0/2808996601 >> 172.21.15.32:0/2808996601 conn(0x7f39280fe060 msgr2=0x7f39280fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.217+0000 7f3927200640 1 -- 172.21.15.32:0/2808996601 shutdown_connections 2024-08-31T07:15:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:51.217+0000 7f3927200640 1 -- 172.21.15.32:0/2808996601 wait complete. 2024-08-31T07:15:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:51.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103290 5.76 MB/sec warmup 100 sec latency 48.693 ms 2024-08-31T07:15:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.244+0000 7f2892000640 1 -- 172.21.15.32:0/759353087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f288c10b460 msgr2=0x7f288c10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.244+0000 7f2892000640 1 --2- 172.21.15.32:0/759353087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f288c10b460 0x7f288c10b870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2878009900 tx=0x7f287802a400 comp rx=0 tx=0).stop 2024-08-31T07:15:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.244+0000 7f2892000640 1 -- 172.21.15.32:0/759353087 shutdown_connections 2024-08-31T07:15:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.244+0000 7f2892000640 1 --2- 172.21.15.32:0/759353087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f288c10bdb0 0x7f288c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:52.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.244+0000 7f2892000640 1 --2- 172.21.15.32:0/759353087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f288c10b460 0x7f288c10b870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:52.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.244+0000 7f2892000640 1 --2- 172.21.15.32:0/759353087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f288c105b20 0x7f288c105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:52.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.244+0000 7f2892000640 1 -- 172.21.15.32:0/759353087 >> 172.21.15.32:0/759353087 conn(0x7f288c101570 msgr2=0x7f288c1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:52.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.244+0000 7f2892000640 1 -- 172.21.15.32:0/759353087 shutdown_connections 2024-08-31T07:15:52.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.244+0000 7f2892000640 1 -- 172.21.15.32:0/759353087 wait complete. 2024-08-31T07:15:52.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.245+0000 7f2892000640 1 Processor -- start 2024-08-31T07:15:52.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.245+0000 7f2892000640 1 -- start start 2024-08-31T07:15:52.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f2892000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f288c105b20 0x7f288c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:52.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f2892000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f288c10b460 0x7f288c19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:52.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f2892000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f288c10bdb0 0x7f288c1a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:52.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f2892000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f288c115ac0 con 0x7f288c105b20 2024-08-31T07:15:52.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f2892000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f288c115940 con 0x7f288c10bdb0 2024-08-31T07:15:52.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f2892000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f288c115c40 con 0x7f288c10b460 2024-08-31T07:15:52.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f288b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f288c105b20 0x7f288c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:52.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f288b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f288c105b20 0x7f288c19c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45766/0 (socket says 172.21.15.32:45766) 2024-08-31T07:15:52.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f288b400640 1 -- 172.21.15.32:0/872683400 learned_addr learned my addr 172.21.15.32:0/872683400 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:52.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f288aa00640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f288c10b460 0x7f288c19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:52.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.246+0000 7f288be00640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f288c10bdb0 0x7f288c1a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.247+0000 7f288b400640 1 -- 172.21.15.32:0/872683400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f288c10b460 msgr2=0x7f288c19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.247+0000 7f288b400640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f288c10b460 0x7f288c19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.247+0000 7f288b400640 1 -- 172.21.15.32:0/872683400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f288c10bdb0 msgr2=0x7f288c1a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.247+0000 7f288b400640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f288c10bdb0 0x7f288c1a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.247+0000 7f288b400640 1 -- 172.21.15.32:0/872683400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c1a23a0 con 0x7f288c105b20 2024-08-31T07:15:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.247+0000 7f288aa00640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f288c10b460 0x7f288c19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.247+0000 7f288b400640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f288c105b20 0x7f288c19c430 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f287c00e9f0 tx=0x7f287c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.248+0000 7f2883e00640 1 -- 172.21.15.32:0/872683400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c00cd80 con 0x7f288c105b20 2024-08-31T07:15:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.248+0000 7f2892000640 1 -- 172.21.15.32:0/872683400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288c1a2690 con 0x7f288c105b20 2024-08-31T07:15:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.248+0000 7f2883e00640 1 -- 172.21.15.32:0/872683400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f287c0108e0 con 0x7f288c105b20 2024-08-31T07:15:52.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.248+0000 7f2883e00640 1 -- 172.21.15.32:0/872683400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c018980 con 0x7f288c105b20 2024-08-31T07:15:52.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.248+0000 7f2892000640 1 -- 172.21.15.32:0/872683400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288c1a8df0 con 0x7f288c105b20 2024-08-31T07:15:52.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.249+0000 7f2892000640 1 -- 172.21.15.32:0/872683400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288c105f30 con 0x7f288c105b20 2024-08-31T07:15:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.251+0000 7f2883e00640 1 -- 172.21.15.32:0/872683400 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f287c010430 con 0x7f288c105b20 2024-08-31T07:15:52.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.253+0000 7f2883e00640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28700b4ca0 0x7f28700b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:52.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.253+0000 7f2883e00640 1 -- 172.21.15.32:0/872683400 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f287c0e5d20 con 0x7f288c105b20 2024-08-31T07:15:52.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.253+0000 7f288aa00640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28700b4ca0 0x7f28700b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:52.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.254+0000 7f288aa00640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28700b4ca0 0x7f28700b71b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f287802a9f0 tx=0x7f2878005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:52.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.255+0000 7f2883e00640 1 -- 172.21.15.32:0/872683400 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f287c0ac4e0 con 0x7f288c105b20 2024-08-31T07:15:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.489+0000 7f2892000640 1 -- 172.21.15.32:0/872683400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f288c19d560 con 0x7f288c105b20 2024-08-31T07:15:52.490 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: pgmap v461: 97 pgs: 97 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 399 KiB/s wr, 81 op/s 2024-08-31T07:15:52.490 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:15:52.490 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.490 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:52.490 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:52 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.490+0000 7f2883e00640 1 -- 172.21.15.32:0/872683400 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11967 (secure 0 0 0) 0x7f287c0b5070 con 0x7f288c105b20 2024-08-31T07:15:52.492 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:15:52.493 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"btime":"2024-08-31T07:14:36:074596+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:14:35.939633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:15:52.493 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 -- 172.21.15.32:0/872683400 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28700b4ca0 msgr2=0x7f28700b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28700b4ca0 0x7f28700b71b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f287802a9f0 tx=0x7f2878005a20 comp rx=0 tx=0).stop 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 -- 172.21.15.32:0/872683400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f288c105b20 msgr2=0x7f288c19c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f288c105b20 0x7f288c19c430 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f287c00e9f0 tx=0x7f287c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 -- 172.21.15.32:0/872683400 shutdown_connections 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28700b4ca0 0x7f28700b71b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f288c10bdb0 0x7f288c1a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f288c10b460 0x7f288c19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 --2- 172.21.15.32:0/872683400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f288c105b20 0x7f288c19c430 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 -- 172.21.15.32:0/872683400 >> 172.21.15.32:0/872683400 conn(0x7f288c101570 msgr2=0x7f288c103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 -- 172.21.15.32:0/872683400 shutdown_connections 2024-08-31T07:15:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:52.493+0000 7f2892000640 1 -- 172.21.15.32:0/872683400 wait complete. 2024-08-31T07:15:52.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:15:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: pgmap v461: 97 pgs: 97 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 399 KiB/s wr, 81 op/s 2024-08-31T07:15:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:15:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:52 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103576 5.70 MB/sec warmup 101 sec latency 716.269 ms 2024-08-31T07:15:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: pgmap v461: 97 pgs: 97 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 399 KiB/s wr, 81 op/s 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:52.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:15:52.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:52.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:52 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:15:53.406 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0' 2024-08-31T07:15:53.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0 2024-08-31T07:15:53.480 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:53.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:53 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/872683400' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:15:53.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:53 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/872683400' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:15:53.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103576 5.64 MB/sec warmup 102 sec latency 1716.361 ms 2024-08-31T07:15:53.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:53 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/872683400' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:15:54.130 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:15:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.147+0000 7fa7d7800640 1 -- 172.21.15.32:0/906737165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d0109250 msgr2=0x7fa7d0109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.147+0000 7fa7d7800640 1 --2- 172.21.15.32:0/906737165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d0109250 0x7fa7d0109660 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009860 tx=0x7fa7bc02a340 comp rx=0 tx=0).stop 2024-08-31T07:15:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.147+0000 7fa7d7800640 1 -- 172.21.15.32:0/906737165 shutdown_connections 2024-08-31T07:15:54.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.147+0000 7fa7d7800640 1 --2- 172.21.15.32:0/906737165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d0109ba0 0x7fa7d01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.147+0000 7fa7d7800640 1 --2- 172.21.15.32:0/906737165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d0109250 0x7fa7d0109660 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.147+0000 7fa7d7800640 1 --2- 172.21.15.32:0/906737165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0103910 0x7fa7d0103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.147+0000 7fa7d7800640 1 -- 172.21.15.32:0/906737165 >> 172.21.15.32:0/906737165 conn(0x7fa7d00ff380 msgr2=0x7fa7d01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:54.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.147+0000 7fa7d7800640 1 -- 172.21.15.32:0/906737165 shutdown_connections 2024-08-31T07:15:54.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.147+0000 7fa7d7800640 1 -- 172.21.15.32:0/906737165 wait complete. 2024-08-31T07:15:54.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.149+0000 7fa7d7800640 1 Processor -- start 2024-08-31T07:15:54.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.149+0000 7fa7d7800640 1 -- start start 2024-08-31T07:15:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7d7800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d0103910 0x7fa7d019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7d7800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d0109250 0x7fa7d019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7d7800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0109ba0 0x7fa7d019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7d7800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d0113860 con 0x7fa7d0109250 2024-08-31T07:15:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7d7800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01136e0 con 0x7fa7d0103910 2024-08-31T07:15:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7d7800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01139e0 con 0x7fa7d0109ba0 2024-08-31T07:15:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7d4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d0103910 0x7fa7d019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7d4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d0103910 0x7fa7d019a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54768/0 (socket says 172.21.15.32:54768) 2024-08-31T07:15:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7d4e00640 1 -- 172.21.15.32:0/370234940 learned_addr learned my addr 172.21.15.32:0/370234940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7d5800640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0109ba0 0x7fa7d019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.150+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d0109250 0x7fa7d019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.151+0000 7fa7d4e00640 1 -- 172.21.15.32:0/370234940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0109ba0 msgr2=0x7fa7d019fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.151+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0109ba0 0x7fa7d019fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.151+0000 7fa7d4e00640 1 -- 172.21.15.32:0/370234940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d0109250 msgr2=0x7fa7d019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.151+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d0109250 0x7fa7d019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.151+0000 7fa7d4e00640 1 -- 172.21.15.32:0/370234940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d01a0180 con 0x7fa7d0103910 2024-08-31T07:15:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.151+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d0103910 0x7fa7d019a210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400e9f0 tx=0x7fa7c400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:54.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.151+0000 7fa7cd600640 1 -- 172.21.15.32:0/370234940 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c400cd80 con 0x7fa7d0103910 2024-08-31T07:15:54.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.152+0000 7fa7cd600640 1 -- 172.21.15.32:0/370234940 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c40054d0 con 0x7fa7d0103910 2024-08-31T07:15:54.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.152+0000 7fa7d7800640 1 -- 172.21.15.32:0/370234940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d01a0470 con 0x7fa7d0103910 2024-08-31T07:15:54.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.152+0000 7fa7d7800640 1 -- 172.21.15.32:0/370234940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d0075390 con 0x7fa7d0103910 2024-08-31T07:15:54.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.152+0000 7fa7cd600640 1 -- 172.21.15.32:0/370234940 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4010690 con 0x7fa7d0103910 2024-08-31T07:15:54.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.155+0000 7fa7de444740 1 Processor -- start 2024-08-31T07:15:54.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.155+0000 7fa7de444740 1 -- start start 2024-08-31T07:15:54.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.155+0000 7fa7de444740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c01950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.155+0000 7fa7cd600640 1 -- 172.21.15.32:0/370234940 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa7c4010830 con 0x7fa7d0103910 2024-08-31T07:15:54.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7de444740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620408331b0 0x562040c01e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7de444740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56204083b390 0x562040c023d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7de444740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562040b78770 con 0x56204083b390 2024-08-31T07:15:54.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7de444740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562040b67d00 con 0x56204085c070 2024-08-31T07:15:54.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7de444740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562040b6bcb0 con 0x5620408331b0 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7d4e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620408331b0 0x562040c01e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7d4e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620408331b0 0x562040c01e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42992/0 (socket says 172.21.15.32:42992) 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7d4e00640 1 -- 172.21.15.32:0/1200730863 learned_addr learned my addr 172.21.15.32:0/1200730863 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7d5800640 1 --2- 172.21.15.32:0/1200730863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c01950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/1200730863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56204083b390 0x562040c023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7d4e00640 1 -- 172.21.15.32:0/1200730863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 msgr2=0x562040c01950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/1200730863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c01950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7d4e00640 1 -- 172.21.15.32:0/1200730863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56204083b390 msgr2=0x562040c023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/1200730863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56204083b390 0x562040c023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7d4e00640 1 -- 172.21.15.32:0/1200730863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620408c7650 con 0x5620408331b0 2024-08-31T07:15:54.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.156+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/1200730863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56204083b390 0x562040c023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:54.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.157+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/1200730863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620408331b0 0x562040c01e90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40e2fd0 tx=0x7fa7c40ebb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d9835611151c9b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:54.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.157+0000 7fa7c2000640 1 -- 172.21.15.32:0/1200730863 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c40f4070 con 0x5620408331b0 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.157+0000 7fa7c2000640 1 -- 172.21.15.32:0/1200730863 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c40f0440 con 0x5620408331b0 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.157+0000 7fa7c2000640 1 -- 172.21.15.32:0/1200730863 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c40e7330 con 0x5620408331b0 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.158+0000 7fa7cd600640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7b80b4d20 0x7fa7b80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.158+0000 7fa7cd600640 1 -- 172.21.15.32:0/370234940 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fa7c40e5890 con 0x7fa7d0103910 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.158+0000 7fa7d5800640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7b80b4d20 0x7fa7b80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.158+0000 7fa7de444740 1 -- 172.21.15.32:0/1200730863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620408331b0 msgr2=0x562040c01e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.158+0000 7fa7de444740 1 --2- 172.21.15.32:0/1200730863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620408331b0 0x562040c01e90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40e2fd0 tx=0x7fa7c40ebb40 comp rx=0 tx=0).stop 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.159+0000 7fa7de444740 1 -- 172.21.15.32:0/1200730863 shutdown_connections 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.159+0000 7fa7de444740 1 --2- 172.21.15.32:0/1200730863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56204083b390 0x562040c023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.159+0000 7fa7de444740 1 --2- 172.21.15.32:0/1200730863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5620408331b0 0x562040c01e90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.159+0000 7fa7de444740 1 --2- 172.21.15.32:0/1200730863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c01950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.159+0000 7fa7de444740 1 -- 172.21.15.32:0/1200730863 >> 172.21.15.32:0/1200730863 conn(0x562040850460 msgr2=0x562040bff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:54.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.159+0000 7fa7de444740 1 -- 172.21.15.32:0/1200730863 shutdown_connections 2024-08-31T07:15:54.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.159+0000 7fa7de444740 1 -- 172.21.15.32:0/1200730863 wait complete. 2024-08-31T07:15:54.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.160+0000 7fa7d5800640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7b80b4d20 0x7fa7b80b7230 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8005fd0 tx=0x7fa7c8010040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:54.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.160+0000 7fa7de444740 1 -- start start 2024-08-31T07:15:54.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.160+0000 7fa7de444740 1 Processor -- start 2024-08-31T07:15:54.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7de444740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c0d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7de444740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204083b390 0x562040c0da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7de444740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204085c070 0x562040c19480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7de444740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5620409ca700 con 0x5620408331b0 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7de444740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562040aeecb0 con 0x56204083b390 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7de444740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562040c10130 con 0x56204085c070 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7de444740 10 client.0 _ll_register_callbacks cb 0x562040b14430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7cfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204083b390 0x562040c0da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7cfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204083b390 0x562040c0da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54784/0 (socket says 172.21.15.32:54784) 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7cfe00640 1 -- 172.21.15.32:0/2207958498 learned_addr learned my addr 172.21.15.32:0/2207958498 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.161+0000 7fa7d5800640 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204085c070 0x562040c19480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c0d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7cfe00640 1 -- 172.21.15.32:0/2207958498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204085c070 msgr2=0x562040c19480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204085c070 0x562040c19480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7cfe00640 1 -- 172.21.15.32:0/2207958498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 msgr2=0x562040c0d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c0d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7cfe00640 1 -- 172.21.15.32:0/2207958498 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562040b2f870 con 0x56204083b390 2024-08-31T07:15:54.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c0d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:15:54.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204083b390 0x562040c0da00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc02a930 tx=0x7fa7bc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d09fe8de226650a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:54.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:15:54.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7b6a00640 1 -- 172.21.15.32:0/2207958498 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc041070 con 0x56204083b390 2024-08-31T07:15:54.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.162+0000 7fa7b6a00640 1 -- 172.21.15.32:0/2207958498 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fa7bc030650 con 0x56204083b390 2024-08-31T07:15:54.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7de444740 1 -- 172.21.15.32:0/2207958498 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7c40f0440 con 0x56204083b390 2024-08-31T07:15:54.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7b6a00640 1 -- 172.21.15.32:0/2207958498 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa7bc04d050 con 0x56204083b390 2024-08-31T07:15:54.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7de444740 10 client.26234 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:54.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7de444740 1 -- 172.21.15.32:0/2207958498 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562040c1c150 con 0x56204083b390 2024-08-31T07:15:54.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7b6a00640 1 -- 172.21.15.32:0/2207958498 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c40f0440 con 0x56204083b390 2024-08-31T07:15:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7b6a00640 1 -- 172.21.15.32:0/2207958498 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fa7bc032440 con 0x56204083b390 2024-08-31T07:15:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7de444740 10 client.26234 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7de444740 10 client.26234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7de444740 20 client.26234 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7de444740 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56204081a9f0 0x562040c1f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7de444740 4 client.26234 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.163+0000 7fa7de444740 1 -- 172.21.15.32:0/2207958498 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562040b7a6c0 con 0x56204081a9f0 2024-08-31T07:15:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.164+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56204081a9f0 0x562040c1f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.164+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56204081a9f0 0x562040c1f730 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76d406f9d8ddb466 server_cookie=f15931bce83fe155 in_seq=0 out_seq=0 2024-08-31T07:15:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.164+0000 7fa7b6a00640 10 client.26234 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:54.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.165+0000 7fa7b6a00640 1 -- 172.21.15.32:0/2207958498 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562040b7a6c0 con 0x56204081a9f0 2024-08-31T07:15:54.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.165+0000 7fa7b6a00640 10 client.26234 handle_command_reply: tid=0 2024-08-31T07:15:54.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.165+0000 7fa7de444740 1 client.26234 shutdown 2024-08-31T07:15:54.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.165+0000 7fa7de444740 1 -- 172.21.15.32:0/2207958498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204083b390 msgr2=0x562040c0da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:54.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.165+0000 7fa7de444740 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204083b390 0x562040c0da00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc02a930 tx=0x7fa7bc0042f0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.166+0000 7fa7de444740 1 -- 172.21.15.32:0/2207958498 shutdown_connections 2024-08-31T07:15:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.166+0000 7fa7de444740 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56204081a9f0 0x562040c1f730 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.166+0000 7fa7de444740 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204085c070 0x562040c19480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.166+0000 7fa7de444740 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204083b390 0x562040c0da00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.166+0000 7fa7de444740 1 --2- 172.21.15.32:0/2207958498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c0d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.166+0000 7fa7b6a00640 0 client.26234 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.166+0000 7fa7de444740 1 -- 172.21.15.32:0/2207958498 >> 172.21.15.32:0/2207958498 conn(0x562040850460 msgr2=0x562040c04ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.166+0000 7fa7de444740 1 -- 172.21.15.32:0/2207958498 shutdown_connections 2024-08-31T07:15:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.166+0000 7fa7de444740 1 -- 172.21.15.32:0/2207958498 wait complete. 2024-08-31T07:15:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.166+0000 7fa7de444740 20 client.26234 trim_cache size 0 max 16384 2024-08-31T07:15:54.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.185+0000 7fa7de444740 1 Processor -- start 2024-08-31T07:15:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.185+0000 7fa7de444740 1 -- start start 2024-08-31T07:15:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.186+0000 7fa7de444740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c05410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.186+0000 7fa7de444740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c08470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.186+0000 7fa7de444740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c19380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.186+0000 7fa7de444740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01b1530 con 0x5620408331b0 2024-08-31T07:15:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.186+0000 7fa7de444740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01b3130 con 0x56204085c070 2024-08-31T07:15:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.186+0000 7fa7de444740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01b1c00 con 0x56204083b390 2024-08-31T07:15:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.186+0000 7fa7cfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c08470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7cfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c08470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:43014/0 (socket says 172.21.15.32:43014) 2024-08-31T07:15:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7cfe00640 1 -- 172.21.15.32:0/1067940637 learned_addr learned my addr 172.21.15.32:0/1067940637 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/1067940637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c05410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7d5800640 1 --2- 172.21.15.32:0/1067940637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c19380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7cfe00640 1 -- 172.21.15.32:0/1067940637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 msgr2=0x562040c19380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/1067940637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c19380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7cfe00640 1 -- 172.21.15.32:0/1067940637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 msgr2=0x562040c05410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/1067940637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c05410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7cfe00640 1 -- 172.21.15.32:0/1067940637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562040c047b0 con 0x56204083b390 2024-08-31T07:15:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7d5800640 1 --2- 172.21.15.32:0/1067940637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c19380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:15:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/1067940637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c08470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009860 tx=0x7fa7bc001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603f665c43b65590 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:54.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.187+0000 7fa7c1600640 1 -- 172.21.15.32:0/1067940637 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc032e60 con 0x56204083b390 2024-08-31T07:15:54.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.188+0000 7fa7c1600640 1 -- 172.21.15.32:0/1067940637 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7bc005d30 con 0x56204083b390 2024-08-31T07:15:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.188+0000 7fa7c1600640 1 -- 172.21.15.32:0/1067940637 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc041040 con 0x56204083b390 2024-08-31T07:15:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.188+0000 7fa7de444740 1 -- 172.21.15.32:0/1067940637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 msgr2=0x562040c08470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.188+0000 7fa7de444740 1 --2- 172.21.15.32:0/1067940637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c08470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009860 tx=0x7fa7bc001e70 comp rx=0 tx=0).stop 2024-08-31T07:15:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.189+0000 7fa7de444740 1 -- 172.21.15.32:0/1067940637 shutdown_connections 2024-08-31T07:15:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.189+0000 7fa7de444740 1 --2- 172.21.15.32:0/1067940637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c19380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.189+0000 7fa7de444740 1 --2- 172.21.15.32:0/1067940637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c08470 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.189+0000 7fa7de444740 1 --2- 172.21.15.32:0/1067940637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c05410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.189+0000 7fa7de444740 1 -- 172.21.15.32:0/1067940637 >> 172.21.15.32:0/1067940637 conn(0x562040850460 msgr2=0x562040c0bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.189+0000 7fa7de444740 1 -- 172.21.15.32:0/1067940637 shutdown_connections 2024-08-31T07:15:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.189+0000 7fa7de444740 1 -- 172.21.15.32:0/1067940637 wait complete. 2024-08-31T07:15:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.189+0000 7fa7de444740 1 -- start start 2024-08-31T07:15:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.190+0000 7fa7de444740 1 Processor -- start 2024-08-31T07:15:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.190+0000 7fa7de444740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c15d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.190+0000 7fa7de444740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c12220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.190+0000 7fa7de444740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.190+0000 7fa7de444740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01b29a0 con 0x5620408331b0 2024-08-31T07:15:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.190+0000 7fa7de444740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01b22d0 con 0x56204085c070 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.190+0000 7fa7de444740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01af530 con 0x56204083b390 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.190+0000 7fa7de444740 10 client.0 _ll_register_callbacks cb 0x7fa7bc0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7cfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c12220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7cfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c12220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:43016/0 (socket says 172.21.15.32:43016) 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7cfe00640 1 -- 172.21.15.32:0/3279378291 learned_addr learned my addr 172.21.15.32:0/3279378291 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c15d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7d5800640 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7cfe00640 1 -- 172.21.15.32:0/3279378291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 msgr2=0x562040c128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7cfe00640 1 -- 172.21.15.32:0/3279378291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 msgr2=0x562040c15d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c15d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7cfe00640 1 -- 172.21.15.32:0/3279378291 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562040c8a140 con 0x56204083b390 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7cfe00640 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c12220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc02a860 tx=0x7fa7bc0327b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ffe78b03753000e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.191+0000 7fa7b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.192+0000 7fa7b6a00640 1 -- 172.21.15.32:0/3279378291 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc041040 con 0x56204083b390 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.192+0000 7fa7b6a00640 1 -- 172.21.15.32:0/3279378291 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fa7bc030650 con 0x56204083b390 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.192+0000 7fa7de444740 1 -- 172.21.15.32:0/3279378291 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7bc005d30 con 0x56204083b390 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.192+0000 7fa7b6a00640 1 -- 172.21.15.32:0/3279378291 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa7bc037040 con 0x56204083b390 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.192+0000 7fa7de444740 10 client.26257 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.192+0000 7fa7de444740 1 -- 172.21.15.32:0/3279378291 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562040c8b190 con 0x56204083b390 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.192+0000 7fa7b6a00640 1 -- 172.21.15.32:0/3279378291 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7bc005d30 con 0x56204083b390 2024-08-31T07:15:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.193+0000 7fa7b6a00640 1 -- 172.21.15.32:0/3279378291 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fa7bc0465d0 con 0x56204083b390 2024-08-31T07:15:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.193+0000 7fa7de444740 10 client.26257 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.193+0000 7fa7de444740 10 client.26257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:15:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.193+0000 7fa7de444740 20 client.26257 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.193+0000 7fa7de444740 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5620407efda0 0x562040c8da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.193+0000 7fa7de444740 4 client.26257 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:15:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.193+0000 7fa7de444740 1 -- 172.21.15.32:0/3279378291 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562040b76160 con 0x5620407efda0 2024-08-31T07:15:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.193+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5620407efda0 0x562040c8da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7d4e00640 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5620407efda0 0x562040c8da50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=458838d44e763733 server_cookie=3a70949e8098d08f in_seq=0 out_seq=0 2024-08-31T07:15:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7b6a00640 10 client.26257 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7b6a00640 1 -- 172.21.15.32:0/3279378291 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x562040b76160 con 0x5620407efda0 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7b6a00640 10 client.26257 handle_command_reply: tid=0 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7de444740 1 client.26257 shutdown 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7de444740 1 -- 172.21.15.32:0/3279378291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 msgr2=0x562040c12220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7de444740 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c12220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc02a860 tx=0x7fa7bc0327b0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7de444740 1 -- 172.21.15.32:0/3279378291 shutdown_connections 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7de444740 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5620407efda0 0x562040c8da50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7de444740 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204085c070 0x562040c128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7de444740 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56204083b390 0x562040c12220 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7de444740 1 --2- 172.21.15.32:0/3279378291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5620408331b0 0x562040c15d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7b6a00640 0 client.26257 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.194+0000 7fa7de444740 1 -- 172.21.15.32:0/3279378291 >> 172.21.15.32:0/3279378291 conn(0x562040850460 msgr2=0x562040c0bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.195+0000 7fa7de444740 1 -- 172.21.15.32:0/3279378291 shutdown_connections 2024-08-31T07:15:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.195+0000 7fa7de444740 1 -- 172.21.15.32:0/3279378291 wait complete. 2024-08-31T07:15:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.195+0000 7fa7de444740 20 client.26257 trim_cache size 0 max 16384 2024-08-31T07:15:54.200 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:15:54.200 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:15:54.200 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:15:54.200 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:15:54.201 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:54.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "08644365-5854-418b-a4c9-1e76ad60211b", 2024-08-31T07:15:54.201 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:15:54.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:54.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:54.201 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:15:54.201 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:15:54.201 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:15:54.201 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:54.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:54.203 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:54.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:54.205 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:15:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 -- 172.21.15.32:0/370234940 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7b80b4d20 msgr2=0x7fa7b80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7b80b4d20 0x7fa7b80b7230 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8005fd0 tx=0x7fa7c8010040 comp rx=0 tx=0).stop 2024-08-31T07:15:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 -- 172.21.15.32:0/370234940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d0103910 msgr2=0x7fa7d019a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d0103910 0x7fa7d019a210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400e9f0 tx=0x7fa7c400edf0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 -- 172.21.15.32:0/370234940 shutdown_connections 2024-08-31T07:15:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa7b80b4d20 0x7fa7b80b7230 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7d0109ba0 0x7fa7d019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d0109250 0x7fa7d019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 --2- 172.21.15.32:0/370234940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d0103910 0x7fa7d019a210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 -- 172.21.15.32:0/370234940 >> 172.21.15.32:0/370234940 conn(0x7fa7d00ff380 msgr2=0x7fa7d01017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 -- 172.21.15.32:0/370234940 shutdown_connections 2024-08-31T07:15:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:54.196+0000 7fa7d7800640 1 -- 172.21.15.32:0/370234940 wait complete. 2024-08-31T07:15:54.529 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:54 smithi032 ceph-mon[17252]: pgmap v462: 97 pgs: 97 active+clean; 63 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 294 KiB/s wr, 65 op/s 2024-08-31T07:15:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:54 smithi073 ceph-mon[21995]: pgmap v462: 97 pgs: 97 active+clean; 63 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 294 KiB/s wr, 65 op/s 2024-08-31T07:15:54.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103576 5.59 MB/sec warmup 103 sec latency 2716.465 ms 2024-08-31T07:15:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:54 smithi084 ceph-mon[22387]: pgmap v462: 97 pgs: 97 active+clean; 63 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 294 KiB/s wr, 65 op/s 2024-08-31T07:15:55.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.54 MB/sec warmup 104 sec latency 3141.272 ms 2024-08-31T07:15:56.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:56 smithi032 ceph-mon[17252]: pgmap v463: 97 pgs: 97 active+clean; 63 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 308 KiB/s wr, 67 op/s 2024-08-31T07:15:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:56 smithi073 ceph-mon[21995]: pgmap v463: 97 pgs: 97 active+clean; 63 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 308 KiB/s wr, 67 op/s 2024-08-31T07:15:56.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.48 MB/sec warmup 105 sec latency 1555.799 ms 2024-08-31T07:15:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:56 smithi084 ceph-mon[22387]: pgmap v463: 97 pgs: 97 active+clean; 63 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 308 KiB/s wr, 67 op/s 2024-08-31T07:15:57.273 INFO:tasks.cephfs.filesystem:scrub status for tag:08644365-5854-418b-a4c9-1e76ad60211b - {'path': '/', 'tag': '08644365-5854-418b-a4c9-1e76ad60211b', 'options': 'recursive,force'} 2024-08-31T07:15:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.248+0000 7f5832e00640 1 -- 172.21.15.32:0/1406125141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c10fd00 msgr2=0x7f582c112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.248+0000 7f5832e00640 1 --2- 172.21.15.32:0/1406125141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c10fd00 0x7f582c112120 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5824009ca0 tx=0x7f582402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.248+0000 7f5832e00640 1 -- 172.21.15.32:0/1406125141 shutdown_connections 2024-08-31T07:15:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.248+0000 7f5832e00640 1 --2- 172.21.15.32:0/1406125141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c10fd00 0x7f582c112120 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.248+0000 7f5832e00640 1 --2- 172.21.15.32:0/1406125141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f582c075e40 0x7f582c10f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.248+0000 7f5832e00640 1 --2- 172.21.15.32:0/1406125141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582c0754f0 0x7f582c075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.248+0000 7f5832e00640 1 -- 172.21.15.32:0/1406125141 >> 172.21.15.32:0/1406125141 conn(0x7f582c0ff190 msgr2=0x7f582c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.248+0000 7f5832e00640 1 -- 172.21.15.32:0/1406125141 shutdown_connections 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.248+0000 7f5832e00640 1 -- 172.21.15.32:0/1406125141 wait complete. 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.249+0000 7f5832e00640 1 Processor -- start 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.249+0000 7f5832e00640 1 -- start start 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.250+0000 7f5832e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582c0754f0 0x7f582c1067b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.250+0000 7f5832e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f582c075e40 0x7f582c106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.250+0000 7f5832e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c10fd00 0x7f582c107230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.250+0000 7f5832e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f582c115870 con 0x7f582c10fd00 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.250+0000 7f5832e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1156f0 con 0x7f582c075e40 2024-08-31T07:15:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.250+0000 7f5832e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1159f0 con 0x7f582c0754f0 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.251+0000 7f5831000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f582c075e40 0x7f582c106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.251+0000 7f5831000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f582c075e40 0x7f582c106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54828/0 (socket says 172.21.15.32:54828) 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.251+0000 7f5831000640 1 -- 172.21.15.32:0/1456444017 learned_addr learned my addr 172.21.15.32:0/1456444017 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.251+0000 7f5832400640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c10fd00 0x7f582c107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.252+0000 7f5831000640 1 -- 172.21.15.32:0/1456444017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582c0754f0 msgr2=0x7f582c1067b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.252+0000 7f5831000640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582c0754f0 0x7f582c1067b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.252+0000 7f5831000640 1 -- 172.21.15.32:0/1456444017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c10fd00 msgr2=0x7f582c107230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.252+0000 7f5831000640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c10fd00 0x7f582c107230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.252+0000 7f5831000640 1 -- 172.21.15.32:0/1456444017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c19ae50 con 0x7f582c075e40 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.253+0000 7f5831000640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f582c075e40 0x7f582c106cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5818009ec0 tx=0x7f581800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.253+0000 7f582a000640 1 -- 172.21.15.32:0/1456444017 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581800cd80 con 0x7f582c075e40 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.253+0000 7f582a000640 1 -- 172.21.15.32:0/1456444017 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58180328f0 con 0x7f582c075e40 2024-08-31T07:15:57.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.253+0000 7f582a000640 1 -- 172.21.15.32:0/1456444017 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581803a860 con 0x7f582c075e40 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.253+0000 7f5832e00640 1 -- 172.21.15.32:0/1456444017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582c19b0e0 con 0x7f582c075e40 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.254+0000 7f5832e00640 1 -- 172.21.15.32:0/1456444017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582c19b510 con 0x7f582c075e40 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.258+0000 7f583afe2740 1 Processor -- start 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.259+0000 7f583afe2740 1 -- start start 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.259+0000 7f583afe2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf4fa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.261+0000 7f5832400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf4fa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.261+0000 7f5832400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf4fa2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54842/0 (socket says 172.21.15.32:54842) 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.261+0000 7f5832400640 1 -- 172.21.15.32:0/1028863722 learned_addr learned my addr 172.21.15.32:0/1028863722 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.261+0000 7f583afe2740 1 --2- 172.21.15.32:0/1028863722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf12a980 0x55debf4fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.261+0000 7f583afe2740 1 --2- 172.21.15.32:0/1028863722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf132b60 0x55debf4fad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.261+0000 7f583afe2740 1 -- 172.21.15.32:0/1028863722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55debf46f530 con 0x55debf132b60 2024-08-31T07:15:57.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.261+0000 7f583afe2740 1 -- 172.21.15.32:0/1028863722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55debf45eac0 con 0x55debf153840 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.261+0000 7f583afe2740 1 -- 172.21.15.32:0/1028863722 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55debf462a70 con 0x55debf12a980 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.262+0000 7f5832400640 1 -- 172.21.15.32:0/1028863722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf12a980 msgr2=0x55debf4fa810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.262+0000 7f5832400640 1 --2- 172.21.15.32:0/1028863722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf12a980 0x55debf4fa810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.262+0000 7f5832400640 1 -- 172.21.15.32:0/1028863722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf132b60 msgr2=0x55debf4fad50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.262+0000 7f5832400640 1 --2- 172.21.15.32:0/1028863722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf132b60 0x55debf4fad50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.262+0000 7f5832400640 1 -- 172.21.15.32:0/1028863722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55debf411880 con 0x55debf153840 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.262+0000 7f582a000640 1 -- 172.21.15.32:0/1456444017 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5818032440 con 0x7f582c075e40 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.263+0000 7f5832400640 1 --2- 172.21.15.32:0/1028863722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf4fa2d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5824005cc0 tx=0x7f582402f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b01d95440c073e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.264+0000 7f581ea00640 1 -- 172.21.15.32:0/1028863722 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824031cf0 con 0x55debf153840 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.264+0000 7f581ea00640 1 -- 172.21.15.32:0/1028863722 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5824036040 con 0x55debf153840 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.266+0000 7f581ea00640 1 -- 172.21.15.32:0/1028863722 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824031cf0 con 0x55debf153840 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.266+0000 7f582a000640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f58140b4dc0 0x7f58140b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.266+0000 7f583afe2740 1 -- 172.21.15.32:0/1028863722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 msgr2=0x55debf4fa2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:57.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.266+0000 7f583afe2740 1 --2- 172.21.15.32:0/1028863722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf4fa2d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5824005cc0 tx=0x7f582402f420 comp rx=0 tx=0).stop 2024-08-31T07:15:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.267+0000 7f583afe2740 1 -- 172.21.15.32:0/1028863722 shutdown_connections 2024-08-31T07:15:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.267+0000 7f583afe2740 1 --2- 172.21.15.32:0/1028863722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf132b60 0x55debf4fad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.267+0000 7f583afe2740 1 --2- 172.21.15.32:0/1028863722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf12a980 0x55debf4fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.267+0000 7f583afe2740 1 --2- 172.21.15.32:0/1028863722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf4fa2d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.267+0000 7f583afe2740 1 -- 172.21.15.32:0/1028863722 >> 172.21.15.32:0/1028863722 conn(0x55debf147990 msgr2=0x55debf4f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.267+0000 7f582a000640 1 -- 172.21.15.32:0/1456444017 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f5818107850 con 0x7f582c075e40 2024-08-31T07:15:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.268+0000 7f5831a00640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f58140b4dc0 0x7f58140b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.268+0000 7f583afe2740 1 -- 172.21.15.32:0/1028863722 shutdown_connections 2024-08-31T07:15:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.269+0000 7f583afe2740 1 -- 172.21.15.32:0/1028863722 wait complete. 2024-08-31T07:15:57.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.270+0000 7f5831a00640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f58140b4dc0 0x7f58140b72d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55debf4f8f50 tx=0x7f5820005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.270+0000 7f583afe2740 1 -- start start 2024-08-31T07:15:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.270+0000 7f583afe2740 1 Processor -- start 2024-08-31T07:15:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.271+0000 7f583afe2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 0x55debf505db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.271+0000 7f583afe2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf5062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.271+0000 7f583afe2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 0x55debf511e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.271+0000 7f583afe2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55debf2c17a0 con 0x55debf153840 2024-08-31T07:15:57.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.271+0000 7f583afe2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55debf3e57f0 con 0x55debf12a980 2024-08-31T07:15:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.271+0000 7f583afe2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55debf508af0 con 0x55debf132b60 2024-08-31T07:15:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.271+0000 7f583afe2740 10 client.0 _ll_register_callbacks cb 0x55debf40ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.271+0000 7f5831000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf5062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.271+0000 7f5831000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf5062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:43036/0 (socket says 172.21.15.32:43036) 2024-08-31T07:15:57.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.271+0000 7f5831000640 1 -- 172.21.15.32:0/4010111847 learned_addr learned my addr 172.21.15.32:0/4010111847 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.272+0000 7f5831a00640 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 0x55debf505db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.272+0000 7f5832400640 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 0x55debf511e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.272+0000 7f5831000640 1 -- 172.21.15.32:0/4010111847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 msgr2=0x55debf505db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.272+0000 7f5831000640 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 0x55debf505db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.272+0000 7f5831000640 1 -- 172.21.15.32:0/4010111847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 msgr2=0x55debf511e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.272+0000 7f5831000640 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 0x55debf511e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.272+0000 7f5831000640 1 -- 172.21.15.32:0/4010111847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55debf1cdb90 con 0x55debf132b60 2024-08-31T07:15:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.272+0000 7f5832400640 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 0x55debf511e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:15:57.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.272+0000 7f5831000640 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf5062f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f58180c9fe0 tx=0x7f5818032870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f1d7b8614bcf08b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.272+0000 7f5813400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:15:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.275+0000 7f5813400640 1 -- 172.21.15.32:0/4010111847 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180d27d0 con 0x55debf132b60 2024-08-31T07:15:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.275+0000 7f5813400640 1 -- 172.21.15.32:0/4010111847 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f58180d23e0 con 0x55debf132b60 2024-08-31T07:15:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.275+0000 7f583afe2740 1 -- 172.21.15.32:0/4010111847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5824036040 con 0x55debf132b60 2024-08-31T07:15:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f5813400640 1 -- 172.21.15.32:0/4010111847 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f58180d7280 con 0x55debf132b60 2024-08-31T07:15:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f583afe2740 10 client.26263 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f583afe2740 1 -- 172.21.15.32:0/4010111847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55debf514b10 con 0x55debf132b60 2024-08-31T07:15:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f5813400640 1 -- 172.21.15.32:0/4010111847 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5824036040 con 0x55debf132b60 2024-08-31T07:15:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f5813400640 1 -- 172.21.15.32:0/4010111847 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f58180cd1c0 con 0x55debf132b60 2024-08-31T07:15:57.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f583afe2740 10 client.26263 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f583afe2740 10 client.26263 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f583afe2740 20 client.26263 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f583afe2740 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55debf1121c0 0x55debf5184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f583afe2740 4 client.26263 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f583afe2740 1 -- 172.21.15.32:0/4010111847 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55debf471480 con 0x55debf1121c0 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.276+0000 7f5831a00640 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55debf1121c0 0x55debf5184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.277+0000 7f5831a00640 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55debf1121c0 0x55debf5184d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b08fdd96a99afa server_cookie=2775f089422e4e60 in_seq=0 out_seq=0 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.277+0000 7f5813400640 10 client.26263 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f5813400640 1 -- 172.21.15.32:0/4010111847 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55debf471480 con 0x55debf1121c0 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f5813400640 10 client.26263 handle_command_reply: tid=0 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f583afe2740 1 client.26263 shutdown 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f583afe2740 1 -- 172.21.15.32:0/4010111847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 msgr2=0x55debf5062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:57.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f583afe2740 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf5062f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f58180c9fe0 tx=0x7f5818032870 comp rx=0 tx=0).stop 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f583afe2740 1 -- 172.21.15.32:0/4010111847 shutdown_connections 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f583afe2740 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55debf1121c0 0x55debf5184d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f583afe2740 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 0x55debf511e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f583afe2740 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf5062f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f583afe2740 1 --2- 172.21.15.32:0/4010111847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 0x55debf505db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f5813400640 0 client.26263 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f583afe2740 1 -- 172.21.15.32:0/4010111847 >> 172.21.15.32:0/4010111847 conn(0x55debf147990 msgr2=0x55debf4fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.278+0000 7f583afe2740 1 -- 172.21.15.32:0/4010111847 shutdown_connections 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.279+0000 7f583afe2740 1 -- 172.21.15.32:0/4010111847 wait complete. 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.279+0000 7f583afe2740 20 client.26263 trim_cache size 0 max 16384 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.309+0000 7f583afe2740 1 Processor -- start 2024-08-31T07:15:57.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.309+0000 7f583afe2740 1 -- start start 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.309+0000 7f583afe2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf12a980 0x55debf4f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.309+0000 7f583afe2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf132b60 0x55debf5077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.309+0000 7f583afe2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf502da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.309+0000 7f583afe2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1ad080 con 0x55debf132b60 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.309+0000 7f583afe2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1aec80 con 0x55debf153840 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.309+0000 7f583afe2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1ad750 con 0x55debf12a980 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.309+0000 7f5831000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf132b60 0x55debf5077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5831000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf132b60 0x55debf5077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45848/0 (socket says 172.21.15.32:45848) 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5831000640 1 -- 172.21.15.32:0/4034898963 learned_addr learned my addr 172.21.15.32:0/4034898963 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5831a00640 1 --2- 172.21.15.32:0/4034898963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf12a980 0x55debf4f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5832400640 1 --2- 172.21.15.32:0/4034898963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf502da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5832400640 1 -- 172.21.15.32:0/4034898963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf12a980 msgr2=0x55debf4f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5832400640 1 --2- 172.21.15.32:0/4034898963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf12a980 0x55debf4f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5832400640 1 -- 172.21.15.32:0/4034898963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf132b60 msgr2=0x55debf5077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5832400640 1 --2- 172.21.15.32:0/4034898963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf132b60 0x55debf5077e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5832400640 1 -- 172.21.15.32:0/4034898963 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55debf506c50 con 0x55debf153840 2024-08-31T07:15:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5831a00640 1 --2- 172.21.15.32:0/4034898963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf12a980 0x55debf4f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.310+0000 7f5832400640 1 --2- 172.21.15.32:0/4034898963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf502da0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5824009bf0 tx=0x7f5824005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a244be3617bd8c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.311+0000 7f581e000640 1 -- 172.21.15.32:0/4034898963 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824031c30 con 0x55debf153840 2024-08-31T07:15:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.311+0000 7f581e000640 1 -- 172.21.15.32:0/4034898963 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5824036070 con 0x55debf153840 2024-08-31T07:15:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.311+0000 7f581e000640 1 -- 172.21.15.32:0/4034898963 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824004030 con 0x55debf153840 2024-08-31T07:15:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.312+0000 7f583afe2740 1 -- 172.21.15.32:0/4034898963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 msgr2=0x55debf502da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.312+0000 7f583afe2740 1 --2- 172.21.15.32:0/4034898963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf502da0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5824009bf0 tx=0x7f5824005fd0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.312+0000 7f583afe2740 1 -- 172.21.15.32:0/4034898963 shutdown_connections 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.312+0000 7f583afe2740 1 --2- 172.21.15.32:0/4034898963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf153840 0x55debf502da0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.312+0000 7f583afe2740 1 --2- 172.21.15.32:0/4034898963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf132b60 0x55debf5077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.312+0000 7f583afe2740 1 --2- 172.21.15.32:0/4034898963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf12a980 0x55debf4f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.312+0000 7f583afe2740 1 -- 172.21.15.32:0/4034898963 >> 172.21.15.32:0/4034898963 conn(0x55debf147990 msgr2=0x55debf5070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.312+0000 7f583afe2740 1 -- 172.21.15.32:0/4034898963 shutdown_connections 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.312+0000 7f583afe2740 1 -- 172.21.15.32:0/4034898963 wait complete. 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.312+0000 7f583afe2740 1 -- start start 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.313+0000 7f583afe2740 1 Processor -- start 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.313+0000 7f583afe2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 0x55debf57c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.313+0000 7f583afe2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf57c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.313+0000 7f583afe2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 0x55debf51a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.313+0000 7f583afe2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1ae4f0 con 0x55debf153840 2024-08-31T07:15:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.313+0000 7f583afe2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1ade20 con 0x55debf12a980 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.313+0000 7f583afe2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f582c107ab0 con 0x55debf132b60 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.313+0000 7f583afe2740 10 client.0 _ll_register_callbacks cb 0x7f58180cd6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 0x55debf57c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf57c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 0x55debf57c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54868/0 (socket says 172.21.15.32:54868) 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831a00640 1 -- 172.21.15.32:0/2483424578 learned_addr learned my addr 172.21.15.32:0/2483424578 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5832400640 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 0x55debf51a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831000640 1 -- 172.21.15.32:0/2483424578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 msgr2=0x55debf57c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831000640 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 0x55debf57c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831000640 1 -- 172.21.15.32:0/2483424578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 msgr2=0x55debf51a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831000640 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 0x55debf51a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831000640 1 -- 172.21.15.32:0/2483424578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55debf51ab60 con 0x55debf132b60 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831a00640 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 0x55debf57c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5832400640 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 0x55debf51a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5831000640 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf57c740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f58180cb1d0 tx=0x7f58180c9cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6231a1a5c812174e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5813400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.314+0000 7f5813400640 1 -- 172.21.15.32:0/2483424578 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180cd8b0 con 0x55debf132b60 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.315+0000 7f5813400640 1 -- 172.21.15.32:0/2483424578 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f58180d23e0 con 0x55debf132b60 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.315+0000 7f583afe2740 1 -- 172.21.15.32:0/2483424578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5824036070 con 0x55debf132b60 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.319+0000 7f5813400640 1 -- 172.21.15.32:0/2483424578 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f58180b92e0 con 0x55debf132b60 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.319+0000 7f583afe2740 10 client.26269 fetch_fsmap learned FSMap version 31 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.319+0000 7f583afe2740 1 -- 172.21.15.32:0/2483424578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55debf583910 con 0x55debf132b60 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.331+0000 7f5813400640 1 -- 172.21.15.32:0/2483424578 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5824036070 con 0x55debf132b60 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.331+0000 7f5813400640 1 -- 172.21.15.32:0/2483424578 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f58180b50d0 con 0x55debf132b60 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.331+0000 7f583afe2740 10 client.26269 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:15:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.331+0000 7f583afe2740 10 client.26269 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.331+0000 7f583afe2740 20 client.26269 populate_metadata read hostname 'smithi032' 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.331+0000 7f583afe2740 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55debf0e7570 0x55debf586080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.331+0000 7f583afe2740 4 client.26269 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}] 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.331+0000 7f583afe2740 1 -- 172.21.15.32:0/2483424578 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}) -- 0x55debf46cf20 con 0x55debf0e7570 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.332+0000 7f5831a00640 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55debf0e7570 0x55debf586080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.332+0000 7f5831a00640 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55debf0e7570 0x55debf586080 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9c018b57e42d15 server_cookie=410cb146cc639b33 in_seq=0 out_seq=0 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:55.332+0000 7f5813400640 10 client.26269 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.102+0000 7f5813400640 1 -- 172.21.15.32:0/2483424578 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55debf46cf20 con 0x55debf0e7570 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.102+0000 7f5813400640 10 client.26269 handle_command_reply: tid=0 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.102+0000 7f583afe2740 1 client.26269 shutdown 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 1 -- 172.21.15.32:0/2483424578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 msgr2=0x55debf57c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf57c740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f58180cb1d0 tx=0x7f58180c9cc0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 1 -- 172.21.15.32:0/2483424578 shutdown_connections 2024-08-31T07:15:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55debf0e7570 0x55debf586080 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55debf153840 0x55debf51a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55debf132b60 0x55debf57c740 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 1 --2- 172.21.15.32:0/2483424578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55debf12a980 0x55debf57c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f5813400640 0 client.26269 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:15:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 1 -- 172.21.15.32:0/2483424578 >> 172.21.15.32:0/2483424578 conn(0x55debf147990 msgr2=0x55debf4f6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 1 -- 172.21.15.32:0/2483424578 shutdown_connections 2024-08-31T07:15:57.377 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 31, 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 10, 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "14376fac": { 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 10, 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:15:57.378 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 141.8, 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:15:57.379 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:15:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 1 -- 172.21.15.32:0/2483424578 wait complete. 2024-08-31T07:15:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.103+0000 7f583afe2740 20 client.26269 trim_cache size 0 max 16384 2024-08-31T07:15:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 -- 172.21.15.32:0/1456444017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f58140b4dc0 msgr2=0x7f58140b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f58140b4dc0 0x7f58140b72d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55debf4f8f50 tx=0x7f5820005f90 comp rx=0 tx=0).stop 2024-08-31T07:15:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 -- 172.21.15.32:0/1456444017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f582c075e40 msgr2=0x7f582c106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:15:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f582c075e40 0x7f582c106cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5818009ec0 tx=0x7f581800edc0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 -- 172.21.15.32:0/1456444017 shutdown_connections 2024-08-31T07:15:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f58140b4dc0 0x7f58140b72d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f582c10fd00 0x7f582c107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f582c075e40 0x7f582c106cf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 --2- 172.21.15.32:0/1456444017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f582c0754f0 0x7f582c1067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:15:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 -- 172.21.15.32:0/1456444017 >> 172.21.15.32:0/1456444017 conn(0x7f582c0ff190 msgr2=0x7f582c109830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:15:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 -- 172.21.15.32:0/1456444017 shutdown_connections 2024-08-31T07:15:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:15:57.104+0000 7f5832e00640 1 -- 172.21.15.32:0/1456444017 wait complete. 2024-08-31T07:15:57.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.43 MB/sec warmup 106 sec latency 2555.861 ms 2024-08-31T07:15:58.206 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '14376fac', quiesce took 7.4 seconds. Will release after: 14.4 2024-08-31T07:15:58.573 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:15:58 smithi032 ceph-mon[17252]: pgmap v464: 97 pgs: 97 active+clean; 67 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 723 KiB/s wr, 92 op/s 2024-08-31T07:15:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:15:58 smithi073 ceph-mon[21995]: pgmap v464: 97 pgs: 97 active+clean; 67 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 723 KiB/s wr, 92 op/s 2024-08-31T07:15:58.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.38 MB/sec warmup 107 sec latency 3555.959 ms 2024-08-31T07:15:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:15:58 smithi084 ceph-mon[22387]: pgmap v464: 97 pgs: 97 active+clean; 67 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 723 KiB/s wr, 92 op/s 2024-08-31T07:15:59.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.33 MB/sec warmup 108 sec latency 4556.135 ms 2024-08-31T07:16:00.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:00 smithi032 ceph-mon[17252]: pgmap v465: 97 pgs: 97 active+clean; 67 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 732 KiB/s wr, 74 op/s 2024-08-31T07:16:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:00 smithi073 ceph-mon[21995]: pgmap v465: 97 pgs: 97 active+clean; 67 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 732 KiB/s wr, 74 op/s 2024-08-31T07:16:00.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.28 MB/sec warmup 109 sec latency 5556.262 ms 2024-08-31T07:16:00.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:00 smithi084 ceph-mon[22387]: pgmap v465: 97 pgs: 97 active+clean; 67 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 732 KiB/s wr, 74 op/s 2024-08-31T07:16:01.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:16:01.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:16:01.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:16:01.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:16:01.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:16:01.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:16:01.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.23 MB/sec warmup 110 sec latency 6556.345 ms 2024-08-31T07:16:01.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:16:01.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:16:01.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:16:02.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:02 smithi032 ceph-mon[17252]: pgmap v466: 97 pgs: 97 active+clean; 66 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-08-31T07:16:02.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:02 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:16:02.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:02 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:16:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:02 smithi073 ceph-mon[21995]: pgmap v466: 97 pgs: 97 active+clean; 66 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-08-31T07:16:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:02 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:16:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:02 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:16:02.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.19 MB/sec warmup 111 sec latency 7556.510 ms 2024-08-31T07:16:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:02 smithi084 ceph-mon[22387]: pgmap v466: 97 pgs: 97 active+clean; 66 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-08-31T07:16:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:02 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:16:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:02 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:16:03.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.14 MB/sec warmup 112 sec latency 8556.629 ms 2024-08-31T07:16:04.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:04 smithi032 ceph-mon[17252]: pgmap v467: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-08-31T07:16:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:04 smithi073 ceph-mon[21995]: pgmap v467: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-08-31T07:16:04.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.10 MB/sec warmup 113 sec latency 9556.740 ms 2024-08-31T07:16:04.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:04 smithi084 ceph-mon[22387]: pgmap v467: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-08-31T07:16:05.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.05 MB/sec warmup 114 sec latency 10556.902 ms 2024-08-31T07:16:06.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:06 smithi032 ceph-mon[17252]: pgmap v468: 97 pgs: 97 active+clean; 63 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 91 op/s 2024-08-31T07:16:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:06 smithi073 ceph-mon[21995]: pgmap v468: 97 pgs: 97 active+clean; 63 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 91 op/s 2024-08-31T07:16:06.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 5.01 MB/sec warmup 115 sec latency 11557.001 ms 2024-08-31T07:16:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:06 smithi084 ceph-mon[22387]: pgmap v468: 97 pgs: 97 active+clean; 63 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 91 op/s 2024-08-31T07:16:07.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 4.96 MB/sec warmup 116 sec latency 12557.104 ms 2024-08-31T07:16:08.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:08 smithi032 ceph-mon[17252]: pgmap v469: 97 pgs: 97 active+clean; 63 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-08-31T07:16:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:08 smithi073 ceph-mon[21995]: pgmap v469: 97 pgs: 97 active+clean; 63 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-08-31T07:16:08.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 4.92 MB/sec warmup 117 sec latency 13557.236 ms 2024-08-31T07:16:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:08 smithi084 ceph-mon[22387]: pgmap v469: 97 pgs: 97 active+clean; 63 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-08-31T07:16:09.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 4.88 MB/sec warmup 118 sec latency 14557.323 ms 2024-08-31T07:16:10.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:10 smithi032 ceph-mon[17252]: pgmap v470: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 733 KiB/s wr, 59 op/s 2024-08-31T07:16:10.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:10 smithi073 ceph-mon[21995]: pgmap v470: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 733 KiB/s wr, 59 op/s 2024-08-31T07:16:10.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 4.84 MB/sec warmup 119 sec latency 15557.401 ms 2024-08-31T07:16:10.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:10 smithi084 ceph-mon[22387]: pgmap v470: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 733 KiB/s wr, 59 op/s 2024-08-31T07:16:12.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:16:12.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:12 smithi032 ceph-mon[17252]: pgmap v471: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 610 KiB/s wr, 52 op/s 2024-08-31T07:16:12.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:12 smithi073 ceph-mon[21995]: pgmap v471: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 610 KiB/s wr, 52 op/s 2024-08-31T07:16:12.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 0.00 MB/sec execute 1 sec latency 17557.580 ms 2024-08-31T07:16:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:12 smithi084 ceph-mon[22387]: pgmap v471: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 610 KiB/s wr, 52 op/s 2024-08-31T07:16:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:13.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 0.00 MB/sec execute 2 sec latency 18557.627 ms 2024-08-31T07:16:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.721+0000 7f2d29400640 1 -- 172.21.15.32:0/2960234969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d24109bb0 msgr2=0x7f2d241103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.721+0000 7f2d29400640 1 --2- 172.21.15.32:0/2960234969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d24109bb0 0x7f2d241103e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009ca0 tx=0x7f2d1c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.721+0000 7f2d29400640 1 -- 172.21.15.32:0/2960234969 shutdown_connections 2024-08-31T07:16:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.721+0000 7f2d29400640 1 --2- 172.21.15.32:0/2960234969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d24109bb0 0x7f2d241103e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.721+0000 7f2d29400640 1 --2- 172.21.15.32:0/2960234969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24109260 0x7f2d24109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.721+0000 7f2d29400640 1 --2- 172.21.15.32:0/2960234969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d24103920 0x7f2d24103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.721+0000 7f2d29400640 1 -- 172.21.15.32:0/2960234969 >> 172.21.15.32:0/2960234969 conn(0x7f2d240ff370 msgr2=0x7f2d241017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.721+0000 7f2d29400640 1 -- 172.21.15.32:0/2960234969 shutdown_connections 2024-08-31T07:16:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.721+0000 7f2d29400640 1 -- 172.21.15.32:0/2960234969 wait complete. 2024-08-31T07:16:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.722+0000 7f2d29400640 1 Processor -- start 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.722+0000 7f2d29400640 1 -- start start 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d29400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d24103920 0x7f2d2419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d29400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d24109260 0x7f2d2419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d29400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24109bb0 0x7f2d2419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d29400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24113870 con 0x7f2d24109260 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d29400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241136f0 con 0x7f2d24103920 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d29400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241139f0 con 0x7f2d24109bb0 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d1be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d24109260 0x7f2d2419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d1be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d24109260 0x7f2d2419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52186/0 (socket says 172.21.15.32:52186) 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d22800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d24103920 0x7f2d2419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d1be00640 1 -- 172.21.15.32:0/2652167959 learned_addr learned my addr 172.21.15.32:0/2652167959 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.723+0000 7f2d23200640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24109bb0 0x7f2d2419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d22800640 1 -- 172.21.15.32:0/2652167959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24109bb0 msgr2=0x7f2d2419fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d22800640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24109bb0 0x7f2d2419fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d22800640 1 -- 172.21.15.32:0/2652167959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d24109260 msgr2=0x7f2d2419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d22800640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d24109260 0x7f2d2419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d22800640 1 -- 172.21.15.32:0/2652167959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d241a0210 con 0x7f2d24103920 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d1be00640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d24109260 0x7f2d2419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d22800640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d24103920 0x7f2d2419a2a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9f0 tx=0x7f2d1400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d1b400640 1 -- 172.21.15.32:0/2652167959 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400ccc0 con 0x7f2d24103920 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d29400640 1 -- 172.21.15.32:0/2652167959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d241a0500 con 0x7f2d24103920 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d1b400640 1 -- 172.21.15.32:0/2652167959 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d140108e0 con 0x7f2d24103920 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d1b400640 1 -- 172.21.15.32:0/2652167959 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14018880 con 0x7f2d24103920 2024-08-31T07:16:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.724+0000 7f2d29400640 1 -- 172.21.15.32:0/2652167959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241a6c60 con 0x7f2d24103920 2024-08-31T07:16:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.726+0000 7f2d29400640 1 -- 172.21.15.32:0/2652167959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d24103d30 con 0x7f2d24103920 2024-08-31T07:16:13.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.728+0000 7f2d1b400640 1 -- 172.21.15.32:0/2652167959 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2d14010430 con 0x7f2d24103920 2024-08-31T07:16:13.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.730+0000 7f2d1b400640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d080b4ca0 0x7f2d080b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:13.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.730+0000 7f2d1b400640 1 -- 172.21.15.32:0/2652167959 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2d140e5d40 con 0x7f2d24103920 2024-08-31T07:16:13.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.730+0000 7f2d1be00640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d080b4ca0 0x7f2d080b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:13.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.730+0000 7f2d1be00640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d080b4ca0 0x7f2d080b71b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2d24107bc0 tx=0x7f2d0c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:13.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.731+0000 7f2d1b400640 1 -- 172.21.15.32:0/2652167959 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2d140ac500 con 0x7f2d24103920 2024-08-31T07:16:13.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.966+0000 7f2d29400640 1 -- 172.21.15.32:0/2652167959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d2419b3d0 con 0x7f2d24103920 2024-08-31T07:16:13.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.967+0000 7f2d1b400640 1 -- 172.21.15.32:0/2652167959 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11967 (secure 0 0 0) 0x7f2d140b5090 con 0x7f2d24103920 2024-08-31T07:16:13.969 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:16:13.969 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"btime":"2024-08-31T07:14:36:074596+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:14:35.939633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:16:13.969 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2024-08-31T07:16:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 -- 172.21.15.32:0/2652167959 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d080b4ca0 msgr2=0x7f2d080b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d080b4ca0 0x7f2d080b71b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2d24107bc0 tx=0x7f2d0c005f90 comp rx=0 tx=0).stop 2024-08-31T07:16:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 -- 172.21.15.32:0/2652167959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d24103920 msgr2=0x7f2d2419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d24103920 0x7f2d2419a2a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9f0 tx=0x7f2d1400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 -- 172.21.15.32:0/2652167959 shutdown_connections 2024-08-31T07:16:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d080b4ca0 0x7f2d080b71b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24109bb0 0x7f2d2419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d24109260 0x7f2d2419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 --2- 172.21.15.32:0/2652167959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d24103920 0x7f2d2419a2a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 -- 172.21.15.32:0/2652167959 >> 172.21.15.32:0/2652167959 conn(0x7f2d240ff370 msgr2=0x7f2d24101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 -- 172.21.15.32:0/2652167959 shutdown_connections 2024-08-31T07:16:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:13.970+0000 7f2d29400640 1 -- 172.21.15.32:0/2652167959 wait complete. 2024-08-31T07:16:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:14 smithi032 ceph-mon[17252]: pgmap v472: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-08-31T07:16:14.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:14 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2652167959' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:14.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:14 smithi073 ceph-mon[21995]: pgmap v472: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-08-31T07:16:14.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:14 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2652167959' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:14.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 0.00 MB/sec execute 3 sec latency 19557.725 ms 2024-08-31T07:16:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:14 smithi084 ceph-mon[22387]: pgmap v472: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 47 KiB/s wr, 32 op/s 2024-08-31T07:16:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:14 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2652167959' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:15.161 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 14376fac --release --await' 2024-08-31T07:16:15.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 14376fac --release --await 2024-08-31T07:16:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:15.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103620 0.00 MB/sec execute 4 sec latency 20557.833 ms 2024-08-31T07:16:16.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.414+0000 7fe852200640 1 -- 172.21.15.32:0/2350891898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe84c10bbe0 msgr2=0x7fe84c112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:16.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.414+0000 7fe852200640 1 --2- 172.21.15.32:0/2350891898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe84c10bbe0 0x7fe84c112410 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe844009ca0 tx=0x7fe84402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.414+0000 7fe852200640 1 -- 172.21.15.32:0/2350891898 shutdown_connections 2024-08-31T07:16:16.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.414+0000 7fe852200640 1 --2- 172.21.15.32:0/2350891898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe84c10bbe0 0x7fe84c112410 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.414+0000 7fe852200640 1 --2- 172.21.15.32:0/2350891898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84c10b290 0x7fe84c10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.414+0000 7fe852200640 1 --2- 172.21.15.32:0/2350891898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe84c105950 0x7fe84c105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.414+0000 7fe852200640 1 -- 172.21.15.32:0/2350891898 >> 172.21.15.32:0/2350891898 conn(0x7fe84c101420 msgr2=0x7fe84c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:16.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.414+0000 7fe852200640 1 -- 172.21.15.32:0/2350891898 shutdown_connections 2024-08-31T07:16:16.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.414+0000 7fe852200640 1 -- 172.21.15.32:0/2350891898 wait complete. 2024-08-31T07:16:16.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.415+0000 7fe852200640 1 Processor -- start 2024-08-31T07:16:16.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.415+0000 7fe852200640 1 -- start start 2024-08-31T07:16:16.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe852200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe84c105950 0x7fe84c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe852200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe84c10b290 0x7fe84c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe852200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84c10bbe0 0x7fe84c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe852200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1158f0 con 0x7fe84c105950 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe852200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c115770 con 0x7fe84c10b290 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe852200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c115a70 con 0x7fe84c10bbe0 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe850e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe84c105950 0x7fe84c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe850e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe84c105950 0x7fe84c19c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53620/0 (socket says 172.21.15.32:53620) 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe850e00640 1 -- 172.21.15.32:0/978197444 learned_addr learned my addr 172.21.15.32:0/978197444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe84be00640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe84c10b290 0x7fe84c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe851800640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84c10bbe0 0x7fe84c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe850e00640 1 -- 172.21.15.32:0/978197444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84c10bbe0 msgr2=0x7fe84c1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe850e00640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84c10bbe0 0x7fe84c1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe850e00640 1 -- 172.21.15.32:0/978197444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe84c10b290 msgr2=0x7fe84c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe850e00640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe84c10b290 0x7fe84c19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe850e00640 1 -- 172.21.15.32:0/978197444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84c1a2190 con 0x7fe84c105950 2024-08-31T07:16:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.416+0000 7fe84be00640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe84c10b290 0x7fe84c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.417+0000 7fe850e00640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe84c105950 0x7fe84c19c220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e9f0 tx=0x7fe84000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.417+0000 7fe849600640 1 -- 172.21.15.32:0/978197444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84000cd80 con 0x7fe84c105950 2024-08-31T07:16:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.417+0000 7fe849600640 1 -- 172.21.15.32:0/978197444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8400108e0 con 0x7fe84c105950 2024-08-31T07:16:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.417+0000 7fe852200640 1 -- 172.21.15.32:0/978197444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84c1a2480 con 0x7fe84c105950 2024-08-31T07:16:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.417+0000 7fe849600640 1 -- 172.21.15.32:0/978197444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe840018980 con 0x7fe84c105950 2024-08-31T07:16:16.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.417+0000 7fe852200640 1 -- 172.21.15.32:0/978197444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe84c1a8be0 con 0x7fe84c105950 2024-08-31T07:16:16.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.420+0000 7fe85a41f740 1 Processor -- start 2024-08-31T07:16:16.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe85a41f740 1 -- start start 2024-08-31T07:16:16.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe849600640 1 -- 172.21.15.32:0/978197444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe840010430 con 0x7fe84c105950 2024-08-31T07:16:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c3c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c3c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 0x55a0f0c3cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe85a41f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a0f0bb2a90 con 0x55a0f0896520 2024-08-31T07:16:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe85a41f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a0f0ba2020 con 0x55a0f086d660 2024-08-31T07:16:16.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe85a41f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0f0ba5fd0 con 0x55a0f0875840 2024-08-31T07:16:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe851800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c3c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe851800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c3c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53622/0 (socket says 172.21.15.32:53622) 2024-08-31T07:16:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe851800640 1 -- 172.21.15.32:0/1134957521 learned_addr learned my addr 172.21.15.32:0/1134957521 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.421+0000 7fe850e00640 1 --2- 172.21.15.32:0/1134957521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c3c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.422+0000 7fe84be00640 1 --2- 172.21.15.32:0/1134957521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 0x55a0f0c3cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.422+0000 7fe851800640 1 -- 172.21.15.32:0/1134957521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 msgr2=0x55a0f0c3cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.422+0000 7fe851800640 1 --2- 172.21.15.32:0/1134957521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 0x55a0f0c3cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.422+0000 7fe851800640 1 -- 172.21.15.32:0/1134957521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 msgr2=0x55a0f0c3c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.422+0000 7fe851800640 1 --2- 172.21.15.32:0/1134957521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c3c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.422+0000 7fe851800640 1 -- 172.21.15.32:0/1134957521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f0901b00 con 0x55a0f0896520 2024-08-31T07:16:16.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.422+0000 7fe850e00640 1 --2- 172.21.15.32:0/1134957521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c3c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:16.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.422+0000 7fe851800640 1 --2- 172.21.15.32:0/1134957521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c3c090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe844009ca0 tx=0x7fe84402f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84435f82b599421 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:16.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.423+0000 7fe83e000640 1 -- 172.21.15.32:0/1134957521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844031db0 con 0x55a0f0896520 2024-08-31T07:16:16.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.423+0000 7fe83e000640 1 -- 172.21.15.32:0/1134957521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe844036030 con 0x55a0f0896520 2024-08-31T07:16:16.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.423+0000 7fe83e000640 1 -- 172.21.15.32:0/1134957521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8440042a0 con 0x55a0f0896520 2024-08-31T07:16:16.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.423+0000 7fe849600640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8340b4d20 0x7fe8340b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.423+0000 7fe849600640 1 -- 172.21.15.32:0/978197444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe8400e5750 con 0x7fe84c105950 2024-08-31T07:16:16.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.423+0000 7fe850e00640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8340b4d20 0x7fe8340b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.424+0000 7fe85a41f740 1 -- 172.21.15.32:0/1134957521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 msgr2=0x55a0f0c3c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.424+0000 7fe85a41f740 1 --2- 172.21.15.32:0/1134957521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c3c090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe844009ca0 tx=0x7fe84402f310 comp rx=0 tx=0).stop 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.424+0000 7fe850e00640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8340b4d20 0x7fe8340b7230 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8400e2fd0 tx=0x7fe8400eb550 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.424+0000 7fe85a41f740 1 -- 172.21.15.32:0/1134957521 shutdown_connections 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.424+0000 7fe85a41f740 1 --2- 172.21.15.32:0/1134957521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 0x55a0f0c3cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.424+0000 7fe85a41f740 1 --2- 172.21.15.32:0/1134957521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c3c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.424+0000 7fe85a41f740 1 --2- 172.21.15.32:0/1134957521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c3c090 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.424+0000 7fe85a41f740 1 -- 172.21.15.32:0/1134957521 >> 172.21.15.32:0/1134957521 conn(0x55a0f088a910 msgr2=0x55a0f0c39f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.424+0000 7fe85a41f740 1 -- 172.21.15.32:0/1134957521 shutdown_connections 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.424+0000 7fe85a41f740 1 -- 172.21.15.32:0/1134957521 wait complete. 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.425+0000 7fe85a41f740 1 -- start start 2024-08-31T07:16:16.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.425+0000 7fe85a41f740 1 Processor -- start 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.425+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f086d660 0x55a0f0c47410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.425+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f0875840 0x55a0f0c47950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.425+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c47e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.425+0000 7fe85a41f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a0f0a04bb0 con 0x55a0f0896520 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.425+0000 7fe85a41f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a0f0b28f90 con 0x55a0f0875840 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.425+0000 7fe85a41f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0f0c4a690 con 0x55a0f086d660 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe85a41f740 10 client.0 _ll_register_callbacks cb 0x55a0f0b4e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe851800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f086d660 0x55a0f0c47410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe851800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c47e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe84be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f0875840 0x55a0f0c47950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe84be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f0875840 0x55a0f0c47950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60724/0 (socket says 172.21.15.32:60724) 2024-08-31T07:16:16.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe84be00640 1 -- 172.21.15.32:0/3912007711 learned_addr learned my addr 172.21.15.32:0/3912007711 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:16.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe851800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c47e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53636/0 (socket says 172.21.15.32:53636) 2024-08-31T07:16:16.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe84be00640 1 -- 172.21.15.32:0/3912007711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f086d660 msgr2=0x55a0f0c47410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:16.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe84be00640 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f086d660 0x55a0f0c47410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe84be00640 1 -- 172.21.15.32:0/3912007711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 msgr2=0x55a0f0c47e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:16.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe84be00640 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c47e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.426+0000 7fe84be00640 1 -- 172.21.15.32:0/3912007711 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f0978960 con 0x55a0f0875840 2024-08-31T07:16:16.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.427+0000 7fe84be00640 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f0875840 0x55a0f0c47950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe838011800 tx=0x7fe838011cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39ba72b318998c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.427+0000 7fe832a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.427+0000 7fe832a00640 1 -- 172.21.15.32:0/3912007711 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838010c60 con 0x55a0f0875840 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.427+0000 7fe85a41f740 1 -- 172.21.15.32:0/3912007711 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a0f0b40e30 con 0x55a0f0875840 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.427+0000 7fe832a00640 1 -- 172.21.15.32:0/3912007711 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe838010e00 con 0x55a0f0875840 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.427+0000 7fe832a00640 1 -- 172.21.15.32:0/3912007711 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe838004020 con 0x55a0f0875840 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.427+0000 7fe85a41f740 10 client.26273 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.427+0000 7fe85a41f740 1 -- 172.21.15.32:0/3912007711 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a0f0c56050 con 0x55a0f0875840 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.428+0000 7fe832a00640 1 -- 172.21.15.32:0/3912007711 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a0f0b40e30 con 0x55a0f0875840 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.428+0000 7fe832a00640 1 -- 172.21.15.32:0/3912007711 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe838024430 con 0x55a0f0875840 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.428+0000 7fe85a41f740 10 client.26273 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.428+0000 7fe85a41f740 10 client.26273 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:16:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.428+0000 7fe85a41f740 20 client.26273 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.428+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0f0854ea0 0x55a0f0c59610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.428+0000 7fe85a41f740 4 client.26273 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.428+0000 7fe85a41f740 1 -- 172.21.15.32:0/3912007711 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0f0bb49e0 con 0x55a0f0854ea0 2024-08-31T07:16:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.429+0000 7fe851800640 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0f0854ea0 0x55a0f0c59610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.429+0000 7fe851800640 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0f0854ea0 0x55a0f0c59610 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd94b1c403f61205 server_cookie=86d95963b3941720 in_seq=0 out_seq=0 2024-08-31T07:16:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.429+0000 7fe832a00640 10 client.26273 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.430+0000 7fe832a00640 1 -- 172.21.15.32:0/3912007711 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a0f0bb49e0 con 0x55a0f0854ea0 2024-08-31T07:16:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.430+0000 7fe832a00640 10 client.26273 handle_command_reply: tid=0 2024-08-31T07:16:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.430+0000 7fe85a41f740 1 client.26273 shutdown 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.430+0000 7fe85a41f740 1 -- 172.21.15.32:0/3912007711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f0875840 msgr2=0x55a0f0c47950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.430+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f0875840 0x55a0f0c47950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe838011800 tx=0x7fe838011cc0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.431+0000 7fe85a41f740 1 -- 172.21.15.32:0/3912007711 shutdown_connections 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.431+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0f0854ea0 0x55a0f0c59610 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.431+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c47e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.431+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f0875840 0x55a0f0c47950 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.431+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3912007711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f086d660 0x55a0f0c47410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.431+0000 7fe832a00640 0 client.26273 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.431+0000 7fe85a41f740 1 -- 172.21.15.32:0/3912007711 >> 172.21.15.32:0/3912007711 conn(0x55a0f088a910 msgr2=0x55a0f0c3f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.431+0000 7fe85a41f740 1 -- 172.21.15.32:0/3912007711 shutdown_connections 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.431+0000 7fe85a41f740 1 -- 172.21.15.32:0/3912007711 wait complete. 2024-08-31T07:16:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.431+0000 7fe85a41f740 20 client.26273 trim_cache size 0 max 16384 2024-08-31T07:16:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.449+0000 7fe85a41f740 1 Processor -- start 2024-08-31T07:16:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.449+0000 7fe85a41f740 1 -- start start 2024-08-31T07:16:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.449+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c53880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.449+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 0x55a0f0c47a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.449+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c47fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.449+0000 7fe85a41f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b12d0 con 0x55a0f0896520 2024-08-31T07:16:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.449+0000 7fe85a41f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b2ed0 con 0x55a0f086d660 2024-08-31T07:16:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe85a41f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b19a0 con 0x55a0f0875840 2024-08-31T07:16:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe84be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 0x55a0f0c47a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe851800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c47fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe84be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 0x55a0f0c47a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34406/0 (socket says 172.21.15.32:34406) 2024-08-31T07:16:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe84be00640 1 -- 172.21.15.32:0/1883747196 learned_addr learned my addr 172.21.15.32:0/1883747196 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe851800640 1 --2- 172.21.15.32:0/1883747196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c53880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe84be00640 1 -- 172.21.15.32:0/1883747196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 msgr2=0x55a0f0c53880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe84be00640 1 --2- 172.21.15.32:0/1883747196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c53880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe84be00640 1 -- 172.21.15.32:0/1883747196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 msgr2=0x55a0f0c47fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe84be00640 1 --2- 172.21.15.32:0/1883747196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c47fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe84be00640 1 -- 172.21.15.32:0/1883747196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f0c42fa0 con 0x55a0f0875840 2024-08-31T07:16:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.450+0000 7fe851800640 1 --2- 172.21.15.32:0/1883747196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c53880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:16:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.451+0000 7fe84be00640 1 --2- 172.21.15.32:0/1883747196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 0x55a0f0c47a90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe83800c010 tx=0x7fe838024dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbc4e2fda714b748 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.451+0000 7fe83d600640 1 -- 172.21.15.32:0/1883747196 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838031070 con 0x55a0f0875840 2024-08-31T07:16:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.451+0000 7fe83d600640 1 -- 172.21.15.32:0/1883747196 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8380054f0 con 0x55a0f0875840 2024-08-31T07:16:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.451+0000 7fe83d600640 1 -- 172.21.15.32:0/1883747196 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8380041c0 con 0x55a0f0875840 2024-08-31T07:16:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.452+0000 7fe85a41f740 1 -- 172.21.15.32:0/1883747196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 msgr2=0x55a0f0c47a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.452+0000 7fe85a41f740 1 --2- 172.21.15.32:0/1883747196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 0x55a0f0c47a90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe83800c010 tx=0x7fe838024dd0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.452+0000 7fe85a41f740 1 -- 172.21.15.32:0/1883747196 shutdown_connections 2024-08-31T07:16:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.452+0000 7fe85a41f740 1 --2- 172.21.15.32:0/1883747196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0896520 0x55a0f0c47fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.452+0000 7fe85a41f740 1 --2- 172.21.15.32:0/1883747196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0875840 0x55a0f0c47a90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.452+0000 7fe85a41f740 1 --2- 172.21.15.32:0/1883747196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c53880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.452+0000 7fe85a41f740 1 -- 172.21.15.32:0/1883747196 >> 172.21.15.32:0/1883747196 conn(0x55a0f088a910 msgr2=0x55a0f0c47380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.452+0000 7fe85a41f740 1 -- 172.21.15.32:0/1883747196 shutdown_connections 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.452+0000 7fe85a41f740 1 -- 172.21.15.32:0/1883747196 wait complete. 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.452+0000 7fe85a41f740 1 -- start start 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe85a41f740 1 Processor -- start 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c4ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0875840 0x55a0f0c4c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe85a41f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0896520 0x55a0f0c4c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe85a41f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b2740 con 0x55a0f0875840 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe85a41f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b2070 con 0x55a0f086d660 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe85a41f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1af2b0 con 0x55a0f0896520 2024-08-31T07:16:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe85a41f740 10 client.0 _ll_register_callbacks cb 0x7fe838011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe84be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0875840 0x55a0f0c4c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe84be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0875840 0x55a0f0c4c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53650/0 (socket says 172.21.15.32:53650) 2024-08-31T07:16:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe84be00640 1 -- 172.21.15.32:0/3079359434 learned_addr learned my addr 172.21.15.32:0/3079359434 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.453+0000 7fe851800640 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c4ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.454+0000 7fe851800640 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0896520 0x55a0f0c4c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.454+0000 7fe84be00640 1 -- 172.21.15.32:0/3079359434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0896520 msgr2=0x55a0f0c4c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.454+0000 7fe84be00640 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0896520 0x55a0f0c4c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.454+0000 7fe84be00640 1 -- 172.21.15.32:0/3079359434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 msgr2=0x55a0f0c4ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.454+0000 7fe84be00640 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c4ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.454+0000 7fe84be00640 1 -- 172.21.15.32:0/3079359434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f0c4cf50 con 0x55a0f0875840 2024-08-31T07:16:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.454+0000 7fe851800640 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c4ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.454+0000 7fe84be00640 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0875840 0x55a0f0c4c420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe838014040 tx=0x7fe83800b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe5b9a17e68aed2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.454+0000 7fe832a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.454+0000 7fe832a00640 1 -- 172.21.15.32:0/3079359434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838031040 con 0x55a0f0875840 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe832a00640 1 -- 172.21.15.32:0/3079359434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe838024430 con 0x55a0f0875840 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe85a41f740 1 -- 172.21.15.32:0/3079359434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8380054f0 con 0x55a0f0875840 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe832a00640 1 -- 172.21.15.32:0/3079359434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe838024630 con 0x55a0f0875840 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe85a41f740 10 client.16269 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe85a41f740 1 -- 172.21.15.32:0/3079359434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a0f0cc4d80 con 0x55a0f0875840 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe832a00640 1 -- 172.21.15.32:0/3079359434 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8380054f0 con 0x55a0f0875840 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe832a00640 1 -- 172.21.15.32:0/3079359434 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe83803d070 con 0x55a0f0875840 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe85a41f740 10 client.16269 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe85a41f740 10 client.16269 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe85a41f740 20 client.16269 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.455+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0f082a250 0x55a0f0cc7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.456+0000 7fe85a41f740 4 client.16269 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "14376fac", "release": true, "await": true}] 2024-08-31T07:16:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.456+0000 7fe85a41f740 1 -- 172.21.15.32:0/3079359434 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "14376fac", "release": true, "await": true}) -- 0x55a0f0bb0480 con 0x55a0f082a250 2024-08-31T07:16:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.456+0000 7fe851800640 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0f082a250 0x55a0f0cc7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.456+0000 7fe851800640 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0f082a250 0x55a0f0cc7500 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dffea3dc0d8e75a server_cookie=c4d3802d1fbd6e64 in_seq=0 out_seq=0 2024-08-31T07:16:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.456+0000 7fe832a00640 10 client.16269 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe832a00640 1 -- 172.21.15.32:0/3079359434 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a0f0bb0480 con 0x55a0f082a250 2024-08-31T07:16:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe832a00640 10 client.16269 handle_command_reply: tid=0 2024-08-31T07:16:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 client.16269 shutdown 2024-08-31T07:16:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 -- 172.21.15.32:0/3079359434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0875840 msgr2=0x55a0f0c4c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0875840 0x55a0f0c4c420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe838014040 tx=0x7fe83800b490 comp rx=0 tx=0).stop 2024-08-31T07:16:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 -- 172.21.15.32:0/3079359434 shutdown_connections 2024-08-31T07:16:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0f082a250 0x55a0f0cc7500 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe832a00640 0 client.16269 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0f0896520 0x55a0f0c4c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0f0875840 0x55a0f0c4c420 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 --2- 172.21.15.32:0/3079359434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0f086d660 0x55a0f0c4ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 -- 172.21.15.32:0/3079359434 >> 172.21.15.32:0/3079359434 conn(0x55a0f088a910 msgr2=0x55a0f0c47380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 -- 172.21.15.32:0/3079359434 shutdown_connections 2024-08-31T07:16:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.468+0000 7fe85a41f740 1 -- 172.21.15.32:0/3079359434 wait complete. 2024-08-31T07:16:16.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe85a41f740 20 client.16269 trim_cache size 0 max 16384 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 31, 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 12, 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: "14376fac": { 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 12, 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:16:16.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 141.8, 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:16.472 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:16:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe852200640 1 -- 172.21.15.32:0/978197444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8340b4d20 msgr2=0x7fe8340b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe852200640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8340b4d20 0x7fe8340b7230 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8400e2fd0 tx=0x7fe8400eb550 comp rx=0 tx=0).stop 2024-08-31T07:16:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe852200640 1 -- 172.21.15.32:0/978197444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe84c105950 msgr2=0x7fe84c19c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe852200640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe84c105950 0x7fe84c19c220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e9f0 tx=0x7fe84000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe852200640 1 -- 172.21.15.32:0/978197444 shutdown_connections 2024-08-31T07:16:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe852200640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8340b4d20 0x7fe8340b7230 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe852200640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84c10bbe0 0x7fe84c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe852200640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe84c10b290 0x7fe84c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe852200640 1 --2- 172.21.15.32:0/978197444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe84c105950 0x7fe84c19c220 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.469+0000 7fe852200640 1 -- 172.21.15.32:0/978197444 >> 172.21.15.32:0/978197444 conn(0x7fe84c101420 msgr2=0x7fe84c108800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.470+0000 7fe852200640 1 -- 172.21.15.32:0/978197444 shutdown_connections 2024-08-31T07:16:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:16.470+0000 7fe852200640 1 -- 172.21.15.32:0/978197444 wait complete. 2024-08-31T07:16:16.492 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:16 smithi032 ceph-mon[17252]: pgmap v473: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-08-31T07:16:16.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:16 smithi073 ceph-mon[21995]: pgmap v473: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-08-31T07:16:16.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103823 0.10 MB/sec execute 5 sec latency 21333.413 ms 2024-08-31T07:16:16.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:16 smithi084 ceph-mon[22387]: pgmap v473: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-08-31T07:16:17.560 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '14376fac', total seconds elapsed: 26.7 2024-08-31T07:16:17.560 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.8 2024-08-31T07:16:17.560 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:16:17.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 106258 2.36 MB/sec execute 6 sec latency 12.982 ms 2024-08-31T07:16:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:18.535 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:18 smithi032 ceph-mon[17252]: pgmap v474: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 48 KiB/s wr, 29 op/s 2024-08-31T07:16:18.535 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:18 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:16:18.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:18 smithi073 ceph-mon[21995]: pgmap v474: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 48 KiB/s wr, 29 op/s 2024-08-31T07:16:18.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:18 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:16:18.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107173 2.73 MB/sec execute 7 sec latency 116.594 ms 2024-08-31T07:16:18.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:18 smithi084 ceph-mon[22387]: pgmap v474: 97 pgs: 97 active+clean; 63 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 48 KiB/s wr, 29 op/s 2024-08-31T07:16:18.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:18 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:16:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.878+0000 7f839b200640 1 -- 172.21.15.32:0/2225433323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c10b290 msgr2=0x7f839c10b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.878+0000 7f839b200640 1 --2- 172.21.15.32:0/2225433323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c10b290 0x7f839c10b6a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8384009860 tx=0x7f838402a340 comp rx=0 tx=0).stop 2024-08-31T07:16:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.878+0000 7f839b200640 1 -- 172.21.15.32:0/2225433323 shutdown_connections 2024-08-31T07:16:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.878+0000 7f839b200640 1 --2- 172.21.15.32:0/2225433323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f839c10bbe0 0x7f839c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.878+0000 7f839b200640 1 --2- 172.21.15.32:0/2225433323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c10b290 0x7f839c10b6a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.878+0000 7f839b200640 1 --2- 172.21.15.32:0/2225433323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839c105950 0x7f839c105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.878+0000 7f839b200640 1 -- 172.21.15.32:0/2225433323 >> 172.21.15.32:0/2225433323 conn(0x7f839c101420 msgr2=0x7f839c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.878+0000 7f839b200640 1 -- 172.21.15.32:0/2225433323 shutdown_connections 2024-08-31T07:16:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.879+0000 7f839b200640 1 -- 172.21.15.32:0/2225433323 wait complete. 2024-08-31T07:16:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.880+0000 7f839b200640 1 Processor -- start 2024-08-31T07:16:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.880+0000 7f839b200640 1 -- start start 2024-08-31T07:16:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.880+0000 7f839b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c105950 0x7f839c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.880+0000 7f839b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f839c10b290 0x7f839c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.880+0000 7f839b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839c10bbe0 0x7f839c1a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.880+0000 7f839b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f839c1158a0 con 0x7f839c10bbe0 2024-08-31T07:16:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.880+0000 7f839b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f839c115720 con 0x7f839c10b290 2024-08-31T07:16:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.880+0000 7f839b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f839c115a20 con 0x7f839c105950 2024-08-31T07:16:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.880+0000 7f839a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839c10bbe0 0x7f839c1a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.881+0000 7f8391400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f839c10b290 0x7f839c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.881+0000 7f839a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839c10bbe0 0x7f839c1a1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53660/0 (socket says 172.21.15.32:53660) 2024-08-31T07:16:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.881+0000 7f839a800640 1 -- 172.21.15.32:0/3730866091 learned_addr learned my addr 172.21.15.32:0/3730866091 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.881+0000 7f8399e00640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c105950 0x7f839c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.881+0000 7f8391400640 1 -- 172.21.15.32:0/3730866091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c105950 msgr2=0x7f839c19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.881+0000 7f8391400640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c105950 0x7f839c19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.881+0000 7f8391400640 1 -- 172.21.15.32:0/3730866091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839c10bbe0 msgr2=0x7f839c1a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.881+0000 7f8391400640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839c10bbe0 0x7f839c1a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.881+0000 7f8391400640 1 -- 172.21.15.32:0/3730866091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c1a2260 con 0x7f839c10b290 2024-08-31T07:16:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.881+0000 7f8391400640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f839c10b290 0x7f839c19c890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f838402a860 tx=0x7f8384032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.882+0000 7f8393400640 1 -- 172.21.15.32:0/3730866091 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384040020 con 0x7f839c10b290 2024-08-31T07:16:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.882+0000 7f839b200640 1 -- 172.21.15.32:0/3730866091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c1a24f0 con 0x7f839c10b290 2024-08-31T07:16:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.882+0000 7f839b200640 1 -- 172.21.15.32:0/3730866091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1a8cb0 con 0x7f839c10b290 2024-08-31T07:16:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.882+0000 7f8393400640 1 -- 172.21.15.32:0/3730866091 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83840328f0 con 0x7f839c10b290 2024-08-31T07:16:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.882+0000 7f8393400640 1 -- 172.21.15.32:0/3730866091 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384037040 con 0x7f839c10b290 2024-08-31T07:16:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.883+0000 7f839b200640 1 -- 172.21.15.32:0/3730866091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83780052c0 con 0x7f839c10b290 2024-08-31T07:16:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.885+0000 7f8393400640 1 -- 172.21.15.32:0/3730866091 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8384043020 con 0x7f839c10b290 2024-08-31T07:16:18.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.887+0000 7f8393400640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83800b4ca0 0x7f83800b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.887+0000 7f8393400640 1 -- 172.21.15.32:0/3730866091 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f8384104870 con 0x7f839c10b290 2024-08-31T07:16:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.887+0000 7f8399e00640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83800b4ca0 0x7f83800b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.888+0000 7f8399e00640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83800b4ca0 0x7f83800b71b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f838c009530 tx=0x7f838c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:18.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:18.891+0000 7f8393400640 1 -- 172.21.15.32:0/3730866091 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f838404a030 con 0x7f839c10b290 2024-08-31T07:16:19.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.123+0000 7f839b200640 1 -- 172.21.15.32:0/3730866091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8378005880 con 0x7f839c10b290 2024-08-31T07:16:19.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.124+0000 7f8393400640 1 -- 172.21.15.32:0/3730866091 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11967 (secure 0 0 0) 0x7f8384030e10 con 0x7f839c10b290 2024-08-31T07:16:19.126 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:16:19.126 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"btime":"2024-08-31T07:14:36:074596+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:14:35.939633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:16:19.126 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2024-08-31T07:16:19.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 -- 172.21.15.32:0/3730866091 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83800b4ca0 msgr2=0x7f83800b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:19.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83800b4ca0 0x7f83800b71b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f838c009530 tx=0x7f838c009260 comp rx=0 tx=0).stop 2024-08-31T07:16:19.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 -- 172.21.15.32:0/3730866091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f839c10b290 msgr2=0x7f839c19c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:19.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f839c10b290 0x7f839c19c890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f838402a860 tx=0x7f8384032d50 comp rx=0 tx=0).stop 2024-08-31T07:16:19.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 -- 172.21.15.32:0/3730866091 shutdown_connections 2024-08-31T07:16:19.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83800b4ca0 0x7f83800b71b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:19.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839c10bbe0 0x7f839c1a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:19.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f839c10b290 0x7f839c19c890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:19.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 --2- 172.21.15.32:0/3730866091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f839c105950 0x7f839c19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:19.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 -- 172.21.15.32:0/3730866091 >> 172.21.15.32:0/3730866091 conn(0x7f839c101420 msgr2=0x7f839c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:19.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 -- 172.21.15.32:0/3730866091 shutdown_connections 2024-08-31T07:16:19.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:19.127+0000 7f839b200640 1 -- 172.21.15.32:0/3730866091 wait complete. 2024-08-31T07:16:19.527 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:19 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:16:19.527 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:19 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:16:19.527 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:19 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:16:19.528 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:19 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3730866091' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:19.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:19 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:16:19.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:19 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:16:19.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:19 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:16:19.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:19 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3730866091' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:19.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107173 2.39 MB/sec execute 8 sec latency 1116.641 ms 2024-08-31T07:16:19.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:19 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:16:19.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:19 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:16:19.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:19 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:16:19.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:19 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3730866091' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:20.051 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0' 2024-08-31T07:16:20.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0 2024-08-31T07:16:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:20.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:20 smithi032 ceph-mon[17252]: pgmap v475: 97 pgs: 97 active+clean; 64 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 69 KiB/s wr, 17 op/s 2024-08-31T07:16:20.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:20 smithi084 ceph-mon[22387]: pgmap v475: 97 pgs: 97 active+clean; 64 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 69 KiB/s wr, 17 op/s 2024-08-31T07:16:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:20 smithi073 ceph-mon[21995]: pgmap v475: 97 pgs: 97 active+clean; 64 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 69 KiB/s wr, 17 op/s 2024-08-31T07:16:20.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107241 2.14 MB/sec execute 9 sec latency 1537.994 ms 2024-08-31T07:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.107+0000 7f9386e00640 1 -- 172.21.15.32:0/2326993180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9380103990 msgr2=0x7f9380103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.107+0000 7f9386e00640 1 --2- 172.21.15.32:0/2326993180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9380103990 0x7f9380103da0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9370009900 tx=0x7f937002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.108+0000 7f9386e00640 1 -- 172.21.15.32:0/2326993180 shutdown_connections 2024-08-31T07:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.108+0000 7f9386e00640 1 --2- 172.21.15.32:0/2326993180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9380109c20 0x7f9380110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.108+0000 7f9386e00640 1 --2- 172.21.15.32:0/2326993180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93801092d0 0x7f93801096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.108+0000 7f9386e00640 1 --2- 172.21.15.32:0/2326993180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9380103990 0x7f9380103da0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.108+0000 7f9386e00640 1 -- 172.21.15.32:0/2326993180 >> 172.21.15.32:0/2326993180 conn(0x7f938007b680 msgr2=0x7f938007ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:21.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.108+0000 7f9386e00640 1 -- 172.21.15.32:0/2326993180 shutdown_connections 2024-08-31T07:16:21.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.108+0000 7f9386e00640 1 -- 172.21.15.32:0/2326993180 wait complete. 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.109+0000 7f9386e00640 1 Processor -- start 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.109+0000 7f9386e00640 1 -- start start 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.109+0000 7f9386e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9380103990 0x7f938019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.109+0000 7f9386e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93801092d0 0x7f938019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.109+0000 7f9386e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9380109c20 0x7f93801a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.109+0000 7f9386e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93801138e0 con 0x7f9380109c20 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.109+0000 7f9386e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9380113760 con 0x7f9380103990 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.109+0000 7f9386e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9380113a60 con 0x7f93801092d0 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f9384e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9380109c20 0x7f93801a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f9384e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9380109c20 0x7f93801a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53684/0 (socket says 172.21.15.32:53684) 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f9384e00640 1 -- 172.21.15.32:0/542731360 learned_addr learned my addr 172.21.15.32:0/542731360 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f937fe00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9380103990 0x7f938019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f937f400640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93801092d0 0x7f938019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f9384e00640 1 -- 172.21.15.32:0/542731360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93801092d0 msgr2=0x7f938019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f9384e00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93801092d0 0x7f938019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f9384e00640 1 -- 172.21.15.32:0/542731360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9380103990 msgr2=0x7f938019c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f9384e00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9380103990 0x7f938019c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f9384e00640 1 -- 172.21.15.32:0/542731360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93801a2440 con 0x7f9380109c20 2024-08-31T07:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f937fe00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9380103990 0x7f938019c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.110+0000 7f9384e00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9380109c20 0x7f93801a1d10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f937800eb10 tx=0x7f937800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.111+0000 7f937cc00640 1 -- 172.21.15.32:0/542731360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378004b20 con 0x7f9380109c20 2024-08-31T07:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.111+0000 7f9386e00640 1 -- 172.21.15.32:0/542731360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93801a2730 con 0x7f9380109c20 2024-08-31T07:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.111+0000 7f937cc00640 1 -- 172.21.15.32:0/542731360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9378004cc0 con 0x7f9380109c20 2024-08-31T07:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.111+0000 7f937cc00640 1 -- 172.21.15.32:0/542731360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378010690 con 0x7f9380109c20 2024-08-31T07:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.111+0000 7f9386e00640 1 -- 172.21.15.32:0/542731360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93801a8e90 con 0x7f9380109c20 2024-08-31T07:16:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.114+0000 7f938da3a740 1 Processor -- start 2024-08-31T07:16:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.114+0000 7f937cc00640 1 -- 172.21.15.32:0/542731360 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9378005020 con 0x7f9380109c20 2024-08-31T07:16:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f938da3a740 1 -- start start 2024-08-31T07:16:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f938da3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b6387840 0x55a1b672e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f938da3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b635e980 0x55a1b672e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f938da3a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6366b60 0x55a1b672eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f938da3a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a1b66a3530 con 0x55a1b6366b60 2024-08-31T07:16:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f938da3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a1b6692ac0 con 0x55a1b6387840 2024-08-31T07:16:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f938da3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a1b6696a70 con 0x55a1b635e980 2024-08-31T07:16:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f937f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6366b60 0x55a1b672eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f937f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6366b60 0x55a1b672eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53686/0 (socket says 172.21.15.32:53686) 2024-08-31T07:16:21.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f937f400640 1 -- 172.21.15.32:0/2415340300 learned_addr learned my addr 172.21.15.32:0/2415340300 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:21.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.115+0000 7f937fe00640 1 --2- 172.21.15.32:0/2415340300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b635e980 0x55a1b672e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.116+0000 7f937f400640 1 --2- 172.21.15.32:0/2415340300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b6387840 0x55a1b672e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.116+0000 7f937f400640 1 -- 172.21.15.32:0/2415340300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b635e980 msgr2=0x55a1b672e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:21.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.116+0000 7f937f400640 1 --2- 172.21.15.32:0/2415340300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b635e980 0x55a1b672e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.116+0000 7f937f400640 1 -- 172.21.15.32:0/2415340300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b6387840 msgr2=0x55a1b672e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:21.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.116+0000 7f937f400640 1 --2- 172.21.15.32:0/2415340300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b6387840 0x55a1b672e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.116+0000 7f937f400640 1 -- 172.21.15.32:0/2415340300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b6645880 con 0x55a1b6366b60 2024-08-31T07:16:21.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.116+0000 7f937fe00640 1 --2- 172.21.15.32:0/2415340300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b635e980 0x55a1b672e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.116+0000 7f937f400640 1 --2- 172.21.15.32:0/2415340300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6366b60 0x55a1b672eb90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f936c011790 tx=0x7f936c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f2f241d432dace server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.116+0000 7f937cc00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93640b4d20 0x7f93640b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.117+0000 7f937cc00640 1 -- 172.21.15.32:0/542731360 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f93780e59f0 con 0x7f9380109c20 2024-08-31T07:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.117+0000 7f9375600640 1 -- 172.21.15.32:0/2415340300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c00f7c0 con 0x55a1b6366b60 2024-08-31T07:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.117+0000 7f937fe00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93640b4d20 0x7f93640b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.117+0000 7f9375600640 1 -- 172.21.15.32:0/2415340300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c00fe10 con 0x55a1b6366b60 2024-08-31T07:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.117+0000 7f9375600640 1 -- 172.21.15.32:0/2415340300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c01fd00 con 0x55a1b6366b60 2024-08-31T07:16:21.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.118+0000 7f938da3a740 1 -- 172.21.15.32:0/2415340300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6366b60 msgr2=0x55a1b672eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.118+0000 7f938da3a740 1 --2- 172.21.15.32:0/2415340300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6366b60 0x55a1b672eb90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f936c011790 tx=0x7f936c005850 comp rx=0 tx=0).stop 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.118+0000 7f937fe00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93640b4d20 0x7f93640b7230 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9370031440 tx=0x7f9370032ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.118+0000 7f938da3a740 1 -- 172.21.15.32:0/2415340300 shutdown_connections 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.118+0000 7f938da3a740 1 --2- 172.21.15.32:0/2415340300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6366b60 0x55a1b672eb90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.118+0000 7f938da3a740 1 --2- 172.21.15.32:0/2415340300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b635e980 0x55a1b672e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.118+0000 7f938da3a740 1 --2- 172.21.15.32:0/2415340300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b6387840 0x55a1b672e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.118+0000 7f938da3a740 1 -- 172.21.15.32:0/2415340300 >> 172.21.15.32:0/2415340300 conn(0x55a1b637b990 msgr2=0x55a1b672c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.118+0000 7f938da3a740 1 -- 172.21.15.32:0/2415340300 shutdown_connections 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.118+0000 7f938da3a740 1 -- 172.21.15.32:0/2415340300 wait complete. 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.119+0000 7f938da3a740 1 -- start start 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.119+0000 7f938da3a740 1 Processor -- start 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.119+0000 7f938da3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 0x55a1b6739d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.119+0000 7f938da3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b673a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.119+0000 7f938da3a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b6745c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.119+0000 7f938da3a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a1b64f57a0 con 0x55a1b6387840 2024-08-31T07:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.119+0000 7f938da3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a1b66197f0 con 0x55a1b635e980 2024-08-31T07:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.119+0000 7f938da3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a1b673c8f0 con 0x55a1b6366b60 2024-08-31T07:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f938da3a740 10 client.0 _ll_register_callbacks cb 0x55a1b663ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f9384e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b6745c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f937fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 0x55a1b6739d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f9384e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b6745c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53696/0 (socket says 172.21.15.32:53696) 2024-08-31T07:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f9384e00640 1 -- 172.21.15.32:0/698645690 learned_addr learned my addr 172.21.15.32:0/698645690 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f937f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b673a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f9384e00640 1 -- 172.21.15.32:0/698645690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 msgr2=0x55a1b673a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f9384e00640 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b673a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f9384e00640 1 -- 172.21.15.32:0/698645690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 msgr2=0x55a1b6739d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f9384e00640 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 0x55a1b6739d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.120+0000 7f9384e00640 1 -- 172.21.15.32:0/698645690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b6401b90 con 0x55a1b6387840 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.121+0000 7f9384e00640 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b6745c10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f93780a8180 tx=0x7f93780aba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5089b1c9bdc8d9f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.121+0000 7f936a000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.121+0000 7f936a000640 1 -- 172.21.15.32:0/698645690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93780ac800 con 0x55a1b6387840 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.121+0000 7f936a000640 1 -- 172.21.15.32:0/698645690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f93780ac570 con 0x55a1b6387840 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.121+0000 7f938da3a740 1 -- 172.21.15.32:0/698645690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936c00fe10 con 0x55a1b6387840 2024-08-31T07:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.121+0000 7f936a000640 1 -- 172.21.15.32:0/698645690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f93780ac100 con 0x55a1b6387840 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.121+0000 7f938da3a740 10 client.16287 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.121+0000 7f938da3a740 1 -- 172.21.15.32:0/698645690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a1b6748a10 con 0x55a1b6387840 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.121+0000 7f936a000640 1 -- 172.21.15.32:0/698645690 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c00fe10 con 0x55a1b6387840 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.122+0000 7f936a000640 1 -- 172.21.15.32:0/698645690 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f93780ab3f0 con 0x55a1b6387840 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.122+0000 7f938da3a740 10 client.16287 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.122+0000 7f938da3a740 10 client.16287 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.122+0000 7f938da3a740 20 client.16287 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.122+0000 7f938da3a740 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a1b63461c0 0x55a1b674c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.122+0000 7f938da3a740 4 client.16287 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.122+0000 7f938da3a740 1 -- 172.21.15.32:0/698645690 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1b66a5480 con 0x55a1b63461c0 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.122+0000 7f937fe00640 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a1b63461c0 0x55a1b674c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.123+0000 7f937fe00640 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a1b63461c0 0x55a1b674c040 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e9d27fc84f639d server_cookie=a9e052ae6529ed62 in_seq=0 out_seq=0 2024-08-31T07:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.123+0000 7f936a000640 10 client.16287 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:21.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f936a000640 1 -- 172.21.15.32:0/698645690 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a1b66a5480 con 0x55a1b63461c0 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f936a000640 10 client.16287 handle_command_reply: tid=0 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 client.16287 shutdown 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 -- 172.21.15.32:0/698645690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 msgr2=0x55a1b6745c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b6745c10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f93780a8180 tx=0x7f93780aba50 comp rx=0 tx=0).stop 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 -- 172.21.15.32:0/698645690 shutdown_connections 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a1b63461c0 0x55a1b674c040 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b6745c10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b673a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 --2- 172.21.15.32:0/698645690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 0x55a1b6739d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f936a000640 0 client.16287 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 -- 172.21.15.32:0/698645690 >> 172.21.15.32:0/698645690 conn(0x55a1b637b990 msgr2=0x55a1b672a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 -- 172.21.15.32:0/698645690 shutdown_connections 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 1 -- 172.21.15.32:0/698645690 wait complete. 2024-08-31T07:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.124+0000 7f938da3a740 20 client.16287 trim_cache size 0 max 16384 2024-08-31T07:16:21.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.143+0000 7f938da3a740 1 Processor -- start 2024-08-31T07:16:21.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.143+0000 7f938da3a740 1 -- start start 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.143+0000 7f938da3a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b635e980 0x55a1b67383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.143+0000 7f938da3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b6733220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.143+0000 7f938da3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b6387840 0x55a1b6733760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.143+0000 7f938da3a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b14e0 con 0x55a1b635e980 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.143+0000 7f938da3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b30e0 con 0x55a1b6387840 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.143+0000 7f938da3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b1bb0 con 0x55a1b6366b60 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.143+0000 7f937fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b635e980 0x55a1b67383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.144+0000 7f937fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b635e980 0x55a1b67383b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53698/0 (socket says 172.21.15.32:53698) 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.144+0000 7f937fe00640 1 -- 172.21.15.32:0/2440761372 learned_addr learned my addr 172.21.15.32:0/2440761372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.144+0000 7f937f400640 1 --2- 172.21.15.32:0/2440761372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b6733220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.144+0000 7f9384e00640 1 --2- 172.21.15.32:0/2440761372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b6387840 0x55a1b6733760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.144+0000 7f937fe00640 1 -- 172.21.15.32:0/2440761372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 msgr2=0x55a1b6733220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.144+0000 7f937fe00640 1 --2- 172.21.15.32:0/2440761372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b6733220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.144+0000 7f937fe00640 1 -- 172.21.15.32:0/2440761372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b6387840 msgr2=0x55a1b6733760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:21.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.144+0000 7f937fe00640 1 --2- 172.21.15.32:0/2440761372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b6387840 0x55a1b6733760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.144+0000 7f937fe00640 1 -- 172.21.15.32:0/2440761372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b673a060 con 0x55a1b635e980 2024-08-31T07:16:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.144+0000 7f937fe00640 1 --2- 172.21.15.32:0/2440761372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b635e980 0x55a1b67383b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9370009900 tx=0x7f937003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75789058a6ed977 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.145+0000 7f9374c00640 1 -- 172.21.15.32:0/2440761372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370035040 con 0x55a1b635e980 2024-08-31T07:16:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.145+0000 7f9374c00640 1 -- 172.21.15.32:0/2440761372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9370004c20 con 0x55a1b635e980 2024-08-31T07:16:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.145+0000 7f9374c00640 1 -- 172.21.15.32:0/2440761372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937003db10 con 0x55a1b635e980 2024-08-31T07:16:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.145+0000 7f938da3a740 1 -- 172.21.15.32:0/2440761372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b635e980 msgr2=0x55a1b67383b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.145+0000 7f938da3a740 1 --2- 172.21.15.32:0/2440761372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b635e980 0x55a1b67383b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9370009900 tx=0x7f937003f000 comp rx=0 tx=0).stop 2024-08-31T07:16:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.145+0000 7f938da3a740 1 -- 172.21.15.32:0/2440761372 shutdown_connections 2024-08-31T07:16:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.145+0000 7f938da3a740 1 --2- 172.21.15.32:0/2440761372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b6387840 0x55a1b6733760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.145+0000 7f938da3a740 1 --2- 172.21.15.32:0/2440761372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b6733220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.145+0000 7f938da3a740 1 --2- 172.21.15.32:0/2440761372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b635e980 0x55a1b67383b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.145+0000 7f938da3a740 1 -- 172.21.15.32:0/2440761372 >> 172.21.15.32:0/2440761372 conn(0x55a1b637b990 msgr2=0x55a1b6734ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.146+0000 7f938da3a740 1 -- 172.21.15.32:0/2440761372 shutdown_connections 2024-08-31T07:16:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.146+0000 7f938da3a740 1 -- 172.21.15.32:0/2440761372 wait complete. 2024-08-31T07:16:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.146+0000 7f938da3a740 1 -- start start 2024-08-31T07:16:21.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.146+0000 7f938da3a740 1 Processor -- start 2024-08-31T07:16:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f938da3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 0x55a1b673d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f938da3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b673d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f938da3a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b67b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f938da3a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b2950 con 0x55a1b6387840 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f938da3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b2280 con 0x55a1b635e980 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f938da3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93801af4c0 con 0x55a1b6366b60 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f938da3a740 10 client.0 _ll_register_callbacks cb 0x7f93780a7fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f937fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 0x55a1b673d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f9384e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b67b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f937fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 0x55a1b673d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60810/0 (socket says 172.21.15.32:60810) 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f937fe00640 1 -- 172.21.15.32:0/3488448431 learned_addr learned my addr 172.21.15.32:0/3488448431 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.147+0000 7f937f400640 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b673d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f9384e00640 1 -- 172.21.15.32:0/3488448431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 msgr2=0x55a1b673d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f9384e00640 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b673d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f9384e00640 1 -- 172.21.15.32:0/3488448431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 msgr2=0x55a1b673d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f9384e00640 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 0x55a1b673d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f9384e00640 1 -- 172.21.15.32:0/3488448431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b67b6f80 con 0x55a1b6387840 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f937fe00640 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 0x55a1b673d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f9384e00640 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b67b6a40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f93780ac040 tx=0x7f937800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38deb5686760fdf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f936a000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f936a000640 1 -- 172.21.15.32:0/3488448431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93780ac800 con 0x55a1b6387840 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f936a000640 1 -- 172.21.15.32:0/3488448431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f93780a7bb0 con 0x55a1b6387840 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f938da3a740 1 -- 172.21.15.32:0/3488448431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9370004c20 con 0x55a1b6387840 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.148+0000 7f936a000640 1 -- 172.21.15.32:0/3488448431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f93780a7db0 con 0x55a1b6387840 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f938da3a740 10 client.16293 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f938da3a740 1 -- 172.21.15.32:0/3488448431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a1b67b80e0 con 0x55a1b6387840 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f936a000640 1 -- 172.21.15.32:0/3488448431 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9370004c20 con 0x55a1b6387840 2024-08-31T07:16:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f936a000640 1 -- 172.21.15.32:0/3488448431 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f93780971c0 con 0x55a1b6387840 2024-08-31T07:16:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f938da3a740 10 client.16293 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f938da3a740 10 client.16293 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:16:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f938da3a740 20 client.16293 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f938da3a740 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a1b631b570 0x55a1b67ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f938da3a740 4 client.16293 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}] 2024-08-31T07:16:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f938da3a740 1 -- 172.21.15.32:0/3488448431 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}) -- 0x55a1b66a0f20 con 0x55a1b631b570 2024-08-31T07:16:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.149+0000 7f937fe00640 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a1b631b570 0x55a1b67ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.150+0000 7f937fe00640 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a1b631b570 0x55a1b67ba940 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927baf56227c12a8 server_cookie=84ab896ca0ea927e in_seq=0 out_seq=0 2024-08-31T07:16:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.150+0000 7f936a000640 10 client.16293 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.224+0000 7f936a000640 1 -- 172.21.15.32:0/3488448431 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a1b66a0f20 con 0x55a1b631b570 2024-08-31T07:16:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.224+0000 7f936a000640 10 client.16293 handle_command_reply: tid=0 2024-08-31T07:16:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.224+0000 7f938da3a740 1 client.16293 shutdown 2024-08-31T07:16:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 1 -- 172.21.15.32:0/3488448431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 msgr2=0x55a1b67b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b67b6a40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f93780ac040 tx=0x7f937800cee0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 1 -- 172.21.15.32:0/3488448431 shutdown_connections 2024-08-31T07:16:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a1b631b570 0x55a1b67ba940 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b6387840 0x55a1b67b6a40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a1b6366b60 0x55a1b673d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 1 --2- 172.21.15.32:0/3488448431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1b635e980 0x55a1b673d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 1 -- 172.21.15.32:0/3488448431 >> 172.21.15.32:0/3488448431 conn(0x55a1b637b990 msgr2=0x55a1b6734ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:21.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f936a000640 0 client.16293 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:21.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 1 -- 172.21.15.32:0/3488448431 shutdown_connections 2024-08-31T07:16:21.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 1 -- 172.21.15.32:0/3488448431 wait complete. 2024-08-31T07:16:21.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.225+0000 7f938da3a740 20 client.16293 trim_cache size 0 max 16384 2024-08-31T07:16:21.227 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 31, 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 14, 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "9fad7fc3": { 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 14, 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 141.8, 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:21.229 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:16:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.226+0000 7f9386e00640 1 -- 172.21.15.32:0/542731360 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93640b4d20 msgr2=0x7f93640b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.226+0000 7f9386e00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93640b4d20 0x7f93640b7230 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9370031440 tx=0x7f9370032ed0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.226+0000 7f9386e00640 1 -- 172.21.15.32:0/542731360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9380109c20 msgr2=0x7f93801a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.226+0000 7f9386e00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9380109c20 0x7f93801a1d10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f937800eb10 tx=0x7f937800c2d0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.227+0000 7f9386e00640 1 -- 172.21.15.32:0/542731360 shutdown_connections 2024-08-31T07:16:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.227+0000 7f9386e00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93640b4d20 0x7f93640b7230 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.227+0000 7f9386e00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9380109c20 0x7f93801a1d10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.227+0000 7f9386e00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93801092d0 0x7f938019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.227+0000 7f9386e00640 1 --2- 172.21.15.32:0/542731360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9380103990 0x7f938019c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:21.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.227+0000 7f9386e00640 1 -- 172.21.15.32:0/542731360 >> 172.21.15.32:0/542731360 conn(0x7f938007b680 msgr2=0x7f9380107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:21.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.228+0000 7f9386e00640 1 -- 172.21.15.32:0/542731360 shutdown_connections 2024-08-31T07:16:21.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:21.228+0000 7f9386e00640 1 -- 172.21.15.32:0/542731360 wait complete. 2024-08-31T07:16:21.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 1.93 MB/sec execute 10 sec latency 1058.851 ms 2024-08-31T07:16:22.151 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9fad7fc3', quiesce took 4.6 seconds. Will release after: 17.200000000000003 2024-08-31T07:16:22.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:22 smithi032 ceph-mon[17252]: pgmap v476: 97 pgs: 97 active+clean; 65 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 224 KiB/s wr, 43 op/s 2024-08-31T07:16:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:22 smithi073 ceph-mon[21995]: pgmap v476: 97 pgs: 97 active+clean; 65 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 224 KiB/s wr, 43 op/s 2024-08-31T07:16:22.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 1.75 MB/sec execute 11 sec latency 1466.862 ms 2024-08-31T07:16:22.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:22 smithi084 ceph-mon[22387]: pgmap v476: 97 pgs: 97 active+clean; 65 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 224 KiB/s wr, 43 op/s 2024-08-31T07:16:23.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 1.61 MB/sec execute 12 sec latency 2467.020 ms 2024-08-31T07:16:24.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:24 smithi032 ceph-mon[17252]: pgmap v477: 97 pgs: 97 active+clean; 66 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 304 KiB/s wr, 55 op/s 2024-08-31T07:16:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:24 smithi073 ceph-mon[21995]: pgmap v477: 97 pgs: 97 active+clean; 66 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 304 KiB/s wr, 55 op/s 2024-08-31T07:16:24.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 1.48 MB/sec execute 13 sec latency 3467.158 ms 2024-08-31T07:16:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:24 smithi084 ceph-mon[22387]: pgmap v477: 97 pgs: 97 active+clean; 66 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 304 KiB/s wr, 55 op/s 2024-08-31T07:16:25.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 1.38 MB/sec execute 14 sec latency 4467.305 ms 2024-08-31T07:16:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:26 smithi032 ceph-mon[17252]: pgmap v478: 97 pgs: 97 active+clean; 66 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 487 KiB/s wr, 60 op/s 2024-08-31T07:16:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:26 smithi073 ceph-mon[21995]: pgmap v478: 97 pgs: 97 active+clean; 66 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 487 KiB/s wr, 60 op/s 2024-08-31T07:16:26.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 1.29 MB/sec execute 15 sec latency 5467.380 ms 2024-08-31T07:16:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:26 smithi084 ceph-mon[22387]: pgmap v478: 97 pgs: 97 active+clean; 66 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 487 KiB/s wr, 60 op/s 2024-08-31T07:16:27.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:16:27.671 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:27.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 1.20 MB/sec execute 16 sec latency 6467.436 ms 2024-08-31T07:16:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.357+0000 7f7869e00640 1 -- 172.21.15.32:0/3465565547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f786410bdb0 msgr2=0x7f78641125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.357+0000 7f7869e00640 1 --2- 172.21.15.32:0/3465565547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f786410bdb0 0x7f78641125e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f785c009ca0 tx=0x7f785c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.357+0000 7f7869e00640 1 -- 172.21.15.32:0/3465565547 shutdown_connections 2024-08-31T07:16:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.357+0000 7f7869e00640 1 --2- 172.21.15.32:0/3465565547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f786410bdb0 0x7f78641125e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.357+0000 7f7869e00640 1 --2- 172.21.15.32:0/3465565547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f786410b460 0x7f786410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.357+0000 7f7869e00640 1 --2- 172.21.15.32:0/3465565547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7864105b20 0x7f7864105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.357+0000 7f7869e00640 1 -- 172.21.15.32:0/3465565547 >> 172.21.15.32:0/3465565547 conn(0x7f78641015b0 msgr2=0x7f7864103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.357+0000 7f7869e00640 1 -- 172.21.15.32:0/3465565547 shutdown_connections 2024-08-31T07:16:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.357+0000 7f7869e00640 1 -- 172.21.15.32:0/3465565547 wait complete. 2024-08-31T07:16:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.358+0000 7f7869e00640 1 Processor -- start 2024-08-31T07:16:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7869e00640 1 -- start start 2024-08-31T07:16:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7869e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7864105b20 0x7f786419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7869e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f786410b460 0x7f786419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7869e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f786410bdb0 0x7f78641a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7869e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7864115a70 con 0x7f7864105b20 2024-08-31T07:16:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7869e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f78641158f0 con 0x7f786410bdb0 2024-08-31T07:16:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7869e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7864115bf0 con 0x7f786410b460 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7862a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7864105b20 0x7f786419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7862a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7864105b20 0x7f786419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53398/0 (socket says 172.21.15.32:53398) 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7862a00640 1 -- 172.21.15.32:0/3901145493 learned_addr learned my addr 172.21.15.32:0/3901145493 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f785be00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f786410b460 0x7f786419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.359+0000 7f7863400640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f786410bdb0 0x7f78641a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.360+0000 7f7862a00640 1 -- 172.21.15.32:0/3901145493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f786410b460 msgr2=0x7f786419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.360+0000 7f7862a00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f786410b460 0x7f786419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.360+0000 7f7862a00640 1 -- 172.21.15.32:0/3901145493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f786410bdb0 msgr2=0x7f78641a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.360+0000 7f7862a00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f786410bdb0 0x7f78641a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.360+0000 7f7862a00640 1 -- 172.21.15.32:0/3901145493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78641a2430 con 0x7f7864105b20 2024-08-31T07:16:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.360+0000 7f7862a00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7864105b20 0x7f786419c4c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f785400e9f0 tx=0x7f785400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.360+0000 7f7868a00640 1 -- 172.21.15.32:0/3901145493 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785400cd80 con 0x7f7864105b20 2024-08-31T07:16:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.360+0000 7f7868a00640 1 -- 172.21.15.32:0/3901145493 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78540108e0 con 0x7f7864105b20 2024-08-31T07:16:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.360+0000 7f7869e00640 1 -- 172.21.15.32:0/3901145493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78641a2720 con 0x7f7864105b20 2024-08-31T07:16:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.361+0000 7f7868a00640 1 -- 172.21.15.32:0/3901145493 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854018980 con 0x7f7864105b20 2024-08-31T07:16:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.361+0000 7f7869e00640 1 -- 172.21.15.32:0/3901145493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78641a8e80 con 0x7f7864105b20 2024-08-31T07:16:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.364+0000 7f7870a05740 1 Processor -- start 2024-08-31T07:16:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.364+0000 7f7870a05740 1 -- start start 2024-08-31T07:16:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.364+0000 7f7870a05740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e57070 0x5624191fc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.364+0000 7f7868a00640 1 -- 172.21.15.32:0/3901145493 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7854010430 con 0x7f7864105b20 2024-08-31T07:16:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.364+0000 7f7870a05740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 0x5624191fce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.364+0000 7f7870a05740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e36390 0x5624191fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7870a05740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562419173770 con 0x562418e57070 2024-08-31T07:16:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7870a05740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562419162d00 con 0x562418e2e1b0 2024-08-31T07:16:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7870a05740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562419166cb0 con 0x562418e36390 2024-08-31T07:16:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7863400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e57070 0x5624191fc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7863400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e57070 0x5624191fc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53414/0 (socket says 172.21.15.32:53414) 2024-08-31T07:16:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7863400640 1 -- 172.21.15.32:0/2535445039 learned_addr learned my addr 172.21.15.32:0/2535445039 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7862a00640 1 --2- 172.21.15.32:0/2535445039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 0x5624191fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f785be00640 1 --2- 172.21.15.32:0/2535445039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e36390 0x5624191fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7863400640 1 -- 172.21.15.32:0/2535445039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e36390 msgr2=0x5624191fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7863400640 1 --2- 172.21.15.32:0/2535445039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e36390 0x5624191fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7863400640 1 -- 172.21.15.32:0/2535445039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 msgr2=0x5624191fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7863400640 1 --2- 172.21.15.32:0/2535445039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 0x5624191fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7863400640 1 -- 172.21.15.32:0/2535445039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562418ec2650 con 0x562418e57070 2024-08-31T07:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.365+0000 7f7862a00640 1 --2- 172.21.15.32:0/2535445039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 0x5624191fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.366+0000 7f7863400640 1 --2- 172.21.15.32:0/2535445039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e57070 0x5624191fc950 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f785c02f410 tx=0x7f785c02f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f22f2b3a24e0eb0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.366+0000 7f7853e00640 1 -- 172.21.15.32:0/2535445039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c031cf0 con 0x562418e57070 2024-08-31T07:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.366+0000 7f7853e00640 1 -- 172.21.15.32:0/2535445039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785c036030 con 0x562418e57070 2024-08-31T07:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.366+0000 7f7853e00640 1 -- 172.21.15.32:0/2535445039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c0042a0 con 0x562418e57070 2024-08-31T07:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.366+0000 7f7868a00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f78480b4d20 0x7f78480b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7868a00640 1 -- 172.21.15.32:0/3901145493 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f78540e5750 con 0x7f7864105b20 2024-08-31T07:16:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7862a00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f78480b4d20 0x7f78480b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7870a05740 1 -- 172.21.15.32:0/2535445039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e57070 msgr2=0x5624191fc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7870a05740 1 --2- 172.21.15.32:0/2535445039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e57070 0x5624191fc950 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f785c02f410 tx=0x7f785c02f440 comp rx=0 tx=0).stop 2024-08-31T07:16:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7870a05740 1 -- 172.21.15.32:0/2535445039 shutdown_connections 2024-08-31T07:16:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7870a05740 1 --2- 172.21.15.32:0/2535445039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e36390 0x5624191fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7870a05740 1 --2- 172.21.15.32:0/2535445039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 0x5624191fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7870a05740 1 --2- 172.21.15.32:0/2535445039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e57070 0x5624191fc950 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7870a05740 1 -- 172.21.15.32:0/2535445039 >> 172.21.15.32:0/2535445039 conn(0x562418e4b460 msgr2=0x5624191fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7870a05740 1 -- 172.21.15.32:0/2535445039 shutdown_connections 2024-08-31T07:16:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7870a05740 1 -- 172.21.15.32:0/2535445039 wait complete. 2024-08-31T07:16:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.367+0000 7f7862a00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f78480b4d20 0x7f78480b7230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f78540e2fd0 tx=0x7f7854000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.368+0000 7f7870a05740 1 -- start start 2024-08-31T07:16:28.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.368+0000 7f7870a05740 1 Processor -- start 2024-08-31T07:16:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7870a05740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x562419208590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7870a05740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e36390 0x562419208af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7870a05740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e57070 0x562419214240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7870a05740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562418fc5700 con 0x562418e2e1b0 2024-08-31T07:16:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7870a05740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5624190e9cb0 con 0x562418e36390 2024-08-31T07:16:28.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7870a05740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56241920af50 con 0x562418e57070 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7870a05740 10 client.0 _ll_register_callbacks cb 0x56241910f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7863400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x562419208590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7863400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x562419208590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53428/0 (socket says 172.21.15.32:53428) 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7863400640 1 -- 172.21.15.32:0/97367704 learned_addr learned my addr 172.21.15.32:0/97367704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f785be00640 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e36390 0x562419208af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.369+0000 7f7863400640 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e57070 0x562419214240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.370+0000 7f7863400640 1 -- 172.21.15.32:0/97367704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e57070 msgr2=0x562419214240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.370+0000 7f7863400640 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e57070 0x562419214240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.370+0000 7f7863400640 1 -- 172.21.15.32:0/97367704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e36390 msgr2=0x562419208af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.370+0000 7f7863400640 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e36390 0x562419208af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.370+0000 7f7863400640 1 -- 172.21.15.32:0/97367704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241912a870 con 0x562418e2e1b0 2024-08-31T07:16:28.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.370+0000 7f7863400640 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x562419208590 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f785c005c90 tx=0x7f785c003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c4b5126b8846eb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.370+0000 7f7850c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.370+0000 7f7850c00640 1 -- 172.21.15.32:0/97367704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c058060 con 0x562418e2e1b0 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.370+0000 7f7850c00640 1 -- 172.21.15.32:0/97367704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f785c031860 con 0x562418e2e1b0 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.370+0000 7f7870a05740 1 -- 172.21.15.32:0/97367704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f785c036030 con 0x562418e2e1b0 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7850c00640 1 -- 172.21.15.32:0/97367704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f785c02f660 con 0x562418e2e1b0 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7870a05740 10 client.16305 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7870a05740 1 -- 172.21.15.32:0/97367704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562419216f90 con 0x562418e2e1b0 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7850c00640 1 -- 172.21.15.32:0/97367704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785c036030 con 0x562418e2e1b0 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7850c00640 1 -- 172.21.15.32:0/97367704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f785c043070 con 0x562418e2e1b0 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7870a05740 10 client.16305 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7870a05740 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:16:28.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7870a05740 20 client.16305 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:28.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7870a05740 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562418e159f0 0x56241921a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7870a05740 4 client.16305 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:28.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.371+0000 7f7870a05740 1 -- 172.21.15.32:0/97367704 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624191756c0 con 0x562418e159f0 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.372+0000 7f785be00640 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562418e159f0 0x56241921a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.372+0000 7f785be00640 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562418e159f0 0x56241921a2f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88274ba17711d863 server_cookie=537983bfcf8cb1b9 in_seq=0 out_seq=0 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.372+0000 7f7850c00640 10 client.16305 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.373+0000 7f7850c00640 1 -- 172.21.15.32:0/97367704 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5624191756c0 con 0x562418e159f0 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.373+0000 7f7850c00640 10 client.16305 handle_command_reply: tid=0 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.373+0000 7f7870a05740 1 client.16305 shutdown 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.373+0000 7f7870a05740 1 -- 172.21.15.32:0/97367704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 msgr2=0x562419208590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.373+0000 7f7870a05740 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x562419208590 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f785c005c90 tx=0x7f785c003d70 comp rx=0 tx=0).stop 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.374+0000 7f7870a05740 1 -- 172.21.15.32:0/97367704 shutdown_connections 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.374+0000 7f7870a05740 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562418e159f0 0x56241921a2f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.374+0000 7f7870a05740 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e57070 0x562419214240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.374+0000 7f7870a05740 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e36390 0x562419208af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.374+0000 7f7850c00640 0 client.16305 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.374+0000 7f7870a05740 1 --2- 172.21.15.32:0/97367704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x562419208590 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.374+0000 7f7870a05740 1 -- 172.21.15.32:0/97367704 >> 172.21.15.32:0/97367704 conn(0x562418e4b460 msgr2=0x5624191ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.374+0000 7f7870a05740 1 -- 172.21.15.32:0/97367704 shutdown_connections 2024-08-31T07:16:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.374+0000 7f7870a05740 1 -- 172.21.15.32:0/97367704 wait complete. 2024-08-31T07:16:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.374+0000 7f7870a05740 20 client.16305 trim_cache size 0 max 16384 2024-08-31T07:16:28.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.392+0000 7f7870a05740 1 Processor -- start 2024-08-31T07:16:28.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.392+0000 7f7870a05740 1 -- start start 2024-08-31T07:16:28.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f7870a05740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 0x562419214140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f7870a05740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e36390 0x562419206c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f7870a05740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e57070 0x562419207150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f7870a05740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78641b14b0 con 0x562418e36390 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f7870a05740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f78641b30b0 con 0x562418e2e1b0 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f7870a05740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f78641b1b80 con 0x562418e57070 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f785be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e36390 0x562419206c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f785be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e36390 0x562419206c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53432/0 (socket says 172.21.15.32:53432) 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f785be00640 1 -- 172.21.15.32:0/1136966834 learned_addr learned my addr 172.21.15.32:0/1136966834 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f7863400640 1 --2- 172.21.15.32:0/1136966834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 0x562419214140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.393+0000 7f7863400640 1 --2- 172.21.15.32:0/1136966834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e57070 0x562419207150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.394+0000 7f785be00640 1 -- 172.21.15.32:0/1136966834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e57070 msgr2=0x562419207150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.394+0000 7f785be00640 1 --2- 172.21.15.32:0/1136966834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e57070 0x562419207150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.394+0000 7f785be00640 1 -- 172.21.15.32:0/1136966834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 msgr2=0x562419214140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.394+0000 7f785be00640 1 --2- 172.21.15.32:0/1136966834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 0x562419214140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.394+0000 7f785be00640 1 -- 172.21.15.32:0/1136966834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624191ff1e0 con 0x562418e36390 2024-08-31T07:16:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.394+0000 7f785be00640 1 --2- 172.21.15.32:0/1136966834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e36390 0x562419206c10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f786406c5b0 tx=0x7f784c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cb7e01044e3c74 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.394+0000 7f7853400640 1 -- 172.21.15.32:0/1136966834 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c005220 con 0x562418e36390 2024-08-31T07:16:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.394+0000 7f7853400640 1 -- 172.21.15.32:0/1136966834 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784c004c50 con 0x562418e36390 2024-08-31T07:16:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.394+0000 7f7853400640 1 -- 172.21.15.32:0/1136966834 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c01e860 con 0x562418e36390 2024-08-31T07:16:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.395+0000 7f7870a05740 1 -- 172.21.15.32:0/1136966834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e36390 msgr2=0x562419206c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.395+0000 7f7870a05740 1 --2- 172.21.15.32:0/1136966834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e36390 0x562419206c10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f786406c5b0 tx=0x7f784c005f90 comp rx=0 tx=0).stop 2024-08-31T07:16:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.395+0000 7f7870a05740 1 -- 172.21.15.32:0/1136966834 shutdown_connections 2024-08-31T07:16:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.395+0000 7f7870a05740 1 --2- 172.21.15.32:0/1136966834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418e57070 0x562419207150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.395+0000 7f7870a05740 1 --2- 172.21.15.32:0/1136966834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e36390 0x562419206c10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.395+0000 7f7870a05740 1 --2- 172.21.15.32:0/1136966834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e2e1b0 0x562419214140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.395+0000 7f7870a05740 1 -- 172.21.15.32:0/1136966834 >> 172.21.15.32:0/1136966834 conn(0x562418e4b460 msgr2=0x5624192034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:28.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.395+0000 7f7870a05740 1 -- 172.21.15.32:0/1136966834 shutdown_connections 2024-08-31T07:16:28.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.395+0000 7f7870a05740 1 -- 172.21.15.32:0/1136966834 wait complete. 2024-08-31T07:16:28.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.396+0000 7f7870a05740 1 -- start start 2024-08-31T07:16:28.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.396+0000 7f7870a05740 1 Processor -- start 2024-08-31T07:16:28.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7870a05740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x56241920c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7870a05740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e57070 0x56241920ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7870a05740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418deada0 0x562419283b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7870a05740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78641b2920 con 0x562418e2e1b0 2024-08-31T07:16:28.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7870a05740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f78641b2250 con 0x562418e57070 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7870a05740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f78641af490 con 0x562418deada0 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7870a05740 10 client.0 _ll_register_callbacks cb 0x7f785c009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7863400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x56241920c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7863400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x56241920c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53446/0 (socket says 172.21.15.32:53446) 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7863400640 1 -- 172.21.15.32:0/2997897764 learned_addr learned my addr 172.21.15.32:0/2997897764 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f785be00640 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e57070 0x56241920ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7863400640 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418deada0 0x562419283b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7863400640 1 -- 172.21.15.32:0/2997897764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418deada0 msgr2=0x562419283b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7863400640 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418deada0 0x562419283b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7863400640 1 -- 172.21.15.32:0/2997897764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e57070 msgr2=0x56241920ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.397+0000 7f7863400640 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e57070 0x56241920ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.398+0000 7f7863400640 1 -- 172.21.15.32:0/2997897764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624192840c0 con 0x562418e2e1b0 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.398+0000 7f785be00640 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e57070 0x56241920ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:28.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.398+0000 7f7863400640 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x56241920c8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f785c006010 tx=0x7f785c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224b20b4db747826 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.398+0000 7f7850c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.398+0000 7f7850c00640 1 -- 172.21.15.32:0/2997897764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c058040 con 0x562418e2e1b0 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.398+0000 7f7850c00640 1 -- 172.21.15.32:0/2997897764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f785c02f800 con 0x562418e2e1b0 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.398+0000 7f7870a05740 1 -- 172.21.15.32:0/2997897764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f784c004c50 con 0x562418e2e1b0 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.398+0000 7f7850c00640 1 -- 172.21.15.32:0/2997897764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f785c031d70 con 0x562418e2e1b0 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.398+0000 7f7870a05740 10 client.16311 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.398+0000 7f7870a05740 1 -- 172.21.15.32:0/2997897764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5624192852c0 con 0x562418e2e1b0 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.399+0000 7f7850c00640 1 -- 172.21.15.32:0/2997897764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784c004c50 con 0x562418e2e1b0 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.399+0000 7f7850c00640 1 -- 172.21.15.32:0/2997897764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f785c065020 con 0x562418e2e1b0 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.399+0000 7f7870a05740 10 client.16311 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.399+0000 7f7870a05740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.399+0000 7f7870a05740 20 client.16311 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.399+0000 7f7870a05740 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562419285950 0x562419287d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.399+0000 7f7870a05740 4 client.16311 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:16:28.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.399+0000 7f7870a05740 1 -- 172.21.15.32:0/2997897764 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562419171160 con 0x562419285950 2024-08-31T07:16:28.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.399+0000 7f785be00640 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562419285950 0x562419287d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.400+0000 7f785be00640 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562419285950 0x562419287d70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe66490fe0f825b6 server_cookie=d547f2b077ffe3c8 in_seq=0 out_seq=0 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.400+0000 7f7850c00640 10 client.16311 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.400+0000 7f7850c00640 1 -- 172.21.15.32:0/2997897764 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x562419171160 con 0x562419285950 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.400+0000 7f7850c00640 10 client.16311 handle_command_reply: tid=0 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.400+0000 7f7870a05740 1 client.16311 shutdown 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.400+0000 7f7870a05740 1 -- 172.21.15.32:0/2997897764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 msgr2=0x56241920c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7870a05740 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x56241920c8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f785c006010 tx=0x7f785c003cd0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7870a05740 1 -- 172.21.15.32:0/2997897764 shutdown_connections 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7870a05740 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562419285950 0x562419287d70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7870a05740 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562418deada0 0x562419283b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7870a05740 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562418e57070 0x56241920ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7870a05740 1 --2- 172.21.15.32:0/2997897764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562418e2e1b0 0x56241920c8d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7850c00640 0 client.16311 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:28.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7870a05740 1 -- 172.21.15.32:0/2997897764 >> 172.21.15.32:0/2997897764 conn(0x562418e4b460 msgr2=0x562419201a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:28.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7870a05740 1 -- 172.21.15.32:0/2997897764 shutdown_connections 2024-08-31T07:16:28.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7870a05740 1 -- 172.21.15.32:0/2997897764 wait complete. 2024-08-31T07:16:28.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.401+0000 7f7870a05740 20 client.16311 trim_cache size 0 max 16384 2024-08-31T07:16:28.406 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:16:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:16:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:16:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:16:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:16:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:16:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:16:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:28.410 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:28.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:28.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:28.410 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:16:28.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 -- 172.21.15.32:0/3901145493 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f78480b4d20 msgr2=0x7f78480b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:28.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f78480b4d20 0x7f78480b7230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f78540e2fd0 tx=0x7f7854000bc0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 -- 172.21.15.32:0/3901145493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7864105b20 msgr2=0x7f786419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7864105b20 0x7f786419c4c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f785400e9f0 tx=0x7f785400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 -- 172.21.15.32:0/3901145493 shutdown_connections 2024-08-31T07:16:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f78480b4d20 0x7f78480b7230 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f786410bdb0 0x7f78641a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f786410b460 0x7f786419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 --2- 172.21.15.32:0/3901145493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7864105b20 0x7f786419c4c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 -- 172.21.15.32:0/3901145493 >> 172.21.15.32:0/3901145493 conn(0x7f78641015b0 msgr2=0x7f786410aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 -- 172.21.15.32:0/3901145493 shutdown_connections 2024-08-31T07:16:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:28.402+0000 7f7869e00640 1 -- 172.21.15.32:0/3901145493 wait complete. 2024-08-31T07:16:28.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:28 smithi032 ceph-mon[17252]: pgmap v479: 97 pgs: 97 active+clean; 67 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 596 KiB/s wr, 71 op/s 2024-08-31T07:16:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:28 smithi073 ceph-mon[21995]: pgmap v479: 97 pgs: 97 active+clean; 67 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 596 KiB/s wr, 71 op/s 2024-08-31T07:16:28.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 1.13 MB/sec execute 17 sec latency 7467.507 ms 2024-08-31T07:16:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:28 smithi084 ceph-mon[22387]: pgmap v479: 97 pgs: 97 active+clean; 67 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 596 KiB/s wr, 71 op/s 2024-08-31T07:16:29.288 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:16:29.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:16:29.669 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:29.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 1.07 MB/sec execute 18 sec latency 8467.566 ms 2024-08-31T07:16:30.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.313+0000 7f9189800640 1 -- 172.21.15.32:0/387340447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9184105b20 msgr2=0x7f9184105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.313+0000 7f9189800640 1 --2- 172.21.15.32:0/387340447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9184105b20 0x7f9184105f30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9174009900 tx=0x7f917402a410 comp rx=0 tx=0).stop 2024-08-31T07:16:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.313+0000 7f9189800640 1 -- 172.21.15.32:0/387340447 shutdown_connections 2024-08-31T07:16:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.313+0000 7f9189800640 1 --2- 172.21.15.32:0/387340447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f918410bdb0 0x7f91841125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.313+0000 7f9189800640 1 --2- 172.21.15.32:0/387340447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f918410b460 0x7f918410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.313+0000 7f9189800640 1 --2- 172.21.15.32:0/387340447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9184105b20 0x7f9184105f30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.313+0000 7f9189800640 1 -- 172.21.15.32:0/387340447 >> 172.21.15.32:0/387340447 conn(0x7f91841015b0 msgr2=0x7f9184103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.313+0000 7f9189800640 1 -- 172.21.15.32:0/387340447 shutdown_connections 2024-08-31T07:16:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.313+0000 7f9189800640 1 -- 172.21.15.32:0/387340447 wait complete. 2024-08-31T07:16:30.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.314+0000 7f9189800640 1 Processor -- start 2024-08-31T07:16:30.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9189800640 1 -- start start 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9189800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9184105b20 0x7f918419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9189800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f918410b460 0x7f918419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9189800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f918410bdb0 0x7f91841a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9189800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9184115ac0 con 0x7f9184105b20 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9189800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9184115940 con 0x7f918410bdb0 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9189800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9184115c40 con 0x7f918410b460 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9182a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9184105b20 0x7f918419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9182000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f918410b460 0x7f918419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9183400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f918410bdb0 0x7f91841a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9182a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9184105b20 0x7f918419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53454/0 (socket says 172.21.15.32:53454) 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.315+0000 7f9182a00640 1 -- 172.21.15.32:0/2624973562 learned_addr learned my addr 172.21.15.32:0/2624973562 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.316+0000 7f9182000640 1 -- 172.21.15.32:0/2624973562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f918410bdb0 msgr2=0x7f91841a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.316+0000 7f9182000640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f918410bdb0 0x7f91841a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.316+0000 7f9182000640 1 -- 172.21.15.32:0/2624973562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9184105b20 msgr2=0x7f918419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.316+0000 7f9182000640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9184105b20 0x7f918419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.316+0000 7f9182000640 1 -- 172.21.15.32:0/2624973562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91841a2440 con 0x7f918410b460 2024-08-31T07:16:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.316+0000 7f9182a00640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9184105b20 0x7f918419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.316+0000 7f9182000640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f918410b460 0x7f918419ca70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f918406c840 tx=0x7f917000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.316+0000 7f917b400640 1 -- 172.21.15.32:0/2624973562 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917000cc40 con 0x7f918410b460 2024-08-31T07:16:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.316+0000 7f9189800640 1 -- 172.21.15.32:0/2624973562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91841a2730 con 0x7f918410b460 2024-08-31T07:16:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.317+0000 7f917b400640 1 -- 172.21.15.32:0/2624973562 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f917000cde0 con 0x7f918410b460 2024-08-31T07:16:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.317+0000 7f917b400640 1 -- 172.21.15.32:0/2624973562 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170004a00 con 0x7f918410b460 2024-08-31T07:16:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.317+0000 7f9189800640 1 -- 172.21.15.32:0/2624973562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91841a8e90 con 0x7f918410b460 2024-08-31T07:16:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.318+0000 7f9189800640 1 -- 172.21.15.32:0/2624973562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9184105f30 con 0x7f918410b460 2024-08-31T07:16:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.321+0000 7f917b400640 1 -- 172.21.15.32:0/2624973562 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9170004ba0 con 0x7f918410b460 2024-08-31T07:16:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.323+0000 7f917b400640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91680b4d20 0x7f91680b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.323+0000 7f917b400640 1 -- 172.21.15.32:0/2624973562 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f91700e5490 con 0x7f918410b460 2024-08-31T07:16:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.323+0000 7f9182a00640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91680b4d20 0x7f91680b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.324+0000 7f917b400640 1 -- 172.21.15.32:0/2624973562 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f91700ac330 con 0x7f918410b460 2024-08-31T07:16:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.324+0000 7f9182a00640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91680b4d20 0x7f91680b7230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f91740096c0 tx=0x7f9174005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.558+0000 7f9189800640 1 -- 172.21.15.32:0/2624973562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f918419d600 con 0x7f918410b460 2024-08-31T07:16:30.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:30 smithi032 ceph-mon[17252]: pgmap v480: 97 pgs: 97 active+clean; 67 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 583 KiB/s wr, 69 op/s 2024-08-31T07:16:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.559+0000 7f917b400640 1 -- 172.21.15.32:0/2624973562 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11967 (secure 0 0 0) 0x7f91700abc30 con 0x7f918410b460 2024-08-31T07:16:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:16:30.562 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"btime":"2024-08-31T07:14:36:074596+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:14:35.939633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:16:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2024-08-31T07:16:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.562+0000 7f9189800640 1 -- 172.21.15.32:0/2624973562 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91680b4d20 msgr2=0x7f91680b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.562+0000 7f9189800640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91680b4d20 0x7f91680b7230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f91740096c0 tx=0x7f9174005a20 comp rx=0 tx=0).stop 2024-08-31T07:16:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.562+0000 7f9189800640 1 -- 172.21.15.32:0/2624973562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f918410b460 msgr2=0x7f918419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.562+0000 7f9189800640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f918410b460 0x7f918419ca70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f918406c840 tx=0x7f917000ee50 comp rx=0 tx=0).stop 2024-08-31T07:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.562+0000 7f9189800640 1 -- 172.21.15.32:0/2624973562 shutdown_connections 2024-08-31T07:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.562+0000 7f9189800640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91680b4d20 0x7f91680b7230 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.562+0000 7f9189800640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f918410bdb0 0x7f91841a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.562+0000 7f9189800640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f918410b460 0x7f918419ca70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.562+0000 7f9189800640 1 --2- 172.21.15.32:0/2624973562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9184105b20 0x7f918419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.562+0000 7f9189800640 1 -- 172.21.15.32:0/2624973562 >> 172.21.15.32:0/2624973562 conn(0x7f91841015b0 msgr2=0x7f91841039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.563+0000 7f9189800640 1 -- 172.21.15.32:0/2624973562 shutdown_connections 2024-08-31T07:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:30.563+0000 7f9189800640 1 -- 172.21.15.32:0/2624973562 wait complete. 2024-08-31T07:16:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:30 smithi073 ceph-mon[21995]: pgmap v480: 97 pgs: 97 active+clean; 67 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 583 KiB/s wr, 69 op/s 2024-08-31T07:16:30.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 1.01 MB/sec execute 19 sec latency 9467.681 ms 2024-08-31T07:16:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:30 smithi084 ceph-mon[22387]: pgmap v480: 97 pgs: 97 active+clean; 67 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 583 KiB/s wr, 69 op/s 2024-08-31T07:16:31.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:16:31.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:31 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2624973562' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:31.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:31 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2624973562' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:31.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.96 MB/sec execute 20 sec latency 10467.723 ms 2024-08-31T07:16:31.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:31 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2624973562' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.475+0000 7f83d9e00640 1 -- 172.21.15.32:0/3398004303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d41099c0 msgr2=0x7f83d41101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.475+0000 7f83d9e00640 1 --2- 172.21.15.32:0/3398004303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d41099c0 0x7f83d41101f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009ca0 tx=0x7f83cc02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.475+0000 7f83d9e00640 1 -- 172.21.15.32:0/3398004303 shutdown_connections 2024-08-31T07:16:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.475+0000 7f83d9e00640 1 --2- 172.21.15.32:0/3398004303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d41099c0 0x7f83d41101f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.475+0000 7f83d9e00640 1 --2- 172.21.15.32:0/3398004303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83d4109070 0x7f83d4109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.475+0000 7f83d9e00640 1 --2- 172.21.15.32:0/3398004303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4103730 0x7f83d4103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.475+0000 7f83d9e00640 1 -- 172.21.15.32:0/3398004303 >> 172.21.15.32:0/3398004303 conn(0x7f83d40ff180 msgr2=0x7f83d41015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.475+0000 7f83d9e00640 1 -- 172.21.15.32:0/3398004303 shutdown_connections 2024-08-31T07:16:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.476+0000 7f83d9e00640 1 -- 172.21.15.32:0/3398004303 wait complete. 2024-08-31T07:16:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d9e00640 1 Processor -- start 2024-08-31T07:16:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d9e00640 1 -- start start 2024-08-31T07:16:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d4103730 0x7f83d419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4109070 0x7f83d419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83d41099c0 0x7f83d419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d9e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41136e0 con 0x7f83d4103730 2024-08-31T07:16:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d9e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4113560 con 0x7f83d4109070 2024-08-31T07:16:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d9e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4113860 con 0x7f83d41099c0 2024-08-31T07:16:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d4103730 0x7f83d419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d4103730 0x7f83d419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53480/0 (socket says 172.21.15.32:53480) 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d9400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83d41099c0 0x7f83d419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.477+0000 7f83d8a00640 1 -- 172.21.15.32:0/3992947926 learned_addr learned my addr 172.21.15.32:0/3992947926 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.478+0000 7f83d3e00640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4109070 0x7f83d419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.478+0000 7f83d9400640 1 -- 172.21.15.32:0/3992947926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4109070 msgr2=0x7f83d419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.478+0000 7f83d9400640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4109070 0x7f83d419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.478+0000 7f83d9400640 1 -- 172.21.15.32:0/3992947926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d4103730 msgr2=0x7f83d419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.478+0000 7f83d9400640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d4103730 0x7f83d419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.478+0000 7f83d9400640 1 -- 172.21.15.32:0/3992947926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d419ffc0 con 0x7f83d41099c0 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.478+0000 7f83d9400640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83d41099c0 0x7f83d419f890 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f83cc005e30 tx=0x7f83cc0317c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.479+0000 7f83d1600640 1 -- 172.21.15.32:0/3992947926 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc003b70 con 0x7f83d41099c0 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.479+0000 7f83d9e00640 1 -- 172.21.15.32:0/3992947926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d41a0250 con 0x7f83d41099c0 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.479+0000 7f83d1600640 1 -- 172.21.15.32:0/3992947926 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83cc003d10 con 0x7f83d41099c0 2024-08-31T07:16:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.479+0000 7f83d1600640 1 -- 172.21.15.32:0/3992947926 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc02f970 con 0x7f83d41099c0 2024-08-31T07:16:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.479+0000 7f83d9e00640 1 -- 172.21.15.32:0/3992947926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d41a6a10 con 0x7f83d41099c0 2024-08-31T07:16:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83e203d740 1 Processor -- start 2024-08-31T07:16:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83d1600640 1 -- 172.21.15.32:0/3992947926 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f83cc02fb10 con 0x7f83d41099c0 2024-08-31T07:16:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83e203d740 1 -- start start 2024-08-31T07:16:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83e203d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f54581b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83e203d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f545820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83e203d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541bb390 0x556f545825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83e203d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556f544f87b0 con 0x556f541b31b0 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83e203d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556f544e7d40 con 0x556f541dc070 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83e203d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556f544ebcf0 con 0x556f541bb390 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f54581b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f54581b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58824/0 (socket says 172.21.15.32:58824) 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.483+0000 7f83d3e00640 1 -- 172.21.15.32:0/2515940821 learned_addr learned my addr 172.21.15.32:0/2515940821 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.484+0000 7f83d3e00640 1 --2- 172.21.15.32:0/2515940821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541bb390 0x556f545825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.484+0000 7f83d3e00640 1 -- 172.21.15.32:0/2515940821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541bb390 msgr2=0x556f545825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.484+0000 7f83d3e00640 1 --2- 172.21.15.32:0/2515940821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541bb390 0x556f545825e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.484+0000 7f83d3e00640 1 -- 172.21.15.32:0/2515940821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 msgr2=0x556f545820a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.484+0000 7f83d3e00640 1 --2- 172.21.15.32:0/2515940821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f545820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.484+0000 7f83d3e00640 1 -- 172.21.15.32:0/2515940821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f54247650 con 0x556f541dc070 2024-08-31T07:16:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.484+0000 7f83d3e00640 1 --2- 172.21.15.32:0/2515940821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f54581b60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009fa0 tx=0x7f83c0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e1f5defc227b40 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.484+0000 7f83c5c00640 1 -- 172.21.15.32:0/2515940821 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0004050 con 0x556f541dc070 2024-08-31T07:16:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.485+0000 7f83c5c00640 1 -- 172.21.15.32:0/2515940821 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c00041f0 con 0x556f541dc070 2024-08-31T07:16:32.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.485+0000 7f83c5c00640 1 -- 172.21.15.32:0/2515940821 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c000f760 con 0x556f541dc070 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.485+0000 7f83d1600640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83bc0b4ca0 0x7f83bc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.485+0000 7f83d1600640 1 -- 172.21.15.32:0/3992947926 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f83cc106520 con 0x7f83d41099c0 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.485+0000 7f83e203d740 1 -- 172.21.15.32:0/2515940821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 msgr2=0x556f54581b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.485+0000 7f83e203d740 1 --2- 172.21.15.32:0/2515940821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f54581b60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009fa0 tx=0x7f83c0005850 comp rx=0 tx=0).stop 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.486+0000 7f83d8a00640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83bc0b4ca0 0x7f83bc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.486+0000 7f83e203d740 1 -- 172.21.15.32:0/2515940821 shutdown_connections 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.486+0000 7f83e203d740 1 --2- 172.21.15.32:0/2515940821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541bb390 0x556f545825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.486+0000 7f83e203d740 1 --2- 172.21.15.32:0/2515940821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f545820a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.486+0000 7f83e203d740 1 --2- 172.21.15.32:0/2515940821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f54581b60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.486+0000 7f83e203d740 1 -- 172.21.15.32:0/2515940821 >> 172.21.15.32:0/2515940821 conn(0x556f541d0460 msgr2=0x556f5457fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.486+0000 7f83e203d740 1 -- 172.21.15.32:0/2515940821 shutdown_connections 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.486+0000 7f83e203d740 1 -- 172.21.15.32:0/2515940821 wait complete. 2024-08-31T07:16:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.486+0000 7f83d8a00640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83bc0b4ca0 0x7f83bc0b71b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x556f545807e0 tx=0x7f83c80046c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.486+0000 7f83e203d740 1 -- start start 2024-08-31T07:16:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.487+0000 7f83e203d740 1 Processor -- start 2024-08-31T07:16:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.487+0000 7f83e203d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541b31b0 0x556f54591b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.487+0000 7f83e203d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541bb390 0x556f54592080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.487+0000 7f83e203d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f5459d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.487+0000 7f83e203d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556f5434a700 con 0x556f541bb390 2024-08-31T07:16:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.487+0000 7f83e203d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556f5446ecf0 con 0x556f541dc070 2024-08-31T07:16:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.487+0000 7f83e203d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556f545945e0 con 0x556f541b31b0 2024-08-31T07:16:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.487+0000 7f83e203d740 10 client.0 _ll_register_callbacks cb 0x556f54494470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541bb390 0x556f54592080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d8a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541b31b0 0x556f54591b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541bb390 0x556f54592080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53488/0 (socket says 172.21.15.32:53488) 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d3e00640 1 -- 172.21.15.32:0/2451802463 learned_addr learned my addr 172.21.15.32:0/2451802463 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d9400640 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f5459d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d8a00640 1 -- 172.21.15.32:0/2451802463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 msgr2=0x556f5459d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d8a00640 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f5459d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d8a00640 1 -- 172.21.15.32:0/2451802463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541bb390 msgr2=0x556f54592080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d8a00640 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541bb390 0x556f54592080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d8a00640 1 -- 172.21.15.32:0/2451802463 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f544af8b0 con 0x556f541b31b0 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d9400640 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f5459d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83d8a00640 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541b31b0 0x556f54591b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f83c8004560 tx=0x7f83c8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be245dec1295fc18 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.488+0000 7f83ba000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.489+0000 7f83ba000640 1 -- 172.21.15.32:0/2451802463 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c801e800 con 0x556f541b31b0 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.489+0000 7f83ba000640 1 -- 172.21.15.32:0/2451802463 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f83c801e9a0 con 0x556f541b31b0 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.489+0000 7f83e203d740 1 -- 172.21.15.32:0/2451802463 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c00041f0 con 0x556f541b31b0 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.489+0000 7f83ba000640 1 -- 172.21.15.32:0/2451802463 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f83c801eba0 con 0x556f541b31b0 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.489+0000 7f83e203d740 10 client.26353 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.489+0000 7f83e203d740 1 -- 172.21.15.32:0/2451802463 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556f545a06c0 con 0x556f541b31b0 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.489+0000 7f83ba000640 1 -- 172.21.15.32:0/2451802463 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c00041f0 con 0x556f541b31b0 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.490+0000 7f83ba000640 1 -- 172.21.15.32:0/2451802463 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f83c8004b10 con 0x556f541b31b0 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.490+0000 7f83e203d740 10 client.26353 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.490+0000 7f83e203d740 10 client.26353 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.490+0000 7f83e203d740 20 client.26353 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.490+0000 7f83e203d740 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x556f5419a9f0 0x556f545a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.490+0000 7f83e203d740 4 client.26353 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:32.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.490+0000 7f83e203d740 1 -- 172.21.15.32:0/2451802463 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f544fa700 con 0x556f5419a9f0 2024-08-31T07:16:32.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.490+0000 7f83d3e00640 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x556f5419a9f0 0x556f545a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.491+0000 7f83d3e00640 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x556f5419a9f0 0x556f545a3bf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71b92ea42267244d server_cookie=676dc19f0fd0074c in_seq=0 out_seq=0 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.491+0000 7f83ba000640 10 client.26353 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83ba000640 1 -- 172.21.15.32:0/2451802463 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f544fa700 con 0x556f5419a9f0 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83ba000640 10 client.26353 handle_command_reply: tid=0 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 client.26353 shutdown 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 -- 172.21.15.32:0/2451802463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541b31b0 msgr2=0x556f54591b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541b31b0 0x556f54591b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f83c8004560 tx=0x7f83c8005f90 comp rx=0 tx=0).stop 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 -- 172.21.15.32:0/2451802463 shutdown_connections 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x556f5419a9f0 0x556f545a3bf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f5459d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541bb390 0x556f54592080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83ba000640 0 client.26353 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:16:32.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 --2- 172.21.15.32:0/2451802463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541b31b0 0x556f54591b40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 -- 172.21.15.32:0/2451802463 >> 172.21.15.32:0/2451802463 conn(0x556f541d0460 msgr2=0x556f5457dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:32.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 -- 172.21.15.32:0/2451802463 shutdown_connections 2024-08-31T07:16:32.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 1 -- 172.21.15.32:0/2451802463 wait complete. 2024-08-31T07:16:32.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.492+0000 7f83e203d740 20 client.26353 trim_cache size 0 max 16384 2024-08-31T07:16:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83e203d740 1 Processor -- start 2024-08-31T07:16:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83e203d740 1 -- start start 2024-08-31T07:16:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83e203d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f54586c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83e203d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541bb390 0x556f54587180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83e203d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541dc070 0x556f5457dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83e203d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41acf00 con 0x556f541b31b0 2024-08-31T07:16:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83e203d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41aeb00 con 0x556f541bb390 2024-08-31T07:16:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83e203d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41ad5d0 con 0x556f541dc070 2024-08-31T07:16:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83d8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f54586c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83d8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f54586c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53496/0 (socket says 172.21.15.32:53496) 2024-08-31T07:16:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83d8a00640 1 -- 172.21.15.32:0/3655086939 learned_addr learned my addr 172.21.15.32:0/3655086939 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83d9400640 1 --2- 172.21.15.32:0/3655086939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541dc070 0x556f5457dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.507+0000 7f83d3e00640 1 --2- 172.21.15.32:0/3655086939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541bb390 0x556f54587180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.508+0000 7f83d8a00640 1 -- 172.21.15.32:0/3655086939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541dc070 msgr2=0x556f5457dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.508+0000 7f83d8a00640 1 --2- 172.21.15.32:0/3655086939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541dc070 0x556f5457dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.508+0000 7f83d8a00640 1 -- 172.21.15.32:0/3655086939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541bb390 msgr2=0x556f54587180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.508+0000 7f83d8a00640 1 --2- 172.21.15.32:0/3655086939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541bb390 0x556f54587180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.508+0000 7f83d8a00640 1 -- 172.21.15.32:0/3655086939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f5457ff60 con 0x556f541b31b0 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.508+0000 7f83d8a00640 1 --2- 172.21.15.32:0/3655086939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f54586c40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f83c80092e0 tx=0x7f83c8011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fad5420fa28ef56 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.508+0000 7f83c5200640 1 -- 172.21.15.32:0/3655086939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c801e600 con 0x556f541b31b0 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.508+0000 7f83c5200640 1 -- 172.21.15.32:0/3655086939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c8004be0 con 0x556f541b31b0 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.508+0000 7f83c5200640 1 -- 172.21.15.32:0/3655086939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8005600 con 0x556f541b31b0 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.509+0000 7f83e203d740 1 -- 172.21.15.32:0/3655086939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 msgr2=0x556f54586c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.509+0000 7f83e203d740 1 --2- 172.21.15.32:0/3655086939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f54586c40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f83c80092e0 tx=0x7f83c8011d60 comp rx=0 tx=0).stop 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.509+0000 7f83e203d740 1 -- 172.21.15.32:0/3655086939 shutdown_connections 2024-08-31T07:16:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.509+0000 7f83e203d740 1 --2- 172.21.15.32:0/3655086939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541dc070 0x556f5457dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.509+0000 7f83e203d740 1 --2- 172.21.15.32:0/3655086939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541bb390 0x556f54587180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.509+0000 7f83e203d740 1 --2- 172.21.15.32:0/3655086939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f54586c40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.509+0000 7f83e203d740 1 -- 172.21.15.32:0/3655086939 >> 172.21.15.32:0/3655086939 conn(0x556f541d0460 msgr2=0x556f545901c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.509+0000 7f83e203d740 1 -- 172.21.15.32:0/3655086939 shutdown_connections 2024-08-31T07:16:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.509+0000 7f83e203d740 1 -- 172.21.15.32:0/3655086939 wait complete. 2024-08-31T07:16:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.510+0000 7f83e203d740 1 -- start start 2024-08-31T07:16:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.510+0000 7f83e203d740 1 Processor -- start 2024-08-31T07:16:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.510+0000 7f83e203d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f5459a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83e203d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541bb390 0x556f54597a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83e203d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f54597fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83e203d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41ae370 con 0x556f541b31b0 2024-08-31T07:16:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83e203d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41adca0 con 0x556f541dc070 2024-08-31T07:16:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83e203d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41aaee0 con 0x556f541bb390 2024-08-31T07:16:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83e203d740 10 client.0 _ll_register_callbacks cb 0x7f83c8004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83d8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f5459a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83d8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f5459a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53510/0 (socket says 172.21.15.32:53510) 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83d8a00640 1 -- 172.21.15.32:0/2993657000 learned_addr learned my addr 172.21.15.32:0/2993657000 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83d3e00640 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541bb390 0x556f54597a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83d9400640 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f54597fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83d8a00640 1 -- 172.21.15.32:0/2993657000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541bb390 msgr2=0x556f54597a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83d8a00640 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541bb390 0x556f54597a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83d8a00640 1 -- 172.21.15.32:0/2993657000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 msgr2=0x556f54597fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83d8a00640 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f54597fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.511+0000 7f83d8a00640 1 -- 172.21.15.32:0/2993657000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f54598670 con 0x556f541b31b0 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.512+0000 7f83d8a00640 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f5459a000 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f83c801d3a0 tx=0x7f83c8004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c58dc210a56e00b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.512+0000 7f83ba000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.512+0000 7f83ba000640 1 -- 172.21.15.32:0/2993657000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c801e600 con 0x556f541b31b0 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.512+0000 7f83ba000640 1 -- 172.21.15.32:0/2993657000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f83c801e7a0 con 0x556f541b31b0 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.512+0000 7f83e203d740 1 -- 172.21.15.32:0/2993657000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c8004be0 con 0x556f541b31b0 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.512+0000 7f83ba000640 1 -- 172.21.15.32:0/2993657000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f83c801e9a0 con 0x556f541b31b0 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.512+0000 7f83e203d740 10 client.16332 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.512+0000 7f83e203d740 1 -- 172.21.15.32:0/2993657000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556f5460f470 con 0x556f541b31b0 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.513+0000 7f83ba000640 1 -- 172.21.15.32:0/2993657000 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c8004be0 con 0x556f541b31b0 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.513+0000 7f83ba000640 1 -- 172.21.15.32:0/2993657000 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f83c803a4b0 con 0x556f541b31b0 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.513+0000 7f83e203d740 10 client.16332 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.513+0000 7f83e203d740 10 client.16332 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.513+0000 7f83e203d740 20 client.16332 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.513+0000 7f83e203d740 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x556f5416fda0 0x556f54611b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.513+0000 7f83e203d740 4 client.16332 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:16:32.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.513+0000 7f83e203d740 1 -- 172.21.15.32:0/2993657000 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f544f61a0 con 0x556f5416fda0 2024-08-31T07:16:32.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.513+0000 7f83d3e00640 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x556f5416fda0 0x556f54611b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83d3e00640 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x556f5416fda0 0x556f54611b40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f889c3af56f9304e server_cookie=282ac1abfd77ac38 in_seq=0 out_seq=0 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83ba000640 10 client.16332 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83ba000640 1 -- 172.21.15.32:0/2993657000 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f544f61a0 con 0x556f5416fda0 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83ba000640 10 client.16332 handle_command_reply: tid=0 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83e203d740 1 client.16332 shutdown 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83e203d740 1 -- 172.21.15.32:0/2993657000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 msgr2=0x556f5459a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83e203d740 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f5459a000 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f83c801d3a0 tx=0x7f83c8004e00 comp rx=0 tx=0).stop 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83e203d740 1 -- 172.21.15.32:0/2993657000 shutdown_connections 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83e203d740 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x556f5416fda0 0x556f54611b40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83e203d740 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f541dc070 0x556f54597fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83e203d740 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f541bb390 0x556f54597a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83e203d740 1 --2- 172.21.15.32:0/2993657000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f541b31b0 0x556f5459a000 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.514+0000 7f83e203d740 1 -- 172.21.15.32:0/2993657000 >> 172.21.15.32:0/2993657000 conn(0x556f541d0460 msgr2=0x556f54586c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.515+0000 7f83ba000640 0 client.16332 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:16:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.515+0000 7f83e203d740 1 -- 172.21.15.32:0/2993657000 shutdown_connections 2024-08-31T07:16:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.515+0000 7f83e203d740 1 -- 172.21.15.32:0/2993657000 wait complete. 2024-08-31T07:16:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.515+0000 7f83e203d740 20 client.16332 trim_cache size 0 max 16384 2024-08-31T07:16:32.520 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:16:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 -- 172.21.15.32:0/3992947926 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83bc0b4ca0 msgr2=0x7f83bc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83bc0b4ca0 0x7f83bc0b71b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x556f545807e0 tx=0x7f83c80046c0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 -- 172.21.15.32:0/3992947926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83d41099c0 msgr2=0x7f83d419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83d41099c0 0x7f83d419f890 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f83cc005e30 tx=0x7f83cc0317c0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 -- 172.21.15.32:0/3992947926 shutdown_connections 2024-08-31T07:16:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f83bc0b4ca0 0x7f83bc0b71b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83d41099c0 0x7f83d419f890 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83d4109070 0x7f83d419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 --2- 172.21.15.32:0/3992947926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d4103730 0x7f83d419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 -- 172.21.15.32:0/3992947926 >> 172.21.15.32:0/3992947926 conn(0x7f83d40ff180 msgr2=0x7f83d41015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 -- 172.21.15.32:0/3992947926 shutdown_connections 2024-08-31T07:16:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:32.516+0000 7f83d9e00640 1 -- 172.21.15.32:0/3992947926 wait complete. 2024-08-31T07:16:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:32 smithi032 ceph-mon[17252]: pgmap v481: 97 pgs: 97 active+clean; 67 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 548 KiB/s wr, 67 op/s 2024-08-31T07:16:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:32 smithi073 ceph-mon[21995]: pgmap v481: 97 pgs: 97 active+clean; 67 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 548 KiB/s wr, 67 op/s 2024-08-31T07:16:32.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.92 MB/sec execute 21 sec latency 11467.787 ms 2024-08-31T07:16:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:32 smithi084 ceph-mon[22387]: pgmap v481: 97 pgs: 97 active+clean; 67 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 548 KiB/s wr, 67 op/s 2024-08-31T07:16:33.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:16:33.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.88 MB/sec execute 22 sec latency 12467.871 ms 2024-08-31T07:16:33.925 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:34.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:34 smithi032 ceph-mon[17252]: pgmap v482: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 394 KiB/s wr, 44 op/s 2024-08-31T07:16:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:34 smithi073 ceph-mon[21995]: pgmap v482: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 394 KiB/s wr, 44 op/s 2024-08-31T07:16:34.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.84 MB/sec execute 23 sec latency 13467.912 ms 2024-08-31T07:16:34.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.695+0000 7f1cc6200640 1 -- 172.21.15.32:0/2528914281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc010bbe0 msgr2=0x7f1cc0112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:34.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.695+0000 7f1cc6200640 1 --2- 172.21.15.32:0/2528914281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc010bbe0 0x7f1cc0112410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009ca0 tx=0x7f1cb802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.696+0000 7f1cc6200640 1 -- 172.21.15.32:0/2528914281 shutdown_connections 2024-08-31T07:16:34.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.696+0000 7f1cc6200640 1 --2- 172.21.15.32:0/2528914281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc010bbe0 0x7f1cc0112410 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.696+0000 7f1cc6200640 1 --2- 172.21.15.32:0/2528914281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cc010b290 0x7f1cc010b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.696+0000 7f1cc6200640 1 --2- 172.21.15.32:0/2528914281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc0105950 0x7f1cc0105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.696+0000 7f1cc6200640 1 -- 172.21.15.32:0/2528914281 >> 172.21.15.32:0/2528914281 conn(0x7f1cc0101420 msgr2=0x7f1cc0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:34.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.696+0000 7f1cc6200640 1 -- 172.21.15.32:0/2528914281 shutdown_connections 2024-08-31T07:16:34.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.696+0000 7f1cc6200640 1 -- 172.21.15.32:0/2528914281 wait complete. 2024-08-31T07:16:34.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.697+0000 7f1cc6200640 1 Processor -- start 2024-08-31T07:16:34.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.697+0000 7f1cc6200640 1 -- start start 2024-08-31T07:16:34.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.697+0000 7f1cc6200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cc0105950 0x7f1cc019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.697+0000 7f1cc6200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc010b290 0x7f1cc019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.697+0000 7f1cc6200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc010bbe0 0x7f1cc01a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.697+0000 7f1cc6200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01158a0 con 0x7f1cc010b290 2024-08-31T07:16:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.697+0000 7f1cc6200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0115720 con 0x7f1cc010bbe0 2024-08-31T07:16:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.697+0000 7f1cc6200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0115a20 con 0x7f1cc0105950 2024-08-31T07:16:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc010b290 0x7f1cc019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cc5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc010bbe0 0x7f1cc01a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cc4e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cc0105950 0x7f1cc019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc010b290 0x7f1cc019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53524/0 (socket says 172.21.15.32:53524) 2024-08-31T07:16:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cbfe00640 1 -- 172.21.15.32:0/1165165731 learned_addr learned my addr 172.21.15.32:0/1165165731 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cc4e00640 1 -- 172.21.15.32:0/1165165731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc010bbe0 msgr2=0x7f1cc01a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cc4e00640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc010bbe0 0x7f1cc01a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cc4e00640 1 -- 172.21.15.32:0/1165165731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc010b290 msgr2=0x7f1cc019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cc4e00640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc010b290 0x7f1cc019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cc4e00640 1 -- 172.21.15.32:0/1165165731 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc01a21c0 con 0x7f1cc0105950 2024-08-31T07:16:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.698+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc010b290 0x7f1cc019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.699+0000 7f1cc4e00640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cc0105950 0x7f1cc019c250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400e9f0 tx=0x7f1cb400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.699+0000 7f1cbd600640 1 -- 172.21.15.32:0/1165165731 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400cd80 con 0x7f1cc0105950 2024-08-31T07:16:34.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.699+0000 7f1cc6200640 1 -- 172.21.15.32:0/1165165731 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc01a24b0 con 0x7f1cc0105950 2024-08-31T07:16:34.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.699+0000 7f1cbd600640 1 -- 172.21.15.32:0/1165165731 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb40035f0 con 0x7f1cc0105950 2024-08-31T07:16:34.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.699+0000 7f1cbd600640 1 -- 172.21.15.32:0/1165165731 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4010690 con 0x7f1cc0105950 2024-08-31T07:16:34.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.699+0000 7f1cc6200640 1 -- 172.21.15.32:0/1165165731 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01a8c10 con 0x7f1cc0105950 2024-08-31T07:16:34.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.703+0000 7f1cce469740 1 Processor -- start 2024-08-31T07:16:34.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.703+0000 7f1cce469740 1 -- start start 2024-08-31T07:16:34.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.703+0000 7f1cce469740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca4fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.703+0000 7f1cce469740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 0x564f5ca500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.703+0000 7f1cce469740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.703+0000 7f1cce469740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564f5c9c67b0 con 0x564f5c6aa070 2024-08-31T07:16:34.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.703+0000 7f1cce469740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564f5c9b5d40 con 0x564f5c6811b0 2024-08-31T07:16:34.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.703+0000 7f1cce469740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564f5c9b9cf0 con 0x564f5c689390 2024-08-31T07:16:34.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.704+0000 7f1cc5800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca4fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.704+0000 7f1cc5800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca4fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53536/0 (socket says 172.21.15.32:53536) 2024-08-31T07:16:34.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.704+0000 7f1cc5800640 1 -- 172.21.15.32:0/561316168 learned_addr learned my addr 172.21.15.32:0/561316168 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:34.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.704+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/561316168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.704+0000 7f1cc5800640 1 -- 172.21.15.32:0/561316168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 msgr2=0x564f5ca505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:34.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.704+0000 7f1cc5800640 1 --2- 172.21.15.32:0/561316168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.704+0000 7f1cc5800640 1 -- 172.21.15.32:0/561316168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 msgr2=0x564f5ca500a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:16:34.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.704+0000 7f1cc5800640 1 --2- 172.21.15.32:0/561316168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 0x564f5ca500a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.704+0000 7f1cc5800640 1 -- 172.21.15.32:0/561316168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f5c715650 con 0x564f5c6aa070 2024-08-31T07:16:34.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.705+0000 7f1cbd600640 1 -- 172.21.15.32:0/1165165731 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1cb40037a0 con 0x7f1cc0105950 2024-08-31T07:16:34.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.705+0000 7f1cc5800640 1 --2- 172.21.15.32:0/561316168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca4fb60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1cb802f410 tx=0x7f1cb802f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f438800201d9d339 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:34.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.705+0000 7f1cb2000640 1 -- 172.21.15.32:0/561316168 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8031cf0 con 0x564f5c6aa070 2024-08-31T07:16:34.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.705+0000 7f1cb2000640 1 -- 172.21.15.32:0/561316168 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb8036030 con 0x564f5c6aa070 2024-08-31T07:16:34.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.705+0000 7f1cb2000640 1 -- 172.21.15.32:0/561316168 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80042a0 con 0x564f5c6aa070 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.706+0000 7f1cce469740 1 -- 172.21.15.32:0/561316168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 msgr2=0x564f5ca4fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.706+0000 7f1cce469740 1 --2- 172.21.15.32:0/561316168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca4fb60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1cb802f410 tx=0x7f1cb802f440 comp rx=0 tx=0).stop 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.706+0000 7f1cce469740 1 -- 172.21.15.32:0/561316168 shutdown_connections 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.706+0000 7f1cce469740 1 --2- 172.21.15.32:0/561316168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.706+0000 7f1cce469740 1 --2- 172.21.15.32:0/561316168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 0x564f5ca500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.706+0000 7f1cce469740 1 --2- 172.21.15.32:0/561316168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca4fb60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.706+0000 7f1cce469740 1 -- 172.21.15.32:0/561316168 >> 172.21.15.32:0/561316168 conn(0x564f5c69e460 msgr2=0x564f5ca4da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.706+0000 7f1cce469740 1 -- 172.21.15.32:0/561316168 shutdown_connections 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.707+0000 7f1cce469740 1 -- 172.21.15.32:0/561316168 wait complete. 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.707+0000 7f1cbd600640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca80b4bb0 0x7f1ca80b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.707+0000 7f1cbd600640 1 -- 172.21.15.32:0/1165165731 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1cb40e57b0 con 0x7f1cc0105950 2024-08-31T07:16:34.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.707+0000 7f1cce469740 1 -- start start 2024-08-31T07:16:34.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.707+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca80b4bb0 0x7f1ca80b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.707+0000 7f1cce469740 1 Processor -- start 2024-08-31T07:16:34.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.708+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca80b4bb0 0x7f1ca80b70c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009fd0 tx=0x7f1cac005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:34.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.708+0000 7f1cce469740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 0x564f5ca5b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.708+0000 7f1cce469740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca5bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.708+0000 7f1cce469740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca67480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.708+0000 7f1cce469740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564f5c818700 con 0x564f5c6aa070 2024-08-31T07:16:34.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.708+0000 7f1cce469740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564f5c93ccf0 con 0x564f5c6811b0 2024-08-31T07:16:34.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.708+0000 7f1cce469740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564f5ca5e160 con 0x564f5c689390 2024-08-31T07:16:34.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.708+0000 7f1cce469740 10 client.0 _ll_register_callbacks cb 0x564f5c962470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:34.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.708+0000 7f1cc4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 0x564f5ca5b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca5bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cc4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 0x564f5ca5b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58898/0 (socket says 172.21.15.32:58898) 2024-08-31T07:16:34.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cc4e00640 1 -- 172.21.15.32:0/2637324364 learned_addr learned my addr 172.21.15.32:0/2637324364 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:34.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cc5800640 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca67480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cc4e00640 1 -- 172.21.15.32:0/2637324364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 msgr2=0x564f5ca5bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cc4e00640 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca5bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cc4e00640 1 -- 172.21.15.32:0/2637324364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 msgr2=0x564f5ca67480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cc4e00640 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca67480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cc4e00640 1 -- 172.21.15.32:0/2637324364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f5c97d8b0 con 0x564f5c6811b0 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cc5800640 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca67480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1cc4e00640 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 0x564f5ca5b780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400e9c0 tx=0x7f1cb40ac680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=836db2091fd263f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.709+0000 7f1ca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1ca6a00640 1 -- 172.21.15.32:0/2637324364 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40b51e0 con 0x564f5c6811b0 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1cce469740 1 -- 172.21.15.32:0/2637324364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb8036030 con 0x564f5c6811b0 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1ca6a00640 1 -- 172.21.15.32:0/2637324364 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1cb40e31a0 con 0x564f5c6811b0 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1ca6a00640 1 -- 172.21.15.32:0/2637324364 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1cb40a8150 con 0x564f5c6811b0 2024-08-31T07:16:34.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1cce469740 10 client.26354 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1cce469740 1 -- 172.21.15.32:0/2637324364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564f5ca6a200 con 0x564f5c6811b0 2024-08-31T07:16:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1ca6a00640 1 -- 172.21.15.32:0/2637324364 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb8036030 con 0x564f5c6811b0 2024-08-31T07:16:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1ca6a00640 1 -- 172.21.15.32:0/2637324364 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1cb40930b0 con 0x564f5c6811b0 2024-08-31T07:16:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1cce469740 10 client.26354 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1cce469740 10 client.26354 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:16:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1cce469740 20 client.26354 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1cce469740 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x564f5c6689f0 0x564f5ca6d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1cce469740 4 client.26354 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.710+0000 7f1cce469740 1 -- 172.21.15.32:0/2637324364 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f5c9c8700 con 0x564f5c6689f0 2024-08-31T07:16:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.711+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x564f5c6689f0 0x564f5ca6d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.711+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x564f5c6689f0 0x564f5ca6d6c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e0c4191b51a73be server_cookie=a02f4d058218c597 in_seq=0 out_seq=0 2024-08-31T07:16:34.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.711+0000 7f1ca6a00640 10 client.26354 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:16:34.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.712+0000 7f1ca6a00640 1 -- 172.21.15.32:0/2637324364 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564f5c9c8700 con 0x564f5c6689f0 2024-08-31T07:16:34.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.712+0000 7f1ca6a00640 10 client.26354 handle_command_reply: tid=0 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.712+0000 7f1cce469740 1 client.26354 shutdown 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 1 -- 172.21.15.32:0/2637324364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 msgr2=0x564f5ca5b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 0x564f5ca5b780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400e9c0 tx=0x7f1cb40ac680 comp rx=0 tx=0).stop 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 1 -- 172.21.15.32:0/2637324364 shutdown_connections 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x564f5c6689f0 0x564f5ca6d6c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5ca67480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca5bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 1 --2- 172.21.15.32:0/2637324364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6811b0 0x564f5ca5b780 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 1 -- 172.21.15.32:0/2637324364 >> 172.21.15.32:0/2637324364 conn(0x564f5c69e460 msgr2=0x564f5ca52d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1ca6a00640 0 client.26354 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 1 -- 172.21.15.32:0/2637324364 shutdown_connections 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 1 -- 172.21.15.32:0/2637324364 wait complete. 2024-08-31T07:16:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.713+0000 7f1cce469740 20 client.26354 trim_cache size 0 max 16384 2024-08-31T07:16:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.727+0000 7f1cce469740 1 Processor -- start 2024-08-31T07:16:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.727+0000 7f1cce469740 1 -- start start 2024-08-31T07:16:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cce469740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6811b0 0x564f5ca67380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cce469740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca59e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cce469740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6aa070 0x564f5ca5a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cce469740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b1370 con 0x564f5c6811b0 2024-08-31T07:16:34.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cce469740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b2f70 con 0x564f5c6aa070 2024-08-31T07:16:34.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cce469740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b1a40 con 0x564f5c689390 2024-08-31T07:16:34.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cc4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6811b0 0x564f5ca67380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cc4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6811b0 0x564f5ca67380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53542/0 (socket says 172.21.15.32:53542) 2024-08-31T07:16:34.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cc4e00640 1 -- 172.21.15.32:0/557055452 learned_addr learned my addr 172.21.15.32:0/557055452 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:34.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca59e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cc5800640 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6aa070 0x564f5ca5a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cbfe00640 1 -- 172.21.15.32:0/557055452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6aa070 msgr2=0x564f5ca5a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6aa070 0x564f5ca5a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cbfe00640 1 -- 172.21.15.32:0/557055452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6811b0 msgr2=0x564f5ca67380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6811b0 0x564f5ca67380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cbfe00640 1 -- 172.21.15.32:0/557055452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f5ca47c70 con 0x564f5c689390 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cc5800640 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6aa070 0x564f5ca5a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.728+0000 7f1cc4e00640 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6811b0 0x564f5ca67380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.729+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca59e00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x564f5ca6de30 tx=0x7f1cac01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c801f612d180383 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.729+0000 7f1cb1600640 1 -- 172.21.15.32:0/557055452 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac01d670 con 0x564f5c689390 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.729+0000 7f1cb1600640 1 -- 172.21.15.32:0/557055452 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cac01dcc0 con 0x564f5c689390 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.729+0000 7f1cb1600640 1 -- 172.21.15.32:0/557055452 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac02bd00 con 0x564f5c689390 2024-08-31T07:16:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.730+0000 7f1cce469740 1 -- 172.21.15.32:0/557055452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 msgr2=0x564f5ca59e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.730+0000 7f1cce469740 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca59e00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x564f5ca6de30 tx=0x7f1cac01b000 comp rx=0 tx=0).stop 2024-08-31T07:16:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.730+0000 7f1cce469740 1 -- 172.21.15.32:0/557055452 shutdown_connections 2024-08-31T07:16:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.730+0000 7f1cce469740 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c6aa070 0x564f5ca5a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.730+0000 7f1cce469740 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c689390 0x564f5ca59e00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.730+0000 7f1cce469740 1 --2- 172.21.15.32:0/557055452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6811b0 0x564f5ca67380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.730+0000 7f1cce469740 1 -- 172.21.15.32:0/557055452 >> 172.21.15.32:0/557055452 conn(0x564f5c69e460 msgr2=0x564f5ca566e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.730+0000 7f1cce469740 1 -- 172.21.15.32:0/557055452 shutdown_connections 2024-08-31T07:16:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.730+0000 7f1cce469740 1 -- 172.21.15.32:0/557055452 wait complete. 2024-08-31T07:16:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.730+0000 7f1cce469740 1 -- start start 2024-08-31T07:16:34.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cce469740 1 Processor -- start 2024-08-31T07:16:34.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cce469740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c6811b0 0x564f5ca5f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cce469740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c689390 0x564f5ca5fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cce469740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5cad70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cce469740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b27e0 con 0x564f5c6aa070 2024-08-31T07:16:34.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cce469740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b2110 con 0x564f5c689390 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cce469740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01af350 con 0x564f5c6811b0 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cce469740 10 client.0 _ll_register_callbacks cb 0x7f1cb40ab9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cc4e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c6811b0 0x564f5ca5f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cc4e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c6811b0 0x564f5ca5f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36806/0 (socket says 172.21.15.32:36806) 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cc4e00640 1 -- 172.21.15.32:0/3244704473 learned_addr learned my addr 172.21.15.32:0/3244704473 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.731+0000 7f1cc5800640 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5cad70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.732+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c689390 0x564f5ca5fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.732+0000 7f1cbfe00640 1 -- 172.21.15.32:0/3244704473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c6811b0 msgr2=0x564f5ca5f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.732+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c6811b0 0x564f5ca5f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.732+0000 7f1cbfe00640 1 -- 172.21.15.32:0/3244704473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 msgr2=0x564f5cad70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.732+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5cad70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.732+0000 7f1cbfe00640 1 -- 172.21.15.32:0/3244704473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f5cad7600 con 0x564f5c689390 2024-08-31T07:16:34.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.732+0000 7f1cc4e00640 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c6811b0 0x564f5ca5f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.732+0000 7f1cc5800640 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5cad70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.732+0000 7f1cbfe00640 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c689390 0x564f5ca5fce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0095e0 tx=0x7f1cac005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d73d63e8a441ca server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.732+0000 7f1ca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.733+0000 7f1ca6a00640 1 -- 172.21.15.32:0/3244704473 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac02b630 con 0x564f5c689390 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.733+0000 7f1cce469740 1 -- 172.21.15.32:0/3244704473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cac01dcc0 con 0x564f5c689390 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.733+0000 7f1ca6a00640 1 -- 172.21.15.32:0/3244704473 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f1cac02ba80 con 0x564f5c689390 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.733+0000 7f1ca6a00640 1 -- 172.21.15.32:0/3244704473 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1cac02bc80 con 0x564f5c689390 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.733+0000 7f1cce469740 10 client.26360 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.733+0000 7f1cce469740 1 -- 172.21.15.32:0/3244704473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564f5cad8720 con 0x564f5c689390 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.733+0000 7f1ca6a00640 1 -- 172.21.15.32:0/3244704473 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cac01dcc0 con 0x564f5c689390 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.734+0000 7f1ca6a00640 1 -- 172.21.15.32:0/3244704473 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1cac039070 con 0x564f5c689390 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.734+0000 7f1cce469740 10 client.26360 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.734+0000 7f1cce469740 10 client.26360 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:16:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.734+0000 7f1cce469740 20 client.26360 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.734+0000 7f1cce469740 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x564f5c63dda0 0x564f5cadad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.734+0000 7f1cce469740 4 client.26360 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.734+0000 7f1cce469740 1 -- 172.21.15.32:0/3244704473 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f5c9c41a0 con 0x564f5c63dda0 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.734+0000 7f1cc4e00640 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x564f5c63dda0 0x564f5cadad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.734+0000 7f1cc4e00640 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x564f5c63dda0 0x564f5cadad80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e63c77b97ede7bc server_cookie=6aae84c1a4631a1b in_seq=0 out_seq=0 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.734+0000 7f1ca6a00640 10 client.26360 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1ca6a00640 1 -- 172.21.15.32:0/3244704473 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f5c9c41a0 con 0x564f5c63dda0 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1ca6a00640 10 client.26360 handle_command_reply: tid=0 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 client.26360 shutdown 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 -- 172.21.15.32:0/3244704473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c689390 msgr2=0x564f5ca5fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c689390 0x564f5ca5fce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0095e0 tx=0x7f1cac005b20 comp rx=0 tx=0).stop 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 -- 172.21.15.32:0/3244704473 shutdown_connections 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x564f5c63dda0 0x564f5cadad80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f5c6aa070 0x564f5cad70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f5c689390 0x564f5ca5fce0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 --2- 172.21.15.32:0/3244704473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564f5c6811b0 0x564f5ca5f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1ca6a00640 0 client.26360 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:16:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 -- 172.21.15.32:0/3244704473 >> 172.21.15.32:0/3244704473 conn(0x564f5c69e460 msgr2=0x564f5ca54c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 -- 172.21.15.32:0/3244704473 shutdown_connections 2024-08-31T07:16:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 1 -- 172.21.15.32:0/3244704473 wait complete. 2024-08-31T07:16:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.735+0000 7f1cce469740 20 client.26360 trim_cache size 0 max 16384 2024-08-31T07:16:34.740 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:16:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 -- 172.21.15.32:0/1165165731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca80b4bb0 msgr2=0x7f1ca80b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca80b4bb0 0x7f1ca80b70c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009fd0 tx=0x7f1cac005f90 comp rx=0 tx=0).stop 2024-08-31T07:16:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 -- 172.21.15.32:0/1165165731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cc0105950 msgr2=0x7f1cc019c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cc0105950 0x7f1cc019c250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400e9f0 tx=0x7f1cb400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 -- 172.21.15.32:0/1165165731 shutdown_connections 2024-08-31T07:16:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1ca80b4bb0 0x7f1ca80b70c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cc010bbe0 0x7f1cc01a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc010b290 0x7f1cc019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 --2- 172.21.15.32:0/1165165731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cc0105950 0x7f1cc019c250 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 -- 172.21.15.32:0/1165165731 >> 172.21.15.32:0/1165165731 conn(0x7f1cc0101420 msgr2=0x7f1cc010a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 -- 172.21.15.32:0/1165165731 shutdown_connections 2024-08-31T07:16:34.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:34.736+0000 7f1cc6200640 1 -- 172.21.15.32:0/1165165731 wait complete. 2024-08-31T07:16:34.742 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:34 smithi084 ceph-mon[22387]: pgmap v482: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 394 KiB/s wr, 44 op/s 2024-08-31T07:16:35.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.80 MB/sec execute 24 sec latency 14468.042 ms 2024-08-31T07:16:35.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:16:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:36.563 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:36 smithi032 ceph-mon[17252]: pgmap v483: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 306 KiB/s wr, 33 op/s 2024-08-31T07:16:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:36 smithi073 ceph-mon[21995]: pgmap v483: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 306 KiB/s wr, 33 op/s 2024-08-31T07:16:36.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.77 MB/sec execute 25 sec latency 15468.105 ms 2024-08-31T07:16:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:36 smithi084 ceph-mon[22387]: pgmap v483: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 306 KiB/s wr, 33 op/s 2024-08-31T07:16:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.864+0000 7f2305000640 1 -- 172.21.15.32:0/1039403079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2300103750 msgr2=0x7f2300103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.864+0000 7f2305000640 1 --2- 172.21.15.32:0/1039403079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2300103750 0x7f2300103b60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009900 tx=0x7f22f002a410 comp rx=0 tx=0).stop 2024-08-31T07:16:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.864+0000 7f2305000640 1 -- 172.21.15.32:0/1039403079 shutdown_connections 2024-08-31T07:16:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.865+0000 7f2305000640 1 --2- 172.21.15.32:0/1039403079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23001099e0 0x7f2300110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.865+0000 7f2305000640 1 --2- 172.21.15.32:0/1039403079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2300109090 0x7f23001094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.865+0000 7f2305000640 1 --2- 172.21.15.32:0/1039403079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2300103750 0x7f2300103b60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.865+0000 7f2305000640 1 -- 172.21.15.32:0/1039403079 >> 172.21.15.32:0/1039403079 conn(0x7f23000ff1a0 msgr2=0x7f23001015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.865+0000 7f2305000640 1 -- 172.21.15.32:0/1039403079 shutdown_connections 2024-08-31T07:16:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.865+0000 7f2305000640 1 -- 172.21.15.32:0/1039403079 wait complete. 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.866+0000 7f2305000640 1 Processor -- start 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.866+0000 7f2305000640 1 -- start start 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.866+0000 7f2305000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2300103750 0x7f2300076ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.866+0000 7f2305000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2300109090 0x7f23000754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.866+0000 7f2305000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23001099e0 0x7f23000759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.866+0000 7f2305000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2300113690 con 0x7f2300109090 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.866+0000 7f2305000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2300113510 con 0x7f2300103750 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.866+0000 7f2305000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2300113810 con 0x7f23001099e0 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22fea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2300109090 0x7f23000754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22ff400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2300103750 0x7f2300076ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23001099e0 0x7f23000759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22fea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2300109090 0x7f23000754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60284/0 (socket says 172.21.15.32:60284) 2024-08-31T07:16:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22ff400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2300103750 0x7f2300076ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45020/0 (socket says 172.21.15.32:45020) 2024-08-31T07:16:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22fea00640 1 -- 172.21.15.32:0/3527669797 learned_addr learned my addr 172.21.15.32:0/3527669797 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22ffe00640 1 -- 172.21.15.32:0/3527669797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2300103750 msgr2=0x7f2300076ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22ffe00640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2300103750 0x7f2300076ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22ffe00640 1 -- 172.21.15.32:0/3527669797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2300109090 msgr2=0x7f23000754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22ffe00640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2300109090 0x7f23000754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22ffe00640 1 -- 172.21.15.32:0/3527669797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2300076210 con 0x7f23001099e0 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.867+0000 7f22fea00640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2300109090 0x7f23000754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.868+0000 7f22ffe00640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23001099e0 0x7f23000759f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009fd0 tx=0x7f22f800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.868+0000 7f22f7e00640 1 -- 172.21.15.32:0/3527669797 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f800ce20 con 0x7f23001099e0 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.868+0000 7f2305000640 1 -- 172.21.15.32:0/3527669797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23001aca40 con 0x7f23001099e0 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.868+0000 7f22f7e00640 1 -- 172.21.15.32:0/3527669797 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f8005020 con 0x7f23001099e0 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.868+0000 7f22f7e00640 1 -- 172.21.15.32:0/3527669797 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8010690 con 0x7f23001099e0 2024-08-31T07:16:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.868+0000 7f2305000640 1 -- 172.21.15.32:0/3527669797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23001acf30 con 0x7f23001099e0 2024-08-31T07:16:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.871+0000 7f230d22c740 1 Processor -- start 2024-08-31T07:16:36.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.872+0000 7f22f7e00640 1 -- 172.21.15.32:0/3527669797 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f22f80054d0 con 0x7f23001099e0 2024-08-31T07:16:36.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.872+0000 7f230d22c740 1 -- start start 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.872+0000 7f230d22c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7eb070 0x56453eb90b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f230d22c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7c21b0 0x56453eb910a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f230d22c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7ca390 0x56453eb915e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f230d22c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56453eb077b0 con 0x56453e7eb070 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f230d22c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56453eaf6d40 con 0x56453e7c21b0 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f230d22c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56453eafacf0 con 0x56453e7ca390 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f22fea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7eb070 0x56453eb90b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f22fea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7eb070 0x56453eb90b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60290/0 (socket says 172.21.15.32:60290) 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f22ff400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7c21b0 0x56453eb910a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f22fea00640 1 -- 172.21.15.32:0/4290636213 learned_addr learned my addr 172.21.15.32:0/4290636213 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f22fea00640 1 --2- 172.21.15.32:0/4290636213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7ca390 0x56453eb915e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f22ff400640 1 -- 172.21.15.32:0/4290636213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7ca390 msgr2=0x56453eb915e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:36.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f22ff400640 1 --2- 172.21.15.32:0/4290636213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7ca390 0x56453eb915e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f22ff400640 1 -- 172.21.15.32:0/4290636213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7eb070 msgr2=0x56453eb90b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f22ff400640 1 --2- 172.21.15.32:0/4290636213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7eb070 0x56453eb90b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.873+0000 7f22ff400640 1 -- 172.21.15.32:0/4290636213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453e856650 con 0x56453e7c21b0 2024-08-31T07:16:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.874+0000 7f22f7e00640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22e40b4ca0 0x7f22e40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.874+0000 7f22ff400640 1 --2- 172.21.15.32:0/4290636213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7c21b0 0x56453eb910a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f230019df50 tx=0x7f22f0032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fed9b0d7906f7dfd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.874+0000 7f22f7e00640 1 -- 172.21.15.32:0/3527669797 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f22f80e5bd0 con 0x7f23001099e0 2024-08-31T07:16:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.874+0000 7f22fea00640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22e40b4ca0 0x7f22e40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.874+0000 7f22f4c00640 1 -- 172.21.15.32:0/4290636213 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0040070 con 0x56453e7c21b0 2024-08-31T07:16:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.874+0000 7f22f4c00640 1 -- 172.21.15.32:0/4290636213 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f0004030 con 0x56453e7c21b0 2024-08-31T07:16:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.874+0000 7f22f4c00640 1 -- 172.21.15.32:0/4290636213 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0035040 con 0x56453e7c21b0 2024-08-31T07:16:36.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.875+0000 7f230d22c740 1 -- 172.21.15.32:0/4290636213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7c21b0 msgr2=0x56453eb910a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:36.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.875+0000 7f230d22c740 1 --2- 172.21.15.32:0/4290636213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7c21b0 0x56453eb910a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f230019df50 tx=0x7f22f0032b20 comp rx=0 tx=0).stop 2024-08-31T07:16:36.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.875+0000 7f22fea00640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22e40b4ca0 0x7f22e40b71b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0058c0 tx=0x7f22ec015040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:36.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.875+0000 7f230d22c740 1 -- 172.21.15.32:0/4290636213 shutdown_connections 2024-08-31T07:16:36.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.875+0000 7f230d22c740 1 --2- 172.21.15.32:0/4290636213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7ca390 0x56453eb915e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.875+0000 7f230d22c740 1 --2- 172.21.15.32:0/4290636213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7c21b0 0x56453eb910a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.875+0000 7f230d22c740 1 --2- 172.21.15.32:0/4290636213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7eb070 0x56453eb90b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.875+0000 7f230d22c740 1 -- 172.21.15.32:0/4290636213 >> 172.21.15.32:0/4290636213 conn(0x56453e7df460 msgr2=0x56453eb8ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:36.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.875+0000 7f230d22c740 1 -- 172.21.15.32:0/4290636213 shutdown_connections 2024-08-31T07:16:36.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.876+0000 7f230d22c740 1 -- 172.21.15.32:0/4290636213 wait complete. 2024-08-31T07:16:36.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.876+0000 7f230d22c740 1 -- start start 2024-08-31T07:16:36.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.876+0000 7f230d22c740 1 Processor -- start 2024-08-31T07:16:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f230d22c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 0x56453eb9c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f230d22c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7ca390 0x56453eb9cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f230d22c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 0x56453eba8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f230d22c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56453e959700 con 0x56453e7c21b0 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f230d22c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56453ea7dcf0 con 0x56453e7eb070 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f230d22c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56453eb9f2e0 con 0x56453e7ca390 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f230d22c740 10 client.0 _ll_register_callbacks cb 0x56453eaa3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f22ff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 0x56453eb9c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 0x56453eba8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 0x56453eba8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45040/0 (socket says 172.21.15.32:45040) 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f22ffe00640 1 -- 172.21.15.32:0/1790751549 learned_addr learned my addr 172.21.15.32:0/1790751549 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.877+0000 7f22fea00640 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7ca390 0x56453eb9cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.878+0000 7f22ffe00640 1 -- 172.21.15.32:0/1790751549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7ca390 msgr2=0x56453eb9cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.878+0000 7f22ffe00640 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7ca390 0x56453eb9cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.878+0000 7f22ffe00640 1 -- 172.21.15.32:0/1790751549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 msgr2=0x56453eb9c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.878+0000 7f22ffe00640 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 0x56453eb9c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.878+0000 7f22ffe00640 1 -- 172.21.15.32:0/1790751549 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56453eabe8b0 con 0x56453e7eb070 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.878+0000 7f22ffe00640 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 0x56453eba8600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005fd0 tx=0x7f22f80ac2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ddb8aa339688083 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.878+0000 7f22e9600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.878+0000 7f22e9600640 1 -- 172.21.15.32:0/1790751549 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f80a8570 con 0x56453e7eb070 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.878+0000 7f230d22c740 1 -- 172.21.15.32:0/1790751549 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f0004030 con 0x56453e7eb070 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.878+0000 7f22e9600640 1 -- 172.21.15.32:0/1790751549 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f22f80b5170 con 0x56453e7eb070 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f22e9600640 1 -- 172.21.15.32:0/1790751549 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f22f80a8710 con 0x56453e7eb070 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f230d22c740 10 client.26372 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f230d22c740 1 -- 172.21.15.32:0/1790751549 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56453ebab380 con 0x56453e7eb070 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f22e9600640 1 -- 172.21.15.32:0/1790751549 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f80ca080 con 0x56453e7eb070 2024-08-31T07:16:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f22e9600640 1 -- 172.21.15.32:0/1790751549 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f22f80aafb0 con 0x56453e7eb070 2024-08-31T07:16:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f230d22c740 10 client.26372 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f230d22c740 10 client.26372 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:16:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f230d22c740 20 client.26372 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f230d22c740 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56453e7a99f0 0x56453ebae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f230d22c740 4 client.26372 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.879+0000 7f230d22c740 1 -- 172.21.15.32:0/1790751549 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56453eb09700 con 0x56453e7a99f0 2024-08-31T07:16:36.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.880+0000 7f22ff400640 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56453e7a99f0 0x56453ebae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.880+0000 7f22ff400640 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56453e7a99f0 0x56453ebae850 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11ad1089ab7d7807 server_cookie=cb55f09740f831b5 in_seq=0 out_seq=0 2024-08-31T07:16:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.880+0000 7f22e9600640 10 client.26372 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:16:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.881+0000 7f22e9600640 1 -- 172.21.15.32:0/1790751549 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56453eb09700 con 0x56453e7a99f0 2024-08-31T07:16:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.881+0000 7f22e9600640 10 client.26372 handle_command_reply: tid=0 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.881+0000 7f230d22c740 1 client.26372 shutdown 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.881+0000 7f230d22c740 1 -- 172.21.15.32:0/1790751549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 msgr2=0x56453eba8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.881+0000 7f230d22c740 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 0x56453eba8600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005fd0 tx=0x7f22f80ac2e0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.881+0000 7f230d22c740 1 -- 172.21.15.32:0/1790751549 shutdown_connections 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.881+0000 7f230d22c740 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56453e7a99f0 0x56453ebae850 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.882+0000 7f230d22c740 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 0x56453eba8600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.882+0000 7f230d22c740 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7ca390 0x56453eb9cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.882+0000 7f230d22c740 1 --2- 172.21.15.32:0/1790751549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 0x56453eb9c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.882+0000 7f230d22c740 1 -- 172.21.15.32:0/1790751549 >> 172.21.15.32:0/1790751549 conn(0x56453e7df460 msgr2=0x56453eb93d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.882+0000 7f22e9600640 0 client.26372 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.882+0000 7f230d22c740 1 -- 172.21.15.32:0/1790751549 shutdown_connections 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.882+0000 7f230d22c740 1 -- 172.21.15.32:0/1790751549 wait complete. 2024-08-31T07:16:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.882+0000 7f230d22c740 20 client.26372 trim_cache size 0 max 16384 2024-08-31T07:16:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.896+0000 7f230d22c740 1 Processor -- start 2024-08-31T07:16:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.896+0000 7f230d22c740 1 -- start start 2024-08-31T07:16:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.896+0000 7f230d22c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 0x56453eb97cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.896+0000 7f230d22c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7ca390 0x56453eb94620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.896+0000 7f230d22c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7eb070 0x56453eb9e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.896+0000 7f230d22c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b5700 con 0x56453e7c21b0 2024-08-31T07:16:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.896+0000 7f230d22c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b7300 con 0x56453e7ca390 2024-08-31T07:16:36.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.896+0000 7f230d22c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b5dd0 con 0x56453e7eb070 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.897+0000 7f22ff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 0x56453eb97cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.897+0000 7f22ff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 0x56453eb97cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60308/0 (socket says 172.21.15.32:60308) 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.897+0000 7f22ff400640 1 -- 172.21.15.32:0/1330941436 learned_addr learned my addr 172.21.15.32:0/1330941436 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.897+0000 7f22ffe00640 1 --2- 172.21.15.32:0/1330941436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7eb070 0x56453eb9e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.897+0000 7f22fea00640 1 --2- 172.21.15.32:0/1330941436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7ca390 0x56453eb94620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.897+0000 7f22ff400640 1 -- 172.21.15.32:0/1330941436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7eb070 msgr2=0x56453eb9e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.897+0000 7f22ff400640 1 --2- 172.21.15.32:0/1330941436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7eb070 0x56453eb9e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.897+0000 7f22ff400640 1 -- 172.21.15.32:0/1330941436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7ca390 msgr2=0x56453eb94620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.897+0000 7f22ff400640 1 --2- 172.21.15.32:0/1330941436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7ca390 0x56453eb94620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.897+0000 7f22ff400640 1 -- 172.21.15.32:0/1330941436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453eb939c0 con 0x56453e7c21b0 2024-08-31T07:16:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.898+0000 7f22ff400640 1 --2- 172.21.15.32:0/1330941436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 0x56453eb97cd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x56453eb8f7e0 tx=0x7f22f00050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e653dc037ddcc63 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.898+0000 7f22ebe00640 1 -- 172.21.15.32:0/1330941436 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0035040 con 0x56453e7c21b0 2024-08-31T07:16:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.898+0000 7f22ebe00640 1 -- 172.21.15.32:0/1330941436 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f00043e0 con 0x56453e7c21b0 2024-08-31T07:16:36.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.898+0000 7f22ebe00640 1 -- 172.21.15.32:0/1330941436 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0040040 con 0x56453e7c21b0 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.899+0000 7f230d22c740 1 -- 172.21.15.32:0/1330941436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 msgr2=0x56453eb97cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.899+0000 7f230d22c740 1 --2- 172.21.15.32:0/1330941436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 0x56453eb97cd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x56453eb8f7e0 tx=0x7f22f00050e0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.899+0000 7f230d22c740 1 -- 172.21.15.32:0/1330941436 shutdown_connections 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.899+0000 7f230d22c740 1 --2- 172.21.15.32:0/1330941436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7eb070 0x56453eb9e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.899+0000 7f230d22c740 1 --2- 172.21.15.32:0/1330941436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7ca390 0x56453eb94620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.899+0000 7f230d22c740 1 --2- 172.21.15.32:0/1330941436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7c21b0 0x56453eb97cd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.899+0000 7f230d22c740 1 -- 172.21.15.32:0/1330941436 >> 172.21.15.32:0/1330941436 conn(0x56453e7df460 msgr2=0x56453eb9da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.899+0000 7f230d22c740 1 -- 172.21.15.32:0/1330941436 shutdown_connections 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.899+0000 7f230d22c740 1 -- 172.21.15.32:0/1330941436 wait complete. 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.899+0000 7f230d22c740 1 -- start start 2024-08-31T07:16:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f230d22c740 1 Processor -- start 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f230d22c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7c21b0 0x56453eba0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f230d22c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7ca390 0x56453eba0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f230d22c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 0x56453eba1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f230d22c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b6b70 con 0x56453e7ca390 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f230d22c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b64a0 con 0x56453e7eb070 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f230d22c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b36e0 con 0x56453e7c21b0 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f230d22c740 10 client.0 _ll_register_callbacks cb 0x7f22f80abe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f22fea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7ca390 0x56453eba0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f22ff400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7c21b0 0x56453eba0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f22fea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7ca390 0x56453eba0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60310/0 (socket says 172.21.15.32:60310) 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.900+0000 7f22fea00640 1 -- 172.21.15.32:0/2620360178 learned_addr learned my addr 172.21.15.32:0/2620360178 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.901+0000 7f22ffe00640 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 0x56453eba1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.901+0000 7f22ff400640 1 -- 172.21.15.32:0/2620360178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 msgr2=0x56453eba1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.901+0000 7f22ff400640 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 0x56453eba1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.901+0000 7f22ff400640 1 -- 172.21.15.32:0/2620360178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7ca390 msgr2=0x56453eba0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.901+0000 7f22ff400640 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7ca390 0x56453eba0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.901+0000 7f22ff400640 1 -- 172.21.15.32:0/2620360178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56453ec182b0 con 0x56453e7c21b0 2024-08-31T07:16:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.901+0000 7f22ff400640 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7c21b0 0x56453eba0600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x56453eb8f7e0 tx=0x7f22f0031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e5d3a515bcceda2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.901+0000 7f22e9600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:16:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.901+0000 7f22e9600640 1 -- 172.21.15.32:0/2620360178 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f005c070 con 0x56453e7c21b0 2024-08-31T07:16:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.902+0000 7f230d22c740 1 -- 172.21.15.32:0/2620360178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f00043e0 con 0x56453e7c21b0 2024-08-31T07:16:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.902+0000 7f22e9600640 1 -- 172.21.15.32:0/2620360178 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f22f0031d10 con 0x56453e7c21b0 2024-08-31T07:16:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.902+0000 7f22e9600640 1 -- 172.21.15.32:0/2620360178 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f22f0068050 con 0x56453e7c21b0 2024-08-31T07:16:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.902+0000 7f230d22c740 10 client.26395 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.902+0000 7f230d22c740 1 -- 172.21.15.32:0/2620360178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56453ec19530 con 0x56453e7c21b0 2024-08-31T07:16:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.902+0000 7f22e9600640 1 -- 172.21.15.32:0/2620360178 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f00043e0 con 0x56453e7c21b0 2024-08-31T07:16:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.902+0000 7f22e9600640 1 -- 172.21.15.32:0/2620360178 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f22f0004cf0 con 0x56453e7c21b0 2024-08-31T07:16:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.903+0000 7f230d22c740 10 client.26395 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.903+0000 7f230d22c740 10 client.26395 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.903+0000 7f230d22c740 20 client.26395 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.903+0000 7f230d22c740 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56453e77eda0 0x56453ec1b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.903+0000 7f230d22c740 4 client.26395 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.903+0000 7f230d22c740 1 -- 172.21.15.32:0/2620360178 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56453eb051a0 con 0x56453e77eda0 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.903+0000 7f22fea00640 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56453e77eda0 0x56453ec1b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f22e9600640 10 client.26395 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f22fea00640 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56453e77eda0 0x56453ec1b820 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17b5716333790b3c server_cookie=b2aa001f2336aa6a in_seq=0 out_seq=0 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f22e9600640 1 -- 172.21.15.32:0/2620360178 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56453eb051a0 con 0x56453e77eda0 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f22e9600640 10 client.26395 handle_command_reply: tid=0 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 client.26395 shutdown 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 -- 172.21.15.32:0/2620360178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7c21b0 msgr2=0x56453eba0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7c21b0 0x56453eba0600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x56453eb8f7e0 tx=0x7f22f0031400 comp rx=0 tx=0).stop 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 -- 172.21.15.32:0/2620360178 shutdown_connections 2024-08-31T07:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56453e77eda0 0x56453ec1b820 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56453e7eb070 0x56453eba1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56453e7ca390 0x56453eba0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f22e9600640 0 client.26395 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 --2- 172.21.15.32:0/2620360178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56453e7c21b0 0x56453eba0600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 -- 172.21.15.32:0/2620360178 >> 172.21.15.32:0/2620360178 conn(0x56453e7df460 msgr2=0x56453eb9e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 -- 172.21.15.32:0/2620360178 shutdown_connections 2024-08-31T07:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 1 -- 172.21.15.32:0/2620360178 wait complete. 2024-08-31T07:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.904+0000 7f230d22c740 20 client.26395 trim_cache size 0 max 16384 2024-08-31T07:16:36.908 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:16:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 -- 172.21.15.32:0/3527669797 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22e40b4ca0 msgr2=0x7f22e40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22e40b4ca0 0x7f22e40b71b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0058c0 tx=0x7f22ec015040 comp rx=0 tx=0).stop 2024-08-31T07:16:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 -- 172.21.15.32:0/3527669797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23001099e0 msgr2=0x7f23000759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23001099e0 0x7f23000759f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009fd0 tx=0x7f22f800ef10 comp rx=0 tx=0).stop 2024-08-31T07:16:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 -- 172.21.15.32:0/3527669797 shutdown_connections 2024-08-31T07:16:36.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f22e40b4ca0 0x7f22e40b71b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23001099e0 0x7f23000759f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2300109090 0x7f23000754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 --2- 172.21.15.32:0/3527669797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2300103750 0x7f2300076ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:36.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 -- 172.21.15.32:0/3527669797 >> 172.21.15.32:0/3527669797 conn(0x7f23000ff1a0 msgr2=0x7f23001015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:36.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 -- 172.21.15.32:0/3527669797 shutdown_connections 2024-08-31T07:16:36.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:36.905+0000 7f2305000640 1 -- 172.21.15.32:0/3527669797 wait complete. 2024-08-31T07:16:37.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.74 MB/sec execute 26 sec latency 16468.175 ms 2024-08-31T07:16:37.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:16:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:38.546 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:38 smithi032 ceph-mon[17252]: pgmap v484: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 123 KiB/s wr, 29 op/s 2024-08-31T07:16:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:38 smithi073 ceph-mon[21995]: pgmap v484: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 123 KiB/s wr, 29 op/s 2024-08-31T07:16:38.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.71 MB/sec execute 27 sec latency 17468.237 ms 2024-08-31T07:16:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:38 smithi084 ceph-mon[22387]: pgmap v484: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 123 KiB/s wr, 29 op/s 2024-08-31T07:16:38.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.840+0000 7f695c800640 1 -- 172.21.15.32:0/1624324815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6954109b90 msgr2=0x7f69541103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:38.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.840+0000 7f695c800640 1 --2- 172.21.15.32:0/1624324815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6954109b90 0x7f69541103c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6950009ca0 tx=0x7f695002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.840+0000 7f695c800640 1 -- 172.21.15.32:0/1624324815 shutdown_connections 2024-08-31T07:16:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.840+0000 7f695c800640 1 --2- 172.21.15.32:0/1624324815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6954109b90 0x7f69541103c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.840+0000 7f695c800640 1 --2- 172.21.15.32:0/1624324815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6954109240 0x7f6954109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.840+0000 7f695c800640 1 --2- 172.21.15.32:0/1624324815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954103900 0x7f6954103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.840+0000 7f695c800640 1 -- 172.21.15.32:0/1624324815 >> 172.21.15.32:0/1624324815 conn(0x7f695407b5f0 msgr2=0x7f695407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.840+0000 7f695c800640 1 -- 172.21.15.32:0/1624324815 shutdown_connections 2024-08-31T07:16:38.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.840+0000 7f695c800640 1 -- 172.21.15.32:0/1624324815 wait complete. 2024-08-31T07:16:38.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.841+0000 7f695c800640 1 Processor -- start 2024-08-31T07:16:38.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.841+0000 7f695c800640 1 -- start start 2024-08-31T07:16:38.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f695c800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954103900 0x7f695419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f695c800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6954109240 0x7f695419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f695c800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6954109b90 0x7f69541a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f695c800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6954113850 con 0x7f6954109b90 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f695c800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69541136d0 con 0x7f6954109240 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f695c800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69541139d0 con 0x7f6954103900 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f6959e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954103900 0x7f695419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f6959e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954103900 0x7f695419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40996/0 (socket says 172.21.15.32:40996) 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f6959e00640 1 -- 172.21.15.32:0/324225856 learned_addr learned my addr 172.21.15.32:0/324225856 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f695a800640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6954109b90 0x7f69541a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.842+0000 7f6959400640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6954109240 0x7f695419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.843+0000 7f6959e00640 1 -- 172.21.15.32:0/324225856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6954109240 msgr2=0x7f695419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.843+0000 7f6959e00640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6954109240 0x7f695419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.843+0000 7f6959e00640 1 -- 172.21.15.32:0/324225856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6954109b90 msgr2=0x7f69541a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.843+0000 7f6959e00640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6954109b90 0x7f69541a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.843+0000 7f6959e00640 1 -- 172.21.15.32:0/324225856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69541a23d0 con 0x7f6954103900 2024-08-31T07:16:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.843+0000 7f6959e00640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954103900 0x7f695419c460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f694800e9f0 tx=0x7f694800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.843+0000 7f694ea00640 1 -- 172.21.15.32:0/324225856 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694800cd80 con 0x7f6954103900 2024-08-31T07:16:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.843+0000 7f695c800640 1 -- 172.21.15.32:0/324225856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69541a26c0 con 0x7f6954103900 2024-08-31T07:16:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.843+0000 7f694ea00640 1 -- 172.21.15.32:0/324225856 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69480054d0 con 0x7f6954103900 2024-08-31T07:16:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.843+0000 7f694ea00640 1 -- 172.21.15.32:0/324225856 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948010690 con 0x7f6954103900 2024-08-31T07:16:38.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.844+0000 7f695c800640 1 -- 172.21.15.32:0/324225856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69541a8e20 con 0x7f6954103900 2024-08-31T07:16:38.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.847+0000 7f69634d8740 1 Processor -- start 2024-08-31T07:16:38.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.847+0000 7f69634d8740 1 -- start start 2024-08-31T07:16:38.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.847+0000 7f694ea00640 1 -- 172.21.15.32:0/324225856 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6948004e90 con 0x7f6954103900 2024-08-31T07:16:38.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.847+0000 7f69634d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff65990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f69634d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 0x561f5ff65ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f69634d8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb9f390 0x561f5ff66410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f69634d8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561f5fedc7b0 con 0x561f5fb9f390 2024-08-31T07:16:38.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f69634d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f5fecbd40 con 0x561f5fb971b0 2024-08-31T07:16:38.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f69634d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561f5fecfcf0 con 0x561f5fbc0070 2024-08-31T07:16:38.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f6959e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 0x561f5ff65ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f695a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff65990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f6959e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 0x561f5ff65ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45062/0 (socket says 172.21.15.32:45062) 2024-08-31T07:16:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f6959400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb9f390 0x561f5ff66410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f6959e00640 1 -- 172.21.15.32:0/3013396786 learned_addr learned my addr 172.21.15.32:0/3013396786 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f695a800640 1 -- 172.21.15.32:0/3013396786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 msgr2=0x561f5ff65ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f695a800640 1 --2- 172.21.15.32:0/3013396786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 0x561f5ff65ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.848+0000 7f695a800640 1 -- 172.21.15.32:0/3013396786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb9f390 msgr2=0x561f5ff66410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.849+0000 7f695a800640 1 --2- 172.21.15.32:0/3013396786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb9f390 0x561f5ff66410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.849+0000 7f695a800640 1 -- 172.21.15.32:0/3013396786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5fc2b650 con 0x561f5fbc0070 2024-08-31T07:16:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.849+0000 7f6959e00640 1 --2- 172.21.15.32:0/3013396786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 0x561f5ff65ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.849+0000 7f6959400640 1 --2- 172.21.15.32:0/3013396786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb9f390 0x561f5ff66410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.849+0000 7f695a800640 1 --2- 172.21.15.32:0/3013396786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff65990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6950005e30 tx=0x7f695002f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3649401bacfb9714 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.849+0000 7f6943000640 1 -- 172.21.15.32:0/3013396786 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950031cf0 con 0x561f5fbc0070 2024-08-31T07:16:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.849+0000 7f6943000640 1 -- 172.21.15.32:0/3013396786 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6950036030 con 0x561f5fbc0070 2024-08-31T07:16:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.849+0000 7f6943000640 1 -- 172.21.15.32:0/3013396786 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950004040 con 0x561f5fbc0070 2024-08-31T07:16:38.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.849+0000 7f694ea00640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f693c0b4d20 0x7f693c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.850+0000 7f694ea00640 1 -- 172.21.15.32:0/324225856 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f69480e58e0 con 0x7f6954103900 2024-08-31T07:16:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.850+0000 7f6959e00640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f693c0b4d20 0x7f693c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.850+0000 7f69634d8740 1 -- 172.21.15.32:0/3013396786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 msgr2=0x561f5ff65990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.850+0000 7f69634d8740 1 --2- 172.21.15.32:0/3013396786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff65990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6950005e30 tx=0x7f695002f410 comp rx=0 tx=0).stop 2024-08-31T07:16:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.850+0000 7f69634d8740 1 -- 172.21.15.32:0/3013396786 shutdown_connections 2024-08-31T07:16:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.850+0000 7f69634d8740 1 --2- 172.21.15.32:0/3013396786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb9f390 0x561f5ff66410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.850+0000 7f69634d8740 1 --2- 172.21.15.32:0/3013396786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 0x561f5ff65ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.850+0000 7f69634d8740 1 --2- 172.21.15.32:0/3013396786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff65990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.851+0000 7f69634d8740 1 -- 172.21.15.32:0/3013396786 >> 172.21.15.32:0/3013396786 conn(0x561f5fbb4460 msgr2=0x561f5ff63880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.851+0000 7f6959e00640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f693c0b4d20 0x7f693c0b7230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f69480a9260 tx=0x7f69480eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.851+0000 7f69634d8740 1 -- 172.21.15.32:0/3013396786 shutdown_connections 2024-08-31T07:16:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.851+0000 7f69634d8740 1 -- 172.21.15.32:0/3013396786 wait complete. 2024-08-31T07:16:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.851+0000 7f69634d8740 1 -- start start 2024-08-31T07:16:38.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.851+0000 7f69634d8740 1 Processor -- start 2024-08-31T07:16:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f69634d8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff71580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f69634d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 0x561f5ff71b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f69634d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff7d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f69634d8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561f5fd2e700 con 0x561f5fb971b0 2024-08-31T07:16:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f69634d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f5fe52cf0 con 0x561f5fb9f390 2024-08-31T07:16:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f69634d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561f5ff73f90 con 0x561f5fbc0070 2024-08-31T07:16:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f69634d8740 10 client.0 _ll_register_callbacks cb 0x561f5fe78470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f695a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff71580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f695a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff71580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60366/0 (socket says 172.21.15.32:60366) 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f695a800640 1 -- 172.21.15.32:0/2930426774 learned_addr learned my addr 172.21.15.32:0/2930426774 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.852+0000 7f6959400640 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 0x561f5ff71b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.853+0000 7f695a800640 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff7d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.853+0000 7f695a800640 1 -- 172.21.15.32:0/2930426774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 msgr2=0x561f5ff7d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.853+0000 7f695a800640 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff7d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.853+0000 7f695a800640 1 -- 172.21.15.32:0/2930426774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 msgr2=0x561f5ff71b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.853+0000 7f695a800640 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 0x561f5ff71b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.853+0000 7f695a800640 1 -- 172.21.15.32:0/2930426774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5fe938b0 con 0x561f5fb971b0 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.853+0000 7f6959400640 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 0x561f5ff71b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.853+0000 7f695a800640 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff71580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f695002b0b0 tx=0x7f695002b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263b6e6b76b163b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.853+0000 7f6937400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.853+0000 7f6937400640 1 -- 172.21.15.32:0/2930426774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950058060 con 0x561f5fb971b0 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.854+0000 7f69634d8740 1 -- 172.21.15.32:0/2930426774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6950036030 con 0x561f5fb971b0 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.854+0000 7f6937400640 1 -- 172.21.15.32:0/2930426774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6950031860 con 0x561f5fb971b0 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.854+0000 7f6937400640 1 -- 172.21.15.32:0/2930426774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f695002f660 con 0x561f5fb971b0 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.854+0000 7f69634d8740 10 client.16377 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.854+0000 7f69634d8740 1 -- 172.21.15.32:0/2930426774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f5ff80020 con 0x561f5fb971b0 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.854+0000 7f6937400640 1 -- 172.21.15.32:0/2930426774 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6950036030 con 0x561f5fb971b0 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.855+0000 7f6937400640 1 -- 172.21.15.32:0/2930426774 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f6950043070 con 0x561f5fb971b0 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.855+0000 7f69634d8740 10 client.16377 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.855+0000 7f69634d8740 10 client.16377 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.855+0000 7f69634d8740 20 client.16377 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.855+0000 7f69634d8740 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561f5fb7e9f0 0x561f5ff83380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.855+0000 7f69634d8740 4 client.16377 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.855+0000 7f69634d8740 1 -- 172.21.15.32:0/2930426774 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f5fede700 con 0x561f5fb7e9f0 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.855+0000 7f6959400640 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561f5fb7e9f0 0x561f5ff83380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.856+0000 7f6959400640 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561f5fb7e9f0 0x561f5ff83380 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4ad7af90016c417 server_cookie=e964e9fb9aeb16bd in_seq=0 out_seq=0 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.856+0000 7f6937400640 10 client.16377 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.856+0000 7f6937400640 1 -- 172.21.15.32:0/2930426774 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561f5fede700 con 0x561f5fb7e9f0 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f6937400640 10 client.16377 handle_command_reply: tid=0 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 client.16377 shutdown 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 -- 172.21.15.32:0/2930426774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 msgr2=0x561f5ff71580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff71580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f695002b0b0 tx=0x7f695002b040 comp rx=0 tx=0).stop 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 -- 172.21.15.32:0/2930426774 shutdown_connections 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff7d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561f5fb7e9f0 0x561f5ff83380 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 0x561f5ff71b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 --2- 172.21.15.32:0/2930426774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff71580 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 -- 172.21.15.32:0/2930426774 >> 172.21.15.32:0/2930426774 conn(0x561f5fbb4460 msgr2=0x561f5ff68b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f6937400640 0 client.16377 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:16:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 -- 172.21.15.32:0/2930426774 shutdown_connections 2024-08-31T07:16:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 1 -- 172.21.15.32:0/2930426774 wait complete. 2024-08-31T07:16:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.857+0000 7f69634d8740 20 client.16377 trim_cache size 0 max 16384 2024-08-31T07:16:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.871+0000 7f69634d8740 1 Processor -- start 2024-08-31T07:16:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.871+0000 7f69634d8740 1 -- start start 2024-08-31T07:16:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f69634d8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff6aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f69634d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 0x561f5ff6afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f69634d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff62e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f69634d8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b1440 con 0x561f5fb971b0 2024-08-31T07:16:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f69634d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b3040 con 0x561f5fb9f390 2024-08-31T07:16:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f69634d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b1b10 con 0x561f5fbc0070 2024-08-31T07:16:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f695a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff6aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f695a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff6aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60378/0 (socket says 172.21.15.32:60378) 2024-08-31T07:16:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f695a800640 1 -- 172.21.15.32:0/3157672526 learned_addr learned my addr 172.21.15.32:0/3157672526 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f6959400640 1 --2- 172.21.15.32:0/3157672526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 0x561f5ff6afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f695a800640 1 --2- 172.21.15.32:0/3157672526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff62e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f695a800640 1 -- 172.21.15.32:0/3157672526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 msgr2=0x561f5ff62e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f695a800640 1 --2- 172.21.15.32:0/3157672526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff62e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f695a800640 1 -- 172.21.15.32:0/3157672526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 msgr2=0x561f5ff6afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f695a800640 1 --2- 172.21.15.32:0/3157672526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 0x561f5ff6afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f695a800640 1 -- 172.21.15.32:0/3157672526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5ff73c00 con 0x561f5fb971b0 2024-08-31T07:16:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.872+0000 7f6959400640 1 --2- 172.21.15.32:0/3157672526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 0x561f5ff6afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.873+0000 7f695a800640 1 --2- 172.21.15.32:0/3157672526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff6aaa0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6950033040 tx=0x7f6950031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95978b37ca0e4b2d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.873+0000 7f6942600640 1 -- 172.21.15.32:0/3157672526 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950058030 con 0x561f5fb971b0 2024-08-31T07:16:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.873+0000 7f6942600640 1 -- 172.21.15.32:0/3157672526 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6950036070 con 0x561f5fb971b0 2024-08-31T07:16:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.873+0000 7f6942600640 1 -- 172.21.15.32:0/3157672526 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950062080 con 0x561f5fb971b0 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.874+0000 7f69634d8740 1 -- 172.21.15.32:0/3157672526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 msgr2=0x561f5ff6aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.874+0000 7f69634d8740 1 --2- 172.21.15.32:0/3157672526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff6aaa0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6950033040 tx=0x7f6950031af0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.874+0000 7f69634d8740 1 -- 172.21.15.32:0/3157672526 shutdown_connections 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.874+0000 7f69634d8740 1 --2- 172.21.15.32:0/3157672526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fbc0070 0x561f5ff62e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.874+0000 7f69634d8740 1 --2- 172.21.15.32:0/3157672526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb9f390 0x561f5ff6afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.874+0000 7f69634d8740 1 --2- 172.21.15.32:0/3157672526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fb971b0 0x561f5ff6aaa0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.874+0000 7f69634d8740 1 -- 172.21.15.32:0/3157672526 >> 172.21.15.32:0/3157672526 conn(0x561f5fbb4460 msgr2=0x561f5ff6fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.874+0000 7f69634d8740 1 -- 172.21.15.32:0/3157672526 shutdown_connections 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.874+0000 7f69634d8740 1 -- 172.21.15.32:0/3157672526 wait complete. 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.875+0000 7f69634d8740 1 -- start start 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.875+0000 7f69634d8740 1 Processor -- start 2024-08-31T07:16:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.875+0000 7f69634d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 0x561f5ff84c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.875+0000 7f69634d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fb9f390 0x561f5ff85220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.875+0000 7f69634d8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fbc0070 0x561f5ffe6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.875+0000 7f69634d8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b28b0 con 0x561f5fbc0070 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.875+0000 7f69634d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b21e0 con 0x561f5fb971b0 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.875+0000 7f69634d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69541af420 con 0x561f5fb9f390 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.875+0000 7f69634d8740 10 client.0 _ll_register_callbacks cb 0x7f6950009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f6959400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fb9f390 0x561f5ff85220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f695a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fbc0070 0x561f5ffe6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f6959400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fb9f390 0x561f5ff85220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41036/0 (socket says 172.21.15.32:41036) 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f6959400640 1 -- 172.21.15.32:0/799321655 learned_addr learned my addr 172.21.15.32:0/799321655 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f695a800640 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 0x561f5ff84c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f6959400640 1 -- 172.21.15.32:0/799321655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 msgr2=0x561f5ff84c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f6959400640 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 0x561f5ff84c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f6959400640 1 -- 172.21.15.32:0/799321655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fbc0070 msgr2=0x561f5ffe6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f6959400640 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fbc0070 0x561f5ffe6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f6959400640 1 -- 172.21.15.32:0/799321655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5ffe7140 con 0x561f5fb9f390 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f6959400640 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fb9f390 0x561f5ff85220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6944012580 tx=0x7f6944013a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3cbdb272a1dfd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.876+0000 7f6937400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.877+0000 7f6937400640 1 -- 172.21.15.32:0/799321655 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944005220 con 0x561f5fb9f390 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.877+0000 7f69634d8740 1 -- 172.21.15.32:0/799321655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6950036070 con 0x561f5fb9f390 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.877+0000 7f6937400640 1 -- 172.21.15.32:0/799321655 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f69440053c0 con 0x561f5fb9f390 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.877+0000 7f6937400640 1 -- 172.21.15.32:0/799321655 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f69440055c0 con 0x561f5fb9f390 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.877+0000 7f69634d8740 10 client.26413 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:38.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.877+0000 7f69634d8740 1 -- 172.21.15.32:0/799321655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f5ffee4a0 con 0x561f5fb9f390 2024-08-31T07:16:38.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.877+0000 7f6937400640 1 -- 172.21.15.32:0/799321655 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6950036070 con 0x561f5fb9f390 2024-08-31T07:16:38.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.878+0000 7f6937400640 1 -- 172.21.15.32:0/799321655 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f6944003950 con 0x561f5fb9f390 2024-08-31T07:16:38.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.878+0000 7f69634d8740 10 client.26413 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:38.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.878+0000 7f69634d8740 10 client.26413 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:16:38.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.878+0000 7f69634d8740 20 client.26413 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:38.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.878+0000 7f69634d8740 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561f5fb53da0 0x561f5fff0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:38.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.878+0000 7f69634d8740 4 client.26413 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:16:38.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.878+0000 7f69634d8740 1 -- 172.21.15.32:0/799321655 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f5feda1a0 con 0x561f5fb53da0 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.878+0000 7f695a800640 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561f5fb53da0 0x561f5fff0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.878+0000 7f695a800640 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561f5fb53da0 0x561f5fff0c40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c551e386f11d34ea server_cookie=bad49fc74f3c1300 in_seq=0 out_seq=0 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.878+0000 7f6937400640 10 client.26413 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f6937400640 1 -- 172.21.15.32:0/799321655 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f5feda1a0 con 0x561f5fb53da0 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f6937400640 10 client.26413 handle_command_reply: tid=0 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 client.26413 shutdown 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 -- 172.21.15.32:0/799321655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fb9f390 msgr2=0x561f5ff85220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fb9f390 0x561f5ff85220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6944012580 tx=0x7f6944013a70 comp rx=0 tx=0).stop 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 -- 172.21.15.32:0/799321655 shutdown_connections 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561f5fbc0070 0x561f5ffe6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561f5fb9f390 0x561f5ff85220 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561f5fb53da0 0x561f5fff0c40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 --2- 172.21.15.32:0/799321655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f5fb971b0 0x561f5ff84c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 -- 172.21.15.32:0/799321655 >> 172.21.15.32:0/799321655 conn(0x561f5fbb4460 msgr2=0x561f5ff6aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:38.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f6937400640 0 client.26413 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:16:38.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 -- 172.21.15.32:0/799321655 shutdown_connections 2024-08-31T07:16:38.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 1 -- 172.21.15.32:0/799321655 wait complete. 2024-08-31T07:16:38.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.879+0000 7f69634d8740 20 client.26413 trim_cache size 0 max 16384 2024-08-31T07:16:38.884 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:16:38.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 -- 172.21.15.32:0/324225856 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f693c0b4d20 msgr2=0x7f693c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:38.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f693c0b4d20 0x7f693c0b7230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f69480a9260 tx=0x7f69480eb530 comp rx=0 tx=0).stop 2024-08-31T07:16:38.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 -- 172.21.15.32:0/324225856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954103900 msgr2=0x7f695419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:38.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954103900 0x7f695419c460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f694800e9f0 tx=0x7f694800edf0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 -- 172.21.15.32:0/324225856 shutdown_connections 2024-08-31T07:16:38.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f693c0b4d20 0x7f693c0b7230 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6954109b90 0x7f69541a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6954109240 0x7f695419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 --2- 172.21.15.32:0/324225856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6954103900 0x7f695419c460 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:38.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 -- 172.21.15.32:0/324225856 >> 172.21.15.32:0/324225856 conn(0x7f695407b5f0 msgr2=0x7f6954105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:38.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 -- 172.21.15.32:0/324225856 shutdown_connections 2024-08-31T07:16:38.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:38.880+0000 7f695c800640 1 -- 172.21.15.32:0/324225856 wait complete. 2024-08-31T07:16:39.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:16:39.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.69 MB/sec execute 28 sec latency 18468.288 ms 2024-08-31T07:16:39.852 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:16:39.896 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.586+0000 7f299fe00640 1 -- 172.21.15.32:0/3570746643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0103730 msgr2=0x7f29a0103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.586+0000 7f299fe00640 1 --2- 172.21.15.32:0/3570746643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0103730 0x7f29a0103b40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2990009900 tx=0x7f299002a410 comp rx=0 tx=0).stop 2024-08-31T07:16:40.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.586+0000 7f299fe00640 1 -- 172.21.15.32:0/3570746643 shutdown_connections 2024-08-31T07:16:40.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.586+0000 7f299fe00640 1 --2- 172.21.15.32:0/3570746643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29a01099c0 0x7f29a01101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:40.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.586+0000 7f299fe00640 1 --2- 172.21.15.32:0/3570746643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a0109070 0x7f29a0109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:40.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.586+0000 7f299fe00640 1 --2- 172.21.15.32:0/3570746643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0103730 0x7f29a0103b40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:40.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.586+0000 7f299fe00640 1 -- 172.21.15.32:0/3570746643 >> 172.21.15.32:0/3570746643 conn(0x7f29a00ff180 msgr2=0x7f29a01015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:40.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.586+0000 7f299fe00640 1 -- 172.21.15.32:0/3570746643 shutdown_connections 2024-08-31T07:16:40.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.586+0000 7f299fe00640 1 -- 172.21.15.32:0/3570746643 wait complete. 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.587+0000 7f299fe00640 1 Processor -- start 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.587+0000 7f299fe00640 1 -- start start 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29a0103730 0x7f29a00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0109070 0x7f29a00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a01099c0 0x7f29a0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0113690 con 0x7f29a0109070 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0113510 con 0x7f29a0103730 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0113810 con 0x7f29a01099c0 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29a0103730 0x7f29a00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0109070 0x7f29a00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29a0103730 0x7f29a00754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45106/0 (socket says 172.21.15.32:45106) 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299ea00640 1 -- 172.21.15.32:0/749879266 learned_addr learned my addr 172.21.15.32:0/749879266 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.588+0000 7f299f400640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a01099c0 0x7f29a0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f299ea00640 1 -- 172.21.15.32:0/749879266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a01099c0 msgr2=0x7f29a0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f299ea00640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a01099c0 0x7f29a0075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f299ea00640 1 -- 172.21.15.32:0/749879266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0109070 msgr2=0x7f29a00759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f299ea00640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0109070 0x7f29a00759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f299ea00640 1 -- 172.21.15.32:0/749879266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a01a8340 con 0x7f29a0103730 2024-08-31T07:16:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f299f400640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a01099c0 0x7f29a0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:16:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f299ea00640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29a0103730 0x7f29a00754b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2990005940 tx=0x7f29900040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f2997400640 1 -- 172.21.15.32:0/749879266 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299003f030 con 0x7f29a0103730 2024-08-31T07:16:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f2997400640 1 -- 172.21.15.32:0/749879266 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29900044c0 con 0x7f29a0103730 2024-08-31T07:16:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f299fe00640 1 -- 172.21.15.32:0/749879266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a01a85d0 con 0x7f29a0103730 2024-08-31T07:16:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f2997400640 1 -- 172.21.15.32:0/749879266 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990035070 con 0x7f29a0103730 2024-08-31T07:16:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.589+0000 7f299fe00640 1 -- 172.21.15.32:0/749879266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a01a8b00 con 0x7f29a0103730 2024-08-31T07:16:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.591+0000 7f299fe00640 1 -- 172.21.15.32:0/749879266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a0103b40 con 0x7f29a0103730 2024-08-31T07:16:40.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.593+0000 7f2997400640 1 -- 172.21.15.32:0/749879266 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2990042030 con 0x7f29a0103730 2024-08-31T07:16:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.594+0000 7f2997400640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29840b4d20 0x7f29840b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.594+0000 7f2997400640 1 -- 172.21.15.32:0/749879266 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2990105b50 con 0x7f29a0103730 2024-08-31T07:16:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.595+0000 7f299e000640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29840b4d20 0x7f29840b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.595+0000 7f299e000640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29840b4d20 0x7f29840b7230 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f29a0108ad0 tx=0x7f298c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:40.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.596+0000 7f2997400640 1 -- 172.21.15.32:0/749879266 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f29900cc960 con 0x7f29a0103730 2024-08-31T07:16:40.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:40 smithi032 ceph-mon[17252]: pgmap v485: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-08-31T07:16:40.672 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:40 smithi073 ceph-mon[21995]: pgmap v485: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-08-31T07:16:40.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.66 MB/sec execute 29 sec latency 19468.355 ms 2024-08-31T07:16:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:40 smithi084 ceph-mon[22387]: pgmap v485: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 19 op/s 2024-08-31T07:16:40.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.843+0000 7f299fe00640 1 -- 172.21.15.32:0/749879266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29a00767b0 con 0x7f29a0103730 2024-08-31T07:16:40.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.844+0000 7f2997400640 1 -- 172.21.15.32:0/749879266 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11967 (secure 0 0 0) 0x7f29900cc260 con 0x7f29a0103730 2024-08-31T07:16:40.845 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:16:40.845 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"btime":"2024-08-31T07:14:36:074596+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:14:35.939633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:16:40.846 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2024-08-31T07:16:40.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.846+0000 7f299fe00640 1 -- 172.21.15.32:0/749879266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29840b4d20 msgr2=0x7f29840b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:40.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.846+0000 7f299fe00640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29840b4d20 0x7f29840b7230 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f29a0108ad0 tx=0x7f298c005f90 comp rx=0 tx=0).stop 2024-08-31T07:16:40.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.846+0000 7f299fe00640 1 -- 172.21.15.32:0/749879266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29a0103730 msgr2=0x7f29a00754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:40.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.846+0000 7f299fe00640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29a0103730 0x7f29a00754b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2990005940 tx=0x7f29900040c0 comp rx=0 tx=0).stop 2024-08-31T07:16:40.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.847+0000 7f299fe00640 1 -- 172.21.15.32:0/749879266 shutdown_connections 2024-08-31T07:16:40.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.847+0000 7f299fe00640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f29840b4d20 0x7f29840b7230 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:40.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.847+0000 7f299fe00640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a01099c0 0x7f29a0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:40.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.847+0000 7f299fe00640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0109070 0x7f29a00759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:40.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.847+0000 7f299fe00640 1 --2- 172.21.15.32:0/749879266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29a0103730 0x7f29a00754b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:40.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.847+0000 7f299fe00640 1 -- 172.21.15.32:0/749879266 >> 172.21.15.32:0/749879266 conn(0x7f29a00ff180 msgr2=0x7f29a01015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:40.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.847+0000 7f299fe00640 1 -- 172.21.15.32:0/749879266 shutdown_connections 2024-08-31T07:16:40.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:40.847+0000 7f299fe00640 1 -- 172.21.15.32:0/749879266 wait complete. 2024-08-31T07:16:41.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:41 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/749879266' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.477+0000 7fe68be00640 1 -- 172.21.15.32:0/1484905244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe68c109090 msgr2=0x7fe68c1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.477+0000 7fe68be00640 1 --2- 172.21.15.32:0/1484905244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe68c109090 0x7fe68c1094a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe678009860 tx=0x7fe67802a340 comp rx=0 tx=0).stop 2024-08-31T07:16:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.477+0000 7fe68be00640 1 -- 172.21.15.32:0/1484905244 shutdown_connections 2024-08-31T07:16:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.477+0000 7fe68be00640 1 --2- 172.21.15.32:0/1484905244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe68c1099e0 0x7fe68c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.477+0000 7fe68be00640 1 --2- 172.21.15.32:0/1484905244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe68c109090 0x7fe68c1094a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.477+0000 7fe68be00640 1 --2- 172.21.15.32:0/1484905244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c103750 0x7fe68c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.477+0000 7fe68be00640 1 -- 172.21.15.32:0/1484905244 >> 172.21.15.32:0/1484905244 conn(0x7fe68c0ff1e0 msgr2=0x7fe68c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.477+0000 7fe68be00640 1 -- 172.21.15.32:0/1484905244 shutdown_connections 2024-08-31T07:16:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.477+0000 7fe68be00640 1 -- 172.21.15.32:0/1484905244 wait complete. 2024-08-31T07:16:41.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.478+0000 7fe68be00640 1 Processor -- start 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.478+0000 7fe68be00640 1 -- start start 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c103750 0x7fe68c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe68c109090 0x7fe68c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe68c1099e0 0x7fe68c1a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c113670 con 0x7fe68c103750 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1134f0 con 0x7fe68c109090 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1137f0 con 0x7fe68c1099e0 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe68c109090 0x7fe68c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c103750 0x7fe68c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe68c109090 0x7fe68c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45132/0 (socket says 172.21.15.32:45132) 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68a000640 1 -- 172.21.15.32:0/1740543636 learned_addr learned my addr 172.21.15.32:0/1740543636 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.479+0000 7fe68b400640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe68c1099e0 0x7fe68c1a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe68a000640 1 -- 172.21.15.32:0/1740543636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe68c1099e0 msgr2=0x7fe68c1a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe68a000640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe68c1099e0 0x7fe68c1a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe68a000640 1 -- 172.21.15.32:0/1740543636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c103750 msgr2=0x7fe68c19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe68a000640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c103750 0x7fe68c19c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe68a000640 1 -- 172.21.15.32:0/1740543636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c1a2230 con 0x7fe68c109090 2024-08-31T07:16:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe68aa00640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c103750 0x7fe68c19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe68a000640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe68c109090 0x7fe68c19c860 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe678035040 tx=0x7fe678032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe683400640 1 -- 172.21.15.32:0/1740543636 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678040020 con 0x7fe68c109090 2024-08-31T07:16:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe683400640 1 -- 172.21.15.32:0/1740543636 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe678030d50 con 0x7fe68c109090 2024-08-31T07:16:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe68be00640 1 -- 172.21.15.32:0/1740543636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c1a24c0 con 0x7fe68c109090 2024-08-31T07:16:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.480+0000 7fe683400640 1 -- 172.21.15.32:0/1740543636 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678037040 con 0x7fe68c109090 2024-08-31T07:16:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.481+0000 7fe68be00640 1 -- 172.21.15.32:0/1740543636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c1a8c80 con 0x7fe68c109090 2024-08-31T07:16:41.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.483+0000 7fe6987fc740 1 Processor -- start 2024-08-31T07:16:41.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.484+0000 7fe6987fc740 1 -- start start 2024-08-31T07:16:41.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.484+0000 7fe683400640 1 -- 172.21.15.32:0/1740543636 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe678043020 con 0x7fe68c109090 2024-08-31T07:16:41.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.484+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 0x55a2d5b22b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.484+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.484+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.484+0000 7fe6987fc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5a997b0 con 0x55a2d575c390 2024-08-31T07:16:41.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.484+0000 7fe6987fc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5a88d40 con 0x55a2d577d070 2024-08-31T07:16:41.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.484+0000 7fe6987fc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5a8ccf0 con 0x55a2d57541b0 2024-08-31T07:16:41.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.484+0000 7fe68b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 0x55a2d5b22b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.484+0000 7fe68a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.485+0000 7fe68b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 0x55a2d5b22b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45134/0 (socket says 172.21.15.32:45134) 2024-08-31T07:16:41.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.485+0000 7fe68b400640 1 -- 172.21.15.32:0/2131913649 learned_addr learned my addr 172.21.15.32:0/2131913649 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:41.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.485+0000 7fe68aa00640 1 --2- 172.21.15.32:0/2131913649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.485+0000 7fe68b400640 1 -- 172.21.15.32:0/2131913649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 msgr2=0x55a2d5b230a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.485+0000 7fe68b400640 1 --2- 172.21.15.32:0/2131913649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b230a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.485+0000 7fe68b400640 1 -- 172.21.15.32:0/2131913649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 msgr2=0x55a2d5b235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.485+0000 7fe68b400640 1 --2- 172.21.15.32:0/2131913649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.485+0000 7fe68b400640 1 -- 172.21.15.32:0/2131913649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d57e8650 con 0x55a2d577d070 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.485+0000 7fe68b400640 1 --2- 172.21.15.32:0/2131913649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 0x55a2d5b22b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe684009fd0 tx=0x7fe6840118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df09cb417bdabab4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.486+0000 7fe677e00640 1 -- 172.21.15.32:0/2131913649 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684004d20 con 0x55a2d577d070 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.486+0000 7fe677e00640 1 -- 172.21.15.32:0/2131913649 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684004050 con 0x55a2d577d070 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.486+0000 7fe677e00640 1 -- 172.21.15.32:0/2131913649 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684005280 con 0x55a2d577d070 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.486+0000 7fe683400640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6700b4d20 0x7fe6700b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.486+0000 7fe683400640 1 -- 172.21.15.32:0/1740543636 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe678104ff0 con 0x7fe68c109090 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.486+0000 7fe68aa00640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6700b4d20 0x7fe6700b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.487+0000 7fe6987fc740 1 -- 172.21.15.32:0/2131913649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 msgr2=0x55a2d5b22b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.487+0000 7fe6987fc740 1 --2- 172.21.15.32:0/2131913649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 0x55a2d5b22b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe684009fd0 tx=0x7fe6840118d0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.487+0000 7fe6987fc740 1 -- 172.21.15.32:0/2131913649 shutdown_connections 2024-08-31T07:16:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.487+0000 7fe6987fc740 1 --2- 172.21.15.32:0/2131913649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.487+0000 7fe6987fc740 1 --2- 172.21.15.32:0/2131913649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b230a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.487+0000 7fe6987fc740 1 --2- 172.21.15.32:0/2131913649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 0x55a2d5b22b60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.487+0000 7fe6987fc740 1 -- 172.21.15.32:0/2131913649 >> 172.21.15.32:0/2131913649 conn(0x55a2d5771460 msgr2=0x55a2d5b20a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:41.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.487+0000 7fe6987fc740 1 -- 172.21.15.32:0/2131913649 shutdown_connections 2024-08-31T07:16:41.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.487+0000 7fe6987fc740 1 -- 172.21.15.32:0/2131913649 wait complete. 2024-08-31T07:16:41.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.487+0000 7fe68aa00640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6700b4d20 0x7fe6700b7230 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55a2d5b217e0 tx=0x7fe67c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:41.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.488+0000 7fe6987fc740 1 -- start start 2024-08-31T07:16:41.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.488+0000 7fe6987fc740 1 Processor -- start 2024-08-31T07:16:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.488+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b2e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.488+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b2ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.488+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 0x55a2d5b3a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.488+0000 7fe6987fc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a2d58eb700 con 0x55a2d575c390 2024-08-31T07:16:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.488+0000 7fe6987fc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5a0fcf0 con 0x55a2d577d070 2024-08-31T07:16:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.488+0000 7fe6987fc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5b31340 con 0x55a2d57541b0 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.488+0000 7fe6987fc740 10 client.0 _ll_register_callbacks cb 0x55a2d5a35470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.489+0000 7fe68a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b2ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.489+0000 7fe68a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b2ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60460/0 (socket says 172.21.15.32:60460) 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.489+0000 7fe68a000640 1 -- 172.21.15.32:0/238375080 learned_addr learned my addr 172.21.15.32:0/238375080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.489+0000 7fe68aa00640 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b2e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.489+0000 7fe68b400640 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 0x55a2d5b3a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.489+0000 7fe68a000640 1 -- 172.21.15.32:0/238375080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 msgr2=0x55a2d5b2e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.489+0000 7fe68a000640 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b2e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.489+0000 7fe68a000640 1 -- 172.21.15.32:0/238375080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 msgr2=0x55a2d5b3a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.489+0000 7fe68a000640 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 0x55a2d5b3a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.489+0000 7fe68a000640 1 -- 172.21.15.32:0/238375080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d5a508b0 con 0x55a2d575c390 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.490+0000 7fe68a000640 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b2ec70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe6780cb6d0 tx=0x7fe678108f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cd70ddb827a6ff server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.490+0000 7fe674c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.490+0000 7fe674c00640 1 -- 172.21.15.32:0/238375080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6780c5e60 con 0x55a2d575c390 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.490+0000 7fe674c00640 1 -- 172.21.15.32:0/238375080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe678102a20 con 0x55a2d575c390 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.490+0000 7fe6987fc740 1 -- 172.21.15.32:0/238375080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe684004050 con 0x55a2d575c390 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.490+0000 7fe674c00640 1 -- 172.21.15.32:0/238375080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe6780c7600 con 0x55a2d575c390 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.490+0000 7fe6987fc740 10 client.16401 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.490+0000 7fe6987fc740 1 -- 172.21.15.32:0/238375080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a2d5b3d3c0 con 0x55a2d575c390 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.490+0000 7fe674c00640 1 -- 172.21.15.32:0/238375080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684004050 con 0x55a2d575c390 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.491+0000 7fe674c00640 1 -- 172.21.15.32:0/238375080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe67804d060 con 0x55a2d575c390 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.491+0000 7fe6987fc740 10 client.16401 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.491+0000 7fe6987fc740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.491+0000 7fe6987fc740 20 client.16401 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.491+0000 7fe6987fc740 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a2d573b9f0 0x55a2d5b40930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.491+0000 7fe6987fc740 4 client.16401 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:41.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.491+0000 7fe6987fc740 1 -- 172.21.15.32:0/238375080 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2d5a9b700 con 0x55a2d573b9f0 2024-08-31T07:16:41.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.491+0000 7fe68aa00640 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a2d573b9f0 0x55a2d5b40930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.492+0000 7fe68aa00640 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a2d573b9f0 0x55a2d5b40930 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f84f4232597135 server_cookie=d1f530bba8d3708d in_seq=0 out_seq=0 2024-08-31T07:16:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.492+0000 7fe674c00640 10 client.16401 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe674c00640 1 -- 172.21.15.32:0/238375080 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2d5a9b700 con 0x55a2d573b9f0 2024-08-31T07:16:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe674c00640 10 client.16401 handle_command_reply: tid=0 2024-08-31T07:16:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 client.16401 shutdown 2024-08-31T07:16:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 -- 172.21.15.32:0/238375080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 msgr2=0x55a2d5b2ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b2ec70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe6780cb6d0 tx=0x7fe678108f10 comp rx=0 tx=0).stop 2024-08-31T07:16:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 -- 172.21.15.32:0/238375080 shutdown_connections 2024-08-31T07:16:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a2d573b9f0 0x55a2d5b40930 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d577d070 0x55a2d5b3a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b2ec70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 --2- 172.21.15.32:0/238375080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b2e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 -- 172.21.15.32:0/238375080 >> 172.21.15.32:0/238375080 conn(0x55a2d5771460 msgr2=0x55a2d5b25cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe674c00640 0 client.16401 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 -- 172.21.15.32:0/238375080 shutdown_connections 2024-08-31T07:16:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 1 -- 172.21.15.32:0/238375080 wait complete. 2024-08-31T07:16:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.493+0000 7fe6987fc740 20 client.16401 trim_cache size 0 max 16384 2024-08-31T07:16:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.507+0000 7fe6987fc740 1 Processor -- start 2024-08-31T07:16:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe6987fc740 1 -- start start 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b28c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d575c390 0x55a2d5b242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d577d070 0x55a2d5b2b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe6987fc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1b1300 con 0x55a2d577d070 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe6987fc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1b2f00 con 0x55a2d575c390 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe6987fc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1b19d0 con 0x55a2d57541b0 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe68b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d577d070 0x55a2d5b2b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe68b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d577d070 0x55a2d5b2b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60474/0 (socket says 172.21.15.32:60474) 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe68b400640 1 -- 172.21.15.32:0/3421016386 learned_addr learned my addr 172.21.15.32:0/3421016386 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe68a000640 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d575c390 0x55a2d5b242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.508+0000 7fe68aa00640 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b28c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.509+0000 7fe68aa00640 1 -- 172.21.15.32:0/3421016386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d575c390 msgr2=0x55a2d5b242a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.509+0000 7fe68aa00640 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d575c390 0x55a2d5b242a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.509+0000 7fe68aa00640 1 -- 172.21.15.32:0/3421016386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d577d070 msgr2=0x55a2d5b2b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.509+0000 7fe68aa00640 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d577d070 0x55a2d5b2b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.509+0000 7fe68aa00640 1 -- 172.21.15.32:0/3421016386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d5b1ac70 con 0x55a2d57541b0 2024-08-31T07:16:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.509+0000 7fe68b400640 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d577d070 0x55a2d5b2b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.509+0000 7fe68a000640 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d575c390 0x55a2d5b242a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.509+0000 7fe68aa00640 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b28c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55a2d5b3e8f0 tx=0x7fe67c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edf32a30b6e4c9bb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.509+0000 7fe677400640 1 -- 172.21.15.32:0/3421016386 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c003da0 con 0x55a2d57541b0 2024-08-31T07:16:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe677400640 1 -- 172.21.15.32:0/3421016386 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe67c00fa10 con 0x55a2d57541b0 2024-08-31T07:16:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe677400640 1 -- 172.21.15.32:0/3421016386 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c02a980 con 0x55a2d57541b0 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe6987fc740 1 -- 172.21.15.32:0/3421016386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 msgr2=0x55a2d5b28c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe6987fc740 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b28c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55a2d5b3e8f0 tx=0x7fe67c01b000 comp rx=0 tx=0).stop 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe6987fc740 1 -- 172.21.15.32:0/3421016386 shutdown_connections 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe6987fc740 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d577d070 0x55a2d5b2b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe6987fc740 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d575c390 0x55a2d5b242a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe6987fc740 1 --2- 172.21.15.32:0/3421016386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d57541b0 0x55a2d5b28c80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe6987fc740 1 -- 172.21.15.32:0/3421016386 >> 172.21.15.32:0/3421016386 conn(0x55a2d5771460 msgr2=0x55a2d5b23b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe6987fc740 1 -- 172.21.15.32:0/3421016386 shutdown_connections 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.510+0000 7fe6987fc740 1 -- 172.21.15.32:0/3421016386 wait complete. 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.511+0000 7fe6987fc740 1 -- start start 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.511+0000 7fe6987fc740 1 Processor -- start 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d57541b0 0x55a2d5b33170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b33750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe6987fc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d577d070 0x55a2d5ba4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe6987fc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1b2770 con 0x55a2d575c390 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe6987fc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1b20a0 con 0x55a2d57541b0 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe6987fc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1af2e0 con 0x55a2d577d070 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe6987fc740 10 client.0 _ll_register_callbacks cb 0x7fe6780c7c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe68a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b33750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe68a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b33750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60490/0 (socket says 172.21.15.32:60490) 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe68a000640 1 -- 172.21.15.32:0/2671337217 learned_addr learned my addr 172.21.15.32:0/2671337217 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe68aa00640 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d57541b0 0x55a2d5b33170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe68a000640 1 -- 172.21.15.32:0/2671337217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d577d070 msgr2=0x55a2d5ba4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe68b400640 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d577d070 0x55a2d5ba4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe68a000640 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d577d070 0x55a2d5ba4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe68a000640 1 -- 172.21.15.32:0/2671337217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d57541b0 msgr2=0x55a2d5b33170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe68a000640 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d57541b0 0x55a2d5b33170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.512+0000 7fe68a000640 1 -- 172.21.15.32:0/2671337217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d5ba4a80 con 0x55a2d575c390 2024-08-31T07:16:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.513+0000 7fe68aa00640 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d57541b0 0x55a2d5b33170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.513+0000 7fe68b400640 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d577d070 0x55a2d5ba4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:16:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.513+0000 7fe68a000640 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b33750 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe6780c77c0 tx=0x7fe67802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae1598fb5fc150b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.513+0000 7fe674c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.513+0000 7fe674c00640 1 -- 172.21.15.32:0/2671337217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6780c5e60 con 0x55a2d575c390 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.513+0000 7fe674c00640 1 -- 172.21.15.32:0/2671337217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe6780d47e0 con 0x55a2d575c390 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.513+0000 7fe6987fc740 1 -- 172.21.15.32:0/2671337217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe67c00fa10 con 0x55a2d575c390 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.513+0000 7fe674c00640 1 -- 172.21.15.32:0/2671337217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe6780caeb0 con 0x55a2d575c390 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.513+0000 7fe6987fc740 10 client.16407 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.513+0000 7fe6987fc740 1 -- 172.21.15.32:0/2671337217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a2d5babe70 con 0x55a2d575c390 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.514+0000 7fe674c00640 1 -- 172.21.15.32:0/2671337217 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe67c00fa10 con 0x55a2d575c390 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.514+0000 7fe674c00640 1 -- 172.21.15.32:0/2671337217 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fe6780f21a0 con 0x55a2d575c390 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.514+0000 7fe6987fc740 10 client.16407 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.514+0000 7fe6987fc740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:16:41.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.514+0000 7fe6987fc740 20 client.16407 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.514+0000 7fe6987fc740 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a2d5710da0 0x55a2d5bae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.514+0000 7fe6987fc740 4 client.16407 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.514+0000 7fe6987fc740 1 -- 172.21.15.32:0/2671337217 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2d5b2f000 con 0x55a2d5710da0 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.514+0000 7fe68aa00640 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a2d5710da0 0x55a2d5bae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.515+0000 7fe68aa00640 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a2d5710da0 0x55a2d5bae540 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5148e5854c870311 server_cookie=5dd855cc5471d979 in_seq=0 out_seq=0 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.515+0000 7fe674c00640 10 client.16407 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.515+0000 7fe674c00640 1 -- 172.21.15.32:0/2671337217 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2d5b2f000 con 0x55a2d5710da0 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.515+0000 7fe674c00640 10 client.16407 handle_command_reply: tid=0 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.515+0000 7fe6987fc740 1 client.16407 shutdown 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.515+0000 7fe6987fc740 1 -- 172.21.15.32:0/2671337217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 msgr2=0x55a2d5b33750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe6987fc740 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b33750 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe6780c77c0 tx=0x7fe67802a340 comp rx=0 tx=0).stop 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe6987fc740 1 -- 172.21.15.32:0/2671337217 shutdown_connections 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe6987fc740 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a2d5710da0 0x55a2d5bae540 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe6987fc740 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a2d577d070 0x55a2d5ba4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe6987fc740 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2d575c390 0x55a2d5b33750 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe674c00640 0 client.16407 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe6987fc740 1 --2- 172.21.15.32:0/2671337217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2d57541b0 0x55a2d5b33170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe6987fc740 1 -- 172.21.15.32:0/2671337217 >> 172.21.15.32:0/2671337217 conn(0x55a2d5771460 msgr2=0x55a2d5b24550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe6987fc740 1 -- 172.21.15.32:0/2671337217 shutdown_connections 2024-08-31T07:16:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe6987fc740 1 -- 172.21.15.32:0/2671337217 wait complete. 2024-08-31T07:16:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.516+0000 7fe6987fc740 20 client.16407 trim_cache size 0 max 16384 2024-08-31T07:16:41.520 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:16:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 -- 172.21.15.32:0/1740543636 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6700b4d20 msgr2=0x7fe6700b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6700b4d20 0x7fe6700b7230 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55a2d5b217e0 tx=0x7fe67c009260 comp rx=0 tx=0).stop 2024-08-31T07:16:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 -- 172.21.15.32:0/1740543636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe68c109090 msgr2=0x7fe68c19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe68c109090 0x7fe68c19c860 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe678035040 tx=0x7fe678032d50 comp rx=0 tx=0).stop 2024-08-31T07:16:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 -- 172.21.15.32:0/1740543636 shutdown_connections 2024-08-31T07:16:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6700b4d20 0x7fe6700b7230 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe68c1099e0 0x7fe68c1a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe68c109090 0x7fe68c19c860 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 --2- 172.21.15.32:0/1740543636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c103750 0x7fe68c19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 -- 172.21.15.32:0/1740543636 >> 172.21.15.32:0/1740543636 conn(0x7fe68c0ff1e0 msgr2=0x7fe68c10e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 -- 172.21.15.32:0/1740543636 shutdown_connections 2024-08-31T07:16:41.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:41.517+0000 7fe68be00640 1 -- 172.21.15.32:0/1740543636 wait complete. 2024-08-31T07:16:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:41 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/749879266' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:41.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.64 MB/sec execute 30 sec latency 20468.468 ms 2024-08-31T07:16:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:41 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/749879266' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:16:42.325 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 9fad7fc3 --release --await' 2024-08-31T07:16:42.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 9fad7fc3 --release --await 2024-08-31T07:16:42.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:42 smithi032 ceph-mon[17252]: pgmap v486: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-08-31T07:16:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:42 smithi073 ceph-mon[21995]: pgmap v486: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-08-31T07:16:42.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.62 MB/sec execute 31 sec latency 21468.642 ms 2024-08-31T07:16:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:42 smithi084 ceph-mon[22387]: pgmap v486: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-08-31T07:16:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.647+0000 7f2e42600640 1 -- 172.21.15.32:0/1556942908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3c103900 msgr2=0x7f2e3c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.647+0000 7f2e42600640 1 --2- 172.21.15.32:0/1556942908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3c103900 0x7f2e3c103d10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009900 tx=0x7f2e2c02a410 comp rx=0 tx=0).stop 2024-08-31T07:16:43.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.647+0000 7f2e42600640 1 -- 172.21.15.32:0/1556942908 shutdown_connections 2024-08-31T07:16:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.647+0000 7f2e42600640 1 --2- 172.21.15.32:0/1556942908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e3c109b90 0x7f2e3c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.647+0000 7f2e42600640 1 --2- 172.21.15.32:0/1556942908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e3c109240 0x7f2e3c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.647+0000 7f2e42600640 1 --2- 172.21.15.32:0/1556942908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3c103900 0x7f2e3c103d10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.647+0000 7f2e42600640 1 -- 172.21.15.32:0/1556942908 >> 172.21.15.32:0/1556942908 conn(0x7f2e3c07b5f0 msgr2=0x7f2e3c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.647+0000 7f2e42600640 1 -- 172.21.15.32:0/1556942908 shutdown_connections 2024-08-31T07:16:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.647+0000 7f2e42600640 1 -- 172.21.15.32:0/1556942908 wait complete. 2024-08-31T07:16:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.648+0000 7f2e42600640 1 Processor -- start 2024-08-31T07:16:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.648+0000 7f2e42600640 1 -- start start 2024-08-31T07:16:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e42600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e3c103900 0x7f2e3c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e42600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e3c109240 0x7f2e3c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e42600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3c109b90 0x7f2e3c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e42600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1138a0 con 0x7f2e3c109b90 2024-08-31T07:16:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e42600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c113720 con 0x7f2e3c109240 2024-08-31T07:16:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e42600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c113a20 con 0x7f2e3c103900 2024-08-31T07:16:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e3be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3c109b90 0x7f2e3c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e3b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e3c103900 0x7f2e3c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e3b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e3c103900 0x7f2e3c19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41138/0 (socket says 172.21.15.32:41138) 2024-08-31T07:16:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e3b400640 1 -- 172.21.15.32:0/750276504 learned_addr learned my addr 172.21.15.32:0/750276504 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:43.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.649+0000 7f2e32a00640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e3c109240 0x7f2e3c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e3b400640 1 -- 172.21.15.32:0/750276504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e3c109240 msgr2=0x7f2e3c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e3b400640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e3c109240 0x7f2e3c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e3b400640 1 -- 172.21.15.32:0/750276504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3c109b90 msgr2=0x7f2e3c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e3b400640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3c109b90 0x7f2e3c19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e3b400640 1 -- 172.21.15.32:0/750276504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c1a0280 con 0x7f2e3c103900 2024-08-31T07:16:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e3be00640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3c109b90 0x7f2e3c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e3b400640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e3c103900 0x7f2e3c19a310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c005ab0 tx=0x7f2e2c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e38c00640 1 -- 172.21.15.32:0/750276504 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c03f070 con 0x7f2e3c103900 2024-08-31T07:16:43.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e38c00640 1 -- 172.21.15.32:0/750276504 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2c031b60 con 0x7f2e3c103900 2024-08-31T07:16:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e42600640 1 -- 172.21.15.32:0/750276504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3c1a0510 con 0x7f2e3c103900 2024-08-31T07:16:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.650+0000 7f2e38c00640 1 -- 172.21.15.32:0/750276504 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c035030 con 0x7f2e3c103900 2024-08-31T07:16:43.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.651+0000 7f2e42600640 1 -- 172.21.15.32:0/750276504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3c1a6cd0 con 0x7f2e3c103900 2024-08-31T07:16:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.654+0000 7f2e49249740 1 Processor -- start 2024-08-31T07:16:43.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.654+0000 7f2e49249740 1 -- start start 2024-08-31T07:16:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.654+0000 7f2e49249740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526575f520 0x565265b04ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.654+0000 7f2e38c00640 1 -- 172.21.15.32:0/750276504 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2e2c042070 con 0x7f2e3c103900 2024-08-31T07:16:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.654+0000 7f2e49249740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565265736660 0x565265b05430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.654+0000 7f2e49249740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b05970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.654+0000 7f2e49249740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x565265a7ba90 con 0x565265736660 2024-08-31T07:16:43.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.654+0000 7f2e49249740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x565265a6b020 con 0x56526575f520 2024-08-31T07:16:43.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.654+0000 7f2e49249740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x565265a6efd0 con 0x56526573e840 2024-08-31T07:16:43.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.654+0000 7f2e3b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565265736660 0x565265b05430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e32a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b05970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526575f520 0x565265b04ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e3b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565265736660 0x565265b05430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60518/0 (socket says 172.21.15.32:60518) 2024-08-31T07:16:43.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e32a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b05970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41152/0 (socket says 172.21.15.32:41152) 2024-08-31T07:16:43.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e3b400640 1 -- 172.21.15.32:0/887783720 learned_addr learned my addr 172.21.15.32:0/887783720 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e3be00640 1 -- 172.21.15.32:0/887783720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 msgr2=0x565265b05970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e3be00640 1 --2- 172.21.15.32:0/887783720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b05970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e3be00640 1 -- 172.21.15.32:0/887783720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565265736660 msgr2=0x565265b05430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e3be00640 1 --2- 172.21.15.32:0/887783720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565265736660 0x565265b05430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e3be00640 1 -- 172.21.15.32:0/887783720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652657cab00 con 0x56526575f520 2024-08-31T07:16:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.655+0000 7f2e3be00640 1 --2- 172.21.15.32:0/887783720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526575f520 0x565265b04ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2e34005fd0 tx=0x7f2e34011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96089b62c602f90f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.656+0000 7f2e30c00640 1 -- 172.21.15.32:0/887783720 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34004d20 con 0x56526575f520 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.656+0000 7f2e30c00640 1 -- 172.21.15.32:0/887783720 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e3400f9a0 con 0x56526575f520 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.656+0000 7f2e30c00640 1 -- 172.21.15.32:0/887783720 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34004450 con 0x56526575f520 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.656+0000 7f2e38c00640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e200b4ca0 0x7f2e200b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.656+0000 7f2e38c00640 1 -- 172.21.15.32:0/750276504 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2e2c105c00 con 0x7f2e3c103900 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.656+0000 7f2e3b400640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e200b4ca0 0x7f2e200b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.657+0000 7f2e49249740 1 -- 172.21.15.32:0/887783720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526575f520 msgr2=0x565265b04ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.657+0000 7f2e49249740 1 --2- 172.21.15.32:0/887783720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526575f520 0x565265b04ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2e34005fd0 tx=0x7f2e34011bf0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.657+0000 7f2e49249740 1 -- 172.21.15.32:0/887783720 shutdown_connections 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.657+0000 7f2e49249740 1 --2- 172.21.15.32:0/887783720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b05970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.657+0000 7f2e49249740 1 --2- 172.21.15.32:0/887783720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565265736660 0x565265b05430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.657+0000 7f2e49249740 1 --2- 172.21.15.32:0/887783720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526575f520 0x565265b04ef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.657+0000 7f2e49249740 1 -- 172.21.15.32:0/887783720 >> 172.21.15.32:0/887783720 conn(0x565265753910 msgr2=0x565265b02de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.657+0000 7f2e49249740 1 -- 172.21.15.32:0/887783720 shutdown_connections 2024-08-31T07:16:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.657+0000 7f2e49249740 1 -- 172.21.15.32:0/887783720 wait complete. 2024-08-31T07:16:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.657+0000 7f2e3b400640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e200b4ca0 0x7f2e200b71b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c031440 tx=0x7f2e2c1096b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.658+0000 7f2e49249740 1 -- start start 2024-08-31T07:16:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.658+0000 7f2e49249740 1 Processor -- start 2024-08-31T07:16:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.658+0000 7f2e49249740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565265736660 0x565265b170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e49249740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e49249740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b14cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e49249740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5652658cdbb0 con 0x56526575f520 2024-08-31T07:16:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e49249740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5652659f1f90 con 0x565265736660 2024-08-31T07:16:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e49249740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x565265b17a00 con 0x56526573e840 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e49249740 10 client.0 _ll_register_callbacks cb 0x565265a17710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e32a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565265736660 0x565265b170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e32a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b145c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41166/0 (socket says 172.21.15.32:41166) 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e32a00640 1 -- 172.21.15.32:0/699214358 learned_addr learned my addr 172.21.15.32:0/699214358 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e3be00640 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b14cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e32a00640 1 -- 172.21.15.32:0/699214358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565265736660 msgr2=0x565265b170d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e32a00640 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565265736660 0x565265b170d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e32a00640 1 -- 172.21.15.32:0/699214358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 msgr2=0x565265b14cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e32a00640 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b14cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e32a00640 1 -- 172.21.15.32:0/699214358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565265841960 con 0x56526573e840 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.659+0000 7f2e3be00640 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b14cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.660+0000 7f2e32a00640 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b145c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2e24012700 tx=0x7f2e24012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d1586262be3da5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.660+0000 7f2e29600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.660+0000 7f2e29600640 1 -- 172.21.15.32:0/699214358 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2400f920 con 0x56526573e840 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.660+0000 7f2e49249740 1 -- 172.21.15.32:0/699214358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565265a09e30 con 0x56526573e840 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.660+0000 7f2e29600640 1 -- 172.21.15.32:0/699214358 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2e2400fac0 con 0x56526573e840 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.660+0000 7f2e29600640 1 -- 172.21.15.32:0/699214358 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2e2400fcc0 con 0x56526573e840 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.660+0000 7f2e49249740 10 client.26446 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.660+0000 7f2e49249740 1 -- 172.21.15.32:0/699214358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565265b233f0 con 0x56526573e840 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.660+0000 7f2e29600640 1 -- 172.21.15.32:0/699214358 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565265a09e30 con 0x56526573e840 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.661+0000 7f2e29600640 1 -- 172.21.15.32:0/699214358 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f2e2401c420 con 0x56526573e840 2024-08-31T07:16:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.661+0000 7f2e49249740 10 client.26446 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.661+0000 7f2e49249740 10 client.26446 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:16:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.661+0000 7f2e49249740 20 client.26446 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.661+0000 7f2e49249740 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56526571dea0 0x565265b26a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.661+0000 7f2e49249740 4 client.26446 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.661+0000 7f2e49249740 1 -- 172.21.15.32:0/699214358 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565265a7d9e0 con 0x56526571dea0 2024-08-31T07:16:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.661+0000 7f2e3be00640 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56526571dea0 0x565265b26a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.662+0000 7f2e3be00640 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56526571dea0 0x565265b26a00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=864f9be68fcc5971 server_cookie=338de46902e67988 in_seq=0 out_seq=0 2024-08-31T07:16:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.662+0000 7f2e29600640 10 client.26446 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e29600640 1 -- 172.21.15.32:0/699214358 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565265a7d9e0 con 0x56526571dea0 2024-08-31T07:16:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e29600640 10 client.26446 handle_command_reply: tid=0 2024-08-31T07:16:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 client.26446 shutdown 2024-08-31T07:16:43.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 -- 172.21.15.32:0/699214358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 msgr2=0x565265b145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b145c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2e24012700 tx=0x7f2e24012bc0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 -- 172.21.15.32:0/699214358 shutdown_connections 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56526571dea0 0x565265b26a00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b14cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b145c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 --2- 172.21.15.32:0/699214358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565265736660 0x565265b170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e29600640 0 client.26446 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 -- 172.21.15.32:0/699214358 >> 172.21.15.32:0/699214358 conn(0x565265753910 msgr2=0x565265b07f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 -- 172.21.15.32:0/699214358 shutdown_connections 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 1 -- 172.21.15.32:0/699214358 wait complete. 2024-08-31T07:16:43.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.663+0000 7f2e49249740 20 client.26446 trim_cache size 0 max 16384 2024-08-31T07:16:43.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.685+0000 7f2e49249740 1 Processor -- start 2024-08-31T07:16:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.685+0000 7f2e49249740 1 -- start start 2024-08-31T07:16:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e49249740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565265736660 0x565265b20c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e49249740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526573e840 0x565265b16480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e49249740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b06630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e49249740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1ad1b0 con 0x56526575f520 2024-08-31T07:16:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e49249740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1aedb0 con 0x56526573e840 2024-08-31T07:16:43.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e49249740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1ad880 con 0x565265736660 2024-08-31T07:16:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e3be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565265736660 0x565265b20c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e3be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565265736660 0x565265b20c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41174/0 (socket says 172.21.15.32:41174) 2024-08-31T07:16:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e3be00640 1 -- 172.21.15.32:0/3611012661 learned_addr learned my addr 172.21.15.32:0/3611012661 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:43.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e3be00640 1 --2- 172.21.15.32:0/3611012661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b06630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.686+0000 7f2e32a00640 1 --2- 172.21.15.32:0/3611012661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526573e840 0x565265b16480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.687+0000 7f2e3be00640 1 -- 172.21.15.32:0/3611012661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526573e840 msgr2=0x565265b16480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.687+0000 7f2e3be00640 1 --2- 172.21.15.32:0/3611012661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526573e840 0x565265b16480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.687+0000 7f2e3be00640 1 -- 172.21.15.32:0/3611012661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 msgr2=0x565265b06630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.687+0000 7f2e3be00640 1 --2- 172.21.15.32:0/3611012661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b06630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.687+0000 7f2e3be00640 1 -- 172.21.15.32:0/3611012661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565265b16ce0 con 0x565265736660 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.687+0000 7f2e32a00640 1 --2- 172.21.15.32:0/3611012661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526573e840 0x565265b16480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.687+0000 7f2e3be00640 1 --2- 172.21.15.32:0/3611012661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565265736660 0x565265b20c30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2e34004130 tx=0x7f2e34009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2775b26ba2534159 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.687+0000 7f2e2be00640 1 -- 172.21.15.32:0/3611012661 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3401e040 con 0x565265736660 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.688+0000 7f2e2be00640 1 -- 172.21.15.32:0/3611012661 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e3400fe50 con 0x565265736660 2024-08-31T07:16:43.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.688+0000 7f2e2be00640 1 -- 172.21.15.32:0/3611012661 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340047d0 con 0x565265736660 2024-08-31T07:16:43.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.688+0000 7f2e49249740 1 -- 172.21.15.32:0/3611012661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565265736660 msgr2=0x565265b20c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.688+0000 7f2e49249740 1 --2- 172.21.15.32:0/3611012661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565265736660 0x565265b20c30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2e34004130 tx=0x7f2e34009500 comp rx=0 tx=0).stop 2024-08-31T07:16:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.689+0000 7f2e49249740 1 -- 172.21.15.32:0/3611012661 shutdown_connections 2024-08-31T07:16:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.689+0000 7f2e49249740 1 --2- 172.21.15.32:0/3611012661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b06630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.689+0000 7f2e49249740 1 --2- 172.21.15.32:0/3611012661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56526573e840 0x565265b16480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.689+0000 7f2e49249740 1 --2- 172.21.15.32:0/3611012661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565265736660 0x565265b20c30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.689+0000 7f2e49249740 1 -- 172.21.15.32:0/3611012661 >> 172.21.15.32:0/3611012661 conn(0x565265753910 msgr2=0x565265b05f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.689+0000 7f2e49249740 1 -- 172.21.15.32:0/3611012661 shutdown_connections 2024-08-31T07:16:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.689+0000 7f2e49249740 1 -- 172.21.15.32:0/3611012661 wait complete. 2024-08-31T07:16:43.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.689+0000 7f2e49249740 1 -- start start 2024-08-31T07:16:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.689+0000 7f2e49249740 1 Processor -- start 2024-08-31T07:16:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e49249740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565265736660 0x565265b8a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e49249740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b8ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e49249740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b19400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e49249740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1ae620 con 0x56526575f520 2024-08-31T07:16:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e49249740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1adf50 con 0x565265736660 2024-08-31T07:16:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e49249740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c1ab190 con 0x56526573e840 2024-08-31T07:16:43.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e49249740 10 client.0 _ll_register_callbacks cb 0x7f2e24009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e32a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b8ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e32a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b8ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41182/0 (socket says 172.21.15.32:41182) 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e32a00640 1 -- 172.21.15.32:0/3932406478 learned_addr learned my addr 172.21.15.32:0/3932406478 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e3be00640 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565265736660 0x565265b8a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.690+0000 7f2e3be00640 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b19400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e32a00640 1 -- 172.21.15.32:0/3932406478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565265736660 msgr2=0x565265b8a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e32a00640 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565265736660 0x565265b8a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e32a00640 1 -- 172.21.15.32:0/3932406478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 msgr2=0x565265b19400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e32a00640 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b19400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e32a00640 1 -- 172.21.15.32:0/3932406478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565265b19b70 con 0x56526573e840 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e32a00640 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b8ac50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x565265b17650 tx=0x7f2e2401c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f48c7e909570a36 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:43.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e29600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e29600640 1 -- 172.21.15.32:0/3932406478 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24029070 con 0x56526573e840 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e29600640 1 -- 172.21.15.32:0/3932406478 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2e24005110 con 0x56526573e840 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e49249740 1 -- 172.21.15.32:0/3932406478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e3400fe50 con 0x56526573e840 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e29600640 1 -- 172.21.15.32:0/3932406478 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2e24035050 con 0x56526573e840 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.691+0000 7f2e49249740 10 client.26452 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.692+0000 7f2e49249740 1 -- 172.21.15.32:0/3932406478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565265b91e20 con 0x56526573e840 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.692+0000 7f2e29600640 1 -- 172.21.15.32:0/3932406478 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e3400fe50 con 0x56526573e840 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.692+0000 7f2e29600640 1 -- 172.21.15.32:0/3932406478 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f2e24003b20 con 0x56526573e840 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.692+0000 7f2e49249740 10 client.26452 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.692+0000 7f2e49249740 10 client.26452 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.692+0000 7f2e49249740 20 client.26452 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.692+0000 7f2e49249740 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5652656f3250 0x565265b946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.692+0000 7f2e49249740 4 client.26452 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9fad7fc3", "release": true, "await": true}] 2024-08-31T07:16:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.692+0000 7f2e49249740 1 -- 172.21.15.32:0/3932406478 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9fad7fc3", "release": true, "await": true}) -- 0x565265a79480 con 0x5652656f3250 2024-08-31T07:16:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.692+0000 7f2e3be00640 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5652656f3250 0x565265b946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.693+0000 7f2e3be00640 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5652656f3250 0x565265b946d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2005895716e1dc4b server_cookie=37783ba30f466250 in_seq=0 out_seq=0 2024-08-31T07:16:43.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.693+0000 7f2e29600640 10 client.26452 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:43.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107242 0.60 MB/sec execute 32 sec latency 22468.716 ms 2024-08-31T07:16:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.702+0000 7f2e29600640 1 -- 172.21.15.32:0/3932406478 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565265a79480 con 0x5652656f3250 2024-08-31T07:16:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.702+0000 7f2e29600640 10 client.26452 handle_command_reply: tid=0 2024-08-31T07:16:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.702+0000 7f2e49249740 1 client.26452 shutdown 2024-08-31T07:16:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 1 -- 172.21.15.32:0/3932406478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 msgr2=0x565265b8ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b8ac50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x565265b17650 tx=0x7f2e2401c6d0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 1 -- 172.21.15.32:0/3932406478 shutdown_connections 2024-08-31T07:16:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5652656f3250 0x565265b946d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56526575f520 0x565265b19400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56526573e840 0x565265b8ac50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 1 --2- 172.21.15.32:0/3932406478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565265736660 0x565265b8a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 1 -- 172.21.15.32:0/3932406478 >> 172.21.15.32:0/3932406478 conn(0x565265753910 msgr2=0x565265b066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e29600640 0 client.26452 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 1 -- 172.21.15.32:0/3932406478 shutdown_connections 2024-08-31T07:16:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 1 -- 172.21.15.32:0/3932406478 wait complete. 2024-08-31T07:16:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.703+0000 7f2e49249740 20 client.26452 trim_cache size 0 max 16384 2024-08-31T07:16:43.705 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:16:43.705 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 31, 2024-08-31T07:16:43.705 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 16, 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "9fad7fc3": { 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 16, 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 141.8, 2024-08-31T07:16:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:43.707 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:16:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 -- 172.21.15.32:0/750276504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e200b4ca0 msgr2=0x7f2e200b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e200b4ca0 0x7f2e200b71b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c031440 tx=0x7f2e2c1096b0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 -- 172.21.15.32:0/750276504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e3c103900 msgr2=0x7f2e3c19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e3c103900 0x7f2e3c19a310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c005ab0 tx=0x7f2e2c003c10 comp rx=0 tx=0).stop 2024-08-31T07:16:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 -- 172.21.15.32:0/750276504 shutdown_connections 2024-08-31T07:16:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e200b4ca0 0x7f2e200b71b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3c109b90 0x7f2e3c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e3c109240 0x7f2e3c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 --2- 172.21.15.32:0/750276504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e3c103900 0x7f2e3c19a310 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 -- 172.21.15.32:0/750276504 >> 172.21.15.32:0/750276504 conn(0x7f2e3c07b5f0 msgr2=0x7f2e3c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 -- 172.21.15.32:0/750276504 shutdown_connections 2024-08-31T07:16:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:43.704+0000 7f2e42600640 1 -- 172.21.15.32:0/750276504 wait complete. 2024-08-31T07:16:43.853 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:16:43.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:16:44.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:44 smithi032 ceph-mon[17252]: pgmap v487: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 KiB/s wr, 16 op/s 2024-08-31T07:16:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:44 smithi073 ceph-mon[21995]: pgmap v487: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 KiB/s wr, 16 op/s 2024-08-31T07:16:44.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 109789 1.01 MB/sec execute 33 sec latency 22478.714 ms 2024-08-31T07:16:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:44 smithi084 ceph-mon[22387]: pgmap v487: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 KiB/s wr, 16 op/s 2024-08-31T07:16:44.811 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9fad7fc3', total seconds elapsed: 27.3 2024-08-31T07:16:44.855 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:45.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.618+0000 7fd5c1c00640 1 -- 172.21.15.32:0/3411004762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5bc0754f0 msgr2=0x7fd5bc075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:45.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.618+0000 7fd5c1c00640 1 --2- 172.21.15.32:0/3411004762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5bc0754f0 0x7fd5bc075900 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009900 tx=0x7fd5ac02a410 comp rx=0 tx=0).stop 2024-08-31T07:16:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.618+0000 7fd5c1c00640 1 -- 172.21.15.32:0/3411004762 shutdown_connections 2024-08-31T07:16:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.618+0000 7fd5c1c00640 1 --2- 172.21.15.32:0/3411004762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5bc10fcf0 0x7fd5bc112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.618+0000 7fd5c1c00640 1 --2- 172.21.15.32:0/3411004762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5bc075e40 0x7fd5bc10f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.618+0000 7fd5c1c00640 1 --2- 172.21.15.32:0/3411004762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5bc0754f0 0x7fd5bc075900 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.618+0000 7fd5c1c00640 1 -- 172.21.15.32:0/3411004762 >> 172.21.15.32:0/3411004762 conn(0x7fd5bc0ff180 msgr2=0x7fd5bc1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.618+0000 7fd5c1c00640 1 -- 172.21.15.32:0/3411004762 shutdown_connections 2024-08-31T07:16:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.618+0000 7fd5c1c00640 1 -- 172.21.15.32:0/3411004762 wait complete. 2024-08-31T07:16:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.619+0000 7fd5c1c00640 1 Processor -- start 2024-08-31T07:16:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.619+0000 7fd5c1c00640 1 -- start start 2024-08-31T07:16:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5c1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5bc0754f0 0x7fd5bc19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5c1c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5bc075e40 0x7fd5bc19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5c1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5bc10fcf0 0x7fd5bc1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5c1c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc1158b0 con 0x7fd5bc0754f0 2024-08-31T07:16:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5c1c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc115730 con 0x7fd5bc10fcf0 2024-08-31T07:16:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5c1c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc115a30 con 0x7fd5bc075e40 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5bc075e40 0x7fd5bc19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5bbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5bc0754f0 0x7fd5bc19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5bc075e40 0x7fd5bc19c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41208/0 (socket says 172.21.15.32:41208) 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5b3e00640 1 -- 172.21.15.32:0/1955424259 learned_addr learned my addr 172.21.15.32:0/1955424259 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5c1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5bc10fcf0 0x7fd5bc1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5b3e00640 1 -- 172.21.15.32:0/1955424259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5bc10fcf0 msgr2=0x7fd5bc1a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5b3e00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5bc10fcf0 0x7fd5bc1a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5b3e00640 1 -- 172.21.15.32:0/1955424259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5bc0754f0 msgr2=0x7fd5bc19c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5b3e00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5bc0754f0 0x7fd5bc19c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5b3e00640 1 -- 172.21.15.32:0/1955424259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc1a21d0 con 0x7fd5bc075e40 2024-08-31T07:16:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.620+0000 7fd5c1200640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5bc10fcf0 0x7fd5bc1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:45.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.621+0000 7fd5b3e00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5bc075e40 0x7fd5bc19c800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400e970 tx=0x7fd5a400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:45.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.621+0000 7fd5b9600640 1 -- 172.21.15.32:0/1955424259 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a400cd80 con 0x7fd5bc075e40 2024-08-31T07:16:45.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.621+0000 7fd5c1c00640 1 -- 172.21.15.32:0/1955424259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc1a24c0 con 0x7fd5bc075e40 2024-08-31T07:16:45.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.621+0000 7fd5b9600640 1 -- 172.21.15.32:0/1955424259 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5a40035f0 con 0x7fd5bc075e40 2024-08-31T07:16:45.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.621+0000 7fd5b9600640 1 -- 172.21.15.32:0/1955424259 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4010690 con 0x7fd5bc075e40 2024-08-31T07:16:45.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.621+0000 7fd5c1c00640 1 -- 172.21.15.32:0/1955424259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc1a8c20 con 0x7fd5bc075e40 2024-08-31T07:16:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.624+0000 7fd5c9e85740 1 Processor -- start 2024-08-31T07:16:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.624+0000 7fd5c9e85740 1 -- start start 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5b9600640 1 -- 172.21.15.32:0/1955424259 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd5a4003aa0 con 0x7fd5bc075e40 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e4b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e4bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 0x564d76e4c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564d76dc1750 con 0x564d76a84670 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564d76db0ce0 con 0x564d76a7c490 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564d76db4c90 con 0x564d76aa5350 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e4b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e4b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41224/0 (socket says 172.21.15.32:41224) 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c1200640 1 -- 172.21.15.32:0/18513681 learned_addr learned my addr 172.21.15.32:0/18513681 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5b3e00640 1 --2- 172.21.15.32:0/18513681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 0x564d76e4c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/18513681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e4bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c1200640 1 -- 172.21.15.32:0/18513681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 msgr2=0x564d76e4bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.625+0000 7fd5c1200640 1 --2- 172.21.15.32:0/18513681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e4bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.626+0000 7fd5c1200640 1 -- 172.21.15.32:0/18513681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 msgr2=0x564d76e4c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.626+0000 7fd5c1200640 1 --2- 172.21.15.32:0/18513681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 0x564d76e4c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.626+0000 7fd5c1200640 1 -- 172.21.15.32:0/18513681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d76b10930 con 0x564d76aa5350 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.626+0000 7fd5b3e00640 1 --2- 172.21.15.32:0/18513681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 0x564d76e4c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.626+0000 7fd5c1200640 1 --2- 172.21.15.32:0/18513681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e4b7d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40117e0 tx=0x7fd5b4011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f00ec885f4e6511c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.626+0000 7fd5b1600640 1 -- 172.21.15.32:0/18513681 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4011310 con 0x564d76aa5350 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.626+0000 7fd5b1600640 1 -- 172.21.15.32:0/18513681 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b4003e20 con 0x564d76aa5350 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.626+0000 7fd5b1600640 1 -- 172.21.15.32:0/18513681 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b40044f0 con 0x564d76aa5350 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.627+0000 7fd5b9600640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5a00b4ca0 0x7fd5a00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.627+0000 7fd5b9600640 1 -- 172.21.15.32:0/1955424259 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fd5a40e5790 con 0x7fd5bc075e40 2024-08-31T07:16:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.627+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5a00b4ca0 0x7fd5a00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.627+0000 7fd5c9e85740 1 -- 172.21.15.32:0/18513681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 msgr2=0x564d76e4b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:45.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.627+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/18513681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e4b7d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40117e0 tx=0x7fd5b4011ca0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.627+0000 7fd5c9e85740 1 -- 172.21.15.32:0/18513681 shutdown_connections 2024-08-31T07:16:45.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.627+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/18513681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 0x564d76e4c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.627+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/18513681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e4bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.627+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/18513681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e4b7d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.627+0000 7fd5c9e85740 1 -- 172.21.15.32:0/18513681 >> 172.21.15.32:0/18513681 conn(0x564d76a99740 msgr2=0x564d76e496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.628+0000 7fd5c9e85740 1 -- 172.21.15.32:0/18513681 shutdown_connections 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.628+0000 7fd5c9e85740 1 -- 172.21.15.32:0/18513681 wait complete. 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.628+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5a00b4ca0 0x7fd5a00b71b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac005ab0 tx=0x7fd5ac005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.628+0000 7fd5c9e85740 1 -- start start 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.628+0000 7fd5c9e85740 1 Processor -- start 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e57370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 0x564d76e578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564d76c13890 con 0x564d76a84670 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564d76d37cd0 con 0x564d76a7c490 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564d76e59fb0 con 0x564d76aa5350 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5c9e85740 10 client.0 _ll_register_callbacks cb 0x564d76dabef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5bbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e57370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 0x564d76e578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5bbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e57370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45262/0 (socket says 172.21.15.32:45262) 2024-08-31T07:16:45.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5c1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.629+0000 7fd5bbe00640 1 -- 172.21.15.32:0/2879706525 learned_addr learned my addr 172.21.15.32:0/2879706525 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:45.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.630+0000 7fd5c1200640 1 -- 172.21.15.32:0/2879706525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 msgr2=0x564d76e57370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:45.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.630+0000 7fd5c1200640 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e57370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.630+0000 7fd5c1200640 1 -- 172.21.15.32:0/2879706525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 msgr2=0x564d76e578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.630+0000 7fd5c1200640 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 0x564d76e578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.630+0000 7fd5c1200640 1 -- 172.21.15.32:0/2879706525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d76d811b0 con 0x564d76aa5350 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.630+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e57370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.630+0000 7fd5b3e00640 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 0x564d76e578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.630+0000 7fd5c1200640 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e632a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40113c0 tx=0x7fd5b40113f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc288ed644d42dc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.630+0000 7fd5aa000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.631+0000 7fd5aa000640 1 -- 172.21.15.32:0/2879706525 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b402a060 con 0x564d76aa5350 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.631+0000 7fd5aa000640 1 -- 172.21.15.32:0/2879706525 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fd5b400f9a0 con 0x564d76aa5350 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.631+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2879706525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b4003e20 con 0x564d76aa5350 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.631+0000 7fd5aa000640 1 -- 172.21.15.32:0/2879706525 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5b4004e50 con 0x564d76aa5350 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.631+0000 7fd5c9e85740 10 client.26464 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.631+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2879706525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564d76e65fc0 con 0x564d76aa5350 2024-08-31T07:16:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.631+0000 7fd5aa000640 1 -- 172.21.15.32:0/2879706525 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b4003e20 con 0x564d76aa5350 2024-08-31T07:16:45.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.632+0000 7fd5aa000640 1 -- 172.21.15.32:0/2879706525 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fd5b4018020 con 0x564d76aa5350 2024-08-31T07:16:45.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.632+0000 7fd5c9e85740 10 client.26464 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:45.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.632+0000 7fd5c9e85740 10 client.26464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:16:45.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.632+0000 7fd5c9e85740 20 client.26464 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:45.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.632+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564d76a63cd0 0x564d76e69320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.632+0000 7fd5c9e85740 4 client.26464 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:45.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.632+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2879706525 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d76dc36a0 con 0x564d76a63cd0 2024-08-31T07:16:45.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.632+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564d76a63cd0 0x564d76e69320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.633+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564d76a63cd0 0x564d76e69320 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab87efbd24fbc0c5 server_cookie=592283896d4ac446 in_seq=0 out_seq=0 2024-08-31T07:16:45.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.633+0000 7fd5aa000640 10 client.26464 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:45.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5aa000640 1 -- 172.21.15.32:0/2879706525 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564d76dc36a0 con 0x564d76a63cd0 2024-08-31T07:16:45.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5aa000640 10 client.26464 handle_command_reply: tid=0 2024-08-31T07:16:45.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 client.26464 shutdown 2024-08-31T07:16:45.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2879706525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 msgr2=0x564d76e632a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e632a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40113c0 tx=0x7fd5b40113f0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2879706525 shutdown_connections 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564d76a63cd0 0x564d76e69320 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76aa5350 0x564d76e632a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76a84670 0x564d76e578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2879706525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e57370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5aa000640 0 client.26464 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2879706525 >> 172.21.15.32:0/2879706525 conn(0x564d76a99740 msgr2=0x564d76e4e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2879706525 shutdown_connections 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2879706525 wait complete. 2024-08-31T07:16:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.634+0000 7fd5c9e85740 20 client.26464 trim_cache size 0 max 16384 2024-08-31T07:16:45.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.653+0000 7fd5c9e85740 1 Processor -- start 2024-08-31T07:16:45.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.653+0000 7fd5c9e85740 1 -- start start 2024-08-31T07:16:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a7c490 0x564d76e50a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a84670 0x564d76e50f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76e4e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc1b1340 con 0x564d76aa5350 2024-08-31T07:16:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc1b2f40 con 0x564d76a84670 2024-08-31T07:16:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc1b1a10 con 0x564d76a7c490 2024-08-31T07:16:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5c1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76e4e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a84670 0x564d76e50f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a84670 0x564d76e50f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45268/0 (socket says 172.21.15.32:45268) 2024-08-31T07:16:45.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5c1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76e4e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60596/0 (socket says 172.21.15.32:60596) 2024-08-31T07:16:45.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5b3e00640 1 -- 172.21.15.32:0/726077033 learned_addr learned my addr 172.21.15.32:0/726077033 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:45.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/726077033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a7c490 0x564d76e50a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5bbe00640 1 -- 172.21.15.32:0/726077033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a84670 msgr2=0x564d76e50f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:45.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/726077033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a84670 0x564d76e50f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5bbe00640 1 -- 172.21.15.32:0/726077033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 msgr2=0x564d76e4e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:45.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.654+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/726077033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76e4e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.655+0000 7fd5bbe00640 1 -- 172.21.15.32:0/726077033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d76e52a00 con 0x564d76a7c490 2024-08-31T07:16:45.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.655+0000 7fd5c1200640 1 --2- 172.21.15.32:0/726077033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76e4e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:45.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.655+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/726077033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a7c490 0x564d76e50a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x564d76e69aa0 tx=0x7fd5ac03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=247294c82b42027f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:45.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.655+0000 7fd5b0c00640 1 -- 172.21.15.32:0/726077033 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac035040 con 0x564d76a7c490 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.655+0000 7fd5b0c00640 1 -- 172.21.15.32:0/726077033 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ac004e20 con 0x564d76a7c490 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.655+0000 7fd5b0c00640 1 -- 172.21.15.32:0/726077033 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac003e60 con 0x564d76a7c490 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.656+0000 7fd5c9e85740 1 -- 172.21.15.32:0/726077033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a7c490 msgr2=0x564d76e50a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.656+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/726077033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a7c490 0x564d76e50a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x564d76e69aa0 tx=0x7fd5ac03f000 comp rx=0 tx=0).stop 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.656+0000 7fd5c9e85740 1 -- 172.21.15.32:0/726077033 shutdown_connections 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.656+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/726077033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76e4e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.656+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/726077033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a84670 0x564d76e50f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.656+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/726077033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a7c490 0x564d76e50a30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.656+0000 7fd5c9e85740 1 -- 172.21.15.32:0/726077033 >> 172.21.15.32:0/726077033 conn(0x564d76a99740 msgr2=0x564d76e559f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.656+0000 7fd5c9e85740 1 -- 172.21.15.32:0/726077033 shutdown_connections 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.656+0000 7fd5c9e85740 1 -- 172.21.15.32:0/726077033 wait complete. 2024-08-31T07:16:45.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.657+0000 7fd5c9e85740 1 -- start start 2024-08-31T07:16:45.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.657+0000 7fd5c9e85740 1 Processor -- start 2024-08-31T07:16:45.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.657+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e5b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.657+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a84670 0x564d76e5baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.657+0000 7fd5c9e85740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76ed2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.657+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc1b27b0 con 0x564d76aa5350 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.657+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc1b20e0 con 0x564d76a7c490 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.657+0000 7fd5c9e85740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc1af320 con 0x564d76a84670 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.657+0000 7fd5c9e85740 10 client.0 _ll_register_callbacks cb 0x7fd5a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5c1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76ed2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a84670 0x564d76e5baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5c1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76ed2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60604/0 (socket says 172.21.15.32:60604) 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a84670 0x564d76e5baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41260/0 (socket says 172.21.15.32:41260) 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5c1200640 1 -- 172.21.15.32:0/2510712292 learned_addr learned my addr 172.21.15.32:0/2510712292 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e5b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5c1200640 1 -- 172.21.15.32:0/2510712292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a84670 msgr2=0x564d76e5baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5c1200640 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a84670 0x564d76e5baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5c1200640 1 -- 172.21.15.32:0/2510712292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 msgr2=0x564d76e5b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5c1200640 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e5b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5c1200640 1 -- 172.21.15.32:0/2510712292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d76ed3240 con 0x564d76aa5350 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5c1200640 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76ed2d00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4004160 tx=0x7fd5b4011570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9767b8a9577c8458 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.658+0000 7fd5aa000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.659+0000 7fd5aa000640 1 -- 172.21.15.32:0/2510712292 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4003c80 con 0x564d76aa5350 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.659+0000 7fd5aa000640 1 -- 172.21.15.32:0/2510712292 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fd5b4003e20 con 0x564d76aa5350 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.659+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2510712292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ac004e20 con 0x564d76aa5350 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.659+0000 7fd5aa000640 1 -- 172.21.15.32:0/2510712292 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5b4040050 con 0x564d76aa5350 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.659+0000 7fd5c9e85740 10 client.16443 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.659+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2510712292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564d76ed43a0 con 0x564d76aa5350 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.659+0000 7fd5aa000640 1 -- 172.21.15.32:0/2510712292 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ac004e20 con 0x564d76aa5350 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.659+0000 7fd5aa000640 1 -- 172.21.15.32:0/2510712292 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fd5b4037070 con 0x564d76aa5350 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.660+0000 7fd5c9e85740 10 client.16443 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:45.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.660+0000 7fd5c9e85740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:16:45.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.660+0000 7fd5c9e85740 20 client.16443 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:45.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.660+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564d76a39080 0x564d76ed6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:45.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.660+0000 7fd5c9e85740 4 client.16443 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:16:45.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.660+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2510712292 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564d76dbf140 con 0x564d76a39080 2024-08-31T07:16:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.660+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564d76a39080 0x564d76ed6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.660+0000 7fd5bbe00640 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564d76a39080 0x564d76ed6b00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e775e434bbde943b server_cookie=1abffa38f889dc91 in_seq=0 out_seq=0 2024-08-31T07:16:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.660+0000 7fd5aa000640 10 client.16443 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5aa000640 1 -- 172.21.15.32:0/2510712292 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564d76dbf140 con 0x564d76a39080 2024-08-31T07:16:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5aa000640 10 client.16443 handle_command_reply: tid=0 2024-08-31T07:16:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5c9e85740 1 client.16443 shutdown 2024-08-31T07:16:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2510712292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 msgr2=0x564d76ed2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76ed2d00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4004160 tx=0x7fd5b4011570 comp rx=0 tx=0).stop 2024-08-31T07:16:45.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2510712292 shutdown_connections 2024-08-31T07:16:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564d76a39080 0x564d76ed6b00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d76aa5350 0x564d76ed2d00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d76a84670 0x564d76e5baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5aa000640 0 client.16443 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5c9e85740 1 --2- 172.21.15.32:0/2510712292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d76a7c490 0x564d76e5b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2510712292 >> 172.21.15.32:0/2510712292 conn(0x564d76a99740 msgr2=0x564d76e4e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.662+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2510712292 shutdown_connections 2024-08-31T07:16:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.663+0000 7fd5c9e85740 1 -- 172.21.15.32:0/2510712292 wait complete. 2024-08-31T07:16:45.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.663+0000 7fd5c9e85740 20 client.16443 trim_cache size 0 max 16384 2024-08-31T07:16:45.665 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:16:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:16:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "fd7d31bf-f49a-41f6-8889-081d61bf9bb8", 2024-08-31T07:16:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:16:45.666 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:16:45.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.663+0000 7fd5c1c00640 1 -- 172.21.15.32:0/1955424259 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5a00b4ca0 msgr2=0x7fd5a00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.663+0000 7fd5c1c00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5a00b4ca0 0x7fd5a00b71b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac005ab0 tx=0x7fd5ac005a20 comp rx=0 tx=0).stop 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.663+0000 7fd5c1c00640 1 -- 172.21.15.32:0/1955424259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5bc075e40 msgr2=0x7fd5bc19c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.663+0000 7fd5c1c00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5bc075e40 0x7fd5bc19c800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400e970 tx=0x7fd5a400ee30 comp rx=0 tx=0).stop 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.664+0000 7fd5c1c00640 1 -- 172.21.15.32:0/1955424259 shutdown_connections 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.664+0000 7fd5c1c00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5a00b4ca0 0x7fd5a00b71b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.664+0000 7fd5c1c00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5bc10fcf0 0x7fd5bc1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.664+0000 7fd5c1c00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5bc075e40 0x7fd5bc19c800 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.664+0000 7fd5c1c00640 1 --2- 172.21.15.32:0/1955424259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5bc0754f0 0x7fd5bc19c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.664+0000 7fd5c1c00640 1 -- 172.21.15.32:0/1955424259 >> 172.21.15.32:0/1955424259 conn(0x7fd5bc0ff180 msgr2=0x7fd5bc109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.664+0000 7fd5c1c00640 1 -- 172.21.15.32:0/1955424259 shutdown_connections 2024-08-31T07:16:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:45.664+0000 7fd5c1c00640 1 -- 172.21.15.32:0/1955424259 wait complete. 2024-08-31T07:16:45.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 110791 1.13 MB/sec execute 34 sec latency 143.991 ms 2024-08-31T07:16:46.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: pgmap v488: 97 pgs: 97 active+clean; 67 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-08-31T07:16:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:16:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:16:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:16:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:16:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:16:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:46 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: pgmap v488: 97 pgs: 97 active+clean; 67 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-08-31T07:16:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:16:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:16:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:16:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:16:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:16:46.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:46 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:16:46.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 113172 1.46 MB/sec execute 35 sec latency 169.317 ms 2024-08-31T07:16:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: pgmap v488: 97 pgs: 97 active+clean; 67 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-08-31T07:16:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:16:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:16:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:16:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:16:46.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:16:46.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:46.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:46 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:16:47.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:16:47.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:16:47.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:16:47.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.586 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.586 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:16:47.586 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:16:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:16:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:16:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:16:47.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 114210 1.59 MB/sec execute 36 sec latency 176.797 ms 2024-08-31T07:16:47.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:16:47.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:16:47.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:16:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:16:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:16:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.766+0000 7f40f5c00640 1 -- 172.21.15.32:0/3570906878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f0075ea0 msgr2=0x7f40f010f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.766+0000 7f40f5c00640 1 --2- 172.21.15.32:0/3570906878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f0075ea0 0x7f40f010f770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009860 tx=0x7f40dc02a340 comp rx=0 tx=0).stop 2024-08-31T07:16:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.766+0000 7f40f5c00640 1 -- 172.21.15.32:0/3570906878 shutdown_connections 2024-08-31T07:16:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.766+0000 7f40f5c00640 1 --2- 172.21.15.32:0/3570906878 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f010fcb0 0x7f40f01120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.766+0000 7f40f5c00640 1 --2- 172.21.15.32:0/3570906878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f0075ea0 0x7f40f010f770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.766+0000 7f40f5c00640 1 --2- 172.21.15.32:0/3570906878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f0075550 0x7f40f0075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.766+0000 7f40f5c00640 1 -- 172.21.15.32:0/3570906878 >> 172.21.15.32:0/3570906878 conn(0x7f40f00ff1e0 msgr2=0x7f40f0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:47.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.766+0000 7f40f5c00640 1 -- 172.21.15.32:0/3570906878 shutdown_connections 2024-08-31T07:16:47.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.766+0000 7f40f5c00640 1 -- 172.21.15.32:0/3570906878 wait complete. 2024-08-31T07:16:47.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.767+0000 7f40f5c00640 1 Processor -- start 2024-08-31T07:16:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.767+0000 7f40f5c00640 1 -- start start 2024-08-31T07:16:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40f5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f0075550 0x7f40f019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40f5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f0075ea0 0x7f40f019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40f5c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f010fcb0 0x7f40f01a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40f5c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0115860 con 0x7f40f0075550 2024-08-31T07:16:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40f5c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01156e0 con 0x7f40f0075ea0 2024-08-31T07:16:47.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40f5c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01159e0 con 0x7f40f010fcb0 2024-08-31T07:16:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f0075ea0 0x7f40f019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f0075550 0x7f40f019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f0075ea0 0x7f40f019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46270/0 (socket says 172.21.15.32:46270) 2024-08-31T07:16:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40ef400640 1 -- 172.21.15.32:0/26599472 learned_addr learned my addr 172.21.15.32:0/26599472 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40f5200640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f010fcb0 0x7f40f01a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40ef400640 1 -- 172.21.15.32:0/26599472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f010fcb0 msgr2=0x7f40f01a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:47.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40ef400640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f010fcb0 0x7f40f01a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.768+0000 7f40ef400640 1 -- 172.21.15.32:0/26599472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f0075550 msgr2=0x7f40f019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.769+0000 7f40ef400640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f0075550 0x7f40f019c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.769+0000 7f40ef400640 1 -- 172.21.15.32:0/26599472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f01a2220 con 0x7f40f0075ea0 2024-08-31T07:16:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.769+0000 7f40ef400640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f0075ea0 0x7f40f019c850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f40dc035040 tx=0x7f40dc032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.769+0000 7f40ecc00640 1 -- 172.21.15.32:0/26599472 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc037040 con 0x7f40f0075ea0 2024-08-31T07:16:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.769+0000 7f40f5c00640 1 -- 172.21.15.32:0/26599472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f01a24b0 con 0x7f40f0075ea0 2024-08-31T07:16:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.769+0000 7f40f5c00640 1 -- 172.21.15.32:0/26599472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f01a8c70 con 0x7f40f0075ea0 2024-08-31T07:16:47.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.769+0000 7f40ecc00640 1 -- 172.21.15.32:0/26599472 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40dc041070 con 0x7f40f0075ea0 2024-08-31T07:16:47.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.769+0000 7f40ecc00640 1 -- 172.21.15.32:0/26599472 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0035f0 con 0x7f40f0075ea0 2024-08-31T07:16:47.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.773+0000 7f40fdddd740 1 Processor -- start 2024-08-31T07:16:47.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.773+0000 7f40fdddd740 1 -- start start 2024-08-31T07:16:47.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.773+0000 7f40ecc00640 1 -- 172.21.15.32:0/26599472 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f40dc0037b0 con 0x7f40f0075ea0 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.773+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 0x557cd62ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.773+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd6300060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.773+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd63005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.773+0000 7f40fdddd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557cd6276770 con 0x557cd5f39390 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.773+0000 7f40fdddd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557cd6265d00 con 0x557cd5f5a070 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.773+0000 7f40fdddd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557cd6269cb0 con 0x557cd5f311b0 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40ef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd63005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40ef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd63005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51772/0 (socket says 172.21.15.32:51772) 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40ef400640 1 -- 172.21.15.32:0/1097698057 learned_addr learned my addr 172.21.15.32:0/1097698057 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40efe00640 1 --2- 172.21.15.32:0/1097698057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd6300060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40f5200640 1 --2- 172.21.15.32:0/1097698057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 0x557cd62ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40ef400640 1 -- 172.21.15.32:0/1097698057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 msgr2=0x557cd6300060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40ef400640 1 --2- 172.21.15.32:0/1097698057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd6300060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40ef400640 1 -- 172.21.15.32:0/1097698057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 msgr2=0x557cd62ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40ef400640 1 --2- 172.21.15.32:0/1097698057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 0x557cd62ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40ef400640 1 -- 172.21.15.32:0/1097698057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd5fc5650 con 0x557cd5f39390 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.774+0000 7f40f5200640 1 --2- 172.21.15.32:0/1097698057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 0x557cd62ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.775+0000 7f40ef400640 1 --2- 172.21.15.32:0/1097698057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd63005a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f40dc102200 tx=0x7f40dc108bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1eacedb35de06c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.775+0000 7f40ecc00640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40d40b4ca0 0x7f40d40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.775+0000 7f40e5600640 1 -- 172.21.15.32:0/1097698057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0c7940 con 0x557cd5f39390 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.775+0000 7f40e5600640 1 -- 172.21.15.32:0/1097698057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40dc116070 con 0x557cd5f39390 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.775+0000 7f40e5600640 1 -- 172.21.15.32:0/1097698057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0cac40 con 0x557cd5f39390 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.775+0000 7f40f5200640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40d40b4ca0 0x7f40d40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.775+0000 7f40ecc00640 1 -- 172.21.15.32:0/26599472 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f40dc104b50 con 0x7f40f0075ea0 2024-08-31T07:16:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40fdddd740 1 -- 172.21.15.32:0/1097698057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 msgr2=0x557cd63005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40f5200640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40d40b4ca0 0x7f40d40b71b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009fd0 tx=0x7f40e8009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40fdddd740 1 --2- 172.21.15.32:0/1097698057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd63005a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f40dc102200 tx=0x7f40dc108bf0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40fdddd740 1 -- 172.21.15.32:0/1097698057 shutdown_connections 2024-08-31T07:16:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40fdddd740 1 --2- 172.21.15.32:0/1097698057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd63005a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40fdddd740 1 --2- 172.21.15.32:0/1097698057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd6300060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40fdddd740 1 --2- 172.21.15.32:0/1097698057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 0x557cd62ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40fdddd740 1 -- 172.21.15.32:0/1097698057 >> 172.21.15.32:0/1097698057 conn(0x557cd5f4e460 msgr2=0x557cd62fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40fdddd740 1 -- 172.21.15.32:0/1097698057 shutdown_connections 2024-08-31T07:16:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40fdddd740 1 -- 172.21.15.32:0/1097698057 wait complete. 2024-08-31T07:16:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.777+0000 7f40fdddd740 1 -- start start 2024-08-31T07:16:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.778+0000 7f40fdddd740 1 Processor -- start 2024-08-31T07:16:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.778+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f311b0 0x557cd630b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.778+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 0x557cd630bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.778+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f5a070 0x557cd631fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.778+0000 7f40fdddd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557cd60c8700 con 0x557cd5f311b0 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.778+0000 7f40fdddd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557cd61eccb0 con 0x557cd5f39390 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.778+0000 7f40fdddd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557cd63168d0 con 0x557cd5f5a070 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.778+0000 7f40fdddd740 10 client.0 _ll_register_callbacks cb 0x557cd6212430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 0x557cd630bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f311b0 0x557cd630b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 0x557cd630bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46284/0 (socket says 172.21.15.32:46284) 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40ef400640 1 -- 172.21.15.32:0/1119131705 learned_addr learned my addr 172.21.15.32:0/1119131705 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f311b0 0x557cd630b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51782/0 (socket says 172.21.15.32:51782) 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40f5200640 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f5a070 0x557cd631fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40ef400640 1 -- 172.21.15.32:0/1119131705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f5a070 msgr2=0x557cd631fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40ef400640 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f5a070 0x557cd631fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40ef400640 1 -- 172.21.15.32:0/1119131705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f311b0 msgr2=0x557cd630b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40ef400640 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f311b0 0x557cd630b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40ef400640 1 -- 172.21.15.32:0/1119131705 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd622d870 con 0x557cd5f39390 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40ef400640 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 0x557cd630bb50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0cb280 tx=0x7f40dc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=233866210a910ff7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40da000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.779+0000 7f40da000640 1 -- 172.21.15.32:0/1119131705 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0cae30 con 0x557cd5f39390 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40fdddd740 1 -- 172.21.15.32:0/1119131705 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40dc116070 con 0x557cd5f39390 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40da000640 1 -- 172.21.15.32:0/1119131705 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f40dc108e40 con 0x557cd5f39390 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40da000640 1 -- 172.21.15.32:0/1119131705 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f40dc0cb0c0 con 0x557cd5f39390 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40fdddd740 10 client.26468 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40fdddd740 1 -- 172.21.15.32:0/1119131705 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557cd6322730 con 0x557cd5f39390 2024-08-31T07:16:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40da000640 1 -- 172.21.15.32:0/1119131705 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40dc116070 con 0x557cd5f39390 2024-08-31T07:16:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40da000640 1 -- 172.21.15.32:0/1119131705 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f40dc078100 con 0x557cd5f39390 2024-08-31T07:16:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40fdddd740 10 client.26468 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40fdddd740 10 client.26468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:16:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40fdddd740 20 client.26468 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40fdddd740 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557cd5f189f0 0x557cd6325940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40fdddd740 4 client.26468 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:16:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.780+0000 7f40fdddd740 1 -- 172.21.15.32:0/1119131705 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cd62786c0 con 0x557cd5f189f0 2024-08-31T07:16:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.781+0000 7f40efe00640 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557cd5f189f0 0x557cd6325940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.781+0000 7f40efe00640 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557cd5f189f0 0x557cd6325940 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71fb61db95bf591c server_cookie=8a9e0c74721190e8 in_seq=0 out_seq=0 2024-08-31T07:16:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.781+0000 7f40da000640 10 client.26468 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40da000640 1 -- 172.21.15.32:0/1119131705 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557cd62786c0 con 0x557cd5f189f0 2024-08-31T07:16:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40da000640 10 client.26468 handle_command_reply: tid=0 2024-08-31T07:16:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40fdddd740 1 client.26468 shutdown 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40fdddd740 1 -- 172.21.15.32:0/1119131705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 msgr2=0x557cd630bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40fdddd740 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 0x557cd630bb50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0cb280 tx=0x7f40dc005f90 comp rx=0 tx=0).stop 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40fdddd740 1 -- 172.21.15.32:0/1119131705 shutdown_connections 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40fdddd740 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557cd5f189f0 0x557cd6325940 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40fdddd740 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f5a070 0x557cd631fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40fdddd740 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 0x557cd630bb50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40fdddd740 1 --2- 172.21.15.32:0/1119131705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f311b0 0x557cd630b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40da000640 0 client.26468 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.782+0000 7f40fdddd740 1 -- 172.21.15.32:0/1119131705 >> 172.21.15.32:0/1119131705 conn(0x557cd5f4e460 msgr2=0x557cd6302cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.783+0000 7f40fdddd740 1 -- 172.21.15.32:0/1119131705 shutdown_connections 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.783+0000 7f40fdddd740 1 -- 172.21.15.32:0/1119131705 wait complete. 2024-08-31T07:16:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.784+0000 7f40fdddd740 20 client.26468 trim_cache size 0 max 16384 2024-08-31T07:16:47.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.802+0000 7f40fdddd740 1 Processor -- start 2024-08-31T07:16:47.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.802+0000 7f40fdddd740 1 -- start start 2024-08-31T07:16:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd6305c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd6301510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 0x557cd62fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40fdddd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40d000e560 con 0x557cd5f39390 2024-08-31T07:16:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40fdddd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40d0010160 con 0x557cd5f5a070 2024-08-31T07:16:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40fdddd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40d000ec30 con 0x557cd5f311b0 2024-08-31T07:16:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40ef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd6301510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40ef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd6301510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51798/0 (socket says 172.21.15.32:51798) 2024-08-31T07:16:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40ef400640 1 -- 172.21.15.32:0/2054556378 learned_addr learned my addr 172.21.15.32:0/2054556378 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40efe00640 1 --2- 172.21.15.32:0/2054556378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd6305c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.803+0000 7f40f5200640 1 --2- 172.21.15.32:0/2054556378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 0x557cd62fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.804+0000 7f40efe00640 1 -- 172.21.15.32:0/2054556378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 msgr2=0x557cd62fbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:47.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.804+0000 7f40efe00640 1 --2- 172.21.15.32:0/2054556378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 0x557cd62fbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.804+0000 7f40efe00640 1 -- 172.21.15.32:0/2054556378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 msgr2=0x557cd6301510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:47.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.804+0000 7f40efe00640 1 --2- 172.21.15.32:0/2054556378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd6301510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.804+0000 7f40efe00640 1 -- 172.21.15.32:0/2054556378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd630bee0 con 0x557cd5f311b0 2024-08-31T07:16:47.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.804+0000 7f40efe00640 1 --2- 172.21.15.32:0/2054556378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd6305c10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x557cd6326080 tx=0x7f40e0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e49973569a800a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:47.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.804+0000 7f40e4c00640 1 -- 172.21.15.32:0/2054556378 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0029070 con 0x557cd5f311b0 2024-08-31T07:16:47.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.804+0000 7f40e4c00640 1 -- 172.21.15.32:0/2054556378 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e0003e20 con 0x557cd5f311b0 2024-08-31T07:16:47.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.804+0000 7f40e4c00640 1 -- 172.21.15.32:0/2054556378 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0010820 con 0x557cd5f311b0 2024-08-31T07:16:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.805+0000 7f40fdddd740 1 -- 172.21.15.32:0/2054556378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 msgr2=0x557cd6305c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.805+0000 7f40fdddd740 1 --2- 172.21.15.32:0/2054556378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd6305c10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x557cd6326080 tx=0x7f40e0009260 comp rx=0 tx=0).stop 2024-08-31T07:16:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.805+0000 7f40fdddd740 1 -- 172.21.15.32:0/2054556378 shutdown_connections 2024-08-31T07:16:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.805+0000 7f40fdddd740 1 --2- 172.21.15.32:0/2054556378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f5a070 0x557cd62fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.805+0000 7f40fdddd740 1 --2- 172.21.15.32:0/2054556378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f39390 0x557cd6301510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.805+0000 7f40fdddd740 1 --2- 172.21.15.32:0/2054556378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd6305c10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.805+0000 7f40fdddd740 1 -- 172.21.15.32:0/2054556378 >> 172.21.15.32:0/2054556378 conn(0x557cd5f4e460 msgr2=0x557cd630c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.806+0000 7f40fdddd740 1 -- 172.21.15.32:0/2054556378 shutdown_connections 2024-08-31T07:16:47.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.806+0000 7f40fdddd740 1 -- 172.21.15.32:0/2054556378 wait complete. 2024-08-31T07:16:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.806+0000 7f40fdddd740 1 -- start start 2024-08-31T07:16:47.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.806+0000 7f40fdddd740 1 Processor -- start 2024-08-31T07:16:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd631c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 0x557cd63188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40fdddd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f5a070 0x557cd6318fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40fdddd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40d000f9d0 con 0x557cd5f5a070 2024-08-31T07:16:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40fdddd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40d000f300 con 0x557cd5f39390 2024-08-31T07:16:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40fdddd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40d000c540 con 0x557cd5f311b0 2024-08-31T07:16:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40fdddd740 10 client.0 _ll_register_callbacks cb 0x7f40dc1089b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:16:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40efe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd631c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40f5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f5a070 0x557cd6318fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40efe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd631c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51752/0 (socket says 172.21.15.32:51752) 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40efe00640 1 -- 172.21.15.32:0/3125318501 learned_addr learned my addr 172.21.15.32:0/3125318501 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40ef400640 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 0x557cd63188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40efe00640 1 -- 172.21.15.32:0/3125318501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 msgr2=0x557cd63188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.807+0000 7f40efe00640 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 0x557cd63188c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.808+0000 7f40efe00640 1 -- 172.21.15.32:0/3125318501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f5a070 msgr2=0x557cd6318fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.808+0000 7f40efe00640 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f5a070 0x557cd6318fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.808+0000 7f40efe00640 1 -- 172.21.15.32:0/3125318501 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd6390350 con 0x557cd5f311b0 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.808+0000 7f40efe00640 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd631c400 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005dc0 tx=0x7f40e0003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2223c66c2ce14589 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.808+0000 7f40da000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.808+0000 7f40da000640 1 -- 172.21.15.32:0/3125318501 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0032060 con 0x557cd5f311b0 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.808+0000 7f40fdddd740 1 -- 172.21.15.32:0/3125318501 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e0003e20 con 0x557cd5f311b0 2024-08-31T07:16:47.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.808+0000 7f40da000640 1 -- 172.21.15.32:0/3125318501 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f40e0005500 con 0x557cd5f311b0 2024-08-31T07:16:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40da000640 1 -- 172.21.15.32:0/3125318501 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f40e0029050 con 0x557cd5f311b0 2024-08-31T07:16:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40fdddd740 10 client.26488 fetch_fsmap learned FSMap version 31 2024-08-31T07:16:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40fdddd740 1 -- 172.21.15.32:0/3125318501 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557cd6391460 con 0x557cd5f311b0 2024-08-31T07:16:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40da000640 1 -- 172.21.15.32:0/3125318501 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e0003e20 con 0x557cd5f311b0 2024-08-31T07:16:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40da000640 1 -- 172.21.15.32:0/3125318501 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f40e0024960 con 0x557cd5f311b0 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40fdddd740 10 client.26488 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40fdddd740 10 client.26488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40fdddd740 20 client.26488 populate_metadata read hostname 'smithi032' 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40fdddd740 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557cd5eedda0 0x557cd6393c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40fdddd740 4 client.26488 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.809+0000 7f40fdddd740 1 -- 172.21.15.32:0/3125318501 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557cd6274160 con 0x557cd5eedda0 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.810+0000 7f40ef400640 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557cd5eedda0 0x557cd6393c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.810+0000 7f40da000640 10 client.26488 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.810+0000 7f40ef400640 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557cd5eedda0 0x557cd6393c60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8ea2f3fe9fbe8a server_cookie=2dea437c2642323 in_seq=0 out_seq=0 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.810+0000 7f40da000640 1 -- 172.21.15.32:0/3125318501 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x557cd6274160 con 0x557cd5eedda0 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.810+0000 7f40da000640 10 client.26488 handle_command_reply: tid=0 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.810+0000 7f40fdddd740 1 client.26488 shutdown 2024-08-31T07:16:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 1 -- 172.21.15.32:0/3125318501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 msgr2=0x557cd631c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd631c400 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005dc0 tx=0x7f40e0003c20 comp rx=0 tx=0).stop 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 1 -- 172.21.15.32:0/3125318501 shutdown_connections 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557cd5eedda0 0x557cd6393c60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557cd5f5a070 0x557cd6318fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd5f39390 0x557cd63188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 1 --2- 172.21.15.32:0/3125318501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557cd5f311b0 0x557cd631c400 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 1 -- 172.21.15.32:0/3125318501 >> 172.21.15.32:0/3125318501 conn(0x557cd5f4e460 msgr2=0x557cd630d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40da000640 0 client.26488 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 1 -- 172.21.15.32:0/3125318501 shutdown_connections 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 1 -- 172.21.15.32:0/3125318501 wait complete. 2024-08-31T07:16:47.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.811+0000 7f40fdddd740 20 client.26488 trim_cache size 0 max 16384 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "fd7d31bf-f49a-41f6-8889-081d61bf9bb8", 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:16:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:16:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 -- 172.21.15.32:0/26599472 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40d40b4ca0 msgr2=0x7f40d40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40d40b4ca0 0x7f40d40b71b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009fd0 tx=0x7f40e8009500 comp rx=0 tx=0).stop 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 -- 172.21.15.32:0/26599472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f0075ea0 msgr2=0x7f40f019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f0075ea0 0x7f40f019c850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f40dc035040 tx=0x7f40dc032d50 comp rx=0 tx=0).stop 2024-08-31T07:16:47.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 -- 172.21.15.32:0/26599472 shutdown_connections 2024-08-31T07:16:47.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40d40b4ca0 0x7f40d40b71b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40f010fcb0 0x7f40f01a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40f0075ea0 0x7f40f019c850 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 --2- 172.21.15.32:0/26599472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f0075550 0x7f40f019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:16:47.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 -- 172.21.15.32:0/26599472 >> 172.21.15.32:0/26599472 conn(0x7f40f00ff1e0 msgr2=0x7f40f0109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:16:47.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 -- 172.21.15.32:0/26599472 shutdown_connections 2024-08-31T07:16:47.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:16:47.812+0000 7f40f5c00640 1 -- 172.21.15.32:0/26599472 wait complete. 2024-08-31T07:16:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:48 smithi073 ceph-mon[21995]: pgmap v489: 97 pgs: 97 active+clean; 67 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 123 KiB/s wr, 34 op/s 2024-08-31T07:16:48.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 114210 1.55 MB/sec execute 37 sec latency 1176.863 ms 2024-08-31T07:16:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:48 smithi084 ceph-mon[22387]: pgmap v489: 97 pgs: 97 active+clean; 67 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 123 KiB/s wr, 34 op/s 2024-08-31T07:16:48.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:48 smithi032 ceph-mon[17252]: pgmap v489: 97 pgs: 97 active+clean; 67 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 123 KiB/s wr, 34 op/s 2024-08-31T07:16:48.982 INFO:tasks.cephfs.filesystem:scrub status for tag:fd7d31bf-f49a-41f6-8889-081d61bf9bb8 - {'path': '/', 'tag': 'fd7d31bf-f49a-41f6-8889-081d61bf9bb8', 'options': 'recursive,force'} 2024-08-31T07:16:49.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 114210 1.51 MB/sec execute 38 sec latency 2176.959 ms 2024-08-31T07:16:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:50 smithi073 ceph-mon[21995]: pgmap v490: 97 pgs: 97 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 356 KiB/s wr, 68 op/s 2024-08-31T07:16:50.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 114266 1.47 MB/sec execute 39 sec latency 2599.219 ms 2024-08-31T07:16:50.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:50 smithi084 ceph-mon[22387]: pgmap v490: 97 pgs: 97 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 356 KiB/s wr, 68 op/s 2024-08-31T07:16:50.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:50 smithi032 ceph-mon[17252]: pgmap v490: 97 pgs: 97 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 356 KiB/s wr, 68 op/s 2024-08-31T07:16:51.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:51 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:16:51.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:51 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:16:51.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:51 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:16:51.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 114266 1.43 MB/sec execute 40 sec latency 1533.976 ms 2024-08-31T07:16:51.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:51 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:16:51.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:51 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:16:51.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:51 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:16:51.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:51 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:16:51.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:51 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:16:51.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:51 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:16:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:52 smithi073 ceph-mon[21995]: pgmap v491: 97 pgs: 97 active+clean; 70 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 434 KiB/s wr, 80 op/s 2024-08-31T07:16:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:52 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:16:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:52 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:16:52.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 114266 1.40 MB/sec execute 41 sec latency 2534.130 ms 2024-08-31T07:16:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:52 smithi084 ceph-mon[22387]: pgmap v491: 97 pgs: 97 active+clean; 70 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 434 KiB/s wr, 80 op/s 2024-08-31T07:16:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:52 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:16:52.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:52 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:16:52.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:52 smithi032 ceph-mon[17252]: pgmap v491: 97 pgs: 97 active+clean; 70 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 434 KiB/s wr, 80 op/s 2024-08-31T07:16:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:52 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:16:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:52 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:16:53.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 114266 1.37 MB/sec execute 42 sec latency 3534.317 ms 2024-08-31T07:16:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:54 smithi073 ceph-mon[21995]: pgmap v492: 97 pgs: 97 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 455 KiB/s wr, 86 op/s 2024-08-31T07:16:54.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 114266 1.33 MB/sec execute 43 sec latency 4534.440 ms 2024-08-31T07:16:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:54 smithi084 ceph-mon[22387]: pgmap v492: 97 pgs: 97 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 455 KiB/s wr, 86 op/s 2024-08-31T07:16:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:54 smithi032 ceph-mon[17252]: pgmap v492: 97 pgs: 97 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 455 KiB/s wr, 86 op/s 2024-08-31T07:16:55.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 115879 1.45 MB/sec execute 44 sec latency 4952.358 ms 2024-08-31T07:16:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:56 smithi073 ceph-mon[21995]: pgmap v493: 97 pgs: 97 active+clean; 71 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 478 KiB/s wr, 89 op/s 2024-08-31T07:16:56.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 117490 1.69 MB/sec execute 45 sec latency 12.170 ms 2024-08-31T07:16:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:56 smithi084 ceph-mon[22387]: pgmap v493: 97 pgs: 97 active+clean; 71 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 478 KiB/s wr, 89 op/s 2024-08-31T07:16:56.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:56 smithi032 ceph-mon[17252]: pgmap v493: 97 pgs: 97 active+clean; 71 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 478 KiB/s wr, 89 op/s 2024-08-31T07:16:57.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:57 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:16:57.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 119545 1.81 MB/sec execute 46 sec latency 11.817 ms 2024-08-31T07:16:57.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:57 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:16:57.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:57 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:16:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:16:58 smithi073 ceph-mon[21995]: pgmap v494: 97 pgs: 97 active+clean; 71 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 501 KiB/s wr, 93 op/s 2024-08-31T07:16:58.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 121087 2.03 MB/sec execute 47 sec latency 10.927 ms 2024-08-31T07:16:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:16:58 smithi084 ceph-mon[22387]: pgmap v494: 97 pgs: 97 active+clean; 71 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 501 KiB/s wr, 93 op/s 2024-08-31T07:16:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:16:58 smithi032 ceph-mon[17252]: pgmap v494: 97 pgs: 97 active+clean; 71 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 501 KiB/s wr, 93 op/s 2024-08-31T07:16:59.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 123100 2.15 MB/sec execute 48 sec latency 67.168 ms 2024-08-31T07:17:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:00 smithi073 ceph-mon[21995]: pgmap v495: 97 pgs: 97 active+clean; 73 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 625 KiB/s wr, 109 op/s 2024-08-31T07:17:00.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124645 2.34 MB/sec execute 49 sec latency 10.655 ms 2024-08-31T07:17:00.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:00 smithi084 ceph-mon[22387]: pgmap v495: 97 pgs: 97 active+clean; 73 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 625 KiB/s wr, 109 op/s 2024-08-31T07:17:00.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:00 smithi032 ceph-mon[17252]: pgmap v495: 97 pgs: 97 active+clean; 73 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 625 KiB/s wr, 109 op/s 2024-08-31T07:17:01.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124905 2.29 MB/sec execute 50 sec latency 727.319 ms 2024-08-31T07:17:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:02 smithi073 ceph-mon[21995]: pgmap v496: 97 pgs: 97 active+clean; 75 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 540 KiB/s wr, 94 op/s 2024-08-31T07:17:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:02 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:17:02.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124905 2.25 MB/sec execute 51 sec latency 1727.438 ms 2024-08-31T07:17:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:02 smithi084 ceph-mon[22387]: pgmap v496: 97 pgs: 97 active+clean; 75 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 540 KiB/s wr, 94 op/s 2024-08-31T07:17:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:02 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:17:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:02 smithi032 ceph-mon[17252]: pgmap v496: 97 pgs: 97 active+clean; 75 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 540 KiB/s wr, 94 op/s 2024-08-31T07:17:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:02 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:17:03.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124905 2.21 MB/sec execute 52 sec latency 2727.603 ms 2024-08-31T07:17:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:04 smithi073 ceph-mon[21995]: pgmap v497: 97 pgs: 97 active+clean; 76 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 630 KiB/s wr, 111 op/s 2024-08-31T07:17:04.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124905 2.16 MB/sec execute 53 sec latency 3727.757 ms 2024-08-31T07:17:04.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:04 smithi084 ceph-mon[22387]: pgmap v497: 97 pgs: 97 active+clean; 76 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 630 KiB/s wr, 111 op/s 2024-08-31T07:17:04.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:04 smithi032 ceph-mon[17252]: pgmap v497: 97 pgs: 97 active+clean; 76 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 630 KiB/s wr, 111 op/s 2024-08-31T07:17:05.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124973 2.13 MB/sec execute 54 sec latency 4145.913 ms 2024-08-31T07:17:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:06 smithi073 ceph-mon[21995]: pgmap v498: 97 pgs: 97 active+clean; 77 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 713 KiB/s wr, 122 op/s 2024-08-31T07:17:06.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124973 2.09 MB/sec execute 55 sec latency 1525.665 ms 2024-08-31T07:17:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:06 smithi084 ceph-mon[22387]: pgmap v498: 97 pgs: 97 active+clean; 77 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 713 KiB/s wr, 122 op/s 2024-08-31T07:17:06.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:06 smithi032 ceph-mon[17252]: pgmap v498: 97 pgs: 97 active+clean; 77 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 713 KiB/s wr, 122 op/s 2024-08-31T07:17:07.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124973 2.05 MB/sec execute 56 sec latency 2525.804 ms 2024-08-31T07:17:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:08 smithi073 ceph-mon[21995]: pgmap v499: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 721 KiB/s wr, 138 op/s 2024-08-31T07:17:08.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124973 2.02 MB/sec execute 57 sec latency 3525.932 ms 2024-08-31T07:17:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:08 smithi084 ceph-mon[22387]: pgmap v499: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 721 KiB/s wr, 138 op/s 2024-08-31T07:17:08.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:08 smithi032 ceph-mon[17252]: pgmap v499: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 721 KiB/s wr, 138 op/s 2024-08-31T07:17:09.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124973 1.98 MB/sec execute 58 sec latency 4525.977 ms 2024-08-31T07:17:10.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 125066 1.95 MB/sec execute 59 sec latency 4799.207 ms 2024-08-31T07:17:10.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:10 smithi084 ceph-mon[22387]: pgmap v500: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 675 KiB/s wr, 130 op/s 2024-08-31T07:17:10.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:10 smithi032 ceph-mon[17252]: pgmap v500: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 675 KiB/s wr, 130 op/s 2024-08-31T07:17:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:10 smithi073 ceph-mon[21995]: pgmap v500: 97 pgs: 97 active+clean; 77 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 675 KiB/s wr, 130 op/s 2024-08-31T07:17:11.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 125095 1.92 MB/sec execute 60 sec latency 1523.846 ms 2024-08-31T07:17:12.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:12 smithi032 ceph-mon[17252]: pgmap v501: 97 pgs: 97 active+clean; 79 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 868 KiB/s wr, 109 op/s 2024-08-31T07:17:12.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 125854 1.96 MB/sec execute 61 sec latency 580.437 ms 2024-08-31T07:17:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:12 smithi084 ceph-mon[22387]: pgmap v501: 97 pgs: 97 active+clean; 79 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 868 KiB/s wr, 109 op/s 2024-08-31T07:17:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:12 smithi073 ceph-mon[21995]: pgmap v501: 97 pgs: 97 active+clean; 79 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 868 KiB/s wr, 109 op/s 2024-08-31T07:17:13.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 128049 2.15 MB/sec execute 62 sec latency 10.787 ms 2024-08-31T07:17:14.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 128449 2.12 MB/sec execute 63 sec latency 573.381 ms 2024-08-31T07:17:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:14 smithi084 ceph-mon[22387]: pgmap v502: 97 pgs: 97 active+clean; 80 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-08-31T07:17:14.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:14 smithi032 ceph-mon[17252]: pgmap v502: 97 pgs: 97 active+clean; 80 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-08-31T07:17:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:14 smithi073 ceph-mon[21995]: pgmap v502: 97 pgs: 97 active+clean; 80 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-08-31T07:17:15.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 128518 2.09 MB/sec execute 64 sec latency 990.093 ms 2024-08-31T07:17:16.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 128518 2.06 MB/sec execute 65 sec latency 1530.952 ms 2024-08-31T07:17:16.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:16 smithi084 ceph-mon[22387]: pgmap v503: 97 pgs: 97 active+clean; 78 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-08-31T07:17:16.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:16 smithi032 ceph-mon[17252]: pgmap v503: 97 pgs: 97 active+clean; 78 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-08-31T07:17:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:16 smithi073 ceph-mon[21995]: pgmap v503: 97 pgs: 97 active+clean; 78 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-08-31T07:17:17.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 128518 2.03 MB/sec execute 66 sec latency 2531.116 ms 2024-08-31T07:17:18.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 128518 2.00 MB/sec execute 67 sec latency 3531.259 ms 2024-08-31T07:17:18.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:18 smithi084 ceph-mon[22387]: pgmap v504: 97 pgs: 97 active+clean; 78 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-08-31T07:17:18.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:18 smithi032 ceph-mon[17252]: pgmap v504: 97 pgs: 97 active+clean; 78 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-08-31T07:17:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:18 smithi073 ceph-mon[21995]: pgmap v504: 97 pgs: 97 active+clean; 78 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-08-31T07:17:18.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:17:19.406 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:17:19.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 128518 1.97 MB/sec execute 68 sec latency 4531.323 ms 2024-08-31T07:17:19.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:19 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:17:19.743 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:19 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:17:19.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:19 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:17:20.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.078+0000 7f2046400640 1 -- 172.21.15.32:0/3473016667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2040075ea0 msgr2=0x7f204010f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:20.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.078+0000 7f2046400640 1 --2- 172.21.15.32:0/3473016667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2040075ea0 0x7f204010f770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f202c009900 tx=0x7f202c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.079+0000 7f2046400640 1 -- 172.21.15.32:0/3473016667 shutdown_connections 2024-08-31T07:17:20.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.079+0000 7f2046400640 1 --2- 172.21.15.32:0/3473016667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f204010fcb0 0x7f20401120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.079+0000 7f2046400640 1 --2- 172.21.15.32:0/3473016667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2040075ea0 0x7f204010f770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.079+0000 7f2046400640 1 --2- 172.21.15.32:0/3473016667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2040075550 0x7f2040075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.079+0000 7f2046400640 1 -- 172.21.15.32:0/3473016667 >> 172.21.15.32:0/3473016667 conn(0x7f20400ff1e0 msgr2=0x7f2040101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:20.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.079+0000 7f2046400640 1 -- 172.21.15.32:0/3473016667 shutdown_connections 2024-08-31T07:17:20.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.079+0000 7f2046400640 1 -- 172.21.15.32:0/3473016667 wait complete. 2024-08-31T07:17:20.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.080+0000 7f2046400640 1 Processor -- start 2024-08-31T07:17:20.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.080+0000 7f2046400640 1 -- start start 2024-08-31T07:17:20.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.080+0000 7f2046400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2040075550 0x7f204019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.080+0000 7f2046400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2040075ea0 0x7f204019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.080+0000 7f2046400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f204010fcb0 0x7f20401a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.080+0000 7f2046400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2040115860 con 0x7f2040075550 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f2046400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20401156e0 con 0x7f2040075ea0 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f2046400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f20401159e0 con 0x7f204010fcb0 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f203fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2040075ea0 0x7f204019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f2045000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2040075550 0x7f204019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f203fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2040075ea0 0x7f204019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55392/0 (socket says 172.21.15.32:55392) 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f203fe00640 1 -- 172.21.15.32:0/1266997225 learned_addr learned my addr 172.21.15.32:0/1266997225 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f2045000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2040075550 0x7f204019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52378/0 (socket says 172.21.15.32:52378) 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f2045a00640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f204010fcb0 0x7f20401a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f203fe00640 1 -- 172.21.15.32:0/1266997225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f204010fcb0 msgr2=0x7f20401a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f203fe00640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f204010fcb0 0x7f20401a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f203fe00640 1 -- 172.21.15.32:0/1266997225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2040075550 msgr2=0x7f204019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f203fe00640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2040075550 0x7f204019c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.081+0000 7f203fe00640 1 -- 172.21.15.32:0/1266997225 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20401a2220 con 0x7f2040075ea0 2024-08-31T07:17:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.082+0000 7f203fe00640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2040075ea0 0x7f204019c850 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f202c032fd0 tx=0x7f202c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.082+0000 7f203d600640 1 -- 172.21.15.32:0/1266997225 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c03f030 con 0x7f2040075ea0 2024-08-31T07:17:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.082+0000 7f2046400640 1 -- 172.21.15.32:0/1266997225 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20401a24b0 con 0x7f2040075ea0 2024-08-31T07:17:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.082+0000 7f203d600640 1 -- 172.21.15.32:0/1266997225 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f202c0043d0 con 0x7f2040075ea0 2024-08-31T07:17:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.082+0000 7f203d600640 1 -- 172.21.15.32:0/1266997225 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c035070 con 0x7f2040075ea0 2024-08-31T07:17:20.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.082+0000 7f2046400640 1 -- 172.21.15.32:0/1266997225 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20401a8c70 con 0x7f2040075ea0 2024-08-31T07:17:20.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.086+0000 7f204e810740 1 Processor -- start 2024-08-31T07:17:20.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.086+0000 7f204e810740 1 -- start start 2024-08-31T07:17:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f204e810740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aaf1070 0x56514ae96b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f204e810740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 0x56514ae97060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f204e810740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aad0390 0x56514ae975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f204e810740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56514ae0d770 con 0x56514aac81b0 2024-08-31T07:17:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f204e810740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56514adfcd00 con 0x56514aaf1070 2024-08-31T07:17:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f204e810740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56514ae00cb0 con 0x56514aad0390 2024-08-31T07:17:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f2045a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aaf1070 0x56514ae96b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f2045a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aaf1070 0x56514ae96b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55394/0 (socket says 172.21.15.32:55394) 2024-08-31T07:17:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f2045a00640 1 -- 172.21.15.32:0/1618894817 learned_addr learned my addr 172.21.15.32:0/1618894817 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f2045000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 0x56514ae97060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.087+0000 7f203d600640 1 -- 172.21.15.32:0/1266997225 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f202c042030 con 0x7f2040075ea0 2024-08-31T07:17:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.088+0000 7f2045a00640 1 -- 172.21.15.32:0/1618894817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aad0390 msgr2=0x56514ae975a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:17:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.088+0000 7f2045a00640 1 --2- 172.21.15.32:0/1618894817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aad0390 0x56514ae975a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.088+0000 7f2045a00640 1 -- 172.21.15.32:0/1618894817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 msgr2=0x56514ae97060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.088+0000 7f2045a00640 1 --2- 172.21.15.32:0/1618894817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 0x56514ae97060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.088+0000 7f2045a00640 1 -- 172.21.15.32:0/1618894817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514ab5c650 con 0x56514aaf1070 2024-08-31T07:17:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.088+0000 7f2045a00640 1 --2- 172.21.15.32:0/1618894817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aaf1070 0x56514ae96b20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2038009c90 tx=0x7f2038011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77dd3dd15c97d641 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.088+0000 7f2032000640 1 -- 172.21.15.32:0/1618894817 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038004230 con 0x56514aaf1070 2024-08-31T07:17:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.088+0000 7f2032000640 1 -- 172.21.15.32:0/1618894817 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2038009e40 con 0x56514aaf1070 2024-08-31T07:17:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.088+0000 7f2032000640 1 -- 172.21.15.32:0/1618894817 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038019700 con 0x56514aaf1070 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.089+0000 7f204e810740 1 -- 172.21.15.32:0/1618894817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aaf1070 msgr2=0x56514ae96b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.089+0000 7f204e810740 1 --2- 172.21.15.32:0/1618894817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aaf1070 0x56514ae96b20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2038009c90 tx=0x7f2038011810 comp rx=0 tx=0).stop 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.089+0000 7f204e810740 1 -- 172.21.15.32:0/1618894817 shutdown_connections 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.089+0000 7f204e810740 1 --2- 172.21.15.32:0/1618894817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aad0390 0x56514ae975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.089+0000 7f204e810740 1 --2- 172.21.15.32:0/1618894817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 0x56514ae97060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.089+0000 7f204e810740 1 --2- 172.21.15.32:0/1618894817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aaf1070 0x56514ae96b20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.089+0000 7f204e810740 1 -- 172.21.15.32:0/1618894817 >> 172.21.15.32:0/1618894817 conn(0x56514aae5460 msgr2=0x56514ae94a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.089+0000 7f204e810740 1 -- 172.21.15.32:0/1618894817 shutdown_connections 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.089+0000 7f204e810740 1 -- 172.21.15.32:0/1618894817 wait complete. 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.089+0000 7f203d600640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f20280b4bb0 0x7f20280b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.090+0000 7f203d600640 1 -- 172.21.15.32:0/1266997225 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f202c105500 con 0x7f2040075ea0 2024-08-31T07:17:20.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.090+0000 7f2045000640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f20280b4bb0 0x7f20280b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.090+0000 7f204e810740 1 -- start start 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.090+0000 7f204e810740 1 Processor -- start 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.090+0000 7f2045000640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f20280b4bb0 0x7f20280b70c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x56514ae957a0 tx=0x7f2034009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f204e810740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 0x56514aea27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f204e810740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 0x56514aea2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f204e810740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aaf1070 0x56514aeae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f204e810740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56514ac5f700 con 0x56514aac81b0 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f204e810740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56514ad83cb0 con 0x56514aad0390 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f204e810740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56514aea5300 con 0x56514aaf1070 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f204e810740 10 client.0 _ll_register_callbacks cb 0x56514ada9430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f2045000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 0x56514aea27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f203fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 0x56514aea2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f2045000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 0x56514aea27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52400/0 (socket says 172.21.15.32:52400) 2024-08-31T07:17:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f2045000640 1 -- 172.21.15.32:0/4265540053 learned_addr learned my addr 172.21.15.32:0/4265540053 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.091+0000 7f2045a00640 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aaf1070 0x56514aeae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f203fe00640 1 -- 172.21.15.32:0/4265540053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aaf1070 msgr2=0x56514aeae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f203fe00640 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aaf1070 0x56514aeae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f203fe00640 1 -- 172.21.15.32:0/4265540053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 msgr2=0x56514aea27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f203fe00640 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 0x56514aea27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f203fe00640 1 -- 172.21.15.32:0/4265540053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514adc4870 con 0x56514aad0390 2024-08-31T07:17:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f2045000640 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 0x56514aea27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f203fe00640 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 0x56514aea2ce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f202c102c70 tx=0x7f202c0cb620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49066b60e4cefe8b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f2026a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:17:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f2026a00640 1 -- 172.21.15.32:0/4265540053 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c049040 con 0x56514aad0390 2024-08-31T07:17:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f2026a00640 1 -- 172.21.15.32:0/4265540053 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f202c0d01c0 con 0x56514aad0390 2024-08-31T07:17:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.092+0000 7f204e810740 1 -- 172.21.15.32:0/4265540053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56514ae0f6c0 con 0x56514aad0390 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f2026a00640 1 -- 172.21.15.32:0/4265540053 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f202c0cc310 con 0x56514aad0390 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f204e810740 10 client.26486 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f204e810740 1 -- 172.21.15.32:0/4265540053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56514aeb1310 con 0x56514aad0390 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f2026a00640 1 -- 172.21.15.32:0/4265540053 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56514ae0f6c0 con 0x56514aad0390 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f2026a00640 1 -- 172.21.15.32:0/4265540053 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f202c102ef0 con 0x56514aad0390 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f204e810740 10 client.26486 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f204e810740 10 client.26486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f204e810740 20 client.26486 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f204e810740 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56514aaaf9f0 0x56514aeb4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f204e810740 4 client.26486 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.093+0000 7f204e810740 1 -- 172.21.15.32:0/4265540053 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56514ae0b160 con 0x56514aaaf9f0 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.094+0000 7f2045000640 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56514aaaf9f0 0x56514aeb4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.094+0000 7f2045000640 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56514aaaf9f0 0x56514aeb4870 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d5c682e260ce33 server_cookie=eb41ecc9bd3d5232 in_seq=0 out_seq=0 2024-08-31T07:17:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.094+0000 7f2026a00640 10 client.26486 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.095+0000 7f2026a00640 1 -- 172.21.15.32:0/4265540053 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56514ae0b160 con 0x56514aaaf9f0 2024-08-31T07:17:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.095+0000 7f2026a00640 10 client.26486 handle_command_reply: tid=0 2024-08-31T07:17:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.095+0000 7f204e810740 1 client.26486 shutdown 2024-08-31T07:17:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.095+0000 7f204e810740 1 -- 172.21.15.32:0/4265540053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 msgr2=0x56514aea2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.095+0000 7f204e810740 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 0x56514aea2ce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f202c102c70 tx=0x7f202c0cb620 comp rx=0 tx=0).stop 2024-08-31T07:17:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.096+0000 7f204e810740 1 -- 172.21.15.32:0/4265540053 shutdown_connections 2024-08-31T07:17:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.096+0000 7f204e810740 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56514aaaf9f0 0x56514aeb4870 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.096+0000 7f204e810740 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aaf1070 0x56514aeae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.096+0000 7f204e810740 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 0x56514aea2ce0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.096+0000 7f2026a00640 0 client.26486 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.096+0000 7f204e810740 1 --2- 172.21.15.32:0/4265540053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aac81b0 0x56514aea27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.096+0000 7f204e810740 1 -- 172.21.15.32:0/4265540053 >> 172.21.15.32:0/4265540053 conn(0x56514aae5460 msgr2=0x56514ae99d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.096+0000 7f204e810740 1 -- 172.21.15.32:0/4265540053 shutdown_connections 2024-08-31T07:17:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.096+0000 7f204e810740 1 -- 172.21.15.32:0/4265540053 wait complete. 2024-08-31T07:17:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.096+0000 7f204e810740 20 client.26486 trim_cache size 0 max 16384 2024-08-31T07:17:20.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.118+0000 7f204e810740 1 Processor -- start 2024-08-31T07:17:20.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.118+0000 7f204e810740 1 -- start start 2024-08-31T07:17:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.118+0000 7f204e810740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aac81b0 0x56514ae94910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.118+0000 7f204e810740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aad0390 0x56514ae9bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.118+0000 7f204e810740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aaf1070 0x56514ae9c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.118+0000 7f204e810740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b1350 con 0x56514aad0390 2024-08-31T07:17:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.118+0000 7f204e810740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b2e90 con 0x56514aac81b0 2024-08-31T07:17:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.118+0000 7f204e810740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b1a20 con 0x56514aaf1070 2024-08-31T07:17:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.118+0000 7f2045000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aac81b0 0x56514ae94910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.119+0000 7f2045000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aac81b0 0x56514ae94910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55410/0 (socket says 172.21.15.32:55410) 2024-08-31T07:17:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.119+0000 7f2045000640 1 -- 172.21.15.32:0/883886012 learned_addr learned my addr 172.21.15.32:0/883886012 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.119+0000 7f2045a00640 1 --2- 172.21.15.32:0/883886012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aaf1070 0x56514ae9c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.119+0000 7f203fe00640 1 --2- 172.21.15.32:0/883886012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aad0390 0x56514ae9bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.119+0000 7f2045000640 1 -- 172.21.15.32:0/883886012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aaf1070 msgr2=0x56514ae9c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.119+0000 7f2045000640 1 --2- 172.21.15.32:0/883886012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aaf1070 0x56514ae9c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.119+0000 7f2045000640 1 -- 172.21.15.32:0/883886012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aad0390 msgr2=0x56514ae9bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.119+0000 7f2045000640 1 --2- 172.21.15.32:0/883886012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aad0390 0x56514ae9bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.119+0000 7f2045000640 1 -- 172.21.15.32:0/883886012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514ae98e10 con 0x56514aac81b0 2024-08-31T07:17:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.119+0000 7f2045000640 1 --2- 172.21.15.32:0/883886012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aac81b0 0x56514ae94910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x56514aeb4fa0 tx=0x7f203401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22d5951bdd0ad37a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:20.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.120+0000 7f2031600640 1 -- 172.21.15.32:0/883886012 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203400f990 con 0x56514aac81b0 2024-08-31T07:17:20.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.120+0000 7f2031600640 1 -- 172.21.15.32:0/883886012 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f203402a470 con 0x56514aac81b0 2024-08-31T07:17:20.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.120+0000 7f2031600640 1 -- 172.21.15.32:0/883886012 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203401e660 con 0x56514aac81b0 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.121+0000 7f204e810740 1 -- 172.21.15.32:0/883886012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aac81b0 msgr2=0x56514ae94910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.121+0000 7f204e810740 1 --2- 172.21.15.32:0/883886012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aac81b0 0x56514ae94910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x56514aeb4fa0 tx=0x7f203401a000 comp rx=0 tx=0).stop 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.121+0000 7f204e810740 1 -- 172.21.15.32:0/883886012 shutdown_connections 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.121+0000 7f204e810740 1 --2- 172.21.15.32:0/883886012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aaf1070 0x56514ae9c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.121+0000 7f204e810740 1 --2- 172.21.15.32:0/883886012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aad0390 0x56514ae9bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.121+0000 7f204e810740 1 --2- 172.21.15.32:0/883886012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aac81b0 0x56514ae94910 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.121+0000 7f204e810740 1 -- 172.21.15.32:0/883886012 >> 172.21.15.32:0/883886012 conn(0x56514aae5460 msgr2=0x56514ae94200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.121+0000 7f204e810740 1 -- 172.21.15.32:0/883886012 shutdown_connections 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.121+0000 7f204e810740 1 -- 172.21.15.32:0/883886012 wait complete. 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.121+0000 7f204e810740 1 -- start start 2024-08-31T07:17:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.122+0000 7f204e810740 1 Processor -- start 2024-08-31T07:17:20.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.122+0000 7f204e810740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aac81b0 0x56514aeaa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.122+0000 7f204e810740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 0x56514aeaace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f204e810740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aaf1070 0x56514aea5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f204e810740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b27c0 con 0x56514aaf1070 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f204e810740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b20f0 con 0x56514aad0390 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f204e810740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f20401af330 con 0x56514aac81b0 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f204e810740 10 client.0 _ll_register_callbacks cb 0x7f202c0c7af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f2045000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aac81b0 0x56514aeaa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f2045000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aac81b0 0x56514aeaa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60444/0 (socket says 172.21.15.32:60444) 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f2045000640 1 -- 172.21.15.32:0/745844850 learned_addr learned my addr 172.21.15.32:0/745844850 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f203fe00640 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 0x56514aeaace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f2045a00640 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aaf1070 0x56514aea5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f2045000640 1 -- 172.21.15.32:0/745844850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 msgr2=0x56514aeaace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f2045000640 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 0x56514aeaace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f2045000640 1 -- 172.21.15.32:0/745844850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aaf1070 msgr2=0x56514aea5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f2045000640 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aaf1070 0x56514aea5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.123+0000 7f2045000640 1 -- 172.21.15.32:0/745844850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514aea5c00 con 0x56514aac81b0 2024-08-31T07:17:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.124+0000 7f2045a00640 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aaf1070 0x56514aea5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:17:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.124+0000 7f2045000640 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aac81b0 0x56514aeaa760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f20340049d0 tx=0x7f2034005ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c22596bf5b1e4cb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.124+0000 7f2026a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:17:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.124+0000 7f2026a00640 1 -- 172.21.15.32:0/745844850 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203403f060 con 0x56514aac81b0 2024-08-31T07:17:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.124+0000 7f2026a00640 1 -- 172.21.15.32:0/745844850 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2034017040 con 0x56514aac81b0 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.125+0000 7f204e810740 1 -- 172.21.15.32:0/745844850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f203402a470 con 0x56514aac81b0 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.125+0000 7f2026a00640 1 -- 172.21.15.32:0/745844850 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f203400fba0 con 0x56514aac81b0 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.125+0000 7f204e810740 10 client.26503 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.125+0000 7f204e810740 1 -- 172.21.15.32:0/745844850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56514af1fb60 con 0x56514aac81b0 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.126+0000 7f2026a00640 1 -- 172.21.15.32:0/745844850 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f203402a470 con 0x56514aac81b0 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.126+0000 7f2026a00640 1 -- 172.21.15.32:0/745844850 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2034031890 con 0x56514aac81b0 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.126+0000 7f204e810740 10 client.26503 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.126+0000 7f204e810740 10 client.26503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.126+0000 7f204e810740 20 client.26503 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.126+0000 7f204e810740 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56514aa84da0 0x56514af22210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.126+0000 7f204e810740 4 client.26503 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:17:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.126+0000 7f204e810740 1 -- 172.21.15.32:0/745844850 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56514ae088a0 con 0x56514aa84da0 2024-08-31T07:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.127+0000 7f203fe00640 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56514aa84da0 0x56514af22210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.127+0000 7f203fe00640 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56514aa84da0 0x56514af22210 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc583ec839decde server_cookie=cfb8d6fd217501b1 in_seq=0 out_seq=0 2024-08-31T07:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.127+0000 7f2026a00640 10 client.26503 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f2026a00640 1 -- 172.21.15.32:0/745844850 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x56514ae088a0 con 0x56514aa84da0 2024-08-31T07:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f2026a00640 10 client.26503 handle_command_reply: tid=0 2024-08-31T07:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f204e810740 1 client.26503 shutdown 2024-08-31T07:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f204e810740 1 -- 172.21.15.32:0/745844850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aac81b0 msgr2=0x56514aeaa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f204e810740 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aac81b0 0x56514aeaa760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f20340049d0 tx=0x7f2034005ae0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f204e810740 1 -- 172.21.15.32:0/745844850 shutdown_connections 2024-08-31T07:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f204e810740 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56514aa84da0 0x56514af22210 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f204e810740 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56514aaf1070 0x56514aea5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f204e810740 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56514aad0390 0x56514aeaace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f204e810740 1 --2- 172.21.15.32:0/745844850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56514aac81b0 0x56514aeaa760 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f204e810740 1 -- 172.21.15.32:0/745844850 >> 172.21.15.32:0/745844850 conn(0x56514aae5460 msgr2=0x56514ae99c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.128+0000 7f2026a00640 0 client.26503 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.129+0000 7f204e810740 1 -- 172.21.15.32:0/745844850 shutdown_connections 2024-08-31T07:17:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.129+0000 7f204e810740 1 -- 172.21.15.32:0/745844850 wait complete. 2024-08-31T07:17:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.129+0000 7f204e810740 20 client.26503 trim_cache size 0 max 16384 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:20.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:20.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:20.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:20.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:20.135 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:17:20.135 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:17:20.135 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:20.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:20.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:20.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:20.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:20.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:20.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:20.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:20.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:20.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:17:20.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:17:20.137 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:17:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 -- 172.21.15.32:0/1266997225 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f20280b4bb0 msgr2=0x7f20280b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f20280b4bb0 0x7f20280b70c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x56514ae957a0 tx=0x7f2034009500 comp rx=0 tx=0).stop 2024-08-31T07:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 -- 172.21.15.32:0/1266997225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2040075ea0 msgr2=0x7f204019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2040075ea0 0x7f204019c850 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f202c032fd0 tx=0x7f202c031a20 comp rx=0 tx=0).stop 2024-08-31T07:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 -- 172.21.15.32:0/1266997225 shutdown_connections 2024-08-31T07:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f20280b4bb0 0x7f20280b70c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f204010fcb0 0x7f20401a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2040075ea0 0x7f204019c850 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 --2- 172.21.15.32:0/1266997225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2040075550 0x7f204019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 -- 172.21.15.32:0/1266997225 >> 172.21.15.32:0/1266997225 conn(0x7f20400ff1e0 msgr2=0x7f20401098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 -- 172.21.15.32:0/1266997225 shutdown_connections 2024-08-31T07:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:20.130+0000 7f2046400640 1 -- 172.21.15.32:0/1266997225 wait complete. 2024-08-31T07:17:20.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 130025 2.03 MB/sec execute 69 sec latency 4946.037 ms 2024-08-31T07:17:20.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:20 smithi084 ceph-mon[22387]: pgmap v505: 97 pgs: 97 active+clean; 78 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2024-08-31T07:17:20.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:17:20.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:17:20.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:17:20.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:20 smithi032 ceph-mon[17252]: pgmap v505: 97 pgs: 97 active+clean; 78 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2024-08-31T07:17:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:17:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:17:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:17:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:20 smithi073 ceph-mon[21995]: pgmap v505: 97 pgs: 97 active+clean; 78 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2024-08-31T07:17:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:17:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:17:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:17:21.081 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:17:21.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:17:21.493 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:17:21.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 131645 2.18 MB/sec execute 70 sec latency 10.810 ms 2024-08-31T07:17:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.156+0000 7f6441e00640 1 -- 172.21.15.32:0/3622394995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643c10b270 msgr2=0x7f643c10b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.156+0000 7f6441e00640 1 --2- 172.21.15.32:0/3622394995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643c10b270 0x7f643c10b680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6428009900 tx=0x7f642802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:17:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.156+0000 7f6441e00640 1 -- 172.21.15.32:0/3622394995 shutdown_connections 2024-08-31T07:17:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.156+0000 7f6441e00640 1 --2- 172.21.15.32:0/3622394995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f643c10bbc0 0x7f643c1123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.156+0000 7f6441e00640 1 --2- 172.21.15.32:0/3622394995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643c10b270 0x7f643c10b680 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.156+0000 7f6441e00640 1 --2- 172.21.15.32:0/3622394995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f643c105930 0x7f643c105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.156+0000 7f6441e00640 1 -- 172.21.15.32:0/3622394995 >> 172.21.15.32:0/3622394995 conn(0x7f643c101380 msgr2=0x7f643c1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.156+0000 7f6441e00640 1 -- 172.21.15.32:0/3622394995 shutdown_connections 2024-08-31T07:17:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.156+0000 7f6441e00640 1 -- 172.21.15.32:0/3622394995 wait complete. 2024-08-31T07:17:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.157+0000 7f6441e00640 1 Processor -- start 2024-08-31T07:17:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6441e00640 1 -- start start 2024-08-31T07:17:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6441e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f643c105930 0x7f643c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6441e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f643c10b270 0x7f643c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6441e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643c10bbc0 0x7f643c1a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6441e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f643c1158d0 con 0x7f643c10bbc0 2024-08-31T07:17:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6441e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f643c115750 con 0x7f643c105930 2024-08-31T07:17:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6441e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f643c115a50 con 0x7f643c10b270 2024-08-31T07:17:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6441400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643c10bbc0 0x7f643c1a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6440a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f643c105930 0x7f643c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6440a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f643c105930 0x7f643c19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55440/0 (socket says 172.21.15.32:55440) 2024-08-31T07:17:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6441400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643c10bbc0 0x7f643c1a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52422/0 (socket says 172.21.15.32:52422) 2024-08-31T07:17:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f6440a00640 1 -- 172.21.15.32:0/3776559505 learned_addr learned my addr 172.21.15.32:0/3776559505 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.158+0000 7f643be00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f643c10b270 0x7f643c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.159+0000 7f6440a00640 1 -- 172.21.15.32:0/3776559505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f643c10b270 msgr2=0x7f643c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.159+0000 7f6440a00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f643c10b270 0x7f643c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.159+0000 7f6440a00640 1 -- 172.21.15.32:0/3776559505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643c10bbc0 msgr2=0x7f643c1a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.159+0000 7f6440a00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643c10bbc0 0x7f643c1a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.159+0000 7f6440a00640 1 -- 172.21.15.32:0/3776559505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c1a2230 con 0x7f643c105930 2024-08-31T07:17:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.159+0000 7f643be00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f643c10b270 0x7f643c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.159+0000 7f6440a00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f643c105930 0x7f643c19c2c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f643000ea60 tx=0x7f643000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.160+0000 7f6439600640 1 -- 172.21.15.32:0/3776559505 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643000cd20 con 0x7f643c105930 2024-08-31T07:17:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.160+0000 7f6439600640 1 -- 172.21.15.32:0/3776559505 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64300035f0 con 0x7f643c105930 2024-08-31T07:17:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.160+0000 7f6439600640 1 -- 172.21.15.32:0/3776559505 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430010690 con 0x7f643c105930 2024-08-31T07:17:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.160+0000 7f6441e00640 1 -- 172.21.15.32:0/3776559505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c1a2520 con 0x7f643c105930 2024-08-31T07:17:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.160+0000 7f6441e00640 1 -- 172.21.15.32:0/3776559505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c1a8c80 con 0x7f643c105930 2024-08-31T07:17:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.161+0000 7f6441e00640 1 -- 172.21.15.32:0/3776559505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c105d40 con 0x7f643c105930 2024-08-31T07:17:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.163+0000 7f6439600640 1 -- 172.21.15.32:0/3776559505 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f64300037a0 con 0x7f643c105930 2024-08-31T07:17:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.165+0000 7f6439600640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f64240b4ca0 0x7f64240b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:22.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.165+0000 7f6439600640 1 -- 172.21.15.32:0/3776559505 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f64300e57f0 con 0x7f643c105930 2024-08-31T07:17:22.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.165+0000 7f643be00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f64240b4ca0 0x7f64240b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:22.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.166+0000 7f643be00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f64240b4ca0 0x7f64240b71b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f642802a9e0 tx=0x7f6428005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:22.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.166+0000 7f6439600640 1 -- 172.21.15.32:0/3776559505 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f64300ac600 con 0x7f643c105930 2024-08-31T07:17:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.404+0000 7f6441e00640 1 -- 172.21.15.32:0/3776559505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f643c19d3f0 con 0x7f643c105930 2024-08-31T07:17:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.405+0000 7f6439600640 1 -- 172.21.15.32:0/3776559505 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11960 (secure 0 0 0) 0x7f643c19d3f0 con 0x7f643c105930 2024-08-31T07:17:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:17:22.408 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"btime":"2024-08-31T07:17:01:405815+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:17:00.944291+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:17:22.408 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2024-08-31T07:17:22.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 -- 172.21.15.32:0/3776559505 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f64240b4ca0 msgr2=0x7f64240b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:22.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f64240b4ca0 0x7f64240b71b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f642802a9e0 tx=0x7f6428005a20 comp rx=0 tx=0).stop 2024-08-31T07:17:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 -- 172.21.15.32:0/3776559505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f643c105930 msgr2=0x7f643c19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f643c105930 0x7f643c19c2c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f643000ea60 tx=0x7f643000ee60 comp rx=0 tx=0).stop 2024-08-31T07:17:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 -- 172.21.15.32:0/3776559505 shutdown_connections 2024-08-31T07:17:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f64240b4ca0 0x7f64240b71b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f643c10bbc0 0x7f643c1a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f643c10b270 0x7f643c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 --2- 172.21.15.32:0/3776559505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f643c105930 0x7f643c19c2c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 -- 172.21.15.32:0/3776559505 >> 172.21.15.32:0/3776559505 conn(0x7f643c101380 msgr2=0x7f643c1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 -- 172.21.15.32:0/3776559505 shutdown_connections 2024-08-31T07:17:22.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:22.408+0000 7f6441e00640 1 -- 172.21.15.32:0/3776559505 wait complete. 2024-08-31T07:17:22.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 131999 2.15 MB/sec execute 71 sec latency 628.922 ms 2024-08-31T07:17:22.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:22 smithi084 ceph-mon[22387]: pgmap v506: 97 pgs: 97 active+clean; 78 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-08-31T07:17:22.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:22 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3776559505' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:17:22.772 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:22 smithi032 ceph-mon[17252]: pgmap v506: 97 pgs: 97 active+clean; 78 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-08-31T07:17:22.772 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:22 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3776559505' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:17:22.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:22 smithi073 ceph-mon[21995]: pgmap v506: 97 pgs: 97 active+clean; 78 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-08-31T07:17:22.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:22 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3776559505' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:17:23.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:17:23.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 131999 2.12 MB/sec execute 72 sec latency 1628.982 ms 2024-08-31T07:17:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:17:24.473 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:24 smithi032 ceph-mon[17252]: pgmap v507: 97 pgs: 97 active+clean; 79 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-08-31T07:17:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.471+0000 7ff45d000640 1 -- 172.21.15.32:0/2684600214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff458075b70 msgr2=0x7ff458075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.471+0000 7ff45d000640 1 --2- 172.21.15.32:0/2684600214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff458075b70 0x7ff458075f80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff444009900 tx=0x7ff44402a410 comp rx=0 tx=0).stop 2024-08-31T07:17:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.471+0000 7ff45d000640 1 -- 172.21.15.32:0/2684600214 shutdown_connections 2024-08-31T07:17:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.471+0000 7ff45d000640 1 --2- 172.21.15.32:0/2684600214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4581141d0 0x7ff4581166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.471+0000 7ff45d000640 1 --2- 172.21.15.32:0/2684600214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff458075b70 0x7ff458075f80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.471+0000 7ff45d000640 1 --2- 172.21.15.32:0/2684600214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff458077120 0x7ff458075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.471+0000 7ff45d000640 1 -- 172.21.15.32:0/2684600214 >> 172.21.15.32:0/2684600214 conn(0x7ff45807b5f0 msgr2=0x7ff45807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.471+0000 7ff45d000640 1 -- 172.21.15.32:0/2684600214 shutdown_connections 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.471+0000 7ff45d000640 1 -- 172.21.15.32:0/2684600214 wait complete. 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.472+0000 7ff45d000640 1 Processor -- start 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff45d000640 1 -- start start 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff45d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff458075b70 0x7ff4581a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff45d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff458077120 0x7ff4581a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff45d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4581141d0 0x7ff4581a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff45d000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff458119e10 con 0x7ff4581141d0 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff45d000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff458119c90 con 0x7ff458075b70 2024-08-31T07:17:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff45d000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff458119f90 con 0x7ff458077120 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff456800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4581141d0 0x7ff4581a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff456800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4581141d0 0x7ff4581a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52430/0 (socket says 172.21.15.32:52430) 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff456800640 1 -- 172.21.15.32:0/328810913 learned_addr learned my addr 172.21.15.32:0/328810913 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff455400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff458077120 0x7ff4581a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.473+0000 7ff455e00640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff458075b70 0x7ff4581a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.474+0000 7ff456800640 1 -- 172.21.15.32:0/328810913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff458077120 msgr2=0x7ff4581a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.474+0000 7ff456800640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff458077120 0x7ff4581a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.474+0000 7ff456800640 1 -- 172.21.15.32:0/328810913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff458075b70 msgr2=0x7ff4581a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.474+0000 7ff456800640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff458075b70 0x7ff4581a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.474+0000 7ff456800640 1 -- 172.21.15.32:0/328810913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4581a6750 con 0x7ff4581141d0 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.474+0000 7ff456800640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4581141d0 0x7ff4581a6020 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff45000eb10 tx=0x7ff45000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.474+0000 7ff44ea00640 1 -- 172.21.15.32:0/328810913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450004b20 con 0x7ff4581141d0 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.474+0000 7ff45d000640 1 -- 172.21.15.32:0/328810913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4581a6a40 con 0x7ff4581141d0 2024-08-31T07:17:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.474+0000 7ff44ea00640 1 -- 172.21.15.32:0/328810913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff450004cc0 con 0x7ff4581141d0 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.474+0000 7ff44ea00640 1 -- 172.21.15.32:0/328810913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450010670 con 0x7ff4581141d0 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.475+0000 7ff45d000640 1 -- 172.21.15.32:0/328810913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4581ad1a0 con 0x7ff4581141d0 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.478+0000 7ff463c5e740 1 Processor -- start 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.478+0000 7ff44ea00640 1 -- 172.21.15.32:0/328810913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff450005020 con 0x7ff4581141d0 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.478+0000 7ff463c5e740 1 -- start start 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 0x55c2aad80990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad80ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 0x55c2aad81410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff463c5e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c2aacf77b0 con 0x55c2aa9b21b0 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff463c5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c2aace6d40 con 0x55c2aa9db070 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff463c5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c2aaceacf0 con 0x55c2aa9ba390 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff455e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad80ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff455400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 0x55c2aad80990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff455e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad80ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52436/0 (socket says 172.21.15.32:52436) 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff455e00640 1 -- 172.21.15.32:0/2705726808 learned_addr learned my addr 172.21.15.32:0/2705726808 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.479+0000 7ff455400640 1 --2- 172.21.15.32:0/2705726808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 0x55c2aad81410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff455e00640 1 -- 172.21.15.32:0/2705726808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 msgr2=0x55c2aad81410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff455e00640 1 --2- 172.21.15.32:0/2705726808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 0x55c2aad81410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff455e00640 1 -- 172.21.15.32:0/2705726808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 msgr2=0x55c2aad80990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff455e00640 1 --2- 172.21.15.32:0/2705726808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 0x55c2aad80990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff455e00640 1 -- 172.21.15.32:0/2705726808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2aaa46650 con 0x55c2aa9b21b0 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff455400640 1 --2- 172.21.15.32:0/2705726808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 0x55c2aad81410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff44ea00640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff43c0b4ca0 0x7ff43c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff455e00640 1 --2- 172.21.15.32:0/2705726808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad80ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff44800bd50 tx=0x7ff4480107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7ca866a4393afa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff44ea00640 1 -- 172.21.15.32:0/328810913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff4500e5a50 con 0x7ff4581141d0 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff455400640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff43c0b4ca0 0x7ff43c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff443400640 1 -- 172.21.15.32:0/2705726808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448005220 con 0x55c2aa9b21b0 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.480+0000 7ff443400640 1 -- 172.21.15.32:0/2705726808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff44800be40 con 0x55c2aa9b21b0 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.481+0000 7ff443400640 1 -- 172.21.15.32:0/2705726808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448011780 con 0x55c2aa9b21b0 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff463c5e740 1 -- 172.21.15.32:0/2705726808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 msgr2=0x55c2aad80ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff455400640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff43c0b4ca0 0x7ff43c0b71b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff444000c00 tx=0x7ff444005780 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff463c5e740 1 --2- 172.21.15.32:0/2705726808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad80ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff44800bd50 tx=0x7ff4480107a0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff463c5e740 1 -- 172.21.15.32:0/2705726808 shutdown_connections 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff463c5e740 1 --2- 172.21.15.32:0/2705726808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 0x55c2aad81410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff463c5e740 1 --2- 172.21.15.32:0/2705726808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad80ed0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff463c5e740 1 --2- 172.21.15.32:0/2705726808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 0x55c2aad80990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff463c5e740 1 -- 172.21.15.32:0/2705726808 >> 172.21.15.32:0/2705726808 conn(0x55c2aa9cf460 msgr2=0x55c2aad7e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff463c5e740 1 -- 172.21.15.32:0/2705726808 shutdown_connections 2024-08-31T07:17:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff463c5e740 1 -- 172.21.15.32:0/2705726808 wait complete. 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.482+0000 7ff463c5e740 1 -- start start 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff463c5e740 1 Processor -- start 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9b21b0 0x55c2aad8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9ba390 0x55c2aad8cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9db070 0x55c2aad98460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff463c5e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c2aab49700 con 0x55c2aa9db070 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff463c5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c2aac6dcf0 con 0x55c2aa9ba390 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff463c5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c2aad8f170 con 0x55c2aa9b21b0 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff463c5e740 10 client.0 _ll_register_callbacks cb 0x55c2aac93470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff456800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9db070 0x55c2aad98460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff455e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9b21b0 0x55c2aad8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff455e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9b21b0 0x55c2aad8c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60482/0 (socket says 172.21.15.32:60482) 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff455e00640 1 -- 172.21.15.32:0/2617611342 learned_addr learned my addr 172.21.15.32:0/2617611342 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.483+0000 7ff455400640 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9ba390 0x55c2aad8cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.484+0000 7ff455e00640 1 -- 172.21.15.32:0/2617611342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9ba390 msgr2=0x55c2aad8cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.484+0000 7ff455e00640 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9ba390 0x55c2aad8cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.484+0000 7ff455e00640 1 -- 172.21.15.32:0/2617611342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9db070 msgr2=0x55c2aad98460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.484+0000 7ff455e00640 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9db070 0x55c2aad98460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.484+0000 7ff455e00640 1 -- 172.21.15.32:0/2617611342 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2aacae8b0 con 0x55c2aa9b21b0 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.484+0000 7ff455e00640 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9b21b0 0x55c2aad8c580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff448005730 tx=0x7ff448011750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691a5644dbc3f300 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.484+0000 7ff437e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.484+0000 7ff437e00640 1 -- 172.21.15.32:0/2617611342 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44802a060 con 0x55c2aa9b21b0 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.484+0000 7ff463c5e740 1 -- 172.21.15.32:0/2617611342 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c2aacf9700 con 0x55c2aa9b21b0 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.484+0000 7ff437e00640 1 -- 172.21.15.32:0/2617611342 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff448011b50 con 0x55c2aa9b21b0 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.485+0000 7ff437e00640 1 -- 172.21.15.32:0/2617611342 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff448011d50 con 0x55c2aa9b21b0 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.485+0000 7ff463c5e740 10 client.26521 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.485+0000 7ff463c5e740 1 -- 172.21.15.32:0/2617611342 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c2aad9b150 con 0x55c2aa9b21b0 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.485+0000 7ff437e00640 1 -- 172.21.15.32:0/2617611342 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c2aacf9700 con 0x55c2aa9b21b0 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.485+0000 7ff437e00640 1 -- 172.21.15.32:0/2617611342 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff448019330 con 0x55c2aa9b21b0 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.485+0000 7ff463c5e740 10 client.26521 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.485+0000 7ff463c5e740 10 client.26521 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.485+0000 7ff463c5e740 20 client.26521 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.486+0000 7ff463c5e740 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55c2aa9999f0 0x55c2aad9e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.486+0000 7ff463c5e740 4 client.26521 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.486+0000 7ff463c5e740 1 -- 172.21.15.32:0/2617611342 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2aacf51a0 con 0x55c2aa9999f0 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.486+0000 7ff455400640 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55c2aa9999f0 0x55c2aad9e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.487+0000 7ff455400640 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55c2aa9999f0 0x55c2aad9e510 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2dd23dead475d93b server_cookie=11ac55b14baaed97 in_seq=0 out_seq=0 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.487+0000 7ff437e00640 10 client.26521 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.487+0000 7ff437e00640 1 -- 172.21.15.32:0/2617611342 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c2aacf51a0 con 0x55c2aa9999f0 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.487+0000 7ff437e00640 10 client.26521 handle_command_reply: tid=0 2024-08-31T07:17:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.487+0000 7ff463c5e740 1 client.26521 shutdown 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 1 -- 172.21.15.32:0/2617611342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9b21b0 msgr2=0x55c2aad8c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9b21b0 0x55c2aad8c580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff448005730 tx=0x7ff448011750 comp rx=0 tx=0).stop 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 1 -- 172.21.15.32:0/2617611342 shutdown_connections 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55c2aa9999f0 0x55c2aad9e510 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9db070 0x55c2aad98460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9ba390 0x55c2aad8cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff437e00640 0 client.26521 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 1 --2- 172.21.15.32:0/2617611342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9b21b0 0x55c2aad8c580 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 1 -- 172.21.15.32:0/2617611342 >> 172.21.15.32:0/2617611342 conn(0x55c2aa9cf460 msgr2=0x55c2aad83b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 1 -- 172.21.15.32:0/2617611342 shutdown_connections 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 1 -- 172.21.15.32:0/2617611342 wait complete. 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.488+0000 7ff463c5e740 20 client.26521 trim_cache size 0 max 16384 2024-08-31T07:17:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.502+0000 7ff463c5e740 1 Processor -- start 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff463c5e740 1 -- start start 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9b21b0 0x55c2aad820d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9ba390 0x55c2aad7de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9db070 0x55c2aad7e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff463c5e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff4581b58e0 con 0x55c2aa9ba390 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff463c5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff4581b7420 con 0x55c2aa9b21b0 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff463c5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff4581b5fb0 con 0x55c2aa9db070 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff455400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9ba390 0x55c2aad7de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff455e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9b21b0 0x55c2aad820d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff455400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9ba390 0x55c2aad7de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52440/0 (socket says 172.21.15.32:52440) 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff455400640 1 -- 172.21.15.32:0/4074270599 learned_addr learned my addr 172.21.15.32:0/4074270599 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.503+0000 7ff456800640 1 --2- 172.21.15.32:0/4074270599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9db070 0x55c2aad7e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.504+0000 7ff455e00640 1 -- 172.21.15.32:0/4074270599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9db070 msgr2=0x55c2aad7e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.504+0000 7ff455e00640 1 --2- 172.21.15.32:0/4074270599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9db070 0x55c2aad7e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.504+0000 7ff455e00640 1 -- 172.21.15.32:0/4074270599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9ba390 msgr2=0x55c2aad7de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.504+0000 7ff455e00640 1 --2- 172.21.15.32:0/4074270599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9ba390 0x55c2aad7de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.504+0000 7ff455e00640 1 -- 172.21.15.32:0/4074270599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2aad8d640 con 0x55c2aa9b21b0 2024-08-31T07:17:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.504+0000 7ff455e00640 1 --2- 172.21.15.32:0/4074270599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9b21b0 0x55c2aad820d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff4480057c0 tx=0x7ff44800b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8bcae9573e4c8fe server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.504+0000 7ff442a00640 1 -- 172.21.15.32:0/4074270599 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44802a070 con 0x55c2aa9b21b0 2024-08-31T07:17:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.504+0000 7ff442a00640 1 -- 172.21.15.32:0/4074270599 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff44800b410 con 0x55c2aa9b21b0 2024-08-31T07:17:24.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.504+0000 7ff442a00640 1 -- 172.21.15.32:0/4074270599 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44801b8e0 con 0x55c2aa9b21b0 2024-08-31T07:17:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.505+0000 7ff463c5e740 1 -- 172.21.15.32:0/4074270599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9b21b0 msgr2=0x55c2aad820d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.505+0000 7ff463c5e740 1 --2- 172.21.15.32:0/4074270599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9b21b0 0x55c2aad820d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff4480057c0 tx=0x7ff44800b630 comp rx=0 tx=0).stop 2024-08-31T07:17:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.505+0000 7ff463c5e740 1 -- 172.21.15.32:0/4074270599 shutdown_connections 2024-08-31T07:17:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.505+0000 7ff463c5e740 1 --2- 172.21.15.32:0/4074270599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9db070 0x55c2aad7e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.505+0000 7ff463c5e740 1 --2- 172.21.15.32:0/4074270599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9ba390 0x55c2aad7de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.505+0000 7ff463c5e740 1 --2- 172.21.15.32:0/4074270599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9b21b0 0x55c2aad820d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.505+0000 7ff463c5e740 1 -- 172.21.15.32:0/4074270599 >> 172.21.15.32:0/4074270599 conn(0x55c2aa9cf460 msgr2=0x55c2aad819c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.505+0000 7ff463c5e740 1 -- 172.21.15.32:0/4074270599 shutdown_connections 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.505+0000 7ff463c5e740 1 -- 172.21.15.32:0/4074270599 wait complete. 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.506+0000 7ff463c5e740 1 -- start start 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.506+0000 7ff463c5e740 1 Processor -- start 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad9f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 0x55c2aad9faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff463c5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 0x55c2aad9ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff463c5e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff4581b6d50 con 0x55c2aa9b21b0 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff463c5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff4581b6680 con 0x55c2aa9db070 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff463c5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff4581b38c0 con 0x55c2aa9ba390 2024-08-31T07:17:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff463c5e740 10 client.0 _ll_register_callbacks cb 0x7ff4480054f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff455e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad9f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff456800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 0x55c2aad9ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff455e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad9f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52450/0 (socket says 172.21.15.32:52450) 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff455e00640 1 -- 172.21.15.32:0/811552908 learned_addr learned my addr 172.21.15.32:0/811552908 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff456800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 0x55c2aad9ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55504/0 (socket says 172.21.15.32:55504) 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff455400640 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 0x55c2aad9faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff456800640 1 -- 172.21.15.32:0/811552908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 msgr2=0x55c2aad9faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff456800640 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 0x55c2aad9faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff456800640 1 -- 172.21.15.32:0/811552908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 msgr2=0x55c2aad9f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff456800640 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad9f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.507+0000 7ff456800640 1 -- 172.21.15.32:0/811552908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2aae07cc0 con 0x55c2aa9db070 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.508+0000 7ff456800640 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 0x55c2aad9ffe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff4500e31c0 tx=0x7ff450005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6836238753d60d29 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.508+0000 7ff437e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:17:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.508+0000 7ff437e00640 1 -- 172.21.15.32:0/811552908 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4500abc70 con 0x55c2aa9db070 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.508+0000 7ff463c5e740 1 -- 172.21.15.32:0/811552908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c2aacf28e0 con 0x55c2aa9db070 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.508+0000 7ff437e00640 1 -- 172.21.15.32:0/811552908 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff4500b5200 con 0x55c2aa9db070 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.508+0000 7ff437e00640 1 -- 172.21.15.32:0/811552908 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff4500ac860 con 0x55c2aa9db070 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.508+0000 7ff463c5e740 10 client.26516 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.508+0000 7ff463c5e740 1 -- 172.21.15.32:0/811552908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c2aae08ff0 con 0x55c2aa9db070 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.508+0000 7ff437e00640 1 -- 172.21.15.32:0/811552908 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c2aacf28e0 con 0x55c2aa9db070 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.509+0000 7ff437e00640 1 -- 172.21.15.32:0/811552908 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff4500a7bf0 con 0x55c2aa9db070 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.509+0000 7ff463c5e740 10 client.26516 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.509+0000 7ff463c5e740 10 client.26516 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.509+0000 7ff463c5e740 20 client.26516 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.509+0000 7ff463c5e740 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55c2aa96eda0 0x55c2aae0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.509+0000 7ff463c5e740 4 client.26516 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.509+0000 7ff463c5e740 1 -- 172.21.15.32:0/811552908 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2aac691a0 con 0x55c2aa96eda0 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.509+0000 7ff455e00640 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55c2aa96eda0 0x55c2aae0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:24.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff455e00640 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55c2aa96eda0 0x55c2aae0b760 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4230ff4634cbe6ba server_cookie=e3422d328e45a358 in_seq=0 out_seq=0 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff437e00640 10 client.26516 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff437e00640 1 -- 172.21.15.32:0/811552908 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2aac691a0 con 0x55c2aa96eda0 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff437e00640 10 client.26516 handle_command_reply: tid=0 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff463c5e740 1 client.26516 shutdown 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff463c5e740 1 -- 172.21.15.32:0/811552908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 msgr2=0x55c2aad9ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff463c5e740 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 0x55c2aad9ffe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff4500e31c0 tx=0x7ff450005600 comp rx=0 tx=0).stop 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff463c5e740 1 -- 172.21.15.32:0/811552908 shutdown_connections 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff463c5e740 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55c2aa96eda0 0x55c2aae0b760 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff463c5e740 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2aa9db070 0x55c2aad9ffe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff463c5e740 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2aa9ba390 0x55c2aad9faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff463c5e740 1 --2- 172.21.15.32:0/811552908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c2aa9b21b0 0x55c2aad9f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.510+0000 7ff437e00640 0 client.26516 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.511+0000 7ff463c5e740 1 -- 172.21.15.32:0/811552908 >> 172.21.15.32:0/811552908 conn(0x55c2aa9cf460 msgr2=0x55c2aad8ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.511+0000 7ff463c5e740 1 -- 172.21.15.32:0/811552908 shutdown_connections 2024-08-31T07:17:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.511+0000 7ff463c5e740 1 -- 172.21.15.32:0/811552908 wait complete. 2024-08-31T07:17:24.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.511+0000 7ff463c5e740 20 client.26516 trim_cache size 0 max 16384 2024-08-31T07:17:24.524 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.511+0000 7ff45d000640 1 -- 172.21.15.32:0/328810913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff43c0b4ca0 msgr2=0x7ff43c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.511+0000 7ff45d000640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff43c0b4ca0 0x7ff43c0b71b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff444000c00 tx=0x7ff444005780 comp rx=0 tx=0).stop 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.511+0000 7ff45d000640 1 -- 172.21.15.32:0/328810913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4581141d0 msgr2=0x7ff4581a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.511+0000 7ff45d000640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4581141d0 0x7ff4581a6020 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff45000eb10 tx=0x7ff45000c2d0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.512+0000 7ff45d000640 1 -- 172.21.15.32:0/328810913 shutdown_connections 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.512+0000 7ff45d000640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff43c0b4ca0 0x7ff43c0b71b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.512+0000 7ff45d000640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4581141d0 0x7ff4581a6020 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.512+0000 7ff45d000640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff458077120 0x7ff4581a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.512+0000 7ff45d000640 1 --2- 172.21.15.32:0/328810913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff458075b70 0x7ff4581a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.512+0000 7ff45d000640 1 -- 172.21.15.32:0/328810913 >> 172.21.15.32:0/328810913 conn(0x7ff45807b5f0 msgr2=0x7ff458112e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.512+0000 7ff45d000640 1 -- 172.21.15.32:0/328810913 shutdown_connections 2024-08-31T07:17:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:24.512+0000 7ff45d000640 1 -- 172.21.15.32:0/328810913 wait complete. 2024-08-31T07:17:24.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 131999 2.10 MB/sec execute 73 sec latency 2629.089 ms 2024-08-31T07:17:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:24 smithi084 ceph-mon[22387]: pgmap v507: 97 pgs: 97 active+clean; 79 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-08-31T07:17:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:24 smithi073 ceph-mon[21995]: pgmap v507: 97 pgs: 97 active+clean; 79 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-08-31T07:17:25.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:17:25.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 133575 2.16 MB/sec execute 74 sec latency 3044.951 ms 2024-08-31T07:17:25.915 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:17:26.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.647+0000 7fe387200640 1 -- 172.21.15.32:0/1366068625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3801050c0 msgr2=0x7fe38010d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:26.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.647+0000 7fe387200640 1 --2- 172.21.15.32:0/1366068625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3801050c0 0x7fe38010d610 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe368009900 tx=0x7fe36802a400 comp rx=0 tx=0).stop 2024-08-31T07:17:26.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.647+0000 7fe387200640 1 -- 172.21.15.32:0/1366068625 shutdown_connections 2024-08-31T07:17:26.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.647+0000 7fe387200640 1 --2- 172.21.15.32:0/1366068625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe38010dbd0 0x7fe3801100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.647+0000 7fe387200640 1 --2- 172.21.15.32:0/1366068625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3801050c0 0x7fe38010d610 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.647+0000 7fe387200640 1 --2- 172.21.15.32:0/1366068625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe380104770 0x7fe380104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.647+0000 7fe387200640 1 -- 172.21.15.32:0/1366068625 >> 172.21.15.32:0/1366068625 conn(0x7fe3800794e0 msgr2=0x7fe3800798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:26.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.648+0000 7fe387200640 1 -- 172.21.15.32:0/1366068625 shutdown_connections 2024-08-31T07:17:26.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.648+0000 7fe387200640 1 -- 172.21.15.32:0/1366068625 wait complete. 2024-08-31T07:17:26.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.649+0000 7fe387200640 1 Processor -- start 2024-08-31T07:17:26.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.649+0000 7fe387200640 1 -- start start 2024-08-31T07:17:26.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.649+0000 7fe387200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe380104770 0x7fe3801024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.649+0000 7fe387200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3801050c0 0x7fe3801029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe387200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe38010dbd0 0x7fe380102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe387200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801137f0 con 0x7fe38010dbd0 2024-08-31T07:17:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe387200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe380113670 con 0x7fe3801050c0 2024-08-31T07:17:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe387200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe380113970 con 0x7fe380104770 2024-08-31T07:17:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe37fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe380104770 0x7fe3801024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe377400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3801050c0 0x7fe3801029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe37fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe380104770 0x7fe3801024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47594/0 (socket says 172.21.15.32:47594) 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe37fe00640 1 -- 172.21.15.32:0/4198709593 learned_addr learned my addr 172.21.15.32:0/4198709593 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe385200640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe38010dbd0 0x7fe380102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe377400640 1 -- 172.21.15.32:0/4198709593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe380104770 msgr2=0x7fe3801024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe377400640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe380104770 0x7fe3801024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe377400640 1 -- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe38010dbd0 msgr2=0x7fe380102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe377400640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe38010dbd0 0x7fe380102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe377400640 1 -- 172.21.15.32:0/4198709593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3801a8540 con 0x7fe3801050c0 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe385200640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe38010dbd0 0x7fe380102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.650+0000 7fe37fe00640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe380104770 0x7fe3801024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:26.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.651+0000 7fe377400640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3801050c0 0x7fe3801029f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe36802a920 tx=0x7fe368031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:26.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.651+0000 7fe37d600640 1 -- 172.21.15.32:0/4198709593 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36803f070 con 0x7fe3801050c0 2024-08-31T07:17:26.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.651+0000 7fe37d600640 1 -- 172.21.15.32:0/4198709593 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe368004490 con 0x7fe3801050c0 2024-08-31T07:17:26.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.651+0000 7fe387200640 1 -- 172.21.15.32:0/4198709593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3801a87d0 con 0x7fe3801050c0 2024-08-31T07:17:26.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.651+0000 7fe37d600640 1 -- 172.21.15.32:0/4198709593 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368035070 con 0x7fe3801050c0 2024-08-31T07:17:26.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.651+0000 7fe387200640 1 -- 172.21.15.32:0/4198709593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3801a8d00 con 0x7fe3801050c0 2024-08-31T07:17:26.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.654+0000 7fe38dec8740 1 Processor -- start 2024-08-31T07:17:26.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.654+0000 7fe38dec8740 1 -- start start 2024-08-31T07:17:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 0x55ce800d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd041b0 0x55ce800d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 0x55ce800d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe38dec8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ce800497b0 con 0x55ce7fd041b0 2024-08-31T07:17:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe38dec8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ce80038d40 con 0x55ce7fd2d070 2024-08-31T07:17:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe38dec8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ce8003ccf0 con 0x55ce7fd0c390 2024-08-31T07:17:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe37d600640 1 -- 172.21.15.32:0/4198709593 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe36803b460 con 0x7fe3801050c0 2024-08-31T07:17:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe385200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 0x55ce800d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe385200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 0x55ce800d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45044/0 (socket says 172.21.15.32:45044) 2024-08-31T07:17:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe385200640 1 -- 172.21.15.32:0/1869312526 learned_addr learned my addr 172.21.15.32:0/1869312526 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe37fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd041b0 0x55ce800d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe377400640 1 --2- 172.21.15.32:0/1869312526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 0x55ce800d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe385200640 1 -- 172.21.15.32:0/1869312526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 msgr2=0x55ce800d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.655+0000 7fe385200640 1 --2- 172.21.15.32:0/1869312526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 0x55ce800d3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.656+0000 7fe385200640 1 -- 172.21.15.32:0/1869312526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd041b0 msgr2=0x55ce800d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.656+0000 7fe385200640 1 --2- 172.21.15.32:0/1869312526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd041b0 0x55ce800d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.656+0000 7fe385200640 1 -- 172.21.15.32:0/1869312526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce7fd98650 con 0x55ce7fd2d070 2024-08-31T07:17:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.656+0000 7fe385200640 1 --2- 172.21.15.32:0/1869312526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 0x55ce800d2990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe378009cc0 tx=0x7fe3780117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72a2f5c286e0d98 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.656+0000 7fe375600640 1 -- 172.21.15.32:0/1869312526 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378012450 con 0x55ce7fd2d070 2024-08-31T07:17:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.656+0000 7fe375600640 1 -- 172.21.15.32:0/1869312526 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe378012aa0 con 0x55ce7fd2d070 2024-08-31T07:17:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.656+0000 7fe375600640 1 -- 172.21.15.32:0/1869312526 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37800fbb0 con 0x55ce7fd2d070 2024-08-31T07:17:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.657+0000 7fe37d600640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3640b4d20 0x7fe3640b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.657+0000 7fe37d600640 1 -- 172.21.15.32:0/4198709593 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe368105970 con 0x7fe3801050c0 2024-08-31T07:17:26.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.657+0000 7fe37fe00640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3640b4d20 0x7fe3640b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.657+0000 7fe38dec8740 1 -- 172.21.15.32:0/1869312526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 msgr2=0x55ce800d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.657+0000 7fe38dec8740 1 --2- 172.21.15.32:0/1869312526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 0x55ce800d2990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe378009cc0 tx=0x7fe3780117b0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.658+0000 7fe38dec8740 1 -- 172.21.15.32:0/1869312526 shutdown_connections 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.658+0000 7fe38dec8740 1 --2- 172.21.15.32:0/1869312526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 0x55ce800d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.658+0000 7fe38dec8740 1 --2- 172.21.15.32:0/1869312526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd041b0 0x55ce800d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.658+0000 7fe38dec8740 1 --2- 172.21.15.32:0/1869312526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 0x55ce800d2990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.658+0000 7fe38dec8740 1 -- 172.21.15.32:0/1869312526 >> 172.21.15.32:0/1869312526 conn(0x55ce7fd21460 msgr2=0x55ce800d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.658+0000 7fe38dec8740 1 -- 172.21.15.32:0/1869312526 shutdown_connections 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.658+0000 7fe38dec8740 1 -- 172.21.15.32:0/1869312526 wait complete. 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.658+0000 7fe37fe00640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3640b4d20 0x7fe3640b7230 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ce800d1610 tx=0x7fe370009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.659+0000 7fe38dec8740 1 -- start start 2024-08-31T07:17:26.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.659+0000 7fe38dec8740 1 Processor -- start 2024-08-31T07:17:26.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.659+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd041b0 0x55ce800de5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.659+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 0x55ce800deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.659+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 0x55ce800ea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.659+0000 7fe38dec8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ce7fe9b700 con 0x55ce7fd2d070 2024-08-31T07:17:26.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.659+0000 7fe38dec8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ce7ffbfcf0 con 0x55ce7fd041b0 2024-08-31T07:17:26.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.659+0000 7fe38dec8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ce800e1170 con 0x55ce7fd0c390 2024-08-31T07:17:26.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.659+0000 7fe38dec8740 10 client.0 _ll_register_callbacks cb 0x55ce7ffe5470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:26.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe377400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 0x55ce800deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe377400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 0x55ce800deb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47606/0 (socket says 172.21.15.32:47606) 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe377400640 1 -- 172.21.15.32:0/2629863155 learned_addr learned my addr 172.21.15.32:0/2629863155 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe37fe00640 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd041b0 0x55ce800de5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe385200640 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 0x55ce800ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe377400640 1 -- 172.21.15.32:0/2629863155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd041b0 msgr2=0x55ce800de5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe377400640 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd041b0 0x55ce800de5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe377400640 1 -- 172.21.15.32:0/2629863155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 msgr2=0x55ce800ea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe377400640 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 0x55ce800ea460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe377400640 1 -- 172.21.15.32:0/2629863155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce800008b0 con 0x55ce7fd0c390 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe385200640 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 0x55ce800ea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe377400640 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 0x55ce800deb00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe3681031a0 tx=0x7fe368000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fa7effd165b807f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.660+0000 7fe36e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.661+0000 7fe36e000640 1 -- 172.21.15.32:0/2629863155 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368048050 con 0x55ce7fd0c390 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.661+0000 7fe36e000640 1 -- 172.21.15.32:0/2629863155 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe3680d4f70 con 0x55ce7fd0c390 2024-08-31T07:17:26.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.661+0000 7fe38dec8740 1 -- 172.21.15.32:0/2629863155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe378012aa0 con 0x55ce7fd0c390 2024-08-31T07:17:26.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.661+0000 7fe36e000640 1 -- 172.21.15.32:0/2629863155 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe3680cb850 con 0x55ce7fd0c390 2024-08-31T07:17:26.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.661+0000 7fe38dec8740 10 client.26539 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:26.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.661+0000 7fe38dec8740 1 -- 172.21.15.32:0/2629863155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ce800ed190 con 0x55ce7fd0c390 2024-08-31T07:17:26.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.661+0000 7fe36e000640 1 -- 172.21.15.32:0/2629863155 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe378012aa0 con 0x55ce7fd0c390 2024-08-31T07:17:26.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.661+0000 7fe36e000640 1 -- 172.21.15.32:0/2629863155 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe36804b060 con 0x55ce7fd0c390 2024-08-31T07:17:26.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.661+0000 7fe38dec8740 10 client.26539 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:26.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.661+0000 7fe38dec8740 10 client.26539 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:17:26.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.662+0000 7fe38dec8740 20 client.26539 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.662+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ce7fceb9f0 0x55ce800f0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.662+0000 7fe38dec8740 4 client.26539 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.662+0000 7fe38dec8740 1 -- 172.21.15.32:0/2629863155 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce8004b700 con 0x55ce7fceb9f0 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.662+0000 7fe37fe00640 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ce7fceb9f0 0x55ce800f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.662+0000 7fe37fe00640 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ce7fceb9f0 0x55ce800f0600 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6ba7f81a12c207 server_cookie=27da3347ad511558 in_seq=0 out_seq=0 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.662+0000 7fe36e000640 10 client.26539 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.663+0000 7fe36e000640 1 -- 172.21.15.32:0/2629863155 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce8004b700 con 0x55ce7fceb9f0 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.663+0000 7fe36e000640 10 client.26539 handle_command_reply: tid=0 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.663+0000 7fe38dec8740 1 client.26539 shutdown 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 1 -- 172.21.15.32:0/2629863155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 msgr2=0x55ce800deb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 0x55ce800deb00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe3681031a0 tx=0x7fe368000bc0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 1 -- 172.21.15.32:0/2629863155 shutdown_connections 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ce7fceb9f0 0x55ce800f0600 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 0x55ce800ea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd0c390 0x55ce800deb00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2629863155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd041b0 0x55ce800de5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 1 -- 172.21.15.32:0/2629863155 >> 172.21.15.32:0/2629863155 conn(0x55ce7fd21460 msgr2=0x55ce800d5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:26.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe36e000640 0 client.26539 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:17:26.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 1 -- 172.21.15.32:0/2629863155 shutdown_connections 2024-08-31T07:17:26.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 1 -- 172.21.15.32:0/2629863155 wait complete. 2024-08-31T07:17:26.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.664+0000 7fe38dec8740 20 client.26539 trim_cache size 0 max 16384 2024-08-31T07:17:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.679+0000 7fe38dec8740 1 Processor -- start 2024-08-31T07:17:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.679+0000 7fe38dec8740 1 -- start start 2024-08-31T07:17:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.679+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800d81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.679+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd0c390 0x55ce800cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.679+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 0x55ce800d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.679+0000 7fe38dec8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801b1560 con 0x55ce7fd0c390 2024-08-31T07:17:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.679+0000 7fe38dec8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801b30a0 con 0x55ce7fd2d070 2024-08-31T07:17:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.679+0000 7fe38dec8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801b1c30 con 0x55ce7fd041b0 2024-08-31T07:17:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe37fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800d81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe385200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 0x55ce800d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe377400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd0c390 0x55ce800cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe37fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800d81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47622/0 (socket says 172.21.15.32:47622) 2024-08-31T07:17:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe37fe00640 1 -- 172.21.15.32:0/62254858 learned_addr learned my addr 172.21.15.32:0/62254858 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe37fe00640 1 -- 172.21.15.32:0/62254858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 msgr2=0x55ce800d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe37fe00640 1 --2- 172.21.15.32:0/62254858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 0x55ce800d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe37fe00640 1 -- 172.21.15.32:0/62254858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd0c390 msgr2=0x55ce800cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe37fe00640 1 --2- 172.21.15.32:0/62254858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd0c390 0x55ce800cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe37fe00640 1 -- 172.21.15.32:0/62254858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce800f0970 con 0x55ce7fd041b0 2024-08-31T07:17:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.680+0000 7fe37fe00640 1 --2- 172.21.15.32:0/62254858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800d81b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe380103f00 tx=0x7fe37001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f13ff2b675b355da server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.681+0000 7fe374c00640 1 -- 172.21.15.32:0/62254858 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37001c9e0 con 0x55ce7fd041b0 2024-08-31T07:17:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.681+0000 7fe374c00640 1 -- 172.21.15.32:0/62254858 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe370003e20 con 0x55ce7fd041b0 2024-08-31T07:17:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.681+0000 7fe374c00640 1 -- 172.21.15.32:0/62254858 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370004620 con 0x55ce7fd041b0 2024-08-31T07:17:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.681+0000 7fe38dec8740 1 -- 172.21.15.32:0/62254858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 msgr2=0x55ce800d81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.681+0000 7fe38dec8740 1 --2- 172.21.15.32:0/62254858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800d81b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe380103f00 tx=0x7fe37001a000 comp rx=0 tx=0).stop 2024-08-31T07:17:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.682+0000 7fe38dec8740 1 -- 172.21.15.32:0/62254858 shutdown_connections 2024-08-31T07:17:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.682+0000 7fe38dec8740 1 --2- 172.21.15.32:0/62254858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd2d070 0x55ce800d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.682+0000 7fe38dec8740 1 --2- 172.21.15.32:0/62254858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd0c390 0x55ce800cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.682+0000 7fe38dec8740 1 --2- 172.21.15.32:0/62254858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800d81b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.682+0000 7fe38dec8740 1 -- 172.21.15.32:0/62254858 >> 172.21.15.32:0/62254858 conn(0x55ce7fd21460 msgr2=0x55ce800d7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.682+0000 7fe38dec8740 1 -- 172.21.15.32:0/62254858 shutdown_connections 2024-08-31T07:17:26.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.682+0000 7fe38dec8740 1 -- 172.21.15.32:0/62254858 wait complete. 2024-08-31T07:17:26.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.682+0000 7fe38dec8740 1 -- start start 2024-08-31T07:17:26.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.682+0000 7fe38dec8740 1 Processor -- start 2024-08-31T07:17:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd0c390 0x55ce800e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe38dec8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 0x55ce800e38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe38dec8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801b29d0 con 0x55ce7fd2d070 2024-08-31T07:17:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe38dec8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801b2300 con 0x55ce7fd0c390 2024-08-31T07:17:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe38dec8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801af540 con 0x55ce7fd041b0 2024-08-31T07:17:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe38dec8740 10 client.0 _ll_register_callbacks cb 0x7fe368009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe37fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe37fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800e6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47628/0 (socket says 172.21.15.32:47628) 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe37fe00640 1 -- 172.21.15.32:0/2704260521 learned_addr learned my addr 172.21.15.32:0/2704260521 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe377400640 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd0c390 0x55ce800e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.683+0000 7fe385200640 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 0x55ce800e38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe37fe00640 1 -- 172.21.15.32:0/2704260521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd0c390 msgr2=0x55ce800e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe37fe00640 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd0c390 0x55ce800e31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe37fe00640 1 -- 172.21.15.32:0/2704260521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 msgr2=0x55ce800e38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe37fe00640 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 0x55ce800e38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe37fe00640 1 -- 172.21.15.32:0/2704260521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce8015af80 con 0x55ce7fd041b0 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe385200640 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 0x55ce800e38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe377400640 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd0c390 0x55ce800e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe37fe00640 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800e6d20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe370004140 tx=0x7fe370004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3e4a34ddfaa9234 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe36e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:17:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe36e000640 1 -- 172.21.15.32:0/2704260521 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37001c9e0 con 0x55ce7fd041b0 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe36e000640 1 -- 172.21.15.32:0/2704260521 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe37001cb80 con 0x55ce7fd041b0 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.684+0000 7fe38dec8740 1 -- 172.21.15.32:0/2704260521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe370003e20 con 0x55ce7fd041b0 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe36e000640 1 -- 172.21.15.32:0/2704260521 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe37001cd80 con 0x55ce7fd041b0 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe38dec8740 10 client.26545 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe38dec8740 1 -- 172.21.15.32:0/2704260521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ce8015c040 con 0x55ce7fd041b0 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe36e000640 1 -- 172.21.15.32:0/2704260521 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe370003e20 con 0x55ce7fd041b0 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe36e000640 1 -- 172.21.15.32:0/2704260521 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe370037070 con 0x55ce7fd041b0 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe38dec8740 10 client.26545 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe38dec8740 10 client.26545 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe38dec8740 20 client.26545 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ce7fcc0da0 0x55ce8015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe38dec8740 4 client.26545 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:17:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.685+0000 7fe38dec8740 1 -- 172.21.15.32:0/2704260521 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce800471a0 con 0x55ce7fcc0da0 2024-08-31T07:17:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.686+0000 7fe377400640 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ce7fcc0da0 0x55ce8015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.686+0000 7fe377400640 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ce7fcc0da0 0x55ce8015e770 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e5a6079257e80d2 server_cookie=995a84ac26110836 in_seq=0 out_seq=0 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.686+0000 7fe36e000640 10 client.26545 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.686+0000 7fe36e000640 1 -- 172.21.15.32:0/2704260521 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce800471a0 con 0x55ce7fcc0da0 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.686+0000 7fe36e000640 10 client.26545 handle_command_reply: tid=0 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.686+0000 7fe38dec8740 1 client.26545 shutdown 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 1 -- 172.21.15.32:0/2704260521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 msgr2=0x55ce800e6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800e6d20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe370004140 tx=0x7fe370004170 comp rx=0 tx=0).stop 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 1 -- 172.21.15.32:0/2704260521 shutdown_connections 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ce7fcc0da0 0x55ce8015e770 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce7fd2d070 0x55ce800e38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce7fd0c390 0x55ce800e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 1 --2- 172.21.15.32:0/2704260521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce7fd041b0 0x55ce800e6d20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 1 -- 172.21.15.32:0/2704260521 >> 172.21.15.32:0/2704260521 conn(0x55ce7fd21460 msgr2=0x55ce800df870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe36e000640 0 client.26545 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:17:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 1 -- 172.21.15.32:0/2704260521 shutdown_connections 2024-08-31T07:17:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 1 -- 172.21.15.32:0/2704260521 wait complete. 2024-08-31T07:17:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.687+0000 7fe38dec8740 20 client.26545 trim_cache size 0 max 16384 2024-08-31T07:17:26.691 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 -- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3640b4d20 msgr2=0x7fe3640b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3640b4d20 0x7fe3640b7230 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ce800d1610 tx=0x7fe370009500 comp rx=0 tx=0).stop 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 -- 172.21.15.32:0/4198709593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3801050c0 msgr2=0x7fe3801029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3801050c0 0x7fe3801029f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe36802a920 tx=0x7fe368031a20 comp rx=0 tx=0).stop 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 -- 172.21.15.32:0/4198709593 shutdown_connections 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe3640b4d20 0x7fe3640b7230 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe38010dbd0 0x7fe380102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3801050c0 0x7fe3801029f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 --2- 172.21.15.32:0/4198709593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe380104770 0x7fe3801024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 -- 172.21.15.32:0/4198709593 >> 172.21.15.32:0/4198709593 conn(0x7fe3800794e0 msgr2=0x7fe3800ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 -- 172.21.15.32:0/4198709593 shutdown_connections 2024-08-31T07:17:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:26.688+0000 7fe387200640 1 -- 172.21.15.32:0/4198709593 wait complete. 2024-08-31T07:17:26.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135206 2.29 MB/sec execute 75 sec latency 10.410 ms 2024-08-31T07:17:26.712 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:26 smithi032 ceph-mon[17252]: pgmap v508: 97 pgs: 97 active+clean; 80 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 887 KiB/s wr, 106 op/s 2024-08-31T07:17:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:26 smithi084 ceph-mon[22387]: pgmap v508: 97 pgs: 97 active+clean; 80 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 887 KiB/s wr, 106 op/s 2024-08-31T07:17:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:26 smithi073 ceph-mon[21995]: pgmap v508: 97 pgs: 97 active+clean; 80 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 887 KiB/s wr, 106 op/s 2024-08-31T07:17:27.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135487 2.26 MB/sec execute 76 sec latency 670.931 ms 2024-08-31T07:17:27.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:17:28.114 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:17:28.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:28 smithi032 ceph-mon[17252]: pgmap v509: 97 pgs: 97 active+clean; 81 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 539 KiB/s wr, 99 op/s 2024-08-31T07:17:28.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135487 2.23 MB/sec execute 77 sec latency 1670.996 ms 2024-08-31T07:17:28.731 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:28 smithi073 ceph-mon[21995]: pgmap v509: 97 pgs: 97 active+clean; 81 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 539 KiB/s wr, 99 op/s 2024-08-31T07:17:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:28 smithi084 ceph-mon[22387]: pgmap v509: 97 pgs: 97 active+clean; 81 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 539 KiB/s wr, 99 op/s 2024-08-31T07:17:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.798+0000 7f6874200640 1 -- 172.21.15.32:0/1028887035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f686c109b90 msgr2=0x7f686c1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.798+0000 7f6874200640 1 --2- 172.21.15.32:0/1028887035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f686c109b90 0x7f686c1103c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6864009ca0 tx=0x7f686402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.798+0000 7f6874200640 1 -- 172.21.15.32:0/1028887035 shutdown_connections 2024-08-31T07:17:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.798+0000 7f6874200640 1 --2- 172.21.15.32:0/1028887035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f686c109b90 0x7f686c1103c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.798+0000 7f6874200640 1 --2- 172.21.15.32:0/1028887035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f686c109240 0x7f686c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.798+0000 7f6874200640 1 --2- 172.21.15.32:0/1028887035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c103900 0x7f686c103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.798+0000 7f6874200640 1 -- 172.21.15.32:0/1028887035 >> 172.21.15.32:0/1028887035 conn(0x7f686c07b5f0 msgr2=0x7f686c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:28.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.798+0000 7f6874200640 1 -- 172.21.15.32:0/1028887035 shutdown_connections 2024-08-31T07:17:28.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.798+0000 7f6874200640 1 -- 172.21.15.32:0/1028887035 wait complete. 2024-08-31T07:17:28.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.799+0000 7f6874200640 1 Processor -- start 2024-08-31T07:17:28.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.799+0000 7f6874200640 1 -- start start 2024-08-31T07:17:28.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6874200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f686c103900 0x7f686c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6874200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f686c109240 0x7f686c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6874200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c109b90 0x7f686c19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6874200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f686c1138a0 con 0x7f686c109240 2024-08-31T07:17:28.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6874200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f686c113720 con 0x7f686c103900 2024-08-31T07:17:28.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6874200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f686c113a20 con 0x7f686c109b90 2024-08-31T07:17:28.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6871800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f686c103900 0x7f686c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6871800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f686c103900 0x7f686c19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45094/0 (socket says 172.21.15.32:45094) 2024-08-31T07:17:28.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6871800640 1 -- 172.21.15.32:0/387864306 learned_addr learned my addr 172.21.15.32:0/387864306 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:28.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6870e00640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f686c109240 0x7f686c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6872200640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c109b90 0x7f686c19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6871800640 1 -- 172.21.15.32:0/387864306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c109b90 msgr2=0x7f686c19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:28.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6871800640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c109b90 0x7f686c19fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6871800640 1 -- 172.21.15.32:0/387864306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f686c109240 msgr2=0x7f686c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:28.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.800+0000 7f6871800640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f686c109240 0x7f686c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.801+0000 7f6871800640 1 -- 172.21.15.32:0/387864306 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c1a0140 con 0x7f686c103900 2024-08-31T07:17:28.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.801+0000 7f6871800640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f686c103900 0x7f686c19a1d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f686000d950 tx=0x7f686000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:28.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.801+0000 7f686a000640 1 -- 172.21.15.32:0/387864306 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860017030 con 0x7f686c103900 2024-08-31T07:17:28.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.801+0000 7f6874200640 1 -- 172.21.15.32:0/387864306 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686c1a0430 con 0x7f686c103900 2024-08-31T07:17:28.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.801+0000 7f686a000640 1 -- 172.21.15.32:0/387864306 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6860004020 con 0x7f686c103900 2024-08-31T07:17:28.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.801+0000 7f6874200640 1 -- 172.21.15.32:0/387864306 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686c1a6b90 con 0x7f686c103900 2024-08-31T07:17:28.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.801+0000 7f686a000640 1 -- 172.21.15.32:0/387864306 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600047a0 con 0x7f686c103900 2024-08-31T07:17:28.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.805+0000 7f687ae32740 1 Processor -- start 2024-08-31T07:17:28.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.805+0000 7f687ae32740 1 -- start start 2024-08-31T07:17:28.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.806+0000 7f687ae32740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 0x56255b1ec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.806+0000 7f687ae32740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae1e1b0 0x56255b1eced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.806+0000 7f687ae32740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae26390 0x56255b1ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.806+0000 7f687ae32740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56255b1637b0 con 0x56255ae47070 2024-08-31T07:17:28.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.806+0000 7f687ae32740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56255b152d40 con 0x56255ae1e1b0 2024-08-31T07:17:28.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.806+0000 7f687ae32740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56255b156cf0 con 0x56255ae26390 2024-08-31T07:17:28.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.806+0000 7f6872200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 0x56255b1ec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.806+0000 7f6872200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 0x56255b1ec990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52690/0 (socket says 172.21.15.32:52690) 2024-08-31T07:17:28.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.806+0000 7f6872200640 1 -- 172.21.15.32:0/2469856131 learned_addr learned my addr 172.21.15.32:0/2469856131 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:28.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.806+0000 7f6870e00640 1 --2- 172.21.15.32:0/2469856131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae26390 0x56255b1ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.807+0000 7f6872200640 1 -- 172.21.15.32:0/2469856131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae26390 msgr2=0x56255b1ed410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:28.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.807+0000 7f6872200640 1 --2- 172.21.15.32:0/2469856131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae26390 0x56255b1ed410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.807+0000 7f6872200640 1 -- 172.21.15.32:0/2469856131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae1e1b0 msgr2=0x56255b1eced0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:17:28.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.807+0000 7f6872200640 1 --2- 172.21.15.32:0/2469856131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae1e1b0 0x56255b1eced0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.807+0000 7f6872200640 1 -- 172.21.15.32:0/2469856131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255aeb2650 con 0x56255ae47070 2024-08-31T07:17:28.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.807+0000 7f686a000640 1 -- 172.21.15.32:0/387864306 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f68600041d0 con 0x7f686c103900 2024-08-31T07:17:28.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.807+0000 7f6872200640 1 --2- 172.21.15.32:0/2469856131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 0x56255b1ec990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6864009ca0 tx=0x7f686402f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484166fb5a9b8644 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:28.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.807+0000 7f685ea00640 1 -- 172.21.15.32:0/2469856131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864031db0 con 0x56255ae47070 2024-08-31T07:17:28.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.807+0000 7f685ea00640 1 -- 172.21.15.32:0/2469856131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864036030 con 0x56255ae47070 2024-08-31T07:17:28.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.807+0000 7f685ea00640 1 -- 172.21.15.32:0/2469856131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864004330 con 0x56255ae47070 2024-08-31T07:17:28.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.808+0000 7f687ae32740 1 -- 172.21.15.32:0/2469856131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 msgr2=0x56255b1ec990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:28.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.808+0000 7f687ae32740 1 --2- 172.21.15.32:0/2469856131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 0x56255b1ec990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6864009ca0 tx=0x7f686402f310 comp rx=0 tx=0).stop 2024-08-31T07:17:28.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f687ae32740 1 -- 172.21.15.32:0/2469856131 shutdown_connections 2024-08-31T07:17:28.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f687ae32740 1 --2- 172.21.15.32:0/2469856131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae26390 0x56255b1ed410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f687ae32740 1 --2- 172.21.15.32:0/2469856131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae1e1b0 0x56255b1eced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f687ae32740 1 --2- 172.21.15.32:0/2469856131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 0x56255b1ec990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f687ae32740 1 -- 172.21.15.32:0/2469856131 >> 172.21.15.32:0/2469856131 conn(0x56255ae3b460 msgr2=0x56255b1ea880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f687ae32740 1 -- 172.21.15.32:0/2469856131 shutdown_connections 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f687ae32740 1 -- 172.21.15.32:0/2469856131 wait complete. 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f686a000640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68540b4c30 0x7f68540b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f686a000640 1 -- 172.21.15.32:0/387864306 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f68600e4d10 con 0x7f686c103900 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f687ae32740 1 -- start start 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.809+0000 7f6870e00640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68540b4c30 0x7f68540b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.810+0000 7f687ae32740 1 Processor -- start 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.810+0000 7f687ae32740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae1e1b0 0x56255b1fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.810+0000 7f687ae32740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 0x56255b1fd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.810+0000 7f6870e00640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68540b4c30 0x7f68540b7140 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f686c108e00 tx=0x7f6858005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:28.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.810+0000 7f687ae32740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 0x56255b202500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.810+0000 7f687ae32740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56255afb5700 con 0x56255ae47070 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.810+0000 7f687ae32740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56255b0d9cf0 con 0x56255ae26390 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.810+0000 7f687ae32740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56255b1faf90 con 0x56255ae1e1b0 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.810+0000 7f687ae32740 10 client.0 _ll_register_callbacks cb 0x56255b0ff470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6871800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae1e1b0 0x56255b1fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6872200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 0x56255b202500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6871800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae1e1b0 0x56255b1fcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47654/0 (socket says 172.21.15.32:47654) 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6871800640 1 -- 172.21.15.32:0/2375119635 learned_addr learned my addr 172.21.15.32:0/2375119635 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6870e00640 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 0x56255b1fd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6871800640 1 -- 172.21.15.32:0/2375119635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 msgr2=0x56255b1fd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6871800640 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 0x56255b1fd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6871800640 1 -- 172.21.15.32:0/2375119635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 msgr2=0x56255b202500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:28.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6871800640 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 0x56255b202500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6871800640 1 -- 172.21.15.32:0/2375119635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56255b11a8b0 con 0x56255ae1e1b0 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6871800640 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae1e1b0 0x56255b1fcf00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6860009610 tx=0x7f68600e29b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c4937af84ad919 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.811+0000 7f6853400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.812+0000 7f6853400640 1 -- 172.21.15.32:0/2375119635 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600afc90 con 0x56255ae1e1b0 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.812+0000 7f6853400640 1 -- 172.21.15.32:0/2375119635 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f68600a76b0 con 0x56255ae1e1b0 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.812+0000 7f687ae32740 1 -- 172.21.15.32:0/2375119635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6864036030 con 0x56255ae1e1b0 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.812+0000 7f6853400640 1 -- 172.21.15.32:0/2375119635 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f68600b4740 con 0x56255ae1e1b0 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.812+0000 7f687ae32740 10 client.26557 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.812+0000 7f687ae32740 1 -- 172.21.15.32:0/2375119635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56255b20b450 con 0x56255ae1e1b0 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.812+0000 7f6853400640 1 -- 172.21.15.32:0/2375119635 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864036030 con 0x56255ae1e1b0 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.813+0000 7f6853400640 1 -- 172.21.15.32:0/2375119635 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f68600cccb0 con 0x56255ae1e1b0 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.813+0000 7f687ae32740 10 client.26557 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.813+0000 7f687ae32740 10 client.26557 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.813+0000 7f687ae32740 20 client.26557 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.813+0000 7f687ae32740 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56255ae059f0 0x56255b20e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.813+0000 7f687ae32740 4 client.26557 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:17:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.813+0000 7f687ae32740 1 -- 172.21.15.32:0/2375119635 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56255b165700 con 0x56255ae059f0 2024-08-31T07:17:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.813+0000 7f6870e00640 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56255ae059f0 0x56255b20e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.814+0000 7f6870e00640 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56255ae059f0 0x56255b20e8e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de45a0c1e1673290 server_cookie=aa3682109852c60e in_seq=0 out_seq=0 2024-08-31T07:17:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.814+0000 7f6853400640 10 client.26557 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.814+0000 7f6853400640 1 -- 172.21.15.32:0/2375119635 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56255b165700 con 0x56255ae059f0 2024-08-31T07:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.814+0000 7f6853400640 10 client.26557 handle_command_reply: tid=0 2024-08-31T07:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 client.26557 shutdown 2024-08-31T07:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 -- 172.21.15.32:0/2375119635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae1e1b0 msgr2=0x56255b1fcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae1e1b0 0x56255b1fcf00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6860009610 tx=0x7f68600e29b0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 -- 172.21.15.32:0/2375119635 shutdown_connections 2024-08-31T07:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56255ae059f0 0x56255b20e8e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae47070 0x56255b202500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 0x56255b1fd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 --2- 172.21.15.32:0/2375119635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae1e1b0 0x56255b1fcf00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f6853400640 0 client.26557 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:17:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 -- 172.21.15.32:0/2375119635 >> 172.21.15.32:0/2375119635 conn(0x56255ae3b460 msgr2=0x56255b1efb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 -- 172.21.15.32:0/2375119635 shutdown_connections 2024-08-31T07:17:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 1 -- 172.21.15.32:0/2375119635 wait complete. 2024-08-31T07:17:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.815+0000 7f687ae32740 20 client.26557 trim_cache size 0 max 16384 2024-08-31T07:17:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.829+0000 7f687ae32740 1 Processor -- start 2024-08-31T07:17:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.829+0000 7f687ae32740 1 -- start start 2024-08-31T07:17:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f687ae32740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae1e1b0 0x56255b1f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f687ae32740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae26390 0x56255b1f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f687ae32740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae47070 0x56255b2092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f687ae32740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f684c00e530 con 0x56255ae26390 2024-08-31T07:17:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f687ae32740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f684c010070 con 0x56255ae1e1b0 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f687ae32740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f684c00ec00 con 0x56255ae47070 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f6870e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae26390 0x56255b1f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f6870e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae26390 0x56255b1f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52708/0 (socket says 172.21.15.32:52708) 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f6870e00640 1 -- 172.21.15.32:0/3172800172 learned_addr learned my addr 172.21.15.32:0/3172800172 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f6871800640 1 --2- 172.21.15.32:0/3172800172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae1e1b0 0x56255b1f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f6872200640 1 --2- 172.21.15.32:0/3172800172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae47070 0x56255b2092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f6870e00640 1 -- 172.21.15.32:0/3172800172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae47070 msgr2=0x56255b2092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f6870e00640 1 --2- 172.21.15.32:0/3172800172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae47070 0x56255b2092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f6870e00640 1 -- 172.21.15.32:0/3172800172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae1e1b0 msgr2=0x56255b1f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f6870e00640 1 --2- 172.21.15.32:0/3172800172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae1e1b0 0x56255b1f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.830+0000 7f6870e00640 1 -- 172.21.15.32:0/3172800172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255b1fd940 con 0x56255ae26390 2024-08-31T07:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.831+0000 7f6870e00640 1 --2- 172.21.15.32:0/3172800172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae26390 0x56255b1f65d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f686c06c500 tx=0x7f6858019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea406628aaf452c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.831+0000 7f685e000640 1 -- 172.21.15.32:0/3172800172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685802c040 con 0x56255ae26390 2024-08-31T07:17:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.831+0000 7f685e000640 1 -- 172.21.15.32:0/3172800172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f685801ddb0 con 0x56255ae26390 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.831+0000 7f685e000640 1 -- 172.21.15.32:0/3172800172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685802a440 con 0x56255ae26390 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.832+0000 7f687ae32740 1 -- 172.21.15.32:0/3172800172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae26390 msgr2=0x56255b1f65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.832+0000 7f687ae32740 1 --2- 172.21.15.32:0/3172800172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae26390 0x56255b1f65d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f686c06c500 tx=0x7f6858019000 comp rx=0 tx=0).stop 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.832+0000 7f687ae32740 1 -- 172.21.15.32:0/3172800172 shutdown_connections 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.832+0000 7f687ae32740 1 --2- 172.21.15.32:0/3172800172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae47070 0x56255b2092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.832+0000 7f687ae32740 1 --2- 172.21.15.32:0/3172800172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae26390 0x56255b1f65d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.832+0000 7f687ae32740 1 --2- 172.21.15.32:0/3172800172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae1e1b0 0x56255b1f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.832+0000 7f687ae32740 1 -- 172.21.15.32:0/3172800172 >> 172.21.15.32:0/3172800172 conn(0x56255ae3b460 msgr2=0x56255b1fb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.832+0000 7f687ae32740 1 -- 172.21.15.32:0/3172800172 shutdown_connections 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.832+0000 7f687ae32740 1 -- 172.21.15.32:0/3172800172 wait complete. 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.833+0000 7f687ae32740 1 -- start start 2024-08-31T07:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.833+0000 7f687ae32740 1 Processor -- start 2024-08-31T07:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.833+0000 7f687ae32740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae1e1b0 0x56255b2030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.833+0000 7f687ae32740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 0x56255b203630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f687ae32740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae47070 0x56255b208450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f687ae32740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f684c00f9a0 con 0x56255ae1e1b0 2024-08-31T07:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f687ae32740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f684c00f2d0 con 0x56255ae26390 2024-08-31T07:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f687ae32740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f684c00c510 con 0x56255ae47070 2024-08-31T07:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f687ae32740 10 client.0 _ll_register_callbacks cb 0x7f68600aae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6870e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 0x56255b203630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6871800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae1e1b0 0x56255b2030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6870e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 0x56255b203630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45116/0 (socket says 172.21.15.32:45116) 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6872200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae47070 0x56255b208450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6870e00640 1 -- 172.21.15.32:0/1387365353 learned_addr learned my addr 172.21.15.32:0/1387365353 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6871800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae1e1b0 0x56255b2030f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52718/0 (socket says 172.21.15.32:52718) 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6871800640 1 -- 172.21.15.32:0/1387365353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae47070 msgr2=0x56255b208450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6871800640 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae47070 0x56255b208450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6871800640 1 -- 172.21.15.32:0/1387365353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 msgr2=0x56255b203630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6871800640 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 0x56255b203630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.834+0000 7f6871800640 1 -- 172.21.15.32:0/1387365353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56255b203ed0 con 0x56255ae1e1b0 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.835+0000 7f6871800640 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae1e1b0 0x56255b2030f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f68600ab3b0 tx=0x7f68600aa980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de39fd3ae95bd87e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.835+0000 7f6853400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.835+0000 7f6853400640 1 -- 172.21.15.32:0/1387365353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600a72e0 con 0x56255ae1e1b0 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.835+0000 7f6853400640 1 -- 172.21.15.32:0/1387365353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f68600e2bd0 con 0x56255ae1e1b0 2024-08-31T07:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.835+0000 7f687ae32740 1 -- 172.21.15.32:0/1387365353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f685801ddb0 con 0x56255ae1e1b0 2024-08-31T07:17:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.835+0000 7f6853400640 1 -- 172.21.15.32:0/1387365353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f68600afc90 con 0x56255ae1e1b0 2024-08-31T07:17:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.835+0000 7f687ae32740 10 client.16536 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.835+0000 7f687ae32740 1 -- 172.21.15.32:0/1387365353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56255b279830 con 0x56255ae1e1b0 2024-08-31T07:17:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.835+0000 7f6853400640 1 -- 172.21.15.32:0/1387365353 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f685801ddb0 con 0x56255ae1e1b0 2024-08-31T07:17:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.835+0000 7f6853400640 1 -- 172.21.15.32:0/1387365353 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f68600c9020 con 0x56255ae1e1b0 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.836+0000 7f687ae32740 10 client.16536 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.836+0000 7f687ae32740 10 client.16536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.836+0000 7f687ae32740 20 client.16536 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.836+0000 7f687ae32740 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56255addada0 0x56255b27bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.836+0000 7f687ae32740 4 client.16536 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.836+0000 7f687ae32740 1 -- 172.21.15.32:0/1387365353 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56255b1611a0 con 0x56255addada0 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.836+0000 7f6870e00640 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56255addada0 0x56255b27bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.836+0000 7f6870e00640 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56255addada0 0x56255b27bff0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64f225d7975de695 server_cookie=d4deaab1f9b9d84b in_seq=0 out_seq=0 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.836+0000 7f6853400640 10 client.16536 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f6853400640 1 -- 172.21.15.32:0/1387365353 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56255b1611a0 con 0x56255addada0 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f6853400640 10 client.16536 handle_command_reply: tid=0 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 client.16536 shutdown 2024-08-31T07:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 -- 172.21.15.32:0/1387365353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae1e1b0 msgr2=0x56255b2030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae1e1b0 0x56255b2030f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f68600ab3b0 tx=0x7f68600aa980 comp rx=0 tx=0).stop 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 -- 172.21.15.32:0/1387365353 shutdown_connections 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56255addada0 0x56255b27bff0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56255ae47070 0x56255b208450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56255ae26390 0x56255b203630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 --2- 172.21.15.32:0/1387365353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56255ae1e1b0 0x56255b2030f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 -- 172.21.15.32:0/1387365353 >> 172.21.15.32:0/1387365353 conn(0x56255ae3b460 msgr2=0x56255b1fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f6853400640 0 client.16536 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 -- 172.21.15.32:0/1387365353 shutdown_connections 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.837+0000 7f687ae32740 1 -- 172.21.15.32:0/1387365353 wait complete. 2024-08-31T07:17:28.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f687ae32740 20 client.16536 trim_cache size 0 max 16384 2024-08-31T07:17:28.842 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:17:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 -- 172.21.15.32:0/387864306 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68540b4c30 msgr2=0x7f68540b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68540b4c30 0x7f68540b7140 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f686c108e00 tx=0x7f6858005f90 comp rx=0 tx=0).stop 2024-08-31T07:17:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 -- 172.21.15.32:0/387864306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f686c103900 msgr2=0x7f686c19a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f686c103900 0x7f686c19a1d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f686000d950 tx=0x7f686000de10 comp rx=0 tx=0).stop 2024-08-31T07:17:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 -- 172.21.15.32:0/387864306 shutdown_connections 2024-08-31T07:17:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68540b4c30 0x7f68540b7140 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c109b90 0x7f686c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f686c109240 0x7f686c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 --2- 172.21.15.32:0/387864306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f686c103900 0x7f686c19a1d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 -- 172.21.15.32:0/387864306 >> 172.21.15.32:0/387864306 conn(0x7f686c07b5f0 msgr2=0x7f686c107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.838+0000 7f6874200640 1 -- 172.21.15.32:0/387864306 shutdown_connections 2024-08-31T07:17:28.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:28.839+0000 7f6874200640 1 -- 172.21.15.32:0/387864306 wait complete. 2024-08-31T07:17:29.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135487 2.20 MB/sec execute 78 sec latency 2671.042 ms 2024-08-31T07:17:29.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:17:30.116 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:17:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:30 smithi032 ceph-mon[17252]: pgmap v510: 97 pgs: 97 active+clean; 82 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 455 KiB/s wr, 80 op/s 2024-08-31T07:17:30.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135611 2.18 MB/sec execute 79 sec latency 3088.689 ms 2024-08-31T07:17:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:30 smithi084 ceph-mon[22387]: pgmap v510: 97 pgs: 97 active+clean; 82 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 455 KiB/s wr, 80 op/s 2024-08-31T07:17:30.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:30 smithi073 ceph-mon[21995]: pgmap v510: 97 pgs: 97 active+clean; 82 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 455 KiB/s wr, 80 op/s 2024-08-31T07:17:30.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.979+0000 7f2ab7e00640 1 -- 172.21.15.32:0/4091122282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab8103730 msgr2=0x7f2ab8103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.979+0000 7f2ab7e00640 1 --2- 172.21.15.32:0/4091122282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab8103730 0x7f2ab8103b40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009900 tx=0x7f2aa802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.979+0000 7f2ab7e00640 1 -- 172.21.15.32:0/4091122282 shutdown_connections 2024-08-31T07:17:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.979+0000 7f2ab7e00640 1 --2- 172.21.15.32:0/4091122282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab81099c0 0x7f2ab81101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.979+0000 7f2ab7e00640 1 --2- 172.21.15.32:0/4091122282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ab8109070 0x7f2ab8109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.979+0000 7f2ab7e00640 1 --2- 172.21.15.32:0/4091122282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab8103730 0x7f2ab8103b40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.979+0000 7f2ab7e00640 1 -- 172.21.15.32:0/4091122282 >> 172.21.15.32:0/4091122282 conn(0x7f2ab80ff180 msgr2=0x7f2ab81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.979+0000 7f2ab7e00640 1 -- 172.21.15.32:0/4091122282 shutdown_connections 2024-08-31T07:17:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.979+0000 7f2ab7e00640 1 -- 172.21.15.32:0/4091122282 wait complete. 2024-08-31T07:17:30.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.980+0000 7f2ab7e00640 1 Processor -- start 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.980+0000 7f2ab7e00640 1 -- start start 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2ab7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab8103730 0x7f2ab819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2ab7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab8109070 0x7f2ab819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2ab7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ab81099c0 0x7f2ab81a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2ab7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8113690 con 0x7f2ab8103730 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2ab7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8113510 con 0x7f2ab81099c0 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2ab7e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8113810 con 0x7f2ab8109070 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2ab6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab8103730 0x7f2ab819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2ab6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab8103730 0x7f2ab819c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52732/0 (socket says 172.21.15.32:52732) 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2ab6a00640 1 -- 172.21.15.32:0/3786171776 learned_addr learned my addr 172.21.15.32:0/3786171776 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2aafe00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab8109070 0x7f2ab819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.981+0000 7f2ab7400640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ab81099c0 0x7f2ab81a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.982+0000 7f2ab6a00640 1 -- 172.21.15.32:0/3786171776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab8109070 msgr2=0x7f2ab819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.982+0000 7f2ab6a00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab8109070 0x7f2ab819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.982+0000 7f2ab6a00640 1 -- 172.21.15.32:0/3786171776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ab81099c0 msgr2=0x7f2ab81a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.982+0000 7f2ab6a00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ab81099c0 0x7f2ab81a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.982+0000 7f2ab6a00640 1 -- 172.21.15.32:0/3786171776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab81a21f0 con 0x7f2ab8103730 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.982+0000 7f2aafe00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab8109070 0x7f2ab819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.982+0000 7f2ab6a00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab8103730 0x7f2ab819c280 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8005940 tx=0x7f2aa80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.983+0000 7f2aaf400640 1 -- 172.21.15.32:0/3786171776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa803f030 con 0x7f2ab8103730 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.983+0000 7f2aaf400640 1 -- 172.21.15.32:0/3786171776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa80044c0 con 0x7f2ab8103730 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.983+0000 7f2aaf400640 1 -- 172.21.15.32:0/3786171776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8035070 con 0x7f2ab8103730 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.983+0000 7f2ab7e00640 1 -- 172.21.15.32:0/3786171776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab81a2480 con 0x7f2ab8103730 2024-08-31T07:17:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.983+0000 7f2ab7e00640 1 -- 172.21.15.32:0/3786171776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab81a8c40 con 0x7f2ab8103730 2024-08-31T07:17:30.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.986+0000 7f2ac4668740 1 Processor -- start 2024-08-31T07:17:30.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.986+0000 7f2ac4668740 1 -- start start 2024-08-31T07:17:30.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.986+0000 7f2aaf400640 1 -- 172.21.15.32:0/3786171776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2aa8042030 con 0x7f2ab8103730 2024-08-31T07:17:30.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa1011b0 0x5562fa4d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 0x5562fa4d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2ac4668740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5562fa4467b0 con 0x5562fa109390 2024-08-31T07:17:30.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2ac4668740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5562fa435d40 con 0x5562fa12a070 2024-08-31T07:17:30.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2ac4668740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5562fa439cf0 con 0x5562fa1011b0 2024-08-31T07:17:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2ab7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2ab7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45138/0 (socket says 172.21.15.32:45138) 2024-08-31T07:17:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2aafe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 0x5562fa4d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2ab7400640 1 -- 172.21.15.32:0/1265781738 learned_addr learned my addr 172.21.15.32:0/1265781738 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:30.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.987+0000 7f2ab6a00640 1 --2- 172.21.15.32:0/1265781738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa1011b0 0x5562fa4d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.988+0000 7f2ab7400640 1 -- 172.21.15.32:0/1265781738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa1011b0 msgr2=0x5562fa4d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.988+0000 7f2ab7400640 1 --2- 172.21.15.32:0/1265781738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa1011b0 0x5562fa4d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.988+0000 7f2ab7400640 1 -- 172.21.15.32:0/1265781738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 msgr2=0x5562fa4d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.988+0000 7f2ab7400640 1 --2- 172.21.15.32:0/1265781738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 0x5562fa4d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.988+0000 7f2ab7400640 1 -- 172.21.15.32:0/1265781738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562fa195650 con 0x5562fa12a070 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.988+0000 7f2ab7400640 1 --2- 172.21.15.32:0/1265781738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4cfb60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009fd0 tx=0x7f2ab00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1fd0663ad6c2b25 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.988+0000 7f2aa7e00640 1 -- 172.21.15.32:0/1265781738 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0004d20 con 0x5562fa12a070 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.988+0000 7f2aa7e00640 1 -- 172.21.15.32:0/1265781738 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ab0004050 con 0x5562fa12a070 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.988+0000 7f2aa7e00640 1 -- 172.21.15.32:0/1265781738 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0005280 con 0x5562fa12a070 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.988+0000 7f2aaf400640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a9c0b4ca0 0x7f2a9c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.989+0000 7f2aaf400640 1 -- 172.21.15.32:0/3786171776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2aa81057e0 con 0x7f2ab8103730 2024-08-31T07:17:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.989+0000 7f2ab6a00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a9c0b4ca0 0x7f2a9c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.989+0000 7f2ac4668740 1 -- 172.21.15.32:0/1265781738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 msgr2=0x5562fa4cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:30.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.989+0000 7f2ac4668740 1 --2- 172.21.15.32:0/1265781738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4cfb60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009fd0 tx=0x7f2ab00118d0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.989+0000 7f2ab6a00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a9c0b4ca0 0x7f2a9c0b71b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8103060 tx=0x7f2aa810b550 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.989+0000 7f2ac4668740 1 -- 172.21.15.32:0/1265781738 shutdown_connections 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.989+0000 7f2ac4668740 1 --2- 172.21.15.32:0/1265781738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 0x5562fa4d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.989+0000 7f2ac4668740 1 --2- 172.21.15.32:0/1265781738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa1011b0 0x5562fa4d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.989+0000 7f2ac4668740 1 --2- 172.21.15.32:0/1265781738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4cfb60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.989+0000 7f2ac4668740 1 -- 172.21.15.32:0/1265781738 >> 172.21.15.32:0/1265781738 conn(0x5562fa11e460 msgr2=0x5562fa4cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.990+0000 7f2ac4668740 1 -- 172.21.15.32:0/1265781738 shutdown_connections 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.990+0000 7f2ac4668740 1 -- 172.21.15.32:0/1265781738 wait complete. 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.990+0000 7f2ac4668740 1 -- start start 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2ac4668740 1 Processor -- start 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa1011b0 0x5562fa4db6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 0x5562fa4dbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4e7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2ac4668740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5562fa298700 con 0x5562fa109390 2024-08-31T07:17:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2ac4668740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5562fa3bccf0 con 0x5562fa12a070 2024-08-31T07:17:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2ac4668740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5562fa4de340 con 0x5562fa1011b0 2024-08-31T07:17:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2ac4668740 10 client.0 _ll_register_callbacks cb 0x5562fa3e2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2aafe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 0x5562fa4dbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2aafe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 0x5562fa4dbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52758/0 (socket says 172.21.15.32:52758) 2024-08-31T07:17:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.991+0000 7f2aafe00640 1 -- 172.21.15.32:0/985412589 learned_addr learned my addr 172.21.15.32:0/985412589 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2ab7400640 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa1011b0 0x5562fa4db6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2ab7400640 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2aafe00640 1 -- 172.21.15.32:0/985412589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa1011b0 msgr2=0x5562fa4db6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2aafe00640 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa1011b0 0x5562fa4db6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2aafe00640 1 -- 172.21.15.32:0/985412589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 msgr2=0x5562fa4e7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2aafe00640 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4e7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2aafe00640 1 -- 172.21.15.32:0/985412589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562fa3fd8b0 con 0x5562fa109390 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2ab7400640 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2aafe00640 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 0x5562fa4dbbe0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0012d20 tx=0x7f2aa000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ecc1ed7daefc7b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2aa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.992+0000 7f2aa4c00640 1 -- 172.21.15.32:0/985412589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0012110 con 0x5562fa109390 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.993+0000 7f2aa4c00640 1 -- 172.21.15.32:0/985412589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2aa0004050 con 0x5562fa109390 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.993+0000 7f2ac4668740 1 -- 172.21.15.32:0/985412589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ab0004050 con 0x5562fa109390 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.993+0000 7f2aa4c00640 1 -- 172.21.15.32:0/985412589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2aa00122b0 con 0x5562fa109390 2024-08-31T07:17:30.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.993+0000 7f2ac4668740 10 client.16548 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.993+0000 7f2ac4668740 1 -- 172.21.15.32:0/985412589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5562fa4ea360 con 0x5562fa109390 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.993+0000 7f2aa4c00640 1 -- 172.21.15.32:0/985412589 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ab0004050 con 0x5562fa109390 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.993+0000 7f2aa4c00640 1 -- 172.21.15.32:0/985412589 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2aa0026450 con 0x5562fa109390 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.993+0000 7f2ac4668740 10 client.16548 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.993+0000 7f2ac4668740 10 client.16548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.993+0000 7f2ac4668740 20 client.16548 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.994+0000 7f2ac4668740 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5562fa0e89f0 0x5562fa4ed880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.994+0000 7f2ac4668740 4 client.16548 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.994+0000 7f2ac4668740 1 -- 172.21.15.32:0/985412589 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562fa448700 con 0x5562fa0e89f0 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.994+0000 7f2ab7400640 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5562fa0e89f0 0x5562fa4ed880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.994+0000 7f2ab7400640 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5562fa0e89f0 0x5562fa4ed880 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c78f96cc29af5828 server_cookie=d13c3d994ef2c1fc in_seq=0 out_seq=0 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.994+0000 7f2aa4c00640 10 client.16548 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.995+0000 7f2aa4c00640 1 -- 172.21.15.32:0/985412589 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5562fa448700 con 0x5562fa0e89f0 2024-08-31T07:17:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.995+0000 7f2aa4c00640 10 client.16548 handle_command_reply: tid=0 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.995+0000 7f2ac4668740 1 client.16548 shutdown 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.995+0000 7f2ac4668740 1 -- 172.21.15.32:0/985412589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 msgr2=0x5562fa4dbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.995+0000 7f2ac4668740 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 0x5562fa4dbbe0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0012d20 tx=0x7f2aa000f790 comp rx=0 tx=0).stop 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.995+0000 7f2ac4668740 1 -- 172.21.15.32:0/985412589 shutdown_connections 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.995+0000 7f2ac4668740 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5562fa0e89f0 0x5562fa4ed880 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.996+0000 7f2ac4668740 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.996+0000 7f2ac4668740 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa109390 0x5562fa4dbbe0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.996+0000 7f2ac4668740 1 --2- 172.21.15.32:0/985412589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa1011b0 0x5562fa4db6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.996+0000 7f2ac4668740 1 -- 172.21.15.32:0/985412589 >> 172.21.15.32:0/985412589 conn(0x5562fa11e460 msgr2=0x5562fa4d2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.996+0000 7f2aa4c00640 0 client.16548 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.996+0000 7f2ac4668740 1 -- 172.21.15.32:0/985412589 shutdown_connections 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.996+0000 7f2ac4668740 1 -- 172.21.15.32:0/985412589 wait complete. 2024-08-31T07:17:30.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:30.996+0000 7f2ac4668740 20 client.16548 trim_cache size 0 max 16384 2024-08-31T07:17:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.010+0000 7f2ac4668740 1 Processor -- start 2024-08-31T07:17:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.010+0000 7f2ac4668740 1 -- start start 2024-08-31T07:17:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa1011b0 0x5562fa4cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 0x5562fa4d5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa12a070 0x5562fa4e7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2ac4668740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81b12b0 con 0x5562fa12a070 2024-08-31T07:17:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2ac4668740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81b2df0 con 0x5562fa1011b0 2024-08-31T07:17:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2ac4668740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81b1980 con 0x5562fa109390 2024-08-31T07:17:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2ab7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa1011b0 0x5562fa4cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2ab7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa12a070 0x5562fa4e7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2aafe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 0x5562fa4d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2ab7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa12a070 0x5562fa4e7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52774/0 (socket says 172.21.15.32:52774) 2024-08-31T07:17:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.011+0000 7f2ab7400640 1 -- 172.21.15.32:0/2112088117 learned_addr learned my addr 172.21.15.32:0/2112088117 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:31.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.012+0000 7f2aafe00640 1 -- 172.21.15.32:0/2112088117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa1011b0 msgr2=0x5562fa4cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:31.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.012+0000 7f2aafe00640 1 --2- 172.21.15.32:0/2112088117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa1011b0 0x5562fa4cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.012+0000 7f2aafe00640 1 -- 172.21.15.32:0/2112088117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa12a070 msgr2=0x5562fa4e7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:31.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.012+0000 7f2aafe00640 1 --2- 172.21.15.32:0/2112088117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa12a070 0x5562fa4e7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.012+0000 7f2aafe00640 1 -- 172.21.15.32:0/2112088117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562fa4d99a0 con 0x5562fa109390 2024-08-31T07:17:31.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.012+0000 7f2ab7400640 1 --2- 172.21.15.32:0/2112088117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa12a070 0x5562fa4e7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:31.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.012+0000 7f2aafe00640 1 --2- 172.21.15.32:0/2112088117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 0x5562fa4d5c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00268a0 tx=0x7f2aa00057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4698900cf9d199c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:31.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.012+0000 7f2aa7400640 1 -- 172.21.15.32:0/2112088117 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0000e60 con 0x5562fa109390 2024-08-31T07:17:31.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.012+0000 7f2aa7400640 1 -- 172.21.15.32:0/2112088117 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa0005580 con 0x5562fa109390 2024-08-31T07:17:31.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.012+0000 7f2aa7400640 1 -- 172.21.15.32:0/2112088117 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0026e60 con 0x5562fa109390 2024-08-31T07:17:31.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.013+0000 7f2ac4668740 1 -- 172.21.15.32:0/2112088117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 msgr2=0x5562fa4d5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:31.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.013+0000 7f2ac4668740 1 --2- 172.21.15.32:0/2112088117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 0x5562fa4d5c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00268a0 tx=0x7f2aa00057d0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.013+0000 7f2ac4668740 1 -- 172.21.15.32:0/2112088117 shutdown_connections 2024-08-31T07:17:31.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.013+0000 7f2ac4668740 1 --2- 172.21.15.32:0/2112088117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa12a070 0x5562fa4e7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.013+0000 7f2ac4668740 1 --2- 172.21.15.32:0/2112088117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 0x5562fa4d5c80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.013+0000 7f2ac4668740 1 --2- 172.21.15.32:0/2112088117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa1011b0 0x5562fa4cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.013+0000 7f2ac4668740 1 -- 172.21.15.32:0/2112088117 >> 172.21.15.32:0/2112088117 conn(0x5562fa11e460 msgr2=0x5562fa4cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.013+0000 7f2ac4668740 1 -- 172.21.15.32:0/2112088117 shutdown_connections 2024-08-31T07:17:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.013+0000 7f2ac4668740 1 -- 172.21.15.32:0/2112088117 wait complete. 2024-08-31T07:17:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.014+0000 7f2ac4668740 1 -- start start 2024-08-31T07:17:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.014+0000 7f2ac4668740 1 Processor -- start 2024-08-31T07:17:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.014+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa1011b0 0x5562fa4e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.014+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 0x5562fa4e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.014+0000 7f2ac4668740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.014+0000 7f2ac4668740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81b2720 con 0x5562fa1011b0 2024-08-31T07:17:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.014+0000 7f2ac4668740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81b2050 con 0x5562fa12a070 2024-08-31T07:17:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.014+0000 7f2ac4668740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81af290 con 0x5562fa109390 2024-08-31T07:17:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.014+0000 7f2ac4668740 10 client.0 _ll_register_callbacks cb 0x7f2aa0012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2ab7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa1011b0 0x5562fa4e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2aafe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 0x5562fa4e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2ab7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa1011b0 0x5562fa4e35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52790/0 (socket says 172.21.15.32:52790) 2024-08-31T07:17:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2ab7400640 1 -- 172.21.15.32:0/1800509330 learned_addr learned my addr 172.21.15.32:0/1800509330 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2ab7400640 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2aafe00640 1 -- 172.21.15.32:0/1800509330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 msgr2=0x5562fa4de580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2aafe00640 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4de580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2aafe00640 1 -- 172.21.15.32:0/1800509330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa1011b0 msgr2=0x5562fa4e35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2aafe00640 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa1011b0 0x5562fa4e35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2aafe00640 1 -- 172.21.15.32:0/1800509330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562fa4e4280 con 0x5562fa109390 2024-08-31T07:17:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.015+0000 7f2ab7400640 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:17:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.016+0000 7f2aafe00640 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 0x5562fa4e3b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5562fa4dc280 tx=0x7f2aa0000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ac790725cc4827 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.016+0000 7f2aa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.016+0000 7f2aa4c00640 1 -- 172.21.15.32:0/1800509330 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00050d0 con 0x5562fa109390 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.016+0000 7f2ac4668740 1 -- 172.21.15.32:0/1800509330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aa0005580 con 0x5562fa109390 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.016+0000 7f2aa4c00640 1 -- 172.21.15.32:0/1800509330 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2aa0005270 con 0x5562fa109390 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.016+0000 7f2aa4c00640 1 -- 172.21.15.32:0/1800509330 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2aa0026a10 con 0x5562fa109390 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.016+0000 7f2ac4668740 10 client.26581 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.016+0000 7f2ac4668740 1 -- 172.21.15.32:0/1800509330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5562fa558720 con 0x5562fa109390 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.017+0000 7f2aa4c00640 1 -- 172.21.15.32:0/1800509330 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa0005580 con 0x5562fa109390 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.017+0000 7f2aa4c00640 1 -- 172.21.15.32:0/1800509330 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2aa0042450 con 0x5562fa109390 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.017+0000 7f2ac4668740 10 client.26581 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.017+0000 7f2ac4668740 10 client.26581 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.017+0000 7f2ac4668740 20 client.26581 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.017+0000 7f2ac4668740 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5562fa0bdda0 0x5562fa55ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:31.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.017+0000 7f2ac4668740 4 client.26581 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.017+0000 7f2ac4668740 1 -- 172.21.15.32:0/1800509330 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562fa4441a0 con 0x5562fa0bdda0 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.017+0000 7f2ab7400640 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5562fa0bdda0 0x5562fa55ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2ab7400640 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5562fa0bdda0 0x5562fa55ad50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=144bd60eac75a229 server_cookie=afdda638c223f910 in_seq=0 out_seq=0 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2aa4c00640 10 client.26581 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2aa4c00640 1 -- 172.21.15.32:0/1800509330 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562fa4441a0 con 0x5562fa0bdda0 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2aa4c00640 10 client.26581 handle_command_reply: tid=0 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2ac4668740 1 client.26581 shutdown 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2ac4668740 1 -- 172.21.15.32:0/1800509330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 msgr2=0x5562fa4e3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2ac4668740 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 0x5562fa4e3b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5562fa4dc280 tx=0x7f2aa0000ee0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2ac4668740 1 -- 172.21.15.32:0/1800509330 shutdown_connections 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2ac4668740 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5562fa0bdda0 0x5562fa55ad50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2ac4668740 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562fa12a070 0x5562fa4de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2ac4668740 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562fa109390 0x5562fa4e3b10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2ac4668740 1 --2- 172.21.15.32:0/1800509330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562fa1011b0 0x5562fa4e35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2aa4c00640 0 client.26581 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:17:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.018+0000 7f2ac4668740 1 -- 172.21.15.32:0/1800509330 >> 172.21.15.32:0/1800509330 conn(0x5562fa11e460 msgr2=0x5562fa4d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.019+0000 7f2ac4668740 1 -- 172.21.15.32:0/1800509330 shutdown_connections 2024-08-31T07:17:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.019+0000 7f2ac4668740 1 -- 172.21.15.32:0/1800509330 wait complete. 2024-08-31T07:17:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.019+0000 7f2ac4668740 20 client.26581 trim_cache size 0 max 16384 2024-08-31T07:17:31.033 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:17:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.019+0000 7f2ab7e00640 1 -- 172.21.15.32:0/3786171776 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a9c0b4ca0 msgr2=0x7f2a9c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.019+0000 7f2ab7e00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a9c0b4ca0 0x7f2a9c0b71b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8103060 tx=0x7f2aa810b550 comp rx=0 tx=0).stop 2024-08-31T07:17:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.019+0000 7f2ab7e00640 1 -- 172.21.15.32:0/3786171776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab8103730 msgr2=0x7f2ab819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.019+0000 7f2ab7e00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab8103730 0x7f2ab819c280 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8005940 tx=0x7f2aa80040c0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.020+0000 7f2ab7e00640 1 -- 172.21.15.32:0/3786171776 shutdown_connections 2024-08-31T07:17:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.020+0000 7f2ab7e00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a9c0b4ca0 0x7f2a9c0b71b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.020+0000 7f2ab7e00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ab81099c0 0x7f2ab81a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.020+0000 7f2ab7e00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ab8109070 0x7f2ab819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.020+0000 7f2ab7e00640 1 --2- 172.21.15.32:0/3786171776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab8103730 0x7f2ab819c280 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.020+0000 7f2ab7e00640 1 -- 172.21.15.32:0/3786171776 >> 172.21.15.32:0/3786171776 conn(0x7f2ab80ff180 msgr2=0x7f2ab810e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:31.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.020+0000 7f2ab7e00640 1 -- 172.21.15.32:0/3786171776 shutdown_connections 2024-08-31T07:17:31.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:31.020+0000 7f2ab7e00640 1 -- 172.21.15.32:0/3786171776 wait complete. 2024-08-31T07:17:31.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135611 2.15 MB/sec execute 80 sec latency 1485.018 ms 2024-08-31T07:17:32.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:17:32.390 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:17:32.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135611 2.13 MB/sec execute 81 sec latency 2485.127 ms 2024-08-31T07:17:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:32 smithi084 ceph-mon[22387]: pgmap v511: 97 pgs: 97 active+clean; 83 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 512 KiB/s wr, 91 op/s 2024-08-31T07:17:32.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:32 smithi032 ceph-mon[17252]: pgmap v511: 97 pgs: 97 active+clean; 83 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 512 KiB/s wr, 91 op/s 2024-08-31T07:17:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:32 smithi073 ceph-mon[21995]: pgmap v511: 97 pgs: 97 active+clean; 83 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 512 KiB/s wr, 91 op/s 2024-08-31T07:17:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.111+0000 7f2d0fe00640 1 -- 172.21.15.32:0/2531290608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1010b290 msgr2=0x7f2d1010b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.111+0000 7f2d0fe00640 1 --2- 172.21.15.32:0/2531290608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1010b290 0x7f2d1010b6a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009900 tx=0x7f2cfc02a410 comp rx=0 tx=0).stop 2024-08-31T07:17:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.111+0000 7f2d0fe00640 1 -- 172.21.15.32:0/2531290608 shutdown_connections 2024-08-31T07:17:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.111+0000 7f2d0fe00640 1 --2- 172.21.15.32:0/2531290608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d1010bbe0 0x7f2d10112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.111+0000 7f2d0fe00640 1 --2- 172.21.15.32:0/2531290608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1010b290 0x7f2d1010b6a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.111+0000 7f2d0fe00640 1 --2- 172.21.15.32:0/2531290608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d10105950 0x7f2d10105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.111+0000 7f2d0fe00640 1 -- 172.21.15.32:0/2531290608 >> 172.21.15.32:0/2531290608 conn(0x7f2d10101420 msgr2=0x7f2d10103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.111+0000 7f2d0fe00640 1 -- 172.21.15.32:0/2531290608 shutdown_connections 2024-08-31T07:17:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.111+0000 7f2d0fe00640 1 -- 172.21.15.32:0/2531290608 wait complete. 2024-08-31T07:17:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.112+0000 7f2d0fe00640 1 Processor -- start 2024-08-31T07:17:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0fe00640 1 -- start start 2024-08-31T07:17:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d10105950 0x7f2d1019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1010b290 0x7f2d1019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d1010bbe0 0x7f2d101a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101158f0 con 0x7f2d1010b290 2024-08-31T07:17:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2d10115770 con 0x7f2d1010bbe0 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d10115a70 con 0x7f2d10105950 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1010b290 0x7f2d1019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d10105950 0x7f2d1019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1010b290 0x7f2d1019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52806/0 (socket says 172.21.15.32:52806) 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0e000640 1 -- 172.21.15.32:0/4066195046 learned_addr learned my addr 172.21.15.32:0/4066195046 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.113+0000 7f2d0f400640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d1010bbe0 0x7f2d101a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.114+0000 7f2d0e000640 1 -- 172.21.15.32:0/4066195046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d10105950 msgr2=0x7f2d1019c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.114+0000 7f2d0e000640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d10105950 0x7f2d1019c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.114+0000 7f2d0e000640 1 -- 172.21.15.32:0/4066195046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d1010bbe0 msgr2=0x7f2d101a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.114+0000 7f2d0e000640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d1010bbe0 0x7f2d101a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.114+0000 7f2d0e000640 1 -- 172.21.15.32:0/4066195046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d101a2250 con 0x7f2d1010b290 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.114+0000 7f2d0ea00640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d10105950 0x7f2d1019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.114+0000 7f2d0e000640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1010b290 0x7f2d1019c880 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc005ab0 tx=0x7f2cfc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.115+0000 7f2d07400640 1 -- 172.21.15.32:0/4066195046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc03f070 con 0x7f2d1010b290 2024-08-31T07:17:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.115+0000 7f2d07400640 1 -- 172.21.15.32:0/4066195046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc031b50 con 0x7f2d1010b290 2024-08-31T07:17:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.115+0000 7f2d0fe00640 1 -- 172.21.15.32:0/4066195046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d101a24e0 con 0x7f2d1010b290 2024-08-31T07:17:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.115+0000 7f2d07400640 1 -- 172.21.15.32:0/4066195046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc035030 con 0x7f2d1010b290 2024-08-31T07:17:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.115+0000 7f2d0fe00640 1 -- 172.21.15.32:0/4066195046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d101a8ca0 con 0x7f2d1010b290 2024-08-31T07:17:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.118+0000 7f2d1c7ef740 1 Processor -- start 2024-08-31T07:17:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.118+0000 7f2d1c7ef740 1 -- start start 2024-08-31T07:17:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.118+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.118+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.118+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.118+0000 7f2d07400640 1 -- 172.21.15.32:0/4066195046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2cfc042070 con 0x7f2d1010b290 2024-08-31T07:17:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.118+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5589609517b0 con 0x55896060c1b0 2024-08-31T07:17:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.118+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558960940d40 con 0x558960614390 2024-08-31T07:17:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.118+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558960944cf0 con 0x558960635070 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52810/0 (socket says 172.21.15.32:52810) 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0ea00640 1 -- 172.21.15.32:0/3932998414 learned_addr learned my addr 172.21.15.32:0/3932998414 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0f400640 1 --2- 172.21.15.32:0/3932998414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0e000640 1 --2- 172.21.15.32:0/3932998414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0ea00640 1 -- 172.21.15.32:0/3932998414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 msgr2=0x5589609daa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0ea00640 1 --2- 172.21.15.32:0/3932998414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609daa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0ea00640 1 -- 172.21.15.32:0/3932998414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 msgr2=0x5589609db4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0ea00640 1 --2- 172.21.15.32:0/3932998414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609db4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0ea00640 1 -- 172.21.15.32:0/3932998414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589606a0650 con 0x55896060c1b0 2024-08-31T07:17:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.119+0000 7f2d0f400640 1 --2- 172.21.15.32:0/3932998414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609daa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.120+0000 7f2d0ea00640 1 --2- 172.21.15.32:0/3932998414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dafb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2d000115c0 tx=0x7f2d00011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac782c875988f3c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.120+0000 7f2cfbe00640 1 -- 172.21.15.32:0/3932998414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000f640 con 0x55896060c1b0 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.120+0000 7f2cfbe00640 1 -- 172.21.15.32:0/3932998414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0000fc90 con 0x55896060c1b0 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.120+0000 7f2cfbe00640 1 -- 172.21.15.32:0/3932998414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00019ce0 con 0x55896060c1b0 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.120+0000 7f2d07400640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2cf40b4ca0 0x7f2cf40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.120+0000 7f2d07400640 1 -- 172.21.15.32:0/4066195046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2cfc105710 con 0x7f2d1010b290 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.120+0000 7f2d0f400640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2cf40b4ca0 0x7f2cf40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.121+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3932998414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 msgr2=0x5589609dafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.121+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/3932998414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dafb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2d000115c0 tx=0x7f2d00011a80 comp rx=0 tx=0).stop 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.121+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3932998414 shutdown_connections 2024-08-31T07:17:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.121+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/3932998414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.121+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/3932998414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dafb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.121+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/3932998414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609daa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.121+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3932998414 >> 172.21.15.32:0/3932998414 conn(0x558960629460 msgr2=0x5589609d8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.121+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3932998414 shutdown_connections 2024-08-31T07:17:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.121+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3932998414 wait complete. 2024-08-31T07:17:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.121+0000 7f2d0f400640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2cf40b4ca0 0x7f2cf40b71b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2d08005fd0 tx=0x7f2d08009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.122+0000 7f2d1c7ef740 1 -- start start 2024-08-31T07:17:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.122+0000 7f2d1c7ef740 1 Processor -- start 2024-08-31T07:17:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960614390 0x5589609e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960635070 0x5589609f2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5589607a3700 con 0x55896060c1b0 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5589608c7cf0 con 0x558960635070 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5589609e9250 con 0x558960614390 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d1c7ef740 10 client.0 _ll_register_callbacks cb 0x5589608ed470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d0e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960614390 0x5589609e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d0e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960614390 0x5589609e6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47754/0 (socket says 172.21.15.32:47754) 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d0e000640 1 -- 172.21.15.32:0/3202684407 learned_addr learned my addr 172.21.15.32:0/3202684407 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.123+0000 7f2d0f400640 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960635070 0x5589609f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2d0e000640 1 -- 172.21.15.32:0/3202684407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960635070 msgr2=0x5589609f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2d0e000640 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960635070 0x5589609f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2d0e000640 1 -- 172.21.15.32:0/3202684407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 msgr2=0x5589609e66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2d0e000640 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609e66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2d0e000640 1 -- 172.21.15.32:0/3202684407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589609088b0 con 0x558960614390 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2d0e000640 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960614390 0x5589609e6c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc031440 tx=0x7f2cfc009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4324ee97cec5c9c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2cf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2cf8c00640 1 -- 172.21.15.32:0/3202684407 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc049030 con 0x558960614390 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2cf8c00640 1 -- 172.21.15.32:0/3202684407 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2cfc0b30e0 con 0x558960614390 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3202684407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0000fc90 con 0x558960614390 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.124+0000 7f2cf8c00640 1 -- 172.21.15.32:0/3202684407 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2cfc0cb770 con 0x558960614390 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.125+0000 7f2d1c7ef740 10 client.26593 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.125+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3202684407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5589609f5310 con 0x558960614390 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.125+0000 7f2cf8c00640 1 -- 172.21.15.32:0/3202684407 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0000fc90 con 0x558960614390 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.125+0000 7f2cf8c00640 1 -- 172.21.15.32:0/3202684407 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2cfc04c060 con 0x558960614390 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.125+0000 7f2d1c7ef740 10 client.26593 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.125+0000 7f2d1c7ef740 10 client.26593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.125+0000 7f2d1c7ef740 20 client.26593 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.125+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589605f39f0 0x5589609f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.125+0000 7f2d1c7ef740 4 client.26593 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:17:33.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.125+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3202684407 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558960953700 con 0x5589605f39f0 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.126+0000 7f2d0ea00640 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589605f39f0 0x5589609f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.126+0000 7f2d0ea00640 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589605f39f0 0x5589609f87e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6da1e20987ab35b server_cookie=93864cb3de14aae6 in_seq=0 out_seq=0 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.126+0000 7f2cf8c00640 10 client.26593 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.127+0000 7f2cf8c00640 1 -- 172.21.15.32:0/3202684407 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558960953700 con 0x5589605f39f0 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.127+0000 7f2cf8c00640 10 client.26593 handle_command_reply: tid=0 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.127+0000 7f2d1c7ef740 1 client.26593 shutdown 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3202684407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960614390 msgr2=0x5589609e6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960614390 0x5589609e6c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc031440 tx=0x7f2cfc009560 comp rx=0 tx=0).stop 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3202684407 shutdown_connections 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589605f39f0 0x5589609f87e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960635070 0x5589609f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960614390 0x5589609e6c90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/3202684407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2cf8c00640 0 client.26593 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3202684407 >> 172.21.15.32:0/3202684407 conn(0x558960629460 msgr2=0x5589609ddc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3202684407 shutdown_connections 2024-08-31T07:17:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/3202684407 wait complete. 2024-08-31T07:17:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.128+0000 7f2d1c7ef740 20 client.26593 trim_cache size 0 max 16384 2024-08-31T07:17:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.142+0000 7f2d1c7ef740 1 Processor -- start 2024-08-31T07:17:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.142+0000 7f2d1c7ef740 1 -- start start 2024-08-31T07:17:33.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.142+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.142+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609d8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101b13b0 con 0x55896060c1b0 2024-08-31T07:17:33.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101b2ef0 con 0x558960614390 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101b1a80 con 0x558960635070 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52824/0 (socket says 172.21.15.32:52824) 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0ea00640 1 -- 172.21.15.32:0/2213454840 learned_addr learned my addr 172.21.15.32:0/2213454840 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0f400640 1 --2- 172.21.15.32:0/2213454840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609d8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0e000640 1 -- 172.21.15.32:0/2213454840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 msgr2=0x5589609d8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0e000640 1 --2- 172.21.15.32:0/2213454840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609d8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0e000640 1 -- 172.21.15.32:0/2213454840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 msgr2=0x5589609dfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0e000640 1 --2- 172.21.15.32:0/2213454840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0e000640 1 -- 172.21.15.32:0/2213454840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589609e9130 con 0x558960614390 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.143+0000 7f2d0ea00640 1 --2- 172.21.15.32:0/2213454840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.144+0000 7f2d0e000640 1 --2- 172.21.15.32:0/2213454840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609e00c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0cbdb0 tx=0x7f2cfc02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5855cb99effe09b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.144+0000 7f2cfb400640 1 -- 172.21.15.32:0/2213454840 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc049020 con 0x558960614390 2024-08-31T07:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.144+0000 7f2cfb400640 1 -- 172.21.15.32:0/2213454840 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc1032e0 con 0x558960614390 2024-08-31T07:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.144+0000 7f2cfb400640 1 -- 172.21.15.32:0/2213454840 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0cb590 con 0x558960614390 2024-08-31T07:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.145+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2213454840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 msgr2=0x5589609e00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.145+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/2213454840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609e00c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0cbdb0 tx=0x7f2cfc02a410 comp rx=0 tx=0).stop 2024-08-31T07:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.145+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2213454840 shutdown_connections 2024-08-31T07:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.145+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/2213454840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609d8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.145+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/2213454840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609e00c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.145+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/2213454840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.145+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2213454840 >> 172.21.15.32:0/2213454840 conn(0x558960629460 msgr2=0x5589609e1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.145+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2213454840 shutdown_connections 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.145+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2213454840 wait complete. 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.145+0000 7f2d1c7ef740 1 -- start start 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d1c7ef740 1 Processor -- start 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609eec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d1c7ef740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609e9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101b2820 con 0x55896060c1b0 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101b2150 con 0x558960614390 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d1c7ef740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101af3b0 con 0x558960635070 2024-08-31T07:17:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d1c7ef740 10 client.0 _ll_register_callbacks cb 0x7f2cfc005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609ee6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52828/0 (socket says 172.21.15.32:52828) 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d0e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609eec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.146+0000 7f2d0ea00640 1 -- 172.21.15.32:0/2594752724 learned_addr learned my addr 172.21.15.32:0/2594752724 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2d0f400640 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2d0e000640 1 -- 172.21.15.32:0/2594752724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 msgr2=0x5589609e9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2d0e000640 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609e9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2d0e000640 1 -- 172.21.15.32:0/2594752724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 msgr2=0x5589609ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2d0e000640 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609ee6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2d0e000640 1 -- 172.21.15.32:0/2594752724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589609e9ad0 con 0x558960614390 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2d0e000640 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609eec20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc005e00 tx=0x7f2cfc103170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b69696a1d6ab887 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2cf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2cf8c00640 1 -- 172.21.15.32:0/2594752724 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc049030 con 0x558960614390 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2cf8c00640 1 -- 172.21.15.32:0/2594752724 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f2cfc0d4cb0 con 0x558960614390 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.147+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2594752724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cfc1032e0 con 0x558960614390 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2cf8c00640 1 -- 172.21.15.32:0/2594752724 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2cfc0b32c0 con 0x558960614390 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2d1c7ef740 10 client.26585 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2594752724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5589609e9ad0 con 0x558960614390 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2cf8c00640 1 -- 172.21.15.32:0/2594752724 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc1032e0 con 0x558960614390 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2cf8c00640 1 -- 172.21.15.32:0/2594752724 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2cfc0cc080 con 0x558960614390 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2d1c7ef740 10 client.26585 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2d1c7ef740 10 client.26585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2d1c7ef740 20 client.26585 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589605c8da0 0x558960a65d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2d1c7ef740 4 client.26585 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:17:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2594752724 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55896094f1a0 con 0x5589605c8da0 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.148+0000 7f2d0ea00640 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589605c8da0 0x558960a65d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.149+0000 7f2d0ea00640 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589605c8da0 0x558960a65d90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=544b9e118fc0029f server_cookie=66c247571eabd5cc in_seq=0 out_seq=0 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.149+0000 7f2cf8c00640 10 client.26585 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.149+0000 7f2cf8c00640 1 -- 172.21.15.32:0/2594752724 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55896094f1a0 con 0x5589605c8da0 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.149+0000 7f2cf8c00640 10 client.26585 handle_command_reply: tid=0 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.149+0000 7f2d1c7ef740 1 client.26585 shutdown 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.149+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2594752724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 msgr2=0x5589609eec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.149+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609eec20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc005e00 tx=0x7f2cfc103170 comp rx=0 tx=0).stop 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2594752724 shutdown_connections 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589605c8da0 0x558960a65d90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558960635070 0x5589609e9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2cf8c00640 0 client.26585 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558960614390 0x5589609eec20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d1c7ef740 1 --2- 172.21.15.32:0/2594752724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896060c1b0 0x5589609ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2594752724 >> 172.21.15.32:0/2594752724 conn(0x558960629460 msgr2=0x5589609dfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2594752724 shutdown_connections 2024-08-31T07:17:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d1c7ef740 1 -- 172.21.15.32:0/2594752724 wait complete. 2024-08-31T07:17:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d1c7ef740 20 client.26585 trim_cache size 0 max 16384 2024-08-31T07:17:33.154 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:17:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d0fe00640 1 -- 172.21.15.32:0/4066195046 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2cf40b4ca0 msgr2=0x7f2cf40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d0fe00640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2cf40b4ca0 0x7f2cf40b71b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2d08005fd0 tx=0x7f2d08009500 comp rx=0 tx=0).stop 2024-08-31T07:17:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d0fe00640 1 -- 172.21.15.32:0/4066195046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1010b290 msgr2=0x7f2d1019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.150+0000 7f2d0fe00640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1010b290 0x7f2d1019c880 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc005ab0 tx=0x7f2cfc003b90 comp rx=0 tx=0).stop 2024-08-31T07:17:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.151+0000 7f2d0fe00640 1 -- 172.21.15.32:0/4066195046 shutdown_connections 2024-08-31T07:17:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.151+0000 7f2d0fe00640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2cf40b4ca0 0x7f2cf40b71b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.151+0000 7f2d0fe00640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d1010bbe0 0x7f2d101a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.151+0000 7f2d0fe00640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1010b290 0x7f2d1019c880 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.151+0000 7f2d0fe00640 1 --2- 172.21.15.32:0/4066195046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d10105950 0x7f2d1019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.151+0000 7f2d0fe00640 1 -- 172.21.15.32:0/4066195046 >> 172.21.15.32:0/4066195046 conn(0x7f2d10101420 msgr2=0x7f2d10103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.151+0000 7f2d0fe00640 1 -- 172.21.15.32:0/4066195046 shutdown_connections 2024-08-31T07:17:33.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:33.151+0000 7f2d0fe00640 1 -- 172.21.15.32:0/4066195046 wait complete. 2024-08-31T07:17:33.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135611 2.10 MB/sec execute 82 sec latency 3485.252 ms 2024-08-31T07:17:34.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135611 2.08 MB/sec execute 83 sec latency 4485.414 ms 2024-08-31T07:17:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:34 smithi084 ceph-mon[22387]: pgmap v512: 97 pgs: 97 active+clean; 83 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 515 KiB/s wr, 91 op/s 2024-08-31T07:17:34.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:34 smithi032 ceph-mon[17252]: pgmap v512: 97 pgs: 97 active+clean; 83 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 515 KiB/s wr, 91 op/s 2024-08-31T07:17:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:34 smithi073 ceph-mon[21995]: pgmap v512: 97 pgs: 97 active+clean; 83 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 515 KiB/s wr, 91 op/s 2024-08-31T07:17:35.111 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:17:35.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:17:35.529 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:17:35.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 137210 2.13 MB/sec execute 84 sec latency 4902.771 ms 2024-08-31T07:17:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.231+0000 7f67a5e00640 1 -- 172.21.15.32:0/1785638700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67a010b2a0 msgr2=0x7f67a010b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.231+0000 7f67a5e00640 1 --2- 172.21.15.32:0/1785638700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67a010b2a0 0x7f67a010b6b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f678c009900 tx=0x7f678c02a410 comp rx=0 tx=0).stop 2024-08-31T07:17:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.231+0000 7f67a5e00640 1 -- 172.21.15.32:0/1785638700 shutdown_connections 2024-08-31T07:17:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.231+0000 7f67a5e00640 1 --2- 172.21.15.32:0/1785638700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a010bbf0 0x7f67a0112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.231+0000 7f67a5e00640 1 --2- 172.21.15.32:0/1785638700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67a010b2a0 0x7f67a010b6b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.231+0000 7f67a5e00640 1 --2- 172.21.15.32:0/1785638700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a0105960 0x7f67a0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.231+0000 7f67a5e00640 1 -- 172.21.15.32:0/1785638700 >> 172.21.15.32:0/1785638700 conn(0x7f67a01013d0 msgr2=0x7f67a0103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.231+0000 7f67a5e00640 1 -- 172.21.15.32:0/1785638700 shutdown_connections 2024-08-31T07:17:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.231+0000 7f67a5e00640 1 -- 172.21.15.32:0/1785638700 wait complete. 2024-08-31T07:17:36.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.233+0000 7f67a5e00640 1 Processor -- start 2024-08-31T07:17:36.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.233+0000 7f67a5e00640 1 -- start start 2024-08-31T07:17:36.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.233+0000 7f67a5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67a0105960 0x7f67a019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.233+0000 7f67a5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a010b2a0 0x7f67a019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.233+0000 7f67a5e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a010bbf0 0x7f67a01a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.233+0000 7f67a5e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01158b0 con 0x7f67a010b2a0 2024-08-31T07:17:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.233+0000 7f67a5e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0115730 con 0x7f67a0105960 2024-08-31T07:17:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.233+0000 7f67a5e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0115a30 con 0x7f67a010bbf0 2024-08-31T07:17:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.234+0000 7f67a4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67a0105960 0x7f67a019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.234+0000 7f679fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a010b2a0 0x7f67a019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.234+0000 7f67a4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67a0105960 0x7f67a019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57616/0 (socket says 172.21.15.32:57616) 2024-08-31T07:17:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.234+0000 7f67a4a00640 1 -- 172.21.15.32:0/2036207543 learned_addr learned my addr 172.21.15.32:0/2036207543 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.234+0000 7f67a5400640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a010bbf0 0x7f67a01a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.234+0000 7f67a4a00640 1 -- 172.21.15.32:0/2036207543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a010bbf0 msgr2=0x7f67a01a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.234+0000 7f67a4a00640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a010bbf0 0x7f67a01a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.234+0000 7f67a4a00640 1 -- 172.21.15.32:0/2036207543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a010b2a0 msgr2=0x7f67a019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.234+0000 7f67a4a00640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a010b2a0 0x7f67a019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.234+0000 7f67a4a00640 1 -- 172.21.15.32:0/2036207543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a01a2250 con 0x7f67a0105960 2024-08-31T07:17:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.235+0000 7f67a5400640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a010bbf0 0x7f67a01a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:17:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.235+0000 7f67a4a00640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67a0105960 0x7f67a019c2e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f679400d950 tx=0x7f679400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.235+0000 7f679d600640 1 -- 172.21.15.32:0/2036207543 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794017030 con 0x7f67a0105960 2024-08-31T07:17:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.235+0000 7f67a5e00640 1 -- 172.21.15.32:0/2036207543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a01a2540 con 0x7f67a0105960 2024-08-31T07:17:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.235+0000 7f679d600640 1 -- 172.21.15.32:0/2036207543 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6794004020 con 0x7f67a0105960 2024-08-31T07:17:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.235+0000 7f679d600640 1 -- 172.21.15.32:0/2036207543 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67940047a0 con 0x7f67a0105960 2024-08-31T07:17:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.235+0000 7f67a5e00640 1 -- 172.21.15.32:0/2036207543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a01a8ca0 con 0x7f67a0105960 2024-08-31T07:17:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.239+0000 7f67ae091740 1 Processor -- start 2024-08-31T07:17:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.239+0000 7f67ae091740 1 -- start start 2024-08-31T07:17:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f67ae091740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf636350 0x560ebf9dc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f67ae091740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf60d490 0x560ebf9dcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f67ae091740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9dd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f67ae091740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560ebf952750 con 0x560ebf636350 2024-08-31T07:17:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f67ae091740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560ebf941ce0 con 0x560ebf60d490 2024-08-31T07:17:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f67ae091740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560ebf945c90 con 0x560ebf615670 2024-08-31T07:17:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f67a5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf636350 0x560ebf9dc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f67a5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf636350 0x560ebf9dc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41452/0 (socket says 172.21.15.32:41452) 2024-08-31T07:17:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f67a5400640 1 -- 172.21.15.32:0/1826946791 learned_addr learned my addr 172.21.15.32:0/1826946791 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f679fe00640 1 --2- 172.21.15.32:0/1826946791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9dd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.240+0000 7f679d600640 1 -- 172.21.15.32:0/2036207543 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6794009a30 con 0x7f67a0105960 2024-08-31T07:17:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.241+0000 7f67a4a00640 1 --2- 172.21.15.32:0/1826946791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf60d490 0x560ebf9dcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.241+0000 7f679fe00640 1 -- 172.21.15.32:0/1826946791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf60d490 msgr2=0x560ebf9dcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.241+0000 7f679fe00640 1 --2- 172.21.15.32:0/1826946791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf60d490 0x560ebf9dcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.241+0000 7f679fe00640 1 -- 172.21.15.32:0/1826946791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf636350 msgr2=0x560ebf9dc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.241+0000 7f679fe00640 1 --2- 172.21.15.32:0/1826946791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf636350 0x560ebf9dc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.241+0000 7f679fe00640 1 -- 172.21.15.32:0/1826946791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ebf6a1930 con 0x560ebf615670 2024-08-31T07:17:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.241+0000 7f679fe00640 1 --2- 172.21.15.32:0/1826946791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9dd250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f678c02a930 tx=0x7f678c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68119cdf185f2874 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.242+0000 7f6791c00640 1 -- 172.21.15.32:0/1826946791 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c040070 con 0x560ebf615670 2024-08-31T07:17:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.242+0000 7f6791c00640 1 -- 172.21.15.32:0/1826946791 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f678c004030 con 0x560ebf615670 2024-08-31T07:17:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.242+0000 7f6791c00640 1 -- 172.21.15.32:0/1826946791 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c035040 con 0x560ebf615670 2024-08-31T07:17:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f679d600640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67880b4ca0 0x7f67880b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f67ae091740 1 -- 172.21.15.32:0/1826946791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 msgr2=0x560ebf9dd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f67ae091740 1 --2- 172.21.15.32:0/1826946791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9dd250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f678c02a930 tx=0x7f678c000bc0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f67ae091740 1 -- 172.21.15.32:0/1826946791 shutdown_connections 2024-08-31T07:17:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f67ae091740 1 --2- 172.21.15.32:0/1826946791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9dd250 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f679d600640 1 -- 172.21.15.32:0/2036207543 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f67940e4fb0 con 0x7f67a0105960 2024-08-31T07:17:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f67ae091740 1 --2- 172.21.15.32:0/1826946791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf60d490 0x560ebf9dcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f67ae091740 1 --2- 172.21.15.32:0/1826946791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf636350 0x560ebf9dc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f67ae091740 1 -- 172.21.15.32:0/1826946791 >> 172.21.15.32:0/1826946791 conn(0x560ebf62a740 msgr2=0x560ebf9da6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f67ae091740 1 -- 172.21.15.32:0/1826946791 shutdown_connections 2024-08-31T07:17:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f67a5400640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67880b4ca0 0x7f67880b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.243+0000 7f67ae091740 1 -- 172.21.15.32:0/1826946791 wait complete. 2024-08-31T07:17:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.244+0000 7f67ae091740 1 -- start start 2024-08-31T07:17:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.244+0000 7f67a5400640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67880b4ca0 0x7f67880b71b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6798005bf0 tx=0x7f6798009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.244+0000 7f67ae091740 1 Processor -- start 2024-08-31T07:17:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67ae091740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9e8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67ae091740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 0x560ebf9e8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67ae091740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9f4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67ae091740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560ebf7a4890 con 0x560ebf615670 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67ae091740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560ebf8c8cd0 con 0x560ebf636350 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67ae091740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560ebf9eaf50 con 0x560ebf60d490 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67ae091740 10 client.0 _ll_register_callbacks cb 0x560ebf93cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67a4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67a5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67a4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9e8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57994/0 (socket says 172.21.15.32:57994) 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f67a4a00640 1 -- 172.21.15.32:0/2438524356 learned_addr learned my addr 172.21.15.32:0/2438524356 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.245+0000 7f679fe00640 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 0x560ebf9e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f67a4a00640 1 -- 172.21.15.32:0/2438524356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 msgr2=0x560ebf9f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f67a4a00640 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f67a4a00640 1 -- 172.21.15.32:0/2438524356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 msgr2=0x560ebf9e8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f67a4a00640 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 0x560ebf9e8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f67a4a00640 1 -- 172.21.15.32:0/2438524356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ebf9121b0 con 0x560ebf60d490 2024-08-31T07:17:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f679fe00640 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 0x560ebf9e8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:17:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f67a5400640 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f67a4a00640 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9e8350 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6794000c00 tx=0x7f6794009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc071d8c9e30d1e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f6786000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:17:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f6786000640 1 -- 172.21.15.32:0/2438524356 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67940ab1d0 con 0x560ebf60d490 2024-08-31T07:17:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f6786000640 1 -- 172.21.15.32:0/2438524356 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f67940a7950 con 0x560ebf60d490 2024-08-31T07:17:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f67ae091740 1 -- 172.21.15.32:0/2438524356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f678c004030 con 0x560ebf60d490 2024-08-31T07:17:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f6786000640 1 -- 172.21.15.32:0/2438524356 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f67940abdc0 con 0x560ebf60d490 2024-08-31T07:17:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.246+0000 7f67ae091740 10 client.26611 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.247+0000 7f67ae091740 1 -- 172.21.15.32:0/2438524356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560ebf9f6f30 con 0x560ebf60d490 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.247+0000 7f6786000640 1 -- 172.21.15.32:0/2438524356 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f678c004030 con 0x560ebf60d490 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.247+0000 7f6786000640 1 -- 172.21.15.32:0/2438524356 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f67940e29a0 con 0x560ebf60d490 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.247+0000 7f67ae091740 10 client.26611 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.247+0000 7f67ae091740 10 client.26611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.247+0000 7f67ae091740 20 client.26611 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.247+0000 7f67ae091740 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ebf5f4cd0 0x560ebf9fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.247+0000 7f67ae091740 4 client.26611 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.247+0000 7f67ae091740 1 -- 172.21.15.32:0/2438524356 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ebf9546a0 con 0x560ebf5f4cd0 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.248+0000 7f679fe00640 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ebf5f4cd0 0x560ebf9fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.248+0000 7f679fe00640 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ebf5f4cd0 0x560ebf9fa2e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102a8884f027d7ae server_cookie=13e46e63a487369 in_seq=0 out_seq=0 2024-08-31T07:17:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.248+0000 7f6786000640 10 client.26611 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f6786000640 1 -- 172.21.15.32:0/2438524356 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560ebf9546a0 con 0x560ebf5f4cd0 2024-08-31T07:17:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f6786000640 10 client.26611 handle_command_reply: tid=0 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f67ae091740 1 client.26611 shutdown 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f67ae091740 1 -- 172.21.15.32:0/2438524356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 msgr2=0x560ebf9e8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f67ae091740 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9e8350 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6794000c00 tx=0x7f6794009f50 comp rx=0 tx=0).stop 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f67ae091740 1 -- 172.21.15.32:0/2438524356 shutdown_connections 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f67ae091740 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ebf5f4cd0 0x560ebf9fa2e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f67ae091740 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f67ae091740 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 0x560ebf9e8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f67ae091740 1 --2- 172.21.15.32:0/2438524356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9e8350 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f67ae091740 1 -- 172.21.15.32:0/2438524356 >> 172.21.15.32:0/2438524356 conn(0x560ebf62a740 msgr2=0x560ebf9df9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.249+0000 7f6786000640 0 client.26611 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.250+0000 7f67ae091740 1 -- 172.21.15.32:0/2438524356 shutdown_connections 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.250+0000 7f67ae091740 1 -- 172.21.15.32:0/2438524356 wait complete. 2024-08-31T07:17:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.250+0000 7f67ae091740 20 client.26611 trim_cache size 0 max 16384 2024-08-31T07:17:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.271+0000 7f67ae091740 1 Processor -- start 2024-08-31T07:17:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.271+0000 7f67ae091740 1 -- start start 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.271+0000 7f67ae091740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf60d490 0x560ebf9dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.271+0000 7f67ae091740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9f4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.271+0000 7f67ae091740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9f4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.271+0000 7f67ae091740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01b13c0 con 0x560ebf60d490 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.271+0000 7f67ae091740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01b2f00 con 0x560ebf636350 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.271+0000 7f67ae091740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01b1a90 con 0x560ebf615670 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f679fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f67a4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf60d490 0x560ebf9dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f679fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58010/0 (socket says 172.21.15.32:58010) 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f679fe00640 1 -- 172.21.15.32:0/824971657 learned_addr learned my addr 172.21.15.32:0/824971657 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f67a5400640 1 --2- 172.21.15.32:0/824971657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f679fe00640 1 -- 172.21.15.32:0/824971657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 msgr2=0x560ebf9f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f679fe00640 1 --2- 172.21.15.32:0/824971657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f679fe00640 1 -- 172.21.15.32:0/824971657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf60d490 msgr2=0x560ebf9dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f679fe00640 1 --2- 172.21.15.32:0/824971657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf60d490 0x560ebf9dffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f679fe00640 1 -- 172.21.15.32:0/824971657 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ebf9e70b0 con 0x560ebf615670 2024-08-31T07:17:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.272+0000 7f67a5400640 1 --2- 172.21.15.32:0/824971657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:17:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.273+0000 7f679fe00640 1 --2- 172.21.15.32:0/824971657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9f4140 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x560ebf9faa30 tx=0x7f678c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6527095837739d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:36.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.273+0000 7f6791200640 1 -- 172.21.15.32:0/824971657 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c040070 con 0x560ebf615670 2024-08-31T07:17:36.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.273+0000 7f6791200640 1 -- 172.21.15.32:0/824971657 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f678c004460 con 0x560ebf615670 2024-08-31T07:17:36.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.273+0000 7f6791200640 1 -- 172.21.15.32:0/824971657 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c035040 con 0x560ebf615670 2024-08-31T07:17:36.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.274+0000 7f67ae091740 1 -- 172.21.15.32:0/824971657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 msgr2=0x560ebf9f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:36.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.274+0000 7f67ae091740 1 --2- 172.21.15.32:0/824971657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9f4140 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x560ebf9faa30 tx=0x7f678c009780 comp rx=0 tx=0).stop 2024-08-31T07:17:36.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.274+0000 7f67ae091740 1 -- 172.21.15.32:0/824971657 shutdown_connections 2024-08-31T07:17:36.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.274+0000 7f67ae091740 1 --2- 172.21.15.32:0/824971657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.274+0000 7f67ae091740 1 --2- 172.21.15.32:0/824971657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf615670 0x560ebf9f4140 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.274+0000 7f67ae091740 1 --2- 172.21.15.32:0/824971657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf60d490 0x560ebf9dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.274+0000 7f67ae091740 1 -- 172.21.15.32:0/824971657 >> 172.21.15.32:0/824971657 conn(0x560ebf62a740 msgr2=0x560ebf9df8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.274+0000 7f67ae091740 1 -- 172.21.15.32:0/824971657 shutdown_connections 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.274+0000 7f67ae091740 1 -- 172.21.15.32:0/824971657 wait complete. 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.274+0000 7f67ae091740 1 -- start start 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.275+0000 7f67ae091740 1 Processor -- start 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.275+0000 7f67ae091740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9f0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.275+0000 7f67ae091740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 0x560ebf9eb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.275+0000 7f67ae091740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.275+0000 7f67ae091740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01b2830 con 0x560ebf615670 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.275+0000 7f67ae091740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01b2160 con 0x560ebf636350 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.275+0000 7f67ae091740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01af3a0 con 0x560ebf60d490 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.275+0000 7f67ae091740 10 client.0 _ll_register_callbacks cb 0x7f67940a7580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.275+0000 7f67a4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9eb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57640/0 (socket says 172.21.15.32:57640) 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a5400640 1 -- 172.21.15.32:0/907181860 learned_addr learned my addr 172.21.15.32:0/907181860 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58026/0 (socket says 172.21.15.32:58026) 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f679fe00640 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 0x560ebf9eb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a5400640 1 -- 172.21.15.32:0/907181860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 msgr2=0x560ebf9f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a5400640 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9f0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a5400640 1 -- 172.21.15.32:0/907181860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 msgr2=0x560ebf9eb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a5400640 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 0x560ebf9eb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a5400640 1 -- 172.21.15.32:0/907181860 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ebf9ebbe0 con 0x560ebf636350 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a4a00640 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f67a5400640 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9eb6a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f679801e560 tx=0x7f6798004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d97cd5eabf908682 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:36.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.276+0000 7f6786000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.277+0000 7f6786000640 1 -- 172.21.15.32:0/907181860 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798004b20 con 0x560ebf636350 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.277+0000 7f67ae091740 1 -- 172.21.15.32:0/907181860 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f678c004460 con 0x560ebf636350 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.277+0000 7f6786000640 1 -- 172.21.15.32:0/907181860 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f6798004cc0 con 0x560ebf636350 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.277+0000 7f6786000640 1 -- 172.21.15.32:0/907181860 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6798005020 con 0x560ebf636350 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.277+0000 7f67ae091740 10 client.26603 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.277+0000 7f67ae091740 1 -- 172.21.15.32:0/907181860 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560ebfa659d0 con 0x560ebf636350 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.277+0000 7f6786000640 1 -- 172.21.15.32:0/907181860 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f678c004460 con 0x560ebf636350 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.278+0000 7f6786000640 1 -- 172.21.15.32:0/907181860 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f6798032690 con 0x560ebf636350 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.278+0000 7f67ae091740 10 client.26603 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.278+0000 7f67ae091740 10 client.26603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.278+0000 7f67ae091740 20 client.26603 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.278+0000 7f67ae091740 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ebf5ca080 0x560ebfa681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.278+0000 7f67ae091740 4 client.26603 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:17:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.278+0000 7f67ae091740 1 -- 172.21.15.32:0/907181860 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560ebf950140 con 0x560ebf5ca080 2024-08-31T07:17:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.278+0000 7f67a4a00640 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ebf5ca080 0x560ebfa681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.279+0000 7f67a4a00640 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ebf5ca080 0x560ebfa681f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ff5c47560aff52 server_cookie=a16f269b8ae56e3b in_seq=0 out_seq=0 2024-08-31T07:17:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.279+0000 7f6786000640 10 client.26603 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:36.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f6786000640 1 -- 172.21.15.32:0/907181860 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ebf950140 con 0x560ebf5ca080 2024-08-31T07:17:36.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f6786000640 10 client.26603 handle_command_reply: tid=0 2024-08-31T07:17:36.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f67ae091740 1 client.26603 shutdown 2024-08-31T07:17:36.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f67ae091740 1 -- 172.21.15.32:0/907181860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 msgr2=0x560ebf9eb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:36.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f67ae091740 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9eb6a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f679801e560 tx=0x7f6798004a40 comp rx=0 tx=0).stop 2024-08-31T07:17:36.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f67ae091740 1 -- 172.21.15.32:0/907181860 shutdown_connections 2024-08-31T07:17:36.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f67ae091740 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ebf5ca080 0x560ebfa681f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f67ae091740 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ebf636350 0x560ebf9eb6a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f67ae091740 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ebf615670 0x560ebf9eb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f6786000640 0 client.26603 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:36.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f67ae091740 1 --2- 172.21.15.32:0/907181860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ebf60d490 0x560ebf9f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.282+0000 7f67ae091740 1 -- 172.21.15.32:0/907181860 >> 172.21.15.32:0/907181860 conn(0x560ebf62a740 msgr2=0x560ebf9d9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:36.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.283+0000 7f67ae091740 1 -- 172.21.15.32:0/907181860 shutdown_connections 2024-08-31T07:17:36.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.283+0000 7f67ae091740 1 -- 172.21.15.32:0/907181860 wait complete. 2024-08-31T07:17:36.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.283+0000 7f67ae091740 20 client.26603 trim_cache size 0 max 16384 2024-08-31T07:17:36.284 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:17:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:17:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "47f0c443-ae9b-4a28-b8df-2ef348dd2274", 2024-08-31T07:17:36.285 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:17:36.285 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:17:36.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 -- 172.21.15.32:0/2036207543 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67880b4ca0 msgr2=0x7f67880b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:36.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67880b4ca0 0x7f67880b71b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6798005bf0 tx=0x7f6798009500 comp rx=0 tx=0).stop 2024-08-31T07:17:36.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 -- 172.21.15.32:0/2036207543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67a0105960 msgr2=0x7f67a019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:36.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67a0105960 0x7f67a019c2e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f679400d950 tx=0x7f679400de10 comp rx=0 tx=0).stop 2024-08-31T07:17:36.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 -- 172.21.15.32:0/2036207543 shutdown_connections 2024-08-31T07:17:36.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f67880b4ca0 0x7f67880b71b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67a010bbf0 0x7f67a01a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a010b2a0 0x7f67a019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 --2- 172.21.15.32:0/2036207543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67a0105960 0x7f67a019c2e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:36.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 -- 172.21.15.32:0/2036207543 >> 172.21.15.32:0/2036207543 conn(0x7f67a01013d0 msgr2=0x7f67a01037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:36.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 -- 172.21.15.32:0/2036207543 shutdown_connections 2024-08-31T07:17:36.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:36.284+0000 7f67a5e00640 1 -- 172.21.15.32:0/2036207543 wait complete. 2024-08-31T07:17:36.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:36 smithi032 ceph-mon[17252]: pgmap v513: 97 pgs: 97 active+clean; 83 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 416 KiB/s wr, 77 op/s 2024-08-31T07:17:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:36 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:17:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:36 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:36.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 138757 2.25 MB/sec execute 85 sec latency 33.954 ms 2024-08-31T07:17:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:36 smithi084 ceph-mon[22387]: pgmap v513: 97 pgs: 97 active+clean; 83 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 416 KiB/s wr, 77 op/s 2024-08-31T07:17:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:36 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:17:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:36 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:36 smithi073 ceph-mon[21995]: pgmap v513: 97 pgs: 97 active+clean; 83 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 416 KiB/s wr, 77 op/s 2024-08-31T07:17:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:36 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:17:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:36 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.244 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:17:37.625 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:17:37.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 139034 2.22 MB/sec execute 86 sec latency 718.791 ms 2024-08-31T07:17:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:17:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:37 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:17:37.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:17:37.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:17:37.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:17:37.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:17:37.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:17:37.766 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:17:37.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.767 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:37 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:17:37.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:17:37.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:17:37.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:17:37.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:17:37.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:17:37.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:17:37.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:37.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:37 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:17:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.349+0000 7f0d29c00640 1 -- 172.21.15.32:0/3866471460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d24075b70 msgr2=0x7f0d24075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.349+0000 7f0d29c00640 1 --2- 172.21.15.32:0/3866471460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d24075b70 0x7f0d24075f80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009900 tx=0x7f0d0c02a410 comp rx=0 tx=0).stop 2024-08-31T07:17:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.349+0000 7f0d29c00640 1 -- 172.21.15.32:0/3866471460 shutdown_connections 2024-08-31T07:17:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.349+0000 7f0d29c00640 1 --2- 172.21.15.32:0/3866471460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d241141d0 0x7f0d241166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.349+0000 7f0d29c00640 1 --2- 172.21.15.32:0/3866471460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d24075b70 0x7f0d24075f80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.349+0000 7f0d29c00640 1 --2- 172.21.15.32:0/3866471460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d24077120 0x7f0d24075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.349+0000 7f0d29c00640 1 -- 172.21.15.32:0/3866471460 >> 172.21.15.32:0/3866471460 conn(0x7f0d2407b5f0 msgr2=0x7f0d2407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.349+0000 7f0d29c00640 1 -- 172.21.15.32:0/3866471460 shutdown_connections 2024-08-31T07:17:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.349+0000 7f0d29c00640 1 -- 172.21.15.32:0/3866471460 wait complete. 2024-08-31T07:17:38.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.350+0000 7f0d29c00640 1 Processor -- start 2024-08-31T07:17:38.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.350+0000 7f0d29c00640 1 -- start start 2024-08-31T07:17:38.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d29c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d24075b70 0x7f0d241a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d29c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d24077120 0x7f0d241a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d29c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d241141d0 0x7f0d241a6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d29c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d24119e20 con 0x7f0d241141d0 2024-08-31T07:17:38.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d29c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0d24119ca0 con 0x7f0d24077120 2024-08-31T07:17:38.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d29c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d24119fa0 con 0x7f0d24075b70 2024-08-31T07:17:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d23400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d241141d0 0x7f0d241a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d22a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d24075b70 0x7f0d241a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d22a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d24075b70 0x7f0d241a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58048/0 (socket says 172.21.15.32:58048) 2024-08-31T07:17:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d22a00640 1 -- 172.21.15.32:0/529049669 learned_addr learned my addr 172.21.15.32:0/529049669 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d1be00640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d24077120 0x7f0d241a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d22a00640 1 -- 172.21.15.32:0/529049669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d24077120 msgr2=0x7f0d241a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:38.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d22a00640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d24077120 0x7f0d241a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d22a00640 1 -- 172.21.15.32:0/529049669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d241141d0 msgr2=0x7f0d241a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d22a00640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d241141d0 0x7f0d241a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.351+0000 7f0d22a00640 1 -- 172.21.15.32:0/529049669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d241a6760 con 0x7f0d24075b70 2024-08-31T07:17:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.352+0000 7f0d22a00640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d24075b70 0x7f0d241a07f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400e9f0 tx=0x7f0d1400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.352+0000 7f0d1b400640 1 -- 172.21.15.32:0/529049669 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400cd80 con 0x7f0d24075b70 2024-08-31T07:17:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.352+0000 7f0d29c00640 1 -- 172.21.15.32:0/529049669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d241a6a50 con 0x7f0d24075b70 2024-08-31T07:17:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.352+0000 7f0d29c00640 1 -- 172.21.15.32:0/529049669 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d241ad1b0 con 0x7f0d24075b70 2024-08-31T07:17:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.352+0000 7f0d1b400640 1 -- 172.21.15.32:0/529049669 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d140108e0 con 0x7f0d24075b70 2024-08-31T07:17:38.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.352+0000 7f0d1b400640 1 -- 172.21.15.32:0/529049669 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14018980 con 0x7f0d24075b70 2024-08-31T07:17:38.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.355+0000 7f0d307f7740 1 Processor -- start 2024-08-31T07:17:38.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.355+0000 7f0d307f7740 1 -- start start 2024-08-31T07:17:38.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055074c070 0x560550af1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605507231b0 0x560550af1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055072b390 0x560550af23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d307f7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560550a68770 con 0x56055072b390 2024-08-31T07:17:38.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d307f7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560550a57d00 con 0x5605507231b0 2024-08-31T07:17:38.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d307f7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560550a5bcb0 con 0x56055074c070 2024-08-31T07:17:38.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d23400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055074c070 0x560550af1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055072b390 0x560550af23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d23400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055074c070 0x560550af1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58050/0 (socket says 172.21.15.32:58050) 2024-08-31T07:17:38.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d23400640 1 -- 172.21.15.32:0/3959770648 learned_addr learned my addr 172.21.15.32:0/3959770648 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:38.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.356+0000 7f0d1b400640 1 -- 172.21.15.32:0/529049669 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0d14010430 con 0x7f0d24075b70 2024-08-31T07:17:38.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.357+0000 7f0d23400640 1 -- 172.21.15.32:0/3959770648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605507231b0 msgr2=0x560550af1e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:17:38.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.357+0000 7f0d23400640 1 --2- 172.21.15.32:0/3959770648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605507231b0 0x560550af1e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.357+0000 7f0d23400640 1 -- 172.21.15.32:0/3959770648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055072b390 msgr2=0x560550af23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:38.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.357+0000 7f0d23400640 1 --2- 172.21.15.32:0/3959770648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055072b390 0x560550af23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.357+0000 7f0d23400640 1 -- 172.21.15.32:0/3959770648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605507b7650 con 0x56055074c070 2024-08-31T07:17:38.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.357+0000 7f0d1be00640 1 --2- 172.21.15.32:0/3959770648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055072b390 0x560550af23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:38.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.357+0000 7f0d23400640 1 --2- 172.21.15.32:0/3959770648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055074c070 0x560550af1950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009fd0 tx=0x7f0d1c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17dd610faea998c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:38.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.357+0000 7f0d13e00640 1 -- 172.21.15.32:0/3959770648 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c004d20 con 0x56055074c070 2024-08-31T07:17:38.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.357+0000 7f0d13e00640 1 -- 172.21.15.32:0/3959770648 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d1c004050 con 0x56055074c070 2024-08-31T07:17:38.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.357+0000 7f0d13e00640 1 -- 172.21.15.32:0/3959770648 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c005280 con 0x56055074c070 2024-08-31T07:17:38.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.358+0000 7f0d307f7740 1 -- 172.21.15.32:0/3959770648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055074c070 msgr2=0x560550af1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:38.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.358+0000 7f0d307f7740 1 --2- 172.21.15.32:0/3959770648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055074c070 0x560550af1950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009fd0 tx=0x7f0d1c0118d0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.358+0000 7f0d307f7740 1 -- 172.21.15.32:0/3959770648 shutdown_connections 2024-08-31T07:17:38.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.358+0000 7f0d307f7740 1 --2- 172.21.15.32:0/3959770648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055072b390 0x560550af23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.358+0000 7f0d307f7740 1 --2- 172.21.15.32:0/3959770648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605507231b0 0x560550af1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.358+0000 7f0d307f7740 1 --2- 172.21.15.32:0/3959770648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055074c070 0x560550af1950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.358+0000 7f0d307f7740 1 -- 172.21.15.32:0/3959770648 >> 172.21.15.32:0/3959770648 conn(0x560550740460 msgr2=0x560550aef840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.358+0000 7f0d307f7740 1 -- 172.21.15.32:0/3959770648 shutdown_connections 2024-08-31T07:17:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.359+0000 7f0d307f7740 1 -- 172.21.15.32:0/3959770648 wait complete. 2024-08-31T07:17:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.359+0000 7f0d1b400640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0d080b4c30 0x7f0d080b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.359+0000 7f0d307f7740 1 -- start start 2024-08-31T07:17:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.359+0000 7f0d1b400640 1 -- 172.21.15.32:0/529049669 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f0d140e5550 con 0x7f0d24075b70 2024-08-31T07:17:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.359+0000 7f0d23400640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0d080b4c30 0x7f0d080b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.359+0000 7f0d307f7740 1 Processor -- start 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 0x560550afd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d23400640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0d080b4c30 0x7f0d080b7140 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c004910 tx=0x7f0d1c000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550afdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550b094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d307f7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5605508ba700 con 0x56055074c070 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d307f7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5605509decb0 con 0x56055072b390 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d307f7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560550b00130 con 0x5605507231b0 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d307f7740 10 client.0 _ll_register_callbacks cb 0x560550a04430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d22a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 0x560550afd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550afdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550afdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57686/0 (socket says 172.21.15.32:57686) 2024-08-31T07:17:38.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d1be00640 1 -- 172.21.15.32:0/548967544 learned_addr learned my addr 172.21.15.32:0/548967544 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.360+0000 7f0d23400640 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550b094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.361+0000 7f0d1be00640 1 -- 172.21.15.32:0/548967544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 msgr2=0x560550afd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.361+0000 7f0d1be00640 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 0x560550afd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.361+0000 7f0d1be00640 1 -- 172.21.15.32:0/548967544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 msgr2=0x560550b094b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.361+0000 7f0d1be00640 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550b094b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.361+0000 7f0d1be00640 1 -- 172.21.15.32:0/548967544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560550a1f870 con 0x56055072b390 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.361+0000 7f0d22a00640 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 0x560550afd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.361+0000 7f0d1be00640 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550afdaa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000c00 tx=0x7f0d0c0043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd16ba708a15c93 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.361+0000 7f0d10c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.362+0000 7f0d10c00640 1 -- 172.21.15.32:0/548967544 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c035040 con 0x56055072b390 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.362+0000 7f0d307f7740 1 -- 172.21.15.32:0/548967544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d1c004050 con 0x56055072b390 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.362+0000 7f0d10c00640 1 -- 172.21.15.32:0/548967544 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f0d0c005450 con 0x56055072b390 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.362+0000 7f0d10c00640 1 -- 172.21.15.32:0/548967544 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d0c005650 con 0x56055072b390 2024-08-31T07:17:38.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.362+0000 7f0d307f7740 10 client.26612 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:38.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.362+0000 7f0d307f7740 1 -- 172.21.15.32:0/548967544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560550b0c1f0 con 0x56055072b390 2024-08-31T07:17:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.362+0000 7f0d10c00640 1 -- 172.21.15.32:0/548967544 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d1c004050 con 0x56055072b390 2024-08-31T07:17:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.363+0000 7f0d10c00640 1 -- 172.21.15.32:0/548967544 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0d0c003a00 con 0x56055072b390 2024-08-31T07:17:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.363+0000 7f0d307f7740 10 client.26612 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.363+0000 7f0d307f7740 10 client.26612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:17:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.363+0000 7f0d307f7740 20 client.26612 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.363+0000 7f0d307f7740 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56055070a9f0 0x560550b0f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.363+0000 7f0d307f7740 4 client.26612 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:17:38.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.363+0000 7f0d307f7740 1 -- 172.21.15.32:0/548967544 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560550a6a6c0 con 0x56055070a9f0 2024-08-31T07:17:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.363+0000 7f0d22a00640 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56055070a9f0 0x560550b0f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.364+0000 7f0d22a00640 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56055070a9f0 0x560550b0f610 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5a0d13f4b92c25a server_cookie=c796e1fb6744626b in_seq=0 out_seq=0 2024-08-31T07:17:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.364+0000 7f0d10c00640 10 client.26612 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.364+0000 7f0d10c00640 1 -- 172.21.15.32:0/548967544 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560550a6a6c0 con 0x56055070a9f0 2024-08-31T07:17:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.364+0000 7f0d10c00640 10 client.26612 handle_command_reply: tid=0 2024-08-31T07:17:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 client.26612 shutdown 2024-08-31T07:17:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 -- 172.21.15.32:0/548967544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 msgr2=0x560550afdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550afdaa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000c00 tx=0x7f0d0c0043b0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 -- 172.21.15.32:0/548967544 shutdown_connections 2024-08-31T07:17:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56055070a9f0 0x560550b0f610 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550b094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550afdaa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 --2- 172.21.15.32:0/548967544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 0x560550afd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 -- 172.21.15.32:0/548967544 >> 172.21.15.32:0/548967544 conn(0x560550740460 msgr2=0x560550af4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d10c00640 0 client.26612 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 -- 172.21.15.32:0/548967544 shutdown_connections 2024-08-31T07:17:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 1 -- 172.21.15.32:0/548967544 wait complete. 2024-08-31T07:17:38.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.365+0000 7f0d307f7740 20 client.26612 trim_cache size 0 max 16384 2024-08-31T07:17:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d307f7740 1 Processor -- start 2024-08-31T07:17:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d307f7740 1 -- start start 2024-08-31T07:17:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605507231b0 0x560550af6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055072b390 0x560550af70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550afbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d307f7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0000e620 con 0x56055074c070 2024-08-31T07:17:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d307f7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00010160 con 0x5605507231b0 2024-08-31T07:17:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d307f7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0000ecf0 con 0x56055072b390 2024-08-31T07:17:38.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055072b390 0x560550af70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d23400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550afbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055072b390 0x560550af70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58064/0 (socket says 172.21.15.32:58064) 2024-08-31T07:17:38.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d1be00640 1 -- 172.21.15.32:0/816605506 learned_addr learned my addr 172.21.15.32:0/816605506 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.384+0000 7f0d22a00640 1 --2- 172.21.15.32:0/816605506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605507231b0 0x560550af6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.385+0000 7f0d1be00640 1 -- 172.21.15.32:0/816605506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605507231b0 msgr2=0x560550af6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.385+0000 7f0d1be00640 1 --2- 172.21.15.32:0/816605506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605507231b0 0x560550af6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.385+0000 7f0d1be00640 1 -- 172.21.15.32:0/816605506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 msgr2=0x560550afbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.385+0000 7f0d1be00640 1 --2- 172.21.15.32:0/816605506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550afbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.385+0000 7f0d1be00640 1 -- 172.21.15.32:0/816605506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560550b0fb70 con 0x56055072b390 2024-08-31T07:17:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.385+0000 7f0d1be00640 1 --2- 172.21.15.32:0/816605506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055072b390 0x560550af70f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009850 tx=0x7f0d0c005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c79b1e7a13729a80 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.385+0000 7f0d13400640 1 -- 172.21.15.32:0/816605506 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c035030 con 0x56055072b390 2024-08-31T07:17:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.385+0000 7f0d13400640 1 -- 172.21.15.32:0/816605506 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0c041070 con 0x56055072b390 2024-08-31T07:17:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.385+0000 7f0d13400640 1 -- 172.21.15.32:0/816605506 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0039b0 con 0x56055072b390 2024-08-31T07:17:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.386+0000 7f0d307f7740 1 -- 172.21.15.32:0/816605506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055072b390 msgr2=0x560550af70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.386+0000 7f0d307f7740 1 --2- 172.21.15.32:0/816605506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055072b390 0x560550af70f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009850 tx=0x7f0d0c005f30 comp rx=0 tx=0).stop 2024-08-31T07:17:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.386+0000 7f0d307f7740 1 -- 172.21.15.32:0/816605506 shutdown_connections 2024-08-31T07:17:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.386+0000 7f0d307f7740 1 --2- 172.21.15.32:0/816605506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550afbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.386+0000 7f0d307f7740 1 --2- 172.21.15.32:0/816605506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56055072b390 0x560550af70f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.386+0000 7f0d307f7740 1 --2- 172.21.15.32:0/816605506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605507231b0 0x560550af6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.386+0000 7f0d307f7740 1 -- 172.21.15.32:0/816605506 >> 172.21.15.32:0/816605506 conn(0x560550740460 msgr2=0x560550af84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.386+0000 7f0d307f7740 1 -- 172.21.15.32:0/816605506 shutdown_connections 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.386+0000 7f0d307f7740 1 -- 172.21.15.32:0/816605506 wait complete. 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.387+0000 7f0d307f7740 1 -- start start 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.387+0000 7f0d307f7740 1 Processor -- start 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 0x560550b05ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550b148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d307f7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550b14df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d307f7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0000fa90 con 0x56055074c070 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d307f7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0000f3c0 con 0x56055072b390 2024-08-31T07:17:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d307f7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0000c600 con 0x5605507231b0 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d307f7740 10 client.0 _ll_register_callbacks cb 0x7f0d0c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550b148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d22a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 0x560550b05ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550b148b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57702/0 (socket says 172.21.15.32:57702) 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d1be00640 1 -- 172.21.15.32:0/3915615152 learned_addr learned my addr 172.21.15.32:0/3915615152 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d23400640 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550b14df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d1be00640 1 -- 172.21.15.32:0/3915615152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 msgr2=0x560550b05ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d1be00640 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 0x560550b05ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d1be00640 1 -- 172.21.15.32:0/3915615152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 msgr2=0x560550b14df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d1be00640 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550b14df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d1be00640 1 -- 172.21.15.32:0/3915615152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560550b15460 con 0x56055072b390 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.388+0000 7f0d23400640 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550b14df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.389+0000 7f0d1be00640 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550b148b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c031440 tx=0x7f0d0c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79171c9654510b75 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:17:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.389+0000 7f0d10c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:17:38.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.389+0000 7f0d10c00640 1 -- 172.21.15.32:0/3915615152 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c035040 con 0x56055072b390 2024-08-31T07:17:38.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.389+0000 7f0d10c00640 1 -- 172.21.15.32:0/3915615152 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f0d0c031b40 con 0x56055072b390 2024-08-31T07:17:38.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.389+0000 7f0d307f7740 1 -- 172.21.15.32:0/3915615152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d0c041070 con 0x56055072b390 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.389+0000 7f0d10c00640 1 -- 172.21.15.32:0/3915615152 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d0c068050 con 0x56055072b390 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.389+0000 7f0d307f7740 10 client.26618 fetch_fsmap learned FSMap version 33 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.389+0000 7f0d307f7740 1 -- 172.21.15.32:0/3915615152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560550b02340 con 0x56055072b390 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.390+0000 7f0d10c00640 1 -- 172.21.15.32:0/3915615152 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0c041070 con 0x56055072b390 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.390+0000 7f0d10c00640 1 -- 172.21.15.32:0/3915615152 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0d0c061060 con 0x56055072b390 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.390+0000 7f0d307f7740 10 client.26618 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.390+0000 7f0d307f7740 10 client.26618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.390+0000 7f0d307f7740 20 client.26618 populate_metadata read hostname 'smithi032' 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.390+0000 7f0d307f7740 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5605506dfda0 0x560550b7c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.390+0000 7f0d307f7740 4 client.26618 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.390+0000 7f0d307f7740 1 -- 172.21.15.32:0/3915615152 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560550a66160 con 0x5605506dfda0 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.390+0000 7f0d22a00640 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5605506dfda0 0x560550b7c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:17:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d22a00640 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5605506dfda0 0x560550b7c4a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21bee5ac70e3b31 server_cookie=a3e8f92518d8a904 in_seq=0 out_seq=0 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d10c00640 10 client.26618 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d10c00640 1 -- 172.21.15.32:0/3915615152 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x560550a66160 con 0x5605506dfda0 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d10c00640 10 client.26618 handle_command_reply: tid=0 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d307f7740 1 client.26618 shutdown 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d307f7740 1 -- 172.21.15.32:0/3915615152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 msgr2=0x560550b148b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d307f7740 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550b148b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c031440 tx=0x7f0d0c005c90 comp rx=0 tx=0).stop 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d307f7740 1 -- 172.21.15.32:0/3915615152 shutdown_connections 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d307f7740 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5605506dfda0 0x560550b7c4a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d307f7740 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56055074c070 0x560550b14df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d307f7740 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56055072b390 0x560550b148b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d307f7740 1 --2- 172.21.15.32:0/3915615152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5605507231b0 0x560550b05ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d10c00640 0 client.26618 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:17:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d307f7740 1 -- 172.21.15.32:0/3915615152 >> 172.21.15.32:0/3915615152 conn(0x560550740460 msgr2=0x560550afbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:38.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.391+0000 7f0d307f7740 1 -- 172.21.15.32:0/3915615152 shutdown_connections 2024-08-31T07:17:38.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d307f7740 1 -- 172.21.15.32:0/3915615152 wait complete. 2024-08-31T07:17:38.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d307f7740 20 client.26618 trim_cache size 0 max 16384 2024-08-31T07:17:38.396 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:17:38.396 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:17:38.396 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:17:38.396 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:17:38.396 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:38.396 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "47f0c443-ae9b-4a28-b8df-2ef348dd2274", 2024-08-31T07:17:38.396 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:17:38.397 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:38.398 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:38.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:17:38.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d29c00640 1 -- 172.21.15.32:0/529049669 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0d080b4c30 msgr2=0x7f0d080b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d29c00640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0d080b4c30 0x7f0d080b7140 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c004910 tx=0x7f0d1c000f40 comp rx=0 tx=0).stop 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d29c00640 1 -- 172.21.15.32:0/529049669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d24075b70 msgr2=0x7f0d241a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d29c00640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d24075b70 0x7f0d241a07f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400e9f0 tx=0x7f0d1400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d29c00640 1 -- 172.21.15.32:0/529049669 shutdown_connections 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d29c00640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0d080b4c30 0x7f0d080b7140 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d29c00640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d241141d0 0x7f0d241a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d29c00640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d24077120 0x7f0d241a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d29c00640 1 --2- 172.21.15.32:0/529049669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d24075b70 0x7f0d241a07f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:17:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.392+0000 7f0d29c00640 1 -- 172.21.15.32:0/529049669 >> 172.21.15.32:0/529049669 conn(0x7f0d2407b5f0 msgr2=0x7f0d24112de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:17:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.393+0000 7f0d29c00640 1 -- 172.21.15.32:0/529049669 shutdown_connections 2024-08-31T07:17:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:17:38.393+0000 7f0d29c00640 1 -- 172.21.15.32:0/529049669 wait complete. 2024-08-31T07:17:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:38 smithi032 ceph-mon[17252]: pgmap v514: 97 pgs: 97 active+clean; 83 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 365 KiB/s wr, 71 op/s 2024-08-31T07:17:38.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 139034 2.20 MB/sec execute 87 sec latency 1718.877 ms 2024-08-31T07:17:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:38 smithi084 ceph-mon[22387]: pgmap v514: 97 pgs: 97 active+clean; 83 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 365 KiB/s wr, 71 op/s 2024-08-31T07:17:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:38 smithi073 ceph-mon[21995]: pgmap v514: 97 pgs: 97 active+clean; 83 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 365 KiB/s wr, 71 op/s 2024-08-31T07:17:39.390 INFO:tasks.cephfs.filesystem:scrub status for tag:47f0c443-ae9b-4a28-b8df-2ef348dd2274 - {'path': '/', 'tag': '47f0c443-ae9b-4a28-b8df-2ef348dd2274', 'options': 'recursive,force'} 2024-08-31T07:17:39.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 139034 2.17 MB/sec execute 88 sec latency 2718.945 ms 2024-08-31T07:17:40.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:40 smithi032 ceph-mon[17252]: pgmap v515: 97 pgs: 97 active+clean; 84 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 345 KiB/s wr, 64 op/s 2024-08-31T07:17:40.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 139090 2.15 MB/sec execute 89 sec latency 3134.697 ms 2024-08-31T07:17:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:40 smithi084 ceph-mon[22387]: pgmap v515: 97 pgs: 97 active+clean; 84 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 345 KiB/s wr, 64 op/s 2024-08-31T07:17:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:40 smithi073 ceph-mon[21995]: pgmap v515: 97 pgs: 97 active+clean; 84 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 345 KiB/s wr, 64 op/s 2024-08-31T07:17:41.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 139090 2.12 MB/sec execute 90 sec latency 1551.467 ms 2024-08-31T07:17:42.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:42 smithi032 ceph-mon[17252]: pgmap v516: 97 pgs: 97 active+clean; 85 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 348 KiB/s wr, 68 op/s 2024-08-31T07:17:42.708 INFO:tasks.workunit.client.0.smithi032.stdout: 1 139119 2.10 MB/sec execute 91 sec latency 2495.717 ms 2024-08-31T07:17:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:42 smithi084 ceph-mon[22387]: pgmap v516: 97 pgs: 97 active+clean; 85 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 348 KiB/s wr, 68 op/s 2024-08-31T07:17:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:42 smithi073 ceph-mon[21995]: pgmap v516: 97 pgs: 97 active+clean; 85 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 348 KiB/s wr, 68 op/s 2024-08-31T07:17:43.708 INFO:tasks.workunit.client.0.smithi032.stdout: 1 141335 2.21 MB/sec execute 92 sec latency 55.125 ms 2024-08-31T07:17:44.708 INFO:tasks.workunit.client.0.smithi032.stdout: 1 142634 2.26 MB/sec execute 93 sec latency 15.053 ms 2024-08-31T07:17:44.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:44 smithi032 ceph-mon[17252]: pgmap v517: 97 pgs: 97 active+clean; 89 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 651 KiB/s wr, 69 op/s 2024-08-31T07:17:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:44 smithi073 ceph-mon[21995]: pgmap v517: 97 pgs: 97 active+clean; 89 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 651 KiB/s wr, 69 op/s 2024-08-31T07:17:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:44 smithi084 ceph-mon[22387]: pgmap v517: 97 pgs: 97 active+clean; 89 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 651 KiB/s wr, 69 op/s 2024-08-31T07:17:45.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 144312 2.31 MB/sec execute 94 sec latency 364.631 ms 2024-08-31T07:17:46.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 145889 2.41 MB/sec execute 95 sec latency 13.372 ms 2024-08-31T07:17:46.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:46 smithi032 ceph-mon[17252]: pgmap v518: 97 pgs: 97 active+clean; 89 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-08-31T07:17:46.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:46 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:17:46.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:46 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:17:46.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:46 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:17:46.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:46 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:17:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:46 smithi073 ceph-mon[21995]: pgmap v518: 97 pgs: 97 active+clean; 89 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-08-31T07:17:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:46 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:17:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:46 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:17:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:46 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:17:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:46 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:17:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:46 smithi084 ceph-mon[22387]: pgmap v518: 97 pgs: 97 active+clean; 89 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-08-31T07:17:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:46 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:17:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:46 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:17:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:46 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:17:46.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:46 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:17:47.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 146126 2.39 MB/sec execute 96 sec latency 779.912 ms 2024-08-31T07:17:47.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:47 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:17:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:47 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:17:47.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:47 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:17:48.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 146126 2.36 MB/sec execute 97 sec latency 1780.013 ms 2024-08-31T07:17:48.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:48 smithi032 ceph-mon[17252]: pgmap v519: 97 pgs: 97 active+clean; 87 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-08-31T07:17:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:48 smithi073 ceph-mon[21995]: pgmap v519: 97 pgs: 97 active+clean; 87 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-08-31T07:17:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:48 smithi084 ceph-mon[22387]: pgmap v519: 97 pgs: 97 active+clean; 87 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-08-31T07:17:49.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 146126 2.34 MB/sec execute 98 sec latency 2780.151 ms 2024-08-31T07:17:50.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 147638 2.37 MB/sec execute 99 sec latency 3187.787 ms 2024-08-31T07:17:50.792 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:50 smithi084 ceph-mon[22387]: pgmap v520: 97 pgs: 97 active+clean; 89 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-08-31T07:17:50.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:50 smithi032 ceph-mon[17252]: pgmap v520: 97 pgs: 97 active+clean; 89 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-08-31T07:17:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:50 smithi073 ceph-mon[21995]: pgmap v520: 97 pgs: 97 active+clean; 89 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-08-31T07:17:51.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 149372 2.48 MB/sec execute 100 sec latency 12.700 ms 2024-08-31T07:17:52.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 151378 2.53 MB/sec execute 101 sec latency 12.764 ms 2024-08-31T07:17:52.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:52 smithi032 ceph-mon[17252]: pgmap v521: 97 pgs: 97 active+clean; 89 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-08-31T07:17:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:52 smithi073 ceph-mon[21995]: pgmap v521: 97 pgs: 97 active+clean; 89 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-08-31T07:17:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:52 smithi084 ceph-mon[22387]: pgmap v521: 97 pgs: 97 active+clean; 89 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-08-31T07:17:53.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 152939 2.62 MB/sec execute 102 sec latency 10.758 ms 2024-08-31T07:17:54.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 153273 2.60 MB/sec execute 103 sec latency 622.335 ms 2024-08-31T07:17:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:54 smithi032 ceph-mon[17252]: pgmap v522: 97 pgs: 97 active+clean; 90 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.6 MiB/s wr, 136 op/s 2024-08-31T07:17:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:54 smithi073 ceph-mon[21995]: pgmap v522: 97 pgs: 97 active+clean; 90 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.6 MiB/s wr, 136 op/s 2024-08-31T07:17:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:54 smithi084 ceph-mon[22387]: pgmap v522: 97 pgs: 97 active+clean; 90 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.6 MiB/s wr, 136 op/s 2024-08-31T07:17:55.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 154849 2.64 MB/sec execute 104 sec latency 1028.696 ms 2024-08-31T07:17:56.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 156461 2.73 MB/sec execute 105 sec latency 14.098 ms 2024-08-31T07:17:56.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:56 smithi032 ceph-mon[17252]: pgmap v523: 97 pgs: 97 active+clean; 93 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-08-31T07:17:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:56 smithi073 ceph-mon[21995]: pgmap v523: 97 pgs: 97 active+clean; 93 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-08-31T07:17:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:56 smithi084 ceph-mon[22387]: pgmap v523: 97 pgs: 97 active+clean; 93 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-08-31T07:17:57.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 156815 2.71 MB/sec execute 106 sec latency 597.141 ms 2024-08-31T07:17:58.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 156815 2.68 MB/sec execute 107 sec latency 1597.324 ms 2024-08-31T07:17:58.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:17:58 smithi032 ceph-mon[17252]: pgmap v524: 97 pgs: 97 active+clean; 94 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-08-31T07:17:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:17:58 smithi073 ceph-mon[21995]: pgmap v524: 97 pgs: 97 active+clean; 94 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-08-31T07:17:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:17:58 smithi084 ceph-mon[22387]: pgmap v524: 97 pgs: 97 active+clean; 94 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-08-31T07:17:59.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 156815 2.66 MB/sec execute 108 sec latency 2597.450 ms 2024-08-31T07:18:00.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 158429 2.69 MB/sec execute 109 sec latency 3003.493 ms 2024-08-31T07:18:00.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:00 smithi032 ceph-mon[17252]: pgmap v525: 97 pgs: 97 active+clean; 96 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 911 KiB/s wr, 147 op/s 2024-08-31T07:18:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:00 smithi073 ceph-mon[21995]: pgmap v525: 97 pgs: 97 active+clean; 96 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 911 KiB/s wr, 147 op/s 2024-08-31T07:18:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:00 smithi084 ceph-mon[22387]: pgmap v525: 97 pgs: 97 active+clean; 96 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 911 KiB/s wr, 147 op/s 2024-08-31T07:18:01.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 159716 2.77 MB/sec execute 110 sec latency 25.516 ms 2024-08-31T07:18:02.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160306 2.75 MB/sec execute 111 sec latency 300.062 ms 2024-08-31T07:18:02.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:02 smithi032 ceph-mon[17252]: pgmap v526: 97 pgs: 97 active+clean; 96 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 816 KiB/s wr, 137 op/s 2024-08-31T07:18:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:02 smithi073 ceph-mon[21995]: pgmap v526: 97 pgs: 97 active+clean; 96 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 816 KiB/s wr, 137 op/s 2024-08-31T07:18:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:02 smithi084 ceph-mon[22387]: pgmap v526: 97 pgs: 97 active+clean; 96 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 816 KiB/s wr, 137 op/s 2024-08-31T07:18:03.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160306 2.73 MB/sec execute 112 sec latency 1300.202 ms 2024-08-31T07:18:04.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160306 2.71 MB/sec execute 113 sec latency 2300.337 ms 2024-08-31T07:18:04.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:04 smithi032 ceph-mon[17252]: pgmap v527: 97 pgs: 97 active+clean; 97 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 837 KiB/s wr, 141 op/s 2024-08-31T07:18:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:04 smithi073 ceph-mon[21995]: pgmap v527: 97 pgs: 97 active+clean; 97 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 837 KiB/s wr, 141 op/s 2024-08-31T07:18:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:04 smithi084 ceph-mon[22387]: pgmap v527: 97 pgs: 97 active+clean; 97 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 837 KiB/s wr, 141 op/s 2024-08-31T07:18:05.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160430 2.68 MB/sec execute 114 sec latency 2706.812 ms 2024-08-31T07:18:05.784 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 32.7 2024-08-31T07:18:05.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:18:06.179 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:06.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:06 smithi032 ceph-mon[17252]: pgmap v528: 97 pgs: 97 active+clean; 98 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 954 KiB/s wr, 158 op/s 2024-08-31T07:18:06.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160430 2.66 MB/sec execute 115 sec latency 1489.459 ms 2024-08-31T07:18:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.851+0000 7f4d69000640 1 -- 172.21.15.32:0/1022863552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d64114310 msgr2=0x7f4d64116730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.851+0000 7f4d69000640 1 --2- 172.21.15.32:0/1022863552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d64114310 0x7f4d64116730 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009ca0 tx=0x7f4d5c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:18:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.851+0000 7f4d69000640 1 -- 172.21.15.32:0/1022863552 shutdown_connections 2024-08-31T07:18:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.851+0000 7f4d69000640 1 --2- 172.21.15.32:0/1022863552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d64114310 0x7f4d64116730 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.851+0000 7f4d69000640 1 --2- 172.21.15.32:0/1022863552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d64078030 0x7f4d64113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.851+0000 7f4d69000640 1 --2- 172.21.15.32:0/1022863552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d640776e0 0x7f4d64077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.851+0000 7f4d69000640 1 -- 172.21.15.32:0/1022863552 >> 172.21.15.32:0/1022863552 conn(0x7f4d64101420 msgr2=0x7f4d64103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.851+0000 7f4d69000640 1 -- 172.21.15.32:0/1022863552 shutdown_connections 2024-08-31T07:18:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.851+0000 7f4d69000640 1 -- 172.21.15.32:0/1022863552 wait complete. 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.852+0000 7f4d69000640 1 Processor -- start 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d69000640 1 -- start start 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d69000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d640776e0 0x7f4d641a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d69000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d64078030 0x7f4d641a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d69000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d64114310 0x7f4d641a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d69000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4d64119c90 con 0x7f4d640776e0 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d69000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4d64119b10 con 0x7f4d64114310 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d69000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4d64119e10 con 0x7f4d64078030 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d63400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d640776e0 0x7f4d641a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d63400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d640776e0 0x7f4d641a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60832/0 (socket says 172.21.15.32:60832) 2024-08-31T07:18:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d63400640 1 -- 172.21.15.32:0/937239799 learned_addr learned my addr 172.21.15.32:0/937239799 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.853+0000 7f4d5be00640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d64078030 0x7f4d641a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.854+0000 7f4d63e00640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d64114310 0x7f4d641a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.854+0000 7f4d5be00640 1 -- 172.21.15.32:0/937239799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d64114310 msgr2=0x7f4d641a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.854+0000 7f4d5be00640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d64114310 0x7f4d641a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.854+0000 7f4d5be00640 1 -- 172.21.15.32:0/937239799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d640776e0 msgr2=0x7f4d641a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.854+0000 7f4d5be00640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d640776e0 0x7f4d641a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.854+0000 7f4d5be00640 1 -- 172.21.15.32:0/937239799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d641a65d0 con 0x7f4d64078030 2024-08-31T07:18:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.854+0000 7f4d5be00640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d64078030 0x7f4d641a0c00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4d6406c660 tx=0x7f4d4c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.855+0000 7f4d60c00640 1 -- 172.21.15.32:0/937239799 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00cc40 con 0x7f4d64078030 2024-08-31T07:18:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.855+0000 7f4d69000640 1 -- 172.21.15.32:0/937239799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d641a68c0 con 0x7f4d64078030 2024-08-31T07:18:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.855+0000 7f4d60c00640 1 -- 172.21.15.32:0/937239799 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d4c00cde0 con 0x7f4d64078030 2024-08-31T07:18:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.855+0000 7f4d60c00640 1 -- 172.21.15.32:0/937239799 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c004a00 con 0x7f4d64078030 2024-08-31T07:18:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.855+0000 7f4d69000640 1 -- 172.21.15.32:0/937239799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d641ad020 con 0x7f4d64078030 2024-08-31T07:18:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.856+0000 7f4d69000640 1 -- 172.21.15.32:0/937239799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d64077af0 con 0x7f4d64078030 2024-08-31T07:18:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.858+0000 7f4d60c00640 1 -- 172.21.15.32:0/937239799 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4d4c004ba0 con 0x7f4d64078030 2024-08-31T07:18:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.860+0000 7f4d60c00640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d480b4d20 0x7f4d480b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.860+0000 7f4d60c00640 1 -- 172.21.15.32:0/937239799 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f4d4c0e5490 con 0x7f4d64078030 2024-08-31T07:18:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.860+0000 7f4d63400640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d480b4d20 0x7f4d480b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.860+0000 7f4d63400640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d480b4d20 0x7f4d480b7230 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009c90 tx=0x7f4d54005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:06.861+0000 7f4d60c00640 1 -- 172.21.15.32:0/937239799 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4d4c0ac330 con 0x7f4d64078030 2024-08-31T07:18:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:06 smithi073 ceph-mon[21995]: pgmap v528: 97 pgs: 97 active+clean; 98 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 954 KiB/s wr, 158 op/s 2024-08-31T07:18:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:06 smithi084 ceph-mon[22387]: pgmap v528: 97 pgs: 97 active+clean; 98 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 954 KiB/s wr, 158 op/s 2024-08-31T07:18:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.092+0000 7f4d69000640 1 -- 172.21.15.32:0/937239799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d641a1790 con 0x7f4d64078030 2024-08-31T07:18:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.093+0000 7f4d60c00640 1 -- 172.21.15.32:0/937239799 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11960 (secure 0 0 0) 0x7f4d4c0abc30 con 0x7f4d64078030 2024-08-31T07:18:07.095 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:18:07.096 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"btime":"2024-08-31T07:17:01:405815+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:17:00.944291+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:18:07.096 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2024-08-31T07:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 -- 172.21.15.32:0/937239799 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d480b4d20 msgr2=0x7f4d480b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d480b4d20 0x7f4d480b7230 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009c90 tx=0x7f4d54005f90 comp rx=0 tx=0).stop 2024-08-31T07:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 -- 172.21.15.32:0/937239799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d64078030 msgr2=0x7f4d641a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d64078030 0x7f4d641a0c00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4d6406c660 tx=0x7f4d4c00ee50 comp rx=0 tx=0).stop 2024-08-31T07:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 -- 172.21.15.32:0/937239799 shutdown_connections 2024-08-31T07:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d480b4d20 0x7f4d480b7230 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d64114310 0x7f4d641a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d64078030 0x7f4d641a0c00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 --2- 172.21.15.32:0/937239799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d640776e0 0x7f4d641a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 -- 172.21.15.32:0/937239799 >> 172.21.15.32:0/937239799 conn(0x7f4d64101420 msgr2=0x7f4d64116340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 -- 172.21.15.32:0/937239799 shutdown_connections 2024-08-31T07:18:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:07.096+0000 7f4d69000640 1 -- 172.21.15.32:0/937239799 wait complete. 2024-08-31T07:18:07.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160430 2.64 MB/sec execute 116 sec latency 2489.577 ms 2024-08-31T07:18:07.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:07 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/937239799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:18:07.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:07 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/937239799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:18:07.971 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 152.7 --await-for 45.0' 2024-08-31T07:18:07.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 152.7 --await-for 45.0 2024-08-31T07:18:07.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:07 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/937239799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:18:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:08.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160430 2.62 MB/sec execute 117 sec latency 3489.695 ms 2024-08-31T07:18:08.850 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:08 smithi032 ceph-mon[17252]: pgmap v529: 97 pgs: 97 active+clean; 99 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 789 KiB/s wr, 151 op/s 2024-08-31T07:18:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:08 smithi073 ceph-mon[21995]: pgmap v529: 97 pgs: 97 active+clean; 99 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 789 KiB/s wr, 151 op/s 2024-08-31T07:18:08.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:08 smithi084 ceph-mon[22387]: pgmap v529: 97 pgs: 97 active+clean; 99 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 789 KiB/s wr, 151 op/s 2024-08-31T07:18:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.040+0000 7fd293200640 1 -- 172.21.15.32:0/1430258714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd294105950 msgr2=0x7fd294105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.040+0000 7fd293200640 1 --2- 172.21.15.32:0/1430258714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd294105950 0x7fd294105d60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd284009900 tx=0x7fd28402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.040+0000 7fd293200640 1 -- 172.21.15.32:0/1430258714 shutdown_connections 2024-08-31T07:18:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.040+0000 7fd293200640 1 --2- 172.21.15.32:0/1430258714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29410bbe0 0x7fd294112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.040+0000 7fd293200640 1 --2- 172.21.15.32:0/1430258714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd29410b290 0x7fd29410b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.040+0000 7fd293200640 1 --2- 172.21.15.32:0/1430258714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd294105950 0x7fd294105d60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.040+0000 7fd293200640 1 -- 172.21.15.32:0/1430258714 >> 172.21.15.32:0/1430258714 conn(0x7fd2941013e0 msgr2=0x7fd294103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.040+0000 7fd293200640 1 -- 172.21.15.32:0/1430258714 shutdown_connections 2024-08-31T07:18:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.040+0000 7fd293200640 1 -- 172.21.15.32:0/1430258714 wait complete. 2024-08-31T07:18:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.041+0000 7fd293200640 1 Processor -- start 2024-08-31T07:18:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.041+0000 7fd293200640 1 -- start start 2024-08-31T07:18:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd293200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd294105950 0x7fd29419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd293200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd29410b290 0x7fd29419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd293200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29410bbe0 0x7fd2941a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd293200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941158a0 con 0x7fd294105950 2024-08-31T07:18:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd293200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd294115720 con 0x7fd29410bbe0 2024-08-31T07:18:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd293200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd294115a20 con 0x7fd29410b290 2024-08-31T07:18:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd291e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd294105950 0x7fd29419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd291e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd294105950 0x7fd29419c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60840/0 (socket says 172.21.15.32:60840) 2024-08-31T07:18:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd291e00640 1 -- 172.21.15.32:0/316556094 learned_addr learned my addr 172.21.15.32:0/316556094 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd292800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29410bbe0 0x7fd2941a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.042+0000 7fd28be00640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd29410b290 0x7fd29419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.043+0000 7fd292800640 1 -- 172.21.15.32:0/316556094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd29410b290 msgr2=0x7fd29419c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.043+0000 7fd292800640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd29410b290 0x7fd29419c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.043+0000 7fd292800640 1 -- 172.21.15.32:0/316556094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd294105950 msgr2=0x7fd29419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.043+0000 7fd292800640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd294105950 0x7fd29419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.043+0000 7fd292800640 1 -- 172.21.15.32:0/316556094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2941a2260 con 0x7fd29410bbe0 2024-08-31T07:18:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.043+0000 7fd28be00640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd29410b290 0x7fd29419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.043+0000 7fd291e00640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd294105950 0x7fd29419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.043+0000 7fd292800640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29410bbe0 0x7fd2941a1b30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00da10 tx=0x7fd28c00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.043+0000 7fd28aa00640 1 -- 172.21.15.32:0/316556094 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00bd70 con 0x7fd29410bbe0 2024-08-31T07:18:09.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.043+0000 7fd293200640 1 -- 172.21.15.32:0/316556094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2941a2550 con 0x7fd29410bbe0 2024-08-31T07:18:09.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.044+0000 7fd28aa00640 1 -- 172.21.15.32:0/316556094 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd28c003e20 con 0x7fd29410bbe0 2024-08-31T07:18:09.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.044+0000 7fd28aa00640 1 -- 172.21.15.32:0/316556094 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c0106c0 con 0x7fd29410bbe0 2024-08-31T07:18:09.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.044+0000 7fd293200640 1 -- 172.21.15.32:0/316556094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2941a8cb0 con 0x7fd29410bbe0 2024-08-31T07:18:09.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.047+0000 7fd29f66a740 1 Processor -- start 2024-08-31T07:18:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.048+0000 7fd28aa00640 1 -- 172.21.15.32:0/316556094 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd28c004270 con 0x7fd29410bbe0 2024-08-31T07:18:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.049+0000 7fd29f66a740 1 -- start start 2024-08-31T07:18:09.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.049+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a4c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.049+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56446467c980 0x564464a4c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.049+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564464684b60 0x564464a4cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.049+0000 7fd29f66a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5644649c1530 con 0x564464684b60 2024-08-31T07:18:09.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.049+0000 7fd29f66a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5644649b0ac0 con 0x56446467c980 2024-08-31T07:18:09.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.049+0000 7fd29f66a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5644649b4a70 con 0x5644646a5840 2024-08-31T07:18:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.049+0000 7fd28be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564464684b60 0x564464a4cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.049+0000 7fd291e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56446467c980 0x564464a4c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.049+0000 7fd28be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564464684b60 0x564464a4cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60856/0 (socket says 172.21.15.32:60856) 2024-08-31T07:18:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd28be00640 1 -- 172.21.15.32:0/1180970832 learned_addr learned my addr 172.21.15.32:0/1180970832 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd28be00640 1 --2- 172.21.15.32:0/1180970832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a4c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd291e00640 1 -- 172.21.15.32:0/1180970832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 msgr2=0x564464a4c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd291e00640 1 --2- 172.21.15.32:0/1180970832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a4c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd291e00640 1 -- 172.21.15.32:0/1180970832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564464684b60 msgr2=0x564464a4cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd291e00640 1 --2- 172.21.15.32:0/1180970832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564464684b60 0x564464a4cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd291e00640 1 -- 172.21.15.32:0/1180970832 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564464963880 con 0x56446467c980 2024-08-31T07:18:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd28aa00640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2780b4c90 0x7fd2780b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd28aa00640 1 -- 172.21.15.32:0/316556094 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fd28c0e58f0 con 0x7fd29410bbe0 2024-08-31T07:18:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd291e00640 1 --2- 172.21.15.32:0/1180970832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56446467c980 0x564464a4c810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd284004770 tx=0x7fd284031520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9448bd6d15f7ec27 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.050+0000 7fd28be00640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2780b4c90 0x7fd2780b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.051+0000 7fd283000640 1 -- 172.21.15.32:0/1180970832 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284035040 con 0x56446467c980 2024-08-31T07:18:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.051+0000 7fd283000640 1 -- 172.21.15.32:0/1180970832 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd284031910 con 0x56446467c980 2024-08-31T07:18:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.051+0000 7fd283000640 1 -- 172.21.15.32:0/1180970832 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284004a00 con 0x56446467c980 2024-08-31T07:18:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.052+0000 7fd29f66a740 1 -- 172.21.15.32:0/1180970832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56446467c980 msgr2=0x564464a4c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.052+0000 7fd28be00640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2780b4c90 0x7fd2780b71a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00bfd0 tx=0x7fd27c004730 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.052+0000 7fd29f66a740 1 --2- 172.21.15.32:0/1180970832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56446467c980 0x564464a4c810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd284004770 tx=0x7fd284031520 comp rx=0 tx=0).stop 2024-08-31T07:18:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.052+0000 7fd29f66a740 1 -- 172.21.15.32:0/1180970832 shutdown_connections 2024-08-31T07:18:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.052+0000 7fd29f66a740 1 --2- 172.21.15.32:0/1180970832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564464684b60 0x564464a4cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.052+0000 7fd29f66a740 1 --2- 172.21.15.32:0/1180970832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56446467c980 0x564464a4c810 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.052+0000 7fd29f66a740 1 --2- 172.21.15.32:0/1180970832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a4c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.052+0000 7fd29f66a740 1 -- 172.21.15.32:0/1180970832 >> 172.21.15.32:0/1180970832 conn(0x564464699990 msgr2=0x564464a4a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.052+0000 7fd29f66a740 1 -- 172.21.15.32:0/1180970832 shutdown_connections 2024-08-31T07:18:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.052+0000 7fd29f66a740 1 -- 172.21.15.32:0/1180970832 wait complete. 2024-08-31T07:18:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.053+0000 7fd29f66a740 1 -- start start 2024-08-31T07:18:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.053+0000 7fd29f66a740 1 Processor -- start 2024-08-31T07:18:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.053+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56446467c980 0x564464a57ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.053+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a58440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.053+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644646a5840 0x564464a63d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.053+0000 7fd29f66a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5644648137a0 con 0x5644646a5840 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.053+0000 7fd29f66a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5644649377f0 con 0x564464684b60 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.053+0000 7fd29f66a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564464a5aa50 con 0x56446467c980 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.053+0000 7fd29f66a740 10 client.0 _ll_register_callbacks cb 0x56446495ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd291e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56446467c980 0x564464a57ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd291e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56446467c980 0x564464a57ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50864/0 (socket says 172.21.15.32:50864) 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd291e00640 1 -- 172.21.15.32:0/3612532705 learned_addr learned my addr 172.21.15.32:0/3612532705 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd292800640 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644646a5840 0x564464a63d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd28be00640 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a58440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd291e00640 1 -- 172.21.15.32:0/3612532705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 msgr2=0x564464a58440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd291e00640 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a58440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd291e00640 1 -- 172.21.15.32:0/3612532705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644646a5840 msgr2=0x564464a63d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd291e00640 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644646a5840 0x564464a63d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd291e00640 1 -- 172.21.15.32:0/3612532705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446471fb90 con 0x56446467c980 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd291e00640 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56446467c980 0x564464a57ec0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd28402abe0 tx=0x7fd284004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54219f1b06242bb3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.054+0000 7fd277400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.055+0000 7fd277400640 1 -- 172.21.15.32:0/3612532705 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28404e020 con 0x56446467c980 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.055+0000 7fd29f66a740 1 -- 172.21.15.32:0/3612532705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd284031910 con 0x56446467c980 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.055+0000 7fd277400640 1 -- 172.21.15.32:0/3612532705 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fd284003800 con 0x56446467c980 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.055+0000 7fd277400640 1 -- 172.21.15.32:0/3612532705 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd284059050 con 0x56446467c980 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.055+0000 7fd29f66a740 10 client.26653 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.055+0000 7fd29f66a740 1 -- 172.21.15.32:0/3612532705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564464a66ab0 con 0x56446467c980 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.055+0000 7fd277400640 1 -- 172.21.15.32:0/3612532705 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd284031910 con 0x56446467c980 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.056+0000 7fd277400640 1 -- 172.21.15.32:0/3612532705 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd2840467b0 con 0x56446467c980 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.056+0000 7fd29f66a740 10 client.26653 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.056+0000 7fd29f66a740 10 client.26653 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.056+0000 7fd29f66a740 20 client.26653 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:09.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.056+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5644646641c0 0x564464a69e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.056+0000 7fd29f66a740 4 client.26653 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:18:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.056+0000 7fd29f66a740 1 -- 172.21.15.32:0/3612532705 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644649c3480 con 0x5644646641c0 2024-08-31T07:18:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.056+0000 7fd28be00640 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5644646641c0 0x564464a69e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.057+0000 7fd28be00640 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5644646641c0 0x564464a69e10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e55603514e9325ca server_cookie=300baf0a5df3ad1e in_seq=0 out_seq=0 2024-08-31T07:18:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.057+0000 7fd277400640 10 client.26653 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd277400640 1 -- 172.21.15.32:0/3612532705 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644649c3480 con 0x5644646641c0 2024-08-31T07:18:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd277400640 10 client.26653 handle_command_reply: tid=0 2024-08-31T07:18:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 client.26653 shutdown 2024-08-31T07:18:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 -- 172.21.15.32:0/3612532705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56446467c980 msgr2=0x564464a57ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56446467c980 0x564464a57ec0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd28402abe0 tx=0x7fd284004270 comp rx=0 tx=0).stop 2024-08-31T07:18:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 -- 172.21.15.32:0/3612532705 shutdown_connections 2024-08-31T07:18:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5644646641c0 0x564464a69e10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644646a5840 0x564464a63d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a58440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3612532705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56446467c980 0x564464a57ec0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd277400640 0 client.26653 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 -- 172.21.15.32:0/3612532705 >> 172.21.15.32:0/3612532705 conn(0x564464699990 msgr2=0x564464a4f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 -- 172.21.15.32:0/3612532705 shutdown_connections 2024-08-31T07:18:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.058+0000 7fd29f66a740 1 -- 172.21.15.32:0/3612532705 wait complete. 2024-08-31T07:18:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.059+0000 7fd29f66a740 20 client.26653 trim_cache size 0 max 16384 2024-08-31T07:18:09.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.077+0000 7fd29f66a740 1 Processor -- start 2024-08-31T07:18:09.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.077+0000 7fd29f66a740 1 -- start start 2024-08-31T07:18:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a49a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a49f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a54eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd29f66a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941b1300 con 0x56446467c980 2024-08-31T07:18:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd29f66a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941b2e40 con 0x564464684b60 2024-08-31T07:18:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd29f66a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941b19d0 con 0x5644646a5840 2024-08-31T07:18:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd291e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a49a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd291e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a49a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60880/0 (socket says 172.21.15.32:60880) 2024-08-31T07:18:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd291e00640 1 -- 172.21.15.32:0/670986247 learned_addr learned my addr 172.21.15.32:0/670986247 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd292800640 1 --2- 172.21.15.32:0/670986247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a54eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd28be00640 1 --2- 172.21.15.32:0/670986247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a49f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd291e00640 1 -- 172.21.15.32:0/670986247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 msgr2=0x564464a54eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd291e00640 1 --2- 172.21.15.32:0/670986247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a54eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd291e00640 1 -- 172.21.15.32:0/670986247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 msgr2=0x564464a49f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd291e00640 1 --2- 172.21.15.32:0/670986247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a49f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd291e00640 1 -- 172.21.15.32:0/670986247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564464a58580 con 0x56446467c980 2024-08-31T07:18:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.078+0000 7fd28be00640 1 --2- 172.21.15.32:0/670986247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a49f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:18:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.079+0000 7fd291e00640 1 --2- 172.21.15.32:0/670986247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a49a30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd284031440 tx=0x7fd284005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dcdcd40cd9aeed server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.079+0000 7fd282600640 1 -- 172.21.15.32:0/670986247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284062070 con 0x56446467c980 2024-08-31T07:18:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.079+0000 7fd282600640 1 -- 172.21.15.32:0/670986247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd284046900 con 0x56446467c980 2024-08-31T07:18:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.079+0000 7fd282600640 1 -- 172.21.15.32:0/670986247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28404e070 con 0x56446467c980 2024-08-31T07:18:09.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.080+0000 7fd29f66a740 1 -- 172.21.15.32:0/670986247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 msgr2=0x564464a49a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.080+0000 7fd29f66a740 1 --2- 172.21.15.32:0/670986247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a49a30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd284031440 tx=0x7fd284005fd0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.080+0000 7fd29f66a740 1 -- 172.21.15.32:0/670986247 shutdown_connections 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.080+0000 7fd29f66a740 1 --2- 172.21.15.32:0/670986247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a54eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.080+0000 7fd29f66a740 1 --2- 172.21.15.32:0/670986247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a49f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.080+0000 7fd29f66a740 1 --2- 172.21.15.32:0/670986247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a49a30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.080+0000 7fd29f66a740 1 -- 172.21.15.32:0/670986247 >> 172.21.15.32:0/670986247 conn(0x564464699990 msgr2=0x564464a56540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.080+0000 7fd29f66a740 1 -- 172.21.15.32:0/670986247 shutdown_connections 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.080+0000 7fd29f66a740 1 -- 172.21.15.32:0/670986247 wait complete. 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.081+0000 7fd29f66a740 1 -- start start 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.081+0000 7fd29f66a740 1 Processor -- start 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.081+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a6d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.081+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a69940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.081+0000 7fd29f66a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a69e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.081+0000 7fd29f66a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941b2770 con 0x56446467c980 2024-08-31T07:18:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.081+0000 7fd29f66a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941b20a0 con 0x564464684b60 2024-08-31T07:18:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.081+0000 7fd29f66a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941af300 con 0x5644646a5840 2024-08-31T07:18:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd29f66a740 10 client.0 _ll_register_callbacks cb 0x7fd284032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd291e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a6d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd292800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a69e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd28be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a69940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd291e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a6d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60882/0 (socket says 172.21.15.32:60882) 2024-08-31T07:18:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd291e00640 1 -- 172.21.15.32:0/3810487775 learned_addr learned my addr 172.21.15.32:0/3810487775 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd28be00640 1 -- 172.21.15.32:0/3810487775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 msgr2=0x564464a69e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd28be00640 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a69e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd28be00640 1 -- 172.21.15.32:0/3810487775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 msgr2=0x564464a6d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd28be00640 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a6d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd28be00640 1 -- 172.21.15.32:0/3810487775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564464a6a470 con 0x564464684b60 2024-08-31T07:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd291e00640 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a6d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.082+0000 7fd292800640 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a69e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.083+0000 7fd28be00640 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a69940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x564464937820 tx=0x7fd27c00b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78860257e9f1794f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.083+0000 7fd277400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.083+0000 7fd277400640 1 -- 172.21.15.32:0/3810487775 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c022620 con 0x564464684b60 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.083+0000 7fd29f66a740 1 -- 172.21.15.32:0/3810487775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd284046900 con 0x564464684b60 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.083+0000 7fd277400640 1 -- 172.21.15.32:0/3810487775 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fd27c0227c0 con 0x564464684b60 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.083+0000 7fd277400640 1 -- 172.21.15.32:0/3810487775 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd27c03b050 con 0x564464684b60 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.083+0000 7fd29f66a740 10 client.26642 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.083+0000 7fd29f66a740 1 -- 172.21.15.32:0/3810487775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564464ad5310 con 0x564464684b60 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.084+0000 7fd277400640 1 -- 172.21.15.32:0/3810487775 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd284046900 con 0x564464684b60 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.084+0000 7fd277400640 1 -- 172.21.15.32:0/3810487775 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fd27c034b30 con 0x564464684b60 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.084+0000 7fd29f66a740 10 client.26642 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.084+0000 7fd29f66a740 10 client.26642 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.084+0000 7fd29f66a740 20 client.26642 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.084+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564464639570 0x564464ad7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.084+0000 7fd29f66a740 4 client.26642 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 152.7, "await_for": 45.0}] 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.084+0000 7fd29f66a740 1 -- 172.21.15.32:0/3810487775 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 152.7, "await_for": 45.0}) -- 0x5644649bef20 con 0x564464639570 2024-08-31T07:18:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.084+0000 7fd291e00640 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564464639570 0x564464ad7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.085+0000 7fd291e00640 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564464639570 0x564464ad7b90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d3bae015d5edd8 server_cookie=e9094f96a06ec62d in_seq=0 out_seq=0 2024-08-31T07:18:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.085+0000 7fd277400640 10 client.26642 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:09.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd277400640 1 -- 172.21.15.32:0/3810487775 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5644649bef20 con 0x564464639570 2024-08-31T07:18:09.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd277400640 10 client.26642 handle_command_reply: tid=0 2024-08-31T07:18:09.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 client.26642 shutdown 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 -- 172.21.15.32:0/3810487775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 msgr2=0x564464a69940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a69940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x564464937820 tx=0x7fd27c00b390 comp rx=0 tx=0).stop 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 -- 172.21.15.32:0/3810487775 shutdown_connections 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564464639570 0x564464ad7b90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644646a5840 0x564464a69e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564464684b60 0x564464a69940 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 --2- 172.21.15.32:0/3810487775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56446467c980 0x564464a6d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 -- 172.21.15.32:0/3810487775 >> 172.21.15.32:0/3810487775 conn(0x564464699990 msgr2=0x564464a49a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd277400640 0 client.26642 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 -- 172.21.15.32:0/3810487775 shutdown_connections 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 1 -- 172.21.15.32:0/3810487775 wait complete. 2024-08-31T07:18:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.174+0000 7fd29f66a740 20 client.26642 trim_cache size 0 max 16384 2024-08-31T07:18:09.179 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:18:09.179 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 33, 2024-08-31T07:18:09.179 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:18:09.179 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 18, 2024-08-31T07:18:09.179 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:18:09.179 INFO:teuthology.orchestra.run.smithi032.stdout: "601ae51e": { 2024-08-31T07:18:09.179 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 18, 2024-08-31T07:18:09.179 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:18:09.179 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:18:09.179 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 152.7, 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:09.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:09.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:09.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:09.181 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 -- 172.21.15.32:0/316556094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2780b4c90 msgr2=0x7fd2780b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2780b4c90 0x7fd2780b71a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00bfd0 tx=0x7fd27c004730 comp rx=0 tx=0).stop 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 -- 172.21.15.32:0/316556094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29410bbe0 msgr2=0x7fd2941a1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29410bbe0 0x7fd2941a1b30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00da10 tx=0x7fd28c00ded0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 -- 172.21.15.32:0/316556094 shutdown_connections 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2780b4c90 0x7fd2780b71a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd29410bbe0 0x7fd2941a1b30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd29410b290 0x7fd29419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 --2- 172.21.15.32:0/316556094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd294105950 0x7fd29419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 -- 172.21.15.32:0/316556094 >> 172.21.15.32:0/316556094 conn(0x7fd2941013e0 msgr2=0x7fd294103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:09.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 -- 172.21.15.32:0/316556094 shutdown_connections 2024-08-31T07:18:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:09.175+0000 7fd293200640 1 -- 172.21.15.32:0/316556094 wait complete. 2024-08-31T07:18:09.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:18:09.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.59 MB/sec execute 118 sec latency 3895.867 ms 2024-08-31T07:18:10.075 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '601ae51e', quiesce took 4.3 seconds. Will release after: 28.400000000000002 2024-08-31T07:18:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:10.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:10 smithi032 ceph-mon[17252]: pgmap v530: 97 pgs: 97 active+clean; 99 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 617 KiB/s wr, 124 op/s 2024-08-31T07:18:10.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.57 MB/sec execute 119 sec latency 1593.926 ms 2024-08-31T07:18:10.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.863+0000 7f2711600640 1 -- 172.21.15.32:0/3955352847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f270c075e40 msgr2=0x7f270c10f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.863+0000 7f2711600640 1 --2- 172.21.15.32:0/3955352847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f270c075e40 0x7f270c10f7b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009900 tx=0x7f26f802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.863+0000 7f2711600640 1 -- 172.21.15.32:0/3955352847 shutdown_connections 2024-08-31T07:18:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.863+0000 7f2711600640 1 --2- 172.21.15.32:0/3955352847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c10fcf0 0x7f270c112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.863+0000 7f2711600640 1 --2- 172.21.15.32:0/3955352847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f270c075e40 0x7f270c10f7b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.863+0000 7f2711600640 1 --2- 172.21.15.32:0/3955352847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f270c0754f0 0x7f270c075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.863+0000 7f2711600640 1 -- 172.21.15.32:0/3955352847 >> 172.21.15.32:0/3955352847 conn(0x7f270c0ff180 msgr2=0x7f270c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.863+0000 7f2711600640 1 -- 172.21.15.32:0/3955352847 shutdown_connections 2024-08-31T07:18:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.863+0000 7f2711600640 1 -- 172.21.15.32:0/3955352847 wait complete. 2024-08-31T07:18:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.864+0000 7f2711600640 1 Processor -- start 2024-08-31T07:18:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.864+0000 7f2711600640 1 -- start start 2024-08-31T07:18:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f2711600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c0754f0 0x7f270c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f2711600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f270c075e40 0x7f270c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f2711600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f270c10fcf0 0x7f270c1a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f2711600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f270c1158b0 con 0x7f270c10fcf0 2024-08-31T07:18:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f2711600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f270c115730 con 0x7f270c0754f0 2024-08-31T07:18:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f2711600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f270c115a30 con 0x7f270c075e40 2024-08-31T07:18:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f270be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c0754f0 0x7f270c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f2710c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f270c10fcf0 0x7f270c1a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f2710c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f270c10fcf0 0x7f270c1a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60910/0 (socket says 172.21.15.32:60910) 2024-08-31T07:18:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f2710c00640 1 -- 172.21.15.32:0/39507540 learned_addr learned my addr 172.21.15.32:0/39507540 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.865+0000 7f270b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f270c075e40 0x7f270c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.866+0000 7f2710c00640 1 -- 172.21.15.32:0/39507540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f270c075e40 msgr2=0x7f270c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.866+0000 7f2710c00640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f270c075e40 0x7f270c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.866+0000 7f2710c00640 1 -- 172.21.15.32:0/39507540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c0754f0 msgr2=0x7f270c19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.866+0000 7f2710c00640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c0754f0 0x7f270c19c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.866+0000 7f2710c00640 1 -- 172.21.15.32:0/39507540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270c1a2230 con 0x7f270c10fcf0 2024-08-31T07:18:10.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.866+0000 7f2710c00640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f270c10fcf0 0x7f270c1a1b00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f270400eb10 tx=0x7f270400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:10.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.866+0000 7f2708c00640 1 -- 172.21.15.32:0/39507540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704004b20 con 0x7f270c10fcf0 2024-08-31T07:18:10.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.866+0000 7f2708c00640 1 -- 172.21.15.32:0/39507540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2704004cc0 con 0x7f270c10fcf0 2024-08-31T07:18:10.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.866+0000 7f2711600640 1 -- 172.21.15.32:0/39507540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270c1a2520 con 0x7f270c10fcf0 2024-08-31T07:18:10.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.866+0000 7f2708c00640 1 -- 172.21.15.32:0/39507540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704010670 con 0x7f270c10fcf0 2024-08-31T07:18:10.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.867+0000 7f2711600640 1 -- 172.21.15.32:0/39507540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270c1a8c80 con 0x7f270c10fcf0 2024-08-31T07:18:10.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.870+0000 7f2719845740 1 Processor -- start 2024-08-31T07:18:10.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.870+0000 7f2708c00640 1 -- 172.21.15.32:0/39507540 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2704005000 con 0x7f270c10fcf0 2024-08-31T07:18:10.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.870+0000 7f2719845740 1 -- start start 2024-08-31T07:18:10.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f2719845740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb4eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f2719845740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7801b0 0x562defb4f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f2719845740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def788390 0x562defb4f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f2719845740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562defac5770 con 0x562def7a9070 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f2719845740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562defab4d00 con 0x562def7801b0 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f2719845740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562defab8cb0 con 0x562def788390 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f270b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb4eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f270b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb4eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60914/0 (socket says 172.21.15.32:60914) 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f270b400640 1 -- 172.21.15.32:0/2948930025 learned_addr learned my addr 172.21.15.32:0/2948930025 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f270be00640 1 --2- 172.21.15.32:0/2948930025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7801b0 0x562defb4f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.871+0000 7f270b400640 1 --2- 172.21.15.32:0/2948930025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def788390 0x562defb4f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f270be00640 1 -- 172.21.15.32:0/2948930025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def788390 msgr2=0x562defb4f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:10.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f270be00640 1 --2- 172.21.15.32:0/2948930025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def788390 0x562defb4f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f270be00640 1 -- 172.21.15.32:0/2948930025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 msgr2=0x562defb4eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:10.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f270be00640 1 --2- 172.21.15.32:0/2948930025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb4eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f270be00640 1 -- 172.21.15.32:0/2948930025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562def814650 con 0x562def7801b0 2024-08-31T07:18:10.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f270b400640 1 --2- 172.21.15.32:0/2948930025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb4eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:10.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f270be00640 1 --2- 172.21.15.32:0/2948930025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7801b0 0x562defb4f060 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f26fc005fd0 tx=0x7f26fc0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de4e81966f3a87 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:10.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f2708c00640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26f00b4ca0 0x7f26f00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f2701200640 1 -- 172.21.15.32:0/2948930025 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc01c680 con 0x562def7801b0 2024-08-31T07:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f2701200640 1 -- 172.21.15.32:0/2948930025 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc00be20 con 0x562def7801b0 2024-08-31T07:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f2701200640 1 -- 172.21.15.32:0/2948930025 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc011c00 con 0x562def7801b0 2024-08-31T07:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.872+0000 7f2708c00640 1 -- 172.21.15.32:0/39507540 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f27040e5a40 con 0x7f270c10fcf0 2024-08-31T07:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.873+0000 7f270b400640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26f00b4ca0 0x7f26f00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.873+0000 7f2719845740 1 -- 172.21.15.32:0/2948930025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7801b0 msgr2=0x562defb4f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.873+0000 7f270b400640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26f00b4ca0 0x7f26f00b71b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f26f8000c00 tx=0x7f26f8005d50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.873+0000 7f2719845740 1 --2- 172.21.15.32:0/2948930025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7801b0 0x562defb4f060 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f26fc005fd0 tx=0x7f26fc0107a0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.874+0000 7f2719845740 1 -- 172.21.15.32:0/2948930025 shutdown_connections 2024-08-31T07:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.874+0000 7f2719845740 1 --2- 172.21.15.32:0/2948930025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def788390 0x562defb4f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.874+0000 7f2719845740 1 --2- 172.21.15.32:0/2948930025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7801b0 0x562defb4f060 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.874+0000 7f2719845740 1 --2- 172.21.15.32:0/2948930025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb4eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.874+0000 7f2719845740 1 -- 172.21.15.32:0/2948930025 >> 172.21.15.32:0/2948930025 conn(0x562def79d460 msgr2=0x562defb4ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.874+0000 7f2719845740 1 -- 172.21.15.32:0/2948930025 shutdown_connections 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.874+0000 7f2719845740 1 -- 172.21.15.32:0/2948930025 wait complete. 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.874+0000 7f2719845740 1 -- start start 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.874+0000 7f2719845740 1 Processor -- start 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f2719845740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7801b0 0x562defb5a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f2719845740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 0x562defb5ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f2719845740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb66620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f2719845740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562def917700 con 0x562def7a9070 2024-08-31T07:18:10.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f2719845740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562defa3bcb0 con 0x562def788390 2024-08-31T07:18:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f2719845740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562defb5d300 con 0x562def7801b0 2024-08-31T07:18:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f2719845740 10 client.0 _ll_register_callbacks cb 0x562defa61430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f2710c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb66620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f2710c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb66620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60918/0 (socket says 172.21.15.32:60918) 2024-08-31T07:18:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f2710c00640 1 -- 172.21.15.32:0/3014887571 learned_addr learned my addr 172.21.15.32:0/3014887571 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f270b400640 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 0x562defb5ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.875+0000 7f270be00640 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7801b0 0x562defb5a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f2710c00640 1 -- 172.21.15.32:0/3014887571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7801b0 msgr2=0x562defb5a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f2710c00640 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7801b0 0x562defb5a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f2710c00640 1 -- 172.21.15.32:0/3014887571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 msgr2=0x562defb5ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f2710c00640 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 0x562defb5ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f2710c00640 1 -- 172.21.15.32:0/3014887571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562defa7c870 con 0x562def7a9070 2024-08-31T07:18:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f2710c00640 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb66620 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f27040a8230 tx=0x7f27040abb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51803a53aceaf0d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f26f5600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:18:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f26f5600640 1 -- 172.21.15.32:0/3014887571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27040ac8b0 con 0x562def7a9070 2024-08-31T07:18:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f26f5600640 1 -- 172.21.15.32:0/3014887571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f27040b5250 con 0x562def7a9070 2024-08-31T07:18:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f2719845740 1 -- 172.21.15.32:0/3014887571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26fc00be20 con 0x562def7a9070 2024-08-31T07:18:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.876+0000 7f26f5600640 1 -- 172.21.15.32:0/3014887571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f27040ac1b0 con 0x562def7a9070 2024-08-31T07:18:10.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.877+0000 7f2719845740 10 client.16641 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:10.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.877+0000 7f2719845740 1 -- 172.21.15.32:0/3014887571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562defb69350 con 0x562def7a9070 2024-08-31T07:18:10.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.877+0000 7f26f5600640 1 -- 172.21.15.32:0/3014887571 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc00be20 con 0x562def7a9070 2024-08-31T07:18:10.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.877+0000 7f26f5600640 1 -- 172.21.15.32:0/3014887571 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f27040d1070 con 0x562def7a9070 2024-08-31T07:18:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.877+0000 7f2719845740 10 client.16641 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.877+0000 7f2719845740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:18:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.877+0000 7f2719845740 20 client.16641 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.877+0000 7f2719845740 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562def7679f0 0x562defb6c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.877+0000 7f2719845740 4 client.16641 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:18:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.877+0000 7f2719845740 1 -- 172.21.15.32:0/3014887571 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562defac76c0 con 0x562def7679f0 2024-08-31T07:18:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.878+0000 7f270be00640 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562def7679f0 0x562defb6c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.878+0000 7f270be00640 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562def7679f0 0x562defb6c8e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc34d2082ec28253 server_cookie=e2faaceae2f5b63e in_seq=0 out_seq=0 2024-08-31T07:18:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.878+0000 7f26f5600640 10 client.16641 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:10.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.879+0000 7f26f5600640 1 -- 172.21.15.32:0/3014887571 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562defac76c0 con 0x562def7679f0 2024-08-31T07:18:10.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.879+0000 7f26f5600640 10 client.16641 handle_command_reply: tid=0 2024-08-31T07:18:10.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.879+0000 7f2719845740 1 client.16641 shutdown 2024-08-31T07:18:10.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.879+0000 7f2719845740 1 -- 172.21.15.32:0/3014887571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 msgr2=0x562defb66620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.879+0000 7f2719845740 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb66620 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f27040a8230 tx=0x7f27040abb00 comp rx=0 tx=0).stop 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.880+0000 7f2719845740 1 -- 172.21.15.32:0/3014887571 shutdown_connections 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.880+0000 7f2719845740 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562def7679f0 0x562defb6c8e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.880+0000 7f2719845740 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7a9070 0x562defb66620 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.880+0000 7f2719845740 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 0x562defb5ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.880+0000 7f2719845740 1 --2- 172.21.15.32:0/3014887571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7801b0 0x562defb5a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.880+0000 7f2719845740 1 -- 172.21.15.32:0/3014887571 >> 172.21.15.32:0/3014887571 conn(0x562def79d460 msgr2=0x562defb51d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.880+0000 7f26f5600640 0 client.16641 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.880+0000 7f2719845740 1 -- 172.21.15.32:0/3014887571 shutdown_connections 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.880+0000 7f2719845740 1 -- 172.21.15.32:0/3014887571 wait complete. 2024-08-31T07:18:10.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.880+0000 7f2719845740 20 client.16641 trim_cache size 0 max 16384 2024-08-31T07:18:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.898+0000 7f2719845740 1 Processor -- start 2024-08-31T07:18:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.898+0000 7f2719845740 1 -- start start 2024-08-31T07:18:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f2719845740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7801b0 0x562defb66520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f2719845740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 0x562defb50260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f2719845740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7a9070 0x562defb576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f2719845740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f270c1b1310 con 0x562def7801b0 2024-08-31T07:18:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f2719845740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f270c1b2e50 con 0x562def788390 2024-08-31T07:18:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f2719845740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f270c1b19e0 con 0x562def7a9070 2024-08-31T07:18:10.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f270be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7801b0 0x562defb66520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f270be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7801b0 0x562defb66520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60932/0 (socket says 172.21.15.32:60932) 2024-08-31T07:18:10.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f270be00640 1 -- 172.21.15.32:0/3112810998 learned_addr learned my addr 172.21.15.32:0/3112810998 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f270b400640 1 --2- 172.21.15.32:0/3112810998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 0x562defb50260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.899+0000 7f2710c00640 1 --2- 172.21.15.32:0/3112810998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7a9070 0x562defb576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.900+0000 7f270be00640 1 -- 172.21.15.32:0/3112810998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7a9070 msgr2=0x562defb576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.900+0000 7f270be00640 1 --2- 172.21.15.32:0/3112810998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7a9070 0x562defb576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.900+0000 7f270be00640 1 -- 172.21.15.32:0/3112810998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 msgr2=0x562defb50260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.900+0000 7f270be00640 1 --2- 172.21.15.32:0/3112810998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 0x562defb50260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.900+0000 7f270be00640 1 -- 172.21.15.32:0/3112810998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562defb5b7a0 con 0x562def7801b0 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.900+0000 7f270b400640 1 --2- 172.21.15.32:0/3112810998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 0x562defb50260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.900+0000 7f270be00640 1 --2- 172.21.15.32:0/3112810998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7801b0 0x562defb66520 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x562defb4d7a0 tx=0x7f26fc0117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aabf25832b8bab0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.900+0000 7f26f7e00640 1 -- 172.21.15.32:0/3112810998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004020 con 0x562def7801b0 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.900+0000 7f26f7e00640 1 -- 172.21.15.32:0/3112810998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc0054f0 con 0x562def7801b0 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.900+0000 7f26f7e00640 1 -- 172.21.15.32:0/3112810998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc011a70 con 0x562def7801b0 2024-08-31T07:18:10.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.901+0000 7f2719845740 1 -- 172.21.15.32:0/3112810998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7801b0 msgr2=0x562defb66520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.901+0000 7f2719845740 1 --2- 172.21.15.32:0/3112810998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7801b0 0x562defb66520 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x562defb4d7a0 tx=0x7f26fc0117a0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.901+0000 7f2719845740 1 -- 172.21.15.32:0/3112810998 shutdown_connections 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.901+0000 7f2719845740 1 --2- 172.21.15.32:0/3112810998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7a9070 0x562defb576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.901+0000 7f2719845740 1 --2- 172.21.15.32:0/3112810998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def788390 0x562defb50260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.901+0000 7f2719845740 1 --2- 172.21.15.32:0/3112810998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def7801b0 0x562defb66520 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.901+0000 7f2719845740 1 -- 172.21.15.32:0/3112810998 >> 172.21.15.32:0/3112810998 conn(0x562def79d460 msgr2=0x562defb4fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.901+0000 7f2719845740 1 -- 172.21.15.32:0/3112810998 shutdown_connections 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.901+0000 7f2719845740 1 -- 172.21.15.32:0/3112810998 wait complete. 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.902+0000 7f2719845740 1 -- start start 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.902+0000 7f2719845740 1 Processor -- start 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.902+0000 7f2719845740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7801b0 0x562defb5e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.902+0000 7f2719845740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def788390 0x562defb5ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.902+0000 7f2719845740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7a9070 0x562defbd62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.902+0000 7f2719845740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f270c1b2780 con 0x562def788390 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.902+0000 7f2719845740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f270c1b20b0 con 0x562def7a9070 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.902+0000 7f2719845740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f270c1af2f0 con 0x562def7801b0 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f2719845740 10 client.0 _ll_register_callbacks cb 0x7f27040abf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f270b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def788390 0x562defb5ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f270b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def788390 0x562defb5ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60938/0 (socket says 172.21.15.32:60938) 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f270b400640 1 -- 172.21.15.32:0/1737780199 learned_addr learned my addr 172.21.15.32:0/1737780199 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f270be00640 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7801b0 0x562defb5e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f2710c00640 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7a9070 0x562defbd62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f270b400640 1 -- 172.21.15.32:0/1737780199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7801b0 msgr2=0x562defb5e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f270b400640 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7801b0 0x562defb5e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f270b400640 1 -- 172.21.15.32:0/1737780199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7a9070 msgr2=0x562defbd62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f270b400640 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7a9070 0x562defbd62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f270b400640 1 -- 172.21.15.32:0/1737780199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562defb5f5b0 con 0x562def788390 2024-08-31T07:18:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.903+0000 7f2710c00640 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7a9070 0x562defbd62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:18:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.904+0000 7f270b400640 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def788390 0x562defb5ed60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f26f80096a0 tx=0x7f26f802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cb01fa7cb48472 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.904+0000 7f26f5600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:18:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.904+0000 7f26f5600640 1 -- 172.21.15.32:0/1737780199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f805a060 con 0x562def788390 2024-08-31T07:18:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.904+0000 7f26f5600640 1 -- 172.21.15.32:0/1737780199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f26f80035f0 con 0x562def788390 2024-08-31T07:18:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.904+0000 7f2719845740 1 -- 172.21.15.32:0/1737780199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26fc0054f0 con 0x562def788390 2024-08-31T07:18:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.904+0000 7f26f5600640 1 -- 172.21.15.32:0/1737780199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f26f8035040 con 0x562def788390 2024-08-31T07:18:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.904+0000 7f2719845740 10 client.16647 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.904+0000 7f2719845740 1 -- 172.21.15.32:0/1737780199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562defbd7800 con 0x562def788390 2024-08-31T07:18:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.904+0000 7f26f5600640 1 -- 172.21.15.32:0/1737780199 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc0054f0 con 0x562def788390 2024-08-31T07:18:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.905+0000 7f26f5600640 1 -- 172.21.15.32:0/1737780199 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f26f8054070 con 0x562def788390 2024-08-31T07:18:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.905+0000 7f2719845740 10 client.16647 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.905+0000 7f2719845740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:18:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.905+0000 7f2719845740 20 client.16647 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.905+0000 7f2719845740 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562def73cda0 0x562defbd9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.905+0000 7f2719845740 4 client.16647 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:18:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.905+0000 7f2719845740 1 -- 172.21.15.32:0/1737780199 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562defac3160 con 0x562def73cda0 2024-08-31T07:18:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.905+0000 7f270be00640 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562def73cda0 0x562defbd9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f270be00640 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562def73cda0 0x562defbd9ff0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aafdfbe5656de3f server_cookie=ddbf8e031f0e4b76 in_seq=0 out_seq=0 2024-08-31T07:18:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f26f5600640 10 client.16647 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f26f5600640 1 -- 172.21.15.32:0/1737780199 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x562defac3160 con 0x562def73cda0 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f26f5600640 10 client.16647 handle_command_reply: tid=0 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f2719845740 1 client.16647 shutdown 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f2719845740 1 -- 172.21.15.32:0/1737780199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def788390 msgr2=0x562defb5ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f2719845740 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def788390 0x562defb5ed60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f26f80096a0 tx=0x7f26f802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f2719845740 1 -- 172.21.15.32:0/1737780199 shutdown_connections 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f2719845740 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562def73cda0 0x562defbd9ff0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f2719845740 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562def7a9070 0x562defbd62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f2719845740 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562def788390 0x562defb5ed60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f2719845740 1 --2- 172.21.15.32:0/1737780199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562def7801b0 0x562defb5e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f2719845740 1 -- 172.21.15.32:0/1737780199 >> 172.21.15.32:0/1737780199 conn(0x562def79d460 msgr2=0x562defb4fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.906+0000 7f26f5600640 0 client.16647 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.907+0000 7f2719845740 1 -- 172.21.15.32:0/1737780199 shutdown_connections 2024-08-31T07:18:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.907+0000 7f2719845740 1 -- 172.21.15.32:0/1737780199 wait complete. 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.907+0000 7f2719845740 20 client.16647 trim_cache size 0 max 16384 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:10.914 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:10.915 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:10.916 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:10.917 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:10.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:10.917 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:18:10.917 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:18:10.917 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:10.917 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:10.917 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:10.917 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:10.917 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:10.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:10.918 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:18:10.918 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:18:10.918 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:10.918 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:10.918 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:10.918 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:10.918 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:10.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:10.919 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:18:10.919 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:18:10.919 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:10.919 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:10.919 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:10.919 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:10.919 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:10.919 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:10.919 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:10.919 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.907+0000 7f2711600640 1 -- 172.21.15.32:0/39507540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26f00b4ca0 msgr2=0x7f26f00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.907+0000 7f2711600640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26f00b4ca0 0x7f26f00b71b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f26f8000c00 tx=0x7f26f8005d50 comp rx=0 tx=0).stop 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.907+0000 7f2711600640 1 -- 172.21.15.32:0/39507540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f270c10fcf0 msgr2=0x7f270c1a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.907+0000 7f2711600640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f270c10fcf0 0x7f270c1a1b00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f270400eb10 tx=0x7f270400c2d0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.908+0000 7f2711600640 1 -- 172.21.15.32:0/39507540 shutdown_connections 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.908+0000 7f2711600640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f26f00b4ca0 0x7f26f00b71b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.908+0000 7f2711600640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f270c10fcf0 0x7f270c1a1b00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.908+0000 7f2711600640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f270c075e40 0x7f270c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.908+0000 7f2711600640 1 --2- 172.21.15.32:0/39507540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c0754f0 0x7f270c19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.908+0000 7f2711600640 1 -- 172.21.15.32:0/39507540 >> 172.21.15.32:0/39507540 conn(0x7f270c0ff180 msgr2=0x7f270c109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:10.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.908+0000 7f2711600640 1 -- 172.21.15.32:0/39507540 shutdown_connections 2024-08-31T07:18:10.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:10.908+0000 7f2711600640 1 -- 172.21.15.32:0/39507540 wait complete. 2024-08-31T07:18:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:10 smithi073 ceph-mon[21995]: pgmap v530: 97 pgs: 97 active+clean; 99 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 617 KiB/s wr, 124 op/s 2024-08-31T07:18:10.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:10 smithi084 ceph-mon[22387]: pgmap v530: 97 pgs: 97 active+clean; 99 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 617 KiB/s wr, 124 op/s 2024-08-31T07:18:11.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.55 MB/sec execute 120 sec latency 2594.054 ms 2024-08-31T07:18:11.957 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:18:11.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:18:12.341 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:12.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.53 MB/sec execute 121 sec latency 3594.126 ms 2024-08-31T07:18:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:12 smithi032 ceph-mon[17252]: pgmap v531: 97 pgs: 97 active+clean; 100 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 526 KiB/s wr, 111 op/s 2024-08-31T07:18:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:12 smithi073 ceph-mon[21995]: pgmap v531: 97 pgs: 97 active+clean; 100 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 526 KiB/s wr, 111 op/s 2024-08-31T07:18:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:12 smithi084 ceph-mon[22387]: pgmap v531: 97 pgs: 97 active+clean; 100 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 526 KiB/s wr, 111 op/s 2024-08-31T07:18:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.045+0000 7fccfda00640 1 -- 172.21.15.32:0/3099953978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccf810b290 msgr2=0x7fccf810b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.045+0000 7fccfda00640 1 --2- 172.21.15.32:0/3099953978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccf810b290 0x7fccf810b6a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009900 tx=0x7fcce002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.045+0000 7fccfda00640 1 -- 172.21.15.32:0/3099953978 shutdown_connections 2024-08-31T07:18:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.045+0000 7fccfda00640 1 --2- 172.21.15.32:0/3099953978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccf810bbe0 0x7fccf8112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.045+0000 7fccfda00640 1 --2- 172.21.15.32:0/3099953978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccf810b290 0x7fccf810b6a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.045+0000 7fccfda00640 1 --2- 172.21.15.32:0/3099953978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccf81059e0 0x7fccf8105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.045+0000 7fccfda00640 1 -- 172.21.15.32:0/3099953978 >> 172.21.15.32:0/3099953978 conn(0x7fccf8101430 msgr2=0x7fccf8103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.046+0000 7fccfda00640 1 -- 172.21.15.32:0/3099953978 shutdown_connections 2024-08-31T07:18:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.046+0000 7fccfda00640 1 -- 172.21.15.32:0/3099953978 wait complete. 2024-08-31T07:18:13.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.047+0000 7fccfda00640 1 Processor -- start 2024-08-31T07:18:13.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.047+0000 7fccfda00640 1 -- start start 2024-08-31T07:18:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccfda00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccf81059e0 0x7fccf819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccfda00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccf810b290 0x7fccf819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccfda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccf810bbe0 0x7fccf81a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccfda00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81158a0 con 0x7fccf810bbe0 2024-08-31T07:18:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccfda00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8115720 con 0x7fccf81059e0 2024-08-31T07:18:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccfda00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8115a20 con 0x7fccf810b290 2024-08-31T07:18:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccfd000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccf810bbe0 0x7fccf81a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccf81059e0 0x7fccf819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccfd000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccf810bbe0 0x7fccf81a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60956/0 (socket says 172.21.15.32:60956) 2024-08-31T07:18:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccf81059e0 0x7fccf819c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:33708/0 (socket says 172.21.15.32:33708) 2024-08-31T07:18:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccfd000640 1 -- 172.21.15.32:0/3823405067 learned_addr learned my addr 172.21.15.32:0/3823405067 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.048+0000 7fccefe00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccf810b290 0x7fccf819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.049+0000 7fccf7e00640 1 -- 172.21.15.32:0/3823405067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccf810b290 msgr2=0x7fccf819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.049+0000 7fccf7e00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccf810b290 0x7fccf819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.049+0000 7fccf7e00640 1 -- 172.21.15.32:0/3823405067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccf810bbe0 msgr2=0x7fccf81a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.049+0000 7fccf7e00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccf810bbe0 0x7fccf81a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.049+0000 7fccf7e00640 1 -- 172.21.15.32:0/3823405067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf81a21e0 con 0x7fccf81059e0 2024-08-31T07:18:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.049+0000 7fccf7e00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccf81059e0 0x7fccf819c270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcce800e9f0 tx=0x7fcce800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.049+0000 7fccf5600640 1 -- 172.21.15.32:0/3823405067 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce800ccc0 con 0x7fccf81059e0 2024-08-31T07:18:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.049+0000 7fccfda00640 1 -- 172.21.15.32:0/3823405067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf81a24d0 con 0x7fccf81059e0 2024-08-31T07:18:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.050+0000 7fccf5600640 1 -- 172.21.15.32:0/3823405067 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce8005020 con 0x7fccf81059e0 2024-08-31T07:18:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.050+0000 7fccf5600640 1 -- 172.21.15.32:0/3823405067 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8004a00 con 0x7fccf81059e0 2024-08-31T07:18:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.050+0000 7fccfda00640 1 -- 172.21.15.32:0/3823405067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf81a8c30 con 0x7fccf81059e0 2024-08-31T07:18:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.051+0000 7fccfda00640 1 -- 172.21.15.32:0/3823405067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf8105df0 con 0x7fccf81059e0 2024-08-31T07:18:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.053+0000 7fccf5600640 1 -- 172.21.15.32:0/3823405067 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcce8004ba0 con 0x7fccf81059e0 2024-08-31T07:18:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.055+0000 7fccf5600640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccdc0b4ca0 0x7fccdc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.055+0000 7fccf5600640 1 -- 172.21.15.32:0/3823405067 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fcce80e5520 con 0x7fccf81059e0 2024-08-31T07:18:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.055+0000 7fccefe00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccdc0b4ca0 0x7fccdc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.056+0000 7fccefe00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccdc0b4ca0 0x7fccdc0b71b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009230 tx=0x7fcce0031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.056+0000 7fccf5600640 1 -- 172.21.15.32:0/3823405067 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcce80ac3c0 con 0x7fccf81059e0 2024-08-31T07:18:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.292+0000 7fccfda00640 1 -- 172.21.15.32:0/3823405067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccf819d3a0 con 0x7fccf81059e0 2024-08-31T07:18:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.293+0000 7fccf5600640 1 -- 172.21.15.32:0/3823405067 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11960 (secure 0 0 0) 0x7fcce80abcc0 con 0x7fccf81059e0 2024-08-31T07:18:13.295 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:18:13.295 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"btime":"2024-08-31T07:17:01:405815+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:17:00.944291+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:18:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2024-08-31T07:18:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.295+0000 7fccfda00640 1 -- 172.21.15.32:0/3823405067 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccdc0b4ca0 msgr2=0x7fccdc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.295+0000 7fccfda00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccdc0b4ca0 0x7fccdc0b71b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009230 tx=0x7fcce0031400 comp rx=0 tx=0).stop 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.295+0000 7fccfda00640 1 -- 172.21.15.32:0/3823405067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccf81059e0 msgr2=0x7fccf819c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.295+0000 7fccfda00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccf81059e0 0x7fccf819c270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcce800e9f0 tx=0x7fcce800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.296+0000 7fccfda00640 1 -- 172.21.15.32:0/3823405067 shutdown_connections 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.296+0000 7fccfda00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccdc0b4ca0 0x7fccdc0b71b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.296+0000 7fccfda00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccf810bbe0 0x7fccf81a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.296+0000 7fccfda00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccf810b290 0x7fccf819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.296+0000 7fccfda00640 1 --2- 172.21.15.32:0/3823405067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccf81059e0 0x7fccf819c270 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.296+0000 7fccfda00640 1 -- 172.21.15.32:0/3823405067 >> 172.21.15.32:0/3823405067 conn(0x7fccf8101430 msgr2=0x7fccf8103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.296+0000 7fccfda00640 1 -- 172.21.15.32:0/3823405067 shutdown_connections 2024-08-31T07:18:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:13.296+0000 7fccfda00640 1 -- 172.21.15.32:0/3823405067 wait complete. 2024-08-31T07:18:13.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.51 MB/sec execute 122 sec latency 4594.203 ms 2024-08-31T07:18:13.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:13 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3823405067' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:18:13.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:13 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3823405067' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:18:13.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:13 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3823405067' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:18:14.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:18:14.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.49 MB/sec execute 123 sec latency 5594.266 ms 2024-08-31T07:18:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:14 smithi032 ceph-mon[17252]: pgmap v532: 97 pgs: 97 active+clean; 100 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 511 KiB/s wr, 102 op/s 2024-08-31T07:18:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:14 smithi073 ceph-mon[21995]: pgmap v532: 97 pgs: 97 active+clean; 100 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 511 KiB/s wr, 102 op/s 2024-08-31T07:18:14.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:14 smithi084 ceph-mon[22387]: pgmap v532: 97 pgs: 97 active+clean; 100 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 511 KiB/s wr, 102 op/s 2024-08-31T07:18:15.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.482+0000 7fbc71000640 1 -- 172.21.15.32:0/4048240072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c10b470 msgr2=0x7fbc6c10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:15.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.482+0000 7fbc71000640 1 --2- 172.21.15.32:0/4048240072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c10b470 0x7fbc6c10b880 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009900 tx=0x7fbc5402a410 comp rx=0 tx=0).stop 2024-08-31T07:18:15.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.482+0000 7fbc71000640 1 -- 172.21.15.32:0/4048240072 shutdown_connections 2024-08-31T07:18:15.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.482+0000 7fbc71000640 1 --2- 172.21.15.32:0/4048240072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc6c10bdc0 0x7fbc6c1125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.482+0000 7fbc71000640 1 --2- 172.21.15.32:0/4048240072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c10b470 0x7fbc6c10b880 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.482+0000 7fbc71000640 1 --2- 172.21.15.32:0/4048240072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6c105b30 0x7fbc6c105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.482+0000 7fbc71000640 1 -- 172.21.15.32:0/4048240072 >> 172.21.15.32:0/4048240072 conn(0x7fbc6c101600 msgr2=0x7fbc6c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:15.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.482+0000 7fbc71000640 1 -- 172.21.15.32:0/4048240072 shutdown_connections 2024-08-31T07:18:15.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.482+0000 7fbc71000640 1 -- 172.21.15.32:0/4048240072 wait complete. 2024-08-31T07:18:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.483+0000 7fbc71000640 1 Processor -- start 2024-08-31T07:18:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.483+0000 7fbc71000640 1 -- start start 2024-08-31T07:18:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc71000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc6c105b30 0x7fbc6c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc71000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6c10b470 0x7fbc6c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc71000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c10bdc0 0x7fbc6c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc71000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c115ad0 con 0x7fbc6c10bdc0 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc71000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c115950 con 0x7fbc6c105b30 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc71000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c115c50 con 0x7fbc6c10b470 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc69e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc6c105b30 0x7fbc6c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc6a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c10bdc0 0x7fbc6c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc69e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc6c105b30 0x7fbc6c19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:33720/0 (socket says 172.21.15.32:33720) 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc69e00640 1 -- 172.21.15.32:0/4088772842 learned_addr learned my addr 172.21.15.32:0/4088772842 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.484+0000 7fbc61400640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6c10b470 0x7fbc6c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc69e00640 1 -- 172.21.15.32:0/4088772842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6c10b470 msgr2=0x7fbc6c19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc69e00640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6c10b470 0x7fbc6c19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc69e00640 1 -- 172.21.15.32:0/4088772842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c10bdc0 msgr2=0x7fbc6c1a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc69e00640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c10bdc0 0x7fbc6c1a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc69e00640 1 -- 172.21.15.32:0/4088772842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c1a23f0 con 0x7fbc6c105b30 2024-08-31T07:18:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc6a800640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c10bdc0 0x7fbc6c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc69e00640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc6c105b30 0x7fbc6c19c480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00d950 tx=0x7fbc5c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc63400640 1 -- 172.21.15.32:0/4088772842 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c017030 con 0x7fbc6c105b30 2024-08-31T07:18:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc71000640 1 -- 172.21.15.32:0/4088772842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c1a26e0 con 0x7fbc6c105b30 2024-08-31T07:18:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc63400640 1 -- 172.21.15.32:0/4088772842 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc5c004020 con 0x7fbc6c105b30 2024-08-31T07:18:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc63400640 1 -- 172.21.15.32:0/4088772842 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0047a0 con 0x7fbc6c105b30 2024-08-31T07:18:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.485+0000 7fbc71000640 1 -- 172.21.15.32:0/4088772842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c1a8e40 con 0x7fbc6c105b30 2024-08-31T07:18:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.488+0000 7fbc77e07740 1 Processor -- start 2024-08-31T07:18:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.488+0000 7fbc77e07740 1 -- start start 2024-08-31T07:18:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b64070 0x55e135f09990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f09ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b43390 0x55e135f0a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc77e07740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e135e807b0 con 0x55e135b3b1b0 2024-08-31T07:18:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc77e07740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e135e6fd40 con 0x55e135b64070 2024-08-31T07:18:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc77e07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e135e73cf0 con 0x55e135b43390 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc6a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b64070 0x55e135f09990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc6a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b64070 0x55e135f09990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:33724/0 (socket says 172.21.15.32:33724) 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc6a800640 1 -- 172.21.15.32:0/1382131109 learned_addr learned my addr 172.21.15.32:0/1382131109 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc63400640 1 -- 172.21.15.32:0/4088772842 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbc5c009a30 con 0x7fbc6c105b30 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.489+0000 7fbc61400640 1 --2- 172.21.15.32:0/1382131109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b43390 0x55e135f0a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.490+0000 7fbc6a800640 1 -- 172.21.15.32:0/1382131109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b43390 msgr2=0x55e135f0a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.490+0000 7fbc6a800640 1 --2- 172.21.15.32:0/1382131109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b43390 0x55e135f0a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.490+0000 7fbc69e00640 1 --2- 172.21.15.32:0/1382131109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f09ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.490+0000 7fbc6a800640 1 -- 172.21.15.32:0/1382131109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 msgr2=0x55e135f09ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.490+0000 7fbc6a800640 1 --2- 172.21.15.32:0/1382131109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f09ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.490+0000 7fbc6a800640 1 -- 172.21.15.32:0/1382131109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e135bcf650 con 0x55e135b64070 2024-08-31T07:18:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.490+0000 7fbc69e00640 1 --2- 172.21.15.32:0/1382131109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f09ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:18:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.490+0000 7fbc6a800640 1 --2- 172.21.15.32:0/1382131109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b64070 0x55e135f09990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbc640107a0 tx=0x7fbc64010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb372b55d93616 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.490+0000 7fbc5b400640 1 -- 172.21.15.32:0/1382131109 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64003b70 con 0x55e135b64070 2024-08-31T07:18:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.491+0000 7fbc5b400640 1 -- 172.21.15.32:0/1382131109 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc64010e50 con 0x55e135b64070 2024-08-31T07:18:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.491+0000 7fbc5b400640 1 -- 172.21.15.32:0/1382131109 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64004600 con 0x55e135b64070 2024-08-31T07:18:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc77e07740 1 -- 172.21.15.32:0/1382131109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b64070 msgr2=0x55e135f09990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc77e07740 1 --2- 172.21.15.32:0/1382131109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b64070 0x55e135f09990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbc640107a0 tx=0x7fbc64010c60 comp rx=0 tx=0).stop 2024-08-31T07:18:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc63400640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbc500b4ca0 0x7fbc500b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc63400640 1 -- 172.21.15.32:0/4088772842 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fbc5c0e4f40 con 0x7fbc6c105b30 2024-08-31T07:18:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc6a800640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbc500b4ca0 0x7fbc500b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc77e07740 1 -- 172.21.15.32:0/1382131109 shutdown_connections 2024-08-31T07:18:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc77e07740 1 --2- 172.21.15.32:0/1382131109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b43390 0x55e135f0a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc77e07740 1 --2- 172.21.15.32:0/1382131109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f09ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc77e07740 1 --2- 172.21.15.32:0/1382131109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b64070 0x55e135f09990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc77e07740 1 -- 172.21.15.32:0/1382131109 >> 172.21.15.32:0/1382131109 conn(0x55e135b58460 msgr2=0x55e135f07880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc77e07740 1 -- 172.21.15.32:0/1382131109 shutdown_connections 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.492+0000 7fbc77e07740 1 -- 172.21.15.32:0/1382131109 wait complete. 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.493+0000 7fbc77e07740 1 -- start start 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.493+0000 7fbc6a800640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbc500b4ca0 0x7fbc500b71b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbc640042a0 tx=0x7fbc64009920 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.493+0000 7fbc77e07740 1 Processor -- start 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f15630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b43390 0x55e135f15bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 0x55e135f21460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc77e07740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e135cd2700 con 0x55e135b3b1b0 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc77e07740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e135df6cf0 con 0x55e135b43390 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc77e07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e135f18170 con 0x55e135b64070 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc77e07740 10 client.0 _ll_register_callbacks cb 0x55e135e1c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc69e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f15630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc69e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f15630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60988/0 (socket says 172.21.15.32:60988) 2024-08-31T07:18:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc69e00640 1 -- 172.21.15.32:0/3050528321 learned_addr learned my addr 172.21.15.32:0/3050528321 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc6a800640 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 0x55e135f21460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.494+0000 7fbc61400640 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b43390 0x55e135f15bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc69e00640 1 -- 172.21.15.32:0/3050528321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 msgr2=0x55e135f21460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc69e00640 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 0x55e135f21460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc69e00640 1 -- 172.21.15.32:0/3050528321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b43390 msgr2=0x55e135f15bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc69e00640 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b43390 0x55e135f15bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc69e00640 1 -- 172.21.15.32:0/3050528321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e135e378b0 con 0x55e135b3b1b0 2024-08-31T07:18:15.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc69e00640 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f15630 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0e2710 tx=0x7fbc5c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0e61bb242cf825 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:15.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:18:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc4fe00640 1 -- 172.21.15.32:0/3050528321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0ab650 con 0x55e135b3b1b0 2024-08-31T07:18:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc4fe00640 1 -- 172.21.15.32:0/3050528321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fbc5c0a7c40 con 0x55e135b3b1b0 2024-08-31T07:18:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc77e07740 1 -- 172.21.15.32:0/3050528321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc64010e50 con 0x55e135b3b1b0 2024-08-31T07:18:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc4fe00640 1 -- 172.21.15.32:0/3050528321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbc5c0afec0 con 0x55e135b3b1b0 2024-08-31T07:18:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc77e07740 10 client.16662 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.495+0000 7fbc77e07740 1 -- 172.21.15.32:0/3050528321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e135f241e0 con 0x55e135b3b1b0 2024-08-31T07:18:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.496+0000 7fbc4fe00640 1 -- 172.21.15.32:0/3050528321 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc64010e50 con 0x55e135b3b1b0 2024-08-31T07:18:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.496+0000 7fbc4fe00640 1 -- 172.21.15.32:0/3050528321 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fbc5c0e2930 con 0x55e135b3b1b0 2024-08-31T07:18:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.496+0000 7fbc77e07740 10 client.16662 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.496+0000 7fbc77e07740 10 client.16662 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:18:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.496+0000 7fbc77e07740 20 client.16662 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.496+0000 7fbc77e07740 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e135b229f0 0x55e135f276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.496+0000 7fbc77e07740 4 client.16662 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:18:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.496+0000 7fbc77e07740 1 -- 172.21.15.32:0/3050528321 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e135e82700 con 0x55e135b229f0 2024-08-31T07:18:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.497+0000 7fbc61400640 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e135b229f0 0x55e135f276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.497+0000 7fbc61400640 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e135b229f0 0x55e135f276b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fdbc1ab71a0f75f3 server_cookie=9839b1dd86b42a37 in_seq=0 out_seq=0 2024-08-31T07:18:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.497+0000 7fbc4fe00640 10 client.16662 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:18:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.498+0000 7fbc4fe00640 1 -- 172.21.15.32:0/3050528321 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e135e82700 con 0x55e135b229f0 2024-08-31T07:18:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.498+0000 7fbc4fe00640 10 client.16662 handle_command_reply: tid=0 2024-08-31T07:18:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.498+0000 7fbc77e07740 1 client.16662 shutdown 2024-08-31T07:18:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 1 -- 172.21.15.32:0/3050528321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 msgr2=0x55e135f15630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f15630 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0e2710 tx=0x7fbc5c009f50 comp rx=0 tx=0).stop 2024-08-31T07:18:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 1 -- 172.21.15.32:0/3050528321 shutdown_connections 2024-08-31T07:18:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e135b229f0 0x55e135f276b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 0x55e135f21460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b43390 0x55e135f15bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 1 --2- 172.21.15.32:0/3050528321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f15630 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 1 -- 172.21.15.32:0/3050528321 >> 172.21.15.32:0/3050528321 conn(0x55e135b58460 msgr2=0x55e135f0cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc4fe00640 0 client.16662 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:18:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 1 -- 172.21.15.32:0/3050528321 shutdown_connections 2024-08-31T07:18:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 1 -- 172.21.15.32:0/3050528321 wait complete. 2024-08-31T07:18:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.499+0000 7fbc77e07740 20 client.16662 trim_cache size 0 max 16384 2024-08-31T07:18:15.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.513+0000 7fbc77e07740 1 Processor -- start 2024-08-31T07:18:15.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.513+0000 7fbc77e07740 1 -- start start 2024-08-31T07:18:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.513+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b3b1b0 0x55e135f0eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.513+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b43390 0x55e135f0efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.513+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b64070 0x55e135f06e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.513+0000 7fbc77e07740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1b1490 con 0x55e135b64070 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.513+0000 7fbc77e07740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1b2fd0 con 0x55e135b3b1b0 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.513+0000 7fbc77e07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1b1b60 con 0x55e135b43390 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc6a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b64070 0x55e135f06e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc6a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b64070 0x55e135f06e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60994/0 (socket says 172.21.15.32:60994) 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc61400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b43390 0x55e135f0efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc6a800640 1 -- 172.21.15.32:0/1261566716 learned_addr learned my addr 172.21.15.32:0/1261566716 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc69e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b3b1b0 0x55e135f0eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc6a800640 1 -- 172.21.15.32:0/1261566716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b43390 msgr2=0x55e135f0efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc6a800640 1 --2- 172.21.15.32:0/1261566716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b43390 0x55e135f0efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc6a800640 1 -- 172.21.15.32:0/1261566716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b3b1b0 msgr2=0x55e135f0eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc6a800640 1 --2- 172.21.15.32:0/1261566716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b3b1b0 0x55e135f0eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc6a800640 1 -- 172.21.15.32:0/1261566716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e135f0bc80 con 0x55e135b64070 2024-08-31T07:18:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.514+0000 7fbc69e00640 1 --2- 172.21.15.32:0/1261566716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b3b1b0 0x55e135f0eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.515+0000 7fbc6a800640 1 --2- 172.21.15.32:0/1261566716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b64070 0x55e135f06e30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbc64004310 tx=0x7fbc64013050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd5933a2013dda4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.515+0000 7fbc5aa00640 1 -- 172.21.15.32:0/1261566716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64036070 con 0x55e135b64070 2024-08-31T07:18:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.515+0000 7fbc5aa00640 1 -- 172.21.15.32:0/1261566716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc64003f80 con 0x55e135b64070 2024-08-31T07:18:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.515+0000 7fbc5aa00640 1 -- 172.21.15.32:0/1261566716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64005340 con 0x55e135b64070 2024-08-31T07:18:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.516+0000 7fbc77e07740 1 -- 172.21.15.32:0/1261566716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b64070 msgr2=0x55e135f06e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.516+0000 7fbc77e07740 1 --2- 172.21.15.32:0/1261566716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b64070 0x55e135f06e30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbc64004310 tx=0x7fbc64013050 comp rx=0 tx=0).stop 2024-08-31T07:18:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.516+0000 7fbc77e07740 1 -- 172.21.15.32:0/1261566716 shutdown_connections 2024-08-31T07:18:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.516+0000 7fbc77e07740 1 --2- 172.21.15.32:0/1261566716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b64070 0x55e135f06e30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.516+0000 7fbc77e07740 1 --2- 172.21.15.32:0/1261566716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b43390 0x55e135f0efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.516+0000 7fbc77e07740 1 --2- 172.21.15.32:0/1261566716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b3b1b0 0x55e135f0eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.516+0000 7fbc77e07740 1 -- 172.21.15.32:0/1261566716 >> 172.21.15.32:0/1261566716 conn(0x55e135b58460 msgr2=0x55e135f13cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.516+0000 7fbc77e07740 1 -- 172.21.15.32:0/1261566716 shutdown_connections 2024-08-31T07:18:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.516+0000 7fbc77e07740 1 -- 172.21.15.32:0/1261566716 wait complete. 2024-08-31T07:18:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.516+0000 7fbc77e07740 1 -- start start 2024-08-31T07:18:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc77e07740 1 Processor -- start 2024-08-31T07:18:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f19a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b43390 0x55e135f19fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc77e07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 0x55e135f91000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc77e07740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1b2900 con 0x55e135b3b1b0 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc77e07740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1b2230 con 0x55e135b43390 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc77e07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1af470 con 0x55e135b64070 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc77e07740 10 client.0 _ll_register_callbacks cb 0x7fbc5c0aafa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc69e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f19a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc69e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f19a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32770/0 (socket says 172.21.15.32:32770) 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.517+0000 7fbc69e00640 1 -- 172.21.15.32:0/816211395 learned_addr learned my addr 172.21.15.32:0/816211395 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc61400640 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b43390 0x55e135f19fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc6a800640 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 0x55e135f91000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc69e00640 1 -- 172.21.15.32:0/816211395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 msgr2=0x55e135f91000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc69e00640 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 0x55e135f91000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc69e00640 1 -- 172.21.15.32:0/816211395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b43390 msgr2=0x55e135f19fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc69e00640 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b43390 0x55e135f19fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc69e00640 1 -- 172.21.15.32:0/816211395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e135f91540 con 0x55e135b3b1b0 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc6a800640 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 0x55e135f91000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc69e00640 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f19a60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0b0080 tx=0x7fbc5c004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f904be936c3413 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc4fe00640 1 -- 172.21.15.32:0/816211395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0ab650 con 0x55e135b3b1b0 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc4fe00640 1 -- 172.21.15.32:0/816211395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fbc5c0a7c40 con 0x55e135b3b1b0 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.518+0000 7fbc77e07740 1 -- 172.21.15.32:0/816211395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc64003f80 con 0x55e135b3b1b0 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc4fe00640 1 -- 172.21.15.32:0/816211395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbc5c0b4970 con 0x55e135b3b1b0 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc77e07740 10 client.16668 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc77e07740 1 -- 172.21.15.32:0/816211395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e135f92750 con 0x55e135b3b1b0 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc4fe00640 1 -- 172.21.15.32:0/816211395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc64003f80 con 0x55e135b3b1b0 2024-08-31T07:18:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc4fe00640 1 -- 172.21.15.32:0/816211395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fbc5c0aa8b0 con 0x55e135b3b1b0 2024-08-31T07:18:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc77e07740 10 client.16668 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc77e07740 10 client.16668 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:18:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc77e07740 20 client.16668 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc77e07740 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e135af7da0 0x55e135f94f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc77e07740 4 client.16668 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:18:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.519+0000 7fbc77e07740 1 -- 172.21.15.32:0/816211395 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e135e7e1a0 con 0x55e135af7da0 2024-08-31T07:18:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.520+0000 7fbc61400640 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e135af7da0 0x55e135f94f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.520+0000 7fbc61400640 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e135af7da0 0x55e135f94f10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fce7253262dd6da6 server_cookie=ec9ef20e3bd2e6e5 in_seq=0 out_seq=0 2024-08-31T07:18:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.520+0000 7fbc4fe00640 10 client.16668 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:18:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.520+0000 7fbc4fe00640 1 -- 172.21.15.32:0/816211395 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e135e7e1a0 con 0x55e135af7da0 2024-08-31T07:18:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc4fe00640 10 client.16668 handle_command_reply: tid=0 2024-08-31T07:18:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 client.16668 shutdown 2024-08-31T07:18:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 -- 172.21.15.32:0/816211395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 msgr2=0x55e135f19a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f19a60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0b0080 tx=0x7fbc5c004590 comp rx=0 tx=0).stop 2024-08-31T07:18:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 -- 172.21.15.32:0/816211395 shutdown_connections 2024-08-31T07:18:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e135af7da0 0x55e135f94f10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e135b64070 0x55e135f91000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e135b43390 0x55e135f19fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 --2- 172.21.15.32:0/816211395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e135b3b1b0 0x55e135f19a60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 -- 172.21.15.32:0/816211395 >> 172.21.15.32:0/816211395 conn(0x55e135b58460 msgr2=0x55e135f138b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc4fe00640 0 client.16668 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:18:15.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 -- 172.21.15.32:0/816211395 shutdown_connections 2024-08-31T07:18:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 1 -- 172.21.15.32:0/816211395 wait complete. 2024-08-31T07:18:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.521+0000 7fbc77e07740 20 client.16668 trim_cache size 0 max 16384 2024-08-31T07:18:15.529 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:18:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 -- 172.21.15.32:0/4088772842 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbc500b4ca0 msgr2=0x7fbc500b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbc500b4ca0 0x7fbc500b71b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbc640042a0 tx=0x7fbc64009920 comp rx=0 tx=0).stop 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 -- 172.21.15.32:0/4088772842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc6c105b30 msgr2=0x7fbc6c19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc6c105b30 0x7fbc6c19c480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00d950 tx=0x7fbc5c00de10 comp rx=0 tx=0).stop 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 -- 172.21.15.32:0/4088772842 shutdown_connections 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbc500b4ca0 0x7fbc500b71b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c10bdc0 0x7fbc6c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6c10b470 0x7fbc6c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 --2- 172.21.15.32:0/4088772842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc6c105b30 0x7fbc6c19c480 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 -- 172.21.15.32:0/4088772842 >> 172.21.15.32:0/4088772842 conn(0x7fbc6c101600 msgr2=0x7fbc6c103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 -- 172.21.15.32:0/4088772842 shutdown_connections 2024-08-31T07:18:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:15.522+0000 7fbc71000640 1 -- 172.21.15.32:0/4088772842 wait complete. 2024-08-31T07:18:15.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.47 MB/sec execute 124 sec latency 6594.346 ms 2024-08-31T07:18:16.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.45 MB/sec execute 125 sec latency 7594.482 ms 2024-08-31T07:18:16.740 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:16 smithi032 ceph-mon[17252]: pgmap v533: 97 pgs: 97 active+clean; 101 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 493 KiB/s wr, 87 op/s 2024-08-31T07:18:16.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:18:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:16 smithi073 ceph-mon[21995]: pgmap v533: 97 pgs: 97 active+clean; 101 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 493 KiB/s wr, 87 op/s 2024-08-31T07:18:16.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:16 smithi084 ceph-mon[22387]: pgmap v533: 97 pgs: 97 active+clean; 101 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 493 KiB/s wr, 87 op/s 2024-08-31T07:18:17.164 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:17.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.43 MB/sec execute 126 sec latency 8594.548 ms 2024-08-31T07:18:17.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.859+0000 7f0714000640 1 -- 172.21.15.32:0/443284990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c075e40 msgr2=0x7f070c10f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:17.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.859+0000 7f0714000640 1 --2- 172.21.15.32:0/443284990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c075e40 0x7f070c10f980 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009900 tx=0x7f06f802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.860+0000 7f0714000640 1 -- 172.21.15.32:0/443284990 shutdown_connections 2024-08-31T07:18:17.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.860+0000 7f0714000640 1 --2- 172.21.15.32:0/443284990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c10fec0 0x7f070c1122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.860+0000 7f0714000640 1 --2- 172.21.15.32:0/443284990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c075e40 0x7f070c10f980 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.860+0000 7f0714000640 1 --2- 172.21.15.32:0/443284990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f070c0754f0 0x7f070c075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.860+0000 7f0714000640 1 -- 172.21.15.32:0/443284990 >> 172.21.15.32:0/443284990 conn(0x7f070c07b5f0 msgr2=0x7f070c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:17.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.860+0000 7f0714000640 1 -- 172.21.15.32:0/443284990 shutdown_connections 2024-08-31T07:18:17.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.860+0000 7f0714000640 1 -- 172.21.15.32:0/443284990 wait complete. 2024-08-31T07:18:17.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.861+0000 7f0714000640 1 Processor -- start 2024-08-31T07:18:17.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.861+0000 7f0714000640 1 -- start start 2024-08-31T07:18:17.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.861+0000 7f0714000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f070c0754f0 0x7f070c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.861+0000 7f0714000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c075e40 0x7f070c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.861+0000 7f0714000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c10fec0 0x7f070c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.861+0000 7f0714000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f070c115a30 con 0x7f070c0754f0 2024-08-31T07:18:17.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.861+0000 7f0714000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1158b0 con 0x7f070c075e40 2024-08-31T07:18:17.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.861+0000 7f0714000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f070c115bb0 con 0x7f070c10fec0 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0711600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f070c0754f0 0x7f070c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0710c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c075e40 0x7f070c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0711600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f070c0754f0 0x7f070c19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43172/0 (socket says 172.21.15.32:43172) 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0711600640 1 -- 172.21.15.32:0/1251138956 learned_addr learned my addr 172.21.15.32:0/1251138956 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0712000640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c10fec0 0x7f070c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0711600640 1 -- 172.21.15.32:0/1251138956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c10fec0 msgr2=0x7f070c1a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0711600640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c10fec0 0x7f070c1a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0711600640 1 -- 172.21.15.32:0/1251138956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c075e40 msgr2=0x7f070c19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0711600640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c075e40 0x7f070c19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0711600640 1 -- 172.21.15.32:0/1251138956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c1a23f0 con 0x7f070c0754f0 2024-08-31T07:18:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.862+0000 7f0711600640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f070c0754f0 0x7f070c19c480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f070000e9f0 tx=0x7f070000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:17.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.863+0000 7f070a000640 1 -- 172.21.15.32:0/1251138956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070000cd80 con 0x7f070c0754f0 2024-08-31T07:18:17.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.863+0000 7f070a000640 1 -- 172.21.15.32:0/1251138956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07000108e0 con 0x7f070c0754f0 2024-08-31T07:18:17.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.863+0000 7f0714000640 1 -- 172.21.15.32:0/1251138956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c1a26e0 con 0x7f070c0754f0 2024-08-31T07:18:17.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.863+0000 7f070a000640 1 -- 172.21.15.32:0/1251138956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700018980 con 0x7f070c0754f0 2024-08-31T07:18:17.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.863+0000 7f0714000640 1 -- 172.21.15.32:0/1251138956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c1a8e40 con 0x7f070c0754f0 2024-08-31T07:18:17.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.866+0000 7f071ac60740 1 Processor -- start 2024-08-31T07:18:17.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f071ac60740 1 -- start start 2024-08-31T07:18:17.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f070a000640 1 -- 172.21.15.32:0/1251138956 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0700010430 con 0x7f070c0754f0 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f071ac60740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 0x56223095a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f071ac60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 0x56223095aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f071ac60740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x56223095b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f071ac60740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5622308d17b0 con 0x562230594390 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f071ac60740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5622308c0d40 con 0x5622305b5070 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f071ac60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5622308c4cf0 con 0x56223058c1b0 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f0711600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 0x56223095aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f0711600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 0x56223095aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54934/0 (socket says 172.21.15.32:54934) 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f0711600640 1 -- 172.21.15.32:0/365121050 learned_addr learned my addr 172.21.15.32:0/365121050 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.867+0000 7f0712000640 1 --2- 172.21.15.32:0/365121050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 0x56223095a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.868+0000 7f0710c00640 1 --2- 172.21.15.32:0/365121050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x56223095b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.868+0000 7f0712000640 1 -- 172.21.15.32:0/365121050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 msgr2=0x56223095aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:17.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.868+0000 7f0712000640 1 --2- 172.21.15.32:0/365121050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 0x56223095aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.868+0000 7f0712000640 1 -- 172.21.15.32:0/365121050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 msgr2=0x56223095b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:17.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.868+0000 7f0712000640 1 --2- 172.21.15.32:0/365121050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x56223095b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.868+0000 7f0712000640 1 -- 172.21.15.32:0/365121050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562230620650 con 0x5622305b5070 2024-08-31T07:18:17.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.868+0000 7f0712000640 1 --2- 172.21.15.32:0/365121050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 0x56223095a990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0704009eb0 tx=0x7f07040107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d9e4b1e5038b92 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.868+0000 7f06fe600640 1 -- 172.21.15.32:0/365121050 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704003b70 con 0x5622305b5070 2024-08-31T07:18:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.868+0000 7f06fe600640 1 -- 172.21.15.32:0/365121050 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0704015070 con 0x5622305b5070 2024-08-31T07:18:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.868+0000 7f06fe600640 1 -- 172.21.15.32:0/365121050 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070401a8d0 con 0x5622305b5070 2024-08-31T07:18:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.869+0000 7f070a000640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06f40b4ca0 0x7f06f40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.869+0000 7f0711600640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06f40b4ca0 0x7f06f40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.869+0000 7f070a000640 1 -- 172.21.15.32:0/1251138956 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f07000e57d0 con 0x7f070c0754f0 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f071ac60740 1 -- 172.21.15.32:0/365121050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 msgr2=0x56223095a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f071ac60740 1 --2- 172.21.15.32:0/365121050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 0x56223095a990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0704009eb0 tx=0x7f07040107a0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f071ac60740 1 -- 172.21.15.32:0/365121050 shutdown_connections 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f071ac60740 1 --2- 172.21.15.32:0/365121050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x56223095b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f071ac60740 1 --2- 172.21.15.32:0/365121050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 0x56223095aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f071ac60740 1 --2- 172.21.15.32:0/365121050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 0x56223095a990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f071ac60740 1 -- 172.21.15.32:0/365121050 >> 172.21.15.32:0/365121050 conn(0x5622305a9460 msgr2=0x562230958880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f071ac60740 1 -- 172.21.15.32:0/365121050 shutdown_connections 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f0711600640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06f40b4ca0 0x7f06f40b71b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f07000e3000 tx=0x7f07000eb550 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f071ac60740 1 -- 172.21.15.32:0/365121050 wait complete. 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.870+0000 7f071ac60740 1 -- start start 2024-08-31T07:18:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.871+0000 7f071ac60740 1 Processor -- start 2024-08-31T07:18:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.871+0000 7f071ac60740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56223058c1b0 0x562230966500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.871+0000 7f071ac60740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562230594390 0x562230966a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.871+0000 7f071ac60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622305b5070 0x562230972490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.871+0000 7f071ac60740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562230723700 con 0x56223058c1b0 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.871+0000 7f071ac60740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562230847cf0 con 0x562230594390 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.871+0000 7f071ac60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562230969170 con 0x5622305b5070 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.871+0000 7f071ac60740 10 client.0 _ll_register_callbacks cb 0x56223086d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.871+0000 7f0712000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56223058c1b0 0x562230966500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0710c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562230594390 0x562230966a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0712000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56223058c1b0 0x562230966500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43194/0 (socket says 172.21.15.32:43194) 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0712000640 1 -- 172.21.15.32:0/149004482 learned_addr learned my addr 172.21.15.32:0/149004482 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0710c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562230594390 0x562230966a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49522/0 (socket says 172.21.15.32:49522) 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0712000640 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622305b5070 0x562230972490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0710c00640 1 -- 172.21.15.32:0/149004482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622305b5070 msgr2=0x562230972490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0710c00640 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622305b5070 0x562230972490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0710c00640 1 -- 172.21.15.32:0/149004482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56223058c1b0 msgr2=0x562230966500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0710c00640 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56223058c1b0 0x562230966500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0710c00640 1 -- 172.21.15.32:0/149004482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622308888b0 con 0x562230594390 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f0710c00640 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562230594390 0x562230966a40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f06f802a910 tx=0x7f06f8032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=458add362957e18a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.872+0000 7f06f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.873+0000 7f06f2a00640 1 -- 172.21.15.32:0/149004482 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8035040 con 0x562230594390 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.873+0000 7f071ac60740 1 -- 172.21.15.32:0/149004482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0704015070 con 0x562230594390 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.873+0000 7f06f2a00640 1 -- 172.21.15.32:0/149004482 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f06f80316a0 con 0x562230594390 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.873+0000 7f06f2a00640 1 -- 172.21.15.32:0/149004482 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f06f80318a0 con 0x562230594390 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.873+0000 7f071ac60740 10 client.26696 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.873+0000 7f071ac60740 1 -- 172.21.15.32:0/149004482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5622309751b0 con 0x562230594390 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.873+0000 7f06f2a00640 1 -- 172.21.15.32:0/149004482 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0704015070 con 0x562230594390 2024-08-31T07:18:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.873+0000 7f06f2a00640 1 -- 172.21.15.32:0/149004482 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f06f8004750 con 0x562230594390 2024-08-31T07:18:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.874+0000 7f071ac60740 10 client.26696 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.874+0000 7f071ac60740 10 client.26696 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:18:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.874+0000 7f071ac60740 20 client.26696 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.874+0000 7f071ac60740 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5622305739f0 0x5622309785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.874+0000 7f071ac60740 4 client.26696 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:18:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.874+0000 7f071ac60740 1 -- 172.21.15.32:0/149004482 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622308d3700 con 0x5622305739f0 2024-08-31T07:18:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.874+0000 7f0712000640 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5622305739f0 0x5622309785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.875+0000 7f0712000640 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5622305739f0 0x5622309785d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96629d716f14fd1c server_cookie=9c90ab2f496d365f in_seq=0 out_seq=0 2024-08-31T07:18:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.875+0000 7f06f2a00640 10 client.26696 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:18:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f06f2a00640 1 -- 172.21.15.32:0/149004482 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5622308d3700 con 0x5622305739f0 2024-08-31T07:18:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f06f2a00640 10 client.26696 handle_command_reply: tid=0 2024-08-31T07:18:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 client.26696 shutdown 2024-08-31T07:18:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 -- 172.21.15.32:0/149004482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562230594390 msgr2=0x562230966a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562230594390 0x562230966a40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f06f802a910 tx=0x7f06f8032e90 comp rx=0 tx=0).stop 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 -- 172.21.15.32:0/149004482 shutdown_connections 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5622305739f0 0x5622309785d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622305b5070 0x562230972490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562230594390 0x562230966a40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 --2- 172.21.15.32:0/149004482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56223058c1b0 0x562230966500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f06f2a00640 0 client.26696 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 -- 172.21.15.32:0/149004482 >> 172.21.15.32:0/149004482 conn(0x5622305a9460 msgr2=0x56223095db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 -- 172.21.15.32:0/149004482 shutdown_connections 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 1 -- 172.21.15.32:0/149004482 wait complete. 2024-08-31T07:18:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.876+0000 7f071ac60740 20 client.26696 trim_cache size 0 max 16384 2024-08-31T07:18:17.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.891+0000 7f071ac60740 1 Processor -- start 2024-08-31T07:18:17.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.891+0000 7f071ac60740 1 -- start start 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.891+0000 7f071ac60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 0x562230957e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.891+0000 7f071ac60740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x562230958370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.891+0000 7f071ac60740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 0x5622309634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.891+0000 7f071ac60740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1b1670 con 0x562230594390 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.891+0000 7f071ac60740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1b31b0 con 0x5622305b5070 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.891+0000 7f071ac60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1b1d40 con 0x56223058c1b0 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.891+0000 7f0710c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x562230958370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0710c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x562230958370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43206/0 (socket says 172.21.15.32:43206) 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0710c00640 1 -- 172.21.15.32:0/1781694451 learned_addr learned my addr 172.21.15.32:0/1781694451 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0712000640 1 --2- 172.21.15.32:0/1781694451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 0x5622309634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0712000640 1 --2- 172.21.15.32:0/1781694451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 0x562230957e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0710c00640 1 -- 172.21.15.32:0/1781694451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 msgr2=0x562230957e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0710c00640 1 --2- 172.21.15.32:0/1781694451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 0x562230957e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0710c00640 1 -- 172.21.15.32:0/1781694451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 msgr2=0x5622309634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0710c00640 1 --2- 172.21.15.32:0/1781694451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 0x5622309634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0710c00640 1 -- 172.21.15.32:0/1781694451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622309675c0 con 0x562230594390 2024-08-31T07:18:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0712000640 1 --2- 172.21.15.32:0/1781694451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 0x562230957e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:18:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.892+0000 7f0710c00640 1 --2- 172.21.15.32:0/1781694451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x562230958370 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f06f80059d0 tx=0x7f06f802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3689d7c64d8074d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:17.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.893+0000 7f06fdc00640 1 -- 172.21.15.32:0/1781694451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8035070 con 0x562230594390 2024-08-31T07:18:17.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.893+0000 7f06fdc00640 1 -- 172.21.15.32:0/1781694451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f80316a0 con 0x562230594390 2024-08-31T07:18:17.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.893+0000 7f06fdc00640 1 -- 172.21.15.32:0/1781694451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8003e30 con 0x562230594390 2024-08-31T07:18:17.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.893+0000 7f071ac60740 1 -- 172.21.15.32:0/1781694451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 msgr2=0x562230958370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:17.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.893+0000 7f071ac60740 1 --2- 172.21.15.32:0/1781694451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x562230958370 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f06f80059d0 tx=0x7f06f802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.893+0000 7f071ac60740 1 -- 172.21.15.32:0/1781694451 shutdown_connections 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.893+0000 7f071ac60740 1 --2- 172.21.15.32:0/1781694451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5622305b5070 0x5622309634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.893+0000 7f071ac60740 1 --2- 172.21.15.32:0/1781694451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x562230958370 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.893+0000 7f071ac60740 1 --2- 172.21.15.32:0/1781694451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56223058c1b0 0x562230957e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.893+0000 7f071ac60740 1 -- 172.21.15.32:0/1781694451 >> 172.21.15.32:0/1781694451 conn(0x5622305a9460 msgr2=0x562230964b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.894+0000 7f071ac60740 1 -- 172.21.15.32:0/1781694451 shutdown_connections 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.894+0000 7f071ac60740 1 -- 172.21.15.32:0/1781694451 wait complete. 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.894+0000 7f071ac60740 1 -- start start 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.894+0000 7f071ac60740 1 Processor -- start 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f071ac60740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56223058c1b0 0x56223096abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f071ac60740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x56223096b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f071ac60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622305b5070 0x5622309e1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f071ac60740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1b2ae0 con 0x562230594390 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f071ac60740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1b2410 con 0x56223058c1b0 2024-08-31T07:18:17.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f071ac60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f070c1af650 con 0x5622305b5070 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f071ac60740 10 client.0 _ll_register_callbacks cb 0x7f06f8032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f0710c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x56223096b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f0710c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x56223096b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43210/0 (socket says 172.21.15.32:43210) 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f0710c00640 1 -- 172.21.15.32:0/808542329 learned_addr learned my addr 172.21.15.32:0/808542329 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f0712000640 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56223058c1b0 0x56223096abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.895+0000 7f0712000640 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622305b5070 0x5622309e1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.896+0000 7f0710c00640 1 -- 172.21.15.32:0/808542329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622305b5070 msgr2=0x5622309e1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.896+0000 7f0710c00640 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622305b5070 0x5622309e1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.896+0000 7f0710c00640 1 -- 172.21.15.32:0/808542329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56223058c1b0 msgr2=0x56223096abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.896+0000 7f0710c00640 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56223058c1b0 0x56223096abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.896+0000 7f0710c00640 1 -- 172.21.15.32:0/808542329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622309e25a0 con 0x562230594390 2024-08-31T07:18:17.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.896+0000 7f0710c00640 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x56223096b140 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f06f8031870 tx=0x7f06f8005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e33acf3ac7241c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.896+0000 7f06f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.896+0000 7f06f2a00640 1 -- 172.21.15.32:0/808542329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8035040 con 0x562230594390 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.896+0000 7f06f2a00640 1 -- 172.21.15.32:0/808542329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f06f8003800 con 0x562230594390 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.896+0000 7f071ac60740 1 -- 172.21.15.32:0/808542329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f80316a0 con 0x562230594390 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f06f2a00640 1 -- 172.21.15.32:0/808542329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f06f8004d10 con 0x562230594390 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f071ac60740 10 client.16686 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f071ac60740 1 -- 172.21.15.32:0/808542329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5622309e3820 con 0x562230594390 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f06f2a00640 1 -- 172.21.15.32:0/808542329 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f80316a0 con 0x562230594390 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f06f2a00640 1 -- 172.21.15.32:0/808542329 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f06f8045880 con 0x562230594390 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f071ac60740 10 client.16686 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f071ac60740 10 client.16686 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:18:17.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f071ac60740 20 client.16686 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f071ac60740 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562230548da0 0x5622309e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f071ac60740 4 client.16686 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.897+0000 7f071ac60740 1 -- 172.21.15.32:0/808542329 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622308cf1a0 con 0x562230548da0 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.898+0000 7f0712000640 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562230548da0 0x5622309e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.898+0000 7f0712000640 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562230548da0 0x5622309e5e90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ec541f615c7ff09 server_cookie=21698ce1ab9ac341 in_seq=0 out_seq=0 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.898+0000 7f06f2a00640 10 client.16686 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.898+0000 7f06f2a00640 1 -- 172.21.15.32:0/808542329 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622308cf1a0 con 0x562230548da0 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.898+0000 7f06f2a00640 10 client.16686 handle_command_reply: tid=0 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.898+0000 7f071ac60740 1 client.16686 shutdown 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 1 -- 172.21.15.32:0/808542329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 msgr2=0x56223096b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x56223096b140 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f06f8031870 tx=0x7f06f8005ec0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 1 -- 172.21.15.32:0/808542329 shutdown_connections 2024-08-31T07:18:17.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562230548da0 0x5622309e5e90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622305b5070 0x5622309e1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562230594390 0x56223096b140 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 1 --2- 172.21.15.32:0/808542329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56223058c1b0 0x56223096abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 1 -- 172.21.15.32:0/808542329 >> 172.21.15.32:0/808542329 conn(0x5622305a9460 msgr2=0x56223095da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:17.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f06f2a00640 0 client.16686 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:18:17.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 1 -- 172.21.15.32:0/808542329 shutdown_connections 2024-08-31T07:18:17.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 1 -- 172.21.15.32:0/808542329 wait complete. 2024-08-31T07:18:17.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.899+0000 7f071ac60740 20 client.16686 trim_cache size 0 max 16384 2024-08-31T07:18:17.903 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:18:17.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.900+0000 7f0714000640 1 -- 172.21.15.32:0/1251138956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06f40b4ca0 msgr2=0x7f06f40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:17.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.900+0000 7f0714000640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06f40b4ca0 0x7f06f40b71b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f07000e3000 tx=0x7f07000eb550 comp rx=0 tx=0).stop 2024-08-31T07:18:17.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.900+0000 7f0714000640 1 -- 172.21.15.32:0/1251138956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f070c0754f0 msgr2=0x7f070c19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:17.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.900+0000 7f0714000640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f070c0754f0 0x7f070c19c480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f070000e9f0 tx=0x7f070000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.900+0000 7f0714000640 1 -- 172.21.15.32:0/1251138956 shutdown_connections 2024-08-31T07:18:17.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.900+0000 7f0714000640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f06f40b4ca0 0x7f06f40b71b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.900+0000 7f0714000640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f070c10fec0 0x7f070c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.900+0000 7f0714000640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c075e40 0x7f070c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.900+0000 7f0714000640 1 --2- 172.21.15.32:0/1251138956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f070c0754f0 0x7f070c19c480 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:17.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.900+0000 7f0714000640 1 -- 172.21.15.32:0/1251138956 >> 172.21.15.32:0/1251138956 conn(0x7f070c07b5f0 msgr2=0x7f070c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:17.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.901+0000 7f0714000640 1 -- 172.21.15.32:0/1251138956 shutdown_connections 2024-08-31T07:18:17.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:17.901+0000 7f0714000640 1 -- 172.21.15.32:0/1251138956 wait complete. 2024-08-31T07:18:18.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:18 smithi032 ceph-mon[17252]: pgmap v534: 97 pgs: 97 active+clean; 101 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 351 KiB/s wr, 68 op/s 2024-08-31T07:18:18.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.41 MB/sec execute 127 sec latency 9601.374 ms 2024-08-31T07:18:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:18 smithi073 ceph-mon[21995]: pgmap v534: 97 pgs: 97 active+clean; 101 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 351 KiB/s wr, 68 op/s 2024-08-31T07:18:18.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:18 smithi084 ceph-mon[22387]: pgmap v534: 97 pgs: 97 active+clean; 101 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 351 KiB/s wr, 68 op/s 2024-08-31T07:18:19.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:18:19.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.39 MB/sec execute 128 sec latency 10601.420 ms 2024-08-31T07:18:19.851 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.527+0000 7f25e2c00640 1 -- 172.21.15.32:0/2899536161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc103750 msgr2=0x7f25dc103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.527+0000 7f25e2c00640 1 --2- 172.21.15.32:0/2899536161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc103750 0x7f25dc103b60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009900 tx=0x7f25d002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.527+0000 7f25e2c00640 1 -- 172.21.15.32:0/2899536161 shutdown_connections 2024-08-31T07:18:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.527+0000 7f25e2c00640 1 --2- 172.21.15.32:0/2899536161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25dc1099e0 0x7f25dc110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.527+0000 7f25e2c00640 1 --2- 172.21.15.32:0/2899536161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc109090 0x7f25dc1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.527+0000 7f25e2c00640 1 --2- 172.21.15.32:0/2899536161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc103750 0x7f25dc103b60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.527+0000 7f25e2c00640 1 -- 172.21.15.32:0/2899536161 >> 172.21.15.32:0/2899536161 conn(0x7f25dc0ff1e0 msgr2=0x7f25dc101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.527+0000 7f25e2c00640 1 -- 172.21.15.32:0/2899536161 shutdown_connections 2024-08-31T07:18:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.527+0000 7f25e2c00640 1 -- 172.21.15.32:0/2899536161 wait complete. 2024-08-31T07:18:20.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.529+0000 7f25e2c00640 1 Processor -- start 2024-08-31T07:18:20.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.529+0000 7f25e2c00640 1 -- start start 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.529+0000 7f25e2c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc103750 0x7f25dc0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.529+0000 7f25e2c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25dc109090 0x7f25dc0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.529+0000 7f25e2c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc1099e0 0x7f25dc075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.529+0000 7f25e2c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1136b0 con 0x7f25dc1099e0 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.529+0000 7f25e2c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc113530 con 0x7f25dc103750 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.529+0000 7f25e2c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc113830 con 0x7f25dc109090 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.529+0000 7f25e1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc103750 0x7f25dc0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.530+0000 7f25e1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc103750 0x7f25dc0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49548/0 (socket says 172.21.15.32:49548) 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.530+0000 7f25e2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc1099e0 0x7f25dc075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.530+0000 7f25e1800640 1 -- 172.21.15.32:0/3944704537 learned_addr learned my addr 172.21.15.32:0/3944704537 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.530+0000 7f25e0e00640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25dc109090 0x7f25dc0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.530+0000 7f25e1800640 1 -- 172.21.15.32:0/3944704537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25dc109090 msgr2=0x7f25dc0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:20.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.530+0000 7f25e1800640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25dc109090 0x7f25dc0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.530+0000 7f25e1800640 1 -- 172.21.15.32:0/3944704537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc1099e0 msgr2=0x7f25dc075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:20.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.530+0000 7f25e1800640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc1099e0 0x7f25dc075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.530+0000 7f25e1800640 1 -- 172.21.15.32:0/3944704537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc1a8360 con 0x7f25dc103750 2024-08-31T07:18:20.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.530+0000 7f25e2200640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc1099e0 0x7f25dc075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:20.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.531+0000 7f25e1800640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc103750 0x7f25dc0754b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25d0005940 tx=0x7f25d00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:20.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.531+0000 7f25da000640 1 -- 172.21.15.32:0/3944704537 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d003f030 con 0x7f25dc103750 2024-08-31T07:18:20.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.531+0000 7f25da000640 1 -- 172.21.15.32:0/3944704537 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d00044c0 con 0x7f25dc103750 2024-08-31T07:18:20.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.531+0000 7f25e2c00640 1 -- 172.21.15.32:0/3944704537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc1a85f0 con 0x7f25dc103750 2024-08-31T07:18:20.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.531+0000 7f25da000640 1 -- 172.21.15.32:0/3944704537 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0035070 con 0x7f25dc103750 2024-08-31T07:18:20.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.531+0000 7f25e2c00640 1 -- 172.21.15.32:0/3944704537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc1a8ad0 con 0x7f25dc103750 2024-08-31T07:18:20.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.535+0000 7f25eae38740 1 Processor -- start 2024-08-31T07:18:20.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.535+0000 7f25eae38740 1 -- start start 2024-08-31T07:18:20.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25eae38740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 0x55b453316b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25eae38740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b4533170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25eae38740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f50390 0x55b4533175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25eae38740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b45328d7b0 con 0x55b452f50390 2024-08-31T07:18:20.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25eae38740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b45327cd40 con 0x55b452f481b0 2024-08-31T07:18:20.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25eae38740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b453280cf0 con 0x55b452f71070 2024-08-31T07:18:20.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25e0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f50390 0x55b4533175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25e2200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 0x55b453316b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25e2200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 0x55b453316b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54970/0 (socket says 172.21.15.32:54970) 2024-08-31T07:18:20.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25e2200640 1 -- 172.21.15.32:0/1177182885 learned_addr learned my addr 172.21.15.32:0/1177182885 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:20.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.536+0000 7f25da000640 1 -- 172.21.15.32:0/3944704537 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f25d0042030 con 0x7f25dc103750 2024-08-31T07:18:20.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.537+0000 7f25e1800640 1 --2- 172.21.15.32:0/1177182885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b4533170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.537+0000 7f25e2200640 1 -- 172.21.15.32:0/1177182885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 msgr2=0x55b4533170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:20.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.537+0000 7f25e2200640 1 --2- 172.21.15.32:0/1177182885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b4533170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.537+0000 7f25e2200640 1 -- 172.21.15.32:0/1177182885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f50390 msgr2=0x55b4533175e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:20.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.537+0000 7f25e2200640 1 --2- 172.21.15.32:0/1177182885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f50390 0x55b4533175e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.537+0000 7f25e2200640 1 -- 172.21.15.32:0/1177182885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b452fdc650 con 0x55b452f71070 2024-08-31T07:18:20.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.537+0000 7f25e0e00640 1 --2- 172.21.15.32:0/1177182885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f50390 0x55b4533175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:20.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.537+0000 7f25e1800640 1 --2- 172.21.15.32:0/1177182885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b4533170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:18:20.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.538+0000 7f25e2200640 1 --2- 172.21.15.32:0/1177182885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 0x55b453316b60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f25d40117e0 tx=0x7f25d4011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a044ce27896dfc8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:20.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.538+0000 7f25cea00640 1 -- 172.21.15.32:0/1177182885 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4011310 con 0x55b452f71070 2024-08-31T07:18:20.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.538+0000 7f25cea00640 1 -- 172.21.15.32:0/1177182885 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d4003e20 con 0x55b452f71070 2024-08-31T07:18:20.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.538+0000 7f25cea00640 1 -- 172.21.15.32:0/1177182885 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d40044f0 con 0x55b452f71070 2024-08-31T07:18:20.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.539+0000 7f25da000640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f25c40b4ca0 0x7f25c40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.539+0000 7f25da000640 1 -- 172.21.15.32:0/3944704537 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f25d0105870 con 0x7f25dc103750 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.539+0000 7f25e1800640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f25c40b4ca0 0x7f25c40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.539+0000 7f25eae38740 1 -- 172.21.15.32:0/1177182885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 msgr2=0x55b453316b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.539+0000 7f25eae38740 1 --2- 172.21.15.32:0/1177182885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 0x55b453316b60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f25d40117e0 tx=0x7f25d4011ca0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.539+0000 7f25eae38740 1 -- 172.21.15.32:0/1177182885 shutdown_connections 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.539+0000 7f25eae38740 1 --2- 172.21.15.32:0/1177182885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f50390 0x55b4533175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.539+0000 7f25eae38740 1 --2- 172.21.15.32:0/1177182885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b4533170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.539+0000 7f25eae38740 1 --2- 172.21.15.32:0/1177182885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 0x55b453316b60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.539+0000 7f25eae38740 1 -- 172.21.15.32:0/1177182885 >> 172.21.15.32:0/1177182885 conn(0x55b452f65460 msgr2=0x55b453314a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.540+0000 7f25e1800640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f25c40b4ca0 0x7f25c40b71b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55b4533157e0 tx=0x7f25d002a3f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.540+0000 7f25eae38740 1 -- 172.21.15.32:0/1177182885 shutdown_connections 2024-08-31T07:18:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.540+0000 7f25eae38740 1 -- 172.21.15.32:0/1177182885 wait complete. 2024-08-31T07:18:20.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.540+0000 7f25eae38740 1 -- start start 2024-08-31T07:18:20.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.541+0000 7f25eae38740 1 Processor -- start 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.541+0000 7f25eae38740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b453322790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.541+0000 7f25eae38740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 0x55b453322d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.541+0000 7f25eae38740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f71070 0x55b45332e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.541+0000 7f25eae38740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b4530df700 con 0x55b452f71070 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.541+0000 7f25eae38740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b453203cf0 con 0x55b452f481b0 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.541+0000 7f25eae38740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b453325340 con 0x55b452f50390 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.541+0000 7f25eae38740 10 client.0 _ll_register_callbacks cb 0x55b453229470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.541+0000 7f25e2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b453322790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25e2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b453322790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49564/0 (socket says 172.21.15.32:49564) 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25e2200640 1 -- 172.21.15.32:0/3273686871 learned_addr learned my addr 172.21.15.32:0/3273686871 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25e2200640 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f71070 0x55b45332e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25e0e00640 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 0x55b453322d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25e2200640 1 -- 172.21.15.32:0/3273686871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 msgr2=0x55b453322d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25e2200640 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 0x55b453322d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25e2200640 1 -- 172.21.15.32:0/3273686871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f71070 msgr2=0x55b45332e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25e2200640 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f71070 0x55b45332e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25e2200640 1 -- 172.21.15.32:0/3273686871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4532448b0 con 0x55b452f481b0 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25e2200640 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b453322790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f25d40115f0 tx=0x7f25d40040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=499981475388059d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.542+0000 7f25c3400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.543+0000 7f25c3400640 1 -- 172.21.15.32:0/3273686871 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d401e020 con 0x55b452f481b0 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.543+0000 7f25c3400640 1 -- 172.21.15.32:0/3273686871 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f25d4004a40 con 0x55b452f481b0 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.543+0000 7f25eae38740 1 -- 172.21.15.32:0/3273686871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d4003e20 con 0x55b452f481b0 2024-08-31T07:18:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.543+0000 7f25c3400640 1 -- 172.21.15.32:0/3273686871 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f25d4033050 con 0x55b452f481b0 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.543+0000 7f25eae38740 10 client.26714 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.543+0000 7f25eae38740 1 -- 172.21.15.32:0/3273686871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b453331400 con 0x55b452f481b0 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.543+0000 7f25c3400640 1 -- 172.21.15.32:0/3273686871 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d4003e20 con 0x55b452f481b0 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.544+0000 7f25c3400640 1 -- 172.21.15.32:0/3273686871 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f25d4018020 con 0x55b452f481b0 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.544+0000 7f25eae38740 10 client.26714 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.544+0000 7f25eae38740 10 client.26714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.544+0000 7f25eae38740 20 client.26714 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.544+0000 7f25eae38740 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55b452f2f9f0 0x55b453334760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.544+0000 7f25eae38740 4 client.26714 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.544+0000 7f25eae38740 1 -- 172.21.15.32:0/3273686871 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b45328f700 con 0x55b452f2f9f0 2024-08-31T07:18:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.544+0000 7f25e0e00640 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55b452f2f9f0 0x55b453334760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.545+0000 7f25e0e00640 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55b452f2f9f0 0x55b453334760 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac5e670222028a93 server_cookie=551b1ef2faaea923 in_seq=0 out_seq=0 2024-08-31T07:18:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.545+0000 7f25c3400640 10 client.26714 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.545+0000 7f25c3400640 1 -- 172.21.15.32:0/3273686871 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b45328f700 con 0x55b452f2f9f0 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25c3400640 10 client.26714 handle_command_reply: tid=0 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 client.26714 shutdown 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 -- 172.21.15.32:0/3273686871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 msgr2=0x55b453322790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b453322790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f25d40115f0 tx=0x7f25d40040d0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 -- 172.21.15.32:0/3273686871 shutdown_connections 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55b452f2f9f0 0x55b453334760 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f71070 0x55b45332e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25c3400640 0 client.26714 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 0x55b453322d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 --2- 172.21.15.32:0/3273686871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f481b0 0x55b453322790 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 -- 172.21.15.32:0/3273686871 >> 172.21.15.32:0/3273686871 conn(0x55b452f65460 msgr2=0x55b453319cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 -- 172.21.15.32:0/3273686871 shutdown_connections 2024-08-31T07:18:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 1 -- 172.21.15.32:0/3273686871 wait complete. 2024-08-31T07:18:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.546+0000 7f25eae38740 20 client.26714 trim_cache size 0 max 16384 2024-08-31T07:18:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.560+0000 7f25eae38740 1 Processor -- start 2024-08-31T07:18:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.560+0000 7f25eae38740 1 -- start start 2024-08-31T07:18:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.561+0000 7f25eae38740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 0x55b45331c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.561+0000 7f25eae38740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 0x55b453314000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.561+0000 7f25eae38740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f71070 0x55b453314540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.561+0000 7f25eae38740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1b1380 con 0x55b452f481b0 2024-08-31T07:18:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.561+0000 7f25eae38740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1b2ec0 con 0x55b452f71070 2024-08-31T07:18:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.561+0000 7f25eae38740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1b1a50 con 0x55b452f50390 2024-08-31T07:18:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.561+0000 7f25e2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 0x55b45331c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.561+0000 7f25e2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 0x55b45331c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43230/0 (socket says 172.21.15.32:43230) 2024-08-31T07:18:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.561+0000 7f25e2200640 1 -- 172.21.15.32:0/1274585980 learned_addr learned my addr 172.21.15.32:0/1274585980 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.561+0000 7f25e0e00640 1 --2- 172.21.15.32:0/1274585980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 0x55b453314000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25e2200640 1 --2- 172.21.15.32:0/1274585980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f71070 0x55b453314540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25e2200640 1 -- 172.21.15.32:0/1274585980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 msgr2=0x55b453314000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25e2200640 1 --2- 172.21.15.32:0/1274585980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 0x55b453314000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25e2200640 1 -- 172.21.15.32:0/1274585980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f71070 msgr2=0x55b453314540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25e2200640 1 --2- 172.21.15.32:0/1274585980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f71070 0x55b453314540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25e2200640 1 -- 172.21.15.32:0/1274585980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b453324cd0 con 0x55b452f481b0 2024-08-31T07:18:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25e0e00640 1 --2- 172.21.15.32:0/1274585980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 0x55b453314000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25e2200640 1 --2- 172.21.15.32:0/1274585980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 0x55b45331c380 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f25d401ab50 tx=0x7f25d40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc00c4953cf88be server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25ce000640 1 -- 172.21.15.32:0/1274585980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d401e070 con 0x55b452f481b0 2024-08-31T07:18:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25ce000640 1 -- 172.21.15.32:0/1274585980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d400fcf0 con 0x55b452f481b0 2024-08-31T07:18:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.562+0000 7f25ce000640 1 -- 172.21.15.32:0/1274585980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4025040 con 0x55b452f481b0 2024-08-31T07:18:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.563+0000 7f25eae38740 1 -- 172.21.15.32:0/1274585980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 msgr2=0x55b45331c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.563+0000 7f25eae38740 1 --2- 172.21.15.32:0/1274585980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 0x55b45331c380 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f25d401ab50 tx=0x7f25d40021d0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.563+0000 7f25eae38740 1 -- 172.21.15.32:0/1274585980 shutdown_connections 2024-08-31T07:18:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.563+0000 7f25eae38740 1 --2- 172.21.15.32:0/1274585980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f71070 0x55b453314540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.563+0000 7f25eae38740 1 --2- 172.21.15.32:0/1274585980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f50390 0x55b453314000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.563+0000 7f25eae38740 1 --2- 172.21.15.32:0/1274585980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 0x55b45331c380 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.563+0000 7f25eae38740 1 -- 172.21.15.32:0/1274585980 >> 172.21.15.32:0/1274585980 conn(0x55b452f65460 msgr2=0x55b45331bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:20.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.564+0000 7f25eae38740 1 -- 172.21.15.32:0/1274585980 shutdown_connections 2024-08-31T07:18:20.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.564+0000 7f25eae38740 1 -- 172.21.15.32:0/1274585980 wait complete. 2024-08-31T07:18:20.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.564+0000 7f25eae38740 1 -- start start 2024-08-31T07:18:20.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.564+0000 7f25eae38740 1 Processor -- start 2024-08-31T07:18:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25eae38740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 0x55b45332ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25eae38740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f50390 0x55b4533270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25eae38740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 0x55b453327630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25eae38740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1b27f0 con 0x55b452f481b0 2024-08-31T07:18:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25eae38740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1b2120 con 0x55b452f50390 2024-08-31T07:18:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25eae38740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1af360 con 0x55b452f71070 2024-08-31T07:18:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25eae38740 10 client.0 _ll_register_callbacks cb 0x7f25d400f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25e0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f50390 0x55b4533270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25e2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 0x55b45332ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25e0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f50390 0x55b4533270f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49586/0 (socket says 172.21.15.32:49586) 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25e0e00640 1 -- 172.21.15.32:0/607145764 learned_addr learned my addr 172.21.15.32:0/607145764 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.565+0000 7f25e2200640 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 0x55b453327630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.566+0000 7f25e0e00640 1 -- 172.21.15.32:0/607145764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 msgr2=0x55b453327630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.566+0000 7f25e0e00640 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 0x55b453327630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.566+0000 7f25e0e00640 1 -- 172.21.15.32:0/607145764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 msgr2=0x55b45332ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.566+0000 7f25e0e00640 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 0x55b45332ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.566+0000 7f25e0e00640 1 -- 172.21.15.32:0/607145764 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b453327c40 con 0x55b452f50390 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.566+0000 7f25e0e00640 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f50390 0x55b4533270f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55b453334d90 tx=0x7f25c80039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269413036c88f462 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.566+0000 7f25c3400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.566+0000 7f25c3400640 1 -- 172.21.15.32:0/607145764 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c802a060 con 0x55b452f50390 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.566+0000 7f25c3400640 1 -- 172.21.15.32:0/607145764 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f25c8025660 con 0x55b452f50390 2024-08-31T07:18:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.566+0000 7f25eae38740 1 -- 172.21.15.32:0/607145764 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d400fcf0 con 0x55b452f50390 2024-08-31T07:18:20.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.571+0000 7f25c3400640 1 -- 172.21.15.32:0/607145764 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f25c8025860 con 0x55b452f50390 2024-08-31T07:18:20.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.571+0000 7f25eae38740 10 client.26720 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:20.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.572+0000 7f25eae38740 1 -- 172.21.15.32:0/607145764 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b45339fe60 con 0x55b452f50390 2024-08-31T07:18:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.572+0000 7f25c3400640 1 -- 172.21.15.32:0/607145764 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d400fcf0 con 0x55b452f50390 2024-08-31T07:18:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.572+0000 7f25c3400640 1 -- 172.21.15.32:0/607145764 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f25c802fc40 con 0x55b452f50390 2024-08-31T07:18:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.572+0000 7f25eae38740 10 client.26720 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.572+0000 7f25eae38740 10 client.26720 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:18:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.572+0000 7f25eae38740 20 client.26720 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.572+0000 7f25eae38740 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55b452f04da0 0x55b4533a2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.572+0000 7f25eae38740 4 client.26720 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:18:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.572+0000 7f25eae38740 1 -- 172.21.15.32:0/607145764 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b45328b1a0 con 0x55b452f04da0 2024-08-31T07:18:20.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.572+0000 7f25e2200640 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55b452f04da0 0x55b4533a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:20.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25c3400640 10 client.26720 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25e2200640 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55b452f04da0 0x55b4533a2650 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e427f1226414eab server_cookie=9235241374d54d0 in_seq=0 out_seq=0 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25c3400640 1 -- 172.21.15.32:0/607145764 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b45328b1a0 con 0x55b452f04da0 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25c3400640 10 client.26720 handle_command_reply: tid=0 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25eae38740 1 client.26720 shutdown 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25eae38740 1 -- 172.21.15.32:0/607145764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f50390 msgr2=0x55b4533270f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25eae38740 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f50390 0x55b4533270f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55b453334d90 tx=0x7f25c80039c0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25eae38740 1 -- 172.21.15.32:0/607145764 shutdown_connections 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25eae38740 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55b452f04da0 0x55b4533a2650 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25eae38740 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b452f71070 0x55b453327630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25eae38740 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b452f50390 0x55b4533270f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25eae38740 1 --2- 172.21.15.32:0/607145764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b452f481b0 0x55b45332ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.573+0000 7f25eae38740 1 -- 172.21.15.32:0/607145764 >> 172.21.15.32:0/607145764 conn(0x55b452f65460 msgr2=0x55b453323850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:20.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.574+0000 7f25c3400640 0 client.26720 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:18:20.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.574+0000 7f25eae38740 1 -- 172.21.15.32:0/607145764 shutdown_connections 2024-08-31T07:18:20.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.574+0000 7f25eae38740 1 -- 172.21.15.32:0/607145764 wait complete. 2024-08-31T07:18:20.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.574+0000 7f25eae38740 20 client.26720 trim_cache size 0 max 16384 2024-08-31T07:18:20.576 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 -- 172.21.15.32:0/3944704537 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f25c40b4ca0 msgr2=0x7f25c40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f25c40b4ca0 0x7f25c40b71b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55b4533157e0 tx=0x7f25d002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 -- 172.21.15.32:0/3944704537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc103750 msgr2=0x7f25dc0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc103750 0x7f25dc0754b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25d0005940 tx=0x7f25d00040c0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 -- 172.21.15.32:0/3944704537 shutdown_connections 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f25c40b4ca0 0x7f25c40b71b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc1099e0 0x7f25dc075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25dc109090 0x7f25dc0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 --2- 172.21.15.32:0/3944704537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25dc103750 0x7f25dc0754b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 -- 172.21.15.32:0/3944704537 >> 172.21.15.32:0/3944704537 conn(0x7f25dc0ff1e0 msgr2=0x7f25dc101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 -- 172.21.15.32:0/3944704537 shutdown_connections 2024-08-31T07:18:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:20.575+0000 7f25e2c00640 1 -- 172.21.15.32:0/3944704537 wait complete. 2024-08-31T07:18:20.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:20 smithi032 ceph-mon[17252]: pgmap v535: 97 pgs: 97 active+clean; 102 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 269 KiB/s wr, 32 op/s 2024-08-31T07:18:20.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:20 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:18:20.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.37 MB/sec execute 129 sec latency 11601.478 ms 2024-08-31T07:18:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:20 smithi073 ceph-mon[21995]: pgmap v535: 97 pgs: 97 active+clean; 102 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 269 KiB/s wr, 32 op/s 2024-08-31T07:18:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:20 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:18:20.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:20 smithi084 ceph-mon[22387]: pgmap v535: 97 pgs: 97 active+clean; 102 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 269 KiB/s wr, 32 op/s 2024-08-31T07:18:20.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:20 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:18:21.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:18:21.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.35 MB/sec execute 130 sec latency 12601.519 ms 2024-08-31T07:18:21.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:21 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi032 to 177.5M 2024-08-31T07:18:21.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:21 smithi032 ceph-mon[17252]: Unable to set osd_memory_target on smithi032 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T07:18:21.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:18:21.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:18:21.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:18:21.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:21 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi032 to 177.5M 2024-08-31T07:18:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:21 smithi073 ceph-mon[21995]: Unable to set osd_memory_target on smithi032 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T07:18:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:18:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:18:21.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:18:21.985 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:21.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T07:18:21.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:21 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi032 to 177.5M 2024-08-31T07:18:21.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:21 smithi084 ceph-mon[22387]: Unable to set osd_memory_target on smithi032 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T07:18:21.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:18:21.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:18:21.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:18:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.682+0000 7fb35da00640 1 -- 172.21.15.32:0/3199098176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb35810fcb0 msgr2=0x7fb3581120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.682+0000 7fb35da00640 1 --2- 172.21.15.32:0/3199098176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb35810fcb0 0x7fb3581120f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb350009ca0 tx=0x7fb35002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.682+0000 7fb35da00640 1 -- 172.21.15.32:0/3199098176 shutdown_connections 2024-08-31T07:18:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.682+0000 7fb35da00640 1 --2- 172.21.15.32:0/3199098176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb35810fcb0 0x7fb3581120f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.682+0000 7fb35da00640 1 --2- 172.21.15.32:0/3199098176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb358075ea0 0x7fb35810f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.682+0000 7fb35da00640 1 --2- 172.21.15.32:0/3199098176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb358075550 0x7fb358075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.682+0000 7fb35da00640 1 -- 172.21.15.32:0/3199098176 >> 172.21.15.32:0/3199098176 conn(0x7fb3580ff1e0 msgr2=0x7fb358101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:22.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.683+0000 7fb35da00640 1 -- 172.21.15.32:0/3199098176 shutdown_connections 2024-08-31T07:18:22.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.683+0000 7fb35da00640 1 -- 172.21.15.32:0/3199098176 wait complete. 2024-08-31T07:18:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.684+0000 7fb35da00640 1 Processor -- start 2024-08-31T07:18:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.684+0000 7fb35da00640 1 -- start start 2024-08-31T07:18:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.684+0000 7fb35da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb358075550 0x7fb35819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.684+0000 7fb35da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb358075ea0 0x7fb35819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.684+0000 7fb35da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb35810fcb0 0x7fb3581a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.684+0000 7fb35da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581158b0 con 0x7fb358075ea0 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.684+0000 7fb35da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb358115730 con 0x7fb35810fcb0 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.684+0000 7fb35da00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb358115a30 con 0x7fb358075550 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.684+0000 7fb357400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb358075ea0 0x7fb35819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.685+0000 7fb357400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb358075ea0 0x7fb35819c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43254/0 (socket says 172.21.15.32:43254) 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.685+0000 7fb357400640 1 -- 172.21.15.32:0/1596025195 learned_addr learned my addr 172.21.15.32:0/1596025195 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.685+0000 7fb35d000640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb35810fcb0 0x7fb3581a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.685+0000 7fb357e00640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb358075550 0x7fb35819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.685+0000 7fb357400640 1 -- 172.21.15.32:0/1596025195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb358075550 msgr2=0x7fb35819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.685+0000 7fb357400640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb358075550 0x7fb35819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.685+0000 7fb357400640 1 -- 172.21.15.32:0/1596025195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb35810fcb0 msgr2=0x7fb3581a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.685+0000 7fb357400640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb35810fcb0 0x7fb3581a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.685+0000 7fb357400640 1 -- 172.21.15.32:0/1596025195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3581a2150 con 0x7fb358075ea0 2024-08-31T07:18:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.685+0000 7fb357400640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb358075ea0 0x7fb35819c780 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb34400e9c0 tx=0x7fb34400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.686+0000 7fb354c00640 1 -- 172.21.15.32:0/1596025195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34400cd80 con 0x7fb358075ea0 2024-08-31T07:18:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.686+0000 7fb35da00640 1 -- 172.21.15.32:0/1596025195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3581a2440 con 0x7fb358075ea0 2024-08-31T07:18:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.686+0000 7fb354c00640 1 -- 172.21.15.32:0/1596025195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3440035f0 con 0x7fb358075ea0 2024-08-31T07:18:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.686+0000 7fb354c00640 1 -- 172.21.15.32:0/1596025195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344010690 con 0x7fb358075ea0 2024-08-31T07:18:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.686+0000 7fb35da00640 1 -- 172.21.15.32:0/1596025195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3581a8ba0 con 0x7fb358075ea0 2024-08-31T07:18:22.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.689+0000 7fb365bde740 1 Processor -- start 2024-08-31T07:18:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.689+0000 7fb365bde740 1 -- start start 2024-08-31T07:18:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.689+0000 7fb365bde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 0x55ab6f662b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.689+0000 7fb365bde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 0x55ab6f6630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.689+0000 7fb354c00640 1 -- 172.21.15.32:0/1596025195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb344004000 con 0x7fb358075ea0 2024-08-31T07:18:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.689+0000 7fb365bde740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f6635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.689+0000 7fb365bde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ab6f5d97b0 con 0x55ab6f2941b0 2024-08-31T07:18:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.689+0000 7fb365bde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ab6f5c8d40 con 0x55ab6f2bd070 2024-08-31T07:18:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.689+0000 7fb365bde740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ab6f5cccf0 con 0x55ab6f29c390 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.690+0000 7fb357e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 0x55ab6f6630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.690+0000 7fb35d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 0x55ab6f662b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.690+0000 7fb357400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f6635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.690+0000 7fb35d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 0x55ab6f662b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49592/0 (socket says 172.21.15.32:49592) 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.690+0000 7fb35d000640 1 -- 172.21.15.32:0/674721919 learned_addr learned my addr 172.21.15.32:0/674721919 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.690+0000 7fb35d000640 1 -- 172.21.15.32:0/674721919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 msgr2=0x55ab6f6635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.690+0000 7fb35d000640 1 --2- 172.21.15.32:0/674721919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f6635e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.690+0000 7fb35d000640 1 -- 172.21.15.32:0/674721919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 msgr2=0x55ab6f6630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.690+0000 7fb35d000640 1 --2- 172.21.15.32:0/674721919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 0x55ab6f6630a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.690+0000 7fb35d000640 1 -- 172.21.15.32:0/674721919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6f328650 con 0x55ab6f2bd070 2024-08-31T07:18:22.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.691+0000 7fb35d000640 1 --2- 172.21.15.32:0/674721919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 0x55ab6f662b60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb350009ca0 tx=0x7fb350009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbdd35a480677cf1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.691+0000 7fb34d600640 1 -- 172.21.15.32:0/674721919 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350031cf0 con 0x55ab6f2bd070 2024-08-31T07:18:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.691+0000 7fb34d600640 1 -- 172.21.15.32:0/674721919 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb350036030 con 0x55ab6f2bd070 2024-08-31T07:18:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.691+0000 7fb34d600640 1 -- 172.21.15.32:0/674721919 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350004280 con 0x55ab6f2bd070 2024-08-31T07:18:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.691+0000 7fb354c00640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb33c0b4ca0 0x7fb33c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb354c00640 1 -- 172.21.15.32:0/1596025195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb3440e59e0 con 0x7fb358075ea0 2024-08-31T07:18:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb357e00640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb33c0b4ca0 0x7fb33c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb365bde740 1 -- 172.21.15.32:0/674721919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 msgr2=0x55ab6f662b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb365bde740 1 --2- 172.21.15.32:0/674721919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 0x55ab6f662b60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb350009ca0 tx=0x7fb350009430 comp rx=0 tx=0).stop 2024-08-31T07:18:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb365bde740 1 -- 172.21.15.32:0/674721919 shutdown_connections 2024-08-31T07:18:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb365bde740 1 --2- 172.21.15.32:0/674721919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f6635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb365bde740 1 --2- 172.21.15.32:0/674721919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 0x55ab6f6630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb365bde740 1 --2- 172.21.15.32:0/674721919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 0x55ab6f662b60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb365bde740 1 -- 172.21.15.32:0/674721919 >> 172.21.15.32:0/674721919 conn(0x55ab6f2b1460 msgr2=0x55ab6f660a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb365bde740 1 -- 172.21.15.32:0/674721919 shutdown_connections 2024-08-31T07:18:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.692+0000 7fb365bde740 1 -- 172.21.15.32:0/674721919 wait complete. 2024-08-31T07:18:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.693+0000 7fb357e00640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb33c0b4ca0 0x7fb33c0b71b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55ab6f6617e0 tx=0x7fb348005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.693+0000 7fb365bde740 1 -- start start 2024-08-31T07:18:22.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.693+0000 7fb365bde740 1 Processor -- start 2024-08-31T07:18:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb365bde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 0x55ab6f66e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb365bde740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f66ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb365bde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 0x55ab6f67a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb365bde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ab6f42b700 con 0x55ab6f2bd070 2024-08-31T07:18:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb365bde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ab6f54fcf0 con 0x55ab6f2941b0 2024-08-31T07:18:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb365bde740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ab6f671160 con 0x55ab6f29c390 2024-08-31T07:18:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb365bde740 10 client.0 _ll_register_callbacks cb 0x55ab6f575470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb357400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f66ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb35d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 0x55ab6f67a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb357400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f66ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55024/0 (socket says 172.21.15.32:55024) 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb357400640 1 -- 172.21.15.32:0/520637632 learned_addr learned my addr 172.21.15.32:0/520637632 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb357e00640 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 0x55ab6f66e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb357400640 1 -- 172.21.15.32:0/520637632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 msgr2=0x55ab6f66e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb357400640 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 0x55ab6f66e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb357400640 1 -- 172.21.15.32:0/520637632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 msgr2=0x55ab6f67a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb357400640 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 0x55ab6f67a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.694+0000 7fb357400640 1 -- 172.21.15.32:0/520637632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6f5908b0 con 0x55ab6f29c390 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.695+0000 7fb357400640 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f66ecd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb3440a93c0 tx=0x7fb344003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c215b69cb6bc364f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.695+0000 7fb342000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.695+0000 7fb342000640 1 -- 172.21.15.32:0/520637632 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3440ab8c0 con 0x55ab6f29c390 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.695+0000 7fb342000640 1 -- 172.21.15.32:0/520637632 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb3440a83e0 con 0x55ab6f29c390 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.695+0000 7fb365bde740 1 -- 172.21.15.32:0/520637632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb350036030 con 0x55ab6f29c390 2024-08-31T07:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.695+0000 7fb342000640 1 -- 172.21.15.32:0/520637632 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb3440abc60 con 0x55ab6f29c390 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.695+0000 7fb365bde740 10 client.26749 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.695+0000 7fb365bde740 1 -- 172.21.15.32:0/520637632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ab6f67d150 con 0x55ab6f29c390 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.696+0000 7fb342000640 1 -- 172.21.15.32:0/520637632 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb350036030 con 0x55ab6f29c390 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.696+0000 7fb342000640 1 -- 172.21.15.32:0/520637632 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb3440042f0 con 0x55ab6f29c390 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.696+0000 7fb365bde740 10 client.26749 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.696+0000 7fb365bde740 10 client.26749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.696+0000 7fb365bde740 20 client.26749 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.696+0000 7fb365bde740 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ab6f27b9f0 0x55ab6f6805a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.696+0000 7fb365bde740 4 client.26749 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.696+0000 7fb365bde740 1 -- 172.21.15.32:0/520637632 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab6f5db700 con 0x55ab6f27b9f0 2024-08-31T07:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.696+0000 7fb357e00640 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ab6f27b9f0 0x55ab6f6805a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.697+0000 7fb357e00640 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ab6f27b9f0 0x55ab6f6805a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3711ab41f5f8f77 server_cookie=b9107ebdefa2a7f1 in_seq=0 out_seq=0 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.697+0000 7fb342000640 10 client.26749 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb342000640 1 -- 172.21.15.32:0/520637632 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab6f5db700 con 0x55ab6f27b9f0 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb342000640 10 client.26749 handle_command_reply: tid=0 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 client.26749 shutdown 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 -- 172.21.15.32:0/520637632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 msgr2=0x55ab6f66ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f66ecd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb3440a93c0 tx=0x7fb344003cd0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 -- 172.21.15.32:0/520637632 shutdown_connections 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 0x55ab6f67a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f66ecd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ab6f27b9f0 0x55ab6f6805a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 --2- 172.21.15.32:0/520637632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 0x55ab6f66e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 -- 172.21.15.32:0/520637632 >> 172.21.15.32:0/520637632 conn(0x55ab6f2b1460 msgr2=0x55ab6f665cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb342000640 0 client.26749 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:18:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 -- 172.21.15.32:0/520637632 shutdown_connections 2024-08-31T07:18:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 1 -- 172.21.15.32:0/520637632 wait complete. 2024-08-31T07:18:22.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.698+0000 7fb365bde740 20 client.26749 trim_cache size 0 max 16384 2024-08-31T07:18:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.712+0000 7fb365bde740 1 Processor -- start 2024-08-31T07:18:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.712+0000 7fb365bde740 1 -- start start 2024-08-31T07:18:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb365bde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 0x55ab6f664550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb365bde740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f668c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb365bde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 0x55ab6f66ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb365bde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581b1290 con 0x55ab6f2941b0 2024-08-31T07:18:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb365bde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581b2dd0 con 0x55ab6f2bd070 2024-08-31T07:18:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb365bde740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581b1960 con 0x55ab6f29c390 2024-08-31T07:18:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb357e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 0x55ab6f664550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb357e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 0x55ab6f664550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43286/0 (socket says 172.21.15.32:43286) 2024-08-31T07:18:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb357e00640 1 -- 172.21.15.32:0/2497541744 learned_addr learned my addr 172.21.15.32:0/2497541744 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb35d000640 1 --2- 172.21.15.32:0/2497541744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 0x55ab6f66ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.713+0000 7fb357400640 1 --2- 172.21.15.32:0/2497541744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f668c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.714+0000 7fb357e00640 1 -- 172.21.15.32:0/2497541744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 msgr2=0x55ab6f668c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.714+0000 7fb357e00640 1 --2- 172.21.15.32:0/2497541744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f668c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.714+0000 7fb357e00640 1 -- 172.21.15.32:0/2497541744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 msgr2=0x55ab6f66ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.714+0000 7fb357e00640 1 --2- 172.21.15.32:0/2497541744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 0x55ab6f66ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.714+0000 7fb357e00640 1 -- 172.21.15.32:0/2497541744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6f6691c0 con 0x55ab6f2941b0 2024-08-31T07:18:22.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.714+0000 7fb357e00640 1 --2- 172.21.15.32:0/2497541744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 0x55ab6f664550 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55ab6f670de0 tx=0x7fb348017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777f819a4a2f48fb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.714+0000 7fb34cc00640 1 -- 172.21.15.32:0/2497541744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34801dd50 con 0x55ab6f2941b0 2024-08-31T07:18:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.714+0000 7fb34cc00640 1 -- 172.21.15.32:0/2497541744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb34802b8e0 con 0x55ab6f2941b0 2024-08-31T07:18:22.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.714+0000 7fb34cc00640 1 -- 172.21.15.32:0/2497541744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480299f0 con 0x55ab6f2941b0 2024-08-31T07:18:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.715+0000 7fb365bde740 1 -- 172.21.15.32:0/2497541744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 msgr2=0x55ab6f664550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.715+0000 7fb365bde740 1 --2- 172.21.15.32:0/2497541744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 0x55ab6f664550 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55ab6f670de0 tx=0x7fb348017000 comp rx=0 tx=0).stop 2024-08-31T07:18:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.715+0000 7fb365bde740 1 -- 172.21.15.32:0/2497541744 shutdown_connections 2024-08-31T07:18:22.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.715+0000 7fb365bde740 1 --2- 172.21.15.32:0/2497541744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2bd070 0x55ab6f66ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.715+0000 7fb365bde740 1 --2- 172.21.15.32:0/2497541744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f668c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.715+0000 7fb365bde740 1 --2- 172.21.15.32:0/2497541744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2941b0 0x55ab6f664550 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.715+0000 7fb365bde740 1 -- 172.21.15.32:0/2497541744 >> 172.21.15.32:0/2497541744 conn(0x55ab6f2b1460 msgr2=0x55ab6f66f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.715+0000 7fb365bde740 1 -- 172.21.15.32:0/2497541744 shutdown_connections 2024-08-31T07:18:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.715+0000 7fb365bde740 1 -- 172.21.15.32:0/2497541744 wait complete. 2024-08-31T07:18:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.716+0000 7fb365bde740 1 -- start start 2024-08-31T07:18:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.716+0000 7fb365bde740 1 Processor -- start 2024-08-31T07:18:22.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb365bde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 0x55ab6f672b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb365bde740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f6730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb365bde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 0x55ab6f6e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb365bde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581b2700 con 0x55ab6f2bd070 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb365bde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581b2030 con 0x55ab6f2941b0 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb365bde740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581af270 con 0x55ab6f29c390 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb365bde740 10 client.0 _ll_register_callbacks cb 0x7fb344009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb357e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 0x55ab6f672b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb357e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 0x55ab6f672b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49614/0 (socket says 172.21.15.32:49614) 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb357e00640 1 -- 172.21.15.32:0/603319651 learned_addr learned my addr 172.21.15.32:0/603319651 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb35d000640 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 0x55ab6f6e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb357400640 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f6730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb357e00640 1 -- 172.21.15.32:0/603319651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 msgr2=0x55ab6f6730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:22.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb357e00640 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f6730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb357e00640 1 -- 172.21.15.32:0/603319651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 msgr2=0x55ab6f6e9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:22.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb357e00640 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 0x55ab6f6e9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb357e00640 1 -- 172.21.15.32:0/603319651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6f6ea520 con 0x55ab6f2941b0 2024-08-31T07:18:22.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.717+0000 7fb35d000640 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 0x55ab6f6e9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:18:22.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.718+0000 7fb357e00640 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 0x55ab6f672b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb348004650 tx=0x7fb34801d550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd53f3893003cd9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:22.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.718+0000 7fb342000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:18:22.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.34 MB/sec execute 131 sec latency 13601.566 ms 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.718+0000 7fb342000640 1 -- 172.21.15.32:0/603319651 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34802b630 con 0x55ab6f2941b0 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.718+0000 7fb365bde740 1 -- 172.21.15.32:0/603319651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb34802b8e0 con 0x55ab6f2941b0 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.718+0000 7fb342000640 1 -- 172.21.15.32:0/603319651 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fb34802ba90 con 0x55ab6f2941b0 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.718+0000 7fb342000640 1 -- 172.21.15.32:0/603319651 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb34802bc90 con 0x55ab6f2941b0 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.718+0000 7fb365bde740 10 client.26738 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.718+0000 7fb365bde740 1 -- 172.21.15.32:0/603319651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ab6f6eb780 con 0x55ab6f2941b0 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.718+0000 7fb342000640 1 -- 172.21.15.32:0/603319651 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb34802b8e0 con 0x55ab6f2941b0 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.719+0000 7fb342000640 1 -- 172.21.15.32:0/603319651 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb348038070 con 0x55ab6f2941b0 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.719+0000 7fb365bde740 10 client.26738 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.719+0000 7fb365bde740 10 client.26738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:18:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.719+0000 7fb365bde740 20 client.26738 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.719+0000 7fb365bde740 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ab6f250da0 0x55ab6f6ede60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.719+0000 7fb365bde740 4 client.26738 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.719+0000 7fb365bde740 1 -- 172.21.15.32:0/603319651 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab6f6653f0 con 0x55ab6f250da0 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.719+0000 7fb357400640 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ab6f250da0 0x55ab6f6ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb357400640 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ab6f250da0 0x55ab6f6ede60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a9d67677d6e72a server_cookie=9ca9b8fca2bb5c60 in_seq=0 out_seq=0 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb342000640 10 client.26738 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb342000640 1 -- 172.21.15.32:0/603319651 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab6f6653f0 con 0x55ab6f250da0 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb342000640 10 client.26738 handle_command_reply: tid=0 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb365bde740 1 client.26738 shutdown 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb365bde740 1 -- 172.21.15.32:0/603319651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 msgr2=0x55ab6f672b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb365bde740 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 0x55ab6f672b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb348004650 tx=0x7fb34801d550 comp rx=0 tx=0).stop 2024-08-31T07:18:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb365bde740 1 -- 172.21.15.32:0/603319651 shutdown_connections 2024-08-31T07:18:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb365bde740 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab6f2bd070 0x55ab6f6e9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb365bde740 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ab6f29c390 0x55ab6f6730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb365bde740 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ab6f250da0 0x55ab6f6ede60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb365bde740 1 --2- 172.21.15.32:0/603319651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab6f2941b0 0x55ab6f672b80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb342000640 0 client.26738 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:18:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.720+0000 7fb365bde740 1 -- 172.21.15.32:0/603319651 >> 172.21.15.32:0/603319651 conn(0x55ab6f2b1460 msgr2=0x55ab6f670210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb365bde740 1 -- 172.21.15.32:0/603319651 shutdown_connections 2024-08-31T07:18:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb365bde740 1 -- 172.21.15.32:0/603319651 wait complete. 2024-08-31T07:18:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb365bde740 20 client.26738 trim_cache size 0 max 16384 2024-08-31T07:18:22.729 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:18:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb35da00640 1 -- 172.21.15.32:0/1596025195 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb33c0b4ca0 msgr2=0x7fb33c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb35da00640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb33c0b4ca0 0x7fb33c0b71b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55ab6f6617e0 tx=0x7fb348005f90 comp rx=0 tx=0).stop 2024-08-31T07:18:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb35da00640 1 -- 172.21.15.32:0/1596025195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb358075ea0 msgr2=0x7fb35819c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb35da00640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb358075ea0 0x7fb35819c780 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb34400e9c0 tx=0x7fb34400ee80 comp rx=0 tx=0).stop 2024-08-31T07:18:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb35da00640 1 -- 172.21.15.32:0/1596025195 shutdown_connections 2024-08-31T07:18:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb35da00640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb33c0b4ca0 0x7fb33c0b71b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb35da00640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb35810fcb0 0x7fb3581a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb35da00640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb358075ea0 0x7fb35819c780 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb35da00640 1 --2- 172.21.15.32:0/1596025195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb358075550 0x7fb35819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.721+0000 7fb35da00640 1 -- 172.21.15.32:0/1596025195 >> 172.21.15.32:0/1596025195 conn(0x7fb3580ff1e0 msgr2=0x7fb3581097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.722+0000 7fb35da00640 1 -- 172.21.15.32:0/1596025195 shutdown_connections 2024-08-31T07:18:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:22.722+0000 7fb35da00640 1 -- 172.21.15.32:0/1596025195 wait complete. 2024-08-31T07:18:22.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:22 smithi073 ceph-mon[21995]: pgmap v536: 97 pgs: 97 active+clean; 104 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 437 KiB/s wr, 37 op/s 2024-08-31T07:18:22.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:22 smithi084 ceph-mon[22387]: pgmap v536: 97 pgs: 97 active+clean; 104 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 437 KiB/s wr, 37 op/s 2024-08-31T07:18:23.094 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:22 smithi032 ceph-mon[17252]: pgmap v536: 97 pgs: 97 active+clean; 104 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 437 KiB/s wr, 37 op/s 2024-08-31T07:18:23.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:18:23.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.32 MB/sec execute 132 sec latency 14601.614 ms 2024-08-31T07:18:23.937 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:24.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:24 smithi032 ceph-mon[17252]: pgmap v537: 97 pgs: 97 active+clean; 105 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 656 KiB/s wr, 38 op/s 2024-08-31T07:18:24.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.30 MB/sec execute 133 sec latency 15601.652 ms 2024-08-31T07:18:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.887+0000 7f40b5200640 1 -- 172.21.15.32:0/423711744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b00754f0 msgr2=0x7f40b0075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.887+0000 7f40b5200640 1 --2- 172.21.15.32:0/423711744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b00754f0 0x7f40b0075900 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.887+0000 7f40b5200640 1 -- 172.21.15.32:0/423711744 shutdown_connections 2024-08-31T07:18:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.887+0000 7f40b5200640 1 --2- 172.21.15.32:0/423711744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40b010fcf0 0x7f40b0112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.887+0000 7f40b5200640 1 --2- 172.21.15.32:0/423711744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0075e40 0x7f40b010f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.887+0000 7f40b5200640 1 --2- 172.21.15.32:0/423711744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b00754f0 0x7f40b0075900 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.887+0000 7f40b5200640 1 -- 172.21.15.32:0/423711744 >> 172.21.15.32:0/423711744 conn(0x7f40b00ff180 msgr2=0x7f40b01015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.887+0000 7f40b5200640 1 -- 172.21.15.32:0/423711744 shutdown_connections 2024-08-31T07:18:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.887+0000 7f40b5200640 1 -- 172.21.15.32:0/423711744 wait complete. 2024-08-31T07:18:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.888+0000 7f40b5200640 1 Processor -- start 2024-08-31T07:18:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40b5200640 1 -- start start 2024-08-31T07:18:24.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40b5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b00754f0 0x7f40b019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40b5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b0075e40 0x7f40b019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40b5200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40b010fcf0 0x7f40b01a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40b5200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0115860 con 0x7f40b0075e40 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40b5200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01156e0 con 0x7f40b00754f0 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40b5200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01159e0 con 0x7f40b010fcf0 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40a6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b0075e40 0x7f40b019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40a6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b0075e40 0x7f40b019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43308/0 (socket says 172.21.15.32:43308) 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40a6a00640 1 -- 172.21.15.32:0/2238733534 learned_addr learned my addr 172.21.15.32:0/2238733534 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40af400640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b00754f0 0x7f40b019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.889+0000 7f40afe00640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40b010fcf0 0x7f40b01a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.890+0000 7f40a6a00640 1 -- 172.21.15.32:0/2238733534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40b010fcf0 msgr2=0x7f40b01a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.890+0000 7f40a6a00640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40b010fcf0 0x7f40b01a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.890+0000 7f40a6a00640 1 -- 172.21.15.32:0/2238733534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b00754f0 msgr2=0x7f40b019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.890+0000 7f40a6a00640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b00754f0 0x7f40b019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.890+0000 7f40a6a00640 1 -- 172.21.15.32:0/2238733534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b01a21c0 con 0x7f40b0075e40 2024-08-31T07:18:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.890+0000 7f40a6a00640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b0075e40 0x7f40b019c7f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f409800e970 tx=0x7f409800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.890+0000 7f40acc00640 1 -- 172.21.15.32:0/2238733534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409800cd80 con 0x7f40b0075e40 2024-08-31T07:18:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.890+0000 7f40acc00640 1 -- 172.21.15.32:0/2238733534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40980035f0 con 0x7f40b0075e40 2024-08-31T07:18:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.891+0000 7f40b5200640 1 -- 172.21.15.32:0/2238733534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b01a24b0 con 0x7f40b0075e40 2024-08-31T07:18:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.891+0000 7f40b5200640 1 -- 172.21.15.32:0/2238733534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b01a8c10 con 0x7f40b0075e40 2024-08-31T07:18:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.891+0000 7f40acc00640 1 -- 172.21.15.32:0/2238733534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098010690 con 0x7f40b0075e40 2024-08-31T07:18:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.894+0000 7f40bd458740 1 Processor -- start 2024-08-31T07:18:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.894+0000 7f40bd458740 1 -- start start 2024-08-31T07:18:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.894+0000 7f40bd458740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59cc5070 0x55ad5a06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.894+0000 7f40bd458740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.894+0000 7f40bd458740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59ca4390 0x55ad5a06b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.894+0000 7f40bd458740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ad59fe17b0 con 0x55ad59c9c1b0 2024-08-31T07:18:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.894+0000 7f40bd458740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ad59fd0d40 con 0x55ad59ca4390 2024-08-31T07:18:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.894+0000 7f40bd458740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ad59fd4cf0 con 0x55ad59cc5070 2024-08-31T07:18:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.894+0000 7f40acc00640 1 -- 172.21.15.32:0/2238733534 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4098003aa0 con 0x7f40b0075e40 2024-08-31T07:18:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.895+0000 7f40af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.895+0000 7f40af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43312/0 (socket says 172.21.15.32:43312) 2024-08-31T07:18:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.895+0000 7f40af400640 1 -- 172.21.15.32:0/2688172922 learned_addr learned my addr 172.21.15.32:0/2688172922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.895+0000 7f40afe00640 1 --2- 172.21.15.32:0/2688172922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59cc5070 0x55ad5a06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.895+0000 7f40a6a00640 1 --2- 172.21.15.32:0/2688172922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59ca4390 0x55ad5a06b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.895+0000 7f40af400640 1 -- 172.21.15.32:0/2688172922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59cc5070 msgr2=0x55ad5a06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.895+0000 7f40af400640 1 --2- 172.21.15.32:0/2688172922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59cc5070 0x55ad5a06ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.895+0000 7f40af400640 1 -- 172.21.15.32:0/2688172922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59ca4390 msgr2=0x55ad5a06b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.895+0000 7f40af400640 1 --2- 172.21.15.32:0/2688172922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59ca4390 0x55ad5a06b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.895+0000 7f40af400640 1 -- 172.21.15.32:0/2688172922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad59d30650 con 0x55ad59c9c1b0 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.896+0000 7f40af400640 1 --2- 172.21.15.32:0/2688172922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06b0a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a0032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb4ecca217be763 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.896+0000 7f40a4c00640 1 -- 172.21.15.32:0/2688172922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0040070 con 0x55ad59c9c1b0 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.896+0000 7f40a4c00640 1 -- 172.21.15.32:0/2688172922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a0004030 con 0x55ad59c9c1b0 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.896+0000 7f40a4c00640 1 -- 172.21.15.32:0/2688172922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0035040 con 0x55ad59c9c1b0 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.896+0000 7f40acc00640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40940b4d20 0x7f40940b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40acc00640 1 -- 172.21.15.32:0/2238733534 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f40980e5750 con 0x7f40b0075e40 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40afe00640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40940b4d20 0x7f40940b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40bd458740 1 -- 172.21.15.32:0/2688172922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 msgr2=0x55ad5a06b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40bd458740 1 --2- 172.21.15.32:0/2688172922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06b0a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a0032b90 comp rx=0 tx=0).stop 2024-08-31T07:18:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40bd458740 1 -- 172.21.15.32:0/2688172922 shutdown_connections 2024-08-31T07:18:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40bd458740 1 --2- 172.21.15.32:0/2688172922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59ca4390 0x55ad5a06b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40bd458740 1 --2- 172.21.15.32:0/2688172922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06b0a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40bd458740 1 --2- 172.21.15.32:0/2688172922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59cc5070 0x55ad5a06ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40bd458740 1 -- 172.21.15.32:0/2688172922 >> 172.21.15.32:0/2688172922 conn(0x55ad59cb9460 msgr2=0x55ad5a068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40bd458740 1 -- 172.21.15.32:0/2688172922 shutdown_connections 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.897+0000 7f40bd458740 1 -- 172.21.15.32:0/2688172922 wait complete. 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.898+0000 7f40afe00640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40940b4d20 0x7f40940b7230 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005fd0 tx=0x7f40a8009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.898+0000 7f40bd458740 1 -- start start 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.898+0000 7f40bd458740 1 Processor -- start 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40bd458740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40bd458740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59ca4390 0x55ad5a076c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40bd458740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a082600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40bd458740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ad59e33700 con 0x55ad59ca4390 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40bd458740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ad59f57cf0 con 0x55ad59cc5070 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40bd458740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ad5a0792e0 con 0x55ad59c9c1b0 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40bd458740 10 client.0 _ll_register_callbacks cb 0x55ad59f7d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40af400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40a6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59ca4390 0x55ad5a076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40af400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55074/0 (socket says 172.21.15.32:55074) 2024-08-31T07:18:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40af400640 1 -- 172.21.15.32:0/3290192739 learned_addr learned my addr 172.21.15.32:0/3290192739 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40afe00640 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a082600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40af400640 1 -- 172.21.15.32:0/3290192739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 msgr2=0x55ad5a082600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40af400640 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a082600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40af400640 1 -- 172.21.15.32:0/3290192739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59ca4390 msgr2=0x55ad5a076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40af400640 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59ca4390 0x55ad5a076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.899+0000 7f40af400640 1 -- 172.21.15.32:0/3290192739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad59f988b0 con 0x55ad59c9c1b0 2024-08-31T07:18:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.900+0000 7f40af400640 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0766e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f40a0032fb0 tx=0x7f40a002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7927f00e759f8f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.900+0000 7f409d600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.900+0000 7f409d600640 1 -- 172.21.15.32:0/3290192739 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0040070 con 0x55ad59c9c1b0 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.900+0000 7f409d600640 1 -- 172.21.15.32:0/3290192739 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f40a0005260 con 0x55ad59c9c1b0 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.900+0000 7f40bd458740 1 -- 172.21.15.32:0/3290192739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a0004030 con 0x55ad59c9c1b0 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.900+0000 7f409d600640 1 -- 172.21.15.32:0/3290192739 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f40a0035040 con 0x55ad59c9c1b0 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.901+0000 7f40bd458740 10 client.26767 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.901+0000 7f40bd458740 1 -- 172.21.15.32:0/3290192739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ad5a085280 con 0x55ad59c9c1b0 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.901+0000 7f409d600640 1 -- 172.21.15.32:0/3290192739 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a0004030 con 0x55ad59c9c1b0 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.901+0000 7f409d600640 1 -- 172.21.15.32:0/3290192739 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f40a0004ba0 con 0x55ad59c9c1b0 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.901+0000 7f40bd458740 10 client.26767 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.901+0000 7f40bd458740 10 client.26767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.901+0000 7f40bd458740 20 client.26767 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.901+0000 7f40bd458740 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ad59c839f0 0x55ad5a0885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.901+0000 7f40bd458740 4 client.26767 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:18:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.901+0000 7f40bd458740 1 -- 172.21.15.32:0/3290192739 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad59fe3700 con 0x55ad59c839f0 2024-08-31T07:18:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.902+0000 7f40a6a00640 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ad59c839f0 0x55ad5a0885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.902+0000 7f40a6a00640 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ad59c839f0 0x55ad5a0885e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94dc115fa485d8e server_cookie=a6e790a0bdf673e in_seq=0 out_seq=0 2024-08-31T07:18:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.902+0000 7f409d600640 10 client.26767 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.903+0000 7f409d600640 1 -- 172.21.15.32:0/3290192739 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad59fe3700 con 0x55ad59c839f0 2024-08-31T07:18:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.903+0000 7f409d600640 10 client.26767 handle_command_reply: tid=0 2024-08-31T07:18:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.903+0000 7f40bd458740 1 client.26767 shutdown 2024-08-31T07:18:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 1 -- 172.21.15.32:0/3290192739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 msgr2=0x55ad5a0766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0766e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f40a0032fb0 tx=0x7f40a002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 1 -- 172.21.15.32:0/3290192739 shutdown_connections 2024-08-31T07:18:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ad59c839f0 0x55ad5a0885e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a082600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59ca4390 0x55ad5a076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 1 --2- 172.21.15.32:0/3290192739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0766e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f409d600640 0 client.26767 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 1 -- 172.21.15.32:0/3290192739 >> 172.21.15.32:0/3290192739 conn(0x55ad59cb9460 msgr2=0x55ad5a06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 1 -- 172.21.15.32:0/3290192739 shutdown_connections 2024-08-31T07:18:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 1 -- 172.21.15.32:0/3290192739 wait complete. 2024-08-31T07:18:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.904+0000 7f40bd458740 20 client.26767 trim_cache size 0 max 16384 2024-08-31T07:18:24.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.918+0000 7f40bd458740 1 Processor -- start 2024-08-31T07:18:24.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.918+0000 7f40bd458740 1 -- start start 2024-08-31T07:18:24.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40bd458740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40bd458740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59ca4390 0x55ad5a06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40bd458740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a068000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40bd458740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01b1310 con 0x55ad59c9c1b0 2024-08-31T07:18:24.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40bd458740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01b2e50 con 0x55ad59cc5070 2024-08-31T07:18:24.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40bd458740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01b19e0 con 0x55ad59ca4390 2024-08-31T07:18:24.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43342/0 (socket says 172.21.15.32:43342) 2024-08-31T07:18:24.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40af400640 1 -- 172.21.15.32:0/648054811 learned_addr learned my addr 172.21.15.32:0/648054811 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:24.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40afe00640 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a068000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40a6a00640 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59ca4390 0x55ad5a06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.919+0000 7f40afe00640 1 -- 172.21.15.32:0/648054811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59ca4390 msgr2=0x55ad5a06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:24.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.920+0000 7f40afe00640 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59ca4390 0x55ad5a06c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.920+0000 7f40afe00640 1 -- 172.21.15.32:0/648054811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 msgr2=0x55ad5a06e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:24.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.920+0000 7f40afe00640 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.920+0000 7f40afe00640 1 -- 172.21.15.32:0/648054811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad5a0710f0 con 0x55ad59cc5070 2024-08-31T07:18:24.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.920+0000 7f40a6a00640 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59ca4390 0x55ad5a06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:24.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.920+0000 7f40af400640 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:24.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.920+0000 7f40afe00640 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a068000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f40a8011e90 tx=0x7f40a801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4745321e82a76805 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:24.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.920+0000 7f409fe00640 1 -- 172.21.15.32:0/648054811 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8004b20 con 0x55ad59cc5070 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.920+0000 7f409fe00640 1 -- 172.21.15.32:0/648054811 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a8004cc0 con 0x55ad59cc5070 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.920+0000 7f409fe00640 1 -- 172.21.15.32:0/648054811 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8005280 con 0x55ad59cc5070 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.921+0000 7f40bd458740 1 -- 172.21.15.32:0/648054811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 msgr2=0x55ad5a068000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.921+0000 7f40bd458740 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a068000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f40a8011e90 tx=0x7f40a801a000 comp rx=0 tx=0).stop 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.921+0000 7f40bd458740 1 -- 172.21.15.32:0/648054811 shutdown_connections 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.921+0000 7f40bd458740 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a068000 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.921+0000 7f40bd458740 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59ca4390 0x55ad5a06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.921+0000 7f40bd458740 1 --2- 172.21.15.32:0/648054811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59c9c1b0 0x55ad5a06e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.921+0000 7f40bd458740 1 -- 172.21.15.32:0/648054811 >> 172.21.15.32:0/648054811 conn(0x55ad59cb9460 msgr2=0x55ad5a06bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.921+0000 7f40bd458740 1 -- 172.21.15.32:0/648054811 shutdown_connections 2024-08-31T07:18:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.921+0000 7f40bd458740 1 -- 172.21.15.32:0/648054811 wait complete. 2024-08-31T07:18:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.922+0000 7f40bd458740 1 -- start start 2024-08-31T07:18:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.922+0000 7f40bd458740 1 Processor -- start 2024-08-31T07:18:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40bd458740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0895a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40bd458740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59ca4390 0x55ad5a089ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40bd458740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a08a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40bd458740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01b2780 con 0x55ad59ca4390 2024-08-31T07:18:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40bd458740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01b20b0 con 0x55ad59cc5070 2024-08-31T07:18:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40bd458740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01af2f0 con 0x55ad59c9c1b0 2024-08-31T07:18:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40bd458740 10 client.0 _ll_register_callbacks cb 0x7f40a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40af400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0895a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40af400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0895a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55084/0 (socket says 172.21.15.32:55084) 2024-08-31T07:18:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40af400640 1 -- 172.21.15.32:0/856846370 learned_addr learned my addr 172.21.15.32:0/856846370 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40a6a00640 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59ca4390 0x55ad5a089ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40afe00640 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a08a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40af400640 1 -- 172.21.15.32:0/856846370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 msgr2=0x55ad5a08a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40af400640 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a08a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40af400640 1 -- 172.21.15.32:0/856846370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59ca4390 msgr2=0x55ad5a089ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40af400640 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59ca4390 0x55ad5a089ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.923+0000 7f40af400640 1 -- 172.21.15.32:0/856846370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad5a0f1d60 con 0x55ad59c9c1b0 2024-08-31T07:18:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.924+0000 7f40af400640 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0895a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a0005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602353285dc52c6b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.924+0000 7f409d600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:18:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.924+0000 7f409d600640 1 -- 172.21.15.32:0/856846370 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0062060 con 0x55ad59c9c1b0 2024-08-31T07:18:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.924+0000 7f409d600640 1 -- 172.21.15.32:0/856846370 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f40a0003af0 con 0x55ad59c9c1b0 2024-08-31T07:18:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.924+0000 7f40bd458740 1 -- 172.21.15.32:0/856846370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a8004cc0 con 0x55ad59c9c1b0 2024-08-31T07:18:24.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.924+0000 7f409d600640 1 -- 172.21.15.32:0/856846370 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f40a0040040 con 0x55ad59c9c1b0 2024-08-31T07:18:24.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.924+0000 7f40bd458740 10 client.26773 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:24.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.924+0000 7f40bd458740 1 -- 172.21.15.32:0/856846370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ad5a0f2f70 con 0x55ad59c9c1b0 2024-08-31T07:18:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:24 smithi073 ceph-mon[21995]: pgmap v537: 97 pgs: 97 active+clean; 105 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 656 KiB/s wr, 38 op/s 2024-08-31T07:18:24.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.925+0000 7f409d600640 1 -- 172.21.15.32:0/856846370 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a8004cc0 con 0x55ad59c9c1b0 2024-08-31T07:18:24.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.925+0000 7f409d600640 1 -- 172.21.15.32:0/856846370 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f40a0031530 con 0x55ad59c9c1b0 2024-08-31T07:18:24.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.925+0000 7f40bd458740 10 client.26773 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.925+0000 7f40bd458740 10 client.26773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.925+0000 7f40bd458740 20 client.26773 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.925+0000 7f40bd458740 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ad59c58da0 0x55ad5a0f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.925+0000 7f40bd458740 4 client.26773 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.925+0000 7f40bd458740 1 -- 172.21.15.32:0/856846370 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad59fdf1a0 con 0x55ad59c58da0 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.926+0000 7f40a6a00640 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ad59c58da0 0x55ad5a0f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.926+0000 7f40a6a00640 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ad59c58da0 0x55ad5a0f56d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=137a152a1a1ef1c5 server_cookie=cf5afaaefe9768c5 in_seq=0 out_seq=0 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.926+0000 7f409d600640 10 client.26773 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.926+0000 7f409d600640 1 -- 172.21.15.32:0/856846370 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad59fdf1a0 con 0x55ad59c58da0 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.926+0000 7f409d600640 10 client.26773 handle_command_reply: tid=0 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.926+0000 7f40bd458740 1 client.26773 shutdown 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.926+0000 7f40bd458740 1 -- 172.21.15.32:0/856846370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 msgr2=0x55ad5a0895a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:24.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.926+0000 7f40bd458740 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0895a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a0005ef0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40bd458740 1 -- 172.21.15.32:0/856846370 shutdown_connections 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40bd458740 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ad59c58da0 0x55ad5a0f56d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40bd458740 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ad59cc5070 0x55ad5a08a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40bd458740 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ad59ca4390 0x55ad5a089ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40bd458740 1 --2- 172.21.15.32:0/856846370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ad59c9c1b0 0x55ad5a0895a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f409d600640 0 client.26773 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40bd458740 1 -- 172.21.15.32:0/856846370 >> 172.21.15.32:0/856846370 conn(0x55ad59cb9460 msgr2=0x55ad5a06e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40bd458740 1 -- 172.21.15.32:0/856846370 shutdown_connections 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40bd458740 1 -- 172.21.15.32:0/856846370 wait complete. 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40bd458740 20 client.26773 trim_cache size 0 max 16384 2024-08-31T07:18:24.933 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:18:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40b5200640 1 -- 172.21.15.32:0/2238733534 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40940b4d20 msgr2=0x7f40940b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40b5200640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40940b4d20 0x7f40940b7230 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005fd0 tx=0x7f40a8009500 comp rx=0 tx=0).stop 2024-08-31T07:18:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40b5200640 1 -- 172.21.15.32:0/2238733534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b0075e40 msgr2=0x7f40b019c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.927+0000 7f40b5200640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b0075e40 0x7f40b019c7f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f409800e970 tx=0x7f409800ee30 comp rx=0 tx=0).stop 2024-08-31T07:18:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.928+0000 7f40b5200640 1 -- 172.21.15.32:0/2238733534 shutdown_connections 2024-08-31T07:18:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.928+0000 7f40b5200640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40940b4d20 0x7f40940b7230 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.928+0000 7f40b5200640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40b010fcf0 0x7f40b01a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.928+0000 7f40b5200640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b0075e40 0x7f40b019c7f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.928+0000 7f40b5200640 1 --2- 172.21.15.32:0/2238733534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b00754f0 0x7f40b019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.928+0000 7f40b5200640 1 -- 172.21.15.32:0/2238733534 >> 172.21.15.32:0/2238733534 conn(0x7f40b00ff180 msgr2=0x7f40b0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.928+0000 7f40b5200640 1 -- 172.21.15.32:0/2238733534 shutdown_connections 2024-08-31T07:18:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:24.928+0000 7f40b5200640 1 -- 172.21.15.32:0/2238733534 wait complete. 2024-08-31T07:18:24.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:24 smithi084 ceph-mon[22387]: pgmap v537: 97 pgs: 97 active+clean; 105 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 656 KiB/s wr, 38 op/s 2024-08-31T07:18:25.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.28 MB/sec execute 134 sec latency 16601.764 ms 2024-08-31T07:18:26.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.27 MB/sec execute 135 sec latency 17601.903 ms 2024-08-31T07:18:26.807 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:18:26.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:18:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:26 smithi073 ceph-mon[21995]: pgmap v538: 97 pgs: 97 active+clean; 104 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 786 KiB/s wr, 38 op/s 2024-08-31T07:18:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:26 smithi084 ceph-mon[22387]: pgmap v538: 97 pgs: 97 active+clean; 104 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 786 KiB/s wr, 38 op/s 2024-08-31T07:18:27.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:26 smithi032 ceph-mon[17252]: pgmap v538: 97 pgs: 97 active+clean; 104 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 786 KiB/s wr, 38 op/s 2024-08-31T07:18:27.195 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:27.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.25 MB/sec execute 136 sec latency 18601.991 ms 2024-08-31T07:18:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.865+0000 7fe87be00640 1 -- 172.21.15.32:0/2463397090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c103750 msgr2=0x7fe87c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.865+0000 7fe87be00640 1 --2- 172.21.15.32:0/2463397090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c103750 0x7fe87c103b60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009ca0 tx=0x7fe86c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.865+0000 7fe87be00640 1 -- 172.21.15.32:0/2463397090 shutdown_connections 2024-08-31T07:18:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.865+0000 7fe87be00640 1 --2- 172.21.15.32:0/2463397090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe87c1099e0 0x7fe87c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.865+0000 7fe87be00640 1 --2- 172.21.15.32:0/2463397090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe87c109090 0x7fe87c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.865+0000 7fe87be00640 1 --2- 172.21.15.32:0/2463397090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c103750 0x7fe87c103b60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.865+0000 7fe87be00640 1 -- 172.21.15.32:0/2463397090 >> 172.21.15.32:0/2463397090 conn(0x7fe87c0ff1a0 msgr2=0x7fe87c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.866+0000 7fe87be00640 1 -- 172.21.15.32:0/2463397090 shutdown_connections 2024-08-31T07:18:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.866+0000 7fe87be00640 1 -- 172.21.15.32:0/2463397090 wait complete. 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe87be00640 1 Processor -- start 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe87be00640 1 -- start start 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe87be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe87c103750 0x7fe87c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe87be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c109090 0x7fe87c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe87be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe87c1099e0 0x7fe87c1a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe87be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c113690 con 0x7fe87c109090 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe87be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c113510 con 0x7fe87c1099e0 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe87be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c113810 con 0x7fe87c103750 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe872000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c109090 0x7fe87c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe872000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c109090 0x7fe87c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48226/0 (socket says 172.21.15.32:48226) 2024-08-31T07:18:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.867+0000 7fe872000640 1 -- 172.21.15.32:0/2803073375 learned_addr learned my addr 172.21.15.32:0/2803073375 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.868+0000 7fe87b400640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe87c1099e0 0x7fe87c1a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.868+0000 7fe87aa00640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe87c103750 0x7fe87c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.868+0000 7fe872000640 1 -- 172.21.15.32:0/2803073375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe87c103750 msgr2=0x7fe87c19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.868+0000 7fe872000640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe87c103750 0x7fe87c19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.868+0000 7fe872000640 1 -- 172.21.15.32:0/2803073375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe87c1099e0 msgr2=0x7fe87c1a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.868+0000 7fe872000640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe87c1099e0 0x7fe87c1a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.868+0000 7fe872000640 1 -- 172.21.15.32:0/2803073375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87c1a21f0 con 0x7fe87c109090 2024-08-31T07:18:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.868+0000 7fe872000640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c109090 0x7fe87c19c820 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe86400e9f0 tx=0x7fe86400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.869+0000 7fe873e00640 1 -- 172.21.15.32:0/2803073375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86400ccc0 con 0x7fe87c109090 2024-08-31T07:18:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.869+0000 7fe873e00640 1 -- 172.21.15.32:0/2803073375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8640108e0 con 0x7fe87c109090 2024-08-31T07:18:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.869+0000 7fe87be00640 1 -- 172.21.15.32:0/2803073375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87c1a24e0 con 0x7fe87c109090 2024-08-31T07:18:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.869+0000 7fe873e00640 1 -- 172.21.15.32:0/2803073375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864018860 con 0x7fe87c109090 2024-08-31T07:18:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.869+0000 7fe87be00640 1 -- 172.21.15.32:0/2803073375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe87c1a8c40 con 0x7fe87c109090 2024-08-31T07:18:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.872+0000 7fe8886d2740 1 Processor -- start 2024-08-31T07:18:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.872+0000 7fe8886d2740 1 -- start start 2024-08-31T07:18:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbf01350 0x562edc2a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2a7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbee0670 0x562edc2a8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe8886d2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562edc21d750 con 0x562edbed8490 2024-08-31T07:18:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe8886d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562edc20cce0 con 0x562edbf01350 2024-08-31T07:18:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe8886d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562edc210c90 con 0x562edbee0670 2024-08-31T07:18:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe87aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2a7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe87aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2a7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48242/0 (socket says 172.21.15.32:48242) 2024-08-31T07:18:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe87aa00640 1 -- 172.21.15.32:0/2118332169 learned_addr learned my addr 172.21.15.32:0/2118332169 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe87b400640 1 --2- 172.21.15.32:0/2118332169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbf01350 0x562edc2a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe87aa00640 1 -- 172.21.15.32:0/2118332169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbee0670 msgr2=0x562edc2a8250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:18:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe87aa00640 1 --2- 172.21.15.32:0/2118332169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbee0670 0x562edc2a8250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe87aa00640 1 -- 172.21.15.32:0/2118332169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbf01350 msgr2=0x562edc2a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe87aa00640 1 --2- 172.21.15.32:0/2118332169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbf01350 0x562edc2a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.873+0000 7fe87aa00640 1 -- 172.21.15.32:0/2118332169 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562edbf6c930 con 0x562edbed8490 2024-08-31T07:18:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.874+0000 7fe873e00640 1 -- 172.21.15.32:0/2803073375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe864018a00 con 0x7fe87c109090 2024-08-31T07:18:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.874+0000 7fe87aa00640 1 --2- 172.21.15.32:0/2118332169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2a7d10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe86c035000 tx=0x7fe86c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bf3f9390d28cc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.874+0000 7fe86be00640 1 -- 172.21.15.32:0/2118332169 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c031cf0 con 0x562edbed8490 2024-08-31T07:18:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.874+0000 7fe86be00640 1 -- 172.21.15.32:0/2118332169 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe86c036030 con 0x562edbed8490 2024-08-31T07:18:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.874+0000 7fe86be00640 1 -- 172.21.15.32:0/2118332169 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0042a0 con 0x562edbed8490 2024-08-31T07:18:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.875+0000 7fe8886d2740 1 -- 172.21.15.32:0/2118332169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 msgr2=0x562edc2a7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.875+0000 7fe8886d2740 1 --2- 172.21.15.32:0/2118332169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2a7d10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe86c035000 tx=0x7fe86c0093f0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.876+0000 7fe8886d2740 1 -- 172.21.15.32:0/2118332169 shutdown_connections 2024-08-31T07:18:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.876+0000 7fe8886d2740 1 --2- 172.21.15.32:0/2118332169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbee0670 0x562edc2a8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.876+0000 7fe8886d2740 1 --2- 172.21.15.32:0/2118332169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2a7d10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.876+0000 7fe8886d2740 1 --2- 172.21.15.32:0/2118332169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbf01350 0x562edc2a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.876+0000 7fe873e00640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8600b4d20 0x7fe8600b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.876+0000 7fe8886d2740 1 -- 172.21.15.32:0/2118332169 >> 172.21.15.32:0/2118332169 conn(0x562edbef5740 msgr2=0x562edc2a56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.876+0000 7fe873e00640 1 -- 172.21.15.32:0/2803073375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe8640e5500 con 0x7fe87c109090 2024-08-31T07:18:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.876+0000 7fe8886d2740 1 -- 172.21.15.32:0/2118332169 shutdown_connections 2024-08-31T07:18:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.876+0000 7fe8886d2740 1 -- 172.21.15.32:0/2118332169 wait complete. 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.876+0000 7fe87b400640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8600b4d20 0x7fe8600b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.877+0000 7fe8886d2740 1 -- start start 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.877+0000 7fe8886d2740 1 Processor -- start 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.877+0000 7fe87b400640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8600b4d20 0x7fe8600b7230 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe874009d60 tx=0x7fe874005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 0x562edc2bc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 0x562edc2c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe8886d2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562edc06f890 con 0x562edbed8490 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe8886d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562edc193cd0 con 0x562edbee0670 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe8886d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562edc2b5dd0 con 0x562edbf01350 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe8886d2740 10 client.0 _ll_register_callbacks cb 0x562edc207ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe872000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 0x562edc2bc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe872000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 0x562edc2bc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35902/0 (socket says 172.21.15.32:35902) 2024-08-31T07:18:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe87aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe872000640 1 -- 172.21.15.32:0/1062733979 learned_addr learned my addr 172.21.15.32:0/1062733979 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.878+0000 7fe87b400640 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 0x562edc2c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe87aa00640 1 -- 172.21.15.32:0/1062733979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 msgr2=0x562edc2c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:27.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe87aa00640 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 0x562edc2c1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe87aa00640 1 -- 172.21.15.32:0/1062733979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 msgr2=0x562edc2bc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe87aa00640 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 0x562edc2bc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe87aa00640 1 -- 172.21.15.32:0/1062733979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562edc1dd1b0 con 0x562edbed8490 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe872000640 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 0x562edc2bc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe87aa00640 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2bc110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe86c02ade0 tx=0x7fe86c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d92bfa9248190ac server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe868c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe868c00640 1 -- 172.21.15.32:0/1062733979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c04a040 con 0x562edbed8490 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe868c00640 1 -- 172.21.15.32:0/1062733979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe86c02f660 con 0x562edbed8490 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.879+0000 7fe8886d2740 1 -- 172.21.15.32:0/1062733979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe86c036030 con 0x562edbed8490 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.880+0000 7fe868c00640 1 -- 172.21.15.32:0/1062733979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe86c003970 con 0x562edbed8490 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.880+0000 7fe8886d2740 10 client.16752 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.880+0000 7fe8886d2740 1 -- 172.21.15.32:0/1062733979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562edc2ca660 con 0x562edbed8490 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.880+0000 7fe868c00640 1 -- 172.21.15.32:0/1062733979 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe86c036030 con 0x562edbed8490 2024-08-31T07:18:27.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.880+0000 7fe868c00640 1 -- 172.21.15.32:0/1062733979 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe86c04d030 con 0x562edbed8490 2024-08-31T07:18:27.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.880+0000 7fe8886d2740 10 client.16752 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:27.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.880+0000 7fe8886d2740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:18:27.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.881+0000 7fe8886d2740 20 client.16752 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:27.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.881+0000 7fe8886d2740 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562edbebfcd0 0x562edc2cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.881+0000 7fe8886d2740 4 client.16752 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:18:27.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.881+0000 7fe8886d2740 1 -- 172.21.15.32:0/1062733979 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562edc21f6a0 con 0x562edbebfcd0 2024-08-31T07:18:27.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.881+0000 7fe872000640 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562edbebfcd0 0x562edc2cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.881+0000 7fe872000640 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562edbebfcd0 0x562edc2cd9c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=565084be273539b4 server_cookie=d056daacab04e18e in_seq=0 out_seq=0 2024-08-31T07:18:27.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.882+0000 7fe868c00640 10 client.16752 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.882+0000 7fe868c00640 1 -- 172.21.15.32:0/1062733979 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562edc21f6a0 con 0x562edbebfcd0 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.882+0000 7fe868c00640 10 client.16752 handle_command_reply: tid=0 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.882+0000 7fe8886d2740 1 client.16752 shutdown 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 1 -- 172.21.15.32:0/1062733979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 msgr2=0x562edc2bc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2bc110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe86c02ade0 tx=0x7fe86c003ce0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 1 -- 172.21.15.32:0/1062733979 shutdown_connections 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562edbebfcd0 0x562edc2cd9c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 0x562edc2c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 0x562edc2bc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 1 --2- 172.21.15.32:0/1062733979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbed8490 0x562edc2bc110 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 1 -- 172.21.15.32:0/1062733979 >> 172.21.15.32:0/1062733979 conn(0x562edbef5740 msgr2=0x562edc2aa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe868c00640 0 client.16752 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:27.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 1 -- 172.21.15.32:0/1062733979 shutdown_connections 2024-08-31T07:18:27.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 1 -- 172.21.15.32:0/1062733979 wait complete. 2024-08-31T07:18:27.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.883+0000 7fe8886d2740 20 client.16752 trim_cache size 0 max 16384 2024-08-31T07:18:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.902+0000 7fe8886d2740 1 Processor -- start 2024-08-31T07:18:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.902+0000 7fe8886d2740 1 -- start start 2024-08-31T07:18:27.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbed8490 0x562edc2b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 0x562edc2b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbf01350 0x562edc2a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe8886d2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe85800e630 con 0x562edbf01350 2024-08-31T07:18:27.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe8886d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe858010170 con 0x562edbee0670 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe8886d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe85800ed00 con 0x562edbed8490 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe87b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbf01350 0x562edc2a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe87aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbed8490 0x562edc2b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe87b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbf01350 0x562edc2a8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48254/0 (socket says 172.21.15.32:48254) 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe87b400640 1 -- 172.21.15.32:0/2214979414 learned_addr learned my addr 172.21.15.32:0/2214979414 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.903+0000 7fe872000640 1 --2- 172.21.15.32:0/2214979414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 0x562edc2b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.904+0000 7fe87aa00640 1 -- 172.21.15.32:0/2214979414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 msgr2=0x562edc2b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.904+0000 7fe87aa00640 1 --2- 172.21.15.32:0/2214979414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 0x562edc2b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.904+0000 7fe87aa00640 1 -- 172.21.15.32:0/2214979414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbf01350 msgr2=0x562edc2a8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.904+0000 7fe87aa00640 1 --2- 172.21.15.32:0/2214979414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbf01350 0x562edc2a8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.904+0000 7fe87aa00640 1 -- 172.21.15.32:0/2214979414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562edc2ab8e0 con 0x562edbed8490 2024-08-31T07:18:27.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.904+0000 7fe87aa00640 1 --2- 172.21.15.32:0/2214979414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbed8490 0x562edc2b76c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009ca0 tx=0x7fe86c005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f93eb286217e402e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:27.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.904+0000 7fe86b400640 1 -- 172.21.15.32:0/2214979414 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c003b30 con 0x562edbed8490 2024-08-31T07:18:27.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.904+0000 7fe86b400640 1 -- 172.21.15.32:0/2214979414 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe86c036070 con 0x562edbed8490 2024-08-31T07:18:27.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.904+0000 7fe86b400640 1 -- 172.21.15.32:0/2214979414 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c030a30 con 0x562edbed8490 2024-08-31T07:18:27.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.905+0000 7fe8886d2740 1 -- 172.21.15.32:0/2214979414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbed8490 msgr2=0x562edc2b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:27.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.905+0000 7fe8886d2740 1 --2- 172.21.15.32:0/2214979414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbed8490 0x562edc2b76c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009ca0 tx=0x7fe86c005d80 comp rx=0 tx=0).stop 2024-08-31T07:18:27.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.905+0000 7fe8886d2740 1 -- 172.21.15.32:0/2214979414 shutdown_connections 2024-08-31T07:18:27.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.905+0000 7fe8886d2740 1 --2- 172.21.15.32:0/2214979414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbf01350 0x562edc2a8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.905+0000 7fe8886d2740 1 --2- 172.21.15.32:0/2214979414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbee0670 0x562edc2b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.905+0000 7fe8886d2740 1 --2- 172.21.15.32:0/2214979414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbed8490 0x562edc2b76c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.905+0000 7fe8886d2740 1 -- 172.21.15.32:0/2214979414 >> 172.21.15.32:0/2214979414 conn(0x562edbef5740 msgr2=0x562edc2a8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:27.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.906+0000 7fe8886d2740 1 -- 172.21.15.32:0/2214979414 shutdown_connections 2024-08-31T07:18:27.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.906+0000 7fe8886d2740 1 -- 172.21.15.32:0/2214979414 wait complete. 2024-08-31T07:18:27.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.906+0000 7fe8886d2740 1 -- start start 2024-08-31T07:18:27.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.906+0000 7fe8886d2740 1 Processor -- start 2024-08-31T07:18:27.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbed8490 0x562edc2c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbee0670 0x562edc2c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe8886d2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 0x562edc2c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe8886d2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe85800faa0 con 0x562edbee0670 2024-08-31T07:18:27.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe8886d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe85800f3d0 con 0x562edbed8490 2024-08-31T07:18:27.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe8886d2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe85800c610 con 0x562edbf01350 2024-08-31T07:18:27.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe8886d2740 10 client.0 _ll_register_callbacks cb 0x7fe874009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe87b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 0x562edc2c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe872000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbee0670 0x562edc2c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe87b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 0x562edc2c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38376/0 (socket says 172.21.15.32:38376) 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe87b400640 1 -- 172.21.15.32:0/557540977 learned_addr learned my addr 172.21.15.32:0/557540977 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.907+0000 7fe87aa00640 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbed8490 0x562edc2c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe87b400640 1 -- 172.21.15.32:0/557540977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbed8490 msgr2=0x562edc2c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe87b400640 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbed8490 0x562edc2c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe87b400640 1 -- 172.21.15.32:0/557540977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbee0670 msgr2=0x562edc2c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe87b400640 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbee0670 0x562edc2c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe87b400640 1 -- 172.21.15.32:0/557540977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562edc2c4390 con 0x562edbf01350 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe872000640 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbee0670 0x562edc2c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe87aa00640 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbed8490 0x562edc2c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe87b400640 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 0x562edc2c3ca0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe8740045f0 tx=0x7fe8740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38959718678ff3ec server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:27.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe868c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:18:27.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe868c00640 1 -- 172.21.15.32:0/557540977 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87402f060 con 0x562edbf01350 2024-08-31T07:18:27.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe868c00640 1 -- 172.21.15.32:0/557540977 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7fe87401f610 con 0x562edbf01350 2024-08-31T07:18:27.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.908+0000 7fe8886d2740 1 -- 172.21.15.32:0/557540977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe86c036070 con 0x562edbf01350 2024-08-31T07:18:27.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe868c00640 1 -- 172.21.15.32:0/557540977 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe87401f810 con 0x562edbf01350 2024-08-31T07:18:27.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe8886d2740 10 client.26788 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:27.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe8886d2740 1 -- 172.21.15.32:0/557540977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562edc338060 con 0x562edbf01350 2024-08-31T07:18:27.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe868c00640 1 -- 172.21.15.32:0/557540977 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe86c036070 con 0x562edbf01350 2024-08-31T07:18:27.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe868c00640 1 -- 172.21.15.32:0/557540977 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe874034b30 con 0x562edbf01350 2024-08-31T07:18:27.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe8886d2740 10 client.26788 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:27.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe8886d2740 10 client.26788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:18:27.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe8886d2740 20 client.26788 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:27.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe8886d2740 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562edbe95080 0x562edc33a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:27.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe8886d2740 4 client.26788 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:18:27.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.909+0000 7fe8886d2740 1 -- 172.21.15.32:0/557540977 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562edc21b140 con 0x562edbe95080 2024-08-31T07:18:27.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.910+0000 7fe87aa00640 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562edbe95080 0x562edc33a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:27.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.910+0000 7fe87aa00640 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562edbe95080 0x562edc33a880 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d14d8eca30800b92 server_cookie=76dd3bfb11a689b9 in_seq=0 out_seq=0 2024-08-31T07:18:27.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.910+0000 7fe868c00640 10 client.26788 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe868c00640 1 -- 172.21.15.32:0/557540977 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562edc21b140 con 0x562edbe95080 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe868c00640 10 client.26788 handle_command_reply: tid=0 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe8886d2740 1 client.26788 shutdown 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe8886d2740 1 -- 172.21.15.32:0/557540977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 msgr2=0x562edc2c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe8886d2740 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 0x562edc2c3ca0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe8740045f0 tx=0x7fe8740046c0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe8886d2740 1 -- 172.21.15.32:0/557540977 shutdown_connections 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe8886d2740 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562edbe95080 0x562edc33a880 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe8886d2740 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562edbf01350 0x562edc2c3ca0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe8886d2740 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562edbee0670 0x562edc2c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe8886d2740 1 --2- 172.21.15.32:0/557540977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562edbed8490 0x562edc2c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe8886d2740 1 -- 172.21.15.32:0/557540977 >> 172.21.15.32:0/557540977 conn(0x562edbef5740 msgr2=0x562edc2a8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:27.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.912+0000 7fe868c00640 0 client.26788 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:27.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.913+0000 7fe8886d2740 1 -- 172.21.15.32:0/557540977 shutdown_connections 2024-08-31T07:18:27.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.913+0000 7fe8886d2740 1 -- 172.21.15.32:0/557540977 wait complete. 2024-08-31T07:18:27.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.913+0000 7fe8886d2740 20 client.26788 trim_cache size 0 max 16384 2024-08-31T07:18:27.915 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:18:27.915 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:18:27.915 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "cb664667-a37e-4c2a-b16d-2ae314b4a7c7", 2024-08-31T07:18:27.915 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:18:27.915 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:18:27.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.913+0000 7fe87be00640 1 -- 172.21.15.32:0/2803073375 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8600b4d20 msgr2=0x7fe8600b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:27.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.913+0000 7fe87be00640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8600b4d20 0x7fe8600b7230 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe874009d60 tx=0x7fe874005f90 comp rx=0 tx=0).stop 2024-08-31T07:18:27.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.914+0000 7fe87be00640 1 -- 172.21.15.32:0/2803073375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c109090 msgr2=0x7fe87c19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:27.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.914+0000 7fe87be00640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c109090 0x7fe87c19c820 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe86400e9f0 tx=0x7fe86400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.914+0000 7fe87be00640 1 -- 172.21.15.32:0/2803073375 shutdown_connections 2024-08-31T07:18:27.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.914+0000 7fe87be00640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8600b4d20 0x7fe8600b7230 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.914+0000 7fe87be00640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe87c1099e0 0x7fe87c1a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.914+0000 7fe87be00640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe87c109090 0x7fe87c19c820 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.914+0000 7fe87be00640 1 --2- 172.21.15.32:0/2803073375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe87c103750 0x7fe87c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:27.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.914+0000 7fe87be00640 1 -- 172.21.15.32:0/2803073375 >> 172.21.15.32:0/2803073375 conn(0x7fe87c0ff1a0 msgr2=0x7fe87c1087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:27.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.914+0000 7fe87be00640 1 -- 172.21.15.32:0/2803073375 shutdown_connections 2024-08-31T07:18:27.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:27.914+0000 7fe87be00640 1 -- 172.21.15.32:0/2803073375 wait complete. 2024-08-31T07:18:28.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:18:28.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.23 MB/sec execute 137 sec latency 19602.058 ms 2024-08-31T07:18:28.916 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: pgmap v539: 97 pgs: 97 active+clean; 104 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 952 KiB/s wr, 49 op/s 2024-08-31T07:18:28.916 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.918 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.918 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:28 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: pgmap v539: 97 pgs: 97 active+clean; 104 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 952 KiB/s wr, 49 op/s 2024-08-31T07:18:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:18:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:18:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:18:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:18:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:18:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:18:28.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:18:28.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:18:28.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:28 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: pgmap v539: 97 pgs: 97 active+clean; 104 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 952 KiB/s wr, 49 op/s 2024-08-31T07:18:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:18:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:18:28.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:18:28.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:18:28.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:18:28.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:18:28.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:18:28.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:18:28.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:28.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:28 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:18:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:29.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.22 MB/sec execute 138 sec latency 20602.119 ms 2024-08-31T07:18:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.816+0000 7f3061800640 1 -- 172.21.15.32:0/3672832770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f305c10b290 msgr2=0x7f305c10b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.816+0000 7f3061800640 1 --2- 172.21.15.32:0/3672832770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f305c10b290 0x7f305c10b6a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3048009900 tx=0x7f304802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.817+0000 7f3061800640 1 -- 172.21.15.32:0/3672832770 shutdown_connections 2024-08-31T07:18:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.817+0000 7f3061800640 1 --2- 172.21.15.32:0/3672832770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c10bbe0 0x7f305c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.817+0000 7f3061800640 1 --2- 172.21.15.32:0/3672832770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f305c10b290 0x7f305c10b6a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.817+0000 7f3061800640 1 --2- 172.21.15.32:0/3672832770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c105950 0x7f305c105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.817+0000 7f3061800640 1 -- 172.21.15.32:0/3672832770 >> 172.21.15.32:0/3672832770 conn(0x7f305c101420 msgr2=0x7f305c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.817+0000 7f3061800640 1 -- 172.21.15.32:0/3672832770 shutdown_connections 2024-08-31T07:18:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.817+0000 7f3061800640 1 -- 172.21.15.32:0/3672832770 wait complete. 2024-08-31T07:18:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.818+0000 7f3061800640 1 Processor -- start 2024-08-31T07:18:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.818+0000 7f3061800640 1 -- start start 2024-08-31T07:18:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.818+0000 7f3061800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c105950 0x7f305c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.818+0000 7f3061800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f305c10b290 0x7f305c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.818+0000 7f3061800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c10bbe0 0x7f305c1a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.818+0000 7f3061800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1158f0 con 0x7f305c10b290 2024-08-31T07:18:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.818+0000 7f3061800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f305c115770 con 0x7f305c10bbe0 2024-08-31T07:18:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.818+0000 7f3061800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f305c115a70 con 0x7f305c105950 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f305b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f305c10b290 0x7f305c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f305be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c105950 0x7f305c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f305b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f305c10b290 0x7f305c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48276/0 (socket says 172.21.15.32:48276) 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f3060e00640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c10bbe0 0x7f305c1a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f305b400640 1 -- 172.21.15.32:0/499650349 learned_addr learned my addr 172.21.15.32:0/499650349 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f305be00640 1 -- 172.21.15.32:0/499650349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c10bbe0 msgr2=0x7f305c1a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f305be00640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c10bbe0 0x7f305c1a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f305be00640 1 -- 172.21.15.32:0/499650349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f305c10b290 msgr2=0x7f305c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f305be00640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f305c10b290 0x7f305c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f305be00640 1 -- 172.21.15.32:0/499650349 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c1a2230 con 0x7f305c105950 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f3060e00640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c10bbe0 0x7f305c1a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.819+0000 7f305be00640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c105950 0x7f305c19c2c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f304c00d6d0 tx=0x7f304c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.820+0000 7f3058c00640 1 -- 172.21.15.32:0/499650349 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c017070 con 0x7f305c105950 2024-08-31T07:18:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.820+0000 7f3061800640 1 -- 172.21.15.32:0/499650349 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305c1a2520 con 0x7f305c105950 2024-08-31T07:18:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.820+0000 7f3058c00640 1 -- 172.21.15.32:0/499650349 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f304c009820 con 0x7f305c105950 2024-08-31T07:18:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.820+0000 7f3058c00640 1 -- 172.21.15.32:0/499650349 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c004c40 con 0x7f305c105950 2024-08-31T07:18:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.820+0000 7f3061800640 1 -- 172.21.15.32:0/499650349 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305c1a8c80 con 0x7f305c105950 2024-08-31T07:18:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.823+0000 7f3069a0e740 1 Processor -- start 2024-08-31T07:18:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.823+0000 7f3069a0e740 1 -- start start 2024-08-31T07:18:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.823+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 0x556146fafb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.823+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fb0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.823+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fb05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.823+0000 7f3069a0e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556146f26770 con 0x556146be9390 2024-08-31T07:18:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.823+0000 7f3069a0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556146f15d00 con 0x556146c0a070 2024-08-31T07:18:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.823+0000 7f3069a0e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556146f19cb0 con 0x556146be11b0 2024-08-31T07:18:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f305b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fb05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f3060e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 0x556146fafb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f3060e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 0x556146fafb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35942/0 (socket says 172.21.15.32:35942) 2024-08-31T07:18:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f3060e00640 1 -- 172.21.15.32:0/3865085942 learned_addr learned my addr 172.21.15.32:0/3865085942 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f305b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fb05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48290/0 (socket says 172.21.15.32:48290) 2024-08-31T07:18:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f3060e00640 1 -- 172.21.15.32:0/3865085942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 msgr2=0x556146fb0060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:18:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f3060e00640 1 --2- 172.21.15.32:0/3865085942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fb0060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f3060e00640 1 -- 172.21.15.32:0/3865085942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 msgr2=0x556146fb05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f3060e00640 1 --2- 172.21.15.32:0/3865085942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fb05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f3060e00640 1 -- 172.21.15.32:0/3865085942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556146c75650 con 0x556146c0a070 2024-08-31T07:18:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.824+0000 7f3058c00640 1 -- 172.21.15.32:0/499650349 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f304c005220 con 0x7f305c105950 2024-08-31T07:18:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.825+0000 7f3060e00640 1 --2- 172.21.15.32:0/3865085942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 0x556146fafb20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f30540117e0 tx=0x7f3054011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b945ed9e570ac7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.825+0000 7f3051600640 1 -- 172.21.15.32:0/3865085942 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054011310 con 0x556146c0a070 2024-08-31T07:18:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.825+0000 7f3051600640 1 -- 172.21.15.32:0/3865085942 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3054003e20 con 0x556146c0a070 2024-08-31T07:18:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.825+0000 7f3051600640 1 -- 172.21.15.32:0/3865085942 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30540044f0 con 0x556146c0a070 2024-08-31T07:18:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3069a0e740 1 -- 172.21.15.32:0/3865085942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 msgr2=0x556146fafb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3069a0e740 1 --2- 172.21.15.32:0/3865085942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 0x556146fafb20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f30540117e0 tx=0x7f3054011ca0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3069a0e740 1 -- 172.21.15.32:0/3865085942 shutdown_connections 2024-08-31T07:18:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3069a0e740 1 --2- 172.21.15.32:0/3865085942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fb05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3069a0e740 1 --2- 172.21.15.32:0/3865085942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fb0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3069a0e740 1 --2- 172.21.15.32:0/3865085942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 0x556146fafb20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3069a0e740 1 -- 172.21.15.32:0/3865085942 >> 172.21.15.32:0/3865085942 conn(0x556146bfe460 msgr2=0x556146fada10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3069a0e740 1 -- 172.21.15.32:0/3865085942 shutdown_connections 2024-08-31T07:18:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3058c00640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30400b4c50 0x7f30400b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3069a0e740 1 -- 172.21.15.32:0/3865085942 wait complete. 2024-08-31T07:18:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f3058c00640 1 -- 172.21.15.32:0/499650349 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f304c0e4f70 con 0x7f305c105950 2024-08-31T07:18:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.826+0000 7f305b400640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30400b4c50 0x7f30400b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.827+0000 7f3069a0e740 1 -- start start 2024-08-31T07:18:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.827+0000 7f3069a0e740 1 Processor -- start 2024-08-31T07:18:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.827+0000 7f305b400640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30400b4c50 0x7f30400b7160 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3048006010 tx=0x7f3048005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be11b0 0x556146fbb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 0x556146fbbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146c0a070 0x556146fc75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f3069a0e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556146d78700 con 0x556146be11b0 2024-08-31T07:18:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f3069a0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556146e9ccb0 con 0x556146be9390 2024-08-31T07:18:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f3069a0e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556146fbe300 con 0x556146c0a070 2024-08-31T07:18:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f3069a0e740 10 client.0 _ll_register_callbacks cb 0x556146ec2430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f305be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be11b0 0x556146fbb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f305be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be11b0 0x556146fbb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48304/0 (socket says 172.21.15.32:48304) 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f305be00640 1 -- 172.21.15.32:0/2539020773 learned_addr learned my addr 172.21.15.32:0/2539020773 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.828+0000 7f305b400640 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 0x556146fbbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f305be00640 1 -- 172.21.15.32:0/2539020773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146c0a070 msgr2=0x556146fc75f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f305be00640 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146c0a070 0x556146fc75f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f305be00640 1 -- 172.21.15.32:0/2539020773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 msgr2=0x556146fbbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f305be00640 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 0x556146fbbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f305be00640 1 -- 172.21.15.32:0/2539020773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556146edd870 con 0x556146be11b0 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f305be00640 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be11b0 0x556146fbb6f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f304c0e2d40 tx=0x7f304c0aaff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a5f363578552ae server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f3046000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f3046000640 1 -- 172.21.15.32:0/2539020773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c020050 con 0x556146be11b0 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f3046000640 1 -- 172.21.15.32:0/2539020773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f304c0099d0 con 0x556146be11b0 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f3069a0e740 1 -- 172.21.15.32:0/2539020773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3054003e20 con 0x556146be11b0 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f3046000640 1 -- 172.21.15.32:0/2539020773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f304c0b4a00 con 0x556146be11b0 2024-08-31T07:18:29.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f3069a0e740 10 client.16770 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.829+0000 7f3069a0e740 1 -- 172.21.15.32:0/2539020773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556146fca320 con 0x556146be11b0 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.830+0000 7f3046000640 1 -- 172.21.15.32:0/2539020773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3054003e20 con 0x556146be11b0 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.830+0000 7f3046000640 1 -- 172.21.15.32:0/2539020773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f304c0d2190 con 0x556146be11b0 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.830+0000 7f3069a0e740 10 client.16770 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.830+0000 7f3069a0e740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.830+0000 7f3069a0e740 20 client.16770 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.830+0000 7f3069a0e740 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556146bc89f0 0x556146fcd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.830+0000 7f3069a0e740 4 client.16770 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.830+0000 7f3069a0e740 1 -- 172.21.15.32:0/2539020773 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556146f286c0 con 0x556146bc89f0 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.830+0000 7f305b400640 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556146bc89f0 0x556146fcd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.831+0000 7f305b400640 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556146bc89f0 0x556146fcd740 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db3483e23cae7850 server_cookie=71dd29ab96428b1d in_seq=0 out_seq=0 2024-08-31T07:18:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.831+0000 7f3046000640 10 client.16770 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3046000640 1 -- 172.21.15.32:0/2539020773 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556146f286c0 con 0x556146bc89f0 2024-08-31T07:18:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3046000640 10 client.16770 handle_command_reply: tid=0 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3069a0e740 1 client.16770 shutdown 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3069a0e740 1 -- 172.21.15.32:0/2539020773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be11b0 msgr2=0x556146fbb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3069a0e740 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be11b0 0x556146fbb6f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f304c0e2d40 tx=0x7f304c0aaff0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3069a0e740 1 -- 172.21.15.32:0/2539020773 shutdown_connections 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3069a0e740 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556146bc89f0 0x556146fcd740 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3069a0e740 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146c0a070 0x556146fc75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3069a0e740 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 0x556146fbbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3069a0e740 1 --2- 172.21.15.32:0/2539020773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be11b0 0x556146fbb6f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3069a0e740 1 -- 172.21.15.32:0/2539020773 >> 172.21.15.32:0/2539020773 conn(0x556146bfe460 msgr2=0x556146fb2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.832+0000 7f3046000640 0 client.16770 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.833+0000 7f3069a0e740 1 -- 172.21.15.32:0/2539020773 shutdown_connections 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.833+0000 7f3069a0e740 1 -- 172.21.15.32:0/2539020773 wait complete. 2024-08-31T07:18:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.833+0000 7f3069a0e740 20 client.16770 trim_cache size 0 max 16384 2024-08-31T07:18:29.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.851+0000 7f3069a0e740 1 Processor -- start 2024-08-31T07:18:29.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.851+0000 7f3069a0e740 1 -- start start 2024-08-31T07:18:29.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.851+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fc74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 0x556146fbd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146c0a070 0x556146fb86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f3069a0e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1b1400 con 0x556146c0a070 2024-08-31T07:18:29.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f3069a0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1b2f40 con 0x556146be9390 2024-08-31T07:18:29.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f3069a0e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1b1ad0 con 0x556146be11b0 2024-08-31T07:18:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f3060e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146c0a070 0x556146fb86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f305b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 0x556146fbd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f3060e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146c0a070 0x556146fb86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48320/0 (socket says 172.21.15.32:48320) 2024-08-31T07:18:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f305be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fc74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f3060e00640 1 -- 172.21.15.32:0/3573569163 learned_addr learned my addr 172.21.15.32:0/3573569163 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f305b400640 1 -- 172.21.15.32:0/3573569163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 msgr2=0x556146fc74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f305b400640 1 --2- 172.21.15.32:0/3573569163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fc74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f305b400640 1 -- 172.21.15.32:0/3573569163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146c0a070 msgr2=0x556146fb86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f305b400640 1 --2- 172.21.15.32:0/3573569163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146c0a070 0x556146fb86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f305b400640 1 -- 172.21.15.32:0/3573569163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556146fb2040 con 0x556146be9390 2024-08-31T07:18:29.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.852+0000 7f3060e00640 1 --2- 172.21.15.32:0/3573569163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146c0a070 0x556146fb86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:29.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.853+0000 7f305b400640 1 --2- 172.21.15.32:0/3573569163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 0x556146fbd120 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3048005ab0 tx=0x7f304803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c669d67584b702 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:29.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.853+0000 7f3050c00640 1 -- 172.21.15.32:0/3573569163 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048035040 con 0x556146be9390 2024-08-31T07:18:29.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.853+0000 7f3050c00640 1 -- 172.21.15.32:0/3573569163 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3048031b50 con 0x556146be9390 2024-08-31T07:18:29.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.853+0000 7f3050c00640 1 -- 172.21.15.32:0/3573569163 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048054a00 con 0x556146be9390 2024-08-31T07:18:29.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.854+0000 7f3069a0e740 1 -- 172.21.15.32:0/3573569163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 msgr2=0x556146fbd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:29.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.854+0000 7f3069a0e740 1 --2- 172.21.15.32:0/3573569163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 0x556146fbd120 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3048005ab0 tx=0x7f304803f000 comp rx=0 tx=0).stop 2024-08-31T07:18:29.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.854+0000 7f3069a0e740 1 -- 172.21.15.32:0/3573569163 shutdown_connections 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.854+0000 7f3069a0e740 1 --2- 172.21.15.32:0/3573569163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146c0a070 0x556146fb86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.854+0000 7f3069a0e740 1 --2- 172.21.15.32:0/3573569163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146be9390 0x556146fbd120 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.854+0000 7f3069a0e740 1 --2- 172.21.15.32:0/3573569163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fc74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.854+0000 7f3069a0e740 1 -- 172.21.15.32:0/3573569163 >> 172.21.15.32:0/3573569163 conn(0x556146bfe460 msgr2=0x556146fbca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.854+0000 7f3069a0e740 1 -- 172.21.15.32:0/3573569163 shutdown_connections 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.854+0000 7f3069a0e740 1 -- 172.21.15.32:0/3573569163 wait complete. 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.855+0000 7f3069a0e740 1 -- start start 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.855+0000 7f3069a0e740 1 Processor -- start 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.855+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fbf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.855+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fbfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.855+0000 7f3069a0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 0x556146fc0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.855+0000 7f3069a0e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1b2870 con 0x556146be9390 2024-08-31T07:18:29.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.855+0000 7f3069a0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1b21a0 con 0x556146c0a070 2024-08-31T07:18:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.855+0000 7f3069a0e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1af3e0 con 0x556146be11b0 2024-08-31T07:18:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.855+0000 7f3069a0e740 10 client.0 _ll_register_callbacks cb 0x7f304c0ab150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f305b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fbfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f305b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fbfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48324/0 (socket says 172.21.15.32:48324) 2024-08-31T07:18:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f305b400640 1 -- 172.21.15.32:0/989496865 learned_addr learned my addr 172.21.15.32:0/989496865 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f3060e00640 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 0x556146fc0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f305be00640 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fbf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f305b400640 1 -- 172.21.15.32:0/989496865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 msgr2=0x556146fbf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f305b400640 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fbf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f305b400640 1 -- 172.21.15.32:0/989496865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 msgr2=0x556146fc0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f305b400640 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 0x556146fc0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f305b400640 1 -- 172.21.15.32:0/989496865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556147037300 con 0x556146be9390 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f305b400640 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fbfca0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3048031f30 tx=0x7f304802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c57e410ab1e053c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.856+0000 7f3046000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.857+0000 7f3046000640 1 -- 172.21.15.32:0/989496865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304806b060 con 0x556146be9390 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.857+0000 7f3046000640 1 -- 172.21.15.32:0/989496865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f30480316b0 con 0x556146be9390 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.857+0000 7f3069a0e740 1 -- 172.21.15.32:0/989496865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3048031b50 con 0x556146be9390 2024-08-31T07:18:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.857+0000 7f3046000640 1 -- 172.21.15.32:0/989496865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f30480543f0 con 0x556146be9390 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.857+0000 7f3069a0e740 10 client.16776 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.857+0000 7f3069a0e740 1 -- 172.21.15.32:0/989496865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556147038590 con 0x556146be9390 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.857+0000 7f3046000640 1 -- 172.21.15.32:0/989496865 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3048031b50 con 0x556146be9390 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.858+0000 7f3046000640 1 -- 172.21.15.32:0/989496865 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f304803d570 con 0x556146be9390 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.858+0000 7f3069a0e740 10 client.16776 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.858+0000 7f3069a0e740 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.858+0000 7f3069a0e740 20 client.16776 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.858+0000 7f3069a0e740 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556146b9dda0 0x55614703acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.858+0000 7f3069a0e740 4 client.16776 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.858+0000 7f3069a0e740 1 -- 172.21.15.32:0/989496865 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556146f24160 con 0x556146b9dda0 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.858+0000 7f305be00640 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556146b9dda0 0x55614703acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f305be00640 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556146b9dda0 0x55614703acc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bfaf71882fd54ef server_cookie=7de47fb151668730 in_seq=0 out_seq=0 2024-08-31T07:18:29.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3046000640 10 client.16776 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3046000640 1 -- 172.21.15.32:0/989496865 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x556146f24160 con 0x556146b9dda0 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3046000640 10 client.16776 handle_command_reply: tid=0 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 client.16776 shutdown 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 -- 172.21.15.32:0/989496865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 msgr2=0x556146fbfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fbfca0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3048031f30 tx=0x7f304802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 -- 172.21.15.32:0/989496865 shutdown_connections 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556146b9dda0 0x55614703acc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556146c0a070 0x556146fc0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556146be9390 0x556146fbfca0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 --2- 172.21.15.32:0/989496865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556146be11b0 0x556146fbf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 -- 172.21.15.32:0/989496865 >> 172.21.15.32:0/989496865 conn(0x556146bfe460 msgr2=0x556146fb0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3046000640 0 client.16776 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 -- 172.21.15.32:0/989496865 shutdown_connections 2024-08-31T07:18:29.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.859+0000 7f3069a0e740 1 -- 172.21.15.32:0/989496865 wait complete. 2024-08-31T07:18:29.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3069a0e740 20 client.16776 trim_cache size 0 max 16384 2024-08-31T07:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:18:29.864 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "cb664667-a37e-4c2a-b16d-2ae314b4a7c7", 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:29.865 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:29.866 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:29.867 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:29.868 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:18:29.869 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:29.870 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3061800640 1 -- 172.21.15.32:0/499650349 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30400b4c50 msgr2=0x7f30400b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3061800640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30400b4c50 0x7f30400b7160 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3048006010 tx=0x7f3048005a20 comp rx=0 tx=0).stop 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3061800640 1 -- 172.21.15.32:0/499650349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c105950 msgr2=0x7f305c19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3061800640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c105950 0x7f305c19c2c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f304c00d6d0 tx=0x7f304c00db90 comp rx=0 tx=0).stop 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3061800640 1 -- 172.21.15.32:0/499650349 shutdown_connections 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3061800640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30400b4c50 0x7f30400b7160 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3061800640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c10bbe0 0x7f305c1a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3061800640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f305c10b290 0x7f305c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3061800640 1 --2- 172.21.15.32:0/499650349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f305c105950 0x7f305c19c2c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.860+0000 7f3061800640 1 -- 172.21.15.32:0/499650349 >> 172.21.15.32:0/499650349 conn(0x7f305c101420 msgr2=0x7f305c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.861+0000 7f3061800640 1 -- 172.21.15.32:0/499650349 shutdown_connections 2024-08-31T07:18:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:29.861+0000 7f3061800640 1 -- 172.21.15.32:0/499650349 wait complete. 2024-08-31T07:18:30.714 INFO:tasks.cephfs.filesystem:scrub status for tag:cb664667-a37e-4c2a-b16d-2ae314b4a7c7 - {'path': '/', 'tag': 'cb664667-a37e-4c2a-b16d-2ae314b4a7c7', 'options': 'recursive,force'} 2024-08-31T07:18:30.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.20 MB/sec execute 139 sec latency 21602.192 ms 2024-08-31T07:18:30.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:30 smithi073 ceph-mon[21995]: pgmap v540: 97 pgs: 97 active+clean; 103 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 974 KiB/s wr, 46 op/s 2024-08-31T07:18:30.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:30 smithi084 ceph-mon[22387]: pgmap v540: 97 pgs: 97 active+clean; 103 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 974 KiB/s wr, 46 op/s 2024-08-31T07:18:31.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:30 smithi032 ceph-mon[17252]: pgmap v540: 97 pgs: 97 active+clean; 103 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 974 KiB/s wr, 46 op/s 2024-08-31T07:18:31.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.19 MB/sec execute 140 sec latency 22602.306 ms 2024-08-31T07:18:32.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.17 MB/sec execute 141 sec latency 23602.381 ms 2024-08-31T07:18:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:32 smithi073 ceph-mon[21995]: pgmap v541: 97 pgs: 97 active+clean; 101 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 986 KiB/s wr, 53 op/s 2024-08-31T07:18:32.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:32 smithi084 ceph-mon[22387]: pgmap v541: 97 pgs: 97 active+clean; 101 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 986 KiB/s wr, 53 op/s 2024-08-31T07:18:33.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:32 smithi032 ceph-mon[17252]: pgmap v541: 97 pgs: 97 active+clean; 101 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 986 KiB/s wr, 53 op/s 2024-08-31T07:18:33.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.16 MB/sec execute 142 sec latency 24602.551 ms 2024-08-31T07:18:34.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.14 MB/sec execute 143 sec latency 25602.700 ms 2024-08-31T07:18:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:34 smithi073 ceph-mon[21995]: pgmap v542: 97 pgs: 97 active+clean; 101 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 944 KiB/s wr, 51 op/s 2024-08-31T07:18:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:34 smithi084 ceph-mon[22387]: pgmap v542: 97 pgs: 97 active+clean; 101 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 944 KiB/s wr, 51 op/s 2024-08-31T07:18:35.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:34 smithi032 ceph-mon[17252]: pgmap v542: 97 pgs: 97 active+clean; 101 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 944 KiB/s wr, 51 op/s 2024-08-31T07:18:35.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.13 MB/sec execute 144 sec latency 26602.833 ms 2024-08-31T07:18:36.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.11 MB/sec execute 145 sec latency 27602.982 ms 2024-08-31T07:18:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:36 smithi073 ceph-mon[21995]: pgmap v543: 97 pgs: 97 active+clean; 100 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 785 KiB/s wr, 51 op/s 2024-08-31T07:18:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:36 smithi084 ceph-mon[22387]: pgmap v543: 97 pgs: 97 active+clean; 100 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 785 KiB/s wr, 51 op/s 2024-08-31T07:18:37.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:36 smithi032 ceph-mon[17252]: pgmap v543: 97 pgs: 97 active+clean; 100 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 785 KiB/s wr, 51 op/s 2024-08-31T07:18:37.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.10 MB/sec execute 146 sec latency 28603.153 ms 2024-08-31T07:18:38.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:18:38.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.08 MB/sec execute 147 sec latency 29603.239 ms 2024-08-31T07:18:38.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:38 smithi032 ceph-mon[17252]: pgmap v544: 97 pgs: 97 active+clean; 100 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 749 KiB/s wr, 50 op/s 2024-08-31T07:18:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:38.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:38 smithi073 ceph-mon[21995]: pgmap v544: 97 pgs: 97 active+clean; 100 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 749 KiB/s wr, 50 op/s 2024-08-31T07:18:39.024 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:38 smithi084 ceph-mon[22387]: pgmap v544: 97 pgs: 97 active+clean; 100 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 749 KiB/s wr, 50 op/s 2024-08-31T07:18:39.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.07 MB/sec execute 148 sec latency 30603.293 ms 2024-08-31T07:18:39.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.730+0000 7f71ff000640 1 -- 172.21.15.32:0/9408094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f8109090 msgr2=0x7f71f81094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:39.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.730+0000 7f71ff000640 1 --2- 172.21.15.32:0/9408094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f8109090 0x7f71f81094a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009900 tx=0x7f71e402a410 comp rx=0 tx=0).stop 2024-08-31T07:18:39.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.730+0000 7f71ff000640 1 -- 172.21.15.32:0/9408094 shutdown_connections 2024-08-31T07:18:39.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.730+0000 7f71ff000640 1 --2- 172.21.15.32:0/9408094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f81099e0 0x7f71f8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:39.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.730+0000 7f71ff000640 1 --2- 172.21.15.32:0/9408094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f8109090 0x7f71f81094a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:39.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.730+0000 7f71ff000640 1 --2- 172.21.15.32:0/9408094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8103750 0x7f71f8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:39.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.730+0000 7f71ff000640 1 -- 172.21.15.32:0/9408094 >> 172.21.15.32:0/9408094 conn(0x7f71f80ff1e0 msgr2=0x7f71f8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:39.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.730+0000 7f71ff000640 1 -- 172.21.15.32:0/9408094 shutdown_connections 2024-08-31T07:18:39.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.730+0000 7f71ff000640 1 -- 172.21.15.32:0/9408094 wait complete. 2024-08-31T07:18:39.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.731+0000 7f71ff000640 1 Processor -- start 2024-08-31T07:18:39.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.732+0000 7f71ff000640 1 -- start start 2024-08-31T07:18:39.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.732+0000 7f71ff000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f8103750 0x7f71f8076ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:39.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.732+0000 7f71ff000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8109090 0x7f71f80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:39.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.732+0000 7f71ff000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f81099e0 0x7f71f80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:39.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.732+0000 7f71ff000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8113670 con 0x7f71f81099e0 2024-08-31T07:18:39.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.732+0000 7f71ff000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81134f0 con 0x7f71f8109090 2024-08-31T07:18:39.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.732+0000 7f71ff000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81137f0 con 0x7f71f8103750 2024-08-31T07:18:39.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fdc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f8103750 0x7f71f8076ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:39.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fdc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f8103750 0x7f71f8076ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50828/0 (socket says 172.21.15.32:50828) 2024-08-31T07:18:39.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fdc00640 1 -- 172.21.15.32:0/257879612 learned_addr learned my addr 172.21.15.32:0/257879612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:39.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fe600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f81099e0 0x7f71f80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:39.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fd200640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8109090 0x7f71f80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:39.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fdc00640 1 -- 172.21.15.32:0/257879612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8109090 msgr2=0x7f71f80754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:39.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fdc00640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8109090 0x7f71f80754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:39.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fdc00640 1 -- 172.21.15.32:0/257879612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f81099e0 msgr2=0x7f71f80759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:39.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fdc00640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f81099e0 0x7f71f80759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:39.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fdc00640 1 -- 172.21.15.32:0/257879612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f8076210 con 0x7f71f8103750 2024-08-31T07:18:39.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.733+0000 7f71fdc00640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f8103750 0x7f71f8076ec0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00ea60 tx=0x7f71ec00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:39.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.734+0000 7f71f6000640 1 -- 172.21.15.32:0/257879612 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00cde0 con 0x7f71f8103750 2024-08-31T07:18:39.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.734+0000 7f71ff000640 1 -- 172.21.15.32:0/257879612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f81acae0 con 0x7f71f8103750 2024-08-31T07:18:39.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.734+0000 7f71f6000640 1 -- 172.21.15.32:0/257879612 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ec0035f0 con 0x7f71f8103750 2024-08-31T07:18:39.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.734+0000 7f71f6000640 1 -- 172.21.15.32:0/257879612 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec010690 con 0x7f71f8103750 2024-08-31T07:18:39.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.734+0000 7f71ff000640 1 -- 172.21.15.32:0/257879612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f81ad040 con 0x7f71f8103750 2024-08-31T07:18:39.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.735+0000 7f71ff000640 1 -- 172.21.15.32:0/257879612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f8103b60 con 0x7f71f8103750 2024-08-31T07:18:39.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.737+0000 7f71f6000640 1 -- 172.21.15.32:0/257879612 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f71ec0037a0 con 0x7f71f8103750 2024-08-31T07:18:39.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.739+0000 7f71f6000640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71e00b4ca0 0x7f71e00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:39.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.739+0000 7f71f6000640 1 -- 172.21.15.32:0/257879612 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7f71ec0e5820 con 0x7f71f8103750 2024-08-31T07:18:39.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.739+0000 7f71fd200640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71e00b4ca0 0x7f71e00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:39.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.740+0000 7f71fd200640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71e00b4ca0 0x7f71e00b71b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009230 tx=0x7f71e4031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:39.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.743+0000 7f71f6000640 1 -- 172.21.15.32:0/257879612 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f71ec0ac630 con 0x7f71f8103750 2024-08-31T07:18:39.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.981+0000 7f71ff000640 1 -- 172.21.15.32:0/257879612 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71f802e430 con 0x7f71f8103750 2024-08-31T07:18:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.982+0000 7f71f6000640 1 -- 172.21.15.32:0/257879612 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11960 (secure 0 0 0) 0x7f71ec0abf30 con 0x7f71f8103750 2024-08-31T07:18:39.984 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:18:39.984 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"btime":"2024-08-31T07:17:01:405815+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:17:00.944291+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:18:39.984 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2024-08-31T07:18:39.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.985+0000 7f71ff000640 1 -- 172.21.15.32:0/257879612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71e00b4ca0 msgr2=0x7f71e00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:39.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.985+0000 7f71ff000640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71e00b4ca0 0x7f71e00b71b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009230 tx=0x7f71e4031400 comp rx=0 tx=0).stop 2024-08-31T07:18:39.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.985+0000 7f71ff000640 1 -- 172.21.15.32:0/257879612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f8103750 msgr2=0x7f71f8076ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:39.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.985+0000 7f71ff000640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f8103750 0x7f71f8076ec0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00ea60 tx=0x7f71ec00ef20 comp rx=0 tx=0).stop 2024-08-31T07:18:39.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.985+0000 7f71ff000640 1 -- 172.21.15.32:0/257879612 shutdown_connections 2024-08-31T07:18:39.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.985+0000 7f71ff000640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71e00b4ca0 0x7f71e00b71b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:39.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.985+0000 7f71ff000640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f81099e0 0x7f71f80759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:39.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.985+0000 7f71ff000640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f8109090 0x7f71f80754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:39.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.985+0000 7f71ff000640 1 --2- 172.21.15.32:0/257879612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f8103750 0x7f71f8076ec0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:39.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.985+0000 7f71ff000640 1 -- 172.21.15.32:0/257879612 >> 172.21.15.32:0/257879612 conn(0x7f71f80ff1e0 msgr2=0x7f71f8101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:39.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.986+0000 7f71ff000640 1 -- 172.21.15.32:0/257879612 shutdown_connections 2024-08-31T07:18:39.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:39.986+0000 7f71ff000640 1 -- 172.21.15.32:0/257879612 wait complete. 2024-08-31T07:18:40.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.05 MB/sec execute 149 sec latency 31603.353 ms 2024-08-31T07:18:40.932 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 601ae51e --release --await' 2024-08-31T07:18:40.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 601ae51e --release --await 2024-08-31T07:18:40.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:40 smithi032 ceph-mon[17252]: pgmap v545: 97 pgs: 97 active+clean; 100 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 518 KiB/s wr, 38 op/s 2024-08-31T07:18:40.984 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:40 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/257879612' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:18:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:40 smithi084 ceph-mon[22387]: pgmap v545: 97 pgs: 97 active+clean; 100 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 518 KiB/s wr, 38 op/s 2024-08-31T07:18:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:40 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/257879612' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:18:41.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:40 smithi073 ceph-mon[21995]: pgmap v545: 97 pgs: 97 active+clean; 100 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 518 KiB/s wr, 38 op/s 2024-08-31T07:18:41.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:40 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/257879612' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:18:41.313 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:18:41.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160431 2.04 MB/sec execute 150 sec latency 32603.416 ms 2024-08-31T07:18:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.012+0000 7ff770600640 1 -- 172.21.15.32:0/2967592179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7681144e0 msgr2=0x7ff768116900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.012+0000 7ff770600640 1 --2- 172.21.15.32:0/2967592179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7681144e0 0x7ff768116900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff760009ca0 tx=0x7ff76002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.012+0000 7ff770600640 1 -- 172.21.15.32:0/2967592179 shutdown_connections 2024-08-31T07:18:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.012+0000 7ff770600640 1 --2- 172.21.15.32:0/2967592179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7681144e0 0x7ff768116900 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.012+0000 7ff770600640 1 --2- 172.21.15.32:0/2967592179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff768078030 0x7ff768113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.012+0000 7ff770600640 1 --2- 172.21.15.32:0/2967592179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7680776e0 0x7ff768077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.012+0000 7ff770600640 1 -- 172.21.15.32:0/2967592179 >> 172.21.15.32:0/2967592179 conn(0x7ff7681015b0 msgr2=0x7ff768103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.012+0000 7ff770600640 1 -- 172.21.15.32:0/2967592179 shutdown_connections 2024-08-31T07:18:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.012+0000 7ff770600640 1 -- 172.21.15.32:0/2967592179 wait complete. 2024-08-31T07:18:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.013+0000 7ff770600640 1 Processor -- start 2024-08-31T07:18:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.013+0000 7ff770600640 1 -- start start 2024-08-31T07:18:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff770600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680776e0 0x7ff7681a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff770600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff768078030 0x7ff7681a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff770600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7681144e0 0x7ff7681a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff770600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff768119e90 con 0x7ff7680776e0 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff770600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff768119d10 con 0x7ff7681144e0 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff770600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff76811a010 con 0x7ff768078030 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680776e0 0x7ff7681a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680776e0 0x7ff7681a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46216/0 (socket says 172.21.15.32:46216) 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff76d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff768078030 0x7ff7681a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff76dc00640 1 -- 172.21.15.32:0/1051175757 learned_addr learned my addr 172.21.15.32:0/1051175757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff76e600640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7681144e0 0x7ff7681a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff76d200640 1 -- 172.21.15.32:0/1051175757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7681144e0 msgr2=0x7ff7681a5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff76d200640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7681144e0 0x7ff7681a5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff76d200640 1 -- 172.21.15.32:0/1051175757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680776e0 msgr2=0x7ff7681a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff76d200640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680776e0 0x7ff7681a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.014+0000 7ff76d200640 1 -- 172.21.15.32:0/1051175757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7681a6720 con 0x7ff768078030 2024-08-31T07:18:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.015+0000 7ff76d200640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff768078030 0x7ff7681a0d50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff76806c840 tx=0x7ff75400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.015+0000 7ff766000640 1 -- 172.21.15.32:0/1051175757 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75400cc40 con 0x7ff768078030 2024-08-31T07:18:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.015+0000 7ff770600640 1 -- 172.21.15.32:0/1051175757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7681a6a10 con 0x7ff768078030 2024-08-31T07:18:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.015+0000 7ff766000640 1 -- 172.21.15.32:0/1051175757 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff75400cde0 con 0x7ff768078030 2024-08-31T07:18:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.015+0000 7ff766000640 1 -- 172.21.15.32:0/1051175757 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754018760 con 0x7ff768078030 2024-08-31T07:18:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.015+0000 7ff770600640 1 -- 172.21.15.32:0/1051175757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7681ad170 con 0x7ff768078030 2024-08-31T07:18:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.019+0000 7ff77721e740 1 Processor -- start 2024-08-31T07:18:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.019+0000 7ff77721e740 1 -- start start 2024-08-31T07:18:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.020+0000 7ff77721e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919eee520 0x56191a293ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.020+0000 7ff77721e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a294430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.020+0000 7ff77721e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919ecd840 0x56191a294970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.020+0000 7ff77721e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56191a20aa90 con 0x561919ec5660 2024-08-31T07:18:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.020+0000 7ff77721e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56191a1fa020 con 0x561919ecd840 2024-08-31T07:18:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.020+0000 7ff77721e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56191a1fdfd0 con 0x561919eee520 2024-08-31T07:18:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.020+0000 7ff76e600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919eee520 0x56191a293ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.020+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a294430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.020+0000 7ff76e600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919eee520 0x56191a293ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50854/0 (socket says 172.21.15.32:50854) 2024-08-31T07:18:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.020+0000 7ff76e600640 1 -- 172.21.15.32:0/3824445359 learned_addr learned my addr 172.21.15.32:0/3824445359 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.021+0000 7ff766000640 1 -- 172.21.15.32:0/1051175757 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff754010a90 con 0x7ff768078030 2024-08-31T07:18:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.021+0000 7ff76e600640 1 -- 172.21.15.32:0/3824445359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919ecd840 msgr2=0x56191a294970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:18:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.021+0000 7ff76e600640 1 --2- 172.21.15.32:0/3824445359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919ecd840 0x56191a294970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.021+0000 7ff76e600640 1 -- 172.21.15.32:0/3824445359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 msgr2=0x56191a294430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.021+0000 7ff76e600640 1 --2- 172.21.15.32:0/3824445359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a294430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.021+0000 7ff76e600640 1 -- 172.21.15.32:0/3824445359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561919f59b00 con 0x561919eee520 2024-08-31T07:18:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.021+0000 7ff76dc00640 1 --2- 172.21.15.32:0/3824445359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a294430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.021+0000 7ff76e600640 1 --2- 172.21.15.32:0/3824445359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919eee520 0x56191a293ef0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff760009ca0 tx=0x7ff760009430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa5713a2ec3be90a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.021+0000 7ff75aa00640 1 -- 172.21.15.32:0/3824445359 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760031cf0 con 0x561919eee520 2024-08-31T07:18:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff75aa00640 1 -- 172.21.15.32:0/3824445359 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff760036030 con 0x561919eee520 2024-08-31T07:18:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff75aa00640 1 -- 172.21.15.32:0/3824445359 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760004280 con 0x561919eee520 2024-08-31T07:18:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff766000640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7500b4d20 0x7ff7500b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff77721e740 1 -- 172.21.15.32:0/3824445359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919eee520 msgr2=0x56191a293ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff77721e740 1 --2- 172.21.15.32:0/3824445359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919eee520 0x56191a293ef0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff760009ca0 tx=0x7ff760009430 comp rx=0 tx=0).stop 2024-08-31T07:18:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff766000640 1 -- 172.21.15.32:0/1051175757 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff7540e5750 con 0x7ff768078030 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff77721e740 1 -- 172.21.15.32:0/3824445359 shutdown_connections 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff77721e740 1 --2- 172.21.15.32:0/3824445359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919ecd840 0x56191a294970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff77721e740 1 --2- 172.21.15.32:0/3824445359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a294430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff76dc00640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7500b4d20 0x7ff7500b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff77721e740 1 --2- 172.21.15.32:0/3824445359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919eee520 0x56191a293ef0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.022+0000 7ff77721e740 1 -- 172.21.15.32:0/3824445359 >> 172.21.15.32:0/3824445359 conn(0x561919ee2910 msgr2=0x56191a291de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.023+0000 7ff77721e740 1 -- 172.21.15.32:0/3824445359 shutdown_connections 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.023+0000 7ff77721e740 1 -- 172.21.15.32:0/3824445359 wait complete. 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.023+0000 7ff76dc00640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7500b4d20 0x7ff7500b7230 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56191a292b70 tx=0x7ff75c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.023+0000 7ff77721e740 1 -- start start 2024-08-31T07:18:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff77721e740 1 Processor -- start 2024-08-31T07:18:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff77721e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a2a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff77721e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a29f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff77721e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 0x56191a29f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff77721e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56191a05cbb0 con 0x561919ec5660 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff77721e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56191a180f90 con 0x561919eee520 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff77721e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56191a2a24f0 con 0x561919ecd840 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff77721e740 10 client.0 _ll_register_callbacks cb 0x56191a1a6710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff76d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a29f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a2a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff76d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a29f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50866/0 (socket says 172.21.15.32:50866) 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff76d200640 1 -- 172.21.15.32:0/795459494 learned_addr learned my addr 172.21.15.32:0/795459494 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.024+0000 7ff76e600640 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 0x56191a29f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff76d200640 1 -- 172.21.15.32:0/795459494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 msgr2=0x56191a29f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff76d200640 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 0x56191a29f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff76d200640 1 -- 172.21.15.32:0/795459494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 msgr2=0x56191a2a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff76d200640 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a2a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff76d200640 1 -- 172.21.15.32:0/795459494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561919fd0960 con 0x561919ecd840 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff76e600640 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 0x56191a29f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff76d200640 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a29f240 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff7540a9180 tx=0x7ff7540ac4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cc1ec328bed83d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff74f400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff74f400640 1 -- 172.21.15.32:0/795459494 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540b06d0 con 0x561919ecd840 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff74f400640 1 -- 172.21.15.32:0/795459494 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff7540932c0 con 0x561919ecd840 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.025+0000 7ff77721e740 1 -- 172.21.15.32:0/795459494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56191a198e30 con 0x561919ecd840 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff74f400640 1 -- 172.21.15.32:0/795459494 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff7540b5180 con 0x561919ecd840 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff77721e740 10 client.26818 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff77721e740 1 -- 172.21.15.32:0/795459494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56191a2adea0 con 0x561919ecd840 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff74f400640 1 -- 172.21.15.32:0/795459494 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56191a198e30 con 0x561919ecd840 2024-08-31T07:18:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff74f400640 1 -- 172.21.15.32:0/795459494 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff7540aac70 con 0x561919ecd840 2024-08-31T07:18:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff77721e740 10 client.26818 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff77721e740 10 client.26818 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:18:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff77721e740 20 client.26818 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff77721e740 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561919eacea0 0x56191a2b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff77721e740 4 client.26818 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:18:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.026+0000 7ff77721e740 1 -- 172.21.15.32:0/795459494 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56191a20c9e0 con 0x561919eacea0 2024-08-31T07:18:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.027+0000 7ff76dc00640 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561919eacea0 0x56191a2b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.027+0000 7ff76dc00640 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561919eacea0 0x56191a2b1250 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4231c5bd1a942a21 server_cookie=a5493605bdd124d2 in_seq=0 out_seq=0 2024-08-31T07:18:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.027+0000 7ff74f400640 10 client.26818 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.028+0000 7ff74f400640 1 -- 172.21.15.32:0/795459494 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56191a20c9e0 con 0x561919eacea0 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.028+0000 7ff74f400640 10 client.26818 handle_command_reply: tid=0 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.028+0000 7ff77721e740 1 client.26818 shutdown 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 1 -- 172.21.15.32:0/795459494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 msgr2=0x56191a29f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a29f240 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff7540a9180 tx=0x7ff7540ac4e0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 1 -- 172.21.15.32:0/795459494 shutdown_connections 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561919eacea0 0x56191a2b1250 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 0x56191a29f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff74f400640 0 client.26818 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a29f240 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 1 --2- 172.21.15.32:0/795459494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a2a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 1 -- 172.21.15.32:0/795459494 >> 172.21.15.32:0/795459494 conn(0x561919ee2910 msgr2=0x56191a297080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 1 -- 172.21.15.32:0/795459494 shutdown_connections 2024-08-31T07:18:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 1 -- 172.21.15.32:0/795459494 wait complete. 2024-08-31T07:18:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.029+0000 7ff77721e740 20 client.26818 trim_cache size 0 max 16384 2024-08-31T07:18:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.047+0000 7ff77721e740 1 Processor -- start 2024-08-31T07:18:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.047+0000 7ff77721e740 1 -- start start 2024-08-31T07:18:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.047+0000 7ff77721e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ec5660 0x56191a2ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.047+0000 7ff77721e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919ecd840 0x56191a2a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff77721e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919eee520 0x56191a297700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff77721e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b58a0 con 0x561919eee520 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff77721e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b73e0 con 0x561919ecd840 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff77721e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b5f70 con 0x561919ec5660 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919eee520 0x56191a297700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919ecd840 0x56191a2a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919eee520 0x56191a297700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46256/0 (socket says 172.21.15.32:46256) 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76e600640 1 -- 172.21.15.32:0/3604537546 learned_addr learned my addr 172.21.15.32:0/3604537546 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76dc00640 1 --2- 172.21.15.32:0/3604537546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ec5660 0x56191a2ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76d200640 1 -- 172.21.15.32:0/3604537546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ec5660 msgr2=0x56191a2ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76d200640 1 --2- 172.21.15.32:0/3604537546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ec5660 0x56191a2ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76d200640 1 -- 172.21.15.32:0/3604537546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919eee520 msgr2=0x56191a297700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76d200640 1 --2- 172.21.15.32:0/3604537546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919eee520 0x56191a297700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76d200640 1 -- 172.21.15.32:0/3604537546 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56191a2a04a0 con 0x561919ecd840 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.048+0000 7ff76dc00640 1 --2- 172.21.15.32:0/3604537546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ec5660 0x56191a2ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.049+0000 7ff76d200640 1 --2- 172.21.15.32:0/3604537546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919ecd840 0x56191a2a10c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff75400c6a0 tx=0x7ff7540ac090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=762fa14e89518daa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.049+0000 7ff75a000640 1 -- 172.21.15.32:0/3604537546 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540b06d0 con 0x561919ecd840 2024-08-31T07:18:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.049+0000 7ff75a000640 1 -- 172.21.15.32:0/3604537546 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7540b1be0 con 0x561919ecd840 2024-08-31T07:18:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.049+0000 7ff75a000640 1 -- 172.21.15.32:0/3604537546 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540aabe0 con 0x561919ecd840 2024-08-31T07:18:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.050+0000 7ff77721e740 1 -- 172.21.15.32:0/3604537546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919ecd840 msgr2=0x56191a2a10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.050+0000 7ff77721e740 1 --2- 172.21.15.32:0/3604537546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919ecd840 0x56191a2a10c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff75400c6a0 tx=0x7ff7540ac090 comp rx=0 tx=0).stop 2024-08-31T07:18:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.050+0000 7ff77721e740 1 -- 172.21.15.32:0/3604537546 shutdown_connections 2024-08-31T07:18:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.050+0000 7ff77721e740 1 --2- 172.21.15.32:0/3604537546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919eee520 0x56191a297700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.050+0000 7ff77721e740 1 --2- 172.21.15.32:0/3604537546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919ecd840 0x56191a2a10c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.050+0000 7ff77721e740 1 --2- 172.21.15.32:0/3604537546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ec5660 0x56191a2ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.050+0000 7ff77721e740 1 -- 172.21.15.32:0/3604537546 >> 172.21.15.32:0/3604537546 conn(0x561919ee2910 msgr2=0x56191a296ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.050+0000 7ff77721e740 1 -- 172.21.15.32:0/3604537546 shutdown_connections 2024-08-31T07:18:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.050+0000 7ff77721e740 1 -- 172.21.15.32:0/3604537546 wait complete. 2024-08-31T07:18:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.051+0000 7ff77721e740 1 -- start start 2024-08-31T07:18:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.051+0000 7ff77721e740 1 Processor -- start 2024-08-31T07:18:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.051+0000 7ff77721e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a2a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.051+0000 7ff77721e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a2a4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.051+0000 7ff77721e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 0x56191a314b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.051+0000 7ff77721e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b6d10 con 0x561919ec5660 2024-08-31T07:18:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.051+0000 7ff77721e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b6640 con 0x561919eee520 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.051+0000 7ff77721e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681b3880 con 0x561919ecd840 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.051+0000 7ff77721e740 10 client.0 _ll_register_callbacks cb 0x7ff7540e3370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a2a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a2a40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46272/0 (socket says 172.21.15.32:46272) 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76dc00640 1 -- 172.21.15.32:0/1068944053 learned_addr learned my addr 172.21.15.32:0/1068944053 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76d200640 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a2a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76e600640 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 0x56191a314b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76d200640 1 -- 172.21.15.32:0/1068944053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 msgr2=0x56191a314b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76d200640 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 0x56191a314b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76d200640 1 -- 172.21.15.32:0/1068944053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 msgr2=0x56191a2a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76d200640 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a2a40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76d200640 1 -- 172.21.15.32:0/1068944053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56191a315200 con 0x561919ecd840 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.052+0000 7ff76dc00640 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a2a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.053+0000 7ff76d200640 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a2a4660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff7540ab100 tx=0x7ff7540ac4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=783482531ddfe889 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.053+0000 7ff74f400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.053+0000 7ff74f400640 1 -- 172.21.15.32:0/1068944053 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540971a0 con 0x561919ecd840 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.053+0000 7ff77721e740 1 -- 172.21.15.32:0/1068944053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7540b1be0 con 0x561919ecd840 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.053+0000 7ff74f400640 1 -- 172.21.15.32:0/1068944053 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8397+0+0 (secure 0 0 0) 0x7ff7540aabe0 con 0x561919ecd840 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.053+0000 7ff74f400640 1 -- 172.21.15.32:0/1068944053 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff7540aae20 con 0x561919ecd840 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.053+0000 7ff77721e740 10 client.26824 fetch_fsmap learned FSMap version 33 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.053+0000 7ff77721e740 1 -- 172.21.15.32:0/1068944053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56191a31c610 con 0x561919ecd840 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.053+0000 7ff74f400640 1 -- 172.21.15.32:0/1068944053 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7540b1be0 con 0x561919ecd840 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.053+0000 7ff74f400640 1 -- 172.21.15.32:0/1068944053 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff7540e51a0 con 0x561919ecd840 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.054+0000 7ff77721e740 10 client.26824 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.054+0000 7ff77721e740 10 client.26824 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:18:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.054+0000 7ff77721e740 20 client.26824 populate_metadata read hostname 'smithi032' 2024-08-31T07:18:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.054+0000 7ff77721e740 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561919e82250 0x56191a31ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:18:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.054+0000 7ff77721e740 4 client.26824 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "601ae51e", "release": true, "await": true}] 2024-08-31T07:18:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.054+0000 7ff77721e740 1 -- 172.21.15.32:0/1068944053 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "601ae51e", "release": true, "await": true}) -- 0x56191a208480 con 0x561919e82250 2024-08-31T07:18:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.054+0000 7ff76dc00640 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561919e82250 0x56191a31ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:18:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.054+0000 7ff76dc00640 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561919e82250 0x56191a31ea50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c19905c1c4310581 server_cookie=390bc67203f470c4 in_seq=0 out_seq=0 2024-08-31T07:18:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.054+0000 7ff74f400640 10 client.26824 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff74f400640 1 -- 172.21.15.32:0/1068944053 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56191a208480 con 0x561919e82250 2024-08-31T07:18:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff74f400640 10 client.26824 handle_command_reply: tid=0 2024-08-31T07:18:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff77721e740 1 client.26824 shutdown 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff77721e740 1 -- 172.21.15.32:0/1068944053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 msgr2=0x56191a2a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff77721e740 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a2a4660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff7540ab100 tx=0x7ff7540ac4e0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff77721e740 1 -- 172.21.15.32:0/1068944053 shutdown_connections 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff77721e740 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561919e82250 0x56191a31ea50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff77721e740 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561919eee520 0x56191a314b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff77721e740 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561919ecd840 0x56191a2a4660 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff77721e740 1 --2- 172.21.15.32:0/1068944053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561919ec5660 0x56191a2a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff74f400640 0 client.26824 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff77721e740 1 -- 172.21.15.32:0/1068944053 >> 172.21.15.32:0/1068944053 conn(0x561919ee2910 msgr2=0x56191a2979b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.075+0000 7ff77721e740 1 -- 172.21.15.32:0/1068944053 shutdown_connections 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.076+0000 7ff77721e740 1 -- 172.21.15.32:0/1068944053 wait complete. 2024-08-31T07:18:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.076+0000 7ff77721e740 20 client.26824 trim_cache size 0 max 16384 2024-08-31T07:18:42.078 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:18:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 33, 2024-08-31T07:18:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:18:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 20, 2024-08-31T07:18:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:18:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: "601ae51e": { 2024-08-31T07:18:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 20, 2024-08-31T07:18:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:18:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 152.7, 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:42.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:42.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:42.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:18:42.080 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:18:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.077+0000 7ff770600640 1 -- 172.21.15.32:0/1051175757 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7500b4d20 msgr2=0x7ff7500b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.077+0000 7ff770600640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7500b4d20 0x7ff7500b7230 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56191a292b70 tx=0x7ff75c005f90 comp rx=0 tx=0).stop 2024-08-31T07:18:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.077+0000 7ff770600640 1 -- 172.21.15.32:0/1051175757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff768078030 msgr2=0x7ff7681a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:18:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.077+0000 7ff770600640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff768078030 0x7ff7681a0d50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff76806c840 tx=0x7ff75400ee50 comp rx=0 tx=0).stop 2024-08-31T07:18:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.077+0000 7ff770600640 1 -- 172.21.15.32:0/1051175757 shutdown_connections 2024-08-31T07:18:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.077+0000 7ff770600640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff7500b4d20 0x7ff7500b7230 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.077+0000 7ff770600640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7681144e0 0x7ff7681a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.077+0000 7ff770600640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff768078030 0x7ff7681a0d50 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff76806c840 tx=0x7ff75400ee50 comp rx=0 tx=0).stop 2024-08-31T07:18:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.077+0000 7ff770600640 1 --2- 172.21.15.32:0/1051175757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680776e0 0x7ff7681a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:18:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.077+0000 7ff770600640 1 -- 172.21.15.32:0/1051175757 >> 172.21.15.32:0/1051175757 conn(0x7ff7681015b0 msgr2=0x7ff768116500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:18:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.078+0000 7ff770600640 1 -- 172.21.15.32:0/1051175757 shutdown_connections 2024-08-31T07:18:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:18:42.078+0000 7ff770600640 1 -- 172.21.15.32:0/1051175757 wait complete. 2024-08-31T07:18:42.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 161780 2.06 MB/sec execute 151 sec latency 32962.439 ms 2024-08-31T07:18:42.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:42 smithi032 ceph-mon[17252]: pgmap v546: 97 pgs: 97 active+clean; 100 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 481 KiB/s wr, 38 op/s 2024-08-31T07:18:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:42 smithi084 ceph-mon[22387]: pgmap v546: 97 pgs: 97 active+clean; 100 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 481 KiB/s wr, 38 op/s 2024-08-31T07:18:43.063 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '601ae51e', total seconds elapsed: 37.3 2024-08-31T07:18:43.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:42 smithi073 ceph-mon[21995]: pgmap v546: 97 pgs: 97 active+clean; 100 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 481 KiB/s wr, 38 op/s 2024-08-31T07:18:43.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 163505 2.13 MB/sec execute 152 sec latency 11.248 ms 2024-08-31T07:18:44.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 165026 2.16 MB/sec execute 153 sec latency 13.112 ms 2024-08-31T07:18:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:44 smithi084 ceph-mon[22387]: pgmap v547: 97 pgs: 97 active+clean; 100 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 421 KiB/s wr, 33 op/s 2024-08-31T07:18:45.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:44 smithi032 ceph-mon[17252]: pgmap v547: 97 pgs: 97 active+clean; 100 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 421 KiB/s wr, 33 op/s 2024-08-31T07:18:45.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:44 smithi073 ceph-mon[21995]: pgmap v547: 97 pgs: 97 active+clean; 100 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 421 KiB/s wr, 33 op/s 2024-08-31T07:18:45.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 166868 2.22 MB/sec execute 154 sec latency 191.682 ms 2024-08-31T07:18:46.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 168157 2.25 MB/sec execute 155 sec latency 16.230 ms 2024-08-31T07:18:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:46 smithi084 ceph-mon[22387]: pgmap v548: 97 pgs: 97 active+clean; 100 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 349 KiB/s wr, 46 op/s 2024-08-31T07:18:47.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:46 smithi032 ceph-mon[17252]: pgmap v548: 97 pgs: 97 active+clean; 100 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 349 KiB/s wr, 46 op/s 2024-08-31T07:18:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:46 smithi073 ceph-mon[21995]: pgmap v548: 97 pgs: 97 active+clean; 100 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 349 KiB/s wr, 46 op/s 2024-08-31T07:18:47.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170434 2.32 MB/sec execute 156 sec latency 14.559 ms 2024-08-31T07:18:48.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170866 2.31 MB/sec execute 157 sec latency 424.652 ms 2024-08-31T07:18:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:48 smithi084 ceph-mon[22387]: pgmap v549: 97 pgs: 97 active+clean; 104 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 671 KiB/s wr, 100 op/s 2024-08-31T07:18:49.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:48 smithi032 ceph-mon[17252]: pgmap v549: 97 pgs: 97 active+clean; 104 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 671 KiB/s wr, 100 op/s 2024-08-31T07:18:49.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:48 smithi073 ceph-mon[21995]: pgmap v549: 97 pgs: 97 active+clean; 104 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 671 KiB/s wr, 100 op/s 2024-08-31T07:18:49.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170866 2.30 MB/sec execute 158 sec latency 1424.749 ms 2024-08-31T07:18:50.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170866 2.28 MB/sec execute 159 sec latency 2424.912 ms 2024-08-31T07:18:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:50 smithi084 ceph-mon[22387]: pgmap v550: 97 pgs: 97 active+clean; 105 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 547 KiB/s wr, 124 op/s 2024-08-31T07:18:51.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:50 smithi032 ceph-mon[17252]: pgmap v550: 97 pgs: 97 active+clean; 105 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 547 KiB/s wr, 124 op/s 2024-08-31T07:18:51.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:50 smithi073 ceph-mon[21995]: pgmap v550: 97 pgs: 97 active+clean; 105 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 547 KiB/s wr, 124 op/s 2024-08-31T07:18:51.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170866 2.27 MB/sec execute 160 sec latency 3425.025 ms 2024-08-31T07:18:51.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:51 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:18:51.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:51 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:18:51.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:51 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:18:51.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:51 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:18:51.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:51 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:18:52.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:51 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:18:52.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:51 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:18:52.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:51 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:18:52.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:51 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:18:52.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:51 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:18:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:51 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:18:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:51 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:18:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:51 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:18:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:51 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:18:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:51 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:18:52.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170866 2.25 MB/sec execute 161 sec latency 4425.165 ms 2024-08-31T07:18:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:52 smithi084 ceph-mon[22387]: pgmap v551: 97 pgs: 97 active+clean; 106 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 646 KiB/s wr, 143 op/s 2024-08-31T07:18:53.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:52 smithi032 ceph-mon[17252]: pgmap v551: 97 pgs: 97 active+clean; 106 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 646 KiB/s wr, 143 op/s 2024-08-31T07:18:53.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:52 smithi073 ceph-mon[21995]: pgmap v551: 97 pgs: 97 active+clean; 106 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 646 KiB/s wr, 143 op/s 2024-08-31T07:18:53.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170866 2.24 MB/sec execute 162 sec latency 5425.292 ms 2024-08-31T07:18:54.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170866 2.23 MB/sec execute 163 sec latency 6425.420 ms 2024-08-31T07:18:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:54 smithi084 ceph-mon[22387]: pgmap v552: 97 pgs: 97 active+clean; 107 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 780 KiB/s wr, 162 op/s 2024-08-31T07:18:55.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:54 smithi032 ceph-mon[17252]: pgmap v552: 97 pgs: 97 active+clean; 107 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 780 KiB/s wr, 162 op/s 2024-08-31T07:18:55.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:54 smithi073 ceph-mon[21995]: pgmap v552: 97 pgs: 97 active+clean; 107 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 780 KiB/s wr, 162 op/s 2024-08-31T07:18:55.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170866 2.21 MB/sec execute 164 sec latency 7425.545 ms 2024-08-31T07:18:55.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:55 smithi084 ceph-mon[22387]: pgmap v553: 97 pgs: 97 active+clean; 107 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 778 KiB/s wr, 160 op/s 2024-08-31T07:18:56.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:55 smithi032 ceph-mon[17252]: pgmap v553: 97 pgs: 97 active+clean; 107 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 778 KiB/s wr, 160 op/s 2024-08-31T07:18:56.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:55 smithi073 ceph-mon[21995]: pgmap v553: 97 pgs: 97 active+clean; 107 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 778 KiB/s wr, 160 op/s 2024-08-31T07:18:56.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170866 2.20 MB/sec execute 165 sec latency 8425.684 ms 2024-08-31T07:18:57.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170866 2.19 MB/sec execute 166 sec latency 9425.750 ms 2024-08-31T07:18:58.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:57 smithi084 ceph-mon[22387]: pgmap v554: 97 pgs: 97 active+clean; 107 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 743 KiB/s wr, 149 op/s 2024-08-31T07:18:58.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:57 smithi032 ceph-mon[17252]: pgmap v554: 97 pgs: 97 active+clean; 107 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 743 KiB/s wr, 149 op/s 2024-08-31T07:18:58.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:57 smithi073 ceph-mon[21995]: pgmap v554: 97 pgs: 97 active+clean; 107 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 743 KiB/s wr, 149 op/s 2024-08-31T07:18:58.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170875 2.17 MB/sec execute 167 sec latency 9909.109 ms 2024-08-31T07:18:59.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 171803 2.19 MB/sec execute 168 sec latency 940.518 ms 2024-08-31T07:19:00.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:18:59 smithi084 ceph-mon[22387]: pgmap v555: 97 pgs: 97 active+clean; 107 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 504 KiB/s wr, 93 op/s 2024-08-31T07:19:00.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:18:59 smithi032 ceph-mon[17252]: pgmap v555: 97 pgs: 97 active+clean; 107 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 504 KiB/s wr, 93 op/s 2024-08-31T07:19:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:18:59 smithi073 ceph-mon[21995]: pgmap v555: 97 pgs: 97 active+clean; 107 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 504 KiB/s wr, 93 op/s 2024-08-31T07:19:00.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:19:00.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 173517 2.23 MB/sec execute 169 sec latency 29.734 ms 2024-08-31T07:19:01.130 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:01.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 174425 2.25 MB/sec execute 170 sec latency 28.553 ms 2024-08-31T07:19:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.766+0000 7f7f8a600640 1 -- 172.21.15.32:0/2890240470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f84109070 msgr2=0x7f7f84109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.766+0000 7f7f8a600640 1 --2- 172.21.15.32:0/2890240470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f84109070 0x7f7f84109480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009900 tx=0x7f7f7002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.766+0000 7f7f8a600640 1 -- 172.21.15.32:0/2890240470 shutdown_connections 2024-08-31T07:19:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.766+0000 7f7f8a600640 1 --2- 172.21.15.32:0/2890240470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f841099c0 0x7f7f841101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.766+0000 7f7f8a600640 1 --2- 172.21.15.32:0/2890240470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f84109070 0x7f7f84109480 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.766+0000 7f7f8a600640 1 --2- 172.21.15.32:0/2890240470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f84103730 0x7f7f84103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.766+0000 7f7f8a600640 1 -- 172.21.15.32:0/2890240470 >> 172.21.15.32:0/2890240470 conn(0x7f7f840ff180 msgr2=0x7f7f841015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.767+0000 7f7f8a600640 1 -- 172.21.15.32:0/2890240470 shutdown_connections 2024-08-31T07:19:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.767+0000 7f7f8a600640 1 -- 172.21.15.32:0/2890240470 wait complete. 2024-08-31T07:19:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.768+0000 7f7f8a600640 1 Processor -- start 2024-08-31T07:19:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.768+0000 7f7f8a600640 1 -- start start 2024-08-31T07:19:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f8a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f84103730 0x7f7f840754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f8a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f84109070 0x7f7f840759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f8a600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f841099c0 0x7f7f84075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f8a600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84113680 con 0x7f7f84109070 2024-08-31T07:19:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f8a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84113500 con 0x7f7f84103730 2024-08-31T07:19:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f8a600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84113800 con 0x7f7f841099c0 2024-08-31T07:19:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f84103730 0x7f7f840754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f83e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f84109070 0x7f7f840759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f841099c0 0x7f7f84075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f841099c0 0x7f7f84075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51974/0 (socket says 172.21.15.32:51974) 2024-08-31T07:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f84103730 0x7f7f840754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50726/0 (socket says 172.21.15.32:50726) 2024-08-31T07:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89c00640 1 -- 172.21.15.32:0/2719223281 learned_addr learned my addr 172.21.15.32:0/2719223281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89200640 1 -- 172.21.15.32:0/2719223281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f841099c0 msgr2=0x7f7f84075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89200640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f841099c0 0x7f7f84075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89200640 1 -- 172.21.15.32:0/2719223281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f84109070 msgr2=0x7f7f840759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89200640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f84109070 0x7f7f840759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89200640 1 -- 172.21.15.32:0/2719223281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f841a8330 con 0x7f7f84103730 2024-08-31T07:19:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.769+0000 7f7f89c00640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f841099c0 0x7f7f84075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.770+0000 7f7f89200640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f84103730 0x7f7f840754b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800e9f0 tx=0x7f7f7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.770+0000 7f7f81600640 1 -- 172.21.15.32:0/2719223281 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7800cd80 con 0x7f7f84103730 2024-08-31T07:19:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.770+0000 7f7f81600640 1 -- 172.21.15.32:0/2719223281 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f780035f0 con 0x7f7f84103730 2024-08-31T07:19:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.770+0000 7f7f8a600640 1 -- 172.21.15.32:0/2719223281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f841a8620 con 0x7f7f84103730 2024-08-31T07:19:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.770+0000 7f7f81600640 1 -- 172.21.15.32:0/2719223281 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78010670 con 0x7f7f84103730 2024-08-31T07:19:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.770+0000 7f7f8a600640 1 -- 172.21.15.32:0/2719223281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f841a8b80 con 0x7f7f84103730 2024-08-31T07:19:01.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.774+0000 7f7f92842740 1 Processor -- start 2024-08-31T07:19:01.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.774+0000 7f7f92842740 1 -- start start 2024-08-31T07:19:01.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.774+0000 7f7f81600640 1 -- 172.21.15.32:0/2719223281 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7f780108d0 con 0x7f7f84103730 2024-08-31T07:19:01.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.775+0000 7f7f92842740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 0x55620c465b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.775+0000 7f7f92842740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0971b0 0x55620c466060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.775+0000 7f7f92842740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c09f390 0x55620c4665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.775+0000 7f7f92842740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55620c3dc770 con 0x55620c0c0070 2024-08-31T07:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.775+0000 7f7f92842740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55620c3cbd00 con 0x55620c0971b0 2024-08-31T07:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.775+0000 7f7f92842740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55620c3cfcb0 con 0x55620c09f390 2024-08-31T07:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.775+0000 7f7f89200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0971b0 0x55620c466060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.775+0000 7f7f89200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0971b0 0x55620c466060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50732/0 (socket says 172.21.15.32:50732) 2024-08-31T07:19:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.775+0000 7f7f89c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 0x55620c465b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.775+0000 7f7f89200640 1 -- 172.21.15.32:0/922875024 learned_addr learned my addr 172.21.15.32:0/922875024 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.776+0000 7f7f89200640 1 -- 172.21.15.32:0/922875024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c09f390 msgr2=0x55620c4665a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:19:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.776+0000 7f7f89200640 1 --2- 172.21.15.32:0/922875024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c09f390 0x55620c4665a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.776+0000 7f7f89200640 1 -- 172.21.15.32:0/922875024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 msgr2=0x55620c465b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.776+0000 7f7f89200640 1 --2- 172.21.15.32:0/922875024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 0x55620c465b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.776+0000 7f7f89200640 1 -- 172.21.15.32:0/922875024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620c12b650 con 0x55620c0971b0 2024-08-31T07:19:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.776+0000 7f7f89c00640 1 --2- 172.21.15.32:0/922875024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 0x55620c465b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.777+0000 7f7f89200640 1 --2- 172.21.15.32:0/922875024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0971b0 0x55620c466060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7f780e2e10 tx=0x7f7f780ebeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab121f14b9273617 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.777+0000 7f7f76000640 1 -- 172.21.15.32:0/922875024 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78105070 con 0x55620c0971b0 2024-08-31T07:19:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.777+0000 7f7f81600640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7f6c0b4d20 0x7f7f6c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.777+0000 7f7f76000640 1 -- 172.21.15.32:0/922875024 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f780ede40 con 0x55620c0971b0 2024-08-31T07:19:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.777+0000 7f7f76000640 1 -- 172.21.15.32:0/922875024 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f781016b0 con 0x55620c0971b0 2024-08-31T07:19:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.777+0000 7f7f81600640 1 -- 172.21.15.32:0/2719223281 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7f7f780e5730 con 0x7f7f84103730 2024-08-31T07:19:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.778+0000 7f7f89c00640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7f6c0b4d20 0x7f7f6c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.779+0000 7f7f92842740 1 -- 172.21.15.32:0/922875024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0971b0 msgr2=0x55620c466060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.779+0000 7f7f92842740 1 --2- 172.21.15.32:0/922875024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0971b0 0x55620c466060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7f780e2e10 tx=0x7f7f780ebeb0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.779+0000 7f7f92842740 1 -- 172.21.15.32:0/922875024 shutdown_connections 2024-08-31T07:19:01.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.779+0000 7f7f92842740 1 --2- 172.21.15.32:0/922875024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c09f390 0x55620c4665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.779+0000 7f7f92842740 1 --2- 172.21.15.32:0/922875024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0971b0 0x55620c466060 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.779+0000 7f7f92842740 1 --2- 172.21.15.32:0/922875024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 0x55620c465b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.779+0000 7f7f92842740 1 -- 172.21.15.32:0/922875024 >> 172.21.15.32:0/922875024 conn(0x55620c0b4460 msgr2=0x55620c463a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:01.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.779+0000 7f7f89c00640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7f6c0b4d20 0x7f7f6c0b7230 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0049d0 tx=0x7f7f7c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:01.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.780+0000 7f7f92842740 1 -- 172.21.15.32:0/922875024 shutdown_connections 2024-08-31T07:19:01.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.780+0000 7f7f92842740 1 -- 172.21.15.32:0/922875024 wait complete. 2024-08-31T07:19:01.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.780+0000 7f7f92842740 1 -- start start 2024-08-31T07:19:01.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.781+0000 7f7f92842740 1 Processor -- start 2024-08-31T07:19:01.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f92842740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c479ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f92842740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c09f390 0x55620c47a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f92842740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0c0070 0x55620c485e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f92842740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55620c22e700 con 0x55620c09f390 2024-08-31T07:19:01.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f92842740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55620c352cb0 con 0x55620c0c0070 2024-08-31T07:19:01.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f92842740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55620c47ca30 con 0x55620c0971b0 2024-08-31T07:19:01.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f92842740 10 client.0 _ll_register_callbacks cb 0x55620c378430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:01.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f89200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c479ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f89200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c479ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51986/0 (socket says 172.21.15.32:51986) 2024-08-31T07:19:01.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f89200640 1 -- 172.21.15.32:0/177793603 learned_addr learned my addr 172.21.15.32:0/177793603 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:01.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f89c00640 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0c0070 0x55620c485e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.782+0000 7f7f83e00640 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c09f390 0x55620c47a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.783+0000 7f7f89200640 1 -- 172.21.15.32:0/177793603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0c0070 msgr2=0x55620c485e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.783+0000 7f7f89200640 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0c0070 0x55620c485e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.783+0000 7f7f89200640 1 -- 172.21.15.32:0/177793603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c09f390 msgr2=0x55620c47a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.783+0000 7f7f89200640 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c09f390 0x55620c47a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.783+0000 7f7f89200640 1 -- 172.21.15.32:0/177793603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620c393870 con 0x55620c0971b0 2024-08-31T07:19:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.783+0000 7f7f89200640 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c479ed0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f780ac000 tx=0x7f7f78005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2ab482c2b7414c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:01.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.783+0000 7f7f6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:01.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.783+0000 7f7f6aa00640 1 -- 172.21.15.32:0/177793603 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78105060 con 0x55620c0971b0 2024-08-31T07:19:01.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.783+0000 7f7f6aa00640 1 -- 172.21.15.32:0/177793603 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7f7f780037a0 con 0x55620c0971b0 2024-08-31T07:19:01.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.783+0000 7f7f92842740 1 -- 172.21.15.32:0/177793603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55620c3de6c0 con 0x55620c0971b0 2024-08-31T07:19:01.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f6aa00640 1 -- 172.21.15.32:0/177793603 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7f780a7f30 con 0x55620c0971b0 2024-08-31T07:19:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f92842740 10 client.26833 fetch_fsmap learned FSMap version 33 2024-08-31T07:19:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f92842740 1 -- 172.21.15.32:0/177793603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55620c488b30 con 0x55620c0971b0 2024-08-31T07:19:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f6aa00640 1 -- 172.21.15.32:0/177793603 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55620c3de6c0 con 0x55620c0971b0 2024-08-31T07:19:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f6aa00640 1 -- 172.21.15.32:0/177793603 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7f780e3120 con 0x55620c0971b0 2024-08-31T07:19:01.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f92842740 10 client.26833 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:19:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f92842740 10 client.26833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:19:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f92842740 20 client.26833 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f92842740 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55620c07e9f0 0x55620c48be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f92842740 4 client.26833 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.784+0000 7f7f92842740 1 -- 172.21.15.32:0/177793603 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55620c3da160 con 0x55620c07e9f0 2024-08-31T07:19:01.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.785+0000 7f7f83e00640 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55620c07e9f0 0x55620c48be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.785+0000 7f7f6aa00640 10 client.26833 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.785+0000 7f7f83e00640 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55620c07e9f0 0x55620c48be90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc2603a256e22a4 server_cookie=893e651999c79e31 in_seq=0 out_seq=0 2024-08-31T07:19:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.786+0000 7f7f6aa00640 1 -- 172.21.15.32:0/177793603 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55620c3da160 con 0x55620c07e9f0 2024-08-31T07:19:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.786+0000 7f7f6aa00640 10 client.26833 handle_command_reply: tid=0 2024-08-31T07:19:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.786+0000 7f7f92842740 1 client.26833 shutdown 2024-08-31T07:19:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 1 -- 172.21.15.32:0/177793603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 msgr2=0x55620c479ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c479ed0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f780ac000 tx=0x7f7f78005bb0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 1 -- 172.21.15.32:0/177793603 shutdown_connections 2024-08-31T07:19:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55620c07e9f0 0x55620c48be90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0c0070 0x55620c485e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c09f390 0x55620c47a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 1 --2- 172.21.15.32:0/177793603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c479ed0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 1 -- 172.21.15.32:0/177793603 >> 172.21.15.32:0/177793603 conn(0x55620c0b4460 msgr2=0x55620c461fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:01.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f6aa00640 0 client.26833 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:01.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 1 -- 172.21.15.32:0/177793603 shutdown_connections 2024-08-31T07:19:01.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 1 -- 172.21.15.32:0/177793603 wait complete. 2024-08-31T07:19:01.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.787+0000 7f7f92842740 20 client.26833 trim_cache size 0 max 16384 2024-08-31T07:19:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.807+0000 7f7f92842740 1 Processor -- start 2024-08-31T07:19:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.807+0000 7f7f92842740 1 -- start start 2024-08-31T07:19:01.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f92842740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c4634e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f92842740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c09f390 0x55620c476ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f92842740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 0x55620c477400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f92842740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84070e50 con 0x55620c0c0070 2024-08-31T07:19:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f92842740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84072990 con 0x55620c09f390 2024-08-31T07:19:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f92842740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84071520 con 0x55620c0971b0 2024-08-31T07:19:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f89200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c4634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f83e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c09f390 0x55620c476ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f89200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c4634e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51988/0 (socket says 172.21.15.32:51988) 2024-08-31T07:19:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f89200640 1 -- 172.21.15.32:0/215943049 learned_addr learned my addr 172.21.15.32:0/215943049 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.808+0000 7f7f89c00640 1 --2- 172.21.15.32:0/215943049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 0x55620c477400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.809+0000 7f7f89200640 1 -- 172.21.15.32:0/215943049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c09f390 msgr2=0x55620c476ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.809+0000 7f7f89200640 1 --2- 172.21.15.32:0/215943049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c09f390 0x55620c476ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.809+0000 7f7f89200640 1 -- 172.21.15.32:0/215943049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 msgr2=0x55620c477400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.809+0000 7f7f89200640 1 --2- 172.21.15.32:0/215943049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 0x55620c477400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.809+0000 7f7f89200640 1 -- 172.21.15.32:0/215943049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620c468930 con 0x55620c0971b0 2024-08-31T07:19:01.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.809+0000 7f7f89200640 1 --2- 172.21.15.32:0/215943049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c4634e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f780abd70 tx=0x7f7f78003d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cd977407ee3cf92 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.809+0000 7f7f75600640 1 -- 172.21.15.32:0/215943049 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78105030 con 0x55620c0971b0 2024-08-31T07:19:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.809+0000 7f7f75600640 1 -- 172.21.15.32:0/215943049 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f780d2070 con 0x55620c0971b0 2024-08-31T07:19:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.809+0000 7f7f75600640 1 -- 172.21.15.32:0/215943049 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780ab7e0 con 0x55620c0971b0 2024-08-31T07:19:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.810+0000 7f7f92842740 1 -- 172.21.15.32:0/215943049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 msgr2=0x55620c4634e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.810+0000 7f7f92842740 1 --2- 172.21.15.32:0/215943049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c4634e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f780abd70 tx=0x7f7f78003d90 comp rx=0 tx=0).stop 2024-08-31T07:19:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.810+0000 7f7f92842740 1 -- 172.21.15.32:0/215943049 shutdown_connections 2024-08-31T07:19:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.810+0000 7f7f92842740 1 --2- 172.21.15.32:0/215943049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c0c0070 0x55620c477400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.810+0000 7f7f92842740 1 --2- 172.21.15.32:0/215943049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c09f390 0x55620c476ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.810+0000 7f7f92842740 1 --2- 172.21.15.32:0/215943049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c4634e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.810+0000 7f7f92842740 1 -- 172.21.15.32:0/215943049 >> 172.21.15.32:0/215943049 conn(0x55620c0b4460 msgr2=0x55620c462dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.810+0000 7f7f92842740 1 -- 172.21.15.32:0/215943049 shutdown_connections 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.811+0000 7f7f92842740 1 -- 172.21.15.32:0/215943049 wait complete. 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.811+0000 7f7f92842740 1 -- start start 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.811+0000 7f7f92842740 1 Processor -- start 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f92842740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c48f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f92842740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c09f390 0x55620c48bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f92842740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0c0070 0x55620c48c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f92842740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7f840722c0 con 0x55620c09f390 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f92842740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84071bf0 con 0x55620c0c0070 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f92842740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8406ee50 con 0x55620c0971b0 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f92842740 10 client.0 _ll_register_callbacks cb 0x7f7f780e5a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f89200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c48f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f89200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c48f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51996/0 (socket says 172.21.15.32:51996) 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f89200640 1 -- 172.21.15.32:0/982446438 learned_addr learned my addr 172.21.15.32:0/982446438 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f89c00640 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0c0070 0x55620c48c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f83e00640 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c09f390 0x55620c48bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f89200640 1 -- 172.21.15.32:0/982446438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0c0070 msgr2=0x55620c48c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f89200640 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0c0070 0x55620c48c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f89200640 1 -- 172.21.15.32:0/982446438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c09f390 msgr2=0x55620c48bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f89200640 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c09f390 0x55620c48bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.812+0000 7f7f89200640 1 -- 172.21.15.32:0/982446438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620c48c7f0 con 0x55620c0971b0 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.813+0000 7f7f89200640 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c48f660 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f780ac000 tx=0x7f7f780eb9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bd67c91cdc12399 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.813+0000 7f7f6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.813+0000 7f7f6aa00640 1 -- 172.21.15.32:0/982446438 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780ab9c0 con 0x55620c0971b0 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.813+0000 7f7f6aa00640 1 -- 172.21.15.32:0/982446438 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7f7f780b0410 con 0x55620c0971b0 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.813+0000 7f7f92842740 1 -- 172.21.15.32:0/982446438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f780d2070 con 0x55620c0971b0 2024-08-31T07:19:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.813+0000 7f7f6aa00640 1 -- 172.21.15.32:0/982446438 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7f780abe40 con 0x55620c0971b0 2024-08-31T07:19:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.813+0000 7f7f92842740 10 client.26839 fetch_fsmap learned FSMap version 33 2024-08-31T07:19:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.813+0000 7f7f92842740 1 -- 172.21.15.32:0/982446438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55620c48c7f0 con 0x55620c0971b0 2024-08-31T07:19:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.814+0000 7f7f6aa00640 1 -- 172.21.15.32:0/982446438 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f780d2070 con 0x55620c0971b0 2024-08-31T07:19:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.814+0000 7f7f6aa00640 1 -- 172.21.15.32:0/982446438 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7f78010bc0 con 0x55620c0971b0 2024-08-31T07:19:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.814+0000 7f7f92842740 10 client.26839 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:19:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.814+0000 7f7f92842740 10 client.26839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:19:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.814+0000 7f7f92842740 20 client.26839 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.814+0000 7f7f92842740 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55620c053da0 0x55620c4f9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.814+0000 7f7f92842740 4 client.26839 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.814+0000 7f7f92842740 1 -- 172.21.15.32:0/982446438 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55620c3d78a0 con 0x55620c053da0 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.814+0000 7f7f83e00640 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55620c053da0 0x55620c4f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f83e00640 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55620c053da0 0x55620c4f9870 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20cc2f22f724ec69 server_cookie=16c148cd7c208ce5 in_seq=0 out_seq=0 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f6aa00640 10 client.26839 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f6aa00640 1 -- 172.21.15.32:0/982446438 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x55620c3d78a0 con 0x55620c053da0 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f6aa00640 10 client.26839 handle_command_reply: tid=0 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f92842740 1 client.26839 shutdown 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f92842740 1 -- 172.21.15.32:0/982446438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 msgr2=0x55620c48f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f92842740 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c48f660 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f780ac000 tx=0x7f7f780eb9f0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f92842740 1 -- 172.21.15.32:0/982446438 shutdown_connections 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f92842740 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55620c053da0 0x55620c4f9870 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f92842740 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55620c0c0070 0x55620c48c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f92842740 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55620c09f390 0x55620c48bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f92842740 1 --2- 172.21.15.32:0/982446438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55620c0971b0 0x55620c48f660 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f6aa00640 0 client.26839 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f92842740 1 -- 172.21.15.32:0/982446438 >> 172.21.15.32:0/982446438 conn(0x55620c0b4460 msgr2=0x55620c467280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.815+0000 7f7f92842740 1 -- 172.21.15.32:0/982446438 shutdown_connections 2024-08-31T07:19:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.816+0000 7f7f92842740 1 -- 172.21.15.32:0/982446438 wait complete. 2024-08-31T07:19:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.816+0000 7f7f92842740 20 client.26839 trim_cache size 0 max 16384 2024-08-31T07:19:01.822 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:19:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:19:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:19:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:19:01.825 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:01.826 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:01.827 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:19:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.816+0000 7f7f8a600640 1 -- 172.21.15.32:0/2719223281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7f6c0b4d20 msgr2=0x7f7f6c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.816+0000 7f7f8a600640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7f6c0b4d20 0x7f7f6c0b7230 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0049d0 tx=0x7f7f7c009500 comp rx=0 tx=0).stop 2024-08-31T07:19:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.816+0000 7f7f8a600640 1 -- 172.21.15.32:0/2719223281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f84103730 msgr2=0x7f7f840754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.816+0000 7f7f8a600640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f84103730 0x7f7f840754b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800e9f0 tx=0x7f7f7800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.817+0000 7f7f8a600640 1 -- 172.21.15.32:0/2719223281 shutdown_connections 2024-08-31T07:19:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.817+0000 7f7f8a600640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7f6c0b4d20 0x7f7f6c0b7230 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.817+0000 7f7f8a600640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f841099c0 0x7f7f84075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.817+0000 7f7f8a600640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f84109070 0x7f7f840759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.817+0000 7f7f8a600640 1 --2- 172.21.15.32:0/2719223281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f84103730 0x7f7f840754b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.817+0000 7f7f8a600640 1 -- 172.21.15.32:0/2719223281 >> 172.21.15.32:0/2719223281 conn(0x7f7f840ff180 msgr2=0x7f7f841015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.817+0000 7f7f8a600640 1 -- 172.21.15.32:0/2719223281 shutdown_connections 2024-08-31T07:19:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:01.817+0000 7f7f8a600640 1 -- 172.21.15.32:0/2719223281 wait complete. 2024-08-31T07:19:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:01 smithi032 ceph-mon[17252]: pgmap v556: 97 pgs: 97 active+clean; 109 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 950 KiB/s wr, 103 op/s 2024-08-31T07:19:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:01 smithi032 ceph-mon[17252]: osdmap e81: 12 total, 12 up, 12 in 2024-08-31T07:19:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:01 smithi073 ceph-mon[21995]: pgmap v556: 97 pgs: 97 active+clean; 109 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 950 KiB/s wr, 103 op/s 2024-08-31T07:19:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:01 smithi073 ceph-mon[21995]: osdmap e81: 12 total, 12 up, 12 in 2024-08-31T07:19:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:01 smithi084 ceph-mon[22387]: pgmap v556: 97 pgs: 97 active+clean; 109 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 950 KiB/s wr, 103 op/s 2024-08-31T07:19:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:01 smithi084 ceph-mon[22387]: osdmap e81: 12 total, 12 up, 12 in 2024-08-31T07:19:02.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 176240 2.28 MB/sec execute 171 sec latency 24.829 ms 2024-08-31T07:19:02.868 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:19:02.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:19:03.329 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:03.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 177774 2.33 MB/sec execute 172 sec latency 11.230 ms 2024-08-31T07:19:04.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.054+0000 7fa527200640 1 -- 172.21.15.32:0/1151931649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa528104eb0 msgr2=0x7fa52810d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:04.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.054+0000 7fa527200640 1 --2- 172.21.15.32:0/1151931649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa528104eb0 0x7fa52810d400 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa514009900 tx=0x7fa51402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.054+0000 7fa527200640 1 -- 172.21.15.32:0/1151931649 shutdown_connections 2024-08-31T07:19:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.054+0000 7fa527200640 1 --2- 172.21.15.32:0/1151931649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa52810d9c0 0x7fa52810feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.054+0000 7fa527200640 1 --2- 172.21.15.32:0/1151931649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa528104eb0 0x7fa52810d400 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.054+0000 7fa527200640 1 --2- 172.21.15.32:0/1151931649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa528104560 0x7fa528104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.054+0000 7fa527200640 1 -- 172.21.15.32:0/1151931649 >> 172.21.15.32:0/1151931649 conn(0x7fa5280fe060 msgr2=0x7fa5281004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.054+0000 7fa527200640 1 -- 172.21.15.32:0/1151931649 shutdown_connections 2024-08-31T07:19:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.054+0000 7fa527200640 1 -- 172.21.15.32:0/1151931649 wait complete. 2024-08-31T07:19:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.056+0000 7fa527200640 1 Processor -- start 2024-08-31T07:19:04.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.056+0000 7fa527200640 1 -- start start 2024-08-31T07:19:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.056+0000 7fa527200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa528104560 0x7fa5281022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.056+0000 7fa527200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa528104eb0 0x7fa5281027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.056+0000 7fa527200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa52810d9c0 0x7fa528102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.056+0000 7fa527200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281135f0 con 0x7fa528104eb0 2024-08-31T07:19:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.056+0000 7fa527200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa528113470 con 0x7fa528104560 2024-08-31T07:19:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.056+0000 7fa527200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa528113770 con 0x7fa52810d9c0 2024-08-31T07:19:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa525400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa528104eb0 0x7fa5281027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa525e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa528104560 0x7fa5281022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa526800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa52810d9c0 0x7fa528102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa525400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa528104eb0 0x7fa5281027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49282/0 (socket says 172.21.15.32:49282) 2024-08-31T07:19:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa525400640 1 -- 172.21.15.32:0/2123545703 learned_addr learned my addr 172.21.15.32:0/2123545703 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa525e00640 1 -- 172.21.15.32:0/2123545703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa52810d9c0 msgr2=0x7fa528102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa525e00640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa52810d9c0 0x7fa528102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa525e00640 1 -- 172.21.15.32:0/2123545703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa528104eb0 msgr2=0x7fa5281027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa525e00640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa528104eb0 0x7fa5281027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa525e00640 1 -- 172.21.15.32:0/2123545703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5281a8320 con 0x7fa528104560 2024-08-31T07:19:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.057+0000 7fa526800640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa52810d9c0 0x7fa528102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.058+0000 7fa525e00640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa528104560 0x7fa5281022a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa51800e9f0 tx=0x7fa51800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.058+0000 7fa51ea00640 1 -- 172.21.15.32:0/2123545703 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51800ccc0 con 0x7fa528104560 2024-08-31T07:19:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.058+0000 7fa51ea00640 1 -- 172.21.15.32:0/2123545703 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5180108e0 con 0x7fa528104560 2024-08-31T07:19:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.058+0000 7fa527200640 1 -- 172.21.15.32:0/2123545703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5281a8610 con 0x7fa528104560 2024-08-31T07:19:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.058+0000 7fa51ea00640 1 -- 172.21.15.32:0/2123545703 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518018880 con 0x7fa528104560 2024-08-31T07:19:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.058+0000 7fa527200640 1 -- 172.21.15.32:0/2123545703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5281a8ba0 con 0x7fa528104560 2024-08-31T07:19:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.059+0000 7fa527200640 1 -- 172.21.15.32:0/2123545703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa528104970 con 0x7fa528104560 2024-08-31T07:19:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.061+0000 7fa51ea00640 1 -- 172.21.15.32:0/2123545703 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa518010430 con 0x7fa528104560 2024-08-31T07:19:04.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.063+0000 7fa51ea00640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa50c0b4d20 0x7fa50c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.063+0000 7fa51ea00640 1 -- 172.21.15.32:0/2123545703 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7fa5180e5860 con 0x7fa528104560 2024-08-31T07:19:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.063+0000 7fa525400640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa50c0b4d20 0x7fa50c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.064+0000 7fa525400640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa50c0b4d20 0x7fa50c0b7230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa52810fb00 tx=0x7fa514009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.067+0000 7fa51ea00640 1 -- 172.21.15.32:0/2123545703 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa5180abfc0 con 0x7fa528104560 2024-08-31T07:19:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.343+0000 7fa527200640 1 -- 172.21.15.32:0/2123545703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5281035a0 con 0x7fa528104560 2024-08-31T07:19:04.346 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:03 smithi032 ceph-mon[17252]: pgmap v558: 97 pgs: 97 active+clean; 111 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-08-31T07:19:04.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.344+0000 7fa51ea00640 1 -- 172.21.15.32:0/2123545703 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11960 (secure 0 0 0) 0x7fa5180b0780 con 0x7fa528104560 2024-08-31T07:19:04.346 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:19:04.347 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"btime":"2024-08-31T07:17:01:405815+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:17:00.944291+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:19:04.347 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2024-08-31T07:19:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.347+0000 7fa527200640 1 -- 172.21.15.32:0/2123545703 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa50c0b4d20 msgr2=0x7fa50c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.347+0000 7fa527200640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa50c0b4d20 0x7fa50c0b7230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa52810fb00 tx=0x7fa514009560 comp rx=0 tx=0).stop 2024-08-31T07:19:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.347+0000 7fa527200640 1 -- 172.21.15.32:0/2123545703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa528104560 msgr2=0x7fa5281022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.347+0000 7fa527200640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa528104560 0x7fa5281022a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa51800e9f0 tx=0x7fa51800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.347+0000 7fa527200640 1 -- 172.21.15.32:0/2123545703 shutdown_connections 2024-08-31T07:19:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.347+0000 7fa527200640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa50c0b4d20 0x7fa50c0b7230 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.347+0000 7fa527200640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa52810d9c0 0x7fa528102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.347+0000 7fa527200640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa528104eb0 0x7fa5281027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.348+0000 7fa527200640 1 --2- 172.21.15.32:0/2123545703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa528104560 0x7fa5281022a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.348+0000 7fa527200640 1 -- 172.21.15.32:0/2123545703 >> 172.21.15.32:0/2123545703 conn(0x7fa5280fe060 msgr2=0x7fa5280fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.348+0000 7fa527200640 1 -- 172.21.15.32:0/2123545703 shutdown_connections 2024-08-31T07:19:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:04.348+0000 7fa527200640 1 -- 172.21.15.32:0/2123545703 wait complete. 2024-08-31T07:19:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:03 smithi073 ceph-mon[21995]: pgmap v558: 97 pgs: 97 active+clean; 111 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-08-31T07:19:04.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:03 smithi084 ceph-mon[22387]: pgmap v558: 97 pgs: 97 active+clean; 111 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-08-31T07:19:04.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 179289 2.35 MB/sec execute 173 sec latency 15.739 ms 2024-08-31T07:19:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:05 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2123545703' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:19:05.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:19:05.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:05 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2123545703' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:19:05.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:05 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2123545703' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:19:05.678 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:05.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 181166 2.41 MB/sec execute 174 sec latency 14.002 ms 2024-08-31T07:19:06.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:06 smithi032 ceph-mon[17252]: pgmap v559: 97 pgs: 97 active+clean; 110 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-08-31T07:19:06.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.368+0000 7f9cffa00640 1 -- 172.21.15.32:0/3657124413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cf8102560 msgr2=0x7f9cf8102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.368+0000 7f9cffa00640 1 --2- 172.21.15.32:0/3657124413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cf8102560 0x7f9cf8102970 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009900 tx=0x7f9cec02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.368+0000 7f9cffa00640 1 -- 172.21.15.32:0/3657124413 shutdown_connections 2024-08-31T07:19:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.368+0000 7f9cffa00640 1 --2- 172.21.15.32:0/3657124413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf81079a0 0x7f9cf8107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.368+0000 7f9cffa00640 1 --2- 172.21.15.32:0/3657124413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cf8102eb0 0x7f9cf81073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.368+0000 7f9cffa00640 1 --2- 172.21.15.32:0/3657124413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cf8102560 0x7f9cf8102970 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.368+0000 7f9cffa00640 1 -- 172.21.15.32:0/3657124413 >> 172.21.15.32:0/3657124413 conn(0x7f9cf80794e0 msgr2=0x7f9cf80798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.368+0000 7f9cffa00640 1 -- 172.21.15.32:0/3657124413 shutdown_connections 2024-08-31T07:19:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.368+0000 7f9cffa00640 1 -- 172.21.15.32:0/3657124413 wait complete. 2024-08-31T07:19:06.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.370+0000 7f9cffa00640 1 Processor -- start 2024-08-31T07:19:06.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.370+0000 7f9cffa00640 1 -- start start 2024-08-31T07:19:06.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.370+0000 7f9cffa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cf8102560 0x7f9cf81a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.370+0000 7f9cffa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf8102eb0 0x7f9cf81a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.370+0000 7f9cffa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cf81079a0 0x7f9cf81a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cffa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf8119e90 con 0x7f9cf8102eb0 2024-08-31T07:19:06.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cffa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf8119d10 con 0x7f9cf8102560 2024-08-31T07:19:06.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cffa00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf811a010 con 0x7f9cf81079a0 2024-08-31T07:19:06.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cfd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cf8102560 0x7f9cf81a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cfd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cf8102560 0x7f9cf81a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36720/0 (socket says 172.21.15.32:36720) 2024-08-31T07:19:06.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf8102eb0 0x7f9cf81a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cfd000640 1 -- 172.21.15.32:0/2416715383 learned_addr learned my addr 172.21.15.32:0/2416715383 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:06.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cfda00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cf81079a0 0x7f9cf81a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cfda00640 1 -- 172.21.15.32:0/2416715383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cf8102560 msgr2=0x7f9cf81a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:06.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cfda00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cf8102560 0x7f9cf81a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cfda00640 1 -- 172.21.15.32:0/2416715383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf8102eb0 msgr2=0x7f9cf81a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:06.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cfda00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf8102eb0 0x7f9cf81a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cfda00640 1 -- 172.21.15.32:0/2416715383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf81a6850 con 0x7f9cf81079a0 2024-08-31T07:19:06.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.371+0000 7f9cfd000640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cf8102560 0x7f9cf81a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:06.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.372+0000 7f9cfda00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cf81079a0 0x7f9cf81a6120 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009fd0 tx=0x7f9cf000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:06.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.372+0000 7f9cf5600640 1 -- 172.21.15.32:0/2416715383 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000ce20 con 0x7f9cf81079a0 2024-08-31T07:19:06.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.372+0000 7f9cf5600640 1 -- 172.21.15.32:0/2416715383 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cf0005020 con 0x7f9cf81079a0 2024-08-31T07:19:06.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.372+0000 7f9cf5600640 1 -- 172.21.15.32:0/2416715383 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0010690 con 0x7f9cf81079a0 2024-08-31T07:19:06.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.372+0000 7f9cffa00640 1 -- 172.21.15.32:0/2416715383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf81a6b40 con 0x7f9cf81079a0 2024-08-31T07:19:06.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.372+0000 7f9cffa00640 1 -- 172.21.15.32:0/2416715383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf81ad2a0 con 0x7f9cf81079a0 2024-08-31T07:19:06.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.376+0000 7f9d06624740 1 Processor -- start 2024-08-31T07:19:06.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.377+0000 7f9cf5600640 1 -- 172.21.15.32:0/2416715383 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9cf00054d0 con 0x7f9cf81079a0 2024-08-31T07:19:06.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.377+0000 7f9d06624740 1 -- start start 2024-08-31T07:19:06.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.377+0000 7f9d06624740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da24070 0x56301ddc98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9d06624740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddc9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9d06624740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da03390 0x56301ddca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9d06624740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56301dd407b0 con 0x56301da24070 2024-08-31T07:19:06.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9d06624740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56301dd2fd40 con 0x56301da03390 2024-08-31T07:19:06.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9d06624740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56301dd33cf0 con 0x56301d9fb1b0 2024-08-31T07:19:06.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cf7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da24070 0x56301ddc98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cfd000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddc9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cf7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da24070 0x56301ddc98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47032/0 (socket says 172.21.15.32:47032) 2024-08-31T07:19:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cf7e00640 1 -- 172.21.15.32:0/251903142 learned_addr learned my addr 172.21.15.32:0/251903142 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/251903142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da03390 0x56301ddca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cfd000640 1 -- 172.21.15.32:0/251903142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da03390 msgr2=0x56301ddca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cfd000640 1 --2- 172.21.15.32:0/251903142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da03390 0x56301ddca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cfd000640 1 -- 172.21.15.32:0/251903142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da24070 msgr2=0x56301ddc98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cfd000640 1 --2- 172.21.15.32:0/251903142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da24070 0x56301ddc98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cfd000640 1 -- 172.21.15.32:0/251903142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56301da8f650 con 0x56301d9fb1b0 2024-08-31T07:19:06.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.378+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/251903142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da24070 0x56301ddc98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:06.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.379+0000 7f9cfd000640 1 --2- 172.21.15.32:0/251903142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddc9df0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000c00 tx=0x7f9cec004480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e28bc1430054c6d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:06.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.379+0000 7f9cea000640 1 -- 172.21.15.32:0/251903142 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec035040 con 0x56301d9fb1b0 2024-08-31T07:19:06.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.379+0000 7f9cea000640 1 -- 172.21.15.32:0/251903142 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cec004c10 con 0x56301d9fb1b0 2024-08-31T07:19:06.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.379+0000 7f9cea000640 1 -- 172.21.15.32:0/251903142 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec003930 con 0x56301d9fb1b0 2024-08-31T07:19:06.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.379+0000 7f9cf5600640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ce00b4df0 0x7f9ce00b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.380+0000 7f9cf5600640 1 -- 172.21.15.32:0/2416715383 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7f9cf00e5b10 con 0x7f9cf81079a0 2024-08-31T07:19:06.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.380+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ce00b4df0 0x7f9ce00b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.380+0000 7f9d06624740 1 -- 172.21.15.32:0/251903142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 msgr2=0x56301ddc9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:06.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.380+0000 7f9d06624740 1 --2- 172.21.15.32:0/251903142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddc9df0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000c00 tx=0x7f9cec004480 comp rx=0 tx=0).stop 2024-08-31T07:19:06.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.380+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ce00b4df0 0x7f9ce00b7300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40058c0 tx=0x7f9ce4005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:06.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.381+0000 7f9d06624740 1 -- 172.21.15.32:0/251903142 shutdown_connections 2024-08-31T07:19:06.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.381+0000 7f9d06624740 1 --2- 172.21.15.32:0/251903142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da03390 0x56301ddca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.381+0000 7f9d06624740 1 --2- 172.21.15.32:0/251903142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddc9df0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.381+0000 7f9d06624740 1 --2- 172.21.15.32:0/251903142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da24070 0x56301ddc98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.381+0000 7f9d06624740 1 -- 172.21.15.32:0/251903142 >> 172.21.15.32:0/251903142 conn(0x56301da18460 msgr2=0x56301ddc77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:06.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.381+0000 7f9d06624740 1 -- 172.21.15.32:0/251903142 shutdown_connections 2024-08-31T07:19:06.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.381+0000 7f9d06624740 1 -- 172.21.15.32:0/251903142 wait complete. 2024-08-31T07:19:06.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.381+0000 7f9d06624740 1 -- start start 2024-08-31T07:19:06.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9d06624740 1 Processor -- start 2024-08-31T07:19:06.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9d06624740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9d06624740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 0x56301ddd5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9d06624740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301dde1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9d06624740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56301db92700 con 0x56301da03390 2024-08-31T07:19:06.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9d06624740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56301dcb6cf0 con 0x56301da24070 2024-08-31T07:19:06.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9d06624740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56301ddd8030 con 0x56301d9fb1b0 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9d06624740 10 client.0 _ll_register_callbacks cb 0x56301dcdc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9cfd000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9cfd000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60332/0 (socket says 172.21.15.32:60332) 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.382+0000 7f9cfd000640 1 -- 172.21.15.32:0/513719492 learned_addr learned my addr 172.21.15.32:0/513719492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.383+0000 7f9cfda00640 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301dde1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.383+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 0x56301ddd5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.383+0000 7f9cfd000640 1 -- 172.21.15.32:0/513719492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 msgr2=0x56301dde1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.383+0000 7f9cfd000640 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301dde1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.383+0000 7f9cfd000640 1 -- 172.21.15.32:0/513719492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 msgr2=0x56301ddd5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.383+0000 7f9cfd000640 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 0x56301ddd5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.383+0000 7f9cfd000640 1 -- 172.21.15.32:0/513719492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56301dcf78b0 con 0x56301d9fb1b0 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.383+0000 7f9cfd000640 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd54a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9cec005f70 tx=0x7f9cec009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30c9eae96d941957 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:06.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.383+0000 7f9cdea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:06.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.384+0000 7f9cdea00640 1 -- 172.21.15.32:0/513719492 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec052060 con 0x56301d9fb1b0 2024-08-31T07:19:06.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.384+0000 7f9cdea00640 1 -- 172.21.15.32:0/513719492 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7f9cec0049f0 con 0x56301d9fb1b0 2024-08-31T07:19:06.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.384+0000 7f9d06624740 1 -- 172.21.15.32:0/513719492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cec004c10 con 0x56301d9fb1b0 2024-08-31T07:19:06.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.384+0000 7f9cdea00640 1 -- 172.21.15.32:0/513719492 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9cec035050 con 0x56301d9fb1b0 2024-08-31T07:19:06.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.384+0000 7f9d06624740 10 client.26857 fetch_fsmap learned FSMap version 33 2024-08-31T07:19:06.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.384+0000 7f9d06624740 1 -- 172.21.15.32:0/513719492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56301dde4090 con 0x56301d9fb1b0 2024-08-31T07:19:06.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.384+0000 7f9cdea00640 1 -- 172.21.15.32:0/513719492 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cec004c10 con 0x56301d9fb1b0 2024-08-31T07:19:06.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.385+0000 7f9cdea00640 1 -- 172.21.15.32:0/513719492 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f9cec003ce0 con 0x56301d9fb1b0 2024-08-31T07:19:06.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.385+0000 7f9d06624740 10 client.26857 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:19:06.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.385+0000 7f9d06624740 10 client.26857 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:19:06.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.385+0000 7f9d06624740 20 client.26857 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.385+0000 7f9d06624740 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56301d9e29f0 0x56301dde73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.385+0000 7f9d06624740 4 client.26857 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.385+0000 7f9d06624740 1 -- 172.21.15.32:0/513719492 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56301dd42700 con 0x56301d9e29f0 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.385+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56301d9e29f0 0x56301dde73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.386+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56301d9e29f0 0x56301dde73f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc72c491b3c656b2 server_cookie=98eb7164af179d0 in_seq=0 out_seq=0 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.386+0000 7f9cdea00640 10 client.26857 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9cdea00640 1 -- 172.21.15.32:0/513719492 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56301dd42700 con 0x56301d9e29f0 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9cdea00640 10 client.26857 handle_command_reply: tid=0 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 client.26857 shutdown 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 -- 172.21.15.32:0/513719492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 msgr2=0x56301ddd54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd54a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9cec005f70 tx=0x7f9cec009520 comp rx=0 tx=0).stop 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 -- 172.21.15.32:0/513719492 shutdown_connections 2024-08-31T07:19:06.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56301d9e29f0 0x56301dde73f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301dde1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 0x56301ddd5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9cdea00640 0 client.26857 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:19:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 --2- 172.21.15.32:0/513719492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd54a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 -- 172.21.15.32:0/513719492 >> 172.21.15.32:0/513719492 conn(0x56301da18460 msgr2=0x56301ddcca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 -- 172.21.15.32:0/513719492 shutdown_connections 2024-08-31T07:19:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 1 -- 172.21.15.32:0/513719492 wait complete. 2024-08-31T07:19:06.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.387+0000 7f9d06624740 20 client.26857 trim_cache size 0 max 16384 2024-08-31T07:19:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.401+0000 7f9d06624740 1 Processor -- start 2024-08-31T07:19:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9d06624740 1 -- start start 2024-08-31T07:19:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9d06624740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9d06624740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 0x56301ddc7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9d06624740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301dde1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9d06624740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf81b5940 con 0x56301da03390 2024-08-31T07:19:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9d06624740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf81b7480 con 0x56301da24070 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9d06624740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf81b6010 con 0x56301d9fb1b0 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9cfd000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9cfd000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60336/0 (socket says 172.21.15.32:60336) 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9cfd000640 1 -- 172.21.15.32:0/1486175270 learned_addr learned my addr 172.21.15.32:0/1486175270 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/1486175270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 0x56301ddc7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.402+0000 7f9cfda00640 1 --2- 172.21.15.32:0/1486175270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301dde1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.403+0000 7f9cfd000640 1 -- 172.21.15.32:0/1486175270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 msgr2=0x56301dde1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.403+0000 7f9cfd000640 1 --2- 172.21.15.32:0/1486175270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301dde1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.403+0000 7f9cfd000640 1 -- 172.21.15.32:0/1486175270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 msgr2=0x56301ddc7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.403+0000 7f9cfd000640 1 --2- 172.21.15.32:0/1486175270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 0x56301ddc7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.403+0000 7f9cfd000640 1 -- 172.21.15.32:0/1486175270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56301dde7760 con 0x56301d9fb1b0 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.403+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/1486175270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 0x56301ddc7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.403+0000 7f9cfd000640 1 --2- 172.21.15.32:0/1486175270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd4190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9cec03c940 tx=0x7f9cec005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7cca0dc1779fe3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.403+0000 7f9ce9600640 1 -- 172.21.15.32:0/1486175270 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec052040 con 0x56301d9fb1b0 2024-08-31T07:19:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.403+0000 7f9ce9600640 1 -- 172.21.15.32:0/1486175270 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cec005850 con 0x56301d9fb1b0 2024-08-31T07:19:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.403+0000 7f9ce9600640 1 -- 172.21.15.32:0/1486175270 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0055f0 con 0x56301d9fb1b0 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.404+0000 7f9d06624740 1 -- 172.21.15.32:0/1486175270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 msgr2=0x56301ddd4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.404+0000 7f9d06624740 1 --2- 172.21.15.32:0/1486175270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd4190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9cec03c940 tx=0x7f9cec005fd0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.404+0000 7f9d06624740 1 -- 172.21.15.32:0/1486175270 shutdown_connections 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.404+0000 7f9d06624740 1 --2- 172.21.15.32:0/1486175270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301dde1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.404+0000 7f9d06624740 1 --2- 172.21.15.32:0/1486175270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301da03390 0x56301ddc7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.404+0000 7f9d06624740 1 --2- 172.21.15.32:0/1486175270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301d9fb1b0 0x56301ddd4190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.404+0000 7f9d06624740 1 -- 172.21.15.32:0/1486175270 >> 172.21.15.32:0/1486175270 conn(0x56301da18460 msgr2=0x56301ddc7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.404+0000 7f9d06624740 1 -- 172.21.15.32:0/1486175270 shutdown_connections 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.404+0000 7f9d06624740 1 -- 172.21.15.32:0/1486175270 wait complete. 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.405+0000 7f9d06624740 1 -- start start 2024-08-31T07:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.405+0000 7f9d06624740 1 Processor -- start 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.405+0000 7f9d06624740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301d9fb1b0 0x56301ddd9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.405+0000 7f9d06624740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301da03390 0x56301ddda290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9d06624740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301de4abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9d06624740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf81b6db0 con 0x56301d9fb1b0 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9d06624740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf81b66e0 con 0x56301da24070 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9d06624740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf81b3940 con 0x56301da03390 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9d06624740 10 client.0 _ll_register_callbacks cb 0x7f9cec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9cf7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301da03390 0x56301ddda290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9cfda00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301de4abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9cf7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301da03390 0x56301ddda290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60346/0 (socket says 172.21.15.32:60346) 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9cf7e00640 1 -- 172.21.15.32:0/4088317760 learned_addr learned my addr 172.21.15.32:0/4088317760 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9cfd000640 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301d9fb1b0 0x56301ddd9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9cf7e00640 1 -- 172.21.15.32:0/4088317760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 msgr2=0x56301de4abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301de4abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9cf7e00640 1 -- 172.21.15.32:0/4088317760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301d9fb1b0 msgr2=0x56301ddd9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301d9fb1b0 0x56301ddd9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.406+0000 7f9cf7e00640 1 -- 172.21.15.32:0/4088317760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56301de4b240 con 0x56301da03390 2024-08-31T07:19:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.407+0000 7f9cf7e00640 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301da03390 0x56301ddda290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x56301dcb6df0 tx=0x7f9ce4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e4a2f14179b5d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.407+0000 7f9cdea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.407+0000 7f9cdea00640 1 -- 172.21.15.32:0/4088317760 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4024880 con 0x56301da03390 2024-08-31T07:19:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.407+0000 7f9d06624740 1 -- 172.21.15.32:0/4088317760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cec005850 con 0x56301da03390 2024-08-31T07:19:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.407+0000 7f9cdea00640 1 -- 172.21.15.32:0/4088317760 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7f9ce4024a20 con 0x56301da03390 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.407+0000 7f9cdea00640 1 -- 172.21.15.32:0/4088317760 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9ce4024c20 con 0x56301da03390 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.407+0000 7f9d06624740 10 client.26863 fetch_fsmap learned FSMap version 33 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.407+0000 7f9d06624740 1 -- 172.21.15.32:0/4088317760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56301de52670 con 0x56301da03390 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.407+0000 7f9cdea00640 1 -- 172.21.15.32:0/4088317760 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cec005850 con 0x56301da03390 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.408+0000 7f9cdea00640 1 -- 172.21.15.32:0/4088317760 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f9ce403c050 con 0x56301da03390 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.408+0000 7f9d06624740 10 client.26863 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.408+0000 7f9d06624740 10 client.26863 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.408+0000 7f9d06624740 20 client.26863 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.408+0000 7f9d06624740 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56301d9b7da0 0x56301de54db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.408+0000 7f9d06624740 4 client.26863 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.408+0000 7f9d06624740 1 -- 172.21.15.32:0/4088317760 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56301dd3e1a0 con 0x56301d9b7da0 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.408+0000 7f9cfd000640 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56301d9b7da0 0x56301de54db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.408+0000 7f9cfd000640 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56301d9b7da0 0x56301de54db0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8098f1f070060ed3 server_cookie=154a90544f694d6b in_seq=0 out_seq=0 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.408+0000 7f9cdea00640 10 client.26863 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9cdea00640 1 -- 172.21.15.32:0/4088317760 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56301dd3e1a0 con 0x56301d9b7da0 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9cdea00640 10 client.26863 handle_command_reply: tid=0 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9d06624740 1 client.26863 shutdown 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9d06624740 1 -- 172.21.15.32:0/4088317760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301da03390 msgr2=0x56301ddda290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9d06624740 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301da03390 0x56301ddda290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x56301dcb6df0 tx=0x7f9ce4005f90 comp rx=0 tx=0).stop 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9d06624740 1 -- 172.21.15.32:0/4088317760 shutdown_connections 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9d06624740 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x56301d9b7da0 0x56301de54db0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9d06624740 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56301da24070 0x56301de4abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9d06624740 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56301da03390 0x56301ddda290 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9d06624740 1 --2- 172.21.15.32:0/4088317760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56301d9fb1b0 0x56301ddd9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9cdea00640 0 client.26863 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9d06624740 1 -- 172.21.15.32:0/4088317760 >> 172.21.15.32:0/4088317760 conn(0x56301da18460 msgr2=0x56301ddca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.409+0000 7f9d06624740 1 -- 172.21.15.32:0/4088317760 shutdown_connections 2024-08-31T07:19:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.410+0000 7f9d06624740 1 -- 172.21.15.32:0/4088317760 wait complete. 2024-08-31T07:19:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.410+0000 7f9d06624740 20 client.26863 trim_cache size 0 max 16384 2024-08-31T07:19:06.414 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:19:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.410+0000 7f9cffa00640 1 -- 172.21.15.32:0/2416715383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ce00b4df0 msgr2=0x7f9ce00b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.410+0000 7f9cffa00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ce00b4df0 0x7f9ce00b7300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40058c0 tx=0x7f9ce4005850 comp rx=0 tx=0).stop 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.410+0000 7f9cffa00640 1 -- 172.21.15.32:0/2416715383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cf81079a0 msgr2=0x7f9cf81a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.410+0000 7f9cffa00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cf81079a0 0x7f9cf81a6120 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009fd0 tx=0x7f9cf000ef10 comp rx=0 tx=0).stop 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.411+0000 7f9cffa00640 1 -- 172.21.15.32:0/2416715383 shutdown_connections 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.411+0000 7f9cffa00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ce00b4df0 0x7f9ce00b7300 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.411+0000 7f9cffa00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cf81079a0 0x7f9cf81a6120 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.411+0000 7f9cffa00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf8102eb0 0x7f9cf81a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.411+0000 7f9cffa00640 1 --2- 172.21.15.32:0/2416715383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cf8102560 0x7f9cf81a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.411+0000 7f9cffa00640 1 -- 172.21.15.32:0/2416715383 >> 172.21.15.32:0/2416715383 conn(0x7f9cf80794e0 msgr2=0x7f9cf81053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.411+0000 7f9cffa00640 1 -- 172.21.15.32:0/2416715383 shutdown_connections 2024-08-31T07:19:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:06.411+0000 7f9cffa00640 1 -- 172.21.15.32:0/2416715383 wait complete. 2024-08-31T07:19:06.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:06 smithi073 ceph-mon[21995]: pgmap v559: 97 pgs: 97 active+clean; 110 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-08-31T07:19:06.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:06 smithi084 ceph-mon[22387]: pgmap v559: 97 pgs: 97 active+clean; 110 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-08-31T07:19:06.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 182322 2.43 MB/sec execute 175 sec latency 15.474 ms 2024-08-31T07:19:07.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:19:07.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 184440 2.49 MB/sec execute 176 sec latency 13.905 ms 2024-08-31T07:19:07.726 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:08.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:08 smithi032 ceph-mon[17252]: pgmap v560: 97 pgs: 97 active+clean; 114 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2024-08-31T07:19:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:08 smithi073 ceph-mon[21995]: pgmap v560: 97 pgs: 97 active+clean; 114 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2024-08-31T07:19:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:08 smithi084 ceph-mon[22387]: pgmap v560: 97 pgs: 97 active+clean; 114 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2024-08-31T07:19:08.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 185528 2.49 MB/sec execute 177 sec latency 14.074 ms 2024-08-31T07:19:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.732+0000 7fe5eae00640 1 -- 172.21.15.32:0/2105284032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5e4105960 msgr2=0x7fe5e4105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.732+0000 7fe5eae00640 1 --2- 172.21.15.32:0/2105284032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5e4105960 0x7fe5e4105d70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009860 tx=0x7fe5d802a340 comp rx=0 tx=0).stop 2024-08-31T07:19:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.732+0000 7fe5eae00640 1 -- 172.21.15.32:0/2105284032 shutdown_connections 2024-08-31T07:19:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.732+0000 7fe5eae00640 1 --2- 172.21.15.32:0/2105284032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e410bbf0 0x7fe5e4112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.732+0000 7fe5eae00640 1 --2- 172.21.15.32:0/2105284032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e410b2a0 0x7fe5e410b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.732+0000 7fe5eae00640 1 --2- 172.21.15.32:0/2105284032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5e4105960 0x7fe5e4105d70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.732+0000 7fe5eae00640 1 -- 172.21.15.32:0/2105284032 >> 172.21.15.32:0/2105284032 conn(0x7fe5e41013d0 msgr2=0x7fe5e4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.733+0000 7fe5eae00640 1 -- 172.21.15.32:0/2105284032 shutdown_connections 2024-08-31T07:19:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.733+0000 7fe5eae00640 1 -- 172.21.15.32:0/2105284032 wait complete. 2024-08-31T07:19:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.734+0000 7fe5eae00640 1 Processor -- start 2024-08-31T07:19:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.734+0000 7fe5eae00640 1 -- start start 2024-08-31T07:19:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5eae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e4105960 0x7fe5e419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5eae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5e410b2a0 0x7fe5e419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5eae00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e410bbf0 0x7fe5e41a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5eae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e41158b0 con 0x7fe5e4105960 2024-08-31T07:19:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5eae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4115730 con 0x7fe5e410b2a0 2024-08-31T07:19:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5eae00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4115a30 con 0x7fe5e410bbf0 2024-08-31T07:19:08.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e4105960 0x7fe5e419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e4105960 0x7fe5e419c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47074/0 (socket says 172.21.15.32:47074) 2024-08-31T07:19:08.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5e9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5e410b2a0 0x7fe5e419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5e9a00640 1 -- 172.21.15.32:0/211078648 learned_addr learned my addr 172.21.15.32:0/211078648 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:08.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5ea400640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e410bbf0 0x7fe5e41a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5e9a00640 1 -- 172.21.15.32:0/211078648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e410bbf0 msgr2=0x7fe5e41a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e410bbf0 0x7fe5e41a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5e9a00640 1 -- 172.21.15.32:0/211078648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5e410b2a0 msgr2=0x7fe5e419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5e410b2a0 0x7fe5e419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5e9a00640 1 -- 172.21.15.32:0/211078648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e41a2290 con 0x7fe5e4105960 2024-08-31T07:19:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5e9000640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5e410b2a0 0x7fe5e419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.735+0000 7fe5ea400640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e410bbf0 0x7fe5e41a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.736+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e4105960 0x7fe5e419c320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80303f0 tx=0x7fe5d8032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.736+0000 7fe5e2000640 1 -- 172.21.15.32:0/211078648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8040020 con 0x7fe5e4105960 2024-08-31T07:19:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.736+0000 7fe5eae00640 1 -- 172.21.15.32:0/211078648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e41a2520 con 0x7fe5e4105960 2024-08-31T07:19:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.736+0000 7fe5e2000640 1 -- 172.21.15.32:0/211078648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d8030dd0 con 0x7fe5e4105960 2024-08-31T07:19:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.736+0000 7fe5e2000640 1 -- 172.21.15.32:0/211078648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8037040 con 0x7fe5e4105960 2024-08-31T07:19:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.736+0000 7fe5eae00640 1 -- 172.21.15.32:0/211078648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e41a8ce0 con 0x7fe5e4105960 2024-08-31T07:19:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.740+0000 7fe5f30c0740 1 Processor -- start 2024-08-31T07:19:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.740+0000 7fe5f30c0740 1 -- start start 2024-08-31T07:19:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.740+0000 7fe5e2000640 1 -- 172.21.15.32:0/211078648 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe5d8043020 con 0x7fe5e4105960 2024-08-31T07:19:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327df8070 0x56332819db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327dcf1b0 0x56332819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x56332819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5633281147b0 con 0x563327df8070 2024-08-31T07:19:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563328103d40 con 0x563327dd7390 2024-08-31T07:19:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563328107cf0 con 0x563327dcf1b0 2024-08-31T07:19:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5e9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x56332819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327dcf1b0 0x56332819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327df8070 0x56332819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5e9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x56332819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36764/0 (socket says 172.21.15.32:36764) 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5e9000640 1 -- 172.21.15.32:0/63728592 learned_addr learned my addr 172.21.15.32:0/63728592 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5e9a00640 1 -- 172.21.15.32:0/63728592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 msgr2=0x56332819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.741+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/63728592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x56332819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.742+0000 7fe5e9a00640 1 -- 172.21.15.32:0/63728592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327df8070 msgr2=0x56332819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.742+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/63728592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327df8070 0x56332819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.742+0000 7fe5e9a00640 1 -- 172.21.15.32:0/63728592 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563327e63650 con 0x563327dcf1b0 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.742+0000 7fe5e9000640 1 --2- 172.21.15.32:0/63728592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x56332819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.742+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/63728592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327dcf1b0 0x56332819e0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8102860 tx=0x7fe5d810d270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e98f219b4195c983 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.742+0000 7fe5d6a00640 1 -- 172.21.15.32:0/63728592 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d810da70 con 0x563327dcf1b0 2024-08-31T07:19:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.742+0000 7fe5d6a00640 1 -- 172.21.15.32:0/63728592 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d8112030 con 0x563327dcf1b0 2024-08-31T07:19:08.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.742+0000 7fe5d6a00640 1 -- 172.21.15.32:0/63728592 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d81163d0 con 0x563327dcf1b0 2024-08-31T07:19:08.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.743+0000 7fe5e2000640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5cc0b4ca0 0x7fe5cc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.743+0000 7fe5e2000640 1 -- 172.21.15.32:0/211078648 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7fe5d8105150 con 0x7fe5e4105960 2024-08-31T07:19:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.743+0000 7fe5ea400640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5cc0b4ca0 0x7fe5cc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.743+0000 7fe5f30c0740 1 -- 172.21.15.32:0/63728592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327dcf1b0 msgr2=0x56332819e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.743+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/63728592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327dcf1b0 0x56332819e0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8102860 tx=0x7fe5d810d270 comp rx=0 tx=0).stop 2024-08-31T07:19:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.743+0000 7fe5f30c0740 1 -- 172.21.15.32:0/63728592 shutdown_connections 2024-08-31T07:19:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.743+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/63728592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x56332819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.743+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/63728592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327dcf1b0 0x56332819e0a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.743+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/63728592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327df8070 0x56332819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.743+0000 7fe5f30c0740 1 -- 172.21.15.32:0/63728592 >> 172.21.15.32:0/63728592 conn(0x563327dec460 msgr2=0x56332819ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.744+0000 7fe5f30c0740 1 -- 172.21.15.32:0/63728592 shutdown_connections 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.744+0000 7fe5f30c0740 1 -- 172.21.15.32:0/63728592 wait complete. 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.744+0000 7fe5ea400640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5cc0b4ca0 0x7fe5cc0b71b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc005fd0 tx=0x7fe5dc0095c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.744+0000 7fe5f30c0740 1 -- start start 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5f30c0740 1 Processor -- start 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dcf1b0 0x5633281a97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dd7390 0x5633281a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563327f66700 con 0x563327dd7390 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56332808acf0 con 0x563327dcf1b0 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5633281ac100 con 0x563327df8070 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5f30c0740 10 client.0 _ll_register_callbacks cb 0x5633280b0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dcf1b0 0x5633281a97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5e9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dd7390 0x5633281a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dcf1b0 0x5633281a97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36780/0 (socket says 172.21.15.32:36780) 2024-08-31T07:19:08.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.745+0000 7fe5e9a00640 1 -- 172.21.15.32:0/3822489062 learned_addr learned my addr 172.21.15.32:0/3822489062 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:08.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.746+0000 7fe5ea400640 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.746+0000 7fe5e9a00640 1 -- 172.21.15.32:0/3822489062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 msgr2=0x5633281b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.746+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.746+0000 7fe5e9a00640 1 -- 172.21.15.32:0/3822489062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dd7390 msgr2=0x5633281a9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.746+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dd7390 0x5633281a9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.746+0000 7fe5e9a00640 1 -- 172.21.15.32:0/3822489062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633280cb8b0 con 0x563327dcf1b0 2024-08-31T07:19:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.746+0000 7fe5ea400640 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:19:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.746+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dcf1b0 0x5633281a97a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8102830 tx=0x7fe5d8032f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e252989eaffcf39a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.746+0000 7fe5cb400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.747+0000 7fe5cb400640 1 -- 172.21.15.32:0/3822489062 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80c76c0 con 0x563327dcf1b0 2024-08-31T07:19:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.747+0000 7fe5cb400640 1 -- 172.21.15.32:0/3822489062 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7fe5d8030450 con 0x563327dcf1b0 2024-08-31T07:19:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.747+0000 7fe5f30c0740 1 -- 172.21.15.32:0/3822489062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d8112030 con 0x563327dcf1b0 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.747+0000 7fe5cb400640 1 -- 172.21.15.32:0/3822489062 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe5d80caf70 con 0x563327dcf1b0 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.747+0000 7fe5f30c0740 10 client.26867 fetch_fsmap learned FSMap version 33 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.747+0000 7fe5f30c0740 1 -- 172.21.15.32:0/3822489062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5633281b80b0 con 0x563327dcf1b0 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.747+0000 7fe5cb400640 1 -- 172.21.15.32:0/3822489062 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d8112030 con 0x563327dcf1b0 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.747+0000 7fe5cb400640 1 -- 172.21.15.32:0/3822489062 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe5d8102ae0 con 0x563327dcf1b0 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.748+0000 7fe5f30c0740 10 client.26867 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.748+0000 7fe5f30c0740 10 client.26867 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.748+0000 7fe5f30c0740 20 client.26867 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.748+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x563327db69f0 0x5633281bb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.748+0000 7fe5f30c0740 4 client.26867 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.748+0000 7fe5f30c0740 1 -- 172.21.15.32:0/3822489062 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563328116700 con 0x563327db69f0 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.748+0000 7fe5e9000640 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x563327db69f0 0x5633281bb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.749+0000 7fe5e9000640 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x563327db69f0 0x5633281bb410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73dffc6a1ceaf3be server_cookie=7c4935aae3a1cc29 in_seq=0 out_seq=0 2024-08-31T07:19:08.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.749+0000 7fe5cb400640 10 client.26867 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:19:08.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5cb400640 1 -- 172.21.15.32:0/3822489062 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563328116700 con 0x563327db69f0 2024-08-31T07:19:08.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5cb400640 10 client.26867 handle_command_reply: tid=0 2024-08-31T07:19:08.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 client.26867 shutdown 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 -- 172.21.15.32:0/3822489062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dcf1b0 msgr2=0x5633281a97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dcf1b0 0x5633281a97a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8102830 tx=0x7fe5d8032f10 comp rx=0 tx=0).stop 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 -- 172.21.15.32:0/3822489062 shutdown_connections 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x563327db69f0 0x5633281bb410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dd7390 0x5633281a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/3822489062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dcf1b0 0x5633281a97a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 -- 172.21.15.32:0/3822489062 >> 172.21.15.32:0/3822489062 conn(0x563327dec460 msgr2=0x5633281a0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5cb400640 0 client.26867 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 -- 172.21.15.32:0/3822489062 shutdown_connections 2024-08-31T07:19:08.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 1 -- 172.21.15.32:0/3822489062 wait complete. 2024-08-31T07:19:08.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.750+0000 7fe5f30c0740 20 client.26867 trim_cache size 0 max 16384 2024-08-31T07:19:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.767+0000 7fe5f30c0740 1 Processor -- start 2024-08-31T07:19:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.767+0000 7fe5f30c0740 1 -- start start 2024-08-31T07:19:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.767+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 0x5633281a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.767+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.767+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.767+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e41b14d0 con 0x563327dcf1b0 2024-08-31T07:19:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.767+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e41b3010 con 0x563327dd7390 2024-08-31T07:19:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.767+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e41b1ba0 con 0x563327df8070 2024-08-31T07:19:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5e9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 0x5633281a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281ab1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60380/0 (socket says 172.21.15.32:60380) 2024-08-31T07:19:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5ea400640 1 -- 172.21.15.32:0/2603258014 learned_addr learned my addr 172.21.15.32:0/2603258014 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5e9000640 1 -- 172.21.15.32:0/2603258014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 msgr2=0x5633281ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5e9000640 1 --2- 172.21.15.32:0/2603258014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5e9000640 1 -- 172.21.15.32:0/2603258014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 msgr2=0x5633281a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5e9000640 1 --2- 172.21.15.32:0/2603258014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 0x5633281a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5e9000640 1 -- 172.21.15.32:0/2603258014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633281bb780 con 0x563327dd7390 2024-08-31T07:19:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.768+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/2603258014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 0x5633281a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.769+0000 7fe5e9000640 1 --2- 172.21.15.32:0/2603258014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281b5130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5633281aa3e0 tx=0x7fe5d0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65dbcd86ebb72ec0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.769+0000 7fe5d6000640 1 -- 172.21.15.32:0/2603258014 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00035f0 con 0x563327dd7390 2024-08-31T07:19:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.769+0000 7fe5d6000640 1 -- 172.21.15.32:0/2603258014 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d0003c40 con 0x563327dd7390 2024-08-31T07:19:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.769+0000 7fe5d6000640 1 -- 172.21.15.32:0/2603258014 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0010b20 con 0x563327dd7390 2024-08-31T07:19:08.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.770+0000 7fe5f30c0740 1 -- 172.21.15.32:0/2603258014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 msgr2=0x5633281b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:08.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.770+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/2603258014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281b5130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5633281aa3e0 tx=0x7fe5d0005f90 comp rx=0 tx=0).stop 2024-08-31T07:19:08.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.770+0000 7fe5f30c0740 1 -- 172.21.15.32:0/2603258014 shutdown_connections 2024-08-31T07:19:08.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.770+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/2603258014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.770+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/2603258014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281b5130 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.770+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/2603258014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 0x5633281a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.770+0000 7fe5f30c0740 1 -- 172.21.15.32:0/2603258014 >> 172.21.15.32:0/2603258014 conn(0x563327dec460 msgr2=0x5633281aaac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:08.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.770+0000 7fe5f30c0740 1 -- 172.21.15.32:0/2603258014 shutdown_connections 2024-08-31T07:19:08.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.770+0000 7fe5f30c0740 1 -- 172.21.15.32:0/2603258014 wait complete. 2024-08-31T07:19:08.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.771+0000 7fe5f30c0740 1 -- start start 2024-08-31T07:19:08.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.771+0000 7fe5f30c0740 1 Processor -- start 2024-08-31T07:19:08.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 0x5633281b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5f30c0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e41b2940 con 0x563327dcf1b0 2024-08-31T07:19:08.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e41b2270 con 0x563327dd7390 2024-08-31T07:19:08.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5f30c0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e41af4d0 con 0x563327df8070 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5f30c0740 10 client.0 _ll_register_callbacks cb 0x7fe5d8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5e9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 0x5633281b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5e9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36804/0 (socket says 172.21.15.32:36804) 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5e9000640 1 -- 172.21.15.32:0/292133090 learned_addr learned my addr 172.21.15.32:0/292133090 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.772+0000 7fe5ea400640 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.773+0000 7fe5e9000640 1 -- 172.21.15.32:0/292133090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 msgr2=0x5633281ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.773+0000 7fe5e9000640 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.773+0000 7fe5e9000640 1 -- 172.21.15.32:0/292133090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 msgr2=0x5633281b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.773+0000 7fe5e9000640 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 0x5633281b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.773+0000 7fe5e9000640 1 -- 172.21.15.32:0/292133090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633281ae8b0 con 0x563327dd7390 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.773+0000 7fe5e9000640 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281add00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0012560 tx=0x7fe5d00045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f24633c1f4cb06ba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.773+0000 7fe5cb400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:08.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.773+0000 7fe5cb400640 1 -- 172.21.15.32:0/292133090 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0035060 con 0x563327dd7390 2024-08-31T07:19:08.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.773+0000 7fe5f30c0740 1 -- 172.21.15.32:0/292133090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d0003c40 con 0x563327dd7390 2024-08-31T07:19:08.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.773+0000 7fe5cb400640 1 -- 172.21.15.32:0/292133090 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7fe5d0003df0 con 0x563327dd7390 2024-08-31T07:19:08.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5cb400640 1 -- 172.21.15.32:0/292133090 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe5d002c040 con 0x563327dd7390 2024-08-31T07:19:08.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5f30c0740 10 client.26873 fetch_fsmap learned FSMap version 33 2024-08-31T07:19:08.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5f30c0740 1 -- 172.21.15.32:0/292133090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563328226a20 con 0x563327dd7390 2024-08-31T07:19:08.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5cb400640 1 -- 172.21.15.32:0/292133090 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d0003c40 con 0x563327dd7390 2024-08-31T07:19:08.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5cb400640 1 -- 172.21.15.32:0/292133090 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe5d002f030 con 0x563327dd7390 2024-08-31T07:19:08.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5f30c0740 10 client.26873 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T07:19:08.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5f30c0740 10 client.26873 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:19:08.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5f30c0740 20 client.26873 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:08.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x563327d8bda0 0x563328229160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:08.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5f30c0740 4 client.26873 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:19:08.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.774+0000 7fe5f30c0740 1 -- 172.21.15.32:0/292133090 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633281121a0 con 0x563327d8bda0 2024-08-31T07:19:08.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.775+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x563327d8bda0 0x563328229160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:08.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.775+0000 7fe5e9a00640 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x563327d8bda0 0x563328229160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fbfe089dd5db2c4a server_cookie=970ed2cf402b84f9 in_seq=0 out_seq=0 2024-08-31T07:19:08.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.775+0000 7fe5cb400640 10 client.26873 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:19:08.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.775+0000 7fe5cb400640 1 -- 172.21.15.32:0/292133090 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633281121a0 con 0x563327d8bda0 2024-08-31T07:19:08.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.775+0000 7fe5cb400640 10 client.26873 handle_command_reply: tid=0 2024-08-31T07:19:08.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.775+0000 7fe5f30c0740 1 client.26873 shutdown 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 1 -- 172.21.15.32:0/292133090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 msgr2=0x5633281add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281add00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0012560 tx=0x7fe5d00045c0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 1 -- 172.21.15.32:0/292133090 shutdown_connections 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x563327d8bda0 0x563328229160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563327df8070 0x5633281ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563327dd7390 0x5633281add00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 1 --2- 172.21.15.32:0/292133090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563327dcf1b0 0x5633281b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5cb400640 0 client.26873 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 1 -- 172.21.15.32:0/292133090 >> 172.21.15.32:0/292133090 conn(0x563327dec460 msgr2=0x56332819eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 1 -- 172.21.15.32:0/292133090 shutdown_connections 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 1 -- 172.21.15.32:0/292133090 wait complete. 2024-08-31T07:19:08.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.776+0000 7fe5f30c0740 20 client.26873 trim_cache size 0 max 16384 2024-08-31T07:19:08.781 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:19:08.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 -- 172.21.15.32:0/211078648 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5cc0b4ca0 msgr2=0x7fe5cc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:08.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5cc0b4ca0 0x7fe5cc0b71b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc005fd0 tx=0x7fe5dc0095c0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 -- 172.21.15.32:0/211078648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e4105960 msgr2=0x7fe5e419c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e4105960 0x7fe5e419c320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80303f0 tx=0x7fe5d8032da0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 -- 172.21.15.32:0/211078648 shutdown_connections 2024-08-31T07:19:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe5cc0b4ca0 0x7fe5cc0b71b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5e410bbf0 0x7fe5e41a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5e410b2a0 0x7fe5e419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 --2- 172.21.15.32:0/211078648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e4105960 0x7fe5e419c320 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 -- 172.21.15.32:0/211078648 >> 172.21.15.32:0/211078648 conn(0x7fe5e41013d0 msgr2=0x7fe5e41037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.777+0000 7fe5eae00640 1 -- 172.21.15.32:0/211078648 shutdown_connections 2024-08-31T07:19:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:08.778+0000 7fe5eae00640 1 -- 172.21.15.32:0/211078648 wait complete. 2024-08-31T07:19:09.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 187453 2.54 MB/sec execute 178 sec latency 41.101 ms 2024-08-31T07:19:09.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:19:10.335 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:10.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:10 smithi032 ceph-mon[17252]: pgmap v561: 97 pgs: 97 active+clean; 115 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2024-08-31T07:19:10.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:10 smithi073 ceph-mon[21995]: pgmap v561: 97 pgs: 97 active+clean; 115 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2024-08-31T07:19:10.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:10 smithi084 ceph-mon[22387]: pgmap v561: 97 pgs: 97 active+clean; 115 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2024-08-31T07:19:10.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 188682 2.56 MB/sec execute 179 sec latency 14.883 ms 2024-08-31T07:19:11.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.074+0000 7f950d200640 1 -- 172.21.15.32:0/2699094568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9508075550 msgr2=0x7f9508075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:11.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.074+0000 7f950d200640 1 --2- 172.21.15.32:0/2699094568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9508075550 0x7f9508075960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009900 tx=0x7f94f802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.074+0000 7f950d200640 1 -- 172.21.15.32:0/2699094568 shutdown_connections 2024-08-31T07:19:11.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.074+0000 7f950d200640 1 --2- 172.21.15.32:0/2699094568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950810fcb0 0x7f95081120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.074+0000 7f950d200640 1 --2- 172.21.15.32:0/2699094568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9508075ea0 0x7f950810f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.074+0000 7f950d200640 1 --2- 172.21.15.32:0/2699094568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9508075550 0x7f9508075960 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.074+0000 7f950d200640 1 -- 172.21.15.32:0/2699094568 >> 172.21.15.32:0/2699094568 conn(0x7f95080ff1e0 msgr2=0x7f9508101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:11.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.074+0000 7f950d200640 1 -- 172.21.15.32:0/2699094568 shutdown_connections 2024-08-31T07:19:11.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.074+0000 7f950d200640 1 -- 172.21.15.32:0/2699094568 wait complete. 2024-08-31T07:19:11.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.076+0000 7f950d200640 1 Processor -- start 2024-08-31T07:19:11.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.076+0000 7f950d200640 1 -- start start 2024-08-31T07:19:11.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f950d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9508075550 0x7f950819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f950d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9508075ea0 0x7f950819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f950d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f950810fcb0 0x7f95081a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f950d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9508115860 con 0x7f9508075550 2024-08-31T07:19:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f950d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f95081156e0 con 0x7f950810fcb0 2024-08-31T07:19:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f950d200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f95081159e0 con 0x7f9508075ea0 2024-08-31T07:19:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f9507e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f950810fcb0 0x7f95081a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f9507400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9508075550 0x7f950819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f9506a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9508075ea0 0x7f950819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f9507e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f950810fcb0 0x7f95081a1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36810/0 (socket says 172.21.15.32:36810) 2024-08-31T07:19:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.077+0000 7f9507e00640 1 -- 172.21.15.32:0/761120148 learned_addr learned my addr 172.21.15.32:0/761120148 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f9507e00640 1 -- 172.21.15.32:0/761120148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9508075ea0 msgr2=0x7f950819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f9507e00640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9508075ea0 0x7f950819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f9507e00640 1 -- 172.21.15.32:0/761120148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9508075550 msgr2=0x7f950819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f9507e00640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9508075550 0x7f950819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f9507e00640 1 -- 172.21.15.32:0/761120148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95081a21c0 con 0x7f950810fcb0 2024-08-31T07:19:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f9507e00640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f950810fcb0 0x7f95081a1a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f950000eb10 tx=0x7f950000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f94ffe00640 1 -- 172.21.15.32:0/761120148 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500004b20 con 0x7f950810fcb0 2024-08-31T07:19:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f94ffe00640 1 -- 172.21.15.32:0/761120148 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9500004cc0 con 0x7f950810fcb0 2024-08-31T07:19:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f950d200640 1 -- 172.21.15.32:0/761120148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95081a24b0 con 0x7f950810fcb0 2024-08-31T07:19:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f94ffe00640 1 -- 172.21.15.32:0/761120148 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500010720 con 0x7f950810fcb0 2024-08-31T07:19:11.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.078+0000 7f950d200640 1 -- 172.21.15.32:0/761120148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95081a8c10 con 0x7f950810fcb0 2024-08-31T07:19:11.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.082+0000 7f95153c3740 1 Processor -- start 2024-08-31T07:19:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.082+0000 7f94ffe00640 1 -- 172.21.15.32:0/761120148 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f95000108c0 con 0x7f950810fcb0 2024-08-31T07:19:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.082+0000 7f95153c3740 1 -- start start 2024-08-31T07:19:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.083+0000 7f95153c3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 0x563a4d603b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.083+0000 7f95153c3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d6040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.083+0000 7f95153c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d6045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.083+0000 7f95153c3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563a4d57a7b0 con 0x563a4d2351b0 2024-08-31T07:19:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.083+0000 7f95153c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563a4d569d40 con 0x563a4d23d390 2024-08-31T07:19:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.083+0000 7f95153c3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563a4d56dcf0 con 0x563a4d25e070 2024-08-31T07:19:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.083+0000 7f9507400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d6040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.083+0000 7f9507400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d6040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47152/0 (socket says 172.21.15.32:47152) 2024-08-31T07:19:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.083+0000 7f9507400640 1 -- 172.21.15.32:0/4073452211 learned_addr learned my addr 172.21.15.32:0/4073452211 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:11.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.084+0000 7f9506a00640 1 --2- 172.21.15.32:0/4073452211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 0x563a4d603b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.084+0000 7f9506a00640 1 --2- 172.21.15.32:0/4073452211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d6045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.084+0000 7f9507400640 1 -- 172.21.15.32:0/4073452211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 msgr2=0x563a4d603b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:11.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.084+0000 7f9507400640 1 --2- 172.21.15.32:0/4073452211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 0x563a4d603b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.084+0000 7f9507400640 1 -- 172.21.15.32:0/4073452211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 msgr2=0x563a4d6045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:11.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.084+0000 7f9507400640 1 --2- 172.21.15.32:0/4073452211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d6045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.084+0000 7f9507400640 1 -- 172.21.15.32:0/4073452211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a4d2c9650 con 0x563a4d2351b0 2024-08-31T07:19:11.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.084+0000 7f9507400640 1 --2- 172.21.15.32:0/4073452211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d6040a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009900 tx=0x7f94f8032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a0951e6ebccccc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:11.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.085+0000 7f94fcc00640 1 -- 172.21.15.32:0/4073452211 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8040070 con 0x563a4d2351b0 2024-08-31T07:19:11.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.085+0000 7f94ffe00640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94ec0b4d20 0x7f94ec0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.085+0000 7f94fcc00640 1 -- 172.21.15.32:0/4073452211 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f8004030 con 0x563a4d2351b0 2024-08-31T07:19:11.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.085+0000 7f94fcc00640 1 -- 172.21.15.32:0/4073452211 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8035040 con 0x563a4d2351b0 2024-08-31T07:19:11.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.085+0000 7f94ffe00640 1 -- 172.21.15.32:0/761120148 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7f95000e5aa0 con 0x7f950810fcb0 2024-08-31T07:19:11.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.086+0000 7f9506a00640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94ec0b4d20 0x7f94ec0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.086+0000 7f95153c3740 1 -- 172.21.15.32:0/4073452211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 msgr2=0x563a4d6040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:11.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.086+0000 7f95153c3740 1 --2- 172.21.15.32:0/4073452211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d6040a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009900 tx=0x7f94f8032b90 comp rx=0 tx=0).stop 2024-08-31T07:19:11.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.086+0000 7f95153c3740 1 -- 172.21.15.32:0/4073452211 shutdown_connections 2024-08-31T07:19:11.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.086+0000 7f95153c3740 1 --2- 172.21.15.32:0/4073452211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d6045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.086+0000 7f95153c3740 1 --2- 172.21.15.32:0/4073452211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d6040a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.086+0000 7f95153c3740 1 --2- 172.21.15.32:0/4073452211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 0x563a4d603b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.086+0000 7f95153c3740 1 -- 172.21.15.32:0/4073452211 >> 172.21.15.32:0/4073452211 conn(0x563a4d252460 msgr2=0x563a4d601a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:11.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.086+0000 7f95153c3740 1 -- 172.21.15.32:0/4073452211 shutdown_connections 2024-08-31T07:19:11.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.087+0000 7f95153c3740 1 -- 172.21.15.32:0/4073452211 wait complete. 2024-08-31T07:19:11.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.087+0000 7f9506a00640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94ec0b4d20 0x7f94ec0b7230 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009710 tx=0x7f94f4004730 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:11.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.087+0000 7f95153c3740 1 -- start start 2024-08-31T07:19:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.087+0000 7f95153c3740 1 Processor -- start 2024-08-31T07:19:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.088+0000 7f95153c3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d60f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.088+0000 7f95153c3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d23d390 0x563a4d60fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.088+0000 7f95153c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d25e070 0x563a4d61b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.088+0000 7f95153c3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563a4d3cc700 con 0x563a4d2351b0 2024-08-31T07:19:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.088+0000 7f95153c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563a4d4f0cf0 con 0x563a4d25e070 2024-08-31T07:19:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.088+0000 7f95153c3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563a4d6122e0 con 0x563a4d23d390 2024-08-31T07:19:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.088+0000 7f95153c3740 10 client.0 _ll_register_callbacks cb 0x563a4d516470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:11.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.088+0000 7f9507400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d60f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.089+0000 7f9507400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d60f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47158/0 (socket says 172.21.15.32:47158) 2024-08-31T07:19:11.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.089+0000 7f9507400640 1 -- 172.21.15.32:0/679258064 learned_addr learned my addr 172.21.15.32:0/679258064 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:11.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.089+0000 7f9507e00640 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d25e070 0x563a4d61b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.089+0000 7f9507e00640 1 -- 172.21.15.32:0/679258064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d23d390 msgr2=0x563a4d60fc00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:19:11.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.089+0000 7f9506a00640 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d23d390 0x563a4d60fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.089+0000 7f9507e00640 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d23d390 0x563a4d60fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.089+0000 7f9507e00640 1 -- 172.21.15.32:0/679258064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 msgr2=0x563a4d60f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:11.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.089+0000 7f9507e00640 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d60f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.089+0000 7f9507e00640 1 -- 172.21.15.32:0/679258064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4d5318b0 con 0x563a4d25e070 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.089+0000 7f9507400640 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d60f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.090+0000 7f9507e00640 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d25e070 0x563a4d61b630 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f95000e31b0 tx=0x7f95000abaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c86c12f6bcf4fcd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.090+0000 7f94f1600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.090+0000 7f94f1600640 1 -- 172.21.15.32:0/679258064 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95000ac850 con 0x563a4d25e070 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.090+0000 7f95153c3740 1 -- 172.21.15.32:0/679258064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f8004030 con 0x563a4d25e070 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.090+0000 7f94f1600640 1 -- 172.21.15.32:0/679258064 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7f9500005000 con 0x563a4d25e070 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.091+0000 7f94f1600640 1 -- 172.21.15.32:0/679258064 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9500005200 con 0x563a4d25e070 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.091+0000 7f95153c3740 10 client.26882 fetch_fsmap learned FSMap version 34 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.091+0000 7f95153c3740 1 -- 172.21.15.32:0/679258064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563a4d61e2f0 con 0x563a4d25e070 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.092+0000 7f94f1600640 1 -- 172.21.15.32:0/679258064 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f8004030 con 0x563a4d25e070 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.092+0000 7f94f1600640 1 -- 172.21.15.32:0/679258064 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f9500010bb0 con 0x563a4d25e070 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.092+0000 7f95153c3740 10 client.26882 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T07:19:11.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.092+0000 7f95153c3740 10 client.26882 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:19:11.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.092+0000 7f95153c3740 20 client.26882 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:11.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.092+0000 7f95153c3740 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x563a4d21c9f0 0x563a4d621860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.092+0000 7f95153c3740 4 client.26882 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:11.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.092+0000 7f95153c3740 1 -- 172.21.15.32:0/679258064 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a4d57c700 con 0x563a4d21c9f0 2024-08-31T07:19:11.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.092+0000 7f9507400640 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x563a4d21c9f0 0x563a4d621860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.093+0000 7f9507400640 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x563a4d21c9f0 0x563a4d621860 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89e3de23f68cfb64 server_cookie=cd24680e6dc8e29b in_seq=0 out_seq=0 2024-08-31T07:19:11.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.093+0000 7f94f1600640 10 client.26882 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:19:11.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.094+0000 7f94f1600640 1 -- 172.21.15.32:0/679258064 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a4d57c700 con 0x563a4d21c9f0 2024-08-31T07:19:11.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.094+0000 7f94f1600640 10 client.26882 handle_command_reply: tid=0 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.094+0000 7f95153c3740 1 client.26882 shutdown 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.094+0000 7f95153c3740 1 -- 172.21.15.32:0/679258064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d25e070 msgr2=0x563a4d61b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.094+0000 7f95153c3740 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d25e070 0x563a4d61b630 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f95000e31b0 tx=0x7f95000abaa0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.095+0000 7f95153c3740 1 -- 172.21.15.32:0/679258064 shutdown_connections 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.095+0000 7f95153c3740 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x563a4d21c9f0 0x563a4d621860 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.095+0000 7f95153c3740 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d25e070 0x563a4d61b630 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.095+0000 7f95153c3740 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d23d390 0x563a4d60fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.095+0000 7f95153c3740 1 --2- 172.21.15.32:0/679258064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d60f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.095+0000 7f95153c3740 1 -- 172.21.15.32:0/679258064 >> 172.21.15.32:0/679258064 conn(0x563a4d252460 msgr2=0x563a4d606cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.095+0000 7f94f1600640 0 client.26882 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.095+0000 7f95153c3740 1 -- 172.21.15.32:0/679258064 shutdown_connections 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.095+0000 7f95153c3740 1 -- 172.21.15.32:0/679258064 wait complete. 2024-08-31T07:19:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.095+0000 7f95153c3740 20 client.26882 trim_cache size 0 max 16384 2024-08-31T07:19:11.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.120+0000 7f95153c3740 1 Processor -- start 2024-08-31T07:19:11.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.121+0000 7f95153c3740 1 -- start start 2024-08-31T07:19:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.121+0000 7f95153c3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d607370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.121+0000 7f95153c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d601000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.121+0000 7f95153c3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 0x563a4d601540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.121+0000 7f95153c3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95081b14b0 con 0x563a4d2351b0 2024-08-31T07:19:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.121+0000 7f95153c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f95081b3020 con 0x563a4d23d390 2024-08-31T07:19:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.121+0000 7f95153c3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f95081b1b80 con 0x563a4d25e070 2024-08-31T07:19:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9506a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d601000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9507400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d607370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9507e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 0x563a4d601540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9506a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d601000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36838/0 (socket says 172.21.15.32:36838) 2024-08-31T07:19:11.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9506a00640 1 -- 172.21.15.32:0/3846323301 learned_addr learned my addr 172.21.15.32:0/3846323301 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9507e00640 1 -- 172.21.15.32:0/3846323301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 msgr2=0x563a4d601000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9507e00640 1 --2- 172.21.15.32:0/3846323301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d601000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9507e00640 1 -- 172.21.15.32:0/3846323301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 msgr2=0x563a4d607370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9507e00640 1 --2- 172.21.15.32:0/3846323301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d607370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9507e00640 1 -- 172.21.15.32:0/3846323301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a4d606170 con 0x563a4d25e070 2024-08-31T07:19:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9507400640 1 --2- 172.21.15.32:0/3846323301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d607370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.122+0000 7f9506a00640 1 --2- 172.21.15.32:0/3846323301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d601000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.123+0000 7f9507e00640 1 --2- 172.21.15.32:0/3846323301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 0x563a4d601540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f95000c3040 tx=0x7f9500005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc605f3fc49c16eb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.123+0000 7f94f3e00640 1 -- 172.21.15.32:0/3846323301 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95000b4fe0 con 0x563a4d25e070 2024-08-31T07:19:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.123+0000 7f94f3e00640 1 -- 172.21.15.32:0/3846323301 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95000ac850 con 0x563a4d25e070 2024-08-31T07:19:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.123+0000 7f94f3e00640 1 -- 172.21.15.32:0/3846323301 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500010bb0 con 0x563a4d25e070 2024-08-31T07:19:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.124+0000 7f95153c3740 1 -- 172.21.15.32:0/3846323301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 msgr2=0x563a4d601540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.124+0000 7f95153c3740 1 --2- 172.21.15.32:0/3846323301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 0x563a4d601540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f95000c3040 tx=0x7f9500005810 comp rx=0 tx=0).stop 2024-08-31T07:19:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.124+0000 7f95153c3740 1 -- 172.21.15.32:0/3846323301 shutdown_connections 2024-08-31T07:19:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.124+0000 7f95153c3740 1 --2- 172.21.15.32:0/3846323301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d25e070 0x563a4d601540 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.124+0000 7f95153c3740 1 --2- 172.21.15.32:0/3846323301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d601000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.124+0000 7f95153c3740 1 --2- 172.21.15.32:0/3846323301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d2351b0 0x563a4d607370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.124+0000 7f95153c3740 1 -- 172.21.15.32:0/3846323301 >> 172.21.15.32:0/3846323301 conn(0x563a4d252460 msgr2=0x563a4d606c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.125+0000 7f95153c3740 1 -- 172.21.15.32:0/3846323301 shutdown_connections 2024-08-31T07:19:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.125+0000 7f95153c3740 1 -- 172.21.15.32:0/3846323301 wait complete. 2024-08-31T07:19:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.125+0000 7f95153c3740 1 -- start start 2024-08-31T07:19:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.125+0000 7f95153c3740 1 Processor -- start 2024-08-31T07:19:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.126+0000 7f95153c3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d2351b0 0x563a4d617a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.126+0000 7f95153c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d613f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.126+0000 7f95153c3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d25e070 0x563a4d614450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.126+0000 7f95153c3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95081b2920 con 0x563a4d25e070 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.126+0000 7f95153c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f95081b2250 con 0x563a4d23d390 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.126+0000 7f95153c3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f95081af4b0 con 0x563a4d2351b0 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.126+0000 7f95153c3740 10 client.0 _ll_register_callbacks cb 0x7f95000abf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.127+0000 7f9506a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d613f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.127+0000 7f9506a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d613f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36840/0 (socket says 172.21.15.32:36840) 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.127+0000 7f9506a00640 1 -- 172.21.15.32:0/3355047097 learned_addr learned my addr 172.21.15.32:0/3355047097 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.127+0000 7f9507400640 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d2351b0 0x563a4d617a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.127+0000 7f9507e00640 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d25e070 0x563a4d614450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.127+0000 7f9507400640 1 -- 172.21.15.32:0/3355047097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 msgr2=0x563a4d613f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.127+0000 7f9507400640 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d613f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.127+0000 7f9507400640 1 -- 172.21.15.32:0/3355047097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d25e070 msgr2=0x563a4d614450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.127+0000 7f9507400640 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d25e070 0x563a4d614450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.127+0000 7f9507400640 1 -- 172.21.15.32:0/3355047097 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4d60a0f0 con 0x563a4d2351b0 2024-08-31T07:19:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.128+0000 7f9507400640 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d2351b0 0x563a4d617a50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x563a4d621fc0 tx=0x7f94f802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed7246221cf5dac server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.128+0000 7f94f1600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.128+0000 7f94f1600640 1 -- 172.21.15.32:0/3355047097 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8040040 con 0x563a4d2351b0 2024-08-31T07:19:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.128+0000 7f94f1600640 1 -- 172.21.15.32:0/3355047097 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8485+0+0 (secure 0 0 0) 0x7f94f80051f0 con 0x563a4d2351b0 2024-08-31T07:19:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.128+0000 7f95153c3740 1 -- 172.21.15.32:0/3355047097 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95000ac850 con 0x563a4d2351b0 2024-08-31T07:19:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.129+0000 7f94f1600640 1 -- 172.21.15.32:0/3355047097 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94f805d050 con 0x563a4d2351b0 2024-08-31T07:19:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.129+0000 7f95153c3740 10 client.26890 fetch_fsmap learned FSMap version 34 2024-08-31T07:19:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.129+0000 7f95153c3740 1 -- 172.21.15.32:0/3355047097 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563a4d695480 con 0x563a4d2351b0 2024-08-31T07:19:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.129+0000 7f94f1600640 1 -- 172.21.15.32:0/3355047097 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95000ac850 con 0x563a4d2351b0 2024-08-31T07:19:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.129+0000 7f94f1600640 1 -- 172.21.15.32:0/3355047097 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f94f8004ac0 con 0x563a4d2351b0 2024-08-31T07:19:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.129+0000 7f95153c3740 10 client.26890 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T07:19:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.129+0000 7f95153c3740 10 client.26890 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:19:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.129+0000 7f95153c3740 20 client.26890 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.130+0000 7f95153c3740 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x563a4d1f1da0 0x563a4d697890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.130+0000 7f95153c3740 4 client.26890 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:19:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.130+0000 7f95153c3740 1 -- 172.21.15.32:0/3355047097 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a4d5781a0 con 0x563a4d1f1da0 2024-08-31T07:19:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.130+0000 7f9506a00640 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x563a4d1f1da0 0x563a4d697890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f9506a00640 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x563a4d1f1da0 0x563a4d697890 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4c16c8f5a79e8a5 server_cookie=eabb7fa1cc864c18 in_seq=0 out_seq=0 2024-08-31T07:19:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f94f1600640 10 client.26890 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:19:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f94f1600640 1 -- 172.21.15.32:0/3355047097 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a4d5781a0 con 0x563a4d1f1da0 2024-08-31T07:19:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f94f1600640 10 client.26890 handle_command_reply: tid=0 2024-08-31T07:19:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f95153c3740 1 client.26890 shutdown 2024-08-31T07:19:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f95153c3740 1 -- 172.21.15.32:0/3355047097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d2351b0 msgr2=0x563a4d617a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f95153c3740 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d2351b0 0x563a4d617a50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x563a4d621fc0 tx=0x7f94f802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f95153c3740 1 -- 172.21.15.32:0/3355047097 shutdown_connections 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f95153c3740 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x563a4d1f1da0 0x563a4d697890 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f95153c3740 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563a4d25e070 0x563a4d614450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f95153c3740 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a4d23d390 0x563a4d613f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f95153c3740 1 --2- 172.21.15.32:0/3355047097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563a4d2351b0 0x563a4d617a50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f95153c3740 1 -- 172.21.15.32:0/3355047097 >> 172.21.15.32:0/3355047097 conn(0x563a4d252460 msgr2=0x563a4d604b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.131+0000 7f94f1600640 0 client.26890 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.132+0000 7f95153c3740 1 -- 172.21.15.32:0/3355047097 shutdown_connections 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.132+0000 7f95153c3740 1 -- 172.21.15.32:0/3355047097 wait complete. 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.132+0000 7f95153c3740 20 client.26890 trim_cache size 0 max 16384 2024-08-31T07:19:11.134 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:19:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 -- 172.21.15.32:0/761120148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94ec0b4d20 msgr2=0x7f94ec0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94ec0b4d20 0x7f94ec0b7230 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009710 tx=0x7f94f4004730 comp rx=0 tx=0).stop 2024-08-31T07:19:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 -- 172.21.15.32:0/761120148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f950810fcb0 msgr2=0x7f95081a1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f950810fcb0 0x7f95081a1a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f950000eb10 tx=0x7f950000c2d0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 -- 172.21.15.32:0/761120148 shutdown_connections 2024-08-31T07:19:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94ec0b4d20 0x7f94ec0b7230 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f950810fcb0 0x7f95081a1a90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9508075ea0 0x7f950819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 --2- 172.21.15.32:0/761120148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9508075550 0x7f950819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:11.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 -- 172.21.15.32:0/761120148 >> 172.21.15.32:0/761120148 conn(0x7f95080ff1e0 msgr2=0x7f9508109850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:11.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 -- 172.21.15.32:0/761120148 shutdown_connections 2024-08-31T07:19:11.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:11.133+0000 7f950d200640 1 -- 172.21.15.32:0/761120148 wait complete. 2024-08-31T07:19:11.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 190366 2.59 MB/sec execute 180 sec latency 50.430 ms 2024-08-31T07:19:12.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:19:12.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:12 smithi032 ceph-mon[17252]: pgmap v562: 97 pgs: 97 active+clean; 119 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-08-31T07:19:12.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:12 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:19:12.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:12 smithi032 ceph-mon[17252]: osdmap e82: 12 total, 12 up, 12 in 2024-08-31T07:19:12.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:12 smithi073 ceph-mon[21995]: pgmap v562: 97 pgs: 97 active+clean; 119 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-08-31T07:19:12.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:12 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:19:12.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:12 smithi073 ceph-mon[21995]: osdmap e82: 12 total, 12 up, 12 in 2024-08-31T07:19:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:12 smithi084 ceph-mon[22387]: pgmap v562: 97 pgs: 97 active+clean; 119 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-08-31T07:19:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:12 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:19:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:12 smithi084 ceph-mon[22387]: osdmap e82: 12 total, 12 up, 12 in 2024-08-31T07:19:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:12.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 191803 2.64 MB/sec execute 181 sec latency 42.155 ms 2024-08-31T07:19:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.322+0000 7f2fa3600640 1 -- 172.21.15.32:0/833656750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c10bdb0 msgr2=0x7f2f9c1125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.322+0000 7f2fa3600640 1 --2- 172.21.15.32:0/833656750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c10bdb0 0x7f2f9c1125e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009ca0 tx=0x7f2f9402a8b0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.322+0000 7f2fa3600640 1 -- 172.21.15.32:0/833656750 shutdown_connections 2024-08-31T07:19:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.322+0000 7f2fa3600640 1 --2- 172.21.15.32:0/833656750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c10bdb0 0x7f2f9c1125e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.322+0000 7f2fa3600640 1 --2- 172.21.15.32:0/833656750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c10b460 0x7f2f9c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.322+0000 7f2fa3600640 1 --2- 172.21.15.32:0/833656750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c105b20 0x7f2f9c105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.322+0000 7f2fa3600640 1 -- 172.21.15.32:0/833656750 >> 172.21.15.32:0/833656750 conn(0x7f2f9c1015b0 msgr2=0x7f2f9c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:13.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.322+0000 7f2fa3600640 1 -- 172.21.15.32:0/833656750 shutdown_connections 2024-08-31T07:19:13.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.322+0000 7f2fa3600640 1 -- 172.21.15.32:0/833656750 wait complete. 2024-08-31T07:19:13.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.323+0000 7f2fa3600640 1 Processor -- start 2024-08-31T07:19:13.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2fa3600640 1 -- start start 2024-08-31T07:19:13.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2fa3600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c105b20 0x7f2f9c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2fa3600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c10b460 0x7f2f9c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2fa3600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c10bdb0 0x7f2f9c1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2fa3600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c115ac0 con 0x7f2f9c10b460 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2fa3600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c115940 con 0x7f2f9c105b20 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2fa3600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c115c40 con 0x7f2f9c10bdb0 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2fa0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c105b20 0x7f2f9c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2f9be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c10b460 0x7f2f9c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2fa0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c105b20 0x7f2f9c19c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36860/0 (socket says 172.21.15.32:36860) 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.324+0000 7f2fa0c00640 1 -- 172.21.15.32:0/823273014 learned_addr learned my addr 172.21.15.32:0/823273014 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2fa1600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c10bdb0 0x7f2f9c1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2fa0c00640 1 -- 172.21.15.32:0/823273014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c10bdb0 msgr2=0x7f2f9c1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c10bdb0 0x7f2f9c1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2fa0c00640 1 -- 172.21.15.32:0/823273014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c10b460 msgr2=0x7f2f9c19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c10b460 0x7f2f9c19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2fa0c00640 1 -- 172.21.15.32:0/823273014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9c1a2380 con 0x7f2f9c105b20 2024-08-31T07:19:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2fa1600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c10bdb0 0x7f2f9c1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c105b20 0x7f2f9c19c410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000ea60 tx=0x7f2f9000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2f99600640 1 -- 172.21.15.32:0/823273014 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000cde0 con 0x7f2f9c105b20 2024-08-31T07:19:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2f99600640 1 -- 172.21.15.32:0/823273014 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f900035f0 con 0x7f2f9c105b20 2024-08-31T07:19:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.325+0000 7f2fa3600640 1 -- 172.21.15.32:0/823273014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9c1a2670 con 0x7f2f9c105b20 2024-08-31T07:19:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.326+0000 7f2f99600640 1 -- 172.21.15.32:0/823273014 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90010690 con 0x7f2f9c105b20 2024-08-31T07:19:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.326+0000 7f2fa3600640 1 -- 172.21.15.32:0/823273014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9c1a8dd0 con 0x7f2f9c105b20 2024-08-31T07:19:13.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.329+0000 7f2faa403740 1 Processor -- start 2024-08-31T07:19:13.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.329+0000 7f2faa403740 1 -- start start 2024-08-31T07:19:13.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.329+0000 7f2f99600640 1 -- 172.21.15.32:0/823273014 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2f900037a0 con 0x7f2f9c105b20 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2faa403740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558454535070 0x5584548da990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2faa403740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548daed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2faa403740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548db410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2faa403740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5584548517b0 con 0x558454535070 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2faa403740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558454840d40 con 0x558454514390 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2faa403740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558454844cf0 con 0x55845450c1b0 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2f9be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548db410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2f9be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548db410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36872/0 (socket says 172.21.15.32:36872) 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2fa0c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548daed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2f9be00640 1 -- 172.21.15.32:0/3390329379 learned_addr learned my addr 172.21.15.32:0/3390329379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.330+0000 7f2fa1600640 1 --2- 172.21.15.32:0/3390329379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558454535070 0x5584548da990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.331+0000 7f2f9be00640 1 -- 172.21.15.32:0/3390329379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 msgr2=0x5584548daed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.331+0000 7f2f9be00640 1 --2- 172.21.15.32:0/3390329379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548daed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.331+0000 7f2f9be00640 1 -- 172.21.15.32:0/3390329379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558454535070 msgr2=0x5584548da990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.331+0000 7f2f9be00640 1 --2- 172.21.15.32:0/3390329379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558454535070 0x5584548da990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.331+0000 7f2f9be00640 1 -- 172.21.15.32:0/3390329379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584545a0650 con 0x558454514390 2024-08-31T07:19:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.331+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/3390329379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548daed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:13.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.331+0000 7f2f9be00640 1 --2- 172.21.15.32:0/3390329379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548db410 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009c20 tx=0x7f2f88011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=140d6e192b38f472 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:13.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.331+0000 7f2f99600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f840b4d20 0x7f2f840b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.332+0000 7f2f99600640 1 -- 172.21.15.32:0/823273014 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f2f900e5830 con 0x7f2f9c105b20 2024-08-31T07:19:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.332+0000 7f2f8dc00640 1 -- 172.21.15.32:0/3390329379 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88004030 con 0x558454514390 2024-08-31T07:19:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.332+0000 7f2f8dc00640 1 -- 172.21.15.32:0/3390329379 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f88005020 con 0x558454514390 2024-08-31T07:19:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.332+0000 7f2f8dc00640 1 -- 172.21.15.32:0/3390329379 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88004ca0 con 0x558454514390 2024-08-31T07:19:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.332+0000 7f2fa1600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f840b4d20 0x7f2f840b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.333+0000 7f2faa403740 1 -- 172.21.15.32:0/3390329379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 msgr2=0x5584548db410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.333+0000 7f2fa1600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f840b4d20 0x7f2f840b7230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2f94006010 tx=0x7f2f94009480 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:13.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.333+0000 7f2faa403740 1 --2- 172.21.15.32:0/3390329379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548db410 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009c20 tx=0x7f2f88011810 comp rx=0 tx=0).stop 2024-08-31T07:19:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.333+0000 7f2faa403740 1 -- 172.21.15.32:0/3390329379 shutdown_connections 2024-08-31T07:19:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.333+0000 7f2faa403740 1 --2- 172.21.15.32:0/3390329379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548db410 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.333+0000 7f2faa403740 1 --2- 172.21.15.32:0/3390329379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548daed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.333+0000 7f2faa403740 1 --2- 172.21.15.32:0/3390329379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558454535070 0x5584548da990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.333+0000 7f2faa403740 1 -- 172.21.15.32:0/3390329379 >> 172.21.15.32:0/3390329379 conn(0x558454529460 msgr2=0x5584548d8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.333+0000 7f2faa403740 1 -- 172.21.15.32:0/3390329379 shutdown_connections 2024-08-31T07:19:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.333+0000 7f2faa403740 1 -- 172.21.15.32:0/3390329379 wait complete. 2024-08-31T07:19:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.334+0000 7f2faa403740 1 -- start start 2024-08-31T07:19:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.334+0000 7f2faa403740 1 Processor -- start 2024-08-31T07:19:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2faa403740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55845450c1b0 0x5584548e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2faa403740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2faa403740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 0x5584548f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2faa403740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5584546a3700 con 0x55845450c1b0 2024-08-31T07:19:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2faa403740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5584547c7cf0 con 0x558454514390 2024-08-31T07:19:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2faa403740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5584548e9170 con 0x558454535070 2024-08-31T07:19:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2faa403740 10 client.0 _ll_register_callbacks cb 0x5584547ed470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2fa1600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 0x5584548f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2f9be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2fa1600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 0x5584548f24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60472/0 (socket says 172.21.15.32:60472) 2024-08-31T07:19:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2fa1600640 1 -- 172.21.15.32:0/928791161 learned_addr learned my addr 172.21.15.32:0/928791161 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55845450c1b0 0x5584548e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2fa1600640 1 -- 172.21.15.32:0/928791161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 msgr2=0x5584548e6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2fa1600640 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548e6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2fa1600640 1 -- 172.21.15.32:0/928791161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55845450c1b0 msgr2=0x5584548e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2fa1600640 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55845450c1b0 0x5584548e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.335+0000 7f2fa1600640 1 -- 172.21.15.32:0/928791161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584548088b0 con 0x558454535070 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.336+0000 7f2fa1600640 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 0x5584548f24c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2f94030d30 tx=0x7f2f9402a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac9adb0ea2de7bee server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.336+0000 7f2f82000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.336+0000 7f2f82000640 1 -- 172.21.15.32:0/928791161 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94053440 con 0x558454535070 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.336+0000 7f2faa403740 1 -- 172.21.15.32:0/928791161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f88005020 con 0x558454535070 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.336+0000 7f2f82000640 1 -- 172.21.15.32:0/928791161 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f2f940535e0 con 0x558454535070 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.336+0000 7f2f82000640 1 -- 172.21.15.32:0/928791161 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2f940537e0 con 0x558454535070 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.336+0000 7f2faa403740 10 client.26902 fetch_fsmap learned FSMap version 34 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.336+0000 7f2faa403740 1 -- 172.21.15.32:0/928791161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5584548f5230 con 0x558454535070 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.337+0000 7f2f82000640 1 -- 172.21.15.32:0/928791161 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f88005020 con 0x558454535070 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.337+0000 7f2f82000640 1 -- 172.21.15.32:0/928791161 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f2f94051020 con 0x558454535070 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.337+0000 7f2faa403740 10 client.26902 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.337+0000 7f2faa403740 10 client.26902 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:19:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.337+0000 7f2faa403740 20 client.26902 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.337+0000 7f2faa403740 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5584544f39f0 0x5584548f8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.337+0000 7f2faa403740 4 client.26902 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.337+0000 7f2faa403740 1 -- 172.21.15.32:0/928791161 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558454853700 con 0x5584544f39f0 2024-08-31T07:19:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.338+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5584544f39f0 0x5584548f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.338+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5584544f39f0 0x5584548f8700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae09e078dcbb8db0 server_cookie=f9e5c05a4ddbbc21 in_seq=0 out_seq=0 2024-08-31T07:19:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.338+0000 7f2f82000640 10 client.26902 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:19:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.339+0000 7f2f82000640 1 -- 172.21.15.32:0/928791161 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558454853700 con 0x5584544f39f0 2024-08-31T07:19:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.339+0000 7f2f82000640 10 client.26902 handle_command_reply: tid=0 2024-08-31T07:19:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.339+0000 7f2faa403740 1 client.26902 shutdown 2024-08-31T07:19:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.339+0000 7f2faa403740 1 -- 172.21.15.32:0/928791161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 msgr2=0x5584548f24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.339+0000 7f2faa403740 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 0x5584548f24c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2f94030d30 tx=0x7f2f9402a8b0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.340+0000 7f2faa403740 1 -- 172.21.15.32:0/928791161 shutdown_connections 2024-08-31T07:19:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.340+0000 7f2faa403740 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 0x5584548f24c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.340+0000 7f2faa403740 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5584544f39f0 0x5584548f8700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.340+0000 7f2faa403740 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.340+0000 7f2faa403740 1 --2- 172.21.15.32:0/928791161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55845450c1b0 0x5584548e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.340+0000 7f2faa403740 1 -- 172.21.15.32:0/928791161 >> 172.21.15.32:0/928791161 conn(0x558454529460 msgr2=0x5584548ddb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.340+0000 7f2f82000640 0 client.26902 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:19:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.340+0000 7f2faa403740 1 -- 172.21.15.32:0/928791161 shutdown_connections 2024-08-31T07:19:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.340+0000 7f2faa403740 1 -- 172.21.15.32:0/928791161 wait complete. 2024-08-31T07:19:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.340+0000 7f2faa403740 20 client.26902 trim_cache size 0 max 16384 2024-08-31T07:19:13.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.355+0000 7f2faa403740 1 Processor -- start 2024-08-31T07:19:13.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.355+0000 7f2faa403740 1 -- start start 2024-08-31T07:19:13.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.355+0000 7f2faa403740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.355+0000 7f2faa403740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558454514390 0x5584548dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.355+0000 7f2faa403740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454535070 0x5584548d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.355+0000 7f2faa403740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c1ad1a0 con 0x558454514390 2024-08-31T07:19:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.355+0000 7f2faa403740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c1aed10 con 0x558454535070 2024-08-31T07:19:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.355+0000 7f2faa403740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c1ad870 con 0x55845450c1b0 2024-08-31T07:19:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa1600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454535070 0x5584548d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa0c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548dfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2f9be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558454514390 0x5584548dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa1600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454535070 0x5584548d7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36898/0 (socket says 172.21.15.32:36898) 2024-08-31T07:19:13.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa0c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548dfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60486/0 (socket says 172.21.15.32:60486) 2024-08-31T07:19:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa1600640 1 -- 172.21.15.32:0/1987865589 learned_addr learned my addr 172.21.15.32:0/1987865589 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa0c00640 1 -- 172.21.15.32:0/1987865589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454535070 msgr2=0x5584548d7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/1987865589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454535070 0x5584548d7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa0c00640 1 -- 172.21.15.32:0/1987865589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558454514390 msgr2=0x5584548dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/1987865589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558454514390 0x5584548dffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa0c00640 1 -- 172.21.15.32:0/1987865589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584548d9170 con 0x55845450c1b0 2024-08-31T07:19:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.356+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/1987865589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548dfaa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2f900e2f70 tx=0x7f2f900eb530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caa82f1624b93eb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.357+0000 7f2f8d200640 1 -- 172.21.15.32:0/1987865589 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900aba20 con 0x55845450c1b0 2024-08-31T07:19:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.357+0000 7f2f8d200640 1 -- 172.21.15.32:0/1987865589 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f900ac610 con 0x55845450c1b0 2024-08-31T07:19:13.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.357+0000 7f2f8d200640 1 -- 172.21.15.32:0/1987865589 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900a81a0 con 0x55845450c1b0 2024-08-31T07:19:13.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.357+0000 7f2faa403740 1 -- 172.21.15.32:0/1987865589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 msgr2=0x5584548dfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:13.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.358+0000 7f2faa403740 1 --2- 172.21.15.32:0/1987865589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548dfaa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2f900e2f70 tx=0x7f2f900eb530 comp rx=0 tx=0).stop 2024-08-31T07:19:13.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.358+0000 7f2faa403740 1 -- 172.21.15.32:0/1987865589 shutdown_connections 2024-08-31T07:19:13.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.358+0000 7f2faa403740 1 --2- 172.21.15.32:0/1987865589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454535070 0x5584548d7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.358+0000 7f2faa403740 1 --2- 172.21.15.32:0/1987865589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558454514390 0x5584548dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.358+0000 7f2faa403740 1 --2- 172.21.15.32:0/1987865589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55845450c1b0 0x5584548dfaa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.358+0000 7f2faa403740 1 -- 172.21.15.32:0/1987865589 >> 172.21.15.32:0/1987865589 conn(0x558454529460 msgr2=0x5584548e4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.358+0000 7f2faa403740 1 -- 172.21.15.32:0/1987865589 shutdown_connections 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.358+0000 7f2faa403740 1 -- 172.21.15.32:0/1987865589 wait complete. 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.358+0000 7f2faa403740 1 -- start start 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.358+0000 7f2faa403740 1 Processor -- start 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2faa403740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55845450c1b0 0x5584548eaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2faa403740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548eb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2faa403740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 0x55845495c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2faa403740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c1ae610 con 0x55845450c1b0 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2faa403740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c1adf40 con 0x558454514390 2024-08-31T07:19:13.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2faa403740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c1ab1c0 con 0x558454535070 2024-08-31T07:19:13.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2faa403740 10 client.0 _ll_register_callbacks cb 0x7f2f94030bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:13.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2f9be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548eb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2f9be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548eb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36900/0 (socket says 172.21.15.32:36900) 2024-08-31T07:19:13.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2fa0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55845450c1b0 0x5584548eaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2f9be00640 1 -- 172.21.15.32:0/3366239134 learned_addr learned my addr 172.21.15.32:0/3366239134 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:13.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.359+0000 7f2fa1600640 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 0x55845495c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.360+0000 7f2f9be00640 1 -- 172.21.15.32:0/3366239134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 msgr2=0x55845495c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.360+0000 7f2f9be00640 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 0x55845495c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.360+0000 7f2f9be00640 1 -- 172.21.15.32:0/3366239134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55845450c1b0 msgr2=0x5584548eaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.360+0000 7f2f9be00640 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55845450c1b0 0x5584548eaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.360+0000 7f2f9be00640 1 -- 172.21.15.32:0/3366239134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55845495c8b0 con 0x558454514390 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.360+0000 7f2f9be00640 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548eb4d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2f88011d10 tx=0x7f2f880045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f1a4dd0f36dd990 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.360+0000 7f2f82000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.360+0000 7f2f82000640 1 -- 172.21.15.32:0/3366239134 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8801e040 con 0x558454514390 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.360+0000 7f2faa403740 1 -- 172.21.15.32:0/3366239134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f900ac610 con 0x558454514390 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.360+0000 7f2f82000640 1 -- 172.21.15.32:0/3366239134 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f2f880037f0 con 0x558454514390 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.361+0000 7f2f82000640 1 -- 172.21.15.32:0/3366239134 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2f88036050 con 0x558454514390 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.361+0000 7f2faa403740 10 client.26906 fetch_fsmap learned FSMap version 34 2024-08-31T07:19:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.361+0000 7f2faa403740 1 -- 172.21.15.32:0/3366239134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558454963ca0 con 0x558454514390 2024-08-31T07:19:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.361+0000 7f2f82000640 1 -- 172.21.15.32:0/3366239134 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f900ac610 con 0x558454514390 2024-08-31T07:19:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.361+0000 7f2f82000640 1 -- 172.21.15.32:0/3366239134 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f2f88018070 con 0x558454514390 2024-08-31T07:19:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.361+0000 7f2faa403740 10 client.26906 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T07:19:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.361+0000 7f2faa403740 10 client.26906 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:19:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.362+0000 7f2faa403740 20 client.26906 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.362+0000 7f2faa403740 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5584544c8da0 0x5584549661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.362+0000 7f2faa403740 4 client.26906 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:19:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.362+0000 7f2faa403740 1 -- 172.21.15.32:0/3366239134 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55845484f1a0 con 0x5584544c8da0 2024-08-31T07:19:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.362+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5584544c8da0 0x5584549661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:13.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2f82000640 10 client.26906 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:19:13.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2fa0c00640 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5584544c8da0 0x5584549661c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb4150220ce55d18 server_cookie=98312fd9a9b6caac in_seq=0 out_seq=0 2024-08-31T07:19:13.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2f82000640 1 -- 172.21.15.32:0/3366239134 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55845484f1a0 con 0x5584544c8da0 2024-08-31T07:19:13.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2f82000640 10 client.26906 handle_command_reply: tid=0 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2faa403740 1 client.26906 shutdown 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2faa403740 1 -- 172.21.15.32:0/3366239134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 msgr2=0x5584548eb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2faa403740 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548eb4d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2f88011d10 tx=0x7f2f880045a0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2faa403740 1 -- 172.21.15.32:0/3366239134 shutdown_connections 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2faa403740 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558454535070 0x55845495c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2faa403740 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5584544c8da0 0x5584549661c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2faa403740 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558454514390 0x5584548eb4d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2faa403740 1 --2- 172.21.15.32:0/3366239134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55845450c1b0 0x5584548eaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2f82000640 0 client.26906 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.363+0000 7f2faa403740 1 -- 172.21.15.32:0/3366239134 >> 172.21.15.32:0/3366239134 conn(0x558454529460 msgr2=0x5584548e4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.364+0000 7f2faa403740 1 -- 172.21.15.32:0/3366239134 shutdown_connections 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.364+0000 7f2faa403740 1 -- 172.21.15.32:0/3366239134 wait complete. 2024-08-31T07:19:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.364+0000 7f2faa403740 20 client.26906 trim_cache size 0 max 16384 2024-08-31T07:19:13.368 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:19:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.364+0000 7f2fa3600640 1 -- 172.21.15.32:0/823273014 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f840b4d20 msgr2=0x7f2f840b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.364+0000 7f2fa3600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f840b4d20 0x7f2f840b7230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2f94006010 tx=0x7f2f94009480 comp rx=0 tx=0).stop 2024-08-31T07:19:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.364+0000 7f2fa3600640 1 -- 172.21.15.32:0/823273014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c105b20 msgr2=0x7f2f9c19c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.364+0000 7f2fa3600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c105b20 0x7f2f9c19c410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000ea60 tx=0x7f2f9000ef20 comp rx=0 tx=0).stop 2024-08-31T07:19:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.365+0000 7f2fa3600640 1 -- 172.21.15.32:0/823273014 shutdown_connections 2024-08-31T07:19:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.365+0000 7f2fa3600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2f840b4d20 0x7f2f840b7230 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.365+0000 7f2fa3600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9c10bdb0 0x7f2f9c1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.365+0000 7f2fa3600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f9c10b460 0x7f2f9c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.365+0000 7f2fa3600640 1 --2- 172.21.15.32:0/823273014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9c105b20 0x7f2f9c19c410 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.365+0000 7f2fa3600640 1 -- 172.21.15.32:0/823273014 >> 172.21.15.32:0/823273014 conn(0x7f2f9c1015b0 msgr2=0x7f2f9c1089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.365+0000 7f2fa3600640 1 -- 172.21.15.32:0/823273014 shutdown_connections 2024-08-31T07:19:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:13.365+0000 7f2fa3600640 1 -- 172.21.15.32:0/823273014 wait complete. 2024-08-31T07:19:13.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 192973 2.65 MB/sec execute 182 sec latency 17.143 ms 2024-08-31T07:19:14.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:14 smithi032 ceph-mon[17252]: pgmap v564: 97 pgs: 97 active+clean; 121 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-08-31T07:19:14.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:19:14.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:14 smithi073 ceph-mon[21995]: pgmap v564: 97 pgs: 97 active+clean; 121 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-08-31T07:19:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:14 smithi084 ceph-mon[22387]: pgmap v564: 97 pgs: 97 active+clean; 121 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-08-31T07:19:14.610 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:14.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 194721 2.68 MB/sec execute 183 sec latency 78.220 ms 2024-08-31T07:19:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.406+0000 7fe172200640 1 -- 172.21.15.32:0/1752665954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c105950 msgr2=0x7fe16c105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.406+0000 7fe172200640 1 --2- 172.21.15.32:0/1752665954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c105950 0x7fe16c105d60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009900 tx=0x7fe15c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.406+0000 7fe172200640 1 -- 172.21.15.32:0/1752665954 shutdown_connections 2024-08-31T07:19:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.406+0000 7fe172200640 1 --2- 172.21.15.32:0/1752665954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe16c10bbe0 0x7fe16c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.406+0000 7fe172200640 1 --2- 172.21.15.32:0/1752665954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c10b290 0x7fe16c10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.406+0000 7fe172200640 1 --2- 172.21.15.32:0/1752665954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c105950 0x7fe16c105d60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.406+0000 7fe172200640 1 -- 172.21.15.32:0/1752665954 >> 172.21.15.32:0/1752665954 conn(0x7fe16c101420 msgr2=0x7fe16c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.406+0000 7fe172200640 1 -- 172.21.15.32:0/1752665954 shutdown_connections 2024-08-31T07:19:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.406+0000 7fe172200640 1 -- 172.21.15.32:0/1752665954 wait complete. 2024-08-31T07:19:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.407+0000 7fe172200640 1 Processor -- start 2024-08-31T07:19:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe172200640 1 -- start start 2024-08-31T07:19:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe172200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c105950 0x7fe16c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe172200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe16c10b290 0x7fe16c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe172200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c10bbe0 0x7fe16c1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe172200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1158a0 con 0x7fe16c10bbe0 2024-08-31T07:19:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe172200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c115720 con 0x7fe16c105950 2024-08-31T07:19:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe172200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c115a20 con 0x7fe16c10b290 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe170e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c105950 0x7fe16c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe163e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe16c10b290 0x7fe16c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe170e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c105950 0x7fe16c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36918/0 (socket says 172.21.15.32:36918) 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe170e00640 1 -- 172.21.15.32:0/631275970 learned_addr learned my addr 172.21.15.32:0/631275970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.408+0000 7fe171800640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c10bbe0 0x7fe16c1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.409+0000 7fe163e00640 1 -- 172.21.15.32:0/631275970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c105950 msgr2=0x7fe16c19c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.409+0000 7fe163e00640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c105950 0x7fe16c19c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.409+0000 7fe163e00640 1 -- 172.21.15.32:0/631275970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c10bbe0 msgr2=0x7fe16c1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.409+0000 7fe163e00640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c10bbe0 0x7fe16c1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.409+0000 7fe163e00640 1 -- 172.21.15.32:0/631275970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16c1a2220 con 0x7fe16c10b290 2024-08-31T07:19:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.409+0000 7fe163e00640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe16c10b290 0x7fe16c19c850 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe16c06c660 tx=0x7fe15400db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.409+0000 7fe16a000640 1 -- 172.21.15.32:0/631275970 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154017070 con 0x7fe16c10b290 2024-08-31T07:19:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.410+0000 7fe16a000640 1 -- 172.21.15.32:0/631275970 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe154009820 con 0x7fe16c10b290 2024-08-31T07:19:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.410+0000 7fe16a000640 1 -- 172.21.15.32:0/631275970 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154004b90 con 0x7fe16c10b290 2024-08-31T07:19:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.410+0000 7fe172200640 1 -- 172.21.15.32:0/631275970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16c1a2510 con 0x7fe16c10b290 2024-08-31T07:19:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.410+0000 7fe172200640 1 -- 172.21.15.32:0/631275970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16c0775a0 con 0x7fe16c10b290 2024-08-31T07:19:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.413+0000 7fe17a4b5740 1 Processor -- start 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.413+0000 7fe17a4b5740 1 -- start start 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.413+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 0x555ead232b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.414+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace641b0 0x555ead2330a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.414+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace6c390 0x555ead2335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.414+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555ead1a97b0 con 0x555eace8d070 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.414+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555ead198d40 con 0x555eace641b0 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.414+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555ead19ccf0 con 0x555eace6c390 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.414+0000 7fe170e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace641b0 0x555ead2330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.414+0000 7fe16a000640 1 -- 172.21.15.32:0/631275970 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe154004d30 con 0x7fe16c10b290 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.414+0000 7fe170e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace641b0 0x555ead2330a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36926/0 (socket says 172.21.15.32:36926) 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.414+0000 7fe170e00640 1 -- 172.21.15.32:0/842803696 learned_addr learned my addr 172.21.15.32:0/842803696 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.414+0000 7fe171800640 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 0x555ead232b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.415+0000 7fe170e00640 1 -- 172.21.15.32:0/842803696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace6c390 msgr2=0x555ead2335e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:15.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.415+0000 7fe163e00640 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace6c390 0x555ead2335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.415+0000 7fe170e00640 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace6c390 0x555ead2335e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.415+0000 7fe170e00640 1 -- 172.21.15.32:0/842803696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 msgr2=0x555ead232b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.415+0000 7fe170e00640 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 0x555ead232b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.415+0000 7fe170e00640 1 -- 172.21.15.32:0/842803696 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eacef8650 con 0x555eace641b0 2024-08-31T07:19:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.415+0000 7fe171800640 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 0x555ead232b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.415+0000 7fe163e00640 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace6c390 0x555ead2335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.415+0000 7fe170e00640 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace641b0 0x555ead2330a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe15c000c00 tx=0x7fe15c031b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df74e159672e949 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.415+0000 7fe162000640 1 -- 172.21.15.32:0/842803696 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c040070 con 0x555eace641b0 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.416+0000 7fe162000640 1 -- 172.21.15.32:0/842803696 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe15c004030 con 0x555eace641b0 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.416+0000 7fe162000640 1 -- 172.21.15.32:0/842803696 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c035040 con 0x555eace641b0 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.416+0000 7fe16a000640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1500b4d20 0x7fe1500b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.416+0000 7fe17a4b5740 1 -- 172.21.15.32:0/842803696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace641b0 msgr2=0x555ead2330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.416+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace641b0 0x555ead2330a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe15c000c00 tx=0x7fe15c031b90 comp rx=0 tx=0).stop 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.417+0000 7fe17a4b5740 1 -- 172.21.15.32:0/842803696 shutdown_connections 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.417+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace6c390 0x555ead2335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.417+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace641b0 0x555ead2330a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.417+0000 7fe16a000640 1 -- 172.21.15.32:0/631275970 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fe1540e4d30 con 0x7fe16c10b290 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.417+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/842803696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 0x555ead232b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.417+0000 7fe17a4b5740 1 -- 172.21.15.32:0/842803696 >> 172.21.15.32:0/842803696 conn(0x555eace81460 msgr2=0x555ead230a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.417+0000 7fe171800640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1500b4d20 0x7fe1500b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.417+0000 7fe17a4b5740 1 -- 172.21.15.32:0/842803696 shutdown_connections 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.417+0000 7fe17a4b5740 1 -- 172.21.15.32:0/842803696 wait complete. 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.418+0000 7fe171800640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1500b4d20 0x7fe1500b7230 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe164005fd0 tx=0x7fe164016040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.418+0000 7fe17a4b5740 1 -- start start 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.418+0000 7fe17a4b5740 1 Processor -- start 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.418+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace641b0 0x555ead23e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.418+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead23ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.418+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace8d070 0x555ead24a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.418+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555eacffb700 con 0x555eace641b0 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.418+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555ead11fcf0 con 0x555eace6c390 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.418+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555ead2412e0 con 0x555eace8d070 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.418+0000 7fe17a4b5740 10 client.0 _ll_register_callbacks cb 0x555ead145470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe163e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead23ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe170e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace641b0 0x555ead23e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe163e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead23ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:36932/0 (socket says 172.21.15.32:36932) 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe163e00640 1 -- 172.21.15.32:0/4287298358 learned_addr learned my addr 172.21.15.32:0/4287298358 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe171800640 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace8d070 0x555ead24a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe163e00640 1 -- 172.21.15.32:0/4287298358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace8d070 msgr2=0x555ead24a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe163e00640 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace8d070 0x555ead24a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe163e00640 1 -- 172.21.15.32:0/4287298358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace641b0 msgr2=0x555ead23e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe163e00640 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace641b0 0x555ead23e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe163e00640 1 -- 172.21.15.32:0/4287298358 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ead1608b0 con 0x555eace6c390 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe163e00640 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead23ece0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe1540a7540 tx=0x7fe1540ab360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d7dfe6c1f9e87c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.419+0000 7fe15aa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.420+0000 7fe15aa00640 1 -- 172.21.15.32:0/4287298358 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154020070 con 0x555eace6c390 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.420+0000 7fe17a4b5740 1 -- 172.21.15.32:0/4287298358 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15c004030 con 0x555eace6c390 2024-08-31T07:19:15.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.420+0000 7fe15aa00640 1 -- 172.21.15.32:0/4287298358 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fe1540e26f0 con 0x555eace6c390 2024-08-31T07:19:15.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.420+0000 7fe15aa00640 1 -- 172.21.15.32:0/4287298358 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe1540e28f0 con 0x555eace6c390 2024-08-31T07:19:15.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.420+0000 7fe17a4b5740 10 client.26918 fetch_fsmap learned FSMap version 34 2024-08-31T07:19:15.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.420+0000 7fe17a4b5740 1 -- 172.21.15.32:0/4287298358 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555ead24d260 con 0x555eace6c390 2024-08-31T07:19:15.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.420+0000 7fe15aa00640 1 -- 172.21.15.32:0/4287298358 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe15c004030 con 0x555eace6c390 2024-08-31T07:19:15.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.421+0000 7fe15aa00640 1 -- 172.21.15.32:0/4287298358 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fe1540b4330 con 0x555eace6c390 2024-08-31T07:19:15.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.421+0000 7fe17a4b5740 10 client.26918 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T07:19:15.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.421+0000 7fe17a4b5740 10 client.26918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:19:15.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.421+0000 7fe17a4b5740 20 client.26918 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:15.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.421+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555eace4b9f0 0x555ead250620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.421+0000 7fe17a4b5740 4 client.26918 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.421+0000 7fe17a4b5740 1 -- 172.21.15.32:0/4287298358 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ead1ab700 con 0x555eace4b9f0 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.421+0000 7fe170e00640 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555eace4b9f0 0x555ead250620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.422+0000 7fe15aa00640 10 client.26918 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.422+0000 7fe170e00640 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555eace4b9f0 0x555ead250620 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421179ce8a632b8f server_cookie=aa8fbe25dbdd2521 in_seq=0 out_seq=0 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.422+0000 7fe15aa00640 1 -- 172.21.15.32:0/4287298358 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555ead1ab700 con 0x555eace4b9f0 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.422+0000 7fe15aa00640 10 client.26918 handle_command_reply: tid=0 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.422+0000 7fe17a4b5740 1 client.26918 shutdown 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 1 -- 172.21.15.32:0/4287298358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 msgr2=0x555ead23ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead23ece0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe1540a7540 tx=0x7fe1540ab360 comp rx=0 tx=0).stop 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 1 -- 172.21.15.32:0/4287298358 shutdown_connections 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555eace4b9f0 0x555ead250620 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace8d070 0x555ead24a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead23ece0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/4287298358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace641b0 0x555ead23e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 1 -- 172.21.15.32:0/4287298358 >> 172.21.15.32:0/4287298358 conn(0x555eace81460 msgr2=0x555ead235d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:15.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe15aa00640 0 client.26918 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:15.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 1 -- 172.21.15.32:0/4287298358 shutdown_connections 2024-08-31T07:19:15.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 1 -- 172.21.15.32:0/4287298358 wait complete. 2024-08-31T07:19:15.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.423+0000 7fe17a4b5740 20 client.26918 trim_cache size 0 max 16384 2024-08-31T07:19:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.439+0000 7fe17a4b5740 1 Processor -- start 2024-08-31T07:19:15.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.439+0000 7fe17a4b5740 1 -- start start 2024-08-31T07:19:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead238380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace6c390 0x555ead24a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace8d070 0x555ead23b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1ad0a0 con 0x555eace6c390 2024-08-31T07:19:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1aec10 con 0x555eace8d070 2024-08-31T07:19:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1ad770 con 0x555eace641b0 2024-08-31T07:19:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe170e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead238380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe170e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead238380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60548/0 (socket says 172.21.15.32:60548) 2024-08-31T07:19:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe170e00640 1 -- 172.21.15.32:0/1833069833 learned_addr learned my addr 172.21.15.32:0/1833069833 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe163e00640 1 --2- 172.21.15.32:0/1833069833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace6c390 0x555ead24a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe171800640 1 --2- 172.21.15.32:0/1833069833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace8d070 0x555ead23b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe170e00640 1 -- 172.21.15.32:0/1833069833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace8d070 msgr2=0x555ead23b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe170e00640 1 --2- 172.21.15.32:0/1833069833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace8d070 0x555ead23b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe170e00640 1 -- 172.21.15.32:0/1833069833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace6c390 msgr2=0x555ead24a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe170e00640 1 --2- 172.21.15.32:0/1833069833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace6c390 0x555ead24a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe170e00640 1 -- 172.21.15.32:0/1833069833 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ead23f860 con 0x555eace641b0 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.440+0000 7fe163e00640 1 --2- 172.21.15.32:0/1833069833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace6c390 0x555ead24a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.441+0000 7fe170e00640 1 --2- 172.21.15.32:0/1833069833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead238380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe15c005130 tx=0x7fe15c005160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6bd14431408ed00 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.441+0000 7fe161600640 1 -- 172.21.15.32:0/1833069833 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c040030 con 0x555eace641b0 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.441+0000 7fe161600640 1 -- 172.21.15.32:0/1833069833 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe15c031950 con 0x555eace641b0 2024-08-31T07:19:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.441+0000 7fe161600640 1 -- 172.21.15.32:0/1833069833 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c035040 con 0x555eace641b0 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.442+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1833069833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 msgr2=0x555ead238380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.442+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/1833069833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead238380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe15c005130 tx=0x7fe15c005160 comp rx=0 tx=0).stop 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.442+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1833069833 shutdown_connections 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.442+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/1833069833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace8d070 0x555ead23b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.442+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/1833069833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace6c390 0x555ead24a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.442+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/1833069833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead238380 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.442+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1833069833 >> 172.21.15.32:0/1833069833 conn(0x555eace81460 msgr2=0x555ead237c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.442+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1833069833 shutdown_connections 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.442+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1833069833 wait complete. 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.443+0000 7fe17a4b5740 1 -- start start 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.443+0000 7fe17a4b5740 1 Processor -- start 2024-08-31T07:19:15.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.443+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead242ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.443+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead2434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.443+0000 7fe17a4b5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 0x555ead2b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.443+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1ae510 con 0x555eace8d070 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.443+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1ade40 con 0x555eace6c390 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.443+0000 7fe17a4b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1ab0a0 con 0x555eace641b0 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe17a4b5740 10 client.0 _ll_register_callbacks cb 0x7fe1540a7980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe170e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead242ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe163e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead2434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe170e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead242ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60564/0 (socket says 172.21.15.32:60564) 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe170e00640 1 -- 172.21.15.32:0/1032345861 learned_addr learned my addr 172.21.15.32:0/1032345861 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe171800640 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 0x555ead2b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe163e00640 1 -- 172.21.15.32:0/1032345861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 msgr2=0x555ead242ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe163e00640 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead242ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe163e00640 1 -- 172.21.15.32:0/1032345861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 msgr2=0x555ead2b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe163e00640 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 0x555ead2b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.444+0000 7fe163e00640 1 -- 172.21.15.32:0/1032345861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ead2b4700 con 0x555eace6c390 2024-08-31T07:19:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.445+0000 7fe163e00640 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead2434d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe154004770 tx=0x7fe1540e2900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9a94c024cd91273 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.445+0000 7fe15aa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.445+0000 7fe15aa00640 1 -- 172.21.15.32:0/1032345861 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540d2060 con 0x555eace6c390 2024-08-31T07:19:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.445+0000 7fe15aa00640 1 -- 172.21.15.32:0/1032345861 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fe1540ccbd0 con 0x555eace6c390 2024-08-31T07:19:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.445+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1032345861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ead1a71a0 con 0x555eace6c390 2024-08-31T07:19:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.445+0000 7fe15aa00640 1 -- 172.21.15.32:0/1032345861 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe1540aac30 con 0x555eace6c390 2024-08-31T07:19:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.445+0000 7fe17a4b5740 10 client.26924 fetch_fsmap learned FSMap version 34 2024-08-31T07:19:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.445+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1032345861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555ead2bba40 con 0x555eace6c390 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.445+0000 7fe15aa00640 1 -- 172.21.15.32:0/1032345861 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555ead1a71a0 con 0x555eace6c390 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.446+0000 7fe15aa00640 1 -- 172.21.15.32:0/1032345861 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fe15405b080 con 0x555eace6c390 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.446+0000 7fe17a4b5740 10 client.26924 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.446+0000 7fe17a4b5740 10 client.26924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.446+0000 7fe17a4b5740 20 client.26924 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.446+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555eace20da0 0x555ead2be140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.446+0000 7fe17a4b5740 4 client.26924 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.446+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1032345861 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ead1a48e0 con 0x555eace20da0 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.446+0000 7fe170e00640 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555eace20da0 0x555ead2be140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.446+0000 7fe170e00640 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555eace20da0 0x555ead2be140 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca973191ee2c7ee server_cookie=5882f80c1acb38db in_seq=0 out_seq=0 2024-08-31T07:19:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.446+0000 7fe15aa00640 10 client.26924 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe15aa00640 1 -- 172.21.15.32:0/1032345861 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ead1a48e0 con 0x555eace20da0 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe15aa00640 10 client.26924 handle_command_reply: tid=0 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 client.26924 shutdown 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1032345861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 msgr2=0x555ead2434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead2434d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe154004770 tx=0x7fe1540e2900 comp rx=0 tx=0).stop 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1032345861 shutdown_connections 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555eace20da0 0x555ead2be140 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555eace8d070 0x555ead2b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555eace6c390 0x555ead2434d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe15aa00640 0 client.26924 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 --2- 172.21.15.32:0/1032345861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555eace641b0 0x555ead242ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1032345861 >> 172.21.15.32:0/1032345861 conn(0x555eace81460 msgr2=0x555ead23fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1032345861 shutdown_connections 2024-08-31T07:19:15.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.447+0000 7fe17a4b5740 1 -- 172.21.15.32:0/1032345861 wait complete. 2024-08-31T07:19:15.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.448+0000 7fe17a4b5740 20 client.26924 trim_cache size 0 max 16384 2024-08-31T07:19:15.452 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:19:15.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.448+0000 7fe172200640 1 -- 172.21.15.32:0/631275970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1500b4d20 msgr2=0x7fe1500b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.448+0000 7fe172200640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1500b4d20 0x7fe1500b7230 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe164005fd0 tx=0x7fe164016040 comp rx=0 tx=0).stop 2024-08-31T07:19:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.448+0000 7fe172200640 1 -- 172.21.15.32:0/631275970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe16c10b290 msgr2=0x7fe16c19c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.448+0000 7fe172200640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe16c10b290 0x7fe16c19c850 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe16c06c660 tx=0x7fe15400db30 comp rx=0 tx=0).stop 2024-08-31T07:19:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.448+0000 7fe172200640 1 -- 172.21.15.32:0/631275970 shutdown_connections 2024-08-31T07:19:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.448+0000 7fe172200640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1500b4d20 0x7fe1500b7230 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.448+0000 7fe172200640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c10bbe0 0x7fe16c1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.448+0000 7fe172200640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe16c10b290 0x7fe16c19c850 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.448+0000 7fe172200640 1 --2- 172.21.15.32:0/631275970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c105950 0x7fe16c19c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.449+0000 7fe172200640 1 -- 172.21.15.32:0/631275970 >> 172.21.15.32:0/631275970 conn(0x7fe16c101420 msgr2=0x7fe16c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.449+0000 7fe172200640 1 -- 172.21.15.32:0/631275970 shutdown_connections 2024-08-31T07:19:15.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:15.449+0000 7fe172200640 1 -- 172.21.15.32:0/631275970 wait complete. 2024-08-31T07:19:15.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 195887 2.70 MB/sec execute 184 sec latency 12.551 ms 2024-08-31T07:19:16.267 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:16 smithi032 ceph-mon[17252]: pgmap v565: 97 pgs: 97 active+clean; 122 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 243 op/s 2024-08-31T07:19:16.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:16 smithi073 ceph-mon[21995]: pgmap v565: 97 pgs: 97 active+clean; 122 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 243 op/s 2024-08-31T07:19:16.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:16 smithi084 ceph-mon[22387]: pgmap v565: 97 pgs: 97 active+clean; 122 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 243 op/s 2024-08-31T07:19:16.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 197496 2.72 MB/sec execute 185 sec latency 25.715 ms 2024-08-31T07:19:17.309 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:19:17.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:19:17.320 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:17 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:19:17.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:17 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:19:17.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:17 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:19:17.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 198894 2.77 MB/sec execute 186 sec latency 152.105 ms 2024-08-31T07:19:17.742 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:18.181 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:18 smithi032 ceph-mon[17252]: pgmap v566: 97 pgs: 97 active+clean; 126 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-08-31T07:19:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:18 smithi073 ceph-mon[21995]: pgmap v566: 97 pgs: 97 active+clean; 126 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-08-31T07:19:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:18 smithi084 ceph-mon[22387]: pgmap v566: 97 pgs: 97 active+clean; 126 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-08-31T07:19:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.541+0000 7f2b6be00640 1 -- 172.21.15.32:0/1863069213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c105950 msgr2=0x7f2b6c105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.541+0000 7f2b6be00640 1 --2- 172.21.15.32:0/1863069213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c105950 0x7f2b6c105d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009900 tx=0x7f2b5c02a410 comp rx=0 tx=0).stop 2024-08-31T07:19:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.541+0000 7f2b6be00640 1 -- 172.21.15.32:0/1863069213 shutdown_connections 2024-08-31T07:19:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.541+0000 7f2b6be00640 1 --2- 172.21.15.32:0/1863069213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c10bbe0 0x7f2b6c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.541+0000 7f2b6be00640 1 --2- 172.21.15.32:0/1863069213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10b290 0x7f2b6c10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.541+0000 7f2b6be00640 1 --2- 172.21.15.32:0/1863069213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c105950 0x7f2b6c105d60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.541+0000 7f2b6be00640 1 -- 172.21.15.32:0/1863069213 >> 172.21.15.32:0/1863069213 conn(0x7f2b6c101420 msgr2=0x7f2b6c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:18.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.541+0000 7f2b6be00640 1 -- 172.21.15.32:0/1863069213 shutdown_connections 2024-08-31T07:19:18.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.542+0000 7f2b6be00640 1 -- 172.21.15.32:0/1863069213 wait complete. 2024-08-31T07:19:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.543+0000 7f2b6be00640 1 Processor -- start 2024-08-31T07:19:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.543+0000 7f2b6be00640 1 -- start start 2024-08-31T07:19:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.543+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c105950 0x7f2b6c19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c10b290 0x7f2b6c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10bbe0 0x7f2b6c1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1158f0 con 0x7f2b6c10b290 2024-08-31T07:19:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c115770 con 0x7f2b6c105950 2024-08-31T07:19:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c115a70 con 0x7f2b6c10bbe0 2024-08-31T07:19:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c105950 0x7f2b6c19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c105950 0x7f2b6c19c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34846/0 (socket says 172.21.15.32:34846) 2024-08-31T07:19:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6aa00640 1 -- 172.21.15.32:0/916717945 learned_addr learned my addr 172.21.15.32:0/916717945 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6b400640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10bbe0 0x7f2b6c1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6a000640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c10b290 0x7f2b6c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6aa00640 1 -- 172.21.15.32:0/916717945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10bbe0 msgr2=0x7f2b6c1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10bbe0 0x7f2b6c1a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6aa00640 1 -- 172.21.15.32:0/916717945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c10b290 msgr2=0x7f2b6c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c10b290 0x7f2b6c19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.544+0000 7f2b6aa00640 1 -- 172.21.15.32:0/916717945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c1a2210 con 0x7f2b6c105950 2024-08-31T07:19:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.545+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c105950 0x7f2b6c19c2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c005ab0 tx=0x7f2b5c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.545+0000 7f2b63400640 1 -- 172.21.15.32:0/916717945 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c03f070 con 0x7f2b6c105950 2024-08-31T07:19:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.545+0000 7f2b6be00640 1 -- 172.21.15.32:0/916717945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c1a24a0 con 0x7f2b6c105950 2024-08-31T07:19:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.545+0000 7f2b63400640 1 -- 172.21.15.32:0/916717945 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c031b60 con 0x7f2b6c105950 2024-08-31T07:19:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.545+0000 7f2b63400640 1 -- 172.21.15.32:0/916717945 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c035030 con 0x7f2b6c105950 2024-08-31T07:19:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.545+0000 7f2b6be00640 1 -- 172.21.15.32:0/916717945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c1a8c60 con 0x7f2b6c105950 2024-08-31T07:19:18.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.548+0000 7f2b7844d740 1 Processor -- start 2024-08-31T07:19:18.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b7844d740 1 -- start start 2024-08-31T07:19:18.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcc37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 0x55aebfcc3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcc4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b7844d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55aebfc39750 con 0x55aebf91d350 2024-08-31T07:19:18.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b7844d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55aebfc28ce0 con 0x55aebf8fc670 2024-08-31T07:19:18.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b7844d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55aebfc2cc90 con 0x55aebf8f4490 2024-08-31T07:19:18.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b6a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcc4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b6b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcc37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b6a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcc4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34856/0 (socket says 172.21.15.32:34856) 2024-08-31T07:19:18.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.549+0000 7f2b6a000640 1 -- 172.21.15.32:0/282313259 learned_addr learned my addr 172.21.15.32:0/282313259 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:18.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.550+0000 7f2b63400640 1 -- 172.21.15.32:0/916717945 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2b5c042070 con 0x7f2b6c105950 2024-08-31T07:19:18.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.550+0000 7f2b6b400640 1 -- 172.21.15.32:0/282313259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 msgr2=0x55aebfcc3d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:19:18.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.550+0000 7f2b6b400640 1 --2- 172.21.15.32:0/282313259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 0x55aebfcc3d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.550+0000 7f2b6b400640 1 -- 172.21.15.32:0/282313259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 msgr2=0x55aebfcc4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.550+0000 7f2b6b400640 1 --2- 172.21.15.32:0/282313259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcc4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.550+0000 7f2b6b400640 1 -- 172.21.15.32:0/282313259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aebf988930 con 0x55aebf91d350 2024-08-31T07:19:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.550+0000 7f2b6b400640 1 --2- 172.21.15.32:0/282313259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcc37d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009f00 tx=0x7f2b64011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366872d5acf56ef8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.550+0000 7f2b57e00640 1 -- 172.21.15.32:0/282313259 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64004d20 con 0x55aebf91d350 2024-08-31T07:19:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.550+0000 7f2b57e00640 1 -- 172.21.15.32:0/282313259 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b64004030 con 0x55aebf91d350 2024-08-31T07:19:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.550+0000 7f2b57e00640 1 -- 172.21.15.32:0/282313259 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64005280 con 0x55aebf91d350 2024-08-31T07:19:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.551+0000 7f2b7844d740 1 -- 172.21.15.32:0/282313259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 msgr2=0x55aebfcc37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.551+0000 7f2b7844d740 1 --2- 172.21.15.32:0/282313259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcc37d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009f00 tx=0x7f2b64011810 comp rx=0 tx=0).stop 2024-08-31T07:19:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.552+0000 7f2b7844d740 1 -- 172.21.15.32:0/282313259 shutdown_connections 2024-08-31T07:19:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.552+0000 7f2b7844d740 1 --2- 172.21.15.32:0/282313259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcc4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.552+0000 7f2b7844d740 1 --2- 172.21.15.32:0/282313259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 0x55aebfcc3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.552+0000 7f2b7844d740 1 --2- 172.21.15.32:0/282313259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcc37d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.552+0000 7f2b7844d740 1 -- 172.21.15.32:0/282313259 >> 172.21.15.32:0/282313259 conn(0x55aebf911740 msgr2=0x55aebfcc16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.552+0000 7f2b7844d740 1 -- 172.21.15.32:0/282313259 shutdown_connections 2024-08-31T07:19:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.552+0000 7f2b63400640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b500b4c50 0x7f2b500b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.552+0000 7f2b7844d740 1 -- 172.21.15.32:0/282313259 wait complete. 2024-08-31T07:19:18.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.552+0000 7f2b63400640 1 -- 172.21.15.32:0/916717945 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f2b5c105c30 con 0x7f2b6c105950 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.552+0000 7f2b6a000640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b500b4c50 0x7f2b500b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.553+0000 7f2b7844d740 1 -- start start 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.553+0000 7f2b7844d740 1 Processor -- start 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 0x55aebfcd82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcd8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6a000640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b500b4c50 0x7f2b500b7160 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009cc0 tx=0x7f2b58005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcdd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b7844d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55aebfa8b890 con 0x55aebf91d350 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b7844d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55aebfbafcd0 con 0x55aebf8fc670 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b7844d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55aebfcd1fb0 con 0x55aebf8f4490 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b7844d740 10 client.0 _ll_register_callbacks cb 0x55aebfc23ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 0x55aebfcd82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 0x55aebfcd82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59264/0 (socket says 172.21.15.32:59264) 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6aa00640 1 -- 172.21.15.32:0/437561987 learned_addr learned my addr 172.21.15.32:0/437561987 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6a000640 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcd8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6b400640 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcdd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6aa00640 1 -- 172.21.15.32:0/437561987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 msgr2=0x55aebfcd8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcd8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6aa00640 1 -- 172.21.15.32:0/437561987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 msgr2=0x55aebfcdd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcdd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.554+0000 7f2b6aa00640 1 -- 172.21.15.32:0/437561987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aebfbf91b0 con 0x55aebf8f4490 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.555+0000 7f2b6b400640 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcdd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.555+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 0x55aebfcd82a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c031440 tx=0x7f2b5c0cbbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b64fec47190569b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.555+0000 7f2b54c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.555+0000 7f2b54c00640 1 -- 172.21.15.32:0/437561987 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c049070 con 0x55aebf8f4490 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.555+0000 7f2b54c00640 1 -- 172.21.15.32:0/437561987 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f2b5c1035f0 con 0x55aebf8f4490 2024-08-31T07:19:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.555+0000 7f2b7844d740 1 -- 172.21.15.32:0/437561987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b64004030 con 0x55aebf8f4490 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.555+0000 7f2b54c00640 1 -- 172.21.15.32:0/437561987 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c0d0b80 con 0x55aebf8f4490 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.555+0000 7f2b7844d740 10 client.26932 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.555+0000 7f2b7844d740 1 -- 172.21.15.32:0/437561987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aebfce67f0 con 0x55aebf8f4490 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.556+0000 7f2b54c00640 1 -- 172.21.15.32:0/437561987 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b64004030 con 0x55aebf8f4490 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.556+0000 7f2b54c00640 1 -- 172.21.15.32:0/437561987 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f2b5c0cc620 con 0x55aebf8f4490 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.556+0000 7f2b7844d740 10 client.26932 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.556+0000 7f2b7844d740 10 client.26932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.556+0000 7f2b7844d740 20 client.26932 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.556+0000 7f2b7844d740 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aebf8dbcd0 0x55aebfce9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.556+0000 7f2b7844d740 4 client.26932 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.556+0000 7f2b7844d740 1 -- 172.21.15.32:0/437561987 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aebfc3b6a0 con 0x55aebf8dbcd0 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.556+0000 7f2b6a000640 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aebf8dbcd0 0x55aebfce9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.557+0000 7f2b54c00640 10 client.26932 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.557+0000 7f2b6a000640 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aebf8dbcd0 0x55aebfce9c60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7466f0b8531c502 server_cookie=1a90e8acd4f425e9 in_seq=0 out_seq=0 2024-08-31T07:19:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b54c00640 1 -- 172.21.15.32:0/437561987 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aebfc3b6a0 con 0x55aebf8dbcd0 2024-08-31T07:19:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b54c00640 10 client.26932 handle_command_reply: tid=0 2024-08-31T07:19:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b7844d740 1 client.26932 shutdown 2024-08-31T07:19:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b7844d740 1 -- 172.21.15.32:0/437561987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 msgr2=0x55aebfcd82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b7844d740 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 0x55aebfcd82a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c031440 tx=0x7f2b5c0cbbf0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b7844d740 1 -- 172.21.15.32:0/437561987 shutdown_connections 2024-08-31T07:19:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b7844d740 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aebf8dbcd0 0x55aebfce9c60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b7844d740 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcdd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b7844d740 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcd8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b54c00640 0 client.26932 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b7844d740 1 --2- 172.21.15.32:0/437561987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8f4490 0x55aebfcd82a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.558+0000 7f2b7844d740 1 -- 172.21.15.32:0/437561987 >> 172.21.15.32:0/437561987 conn(0x55aebf911740 msgr2=0x55aebfcc69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.559+0000 7f2b7844d740 1 -- 172.21.15.32:0/437561987 shutdown_connections 2024-08-31T07:19:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.559+0000 7f2b7844d740 1 -- 172.21.15.32:0/437561987 wait complete. 2024-08-31T07:19:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.559+0000 7f2b7844d740 20 client.26932 trim_cache size 0 max 16384 2024-08-31T07:19:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.579+0000 7f2b7844d740 1 Processor -- start 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.579+0000 7f2b7844d740 1 -- start start 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.579+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8f4490 0x55aebfcd9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.579+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8fc670 0x55aebfcd5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.579+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcd57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.579+0000 7f2b7844d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1acfa0 con 0x55aebf91d350 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.579+0000 7f2b7844d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1aeba0 con 0x55aebf8f4490 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.579+0000 7f2b7844d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1ad670 con 0x55aebf8fc670 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.580+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8f4490 0x55aebfcd9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.580+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8f4490 0x55aebfcd9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34872/0 (socket says 172.21.15.32:34872) 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.580+0000 7f2b6aa00640 1 -- 172.21.15.32:0/3508563101 learned_addr learned my addr 172.21.15.32:0/3508563101 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.580+0000 7f2b6b400640 1 --2- 172.21.15.32:0/3508563101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcd57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.580+0000 7f2b6a000640 1 --2- 172.21.15.32:0/3508563101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8fc670 0x55aebfcd5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.580+0000 7f2b6aa00640 1 -- 172.21.15.32:0/3508563101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8fc670 msgr2=0x55aebfcd5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.580+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/3508563101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8fc670 0x55aebfcd5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.580+0000 7f2b6aa00640 1 -- 172.21.15.32:0/3508563101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 msgr2=0x55aebfcd57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.580+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/3508563101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcd57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.580+0000 7f2b6aa00640 1 -- 172.21.15.32:0/3508563101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aebfcc78e0 con 0x55aebf8f4490 2024-08-31T07:19:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.581+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/3508563101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8f4490 0x55aebfcd9cd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0c8440 tx=0x7f2b5c0cc540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f760161bad35751 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.581+0000 7f2b57400640 1 -- 172.21.15.32:0/3508563101 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c049070 con 0x55aebf8f4490 2024-08-31T07:19:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.581+0000 7f2b57400640 1 -- 172.21.15.32:0/3508563101 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c0d5630 con 0x55aebf8f4490 2024-08-31T07:19:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.581+0000 7f2b57400640 1 -- 172.21.15.32:0/3508563101 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c03b460 con 0x55aebf8f4490 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.582+0000 7f2b7844d740 1 -- 172.21.15.32:0/3508563101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8f4490 msgr2=0x55aebfcd9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.582+0000 7f2b7844d740 1 --2- 172.21.15.32:0/3508563101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8f4490 0x55aebfcd9cd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0c8440 tx=0x7f2b5c0cc540 comp rx=0 tx=0).stop 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.582+0000 7f2b7844d740 1 -- 172.21.15.32:0/3508563101 shutdown_connections 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.582+0000 7f2b7844d740 1 --2- 172.21.15.32:0/3508563101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf91d350 0x55aebfcd57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.582+0000 7f2b7844d740 1 --2- 172.21.15.32:0/3508563101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf8fc670 0x55aebfcd5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.582+0000 7f2b7844d740 1 --2- 172.21.15.32:0/3508563101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8f4490 0x55aebfcd9cd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.582+0000 7f2b7844d740 1 -- 172.21.15.32:0/3508563101 >> 172.21.15.32:0/3508563101 conn(0x55aebf911740 msgr2=0x55aebfcd95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.582+0000 7f2b7844d740 1 -- 172.21.15.32:0/3508563101 shutdown_connections 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.582+0000 7f2b7844d740 1 -- 172.21.15.32:0/3508563101 wait complete. 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.583+0000 7f2b7844d740 1 -- start start 2024-08-31T07:19:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.583+0000 7f2b7844d740 1 Processor -- start 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf8f4490 0x55aebfce4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcde6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b7844d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf91d350 0x55aebfcdec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b7844d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1ae410 con 0x55aebf8f4490 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b7844d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1add40 con 0x55aebf8fc670 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b7844d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1aafa0 con 0x55aebf91d350 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b7844d740 10 client.0 _ll_register_callbacks cb 0x7f2b5c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf8f4490 0x55aebfce4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcde6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcde6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34888/0 (socket says 172.21.15.32:34888) 2024-08-31T07:19:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6a000640 1 -- 172.21.15.32:0/704089737 learned_addr learned my addr 172.21.15.32:0/704089737 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6b400640 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf91d350 0x55aebfcdec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6a000640 1 -- 172.21.15.32:0/704089737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf91d350 msgr2=0x55aebfcdec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6a000640 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf91d350 0x55aebfcdec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6a000640 1 -- 172.21.15.32:0/704089737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf8f4490 msgr2=0x55aebfce4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6a000640 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf8f4490 0x55aebfce4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6a000640 1 -- 172.21.15.32:0/704089737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aebfcdf1e0 con 0x55aebf8fc670 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.584+0000 7f2b6b400640 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf91d350 0x55aebfcdec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.585+0000 7f2b6a000640 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcde6e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2b58004180 tx=0x7f2b580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8308ef8c732c346b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.585+0000 7f2b54c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.585+0000 7f2b54c00640 1 -- 172.21.15.32:0/704089737 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58036060 con 0x55aebf8fc670 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.585+0000 7f2b54c00640 1 -- 172.21.15.32:0/704089737 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f2b58031440 con 0x55aebf8fc670 2024-08-31T07:19:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.585+0000 7f2b7844d740 1 -- 172.21.15.32:0/704089737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5c0d5630 con 0x55aebf8fc670 2024-08-31T07:19:18.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.585+0000 7f2b54c00640 1 -- 172.21.15.32:0/704089737 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2b58031640 con 0x55aebf8fc670 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.585+0000 7f2b7844d740 10 client.26942 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.585+0000 7f2b7844d740 1 -- 172.21.15.32:0/704089737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aebfd55470 con 0x55aebf8fc670 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.586+0000 7f2b54c00640 1 -- 172.21.15.32:0/704089737 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c0d5630 con 0x55aebf8fc670 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.586+0000 7f2b54c00640 1 -- 172.21.15.32:0/704089737 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f2b5801f830 con 0x55aebf8fc670 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.586+0000 7f2b7844d740 10 client.26942 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.586+0000 7f2b7844d740 10 client.26942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.586+0000 7f2b7844d740 20 client.26942 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.586+0000 7f2b7844d740 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aebf8b1080 0x55aebfd57cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.586+0000 7f2b7844d740 4 client.26942 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.586+0000 7f2b7844d740 1 -- 172.21.15.32:0/704089737 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aebfcc1c20 con 0x55aebf8b1080 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.586+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aebf8b1080 0x55aebfd57cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.587+0000 7f2b6aa00640 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aebf8b1080 0x55aebfd57cd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f58fc6318dfe6cd4 server_cookie=146fbc6315b5c32 in_seq=0 out_seq=0 2024-08-31T07:19:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.587+0000 7f2b54c00640 10 client.26942 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:18.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b54c00640 1 -- 172.21.15.32:0/704089737 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aebfcc1c20 con 0x55aebf8b1080 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b54c00640 10 client.26942 handle_command_reply: tid=0 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 client.26942 shutdown 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 -- 172.21.15.32:0/704089737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 msgr2=0x55aebfcde6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcde6e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2b58004180 tx=0x7f2b580046c0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 -- 172.21.15.32:0/704089737 shutdown_connections 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aebf8b1080 0x55aebfd57cd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aebf91d350 0x55aebfcdec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b54c00640 0 client.26942 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aebf8fc670 0x55aebfcde6e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 --2- 172.21.15.32:0/704089737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aebf8f4490 0x55aebfce4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 -- 172.21.15.32:0/704089737 >> 172.21.15.32:0/704089737 conn(0x55aebf911740 msgr2=0x55aebfcd9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 -- 172.21.15.32:0/704089737 shutdown_connections 2024-08-31T07:19:18.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.589+0000 7f2b7844d740 1 -- 172.21.15.32:0/704089737 wait complete. 2024-08-31T07:19:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.590+0000 7f2b7844d740 20 client.26942 trim_cache size 0 max 16384 2024-08-31T07:19:18.592 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:19:18.592 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:19:18.592 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8", 2024-08-31T07:19:18.592 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:19:18.592 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:19:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.590+0000 7f2b6be00640 1 -- 172.21.15.32:0/916717945 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b500b4c50 msgr2=0x7f2b500b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.590+0000 7f2b6be00640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b500b4c50 0x7f2b500b7160 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009cc0 tx=0x7f2b58005f90 comp rx=0 tx=0).stop 2024-08-31T07:19:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.590+0000 7f2b6be00640 1 -- 172.21.15.32:0/916717945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c105950 msgr2=0x7f2b6c19c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.590+0000 7f2b6be00640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c105950 0x7f2b6c19c2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c005ab0 tx=0x7f2b5c003c10 comp rx=0 tx=0).stop 2024-08-31T07:19:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.591+0000 7f2b6be00640 1 -- 172.21.15.32:0/916717945 shutdown_connections 2024-08-31T07:19:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.591+0000 7f2b6be00640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b500b4c50 0x7f2b500b7160 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.591+0000 7f2b6be00640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10bbe0 0x7f2b6c1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.591+0000 7f2b6be00640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c10b290 0x7f2b6c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.591+0000 7f2b6be00640 1 --2- 172.21.15.32:0/916717945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c105950 0x7f2b6c19c2a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.591+0000 7f2b6be00640 1 -- 172.21.15.32:0/916717945 >> 172.21.15.32:0/916717945 conn(0x7f2b6c101420 msgr2=0x7f2b6c10a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.591+0000 7f2b6be00640 1 -- 172.21.15.32:0/916717945 shutdown_connections 2024-08-31T07:19:18.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:18.591+0000 7f2b6be00640 1 -- 172.21.15.32:0/916717945 wait complete. 2024-08-31T07:19:18.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 199794 2.77 MB/sec execute 187 sec latency 127.256 ms 2024-08-31T07:19:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:19:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:19:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:19:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:19:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:19:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:19:19.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:19 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:19:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:19:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:19:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:19:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:19:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:19:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:19 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:19:19.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:19:19.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:19:19.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:19:19.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:19:19.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:19:19.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:19 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:19.498 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:19:19.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 201532 2.81 MB/sec execute 188 sec latency 196.258 ms 2024-08-31T07:19:19.947 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:20.330 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:20 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:19:20.330 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:20 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:19:20.331 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:20 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.331 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:20 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.331 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:20 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:19:20.331 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:20 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:19:20.331 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:20 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.331 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:20 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.331 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:20 smithi032 ceph-mon[17252]: pgmap v567: 97 pgs: 97 active+clean; 129 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.7 MiB/s wr, 298 op/s 2024-08-31T07:19:20.391 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:20 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:19:20.391 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:20 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:19:20.391 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:20 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.391 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:20 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.392 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:20 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:19:20.392 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:20 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:19:20.392 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:20 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.392 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:20 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.392 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:20 smithi084 ceph-mon[22387]: pgmap v567: 97 pgs: 97 active+clean; 129 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.7 MiB/s wr, 298 op/s 2024-08-31T07:19:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:20 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:19:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:20 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:19:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:20 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:20 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:20 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:19:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:20 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:19:20.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:20 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:20 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:19:20.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:20 smithi073 ceph-mon[21995]: pgmap v567: 97 pgs: 97 active+clean; 129 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.7 MiB/s wr, 298 op/s 2024-08-31T07:19:20.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.694+0000 7fcfae800640 1 -- 172.21.15.32:0/1419673087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfa81099e0 msgr2=0x7fcfa8110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:20.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.694+0000 7fcfae800640 1 --2- 172.21.15.32:0/1419673087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfa81099e0 0x7fcfa8110210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009ca0 tx=0x7fcfa002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.694+0000 7fcfae800640 1 -- 172.21.15.32:0/1419673087 shutdown_connections 2024-08-31T07:19:20.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.694+0000 7fcfae800640 1 --2- 172.21.15.32:0/1419673087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfa81099e0 0x7fcfa8110210 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.694+0000 7fcfae800640 1 --2- 172.21.15.32:0/1419673087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa8109090 0x7fcfa81094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.694+0000 7fcfae800640 1 --2- 172.21.15.32:0/1419673087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa8103750 0x7fcfa8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.694+0000 7fcfae800640 1 -- 172.21.15.32:0/1419673087 >> 172.21.15.32:0/1419673087 conn(0x7fcfa80ff1e0 msgr2=0x7fcfa8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:20.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.694+0000 7fcfae800640 1 -- 172.21.15.32:0/1419673087 shutdown_connections 2024-08-31T07:19:20.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.694+0000 7fcfae800640 1 -- 172.21.15.32:0/1419673087 wait complete. 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.695+0000 7fcfae800640 1 Processor -- start 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.695+0000 7fcfae800640 1 -- start start 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfae800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa8103750 0x7fcfa819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfae800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa8109090 0x7fcfa819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfae800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfa81099e0 0x7fcfa81a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfae800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81136e0 con 0x7fcfa8109090 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfae800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8113560 con 0x7fcfa81099e0 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfae800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8113860 con 0x7fcfa8103750 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfad400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa8103750 0x7fcfa819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfad400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa8103750 0x7fcfa819c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59302/0 (socket says 172.21.15.32:59302) 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfad400640 1 -- 172.21.15.32:0/3299019851 learned_addr learned my addr 172.21.15.32:0/3299019851 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfade00640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfa81099e0 0x7fcfa81a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfaca00640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa8109090 0x7fcfa819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfad400640 1 -- 172.21.15.32:0/3299019851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfa81099e0 msgr2=0x7fcfa81a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:20.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfad400640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfa81099e0 0x7fcfa81a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfad400640 1 -- 172.21.15.32:0/3299019851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa8109090 msgr2=0x7fcfa819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:20.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfad400640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa8109090 0x7fcfa819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.696+0000 7fcfad400640 1 -- 172.21.15.32:0/3299019851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa81a21e0 con 0x7fcfa8103750 2024-08-31T07:19:20.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.697+0000 7fcfad400640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa8103750 0x7fcfa819c270 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00e9f0 tx=0x7fcf9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:20.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.697+0000 7fcfa6000640 1 -- 172.21.15.32:0/3299019851 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00cd80 con 0x7fcfa8103750 2024-08-31T07:19:20.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.697+0000 7fcfae800640 1 -- 172.21.15.32:0/3299019851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa81a24d0 con 0x7fcfa8103750 2024-08-31T07:19:20.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.697+0000 7fcfa6000640 1 -- 172.21.15.32:0/3299019851 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf9c0108e0 con 0x7fcfa8103750 2024-08-31T07:19:20.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.697+0000 7fcfa6000640 1 -- 172.21.15.32:0/3299019851 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c018980 con 0x7fcfa8103750 2024-08-31T07:19:20.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.697+0000 7fcfae800640 1 -- 172.21.15.32:0/3299019851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa81a8c30 con 0x7fcfa8103750 2024-08-31T07:19:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.701+0000 7fcfb6a54740 1 Processor -- start 2024-08-31T07:19:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.701+0000 7fcfb6a54740 1 -- start start 2024-08-31T07:19:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.701+0000 7fcfa6000640 1 -- 172.21.15.32:0/3299019851 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcf9c010430 con 0x7fcfa8103750 2024-08-31T07:19:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.701+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b32070 0x55b893ed7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.701+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ed8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.701+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b11390 0x55b893ed85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.701+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b893e4e770 con 0x55b893b32070 2024-08-31T07:19:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.701+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b893e3dd00 con 0x55b893b11390 2024-08-31T07:19:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.701+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b893e41cb0 con 0x55b893b091b0 2024-08-31T07:19:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfaca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b11390 0x55b893ed85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfaca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b11390 0x55b893ed85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34910/0 (socket says 172.21.15.32:34910) 2024-08-31T07:19:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfaca00640 1 -- 172.21.15.32:0/643734605 learned_addr learned my addr 172.21.15.32:0/643734605 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfade00640 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b32070 0x55b893ed7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfad400640 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ed8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfaca00640 1 -- 172.21.15.32:0/643734605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 msgr2=0x55b893ed8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfaca00640 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ed8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfaca00640 1 -- 172.21.15.32:0/643734605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b32070 msgr2=0x55b893ed7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfaca00640 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b32070 0x55b893ed7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfaca00640 1 -- 172.21.15.32:0/643734605 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b893b9d650 con 0x55b893b11390 2024-08-31T07:19:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfade00640 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b32070 0x55b893ed7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.702+0000 7fcfad400640 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ed8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.703+0000 7fcfaca00640 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b11390 0x55b893ed85a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009c20 tx=0x7fcf94011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f67eaf64146709f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.703+0000 7fcfa6000640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf900b4ca0 0x7fcf900b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.703+0000 7fcf9a600640 1 -- 172.21.15.32:0/643734605 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94004230 con 0x55b893b11390 2024-08-31T07:19:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.703+0000 7fcf9a600640 1 -- 172.21.15.32:0/643734605 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf940043d0 con 0x55b893b11390 2024-08-31T07:19:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.703+0000 7fcf9a600640 1 -- 172.21.15.32:0/643734605 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94004d50 con 0x55b893b11390 2024-08-31T07:19:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.703+0000 7fcfa6000640 1 -- 172.21.15.32:0/3299019851 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fcf9c0e5860 con 0x7fcfa8103750 2024-08-31T07:19:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.703+0000 7fcfade00640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf900b4ca0 0x7fcf900b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.704+0000 7fcfb6a54740 1 -- 172.21.15.32:0/643734605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b11390 msgr2=0x55b893ed85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.704+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b11390 0x55b893ed85a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009c20 tx=0x7fcf94011810 comp rx=0 tx=0).stop 2024-08-31T07:19:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.704+0000 7fcfb6a54740 1 -- 172.21.15.32:0/643734605 shutdown_connections 2024-08-31T07:19:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.704+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b11390 0x55b893ed85a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.704+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ed8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.704+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/643734605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b32070 0x55b893ed7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.704+0000 7fcfade00640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf900b4ca0 0x7fcf900b71b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0006010 tx=0x7fcfa0005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.704+0000 7fcfb6a54740 1 -- 172.21.15.32:0/643734605 >> 172.21.15.32:0/643734605 conn(0x55b893b26460 msgr2=0x55b893ed5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.704+0000 7fcfb6a54740 1 -- 172.21.15.32:0/643734605 shutdown_connections 2024-08-31T07:19:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.704+0000 7fcfb6a54740 1 -- 172.21.15.32:0/643734605 wait complete. 2024-08-31T07:19:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.705+0000 7fcfb6a54740 1 -- start start 2024-08-31T07:19:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.705+0000 7fcfb6a54740 1 Processor -- start 2024-08-31T07:19:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ee36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 0x55b893eef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b893ca0700 con 0x55b893b11390 2024-08-31T07:19:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b893dc4cb0 con 0x55b893b32070 2024-08-31T07:19:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b893ee6300 con 0x55b893b091b0 2024-08-31T07:19:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfb6a54740 10 client.0 _ll_register_callbacks cb 0x55b893dea430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfad400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ee36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfad400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ee36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59324/0 (socket says 172.21.15.32:59324) 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfad400640 1 -- 172.21.15.32:0/2331062393 learned_addr learned my addr 172.21.15.32:0/2331062393 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfaca00640 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.706+0000 7fcfade00640 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 0x55b893eef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcfade00640 1 -- 172.21.15.32:0/2331062393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 msgr2=0x55b893ee36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcfade00640 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ee36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcfade00640 1 -- 172.21.15.32:0/2331062393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 msgr2=0x55b893ee3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcfade00640 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcfade00640 1 -- 172.21.15.32:0/2331062393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b893e05870 con 0x55b893b32070 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcfaca00640 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcfad400640 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ee36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcfade00640 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 0x55b893eef5f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0005e30 tx=0x7fcfa002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=307ee18772163cb6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcf8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcf8ea00640 1 -- 172.21.15.32:0/2331062393 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa002f610 con 0x55b893b32070 2024-08-31T07:19:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcf8ea00640 1 -- 172.21.15.32:0/2331062393 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fcfa002f7b0 con 0x55b893b32070 2024-08-31T07:19:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.707+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2331062393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf940043d0 con 0x55b893b32070 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcf8ea00640 1 -- 172.21.15.32:0/2331062393 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcfa002f9b0 con 0x55b893b32070 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcfb6a54740 10 client.26954 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2331062393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b893ef2320 con 0x55b893b32070 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcf8ea00640 1 -- 172.21.15.32:0/2331062393 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf940043d0 con 0x55b893b32070 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcf8ea00640 1 -- 172.21.15.32:0/2331062393 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fcfa0051070 con 0x55b893b32070 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcfb6a54740 10 client.26954 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcfb6a54740 10 client.26954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcfb6a54740 20 client.26954 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b893af09f0 0x55b893ef5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcfb6a54740 4 client.26954 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.708+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2331062393 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b893e506c0 con 0x55b893af09f0 2024-08-31T07:19:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.709+0000 7fcfad400640 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b893af09f0 0x55b893ef5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.709+0000 7fcf8ea00640 10 client.26954 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.709+0000 7fcfad400640 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b893af09f0 0x55b893ef5830 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f1e092a7d7e5bf server_cookie=f2315b4543fcbd28 in_seq=0 out_seq=0 2024-08-31T07:19:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.710+0000 7fcf8ea00640 1 -- 172.21.15.32:0/2331062393 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b893e506c0 con 0x55b893af09f0 2024-08-31T07:19:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.710+0000 7fcf8ea00640 10 client.26954 handle_command_reply: tid=0 2024-08-31T07:19:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.710+0000 7fcfb6a54740 1 client.26954 shutdown 2024-08-31T07:19:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.710+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2331062393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 msgr2=0x55b893eef5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.710+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 0x55b893eef5f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0005e30 tx=0x7fcfa002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.711+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2331062393 shutdown_connections 2024-08-31T07:19:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.711+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b893af09f0 0x55b893ef5830 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.711+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 0x55b893eef5f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.711+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.711+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/2331062393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893ee36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.711+0000 7fcf8ea00640 0 client.26954 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.711+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2331062393 >> 172.21.15.32:0/2331062393 conn(0x55b893b26460 msgr2=0x55b893edad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.711+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2331062393 shutdown_connections 2024-08-31T07:19:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.711+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2331062393 wait complete. 2024-08-31T07:19:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.711+0000 7fcfb6a54740 20 client.26954 trim_cache size 0 max 16384 2024-08-31T07:19:20.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 202760 2.83 MB/sec execute 189 sec latency 11.866 ms 2024-08-31T07:19:20.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfb6a54740 1 Processor -- start 2024-08-31T07:19:20.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfb6a54740 1 -- start start 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893eef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 0x55b893ee06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81b13a0 con 0x55b893b11390 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81b2fa0 con 0x55b893b32070 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81b1a70 con 0x55b893b091b0 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfad400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893eef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfad400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893eef4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59338/0 (socket says 172.21.15.32:59338) 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfad400640 1 -- 172.21.15.32:0/2165263793 learned_addr learned my addr 172.21.15.32:0/2165263793 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.731+0000 7fcfaca00640 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.732+0000 7fcfade00640 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 0x55b893ee06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.732+0000 7fcfad400640 1 -- 172.21.15.32:0/2165263793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 msgr2=0x55b893ee06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.732+0000 7fcfad400640 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 0x55b893ee06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.732+0000 7fcfad400640 1 -- 172.21.15.32:0/2165263793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 msgr2=0x55b893ee5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.732+0000 7fcfad400640 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.732+0000 7fcfad400640 1 -- 172.21.15.32:0/2165263793 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b893ecfc30 con 0x55b893b091b0 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.732+0000 7fcfade00640 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 0x55b893ee06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.732+0000 7fcfaca00640 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.732+0000 7fcfad400640 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893eef4f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0abcf0 tx=0x7fcf9c0e7cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dc30b90694ca973 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.733+0000 7fcf99c00640 1 -- 172.21.15.32:0/2165263793 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c010a90 con 0x55b893b091b0 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.733+0000 7fcf99c00640 1 -- 172.21.15.32:0/2165263793 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf9c0a8280 con 0x55b893b091b0 2024-08-31T07:19:20.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.733+0000 7fcf99c00640 1 -- 172.21.15.32:0/2165263793 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0ab4b0 con 0x55b893b091b0 2024-08-31T07:19:20.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.734+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2165263793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 msgr2=0x55b893eef4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.734+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893eef4f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0abcf0 tx=0x7fcf9c0e7cb0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.734+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2165263793 shutdown_connections 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.734+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b32070 0x55b893ee06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.734+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.734+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/2165263793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b091b0 0x55b893eef4f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.734+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2165263793 >> 172.21.15.32:0/2165263793 conn(0x55b893b26460 msgr2=0x55b893ee4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.734+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2165263793 shutdown_connections 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.734+0000 7fcfb6a54740 1 -- 172.21.15.32:0/2165263793 wait complete. 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.735+0000 7fcfb6a54740 1 -- start start 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.735+0000 7fcfb6a54740 1 Processor -- start 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.735+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b091b0 0x55b893eebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfb6a54740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b32070 0x55b893ee7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81b2810 con 0x55b893b11390 2024-08-31T07:19:20.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81b2140 con 0x55b893b091b0 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfb6a54740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81af3a0 con 0x55b893b32070 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfb6a54740 10 client.0 _ll_register_callbacks cb 0x7fcfa0031f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfad400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b091b0 0x55b893eebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfade00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b32070 0x55b893ee7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfad400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b091b0 0x55b893eebe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34944/0 (socket says 172.21.15.32:34944) 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfad400640 1 -- 172.21.15.32:0/147320842 learned_addr learned my addr 172.21.15.32:0/147320842 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfad400640 1 -- 172.21.15.32:0/147320842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b32070 msgr2=0x55b893ee7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfad400640 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b32070 0x55b893ee7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfaca00640 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfad400640 1 -- 172.21.15.32:0/147320842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 msgr2=0x55b893ee7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfad400640 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfad400640 1 -- 172.21.15.32:0/147320842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b893ee81c0 con 0x55b893b091b0 2024-08-31T07:19:20.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfade00640 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b32070 0x55b893ee7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.736+0000 7fcfaca00640 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.737+0000 7fcfad400640 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b091b0 0x55b893eebe90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00e9c0 tx=0x7fcf9c0b0930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d6dfe2a4b0fa65 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.737+0000 7fcf8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.737+0000 7fcf8ea00640 1 -- 172.21.15.32:0/147320842 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0abb00 con 0x55b893b091b0 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.737+0000 7fcf8ea00640 1 -- 172.21.15.32:0/147320842 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fcf9c0b3b30 con 0x55b893b091b0 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.737+0000 7fcfb6a54740 1 -- 172.21.15.32:0/147320842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf9c0a8280 con 0x55b893b091b0 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.737+0000 7fcf8ea00640 1 -- 172.21.15.32:0/147320842 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcf9c0b3db0 con 0x55b893b091b0 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.737+0000 7fcfb6a54740 10 client.26960 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.737+0000 7fcfb6a54740 1 -- 172.21.15.32:0/147320842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b893f60320 con 0x55b893b091b0 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.738+0000 7fcf8ea00640 1 -- 172.21.15.32:0/147320842 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf9c0a8280 con 0x55b893b091b0 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.738+0000 7fcf8ea00640 1 -- 172.21.15.32:0/147320842 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fcf9c0a7960 con 0x55b893b091b0 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.738+0000 7fcfb6a54740 10 client.26960 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.738+0000 7fcfb6a54740 10 client.26960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:19:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.738+0000 7fcfb6a54740 20 client.26960 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.738+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b893ac5da0 0x55b893f62b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.738+0000 7fcfb6a54740 4 client.26960 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:19:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.738+0000 7fcfb6a54740 1 -- 172.21.15.32:0/147320842 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b893e4c160 con 0x55b893ac5da0 2024-08-31T07:19:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.738+0000 7fcfaca00640 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b893ac5da0 0x55b893f62b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.739+0000 7fcf8ea00640 10 client.26960 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.739+0000 7fcfaca00640 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b893ac5da0 0x55b893f62b80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea2d44d8db55791 server_cookie=4eca54a74fcf8656 in_seq=0 out_seq=0 2024-08-31T07:19:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.739+0000 7fcf8ea00640 1 -- 172.21.15.32:0/147320842 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x55b893e4c160 con 0x55b893ac5da0 2024-08-31T07:19:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.739+0000 7fcf8ea00640 10 client.26960 handle_command_reply: tid=0 2024-08-31T07:19:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.739+0000 7fcfb6a54740 1 client.26960 shutdown 2024-08-31T07:19:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 1 -- 172.21.15.32:0/147320842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b091b0 msgr2=0x55b893eebe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b091b0 0x55b893eebe90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00e9c0 tx=0x7fcf9c0b0930 comp rx=0 tx=0).stop 2024-08-31T07:19:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 1 -- 172.21.15.32:0/147320842 shutdown_connections 2024-08-31T07:19:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b893ac5da0 0x55b893f62b80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893b32070 0x55b893ee7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b893b11390 0x55b893ee7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 1 --2- 172.21.15.32:0/147320842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b893b091b0 0x55b893eebe90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 1 -- 172.21.15.32:0/147320842 >> 172.21.15.32:0/147320842 conn(0x55b893b26460 msgr2=0x55b893ed8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcf8ea00640 0 client.26960 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 1 -- 172.21.15.32:0/147320842 shutdown_connections 2024-08-31T07:19:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 1 -- 172.21.15.32:0/147320842 wait complete. 2024-08-31T07:19:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.740+0000 7fcfb6a54740 20 client.26960 trim_cache size 0 max 16384 2024-08-31T07:19:20.744 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:19:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:19:20.744 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8", 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:20.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:20.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:19:20.747 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:20.748 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:20.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:20.750 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 -- 172.21.15.32:0/3299019851 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf900b4ca0 msgr2=0x7fcf900b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf900b4ca0 0x7fcf900b71b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0006010 tx=0x7fcfa0005d80 comp rx=0 tx=0).stop 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 -- 172.21.15.32:0/3299019851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa8103750 msgr2=0x7fcfa819c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa8103750 0x7fcfa819c270 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00e9f0 tx=0x7fcf9c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 -- 172.21.15.32:0/3299019851 shutdown_connections 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf900b4ca0 0x7fcf900b71b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfa81099e0 0x7fcfa81a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa8109090 0x7fcfa819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 --2- 172.21.15.32:0/3299019851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfa8103750 0x7fcfa819c270 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 -- 172.21.15.32:0/3299019851 >> 172.21.15.32:0/3299019851 conn(0x7fcfa80ff1e0 msgr2=0x7fcfa81086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 -- 172.21.15.32:0/3299019851 shutdown_connections 2024-08-31T07:19:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:20.741+0000 7fcfae800640 1 -- 172.21.15.32:0/3299019851 wait complete. 2024-08-31T07:19:21.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:21 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:19:21.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:21 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:19:21.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:21 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:19:21.638 INFO:tasks.cephfs.filesystem:scrub status for tag:7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8 - {'path': '/', 'tag': '7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8', 'options': 'recursive,force'} 2024-08-31T07:19:21.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 204012 2.84 MB/sec execute 190 sec latency 44.103 ms 2024-08-31T07:19:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: pgmap v568: 97 pgs: 97 active+clean; 130 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 260 op/s 2024-08-31T07:19:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi073 to 176.0M 2024-08-31T07:19:22.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi084 to 176.0M 2024-08-31T07:19:22.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: Unable to set osd_memory_target on smithi073 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-08-31T07:19:22.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: Unable to set osd_memory_target on smithi084 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-08-31T07:19:22.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:19:22.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:19:22.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:19:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: pgmap v568: 97 pgs: 97 active+clean; 130 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 260 op/s 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi073 to 176.0M 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi084 to 176.0M 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: Unable to set osd_memory_target on smithi073 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: Unable to set osd_memory_target on smithi084 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:19:22.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:19:22.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:19:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: pgmap v568: 97 pgs: 97 active+clean; 130 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 260 op/s 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi073 to 176.0M 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi084 to 176.0M 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: Unable to set osd_memory_target on smithi073 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-08-31T07:19:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: Unable to set osd_memory_target on smithi084 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-08-31T07:19:22.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:19:22.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:19:22.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:19:22.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 205890 2.89 MB/sec execute 191 sec latency 48.696 ms 2024-08-31T07:19:23.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 206961 2.90 MB/sec execute 192 sec latency 12.817 ms 2024-08-31T07:19:24.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:24 smithi032 ceph-mon[17252]: pgmap v569: 97 pgs: 97 active+clean; 131 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 224 op/s 2024-08-31T07:19:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:24 smithi073 ceph-mon[21995]: pgmap v569: 97 pgs: 97 active+clean; 131 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 224 op/s 2024-08-31T07:19:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:24 smithi084 ceph-mon[22387]: pgmap v569: 97 pgs: 97 active+clean; 131 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 224 op/s 2024-08-31T07:19:24.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 208467 2.93 MB/sec execute 193 sec latency 82.040 ms 2024-08-31T07:19:25.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 209757 2.95 MB/sec execute 194 sec latency 21.002 ms 2024-08-31T07:19:26.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:26 smithi032 ceph-mon[17252]: pgmap v570: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-08-31T07:19:26.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:26 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:19:26.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:26 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:19:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:26 smithi073 ceph-mon[21995]: pgmap v570: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-08-31T07:19:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:26 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:19:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:26 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:19:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:26 smithi084 ceph-mon[22387]: pgmap v570: 97 pgs: 97 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-08-31T07:19:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:26 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:19:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:26 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:19:26.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 210982 2.97 MB/sec execute 195 sec latency 17.603 ms 2024-08-31T07:19:27.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:27 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:19:27.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:27 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:19:27.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:27 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:19:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:27 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:19:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:27 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:19:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:27 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:19:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:27 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:19:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:27 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:19:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:27 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:19:27.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 212953 3.02 MB/sec execute 196 sec latency 56.004 ms 2024-08-31T07:19:28.330 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:28 smithi073 ceph-mon[21995]: pgmap v571: 97 pgs: 97 active+clean; 138 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-08-31T07:19:28.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:28 smithi032 ceph-mon[17252]: pgmap v571: 97 pgs: 97 active+clean; 138 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-08-31T07:19:28.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:28 smithi084 ceph-mon[22387]: pgmap v571: 97 pgs: 97 active+clean; 138 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-08-31T07:19:28.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 213926 3.02 MB/sec execute 197 sec latency 18.056 ms 2024-08-31T07:19:29.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 215587 3.05 MB/sec execute 198 sec latency 16.275 ms 2024-08-31T07:19:30.336 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:30 smithi032 ceph-mon[17252]: pgmap v572: 97 pgs: 97 active+clean; 139 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 269 op/s 2024-08-31T07:19:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:30 smithi073 ceph-mon[21995]: pgmap v572: 97 pgs: 97 active+clean; 139 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 269 op/s 2024-08-31T07:19:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:30 smithi084 ceph-mon[22387]: pgmap v572: 97 pgs: 97 active+clean; 139 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 269 op/s 2024-08-31T07:19:30.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 216699 3.07 MB/sec execute 199 sec latency 159.431 ms 2024-08-31T07:19:31.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 217878 3.08 MB/sec execute 200 sec latency 15.502 ms 2024-08-31T07:19:32.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:32 smithi032 ceph-mon[17252]: pgmap v573: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 218 op/s 2024-08-31T07:19:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:32 smithi073 ceph-mon[21995]: pgmap v573: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 218 op/s 2024-08-31T07:19:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:32 smithi084 ceph-mon[22387]: pgmap v573: 97 pgs: 97 active+clean; 140 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 218 op/s 2024-08-31T07:19:32.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 219146 3.10 MB/sec execute 201 sec latency 17.309 ms 2024-08-31T07:19:33.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 220368 3.12 MB/sec execute 202 sec latency 120.962 ms 2024-08-31T07:19:34.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:34 smithi032 ceph-mon[17252]: pgmap v574: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-08-31T07:19:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:34 smithi073 ceph-mon[21995]: pgmap v574: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-08-31T07:19:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:34 smithi084 ceph-mon[22387]: pgmap v574: 97 pgs: 97 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-08-31T07:19:34.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 221556 3.13 MB/sec execute 203 sec latency 16.586 ms 2024-08-31T07:19:35.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 222536 3.13 MB/sec execute 204 sec latency 419.345 ms 2024-08-31T07:19:36.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:36 smithi032 ceph-mon[17252]: pgmap v575: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-08-31T07:19:36.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:36 smithi073 ceph-mon[21995]: pgmap v575: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-08-31T07:19:36.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:36 smithi084 ceph-mon[22387]: pgmap v575: 97 pgs: 97 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-08-31T07:19:36.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 222709 3.13 MB/sec execute 205 sec latency 792.103 ms 2024-08-31T07:19:37.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 223997 3.15 MB/sec execute 206 sec latency 17.501 ms 2024-08-31T07:19:38.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:38 smithi032 ceph-mon[17252]: pgmap v576: 97 pgs: 97 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2024-08-31T07:19:38.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:38 smithi073 ceph-mon[21995]: pgmap v576: 97 pgs: 97 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2024-08-31T07:19:38.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:38 smithi084 ceph-mon[22387]: pgmap v576: 97 pgs: 97 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2024-08-31T07:19:38.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 225911 3.17 MB/sec execute 207 sec latency 17.182 ms 2024-08-31T07:19:39.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 227335 3.21 MB/sec execute 208 sec latency 125.679 ms 2024-08-31T07:19:40.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:40 smithi032 ceph-mon[17252]: pgmap v577: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-08-31T07:19:40.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:40 smithi073 ceph-mon[21995]: pgmap v577: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-08-31T07:19:40.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:40 smithi084 ceph-mon[22387]: pgmap v577: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-08-31T07:19:40.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 227969 3.20 MB/sec execute 209 sec latency 251.084 ms 2024-08-31T07:19:41.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 227969 3.19 MB/sec execute 210 sec latency 1251.146 ms 2024-08-31T07:19:42.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:42 smithi032 ceph-mon[17252]: pgmap v578: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-08-31T07:19:42.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:42 smithi073 ceph-mon[21995]: pgmap v578: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-08-31T07:19:42.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:42 smithi084 ceph-mon[22387]: pgmap v578: 97 pgs: 97 active+clean; 152 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-08-31T07:19:42.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 229782 3.22 MB/sec execute 211 sec latency 1322.627 ms 2024-08-31T07:19:43.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 230971 3.25 MB/sec execute 212 sec latency 282.666 ms 2024-08-31T07:19:43.793 INFO:tasks.quiescer.fs.[cephfs]:Total duration (56.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-31T07:19:43.793 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.0 2024-08-31T07:19:43.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:19:44.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:44 smithi032 ceph-mon[17252]: pgmap v579: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-08-31T07:19:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:44 smithi073 ceph-mon[21995]: pgmap v579: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-08-31T07:19:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:44 smithi084 ceph-mon[22387]: pgmap v579: 97 pgs: 97 active+clean; 153 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-08-31T07:19:44.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 232127 3.25 MB/sec execute 213 sec latency 14.368 ms 2024-08-31T07:19:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.968+0000 7f3793200640 1 -- 172.21.15.32:0/1453769439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f378c105b50 msgr2=0x7f378c105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.968+0000 7f3793200640 1 --2- 172.21.15.32:0/1453769439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f378c105b50 0x7f378c105f60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f377c009860 tx=0x7f377c02a340 comp rx=0 tx=0).stop 2024-08-31T07:19:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.968+0000 7f3793200640 1 -- 172.21.15.32:0/1453769439 shutdown_connections 2024-08-31T07:19:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.968+0000 7f3793200640 1 --2- 172.21.15.32:0/1453769439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c10bde0 0x7f378c112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.968+0000 7f3793200640 1 --2- 172.21.15.32:0/1453769439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f378c10b490 0x7f378c10b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.968+0000 7f3793200640 1 --2- 172.21.15.32:0/1453769439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f378c105b50 0x7f378c105f60 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.968+0000 7f3793200640 1 -- 172.21.15.32:0/1453769439 >> 172.21.15.32:0/1453769439 conn(0x7f378c101560 msgr2=0x7f378c1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.968+0000 7f3793200640 1 -- 172.21.15.32:0/1453769439 shutdown_connections 2024-08-31T07:19:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.968+0000 7f3793200640 1 -- 172.21.15.32:0/1453769439 wait complete. 2024-08-31T07:19:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.969+0000 7f3793200640 1 Processor -- start 2024-08-31T07:19:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f3793200640 1 -- start start 2024-08-31T07:19:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f3793200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c105b50 0x7f378c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f3793200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f378c10b490 0x7f378c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f3793200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f378c10bde0 0x7f378c1a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f3793200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f378c115aa0 con 0x7f378c10b490 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f3793200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f378c115920 con 0x7f378c105b50 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f3793200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f378c115c20 con 0x7f378c10bde0 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f378be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c105b50 0x7f378c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f378be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c105b50 0x7f378c19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37136/0 (socket says 172.21.15.32:37136) 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f378be00640 1 -- 172.21.15.32:0/4273126208 learned_addr learned my addr 172.21.15.32:0/4273126208 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.970+0000 7f3791200640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f378c10bde0 0x7f378c1a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.971+0000 7f378b400640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f378c10b490 0x7f378c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.971+0000 7f378be00640 1 -- 172.21.15.32:0/4273126208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f378c10bde0 msgr2=0x7f378c1a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.971+0000 7f378be00640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f378c10bde0 0x7f378c1a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.971+0000 7f378be00640 1 -- 172.21.15.32:0/4273126208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f378c10b490 msgr2=0x7f378c19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.971+0000 7f378be00640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f378c10b490 0x7f378c19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.971+0000 7f378be00640 1 -- 172.21.15.32:0/4273126208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c1a2480 con 0x7f378c105b50 2024-08-31T07:19:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.971+0000 7f3791200640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f378c10bde0 0x7f378c1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.971+0000 7f378be00640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c105b50 0x7f378c19c510 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f377c035040 tx=0x7f377c032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.972+0000 7f3788c00640 1 -- 172.21.15.32:0/4273126208 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c037040 con 0x7f378c105b50 2024-08-31T07:19:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.972+0000 7f3793200640 1 -- 172.21.15.32:0/4273126208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378c1a2710 con 0x7f378c105b50 2024-08-31T07:19:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.972+0000 7f3788c00640 1 -- 172.21.15.32:0/4273126208 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377c041070 con 0x7f378c105b50 2024-08-31T07:19:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.972+0000 7f3788c00640 1 -- 172.21.15.32:0/4273126208 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c003850 con 0x7f378c105b50 2024-08-31T07:19:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.972+0000 7f3793200640 1 -- 172.21.15.32:0/4273126208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378c1a8ed0 con 0x7f378c105b50 2024-08-31T07:19:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.973+0000 7f3793200640 1 -- 172.21.15.32:0/4273126208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378c106dd0 con 0x7f378c105b50 2024-08-31T07:19:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.976+0000 7f3788c00640 1 -- 172.21.15.32:0/4273126208 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f377c0039f0 con 0x7f378c105b50 2024-08-31T07:19:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.977+0000 7f3788c00640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f37700b4ca0 0x7f37700b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.978+0000 7f3788c00640 1 -- 172.21.15.32:0/4273126208 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f377c105060 con 0x7f378c105b50 2024-08-31T07:19:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.978+0000 7f378b400640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f37700b4ca0 0x7f37700b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.979+0000 7f378b400640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f37700b4ca0 0x7f37700b71b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f378c103da0 tx=0x7f3778005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:44.979+0000 7f3788c00640 1 -- 172.21.15.32:0/4273126208 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f377c0cbe40 con 0x7f378c105b50 2024-08-31T07:19:45.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.245+0000 7f3793200640 1 -- 172.21.15.32:0/4273126208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f378c105f60 con 0x7f378c105b50 2024-08-31T07:19:45.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.246+0000 7f3788c00640 1 -- 172.21.15.32:0/4273126208 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11967 (secure 0 0 0) 0x7f378c105f60 con 0x7f378c105b50 2024-08-31T07:19:45.247 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:19:45.247 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"btime":"2024-08-31T07:19:16:021228+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:19:15.946242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:19:45.247 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 35 2024-08-31T07:19:45.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.248+0000 7f3793200640 1 -- 172.21.15.32:0/4273126208 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f37700b4ca0 msgr2=0x7f37700b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.248+0000 7f3793200640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f37700b4ca0 0x7f37700b71b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f378c103da0 tx=0x7f3778005f90 comp rx=0 tx=0).stop 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.248+0000 7f3793200640 1 -- 172.21.15.32:0/4273126208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c105b50 msgr2=0x7f378c19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.248+0000 7f3793200640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c105b50 0x7f378c19c510 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f377c035040 tx=0x7f377c032d50 comp rx=0 tx=0).stop 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.249+0000 7f3793200640 1 -- 172.21.15.32:0/4273126208 shutdown_connections 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.249+0000 7f3793200640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f37700b4ca0 0x7f37700b71b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.249+0000 7f3793200640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f378c10bde0 0x7f378c1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.249+0000 7f3793200640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f378c10b490 0x7f378c19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.249+0000 7f3793200640 1 --2- 172.21.15.32:0/4273126208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f378c105b50 0x7f378c19c510 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.249+0000 7f3793200640 1 -- 172.21.15.32:0/4273126208 >> 172.21.15.32:0/4273126208 conn(0x7f378c101560 msgr2=0x7f378c109c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.249+0000 7f3793200640 1 -- 172.21.15.32:0/4273126208 shutdown_connections 2024-08-31T07:19:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:45.249+0000 7f3793200640 1 -- 172.21.15.32:0/4273126208 wait complete. 2024-08-31T07:19:45.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 233326 3.27 MB/sec execute 214 sec latency 141.974 ms 2024-08-31T07:19:46.160 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:46 smithi032 ceph-mon[17252]: pgmap v580: 97 pgs: 97 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-08-31T07:19:46.160 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:46 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4273126208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:19:46.308 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 148.0 --await-for 45.0' 2024-08-31T07:19:46.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 148.0 --await-for 45.0 2024-08-31T07:19:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:46 smithi073 ceph-mon[21995]: pgmap v580: 97 pgs: 97 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-08-31T07:19:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:46 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4273126208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:19:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:46 smithi084 ceph-mon[22387]: pgmap v580: 97 pgs: 97 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-08-31T07:19:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:46 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4273126208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:19:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:46.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 233364 3.25 MB/sec execute 215 sec latency 483.896 ms 2024-08-31T07:19:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.476+0000 7fe10ea00640 1 -- 172.21.15.32:0/1844620290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe108103920 msgr2=0x7fe108103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.476+0000 7fe10ea00640 1 --2- 172.21.15.32:0/1844620290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe108103920 0x7fe108103d30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009900 tx=0x7fe0f802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.477+0000 7fe10ea00640 1 -- 172.21.15.32:0/1844620290 shutdown_connections 2024-08-31T07:19:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.477+0000 7fe10ea00640 1 --2- 172.21.15.32:0/1844620290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe108109bb0 0x7fe1081103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.477+0000 7fe10ea00640 1 --2- 172.21.15.32:0/1844620290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108109260 0x7fe108109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.477+0000 7fe10ea00640 1 --2- 172.21.15.32:0/1844620290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe108103920 0x7fe108103d30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.477+0000 7fe10ea00640 1 -- 172.21.15.32:0/1844620290 >> 172.21.15.32:0/1844620290 conn(0x7fe1080ff370 msgr2=0x7fe1081017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.477+0000 7fe10ea00640 1 -- 172.21.15.32:0/1844620290 shutdown_connections 2024-08-31T07:19:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.477+0000 7fe10ea00640 1 -- 172.21.15.32:0/1844620290 wait complete. 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.478+0000 7fe10ea00640 1 Processor -- start 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.478+0000 7fe10ea00640 1 -- start start 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.478+0000 7fe10ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108103920 0x7fe10819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.478+0000 7fe10ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe108109260 0x7fe10819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe10ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe108109bb0 0x7fe10819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe10ea00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081138c0 con 0x7fe108103920 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe10ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe108113740 con 0x7fe108109bb0 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe10ea00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe108113a40 con 0x7fe108109260 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe107e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108103920 0x7fe10819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe107e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108103920 0x7fe10819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35988/0 (socket says 172.21.15.32:35988) 2024-08-31T07:19:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe107400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe108109260 0x7fe10819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe107e00640 1 -- 172.21.15.32:0/2092660803 learned_addr learned my addr 172.21.15.32:0/2092660803 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe10ca00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe108109bb0 0x7fe10819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe107400640 1 -- 172.21.15.32:0/2092660803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe108109bb0 msgr2=0x7fe10819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe107400640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe108109bb0 0x7fe10819faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe107400640 1 -- 172.21.15.32:0/2092660803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108103920 msgr2=0x7fe10819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe107400640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108103920 0x7fe10819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe107400640 1 -- 172.21.15.32:0/2092660803 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1081a0220 con 0x7fe108109260 2024-08-31T07:19:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.479+0000 7fe107e00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108103920 0x7fe10819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.480+0000 7fe107400640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe108109260 0x7fe10819a850 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe10806c730 tx=0x7fe0f400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.480+0000 7fe104c00640 1 -- 172.21.15.32:0/2092660803 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400cc40 con 0x7fe108109260 2024-08-31T07:19:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.480+0000 7fe10ea00640 1 -- 172.21.15.32:0/2092660803 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1081a0510 con 0x7fe108109260 2024-08-31T07:19:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.480+0000 7fe10ea00640 1 -- 172.21.15.32:0/2092660803 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1081a6c70 con 0x7fe108109260 2024-08-31T07:19:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.480+0000 7fe104c00640 1 -- 172.21.15.32:0/2092660803 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f400cde0 con 0x7fe108109260 2024-08-31T07:19:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.480+0000 7fe104c00640 1 -- 172.21.15.32:0/2092660803 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4018640 con 0x7fe108109260 2024-08-31T07:19:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.484+0000 7fe115640740 1 Processor -- start 2024-08-31T07:19:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.484+0000 7fe104c00640 1 -- 172.21.15.32:0/2092660803 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe0f4010430 con 0x7fe108109260 2024-08-31T07:19:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.484+0000 7fe115640740 1 -- start start 2024-08-31T07:19:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.484+0000 7fe115640740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad130110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.484+0000 7fe115640740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad130650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.484+0000 7fe115640740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad130b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe115640740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ddad0a5530 con 0x55ddacd60980 2024-08-31T07:19:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe115640740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ddad094ac0 con 0x55ddacd68b60 2024-08-31T07:19:47.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe115640740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ddad098a70 con 0x55ddacd89840 2024-08-31T07:19:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe10ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad130110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe107e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad130650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe10ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad130110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54158/0 (socket says 172.21.15.32:54158) 2024-08-31T07:19:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe10ca00640 1 -- 172.21.15.32:0/2958900862 learned_addr learned my addr 172.21.15.32:0/2958900862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe10ca00640 1 -- 172.21.15.32:0/2958900862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 msgr2=0x55ddad130b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe107400640 1 --2- 172.21.15.32:0/2958900862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad130b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe10ca00640 1 --2- 172.21.15.32:0/2958900862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad130b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe10ca00640 1 -- 172.21.15.32:0/2958900862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 msgr2=0x55ddad130650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe10ca00640 1 --2- 172.21.15.32:0/2958900862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad130650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe10ca00640 1 -- 172.21.15.32:0/2958900862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddad047880 con 0x55ddacd89840 2024-08-31T07:19:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe107400640 1 --2- 172.21.15.32:0/2958900862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad130b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:19:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.485+0000 7fe107e00640 1 --2- 172.21.15.32:0/2958900862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad130650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.486+0000 7fe10ca00640 1 --2- 172.21.15.32:0/2958900862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad130110 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe100009f50 tx=0x7fe1000118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52db1d1d933dde9d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.486+0000 7fe0fd200640 1 -- 172.21.15.32:0/2958900862 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10001e070 con 0x55ddacd89840 2024-08-31T07:19:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.486+0000 7fe0fd200640 1 -- 172.21.15.32:0/2958900862 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe100003c30 con 0x55ddacd89840 2024-08-31T07:19:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.486+0000 7fe0fd200640 1 -- 172.21.15.32:0/2958900862 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10000f8d0 con 0x55ddacd89840 2024-08-31T07:19:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe104c00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe0ec0b4ca0 0x7fe0ec0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe115640740 1 -- 172.21.15.32:0/2958900862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 msgr2=0x55ddad130110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe115640740 1 --2- 172.21.15.32:0/2958900862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad130110 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe100009f50 tx=0x7fe1000118d0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe104c00640 1 -- 172.21.15.32:0/2092660803 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fe0f40e56c0 con 0x7fe108109260 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe107e00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe0ec0b4ca0 0x7fe0ec0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe115640740 1 -- 172.21.15.32:0/2958900862 shutdown_connections 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe115640740 1 --2- 172.21.15.32:0/2958900862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad130b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe115640740 1 --2- 172.21.15.32:0/2958900862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad130650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe115640740 1 --2- 172.21.15.32:0/2958900862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad130110 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe115640740 1 -- 172.21.15.32:0/2958900862 >> 172.21.15.32:0/2958900862 conn(0x55ddacd7d990 msgr2=0x55ddad12e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe115640740 1 -- 172.21.15.32:0/2958900862 shutdown_connections 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.487+0000 7fe115640740 1 -- 172.21.15.32:0/2958900862 wait complete. 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.488+0000 7fe107e00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe0ec0b4ca0 0x7fe0ec0b71b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55ddad12ed90 tx=0x7fe0f8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.488+0000 7fe115640740 1 -- start start 2024-08-31T07:19:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.488+0000 7fe115640740 1 Processor -- start 2024-08-31T07:19:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe115640740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad13bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe115640740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe115640740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad147c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe115640740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ddacef77a0 con 0x55ddacd60980 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe115640740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ddad01b7f0 con 0x55ddacd68b60 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe115640740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ddad13e8f0 con 0x55ddacd89840 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe115640740 10 client.0 _ll_register_callbacks cb 0x55ddad040e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe107e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad13bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe10ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe107400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe10ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54162/0 (socket says 172.21.15.32:54162) 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe107400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49862/0 (socket says 172.21.15.32:49862) 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe10ca00640 1 -- 172.21.15.32:0/1328472746 learned_addr learned my addr 172.21.15.32:0/1328472746 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe10ca00640 1 -- 172.21.15.32:0/1328472746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 msgr2=0x55ddad13c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.489+0000 7fe10ca00640 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.490+0000 7fe10ca00640 1 -- 172.21.15.32:0/1328472746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 msgr2=0x55ddad13bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.490+0000 7fe10ca00640 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad13bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.490+0000 7fe10ca00640 1 -- 172.21.15.32:0/1328472746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddace03b90 con 0x55ddacd89840 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.490+0000 7fe107400640 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.490+0000 7fe10ca00640 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad147c70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe10001c040 tx=0x7fe100004ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44dcf1dc77a30c79 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.490+0000 7fe0f1600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.490+0000 7fe0f1600640 1 -- 172.21.15.32:0/1328472746 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10001e040 con 0x55ddacd89840 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.490+0000 7fe0f1600640 1 -- 172.21.15.32:0/1328472746 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fe100004050 con 0x55ddacd89840 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.490+0000 7fe115640740 1 -- 172.21.15.32:0/1328472746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe100003c30 con 0x55ddacd89840 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe0f1600640 1 -- 172.21.15.32:0/1328472746 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe100032050 con 0x55ddacd89840 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe115640740 10 client.26974 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe115640740 1 -- 172.21.15.32:0/1328472746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ddad14a850 con 0x55ddacd89840 2024-08-31T07:19:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe0f1600640 1 -- 172.21.15.32:0/1328472746 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe100003c30 con 0x55ddacd89840 2024-08-31T07:19:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe0f1600640 1 -- 172.21.15.32:0/1328472746 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fe100021030 con 0x55ddacd89840 2024-08-31T07:19:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe115640740 10 client.26974 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe115640740 10 client.26974 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:19:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe115640740 20 client.26974 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe115640740 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ddacd481c0 0x55ddad14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe115640740 4 client.26974 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.491+0000 7fe115640740 1 -- 172.21.15.32:0/1328472746 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddad0a7480 con 0x55ddacd481c0 2024-08-31T07:19:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.492+0000 7fe107e00640 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ddacd481c0 0x55ddad14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.492+0000 7fe107e00640 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ddacd481c0 0x55ddad14dc60 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=300fdc5e825ac36a server_cookie=cb16f4d0f3ed1839 in_seq=0 out_seq=0 2024-08-31T07:19:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.492+0000 7fe0f1600640 10 client.26974 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe0f1600640 1 -- 172.21.15.32:0/1328472746 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ddad0a7480 con 0x55ddacd481c0 2024-08-31T07:19:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe0f1600640 10 client.26974 handle_command_reply: tid=0 2024-08-31T07:19:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe115640740 1 client.26974 shutdown 2024-08-31T07:19:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe115640740 1 -- 172.21.15.32:0/1328472746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 msgr2=0x55ddad147c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe115640740 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad147c70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe10001c040 tx=0x7fe100004ee0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe115640740 1 -- 172.21.15.32:0/1328472746 shutdown_connections 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe115640740 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ddacd481c0 0x55ddad14dc60 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe115640740 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad147c70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe115640740 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe115640740 1 --2- 172.21.15.32:0/1328472746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad13bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe0f1600640 0 client.26974 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.493+0000 7fe115640740 1 -- 172.21.15.32:0/1328472746 >> 172.21.15.32:0/1328472746 conn(0x55ddacd7d990 msgr2=0x55ddad1332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.494+0000 7fe115640740 1 -- 172.21.15.32:0/1328472746 shutdown_connections 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.494+0000 7fe115640740 1 -- 172.21.15.32:0/1328472746 wait complete. 2024-08-31T07:19:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.494+0000 7fe115640740 20 client.26974 trim_cache size 0 max 16384 2024-08-31T07:19:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.513+0000 7fe115640740 1 Processor -- start 2024-08-31T07:19:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe115640740 1 -- start start 2024-08-31T07:19:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe115640740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad133880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe115640740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe115640740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad138c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe115640740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e800e6c0 con 0x55ddacd60980 2024-08-31T07:19:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe115640740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e80102c0 con 0x55ddacd68b60 2024-08-31T07:19:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe115640740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e800ed90 con 0x55ddacd89840 2024-08-31T07:19:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe107400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe107e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad133880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe107400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49872/0 (socket says 172.21.15.32:49872) 2024-08-31T07:19:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe107400640 1 -- 172.21.15.32:0/4250757664 learned_addr learned my addr 172.21.15.32:0/4250757664 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.514+0000 7fe10ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad138c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.515+0000 7fe107400640 1 -- 172.21.15.32:0/4250757664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 msgr2=0x55ddad138c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:47.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.515+0000 7fe107400640 1 --2- 172.21.15.32:0/4250757664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad138c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.515+0000 7fe107400640 1 -- 172.21.15.32:0/4250757664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 msgr2=0x55ddad133880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:47.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.515+0000 7fe107400640 1 --2- 172.21.15.32:0/4250757664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad133880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.515+0000 7fe107400640 1 -- 172.21.15.32:0/4250757664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddad13cb20 con 0x55ddacd68b60 2024-08-31T07:19:47.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.515+0000 7fe107400640 1 --2- 172.21.15.32:0/4250757664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13d6b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40a7e80 tx=0x7fe0f400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8e0433a1344470 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:47.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.515+0000 7fe0f3e00640 1 -- 172.21.15.32:0/4250757664 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40e3290 con 0x55ddacd68b60 2024-08-31T07:19:47.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.515+0000 7fe0f3e00640 1 -- 172.21.15.32:0/4250757664 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f4093070 con 0x55ddacd68b60 2024-08-31T07:19:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.515+0000 7fe0f3e00640 1 -- 172.21.15.32:0/4250757664 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40e5da0 con 0x55ddacd68b60 2024-08-31T07:19:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.516+0000 7fe115640740 1 -- 172.21.15.32:0/4250757664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 msgr2=0x55ddad13d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.516+0000 7fe115640740 1 --2- 172.21.15.32:0/4250757664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13d6b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40a7e80 tx=0x7fe0f400cf90 comp rx=0 tx=0).stop 2024-08-31T07:19:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.516+0000 7fe115640740 1 -- 172.21.15.32:0/4250757664 shutdown_connections 2024-08-31T07:19:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.516+0000 7fe115640740 1 --2- 172.21.15.32:0/4250757664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd89840 0x55ddad138c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.516+0000 7fe115640740 1 --2- 172.21.15.32:0/4250757664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd68b60 0x55ddad13d6b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.516+0000 7fe115640740 1 --2- 172.21.15.32:0/4250757664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd60980 0x55ddad133880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.516+0000 7fe115640740 1 -- 172.21.15.32:0/4250757664 >> 172.21.15.32:0/4250757664 conn(0x55ddacd7d990 msgr2=0x55ddad13cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.516+0000 7fe115640740 1 -- 172.21.15.32:0/4250757664 shutdown_connections 2024-08-31T07:19:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.516+0000 7fe115640740 1 -- 172.21.15.32:0/4250757664 wait complete. 2024-08-31T07:19:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.517+0000 7fe115640740 1 -- start start 2024-08-31T07:19:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.517+0000 7fe115640740 1 Processor -- start 2024-08-31T07:19:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.517+0000 7fe115640740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd60980 0x55ddad1440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.517+0000 7fe115640740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd68b60 0x55ddad1405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.517+0000 7fe115640740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd89840 0x55ddad140af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.517+0000 7fe115640740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e800fb30 con 0x55ddacd68b60 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.517+0000 7fe115640740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e800f460 con 0x55ddacd89840 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.517+0000 7fe115640740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e800c6c0 con 0x55ddacd60980 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe115640740 10 client.0 _ll_register_callbacks cb 0x7fe100009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe107e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd60980 0x55ddad1440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe10ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd89840 0x55ddad140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe107e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd60980 0x55ddad1440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54190/0 (socket says 172.21.15.32:54190) 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe107400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd68b60 0x55ddad1405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe107e00640 1 -- 172.21.15.32:0/1276363897 learned_addr learned my addr 172.21.15.32:0/1276363897 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe107e00640 1 -- 172.21.15.32:0/1276363897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd89840 msgr2=0x55ddad140af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe107e00640 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd89840 0x55ddad140af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe107e00640 1 -- 172.21.15.32:0/1276363897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd68b60 msgr2=0x55ddad1405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe107e00640 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd68b60 0x55ddad1405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe107e00640 1 -- 172.21.15.32:0/1276363897 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddad141030 con 0x55ddacd60980 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe107e00640 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd60980 0x55ddad1440f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0f803ea70 tx=0x7fe0f8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e8ad675dff8172 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.518+0000 7fe0f1600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.519+0000 7fe0f1600640 1 -- 172.21.15.32:0/1276363897 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8035040 con 0x55ddacd60980 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.519+0000 7fe0f1600640 1 -- 172.21.15.32:0/1276363897 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fe0f8003830 con 0x55ddacd60980 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.519+0000 7fe115640740 1 -- 172.21.15.32:0/1276363897 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f4093070 con 0x55ddacd60980 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.519+0000 7fe0f1600640 1 -- 172.21.15.32:0/1276363897 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe0f8003a30 con 0x55ddacd60980 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.519+0000 7fe115640740 10 client.26980 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.519+0000 7fe115640740 1 -- 172.21.15.32:0/1276363897 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ddad1b9260 con 0x55ddacd60980 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.519+0000 7fe0f1600640 1 -- 172.21.15.32:0/1276363897 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f4093070 con 0x55ddacd60980 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.520+0000 7fe0f1600640 1 -- 172.21.15.32:0/1276363897 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fe0f8056410 con 0x55ddacd60980 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.520+0000 7fe115640740 10 client.26980 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.520+0000 7fe115640740 10 client.26980 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:19:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.520+0000 7fe115640740 20 client.26980 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.520+0000 7fe115640740 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ddacd1d570 0x55ddad1bbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.520+0000 7fe115640740 4 client.26980 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.0, "await_for": 45.0}] 2024-08-31T07:19:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.520+0000 7fe115640740 1 -- 172.21.15.32:0/1276363897 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.0, "await_for": 45.0}) -- 0x55ddad0a2f20 con 0x55ddacd1d570 2024-08-31T07:19:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.520+0000 7fe107400640 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ddacd1d570 0x55ddad1bbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.520+0000 7fe107400640 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ddacd1d570 0x55ddad1bbb80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff48d2900ed7d57 server_cookie=3163f6176dd3df41 in_seq=0 out_seq=0 2024-08-31T07:19:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.520+0000 7fe0f1600640 10 client.26980 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.633+0000 7fe0f1600640 1 -- 172.21.15.32:0/1276363897 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ddad0a2f20 con 0x55ddacd1d570 2024-08-31T07:19:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.633+0000 7fe0f1600640 10 client.26980 handle_command_reply: tid=0 2024-08-31T07:19:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.633+0000 7fe115640740 1 client.26980 shutdown 2024-08-31T07:19:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 1 -- 172.21.15.32:0/1276363897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd60980 msgr2=0x55ddad1440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd60980 0x55ddad1440f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0f803ea70 tx=0x7fe0f8009260 comp rx=0 tx=0).stop 2024-08-31T07:19:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 1 -- 172.21.15.32:0/1276363897 shutdown_connections 2024-08-31T07:19:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ddacd1d570 0x55ddad1bbb80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ddacd89840 0x55ddad140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ddacd68b60 0x55ddad1405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 1 --2- 172.21.15.32:0/1276363897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddacd60980 0x55ddad1440f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe0f1600640 0 client.26980 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 1 -- 172.21.15.32:0/1276363897 >> 172.21.15.32:0/1276363897 conn(0x55ddacd7d990 msgr2=0x55ddad13cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 1 -- 172.21.15.32:0/1276363897 shutdown_connections 2024-08-31T07:19:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 1 -- 172.21.15.32:0/1276363897 wait complete. 2024-08-31T07:19:47.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.634+0000 7fe115640740 20 client.26980 trim_cache size 0 max 16384 2024-08-31T07:19:47.637 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:19:47.637 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 35, 2024-08-31T07:19:47.637 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 22, 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "93e49528": { 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 22, 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 148.0, 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:19:47.638 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:19:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:19:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:19:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:19:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:47.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:47.639 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 -- 172.21.15.32:0/2092660803 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe0ec0b4ca0 msgr2=0x7fe0ec0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe0ec0b4ca0 0x7fe0ec0b71b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55ddad12ed90 tx=0x7fe0f8005a20 comp rx=0 tx=0).stop 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 -- 172.21.15.32:0/2092660803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe108109260 msgr2=0x7fe10819a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe108109260 0x7fe10819a850 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe10806c730 tx=0x7fe0f400ee50 comp rx=0 tx=0).stop 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 -- 172.21.15.32:0/2092660803 shutdown_connections 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe0ec0b4ca0 0x7fe0ec0b71b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe108109bb0 0x7fe10819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe108109260 0x7fe10819a850 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 --2- 172.21.15.32:0/2092660803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108103920 0x7fe10819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 -- 172.21.15.32:0/2092660803 >> 172.21.15.32:0/2092660803 conn(0x7fe1080ff370 msgr2=0x7fe108101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 -- 172.21.15.32:0/2092660803 shutdown_connections 2024-08-31T07:19:47.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:47.635+0000 7fe10ea00640 1 -- 172.21.15.32:0/2092660803 wait complete. 2024-08-31T07:19:47.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.27 MB/sec execute 216 sec latency 474.273 ms 2024-08-31T07:19:47.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:47 smithi032 ceph-mon[17252]: pgmap v581: 97 pgs: 97 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 2.4 MiB/s wr, 259 op/s 2024-08-31T07:19:48.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:47 smithi073 ceph-mon[21995]: pgmap v581: 97 pgs: 97 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 2.4 MiB/s wr, 259 op/s 2024-08-31T07:19:48.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:47 smithi084 ceph-mon[22387]: pgmap v581: 97 pgs: 97 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 2.4 MiB/s wr, 259 op/s 2024-08-31T07:19:48.542 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '93e49528', quiesce took 4.7 seconds. Will release after: 23.3 2024-08-31T07:19:48.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.26 MB/sec execute 217 sec latency 1202.628 ms 2024-08-31T07:19:49.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.24 MB/sec execute 218 sec latency 2202.763 ms 2024-08-31T07:19:50.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:49 smithi032 ceph-mon[17252]: pgmap v582: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-08-31T07:19:50.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:49 smithi073 ceph-mon[21995]: pgmap v582: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-08-31T07:19:50.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:49 smithi084 ceph-mon[22387]: pgmap v582: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-08-31T07:19:50.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.23 MB/sec execute 219 sec latency 3202.848 ms 2024-08-31T07:19:51.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:19:51.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.21 MB/sec execute 220 sec latency 4202.902 ms 2024-08-31T07:19:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:52.342 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:52 smithi032 ceph-mon[17252]: pgmap v583: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.2 MiB/s wr, 230 op/s 2024-08-31T07:19:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:52 smithi073 ceph-mon[21995]: pgmap v583: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.2 MiB/s wr, 230 op/s 2024-08-31T07:19:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:52 smithi084 ceph-mon[22387]: pgmap v583: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.2 MiB/s wr, 230 op/s 2024-08-31T07:19:52.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.20 MB/sec execute 221 sec latency 5202.960 ms 2024-08-31T07:19:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.890+0000 7fe2eee00640 1 -- 172.21.15.32:0/3812857014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8109260 msgr2=0x7fe2e8109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.890+0000 7fe2eee00640 1 --2- 172.21.15.32:0/3812857014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8109260 0x7fe2e8109670 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009900 tx=0x7fe2d002a410 comp rx=0 tx=0).stop 2024-08-31T07:19:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.890+0000 7fe2eee00640 1 -- 172.21.15.32:0/3812857014 shutdown_connections 2024-08-31T07:19:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.890+0000 7fe2eee00640 1 --2- 172.21.15.32:0/3812857014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2e8109bb0 0x7fe2e81103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.890+0000 7fe2eee00640 1 --2- 172.21.15.32:0/3812857014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8109260 0x7fe2e8109670 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.890+0000 7fe2eee00640 1 --2- 172.21.15.32:0/3812857014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e8103920 0x7fe2e8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.890+0000 7fe2eee00640 1 -- 172.21.15.32:0/3812857014 >> 172.21.15.32:0/3812857014 conn(0x7fe2e80ff370 msgr2=0x7fe2e81017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.890+0000 7fe2eee00640 1 -- 172.21.15.32:0/3812857014 shutdown_connections 2024-08-31T07:19:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.890+0000 7fe2eee00640 1 -- 172.21.15.32:0/3812857014 wait complete. 2024-08-31T07:19:52.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.891+0000 7fe2eee00640 1 Processor -- start 2024-08-31T07:19:52.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.891+0000 7fe2eee00640 1 -- start start 2024-08-31T07:19:52.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2eee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8103920 0x7fe2e819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2eee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2e8109260 0x7fe2e819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2eee00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e8109bb0 0x7fe2e819fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2eee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e81138c0 con 0x7fe2e8103920 2024-08-31T07:19:52.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2eee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e8113740 con 0x7fe2e8109260 2024-08-31T07:19:52.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2eee00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e8113a40 con 0x7fe2e8109bb0 2024-08-31T07:19:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8103920 0x7fe2e819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8103920 0x7fe2e819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36042/0 (socket says 172.21.15.32:36042) 2024-08-31T07:19:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2dfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2e8109260 0x7fe2e819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2e7e00640 1 -- 172.21.15.32:0/28841542 learned_addr learned my addr 172.21.15.32:0/28841542 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.892+0000 7fe2ece00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e8109bb0 0x7fe2e819fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2e7e00640 1 -- 172.21.15.32:0/28841542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e8109bb0 msgr2=0x7fe2e819fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2e7e00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e8109bb0 0x7fe2e819fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2e7e00640 1 -- 172.21.15.32:0/28841542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2e8109260 msgr2=0x7fe2e819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2e7e00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2e8109260 0x7fe2e819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2e7e00640 1 -- 172.21.15.32:0/28841542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e81a01c0 con 0x7fe2e8103920 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2ece00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e8109bb0 0x7fe2e819fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2e7e00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8103920 0x7fe2e819a250 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800e9f0 tx=0x7fe2d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2e5600640 1 -- 172.21.15.32:0/28841542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d800cd80 con 0x7fe2e8103920 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2e5600640 1 -- 172.21.15.32:0/28841542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2d80054d0 con 0x7fe2e8103920 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2eee00640 1 -- 172.21.15.32:0/28841542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e81a04b0 con 0x7fe2e8103920 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.893+0000 7fe2e5600640 1 -- 172.21.15.32:0/28841542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8010690 con 0x7fe2e8103920 2024-08-31T07:19:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.894+0000 7fe2eee00640 1 -- 172.21.15.32:0/28841542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e81a6c10 con 0x7fe2e8103920 2024-08-31T07:19:52.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.897+0000 7fe2f5a7b740 1 Processor -- start 2024-08-31T07:19:52.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.897+0000 7fe2f5a7b740 1 -- start start 2024-08-31T07:19:52.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.897+0000 7fe2e5600640 1 -- 172.21.15.32:0/28841542 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe2d8004e90 con 0x7fe2e8103920 2024-08-31T07:19:52.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.897+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x559340164950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.897+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd961b0 0x559340164e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd9e390 0x5593401653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5593400db770 con 0x55933fdbf070 2024-08-31T07:19:52.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5593400cad00 con 0x55933fd9e390 2024-08-31T07:19:52.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5593400cecb0 con 0x55933fd961b0 2024-08-31T07:19:52.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2ece00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x559340164950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2ece00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x559340164950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36046/0 (socket says 172.21.15.32:36046) 2024-08-31T07:19:52.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2e7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd961b0 0x559340164e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2ece00640 1 -- 172.21.15.32:0/746343176 learned_addr learned my addr 172.21.15.32:0/746343176 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:52.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2dfe00640 1 --2- 172.21.15.32:0/746343176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd9e390 0x5593401653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2ece00640 1 -- 172.21.15.32:0/746343176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd961b0 msgr2=0x559340164e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:52.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2ece00640 1 --2- 172.21.15.32:0/746343176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd961b0 0x559340164e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2ece00640 1 -- 172.21.15.32:0/746343176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd9e390 msgr2=0x5593401653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:52.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2ece00640 1 --2- 172.21.15.32:0/746343176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd9e390 0x5593401653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2ece00640 1 -- 172.21.15.32:0/746343176 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933fe2a650 con 0x55933fdbf070 2024-08-31T07:19:52.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.898+0000 7fe2dfe00640 1 --2- 172.21.15.32:0/746343176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd9e390 0x5593401653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:52.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.899+0000 7fe2ece00640 1 --2- 172.21.15.32:0/746343176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x559340164950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0011810 tx=0x7fe2e0011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842deece367cc4ac server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:52.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.899+0000 7fe2dd600640 1 -- 172.21.15.32:0/746343176 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0011310 con 0x55933fdbf070 2024-08-31T07:19:52.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.899+0000 7fe2dd600640 1 -- 172.21.15.32:0/746343176 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2e0003e20 con 0x55933fdbf070 2024-08-31T07:19:52.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.899+0000 7fe2dd600640 1 -- 172.21.15.32:0/746343176 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e00044b0 con 0x55933fdbf070 2024-08-31T07:19:52.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.900+0000 7fe2e5600640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2cc0b4ca0 0x7fe2cc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.900+0000 7fe2e5600640 1 -- 172.21.15.32:0/28841542 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fe2d80e59a0 con 0x7fe2e8103920 2024-08-31T07:19:52.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.900+0000 7fe2e7e00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2cc0b4ca0 0x7fe2cc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.900+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/746343176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 msgr2=0x559340164950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:52.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.900+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/746343176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x559340164950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0011810 tx=0x7fe2e0011cd0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.900+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/746343176 shutdown_connections 2024-08-31T07:19:52.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.900+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/746343176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd9e390 0x5593401653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.900+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/746343176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd961b0 0x559340164e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.900+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/746343176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x559340164950 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.900+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/746343176 >> 172.21.15.32:0/746343176 conn(0x55933fdb3460 msgr2=0x559340162840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:52.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.901+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/746343176 shutdown_connections 2024-08-31T07:19:52.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.901+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/746343176 wait complete. 2024-08-31T07:19:52.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.901+0000 7fe2e7e00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2cc0b4ca0 0x7fe2cc0b71b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80a92f0 tx=0x7fe2d80eb4b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.901+0000 7fe2f5a7b740 1 -- start start 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.901+0000 7fe2f5a7b740 1 Processor -- start 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.902+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x559340174880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.902+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x559340174dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.902+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401807c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.902+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55933ff2d700 con 0x55933fdbf070 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.902+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559340051cb0 con 0x55933fd961b0 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.902+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559340177460 con 0x55933fd9e390 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.902+0000 7fe2f5a7b740 10 client.0 _ll_register_callbacks cb 0x559340077430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.902+0000 7fe2ece00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401807c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.902+0000 7fe2ece00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401807c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36048/0 (socket says 172.21.15.32:36048) 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.902+0000 7fe2ece00640 1 -- 172.21.15.32:0/2987299829 learned_addr learned my addr 172.21.15.32:0/2987299829 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.903+0000 7fe2dfe00640 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x559340174dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.903+0000 7fe2ece00640 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x559340174880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.903+0000 7fe2ece00640 1 -- 172.21.15.32:0/2987299829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 msgr2=0x559340174dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.903+0000 7fe2ece00640 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x559340174dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.903+0000 7fe2ece00640 1 -- 172.21.15.32:0/2987299829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 msgr2=0x559340174880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.903+0000 7fe2ece00640 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x559340174880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.903+0000 7fe2ece00640 1 -- 172.21.15.32:0/2987299829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559340092870 con 0x55933fdbf070 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.903+0000 7fe2ece00640 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401807c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0003df0 tx=0x7fe2e001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28473c5e18a8d8a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.903+0000 7fe2d6000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2d6000640 1 -- 172.21.15.32:0/2987299829 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0004c40 con 0x55933fdbf070 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2d6000640 1 -- 172.21.15.32:0/2987299829 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fe2e0004de0 con 0x55933fdbf070 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/2987299829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e0003e20 con 0x55933fdbf070 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2d6000640 1 -- 172.21.15.32:0/2987299829 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe2e0033050 con 0x55933fdbf070 2024-08-31T07:19:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2f5a7b740 10 client.16980 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/2987299829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559340183440 con 0x55933fdbf070 2024-08-31T07:19:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2d6000640 1 -- 172.21.15.32:0/2987299829 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2e0003e20 con 0x55933fdbf070 2024-08-31T07:19:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2d6000640 1 -- 172.21.15.32:0/2987299829 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fe2e0018020 con 0x55933fdbf070 2024-08-31T07:19:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2f5a7b740 10 client.16980 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2f5a7b740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:19:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.904+0000 7fe2f5a7b740 20 client.16980 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.905+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55933fd7d9f0 0x5593401867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.905+0000 7fe2f5a7b740 4 client.16980 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.905+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/2987299829 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593400dd6c0 con 0x55933fd7d9f0 2024-08-31T07:19:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.905+0000 7fe2dfe00640 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55933fd7d9f0 0x5593401867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.905+0000 7fe2dfe00640 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55933fd7d9f0 0x5593401867a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e74c093bc4b7dd16 server_cookie=c7e968ecba2cb769 in_seq=0 out_seq=0 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.905+0000 7fe2d6000640 10 client.16980 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.906+0000 7fe2d6000640 1 -- 172.21.15.32:0/2987299829 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5593400dd6c0 con 0x55933fd7d9f0 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.906+0000 7fe2d6000640 10 client.16980 handle_command_reply: tid=0 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.906+0000 7fe2f5a7b740 1 client.16980 shutdown 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/2987299829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 msgr2=0x5593401807c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401807c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0003df0 tx=0x7fe2e001e040 comp rx=0 tx=0).stop 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/2987299829 shutdown_connections 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55933fd7d9f0 0x5593401867a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401807c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x559340174dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/2987299829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x559340174880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2d6000640 0 client.16980 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:52.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/2987299829 >> 172.21.15.32:0/2987299829 conn(0x55933fdb3460 msgr2=0x559340167ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:52.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/2987299829 shutdown_connections 2024-08-31T07:19:52.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/2987299829 wait complete. 2024-08-31T07:19:52.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.907+0000 7fe2f5a7b740 20 client.16980 trim_cache size 0 max 16384 2024-08-31T07:19:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2f5a7b740 1 Processor -- start 2024-08-31T07:19:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2f5a7b740 1 -- start start 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x559340176560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x559340166090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e81ad200 con 0x55933fdbf070 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e81aee00 con 0x55933fd961b0 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e81ad8d0 con 0x55933fd9e390 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2ece00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2ece00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401806c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36060/0 (socket says 172.21.15.32:36060) 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2ece00640 1 -- 172.21.15.32:0/1015876002 learned_addr learned my addr 172.21.15.32:0/1015876002 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2dfe00640 1 --2- 172.21.15.32:0/1015876002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x559340166090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.926+0000 7fe2ece00640 1 --2- 172.21.15.32:0/1015876002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x559340176560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.927+0000 7fe2ece00640 1 -- 172.21.15.32:0/1015876002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 msgr2=0x559340166090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.927+0000 7fe2ece00640 1 --2- 172.21.15.32:0/1015876002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x559340166090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.927+0000 7fe2ece00640 1 -- 172.21.15.32:0/1015876002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 msgr2=0x559340176560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.927+0000 7fe2ece00640 1 --2- 172.21.15.32:0/1015876002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x559340176560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.927+0000 7fe2ece00640 1 -- 172.21.15.32:0/1015876002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559340172b80 con 0x55933fdbf070 2024-08-31T07:19:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.927+0000 7fe2ece00640 1 --2- 172.21.15.32:0/1015876002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401806c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0005d70 tx=0x7fe2e0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ee8b025c199225 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.927+0000 7fe2dcc00640 1 -- 172.21.15.32:0/1015876002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0003c30 con 0x55933fdbf070 2024-08-31T07:19:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.927+0000 7fe2dcc00640 1 -- 172.21.15.32:0/1015876002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2e000fe50 con 0x55933fdbf070 2024-08-31T07:19:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.928+0000 7fe2dcc00640 1 -- 172.21.15.32:0/1015876002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e00049b0 con 0x55933fdbf070 2024-08-31T07:19:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.928+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/1015876002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 msgr2=0x5593401806c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.928+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/1015876002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401806c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0005d70 tx=0x7fe2e0009500 comp rx=0 tx=0).stop 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.928+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/1015876002 shutdown_connections 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.928+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/1015876002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401806c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.928+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/1015876002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x559340166090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.928+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/1015876002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x559340176560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.928+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/1015876002 >> 172.21.15.32:0/1015876002 conn(0x55933fdb3460 msgr2=0x559340165980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.928+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/1015876002 shutdown_connections 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.929+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/1015876002 wait complete. 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.929+0000 7fe2f5a7b740 1 -- start start 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.929+0000 7fe2f5a7b740 1 Processor -- start 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x5593401790e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x5593401796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2f5a7b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401ea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e81ae670 con 0x55933fdbf070 2024-08-31T07:19:52.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e81adfa0 con 0x55933fd961b0 2024-08-31T07:19:52.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2f5a7b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e81ab200 con 0x55933fd9e390 2024-08-31T07:19:52.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2f5a7b740 10 client.0 _ll_register_callbacks cb 0x7fe2e0011170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2dfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x5593401796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2dfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x5593401796c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54222/0 (socket says 172.21.15.32:54222) 2024-08-31T07:19:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2dfe00640 1 -- 172.21.15.32:0/3151072314 learned_addr learned my addr 172.21.15.32:0/3151072314 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2ece00640 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401ea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2ece00640 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x5593401790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2dfe00640 1 -- 172.21.15.32:0/3151072314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 msgr2=0x5593401790e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2dfe00640 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x5593401790e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2dfe00640 1 -- 172.21.15.32:0/3151072314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 msgr2=0x5593401ea1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2dfe00640 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401ea1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.930+0000 7fe2dfe00640 1 -- 172.21.15.32:0/3151072314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593401ea840 con 0x55933fd9e390 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.931+0000 7fe2dfe00640 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x5593401796c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0031440 tx=0x7fe2d0004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba6ae0f7140638ce server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.931+0000 7fe2d6000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.931+0000 7fe2d6000640 1 -- 172.21.15.32:0/3151072314 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0035040 con 0x55933fd9e390 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.931+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/3151072314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e000fe50 con 0x55933fd9e390 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.931+0000 7fe2d6000640 1 -- 172.21.15.32:0/3151072314 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fe2d0005590 con 0x55933fd9e390 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.931+0000 7fe2d6000640 1 -- 172.21.15.32:0/3151072314 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe2d0004750 con 0x55933fd9e390 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.931+0000 7fe2f5a7b740 10 client.26998 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.931+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/3151072314 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5593401f1c40 con 0x55933fd9e390 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.932+0000 7fe2d6000640 1 -- 172.21.15.32:0/3151072314 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2e000fe50 con 0x55933fd9e390 2024-08-31T07:19:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.932+0000 7fe2d6000640 1 -- 172.21.15.32:0/3151072314 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fe2d004c020 con 0x55933fd9e390 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.932+0000 7fe2f5a7b740 10 client.26998 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.932+0000 7fe2f5a7b740 10 client.26998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.932+0000 7fe2f5a7b740 20 client.26998 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.932+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55933fd52da0 0x5593401f4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.932+0000 7fe2f5a7b740 4 client.26998 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.932+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/3151072314 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593400d9160 con 0x55933fd52da0 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.932+0000 7fe2ece00640 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55933fd52da0 0x5593401f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.933+0000 7fe2ece00640 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55933fd52da0 0x5593401f4450 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3869c9eb70e89dbd server_cookie=330ca312b91a9428 in_seq=0 out_seq=0 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.933+0000 7fe2d6000640 10 client.26998 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.933+0000 7fe2d6000640 1 -- 172.21.15.32:0/3151072314 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x5593400d9160 con 0x55933fd52da0 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.933+0000 7fe2d6000640 10 client.26998 handle_command_reply: tid=0 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.933+0000 7fe2f5a7b740 1 client.26998 shutdown 2024-08-31T07:19:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/3151072314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 msgr2=0x5593401796c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:52.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x5593401796c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0031440 tx=0x7fe2d0004530 comp rx=0 tx=0).stop 2024-08-31T07:19:52.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/3151072314 shutdown_connections 2024-08-31T07:19:52.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55933fd52da0 0x5593401f4450 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55933fdbf070 0x5593401ea1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55933fd9e390 0x5593401796c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 1 --2- 172.21.15.32:0/3151072314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55933fd961b0 0x5593401790e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2d6000640 0 client.26998 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:19:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/3151072314 >> 172.21.15.32:0/3151072314 conn(0x55933fdb3460 msgr2=0x559340176480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/3151072314 shutdown_connections 2024-08-31T07:19:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 1 -- 172.21.15.32:0/3151072314 wait complete. 2024-08-31T07:19:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.934+0000 7fe2f5a7b740 20 client.26998 trim_cache size 0 max 16384 2024-08-31T07:19:52.939 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:19:52.939 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:19:52.940 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:52.941 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:52.942 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:52.943 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:19:52.944 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:19:52.945 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:19:52.945 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:19:52.945 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:19:52.945 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:19:52.945 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:19:52.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:52.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:19:52.945 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:19:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 -- 172.21.15.32:0/28841542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2cc0b4ca0 msgr2=0x7fe2cc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2cc0b4ca0 0x7fe2cc0b71b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80a92f0 tx=0x7fe2d80eb4b0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 -- 172.21.15.32:0/28841542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8103920 msgr2=0x7fe2e819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8103920 0x7fe2e819a250 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800e9f0 tx=0x7fe2d800edf0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 -- 172.21.15.32:0/28841542 shutdown_connections 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe2cc0b4ca0 0x7fe2cc0b71b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e8109bb0 0x7fe2e819fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2e8109260 0x7fe2e819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 --2- 172.21.15.32:0/28841542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8103920 0x7fe2e819a250 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 -- 172.21.15.32:0/28841542 >> 172.21.15.32:0/28841542 conn(0x7fe2e80ff370 msgr2=0x7fe2e8101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 -- 172.21.15.32:0/28841542 shutdown_connections 2024-08-31T07:19:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:52.935+0000 7fe2eee00640 1 -- 172.21.15.32:0/28841542 wait complete. 2024-08-31T07:19:53.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.18 MB/sec execute 222 sec latency 6203.068 ms 2024-08-31T07:19:53.846 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:19:53.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:19:54.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:54 smithi032 ceph-mon[17252]: pgmap v584: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-08-31T07:19:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:54.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:54 smithi073 ceph-mon[21995]: pgmap v584: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-08-31T07:19:54.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:54 smithi084 ceph-mon[22387]: pgmap v584: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-08-31T07:19:54.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.17 MB/sec execute 223 sec latency 7203.179 ms 2024-08-31T07:19:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.907+0000 7f1a6e000640 1 -- 172.21.15.32:0/3518531951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a68109b90 msgr2=0x7f1a681103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.907+0000 7f1a6e000640 1 --2- 172.21.15.32:0/3518531951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a68109b90 0x7f1a681103c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009ca0 tx=0x7f1a6002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:19:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.907+0000 7f1a6e000640 1 -- 172.21.15.32:0/3518531951 shutdown_connections 2024-08-31T07:19:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.907+0000 7f1a6e000640 1 --2- 172.21.15.32:0/3518531951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a68109b90 0x7f1a681103c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.907+0000 7f1a6e000640 1 --2- 172.21.15.32:0/3518531951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a68109240 0x7f1a68109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.907+0000 7f1a6e000640 1 --2- 172.21.15.32:0/3518531951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a68103900 0x7f1a68103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.907+0000 7f1a6e000640 1 -- 172.21.15.32:0/3518531951 >> 172.21.15.32:0/3518531951 conn(0x7f1a6807b5f0 msgr2=0x7f1a6807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.907+0000 7f1a6e000640 1 -- 172.21.15.32:0/3518531951 shutdown_connections 2024-08-31T07:19:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.908+0000 7f1a6e000640 1 -- 172.21.15.32:0/3518531951 wait complete. 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.909+0000 7f1a6e000640 1 Processor -- start 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.909+0000 7f1a6e000640 1 -- start start 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.909+0000 7f1a6e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a68103900 0x7f1a6819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.909+0000 7f1a6e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a68109240 0x7f1a6819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.909+0000 7f1a6e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a68109b90 0x7f1a681a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.909+0000 7f1a6e000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68113850 con 0x7f1a68109240 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.909+0000 7f1a6e000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681136d0 con 0x7f1a68109b90 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.909+0000 7f1a6e000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681139d0 con 0x7f1a68103900 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.909+0000 7f1a67400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a68103900 0x7f1a6819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.910+0000 7f1a67400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a68103900 0x7f1a6819c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54232/0 (socket says 172.21.15.32:54232) 2024-08-31T07:19:54.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.910+0000 7f1a67400640 1 -- 172.21.15.32:0/3101033580 learned_addr learned my addr 172.21.15.32:0/3101033580 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.910+0000 7f1a67e00640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a68109b90 0x7f1a681a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.910+0000 7f1a5fe00640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a68109240 0x7f1a6819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.910+0000 7f1a67400640 1 -- 172.21.15.32:0/3101033580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a68109b90 msgr2=0x7f1a681a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.910+0000 7f1a67400640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a68109b90 0x7f1a681a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.910+0000 7f1a67400640 1 -- 172.21.15.32:0/3101033580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a68109240 msgr2=0x7f1a6819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.910+0000 7f1a67400640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a68109240 0x7f1a6819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.910+0000 7f1a67400640 1 -- 172.21.15.32:0/3101033580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a681a2310 con 0x7f1a68103900 2024-08-31T07:19:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.910+0000 7f1a67400640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a68103900 0x7f1a6819c3a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800e9f0 tx=0x7f1a5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.911+0000 7f1a64c00640 1 -- 172.21.15.32:0/3101033580 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800cd80 con 0x7f1a68103900 2024-08-31T07:19:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.911+0000 7f1a64c00640 1 -- 172.21.15.32:0/3101033580 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a580108e0 con 0x7f1a68103900 2024-08-31T07:19:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.911+0000 7f1a64c00640 1 -- 172.21.15.32:0/3101033580 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58018980 con 0x7f1a68103900 2024-08-31T07:19:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.911+0000 7f1a6e000640 1 -- 172.21.15.32:0/3101033580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a681a2600 con 0x7f1a68103900 2024-08-31T07:19:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.911+0000 7f1a6e000640 1 -- 172.21.15.32:0/3101033580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a681a8d60 con 0x7f1a68103900 2024-08-31T07:19:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.912+0000 7f1a6e000640 1 -- 172.21.15.32:0/3101033580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a68103d10 con 0x7f1a68103900 2024-08-31T07:19:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.915+0000 7f1a64c00640 1 -- 172.21.15.32:0/3101033580 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1a58010430 con 0x7f1a68103900 2024-08-31T07:19:54.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.917+0000 7f1a64c00640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1a4c0b4ca0 0x7f1a4c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:54.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.917+0000 7f1a64c00640 1 -- 172.21.15.32:0/3101033580 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f1a580e5e00 con 0x7f1a68103900 2024-08-31T07:19:54.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.917+0000 7f1a5fe00640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1a4c0b4ca0 0x7f1a4c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:54.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.917+0000 7f1a5fe00640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1a4c0b4ca0 0x7f1a4c0b71b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1a68108f10 tx=0x7f1a50005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:54.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:54.919+0000 7f1a64c00640 1 -- 172.21.15.32:0/3101033580 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1a580acbe0 con 0x7f1a68103900 2024-08-31T07:19:55.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.156+0000 7f1a6e000640 1 -- 172.21.15.32:0/3101033580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a681a9050 con 0x7f1a68103900 2024-08-31T07:19:55.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.157+0000 7f1a64c00640 1 -- 172.21.15.32:0/3101033580 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11967 (secure 0 0 0) 0x7f1a580b5120 con 0x7f1a68103900 2024-08-31T07:19:55.159 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:19:55.159 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 35 2024-08-31T07:19:55.160 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"btime":"2024-08-31T07:19:16:021228+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:19:15.946242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.160+0000 7f1a6e000640 1 -- 172.21.15.32:0/3101033580 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1a4c0b4ca0 msgr2=0x7f1a4c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.160+0000 7f1a6e000640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1a4c0b4ca0 0x7f1a4c0b71b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1a68108f10 tx=0x7f1a50005f90 comp rx=0 tx=0).stop 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.160+0000 7f1a6e000640 1 -- 172.21.15.32:0/3101033580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a68103900 msgr2=0x7f1a6819c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.160+0000 7f1a6e000640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a68103900 0x7f1a6819c3a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800e9f0 tx=0x7f1a5800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.160+0000 7f1a6e000640 1 -- 172.21.15.32:0/3101033580 shutdown_connections 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.160+0000 7f1a6e000640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1a4c0b4ca0 0x7f1a4c0b71b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.160+0000 7f1a6e000640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a68109b90 0x7f1a681a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.160+0000 7f1a6e000640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a68109240 0x7f1a6819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.160+0000 7f1a6e000640 1 --2- 172.21.15.32:0/3101033580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a68103900 0x7f1a6819c3a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.161+0000 7f1a6e000640 1 -- 172.21.15.32:0/3101033580 >> 172.21.15.32:0/3101033580 conn(0x7f1a6807b5f0 msgr2=0x7f1a68107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.161+0000 7f1a6e000640 1 -- 172.21.15.32:0/3101033580 shutdown_connections 2024-08-31T07:19:55.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:55.161+0000 7f1a6e000640 1 -- 172.21.15.32:0/3101033580 wait complete. 2024-08-31T07:19:55.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.16 MB/sec execute 224 sec latency 8203.253 ms 2024-08-31T07:19:56.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:56 smithi032 ceph-mon[17252]: pgmap v585: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-08-31T07:19:56.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:56 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3101033580' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:19:56.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:19:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:56 smithi073 ceph-mon[21995]: pgmap v585: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-08-31T07:19:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:56 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3101033580' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:19:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:56.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:56 smithi084 ceph-mon[22387]: pgmap v585: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-08-31T07:19:56.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:56 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3101033580' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:19:56.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.14 MB/sec execute 225 sec latency 9203.316 ms 2024-08-31T07:19:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.195+0000 7fa247e00640 1 -- 172.21.15.32:0/2910887185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2481099e0 msgr2=0x7fa248110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.195+0000 7fa247e00640 1 --2- 172.21.15.32:0/2910887185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2481099e0 0x7fa248110210 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa240009be0 tx=0x7fa24002a6c0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.196+0000 7fa247e00640 1 -- 172.21.15.32:0/2910887185 shutdown_connections 2024-08-31T07:19:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.196+0000 7fa247e00640 1 --2- 172.21.15.32:0/2910887185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2481099e0 0x7fa248110210 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.196+0000 7fa247e00640 1 --2- 172.21.15.32:0/2910887185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa248109090 0x7fa2481094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.196+0000 7fa247e00640 1 --2- 172.21.15.32:0/2910887185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa248103750 0x7fa248103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.196+0000 7fa247e00640 1 -- 172.21.15.32:0/2910887185 >> 172.21.15.32:0/2910887185 conn(0x7fa2480ff1e0 msgr2=0x7fa248101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.196+0000 7fa247e00640 1 -- 172.21.15.32:0/2910887185 shutdown_connections 2024-08-31T07:19:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.196+0000 7fa247e00640 1 -- 172.21.15.32:0/2910887185 wait complete. 2024-08-31T07:19:57.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.197+0000 7fa247e00640 1 Processor -- start 2024-08-31T07:19:57.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.197+0000 7fa247e00640 1 -- start start 2024-08-31T07:19:57.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa247e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa248103750 0x7fa248199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa247e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa248109090 0x7fa24819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa247e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2481099e0 0x7fa24819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa247e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa248113690 con 0x7fa2481099e0 2024-08-31T07:19:57.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa247e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa248113510 con 0x7fa248103750 2024-08-31T07:19:57.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa247e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa248113810 con 0x7fa248109090 2024-08-31T07:19:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa246a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa248103750 0x7fa248199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa246a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa248103750 0x7fa248199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60102/0 (socket says 172.21.15.32:60102) 2024-08-31T07:19:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa247400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2481099e0 0x7fa24819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa246a00640 1 -- 172.21.15.32:0/3783192658 learned_addr learned my addr 172.21.15.32:0/3783192658 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.198+0000 7fa237e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa248109090 0x7fa24819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa237e00640 1 -- 172.21.15.32:0/3783192658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa248103750 msgr2=0x7fa248199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa237e00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa248103750 0x7fa248199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa237e00640 1 -- 172.21.15.32:0/3783192658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2481099e0 msgr2=0x7fa24819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa237e00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2481099e0 0x7fa24819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa237e00640 1 -- 172.21.15.32:0/3783192658 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24819ff50 con 0x7fa248109090 2024-08-31T07:19:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa246a00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa248103750 0x7fa248199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa237e00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa248109090 0x7fa24819a580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa24806b730 tx=0x7fa23000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa23fe00640 1 -- 172.21.15.32:0/3783192658 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23000cc40 con 0x7fa248109090 2024-08-31T07:19:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa247e00640 1 -- 172.21.15.32:0/3783192658 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2481a0240 con 0x7fa248109090 2024-08-31T07:19:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa23fe00640 1 -- 172.21.15.32:0/3783192658 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa23000cde0 con 0x7fa248109090 2024-08-31T07:19:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa23fe00640 1 -- 172.21.15.32:0/3783192658 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230018760 con 0x7fa248109090 2024-08-31T07:19:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.199+0000 7fa247e00640 1 -- 172.21.15.32:0/3783192658 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2481a69a0 con 0x7fa248109090 2024-08-31T07:19:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.202+0000 7fa2542b9740 1 Processor -- start 2024-08-31T07:19:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.202+0000 7fa2542b9740 1 -- start start 2024-08-31T07:19:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31ef94b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 0x55e31ef950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebce390 0x55e31ef955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa2542b9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e31ef0b7b0 con 0x55e31ebce390 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa2542b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e31eefad40 con 0x55e31ebef070 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa2542b9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e31eefecf0 con 0x55e31ebc61b0 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa247400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31ef94b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa246a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 0x55e31ef950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa247400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31ef94b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60110/0 (socket says 172.21.15.32:60110) 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa247400640 1 -- 172.21.15.32:0/2826352360 learned_addr learned my addr 172.21.15.32:0/2826352360 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.203+0000 7fa23fe00640 1 -- 172.21.15.32:0/3783192658 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa230010a90 con 0x7fa248109090 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.204+0000 7fa247400640 1 -- 172.21.15.32:0/2826352360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 msgr2=0x55e31ef950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.204+0000 7fa247400640 1 --2- 172.21.15.32:0/2826352360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 0x55e31ef950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.204+0000 7fa247400640 1 -- 172.21.15.32:0/2826352360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebce390 msgr2=0x55e31ef955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.204+0000 7fa247400640 1 --2- 172.21.15.32:0/2826352360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebce390 0x55e31ef955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.204+0000 7fa247400640 1 -- 172.21.15.32:0/2826352360 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e31ec5a650 con 0x55e31ebef070 2024-08-31T07:19:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.204+0000 7fa247400640 1 --2- 172.21.15.32:0/2826352360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31ef94b60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa240009be0 tx=0x7fa240030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c1cf73bd2e5869c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.205+0000 7fa237400640 1 -- 172.21.15.32:0/2826352360 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240041030 con 0x55e31ebef070 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.205+0000 7fa237400640 1 -- 172.21.15.32:0/2826352360 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa240003e20 con 0x55e31ebef070 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.205+0000 7fa237400640 1 -- 172.21.15.32:0/2826352360 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240036040 con 0x55e31ebef070 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.205+0000 7fa23fe00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa2280b4d20 0x7fa2280b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.205+0000 7fa2542b9740 1 -- 172.21.15.32:0/2826352360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 msgr2=0x55e31ef94b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.205+0000 7fa2542b9740 1 --2- 172.21.15.32:0/2826352360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31ef94b60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa240009be0 tx=0x7fa240030430 comp rx=0 tx=0).stop 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa23fe00640 1 -- 172.21.15.32:0/3783192658 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fa2300e5780 con 0x7fa248109090 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa2542b9740 1 -- 172.21.15.32:0/2826352360 shutdown_connections 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa2542b9740 1 --2- 172.21.15.32:0/2826352360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebce390 0x55e31ef955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa2542b9740 1 --2- 172.21.15.32:0/2826352360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 0x55e31ef950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa2542b9740 1 --2- 172.21.15.32:0/2826352360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31ef94b60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa2542b9740 1 -- 172.21.15.32:0/2826352360 >> 172.21.15.32:0/2826352360 conn(0x55e31ebe3460 msgr2=0x55e31ef92a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa246a00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa2280b4d20 0x7fa2280b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa2542b9740 1 -- 172.21.15.32:0/2826352360 shutdown_connections 2024-08-31T07:19:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa2542b9740 1 -- 172.21.15.32:0/2826352360 wait complete. 2024-08-31T07:19:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa246a00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa2280b4d20 0x7fa2280b7230 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa238009fd0 tx=0x7fa238014040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.206+0000 7fa2542b9740 1 -- start start 2024-08-31T07:19:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.207+0000 7fa2542b9740 1 Processor -- start 2024-08-31T07:19:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.207+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efa4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.207+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 0x55e31efa5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.207+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 0x55e31efb0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.207+0000 7fa2542b9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e31ed5d700 con 0x55e31ebef070 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.207+0000 7fa2542b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e31ee81cf0 con 0x55e31ebc61b0 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.207+0000 7fa2542b9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e31efa7640 con 0x55e31ebce390 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.207+0000 7fa2542b9740 10 client.0 _ll_register_callbacks cb 0x55e31eea7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa247400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 0x55e31efb0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa246a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efa4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa246a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efa4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60114/0 (socket says 172.21.15.32:60114) 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa246a00640 1 -- 172.21.15.32:0/1272729747 learned_addr learned my addr 172.21.15.32:0/1272729747 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa247400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 0x55e31efb0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41590/0 (socket says 172.21.15.32:41590) 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa237e00640 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 0x55e31efa5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa246a00640 1 -- 172.21.15.32:0/1272729747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 msgr2=0x55e31efa5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa246a00640 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 0x55e31efa5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa246a00640 1 -- 172.21.15.32:0/1272729747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 msgr2=0x55e31efb0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa246a00640 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 0x55e31efb0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa246a00640 1 -- 172.21.15.32:0/1272729747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e31eec28b0 con 0x55e31ebc61b0 2024-08-31T07:19:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.208+0000 7fa237e00640 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 0x55e31efa5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.209+0000 7fa246a00640 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efa4b40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa238009d40 tx=0x7fa238004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f58407cf082cfccf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.209+0000 7fa22fe00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.209+0000 7fa22fe00640 1 -- 172.21.15.32:0/1272729747 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23802a060 con 0x55e31ebc61b0 2024-08-31T07:19:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.209+0000 7fa2542b9740 1 -- 172.21.15.32:0/1272729747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa240003e20 con 0x55e31ebc61b0 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.209+0000 7fa22fe00640 1 -- 172.21.15.32:0/1272729747 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fa238003800 con 0x55e31ebc61b0 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.209+0000 7fa22fe00640 1 -- 172.21.15.32:0/1272729747 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa238003a00 con 0x55e31ebc61b0 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.209+0000 7fa2542b9740 10 client.27017 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.209+0000 7fa2542b9740 1 -- 172.21.15.32:0/1272729747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e31efb3740 con 0x55e31ebc61b0 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.209+0000 7fa22fe00640 1 -- 172.21.15.32:0/1272729747 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa240003e20 con 0x55e31ebc61b0 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.210+0000 7fa22fe00640 1 -- 172.21.15.32:0/1272729747 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fa238011440 con 0x55e31ebc61b0 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.210+0000 7fa2542b9740 10 client.27017 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.210+0000 7fa2542b9740 10 client.27017 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.210+0000 7fa2542b9740 20 client.27017 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.210+0000 7fa2542b9740 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e31ebad9f0 0x55e31efb6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.210+0000 7fa2542b9740 4 client.27017 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.210+0000 7fa2542b9740 1 -- 172.21.15.32:0/1272729747 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e31ef0d700 con 0x55e31ebad9f0 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.210+0000 7fa237e00640 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e31ebad9f0 0x55e31efb6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.211+0000 7fa237e00640 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e31ebad9f0 0x55e31efb6ba0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b4b392ba200cad7 server_cookie=c206eda1219d6b0d in_seq=0 out_seq=0 2024-08-31T07:19:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.211+0000 7fa22fe00640 10 client.27017 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:19:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa22fe00640 1 -- 172.21.15.32:0/1272729747 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e31ef0d700 con 0x55e31ebad9f0 2024-08-31T07:19:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa22fe00640 10 client.27017 handle_command_reply: tid=0 2024-08-31T07:19:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 client.27017 shutdown 2024-08-31T07:19:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 -- 172.21.15.32:0/1272729747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 msgr2=0x55e31efa4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efa4b40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa238009d40 tx=0x7fa238004730 comp rx=0 tx=0).stop 2024-08-31T07:19:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 -- 172.21.15.32:0/1272729747 shutdown_connections 2024-08-31T07:19:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e31ebad9f0 0x55e31efb6ba0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 0x55e31efb0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 0x55e31efa5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa22fe00640 0 client.27017 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:19:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 --2- 172.21.15.32:0/1272729747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efa4b40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 -- 172.21.15.32:0/1272729747 >> 172.21.15.32:0/1272729747 conn(0x55e31ebe3460 msgr2=0x55e31ef90ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 -- 172.21.15.32:0/1272729747 shutdown_connections 2024-08-31T07:19:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 1 -- 172.21.15.32:0/1272729747 wait complete. 2024-08-31T07:19:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.212+0000 7fa2542b9740 20 client.27017 trim_cache size 0 max 16384 2024-08-31T07:19:57.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.226+0000 7fa2542b9740 1 Processor -- start 2024-08-31T07:19:57.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.226+0000 7fa2542b9740 1 -- start start 2024-08-31T07:19:57.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 0x55e31ef90ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebce390 0x55e31ef91530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31efa1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa2542b9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481acf50 con 0x55e31ebce390 2024-08-31T07:19:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa2542b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481aeb50 con 0x55e31ebef070 2024-08-31T07:19:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa2542b9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481ad620 con 0x55e31ebc61b0 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa246a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 0x55e31ef90ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa237e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebce390 0x55e31ef91530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa246a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 0x55e31ef90ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54328/0 (socket says 172.21.15.32:54328) 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa246a00640 1 -- 172.21.15.32:0/2084484250 learned_addr learned my addr 172.21.15.32:0/2084484250 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa247400640 1 --2- 172.21.15.32:0/2084484250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31efa1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa246a00640 1 -- 172.21.15.32:0/2084484250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 msgr2=0x55e31efa1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa246a00640 1 --2- 172.21.15.32:0/2084484250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31efa1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa246a00640 1 -- 172.21.15.32:0/2084484250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebce390 msgr2=0x55e31ef91530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa246a00640 1 --2- 172.21.15.32:0/2084484250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebce390 0x55e31ef91530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa246a00640 1 -- 172.21.15.32:0/2084484250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e31efa2e40 con 0x55e31ebc61b0 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.227+0000 7fa247400640 1 --2- 172.21.15.32:0/2084484250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31efa1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.228+0000 7fa246a00640 1 --2- 172.21.15.32:0/2084484250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 0x55e31ef90ff0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa238011860 tx=0x7fa23800fea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4808c3f94ed26176 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.228+0000 7fa236a00640 1 -- 172.21.15.32:0/2084484250 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23803c070 con 0x55e31ebc61b0 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.228+0000 7fa236a00640 1 -- 172.21.15.32:0/2084484250 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa238003800 con 0x55e31ebc61b0 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.228+0000 7fa236a00640 1 -- 172.21.15.32:0/2084484250 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23802f750 con 0x55e31ebc61b0 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.229+0000 7fa2542b9740 1 -- 172.21.15.32:0/2084484250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 msgr2=0x55e31ef90ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.229+0000 7fa2542b9740 1 --2- 172.21.15.32:0/2084484250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 0x55e31ef90ff0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa238011860 tx=0x7fa23800fea0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.229+0000 7fa2542b9740 1 -- 172.21.15.32:0/2084484250 shutdown_connections 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.229+0000 7fa2542b9740 1 --2- 172.21.15.32:0/2084484250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebef070 0x55e31efa1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.229+0000 7fa2542b9740 1 --2- 172.21.15.32:0/2084484250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebce390 0x55e31ef91530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.229+0000 7fa2542b9740 1 --2- 172.21.15.32:0/2084484250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebc61b0 0x55e31ef90ff0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.229+0000 7fa2542b9740 1 -- 172.21.15.32:0/2084484250 >> 172.21.15.32:0/2084484250 conn(0x55e31ebe3460 msgr2=0x55e31efa31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.229+0000 7fa2542b9740 1 -- 172.21.15.32:0/2084484250 shutdown_connections 2024-08-31T07:19:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.229+0000 7fa2542b9740 1 -- 172.21.15.32:0/2084484250 wait complete. 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.229+0000 7fa2542b9740 1 -- start start 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.230+0000 7fa2542b9740 1 Processor -- start 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.230+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efacd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.230+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 0x55e31efad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.230+0000 7fa2542b9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 0x55e31efa7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.230+0000 7fa2542b9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481ae3c0 con 0x55e31ebef070 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.230+0000 7fa2542b9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481adcf0 con 0x55e31ebc61b0 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.230+0000 7fa2542b9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481aaf50 con 0x55e31ebce390 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.230+0000 7fa2542b9740 10 client.0 _ll_register_callbacks cb 0x7fa2380043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.230+0000 7fa246a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efacd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.230+0000 7fa247400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 0x55e31efa7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa246a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efacd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60130/0 (socket says 172.21.15.32:60130) 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa246a00640 1 -- 172.21.15.32:0/3957675313 learned_addr learned my addr 172.21.15.32:0/3957675313 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa247400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 0x55e31efa7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41606/0 (socket says 172.21.15.32:41606) 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa237e00640 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 0x55e31efad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa246a00640 1 -- 172.21.15.32:0/3957675313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 msgr2=0x55e31efad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa246a00640 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 0x55e31efad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa246a00640 1 -- 172.21.15.32:0/3957675313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 msgr2=0x55e31efa7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa246a00640 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 0x55e31efa7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa246a00640 1 -- 172.21.15.32:0/3957675313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e31efa81b0 con 0x55e31ebc61b0 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa246a00640 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efacd20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa238011950 tx=0x7fa2380121f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d964a078d537bd75 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa22fe00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.231+0000 7fa22fe00640 1 -- 172.21.15.32:0/3957675313 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23803c070 con 0x55e31ebc61b0 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.232+0000 7fa2542b9740 1 -- 172.21.15.32:0/3957675313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa238003800 con 0x55e31ebc61b0 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.232+0000 7fa22fe00640 1 -- 172.21.15.32:0/3957675313 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7fa238003ba0 con 0x55e31ebc61b0 2024-08-31T07:19:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.232+0000 7fa22fe00640 1 -- 172.21.15.32:0/3957675313 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa23802a040 con 0x55e31ebc61b0 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.232+0000 7fa2542b9740 10 client.27023 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.232+0000 7fa2542b9740 1 -- 172.21.15.32:0/3957675313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e31f022140 con 0x55e31ebc61b0 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.232+0000 7fa22fe00640 1 -- 172.21.15.32:0/3957675313 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa238003800 con 0x55e31ebc61b0 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.233+0000 7fa22fe00640 1 -- 172.21.15.32:0/3957675313 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fa238004860 con 0x55e31ebc61b0 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.233+0000 7fa2542b9740 10 client.27023 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.233+0000 7fa2542b9740 10 client.27023 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.233+0000 7fa2542b9740 20 client.27023 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.233+0000 7fa2542b9740 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e31eb82da0 0x55e31f0247b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.233+0000 7fa2542b9740 4 client.27023 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.233+0000 7fa2542b9740 1 -- 172.21.15.32:0/3957675313 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e31ef091a0 con 0x55e31eb82da0 2024-08-31T07:19:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.233+0000 7fa237e00640 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e31eb82da0 0x55e31f0247b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.233+0000 7fa237e00640 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e31eb82da0 0x55e31f0247b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a02d6adf8ad3f499 server_cookie=93d097d8f80252e4 in_seq=0 out_seq=0 2024-08-31T07:19:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.233+0000 7fa22fe00640 10 client.27023 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:19:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa22fe00640 1 -- 172.21.15.32:0/3957675313 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e31ef091a0 con 0x55e31eb82da0 2024-08-31T07:19:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa22fe00640 10 client.27023 handle_command_reply: tid=0 2024-08-31T07:19:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 client.27023 shutdown 2024-08-31T07:19:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 -- 172.21.15.32:0/3957675313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 msgr2=0x55e31efacd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efacd20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa238011950 tx=0x7fa2380121f0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 -- 172.21.15.32:0/3957675313 shutdown_connections 2024-08-31T07:19:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e31eb82da0 0x55e31f0247b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e31ebef070 0x55e31efa7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e31ebce390 0x55e31efad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 --2- 172.21.15.32:0/3957675313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31ebc61b0 0x55e31efacd20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa22fe00640 0 client.27023 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:19:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 -- 172.21.15.32:0/3957675313 >> 172.21.15.32:0/3957675313 conn(0x55e31ebe3460 msgr2=0x55e31ef99c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 -- 172.21.15.32:0/3957675313 shutdown_connections 2024-08-31T07:19:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 1 -- 172.21.15.32:0/3957675313 wait complete. 2024-08-31T07:19:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.234+0000 7fa2542b9740 20 client.27023 trim_cache size 0 max 16384 2024-08-31T07:19:57.240 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:19:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 -- 172.21.15.32:0/3783192658 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa2280b4d20 msgr2=0x7fa2280b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa2280b4d20 0x7fa2280b7230 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa238009fd0 tx=0x7fa238014040 comp rx=0 tx=0).stop 2024-08-31T07:19:57.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 -- 172.21.15.32:0/3783192658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa248109090 msgr2=0x7fa24819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa248109090 0x7fa24819a580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa24806b730 tx=0x7fa23000ee50 comp rx=0 tx=0).stop 2024-08-31T07:19:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 -- 172.21.15.32:0/3783192658 shutdown_connections 2024-08-31T07:19:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa2280b4d20 0x7fa2280b7230 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2481099e0 0x7fa24819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa248109090 0x7fa24819a580 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 --2- 172.21.15.32:0/3783192658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa248103750 0x7fa248199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 -- 172.21.15.32:0/3783192658 >> 172.21.15.32:0/3783192658 conn(0x7fa2480ff1e0 msgr2=0x7fa248101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 -- 172.21.15.32:0/3783192658 shutdown_connections 2024-08-31T07:19:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:57.235+0000 7fa247e00640 1 -- 172.21.15.32:0/3783192658 wait complete. 2024-08-31T07:19:57.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.13 MB/sec execute 226 sec latency 10203.400 ms 2024-08-31T07:19:58.273 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:19:58 smithi032 ceph-mon[17252]: pgmap v586: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-08-31T07:19:58.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:19:58.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:19:58 smithi073 ceph-mon[21995]: pgmap v586: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-08-31T07:19:58.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:19:58 smithi084 ceph-mon[22387]: pgmap v586: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-08-31T07:19:58.696 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:19:58.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.11 MB/sec execute 227 sec latency 11203.449 ms 2024-08-31T07:19:59.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.370+0000 7effff200640 1 -- 172.21.15.32:0/3519166010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 msgr2=0x7efff8112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:59.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.370+0000 7effff200640 1 --2- 172.21.15.32:0/3519166010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 0x7efff8112410 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efff4009ca0 tx=0x7efff402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.370+0000 7effff200640 1 -- 172.21.15.32:0/3519166010 shutdown_connections 2024-08-31T07:19:59.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.370+0000 7effff200640 1 --2- 172.21.15.32:0/3519166010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810bbe0 0x7efff8112410 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.370+0000 7effff200640 1 --2- 172.21.15.32:0/3519166010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810b290 0x7efff810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.370+0000 7effff200640 1 --2- 172.21.15.32:0/3519166010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff8105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.370+0000 7effff200640 1 -- 172.21.15.32:0/3519166010 >> 172.21.15.32:0/3519166010 conn(0x7efff8101420 msgr2=0x7efff8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:59.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.370+0000 7effff200640 1 -- 172.21.15.32:0/3519166010 shutdown_connections 2024-08-31T07:19:59.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.370+0000 7effff200640 1 -- 172.21.15.32:0/3519166010 wait complete. 2024-08-31T07:19:59.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.371+0000 7effff200640 1 Processor -- start 2024-08-31T07:19:59.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.371+0000 7effff200640 1 -- start start 2024-08-31T07:19:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7effff200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7effff200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810b290 0x7efff819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7effff200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810bbe0 0x7efff81a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7effff200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efff81158f0 con 0x7efff810b290 2024-08-31T07:19:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7effff200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efff8115770 con 0x7efff810bbe0 2024-08-31T07:19:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7effff200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efff8115a70 con 0x7efff8105950 2024-08-31T07:19:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7efffd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810b290 0x7efff819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7efffde00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7efffde00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff819c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54352/0 (socket says 172.21.15.32:54352) 2024-08-31T07:19:59.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7efffde00640 1 -- 172.21.15.32:0/1667726485 learned_addr learned my addr 172.21.15.32:0/1667726485 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:59.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7efffe800640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810bbe0 0x7efff81a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7efffde00640 1 -- 172.21.15.32:0/1667726485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810bbe0 msgr2=0x7efff81a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:59.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7efffde00640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810bbe0 0x7efff81a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7efffde00640 1 -- 172.21.15.32:0/1667726485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810b290 msgr2=0x7efff819c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:59.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7efffde00640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810b290 0x7efff819c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.372+0000 7efffde00640 1 -- 172.21.15.32:0/1667726485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff81a22f0 con 0x7efff8105950 2024-08-31T07:19:59.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.373+0000 7efffd400640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810b290 0x7efff819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:59.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.373+0000 7efffde00640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff819c380 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7effec00e9f0 tx=0x7effec00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:59.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.373+0000 7efff2a00640 1 -- 172.21.15.32:0/1667726485 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec00cd80 con 0x7efff8105950 2024-08-31T07:19:59.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.373+0000 7effff200640 1 -- 172.21.15.32:0/1667726485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff81a25e0 con 0x7efff8105950 2024-08-31T07:19:59.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.373+0000 7efff2a00640 1 -- 172.21.15.32:0/1667726485 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effec0054d0 con 0x7efff8105950 2024-08-31T07:19:59.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.373+0000 7efff2a00640 1 -- 172.21.15.32:0/1667726485 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec010690 con 0x7efff8105950 2024-08-31T07:19:59.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.373+0000 7effff200640 1 -- 172.21.15.32:0/1667726485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff81a8d40 con 0x7efff8105950 2024-08-31T07:19:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.376+0000 7f0007122740 1 Processor -- start 2024-08-31T07:19:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.376+0000 7f0007122740 1 -- start start 2024-08-31T07:19:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7f0007122740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81da3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7f0007122740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819d51b0 0x556f81da40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7f0007122740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 0x556f81da45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7f0007122740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556f81d1a7b0 con 0x556f819fe070 2024-08-31T07:19:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7f0007122740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556f81d09d40 con 0x556f819dd390 2024-08-31T07:19:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7f0007122740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556f81d0dcf0 con 0x556f819d51b0 2024-08-31T07:19:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7efffe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81da3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7efffe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81da3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41624/0 (socket says 172.21.15.32:41624) 2024-08-31T07:19:59.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7efffe800640 1 -- 172.21.15.32:0/940696044 learned_addr learned my addr 172.21.15.32:0/940696044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:59.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7efffd400640 1 --2- 172.21.15.32:0/940696044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 0x556f81da45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.377+0000 7efff2a00640 1 -- 172.21.15.32:0/1667726485 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7effec004e90 con 0x7efff8105950 2024-08-31T07:19:59.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.378+0000 7efffe800640 1 -- 172.21.15.32:0/940696044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819d51b0 msgr2=0x556f81da40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:59.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.378+0000 7efffe800640 1 --2- 172.21.15.32:0/940696044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819d51b0 0x556f81da40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.378+0000 7efffe800640 1 -- 172.21.15.32:0/940696044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 msgr2=0x556f81da45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:59.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.378+0000 7efffe800640 1 --2- 172.21.15.32:0/940696044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 0x556f81da45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.378+0000 7efffe800640 1 -- 172.21.15.32:0/940696044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f81a69650 con 0x556f819fe070 2024-08-31T07:19:59.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.378+0000 7efffd400640 1 --2- 172.21.15.32:0/940696044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 0x556f81da45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:19:59.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.378+0000 7efffe800640 1 --2- 172.21.15.32:0/940696044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81da3b60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efff4009ca0 tx=0x7efff4009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f50276a11760be server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.378+0000 7effe7000640 1 -- 172.21.15.32:0/940696044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4031cf0 con 0x556f819fe070 2024-08-31T07:19:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.378+0000 7effe7000640 1 -- 172.21.15.32:0/940696044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4036030 con 0x556f819fe070 2024-08-31T07:19:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.378+0000 7effe7000640 1 -- 172.21.15.32:0/940696044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4004240 con 0x556f819fe070 2024-08-31T07:19:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7f0007122740 1 -- 172.21.15.32:0/940696044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 msgr2=0x556f81da3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7f0007122740 1 --2- 172.21.15.32:0/940696044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81da3b60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efff4009ca0 tx=0x7efff4009430 comp rx=0 tx=0).stop 2024-08-31T07:19:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7f0007122740 1 -- 172.21.15.32:0/940696044 shutdown_connections 2024-08-31T07:19:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7f0007122740 1 --2- 172.21.15.32:0/940696044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 0x556f81da45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7f0007122740 1 --2- 172.21.15.32:0/940696044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819d51b0 0x556f81da40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7f0007122740 1 --2- 172.21.15.32:0/940696044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81da3b60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7f0007122740 1 -- 172.21.15.32:0/940696044 >> 172.21.15.32:0/940696044 conn(0x556f819f2460 msgr2=0x556f81da1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7f0007122740 1 -- 172.21.15.32:0/940696044 shutdown_connections 2024-08-31T07:19:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7efff2a00640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe00b4c50 0x7effe00b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7efff2a00640 1 -- 172.21.15.32:0/1667726485 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7effec0e59a0 con 0x7efff8105950 2024-08-31T07:19:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7f0007122740 1 -- 172.21.15.32:0/940696044 wait complete. 2024-08-31T07:19:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.380+0000 7efffd400640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe00b4c50 0x7effe00b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.381+0000 7f0007122740 1 -- start start 2024-08-31T07:19:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.381+0000 7efffd400640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe00b4c50 0x7effe00b7160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efff8104b20 tx=0x7effe8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.381+0000 7f0007122740 1 Processor -- start 2024-08-31T07:19:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7f0007122740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81daf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7f0007122740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819dd390 0x556f81dafd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7f0007122740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81dbb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7f0007122740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556f81b6c700 con 0x556f819fe070 2024-08-31T07:19:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7f0007122740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556f81c90cf0 con 0x556f819d51b0 2024-08-31T07:19:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7f0007122740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556f81db2160 con 0x556f819dd390 2024-08-31T07:19:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7f0007122740 10 client.0 _ll_register_callbacks cb 0x556f81cb6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81daf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffd400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819dd390 0x556f81dafd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81daf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60180/0 (socket says 172.21.15.32:60180) 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffde00640 1 -- 172.21.15.32:0/1794735082 learned_addr learned my addr 172.21.15.32:0/1794735082 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffe800640 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81dbb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffde00640 1 -- 172.21.15.32:0/1794735082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819dd390 msgr2=0x556f81dafd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffde00640 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819dd390 0x556f81dafd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffde00640 1 -- 172.21.15.32:0/1794735082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 msgr2=0x556f81dbb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffde00640 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81dbb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffde00640 1 -- 172.21.15.32:0/1794735082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f81cd18b0 con 0x556f819d51b0 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.382+0000 7efffe800640 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81dbb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.383+0000 7efffde00640 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81daf790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7effec0a92f0 tx=0x7effec0ab8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa0ef122cdd06d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.383+0000 7effdb400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.383+0000 7effdb400640 1 -- 172.21.15.32:0/1794735082 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0e3570 con 0x556f819d51b0 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.383+0000 7f0007122740 1 -- 172.21.15.32:0/1794735082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff4036030 con 0x556f819d51b0 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.383+0000 7effdb400640 1 -- 172.21.15.32:0/1794735082 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7effec0b4f10 con 0x556f819d51b0 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.383+0000 7effdb400640 1 -- 172.21.15.32:0/1794735082 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7effec0ac080 con 0x556f819d51b0 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.383+0000 7f0007122740 10 client.27035 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.383+0000 7f0007122740 1 -- 172.21.15.32:0/1794735082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556f81dbe250 con 0x556f819d51b0 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.383+0000 7effdb400640 1 -- 172.21.15.32:0/1794735082 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4036030 con 0x556f819d51b0 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.384+0000 7effdb400640 1 -- 172.21.15.32:0/1794735082 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7effec0a7a10 con 0x556f819d51b0 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.384+0000 7f0007122740 10 client.27035 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.384+0000 7f0007122740 10 client.27035 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:19:59.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.384+0000 7f0007122740 20 client.27035 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.384+0000 7f0007122740 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556f819bc9f0 0x556f81dc1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.384+0000 7f0007122740 4 client.27035 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:19:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.384+0000 7f0007122740 1 -- 172.21.15.32:0/1794735082 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f81d1c700 con 0x556f819bc9f0 2024-08-31T07:19:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.384+0000 7efffd400640 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556f819bc9f0 0x556f81dc1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.385+0000 7efffd400640 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556f819bc9f0 0x556f81dc1770 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5bd5777747d813d7 server_cookie=482f5a2518d24e56 in_seq=0 out_seq=0 2024-08-31T07:19:59.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.385+0000 7effdb400640 10 client.27035 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:19:59.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7effdb400640 1 -- 172.21.15.32:0/1794735082 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f81d1c700 con 0x556f819bc9f0 2024-08-31T07:19:59.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7effdb400640 10 client.27035 handle_command_reply: tid=0 2024-08-31T07:19:59.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 client.27035 shutdown 2024-08-31T07:19:59.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 -- 172.21.15.32:0/1794735082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 msgr2=0x556f81daf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81daf790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7effec0a92f0 tx=0x7effec0ab8b0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 -- 172.21.15.32:0/1794735082 shutdown_connections 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556f819bc9f0 0x556f81dc1770 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81dbb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819dd390 0x556f81dafd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 --2- 172.21.15.32:0/1794735082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81daf790 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7effdb400640 0 client.27035 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 -- 172.21.15.32:0/1794735082 >> 172.21.15.32:0/1794735082 conn(0x556f819f2460 msgr2=0x556f81da6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 -- 172.21.15.32:0/1794735082 shutdown_connections 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 1 -- 172.21.15.32:0/1794735082 wait complete. 2024-08-31T07:19:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.386+0000 7f0007122740 20 client.27035 trim_cache size 0 max 16384 2024-08-31T07:19:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.400+0000 7f0007122740 1 Processor -- start 2024-08-31T07:19:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.400+0000 7f0007122740 1 -- start start 2024-08-31T07:19:59.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7f0007122740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81da9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7f0007122740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819dd390 0x556f81dbb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7f0007122740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81dac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7f0007122740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b1360 con 0x556f819fe070 2024-08-31T07:19:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7f0007122740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b2f60 con 0x556f819d51b0 2024-08-31T07:19:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7f0007122740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b1a30 con 0x556f819dd390 2024-08-31T07:19:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7efffe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81dac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7efffde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81da9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7efffde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81da9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60192/0 (socket says 172.21.15.32:60192) 2024-08-31T07:19:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7efffde00640 1 -- 172.21.15.32:0/2815060871 learned_addr learned my addr 172.21.15.32:0/2815060871 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.401+0000 7efffd400640 1 --2- 172.21.15.32:0/2815060871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819dd390 0x556f81dbb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.402+0000 7efffde00640 1 -- 172.21.15.32:0/2815060871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819dd390 msgr2=0x556f81dbb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.402+0000 7efffde00640 1 --2- 172.21.15.32:0/2815060871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819dd390 0x556f81dbb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.402+0000 7efffde00640 1 -- 172.21.15.32:0/2815060871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 msgr2=0x556f81dac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.402+0000 7efffde00640 1 --2- 172.21.15.32:0/2815060871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81dac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.402+0000 7efffde00640 1 -- 172.21.15.32:0/2815060871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f81da5e50 con 0x556f819d51b0 2024-08-31T07:19:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.402+0000 7efffde00640 1 --2- 172.21.15.32:0/2815060871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81da9380 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7effec0ac240 tx=0x7effec004a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=892627182b0aed29 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:59.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.402+0000 7effe6600640 1 -- 172.21.15.32:0/2815060871 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0cb070 con 0x556f819d51b0 2024-08-31T07:19:59.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.402+0000 7effe6600640 1 -- 172.21.15.32:0/2815060871 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effec005680 con 0x556f819d51b0 2024-08-31T07:19:59.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.402+0000 7effe6600640 1 -- 172.21.15.32:0/2815060871 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0970d0 con 0x556f819d51b0 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.403+0000 7f0007122740 1 -- 172.21.15.32:0/2815060871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 msgr2=0x556f81da9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.403+0000 7f0007122740 1 --2- 172.21.15.32:0/2815060871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81da9380 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7effec0ac240 tx=0x7effec004a00 comp rx=0 tx=0).stop 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.403+0000 7f0007122740 1 -- 172.21.15.32:0/2815060871 shutdown_connections 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.403+0000 7f0007122740 1 --2- 172.21.15.32:0/2815060871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819fe070 0x556f81dac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.403+0000 7f0007122740 1 --2- 172.21.15.32:0/2815060871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819dd390 0x556f81dbb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.403+0000 7f0007122740 1 --2- 172.21.15.32:0/2815060871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819d51b0 0x556f81da9380 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.403+0000 7f0007122740 1 -- 172.21.15.32:0/2815060871 >> 172.21.15.32:0/2815060871 conn(0x556f819f2460 msgr2=0x556f81da8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.403+0000 7f0007122740 1 -- 172.21.15.32:0/2815060871 shutdown_connections 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.403+0000 7f0007122740 1 -- 172.21.15.32:0/2815060871 wait complete. 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.404+0000 7f0007122740 1 -- start start 2024-08-31T07:19:59.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.404+0000 7f0007122740 1 Processor -- start 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.404+0000 7f0007122740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819d51b0 0x556f81db7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.404+0000 7f0007122740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 0x556f81db3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.404+0000 7f0007122740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819fe070 0x556f81db44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7f0007122740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b27d0 con 0x556f819d51b0 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7f0007122740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b2100 con 0x556f819dd390 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7f0007122740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efff81af360 con 0x556f819fe070 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7f0007122740 10 client.0 _ll_register_callbacks cb 0x7effec0abec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffe800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819fe070 0x556f81db44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819d51b0 0x556f81db7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffe800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819fe070 0x556f81db44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54374/0 (socket says 172.21.15.32:54374) 2024-08-31T07:19:59.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffe800640 1 -- 172.21.15.32:0/3847137844 learned_addr learned my addr 172.21.15.32:0/3847137844 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:19:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819d51b0 0x556f81db7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41654/0 (socket says 172.21.15.32:41654) 2024-08-31T07:19:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffd400640 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 0x556f81db3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffe800640 1 -- 172.21.15.32:0/3847137844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 msgr2=0x556f81db3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffe800640 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 0x556f81db3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffe800640 1 -- 172.21.15.32:0/3847137844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819d51b0 msgr2=0x556f81db7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:19:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffe800640 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819d51b0 0x556f81db7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.405+0000 7efffe800640 1 -- 172.21.15.32:0/3847137844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f81db4b30 con 0x556f819fe070 2024-08-31T07:19:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.406+0000 7efffe800640 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819fe070 0x556f81db44c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efff4004af0 tx=0x7efff402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a7ee96b9a2468f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:19:59.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.406+0000 7effdb400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:19:59.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.406+0000 7effdb400640 1 -- 172.21.15.32:0/3847137844 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff404a040 con 0x556f819fe070 2024-08-31T07:19:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.406+0000 7effdb400640 1 -- 172.21.15.32:0/3847137844 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7efff4031860 con 0x556f819fe070 2024-08-31T07:19:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.406+0000 7f0007122740 1 -- 172.21.15.32:0/3847137844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effec005680 con 0x556f819fe070 2024-08-31T07:19:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.406+0000 7effdb400640 1 -- 172.21.15.32:0/3847137844 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efff4003970 con 0x556f819fe070 2024-08-31T07:19:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.406+0000 7f0007122740 10 client.27037 fetch_fsmap learned FSMap version 35 2024-08-31T07:19:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.406+0000 7f0007122740 1 -- 172.21.15.32:0/3847137844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556f81e2cf50 con 0x556f819fe070 2024-08-31T07:19:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.407+0000 7effdb400640 1 -- 172.21.15.32:0/3847137844 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effec005680 con 0x556f819fe070 2024-08-31T07:19:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.407+0000 7effdb400640 1 -- 172.21.15.32:0/3847137844 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7efff4043070 con 0x556f819fe070 2024-08-31T07:19:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.407+0000 7f0007122740 10 client.27037 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:19:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.407+0000 7f0007122740 10 client.27037 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.407+0000 7f0007122740 20 client.27037 populate_metadata read hostname 'smithi032' 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.407+0000 7f0007122740 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556f81991da0 0x556f81e2f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.407+0000 7f0007122740 4 client.27037 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.407+0000 7f0007122740 1 -- 172.21.15.32:0/3847137844 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f81d181a0 con 0x556f81991da0 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.407+0000 7efffde00640 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556f81991da0 0x556f81e2f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7efffde00640 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556f81991da0 0x556f81e2f4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=40ef5004bb20b801 server_cookie=921b7d8e4d37abf in_seq=0 out_seq=0 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7effdb400640 10 client.27037 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7effdb400640 1 -- 172.21.15.32:0/3847137844 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f81d181a0 con 0x556f81991da0 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7effdb400640 10 client.27037 handle_command_reply: tid=0 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7f0007122740 1 client.27037 shutdown 2024-08-31T07:19:59.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7f0007122740 1 -- 172.21.15.32:0/3847137844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819fe070 msgr2=0x556f81db44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7f0007122740 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819fe070 0x556f81db44c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efff4004af0 tx=0x7efff402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7f0007122740 1 -- 172.21.15.32:0/3847137844 shutdown_connections 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7f0007122740 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556f81991da0 0x556f81e2f4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7f0007122740 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f819fe070 0x556f81db44c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7f0007122740 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f819dd390 0x556f81db3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7f0007122740 1 --2- 172.21.15.32:0/3847137844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f819d51b0 0x556f81db7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7effdb400640 0 client.27037 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.408+0000 7f0007122740 1 -- 172.21.15.32:0/3847137844 >> 172.21.15.32:0/3847137844 conn(0x556f819f2460 msgr2=0x556f81db0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.409+0000 7f0007122740 1 -- 172.21.15.32:0/3847137844 shutdown_connections 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.409+0000 7f0007122740 1 -- 172.21.15.32:0/3847137844 wait complete. 2024-08-31T07:19:59.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.409+0000 7f0007122740 20 client.27037 trim_cache size 0 max 16384 2024-08-31T07:19:59.415 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:19:59.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.409+0000 7effff200640 1 -- 172.21.15.32:0/1667726485 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe00b4c50 msgr2=0x7effe00b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:59.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.409+0000 7effff200640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe00b4c50 0x7effe00b7160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efff8104b20 tx=0x7effe8005f90 comp rx=0 tx=0).stop 2024-08-31T07:19:59.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.409+0000 7effff200640 1 -- 172.21.15.32:0/1667726485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 msgr2=0x7efff819c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:19:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.409+0000 7effff200640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff819c380 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7effec00e9f0 tx=0x7effec00edf0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.410+0000 7effff200640 1 -- 172.21.15.32:0/1667726485 shutdown_connections 2024-08-31T07:19:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.410+0000 7effff200640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe00b4c50 0x7effe00b7160 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.410+0000 7effff200640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efff810bbe0 0x7efff81a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.410+0000 7effff200640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efff810b290 0x7efff819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.410+0000 7effff200640 1 --2- 172.21.15.32:0/1667726485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efff8105950 0x7efff819c380 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:19:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.410+0000 7effff200640 1 -- 172.21.15.32:0/1667726485 >> 172.21.15.32:0/1667726485 conn(0x7efff8101420 msgr2=0x7efff810a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:19:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.410+0000 7effff200640 1 -- 172.21.15.32:0/1667726485 shutdown_connections 2024-08-31T07:19:59.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:19:59.410+0000 7effff200640 1 -- 172.21.15.32:0/1667726485 wait complete. 2024-08-31T07:19:59.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.10 MB/sec execute 228 sec latency 12203.513 ms 2024-08-31T07:20:00.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:00 smithi032 ceph-mon[17252]: pgmap v587: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 89 op/s 2024-08-31T07:20:00.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:00 smithi032 ceph-mon[17252]: overall HEALTH_OK 2024-08-31T07:20:00.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:20:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:00 smithi073 ceph-mon[21995]: pgmap v587: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 89 op/s 2024-08-31T07:20:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:00 smithi073 ceph-mon[21995]: overall HEALTH_OK 2024-08-31T07:20:00.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:00 smithi084 ceph-mon[22387]: pgmap v587: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.3 MiB/s wr, 89 op/s 2024-08-31T07:20:00.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:00 smithi084 ceph-mon[22387]: overall HEALTH_OK 2024-08-31T07:20:00.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.09 MB/sec execute 229 sec latency 13203.554 ms 2024-08-31T07:20:00.761 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:01.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.07 MB/sec execute 230 sec latency 14203.596 ms 2024-08-31T07:20:02.256 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:02 smithi032 ceph-mon[17252]: pgmap v588: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-08-31T07:20:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:02 smithi073 ceph-mon[21995]: pgmap v588: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-08-31T07:20:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.438+0000 7f0d01800640 1 -- 172.21.15.32:0/3504701338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc103900 msgr2=0x7f0cfc103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.438+0000 7f0d01800640 1 --2- 172.21.15.32:0/3504701338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc103900 0x7f0cfc103d10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009900 tx=0x7f0cec02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.438+0000 7f0d01800640 1 -- 172.21.15.32:0/3504701338 shutdown_connections 2024-08-31T07:20:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.438+0000 7f0d01800640 1 --2- 172.21.15.32:0/3504701338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc109b90 0x7f0cfc1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.438+0000 7f0d01800640 1 --2- 172.21.15.32:0/3504701338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cfc109240 0x7f0cfc109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.438+0000 7f0d01800640 1 --2- 172.21.15.32:0/3504701338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc103900 0x7f0cfc103d10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.438+0000 7f0d01800640 1 -- 172.21.15.32:0/3504701338 >> 172.21.15.32:0/3504701338 conn(0x7f0cfc07b5f0 msgr2=0x7f0cfc07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.438+0000 7f0d01800640 1 -- 172.21.15.32:0/3504701338 shutdown_connections 2024-08-31T07:20:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.438+0000 7f0d01800640 1 -- 172.21.15.32:0/3504701338 wait complete. 2024-08-31T07:20:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.439+0000 7f0d01800640 1 Processor -- start 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0d01800640 1 -- start start 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0d01800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc103900 0x7f0cfc19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0d01800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cfc109240 0x7f0cfc19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0d01800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc109b90 0x7f0cfc19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0d01800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc113850 con 0x7f0cfc109b90 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0d01800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1136d0 con 0x7f0cfc103900 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0d01800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1139d0 con 0x7f0cfc109240 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0cfb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc109b90 0x7f0cfc19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0cfb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc109b90 0x7f0cfc19fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41666/0 (socket says 172.21.15.32:41666) 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0cfaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc103900 0x7f0cfc19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0cfb400640 1 -- 172.21.15.32:0/3070983406 learned_addr learned my addr 172.21.15.32:0/3070983406 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.440+0000 7f0cfa000640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cfc109240 0x7f0cfc19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.441+0000 7f0cfa000640 1 -- 172.21.15.32:0/3070983406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc103900 msgr2=0x7f0cfc19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.441+0000 7f0cfa000640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc103900 0x7f0cfc19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.441+0000 7f0cfa000640 1 -- 172.21.15.32:0/3070983406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc109b90 msgr2=0x7f0cfc19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.441+0000 7f0cfa000640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc109b90 0x7f0cfc19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.441+0000 7f0cfa000640 1 -- 172.21.15.32:0/3070983406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc1a0230 con 0x7f0cfc109240 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.441+0000 7f0cfb400640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc109b90 0x7f0cfc19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.441+0000 7f0cfa000640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cfc109240 0x7f0cfc19a860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc06c790 tx=0x7f0ce800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.441+0000 7f0cf3400640 1 -- 172.21.15.32:0/3070983406 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800cc40 con 0x7f0cfc109240 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.441+0000 7f0d01800640 1 -- 172.21.15.32:0/3070983406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc1a0520 con 0x7f0cfc109240 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.442+0000 7f0cf3400640 1 -- 172.21.15.32:0/3070983406 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce800cde0 con 0x7f0cfc109240 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.442+0000 7f0cf3400640 1 -- 172.21.15.32:0/3070983406 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8004a00 con 0x7f0cfc109240 2024-08-31T07:20:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.442+0000 7f0d01800640 1 -- 172.21.15.32:0/3070983406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc1a6c80 con 0x7f0cfc109240 2024-08-31T07:20:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0d083d0740 1 Processor -- start 2024-08-31T07:20:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0d083d0740 1 -- start start 2024-08-31T07:20:02.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 0x56209dbeddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d827390 0x56209dbee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0d083d0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56209db647b0 con 0x56209d848070 2024-08-31T07:20:02.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0d083d0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56209db53d40 con 0x56209d827390 2024-08-31T07:20:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0d083d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56209db57cf0 con 0x56209d81f1b0 2024-08-31T07:20:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0cfb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0cfb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbed8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41678/0 (socket says 172.21.15.32:41678) 2024-08-31T07:20:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0cfaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 0x56209dbeddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.445+0000 7f0cfb400640 1 -- 172.21.15.32:0/677094378 learned_addr learned my addr 172.21.15.32:0/677094378 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:02.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.446+0000 7f0cfb400640 1 -- 172.21.15.32:0/677094378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 msgr2=0x56209dbeddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:02.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.446+0000 7f0cfb400640 1 --2- 172.21.15.32:0/677094378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 0x56209dbeddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.446+0000 7f0cfb400640 1 -- 172.21.15.32:0/677094378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d827390 msgr2=0x56209dbee330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:20:02.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.446+0000 7f0cfb400640 1 --2- 172.21.15.32:0/677094378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d827390 0x56209dbee330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.446+0000 7f0cfb400640 1 -- 172.21.15.32:0/677094378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209d8b3650 con 0x56209d848070 2024-08-31T07:20:02.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.446+0000 7f0cf3400640 1 -- 172.21.15.32:0/3070983406 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0ce8004ba0 con 0x7f0cfc109240 2024-08-31T07:20:02.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.446+0000 7f0cfb400640 1 --2- 172.21.15.32:0/677094378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbed8b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400e830 tx=0x7f0cf400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6532023bd7d8536 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:02.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.447+0000 7f0ce7e00640 1 -- 172.21.15.32:0/677094378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4004d20 con 0x56209d848070 2024-08-31T07:20:02.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.447+0000 7f0ce7e00640 1 -- 172.21.15.32:0/677094378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf4011e50 con 0x56209d848070 2024-08-31T07:20:02.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.447+0000 7f0ce7e00640 1 -- 172.21.15.32:0/677094378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400f3c0 con 0x56209d848070 2024-08-31T07:20:02.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.447+0000 7f0d083d0740 1 -- 172.21.15.32:0/677094378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 msgr2=0x56209dbed8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:02.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.447+0000 7f0d083d0740 1 --2- 172.21.15.32:0/677094378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbed8b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400e830 tx=0x7f0cf400ecf0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0d083d0740 1 -- 172.21.15.32:0/677094378 shutdown_connections 2024-08-31T07:20:02.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0d083d0740 1 --2- 172.21.15.32:0/677094378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d827390 0x56209dbee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0d083d0740 1 --2- 172.21.15.32:0/677094378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 0x56209dbeddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0d083d0740 1 --2- 172.21.15.32:0/677094378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbed8b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0d083d0740 1 -- 172.21.15.32:0/677094378 >> 172.21.15.32:0/677094378 conn(0x56209d83c460 msgr2=0x56209dbeb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:02.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0d083d0740 1 -- 172.21.15.32:0/677094378 shutdown_connections 2024-08-31T07:20:02.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0d083d0740 1 -- 172.21.15.32:0/677094378 wait complete. 2024-08-31T07:20:02.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0cf3400640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ce00b4bb0 0x7f0ce00b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0d083d0740 1 -- start start 2024-08-31T07:20:02.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0cf3400640 1 -- 172.21.15.32:0/3070983406 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f0ce80e5440 con 0x7f0cfc109240 2024-08-31T07:20:02.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.448+0000 7f0cfb400640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ce00b4bb0 0x7f0ce00b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.449+0000 7f0d083d0740 1 Processor -- start 2024-08-31T07:20:02.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.449+0000 7f0cfb400640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ce00b4bb0 0x7f0ce00b70c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4005650 tx=0x7f0cf401d040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:02.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.449+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 0x56209dbf9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.449+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 0x56209dbf9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.449+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d848070 0x56209dbff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.449+0000 7f0d083d0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56209d9b6700 con 0x56209d827390 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.449+0000 7f0d083d0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56209dadacf0 con 0x56209d81f1b0 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.449+0000 7f0d083d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56209dbfc090 con 0x56209d848070 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.449+0000 7f0d083d0740 10 client.0 _ll_register_callbacks cb 0x56209db00470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfa000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 0x56209dbf9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfa000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 0x56209dbf9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41688/0 (socket says 172.21.15.32:41688) 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfa000640 1 -- 172.21.15.32:0/764285924 learned_addr learned my addr 172.21.15.32:0/764285924 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfaa00640 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 0x56209dbf9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfb400640 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d848070 0x56209dbff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfaa00640 1 -- 172.21.15.32:0/764285924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d848070 msgr2=0x56209dbff350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:02.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfaa00640 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d848070 0x56209dbff350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfaa00640 1 -- 172.21.15.32:0/764285924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 msgr2=0x56209dbf9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfaa00640 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 0x56209dbf9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfaa00640 1 -- 172.21.15.32:0/764285924 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209db1b8b0 con 0x56209d81f1b0 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.450+0000 7f0cfa000640 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 0x56209dbf9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.451+0000 7f0cfaa00640 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 0x56209dbf9520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009230 tx=0x7f0cec005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbf7c0753d1ce287 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.451+0000 7f0ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.451+0000 7f0ce4c00640 1 -- 172.21.15.32:0/764285924 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec035040 con 0x56209d81f1b0 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.451+0000 7f0d083d0740 1 -- 172.21.15.32:0/764285924 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf4011e50 con 0x56209d81f1b0 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.451+0000 7f0ce4c00640 1 -- 172.21.15.32:0/764285924 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f0cec0317e0 con 0x56209d81f1b0 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.451+0000 7f0ce4c00640 1 -- 172.21.15.32:0/764285924 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0cec004040 con 0x56209d81f1b0 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.451+0000 7f0d083d0740 10 client.27050 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.451+0000 7f0d083d0740 1 -- 172.21.15.32:0/764285924 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56209dc10990 con 0x56209d81f1b0 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.452+0000 7f0ce4c00640 1 -- 172.21.15.32:0/764285924 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf4011e50 con 0x56209d81f1b0 2024-08-31T07:20:02.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.452+0000 7f0ce4c00640 1 -- 172.21.15.32:0/764285924 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f0cec004870 con 0x56209d81f1b0 2024-08-31T07:20:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.452+0000 7f0d083d0740 10 client.27050 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.452+0000 7f0d083d0740 10 client.27050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:20:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.452+0000 7f0d083d0740 20 client.27050 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.452+0000 7f0d083d0740 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56209d8069f0 0x56209dc13db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.452+0000 7f0d083d0740 4 client.27050 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.452+0000 7f0d083d0740 1 -- 172.21.15.32:0/764285924 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56209db66700 con 0x56209d8069f0 2024-08-31T07:20:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.452+0000 7f0cfa000640 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56209d8069f0 0x56209dc13db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.453+0000 7f0cfa000640 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56209d8069f0 0x56209dc13db0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0921d4c6f3db1fc server_cookie=437cfbea770b1d1c in_seq=0 out_seq=0 2024-08-31T07:20:02.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.453+0000 7f0ce4c00640 10 client.27050 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:20:02.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0ce4c00640 1 -- 172.21.15.32:0/764285924 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56209db66700 con 0x56209d8069f0 2024-08-31T07:20:02.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0ce4c00640 10 client.27050 handle_command_reply: tid=0 2024-08-31T07:20:02.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 client.27050 shutdown 2024-08-31T07:20:02.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 -- 172.21.15.32:0/764285924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 msgr2=0x56209dbf9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 0x56209dbf9520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009230 tx=0x7f0cec005f10 comp rx=0 tx=0).stop 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 -- 172.21.15.32:0/764285924 shutdown_connections 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56209d8069f0 0x56209dc13db0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d848070 0x56209dbff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 0x56209dbf9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 --2- 172.21.15.32:0/764285924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 0x56209dbf9520 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0ce4c00640 0 client.27050 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 -- 172.21.15.32:0/764285924 >> 172.21.15.32:0/764285924 conn(0x56209d83c460 msgr2=0x56209dbf0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 -- 172.21.15.32:0/764285924 shutdown_connections 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.454+0000 7f0d083d0740 1 -- 172.21.15.32:0/764285924 wait complete. 2024-08-31T07:20:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.455+0000 7f0d083d0740 20 client.27050 trim_cache size 0 max 16384 2024-08-31T07:20:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.469+0000 7f0d083d0740 1 Processor -- start 2024-08-31T07:20:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.469+0000 7f0d083d0740 1 -- start start 2024-08-31T07:20:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.469+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 0x56209dbff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.469+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d827390 0x56209dbf7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.469+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbf8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.469+0000 7f0d083d0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1ad140 con 0x56209d848070 2024-08-31T07:20:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.469+0000 7f0d083d0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1aed40 con 0x56209d81f1b0 2024-08-31T07:20:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.469+0000 7f0d083d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1ad810 con 0x56209d827390 2024-08-31T07:20:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbf8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbf8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41692/0 (socket says 172.21.15.32:41692) 2024-08-31T07:20:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfb400640 1 -- 172.21.15.32:0/3310324861 learned_addr learned my addr 172.21.15.32:0/3310324861 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfaa00640 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 0x56209dbff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfa000640 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d827390 0x56209dbf7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfa000640 1 -- 172.21.15.32:0/3310324861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 msgr2=0x56209dbff2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfa000640 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 0x56209dbff2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfa000640 1 -- 172.21.15.32:0/3310324861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 msgr2=0x56209dbf8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfa000640 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbf8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfa000640 1 -- 172.21.15.32:0/3310324861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209dc14120 con 0x56209d827390 2024-08-31T07:20:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfaa00640 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 0x56209dbff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfb400640 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbf8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.470+0000 7f0cfa000640 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d827390 0x56209dbf7bc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80abb10 tx=0x7f0ce8093040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=152e27b03da721e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0ce7400640 1 -- 172.21.15.32:0/3310324861 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80ac280 con 0x56209d827390 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0ce7400640 1 -- 172.21.15.32:0/3310324861 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce80abb80 con 0x56209d827390 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0ce7400640 1 -- 172.21.15.32:0/3310324861 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80abde0 con 0x56209d827390 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0d083d0740 1 -- 172.21.15.32:0/3310324861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d827390 msgr2=0x56209dbf7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0d083d0740 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d827390 0x56209dbf7bc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80abb10 tx=0x7f0ce8093040 comp rx=0 tx=0).stop 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0d083d0740 1 -- 172.21.15.32:0/3310324861 shutdown_connections 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0d083d0740 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d848070 0x56209dbf8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0d083d0740 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d827390 0x56209dbf7bc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0d083d0740 1 --2- 172.21.15.32:0/3310324861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d81f1b0 0x56209dbff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0d083d0740 1 -- 172.21.15.32:0/3310324861 >> 172.21.15.32:0/3310324861 conn(0x56209d83c460 msgr2=0x56209dbf4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.471+0000 7f0d083d0740 1 -- 172.21.15.32:0/3310324861 shutdown_connections 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.472+0000 7f0d083d0740 1 -- 172.21.15.32:0/3310324861 wait complete. 2024-08-31T07:20:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.472+0000 7f0d083d0740 1 -- start start 2024-08-31T07:20:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.472+0000 7f0d083d0740 1 Processor -- start 2024-08-31T07:20:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 0x56209dc0a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 0x56209dc0a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0d083d0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d848070 0x56209dc7d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0d083d0740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1ae5b0 con 0x56209d827390 2024-08-31T07:20:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0d083d0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1adee0 con 0x56209d848070 2024-08-31T07:20:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0d083d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1ab140 con 0x56209d81f1b0 2024-08-31T07:20:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0d083d0740 10 client.0 _ll_register_callbacks cb 0x7f0cec032c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0cfaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 0x56209dc0a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0cfaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 0x56209dc0a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54442/0 (socket says 172.21.15.32:54442) 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0cfa000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 0x56209dc0a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0cfaa00640 1 -- 172.21.15.32:0/2008304491 learned_addr learned my addr 172.21.15.32:0/2008304491 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0cfb400640 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d848070 0x56209dc7d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0cfaa00640 1 -- 172.21.15.32:0/2008304491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d848070 msgr2=0x56209dc7d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.473+0000 7f0cfaa00640 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d848070 0x56209dc7d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0cfaa00640 1 -- 172.21.15.32:0/2008304491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 msgr2=0x56209dc0a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0cfaa00640 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 0x56209dc0a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0cfaa00640 1 -- 172.21.15.32:0/2008304491 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209dc0afe0 con 0x56209d81f1b0 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0cfaa00640 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 0x56209dc0a270 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0092a0 tx=0x7f0cec0317f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1889bee79482003d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0ce4c00640 1 -- 172.21.15.32:0/2008304491 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec035040 con 0x56209d81f1b0 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0d083d0740 1 -- 172.21.15.32:0/2008304491 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce80abb80 con 0x56209d81f1b0 2024-08-31T07:20:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0ce4c00640 1 -- 172.21.15.32:0/2008304491 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7f0cec0046c0 con 0x56209d81f1b0 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0ce4c00640 1 -- 172.21.15.32:0/2008304491 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0cec060050 con 0x56209d81f1b0 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0d083d0740 10 client.27055 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.474+0000 7f0d083d0740 1 -- 172.21.15.32:0/2008304491 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56209dc7ec70 con 0x56209d81f1b0 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.475+0000 7f0ce4c00640 1 -- 172.21.15.32:0/2008304491 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce80abb80 con 0x56209d81f1b0 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.475+0000 7f0ce4c00640 1 -- 172.21.15.32:0/2008304491 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f0cec0035f0 con 0x56209d81f1b0 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.475+0000 7f0d083d0740 10 client.27055 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.475+0000 7f0d083d0740 10 client.27055 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.475+0000 7f0d083d0740 20 client.27055 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.475+0000 7f0d083d0740 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56209d7dbda0 0x56209dc81460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.475+0000 7f0d083d0740 4 client.27055 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.475+0000 7f0d083d0740 1 -- 172.21.15.32:0/2008304491 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56209db621a0 con 0x56209d7dbda0 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.476+0000 7f0cfa000640 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56209d7dbda0 0x56209dc81460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.476+0000 7f0cfa000640 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56209d7dbda0 0x56209dc81460 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8a4f74d2a3fa407 server_cookie=2742863b63ebeb45 in_seq=0 out_seq=0 2024-08-31T07:20:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.476+0000 7f0ce4c00640 10 client.27055 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.476+0000 7f0ce4c00640 1 -- 172.21.15.32:0/2008304491 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56209db621a0 con 0x56209d7dbda0 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.476+0000 7f0ce4c00640 10 client.27055 handle_command_reply: tid=0 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.476+0000 7f0d083d0740 1 client.27055 shutdown 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.476+0000 7f0d083d0740 1 -- 172.21.15.32:0/2008304491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 msgr2=0x56209dc0a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.476+0000 7f0d083d0740 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 0x56209dc0a270 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0092a0 tx=0x7f0cec0317f0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.477+0000 7f0d083d0740 1 -- 172.21.15.32:0/2008304491 shutdown_connections 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.477+0000 7f0d083d0740 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56209d7dbda0 0x56209dc81460 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.477+0000 7f0d083d0740 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56209d848070 0x56209dc7d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.477+0000 7f0d083d0740 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56209d827390 0x56209dc0a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.477+0000 7f0d083d0740 1 --2- 172.21.15.32:0/2008304491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56209d81f1b0 0x56209dc0a270 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.477+0000 7f0ce4c00640 0 client.27055 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.477+0000 7f0d083d0740 1 -- 172.21.15.32:0/2008304491 >> 172.21.15.32:0/2008304491 conn(0x56209d83c460 msgr2=0x56209dbf29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.477+0000 7f0d083d0740 1 -- 172.21.15.32:0/2008304491 shutdown_connections 2024-08-31T07:20:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.477+0000 7f0d083d0740 1 -- 172.21.15.32:0/2008304491 wait complete. 2024-08-31T07:20:02.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.477+0000 7f0d083d0740 20 client.27055 trim_cache size 0 max 16384 2024-08-31T07:20:02.482 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 -- 172.21.15.32:0/3070983406 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ce00b4bb0 msgr2=0x7f0ce00b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ce00b4bb0 0x7f0ce00b70c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4005650 tx=0x7f0cf401d040 comp rx=0 tx=0).stop 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 -- 172.21.15.32:0/3070983406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cfc109240 msgr2=0x7f0cfc19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cfc109240 0x7f0cfc19a860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc06c790 tx=0x7f0ce800ee50 comp rx=0 tx=0).stop 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 -- 172.21.15.32:0/3070983406 shutdown_connections 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0ce00b4bb0 0x7f0ce00b70c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc109b90 0x7f0cfc19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cfc109240 0x7f0cfc19a860 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 --2- 172.21.15.32:0/3070983406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc103900 0x7f0cfc19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 -- 172.21.15.32:0/3070983406 >> 172.21.15.32:0/3070983406 conn(0x7f0cfc07b5f0 msgr2=0x7f0cfc107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 -- 172.21.15.32:0/3070983406 shutdown_connections 2024-08-31T07:20:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:02.478+0000 7f0d01800640 1 -- 172.21.15.32:0/3070983406 wait complete. 2024-08-31T07:20:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:02 smithi084 ceph-mon[22387]: pgmap v588: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-08-31T07:20:02.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.06 MB/sec execute 231 sec latency 15203.639 ms 2024-08-31T07:20:03.509 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:20:03.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.05 MB/sec execute 232 sec latency 16203.689 ms 2024-08-31T07:20:03.892 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:04.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:04 smithi032 ceph-mon[17252]: pgmap v589: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-08-31T07:20:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:04 smithi073 ceph-mon[21995]: pgmap v589: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-08-31T07:20:04.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:04 smithi084 ceph-mon[22387]: pgmap v589: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-08-31T07:20:04.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.03 MB/sec execute 233 sec latency 17203.764 ms 2024-08-31T07:20:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.854+0000 7faf28c00640 1 -- 172.21.15.32:0/1423626642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf24075ea0 msgr2=0x7faf2410f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.854+0000 7faf28c00640 1 --2- 172.21.15.32:0/1423626642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf24075ea0 0x7faf2410f770 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009900 tx=0x7faf0c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.854+0000 7faf28c00640 1 -- 172.21.15.32:0/1423626642 shutdown_connections 2024-08-31T07:20:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.854+0000 7faf28c00640 1 --2- 172.21.15.32:0/1423626642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf2410fcb0 0x7faf241120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.854+0000 7faf28c00640 1 --2- 172.21.15.32:0/1423626642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf24075ea0 0x7faf2410f770 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.854+0000 7faf28c00640 1 --2- 172.21.15.32:0/1423626642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24075550 0x7faf24075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.854+0000 7faf28c00640 1 -- 172.21.15.32:0/1423626642 >> 172.21.15.32:0/1423626642 conn(0x7faf240ff1e0 msgr2=0x7faf24101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.854+0000 7faf28c00640 1 -- 172.21.15.32:0/1423626642 shutdown_connections 2024-08-31T07:20:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.854+0000 7faf28c00640 1 -- 172.21.15.32:0/1423626642 wait complete. 2024-08-31T07:20:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.855+0000 7faf28c00640 1 Processor -- start 2024-08-31T07:20:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.855+0000 7faf28c00640 1 -- start start 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.855+0000 7faf28c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf24075550 0x7faf2419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.855+0000 7faf28c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24075ea0 0x7faf2419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf28c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf2410fcb0 0x7faf241a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf28c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf241158b0 con 0x7faf24075550 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf28c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf24115730 con 0x7faf2410fcb0 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf28c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf24115a30 con 0x7faf24075ea0 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf23400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf24075550 0x7faf2419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf13e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24075ea0 0x7faf2419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf23e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf2410fcb0 0x7faf241a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf13e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24075ea0 0x7faf2419c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54450/0 (socket says 172.21.15.32:54450) 2024-08-31T07:20:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf23400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf24075550 0x7faf2419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41734/0 (socket says 172.21.15.32:41734) 2024-08-31T07:20:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf23e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf2410fcb0 0x7faf241a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60276/0 (socket says 172.21.15.32:60276) 2024-08-31T07:20:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf13e00640 1 -- 172.21.15.32:0/4252471368 learned_addr learned my addr 172.21.15.32:0/4252471368 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf23e00640 1 -- 172.21.15.32:0/4252471368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24075ea0 msgr2=0x7faf2419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf23e00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24075ea0 0x7faf2419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf23e00640 1 -- 172.21.15.32:0/4252471368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf24075550 msgr2=0x7faf2419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf23e00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf24075550 0x7faf2419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.856+0000 7faf23e00640 1 -- 172.21.15.32:0/4252471368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf241a2230 con 0x7faf2410fcb0 2024-08-31T07:20:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.857+0000 7faf23e00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf2410fcb0 0x7faf241a1b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00e970 tx=0x7faf1c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.857+0000 7faf20c00640 1 -- 172.21.15.32:0/4252471368 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00ccc0 con 0x7faf2410fcb0 2024-08-31T07:20:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.857+0000 7faf20c00640 1 -- 172.21.15.32:0/4252471368 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf1c005020 con 0x7faf2410fcb0 2024-08-31T07:20:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.857+0000 7faf28c00640 1 -- 172.21.15.32:0/4252471368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf241a2520 con 0x7faf2410fcb0 2024-08-31T07:20:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.857+0000 7faf20c00640 1 -- 172.21.15.32:0/4252471368 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c004a00 con 0x7faf2410fcb0 2024-08-31T07:20:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.857+0000 7faf28c00640 1 -- 172.21.15.32:0/4252471368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf241a8c80 con 0x7faf2410fcb0 2024-08-31T07:20:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.860+0000 7faf30dd3740 1 Processor -- start 2024-08-31T07:20:04.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.861+0000 7faf20c00640 1 -- 172.21.15.32:0/4252471368 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7faf1c004ba0 con 0x7faf2410fcb0 2024-08-31T07:20:04.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.861+0000 7faf30dd3740 1 -- start start 2024-08-31T07:20:04.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e4ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e4b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e4b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf30dd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ef92dc17b0 con 0x55ef92a84390 2024-08-31T07:20:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf30dd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ef92db0d40 con 0x55ef92a7c1b0 2024-08-31T07:20:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf30dd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ef92db4cf0 con 0x55ef92aa5070 2024-08-31T07:20:04.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf13e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e4ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf13e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e4ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54460/0 (socket says 172.21.15.32:54460) 2024-08-31T07:20:04.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf23400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e4b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf13e00640 1 -- 172.21.15.32:0/1226765646 learned_addr learned my addr 172.21.15.32:0/1226765646 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.862+0000 7faf13e00640 1 --2- 172.21.15.32:0/1226765646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e4b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf23400640 1 -- 172.21.15.32:0/1226765646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 msgr2=0x55ef92e4ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf23400640 1 --2- 172.21.15.32:0/1226765646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e4ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf23400640 1 -- 172.21.15.32:0/1226765646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 msgr2=0x55ef92e4b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf23400640 1 --2- 172.21.15.32:0/1226765646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e4b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf23400640 1 -- 172.21.15.32:0/1226765646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef92b10650 con 0x55ef92a7c1b0 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf23400640 1 --2- 172.21.15.32:0/1226765646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e4b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faf14009f80 tx=0x7faf140118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16a603227f97e85b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf20c00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf040b4ca0 0x7faf040b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf19200640 1 -- 172.21.15.32:0/1226765646 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14004d20 con 0x55ef92a7c1b0 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf19200640 1 -- 172.21.15.32:0/1226765646 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf14004050 con 0x55ef92a7c1b0 2024-08-31T07:20:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf19200640 1 -- 172.21.15.32:0/1226765646 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14005260 con 0x55ef92a7c1b0 2024-08-31T07:20:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.863+0000 7faf20c00640 1 -- 172.21.15.32:0/4252471368 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7faf1c0e54b0 con 0x7faf2410fcb0 2024-08-31T07:20:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.864+0000 7faf13e00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf040b4ca0 0x7faf040b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.864+0000 7faf30dd3740 1 -- 172.21.15.32:0/1226765646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 msgr2=0x55ef92e4b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.864+0000 7faf13e00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf040b4ca0 0x7faf040b71b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0096c0 tx=0x7faf0c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.864+0000 7faf30dd3740 1 --2- 172.21.15.32:0/1226765646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e4b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faf14009f80 tx=0x7faf140118d0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.864+0000 7faf30dd3740 1 -- 172.21.15.32:0/1226765646 shutdown_connections 2024-08-31T07:20:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.864+0000 7faf30dd3740 1 --2- 172.21.15.32:0/1226765646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e4b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.864+0000 7faf30dd3740 1 --2- 172.21.15.32:0/1226765646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e4b0a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.864+0000 7faf30dd3740 1 --2- 172.21.15.32:0/1226765646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e4ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.864+0000 7faf30dd3740 1 -- 172.21.15.32:0/1226765646 >> 172.21.15.32:0/1226765646 conn(0x55ef92a99460 msgr2=0x55ef92e48a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.865+0000 7faf30dd3740 1 -- 172.21.15.32:0/1226765646 shutdown_connections 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.865+0000 7faf30dd3740 1 -- 172.21.15.32:0/1226765646 wait complete. 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.865+0000 7faf30dd3740 1 -- start start 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.865+0000 7faf30dd3740 1 Processor -- start 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.866+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e56720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.866+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e56c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.866+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e62680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.866+0000 7faf30dd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ef92c13700 con 0x55ef92a84390 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.866+0000 7faf30dd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ef92d37cf0 con 0x55ef92a7c1b0 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.866+0000 7faf30dd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ef92e59390 con 0x55ef92aa5070 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.866+0000 7faf30dd3740 10 client.0 _ll_register_callbacks cb 0x55ef92d5d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.866+0000 7faf13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e56c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.866+0000 7faf13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e56c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41758/0 (socket says 172.21.15.32:41758) 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.866+0000 7faf13e00640 1 -- 172.21.15.32:0/1342507062 learned_addr learned my addr 172.21.15.32:0/1342507062 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.867+0000 7faf23400640 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e56720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.867+0000 7faf23e00640 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e62680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.867+0000 7faf13e00640 1 -- 172.21.15.32:0/1342507062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 msgr2=0x55ef92e62680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.867+0000 7faf13e00640 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e62680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.867+0000 7faf13e00640 1 -- 172.21.15.32:0/1342507062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 msgr2=0x55ef92e56720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.867+0000 7faf13e00640 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e56720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.867+0000 7faf13e00640 1 -- 172.21.15.32:0/1342507062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef92d788b0 con 0x55ef92a84390 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.867+0000 7faf13e00640 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e56c60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7faf0c005ab0 tx=0x7faf0c02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f2908046124957 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.867+0000 7faf10c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:20:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.868+0000 7faf10c00640 1 -- 172.21.15.32:0/1342507062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0316a0 con 0x55ef92a84390 2024-08-31T07:20:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.868+0000 7faf30dd3740 1 -- 172.21.15.32:0/1342507062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf14004050 con 0x55ef92a84390 2024-08-31T07:20:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.868+0000 7faf10c00640 1 -- 172.21.15.32:0/1342507062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7faf0c031840 con 0x55ef92a84390 2024-08-31T07:20:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.868+0000 7faf10c00640 1 -- 172.21.15.32:0/1342507062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faf0c031a40 con 0x55ef92a84390 2024-08-31T07:20:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.868+0000 7faf30dd3740 10 client.17055 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.868+0000 7faf30dd3740 1 -- 172.21.15.32:0/1342507062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ef92e653b0 con 0x55ef92a84390 2024-08-31T07:20:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.868+0000 7faf10c00640 1 -- 172.21.15.32:0/1342507062 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf14004050 con 0x55ef92a84390 2024-08-31T07:20:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.869+0000 7faf10c00640 1 -- 172.21.15.32:0/1342507062 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7faf0c03d450 con 0x55ef92a84390 2024-08-31T07:20:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.869+0000 7faf30dd3740 10 client.17055 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.869+0000 7faf30dd3740 10 client.17055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:20:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.869+0000 7faf30dd3740 20 client.17055 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.869+0000 7faf30dd3740 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ef92a639f0 0x55ef92e68830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.869+0000 7faf30dd3740 4 client.17055 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.869+0000 7faf30dd3740 1 -- 172.21.15.32:0/1342507062 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef92dc3700 con 0x55ef92a639f0 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.869+0000 7faf23400640 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ef92a639f0 0x55ef92e68830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.870+0000 7faf23400640 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ef92a639f0 0x55ef92e68830 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc25c880388884d4 server_cookie=88302c7a84b1e226 in_seq=0 out_seq=0 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.870+0000 7faf10c00640 10 client.17055 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.870+0000 7faf10c00640 1 -- 172.21.15.32:0/1342507062 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ef92dc3700 con 0x55ef92a639f0 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.870+0000 7faf10c00640 10 client.17055 handle_command_reply: tid=0 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.870+0000 7faf30dd3740 1 client.17055 shutdown 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 1 -- 172.21.15.32:0/1342507062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 msgr2=0x55ef92e56c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e56c60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7faf0c005ab0 tx=0x7faf0c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 1 -- 172.21.15.32:0/1342507062 shutdown_connections 2024-08-31T07:20:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e62680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e56c60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ef92a639f0 0x55ef92e68830 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 1 --2- 172.21.15.32:0/1342507062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e56720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 1 -- 172.21.15.32:0/1342507062 >> 172.21.15.32:0/1342507062 conn(0x55ef92a99460 msgr2=0x55ef92e4dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf10c00640 0 client.17055 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:20:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 1 -- 172.21.15.32:0/1342507062 shutdown_connections 2024-08-31T07:20:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 1 -- 172.21.15.32:0/1342507062 wait complete. 2024-08-31T07:20:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.871+0000 7faf30dd3740 20 client.17055 trim_cache size 0 max 16384 2024-08-31T07:20:04.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.885+0000 7faf30dd3740 1 Processor -- start 2024-08-31T07:20:04.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.885+0000 7faf30dd3740 1 -- start start 2024-08-31T07:20:04.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.885+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e62580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.885+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.885+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.885+0000 7faf30dd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf241b14a0 con 0x55ef92a84390 2024-08-31T07:20:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.885+0000 7faf30dd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf241b30a0 con 0x55ef92a7c1b0 2024-08-31T07:20:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.885+0000 7faf30dd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf241b1b70 con 0x55ef92aa5070 2024-08-31T07:20:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e4fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41774/0 (socket says 172.21.15.32:41774) 2024-08-31T07:20:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf13e00640 1 -- 172.21.15.32:0/3431647774 learned_addr learned my addr 172.21.15.32:0/3431647774 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf23400640 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e62580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf23e00640 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf13e00640 1 -- 172.21.15.32:0/3431647774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 msgr2=0x55ef92e501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf13e00640 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf13e00640 1 -- 172.21.15.32:0/3431647774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 msgr2=0x55ef92e62580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf13e00640 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e62580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf13e00640 1 -- 172.21.15.32:0/3431647774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef92e52090 con 0x55ef92a84390 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf23e00640 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.886+0000 7faf23400640 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e62580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.887+0000 7faf13e00640 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e4fc70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7faf0c03d870 tx=0x7faf0c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596624d49c6106df server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.887+0000 7faf13400640 1 -- 172.21.15.32:0/3431647774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c03dc60 con 0x55ef92a84390 2024-08-31T07:20:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.887+0000 7faf13400640 1 -- 172.21.15.32:0/3431647774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c062070 con 0x55ef92a84390 2024-08-31T07:20:04.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.887+0000 7faf13400640 1 -- 172.21.15.32:0/3431647774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c05d6e0 con 0x55ef92a84390 2024-08-31T07:20:04.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.888+0000 7faf30dd3740 1 -- 172.21.15.32:0/3431647774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 msgr2=0x55ef92e4fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:04.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.888+0000 7faf30dd3740 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e4fc70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7faf0c03d870 tx=0x7faf0c031400 comp rx=0 tx=0).stop 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.888+0000 7faf30dd3740 1 -- 172.21.15.32:0/3431647774 shutdown_connections 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.888+0000 7faf30dd3740 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92aa5070 0x55ef92e501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.888+0000 7faf30dd3740 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a84390 0x55ef92e4fc70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.888+0000 7faf30dd3740 1 --2- 172.21.15.32:0/3431647774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92a7c1b0 0x55ef92e62580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.888+0000 7faf30dd3740 1 -- 172.21.15.32:0/3431647774 >> 172.21.15.32:0/3431647774 conn(0x55ef92a99460 msgr2=0x55ef92e54da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.888+0000 7faf30dd3740 1 -- 172.21.15.32:0/3431647774 shutdown_connections 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.888+0000 7faf30dd3740 1 -- 172.21.15.32:0/3431647774 wait complete. 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.888+0000 7faf30dd3740 1 -- start start 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.889+0000 7faf30dd3740 1 Processor -- start 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.889+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a7c1b0 0x55ef92e5a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.889+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92a84390 0x55ef92e5ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.889+0000 7faf30dd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92aa5070 0x55ef92e5b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.889+0000 7faf30dd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faf241b2910 con 0x55ef92a7c1b0 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.889+0000 7faf30dd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf241b2240 con 0x55ef92aa5070 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.889+0000 7faf30dd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf241af4a0 con 0x55ef92a84390 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.889+0000 7faf30dd3740 10 client.0 _ll_register_callbacks cb 0x7faf0c003760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.889+0000 7faf23400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a7c1b0 0x55ef92e5a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf23400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a7c1b0 0x55ef92e5a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41786/0 (socket says 172.21.15.32:41786) 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf23400640 1 -- 172.21.15.32:0/3482101838 learned_addr learned my addr 172.21.15.32:0/3482101838 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf13e00640 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92a84390 0x55ef92e5ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf23e00640 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92aa5070 0x55ef92e5b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf23400640 1 -- 172.21.15.32:0/3482101838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92a84390 msgr2=0x55ef92e5ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf23400640 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92a84390 0x55ef92e5ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf23400640 1 -- 172.21.15.32:0/3482101838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92aa5070 msgr2=0x55ef92e5b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf23400640 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92aa5070 0x55ef92e5b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf23400640 1 -- 172.21.15.32:0/3482101838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef92ed2320 con 0x55ef92a7c1b0 2024-08-31T07:20:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf23400640 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a7c1b0 0x55ef92e5a6d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faf14014040 tx=0x7faf14009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492d3bc74caf055 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf10c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:20:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.890+0000 7faf10c00640 1 -- 172.21.15.32:0/3482101838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14003a30 con 0x55ef92a7c1b0 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf10c00640 1 -- 172.21.15.32:0/3482101838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8441+0+0 (secure 0 0 0) 0x7faf1401f040 con 0x55ef92a7c1b0 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf30dd3740 1 -- 172.21.15.32:0/3482101838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0c062070 con 0x55ef92a7c1b0 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf10c00640 1 -- 172.21.15.32:0/3482101838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faf1403b050 con 0x55ef92a7c1b0 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf30dd3740 10 client.17061 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf30dd3740 1 -- 172.21.15.32:0/3482101838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ef92ed3580 con 0x55ef92a7c1b0 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf10c00640 1 -- 172.21.15.32:0/3482101838 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c062070 con 0x55ef92a7c1b0 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf10c00640 1 -- 172.21.15.32:0/3482101838 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7faf1400fcd0 con 0x55ef92a7c1b0 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf30dd3740 10 client.17061 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf30dd3740 10 client.17061 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf30dd3740 20 client.17061 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf30dd3740 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ef92a38da0 0x55ef92ed5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf30dd3740 4 client.17061 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:20:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.891+0000 7faf30dd3740 1 -- 172.21.15.32:0/3482101838 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef92dbf1a0 con 0x55ef92a38da0 2024-08-31T07:20:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.892+0000 7faf13e00640 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ef92a38da0 0x55ef92ed5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.892+0000 7faf13e00640 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ef92a38da0 0x55ef92ed5c00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c8ff9e1ae8074f server_cookie=3b3ecf424439ffb in_seq=0 out_seq=0 2024-08-31T07:20:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.892+0000 7faf10c00640 10 client.17061 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:20:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf10c00640 1 -- 172.21.15.32:0/3482101838 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef92dbf1a0 con 0x55ef92a38da0 2024-08-31T07:20:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf10c00640 10 client.17061 handle_command_reply: tid=0 2024-08-31T07:20:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 client.17061 shutdown 2024-08-31T07:20:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 -- 172.21.15.32:0/3482101838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a7c1b0 msgr2=0x55ef92e5a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a7c1b0 0x55ef92e5a6d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faf14014040 tx=0x7faf14009500 comp rx=0 tx=0).stop 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 -- 172.21.15.32:0/3482101838 shutdown_connections 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ef92a38da0 0x55ef92ed5c00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef92aa5070 0x55ef92e5b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef92a84390 0x55ef92e5ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 --2- 172.21.15.32:0/3482101838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ef92a7c1b0 0x55ef92e5a6d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 -- 172.21.15.32:0/3482101838 >> 172.21.15.32:0/3482101838 conn(0x55ef92a99460 msgr2=0x55ef92e4fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf10c00640 0 client.17061 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 -- 172.21.15.32:0/3482101838 shutdown_connections 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 1 -- 172.21.15.32:0/3482101838 wait complete. 2024-08-31T07:20:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.893+0000 7faf30dd3740 20 client.17061 trim_cache size 0 max 16384 2024-08-31T07:20:04.900 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:20:04.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 -- 172.21.15.32:0/4252471368 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf040b4ca0 msgr2=0x7faf040b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:04.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf040b4ca0 0x7faf040b71b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0096c0 tx=0x7faf0c005a20 comp rx=0 tx=0).stop 2024-08-31T07:20:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 -- 172.21.15.32:0/4252471368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf2410fcb0 msgr2=0x7faf241a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf2410fcb0 0x7faf241a1b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00e970 tx=0x7faf1c00ee30 comp rx=0 tx=0).stop 2024-08-31T07:20:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 -- 172.21.15.32:0/4252471368 shutdown_connections 2024-08-31T07:20:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faf040b4ca0 0x7faf040b71b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf2410fcb0 0x7faf241a1b00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf24075ea0 0x7faf2419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 --2- 172.21.15.32:0/4252471368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf24075550 0x7faf2419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 -- 172.21.15.32:0/4252471368 >> 172.21.15.32:0/4252471368 conn(0x7faf240ff1e0 msgr2=0x7faf24109830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 -- 172.21.15.32:0/4252471368 shutdown_connections 2024-08-31T07:20:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:04.894+0000 7faf28c00640 1 -- 172.21.15.32:0/4252471368 wait complete. 2024-08-31T07:20:05.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.02 MB/sec execute 234 sec latency 18203.894 ms 2024-08-31T07:20:05.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:20:06.207 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:06 smithi032 ceph-mon[17252]: pgmap v590: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-08-31T07:20:06.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:06 smithi073 ceph-mon[21995]: pgmap v590: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-08-31T07:20:06.464 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:06.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:06 smithi084 ceph-mon[22387]: pgmap v590: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-08-31T07:20:06.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.01 MB/sec execute 235 sec latency 19203.974 ms 2024-08-31T07:20:07.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.131+0000 7fd8c4a00640 1 -- 172.21.15.32:0/2234892232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8c0109070 msgr2=0x7fd8c0109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:07.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.131+0000 7fd8c4a00640 1 --2- 172.21.15.32:0/2234892232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8c0109070 0x7fd8c0109480 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009900 tx=0x7fd8a802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.131+0000 7fd8c4a00640 1 -- 172.21.15.32:0/2234892232 shutdown_connections 2024-08-31T07:20:07.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.131+0000 7fd8c4a00640 1 --2- 172.21.15.32:0/2234892232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8c01099c0 0x7fd8c01101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.131+0000 7fd8c4a00640 1 --2- 172.21.15.32:0/2234892232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8c0109070 0x7fd8c0109480 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.131+0000 7fd8c4a00640 1 --2- 172.21.15.32:0/2234892232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c0103730 0x7fd8c0103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.131+0000 7fd8c4a00640 1 -- 172.21.15.32:0/2234892232 >> 172.21.15.32:0/2234892232 conn(0x7fd8c00ff180 msgr2=0x7fd8c01015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:07.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.131+0000 7fd8c4a00640 1 -- 172.21.15.32:0/2234892232 shutdown_connections 2024-08-31T07:20:07.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.131+0000 7fd8c4a00640 1 -- 172.21.15.32:0/2234892232 wait complete. 2024-08-31T07:20:07.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.132+0000 7fd8c4a00640 1 Processor -- start 2024-08-31T07:20:07.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.132+0000 7fd8c4a00640 1 -- start start 2024-08-31T07:20:07.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8c4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c0103730 0x7fd8c019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8c4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8c0109070 0x7fd8c019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8c4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8c01099c0 0x7fd8c01a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8c4a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0113680 con 0x7fd8c01099c0 2024-08-31T07:20:07.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8c4a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0113500 con 0x7fd8c0109070 2024-08-31T07:20:07.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8c4a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0113800 con 0x7fd8c0103730 2024-08-31T07:20:07.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8bf400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c0103730 0x7fd8c019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8bf400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c0103730 0x7fd8c019c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34774/0 (socket says 172.21.15.32:34774) 2024-08-31T07:20:07.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8bf400640 1 -- 172.21.15.32:0/1069242880 learned_addr learned my addr 172.21.15.32:0/1069242880 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:07.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8c01099c0 0x7fd8c01a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.133+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8c0109070 0x7fd8c019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8bf400640 1 -- 172.21.15.32:0/1069242880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8c0109070 msgr2=0x7fd8c019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8bf400640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8c0109070 0x7fd8c019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8bf400640 1 -- 172.21.15.32:0/1069242880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8c01099c0 msgr2=0x7fd8c01a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8bf400640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8c01099c0 0x7fd8c01a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8bf400640 1 -- 172.21.15.32:0/1069242880 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c01a2240 con 0x7fd8c0103730 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8bf400640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c0103730 0x7fd8c019c2d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000e9f0 tx=0x7fd8b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8bcc00640 1 -- 172.21.15.32:0/1069242880 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000cd80 con 0x7fd8c0103730 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8c4a00640 1 -- 172.21.15.32:0/1069242880 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c01a2530 con 0x7fd8c0103730 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8bcc00640 1 -- 172.21.15.32:0/1069242880 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b00108e0 con 0x7fd8c0103730 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8bcc00640 1 -- 172.21.15.32:0/1069242880 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0018980 con 0x7fd8c0103730 2024-08-31T07:20:07.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.134+0000 7fd8c4a00640 1 -- 172.21.15.32:0/1069242880 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c01a8c90 con 0x7fd8c0103730 2024-08-31T07:20:07.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.138+0000 7fd8ccc62740 1 Processor -- start 2024-08-31T07:20:07.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.139+0000 7fd8ccc62740 1 -- start start 2024-08-31T07:20:07.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.139+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae981070 0x5615aed26b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.139+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae9581b0 0x5615aed270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.139+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 0x5615aed275e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.139+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5615aec9d7b0 con 0x5615ae9581b0 2024-08-31T07:20:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.139+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5615aec8cd40 con 0x5615ae981070 2024-08-31T07:20:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.139+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615aec90cf0 con 0x5615ae960390 2024-08-31T07:20:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae981070 0x5615aed26b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bcc00640 1 -- 172.21.15.32:0/1069242880 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd8b0010430 con 0x7fd8c0103730 2024-08-31T07:20:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae981070 0x5615aed26b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48076/0 (socket says 172.21.15.32:48076) 2024-08-31T07:20:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bfe00640 1 -- 172.21.15.32:0/3337687869 learned_addr learned my addr 172.21.15.32:0/3337687869 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:07.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/3337687869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 0x5615aed275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bf400640 1 --2- 172.21.15.32:0/3337687869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae9581b0 0x5615aed270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bfe00640 1 -- 172.21.15.32:0/3337687869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 msgr2=0x5615aed275e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/3337687869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 0x5615aed275e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bfe00640 1 -- 172.21.15.32:0/3337687869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae9581b0 msgr2=0x5615aed270a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/3337687869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae9581b0 0x5615aed270a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bfe00640 1 -- 172.21.15.32:0/3337687869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ae9ec650 con 0x5615ae981070 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.140+0000 7fd8bf400640 1 --2- 172.21.15.32:0/3337687869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae9581b0 0x5615aed270a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.141+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/3337687869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae981070 0x5615aed26b60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009fd0 tx=0x7fd8b80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba3ab4b45f7365e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.141+0000 7fd8afe00640 1 -- 172.21.15.32:0/3337687869 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8004d20 con 0x5615ae981070 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.141+0000 7fd8afe00640 1 -- 172.21.15.32:0/3337687869 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b8004050 con 0x5615ae981070 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.141+0000 7fd8afe00640 1 -- 172.21.15.32:0/3337687869 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8005280 con 0x5615ae981070 2024-08-31T07:20:07.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.141+0000 7fd8bcc00640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8a40b4ca0 0x7fd8a40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8ccc62740 1 -- 172.21.15.32:0/3337687869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae981070 msgr2=0x5615aed26b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/3337687869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae981070 0x5615aed26b60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009fd0 tx=0x7fd8b80118d0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8bcc00640 1 -- 172.21.15.32:0/1069242880 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8485+0+0 (secure 0 0 0) 0x7fd8b00e5890 con 0x7fd8c0103730 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8ccc62740 1 -- 172.21.15.32:0/3337687869 shutdown_connections 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/3337687869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 0x5615aed275e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/3337687869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae9581b0 0x5615aed270a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/3337687869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae981070 0x5615aed26b60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8ccc62740 1 -- 172.21.15.32:0/3337687869 >> 172.21.15.32:0/3337687869 conn(0x5615ae975460 msgr2=0x5615aed24a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8bf400640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8a40b4ca0 0x7fd8a40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8ccc62740 1 -- 172.21.15.32:0/3337687869 shutdown_connections 2024-08-31T07:20:07.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.142+0000 7fd8ccc62740 1 -- 172.21.15.32:0/3337687869 wait complete. 2024-08-31T07:20:07.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.143+0000 7fd8bf400640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8a40b4ca0 0x7fd8a40b71b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5615aed257e0 tx=0x7fd8b0000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:07.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.143+0000 7fd8ccc62740 1 -- start start 2024-08-31T07:20:07.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.143+0000 7fd8ccc62740 1 Processor -- start 2024-08-31T07:20:07.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.143+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed32750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.143+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae960390 0x5615aed32c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.143+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 0x5615aed3e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.143+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5615aeaef700 con 0x5615ae960390 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.143+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5615aec13cf0 con 0x5615ae9581b0 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.143+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615aed35340 con 0x5615ae981070 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8ccc62740 10 client.0 _ll_register_callbacks cb 0x5615aec39470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed32750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed32750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48086/0 (socket says 172.21.15.32:48086) 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8bfe00640 1 -- 172.21.15.32:0/2857313569 learned_addr learned my addr 172.21.15.32:0/2857313569 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae960390 0x5615aed32c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 0x5615aed3e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8b7e00640 1 -- 172.21.15.32:0/2857313569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 msgr2=0x5615aed3e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 0x5615aed3e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8b7e00640 1 -- 172.21.15.32:0/2857313569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 msgr2=0x5615aed32750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed32750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.144+0000 7fd8b7e00640 1 -- 172.21.15.32:0/2857313569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615aec548b0 con 0x5615ae960390 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.145+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae960390 0x5615aed32c90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009230 tx=0x7fd8a8004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ebbe3f207fd919 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.145+0000 7fd8acc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.145+0000 7fd8acc00640 1 -- 172.21.15.32:0/2857313569 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8035040 con 0x5615ae960390 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.145+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2857313569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b8004050 con 0x5615ae960390 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.145+0000 7fd8acc00640 1 -- 172.21.15.32:0/2857313569 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8485+0+0 (secure 0 0 0) 0x7fd8a8005450 con 0x5615ae960390 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.145+0000 7fd8acc00640 1 -- 172.21.15.32:0/2857313569 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd8a8005650 con 0x5615ae960390 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.145+0000 7fd8ccc62740 10 client.17070 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.145+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2857313569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5615aed41240 con 0x5615ae960390 2024-08-31T07:20:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.146+0000 7fd8acc00640 1 -- 172.21.15.32:0/2857313569 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b8004050 con 0x5615ae960390 2024-08-31T07:20:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.146+0000 7fd8acc00640 1 -- 172.21.15.32:0/2857313569 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fd8a80038f0 con 0x5615ae960390 2024-08-31T07:20:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.146+0000 7fd8ccc62740 10 client.17070 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.146+0000 7fd8ccc62740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:20:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.146+0000 7fd8ccc62740 20 client.17070 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.146+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5615ae93f9f0 0x5615aed447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.146+0000 7fd8ccc62740 4 client.17070 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.146+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2857313569 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615aec9f700 con 0x5615ae93f9f0 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.147+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5615ae93f9f0 0x5615aed447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.147+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5615ae93f9f0 0x5615aed447e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8def32d06e27bee7 server_cookie=3d26674253f1a2bf in_seq=0 out_seq=0 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.147+0000 7fd8acc00640 10 client.17070 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.148+0000 7fd8acc00640 1 -- 172.21.15.32:0/2857313569 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615aec9f700 con 0x5615ae93f9f0 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.148+0000 7fd8acc00640 10 client.17070 handle_command_reply: tid=0 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.148+0000 7fd8ccc62740 1 client.17070 shutdown 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.148+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2857313569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae960390 msgr2=0x5615aed32c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.148+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae960390 0x5615aed32c90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009230 tx=0x7fd8a8004340 comp rx=0 tx=0).stop 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.149+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2857313569 shutdown_connections 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.149+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5615ae93f9f0 0x5615aed447e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.149+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 0x5615aed3e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.149+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae960390 0x5615aed32c90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.149+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2857313569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed32750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.149+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2857313569 >> 172.21.15.32:0/2857313569 conn(0x5615ae975460 msgr2=0x5615aed29cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:07.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.149+0000 7fd8acc00640 0 client.17070 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:07.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.149+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2857313569 shutdown_connections 2024-08-31T07:20:07.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.149+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2857313569 wait complete. 2024-08-31T07:20:07.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.149+0000 7fd8ccc62740 20 client.17070 trim_cache size 0 max 16384 2024-08-31T07:20:07.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.163+0000 7fd8ccc62740 1 Processor -- start 2024-08-31T07:20:07.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.163+0000 7fd8ccc62740 1 -- start start 2024-08-31T07:20:07.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.163+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed2bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.163+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 0x5615aed2c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.163+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae981070 0x5615aed24240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.163+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01b1530 con 0x5615ae981070 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.163+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01b3130 con 0x5615ae9581b0 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.163+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01b1c00 con 0x5615ae960390 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae981070 0x5615aed24240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8b7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 0x5615aed2c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae981070 0x5615aed24240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54096/0 (socket says 172.21.15.32:54096) 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8b7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 0x5615aed2c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34808/0 (socket says 172.21.15.32:34808) 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8bfe00640 1 -- 172.21.15.32:0/1298855587 learned_addr learned my addr 172.21.15.32:0/1298855587 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:07.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/1298855587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed2bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8b7e00640 1 -- 172.21.15.32:0/1298855587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 msgr2=0x5615aed2bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:07.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/1298855587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed2bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8b7e00640 1 -- 172.21.15.32:0/1298855587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae981070 msgr2=0x5615aed24240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/1298855587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae981070 0x5615aed24240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8b7e00640 1 -- 172.21.15.32:0/1298855587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615aed2e0c0 con 0x5615ae960390 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.164+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/1298855587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 0x5615aed2c1b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8003cc0 tx=0x7fd8a8031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb5c16f148600a47 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.165+0000 7fd8af400640 1 -- 172.21.15.32:0/1298855587 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8035070 con 0x5615ae960390 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.165+0000 7fd8af400640 1 -- 172.21.15.32:0/1298855587 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a8041070 con 0x5615ae960390 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.165+0000 7fd8af400640 1 -- 172.21.15.32:0/1298855587 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8053070 con 0x5615ae960390 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 -- 172.21.15.32:0/1298855587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 msgr2=0x5615aed2c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/1298855587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 0x5615aed2c1b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8003cc0 tx=0x7fd8a8031400 comp rx=0 tx=0).stop 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 -- 172.21.15.32:0/1298855587 shutdown_connections 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/1298855587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae981070 0x5615aed24240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/1298855587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae960390 0x5615aed2c1b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/1298855587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed2bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 -- 172.21.15.32:0/1298855587 >> 172.21.15.32:0/1298855587 conn(0x5615ae975460 msgr2=0x5615aed30dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:07.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 -- 172.21.15.32:0/1298855587 shutdown_connections 2024-08-31T07:20:07.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 -- 172.21.15.32:0/1298855587 wait complete. 2024-08-31T07:20:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 -- start start 2024-08-31T07:20:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.166+0000 7fd8ccc62740 1 Processor -- start 2024-08-31T07:20:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed3af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 0x5615aed26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8ccc62740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae914da0 0x5615aed27180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01b29a0 con 0x5615ae914da0 2024-08-31T07:20:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01b22d0 con 0x5615ae9581b0 2024-08-31T07:20:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8ccc62740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01af530 con 0x5615ae981070 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8ccc62740 10 client.0 _ll_register_callbacks cb 0x7fd8a8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed3af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8b7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 0x5615aed26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae914da0 0x5615aed27180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8b7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 0x5615aed26a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34822/0 (socket says 172.21.15.32:34822) 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.167+0000 7fd8b7e00640 1 -- 172.21.15.32:0/2443857736 learned_addr learned my addr 172.21.15.32:0/2443857736 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8b7e00640 1 -- 172.21.15.32:0/2443857736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 msgr2=0x5615aed3af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed3af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8b7e00640 1 -- 172.21.15.32:0/2443857736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae914da0 msgr2=0x5615aed27180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae914da0 0x5615aed27180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8b7e00640 1 -- 172.21.15.32:0/2443857736 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615aed39bf0 con 0x5615ae981070 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed3af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8b7e00640 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 0x5615aed26a70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8003c90 tx=0x7fd8a80038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1053e1f0a4ddb36b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8acc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:20:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8acc00640 1 -- 172.21.15.32:0/2443857736 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8053070 con 0x5615ae981070 2024-08-31T07:20:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2443857736 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8a8041070 con 0x5615ae981070 2024-08-31T07:20:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.168+0000 7fd8acc00640 1 -- 172.21.15.32:0/2443857736 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8485+0+0 (secure 0 0 0) 0x7fd8a8031d30 con 0x5615ae981070 2024-08-31T07:20:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8acc00640 1 -- 172.21.15.32:0/2443857736 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd8a8035040 con 0x5615ae981070 2024-08-31T07:20:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8ccc62740 10 client.27091 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2443857736 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5615aedaf360 con 0x5615ae981070 2024-08-31T07:20:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8acc00640 1 -- 172.21.15.32:0/2443857736 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a8041070 con 0x5615ae981070 2024-08-31T07:20:07.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8acc00640 1 -- 172.21.15.32:0/2443857736 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fd8a805e030 con 0x5615ae981070 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8ccc62740 10 client.27091 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8ccc62740 10 client.27091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8ccc62740 20 client.27091 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5615aedafa50 0x5615aedb1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8ccc62740 4 client.27091 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.169+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2443857736 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615aec9b1a0 con 0x5615aedafa50 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.170+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5615aedafa50 0x5615aedb1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.170+0000 7fd8bfe00640 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5615aedafa50 0x5615aedb1e70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e525b39e8b1ff8 server_cookie=7263ef2234434368 in_seq=0 out_seq=0 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.170+0000 7fd8acc00640 10 client.27091 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8acc00640 1 -- 172.21.15.32:0/2443857736 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615aec9b1a0 con 0x5615aedafa50 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8acc00640 10 client.27091 handle_command_reply: tid=0 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 client.27091 shutdown 2024-08-31T07:20:07.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2443857736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 msgr2=0x5615aed26a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 0x5615aed26a70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8003c90 tx=0x7fd8a80038a0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2443857736 shutdown_connections 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5615aedafa50 0x5615aedb1e70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5615ae914da0 0x5615aed27180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615ae981070 0x5615aed26a70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 --2- 172.21.15.32:0/2443857736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615ae9581b0 0x5615aed3af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8acc00640 0 client.27091 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2443857736 >> 172.21.15.32:0/2443857736 conn(0x5615ae975460 msgr2=0x5615aed24240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2443857736 shutdown_connections 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 1 -- 172.21.15.32:0/2443857736 wait complete. 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.171+0000 7fd8ccc62740 20 client.27091 trim_cache size 0 max 16384 2024-08-31T07:20:07.175 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:20:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 -- 172.21.15.32:0/1069242880 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8a40b4ca0 msgr2=0x7fd8a40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8a40b4ca0 0x7fd8a40b71b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5615aed257e0 tx=0x7fd8b0000bc0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 -- 172.21.15.32:0/1069242880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c0103730 msgr2=0x7fd8c019c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c0103730 0x7fd8c019c2d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000e9f0 tx=0x7fd8b000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 -- 172.21.15.32:0/1069242880 shutdown_connections 2024-08-31T07:20:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd8a40b4ca0 0x7fd8a40b71b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8c01099c0 0x7fd8c01a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8c0109070 0x7fd8c019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 --2- 172.21.15.32:0/1069242880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c0103730 0x7fd8c019c2d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 -- 172.21.15.32:0/1069242880 >> 172.21.15.32:0/1069242880 conn(0x7fd8c00ff180 msgr2=0x7fd8c010e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 -- 172.21.15.32:0/1069242880 shutdown_connections 2024-08-31T07:20:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:07.172+0000 7fd8c4a00640 1 -- 172.21.15.32:0/1069242880 wait complete. 2024-08-31T07:20:07.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:07 smithi032 ceph-mon[17252]: osdmap e83: 12 total, 12 up, 12 in 2024-08-31T07:20:07.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:07 smithi073 ceph-mon[21995]: osdmap e83: 12 total, 12 up, 12 in 2024-08-31T07:20:07.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:07 smithi084 ceph-mon[22387]: osdmap e83: 12 total, 12 up, 12 in 2024-08-31T07:20:07.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 3.00 MB/sec execute 236 sec latency 20204.085 ms 2024-08-31T07:20:08.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:08 smithi032 ceph-mon[17252]: pgmap v592: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-08-31T07:20:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:08 smithi073 ceph-mon[21995]: pgmap v592: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-08-31T07:20:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:08 smithi084 ceph-mon[22387]: pgmap v592: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-08-31T07:20:08.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 2.98 MB/sec execute 237 sec latency 21204.247 ms 2024-08-31T07:20:08.911 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:20:08.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:20:09.311 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:09.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 2.97 MB/sec execute 238 sec latency 22204.377 ms 2024-08-31T07:20:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:09.999+0000 7f3e20c00640 1 -- 172.21.15.32:0/3316969818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e1c103730 msgr2=0x7f3e1c103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:09.999+0000 7f3e20c00640 1 --2- 172.21.15.32:0/3316969818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e1c103730 0x7f3e1c103b40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009900 tx=0x7f3e0c02a410 comp rx=0 tx=0).stop 2024-08-31T07:20:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:09.999+0000 7f3e20c00640 1 -- 172.21.15.32:0/3316969818 shutdown_connections 2024-08-31T07:20:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:09.999+0000 7f3e20c00640 1 --2- 172.21.15.32:0/3316969818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e1c1099c0 0x7f3e1c1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:09.999+0000 7f3e20c00640 1 --2- 172.21.15.32:0/3316969818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1c109070 0x7f3e1c109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:09.999+0000 7f3e20c00640 1 --2- 172.21.15.32:0/3316969818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e1c103730 0x7f3e1c103b40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:09.999+0000 7f3e20c00640 1 -- 172.21.15.32:0/3316969818 >> 172.21.15.32:0/3316969818 conn(0x7f3e1c0ff180 msgr2=0x7f3e1c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:09.999+0000 7f3e20c00640 1 -- 172.21.15.32:0/3316969818 shutdown_connections 2024-08-31T07:20:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:09.999+0000 7f3e20c00640 1 -- 172.21.15.32:0/3316969818 wait complete. 2024-08-31T07:20:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.001+0000 7f3e20c00640 1 Processor -- start 2024-08-31T07:20:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.001+0000 7f3e20c00640 1 -- start start 2024-08-31T07:20:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e20c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e1c103730 0x7f3e1c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e20c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1c109070 0x7f3e1c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e20c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e1c1099c0 0x7f3e1c1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e20c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c113670 con 0x7f3e1c1099c0 2024-08-31T07:20:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e20c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1134f0 con 0x7f3e1c103730 2024-08-31T07:20:10.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e20c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1137f0 con 0x7f3e1c109070 2024-08-31T07:20:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e1aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1c109070 0x7f3e1c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e1aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1c109070 0x7f3e1c19c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34834/0 (socket says 172.21.15.32:34834) 2024-08-31T07:20:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e1aa00640 1 -- 172.21.15.32:0/1662742386 learned_addr learned my addr 172.21.15.32:0/1662742386 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e1b400640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e1c103730 0x7f3e1c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.002+0000 7f3e1be00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e1c1099c0 0x7f3e1c1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e1aa00640 1 -- 172.21.15.32:0/1662742386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e1c103730 msgr2=0x7f3e1c19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e1aa00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e1c103730 0x7f3e1c19c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e1aa00640 1 -- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e1c1099c0 msgr2=0x7f3e1c1a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e1aa00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e1c1099c0 0x7f3e1c1a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e1aa00640 1 -- 172.21.15.32:0/1662742386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1c1a2270 con 0x7f3e1c109070 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e1be00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e1c1099c0 0x7f3e1c1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e1b400640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e1c103730 0x7f3e1c19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e1aa00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1c109070 0x7f3e1c19c8a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800e9f0 tx=0x7f3e0800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e13e00640 1 -- 172.21.15.32:0/1662742386 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800ccc0 con 0x7f3e1c109070 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e20c00640 1 -- 172.21.15.32:0/1662742386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1c1a2560 con 0x7f3e1c109070 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e13e00640 1 -- 172.21.15.32:0/1662742386 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e080108e0 con 0x7f3e1c109070 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e13e00640 1 -- 172.21.15.32:0/1662742386 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08018860 con 0x7f3e1c109070 2024-08-31T07:20:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.003+0000 7f3e20c00640 1 -- 172.21.15.32:0/1662742386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1c1a8cc0 con 0x7f3e1c109070 2024-08-31T07:20:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.006+0000 7f3e28e4c740 1 Processor -- start 2024-08-31T07:20:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e28e4c740 1 -- start start 2024-08-31T07:20:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13977350 0x55df13d1d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 0x55df13d1dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13956670 0x55df13d1e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55df13c93750 con 0x55df13956670 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55df13c82ce0 con 0x55df13977350 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55df13c86c90 con 0x55df1394e490 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e1be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13977350 0x55df13d1d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e1be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13977350 0x55df13d1d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48128/0 (socket says 172.21.15.32:48128) 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e1be00640 1 -- 172.21.15.32:0/877322254 learned_addr learned my addr 172.21.15.32:0/877322254 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.007+0000 7f3e1b400640 1 --2- 172.21.15.32:0/877322254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 0x55df13d1dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.008+0000 7f3e1be00640 1 -- 172.21.15.32:0/877322254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 msgr2=0x55df13d1dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.008+0000 7f3e1be00640 1 --2- 172.21.15.32:0/877322254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 0x55df13d1dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.008+0000 7f3e1be00640 1 -- 172.21.15.32:0/877322254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13956670 msgr2=0x55df13d1e250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.008+0000 7f3e1be00640 1 --2- 172.21.15.32:0/877322254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13956670 0x55df13d1e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.008+0000 7f3e1be00640 1 -- 172.21.15.32:0/877322254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df139e2930 con 0x55df13977350 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.008+0000 7f3e13e00640 1 -- 172.21.15.32:0/1662742386 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3e08010430 con 0x7f3e1c109070 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.008+0000 7f3e1be00640 1 --2- 172.21.15.32:0/877322254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13977350 0x55df13d1d7d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3e14010450 tx=0x7f3e14010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894a425f2f7024d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.008+0000 7f3e07e00640 1 -- 172.21.15.32:0/877322254 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14017070 con 0x55df13977350 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.008+0000 7f3e07e00640 1 -- 172.21.15.32:0/877322254 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e14011c10 con 0x55df13977350 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.008+0000 7f3e07e00640 1 -- 172.21.15.32:0/877322254 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1401ac50 con 0x55df13977350 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.009+0000 7f3e28e4c740 1 -- 172.21.15.32:0/877322254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13977350 msgr2=0x55df13d1d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.009+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/877322254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13977350 0x55df13d1d7d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3e14010450 tx=0x7f3e14010910 comp rx=0 tx=0).stop 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.009+0000 7f3e28e4c740 1 -- 172.21.15.32:0/877322254 shutdown_connections 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.009+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/877322254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13956670 0x55df13d1e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.009+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/877322254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 0x55df13d1dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.009+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/877322254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13977350 0x55df13d1d7d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.009+0000 7f3e28e4c740 1 -- 172.21.15.32:0/877322254 >> 172.21.15.32:0/877322254 conn(0x55df1396b740 msgr2=0x55df13d1b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.009+0000 7f3e28e4c740 1 -- 172.21.15.32:0/877322254 shutdown_connections 2024-08-31T07:20:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.009+0000 7f3e28e4c740 1 -- 172.21.15.32:0/877322254 wait complete. 2024-08-31T07:20:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.010+0000 7f3e13e00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e000b4bb0 0x7f3e000b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.010+0000 7f3e13e00640 1 -- 172.21.15.32:0/1662742386 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8485+0+0 (secure 0 0 0) 0x7f3e080e58d0 con 0x7f3e1c109070 2024-08-31T07:20:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.010+0000 7f3e1b400640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e000b4bb0 0x7f3e000b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.010+0000 7f3e28e4c740 1 -- start start 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.010+0000 7f3e28e4c740 1 Processor -- start 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e1b400640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e000b4bb0 0x7f3e000b70c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c031440 tx=0x7f3e0c032ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df1394e490 0x55df13d29330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 0x55df13d29870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 0x55df13d352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55df13ae5890 con 0x55df1394e490 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55df13c09cd0 con 0x55df13956670 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55df13d2bfb0 con 0x55df13977350 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e28e4c740 10 client.0 _ll_register_callbacks cb 0x55df13c7def0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e1b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df1394e490 0x55df13d29330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e1be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 0x55df13d352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e1aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 0x55df13d29870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e1be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 0x55df13d352a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34856/0 (socket says 172.21.15.32:34856) 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e1aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 0x55df13d29870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48138/0 (socket says 172.21.15.32:48138) 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.011+0000 7f3e1be00640 1 -- 172.21.15.32:0/2390805419 learned_addr learned my addr 172.21.15.32:0/2390805419 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.012+0000 7f3e1b400640 1 -- 172.21.15.32:0/2390805419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 msgr2=0x55df13d352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.012+0000 7f3e1b400640 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 0x55df13d352a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.012+0000 7f3e1b400640 1 -- 172.21.15.32:0/2390805419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 msgr2=0x55df13d29870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.012+0000 7f3e1b400640 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 0x55df13d29870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.012+0000 7f3e1b400640 1 -- 172.21.15.32:0/2390805419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df13c531b0 con 0x55df1394e490 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.012+0000 7f3e1aa00640 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 0x55df13d29870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.012+0000 7f3e1be00640 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 0x55df13d352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.012+0000 7f3e1b400640 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df1394e490 0x55df13d29330 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009230 tx=0x7f3e0c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b3d6edb8fb5471 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.012+0000 7f3e04c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:20:10.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.013+0000 7f3e04c00640 1 -- 172.21.15.32:0/2390805419 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c04d070 con 0x55df1394e490 2024-08-31T07:20:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.013+0000 7f3e04c00640 1 -- 172.21.15.32:0/2390805419 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8485+0+0 (secure 0 0 0) 0x7f3e0c004050 con 0x55df1394e490 2024-08-31T07:20:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.013+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2390805419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e14011c10 con 0x55df1394e490 2024-08-31T07:20:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.013+0000 7f3e04c00640 1 -- 172.21.15.32:0/2390805419 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3e0c059050 con 0x55df1394e490 2024-08-31T07:20:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.013+0000 7f3e28e4c740 10 client.17085 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:10.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.013+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2390805419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55df13d37f70 con 0x55df1394e490 2024-08-31T07:20:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.013+0000 7f3e04c00640 1 -- 172.21.15.32:0/2390805419 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e14011c10 con 0x55df1394e490 2024-08-31T07:20:10.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.014+0000 7f3e04c00640 1 -- 172.21.15.32:0/2390805419 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f3e0c003680 con 0x55df1394e490 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.014+0000 7f3e28e4c740 10 client.17085 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.014+0000 7f3e28e4c740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.014+0000 7f3e28e4c740 20 client.17085 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.014+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55df13935cd0 0x55df13d3b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.014+0000 7f3e28e4c740 4 client.17085 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.014+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2390805419 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df13c956a0 con 0x55df13935cd0 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.014+0000 7f3e1aa00640 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55df13935cd0 0x55df13d3b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.015+0000 7f3e1aa00640 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55df13935cd0 0x55df13d3b5b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b2f904a72b9b6f server_cookie=54e8d4a1b01dba86 in_seq=0 out_seq=0 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.015+0000 7f3e04c00640 10 client.17085 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.015+0000 7f3e04c00640 1 -- 172.21.15.32:0/2390805419 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df13c956a0 con 0x55df13935cd0 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.015+0000 7f3e04c00640 10 client.17085 handle_command_reply: tid=0 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.015+0000 7f3e28e4c740 1 client.17085 shutdown 2024-08-31T07:20:10.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2390805419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df1394e490 msgr2=0x55df13d29330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df1394e490 0x55df13d29330 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009230 tx=0x7f3e0c009260 comp rx=0 tx=0).stop 2024-08-31T07:20:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2390805419 shutdown_connections 2024-08-31T07:20:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55df13935cd0 0x55df13d3b5b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 0x55df13d352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 0x55df13d29870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2390805419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df1394e490 0x55df13d29330 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e04c00640 0 client.17085 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2390805419 >> 172.21.15.32:0/2390805419 conn(0x55df1396b740 msgr2=0x55df13d209b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2390805419 shutdown_connections 2024-08-31T07:20:10.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2390805419 wait complete. 2024-08-31T07:20:10.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.016+0000 7f3e28e4c740 20 client.17085 trim_cache size 0 max 16384 2024-08-31T07:20:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e28e4c740 1 Processor -- start 2024-08-31T07:20:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e28e4c740 1 -- start start 2024-08-31T07:20:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 0x55df13d1b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 0x55df13d1f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13977350 0x55df13d2ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1b13d0 con 0x55df13977350 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1b2fd0 con 0x55df13956670 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1b1aa0 con 0x55df1394e490 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e1be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13977350 0x55df13d2ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e1b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 0x55df13d1b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e1b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 0x55df13d1b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34868/0 (socket says 172.21.15.32:34868) 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.035+0000 7f3e1b400640 1 -- 172.21.15.32:0/753019232 learned_addr learned my addr 172.21.15.32:0/753019232 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.036+0000 7f3e1aa00640 1 --2- 172.21.15.32:0/753019232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 0x55df13d1f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.036+0000 7f3e1b400640 1 -- 172.21.15.32:0/753019232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 msgr2=0x55df13d1f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.036+0000 7f3e1b400640 1 --2- 172.21.15.32:0/753019232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 0x55df13d1f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.036+0000 7f3e1b400640 1 -- 172.21.15.32:0/753019232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13977350 msgr2=0x55df13d2ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.036+0000 7f3e1b400640 1 --2- 172.21.15.32:0/753019232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13977350 0x55df13d2ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.036+0000 7f3e1b400640 1 -- 172.21.15.32:0/753019232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df13d1fcf0 con 0x55df1394e490 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.036+0000 7f3e1b400640 1 --2- 172.21.15.32:0/753019232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 0x55df13d1b5a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c006010 tx=0x7f3e0c031710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d597891dc8fb6841 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.036+0000 7f3e07400640 1 -- 172.21.15.32:0/753019232 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c061070 con 0x55df1394e490 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.036+0000 7f3e07400640 1 -- 172.21.15.32:0/753019232 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0c0044b0 con 0x55df1394e490 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.037+0000 7f3e07400640 1 -- 172.21.15.32:0/753019232 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c04d050 con 0x55df1394e490 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.037+0000 7f3e28e4c740 1 -- 172.21.15.32:0/753019232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 msgr2=0x55df13d1b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.037+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/753019232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 0x55df13d1b5a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c006010 tx=0x7f3e0c031710 comp rx=0 tx=0).stop 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.037+0000 7f3e28e4c740 1 -- 172.21.15.32:0/753019232 shutdown_connections 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.037+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/753019232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13977350 0x55df13d2ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.037+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/753019232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df13956670 0x55df13d1f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.037+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/753019232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df1394e490 0x55df13d1b5a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.037+0000 7f3e28e4c740 1 -- 172.21.15.32:0/753019232 >> 172.21.15.32:0/753019232 conn(0x55df1396b740 msgr2=0x55df13d2a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.037+0000 7f3e28e4c740 1 -- 172.21.15.32:0/753019232 shutdown_connections 2024-08-31T07:20:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.038+0000 7f3e28e4c740 1 -- 172.21.15.32:0/753019232 wait complete. 2024-08-31T07:20:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.038+0000 7f3e28e4c740 1 -- start start 2024-08-31T07:20:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.038+0000 7f3e28e4c740 1 Processor -- start 2024-08-31T07:20:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df1394e490 0x55df13d2d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13956670 0x55df13d2da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e28e4c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 0x55df13da4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1b2840 con 0x55df13956670 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1b2170 con 0x55df1394e490 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e28e4c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1af3d0 con 0x55df13977350 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e28e4c740 10 client.0 _ll_register_callbacks cb 0x7f3e0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e1aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13956670 0x55df13d2da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e1b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df1394e490 0x55df13d2d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e1b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df1394e490 0x55df13d2d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48150/0 (socket says 172.21.15.32:48150) 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e1b400640 1 -- 172.21.15.32:0/2464453339 learned_addr learned my addr 172.21.15.32:0/2464453339 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e1be00640 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 0x55df13da4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e1b400640 1 -- 172.21.15.32:0/2464453339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 msgr2=0x55df13da4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e1b400640 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 0x55df13da4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e1b400640 1 -- 172.21.15.32:0/2464453339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13956670 msgr2=0x55df13d2da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e1b400640 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13956670 0x55df13d2da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.039+0000 7f3e1b400640 1 -- 172.21.15.32:0/2464453339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df13da54b0 con 0x55df1394e490 2024-08-31T07:20:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.040+0000 7f3e1b400640 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df1394e490 0x55df13d2d550 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c000c00 tx=0x7f3e0c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c18785144ae265 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.040+0000 7f3e04c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.040+0000 7f3e04c00640 1 -- 172.21.15.32:0/2464453339 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c06a020 con 0x55df1394e490 2024-08-31T07:20:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.040+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2464453339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e0c0044b0 con 0x55df1394e490 2024-08-31T07:20:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.040+0000 7f3e04c00640 1 -- 172.21.15.32:0/2464453339 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8485+0+0 (secure 0 0 0) 0x7f3e0c004010 con 0x55df1394e490 2024-08-31T07:20:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.040+0000 7f3e04c00640 1 -- 172.21.15.32:0/2464453339 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3e0c075050 con 0x55df1394e490 2024-08-31T07:20:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.040+0000 7f3e28e4c740 10 client.27110 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.040+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2464453339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55df13da65b0 con 0x55df1394e490 2024-08-31T07:20:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.041+0000 7f3e04c00640 1 -- 172.21.15.32:0/2464453339 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0c0044b0 con 0x55df1394e490 2024-08-31T07:20:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.041+0000 7f3e04c00640 1 -- 172.21.15.32:0/2464453339 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f3e0c0664e0 con 0x55df1394e490 2024-08-31T07:20:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.041+0000 7f3e28e4c740 10 client.27110 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.041+0000 7f3e28e4c740 10 client.27110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:20:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.041+0000 7f3e28e4c740 20 client.27110 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.041+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55df1390b080 0x55df13da8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:10.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.041+0000 7f3e28e4c740 4 client.27110 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:20:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.041+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2464453339 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55df13c91140 con 0x55df1390b080 2024-08-31T07:20:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.041+0000 7f3e1aa00640 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55df1390b080 0x55df13da8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.042+0000 7f3e1aa00640 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55df1390b080 0x55df13da8d30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f9e1acdcb6bee1a server_cookie=4d98fc2f374648c7 in_seq=0 out_seq=0 2024-08-31T07:20:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.042+0000 7f3e04c00640 10 client.27110 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e04c00640 1 -- 172.21.15.32:0/2464453339 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55df13c91140 con 0x55df1390b080 2024-08-31T07:20:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e04c00640 10 client.27110 handle_command_reply: tid=0 2024-08-31T07:20:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 client.27110 shutdown 2024-08-31T07:20:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2464453339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df1394e490 msgr2=0x55df13d2d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df1394e490 0x55df13d2d550 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c000c00 tx=0x7f3e0c003820 comp rx=0 tx=0).stop 2024-08-31T07:20:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2464453339 shutdown_connections 2024-08-31T07:20:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55df1390b080 0x55df13da8d30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df13977350 0x55df13da4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55df13956670 0x55df13d2da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 --2- 172.21.15.32:0/2464453339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55df1394e490 0x55df13d2d550 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e04c00640 0 client.27110 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2464453339 >> 172.21.15.32:0/2464453339 conn(0x55df1396b740 msgr2=0x55df13d21290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2464453339 shutdown_connections 2024-08-31T07:20:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 1 -- 172.21.15.32:0/2464453339 wait complete. 2024-08-31T07:20:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.044+0000 7f3e28e4c740 20 client.27110 trim_cache size 0 max 16384 2024-08-31T07:20:10.047 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:20:10.047 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:20:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22", 2024-08-31T07:20:10.048 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:20:10.048 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:20:10.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 -- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e000b4bb0 msgr2=0x7f3e000b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:10.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e000b4bb0 0x7f3e000b70c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c031440 tx=0x7f3e0c032ed0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 -- 172.21.15.32:0/1662742386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1c109070 msgr2=0x7f3e1c19c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1c109070 0x7f3e1c19c8a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800e9f0 tx=0x7f3e0800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 -- 172.21.15.32:0/1662742386 shutdown_connections 2024-08-31T07:20:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3e000b4bb0 0x7f3e000b70c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e1c1099c0 0x7f3e1c1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1c109070 0x7f3e1c19c8a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 --2- 172.21.15.32:0/1662742386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e1c103730 0x7f3e1c19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 -- 172.21.15.32:0/1662742386 >> 172.21.15.32:0/1662742386 conn(0x7f3e1c0ff180 msgr2=0x7f3e1c10e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 -- 172.21.15.32:0/1662742386 shutdown_connections 2024-08-31T07:20:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:10.045+0000 7f3e20c00640 1 -- 172.21.15.32:0/1662742386 wait complete. 2024-08-31T07:20:10.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:10 smithi032 ceph-mon[17252]: pgmap v593: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-08-31T07:20:10.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:10 smithi073 ceph-mon[21995]: pgmap v593: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-08-31T07:20:10.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:10 smithi084 ceph-mon[22387]: pgmap v593: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-08-31T07:20:10.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 2.96 MB/sec execute 239 sec latency 23204.468 ms 2024-08-31T07:20:10.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:20:11.173 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:20:11.173 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.173 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:20:11.173 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:20:11.173 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:20:11.173 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:20:11.173 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.173 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:20:11.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:20:11.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:20:11.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:20:11.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:11 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.314 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:20:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.431 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:11 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:20:11.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:20:11.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:20:11.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:11 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:20:11.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 2.95 MB/sec execute 240 sec latency 24211.119 ms 2024-08-31T07:20:11.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:20:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.070+0000 7fdd40000640 1 -- 172.21.15.32:0/2096646156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd38109260 msgr2=0x7fdd38109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.070+0000 7fdd40000640 1 --2- 172.21.15.32:0/2096646156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd38109260 0x7fdd38109670 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009900 tx=0x7fdd2402a410 comp rx=0 tx=0).stop 2024-08-31T07:20:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.070+0000 7fdd40000640 1 -- 172.21.15.32:0/2096646156 shutdown_connections 2024-08-31T07:20:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.070+0000 7fdd40000640 1 --2- 172.21.15.32:0/2096646156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd38109bb0 0x7fdd381103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.070+0000 7fdd40000640 1 --2- 172.21.15.32:0/2096646156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd38109260 0x7fdd38109670 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.070+0000 7fdd40000640 1 --2- 172.21.15.32:0/2096646156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd38103920 0x7fdd38103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.070+0000 7fdd40000640 1 -- 172.21.15.32:0/2096646156 >> 172.21.15.32:0/2096646156 conn(0x7fdd380ff3b0 msgr2=0x7fdd38101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.071+0000 7fdd40000640 1 -- 172.21.15.32:0/2096646156 shutdown_connections 2024-08-31T07:20:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.071+0000 7fdd40000640 1 -- 172.21.15.32:0/2096646156 wait complete. 2024-08-31T07:20:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.072+0000 7fdd40000640 1 Processor -- start 2024-08-31T07:20:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.072+0000 7fdd40000640 1 -- start start 2024-08-31T07:20:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.072+0000 7fdd40000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd38103920 0x7fdd3819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.072+0000 7fdd40000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd38109260 0x7fdd3819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.072+0000 7fdd40000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd38109bb0 0x7fdd381a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.072+0000 7fdd40000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdd38113870 con 0x7fdd38109bb0 2024-08-31T07:20:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.072+0000 7fdd40000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381136f0 con 0x7fdd38103920 2024-08-31T07:20:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.072+0000 7fdd40000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381139f0 con 0x7fdd38109260 2024-08-31T07:20:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd38109bb0 0x7fdd381a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd38103920 0x7fdd3819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd38109bb0 0x7fdd381a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54164/0 (socket says 172.21.15.32:54164) 2024-08-31T07:20:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3e000640 1 -- 172.21.15.32:0/1502650081 learned_addr learned my addr 172.21.15.32:0/1502650081 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3cc00640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd38109260 0x7fdd3819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3d600640 1 -- 172.21.15.32:0/1502650081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd38109260 msgr2=0x7fdd3819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3d600640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd38109260 0x7fdd3819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3d600640 1 -- 172.21.15.32:0/1502650081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd38109bb0 msgr2=0x7fdd381a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3d600640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd38109bb0 0x7fdd381a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3d600640 1 -- 172.21.15.32:0/1502650081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd381a23d0 con 0x7fdd38103920 2024-08-31T07:20:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.073+0000 7fdd3e000640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd38109bb0 0x7fdd381a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.074+0000 7fdd3d600640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd38103920 0x7fdd3819c460 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00e9f0 tx=0x7fdd2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:12.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.074+0000 7fdd36000640 1 -- 172.21.15.32:0/1502650081 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c00ccc0 con 0x7fdd38103920 2024-08-31T07:20:12.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.074+0000 7fdd40000640 1 -- 172.21.15.32:0/1502650081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd381a26c0 con 0x7fdd38103920 2024-08-31T07:20:12.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.074+0000 7fdd36000640 1 -- 172.21.15.32:0/1502650081 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd2c005020 con 0x7fdd38103920 2024-08-31T07:20:12.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.074+0000 7fdd36000640 1 -- 172.21.15.32:0/1502650081 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c004a00 con 0x7fdd38103920 2024-08-31T07:20:12.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.074+0000 7fdd40000640 1 -- 172.21.15.32:0/1502650081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd381a8e20 con 0x7fdd38103920 2024-08-31T07:20:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.077+0000 7fdd46dd3740 1 Processor -- start 2024-08-31T07:20:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.077+0000 7fdd46dd3740 1 -- start start 2024-08-31T07:20:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.078+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740df2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.078+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740df2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.078+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740df32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.078+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e740d69770 con 0x55e740a241b0 2024-08-31T07:20:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.078+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e740d58d00 con 0x55e740a2c390 2024-08-31T07:20:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.078+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e740d5ccb0 con 0x55e740a4d070 2024-08-31T07:20:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.078+0000 7fdd3e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740df2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.078+0000 7fdd3e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740df2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34904/0 (socket says 172.21.15.32:34904) 2024-08-31T07:20:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.078+0000 7fdd3e000640 1 -- 172.21.15.32:0/2067210535 learned_addr learned my addr 172.21.15.32:0/2067210535 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.079+0000 7fdd3e000640 1 -- 172.21.15.32:0/2067210535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 msgr2=0x55e740df32f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.079+0000 7fdd3e000640 1 --2- 172.21.15.32:0/2067210535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740df32f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.079+0000 7fdd3e000640 1 -- 172.21.15.32:0/2067210535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 msgr2=0x55e740df2db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.079+0000 7fdd3e000640 1 --2- 172.21.15.32:0/2067210535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740df2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.079+0000 7fdd3e000640 1 -- 172.21.15.32:0/2067210535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e740ab8650 con 0x55e740a4d070 2024-08-31T07:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.079+0000 7fdd3e000640 1 --2- 172.21.15.32:0/2067210535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740df2870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdd300118b0 tx=0x7fdd30011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a14ca29cd77618d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.079+0000 7fdd2a600640 1 -- 172.21.15.32:0/2067210535 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3001e070 con 0x55e740a4d070 2024-08-31T07:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.079+0000 7fdd2a600640 1 -- 172.21.15.32:0/2067210535 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3000fe70 con 0x55e740a4d070 2024-08-31T07:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.079+0000 7fdd36000640 1 -- 172.21.15.32:0/1502650081 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fdd2c004ba0 con 0x7fdd38103920 2024-08-31T07:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.079+0000 7fdd2a600640 1 -- 172.21.15.32:0/2067210535 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30010bf0 con 0x55e740a4d070 2024-08-31T07:20:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.081+0000 7fdd46dd3740 1 -- 172.21.15.32:0/2067210535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 msgr2=0x55e740df2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.081+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/2067210535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740df2870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdd300118b0 tx=0x7fdd30011d70 comp rx=0 tx=0).stop 2024-08-31T07:20:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.081+0000 7fdd46dd3740 1 -- 172.21.15.32:0/2067210535 shutdown_connections 2024-08-31T07:20:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.081+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/2067210535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740df32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.081+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/2067210535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740df2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.081+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/2067210535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740df2870 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.081+0000 7fdd46dd3740 1 -- 172.21.15.32:0/2067210535 >> 172.21.15.32:0/2067210535 conn(0x55e740a41460 msgr2=0x55e740df0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.081+0000 7fdd46dd3740 1 -- 172.21.15.32:0/2067210535 shutdown_connections 2024-08-31T07:20:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.081+0000 7fdd46dd3740 1 -- 172.21.15.32:0/2067210535 wait complete. 2024-08-31T07:20:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.082+0000 7fdd36000640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdd200b4bb0 0x7fdd200b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.082+0000 7fdd46dd3740 1 -- start start 2024-08-31T07:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.082+0000 7fdd36000640 1 -- 172.21.15.32:0/1502650081 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7fdd2c0e54b0 con 0x7fdd38103920 2024-08-31T07:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.082+0000 7fdd46dd3740 1 Processor -- start 2024-08-31T07:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.082+0000 7fdd3e000640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdd200b4bb0 0x7fdd200b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a241b0 0x55e740dfe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a2c390 0x55e740dfe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a4d070 0x55e740e0a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e740bbb700 con 0x55e740a4d070 2024-08-31T07:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e740cdfcb0 con 0x55e740a241b0 2024-08-31T07:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e740e01050 con 0x55e740a2c390 2024-08-31T07:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd46dd3740 10 client.0 _ll_register_callbacks cb 0x55e740d05430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:12.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd3e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a4d070 0x55e740e0a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd3e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a4d070 0x55e740e0a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54178/0 (socket says 172.21.15.32:54178) 2024-08-31T07:20:12.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd3e000640 1 -- 172.21.15.32:0/68683980 learned_addr learned my addr 172.21.15.32:0/68683980 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd3cc00640 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a2c390 0x55e740dfe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd3d600640 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a241b0 0x55e740dfe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.083+0000 7fdd3e000640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdd200b4bb0 0x7fdd200b70c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdd30010650 tx=0x7fdd30009a30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.084+0000 7fdd3cc00640 1 -- 172.21.15.32:0/68683980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a241b0 msgr2=0x55e740dfe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.084+0000 7fdd3cc00640 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a241b0 0x55e740dfe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.084+0000 7fdd3cc00640 1 -- 172.21.15.32:0/68683980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a4d070 msgr2=0x55e740e0a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.084+0000 7fdd3cc00640 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a4d070 0x55e740e0a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.084+0000 7fdd3cc00640 1 -- 172.21.15.32:0/68683980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e740d20870 con 0x55e740a2c390 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.084+0000 7fdd3d600640 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a241b0 0x55e740dfe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.084+0000 7fdd3e000640 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a4d070 0x55e740e0a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.084+0000 7fdd3cc00640 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a2c390 0x55e740dfe930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdd2402a930 tx=0x7fdd24032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3357774a998f0c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.084+0000 7fdd1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:20:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.085+0000 7fdd1ea00640 1 -- 172.21.15.32:0/68683980 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24040070 con 0x55e740a2c390 2024-08-31T07:20:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.085+0000 7fdd46dd3740 1 -- 172.21.15.32:0/68683980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3000fe70 con 0x55e740a2c390 2024-08-31T07:20:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.085+0000 7fdd1ea00640 1 -- 172.21.15.32:0/68683980 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7fdd240318b0 con 0x55e740a2c390 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.085+0000 7fdd1ea00640 1 -- 172.21.15.32:0/68683980 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdd24035040 con 0x55e740a2c390 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.085+0000 7fdd46dd3740 10 client.27121 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.085+0000 7fdd46dd3740 1 -- 172.21.15.32:0/68683980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e740e0d060 con 0x55e740a2c390 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.086+0000 7fdd1ea00640 1 -- 172.21.15.32:0/68683980 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3000fe70 con 0x55e740a2c390 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.086+0000 7fdd1ea00640 1 -- 172.21.15.32:0/68683980 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fdd24004750 con 0x55e740a2c390 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.086+0000 7fdd46dd3740 10 client.27121 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.086+0000 7fdd46dd3740 10 client.27121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.086+0000 7fdd46dd3740 20 client.27121 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.086+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e740a0b9f0 0x55e740e10520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.086+0000 7fdd46dd3740 4 client.27121 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.086+0000 7fdd46dd3740 1 -- 172.21.15.32:0/68683980 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e740d6b6c0 con 0x55e740a0b9f0 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.086+0000 7fdd3d600640 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e740a0b9f0 0x55e740e10520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.087+0000 7fdd3d600640 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e740a0b9f0 0x55e740e10520 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c231235149c806 server_cookie=b1c7c82a9f305f55 in_seq=0 out_seq=0 2024-08-31T07:20:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.087+0000 7fdd1ea00640 10 client.27121 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd1ea00640 1 -- 172.21.15.32:0/68683980 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e740d6b6c0 con 0x55e740a0b9f0 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd1ea00640 10 client.27121 handle_command_reply: tid=0 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd46dd3740 1 client.27121 shutdown 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd46dd3740 1 -- 172.21.15.32:0/68683980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a2c390 msgr2=0x55e740dfe930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a2c390 0x55e740dfe930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdd2402a930 tx=0x7fdd24032d50 comp rx=0 tx=0).stop 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd46dd3740 1 -- 172.21.15.32:0/68683980 shutdown_connections 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e740a0b9f0 0x55e740e10520 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a4d070 0x55e740e0a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a2c390 0x55e740dfe930 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/68683980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a241b0 0x55e740dfe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd1ea00640 0 client.27121 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.088+0000 7fdd46dd3740 1 -- 172.21.15.32:0/68683980 >> 172.21.15.32:0/68683980 conn(0x55e740a41460 msgr2=0x55e740df5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:12.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.089+0000 7fdd46dd3740 1 -- 172.21.15.32:0/68683980 shutdown_connections 2024-08-31T07:20:12.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.089+0000 7fdd46dd3740 1 -- 172.21.15.32:0/68683980 wait complete. 2024-08-31T07:20:12.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.089+0000 7fdd46dd3740 20 client.27121 trim_cache size 0 max 16384 2024-08-31T07:20:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.122+0000 7fdd46dd3740 1 Processor -- start 2024-08-31T07:20:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.122+0000 7fdd46dd3740 1 -- start start 2024-08-31T07:20:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.122+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740df93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.122+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740dfb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.122+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740dfb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.122+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381ad290 con 0x55e740a241b0 2024-08-31T07:20:12.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.122+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381aee90 con 0x55e740a2c390 2024-08-31T07:20:12.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.122+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381ad960 con 0x55e740a4d070 2024-08-31T07:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.123+0000 7fdd3d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740df93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.123+0000 7fdd3d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740df93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54188/0 (socket says 172.21.15.32:54188) 2024-08-31T07:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.123+0000 7fdd3d600640 1 -- 172.21.15.32:0/2800115289 learned_addr learned my addr 172.21.15.32:0/2800115289 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.123+0000 7fdd3e000640 1 --2- 172.21.15.32:0/2800115289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740dfb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.123+0000 7fdd3cc00640 1 --2- 172.21.15.32:0/2800115289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740dfb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.123+0000 7fdd3d600640 1 -- 172.21.15.32:0/2800115289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 msgr2=0x55e740dfb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.123+0000 7fdd3d600640 1 --2- 172.21.15.32:0/2800115289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740dfb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.123+0000 7fdd3d600640 1 -- 172.21.15.32:0/2800115289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 msgr2=0x55e740dfb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.123+0000 7fdd3d600640 1 --2- 172.21.15.32:0/2800115289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740dfb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.123+0000 7fdd3d600640 1 -- 172.21.15.32:0/2800115289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e740dff4b0 con 0x55e740a241b0 2024-08-31T07:20:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.124+0000 7fdd3d600640 1 --2- 172.21.15.32:0/2800115289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740df93c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0abb20 tx=0x7fdd2c0a73e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cabbe20abf244ca server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.124+0000 7fdd29c00640 1 -- 172.21.15.32:0/2800115289 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c01c040 con 0x55e740a241b0 2024-08-31T07:20:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.124+0000 7fdd29c00640 1 -- 172.21.15.32:0/2800115289 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd2c0b01d0 con 0x55e740a241b0 2024-08-31T07:20:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.124+0000 7fdd29c00640 1 -- 172.21.15.32:0/2800115289 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c0a7e20 con 0x55e740a241b0 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.125+0000 7fdd46dd3740 1 -- 172.21.15.32:0/2800115289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 msgr2=0x55e740df93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.125+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/2800115289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740df93c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0abb20 tx=0x7fdd2c0a73e0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.125+0000 7fdd46dd3740 1 -- 172.21.15.32:0/2800115289 shutdown_connections 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.125+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/2800115289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740dfb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.125+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/2800115289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740dfb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.125+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/2800115289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740df93c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.125+0000 7fdd46dd3740 1 -- 172.21.15.32:0/2800115289 >> 172.21.15.32:0/2800115289 conn(0x55e740a41460 msgr2=0x55e740dfca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.125+0000 7fdd46dd3740 1 -- 172.21.15.32:0/2800115289 shutdown_connections 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.126+0000 7fdd46dd3740 1 -- 172.21.15.32:0/2800115289 wait complete. 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.126+0000 7fdd46dd3740 1 -- start start 2024-08-31T07:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.126+0000 7fdd46dd3740 1 Processor -- start 2024-08-31T07:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740e06900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740e01260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd46dd3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740e017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381ae700 con 0x55e740a241b0 2024-08-31T07:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381ae030 con 0x55e740a2c390 2024-08-31T07:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd46dd3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381ab2d0 con 0x55e740a4d070 2024-08-31T07:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd46dd3740 10 client.0 _ll_register_callbacks cb 0x7fdd24009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd3d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740e06900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd3d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740e06900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54204/0 (socket says 172.21.15.32:54204) 2024-08-31T07:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd3e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740e017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.127+0000 7fdd3d600640 1 -- 172.21.15.32:0/130609340 learned_addr learned my addr 172.21.15.32:0/130609340 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.128+0000 7fdd3cc00640 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740e01260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.128+0000 7fdd3e000640 1 -- 172.21.15.32:0/130609340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 msgr2=0x55e740e01260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.128+0000 7fdd3e000640 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740e01260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.128+0000 7fdd3e000640 1 -- 172.21.15.32:0/130609340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 msgr2=0x55e740e06900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.128+0000 7fdd3e000640 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740e06900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.128+0000 7fdd3e000640 1 -- 172.21.15.32:0/130609340 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e740e01de0 con 0x55e740a4d070 2024-08-31T07:20:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.128+0000 7fdd3cc00640 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740e01260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.128+0000 7fdd3e000640 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740e017a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000f7b0 tx=0x7fdd30011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b31efea4a387f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.128+0000 7fdd1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:20:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.129+0000 7fdd1ea00640 1 -- 172.21.15.32:0/130609340 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30040060 con 0x55e740a4d070 2024-08-31T07:20:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.129+0000 7fdd1ea00640 1 -- 172.21.15.32:0/130609340 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7fdd30019620 con 0x55e740a4d070 2024-08-31T07:20:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.129+0000 7fdd46dd3740 1 -- 172.21.15.32:0/130609340 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd2c0b01d0 con 0x55e740a4d070 2024-08-31T07:20:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.130+0000 7fdd1ea00640 1 -- 172.21.15.32:0/130609340 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdd3001e040 con 0x55e740a4d070 2024-08-31T07:20:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.130+0000 7fdd46dd3740 10 client.27127 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.130+0000 7fdd46dd3740 1 -- 172.21.15.32:0/130609340 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e740e7bca0 con 0x55e740a4d070 2024-08-31T07:20:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.130+0000 7fdd1ea00640 1 -- 172.21.15.32:0/130609340 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd2c0b01d0 con 0x55e740a4d070 2024-08-31T07:20:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.130+0000 7fdd1ea00640 1 -- 172.21.15.32:0/130609340 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fdd30021030 con 0x55e740a4d070 2024-08-31T07:20:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.131+0000 7fdd46dd3740 10 client.27127 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.131+0000 7fdd46dd3740 10 client.27127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:20:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.131+0000 7fdd46dd3740 20 client.27127 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.131+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e7409e0da0 0x55e740e7e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.131+0000 7fdd46dd3740 4 client.27127 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:20:12.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.131+0000 7fdd46dd3740 1 -- 172.21.15.32:0/130609340 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e740d67160 con 0x55e7409e0da0 2024-08-31T07:20:12.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.131+0000 7fdd3d600640 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e7409e0da0 0x55e740e7e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:12.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.132+0000 7fdd1ea00640 10 client.27127 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:12.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.132+0000 7fdd3d600640 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e7409e0da0 0x55e740e7e390 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a54e6000e52a4d server_cookie=e12a94ca6076197d in_seq=0 out_seq=0 2024-08-31T07:20:12.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd1ea00640 1 -- 172.21.15.32:0/130609340 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x55e740d67160 con 0x55e7409e0da0 2024-08-31T07:20:12.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd1ea00640 10 client.27127 handle_command_reply: tid=0 2024-08-31T07:20:12.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 client.27127 shutdown 2024-08-31T07:20:12.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 -- 172.21.15.32:0/130609340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 msgr2=0x55e740e017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:12.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740e017a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000f7b0 tx=0x7fdd30011d70 comp rx=0 tx=0).stop 2024-08-31T07:20:12.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 -- 172.21.15.32:0/130609340 shutdown_connections 2024-08-31T07:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e7409e0da0 0x55e740e7e390 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e740a4d070 0x55e740e017a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e740a2c390 0x55e740e01260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 --2- 172.21.15.32:0/130609340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e740a241b0 0x55e740e06900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 -- 172.21.15.32:0/130609340 >> 172.21.15.32:0/130609340 conn(0x55e740a41460 msgr2=0x55e740df93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd1ea00640 0 client.27127 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 -- 172.21.15.32:0/130609340 shutdown_connections 2024-08-31T07:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.133+0000 7fdd46dd3740 1 -- 172.21.15.32:0/130609340 wait complete. 2024-08-31T07:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.134+0000 7fdd46dd3740 20 client.27127 trim_cache size 0 max 16384 2024-08-31T07:20:12.136 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:20:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:20:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:20:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:20:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22", 2024-08-31T07:20:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:20:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:12.136 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:12.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:20:12.138 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:12.139 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:20:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:20:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:12.144 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.134+0000 7fdd40000640 1 -- 172.21.15.32:0/1502650081 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdd200b4bb0 msgr2=0x7fdd200b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.134+0000 7fdd40000640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdd200b4bb0 0x7fdd200b70c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdd30010650 tx=0x7fdd30009a30 comp rx=0 tx=0).stop 2024-08-31T07:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.134+0000 7fdd40000640 1 -- 172.21.15.32:0/1502650081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd38103920 msgr2=0x7fdd3819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.134+0000 7fdd40000640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd38103920 0x7fdd3819c460 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00e9f0 tx=0x7fdd2c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.135+0000 7fdd40000640 1 -- 172.21.15.32:0/1502650081 shutdown_connections 2024-08-31T07:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.135+0000 7fdd40000640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdd200b4bb0 0x7fdd200b70c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.135+0000 7fdd40000640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd38109bb0 0x7fdd381a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.135+0000 7fdd40000640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd38109260 0x7fdd3819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.135+0000 7fdd40000640 1 --2- 172.21.15.32:0/1502650081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd38103920 0x7fdd3819c460 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.135+0000 7fdd40000640 1 -- 172.21.15.32:0/1502650081 >> 172.21.15.32:0/1502650081 conn(0x7fdd380ff3b0 msgr2=0x7fdd3810e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.135+0000 7fdd40000640 1 -- 172.21.15.32:0/1502650081 shutdown_connections 2024-08-31T07:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:12.135+0000 7fdd40000640 1 -- 172.21.15.32:0/1502650081 wait complete. 2024-08-31T07:20:12.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:12 smithi032 ceph-mon[17252]: pgmap v594: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-08-31T07:20:12.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:12 smithi032 ceph-mon[17252]: osdmap e84: 12 total, 12 up, 12 in 2024-08-31T07:20:12.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:12 smithi073 ceph-mon[21995]: pgmap v594: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-08-31T07:20:12.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:12 smithi073 ceph-mon[21995]: osdmap e84: 12 total, 12 up, 12 in 2024-08-31T07:20:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:12 smithi084 ceph-mon[22387]: pgmap v594: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-08-31T07:20:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:12 smithi084 ceph-mon[22387]: osdmap e84: 12 total, 12 up, 12 in 2024-08-31T07:20:12.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 2.93 MB/sec execute 241 sec latency 25211.176 ms 2024-08-31T07:20:13.059 INFO:tasks.cephfs.filesystem:scrub status for tag:fa02e0e6-4fcf-48e2-9f34-5121c7f65c22 - {'path': '/', 'tag': 'fa02e0e6-4fcf-48e2-9f34-5121c7f65c22', 'options': 'recursive,force'} 2024-08-31T07:20:13.096 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:13.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 2.92 MB/sec execute 242 sec latency 26211.243 ms 2024-08-31T07:20:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.812+0000 7f94b7000640 1 -- 172.21.15.32:0/518937594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94b0103750 msgr2=0x7f94b0103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.812+0000 7f94b7000640 1 --2- 172.21.15.32:0/518937594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94b0103750 0x7f94b0103b60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009be0 tx=0x7f94a402a6c0 comp rx=0 tx=0).stop 2024-08-31T07:20:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.812+0000 7f94b7000640 1 -- 172.21.15.32:0/518937594 shutdown_connections 2024-08-31T07:20:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.812+0000 7f94b7000640 1 --2- 172.21.15.32:0/518937594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b01099e0 0x7f94b0110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.812+0000 7f94b7000640 1 --2- 172.21.15.32:0/518937594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b0109090 0x7f94b01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.812+0000 7f94b7000640 1 --2- 172.21.15.32:0/518937594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94b0103750 0x7f94b0103b60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.812+0000 7f94b7000640 1 -- 172.21.15.32:0/518937594 >> 172.21.15.32:0/518937594 conn(0x7f94b00ff1e0 msgr2=0x7f94b0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.812+0000 7f94b7000640 1 -- 172.21.15.32:0/518937594 shutdown_connections 2024-08-31T07:20:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.812+0000 7f94b7000640 1 -- 172.21.15.32:0/518937594 wait complete. 2024-08-31T07:20:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.814+0000 7f94b7000640 1 Processor -- start 2024-08-31T07:20:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.814+0000 7f94b7000640 1 -- start start 2024-08-31T07:20:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.814+0000 7f94b7000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b0103750 0x7f94b019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.814+0000 7f94b7000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94b0109090 0x7f94b019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:13.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.814+0000 7f94b7000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b01099e0 0x7f94b01a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.814+0000 7f94b7000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0113690 con 0x7f94b01099e0 2024-08-31T07:20:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.814+0000 7f94b7000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0113510 con 0x7f94b0109090 2024-08-31T07:20:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.814+0000 7f94b7000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0113810 con 0x7f94b0103750 2024-08-31T07:20:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.814+0000 7f94b5c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b0103750 0x7f94b019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.814+0000 7f94b5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94b0109090 0x7f94b019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.815+0000 7f94b5c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b0103750 0x7f94b019c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34966/0 (socket says 172.21.15.32:34966) 2024-08-31T07:20:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.815+0000 7f94b5c00640 1 -- 172.21.15.32:0/1835904369 learned_addr learned my addr 172.21.15.32:0/1835904369 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.815+0000 7f94b6600640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b01099e0 0x7f94b01a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.815+0000 7f94b5c00640 1 -- 172.21.15.32:0/1835904369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94b0109090 msgr2=0x7f94b019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.815+0000 7f94b5c00640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94b0109090 0x7f94b019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.815+0000 7f94b5c00640 1 -- 172.21.15.32:0/1835904369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b01099e0 msgr2=0x7f94b01a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.815+0000 7f94b5c00640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b01099e0 0x7f94b01a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.815+0000 7f94b5c00640 1 -- 172.21.15.32:0/1835904369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b01a2230 con 0x7f94b0103750 2024-08-31T07:20:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.815+0000 7f94b5200640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94b0109090 0x7f94b019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.815+0000 7f94b5c00640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b0103750 0x7f94b019c2c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f94a4004af0 tx=0x7f94a4003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.816+0000 7f94ae000640 1 -- 172.21.15.32:0/1835904369 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4040020 con 0x7f94b0103750 2024-08-31T07:20:13.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.816+0000 7f94b7000640 1 -- 172.21.15.32:0/1835904369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b01a24c0 con 0x7f94b0103750 2024-08-31T07:20:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.816+0000 7f94ae000640 1 -- 172.21.15.32:0/1835904369 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a4003e10 con 0x7f94b0103750 2024-08-31T07:20:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.816+0000 7f94ae000640 1 -- 172.21.15.32:0/1835904369 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4036040 con 0x7f94b0103750 2024-08-31T07:20:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.816+0000 7f94b7000640 1 -- 172.21.15.32:0/1835904369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b01a8c80 con 0x7f94b0103750 2024-08-31T07:20:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.817+0000 7f94b7000640 1 -- 172.21.15.32:0/1835904369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b0103b60 con 0x7f94b0103750 2024-08-31T07:20:13.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.820+0000 7f94ae000640 1 -- 172.21.15.32:0/1835904369 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f94a4043020 con 0x7f94b0103750 2024-08-31T07:20:13.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.822+0000 7f94ae000640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94980b4ca0 0x7f94980b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:13.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.822+0000 7f94ae000640 1 -- 172.21.15.32:0/1835904369 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f94a41064e0 con 0x7f94b0103750 2024-08-31T07:20:13.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.822+0000 7f94b5200640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94980b4ca0 0x7f94980b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:13.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.823+0000 7f94b5200640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94980b4ca0 0x7f94980b71b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f94b0108c40 tx=0x7f949c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:13.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:13.824+0000 7f94ae000640 1 -- 172.21.15.32:0/1835904369 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f94a404a020 con 0x7f94b0103750 2024-08-31T07:20:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.055+0000 7f94b7000640 1 -- 172.21.15.32:0/1835904369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94b019d3f0 con 0x7f94b0103750 2024-08-31T07:20:14.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.056+0000 7f94ae000640 1 -- 172.21.15.32:0/1835904369 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11967 (secure 0 0 0) 0x7f94a40cd2c0 con 0x7f94b0103750 2024-08-31T07:20:14.057 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:20:14.057 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"btime":"2024-08-31T07:19:16:021228+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:19:15.946242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:20:14.057 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 35 2024-08-31T07:20:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 -- 172.21.15.32:0/1835904369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94980b4ca0 msgr2=0x7f94980b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94980b4ca0 0x7f94980b71b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f94b0108c40 tx=0x7f949c005f90 comp rx=0 tx=0).stop 2024-08-31T07:20:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 -- 172.21.15.32:0/1835904369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b0103750 msgr2=0x7f94b019c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b0103750 0x7f94b019c2c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f94a4004af0 tx=0x7f94a4003c20 comp rx=0 tx=0).stop 2024-08-31T07:20:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 -- 172.21.15.32:0/1835904369 shutdown_connections 2024-08-31T07:20:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f94980b4ca0 0x7f94980b71b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b01099e0 0x7f94b01a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94b0109090 0x7f94b019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 --2- 172.21.15.32:0/1835904369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b0103750 0x7f94b019c2c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 -- 172.21.15.32:0/1835904369 >> 172.21.15.32:0/1835904369 conn(0x7f94b00ff1e0 msgr2=0x7f94b010e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 -- 172.21.15.32:0/1835904369 shutdown_connections 2024-08-31T07:20:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:14.059+0000 7f94b7000640 1 -- 172.21.15.32:0/1835904369 wait complete. 2024-08-31T07:20:14.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:14 smithi032 ceph-mon[17252]: pgmap v596: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 993 KiB/s wr, 57 op/s 2024-08-31T07:20:14.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:14 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1835904369' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:14.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:14 smithi073 ceph-mon[21995]: pgmap v596: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 993 KiB/s wr, 57 op/s 2024-08-31T07:20:14.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:14 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1835904369' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:14 smithi084 ceph-mon[22387]: pgmap v596: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 993 KiB/s wr, 57 op/s 2024-08-31T07:20:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:14 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1835904369' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:14.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 2.91 MB/sec execute 243 sec latency 27211.387 ms 2024-08-31T07:20:14.875 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 93e49528 --release --await' 2024-08-31T07:20:14.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 93e49528 --release --await 2024-08-31T07:20:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:15.734 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234423 2.90 MB/sec execute 244 sec latency 28211.497 ms 2024-08-31T07:20:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.945+0000 7f32e7e00640 1 -- 172.21.15.32:0/2188973489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e810b290 msgr2=0x7f32e810b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.945+0000 7f32e7e00640 1 --2- 172.21.15.32:0/2188973489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e810b290 0x7f32e810b6a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009900 tx=0x7f32d402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.945+0000 7f32e7e00640 1 -- 172.21.15.32:0/2188973489 shutdown_connections 2024-08-31T07:20:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.945+0000 7f32e7e00640 1 --2- 172.21.15.32:0/2188973489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e810bbe0 0x7f32e8112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.945+0000 7f32e7e00640 1 --2- 172.21.15.32:0/2188973489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e810b290 0x7f32e810b6a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.945+0000 7f32e7e00640 1 --2- 172.21.15.32:0/2188973489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32e8105950 0x7f32e8105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.945+0000 7f32e7e00640 1 -- 172.21.15.32:0/2188973489 >> 172.21.15.32:0/2188973489 conn(0x7f32e8101420 msgr2=0x7f32e8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.945+0000 7f32e7e00640 1 -- 172.21.15.32:0/2188973489 shutdown_connections 2024-08-31T07:20:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.945+0000 7f32e7e00640 1 -- 172.21.15.32:0/2188973489 wait complete. 2024-08-31T07:20:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.947+0000 7f32e7e00640 1 Processor -- start 2024-08-31T07:20:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.947+0000 7f32e7e00640 1 -- start start 2024-08-31T07:20:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.947+0000 7f32e7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32e8105950 0x7f32e819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.947+0000 7f32e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e810b290 0x7f32e819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.947+0000 7f32e7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e810bbe0 0x7f32e81a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.947+0000 7f32e7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81158a0 con 0x7f32e810b290 2024-08-31T07:20:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.947+0000 7f32e7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32e8115720 con 0x7f32e8105950 2024-08-31T07:20:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.947+0000 7f32e7e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32e8115a20 con 0x7f32e810bbe0 2024-08-31T07:20:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.948+0000 7f32e6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e810b290 0x7f32e819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.948+0000 7f32e6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32e8105950 0x7f32e819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.948+0000 7f32e6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32e8105950 0x7f32e819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54192/0 (socket says 172.21.15.32:54192) 2024-08-31T07:20:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.948+0000 7f32e6a00640 1 -- 172.21.15.32:0/4029061148 learned_addr learned my addr 172.21.15.32:0/4029061148 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:15.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.948+0000 7f32e7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e810bbe0 0x7f32e81a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.948+0000 7f32e6a00640 1 -- 172.21.15.32:0/4029061148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e810bbe0 msgr2=0x7f32e81a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:15.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.948+0000 7f32e6a00640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e810bbe0 0x7f32e81a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.948+0000 7f32e6a00640 1 -- 172.21.15.32:0/4029061148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e810b290 msgr2=0x7f32e819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.948+0000 7f32e6a00640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e810b290 0x7f32e819c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.948+0000 7f32e6a00640 1 -- 172.21.15.32:0/4029061148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e81a2200 con 0x7f32e8105950 2024-08-31T07:20:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.949+0000 7f32e6a00640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32e8105950 0x7f32e819c290 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f32d800ea60 tx=0x7f32d800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.949+0000 7f32df400640 1 -- 172.21.15.32:0/4029061148 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d800cde0 con 0x7f32e8105950 2024-08-31T07:20:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.949+0000 7f32e7e00640 1 -- 172.21.15.32:0/4029061148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e81a24f0 con 0x7f32e8105950 2024-08-31T07:20:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.949+0000 7f32df400640 1 -- 172.21.15.32:0/4029061148 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32d80035f0 con 0x7f32e8105950 2024-08-31T07:20:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.949+0000 7f32e7e00640 1 -- 172.21.15.32:0/4029061148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e81a8c50 con 0x7f32e8105950 2024-08-31T07:20:15.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.949+0000 7f32df400640 1 -- 172.21.15.32:0/4029061148 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8010690 con 0x7f32e8105950 2024-08-31T07:20:15.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.953+0000 7f32f444d740 1 Processor -- start 2024-08-31T07:20:15.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.953+0000 7f32f444d740 1 -- start start 2024-08-31T07:20:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.953+0000 7f32f444d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82bca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.953+0000 7f32f444d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.953+0000 7f32f444d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bcab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.953+0000 7f32f444d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555b82b40a90 con 0x555b82803840 2024-08-31T07:20:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.953+0000 7f32f444d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555b82b30020 con 0x555b827fb660 2024-08-31T07:20:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.953+0000 7f32f444d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555b82b33fd0 con 0x555b82824520 2024-08-31T07:20:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.954+0000 7f32e7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82bca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.954+0000 7f32e6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bcab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.954+0000 7f32e7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82bca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34862/0 (socket says 172.21.15.32:34862) 2024-08-31T07:20:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.954+0000 7f32e7400640 1 -- 172.21.15.32:0/180993333 learned_addr learned my addr 172.21.15.32:0/180993333 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.954+0000 7f32e7400640 1 -- 172.21.15.32:0/180993333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 msgr2=0x555b82bca5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:20:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.954+0000 7f32e7400640 1 --2- 172.21.15.32:0/180993333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bca5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.954+0000 7f32e7400640 1 -- 172.21.15.32:0/180993333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 msgr2=0x555b82bcab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.954+0000 7f32e7400640 1 --2- 172.21.15.32:0/180993333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bcab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.954+0000 7f32e7400640 1 -- 172.21.15.32:0/180993333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b8288fb00 con 0x555b82824520 2024-08-31T07:20:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.954+0000 7f32e7400640 1 --2- 172.21.15.32:0/180993333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82bca090 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009fd0 tx=0x7f32e00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5a2deca79196140 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.955+0000 7f32df400640 1 -- 172.21.15.32:0/4029061148 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f32d8004000 con 0x7f32e8105950 2024-08-31T07:20:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.955+0000 7f32d3e00640 1 -- 172.21.15.32:0/180993333 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0004d20 con 0x555b82824520 2024-08-31T07:20:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.955+0000 7f32d3e00640 1 -- 172.21.15.32:0/180993333 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e0004050 con 0x555b82824520 2024-08-31T07:20:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.955+0000 7f32d3e00640 1 -- 172.21.15.32:0/180993333 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0005240 con 0x555b82824520 2024-08-31T07:20:15.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.956+0000 7f32f444d740 1 -- 172.21.15.32:0/180993333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 msgr2=0x555b82bca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.956+0000 7f32f444d740 1 --2- 172.21.15.32:0/180993333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82bca090 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009fd0 tx=0x7f32e00118d0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.956+0000 7f32f444d740 1 -- 172.21.15.32:0/180993333 shutdown_connections 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.956+0000 7f32f444d740 1 --2- 172.21.15.32:0/180993333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bcab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.956+0000 7f32f444d740 1 --2- 172.21.15.32:0/180993333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.956+0000 7f32f444d740 1 --2- 172.21.15.32:0/180993333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82bca090 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.956+0000 7f32f444d740 1 -- 172.21.15.32:0/180993333 >> 172.21.15.32:0/180993333 conn(0x555b82818910 msgr2=0x555b82bc7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.956+0000 7f32f444d740 1 -- 172.21.15.32:0/180993333 shutdown_connections 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.956+0000 7f32f444d740 1 -- 172.21.15.32:0/180993333 wait complete. 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.957+0000 7f32df400640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32cc0b4c30 0x7f32cc0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.957+0000 7f32f444d740 1 -- start start 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.957+0000 7f32df400640 1 -- 172.21.15.32:0/4029061148 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f32d80e54e0 con 0x7f32e8105950 2024-08-31T07:20:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.957+0000 7f32f444d740 1 Processor -- start 2024-08-31T07:20:15.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.957+0000 7f32e7400640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32cc0b4c30 0x7f32cc0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32f444d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bd7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32f444d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bd54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32f444d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82bd5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32f444d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555b82992bb0 con 0x555b82803840 2024-08-31T07:20:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32f444d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555b82ab6f90 con 0x555b827fb660 2024-08-31T07:20:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32f444d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555b82bd8870 con 0x555b82824520 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32e7400640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32cc0b4c30 0x7f32cc0b7140 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f32e001b040 tx=0x7f32e0004950 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32f444d740 10 client.0 _ll_register_callbacks cb 0x555b82adc710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32e6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bd54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32e6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bd54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47928/0 (socket says 172.21.15.32:47928) 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32e6000640 1 -- 172.21.15.32:0/2497493441 learned_addr learned my addr 172.21.15.32:0/2497493441 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32e7400640 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82bd5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32e6a00640 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bd7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.958+0000 7f32e6000640 1 -- 172.21.15.32:0/2497493441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 msgr2=0x555b82bd5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.959+0000 7f32e6000640 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82bd5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.959+0000 7f32e6000640 1 -- 172.21.15.32:0/2497493441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 msgr2=0x555b82bd7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.959+0000 7f32e6000640 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bd7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.959+0000 7f32e6000640 1 -- 172.21.15.32:0/2497493441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b82906960 con 0x555b82803840 2024-08-31T07:20:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.959+0000 7f32e6000640 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bd54a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009900 tx=0x7f32d4032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992c60d77c64ad74 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.959+0000 7f32d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:20:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.959+0000 7f32d0c00640 1 -- 172.21.15.32:0/2497493441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4035040 con 0x555b82803840 2024-08-31T07:20:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.959+0000 7f32d0c00640 1 -- 172.21.15.32:0/2497493441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f32d40316a0 con 0x555b82803840 2024-08-31T07:20:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.959+0000 7f32f444d740 1 -- 172.21.15.32:0/2497493441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b82acee30 con 0x555b82803840 2024-08-31T07:20:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32d0c00640 1 -- 172.21.15.32:0/2497493441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f32d40318a0 con 0x555b82803840 2024-08-31T07:20:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32f444d740 10 client.17124 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32f444d740 1 -- 172.21.15.32:0/2497493441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555b82be4260 con 0x555b82803840 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32d0c00640 1 -- 172.21.15.32:0/2497493441 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555b82acee30 con 0x555b82803840 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32d0c00640 1 -- 172.21.15.32:0/2497493441 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f32d4004750 con 0x555b82803840 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32f444d740 10 client.17124 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32f444d740 10 client.17124 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32f444d740 20 client.17124 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32f444d740 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555b827e2ea0 0x555b82be7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32f444d740 4 client.17124 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.960+0000 7f32f444d740 1 -- 172.21.15.32:0/2497493441 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b82b429e0 con 0x555b827e2ea0 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.961+0000 7f32e6a00640 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555b827e2ea0 0x555b82be7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.961+0000 7f32e6a00640 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555b827e2ea0 0x555b82be7880 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162a1a44f39d3f55 server_cookie=938b186b53b67476 in_seq=0 out_seq=0 2024-08-31T07:20:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.961+0000 7f32d0c00640 10 client.17124 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32d0c00640 1 -- 172.21.15.32:0/2497493441 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b82b429e0 con 0x555b827e2ea0 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32d0c00640 10 client.17124 handle_command_reply: tid=0 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32f444d740 1 client.17124 shutdown 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32f444d740 1 -- 172.21.15.32:0/2497493441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 msgr2=0x555b82bd54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32f444d740 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bd54a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009900 tx=0x7f32d4032e90 comp rx=0 tx=0).stop 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32f444d740 1 -- 172.21.15.32:0/2497493441 shutdown_connections 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32f444d740 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555b827e2ea0 0x555b82be7880 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32f444d740 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82bd5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32f444d740 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bd54a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32f444d740 1 --2- 172.21.15.32:0/2497493441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bd7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32f444d740 1 -- 172.21.15.32:0/2497493441 >> 172.21.15.32:0/2497493441 conn(0x555b82818910 msgr2=0x555b82bcd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32d0c00640 0 client.17124 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.962+0000 7f32f444d740 1 -- 172.21.15.32:0/2497493441 shutdown_connections 2024-08-31T07:20:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.963+0000 7f32f444d740 1 -- 172.21.15.32:0/2497493441 wait complete. 2024-08-31T07:20:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.963+0000 7f32f444d740 20 client.17124 trim_cache size 0 max 16384 2024-08-31T07:20:15.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.981+0000 7f32f444d740 1 Processor -- start 2024-08-31T07:20:15.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.981+0000 7f32f444d740 1 -- start start 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.981+0000 7f32f444d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b827fb660 0x555b82bcb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.981+0000 7f32f444d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b82803840 0x555b82be1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.981+0000 7f32f444d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82824520 0x555b82bd7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.981+0000 7f32f444d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32c800e6b0 con 0x555b82824520 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.981+0000 7f32f444d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80102b0 con 0x555b82803840 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32f444d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32c800ed80 con 0x555b827fb660 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82824520 0x555b82bd7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82824520 0x555b82bd7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47932/0 (socket says 172.21.15.32:47932) 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e7400640 1 -- 172.21.15.32:0/3626906768 learned_addr learned my addr 172.21.15.32:0/3626906768 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e6a00640 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b827fb660 0x555b82bcb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e6000640 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b82803840 0x555b82be1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e6000640 1 -- 172.21.15.32:0/3626906768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b827fb660 msgr2=0x555b82bcb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e6000640 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b827fb660 0x555b82bcb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e6000640 1 -- 172.21.15.32:0/3626906768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82824520 msgr2=0x555b82bd7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e6000640 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82824520 0x555b82bd7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e6000640 1 -- 172.21.15.32:0/3626906768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b82bccef0 con 0x555b82803840 2024-08-31T07:20:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e6a00640 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b827fb660 0x555b82bcb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.982+0000 7f32e7400640 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82824520 0x555b82bd7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.983+0000 7f32e6000640 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b82803840 0x555b82be1af0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f32d402a910 tx=0x7f32d4009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e752b2b12f5bf994 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.983+0000 7f32d3400640 1 -- 172.21.15.32:0/3626906768 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4035030 con 0x555b82803840 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.983+0000 7f32d3400640 1 -- 172.21.15.32:0/3626906768 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32d40048c0 con 0x555b82803840 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.983+0000 7f32d3400640 1 -- 172.21.15.32:0/3626906768 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4000e60 con 0x555b82803840 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.984+0000 7f32f444d740 1 -- 172.21.15.32:0/3626906768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b82803840 msgr2=0x555b82be1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.984+0000 7f32f444d740 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b82803840 0x555b82be1af0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f32d402a910 tx=0x7f32d4009680 comp rx=0 tx=0).stop 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.984+0000 7f32f444d740 1 -- 172.21.15.32:0/3626906768 shutdown_connections 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.984+0000 7f32f444d740 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82824520 0x555b82bd7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.984+0000 7f32f444d740 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b82803840 0x555b82be1af0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.984+0000 7f32f444d740 1 --2- 172.21.15.32:0/3626906768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b827fb660 0x555b82bcb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.984+0000 7f32f444d740 1 -- 172.21.15.32:0/3626906768 >> 172.21.15.32:0/3626906768 conn(0x555b82818910 msgr2=0x555b82bcffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.984+0000 7f32f444d740 1 -- 172.21.15.32:0/3626906768 shutdown_connections 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.984+0000 7f32f444d740 1 -- 172.21.15.32:0/3626906768 wait complete. 2024-08-31T07:20:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.984+0000 7f32f444d740 1 -- start start 2024-08-31T07:20:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32f444d740 1 Processor -- start 2024-08-31T07:20:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32f444d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bdc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32f444d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bdc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32f444d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82c51250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32f444d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32c800fb20 con 0x555b82803840 2024-08-31T07:20:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32f444d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32c800f450 con 0x555b827fb660 2024-08-31T07:20:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32f444d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32c800c6b0 con 0x555b82824520 2024-08-31T07:20:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32f444d740 10 client.0 _ll_register_callbacks cb 0x7f32d4032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32e6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bdc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32e6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bdc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47946/0 (socket says 172.21.15.32:47946) 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.985+0000 7f32e6000640 1 -- 172.21.15.32:0/1967742437 learned_addr learned my addr 172.21.15.32:0/1967742437 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32e6a00640 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bdc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32e7400640 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82c51250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32e6a00640 1 -- 172.21.15.32:0/1967742437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 msgr2=0x555b82c51250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32e6a00640 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82c51250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32e6a00640 1 -- 172.21.15.32:0/1967742437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 msgr2=0x555b82bdc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32e6a00640 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bdc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32e6a00640 1 -- 172.21.15.32:0/1967742437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b82c51810 con 0x555b827fb660 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32e6a00640 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bdc340 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f32d80e2c20 tx=0x7f32d80e53f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f989bc891d4eae59 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32d0c00640 1 -- 172.21.15.32:0/1967742437 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d80abbc0 con 0x555b827fb660 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32d0c00640 1 -- 172.21.15.32:0/1967742437 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f32d80a7e50 con 0x555b827fb660 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.986+0000 7f32f444d740 1 -- 172.21.15.32:0/1967742437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32d40048c0 con 0x555b827fb660 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32d0c00640 1 -- 172.21.15.32:0/1967742437 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f32d80b0430 con 0x555b827fb660 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32f444d740 10 client.27146 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32f444d740 1 -- 172.21.15.32:0/1967742437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555b82c52b30 con 0x555b827fb660 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32d0c00640 1 -- 172.21.15.32:0/1967742437 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32d40048c0 con 0x555b827fb660 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32d0c00640 1 -- 172.21.15.32:0/1967742437 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f32d80a75a0 con 0x555b827fb660 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32f444d740 10 client.27146 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32f444d740 10 client.27146 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32f444d740 20 client.27146 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32f444d740 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555b827b8250 0x555b82c55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32f444d740 4 client.27146 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "93e49528", "release": true, "await": true}] 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.987+0000 7f32f444d740 1 -- 172.21.15.32:0/1967742437 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "93e49528", "release": true, "await": true}) -- 0x555b82b3e480 con 0x555b827b8250 2024-08-31T07:20:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.988+0000 7f32e6000640 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555b827b8250 0x555b82c55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:15.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.988+0000 7f32e6000640 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555b827b8250 0x555b82c55340 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59fb7a3bb26c8e6 server_cookie=8c275480a39e89f3 in_seq=0 out_seq=0 2024-08-31T07:20:15.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:15.988+0000 7f32d0c00640 10 client.27146 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:16.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32d0c00640 1 -- 172.21.15.32:0/1967742437 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555b82b3e480 con 0x555b827b8250 2024-08-31T07:20:16.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32d0c00640 10 client.27146 handle_command_reply: tid=0 2024-08-31T07:20:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 client.27146 shutdown 2024-08-31T07:20:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 -- 172.21.15.32:0/1967742437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 msgr2=0x555b82bdc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bdc340 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f32d80e2c20 tx=0x7f32d80e53f0 comp rx=0 tx=0).stop 2024-08-31T07:20:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 -- 172.21.15.32:0/1967742437 shutdown_connections 2024-08-31T07:20:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555b827b8250 0x555b82c55340 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b82824520 0x555b82c51250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b82803840 0x555b82bdc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 --2- 172.21.15.32:0/1967742437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b827fb660 0x555b82bdc340 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32d0c00640 0 client.27146 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 -- 172.21.15.32:0/1967742437 >> 172.21.15.32:0/1967742437 conn(0x555b82818910 msgr2=0x555b82bd66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:16.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 -- 172.21.15.32:0/1967742437 shutdown_connections 2024-08-31T07:20:16.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 1 -- 172.21.15.32:0/1967742437 wait complete. 2024-08-31T07:20:16.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.009+0000 7f32f444d740 20 client.27146 trim_cache size 0 max 16384 2024-08-31T07:20:16.012 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:20:16.012 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 35, 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 24, 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "93e49528": { 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 24, 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:20:16.013 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 148.0, 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:16.014 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:20:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 -- 172.21.15.32:0/4029061148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32cc0b4c30 msgr2=0x7f32cc0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32cc0b4c30 0x7f32cc0b7140 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f32e001b040 tx=0x7f32e0004950 comp rx=0 tx=0).stop 2024-08-31T07:20:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 -- 172.21.15.32:0/4029061148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32e8105950 msgr2=0x7f32e819c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32e8105950 0x7f32e819c290 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f32d800ea60 tx=0x7f32d800ef20 comp rx=0 tx=0).stop 2024-08-31T07:20:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 -- 172.21.15.32:0/4029061148 shutdown_connections 2024-08-31T07:20:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f32cc0b4c30 0x7f32cc0b7140 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:16.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e810bbe0 0x7f32e81a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32e810b290 0x7f32e819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 --2- 172.21.15.32:0/4029061148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32e8105950 0x7f32e819c290 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 -- 172.21.15.32:0/4029061148 >> 172.21.15.32:0/4029061148 conn(0x7f32e8101420 msgr2=0x7f32e8103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 -- 172.21.15.32:0/4029061148 shutdown_connections 2024-08-31T07:20:16.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:16.012+0000 7f32e7e00640 1 -- 172.21.15.32:0/4029061148 wait complete. 2024-08-31T07:20:16.273 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:16 smithi032 ceph-mon[17252]: pgmap v597: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 949 KiB/s wr, 55 op/s 2024-08-31T07:20:16.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:16 smithi073 ceph-mon[21995]: pgmap v597: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 949 KiB/s wr, 55 op/s 2024-08-31T07:20:16.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:16 smithi084 ceph-mon[22387]: pgmap v597: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 949 KiB/s wr, 55 op/s 2024-08-31T07:20:16.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234674 2.89 MB/sec execute 245 sec latency 28489.058 ms 2024-08-31T07:20:17.371 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '93e49528', total seconds elapsed: 33.6 2024-08-31T07:20:17.371 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.0 2024-08-31T07:20:17.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:20:17.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 235206 2.88 MB/sec execute 246 sec latency 727.165 ms 2024-08-31T07:20:17.779 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:18.195 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:18 smithi032 ceph-mon[17252]: pgmap v598: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-08-31T07:20:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:18 smithi073 ceph-mon[21995]: pgmap v598: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-08-31T07:20:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:18 smithi084 ceph-mon[22387]: pgmap v598: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-08-31T07:20:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.542+0000 7f4fea400640 1 -- 172.21.15.32:0/1333831407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe4105950 msgr2=0x7f4fe4105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.542+0000 7f4fea400640 1 --2- 172.21.15.32:0/1333831407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe4105950 0x7f4fe4105d60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009900 tx=0x7f4fd802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.542+0000 7f4fea400640 1 -- 172.21.15.32:0/1333831407 shutdown_connections 2024-08-31T07:20:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.542+0000 7f4fea400640 1 --2- 172.21.15.32:0/1333831407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe410bbe0 0x7f4fe4112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.542+0000 7f4fea400640 1 --2- 172.21.15.32:0/1333831407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe410b290 0x7f4fe410b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:18.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.542+0000 7f4fea400640 1 --2- 172.21.15.32:0/1333831407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe4105950 0x7f4fe4105d60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:18.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.542+0000 7f4fea400640 1 -- 172.21.15.32:0/1333831407 >> 172.21.15.32:0/1333831407 conn(0x7f4fe4101420 msgr2=0x7f4fe4103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:18.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.542+0000 7f4fea400640 1 -- 172.21.15.32:0/1333831407 shutdown_connections 2024-08-31T07:20:18.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.542+0000 7f4fea400640 1 -- 172.21.15.32:0/1333831407 wait complete. 2024-08-31T07:20:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.543+0000 7f4fea400640 1 Processor -- start 2024-08-31T07:20:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.543+0000 7f4fea400640 1 -- start start 2024-08-31T07:20:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fea400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe4105950 0x7f4fe419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fea400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe410b290 0x7f4fe419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fea400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe410bbe0 0x7f4fe41a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fea400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41158f0 con 0x7f4fe410b290 2024-08-31T07:20:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fea400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4115770 con 0x7f4fe4105950 2024-08-31T07:20:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fea400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4115a70 con 0x7f4fe410bbe0 2024-08-31T07:20:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fe9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe4105950 0x7f4fe419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fe9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe4105950 0x7f4fe419c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54246/0 (socket says 172.21.15.32:54246) 2024-08-31T07:20:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fe9000640 1 -- 172.21.15.32:0/2473055163 learned_addr learned my addr 172.21.15.32:0/2473055163 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fe3e00640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe410b290 0x7f4fe419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fe9a00640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe410bbe0 0x7f4fe41a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fe9000640 1 -- 172.21.15.32:0/2473055163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe410bbe0 msgr2=0x7f4fe41a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fe9000640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe410bbe0 0x7f4fe41a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fe9000640 1 -- 172.21.15.32:0/2473055163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe410b290 msgr2=0x7f4fe419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.544+0000 7f4fe9000640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe410b290 0x7f4fe419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.545+0000 7f4fe9000640 1 -- 172.21.15.32:0/2473055163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe41a2210 con 0x7f4fe4105950 2024-08-31T07:20:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.545+0000 7f4fe9000640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe4105950 0x7f4fe419c2a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8005ab0 tx=0x7f4fd8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.545+0000 7f4fe1600640 1 -- 172.21.15.32:0/2473055163 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd803f070 con 0x7f4fe4105950 2024-08-31T07:20:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.545+0000 7f4fea400640 1 -- 172.21.15.32:0/2473055163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe41a24a0 con 0x7f4fe4105950 2024-08-31T07:20:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.545+0000 7f4fea400640 1 -- 172.21.15.32:0/2473055163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe41a8c60 con 0x7f4fe4105950 2024-08-31T07:20:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.545+0000 7f4fe1600640 1 -- 172.21.15.32:0/2473055163 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd8031b60 con 0x7f4fe4105950 2024-08-31T07:20:18.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.545+0000 7f4fe1600640 1 -- 172.21.15.32:0/2473055163 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8049060 con 0x7f4fe4105950 2024-08-31T07:20:18.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.546+0000 7f4fea400640 1 -- 172.21.15.32:0/2473055163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe4105d60 con 0x7f4fe4105950 2024-08-31T07:20:18.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.549+0000 7f4fe1600640 1 -- 172.21.15.32:0/2473055163 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4fd8042070 con 0x7f4fe4105950 2024-08-31T07:20:18.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.550+0000 7f4fe1600640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4fcc0b4d20 0x7f4fcc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:18.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.550+0000 7f4fe1600640 1 -- 172.21.15.32:0/2473055163 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f4fd8105f50 con 0x7f4fe4105950 2024-08-31T07:20:18.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.550+0000 7f4fe3e00640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4fcc0b4d20 0x7f4fcc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.551+0000 7f4fe3e00640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4fcc0b4d20 0x7f4fcc0b7230 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4104b80 tx=0x7f4fd0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.552+0000 7f4fe1600640 1 -- 172.21.15.32:0/2473055163 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4fd80cc6e0 con 0x7f4fe4105950 2024-08-31T07:20:18.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 235839 2.88 MB/sec execute 247 sec latency 551.172 ms 2024-08-31T07:20:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.789+0000 7f4fea400640 1 -- 172.21.15.32:0/2473055163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fe419d3d0 con 0x7f4fe4105950 2024-08-31T07:20:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.790+0000 7f4fe1600640 1 -- 172.21.15.32:0/2473055163 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11967 (secure 0 0 0) 0x7f4fe419d3d0 con 0x7f4fe4105950 2024-08-31T07:20:18.791 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:20:18.791 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"btime":"2024-08-31T07:19:16:021228+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:19:15.946242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:20:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 35 2024-08-31T07:20:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.792+0000 7f4fea400640 1 -- 172.21.15.32:0/2473055163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4fcc0b4d20 msgr2=0x7f4fcc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.792+0000 7f4fea400640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4fcc0b4d20 0x7f4fcc0b7230 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4104b80 tx=0x7f4fd0005f90 comp rx=0 tx=0).stop 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.792+0000 7f4fea400640 1 -- 172.21.15.32:0/2473055163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe4105950 msgr2=0x7f4fe419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.792+0000 7f4fea400640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe4105950 0x7f4fe419c2a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8005ab0 tx=0x7f4fd8003c10 comp rx=0 tx=0).stop 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.792+0000 7f4fea400640 1 -- 172.21.15.32:0/2473055163 shutdown_connections 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.792+0000 7f4fea400640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4fcc0b4d20 0x7f4fcc0b7230 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.792+0000 7f4fea400640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe410bbe0 0x7f4fe41a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.792+0000 7f4fea400640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe410b290 0x7f4fe419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.792+0000 7f4fea400640 1 --2- 172.21.15.32:0/2473055163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe4105950 0x7f4fe419c2a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.792+0000 7f4fea400640 1 -- 172.21.15.32:0/2473055163 >> 172.21.15.32:0/2473055163 conn(0x7f4fe4101420 msgr2=0x7f4fe410a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.793+0000 7f4fea400640 1 -- 172.21.15.32:0/2473055163 shutdown_connections 2024-08-31T07:20:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:18.793+0000 7f4fea400640 1 -- 172.21.15.32:0/2473055163 wait complete. 2024-08-31T07:20:19.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:19 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2473055163' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:19.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:19 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2473055163' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:19.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:19 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2473055163' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:19.721 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 148.0 --await-for 45.0' 2024-08-31T07:20:19.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 148.0 --await-for 45.0 2024-08-31T07:20:19.743 INFO:tasks.workunit.client.0.smithi032.stdout: 1 235839 2.87 MB/sec execute 248 sec latency 1551.229 ms 2024-08-31T07:20:20.098 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:20.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:20 smithi032 ceph-mon[17252]: pgmap v599: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-08-31T07:20:20.390 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:20 smithi084 ceph-mon[22387]: pgmap v599: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-08-31T07:20:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:20 smithi073 ceph-mon[21995]: pgmap v599: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-08-31T07:20:20.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237114 2.89 MB/sec execute 249 sec latency 1928.832 ms 2024-08-31T07:20:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.834+0000 7fd23a000640 1 -- 172.21.15.32:0/1828328541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd23410b460 msgr2=0x7fd23410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.834+0000 7fd23a000640 1 --2- 172.21.15.32:0/1828328541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd23410b460 0x7fd23410b870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd220009900 tx=0x7fd22002a410 comp rx=0 tx=0).stop 2024-08-31T07:20:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.834+0000 7fd23a000640 1 -- 172.21.15.32:0/1828328541 shutdown_connections 2024-08-31T07:20:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.834+0000 7fd23a000640 1 --2- 172.21.15.32:0/1828328541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd23410bdb0 0x7fd2341125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.834+0000 7fd23a000640 1 --2- 172.21.15.32:0/1828328541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd23410b460 0x7fd23410b870 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.834+0000 7fd23a000640 1 --2- 172.21.15.32:0/1828328541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd234105b20 0x7fd234105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.834+0000 7fd23a000640 1 -- 172.21.15.32:0/1828328541 >> 172.21.15.32:0/1828328541 conn(0x7fd234101570 msgr2=0x7fd2341039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.834+0000 7fd23a000640 1 -- 172.21.15.32:0/1828328541 shutdown_connections 2024-08-31T07:20:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.834+0000 7fd23a000640 1 -- 172.21.15.32:0/1828328541 wait complete. 2024-08-31T07:20:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.835+0000 7fd23a000640 1 Processor -- start 2024-08-31T07:20:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.835+0000 7fd23a000640 1 -- start start 2024-08-31T07:20:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd23a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd234105b20 0x7fd23419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd23a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23410b460 0x7fd23419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd23a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd23410bdb0 0x7fd2341a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd23a000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd234115a70 con 0x7fd23410b460 2024-08-31T07:20:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd23a000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341158f0 con 0x7fd23410bdb0 2024-08-31T07:20:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd23a000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd234115bf0 con 0x7fd234105b20 2024-08-31T07:20:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd233400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd234105b20 0x7fd23419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd232a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23410b460 0x7fd23419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd233400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd234105b20 0x7fd23419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34908/0 (socket says 172.21.15.32:34908) 2024-08-31T07:20:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd233400640 1 -- 172.21.15.32:0/2220206145 learned_addr learned my addr 172.21.15.32:0/2220206145 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.836+0000 7fd233e00640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd23410bdb0 0x7fd2341a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.837+0000 7fd233400640 1 -- 172.21.15.32:0/2220206145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd23410bdb0 msgr2=0x7fd2341a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.837+0000 7fd233400640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd23410bdb0 0x7fd2341a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.837+0000 7fd233400640 1 -- 172.21.15.32:0/2220206145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23410b460 msgr2=0x7fd23419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.837+0000 7fd233400640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23410b460 0x7fd23419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.837+0000 7fd233400640 1 -- 172.21.15.32:0/2220206145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2341a23b0 con 0x7fd234105b20 2024-08-31T07:20:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.837+0000 7fd233400640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd234105b20 0x7fd23419c440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd22400e9f0 tx=0x7fd22400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.837+0000 7fd22be00640 1 -- 172.21.15.32:0/2220206145 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22400ccc0 con 0x7fd234105b20 2024-08-31T07:20:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.837+0000 7fd22be00640 1 -- 172.21.15.32:0/2220206145 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2240108e0 con 0x7fd234105b20 2024-08-31T07:20:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.837+0000 7fd22be00640 1 -- 172.21.15.32:0/2220206145 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224018880 con 0x7fd234105b20 2024-08-31T07:20:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.837+0000 7fd23a000640 1 -- 172.21.15.32:0/2220206145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2341a26a0 con 0x7fd234105b20 2024-08-31T07:20:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.838+0000 7fd23a000640 1 -- 172.21.15.32:0/2220206145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2341a8e00 con 0x7fd234105b20 2024-08-31T07:20:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.841+0000 7fd240bde740 1 Processor -- start 2024-08-31T07:20:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.841+0000 7fd240bde740 1 -- start start 2024-08-31T07:20:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.841+0000 7fd22be00640 1 -- 172.21.15.32:0/2220206145 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd224010430 con 0x7fd234105b20 2024-08-31T07:20:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.841+0000 7fd240bde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55664181c840 0x556641bc3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.841+0000 7fd240bde740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417f3980 0x556641bc3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.841+0000 7fd240bde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417fbb60 0x556641bc3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.841+0000 7fd240bde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556641b38530 con 0x55664181c840 2024-08-31T07:20:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.841+0000 7fd240bde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556641b27ac0 con 0x5566417fbb60 2024-08-31T07:20:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.841+0000 7fd240bde740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556641b2ba70 con 0x5566417f3980 2024-08-31T07:20:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd233400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417f3980 0x556641bc3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd233400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417f3980 0x556641bc3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34922/0 (socket says 172.21.15.32:34922) 2024-08-31T07:20:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd233400640 1 -- 172.21.15.32:0/2139873711 learned_addr learned my addr 172.21.15.32:0/2139873711 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd233e00640 1 --2- 172.21.15.32:0/2139873711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55664181c840 0x556641bc3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd232a00640 1 --2- 172.21.15.32:0/2139873711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417fbb60 0x556641bc3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd233400640 1 -- 172.21.15.32:0/2139873711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417fbb60 msgr2=0x556641bc3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd233400640 1 --2- 172.21.15.32:0/2139873711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417fbb60 0x556641bc3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd233400640 1 -- 172.21.15.32:0/2139873711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55664181c840 msgr2=0x556641bc3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd233400640 1 --2- 172.21.15.32:0/2139873711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55664181c840 0x556641bc3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd233400640 1 -- 172.21.15.32:0/2139873711 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556641ada880 con 0x5566417f3980 2024-08-31T07:20:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.842+0000 7fd233400640 1 --2- 172.21.15.32:0/2139873711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417f3980 0x556641bc3650 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd2240e3020 tx=0x7fd2240ed2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36a73f1c61a48f19 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:20.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.843+0000 7fd21fe00640 1 -- 172.21.15.32:0/2139873711 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2240f4070 con 0x5566417f3980 2024-08-31T07:20:20.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.843+0000 7fd21fe00640 1 -- 172.21.15.32:0/2139873711 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2240f08d0 con 0x5566417f3980 2024-08-31T07:20:20.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.843+0000 7fd21fe00640 1 -- 172.21.15.32:0/2139873711 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2240ef8c0 con 0x5566417f3980 2024-08-31T07:20:20.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.843+0000 7fd22be00640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2180b4ca0 0x7fd2180b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd22be00640 1 -- 172.21.15.32:0/2220206145 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7fd2240e59a0 con 0x7fd234105b20 2024-08-31T07:20:20.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd233e00640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2180b4ca0 0x7fd2180b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd240bde740 1 -- 172.21.15.32:0/2139873711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417f3980 msgr2=0x556641bc3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:20.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd240bde740 1 --2- 172.21.15.32:0/2139873711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417f3980 0x556641bc3650 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd2240e3020 tx=0x7fd2240ed2a0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd240bde740 1 -- 172.21.15.32:0/2139873711 shutdown_connections 2024-08-31T07:20:20.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd240bde740 1 --2- 172.21.15.32:0/2139873711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417fbb60 0x556641bc3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd240bde740 1 --2- 172.21.15.32:0/2139873711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417f3980 0x556641bc3650 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd240bde740 1 --2- 172.21.15.32:0/2139873711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55664181c840 0x556641bc3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd240bde740 1 -- 172.21.15.32:0/2139873711 >> 172.21.15.32:0/2139873711 conn(0x556641810990 msgr2=0x556641bc1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd240bde740 1 -- 172.21.15.32:0/2139873711 shutdown_connections 2024-08-31T07:20:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.844+0000 7fd240bde740 1 -- 172.21.15.32:0/2139873711 wait complete. 2024-08-31T07:20:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.845+0000 7fd233e00640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2180b4ca0 0x7fd2180b71b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd22c005fd0 tx=0x7fd22c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.845+0000 7fd240bde740 1 -- start start 2024-08-31T07:20:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.845+0000 7fd240bde740 1 Processor -- start 2024-08-31T07:20:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd240bde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bcec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd240bde740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bcf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd240bde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55664181c840 0x556641bdac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd240bde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55664198a7a0 con 0x55664181c840 2024-08-31T07:20:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd240bde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556641aae7f0 con 0x5566417f3980 2024-08-31T07:20:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd240bde740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556641bd1940 con 0x5566417fbb60 2024-08-31T07:20:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd240bde740 10 client.0 _ll_register_callbacks cb 0x556641ad3e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd232a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bcf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd233e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55664181c840 0x556641bdac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd233400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bcec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd232a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bcf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34930/0 (socket says 172.21.15.32:34930) 2024-08-31T07:20:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.846+0000 7fd232a00640 1 -- 172.21.15.32:0/1170760575 learned_addr learned my addr 172.21.15.32:0/1170760575 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd232a00640 1 -- 172.21.15.32:0/1170760575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 msgr2=0x556641bcec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd232a00640 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bcec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd232a00640 1 -- 172.21.15.32:0/1170760575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55664181c840 msgr2=0x556641bdac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd232a00640 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55664181c840 0x556641bdac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd232a00640 1 -- 172.21.15.32:0/1170760575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556641896b90 con 0x5566417fbb60 2024-08-31T07:20:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd233400640 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bcec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd232a00640 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bcf1c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd220009900 tx=0x7fd220032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d424bf4820848ce8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd21cc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd21cc00640 1 -- 172.21.15.32:0/1170760575 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220035040 con 0x5566417fbb60 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd240bde740 1 -- 172.21.15.32:0/1170760575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2240f08d0 con 0x5566417fbb60 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.847+0000 7fd21cc00640 1 -- 172.21.15.32:0/1170760575 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7fd220031760 con 0x5566417fbb60 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd21cc00640 1 -- 172.21.15.32:0/1170760575 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd220004040 con 0x5566417fbb60 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd240bde740 10 client.27169 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd240bde740 1 -- 172.21.15.32:0/1170760575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556641bdd950 con 0x5566417fbb60 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd21cc00640 1 -- 172.21.15.32:0/1170760575 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2240f08d0 con 0x5566417fbb60 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd21cc00640 1 -- 172.21.15.32:0/1170760575 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fd220004750 con 0x5566417fbb60 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd240bde740 10 client.27169 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd240bde740 10 client.27169 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:20:20.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd240bde740 20 client.27169 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:20.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd240bde740 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5566417db1c0 0x556641be0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd240bde740 4 client.27169 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:20.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.848+0000 7fd240bde740 1 -- 172.21.15.32:0/1170760575 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556641b3a480 con 0x5566417db1c0 2024-08-31T07:20:20.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.849+0000 7fd233400640 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5566417db1c0 0x556641be0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.849+0000 7fd233400640 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5566417db1c0 0x556641be0db0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2002717e8100d3f server_cookie=4e61f0c2468476b9 in_seq=0 out_seq=0 2024-08-31T07:20:20.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.849+0000 7fd21cc00640 10 client.27169 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:20.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd21cc00640 1 -- 172.21.15.32:0/1170760575 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556641b3a480 con 0x5566417db1c0 2024-08-31T07:20:20.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd21cc00640 10 client.27169 handle_command_reply: tid=0 2024-08-31T07:20:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd240bde740 1 client.27169 shutdown 2024-08-31T07:20:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd240bde740 1 -- 172.21.15.32:0/1170760575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 msgr2=0x556641bcf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd240bde740 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bcf1c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd220009900 tx=0x7fd220032e90 comp rx=0 tx=0).stop 2024-08-31T07:20:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd240bde740 1 -- 172.21.15.32:0/1170760575 shutdown_connections 2024-08-31T07:20:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd240bde740 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5566417db1c0 0x556641be0db0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd240bde740 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55664181c840 0x556641bdac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd240bde740 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bcf1c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd240bde740 1 --2- 172.21.15.32:0/1170760575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bcec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd240bde740 1 -- 172.21.15.32:0/1170760575 >> 172.21.15.32:0/1170760575 conn(0x556641810990 msgr2=0x556641bc4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd21cc00640 0 client.27169 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.850+0000 7fd240bde740 1 -- 172.21.15.32:0/1170760575 shutdown_connections 2024-08-31T07:20:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.851+0000 7fd240bde740 1 -- 172.21.15.32:0/1170760575 wait complete. 2024-08-31T07:20:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.851+0000 7fd240bde740 20 client.27169 trim_cache size 0 max 16384 2024-08-31T07:20:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd240bde740 1 Processor -- start 2024-08-31T07:20:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd240bde740 1 -- start start 2024-08-31T07:20:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd240bde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417f3980 0x556641bc4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd240bde740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bc07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd240bde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55664181c840 0x556641bc0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd240bde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341b1550 con 0x5566417f3980 2024-08-31T07:20:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd240bde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341b3150 con 0x55664181c840 2024-08-31T07:20:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd240bde740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341b1c20 con 0x5566417fbb60 2024-08-31T07:20:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd232a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bc07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd233400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417f3980 0x556641bc4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd232a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bc07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34940/0 (socket says 172.21.15.32:34940) 2024-08-31T07:20:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd232a00640 1 -- 172.21.15.32:0/2570280465 learned_addr learned my addr 172.21.15.32:0/2570280465 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.871+0000 7fd233e00640 1 --2- 172.21.15.32:0/2570280465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55664181c840 0x556641bc0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.872+0000 7fd232a00640 1 -- 172.21.15.32:0/2570280465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55664181c840 msgr2=0x556641bc0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.872+0000 7fd232a00640 1 --2- 172.21.15.32:0/2570280465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55664181c840 0x556641bc0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.872+0000 7fd232a00640 1 -- 172.21.15.32:0/2570280465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417f3980 msgr2=0x556641bc4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.872+0000 7fd232a00640 1 --2- 172.21.15.32:0/2570280465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417f3980 0x556641bc4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.872+0000 7fd232a00640 1 -- 172.21.15.32:0/2570280465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556641bc5890 con 0x5566417fbb60 2024-08-31T07:20:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.872+0000 7fd233e00640 1 --2- 172.21.15.32:0/2570280465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55664181c840 0x556641bc0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.872+0000 7fd233400640 1 --2- 172.21.15.32:0/2570280465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417f3980 0x556641bc4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.872+0000 7fd232a00640 1 --2- 172.21.15.32:0/2570280465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bc07f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd2200048d0 tx=0x7fd220004900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dde6c5373ba6aed server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.872+0000 7fd21f400640 1 -- 172.21.15.32:0/2570280465 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220035070 con 0x5566417fbb60 2024-08-31T07:20:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.872+0000 7fd21f400640 1 -- 172.21.15.32:0/2570280465 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2200043e0 con 0x5566417fbb60 2024-08-31T07:20:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.873+0000 7fd21f400640 1 -- 172.21.15.32:0/2570280465 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220004d40 con 0x5566417fbb60 2024-08-31T07:20:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.873+0000 7fd240bde740 1 -- 172.21.15.32:0/2570280465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 msgr2=0x556641bc07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.873+0000 7fd240bde740 1 --2- 172.21.15.32:0/2570280465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bc07f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd2200048d0 tx=0x7fd220004900 comp rx=0 tx=0).stop 2024-08-31T07:20:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.873+0000 7fd240bde740 1 -- 172.21.15.32:0/2570280465 shutdown_connections 2024-08-31T07:20:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.873+0000 7fd240bde740 1 --2- 172.21.15.32:0/2570280465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55664181c840 0x556641bc0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.873+0000 7fd240bde740 1 --2- 172.21.15.32:0/2570280465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566417fbb60 0x556641bc07f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.873+0000 7fd240bde740 1 --2- 172.21.15.32:0/2570280465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417f3980 0x556641bc4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.873+0000 7fd240bde740 1 -- 172.21.15.32:0/2570280465 >> 172.21.15.32:0/2570280465 conn(0x556641810990 msgr2=0x556641bc4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.874+0000 7fd240bde740 1 -- 172.21.15.32:0/2570280465 shutdown_connections 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.874+0000 7fd240bde740 1 -- 172.21.15.32:0/2570280465 wait complete. 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.874+0000 7fd240bde740 1 -- start start 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.874+0000 7fd240bde740 1 Processor -- start 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd240bde740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bd2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd240bde740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417fbb60 0x556641bd3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd240bde740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55664181c840 0x556641c4a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd240bde740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341b29c0 con 0x5566417fbb60 2024-08-31T07:20:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd240bde740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341b22f0 con 0x5566417f3980 2024-08-31T07:20:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd240bde740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341af550 con 0x55664181c840 2024-08-31T07:20:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd240bde740 10 client.0 _ll_register_callbacks cb 0x7fd220032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd233400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bd2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd233400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bd2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54296/0 (socket says 172.21.15.32:54296) 2024-08-31T07:20:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd233400640 1 -- 172.21.15.32:0/2908321852 learned_addr learned my addr 172.21.15.32:0/2908321852 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd233e00640 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55664181c840 0x556641c4a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.875+0000 7fd232a00640 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417fbb60 0x556641bd3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd233400640 1 -- 172.21.15.32:0/2908321852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55664181c840 msgr2=0x556641c4a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd233400640 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55664181c840 0x556641c4a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd233400640 1 -- 172.21.15.32:0/2908321852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417fbb60 msgr2=0x556641bd3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd233400640 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417fbb60 0x556641bd3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd233400640 1 -- 172.21.15.32:0/2908321852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556641c4acf0 con 0x5566417f3980 2024-08-31T07:20:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd232a00640 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417fbb60 0x556641bd3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd233400640 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bd2ee0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd2240a8100 tx=0x7fd2240ef420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6545099919dfc159 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd21cc00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd21cc00640 1 -- 172.21.15.32:0/2908321852 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2240f4040 con 0x5566417f3980 2024-08-31T07:20:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd21cc00640 1 -- 172.21.15.32:0/2908321852 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7fd224010a90 con 0x5566417f3980 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd240bde740 1 -- 172.21.15.32:0/2908321852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2200043e0 con 0x5566417f3980 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd21cc00640 1 -- 172.21.15.32:0/2908321852 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd2240a7f40 con 0x5566417f3980 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd240bde740 10 client.27170 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.876+0000 7fd240bde740 1 -- 172.21.15.32:0/2908321852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556641c4bdc0 con 0x5566417f3980 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.877+0000 7fd21cc00640 1 -- 172.21.15.32:0/2908321852 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2200043e0 con 0x5566417f3980 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.877+0000 7fd21cc00640 1 -- 172.21.15.32:0/2908321852 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fd2240edce0 con 0x5566417f3980 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.877+0000 7fd240bde740 10 client.27170 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.877+0000 7fd240bde740 10 client.27170 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.877+0000 7fd240bde740 20 client.27170 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.877+0000 7fd240bde740 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5566417b0570 0x556641c4e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.877+0000 7fd240bde740 4 client.27170 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.0, "await_for": 45.0}] 2024-08-31T07:20:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.877+0000 7fd240bde740 1 -- 172.21.15.32:0/2908321852 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.0, "await_for": 45.0}) -- 0x556641b35f20 con 0x5566417b0570 2024-08-31T07:20:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.877+0000 7fd232a00640 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5566417b0570 0x556641c4e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.878+0000 7fd232a00640 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5566417b0570 0x556641c4e230 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48eff0c4dd91b95c server_cookie=467a8b00948ebe89 in_seq=0 out_seq=0 2024-08-31T07:20:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.878+0000 7fd21cc00640 10 client.27170 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.937+0000 7fd21cc00640 1 -- 172.21.15.32:0/2908321852 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556641b35f20 con 0x5566417b0570 2024-08-31T07:20:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.937+0000 7fd21cc00640 10 client.27170 handle_command_reply: tid=0 2024-08-31T07:20:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.937+0000 7fd240bde740 1 client.27170 shutdown 2024-08-31T07:20:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 1 -- 172.21.15.32:0/2908321852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 msgr2=0x556641bd2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bd2ee0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd2240a8100 tx=0x7fd2240ef420 comp rx=0 tx=0).stop 2024-08-31T07:20:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 1 -- 172.21.15.32:0/2908321852 shutdown_connections 2024-08-31T07:20:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5566417b0570 0x556641c4e230 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55664181c840 0x556641c4a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5566417fbb60 0x556641bd3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 1 --2- 172.21.15.32:0/2908321852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5566417f3980 0x556641bd2ee0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd21cc00640 0 client.27170 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 1 -- 172.21.15.32:0/2908321852 >> 172.21.15.32:0/2908321852 conn(0x556641810990 msgr2=0x556641bcff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 1 -- 172.21.15.32:0/2908321852 shutdown_connections 2024-08-31T07:20:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 1 -- 172.21.15.32:0/2908321852 wait complete. 2024-08-31T07:20:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.938+0000 7fd240bde740 20 client.27170 trim_cache size 0 max 16384 2024-08-31T07:20:20.941 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:20:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 35, 2024-08-31T07:20:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:20:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 26, 2024-08-31T07:20:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:20:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "d2dfb577": { 2024-08-31T07:20:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 26, 2024-08-31T07:20:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:20:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:20:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 148.0, 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:20.943 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:20:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 -- 172.21.15.32:0/2220206145 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2180b4ca0 msgr2=0x7fd2180b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2180b4ca0 0x7fd2180b71b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd22c005fd0 tx=0x7fd22c009500 comp rx=0 tx=0).stop 2024-08-31T07:20:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 -- 172.21.15.32:0/2220206145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd234105b20 msgr2=0x7fd23419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd234105b20 0x7fd23419c440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd22400e9f0 tx=0x7fd22400edf0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 -- 172.21.15.32:0/2220206145 shutdown_connections 2024-08-31T07:20:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd2180b4ca0 0x7fd2180b71b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd23410bdb0 0x7fd2341a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd23410b460 0x7fd23419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 --2- 172.21.15.32:0/2220206145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd234105b20 0x7fd23419c440 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 -- 172.21.15.32:0/2220206145 >> 172.21.15.32:0/2220206145 conn(0x7fd234101570 msgr2=0x7fd234103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 -- 172.21.15.32:0/2220206145 shutdown_connections 2024-08-31T07:20:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:20.939+0000 7fd23a000640 1 -- 172.21.15.32:0/2220206145 wait complete. 2024-08-31T07:20:21.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.89 MB/sec execute 250 sec latency 839.294 ms 2024-08-31T07:20:21.775 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd2dfb577', quiesce took 4.4 seconds. Will release after: 23.6 2024-08-31T07:20:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:22 smithi073 ceph-mon[21995]: pgmap v600: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-08-31T07:20:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:22 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:20:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:22 smithi084 ceph-mon[22387]: pgmap v600: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-08-31T07:20:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:22 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:20:22.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:22 smithi032 ceph-mon[17252]: pgmap v600: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-08-31T07:20:22.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:22 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:20:22.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.87 MB/sec execute 251 sec latency 1839.414 ms 2024-08-31T07:20:23.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:20:23.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:20:23.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:20:23.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:20:23.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:20:23.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:20:23.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:20:23.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:20:23.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:20:23.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.86 MB/sec execute 252 sec latency 2839.575 ms 2024-08-31T07:20:24.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.85 MB/sec execute 253 sec latency 3839.697 ms 2024-08-31T07:20:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:24 smithi073 ceph-mon[21995]: pgmap v601: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-08-31T07:20:24.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:24 smithi084 ceph-mon[22387]: pgmap v601: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-08-31T07:20:25.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:24 smithi032 ceph-mon[17252]: pgmap v601: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-08-31T07:20:25.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.84 MB/sec execute 254 sec latency 4839.830 ms 2024-08-31T07:20:26.744 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.83 MB/sec execute 255 sec latency 5840.041 ms 2024-08-31T07:20:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:26 smithi073 ceph-mon[21995]: pgmap v602: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 966 KiB/s wr, 80 op/s 2024-08-31T07:20:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:26 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:20:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:26 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:20:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:26 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:20:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:26 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:20:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:26 smithi084 ceph-mon[22387]: pgmap v602: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 966 KiB/s wr, 80 op/s 2024-08-31T07:20:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:26 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:20:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:26 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:20:26.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:26 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:20:26.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:26 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:20:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:26 smithi032 ceph-mon[17252]: pgmap v602: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 966 KiB/s wr, 80 op/s 2024-08-31T07:20:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:26 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:20:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:26 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:20:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:26 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:20:27.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:26 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:20:27.745 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.82 MB/sec execute 256 sec latency 6840.185 ms 2024-08-31T07:20:27.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:27 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:20:27.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:27 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:20:28.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:27 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:20:28.745 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.81 MB/sec execute 257 sec latency 7840.291 ms 2024-08-31T07:20:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:28 smithi073 ceph-mon[21995]: pgmap v603: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 911 KiB/s wr, 89 op/s 2024-08-31T07:20:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:28 smithi084 ceph-mon[22387]: pgmap v603: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 911 KiB/s wr, 89 op/s 2024-08-31T07:20:29.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:28 smithi032 ceph-mon[17252]: pgmap v603: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 911 KiB/s wr, 89 op/s 2024-08-31T07:20:29.745 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.80 MB/sec execute 258 sec latency 8840.346 ms 2024-08-31T07:20:30.745 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.79 MB/sec execute 259 sec latency 9840.487 ms 2024-08-31T07:20:30.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:30 smithi073 ceph-mon[21995]: pgmap v604: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 663 KiB/s wr, 76 op/s 2024-08-31T07:20:30.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:30 smithi084 ceph-mon[22387]: pgmap v604: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 663 KiB/s wr, 76 op/s 2024-08-31T07:20:31.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:30 smithi032 ceph-mon[17252]: pgmap v604: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 663 KiB/s wr, 76 op/s 2024-08-31T07:20:31.745 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.78 MB/sec execute 260 sec latency 10840.661 ms 2024-08-31T07:20:32.745 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.76 MB/sec execute 261 sec latency 11840.815 ms 2024-08-31T07:20:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:32 smithi073 ceph-mon[21995]: pgmap v605: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 508 KiB/s wr, 68 op/s 2024-08-31T07:20:32.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:32 smithi084 ceph-mon[22387]: pgmap v605: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 508 KiB/s wr, 68 op/s 2024-08-31T07:20:33.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:32 smithi032 ceph-mon[17252]: pgmap v605: 97 pgs: 97 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 508 KiB/s wr, 68 op/s 2024-08-31T07:20:33.745 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.75 MB/sec execute 262 sec latency 12840.954 ms 2024-08-31T07:20:34.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.74 MB/sec execute 263 sec latency 13841.103 ms 2024-08-31T07:20:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:34 smithi073 ceph-mon[21995]: pgmap v606: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 292 KiB/s wr, 47 op/s 2024-08-31T07:20:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:34 smithi084 ceph-mon[22387]: pgmap v606: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 292 KiB/s wr, 47 op/s 2024-08-31T07:20:35.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:34 smithi032 ceph-mon[17252]: pgmap v606: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 292 KiB/s wr, 47 op/s 2024-08-31T07:20:35.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.73 MB/sec execute 264 sec latency 14841.191 ms 2024-08-31T07:20:36.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.72 MB/sec execute 265 sec latency 15841.336 ms 2024-08-31T07:20:36.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:36 smithi073 ceph-mon[21995]: pgmap v607: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 198 KiB/s wr, 31 op/s 2024-08-31T07:20:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:36 smithi084 ceph-mon[22387]: pgmap v607: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 198 KiB/s wr, 31 op/s 2024-08-31T07:20:37.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:36 smithi032 ceph-mon[17252]: pgmap v607: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 198 KiB/s wr, 31 op/s 2024-08-31T07:20:37.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.71 MB/sec execute 266 sec latency 16841.486 ms 2024-08-31T07:20:38.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.70 MB/sec execute 267 sec latency 17841.619 ms 2024-08-31T07:20:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:38 smithi073 ceph-mon[21995]: pgmap v608: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 184 KiB/s wr, 30 op/s 2024-08-31T07:20:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:38 smithi084 ceph-mon[22387]: pgmap v608: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 184 KiB/s wr, 30 op/s 2024-08-31T07:20:39.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:38 smithi032 ceph-mon[17252]: pgmap v608: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 184 KiB/s wr, 30 op/s 2024-08-31T07:20:39.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.69 MB/sec execute 268 sec latency 18841.776 ms 2024-08-31T07:20:40.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.68 MB/sec execute 269 sec latency 19841.883 ms 2024-08-31T07:20:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:40 smithi073 ceph-mon[21995]: pgmap v609: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 97 KiB/s wr, 14 op/s 2024-08-31T07:20:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:40 smithi084 ceph-mon[22387]: pgmap v609: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 97 KiB/s wr, 14 op/s 2024-08-31T07:20:41.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:40 smithi032 ceph-mon[17252]: pgmap v609: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 97 KiB/s wr, 14 op/s 2024-08-31T07:20:41.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.67 MB/sec execute 270 sec latency 20841.942 ms 2024-08-31T07:20:42.746 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.66 MB/sec execute 271 sec latency 21842.021 ms 2024-08-31T07:20:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:42 smithi073 ceph-mon[21995]: pgmap v610: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 103 KiB/s wr, 16 op/s 2024-08-31T07:20:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:42 smithi084 ceph-mon[22387]: pgmap v610: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 103 KiB/s wr, 16 op/s 2024-08-31T07:20:43.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:20:43.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:42 smithi032 ceph-mon[17252]: pgmap v610: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 103 KiB/s wr, 16 op/s 2024-08-31T07:20:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:43.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.65 MB/sec execute 272 sec latency 22842.143 ms 2024-08-31T07:20:44.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.186+0000 7f0f1e800640 1 -- 172.21.15.32:0/1459598391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f18109b90 msgr2=0x7f0f181103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:44.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.186+0000 7f0f1e800640 1 --2- 172.21.15.32:0/1459598391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f18109b90 0x7f0f181103c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009ca0 tx=0x7f0f1002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.186+0000 7f0f1e800640 1 -- 172.21.15.32:0/1459598391 shutdown_connections 2024-08-31T07:20:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.186+0000 7f0f1e800640 1 --2- 172.21.15.32:0/1459598391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f18109b90 0x7f0f181103c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.186+0000 7f0f1e800640 1 --2- 172.21.15.32:0/1459598391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f18109240 0x7f0f18109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.186+0000 7f0f1e800640 1 --2- 172.21.15.32:0/1459598391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f18103900 0x7f0f18103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.186+0000 7f0f1e800640 1 -- 172.21.15.32:0/1459598391 >> 172.21.15.32:0/1459598391 conn(0x7f0f1807b5f0 msgr2=0x7f0f1807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.186+0000 7f0f1e800640 1 -- 172.21.15.32:0/1459598391 shutdown_connections 2024-08-31T07:20:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.186+0000 7f0f1e800640 1 -- 172.21.15.32:0/1459598391 wait complete. 2024-08-31T07:20:44.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.187+0000 7f0f1e800640 1 Processor -- start 2024-08-31T07:20:44.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.187+0000 7f0f1e800640 1 -- start start 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f1e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f18103900 0x7f0f1819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f1e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f18109240 0x7f0f1819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f1e800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f18109b90 0x7f0f181a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f1e800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181138a0 con 0x7f0f18103900 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f1e800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18113720 con 0x7f0f18109240 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f1e800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18113a20 con 0x7f0f18109b90 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f0ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f18109240 0x7f0f1819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f0ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f18109240 0x7f0f1819c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55692/0 (socket says 172.21.15.32:55692) 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f0ea00640 1 -- 172.21.15.32:0/419293057 learned_addr learned my addr 172.21.15.32:0/419293057 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f17400640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f18103900 0x7f0f1819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f17e00640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f18109b90 0x7f0f181a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f0ea00640 1 -- 172.21.15.32:0/419293057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f18109b90 msgr2=0x7f0f181a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f18109b90 0x7f0f181a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f0ea00640 1 -- 172.21.15.32:0/419293057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f18103900 msgr2=0x7f0f1819c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f18103900 0x7f0f1819c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.188+0000 7f0f0ea00640 1 -- 172.21.15.32:0/419293057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f181a2320 con 0x7f0f18109240 2024-08-31T07:20:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.189+0000 7f0f17e00640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f18109b90 0x7f0f181a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.189+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f18109240 0x7f0f1819c950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e9c0 tx=0x7f0f0000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.189+0000 7f0f14c00640 1 -- 172.21.15.32:0/419293057 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000ccc0 con 0x7f0f18109240 2024-08-31T07:20:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.189+0000 7f0f1e800640 1 -- 172.21.15.32:0/419293057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f181a2610 con 0x7f0f18109240 2024-08-31T07:20:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.189+0000 7f0f14c00640 1 -- 172.21.15.32:0/419293057 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f000054d0 con 0x7f0f18109240 2024-08-31T07:20:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.189+0000 7f0f14c00640 1 -- 172.21.15.32:0/419293057 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00010430 con 0x7f0f18109240 2024-08-31T07:20:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.189+0000 7f0f1e800640 1 -- 172.21.15.32:0/419293057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a8d70 con 0x7f0f18109240 2024-08-31T07:20:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.192+0000 7f0f255e7740 1 Processor -- start 2024-08-31T07:20:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.192+0000 7f0f255e7740 1 -- start start 2024-08-31T07:20:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 0x55aa400ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f255e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55aa40040770 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f255e7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55aa4002fd00 con 0x55aa3fd03390 2024-08-31T07:20:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f255e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55aa40033cb0 con 0x55aa3fd24070 2024-08-31T07:20:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f17400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f17e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f17400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56900/0 (socket says 172.21.15.32:56900) 2024-08-31T07:20:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f17e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44250/0 (socket says 172.21.15.32:44250) 2024-08-31T07:20:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f17400640 1 -- 172.21.15.32:0/1547386302 learned_addr learned my addr 172.21.15.32:0/1547386302 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.193+0000 7f0f14c00640 1 -- 172.21.15.32:0/419293057 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0f00005020 con 0x7f0f18109240 2024-08-31T07:20:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.194+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/1547386302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 0x55aa400ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.194+0000 7f0f17400640 1 -- 172.21.15.32:0/1547386302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 msgr2=0x55aa400c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.194+0000 7f0f17400640 1 --2- 172.21.15.32:0/1547386302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.194+0000 7f0f17400640 1 -- 172.21.15.32:0/1547386302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 msgr2=0x55aa400ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.194+0000 7f0f17400640 1 --2- 172.21.15.32:0/1547386302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 0x55aa400ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.194+0000 7f0f17400640 1 -- 172.21.15.32:0/1547386302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa3fd8f650 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.194+0000 7f0f17e00640 1 --2- 172.21.15.32:0/1547386302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.194+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/1547386302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 0x55aa400ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.194+0000 7f0f17400640 1 --2- 172.21.15.32:0/1547386302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400c9e90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009cf0 tx=0x7f0f08011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420f36aee162644c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.195+0000 7f0f0cc00640 1 -- 172.21.15.32:0/1547386302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08004230 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.195+0000 7f0f0cc00640 1 -- 172.21.15.32:0/1547386302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f080043d0 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.195+0000 7f0f0cc00640 1 -- 172.21.15.32:0/1547386302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08019880 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.195+0000 7f0f14c00640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0efc0b4ca0 0x7f0efc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f14c00640 1 -- 172.21.15.32:0/419293057 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f0f000e5930 con 0x7f0f18109240 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f255e7740 1 -- 172.21.15.32:0/1547386302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 msgr2=0x55aa400c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f255e7740 1 --2- 172.21.15.32:0/1547386302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400c9e90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009cf0 tx=0x7f0f08011810 comp rx=0 tx=0).stop 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f17e00640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0efc0b4ca0 0x7f0efc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f255e7740 1 -- 172.21.15.32:0/1547386302 shutdown_connections 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f255e7740 1 --2- 172.21.15.32:0/1547386302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 0x55aa400ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f255e7740 1 --2- 172.21.15.32:0/1547386302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400c9e90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f255e7740 1 --2- 172.21.15.32:0/1547386302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f255e7740 1 -- 172.21.15.32:0/1547386302 >> 172.21.15.32:0/1547386302 conn(0x55aa3fd18460 msgr2=0x55aa400c7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f255e7740 1 -- 172.21.15.32:0/1547386302 shutdown_connections 2024-08-31T07:20:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.196+0000 7f0f255e7740 1 -- 172.21.15.32:0/1547386302 wait complete. 2024-08-31T07:20:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.197+0000 7f0f255e7740 1 -- start start 2024-08-31T07:20:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.197+0000 7f0f17e00640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0efc0b4ca0 0x7f0efc0b71b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0f1002f490 tx=0x7f0f100099a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.197+0000 7f0f255e7740 1 Processor -- start 2024-08-31T07:20:44.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.197+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.197+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd03390 0x55aa400d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.197+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd24070 0x55aa400e1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.197+0000 7f0f255e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55aa3fe92700 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.197+0000 7f0f255e7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55aa3ffb6cb0 con 0x55aa3fd24070 2024-08-31T07:20:44.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.197+0000 7f0f255e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55aa400d8130 con 0x55aa3fd03390 2024-08-31T07:20:44.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.197+0000 7f0f255e7740 10 client.0 _ll_register_callbacks cb 0x55aa3ffdc430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.198+0000 7f0f17e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd24070 0x55aa400e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.198+0000 7f0f17400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.198+0000 7f0f0ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd03390 0x55aa400d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.198+0000 7f0f17e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd24070 0x55aa400e1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55700/0 (socket says 172.21.15.32:55700) 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.198+0000 7f0f17e00640 1 -- 172.21.15.32:0/3804654569 learned_addr learned my addr 172.21.15.32:0/3804654569 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.198+0000 7f0f17400640 1 -- 172.21.15.32:0/3804654569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd03390 msgr2=0x55aa400d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.198+0000 7f0f17400640 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd03390 0x55aa400d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.198+0000 7f0f17400640 1 -- 172.21.15.32:0/3804654569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd24070 msgr2=0x55aa400e1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.198+0000 7f0f17400640 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd24070 0x55aa400e1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.198+0000 7f0f17400640 1 -- 172.21.15.32:0/3804654569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa3fff7870 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.199+0000 7f0f17400640 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0f08003f50 tx=0x7f0f080038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50ee472f538afbf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.199+0000 7f0f05600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.199+0000 7f0f05600640 1 -- 172.21.15.32:0/3804654569 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08027070 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.199+0000 7f0f05600640 1 -- 172.21.15.32:0/3804654569 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f0f08005240 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.199+0000 7f0f255e7740 1 -- 172.21.15.32:0/3804654569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f080043d0 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.199+0000 7f0f05600640 1 -- 172.21.15.32:0/3804654569 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0f08033050 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.199+0000 7f0f255e7740 10 client.17166 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.199+0000 7f0f255e7740 1 -- 172.21.15.32:0/3804654569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aa400e4080 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.200+0000 7f0f05600640 1 -- 172.21.15.32:0/3804654569 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f080043d0 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.200+0000 7f0f05600640 1 -- 172.21.15.32:0/3804654569 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f0f08004b70 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.200+0000 7f0f255e7740 10 client.17166 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.200+0000 7f0f255e7740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.200+0000 7f0f255e7740 20 client.17166 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.200+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aa3fce29f0 0x55aa400e7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.200+0000 7f0f255e7740 4 client.17166 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.200+0000 7f0f255e7740 1 -- 172.21.15.32:0/3804654569 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa400426c0 con 0x55aa3fce29f0 2024-08-31T07:20:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.200+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aa3fce29f0 0x55aa400e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.201+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aa3fce29f0 0x55aa400e7440 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbae0ff6f27f63d0 server_cookie=17500dbeb42d0fc7 in_seq=0 out_seq=0 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.201+0000 7f0f05600640 10 client.17166 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f05600640 1 -- 172.21.15.32:0/3804654569 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aa400426c0 con 0x55aa3fce29f0 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f05600640 10 client.17166 handle_command_reply: tid=0 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 client.17166 shutdown 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 -- 172.21.15.32:0/3804654569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 msgr2=0x55aa400d54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0f08003f50 tx=0x7f0f080038a0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 -- 172.21.15.32:0/3804654569 shutdown_connections 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aa3fce29f0 0x55aa400e7440 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd24070 0x55aa400e1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd03390 0x55aa400d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3804654569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 -- 172.21.15.32:0/3804654569 >> 172.21.15.32:0/3804654569 conn(0x55aa3fd18460 msgr2=0x55aa400ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f05600640 0 client.17166 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 -- 172.21.15.32:0/3804654569 shutdown_connections 2024-08-31T07:20:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.202+0000 7f0f255e7740 1 -- 172.21.15.32:0/3804654569 wait complete. 2024-08-31T07:20:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.203+0000 7f0f255e7740 20 client.17166 trim_cache size 0 max 16384 2024-08-31T07:20:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.221+0000 7f0f255e7740 1 Processor -- start 2024-08-31T07:20:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.221+0000 7f0f255e7740 1 -- start start 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.221+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.221+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 0x55aa400cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.221+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.221+0000 7f0f255e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181b1540 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.221+0000 7f0f255e7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181b3140 con 0x55aa3fd03390 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.221+0000 7f0f255e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181b1c10 con 0x55aa3fd24070 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56916/0 (socket says 172.21.15.32:56916) 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17400640 1 -- 172.21.15.32:0/3082531640 learned_addr learned my addr 172.21.15.32:0/3082531640 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/3082531640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 0x55aa400cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17e00640 1 --2- 172.21.15.32:0/3082531640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17400640 1 -- 172.21.15.32:0/3082531640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 msgr2=0x55aa400c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17400640 1 --2- 172.21.15.32:0/3082531640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17400640 1 -- 172.21.15.32:0/3082531640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 msgr2=0x55aa400cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17400640 1 --2- 172.21.15.32:0/3082531640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 0x55aa400cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17400640 1 -- 172.21.15.32:0/3082531640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa400d4230 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17e00640 1 --2- 172.21.15.32:0/3082531640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.222+0000 7f0f17400640 1 --2- 172.21.15.32:0/3082531640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0f08005e00 tx=0x7f0f080059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf5ed520d8a4d48 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.223+0000 7f0f07e00640 1 -- 172.21.15.32:0/3082531640 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08027070 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.223+0000 7f0f07e00640 1 -- 172.21.15.32:0/3082531640 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f0803c070 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.223+0000 7f0f07e00640 1 -- 172.21.15.32:0/3082531640 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08005320 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 -- 172.21.15.32:0/3082531640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 msgr2=0x55aa400d54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3082531640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0f08005e00 tx=0x7f0f080059b0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 -- 172.21.15.32:0/3082531640 shutdown_connections 2024-08-31T07:20:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3082531640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd24070 0x55aa400c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3082531640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fd03390 0x55aa400cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3082531640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d54b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 -- 172.21.15.32:0/3082531640 >> 172.21.15.32:0/3082531640 conn(0x55aa3fd18460 msgr2=0x55aa400cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 -- 172.21.15.32:0/3082531640 shutdown_connections 2024-08-31T07:20:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 -- 172.21.15.32:0/3082531640 wait complete. 2024-08-31T07:20:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 -- start start 2024-08-31T07:20:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.224+0000 7f0f255e7740 1 Processor -- start 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd03390 0x55aa400da000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f255e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fd24070 0x55aa40150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f255e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181b29b0 con 0x55aa3fd24070 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f255e7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181b22e0 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f255e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181af540 con 0x55aa3fd03390 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f255e7740 10 client.0 _ll_register_callbacks cb 0x7f0f08005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fd24070 0x55aa40150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f17400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fd24070 0x55aa40150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56928/0 (socket says 172.21.15.32:56928) 2024-08-31T07:20:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f17e00640 1 -- 172.21.15.32:0/3713744229 learned_addr learned my addr 172.21.15.32:0/3713744229 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.225+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd03390 0x55aa400da000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.226+0000 7f0f17400640 1 -- 172.21.15.32:0/3713744229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd03390 msgr2=0x55aa400da000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.226+0000 7f0f17400640 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd03390 0x55aa400da000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.226+0000 7f0f17400640 1 -- 172.21.15.32:0/3713744229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fd24070 msgr2=0x55aa40150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.226+0000 7f0f17400640 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fd24070 0x55aa40150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.226+0000 7f0f17400640 1 -- 172.21.15.32:0/3713744229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa40151330 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.226+0000 7f0f17400640 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d9ac0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f08039040 tx=0x7f0f08011d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea04702a6ab1edf7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.226+0000 7f0f05600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.226+0000 7f0f05600640 1 -- 172.21.15.32:0/3713744229 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08027040 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.227+0000 7f0f05600640 1 -- 172.21.15.32:0/3713744229 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f0f08019d90 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.227+0000 7f0f255e7740 1 -- 172.21.15.32:0/3713744229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f0803c070 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.227+0000 7f0f05600640 1 -- 172.21.15.32:0/3713744229 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0f0804e050 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.227+0000 7f0f255e7740 10 client.27185 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.227+0000 7f0f255e7740 1 -- 172.21.15.32:0/3713744229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aa401525c0 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.227+0000 7f0f05600640 1 -- 172.21.15.32:0/3713744229 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f0803c070 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.227+0000 7f0f05600640 1 -- 172.21.15.32:0/3713744229 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f0f080235f0 con 0x55aa3fcfb1b0 2024-08-31T07:20:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.227+0000 7f0f255e7740 10 client.27185 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.227+0000 7f0f255e7740 10 client.27185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:20:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.227+0000 7f0f255e7740 20 client.27185 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.228+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aa3fcb7da0 0x55aa40154d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.228+0000 7f0f255e7740 4 client.27185 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.228+0000 7f0f255e7740 1 -- 172.21.15.32:0/3713744229 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa4003e160 con 0x55aa3fcb7da0 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.228+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aa3fcb7da0 0x55aa40154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.228+0000 7f0f0ea00640 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aa3fcb7da0 0x55aa40154d10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b29bb474ce5ccd38 server_cookie=419a4f8fed52b7de in_seq=0 out_seq=0 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.228+0000 7f0f05600640 10 client.27185 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f05600640 1 -- 172.21.15.32:0/3713744229 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55aa4003e160 con 0x55aa3fcb7da0 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f05600640 10 client.27185 handle_command_reply: tid=0 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 client.27185 shutdown 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 -- 172.21.15.32:0/3713744229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fcfb1b0 msgr2=0x55aa400d9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d9ac0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f08039040 tx=0x7f0f08011d80 comp rx=0 tx=0).stop 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 -- 172.21.15.32:0/3713744229 shutdown_connections 2024-08-31T07:20:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55aa3fcb7da0 0x55aa40154d10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa3fd24070 0x55aa40150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa3fd03390 0x55aa400da000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f05600640 0 client.27185 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 --2- 172.21.15.32:0/3713744229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3fcfb1b0 0x55aa400d9ac0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 -- 172.21.15.32:0/3713744229 >> 172.21.15.32:0/3713744229 conn(0x55aa3fd18460 msgr2=0x55aa400ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 -- 172.21.15.32:0/3713744229 shutdown_connections 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 1 -- 172.21.15.32:0/3713744229 wait complete. 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.229+0000 7f0f255e7740 20 client.27185 trim_cache size 0 max 16384 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:20:44.234 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:44.235 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:20:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:20:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:44.241 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:20:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.230+0000 7f0f1e800640 1 -- 172.21.15.32:0/419293057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0efc0b4ca0 msgr2=0x7f0efc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.230+0000 7f0f1e800640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0efc0b4ca0 0x7f0efc0b71b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0f1002f490 tx=0x7f0f100099a0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.230+0000 7f0f1e800640 1 -- 172.21.15.32:0/419293057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f18109240 msgr2=0x7f0f1819c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.230+0000 7f0f1e800640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f18109240 0x7f0f1819c950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e9c0 tx=0x7f0f0000ee80 comp rx=0 tx=0).stop 2024-08-31T07:20:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.230+0000 7f0f1e800640 1 -- 172.21.15.32:0/419293057 shutdown_connections 2024-08-31T07:20:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.230+0000 7f0f1e800640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0efc0b4ca0 0x7f0efc0b71b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.230+0000 7f0f1e800640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f18109b90 0x7f0f181a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.230+0000 7f0f1e800640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f18109240 0x7f0f1819c950 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.230+0000 7f0f1e800640 1 --2- 172.21.15.32:0/419293057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f18103900 0x7f0f1819c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.230+0000 7f0f1e800640 1 -- 172.21.15.32:0/419293057 >> 172.21.15.32:0/419293057 conn(0x7f0f1807b5f0 msgr2=0x7f0f18107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.231+0000 7f0f1e800640 1 -- 172.21.15.32:0/419293057 shutdown_connections 2024-08-31T07:20:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:44.231+0000 7f0f1e800640 1 -- 172.21.15.32:0/419293057 wait complete. 2024-08-31T07:20:44.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.64 MB/sec execute 273 sec latency 23842.226 ms 2024-08-31T07:20:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:44 smithi073 ceph-mon[21995]: pgmap v611: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 73 KiB/s wr, 16 op/s 2024-08-31T07:20:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:44 smithi084 ceph-mon[22387]: pgmap v611: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 73 KiB/s wr, 16 op/s 2024-08-31T07:20:45.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:44 smithi032 ceph-mon[17252]: pgmap v611: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 73 KiB/s wr, 16 op/s 2024-08-31T07:20:45.281 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:20:45.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:20:45.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:20:45.687 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:45.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.63 MB/sec execute 274 sec latency 24842.279 ms 2024-08-31T07:20:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.419+0000 7efe2f600640 1 -- 172.21.15.32:0/3291779340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe281141e0 msgr2=0x7efe281166b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.419+0000 7efe2f600640 1 --2- 172.21.15.32:0/3291779340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe281141e0 0x7efe281166b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efe20009ca0 tx=0x7efe2002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:20:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.419+0000 7efe2f600640 1 -- 172.21.15.32:0/3291779340 shutdown_connections 2024-08-31T07:20:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.419+0000 7efe2f600640 1 --2- 172.21.15.32:0/3291779340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe281141e0 0x7efe281166b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.419+0000 7efe2f600640 1 --2- 172.21.15.32:0/3291779340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe28075b70 0x7efe28075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.419+0000 7efe2f600640 1 --2- 172.21.15.32:0/3291779340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe28077120 0x7efe28075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.419+0000 7efe2f600640 1 -- 172.21.15.32:0/3291779340 >> 172.21.15.32:0/3291779340 conn(0x7efe280ff380 msgr2=0x7efe281017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.420+0000 7efe2f600640 1 -- 172.21.15.32:0/3291779340 shutdown_connections 2024-08-31T07:20:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.420+0000 7efe2f600640 1 -- 172.21.15.32:0/3291779340 wait complete. 2024-08-31T07:20:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.421+0000 7efe2f600640 1 Processor -- start 2024-08-31T07:20:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.421+0000 7efe2f600640 1 -- start start 2024-08-31T07:20:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe2f600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe28075b70 0x7efe281a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe2f600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe28077120 0x7efe281a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe2f600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe281141e0 0x7efe281a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe2f600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efe28119e20 con 0x7efe281141e0 2024-08-31T07:20:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe2f600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe28119ca0 con 0x7efe28077120 2024-08-31T07:20:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe2f600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe28119fa0 con 0x7efe28075b70 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe2cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe28075b70 0x7efe281a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe2d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe281141e0 0x7efe281a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe2cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe28075b70 0x7efe281a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60380/0 (socket says 172.21.15.32:60380) 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe2cc00640 1 -- 172.21.15.32:0/1039371958 learned_addr learned my addr 172.21.15.32:0/1039371958 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.422+0000 7efe27e00640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe28077120 0x7efe281a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.423+0000 7efe2cc00640 1 -- 172.21.15.32:0/1039371958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe28077120 msgr2=0x7efe281a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.423+0000 7efe2cc00640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe28077120 0x7efe281a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.423+0000 7efe2cc00640 1 -- 172.21.15.32:0/1039371958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe281141e0 msgr2=0x7efe281a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.423+0000 7efe2cc00640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe281141e0 0x7efe281a6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.423+0000 7efe2cc00640 1 -- 172.21.15.32:0/1039371958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe281a6780 con 0x7efe28075b70 2024-08-31T07:20:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.423+0000 7efe2cc00640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe28075b70 0x7efe281a0810 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00ea60 tx=0x7efe1c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.423+0000 7efe25600640 1 -- 172.21.15.32:0/1039371958 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c00cde0 con 0x7efe28075b70 2024-08-31T07:20:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.424+0000 7efe25600640 1 -- 172.21.15.32:0/1039371958 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe1c0035f0 con 0x7efe28075b70 2024-08-31T07:20:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.424+0000 7efe2f600640 1 -- 172.21.15.32:0/1039371958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe281a6a70 con 0x7efe28075b70 2024-08-31T07:20:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.424+0000 7efe25600640 1 -- 172.21.15.32:0/1039371958 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c010690 con 0x7efe28075b70 2024-08-31T07:20:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.424+0000 7efe2f600640 1 -- 172.21.15.32:0/1039371958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe281ad1d0 con 0x7efe28075b70 2024-08-31T07:20:46.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.425+0000 7efe2f600640 1 -- 172.21.15.32:0/1039371958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe28075f80 con 0x7efe28075b70 2024-08-31T07:20:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.428+0000 7efe25600640 1 -- 172.21.15.32:0/1039371958 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7efe1c0037a0 con 0x7efe28075b70 2024-08-31T07:20:46.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.430+0000 7efe25600640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe100b4d20 0x7efe100b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.430+0000 7efe25600640 1 -- 172.21.15.32:0/1039371958 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7efe1c0e5930 con 0x7efe28075b70 2024-08-31T07:20:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.430+0000 7efe27e00640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe100b4d20 0x7efe100b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.431+0000 7efe25600640 1 -- 172.21.15.32:0/1039371958 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efe1c056130 con 0x7efe28075b70 2024-08-31T07:20:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.432+0000 7efe27e00640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe100b4d20 0x7efe100b7230 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efe28100270 tx=0x7efe14005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.670+0000 7efe2f600640 1 -- 172.21.15.32:0/1039371958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe281a1940 con 0x7efe28075b70 2024-08-31T07:20:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.671+0000 7efe25600640 1 -- 172.21.15.32:0/1039371958 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11967 (secure 0 0 0) 0x7efe1c0ac710 con 0x7efe28075b70 2024-08-31T07:20:46.672 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:20:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 35 2024-08-31T07:20:46.673 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"btime":"2024-08-31T07:19:16:021228+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:19:15.946242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:20:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.673+0000 7efe2f600640 1 -- 172.21.15.32:0/1039371958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe100b4d20 msgr2=0x7efe100b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.673+0000 7efe2f600640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe100b4d20 0x7efe100b7230 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efe28100270 tx=0x7efe14005f90 comp rx=0 tx=0).stop 2024-08-31T07:20:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.673+0000 7efe2f600640 1 -- 172.21.15.32:0/1039371958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe28075b70 msgr2=0x7efe281a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.673+0000 7efe2f600640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe28075b70 0x7efe281a0810 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00ea60 tx=0x7efe1c00ef20 comp rx=0 tx=0).stop 2024-08-31T07:20:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.674+0000 7efe2f600640 1 -- 172.21.15.32:0/1039371958 shutdown_connections 2024-08-31T07:20:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.674+0000 7efe2f600640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe100b4d20 0x7efe100b7230 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.674+0000 7efe2f600640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe281141e0 0x7efe281a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.674+0000 7efe2f600640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe28077120 0x7efe281a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.674+0000 7efe2f600640 1 --2- 172.21.15.32:0/1039371958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe28075b70 0x7efe281a0810 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.674+0000 7efe2f600640 1 -- 172.21.15.32:0/1039371958 >> 172.21.15.32:0/1039371958 conn(0x7efe280ff380 msgr2=0x7efe281100e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.674+0000 7efe2f600640 1 -- 172.21.15.32:0/1039371958 shutdown_connections 2024-08-31T07:20:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:46.674+0000 7efe2f600640 1 -- 172.21.15.32:0/1039371958 wait complete. 2024-08-31T07:20:46.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.62 MB/sec execute 275 sec latency 25842.344 ms 2024-08-31T07:20:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:46 smithi073 ceph-mon[21995]: pgmap v612: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-08-31T07:20:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:46 smithi084 ceph-mon[22387]: pgmap v612: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-08-31T07:20:46.993 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:46 smithi032 ceph-mon[17252]: pgmap v612: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-08-31T07:20:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.443+0000 7faab7200640 1 -- 172.21.15.32:0/246249799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8109090 msgr2=0x7faab81094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.443+0000 7faab7200640 1 --2- 172.21.15.32:0/246249799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8109090 0x7faab81094a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009900 tx=0x7faaa402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.443+0000 7faab7200640 1 -- 172.21.15.32:0/246249799 shutdown_connections 2024-08-31T07:20:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.443+0000 7faab7200640 1 --2- 172.21.15.32:0/246249799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab81099e0 0x7faab8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.443+0000 7faab7200640 1 --2- 172.21.15.32:0/246249799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8109090 0x7faab81094a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.443+0000 7faab7200640 1 --2- 172.21.15.32:0/246249799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab8103750 0x7faab8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.443+0000 7faab7200640 1 -- 172.21.15.32:0/246249799 >> 172.21.15.32:0/246249799 conn(0x7faab80ff1e0 msgr2=0x7faab8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.443+0000 7faab7200640 1 -- 172.21.15.32:0/246249799 shutdown_connections 2024-08-31T07:20:47.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.443+0000 7faab7200640 1 -- 172.21.15.32:0/246249799 wait complete. 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.444+0000 7faab7200640 1 Processor -- start 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.444+0000 7faab7200640 1 -- start start 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8103750 0x7faab80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab7200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab8109090 0x7faab80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab81099e0 0x7faab8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faab8113670 con 0x7faab81099e0 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faab81134f0 con 0x7faab8103750 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab7200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faab81137f0 con 0x7faab8109090 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8103750 0x7faab80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8103750 0x7faab80754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37398/0 (socket says 172.21.15.32:37398) 2024-08-31T07:20:47.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab5e00640 1 -- 172.21.15.32:0/4157164474 learned_addr learned my addr 172.21.15.32:0/4157164474 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:47.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab5400640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab8109090 0x7faab80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:47.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.445+0000 7faab6800640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab81099e0 0x7faab8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:47.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.446+0000 7faab5e00640 1 -- 172.21.15.32:0/4157164474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab8109090 msgr2=0x7faab80759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.446+0000 7faab5e00640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab8109090 0x7faab80759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.446+0000 7faab5e00640 1 -- 172.21.15.32:0/4157164474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab81099e0 msgr2=0x7faab8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.446+0000 7faab5e00640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab81099e0 0x7faab8075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.446+0000 7faab5e00640 1 -- 172.21.15.32:0/4157164474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab81a8380 con 0x7faab8103750 2024-08-31T07:20:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.446+0000 7faab5e00640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8103750 0x7faab80754b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faaa800e9f0 tx=0x7faaa800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.446+0000 7faaaea00640 1 -- 172.21.15.32:0/4157164474 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa800cd80 con 0x7faab8103750 2024-08-31T07:20:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.446+0000 7faab7200640 1 -- 172.21.15.32:0/4157164474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab81a8670 con 0x7faab8103750 2024-08-31T07:20:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.446+0000 7faaaea00640 1 -- 172.21.15.32:0/4157164474 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa80054d0 con 0x7faab8103750 2024-08-31T07:20:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.447+0000 7faaaea00640 1 -- 172.21.15.32:0/4157164474 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8010470 con 0x7faab8103750 2024-08-31T07:20:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.447+0000 7faab7200640 1 -- 172.21.15.32:0/4157164474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab81a8b80 con 0x7faab8103750 2024-08-31T07:20:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.448+0000 7faab7200640 1 -- 172.21.15.32:0/4157164474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab81049d0 con 0x7faab8103750 2024-08-31T07:20:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.450+0000 7faaaea00640 1 -- 172.21.15.32:0/4157164474 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7faaa80046c0 con 0x7faab8103750 2024-08-31T07:20:47.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.452+0000 7faaaea00640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa9c0b4ca0 0x7faa9c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.452+0000 7faaaea00640 1 -- 172.21.15.32:0/4157164474 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7faaa80e5b10 con 0x7faab8103750 2024-08-31T07:20:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.452+0000 7faab5400640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa9c0b4ca0 0x7faa9c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:47.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.453+0000 7faab5400640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa9c0b4ca0 0x7faa9c0b71b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009230 tx=0x7faaa4031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.453+0000 7faaaea00640 1 -- 172.21.15.32:0/4157164474 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faaa80ac8f0 con 0x7faab8103750 2024-08-31T07:20:47.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.695+0000 7faab7200640 1 -- 172.21.15.32:0/4157164474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faab8103b60 con 0x7faab8103750 2024-08-31T07:20:47.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.697+0000 7faaaea00640 1 -- 172.21.15.32:0/4157164474 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11967 (secure 0 0 0) 0x7faaa80ac1f0 con 0x7faab8103750 2024-08-31T07:20:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:20:47.698 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"btime":"2024-08-31T07:19:16:021228+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:19:15.946242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:20:47.698 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 35 2024-08-31T07:20:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 -- 172.21.15.32:0/4157164474 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa9c0b4ca0 msgr2=0x7faa9c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa9c0b4ca0 0x7faa9c0b71b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009230 tx=0x7faaa4031400 comp rx=0 tx=0).stop 2024-08-31T07:20:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 -- 172.21.15.32:0/4157164474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8103750 msgr2=0x7faab80754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8103750 0x7faab80754b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faaa800e9f0 tx=0x7faaa800edf0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 -- 172.21.15.32:0/4157164474 shutdown_connections 2024-08-31T07:20:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa9c0b4ca0 0x7faa9c0b71b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab81099e0 0x7faab8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab8109090 0x7faab80759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 --2- 172.21.15.32:0/4157164474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8103750 0x7faab80754b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 -- 172.21.15.32:0/4157164474 >> 172.21.15.32:0/4157164474 conn(0x7faab80ff1e0 msgr2=0x7faab8101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.699+0000 7faab7200640 1 -- 172.21.15.32:0/4157164474 shutdown_connections 2024-08-31T07:20:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:47.700+0000 7faab7200640 1 -- 172.21.15.32:0/4157164474 wait complete. 2024-08-31T07:20:47.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.61 MB/sec execute 276 sec latency 26842.386 ms 2024-08-31T07:20:47.883 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:47 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1039371958' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:47.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:20:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:47 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1039371958' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:47.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:47 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1039371958' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:48.625 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id d2dfb577 --release --await' 2024-08-31T07:20:48.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id d2dfb577 --release --await 2024-08-31T07:20:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:48.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.60 MB/sec execute 277 sec latency 27842.449 ms 2024-08-31T07:20:48.844 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:48 smithi032 ceph-mon[17252]: pgmap v613: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-08-31T07:20:48.844 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:48 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4157164474' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:48 smithi073 ceph-mon[21995]: pgmap v613: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-08-31T07:20:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:48 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4157164474' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:48 smithi084 ceph-mon[22387]: pgmap v613: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-08-31T07:20:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:48 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4157164474' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:20:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:49.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.382+0000 7f9a7f200640 1 -- 172.21.15.32:0/912764749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a78102b30 msgr2=0x7f9a7810f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:49.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.382+0000 7f9a7f200640 1 --2- 172.21.15.32:0/912764749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a78102b30 0x7f9a7810f850 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009860 tx=0x7f9a6802a3d0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.383+0000 7f9a7f200640 1 -- 172.21.15.32:0/912764749 shutdown_connections 2024-08-31T07:20:49.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.383+0000 7f9a7f200640 1 --2- 172.21.15.32:0/912764749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a7810fd90 0x7f9a781121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.383+0000 7f9a7f200640 1 --2- 172.21.15.32:0/912764749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a78102b30 0x7f9a7810f850 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.383+0000 7f9a7f200640 1 --2- 172.21.15.32:0/912764749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a781021e0 0x7f9a781025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.383+0000 7f9a7f200640 1 -- 172.21.15.32:0/912764749 >> 172.21.15.32:0/912764749 conn(0x7f9a780fe060 msgr2=0x7f9a781004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:49.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.383+0000 7f9a7f200640 1 -- 172.21.15.32:0/912764749 shutdown_connections 2024-08-31T07:20:49.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.383+0000 7f9a7f200640 1 -- 172.21.15.32:0/912764749 wait complete. 2024-08-31T07:20:49.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.384+0000 7f9a7f200640 1 Processor -- start 2024-08-31T07:20:49.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.384+0000 7f9a7f200640 1 -- start start 2024-08-31T07:20:49.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.384+0000 7f9a7f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a781021e0 0x7f9a7819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.384+0000 7f9a7f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a78102b30 0x7f9a7819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.384+0000 7f9a7f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a7810fd90 0x7f9a781a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.384+0000 7f9a7f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78115840 con 0x7f9a781021e0 2024-08-31T07:20:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.384+0000 7f9a7f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781156c0 con 0x7f9a78102b30 2024-08-31T07:20:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.384+0000 7f9a7f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781159c0 con 0x7f9a7810fd90 2024-08-31T07:20:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a781021e0 0x7f9a7819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a781021e0 0x7f9a7819c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58816/0 (socket says 172.21.15.32:58816) 2024-08-31T07:20:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7de00640 1 -- 172.21.15.32:0/1075477666 learned_addr learned my addr 172.21.15.32:0/1075477666 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7e800640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a7810fd90 0x7f9a781a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7d400640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a78102b30 0x7f9a7819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7d400640 1 -- 172.21.15.32:0/1075477666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a7810fd90 msgr2=0x7f9a781a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7d400640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a7810fd90 0x7f9a781a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7d400640 1 -- 172.21.15.32:0/1075477666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a781021e0 msgr2=0x7f9a7819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7d400640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a781021e0 0x7f9a7819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7d400640 1 -- 172.21.15.32:0/1075477666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a781a21e0 con 0x7f9a78102b30 2024-08-31T07:20:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.385+0000 7f9a7de00640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a781021e0 0x7f9a7819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.386+0000 7f9a7d400640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a78102b30 0x7f9a7819c810 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9a6802a8f0 tx=0x7f9a68030fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.386+0000 7f9a72a00640 1 -- 172.21.15.32:0/1075477666 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68040070 con 0x7f9a78102b30 2024-08-31T07:20:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.386+0000 7f9a7f200640 1 -- 172.21.15.32:0/1075477666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a781a2470 con 0x7f9a78102b30 2024-08-31T07:20:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.386+0000 7f9a72a00640 1 -- 172.21.15.32:0/1075477666 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a68005570 con 0x7f9a78102b30 2024-08-31T07:20:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.386+0000 7f9a72a00640 1 -- 172.21.15.32:0/1075477666 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680046c0 con 0x7f9a78102b30 2024-08-31T07:20:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.386+0000 7f9a7f200640 1 -- 172.21.15.32:0/1075477666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a781a8c30 con 0x7f9a78102b30 2024-08-31T07:20:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.389+0000 7f9a8741a740 1 Processor -- start 2024-08-31T07:20:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a8741a740 1 -- start start 2024-08-31T07:20:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdeb070 0x558dac190b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac1910a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdca390 0x558dac1915e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a8741a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558dac1077b0 con 0x558dabdc21b0 2024-08-31T07:20:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a8741a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558dac0f6d40 con 0x558dabdeb070 2024-08-31T07:20:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a8741a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558dac0facf0 con 0x558dabdca390 2024-08-31T07:20:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a7de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac1910a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a7de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac1910a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58822/0 (socket says 172.21.15.32:58822) 2024-08-31T07:20:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a7de00640 1 -- 172.21.15.32:0/693834965 learned_addr learned my addr 172.21.15.32:0/693834965 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.390+0000 7f9a7e800640 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdeb070 0x558dac190b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.391+0000 7f9a72a00640 1 -- 172.21.15.32:0/1075477666 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9a68043070 con 0x7f9a78102b30 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.391+0000 7f9a7d400640 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdca390 0x558dac1915e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.391+0000 7f9a7de00640 1 -- 172.21.15.32:0/693834965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdca390 msgr2=0x558dac1915e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.391+0000 7f9a7de00640 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdca390 0x558dac1915e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.391+0000 7f9a7de00640 1 -- 172.21.15.32:0/693834965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdeb070 msgr2=0x558dac190b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.391+0000 7f9a7de00640 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdeb070 0x558dac190b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.391+0000 7f9a7de00640 1 -- 172.21.15.32:0/693834965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dabe56650 con 0x558dabdc21b0 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.391+0000 7f9a7d400640 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdca390 0x558dac1915e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.391+0000 7f9a7e800640 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdeb070 0x558dac190b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.391+0000 7f9a7de00640 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac1910a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c011630 tx=0x7f9a6c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506db8e3e5557701 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:49.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.392+0000 7f9a67000640 1 -- 172.21.15.32:0/693834965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c01e070 con 0x558dabdc21b0 2024-08-31T07:20:49.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.392+0000 7f9a67000640 1 -- 172.21.15.32:0/693834965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a6c00fba0 con 0x558dabdc21b0 2024-08-31T07:20:49.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.392+0000 7f9a67000640 1 -- 172.21.15.32:0/693834965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c019ca0 con 0x558dabdc21b0 2024-08-31T07:20:49.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.393+0000 7f9a72a00640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a600b4ca0 0x7f9a600b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.393+0000 7f9a72a00640 1 -- 172.21.15.32:0/1075477666 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f9a68105fd0 con 0x7f9a78102b30 2024-08-31T07:20:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.393+0000 7f9a7e800640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a600b4ca0 0x7f9a600b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.393+0000 7f9a8741a740 1 -- 172.21.15.32:0/693834965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 msgr2=0x558dac1910a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.393+0000 7f9a8741a740 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac1910a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c011630 tx=0x7f9a6c011af0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.393+0000 7f9a8741a740 1 -- 172.21.15.32:0/693834965 shutdown_connections 2024-08-31T07:20:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.393+0000 7f9a8741a740 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdca390 0x558dac1915e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.393+0000 7f9a8741a740 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac1910a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.393+0000 7f9a8741a740 1 --2- 172.21.15.32:0/693834965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdeb070 0x558dac190b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.393+0000 7f9a8741a740 1 -- 172.21.15.32:0/693834965 >> 172.21.15.32:0/693834965 conn(0x558dabddf460 msgr2=0x558dac18ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.394+0000 7f9a8741a740 1 -- 172.21.15.32:0/693834965 shutdown_connections 2024-08-31T07:20:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.394+0000 7f9a8741a740 1 -- 172.21.15.32:0/693834965 wait complete. 2024-08-31T07:20:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.394+0000 7f9a7e800640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a600b4ca0 0x7f9a600b71b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009fd0 tx=0x7f9a74010040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.394+0000 7f9a8741a740 1 -- start start 2024-08-31T07:20:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.395+0000 7f9a8741a740 1 Processor -- start 2024-08-31T07:20:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.395+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdc21b0 0x558dac19c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdca390 0x558dac19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac1a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a8741a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558dabf59700 con 0x558dabdca390 2024-08-31T07:20:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a8741a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558dac07dcf0 con 0x558dabdc21b0 2024-08-31T07:20:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a8741a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558dac19f340 con 0x558dabdeb070 2024-08-31T07:20:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a8741a740 10 client.0 _ll_register_callbacks cb 0x558dac0a3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdc21b0 0x558dac19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdc21b0 0x558dac19c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37416/0 (socket says 172.21.15.32:37416) 2024-08-31T07:20:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7de00640 1 -- 172.21.15.32:0/1504376922 learned_addr learned my addr 172.21.15.32:0/1504376922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7e800640 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac1a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7d400640 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdca390 0x558dac19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7de00640 1 -- 172.21.15.32:0/1504376922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 msgr2=0x558dac1a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7de00640 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac1a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7de00640 1 -- 172.21.15.32:0/1504376922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdca390 msgr2=0x558dac19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7de00640 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdca390 0x558dac19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7de00640 1 -- 172.21.15.32:0/1504376922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dac0be8b0 con 0x558dabdc21b0 2024-08-31T07:20:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.396+0000 7f9a7d400640 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdca390 0x558dac19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.397+0000 7f9a7de00640 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdc21b0 0x558dac19c740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009f40 tx=0x7f9a6c00fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d42c9eb92a21f69a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.397+0000 7f9a5b400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.397+0000 7f9a5b400640 1 -- 172.21.15.32:0/1504376922 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c026020 con 0x558dabdc21b0 2024-08-31T07:20:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.397+0000 7f9a5b400640 1 -- 172.21.15.32:0/1504376922 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f9a6c00fd50 con 0x558dabdc21b0 2024-08-31T07:20:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.397+0000 7f9a8741a740 1 -- 172.21.15.32:0/1504376922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a6c00fba0 con 0x558dabdc21b0 2024-08-31T07:20:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.397+0000 7f9a5b400640 1 -- 172.21.15.32:0/1504376922 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9a6c031050 con 0x558dabdc21b0 2024-08-31T07:20:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.398+0000 7f9a8741a740 10 client.27209 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.398+0000 7f9a8741a740 1 -- 172.21.15.32:0/1504376922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558dac1ab2c0 con 0x558dabdc21b0 2024-08-31T07:20:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.399+0000 7f9a5b400640 1 -- 172.21.15.32:0/1504376922 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a6c00fba0 con 0x558dabdc21b0 2024-08-31T07:20:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.399+0000 7f9a5b400640 1 -- 172.21.15.32:0/1504376922 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f9a6c021070 con 0x558dabdc21b0 2024-08-31T07:20:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.399+0000 7f9a8741a740 10 client.27209 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.399+0000 7f9a8741a740 10 client.27209 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.399+0000 7f9a8741a740 20 client.27209 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.399+0000 7f9a8741a740 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558dabda99f0 0x558dac1ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.399+0000 7f9a8741a740 4 client.27209 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.399+0000 7f9a8741a740 1 -- 172.21.15.32:0/1504376922 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dac109700 con 0x558dabda99f0 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.400+0000 7f9a7d400640 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558dabda99f0 0x558dac1ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.401+0000 7f9a5b400640 10 client.27209 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.401+0000 7f9a7d400640 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558dabda99f0 0x558dac1ae6d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=19f10214d3bbb555 server_cookie=9e7077e659e00a31 in_seq=0 out_seq=0 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a5b400640 1 -- 172.21.15.32:0/1504376922 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558dac109700 con 0x558dabda99f0 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a5b400640 10 client.27209 handle_command_reply: tid=0 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 client.27209 shutdown 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 -- 172.21.15.32:0/1504376922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdc21b0 msgr2=0x558dac19c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdc21b0 0x558dac19c740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009f40 tx=0x7f9a6c00fb20 comp rx=0 tx=0).stop 2024-08-31T07:20:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 -- 172.21.15.32:0/1504376922 shutdown_connections 2024-08-31T07:20:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558dabda99f0 0x558dac1ae6d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac1a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdca390 0x558dac19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 --2- 172.21.15.32:0/1504376922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdc21b0 0x558dac19c740 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 -- 172.21.15.32:0/1504376922 >> 172.21.15.32:0/1504376922 conn(0x558dabddf460 msgr2=0x558dac193cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a5b400640 0 client.27209 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:20:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 -- 172.21.15.32:0/1504376922 shutdown_connections 2024-08-31T07:20:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.402+0000 7f9a8741a740 1 -- 172.21.15.32:0/1504376922 wait complete. 2024-08-31T07:20:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.403+0000 7f9a8741a740 20 client.27209 trim_cache size 0 max 16384 2024-08-31T07:20:49.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.421+0000 7f9a8741a740 1 Processor -- start 2024-08-31T07:20:49.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.421+0000 7f9a8741a740 1 -- start start 2024-08-31T07:20:49.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.421+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac195de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac196320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a8741a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781b1320 con 0x558dabdc21b0 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a8741a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781b2f20 con 0x558dabdca390 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a8741a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781b19f0 con 0x558dabdeb070 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37430/0 (socket says 172.21.15.32:37430) 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a7d400640 1 -- 172.21.15.32:0/680645886 learned_addr learned my addr 172.21.15.32:0/680645886 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a7de00640 1 --2- 172.21.15.32:0/680645886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a7d400640 1 -- 172.21.15.32:0/680645886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 msgr2=0x558dac19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a7d400640 1 --2- 172.21.15.32:0/680645886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a7d400640 1 -- 172.21.15.32:0/680645886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 msgr2=0x558dac195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a7d400640 1 --2- 172.21.15.32:0/680645886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a7d400640 1 -- 172.21.15.32:0/680645886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dac192e50 con 0x558dabdca390 2024-08-31T07:20:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.422+0000 7f9a7de00640 1 --2- 172.21.15.32:0/680645886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.423+0000 7f9a7d400640 1 --2- 172.21.15.32:0/680645886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac196320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9a68000c00 tx=0x7f9a680c8040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac3f0bde8d4f07b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.423+0000 7f9a66600640 1 -- 172.21.15.32:0/680645886 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6804a040 con 0x558dabdca390 2024-08-31T07:20:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.423+0000 7f9a66600640 1 -- 172.21.15.32:0/680645886 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a68031400 con 0x558dabdca390 2024-08-31T07:20:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.423+0000 7f9a66600640 1 -- 172.21.15.32:0/680645886 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680d5a30 con 0x558dabdca390 2024-08-31T07:20:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.424+0000 7f9a8741a740 1 -- 172.21.15.32:0/680645886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 msgr2=0x558dac196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:49.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.424+0000 7f9a8741a740 1 --2- 172.21.15.32:0/680645886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac196320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9a68000c00 tx=0x7f9a680c8040 comp rx=0 tx=0).stop 2024-08-31T07:20:49.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.424+0000 7f9a8741a740 1 -- 172.21.15.32:0/680645886 shutdown_connections 2024-08-31T07:20:49.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.424+0000 7f9a8741a740 1 --2- 172.21.15.32:0/680645886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.424+0000 7f9a8741a740 1 --2- 172.21.15.32:0/680645886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac196320 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.424+0000 7f9a8741a740 1 --2- 172.21.15.32:0/680645886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.424+0000 7f9a8741a740 1 -- 172.21.15.32:0/680645886 >> 172.21.15.32:0/680645886 conn(0x558dabddf460 msgr2=0x558dac1976f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:49.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.424+0000 7f9a8741a740 1 -- 172.21.15.32:0/680645886 shutdown_connections 2024-08-31T07:20:49.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.424+0000 7f9a8741a740 1 -- 172.21.15.32:0/680645886 wait complete. 2024-08-31T07:20:49.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.425+0000 7f9a8741a740 1 -- start start 2024-08-31T07:20:49.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.425+0000 7f9a8741a740 1 Processor -- start 2024-08-31T07:20:49.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac195860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac195da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a8741a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a8741a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781b2790 con 0x558dabdc21b0 2024-08-31T07:20:49.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a8741a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781b20c0 con 0x558dabdca390 2024-08-31T07:20:49.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a8741a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9a781af320 con 0x558dabdeb070 2024-08-31T07:20:49.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a8741a740 10 client.0 _ll_register_callbacks cb 0x7f9a6c0114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a7de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37440/0 (socket says 172.21.15.32:37440) 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a7d400640 1 -- 172.21.15.32:0/3131292987 learned_addr learned my addr 172.21.15.32:0/3131292987 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a7e800640 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.426+0000 7f9a7e800640 1 -- 172.21.15.32:0/3131292987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 msgr2=0x558dac195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a7e800640 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a7e800640 1 -- 172.21.15.32:0/3131292987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 msgr2=0x558dac195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a7e800640 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac195860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a7e800640 1 -- 172.21.15.32:0/3131292987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dac1a3cc0 con 0x558dabdeb070 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a7d400640 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:49.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a7de00640 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:49.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a7e800640 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac1962e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e560 tx=0x7f9a740095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50709624dd61aece server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:49.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a5b400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:20:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a5b400640 1 -- 172.21.15.32:0/3131292987 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7402a020 con 0x558dabdeb070 2024-08-31T07:20:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a5b400640 1 -- 172.21.15.32:0/3131292987 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f9a7400dd60 con 0x558dabdeb070 2024-08-31T07:20:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.427+0000 7f9a8741a740 1 -- 172.21.15.32:0/3131292987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558dac1051a0 con 0x558dabdeb070 2024-08-31T07:20:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a5b400640 1 -- 172.21.15.32:0/3131292987 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9a74035050 con 0x558dabdeb070 2024-08-31T07:20:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a8741a740 10 client.27217 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a8741a740 1 -- 172.21.15.32:0/3131292987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558dac2192f0 con 0x558dabdeb070 2024-08-31T07:20:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a5b400640 1 -- 172.21.15.32:0/3131292987 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558dac1051a0 con 0x558dabdeb070 2024-08-31T07:20:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a5b400640 1 -- 172.21.15.32:0/3131292987 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f9a7400b040 con 0x558dabdeb070 2024-08-31T07:20:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a8741a740 10 client.27217 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:49.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a8741a740 10 client.27217 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:20:49.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a8741a740 20 client.27217 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:49.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a8741a740 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558dabd7eda0 0x558dac21bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:49.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a8741a740 4 client.27217 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:20:49.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.428+0000 7f9a8741a740 1 -- 172.21.15.32:0/3131292987 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dac1028e0 con 0x558dabd7eda0 2024-08-31T07:20:49.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.429+0000 7f9a7de00640 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558dabd7eda0 0x558dac21bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:49.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.429+0000 7f9a7de00640 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558dabd7eda0 0x558dac21bb30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f0f2efe176fd84f9 server_cookie=4de49206facd2185 in_seq=0 out_seq=0 2024-08-31T07:20:49.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.429+0000 7f9a5b400640 10 client.27217 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:20:49.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a5b400640 1 -- 172.21.15.32:0/3131292987 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dac1028e0 con 0x558dabd7eda0 2024-08-31T07:20:49.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a5b400640 10 client.27217 handle_command_reply: tid=0 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 client.27217 shutdown 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 -- 172.21.15.32:0/3131292987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 msgr2=0x558dac1962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac1962e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e560 tx=0x7f9a740095c0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 -- 172.21.15.32:0/3131292987 shutdown_connections 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558dabd7eda0 0x558dac21bb30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dabdeb070 0x558dac1962e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558dabdca390 0x558dac195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 --2- 172.21.15.32:0/3131292987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558dabdc21b0 0x558dac195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 -- 172.21.15.32:0/3131292987 >> 172.21.15.32:0/3131292987 conn(0x558dabddf460 msgr2=0x558dac1976f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a5b400640 0 client.27217 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 -- 172.21.15.32:0/3131292987 shutdown_connections 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 1 -- 172.21.15.32:0/3131292987 wait complete. 2024-08-31T07:20:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.430+0000 7f9a8741a740 20 client.27217 trim_cache size 0 max 16384 2024-08-31T07:20:49.436 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:20:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 -- 172.21.15.32:0/1075477666 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a600b4ca0 msgr2=0x7f9a600b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a600b4ca0 0x7f9a600b71b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009fd0 tx=0x7f9a74010040 comp rx=0 tx=0).stop 2024-08-31T07:20:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 -- 172.21.15.32:0/1075477666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a78102b30 msgr2=0x7f9a7819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a78102b30 0x7f9a7819c810 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9a6802a8f0 tx=0x7f9a68030fa0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 -- 172.21.15.32:0/1075477666 shutdown_connections 2024-08-31T07:20:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9a600b4ca0 0x7f9a600b71b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a7810fd90 0x7f9a781a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a78102b30 0x7f9a7819c810 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 --2- 172.21.15.32:0/1075477666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a781021e0 0x7f9a7819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 -- 172.21.15.32:0/1075477666 >> 172.21.15.32:0/1075477666 conn(0x7f9a780fe060 msgr2=0x7f9a78111ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.431+0000 7f9a7f200640 1 -- 172.21.15.32:0/1075477666 shutdown_connections 2024-08-31T07:20:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:49.432+0000 7f9a7f200640 1 -- 172.21.15.32:0/1075477666 wait complete. 2024-08-31T07:20:49.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.60 MB/sec execute 278 sec latency 28842.553 ms 2024-08-31T07:20:50.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:20:50.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237420 2.59 MB/sec execute 279 sec latency 29842.623 ms 2024-08-31T07:20:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.821+0000 7f09c0600640 1 -- 172.21.15.32:0/2832872856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b8075550 msgr2=0x7f09b8075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.821+0000 7f09c0600640 1 --2- 172.21.15.32:0/2832872856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b8075550 0x7f09b8075960 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009900 tx=0x7f09ac02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.821+0000 7f09c0600640 1 -- 172.21.15.32:0/2832872856 shutdown_connections 2024-08-31T07:20:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.821+0000 7f09c0600640 1 --2- 172.21.15.32:0/2832872856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09b810fe80 0x7f09b81122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.821+0000 7f09c0600640 1 --2- 172.21.15.32:0/2832872856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09b8075ea0 0x7f09b810f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.821+0000 7f09c0600640 1 --2- 172.21.15.32:0/2832872856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b8075550 0x7f09b8075960 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.821+0000 7f09c0600640 1 -- 172.21.15.32:0/2832872856 >> 172.21.15.32:0/2832872856 conn(0x7f09b80ff3b0 msgr2=0x7f09b8101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.821+0000 7f09c0600640 1 -- 172.21.15.32:0/2832872856 shutdown_connections 2024-08-31T07:20:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.821+0000 7f09c0600640 1 -- 172.21.15.32:0/2832872856 wait complete. 2024-08-31T07:20:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.823+0000 7f09c0600640 1 Processor -- start 2024-08-31T07:20:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.823+0000 7f09c0600640 1 -- start start 2024-08-31T07:20:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.823+0000 7f09c0600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09b8075550 0x7f09b819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.823+0000 7f09c0600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09b8075ea0 0x7f09b819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.823+0000 7f09c0600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b810fe80 0x7f09b81a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.823+0000 7f09c0600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8115a80 con 0x7f09b810fe80 2024-08-31T07:20:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.823+0000 7f09c0600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8115900 con 0x7f09b8075550 2024-08-31T07:20:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.823+0000 7f09c0600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8115c00 con 0x7f09b8075ea0 2024-08-31T07:20:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.824+0000 7f09be600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b810fe80 0x7f09b81a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.824+0000 7f09bdc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09b8075550 0x7f09b819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.824+0000 7f09be600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b810fe80 0x7f09b81a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58854/0 (socket says 172.21.15.32:58854) 2024-08-31T07:20:50.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.824+0000 7f09be600640 1 -- 172.21.15.32:0/503452728 learned_addr learned my addr 172.21.15.32:0/503452728 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:50.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.824+0000 7f09bd200640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09b8075ea0 0x7f09b819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.824+0000 7f09bdc00640 1 -- 172.21.15.32:0/503452728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09b8075ea0 msgr2=0x7f09b819c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:50.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.824+0000 7f09bdc00640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09b8075ea0 0x7f09b819c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.824+0000 7f09bdc00640 1 -- 172.21.15.32:0/503452728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b810fe80 msgr2=0x7f09b81a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.824+0000 7f09bdc00640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b810fe80 0x7f09b81a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.824+0000 7f09bdc00640 1 -- 172.21.15.32:0/503452728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b81a2320 con 0x7f09b8075550 2024-08-31T07:20:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.825+0000 7f09bdc00640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09b8075550 0x7f09b819c3b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f09ac02a9e0 tx=0x7f09ac003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.825+0000 7f09b6000640 1 -- 172.21.15.32:0/503452728 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac03f070 con 0x7f09b8075550 2024-08-31T07:20:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.825+0000 7f09c0600640 1 -- 172.21.15.32:0/503452728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b81a25b0 con 0x7f09b8075550 2024-08-31T07:20:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.825+0000 7f09b6000640 1 -- 172.21.15.32:0/503452728 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09ac031b50 con 0x7f09b8075550 2024-08-31T07:20:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.825+0000 7f09b6000640 1 -- 172.21.15.32:0/503452728 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac035030 con 0x7f09b8075550 2024-08-31T07:20:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.825+0000 7f09c0600640 1 -- 172.21.15.32:0/503452728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b81a8d70 con 0x7f09b8075550 2024-08-31T07:20:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.828+0000 7f09c71e7740 1 Processor -- start 2024-08-31T07:20:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.829+0000 7f09c71e7740 1 -- start start 2024-08-31T07:20:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.829+0000 7f09b6000640 1 -- 172.21.15.32:0/503452728 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f09ac042070 con 0x7f09b8075550 2024-08-31T07:20:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.829+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 0x55d6ea401ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.829+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea033660 0x55d6ea402430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.829+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea402970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.829+0000 7f09c71e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea378a90 con 0x55d6ea05c520 2024-08-31T07:20:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.829+0000 7f09c71e7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea368020 con 0x55d6ea03b840 2024-08-31T07:20:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.829+0000 7f09c71e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea36bfd0 con 0x55d6ea033660 2024-08-31T07:20:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.830+0000 7f09be600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 0x55d6ea401ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.830+0000 7f09be600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 0x55d6ea401ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58856/0 (socket says 172.21.15.32:58856) 2024-08-31T07:20:50.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.830+0000 7f09be600640 1 -- 172.21.15.32:0/2033385063 learned_addr learned my addr 172.21.15.32:0/2033385063 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:50.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.830+0000 7f09bd200640 1 --2- 172.21.15.32:0/2033385063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea402970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.830+0000 7f09bdc00640 1 --2- 172.21.15.32:0/2033385063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea033660 0x55d6ea402430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.830+0000 7f09bd200640 1 -- 172.21.15.32:0/2033385063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea033660 msgr2=0x55d6ea402430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.830+0000 7f09bd200640 1 --2- 172.21.15.32:0/2033385063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea033660 0x55d6ea402430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.830+0000 7f09bd200640 1 -- 172.21.15.32:0/2033385063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 msgr2=0x55d6ea401ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.830+0000 7f09bd200640 1 --2- 172.21.15.32:0/2033385063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 0x55d6ea401ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.830+0000 7f09bd200640 1 -- 172.21.15.32:0/2033385063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ea0c7b00 con 0x55d6ea03b840 2024-08-31T07:20:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.831+0000 7f09bd200640 1 --2- 172.21.15.32:0/2033385063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea402970 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f09b806c710 tx=0x7f09a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293f2156ba4f941c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.831+0000 7f09aaa00640 1 -- 172.21.15.32:0/2033385063 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4009d70 con 0x55d6ea03b840 2024-08-31T07:20:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.831+0000 7f09aaa00640 1 -- 172.21.15.32:0/2033385063 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a4005020 con 0x55d6ea03b840 2024-08-31T07:20:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.831+0000 7f09aaa00640 1 -- 172.21.15.32:0/2033385063 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4004ad0 con 0x55d6ea03b840 2024-08-31T07:20:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.831+0000 7f09b6000640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f09a00b4ca0 0x7f09a00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.831+0000 7f09b6000640 1 -- 172.21.15.32:0/503452728 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f09ac105a60 con 0x7f09b8075550 2024-08-31T07:20:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.832+0000 7f09be600640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f09a00b4ca0 0x7f09a00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.832+0000 7f09c71e7740 1 -- 172.21.15.32:0/2033385063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 msgr2=0x55d6ea402970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.832+0000 7f09c71e7740 1 --2- 172.21.15.32:0/2033385063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea402970 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f09b806c710 tx=0x7f09a4011810 comp rx=0 tx=0).stop 2024-08-31T07:20:50.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.833+0000 7f09be600640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f09a00b4ca0 0x7f09a00b71b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f09b00116d0 tx=0x7f09b0009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:50.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.833+0000 7f09c71e7740 1 -- 172.21.15.32:0/2033385063 shutdown_connections 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.833+0000 7f09c71e7740 1 --2- 172.21.15.32:0/2033385063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea402970 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.833+0000 7f09c71e7740 1 --2- 172.21.15.32:0/2033385063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea033660 0x55d6ea402430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.833+0000 7f09c71e7740 1 --2- 172.21.15.32:0/2033385063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 0x55d6ea401ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.833+0000 7f09c71e7740 1 -- 172.21.15.32:0/2033385063 >> 172.21.15.32:0/2033385063 conn(0x55d6ea050910 msgr2=0x55d6ea3ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.833+0000 7f09c71e7740 1 -- 172.21.15.32:0/2033385063 shutdown_connections 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.833+0000 7f09c71e7740 1 -- 172.21.15.32:0/2033385063 wait complete. 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.834+0000 7f09c71e7740 1 -- start start 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.834+0000 7f09c71e7740 1 Processor -- start 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.834+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea033660 0x55d6ea40d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.834+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea40d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.834+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea05c520 0x55d6ea40dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.834+0000 7f09c71e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea1cabb0 con 0x55d6ea033660 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.834+0000 7f09c71e7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea2eef90 con 0x55d6ea03b840 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.834+0000 7f09c71e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea418ee0 con 0x55d6ea05c520 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09c71e7740 10 client.0 _ll_register_callbacks cb 0x55d6ea314710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09bd200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea40d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09bdc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea033660 0x55d6ea40d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09bd200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea40d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37474/0 (socket says 172.21.15.32:37474) 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09bd200640 1 -- 172.21.15.32:0/2354089683 learned_addr learned my addr 172.21.15.32:0/2354089683 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09be600640 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea05c520 0x55d6ea40dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09bd200640 1 -- 172.21.15.32:0/2354089683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea05c520 msgr2=0x55d6ea40dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09bd200640 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea05c520 0x55d6ea40dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09bd200640 1 -- 172.21.15.32:0/2354089683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea033660 msgr2=0x55d6ea40d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09bd200640 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea033660 0x55d6ea40d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09bd200640 1 -- 172.21.15.32:0/2354089683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ea13e960 con 0x55d6ea03b840 2024-08-31T07:20:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.835+0000 7f09bdc00640 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea033660 0x55d6ea40d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:50.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.836+0000 7f09bd200640 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea40d830 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009580 tx=0x7f09a4019950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3914e0e02f1ca03f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:50.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.836+0000 7f099f400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.836+0000 7f099f400640 1 -- 172.21.15.32:0/2354089683 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4029020 con 0x55d6ea03b840 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.836+0000 7f09c71e7740 1 -- 172.21.15.32:0/2354089683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d6ea306e30 con 0x55d6ea03b840 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.836+0000 7f099f400640 1 -- 172.21.15.32:0/2354089683 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f09a4019d50 con 0x55d6ea03b840 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.836+0000 7f099f400640 1 -- 172.21.15.32:0/2354089683 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f09a4034050 con 0x55d6ea03b840 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.836+0000 7f09c71e7740 10 client.27227 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.836+0000 7f09c71e7740 1 -- 172.21.15.32:0/2354089683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d6ea424900 con 0x55d6ea03b840 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.837+0000 7f099f400640 1 -- 172.21.15.32:0/2354089683 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d6ea306e30 con 0x55d6ea03b840 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.837+0000 7f099f400640 1 -- 172.21.15.32:0/2354089683 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f09a4023420 con 0x55d6ea03b840 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.837+0000 7f09c71e7740 10 client.27227 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.837+0000 7f09c71e7740 10 client.27227 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.837+0000 7f09c71e7740 20 client.27227 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.837+0000 7f09c71e7740 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55d6ea01aea0 0x55d6ea427d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.837+0000 7f09c71e7740 4 client.27227 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.837+0000 7f09c71e7740 1 -- 172.21.15.32:0/2354089683 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6ea37a9e0 con 0x55d6ea01aea0 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.837+0000 7f09bdc00640 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55d6ea01aea0 0x55d6ea427d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.838+0000 7f09bdc00640 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55d6ea01aea0 0x55d6ea427d20 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=274e8c108e64ef6 server_cookie=abd79331fca5dfe9 in_seq=0 out_seq=0 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.838+0000 7f099f400640 10 client.27227 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f099f400640 1 -- 172.21.15.32:0/2354089683 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f09ac0acf90 con 0x55d6ea01aea0 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f099f400640 10 client.27227 handle_command_reply: tid=0 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f09c71e7740 1 client.27227 shutdown 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f09c71e7740 1 -- 172.21.15.32:0/2354089683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 msgr2=0x55d6ea40d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f09c71e7740 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea40d830 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009580 tx=0x7f09a4019950 comp rx=0 tx=0).stop 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f09c71e7740 1 -- 172.21.15.32:0/2354089683 shutdown_connections 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f09c71e7740 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55d6ea01aea0 0x55d6ea427d20 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f09c71e7740 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea05c520 0x55d6ea40dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f09c71e7740 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea03b840 0x55d6ea40d830 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f09c71e7740 1 --2- 172.21.15.32:0/2354089683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea033660 0x55d6ea40d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f09c71e7740 1 -- 172.21.15.32:0/2354089683 >> 172.21.15.32:0/2354089683 conn(0x55d6ea050910 msgr2=0x55d6ea4050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:50.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f099f400640 0 client.27227 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:50.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.839+0000 7f09c71e7740 1 -- 172.21.15.32:0/2354089683 shutdown_connections 2024-08-31T07:20:50.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.840+0000 7f09c71e7740 1 -- 172.21.15.32:0/2354089683 wait complete. 2024-08-31T07:20:50.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.840+0000 7f09c71e7740 20 client.27227 trim_cache size 0 max 16384 2024-08-31T07:20:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.860+0000 7f09c71e7740 1 Processor -- start 2024-08-31T07:20:50.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.860+0000 7f09c71e7740 1 -- start start 2024-08-31T07:20:50.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.860+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 0x55d6ea40a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.860+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea03b840 0x55d6ea40aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.860+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 0x55d6ea42d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.860+0000 7f09c71e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09b81ad050 con 0x55d6ea05c520 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.860+0000 7f09c71e7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f09b81aec50 con 0x55d6ea033660 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.860+0000 7f09c71e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f09b81ad720 con 0x55d6ea03b840 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09be600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 0x55d6ea42d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09bd200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea03b840 0x55d6ea40aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09be600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 0x55d6ea42d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58870/0 (socket says 172.21.15.32:58870) 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09be600640 1 -- 172.21.15.32:0/1762534590 learned_addr learned my addr 172.21.15.32:0/1762534590 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09bdc00640 1 --2- 172.21.15.32:0/1762534590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 0x55d6ea40a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09bd200640 1 -- 172.21.15.32:0/1762534590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 msgr2=0x55d6ea40a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09bd200640 1 --2- 172.21.15.32:0/1762534590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 0x55d6ea40a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09bd200640 1 -- 172.21.15.32:0/1762534590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 msgr2=0x55d6ea42d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09bd200640 1 --2- 172.21.15.32:0/1762534590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 0x55d6ea42d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09bd200640 1 -- 172.21.15.32:0/1762534590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ea40e5b0 con 0x55d6ea03b840 2024-08-31T07:20:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:50.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.861+0000 7f09bd200640 1 --2- 172.21.15.32:0/1762534590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea03b840 0x55d6ea40aa20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f09a40239e0 tx=0x7f09a4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f016b81c86cb9625 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:50.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.862+0000 7f09aa000640 1 -- 172.21.15.32:0/1762534590 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4029070 con 0x55d6ea03b840 2024-08-31T07:20:50.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.862+0000 7f09aa000640 1 -- 172.21.15.32:0/1762534590 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a4023a60 con 0x55d6ea03b840 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.862+0000 7f09aa000640 1 -- 172.21.15.32:0/1762534590 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4025c80 con 0x55d6ea03b840 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 -- 172.21.15.32:0/1762534590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea03b840 msgr2=0x55d6ea40aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 --2- 172.21.15.32:0/1762534590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea03b840 0x55d6ea40aa20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f09a40239e0 tx=0x7f09a4005fd0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 -- 172.21.15.32:0/1762534590 shutdown_connections 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 --2- 172.21.15.32:0/1762534590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea05c520 0x55d6ea42d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 --2- 172.21.15.32:0/1762534590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea03b840 0x55d6ea40aa20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 --2- 172.21.15.32:0/1762534590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 0x55d6ea40a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 -- 172.21.15.32:0/1762534590 >> 172.21.15.32:0/1762534590 conn(0x55d6ea050910 msgr2=0x55d6ea40da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 -- 172.21.15.32:0/1762534590 shutdown_connections 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 -- 172.21.15.32:0/1762534590 wait complete. 2024-08-31T07:20:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 -- start start 2024-08-31T07:20:50.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.863+0000 7f09c71e7740 1 Processor -- start 2024-08-31T07:20:50.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 0x55d6ea41c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea03b840 0x55d6ea41c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09c71e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea05c520 0x55d6ea41cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09c71e7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09b81ae4c0 con 0x55d6ea03b840 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09c71e7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f09b81addf0 con 0x55d6ea033660 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09c71e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f09b81ab110 con 0x55d6ea05c520 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09c71e7740 10 client.0 _ll_register_callbacks cb 0x7f09a40197f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09bd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea03b840 0x55d6ea41c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09bdc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 0x55d6ea41c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09bd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea03b840 0x55d6ea41c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58876/0 (socket says 172.21.15.32:58876) 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.864+0000 7f09bd200640 1 -- 172.21.15.32:0/1004170955 learned_addr learned my addr 172.21.15.32:0/1004170955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.865+0000 7f09be600640 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea05c520 0x55d6ea41cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.865+0000 7f09bdc00640 1 -- 172.21.15.32:0/1004170955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea05c520 msgr2=0x55d6ea41cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.865+0000 7f09bdc00640 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea05c520 0x55d6ea41cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.865+0000 7f09bdc00640 1 -- 172.21.15.32:0/1004170955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea03b840 msgr2=0x55d6ea41c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.865+0000 7f09bdc00640 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea03b840 0x55d6ea41c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.865+0000 7f09bdc00640 1 -- 172.21.15.32:0/1004170955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ea491850 con 0x55d6ea033660 2024-08-31T07:20:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.865+0000 7f09bdc00640 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 0x55d6ea41c2b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f09ac1031a0 tx=0x7f09ac02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaaf46fe88017d56 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.865+0000 7f099f400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.865+0000 7f099f400640 1 -- 172.21.15.32:0/1004170955 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac049040 con 0x55d6ea033660 2024-08-31T07:20:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f099f400640 1 -- 172.21.15.32:0/1004170955 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f09ac103420 con 0x55d6ea033660 2024-08-31T07:20:50.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f09c71e7740 1 -- 172.21.15.32:0/1004170955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a4023a60 con 0x55d6ea033660 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f099f400640 1 -- 172.21.15.32:0/1004170955 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f09ac103620 con 0x55d6ea033660 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f09c71e7740 10 client.27233 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f09c71e7740 1 -- 172.21.15.32:0/1004170955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d6ea492b70 con 0x55d6ea033660 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f099f400640 1 -- 172.21.15.32:0/1004170955 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a4023a60 con 0x55d6ea033660 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f099f400640 1 -- 172.21.15.32:0/1004170955 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f09ac0d4fd0 con 0x55d6ea033660 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f09c71e7740 10 client.27233 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f09c71e7740 10 client.27233 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f09c71e7740 20 client.27233 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f09c71e7740 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55d6e9ff0250 0x55d6ea4953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f09c71e7740 4 client.27233 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d2dfb577", "release": true, "await": true}] 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.866+0000 7f09c71e7740 1 -- 172.21.15.32:0/1004170955 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d2dfb577", "release": true, "await": true}) -- 0x55d6ea376480 con 0x55d6e9ff0250 2024-08-31T07:20:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.867+0000 7f09bd200640 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55d6e9ff0250 0x55d6ea4953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:50.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.867+0000 7f09bd200640 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55d6e9ff0250 0x55d6ea4953a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7eaa23f1adce33c server_cookie=be16556b602d83bf in_seq=0 out_seq=0 2024-08-31T07:20:50.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.867+0000 7f099f400640 10 client.27233 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:50.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.874+0000 7f099f400640 1 -- 172.21.15.32:0/1004170955 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d6ea376480 con 0x55d6e9ff0250 2024-08-31T07:20:50.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.874+0000 7f099f400640 10 client.27233 handle_command_reply: tid=0 2024-08-31T07:20:50.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.874+0000 7f09c71e7740 1 client.27233 shutdown 2024-08-31T07:20:50.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f09c71e7740 1 -- 172.21.15.32:0/1004170955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 msgr2=0x55d6ea41c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:50.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f09c71e7740 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 0x55d6ea41c2b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f09ac1031a0 tx=0x7f09ac02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f09c71e7740 1 -- 172.21.15.32:0/1004170955 shutdown_connections 2024-08-31T07:20:50.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f09c71e7740 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55d6e9ff0250 0x55d6ea4953a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f09c71e7740 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d6ea05c520 0x55d6ea41cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f09c71e7740 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6ea03b840 0x55d6ea41c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f099f400640 0 client.27233 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:50.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f09c71e7740 1 --2- 172.21.15.32:0/1004170955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d6ea033660 0x55d6ea41c2b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f09c71e7740 1 -- 172.21.15.32:0/1004170955 >> 172.21.15.32:0/1004170955 conn(0x55d6ea050910 msgr2=0x55d6ea40bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:50.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f09c71e7740 1 -- 172.21.15.32:0/1004170955 shutdown_connections 2024-08-31T07:20:50.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.875+0000 7f09c71e7740 1 -- 172.21.15.32:0/1004170955 wait complete. 2024-08-31T07:20:50.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.876+0000 7f09c71e7740 20 client.27233 trim_cache size 0 max 16384 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 35, 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 28, 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "d2dfb577": { 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 28, 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:20:50.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 148.0, 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:20:50.879 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:20:50.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.876+0000 7f09c0600640 1 -- 172.21.15.32:0/503452728 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f09a00b4ca0 msgr2=0x7f09a00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:50.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.876+0000 7f09c0600640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f09a00b4ca0 0x7f09a00b71b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f09b00116d0 tx=0x7f09b0009500 comp rx=0 tx=0).stop 2024-08-31T07:20:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.876+0000 7f09c0600640 1 -- 172.21.15.32:0/503452728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09b8075550 msgr2=0x7f09b819c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.876+0000 7f09c0600640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09b8075550 0x7f09b819c3b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f09ac02a9e0 tx=0x7f09ac003b90 comp rx=0 tx=0).stop 2024-08-31T07:20:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.877+0000 7f09c0600640 1 -- 172.21.15.32:0/503452728 shutdown_connections 2024-08-31T07:20:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.877+0000 7f09c0600640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f09a00b4ca0 0x7f09a00b71b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.877+0000 7f09c0600640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b810fe80 0x7f09b81a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.877+0000 7f09c0600640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09b8075ea0 0x7f09b819c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.877+0000 7f09c0600640 1 --2- 172.21.15.32:0/503452728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09b8075550 0x7f09b819c3b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.877+0000 7f09c0600640 1 -- 172.21.15.32:0/503452728 >> 172.21.15.32:0/503452728 conn(0x7f09b80ff3b0 msgr2=0x7f09b81078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.877+0000 7f09c0600640 1 -- 172.21.15.32:0/503452728 shutdown_connections 2024-08-31T07:20:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:50.877+0000 7f09c0600640 1 -- 172.21.15.32:0/503452728 wait complete. 2024-08-31T07:20:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:50 smithi073 ceph-mon[21995]: pgmap v614: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-08-31T07:20:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:50 smithi084 ceph-mon[22387]: pgmap v614: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-08-31T07:20:51.009 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:50 smithi032 ceph-mon[17252]: pgmap v614: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-08-31T07:20:51.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 238299 2.59 MB/sec execute 280 sec latency 29972.838 ms 2024-08-31T07:20:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.876+0000 7efbfca00640 1 -- 172.21.15.32:0/1792576451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efbf8109090 msgr2=0x7efbf81094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.876+0000 7efbfca00640 1 --2- 172.21.15.32:0/1792576451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efbf8109090 0x7efbf81094a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009900 tx=0x7efbe402a410 comp rx=0 tx=0).stop 2024-08-31T07:20:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.876+0000 7efbfca00640 1 -- 172.21.15.32:0/1792576451 shutdown_connections 2024-08-31T07:20:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.876+0000 7efbfca00640 1 --2- 172.21.15.32:0/1792576451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbf81099e0 0x7efbf8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.876+0000 7efbfca00640 1 --2- 172.21.15.32:0/1792576451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efbf8109090 0x7efbf81094a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.876+0000 7efbfca00640 1 --2- 172.21.15.32:0/1792576451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8103750 0x7efbf8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.876+0000 7efbfca00640 1 -- 172.21.15.32:0/1792576451 >> 172.21.15.32:0/1792576451 conn(0x7efbf80ff1e0 msgr2=0x7efbf8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:51.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.876+0000 7efbfca00640 1 -- 172.21.15.32:0/1792576451 shutdown_connections 2024-08-31T07:20:51.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.876+0000 7efbfca00640 1 -- 172.21.15.32:0/1792576451 wait complete. 2024-08-31T07:20:51.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.877+0000 7efbfca00640 1 Processor -- start 2024-08-31T07:20:51.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbfca00640 1 -- start start 2024-08-31T07:20:51.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbfca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8103750 0x7efbf80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbfca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efbf8109090 0x7efbf80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbfca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbf81099e0 0x7efbf8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbfca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81136a0 con 0x7efbf8103750 2024-08-31T07:20:51.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbfca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8113520 con 0x7efbf81099e0 2024-08-31T07:20:51.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbfca00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8113820 con 0x7efbf8109090 2024-08-31T07:20:51.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbf6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8103750 0x7efbf80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbf5e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efbf8109090 0x7efbf80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbf5e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efbf8109090 0x7efbf80759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60546/0 (socket says 172.21.15.32:60546) 2024-08-31T07:20:51.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.878+0000 7efbf5e00640 1 -- 172.21.15.32:0/1888474522 learned_addr learned my addr 172.21.15.32:0/1888474522 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:51.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.879+0000 7efbf7200640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbf81099e0 0x7efbf8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.879+0000 7efbf5e00640 1 -- 172.21.15.32:0/1888474522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbf81099e0 msgr2=0x7efbf8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.879+0000 7efbf5e00640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbf81099e0 0x7efbf8075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.879+0000 7efbf5e00640 1 -- 172.21.15.32:0/1888474522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8103750 msgr2=0x7efbf80754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.879+0000 7efbf5e00640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8103750 0x7efbf80754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.879+0000 7efbf5e00640 1 -- 172.21.15.32:0/1888474522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf81a8370 con 0x7efbf8109090 2024-08-31T07:20:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.879+0000 7efbf5e00640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efbf8109090 0x7efbf80759f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efbe4005ab0 tx=0x7efbe4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.880+0000 7efbef400640 1 -- 172.21.15.32:0/1888474522 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe403f070 con 0x7efbf8109090 2024-08-31T07:20:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.880+0000 7efbef400640 1 -- 172.21.15.32:0/1888474522 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe4031b50 con 0x7efbf8109090 2024-08-31T07:20:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.880+0000 7efbfca00640 1 -- 172.21.15.32:0/1888474522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf81a8600 con 0x7efbf8109090 2024-08-31T07:20:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.880+0000 7efbef400640 1 -- 172.21.15.32:0/1888474522 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe4035030 con 0x7efbf8109090 2024-08-31T07:20:51.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.880+0000 7efbfca00640 1 -- 172.21.15.32:0/1888474522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf81a8b30 con 0x7efbf8109090 2024-08-31T07:20:51.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.883+0000 7efc03fc4740 1 Processor -- start 2024-08-31T07:20:51.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.883+0000 7efc03fc4740 1 -- start start 2024-08-31T07:20:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e07b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efc03fc4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c325d7e7b0 con 0x55c325a41390 2024-08-31T07:20:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efc03fc4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c325d6dd40 con 0x55c325a62070 2024-08-31T07:20:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efc03fc4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c325d71cf0 con 0x55c325a391b0 2024-08-31T07:20:51.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efbf6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efbf6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60562/0 (socket says 172.21.15.32:60562) 2024-08-31T07:20:51.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efbf6800640 1 -- 172.21.15.32:0/2848143459 learned_addr learned my addr 172.21.15.32:0/2848143459 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:51.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efbf7200640 1 --2- 172.21.15.32:0/2848143459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e07b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.884+0000 7efbef400640 1 -- 172.21.15.32:0/1888474522 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7efbe4042070 con 0x7efbf8109090 2024-08-31T07:20:51.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.885+0000 7efbf6800640 1 -- 172.21.15.32:0/2848143459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 msgr2=0x55c325e07b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.885+0000 7efbf6800640 1 --2- 172.21.15.32:0/2848143459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e07b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.885+0000 7efbf6800640 1 -- 172.21.15.32:0/2848143459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 msgr2=0x55c325e085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.885+0000 7efbf6800640 1 --2- 172.21.15.32:0/2848143459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.885+0000 7efbf6800640 1 -- 172.21.15.32:0/2848143459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c325acd650 con 0x55c325a391b0 2024-08-31T07:20:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.885+0000 7efbf7200640 1 --2- 172.21.15.32:0/2848143459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e07b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.885+0000 7efbf6800640 1 --2- 172.21.15.32:0/2848143459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e080a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009c90 tx=0x7efbe8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e262915ae0e30e02 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.886+0000 7efbe3400640 1 -- 172.21.15.32:0/2848143459 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8004230 con 0x55c325a391b0 2024-08-31T07:20:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.886+0000 7efbe3400640 1 -- 172.21.15.32:0/2848143459 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe80043d0 con 0x55c325a391b0 2024-08-31T07:20:51.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.886+0000 7efbe3400640 1 -- 172.21.15.32:0/2848143459 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8019750 con 0x55c325a391b0 2024-08-31T07:20:51.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.886+0000 7efbef400640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbdc0b4d20 0x7efbdc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.887+0000 7efbef400640 1 -- 172.21.15.32:0/1888474522 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7efbe4105a80 con 0x7efbf8109090 2024-08-31T07:20:51.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.887+0000 7efc03fc4740 1 -- 172.21.15.32:0/2848143459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 msgr2=0x55c325e080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:51.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.887+0000 7efc03fc4740 1 --2- 172.21.15.32:0/2848143459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e080a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009c90 tx=0x7efbe8011810 comp rx=0 tx=0).stop 2024-08-31T07:20:51.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.887+0000 7efbf7200640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbdc0b4d20 0x7efbdc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.887+0000 7efc03fc4740 1 -- 172.21.15.32:0/2848143459 shutdown_connections 2024-08-31T07:20:51.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.887+0000 7efc03fc4740 1 --2- 172.21.15.32:0/2848143459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.887+0000 7efc03fc4740 1 --2- 172.21.15.32:0/2848143459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e080a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.887+0000 7efc03fc4740 1 --2- 172.21.15.32:0/2848143459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e07b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.887+0000 7efc03fc4740 1 -- 172.21.15.32:0/2848143459 >> 172.21.15.32:0/2848143459 conn(0x55c325a56460 msgr2=0x55c325e05a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:51.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.888+0000 7efc03fc4740 1 -- 172.21.15.32:0/2848143459 shutdown_connections 2024-08-31T07:20:51.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.888+0000 7efc03fc4740 1 -- 172.21.15.32:0/2848143459 wait complete. 2024-08-31T07:20:51.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.888+0000 7efbf7200640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbdc0b4d20 0x7efbdc0b7230 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009fd0 tx=0x7efbf0009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:51.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.889+0000 7efc03fc4740 1 -- start start 2024-08-31T07:20:51.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.889+0000 7efc03fc4740 1 Processor -- start 2024-08-31T07:20:51.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.889+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e13770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e13cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e1f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efc03fc4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c325bd0700 con 0x55c325a41390 2024-08-31T07:20:51.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efc03fc4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c325cf4cf0 con 0x55c325a62070 2024-08-31T07:20:51.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efc03fc4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c325e16340 con 0x55c325a391b0 2024-08-31T07:20:51.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efc03fc4740 10 client.0 _ll_register_callbacks cb 0x55c325d1a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:51.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efbf7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e1f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efbf7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e1f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37524/0 (socket says 172.21.15.32:37524) 2024-08-31T07:20:51.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efbf6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e13770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efbf7200640 1 -- 172.21.15.32:0/1468262201 learned_addr learned my addr 172.21.15.32:0/1468262201 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:51.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efbf5e00640 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e13cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efbf7200640 1 -- 172.21.15.32:0/1468262201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 msgr2=0x55c325e13770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:51.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efbf7200640 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e13770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efbf7200640 1 -- 172.21.15.32:0/1468262201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 msgr2=0x55c325e13cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:51.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.890+0000 7efbf7200640 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e13cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.891+0000 7efbf7200640 1 -- 172.21.15.32:0/1468262201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c325d358b0 con 0x55c325a62070 2024-08-31T07:20:51.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.891+0000 7efbf7200640 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e1f690 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efbf0005f40 tx=0x7efbf0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1c168769a791d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:51.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.891+0000 7efbd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.892+0000 7efbd7e00640 1 -- 172.21.15.32:0/1468262201 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0028cf0 con 0x55c325a62070 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.892+0000 7efc03fc4740 1 -- 172.21.15.32:0/1468262201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbe80043d0 con 0x55c325a62070 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.892+0000 7efbd7e00640 1 -- 172.21.15.32:0/1468262201 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7efbf001d520 con 0x55c325a62070 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.892+0000 7efbd7e00640 1 -- 172.21.15.32:0/1468262201 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efbf001d720 con 0x55c325a62070 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.892+0000 7efc03fc4740 10 client.27245 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.892+0000 7efc03fc4740 1 -- 172.21.15.32:0/1468262201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c325e224b0 con 0x55c325a62070 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.892+0000 7efbd7e00640 1 -- 172.21.15.32:0/1468262201 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe80043d0 con 0x55c325a62070 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.893+0000 7efbd7e00640 1 -- 172.21.15.32:0/1468262201 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7efbf003d990 con 0x55c325a62070 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.893+0000 7efc03fc4740 10 client.27245 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.893+0000 7efc03fc4740 10 client.27245 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.893+0000 7efc03fc4740 20 client.27245 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:51.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.893+0000 7efc03fc4740 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55c325a209f0 0x55c325e25920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.893+0000 7efc03fc4740 4 client.27245 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:51.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.893+0000 7efc03fc4740 1 -- 172.21.15.32:0/1468262201 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c325d80700 con 0x55c325a209f0 2024-08-31T07:20:51.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.893+0000 7efbf6800640 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55c325a209f0 0x55c325e25920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.894+0000 7efbf6800640 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55c325a209f0 0x55c325e25920 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7977ecee7aea4d48 server_cookie=287bddc9164ca3ed in_seq=0 out_seq=0 2024-08-31T07:20:51.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.894+0000 7efbd7e00640 10 client.27245 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:20:51.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efbd7e00640 1 -- 172.21.15.32:0/1468262201 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c325d80700 con 0x55c325a209f0 2024-08-31T07:20:51.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efbd7e00640 10 client.27245 handle_command_reply: tid=0 2024-08-31T07:20:51.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efc03fc4740 1 client.27245 shutdown 2024-08-31T07:20:51.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efc03fc4740 1 -- 172.21.15.32:0/1468262201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 msgr2=0x55c325e1f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:51.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efc03fc4740 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e1f690 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efbf0005f40 tx=0x7efbf0004a40 comp rx=0 tx=0).stop 2024-08-31T07:20:51.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efc03fc4740 1 -- 172.21.15.32:0/1468262201 shutdown_connections 2024-08-31T07:20:51.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efc03fc4740 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55c325a209f0 0x55c325e25920 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efc03fc4740 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e1f690 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efbd7e00640 0 client.27245 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:20:51.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efc03fc4740 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e13cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efc03fc4740 1 --2- 172.21.15.32:0/1468262201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e13770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.895+0000 7efc03fc4740 1 -- 172.21.15.32:0/1468262201 >> 172.21.15.32:0/1468262201 conn(0x55c325a56460 msgr2=0x55c325e0acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:51.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.896+0000 7efc03fc4740 1 -- 172.21.15.32:0/1468262201 shutdown_connections 2024-08-31T07:20:51.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.896+0000 7efc03fc4740 1 -- 172.21.15.32:0/1468262201 wait complete. 2024-08-31T07:20:51.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.896+0000 7efc03fc4740 20 client.27245 trim_cache size 0 max 16384 2024-08-31T07:20:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.911+0000 7efc03fc4740 1 Processor -- start 2024-08-31T07:20:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.911+0000 7efc03fc4740 1 -- start start 2024-08-31T07:20:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.911+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e0cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.911+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e0d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.911+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e05000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.911+0000 7efc03fc4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81acfd0 con 0x55c325a41390 2024-08-31T07:20:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.911+0000 7efc03fc4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81aebd0 con 0x55c325a62070 2024-08-31T07:20:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.911+0000 7efc03fc4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81ad6a0 con 0x55c325a391b0 2024-08-31T07:20:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e05000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e0d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e0cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e05000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37540/0 (socket says 172.21.15.32:37540) 2024-08-31T07:20:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf7200640 1 -- 172.21.15.32:0/4122656873 learned_addr learned my addr 172.21.15.32:0/4122656873 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf7200640 1 -- 172.21.15.32:0/4122656873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 msgr2=0x55c325e0cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf7200640 1 --2- 172.21.15.32:0/4122656873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e0cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf7200640 1 -- 172.21.15.32:0/4122656873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 msgr2=0x55c325e0d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf7200640 1 --2- 172.21.15.32:0/4122656873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e0d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf7200640 1 -- 172.21.15.32:0/4122656873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c325e0a080 con 0x55c325a62070 2024-08-31T07:20:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.912+0000 7efbf7200640 1 --2- 172.21.15.32:0/4122656873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e05000 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efbf0004a10 tx=0x7efbf0005c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0431c86ab02ca8d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.913+0000 7efbe2a00640 1 -- 172.21.15.32:0/4122656873 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0004ab0 con 0x55c325a62070 2024-08-31T07:20:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.913+0000 7efbe2a00640 1 -- 172.21.15.32:0/4122656873 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf001dd70 con 0x55c325a62070 2024-08-31T07:20:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.913+0000 7efbe2a00640 1 -- 172.21.15.32:0/4122656873 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf003d970 con 0x55c325a62070 2024-08-31T07:20:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.914+0000 7efc03fc4740 1 -- 172.21.15.32:0/4122656873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 msgr2=0x55c325e05000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.914+0000 7efc03fc4740 1 --2- 172.21.15.32:0/4122656873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e05000 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efbf0004a10 tx=0x7efbf0005c10 comp rx=0 tx=0).stop 2024-08-31T07:20:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.914+0000 7efc03fc4740 1 -- 172.21.15.32:0/4122656873 shutdown_connections 2024-08-31T07:20:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.914+0000 7efc03fc4740 1 --2- 172.21.15.32:0/4122656873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a62070 0x55c325e05000 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.914+0000 7efc03fc4740 1 --2- 172.21.15.32:0/4122656873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e0d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.914+0000 7efc03fc4740 1 --2- 172.21.15.32:0/4122656873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a391b0 0x55c325e0cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.914+0000 7efc03fc4740 1 -- 172.21.15.32:0/4122656873 >> 172.21.15.32:0/4122656873 conn(0x55c325a56460 msgr2=0x55c325e11df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.914+0000 7efc03fc4740 1 -- 172.21.15.32:0/4122656873 shutdown_connections 2024-08-31T07:20:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.915+0000 7efc03fc4740 1 -- 172.21.15.32:0/4122656873 wait complete. 2024-08-31T07:20:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.915+0000 7efc03fc4740 1 -- start start 2024-08-31T07:20:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.915+0000 7efc03fc4740 1 Processor -- start 2024-08-31T07:20:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.916+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a391b0 0x55c325e18100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.916+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.916+0000 7efc03fc4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a62070 0x55c325e89390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.916+0000 7efc03fc4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81ae440 con 0x55c325a41390 2024-08-31T07:20:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.916+0000 7efc03fc4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81add70 con 0x55c325a391b0 2024-08-31T07:20:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.916+0000 7efc03fc4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81aaff0 con 0x55c325a62070 2024-08-31T07:20:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.916+0000 7efc03fc4740 10 client.0 _ll_register_callbacks cb 0x7efbf000f9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:51.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.916+0000 7efbf5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.916+0000 7efbf5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e186e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58932/0 (socket says 172.21.15.32:58932) 2024-08-31T07:20:51.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.916+0000 7efbf5e00640 1 -- 172.21.15.32:0/3293157428 learned_addr learned my addr 172.21.15.32:0/3293157428 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.917+0000 7efbf7200640 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a62070 0x55c325e89390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.917+0000 7efbf7200640 1 -- 172.21.15.32:0/3293157428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a391b0 msgr2=0x55c325e18100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.917+0000 7efbf7200640 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a391b0 0x55c325e18100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.917+0000 7efbf7200640 1 -- 172.21.15.32:0/3293157428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 msgr2=0x55c325e186e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.917+0000 7efbf7200640 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e186e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.917+0000 7efbf7200640 1 -- 172.21.15.32:0/3293157428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c325e89a00 con 0x55c325a62070 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.917+0000 7efbf5e00640 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.917+0000 7efbf7200640 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a62070 0x55c325e89390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efbf0011f80 tx=0x7efbf0004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48119fa7d9cc2d70 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.917+0000 7efbd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.918+0000 7efbd7e00640 1 -- 172.21.15.32:0/3293157428 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf001dad0 con 0x55c325a62070 2024-08-31T07:20:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.918+0000 7efc03fc4740 1 -- 172.21.15.32:0/3293157428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf001dd70 con 0x55c325a62070 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.918+0000 7efbd7e00640 1 -- 172.21.15.32:0/3293157428 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7efbf003d570 con 0x55c325a62070 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.918+0000 7efbd7e00640 1 -- 172.21.15.32:0/3293157428 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efbf003d790 con 0x55c325a62070 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.918+0000 7efc03fc4740 10 client.27253 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.918+0000 7efc03fc4740 1 -- 172.21.15.32:0/3293157428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c325e90df0 con 0x55c325a62070 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.918+0000 7efbd7e00640 1 -- 172.21.15.32:0/3293157428 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf001dd70 con 0x55c325a62070 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.919+0000 7efbd7e00640 1 -- 172.21.15.32:0/3293157428 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7efbf003cbe0 con 0x55c325a62070 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.919+0000 7efc03fc4740 10 client.27253 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.919+0000 7efc03fc4740 10 client.27253 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.919+0000 7efc03fc4740 20 client.27253 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.919+0000 7efc03fc4740 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55c3259f5da0 0x55c325e93420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.919+0000 7efc03fc4740 4 client.27253 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.919+0000 7efc03fc4740 1 -- 172.21.15.32:0/3293157428 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c325d7c1a0 con 0x55c3259f5da0 2024-08-31T07:20:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.919+0000 7efbf6800640 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55c3259f5da0 0x55c325e93420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efbd7e00640 10 client.27253 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:20:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efbf6800640 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55c3259f5da0 0x55c325e93420 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=785c7d561ee14236 server_cookie=4bad8dda2fc2e33d in_seq=0 out_seq=0 2024-08-31T07:20:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efbd7e00640 1 -- 172.21.15.32:0/3293157428 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c325d7c1a0 con 0x55c3259f5da0 2024-08-31T07:20:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efbd7e00640 10 client.27253 handle_command_reply: tid=0 2024-08-31T07:20:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efc03fc4740 1 client.27253 shutdown 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efc03fc4740 1 -- 172.21.15.32:0/3293157428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a62070 msgr2=0x55c325e89390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efc03fc4740 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a62070 0x55c325e89390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efbf0011f80 tx=0x7efbf0004c70 comp rx=0 tx=0).stop 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efc03fc4740 1 -- 172.21.15.32:0/3293157428 shutdown_connections 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efc03fc4740 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55c3259f5da0 0x55c325e93420 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efc03fc4740 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c325a62070 0x55c325e89390 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efc03fc4740 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c325a41390 0x55c325e186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efc03fc4740 1 --2- 172.21.15.32:0/3293157428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c325a391b0 0x55c325e18100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efc03fc4740 1 -- 172.21.15.32:0/3293157428 >> 172.21.15.32:0/3293157428 conn(0x55c325a56460 msgr2=0x55c325e0cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efbd7e00640 0 client.27253 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.920+0000 7efc03fc4740 1 -- 172.21.15.32:0/3293157428 shutdown_connections 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.921+0000 7efc03fc4740 1 -- 172.21.15.32:0/3293157428 wait complete. 2024-08-31T07:20:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.921+0000 7efc03fc4740 20 client.27253 trim_cache size 0 max 16384 2024-08-31T07:20:51.924 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:20:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.921+0000 7efbfca00640 1 -- 172.21.15.32:0/1888474522 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbdc0b4d20 msgr2=0x7efbdc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.921+0000 7efbfca00640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbdc0b4d20 0x7efbdc0b7230 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009fd0 tx=0x7efbf0009500 comp rx=0 tx=0).stop 2024-08-31T07:20:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.921+0000 7efbfca00640 1 -- 172.21.15.32:0/1888474522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efbf8109090 msgr2=0x7efbf80759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.921+0000 7efbfca00640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efbf8109090 0x7efbf80759f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efbe4005ab0 tx=0x7efbe4003b90 comp rx=0 tx=0).stop 2024-08-31T07:20:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.922+0000 7efbfca00640 1 -- 172.21.15.32:0/1888474522 shutdown_connections 2024-08-31T07:20:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.922+0000 7efbfca00640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efbdc0b4d20 0x7efbdc0b7230 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.922+0000 7efbfca00640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbf81099e0 0x7efbf8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.922+0000 7efbfca00640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efbf8109090 0x7efbf80759f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.922+0000 7efbfca00640 1 --2- 172.21.15.32:0/1888474522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8103750 0x7efbf80754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.922+0000 7efbfca00640 1 -- 172.21.15.32:0/1888474522 >> 172.21.15.32:0/1888474522 conn(0x7efbf80ff1e0 msgr2=0x7efbf8101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.922+0000 7efbfca00640 1 -- 172.21.15.32:0/1888474522 shutdown_connections 2024-08-31T07:20:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:51.922+0000 7efbfca00640 1 -- 172.21.15.32:0/1888474522 wait complete. 2024-08-31T07:20:52.482 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd2dfb577', total seconds elapsed: 35.1 2024-08-31T07:20:52.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 239364 2.60 MB/sec execute 281 sec latency 267.091 ms 2024-08-31T07:20:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:52 smithi073 ceph-mon[21995]: pgmap v615: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-08-31T07:20:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:52 smithi084 ceph-mon[22387]: pgmap v615: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-08-31T07:20:53.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:52 smithi032 ceph-mon[17252]: pgmap v615: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-08-31T07:20:53.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:20:53.636 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:53.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 239388 2.59 MB/sec execute 282 sec latency 989.147 ms 2024-08-31T07:20:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.328+0000 7f000ac00640 1 -- 172.21.15.32:0/1321995130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0004105b20 msgr2=0x7f0004105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.328+0000 7f000ac00640 1 --2- 172.21.15.32:0/1321995130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0004105b20 0x7f0004105f30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efff4009900 tx=0x7efff402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.328+0000 7f000ac00640 1 -- 172.21.15.32:0/1321995130 shutdown_connections 2024-08-31T07:20:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.328+0000 7f000ac00640 1 --2- 172.21.15.32:0/1321995130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f000410bdb0 0x7f00041125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.328+0000 7f000ac00640 1 --2- 172.21.15.32:0/1321995130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f000410b460 0x7f000410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.328+0000 7f000ac00640 1 --2- 172.21.15.32:0/1321995130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0004105b20 0x7f0004105f30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.328+0000 7f000ac00640 1 -- 172.21.15.32:0/1321995130 >> 172.21.15.32:0/1321995130 conn(0x7f00041015b0 msgr2=0x7f0004103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.328+0000 7f000ac00640 1 -- 172.21.15.32:0/1321995130 shutdown_connections 2024-08-31T07:20:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.328+0000 7f000ac00640 1 -- 172.21.15.32:0/1321995130 wait complete. 2024-08-31T07:20:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.329+0000 7f000ac00640 1 Processor -- start 2024-08-31T07:20:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.329+0000 7f000ac00640 1 -- start start 2024-08-31T07:20:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7f000ac00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0004105b20 0x7f000419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7f000ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f000410b460 0x7f000419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7f000ac00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f000410bdb0 0x7f00041a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7f000ac00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0004115a70 con 0x7f0004105b20 2024-08-31T07:20:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7f000ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00041158f0 con 0x7f000410b460 2024-08-31T07:20:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7f000ac00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0004115bf0 con 0x7f000410bdb0 2024-08-31T07:20:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7f0003e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0004105b20 0x7f000419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7f0003e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0004105b20 0x7f000419c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58948/0 (socket says 172.21.15.32:58948) 2024-08-31T07:20:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7f0003e00640 1 -- 172.21.15.32:0/1214273859 learned_addr learned my addr 172.21.15.32:0/1214273859 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7f0008c00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f000410bdb0 0x7f00041a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.330+0000 7efffbe00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f000410b460 0x7f000419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.331+0000 7f0003e00640 1 -- 172.21.15.32:0/1214273859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f000410bdb0 msgr2=0x7f00041a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.331+0000 7f0003e00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f000410bdb0 0x7f00041a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.331+0000 7f0003e00640 1 -- 172.21.15.32:0/1214273859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f000410b460 msgr2=0x7f000419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.331+0000 7f0003e00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f000410b460 0x7f000419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.331+0000 7f0003e00640 1 -- 172.21.15.32:0/1214273859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00041a2410 con 0x7f0004105b20 2024-08-31T07:20:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.331+0000 7f0003e00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0004105b20 0x7f000419c4a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efff4005940 tx=0x7efff4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.331+0000 7f0001600640 1 -- 172.21.15.32:0/1214273859 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff403f030 con 0x7f0004105b20 2024-08-31T07:20:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.332+0000 7f0001600640 1 -- 172.21.15.32:0/1214273859 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4003aa0 con 0x7f0004105b20 2024-08-31T07:20:54.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.332+0000 7f000ac00640 1 -- 172.21.15.32:0/1214273859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00041a26a0 con 0x7f0004105b20 2024-08-31T07:20:54.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.332+0000 7f0001600640 1 -- 172.21.15.32:0/1214273859 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4035070 con 0x7f0004105b20 2024-08-31T07:20:54.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.332+0000 7f000ac00640 1 -- 172.21.15.32:0/1214273859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00041a8e60 con 0x7f0004105b20 2024-08-31T07:20:54.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.335+0000 7f00113fd740 1 Processor -- start 2024-08-31T07:20:54.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.335+0000 7f00113fd740 1 -- start start 2024-08-31T07:20:54.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.335+0000 7f0001600640 1 -- 172.21.15.32:0/1214273859 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7efff4042030 con 0x7f0004105b20 2024-08-31T07:20:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.335+0000 7f00113fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9c7070 0x55904cd6c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.335+0000 7f00113fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 0x55904cd6ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.335+0000 7f00113fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd6d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.335+0000 7f00113fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55904cce37b0 con 0x55904c9c7070 2024-08-31T07:20:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.335+0000 7f00113fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55904ccd2d40 con 0x55904c99e1b0 2024-08-31T07:20:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.335+0000 7f00113fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55904ccd6cf0 con 0x55904c9a6390 2024-08-31T07:20:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7f0003e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 0x55904cd6ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7f0008c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9c7070 0x55904cd6c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7f0003e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 0x55904cd6ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37594/0 (socket says 172.21.15.32:37594) 2024-08-31T07:20:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7f0003e00640 1 -- 172.21.15.32:0/2290288019 learned_addr learned my addr 172.21.15.32:0/2290288019 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7efffbe00640 1 --2- 172.21.15.32:0/2290288019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd6d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7f0003e00640 1 -- 172.21.15.32:0/2290288019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 msgr2=0x55904cd6d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7f0003e00640 1 --2- 172.21.15.32:0/2290288019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd6d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7f0003e00640 1 -- 172.21.15.32:0/2290288019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9c7070 msgr2=0x55904cd6c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7f0003e00640 1 --2- 172.21.15.32:0/2290288019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9c7070 0x55904cd6c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7f0003e00640 1 -- 172.21.15.32:0/2290288019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904ca32650 con 0x55904c99e1b0 2024-08-31T07:20:54.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.336+0000 7efffbe00640 1 --2- 172.21.15.32:0/2290288019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd6d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.337+0000 7f0003e00640 1 --2- 172.21.15.32:0/2290288019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 0x55904cd6ced0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efff4102e60 tx=0x7efff410bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f501f6f0d3f869 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.337+0000 7efff9200640 1 -- 172.21.15.32:0/2290288019 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4114070 con 0x55904c99e1b0 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.337+0000 7efff9200640 1 -- 172.21.15.32:0/2290288019 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff410dd30 con 0x55904c99e1b0 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.337+0000 7efff9200640 1 -- 172.21.15.32:0/2290288019 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff41079c0 con 0x55904c99e1b0 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.337+0000 7f0001600640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe80b4d20 0x7effe80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f0001600640 1 -- 172.21.15.32:0/1214273859 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7efff41057e0 con 0x7f0004105b20 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f0008c00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe80b4d20 0x7effe80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f00113fd740 1 -- 172.21.15.32:0/2290288019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 msgr2=0x55904cd6ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f00113fd740 1 --2- 172.21.15.32:0/2290288019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 0x55904cd6ced0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efff4102e60 tx=0x7efff410bcb0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f0008c00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe80b4d20 0x7effe80b7230 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efffc005fd0 tx=0x7efffc0096a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f00113fd740 1 -- 172.21.15.32:0/2290288019 shutdown_connections 2024-08-31T07:20:54.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f00113fd740 1 --2- 172.21.15.32:0/2290288019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd6d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f00113fd740 1 --2- 172.21.15.32:0/2290288019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 0x55904cd6ced0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f00113fd740 1 --2- 172.21.15.32:0/2290288019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9c7070 0x55904cd6c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f00113fd740 1 -- 172.21.15.32:0/2290288019 >> 172.21.15.32:0/2290288019 conn(0x55904c9bb460 msgr2=0x55904cd6a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f00113fd740 1 -- 172.21.15.32:0/2290288019 shutdown_connections 2024-08-31T07:20:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.338+0000 7f00113fd740 1 -- 172.21.15.32:0/2290288019 wait complete. 2024-08-31T07:20:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.339+0000 7f00113fd740 1 -- start start 2024-08-31T07:20:54.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.339+0000 7f00113fd740 1 Processor -- start 2024-08-31T07:20:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7f00113fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c99e1b0 0x55904cd785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7f00113fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd78b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7f00113fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 0x55904cd84190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7f00113fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55904cb35700 con 0x55904c99e1b0 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7f00113fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55904cc59cf0 con 0x55904c9c7070 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7f00113fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55904cd7af30 con 0x55904c9a6390 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7f00113fd740 10 client.0 _ll_register_callbacks cb 0x55904cc7f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7efffbe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd78b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7f0003e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c99e1b0 0x55904cd785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7efffbe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd78b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60628/0 (socket says 172.21.15.32:60628) 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7efffbe00640 1 -- 172.21.15.32:0/2109446243 learned_addr learned my addr 172.21.15.32:0/2109446243 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.340+0000 7f0008c00640 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 0x55904cd84190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7efffbe00640 1 -- 172.21.15.32:0/2109446243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 msgr2=0x55904cd84190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7efffbe00640 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 0x55904cd84190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7efffbe00640 1 -- 172.21.15.32:0/2109446243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c99e1b0 msgr2=0x55904cd785e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7efffbe00640 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c99e1b0 0x55904cd785e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7efffbe00640 1 -- 172.21.15.32:0/2109446243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904cc9a8b0 con 0x55904c9a6390 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7f0008c00640 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 0x55904cd84190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7f0003e00640 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c99e1b0 0x55904cd785e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7efffbe00640 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd78b20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7effec0126e0 tx=0x7effec012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=646eede1cbcee43 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7efff1600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7efff1600640 1 -- 172.21.15.32:0/2109446243 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec012190 con 0x55904c9a6390 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7f00113fd740 1 -- 172.21.15.32:0/2109446243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff410dd30 con 0x55904c9a6390 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7efff1600640 1 -- 172.21.15.32:0/2109446243 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7effec00f920 con 0x55904c9a6390 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.341+0000 7efff1600640 1 -- 172.21.15.32:0/2109446243 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7effec00fb20 con 0x55904c9a6390 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.342+0000 7f00113fd740 10 client.27265 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.342+0000 7f00113fd740 1 -- 172.21.15.32:0/2109446243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55904cd86f10 con 0x55904c9a6390 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.342+0000 7efff1600640 1 -- 172.21.15.32:0/2109446243 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff410dd30 con 0x55904c9a6390 2024-08-31T07:20:54.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.342+0000 7efff1600640 1 -- 172.21.15.32:0/2109446243 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7effec01c420 con 0x55904c9a6390 2024-08-31T07:20:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.342+0000 7f00113fd740 10 client.27265 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.342+0000 7f00113fd740 10 client.27265 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:20:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.342+0000 7f00113fd740 20 client.27265 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.342+0000 7f00113fd740 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55904c9859f0 0x55904cd8a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.342+0000 7f00113fd740 4 client.27265 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.342+0000 7f00113fd740 1 -- 172.21.15.32:0/2109446243 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55904cce5700 con 0x55904c9859f0 2024-08-31T07:20:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.343+0000 7f0003e00640 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55904c9859f0 0x55904cd8a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.343+0000 7f0003e00640 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55904c9859f0 0x55904cd8a3e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb4d9234d6402fa3 server_cookie=3c7e2a148c0168c6 in_seq=0 out_seq=0 2024-08-31T07:20:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.343+0000 7efff1600640 10 client.27265 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:20:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7efff1600640 1 -- 172.21.15.32:0/2109446243 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55904cce5700 con 0x55904c9859f0 2024-08-31T07:20:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7efff1600640 10 client.27265 handle_command_reply: tid=0 2024-08-31T07:20:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 client.27265 shutdown 2024-08-31T07:20:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 -- 172.21.15.32:0/2109446243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 msgr2=0x55904cd78b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd78b20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7effec0126e0 tx=0x7effec012ba0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 -- 172.21.15.32:0/2109446243 shutdown_connections 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55904c9859f0 0x55904cd8a3e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 0x55904cd84190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9a6390 0x55904cd78b20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 --2- 172.21.15.32:0/2109446243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c99e1b0 0x55904cd785e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7efff1600640 0 client.27265 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 -- 172.21.15.32:0/2109446243 >> 172.21.15.32:0/2109446243 conn(0x55904c9bb460 msgr2=0x55904cd6fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 -- 172.21.15.32:0/2109446243 shutdown_connections 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.344+0000 7f00113fd740 1 -- 172.21.15.32:0/2109446243 wait complete. 2024-08-31T07:20:54.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.345+0000 7f00113fd740 20 client.27265 trim_cache size 0 max 16384 2024-08-31T07:20:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.359+0000 7f00113fd740 1 Processor -- start 2024-08-31T07:20:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.359+0000 7f00113fd740 1 -- start start 2024-08-31T07:20:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.359+0000 7f00113fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c99e1b0 0x55904cd73b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.359+0000 7f00113fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.359+0000 7f00113fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 0x55904cd84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.359+0000 7f00113fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f00041b1610 con 0x55904c9a6390 2024-08-31T07:20:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.359+0000 7f00113fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00041b3210 con 0x55904c9c7070 2024-08-31T07:20:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.359+0000 7f00113fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f00041b1ce0 con 0x55904c99e1b0 2024-08-31T07:20:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7efffbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7efffbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58984/0 (socket says 172.21.15.32:58984) 2024-08-31T07:20:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7efffbe00640 1 -- 172.21.15.32:0/3829948642 learned_addr learned my addr 172.21.15.32:0/3829948642 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7f0003e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c99e1b0 0x55904cd73b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7f0008c00640 1 --2- 172.21.15.32:0/3829948642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 0x55904cd84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7f0003e00640 1 -- 172.21.15.32:0/3829948642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 msgr2=0x55904cd84090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7f0003e00640 1 --2- 172.21.15.32:0/3829948642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 0x55904cd84090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7f0003e00640 1 -- 172.21.15.32:0/3829948642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 msgr2=0x55904cd7a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7f0003e00640 1 --2- 172.21.15.32:0/3829948642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7f0003e00640 1 -- 172.21.15.32:0/3829948642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904cd6ec80 con 0x55904c99e1b0 2024-08-31T07:20:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7efffbe00640 1 --2- 172.21.15.32:0/3829948642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.360+0000 7f0003e00640 1 --2- 172.21.15.32:0/3829948642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c99e1b0 0x55904cd73b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efff4006010 tx=0x7efff4107520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7577c648bf186cd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.361+0000 7efff3e00640 1 -- 172.21.15.32:0/3829948642 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4049070 con 0x55904c99e1b0 2024-08-31T07:20:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.361+0000 7efff3e00640 1 -- 172.21.15.32:0/3829948642 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff410b300 con 0x55904c99e1b0 2024-08-31T07:20:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.361+0000 7efff3e00640 1 -- 172.21.15.32:0/3829948642 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4114070 con 0x55904c99e1b0 2024-08-31T07:20:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 -- 172.21.15.32:0/3829948642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c99e1b0 msgr2=0x55904cd73b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 --2- 172.21.15.32:0/3829948642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c99e1b0 0x55904cd73b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efff4006010 tx=0x7efff4107520 comp rx=0 tx=0).stop 2024-08-31T07:20:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 -- 172.21.15.32:0/3829948642 shutdown_connections 2024-08-31T07:20:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 --2- 172.21.15.32:0/3829948642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c9c7070 0x55904cd84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 --2- 172.21.15.32:0/3829948642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 --2- 172.21.15.32:0/3829948642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c99e1b0 0x55904cd73b90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 -- 172.21.15.32:0/3829948642 >> 172.21.15.32:0/3829948642 conn(0x55904c9bb460 msgr2=0x55904cd79900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 -- 172.21.15.32:0/3829948642 shutdown_connections 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 -- 172.21.15.32:0/3829948642 wait complete. 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 -- start start 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.362+0000 7f00113fd740 1 Processor -- start 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7f00113fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 0x55904cd805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7f00113fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7f00113fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9c7070 0x55904cd7cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7f00113fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f00041b2a80 con 0x55904c9a6390 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7f00113fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00041b23b0 con 0x55904c99e1b0 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7f00113fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f00041af610 con 0x55904c9c7070 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7f00113fd740 10 client.0 _ll_register_callbacks cb 0x7effec009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7efffbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7efffbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58994/0 (socket says 172.21.15.32:58994) 2024-08-31T07:20:54.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7efffbe00640 1 -- 172.21.15.32:0/3353459144 learned_addr learned my addr 172.21.15.32:0/3353459144 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:54.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7f0003e00640 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 0x55904cd805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.363+0000 7f0008c00640 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9c7070 0x55904cd7cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7efffbe00640 1 -- 172.21.15.32:0/3353459144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9c7070 msgr2=0x55904cd7cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7efffbe00640 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9c7070 0x55904cd7cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7efffbe00640 1 -- 172.21.15.32:0/3353459144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 msgr2=0x55904cd805d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7efffbe00640 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 0x55904cd805d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7efffbe00640 1 -- 172.21.15.32:0/3353459144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904cd7d510 con 0x55904c9a6390 2024-08-31T07:20:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7f0008c00640 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9c7070 0x55904cd7cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7efffbe00640 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7ca90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7effec001eb0 tx=0x7effec0123c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a214f00f91384b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7efff1600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7efff1600640 1 -- 172.21.15.32:0/3353459144 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec029070 con 0x55904c9a6390 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7efff1600640 1 -- 172.21.15.32:0/3353459144 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7effec00f920 con 0x55904c9a6390 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.364+0000 7f00113fd740 1 -- 172.21.15.32:0/3353459144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff410b300 con 0x55904c9a6390 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7efff1600640 1 -- 172.21.15.32:0/3353459144 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7effec01c420 con 0x55904c9a6390 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7f00113fd740 10 client.17253 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7f00113fd740 1 -- 172.21.15.32:0/3353459144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55904cdf59f0 con 0x55904c9a6390 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7efff1600640 1 -- 172.21.15.32:0/3353459144 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff410b300 con 0x55904c9a6390 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7efff1600640 1 -- 172.21.15.32:0/3353459144 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7effec01cce0 con 0x55904c9a6390 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7f00113fd740 10 client.17253 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7f00113fd740 10 client.17253 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7f00113fd740 20 client.17253 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7f00113fd740 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55904c95ada0 0x55904cdf8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7f00113fd740 4 client.17253 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.365+0000 7f00113fd740 1 -- 172.21.15.32:0/3353459144 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55904cce11a0 con 0x55904c95ada0 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.366+0000 7f0003e00640 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55904c95ada0 0x55904cdf8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.366+0000 7f0003e00640 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55904c95ada0 0x55904cdf8190 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ecb3c156680084 server_cookie=5e4c0bd72c0c1a1e in_seq=0 out_seq=0 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.366+0000 7efff1600640 10 client.17253 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7efff1600640 1 -- 172.21.15.32:0/3353459144 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55904cce11a0 con 0x55904c95ada0 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7efff1600640 10 client.17253 handle_command_reply: tid=0 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 client.17253 shutdown 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 -- 172.21.15.32:0/3353459144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 msgr2=0x55904cd7ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7ca90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7effec001eb0 tx=0x7effec0123c0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 -- 172.21.15.32:0/3353459144 shutdown_connections 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55904c95ada0 0x55904cdf8190 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55904c9c7070 0x55904cd7cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55904c9a6390 0x55904cd7ca90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 --2- 172.21.15.32:0/3353459144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55904c99e1b0 0x55904cd805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7efff1600640 0 client.17253 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:20:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 -- 172.21.15.32:0/3353459144 >> 172.21.15.32:0/3353459144 conn(0x55904c9bb460 msgr2=0x55904cd79860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 -- 172.21.15.32:0/3353459144 shutdown_connections 2024-08-31T07:20:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 1 -- 172.21.15.32:0/3353459144 wait complete. 2024-08-31T07:20:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.367+0000 7f00113fd740 20 client.17253 trim_cache size 0 max 16384 2024-08-31T07:20:54.371 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:20:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.368+0000 7f000ac00640 1 -- 172.21.15.32:0/1214273859 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe80b4d20 msgr2=0x7effe80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.368+0000 7f000ac00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe80b4d20 0x7effe80b7230 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efffc005fd0 tx=0x7efffc0096a0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.368+0000 7f000ac00640 1 -- 172.21.15.32:0/1214273859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0004105b20 msgr2=0x7f000419c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.368+0000 7f000ac00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0004105b20 0x7f000419c4a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efff4005940 tx=0x7efff4004270 comp rx=0 tx=0).stop 2024-08-31T07:20:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.368+0000 7f000ac00640 1 -- 172.21.15.32:0/1214273859 shutdown_connections 2024-08-31T07:20:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.368+0000 7f000ac00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effe80b4d20 0x7effe80b7230 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.368+0000 7f000ac00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f000410bdb0 0x7f00041a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.368+0000 7f000ac00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f000410b460 0x7f000419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.368+0000 7f000ac00640 1 --2- 172.21.15.32:0/1214273859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0004105b20 0x7f000419c4a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:54.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.368+0000 7f000ac00640 1 -- 172.21.15.32:0/1214273859 >> 172.21.15.32:0/1214273859 conn(0x7f00041015b0 msgr2=0x7f00041039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:54.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.369+0000 7f000ac00640 1 -- 172.21.15.32:0/1214273859 shutdown_connections 2024-08-31T07:20:54.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:54.369+0000 7f000ac00640 1 -- 172.21.15.32:0/1214273859 wait complete. 2024-08-31T07:20:54.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 239388 2.58 MB/sec execute 283 sec latency 1989.263 ms 2024-08-31T07:20:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:54 smithi073 ceph-mon[21995]: pgmap v616: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 63 KiB/s wr, 18 op/s 2024-08-31T07:20:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:54 smithi084 ceph-mon[22387]: pgmap v616: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 63 KiB/s wr, 18 op/s 2024-08-31T07:20:55.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:54 smithi032 ceph-mon[17252]: pgmap v616: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 63 KiB/s wr, 18 op/s 2024-08-31T07:20:55.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:20:55.718 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:55.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 240677 2.60 MB/sec execute 284 sec latency 2362.453 ms 2024-08-31T07:20:56.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.396+0000 7fb65ec00640 1 -- 172.21.15.32:0/847301972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6581099e0 msgr2=0x7fb658110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.396+0000 7fb65ec00640 1 --2- 172.21.15.32:0/847301972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6581099e0 0x7fb658110210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb650009be0 tx=0x7fb65002a6d0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.396+0000 7fb65ec00640 1 -- 172.21.15.32:0/847301972 shutdown_connections 2024-08-31T07:20:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.396+0000 7fb65ec00640 1 --2- 172.21.15.32:0/847301972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6581099e0 0x7fb658110210 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.396+0000 7fb65ec00640 1 --2- 172.21.15.32:0/847301972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb658109090 0x7fb6581094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.396+0000 7fb65ec00640 1 --2- 172.21.15.32:0/847301972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb658103750 0x7fb658103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.396+0000 7fb65ec00640 1 -- 172.21.15.32:0/847301972 >> 172.21.15.32:0/847301972 conn(0x7fb6580ff1e0 msgr2=0x7fb658101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.396+0000 7fb65ec00640 1 -- 172.21.15.32:0/847301972 shutdown_connections 2024-08-31T07:20:56.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.396+0000 7fb65ec00640 1 -- 172.21.15.32:0/847301972 wait complete. 2024-08-31T07:20:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.398+0000 7fb65ec00640 1 Processor -- start 2024-08-31T07:20:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.398+0000 7fb65ec00640 1 -- start start 2024-08-31T07:20:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.398+0000 7fb65ec00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb658103750 0x7fb6580754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.398+0000 7fb65ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb658109090 0x7fb6580759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.398+0000 7fb65ec00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6581099e0 0x7fb658075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ec00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb658113690 con 0x7fb658103750 2024-08-31T07:20:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb658113510 con 0x7fb658109090 2024-08-31T07:20:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ec00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb658113810 con 0x7fb6581099e0 2024-08-31T07:20:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb658109090 0x7fb6580759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb658109090 0x7fb6580759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59254/0 (socket says 172.21.15.32:59254) 2024-08-31T07:20:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65e200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6581099e0 0x7fb658075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ce00640 1 -- 172.21.15.32:0/1560528775 learned_addr learned my addr 172.21.15.32:0/1560528775 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:56.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65d800640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb658103750 0x7fb6580754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ce00640 1 -- 172.21.15.32:0/1560528775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6581099e0 msgr2=0x7fb658075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:56.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ce00640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6581099e0 0x7fb658075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ce00640 1 -- 172.21.15.32:0/1560528775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb658103750 msgr2=0x7fb6580754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:56.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ce00640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb658103750 0x7fb6580754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.399+0000 7fb65ce00640 1 -- 172.21.15.32:0/1560528775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6581a8280 con 0x7fb658109090 2024-08-31T07:20:56.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.400+0000 7fb65ce00640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb658109090 0x7fb6580759f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb6440058c0 tx=0x7fb64400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:56.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.400+0000 7fb656000640 1 -- 172.21.15.32:0/1560528775 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64400cd30 con 0x7fb658109090 2024-08-31T07:20:56.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.400+0000 7fb656000640 1 -- 172.21.15.32:0/1560528775 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6440035f0 con 0x7fb658109090 2024-08-31T07:20:56.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.400+0000 7fb656000640 1 -- 172.21.15.32:0/1560528775 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6440044e0 con 0x7fb658109090 2024-08-31T07:20:56.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.400+0000 7fb65ec00640 1 -- 172.21.15.32:0/1560528775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6581a8570 con 0x7fb658109090 2024-08-31T07:20:56.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.400+0000 7fb65ec00640 1 -- 172.21.15.32:0/1560528775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581a8a60 con 0x7fb658109090 2024-08-31T07:20:56.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.403+0000 7fb666e7c740 1 Processor -- start 2024-08-31T07:20:56.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.403+0000 7fb666e7c740 1 -- start start 2024-08-31T07:20:56.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb656000640 1 -- 172.21.15.32:0/1560528775 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb644003aa0 con 0x7fb658109090 2024-08-31T07:20:56.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bc12070 0x55668bfb7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbe91b0 0x55668bfb80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 0x55668bfb85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb666e7c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55668bf2e7b0 con 0x55668bc12070 2024-08-31T07:20:56.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb666e7c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55668bf1dd40 con 0x55668bbe91b0 2024-08-31T07:20:56.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb666e7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55668bf21cf0 con 0x55668bbf1390 2024-08-31T07:20:56.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb65d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbe91b0 0x55668bfb80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb65ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 0x55668bfb85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb65e200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bc12070 0x55668bfb7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb65d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbe91b0 0x55668bfb80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59262/0 (socket says 172.21.15.32:59262) 2024-08-31T07:20:56.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.404+0000 7fb65d800640 1 -- 172.21.15.32:0/941979734 learned_addr learned my addr 172.21.15.32:0/941979734 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:56.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.405+0000 7fb65d800640 1 -- 172.21.15.32:0/941979734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 msgr2=0x55668bfb85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.405+0000 7fb65d800640 1 --2- 172.21.15.32:0/941979734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 0x55668bfb85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.405+0000 7fb65d800640 1 -- 172.21.15.32:0/941979734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bc12070 msgr2=0x55668bfb7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.405+0000 7fb65d800640 1 --2- 172.21.15.32:0/941979734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bc12070 0x55668bfb7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.405+0000 7fb65d800640 1 -- 172.21.15.32:0/941979734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668bc7d650 con 0x55668bbe91b0 2024-08-31T07:20:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.405+0000 7fb65d800640 1 --2- 172.21.15.32:0/941979734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbe91b0 0x55668bfb80a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00bd10 tx=0x7fb64c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeca84e5288f6045 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.405+0000 7fb64aa00640 1 -- 172.21.15.32:0/941979734 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c005220 con 0x55668bbe91b0 2024-08-31T07:20:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.405+0000 7fb64aa00640 1 -- 172.21.15.32:0/941979734 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c004c50 con 0x55668bbe91b0 2024-08-31T07:20:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.405+0000 7fb64aa00640 1 -- 172.21.15.32:0/941979734 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0117e0 con 0x55668bbe91b0 2024-08-31T07:20:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.406+0000 7fb656000640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb6400b4d20 0x7fb6400b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.406+0000 7fb656000640 1 -- 172.21.15.32:0/1560528775 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7fb6440e5660 con 0x7fb658109090 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.406+0000 7fb666e7c740 1 -- 172.21.15.32:0/941979734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbe91b0 msgr2=0x55668bfb80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.406+0000 7fb666e7c740 1 --2- 172.21.15.32:0/941979734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbe91b0 0x55668bfb80a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00bd10 tx=0x7fb64c0107a0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.406+0000 7fb65e200640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb6400b4d20 0x7fb6400b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.407+0000 7fb666e7c740 1 -- 172.21.15.32:0/941979734 shutdown_connections 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.407+0000 7fb666e7c740 1 --2- 172.21.15.32:0/941979734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 0x55668bfb85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.407+0000 7fb666e7c740 1 --2- 172.21.15.32:0/941979734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbe91b0 0x55668bfb80a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.407+0000 7fb666e7c740 1 --2- 172.21.15.32:0/941979734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bc12070 0x55668bfb7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.407+0000 7fb666e7c740 1 -- 172.21.15.32:0/941979734 >> 172.21.15.32:0/941979734 conn(0x55668bc06460 msgr2=0x55668bfb5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.407+0000 7fb666e7c740 1 -- 172.21.15.32:0/941979734 shutdown_connections 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.407+0000 7fb666e7c740 1 -- 172.21.15.32:0/941979734 wait complete. 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.407+0000 7fb65e200640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb6400b4d20 0x7fb6400b7230 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb650004af0 tx=0x7fb650005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.407+0000 7fb666e7c740 1 -- start start 2024-08-31T07:20:56.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.408+0000 7fb666e7c740 1 Processor -- start 2024-08-31T07:20:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.408+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbe91b0 0x55668bfc3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.408+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 0x55668bfc3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.408+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfcf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.408+0000 7fb666e7c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55668bd80700 con 0x55668bbe91b0 2024-08-31T07:20:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.408+0000 7fb666e7c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55668bea4cf0 con 0x55668bc12070 2024-08-31T07:20:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.408+0000 7fb666e7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55668bfc6340 con 0x55668bbf1390 2024-08-31T07:20:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.408+0000 7fb666e7c740 10 client.0 _ll_register_callbacks cb 0x55668beca470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 0x55668bfc3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfcf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 0x55668bfc3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:49548/0 (socket says 172.21.15.32:49548) 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbe91b0 0x55668bfc3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65ce00640 1 -- 172.21.15.32:0/1834398735 learned_addr learned my addr 172.21.15.32:0/1834398735 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65ce00640 1 -- 172.21.15.32:0/1834398735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 msgr2=0x55668bfcf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65ce00640 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfcf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65ce00640 1 -- 172.21.15.32:0/1834398735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbe91b0 msgr2=0x55668bfc3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65ce00640 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbe91b0 0x55668bfc3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65ce00640 1 -- 172.21.15.32:0/1834398735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668bee58b0 con 0x55668bbf1390 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65e200640 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfcf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb65ce00640 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 0x55668bfc3c90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb644000c00 tx=0x7fb6440a82b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e985638ec29d12a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.409+0000 7fb63f400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.410+0000 7fb63f400640 1 -- 172.21.15.32:0/1834398735 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644009530 con 0x55668bbf1390 2024-08-31T07:20:56.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.410+0000 7fb63f400640 1 -- 172.21.15.32:0/1834398735 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7fb6440b4c30 con 0x55668bbf1390 2024-08-31T07:20:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.410+0000 7fb666e7c740 1 -- 172.21.15.32:0/1834398735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c004c50 con 0x55668bbf1390 2024-08-31T07:20:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.410+0000 7fb63f400640 1 -- 172.21.15.32:0/1834398735 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb6440ab8b0 con 0x55668bbf1390 2024-08-31T07:20:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.410+0000 7fb666e7c740 10 client.27283 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.410+0000 7fb666e7c740 1 -- 172.21.15.32:0/1834398735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55668bfd2350 con 0x55668bbf1390 2024-08-31T07:20:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.410+0000 7fb63f400640 1 -- 172.21.15.32:0/1834398735 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c004c50 con 0x55668bbf1390 2024-08-31T07:20:56.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.410+0000 7fb63f400640 1 -- 172.21.15.32:0/1834398735 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fb6440b01f0 con 0x55668bbf1390 2024-08-31T07:20:56.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.410+0000 7fb666e7c740 10 client.27283 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:56.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.410+0000 7fb666e7c740 10 client.27283 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:20:56.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.411+0000 7fb666e7c740 20 client.27283 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:56.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.411+0000 7fb666e7c740 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55668bbd09f0 0x55668bfd5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.411+0000 7fb666e7c740 4 client.27283 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.411+0000 7fb666e7c740 1 -- 172.21.15.32:0/1834398735 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55668bf30700 con 0x55668bbd09f0 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.411+0000 7fb65d800640 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55668bbd09f0 0x55668bfd5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.412+0000 7fb65d800640 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55668bbd09f0 0x55668bfd5870 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b2f9eea7a16af4b server_cookie=9c4be3300333a7bd in_seq=0 out_seq=0 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.412+0000 7fb63f400640 10 client.27283 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.412+0000 7fb63f400640 1 -- 172.21.15.32:0/1834398735 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55668bf30700 con 0x55668bbd09f0 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.412+0000 7fb63f400640 10 client.27283 handle_command_reply: tid=0 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.412+0000 7fb666e7c740 1 client.27283 shutdown 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 1 -- 172.21.15.32:0/1834398735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 msgr2=0x55668bfc3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 0x55668bfc3c90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb644000c00 tx=0x7fb6440a82b0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 1 -- 172.21.15.32:0/1834398735 shutdown_connections 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55668bbd09f0 0x55668bfd5870 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfcf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbf1390 0x55668bfc3c90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 1 --2- 172.21.15.32:0/1834398735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbe91b0 0x55668bfc3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 1 -- 172.21.15.32:0/1834398735 >> 172.21.15.32:0/1834398735 conn(0x55668bc06460 msgr2=0x55668bfbad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb63f400640 0 client.27283 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:20:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 1 -- 172.21.15.32:0/1834398735 shutdown_connections 2024-08-31T07:20:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 1 -- 172.21.15.32:0/1834398735 wait complete. 2024-08-31T07:20:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.413+0000 7fb666e7c740 20 client.27283 trim_cache size 0 max 16384 2024-08-31T07:20:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb666e7c740 1 Processor -- start 2024-08-31T07:20:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb666e7c740 1 -- start start 2024-08-31T07:20:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbe91b0 0x55668bfbed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbf1390 0x55668bfcf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfb9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb666e7c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581b1360 con 0x55668bbf1390 2024-08-31T07:20:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb666e7c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581b2f60 con 0x55668bc12070 2024-08-31T07:20:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb666e7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581b1a30 con 0x55668bbe91b0 2024-08-31T07:20:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb65ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbf1390 0x55668bfcf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb65e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfb9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb65d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbe91b0 0x55668bfbed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb65e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfb9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59278/0 (socket says 172.21.15.32:59278) 2024-08-31T07:20:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.428+0000 7fb65e200640 1 -- 172.21.15.32:0/2454791842 learned_addr learned my addr 172.21.15.32:0/2454791842 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.429+0000 7fb65d800640 1 -- 172.21.15.32:0/2454791842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 msgr2=0x55668bfb9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.429+0000 7fb65d800640 1 --2- 172.21.15.32:0/2454791842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfb9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.429+0000 7fb65d800640 1 -- 172.21.15.32:0/2454791842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbf1390 msgr2=0x55668bfcf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.429+0000 7fb65d800640 1 --2- 172.21.15.32:0/2454791842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbf1390 0x55668bfcf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.429+0000 7fb65d800640 1 -- 172.21.15.32:0/2454791842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668bfafc70 con 0x55668bbe91b0 2024-08-31T07:20:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.429+0000 7fb65e200640 1 --2- 172.21.15.32:0/2454791842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfb9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.429+0000 7fb65ce00640 1 --2- 172.21.15.32:0/2454791842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbf1390 0x55668bfcf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:20:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.429+0000 7fb65d800640 1 --2- 172.21.15.32:0/2454791842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbe91b0 0x55668bfbed00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0057c0 tx=0x7fb64c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d303491e2d60e59b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.429+0000 7fb64a000640 1 -- 172.21.15.32:0/2454791842 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c004220 con 0x55668bbe91b0 2024-08-31T07:20:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.430+0000 7fb64a000640 1 -- 172.21.15.32:0/2454791842 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c0043c0 con 0x55668bbe91b0 2024-08-31T07:20:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.430+0000 7fb64a000640 1 -- 172.21.15.32:0/2454791842 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c01b6b0 con 0x55668bbe91b0 2024-08-31T07:20:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.430+0000 7fb666e7c740 1 -- 172.21.15.32:0/2454791842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbe91b0 msgr2=0x55668bfbed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.430+0000 7fb666e7c740 1 --2- 172.21.15.32:0/2454791842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbe91b0 0x55668bfbed00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0057c0 tx=0x7fb64c005720 comp rx=0 tx=0).stop 2024-08-31T07:20:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.430+0000 7fb666e7c740 1 -- 172.21.15.32:0/2454791842 shutdown_connections 2024-08-31T07:20:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.430+0000 7fb666e7c740 1 --2- 172.21.15.32:0/2454791842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bc12070 0x55668bfb9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.430+0000 7fb666e7c740 1 --2- 172.21.15.32:0/2454791842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbf1390 0x55668bfcf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.430+0000 7fb666e7c740 1 --2- 172.21.15.32:0/2454791842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bbe91b0 0x55668bfbed00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.430+0000 7fb666e7c740 1 -- 172.21.15.32:0/2454791842 >> 172.21.15.32:0/2454791842 conn(0x55668bc06460 msgr2=0x55668bfc4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.431+0000 7fb666e7c740 1 -- 172.21.15.32:0/2454791842 shutdown_connections 2024-08-31T07:20:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.431+0000 7fb666e7c740 1 -- 172.21.15.32:0/2454791842 wait complete. 2024-08-31T07:20:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.431+0000 7fb666e7c740 1 -- start start 2024-08-31T07:20:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.431+0000 7fb666e7c740 1 Processor -- start 2024-08-31T07:20:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbe91b0 0x55668bfc79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbf1390 0x55668bfc7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb666e7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bc12070 0x55668c03f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb666e7c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581b27d0 con 0x55668bbe91b0 2024-08-31T07:20:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb666e7c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581b2100 con 0x55668bbf1390 2024-08-31T07:20:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb666e7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581af360 con 0x55668bc12070 2024-08-31T07:20:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb666e7c740 10 client.0 _ll_register_callbacks cb 0x7fb6440eb240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb65ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbf1390 0x55668bfc7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb65ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbf1390 0x55668bfc7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59288/0 (socket says 172.21.15.32:59288) 2024-08-31T07:20:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb65d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbe91b0 0x55668bfc79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.432+0000 7fb65ce00640 1 -- 172.21.15.32:0/2973170837 learned_addr learned my addr 172.21.15.32:0/2973170837 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb65e200640 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bc12070 0x55668c03f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb65ce00640 1 -- 172.21.15.32:0/2973170837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bc12070 msgr2=0x55668c03f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb65ce00640 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bc12070 0x55668c03f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb65ce00640 1 -- 172.21.15.32:0/2973170837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbe91b0 msgr2=0x55668bfc79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb65ce00640 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbe91b0 0x55668bfc79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb65ce00640 1 -- 172.21.15.32:0/2973170837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668c03f750 con 0x55668bbf1390 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb65ce00640 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbf1390 0x55668bfc7f60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb6440a7e20 tx=0x7fb644092fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ad5e5731677670 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb63f400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb63f400640 1 -- 172.21.15.32:0/2973170837 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644009530 con 0x55668bbf1390 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb63f400640 1 -- 172.21.15.32:0/2973170837 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7fb6440b4e40 con 0x55668bbf1390 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb666e7c740 1 -- 172.21.15.32:0/2973170837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c0043c0 con 0x55668bbf1390 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.433+0000 7fb63f400640 1 -- 172.21.15.32:0/2973170837 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb6440ac4a0 con 0x55668bbf1390 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb666e7c740 10 client.27284 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb666e7c740 1 -- 172.21.15.32:0/2973170837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55668c0408d0 con 0x55668bbf1390 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb63f400640 1 -- 172.21.15.32:0/2973170837 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c0043c0 con 0x55668bbf1390 2024-08-31T07:20:56.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb63f400640 1 -- 172.21.15.32:0/2973170837 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fb6440a78a0 con 0x55668bbf1390 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb666e7c740 10 client.27284 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb666e7c740 10 client.27284 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb666e7c740 20 client.27284 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb666e7c740 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55668bba5da0 0x55668c042f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb666e7c740 4 client.27284 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb666e7c740 1 -- 172.21.15.32:0/2973170837 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55668bf2c1a0 con 0x55668bba5da0 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.434+0000 7fb65d800640 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55668bba5da0 0x55668c042f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.435+0000 7fb63f400640 10 client.27284 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.435+0000 7fb65d800640 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55668bba5da0 0x55668c042f00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8639530c2250416 server_cookie=68e3e2996ed5c0b5 in_seq=0 out_seq=0 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.435+0000 7fb63f400640 1 -- 172.21.15.32:0/2973170837 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55668bf2c1a0 con 0x55668bba5da0 2024-08-31T07:20:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.435+0000 7fb63f400640 10 client.27284 handle_command_reply: tid=0 2024-08-31T07:20:56.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.435+0000 7fb666e7c740 1 client.27284 shutdown 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 1 -- 172.21.15.32:0/2973170837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbf1390 msgr2=0x55668bfc7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbf1390 0x55668bfc7f60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb6440a7e20 tx=0x7fb644092fa0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 1 -- 172.21.15.32:0/2973170837 shutdown_connections 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55668bc12070 0x55668c03f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55668bba5da0 0x55668c042f00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668bbf1390 0x55668bfc7f60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 1 --2- 172.21.15.32:0/2973170837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55668bbe91b0 0x55668bfc79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 1 -- 172.21.15.32:0/2973170837 >> 172.21.15.32:0/2973170837 conn(0x55668bc06460 msgr2=0x55668bfb9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb63f400640 0 client.27284 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 1 -- 172.21.15.32:0/2973170837 shutdown_connections 2024-08-31T07:20:56.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 1 -- 172.21.15.32:0/2973170837 wait complete. 2024-08-31T07:20:56.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.436+0000 7fb666e7c740 20 client.27284 trim_cache size 0 max 16384 2024-08-31T07:20:56.441 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 -- 172.21.15.32:0/1560528775 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb6400b4d20 msgr2=0x7fb6400b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb6400b4d20 0x7fb6400b7230 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb650004af0 tx=0x7fb650005fd0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 -- 172.21.15.32:0/1560528775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb658109090 msgr2=0x7fb6580759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb658109090 0x7fb6580759f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb6440058c0 tx=0x7fb64400ee50 comp rx=0 tx=0).stop 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 -- 172.21.15.32:0/1560528775 shutdown_connections 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb6400b4d20 0x7fb6400b7230 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6581099e0 0x7fb658075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb658109090 0x7fb6580759f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 --2- 172.21.15.32:0/1560528775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb658103750 0x7fb6580754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 -- 172.21.15.32:0/1560528775 >> 172.21.15.32:0/1560528775 conn(0x7fb6580ff1e0 msgr2=0x7fb658101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:56.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 -- 172.21.15.32:0/1560528775 shutdown_connections 2024-08-31T07:20:56.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:56.437+0000 7fb65ec00640 1 -- 172.21.15.32:0/1560528775 wait complete. 2024-08-31T07:20:56.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 241993 2.62 MB/sec execute 285 sec latency 17.188 ms 2024-08-31T07:20:56.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:56 smithi032 ceph-mon[17252]: pgmap v617: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 202 KiB/s wr, 24 op/s 2024-08-31T07:20:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:56 smithi073 ceph-mon[21995]: pgmap v617: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 202 KiB/s wr, 24 op/s 2024-08-31T07:20:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:56 smithi084 ceph-mon[22387]: pgmap v617: 97 pgs: 97 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 202 KiB/s wr, 24 op/s 2024-08-31T07:20:57.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:20:57.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 243331 2.63 MB/sec execute 286 sec latency 87.216 ms 2024-08-31T07:20:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:20:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.639+0000 7f864f200640 1 -- 172.21.15.32:0/683241493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8650109090 msgr2=0x7f86501094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.639+0000 7f864f200640 1 --2- 172.21.15.32:0/683241493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8650109090 0x7f86501094a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f863c009900 tx=0x7f863c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.640+0000 7f864f200640 1 -- 172.21.15.32:0/683241493 shutdown_connections 2024-08-31T07:20:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.640+0000 7f864f200640 1 --2- 172.21.15.32:0/683241493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86501099e0 0x7f8650110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.640+0000 7f864f200640 1 --2- 172.21.15.32:0/683241493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8650109090 0x7f86501094a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.640+0000 7f864f200640 1 --2- 172.21.15.32:0/683241493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8650103750 0x7f8650103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.640+0000 7f864f200640 1 -- 172.21.15.32:0/683241493 >> 172.21.15.32:0/683241493 conn(0x7f86500ff1e0 msgr2=0x7f8650101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:58.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.640+0000 7f864f200640 1 -- 172.21.15.32:0/683241493 shutdown_connections 2024-08-31T07:20:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.640+0000 7f864f200640 1 -- 172.21.15.32:0/683241493 wait complete. 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.641+0000 7f864f200640 1 Processor -- start 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.641+0000 7f864f200640 1 -- start start 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.641+0000 7f864f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8650103750 0x7f865019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.641+0000 7f864f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8650109090 0x7f865019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86501099e0 0x7f86501a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f86501136c0 con 0x7f8650109090 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8650113540 con 0x7f8650103750 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8650113840 con 0x7f86501099e0 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8650103750 0x7f865019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8650103750 0x7f865019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59304/0 (socket says 172.21.15.32:59304) 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864de00640 1 -- 172.21.15.32:0/1931075974 learned_addr learned my addr 172.21.15.32:0/1931075974 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864d400640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8650109090 0x7f865019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864e800640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86501099e0 0x7f86501a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864de00640 1 -- 172.21.15.32:0/1931075974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86501099e0 msgr2=0x7f86501a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864de00640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86501099e0 0x7f86501a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864de00640 1 -- 172.21.15.32:0/1931075974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8650109090 msgr2=0x7f865019c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864de00640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8650109090 0x7f865019c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.642+0000 7f864de00640 1 -- 172.21.15.32:0/1931075974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86501a2220 con 0x7f8650103750 2024-08-31T07:20:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.643+0000 7f864de00640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8650103750 0x7f865019c2b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f864000e9f0 tx=0x7f864000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.643+0000 7f8646a00640 1 -- 172.21.15.32:0/1931075974 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864000cd80 con 0x7f8650103750 2024-08-31T07:20:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.643+0000 7f8646a00640 1 -- 172.21.15.32:0/1931075974 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86400108e0 con 0x7f8650103750 2024-08-31T07:20:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.643+0000 7f864f200640 1 -- 172.21.15.32:0/1931075974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86501a2510 con 0x7f8650103750 2024-08-31T07:20:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.643+0000 7f8646a00640 1 -- 172.21.15.32:0/1931075974 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640018980 con 0x7f8650103750 2024-08-31T07:20:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.643+0000 7f864f200640 1 -- 172.21.15.32:0/1931075974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86501a8c70 con 0x7f8650103750 2024-08-31T07:20:58.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f865b7d1740 1 Processor -- start 2024-08-31T07:20:58.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f865b7d1740 1 -- start start 2024-08-31T07:20:58.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f8646a00640 1 -- 172.21.15.32:0/1931075974 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8640010430 con 0x7f8650103750 2024-08-31T07:20:58.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 0x564e89d7eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 0x564e89d7f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b8390 0x564e89d7f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f865b7d1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564e89cf57b0 con 0x564e899b01b0 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f865b7d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564e89ce4d40 con 0x564e899d9070 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f865b7d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564e89ce8cf0 con 0x564e899b8390 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f864e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 0x564e89d7eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f864e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 0x564e89d7eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59310/0 (socket says 172.21.15.32:59310) 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f864e800640 1 -- 172.21.15.32:0/460978042 learned_addr learned my addr 172.21.15.32:0/460978042 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.648+0000 7f864d400640 1 --2- 172.21.15.32:0/460978042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b8390 0x564e89d7f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.647+0000 7f864de00640 1 --2- 172.21.15.32:0/460978042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 0x564e89d7f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.648+0000 7f864e800640 1 -- 172.21.15.32:0/460978042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b8390 msgr2=0x564e89d7f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:58.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.648+0000 7f864e800640 1 --2- 172.21.15.32:0/460978042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b8390 0x564e89d7f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.648+0000 7f864e800640 1 -- 172.21.15.32:0/460978042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 msgr2=0x564e89d7f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:58.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.648+0000 7f864e800640 1 --2- 172.21.15.32:0/460978042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 0x564e89d7f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.648+0000 7f864e800640 1 -- 172.21.15.32:0/460978042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e89a44650 con 0x564e899d9070 2024-08-31T07:20:58.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.648+0000 7f864de00640 1 --2- 172.21.15.32:0/460978042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 0x564e89d7f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:58.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.648+0000 7f864e800640 1 --2- 172.21.15.32:0/460978042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 0x564e89d7eb60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8648009fd0 tx=0x7f86480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5143c9cb2ddc8fa2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:58.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.649+0000 7f863b000640 1 -- 172.21.15.32:0/460978042 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648004d20 con 0x564e899d9070 2024-08-31T07:20:58.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.649+0000 7f863b000640 1 -- 172.21.15.32:0/460978042 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8648004050 con 0x564e899d9070 2024-08-31T07:20:58.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.649+0000 7f863b000640 1 -- 172.21.15.32:0/460978042 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648005280 con 0x564e899d9070 2024-08-31T07:20:58.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.649+0000 7f8646a00640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86340b4d20 0x7f86340b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.649+0000 7f8646a00640 1 -- 172.21.15.32:0/1931075974 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f86400e5780 con 0x7f8650103750 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.649+0000 7f864de00640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86340b4d20 0x7f86340b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.649+0000 7f865b7d1740 1 -- 172.21.15.32:0/460978042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 msgr2=0x564e89d7eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.649+0000 7f865b7d1740 1 --2- 172.21.15.32:0/460978042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 0x564e89d7eb60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8648009fd0 tx=0x7f86480118d0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.650+0000 7f865b7d1740 1 -- 172.21.15.32:0/460978042 shutdown_connections 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.650+0000 7f865b7d1740 1 --2- 172.21.15.32:0/460978042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b8390 0x564e89d7f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.650+0000 7f865b7d1740 1 --2- 172.21.15.32:0/460978042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 0x564e89d7f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.650+0000 7f865b7d1740 1 --2- 172.21.15.32:0/460978042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 0x564e89d7eb60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.650+0000 7f865b7d1740 1 -- 172.21.15.32:0/460978042 >> 172.21.15.32:0/460978042 conn(0x564e899cd460 msgr2=0x564e89d7ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.650+0000 7f865b7d1740 1 -- 172.21.15.32:0/460978042 shutdown_connections 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.650+0000 7f864de00640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86340b4d20 0x7f86340b7230 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86400e2fd0 tx=0x7f8640000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:58.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.650+0000 7f865b7d1740 1 -- 172.21.15.32:0/460978042 wait complete. 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.651+0000 7f865b7d1740 1 -- start start 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.651+0000 7f865b7d1740 1 Processor -- start 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 0x564e89d8a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d8acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899d9070 0x564e89d96660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f865b7d1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564e89b47700 con 0x564e899d9070 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f865b7d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564e89c6bcf0 con 0x564e899b8390 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f865b7d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564e89d8d340 con 0x564e899b01b0 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f865b7d1740 10 client.0 _ll_register_callbacks cb 0x564e89c91470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d8acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d8acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59312/0 (socket says 172.21.15.32:59312) 2024-08-31T07:20:58.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864d400640 1 -- 172.21.15.32:0/1605045376 learned_addr learned my addr 172.21.15.32:0/1605045376 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864e800640 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 0x564e89d8a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864e800640 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899d9070 0x564e89d96660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864d400640 1 -- 172.21.15.32:0/1605045376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 msgr2=0x564e89d8a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864d400640 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 0x564e89d8a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864d400640 1 -- 172.21.15.32:0/1605045376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899d9070 msgr2=0x564e89d96660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864d400640 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899d9070 0x564e89d96660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864d400640 1 -- 172.21.15.32:0/1605045376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e89cac8b0 con 0x564e899b8390 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.652+0000 7f864e800640 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 0x564e89d8a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.653+0000 7f864d400640 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d8acb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f865006b730 tx=0x7f863c0043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf9cc9f5dee2860 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.653+0000 7f862f400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.653+0000 7f862f400640 1 -- 172.21.15.32:0/1605045376 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c035040 con 0x564e899b8390 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.653+0000 7f862f400640 1 -- 172.21.15.32:0/1605045376 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f863c005450 con 0x564e899b8390 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.653+0000 7f865b7d1740 1 -- 172.21.15.32:0/1605045376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8648004050 con 0x564e899b8390 2024-08-31T07:20:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.653+0000 7f862f400640 1 -- 172.21.15.32:0/1605045376 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f863c005650 con 0x564e899b8390 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.653+0000 7f865b7d1740 10 client.27296 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.653+0000 7f865b7d1740 1 -- 172.21.15.32:0/1605045376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564e89d99380 con 0x564e899b8390 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.654+0000 7f862f400640 1 -- 172.21.15.32:0/1605045376 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8648004050 con 0x564e899b8390 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.654+0000 7f862f400640 1 -- 172.21.15.32:0/1605045376 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f863c003960 con 0x564e899b8390 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.654+0000 7f865b7d1740 10 client.27296 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.654+0000 7f865b7d1740 10 client.27296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.654+0000 7f865b7d1740 20 client.27296 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.654+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564e899979f0 0x564e89d9c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.654+0000 7f865b7d1740 4 client.27296 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.654+0000 7f865b7d1740 1 -- 172.21.15.32:0/1605045376 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e89cf7700 con 0x564e899979f0 2024-08-31T07:20:58.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.654+0000 7f864e800640 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564e899979f0 0x564e89d9c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.655+0000 7f864e800640 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564e899979f0 0x564e89d9c8c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce663943f913981 server_cookie=975abc112e79b85b in_seq=0 out_seq=0 2024-08-31T07:20:58.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.655+0000 7f862f400640 10 client.27296 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f862f400640 1 -- 172.21.15.32:0/1605045376 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e89cf7700 con 0x564e899979f0 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f862f400640 10 client.27296 handle_command_reply: tid=0 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 client.27296 shutdown 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 -- 172.21.15.32:0/1605045376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 msgr2=0x564e89d8acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d8acb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f865006b730 tx=0x7f863c0043b0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 -- 172.21.15.32:0/1605045376 shutdown_connections 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564e899979f0 0x564e89d9c8c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899d9070 0x564e89d96660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d8acb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1605045376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 0x564e89d8a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f862f400640 0 client.27296 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 -- 172.21.15.32:0/1605045376 >> 172.21.15.32:0/1605045376 conn(0x564e899cd460 msgr2=0x564e89d81cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 -- 172.21.15.32:0/1605045376 shutdown_connections 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.656+0000 7f865b7d1740 1 -- 172.21.15.32:0/1605045376 wait complete. 2024-08-31T07:20:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.657+0000 7f865b7d1740 20 client.27296 trim_cache size 0 max 16384 2024-08-31T07:20:58.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.670+0000 7f865b7d1740 1 Processor -- start 2024-08-31T07:20:58.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f865b7d1740 1 -- start start 2024-08-31T07:20:58.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 0x564e89d96560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d96aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899d9070 0x564e89d83c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f865b7d1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f86501b1450 con 0x564e899b01b0 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f865b7d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86501b3050 con 0x564e899b8390 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f865b7d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86501b1b20 con 0x564e899d9070 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f864d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d96aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f864d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d96aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59316/0 (socket says 172.21.15.32:59316) 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f864d400640 1 -- 172.21.15.32:0/4062295481 learned_addr learned my addr 172.21.15.32:0/4062295481 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.671+0000 7f864e800640 1 --2- 172.21.15.32:0/4062295481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 0x564e89d96560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f864d400640 1 -- 172.21.15.32:0/4062295481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899d9070 msgr2=0x564e89d83c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f864e800640 1 --2- 172.21.15.32:0/4062295481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899d9070 0x564e89d83c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f864d400640 1 --2- 172.21.15.32:0/4062295481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899d9070 0x564e89d83c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f864d400640 1 -- 172.21.15.32:0/4062295481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 msgr2=0x564e89d96560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:58.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f864d400640 1 --2- 172.21.15.32:0/4062295481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 0x564e89d96560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f864d400640 1 -- 172.21.15.32:0/4062295481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e89d8b830 con 0x564e899b8390 2024-08-31T07:20:58.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f864e800640 1 --2- 172.21.15.32:0/4062295481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899d9070 0x564e89d83c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:20:58.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f864d400640 1 --2- 172.21.15.32:0/4062295481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d96aa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f863c000c00 tx=0x7f863c031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cbd4ab3cbf32a2b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:58.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f863a600640 1 -- 172.21.15.32:0/4062295481 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c005450 con 0x564e899b8390 2024-08-31T07:20:58.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f863a600640 1 -- 172.21.15.32:0/4062295481 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f863c041070 con 0x564e899b8390 2024-08-31T07:20:58.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.672+0000 7f863a600640 1 -- 172.21.15.32:0/4062295481 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c0046c0 con 0x564e899b8390 2024-08-31T07:20:58.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.673+0000 7f865b7d1740 1 -- 172.21.15.32:0/4062295481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 msgr2=0x564e89d96aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:58.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.673+0000 7f865b7d1740 1 --2- 172.21.15.32:0/4062295481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d96aa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f863c000c00 tx=0x7f863c031700 comp rx=0 tx=0).stop 2024-08-31T07:20:58.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.673+0000 7f865b7d1740 1 -- 172.21.15.32:0/4062295481 shutdown_connections 2024-08-31T07:20:58.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.673+0000 7f865b7d1740 1 --2- 172.21.15.32:0/4062295481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899d9070 0x564e89d83c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.673+0000 7f865b7d1740 1 --2- 172.21.15.32:0/4062295481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899b8390 0x564e89d96aa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.673+0000 7f865b7d1740 1 --2- 172.21.15.32:0/4062295481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b01b0 0x564e89d96560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.673+0000 7f865b7d1740 1 -- 172.21.15.32:0/4062295481 >> 172.21.15.32:0/4062295481 conn(0x564e899cd460 msgr2=0x564e89d85720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:58.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.673+0000 7f865b7d1740 1 -- 172.21.15.32:0/4062295481 shutdown_connections 2024-08-31T07:20:58.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.673+0000 7f865b7d1740 1 -- 172.21.15.32:0/4062295481 wait complete. 2024-08-31T07:20:58.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.674+0000 7f865b7d1740 1 -- start start 2024-08-31T07:20:58.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.674+0000 7f865b7d1740 1 Processor -- start 2024-08-31T07:20:58.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 0x564e89d8ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b8390 0x564e89d8f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f865b7d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 0x564e89e00230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f865b7d1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f86501b28c0 con 0x564e899b8390 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f865b7d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86501b21f0 con 0x564e899d9070 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f865b7d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86501af450 con 0x564e899b01b0 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f865b7d1740 10 client.0 _ll_register_callbacks cb 0x7f863c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f864d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b8390 0x564e89d8f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f864d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b8390 0x564e89d8f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57416/0 (socket says 172.21.15.32:57416) 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f864d400640 1 -- 172.21.15.32:0/1037653852 learned_addr learned my addr 172.21.15.32:0/1037653852 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f864e800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 0x564e89d8ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.675+0000 7f864e800640 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 0x564e89e00230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.676+0000 7f864d400640 1 -- 172.21.15.32:0/1037653852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 msgr2=0x564e89d8ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.676+0000 7f864d400640 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 0x564e89d8ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.676+0000 7f864d400640 1 -- 172.21.15.32:0/1037653852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 msgr2=0x564e89e00230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:20:58.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.676+0000 7f864d400640 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 0x564e89e00230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.676+0000 7f864d400640 1 -- 172.21.15.32:0/1037653852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e89e008a0 con 0x564e899b8390 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.676+0000 7f864d400640 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b8390 0x564e89d8f410 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f863c02a9b0 tx=0x7f863c031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e594953ce554d69d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.676+0000 7f862f400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.676+0000 7f862f400640 1 -- 172.21.15.32:0/1037653852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c035040 con 0x564e899b8390 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.676+0000 7f862f400640 1 -- 172.21.15.32:0/1037653852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8441+0+0 (secure 0 0 0) 0x7f863c005450 con 0x564e899b8390 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.676+0000 7f865b7d1740 1 -- 172.21.15.32:0/1037653852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f863c041070 con 0x564e899b8390 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f862f400640 1 -- 172.21.15.32:0/1037653852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f863c069050 con 0x564e899b8390 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f865b7d1740 10 client.17289 fetch_fsmap learned FSMap version 35 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f865b7d1740 1 -- 172.21.15.32:0/1037653852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564e89e07b10 con 0x564e899b8390 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f862f400640 1 -- 172.21.15.32:0/1037653852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f863c041070 con 0x564e899b8390 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f862f400640 1 -- 172.21.15.32:0/1037653852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f863c062060 con 0x564e899b8390 2024-08-31T07:20:58.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f865b7d1740 10 client.17289 fetch_fsmap finished waiting for FSMap version 35 2024-08-31T07:20:58.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f865b7d1740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:20:58.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f865b7d1740 20 client.17289 populate_metadata read hostname 'smithi032' 2024-08-31T07:20:58.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564e8996cda0 0x564e89e0a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:20:58.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f865b7d1740 4 client.17289 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:20:58.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.677+0000 7f865b7d1740 1 -- 172.21.15.32:0/1037653852 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e89cf31a0 con 0x564e8996cda0 2024-08-31T07:20:58.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.678+0000 7f864e800640 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564e8996cda0 0x564e89e0a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:20:58.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.678+0000 7f864e800640 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564e8996cda0 0x564e89e0a2e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5839894af24e9d server_cookie=c7a7c6f29150f6e2 in_seq=0 out_seq=0 2024-08-31T07:20:58.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.678+0000 7f862f400640 10 client.17289 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:58.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.678+0000 7f862f400640 1 -- 172.21.15.32:0/1037653852 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e89cf31a0 con 0x564e8996cda0 2024-08-31T07:20:58.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.678+0000 7f862f400640 10 client.17289 handle_command_reply: tid=0 2024-08-31T07:20:58.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.678+0000 7f865b7d1740 1 client.17289 shutdown 2024-08-31T07:20:58.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 1 -- 172.21.15.32:0/1037653852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b8390 msgr2=0x564e89d8f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b8390 0x564e89d8f410 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f863c02a9b0 tx=0x7f863c031b50 comp rx=0 tx=0).stop 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 1 -- 172.21.15.32:0/1037653852 shutdown_connections 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564e8996cda0 0x564e89e0a2e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f862f400640 0 client.17289 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e899d9070 0x564e89e00230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564e899b8390 0x564e89d8f410 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 1 --2- 172.21.15.32:0/1037653852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e899b01b0 0x564e89d8ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 1 -- 172.21.15.32:0/1037653852 >> 172.21.15.32:0/1037653852 conn(0x564e899cd460 msgr2=0x564e89d85720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 1 -- 172.21.15.32:0/1037653852 shutdown_connections 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 1 -- 172.21.15.32:0/1037653852 wait complete. 2024-08-31T07:20:58.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.679+0000 7f865b7d1740 20 client.17289 trim_cache size 0 max 16384 2024-08-31T07:20:58.684 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:20:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.680+0000 7f864f200640 1 -- 172.21.15.32:0/1931075974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86340b4d20 msgr2=0x7f86340b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.680+0000 7f864f200640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86340b4d20 0x7f86340b7230 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86400e2fd0 tx=0x7f8640000bc0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.680+0000 7f864f200640 1 -- 172.21.15.32:0/1931075974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8650103750 msgr2=0x7f865019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:20:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.680+0000 7f864f200640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8650103750 0x7f865019c2b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f864000e9f0 tx=0x7f864000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.680+0000 7f864f200640 1 -- 172.21.15.32:0/1931075974 shutdown_connections 2024-08-31T07:20:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.680+0000 7f864f200640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f86340b4d20 0x7f86340b7230 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.680+0000 7f864f200640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86501099e0 0x7f86501a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.680+0000 7f864f200640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8650109090 0x7f865019c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.680+0000 7f864f200640 1 --2- 172.21.15.32:0/1931075974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8650103750 0x7f865019c2b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:20:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.680+0000 7f864f200640 1 -- 172.21.15.32:0/1931075974 >> 172.21.15.32:0/1931075974 conn(0x7f86500ff1e0 msgr2=0x7f8650108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:20:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.681+0000 7f864f200640 1 -- 172.21.15.32:0/1931075974 shutdown_connections 2024-08-31T07:20:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:20:58.681+0000 7f864f200640 1 -- 172.21.15.32:0/1931075974 wait complete. 2024-08-31T07:20:58.747 INFO:tasks.workunit.client.0.smithi032.stdout: 1 245087 2.66 MB/sec execute 287 sec latency 13.273 ms 2024-08-31T07:20:58.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:20:58 smithi032 ceph-mon[17252]: pgmap v618: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 494 KiB/s wr, 66 op/s 2024-08-31T07:20:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:20:58 smithi084 ceph-mon[22387]: pgmap v618: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 494 KiB/s wr, 66 op/s 2024-08-31T07:20:59.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:20:58 smithi073 ceph-mon[21995]: pgmap v618: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 494 KiB/s wr, 66 op/s 2024-08-31T07:20:59.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 246305 2.67 MB/sec execute 288 sec latency 36.040 ms 2024-08-31T07:21:00.652 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:21:00.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:21:00.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 247182 2.67 MB/sec execute 289 sec latency 235.075 ms 2024-08-31T07:21:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:00 smithi084 ceph-mon[22387]: pgmap v619: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 520 KiB/s wr, 63 op/s 2024-08-31T07:21:01.073 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:01.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:00 smithi032 ceph-mon[17252]: pgmap v619: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 520 KiB/s wr, 63 op/s 2024-08-31T07:21:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:00 smithi073 ceph-mon[21995]: pgmap v619: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 520 KiB/s wr, 63 op/s 2024-08-31T07:21:01.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 248712 2.70 MB/sec execute 290 sec latency 24.957 ms 2024-08-31T07:21:01.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.824+0000 7f529cc00640 1 -- 172.21.15.32:0/1110890609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5298105960 msgr2=0x7f5298105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:01.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.824+0000 7f529cc00640 1 --2- 172.21.15.32:0/1110890609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5298105960 0x7f5298105d70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5288009900 tx=0x7f528802a410 comp rx=0 tx=0).stop 2024-08-31T07:21:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.824+0000 7f529cc00640 1 -- 172.21.15.32:0/1110890609 shutdown_connections 2024-08-31T07:21:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.824+0000 7f529cc00640 1 --2- 172.21.15.32:0/1110890609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529810bbf0 0x7f5298112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.824+0000 7f529cc00640 1 --2- 172.21.15.32:0/1110890609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529810b2a0 0x7f529810b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.824+0000 7f529cc00640 1 --2- 172.21.15.32:0/1110890609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5298105960 0x7f5298105d70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.824+0000 7f529cc00640 1 -- 172.21.15.32:0/1110890609 >> 172.21.15.32:0/1110890609 conn(0x7f52981013d0 msgr2=0x7f5298103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.824+0000 7f529cc00640 1 -- 172.21.15.32:0/1110890609 shutdown_connections 2024-08-31T07:21:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.824+0000 7f529cc00640 1 -- 172.21.15.32:0/1110890609 wait complete. 2024-08-31T07:21:01.826 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:01 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.825+0000 7f529cc00640 1 Processor -- start 2024-08-31T07:21:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f529cc00640 1 -- start start 2024-08-31T07:21:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f529cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5298105960 0x7f529819c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f529cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529810b2a0 0x7f529819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f529cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f529810bbf0 0x7f52981a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f529cc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52981158b0 con 0x7f5298105960 2024-08-31T07:21:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f529cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5298115730 con 0x7f529810bbf0 2024-08-31T07:21:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f529cc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5298115a30 con 0x7f529810b2a0 2024-08-31T07:21:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f5297e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f529810bbf0 0x7f52981a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f5297400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5298105960 0x7f529819c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f5296a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529810b2a0 0x7f529819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f5297e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f529810bbf0 0x7f52981a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59340/0 (socket says 172.21.15.32:59340) 2024-08-31T07:21:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.826+0000 7f5297e00640 1 -- 172.21.15.32:0/2582997310 learned_addr learned my addr 172.21.15.32:0/2582997310 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.827+0000 7f5297e00640 1 -- 172.21.15.32:0/2582997310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529810b2a0 msgr2=0x7f529819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.827+0000 7f5297e00640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529810b2a0 0x7f529819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.827+0000 7f5297e00640 1 -- 172.21.15.32:0/2582997310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5298105960 msgr2=0x7f529819c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.827+0000 7f5297e00640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5298105960 0x7f529819c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.827+0000 7f5297e00640 1 -- 172.21.15.32:0/2582997310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52981a21b0 con 0x7f529810bbf0 2024-08-31T07:21:01.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.827+0000 7f5297e00640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f529810bbf0 0x7f52981a1a80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f529000eb10 tx=0x7f529000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:01.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.828+0000 7f528fe00640 1 -- 172.21.15.32:0/2582997310 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290004b20 con 0x7f529810bbf0 2024-08-31T07:21:01.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.828+0000 7f529cc00640 1 -- 172.21.15.32:0/2582997310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52981a24a0 con 0x7f529810bbf0 2024-08-31T07:21:01.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.828+0000 7f528fe00640 1 -- 172.21.15.32:0/2582997310 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5290004cc0 con 0x7f529810bbf0 2024-08-31T07:21:01.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.828+0000 7f528fe00640 1 -- 172.21.15.32:0/2582997310 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290010670 con 0x7f529810bbf0 2024-08-31T07:21:01.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.828+0000 7f529cc00640 1 -- 172.21.15.32:0/2582997310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52981a8c00 con 0x7f529810bbf0 2024-08-31T07:21:01.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.831+0000 7f52a4e29740 1 Processor -- start 2024-08-31T07:21:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.831+0000 7f528fe00640 1 -- 172.21.15.32:0/2582997310 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5290005020 con 0x7f529810bbf0 2024-08-31T07:21:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.832+0000 7f52a4e29740 1 -- start start 2024-08-31T07:21:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.832+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.832+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d57861d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.832+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d57862250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.832+0000 7f52a4e29740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562d577d7750 con 0x562d57492490 2024-08-31T07:21:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.832+0000 7f52a4e29740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562d577c6ce0 con 0x562d574bb350 2024-08-31T07:21:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.832+0000 7f52a4e29740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562d577cac90 con 0x562d5749a670 2024-08-31T07:21:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.833+0000 7f5296a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.833+0000 7f5296a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59352/0 (socket says 172.21.15.32:59352) 2024-08-31T07:21:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.833+0000 7f5296a00640 1 -- 172.21.15.32:0/3651473060 learned_addr learned my addr 172.21.15.32:0/3651473060 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.833+0000 7f5296a00640 1 --2- 172.21.15.32:0/3651473060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d57862250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.833+0000 7f5296a00640 1 -- 172.21.15.32:0/3651473060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 msgr2=0x562d57862250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.833+0000 7f5296a00640 1 --2- 172.21.15.32:0/3651473060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d57862250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.833+0000 7f5296a00640 1 -- 172.21.15.32:0/3651473060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 msgr2=0x562d57861d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.833+0000 7f5296a00640 1 --2- 172.21.15.32:0/3651473060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d57861d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.833+0000 7f5296a00640 1 -- 172.21.15.32:0/3651473060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d57526930 con 0x562d574bb350 2024-08-31T07:21:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.833+0000 7f5296a00640 1 --2- 172.21.15.32:0/3651473060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578617d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5284011710 tx=0x7f5284005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d3ae1d0f4a747a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.834+0000 7f528fe00640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f527c0b4ca0 0x7f527c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.834+0000 7f5283e00640 1 -- 172.21.15.32:0/3651473060 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284005020 con 0x562d574bb350 2024-08-31T07:21:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.834+0000 7f5283e00640 1 -- 172.21.15.32:0/3651473060 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f528400f840 con 0x562d574bb350 2024-08-31T07:21:01.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.834+0000 7f5283e00640 1 -- 172.21.15.32:0/3651473060 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284004a00 con 0x562d574bb350 2024-08-31T07:21:01.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.834+0000 7f528fe00640 1 -- 172.21.15.32:0/2582997310 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8485+0+0 (secure 0 0 0) 0x7f52900e5b70 con 0x7f529810bbf0 2024-08-31T07:21:01.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.834+0000 7f5297400640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f527c0b4ca0 0x7f527c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.835+0000 7f52a4e29740 1 -- 172.21.15.32:0/3651473060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 msgr2=0x562d578617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.835+0000 7f5297400640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f527c0b4ca0 0x7f527c0b71b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5288031440 tx=0x7f52880095d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.835+0000 7f52a4e29740 1 --2- 172.21.15.32:0/3651473060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578617d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5284011710 tx=0x7f5284005850 comp rx=0 tx=0).stop 2024-08-31T07:21:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.835+0000 7f52a4e29740 1 -- 172.21.15.32:0/3651473060 shutdown_connections 2024-08-31T07:21:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.835+0000 7f52a4e29740 1 --2- 172.21.15.32:0/3651473060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d57862250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.835+0000 7f52a4e29740 1 --2- 172.21.15.32:0/3651473060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d57861d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.835+0000 7f52a4e29740 1 --2- 172.21.15.32:0/3651473060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578617d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.835+0000 7f52a4e29740 1 -- 172.21.15.32:0/3651473060 >> 172.21.15.32:0/3651473060 conn(0x562d574af740 msgr2=0x562d5785f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.836+0000 7f52a4e29740 1 -- 172.21.15.32:0/3651473060 shutdown_connections 2024-08-31T07:21:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.836+0000 7f52a4e29740 1 -- 172.21.15.32:0/3651473060 wait complete. 2024-08-31T07:21:01.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.836+0000 7f52a4e29740 1 -- start start 2024-08-31T07:21:01.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.836+0000 7f52a4e29740 1 Processor -- start 2024-08-31T07:21:01.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d5786d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d5786d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578792a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f52a4e29740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562d57629890 con 0x562d57492490 2024-08-31T07:21:01.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f52a4e29740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562d5774dcd0 con 0x562d574bb350 2024-08-31T07:21:01.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f52a4e29740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562d5786ffb0 con 0x562d5749a670 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f52a4e29740 10 client.0 _ll_register_callbacks cb 0x562d577c1ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f5296a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d5786d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f5296a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d5786d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:49644/0 (socket says 172.21.15.32:49644) 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f5296a00640 1 -- 172.21.15.32:0/1500117498 learned_addr learned my addr 172.21.15.32:0/1500117498 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.837+0000 7f5297400640 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d5786d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.838+0000 7f5297e00640 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.838+0000 7f5296a00640 1 -- 172.21.15.32:0/1500117498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 msgr2=0x562d578792a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.838+0000 7f5296a00640 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578792a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.838+0000 7f5296a00640 1 -- 172.21.15.32:0/1500117498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 msgr2=0x562d5786d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.838+0000 7f5296a00640 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d5786d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.838+0000 7f5296a00640 1 -- 172.21.15.32:0/1500117498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d577971b0 con 0x562d5749a670 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.838+0000 7f5297e00640 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578792a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:21:01.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.838+0000 7f5296a00640 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d5786d9e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f52840058c0 tx=0x7f528400fef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9235014766ddc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.838+0000 7f5280c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.839+0000 7f5280c00640 1 -- 172.21.15.32:0/1500117498 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528402d060 con 0x562d5749a670 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.839+0000 7f5280c00640 1 -- 172.21.15.32:0/1500117498 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8485+0+0 (secure 0 0 0) 0x7f52840052f0 con 0x562d5749a670 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.839+0000 7f52a4e29740 1 -- 172.21.15.32:0/1500117498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f528400f840 con 0x562d5749a670 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.839+0000 7f5280c00640 1 -- 172.21.15.32:0/1500117498 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5284005530 con 0x562d5749a670 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.839+0000 7f52a4e29740 10 client.27316 fetch_fsmap learned FSMap version 36 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.839+0000 7f52a4e29740 1 -- 172.21.15.32:0/1500117498 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562d5787c050 con 0x562d5749a670 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.839+0000 7f5280c00640 1 -- 172.21.15.32:0/1500117498 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f528400f840 con 0x562d5749a670 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.839+0000 7f5280c00640 1 -- 172.21.15.32:0/1500117498 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 36) ==== 8382+0+0 (secure 0 0 0) 0x7f5284027030 con 0x562d5749a670 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.839+0000 7f52a4e29740 10 client.27316 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.840+0000 7f52a4e29740 10 client.27316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.840+0000 7f52a4e29740 20 client.27316 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.840+0000 7f52a4e29740 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d57479cd0 0x562d5787f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.840+0000 7f52a4e29740 4 client.27316 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:21:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.840+0000 7f52a4e29740 1 -- 172.21.15.32:0/1500117498 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d577d96a0 con 0x562d57479cd0 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.840+0000 7f5297400640 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d57479cd0 0x562d5787f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.840+0000 7f5297400640 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d57479cd0 0x562d5787f3b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5371e8456900aff server_cookie=98aec67c12193c00 in_seq=0 out_seq=0 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.840+0000 7f5280c00640 10 client.27316 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f5280c00640 1 -- 172.21.15.32:0/1500117498 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d577d96a0 con 0x562d57479cd0 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f5280c00640 10 client.27316 handle_command_reply: tid=0 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f52a4e29740 1 client.27316 shutdown 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f52a4e29740 1 -- 172.21.15.32:0/1500117498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 msgr2=0x562d5786d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f52a4e29740 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d5786d9e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f52840058c0 tx=0x7f528400fef0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f52a4e29740 1 -- 172.21.15.32:0/1500117498 shutdown_connections 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f52a4e29740 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d57479cd0 0x562d5787f3b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f52a4e29740 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578792a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f52a4e29740 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d5786d9e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f52a4e29740 1 --2- 172.21.15.32:0/1500117498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d5786d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.841+0000 7f5280c00640 0 client.27316 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.842+0000 7f52a4e29740 1 -- 172.21.15.32:0/1500117498 >> 172.21.15.32:0/1500117498 conn(0x562d574af740 msgr2=0x562d578649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.842+0000 7f52a4e29740 1 -- 172.21.15.32:0/1500117498 shutdown_connections 2024-08-31T07:21:01.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.842+0000 7f52a4e29740 1 -- 172.21.15.32:0/1500117498 wait complete. 2024-08-31T07:21:01.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.842+0000 7f52a4e29740 20 client.27316 trim_cache size 0 max 16384 2024-08-31T07:21:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.861+0000 7f52a4e29740 1 Processor -- start 2024-08-31T07:21:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.861+0000 7f52a4e29740 1 -- start start 2024-08-31T07:21:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d5785f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f52a4e29740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52981b1430 con 0x562d57492490 2024-08-31T07:21:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f52a4e29740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f52981b3060 con 0x562d574bb350 2024-08-31T07:21:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f52a4e29740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f52981b1b00 con 0x562d5749a670 2024-08-31T07:21:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f5296a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f5297400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d5785f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f5296a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:49648/0 (socket says 172.21.15.32:49648) 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f5296a00640 1 -- 172.21.15.32:0/3401244570 learned_addr learned my addr 172.21.15.32:0/3401244570 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f5297e00640 1 --2- 172.21.15.32:0/3401244570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f5296a00640 1 -- 172.21.15.32:0/3401244570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 msgr2=0x562d578648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f5296a00640 1 --2- 172.21.15.32:0/3401244570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f5296a00640 1 -- 172.21.15.32:0/3401244570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 msgr2=0x562d5785f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.862+0000 7f5296a00640 1 --2- 172.21.15.32:0/3401244570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d5785f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.863+0000 7f5296a00640 1 -- 172.21.15.32:0/3401244570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d5786dae0 con 0x562d5749a670 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.863+0000 7f5296a00640 1 --2- 172.21.15.32:0/3401244570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578791a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5284004f90 tx=0x7f5284009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9c825ab5f2515f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.863+0000 7f5283400640 1 -- 172.21.15.32:0/3401244570 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528402d040 con 0x562d5749a670 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.863+0000 7f5283400640 1 -- 172.21.15.32:0/3401244570 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f528400f790 con 0x562d5749a670 2024-08-31T07:21:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.863+0000 7f5283400640 1 -- 172.21.15.32:0/3401244570 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284003d90 con 0x562d5749a670 2024-08-31T07:21:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.864+0000 7f52a4e29740 1 -- 172.21.15.32:0/3401244570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 msgr2=0x562d578791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.864+0000 7f52a4e29740 1 --2- 172.21.15.32:0/3401244570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578791a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5284004f90 tx=0x7f5284009640 comp rx=0 tx=0).stop 2024-08-31T07:21:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.864+0000 7f52a4e29740 1 -- 172.21.15.32:0/3401244570 shutdown_connections 2024-08-31T07:21:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.864+0000 7f52a4e29740 1 --2- 172.21.15.32:0/3401244570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d574bb350 0x562d578648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.864+0000 7f52a4e29740 1 --2- 172.21.15.32:0/3401244570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578791a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.864+0000 7f52a4e29740 1 --2- 172.21.15.32:0/3401244570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d57492490 0x562d5785f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.864+0000 7f52a4e29740 1 -- 172.21.15.32:0/3401244570 >> 172.21.15.32:0/3401244570 conn(0x562d574af740 msgr2=0x562d5785f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.864+0000 7f52a4e29740 1 -- 172.21.15.32:0/3401244570 shutdown_connections 2024-08-31T07:21:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.864+0000 7f52a4e29740 1 -- 172.21.15.32:0/3401244570 wait complete. 2024-08-31T07:21:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.865+0000 7f52a4e29740 1 -- start start 2024-08-31T07:21:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.865+0000 7f52a4e29740 1 Processor -- start 2024-08-31T07:21:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d57492490 0x562d57875810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f52a4e29740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d574bb350 0x562d57870700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f52a4e29740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52981b28d0 con 0x562d574bb350 2024-08-31T07:21:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f52a4e29740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f52981b2200 con 0x562d57492490 2024-08-31T07:21:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f52a4e29740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f52981af430 con 0x562d5749a670 2024-08-31T07:21:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f52a4e29740 10 client.0 _ll_register_callbacks cb 0x7f5284009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5296a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5297400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d57492490 0x562d57875810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5296a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:49650/0 (socket says 172.21.15.32:49650) 2024-08-31T07:21:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5296a00640 1 -- 172.21.15.32:0/4106292137 learned_addr learned my addr 172.21.15.32:0/4106292137 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5297400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d57492490 0x562d57875810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59384/0 (socket says 172.21.15.32:59384) 2024-08-31T07:21:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5297e00640 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d574bb350 0x562d57870700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5297400640 1 -- 172.21.15.32:0/4106292137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 msgr2=0x562d578701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5297400640 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5297400640 1 -- 172.21.15.32:0/4106292137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d574bb350 msgr2=0x562d57870700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5297400640 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d574bb350 0x562d57870700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5297400640 1 -- 172.21.15.32:0/4106292137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d57870cc0 con 0x562d57492490 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5296a00640 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.866+0000 7f5297e00640 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d574bb350 0x562d57870700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.867+0000 7f5297400640 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d57492490 0x562d57875810 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5288009230 tx=0x7f528803d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b26a8c6918a9158d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.867+0000 7f5280c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.867+0000 7f5280c00640 1 -- 172.21.15.32:0/4106292137 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288035040 con 0x562d57492490 2024-08-31T07:21:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.867+0000 7f5280c00640 1 -- 172.21.15.32:0/4106292137 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8485+0+0 (secure 0 0 0) 0x7f5288004040 con 0x562d57492490 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.867+0000 7f52a4e29740 1 -- 172.21.15.32:0/4106292137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f528400f790 con 0x562d57492490 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.867+0000 7f5280c00640 1 -- 172.21.15.32:0/4106292137 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5288004240 con 0x562d57492490 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.867+0000 7f52a4e29740 10 client.27317 fetch_fsmap learned FSMap version 36 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.867+0000 7f52a4e29740 1 -- 172.21.15.32:0/4106292137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562d578eaa90 con 0x562d57492490 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.868+0000 7f5280c00640 1 -- 172.21.15.32:0/4106292137 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f528400f790 con 0x562d57492490 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.868+0000 7f5280c00640 1 -- 172.21.15.32:0/4106292137 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8382+0+0 (secure 0 0 0) 0x7f5288004950 con 0x562d57492490 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.868+0000 7f52a4e29740 10 client.27317 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.868+0000 7f52a4e29740 10 client.27317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.868+0000 7f52a4e29740 20 client.27317 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.868+0000 7f52a4e29740 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d5744f080 0x562d578ed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.868+0000 7f52a4e29740 4 client.27317 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.868+0000 7f52a4e29740 1 -- 172.21.15.32:0/4106292137 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562d577d5140 con 0x562d5744f080 2024-08-31T07:21:01.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.868+0000 7f5296a00640 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d5744f080 0x562d578ed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.869+0000 7f5296a00640 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d5744f080 0x562d578ed2f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3a343fef900f0c2 server_cookie=420698ff76340108 in_seq=0 out_seq=0 2024-08-31T07:21:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.869+0000 7f5280c00640 10 client.27317 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f5280c00640 1 -- 172.21.15.32:0/4106292137 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562d577d5140 con 0x562d5744f080 2024-08-31T07:21:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f5280c00640 10 client.27317 handle_command_reply: tid=0 2024-08-31T07:21:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f52a4e29740 1 client.27317 shutdown 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f52a4e29740 1 -- 172.21.15.32:0/4106292137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d57492490 msgr2=0x562d57875810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f52a4e29740 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d57492490 0x562d57875810 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5288009230 tx=0x7f528803d000 comp rx=0 tx=0).stop 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f52a4e29740 1 -- 172.21.15.32:0/4106292137 shutdown_connections 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f52a4e29740 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d5744f080 0x562d578ed2f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f52a4e29740 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d574bb350 0x562d57870700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f52a4e29740 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d5749a670 0x562d578701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f5280c00640 0 client.27317 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f52a4e29740 1 --2- 172.21.15.32:0/4106292137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d57492490 0x562d57875810 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.870+0000 7f52a4e29740 1 -- 172.21.15.32:0/4106292137 >> 172.21.15.32:0/4106292137 conn(0x562d574af740 msgr2=0x562d5785efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.871+0000 7f52a4e29740 1 -- 172.21.15.32:0/4106292137 shutdown_connections 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.871+0000 7f52a4e29740 1 -- 172.21.15.32:0/4106292137 wait complete. 2024-08-31T07:21:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.871+0000 7f52a4e29740 20 client.27317 trim_cache size 0 max 16384 2024-08-31T07:21:01.875 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:21:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:21:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "4ce84d20-d56a-4c0e-b303-5ada6c73ea15", 2024-08-31T07:21:01.875 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:21:01.875 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:21:01.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 -- 172.21.15.32:0/2582997310 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f527c0b4ca0 msgr2=0x7f527c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f527c0b4ca0 0x7f527c0b71b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5288031440 tx=0x7f52880095d0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 -- 172.21.15.32:0/2582997310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f529810bbf0 msgr2=0x7f52981a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f529810bbf0 0x7f52981a1a80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f529000eb10 tx=0x7f529000c2d0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 -- 172.21.15.32:0/2582997310 shutdown_connections 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f527c0b4ca0 0x7f527c0b71b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f529810bbf0 0x7f52981a1a80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f529810b2a0 0x7f529819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 --2- 172.21.15.32:0/2582997310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5298105960 0x7f529819c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 -- 172.21.15.32:0/2582997310 >> 172.21.15.32:0/2582997310 conn(0x7f52981013d0 msgr2=0x7f52981037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 -- 172.21.15.32:0/2582997310 shutdown_connections 2024-08-31T07:21:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:01.872+0000 7f529cc00640 1 -- 172.21.15.32:0/2582997310 wait complete. 2024-08-31T07:21:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:01 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:01 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:02.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 249801 2.71 MB/sec execute 291 sec latency 107.589 ms 2024-08-31T07:21:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: pgmap v620: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 946 KiB/s wr, 132 op/s 2024-08-31T07:21:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: osdmap e85: 12 total, 12 up, 12 in 2024-08-31T07:21:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:02.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:02.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:02.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:02.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:02.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:02.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:02.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:02.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:02 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: pgmap v620: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 946 KiB/s wr, 132 op/s 2024-08-31T07:21:03.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: osdmap e85: 12 total, 12 up, 12 in 2024-08-31T07:21:03.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:21:03.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:03.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:03.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:03.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:03.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:03.058 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.058 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.058 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:02 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:21:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: pgmap v620: 97 pgs: 97 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 946 KiB/s wr, 132 op/s 2024-08-31T07:21:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: osdmap e85: 12 total, 12 up, 12 in 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:03.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:03.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:03.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.181 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:02 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:03.748 INFO:tasks.workunit.client.0.smithi032.stdout: 1 250021 2.70 MB/sec execute 292 sec latency 815.150 ms 2024-08-31T07:21:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.266+0000 7fc746400640 1 -- 172.21.15.32:0/4278299289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc740075b70 msgr2=0x7fc740075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.266+0000 7fc746400640 1 --2- 172.21.15.32:0/4278299289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc740075b70 0x7fc740075f80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009900 tx=0x7fc72c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.266+0000 7fc746400640 1 -- 172.21.15.32:0/4278299289 shutdown_connections 2024-08-31T07:21:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.266+0000 7fc746400640 1 --2- 172.21.15.32:0/4278299289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc740114020 0x7fc7401164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.266+0000 7fc746400640 1 --2- 172.21.15.32:0/4278299289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc740075b70 0x7fc740075f80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.266+0000 7fc746400640 1 --2- 172.21.15.32:0/4278299289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740077120 0x7fc740075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.266+0000 7fc746400640 1 -- 172.21.15.32:0/4278299289 >> 172.21.15.32:0/4278299289 conn(0x7fc7400ff1e0 msgr2=0x7fc740101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.267+0000 7fc746400640 1 -- 172.21.15.32:0/4278299289 shutdown_connections 2024-08-31T07:21:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.267+0000 7fc746400640 1 -- 172.21.15.32:0/4278299289 wait complete. 2024-08-31T07:21:04.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc746400640 1 Processor -- start 2024-08-31T07:21:04.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc746400640 1 -- start start 2024-08-31T07:21:04.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc746400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc740075b70 0x7fc7401a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc746400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc740077120 0x7fc7401a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc746400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740114020 0x7fc7401a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc746400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc740119c60 con 0x7fc740077120 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc746400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc740119ae0 con 0x7fc740114020 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc746400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc740119de0 con 0x7fc740075b70 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc73fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc740077120 0x7fc7401a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc745a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740114020 0x7fc7401a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc745a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740114020 0x7fc7401a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59410/0 (socket says 172.21.15.32:59410) 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc73fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc740077120 0x7fc7401a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57474/0 (socket says 172.21.15.32:57474) 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc745a00640 1 -- 172.21.15.32:0/2778000126 learned_addr learned my addr 172.21.15.32:0/2778000126 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.268+0000 7fc745000640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc740075b70 0x7fc7401a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.269+0000 7fc745a00640 1 -- 172.21.15.32:0/2778000126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc740075b70 msgr2=0x7fc7401a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.269+0000 7fc745a00640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc740075b70 0x7fc7401a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.269+0000 7fc745a00640 1 -- 172.21.15.32:0/2778000126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc740077120 msgr2=0x7fc7401a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.269+0000 7fc745a00640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc740077120 0x7fc7401a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.269+0000 7fc745a00640 1 -- 172.21.15.32:0/2778000126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7401a6600 con 0x7fc740114020 2024-08-31T07:21:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.269+0000 7fc745a00640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740114020 0x7fc7401a5ed0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc73800eb10 tx=0x7fc73800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.269+0000 7fc73d600640 1 -- 172.21.15.32:0/2778000126 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738004b20 con 0x7fc740114020 2024-08-31T07:21:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.270+0000 7fc73d600640 1 -- 172.21.15.32:0/2778000126 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc738004cc0 con 0x7fc740114020 2024-08-31T07:21:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.270+0000 7fc746400640 1 -- 172.21.15.32:0/2778000126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7401a68f0 con 0x7fc740114020 2024-08-31T07:21:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.270+0000 7fc73d600640 1 -- 172.21.15.32:0/2778000126 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738010670 con 0x7fc740114020 2024-08-31T07:21:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.270+0000 7fc746400640 1 -- 172.21.15.32:0/2778000126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7401ad050 con 0x7fc740114020 2024-08-31T07:21:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.273+0000 7fc74e5ca740 1 Processor -- start 2024-08-31T07:21:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.274+0000 7fc73d600640 1 -- 172.21.15.32:0/2778000126 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc738005020 con 0x7fc740114020 2024-08-31T07:21:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.274+0000 7fc74e5ca740 1 -- start start 2024-08-31T07:21:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.274+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718739b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.274+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b71873a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.274+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b71873a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.274+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b7186b0770 con 0x55b718373390 2024-08-31T07:21:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.274+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b71869fd00 con 0x55b718394070 2024-08-31T07:21:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.274+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b7186a3cb0 con 0x55b71836b1b0 2024-08-31T07:21:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc745000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b71873a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc73fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b71873a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc745000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b71873a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:49668/0 (socket says 172.21.15.32:49668) 2024-08-31T07:21:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc745000640 1 -- 172.21.15.32:0/893734613 learned_addr learned my addr 172.21.15.32:0/893734613 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc73fe00640 1 --2- 172.21.15.32:0/893734613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718739b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc745000640 1 -- 172.21.15.32:0/893734613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 msgr2=0x55b718739b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc745000640 1 --2- 172.21.15.32:0/893734613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718739b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc745000640 1 -- 172.21.15.32:0/893734613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 msgr2=0x55b71873a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc745000640 1 --2- 172.21.15.32:0/893734613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b71873a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc745000640 1 -- 172.21.15.32:0/893734613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7183ff650 con 0x55b71836b1b0 2024-08-31T07:21:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc73fe00640 1 --2- 172.21.15.32:0/893734613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718739b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:21:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc73d600640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7280b4ca0 0x7fc7280b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.275+0000 7fc745000640 1 --2- 172.21.15.32:0/893734613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b71873a060 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc734009cf0 tx=0x7fc734011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c5127185692747 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.276+0000 7fc73d600640 1 -- 172.21.15.32:0/2778000126 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8485+0+0 (secure 0 0 0) 0x7fc7380e5b70 con 0x7fc740114020 2024-08-31T07:21:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.276+0000 7fc73fe00640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7280b4ca0 0x7fc7280b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.276+0000 7fc732000640 1 -- 172.21.15.32:0/893734613 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73401e070 con 0x55b71836b1b0 2024-08-31T07:21:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.276+0000 7fc732000640 1 -- 172.21.15.32:0/893734613 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc734019ae0 con 0x55b71836b1b0 2024-08-31T07:21:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.276+0000 7fc732000640 1 -- 172.21.15.32:0/893734613 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73400e9e0 con 0x55b71836b1b0 2024-08-31T07:21:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc74e5ca740 1 -- 172.21.15.32:0/893734613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 msgr2=0x55b71873a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc73fe00640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7280b4ca0 0x7fc7280b71b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc72c000c00 tx=0x7fc72c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/893734613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b71873a060 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc734009cf0 tx=0x7fc734011810 comp rx=0 tx=0).stop 2024-08-31T07:21:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc74e5ca740 1 -- 172.21.15.32:0/893734613 shutdown_connections 2024-08-31T07:21:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/893734613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b71873a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/893734613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b71873a060 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/893734613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718739b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc74e5ca740 1 -- 172.21.15.32:0/893734613 >> 172.21.15.32:0/893734613 conn(0x55b718388460 msgr2=0x55b718737a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc74e5ca740 1 -- 172.21.15.32:0/893734613 shutdown_connections 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc74e5ca740 1 -- 172.21.15.32:0/893734613 wait complete. 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.277+0000 7fc74e5ca740 1 -- start start 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.278+0000 7fc74e5ca740 1 Processor -- start 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.278+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.278+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b718745c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.278+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718751670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.278+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b718502700 con 0x55b718373390 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.278+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b718626cb0 con 0x55b718394070 2024-08-31T07:21:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.278+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b718748350 con 0x55b71836b1b0 2024-08-31T07:21:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.278+0000 7fc74e5ca740 10 client.0 _ll_register_callbacks cb 0x55b71864c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc745000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc73fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b718745c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc745000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:49674/0 (socket says 172.21.15.32:49674) 2024-08-31T07:21:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc745000640 1 -- 172.21.15.32:0/1185029255 learned_addr learned my addr 172.21.15.32:0/1185029255 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc745a00640 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718751670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc745000640 1 -- 172.21.15.32:0/1185029255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 msgr2=0x55b718751670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc745000640 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718751670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc745000640 1 -- 172.21.15.32:0/1185029255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 msgr2=0x55b718745c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc745000640 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b718745c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc745000640 1 -- 172.21.15.32:0/1185029255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b718667870 con 0x55b71836b1b0 2024-08-31T07:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc745000640 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745710 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc7340097b0 tx=0x7fc734019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14dc290630a2866e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.279+0000 7fc726a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.280+0000 7fc726a00640 1 -- 172.21.15.32:0/1185029255 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73401e050 con 0x55b71836b1b0 2024-08-31T07:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.280+0000 7fc74e5ca740 1 -- 172.21.15.32:0/1185029255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc734019ae0 con 0x55b71836b1b0 2024-08-31T07:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.280+0000 7fc726a00640 1 -- 172.21.15.32:0/1185029255 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8485+0+0 (secure 0 0 0) 0x7fc734005240 con 0x55b71836b1b0 2024-08-31T07:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.280+0000 7fc726a00640 1 -- 172.21.15.32:0/1185029255 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc73400e3b0 con 0x55b71836b1b0 2024-08-31T07:21:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.280+0000 7fc74e5ca740 10 client.27334 fetch_fsmap learned FSMap version 36 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.280+0000 7fc74e5ca740 1 -- 172.21.15.32:0/1185029255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b718754350 con 0x55b71836b1b0 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.280+0000 7fc726a00640 1 -- 172.21.15.32:0/1185029255 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc734019ae0 con 0x55b71836b1b0 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.281+0000 7fc726a00640 1 -- 172.21.15.32:0/1185029255 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 36) ==== 8382+0+0 (secure 0 0 0) 0x7fc7340046c0 con 0x55b71836b1b0 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.281+0000 7fc74e5ca740 10 client.27334 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.281+0000 7fc74e5ca740 10 client.27334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.281+0000 7fc74e5ca740 20 client.27334 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.281+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b7183529f0 0x55b718757710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.281+0000 7fc74e5ca740 4 client.27334 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.281+0000 7fc74e5ca740 1 -- 172.21.15.32:0/1185029255 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7186b26c0 con 0x55b7183529f0 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.281+0000 7fc73fe00640 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b7183529f0 0x55b718757710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.282+0000 7fc73fe00640 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b7183529f0 0x55b718757710 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc7fd2c7e5eab2a server_cookie=97ff0f2cf480d176 in_seq=0 out_seq=0 2024-08-31T07:21:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.282+0000 7fc726a00640 10 client.27334 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.282+0000 7fc726a00640 1 -- 172.21.15.32:0/1185029255 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7186b26c0 con 0x55b7183529f0 2024-08-31T07:21:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.282+0000 7fc726a00640 10 client.27334 handle_command_reply: tid=0 2024-08-31T07:21:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.282+0000 7fc74e5ca740 1 client.27334 shutdown 2024-08-31T07:21:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 1 -- 172.21.15.32:0/1185029255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 msgr2=0x55b718745710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745710 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc7340097b0 tx=0x7fc734019660 comp rx=0 tx=0).stop 2024-08-31T07:21:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 1 -- 172.21.15.32:0/1185029255 shutdown_connections 2024-08-31T07:21:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b7183529f0 0x55b718757710 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718751670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b718745c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/1185029255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745710 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc726a00640 0 client.27334 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 1 -- 172.21.15.32:0/1185029255 >> 172.21.15.32:0/1185029255 conn(0x55b718388460 msgr2=0x55b71873ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 1 -- 172.21.15.32:0/1185029255 shutdown_connections 2024-08-31T07:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 1 -- 172.21.15.32:0/1185029255 wait complete. 2024-08-31T07:21:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.283+0000 7fc74e5ca740 20 client.27334 trim_cache size 0 max 16384 2024-08-31T07:21:04.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.302+0000 7fc74e5ca740 1 Processor -- start 2024-08-31T07:21:04.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.302+0000 7fc74e5ca740 1 -- start start 2024-08-31T07:21:04.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.302+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.302+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b71873f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.302+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718751570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.302+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401b58b0 con 0x55b718373390 2024-08-31T07:21:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.302+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401b74e0 con 0x55b718394070 2024-08-31T07:21:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.302+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401b5f80 con 0x55b71836b1b0 2024-08-31T07:21:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc73fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b71873f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718751570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718751570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59444/0 (socket says 172.21.15.32:59444) 2024-08-31T07:21:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745a00640 1 -- 172.21.15.32:0/3271915858 learned_addr learned my addr 172.21.15.32:0/3271915858 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745000640 1 --2- 172.21.15.32:0/3271915858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745a00640 1 -- 172.21.15.32:0/3271915858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 msgr2=0x55b718745d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:04.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745a00640 1 --2- 172.21.15.32:0/3271915858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745a00640 1 -- 172.21.15.32:0/3271915858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 msgr2=0x55b71873f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:04.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745a00640 1 --2- 172.21.15.32:0/3271915858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b71873f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745a00640 1 -- 172.21.15.32:0/3271915858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b71873c980 con 0x55b718394070 2024-08-31T07:21:04.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745000640 1 --2- 172.21.15.32:0/3271915858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:04.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.303+0000 7fc745a00640 1 --2- 172.21.15.32:0/3271915858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718751570 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc7380a82a0 tx=0x7fc73800cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=149eb1732aafc9f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc731600640 1 -- 172.21.15.32:0/3271915858 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7380ac220 con 0x55b718394070 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc731600640 1 -- 172.21.15.32:0/3271915858 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7380b0a90 con 0x55b718394070 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc731600640 1 -- 172.21.15.32:0/3271915858 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7380b50b0 con 0x55b718394070 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc74e5ca740 1 -- 172.21.15.32:0/3271915858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 msgr2=0x55b718751570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/3271915858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718751570 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc7380a82a0 tx=0x7fc73800cee0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc74e5ca740 1 -- 172.21.15.32:0/3271915858 shutdown_connections 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/3271915858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b718751570 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/3271915858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b718373390 0x55b71873f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/3271915858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b71836b1b0 0x55b718745d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc74e5ca740 1 -- 172.21.15.32:0/3271915858 >> 172.21.15.32:0/3271915858 conn(0x55b718388460 msgr2=0x55b71873ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:04.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.304+0000 7fc74e5ca740 1 -- 172.21.15.32:0/3271915858 shutdown_connections 2024-08-31T07:21:04.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.305+0000 7fc74e5ca740 1 -- 172.21.15.32:0/3271915858 wait complete. 2024-08-31T07:21:04.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.305+0000 7fc74e5ca740 1 -- start start 2024-08-31T07:21:04.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.305+0000 7fc74e5ca740 1 Processor -- start 2024-08-31T07:21:04.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b71836b1b0 0x55b71874ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b718373390 0x55b718749280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc74e5ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b7187497e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401b6d50 con 0x55b71836b1b0 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401b6680 con 0x55b718394070 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc74e5ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401b38b0 con 0x55b718373390 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc74e5ca740 10 client.0 _ll_register_callbacks cb 0x7fc734005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc745000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b71836b1b0 0x55b71874ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc745000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b71836b1b0 0x55b71874ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57518/0 (socket says 172.21.15.32:57518) 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc745000640 1 -- 172.21.15.32:0/2172962943 learned_addr learned my addr 172.21.15.32:0/2172962943 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc73fe00640 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b718373390 0x55b718749280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc745a00640 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b7187497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc745000640 1 -- 172.21.15.32:0/2172962943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b718373390 msgr2=0x55b718749280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc745000640 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b718373390 0x55b718749280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc745000640 1 -- 172.21.15.32:0/2172962943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 msgr2=0x55b7187497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc745000640 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b7187497e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.306+0000 7fc745000640 1 -- 172.21.15.32:0/2172962943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b718749f70 con 0x55b71836b1b0 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.307+0000 7fc745000640 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b71836b1b0 0x55b71874ddd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc734004b60 tx=0x7fc734009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ae7a8ffb2824ec server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.307+0000 7fc726a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.307+0000 7fc726a00640 1 -- 172.21.15.32:0/2172962943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73401e040 con 0x55b71836b1b0 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.307+0000 7fc726a00640 1 -- 172.21.15.32:0/2172962943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8485+0+0 (secure 0 0 0) 0x7fc73400e8f0 con 0x55b71836b1b0 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.307+0000 7fc74e5ca740 1 -- 172.21.15.32:0/2172962943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7380b0a90 con 0x55b71836b1b0 2024-08-31T07:21:04.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.307+0000 7fc726a00640 1 -- 172.21.15.32:0/2172962943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc73400eb30 con 0x55b71836b1b0 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.307+0000 7fc74e5ca740 10 client.17322 fetch_fsmap learned FSMap version 36 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.307+0000 7fc74e5ca740 1 -- 172.21.15.32:0/2172962943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b7187c1ff0 con 0x55b71836b1b0 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.308+0000 7fc726a00640 1 -- 172.21.15.32:0/2172962943 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7380b0a90 con 0x55b71836b1b0 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.308+0000 7fc726a00640 1 -- 172.21.15.32:0/2172962943 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 36) ==== 8382+0+0 (secure 0 0 0) 0x7fc734004030 con 0x55b71836b1b0 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.308+0000 7fc74e5ca740 10 client.17322 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.308+0000 7fc74e5ca740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.308+0000 7fc74e5ca740 20 client.17322 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.308+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b718327da0 0x55b7187c4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.308+0000 7fc74e5ca740 4 client.17322 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.308+0000 7fc74e5ca740 1 -- 172.21.15.32:0/2172962943 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7186ae160 con 0x55b718327da0 2024-08-31T07:21:04.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.308+0000 7fc73fe00640 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b718327da0 0x55b7187c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.309+0000 7fc73fe00640 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b718327da0 0x55b7187c4810 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13a707df274fb945 server_cookie=f3ed5d62796a3251 in_seq=0 out_seq=0 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.309+0000 7fc726a00640 10 client.17322 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.309+0000 7fc726a00640 1 -- 172.21.15.32:0/2172962943 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x55b7186ae160 con 0x55b718327da0 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.309+0000 7fc726a00640 10 client.17322 handle_command_reply: tid=0 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.309+0000 7fc74e5ca740 1 client.17322 shutdown 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.309+0000 7fc74e5ca740 1 -- 172.21.15.32:0/2172962943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b71836b1b0 msgr2=0x55b71874ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.309+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b71836b1b0 0x55b71874ddd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc734004b60 tx=0x7fc734009910 comp rx=0 tx=0).stop 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.310+0000 7fc74e5ca740 1 -- 172.21.15.32:0/2172962943 shutdown_connections 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.310+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b718327da0 0x55b7187c4810 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.310+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b718394070 0x55b7187497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.310+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b718373390 0x55b718749280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.310+0000 7fc726a00640 0 client.17322 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:04.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.310+0000 7fc74e5ca740 1 --2- 172.21.15.32:0/2172962943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b71836b1b0 0x55b71874ddd0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.310+0000 7fc74e5ca740 1 -- 172.21.15.32:0/2172962943 >> 172.21.15.32:0/2172962943 conn(0x55b718388460 msgr2=0x55b71873ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:04.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.310+0000 7fc74e5ca740 1 -- 172.21.15.32:0/2172962943 shutdown_connections 2024-08-31T07:21:04.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.310+0000 7fc74e5ca740 1 -- 172.21.15.32:0/2172962943 wait complete. 2024-08-31T07:21:04.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.310+0000 7fc74e5ca740 20 client.17322 trim_cache size 0 max 16384 2024-08-31T07:21:04.315 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:21:04.316 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:21:04.316 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:21:04.316 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:21:04.316 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.316 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "4ce84d20-d56a-4c0e-b303-5ada6c73ea15", 2024-08-31T07:21:04.316 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:21:04.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:21:04.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.316 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:21:04.316 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.318 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:21:04.318 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:21:04.318 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:21:04.319 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:21:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:21:04.324 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:21:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 -- 172.21.15.32:0/2778000126 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7280b4ca0 msgr2=0x7fc7280b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7280b4ca0 0x7fc7280b71b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc72c000c00 tx=0x7fc72c031400 comp rx=0 tx=0).stop 2024-08-31T07:21:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 -- 172.21.15.32:0/2778000126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740114020 msgr2=0x7fc7401a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740114020 0x7fc7401a5ed0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc73800eb10 tx=0x7fc73800c2d0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 -- 172.21.15.32:0/2778000126 shutdown_connections 2024-08-31T07:21:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc7280b4ca0 0x7fc7280b71b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740114020 0x7fc7401a5ed0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc740077120 0x7fc7401a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 --2- 172.21.15.32:0/2778000126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc740075b70 0x7fc7401a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 -- 172.21.15.32:0/2778000126 >> 172.21.15.32:0/2778000126 conn(0x7fc7400ff1e0 msgr2=0x7fc74010ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 -- 172.21.15.32:0/2778000126 shutdown_connections 2024-08-31T07:21:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:04.311+0000 7fc746400640 1 -- 172.21.15.32:0/2778000126 wait complete. 2024-08-31T07:21:04.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 250021 2.69 MB/sec execute 293 sec latency 1815.281 ms 2024-08-31T07:21:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:04 smithi084 ceph-mon[22387]: pgmap v622: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-08-31T07:21:05.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:04 smithi032 ceph-mon[17252]: pgmap v622: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-08-31T07:21:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:04 smithi073 ceph-mon[21995]: pgmap v622: 97 pgs: 97 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-08-31T07:21:05.260 INFO:tasks.cephfs.filesystem:scrub status for tag:4ce84d20-d56a-4c0e-b303-5ada6c73ea15 - {'path': '/', 'tag': '4ce84d20-d56a-4c0e-b303-5ada6c73ea15', 'options': 'recursive,force'} 2024-08-31T07:21:05.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 251322 2.71 MB/sec execute 294 sec latency 2187.506 ms 2024-08-31T07:21:06.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 252602 2.72 MB/sec execute 295 sec latency 18.108 ms 2024-08-31T07:21:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:06 smithi084 ceph-mon[22387]: pgmap v623: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-08-31T07:21:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:06 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:07.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:06 smithi032 ceph-mon[17252]: pgmap v623: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-08-31T07:21:07.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:06 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:07.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:06 smithi073 ceph-mon[21995]: pgmap v623: 97 pgs: 97 active+clean; 170 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-08-31T07:21:07.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:06 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:07.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 253971 2.73 MB/sec execute 296 sec latency 51.643 ms 2024-08-31T07:21:08.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 255715 2.77 MB/sec execute 297 sec latency 13.514 ms 2024-08-31T07:21:08.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:08 smithi084 ceph-mon[22387]: pgmap v624: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 216 op/s 2024-08-31T07:21:09.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:08 smithi032 ceph-mon[17252]: pgmap v624: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 216 op/s 2024-08-31T07:21:09.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:08 smithi073 ceph-mon[21995]: pgmap v624: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 216 op/s 2024-08-31T07:21:09.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 256857 2.77 MB/sec execute 298 sec latency 20.368 ms 2024-08-31T07:21:10.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 258438 2.79 MB/sec execute 299 sec latency 42.310 ms 2024-08-31T07:21:10.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:10 smithi084 ceph-mon[22387]: pgmap v625: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-08-31T07:21:11.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:10 smithi032 ceph-mon[17252]: pgmap v625: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-08-31T07:21:11.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:10 smithi073 ceph-mon[21995]: pgmap v625: 97 pgs: 97 active+clean; 172 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-08-31T07:21:11.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 259691 2.80 MB/sec execute 300 sec latency 13.190 ms 2024-08-31T07:21:11.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:11 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:21:11.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:11 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:21:11.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:11 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:21:11.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:11 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:21:11.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:11 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:21:12.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:11 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:21:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:11 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:21:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:11 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:21:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:11 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:21:12.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:11 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:21:12.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:11 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:21:12.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:11 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:21:12.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:11 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:21:12.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:11 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:21:12.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:11 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:21:12.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 260661 2.81 MB/sec execute 301 sec latency 198.692 ms 2024-08-31T07:21:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:12 smithi084 ceph-mon[22387]: pgmap v626: 97 pgs: 97 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2024-08-31T07:21:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:12 smithi084 ceph-mon[22387]: osdmap e86: 12 total, 12 up, 12 in 2024-08-31T07:21:13.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:12 smithi032 ceph-mon[17252]: pgmap v626: 97 pgs: 97 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2024-08-31T07:21:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:12 smithi032 ceph-mon[17252]: osdmap e86: 12 total, 12 up, 12 in 2024-08-31T07:21:13.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:12 smithi073 ceph-mon[21995]: pgmap v626: 97 pgs: 97 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2024-08-31T07:21:13.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:12 smithi073 ceph-mon[21995]: osdmap e86: 12 total, 12 up, 12 in 2024-08-31T07:21:13.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 261447 2.81 MB/sec execute 302 sec latency 826.440 ms 2024-08-31T07:21:14.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 262953 2.84 MB/sec execute 303 sec latency 11.090 ms 2024-08-31T07:21:14.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:14 smithi084 ceph-mon[22387]: pgmap v628: 97 pgs: 97 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-08-31T07:21:15.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:14 smithi032 ceph-mon[17252]: pgmap v628: 97 pgs: 97 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-08-31T07:21:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:14 smithi073 ceph-mon[21995]: pgmap v628: 97 pgs: 97 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-08-31T07:21:15.749 INFO:tasks.workunit.client.0.smithi032.stdout: 1 264204 2.85 MB/sec execute 304 sec latency 85.792 ms 2024-08-31T07:21:16.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 264204 2.84 MB/sec execute 305 sec latency 1085.910 ms 2024-08-31T07:21:16.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:16 smithi084 ceph-mon[22387]: pgmap v629: 97 pgs: 97 active+clean; 181 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-08-31T07:21:17.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:16 smithi032 ceph-mon[17252]: pgmap v629: 97 pgs: 97 active+clean; 181 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-08-31T07:21:17.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:16 smithi073 ceph-mon[21995]: pgmap v629: 97 pgs: 97 active+clean; 181 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-08-31T07:21:17.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 264204 2.83 MB/sec execute 306 sec latency 2086.046 ms 2024-08-31T07:21:18.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 264204 2.82 MB/sec execute 307 sec latency 3086.149 ms 2024-08-31T07:21:18.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:18 smithi084 ceph-mon[22387]: pgmap v630: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-08-31T07:21:19.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:18 smithi032 ceph-mon[17252]: pgmap v630: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-08-31T07:21:19.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:18 smithi073 ceph-mon[21995]: pgmap v630: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-08-31T07:21:19.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 264204 2.81 MB/sec execute 308 sec latency 4086.269 ms 2024-08-31T07:21:20.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 265875 2.83 MB/sec execute 309 sec latency 4302.964 ms 2024-08-31T07:21:20.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:20 smithi084 ceph-mon[22387]: pgmap v631: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-08-31T07:21:21.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:20 smithi032 ceph-mon[17252]: pgmap v631: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-08-31T07:21:21.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:20 smithi073 ceph-mon[21995]: pgmap v631: 97 pgs: 97 active+clean; 185 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-08-31T07:21:21.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 267005 2.84 MB/sec execute 310 sec latency 18.143 ms 2024-08-31T07:21:22.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 267752 2.84 MB/sec execute 311 sec latency 523.514 ms 2024-08-31T07:21:22.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:22 smithi084 ceph-mon[22387]: pgmap v632: 97 pgs: 97 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-08-31T07:21:23.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:22 smithi032 ceph-mon[17252]: pgmap v632: 97 pgs: 97 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-08-31T07:21:23.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:22 smithi073 ceph-mon[21995]: pgmap v632: 97 pgs: 97 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-08-31T07:21:23.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 267752 2.83 MB/sec execute 312 sec latency 1523.557 ms 2024-08-31T07:21:24.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:21:24.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:21:24.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:21:24.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:23 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:21:24.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:21:24.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:21:24.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:21:24.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:23 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:21:24.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:21:24.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:21:24.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:21:24.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:23 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:21:24.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 267752 2.83 MB/sec execute 313 sec latency 2523.606 ms 2024-08-31T07:21:25.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:24 smithi032 ceph-mon[17252]: pgmap v633: 97 pgs: 97 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-08-31T07:21:25.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:24 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:25.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:24 smithi073 ceph-mon[21995]: pgmap v633: 97 pgs: 97 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-08-31T07:21:25.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:24 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:25.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:24 smithi084 ceph-mon[22387]: pgmap v633: 97 pgs: 97 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-08-31T07:21:25.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:24 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:25.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 269051 2.84 MB/sec execute 314 sec latency 2895.131 ms 2024-08-31T07:21:26.750 INFO:tasks.workunit.client.0.smithi032.stdout: 1 270269 2.85 MB/sec execute 315 sec latency 13.148 ms 2024-08-31T07:21:27.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:26 smithi032 ceph-mon[17252]: pgmap v634: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 886 KiB/s wr, 114 op/s 2024-08-31T07:21:27.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:26 smithi073 ceph-mon[21995]: pgmap v634: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 886 KiB/s wr, 114 op/s 2024-08-31T07:21:27.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:26 smithi084 ceph-mon[22387]: pgmap v634: 97 pgs: 97 active+clean; 188 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 886 KiB/s wr, 114 op/s 2024-08-31T07:21:27.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 271294 2.86 MB/sec execute 316 sec latency 308.441 ms 2024-08-31T07:21:28.009 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:27 smithi032 ceph-mon[17252]: pgmap v635: 97 pgs: 97 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-08-31T07:21:28.080 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:27 smithi073 ceph-mon[21995]: pgmap v635: 97 pgs: 97 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-08-31T07:21:28.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:27 smithi084 ceph-mon[22387]: pgmap v635: 97 pgs: 97 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-08-31T07:21:28.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 271294 2.85 MB/sec execute 317 sec latency 1308.533 ms 2024-08-31T07:21:29.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 271294 2.84 MB/sec execute 318 sec latency 2308.670 ms 2024-08-31T07:21:30.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:30 smithi032 ceph-mon[17252]: pgmap v636: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 973 KiB/s wr, 105 op/s 2024-08-31T07:21:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:30 smithi073 ceph-mon[21995]: pgmap v636: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 973 KiB/s wr, 105 op/s 2024-08-31T07:21:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:30 smithi084 ceph-mon[22387]: pgmap v636: 97 pgs: 97 active+clean; 194 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 973 KiB/s wr, 105 op/s 2024-08-31T07:21:30.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 272578 2.86 MB/sec execute 319 sec latency 2679.200 ms 2024-08-31T07:21:31.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 273818 2.87 MB/sec execute 320 sec latency 16.849 ms 2024-08-31T07:21:32.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:32 smithi032 ceph-mon[17252]: pgmap v637: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-08-31T07:21:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:32 smithi073 ceph-mon[21995]: pgmap v637: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-08-31T07:21:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:32 smithi084 ceph-mon[22387]: pgmap v637: 97 pgs: 97 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-08-31T07:21:32.751 INFO:tasks.workunit.client.0.smithi032.stdout: 1 275293 2.88 MB/sec execute 321 sec latency 160.564 ms 2024-08-31T07:21:33.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 276155 2.89 MB/sec execute 322 sec latency 332.369 ms 2024-08-31T07:21:34.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:34 smithi032 ceph-mon[17252]: pgmap v638: 97 pgs: 97 active+clean; 197 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-08-31T07:21:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:34 smithi073 ceph-mon[21995]: pgmap v638: 97 pgs: 97 active+clean; 197 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-08-31T07:21:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:34 smithi084 ceph-mon[22387]: pgmap v638: 97 pgs: 97 active+clean; 197 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-08-31T07:21:34.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 277092 2.90 MB/sec execute 323 sec latency 639.269 ms 2024-08-31T07:21:35.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:21:35.690 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:35.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 278385 2.91 MB/sec execute 324 sec latency 28.379 ms 2024-08-31T07:21:36.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:36 smithi032 ceph-mon[17252]: pgmap v639: 97 pgs: 97 active+clean; 196 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-08-31T07:21:36.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.345+0000 7f3df4a00640 1 -- 172.21.15.32:0/4215556981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df010b290 msgr2=0x7f3df010b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:36.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.345+0000 7f3df4a00640 1 --2- 172.21.15.32:0/4215556981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df010b290 0x7f3df010b6a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009860 tx=0x7f3ddc02a340 comp rx=0 tx=0).stop 2024-08-31T07:21:36.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.345+0000 7f3df4a00640 1 -- 172.21.15.32:0/4215556981 shutdown_connections 2024-08-31T07:21:36.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.345+0000 7f3df4a00640 1 --2- 172.21.15.32:0/4215556981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df010bbe0 0x7f3df0112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.345+0000 7f3df4a00640 1 --2- 172.21.15.32:0/4215556981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df010b290 0x7f3df010b6a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.345+0000 7f3df4a00640 1 --2- 172.21.15.32:0/4215556981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df0105950 0x7f3df0105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.345+0000 7f3df4a00640 1 -- 172.21.15.32:0/4215556981 >> 172.21.15.32:0/4215556981 conn(0x7f3df0101420 msgr2=0x7f3df0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:36.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.345+0000 7f3df4a00640 1 -- 172.21.15.32:0/4215556981 shutdown_connections 2024-08-31T07:21:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.345+0000 7f3df4a00640 1 -- 172.21.15.32:0/4215556981 wait complete. 2024-08-31T07:21:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.346+0000 7f3df4a00640 1 Processor -- start 2024-08-31T07:21:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.346+0000 7f3df4a00640 1 -- start start 2024-08-31T07:21:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.346+0000 7f3df4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df0105950 0x7f3df019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.346+0000 7f3df4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df010b290 0x7f3df019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.346+0000 7f3df4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df010bbe0 0x7f3df01a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.346+0000 7f3df4a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01158a0 con 0x7f3df010b290 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.346+0000 7f3df4a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0115720 con 0x7f3df010bbe0 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.346+0000 7f3df4a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0115a20 con 0x7f3df0105950 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3defe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df010bbe0 0x7f3df01a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3def400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df0105950 0x7f3df019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3deea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df010b290 0x7f3df019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3defe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df010bbe0 0x7f3df01a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38764/0 (socket says 172.21.15.32:38764) 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3def400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df0105950 0x7f3df019c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35398/0 (socket says 172.21.15.32:35398) 2024-08-31T07:21:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3def400640 1 -- 172.21.15.32:0/411422801 learned_addr learned my addr 172.21.15.32:0/411422801 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3def400640 1 -- 172.21.15.32:0/411422801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df010bbe0 msgr2=0x7f3df01a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3def400640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df010bbe0 0x7f3df01a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3def400640 1 -- 172.21.15.32:0/411422801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df010b290 msgr2=0x7f3df019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3def400640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df010b290 0x7f3df019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3def400640 1 -- 172.21.15.32:0/411422801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df01a2240 con 0x7f3df0105950 2024-08-31T07:21:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.347+0000 7f3defe00640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df010bbe0 0x7f3df01a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.348+0000 7f3def400640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df0105950 0x7f3df019c2d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3de000e9c0 tx=0x7f3de000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.348+0000 7f3de7e00640 1 -- 172.21.15.32:0/411422801 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de000cde0 con 0x7f3df0105950 2024-08-31T07:21:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.348+0000 7f3de7e00640 1 -- 172.21.15.32:0/411422801 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de00035f0 con 0x7f3df0105950 2024-08-31T07:21:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.348+0000 7f3df4a00640 1 -- 172.21.15.32:0/411422801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df01a2530 con 0x7f3df0105950 2024-08-31T07:21:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.348+0000 7f3de7e00640 1 -- 172.21.15.32:0/411422801 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0010690 con 0x7f3df0105950 2024-08-31T07:21:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.348+0000 7f3df4a00640 1 -- 172.21.15.32:0/411422801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01a8c90 con 0x7f3df0105950 2024-08-31T07:21:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.352+0000 7f3dfcbe9740 1 Processor -- start 2024-08-31T07:21:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.353+0000 7f3dfcbe9740 1 -- start start 2024-08-31T07:21:36.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.353+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b1707070 0x5577b1aacb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.353+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 0x5577b1aad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.353+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16e6390 0x5577b1aad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.353+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5577b1a23770 con 0x5577b1707070 2024-08-31T07:21:36.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.353+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5577b1a12d00 con 0x5577b16de1b0 2024-08-31T07:21:36.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.353+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5577b1a16cb0 con 0x5577b16e6390 2024-08-31T07:21:36.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.353+0000 7f3defe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b1707070 0x5577b1aacb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.353+0000 7f3de7e00640 1 -- 172.21.15.32:0/411422801 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3de0004000 con 0x7f3df0105950 2024-08-31T07:21:36.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3deea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16e6390 0x5577b1aad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3defe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b1707070 0x5577b1aacb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43168/0 (socket says 172.21.15.32:43168) 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3defe00640 1 -- 172.21.15.32:0/3674592933 learned_addr learned my addr 172.21.15.32:0/3674592933 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3def400640 1 --2- 172.21.15.32:0/3674592933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 0x5577b1aad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3deea00640 1 -- 172.21.15.32:0/3674592933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 msgr2=0x5577b1aad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3deea00640 1 --2- 172.21.15.32:0/3674592933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 0x5577b1aad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3deea00640 1 -- 172.21.15.32:0/3674592933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b1707070 msgr2=0x5577b1aacb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3deea00640 1 --2- 172.21.15.32:0/3674592933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b1707070 0x5577b1aacb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3deea00640 1 -- 172.21.15.32:0/3674592933 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b1772650 con 0x5577b16e6390 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3defe00640 1 --2- 172.21.15.32:0/3674592933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b1707070 0x5577b1aacb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3def400640 1 --2- 172.21.15.32:0/3674592933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 0x5577b1aad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.354+0000 7f3deea00640 1 --2- 172.21.15.32:0/3674592933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16e6390 0x5577b1aad5a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009780 tx=0x7f3ddc005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584914ddeacf0354 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.355+0000 7f3ddbe00640 1 -- 172.21.15.32:0/3674592933 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc041020 con 0x5577b16e6390 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.355+0000 7f3ddbe00640 1 -- 172.21.15.32:0/3674592933 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc030450 con 0x5577b16e6390 2024-08-31T07:21:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.355+0000 7f3ddbe00640 1 -- 172.21.15.32:0/3674592933 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc037040 con 0x5577b16e6390 2024-08-31T07:21:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.355+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3674592933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16e6390 msgr2=0x5577b1aad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.355+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/3674592933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16e6390 0x5577b1aad5a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009780 tx=0x7f3ddc005d10 comp rx=0 tx=0).stop 2024-08-31T07:21:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.355+0000 7f3de7e00640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3dd40b4ca0 0x7f3dd40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.356+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3674592933 shutdown_connections 2024-08-31T07:21:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.356+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/3674592933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16e6390 0x5577b1aad5a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.356+0000 7f3de7e00640 1 -- 172.21.15.32:0/411422801 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f3de00e5860 con 0x7f3df0105950 2024-08-31T07:21:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.356+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/3674592933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 0x5577b1aad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.356+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/3674592933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b1707070 0x5577b1aacb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.356+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3674592933 >> 172.21.15.32:0/3674592933 conn(0x5577b16fb460 msgr2=0x5577b1aaaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.356+0000 7f3defe00640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3dd40b4ca0 0x7f3dd40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.356+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3674592933 shutdown_connections 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.356+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3674592933 wait complete. 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.357+0000 7f3dfcbe9740 1 -- start start 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.357+0000 7f3defe00640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3dd40b4ca0 0x7f3dd40b71b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3de8005fd0 tx=0x7f3de8009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.357+0000 7f3dfcbe9740 1 Processor -- start 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.357+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1ab86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.357+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.357+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 0x5577b1ac4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.357+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5577b1875700 con 0x5577b16e6390 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.357+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5577b1999cb0 con 0x5577b1707070 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.357+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5577b1abb2a0 con 0x5577b16de1b0 2024-08-31T07:21:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.357+0000 7f3dfcbe9740 10 client.0 _ll_register_callbacks cb 0x5577b19bf430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:36.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3deea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3deea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43184/0 (socket says 172.21.15.32:43184) 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3deea00640 1 -- 172.21.15.32:0/3738720313 learned_addr learned my addr 172.21.15.32:0/3738720313 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3defe00640 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 0x5577b1ac4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3def400640 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1ab86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3defe00640 1 -- 172.21.15.32:0/3738720313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 msgr2=0x5577b1ab86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3defe00640 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1ab86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3defe00640 1 -- 172.21.15.32:0/3738720313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 msgr2=0x5577b1ab8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3defe00640 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3defe00640 1 -- 172.21.15.32:0/3738720313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b19da870 con 0x5577b1707070 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3deea00640 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.358+0000 7f3def400640 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1ab86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:36.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.359+0000 7f3defe00640 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 0x5577b1ac4590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3de8003b80 tx=0x7f3de8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa498297f7175ace server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.359+0000 7f3dd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.359+0000 7f3dd8c00640 1 -- 172.21.15.32:0/3738720313 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de800fd60 con 0x5577b1707070 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.359+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3738720313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ddc030450 con 0x5577b1707070 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.359+0000 7f3dd8c00640 1 -- 172.21.15.32:0/3738720313 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f3de8004030 con 0x5577b1707070 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.359+0000 7f3dd8c00640 1 -- 172.21.15.32:0/3738720313 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3de802e020 con 0x5577b1707070 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.359+0000 7f3dfcbe9740 10 client.27341 fetch_fsmap learned FSMap version 38 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.360+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3738720313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5577b1ac72d0 con 0x5577b1707070 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.360+0000 7f3dd8c00640 1 -- 172.21.15.32:0/3738720313 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc030450 con 0x5577b1707070 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.360+0000 7f3dd8c00640 1 -- 172.21.15.32:0/3738720313 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 38) ==== 8398+0+0 (secure 0 0 0) 0x7f3de8011850 con 0x5577b1707070 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.360+0000 7f3dfcbe9740 10 client.27341 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.360+0000 7f3dfcbe9740 10 client.27341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.360+0000 7f3dfcbe9740 20 client.27341 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.360+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5577b16c59f0 0x5577b1aca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.360+0000 7f3dfcbe9740 4 client.27341 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:21:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.360+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3738720313 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577b1a256c0 con 0x5577b16c59f0 2024-08-31T07:21:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.361+0000 7f3def400640 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5577b16c59f0 0x5577b1aca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.361+0000 7f3def400640 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5577b16c59f0 0x5577b1aca800 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a00bcb3c4fa3db server_cookie=bc9e19f1ecfd249e in_seq=0 out_seq=0 2024-08-31T07:21:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.361+0000 7f3dd8c00640 10 client.27341 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dd8c00640 1 -- 172.21.15.32:0/3738720313 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577b1a256c0 con 0x5577b16c59f0 2024-08-31T07:21:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dd8c00640 10 client.27341 handle_command_reply: tid=0 2024-08-31T07:21:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 client.27341 shutdown 2024-08-31T07:21:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3738720313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 msgr2=0x5577b1ac4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 0x5577b1ac4590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3de8003b80 tx=0x7f3de8004ab0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3738720313 shutdown_connections 2024-08-31T07:21:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5577b16c59f0 0x5577b1aca800 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 0x5577b1ac4590 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/3738720313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1ab86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3738720313 >> 172.21.15.32:0/3738720313 conn(0x5577b16fb460 msgr2=0x5577b1aafd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dd8c00640 0 client.27341 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3738720313 shutdown_connections 2024-08-31T07:21:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/3738720313 wait complete. 2024-08-31T07:21:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.362+0000 7f3dfcbe9740 20 client.27341 trim_cache size 0 max 16384 2024-08-31T07:21:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3dfcbe9740 1 Processor -- start 2024-08-31T07:21:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3dfcbe9740 1 -- start start 2024-08-31T07:21:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 0x5577b1ab3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b1707070 0x5577b1aba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b13f0 con 0x5577b16e6390 2024-08-31T07:21:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b30e0 con 0x5577b16de1b0 2024-08-31T07:21:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b1ac0 con 0x5577b1707070 2024-08-31T07:21:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3deea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3def400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 0x5577b1ab3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3deea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43190/0 (socket says 172.21.15.32:43190) 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3deea00640 1 -- 172.21.15.32:0/885285578 learned_addr learned my addr 172.21.15.32:0/885285578 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3defe00640 1 --2- 172.21.15.32:0/885285578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b1707070 0x5577b1aba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3def400640 1 -- 172.21.15.32:0/885285578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b1707070 msgr2=0x5577b1aba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3def400640 1 --2- 172.21.15.32:0/885285578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b1707070 0x5577b1aba100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3def400640 1 -- 172.21.15.32:0/885285578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 msgr2=0x5577b1ab05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3def400640 1 --2- 172.21.15.32:0/885285578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3def400640 1 -- 172.21.15.32:0/885285578 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b1ab69d0 con 0x5577b16de1b0 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3def400640 1 --2- 172.21.15.32:0/885285578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 0x5577b1ab3c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3de00e2fa0 tx=0x7f3de00e7d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d905e4a33fb730dd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3ddb400640 1 -- 172.21.15.32:0/885285578 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00abf40 con 0x5577b16de1b0 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3ddb400640 1 -- 172.21.15.32:0/885285578 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de00b07b0 con 0x5577b16de1b0 2024-08-31T07:21:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.382+0000 7f3ddb400640 1 -- 172.21.15.32:0/885285578 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00e3220 con 0x5577b16de1b0 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.383+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/885285578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 msgr2=0x5577b1ab3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.383+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/885285578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 0x5577b1ab3c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3de00e2fa0 tx=0x7f3de00e7d20 comp rx=0 tx=0).stop 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.383+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/885285578 shutdown_connections 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.383+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/885285578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b1707070 0x5577b1aba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.383+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/885285578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1ab05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.383+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/885285578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b16de1b0 0x5577b1ab3c80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.383+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/885285578 >> 172.21.15.32:0/885285578 conn(0x5577b16fb460 msgr2=0x5577b1ab99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.383+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/885285578 shutdown_connections 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.383+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/885285578 wait complete. 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.384+0000 7f3dfcbe9740 1 -- start start 2024-08-31T07:21:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.384+0000 7f3dfcbe9740 1 Processor -- start 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1abc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1abcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3dfcbe9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 0x5577b1b34200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b2950 con 0x5577b16e6390 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b2250 con 0x5577b1707070 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3dfcbe9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01af3f0 con 0x5577b16de1b0 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3dfcbe9740 10 client.0 _ll_register_callbacks cb 0x7f3de8003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3def400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1abc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3deea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1abcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3def400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1abc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35436/0 (socket says 172.21.15.32:35436) 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3def400640 1 -- 172.21.15.32:0/2383854954 learned_addr learned my addr 172.21.15.32:0/2383854954 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3defe00640 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 0x5577b1b34200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3def400640 1 -- 172.21.15.32:0/2383854954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 msgr2=0x5577b1b34200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3def400640 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 0x5577b1b34200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3def400640 1 -- 172.21.15.32:0/2383854954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 msgr2=0x5577b1abcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3def400640 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1abcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.385+0000 7f3def400640 1 -- 172.21.15.32:0/2383854954 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b1b34740 con 0x5577b16de1b0 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.386+0000 7f3def400640 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1abc840 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3de00a91b0 tx=0x7f3de0003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aca09dc81750925 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.386+0000 7f3dd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.386+0000 7f3dd8c00640 1 -- 172.21.15.32:0/2383854954 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00971b0 con 0x5577b16de1b0 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.386+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/2383854954 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de00b07b0 con 0x5577b16de1b0 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.386+0000 7f3dd8c00640 1 -- 172.21.15.32:0/2383854954 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f3de00a8530 con 0x5577b16de1b0 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.386+0000 7f3dd8c00640 1 -- 172.21.15.32:0/2383854954 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3de00e3220 con 0x5577b16de1b0 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.386+0000 7f3dfcbe9740 10 client.27358 fetch_fsmap learned FSMap version 38 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.386+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/2383854954 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5577b1b35810 con 0x5577b16de1b0 2024-08-31T07:21:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.386+0000 7f3dd8c00640 1 -- 172.21.15.32:0/2383854954 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de00b07b0 con 0x5577b16de1b0 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.387+0000 7f3dd8c00640 1 -- 172.21.15.32:0/2383854954 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 38) ==== 8398+0+0 (secure 0 0 0) 0x7f3de00b3b30 con 0x5577b16de1b0 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.387+0000 7f3dfcbe9740 10 client.27358 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.387+0000 7f3dfcbe9740 10 client.27358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.387+0000 7f3dfcbe9740 20 client.27358 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.387+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5577b169ada0 0x5577b1b38010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.387+0000 7f3dfcbe9740 4 client.27358 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.387+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/2383854954 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577b1a21160 con 0x5577b169ada0 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.387+0000 7f3deea00640 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5577b169ada0 0x5577b1b38010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3deea00640 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5577b169ada0 0x5577b1b38010 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb83cd8726488d34 server_cookie=78d485b92dde584f in_seq=0 out_seq=0 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dd8c00640 10 client.27358 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dd8c00640 1 -- 172.21.15.32:0/2383854954 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x5577b1a21160 con 0x5577b169ada0 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dd8c00640 10 client.27358 handle_command_reply: tid=0 2024-08-31T07:21:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 client.27358 shutdown 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/2383854954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 msgr2=0x5577b1abc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1abc840 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3de00a91b0 tx=0x7f3de0003cd0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/2383854954 shutdown_connections 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5577b169ada0 0x5577b1b38010 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577b1707070 0x5577b1b34200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577b16e6390 0x5577b1abcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 --2- 172.21.15.32:0/2383854954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577b16de1b0 0x5577b1abc840 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dd8c00640 0 client.27358 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/2383854954 >> 172.21.15.32:0/2383854954 conn(0x5577b16fb460 msgr2=0x5577b1aae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/2383854954 shutdown_connections 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.388+0000 7f3dfcbe9740 1 -- 172.21.15.32:0/2383854954 wait complete. 2024-08-31T07:21:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3dfcbe9740 20 client.27358 trim_cache size 0 max 16384 2024-08-31T07:21:36.394 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:21:36.394 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:21:36.394 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:21:36.394 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:21:36.394 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:21:36.394 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:21:36.394 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:21:36.395 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:21:36.398 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:36.399 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:36.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3df4a00640 1 -- 172.21.15.32:0/411422801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3dd40b4ca0 msgr2=0x7f3dd40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3df4a00640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3dd40b4ca0 0x7f3dd40b71b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3de8005fd0 tx=0x7f3de8009500 comp rx=0 tx=0).stop 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3df4a00640 1 -- 172.21.15.32:0/411422801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df0105950 msgr2=0x7f3df019c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3df4a00640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df0105950 0x7f3df019c2d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3de000e9c0 tx=0x7f3de000ee80 comp rx=0 tx=0).stop 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3df4a00640 1 -- 172.21.15.32:0/411422801 shutdown_connections 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3df4a00640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3dd40b4ca0 0x7f3dd40b71b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3df4a00640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df010bbe0 0x7f3df01a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3df4a00640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df010b290 0x7f3df019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3df4a00640 1 --2- 172.21.15.32:0/411422801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df0105950 0x7f3df019c2d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.389+0000 7f3df4a00640 1 -- 172.21.15.32:0/411422801 >> 172.21.15.32:0/411422801 conn(0x7f3df0101420 msgr2=0x7f3df0103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.390+0000 7f3df4a00640 1 -- 172.21.15.32:0/411422801 shutdown_connections 2024-08-31T07:21:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:36.390+0000 7f3df4a00640 1 -- 172.21.15.32:0/411422801 wait complete. 2024-08-31T07:21:36.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:36 smithi073 ceph-mon[21995]: pgmap v639: 97 pgs: 97 active+clean; 196 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-08-31T07:21:36.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:36 smithi084 ceph-mon[22387]: pgmap v639: 97 pgs: 97 active+clean; 196 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-08-31T07:21:36.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 278385 2.90 MB/sec execute 325 sec latency 1028.443 ms 2024-08-31T07:21:37.599 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:21:37.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:21:37.752 INFO:tasks.workunit.client.0.smithi032.stdout: 1 278385 2.89 MB/sec execute 326 sec latency 2028.484 ms 2024-08-31T07:21:37.983 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:38.293 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:38 smithi032 ceph-mon[17252]: pgmap v640: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-08-31T07:21:38.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:38 smithi073 ceph-mon[21995]: pgmap v640: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-08-31T07:21:38.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:38 smithi084 ceph-mon[22387]: pgmap v640: 97 pgs: 97 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-08-31T07:21:38.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 278385 2.88 MB/sec execute 327 sec latency 3028.546 ms 2024-08-31T07:21:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.369+0000 7f5140400640 1 -- 172.21.15.32:0/2656321034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5138110020 msgr2=0x7f5138112440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.369+0000 7f5140400640 1 --2- 172.21.15.32:0/2656321034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5138110020 0x7f5138112440 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5130009be0 tx=0x7f513002a7e0 comp rx=0 tx=0).stop 2024-08-31T07:21:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.370+0000 7f5140400640 1 -- 172.21.15.32:0/2656321034 shutdown_connections 2024-08-31T07:21:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.370+0000 7f5140400640 1 --2- 172.21.15.32:0/2656321034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5138110020 0x7f5138112440 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.370+0000 7f5140400640 1 --2- 172.21.15.32:0/2656321034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5138102dc0 0x7f513810fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.370+0000 7f5140400640 1 --2- 172.21.15.32:0/2656321034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5138102470 0x7f5138102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.370+0000 7f5140400640 1 -- 172.21.15.32:0/2656321034 >> 172.21.15.32:0/2656321034 conn(0x7f5138079500 msgr2=0x7f5138079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.370+0000 7f5140400640 1 -- 172.21.15.32:0/2656321034 shutdown_connections 2024-08-31T07:21:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.370+0000 7f5140400640 1 -- 172.21.15.32:0/2656321034 wait complete. 2024-08-31T07:21:39.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.371+0000 7f5140400640 1 Processor -- start 2024-08-31T07:21:39.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.371+0000 7f5140400640 1 -- start start 2024-08-31T07:21:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f5140400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5138102470 0x7f513819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f5140400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5138102dc0 0x7f513819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f5140400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5138110020 0x7f51381a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f5140400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5138115a30 con 0x7f5138110020 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f5140400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51381158b0 con 0x7f5138102470 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f5140400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5138115bb0 con 0x7f5138102dc0 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5138102470 0x7f513819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5138102470 0x7f513819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38830/0 (socket says 172.21.15.32:38830) 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513da00640 1 -- 172.21.15.32:0/1035511996 learned_addr learned my addr 172.21.15.32:0/1035511996 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513d000640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5138102dc0 0x7f513819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513e400640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5138110020 0x7f51381a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513da00640 1 -- 172.21.15.32:0/1035511996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5138102dc0 msgr2=0x7f513819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513da00640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5138102dc0 0x7f513819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513da00640 1 -- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5138110020 msgr2=0x7f51381a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513da00640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5138110020 0x7f51381a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513da00640 1 -- 172.21.15.32:0/1035511996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51381a22b0 con 0x7f5138102470 2024-08-31T07:21:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.372+0000 7f513e400640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5138110020 0x7f51381a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:21:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.373+0000 7f513da00640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5138102470 0x7f513819c340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f512c00ea60 tx=0x7f512c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.373+0000 7f5136000640 1 -- 172.21.15.32:0/1035511996 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c00cde0 con 0x7f5138102470 2024-08-31T07:21:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.373+0000 7f5136000640 1 -- 172.21.15.32:0/1035511996 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512c0035f0 con 0x7f5138102470 2024-08-31T07:21:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.373+0000 7f5136000640 1 -- 172.21.15.32:0/1035511996 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c010690 con 0x7f5138102470 2024-08-31T07:21:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.373+0000 7f5140400640 1 -- 172.21.15.32:0/1035511996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51381a25a0 con 0x7f5138102470 2024-08-31T07:21:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.373+0000 7f5140400640 1 -- 172.21.15.32:0/1035511996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51381a8d00 con 0x7f5138102470 2024-08-31T07:21:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.374+0000 7f5140400640 1 -- 172.21.15.32:0/1035511996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513806bf10 con 0x7f5138102470 2024-08-31T07:21:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.377+0000 7f5136000640 1 -- 172.21.15.32:0/1035511996 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f512c0037a0 con 0x7f5138102470 2024-08-31T07:21:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.379+0000 7f5136000640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51200b4d20 0x7f51200b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.379+0000 7f5136000640 1 -- 172.21.15.32:0/1035511996 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f512c0e5800 con 0x7f5138102470 2024-08-31T07:21:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.379+0000 7f513d000640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51200b4d20 0x7f51200b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.380+0000 7f513d000640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51200b4d20 0x7f51200b7230 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5138103fb0 tx=0x7f5124009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.381+0000 7f5136000640 1 -- 172.21.15.32:0/1035511996 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f512c0ac5e0 con 0x7f5138102470 2024-08-31T07:21:39.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.663+0000 7f5140400640 1 -- 172.21.15.32:0/1035511996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5138102880 con 0x7f5138102470 2024-08-31T07:21:39.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.664+0000 7f5136000640 1 -- 172.21.15.32:0/1035511996 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11975 (secure 0 0 0) 0x7f512c0abee0 con 0x7f5138102470 2024-08-31T07:21:39.666 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:21:39.666 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":38,"btime":"2024-08-31T07:21:23:735547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:21:23.365995+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:21:39.666 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 38 2024-08-31T07:21:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 -- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51200b4d20 msgr2=0x7f51200b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51200b4d20 0x7f51200b7230 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5138103fb0 tx=0x7f5124009260 comp rx=0 tx=0).stop 2024-08-31T07:21:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 -- 172.21.15.32:0/1035511996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5138102470 msgr2=0x7f513819c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5138102470 0x7f513819c340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f512c00ea60 tx=0x7f512c00ef20 comp rx=0 tx=0).stop 2024-08-31T07:21:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 -- 172.21.15.32:0/1035511996 shutdown_connections 2024-08-31T07:21:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51200b4d20 0x7f51200b7230 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5138110020 0x7f51381a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5138102dc0 0x7f513819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 --2- 172.21.15.32:0/1035511996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5138102470 0x7f513819c340 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 -- 172.21.15.32:0/1035511996 >> 172.21.15.32:0/1035511996 conn(0x7f5138079500 msgr2=0x7f5138109b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.667+0000 7f5140400640 1 -- 172.21.15.32:0/1035511996 shutdown_connections 2024-08-31T07:21:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:39.668+0000 7f5140400640 1 -- 172.21.15.32:0/1035511996 wait complete. 2024-08-31T07:21:39.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 279966 2.90 MB/sec execute 328 sec latency 3283.298 ms 2024-08-31T07:21:40.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:40 smithi032 ceph-mon[17252]: pgmap v641: 97 pgs: 97 active+clean; 201 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-08-31T07:21:40.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:40 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1035511996' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:21:40.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:40 smithi073 ceph-mon[21995]: pgmap v641: 97 pgs: 97 active+clean; 201 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-08-31T07:21:40.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:40 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1035511996' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:21:40.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:40 smithi084 ceph-mon[22387]: pgmap v641: 97 pgs: 97 active+clean; 201 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-08-31T07:21:40.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:40 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1035511996' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:21:40.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:21:40.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 281052 2.91 MB/sec execute 329 sec latency 45.426 ms 2024-08-31T07:21:41.009 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.695+0000 7f15ba400640 1 -- 172.21.15.32:0/1143490441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15b4109090 msgr2=0x7f15b41094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.695+0000 7f15ba400640 1 --2- 172.21.15.32:0/1143490441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15b4109090 0x7f15b41094a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009900 tx=0x7f15a002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.695+0000 7f15ba400640 1 -- 172.21.15.32:0/1143490441 shutdown_connections 2024-08-31T07:21:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.695+0000 7f15ba400640 1 --2- 172.21.15.32:0/1143490441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b41099e0 0x7f15b4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.695+0000 7f15ba400640 1 --2- 172.21.15.32:0/1143490441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15b4109090 0x7f15b41094a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.695+0000 7f15ba400640 1 --2- 172.21.15.32:0/1143490441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b4103750 0x7f15b4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.695+0000 7f15ba400640 1 -- 172.21.15.32:0/1143490441 >> 172.21.15.32:0/1143490441 conn(0x7f15b40ff1e0 msgr2=0x7f15b4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.695+0000 7f15ba400640 1 -- 172.21.15.32:0/1143490441 shutdown_connections 2024-08-31T07:21:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.695+0000 7f15ba400640 1 -- 172.21.15.32:0/1143490441 wait complete. 2024-08-31T07:21:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.697+0000 7f15ba400640 1 Processor -- start 2024-08-31T07:21:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.697+0000 7f15ba400640 1 -- start start 2024-08-31T07:21:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.697+0000 7f15ba400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b4103750 0x7f15b419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.697+0000 7f15ba400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15b4109090 0x7f15b419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15ba400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b41099e0 0x7f15b41a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15ba400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4113670 con 0x7f15b4109090 2024-08-31T07:21:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15ba400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f15b41134f0 con 0x7f15b4103750 2024-08-31T07:21:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15ba400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f15b41137f0 con 0x7f15b41099e0 2024-08-31T07:21:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15b4109090 0x7f15b419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b4103750 0x7f15b419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b9a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b41099e0 0x7f15b41a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15b4109090 0x7f15b419c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43210/0 (socket says 172.21.15.32:43210) 2024-08-31T07:21:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b3e00640 1 -- 172.21.15.32:0/3789590954 learned_addr learned my addr 172.21.15.32:0/3789590954 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b9000640 1 -- 172.21.15.32:0/3789590954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b41099e0 msgr2=0x7f15b41a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b9000640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b41099e0 0x7f15b41a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b9000640 1 -- 172.21.15.32:0/3789590954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15b4109090 msgr2=0x7f15b419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b9000640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15b4109090 0x7f15b419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b9000640 1 -- 172.21.15.32:0/3789590954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b41a2250 con 0x7f15b4103750 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b9a00640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b41099e0 0x7f15b41a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.698+0000 7f15b3e00640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15b4109090 0x7f15b419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.699+0000 7f15b9000640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b4103750 0x7f15b419c2e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15a800e9f0 tx=0x7f15a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.699+0000 7f15b1600640 1 -- 172.21.15.32:0/3789590954 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a800cd80 con 0x7f15b4103750 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.699+0000 7f15ba400640 1 -- 172.21.15.32:0/3789590954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b41a2540 con 0x7f15b4103750 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.699+0000 7f15ba400640 1 -- 172.21.15.32:0/3789590954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b41a8ca0 con 0x7f15b4103750 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.699+0000 7f15b1600640 1 -- 172.21.15.32:0/3789590954 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a80035f0 con 0x7f15b4103750 2024-08-31T07:21:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.699+0000 7f15b1600640 1 -- 172.21.15.32:0/3789590954 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8010690 con 0x7f15b4103750 2024-08-31T07:21:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.702+0000 7f15c2672740 1 Processor -- start 2024-08-31T07:21:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.702+0000 7f15c2672740 1 -- start start 2024-08-31T07:21:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15c2672740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4aea070 0x5616a4e8fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15c2672740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4ac11b0 0x5616a4e8ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15c2672740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 0x5616a4e904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15c2672740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5616a4e067b0 con 0x5616a4ac9390 2024-08-31T07:21:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15c2672740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5616a4df5d40 con 0x5616a4aea070 2024-08-31T07:21:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15c2672740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5616a4df9cf0 con 0x5616a4ac11b0 2024-08-31T07:21:41.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15b1600640 1 -- 172.21.15.32:0/3789590954 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f15a80037a0 con 0x7f15b4103750 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 0x5616a4e904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 0x5616a4e904f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43224/0 (socket says 172.21.15.32:43224) 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15b3e00640 1 -- 172.21.15.32:0/2041894988 learned_addr learned my addr 172.21.15.32:0/2041894988 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15b9a00640 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4aea070 0x5616a4e8fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.703+0000 7f15b9000640 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4ac11b0 0x5616a4e8ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15b3e00640 1 -- 172.21.15.32:0/2041894988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4ac11b0 msgr2=0x5616a4e8ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15b3e00640 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4ac11b0 0x5616a4e8ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15b3e00640 1 -- 172.21.15.32:0/2041894988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4aea070 msgr2=0x5616a4e8fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15b3e00640 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4aea070 0x5616a4e8fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15b3e00640 1 -- 172.21.15.32:0/2041894988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616a4b55650 con 0x5616a4ac9390 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15b9a00640 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4aea070 0x5616a4e8fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15b9000640 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4ac11b0 0x5616a4e8ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:21:41.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15b3e00640 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 0x5616a4e904f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f15a0006010 tx=0x7f15a0005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdccc300a4fcb12b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15a6000640 1 -- 172.21.15.32:0/2041894988 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0035040 con 0x5616a4ac9390 2024-08-31T07:21:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15a6000640 1 -- 172.21.15.32:0/2041894988 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a0031720 con 0x5616a4ac9390 2024-08-31T07:21:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.704+0000 7f15a6000640 1 -- 172.21.15.32:0/2041894988 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0004a00 con 0x5616a4ac9390 2024-08-31T07:21:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.705+0000 7f15b1600640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f159c0b4d20 0x7f159c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.705+0000 7f15b1600640 1 -- 172.21.15.32:0/3789590954 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f15a80e56f0 con 0x7f15b4103750 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.705+0000 7f15b9a00640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f159c0b4d20 0x7f159c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.705+0000 7f15c2672740 1 -- 172.21.15.32:0/2041894988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 msgr2=0x5616a4e904f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.705+0000 7f15c2672740 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 0x5616a4e904f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f15a0006010 tx=0x7f15a0005e20 comp rx=0 tx=0).stop 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.706+0000 7f15c2672740 1 -- 172.21.15.32:0/2041894988 shutdown_connections 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.706+0000 7f15c2672740 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 0x5616a4e904f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.706+0000 7f15c2672740 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4ac11b0 0x5616a4e8ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.706+0000 7f15c2672740 1 --2- 172.21.15.32:0/2041894988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4aea070 0x5616a4e8fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.706+0000 7f15c2672740 1 -- 172.21.15.32:0/2041894988 >> 172.21.15.32:0/2041894988 conn(0x5616a4ade460 msgr2=0x5616a4e8d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.706+0000 7f15c2672740 1 -- 172.21.15.32:0/2041894988 shutdown_connections 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.706+0000 7f15b9a00640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f159c0b4d20 0x7f159c0b7230 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009fd0 tx=0x7f15ac009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.706+0000 7f15c2672740 1 -- 172.21.15.32:0/2041894988 wait complete. 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.706+0000 7f15c2672740 1 -- start start 2024-08-31T07:21:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.707+0000 7f15c2672740 1 Processor -- start 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.707+0000 7f15c2672740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac11b0 0x5616a4e9b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.707+0000 7f15c2672740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 0x5616a4e9bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.707+0000 7f15c2672740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4ea7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.707+0000 7f15c2672740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5616a4c58700 con 0x5616a4ac9390 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.707+0000 7f15c2672740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5616a4d7ccf0 con 0x5616a4ac11b0 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.707+0000 7f15c2672740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5616a4e9e1f0 con 0x5616a4aea070 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.707+0000 7f15c2672740 10 client.0 _ll_register_callbacks cb 0x5616a4da2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac11b0 0x5616a4e9b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 0x5616a4e9bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac11b0 0x5616a4e9b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38876/0 (socket says 172.21.15.32:38876) 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9000640 1 -- 172.21.15.32:0/1685673441 learned_addr learned my addr 172.21.15.32:0/1685673441 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9a00640 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4ea7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9000640 1 -- 172.21.15.32:0/1685673441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 msgr2=0x5616a4ea7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9000640 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4ea7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9000640 1 -- 172.21.15.32:0/1685673441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 msgr2=0x5616a4e9bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9000640 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 0x5616a4e9bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9000640 1 -- 172.21.15.32:0/1685673441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616a4dbd8b0 con 0x5616a4ac11b0 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9a00640 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4ea7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f15b9000640 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac11b0 0x5616a4e9b600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f15a80abdc0 tx=0x7f15a8003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94015a4a799a9c65 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.708+0000 7f159aa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.709+0000 7f159aa00640 1 -- 172.21.15.32:0/1685673441 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a80b06a0 con 0x5616a4ac11b0 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.709+0000 7f15c2672740 1 -- 172.21.15.32:0/1685673441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a0031720 con 0x5616a4ac11b0 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.709+0000 7f159aa00640 1 -- 172.21.15.32:0/1685673441 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f15a80e3110 con 0x5616a4ac11b0 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.709+0000 7f159aa00640 1 -- 172.21.15.32:0/1685673441 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f15a80b5150 con 0x5616a4ac11b0 2024-08-31T07:21:41.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.709+0000 7f15c2672740 10 client.27365 fetch_fsmap learned FSMap version 38 2024-08-31T07:21:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.709+0000 7f15c2672740 1 -- 172.21.15.32:0/1685673441 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5616a4eaa220 con 0x5616a4ac11b0 2024-08-31T07:21:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.709+0000 7f159aa00640 1 -- 172.21.15.32:0/1685673441 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a0031720 con 0x5616a4ac11b0 2024-08-31T07:21:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.710+0000 7f159aa00640 1 -- 172.21.15.32:0/1685673441 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 38) ==== 8398+0+0 (secure 0 0 0) 0x7f15a80b02b0 con 0x5616a4ac11b0 2024-08-31T07:21:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.710+0000 7f15c2672740 10 client.27365 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T07:21:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.710+0000 7f15c2672740 10 client.27365 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:21:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.710+0000 7f15c2672740 20 client.27365 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.710+0000 7f15c2672740 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5616a4aa89f0 0x5616a4ead6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.710+0000 7f15c2672740 4 client.27365 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:21:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.710+0000 7f15c2672740 1 -- 172.21.15.32:0/1685673441 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616a4e08700 con 0x5616a4aa89f0 2024-08-31T07:21:41.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.710+0000 7f15b3e00640 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5616a4aa89f0 0x5616a4ead6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.711+0000 7f15b3e00640 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5616a4aa89f0 0x5616a4ead6c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6db79290a712080e server_cookie=1828091f783e1c09 in_seq=0 out_seq=0 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.711+0000 7f159aa00640 10 client.27365 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.711+0000 7f159aa00640 1 -- 172.21.15.32:0/1685673441 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5616a4e08700 con 0x5616a4aa89f0 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.711+0000 7f159aa00640 10 client.27365 handle_command_reply: tid=0 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 client.27365 shutdown 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 -- 172.21.15.32:0/1685673441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac11b0 msgr2=0x5616a4e9b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac11b0 0x5616a4e9b600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f15a80abdc0 tx=0x7f15a8003cf0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 -- 172.21.15.32:0/1685673441 shutdown_connections 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5616a4aa89f0 0x5616a4ead6c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4ea7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac9390 0x5616a4e9bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 --2- 172.21.15.32:0/1685673441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac11b0 0x5616a4e9b600 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f159aa00640 0 client.27365 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 -- 172.21.15.32:0/1685673441 >> 172.21.15.32:0/1685673441 conn(0x5616a4ade460 msgr2=0x5616a4e92c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:41.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 -- 172.21.15.32:0/1685673441 shutdown_connections 2024-08-31T07:21:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 1 -- 172.21.15.32:0/1685673441 wait complete. 2024-08-31T07:21:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.712+0000 7f15c2672740 20 client.27365 trim_cache size 0 max 16384 2024-08-31T07:21:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.726+0000 7f15c2672740 1 Processor -- start 2024-08-31T07:21:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.726+0000 7f15c2672740 1 -- start start 2024-08-31T07:21:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15c2672740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4e8d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15c2672740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 0x5616a4ea7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15c2672740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4e92b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15c2672740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15b41b14b0 con 0x5616a4ac11b0 2024-08-31T07:21:41.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15c2672740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f15b41b31a0 con 0x5616a4ac9390 2024-08-31T07:21:41.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15c2672740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f15b41b1b80 con 0x5616a4aea070 2024-08-31T07:21:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4e8d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b9a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4e92b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4e8d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43236/0 (socket says 172.21.15.32:43236) 2024-08-31T07:21:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b9000640 1 -- 172.21.15.32:0/3654947245 learned_addr learned my addr 172.21.15.32:0/3654947245 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 0x5616a4ea7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b3e00640 1 -- 172.21.15.32:0/3654947245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 msgr2=0x5616a4e92b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b3e00640 1 --2- 172.21.15.32:0/3654947245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4e92b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b3e00640 1 -- 172.21.15.32:0/3654947245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 msgr2=0x5616a4e8d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:41.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b3e00640 1 --2- 172.21.15.32:0/3654947245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4e8d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b3e00640 1 -- 172.21.15.32:0/3654947245 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616a4e91d60 con 0x5616a4ac9390 2024-08-31T07:21:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.727+0000 7f15b9000640 1 --2- 172.21.15.32:0/3654947245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4e8d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.728+0000 7f15b3e00640 1 --2- 172.21.15.32:0/3654947245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 0x5616a4ea7410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5616a4eaddf0 tx=0x7f15a002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25b3b90a5da444dc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.728+0000 7f15a5600640 1 -- 172.21.15.32:0/3654947245 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0035040 con 0x5616a4ac9390 2024-08-31T07:21:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.728+0000 7f15a5600640 1 -- 172.21.15.32:0/3654947245 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a0004040 con 0x5616a4ac9390 2024-08-31T07:21:41.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.728+0000 7f15a5600640 1 -- 172.21.15.32:0/3654947245 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0003850 con 0x5616a4ac9390 2024-08-31T07:21:41.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.729+0000 7f15c2672740 1 -- 172.21.15.32:0/3654947245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 msgr2=0x5616a4ea7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.729+0000 7f15c2672740 1 --2- 172.21.15.32:0/3654947245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 0x5616a4ea7410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5616a4eaddf0 tx=0x7f15a002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.729+0000 7f15c2672740 1 -- 172.21.15.32:0/3654947245 shutdown_connections 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.729+0000 7f15c2672740 1 --2- 172.21.15.32:0/3654947245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4e92b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.729+0000 7f15c2672740 1 --2- 172.21.15.32:0/3654947245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 0x5616a4ea7410 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.729+0000 7f15c2672740 1 --2- 172.21.15.32:0/3654947245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4e8d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.729+0000 7f15c2672740 1 -- 172.21.15.32:0/3654947245 >> 172.21.15.32:0/3654947245 conn(0x5616a4ade460 msgr2=0x5616a4e8d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.729+0000 7f15c2672740 1 -- 172.21.15.32:0/3654947245 shutdown_connections 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.729+0000 7f15c2672740 1 -- 172.21.15.32:0/3654947245 wait complete. 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.730+0000 7f15c2672740 1 -- start start 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.730+0000 7f15c2672740 1 Processor -- start 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.730+0000 7f15c2672740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4ea3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.730+0000 7f15c2672740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 0x5616a4ea3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.730+0000 7f15c2672740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4e9e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.730+0000 7f15c2672740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15b41b2a10 con 0x5616a4ac11b0 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.730+0000 7f15c2672740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f15b41b2310 con 0x5616a4ac9390 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.730+0000 7f15c2672740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f15b41af4b0 con 0x5616a4aea070 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.730+0000 7f15c2672740 10 client.0 _ll_register_callbacks cb 0x7f15a80ab5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4ea3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4e9e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4e9e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35524/0 (socket says 172.21.15.32:35524) 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4ea3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43244/0 (socket says 172.21.15.32:43244) 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9a00640 1 -- 172.21.15.32:0/3793731230 learned_addr learned my addr 172.21.15.32:0/3793731230 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b3e00640 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 0x5616a4ea3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9a00640 1 -- 172.21.15.32:0/3793731230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 msgr2=0x5616a4ea3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9a00640 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 0x5616a4ea3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9a00640 1 -- 172.21.15.32:0/3793731230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 msgr2=0x5616a4ea3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9a00640 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4ea3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9a00640 1 -- 172.21.15.32:0/3793731230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616a4e9ebc0 con 0x5616a4aea070 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f15b9a00640 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4e9e430 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f15ac011880 tx=0x7f15ac004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ce915a98179a3e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.731+0000 7f159aa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.732+0000 7f159aa00640 1 -- 172.21.15.32:0/3793731230 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac010b40 con 0x5616a4aea070 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.732+0000 7f15c2672740 1 -- 172.21.15.32:0/3793731230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a0004040 con 0x5616a4aea070 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.732+0000 7f159aa00640 1 -- 172.21.15.32:0/3793731230 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f15ac010ce0 con 0x5616a4aea070 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.732+0000 7f159aa00640 1 -- 172.21.15.32:0/3793731230 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f15ac037050 con 0x5616a4aea070 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.732+0000 7f15c2672740 10 client.27379 fetch_fsmap learned FSMap version 38 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.732+0000 7f15c2672740 1 -- 172.21.15.32:0/3793731230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5616a4f18b30 con 0x5616a4aea070 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.732+0000 7f159aa00640 1 -- 172.21.15.32:0/3793731230 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a0004040 con 0x5616a4aea070 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.733+0000 7f159aa00640 1 -- 172.21.15.32:0/3793731230 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 38) ==== 8398+0+0 (secure 0 0 0) 0x7f15ac029400 con 0x5616a4aea070 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.733+0000 7f15c2672740 10 client.27379 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T07:21:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.733+0000 7f15c2672740 10 client.27379 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.733+0000 7f15c2672740 20 client.27379 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.733+0000 7f15c2672740 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5616a4a7dda0 0x5616a4f1b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.733+0000 7f15c2672740 4 client.27379 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.733+0000 7f15c2672740 1 -- 172.21.15.32:0/3793731230 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616a4e041a0 con 0x5616a4a7dda0 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.733+0000 7f15b9000640 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5616a4a7dda0 0x5616a4f1b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15b9000640 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5616a4a7dda0 0x5616a4f1b320 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c830e4e2e345b08 server_cookie=6ed9290b2ac83820 in_seq=0 out_seq=0 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f159aa00640 10 client.27379 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f159aa00640 1 -- 172.21.15.32:0/3793731230 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616a4e041a0 con 0x5616a4a7dda0 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f159aa00640 10 client.27379 handle_command_reply: tid=0 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 client.27379 shutdown 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 -- 172.21.15.32:0/3793731230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 msgr2=0x5616a4e9e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4e9e430 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f15ac011880 tx=0x7f15ac004ab0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 -- 172.21.15.32:0/3793731230 shutdown_connections 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5616a4a7dda0 0x5616a4f1b320 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5616a4aea070 0x5616a4e9e430 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f159aa00640 0 client.27379 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:21:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5616a4ac9390 0x5616a4ea3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 --2- 172.21.15.32:0/3793731230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616a4ac11b0 0x5616a4ea3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 -- 172.21.15.32:0/3793731230 >> 172.21.15.32:0/3793731230 conn(0x5616a4ade460 msgr2=0x5616a4e8bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 -- 172.21.15.32:0/3793731230 shutdown_connections 2024-08-31T07:21:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 1 -- 172.21.15.32:0/3793731230 wait complete. 2024-08-31T07:21:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.734+0000 7f15c2672740 20 client.27379 trim_cache size 0 max 16384 2024-08-31T07:21:41.738 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:21:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 -- 172.21.15.32:0/3789590954 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f159c0b4d20 msgr2=0x7f159c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f159c0b4d20 0x7f159c0b7230 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009fd0 tx=0x7f15ac009500 comp rx=0 tx=0).stop 2024-08-31T07:21:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 -- 172.21.15.32:0/3789590954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b4103750 msgr2=0x7f15b419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b4103750 0x7f15b419c2e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15a800e9f0 tx=0x7f15a800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 -- 172.21.15.32:0/3789590954 shutdown_connections 2024-08-31T07:21:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f159c0b4d20 0x7f159c0b7230 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b41099e0 0x7f15b41a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15b4109090 0x7f15b419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 --2- 172.21.15.32:0/3789590954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b4103750 0x7f15b419c2e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 -- 172.21.15.32:0/3789590954 >> 172.21.15.32:0/3789590954 conn(0x7f15b40ff1e0 msgr2=0x7f15b4108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 -- 172.21.15.32:0/3789590954 shutdown_connections 2024-08-31T07:21:41.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:41.735+0000 7f15ba400640 1 -- 172.21.15.32:0/3789590954 wait complete. 2024-08-31T07:21:41.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 281934 2.91 MB/sec execute 330 sec latency 239.719 ms 2024-08-31T07:21:42.338 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:42 smithi032 ceph-mon[17252]: pgmap v642: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-08-31T07:21:42.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:42 smithi073 ceph-mon[21995]: pgmap v642: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-08-31T07:21:42.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:42 smithi084 ceph-mon[22387]: pgmap v642: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-08-31T07:21:42.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 281934 2.90 MB/sec execute 331 sec latency 1239.834 ms 2024-08-31T07:21:42.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:21:43.257 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:43.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 281934 2.89 MB/sec execute 332 sec latency 2239.919 ms 2024-08-31T07:21:44.303 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:44 smithi032 ceph-mon[17252]: pgmap v643: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-08-31T07:21:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:44 smithi073 ceph-mon[21995]: pgmap v643: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-08-31T07:21:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:44 smithi084 ceph-mon[22387]: pgmap v643: 97 pgs: 97 active+clean; 202 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-08-31T07:21:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.607+0000 7fce52600640 1 -- 172.21.15.32:0/301545330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c0776e0 msgr2=0x7fce4c077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.607+0000 7fce52600640 1 --2- 172.21.15.32:0/301545330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c0776e0 0x7fce4c077af0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009900 tx=0x7fce3c02a410 comp rx=0 tx=0).stop 2024-08-31T07:21:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.607+0000 7fce52600640 1 -- 172.21.15.32:0/301545330 shutdown_connections 2024-08-31T07:21:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.607+0000 7fce52600640 1 --2- 172.21.15.32:0/301545330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce4c1144e0 0x7fce4c116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.607+0000 7fce52600640 1 --2- 172.21.15.32:0/301545330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce4c078030 0x7fce4c113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.607+0000 7fce52600640 1 --2- 172.21.15.32:0/301545330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c0776e0 0x7fce4c077af0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.607+0000 7fce52600640 1 -- 172.21.15.32:0/301545330 >> 172.21.15.32:0/301545330 conn(0x7fce4c1015b0 msgr2=0x7fce4c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.607+0000 7fce52600640 1 -- 172.21.15.32:0/301545330 shutdown_connections 2024-08-31T07:21:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.607+0000 7fce52600640 1 -- 172.21.15.32:0/301545330 wait complete. 2024-08-31T07:21:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.608+0000 7fce52600640 1 Processor -- start 2024-08-31T07:21:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.608+0000 7fce52600640 1 -- start start 2024-08-31T07:21:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce52600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c0776e0 0x7fce4c1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce52600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce4c078030 0x7fce4c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce52600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce4c1144e0 0x7fce4c1a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce52600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c119e60 con 0x7fce4c0776e0 2024-08-31T07:21:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce52600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c119ce0 con 0x7fce4c1144e0 2024-08-31T07:21:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce52600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c119fe0 con 0x7fce4c078030 2024-08-31T07:21:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce4b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c0776e0 0x7fce4c1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce4b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c0776e0 0x7fce4c1a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43262/0 (socket says 172.21.15.32:43262) 2024-08-31T07:21:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce4b400640 1 -- 172.21.15.32:0/1974394481 learned_addr learned my addr 172.21.15.32:0/1974394481 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce42a00640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce4c078030 0x7fce4c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.609+0000 7fce4be00640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce4c1144e0 0x7fce4c1a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.610+0000 7fce4b400640 1 -- 172.21.15.32:0/1974394481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce4c078030 msgr2=0x7fce4c1a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.610+0000 7fce4b400640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce4c078030 0x7fce4c1a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.610+0000 7fce4b400640 1 -- 172.21.15.32:0/1974394481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce4c1144e0 msgr2=0x7fce4c1a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.610+0000 7fce4b400640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce4c1144e0 0x7fce4c1a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.610+0000 7fce4b400640 1 -- 172.21.15.32:0/1974394481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c1a67a0 con 0x7fce4c0776e0 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.610+0000 7fce4be00640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce4c1144e0 0x7fce4c1a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.610+0000 7fce4b400640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c0776e0 0x7fce4c1a0830 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fce3c005940 tx=0x7fce3c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.610+0000 7fce48c00640 1 -- 172.21.15.32:0/1974394481 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c03f030 con 0x7fce4c0776e0 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.611+0000 7fce48c00640 1 -- 172.21.15.32:0/1974394481 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c0044c0 con 0x7fce4c0776e0 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.611+0000 7fce52600640 1 -- 172.21.15.32:0/1974394481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4c1a6a30 con 0x7fce4c0776e0 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.611+0000 7fce48c00640 1 -- 172.21.15.32:0/1974394481 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c035070 con 0x7fce4c0776e0 2024-08-31T07:21:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.611+0000 7fce52600640 1 -- 172.21.15.32:0/1974394481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4c1ad1f0 con 0x7fce4c0776e0 2024-08-31T07:21:44.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.614+0000 7fce592d1740 1 Processor -- start 2024-08-31T07:21:44.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.614+0000 7fce592d1740 1 -- start start 2024-08-31T07:21:44.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.614+0000 7fce48c00640 1 -- 172.21.15.32:0/1974394481 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fce3c042030 con 0x7fce4c0776e0 2024-08-31T07:21:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce592d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd4a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce592d1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd4aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce592d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 0x56253fd4b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce592d1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56253fcc17b0 con 0x56253f97c1b0 2024-08-31T07:21:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce592d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56253fcb0d40 con 0x56253f9a5070 2024-08-31T07:21:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce592d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56253fcb4cf0 con 0x56253f984390 2024-08-31T07:21:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce4b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd4aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce4be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd4a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce4b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd4aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43270/0 (socket says 172.21.15.32:43270) 2024-08-31T07:21:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce42a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 0x56253fd4b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.615+0000 7fce4b400640 1 -- 172.21.15.32:0/1850674931 learned_addr learned my addr 172.21.15.32:0/1850674931 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.616+0000 7fce42a00640 1 -- 172.21.15.32:0/1850674931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 msgr2=0x56253fd4a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.616+0000 7fce42a00640 1 --2- 172.21.15.32:0/1850674931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd4a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.616+0000 7fce42a00640 1 -- 172.21.15.32:0/1850674931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 msgr2=0x56253fd4aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.616+0000 7fce42a00640 1 --2- 172.21.15.32:0/1850674931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd4aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.616+0000 7fce42a00640 1 -- 172.21.15.32:0/1850674931 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253fa10650 con 0x56253f984390 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.616+0000 7fce4b400640 1 --2- 172.21.15.32:0/1850674931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd4aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.616+0000 7fce42a00640 1 --2- 172.21.15.32:0/1850674931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 0x56253fd4b410 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fce4c06c5b0 tx=0x7fce34011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d1819dc5e350763 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.616+0000 7fce48c00640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 0x7fce300b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.616+0000 7fce40c00640 1 -- 172.21.15.32:0/1850674931 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34004050 con 0x56253f984390 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.616+0000 7fce40c00640 1 -- 172.21.15.32:0/1850674931 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce340054d0 con 0x56253f984390 2024-08-31T07:21:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.617+0000 7fce40c00640 1 -- 172.21.15.32:0/1850674931 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34004c40 con 0x56253f984390 2024-08-31T07:21:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.617+0000 7fce48c00640 1 -- 172.21.15.32:0/1974394481 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7fce3c105840 con 0x7fce4c0776e0 2024-08-31T07:21:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.617+0000 7fce4be00640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 0x7fce300b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce592d1740 1 -- 172.21.15.32:0/1850674931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 msgr2=0x56253fd4b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce4be00640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 0x7fce300b7230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fce44009fd0 tx=0x7fce44009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce592d1740 1 --2- 172.21.15.32:0/1850674931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 0x56253fd4b410 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fce4c06c5b0 tx=0x7fce34011810 comp rx=0 tx=0).stop 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce592d1740 1 -- 172.21.15.32:0/1850674931 shutdown_connections 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce592d1740 1 --2- 172.21.15.32:0/1850674931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 0x56253fd4b410 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce592d1740 1 --2- 172.21.15.32:0/1850674931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd4aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce592d1740 1 --2- 172.21.15.32:0/1850674931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd4a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce592d1740 1 -- 172.21.15.32:0/1850674931 >> 172.21.15.32:0/1850674931 conn(0x56253f999460 msgr2=0x56253fd48880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce592d1740 1 -- 172.21.15.32:0/1850674931 shutdown_connections 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce592d1740 1 -- 172.21.15.32:0/1850674931 wait complete. 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.618+0000 7fce592d1740 1 -- start start 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.619+0000 7fce592d1740 1 Processor -- start 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.619+0000 7fce592d1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd56590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.619+0000 7fce592d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 0x56253fd56ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.619+0000 7fce592d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd62460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.619+0000 7fce592d1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56253fb13700 con 0x56253f97c1b0 2024-08-31T07:21:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.619+0000 7fce592d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56253fc37cf0 con 0x56253f9a5070 2024-08-31T07:21:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.619+0000 7fce592d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56253fd59170 con 0x56253f984390 2024-08-31T07:21:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.619+0000 7fce592d1740 10 client.0 _ll_register_callbacks cb 0x56253fc5d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce4b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd56590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce4be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd62460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce42a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 0x56253fd56ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce4b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd56590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43276/0 (socket says 172.21.15.32:43276) 2024-08-31T07:21:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce4b400640 1 -- 172.21.15.32:0/2062599351 learned_addr learned my addr 172.21.15.32:0/2062599351 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce4b400640 1 -- 172.21.15.32:0/2062599351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 msgr2=0x56253fd56ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce4b400640 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 0x56253fd56ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce4b400640 1 -- 172.21.15.32:0/2062599351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 msgr2=0x56253fd62460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce4b400640 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd62460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce4b400640 1 -- 172.21.15.32:0/2062599351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253fc788b0 con 0x56253f97c1b0 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce4b400640 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd56590 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0cbcd0 tx=0x7fce3c0c84e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a82b6ff6d6ffdfa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.620+0000 7fce39600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.621+0000 7fce39600640 1 -- 172.21.15.32:0/2062599351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c049070 con 0x56253f97c1b0 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.621+0000 7fce39600640 1 -- 172.21.15.32:0/2062599351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7fce3c0cc230 con 0x56253f97c1b0 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.621+0000 7fce592d1740 1 -- 172.21.15.32:0/2062599351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce340054d0 con 0x56253f97c1b0 2024-08-31T07:21:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.621+0000 7fce39600640 1 -- 172.21.15.32:0/2062599351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fce3c0cb740 con 0x56253f97c1b0 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.621+0000 7fce592d1740 10 client.17376 fetch_fsmap learned FSMap version 38 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.621+0000 7fce592d1740 1 -- 172.21.15.32:0/2062599351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56253fd65180 con 0x56253f97c1b0 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.621+0000 7fce39600640 1 -- 172.21.15.32:0/2062599351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce340054d0 con 0x56253f97c1b0 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.622+0000 7fce39600640 1 -- 172.21.15.32:0/2062599351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) ==== 8398+0+0 (secure 0 0 0) 0x7fce3c0d03b0 con 0x56253f97c1b0 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.622+0000 7fce592d1740 10 client.17376 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.622+0000 7fce592d1740 10 client.17376 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.622+0000 7fce592d1740 20 client.17376 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.622+0000 7fce592d1740 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56253f9639f0 0x56253fd68790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.622+0000 7fce592d1740 4 client.17376 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.622+0000 7fce592d1740 1 -- 172.21.15.32:0/2062599351 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56253fcc3700 con 0x56253f9639f0 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.622+0000 7fce42a00640 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56253f9639f0 0x56253fd68790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.623+0000 7fce42a00640 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56253f9639f0 0x56253fd68790 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17f93978f673e238 server_cookie=e2dfd22c9d434137 in_seq=0 out_seq=0 2024-08-31T07:21:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.623+0000 7fce39600640 10 client.17376 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:21:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce39600640 1 -- 172.21.15.32:0/2062599351 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56253fcc3700 con 0x56253f9639f0 2024-08-31T07:21:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce39600640 10 client.17376 handle_command_reply: tid=0 2024-08-31T07:21:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 client.17376 shutdown 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 -- 172.21.15.32:0/2062599351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 msgr2=0x56253fd56590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd56590 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0cbcd0 tx=0x7fce3c0c84e0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 -- 172.21.15.32:0/2062599351 shutdown_connections 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56253f9639f0 0x56253fd68790 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd62460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f984390 0x56253fd56ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 --2- 172.21.15.32:0/2062599351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f97c1b0 0x56253fd56590 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 -- 172.21.15.32:0/2062599351 >> 172.21.15.32:0/2062599351 conn(0x56253f999460 msgr2=0x56253fd4db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce39600640 0 client.17376 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 -- 172.21.15.32:0/2062599351 shutdown_connections 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 1 -- 172.21.15.32:0/2062599351 wait complete. 2024-08-31T07:21:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.624+0000 7fce592d1740 20 client.17376 trim_cache size 0 max 16384 2024-08-31T07:21:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.638+0000 7fce592d1740 1 Processor -- start 2024-08-31T07:21:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.638+0000 7fce592d1740 1 -- start start 2024-08-31T07:21:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce592d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f97c1b0 0x56253fd51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce592d1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 0x56253fd4c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce592d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f9a5070 0x56253fd47e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce592d1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c1b5950 con 0x56253f984390 2024-08-31T07:21:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce592d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c1b7640 con 0x56253f97c1b0 2024-08-31T07:21:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce592d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c1b6020 con 0x56253f9a5070 2024-08-31T07:21:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce42a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 0x56253fd4c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce4b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f97c1b0 0x56253fd51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce42a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 0x56253fd4c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43282/0 (socket says 172.21.15.32:43282) 2024-08-31T07:21:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce42a00640 1 -- 172.21.15.32:0/1846756245 learned_addr learned my addr 172.21.15.32:0/1846756245 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce4be00640 1 --2- 172.21.15.32:0/1846756245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f9a5070 0x56253fd47e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce4b400640 1 -- 172.21.15.32:0/1846756245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f9a5070 msgr2=0x56253fd47e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce4b400640 1 --2- 172.21.15.32:0/1846756245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f9a5070 0x56253fd47e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce4b400640 1 -- 172.21.15.32:0/1846756245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 msgr2=0x56253fd4c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.639+0000 7fce4b400640 1 --2- 172.21.15.32:0/1846756245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 0x56253fd4c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.640+0000 7fce4b400640 1 -- 172.21.15.32:0/1846756245 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253fd49170 con 0x56253f97c1b0 2024-08-31T07:21:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.640+0000 7fce4b400640 1 --2- 172.21.15.32:0/1846756245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f97c1b0 0x56253fd51b40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0c8050 tx=0x7fce3c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=825d8b3c0d4c449c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.640+0000 7fce3be00640 1 -- 172.21.15.32:0/1846756245 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c079070 con 0x56253f97c1b0 2024-08-31T07:21:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.640+0000 7fce3be00640 1 -- 172.21.15.32:0/1846756245 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c0cbae0 con 0x56253f97c1b0 2024-08-31T07:21:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.640+0000 7fce3be00640 1 -- 172.21.15.32:0/1846756245 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0f2020 con 0x56253f97c1b0 2024-08-31T07:21:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.641+0000 7fce592d1740 1 -- 172.21.15.32:0/1846756245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f97c1b0 msgr2=0x56253fd51b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.641+0000 7fce592d1740 1 --2- 172.21.15.32:0/1846756245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f97c1b0 0x56253fd51b40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0c8050 tx=0x7fce3c02a410 comp rx=0 tx=0).stop 2024-08-31T07:21:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.641+0000 7fce592d1740 1 -- 172.21.15.32:0/1846756245 shutdown_connections 2024-08-31T07:21:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.641+0000 7fce592d1740 1 --2- 172.21.15.32:0/1846756245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f9a5070 0x56253fd47e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.641+0000 7fce592d1740 1 --2- 172.21.15.32:0/1846756245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 0x56253fd4c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.641+0000 7fce592d1740 1 --2- 172.21.15.32:0/1846756245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f97c1b0 0x56253fd51b40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.641+0000 7fce592d1740 1 -- 172.21.15.32:0/1846756245 >> 172.21.15.32:0/1846756245 conn(0x56253f999460 msgr2=0x56253fd4b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.641+0000 7fce592d1740 1 -- 172.21.15.32:0/1846756245 shutdown_connections 2024-08-31T07:21:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.641+0000 7fce592d1740 1 -- 172.21.15.32:0/1846756245 wait complete. 2024-08-31T07:21:44.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.642+0000 7fce592d1740 1 -- start start 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.642+0000 7fce592d1740 1 Processor -- start 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.642+0000 7fce592d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f97c1b0 0x56253fd5a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.642+0000 7fce592d1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 0x56253fd5a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce592d1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd5ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce592d1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c1b6eb0 con 0x56253f984390 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce592d1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c1b67b0 con 0x56253f9a5070 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce592d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c1b3950 con 0x56253f97c1b0 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce592d1740 10 client.0 _ll_register_callbacks cb 0x7fce3c03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce4b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f97c1b0 0x56253fd5a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce4b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f97c1b0 0x56253fd5a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35580/0 (socket says 172.21.15.32:35580) 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce4b400640 1 -- 172.21.15.32:0/445079258 learned_addr learned my addr 172.21.15.32:0/445079258 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce42a00640 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 0x56253fd5a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce4be00640 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd5ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce4b400640 1 -- 172.21.15.32:0/445079258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 msgr2=0x56253fd5ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce4b400640 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd5ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce4b400640 1 -- 172.21.15.32:0/445079258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 msgr2=0x56253fd5a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce4b400640 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 0x56253fd5a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.643+0000 7fce4b400640 1 -- 172.21.15.32:0/445079258 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253fdd2190 con 0x56253f97c1b0 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.644+0000 7fce4b400640 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f97c1b0 0x56253fd5a2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fce3c031440 tx=0x7fce3c0b73d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b14e42ec4f4b68 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.644+0000 7fce39600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.644+0000 7fce39600640 1 -- 172.21.15.32:0/445079258 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c079070 con 0x56253f97c1b0 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.644+0000 7fce592d1740 1 -- 172.21.15.32:0/445079258 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce3c0cbae0 con 0x56253f97c1b0 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.644+0000 7fce39600640 1 -- 172.21.15.32:0/445079258 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7fce3c0cc230 con 0x56253f97c1b0 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.644+0000 7fce39600640 1 -- 172.21.15.32:0/445079258 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fce3c0ea020 con 0x56253f97c1b0 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.644+0000 7fce592d1740 10 client.27397 fetch_fsmap learned FSMap version 38 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.644+0000 7fce592d1740 1 -- 172.21.15.32:0/445079258 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56253fdd2190 con 0x56253f97c1b0 2024-08-31T07:21:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.644+0000 7fce39600640 1 -- 172.21.15.32:0/445079258 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c0cbae0 con 0x56253f97c1b0 2024-08-31T07:21:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.645+0000 7fce39600640 1 -- 172.21.15.32:0/445079258 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 38) ==== 8398+0+0 (secure 0 0 0) 0x7fce3c0c7940 con 0x56253f97c1b0 2024-08-31T07:21:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.645+0000 7fce592d1740 10 client.27397 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T07:21:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.645+0000 7fce592d1740 10 client.27397 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:21:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.645+0000 7fce592d1740 20 client.27397 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.645+0000 7fce592d1740 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56253f938da0 0x56253fdd55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.645+0000 7fce592d1740 4 client.27397 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:21:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.645+0000 7fce592d1740 1 -- 172.21.15.32:0/445079258 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56253fcbf1a0 con 0x56253f938da0 2024-08-31T07:21:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.645+0000 7fce42a00640 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56253f938da0 0x56253fdd55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce42a00640 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56253f938da0 0x56253fdd55d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7764d7e41da1d69e server_cookie=715b81fd17f9c6a7 in_seq=0 out_seq=0 2024-08-31T07:21:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce39600640 10 client.27397 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:21:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce39600640 1 -- 172.21.15.32:0/445079258 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56253fcbf1a0 con 0x56253f938da0 2024-08-31T07:21:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce39600640 10 client.27397 handle_command_reply: tid=0 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 client.27397 shutdown 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 -- 172.21.15.32:0/445079258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f97c1b0 msgr2=0x56253fd5a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f97c1b0 0x56253fd5a2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fce3c031440 tx=0x7fce3c0b73d0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 -- 172.21.15.32:0/445079258 shutdown_connections 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56253f938da0 0x56253fdd55d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56253f9a5070 0x56253fd5ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56253f984390 0x56253fd5a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce39600640 0 client.27397 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 --2- 172.21.15.32:0/445079258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253f97c1b0 0x56253fd5a2f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 -- 172.21.15.32:0/445079258 >> 172.21.15.32:0/445079258 conn(0x56253f999460 msgr2=0x56253fd57690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 -- 172.21.15.32:0/445079258 shutdown_connections 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 1 -- 172.21.15.32:0/445079258 wait complete. 2024-08-31T07:21:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.646+0000 7fce592d1740 20 client.27397 trim_cache size 0 max 16384 2024-08-31T07:21:44.651 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:21:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.647+0000 7fce52600640 1 -- 172.21.15.32:0/1974394481 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 msgr2=0x7fce300b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.647+0000 7fce52600640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 0x7fce300b7230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fce44009fd0 tx=0x7fce44009500 comp rx=0 tx=0).stop 2024-08-31T07:21:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.647+0000 7fce52600640 1 -- 172.21.15.32:0/1974394481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c0776e0 msgr2=0x7fce4c1a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.647+0000 7fce52600640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c0776e0 0x7fce4c1a0830 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fce3c005940 tx=0x7fce3c0040c0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.647+0000 7fce52600640 1 -- 172.21.15.32:0/1974394481 shutdown_connections 2024-08-31T07:21:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.647+0000 7fce52600640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 0x7fce300b7230 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.647+0000 7fce52600640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce4c1144e0 0x7fce4c1a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.647+0000 7fce52600640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce4c078030 0x7fce4c1a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.647+0000 7fce52600640 1 --2- 172.21.15.32:0/1974394481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce4c0776e0 0x7fce4c1a0830 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.647+0000 7fce52600640 1 -- 172.21.15.32:0/1974394481 >> 172.21.15.32:0/1974394481 conn(0x7fce4c1015b0 msgr2=0x7fce4c1165f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.648+0000 7fce52600640 1 -- 172.21.15.32:0/1974394481 shutdown_connections 2024-08-31T07:21:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:44.648+0000 7fce52600640 1 -- 172.21.15.32:0/1974394481 wait complete. 2024-08-31T07:21:44.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 281934 2.89 MB/sec execute 333 sec latency 3240.042 ms 2024-08-31T07:21:45.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:21:45.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 283263 2.90 MB/sec execute 334 sec latency 3612.060 ms 2024-08-31T07:21:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:46.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:46 smithi032 ceph-mon[17252]: pgmap v644: 97 pgs: 97 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-08-31T07:21:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:46 smithi073 ceph-mon[21995]: pgmap v644: 97 pgs: 97 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-08-31T07:21:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:46 smithi084 ceph-mon[22387]: pgmap v644: 97 pgs: 97 active+clean; 204 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-08-31T07:21:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.715+0000 7fea8ec00640 1 -- 172.21.15.32:0/3621533553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea88103750 msgr2=0x7fea88103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.715+0000 7fea8ec00640 1 --2- 172.21.15.32:0/3621533553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea88103750 0x7fea88103b60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009900 tx=0x7fea7c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.716+0000 7fea8ec00640 1 -- 172.21.15.32:0/3621533553 shutdown_connections 2024-08-31T07:21:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.716+0000 7fea8ec00640 1 --2- 172.21.15.32:0/3621533553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea881099e0 0x7fea88110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.716+0000 7fea8ec00640 1 --2- 172.21.15.32:0/3621533553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea88109090 0x7fea881094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.716+0000 7fea8ec00640 1 --2- 172.21.15.32:0/3621533553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea88103750 0x7fea88103b60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.716+0000 7fea8ec00640 1 -- 172.21.15.32:0/3621533553 >> 172.21.15.32:0/3621533553 conn(0x7fea880ff1a0 msgr2=0x7fea881015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.716+0000 7fea8ec00640 1 -- 172.21.15.32:0/3621533553 shutdown_connections 2024-08-31T07:21:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.716+0000 7fea8ec00640 1 -- 172.21.15.32:0/3621533553 wait complete. 2024-08-31T07:21:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.717+0000 7fea8ec00640 1 Processor -- start 2024-08-31T07:21:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.717+0000 7fea8ec00640 1 -- start start 2024-08-31T07:21:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea88103750 0x7fea8819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8ec00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea88109090 0x7fea8819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8ec00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea881099e0 0x7fea8819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8ec00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fea88113700 con 0x7fea881099e0 2024-08-31T07:21:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea88113580 con 0x7fea88103750 2024-08-31T07:21:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8ec00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fea88113880 con 0x7fea88109090 2024-08-31T07:21:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea88109090 0x7fea8819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea88109090 0x7fea8819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46080/0 (socket says 172.21.15.32:46080) 2024-08-31T07:21:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8ce00640 1 -- 172.21.15.32:0/2287077158 learned_addr learned my addr 172.21.15.32:0/2287077158 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8d800640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea88103750 0x7fea8819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.718+0000 7fea8e200640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea881099e0 0x7fea8819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea8ce00640 1 -- 172.21.15.32:0/2287077158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea88103750 msgr2=0x7fea8819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea8ce00640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea88103750 0x7fea8819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea8ce00640 1 -- 172.21.15.32:0/2287077158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea881099e0 msgr2=0x7fea8819f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea8ce00640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea881099e0 0x7fea8819f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea8ce00640 1 -- 172.21.15.32:0/2287077158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8819ffe0 con 0x7fea88109090 2024-08-31T07:21:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea8ce00640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea88109090 0x7fea8819a610 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fea7400e970 tx=0x7fea7400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea86000640 1 -- 172.21.15.32:0/2287077158 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7400cd80 con 0x7fea88109090 2024-08-31T07:21:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea86000640 1 -- 172.21.15.32:0/2287077158 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea740035f0 con 0x7fea88109090 2024-08-31T07:21:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea8ec00640 1 -- 172.21.15.32:0/2287077158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea881a02d0 con 0x7fea88109090 2024-08-31T07:21:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea86000640 1 -- 172.21.15.32:0/2287077158 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74010690 con 0x7fea88109090 2024-08-31T07:21:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.719+0000 7fea8ec00640 1 -- 172.21.15.32:0/2287077158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881a6a30 con 0x7fea88109090 2024-08-31T07:21:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.723+0000 7fea86000640 1 -- 172.21.15.32:0/2287077158 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fea74003aa0 con 0x7fea88109090 2024-08-31T07:21:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.723+0000 7fea96e06740 1 Processor -- start 2024-08-31T07:21:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.724+0000 7fea96e06740 1 -- start start 2024-08-31T07:21:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.724+0000 7fea96e06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864430ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.724+0000 7fea96e06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 0x55864430efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.724+0000 7fea96e06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x55864430f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.724+0000 7fea96e06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5586442857b0 con 0x558643f401b0 2024-08-31T07:21:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.724+0000 7fea96e06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558644274d40 con 0x558643f69070 2024-08-31T07:21:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.724+0000 7fea96e06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558644278cf0 con 0x558643f48390 2024-08-31T07:21:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.724+0000 7fea8e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864430ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.725+0000 7fea8e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864430ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37822/0 (socket says 172.21.15.32:37822) 2024-08-31T07:21:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.725+0000 7fea8e200640 1 -- 172.21.15.32:0/3209983495 learned_addr learned my addr 172.21.15.32:0/3209983495 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.725+0000 7fea8ce00640 1 --2- 172.21.15.32:0/3209983495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x55864430f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.725+0000 7fea8d800640 1 --2- 172.21.15.32:0/3209983495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 0x55864430efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.725+0000 7fea8e200640 1 -- 172.21.15.32:0/3209983495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 msgr2=0x55864430f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.725+0000 7fea8e200640 1 --2- 172.21.15.32:0/3209983495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x55864430f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.725+0000 7fea8e200640 1 -- 172.21.15.32:0/3209983495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 msgr2=0x55864430efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.725+0000 7fea8e200640 1 --2- 172.21.15.32:0/3209983495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 0x55864430efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.725+0000 7fea8e200640 1 -- 172.21.15.32:0/3209983495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558643fd4650 con 0x558643f69070 2024-08-31T07:21:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.725+0000 7fea8d800640 1 --2- 172.21.15.32:0/3209983495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 0x55864430efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:21:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.726+0000 7fea8e200640 1 --2- 172.21.15.32:0/3209983495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864430ea70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fea80009fd0 tx=0x7fea800118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929f36ef5b24d1ac server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.726+0000 7fea86000640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fea700b4ca0 0x7fea700b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.726+0000 7fea86000640 1 -- 172.21.15.32:0/2287077158 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7fea740e5800 con 0x7fea88109090 2024-08-31T07:21:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.726+0000 7fea7aa00640 1 -- 172.21.15.32:0/3209983495 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80004d20 con 0x558643f69070 2024-08-31T07:21:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.726+0000 7fea8d800640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fea700b4ca0 0x7fea700b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.726+0000 7fea7aa00640 1 -- 172.21.15.32:0/3209983495 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea80004050 con 0x558643f69070 2024-08-31T07:21:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.726+0000 7fea7aa00640 1 -- 172.21.15.32:0/3209983495 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80005280 con 0x558643f69070 2024-08-31T07:21:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.727+0000 7fea96e06740 1 -- 172.21.15.32:0/3209983495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 msgr2=0x55864430ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.727+0000 7fea8d800640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fea700b4ca0 0x7fea700b71b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fea7c005ab0 tx=0x7fea7c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.727+0000 7fea96e06740 1 --2- 172.21.15.32:0/3209983495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864430ea70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fea80009fd0 tx=0x7fea800118d0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.727+0000 7fea96e06740 1 -- 172.21.15.32:0/3209983495 shutdown_connections 2024-08-31T07:21:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.727+0000 7fea96e06740 1 --2- 172.21.15.32:0/3209983495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x55864430f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.727+0000 7fea96e06740 1 --2- 172.21.15.32:0/3209983495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 0x55864430efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.727+0000 7fea96e06740 1 --2- 172.21.15.32:0/3209983495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864430ea70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.727+0000 7fea96e06740 1 -- 172.21.15.32:0/3209983495 >> 172.21.15.32:0/3209983495 conn(0x558643f5d460 msgr2=0x55864430c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.727+0000 7fea96e06740 1 -- 172.21.15.32:0/3209983495 shutdown_connections 2024-08-31T07:21:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.727+0000 7fea96e06740 1 -- 172.21.15.32:0/3209983495 wait complete. 2024-08-31T07:21:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.728+0000 7fea96e06740 1 -- start start 2024-08-31T07:21:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.728+0000 7fea96e06740 1 Processor -- start 2024-08-31T07:21:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea96e06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f401b0 0x55864431a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea96e06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f48390 0x55864431ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea96e06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864432a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea96e06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5586440d7700 con 0x558643f48390 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea96e06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5586441fbcf0 con 0x558643f69070 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea96e06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55864431d250 con 0x558643f401b0 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea96e06740 10 client.0 _ll_register_callbacks cb 0x558644221470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea8d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f401b0 0x55864431a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea8d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f401b0 0x55864431a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46108/0 (socket says 172.21.15.32:46108) 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea8d800640 1 -- 172.21.15.32:0/3583195207 learned_addr learned my addr 172.21.15.32:0/3583195207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea8ce00640 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f48390 0x55864431ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.729+0000 7fea8e200640 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864432a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.730+0000 7fea8d800640 1 -- 172.21.15.32:0/3583195207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 msgr2=0x55864432a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.730+0000 7fea8d800640 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864432a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.730+0000 7fea8d800640 1 -- 172.21.15.32:0/3583195207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f48390 msgr2=0x55864431ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.730+0000 7fea8d800640 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f48390 0x55864431ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.730+0000 7fea8d800640 1 -- 172.21.15.32:0/3583195207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864423c8b0 con 0x558643f401b0 2024-08-31T07:21:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.730+0000 7fea8d800640 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f401b0 0x55864431a6e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009900 tx=0x7fea7c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db63fc2994ff9f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.730+0000 7fea6f400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:21:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.730+0000 7fea6f400640 1 -- 172.21.15.32:0/3583195207 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c04d070 con 0x558643f401b0 2024-08-31T07:21:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.730+0000 7fea6f400640 1 -- 172.21.15.32:0/3583195207 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7fea7c004b80 con 0x558643f401b0 2024-08-31T07:21:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.730+0000 7fea96e06740 1 -- 172.21.15.32:0/3583195207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea80004050 con 0x558643f401b0 2024-08-31T07:21:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea6f400640 1 -- 172.21.15.32:0/3583195207 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fea7c059050 con 0x558643f401b0 2024-08-31T07:21:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea96e06740 10 client.27409 fetch_fsmap learned FSMap version 38 2024-08-31T07:21:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea96e06740 1 -- 172.21.15.32:0/3583195207 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55864432d6f0 con 0x558643f401b0 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea6f400640 1 -- 172.21.15.32:0/3583195207 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea80004050 con 0x558643f401b0 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea6f400640 1 -- 172.21.15.32:0/3583195207 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 38) ==== 8398+0+0 (secure 0 0 0) 0x7fea7c032540 con 0x558643f401b0 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea96e06740 10 client.27409 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea96e06740 10 client.27409 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea96e06740 20 client.27409 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea96e06740 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558643f279f0 0x558644330d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea96e06740 4 client.27409 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.731+0000 7fea96e06740 1 -- 172.21.15.32:0/3583195207 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558644287700 con 0x558643f279f0 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.732+0000 7fea8ce00640 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558643f279f0 0x558644330d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.732+0000 7fea8ce00640 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558643f279f0 0x558644330d50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84971ef128869836 server_cookie=1627064f5f82d9db in_seq=0 out_seq=0 2024-08-31T07:21:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.732+0000 7fea6f400640 10 client.27409 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:21:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea6f400640 1 -- 172.21.15.32:0/3583195207 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558644287700 con 0x558643f279f0 2024-08-31T07:21:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea6f400640 10 client.27409 handle_command_reply: tid=0 2024-08-31T07:21:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea96e06740 1 client.27409 shutdown 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea96e06740 1 -- 172.21.15.32:0/3583195207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f401b0 msgr2=0x55864431a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea96e06740 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f401b0 0x55864431a6e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009900 tx=0x7fea7c009260 comp rx=0 tx=0).stop 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea96e06740 1 -- 172.21.15.32:0/3583195207 shutdown_connections 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea96e06740 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558643f279f0 0x558644330d50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea96e06740 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864432a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea96e06740 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f48390 0x55864431ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea96e06740 1 --2- 172.21.15.32:0/3583195207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f401b0 0x55864431a6e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea96e06740 1 -- 172.21.15.32:0/3583195207 >> 172.21.15.32:0/3583195207 conn(0x558643f5d460 msgr2=0x558644311c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.733+0000 7fea6f400640 0 client.27409 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.734+0000 7fea96e06740 1 -- 172.21.15.32:0/3583195207 shutdown_connections 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.734+0000 7fea96e06740 1 -- 172.21.15.32:0/3583195207 wait complete. 2024-08-31T07:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.734+0000 7fea96e06740 20 client.27409 trim_cache size 0 max 16384 2024-08-31T07:21:46.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.750+0000 7fea96e06740 1 Processor -- start 2024-08-31T07:21:46.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.750+0000 7fea96e06740 1 -- start start 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea96e06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 0x558644314290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea96e06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x55864432a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea96e06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864432ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea96e06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fea881acfb0 con 0x558643f401b0 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea96e06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea881aeca0 con 0x558643f69070 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea96e06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fea881ad680 con 0x558643f48390 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea8e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864432ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea8d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 0x558644314290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x55864432a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea8e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864432ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37832/0 (socket says 172.21.15.32:37832) 2024-08-31T07:21:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea8e200640 1 -- 172.21.15.32:0/3829497261 learned_addr learned my addr 172.21.15.32:0/3829497261 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea8ce00640 1 -- 172.21.15.32:0/3829497261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 msgr2=0x55864432ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea8ce00640 1 --2- 172.21.15.32:0/3829497261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864432ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea8ce00640 1 -- 172.21.15.32:0/3829497261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 msgr2=0x558644314290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea8ce00640 1 --2- 172.21.15.32:0/3829497261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 0x558644314290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.751+0000 7fea8ce00640 1 -- 172.21.15.32:0/3829497261 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558644310d60 con 0x558643f48390 2024-08-31T07:21:46.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.752+0000 7fea8ce00640 1 --2- 172.21.15.32:0/3829497261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x55864432a900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fea740e2f40 tx=0x7fea740eb530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b2f8c7eb0a3f740 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:46.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.752+0000 7fea7a000640 1 -- 172.21.15.32:0/3829497261 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea740ab9f0 con 0x558643f48390 2024-08-31T07:21:46.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.752+0000 7fea7a000640 1 -- 172.21.15.32:0/3829497261 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea740ac5e0 con 0x558643f48390 2024-08-31T07:21:46.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.752+0000 7fea7a000640 1 -- 172.21.15.32:0/3829497261 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea740e31c0 con 0x558643f48390 2024-08-31T07:21:46.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 284585 2.91 MB/sec execute 335 sec latency 18.220 ms 2024-08-31T07:21:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.753+0000 7fea96e06740 1 -- 172.21.15.32:0/3829497261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 msgr2=0x55864432a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.753+0000 7fea96e06740 1 --2- 172.21.15.32:0/3829497261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x55864432a900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fea740e2f40 tx=0x7fea740eb530 comp rx=0 tx=0).stop 2024-08-31T07:21:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.753+0000 7fea96e06740 1 -- 172.21.15.32:0/3829497261 shutdown_connections 2024-08-31T07:21:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.753+0000 7fea96e06740 1 --2- 172.21.15.32:0/3829497261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f69070 0x55864432ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.753+0000 7fea96e06740 1 --2- 172.21.15.32:0/3829497261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x55864432a900 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.753+0000 7fea96e06740 1 --2- 172.21.15.32:0/3829497261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f401b0 0x558644314290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.753+0000 7fea96e06740 1 -- 172.21.15.32:0/3829497261 >> 172.21.15.32:0/3829497261 conn(0x558643f5d460 msgr2=0x558644313b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.753+0000 7fea96e06740 1 -- 172.21.15.32:0/3829497261 shutdown_connections 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.753+0000 7fea96e06740 1 -- 172.21.15.32:0/3829497261 wait complete. 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.754+0000 7fea96e06740 1 -- start start 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.754+0000 7fea96e06740 1 Processor -- start 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.754+0000 7fea96e06740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f401b0 0x558644323150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.754+0000 7fea96e06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x5586443236d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.754+0000 7fea96e06740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f69070 0x55864439a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea96e06740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fea881ae510 con 0x558643f69070 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea96e06740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea881ade10 con 0x558643f401b0 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea96e06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fea881aafd0 con 0x558643f48390 2024-08-31T07:21:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea96e06740 10 client.0 _ll_register_callbacks cb 0x7fea7c0095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x5586443236d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x5586443236d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46138/0 (socket says 172.21.15.32:46138) 2024-08-31T07:21:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea8d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f401b0 0x558644323150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea8ce00640 1 -- 172.21.15.32:0/1022387525 learned_addr learned my addr 172.21.15.32:0/1022387525 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea8e200640 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f69070 0x55864439a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea8d800640 1 -- 172.21.15.32:0/1022387525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 msgr2=0x5586443236d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea8d800640 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x5586443236d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea8d800640 1 -- 172.21.15.32:0/1022387525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f69070 msgr2=0x55864439a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea8d800640 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f69070 0x55864439a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.755+0000 7fea8d800640 1 -- 172.21.15.32:0/1022387525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864439ad20 con 0x558643f401b0 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.756+0000 7fea8d800640 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f401b0 0x558644323150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fea7c02a9e0 tx=0x7fea7c0326e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7094e5aabc925e82 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.756+0000 7fea6f400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.756+0000 7fea6f400640 1 -- 172.21.15.32:0/1022387525 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c061070 con 0x558643f401b0 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.756+0000 7fea6f400640 1 -- 172.21.15.32:0/1022387525 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7fea7c004980 con 0x558643f401b0 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.756+0000 7fea96e06740 1 -- 172.21.15.32:0/1022387525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea740ac5e0 con 0x558643f401b0 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.756+0000 7fea6f400640 1 -- 172.21.15.32:0/1022387525 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fea7c06d050 con 0x558643f401b0 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.756+0000 7fea96e06740 10 client.27407 fetch_fsmap learned FSMap version 38 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.756+0000 7fea96e06740 1 -- 172.21.15.32:0/1022387525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55864439bfa0 con 0x558643f401b0 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.757+0000 7fea6f400640 1 -- 172.21.15.32:0/1022387525 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea740ac5e0 con 0x558643f401b0 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.757+0000 7fea6f400640 1 -- 172.21.15.32:0/1022387525 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 38) ==== 8398+0+0 (secure 0 0 0) 0x7fea7c003cd0 con 0x558643f401b0 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.757+0000 7fea96e06740 10 client.27407 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.757+0000 7fea96e06740 10 client.27407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.757+0000 7fea96e06740 20 client.27407 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.757+0000 7fea96e06740 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558643efcda0 0x55864439e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:46.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.757+0000 7fea96e06740 4 client.27407 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:21:46.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.757+0000 7fea96e06740 1 -- 172.21.15.32:0/1022387525 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586442831a0 con 0x558643efcda0 2024-08-31T07:21:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.758+0000 7fea8ce00640 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558643efcda0 0x55864439e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.758+0000 7fea8ce00640 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558643efcda0 0x55864439e7f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bdbb320422b7929c server_cookie=6671baaa4f66a477 in_seq=0 out_seq=0 2024-08-31T07:21:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.758+0000 7fea6f400640 10 client.27407 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:21:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.758+0000 7fea6f400640 1 -- 172.21.15.32:0/1022387525 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586442831a0 con 0x558643efcda0 2024-08-31T07:21:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.758+0000 7fea6f400640 10 client.27407 handle_command_reply: tid=0 2024-08-31T07:21:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.758+0000 7fea96e06740 1 client.27407 shutdown 2024-08-31T07:21:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 1 -- 172.21.15.32:0/1022387525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f401b0 msgr2=0x558644323150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f401b0 0x558644323150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fea7c02a9e0 tx=0x7fea7c0326e0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 1 -- 172.21.15.32:0/1022387525 shutdown_connections 2024-08-31T07:21:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558643efcda0 0x55864439e7f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558643f69070 0x55864439a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558643f48390 0x5586443236d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 1 --2- 172.21.15.32:0/1022387525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558643f401b0 0x558644323150 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea6f400640 0 client.27407 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:21:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 1 -- 172.21.15.32:0/1022387525 >> 172.21.15.32:0/1022387525 conn(0x558643f5d460 msgr2=0x55864430faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 1 -- 172.21.15.32:0/1022387525 shutdown_connections 2024-08-31T07:21:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 1 -- 172.21.15.32:0/1022387525 wait complete. 2024-08-31T07:21:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.759+0000 7fea96e06740 20 client.27407 trim_cache size 0 max 16384 2024-08-31T07:21:46.762 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:21:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.760+0000 7fea8ec00640 1 -- 172.21.15.32:0/2287077158 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fea700b4ca0 msgr2=0x7fea700b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.760+0000 7fea8ec00640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fea700b4ca0 0x7fea700b71b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fea7c005ab0 tx=0x7fea7c005a20 comp rx=0 tx=0).stop 2024-08-31T07:21:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.760+0000 7fea8ec00640 1 -- 172.21.15.32:0/2287077158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea88109090 msgr2=0x7fea8819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.760+0000 7fea8ec00640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea88109090 0x7fea8819a610 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fea7400e970 tx=0x7fea7400ee30 comp rx=0 tx=0).stop 2024-08-31T07:21:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.760+0000 7fea8ec00640 1 -- 172.21.15.32:0/2287077158 shutdown_connections 2024-08-31T07:21:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.760+0000 7fea8ec00640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fea700b4ca0 0x7fea700b71b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.760+0000 7fea8ec00640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea881099e0 0x7fea8819f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.760+0000 7fea8ec00640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea88109090 0x7fea8819a610 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.760+0000 7fea8ec00640 1 --2- 172.21.15.32:0/2287077158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea88103750 0x7fea8819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:46.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.760+0000 7fea8ec00640 1 -- 172.21.15.32:0/2287077158 >> 172.21.15.32:0/2287077158 conn(0x7fea880ff1a0 msgr2=0x7fea881015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:46.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.761+0000 7fea8ec00640 1 -- 172.21.15.32:0/2287077158 shutdown_connections 2024-08-31T07:21:46.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:46.761+0000 7fea8ec00640 1 -- 172.21.15.32:0/2287077158 wait complete. 2024-08-31T07:21:47.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:21:47.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 285476 2.92 MB/sec execute 336 sec latency 269.703 ms 2024-08-31T07:21:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:48.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:48 smithi032 ceph-mon[17252]: pgmap v645: 97 pgs: 97 active+clean; 206 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 149 op/s 2024-08-31T07:21:48.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:48 smithi073 ceph-mon[21995]: pgmap v645: 97 pgs: 97 active+clean; 206 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 149 op/s 2024-08-31T07:21:48.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:48 smithi084 ceph-mon[22387]: pgmap v645: 97 pgs: 97 active+clean; 206 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 149 op/s 2024-08-31T07:21:48.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 285476 2.91 MB/sec execute 337 sec latency 1269.748 ms 2024-08-31T07:21:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.050+0000 7f2e37000640 1 -- 172.21.15.32:0/1612126650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e301088b0 msgr2=0x7f2e3010f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.050+0000 7f2e37000640 1 --2- 172.21.15.32:0/1612126650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e301088b0 0x7f2e3010f0e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009ca0 tx=0x7f2e2802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.050+0000 7f2e37000640 1 -- 172.21.15.32:0/1612126650 shutdown_connections 2024-08-31T07:21:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.050+0000 7f2e37000640 1 --2- 172.21.15.32:0/1612126650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e301088b0 0x7f2e3010f0e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.050+0000 7f2e37000640 1 --2- 172.21.15.32:0/1612126650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e30107f60 0x7f2e30108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.050+0000 7f2e37000640 1 --2- 172.21.15.32:0/1612126650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e30102620 0x7f2e30102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.050+0000 7f2e37000640 1 -- 172.21.15.32:0/1612126650 >> 172.21.15.32:0/1612126650 conn(0x7f2e300fe070 msgr2=0x7f2e301004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.050+0000 7f2e37000640 1 -- 172.21.15.32:0/1612126650 shutdown_connections 2024-08-31T07:21:49.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.050+0000 7f2e37000640 1 -- 172.21.15.32:0/1612126650 wait complete. 2024-08-31T07:21:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.052+0000 7f2e37000640 1 Processor -- start 2024-08-31T07:21:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.052+0000 7f2e37000640 1 -- start start 2024-08-31T07:21:49.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e37000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e30102620 0x7f2e300fe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e37000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e30107f60 0x7f2e300feae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e37000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e301088b0 0x7f2e300ff020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e37000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301125d0 con 0x7f2e30107f60 2024-08-31T07:21:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e37000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30112450 con 0x7f2e30102620 2024-08-31T07:21:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e37000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30112750 con 0x7f2e301088b0 2024-08-31T07:21:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e35c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e30102620 0x7f2e300fe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e35c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e30102620 0x7f2e300fe5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37842/0 (socket says 172.21.15.32:37842) 2024-08-31T07:21:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e35c00640 1 -- 172.21.15.32:0/754545577 learned_addr learned my addr 172.21.15.32:0/754545577 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e36600640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e301088b0 0x7f2e300ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.053+0000 7f2e35200640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e30107f60 0x7f2e300feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.054+0000 7f2e35c00640 1 -- 172.21.15.32:0/754545577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e301088b0 msgr2=0x7f2e300ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.054+0000 7f2e35c00640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e301088b0 0x7f2e300ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.054+0000 7f2e35c00640 1 -- 172.21.15.32:0/754545577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e30107f60 msgr2=0x7f2e300feae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.054+0000 7f2e35c00640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e30107f60 0x7f2e300feae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.054+0000 7f2e35c00640 1 -- 172.21.15.32:0/754545577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e301a8310 con 0x7f2e30102620 2024-08-31T07:21:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.054+0000 7f2e36600640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e301088b0 0x7f2e300ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.054+0000 7f2e35c00640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e30102620 0x7f2e300fe5a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400d950 tx=0x7f2e2400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.057+0000 7f2e2e000640 1 -- 172.21.15.32:0/754545577 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24017030 con 0x7f2e30102620 2024-08-31T07:21:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.057+0000 7f2e2e000640 1 -- 172.21.15.32:0/754545577 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e24004020 con 0x7f2e30102620 2024-08-31T07:21:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.057+0000 7f2e37000640 1 -- 172.21.15.32:0/754545577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e301a8600 con 0x7f2e30102620 2024-08-31T07:21:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.057+0000 7f2e2e000640 1 -- 172.21.15.32:0/754545577 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e240047a0 con 0x7f2e30102620 2024-08-31T07:21:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.057+0000 7f2e37000640 1 -- 172.21.15.32:0/754545577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e301a8b90 con 0x7f2e30102620 2024-08-31T07:21:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.060+0000 7f2e3f256740 1 Processor -- start 2024-08-31T07:21:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e3f256740 1 -- start start 2024-08-31T07:21:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e2e000640 1 -- 172.21.15.32:0/754545577 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2e24009a30 con 0x7f2e30102620 2024-08-31T07:21:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba3ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb66c1b0 0x563efba3b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 0x563efba3b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e3f256740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563efb9b17b0 con 0x563efb695070 2024-08-31T07:21:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e3f256740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563efb9a0d40 con 0x563efb674390 2024-08-31T07:21:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e3f256740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563efb9a4cf0 con 0x563efb66c1b0 2024-08-31T07:21:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e35200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 0x563efba3b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e36600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba3ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e35200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 0x563efba3b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37858/0 (socket says 172.21.15.32:37858) 2024-08-31T07:21:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e35200640 1 -- 172.21.15.32:0/4112800701 learned_addr learned my addr 172.21.15.32:0/4112800701 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.061+0000 7f2e35c00640 1 --2- 172.21.15.32:0/4112800701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb66c1b0 0x563efba3b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.062+0000 7f2e36600640 1 -- 172.21.15.32:0/4112800701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb66c1b0 msgr2=0x563efba3b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.062+0000 7f2e36600640 1 --2- 172.21.15.32:0/4112800701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb66c1b0 0x563efba3b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.062+0000 7f2e36600640 1 -- 172.21.15.32:0/4112800701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 msgr2=0x563efba3b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.062+0000 7f2e36600640 1 --2- 172.21.15.32:0/4112800701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 0x563efba3b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.062+0000 7f2e36600640 1 -- 172.21.15.32:0/4112800701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563efb700650 con 0x563efb695070 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.062+0000 7f2e36600640 1 --2- 172.21.15.32:0/4112800701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba3ab60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2e28005e30 tx=0x7f2e28009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755dceb66de6e4e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.063+0000 7f2e22a00640 1 -- 172.21.15.32:0/4112800701 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28031650 con 0x563efb695070 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.063+0000 7f2e22a00640 1 -- 172.21.15.32:0/4112800701 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e28036040 con 0x563efb695070 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.063+0000 7f2e22a00640 1 -- 172.21.15.32:0/4112800701 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28004510 con 0x563efb695070 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.063+0000 7f2e2e000640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e180b4d20 0x7f2e180b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.063+0000 7f2e2e000640 1 -- 172.21.15.32:0/754545577 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f2e240e4f20 con 0x7f2e30102620 2024-08-31T07:21:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.063+0000 7f2e35c00640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e180b4d20 0x7f2e180b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.063+0000 7f2e3f256740 1 -- 172.21.15.32:0/4112800701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 msgr2=0x563efba3ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.063+0000 7f2e3f256740 1 --2- 172.21.15.32:0/4112800701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba3ab60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2e28005e30 tx=0x7f2e28009950 comp rx=0 tx=0).stop 2024-08-31T07:21:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.064+0000 7f2e3f256740 1 -- 172.21.15.32:0/4112800701 shutdown_connections 2024-08-31T07:21:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.064+0000 7f2e3f256740 1 --2- 172.21.15.32:0/4112800701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 0x563efba3b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.064+0000 7f2e3f256740 1 --2- 172.21.15.32:0/4112800701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb66c1b0 0x563efba3b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.064+0000 7f2e3f256740 1 --2- 172.21.15.32:0/4112800701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba3ab60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.064+0000 7f2e3f256740 1 -- 172.21.15.32:0/4112800701 >> 172.21.15.32:0/4112800701 conn(0x563efb689460 msgr2=0x563efba38a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.064+0000 7f2e3f256740 1 -- 172.21.15.32:0/4112800701 shutdown_connections 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.064+0000 7f2e3f256740 1 -- 172.21.15.32:0/4112800701 wait complete. 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.064+0000 7f2e35c00640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e180b4d20 0x7f2e180b7230 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x563efba397e0 tx=0x7f2e240e89d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.064+0000 7f2e3f256740 1 -- start start 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.065+0000 7f2e3f256740 1 Processor -- start 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.065+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb66c1b0 0x563efba46700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.065+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 0x563efba46c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.065+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba52450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.065+0000 7f2e3f256740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563efb803700 con 0x563efb695070 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.065+0000 7f2e3f256740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563efb927cf0 con 0x563efb66c1b0 2024-08-31T07:21:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.065+0000 7f2e3f256740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563efba49160 con 0x563efb674390 2024-08-31T07:21:49.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.065+0000 7f2e3f256740 10 client.0 _ll_register_callbacks cb 0x563efb94d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:49.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e36600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba52450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e36600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba52450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53528/0 (socket says 172.21.15.32:53528) 2024-08-31T07:21:49.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e36600640 1 -- 172.21.15.32:0/1007134803 learned_addr learned my addr 172.21.15.32:0/1007134803 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:49.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e35200640 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 0x563efba46c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e36600640 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb66c1b0 0x563efba46700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e36600640 1 -- 172.21.15.32:0/1007134803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 msgr2=0x563efba46c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e36600640 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 0x563efba46c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e36600640 1 -- 172.21.15.32:0/1007134803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb66c1b0 msgr2=0x563efba46700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e36600640 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb66c1b0 0x563efba46700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e36600640 1 -- 172.21.15.32:0/1007134803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563efb9688b0 con 0x563efb695070 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.066+0000 7f2e35200640 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 0x563efba46c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.067+0000 7f2e36600640 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba52450 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2e280041b0 tx=0x7f2e28009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd613b73ffb090a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.067+0000 7f2e17400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.067+0000 7f2e17400640 1 -- 172.21.15.32:0/1007134803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28031660 con 0x563efb695070 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.067+0000 7f2e3f256740 1 -- 172.21.15.32:0/1007134803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e28036040 con 0x563efb695070 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.067+0000 7f2e17400640 1 -- 172.21.15.32:0/1007134803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f2e28031800 con 0x563efb695070 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.067+0000 7f2e17400640 1 -- 172.21.15.32:0/1007134803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2e28004b40 con 0x563efb695070 2024-08-31T07:21:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.067+0000 7f2e3f256740 10 client.17412 fetch_fsmap learned FSMap version 38 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.067+0000 7f2e3f256740 1 -- 172.21.15.32:0/1007134803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563efba55170 con 0x563efb695070 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.068+0000 7f2e17400640 1 -- 172.21.15.32:0/1007134803 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e28036040 con 0x563efb695070 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.068+0000 7f2e17400640 1 -- 172.21.15.32:0/1007134803 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) ==== 8398+0+0 (secure 0 0 0) 0x7f2e28051040 con 0x563efb695070 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.068+0000 7f2e3f256740 10 client.17412 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.068+0000 7f2e3f256740 10 client.17412 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.068+0000 7f2e3f256740 20 client.17412 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.068+0000 7f2e3f256740 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x563efb6539f0 0x563efba584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.068+0000 7f2e3f256740 4 client.17412 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.068+0000 7f2e3f256740 1 -- 172.21.15.32:0/1007134803 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563efb9b3700 con 0x563efb6539f0 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.068+0000 7f2e35200640 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x563efb6539f0 0x563efba584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.069+0000 7f2e35200640 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x563efb6539f0 0x563efba584d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60a0c0b2b114b675 server_cookie=9231539f000eea76 in_seq=0 out_seq=0 2024-08-31T07:21:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.069+0000 7f2e17400640 10 client.17412 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:21:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e17400640 1 -- 172.21.15.32:0/1007134803 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563efb9b3700 con 0x563efb6539f0 2024-08-31T07:21:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e17400640 10 client.17412 handle_command_reply: tid=0 2024-08-31T07:21:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 client.17412 shutdown 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 -- 172.21.15.32:0/1007134803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 msgr2=0x563efba52450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba52450 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2e280041b0 tx=0x7f2e28009360 comp rx=0 tx=0).stop 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 -- 172.21.15.32:0/1007134803 shutdown_connections 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb695070 0x563efba52450 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 0x563efba46c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x563efb6539f0 0x563efba584d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 --2- 172.21.15.32:0/1007134803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb66c1b0 0x563efba46700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 -- 172.21.15.32:0/1007134803 >> 172.21.15.32:0/1007134803 conn(0x563efb689460 msgr2=0x563efba3dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e17400640 0 client.17412 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 -- 172.21.15.32:0/1007134803 shutdown_connections 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.070+0000 7f2e3f256740 1 -- 172.21.15.32:0/1007134803 wait complete. 2024-08-31T07:21:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.071+0000 7f2e3f256740 20 client.17412 trim_cache size 0 max 16384 2024-08-31T07:21:49.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.084+0000 7f2e3f256740 1 Processor -- start 2024-08-31T07:21:49.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e3f256740 1 -- start start 2024-08-31T07:21:49.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba52350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 0x563efba38240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb695070 0x563efba38780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e3f256740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301b1250 con 0x563efb66c1b0 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e3f256740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301b2f40 con 0x563efb695070 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e3f256740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301b1920 con 0x563efb674390 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e36600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba52350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e36600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba52350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53530/0 (socket says 172.21.15.32:53530) 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e36600640 1 -- 172.21.15.32:0/1580916925 learned_addr learned my addr 172.21.15.32:0/1580916925 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e35200640 1 --2- 172.21.15.32:0/1580916925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 0x563efba38240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.085+0000 7f2e36600640 1 --2- 172.21.15.32:0/1580916925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb695070 0x563efba38780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.086+0000 7f2e36600640 1 -- 172.21.15.32:0/1580916925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 msgr2=0x563efba38240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.086+0000 7f2e36600640 1 --2- 172.21.15.32:0/1580916925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 0x563efba38240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.086+0000 7f2e36600640 1 -- 172.21.15.32:0/1580916925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb695070 msgr2=0x563efba38780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.086+0000 7f2e36600640 1 --2- 172.21.15.32:0/1580916925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb695070 0x563efba38780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.086+0000 7f2e36600640 1 -- 172.21.15.32:0/1580916925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563efba477c0 con 0x563efb66c1b0 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.086+0000 7f2e36600640 1 --2- 172.21.15.32:0/1580916925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba52350 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2e280048e0 tx=0x7f2e28005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d30e9e9b7d8244d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.086+0000 7f2e22000640 1 -- 172.21.15.32:0/1580916925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2802fd20 con 0x563efb66c1b0 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.086+0000 7f2e22000640 1 -- 172.21.15.32:0/1580916925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e28036030 con 0x563efb66c1b0 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.086+0000 7f2e22000640 1 -- 172.21.15.32:0/1580916925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28003b60 con 0x563efb66c1b0 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.087+0000 7f2e3f256740 1 -- 172.21.15.32:0/1580916925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 msgr2=0x563efba52350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.087+0000 7f2e3f256740 1 --2- 172.21.15.32:0/1580916925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba52350 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2e280048e0 tx=0x7f2e28005d80 comp rx=0 tx=0).stop 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.087+0000 7f2e3f256740 1 -- 172.21.15.32:0/1580916925 shutdown_connections 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.087+0000 7f2e3f256740 1 --2- 172.21.15.32:0/1580916925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb695070 0x563efba38780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.087+0000 7f2e3f256740 1 --2- 172.21.15.32:0/1580916925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb674390 0x563efba38240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.087+0000 7f2e3f256740 1 --2- 172.21.15.32:0/1580916925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba52350 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.087+0000 7f2e3f256740 1 -- 172.21.15.32:0/1580916925 >> 172.21.15.32:0/1580916925 conn(0x563efb689460 msgr2=0x563efba3fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.088+0000 7f2e3f256740 1 -- 172.21.15.32:0/1580916925 shutdown_connections 2024-08-31T07:21:49.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.088+0000 7f2e3f256740 1 -- 172.21.15.32:0/1580916925 wait complete. 2024-08-31T07:21:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.088+0000 7f2e3f256740 1 -- start start 2024-08-31T07:21:49.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.088+0000 7f2e3f256740 1 Processor -- start 2024-08-31T07:21:49.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba4ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 0x563efba4a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e3f256740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb695070 0x563efba4a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e3f256740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301b27b0 con 0x563efb66c1b0 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e3f256740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301b20b0 con 0x563efb674390 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e3f256740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301af250 con 0x563efb695070 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e3f256740 10 client.0 _ll_register_callbacks cb 0x7f2e280047b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e36600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba4ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e35200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 0x563efba4a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e36600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba4ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53536/0 (socket says 172.21.15.32:53536) 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e36600640 1 -- 172.21.15.32:0/4127792047 learned_addr learned my addr 172.21.15.32:0/4127792047 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e36600640 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb695070 0x563efba4a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e35200640 1 -- 172.21.15.32:0/4127792047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb695070 msgr2=0x563efba4a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e35200640 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb695070 0x563efba4a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e35200640 1 -- 172.21.15.32:0/4127792047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 msgr2=0x563efba4ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e35200640 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba4ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e35200640 1 -- 172.21.15.32:0/4127792047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563efba4afe0 con 0x563efb674390 2024-08-31T07:21:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.089+0000 7f2e36600640 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb695070 0x563efba4a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:21:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.090+0000 7f2e35200640 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 0x563efba4a1d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x563efb9a4e00 tx=0x7f2e1c012e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b7a7c0f3ee4393b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.090+0000 7f2e17400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:21:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.090+0000 7f2e17400640 1 -- 172.21.15.32:0/4127792047 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c020970 con 0x563efb674390 2024-08-31T07:21:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.090+0000 7f2e17400640 1 -- 172.21.15.32:0/4127792047 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f2e1c020b10 con 0x563efb674390 2024-08-31T07:21:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.090+0000 7f2e3f256740 1 -- 172.21.15.32:0/4127792047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e28036030 con 0x563efb674390 2024-08-31T07:21:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.090+0000 7f2e17400640 1 -- 172.21.15.32:0/4127792047 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2e1c020d10 con 0x563efb674390 2024-08-31T07:21:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.090+0000 7f2e3f256740 10 client.27425 fetch_fsmap learned FSMap version 38 2024-08-31T07:21:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.090+0000 7f2e3f256740 1 -- 172.21.15.32:0/4127792047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563efbac2db0 con 0x563efb674390 2024-08-31T07:21:49.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.091+0000 7f2e17400640 1 -- 172.21.15.32:0/4127792047 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e28036030 con 0x563efb674390 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.091+0000 7f2e17400640 1 -- 172.21.15.32:0/4127792047 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 38) ==== 8398+0+0 (secure 0 0 0) 0x7f2e1c026410 con 0x563efb674390 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.091+0000 7f2e3f256740 10 client.27425 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.091+0000 7f2e3f256740 10 client.27425 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.091+0000 7f2e3f256740 20 client.27425 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.091+0000 7f2e3f256740 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x563efb628da0 0x563efbac5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.091+0000 7f2e3f256740 4 client.27425 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.091+0000 7f2e3f256740 1 -- 172.21.15.32:0/4127792047 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563efb9af1a0 con 0x563efb628da0 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.092+0000 7f2e36600640 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x563efb628da0 0x563efbac5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.092+0000 7f2e36600640 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x563efb628da0 0x563efbac5550 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa5e8d30a1975d77 server_cookie=f7a1596125648ad4 in_seq=0 out_seq=0 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.092+0000 7f2e17400640 10 client.27425 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:21:49.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.092+0000 7f2e17400640 1 -- 172.21.15.32:0/4127792047 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563efb9af1a0 con 0x563efb628da0 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.092+0000 7f2e17400640 10 client.27425 handle_command_reply: tid=0 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.092+0000 7f2e3f256740 1 client.27425 shutdown 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 1 -- 172.21.15.32:0/4127792047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 msgr2=0x563efba4a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 0x563efba4a1d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x563efb9a4e00 tx=0x7f2e1c012e00 comp rx=0 tx=0).stop 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 1 -- 172.21.15.32:0/4127792047 shutdown_connections 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563efb695070 0x563efba4a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x563efb628da0 0x563efbac5550 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563efb674390 0x563efba4a1d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 1 --2- 172.21.15.32:0/4127792047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efb66c1b0 0x563efba4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e17400640 0 client.27425 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 1 -- 172.21.15.32:0/4127792047 >> 172.21.15.32:0/4127792047 conn(0x563efb689460 msgr2=0x563efba38240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 1 -- 172.21.15.32:0/4127792047 shutdown_connections 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 1 -- 172.21.15.32:0/4127792047 wait complete. 2024-08-31T07:21:49.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.093+0000 7f2e3f256740 20 client.27425 trim_cache size 0 max 16384 2024-08-31T07:21:49.097 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:21:49.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 -- 172.21.15.32:0/754545577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e180b4d20 msgr2=0x7f2e180b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:49.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e180b4d20 0x7f2e180b7230 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x563efba397e0 tx=0x7f2e240e89d0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 -- 172.21.15.32:0/754545577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e30102620 msgr2=0x7f2e300fe5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e30102620 0x7f2e300fe5a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400d950 tx=0x7f2e2400de10 comp rx=0 tx=0).stop 2024-08-31T07:21:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 -- 172.21.15.32:0/754545577 shutdown_connections 2024-08-31T07:21:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2e180b4d20 0x7f2e180b7230 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e301088b0 0x7f2e300ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e30107f60 0x7f2e300feae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 --2- 172.21.15.32:0/754545577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e30102620 0x7f2e300fe5a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 -- 172.21.15.32:0/754545577 >> 172.21.15.32:0/754545577 conn(0x7f2e300fe070 msgr2=0x7f2e30100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 -- 172.21.15.32:0/754545577 shutdown_connections 2024-08-31T07:21:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:49.094+0000 7f2e37000640 1 -- 172.21.15.32:0/754545577 wait complete. 2024-08-31T07:21:49.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 285476 2.90 MB/sec execute 338 sec latency 2269.835 ms 2024-08-31T07:21:49.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:21:50.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:50 smithi032 ceph-mon[17252]: pgmap v646: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-08-31T07:21:50.381 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:50.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:50 smithi073 ceph-mon[21995]: pgmap v646: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-08-31T07:21:50.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:50 smithi084 ceph-mon[22387]: pgmap v646: 97 pgs: 97 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-08-31T07:21:50.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 286797 2.91 MB/sec execute 339 sec latency 2637.816 ms 2024-08-31T07:21:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.080+0000 7f6923000640 1 -- 172.21.15.32:0/1252411034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691c1099e0 msgr2=0x7f691c110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.080+0000 7f6923000640 1 --2- 172.21.15.32:0/1252411034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691c1099e0 0x7f691c110210 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6914009ca0 tx=0x7f691402a8b0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.080+0000 7f6923000640 1 -- 172.21.15.32:0/1252411034 shutdown_connections 2024-08-31T07:21:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.080+0000 7f6923000640 1 --2- 172.21.15.32:0/1252411034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691c1099e0 0x7f691c110210 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.080+0000 7f6923000640 1 --2- 172.21.15.32:0/1252411034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f691c109090 0x7f691c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.080+0000 7f6923000640 1 --2- 172.21.15.32:0/1252411034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f691c103750 0x7f691c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.080+0000 7f6923000640 1 -- 172.21.15.32:0/1252411034 >> 172.21.15.32:0/1252411034 conn(0x7f691c0ff1e0 msgr2=0x7f691c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.080+0000 7f6923000640 1 -- 172.21.15.32:0/1252411034 shutdown_connections 2024-08-31T07:21:51.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.080+0000 7f6923000640 1 -- 172.21.15.32:0/1252411034 wait complete. 2024-08-31T07:21:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.082+0000 7f6923000640 1 Processor -- start 2024-08-31T07:21:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.082+0000 7f6923000640 1 -- start start 2024-08-31T07:21:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.082+0000 7f6923000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691c103750 0x7f691c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.082+0000 7f6923000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f691c109090 0x7f691c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.082+0000 7f6923000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f691c1099e0 0x7f691c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.082+0000 7f6923000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1136e0 con 0x7f691c1099e0 2024-08-31T07:21:51.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.082+0000 7f6923000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f691c113560 con 0x7f691c103750 2024-08-31T07:21:51.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.082+0000 7f6923000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f691c113860 con 0x7f691c109090 2024-08-31T07:21:51.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6921c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691c103750 0x7f691c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6922600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f691c1099e0 0x7f691c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6921200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f691c109090 0x7f691c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6921c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691c103750 0x7f691c0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37892/0 (socket says 172.21.15.32:37892) 2024-08-31T07:21:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6921c00640 1 -- 172.21.15.32:0/2861635320 learned_addr learned my addr 172.21.15.32:0/2861635320 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6921c00640 1 -- 172.21.15.32:0/2861635320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f691c109090 msgr2=0x7f691c0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6921c00640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f691c109090 0x7f691c0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6921c00640 1 -- 172.21.15.32:0/2861635320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f691c1099e0 msgr2=0x7f691c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6921c00640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f691c1099e0 0x7f691c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6921c00640 1 -- 172.21.15.32:0/2861635320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c1a8390 con 0x7f691c103750 2024-08-31T07:21:51.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.083+0000 7f6921200640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f691c109090 0x7f691c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:51.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.084+0000 7f6921c00640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691c103750 0x7f691c0754b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f691000e9f0 tx=0x7f691000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:51.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.084+0000 7f691a000640 1 -- 172.21.15.32:0/2861635320 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691000cd80 con 0x7f691c103750 2024-08-31T07:21:51.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.084+0000 7f691a000640 1 -- 172.21.15.32:0/2861635320 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69100108e0 con 0x7f691c103750 2024-08-31T07:21:51.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.084+0000 7f691a000640 1 -- 172.21.15.32:0/2861635320 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910018980 con 0x7f691c103750 2024-08-31T07:21:51.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.084+0000 7f6923000640 1 -- 172.21.15.32:0/2861635320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c1a8680 con 0x7f691c103750 2024-08-31T07:21:51.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.084+0000 7f6923000640 1 -- 172.21.15.32:0/2861635320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c1a8c10 con 0x7f691c103750 2024-08-31T07:21:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.087+0000 7f692b275740 1 Processor -- start 2024-08-31T07:21:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.088+0000 7f692b275740 1 -- start start 2024-08-31T07:21:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.088+0000 7f691a000640 1 -- 172.21.15.32:0/2861635320 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6910010430 con 0x7f691c103750 2024-08-31T07:21:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.088+0000 7f692b275740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f04ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.088+0000 7f692b275740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f04b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.088+0000 7f692b275740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f04b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.088+0000 7f692b275740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ca0efc17b0 con 0x55ca0eca5070 2024-08-31T07:21:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.088+0000 7f692b275740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ca0efb0d40 con 0x55ca0ec84390 2024-08-31T07:21:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.088+0000 7f692b275740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ca0efb4cf0 con 0x55ca0ec7c1b0 2024-08-31T07:21:51.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.088+0000 7f6921c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f04b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6922600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f04ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6922600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f04ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53570/0 (socket says 172.21.15.32:53570) 2024-08-31T07:21:51.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6922600640 1 -- 172.21.15.32:0/2081193531 learned_addr learned my addr 172.21.15.32:0/2081193531 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:51.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6921200640 1 --2- 172.21.15.32:0/2081193531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f04b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6922600640 1 -- 172.21.15.32:0/2081193531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 msgr2=0x55ca0f04b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:51.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6922600640 1 --2- 172.21.15.32:0/2081193531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f04b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6922600640 1 -- 172.21.15.32:0/2081193531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 msgr2=0x55ca0f04b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:51.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6922600640 1 --2- 172.21.15.32:0/2081193531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f04b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6922600640 1 -- 172.21.15.32:0/2081193531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca0ed10650 con 0x55ca0eca5070 2024-08-31T07:21:51.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6921c00640 1 --2- 172.21.15.32:0/2081193531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f04b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:21:51.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.089+0000 7f6921200640 1 --2- 172.21.15.32:0/2081193531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f04b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:51.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.090+0000 7f6922600640 1 --2- 172.21.15.32:0/2081193531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f04ab60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6914009ca0 tx=0x7f6914030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90494694344e8253 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:51.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.090+0000 7f691a000640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69040b4d20 0x7f69040b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.090+0000 7f690ea00640 1 -- 172.21.15.32:0/2081193531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914030800 con 0x55ca0eca5070 2024-08-31T07:21:51.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.090+0000 7f690ea00640 1 -- 172.21.15.32:0/2081193531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6914036030 con 0x55ca0eca5070 2024-08-31T07:21:51.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.090+0000 7f690ea00640 1 -- 172.21.15.32:0/2081193531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69140043c0 con 0x55ca0eca5070 2024-08-31T07:21:51.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.090+0000 7f691a000640 1 -- 172.21.15.32:0/2861635320 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f69100e57e0 con 0x7f691c103750 2024-08-31T07:21:51.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.090+0000 7f6921c00640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69040b4d20 0x7f69040b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.091+0000 7f692b275740 1 -- 172.21.15.32:0/2081193531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 msgr2=0x55ca0f04ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.091+0000 7f692b275740 1 --2- 172.21.15.32:0/2081193531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f04ab60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6914009ca0 tx=0x7f6914030fd0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.091+0000 7f6921c00640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69040b4d20 0x7f69040b7230 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55ca0f0497e0 tx=0x7f6910000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.091+0000 7f692b275740 1 -- 172.21.15.32:0/2081193531 shutdown_connections 2024-08-31T07:21:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.091+0000 7f692b275740 1 --2- 172.21.15.32:0/2081193531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f04b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.091+0000 7f692b275740 1 --2- 172.21.15.32:0/2081193531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f04b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.091+0000 7f692b275740 1 --2- 172.21.15.32:0/2081193531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f04ab60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.091+0000 7f692b275740 1 -- 172.21.15.32:0/2081193531 >> 172.21.15.32:0/2081193531 conn(0x55ca0ec99460 msgr2=0x55ca0f048a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.092+0000 7f692b275740 1 -- 172.21.15.32:0/2081193531 shutdown_connections 2024-08-31T07:21:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.092+0000 7f692b275740 1 -- 172.21.15.32:0/2081193531 wait complete. 2024-08-31T07:21:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.092+0000 7f692b275740 1 -- start start 2024-08-31T07:21:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.092+0000 7f692b275740 1 Processor -- start 2024-08-31T07:21:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f692b275740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f056780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f692b275740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec84390 0x55ca0f056cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f692b275740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f062600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f692b275740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ca0ee13700 con 0x55ca0eca5070 2024-08-31T07:21:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f692b275740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ca0ef37cf0 con 0x55ca0ec7c1b0 2024-08-31T07:21:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f692b275740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ca0f0592e0 con 0x55ca0ec84390 2024-08-31T07:21:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f692b275740 10 client.0 _ll_register_callbacks cb 0x55ca0ef5d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f6922600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f056780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f6922600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f056780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37918/0 (socket says 172.21.15.32:37918) 2024-08-31T07:21:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f6922600640 1 -- 172.21.15.32:0/541674016 learned_addr learned my addr 172.21.15.32:0/541674016 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f6921200640 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec84390 0x55ca0f056cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.093+0000 7f6922600640 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f062600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.094+0000 7f6921200640 1 -- 172.21.15.32:0/541674016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 msgr2=0x55ca0f056780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:51.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.094+0000 7f6921200640 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f056780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.094+0000 7f6921200640 1 -- 172.21.15.32:0/541674016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 msgr2=0x55ca0f062600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.094+0000 7f6921200640 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f062600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.094+0000 7f6921200640 1 -- 172.21.15.32:0/541674016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca0ef788b0 con 0x55ca0ec84390 2024-08-31T07:21:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.094+0000 7f6921200640 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec84390 0x55ca0f056cc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f691c06b4a0 tx=0x7f690800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2596f6fd9e07c7a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.094+0000 7f6903400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:21:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.094+0000 7f6903400640 1 -- 172.21.15.32:0/541674016 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69080120d0 con 0x55ca0ec84390 2024-08-31T07:21:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.094+0000 7f692b275740 1 -- 172.21.15.32:0/541674016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6914036030 con 0x55ca0ec84390 2024-08-31T07:21:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.095+0000 7f6903400640 1 -- 172.21.15.32:0/541674016 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f6908004030 con 0x55ca0ec84390 2024-08-31T07:21:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.095+0000 7f6903400640 1 -- 172.21.15.32:0/541674016 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6908012270 con 0x55ca0ec84390 2024-08-31T07:21:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.095+0000 7f692b275740 10 client.27442 fetch_fsmap learned FSMap version 39 2024-08-31T07:21:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.095+0000 7f692b275740 1 -- 172.21.15.32:0/541674016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ca0f0653e0 con 0x55ca0ec84390 2024-08-31T07:21:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.095+0000 7f6903400640 1 -- 172.21.15.32:0/541674016 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6914036030 con 0x55ca0ec84390 2024-08-31T07:21:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.095+0000 7f6903400640 1 -- 172.21.15.32:0/541674016 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7f6908026480 con 0x55ca0ec84390 2024-08-31T07:21:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.095+0000 7f692b275740 10 client.27442 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T07:21:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.095+0000 7f692b275740 10 client.27442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.096+0000 7f692b275740 20 client.27442 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.096+0000 7f692b275740 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ca0ec639f0 0x55ca0f068900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.096+0000 7f692b275740 4 client.27442 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:21:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.096+0000 7f692b275740 1 -- 172.21.15.32:0/541674016 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca0efc3700 con 0x55ca0ec639f0 2024-08-31T07:21:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.096+0000 7f6922600640 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ca0ec639f0 0x55ca0f068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.096+0000 7f6922600640 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ca0ec639f0 0x55ca0f068900 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f85a70da146e7a server_cookie=2ab073aa22321c3a in_seq=0 out_seq=0 2024-08-31T07:21:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.096+0000 7f6903400640 10 client.27442 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:51.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.097+0000 7f6903400640 1 -- 172.21.15.32:0/541674016 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca0efc3700 con 0x55ca0ec639f0 2024-08-31T07:21:51.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.097+0000 7f6903400640 10 client.27442 handle_command_reply: tid=0 2024-08-31T07:21:51.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 client.27442 shutdown 2024-08-31T07:21:51.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 -- 172.21.15.32:0/541674016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec84390 msgr2=0x55ca0f056cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:51.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec84390 0x55ca0f056cc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f691c06b4a0 tx=0x7f690800f790 comp rx=0 tx=0).stop 2024-08-31T07:21:51.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 -- 172.21.15.32:0/541674016 shutdown_connections 2024-08-31T07:21:51.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ca0ec639f0 0x55ca0f068900 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f062600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec84390 0x55ca0f056cc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 --2- 172.21.15.32:0/541674016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f056780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f6903400640 0 client.27442 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:51.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 -- 172.21.15.32:0/541674016 >> 172.21.15.32:0/541674016 conn(0x55ca0ec99460 msgr2=0x55ca0f04dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:51.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 -- 172.21.15.32:0/541674016 shutdown_connections 2024-08-31T07:21:51.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 1 -- 172.21.15.32:0/541674016 wait complete. 2024-08-31T07:21:51.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.098+0000 7f692b275740 20 client.27442 trim_cache size 0 max 16384 2024-08-31T07:21:51.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.115+0000 7f692b275740 1 Processor -- start 2024-08-31T07:21:51.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.115+0000 7f692b275740 1 -- start start 2024-08-31T07:21:51.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f692b275740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f050380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f692b275740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0ec84390 0x55ca0f062500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f692b275740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0eca5070 0x55ca0f062a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f692b275740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1b1400 con 0x55ca0ec84390 2024-08-31T07:21:51.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f692b275740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1b3060 con 0x55ca0ec7c1b0 2024-08-31T07:21:51.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f692b275740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1b1ad0 con 0x55ca0eca5070 2024-08-31T07:21:51.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f6921200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0ec84390 0x55ca0f062500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f6922600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f050380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f6921200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0ec84390 0x55ca0f062500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53592/0 (socket says 172.21.15.32:53592) 2024-08-31T07:21:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f6921200640 1 -- 172.21.15.32:0/3046074489 learned_addr learned my addr 172.21.15.32:0/3046074489 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.116+0000 7f6922600640 1 --2- 172.21.15.32:0/3046074489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0eca5070 0x55ca0f062a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.117+0000 7f6921200640 1 -- 172.21.15.32:0/3046074489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0eca5070 msgr2=0x55ca0f062a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.117+0000 7f6921200640 1 --2- 172.21.15.32:0/3046074489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0eca5070 0x55ca0f062a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.117+0000 7f6921200640 1 -- 172.21.15.32:0/3046074489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 msgr2=0x55ca0f050380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.117+0000 7f6921200640 1 --2- 172.21.15.32:0/3046074489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f050380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.117+0000 7f6921200640 1 -- 172.21.15.32:0/3046074489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca0f044200 con 0x55ca0ec84390 2024-08-31T07:21:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.117+0000 7f6921200640 1 --2- 172.21.15.32:0/3046074489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0ec84390 0x55ca0f062500 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6908000c00 tx=0x7f690800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a533fa215a9d27 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.117+0000 7f690e000640 1 -- 172.21.15.32:0/3046074489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908026540 con 0x55ca0ec84390 2024-08-31T07:21:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.117+0000 7f690e000640 1 -- 172.21.15.32:0/3046074489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6908005560 con 0x55ca0ec84390 2024-08-31T07:21:51.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.117+0000 7f690e000640 1 -- 172.21.15.32:0/3046074489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908003960 con 0x55ca0ec84390 2024-08-31T07:21:51.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.118+0000 7f692b275740 1 -- 172.21.15.32:0/3046074489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0ec84390 msgr2=0x55ca0f062500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:51.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.118+0000 7f692b275740 1 --2- 172.21.15.32:0/3046074489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0ec84390 0x55ca0f062500 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6908000c00 tx=0x7f690800fc90 comp rx=0 tx=0).stop 2024-08-31T07:21:51.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.118+0000 7f692b275740 1 -- 172.21.15.32:0/3046074489 shutdown_connections 2024-08-31T07:21:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.118+0000 7f692b275740 1 --2- 172.21.15.32:0/3046074489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0eca5070 0x55ca0f062a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.118+0000 7f692b275740 1 --2- 172.21.15.32:0/3046074489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0ec84390 0x55ca0f062500 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.118+0000 7f692b275740 1 --2- 172.21.15.32:0/3046074489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f050380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.118+0000 7f692b275740 1 -- 172.21.15.32:0/3046074489 >> 172.21.15.32:0/3046074489 conn(0x55ca0ec99460 msgr2=0x55ca0f04fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.118+0000 7f692b275740 1 -- 172.21.15.32:0/3046074489 shutdown_connections 2024-08-31T07:21:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.118+0000 7f692b275740 1 -- 172.21.15.32:0/3046074489 wait complete. 2024-08-31T07:21:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.119+0000 7f692b275740 1 -- start start 2024-08-31T07:21:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.119+0000 7f692b275740 1 Processor -- start 2024-08-31T07:21:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f692b275740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f05b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f692b275740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f05bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f692b275740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f05c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f692b275740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1b2960 con 0x55ca0eca5070 2024-08-31T07:21:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f692b275740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1b2260 con 0x55ca0ec84390 2024-08-31T07:21:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f692b275740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1af400 con 0x55ca0ec7c1b0 2024-08-31T07:21:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f692b275740 10 client.0 _ll_register_callbacks cb 0x7f6908012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f6921200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f05bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f6921200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f05bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37938/0 (socket says 172.21.15.32:37938) 2024-08-31T07:21:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f6921200640 1 -- 172.21.15.32:0/2388950333 learned_addr learned my addr 172.21.15.32:0/2388950333 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f6922600640 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f05b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.120+0000 7f6922600640 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f05c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f6921200640 1 -- 172.21.15.32:0/2388950333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 msgr2=0x55ca0f05b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f6921200640 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f05b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f6921200640 1 -- 172.21.15.32:0/2388950333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 msgr2=0x55ca0f05c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f6921200640 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f05c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f6921200640 1 -- 172.21.15.32:0/2388950333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca0f0d3340 con 0x55ca0ec84390 2024-08-31T07:21:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f6922600640 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f05c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:21:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f6921200640 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f05bb20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6908009700 tx=0x7f6908005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c33c73d1dec107cf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f6903400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:21:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f6903400640 1 -- 172.21.15.32:0/2388950333 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69080052d0 con 0x55ca0ec84390 2024-08-31T07:21:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f6903400640 1 -- 172.21.15.32:0/2388950333 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f6908018030 con 0x55ca0ec84390 2024-08-31T07:21:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.121+0000 7f692b275740 1 -- 172.21.15.32:0/2388950333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6908005560 con 0x55ca0ec84390 2024-08-31T07:21:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f6903400640 1 -- 172.21.15.32:0/2388950333 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f69080035f0 con 0x55ca0ec84390 2024-08-31T07:21:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f692b275740 10 client.27443 fetch_fsmap learned FSMap version 39 2024-08-31T07:21:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f692b275740 1 -- 172.21.15.32:0/2388950333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ca0f0d45a0 con 0x55ca0ec84390 2024-08-31T07:21:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f6903400640 1 -- 172.21.15.32:0/2388950333 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6908005560 con 0x55ca0ec84390 2024-08-31T07:21:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f6903400640 1 -- 172.21.15.32:0/2388950333 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7f690801c7d0 con 0x55ca0ec84390 2024-08-31T07:21:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f692b275740 10 client.27443 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T07:21:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f692b275740 10 client.27443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f692b275740 20 client.27443 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f692b275740 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ca0ec38da0 0x55ca0f0d6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f692b275740 4 client.27443 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:21:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.122+0000 7f692b275740 1 -- 172.21.15.32:0/2388950333 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca0efbf1a0 con 0x55ca0ec38da0 2024-08-31T07:21:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.123+0000 7f6922600640 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ca0ec38da0 0x55ca0f0d6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.123+0000 7f6922600640 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ca0ec38da0 0x55ca0f0d6cf0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9ca5e12e2d2658 server_cookie=6cc7889b0c9685a1 in_seq=0 out_seq=0 2024-08-31T07:21:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.123+0000 7f6903400640 10 client.27443 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.123+0000 7f6903400640 1 -- 172.21.15.32:0/2388950333 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca0efbf1a0 con 0x55ca0ec38da0 2024-08-31T07:21:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.123+0000 7f6903400640 10 client.27443 handle_command_reply: tid=0 2024-08-31T07:21:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 client.27443 shutdown 2024-08-31T07:21:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 -- 172.21.15.32:0/2388950333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 msgr2=0x55ca0f05bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f05bb20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6908009700 tx=0x7f6908005ab0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 -- 172.21.15.32:0/2388950333 shutdown_connections 2024-08-31T07:21:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ca0ec38da0 0x55ca0f0d6cf0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ca0eca5070 0x55ca0f05c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca0ec84390 0x55ca0f05bb20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 --2- 172.21.15.32:0/2388950333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ca0ec7c1b0 0x55ca0f05b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 -- 172.21.15.32:0/2388950333 >> 172.21.15.32:0/2388950333 conn(0x55ca0ec99460 msgr2=0x55ca0f04bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f6903400640 0 client.27443 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 -- 172.21.15.32:0/2388950333 shutdown_connections 2024-08-31T07:21:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 1 -- 172.21.15.32:0/2388950333 wait complete. 2024-08-31T07:21:51.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.124+0000 7f692b275740 20 client.27443 trim_cache size 0 max 16384 2024-08-31T07:21:51.137 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:21:51.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 -- 172.21.15.32:0/2861635320 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69040b4d20 msgr2=0x7f69040b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:51.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69040b4d20 0x7f69040b7230 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55ca0f0497e0 tx=0x7f6910000bc0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 -- 172.21.15.32:0/2861635320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691c103750 msgr2=0x7f691c0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691c103750 0x7f691c0754b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f691000e9f0 tx=0x7f691000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 -- 172.21.15.32:0/2861635320 shutdown_connections 2024-08-31T07:21:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69040b4d20 0x7f69040b7230 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f691c1099e0 0x7f691c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f691c109090 0x7f691c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 --2- 172.21.15.32:0/2861635320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f691c103750 0x7f691c0754b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:51.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 -- 172.21.15.32:0/2861635320 >> 172.21.15.32:0/2861635320 conn(0x7f691c0ff1e0 msgr2=0x7f691c1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:51.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 -- 172.21.15.32:0/2861635320 shutdown_connections 2024-08-31T07:21:51.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:51.125+0000 7f6923000640 1 -- 172.21.15.32:0/2861635320 wait complete. 2024-08-31T07:21:51.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 288074 2.92 MB/sec execute 340 sec latency 18.319 ms 2024-08-31T07:21:52.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:52 smithi032 ceph-mon[17252]: pgmap v647: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-08-31T07:21:52.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:52 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:52 smithi073 ceph-mon[21995]: pgmap v647: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-08-31T07:21:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:52 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:52 smithi084 ceph-mon[22387]: pgmap v647: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-08-31T07:21:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:52 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:21:52.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 289473 2.93 MB/sec execute 341 sec latency 58.581 ms 2024-08-31T07:21:53.051 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:21:53.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:21:53.479 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:53.753 INFO:tasks.workunit.client.0.smithi032.stdout: 1 291016 2.96 MB/sec execute 342 sec latency 94.480 ms 2024-08-31T07:21:54.324 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:54 smithi032 ceph-mon[17252]: pgmap v648: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-08-31T07:21:54.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.425+0000 7f999da00640 1 -- 172.21.15.32:0/3706975684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f999810bbf0 msgr2=0x7f9998112420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:54.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.425+0000 7f999da00640 1 --2- 172.21.15.32:0/3706975684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f999810bbf0 0x7f9998112420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9990009be0 tx=0x7f999002a6d0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.425+0000 7f999da00640 1 -- 172.21.15.32:0/3706975684 shutdown_connections 2024-08-31T07:21:54.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.425+0000 7f999da00640 1 --2- 172.21.15.32:0/3706975684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f999810bbf0 0x7f9998112420 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.425+0000 7f999da00640 1 --2- 172.21.15.32:0/3706975684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999810b2a0 0x7f999810b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.425+0000 7f999da00640 1 --2- 172.21.15.32:0/3706975684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9998105960 0x7f9998105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.425+0000 7f999da00640 1 -- 172.21.15.32:0/3706975684 >> 172.21.15.32:0/3706975684 conn(0x7f99981013d0 msgr2=0x7f9998103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:54.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.425+0000 7f999da00640 1 -- 172.21.15.32:0/3706975684 shutdown_connections 2024-08-31T07:21:54.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.425+0000 7f999da00640 1 -- 172.21.15.32:0/3706975684 wait complete. 2024-08-31T07:21:54.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.426+0000 7f999da00640 1 Processor -- start 2024-08-31T07:21:54.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.426+0000 7f999da00640 1 -- start start 2024-08-31T07:21:54.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9998105960 0x7f999819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f999810b2a0 0x7f999819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f999810bbf0 0x7f99981a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99981158b0 con 0x7f999810b2a0 2024-08-31T07:21:54.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9998115730 con 0x7f9998105960 2024-08-31T07:21:54.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999da00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9998115a30 con 0x7f999810bbf0 2024-08-31T07:21:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f999810bbf0 0x7f99981a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f9997400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f999810b2a0 0x7f999819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f999810bbf0 0x7f99981a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46226/0 (socket says 172.21.15.32:46226) 2024-08-31T07:21:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999d000640 1 -- 172.21.15.32:0/2218632156 learned_addr learned my addr 172.21.15.32:0/2218632156 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f9997e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9998105960 0x7f999819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999d000640 1 -- 172.21.15.32:0/2218632156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9998105960 msgr2=0x7f999819c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999d000640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9998105960 0x7f999819c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999d000640 1 -- 172.21.15.32:0/2218632156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f999810b2a0 msgr2=0x7f999819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999d000640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f999810b2a0 0x7f999819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.427+0000 7f999d000640 1 -- 172.21.15.32:0/2218632156 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99981a2190 con 0x7f999810bbf0 2024-08-31T07:21:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.428+0000 7f999d000640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f999810bbf0 0x7f99981a1a60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9990004af0 tx=0x7f9990003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.428+0000 7f9994c00640 1 -- 172.21.15.32:0/2218632156 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990041020 con 0x7f999810bbf0 2024-08-31T07:21:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.428+0000 7f9994c00640 1 -- 172.21.15.32:0/2218632156 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9990003e10 con 0x7f999810bbf0 2024-08-31T07:21:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.428+0000 7f999da00640 1 -- 172.21.15.32:0/2218632156 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99981a2420 con 0x7f999810bbf0 2024-08-31T07:21:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.428+0000 7f9994c00640 1 -- 172.21.15.32:0/2218632156 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990036040 con 0x7f999810bbf0 2024-08-31T07:21:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.428+0000 7f999da00640 1 -- 172.21.15.32:0/2218632156 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99981a8be0 con 0x7f999810bbf0 2024-08-31T07:21:54.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:54 smithi073 ceph-mon[21995]: pgmap v648: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-08-31T07:21:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.432+0000 7f99a5dc5740 1 Processor -- start 2024-08-31T07:21:54.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.432+0000 7f9994c00640 1 -- 172.21.15.32:0/2218632156 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9990044020 con 0x7f999810bbf0 2024-08-31T07:21:54.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f99a5dc5740 1 -- start start 2024-08-31T07:21:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcfc0350 0x55fafd3667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf97490 0x55fafd366d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 0x55fafd367250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fafd2dc750 con 0x55fafcf97490 2024-08-31T07:21:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fafd2cbce0 con 0x55fafcfc0350 2024-08-31T07:21:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fafd2cfc90 con 0x55fafcf9f670 2024-08-31T07:21:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f9997400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcfc0350 0x55fafd3667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f9997e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf97490 0x55fafd366d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f9997400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcfc0350 0x55fafd3667d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37960/0 (socket says 172.21.15.32:37960) 2024-08-31T07:21:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f9997400640 1 -- 172.21.15.32:0/4155410550 learned_addr learned my addr 172.21.15.32:0/4155410550 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.433+0000 7f9997400640 1 --2- 172.21.15.32:0/4155410550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 0x55fafd367250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.434+0000 7f9997400640 1 -- 172.21.15.32:0/4155410550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 msgr2=0x55fafd367250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.434+0000 7f9997400640 1 --2- 172.21.15.32:0/4155410550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 0x55fafd367250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.434+0000 7f9997400640 1 -- 172.21.15.32:0/4155410550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf97490 msgr2=0x55fafd366d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.434+0000 7f9997400640 1 --2- 172.21.15.32:0/4155410550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf97490 0x55fafd366d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.434+0000 7f9997400640 1 -- 172.21.15.32:0/4155410550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fafd02b930 con 0x55fafcfc0350 2024-08-31T07:21:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.434+0000 7f9997e00640 1 --2- 172.21.15.32:0/4155410550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf97490 0x55fafd366d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.434+0000 7f9997400640 1 --2- 172.21.15.32:0/4155410550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcfc0350 0x55fafd3667d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9984009820 tx=0x7f99840097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ad0ad30491d32d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.435+0000 7f998d600640 1 -- 172.21.15.32:0/4155410550 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984004020 con 0x55fafcfc0350 2024-08-31T07:21:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.435+0000 7f998d600640 1 -- 172.21.15.32:0/4155410550 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99840054d0 con 0x55fafcfc0350 2024-08-31T07:21:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.435+0000 7f9994c00640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f997c0b4ca0 0x7f997c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.435+0000 7f998d600640 1 -- 172.21.15.32:0/4155410550 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984004bc0 con 0x55fafcfc0350 2024-08-31T07:21:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.435+0000 7f9994c00640 1 -- 172.21.15.32:0/2218632156 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f99901060c0 con 0x7f999810bbf0 2024-08-31T07:21:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.435+0000 7f9997e00640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f997c0b4ca0 0x7f997c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.436+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4155410550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcfc0350 msgr2=0x55fafd3667d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.436+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/4155410550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcfc0350 0x55fafd3667d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9984009820 tx=0x7f99840097b0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.436+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4155410550 shutdown_connections 2024-08-31T07:21:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.436+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/4155410550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 0x55fafd367250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.436+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/4155410550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf97490 0x55fafd366d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.436+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/4155410550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcfc0350 0x55fafd3667d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.436+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4155410550 >> 172.21.15.32:0/4155410550 conn(0x55fafcfb4740 msgr2=0x55fafd3646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.436+0000 7f9997e00640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f997c0b4ca0 0x7f997c0b71b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55fafd365450 tx=0x7f9988005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.436+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4155410550 shutdown_connections 2024-08-31T07:21:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.436+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4155410550 wait complete. 2024-08-31T07:21:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.437+0000 7f99a5dc5740 1 -- start start 2024-08-31T07:21:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.437+0000 7f99a5dc5740 1 Processor -- start 2024-08-31T07:21:54.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.437+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd372470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.437+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 0x55fafd372a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd37e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fafd12e890 con 0x55fafcf9f670 2024-08-31T07:21:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fafd252cd0 con 0x55fafcf97490 2024-08-31T07:21:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fafd374fb0 con 0x55fafcfc0350 2024-08-31T07:21:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f99a5dc5740 10 client.0 _ll_register_callbacks cb 0x55fafd2c6ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f9997e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd372470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f9997e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd372470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37974/0 (socket says 172.21.15.32:37974) 2024-08-31T07:21:54.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f9997e00640 1 -- 172.21.15.32:0/4092703825 learned_addr learned my addr 172.21.15.32:0/4092703825 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f999d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd37e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f9997400640 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 0x55fafd372a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f999d000640 1 -- 172.21.15.32:0/4092703825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 msgr2=0x55fafd372470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f999d000640 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd372470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f999d000640 1 -- 172.21.15.32:0/4092703825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 msgr2=0x55fafd372a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f999d000640 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 0x55fafd372a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f999d000640 1 -- 172.21.15.32:0/4092703825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fafd29c1b0 con 0x55fafcfc0350 2024-08-31T07:21:54.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f9997400640 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 0x55fafd372a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:21:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.438+0000 7f9997e00640 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd372470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.439+0000 7f999d000640 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd37e2d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9990103800 tx=0x7f99900cbfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=706404f2ab734120 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.439+0000 7f9982000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:21:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.439+0000 7f9982000640 1 -- 172.21.15.32:0/4092703825 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99900ccea0 con 0x55fafcfc0350 2024-08-31T07:21:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.439+0000 7f9982000640 1 -- 172.21.15.32:0/4092703825 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f99900d5630 con 0x55fafcfc0350 2024-08-31T07:21:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.439+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4092703825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99840054d0 con 0x55fafcfc0350 2024-08-31T07:21:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.439+0000 7f9982000640 1 -- 172.21.15.32:0/4092703825 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f99900f3020 con 0x55fafcfc0350 2024-08-31T07:21:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.439+0000 7f99a5dc5740 10 client.27460 fetch_fsmap learned FSMap version 39 2024-08-31T07:21:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.439+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4092703825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fafd381080 con 0x55fafcfc0350 2024-08-31T07:21:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.439+0000 7f9982000640 1 -- 172.21.15.32:0/4092703825 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99840054d0 con 0x55fafcfc0350 2024-08-31T07:21:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.440+0000 7f9982000640 1 -- 172.21.15.32:0/4092703825 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7f9990004b40 con 0x55fafcfc0350 2024-08-31T07:21:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.440+0000 7f99a5dc5740 10 client.27460 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T07:21:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.440+0000 7f99a5dc5740 10 client.27460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.440+0000 7f99a5dc5740 20 client.27460 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.440+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fafcf7ecd0 0x55fafd384520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.440+0000 7f99a5dc5740 4 client.27460 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:21:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.440+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4092703825 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fafd2de6a0 con 0x55fafcf7ecd0 2024-08-31T07:21:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.440+0000 7f9997e00640 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fafcf7ecd0 0x55fafd384520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.441+0000 7f9997e00640 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fafcf7ecd0 0x55fafd384520 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe78539721928774 server_cookie=d615b75e7f1fc8e7 in_seq=0 out_seq=0 2024-08-31T07:21:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.441+0000 7f9982000640 10 client.27460 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f9982000640 1 -- 172.21.15.32:0/4092703825 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fafd2de6a0 con 0x55fafcf7ecd0 2024-08-31T07:21:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f9982000640 10 client.27460 handle_command_reply: tid=0 2024-08-31T07:21:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 client.27460 shutdown 2024-08-31T07:21:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4092703825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 msgr2=0x55fafd37e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd37e2d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9990103800 tx=0x7f99900cbfd0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4092703825 shutdown_connections 2024-08-31T07:21:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fafcf7ecd0 0x55fafd384520 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd37e2d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 0x55fafd372a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/4092703825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd372470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4092703825 >> 172.21.15.32:0/4092703825 conn(0x55fafcfb4740 msgr2=0x55fafd369960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f9982000640 0 client.27460 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4092703825 shutdown_connections 2024-08-31T07:21:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 1 -- 172.21.15.32:0/4092703825 wait complete. 2024-08-31T07:21:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.442+0000 7f99a5dc5740 20 client.27460 trim_cache size 0 max 16384 2024-08-31T07:21:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.462+0000 7f99a5dc5740 1 Processor -- start 2024-08-31T07:21:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.462+0000 7f99a5dc5740 1 -- start start 2024-08-31T07:21:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd37e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 0x55fafd36ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcfc0350 0x55fafd36bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99981b1340 con 0x55fafcfc0350 2024-08-31T07:21:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99981b2fa0 con 0x55fafcf97490 2024-08-31T07:21:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f99981b1a10 con 0x55fafcf9f670 2024-08-31T07:21:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f9997400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 0x55fafd36ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f9997400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 0x55fafd36ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46246/0 (socket says 172.21.15.32:46246) 2024-08-31T07:21:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f9997400640 1 -- 172.21.15.32:0/3614839726 learned_addr learned my addr 172.21.15.32:0/3614839726 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f9997e00640 1 --2- 172.21.15.32:0/3614839726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd37e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.463+0000 7f999d000640 1 --2- 172.21.15.32:0/3614839726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcfc0350 0x55fafd36bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.464+0000 7f9997400640 1 -- 172.21.15.32:0/3614839726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 msgr2=0x55fafd37e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.464+0000 7f9997400640 1 --2- 172.21.15.32:0/3614839726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd37e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.464+0000 7f9997400640 1 -- 172.21.15.32:0/3614839726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcfc0350 msgr2=0x55fafd36bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.464+0000 7f9997400640 1 --2- 172.21.15.32:0/3614839726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcfc0350 0x55fafd36bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.464+0000 7f9997400640 1 -- 172.21.15.32:0/3614839726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fafd36a850 con 0x55fafcf9f670 2024-08-31T07:21:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.464+0000 7f9997e00640 1 --2- 172.21.15.32:0/3614839726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd37e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.464+0000 7f9997400640 1 --2- 172.21.15.32:0/3614839726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 0x55fafd36ba50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9984009570 tx=0x7f9984009a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=842c63086b840b6e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.464+0000 7f998cc00640 1 -- 172.21.15.32:0/3614839726 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984005040 con 0x55fafcf9f670 2024-08-31T07:21:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.465+0000 7f998cc00640 1 -- 172.21.15.32:0/3614839726 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f998400f790 con 0x55fafcf9f670 2024-08-31T07:21:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.465+0000 7f998cc00640 1 -- 172.21.15.32:0/3614839726 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840048c0 con 0x55fafcf9f670 2024-08-31T07:21:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.465+0000 7f99a5dc5740 1 -- 172.21.15.32:0/3614839726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 msgr2=0x55fafd36ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.465+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/3614839726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 0x55fafd36ba50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9984009570 tx=0x7f9984009a90 comp rx=0 tx=0).stop 2024-08-31T07:21:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.465+0000 7f99a5dc5740 1 -- 172.21.15.32:0/3614839726 shutdown_connections 2024-08-31T07:21:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.465+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/3614839726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcfc0350 0x55fafd36bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.465+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/3614839726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcf9f670 0x55fafd36ba50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.465+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/3614839726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd37e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.465+0000 7f99a5dc5740 1 -- 172.21.15.32:0/3614839726 >> 172.21.15.32:0/3614839726 conn(0x55fafcfb4740 msgr2=0x55fafd36d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.466+0000 7f99a5dc5740 1 -- 172.21.15.32:0/3614839726 shutdown_connections 2024-08-31T07:21:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.466+0000 7f99a5dc5740 1 -- 172.21.15.32:0/3614839726 wait complete. 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.466+0000 7f99a5dc5740 1 -- start start 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.466+0000 7f99a5dc5740 1 Processor -- start 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd37a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 0x55fafd37aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f99a5dc5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd3751f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99981b28a0 con 0x55fafcf9f670 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99981b21a0 con 0x55fafcf97490 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f99a5dc5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f99981af340 con 0x55fafcfc0350 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f99a5dc5740 10 client.0 _ll_register_callbacks cb 0x7f9990009330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f999d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd3751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f9997e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd37a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f999d000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd3751f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46250/0 (socket says 172.21.15.32:46250) 2024-08-31T07:21:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f999d000640 1 -- 172.21.15.32:0/1151002039 learned_addr learned my addr 172.21.15.32:0/1151002039 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.467+0000 7f9997400640 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 0x55fafd37aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f999d000640 1 -- 172.21.15.32:0/1151002039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 msgr2=0x55fafd37a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f999d000640 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd37a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f999d000640 1 -- 172.21.15.32:0/1151002039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 msgr2=0x55fafd37aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f999d000640 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 0x55fafd37aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f999d000640 1 -- 172.21.15.32:0/1151002039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fafd3758e0 con 0x55fafcfc0350 2024-08-31T07:21:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f9997e00640 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd37a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f9997400640 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 0x55fafd37aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:21:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f999d000640 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd3751f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9990006010 tx=0x7f99900cbc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0ded71b6880a730 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f9982000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:21:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f9982000640 1 -- 172.21.15.32:0/1151002039 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99900ccea0 con 0x55fafcfc0350 2024-08-31T07:21:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f9982000640 1 -- 172.21.15.32:0/1151002039 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7f99900e72f0 con 0x55fafcfc0350 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.468+0000 7f99a5dc5740 1 -- 172.21.15.32:0/1151002039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f998400f790 con 0x55fafcfc0350 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f9982000640 1 -- 172.21.15.32:0/1151002039 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f99900eb020 con 0x55fafcfc0350 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f99a5dc5740 10 client.27466 fetch_fsmap learned FSMap version 39 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f99a5dc5740 1 -- 172.21.15.32:0/1151002039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fafd3758e0 con 0x55fafcfc0350 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f9982000640 1 -- 172.21.15.32:0/1151002039 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f998400f790 con 0x55fafcfc0350 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f9982000640 1 -- 172.21.15.32:0/1151002039 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7f99900cdad0 con 0x55fafcfc0350 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f99a5dc5740 10 client.27466 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f99a5dc5740 10 client.27466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f99a5dc5740 20 client.27466 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fafcf54080 0x55fafd3f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f99a5dc5740 4 client.27466 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:21:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.469+0000 7f99a5dc5740 1 -- 172.21.15.32:0/1151002039 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fafd2da140 con 0x55fafcf54080 2024-08-31T07:21:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.470+0000 7f9997e00640 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fafcf54080 0x55fafd3f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.470+0000 7f9982000640 10 client.27466 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.470+0000 7f9997e00640 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fafcf54080 0x55fafd3f1d90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a72bd94142dbc874 server_cookie=59834b050f58a0dc in_seq=0 out_seq=0 2024-08-31T07:21:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f9982000640 1 -- 172.21.15.32:0/1151002039 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fafd2da140 con 0x55fafcf54080 2024-08-31T07:21:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f9982000640 10 client.27466 handle_command_reply: tid=0 2024-08-31T07:21:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f99a5dc5740 1 client.27466 shutdown 2024-08-31T07:21:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f99a5dc5740 1 -- 172.21.15.32:0/1151002039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 msgr2=0x55fafd3751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd3751f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9990006010 tx=0x7f99900cbc10 comp rx=0 tx=0).stop 2024-08-31T07:21:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f99a5dc5740 1 -- 172.21.15.32:0/1151002039 shutdown_connections 2024-08-31T07:21:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fafcf54080 0x55fafd3f1d90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fafcfc0350 0x55fafd3751f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fafcf9f670 0x55fafd37aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f99a5dc5740 1 --2- 172.21.15.32:0/1151002039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fafcf97490 0x55fafd37a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f99a5dc5740 1 -- 172.21.15.32:0/1151002039 >> 172.21.15.32:0/1151002039 conn(0x55fafcfb4740 msgr2=0x55fafd363fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f9982000640 0 client.27466 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.472+0000 7f99a5dc5740 1 -- 172.21.15.32:0/1151002039 shutdown_connections 2024-08-31T07:21:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.473+0000 7f99a5dc5740 1 -- 172.21.15.32:0/1151002039 wait complete. 2024-08-31T07:21:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.473+0000 7f99a5dc5740 20 client.27466 trim_cache size 0 max 16384 2024-08-31T07:21:54.477 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:21:54.478 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:21:54.478 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "7f58fa9a-908d-4f85-9621-856bff2ad0f2", 2024-08-31T07:21:54.478 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:21:54.478 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:21:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.473+0000 7f999da00640 1 -- 172.21.15.32:0/2218632156 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f997c0b4ca0 msgr2=0x7f997c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.473+0000 7f999da00640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f997c0b4ca0 0x7f997c0b71b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55fafd365450 tx=0x7f9988005f90 comp rx=0 tx=0).stop 2024-08-31T07:21:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.473+0000 7f999da00640 1 -- 172.21.15.32:0/2218632156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f999810bbf0 msgr2=0x7f99981a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.473+0000 7f999da00640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f999810bbf0 0x7f99981a1a60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9990004af0 tx=0x7f9990003c20 comp rx=0 tx=0).stop 2024-08-31T07:21:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.474+0000 7f999da00640 1 -- 172.21.15.32:0/2218632156 shutdown_connections 2024-08-31T07:21:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.474+0000 7f999da00640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f997c0b4ca0 0x7f997c0b71b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.474+0000 7f999da00640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f999810bbf0 0x7f99981a1a60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.474+0000 7f999da00640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f999810b2a0 0x7f999819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.474+0000 7f999da00640 1 --2- 172.21.15.32:0/2218632156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9998105960 0x7f999819c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:54.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.474+0000 7f999da00640 1 -- 172.21.15.32:0/2218632156 >> 172.21.15.32:0/2218632156 conn(0x7f99981013d0 msgr2=0x7f99981037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:54.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.474+0000 7f999da00640 1 -- 172.21.15.32:0/2218632156 shutdown_connections 2024-08-31T07:21:54.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:54.474+0000 7f999da00640 1 -- 172.21.15.32:0/2218632156 wait complete. 2024-08-31T07:21:54.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:54 smithi084 ceph-mon[22387]: pgmap v648: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-08-31T07:21:54.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 291469 2.95 MB/sec execute 343 sec latency 253.722 ms 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:55.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:55 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:21:55.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:55.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:55.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:55.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:55.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:55.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:55.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:55 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:21:55.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:55.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:55.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:55.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:55.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:55.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:55.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:55 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:55.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:21:55.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 292567 2.96 MB/sec execute 344 sec latency 281.397 ms 2024-08-31T07:21:55.931 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:21:56.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:56 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:56.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:56 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:56.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:56 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:56.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:56 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:21:56.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:56 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:56.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:56 smithi032 ceph-mon[17252]: pgmap v649: 97 pgs: 97 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-08-31T07:21:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:56 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:56 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:56 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:56 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:21:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:56 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:56 smithi073 ceph-mon[21995]: pgmap v649: 97 pgs: 97 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-08-31T07:21:56.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:56 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:56.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:56 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:56.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:56 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:56.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:56 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:21:56.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:56 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:21:56.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:56 smithi084 ceph-mon[22387]: pgmap v649: 97 pgs: 97 active+clean; 210 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-08-31T07:21:56.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.648+0000 7fdb63600640 1 -- 172.21.15.32:0/1482287045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c075e20 msgr2=0x7fdb5c10f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.648+0000 7fdb63600640 1 --2- 172.21.15.32:0/1482287045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c075e20 0x7fdb5c10f9a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009900 tx=0x7fdb4802a400 comp rx=0 tx=0).stop 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.648+0000 7fdb63600640 1 -- 172.21.15.32:0/1482287045 shutdown_connections 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.648+0000 7fdb63600640 1 --2- 172.21.15.32:0/1482287045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c10fee0 0x7fdb5c112300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.648+0000 7fdb63600640 1 --2- 172.21.15.32:0/1482287045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c075e20 0x7fdb5c10f9a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.648+0000 7fdb63600640 1 --2- 172.21.15.32:0/1482287045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb5c0754d0 0x7fdb5c0758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.648+0000 7fdb63600640 1 -- 172.21.15.32:0/1482287045 >> 172.21.15.32:0/1482287045 conn(0x7fdb5c07b5f0 msgr2=0x7fdb5c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.648+0000 7fdb63600640 1 -- 172.21.15.32:0/1482287045 shutdown_connections 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.649+0000 7fdb63600640 1 -- 172.21.15.32:0/1482287045 wait complete. 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb63600640 1 Processor -- start 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb63600640 1 -- start start 2024-08-31T07:21:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb63600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0754d0 0x7fdb5c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb63600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb5c075e20 0x7fdb5c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb63600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c10fee0 0x7fdb5c1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb63600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1159e0 con 0x7fdb5c0754d0 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb63600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c115860 con 0x7fdb5c10fee0 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb63600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c115b60 con 0x7fdb5c075e20 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb60c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0754d0 0x7fdb5c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb5c075e20 0x7fdb5c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb60c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0754d0 0x7fdb5c19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46706/0 (socket says 172.21.15.32:46706) 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.650+0000 7fdb60c00640 1 -- 172.21.15.32:0/3531051978 learned_addr learned my addr 172.21.15.32:0/3531051978 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.651+0000 7fdb61600640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c10fee0 0x7fdb5c1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.651+0000 7fdb5be00640 1 -- 172.21.15.32:0/3531051978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c10fee0 msgr2=0x7fdb5c1a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.651+0000 7fdb5be00640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c10fee0 0x7fdb5c1a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.651+0000 7fdb5be00640 1 -- 172.21.15.32:0/3531051978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0754d0 msgr2=0x7fdb5c19c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.651+0000 7fdb5be00640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0754d0 0x7fdb5c19c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.651+0000 7fdb5be00640 1 -- 172.21.15.32:0/3531051978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c1a2400 con 0x7fdb5c075e20 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.651+0000 7fdb60c00640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0754d0 0x7fdb5c19c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.651+0000 7fdb5be00640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb5c075e20 0x7fdb5c19ca30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdb4802a920 tx=0x7fdb48031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.652+0000 7fdb59600640 1 -- 172.21.15.32:0/3531051978 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4803f070 con 0x7fdb5c075e20 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.652+0000 7fdb59600640 1 -- 172.21.15.32:0/3531051978 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb48004490 con 0x7fdb5c075e20 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.652+0000 7fdb63600640 1 -- 172.21.15.32:0/3531051978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c1a2690 con 0x7fdb5c075e20 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.652+0000 7fdb59600640 1 -- 172.21.15.32:0/3531051978 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48035070 con 0x7fdb5c075e20 2024-08-31T07:21:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.652+0000 7fdb63600640 1 -- 172.21.15.32:0/3531051978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c1a8e50 con 0x7fdb5c075e20 2024-08-31T07:21:56.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.655+0000 7fdb6a260740 1 Processor -- start 2024-08-31T07:21:56.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.655+0000 7fdb6a260740 1 -- start start 2024-08-31T07:21:56.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.655+0000 7fdb59600640 1 -- 172.21.15.32:0/3531051978 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fdb4803b460 con 0x7fdb5c075e20 2024-08-31T07:21:56.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.655+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa4233a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.655+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f6c1b0 0x55fa4233adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.655+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa4233b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb6a260740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fa422b1770 con 0x55fa41f6c1b0 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb6a260740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fa422a0d00 con 0x55fa41f95070 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb6a260740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fa422a4cb0 con 0x55fa41f74390 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb61600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa4233a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb60c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f6c1b0 0x55fa4233adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa4233b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb61600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa4233a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60844/0 (socket says 172.21.15.32:60844) 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb61600640 1 -- 172.21.15.32:0/574405585 learned_addr learned my addr 172.21.15.32:0/574405585 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb60c00640 1 -- 172.21.15.32:0/574405585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 msgr2=0x55fa4233b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb60c00640 1 --2- 172.21.15.32:0/574405585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa4233b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb60c00640 1 -- 172.21.15.32:0/574405585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 msgr2=0x55fa4233a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb60c00640 1 --2- 172.21.15.32:0/574405585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa4233a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.656+0000 7fdb60c00640 1 -- 172.21.15.32:0/574405585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa42000650 con 0x55fa41f6c1b0 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.657+0000 7fdb60c00640 1 --2- 172.21.15.32:0/574405585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f6c1b0 0x55fa4233adb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdb500114f0 tx=0x7fdb500119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268911a7c7a87bbe server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.657+0000 7fdb4dc00640 1 -- 172.21.15.32:0/574405585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5001e070 con 0x55fa41f6c1b0 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.657+0000 7fdb4dc00640 1 -- 172.21.15.32:0/574405585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb500035f0 con 0x55fa41f6c1b0 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.657+0000 7fdb4dc00640 1 -- 172.21.15.32:0/574405585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5000f820 con 0x55fa41f6c1b0 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.658+0000 7fdb6a260740 1 -- 172.21.15.32:0/574405585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f6c1b0 msgr2=0x55fa4233adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.658+0000 7fdb6a260740 1 --2- 172.21.15.32:0/574405585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f6c1b0 0x55fa4233adb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdb500114f0 tx=0x7fdb500119b0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.658+0000 7fdb6a260740 1 -- 172.21.15.32:0/574405585 shutdown_connections 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.658+0000 7fdb6a260740 1 --2- 172.21.15.32:0/574405585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa4233b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.658+0000 7fdb6a260740 1 --2- 172.21.15.32:0/574405585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f6c1b0 0x55fa4233adb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.658+0000 7fdb6a260740 1 --2- 172.21.15.32:0/574405585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa4233a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.658+0000 7fdb6a260740 1 -- 172.21.15.32:0/574405585 >> 172.21.15.32:0/574405585 conn(0x55fa41f89460 msgr2=0x55fa42338760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.658+0000 7fdb59600640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb440b4ca0 0x7fdb440b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.659+0000 7fdb59600640 1 -- 172.21.15.32:0/3531051978 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7fdb481059b0 con 0x7fdb5c075e20 2024-08-31T07:21:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.659+0000 7fdb6a260740 1 -- 172.21.15.32:0/574405585 shutdown_connections 2024-08-31T07:21:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.659+0000 7fdb61600640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb440b4ca0 0x7fdb440b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.659+0000 7fdb6a260740 1 -- 172.21.15.32:0/574405585 wait complete. 2024-08-31T07:21:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.659+0000 7fdb6a260740 1 -- start start 2024-08-31T07:21:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.660+0000 7fdb61600640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb440b4ca0 0x7fdb440b71b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdb54005fd0 tx=0x7fdb54009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.660+0000 7fdb6a260740 1 Processor -- start 2024-08-31T07:21:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.660+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 0x55fa42346420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.660+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.660+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 0x55fa42352340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.660+0000 7fdb6a260740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fa42103700 con 0x55fa41f95070 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.660+0000 7fdb6a260740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fa42227cb0 con 0x55fa41f6c1b0 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.660+0000 7fdb6a260740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fa42349050 con 0x55fa41f74390 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.660+0000 7fdb6a260740 10 client.0 _ll_register_callbacks cb 0x55fa4224d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb61600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 0x55fa42352340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42884/0 (socket says 172.21.15.32:42884) 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb5be00640 1 -- 172.21.15.32:0/3416473854 learned_addr learned my addr 172.21.15.32:0/3416473854 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb60c00640 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 0x55fa42346420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb5be00640 1 -- 172.21.15.32:0/3416473854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 msgr2=0x55fa42346420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb5be00640 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 0x55fa42346420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb5be00640 1 -- 172.21.15.32:0/3416473854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 msgr2=0x55fa42352340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:56.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb5be00640 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 0x55fa42352340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb5be00640 1 -- 172.21.15.32:0/3416473854 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa42268870 con 0x55fa41f74390 2024-08-31T07:21:56.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb60c00640 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 0x55fa42346420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:56.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb61600640 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 0x55fa42352340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:56.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb5be00640 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdb480c93b0 tx=0x7fdb48003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f437cd07a60fe74f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:56.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.661+0000 7fdb42000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:21:56.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb42000640 1 -- 172.21.15.32:0/3416473854 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48048030 con 0x55fa41f74390 2024-08-31T07:21:56.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb42000640 1 -- 172.21.15.32:0/3416473854 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7fdb480d4f20 con 0x55fa41f74390 2024-08-31T07:21:56.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb6a260740 1 -- 172.21.15.32:0/3416473854 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb500035f0 con 0x55fa41f74390 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb42000640 1 -- 172.21.15.32:0/3416473854 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdb480cba90 con 0x55fa41f74390 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb6a260740 10 client.27478 fetch_fsmap learned FSMap version 39 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb6a260740 1 -- 172.21.15.32:0/3416473854 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fa42355010 con 0x55fa41f74390 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb42000640 1 -- 172.21.15.32:0/3416473854 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb500035f0 con 0x55fa41f74390 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb42000640 1 -- 172.21.15.32:0/3416473854 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7fdb4804b060 con 0x55fa41f74390 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb6a260740 10 client.27478 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb6a260740 10 client.27478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb6a260740 20 client.27478 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb6a260740 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fa41f539f0 0x55fa42358500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb6a260740 4 client.27478 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:21:56.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.662+0000 7fdb6a260740 1 -- 172.21.15.32:0/3416473854 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa422b36c0 con 0x55fa41f539f0 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.663+0000 7fdb60c00640 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fa41f539f0 0x55fa42358500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.663+0000 7fdb60c00640 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fa41f539f0 0x55fa42358500 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348254c9df10998d server_cookie=b7559efbcf1db217 in_seq=0 out_seq=0 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.663+0000 7fdb42000640 10 client.27478 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.664+0000 7fdb42000640 1 -- 172.21.15.32:0/3416473854 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa422b36c0 con 0x55fa41f539f0 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.664+0000 7fdb42000640 10 client.27478 handle_command_reply: tid=0 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 client.27478 shutdown 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 -- 172.21.15.32:0/3416473854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 msgr2=0x55fa42346960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdb480c93b0 tx=0x7fdb48003f50 comp rx=0 tx=0).stop 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 -- 172.21.15.32:0/3416473854 shutdown_connections 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fa41f539f0 0x55fa42358500 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 0x55fa42352340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346960 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 --2- 172.21.15.32:0/3416473854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 0x55fa42346420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 -- 172.21.15.32:0/3416473854 >> 172.21.15.32:0/3416473854 conn(0x55fa41f89460 msgr2=0x55fa4233da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:56.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb42000640 0 client.27478 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:56.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 -- 172.21.15.32:0/3416473854 shutdown_connections 2024-08-31T07:21:56.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 1 -- 172.21.15.32:0/3416473854 wait complete. 2024-08-31T07:21:56.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.665+0000 7fdb6a260740 20 client.27478 trim_cache size 0 max 16384 2024-08-31T07:21:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb6a260740 1 Processor -- start 2024-08-31T07:21:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb6a260740 1 -- start start 2024-08-31T07:21:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 0x55fa423387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 0x55fa42343410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb6a260740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1b15c0 con 0x55fa41f95070 2024-08-31T07:21:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb6a260740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1b3220 con 0x55fa41f6c1b0 2024-08-31T07:21:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb6a260740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1b1c90 con 0x55fa41f74390 2024-08-31T07:21:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb61600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 0x55fa42343410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb61600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 0x55fa42343410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46724/0 (socket says 172.21.15.32:46724) 2024-08-31T07:21:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb61600640 1 -- 172.21.15.32:0/429830936 learned_addr learned my addr 172.21.15.32:0/429830936 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.684+0000 7fdb60c00640 1 --2- 172.21.15.32:0/429830936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 0x55fa423387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.685+0000 7fdb5be00640 1 -- 172.21.15.32:0/429830936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 msgr2=0x55fa423387d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.685+0000 7fdb5be00640 1 --2- 172.21.15.32:0/429830936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 0x55fa423387d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.685+0000 7fdb5be00640 1 -- 172.21.15.32:0/429830936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 msgr2=0x55fa42343410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.685+0000 7fdb5be00640 1 --2- 172.21.15.32:0/429830936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 0x55fa42343410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.685+0000 7fdb5be00640 1 -- 172.21.15.32:0/429830936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa42345180 con 0x55fa41f74390 2024-08-31T07:21:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.685+0000 7fdb5be00640 1 --2- 172.21.15.32:0/429830936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdb48105d70 tx=0x7fdb4803b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bffc8d07bb2eae6f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.685+0000 7fdb4d200640 1 -- 172.21.15.32:0/429830936 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48048070 con 0x55fa41f74390 2024-08-31T07:21:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.685+0000 7fdb4d200640 1 -- 172.21.15.32:0/429830936 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb480cc140 con 0x55fa41f74390 2024-08-31T07:21:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.685+0000 7fdb4d200640 1 -- 172.21.15.32:0/429830936 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48103580 con 0x55fa41f74390 2024-08-31T07:21:56.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.686+0000 7fdb6a260740 1 -- 172.21.15.32:0/429830936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 msgr2=0x55fa42346420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:56.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.686+0000 7fdb6a260740 1 --2- 172.21.15.32:0/429830936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdb48105d70 tx=0x7fdb4803b790 comp rx=0 tx=0).stop 2024-08-31T07:21:56.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.686+0000 7fdb6a260740 1 -- 172.21.15.32:0/429830936 shutdown_connections 2024-08-31T07:21:56.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.686+0000 7fdb6a260740 1 --2- 172.21.15.32:0/429830936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f95070 0x55fa42343410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.686+0000 7fdb6a260740 1 --2- 172.21.15.32:0/429830936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f74390 0x55fa42346420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.686+0000 7fdb6a260740 1 --2- 172.21.15.32:0/429830936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f6c1b0 0x55fa423387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.686+0000 7fdb6a260740 1 -- 172.21.15.32:0/429830936 >> 172.21.15.32:0/429830936 conn(0x55fa41f89460 msgr2=0x55fa423380c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:56.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.686+0000 7fdb6a260740 1 -- 172.21.15.32:0/429830936 shutdown_connections 2024-08-31T07:21:56.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.686+0000 7fdb6a260740 1 -- 172.21.15.32:0/429830936 wait complete. 2024-08-31T07:21:56.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.687+0000 7fdb6a260740 1 -- start start 2024-08-31T07:21:56.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.687+0000 7fdb6a260740 1 Processor -- start 2024-08-31T07:21:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.687+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f6c1b0 0x55fa4234e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f74390 0x55fa4234ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb6a260740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa42349320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb6a260740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1b2b20 con 0x55fa41f74390 2024-08-31T07:21:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb6a260740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1b2420 con 0x55fa41f95070 2024-08-31T07:21:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb6a260740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1af5c0 con 0x55fa41f6c1b0 2024-08-31T07:21:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb6a260740 10 client.0 _ll_register_callbacks cb 0x7fdb48005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:21:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb61600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa42349320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f74390 0x55fa4234ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb60c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f6c1b0 0x55fa4234e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb61600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa42349320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:60872/0 (socket says 172.21.15.32:60872) 2024-08-31T07:21:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb5be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f74390 0x55fa4234ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46736/0 (socket says 172.21.15.32:46736) 2024-08-31T07:21:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb61600640 1 -- 172.21.15.32:0/2807186763 learned_addr learned my addr 172.21.15.32:0/2807186763 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:21:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb61600640 1 -- 172.21.15.32:0/2807186763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f6c1b0 msgr2=0x55fa4234e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb61600640 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f6c1b0 0x55fa4234e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb61600640 1 -- 172.21.15.32:0/2807186763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f74390 msgr2=0x55fa4234ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb61600640 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f74390 0x55fa4234ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb61600640 1 -- 172.21.15.32:0/2807186763 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa42349ab0 con 0x55fa41f95070 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb60c00640 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f6c1b0 0x55fa4234e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.688+0000 7fdb5be00640 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f74390 0x55fa4234ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.689+0000 7fdb61600640 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa42349320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdb540045e0 tx=0x7fdb54004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6043330836c0989 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.689+0000 7fdb42000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.689+0000 7fdb42000640 1 -- 172.21.15.32:0/2807186763 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5402e060 con 0x55fa41f95070 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.689+0000 7fdb6a260740 1 -- 172.21.15.32:0/2807186763 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb480cc140 con 0x55fa41f95070 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.689+0000 7fdb42000640 1 -- 172.21.15.32:0/2807186763 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8441+0+0 (secure 0 0 0) 0x7fdb5401e430 con 0x55fa41f95070 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.689+0000 7fdb42000640 1 -- 172.21.15.32:0/2807186763 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdb5400fe50 con 0x55fa41f95070 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.689+0000 7fdb6a260740 10 client.27479 fetch_fsmap learned FSMap version 39 2024-08-31T07:21:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.689+0000 7fdb6a260740 1 -- 172.21.15.32:0/2807186763 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fa423c39f0 con 0x55fa41f95070 2024-08-31T07:21:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.690+0000 7fdb42000640 1 -- 172.21.15.32:0/2807186763 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb480cc140 con 0x55fa41f95070 2024-08-31T07:21:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.690+0000 7fdb42000640 1 -- 172.21.15.32:0/2807186763 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7fdb54033760 con 0x55fa41f95070 2024-08-31T07:21:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.690+0000 7fdb6a260740 10 client.27479 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T07:21:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.690+0000 7fdb6a260740 10 client.27479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:21:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.690+0000 7fdb6a260740 20 client.27479 populate_metadata read hostname 'smithi032' 2024-08-31T07:21:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.690+0000 7fdb6a260740 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fa41f28da0 0x55fa423c6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:21:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.690+0000 7fdb6a260740 4 client.27479 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.690+0000 7fdb6a260740 1 -- 172.21.15.32:0/2807186763 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa422af160 con 0x55fa41f28da0 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.690+0000 7fdb60c00640 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fa41f28da0 0x55fa423c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.691+0000 7fdb60c00640 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fa41f28da0 0x55fa423c6210 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be09c9e4712719ac server_cookie=5ed07013119bc7fe in_seq=0 out_seq=0 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.691+0000 7fdb42000640 10 client.27479 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.691+0000 7fdb42000640 1 -- 172.21.15.32:0/2807186763 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x55fa422af160 con 0x55fa41f28da0 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.691+0000 7fdb42000640 10 client.27479 handle_command_reply: tid=0 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.691+0000 7fdb6a260740 1 client.27479 shutdown 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.691+0000 7fdb6a260740 1 -- 172.21.15.32:0/2807186763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 msgr2=0x55fa42349320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.691+0000 7fdb6a260740 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa42349320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdb540045e0 tx=0x7fdb54004a40 comp rx=0 tx=0).stop 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.692+0000 7fdb6a260740 1 -- 172.21.15.32:0/2807186763 shutdown_connections 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.692+0000 7fdb6a260740 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55fa41f28da0 0x55fa423c6210 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.692+0000 7fdb6a260740 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa41f95070 0x55fa42349320 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.692+0000 7fdb6a260740 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa41f74390 0x55fa4234ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.692+0000 7fdb6a260740 1 --2- 172.21.15.32:0/2807186763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa41f6c1b0 0x55fa4234e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.692+0000 7fdb42000640 0 client.27479 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:21:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.692+0000 7fdb6a260740 1 -- 172.21.15.32:0/2807186763 >> 172.21.15.32:0/2807186763 conn(0x55fa41f89460 msgr2=0x55fa42336da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.692+0000 7fdb6a260740 1 -- 172.21.15.32:0/2807186763 shutdown_connections 2024-08-31T07:21:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.692+0000 7fdb6a260740 1 -- 172.21.15.32:0/2807186763 wait complete. 2024-08-31T07:21:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.692+0000 7fdb6a260740 20 client.27479 trim_cache size 0 max 16384 2024-08-31T07:21:56.696 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:21:56.696 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "7f58fa9a-908d-4f85-9621-856bff2ad0f2", 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.697 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.699 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:21:56.701 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:21:56.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 -- 172.21.15.32:0/3531051978 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb440b4ca0 msgr2=0x7fdb440b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb440b4ca0 0x7fdb440b71b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdb54005fd0 tx=0x7fdb54009500 comp rx=0 tx=0).stop 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 -- 172.21.15.32:0/3531051978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb5c075e20 msgr2=0x7fdb5c19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb5c075e20 0x7fdb5c19ca30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdb4802a920 tx=0x7fdb48031a20 comp rx=0 tx=0).stop 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 -- 172.21.15.32:0/3531051978 shutdown_connections 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdb440b4ca0 0x7fdb440b71b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c10fee0 0x7fdb5c1a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb5c075e20 0x7fdb5c19ca30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 --2- 172.21.15.32:0/3531051978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb5c0754d0 0x7fdb5c19c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 -- 172.21.15.32:0/3531051978 >> 172.21.15.32:0/3531051978 conn(0x7fdb5c07b5f0 msgr2=0x7fdb5c1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 -- 172.21.15.32:0/3531051978 shutdown_connections 2024-08-31T07:21:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:21:56.693+0000 7fdb63600640 1 -- 172.21.15.32:0/3531051978 wait complete. 2024-08-31T07:21:56.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 292567 2.95 MB/sec execute 345 sec latency 1281.502 ms 2024-08-31T07:21:57.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 292567 2.94 MB/sec execute 346 sec latency 2281.595 ms 2024-08-31T07:21:57.914 INFO:tasks.cephfs.filesystem:scrub status for tag:7f58fa9a-908d-4f85-9621-856bff2ad0f2 - {'path': '/', 'tag': '7f58fa9a-908d-4f85-9621-856bff2ad0f2', 'options': 'recursive,force'} 2024-08-31T07:21:58.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:21:58 smithi032 ceph-mon[17252]: pgmap v650: 97 pgs: 97 active+clean; 214 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-08-31T07:21:58.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:21:58 smithi073 ceph-mon[21995]: pgmap v650: 97 pgs: 97 active+clean; 214 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-08-31T07:21:58.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:21:58 smithi084 ceph-mon[22387]: pgmap v650: 97 pgs: 97 active+clean; 214 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-08-31T07:21:58.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 292567 2.93 MB/sec execute 347 sec latency 3281.661 ms 2024-08-31T07:21:59.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 292567 2.93 MB/sec execute 348 sec latency 4281.770 ms 2024-08-31T07:22:00.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:00 smithi032 ceph-mon[17252]: pgmap v651: 97 pgs: 97 active+clean; 213 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-08-31T07:22:00.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:00 smithi073 ceph-mon[21995]: pgmap v651: 97 pgs: 97 active+clean; 213 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-08-31T07:22:00.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:00 smithi084 ceph-mon[22387]: pgmap v651: 97 pgs: 97 active+clean; 213 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-08-31T07:22:00.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 294108 2.94 MB/sec execute 349 sec latency 4499.665 ms 2024-08-31T07:22:01.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:22:01.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:22:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:22:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:22:01.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:22:01.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:22:01.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 294881 2.95 MB/sec execute 350 sec latency 27.916 ms 2024-08-31T07:22:02.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:02 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:22:02.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:02 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:22:02.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:02 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:22:02.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:02 smithi032 ceph-mon[17252]: pgmap v652: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-08-31T07:22:02.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:02 smithi032 ceph-mon[17252]: osdmap e87: 12 total, 12 up, 12 in 2024-08-31T07:22:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:02 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:22:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:02 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:22:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:02 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:22:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:02 smithi073 ceph-mon[21995]: pgmap v652: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-08-31T07:22:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:02 smithi073 ceph-mon[21995]: osdmap e87: 12 total, 12 up, 12 in 2024-08-31T07:22:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:02 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:22:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:02 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:22:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:02 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:22:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:02 smithi084 ceph-mon[22387]: pgmap v652: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-08-31T07:22:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:02 smithi084 ceph-mon[22387]: osdmap e87: 12 total, 12 up, 12 in 2024-08-31T07:22:02.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 296034 2.96 MB/sec execute 351 sec latency 19.797 ms 2024-08-31T07:22:03.754 INFO:tasks.workunit.client.0.smithi032.stdout: 1 296117 2.95 MB/sec execute 352 sec latency 936.714 ms 2024-08-31T07:22:04.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:04 smithi032 ceph-mon[17252]: pgmap v654: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-08-31T07:22:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:04 smithi073 ceph-mon[21995]: pgmap v654: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-08-31T07:22:04.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:04 smithi084 ceph-mon[22387]: pgmap v654: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-08-31T07:22:04.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 296117 2.94 MB/sec execute 353 sec latency 1936.855 ms 2024-08-31T07:22:05.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 297418 2.95 MB/sec execute 354 sec latency 2307.908 ms 2024-08-31T07:22:06.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:06 smithi032 ceph-mon[17252]: pgmap v655: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-08-31T07:22:06.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:06 smithi073 ceph-mon[21995]: pgmap v655: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-08-31T07:22:06.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:06 smithi084 ceph-mon[22387]: pgmap v655: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-08-31T07:22:06.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 298665 2.96 MB/sec execute 355 sec latency 17.599 ms 2024-08-31T07:22:07.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:07 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:22:07.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:07 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:22:07.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:07 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:22:07.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 299665 2.97 MB/sec execute 356 sec latency 174.438 ms 2024-08-31T07:22:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:08 smithi073 ceph-mon[21995]: pgmap v656: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-08-31T07:22:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:08 smithi084 ceph-mon[22387]: pgmap v656: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-08-31T07:22:08.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:08 smithi032 ceph-mon[17252]: pgmap v656: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-08-31T07:22:08.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 299665 2.96 MB/sec execute 357 sec latency 1174.577 ms 2024-08-31T07:22:09.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 299665 2.95 MB/sec execute 358 sec latency 2174.741 ms 2024-08-31T07:22:10.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:10 smithi073 ceph-mon[21995]: pgmap v657: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-08-31T07:22:10.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:10 smithi084 ceph-mon[22387]: pgmap v657: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-08-31T07:22:10.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:10 smithi032 ceph-mon[17252]: pgmap v657: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-08-31T07:22:10.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 300965 2.96 MB/sec execute 359 sec latency 2542.168 ms 2024-08-31T07:22:11.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 302233 2.97 MB/sec execute 360 sec latency 14.180 ms 2024-08-31T07:22:12.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:12 smithi073 ceph-mon[21995]: pgmap v658: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-08-31T07:22:12.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:12 smithi073 ceph-mon[21995]: osdmap e88: 12 total, 12 up, 12 in 2024-08-31T07:22:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:12 smithi084 ceph-mon[22387]: pgmap v658: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-08-31T07:22:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:12 smithi084 ceph-mon[22387]: osdmap e88: 12 total, 12 up, 12 in 2024-08-31T07:22:12.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:12 smithi032 ceph-mon[17252]: pgmap v658: 97 pgs: 97 active+clean; 218 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-08-31T07:22:12.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:12 smithi032 ceph-mon[17252]: osdmap e88: 12 total, 12 up, 12 in 2024-08-31T07:22:12.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 303662 2.98 MB/sec execute 361 sec latency 183.558 ms 2024-08-31T07:22:13.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 305348 3.01 MB/sec execute 362 sec latency 40.987 ms 2024-08-31T07:22:14.297 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:14 smithi032 ceph-mon[17252]: pgmap v660: 97 pgs: 97 active+clean; 221 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-08-31T07:22:14.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:14 smithi073 ceph-mon[21995]: pgmap v660: 97 pgs: 97 active+clean; 221 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-08-31T07:22:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:14 smithi084 ceph-mon[22387]: pgmap v660: 97 pgs: 97 active+clean; 221 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-08-31T07:22:14.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 305776 3.00 MB/sec execute 363 sec latency 275.742 ms 2024-08-31T07:22:15.755 INFO:tasks.workunit.client.0.smithi032.stdout: 1 306757 3.01 MB/sec execute 364 sec latency 347.755 ms 2024-08-31T07:22:16.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:16 smithi032 ceph-mon[17252]: pgmap v661: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-08-31T07:22:16.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:16 smithi073 ceph-mon[21995]: pgmap v661: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-08-31T07:22:16.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:16 smithi084 ceph-mon[22387]: pgmap v661: 97 pgs: 97 active+clean; 222 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-08-31T07:22:16.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 306757 3.00 MB/sec execute 365 sec latency 1347.892 ms 2024-08-31T07:22:17.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 306757 2.99 MB/sec execute 366 sec latency 2348.057 ms 2024-08-31T07:22:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:18 smithi073 ceph-mon[21995]: pgmap v662: 97 pgs: 97 active+clean; 227 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-08-31T07:22:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:18 smithi084 ceph-mon[22387]: pgmap v662: 97 pgs: 97 active+clean; 227 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-08-31T07:22:18.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:18 smithi032 ceph-mon[17252]: pgmap v662: 97 pgs: 97 active+clean; 227 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-08-31T07:22:18.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 307514 2.99 MB/sec execute 367 sec latency 2984.146 ms 2024-08-31T07:22:19.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 308440 3.00 MB/sec execute 368 sec latency 272.167 ms 2024-08-31T07:22:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:20 smithi073 ceph-mon[21995]: pgmap v663: 97 pgs: 97 active+clean; 227 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-08-31T07:22:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:20 smithi084 ceph-mon[22387]: pgmap v663: 97 pgs: 97 active+clean; 227 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-08-31T07:22:20.580 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:20 smithi032 ceph-mon[17252]: pgmap v663: 97 pgs: 97 active+clean; 227 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-08-31T07:22:20.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 309525 3.01 MB/sec execute 369 sec latency 19.556 ms 2024-08-31T07:22:21.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 310305 3.01 MB/sec execute 370 sec latency 478.111 ms 2024-08-31T07:22:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:22 smithi073 ceph-mon[21995]: pgmap v664: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-08-31T07:22:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:22 smithi084 ceph-mon[22387]: pgmap v664: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-08-31T07:22:22.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:22 smithi032 ceph-mon[17252]: pgmap v664: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-08-31T07:22:22.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 310305 3.00 MB/sec execute 371 sec latency 1478.295 ms 2024-08-31T07:22:23.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 310305 2.99 MB/sec execute 372 sec latency 2478.344 ms 2024-08-31T07:22:24.402 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:24 smithi032 ceph-mon[17252]: pgmap v665: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-08-31T07:22:24.402 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:22:24.402 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.402 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.402 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.402 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:24 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:24 smithi073 ceph-mon[21995]: pgmap v665: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-08-31T07:22:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:22:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:24 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:24 smithi084 ceph-mon[22387]: pgmap v665: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-08-31T07:22:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:22:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:24 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:24.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 310305 2.99 MB/sec execute 373 sec latency 3478.388 ms 2024-08-31T07:22:25.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:25.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:25 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:25.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:25.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:25 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:25.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:25.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:25 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:25.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 311326 2.99 MB/sec execute 374 sec latency 3860.431 ms 2024-08-31T07:22:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:26 smithi073 ceph-mon[21995]: pgmap v666: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-08-31T07:22:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:26 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:26 smithi084 ceph-mon[22387]: pgmap v666: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-08-31T07:22:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:26 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 312627 3.01 MB/sec execute 375 sec latency 13.890 ms 2024-08-31T07:22:26.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:26 smithi032 ceph-mon[17252]: pgmap v666: 97 pgs: 97 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-08-31T07:22:26.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:26.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:26 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:27.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 313847 3.01 MB/sec execute 376 sec latency 167.791 ms 2024-08-31T07:22:27.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:27 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:27.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:27 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:27.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:27 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:22:27.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:27 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:22:27.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:27 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:27.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:22:27.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:27 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:27.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:27 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:27.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:27 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:22:27.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:27 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:22:27.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:27 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:27.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:27 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:27.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:27 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:27.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:27 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:22:27.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:27 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:22:27.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:27 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:22:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:28.743 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:28 smithi032 ceph-mon[17252]: pgmap v667: 97 pgs: 97 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-08-31T07:22:28.756 INFO:tasks.workunit.client.0.smithi032.stdout: 1 313847 3.01 MB/sec execute 377 sec latency 1167.831 ms 2024-08-31T07:22:28.830 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:28 smithi073 ceph-mon[21995]: pgmap v667: 97 pgs: 97 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-08-31T07:22:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:28 smithi084 ceph-mon[22387]: pgmap v667: 97 pgs: 97 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-08-31T07:22:29.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.077+0000 7fb337000640 1 -- 172.21.15.32:0/4100684597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb33010b290 msgr2=0x7fb33010b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:29.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.077+0000 7fb337000640 1 --2- 172.21.15.32:0/4100684597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb33010b290 0x7fb33010b6a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009900 tx=0x7fb31c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.077+0000 7fb337000640 1 -- 172.21.15.32:0/4100684597 shutdown_connections 2024-08-31T07:22:29.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.077+0000 7fb337000640 1 --2- 172.21.15.32:0/4100684597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb33010bbe0 0x7fb330112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.077+0000 7fb337000640 1 --2- 172.21.15.32:0/4100684597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb33010b290 0x7fb33010b6a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.077+0000 7fb337000640 1 --2- 172.21.15.32:0/4100684597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330105950 0x7fb330105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.077+0000 7fb337000640 1 -- 172.21.15.32:0/4100684597 >> 172.21.15.32:0/4100684597 conn(0x7fb330101420 msgr2=0x7fb330103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.077+0000 7fb337000640 1 -- 172.21.15.32:0/4100684597 shutdown_connections 2024-08-31T07:22:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.077+0000 7fb337000640 1 -- 172.21.15.32:0/4100684597 wait complete. 2024-08-31T07:22:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.078+0000 7fb337000640 1 Processor -- start 2024-08-31T07:22:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.078+0000 7fb337000640 1 -- start start 2024-08-31T07:22:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.078+0000 7fb337000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330105950 0x7fb33019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.078+0000 7fb337000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb33010b290 0x7fb33019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.078+0000 7fb337000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb33010bbe0 0x7fb3301a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.078+0000 7fb337000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301158a0 con 0x7fb33010b290 2024-08-31T07:22:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.078+0000 7fb337000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb330115720 con 0x7fb330105950 2024-08-31T07:22:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.078+0000 7fb337000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb330115a20 con 0x7fb33010bbe0 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb335200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb33010b290 0x7fb33019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb335c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330105950 0x7fb33019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb335200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb33010b290 0x7fb33019c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46040/0 (socket says 172.21.15.32:46040) 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb335200640 1 -- 172.21.15.32:0/3441907451 learned_addr learned my addr 172.21.15.32:0/3441907451 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb336600640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb33010bbe0 0x7fb3301a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb335c00640 1 -- 172.21.15.32:0/3441907451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb33010bbe0 msgr2=0x7fb3301a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb335c00640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb33010bbe0 0x7fb3301a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb335c00640 1 -- 172.21.15.32:0/3441907451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb33010b290 msgr2=0x7fb33019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb335c00640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb33010b290 0x7fb33019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb335c00640 1 -- 172.21.15.32:0/3441907451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3301a2280 con 0x7fb330105950 2024-08-31T07:22:29.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.079+0000 7fb335200640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb33010b290 0x7fb33019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.080+0000 7fb335c00640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330105950 0x7fb33019c310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb32400e9f0 tx=0x7fb32400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.080+0000 7fb32e000640 1 -- 172.21.15.32:0/3441907451 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32400ccc0 con 0x7fb330105950 2024-08-31T07:22:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.080+0000 7fb337000640 1 -- 172.21.15.32:0/3441907451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3301a2570 con 0x7fb330105950 2024-08-31T07:22:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.080+0000 7fb32e000640 1 -- 172.21.15.32:0/3441907451 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3240108e0 con 0x7fb330105950 2024-08-31T07:22:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.080+0000 7fb32e000640 1 -- 172.21.15.32:0/3441907451 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324018880 con 0x7fb330105950 2024-08-31T07:22:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.080+0000 7fb337000640 1 -- 172.21.15.32:0/3441907451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3301a8cd0 con 0x7fb330105950 2024-08-31T07:22:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.083+0000 7fb33f3dd740 1 Processor -- start 2024-08-31T07:22:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.083+0000 7fb33f3dd740 1 -- start start 2024-08-31T07:22:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 0x561ac8aceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 0x561ac8acf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8acf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561ac8a45770 con 0x561ac8708390 2024-08-31T07:22:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561ac8a34d00 con 0x561ac8729070 2024-08-31T07:22:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561ac8a38cb0 con 0x561ac87001b0 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb335200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8acf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb32e000640 1 -- 172.21.15.32:0/3441907451 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb324010430 con 0x7fb330105950 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb335200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8acf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46046/0 (socket says 172.21.15.32:46046) 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb335200640 1 -- 172.21.15.32:0/3102081256 learned_addr learned my addr 172.21.15.32:0/3102081256 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.084+0000 7fb336600640 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 0x561ac8aceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.085+0000 7fb335200640 1 -- 172.21.15.32:0/3102081256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 msgr2=0x561ac8acf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.085+0000 7fb335c00640 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 0x561ac8acf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.085+0000 7fb335200640 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 0x561ac8acf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.085+0000 7fb335200640 1 -- 172.21.15.32:0/3102081256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 msgr2=0x561ac8aceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.085+0000 7fb335200640 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 0x561ac8aceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.085+0000 7fb335200640 1 -- 172.21.15.32:0/3102081256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac8794650 con 0x561ac8708390 2024-08-31T07:22:29.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.085+0000 7fb335c00640 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 0x561ac8acf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:22:29.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.085+0000 7fb336600640 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 0x561ac8aceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:29.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.085+0000 7fb335200640 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8acf5a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb31c006010 tx=0x7fb31c005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56262658eca7ffb1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.086+0000 7fb322a00640 1 -- 172.21.15.32:0/3102081256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c035040 con 0x561ac8708390 2024-08-31T07:22:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.086+0000 7fb322a00640 1 -- 172.21.15.32:0/3102081256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb31c031720 con 0x561ac8708390 2024-08-31T07:22:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.086+0000 7fb322a00640 1 -- 172.21.15.32:0/3102081256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c005650 con 0x561ac8708390 2024-08-31T07:22:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.086+0000 7fb32e000640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3180b4ca0 0x7fb3180b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.086+0000 7fb32e000640 1 -- 172.21.15.32:0/3441907451 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fb3240e5860 con 0x7fb330105950 2024-08-31T07:22:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.086+0000 7fb33f3dd740 1 -- 172.21.15.32:0/3102081256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 msgr2=0x561ac8acf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.086+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8acf5a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb31c006010 tx=0x7fb31c005e20 comp rx=0 tx=0).stop 2024-08-31T07:22:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.086+0000 7fb336600640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3180b4ca0 0x7fb3180b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.087+0000 7fb33f3dd740 1 -- 172.21.15.32:0/3102081256 shutdown_connections 2024-08-31T07:22:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.087+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8acf5a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.087+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 0x561ac8acf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.087+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/3102081256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 0x561ac8aceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.087+0000 7fb33f3dd740 1 -- 172.21.15.32:0/3102081256 >> 172.21.15.32:0/3102081256 conn(0x561ac871d460 msgr2=0x561ac8acca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.087+0000 7fb33f3dd740 1 -- 172.21.15.32:0/3102081256 shutdown_connections 2024-08-31T07:22:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.087+0000 7fb336600640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3180b4ca0 0x7fb3180b71b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb328009fd0 tx=0x7fb328009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.087+0000 7fb33f3dd740 1 -- 172.21.15.32:0/3102081256 wait complete. 2024-08-31T07:22:29.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.088+0000 7fb33f3dd740 1 -- start start 2024-08-31T07:22:29.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.088+0000 7fb33f3dd740 1 Processor -- start 2024-08-31T07:22:29.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ada730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 0x561ac8adacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac8729070 0x561ac8ae6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561ac8897700 con 0x561ac87001b0 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561ac89bbcb0 con 0x561ac8708390 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561ac8add2a0 con 0x561ac8729070 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb33f3dd740 10 client.0 _ll_register_callbacks cb 0x561ac89e1430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb335c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ada730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb335c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ada730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46058/0 (socket says 172.21.15.32:46058) 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb335c00640 1 -- 172.21.15.32:0/1247631134 learned_addr learned my addr 172.21.15.32:0/1247631134 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb335200640 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 0x561ac8adacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb336600640 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac8729070 0x561ac8ae6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb335c00640 1 -- 172.21.15.32:0/1247631134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac8729070 msgr2=0x561ac8ae6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb335c00640 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac8729070 0x561ac8ae6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb335c00640 1 -- 172.21.15.32:0/1247631134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 msgr2=0x561ac8adacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb335c00640 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 0x561ac8adacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.089+0000 7fb335c00640 1 -- 172.21.15.32:0/1247631134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ac89fc870 con 0x561ac87001b0 2024-08-31T07:22:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.090+0000 7fb335c00640 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ada730 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb3240e3000 tx=0x7fb32400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568ba7617b06a60e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.090+0000 7fb317400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:22:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.090+0000 7fb317400640 1 -- 172.21.15.32:0/1247631134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3240e3430 con 0x561ac87001b0 2024-08-31T07:22:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.090+0000 7fb317400640 1 -- 172.21.15.32:0/1247631134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fb3240b0410 con 0x561ac87001b0 2024-08-31T07:22:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.090+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1247631134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb31c031720 con 0x561ac87001b0 2024-08-31T07:22:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.090+0000 7fb317400640 1 -- 172.21.15.32:0/1247631134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb3240b5260 con 0x561ac87001b0 2024-08-31T07:22:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.090+0000 7fb33f3dd740 10 client.17484 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.090+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1247631134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561ac8ae9270 con 0x561ac87001b0 2024-08-31T07:22:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.090+0000 7fb317400640 1 -- 172.21.15.32:0/1247631134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb31c031720 con 0x561ac87001b0 2024-08-31T07:22:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.091+0000 7fb317400640 1 -- 172.21.15.32:0/1247631134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb3240a7b60 con 0x561ac87001b0 2024-08-31T07:22:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.091+0000 7fb33f3dd740 10 client.17484 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.091+0000 7fb33f3dd740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.091+0000 7fb33f3dd740 20 client.17484 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.091+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561ac86e79f0 0x561ac8aec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.091+0000 7fb33f3dd740 4 client.17484 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.091+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1247631134 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ac8a476c0 con 0x561ac86e79f0 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.091+0000 7fb335200640 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561ac86e79f0 0x561ac8aec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.092+0000 7fb335200640 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561ac86e79f0 0x561ac8aec780 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e4d0cd02fdf8bc server_cookie=6100565036458ede in_seq=0 out_seq=0 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.092+0000 7fb317400640 10 client.17484 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb317400640 1 -- 172.21.15.32:0/1247631134 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ac8a476c0 con 0x561ac86e79f0 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb317400640 10 client.17484 handle_command_reply: tid=0 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 client.17484 shutdown 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1247631134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 msgr2=0x561ac8ada730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ada730 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb3240e3000 tx=0x7fb32400ce60 comp rx=0 tx=0).stop 2024-08-31T07:22:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1247631134 shutdown_connections 2024-08-31T07:22:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561ac86e79f0 0x561ac8aec780 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac8729070 0x561ac8ae6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 0x561ac8adacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1247631134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ada730 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1247631134 >> 172.21.15.32:0/1247631134 conn(0x561ac871d460 msgr2=0x561ac8ad1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb317400640 0 client.17484 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1247631134 shutdown_connections 2024-08-31T07:22:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.093+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1247631134 wait complete. 2024-08-31T07:22:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.094+0000 7fb33f3dd740 20 client.17484 trim_cache size 0 max 16384 2024-08-31T07:22:29.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.112+0000 7fb33f3dd740 1 Processor -- start 2024-08-31T07:22:29.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.112+0000 7fb33f3dd740 1 -- start start 2024-08-31T07:22:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.112+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ad4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.112+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 0x561ac8ae6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.112+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac8729070 0x561ac8acc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.112+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301b1420 con 0x561ac87001b0 2024-08-31T07:22:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.112+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301b3050 con 0x561ac8708390 2024-08-31T07:22:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.112+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301b1af0 con 0x561ac8729070 2024-08-31T07:22:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.113+0000 7fb335c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ad4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.113+0000 7fb335c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ad4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46068/0 (socket says 172.21.15.32:46068) 2024-08-31T07:22:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.113+0000 7fb335c00640 1 -- 172.21.15.32:0/2950617814 learned_addr learned my addr 172.21.15.32:0/2950617814 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:29.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.113+0000 7fb335200640 1 --2- 172.21.15.32:0/2950617814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 0x561ac8ae6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.113+0000 7fb336600640 1 --2- 172.21.15.32:0/2950617814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac8729070 0x561ac8acc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.113+0000 7fb335200640 1 -- 172.21.15.32:0/2950617814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac8729070 msgr2=0x561ac8acc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.113+0000 7fb335200640 1 --2- 172.21.15.32:0/2950617814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac8729070 0x561ac8acc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.113+0000 7fb335200640 1 -- 172.21.15.32:0/2950617814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 msgr2=0x561ac8ad4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.113+0000 7fb335200640 1 --2- 172.21.15.32:0/2950617814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ad4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.113+0000 7fb335200640 1 -- 172.21.15.32:0/2950617814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac8ad9490 con 0x561ac8708390 2024-08-31T07:22:29.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.114+0000 7fb335200640 1 --2- 172.21.15.32:0/2950617814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 0x561ac8ae6490 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x561ac8aecee0 tx=0x7fb31c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc5346982d359203 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:29.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.114+0000 7fb322000640 1 -- 172.21.15.32:0/2950617814 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c035040 con 0x561ac8708390 2024-08-31T07:22:29.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.114+0000 7fb322000640 1 -- 172.21.15.32:0/2950617814 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb31c0035f0 con 0x561ac8708390 2024-08-31T07:22:29.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.114+0000 7fb322000640 1 -- 172.21.15.32:0/2950617814 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c052680 con 0x561ac8708390 2024-08-31T07:22:29.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.115+0000 7fb33f3dd740 1 -- 172.21.15.32:0/2950617814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 msgr2=0x561ac8ae6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:29.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.115+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/2950617814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 0x561ac8ae6490 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x561ac8aecee0 tx=0x7fb31c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.115+0000 7fb33f3dd740 1 -- 172.21.15.32:0/2950617814 shutdown_connections 2024-08-31T07:22:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.115+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/2950617814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac8729070 0x561ac8acc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.115+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/2950617814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8708390 0x561ac8ae6490 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.115+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/2950617814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac87001b0 0x561ac8ad4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.115+0000 7fb33f3dd740 1 -- 172.21.15.32:0/2950617814 >> 172.21.15.32:0/2950617814 conn(0x561ac871d460 msgr2=0x561ac8ad3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.115+0000 7fb33f3dd740 1 -- 172.21.15.32:0/2950617814 shutdown_connections 2024-08-31T07:22:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.115+0000 7fb33f3dd740 1 -- 172.21.15.32:0/2950617814 wait complete. 2024-08-31T07:22:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.116+0000 7fb33f3dd740 1 -- start start 2024-08-31T07:22:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.116+0000 7fb33f3dd740 1 Processor -- start 2024-08-31T07:22:29.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.116+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 0x561ac8ae2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.116+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8adef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.116+0000 7fb33f3dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 0x561ac8adf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.116+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301b2980 con 0x561ac8708390 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.116+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301b2280 con 0x561ac8729070 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.116+0000 7fb33f3dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301af420 con 0x561ac87001b0 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.116+0000 7fb33f3dd740 10 client.0 _ll_register_callbacks cb 0x7fb3240ab810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb335200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8adef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb335c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 0x561ac8ae2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb336600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 0x561ac8adf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb335200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8adef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46074/0 (socket says 172.21.15.32:46074) 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb335c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 0x561ac8ae2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34610/0 (socket says 172.21.15.32:34610) 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb335200640 1 -- 172.21.15.32:0/1000343850 learned_addr learned my addr 172.21.15.32:0/1000343850 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb335200640 1 -- 172.21.15.32:0/1000343850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 msgr2=0x561ac8ae2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb335200640 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 0x561ac8ae2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb335200640 1 -- 172.21.15.32:0/1000343850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 msgr2=0x561ac8adf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb335200640 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 0x561ac8adf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.117+0000 7fb335200640 1 -- 172.21.15.32:0/1000343850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ac8adfa30 con 0x561ac8708390 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.118+0000 7fb335200640 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8adef80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0095c0 tx=0x7fb31c005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9c0538b1c8cd2c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.118+0000 7fb317400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.118+0000 7fb317400640 1 -- 172.21.15.32:0/1000343850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c035050 con 0x561ac8708390 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.118+0000 7fb317400640 1 -- 172.21.15.32:0/1000343850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fb31c004000 con 0x561ac8708390 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.118+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1000343850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb31c0035f0 con 0x561ac8708390 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.118+0000 7fb317400640 1 -- 172.21.15.32:0/1000343850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb31c004200 con 0x561ac8708390 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.118+0000 7fb33f3dd740 10 client.17490 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.118+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1000343850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561ac8b57e80 con 0x561ac8708390 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.118+0000 7fb317400640 1 -- 172.21.15.32:0/1000343850 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb31c0035f0 con 0x561ac8708390 2024-08-31T07:22:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.119+0000 7fb317400640 1 -- 172.21.15.32:0/1000343850 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb31c0524b0 con 0x561ac8708390 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.119+0000 7fb33f3dd740 10 client.17490 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.119+0000 7fb33f3dd740 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.119+0000 7fb33f3dd740 20 client.17490 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.119+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561ac86bcda0 0x561ac8b5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.119+0000 7fb33f3dd740 4 client.17490 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.119+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1000343850 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ac8a43160 con 0x561ac86bcda0 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.119+0000 7fb335c00640 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561ac86bcda0 0x561ac8b5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb335c00640 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561ac86bcda0 0x561ac8b5a120 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d584d50f316ff07d server_cookie=218d8c2421a1941f in_seq=0 out_seq=0 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb317400640 10 client.17490 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb317400640 1 -- 172.21.15.32:0/1000343850 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x561ac8a43160 con 0x561ac86bcda0 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb317400640 10 client.17490 handle_command_reply: tid=0 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb33f3dd740 1 client.17490 shutdown 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1000343850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 msgr2=0x561ac8adef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8adef80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0095c0 tx=0x7fb31c005490 comp rx=0 tx=0).stop 2024-08-31T07:22:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1000343850 shutdown_connections 2024-08-31T07:22:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561ac86bcda0 0x561ac8b5a120 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561ac8729070 0x561ac8adf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561ac8708390 0x561ac8adef80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb33f3dd740 1 --2- 172.21.15.32:0/1000343850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561ac87001b0 0x561ac8ae2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb317400640 0 client.17490 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1000343850 >> 172.21.15.32:0/1000343850 conn(0x561ac871d460 msgr2=0x561ac8acc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.120+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1000343850 shutdown_connections 2024-08-31T07:22:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb33f3dd740 1 -- 172.21.15.32:0/1000343850 wait complete. 2024-08-31T07:22:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb33f3dd740 20 client.17490 trim_cache size 0 max 16384 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:22:29.130 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.131 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:29.134 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:22:29.135 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:22:29.135 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:29.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:29.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:29.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:29.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:29.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:29.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:29.135 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:22:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb337000640 1 -- 172.21.15.32:0/3441907451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3180b4ca0 msgr2=0x7fb3180b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb337000640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3180b4ca0 0x7fb3180b71b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb328009fd0 tx=0x7fb328009500 comp rx=0 tx=0).stop 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb337000640 1 -- 172.21.15.32:0/3441907451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330105950 msgr2=0x7fb33019c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb337000640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330105950 0x7fb33019c310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb32400e9f0 tx=0x7fb32400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb337000640 1 -- 172.21.15.32:0/3441907451 shutdown_connections 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb337000640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3180b4ca0 0x7fb3180b71b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb337000640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb33010bbe0 0x7fb3301a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb337000640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb33010b290 0x7fb33019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb337000640 1 --2- 172.21.15.32:0/3441907451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330105950 0x7fb33019c310 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.121+0000 7fb337000640 1 -- 172.21.15.32:0/3441907451 >> 172.21.15.32:0/3441907451 conn(0x7fb330101420 msgr2=0x7fb330103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.122+0000 7fb337000640 1 -- 172.21.15.32:0/3441907451 shutdown_connections 2024-08-31T07:22:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:29.122+0000 7fb337000640 1 -- 172.21.15.32:0/3441907451 wait complete. 2024-08-31T07:22:29.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 313847 3.00 MB/sec execute 378 sec latency 2167.869 ms 2024-08-31T07:22:30.072 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:22:30.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:22:30.497 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:30.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 315149 3.01 MB/sec execute 379 sec latency 2532.462 ms 2024-08-31T07:22:30.815 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:30 smithi032 ceph-mon[17252]: pgmap v668: 97 pgs: 97 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-08-31T07:22:30.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:30 smithi073 ceph-mon[21995]: pgmap v668: 97 pgs: 97 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-08-31T07:22:30.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:30 smithi084 ceph-mon[22387]: pgmap v668: 97 pgs: 97 active+clean; 231 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-08-31T07:22:31.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.170+0000 7f243d200640 1 -- 172.21.15.32:0/501210781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243810bbe0 msgr2=0x7f2438112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:31.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.170+0000 7f243d200640 1 --2- 172.21.15.32:0/501210781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243810bbe0 0x7f2438112410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2430009ca0 tx=0x7f243002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.170+0000 7f243d200640 1 -- 172.21.15.32:0/501210781 shutdown_connections 2024-08-31T07:22:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.170+0000 7f243d200640 1 --2- 172.21.15.32:0/501210781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243810bbe0 0x7f2438112410 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.170+0000 7f243d200640 1 --2- 172.21.15.32:0/501210781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f243810b290 0x7f243810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.170+0000 7f243d200640 1 --2- 172.21.15.32:0/501210781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2438105950 0x7f2438105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.170+0000 7f243d200640 1 -- 172.21.15.32:0/501210781 >> 172.21.15.32:0/501210781 conn(0x7f2438101420 msgr2=0x7f2438103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.170+0000 7f243d200640 1 -- 172.21.15.32:0/501210781 shutdown_connections 2024-08-31T07:22:31.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.170+0000 7f243d200640 1 -- 172.21.15.32:0/501210781 wait complete. 2024-08-31T07:22:31.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.171+0000 7f243d200640 1 Processor -- start 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f243d200640 1 -- start start 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f243d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2438105950 0x7f243819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f243d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243810b290 0x7f243819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f243d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f243810bbe0 0x7f24381a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f243d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24381158a0 con 0x7f2438105950 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f243d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2438115720 con 0x7f243810b290 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f243d200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2438115a20 con 0x7f243810bbe0 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f2436a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243810b290 0x7f243819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f2437400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2438105950 0x7f243819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f2436a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243810b290 0x7f243819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59392/0 (socket says 172.21.15.32:59392) 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.172+0000 7f2436a00640 1 -- 172.21.15.32:0/282388322 learned_addr learned my addr 172.21.15.32:0/282388322 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:31.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.173+0000 7f2437e00640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f243810bbe0 0x7f24381a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.173+0000 7f2436a00640 1 -- 172.21.15.32:0/282388322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f243810bbe0 msgr2=0x7f24381a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.173+0000 7f2436a00640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f243810bbe0 0x7f24381a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.173+0000 7f2436a00640 1 -- 172.21.15.32:0/282388322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2438105950 msgr2=0x7f243819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.173+0000 7f2436a00640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2438105950 0x7f243819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.173+0000 7f2436a00640 1 -- 172.21.15.32:0/282388322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24381a21c0 con 0x7f243810b290 2024-08-31T07:22:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.173+0000 7f2437e00640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f243810bbe0 0x7f24381a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:22:31.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.173+0000 7f2437400640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2438105950 0x7f243819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.173+0000 7f2436a00640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243810b290 0x7f243819c7f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f242400e9f0 tx=0x7f242400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.174+0000 7f242fe00640 1 -- 172.21.15.32:0/282388322 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242400ccc0 con 0x7f243810b290 2024-08-31T07:22:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.174+0000 7f243d200640 1 -- 172.21.15.32:0/282388322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24381a24b0 con 0x7f243810b290 2024-08-31T07:22:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.174+0000 7f242fe00640 1 -- 172.21.15.32:0/282388322 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24240108e0 con 0x7f243810b290 2024-08-31T07:22:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.174+0000 7f242fe00640 1 -- 172.21.15.32:0/282388322 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424018860 con 0x7f243810b290 2024-08-31T07:22:31.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.174+0000 7f243d200640 1 -- 172.21.15.32:0/282388322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24381a8c10 con 0x7f243810b290 2024-08-31T07:22:31.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.175+0000 7f243d200640 1 -- 172.21.15.32:0/282388322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2438105d60 con 0x7f243810b290 2024-08-31T07:22:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.178+0000 7f242fe00640 1 -- 172.21.15.32:0/282388322 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2424010430 con 0x7f243810b290 2024-08-31T07:22:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.180+0000 7f242fe00640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f241c0b4ca0 0x7f241c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.180+0000 7f242fe00640 1 -- 172.21.15.32:0/282388322 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f24240e5b40 con 0x7f243810b290 2024-08-31T07:22:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.180+0000 7f2437400640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f241c0b4ca0 0x7f241c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.181+0000 7f2437400640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f241c0b4ca0 0x7f241c0b71b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2428009fc0 tx=0x7f2428005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:31.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.181+0000 7f242fe00640 1 -- 172.21.15.32:0/282388322 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f24240ac920 con 0x7f243810b290 2024-08-31T07:22:31.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.430+0000 7f243d200640 1 -- 172.21.15.32:0/282388322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f243819d380 con 0x7f243810b290 2024-08-31T07:22:31.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.431+0000 7f242fe00640 1 -- 172.21.15.32:0/282388322 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11968 (secure 0 0 0) 0x7f24240ac220 con 0x7f243810b290 2024-08-31T07:22:31.432 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:22:31.432 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"btime":"2024-08-31T07:22:06:095426+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:22:05.949269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:22:31.432 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 40 2024-08-31T07:22:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 -- 172.21.15.32:0/282388322 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f241c0b4ca0 msgr2=0x7f241c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f241c0b4ca0 0x7f241c0b71b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2428009fc0 tx=0x7f2428005f90 comp rx=0 tx=0).stop 2024-08-31T07:22:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 -- 172.21.15.32:0/282388322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243810b290 msgr2=0x7f243819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243810b290 0x7f243819c7f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f242400e9f0 tx=0x7f242400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 -- 172.21.15.32:0/282388322 shutdown_connections 2024-08-31T07:22:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f241c0b4ca0 0x7f241c0b71b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f243810bbe0 0x7f24381a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f243810b290 0x7f243819c7f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 --2- 172.21.15.32:0/282388322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2438105950 0x7f243819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:31.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 -- 172.21.15.32:0/282388322 >> 172.21.15.32:0/282388322 conn(0x7f2438101420 msgr2=0x7f243810a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:31.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 -- 172.21.15.32:0/282388322 shutdown_connections 2024-08-31T07:22:31.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:31.433+0000 7f243d200640 1 -- 172.21.15.32:0/282388322 wait complete. 2024-08-31T07:22:31.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 316313 3.02 MB/sec execute 380 sec latency 19.757 ms 2024-08-31T07:22:31.852 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:31 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/282388322' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:22:31.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:31 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/282388322' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:22:31.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:31 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/282388322' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:22:32.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:22:32.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:32 smithi032 ceph-mon[17252]: pgmap v669: 97 pgs: 97 active+clean; 232 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-08-31T07:22:32.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 317846 3.03 MB/sec execute 381 sec latency 119.178 ms 2024-08-31T07:22:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:32 smithi073 ceph-mon[21995]: pgmap v669: 97 pgs: 97 active+clean; 232 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-08-31T07:22:32.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:32 smithi084 ceph-mon[22387]: pgmap v669: 97 pgs: 97 active+clean; 232 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-08-31T07:22:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.468+0000 7f9c5fc00640 1 -- 172.21.15.32:0/1805359064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c581047b0 msgr2=0x7f9c58104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.468+0000 7f9c5fc00640 1 --2- 172.21.15.32:0/1805359064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c581047b0 0x7f9c58104bc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009900 tx=0x7f9c4c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.468+0000 7f9c5fc00640 1 -- 172.21.15.32:0/1805359064 shutdown_connections 2024-08-31T07:22:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.468+0000 7f9c5fc00640 1 --2- 172.21.15.32:0/1805359064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c5810dbe0 0x7f9c581100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.468+0000 7f9c5fc00640 1 --2- 172.21.15.32:0/1805359064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c58105100 0x7f9c5810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.468+0000 7f9c5fc00640 1 --2- 172.21.15.32:0/1805359064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c581047b0 0x7f9c58104bc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.468+0000 7f9c5fc00640 1 -- 172.21.15.32:0/1805359064 >> 172.21.15.32:0/1805359064 conn(0x7f9c580794c0 msgr2=0x7f9c580798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.468+0000 7f9c5fc00640 1 -- 172.21.15.32:0/1805359064 shutdown_connections 2024-08-31T07:22:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.468+0000 7f9c5fc00640 1 -- 172.21.15.32:0/1805359064 wait complete. 2024-08-31T07:22:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.469+0000 7f9c5fc00640 1 Processor -- start 2024-08-31T07:22:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.469+0000 7f9c5fc00640 1 -- start start 2024-08-31T07:22:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c5fc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c581047b0 0x7f9c58103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c5fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c58105100 0x7f9c581024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c5fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c5810dbe0 0x7f9c58102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c5fc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58113820 con 0x7f9c58105100 2024-08-31T07:22:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c5fc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581136a0 con 0x7f9c5810dbe0 2024-08-31T07:22:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c5fc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581139a0 con 0x7f9c581047b0 2024-08-31T07:22:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c581047b0 0x7f9c58103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c57e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c58105100 0x7f9c581024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c5dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c5810dbe0 0x7f9c58102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c581047b0 0x7f9c58103f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34646/0 (socket says 172.21.15.32:34646) 2024-08-31T07:22:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.470+0000 7f9c5d200640 1 -- 172.21.15.32:0/2023372759 learned_addr learned my addr 172.21.15.32:0/2023372759 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.471+0000 7f9c5d200640 1 -- 172.21.15.32:0/2023372759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c5810dbe0 msgr2=0x7f9c58102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.471+0000 7f9c5d200640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c5810dbe0 0x7f9c58102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.471+0000 7f9c5d200640 1 -- 172.21.15.32:0/2023372759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c58105100 msgr2=0x7f9c581024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.471+0000 7f9c5d200640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c58105100 0x7f9c581024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.471+0000 7f9c5d200640 1 -- 172.21.15.32:0/2023372759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c58103250 con 0x7f9c581047b0 2024-08-31T07:22:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.471+0000 7f9c5d200640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c581047b0 0x7f9c58103f00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c005940 tx=0x7f9c4c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.472+0000 7f9c55600640 1 -- 172.21.15.32:0/2023372759 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c03f030 con 0x7f9c581047b0 2024-08-31T07:22:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.472+0000 7f9c5fc00640 1 -- 172.21.15.32:0/2023372759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c581acbb0 con 0x7f9c581047b0 2024-08-31T07:22:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.472+0000 7f9c5fc00640 1 -- 172.21.15.32:0/2023372759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c581ad0e0 con 0x7f9c581047b0 2024-08-31T07:22:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.472+0000 7f9c55600640 1 -- 172.21.15.32:0/2023372759 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c4c0044c0 con 0x7f9c581047b0 2024-08-31T07:22:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.472+0000 7f9c55600640 1 -- 172.21.15.32:0/2023372759 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c049060 con 0x7f9c581047b0 2024-08-31T07:22:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.475+0000 7f9c66891740 1 Processor -- start 2024-08-31T07:22:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.476+0000 7f9c66891740 1 -- start start 2024-08-31T07:22:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.476+0000 7f9c66891740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d497990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.476+0000 7f9c66891740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0c91b0 0x55971d497ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.476+0000 7f9c66891740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d498410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.476+0000 7f9c66891740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55971d40e7b0 con 0x55971d0f2070 2024-08-31T07:22:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.476+0000 7f9c66891740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55971d3fdd40 con 0x55971d0c91b0 2024-08-31T07:22:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.476+0000 7f9c66891740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55971d401cf0 con 0x55971d0d1390 2024-08-31T07:22:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.476+0000 7f9c57e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d498410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.476+0000 7f9c5dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d497990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.477+0000 7f9c57e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d498410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34652/0 (socket says 172.21.15.32:34652) 2024-08-31T07:22:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.477+0000 7f9c57e00640 1 -- 172.21.15.32:0/20588787 learned_addr learned my addr 172.21.15.32:0/20588787 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.477+0000 7f9c55600640 1 -- 172.21.15.32:0/2023372759 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9c4c042030 con 0x7f9c581047b0 2024-08-31T07:22:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.477+0000 7f9c57e00640 1 -- 172.21.15.32:0/20588787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0c91b0 msgr2=0x55971d497ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.477+0000 7f9c57e00640 1 --2- 172.21.15.32:0/20588787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0c91b0 0x55971d497ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.477+0000 7f9c57e00640 1 -- 172.21.15.32:0/20588787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 msgr2=0x55971d497990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.477+0000 7f9c57e00640 1 --2- 172.21.15.32:0/20588787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d497990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.477+0000 7f9c57e00640 1 -- 172.21.15.32:0/20588787 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971d15d650 con 0x55971d0d1390 2024-08-31T07:22:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.477+0000 7f9c57e00640 1 --2- 172.21.15.32:0/20588787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d498410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009c20 tx=0x7f9c44011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b64537de1ba4c3b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.478+0000 7f9c4a000640 1 -- 172.21.15.32:0/20588787 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44004030 con 0x55971d0d1390 2024-08-31T07:22:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.478+0000 7f9c4a000640 1 -- 172.21.15.32:0/20588787 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c44005020 con 0x55971d0d1390 2024-08-31T07:22:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.478+0000 7f9c4a000640 1 -- 172.21.15.32:0/20588787 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44004ca0 con 0x55971d0d1390 2024-08-31T07:22:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c66891740 1 -- 172.21.15.32:0/20588787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 msgr2=0x55971d498410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c66891740 1 --2- 172.21.15.32:0/20588787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d498410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009c20 tx=0x7f9c44011810 comp rx=0 tx=0).stop 2024-08-31T07:22:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c66891740 1 -- 172.21.15.32:0/20588787 shutdown_connections 2024-08-31T07:22:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c66891740 1 --2- 172.21.15.32:0/20588787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d498410 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c66891740 1 --2- 172.21.15.32:0/20588787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0c91b0 0x55971d497ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c66891740 1 --2- 172.21.15.32:0/20588787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d497990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c66891740 1 -- 172.21.15.32:0/20588787 >> 172.21.15.32:0/20588787 conn(0x55971d0e6460 msgr2=0x55971d495880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c55600640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c400b4ca0 0x7f9c400b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c66891740 1 -- 172.21.15.32:0/20588787 shutdown_connections 2024-08-31T07:22:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c66891740 1 -- 172.21.15.32:0/20588787 wait complete. 2024-08-31T07:22:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.479+0000 7f9c55600640 1 -- 172.21.15.32:0/2023372759 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f9c4c104ff0 con 0x7f9c581047b0 2024-08-31T07:22:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.480+0000 7f9c5dc00640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c400b4ca0 0x7f9c400b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.480+0000 7f9c66891740 1 -- start start 2024-08-31T07:22:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.480+0000 7f9c66891740 1 Processor -- start 2024-08-31T07:22:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.480+0000 7f9c5dc00640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c400b4ca0 0x7f9c400b71b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c581045b0 tx=0x7f9c50009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c66891740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0c91b0 0x55971d4a3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c66891740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 0x55971d4a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c66891740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d4af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c66891740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55971d260700 con 0x55971d0f2070 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c66891740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55971d384cf0 con 0x55971d0d1390 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c66891740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55971d4a6170 con 0x55971d0c91b0 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c66891740 10 client.0 _ll_register_callbacks cb 0x55971d3aa470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c57e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 0x55971d4a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c57e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 0x55971d4a3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59412/0 (socket says 172.21.15.32:59412) 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c57e00640 1 -- 172.21.15.32:0/551287616 learned_addr learned my addr 172.21.15.32:0/551287616 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.481+0000 7f9c5dc00640 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d4af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.482+0000 7f9c5d200640 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0c91b0 0x55971d4a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.482+0000 7f9c57e00640 1 -- 172.21.15.32:0/551287616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0c91b0 msgr2=0x55971d4a3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.482+0000 7f9c57e00640 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0c91b0 0x55971d4a3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.482+0000 7f9c57e00640 1 -- 172.21.15.32:0/551287616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 msgr2=0x55971d4af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.482+0000 7f9c57e00640 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d4af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.482+0000 7f9c57e00640 1 -- 172.21.15.32:0/551287616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971d3c58b0 con 0x55971d0d1390 2024-08-31T07:22:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.482+0000 7f9c57e00640 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 0x55971d4a3a90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c44004270 tx=0x7f9c440042a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77f5a656e1225d71 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.482+0000 7f9c3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:22:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.483+0000 7f9c3ea00640 1 -- 172.21.15.32:0/551287616 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44027020 con 0x55971d0d1390 2024-08-31T07:22:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.483+0000 7f9c3ea00640 1 -- 172.21.15.32:0/551287616 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f9c440043c0 con 0x55971d0d1390 2024-08-31T07:22:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.483+0000 7f9c66891740 1 -- 172.21.15.32:0/551287616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c44005020 con 0x55971d0d1390 2024-08-31T07:22:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.483+0000 7f9c3ea00640 1 -- 172.21.15.32:0/551287616 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9c44032050 con 0x55971d0d1390 2024-08-31T07:22:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.483+0000 7f9c66891740 10 client.27512 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.483+0000 7f9c66891740 1 -- 172.21.15.32:0/551287616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55971d4b21c0 con 0x55971d0d1390 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.484+0000 7f9c3ea00640 1 -- 172.21.15.32:0/551287616 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c44005020 con 0x55971d0d1390 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.484+0000 7f9c3ea00640 1 -- 172.21.15.32:0/551287616 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f9c44018070 con 0x55971d0d1390 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.484+0000 7f9c66891740 10 client.27512 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.484+0000 7f9c66891740 10 client.27512 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.484+0000 7f9c66891740 20 client.27512 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.484+0000 7f9c66891740 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55971d0b09f0 0x55971d4b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.484+0000 7f9c66891740 4 client.27512 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.484+0000 7f9c66891740 1 -- 172.21.15.32:0/551287616 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55971d410700 con 0x55971d0b09f0 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.484+0000 7f9c5d200640 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55971d0b09f0 0x55971d4b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.485+0000 7f9c5d200640 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55971d0b09f0 0x55971d4b5520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=235d0a82d548c7b server_cookie=1a114d63e39ae74c in_seq=0 out_seq=0 2024-08-31T07:22:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.485+0000 7f9c3ea00640 10 client.27512 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:22:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c3ea00640 1 -- 172.21.15.32:0/551287616 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55971d410700 con 0x55971d0b09f0 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c3ea00640 10 client.27512 handle_command_reply: tid=0 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c66891740 1 client.27512 shutdown 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c66891740 1 -- 172.21.15.32:0/551287616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 msgr2=0x55971d4a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c66891740 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 0x55971d4a3a90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c44004270 tx=0x7f9c440042a0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c66891740 1 -- 172.21.15.32:0/551287616 shutdown_connections 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c66891740 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55971d0b09f0 0x55971d4b5520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c66891740 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d4af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c66891740 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 0x55971d4a3a90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c66891740 1 --2- 172.21.15.32:0/551287616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0c91b0 0x55971d4a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c66891740 1 -- 172.21.15.32:0/551287616 >> 172.21.15.32:0/551287616 conn(0x55971d0e6460 msgr2=0x55971d49ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.486+0000 7f9c3ea00640 0 client.27512 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.487+0000 7f9c66891740 1 -- 172.21.15.32:0/551287616 shutdown_connections 2024-08-31T07:22:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.487+0000 7f9c66891740 1 -- 172.21.15.32:0/551287616 wait complete. 2024-08-31T07:22:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.487+0000 7f9c66891740 20 client.27512 trim_cache size 0 max 16384 2024-08-31T07:22:33.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.502+0000 7f9c66891740 1 Processor -- start 2024-08-31T07:22:33.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c66891740 1 -- start start 2024-08-31T07:22:33.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c66891740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0c91b0 0x55971d49eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c66891740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d49b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c66891740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d4990d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c66891740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3800e6b0 con 0x55971d0f2070 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c66891740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c380102e0 con 0x55971d0c91b0 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c66891740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3800ed80 con 0x55971d0d1390 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c57e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d49b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0c91b0 0x55971d49eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c57e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d49b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34678/0 (socket says 172.21.15.32:34678) 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.503+0000 7f9c57e00640 1 -- 172.21.15.32:0/4196626361 learned_addr learned my addr 172.21.15.32:0/4196626361 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c5dc00640 1 --2- 172.21.15.32:0/4196626361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d4990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c57e00640 1 -- 172.21.15.32:0/4196626361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0c91b0 msgr2=0x55971d49eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c57e00640 1 --2- 172.21.15.32:0/4196626361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0c91b0 0x55971d49eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c57e00640 1 -- 172.21.15.32:0/4196626361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 msgr2=0x55971d4990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c57e00640 1 --2- 172.21.15.32:0/4196626361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d4990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c57e00640 1 -- 172.21.15.32:0/4196626361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971d49a7f0 con 0x55971d0d1390 2024-08-31T07:22:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c5dc00640 1 --2- 172.21.15.32:0/4196626361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d4990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:22:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c5d200640 1 --2- 172.21.15.32:0/4196626361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0c91b0 0x55971d49eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c57e00640 1 --2- 172.21.15.32:0/4196626361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d49b450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9c44003d40 tx=0x7f9c44005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ef55d7e81f1c25 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c49600640 1 -- 172.21.15.32:0/4196626361 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44027070 con 0x55971d0d1390 2024-08-31T07:22:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c49600640 1 -- 172.21.15.32:0/4196626361 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c44009dd0 con 0x55971d0d1390 2024-08-31T07:22:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.504+0000 7f9c49600640 1 -- 172.21.15.32:0/4196626361 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4401e040 con 0x55971d0d1390 2024-08-31T07:22:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.505+0000 7f9c66891740 1 -- 172.21.15.32:0/4196626361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 msgr2=0x55971d49b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.505+0000 7f9c66891740 1 --2- 172.21.15.32:0/4196626361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d49b450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9c44003d40 tx=0x7f9c44005740 comp rx=0 tx=0).stop 2024-08-31T07:22:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.506+0000 7f9c66891740 1 -- 172.21.15.32:0/4196626361 shutdown_connections 2024-08-31T07:22:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.506+0000 7f9c66891740 1 --2- 172.21.15.32:0/4196626361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0f2070 0x55971d4990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.506+0000 7f9c66891740 1 --2- 172.21.15.32:0/4196626361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0d1390 0x55971d49b450 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.506+0000 7f9c66891740 1 --2- 172.21.15.32:0/4196626361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0c91b0 0x55971d49eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.506+0000 7f9c66891740 1 -- 172.21.15.32:0/4196626361 >> 172.21.15.32:0/4196626361 conn(0x55971d0e6460 msgr2=0x55971d4989c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:33.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.506+0000 7f9c66891740 1 -- 172.21.15.32:0/4196626361 shutdown_connections 2024-08-31T07:22:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.506+0000 7f9c66891740 1 -- 172.21.15.32:0/4196626361 wait complete. 2024-08-31T07:22:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.506+0000 7f9c66891740 1 -- start start 2024-08-31T07:22:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.507+0000 7f9c66891740 1 Processor -- start 2024-08-31T07:22:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.507+0000 7f9c66891740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0c91b0 0x55971d4a7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.507+0000 7f9c66891740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 0x55971d4a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.507+0000 7f9c66891740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0f2070 0x55971d4a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.507+0000 7f9c66891740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3800fc10 con 0x55971d0c91b0 2024-08-31T07:22:33.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.507+0000 7f9c66891740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3800f510 con 0x55971d0d1390 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.507+0000 7f9c66891740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3800c6b0 con 0x55971d0f2070 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.507+0000 7f9c66891740 10 client.0 _ll_register_callbacks cb 0x7f9c440055e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.508+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0c91b0 0x55971d4a7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.508+0000 7f9c57e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 0x55971d4a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.508+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0c91b0 0x55971d4a7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46128/0 (socket says 172.21.15.32:46128) 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.508+0000 7f9c5d200640 1 -- 172.21.15.32:0/3464220840 learned_addr learned my addr 172.21.15.32:0/3464220840 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.508+0000 7f9c5d200640 1 -- 172.21.15.32:0/3464220840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0f2070 msgr2=0x55971d4a7e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.508+0000 7f9c5d200640 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0f2070 0x55971d4a7e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.508+0000 7f9c5d200640 1 -- 172.21.15.32:0/3464220840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 msgr2=0x55971d4a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.508+0000 7f9c5d200640 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 0x55971d4a7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.508+0000 7f9c5d200640 1 -- 172.21.15.32:0/3464220840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971d51edd0 con 0x55971d0c91b0 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.509+0000 7f9c5d200640 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0c91b0 0x55971d4a7410 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c102840 tx=0x7f9c4c1053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9226a15883bd4e8a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.509+0000 7f9c3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:22:33.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.509+0000 7f9c3ea00640 1 -- 172.21.15.32:0/3464220840 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c035040 con 0x55971d0c91b0 2024-08-31T07:22:33.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.509+0000 7f9c66891740 1 -- 172.21.15.32:0/3464220840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c44009dd0 con 0x55971d0c91b0 2024-08-31T07:22:33.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.509+0000 7f9c3ea00640 1 -- 172.21.15.32:0/3464220840 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f9c4c102a10 con 0x55971d0c91b0 2024-08-31T07:22:33.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.509+0000 7f9c3ea00640 1 -- 172.21.15.32:0/3464220840 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9c4c0cffa0 con 0x55971d0c91b0 2024-08-31T07:22:33.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.509+0000 7f9c66891740 10 client.17511 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:33.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.509+0000 7f9c66891740 1 -- 172.21.15.32:0/3464220840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55971d51ffb0 con 0x55971d0c91b0 2024-08-31T07:22:33.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.510+0000 7f9c3ea00640 1 -- 172.21.15.32:0/3464220840 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c44009dd0 con 0x55971d0c91b0 2024-08-31T07:22:33.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.510+0000 7f9c3ea00640 1 -- 172.21.15.32:0/3464220840 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f9c4c0d45c0 con 0x55971d0c91b0 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.510+0000 7f9c66891740 10 client.17511 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.510+0000 7f9c66891740 10 client.17511 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.510+0000 7f9c66891740 20 client.17511 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.510+0000 7f9c66891740 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55971d085da0 0x55971d5226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.510+0000 7f9c66891740 4 client.17511 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.510+0000 7f9c66891740 1 -- 172.21.15.32:0/3464220840 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55971d40c1a0 con 0x55971d085da0 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.511+0000 7f9c57e00640 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55971d085da0 0x55971d5226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.511+0000 7f9c57e00640 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55971d085da0 0x55971d5226d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cc364dfb519d9f29 server_cookie=ab3bf5a02e089d15 in_seq=0 out_seq=0 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.511+0000 7f9c3ea00640 10 client.17511 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c3ea00640 1 -- 172.21.15.32:0/3464220840 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55971d40c1a0 con 0x55971d085da0 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c3ea00640 10 client.17511 handle_command_reply: tid=0 2024-08-31T07:22:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c66891740 1 client.17511 shutdown 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c66891740 1 -- 172.21.15.32:0/3464220840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0c91b0 msgr2=0x55971d4a7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c66891740 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0c91b0 0x55971d4a7410 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c102840 tx=0x7f9c4c1053b0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c66891740 1 -- 172.21.15.32:0/3464220840 shutdown_connections 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c66891740 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55971d085da0 0x55971d5226d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c66891740 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55971d0f2070 0x55971d4a7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c66891740 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55971d0d1390 0x55971d4a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c66891740 1 --2- 172.21.15.32:0/3464220840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55971d0c91b0 0x55971d4a7410 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c66891740 1 -- 172.21.15.32:0/3464220840 >> 172.21.15.32:0/3464220840 conn(0x55971d0e6460 msgr2=0x55971d49b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.512+0000 7f9c3ea00640 0 client.17511 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.513+0000 7f9c66891740 1 -- 172.21.15.32:0/3464220840 shutdown_connections 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.513+0000 7f9c66891740 1 -- 172.21.15.32:0/3464220840 wait complete. 2024-08-31T07:22:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.513+0000 7f9c66891740 20 client.17511 trim_cache size 0 max 16384 2024-08-31T07:22:33.516 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:22:33.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 -- 172.21.15.32:0/2023372759 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c400b4ca0 msgr2=0x7f9c400b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:33.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c400b4ca0 0x7f9c400b71b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c581045b0 tx=0x7f9c50009500 comp rx=0 tx=0).stop 2024-08-31T07:22:33.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 -- 172.21.15.32:0/2023372759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c581047b0 msgr2=0x7f9c58103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:33.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c581047b0 0x7f9c58103f00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c005940 tx=0x7f9c4c0040c0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 -- 172.21.15.32:0/2023372759 shutdown_connections 2024-08-31T07:22:33.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c400b4ca0 0x7f9c400b71b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c5810dbe0 0x7f9c58102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c58105100 0x7f9c581024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 --2- 172.21.15.32:0/2023372759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c581047b0 0x7f9c58103f00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:33.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 -- 172.21.15.32:0/2023372759 >> 172.21.15.32:0/2023372759 conn(0x7f9c580794c0 msgr2=0x7f9c581005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:33.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 -- 172.21.15.32:0/2023372759 shutdown_connections 2024-08-31T07:22:33.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:33.514+0000 7f9c5fc00640 1 -- 172.21.15.32:0/2023372759 wait complete. 2024-08-31T07:22:33.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 319538 3.05 MB/sec execute 382 sec latency 13.946 ms 2024-08-31T07:22:34.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:22:34.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 320284 3.05 MB/sec execute 383 sec latency 157.495 ms 2024-08-31T07:22:34.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:34 smithi032 ceph-mon[17252]: pgmap v670: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-08-31T07:22:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:34 smithi073 ceph-mon[21995]: pgmap v670: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-08-31T07:22:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:34 smithi084 ceph-mon[22387]: pgmap v670: 97 pgs: 97 active+clean; 235 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-08-31T07:22:35.105 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:35.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 320938 3.05 MB/sec execute 384 sec latency 468.880 ms 2024-08-31T07:22:35.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.779+0000 7fc2dd800640 1 -- 172.21.15.32:0/820997496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2d8104440 msgr2=0x7fc2d806b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:35.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.779+0000 7fc2dd800640 1 --2- 172.21.15.32:0/820997496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2d8104440 0x7fc2d806b860 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009900 tx=0x7fc2c802a400 comp rx=0 tx=0).stop 2024-08-31T07:22:35.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.779+0000 7fc2dd800640 1 -- 172.21.15.32:0/820997496 shutdown_connections 2024-08-31T07:22:35.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.779+0000 7fc2dd800640 1 --2- 172.21.15.32:0/820997496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2d810fde0 0x7fc2d81122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.779+0000 7fc2dd800640 1 --2- 172.21.15.32:0/820997496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d806be80 0x7fc2d806c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.779+0000 7fc2dd800640 1 --2- 172.21.15.32:0/820997496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2d8104440 0x7fc2d806b860 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.779+0000 7fc2dd800640 1 -- 172.21.15.32:0/820997496 >> 172.21.15.32:0/820997496 conn(0x7fc2d8079740 msgr2=0x7fc2d8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:35.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.779+0000 7fc2dd800640 1 -- 172.21.15.32:0/820997496 shutdown_connections 2024-08-31T07:22:35.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.779+0000 7fc2dd800640 1 -- 172.21.15.32:0/820997496 wait complete. 2024-08-31T07:22:35.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2dd800640 1 Processor -- start 2024-08-31T07:22:35.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2dd800640 1 -- start start 2024-08-31T07:22:35.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2dd800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2d806be80 0x7fc2d819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2dd800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d8104440 0x7fc2d819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2dd800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2d810fde0 0x7fc2d81a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2dd800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8115a80 con 0x7fc2d8104440 2024-08-31T07:22:35.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2dd800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8115900 con 0x7fc2d810fde0 2024-08-31T07:22:35.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2dd800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8115c00 con 0x7fc2d806be80 2024-08-31T07:22:35.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2d6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2d806be80 0x7fc2d819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2ce000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d8104440 0x7fc2d819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2d6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2d806be80 0x7fc2d819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34708/0 (socket says 172.21.15.32:34708) 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.781+0000 7fc2d6a00640 1 -- 172.21.15.32:0/3572395569 learned_addr learned my addr 172.21.15.32:0/3572395569 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.782+0000 7fc2d7400640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2d810fde0 0x7fc2d81a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.782+0000 7fc2d7400640 1 -- 172.21.15.32:0/3572395569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2d806be80 msgr2=0x7fc2d819c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.782+0000 7fc2d7400640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2d806be80 0x7fc2d819c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.782+0000 7fc2d7400640 1 -- 172.21.15.32:0/3572395569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d8104440 msgr2=0x7fc2d819c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.782+0000 7fc2d7400640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d8104440 0x7fc2d819c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.782+0000 7fc2d7400640 1 -- 172.21.15.32:0/3572395569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d81a23a0 con 0x7fc2d810fde0 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.782+0000 7fc2d6a00640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2d806be80 0x7fc2d819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.782+0000 7fc2ce000640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d8104440 0x7fc2d819c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.782+0000 7fc2d7400640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2d810fde0 0x7fc2d81a1c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000e940 tx=0x7fc2d000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.783+0000 7fc2cfe00640 1 -- 172.21.15.32:0/3572395569 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0004d20 con 0x7fc2d810fde0 2024-08-31T07:22:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.783+0000 7fc2cfe00640 1 -- 172.21.15.32:0/3572395569 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d0005020 con 0x7fc2d810fde0 2024-08-31T07:22:35.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.783+0000 7fc2cfe00640 1 -- 172.21.15.32:0/3572395569 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00116d0 con 0x7fc2d810fde0 2024-08-31T07:22:35.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.783+0000 7fc2dd800640 1 -- 172.21.15.32:0/3572395569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d81a2690 con 0x7fc2d810fde0 2024-08-31T07:22:35.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.783+0000 7fc2dd800640 1 -- 172.21.15.32:0/3572395569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d81a8df0 con 0x7fc2d810fde0 2024-08-31T07:22:35.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.786+0000 7fc2e4468740 1 Processor -- start 2024-08-31T07:22:35.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.786+0000 7fc2cfe00640 1 -- 172.21.15.32:0/3572395569 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc2d0005460 con 0x7fc2d810fde0 2024-08-31T07:22:35.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.787+0000 7fc2e4468740 1 -- start start 2024-08-31T07:22:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.787+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3d5070 0x55570e77a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.787+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3ac1b0 0x55570e77adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.787+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 0x55570e77b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.787+0000 7fc2e4468740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55570e6f17b0 con 0x55570e3ac1b0 2024-08-31T07:22:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.787+0000 7fc2e4468740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55570e6e0d40 con 0x55570e3d5070 2024-08-31T07:22:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.787+0000 7fc2e4468740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55570e6e4cf0 con 0x55570e3b4390 2024-08-31T07:22:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.788+0000 7fc2d6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3ac1b0 0x55570e77adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.788+0000 7fc2d6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3ac1b0 0x55570e77adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46152/0 (socket says 172.21.15.32:46152) 2024-08-31T07:22:35.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.788+0000 7fc2d6a00640 1 -- 172.21.15.32:0/3915211375 learned_addr learned my addr 172.21.15.32:0/3915211375 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:35.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.788+0000 7fc2ce000640 1 --2- 172.21.15.32:0/3915211375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 0x55570e77b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.788+0000 7fc2ce000640 1 --2- 172.21.15.32:0/3915211375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3d5070 0x55570e77a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.788+0000 7fc2d6a00640 1 -- 172.21.15.32:0/3915211375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 msgr2=0x55570e77b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.788+0000 7fc2d6a00640 1 --2- 172.21.15.32:0/3915211375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 0x55570e77b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.788+0000 7fc2d6a00640 1 -- 172.21.15.32:0/3915211375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3d5070 msgr2=0x55570e77a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.788+0000 7fc2d6a00640 1 --2- 172.21.15.32:0/3915211375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3d5070 0x55570e77a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.788+0000 7fc2d6a00640 1 -- 172.21.15.32:0/3915211375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570e440650 con 0x55570e3ac1b0 2024-08-31T07:22:35.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.789+0000 7fc2d6a00640 1 --2- 172.21.15.32:0/3915211375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3ac1b0 0x55570e77adf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8000c00 tx=0x7fc2c80314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e9564106999eb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:35.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.789+0000 7fc2cfe00640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc2bc0b4d20 0x7fc2bc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.789+0000 7fc2c7e00640 1 -- 172.21.15.32:0/3915211375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8035040 con 0x55570e3ac1b0 2024-08-31T07:22:35.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.789+0000 7fc2cfe00640 1 -- 172.21.15.32:0/3572395569 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fc2d00e5860 con 0x7fc2d810fde0 2024-08-31T07:22:35.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.789+0000 7fc2c7e00640 1 -- 172.21.15.32:0/3915211375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c8031890 con 0x55570e3ac1b0 2024-08-31T07:22:35.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.789+0000 7fc2c7e00640 1 -- 172.21.15.32:0/3915211375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8004a00 con 0x55570e3ac1b0 2024-08-31T07:22:35.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.789+0000 7fc2ce000640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc2bc0b4d20 0x7fc2bc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.790+0000 7fc2e4468740 1 -- 172.21.15.32:0/3915211375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3ac1b0 msgr2=0x55570e77adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:35.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.790+0000 7fc2e4468740 1 --2- 172.21.15.32:0/3915211375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3ac1b0 0x55570e77adf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8000c00 tx=0x7fc2c80314a0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.790+0000 7fc2ce000640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc2bc0b4d20 0x7fc2bc0b7230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc2d806ae40 tx=0x7fc2c0004730 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:35.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.790+0000 7fc2e4468740 1 -- 172.21.15.32:0/3915211375 shutdown_connections 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.790+0000 7fc2e4468740 1 --2- 172.21.15.32:0/3915211375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 0x55570e77b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.790+0000 7fc2e4468740 1 --2- 172.21.15.32:0/3915211375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3ac1b0 0x55570e77adf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.790+0000 7fc2e4468740 1 --2- 172.21.15.32:0/3915211375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3d5070 0x55570e77a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.790+0000 7fc2e4468740 1 -- 172.21.15.32:0/3915211375 >> 172.21.15.32:0/3915211375 conn(0x55570e3c9460 msgr2=0x55570e7787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.790+0000 7fc2e4468740 1 -- 172.21.15.32:0/3915211375 shutdown_connections 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.790+0000 7fc2e4468740 1 -- 172.21.15.32:0/3915211375 wait complete. 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.791+0000 7fc2e4468740 1 -- start start 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.791+0000 7fc2e4468740 1 Processor -- start 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.791+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e786510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.791+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 0x55570e786a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3d5070 0x55570e792350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2e4468740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55570e543700 con 0x55570e3d5070 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2e4468740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55570e667cf0 con 0x55570e3ac1b0 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2e4468740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55570e789030 con 0x55570e3b4390 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2e4468740 10 client.0 _ll_register_callbacks cb 0x55570e68d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2d6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e786510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2ce000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 0x55570e786a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2d7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3d5070 0x55570e792350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2ce000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 0x55570e786a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34726/0 (socket says 172.21.15.32:34726) 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2d7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3d5070 0x55570e792350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46160/0 (socket says 172.21.15.32:46160) 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2d6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e786510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59458/0 (socket says 172.21.15.32:59458) 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2ce000640 1 -- 172.21.15.32:0/385036962 learned_addr learned my addr 172.21.15.32:0/385036962 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2ce000640 1 -- 172.21.15.32:0/385036962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 msgr2=0x55570e786510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2ce000640 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e786510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2ce000640 1 -- 172.21.15.32:0/385036962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3d5070 msgr2=0x55570e792350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2ce000640 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3d5070 0x55570e792350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.792+0000 7fc2ce000640 1 -- 172.21.15.32:0/385036962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570e6a88b0 con 0x55570e3b4390 2024-08-31T07:22:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.793+0000 7fc2ce000640 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 0x55570e786a90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0003cd0 tx=0x7fc2c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eddd81c51b8653c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.793+0000 7fc2c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:22:35.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.793+0000 7fc2c4c00640 1 -- 172.21.15.32:0/385036962 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0031060 con 0x55570e3b4390 2024-08-31T07:22:35.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.793+0000 7fc2c4c00640 1 -- 172.21.15.32:0/385036962 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fc2c001a050 con 0x55570e3b4390 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.793+0000 7fc2e4468740 1 -- 172.21.15.32:0/385036962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c8031890 con 0x55570e3b4390 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.794+0000 7fc2c4c00640 1 -- 172.21.15.32:0/385036962 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc2c0021920 con 0x55570e3b4390 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.794+0000 7fc2e4468740 10 client.27532 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.794+0000 7fc2e4468740 1 -- 172.21.15.32:0/385036962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55570e795120 con 0x55570e3b4390 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.794+0000 7fc2c4c00640 1 -- 172.21.15.32:0/385036962 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c8031890 con 0x55570e3b4390 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.794+0000 7fc2c4c00640 1 -- 172.21.15.32:0/385036962 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fc2c001f910 con 0x55570e3b4390 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.795+0000 7fc2e4468740 10 client.27532 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.795+0000 7fc2e4468740 10 client.27532 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.795+0000 7fc2e4468740 20 client.27532 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.795+0000 7fc2e4468740 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55570e3939f0 0x55570e798710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.795+0000 7fc2e4468740 4 client.27532 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.795+0000 7fc2e4468740 1 -- 172.21.15.32:0/385036962 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55570e6f3700 con 0x55570e3939f0 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.795+0000 7fc2d6a00640 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55570e3939f0 0x55570e798710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.796+0000 7fc2d6a00640 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55570e3939f0 0x55570e798710 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b20f63a1d87176aa server_cookie=a534a6f5b2aa9a1d in_seq=0 out_seq=0 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.796+0000 7fc2c4c00640 10 client.27532 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2c4c00640 1 -- 172.21.15.32:0/385036962 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55570e6f3700 con 0x55570e3939f0 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2c4c00640 10 client.27532 handle_command_reply: tid=0 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2e4468740 1 client.27532 shutdown 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2e4468740 1 -- 172.21.15.32:0/385036962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 msgr2=0x55570e786a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2e4468740 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 0x55570e786a90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0003cd0 tx=0x7fc2c00046c0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2e4468740 1 -- 172.21.15.32:0/385036962 shutdown_connections 2024-08-31T07:22:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2e4468740 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55570e3939f0 0x55570e798710 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2e4468740 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3d5070 0x55570e792350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2e4468740 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3b4390 0x55570e786a90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2e4468740 1 --2- 172.21.15.32:0/385036962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e786510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2c4c00640 0 client.27532 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:22:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2e4468740 1 -- 172.21.15.32:0/385036962 >> 172.21.15.32:0/385036962 conn(0x55570e3c9460 msgr2=0x55570e77da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.797+0000 7fc2e4468740 1 -- 172.21.15.32:0/385036962 shutdown_connections 2024-08-31T07:22:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.798+0000 7fc2e4468740 1 -- 172.21.15.32:0/385036962 wait complete. 2024-08-31T07:22:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.798+0000 7fc2e4468740 20 client.27532 trim_cache size 0 max 16384 2024-08-31T07:22:35.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.812+0000 7fc2e4468740 1 Processor -- start 2024-08-31T07:22:35.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.812+0000 7fc2e4468740 1 -- start start 2024-08-31T07:22:35.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.812+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e7800d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.812+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 0x55570e7780c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.812+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 0x55570e778600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.812+0000 7fc2e4468740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81b1620 con 0x55570e3b4390 2024-08-31T07:22:35.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.812+0000 7fc2e4468740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81b3250 con 0x55570e3ac1b0 2024-08-31T07:22:35.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.812+0000 7fc2e4468740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81b1cf0 con 0x55570e3d5070 2024-08-31T07:22:35.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2ce000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 0x55570e7780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2ce000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 0x55570e7780c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46162/0 (socket says 172.21.15.32:46162) 2024-08-31T07:22:35.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2ce000640 1 -- 172.21.15.32:0/3091881493 learned_addr learned my addr 172.21.15.32:0/3091881493 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:35.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2d7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 0x55570e778600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2d6a00640 1 --2- 172.21.15.32:0/3091881493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e7800d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2ce000640 1 -- 172.21.15.32:0/3091881493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 msgr2=0x55570e778600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:35.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2ce000640 1 --2- 172.21.15.32:0/3091881493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 0x55570e778600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2ce000640 1 -- 172.21.15.32:0/3091881493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 msgr2=0x55570e7800d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:35.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2ce000640 1 --2- 172.21.15.32:0/3091881493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e7800d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2ce000640 1 -- 172.21.15.32:0/3091881493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570e77cba0 con 0x55570e3b4390 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2d7400640 1 --2- 172.21.15.32:0/3091881493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 0x55570e778600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.813+0000 7fc2ce000640 1 --2- 172.21.15.32:0/3091881493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 0x55570e7780c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc2c001fea0 tx=0x7fc2c00117c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f56914c1b60e9e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.814+0000 7fc2c7400640 1 -- 172.21.15.32:0/3091881493 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0021ac0 con 0x55570e3b4390 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.814+0000 7fc2c7400640 1 -- 172.21.15.32:0/3091881493 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c0021c60 con 0x55570e3b4390 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.814+0000 7fc2c7400640 1 -- 172.21.15.32:0/3091881493 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c001fa10 con 0x55570e3b4390 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.814+0000 7fc2e4468740 1 -- 172.21.15.32:0/3091881493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 msgr2=0x55570e7780c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.815+0000 7fc2e4468740 1 --2- 172.21.15.32:0/3091881493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 0x55570e7780c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc2c001fea0 tx=0x7fc2c00117c0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.815+0000 7fc2e4468740 1 -- 172.21.15.32:0/3091881493 shutdown_connections 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.815+0000 7fc2e4468740 1 --2- 172.21.15.32:0/3091881493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 0x55570e778600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.815+0000 7fc2e4468740 1 --2- 172.21.15.32:0/3091881493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 0x55570e7780c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.815+0000 7fc2e4468740 1 --2- 172.21.15.32:0/3091881493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e7800d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.815+0000 7fc2e4468740 1 -- 172.21.15.32:0/3091881493 >> 172.21.15.32:0/3091881493 conn(0x55570e3c9460 msgr2=0x55570e77f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:35.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.815+0000 7fc2e4468740 1 -- 172.21.15.32:0/3091881493 shutdown_connections 2024-08-31T07:22:35.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.815+0000 7fc2e4468740 1 -- 172.21.15.32:0/3091881493 wait complete. 2024-08-31T07:22:35.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.815+0000 7fc2e4468740 1 -- start start 2024-08-31T07:22:35.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.815+0000 7fc2e4468740 1 Processor -- start 2024-08-31T07:22:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e78eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 0x55570e789270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2e4468740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 0x55570e7898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2e4468740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81b2b80 con 0x55570e3b4390 2024-08-31T07:22:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2e4468740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81b2480 con 0x55570e3ac1b0 2024-08-31T07:22:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2e4468740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d81af620 con 0x55570e3d5070 2024-08-31T07:22:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2e4468740 10 client.0 _ll_register_callbacks cb 0x7fc2c0003b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2d6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e78eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2d6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e78eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59474/0 (socket says 172.21.15.32:59474) 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2d7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 0x55570e7898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2d6a00640 1 -- 172.21.15.32:0/2948592845 learned_addr learned my addr 172.21.15.32:0/2948592845 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.816+0000 7fc2ce000640 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 0x55570e789270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2d6a00640 1 -- 172.21.15.32:0/2948592845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 msgr2=0x55570e7898a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2d6a00640 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 0x55570e7898a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2d6a00640 1 -- 172.21.15.32:0/2948592845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 msgr2=0x55570e789270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2d6a00640 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 0x55570e789270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2d6a00640 1 -- 172.21.15.32:0/2948592845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570e78a030 con 0x55570e3ac1b0 2024-08-31T07:22:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2d7400640 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 0x55570e7898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2d6a00640 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e78eb90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8031440 tx=0x7fc2c802a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6db0d99734b4a38 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2c4c00640 1 -- 172.21.15.32:0/2948592845 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8035040 con 0x55570e3ac1b0 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2c4c00640 1 -- 172.21.15.32:0/2948592845 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fc2c8004240 con 0x55570e3ac1b0 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2e4468740 1 -- 172.21.15.32:0/2948592845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c0021c60 con 0x55570e3ac1b0 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2c4c00640 1 -- 172.21.15.32:0/2948592845 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc2c8069050 con 0x55570e3ac1b0 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2e4468740 10 client.27536 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.817+0000 7fc2e4468740 1 -- 172.21.15.32:0/2948592845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55570e804110 con 0x55570e3ac1b0 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.818+0000 7fc2c4c00640 1 -- 172.21.15.32:0/2948592845 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c0021c60 con 0x55570e3ac1b0 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.818+0000 7fc2c4c00640 1 -- 172.21.15.32:0/2948592845 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fc2c8003860 con 0x55570e3ac1b0 2024-08-31T07:22:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.818+0000 7fc2e4468740 10 client.27536 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.818+0000 7fc2e4468740 10 client.27536 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.818+0000 7fc2e4468740 20 client.27536 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.818+0000 7fc2e4468740 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55570e368da0 0x55570e806570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.818+0000 7fc2e4468740 4 client.27536 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.818+0000 7fc2e4468740 1 -- 172.21.15.32:0/2948592845 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55570e6ef1a0 con 0x55570e368da0 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.818+0000 7fc2ce000640 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55570e368da0 0x55570e806570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2ce000640 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55570e368da0 0x55570e806570 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=630cd7a334106bf6 server_cookie=a77d40ec827082db in_seq=0 out_seq=0 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2c4c00640 10 client.27536 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2c4c00640 1 -- 172.21.15.32:0/2948592845 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55570e6ef1a0 con 0x55570e368da0 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2c4c00640 10 client.27536 handle_command_reply: tid=0 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2e4468740 1 client.27536 shutdown 2024-08-31T07:22:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2e4468740 1 -- 172.21.15.32:0/2948592845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 msgr2=0x55570e78eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2e4468740 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e78eb90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8031440 tx=0x7fc2c802a400 comp rx=0 tx=0).stop 2024-08-31T07:22:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2e4468740 1 -- 172.21.15.32:0/2948592845 shutdown_connections 2024-08-31T07:22:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2e4468740 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55570e368da0 0x55570e806570 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2e4468740 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55570e3d5070 0x55570e7898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2e4468740 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55570e3b4390 0x55570e789270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2c4c00640 0 client.27536 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:22:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2e4468740 1 --2- 172.21.15.32:0/2948592845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55570e3ac1b0 0x55570e78eb90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2e4468740 1 -- 172.21.15.32:0/2948592845 >> 172.21.15.32:0/2948592845 conn(0x55570e3c9460 msgr2=0x55570e778030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.819+0000 7fc2e4468740 1 -- 172.21.15.32:0/2948592845 shutdown_connections 2024-08-31T07:22:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2e4468740 1 -- 172.21.15.32:0/2948592845 wait complete. 2024-08-31T07:22:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2e4468740 20 client.27536 trim_cache size 0 max 16384 2024-08-31T07:22:35.826 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:22:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2dd800640 1 -- 172.21.15.32:0/3572395569 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc2bc0b4d20 msgr2=0x7fc2bc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2dd800640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc2bc0b4d20 0x7fc2bc0b7230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc2d806ae40 tx=0x7fc2c0004730 comp rx=0 tx=0).stop 2024-08-31T07:22:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2dd800640 1 -- 172.21.15.32:0/3572395569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2d810fde0 msgr2=0x7fc2d81a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2dd800640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2d810fde0 0x7fc2d81a1c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000e940 tx=0x7fc2d000ee00 comp rx=0 tx=0).stop 2024-08-31T07:22:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2dd800640 1 -- 172.21.15.32:0/3572395569 shutdown_connections 2024-08-31T07:22:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2dd800640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc2bc0b4d20 0x7fc2bc0b7230 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2dd800640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2d810fde0 0x7fc2d81a1c70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2dd800640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d8104440 0x7fc2d819c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2dd800640 1 --2- 172.21.15.32:0/3572395569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2d806be80 0x7fc2d819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.820+0000 7fc2dd800640 1 -- 172.21.15.32:0/3572395569 >> 172.21.15.32:0/3572395569 conn(0x7fc2d8079740 msgr2=0x7fc2d8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.821+0000 7fc2dd800640 1 -- 172.21.15.32:0/3572395569 shutdown_connections 2024-08-31T07:22:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:35.821+0000 7fc2dd800640 1 -- 172.21.15.32:0/3572395569 wait complete. 2024-08-31T07:22:36.483 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.5 2024-08-31T07:22:36.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:22:36.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:22:36.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 320938 3.04 MB/sec execute 385 sec latency 1213.278 ms 2024-08-31T07:22:36.786 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:36 smithi032 ceph-mon[17252]: pgmap v671: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-08-31T07:22:36.878 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:36 smithi073 ceph-mon[21995]: pgmap v671: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-08-31T07:22:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:36 smithi084 ceph-mon[22387]: pgmap v671: 97 pgs: 97 active+clean; 236 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-08-31T07:22:37.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.528+0000 7f5ce9400640 1 -- 172.21.15.32:0/1650144336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce4105b20 msgr2=0x7f5ce4105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:37.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.528+0000 7f5ce9400640 1 --2- 172.21.15.32:0/1650144336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce4105b20 0x7f5ce4105f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009900 tx=0x7f5cd402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.528+0000 7f5ce9400640 1 -- 172.21.15.32:0/1650144336 shutdown_connections 2024-08-31T07:22:37.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.528+0000 7f5ce9400640 1 --2- 172.21.15.32:0/1650144336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce410bdb0 0x7f5ce41125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.528+0000 7f5ce9400640 1 --2- 172.21.15.32:0/1650144336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ce410b460 0x7f5ce410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.528+0000 7f5ce9400640 1 --2- 172.21.15.32:0/1650144336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce4105b20 0x7f5ce4105f30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.528+0000 7f5ce9400640 1 -- 172.21.15.32:0/1650144336 >> 172.21.15.32:0/1650144336 conn(0x7f5ce41015b0 msgr2=0x7f5ce4103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:37.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.529+0000 7f5ce9400640 1 -- 172.21.15.32:0/1650144336 shutdown_connections 2024-08-31T07:22:37.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.529+0000 7f5ce9400640 1 -- 172.21.15.32:0/1650144336 wait complete. 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.530+0000 7f5ce9400640 1 Processor -- start 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.530+0000 7f5ce9400640 1 -- start start 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.530+0000 7f5ce9400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4105b20 0x7f5ce419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.530+0000 7f5ce9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ce410b460 0x7f5ce419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce410bdb0 0x7f5ce41a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce9400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4115a70 con 0x7f5ce410bdb0 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce9400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce41158f0 con 0x7f5ce410b460 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce9400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4115bf0 con 0x7f5ce4105b20 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ce410b460 0x7f5ce419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ce410b460 0x7f5ce419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55734/0 (socket says 172.21.15.32:55734) 2024-08-31T07:22:37.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce1e00640 1 -- 172.21.15.32:0/3016727490 learned_addr learned my addr 172.21.15.32:0/3016727490 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:37.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce2800640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4105b20 0x7f5ce419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:37.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce3200640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce410bdb0 0x7f5ce41a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:37.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce1e00640 1 -- 172.21.15.32:0/3016727490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4105b20 msgr2=0x7f5ce419c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:37.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce1e00640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4105b20 0x7f5ce419c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce1e00640 1 -- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce410bdb0 msgr2=0x7f5ce41a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:37.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce1e00640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce410bdb0 0x7f5ce41a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.531+0000 7f5ce1e00640 1 -- 172.21.15.32:0/3016727490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce41a2350 con 0x7f5ce410b460 2024-08-31T07:22:37.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.532+0000 7f5ce3200640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce410bdb0 0x7f5ce41a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:22:37.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.532+0000 7f5ce1e00640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ce410b460 0x7f5ce419c980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e9f0 tx=0x7f5cd000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:37.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.532+0000 7f5cdb400640 1 -- 172.21.15.32:0/3016727490 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd000ccc0 con 0x7f5ce410b460 2024-08-31T07:22:37.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.532+0000 7f5cdb400640 1 -- 172.21.15.32:0/3016727490 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cd00108e0 con 0x7f5ce410b460 2024-08-31T07:22:37.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.532+0000 7f5ce9400640 1 -- 172.21.15.32:0/3016727490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce41a2640 con 0x7f5ce410b460 2024-08-31T07:22:37.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.533+0000 7f5ce9400640 1 -- 172.21.15.32:0/3016727490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce41a8da0 con 0x7f5ce410b460 2024-08-31T07:22:37.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.533+0000 7f5cdb400640 1 -- 172.21.15.32:0/3016727490 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0018860 con 0x7f5ce410b460 2024-08-31T07:22:37.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.534+0000 7f5ce9400640 1 -- 172.21.15.32:0/3016727490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc40052c0 con 0x7f5ce410b460 2024-08-31T07:22:37.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.537+0000 7f5cdb400640 1 -- 172.21.15.32:0/3016727490 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5cd0010430 con 0x7f5ce410b460 2024-08-31T07:22:37.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.539+0000 7f5cdb400640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5cc80b4ca0 0x7f5cc80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:37.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.539+0000 7f5cdb400640 1 -- 172.21.15.32:0/3016727490 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f5cd00e56a0 con 0x7f5ce410b460 2024-08-31T07:22:37.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.539+0000 7f5ce2800640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5cc80b4ca0 0x7f5cc80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:37.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.540+0000 7f5ce2800640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5cc80b4ca0 0x7f5cc80b71b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4031440 tx=0x7f5cd4009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:37.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.543+0000 7f5cdb400640 1 -- 172.21.15.32:0/3016727490 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5cd00ac480 con 0x7f5ce410b460 2024-08-31T07:22:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:37.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 320938 3.03 MB/sec execute 386 sec latency 2213.331 ms 2024-08-31T07:22:37.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.781+0000 7f5ce9400640 1 -- 172.21.15.32:0/3016727490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cc4005880 con 0x7f5ce410b460 2024-08-31T07:22:37.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.782+0000 7f5cdb400640 1 -- 172.21.15.32:0/3016727490 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11968 (secure 0 0 0) 0x7f5cd00abd80 con 0x7f5ce410b460 2024-08-31T07:22:37.783 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:22:37.783 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"btime":"2024-08-31T07:22:06:095426+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:22:05.949269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:22:37.784 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 40 2024-08-31T07:22:37.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.784+0000 7f5ce9400640 1 -- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5cc80b4ca0 msgr2=0x7f5cc80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:37.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.784+0000 7f5ce9400640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5cc80b4ca0 0x7f5cc80b71b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4031440 tx=0x7f5cd4009560 comp rx=0 tx=0).stop 2024-08-31T07:22:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.784+0000 7f5ce9400640 1 -- 172.21.15.32:0/3016727490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ce410b460 msgr2=0x7f5ce419c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.784+0000 7f5ce9400640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ce410b460 0x7f5ce419c980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e9f0 tx=0x7f5cd000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.785+0000 7f5ce9400640 1 -- 172.21.15.32:0/3016727490 shutdown_connections 2024-08-31T07:22:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.785+0000 7f5ce9400640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5cc80b4ca0 0x7f5cc80b71b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.785+0000 7f5ce9400640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce410bdb0 0x7f5ce41a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.785+0000 7f5ce9400640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ce410b460 0x7f5ce419c980 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.785+0000 7f5ce9400640 1 --2- 172.21.15.32:0/3016727490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ce4105b20 0x7f5ce419c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.785+0000 7f5ce9400640 1 -- 172.21.15.32:0/3016727490 >> 172.21.15.32:0/3016727490 conn(0x7f5ce41015b0 msgr2=0x7f5ce41039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.785+0000 7f5ce9400640 1 -- 172.21.15.32:0/3016727490 shutdown_connections 2024-08-31T07:22:37.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:37.785+0000 7f5ce9400640 1 -- 172.21.15.32:0/3016727490 wait complete. 2024-08-31T07:22:38.746 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 135.5 --await-for 45.0' 2024-08-31T07:22:38.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 135.5 --await-for 45.0 2024-08-31T07:22:38.757 INFO:tasks.workunit.client.0.smithi032.stdout: 1 321695 3.03 MB/sec execute 387 sec latency 2847.481 ms 2024-08-31T07:22:38.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:38 smithi032 ceph-mon[17252]: pgmap v672: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-08-31T07:22:38.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:38 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3016727490' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:22:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:38 smithi073 ceph-mon[21995]: pgmap v672: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-08-31T07:22:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:38 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3016727490' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:22:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:38 smithi084 ceph-mon[22387]: pgmap v672: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-08-31T07:22:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:38 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3016727490' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:22:39.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.060+0000 7fb1ade00640 1 -- 172.21.15.32:0/63552639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1a81099e0 msgr2=0x7fb1a8110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:39.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.060+0000 7fb1ade00640 1 --2- 172.21.15.32:0/63552639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1a81099e0 0x7fb1a8110210 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009ca0 tx=0x7fb1a002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.060+0000 7fb1ade00640 1 -- 172.21.15.32:0/63552639 shutdown_connections 2024-08-31T07:22:39.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.060+0000 7fb1ade00640 1 --2- 172.21.15.32:0/63552639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1a81099e0 0x7fb1a8110210 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.060+0000 7fb1ade00640 1 --2- 172.21.15.32:0/63552639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1a8109090 0x7fb1a81094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.060+0000 7fb1ade00640 1 --2- 172.21.15.32:0/63552639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1a8103750 0x7fb1a8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.060+0000 7fb1ade00640 1 -- 172.21.15.32:0/63552639 >> 172.21.15.32:0/63552639 conn(0x7fb1a80ff1e0 msgr2=0x7fb1a8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:39.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.060+0000 7fb1ade00640 1 -- 172.21.15.32:0/63552639 shutdown_connections 2024-08-31T07:22:39.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.060+0000 7fb1ade00640 1 -- 172.21.15.32:0/63552639 wait complete. 2024-08-31T07:22:39.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.061+0000 7fb1ade00640 1 Processor -- start 2024-08-31T07:22:39.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1ade00640 1 -- start start 2024-08-31T07:22:39.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1ade00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1a8103750 0x7fb1a819c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1ade00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1a8109090 0x7fb1a819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1ade00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1a81099e0 0x7fb1a81a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1ade00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a8113690 con 0x7fb1a8103750 2024-08-31T07:22:39.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1ade00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a8113510 con 0x7fb1a8109090 2024-08-31T07:22:39.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1ade00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a8113810 con 0x7fb1a81099e0 2024-08-31T07:22:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1ad400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1a81099e0 0x7fb1a81a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1ad400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1a81099e0 0x7fb1a81a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:53938/0 (socket says 172.21.15.32:53938) 2024-08-31T07:22:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1a8109090 0x7fb1a819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.062+0000 7fb1ad400640 1 -- 172.21.15.32:0/955781510 learned_addr learned my addr 172.21.15.32:0/955781510 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.063+0000 7fb1aca00640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1a8103750 0x7fb1a819c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.063+0000 7fb1ad400640 1 -- 172.21.15.32:0/955781510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1a8109090 msgr2=0x7fb1a819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.063+0000 7fb1ad400640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1a8109090 0x7fb1a819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.063+0000 7fb1ad400640 1 -- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1a8103750 msgr2=0x7fb1a819c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.063+0000 7fb1ad400640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1a8103750 0x7fb1a819c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.063+0000 7fb1ad400640 1 -- 172.21.15.32:0/955781510 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a81a2170 con 0x7fb1a81099e0 2024-08-31T07:22:39.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.063+0000 7fb1aca00640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1a8103750 0x7fb1a819c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:22:39.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.063+0000 7fb1ad400640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1a81099e0 0x7fb1a81a1a40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0005cc0 tx=0x7fb1a0031700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:39.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.064+0000 7fb1a5600640 1 -- 172.21.15.32:0/955781510 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0003b70 con 0x7fb1a81099e0 2024-08-31T07:22:39.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.064+0000 7fb1a5600640 1 -- 172.21.15.32:0/955781510 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1a00318f0 con 0x7fb1a81099e0 2024-08-31T07:22:39.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.064+0000 7fb1a5600640 1 -- 172.21.15.32:0/955781510 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a002f950 con 0x7fb1a81099e0 2024-08-31T07:22:39.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.064+0000 7fb1ade00640 1 -- 172.21.15.32:0/955781510 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a81a2400 con 0x7fb1a81099e0 2024-08-31T07:22:39.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.064+0000 7fb1ade00640 1 -- 172.21.15.32:0/955781510 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a81a8bc0 con 0x7fb1a81099e0 2024-08-31T07:22:39.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.068+0000 7fb1a5600640 1 -- 172.21.15.32:0/955781510 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb1a002fb30 con 0x7fb1a81099e0 2024-08-31T07:22:39.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.069+0000 7fb1b6221740 1 Processor -- start 2024-08-31T07:22:39.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.069+0000 7fb1b6221740 1 -- start start 2024-08-31T07:22:39.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.069+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb849b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.069+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb47b1b0 0x557fdb84a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.069+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb483390 0x557fdb84a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.069+0000 7fb1b6221740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557fdb7c07b0 con 0x557fdb47b1b0 2024-08-31T07:22:39.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.069+0000 7fb1b6221740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557fdb7afd40 con 0x557fdb483390 2024-08-31T07:22:39.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.069+0000 7fb1b6221740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557fdb7b3cf0 con 0x557fdb4a4070 2024-08-31T07:22:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a5600640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1900b4d20 0x7fb1900b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb849b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb849b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:53946/0 (socket says 172.21.15.32:53946) 2024-08-31T07:22:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a7e00640 1 -- 172.21.15.32:0/554449691 learned_addr learned my addr 172.21.15.32:0/554449691 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/554449691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb483390 0x557fdb84a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1aca00640 1 --2- 172.21.15.32:0/554449691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb47b1b0 0x557fdb84a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1ad400640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1900b4d20 0x7fb1900b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a5600640 1 -- 172.21.15.32:0/955781510 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fb1a0106730 con 0x7fb1a81099e0 2024-08-31T07:22:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a7e00640 1 -- 172.21.15.32:0/554449691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb483390 msgr2=0x557fdb84a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/554449691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb483390 0x557fdb84a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a7e00640 1 -- 172.21.15.32:0/554449691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb47b1b0 msgr2=0x557fdb84a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/554449691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb47b1b0 0x557fdb84a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.070+0000 7fb1a7e00640 1 -- 172.21.15.32:0/554449691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fdb50f650 con 0x557fdb4a4070 2024-08-31T07:22:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.071+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/554449691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb849b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb194011710 tx=0x7fb194005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a6eee6d081988f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:39.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.071+0000 7fb199c00640 1 -- 172.21.15.32:0/554449691 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194005020 con 0x557fdb4a4070 2024-08-31T07:22:39.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.071+0000 7fb199c00640 1 -- 172.21.15.32:0/554449691 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb19400f840 con 0x557fdb4a4070 2024-08-31T07:22:39.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.071+0000 7fb199c00640 1 -- 172.21.15.32:0/554449691 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194004a00 con 0x557fdb4a4070 2024-08-31T07:22:39.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.072+0000 7fb1b6221740 1 -- 172.21.15.32:0/554449691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 msgr2=0x557fdb849b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:39.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.072+0000 7fb1ad400640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1900b4d20 0x7fb1900b7230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00cb100 tx=0x7fb1a002a8c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:39.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.072+0000 7fb1b6221740 1 --2- 172.21.15.32:0/554449691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb849b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb194011710 tx=0x7fb194005850 comp rx=0 tx=0).stop 2024-08-31T07:22:39.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.072+0000 7fb1b6221740 1 -- 172.21.15.32:0/554449691 shutdown_connections 2024-08-31T07:22:39.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.072+0000 7fb1b6221740 1 --2- 172.21.15.32:0/554449691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb483390 0x557fdb84a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.072+0000 7fb1b6221740 1 --2- 172.21.15.32:0/554449691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb47b1b0 0x557fdb84a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.072+0000 7fb1b6221740 1 --2- 172.21.15.32:0/554449691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb849b60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.072+0000 7fb1b6221740 1 -- 172.21.15.32:0/554449691 >> 172.21.15.32:0/554449691 conn(0x557fdb498460 msgr2=0x557fdb847a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:39.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.072+0000 7fb1b6221740 1 -- 172.21.15.32:0/554449691 shutdown_connections 2024-08-31T07:22:39.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.072+0000 7fb1b6221740 1 -- 172.21.15.32:0/554449691 wait complete. 2024-08-31T07:22:39.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.073+0000 7fb1b6221740 1 -- start start 2024-08-31T07:22:39.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.073+0000 7fb1b6221740 1 Processor -- start 2024-08-31T07:22:39.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb855760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 0x557fdb855ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb4a4070 0x557fdb8616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1b6221740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557fdb612700 con 0x557fdb4a4070 2024-08-31T07:22:39.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1b6221740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557fdb736cf0 con 0x557fdb47b1b0 2024-08-31T07:22:39.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1b6221740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557fdb858340 con 0x557fdb483390 2024-08-31T07:22:39.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1b6221740 10 client.0 _ll_register_callbacks cb 0x557fdb75c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:39.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1aca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 0x557fdb855ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb855760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1aca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 0x557fdb855ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:53956/0 (socket says 172.21.15.32:53956) 2024-08-31T07:22:39.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.074+0000 7fb1aca00640 1 -- 172.21.15.32:0/1927969963 learned_addr learned my addr 172.21.15.32:0/1927969963 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:39.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.075+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb4a4070 0x557fdb8616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.075+0000 7fb1aca00640 1 -- 172.21.15.32:0/1927969963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 msgr2=0x557fdb855760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:39.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.075+0000 7fb1aca00640 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb855760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.075+0000 7fb1aca00640 1 -- 172.21.15.32:0/1927969963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb4a4070 msgr2=0x557fdb8616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.075+0000 7fb1aca00640 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb4a4070 0x557fdb8616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.075+0000 7fb1aca00640 1 -- 172.21.15.32:0/1927969963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fdb7778b0 con 0x557fdb483390 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.075+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb855760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.075+0000 7fb1aca00640 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 0x557fdb855ca0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009d70 tx=0x7fb19c0039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3392ea6918e621 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.075+0000 7fb18e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.075+0000 7fb18e000640 1 -- 172.21.15.32:0/1927969963 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c013930 con 0x557fdb483390 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb1b6221740 1 -- 172.21.15.32:0/1927969963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb19400f840 con 0x557fdb483390 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb18e000640 1 -- 172.21.15.32:0/1927969963 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fb19c013ad0 con 0x557fdb483390 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb18e000640 1 -- 172.21.15.32:0/1927969963 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb19c013cd0 con 0x557fdb483390 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb1b6221740 10 client.27556 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb1b6221740 1 -- 172.21.15.32:0/1927969963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557fdb8644c0 con 0x557fdb483390 2024-08-31T07:22:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb18e000640 1 -- 172.21.15.32:0/1927969963 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb19400f840 con 0x557fdb483390 2024-08-31T07:22:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb18e000640 1 -- 172.21.15.32:0/1927969963 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb19c01a070 con 0x557fdb483390 2024-08-31T07:22:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb1b6221740 10 client.27556 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb1b6221740 10 client.27556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:22:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb1b6221740 20 client.27556 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.076+0000 7fb1b6221740 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557fdb4629f0 0x557fdb867a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.077+0000 7fb1b6221740 4 client.27556 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:22:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.077+0000 7fb1b6221740 1 -- 172.21.15.32:0/1927969963 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fdb7c2700 con 0x557fdb4629f0 2024-08-31T07:22:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.077+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557fdb4629f0 0x557fdb867a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.077+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557fdb4629f0 0x557fdb867a20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d98833f9941171b server_cookie=6e8aef268c8b8e43 in_seq=0 out_seq=0 2024-08-31T07:22:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.077+0000 7fb18e000640 10 client.27556 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:22:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.078+0000 7fb18e000640 1 -- 172.21.15.32:0/1927969963 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557fdb7c2700 con 0x557fdb4629f0 2024-08-31T07:22:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.078+0000 7fb18e000640 10 client.27556 handle_command_reply: tid=0 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.078+0000 7fb1b6221740 1 client.27556 shutdown 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 1 -- 172.21.15.32:0/1927969963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 msgr2=0x557fdb855ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 0x557fdb855ca0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009d70 tx=0x7fb19c0039c0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 1 -- 172.21.15.32:0/1927969963 shutdown_connections 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557fdb4629f0 0x557fdb867a20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb4a4070 0x557fdb8616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb18e000640 0 client.27556 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 0x557fdb855ca0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 1 --2- 172.21.15.32:0/1927969963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb855760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 1 -- 172.21.15.32:0/1927969963 >> 172.21.15.32:0/1927969963 conn(0x557fdb498460 msgr2=0x557fdb84ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 1 -- 172.21.15.32:0/1927969963 shutdown_connections 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 1 -- 172.21.15.32:0/1927969963 wait complete. 2024-08-31T07:22:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.079+0000 7fb1b6221740 20 client.27556 trim_cache size 0 max 16384 2024-08-31T07:22:39.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.095+0000 7fb1b6221740 1 Processor -- start 2024-08-31T07:22:39.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.096+0000 7fb1b6221740 1 -- start start 2024-08-31T07:22:39.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.096+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb47b1b0 0x557fdb850d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.096+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb483390 0x557fdb84b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.096+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb8615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.096+0000 7fb1b6221740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb18800e6b0 con 0x557fdb47b1b0 2024-08-31T07:22:39.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.096+0000 7fb1b6221740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1880102e0 con 0x557fdb483390 2024-08-31T07:22:39.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.096+0000 7fb1b6221740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb18800ed80 con 0x557fdb4a4070 2024-08-31T07:22:39.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.096+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb47b1b0 0x557fdb850d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.097+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb8615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.097+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb8615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:53968/0 (socket says 172.21.15.32:53968) 2024-08-31T07:22:39.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.097+0000 7fb1a7e00640 1 -- 172.21.15.32:0/3786663818 learned_addr learned my addr 172.21.15.32:0/3786663818 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:39.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.097+0000 7fb1aca00640 1 --2- 172.21.15.32:0/3786663818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb483390 0x557fdb84b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.097+0000 7fb1a7e00640 1 -- 172.21.15.32:0/3786663818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb483390 msgr2=0x557fdb84b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:39.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.097+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/3786663818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb483390 0x557fdb84b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.097+0000 7fb1a7e00640 1 -- 172.21.15.32:0/3786663818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb47b1b0 msgr2=0x557fdb850d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:39.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.097+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/3786663818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb47b1b0 0x557fdb850d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.097+0000 7fb1a7e00640 1 -- 172.21.15.32:0/3786663818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fdb853a60 con 0x557fdb4a4070 2024-08-31T07:22:39.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.097+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/3786663818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb8615c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb1940057c0 tx=0x7fb194005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=423dc28cc0696ca8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:39.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.098+0000 7fb199200640 1 -- 172.21.15.32:0/3786663818 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19400fb10 con 0x557fdb4a4070 2024-08-31T07:22:39.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.098+0000 7fb199200640 1 -- 172.21.15.32:0/3786663818 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb19400f840 con 0x557fdb4a4070 2024-08-31T07:22:39.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.098+0000 7fb199200640 1 -- 172.21.15.32:0/3786663818 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194003bf0 con 0x557fdb4a4070 2024-08-31T07:22:39.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.099+0000 7fb1b6221740 1 -- 172.21.15.32:0/3786663818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 msgr2=0x557fdb8615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:39.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.099+0000 7fb1b6221740 1 --2- 172.21.15.32:0/3786663818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb8615c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb1940057c0 tx=0x7fb194005850 comp rx=0 tx=0).stop 2024-08-31T07:22:39.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.099+0000 7fb1b6221740 1 -- 172.21.15.32:0/3786663818 shutdown_connections 2024-08-31T07:22:39.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.099+0000 7fb1b6221740 1 --2- 172.21.15.32:0/3786663818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb4a4070 0x557fdb8615c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.099+0000 7fb1b6221740 1 --2- 172.21.15.32:0/3786663818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb483390 0x557fdb84b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.099+0000 7fb1b6221740 1 --2- 172.21.15.32:0/3786663818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb47b1b0 0x557fdb850d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.099+0000 7fb1b6221740 1 -- 172.21.15.32:0/3786663818 >> 172.21.15.32:0/3786663818 conn(0x557fdb498460 msgr2=0x557fdb84ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:39.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.099+0000 7fb1b6221740 1 -- 172.21.15.32:0/3786663818 shutdown_connections 2024-08-31T07:22:39.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.099+0000 7fb1b6221740 1 -- 172.21.15.32:0/3786663818 wait complete. 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.100+0000 7fb1b6221740 1 -- start start 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.100+0000 7fb1b6221740 1 Processor -- start 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb85a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 0x557fdb85a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1b6221740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb4a4070 0x557fdb8cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1b6221740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb18800fc10 con 0x557fdb4a4070 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1b6221740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb18800f510 con 0x557fdb47b1b0 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1b6221740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb18800c6b0 con 0x557fdb483390 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1b6221740 10 client.0 _ll_register_callbacks cb 0x7fb19c009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb85a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb85a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55790/0 (socket says 172.21.15.32:55790) 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1a7e00640 1 -- 172.21.15.32:0/2050691001 learned_addr learned my addr 172.21.15.32:0/2050691001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1aca00640 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 0x557fdb85a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.101+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb4a4070 0x557fdb8cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.102+0000 7fb1a7e00640 1 -- 172.21.15.32:0/2050691001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 msgr2=0x557fdb85a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:39.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.102+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 0x557fdb85a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.102+0000 7fb1a7e00640 1 -- 172.21.15.32:0/2050691001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb4a4070 msgr2=0x557fdb8cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.102+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb4a4070 0x557fdb8cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.102+0000 7fb1a7e00640 1 -- 172.21.15.32:0/2050691001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fdb8cbbf0 con 0x557fdb47b1b0 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.102+0000 7fb1a7e00640 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb85a1c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb194010010 tx=0x7fb1940098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc567006ac5555f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.102+0000 7fb18e000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.102+0000 7fb18e000640 1 -- 172.21.15.32:0/2050691001 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1940050c0 con 0x557fdb47b1b0 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.103+0000 7fb18e000640 1 -- 172.21.15.32:0/2050691001 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fb194005260 con 0x557fdb47b1b0 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.103+0000 7fb1b6221740 1 -- 172.21.15.32:0/2050691001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb19400f840 con 0x557fdb47b1b0 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.103+0000 7fb18e000640 1 -- 172.21.15.32:0/2050691001 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb19403e020 con 0x557fdb47b1b0 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.103+0000 7fb1b6221740 10 client.27560 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.103+0000 7fb1b6221740 1 -- 172.21.15.32:0/2050691001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557fdb8d2fe0 con 0x557fdb47b1b0 2024-08-31T07:22:39.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.104+0000 7fb18e000640 1 -- 172.21.15.32:0/2050691001 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb19400f840 con 0x557fdb47b1b0 2024-08-31T07:22:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.105+0000 7fb18e000640 1 -- 172.21.15.32:0/2050691001 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb19401d030 con 0x557fdb47b1b0 2024-08-31T07:22:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.105+0000 7fb1b6221740 10 client.27560 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.105+0000 7fb1b6221740 10 client.27560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:22:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.105+0000 7fb1b6221740 20 client.27560 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.105+0000 7fb1b6221740 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557fdb437da0 0x557fdb8d5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.105+0000 7fb1b6221740 4 client.27560 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:22:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.105+0000 7fb1b6221740 1 -- 172.21.15.32:0/2050691001 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557fdb7be1a0 con 0x557fdb437da0 2024-08-31T07:22:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.105+0000 7fb1aca00640 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557fdb437da0 0x557fdb8d5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:39.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb1aca00640 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557fdb437da0 0x557fdb8d5560 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7d94496b2e9905d server_cookie=764a19051c3a8c06 in_seq=0 out_seq=0 2024-08-31T07:22:39.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb18e000640 10 client.27560 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:22:39.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb18e000640 1 -- 172.21.15.32:0/2050691001 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557fdb7be1a0 con 0x557fdb437da0 2024-08-31T07:22:39.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb18e000640 10 client.27560 handle_command_reply: tid=0 2024-08-31T07:22:39.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb1b6221740 1 client.27560 shutdown 2024-08-31T07:22:39.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb1b6221740 1 -- 172.21.15.32:0/2050691001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 msgr2=0x557fdb85a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:39.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb1b6221740 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb85a1c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb194010010 tx=0x7fb1940098d0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb1b6221740 1 -- 172.21.15.32:0/2050691001 shutdown_connections 2024-08-31T07:22:39.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb1b6221740 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557fdb437da0 0x557fdb8d5560 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb1b6221740 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fdb4a4070 0x557fdb8cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb1b6221740 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557fdb483390 0x557fdb85a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb1b6221740 1 --2- 172.21.15.32:0/2050691001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fdb47b1b0 0x557fdb85a1c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb18e000640 0 client.27560 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:22:39.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.106+0000 7fb1b6221740 1 -- 172.21.15.32:0/2050691001 >> 172.21.15.32:0/2050691001 conn(0x557fdb498460 msgr2=0x557fdb857370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:39.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.107+0000 7fb1b6221740 1 -- 172.21.15.32:0/2050691001 shutdown_connections 2024-08-31T07:22:39.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.107+0000 7fb1b6221740 1 -- 172.21.15.32:0/2050691001 wait complete. 2024-08-31T07:22:39.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.107+0000 7fb1b6221740 20 client.27560 trim_cache size 0 max 16384 2024-08-31T07:22:39.109 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:22:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 -- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1900b4d20 msgr2=0x7fb1900b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1900b4d20 0x7fb1900b7230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00cb100 tx=0x7fb1a002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 -- 172.21.15.32:0/955781510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1a81099e0 msgr2=0x7fb1a81a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1a81099e0 0x7fb1a81a1a40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0005cc0 tx=0x7fb1a0031700 comp rx=0 tx=0).stop 2024-08-31T07:22:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 -- 172.21.15.32:0/955781510 shutdown_connections 2024-08-31T07:22:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1900b4d20 0x7fb1900b7230 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1a81099e0 0x7fb1a81a1a40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1a8109090 0x7fb1a819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 --2- 172.21.15.32:0/955781510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1a8103750 0x7fb1a819c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:39.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 -- 172.21.15.32:0/955781510 >> 172.21.15.32:0/955781510 conn(0x7fb1a80ff1e0 msgr2=0x7fb1a81076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:39.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 -- 172.21.15.32:0/955781510 shutdown_connections 2024-08-31T07:22:39.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:39.108+0000 7fb1ade00640 1 -- 172.21.15.32:0/955781510 wait complete. 2024-08-31T07:22:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:39.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323132 3.05 MB/sec execute 388 sec latency 24.075 ms 2024-08-31T07:22:40.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:22:40.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323217 3.05 MB/sec execute 389 sec latency 513.682 ms 2024-08-31T07:22:40.797 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:40.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:40 smithi032 ceph-mon[17252]: pgmap v673: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-08-31T07:22:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.817+0000 7f05a3800640 1 -- 172.21.15.32:0/3464447209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c103920 msgr2=0x7f059c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.817+0000 7f05a3800640 1 --2- 172.21.15.32:0/3464447209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c103920 0x7f059c103d30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0590009900 tx=0x7f059002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.818+0000 7f05a3800640 1 -- 172.21.15.32:0/3464447209 shutdown_connections 2024-08-31T07:22:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.818+0000 7f05a3800640 1 --2- 172.21.15.32:0/3464447209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f059c109bb0 0x7f059c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.818+0000 7f05a3800640 1 --2- 172.21.15.32:0/3464447209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c109260 0x7f059c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.818+0000 7f05a3800640 1 --2- 172.21.15.32:0/3464447209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c103920 0x7f059c103d30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.818+0000 7f05a3800640 1 -- 172.21.15.32:0/3464447209 >> 172.21.15.32:0/3464447209 conn(0x7f059c0ff370 msgr2=0x7f059c1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.818+0000 7f05a3800640 1 -- 172.21.15.32:0/3464447209 shutdown_connections 2024-08-31T07:22:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.818+0000 7f05a3800640 1 -- 172.21.15.32:0/3464447209 wait complete. 2024-08-31T07:22:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.819+0000 7f05a3800640 1 Processor -- start 2024-08-31T07:22:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.819+0000 7f05a3800640 1 -- start start 2024-08-31T07:22:40.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f05a3800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f059c103920 0x7f059c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f05a3800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c109260 0x7f059c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f05a3800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c109bb0 0x7f059c1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f05a3800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f059c1138c0 con 0x7f059c103920 2024-08-31T07:22:40.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f05a3800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f059c113740 con 0x7f059c109260 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f05a3800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f059c113a40 con 0x7f059c109bb0 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f05a0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f059c103920 0x7f059c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f059be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c109260 0x7f059c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f05a0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f059c103920 0x7f059c19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59680/0 (socket says 172.21.15.32:59680) 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f05a0e00640 1 -- 172.21.15.32:0/2663929233 learned_addr learned my addr 172.21.15.32:0/2663929233 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f05a1800640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c109bb0 0x7f059c1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.820+0000 7f059be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c109260 0x7f059c19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55814/0 (socket says 172.21.15.32:55814) 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.821+0000 7f05a0e00640 1 -- 172.21.15.32:0/2663929233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c109bb0 msgr2=0x7f059c1a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.821+0000 7f05a0e00640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c109bb0 0x7f059c1a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.821+0000 7f05a0e00640 1 -- 172.21.15.32:0/2663929233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c109260 msgr2=0x7f059c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.821+0000 7f05a0e00640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c109260 0x7f059c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.821+0000 7f05a0e00640 1 -- 172.21.15.32:0/2663929233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c1a2400 con 0x7f059c103920 2024-08-31T07:22:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.821+0000 7f05a0e00640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f059c103920 0x7f059c19c490 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0590005ab0 tx=0x7f0590003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:40.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.821+0000 7f0599600640 1 -- 172.21.15.32:0/2663929233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059003f070 con 0x7f059c103920 2024-08-31T07:22:40.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.821+0000 7f05a3800640 1 -- 172.21.15.32:0/2663929233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059c1a2690 con 0x7f059c103920 2024-08-31T07:22:40.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.821+0000 7f05a3800640 1 -- 172.21.15.32:0/2663929233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059c1a8e50 con 0x7f059c103920 2024-08-31T07:22:40.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.821+0000 7f0599600640 1 -- 172.21.15.32:0/2663929233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0590031b60 con 0x7f059c103920 2024-08-31T07:22:40.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.822+0000 7f0599600640 1 -- 172.21.15.32:0/2663929233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590035030 con 0x7f059c103920 2024-08-31T07:22:40.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.825+0000 7f05aa43f740 1 Processor -- start 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.825+0000 7f05aa43f740 1 -- start start 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.825+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938c8840 0x561993c6f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.825+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.825+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938a7b60 0x561993c6fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.825+0000 7f05aa43f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561993be4530 con 0x5619938a7b60 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.825+0000 7f05aa43f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561993bd3ac0 con 0x56199389f980 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.825+0000 7f05aa43f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561993bd7a70 con 0x5619938c8840 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.826+0000 7f0599600640 1 -- 172.21.15.32:0/2663929233 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0590042070 con 0x7f059c103920 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.826+0000 7f05a0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.826+0000 7f05a0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55820/0 (socket says 172.21.15.32:55820) 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.826+0000 7f05a0e00640 1 -- 172.21.15.32:0/2776230402 learned_addr learned my addr 172.21.15.32:0/2776230402 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.826+0000 7f059be00640 1 --2- 172.21.15.32:0/2776230402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938a7b60 0x561993c6fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.826+0000 7f05a0e00640 1 -- 172.21.15.32:0/2776230402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938c8840 msgr2=0x561993c6f110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:22:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.826+0000 7f05a0e00640 1 --2- 172.21.15.32:0/2776230402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938c8840 0x561993c6f110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.826+0000 7f05a0e00640 1 -- 172.21.15.32:0/2776230402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938a7b60 msgr2=0x561993c6fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.826+0000 7f05a0e00640 1 --2- 172.21.15.32:0/2776230402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938a7b60 0x561993c6fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.826+0000 7f05a0e00640 1 -- 172.21.15.32:0/2776230402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561993b86880 con 0x56199389f980 2024-08-31T07:22:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.827+0000 7f05a0e00640 1 --2- 172.21.15.32:0/2776230402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6f650 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f05901032a0 tx=0x7f0590109de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc9d858b40e21d8d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.827+0000 7f058e000640 1 -- 172.21.15.32:0/2776230402 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590115070 con 0x56199389f980 2024-08-31T07:22:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.827+0000 7f058e000640 1 -- 172.21.15.32:0/2776230402 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059010ed50 con 0x56199389f980 2024-08-31T07:22:40.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.827+0000 7f058e000640 1 -- 172.21.15.32:0/2776230402 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05901105a0 con 0x56199389f980 2024-08-31T07:22:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.828+0000 7f0599600640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4da0 0x7f05840b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.828+0000 7f0599600640 1 -- 172.21.15.32:0/2663929233 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f0590105c20 con 0x7f059c103920 2024-08-31T07:22:40.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.828+0000 7f05a1800640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4da0 0x7f05840b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.828+0000 7f05aa43f740 1 -- 172.21.15.32:0/2776230402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 msgr2=0x561993c6f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.828+0000 7f05aa43f740 1 --2- 172.21.15.32:0/2776230402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6f650 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f05901032a0 tx=0x7f0590109de0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.829+0000 7f05aa43f740 1 -- 172.21.15.32:0/2776230402 shutdown_connections 2024-08-31T07:22:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.829+0000 7f05aa43f740 1 --2- 172.21.15.32:0/2776230402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938a7b60 0x561993c6fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.829+0000 7f05aa43f740 1 --2- 172.21.15.32:0/2776230402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6f650 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.829+0000 7f05aa43f740 1 --2- 172.21.15.32:0/2776230402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938c8840 0x561993c6f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.829+0000 7f05aa43f740 1 -- 172.21.15.32:0/2776230402 >> 172.21.15.32:0/2776230402 conn(0x5619938bc990 msgr2=0x561993c6d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.829+0000 7f05aa43f740 1 -- 172.21.15.32:0/2776230402 shutdown_connections 2024-08-31T07:22:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.829+0000 7f05a1800640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4da0 0x7f05840b72b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f05940099a0 tx=0x7f0594009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.829+0000 7f05aa43f740 1 -- 172.21.15.32:0/2776230402 wait complete. 2024-08-31T07:22:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.830+0000 7f05aa43f740 1 -- start start 2024-08-31T07:22:40.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.830+0000 7f05aa43f740 1 Processor -- start 2024-08-31T07:22:40.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56199389f980 0x561993c834b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619938a7b60 0x561993c839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 0x561993c8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f05aa43f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561993a367a0 con 0x5619938c8840 2024-08-31T07:22:40.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f05aa43f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561993b5a7f0 con 0x5619938a7b60 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f05aa43f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561993c85de0 con 0x56199389f980 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f05aa43f740 10 client.0 _ll_register_callbacks cb 0x561993b7fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f059be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619938a7b60 0x561993c839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f059be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619938a7b60 0x561993c839f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55828/0 (socket says 172.21.15.32:55828) 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f059be00640 1 -- 172.21.15.32:0/2269410876 learned_addr learned my addr 172.21.15.32:0/2269410876 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f05a1800640 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 0x561993c8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f05a0e00640 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56199389f980 0x561993c834b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f059be00640 1 -- 172.21.15.32:0/2269410876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56199389f980 msgr2=0x561993c834b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f059be00640 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56199389f980 0x561993c834b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f059be00640 1 -- 172.21.15.32:0/2269410876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 msgr2=0x561993c8f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f059be00640 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 0x561993c8f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f059be00640 1 -- 172.21.15.32:0/2269410876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561993942b90 con 0x5619938a7b60 2024-08-31T07:22:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.831+0000 7f05a1800640 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 0x561993c8f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:22:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.832+0000 7f059be00640 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619938a7b60 0x561993c839f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f05880126e0 tx=0x7f0588012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d912f23d71872fb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.832+0000 7f0582a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:22:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.832+0000 7f0582a00640 1 -- 172.21.15.32:0/2269410876 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588012190 con 0x5619938a7b60 2024-08-31T07:22:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.832+0000 7f0582a00640 1 -- 172.21.15.32:0/2269410876 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f0588004020 con 0x5619938a7b60 2024-08-31T07:22:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.832+0000 7f05aa43f740 1 -- 172.21.15.32:0/2269410876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f059010ed50 con 0x5619938a7b60 2024-08-31T07:22:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.832+0000 7f0582a00640 1 -- 172.21.15.32:0/2269410876 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0588004220 con 0x5619938a7b60 2024-08-31T07:22:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.832+0000 7f05aa43f740 10 client.27572 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.832+0000 7f05aa43f740 1 -- 172.21.15.32:0/2269410876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561993c91de0 con 0x5619938a7b60 2024-08-31T07:22:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.833+0000 7f0582a00640 1 -- 172.21.15.32:0/2269410876 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059010ed50 con 0x5619938a7b60 2024-08-31T07:22:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.833+0000 7f0582a00640 1 -- 172.21.15.32:0/2269410876 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f0588003890 con 0x5619938a7b60 2024-08-31T07:22:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.833+0000 7f05aa43f740 10 client.27572 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.833+0000 7f05aa43f740 10 client.27572 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:22:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.833+0000 7f05aa43f740 20 client.27572 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.833+0000 7f05aa43f740 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5619938871c0 0x561993c95280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.833+0000 7f05aa43f740 4 client.27572 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.833+0000 7f05aa43f740 1 -- 172.21.15.32:0/2269410876 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561993be6480 con 0x5619938871c0 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.833+0000 7f05a0e00640 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5619938871c0 0x561993c95280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.834+0000 7f05a0e00640 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5619938871c0 0x561993c95280 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bceb8bdd24c05b1 server_cookie=b2f1f6210ae6cd48 in_seq=0 out_seq=0 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.834+0000 7f0582a00640 10 client.27572 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f0582a00640 1 -- 172.21.15.32:0/2269410876 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561993be6480 con 0x5619938871c0 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f0582a00640 10 client.27572 handle_command_reply: tid=0 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 client.27572 shutdown 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 -- 172.21.15.32:0/2269410876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619938a7b60 msgr2=0x561993c839f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619938a7b60 0x561993c839f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f05880126e0 tx=0x7f0588012ba0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 -- 172.21.15.32:0/2269410876 shutdown_connections 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5619938871c0 0x561993c95280 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 0x561993c8f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f0582a00640 0 client.27572 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619938a7b60 0x561993c839f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 --2- 172.21.15.32:0/2269410876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56199389f980 0x561993c834b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 -- 172.21.15.32:0/2269410876 >> 172.21.15.32:0/2269410876 conn(0x5619938bc990 msgr2=0x561993c6b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 -- 172.21.15.32:0/2269410876 shutdown_connections 2024-08-31T07:22:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.835+0000 7f05aa43f740 1 -- 172.21.15.32:0/2269410876 wait complete. 2024-08-31T07:22:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.836+0000 7f05aa43f740 20 client.27572 trim_cache size 0 max 16384 2024-08-31T07:22:40.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.868+0000 7f05aa43f740 1 Processor -- start 2024-08-31T07:22:40.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.868+0000 7f05aa43f740 1 -- start start 2024-08-31T07:22:40.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.868+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.868+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938a7b60 0x561993c6f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.868+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 0x561993c8efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.868+0000 7f05aa43f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f059c1ad200 con 0x5619938c8840 2024-08-31T07:22:40.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.868+0000 7f05aa43f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f059c1aee30 con 0x56199389f980 2024-08-31T07:22:40.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.868+0000 7f05aa43f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f059c1ad8d0 con 0x5619938a7b60 2024-08-31T07:22:40.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a1800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 0x561993c8efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a1800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 0x561993c8efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59714/0 (socket says 172.21.15.32:59714) 2024-08-31T07:22:40.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55832/0 (socket says 172.21.15.32:55832) 2024-08-31T07:22:40.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a1800640 1 -- 172.21.15.32:0/1281702815 learned_addr learned my addr 172.21.15.32:0/1281702815 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:40.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f059be00640 1 --2- 172.21.15.32:0/1281702815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938a7b60 0x561993c6f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a0e00640 1 -- 172.21.15.32:0/1281702815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938a7b60 msgr2=0x561993c6f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:40.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a0e00640 1 --2- 172.21.15.32:0/1281702815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938a7b60 0x561993c6f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a0e00640 1 -- 172.21.15.32:0/1281702815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 msgr2=0x561993c8efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:40.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a0e00640 1 --2- 172.21.15.32:0/1281702815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 0x561993c8efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a0e00640 1 -- 172.21.15.32:0/1281702815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561993c6ff40 con 0x56199389f980 2024-08-31T07:22:40.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.869+0000 7f05a0e00640 1 --2- 172.21.15.32:0/1281702815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6ef80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f05900cc000 tx=0x7f0590111d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb6df8fed4383a57 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.870+0000 7f058d600640 1 -- 172.21.15.32:0/1281702815 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590049070 con 0x56199389f980 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.870+0000 7f058d600640 1 -- 172.21.15.32:0/1281702815 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05900cbe10 con 0x56199389f980 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.870+0000 7f058d600640 1 -- 172.21.15.32:0/1281702815 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590115030 con 0x56199389f980 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.871+0000 7f05aa43f740 1 -- 172.21.15.32:0/1281702815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 msgr2=0x561993c6ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.871+0000 7f05aa43f740 1 --2- 172.21.15.32:0/1281702815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6ef80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f05900cc000 tx=0x7f0590111d80 comp rx=0 tx=0).stop 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.871+0000 7f05aa43f740 1 -- 172.21.15.32:0/1281702815 shutdown_connections 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.871+0000 7f05aa43f740 1 --2- 172.21.15.32:0/1281702815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938c8840 0x561993c8efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.871+0000 7f05aa43f740 1 --2- 172.21.15.32:0/1281702815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938a7b60 0x561993c6f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.871+0000 7f05aa43f740 1 --2- 172.21.15.32:0/1281702815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c6ef80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.871+0000 7f05aa43f740 1 -- 172.21.15.32:0/1281702815 >> 172.21.15.32:0/1281702815 conn(0x5619938bc990 msgr2=0x561993c6c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.871+0000 7f05aa43f740 1 -- 172.21.15.32:0/1281702815 shutdown_connections 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.871+0000 7f05aa43f740 1 -- 172.21.15.32:0/1281702815 wait complete. 2024-08-31T07:22:40.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.871+0000 7f05aa43f740 1 -- start start 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05aa43f740 1 Processor -- start 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c8b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938a7b60 0x561993c8b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05aa43f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938c8840 0x561993c85ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05aa43f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f059c1ae760 con 0x5619938a7b60 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05aa43f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f059c1ae060 con 0x56199389f980 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05aa43f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f059c1ab220 con 0x5619938c8840 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05aa43f740 10 client.0 _ll_register_callbacks cb 0x7f0588009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05a0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c8b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05a0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c8b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55842/0 (socket says 172.21.15.32:55842) 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f05a0e00640 1 -- 172.21.15.32:0/3572310883 learned_addr learned my addr 172.21.15.32:0/3572310883 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.872+0000 7f059be00640 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938a7b60 0x561993c8b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.873+0000 7f05a1800640 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938c8840 0x561993c85ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.873+0000 7f05a0e00640 1 -- 172.21.15.32:0/3572310883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938c8840 msgr2=0x561993c85ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.873+0000 7f05a0e00640 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938c8840 0x561993c85ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.873+0000 7f05a0e00640 1 -- 172.21.15.32:0/3572310883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938a7b60 msgr2=0x561993c8b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.873+0000 7f05a0e00640 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938a7b60 0x561993c8b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.873+0000 7f05a0e00640 1 -- 172.21.15.32:0/3572310883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561993c86780 con 0x56199389f980 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.873+0000 7f05a0e00640 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c8b2d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f059002a9b0 tx=0x7f05900045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba00a8b61a8ccd0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.873+0000 7f0582a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.873+0000 7f0582a00640 1 -- 172.21.15.32:0/3572310883 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590049070 con 0x56199389f980 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.873+0000 7f0582a00640 1 -- 172.21.15.32:0/3572310883 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f05900c88f0 con 0x56199389f980 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f05aa43f740 1 -- 172.21.15.32:0/3572310883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05900cbe10 con 0x56199389f980 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f0582a00640 1 -- 172.21.15.32:0/3572310883 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0590115070 con 0x56199389f980 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f05aa43f740 10 client.27578 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f05aa43f740 1 -- 172.21.15.32:0/3572310883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561993d00620 con 0x56199389f980 2024-08-31T07:22:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f0582a00640 1 -- 172.21.15.32:0/3572310883 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05900cbe10 con 0x56199389f980 2024-08-31T07:22:40.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f0582a00640 1 -- 172.21.15.32:0/3572310883 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f05900c8590 con 0x56199389f980 2024-08-31T07:22:40.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f05aa43f740 10 client.27578 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:40.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f05aa43f740 10 client.27578 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:22:40.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f05aa43f740 20 client.27578 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:40.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f05aa43f740 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56199385c570 0x561993d02bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:40.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f05aa43f740 4 client.27578 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.5, "await_for": 45.0}] 2024-08-31T07:22:40.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.874+0000 7f05aa43f740 1 -- 172.21.15.32:0/3572310883 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.5, "await_for": 45.0}) -- 0x561993be1f20 con 0x56199385c570 2024-08-31T07:22:40.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.875+0000 7f059be00640 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56199385c570 0x561993d02bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:40.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.875+0000 7f059be00640 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56199385c570 0x561993d02bb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a948a0c69946c7b6 server_cookie=dc23c1856e60b363 in_seq=0 out_seq=0 2024-08-31T07:22:40.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.875+0000 7f0582a00640 10 client.27578 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:40 smithi073 ceph-mon[21995]: pgmap v673: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-08-31T07:22:40.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.959+0000 7f0582a00640 1 -- 172.21.15.32:0/3572310883 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561993be1f20 con 0x56199385c570 2024-08-31T07:22:40.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.959+0000 7f0582a00640 10 client.27578 handle_command_reply: tid=0 2024-08-31T07:22:40.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.959+0000 7f05aa43f740 1 client.27578 shutdown 2024-08-31T07:22:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 1 -- 172.21.15.32:0/3572310883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 msgr2=0x561993c8b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c8b2d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f059002a9b0 tx=0x7f05900045e0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 1 -- 172.21.15.32:0/3572310883 shutdown_connections 2024-08-31T07:22:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56199385c570 0x561993d02bb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619938c8840 0x561993c85ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619938a7b60 0x561993c8b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 1 --2- 172.21.15.32:0/3572310883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56199389f980 0x561993c8b2d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 1 -- 172.21.15.32:0/3572310883 >> 172.21.15.32:0/3572310883 conn(0x5619938bc990 msgr2=0x561993c70140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f0582a00640 0 client.27578 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:40.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 1 -- 172.21.15.32:0/3572310883 shutdown_connections 2024-08-31T07:22:40.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 1 -- 172.21.15.32:0/3572310883 wait complete. 2024-08-31T07:22:40.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.960+0000 7f05aa43f740 20 client.27578 trim_cache size 0 max 16384 2024-08-31T07:22:40.963 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:22:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 40, 2024-08-31T07:22:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:22:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 30, 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "de2c8f7e": { 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 30, 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 135.5, 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:22:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:22:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:22:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:22:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:22:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:40.965 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.961+0000 7f05a3800640 1 -- 172.21.15.32:0/2663929233 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4da0 msgr2=0x7f05840b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.961+0000 7f05a3800640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4da0 0x7f05840b72b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f05940099a0 tx=0x7f0594009500 comp rx=0 tx=0).stop 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.961+0000 7f05a3800640 1 -- 172.21.15.32:0/2663929233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f059c103920 msgr2=0x7f059c19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.961+0000 7f05a3800640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f059c103920 0x7f059c19c490 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0590005ab0 tx=0x7f0590003c10 comp rx=0 tx=0).stop 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.961+0000 7f05a3800640 1 -- 172.21.15.32:0/2663929233 shutdown_connections 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.961+0000 7f05a3800640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4da0 0x7f05840b72b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.961+0000 7f05a3800640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f059c109bb0 0x7f059c1a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.961+0000 7f05a3800640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c109260 0x7f059c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.961+0000 7f05a3800640 1 --2- 172.21.15.32:0/2663929233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f059c103920 0x7f059c19c490 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.961+0000 7f05a3800640 1 -- 172.21.15.32:0/2663929233 >> 172.21.15.32:0/2663929233 conn(0x7f059c0ff370 msgr2=0x7f059c10e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.962+0000 7f05a3800640 1 -- 172.21.15.32:0/2663929233 shutdown_connections 2024-08-31T07:22:40.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:40.962+0000 7f05a3800640 1 -- 172.21.15.32:0/2663929233 wait complete. 2024-08-31T07:22:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:40 smithi084 ceph-mon[22387]: pgmap v673: 97 pgs: 97 active+clean; 241 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-08-31T07:22:41.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 3.04 MB/sec execute 390 sec latency 804.559 ms 2024-08-31T07:22:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.818+0000 7fead8200640 1 -- 172.21.15.32:0/3302392890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead00776e0 msgr2=0x7fead0077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.818+0000 7fead8200640 1 --2- 172.21.15.32:0/3302392890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead00776e0 0x7fead0077af0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feac4009900 tx=0x7feac402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.819+0000 7fead8200640 1 -- 172.21.15.32:0/3302392890 shutdown_connections 2024-08-31T07:22:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.819+0000 7fead8200640 1 --2- 172.21.15.32:0/3302392890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fead01144e0 0x7fead0116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.819+0000 7fead8200640 1 --2- 172.21.15.32:0/3302392890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead0078030 0x7fead0113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.819+0000 7fead8200640 1 --2- 172.21.15.32:0/3302392890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead00776e0 0x7fead0077af0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.819+0000 7fead8200640 1 -- 172.21.15.32:0/3302392890 >> 172.21.15.32:0/3302392890 conn(0x7fead01015b0 msgr2=0x7fead0103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.819+0000 7fead8200640 1 -- 172.21.15.32:0/3302392890 shutdown_connections 2024-08-31T07:22:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.819+0000 7fead8200640 1 -- 172.21.15.32:0/3302392890 wait complete. 2024-08-31T07:22:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.820+0000 7fead8200640 1 Processor -- start 2024-08-31T07:22:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.820+0000 7fead8200640 1 -- start start 2024-08-31T07:22:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.820+0000 7fead8200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead00776e0 0x7fead01a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.820+0000 7fead8200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead0078030 0x7fead01a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.820+0000 7fead8200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fead01144e0 0x7fead01a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.820+0000 7fead8200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fead0119e90 con 0x7fead0078030 2024-08-31T07:22:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.820+0000 7fead8200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fead0119d10 con 0x7fead01144e0 2024-08-31T07:22:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.820+0000 7fead8200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fead011a010 con 0x7fead00776e0 2024-08-31T07:22:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead0078030 0x7fead01a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead6200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fead01144e0 0x7fead01a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead6200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fead01144e0 0x7fead01a60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55848/0 (socket says 172.21.15.32:55848) 2024-08-31T07:22:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead6200640 1 -- 172.21.15.32:0/326044452 learned_addr learned my addr 172.21.15.32:0/326044452 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead5800640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead00776e0 0x7fead01a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead6200640 1 -- 172.21.15.32:0/326044452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead00776e0 msgr2=0x7fead01a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead6200640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead00776e0 0x7fead01a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead6200640 1 -- 172.21.15.32:0/326044452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead0078030 msgr2=0x7fead01a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead6200640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead0078030 0x7fead01a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead6200640 1 -- 172.21.15.32:0/326044452 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead01a67f0 con 0x7fead01144e0 2024-08-31T07:22:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.821+0000 7fead6200640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fead01144e0 0x7fead01a60c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feac8009fd0 tx=0x7feac800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.822+0000 7feace000640 1 -- 172.21.15.32:0/326044452 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8017070 con 0x7fead01144e0 2024-08-31T07:22:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.822+0000 7feace000640 1 -- 172.21.15.32:0/326044452 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac8003e20 con 0x7fead01144e0 2024-08-31T07:22:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.822+0000 7feace000640 1 -- 172.21.15.32:0/326044452 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8010460 con 0x7fead01144e0 2024-08-31T07:22:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.822+0000 7fead8200640 1 -- 172.21.15.32:0/326044452 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead01a6ae0 con 0x7fead01144e0 2024-08-31T07:22:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.822+0000 7fead8200640 1 -- 172.21.15.32:0/326044452 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead01ad240 con 0x7fead01144e0 2024-08-31T07:22:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.825+0000 7feadee6b740 1 Processor -- start 2024-08-31T07:22:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.825+0000 7feace000640 1 -- 172.21.15.32:0/326044452 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7feac8003970 con 0x7fead01144e0 2024-08-31T07:22:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.825+0000 7feadee6b740 1 -- start start 2024-08-31T07:22:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7feadee6b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d54c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7feadee6b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 0x55a49d54ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7feadee6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d186390 0x55a49d54d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7feadee6b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a49d4c37b0 con 0x55a49d1a7070 2024-08-31T07:22:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7feadee6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a49d4b2d40 con 0x55a49d186390 2024-08-31T07:22:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7feadee6b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a49d4b6cf0 con 0x55a49d17e1b0 2024-08-31T07:22:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7fead4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d54c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7fead4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d54c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59758/0 (socket says 172.21.15.32:59758) 2024-08-31T07:22:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7fead5800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 0x55a49d54ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7fead4e00640 1 -- 172.21.15.32:0/3452213799 learned_addr learned my addr 172.21.15.32:0/3452213799 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.826+0000 7fead4e00640 1 --2- 172.21.15.32:0/3452213799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d186390 0x55a49d54d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7fead5800640 1 -- 172.21.15.32:0/3452213799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d186390 msgr2=0x55a49d54d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7fead5800640 1 --2- 172.21.15.32:0/3452213799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d186390 0x55a49d54d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7fead5800640 1 -- 172.21.15.32:0/3452213799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 msgr2=0x55a49d54c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7fead5800640 1 --2- 172.21.15.32:0/3452213799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d54c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7fead5800640 1 -- 172.21.15.32:0/3452213799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49d212650 con 0x55a49d17e1b0 2024-08-31T07:22:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7fead4e00640 1 --2- 172.21.15.32:0/3452213799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d54c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7fead5800640 1 --2- 172.21.15.32:0/3452213799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 0x55a49d54ced0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7feac402a9e0 tx=0x7feac4004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b98fbee83249edf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7feace000640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feab80b4ca0 0x7feab80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7feac2a00640 1 -- 172.21.15.32:0/3452213799 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4035040 con 0x55a49d17e1b0 2024-08-31T07:22:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7feace000640 1 -- 172.21.15.32:0/326044452 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7feac80e50b0 con 0x7fead01144e0 2024-08-31T07:22:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7feac2a00640 1 -- 172.21.15.32:0/3452213799 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac40054f0 con 0x55a49d17e1b0 2024-08-31T07:22:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.827+0000 7feac2a00640 1 -- 172.21.15.32:0/3452213799 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4003850 con 0x55a49d17e1b0 2024-08-31T07:22:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.828+0000 7fead4e00640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feab80b4ca0 0x7feab80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.828+0000 7feadee6b740 1 -- 172.21.15.32:0/3452213799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 msgr2=0x55a49d54ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.828+0000 7fead4e00640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feab80b4ca0 0x7feab80b71b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feabc00bfd0 tx=0x7feabc00ba40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.828+0000 7feadee6b740 1 --2- 172.21.15.32:0/3452213799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 0x55a49d54ced0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7feac402a9e0 tx=0x7feac4004180 comp rx=0 tx=0).stop 2024-08-31T07:22:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.829+0000 7feadee6b740 1 -- 172.21.15.32:0/3452213799 shutdown_connections 2024-08-31T07:22:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.829+0000 7feadee6b740 1 --2- 172.21.15.32:0/3452213799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d186390 0x55a49d54d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.829+0000 7feadee6b740 1 --2- 172.21.15.32:0/3452213799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 0x55a49d54ced0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.829+0000 7feadee6b740 1 --2- 172.21.15.32:0/3452213799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d54c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.829+0000 7feadee6b740 1 -- 172.21.15.32:0/3452213799 >> 172.21.15.32:0/3452213799 conn(0x55a49d19b460 msgr2=0x55a49d54a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.829+0000 7feadee6b740 1 -- 172.21.15.32:0/3452213799 shutdown_connections 2024-08-31T07:22:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.829+0000 7feadee6b740 1 -- 172.21.15.32:0/3452213799 wait complete. 2024-08-31T07:22:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.829+0000 7feadee6b740 1 -- start start 2024-08-31T07:22:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.830+0000 7feadee6b740 1 Processor -- start 2024-08-31T07:22:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.830+0000 7feadee6b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 0x55a49d558570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.830+0000 7feadee6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d186390 0x55a49d558ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.830+0000 7feadee6b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d564400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.830+0000 7feadee6b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a49d315700 con 0x55a49d1a7070 2024-08-31T07:22:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.830+0000 7feadee6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a49d439cf0 con 0x55a49d186390 2024-08-31T07:22:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.830+0000 7feadee6b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a49d55b110 con 0x55a49d17e1b0 2024-08-31T07:22:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.830+0000 7feadee6b740 10 client.0 _ll_register_callbacks cb 0x55a49d45f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.830+0000 7fead6200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d564400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.830+0000 7fead5800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 0x55a49d558570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7fead5800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 0x55a49d558570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54064/0 (socket says 172.21.15.32:54064) 2024-08-31T07:22:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7fead5800640 1 -- 172.21.15.32:0/3891821935 learned_addr learned my addr 172.21.15.32:0/3891821935 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7fead4e00640 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d186390 0x55a49d558ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7fead6200640 1 -- 172.21.15.32:0/3891821935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 msgr2=0x55a49d558570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7fead6200640 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 0x55a49d558570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7fead6200640 1 -- 172.21.15.32:0/3891821935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d186390 msgr2=0x55a49d558ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7fead6200640 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d186390 0x55a49d558ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7fead6200640 1 -- 172.21.15.32:0/3891821935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49d47a8b0 con 0x55a49d1a7070 2024-08-31T07:22:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7fead6200640 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d564400 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7feac80e28f0 tx=0x7feac8003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ee98ab3d1aa6c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7feab7400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:22:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7feab7400640 1 -- 172.21.15.32:0/3891821935 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8020040 con 0x55a49d1a7070 2024-08-31T07:22:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7feab7400640 1 -- 172.21.15.32:0/3891821935 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7feac80a7bd0 con 0x55a49d1a7070 2024-08-31T07:22:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.831+0000 7feadee6b740 1 -- 172.21.15.32:0/3891821935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac40054f0 con 0x55a49d1a7070 2024-08-31T07:22:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feab7400640 1 -- 172.21.15.32:0/3891821935 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7feac80b0100 con 0x55a49d1a7070 2024-08-31T07:22:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feadee6b740 10 client.17580 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feadee6b740 1 -- 172.21.15.32:0/3891821935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a49d567150 con 0x55a49d1a7070 2024-08-31T07:22:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feab7400640 1 -- 172.21.15.32:0/3891821935 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac40054f0 con 0x55a49d1a7070 2024-08-31T07:22:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feab7400640 1 -- 172.21.15.32:0/3891821935 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7feac80aa610 con 0x55a49d1a7070 2024-08-31T07:22:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feadee6b740 10 client.17580 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feadee6b740 10 client.17580 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:22:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feadee6b740 20 client.17580 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feadee6b740 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a49d1659f0 0x55a49d56a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feadee6b740 4 client.17580 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:22:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.832+0000 7feadee6b740 1 -- 172.21.15.32:0/3891821935 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a49d4c5700 con 0x55a49d1659f0 2024-08-31T07:22:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.833+0000 7fead5800640 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a49d1659f0 0x55a49d56a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.833+0000 7fead5800640 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a49d1659f0 0x55a49d56a660 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7cb86e3e6dbba1be server_cookie=d37ced8f8fb08367 in_seq=0 out_seq=0 2024-08-31T07:22:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.833+0000 7feab7400640 10 client.17580 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:22:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.834+0000 7feab7400640 1 -- 172.21.15.32:0/3891821935 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a49d4c5700 con 0x55a49d1659f0 2024-08-31T07:22:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.834+0000 7feab7400640 10 client.17580 handle_command_reply: tid=0 2024-08-31T07:22:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.834+0000 7feadee6b740 1 client.17580 shutdown 2024-08-31T07:22:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.834+0000 7feadee6b740 1 -- 172.21.15.32:0/3891821935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 msgr2=0x55a49d564400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.834+0000 7feadee6b740 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d564400 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7feac80e28f0 tx=0x7feac8003c60 comp rx=0 tx=0).stop 2024-08-31T07:22:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.834+0000 7feadee6b740 1 -- 172.21.15.32:0/3891821935 shutdown_connections 2024-08-31T07:22:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.834+0000 7feadee6b740 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1a7070 0x55a49d564400 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.834+0000 7feadee6b740 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a49d1659f0 0x55a49d56a660 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.834+0000 7feadee6b740 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d186390 0x55a49d558ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.834+0000 7feadee6b740 1 --2- 172.21.15.32:0/3891821935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d17e1b0 0x55a49d558570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.835+0000 7feab7400640 0 client.17580 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:22:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.835+0000 7feadee6b740 1 -- 172.21.15.32:0/3891821935 >> 172.21.15.32:0/3891821935 conn(0x55a49d19b460 msgr2=0x55a49d54fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.835+0000 7feadee6b740 1 -- 172.21.15.32:0/3891821935 shutdown_connections 2024-08-31T07:22:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.835+0000 7feadee6b740 1 -- 172.21.15.32:0/3891821935 wait complete. 2024-08-31T07:22:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.835+0000 7feadee6b740 20 client.17580 trim_cache size 0 max 16384 2024-08-31T07:22:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.849+0000 7feadee6b740 1 Processor -- start 2024-08-31T07:22:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.849+0000 7feadee6b740 1 -- start start 2024-08-31T07:22:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7feadee6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d1659f0 0x55a49d54a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7feadee6b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d17e1b0 0x55a49d555560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7feadee6b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 0x55a49d555aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7feadee6b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fead01b58a0 con 0x55a49d17e1b0 2024-08-31T07:22:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7feadee6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fead01b74d0 con 0x55a49d1659f0 2024-08-31T07:22:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7feadee6b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fead01b5f70 con 0x55a49d186390 2024-08-31T07:22:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7fead4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d17e1b0 0x55a49d555560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7fead5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d1659f0 0x55a49d54a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7fead4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d17e1b0 0x55a49d555560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59774/0 (socket says 172.21.15.32:59774) 2024-08-31T07:22:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7fead5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d1659f0 0x55a49d54a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55878/0 (socket says 172.21.15.32:55878) 2024-08-31T07:22:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7fead4e00640 1 -- 172.21.15.32:0/1786997179 learned_addr learned my addr 172.21.15.32:0/1786997179 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.850+0000 7fead6200640 1 --2- 172.21.15.32:0/1786997179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 0x55a49d555aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.851+0000 7fead5800640 1 -- 172.21.15.32:0/1786997179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 msgr2=0x55a49d555aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.851+0000 7fead5800640 1 --2- 172.21.15.32:0/1786997179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 0x55a49d555aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.851+0000 7fead5800640 1 -- 172.21.15.32:0/1786997179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d17e1b0 msgr2=0x55a49d555560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.851+0000 7fead5800640 1 --2- 172.21.15.32:0/1786997179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d17e1b0 0x55a49d555560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.851+0000 7fead5800640 1 -- 172.21.15.32:0/1786997179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49d54ec80 con 0x55a49d1659f0 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.851+0000 7fead5800640 1 --2- 172.21.15.32:0/1786997179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d1659f0 0x55a49d54a780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feac40057c0 tx=0x7feac4004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72697489091f8834 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.851+0000 7feac2000640 1 -- 172.21.15.32:0/1786997179 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4035040 con 0x55a49d1659f0 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.851+0000 7feac2000640 1 -- 172.21.15.32:0/1786997179 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac40048c0 con 0x55a49d1659f0 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.851+0000 7feac2000640 1 -- 172.21.15.32:0/1786997179 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4003850 con 0x55a49d1659f0 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.852+0000 7feadee6b740 1 -- 172.21.15.32:0/1786997179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d1659f0 msgr2=0x55a49d54a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.852+0000 7feadee6b740 1 --2- 172.21.15.32:0/1786997179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d1659f0 0x55a49d54a780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feac40057c0 tx=0x7feac4004cb0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.852+0000 7feadee6b740 1 -- 172.21.15.32:0/1786997179 shutdown_connections 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.852+0000 7feadee6b740 1 --2- 172.21.15.32:0/1786997179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 0x55a49d555aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.852+0000 7feadee6b740 1 --2- 172.21.15.32:0/1786997179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d17e1b0 0x55a49d555560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.852+0000 7feadee6b740 1 --2- 172.21.15.32:0/1786997179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d1659f0 0x55a49d54a780 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.852+0000 7feadee6b740 1 -- 172.21.15.32:0/1786997179 >> 172.21.15.32:0/1786997179 conn(0x55a49d19b460 msgr2=0x55a49d54a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.852+0000 7feadee6b740 1 -- 172.21.15.32:0/1786997179 shutdown_connections 2024-08-31T07:22:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.852+0000 7feadee6b740 1 -- 172.21.15.32:0/1786997179 wait complete. 2024-08-31T07:22:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.853+0000 7feadee6b740 1 -- start start 2024-08-31T07:22:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.853+0000 7feadee6b740 1 Processor -- start 2024-08-31T07:22:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.853+0000 7feadee6b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1659f0 0x55a49d560880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.853+0000 7feadee6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d17e1b0 0x55a49d55b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.853+0000 7feadee6b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 0x55a49d55b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.853+0000 7feadee6b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fead01b6e00 con 0x55a49d1659f0 2024-08-31T07:22:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.853+0000 7feadee6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fead01b6700 con 0x55a49d17e1b0 2024-08-31T07:22:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.853+0000 7feadee6b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fead01b38a0 con 0x55a49d186390 2024-08-31T07:22:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.853+0000 7feadee6b740 10 client.0 _ll_register_callbacks cb 0x7feac80ab0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead6200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 0x55a49d55b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d17e1b0 0x55a49d55b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead6200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 0x55a49d55b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54082/0 (socket says 172.21.15.32:54082) 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead6200640 1 -- 172.21.15.32:0/1478971613 learned_addr learned my addr 172.21.15.32:0/1478971613 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead5800640 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1659f0 0x55a49d560880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead6200640 1 -- 172.21.15.32:0/1478971613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d17e1b0 msgr2=0x55a49d55b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead6200640 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d17e1b0 0x55a49d55b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead6200640 1 -- 172.21.15.32:0/1478971613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1659f0 msgr2=0x55a49d560880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead6200640 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1659f0 0x55a49d560880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead6200640 1 -- 172.21.15.32:0/1478971613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49d55bda0 con 0x55a49d186390 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7fead6200640 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 0x55a49d55b860 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feac80e2e70 tx=0x7feac80a7e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f841234817e5db98 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.854+0000 7feab7400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feab7400640 1 -- 172.21.15.32:0/1478971613 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8014070 con 0x55a49d186390 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feab7400640 1 -- 172.21.15.32:0/1478971613 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7feac80a7f30 con 0x55a49d186390 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feadee6b740 1 -- 172.21.15.32:0/1478971613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac40048c0 con 0x55a49d186390 2024-08-31T07:22:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feab7400640 1 -- 172.21.15.32:0/1478971613 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7feac8020070 con 0x55a49d186390 2024-08-31T07:22:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feadee6b740 10 client.27595 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feadee6b740 1 -- 172.21.15.32:0/1478971613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a49d5d5c70 con 0x55a49d186390 2024-08-31T07:22:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feab7400640 1 -- 172.21.15.32:0/1478971613 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac40048c0 con 0x55a49d186390 2024-08-31T07:22:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feab7400640 1 -- 172.21.15.32:0/1478971613 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7feac80b1290 con 0x55a49d186390 2024-08-31T07:22:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feadee6b740 10 client.27595 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feadee6b740 10 client.27595 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:22:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feadee6b740 20 client.27595 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.855+0000 7feadee6b740 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a49d1a7070 0x55a49d5d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.856+0000 7feadee6b740 4 client.27595 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:22:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.856+0000 7feadee6b740 1 -- 172.21.15.32:0/1478971613 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a49d4c11a0 con 0x55a49d1a7070 2024-08-31T07:22:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.856+0000 7fead5800640 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a49d1a7070 0x55a49d5d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7fead5800640 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a49d1a7070 0x55a49d5d82a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45f409f819a6180a server_cookie=9979d53e3559088a in_seq=0 out_seq=0 2024-08-31T07:22:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feab7400640 10 client.27595 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:22:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feab7400640 1 -- 172.21.15.32:0/1478971613 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a49d4c11a0 con 0x55a49d1a7070 2024-08-31T07:22:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feab7400640 10 client.27595 handle_command_reply: tid=0 2024-08-31T07:22:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 client.27595 shutdown 2024-08-31T07:22:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 -- 172.21.15.32:0/1478971613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 msgr2=0x55a49d55b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 0x55a49d55b860 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feac80e2e70 tx=0x7feac80a7e80 comp rx=0 tx=0).stop 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 -- 172.21.15.32:0/1478971613 shutdown_connections 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a49d186390 0x55a49d55b860 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55a49d1a7070 0x55a49d5d82a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a49d17e1b0 0x55a49d55b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 --2- 172.21.15.32:0/1478971613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a49d1659f0 0x55a49d560880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feab7400640 0 client.27595 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 -- 172.21.15.32:0/1478971613 >> 172.21.15.32:0/1478971613 conn(0x55a49d19b460 msgr2=0x55a49d54d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 -- 172.21.15.32:0/1478971613 shutdown_connections 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.857+0000 7feadee6b740 1 -- 172.21.15.32:0/1478971613 wait complete. 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7feadee6b740 20 client.27595 trim_cache size 0 max 16384 2024-08-31T07:22:41.862 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:22:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7fead8200640 1 -- 172.21.15.32:0/326044452 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feab80b4ca0 msgr2=0x7feab80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7fead8200640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feab80b4ca0 0x7feab80b71b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feabc00bfd0 tx=0x7feabc00ba40 comp rx=0 tx=0).stop 2024-08-31T07:22:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7fead8200640 1 -- 172.21.15.32:0/326044452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fead01144e0 msgr2=0x7fead01a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7fead8200640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fead01144e0 0x7fead01a60c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feac8009fd0 tx=0x7feac800b260 comp rx=0 tx=0).stop 2024-08-31T07:22:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7fead8200640 1 -- 172.21.15.32:0/326044452 shutdown_connections 2024-08-31T07:22:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7fead8200640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7feab80b4ca0 0x7feab80b71b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7fead8200640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fead01144e0 0x7fead01a60c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7fead8200640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead0078030 0x7fead01a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7fead8200640 1 --2- 172.21.15.32:0/326044452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead00776e0 0x7fead01a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.858+0000 7fead8200640 1 -- 172.21.15.32:0/326044452 >> 172.21.15.32:0/326044452 conn(0x7fead01015b0 msgr2=0x7fead01165c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.859+0000 7fead8200640 1 -- 172.21.15.32:0/326044452 shutdown_connections 2024-08-31T07:22:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:41.859+0000 7fead8200640 1 -- 172.21.15.32:0/326044452 wait complete. 2024-08-31T07:22:42.320 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'de2c8f7e', quiesce took 5.8 seconds. Will release after: 9.7 2024-08-31T07:22:42.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 3.03 MB/sec execute 391 sec latency 1804.694 ms 2024-08-31T07:22:42.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:42 smithi032 ceph-mon[17252]: pgmap v674: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-08-31T07:22:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:42 smithi073 ceph-mon[21995]: pgmap v674: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-08-31T07:22:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:42 smithi084 ceph-mon[22387]: pgmap v674: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-08-31T07:22:43.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:22:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:43.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 3.02 MB/sec execute 392 sec latency 2804.855 ms 2024-08-31T07:22:44.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.137+0000 7f8dae000640 1 -- 172.21.15.32:0/1432793707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8da8103750 msgr2=0x7f8da8103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:44.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.137+0000 7f8dae000640 1 --2- 172.21.15.32:0/1432793707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8da8103750 0x7f8da8103b60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009900 tx=0x7f8d9802a410 comp rx=0 tx=0).stop 2024-08-31T07:22:44.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.137+0000 7f8dae000640 1 -- 172.21.15.32:0/1432793707 shutdown_connections 2024-08-31T07:22:44.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.137+0000 7f8dae000640 1 --2- 172.21.15.32:0/1432793707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da81099e0 0x7f8da8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.137+0000 7f8dae000640 1 --2- 172.21.15.32:0/1432793707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8da8109090 0x7f8da81094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.137+0000 7f8dae000640 1 --2- 172.21.15.32:0/1432793707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8da8103750 0x7f8da8103b60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.137+0000 7f8dae000640 1 -- 172.21.15.32:0/1432793707 >> 172.21.15.32:0/1432793707 conn(0x7f8da80ff1a0 msgr2=0x7f8da81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:44.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.137+0000 7f8dae000640 1 -- 172.21.15.32:0/1432793707 shutdown_connections 2024-08-31T07:22:44.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.137+0000 7f8dae000640 1 -- 172.21.15.32:0/1432793707 wait complete. 2024-08-31T07:22:44.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.138+0000 7f8dae000640 1 Processor -- start 2024-08-31T07:22:44.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8dae000640 1 -- start start 2024-08-31T07:22:44.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8dae000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da8103750 0x7f8da819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8dae000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8da8109090 0x7f8da819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8dae000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8da81099e0 0x7f8da81a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8dae000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81136e0 con 0x7f8da8109090 2024-08-31T07:22:44.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8dae000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8113560 con 0x7f8da81099e0 2024-08-31T07:22:44.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8dae000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8113860 con 0x7f8da8103750 2024-08-31T07:22:44.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8da8109090 0x7f8da819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8da8109090 0x7f8da819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59798/0 (socket says 172.21.15.32:59798) 2024-08-31T07:22:44.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8da7e00640 1 -- 172.21.15.32:0/4222504019 learned_addr learned my addr 172.21.15.32:0/4222504019 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.139+0000 7f8dacc00640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da8103750 0x7f8da819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.140+0000 7f8dad600640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8da81099e0 0x7f8da81a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.140+0000 7f8da7e00640 1 -- 172.21.15.32:0/4222504019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da8103750 msgr2=0x7f8da819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.140+0000 7f8da7e00640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da8103750 0x7f8da819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.140+0000 7f8da7e00640 1 -- 172.21.15.32:0/4222504019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8da81099e0 msgr2=0x7f8da81a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.140+0000 7f8da7e00640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8da81099e0 0x7f8da81a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.140+0000 7f8da7e00640 1 -- 172.21.15.32:0/4222504019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da81a2220 con 0x7f8da8109090 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.140+0000 7f8da7e00640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8da8109090 0x7f8da819c850 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400e9f0 tx=0x7f8d9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.140+0000 7f8da5600640 1 -- 172.21.15.32:0/4222504019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400ccc0 con 0x7f8da8109090 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.141+0000 7f8da5600640 1 -- 172.21.15.32:0/4222504019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d940108e0 con 0x7f8da8109090 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.141+0000 7f8dae000640 1 -- 172.21.15.32:0/4222504019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da81a2510 con 0x7f8da8109090 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.141+0000 7f8da5600640 1 -- 172.21.15.32:0/4222504019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94018860 con 0x7f8da8109090 2024-08-31T07:22:44.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.141+0000 7f8dae000640 1 -- 172.21.15.32:0/4222504019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da81a8c70 con 0x7f8da8109090 2024-08-31T07:22:44.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.144+0000 7f8db6255740 1 Processor -- start 2024-08-31T07:22:44.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.144+0000 7f8db6255740 1 -- start start 2024-08-31T07:22:44.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.144+0000 7f8da5600640 1 -- 172.21.15.32:0/4222504019 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8d94010430 con 0x7f8da8109090 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8db6255740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8db6255740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8db6255740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275def3390 0x56275e2ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8db6255740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56275e2307b0 con 0x56275def3390 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8db6255740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56275e21fd40 con 0x56275deeb1b0 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8db6255740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56275e223cf0 con 0x56275df14070 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8dacc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8dacc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2ba0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55916/0 (socket says 172.21.15.32:55916) 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8dacc00640 1 -- 172.21.15.32:0/1121361548 learned_addr learned my addr 172.21.15.32:0/1121361548 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8dad600640 1 --2- 172.21.15.32:0/1121361548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8da7e00640 1 --2- 172.21.15.32:0/1121361548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275def3390 0x56275e2ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8dacc00640 1 -- 172.21.15.32:0/1121361548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 msgr2=0x56275e2b9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:44.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8dacc00640 1 --2- 172.21.15.32:0/1121361548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2b9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8dacc00640 1 -- 172.21.15.32:0/1121361548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275def3390 msgr2=0x56275e2ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8dacc00640 1 --2- 172.21.15.32:0/1121361548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275def3390 0x56275e2ba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8dacc00640 1 -- 172.21.15.32:0/1121361548 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275df7f650 con 0x56275deeb1b0 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.145+0000 7f8dad600640 1 --2- 172.21.15.32:0/1121361548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2b9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.146+0000 7f8dacc00640 1 --2- 172.21.15.32:0/1121361548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2ba0a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009230 tx=0x7f8d98004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fa322bbfff09d39 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.146+0000 7f8d9dc00640 1 -- 172.21.15.32:0/1121361548 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98035040 con 0x56275deeb1b0 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.146+0000 7f8da5600640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d8c0b4d20 0x7f8d8c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.146+0000 7f8d9dc00640 1 -- 172.21.15.32:0/1121361548 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d980054f0 con 0x56275deeb1b0 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.146+0000 7f8d9dc00640 1 -- 172.21.15.32:0/1121361548 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d980035f0 con 0x56275deeb1b0 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.146+0000 7f8da5600640 1 -- 172.21.15.32:0/4222504019 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f8d940e5770 con 0x7f8da8109090 2024-08-31T07:22:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.146+0000 7f8dad600640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d8c0b4d20 0x7f8d8c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.147+0000 7f8db6255740 1 -- 172.21.15.32:0/1121361548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 msgr2=0x56275e2ba0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.147+0000 7f8db6255740 1 --2- 172.21.15.32:0/1121361548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2ba0a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009230 tx=0x7f8d98004170 comp rx=0 tx=0).stop 2024-08-31T07:22:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.147+0000 7f8dad600640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d8c0b4d20 0x7f8d8c0b7230 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8da0005fd0 tx=0x7f8da0009600 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.147+0000 7f8db6255740 1 -- 172.21.15.32:0/1121361548 shutdown_connections 2024-08-31T07:22:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.147+0000 7f8db6255740 1 --2- 172.21.15.32:0/1121361548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275def3390 0x56275e2ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.147+0000 7f8db6255740 1 --2- 172.21.15.32:0/1121361548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2ba0a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.147+0000 7f8db6255740 1 --2- 172.21.15.32:0/1121361548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2b9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.147+0000 7f8db6255740 1 -- 172.21.15.32:0/1121361548 >> 172.21.15.32:0/1121361548 conn(0x56275df08460 msgr2=0x56275e2b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.147+0000 7f8db6255740 1 -- 172.21.15.32:0/1121361548 shutdown_connections 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.147+0000 7f8db6255740 1 -- 172.21.15.32:0/1121361548 wait complete. 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.148+0000 7f8db6255740 1 -- start start 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.148+0000 7f8db6255740 1 Processor -- start 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8db6255740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8db6255740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275def3390 0x56275e2c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8db6255740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275df14070 0x56275e2d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8db6255740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56275e082700 con 0x56275df14070 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8db6255740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56275e1a6cf0 con 0x56275deeb1b0 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8db6255740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56275e2c82e0 con 0x56275def3390 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8db6255740 10 client.0 _ll_register_callbacks cb 0x56275e1cc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8dacc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8dad600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275df14070 0x56275e2d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275def3390 0x56275e2c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8dacc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55932/0 (socket says 172.21.15.32:55932) 2024-08-31T07:22:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8dacc00640 1 -- 172.21.15.32:0/467309614 learned_addr learned my addr 172.21.15.32:0/467309614 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8da7e00640 1 -- 172.21.15.32:0/467309614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 msgr2=0x56275e2c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8da7e00640 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8da7e00640 1 -- 172.21.15.32:0/467309614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275df14070 msgr2=0x56275e2d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8da7e00640 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275df14070 0x56275e2d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.149+0000 7f8da7e00640 1 -- 172.21.15.32:0/467309614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275e1e78b0 con 0x56275def3390 2024-08-31T07:22:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.150+0000 7f8da7e00640 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275def3390 0x56275e2c5c60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8d940e2eb0 tx=0x7f8d940096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac026000efd300eb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.150+0000 7f8d92000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:22:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.150+0000 7f8d92000640 1 -- 172.21.15.32:0/467309614 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d940e3340 con 0x56275def3390 2024-08-31T07:22:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.150+0000 7f8db6255740 1 -- 172.21.15.32:0/467309614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d980054f0 con 0x56275def3390 2024-08-31T07:22:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.150+0000 7f8d92000640 1 -- 172.21.15.32:0/467309614 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f8d940930f0 con 0x56275def3390 2024-08-31T07:22:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.150+0000 7f8d92000640 1 -- 172.21.15.32:0/467309614 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8d940b06c0 con 0x56275def3390 2024-08-31T07:22:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.150+0000 7f8db6255740 10 client.27607 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.150+0000 7f8db6255740 1 -- 172.21.15.32:0/467309614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56275e2d43b0 con 0x56275def3390 2024-08-31T07:22:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.151+0000 7f8d92000640 1 -- 172.21.15.32:0/467309614 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d980054f0 con 0x56275def3390 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.151+0000 7f8d92000640 1 -- 172.21.15.32:0/467309614 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f8d940a7870 con 0x56275def3390 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.151+0000 7f8db6255740 10 client.27607 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.151+0000 7f8db6255740 10 client.27607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.151+0000 7f8db6255740 20 client.27607 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.151+0000 7f8db6255740 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56275ded29f0 0x56275e2d78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.151+0000 7f8db6255740 4 client.27607 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.151+0000 7f8db6255740 1 -- 172.21.15.32:0/467309614 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56275e232700 con 0x56275ded29f0 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.152+0000 7f8dacc00640 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56275ded29f0 0x56275e2d78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.152+0000 7f8dacc00640 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56275ded29f0 0x56275e2d78b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84fa9959aceb7451 server_cookie=90b051ef0ce75c0b in_seq=0 out_seq=0 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.152+0000 7f8d92000640 10 client.27607 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.153+0000 7f8d92000640 1 -- 172.21.15.32:0/467309614 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56275e232700 con 0x56275ded29f0 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.153+0000 7f8d92000640 10 client.27607 handle_command_reply: tid=0 2024-08-31T07:22:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.153+0000 7f8db6255740 1 client.27607 shutdown 2024-08-31T07:22:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 1 -- 172.21.15.32:0/467309614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275def3390 msgr2=0x56275e2c5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275def3390 0x56275e2c5c60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8d940e2eb0 tx=0x7f8d940096a0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 1 -- 172.21.15.32:0/467309614 shutdown_connections 2024-08-31T07:22:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56275ded29f0 0x56275e2d78b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275df14070 0x56275e2d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275def3390 0x56275e2c5c60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 1 --2- 172.21.15.32:0/467309614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275deeb1b0 0x56275e2c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 1 -- 172.21.15.32:0/467309614 >> 172.21.15.32:0/467309614 conn(0x56275df08460 msgr2=0x56275e2bccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8d92000640 0 client.27607 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 1 -- 172.21.15.32:0/467309614 shutdown_connections 2024-08-31T07:22:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 1 -- 172.21.15.32:0/467309614 wait complete. 2024-08-31T07:22:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.154+0000 7f8db6255740 20 client.27607 trim_cache size 0 max 16384 2024-08-31T07:22:44.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.168+0000 7f8db6255740 1 Processor -- start 2024-08-31T07:22:44.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.168+0000 7f8db6255740 1 -- start start 2024-08-31T07:22:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8db6255740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275deeb1b0 0x56275e2d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8db6255740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275def3390 0x56275e2c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8db6255740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8db6255740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81b1450 con 0x56275deeb1b0 2024-08-31T07:22:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8db6255740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81b3080 con 0x56275def3390 2024-08-31T07:22:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8db6255740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81b1b20 con 0x56275df14070 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dad600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dacc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275deeb1b0 0x56275e2d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dad600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2c2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54120/0 (socket says 172.21.15.32:54120) 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dad600640 1 -- 172.21.15.32:0/4166324367 learned_addr learned my addr 172.21.15.32:0/4166324367 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dacc00640 1 --2- 172.21.15.32:0/4166324367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275deeb1b0 0x56275e2d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59818/0 (socket says 172.21.15.32:59818) 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8da7e00640 1 --2- 172.21.15.32:0/4166324367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275def3390 0x56275e2c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dad600640 1 -- 172.21.15.32:0/4166324367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275def3390 msgr2=0x56275e2c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dad600640 1 --2- 172.21.15.32:0/4166324367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275def3390 0x56275e2c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dad600640 1 -- 172.21.15.32:0/4166324367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275deeb1b0 msgr2=0x56275e2d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dad600640 1 --2- 172.21.15.32:0/4166324367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275deeb1b0 0x56275e2d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dad600640 1 -- 172.21.15.32:0/4166324367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275e2c67e0 con 0x56275df14070 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.169+0000 7f8dacc00640 1 --2- 172.21.15.32:0/4166324367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275deeb1b0 0x56275e2d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.170+0000 7f8dad600640 1 --2- 172.21.15.32:0/4166324367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2c2c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8da0010780 tx=0x7f8da0019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c2990efc930cf0b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.170+0000 7f8d9d200640 1 -- 172.21.15.32:0/4166324367 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da001c840 con 0x56275df14070 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.170+0000 7f8d9d200640 1 -- 172.21.15.32:0/4166324367 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da0028430 con 0x56275df14070 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.170+0000 7f8d9d200640 1 -- 172.21.15.32:0/4166324367 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da001b5f0 con 0x56275df14070 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.171+0000 7f8db6255740 1 -- 172.21.15.32:0/4166324367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 msgr2=0x56275e2c2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.171+0000 7f8db6255740 1 --2- 172.21.15.32:0/4166324367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2c2c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8da0010780 tx=0x7f8da0019000 comp rx=0 tx=0).stop 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.171+0000 7f8db6255740 1 -- 172.21.15.32:0/4166324367 shutdown_connections 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.171+0000 7f8db6255740 1 --2- 172.21.15.32:0/4166324367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275df14070 0x56275e2c2c50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.171+0000 7f8db6255740 1 --2- 172.21.15.32:0/4166324367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275def3390 0x56275e2c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.171+0000 7f8db6255740 1 --2- 172.21.15.32:0/4166324367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275deeb1b0 0x56275e2d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.171+0000 7f8db6255740 1 -- 172.21.15.32:0/4166324367 >> 172.21.15.32:0/4166324367 conn(0x56275df08460 msgr2=0x56275e2bec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.171+0000 7f8db6255740 1 -- 172.21.15.32:0/4166324367 shutdown_connections 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.171+0000 7f8db6255740 1 -- 172.21.15.32:0/4166324367 wait complete. 2024-08-31T07:22:44.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.171+0000 7f8db6255740 1 -- start start 2024-08-31T07:22:44.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.172+0000 7f8db6255740 1 Processor -- start 2024-08-31T07:22:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8db6255740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275deeb1b0 0x56275e2c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8db6255740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275def3390 0x56275e2ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8db6255740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275df14070 0x56275e341270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8db6255740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81b29b0 con 0x56275def3390 2024-08-31T07:22:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8db6255740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81b22b0 con 0x56275df14070 2024-08-31T07:22:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8db6255740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81af450 con 0x56275deeb1b0 2024-08-31T07:22:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8db6255740 10 client.0 _ll_register_callbacks cb 0x7f8d940ab740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8dacc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275deeb1b0 0x56275e2c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8dacc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275deeb1b0 0x56275e2c9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:54134/0 (socket says 172.21.15.32:54134) 2024-08-31T07:22:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8dacc00640 1 -- 172.21.15.32:0/3021113514 learned_addr learned my addr 172.21.15.32:0/3021113514 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275def3390 0x56275e2ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8dad600640 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275df14070 0x56275e341270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8dacc00640 1 -- 172.21.15.32:0/3021113514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275df14070 msgr2=0x56275e341270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8dacc00640 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275df14070 0x56275e341270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8dacc00640 1 -- 172.21.15.32:0/3021113514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275def3390 msgr2=0x56275e2ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8dacc00640 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275def3390 0x56275e2ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.173+0000 7f8dacc00640 1 -- 172.21.15.32:0/3021113514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275e3417b0 con 0x56275deeb1b0 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.174+0000 7f8dacc00640 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275deeb1b0 0x56275e2c9c00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005f70 tx=0x7f8d98009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b5a3a3953e1014d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.174+0000 7f8d92000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.174+0000 7f8d92000640 1 -- 172.21.15.32:0/3021113514 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9805c060 con 0x56275deeb1b0 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.174+0000 7f8d92000640 1 -- 172.21.15.32:0/3021113514 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f8d98004f00 con 0x56275deeb1b0 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.174+0000 7f8db6255740 1 -- 172.21.15.32:0/3021113514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8da0028430 con 0x56275deeb1b0 2024-08-31T07:22:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.174+0000 7f8d92000640 1 -- 172.21.15.32:0/3021113514 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8d98035040 con 0x56275deeb1b0 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.174+0000 7f8db6255740 10 client.27613 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.174+0000 7f8db6255740 1 -- 172.21.15.32:0/3021113514 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56275e342990 con 0x56275deeb1b0 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.175+0000 7f8d92000640 1 -- 172.21.15.32:0/3021113514 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da0028430 con 0x56275deeb1b0 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.175+0000 7f8d92000640 1 -- 172.21.15.32:0/3021113514 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f8d98003b10 con 0x56275deeb1b0 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.175+0000 7f8db6255740 10 client.27613 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.175+0000 7f8db6255740 10 client.27613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.175+0000 7f8db6255740 20 client.27613 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.175+0000 7f8db6255740 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56275dea7da0 0x56275e344d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.175+0000 7f8db6255740 4 client.27613 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.175+0000 7f8db6255740 1 -- 172.21.15.32:0/3021113514 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56275e22e1a0 con 0x56275dea7da0 2024-08-31T07:22:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.175+0000 7f8da7e00640 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56275dea7da0 0x56275e344d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8da7e00640 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56275dea7da0 0x56275e344d30 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd770d70b0563d4 server_cookie=8e31c1781064b216 in_seq=0 out_seq=0 2024-08-31T07:22:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8d92000640 10 client.27613 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8d92000640 1 -- 172.21.15.32:0/3021113514 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56275e22e1a0 con 0x56275dea7da0 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8d92000640 10 client.27613 handle_command_reply: tid=0 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8db6255740 1 client.27613 shutdown 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8db6255740 1 -- 172.21.15.32:0/3021113514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275deeb1b0 msgr2=0x56275e2c9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8db6255740 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275deeb1b0 0x56275e2c9c00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005f70 tx=0x7f8d98009260 comp rx=0 tx=0).stop 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8db6255740 1 -- 172.21.15.32:0/3021113514 shutdown_connections 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8db6255740 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56275dea7da0 0x56275e344d30 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8db6255740 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56275df14070 0x56275e341270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8db6255740 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56275def3390 0x56275e2ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8d92000640 0 client.27613 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8db6255740 1 --2- 172.21.15.32:0/3021113514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56275deeb1b0 0x56275e2c9c00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.176+0000 7f8db6255740 1 -- 172.21.15.32:0/3021113514 >> 172.21.15.32:0/3021113514 conn(0x56275df08460 msgr2=0x56275e2bab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.177+0000 7f8db6255740 1 -- 172.21.15.32:0/3021113514 shutdown_connections 2024-08-31T07:22:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.177+0000 7f8db6255740 1 -- 172.21.15.32:0/3021113514 wait complete. 2024-08-31T07:22:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.177+0000 7f8db6255740 20 client.27613 trim_cache size 0 max 16384 2024-08-31T07:22:44.181 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:22:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.177+0000 7f8dae000640 1 -- 172.21.15.32:0/4222504019 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d8c0b4d20 msgr2=0x7f8d8c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.177+0000 7f8dae000640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d8c0b4d20 0x7f8d8c0b7230 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8da0005fd0 tx=0x7f8da0009600 comp rx=0 tx=0).stop 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.177+0000 7f8dae000640 1 -- 172.21.15.32:0/4222504019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8da8109090 msgr2=0x7f8da819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.177+0000 7f8dae000640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8da8109090 0x7f8da819c850 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400e9f0 tx=0x7f8d9400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.178+0000 7f8dae000640 1 -- 172.21.15.32:0/4222504019 shutdown_connections 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.178+0000 7f8dae000640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8d8c0b4d20 0x7f8d8c0b7230 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.178+0000 7f8dae000640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8da81099e0 0x7f8da81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.178+0000 7f8dae000640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8da8109090 0x7f8da819c850 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.178+0000 7f8dae000640 1 --2- 172.21.15.32:0/4222504019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da8103750 0x7f8da819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.178+0000 7f8dae000640 1 -- 172.21.15.32:0/4222504019 >> 172.21.15.32:0/4222504019 conn(0x7f8da80ff1a0 msgr2=0x7f8da8108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.178+0000 7f8dae000640 1 -- 172.21.15.32:0/4222504019 shutdown_connections 2024-08-31T07:22:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:44.178+0000 7f8dae000640 1 -- 172.21.15.32:0/4222504019 wait complete. 2024-08-31T07:22:44.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 3.02 MB/sec execute 393 sec latency 3804.907 ms 2024-08-31T07:22:44.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:44 smithi032 ceph-mon[17252]: pgmap v675: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-08-31T07:22:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:44 smithi073 ceph-mon[21995]: pgmap v675: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-08-31T07:22:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:44 smithi084 ceph-mon[22387]: pgmap v675: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-08-31T07:22:45.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 3.01 MB/sec execute 394 sec latency 4804.960 ms 2024-08-31T07:22:46.082 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:22:46.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:22:46.460 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:46.766 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 3.00 MB/sec execute 395 sec latency 5805.004 ms 2024-08-31T07:22:46.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:46 smithi032 ceph-mon[17252]: pgmap v676: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-08-31T07:22:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:46 smithi073 ceph-mon[21995]: pgmap v676: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-08-31T07:22:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:46 smithi084 ceph-mon[22387]: pgmap v676: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-08-31T07:22:47.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.092+0000 7fe9c0800640 1 -- 172.21.15.32:0/620793564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9b8109240 msgr2=0x7fe9b8109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:47.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.092+0000 7fe9c0800640 1 --2- 172.21.15.32:0/620793564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9b8109240 0x7fe9b8109650 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009900 tx=0x7fe9a802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.092+0000 7fe9c0800640 1 -- 172.21.15.32:0/620793564 shutdown_connections 2024-08-31T07:22:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.092+0000 7fe9c0800640 1 --2- 172.21.15.32:0/620793564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9b8109b90 0x7fe9b81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.092+0000 7fe9c0800640 1 --2- 172.21.15.32:0/620793564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9b8109240 0x7fe9b8109650 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.092+0000 7fe9c0800640 1 --2- 172.21.15.32:0/620793564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9b8103900 0x7fe9b8103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.092+0000 7fe9c0800640 1 -- 172.21.15.32:0/620793564 >> 172.21.15.32:0/620793564 conn(0x7fe9b807b5f0 msgr2=0x7fe9b807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.092+0000 7fe9c0800640 1 -- 172.21.15.32:0/620793564 shutdown_connections 2024-08-31T07:22:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.092+0000 7fe9c0800640 1 -- 172.21.15.32:0/620793564 wait complete. 2024-08-31T07:22:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.093+0000 7fe9c0800640 1 Processor -- start 2024-08-31T07:22:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.093+0000 7fe9c0800640 1 -- start start 2024-08-31T07:22:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9c0800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9b8103900 0x7fe9b819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9c0800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9b8109240 0x7fe9b819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9c0800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9b8109b90 0x7fe9b81a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9c0800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8113850 con 0x7fe9b8109b90 2024-08-31T07:22:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9c0800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b81136d0 con 0x7fe9b8103900 2024-08-31T07:22:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9c0800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b81139d0 con 0x7fe9b8109240 2024-08-31T07:22:47.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9b8103900 0x7fe9b819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9be800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9b8109b90 0x7fe9b81a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9b8109240 0x7fe9b819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9b8103900 0x7fe9b819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54666/0 (socket says 172.21.15.32:54666) 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9bde00640 1 -- 172.21.15.32:0/353897513 learned_addr learned my addr 172.21.15.32:0/353897513 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9bde00640 1 -- 172.21.15.32:0/353897513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9b8109240 msgr2=0x7fe9b819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9bde00640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9b8109240 0x7fe9b819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9bde00640 1 -- 172.21.15.32:0/353897513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9b8109b90 msgr2=0x7fe9b81a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9bde00640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9b8109b90 0x7fe9b81a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.094+0000 7fe9bde00640 1 -- 172.21.15.32:0/353897513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b81a2470 con 0x7fe9b8103900 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.095+0000 7fe9bde00640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9b8103900 0x7fe9b819c500 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00e9f0 tx=0x7fe9ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.095+0000 7fe9b2a00640 1 -- 172.21.15.32:0/353897513 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac00cd80 con 0x7fe9b8103900 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.095+0000 7fe9c0800640 1 -- 172.21.15.32:0/353897513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b81a2760 con 0x7fe9b8103900 2024-08-31T07:22:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.095+0000 7fe9c0800640 1 -- 172.21.15.32:0/353897513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b81a8ec0 con 0x7fe9b8103900 2024-08-31T07:22:47.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.095+0000 7fe9b2a00640 1 -- 172.21.15.32:0/353897513 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9ac0108e0 con 0x7fe9b8103900 2024-08-31T07:22:47.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.095+0000 7fe9b2a00640 1 -- 172.21.15.32:0/353897513 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0188e0 con 0x7fe9b8103900 2024-08-31T07:22:47.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.098+0000 7fe9c73dd740 1 Processor -- start 2024-08-31T07:22:47.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.098+0000 7fe9c73dd740 1 -- start start 2024-08-31T07:22:47.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.099+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2e8a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.099+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2abb490 0x55a8d2e8ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.099+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2e8b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.099+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a8d2e00750 con 0x55a8d2ac3670 2024-08-31T07:22:47.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.099+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a8d2defce0 con 0x55a8d2abb490 2024-08-31T07:22:47.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.099+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a8d2df3c90 con 0x55a8d2ae4350 2024-08-31T07:22:47.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.099+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2e8b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.099+0000 7fe9be800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2e8a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.099+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2e8b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40430/0 (socket says 172.21.15.32:40430) 2024-08-31T07:22:47.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.099+0000 7fe9bd400640 1 -- 172.21.15.32:0/826036871 learned_addr learned my addr 172.21.15.32:0/826036871 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:47.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.100+0000 7fe9be800640 1 -- 172.21.15.32:0/826036871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2abb490 msgr2=0x55a8d2e8ab40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:22:47.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.100+0000 7fe9be800640 1 --2- 172.21.15.32:0/826036871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2abb490 0x55a8d2e8ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.100+0000 7fe9be800640 1 -- 172.21.15.32:0/826036871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 msgr2=0x55a8d2e8b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:47.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.100+0000 7fe9b2a00640 1 -- 172.21.15.32:0/353897513 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe9ac010430 con 0x7fe9b8103900 2024-08-31T07:22:47.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.100+0000 7fe9be800640 1 --2- 172.21.15.32:0/826036871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2e8b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.100+0000 7fe9be800640 1 -- 172.21.15.32:0/826036871 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d2b4f930 con 0x55a8d2ae4350 2024-08-31T07:22:47.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.100+0000 7fe9be800640 1 --2- 172.21.15.32:0/826036871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2e8a600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009f00 tx=0x7fe9b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5776538f54f7f64b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:47.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.100+0000 7fe9a7000640 1 -- 172.21.15.32:0/826036871 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4004d20 con 0x55a8d2ae4350 2024-08-31T07:22:47.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.100+0000 7fe9a7000640 1 -- 172.21.15.32:0/826036871 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b4004050 con 0x55a8d2ae4350 2024-08-31T07:22:47.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.100+0000 7fe9a7000640 1 -- 172.21.15.32:0/826036871 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4005280 con 0x55a8d2ae4350 2024-08-31T07:22:47.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.101+0000 7fe9c73dd740 1 -- 172.21.15.32:0/826036871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 msgr2=0x55a8d2e8a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:47.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.101+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/826036871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2e8a600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009f00 tx=0x7fe9b4011810 comp rx=0 tx=0).stop 2024-08-31T07:22:47.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.101+0000 7fe9b2a00640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9a00b4d20 0x7fe9a00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.101+0000 7fe9c73dd740 1 -- 172.21.15.32:0/826036871 shutdown_connections 2024-08-31T07:22:47.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.101+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/826036871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2e8b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.102+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/826036871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2abb490 0x55a8d2e8ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.102+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/826036871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2e8a600 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.102+0000 7fe9c73dd740 1 -- 172.21.15.32:0/826036871 >> 172.21.15.32:0/826036871 conn(0x55a8d2ad8740 msgr2=0x55a8d2e884f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:47.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.102+0000 7fe9b2a00640 1 -- 172.21.15.32:0/353897513 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fe9ac0e56d0 con 0x7fe9b8103900 2024-08-31T07:22:47.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.102+0000 7fe9be800640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9a00b4d20 0x7fe9a00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.102+0000 7fe9c73dd740 1 -- 172.21.15.32:0/826036871 shutdown_connections 2024-08-31T07:22:47.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.102+0000 7fe9c73dd740 1 -- 172.21.15.32:0/826036871 wait complete. 2024-08-31T07:22:47.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.102+0000 7fe9c73dd740 1 -- start start 2024-08-31T07:22:47.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.102+0000 7fe9be800640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9a00b4d20 0x7fe9a00b7230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009d40 tx=0x7fe9b4004990 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:47.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9c73dd740 1 Processor -- start 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2abb490 0x55a8d2e96110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 0x55a8d2e96650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2ea2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a8d2c52890 con 0x55a8d2abb490 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a8d2d76cd0 con 0x55a8d2ac3670 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a8d2e98de0 con 0x55a8d2ae4350 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9c73dd740 10 client.0 _ll_register_callbacks cb 0x55a8d2deaef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2abb490 0x55a8d2e96110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9be800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2ea2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2abb490 0x55a8d2e96110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40432/0 (socket says 172.21.15.32:40432) 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.103+0000 7fe9bde00640 1 -- 172.21.15.32:0/3223112855 learned_addr learned my addr 172.21.15.32:0/3223112855 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:47.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.104+0000 7fe9bd400640 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 0x55a8d2e96650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.104+0000 7fe9be800640 1 -- 172.21.15.32:0/3223112855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 msgr2=0x55a8d2e96650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.104+0000 7fe9be800640 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 0x55a8d2e96650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.104+0000 7fe9be800640 1 -- 172.21.15.32:0/3223112855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2abb490 msgr2=0x55a8d2e96110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.104+0000 7fe9be800640 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2abb490 0x55a8d2e96110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.104+0000 7fe9be800640 1 -- 172.21.15.32:0/3223112855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d2dc01b0 con 0x55a8d2ae4350 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.104+0000 7fe9bde00640 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2abb490 0x55a8d2e96110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.104+0000 7fe9be800640 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2ea2160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4005fd0 tx=0x7fe9b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a2ac1fbd1a117df server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.104+0000 7fe99b400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.105+0000 7fe99b400640 1 -- 172.21.15.32:0/3223112855 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4022830 con 0x55a8d2ae4350 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.105+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3223112855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b4004050 con 0x55a8d2ae4350 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.105+0000 7fe99b400640 1 -- 172.21.15.32:0/3223112855 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fe9b40229d0 con 0x55a8d2ae4350 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.105+0000 7fe99b400640 1 -- 172.21.15.32:0/3223112855 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe9b403e050 con 0x55a8d2ae4350 2024-08-31T07:22:47.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.105+0000 7fe9c73dd740 10 client.27625 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:47.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.105+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3223112855 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a8d2ea4d70 con 0x55a8d2ae4350 2024-08-31T07:22:47.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.105+0000 7fe99b400640 1 -- 172.21.15.32:0/3223112855 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b4004050 con 0x55a8d2ae4350 2024-08-31T07:22:47.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.105+0000 7fe99b400640 1 -- 172.21.15.32:0/3223112855 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fe9b4030ad0 con 0x55a8d2ae4350 2024-08-31T07:22:47.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.105+0000 7fe9c73dd740 10 client.27625 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:47.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.105+0000 7fe9c73dd740 10 client.27625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:22:47.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.106+0000 7fe9c73dd740 20 client.27625 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:47.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.106+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a8d2aa2cd0 0x55a8d2ea8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.106+0000 7fe9c73dd740 4 client.27625 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:22:47.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.106+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3223112855 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8d2e026a0 con 0x55a8d2aa2cd0 2024-08-31T07:22:47.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.106+0000 7fe9bde00640 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a8d2aa2cd0 0x55a8d2ea8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.106+0000 7fe99b400640 10 client.27625 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.106+0000 7fe9bde00640 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a8d2aa2cd0 0x55a8d2ea8130 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9385382b5042e1f0 server_cookie=c0e5d7ed5ff207dc in_seq=0 out_seq=0 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.107+0000 7fe99b400640 1 -- 172.21.15.32:0/3223112855 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8d2e026a0 con 0x55a8d2aa2cd0 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.107+0000 7fe99b400640 10 client.27625 handle_command_reply: tid=0 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.107+0000 7fe9c73dd740 1 client.27625 shutdown 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.107+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3223112855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 msgr2=0x55a8d2ea2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.107+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2ea2160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4005fd0 tx=0x7fe9b4011810 comp rx=0 tx=0).stop 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.108+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3223112855 shutdown_connections 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.108+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a8d2aa2cd0 0x55a8d2ea8130 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.108+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2ea2160 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.108+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 0x55a8d2e96650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.108+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3223112855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2abb490 0x55a8d2e96110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.108+0000 7fe99b400640 0 client.27625 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.108+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3223112855 >> 172.21.15.32:0/3223112855 conn(0x55a8d2ad8740 msgr2=0x55a8d2e8d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.108+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3223112855 shutdown_connections 2024-08-31T07:22:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.108+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3223112855 wait complete. 2024-08-31T07:22:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.108+0000 7fe9c73dd740 20 client.27625 trim_cache size 0 max 16384 2024-08-31T07:22:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.126+0000 7fe9c73dd740 1 Processor -- start 2024-08-31T07:22:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.126+0000 7fe9c73dd740 1 -- start start 2024-08-31T07:22:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2abb490 0x55a8d2e97b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2ea2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2ea25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c00e680 con 0x55a8d2ac3670 2024-08-31T07:22:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c0102b0 con 0x55a8d2abb490 2024-08-31T07:22:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c00ed50 con 0x55a8d2ae4350 2024-08-31T07:22:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2ea2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9be800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2ea25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2ea2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40440/0 (socket says 172.21.15.32:40440) 2024-08-31T07:22:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9bd400640 1 -- 172.21.15.32:0/2064955581 learned_addr learned my addr 172.21.15.32:0/2064955581 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.127+0000 7fe9bde00640 1 --2- 172.21.15.32:0/2064955581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2abb490 0x55a8d2e97b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.128+0000 7fe9bd400640 1 -- 172.21.15.32:0/2064955581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 msgr2=0x55a8d2ea25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.128+0000 7fe9bd400640 1 --2- 172.21.15.32:0/2064955581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2ea25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.128+0000 7fe9bd400640 1 -- 172.21.15.32:0/2064955581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2abb490 msgr2=0x55a8d2e97b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.128+0000 7fe9bd400640 1 --2- 172.21.15.32:0/2064955581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2abb490 0x55a8d2e97b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.128+0000 7fe9bd400640 1 -- 172.21.15.32:0/2064955581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d2e94e70 con 0x55a8d2ac3670 2024-08-31T07:22:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.128+0000 7fe9bd400640 1 --2- 172.21.15.32:0/2064955581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2ea2060 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009900 tx=0x7fe9a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e4b8e86abdd9c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.128+0000 7fe9a6600640 1 -- 172.21.15.32:0/2064955581 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8035040 con 0x55a8d2ac3670 2024-08-31T07:22:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.128+0000 7fe9a6600640 1 -- 172.21.15.32:0/2064955581 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a80316a0 con 0x55a8d2ac3670 2024-08-31T07:22:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.128+0000 7fe9a6600640 1 -- 172.21.15.32:0/2064955581 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8004750 con 0x55a8d2ac3670 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.129+0000 7fe9c73dd740 1 -- 172.21.15.32:0/2064955581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 msgr2=0x55a8d2ea2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.129+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/2064955581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2ea2060 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009900 tx=0x7fe9a8000bc0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.129+0000 7fe9c73dd740 1 -- 172.21.15.32:0/2064955581 shutdown_connections 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.129+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/2064955581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2ae4350 0x55a8d2ea25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.129+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/2064955581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ac3670 0x55a8d2ea2060 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.129+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/2064955581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2abb490 0x55a8d2e97b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.129+0000 7fe9c73dd740 1 -- 172.21.15.32:0/2064955581 >> 172.21.15.32:0/2064955581 conn(0x55a8d2ad8740 msgr2=0x55a8d2e97430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.129+0000 7fe9c73dd740 1 -- 172.21.15.32:0/2064955581 shutdown_connections 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.129+0000 7fe9c73dd740 1 -- 172.21.15.32:0/2064955581 wait complete. 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.130+0000 7fe9c73dd740 1 -- start start 2024-08-31T07:22:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.130+0000 7fe9c73dd740 1 Processor -- start 2024-08-31T07:22:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.130+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2abb490 0x55a8d2e9a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.130+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 0x55a8d2e9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.130+0000 7fe9c73dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ae4350 0x55a8d2f0ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.130+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c00fbe0 con 0x55a8d2ae4350 2024-08-31T07:22:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.130+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c00f4e0 con 0x55a8d2ac3670 2024-08-31T07:22:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.130+0000 7fe9c73dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c00c680 con 0x55a8d2abb490 2024-08-31T07:22:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9c73dd740 10 client.0 _ll_register_callbacks cb 0x7fe9b40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9be800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ae4350 0x55a8d2f0ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2abb490 0x55a8d2e9a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 0x55a8d2e9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9be800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ae4350 0x55a8d2f0ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40448/0 (socket says 172.21.15.32:40448) 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9be800640 1 -- 172.21.15.32:0/3792149475 learned_addr learned my addr 172.21.15.32:0/3792149475 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9bd400640 1 -- 172.21.15.32:0/3792149475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2abb490 msgr2=0x55a8d2e9a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9bd400640 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2abb490 0x55a8d2e9a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9bd400640 1 -- 172.21.15.32:0/3792149475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ae4350 msgr2=0x55a8d2f0ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9bd400640 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ae4350 0x55a8d2f0ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9bd400640 1 -- 172.21.15.32:0/3792149475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d2f0c0e0 con 0x55a8d2ac3670 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.131+0000 7fe9be800640 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ae4350 0x55a8d2f0ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.132+0000 7fe9bd400640 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 0x55a8d2e9af80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8032fd0 tx=0x7fe9a8032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1868a1889912775 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.132+0000 7fe99b400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.132+0000 7fe99b400640 1 -- 172.21.15.32:0/3792149475 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8035040 con 0x55a8d2ac3670 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.132+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3792149475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a80316a0 con 0x55a8d2ac3670 2024-08-31T07:22:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.132+0000 7fe99b400640 1 -- 172.21.15.32:0/3792149475 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fe9a8004040 con 0x55a8d2ac3670 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.132+0000 7fe99b400640 1 -- 172.21.15.32:0/3792149475 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe9a8004240 con 0x55a8d2ac3670 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.132+0000 7fe9c73dd740 10 client.27629 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.132+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3792149475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a8d2f13460 con 0x55a8d2ac3670 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.132+0000 7fe99b400640 1 -- 172.21.15.32:0/3792149475 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a80316a0 con 0x55a8d2ac3670 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.133+0000 7fe99b400640 1 -- 172.21.15.32:0/3792149475 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fe9a8004750 con 0x55a8d2ac3670 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.133+0000 7fe9c73dd740 10 client.27629 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.133+0000 7fe9c73dd740 10 client.27629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.133+0000 7fe9c73dd740 20 client.27629 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.133+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a8d2a78080 0x55a8d2f15c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.133+0000 7fe9c73dd740 4 client.27629 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.133+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3792149475 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a8d2dfe140 con 0x55a8d2a78080 2024-08-31T07:22:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.133+0000 7fe9bde00640 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a8d2a78080 0x55a8d2f15c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.134+0000 7fe9bde00640 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a8d2a78080 0x55a8d2f15c00 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c124e29bb4bdda7 server_cookie=dfa9d3280e28391f in_seq=0 out_seq=0 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.134+0000 7fe99b400640 10 client.27629 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.135+0000 7fe99b400640 1 -- 172.21.15.32:0/3792149475 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a8d2dfe140 con 0x55a8d2a78080 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.135+0000 7fe99b400640 10 client.27629 handle_command_reply: tid=0 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.135+0000 7fe9c73dd740 1 client.27629 shutdown 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.135+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3792149475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 msgr2=0x55a8d2e9af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.135+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 0x55a8d2e9af80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8032fd0 tx=0x7fe9a8032b20 comp rx=0 tx=0).stop 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3792149475 shutdown_connections 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a8d2a78080 0x55a8d2f15c00 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8d2ae4350 0x55a8d2f0ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8d2ac3670 0x55a8d2e9af80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c73dd740 1 --2- 172.21.15.32:0/3792149475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8d2abb490 0x55a8d2e9a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3792149475 >> 172.21.15.32:0/3792149475 conn(0x55a8d2ad8740 msgr2=0x55a8d2e97b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe99b400640 0 client.27629 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3792149475 shutdown_connections 2024-08-31T07:22:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c73dd740 1 -- 172.21.15.32:0/3792149475 wait complete. 2024-08-31T07:22:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c73dd740 20 client.27629 trim_cache size 0 max 16384 2024-08-31T07:22:47.140 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:22:47.140 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:22:47.140 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "a047226e-5dfa-4aa9-bb91-ffd989a87ee6", 2024-08-31T07:22:47.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:22:47.140 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:22:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c0800640 1 -- 172.21.15.32:0/353897513 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9a00b4d20 msgr2=0x7fe9a00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.136+0000 7fe9c0800640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9a00b4d20 0x7fe9a00b7230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009d40 tx=0x7fe9b4004990 comp rx=0 tx=0).stop 2024-08-31T07:22:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.137+0000 7fe9c0800640 1 -- 172.21.15.32:0/353897513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9b8103900 msgr2=0x7fe9b819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.137+0000 7fe9c0800640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9b8103900 0x7fe9b819c500 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00e9f0 tx=0x7fe9ac00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.137+0000 7fe9c0800640 1 -- 172.21.15.32:0/353897513 shutdown_connections 2024-08-31T07:22:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.137+0000 7fe9c0800640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe9a00b4d20 0x7fe9a00b7230 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.137+0000 7fe9c0800640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9b8109b90 0x7fe9b81a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.137+0000 7fe9c0800640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9b8109240 0x7fe9b819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.137+0000 7fe9c0800640 1 --2- 172.21.15.32:0/353897513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9b8103900 0x7fe9b819c500 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.137+0000 7fe9c0800640 1 -- 172.21.15.32:0/353897513 >> 172.21.15.32:0/353897513 conn(0x7fe9b807b5f0 msgr2=0x7fe9b8104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.137+0000 7fe9c0800640 1 -- 172.21.15.32:0/353897513 shutdown_connections 2024-08-31T07:22:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:47.137+0000 7fe9c0800640 1 -- 172.21.15.32:0/353897513 wait complete. 2024-08-31T07:22:47.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 2.99 MB/sec execute 396 sec latency 6805.099 ms 2024-08-31T07:22:47.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:22:47.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:22:47.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:47.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:22:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:22:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:47.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:47.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:22:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:22:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:47.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:22:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:48.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 2.99 MB/sec execute 397 sec latency 7805.176 ms 2024-08-31T07:22:48.833 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: pgmap v677: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-08-31T07:22:48.833 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:22:48.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:22:48.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:22:48.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:22:48.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:22:48.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.834 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:48 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: pgmap v677: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-08-31T07:22:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:22:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:22:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:22:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:22:48.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:22:48.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:48 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: pgmap v677: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-08-31T07:22:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:22:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:22:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:22:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:22:48.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:22:48.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:48.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:48 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:22:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.293+0000 7fbb4e000640 1 -- 172.21.15.32:0/2228643633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb48078030 msgr2=0x7fbb48113dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.293+0000 7fbb4e000640 1 --2- 172.21.15.32:0/2228643633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb48078030 0x7fbb48113dd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009900 tx=0x7fbb3402a410 comp rx=0 tx=0).stop 2024-08-31T07:22:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.293+0000 7fbb4e000640 1 -- 172.21.15.32:0/2228643633 shutdown_connections 2024-08-31T07:22:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.293+0000 7fbb4e000640 1 --2- 172.21.15.32:0/2228643633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb48114310 0x7fbb48116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.293+0000 7fbb4e000640 1 --2- 172.21.15.32:0/2228643633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb48078030 0x7fbb48113dd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.293+0000 7fbb4e000640 1 --2- 172.21.15.32:0/2228643633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb480776e0 0x7fbb48077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.293+0000 7fbb4e000640 1 -- 172.21.15.32:0/2228643633 >> 172.21.15.32:0/2228643633 conn(0x7fbb48101420 msgr2=0x7fbb48103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:49.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.293+0000 7fbb4e000640 1 -- 172.21.15.32:0/2228643633 shutdown_connections 2024-08-31T07:22:49.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.293+0000 7fbb4e000640 1 -- 172.21.15.32:0/2228643633 wait complete. 2024-08-31T07:22:49.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.294+0000 7fbb4e000640 1 Processor -- start 2024-08-31T07:22:49.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.294+0000 7fbb4e000640 1 -- start start 2024-08-31T07:22:49.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb4e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb480776e0 0x7fbb481a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb4e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48078030 0x7fbb481a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb4e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb48114310 0x7fbb481a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb4e000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48119cc0 con 0x7fbb480776e0 2024-08-31T07:22:49.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb4e000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48119b40 con 0x7fbb48114310 2024-08-31T07:22:49.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb4e000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48119e40 con 0x7fbb48078030 2024-08-31T07:22:49.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb47e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48078030 0x7fbb481a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb4cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb480776e0 0x7fbb481a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb47e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48078030 0x7fbb481a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:48412/0 (socket says 172.21.15.32:48412) 2024-08-31T07:22:49.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb4cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb480776e0 0x7fbb481a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40476/0 (socket says 172.21.15.32:40476) 2024-08-31T07:22:49.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb47e00640 1 -- 172.21.15.32:0/2226499363 learned_addr learned my addr 172.21.15.32:0/2226499363 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:49.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb4d600640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb48114310 0x7fbb481a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb47e00640 1 -- 172.21.15.32:0/2226499363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb48114310 msgr2=0x7fbb481a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:49.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.295+0000 7fbb47e00640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb48114310 0x7fbb481a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.296+0000 7fbb47e00640 1 -- 172.21.15.32:0/2226499363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb480776e0 msgr2=0x7fbb481a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:49.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.296+0000 7fbb47e00640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb480776e0 0x7fbb481a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.296+0000 7fbb47e00640 1 -- 172.21.15.32:0/2226499363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb481a65e0 con 0x7fbb48078030 2024-08-31T07:22:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.296+0000 7fbb47e00640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48078030 0x7fbb481a0c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbb34005ab0 tx=0x7fbb34003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.296+0000 7fbb45600640 1 -- 172.21.15.32:0/2226499363 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3403f070 con 0x7fbb48078030 2024-08-31T07:22:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.296+0000 7fbb4e000640 1 -- 172.21.15.32:0/2226499363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb481a6870 con 0x7fbb48078030 2024-08-31T07:22:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.296+0000 7fbb45600640 1 -- 172.21.15.32:0/2226499363 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb34031b50 con 0x7fbb48078030 2024-08-31T07:22:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.296+0000 7fbb45600640 1 -- 172.21.15.32:0/2226499363 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34035030 con 0x7fbb48078030 2024-08-31T07:22:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.296+0000 7fbb4e000640 1 -- 172.21.15.32:0/2226499363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb481ad030 con 0x7fbb48078030 2024-08-31T07:22:49.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.299+0000 7fbb562a7740 1 Processor -- start 2024-08-31T07:22:49.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.299+0000 7fbb562a7740 1 -- start start 2024-08-31T07:22:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.300+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9352a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.300+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 0x55a6b9352f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.300+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f8c390 0x55a6b93534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.300+0000 7fbb562a7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a6b92c9770 con 0x55a6b8fad070 2024-08-31T07:22:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.300+0000 7fbb562a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a6b92b8d00 con 0x55a6b8f8c390 2024-08-31T07:22:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.300+0000 7fbb562a7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a6b92bccb0 con 0x55a6b8f841b0 2024-08-31T07:22:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.300+0000 7fbb4d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9352a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.300+0000 7fbb4d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9352a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40488/0 (socket says 172.21.15.32:40488) 2024-08-31T07:22:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.300+0000 7fbb4cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 0x55a6b9352f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.300+0000 7fbb4d600640 1 -- 172.21.15.32:0/2735663332 learned_addr learned my addr 172.21.15.32:0/2735663332 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb45600640 1 -- 172.21.15.32:0/2226499363 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbb34042070 con 0x7fbb48078030 2024-08-31T07:22:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb4cc00640 1 -- 172.21.15.32:0/2735663332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f8c390 msgr2=0x55a6b93534b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:22:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb4cc00640 1 --2- 172.21.15.32:0/2735663332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f8c390 0x55a6b93534b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb4cc00640 1 -- 172.21.15.32:0/2735663332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 msgr2=0x55a6b9352a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb4cc00640 1 --2- 172.21.15.32:0/2735663332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9352a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb4cc00640 1 -- 172.21.15.32:0/2735663332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6b9018650 con 0x55a6b8f841b0 2024-08-31T07:22:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb4d600640 1 --2- 172.21.15.32:0/2735663332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9352a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb4cc00640 1 --2- 172.21.15.32:0/2735663332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 0x55a6b9352f70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009c20 tx=0x7fbb3c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd94f1daa4da854 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:49.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb39c00640 1 -- 172.21.15.32:0/2735663332 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c004050 con 0x55a6b8f841b0 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb39c00640 1 -- 172.21.15.32:0/2735663332 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb3c0054d0 con 0x55a6b8f841b0 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.301+0000 7fbb39c00640 1 -- 172.21.15.32:0/2735663332 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c004ca0 con 0x55a6b8f841b0 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.302+0000 7fbb562a7740 1 -- 172.21.15.32:0/2735663332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 msgr2=0x55a6b9352f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.302+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2735663332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 0x55a6b9352f70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009c20 tx=0x7fbb3c011810 comp rx=0 tx=0).stop 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.302+0000 7fbb562a7740 1 -- 172.21.15.32:0/2735663332 shutdown_connections 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.302+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2735663332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f8c390 0x55a6b93534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.302+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2735663332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 0x55a6b9352f70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.302+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2735663332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9352a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.302+0000 7fbb562a7740 1 -- 172.21.15.32:0/2735663332 >> 172.21.15.32:0/2735663332 conn(0x55a6b8fa1460 msgr2=0x55a6b9350920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.302+0000 7fbb562a7740 1 -- 172.21.15.32:0/2735663332 shutdown_connections 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.302+0000 7fbb562a7740 1 -- 172.21.15.32:0/2735663332 wait complete. 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.303+0000 7fbb45600640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbb300b4bc0 0x7fbb300b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.303+0000 7fbb562a7740 1 -- start start 2024-08-31T07:22:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.303+0000 7fbb45600640 1 -- 172.21.15.32:0/2226499363 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fbb34105790 con 0x7fbb48078030 2024-08-31T07:22:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.303+0000 7fbb4d600640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbb300b4bc0 0x7fbb300b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.303+0000 7fbb562a7740 1 Processor -- start 2024-08-31T07:22:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.304+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 0x55a6b935e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.304+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b935ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.304+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b936a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.304+0000 7fbb562a7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a6b911b700 con 0x55a6b8fad070 2024-08-31T07:22:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.304+0000 7fbb562a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a6b923fcb0 con 0x55a6b8f841b0 2024-08-31T07:22:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.304+0000 7fbb562a7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a6b9361210 con 0x55a6b8f8c390 2024-08-31T07:22:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.304+0000 7fbb4d600640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbb300b4bc0 0x7fbb300b70d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbb40005fd0 tx=0x7fbb40009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.304+0000 7fbb562a7740 10 client.0 _ll_register_callbacks cb 0x55a6b9265430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb4cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 0x55a6b935e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb4d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b936a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb47e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b935ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb47e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b935ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:48436/0 (socket says 172.21.15.32:48436) 2024-08-31T07:22:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb47e00640 1 -- 172.21.15.32:0/2274009788 learned_addr learned my addr 172.21.15.32:0/2274009788 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb4cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 0x55a6b935e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54722/0 (socket says 172.21.15.32:54722) 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb47e00640 1 -- 172.21.15.32:0/2274009788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 msgr2=0x55a6b935e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb47e00640 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 0x55a6b935e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb47e00640 1 -- 172.21.15.32:0/2274009788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 msgr2=0x55a6b936a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb47e00640 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b936a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb47e00640 1 -- 172.21.15.32:0/2274009788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6b9280870 con 0x55a6b8f8c390 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb4d600640 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b936a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb47e00640 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b935ec10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb34006010 tx=0x7fbb340cb930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42cf4be763c470e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.305+0000 7fbb2e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.306+0000 7fbb2e000640 1 -- 172.21.15.32:0/2274009788 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34049070 con 0x55a6b8f8c390 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.306+0000 7fbb562a7740 1 -- 172.21.15.32:0/2274009788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb3c0054d0 con 0x55a6b8f8c390 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.306+0000 7fbb2e000640 1 -- 172.21.15.32:0/2274009788 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fbb34103150 con 0x55a6b8f8c390 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.306+0000 7fbb2e000640 1 -- 172.21.15.32:0/2274009788 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbb340d06e0 con 0x55a6b8f8c390 2024-08-31T07:22:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.306+0000 7fbb562a7740 10 client.27643 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.306+0000 7fbb562a7740 1 -- 172.21.15.32:0/2274009788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a6b936d2a0 con 0x55a6b8f8c390 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.306+0000 7fbb2e000640 1 -- 172.21.15.32:0/2274009788 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb3c0054d0 con 0x55a6b8f8c390 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.306+0000 7fbb2e000640 1 -- 172.21.15.32:0/2274009788 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fbb3404c060 con 0x55a6b8f8c390 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.307+0000 7fbb562a7740 10 client.27643 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.307+0000 7fbb562a7740 10 client.27643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.307+0000 7fbb562a7740 20 client.27643 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.307+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a6b8f6b9f0 0x55a6b93706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.307+0000 7fbb562a7740 4 client.27643 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.307+0000 7fbb562a7740 1 -- 172.21.15.32:0/2274009788 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6b92cb6c0 con 0x55a6b8f6b9f0 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.307+0000 7fbb4cc00640 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a6b8f6b9f0 0x55a6b93706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.308+0000 7fbb4cc00640 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a6b8f6b9f0 0x55a6b93706c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871f542f1724b8bb server_cookie=1f5d64c75b9199fe in_seq=0 out_seq=0 2024-08-31T07:22:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.308+0000 7fbb2e000640 10 client.27643 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb2e000640 1 -- 172.21.15.32:0/2274009788 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6b92cb6c0 con 0x55a6b8f6b9f0 2024-08-31T07:22:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb2e000640 10 client.27643 handle_command_reply: tid=0 2024-08-31T07:22:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 client.27643 shutdown 2024-08-31T07:22:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 -- 172.21.15.32:0/2274009788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 msgr2=0x55a6b935ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b935ec10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb34006010 tx=0x7fbb340cb930 comp rx=0 tx=0).stop 2024-08-31T07:22:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 -- 172.21.15.32:0/2274009788 shutdown_connections 2024-08-31T07:22:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a6b8f6b9f0 0x55a6b93706c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b936a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b935ec10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2274009788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 0x55a6b935e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb2e000640 0 client.27643 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 -- 172.21.15.32:0/2274009788 >> 172.21.15.32:0/2274009788 conn(0x55a6b8fa1460 msgr2=0x55a6b9355c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 -- 172.21.15.32:0/2274009788 shutdown_connections 2024-08-31T07:22:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 1 -- 172.21.15.32:0/2274009788 wait complete. 2024-08-31T07:22:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.309+0000 7fbb562a7740 20 client.27643 trim_cache size 0 max 16384 2024-08-31T07:22:49.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.328+0000 7fbb562a7740 1 Processor -- start 2024-08-31T07:22:49.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.328+0000 7fbb562a7740 1 -- start start 2024-08-31T07:22:49.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 0x55a6b9358250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b934ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9350450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb562a7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b57b0 con 0x55a6b8fad070 2024-08-31T07:22:49.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb562a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b73e0 con 0x55a6b8f841b0 2024-08-31T07:22:49.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb562a7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b5e80 con 0x55a6b8f8c390 2024-08-31T07:22:49.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb4d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9350450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb47e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b934ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb4cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 0x55a6b9358250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb47e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b934ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:48444/0 (socket says 172.21.15.32:48444) 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb4d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9350450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40498/0 (socket says 172.21.15.32:40498) 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.329+0000 7fbb47e00640 1 -- 172.21.15.32:0/2452965946 learned_addr learned my addr 172.21.15.32:0/2452965946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.330+0000 7fbb47e00640 1 -- 172.21.15.32:0/2452965946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 msgr2=0x55a6b9358250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.330+0000 7fbb47e00640 1 --2- 172.21.15.32:0/2452965946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 0x55a6b9358250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.330+0000 7fbb47e00640 1 -- 172.21.15.32:0/2452965946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 msgr2=0x55a6b9350450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.330+0000 7fbb47e00640 1 --2- 172.21.15.32:0/2452965946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9350450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.330+0000 7fbb47e00640 1 -- 172.21.15.32:0/2452965946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6b935d410 con 0x55a6b8f8c390 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.330+0000 7fbb47e00640 1 --2- 172.21.15.32:0/2452965946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b934ff10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbb340d0650 tx=0x7fbb340cbda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caaea48574af3f7b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.330+0000 7fbb39200640 1 -- 172.21.15.32:0/2452965946 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34049020 con 0x55a6b8f8c390 2024-08-31T07:22:49.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.330+0000 7fbb39200640 1 -- 172.21.15.32:0/2452965946 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb340d0330 con 0x55a6b8f8c390 2024-08-31T07:22:49.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.330+0000 7fbb39200640 1 -- 172.21.15.32:0/2452965946 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb340d4d00 con 0x55a6b8f8c390 2024-08-31T07:22:49.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.331+0000 7fbb562a7740 1 -- 172.21.15.32:0/2452965946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 msgr2=0x55a6b934ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.331+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2452965946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b934ff10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbb340d0650 tx=0x7fbb340cbda0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.331+0000 7fbb562a7740 1 -- 172.21.15.32:0/2452965946 shutdown_connections 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.331+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2452965946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8fad070 0x55a6b9350450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.331+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2452965946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f8c390 0x55a6b934ff10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.331+0000 7fbb562a7740 1 --2- 172.21.15.32:0/2452965946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8f841b0 0x55a6b9358250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.331+0000 7fbb562a7740 1 -- 172.21.15.32:0/2452965946 >> 172.21.15.32:0/2452965946 conn(0x55a6b8fa1460 msgr2=0x55a6b9357b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.331+0000 7fbb562a7740 1 -- 172.21.15.32:0/2452965946 shutdown_connections 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.331+0000 7fbb562a7740 1 -- 172.21.15.32:0/2452965946 wait complete. 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.332+0000 7fbb562a7740 1 -- start start 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.332+0000 7fbb562a7740 1 Processor -- start 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 0x55a6b9362dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8f8c390 0x55a6b9363370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb562a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8fad070 0x55a6b93d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb562a7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b6d10 con 0x55a6b8f8c390 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb562a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b6610 con 0x55a6b8fad070 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb562a7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b37b0 con 0x55a6b8f841b0 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb562a7740 10 client.0 _ll_register_callbacks cb 0x7fbb340cbcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb4d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8fad070 0x55a6b93d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb4d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8fad070 0x55a6b93d4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54742/0 (socket says 172.21.15.32:54742) 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb4d600640 1 -- 172.21.15.32:0/820982939 learned_addr learned my addr 172.21.15.32:0/820982939 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb4cc00640 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 0x55a6b9362dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb47e00640 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8f8c390 0x55a6b9363370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb4d600640 1 -- 172.21.15.32:0/820982939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 msgr2=0x55a6b9362dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb4d600640 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 0x55a6b9362dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb4d600640 1 -- 172.21.15.32:0/820982939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8f8c390 msgr2=0x55a6b9363370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb4d600640 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8f8c390 0x55a6b9363370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.333+0000 7fbb4d600640 1 -- 172.21.15.32:0/820982939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6b9370a30 con 0x55a6b8fad070 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.334+0000 7fbb4d600640 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8fad070 0x55a6b93d4050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbb4001c560 tx=0x7fbb40004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed4475ea9e3b6be server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.334+0000 7fbb2e000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.334+0000 7fbb2e000640 1 -- 172.21.15.32:0/820982939 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40029410 con 0x55a6b8fad070 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.334+0000 7fbb2e000640 1 -- 172.21.15.32:0/820982939 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fbb400295b0 con 0x55a6b8fad070 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.334+0000 7fbb562a7740 1 -- 172.21.15.32:0/820982939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb340d0330 con 0x55a6b8fad070 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.334+0000 7fbb2e000640 1 -- 172.21.15.32:0/820982939 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbb40038050 con 0x55a6b8fad070 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.334+0000 7fbb562a7740 10 client.27644 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.334+0000 7fbb562a7740 1 -- 172.21.15.32:0/820982939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a6b93db890 con 0x55a6b8fad070 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.335+0000 7fbb2e000640 1 -- 172.21.15.32:0/820982939 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb340d0330 con 0x55a6b8fad070 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.335+0000 7fbb2e000640 1 -- 172.21.15.32:0/820982939 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fbb40031990 con 0x55a6b8fad070 2024-08-31T07:22:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.335+0000 7fbb562a7740 10 client.27644 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.335+0000 7fbb562a7740 10 client.27644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:22:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.335+0000 7fbb562a7740 20 client.27644 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.335+0000 7fbb562a7740 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a6b8f40da0 0x55a6b93de0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.335+0000 7fbb562a7740 4 client.27644 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:22:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.335+0000 7fbb562a7740 1 -- 172.21.15.32:0/820982939 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6b92c7160 con 0x55a6b8f40da0 2024-08-31T07:22:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.335+0000 7fbb4cc00640 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a6b8f40da0 0x55a6b93de0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.336+0000 7fbb4cc00640 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a6b8f40da0 0x55a6b93de0d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c6192a5224079b3 server_cookie=f88bdaf7697dc320 in_seq=0 out_seq=0 2024-08-31T07:22:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.336+0000 7fbb2e000640 10 client.27644 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.336+0000 7fbb2e000640 1 -- 172.21.15.32:0/820982939 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x55a6b92c7160 con 0x55a6b8f40da0 2024-08-31T07:22:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.336+0000 7fbb2e000640 10 client.27644 handle_command_reply: tid=0 2024-08-31T07:22:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.336+0000 7fbb562a7740 1 client.27644 shutdown 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.336+0000 7fbb562a7740 1 -- 172.21.15.32:0/820982939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8fad070 msgr2=0x55a6b93d4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.336+0000 7fbb562a7740 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8fad070 0x55a6b93d4050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbb4001c560 tx=0x7fbb40004ab0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.336+0000 7fbb562a7740 1 -- 172.21.15.32:0/820982939 shutdown_connections 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.336+0000 7fbb562a7740 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a6b8f40da0 0x55a6b93de0d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.337+0000 7fbb562a7740 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a6b8fad070 0x55a6b93d4050 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.337+0000 7fbb562a7740 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a6b8f8c390 0x55a6b9363370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.337+0000 7fbb562a7740 1 --2- 172.21.15.32:0/820982939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a6b8f841b0 0x55a6b9362dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.337+0000 7fbb562a7740 1 -- 172.21.15.32:0/820982939 >> 172.21.15.32:0/820982939 conn(0x55a6b8fa1460 msgr2=0x55a6b9353960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.337+0000 7fbb2e000640 0 client.27644 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.337+0000 7fbb562a7740 1 -- 172.21.15.32:0/820982939 shutdown_connections 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.337+0000 7fbb562a7740 1 -- 172.21.15.32:0/820982939 wait complete. 2024-08-31T07:22:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.337+0000 7fbb562a7740 20 client.27644 trim_cache size 0 max 16384 2024-08-31T07:22:49.342 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:22:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:22:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:22:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:22:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "a047226e-5dfa-4aa9-bb91-ffd989a87ee6", 2024-08-31T07:22:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:22:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:22:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:22:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:22:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:22:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:22:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:22:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:22:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:22:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:22:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:49.351 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 -- 172.21.15.32:0/2226499363 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbb300b4bc0 msgr2=0x7fbb300b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbb300b4bc0 0x7fbb300b70d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbb40005fd0 tx=0x7fbb40009500 comp rx=0 tx=0).stop 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 -- 172.21.15.32:0/2226499363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48078030 msgr2=0x7fbb481a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48078030 0x7fbb481a0c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbb34005ab0 tx=0x7fbb34003b90 comp rx=0 tx=0).stop 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 -- 172.21.15.32:0/2226499363 shutdown_connections 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbb300b4bc0 0x7fbb300b70d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb48114310 0x7fbb481a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48078030 0x7fbb481a0c10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 --2- 172.21.15.32:0/2226499363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb480776e0 0x7fbb481a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 -- 172.21.15.32:0/2226499363 >> 172.21.15.32:0/2226499363 conn(0x7fbb48101420 msgr2=0x7fbb48116390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 -- 172.21.15.32:0/2226499363 shutdown_connections 2024-08-31T07:22:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:49.338+0000 7fbb4e000640 1 -- 172.21.15.32:0/2226499363 wait complete. 2024-08-31T07:22:49.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 2.98 MB/sec execute 398 sec latency 8805.277 ms 2024-08-31T07:22:50.532 INFO:tasks.cephfs.filesystem:scrub status for tag:a047226e-5dfa-4aa9-bb91-ffd989a87ee6 - {'path': '/', 'tag': 'a047226e-5dfa-4aa9-bb91-ffd989a87ee6', 'options': 'recursive,force'} 2024-08-31T07:22:50.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 2.97 MB/sec execute 399 sec latency 9805.390 ms 2024-08-31T07:22:50.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:50 smithi032 ceph-mon[17252]: pgmap v678: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 78 op/s 2024-08-31T07:22:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:50 smithi073 ceph-mon[21995]: pgmap v678: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 78 op/s 2024-08-31T07:22:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:50 smithi084 ceph-mon[22387]: pgmap v678: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 78 op/s 2024-08-31T07:22:51.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 2.96 MB/sec execute 400 sec latency 10805.544 ms 2024-08-31T07:22:52.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:22:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:52.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 2.96 MB/sec execute 401 sec latency 11805.616 ms 2024-08-31T07:22:52.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:52 smithi032 ceph-mon[17252]: pgmap v679: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-08-31T07:22:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:52 smithi073 ceph-mon[21995]: pgmap v679: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-08-31T07:22:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:52 smithi084 ceph-mon[22387]: pgmap v679: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.2 MiB/s wr, 86 op/s 2024-08-31T07:22:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.066+0000 7f5b04c00640 1 -- 172.21.15.32:0/2658502240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00109090 msgr2=0x7f5b001094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.066+0000 7f5b04c00640 1 --2- 172.21.15.32:0/2658502240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00109090 0x7f5b001094a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009900 tx=0x7f5aec02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:22:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.067+0000 7f5b04c00640 1 -- 172.21.15.32:0/2658502240 shutdown_connections 2024-08-31T07:22:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.067+0000 7f5b04c00640 1 --2- 172.21.15.32:0/2658502240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b001099e0 0x7f5b00110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.067+0000 7f5b04c00640 1 --2- 172.21.15.32:0/2658502240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00109090 0x7f5b001094a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.067+0000 7f5b04c00640 1 --2- 172.21.15.32:0/2658502240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00103750 0x7f5b00103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.067+0000 7f5b04c00640 1 -- 172.21.15.32:0/2658502240 >> 172.21.15.32:0/2658502240 conn(0x7f5b000ff1a0 msgr2=0x7f5b001015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.067+0000 7f5b04c00640 1 -- 172.21.15.32:0/2658502240 shutdown_connections 2024-08-31T07:22:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.067+0000 7f5b04c00640 1 -- 172.21.15.32:0/2658502240 wait complete. 2024-08-31T07:22:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.068+0000 7f5b04c00640 1 Processor -- start 2024-08-31T07:22:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.068+0000 7f5b04c00640 1 -- start start 2024-08-31T07:22:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5b04c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00103750 0x7f5b000754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5b04c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b00109090 0x7f5b000759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5b04c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b001099e0 0x7f5b00075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5b04c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5b00113670 con 0x7f5b00103750 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5b04c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001134f0 con 0x7f5b001099e0 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5b04c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001137f0 con 0x7f5b00109090 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5aff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00103750 0x7f5b000754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5aff400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00103750 0x7f5b000754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40532/0 (socket says 172.21.15.32:40532) 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5aff400640 1 -- 172.21.15.32:0/740026282 learned_addr learned my addr 172.21.15.32:0/740026282 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5affe00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b001099e0 0x7f5b00075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.069+0000 7f5afea00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b00109090 0x7f5b000759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5affe00640 1 -- 172.21.15.32:0/740026282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b00109090 msgr2=0x7f5b000759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5affe00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b00109090 0x7f5b000759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5affe00640 1 -- 172.21.15.32:0/740026282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00103750 msgr2=0x7f5b000754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5affe00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00103750 0x7f5b000754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:53.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5affe00640 1 -- 172.21.15.32:0/740026282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b001a8320 con 0x7f5b001099e0 2024-08-31T07:22:53.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5aff400640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00103750 0x7f5b000754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:22:53.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5affe00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b001099e0 0x7f5b00075f30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009fd0 tx=0x7f5af800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:53.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5af7e00640 1 -- 172.21.15.32:0/740026282 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af800ce20 con 0x7f5b001099e0 2024-08-31T07:22:53.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5b04c00640 1 -- 172.21.15.32:0/740026282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b001a8610 con 0x7f5b001099e0 2024-08-31T07:22:53.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5af7e00640 1 -- 172.21.15.32:0/740026282 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af8005020 con 0x7f5b001099e0 2024-08-31T07:22:53.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5af7e00640 1 -- 172.21.15.32:0/740026282 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8010690 con 0x7f5b001099e0 2024-08-31T07:22:53.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.070+0000 7f5b04c00640 1 -- 172.21.15.32:0/740026282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b001a8b20 con 0x7f5b001099e0 2024-08-31T07:22:53.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.071+0000 7f5b04c00640 1 -- 172.21.15.32:0/740026282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b00103b60 con 0x7f5b001099e0 2024-08-31T07:22:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.074+0000 7f5af7e00640 1 -- 172.21.15.32:0/740026282 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5af80054d0 con 0x7f5b001099e0 2024-08-31T07:22:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.076+0000 7f5af7e00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ae40b4ca0 0x7f5ae40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.076+0000 7f5af7e00640 1 -- 172.21.15.32:0/740026282 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7f5af80e5be0 con 0x7f5b001099e0 2024-08-31T07:22:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.076+0000 7f5aff400640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ae40b4ca0 0x7f5ae40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.077+0000 7f5aff400640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ae40b4ca0 0x7f5ae40b71b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b00076f00 tx=0x7f5af0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:53.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.077+0000 7f5af7e00640 1 -- 172.21.15.32:0/740026282 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5af80ac9c0 con 0x7f5b001099e0 2024-08-31T07:22:53.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.315+0000 7f5b04c00640 1 -- 172.21.15.32:0/740026282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b000767b0 con 0x7f5b001099e0 2024-08-31T07:22:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.316+0000 7f5af7e00640 1 -- 172.21.15.32:0/740026282 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11968 (secure 0 0 0) 0x7f5af80ac2c0 con 0x7f5b001099e0 2024-08-31T07:22:53.317 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:22:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"btime":"2024-08-31T07:22:06:095426+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:22:05.949269+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:22:53.318 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 40 2024-08-31T07:22:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 -- 172.21.15.32:0/740026282 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ae40b4ca0 msgr2=0x7f5ae40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ae40b4ca0 0x7f5ae40b71b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b00076f00 tx=0x7f5af0005f90 comp rx=0 tx=0).stop 2024-08-31T07:22:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 -- 172.21.15.32:0/740026282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b001099e0 msgr2=0x7f5b00075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b001099e0 0x7f5b00075f30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009fd0 tx=0x7f5af800ef10 comp rx=0 tx=0).stop 2024-08-31T07:22:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 -- 172.21.15.32:0/740026282 shutdown_connections 2024-08-31T07:22:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5ae40b4ca0 0x7f5ae40b71b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b001099e0 0x7f5b00075f30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:53.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b00109090 0x7f5b000759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:53.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 --2- 172.21.15.32:0/740026282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b00103750 0x7f5b000754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:53.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 -- 172.21.15.32:0/740026282 >> 172.21.15.32:0/740026282 conn(0x7f5b000ff1a0 msgr2=0x7f5b001015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:53.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 -- 172.21.15.32:0/740026282 shutdown_connections 2024-08-31T07:22:53.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:53.319+0000 7f5b04c00640 1 -- 172.21.15.32:0/740026282 wait complete. 2024-08-31T07:22:53.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 2.95 MB/sec execute 402 sec latency 12805.657 ms 2024-08-31T07:22:53.850 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:53 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/740026282' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:22:53.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:53 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/740026282' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:22:53.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:53 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/740026282' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:22:54.242 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id de2c8f7e --release --await' 2024-08-31T07:22:54.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id de2c8f7e --release --await 2024-08-31T07:22:54.624 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:22:54.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323222 2.94 MB/sec execute 403 sec latency 13805.774 ms 2024-08-31T07:22:54.845 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:54 smithi032 ceph-mon[17252]: pgmap v680: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 947 KiB/s wr, 58 op/s 2024-08-31T07:22:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:54 smithi073 ceph-mon[21995]: pgmap v680: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 947 KiB/s wr, 58 op/s 2024-08-31T07:22:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:54 smithi084 ceph-mon[22387]: pgmap v680: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 947 KiB/s wr, 58 op/s 2024-08-31T07:22:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.535+0000 7fa485000640 1 -- 172.21.15.32:0/1226674106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa48010b290 msgr2=0x7fa48010b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.535+0000 7fa485000640 1 --2- 172.21.15.32:0/1226674106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa48010b290 0x7fa48010b6a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009900 tx=0x7fa46c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.535+0000 7fa485000640 1 -- 172.21.15.32:0/1226674106 shutdown_connections 2024-08-31T07:22:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.535+0000 7fa485000640 1 --2- 172.21.15.32:0/1226674106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48010bbe0 0x7fa480112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.535+0000 7fa485000640 1 --2- 172.21.15.32:0/1226674106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa48010b290 0x7fa48010b6a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.535+0000 7fa485000640 1 --2- 172.21.15.32:0/1226674106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa480105950 0x7fa480105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.535+0000 7fa485000640 1 -- 172.21.15.32:0/1226674106 >> 172.21.15.32:0/1226674106 conn(0x7fa480101420 msgr2=0x7fa480103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.535+0000 7fa485000640 1 -- 172.21.15.32:0/1226674106 shutdown_connections 2024-08-31T07:22:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.535+0000 7fa485000640 1 -- 172.21.15.32:0/1226674106 wait complete. 2024-08-31T07:22:55.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.536+0000 7fa485000640 1 Processor -- start 2024-08-31T07:22:55.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa485000640 1 -- start start 2024-08-31T07:22:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa485000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa480105950 0x7fa48019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa485000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa48010b290 0x7fa48019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa485000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48010bbe0 0x7fa4801a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa485000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801158a0 con 0x7fa48010b290 2024-08-31T07:22:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa485000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa480115720 con 0x7fa480105950 2024-08-31T07:22:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa485000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa480115a20 con 0x7fa48010bbe0 2024-08-31T07:22:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa47ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa48010b290 0x7fa48019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa47ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa48010b290 0x7fa48019c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40552/0 (socket says 172.21.15.32:40552) 2024-08-31T07:22:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa47ea00640 1 -- 172.21.15.32:0/1724852622 learned_addr learned my addr 172.21.15.32:0/1724852622 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.537+0000 7fa47f400640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa480105950 0x7fa48019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.538+0000 7fa47fe00640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48010bbe0 0x7fa4801a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.538+0000 7fa47f400640 1 -- 172.21.15.32:0/1724852622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48010bbe0 msgr2=0x7fa4801a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.538+0000 7fa47f400640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48010bbe0 0x7fa4801a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.538+0000 7fa47f400640 1 -- 172.21.15.32:0/1724852622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa48010b290 msgr2=0x7fa48019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.538+0000 7fa47f400640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa48010b290 0x7fa48019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.538+0000 7fa47f400640 1 -- 172.21.15.32:0/1724852622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4801a21e0 con 0x7fa480105950 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.538+0000 7fa47f400640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa480105950 0x7fa48019c270 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa47000e9f0 tx=0x7fa47000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.539+0000 7fa477e00640 1 -- 172.21.15.32:0/1724852622 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47000ccc0 con 0x7fa480105950 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.539+0000 7fa477e00640 1 -- 172.21.15.32:0/1724852622 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4700108e0 con 0x7fa480105950 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.539+0000 7fa477e00640 1 -- 172.21.15.32:0/1724852622 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470018880 con 0x7fa480105950 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.539+0000 7fa485000640 1 -- 172.21.15.32:0/1724852622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4801a24d0 con 0x7fa480105950 2024-08-31T07:22:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.539+0000 7fa485000640 1 -- 172.21.15.32:0/1724852622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4801a8c30 con 0x7fa480105950 2024-08-31T07:22:55.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.542+0000 7fa48d244740 1 Processor -- start 2024-08-31T07:22:55.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.542+0000 7fa48d244740 1 -- start start 2024-08-31T07:22:55.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.542+0000 7fa48d244740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696ea3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.542+0000 7fa48d244740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696ad4660 0x559696ea35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.542+0000 7fa48d244740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696adc840 0x559696ea3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.542+0000 7fa48d244740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559696e19a90 con 0x559696adc840 2024-08-31T07:22:55.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.542+0000 7fa48d244740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559696e09020 con 0x559696afd520 2024-08-31T07:22:55.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.542+0000 7fa48d244740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559696e0cfd0 con 0x559696ad4660 2024-08-31T07:22:55.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.543+0000 7fa47fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696ea3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.543+0000 7fa47ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696adc840 0x559696ea3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.543+0000 7fa47fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696ea3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54792/0 (socket says 172.21.15.32:54792) 2024-08-31T07:22:55.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.543+0000 7fa47fe00640 1 -- 172.21.15.32:0/920517022 learned_addr learned my addr 172.21.15.32:0/920517022 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:55.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.543+0000 7fa47fe00640 1 -- 172.21.15.32:0/920517022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696ad4660 msgr2=0x559696ea35d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:22:55.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.543+0000 7fa47fe00640 1 --2- 172.21.15.32:0/920517022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696ad4660 0x559696ea35d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.543+0000 7fa47fe00640 1 -- 172.21.15.32:0/920517022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696adc840 msgr2=0x559696ea3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.543+0000 7fa47fe00640 1 --2- 172.21.15.32:0/920517022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696adc840 0x559696ea3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.543+0000 7fa47fe00640 1 -- 172.21.15.32:0/920517022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559696b68b00 con 0x559696afd520 2024-08-31T07:22:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.543+0000 7fa47fe00640 1 --2- 172.21.15.32:0/920517022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696ea3090 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa478010370 tx=0x7fa478010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1afb33b7d8147884 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.544+0000 7fa474c00640 1 -- 172.21.15.32:0/920517022 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478016070 con 0x559696afd520 2024-08-31T07:22:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.544+0000 7fa474c00640 1 -- 172.21.15.32:0/920517022 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa478004d20 con 0x559696afd520 2024-08-31T07:22:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.544+0000 7fa474c00640 1 -- 172.21.15.32:0/920517022 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478005380 con 0x559696afd520 2024-08-31T07:22:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.545+0000 7fa48d244740 1 -- 172.21.15.32:0/920517022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 msgr2=0x559696ea3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.545+0000 7fa48d244740 1 --2- 172.21.15.32:0/920517022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696ea3090 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa478010370 tx=0x7fa478010830 comp rx=0 tx=0).stop 2024-08-31T07:22:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.545+0000 7fa48d244740 1 -- 172.21.15.32:0/920517022 shutdown_connections 2024-08-31T07:22:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.545+0000 7fa48d244740 1 --2- 172.21.15.32:0/920517022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696adc840 0x559696ea3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.545+0000 7fa48d244740 1 --2- 172.21.15.32:0/920517022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696ad4660 0x559696ea35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.545+0000 7fa48d244740 1 --2- 172.21.15.32:0/920517022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696ea3090 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.545+0000 7fa48d244740 1 -- 172.21.15.32:0/920517022 >> 172.21.15.32:0/920517022 conn(0x559696af1910 msgr2=0x559696ea0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.545+0000 7fa477e00640 1 -- 172.21.15.32:0/1724852622 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa47001f020 con 0x7fa480105950 2024-08-31T07:22:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.545+0000 7fa48d244740 1 -- 172.21.15.32:0/920517022 shutdown_connections 2024-08-31T07:22:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.545+0000 7fa48d244740 1 -- 172.21.15.32:0/920517022 wait complete. 2024-08-31T07:22:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.546+0000 7fa48d244740 1 -- start start 2024-08-31T07:22:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.546+0000 7fa48d244740 1 Processor -- start 2024-08-31T07:22:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa48d244740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 0x559696eae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa48d244740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696eaea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa48d244740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696abbea0 0x559696eaef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa48d244740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559696c6bbb0 con 0x559696abbea0 2024-08-31T07:22:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa48d244740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559696d8ff90 con 0x559696afd520 2024-08-31T07:22:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa48d244740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559696eb1870 con 0x559696adc840 2024-08-31T07:22:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa48d244740 10 client.0 _ll_register_callbacks cb 0x559696db5710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa47fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 0x559696eae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa47fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 0x559696eae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:48480/0 (socket says 172.21.15.32:48480) 2024-08-31T07:22:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa47f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696eaea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa47fe00640 1 -- 172.21.15.32:0/4032454243 learned_addr learned my addr 172.21.15.32:0/4032454243 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa477e00640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4640b4c30 0x7fa4640b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.547+0000 7fa47ea00640 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696abbea0 0x559696eaef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa477e00640 1 -- 172.21.15.32:0/1724852622 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fa4700e55f0 con 0x7fa480105950 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa47fe00640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4640b4c30 0x7fa4640b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa47f400640 1 -- 172.21.15.32:0/4032454243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 msgr2=0x559696eae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa47f400640 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 0x559696eae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa47f400640 1 -- 172.21.15.32:0/4032454243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696abbea0 msgr2=0x559696eaef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa47f400640 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696abbea0 0x559696eaef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa47f400640 1 -- 172.21.15.32:0/4032454243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559696bdf960 con 0x559696afd520 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa47f400640 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696eaea10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa4700e2d30 tx=0x7fa47000ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bf976461da45d5b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa469600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa47fe00640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4640b4c30 0x7fa4640b7140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa478010fe0 tx=0x7fa478009600 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.548+0000 7fa469600640 1 -- 172.21.15.32:0/4032454243 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4700ac3d0 con 0x559696afd520 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa469600640 1 -- 172.21.15.32:0/4032454243 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fa470093070 con 0x559696afd520 2024-08-31T07:22:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa48d244740 1 -- 172.21.15.32:0/4032454243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559696da7e30 con 0x559696afd520 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa469600640 1 -- 172.21.15.32:0/4032454243 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa4700abcd0 con 0x559696afd520 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa48d244740 10 client.27662 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa48d244740 1 -- 172.21.15.32:0/4032454243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559696ebd340 con 0x559696afd520 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa469600640 1 -- 172.21.15.32:0/4032454243 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559696da7e30 con 0x559696afd520 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa469600640 1 -- 172.21.15.32:0/4032454243 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fa4700b01a0 con 0x559696afd520 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa48d244740 10 client.27662 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa48d244740 10 client.27662 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa48d244740 20 client.27662 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa48d244740 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559696a91250 0x559696ec0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa48d244740 4 client.27662 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.549+0000 7fa48d244740 1 -- 172.21.15.32:0/4032454243 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559696e1b9e0 con 0x559696a91250 2024-08-31T07:22:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.550+0000 7fa47ea00640 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559696a91250 0x559696ec0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.550+0000 7fa47ea00640 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559696a91250 0x559696ec0860 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=107d5084e46ef05 server_cookie=cc5728422acecb2b in_seq=0 out_seq=0 2024-08-31T07:22:55.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.550+0000 7fa469600640 10 client.27662 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.551+0000 7fa469600640 1 -- 172.21.15.32:0/4032454243 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559696e1b9e0 con 0x559696a91250 2024-08-31T07:22:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.551+0000 7fa469600640 10 client.27662 handle_command_reply: tid=0 2024-08-31T07:22:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.551+0000 7fa48d244740 1 client.27662 shutdown 2024-08-31T07:22:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 1 -- 172.21.15.32:0/4032454243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 msgr2=0x559696eaea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696eaea10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa4700e2d30 tx=0x7fa47000ce60 comp rx=0 tx=0).stop 2024-08-31T07:22:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 1 -- 172.21.15.32:0/4032454243 shutdown_connections 2024-08-31T07:22:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559696a91250 0x559696ec0860 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696abbea0 0x559696eaef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696eaea10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 1 --2- 172.21.15.32:0/4032454243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 0x559696eae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa469600640 0 client.27662 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 1 -- 172.21.15.32:0/4032454243 >> 172.21.15.32:0/4032454243 conn(0x559696af1910 msgr2=0x559696ea6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 1 -- 172.21.15.32:0/4032454243 shutdown_connections 2024-08-31T07:22:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 1 -- 172.21.15.32:0/4032454243 wait complete. 2024-08-31T07:22:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.552+0000 7fa48d244740 20 client.27662 trim_cache size 0 max 16384 2024-08-31T07:22:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.570+0000 7fa48d244740 1 Processor -- start 2024-08-31T07:22:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.570+0000 7fa48d244740 1 -- start start 2024-08-31T07:22:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa48d244740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696abbea0 0x559696ebaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa48d244740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 0x559696eab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa48d244740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696afd520 0x559696eabc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa48d244740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801b14a0 con 0x559696afd520 2024-08-31T07:22:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa48d244740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801b30d0 con 0x559696abbea0 2024-08-31T07:22:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa48d244740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801b1b70 con 0x559696adc840 2024-08-31T07:22:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696afd520 0x559696eabc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696abbea0 0x559696ebaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696abbea0 0x559696ebaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54818/0 (socket says 172.21.15.32:54818) 2024-08-31T07:22:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696afd520 0x559696eabc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40598/0 (socket says 172.21.15.32:40598) 2024-08-31T07:22:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47f400640 1 -- 172.21.15.32:0/541010928 learned_addr learned my addr 172.21.15.32:0/541010928 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47ea00640 1 --2- 172.21.15.32:0/541010928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 0x559696eab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47f400640 1 -- 172.21.15.32:0/541010928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 msgr2=0x559696eab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47f400640 1 --2- 172.21.15.32:0/541010928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 0x559696eab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47f400640 1 -- 172.21.15.32:0/541010928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696afd520 msgr2=0x559696eabc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47f400640 1 --2- 172.21.15.32:0/541010928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696afd520 0x559696eabc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.571+0000 7fa47f400640 1 -- 172.21.15.32:0/541010928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559696ea1d10 con 0x559696abbea0 2024-08-31T07:22:55.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.572+0000 7fa47f400640 1 --2- 172.21.15.32:0/541010928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696abbea0 0x559696ebaa90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa470000c00 tx=0x7fa4700bad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7fe1137b2436c1c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:55.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.572+0000 7fa46be00640 1 -- 172.21.15.32:0/541010928 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4700b4ff0 con 0x559696abbea0 2024-08-31T07:22:55.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.572+0000 7fa46be00640 1 -- 172.21.15.32:0/541010928 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4700b0540 con 0x559696abbea0 2024-08-31T07:22:55.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.572+0000 7fa46be00640 1 -- 172.21.15.32:0/541010928 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4700b0130 con 0x559696abbea0 2024-08-31T07:22:55.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.573+0000 7fa48d244740 1 -- 172.21.15.32:0/541010928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696abbea0 msgr2=0x559696ebaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:55.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.573+0000 7fa48d244740 1 --2- 172.21.15.32:0/541010928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696abbea0 0x559696ebaa90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa470000c00 tx=0x7fa4700bad70 comp rx=0 tx=0).stop 2024-08-31T07:22:55.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.573+0000 7fa48d244740 1 -- 172.21.15.32:0/541010928 shutdown_connections 2024-08-31T07:22:55.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.573+0000 7fa48d244740 1 --2- 172.21.15.32:0/541010928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696afd520 0x559696eabc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.573+0000 7fa48d244740 1 --2- 172.21.15.32:0/541010928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696adc840 0x559696eab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.573+0000 7fa48d244740 1 --2- 172.21.15.32:0/541010928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696abbea0 0x559696ebaa90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.573+0000 7fa48d244740 1 -- 172.21.15.32:0/541010928 >> 172.21.15.32:0/541010928 conn(0x559696af1910 msgr2=0x559696eaf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.573+0000 7fa48d244740 1 -- 172.21.15.32:0/541010928 shutdown_connections 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.573+0000 7fa48d244740 1 -- 172.21.15.32:0/541010928 wait complete. 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.573+0000 7fa48d244740 1 -- start start 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.574+0000 7fa48d244740 1 Processor -- start 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.574+0000 7fa48d244740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696abbea0 0x559696eb4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.574+0000 7fa48d244740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696adc840 0x559696eb4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.574+0000 7fa48d244740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696f25250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.574+0000 7fa48d244740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801b2a00 con 0x559696adc840 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.574+0000 7fa48d244740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801b2300 con 0x559696afd520 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.574+0000 7fa48d244740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801af4a0 con 0x559696abbea0 2024-08-31T07:22:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.574+0000 7fa48d244740 10 client.0 _ll_register_callbacks cb 0x7fa4700ab440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:22:55.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696adc840 0x559696eb4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696abbea0 0x559696eb4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696abbea0 0x559696eb4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:48492/0 (socket says 172.21.15.32:48492) 2024-08-31T07:22:55.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47f400640 1 -- 172.21.15.32:0/2782404281 learned_addr learned my addr 172.21.15.32:0/2782404281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:22:55.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47fe00640 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696f25250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47f400640 1 -- 172.21.15.32:0/2782404281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 msgr2=0x559696f25250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47f400640 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696f25250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47f400640 1 -- 172.21.15.32:0/2782404281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696adc840 msgr2=0x559696eb4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47f400640 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696adc840 0x559696eb4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47f400640 1 -- 172.21.15.32:0/2782404281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559696ea4fd0 con 0x559696abbea0 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa47f400640 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696abbea0 0x559696eb4440 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa470009230 tx=0x7fa47000ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8dbfd0302137d45 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.575+0000 7fa469600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa469600640 1 -- 172.21.15.32:0/2782404281 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470018c80 con 0x559696abbea0 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa48d244740 1 -- 172.21.15.32:0/2782404281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4700b0540 con 0x559696abbea0 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa469600640 1 -- 172.21.15.32:0/2782404281 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8441+0+0 (secure 0 0 0) 0x7fa470010a90 con 0x559696abbea0 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa469600640 1 -- 172.21.15.32:0/2782404281 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa4700ac3d0 con 0x559696abbea0 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa48d244740 10 client.27670 fetch_fsmap learned FSMap version 40 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa48d244740 1 -- 172.21.15.32:0/2782404281 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559696f2ca80 con 0x559696abbea0 2024-08-31T07:22:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa469600640 1 -- 172.21.15.32:0/2782404281 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4700b0540 con 0x559696abbea0 2024-08-31T07:22:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa469600640 1 -- 172.21.15.32:0/2782404281 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fa4700ab050 con 0x559696abbea0 2024-08-31T07:22:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa48d244740 10 client.27670 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T07:22:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa48d244740 10 client.27670 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:22:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa48d244740 20 client.27670 populate_metadata read hostname 'smithi032' 2024-08-31T07:22:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa48d244740 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559696f2ce70 0x559696f2f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:22:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa48d244740 4 client.27670 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "de2c8f7e", "release": true, "await": true}] 2024-08-31T07:22:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.576+0000 7fa48d244740 1 -- 172.21.15.32:0/2782404281 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "de2c8f7e", "release": true, "await": true}) -- 0x559696e17480 con 0x559696f2ce70 2024-08-31T07:22:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.577+0000 7fa47ea00640 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559696f2ce70 0x559696f2f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:22:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.577+0000 7fa47ea00640 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559696f2ce70 0x559696f2f2c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c3b726d34eaad5 server_cookie=4a186eedebd52d66 in_seq=0 out_seq=0 2024-08-31T07:22:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.577+0000 7fa469600640 10 client.27670 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:55.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.599+0000 7fa469600640 1 -- 172.21.15.32:0/2782404281 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559696e17480 con 0x559696f2ce70 2024-08-31T07:22:55.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.599+0000 7fa469600640 10 client.27670 handle_command_reply: tid=0 2024-08-31T07:22:55.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.601+0000 7fa48d244740 1 client.27670 shutdown 2024-08-31T07:22:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.602+0000 7fa48d244740 1 -- 172.21.15.32:0/2782404281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696abbea0 msgr2=0x559696eb4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.602+0000 7fa48d244740 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696abbea0 0x559696eb4440 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa470009230 tx=0x7fa47000ce60 comp rx=0 tx=0).stop 2024-08-31T07:22:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.603+0000 7fa48d244740 1 -- 172.21.15.32:0/2782404281 shutdown_connections 2024-08-31T07:22:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.603+0000 7fa48d244740 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559696f2ce70 0x559696f2f2c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.603+0000 7fa48d244740 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559696afd520 0x559696f25250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.603+0000 7fa48d244740 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559696adc840 0x559696eb4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.603+0000 7fa48d244740 1 --2- 172.21.15.32:0/2782404281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559696abbea0 0x559696eb4440 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.603+0000 7fa48d244740 1 -- 172.21.15.32:0/2782404281 >> 172.21.15.32:0/2782404281 conn(0x559696af1910 msgr2=0x559696ea3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.603+0000 7fa469600640 0 client.27670 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:22:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.604+0000 7fa48d244740 1 -- 172.21.15.32:0/2782404281 shutdown_connections 2024-08-31T07:22:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.604+0000 7fa48d244740 1 -- 172.21.15.32:0/2782404281 wait complete. 2024-08-31T07:22:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.604+0000 7fa48d244740 20 client.27670 trim_cache size 0 max 16384 2024-08-31T07:22:55.606 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 40, 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 32, 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "de2c8f7e": { 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 32, 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:22:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 135.5, 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:22:55.608 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:22:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.607+0000 7fa485000640 1 -- 172.21.15.32:0/1724852622 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4640b4c30 msgr2=0x7fa4640b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.607+0000 7fa485000640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4640b4c30 0x7fa4640b7140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa478010fe0 tx=0x7fa478009600 comp rx=0 tx=0).stop 2024-08-31T07:22:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.607+0000 7fa485000640 1 -- 172.21.15.32:0/1724852622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa480105950 msgr2=0x7fa48019c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:22:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.607+0000 7fa485000640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa480105950 0x7fa48019c270 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa47000e9f0 tx=0x7fa47000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.609+0000 7fa485000640 1 -- 172.21.15.32:0/1724852622 shutdown_connections 2024-08-31T07:22:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.609+0000 7fa485000640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa4640b4c30 0x7fa4640b7140 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.609+0000 7fa485000640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48010bbe0 0x7fa4801a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.609+0000 7fa485000640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa48010b290 0x7fa48019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.609+0000 7fa485000640 1 --2- 172.21.15.32:0/1724852622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa480105950 0x7fa48019c270 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:22:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.609+0000 7fa485000640 1 -- 172.21.15.32:0/1724852622 >> 172.21.15.32:0/1724852622 conn(0x7fa480101420 msgr2=0x7fa480103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:22:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.610+0000 7fa485000640 1 -- 172.21.15.32:0/1724852622 shutdown_connections 2024-08-31T07:22:55.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:22:55.610+0000 7fa485000640 1 -- 172.21.15.32:0/1724852622 wait complete. 2024-08-31T07:22:55.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323365 2.93 MB/sec execute 404 sec latency 14652.012 ms 2024-08-31T07:22:56.516 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'de2c8f7e', total seconds elapsed: 20.0 2024-08-31T07:22:56.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 324487 2.94 MB/sec execute 405 sec latency 224.060 ms 2024-08-31T07:22:56.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:56 smithi032 ceph-mon[17252]: pgmap v681: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.0 MiB/s wr, 57 op/s 2024-08-31T07:22:56.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:56 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:22:56.936 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:56 smithi073 ceph-mon[21995]: pgmap v681: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.0 MiB/s wr, 57 op/s 2024-08-31T07:22:56.936 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:56 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:22:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:56 smithi084 ceph-mon[22387]: pgmap v681: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.0 MiB/s wr, 57 op/s 2024-08-31T07:22:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:56 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:22:57.767 INFO:tasks.workunit.client.0.smithi032.stdout: 1 324487 2.93 MB/sec execute 406 sec latency 1224.112 ms 2024-08-31T07:22:58.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 324487 2.92 MB/sec execute 407 sec latency 2224.239 ms 2024-08-31T07:22:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:22:58 smithi073 ceph-mon[21995]: pgmap v682: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 931 KiB/s wr, 57 op/s 2024-08-31T07:22:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:22:58 smithi084 ceph-mon[22387]: pgmap v682: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 931 KiB/s wr, 57 op/s 2024-08-31T07:22:59.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:22:58 smithi032 ceph-mon[17252]: pgmap v682: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 931 KiB/s wr, 57 op/s 2024-08-31T07:22:59.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 324487 2.92 MB/sec execute 408 sec latency 3224.344 ms 2024-08-31T07:23:00.648 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:00 smithi032 ceph-mon[17252]: pgmap v683: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 862 KiB/s wr, 48 op/s 2024-08-31T07:23:00.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 324528 2.91 MB/sec execute 409 sec latency 3926.025 ms 2024-08-31T07:23:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:00 smithi073 ceph-mon[21995]: pgmap v683: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 862 KiB/s wr, 48 op/s 2024-08-31T07:23:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:00 smithi084 ceph-mon[22387]: pgmap v683: 97 pgs: 97 active+clean; 244 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 862 KiB/s wr, 48 op/s 2024-08-31T07:23:01.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 325811 2.92 MB/sec execute 410 sec latency 31.368 ms 2024-08-31T07:23:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:23:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:23:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:23:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:23:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:23:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:23:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:23:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:23:02.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:23:02.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:23:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:23:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:23:02.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 327060 2.93 MB/sec execute 411 sec latency 19.965 ms 2024-08-31T07:23:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:02 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:23:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:02 smithi073 ceph-mon[21995]: pgmap v684: 97 pgs: 97 active+clean; 246 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 984 KiB/s wr, 80 op/s 2024-08-31T07:23:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:02 smithi073 ceph-mon[21995]: osdmap e89: 12 total, 12 up, 12 in 2024-08-31T07:23:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:02 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:23:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:02 smithi084 ceph-mon[22387]: pgmap v684: 97 pgs: 97 active+clean; 246 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 984 KiB/s wr, 80 op/s 2024-08-31T07:23:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:02 smithi084 ceph-mon[22387]: osdmap e89: 12 total, 12 up, 12 in 2024-08-31T07:23:03.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:02 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:23:03.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:02 smithi032 ceph-mon[17252]: pgmap v684: 97 pgs: 97 active+clean; 246 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 984 KiB/s wr, 80 op/s 2024-08-31T07:23:03.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:02 smithi032 ceph-mon[17252]: osdmap e89: 12 total, 12 up, 12 in 2024-08-31T07:23:03.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 328029 2.94 MB/sec execute 412 sec latency 329.626 ms 2024-08-31T07:23:04.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 328029 2.93 MB/sec execute 413 sec latency 1329.773 ms 2024-08-31T07:23:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:04 smithi073 ceph-mon[21995]: pgmap v686: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 986 KiB/s wr, 103 op/s 2024-08-31T07:23:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:04 smithi084 ceph-mon[22387]: pgmap v686: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 986 KiB/s wr, 103 op/s 2024-08-31T07:23:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:04 smithi032 ceph-mon[17252]: pgmap v686: 97 pgs: 97 active+clean; 245 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 986 KiB/s wr, 103 op/s 2024-08-31T07:23:05.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 328253 2.92 MB/sec execute 414 sec latency 1682.501 ms 2024-08-31T07:23:06.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 328946 2.92 MB/sec execute 415 sec latency 1048.635 ms 2024-08-31T07:23:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:06 smithi073 ceph-mon[21995]: pgmap v687: 97 pgs: 97 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-08-31T07:23:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:06 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:23:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:06 smithi084 ceph-mon[22387]: pgmap v687: 97 pgs: 97 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-08-31T07:23:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:06 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:23:07.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:06 smithi032 ceph-mon[17252]: pgmap v687: 97 pgs: 97 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-08-31T07:23:07.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:06 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:23:07.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 330428 2.94 MB/sec execute 416 sec latency 99.788 ms 2024-08-31T07:23:08.768 INFO:tasks.workunit.client.0.smithi032.stdout: 1 331577 2.95 MB/sec execute 417 sec latency 55.059 ms 2024-08-31T07:23:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:08 smithi073 ceph-mon[21995]: pgmap v688: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-08-31T07:23:08.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:08 smithi084 ceph-mon[22387]: pgmap v688: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-08-31T07:23:09.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:08 smithi032 ceph-mon[17252]: pgmap v688: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-08-31T07:23:09.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 331577 2.94 MB/sec execute 418 sec latency 1055.140 ms 2024-08-31T07:23:10.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 332456 2.94 MB/sec execute 419 sec latency 1412.449 ms 2024-08-31T07:23:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:10 smithi073 ceph-mon[21995]: pgmap v689: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-08-31T07:23:10.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:10 smithi084 ceph-mon[22387]: pgmap v689: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-08-31T07:23:11.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:10 smithi032 ceph-mon[17252]: pgmap v689: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-08-31T07:23:11.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 333912 2.96 MB/sec execute 420 sec latency 11.811 ms 2024-08-31T07:23:12.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 335066 2.96 MB/sec execute 421 sec latency 19.624 ms 2024-08-31T07:23:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:12 smithi073 ceph-mon[21995]: pgmap v690: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-08-31T07:23:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:12 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:23:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:12 smithi073 ceph-mon[21995]: osdmap e90: 12 total, 12 up, 12 in 2024-08-31T07:23:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:12 smithi084 ceph-mon[22387]: pgmap v690: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-08-31T07:23:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:12 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:23:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:12 smithi084 ceph-mon[22387]: osdmap e90: 12 total, 12 up, 12 in 2024-08-31T07:23:13.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:12 smithi032 ceph-mon[17252]: pgmap v690: 97 pgs: 97 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-08-31T07:23:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:12 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:23:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:12 smithi032 ceph-mon[17252]: osdmap e90: 12 total, 12 up, 12 in 2024-08-31T07:23:13.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 335120 2.96 MB/sec execute 422 sec latency 950.447 ms 2024-08-31T07:23:14.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 335120 2.95 MB/sec execute 423 sec latency 1950.616 ms 2024-08-31T07:23:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:14 smithi073 ceph-mon[21995]: pgmap v692: 97 pgs: 97 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-08-31T07:23:14.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:14 smithi084 ceph-mon[22387]: pgmap v692: 97 pgs: 97 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-08-31T07:23:15.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:14 smithi032 ceph-mon[17252]: pgmap v692: 97 pgs: 97 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-08-31T07:23:15.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 335274 2.94 MB/sec execute 424 sec latency 2152.660 ms 2024-08-31T07:23:16.769 INFO:tasks.workunit.client.0.smithi032.stdout: 1 335274 2.94 MB/sec execute 425 sec latency 1717.579 ms 2024-08-31T07:23:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:16 smithi073 ceph-mon[21995]: pgmap v693: 97 pgs: 97 active+clean; 256 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-08-31T07:23:16.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:16 smithi084 ceph-mon[22387]: pgmap v693: 97 pgs: 97 active+clean; 256 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-08-31T07:23:17.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:16 smithi032 ceph-mon[17252]: pgmap v693: 97 pgs: 97 active+clean; 256 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-08-31T07:23:17.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 335274 2.93 MB/sec execute 426 sec latency 2717.705 ms 2024-08-31T07:23:18.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 335274 2.92 MB/sec execute 427 sec latency 3717.838 ms 2024-08-31T07:23:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:18 smithi073 ceph-mon[21995]: pgmap v694: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-08-31T07:23:18.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:18 smithi084 ceph-mon[22387]: pgmap v694: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-08-31T07:23:19.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:18 smithi032 ceph-mon[17252]: pgmap v694: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-08-31T07:23:19.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 335274 2.92 MB/sec execute 428 sec latency 4717.939 ms 2024-08-31T07:23:20.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:23:20.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 335274 2.91 MB/sec execute 429 sec latency 5718.007 ms 2024-08-31T07:23:20.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:20 smithi032 ceph-mon[17252]: pgmap v695: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-08-31T07:23:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:20 smithi073 ceph-mon[21995]: pgmap v695: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-08-31T07:23:20.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:20 smithi084 ceph-mon[22387]: pgmap v695: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-08-31T07:23:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.644+0000 7f3600c00640 1 -- 172.21.15.32:0/1516385236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35fc10dbd0 msgr2=0x7f35fc1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.644+0000 7f3600c00640 1 --2- 172.21.15.32:0/1516385236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35fc10dbd0 0x7f35fc1100c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009be0 tx=0x7f35f402a6d0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.644+0000 7f3600c00640 1 -- 172.21.15.32:0/1516385236 shutdown_connections 2024-08-31T07:23:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.644+0000 7f3600c00640 1 --2- 172.21.15.32:0/1516385236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35fc10dbd0 0x7f35fc1100c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.644+0000 7f3600c00640 1 --2- 172.21.15.32:0/1516385236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35fc1050c0 0x7f35fc10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.644+0000 7f3600c00640 1 --2- 172.21.15.32:0/1516385236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35fc104770 0x7f35fc104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.644+0000 7f3600c00640 1 -- 172.21.15.32:0/1516385236 >> 172.21.15.32:0/1516385236 conn(0x7f35fc0794e0 msgr2=0x7f35fc0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.645+0000 7f3600c00640 1 -- 172.21.15.32:0/1516385236 shutdown_connections 2024-08-31T07:23:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.645+0000 7f3600c00640 1 -- 172.21.15.32:0/1516385236 wait complete. 2024-08-31T07:23:21.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.646+0000 7f3600c00640 1 Processor -- start 2024-08-31T07:23:21.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.646+0000 7f3600c00640 1 -- start start 2024-08-31T07:23:21.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.646+0000 7f3600c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35fc104770 0x7f35fc1024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.646+0000 7f3600c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35fc1050c0 0x7f35fc1029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.646+0000 7f3600c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35fc10dbd0 0x7f35fc102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.646+0000 7f3600c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc113830 con 0x7f35fc10dbd0 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.646+0000 7f3600c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1136b0 con 0x7f35fc104770 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.646+0000 7f3600c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1139b0 con 0x7f35fc1050c0 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.646+0000 7f35f9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35fc104770 0x7f35fc1024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.647+0000 7f35f9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35fc104770 0x7f35fc1024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52706/0 (socket says 172.21.15.32:52706) 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.647+0000 7f35f9e00640 1 -- 172.21.15.32:0/3286367769 learned_addr learned my addr 172.21.15.32:0/3286367769 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.647+0000 7f35f9400640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35fc1050c0 0x7f35fc1029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.647+0000 7f35fa800640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35fc10dbd0 0x7f35fc102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.647+0000 7f35f9e00640 1 -- 172.21.15.32:0/3286367769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35fc1050c0 msgr2=0x7f35fc1029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.647+0000 7f35f9e00640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35fc1050c0 0x7f35fc1029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.647+0000 7f35f9e00640 1 -- 172.21.15.32:0/3286367769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35fc10dbd0 msgr2=0x7f35fc102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.647+0000 7f35f9e00640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35fc10dbd0 0x7f35fc102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.647+0000 7f35f9e00640 1 -- 172.21.15.32:0/3286367769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc1a8560 con 0x7f35fc104770 2024-08-31T07:23:21.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.647+0000 7f35f9e00640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35fc104770 0x7f35fc1024b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00e9c0 tx=0x7f35ec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:21.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.648+0000 7f35f2a00640 1 -- 172.21.15.32:0/3286367769 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec00cd80 con 0x7f35fc104770 2024-08-31T07:23:21.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.648+0000 7f3600c00640 1 -- 172.21.15.32:0/3286367769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35fc1a8850 con 0x7f35fc104770 2024-08-31T07:23:21.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.648+0000 7f35f2a00640 1 -- 172.21.15.32:0/3286367769 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ec0035f0 con 0x7f35fc104770 2024-08-31T07:23:21.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.648+0000 7f35f2a00640 1 -- 172.21.15.32:0/3286367769 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec010690 con 0x7f35fc104770 2024-08-31T07:23:21.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.648+0000 7f3600c00640 1 -- 172.21.15.32:0/3286367769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35fc1a8d90 con 0x7f35fc104770 2024-08-31T07:23:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.651+0000 7f36079db740 1 Processor -- start 2024-08-31T07:23:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.651+0000 7f36079db740 1 -- start start 2024-08-31T07:23:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.651+0000 7f35f2a00640 1 -- 172.21.15.32:0/3286367769 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f35ec0037a0 con 0x7f35fc104770 2024-08-31T07:23:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f36079db740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad622070 0x562aad9c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f36079db740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f36079db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f36079db740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562aad93e770 con 0x562aad622070 2024-08-31T07:23:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f36079db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562aad92dd00 con 0x562aad601390 2024-08-31T07:23:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f36079db740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562aad931cb0 con 0x562aad5f91b0 2024-08-31T07:23:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f35f9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f35fa800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad622070 0x562aad9c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f35f9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37700/0 (socket says 172.21.15.32:37700) 2024-08-31T07:23:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f35f9e00640 1 -- 172.21.15.32:0/274083567 learned_addr learned my addr 172.21.15.32:0/274083567 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.652+0000 7f35f9400640 1 --2- 172.21.15.32:0/274083567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.653+0000 7f35f9e00640 1 -- 172.21.15.32:0/274083567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 msgr2=0x562aad9c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.653+0000 7f35f9e00640 1 --2- 172.21.15.32:0/274083567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.653+0000 7f35f9e00640 1 -- 172.21.15.32:0/274083567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad622070 msgr2=0x562aad9c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.653+0000 7f35f9e00640 1 --2- 172.21.15.32:0/274083567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad622070 0x562aad9c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.653+0000 7f35f9e00640 1 -- 172.21.15.32:0/274083567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aad68d650 con 0x562aad5f91b0 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.653+0000 7f35f9400640 1 --2- 172.21.15.32:0/274083567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.653+0000 7f35f9e00640 1 --2- 172.21.15.32:0/274083567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9c7e90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0e2f90 tx=0x7f35ec0ebb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69306d232fda2942 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.654+0000 7f35f2a00640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35e00b4ca0 0x7f35e00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.654+0000 7f35e7000640 1 -- 172.21.15.32:0/274083567 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0f4070 con 0x562aad5f91b0 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.654+0000 7f35e7000640 1 -- 172.21.15.32:0/274083567 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ec0abb00 con 0x562aad5f91b0 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.654+0000 7f35e7000640 1 -- 172.21.15.32:0/274083567 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0be290 con 0x562aad5f91b0 2024-08-31T07:23:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.654+0000 7f35f2a00640 1 -- 172.21.15.32:0/3286367769 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f35ec0e5910 con 0x7f35fc104770 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.654+0000 7f35fa800640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35e00b4ca0 0x7f35e00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f36079db740 1 -- 172.21.15.32:0/274083567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 msgr2=0x562aad9c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f36079db740 1 --2- 172.21.15.32:0/274083567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9c7e90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0e2f90 tx=0x7f35ec0ebb20 comp rx=0 tx=0).stop 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f36079db740 1 -- 172.21.15.32:0/274083567 shutdown_connections 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f36079db740 1 --2- 172.21.15.32:0/274083567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f36079db740 1 --2- 172.21.15.32:0/274083567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9c7e90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f36079db740 1 --2- 172.21.15.32:0/274083567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad622070 0x562aad9c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f36079db740 1 -- 172.21.15.32:0/274083567 >> 172.21.15.32:0/274083567 conn(0x562aad616460 msgr2=0x562aad9c5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f35fa800640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35e00b4ca0 0x7f35e00b71b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35f4004af0 tx=0x7f35f4005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f36079db740 1 -- 172.21.15.32:0/274083567 shutdown_connections 2024-08-31T07:23:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f36079db740 1 -- 172.21.15.32:0/274083567 wait complete. 2024-08-31T07:23:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.655+0000 7f36079db740 1 -- start start 2024-08-31T07:23:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.656+0000 7f36079db740 1 Processor -- start 2024-08-31T07:23:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.656+0000 7f36079db740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad5f91b0 0x562aad9d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.656+0000 7f36079db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.656+0000 7f36079db740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad622070 0x562aad9df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.656+0000 7f36079db740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562aad790700 con 0x562aad5f91b0 2024-08-31T07:23:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.656+0000 7f36079db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562aad8b4cb0 con 0x562aad601390 2024-08-31T07:23:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.656+0000 7f36079db740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562aad9d6130 con 0x562aad622070 2024-08-31T07:23:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.656+0000 7f36079db740 10 client.0 _ll_register_callbacks cb 0x562aad8da430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.656+0000 7f35f9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.656+0000 7f35f9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9d3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52724/0 (socket says 172.21.15.32:52724) 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.657+0000 7f35f9400640 1 -- 172.21.15.32:0/1600151743 learned_addr learned my addr 172.21.15.32:0/1600151743 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.657+0000 7f35f9e00640 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad5f91b0 0x562aad9d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.657+0000 7f35fa800640 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad622070 0x562aad9df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.657+0000 7f35f9400640 1 -- 172.21.15.32:0/1600151743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad622070 msgr2=0x562aad9df450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.657+0000 7f35f9400640 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad622070 0x562aad9df450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.657+0000 7f35f9400640 1 -- 172.21.15.32:0/1600151743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad5f91b0 msgr2=0x562aad9d3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.657+0000 7f35f9400640 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad5f91b0 0x562aad9d3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.657+0000 7f35f9400640 1 -- 172.21.15.32:0/1600151743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aad8f5870 con 0x562aad601390 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.657+0000 7f35f9400640 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9d3a80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f35e8012680 tx=0x7f35e8012b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94211ab7c242939 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.657+0000 7f35db400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.658+0000 7f35db400640 1 -- 172.21.15.32:0/1600151743 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8004020 con 0x562aad601390 2024-08-31T07:23:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.658+0000 7f35db400640 1 -- 172.21.15.32:0/1600151743 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f35e80041c0 con 0x562aad601390 2024-08-31T07:23:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.658+0000 7f36079db740 1 -- 172.21.15.32:0/1600151743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ec0abb00 con 0x562aad601390 2024-08-31T07:23:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.662+0000 7f35db400640 1 -- 172.21.15.32:0/1600151743 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f35e80043c0 con 0x562aad601390 2024-08-31T07:23:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.662+0000 7f36079db740 10 client.27680 fetch_fsmap learned FSMap version 43 2024-08-31T07:23:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.662+0000 7f36079db740 1 -- 172.21.15.32:0/1600151743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562aad9e21a0 con 0x562aad601390 2024-08-31T07:23:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.663+0000 7f35db400640 1 -- 172.21.15.32:0/1600151743 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ec0abb00 con 0x562aad601390 2024-08-31T07:23:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.663+0000 7f35db400640 1 -- 172.21.15.32:0/1600151743 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f35e8003b50 con 0x562aad601390 2024-08-31T07:23:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.663+0000 7f36079db740 10 client.27680 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T07:23:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.663+0000 7f36079db740 10 client.27680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:23:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.663+0000 7f36079db740 20 client.27680 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.663+0000 7f36079db740 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562aad5e09f0 0x562aad9e5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.663+0000 7f36079db740 4 client.27680 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.663+0000 7f36079db740 1 -- 172.21.15.32:0/1600151743 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aad9406c0 con 0x562aad5e09f0 2024-08-31T07:23:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.663+0000 7f35f9e00640 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562aad5e09f0 0x562aad9e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.664+0000 7f35f9e00640 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562aad5e09f0 0x562aad9e5600 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab06e371f68bd2c server_cookie=7ecbef9e0292e7fa in_seq=0 out_seq=0 2024-08-31T07:23:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.664+0000 7f35db400640 10 client.27680 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f35db400640 1 -- 172.21.15.32:0/1600151743 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562aad9406c0 con 0x562aad5e09f0 2024-08-31T07:23:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f35db400640 10 client.27680 handle_command_reply: tid=0 2024-08-31T07:23:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 client.27680 shutdown 2024-08-31T07:23:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 -- 172.21.15.32:0/1600151743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 msgr2=0x562aad9d3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9d3a80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f35e8012680 tx=0x7f35e8012b40 comp rx=0 tx=0).stop 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 -- 172.21.15.32:0/1600151743 shutdown_connections 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562aad5e09f0 0x562aad9e5600 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad622070 0x562aad9df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad601390 0x562aad9d3a80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 --2- 172.21.15.32:0/1600151743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad5f91b0 0x562aad9d3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 -- 172.21.15.32:0/1600151743 >> 172.21.15.32:0/1600151743 conn(0x562aad616460 msgr2=0x562aad9cab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f35db400640 0 client.27680 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 -- 172.21.15.32:0/1600151743 shutdown_connections 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.665+0000 7f36079db740 1 -- 172.21.15.32:0/1600151743 wait complete. 2024-08-31T07:23:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.666+0000 7f36079db740 20 client.27680 trim_cache size 0 max 16384 2024-08-31T07:23:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.686+0000 7f36079db740 1 Processor -- start 2024-08-31T07:23:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.686+0000 7f36079db740 1 -- start start 2024-08-31T07:23:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f36079db740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f36079db740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad601390 0x562aad9ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f36079db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad622070 0x562aad9cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f36079db740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1b1710 con 0x562aad601390 2024-08-31T07:23:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f36079db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1b32b0 con 0x562aad622070 2024-08-31T07:23:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f36079db740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1b1de0 con 0x562aad5f91b0 2024-08-31T07:23:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35f9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35fa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad622070 0x562aad9cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35f9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9df350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37724/0 (socket says 172.21.15.32:37724) 2024-08-31T07:23:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35f9e00640 1 -- 172.21.15.32:0/3762358058 learned_addr learned my addr 172.21.15.32:0/3762358058 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35f9400640 1 --2- 172.21.15.32:0/3762358058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad601390 0x562aad9ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35f9e00640 1 -- 172.21.15.32:0/3762358058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad622070 msgr2=0x562aad9cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:21.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35f9e00640 1 --2- 172.21.15.32:0/3762358058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad622070 0x562aad9cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35f9e00640 1 -- 172.21.15.32:0/3762358058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad601390 msgr2=0x562aad9ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:21.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35f9e00640 1 --2- 172.21.15.32:0/3762358058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad601390 0x562aad9ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35f9e00640 1 -- 172.21.15.32:0/3762358058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aad9bfaf0 con 0x562aad5f91b0 2024-08-31T07:23:21.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.687+0000 7f35f9400640 1 --2- 172.21.15.32:0/3762358058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad601390 0x562aad9ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:21.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.688+0000 7f35f9e00640 1 --2- 172.21.15.32:0/3762358058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9df350 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0058c0 tx=0x7f35ec003d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c67618334c8873cd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:21.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.688+0000 7f35e6600640 1 -- 172.21.15.32:0/3762358058 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0bb700 con 0x562aad5f91b0 2024-08-31T07:23:21.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.688+0000 7f35e6600640 1 -- 172.21.15.32:0/3762358058 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ec0abb00 con 0x562aad5f91b0 2024-08-31T07:23:21.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.688+0000 7f35e6600640 1 -- 172.21.15.32:0/3762358058 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0bbc90 con 0x562aad5f91b0 2024-08-31T07:23:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.689+0000 7f36079db740 1 -- 172.21.15.32:0/3762358058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 msgr2=0x562aad9df350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.689+0000 7f36079db740 1 --2- 172.21.15.32:0/3762358058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9df350 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0058c0 tx=0x7f35ec003d90 comp rx=0 tx=0).stop 2024-08-31T07:23:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.689+0000 7f36079db740 1 -- 172.21.15.32:0/3762358058 shutdown_connections 2024-08-31T07:23:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.689+0000 7f36079db740 1 --2- 172.21.15.32:0/3762358058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad622070 0x562aad9cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.689+0000 7f36079db740 1 --2- 172.21.15.32:0/3762358058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad601390 0x562aad9ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.689+0000 7f36079db740 1 --2- 172.21.15.32:0/3762358058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad5f91b0 0x562aad9df350 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.689+0000 7f36079db740 1 -- 172.21.15.32:0/3762358058 >> 172.21.15.32:0/3762358058 conn(0x562aad616460 msgr2=0x562aad9d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.689+0000 7f36079db740 1 -- 172.21.15.32:0/3762358058 shutdown_connections 2024-08-31T07:23:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.689+0000 7f36079db740 1 -- 172.21.15.32:0/3762358058 wait complete. 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.690+0000 7f36079db740 1 -- start start 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.690+0000 7f36079db740 1 Processor -- start 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.690+0000 7f36079db740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad5f91b0 0x562aad9db5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.690+0000 7f36079db740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad601390 0x562aad9dbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f36079db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad622070 0x562aad9d6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f36079db740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1b2bb0 con 0x562aad5f91b0 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f36079db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1b24b0 con 0x562aad622070 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f36079db740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1af710 con 0x562aad601390 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f36079db740 10 client.0 _ll_register_callbacks cb 0x7f35e8009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f35f9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad5f91b0 0x562aad9db5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f35f9400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad601390 0x562aad9dbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f35f9400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad601390 0x562aad9dbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37728/0 (socket says 172.21.15.32:37728) 2024-08-31T07:23:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f35f9400640 1 -- 172.21.15.32:0/3122303861 learned_addr learned my addr 172.21.15.32:0/3122303861 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:21.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f35fa800640 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad622070 0x562aad9d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f35f9400640 1 -- 172.21.15.32:0/3122303861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad622070 msgr2=0x562aad9d6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:21.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f35f9400640 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad622070 0x562aad9d6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f35f9400640 1 -- 172.21.15.32:0/3122303861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad5f91b0 msgr2=0x562aad9db5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:21.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f35f9400640 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad5f91b0 0x562aad9db5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.691+0000 7f35f9400640 1 -- 172.21.15.32:0/3122303861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aad9d3f00 con 0x562aad601390 2024-08-31T07:23:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.692+0000 7f35f9400640 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad601390 0x562aad9dbb00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f35e8003f70 tx=0x7f35e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f51897146e837ae server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.692+0000 7f35db400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:23:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.692+0000 7f35db400640 1 -- 172.21.15.32:0/3122303861 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e801cbc0 con 0x562aad601390 2024-08-31T07:23:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.692+0000 7f36079db740 1 -- 172.21.15.32:0/3122303861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35ec0abb00 con 0x562aad601390 2024-08-31T07:23:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.692+0000 7f35db400640 1 -- 172.21.15.32:0/3122303861 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f35e801cd60 con 0x562aad601390 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.692+0000 7f35db400640 1 -- 172.21.15.32:0/3122303861 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f35e8032020 con 0x562aad601390 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f36079db740 10 client.27688 fetch_fsmap learned FSMap version 43 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f36079db740 1 -- 172.21.15.32:0/3122303861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562aada50650 con 0x562aad601390 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f35db400640 1 -- 172.21.15.32:0/3122303861 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ec0abb00 con 0x562aad601390 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f35db400640 1 -- 172.21.15.32:0/3122303861 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f35e8003ac0 con 0x562aad601390 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f36079db740 10 client.27688 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f36079db740 10 client.27688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f36079db740 20 client.27688 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f36079db740 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562aad5b5da0 0x562aada52e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f36079db740 4 client.27688 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f36079db740 1 -- 172.21.15.32:0/3122303861 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562aad93c160 con 0x562aad5b5da0 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.693+0000 7f35f9e00640 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562aad5b5da0 0x562aada52e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.694+0000 7f35f9e00640 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562aad5b5da0 0x562aada52e70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a711c7286705cb server_cookie=af218223454a8998 in_seq=0 out_seq=0 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.694+0000 7f35db400640 10 client.27688 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.694+0000 7f35db400640 1 -- 172.21.15.32:0/3122303861 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x562aad93c160 con 0x562aad5b5da0 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.694+0000 7f35db400640 10 client.27688 handle_command_reply: tid=0 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.694+0000 7f36079db740 1 client.27688 shutdown 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 1 -- 172.21.15.32:0/3122303861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad601390 msgr2=0x562aad9dbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad601390 0x562aad9dbb00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f35e8003f70 tx=0x7f35e800f790 comp rx=0 tx=0).stop 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 1 -- 172.21.15.32:0/3122303861 shutdown_connections 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562aad5b5da0 0x562aada52e70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562aad622070 0x562aad9d6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562aad601390 0x562aad9dbb00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 1 --2- 172.21.15.32:0/3122303861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562aad5f91b0 0x562aad9db5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 1 -- 172.21.15.32:0/3122303861 >> 172.21.15.32:0/3122303861 conn(0x562aad616460 msgr2=0x562aad9c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f35db400640 0 client.27688 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 1 -- 172.21.15.32:0/3122303861 shutdown_connections 2024-08-31T07:23:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 1 -- 172.21.15.32:0/3122303861 wait complete. 2024-08-31T07:23:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.695+0000 7f36079db740 20 client.27688 trim_cache size 0 max 16384 2024-08-31T07:23:21.699 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:23:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:23:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:23:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:21.708 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:23:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 -- 172.21.15.32:0/3286367769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35e00b4ca0 msgr2=0x7f35e00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35e00b4ca0 0x7f35e00b71b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35f4004af0 tx=0x7f35f4005fd0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 -- 172.21.15.32:0/3286367769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35fc104770 msgr2=0x7f35fc1024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35fc104770 0x7f35fc1024b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00e9c0 tx=0x7f35ec00ee80 comp rx=0 tx=0).stop 2024-08-31T07:23:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 -- 172.21.15.32:0/3286367769 shutdown_connections 2024-08-31T07:23:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35e00b4ca0 0x7f35e00b71b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35fc10dbd0 0x7f35fc102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35fc1050c0 0x7f35fc1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 --2- 172.21.15.32:0/3286367769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35fc104770 0x7f35fc1024b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:21.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 -- 172.21.15.32:0/3286367769 >> 172.21.15.32:0/3286367769 conn(0x7f35fc0794e0 msgr2=0x7f35fc0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:21.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 -- 172.21.15.32:0/3286367769 shutdown_connections 2024-08-31T07:23:21.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:21.696+0000 7f3600c00640 1 -- 172.21.15.32:0/3286367769 wait complete. 2024-08-31T07:23:21.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 337045 2.93 MB/sec execute 430 sec latency 5914.455 ms 2024-08-31T07:23:22.727 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:23:22.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:23:22.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 338045 2.93 MB/sec execute 431 sec latency 20.192 ms 2024-08-31T07:23:22.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:22 smithi073 ceph-mon[21995]: pgmap v696: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 950 KiB/s wr, 105 op/s 2024-08-31T07:23:22.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:22 smithi084 ceph-mon[22387]: pgmap v696: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 950 KiB/s wr, 105 op/s 2024-08-31T07:23:23.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:22 smithi032 ceph-mon[17252]: pgmap v696: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 950 KiB/s wr, 105 op/s 2024-08-31T07:23:23.128 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:23.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 338670 2.93 MB/sec execute 432 sec latency 416.519 ms 2024-08-31T07:23:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.793+0000 7efe40400640 1 -- 172.21.15.32:0/3473256975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3810fe60 msgr2=0x7efe381122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.793+0000 7efe40400640 1 --2- 172.21.15.32:0/3473256975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3810fe60 0x7efe381122a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efe30009be0 tx=0x7efe3002a6c0 comp rx=0 tx=0).stop 2024-08-31T07:23:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.793+0000 7efe40400640 1 -- 172.21.15.32:0/3473256975 shutdown_connections 2024-08-31T07:23:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.793+0000 7efe40400640 1 --2- 172.21.15.32:0/3473256975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3810fe60 0x7efe381122a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.793+0000 7efe40400640 1 --2- 172.21.15.32:0/3473256975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe38075ea0 0x7efe3810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:23.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.793+0000 7efe40400640 1 --2- 172.21.15.32:0/3473256975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe38075550 0x7efe38075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:23.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.793+0000 7efe40400640 1 -- 172.21.15.32:0/3473256975 >> 172.21.15.32:0/3473256975 conn(0x7efe3807b5f0 msgr2=0x7efe3807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:23.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.793+0000 7efe40400640 1 -- 172.21.15.32:0/3473256975 shutdown_connections 2024-08-31T07:23:23.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.793+0000 7efe40400640 1 -- 172.21.15.32:0/3473256975 wait complete. 2024-08-31T07:23:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.794+0000 7efe40400640 1 Processor -- start 2024-08-31T07:23:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe40400640 1 -- start start 2024-08-31T07:23:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe40400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe38075550 0x7efe3819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe40400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe38075ea0 0x7efe3819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe40400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3810fe60 0x7efe381a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe40400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efe38115a10 con 0x7efe38075ea0 2024-08-31T07:23:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe40400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe38115890 con 0x7efe3810fe60 2024-08-31T07:23:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe40400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe38115b90 con 0x7efe38075550 2024-08-31T07:23:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe3d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe38075ea0 0x7efe3819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe3da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe38075550 0x7efe3819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe3d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe38075ea0 0x7efe3819c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33114/0 (socket says 172.21.15.32:33114) 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe3d000640 1 -- 172.21.15.32:0/3291787675 learned_addr learned my addr 172.21.15.32:0/3291787675 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.795+0000 7efe3da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe38075550 0x7efe3819c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37760/0 (socket says 172.21.15.32:37760) 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.796+0000 7efe3e400640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3810fe60 0x7efe381a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.796+0000 7efe3da00640 1 -- 172.21.15.32:0/3291787675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3810fe60 msgr2=0x7efe381a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.796+0000 7efe3da00640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3810fe60 0x7efe381a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.796+0000 7efe3da00640 1 -- 172.21.15.32:0/3291787675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe38075ea0 msgr2=0x7efe3819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.796+0000 7efe3da00640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe38075ea0 0x7efe3819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.796+0000 7efe3da00640 1 -- 172.21.15.32:0/3291787675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe381a2330 con 0x7efe38075550 2024-08-31T07:23:23.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.796+0000 7efe3d000640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe38075ea0 0x7efe3819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.796+0000 7efe3da00640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe38075550 0x7efe3819c3c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00e9f0 tx=0x7efe2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.796+0000 7efe36000640 1 -- 172.21.15.32:0/3291787675 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c00ccc0 con 0x7efe38075550 2024-08-31T07:23:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.797+0000 7efe40400640 1 -- 172.21.15.32:0/3291787675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe381a2620 con 0x7efe38075550 2024-08-31T07:23:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.797+0000 7efe36000640 1 -- 172.21.15.32:0/3291787675 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe2c005020 con 0x7efe38075550 2024-08-31T07:23:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.797+0000 7efe36000640 1 -- 172.21.15.32:0/3291787675 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c004a00 con 0x7efe38075550 2024-08-31T07:23:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.797+0000 7efe40400640 1 -- 172.21.15.32:0/3291787675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe381a8d80 con 0x7efe38075550 2024-08-31T07:23:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.798+0000 7efe40400640 1 -- 172.21.15.32:0/3291787675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe38075960 con 0x7efe38075550 2024-08-31T07:23:23.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.801+0000 7efe36000640 1 -- 172.21.15.32:0/3291787675 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7efe2c004ba0 con 0x7efe38075550 2024-08-31T07:23:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.802+0000 7efe36000640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe200b4ca0 0x7efe200b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.802+0000 7efe36000640 1 -- 172.21.15.32:0/3291787675 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7efe2c0e5420 con 0x7efe38075550 2024-08-31T07:23:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.802+0000 7efe3d000640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe200b4ca0 0x7efe200b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.803+0000 7efe3d000640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe200b4ca0 0x7efe200b71b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efe240096c0 tx=0x7efe24009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:23.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:23.803+0000 7efe36000640 1 -- 172.21.15.32:0/3291787675 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efe2c0ac260 con 0x7efe38075550 2024-08-31T07:23:24.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.037+0000 7efe40400640 1 -- 172.21.15.32:0/3291787675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe3819d4f0 con 0x7efe38075550 2024-08-31T07:23:24.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.038+0000 7efe36000640 1 -- 172.21.15.32:0/3291787675 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11962 (secure 0 0 0) 0x7efe2c0abb60 con 0x7efe38075550 2024-08-31T07:23:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:23:24.039 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":43,"btime":"2024-08-31T07:23:11:637919+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:23:10.951692+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:23:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 43 2024-08-31T07:23:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.040+0000 7efe40400640 1 -- 172.21.15.32:0/3291787675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe200b4ca0 msgr2=0x7efe200b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.040+0000 7efe40400640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe200b4ca0 0x7efe200b71b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efe240096c0 tx=0x7efe24009260 comp rx=0 tx=0).stop 2024-08-31T07:23:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.040+0000 7efe40400640 1 -- 172.21.15.32:0/3291787675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe38075550 msgr2=0x7efe3819c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.040+0000 7efe40400640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe38075550 0x7efe3819c3c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00e9f0 tx=0x7efe2c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:23:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.040+0000 7efe40400640 1 -- 172.21.15.32:0/3291787675 shutdown_connections 2024-08-31T07:23:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.040+0000 7efe40400640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efe200b4ca0 0x7efe200b71b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.040+0000 7efe40400640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3810fe60 0x7efe381a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.040+0000 7efe40400640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe38075ea0 0x7efe3819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.040+0000 7efe40400640 1 --2- 172.21.15.32:0/3291787675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe38075550 0x7efe3819c3c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.040+0000 7efe40400640 1 -- 172.21.15.32:0/3291787675 >> 172.21.15.32:0/3291787675 conn(0x7efe3807b5f0 msgr2=0x7efe381003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.041+0000 7efe40400640 1 -- 172.21.15.32:0/3291787675 shutdown_connections 2024-08-31T07:23:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:24.041+0000 7efe40400640 1 -- 172.21.15.32:0/3291787675 wait complete. 2024-08-31T07:23:24.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 338670 2.93 MB/sec execute 433 sec latency 1416.591 ms 2024-08-31T07:23:24.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:24 smithi032 ceph-mon[17252]: pgmap v697: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 492 KiB/s wr, 68 op/s 2024-08-31T07:23:24.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:24 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3291787675' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:24 smithi073 ceph-mon[21995]: pgmap v697: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 492 KiB/s wr, 68 op/s 2024-08-31T07:23:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:24 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3291787675' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:24.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:24 smithi084 ceph-mon[22387]: pgmap v697: 97 pgs: 97 active+clean; 257 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 492 KiB/s wr, 68 op/s 2024-08-31T07:23:24.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:24 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3291787675' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:25.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:23:25.302 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-31T07:23:25.303 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-08-31T07:23:25.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:23:25.467 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:25.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 339974 2.94 MB/sec execute 434 sec latency 1770.642 ms 2024-08-31T07:23:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.175+0000 7fef97e00640 1 -- 172.21.15.32:0/2135516757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef98103750 msgr2=0x7fef98103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.175+0000 7fef97e00640 1 --2- 172.21.15.32:0/2135516757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef98103750 0x7fef98103b60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fef88009900 tx=0x7fef8802a410 comp rx=0 tx=0).stop 2024-08-31T07:23:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.175+0000 7fef97e00640 1 -- 172.21.15.32:0/2135516757 shutdown_connections 2024-08-31T07:23:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.175+0000 7fef97e00640 1 --2- 172.21.15.32:0/2135516757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef981099e0 0x7fef98110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.175+0000 7fef97e00640 1 --2- 172.21.15.32:0/2135516757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef98109090 0x7fef981094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.175+0000 7fef97e00640 1 --2- 172.21.15.32:0/2135516757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef98103750 0x7fef98103b60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.175+0000 7fef97e00640 1 -- 172.21.15.32:0/2135516757 >> 172.21.15.32:0/2135516757 conn(0x7fef980ff1e0 msgr2=0x7fef98101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.175+0000 7fef97e00640 1 -- 172.21.15.32:0/2135516757 shutdown_connections 2024-08-31T07:23:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.175+0000 7fef97e00640 1 -- 172.21.15.32:0/2135516757 wait complete. 2024-08-31T07:23:26.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.176+0000 7fef97e00640 1 Processor -- start 2024-08-31T07:23:26.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.176+0000 7fef97e00640 1 -- start start 2024-08-31T07:23:26.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef98103750 0x7fef9819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef98109090 0x7fef9819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef97e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef981099e0 0x7fef981a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef97e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef98113690 con 0x7fef98103750 2024-08-31T07:23:26.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef97e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fef98113510 con 0x7fef981099e0 2024-08-31T07:23:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef97e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fef98113810 con 0x7fef98109090 2024-08-31T07:23:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef96000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef98109090 0x7fef9819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef96a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef98103750 0x7fef9819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef96000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef98109090 0x7fef9819c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37632/0 (socket says 172.21.15.32:37632) 2024-08-31T07:23:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef96000640 1 -- 172.21.15.32:0/18790946 learned_addr learned my addr 172.21.15.32:0/18790946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.177+0000 7fef97400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef981099e0 0x7fef981a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef96000640 1 -- 172.21.15.32:0/18790946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef981099e0 msgr2=0x7fef981a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef96000640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef981099e0 0x7fef981a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef96000640 1 -- 172.21.15.32:0/18790946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef98103750 msgr2=0x7fef9819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef96000640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef98103750 0x7fef9819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef96000640 1 -- 172.21.15.32:0/18790946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef981a2250 con 0x7fef98109090 2024-08-31T07:23:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef96000640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef98109090 0x7fef9819c880 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fef8400e9f0 tx=0x7fef8400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef8f400640 1 -- 172.21.15.32:0/18790946 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8400ccc0 con 0x7fef98109090 2024-08-31T07:23:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef97e00640 1 -- 172.21.15.32:0/18790946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef981a2540 con 0x7fef98109090 2024-08-31T07:23:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef8f400640 1 -- 172.21.15.32:0/18790946 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef840108e0 con 0x7fef98109090 2024-08-31T07:23:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef8f400640 1 -- 172.21.15.32:0/18790946 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84018860 con 0x7fef98109090 2024-08-31T07:23:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.178+0000 7fef97e00640 1 -- 172.21.15.32:0/18790946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef981a8ca0 con 0x7fef98109090 2024-08-31T07:23:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.183+0000 7fefa428f740 1 Processor -- start 2024-08-31T07:23:26.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.183+0000 7fefa428f740 1 -- start start 2024-08-31T07:23:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.183+0000 7fefa428f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 0x55e739dcea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.183+0000 7fefa428f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dcefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.183+0000 7fefa428f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 0x55e739dcf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.183+0000 7fefa428f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e739d457b0 con 0x55e739a29070 2024-08-31T07:23:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.183+0000 7fefa428f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e739d34d40 con 0x55e739a001b0 2024-08-31T07:23:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.183+0000 7fefa428f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e739d38cf0 con 0x55e739a08390 2024-08-31T07:23:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef96a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dcefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef96a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dcefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53372/0 (socket says 172.21.15.32:53372) 2024-08-31T07:23:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef96a00640 1 -- 172.21.15.32:0/778706885 learned_addr learned my addr 172.21.15.32:0/778706885 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef97400640 1 --2- 172.21.15.32:0/778706885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 0x55e739dcea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef8f400640 1 -- 172.21.15.32:0/18790946 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fef84010430 con 0x7fef98109090 2024-08-31T07:23:26.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef96a00640 1 -- 172.21.15.32:0/778706885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 msgr2=0x55e739dcf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:26.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef96a00640 1 --2- 172.21.15.32:0/778706885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 0x55e739dcf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef96a00640 1 -- 172.21.15.32:0/778706885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 msgr2=0x55e739dcea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef96a00640 1 --2- 172.21.15.32:0/778706885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 0x55e739dcea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef96a00640 1 -- 172.21.15.32:0/778706885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e739a94650 con 0x55e739a001b0 2024-08-31T07:23:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.184+0000 7fef97400640 1 --2- 172.21.15.32:0/778706885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 0x55e739dcea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.185+0000 7fef96a00640 1 --2- 172.21.15.32:0/778706885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dcefb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef88009900 tx=0x7fef88032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb5eb4223fad0e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.185+0000 7fef83400640 1 -- 172.21.15.32:0/778706885 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88040070 con 0x55e739a001b0 2024-08-31T07:23:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.185+0000 7fef83400640 1 -- 172.21.15.32:0/778706885 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef88004030 con 0x55e739a001b0 2024-08-31T07:23:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.185+0000 7fef83400640 1 -- 172.21.15.32:0/778706885 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88035040 con 0x55e739a001b0 2024-08-31T07:23:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.186+0000 7fef8f400640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef7c0b4d20 0x7fef7c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.186+0000 7fef8f400640 1 -- 172.21.15.32:0/18790946 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fef840e57e0 con 0x7fef98109090 2024-08-31T07:23:26.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.186+0000 7fefa428f740 1 -- 172.21.15.32:0/778706885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 msgr2=0x55e739dcefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:26.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.186+0000 7fefa428f740 1 --2- 172.21.15.32:0/778706885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dcefb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef88009900 tx=0x7fef88032b90 comp rx=0 tx=0).stop 2024-08-31T07:23:26.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.186+0000 7fef97400640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef7c0b4d20 0x7fef7c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.186+0000 7fefa428f740 1 -- 172.21.15.32:0/778706885 shutdown_connections 2024-08-31T07:23:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.186+0000 7fefa428f740 1 --2- 172.21.15.32:0/778706885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 0x55e739dcf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.186+0000 7fefa428f740 1 --2- 172.21.15.32:0/778706885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dcefb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.186+0000 7fefa428f740 1 --2- 172.21.15.32:0/778706885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 0x55e739dcea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.186+0000 7fefa428f740 1 -- 172.21.15.32:0/778706885 >> 172.21.15.32:0/778706885 conn(0x55e739a1d460 msgr2=0x55e739dcc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.187+0000 7fefa428f740 1 -- 172.21.15.32:0/778706885 shutdown_connections 2024-08-31T07:23:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.187+0000 7fefa428f740 1 -- 172.21.15.32:0/778706885 wait complete. 2024-08-31T07:23:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.187+0000 7fef97400640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef7c0b4d20 0x7fef7c0b7230 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fef90005fd0 tx=0x7fef90009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.187+0000 7fefa428f740 1 -- start start 2024-08-31T07:23:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fefa428f740 1 Processor -- start 2024-08-31T07:23:26.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fefa428f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dda6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fefa428f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a08390 0x55e739ddac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fefa428f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 0x55e739de6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fefa428f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e739b97700 con 0x55e739a08390 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fefa428f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e739cbbcf0 con 0x55e739a001b0 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fefa428f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e739ddd1f0 con 0x55e739a29070 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fefa428f740 10 client.0 _ll_register_callbacks cb 0x55e739ce1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fef96a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dda6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fef97400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 0x55e739de6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fef96a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dda6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53374/0 (socket says 172.21.15.32:53374) 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.188+0000 7fef96a00640 1 -- 172.21.15.32:0/2581884703 learned_addr learned my addr 172.21.15.32:0/2581884703 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.189+0000 7fef96000640 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a08390 0x55e739ddac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.189+0000 7fef96a00640 1 -- 172.21.15.32:0/2581884703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 msgr2=0x55e739de6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.189+0000 7fef96a00640 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 0x55e739de6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.189+0000 7fef96a00640 1 -- 172.21.15.32:0/2581884703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a08390 msgr2=0x55e739ddac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.189+0000 7fef96a00640 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a08390 0x55e739ddac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.189+0000 7fef96a00640 1 -- 172.21.15.32:0/2581884703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e739cfc8b0 con 0x55e739a001b0 2024-08-31T07:23:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.189+0000 7fef96a00640 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dda6d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fef8802a930 tx=0x7fef8802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99273cb7596bdc2f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.189+0000 7fef77e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:23:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.189+0000 7fef77e00640 1 -- 172.21.15.32:0/2581884703 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88035070 con 0x55e739a001b0 2024-08-31T07:23:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.190+0000 7fef77e00640 1 -- 172.21.15.32:0/2581884703 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fef88005460 con 0x55e739a001b0 2024-08-31T07:23:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.190+0000 7fefa428f740 1 -- 172.21.15.32:0/2581884703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef88004030 con 0x55e739a001b0 2024-08-31T07:23:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.190+0000 7fef77e00640 1 -- 172.21.15.32:0/2581884703 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fef88040040 con 0x55e739a001b0 2024-08-31T07:23:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.190+0000 7fefa428f740 10 client.27704 fetch_fsmap learned FSMap version 43 2024-08-31T07:23:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.190+0000 7fefa428f740 1 -- 172.21.15.32:0/2581884703 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e739de9280 con 0x55e739a001b0 2024-08-31T07:23:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.190+0000 7fef77e00640 1 -- 172.21.15.32:0/2581884703 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef88004030 con 0x55e739a001b0 2024-08-31T07:23:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.190+0000 7fef77e00640 1 -- 172.21.15.32:0/2581884703 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7fef88004cf0 con 0x55e739a001b0 2024-08-31T07:23:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.190+0000 7fefa428f740 10 client.27704 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T07:23:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.190+0000 7fefa428f740 10 client.27704 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:23:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.190+0000 7fefa428f740 20 client.27704 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.191+0000 7fefa428f740 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e7399e79f0 0x55e739dec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.191+0000 7fefa428f740 4 client.27704 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.191+0000 7fefa428f740 1 -- 172.21.15.32:0/2581884703 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e739d47700 con 0x55e7399e79f0 2024-08-31T07:23:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.191+0000 7fef96000640 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e7399e79f0 0x55e739dec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.191+0000 7fef96000640 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e7399e79f0 0x55e739dec5e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a2f29c64402b153d server_cookie=a95639581259ce in_seq=0 out_seq=0 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.191+0000 7fef77e00640 10 client.27704 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.192+0000 7fef77e00640 1 -- 172.21.15.32:0/2581884703 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e739d47700 con 0x55e7399e79f0 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.192+0000 7fef77e00640 10 client.27704 handle_command_reply: tid=0 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 client.27704 shutdown 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 -- 172.21.15.32:0/2581884703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 msgr2=0x55e739dda6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dda6d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fef8802a930 tx=0x7fef8802a410 comp rx=0 tx=0).stop 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 -- 172.21.15.32:0/2581884703 shutdown_connections 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e7399e79f0 0x55e739dec5e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 0x55e739de6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a08390 0x55e739ddac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 --2- 172.21.15.32:0/2581884703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dda6d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 -- 172.21.15.32:0/2581884703 >> 172.21.15.32:0/2581884703 conn(0x55e739a1d460 msgr2=0x55e739dd1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fef77e00640 0 client.27704 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:23:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 -- 172.21.15.32:0/2581884703 shutdown_connections 2024-08-31T07:23:26.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 1 -- 172.21.15.32:0/2581884703 wait complete. 2024-08-31T07:23:26.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.193+0000 7fefa428f740 20 client.27704 trim_cache size 0 max 16384 2024-08-31T07:23:26.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.209+0000 7fefa428f740 1 Processor -- start 2024-08-31T07:23:26.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.210+0000 7fefa428f740 1 -- start start 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.210+0000 7fefa428f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dd5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.210+0000 7fefa428f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a08390 0x55e739dd01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.210+0000 7fefa428f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 0x55e739dcbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.210+0000 7fefa428f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef981b1440 con 0x55e739a08390 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.210+0000 7fefa428f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fef981b2fe0 con 0x55e739a001b0 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.210+0000 7fefa428f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fef981b1b10 con 0x55e739a29070 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.211+0000 7fef97400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 0x55e739dcbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.211+0000 7fef97400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 0x55e739dcbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37644/0 (socket says 172.21.15.32:37644) 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.211+0000 7fef97400640 1 -- 172.21.15.32:0/3465881214 learned_addr learned my addr 172.21.15.32:0/3465881214 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.211+0000 7fef96a00640 1 --2- 172.21.15.32:0/3465881214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dd5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.211+0000 7fef96000640 1 --2- 172.21.15.32:0/3465881214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a08390 0x55e739dd01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.211+0000 7fef97400640 1 -- 172.21.15.32:0/3465881214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 msgr2=0x55e739dd5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.211+0000 7fef97400640 1 --2- 172.21.15.32:0/3465881214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dd5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.211+0000 7fef97400640 1 -- 172.21.15.32:0/3465881214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a08390 msgr2=0x55e739dd01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.211+0000 7fef97400640 1 --2- 172.21.15.32:0/3465881214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a08390 0x55e739dd01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.211+0000 7fef97400640 1 -- 172.21.15.32:0/3465881214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e739dd1300 con 0x55e739a29070 2024-08-31T07:23:26.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.212+0000 7fef97400640 1 --2- 172.21.15.32:0/3465881214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 0x55e739dcbf50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fef90004070 tx=0x7fef9001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c6f4aa32b5ea32d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.212+0000 7fef82a00640 1 -- 172.21.15.32:0/3465881214 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90004b20 con 0x55e739a29070 2024-08-31T07:23:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.212+0000 7fef82a00640 1 -- 172.21.15.32:0/3465881214 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef90004cc0 con 0x55e739a29070 2024-08-31T07:23:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.212+0000 7fef82a00640 1 -- 172.21.15.32:0/3465881214 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90005540 con 0x55e739a29070 2024-08-31T07:23:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.213+0000 7fefa428f740 1 -- 172.21.15.32:0/3465881214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 msgr2=0x55e739dcbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.213+0000 7fefa428f740 1 --2- 172.21.15.32:0/3465881214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 0x55e739dcbf50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fef90004070 tx=0x7fef9001b000 comp rx=0 tx=0).stop 2024-08-31T07:23:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.213+0000 7fefa428f740 1 -- 172.21.15.32:0/3465881214 shutdown_connections 2024-08-31T07:23:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.213+0000 7fefa428f740 1 --2- 172.21.15.32:0/3465881214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a29070 0x55e739dcbf50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.213+0000 7fefa428f740 1 --2- 172.21.15.32:0/3465881214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a08390 0x55e739dd01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.213+0000 7fefa428f740 1 --2- 172.21.15.32:0/3465881214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739dd5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.213+0000 7fefa428f740 1 -- 172.21.15.32:0/3465881214 >> 172.21.15.32:0/3465881214 conn(0x55e739a1d460 msgr2=0x55e739dcfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.213+0000 7fefa428f740 1 -- 172.21.15.32:0/3465881214 shutdown_connections 2024-08-31T07:23:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.213+0000 7fefa428f740 1 -- 172.21.15.32:0/3465881214 wait complete. 2024-08-31T07:23:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.214+0000 7fefa428f740 1 -- start start 2024-08-31T07:23:26.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.214+0000 7fefa428f740 1 Processor -- start 2024-08-31T07:23:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fefa428f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739defe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fefa428f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 0x55e739ded340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fefa428f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 0x55e739ded900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fefa428f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef981b28e0 con 0x55e739a29070 2024-08-31T07:23:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fefa428f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fef981b21e0 con 0x55e739a001b0 2024-08-31T07:23:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fefa428f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fef981af440 con 0x55e739a08390 2024-08-31T07:23:26.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fefa428f740 10 client.0 _ll_register_callbacks cb 0x7fef88032d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef96000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 0x55e739ded340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef96000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 0x55e739ded340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37650/0 (socket says 172.21.15.32:37650) 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef96000640 1 -- 172.21.15.32:0/2372852730 learned_addr learned my addr 172.21.15.32:0/2372852730 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef96a00640 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739defe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef97400640 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 0x55e739ded900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef96000640 1 -- 172.21.15.32:0/2372852730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 msgr2=0x55e739defe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef96000640 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739defe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef96000640 1 -- 172.21.15.32:0/2372852730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 msgr2=0x55e739ded900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef96000640 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 0x55e739ded900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef96000640 1 -- 172.21.15.32:0/2372852730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e739dee090 con 0x55e739a08390 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef96a00640 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739defe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.215+0000 7fef97400640 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 0x55e739ded900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:26.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.216+0000 7fef96000640 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 0x55e739ded340 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fef8400e9c0 tx=0x7fef8400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d88d5756edff6c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.216+0000 7fef77e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:23:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.216+0000 7fef77e00640 1 -- 172.21.15.32:0/2372852730 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef840b0730 con 0x55e739a08390 2024-08-31T07:23:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.216+0000 7fef77e00640 1 -- 172.21.15.32:0/2372852730 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fef840a8200 con 0x55e739a08390 2024-08-31T07:23:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.216+0000 7fefa428f740 1 -- 172.21.15.32:0/2372852730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef90004cc0 con 0x55e739a08390 2024-08-31T07:23:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.216+0000 7fef77e00640 1 -- 172.21.15.32:0/2372852730 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fef840b51e0 con 0x55e739a08390 2024-08-31T07:23:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.216+0000 7fefa428f740 10 client.27709 fetch_fsmap learned FSMap version 43 2024-08-31T07:23:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.216+0000 7fefa428f740 1 -- 172.21.15.32:0/2372852730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e739e56d80 con 0x55e739a08390 2024-08-31T07:23:26.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.217+0000 7fef77e00640 1 -- 172.21.15.32:0/2372852730 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef90004cc0 con 0x55e739a08390 2024-08-31T07:23:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.218+0000 7fef77e00640 1 -- 172.21.15.32:0/2372852730 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7fef840970d0 con 0x55e739a08390 2024-08-31T07:23:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.218+0000 7fefa428f740 10 client.27709 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T07:23:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.218+0000 7fefa428f740 10 client.27709 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:23:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.218+0000 7fefa428f740 20 client.27709 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.218+0000 7fefa428f740 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e7399bcda0 0x55e739e59470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.218+0000 7fefa428f740 4 client.27709 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:23:26.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.218+0000 7fefa428f740 1 -- 172.21.15.32:0/2372852730 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e739d431a0 con 0x55e7399bcda0 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.218+0000 7fef96a00640 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e7399bcda0 0x55e739e59470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fef96a00640 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e7399bcda0 0x55e739e59470 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d24978f76227063c server_cookie=c9132c5fef43527e in_seq=0 out_seq=0 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fef77e00640 10 client.27709 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fef77e00640 1 -- 172.21.15.32:0/2372852730 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e739d431a0 con 0x55e7399bcda0 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fef77e00640 10 client.27709 handle_command_reply: tid=0 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fefa428f740 1 client.27709 shutdown 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fefa428f740 1 -- 172.21.15.32:0/2372852730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 msgr2=0x55e739ded340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fefa428f740 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 0x55e739ded340 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fef8400e9c0 tx=0x7fef8400cef0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fefa428f740 1 -- 172.21.15.32:0/2372852730 shutdown_connections 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fefa428f740 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55e7399bcda0 0x55e739e59470 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fefa428f740 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e739a29070 0x55e739ded900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fef77e00640 0 client.27709 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fefa428f740 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e739a08390 0x55e739ded340 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fefa428f740 1 --2- 172.21.15.32:0/2372852730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e739a001b0 0x55e739defe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.219+0000 7fefa428f740 1 -- 172.21.15.32:0/2372852730 >> 172.21.15.32:0/2372852730 conn(0x55e739a1d460 msgr2=0x55e739dd0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:26.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.220+0000 7fefa428f740 1 -- 172.21.15.32:0/2372852730 shutdown_connections 2024-08-31T07:23:26.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.220+0000 7fefa428f740 1 -- 172.21.15.32:0/2372852730 wait complete. 2024-08-31T07:23:26.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.220+0000 7fefa428f740 20 client.27709 trim_cache size 0 max 16384 2024-08-31T07:23:26.223 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.221+0000 7fef97e00640 1 -- 172.21.15.32:0/18790946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef7c0b4d20 msgr2=0x7fef7c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.221+0000 7fef97e00640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef7c0b4d20 0x7fef7c0b7230 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fef90005fd0 tx=0x7fef90009500 comp rx=0 tx=0).stop 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.221+0000 7fef97e00640 1 -- 172.21.15.32:0/18790946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef98109090 msgr2=0x7fef9819c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.221+0000 7fef97e00640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef98109090 0x7fef9819c880 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fef8400e9f0 tx=0x7fef8400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.221+0000 7fef97e00640 1 -- 172.21.15.32:0/18790946 shutdown_connections 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.221+0000 7fef97e00640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fef7c0b4d20 0x7fef7c0b7230 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.221+0000 7fef97e00640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef981099e0 0x7fef981a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.221+0000 7fef97e00640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef98109090 0x7fef9819c880 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.221+0000 7fef97e00640 1 --2- 172.21.15.32:0/18790946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef98103750 0x7fef9819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.221+0000 7fef97e00640 1 -- 172.21.15.32:0/18790946 >> 172.21.15.32:0/18790946 conn(0x7fef980ff1e0 msgr2=0x7fef981087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.222+0000 7fef97e00640 1 -- 172.21.15.32:0/18790946 shutdown_connections 2024-08-31T07:23:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:26.222+0000 7fef97e00640 1 -- 172.21.15.32:0/18790946 wait complete. 2024-08-31T07:23:26.713 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:26.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 341117 2.94 MB/sec execute 435 sec latency 13.374 ms 2024-08-31T07:23:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:26 smithi073 ceph-mon[21995]: pgmap v698: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 678 KiB/s wr, 79 op/s 2024-08-31T07:23:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:26 smithi084 ceph-mon[22387]: pgmap v698: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 678 KiB/s wr, 79 op/s 2024-08-31T07:23:27.009 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:26 smithi032 ceph-mon[17252]: pgmap v698: 97 pgs: 97 active+clean; 260 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 678 KiB/s wr, 79 op/s 2024-08-31T07:23:27.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:23:27.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 342584 2.95 MB/sec execute 436 sec latency 81.096 ms 2024-08-31T07:23:27.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:27 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:23:27.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:27 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:23:28.044 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:27 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:23:28.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.096+0000 7f36cb200640 1 -- 172.21.15.32:0/2673267344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c4075b70 msgr2=0x7f36c4075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:28.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.096+0000 7f36cb200640 1 --2- 172.21.15.32:0/2673267344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c4075b70 0x7f36c4075f80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009900 tx=0x7f36b402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.096+0000 7f36cb200640 1 -- 172.21.15.32:0/2673267344 shutdown_connections 2024-08-31T07:23:28.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.096+0000 7f36cb200640 1 --2- 172.21.15.32:0/2673267344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36c4114020 0x7f36c41164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.096+0000 7f36cb200640 1 --2- 172.21.15.32:0/2673267344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c4075b70 0x7f36c4075f80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.096+0000 7f36cb200640 1 --2- 172.21.15.32:0/2673267344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c4077120 0x7f36c4075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.096+0000 7f36cb200640 1 -- 172.21.15.32:0/2673267344 >> 172.21.15.32:0/2673267344 conn(0x7f36c40ff1e0 msgr2=0x7f36c4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:28.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.096+0000 7f36cb200640 1 -- 172.21.15.32:0/2673267344 shutdown_connections 2024-08-31T07:23:28.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.096+0000 7f36cb200640 1 -- 172.21.15.32:0/2673267344 wait complete. 2024-08-31T07:23:28.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.098+0000 7f36cb200640 1 Processor -- start 2024-08-31T07:23:28.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.098+0000 7f36cb200640 1 -- start start 2024-08-31T07:23:28.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.098+0000 7f36cb200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c4075b70 0x7f36c41a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.098+0000 7f36cb200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36c4077120 0x7f36c41a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.098+0000 7f36cb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c4114020 0x7f36c41a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.098+0000 7f36cb200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4119cb0 con 0x7f36c4075b70 2024-08-31T07:23:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.098+0000 7f36cb200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4119b30 con 0x7f36c4114020 2024-08-31T07:23:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.098+0000 7f36cb200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4119e30 con 0x7f36c4077120 2024-08-31T07:23:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36ca800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c4114020 0x7f36c41a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36ca800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c4114020 0x7f36c41a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53404/0 (socket says 172.21.15.32:53404) 2024-08-31T07:23:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36ca800640 1 -- 172.21.15.32:0/1094711492 learned_addr learned my addr 172.21.15.32:0/1094711492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36c9e00640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c4075b70 0x7f36c41a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36c9400640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36c4077120 0x7f36c41a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:28.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36ca800640 1 -- 172.21.15.32:0/1094711492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36c4077120 msgr2=0x7f36c41a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:28.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36ca800640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36c4077120 0x7f36c41a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36ca800640 1 -- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c4075b70 msgr2=0x7f36c41a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:28.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36ca800640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c4075b70 0x7f36c41a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36ca800640 1 -- 172.21.15.32:0/1094711492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c41a65f0 con 0x7f36c4114020 2024-08-31T07:23:28.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.099+0000 7f36c9e00640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c4075b70 0x7f36c41a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:28.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.100+0000 7f36ca800640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c4114020 0x7f36c41a5ec0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f36c000eb10 tx=0x7f36c000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:28.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.100+0000 7f36bea00640 1 -- 172.21.15.32:0/1094711492 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0004b20 con 0x7f36c4114020 2024-08-31T07:23:28.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.100+0000 7f36bea00640 1 -- 172.21.15.32:0/1094711492 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c0004cc0 con 0x7f36c4114020 2024-08-31T07:23:28.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.100+0000 7f36bea00640 1 -- 172.21.15.32:0/1094711492 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0010670 con 0x7f36c4114020 2024-08-31T07:23:28.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.100+0000 7f36cb200640 1 -- 172.21.15.32:0/1094711492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c41a68e0 con 0x7f36c4114020 2024-08-31T07:23:28.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.100+0000 7f36cb200640 1 -- 172.21.15.32:0/1094711492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c41ad040 con 0x7f36c4114020 2024-08-31T07:23:28.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.102+0000 7f36cb200640 1 -- 172.21.15.32:0/1094711492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c4076a60 con 0x7f36c4114020 2024-08-31T07:23:28.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.104+0000 7f36bea00640 1 -- 172.21.15.32:0/1094711492 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f36c0005020 con 0x7f36c4114020 2024-08-31T07:23:28.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.106+0000 7f36bea00640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f36ac0b4d20 0x7f36ac0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:28.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.106+0000 7f36bea00640 1 -- 172.21.15.32:0/1094711492 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f36c00e5af0 con 0x7f36c4114020 2024-08-31T07:23:28.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.106+0000 7f36c9e00640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f36ac0b4d20 0x7f36ac0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:28.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.107+0000 7f36c9e00640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f36ac0b4d20 0x7f36ac0b7230 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009c30 tx=0x7f36b8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:28.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.110+0000 7f36bea00640 1 -- 172.21.15.32:0/1094711492 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f36c00ac8d0 con 0x7f36c4114020 2024-08-31T07:23:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:28.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.390+0000 7f36cb200640 1 -- 172.21.15.32:0/1094711492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36c402e430 con 0x7f36c4114020 2024-08-31T07:23:28.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.391+0000 7f36bea00640 1 -- 172.21.15.32:0/1094711492 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11962 (secure 0 0 0) 0x7f36c00ac1d0 con 0x7f36c4114020 2024-08-31T07:23:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:23:28.392 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":43,"btime":"2024-08-31T07:23:11:637919+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:23:10.951692+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:23:28.393 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 43 2024-08-31T07:23:28.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 -- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f36ac0b4d20 msgr2=0x7f36ac0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:28.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f36ac0b4d20 0x7f36ac0b7230 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009c30 tx=0x7f36b8005f90 comp rx=0 tx=0).stop 2024-08-31T07:23:28.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 -- 172.21.15.32:0/1094711492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c4114020 msgr2=0x7f36c41a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:28.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c4114020 0x7f36c41a5ec0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f36c000eb10 tx=0x7f36c000c2d0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 -- 172.21.15.32:0/1094711492 shutdown_connections 2024-08-31T07:23:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f36ac0b4d20 0x7f36ac0b7230 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c4114020 0x7f36c41a5ec0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36c4077120 0x7f36c41a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 --2- 172.21.15.32:0/1094711492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c4075b70 0x7f36c41a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 -- 172.21.15.32:0/1094711492 >> 172.21.15.32:0/1094711492 conn(0x7f36c40ff1e0 msgr2=0x7f36c410ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 -- 172.21.15.32:0/1094711492 shutdown_connections 2024-08-31T07:23:28.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:28.394+0000 7f36cb200640 1 -- 172.21.15.32:0/1094711492 wait complete. 2024-08-31T07:23:28.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 344309 2.97 MB/sec execute 437 sec latency 14.583 ms 2024-08-31T07:23:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:28 smithi073 ceph-mon[21995]: pgmap v699: 97 pgs: 97 active+clean; 262 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 691 KiB/s wr, 87 op/s 2024-08-31T07:23:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:28 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1094711492' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:23:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:23:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:28 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:23:28.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:28 smithi032 ceph-mon[17252]: pgmap v699: 97 pgs: 97 active+clean; 262 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 691 KiB/s wr, 87 op/s 2024-08-31T07:23:28.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:28 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1094711492' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:28.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:23:28.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:23:28.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:28 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:23:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:28 smithi084 ceph-mon[22387]: pgmap v699: 97 pgs: 97 active+clean; 262 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 691 KiB/s wr, 87 op/s 2024-08-31T07:23:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:28 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1094711492' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:23:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:23:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:28 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:23:29.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.006+0000 7ff1f3200640 1 -- 172.21.15.32:0/2818252247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1f4104570 msgr2=0x7ff1f4104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:29.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.006+0000 7ff1f3200640 1 --2- 172.21.15.32:0/2818252247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1f4104570 0x7ff1f4104980 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009900 tx=0x7ff1e402a410 comp rx=0 tx=0).stop 2024-08-31T07:23:29.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.006+0000 7ff1f3200640 1 -- 172.21.15.32:0/2818252247 shutdown_connections 2024-08-31T07:23:29.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.006+0000 7ff1f3200640 1 --2- 172.21.15.32:0/2818252247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f410d9d0 0x7ff1f410fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.006+0000 7ff1f3200640 1 --2- 172.21.15.32:0/2818252247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f4104ec0 0x7ff1f410d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.006+0000 7ff1f3200640 1 --2- 172.21.15.32:0/2818252247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1f4104570 0x7ff1f4104980 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.006+0000 7ff1f3200640 1 -- 172.21.15.32:0/2818252247 >> 172.21.15.32:0/2818252247 conn(0x7ff1f40fe070 msgr2=0x7ff1f41004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.006+0000 7ff1f3200640 1 -- 172.21.15.32:0/2818252247 shutdown_connections 2024-08-31T07:23:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.006+0000 7ff1f3200640 1 -- 172.21.15.32:0/2818252247 wait complete. 2024-08-31T07:23:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.008+0000 7ff1f3200640 1 Processor -- start 2024-08-31T07:23:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.008+0000 7ff1f3200640 1 -- start start 2024-08-31T07:23:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.008+0000 7ff1f3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f4104570 0x7ff1f419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1f3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f4104ec0 0x7ff1f419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1f3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1f410d9d0 0x7ff1f41a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1f3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4113640 con 0x7ff1f4104570 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1f3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41134c0 con 0x7ff1f4104ec0 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1f3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41137c0 con 0x7ff1f410d9d0 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f4104ec0 0x7ff1f419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f4104ec0 0x7ff1f419c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53416/0 (socket says 172.21.15.32:53416) 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1ebe00640 1 -- 172.21.15.32:0/3838050491 learned_addr learned my addr 172.21.15.32:0/3838050491 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f4104570 0x7ff1f419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1f2800640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1f410d9d0 0x7ff1f41a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1ebe00640 1 -- 172.21.15.32:0/3838050491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1f410d9d0 msgr2=0x7ff1f41a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1ebe00640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1f410d9d0 0x7ff1f41a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1ebe00640 1 -- 172.21.15.32:0/3838050491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f4104570 msgr2=0x7ff1f419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1ebe00640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f4104570 0x7ff1f419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.009+0000 7ff1ebe00640 1 -- 172.21.15.32:0/3838050491 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f41a22a0 con 0x7ff1f4104ec0 2024-08-31T07:23:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.010+0000 7ff1ebe00640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f4104ec0 0x7ff1f419c8d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e9f0 tx=0x7ff1dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.010+0000 7ff1eaa00640 1 -- 172.21.15.32:0/3838050491 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00ccc0 con 0x7ff1f4104ec0 2024-08-31T07:23:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.010+0000 7ff1eaa00640 1 -- 172.21.15.32:0/3838050491 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc0108e0 con 0x7ff1f4104ec0 2024-08-31T07:23:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.010+0000 7ff1eaa00640 1 -- 172.21.15.32:0/3838050491 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc018880 con 0x7ff1f4104ec0 2024-08-31T07:23:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.010+0000 7ff1f3200640 1 -- 172.21.15.32:0/3838050491 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f41a2590 con 0x7ff1f4104ec0 2024-08-31T07:23:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.010+0000 7ff1f3200640 1 -- 172.21.15.32:0/3838050491 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f41a8cf0 con 0x7ff1f4104ec0 2024-08-31T07:23:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.014+0000 7ff1ffc6b740 1 Processor -- start 2024-08-31T07:23:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1ffc6b740 1 -- start start 2024-08-31T07:23:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1eaa00640 1 -- 172.21.15.32:0/3838050491 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff1dc010430 con 0x7ff1f4104ec0 2024-08-31T07:23:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b994a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b994fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 0x55b81b9954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b81b90b7b0 con 0x55b81b5ef070 2024-08-31T07:23:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b81b8fad40 con 0x55b81b5c61b0 2024-08-31T07:23:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b81b8fecf0 con 0x55b81b5ce390 2024-08-31T07:23:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1f1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b994fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1f2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b994a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1f1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b994fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53418/0 (socket says 172.21.15.32:53418) 2024-08-31T07:23:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.015+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2372413343 learned_addr learned my addr 172.21.15.32:0/2372413343 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.016+0000 7ff1ebe00640 1 --2- 172.21.15.32:0/2372413343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 0x55b81b9954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.016+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2372413343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 msgr2=0x55b81b9954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.016+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/2372413343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 0x55b81b9954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.016+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2372413343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 msgr2=0x55b81b994a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.016+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/2372413343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b994a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.016+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2372413343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81b65a650 con 0x55b81b5c61b0 2024-08-31T07:23:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.016+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/2372413343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b994fb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009900 tx=0x7ff1e4032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f5d5a35f626697e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.016+0000 7ff1e3400640 1 -- 172.21.15.32:0/2372413343 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4040070 con 0x55b81b5c61b0 2024-08-31T07:23:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.017+0000 7ff1e3400640 1 -- 172.21.15.32:0/2372413343 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e4004030 con 0x55b81b5c61b0 2024-08-31T07:23:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.017+0000 7ff1e3400640 1 -- 172.21.15.32:0/2372413343 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4035040 con 0x55b81b5c61b0 2024-08-31T07:23:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1eaa00640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff1d80b4d20 0x7ff1d80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1eaa00640 1 -- 172.21.15.32:0/3838050491 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7ff1dc0e5810 con 0x7ff1f4104ec0 2024-08-31T07:23:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1f2800640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff1d80b4d20 0x7ff1d80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2372413343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 msgr2=0x55b81b994fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/2372413343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b994fb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009900 tx=0x7ff1e4032b90 comp rx=0 tx=0).stop 2024-08-31T07:23:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2372413343 shutdown_connections 2024-08-31T07:23:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/2372413343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 0x55b81b9954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/2372413343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b994fb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/2372413343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b994a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2372413343 >> 172.21.15.32:0/2372413343 conn(0x55b81b5e3460 msgr2=0x55b81b992960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.018+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2372413343 shutdown_connections 2024-08-31T07:23:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.019+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2372413343 wait complete. 2024-08-31T07:23:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.019+0000 7ff1f2800640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff1d80b4d20 0x7ff1d80b7230 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005fd0 tx=0x7ff1ec009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.019+0000 7ff1ffc6b740 1 -- start start 2024-08-31T07:23:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.019+0000 7ff1ffc6b740 1 Processor -- start 2024-08-31T07:23:29.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b9a0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 0x55b81b9a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b9ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b81b75d700 con 0x55b81b5ef070 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b81b881cf0 con 0x55b81b5c61b0 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b81b9a31f0 con 0x55b81b5ce390 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1ffc6b740 10 client.0 _ll_register_callbacks cb 0x55b81b8a7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1f1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b9a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 0x55b81b9a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1f1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b9a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53430/0 (socket says 172.21.15.32:53430) 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2422519668 learned_addr learned my addr 172.21.15.32:0/2422519668 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.020+0000 7ff1f2800640 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b9ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2422519668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 msgr2=0x55b81b9a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 0x55b81b9a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2422519668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 msgr2=0x55b81b9ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b9ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2422519668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81b8c28b0 con 0x55b81b5c61b0 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1ebe00640 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 0x55b81b9a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b9a0610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4031440 tx=0x7ff1e402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b6a4607ee8f0625 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1d7e00640 1 -- 172.21.15.32:0/2422519668 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4040070 con 0x55b81b5c61b0 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2422519668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e4004030 con 0x55b81b5c61b0 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.021+0000 7ff1d7e00640 1 -- 172.21.15.32:0/2422519668 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7ff1e4005460 con 0x55b81b5c61b0 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1d7e00640 1 -- 172.21.15.32:0/2422519668 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff1e4056050 con 0x55b81b5c61b0 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1ffc6b740 10 client.27728 fetch_fsmap learned FSMap version 43 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2422519668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b81b9af1e0 con 0x55b81b5c61b0 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1d7e00640 1 -- 172.21.15.32:0/2422519668 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e4004030 con 0x55b81b5c61b0 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1d7e00640 1 -- 172.21.15.32:0/2422519668 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7ff1e4004c50 con 0x55b81b5c61b0 2024-08-31T07:23:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1ffc6b740 10 client.27728 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T07:23:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1ffc6b740 10 client.27728 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:23:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1ffc6b740 20 client.27728 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55b81b5ad9f0 0x55b81b9b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1ffc6b740 4 client.27728 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.022+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2422519668 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b81b90d700 con 0x55b81b5ad9f0 2024-08-31T07:23:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.023+0000 7ff1ebe00640 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55b81b5ad9f0 0x55b81b9b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.023+0000 7ff1ebe00640 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55b81b5ad9f0 0x55b81b9b2540 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fce0c97ba018875f server_cookie=404afe3175c1d5b5 in_seq=0 out_seq=0 2024-08-31T07:23:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.023+0000 7ff1d7e00640 10 client.27728 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:23:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.024+0000 7ff1d7e00640 1 -- 172.21.15.32:0/2422519668 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b81b90d700 con 0x55b81b5ad9f0 2024-08-31T07:23:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.024+0000 7ff1d7e00640 10 client.27728 handle_command_reply: tid=0 2024-08-31T07:23:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.024+0000 7ff1ffc6b740 1 client.27728 shutdown 2024-08-31T07:23:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.024+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2422519668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 msgr2=0x55b81b9a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.024+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b9a0610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4031440 tx=0x7ff1e402a410 comp rx=0 tx=0).stop 2024-08-31T07:23:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.024+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2422519668 shutdown_connections 2024-08-31T07:23:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.024+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55b81b5ad9f0 0x55b81b9b2540 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.025+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b9ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.025+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5ce390 0x55b81b9a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.025+0000 7ff1d7e00640 0 client.27728 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:23:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.025+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/2422519668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5c61b0 0x55b81b9a0610 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.025+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2422519668 >> 172.21.15.32:0/2422519668 conn(0x55b81b5e3460 msgr2=0x55b81b997c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.025+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2422519668 shutdown_connections 2024-08-31T07:23:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.025+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/2422519668 wait complete. 2024-08-31T07:23:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.025+0000 7ff1ffc6b740 20 client.27728 trim_cache size 0 max 16384 2024-08-31T07:23:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.041+0000 7ff1ffc6b740 1 Processor -- start 2024-08-31T07:23:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.041+0000 7ff1ffc6b740 1 -- start start 2024-08-31T07:23:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5ce390 0x55b81b9ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b999b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b1460 con 0x55b81b5ef070 2024-08-31T07:23:29.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b3000 con 0x55b81b5ce390 2024-08-31T07:23:29.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b1b30 con 0x55b81b5c61b0 2024-08-31T07:23:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1f1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5ce390 0x55b81b9ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1f1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9ac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37710/0 (socket says 172.21.15.32:37710) 2024-08-31T07:23:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1f1e00640 1 -- 172.21.15.32:0/1469889022 learned_addr learned my addr 172.21.15.32:0/1469889022 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1f2800640 1 --2- 172.21.15.32:0/1469889022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b999b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1f1e00640 1 -- 172.21.15.32:0/1469889022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5ce390 msgr2=0x55b81b9ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/1469889022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5ce390 0x55b81b9ac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1f1e00640 1 -- 172.21.15.32:0/1469889022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 msgr2=0x55b81b999b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/1469889022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b999b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1f1e00640 1 -- 172.21.15.32:0/1469889022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81b9978d0 con 0x55b81b5c61b0 2024-08-31T07:23:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.042+0000 7ff1f2800640 1 --2- 172.21.15.32:0/1469889022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b999b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.043+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/1469889022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9ac4b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff1e402a930 tx=0x7ff1e40050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d461f38383f018da server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.043+0000 7ff1e2a00640 1 -- 172.21.15.32:0/1469889022 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4040070 con 0x55b81b5c61b0 2024-08-31T07:23:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.043+0000 7ff1e2a00640 1 -- 172.21.15.32:0/1469889022 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e404d070 con 0x55b81b5c61b0 2024-08-31T07:23:29.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.043+0000 7ff1e2a00640 1 -- 172.21.15.32:0/1469889022 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4035040 con 0x55b81b5c61b0 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.044+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/1469889022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 msgr2=0x55b81b9ac4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.044+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/1469889022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9ac4b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff1e402a930 tx=0x7ff1e40050b0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.044+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/1469889022 shutdown_connections 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.044+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/1469889022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ef070 0x55b81b999b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.044+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/1469889022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5ce390 0x55b81b9ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.044+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/1469889022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9ac4b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.044+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/1469889022 >> 172.21.15.32:0/1469889022 conn(0x55b81b5e3460 msgr2=0x55b81b99ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.044+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/1469889022 shutdown_connections 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.044+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/1469889022 wait complete. 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.045+0000 7ff1ffc6b740 1 -- start start 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.045+0000 7ff1ffc6b740 1 Processor -- start 2024-08-31T07:23:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.045+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.045+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ce390 0x55b81b9b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.045+0000 7ff1ffc6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5ef070 0x55b81b9b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.045+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b2900 con 0x55b81b5ce390 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.045+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b2200 con 0x55b81b5ef070 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.045+0000 7ff1ffc6b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41af460 con 0x55b81b5c61b0 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1ffc6b740 10 client.0 _ll_register_callbacks cb 0x7ff1e4005dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1f1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1f1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37712/0 (socket says 172.21.15.32:37712) 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1f1e00640 1 -- 172.21.15.32:0/652076890 learned_addr learned my addr 172.21.15.32:0/652076890 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1ebe00640 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ce390 0x55b81b9b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1f2800640 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5ef070 0x55b81b9b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1f1e00640 1 -- 172.21.15.32:0/652076890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5ef070 msgr2=0x55b81b9b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5ef070 0x55b81b9b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1f1e00640 1 -- 172.21.15.32:0/652076890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ce390 msgr2=0x55b81b9b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ce390 0x55b81b9b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1f1e00640 1 -- 172.21.15.32:0/652076890 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81ba1bcc0 con 0x55b81b5c61b0 2024-08-31T07:23:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9b5fa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40036e0 tx=0x7ff1e4003990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79c4c7cee55d2d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.046+0000 7ff1d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:23:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.047+0000 7ff1d7e00640 1 -- 172.21.15.32:0/652076890 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4035040 con 0x55b81b5c61b0 2024-08-31T07:23:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.047+0000 7ff1d7e00640 1 -- 172.21.15.32:0/652076890 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7ff1e40053f0 con 0x55b81b5c61b0 2024-08-31T07:23:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.047+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/652076890 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e404d070 con 0x55b81b5c61b0 2024-08-31T07:23:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.047+0000 7ff1d7e00640 1 -- 172.21.15.32:0/652076890 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff1e4070050 con 0x55b81b5c61b0 2024-08-31T07:23:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.047+0000 7ff1ffc6b740 10 client.27733 fetch_fsmap learned FSMap version 43 2024-08-31T07:23:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.047+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/652076890 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b81ba1ce50 con 0x55b81b5c61b0 2024-08-31T07:23:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.047+0000 7ff1d7e00640 1 -- 172.21.15.32:0/652076890 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e404d070 con 0x55b81b5c61b0 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.048+0000 7ff1d7e00640 1 -- 172.21.15.32:0/652076890 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7ff1e4004860 con 0x55b81b5c61b0 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.048+0000 7ff1ffc6b740 10 client.27733 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.048+0000 7ff1ffc6b740 10 client.27733 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.048+0000 7ff1ffc6b740 20 client.27733 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.048+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55b81b582da0 0x55b81ba1f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.048+0000 7ff1ffc6b740 4 client.27733 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.048+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/652076890 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b81b9091a0 con 0x55b81b582da0 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.048+0000 7ff1ebe00640 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55b81b582da0 0x55b81ba1f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.048+0000 7ff1ebe00640 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55b81b582da0 0x55b81ba1f570 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18b83a216b004d31 server_cookie=c44c1399ac0890b6 in_seq=0 out_seq=0 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.048+0000 7ff1d7e00640 10 client.27733 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1d7e00640 1 -- 172.21.15.32:0/652076890 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b81b9091a0 con 0x55b81b582da0 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1d7e00640 10 client.27733 handle_command_reply: tid=0 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 client.27733 shutdown 2024-08-31T07:23:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/652076890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 msgr2=0x55b81b9b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9b5fa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40036e0 tx=0x7ff1e4003990 comp rx=0 tx=0).stop 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/652076890 shutdown_connections 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55b81b582da0 0x55b81ba1f570 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b81b5ef070 0x55b81b9b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b81b5ce390 0x55b81b9b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 --2- 172.21.15.32:0/652076890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b81b5c61b0 0x55b81b9b5fa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1d7e00640 0 client.27733 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/652076890 >> 172.21.15.32:0/652076890 conn(0x55b81b5e3460 msgr2=0x55b81b999b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/652076890 shutdown_connections 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 1 -- 172.21.15.32:0/652076890 wait complete. 2024-08-31T07:23:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.049+0000 7ff1ffc6b740 20 client.27733 trim_cache size 0 max 16384 2024-08-31T07:23:29.054 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:23:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 -- 172.21.15.32:0/3838050491 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff1d80b4d20 msgr2=0x7ff1d80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff1d80b4d20 0x7ff1d80b7230 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005fd0 tx=0x7ff1ec009500 comp rx=0 tx=0).stop 2024-08-31T07:23:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 -- 172.21.15.32:0/3838050491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f4104ec0 msgr2=0x7ff1f419c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f4104ec0 0x7ff1f419c8d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e9f0 tx=0x7ff1dc00edf0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 -- 172.21.15.32:0/3838050491 shutdown_connections 2024-08-31T07:23:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff1d80b4d20 0x7ff1d80b7230 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1f410d9d0 0x7ff1f41a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1f4104ec0 0x7ff1f419c8d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 --2- 172.21.15.32:0/3838050491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f4104570 0x7ff1f419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 -- 172.21.15.32:0/3838050491 >> 172.21.15.32:0/3838050491 conn(0x7ff1f40fe070 msgr2=0x7ff1f4102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 -- 172.21.15.32:0/3838050491 shutdown_connections 2024-08-31T07:23:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:29.050+0000 7ff1f3200640 1 -- 172.21.15.32:0/3838050491 wait complete. 2024-08-31T07:23:29.721 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-08-31T07:23:29.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-08-31T07:23:29.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345456 2.98 MB/sec execute 438 sec latency 17.751 ms 2024-08-31T07:23:30.509 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:23:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:30.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345759 2.97 MB/sec execute 439 sec latency 569.926 ms 2024-08-31T07:23:30.805 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:30 smithi032 ceph-mon[17252]: pgmap v700: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 614 KiB/s wr, 73 op/s 2024-08-31T07:23:30.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:30 smithi073 ceph-mon[21995]: pgmap v700: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 614 KiB/s wr, 73 op/s 2024-08-31T07:23:30.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:30 smithi084 ceph-mon[22387]: pgmap v700: 97 pgs: 97 active+clean; 263 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 614 KiB/s wr, 73 op/s 2024-08-31T07:23:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.435+0000 7f463a600640 1 -- 172.21.15.32:0/3548676649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f463410bdb0 msgr2=0x7f46341125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.435+0000 7f463a600640 1 --2- 172.21.15.32:0/3548676649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f463410bdb0 0x7f46341125e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f462c009be0 tx=0x7f462c02a6c0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.435+0000 7f463a600640 1 -- 172.21.15.32:0/3548676649 shutdown_connections 2024-08-31T07:23:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.435+0000 7f463a600640 1 --2- 172.21.15.32:0/3548676649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f463410bdb0 0x7f46341125e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.435+0000 7f463a600640 1 --2- 172.21.15.32:0/3548676649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f463410b460 0x7f463410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.435+0000 7f463a600640 1 --2- 172.21.15.32:0/3548676649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4634105b20 0x7f4634105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.435+0000 7f463a600640 1 -- 172.21.15.32:0/3548676649 >> 172.21.15.32:0/3548676649 conn(0x7f4634101570 msgr2=0x7f46341039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.435+0000 7f463a600640 1 -- 172.21.15.32:0/3548676649 shutdown_connections 2024-08-31T07:23:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.435+0000 7f463a600640 1 -- 172.21.15.32:0/3548676649 wait complete. 2024-08-31T07:23:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.437+0000 7f463a600640 1 Processor -- start 2024-08-31T07:23:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.437+0000 7f463a600640 1 -- start start 2024-08-31T07:23:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.437+0000 7f463a600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634105b20 0x7f463419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.437+0000 7f463a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f463410b460 0x7f463419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.437+0000 7f463a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f463410bdb0 0x7f46341a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.437+0000 7f463a600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4634115a70 con 0x7f463410bdb0 2024-08-31T07:23:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.437+0000 7f463a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46341158f0 con 0x7f463410b460 2024-08-31T07:23:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.437+0000 7f463a600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4634115bf0 con 0x7f4634105b20 2024-08-31T07:23:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.438+0000 7f4633400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634105b20 0x7f463419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.438+0000 7f4633e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f463410bdb0 0x7f46341a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.438+0000 7f4633400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634105b20 0x7f463419c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37730/0 (socket says 172.21.15.32:37730) 2024-08-31T07:23:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.438+0000 7f4633400640 1 -- 172.21.15.32:0/58912876 learned_addr learned my addr 172.21.15.32:0/58912876 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.438+0000 7f462be00640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f463410b460 0x7f463419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.438+0000 7f4633400640 1 -- 172.21.15.32:0/58912876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f463410b460 msgr2=0x7f463419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.438+0000 7f4633400640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f463410b460 0x7f463419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.438+0000 7f4633400640 1 -- 172.21.15.32:0/58912876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f463410bdb0 msgr2=0x7f46341a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.438+0000 7f4633400640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f463410bdb0 0x7f46341a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.438+0000 7f4633400640 1 -- 172.21.15.32:0/58912876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46341a2310 con 0x7f4634105b20 2024-08-31T07:23:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.439+0000 7f4633400640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634105b20 0x7f463419c3a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f462400e9f0 tx=0x7f462400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.439+0000 7f4630c00640 1 -- 172.21.15.32:0/58912876 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462400ccc0 con 0x7f4634105b20 2024-08-31T07:23:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.439+0000 7f463a600640 1 -- 172.21.15.32:0/58912876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46341a2600 con 0x7f4634105b20 2024-08-31T07:23:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.439+0000 7f4630c00640 1 -- 172.21.15.32:0/58912876 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46240108e0 con 0x7f4634105b20 2024-08-31T07:23:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.439+0000 7f463a600640 1 -- 172.21.15.32:0/58912876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46341a8d60 con 0x7f4634105b20 2024-08-31T07:23:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.439+0000 7f4630c00640 1 -- 172.21.15.32:0/58912876 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624018880 con 0x7f4634105b20 2024-08-31T07:23:31.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.442+0000 7f46413ca740 1 Processor -- start 2024-08-31T07:23:31.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.443+0000 7f46413ca740 1 -- start start 2024-08-31T07:23:31.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.443+0000 7f4630c00640 1 -- 172.21.15.32:0/58912876 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4624010430 con 0x7f4634105b20 2024-08-31T07:23:31.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.443+0000 7f46413ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 0x558223480c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.443+0000 7f46413ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 0x558223481150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.443+0000 7f46413ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223481690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.443+0000 7f46413ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5582233f6450 con 0x5582230da850 2024-08-31T07:23:31.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.443+0000 7f46413ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5582233e59e0 con 0x5582230b1990 2024-08-31T07:23:31.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.443+0000 7f46413ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5582233e9990 con 0x5582230b9b70 2024-08-31T07:23:31.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.443+0000 7f462be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223481690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.444+0000 7f4633e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 0x558223480c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.444+0000 7f462be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223481690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37732/0 (socket says 172.21.15.32:37732) 2024-08-31T07:23:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.444+0000 7f462be00640 1 -- 172.21.15.32:0/2576836333 learned_addr learned my addr 172.21.15.32:0/2576836333 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.444+0000 7f4633400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 0x558223481150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.444+0000 7f462be00640 1 -- 172.21.15.32:0/2576836333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 msgr2=0x558223481150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.444+0000 7f462be00640 1 --2- 172.21.15.32:0/2576836333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 0x558223481150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.444+0000 7f462be00640 1 -- 172.21.15.32:0/2576836333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 msgr2=0x558223480c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.444+0000 7f462be00640 1 --2- 172.21.15.32:0/2576836333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 0x558223480c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.444+0000 7f462be00640 1 -- 172.21.15.32:0/2576836333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558223378fa0 con 0x5582230b9b70 2024-08-31T07:23:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.445+0000 7f462be00640 1 --2- 172.21.15.32:0/2576836333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223481690 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f461c009c60 tx=0x7f461c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f286b76c0ad11420 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:31.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.445+0000 7f4628c00640 1 -- 172.21.15.32:0/2576836333 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c01e070 con 0x5582230b9b70 2024-08-31T07:23:31.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.445+0000 7f4628c00640 1 -- 172.21.15.32:0/2576836333 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c009e10 con 0x5582230b9b70 2024-08-31T07:23:31.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.445+0000 7f4628c00640 1 -- 172.21.15.32:0/2576836333 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c019690 con 0x5582230b9b70 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.445+0000 7f4630c00640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46180b4ca0 0x7f46180b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.445+0000 7f4630c00640 1 -- 172.21.15.32:0/58912876 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f46240e5850 con 0x7f4634105b20 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.445+0000 7f4633e00640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46180b4ca0 0x7f46180b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.446+0000 7f46413ca740 1 -- 172.21.15.32:0/2576836333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 msgr2=0x558223481690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.446+0000 7f46413ca740 1 --2- 172.21.15.32:0/2576836333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223481690 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f461c009c60 tx=0x7f461c011810 comp rx=0 tx=0).stop 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.446+0000 7f46413ca740 1 -- 172.21.15.32:0/2576836333 shutdown_connections 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.446+0000 7f46413ca740 1 --2- 172.21.15.32:0/2576836333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223481690 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.446+0000 7f46413ca740 1 --2- 172.21.15.32:0/2576836333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 0x558223481150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.446+0000 7f46413ca740 1 --2- 172.21.15.32:0/2576836333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 0x558223480c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.446+0000 7f46413ca740 1 -- 172.21.15.32:0/2576836333 >> 172.21.15.32:0/2576836333 conn(0x5582230cec40 msgr2=0x55822347eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.446+0000 7f4633e00640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46180b4ca0 0x7f46180b71b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f462c004af0 tx=0x7f462c005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.446+0000 7f46413ca740 1 -- 172.21.15.32:0/2576836333 shutdown_connections 2024-08-31T07:23:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.447+0000 7f46413ca740 1 -- 172.21.15.32:0/2576836333 wait complete. 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.447+0000 7f46413ca740 1 -- start start 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.447+0000 7f46413ca740 1 Processor -- start 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.448+0000 7f46413ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 0x55822348c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.448+0000 7f46413ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x55822348c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.448+0000 7f46413ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230da850 0x5582234a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.448+0000 7f46413ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5582232487b0 con 0x5582230b1990 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.448+0000 7f46413ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558223380eb0 con 0x5582230da850 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.448+0000 7f46413ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55822348f3f0 con 0x5582230b9b70 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.448+0000 7f46413ca740 10 client.0 _ll_register_callbacks cb 0x5582233e30a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.448+0000 7f4633400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 0x55822348c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.448+0000 7f462be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x55822348c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.449+0000 7f4633400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 0x55822348c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58898/0 (socket says 172.21.15.32:58898) 2024-08-31T07:23:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.449+0000 7f4633400640 1 -- 172.21.15.32:0/180989313 learned_addr learned my addr 172.21.15.32:0/180989313 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.449+0000 7f4633e00640 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230da850 0x5582234a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.449+0000 7f462be00640 1 -- 172.21.15.32:0/180989313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230da850 msgr2=0x5582234a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.449+0000 7f462be00640 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230da850 0x5582234a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.449+0000 7f462be00640 1 -- 172.21.15.32:0/180989313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 msgr2=0x55822348c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.449+0000 7f462be00640 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 0x55822348c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.449+0000 7f462be00640 1 -- 172.21.15.32:0/180989313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55822338bf40 con 0x5582230b9b70 2024-08-31T07:23:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.450+0000 7f462be00640 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x55822348c8e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f461c006010 tx=0x7f461c004280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc48fb4bc5b3564b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.450+0000 7f4621600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.450+0000 7f4621600640 1 -- 172.21.15.32:0/180989313 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c027020 con 0x5582230b9b70 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.450+0000 7f4621600640 1 -- 172.21.15.32:0/180989313 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f461c005040 con 0x5582230b9b70 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.450+0000 7f46413ca740 1 -- 172.21.15.32:0/180989313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c009e10 con 0x5582230b9b70 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.450+0000 7f4621600640 1 -- 172.21.15.32:0/180989313 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f461c032050 con 0x5582230b9b70 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.450+0000 7f46413ca740 10 client.27745 fetch_fsmap learned FSMap version 43 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.450+0000 7f46413ca740 1 -- 172.21.15.32:0/180989313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558223154ba0 con 0x5582230b9b70 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.451+0000 7f4621600640 1 -- 172.21.15.32:0/180989313 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c009e10 con 0x5582230b9b70 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.451+0000 7f4621600640 1 -- 172.21.15.32:0/180989313 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f461c004a40 con 0x5582230b9b70 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.451+0000 7f46413ca740 10 client.27745 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.451+0000 7f46413ca740 10 client.27745 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:23:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.451+0000 7f46413ca740 20 client.27745 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.451+0000 7f46413ca740 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5582230991d0 0x5582234a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.451+0000 7f46413ca740 4 client.27745 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.451+0000 7f46413ca740 1 -- 172.21.15.32:0/180989313 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582233f83a0 con 0x5582230991d0 2024-08-31T07:23:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.452+0000 7f4633400640 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5582230991d0 0x5582234a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.453+0000 7f4633400640 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5582230991d0 0x5582234a6bb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a7f6828bb2b21f server_cookie=556acd7d5807fcfe in_seq=0 out_seq=0 2024-08-31T07:23:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.453+0000 7f4621600640 10 client.27745 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.453+0000 7f4621600640 1 -- 172.21.15.32:0/180989313 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582233f83a0 con 0x5582230991d0 2024-08-31T07:23:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.453+0000 7f4621600640 10 client.27745 handle_command_reply: tid=0 2024-08-31T07:23:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f46413ca740 1 client.27745 shutdown 2024-08-31T07:23:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f46413ca740 1 -- 172.21.15.32:0/180989313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 msgr2=0x55822348c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f46413ca740 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x55822348c8e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f461c006010 tx=0x7f461c004280 comp rx=0 tx=0).stop 2024-08-31T07:23:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f46413ca740 1 -- 172.21.15.32:0/180989313 shutdown_connections 2024-08-31T07:23:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f46413ca740 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5582230991d0 0x5582234a6bb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f46413ca740 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230da850 0x5582234a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f46413ca740 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x55822348c8e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f46413ca740 1 --2- 172.21.15.32:0/180989313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 0x55822348c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f46413ca740 1 -- 172.21.15.32:0/180989313 >> 172.21.15.32:0/180989313 conn(0x5582230cec40 msgr2=0x558223483da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:31.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f4621600640 0 client.27745 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:31.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.454+0000 7f46413ca740 1 -- 172.21.15.32:0/180989313 shutdown_connections 2024-08-31T07:23:31.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.455+0000 7f46413ca740 1 -- 172.21.15.32:0/180989313 wait complete. 2024-08-31T07:23:31.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.455+0000 7f46413ca740 20 client.27745 trim_cache size 0 max 16384 2024-08-31T07:23:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.473+0000 7f46413ca740 1 Processor -- start 2024-08-31T07:23:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.473+0000 7f46413ca740 1 -- start start 2024-08-31T07:23:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.473+0000 7f46413ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 0x55822348e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.473+0000 7f46413ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x55822348c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.473+0000 7f46413ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 0x558223482350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.473+0000 7f46413ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f461000e6b0 con 0x5582230da850 2024-08-31T07:23:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.473+0000 7f46413ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4610010250 con 0x5582230b1990 2024-08-31T07:23:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.473+0000 7f46413ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f461000ed80 con 0x5582230b9b70 2024-08-31T07:23:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f4633e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 0x558223482350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f462be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x55822348c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f4633e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 0x558223482350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58900/0 (socket says 172.21.15.32:58900) 2024-08-31T07:23:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f4633e00640 1 -- 172.21.15.32:0/3403220862 learned_addr learned my addr 172.21.15.32:0/3403220862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f4633400640 1 --2- 172.21.15.32:0/3403220862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 0x55822348e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f462be00640 1 -- 172.21.15.32:0/3403220862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 msgr2=0x55822348e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f462be00640 1 --2- 172.21.15.32:0/3403220862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 0x55822348e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f462be00640 1 -- 172.21.15.32:0/3403220862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 msgr2=0x558223482350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f462be00640 1 --2- 172.21.15.32:0/3403220862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 0x558223482350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f462be00640 1 -- 172.21.15.32:0/3403220862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558223483a70 con 0x5582230b9b70 2024-08-31T07:23:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.474+0000 7f4633400640 1 --2- 172.21.15.32:0/3403220862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 0x55822348e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.475+0000 7f462be00640 1 --2- 172.21.15.32:0/3403220862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x55822348c7c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f461c004e90 tx=0x7f461c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51df4860ecbf74b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.475+0000 7f4623e00640 1 -- 172.21.15.32:0/3403220862 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c027070 con 0x5582230b9b70 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.475+0000 7f4623e00640 1 -- 172.21.15.32:0/3403220862 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c019dd0 con 0x5582230b9b70 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.475+0000 7f4623e00640 1 -- 172.21.15.32:0/3403220862 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c01e040 con 0x5582230b9b70 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.476+0000 7f46413ca740 1 -- 172.21.15.32:0/3403220862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 msgr2=0x55822348c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.476+0000 7f46413ca740 1 --2- 172.21.15.32:0/3403220862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x55822348c7c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f461c004e90 tx=0x7f461c011810 comp rx=0 tx=0).stop 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.476+0000 7f46413ca740 1 -- 172.21.15.32:0/3403220862 shutdown_connections 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.476+0000 7f46413ca740 1 --2- 172.21.15.32:0/3403220862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230da850 0x558223482350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.476+0000 7f46413ca740 1 --2- 172.21.15.32:0/3403220862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x55822348c7c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.476+0000 7f46413ca740 1 --2- 172.21.15.32:0/3403220862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230b1990 0x55822348e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.476+0000 7f46413ca740 1 -- 172.21.15.32:0/3403220862 >> 172.21.15.32:0/3403220862 conn(0x5582230cec40 msgr2=0x558223481c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.476+0000 7f46413ca740 1 -- 172.21.15.32:0/3403220862 shutdown_connections 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.476+0000 7f46413ca740 1 -- 172.21.15.32:0/3403220862 wait complete. 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.477+0000 7f46413ca740 1 -- start start 2024-08-31T07:23:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.477+0000 7f46413ca740 1 Processor -- start 2024-08-31T07:23:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f46413ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 0x558223480a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f46413ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223480fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f46413ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230da850 0x5582234814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f46413ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f461000fb50 con 0x5582230b1990 2024-08-31T07:23:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f46413ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f461000f450 con 0x5582230da850 2024-08-31T07:23:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f46413ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f461000c6b0 con 0x5582230b9b70 2024-08-31T07:23:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f46413ca740 10 client.0 _ll_register_callbacks cb 0x7f461c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f4633400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 0x558223480a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f462be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223480fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f462be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223480fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37748/0 (socket says 172.21.15.32:37748) 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f462be00640 1 -- 172.21.15.32:0/3528597946 learned_addr learned my addr 172.21.15.32:0/3528597946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f4633e00640 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230da850 0x5582234814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f462be00640 1 -- 172.21.15.32:0/3528597946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230da850 msgr2=0x5582234814e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f462be00640 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230da850 0x5582234814e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f462be00640 1 -- 172.21.15.32:0/3528597946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 msgr2=0x558223480a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f462be00640 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 0x558223480a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f462be00640 1 -- 172.21.15.32:0/3528597946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558223499450 con 0x5582230b9b70 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.478+0000 7f4633400640 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 0x558223480a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.479+0000 7f462be00640 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223480fa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f461c003c80 tx=0x7f461c009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7907f4eb8ea312f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.479+0000 7f4621600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:23:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.479+0000 7f4621600640 1 -- 172.21.15.32:0/3528597946 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c044060 con 0x5582230b9b70 2024-08-31T07:23:31.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.479+0000 7f4621600640 1 -- 172.21.15.32:0/3528597946 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f461c009260 con 0x5582230b9b70 2024-08-31T07:23:31.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.479+0000 7f46413ca740 1 -- 172.21.15.32:0/3528597946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c019dd0 con 0x5582230b9b70 2024-08-31T07:23:31.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.479+0000 7f4621600640 1 -- 172.21.15.32:0/3528597946 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f461c01e030 con 0x5582230b9b70 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.479+0000 7f46413ca740 10 client.27751 fetch_fsmap learned FSMap version 43 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.479+0000 7f46413ca740 1 -- 172.21.15.32:0/3528597946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5582235116c0 con 0x5582230b9b70 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.480+0000 7f4621600640 1 -- 172.21.15.32:0/3528597946 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c019dd0 con 0x5582230b9b70 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.480+0000 7f4621600640 1 -- 172.21.15.32:0/3528597946 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f461c005380 con 0x5582230b9b70 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.480+0000 7f46413ca740 10 client.27751 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.480+0000 7f46413ca740 10 client.27751 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.480+0000 7f46413ca740 20 client.27751 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.480+0000 7f46413ca740 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55822306e580 0x558223513e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.480+0000 7f46413ca740 4 client.27751 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.480+0000 7f46413ca740 1 -- 172.21.15.32:0/3528597946 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x5582233f3e40 con 0x55822306e580 2024-08-31T07:23:31.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.480+0000 7f4633400640 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55822306e580 0x558223513e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:31.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.481+0000 7f4633400640 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55822306e580 0x558223513e80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb361f7d4fe98a2 server_cookie=e0490736b3d0b173 in_seq=0 out_seq=0 2024-08-31T07:23:31.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.481+0000 7f4621600640 10 client.27751 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f4621600640 1 -- 172.21.15.32:0/3528597946 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5582233f3e40 con 0x55822306e580 2024-08-31T07:23:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f4621600640 10 client.27751 handle_command_reply: tid=0 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f46413ca740 1 client.27751 shutdown 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f46413ca740 1 -- 172.21.15.32:0/3528597946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 msgr2=0x558223480fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f46413ca740 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223480fa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f461c003c80 tx=0x7f461c009930 comp rx=0 tx=0).stop 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f46413ca740 1 -- 172.21.15.32:0/3528597946 shutdown_connections 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f46413ca740 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55822306e580 0x558223513e80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f46413ca740 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582230da850 0x5582234814e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f46413ca740 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5582230b9b70 0x558223480fa0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f46413ca740 1 --2- 172.21.15.32:0/3528597946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5582230b1990 0x558223480a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.556+0000 7f46413ca740 1 -- 172.21.15.32:0/3528597946 >> 172.21.15.32:0/3528597946 conn(0x5582230cec40 msgr2=0x558223481c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.557+0000 7f4621600640 0 client.27751 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.557+0000 7f46413ca740 1 -- 172.21.15.32:0/3528597946 shutdown_connections 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.557+0000 7f46413ca740 1 -- 172.21.15.32:0/3528597946 wait complete. 2024-08-31T07:23:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.557+0000 7f46413ca740 20 client.27751 trim_cache size 0 max 16384 2024-08-31T07:23:31.559 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:23:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 43, 2024-08-31T07:23:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:23:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 34, 2024-08-31T07:23:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:23:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "500e9d42": { 2024-08-31T07:23:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 34, 2024-08-31T07:23:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:23:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 145.2, 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:31.561 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:23:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 -- 172.21.15.32:0/58912876 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46180b4ca0 msgr2=0x7f46180b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46180b4ca0 0x7f46180b71b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f462c004af0 tx=0x7f462c005fd0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 -- 172.21.15.32:0/58912876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634105b20 msgr2=0x7f463419c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634105b20 0x7f463419c3a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f462400e9f0 tx=0x7f462400edf0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 -- 172.21.15.32:0/58912876 shutdown_connections 2024-08-31T07:23:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46180b4ca0 0x7f46180b71b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f463410bdb0 0x7f46341a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f463410b460 0x7f463419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 --2- 172.21.15.32:0/58912876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4634105b20 0x7f463419c3a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 -- 172.21.15.32:0/58912876 >> 172.21.15.32:0/58912876 conn(0x7f4634101570 msgr2=0x7f4634108a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 -- 172.21.15.32:0/58912876 shutdown_connections 2024-08-31T07:23:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:31.558+0000 7f463a600640 1 -- 172.21.15.32:0/58912876 wait complete. 2024-08-31T07:23:31.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.97 MB/sec execute 440 sec latency 1313.209 ms 2024-08-31T07:23:32.734 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '500e9d42', quiesce took 7.4 seconds. Will release after: 17.85 2024-08-31T07:23:32.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.96 MB/sec execute 441 sec latency 1245.959 ms 2024-08-31T07:23:32.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:32 smithi032 ceph-mon[17252]: pgmap v701: 97 pgs: 97 active+clean; 267 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1005 KiB/s wr, 118 op/s 2024-08-31T07:23:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.557+0000 7faa06600640 1 -- 172.21.15.32:0/749731340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00104ec0 msgr2=0x7faa0010d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.557+0000 7faa06600640 1 --2- 172.21.15.32:0/749731340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00104ec0 0x7faa0010d410 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009900 tx=0x7fa9e802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.557+0000 7faa06600640 1 -- 172.21.15.32:0/749731340 shutdown_connections 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.557+0000 7faa06600640 1 --2- 172.21.15.32:0/749731340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0010d9d0 0x7faa0010fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.557+0000 7faa06600640 1 --2- 172.21.15.32:0/749731340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00104ec0 0x7faa0010d410 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.557+0000 7faa06600640 1 --2- 172.21.15.32:0/749731340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa00104570 0x7faa00104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.557+0000 7faa06600640 1 -- 172.21.15.32:0/749731340 >> 172.21.15.32:0/749731340 conn(0x7faa000fe070 msgr2=0x7faa001004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.557+0000 7faa06600640 1 -- 172.21.15.32:0/749731340 shutdown_connections 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.557+0000 7faa06600640 1 -- 172.21.15.32:0/749731340 wait complete. 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.558+0000 7faa06600640 1 Processor -- start 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.558+0000 7faa06600640 1 -- start start 2024-08-31T07:23:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.558+0000 7faa06600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa00104570 0x7faa001022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.558+0000 7faa06600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00104ec0 0x7faa001027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.558+0000 7faa06600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0010d9d0 0x7faa00102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa06600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa00113660 con 0x7faa00104ec0 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa06600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faa001134e0 con 0x7faa00104570 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa06600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faa001137e0 con 0x7faa0010d9d0 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7fa9f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00104ec0 0x7faa001027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa05200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa00104570 0x7faa001022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa05200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa00104570 0x7faa001022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53518/0 (socket says 172.21.15.32:53518) 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7fa9f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00104ec0 0x7faa001027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58942/0 (socket says 172.21.15.32:58942) 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa05200640 1 -- 172.21.15.32:0/4074946464 learned_addr learned my addr 172.21.15.32:0/4074946464 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa05c00640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0010d9d0 0x7faa00102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa05200640 1 -- 172.21.15.32:0/4074946464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0010d9d0 msgr2=0x7faa00102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa05200640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0010d9d0 0x7faa00102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa05200640 1 -- 172.21.15.32:0/4074946464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00104ec0 msgr2=0x7faa001027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa05200640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00104ec0 0x7faa001027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7faa05200640 1 -- 172.21.15.32:0/4074946464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa001a8310 con 0x7faa00104570 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.559+0000 7fa9f7e00640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00104ec0 0x7faa001027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.560+0000 7faa05200640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa00104570 0x7faa001022b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000e9f0 tx=0x7fa9f000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.560+0000 7fa9fe000640 1 -- 172.21.15.32:0/4074946464 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000cd80 con 0x7faa00104570 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.560+0000 7faa06600640 1 -- 172.21.15.32:0/4074946464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa001a8600 con 0x7faa00104570 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.560+0000 7fa9fe000640 1 -- 172.21.15.32:0/4074946464 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f00054d0 con 0x7faa00104570 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.560+0000 7fa9fe000640 1 -- 172.21.15.32:0/4074946464 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0010690 con 0x7faa00104570 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.560+0000 7faa06600640 1 -- 172.21.15.32:0/4074946464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001a8b10 con 0x7faa00104570 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.563+0000 7faa0ea08740 1 Processor -- start 2024-08-31T07:23:32.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.563+0000 7faa0ea08740 1 -- start start 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.563+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f47a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.563+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f47fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.563+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.563+0000 7faa0ea08740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562059ebe7b0 con 0x562059b81390 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.563+0000 7faa0ea08740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562059eadd40 con 0x562059b791b0 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.563+0000 7faa0ea08740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562059eb1cf0 con 0x562059ba2070 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7fa9f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7faa05c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f47a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7faa05c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f47a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37776/0 (socket says 172.21.15.32:37776) 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7fa9f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58952/0 (socket says 172.21.15.32:58952) 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7faa05c00640 1 -- 172.21.15.32:0/725276665 learned_addr learned my addr 172.21.15.32:0/725276665 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7faa05c00640 1 -- 172.21.15.32:0/725276665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 msgr2=0x562059f47fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7faa05c00640 1 --2- 172.21.15.32:0/725276665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f47fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7faa05c00640 1 -- 172.21.15.32:0/725276665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 msgr2=0x562059f484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7faa05c00640 1 --2- 172.21.15.32:0/725276665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7faa05c00640 1 -- 172.21.15.32:0/725276665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562059c0d650 con 0x562059ba2070 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.564+0000 7fa9fe000640 1 -- 172.21.15.32:0/4074946464 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa9f0004e90 con 0x7faa00104570 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.565+0000 7faa05c00640 1 --2- 172.21.15.32:0/725276665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f47a70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009fd0 tx=0x7fa9f80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d190293358c4aad6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.565+0000 7fa9f6000640 1 -- 172.21.15.32:0/725276665 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8004d20 con 0x562059ba2070 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.565+0000 7fa9f6000640 1 -- 172.21.15.32:0/725276665 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f8004050 con 0x562059ba2070 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.565+0000 7fa9f6000640 1 -- 172.21.15.32:0/725276665 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8005260 con 0x562059ba2070 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7faa0ea08740 1 -- 172.21.15.32:0/725276665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 msgr2=0x562059f47a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7faa0ea08740 1 --2- 172.21.15.32:0/725276665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f47a70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009fd0 tx=0x7fa9f80118d0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7faa0ea08740 1 -- 172.21.15.32:0/725276665 shutdown_connections 2024-08-31T07:23:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7faa0ea08740 1 --2- 172.21.15.32:0/725276665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7faa0ea08740 1 --2- 172.21.15.32:0/725276665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f47fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7faa0ea08740 1 --2- 172.21.15.32:0/725276665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f47a70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7faa0ea08740 1 -- 172.21.15.32:0/725276665 >> 172.21.15.32:0/725276665 conn(0x562059b96460 msgr2=0x562059f45960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7faa0ea08740 1 -- 172.21.15.32:0/725276665 shutdown_connections 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7faa0ea08740 1 -- 172.21.15.32:0/725276665 wait complete. 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7fa9fe000640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa9e40b4bb0 0x7fa9e40b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7fa9fe000640 1 -- 172.21.15.32:0/4074946464 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fa9f00e59a0 con 0x7faa00104570 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.566+0000 7fa9f7e00640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa9e40b4bb0 0x7fa9e40b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.567+0000 7faa0ea08740 1 -- start start 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.567+0000 7faa0ea08740 1 Processor -- start 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.567+0000 7fa9f7e00640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa9e40b4bb0 0x7fa9e40b70c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa9e802a9e0 tx=0x7fa9e8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f53640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f53b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f5f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa0ea08740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562059d10700 con 0x562059b81390 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa0ea08740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562059e34cf0 con 0x562059b791b0 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa0ea08740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562059f56250 con 0x562059ba2070 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa0ea08740 10 client.0 _ll_register_callbacks cb 0x562059e5a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7fa9f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7fa9f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f53b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58966/0 (socket says 172.21.15.32:58966) 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7fa9f7e00640 1 -- 172.21.15.32:0/724595955 learned_addr learned my addr 172.21.15.32:0/724595955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa05c00640 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f5f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa05200640 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f53640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa05c00640 1 -- 172.21.15.32:0/724595955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 msgr2=0x562059f53640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa05c00640 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f53640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa05c00640 1 -- 172.21.15.32:0/724595955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 msgr2=0x562059f53b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa05c00640 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f53b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7faa05c00640 1 -- 172.21.15.32:0/724595955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562059e758b0 con 0x562059ba2070 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.568+0000 7fa9f7e00640 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f53b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.569+0000 7faa05c00640 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f5f570 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8004920 tx=0x7fa9f8003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ef013192b2ca7cb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.569+0000 7fa9eea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.569+0000 7fa9eea00640 1 -- 172.21.15.32:0/724595955 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8028020 con 0x562059ba2070 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.569+0000 7faa0ea08740 1 -- 172.21.15.32:0/724595955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f8004050 con 0x562059ba2070 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.569+0000 7fa9eea00640 1 -- 172.21.15.32:0/724595955 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fa9f8004410 con 0x562059ba2070 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.569+0000 7fa9eea00640 1 -- 172.21.15.32:0/724595955 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa9f8033050 con 0x562059ba2070 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.570+0000 7faa0ea08740 10 client.27763 fetch_fsmap learned FSMap version 43 2024-08-31T07:23:32.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.570+0000 7faa0ea08740 1 -- 172.21.15.32:0/724595955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562059f622a0 con 0x562059ba2070 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.570+0000 7fa9eea00640 1 -- 172.21.15.32:0/724595955 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f8004050 con 0x562059ba2070 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.570+0000 7fa9eea00640 1 -- 172.21.15.32:0/724595955 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7fa9f800f4e0 con 0x562059ba2070 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.570+0000 7faa0ea08740 10 client.27763 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.570+0000 7faa0ea08740 10 client.27763 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.570+0000 7faa0ea08740 20 client.27763 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.570+0000 7faa0ea08740 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x562059b609f0 0x562059f656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.570+0000 7faa0ea08740 4 client.27763 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.570+0000 7faa0ea08740 1 -- 172.21.15.32:0/724595955 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562059ec0700 con 0x562059b609f0 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.571+0000 7faa05200640 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x562059b609f0 0x562059f656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.571+0000 7faa05200640 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x562059b609f0 0x562059f656c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f103df77267c46c1 server_cookie=5b5e11cb0c4d69ff in_seq=0 out_seq=0 2024-08-31T07:23:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.571+0000 7fa9eea00640 10 client.27763 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:23:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7fa9eea00640 1 -- 172.21.15.32:0/724595955 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562059ec0700 con 0x562059b609f0 2024-08-31T07:23:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7fa9eea00640 10 client.27763 handle_command_reply: tid=0 2024-08-31T07:23:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7faa0ea08740 1 client.27763 shutdown 2024-08-31T07:23:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7faa0ea08740 1 -- 172.21.15.32:0/724595955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 msgr2=0x562059f5f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7faa0ea08740 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f5f570 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8004920 tx=0x7fa9f8003ce0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7faa0ea08740 1 -- 172.21.15.32:0/724595955 shutdown_connections 2024-08-31T07:23:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7faa0ea08740 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x562059b609f0 0x562059f656c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7faa0ea08740 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059ba2070 0x562059f5f570 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7faa0ea08740 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f53b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7faa0ea08740 1 --2- 172.21.15.32:0/724595955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f53640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7fa9eea00640 0 client.27763 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:23:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.572+0000 7faa0ea08740 1 -- 172.21.15.32:0/724595955 >> 172.21.15.32:0/724595955 conn(0x562059b96460 msgr2=0x562059f4ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.573+0000 7faa0ea08740 1 -- 172.21.15.32:0/724595955 shutdown_connections 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.573+0000 7faa0ea08740 1 -- 172.21.15.32:0/724595955 wait complete. 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.573+0000 7faa0ea08740 20 client.27763 trim_cache size 0 max 16384 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa0ea08740 1 Processor -- start 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa0ea08740 1 -- start start 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f4d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b81390 0x562059f5f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059ba2070 0x562059f44f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa0ea08740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b13d0 con 0x562059ba2070 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa0ea08740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b2f70 con 0x562059b791b0 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa0ea08740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b1aa0 con 0x562059b81390 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa05200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f4d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa05c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059ba2070 0x562059f44f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7faa05200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f4d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53544/0 (socket says 172.21.15.32:53544) 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.587+0000 7fa9f7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b81390 0x562059f5f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.588+0000 7faa05200640 1 -- 172.21.15.32:0/3707933660 learned_addr learned my addr 172.21.15.32:0/3707933660 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.588+0000 7fa9f7e00640 1 -- 172.21.15.32:0/3707933660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 msgr2=0x562059f4d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.588+0000 7fa9f7e00640 1 --2- 172.21.15.32:0/3707933660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f4d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.588+0000 7fa9f7e00640 1 -- 172.21.15.32:0/3707933660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059ba2070 msgr2=0x562059f44f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.588+0000 7fa9f7e00640 1 --2- 172.21.15.32:0/3707933660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059ba2070 0x562059f44f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.588+0000 7fa9f7e00640 1 -- 172.21.15.32:0/3707933660 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562059f54700 con 0x562059b81390 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.588+0000 7fa9f7e00640 1 --2- 172.21.15.32:0/3707933660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b81390 0x562059f5f470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e8040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2d665acc818a67 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.589+0000 7fa9f5600640 1 -- 172.21.15.32:0/3707933660 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8035040 con 0x562059b81390 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.589+0000 7fa9f5600640 1 -- 172.21.15.32:0/3707933660 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e8004c40 con 0x562059b81390 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.589+0000 7fa9f5600640 1 -- 172.21.15.32:0/3707933660 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8031700 con 0x562059b81390 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.590+0000 7faa0ea08740 1 -- 172.21.15.32:0/3707933660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b81390 msgr2=0x562059f5f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.590+0000 7faa0ea08740 1 --2- 172.21.15.32:0/3707933660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b81390 0x562059f5f470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e8040000 comp rx=0 tx=0).stop 2024-08-31T07:23:32.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.590+0000 7faa0ea08740 1 -- 172.21.15.32:0/3707933660 shutdown_connections 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.590+0000 7faa0ea08740 1 --2- 172.21.15.32:0/3707933660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059ba2070 0x562059f44f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.590+0000 7faa0ea08740 1 --2- 172.21.15.32:0/3707933660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b81390 0x562059f5f470 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.590+0000 7faa0ea08740 1 --2- 172.21.15.32:0/3707933660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059b791b0 0x562059f4d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.590+0000 7faa0ea08740 1 -- 172.21.15.32:0/3707933660 >> 172.21.15.32:0/3707933660 conn(0x562059b96460 msgr2=0x562059f4cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.590+0000 7faa0ea08740 1 -- 172.21.15.32:0/3707933660 shutdown_connections 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.590+0000 7faa0ea08740 1 -- 172.21.15.32:0/3707933660 wait complete. 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.591+0000 7faa0ea08740 1 -- start start 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.591+0000 7faa0ea08740 1 Processor -- start 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.591+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b791b0 0x562059f57490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.591+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.591+0000 7faa0ea08740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059ba2070 0x562059f57f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.591+0000 7faa0ea08740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b2870 con 0x562059b81390 2024-08-31T07:23:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.591+0000 7faa0ea08740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b2170 con 0x562059ba2070 2024-08-31T07:23:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.591+0000 7faa0ea08740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faa001af3d0 con 0x562059b791b0 2024-08-31T07:23:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.591+0000 7faa0ea08740 10 client.0 _ll_register_callbacks cb 0x7fa9f8004e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.591+0000 7fa9f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7faa05c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059ba2070 0x562059f57f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7fa9f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58988/0 (socket says 172.21.15.32:58988) 2024-08-31T07:23:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7fa9f7e00640 1 -- 172.21.15.32:0/710598361 learned_addr learned my addr 172.21.15.32:0/710598361 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7faa05200640 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b791b0 0x562059f57490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7fa9f7e00640 1 -- 172.21.15.32:0/710598361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b791b0 msgr2=0x562059f57490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7fa9f7e00640 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b791b0 0x562059f57490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7fa9f7e00640 1 -- 172.21.15.32:0/710598361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059ba2070 msgr2=0x562059f57f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7fa9f7e00640 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059ba2070 0x562059f57f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7fa9f7e00640 1 -- 172.21.15.32:0/710598361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562059fcef20 con 0x562059b81390 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7faa05c00640 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059ba2070 0x562059f57f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7fa9f7e00640 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f579d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8003d60 tx=0x7fa9e8003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781f005396229618 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.592+0000 7fa9eea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.593+0000 7fa9eea00640 1 -- 172.21.15.32:0/710598361 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8055070 con 0x562059b81390 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.593+0000 7fa9eea00640 1 -- 172.21.15.32:0/710598361 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fa9e80314a0 con 0x562059b81390 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.593+0000 7faa0ea08740 1 -- 172.21.15.32:0/710598361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e8004c40 con 0x562059b81390 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.593+0000 7fa9eea00640 1 -- 172.21.15.32:0/710598361 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa9e8061050 con 0x562059b81390 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.593+0000 7faa0ea08740 10 client.17766 fetch_fsmap learned FSMap version 43 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.593+0000 7faa0ea08740 1 -- 172.21.15.32:0/710598361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562059fd0130 con 0x562059b81390 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.593+0000 7fa9eea00640 1 -- 172.21.15.32:0/710598361 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e8004c40 con 0x562059b81390 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.594+0000 7fa9eea00640 1 -- 172.21.15.32:0/710598361 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7fa9e8044070 con 0x562059b81390 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.594+0000 7faa0ea08740 10 client.17766 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.594+0000 7faa0ea08740 10 client.17766 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:23:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.594+0000 7faa0ea08740 20 client.17766 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.594+0000 7faa0ea08740 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x562059b35da0 0x562059fd2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.594+0000 7faa0ea08740 4 client.17766 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.594+0000 7faa0ea08740 1 -- 172.21.15.32:0/710598361 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562059ebc1a0 con 0x562059b35da0 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.594+0000 7faa05200640 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x562059b35da0 0x562059fd2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.595+0000 7faa05200640 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x562059b35da0 0x562059fd2900 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1955ede64b9413c5 server_cookie=45477bfb9b1cd6dc in_seq=0 out_seq=0 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.595+0000 7fa9eea00640 10 client.17766 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.595+0000 7fa9eea00640 1 -- 172.21.15.32:0/710598361 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562059ebc1a0 con 0x562059b35da0 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.595+0000 7fa9eea00640 10 client.17766 handle_command_reply: tid=0 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.595+0000 7faa0ea08740 1 client.17766 shutdown 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.595+0000 7faa0ea08740 1 -- 172.21.15.32:0/710598361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 msgr2=0x562059f579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.595+0000 7faa0ea08740 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f579d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8003d60 tx=0x7fa9e8003d90 comp rx=0 tx=0).stop 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.595+0000 7faa0ea08740 1 -- 172.21.15.32:0/710598361 shutdown_connections 2024-08-31T07:23:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.595+0000 7faa0ea08740 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x562059b35da0 0x562059fd2900 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.596+0000 7faa0ea08740 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562059ba2070 0x562059f57f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.596+0000 7faa0ea08740 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562059b81390 0x562059f579d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.596+0000 7fa9eea00640 0 client.17766 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:23:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.596+0000 7faa0ea08740 1 --2- 172.21.15.32:0/710598361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562059b791b0 0x562059f57490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.596+0000 7faa0ea08740 1 -- 172.21.15.32:0/710598361 >> 172.21.15.32:0/710598361 conn(0x562059b96460 msgr2=0x562059f48950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.596+0000 7faa0ea08740 1 -- 172.21.15.32:0/710598361 shutdown_connections 2024-08-31T07:23:32.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.596+0000 7faa0ea08740 1 -- 172.21.15.32:0/710598361 wait complete. 2024-08-31T07:23:32.893 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.596+0000 7faa0ea08740 20 client.17766 trim_cache size 0 max 16384 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 -- 172.21.15.32:0/4074946464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa9e40b4bb0 msgr2=0x7fa9e40b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa9e40b4bb0 0x7fa9e40b70c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa9e802a9e0 tx=0x7fa9e8005a20 comp rx=0 tx=0).stop 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 -- 172.21.15.32:0/4074946464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa00104570 msgr2=0x7faa001022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa00104570 0x7faa001022b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000e9f0 tx=0x7fa9f000edf0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 -- 172.21.15.32:0/4074946464 shutdown_connections 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa9e40b4bb0 0x7fa9e40b70c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa0010d9d0 0x7faa00102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa00104ec0 0x7faa001027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 --2- 172.21.15.32:0/4074946464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa00104570 0x7faa001022b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 -- 172.21.15.32:0/4074946464 >> 172.21.15.32:0/4074946464 conn(0x7faa000fe070 msgr2=0x7faa001074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 -- 172.21.15.32:0/4074946464 shutdown_connections 2024-08-31T07:23:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:32.597+0000 7faa06600640 1 -- 172.21.15.32:0/4074946464 wait complete. 2024-08-31T07:23:32.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:32 smithi084 ceph-mon[22387]: pgmap v701: 97 pgs: 97 active+clean; 267 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1005 KiB/s wr, 118 op/s 2024-08-31T07:23:33.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:32 smithi073 ceph-mon[21995]: pgmap v701: 97 pgs: 97 active+clean; 267 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1005 KiB/s wr, 118 op/s 2024-08-31T07:23:33.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:23:33.770 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.95 MB/sec execute 442 sec latency 2246.036 ms 2024-08-31T07:23:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:34.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.95 MB/sec execute 443 sec latency 3246.102 ms 2024-08-31T07:23:34.815 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:34 smithi032 ceph-mon[17252]: pgmap v702: 97 pgs: 97 active+clean; 269 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-08-31T07:23:34.815 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:34 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:23:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.888+0000 7f710ee00640 1 -- 172.21.15.32:0/794295586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7108105950 msgr2=0x7f7108105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.888+0000 7f710ee00640 1 --2- 172.21.15.32:0/794295586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7108105950 0x7f7108105d60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009900 tx=0x7f70fc02a410 comp rx=0 tx=0).stop 2024-08-31T07:23:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.888+0000 7f710ee00640 1 -- 172.21.15.32:0/794295586 shutdown_connections 2024-08-31T07:23:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.888+0000 7f710ee00640 1 --2- 172.21.15.32:0/794295586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710810bbe0 0x7f7108112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.888+0000 7f710ee00640 1 --2- 172.21.15.32:0/794295586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f710810b290 0x7f710810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.888+0000 7f710ee00640 1 --2- 172.21.15.32:0/794295586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7108105950 0x7f7108105d60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.888+0000 7f710ee00640 1 -- 172.21.15.32:0/794295586 >> 172.21.15.32:0/794295586 conn(0x7f7108101420 msgr2=0x7f7108103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.888+0000 7f710ee00640 1 -- 172.21.15.32:0/794295586 shutdown_connections 2024-08-31T07:23:34.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.888+0000 7f710ee00640 1 -- 172.21.15.32:0/794295586 wait complete. 2024-08-31T07:23:34.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.889+0000 7f710ee00640 1 Processor -- start 2024-08-31T07:23:34.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.889+0000 7f710ee00640 1 -- start start 2024-08-31T07:23:34.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7108105950 0x7f710819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710ee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710810b290 0x7f710819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710ee00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f710810bbe0 0x7f71081a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710ee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71081158a0 con 0x7f710810b290 2024-08-31T07:23:34.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7108115720 con 0x7f7108105950 2024-08-31T07:23:34.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710ee00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7108115a20 con 0x7f710810bbe0 2024-08-31T07:23:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710810b290 0x7f710819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710810b290 0x7f710819c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59010/0 (socket says 172.21.15.32:59010) 2024-08-31T07:23:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710d000640 1 -- 172.21.15.32:0/1334810347 learned_addr learned my addr 172.21.15.32:0/1334810347 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710da00640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7108105950 0x7f710819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.890+0000 7f710e400640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f710810bbe0 0x7f71081a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f710d000640 1 -- 172.21.15.32:0/1334810347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f710810bbe0 msgr2=0x7f71081a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f710d000640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f710810bbe0 0x7f71081a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f710d000640 1 -- 172.21.15.32:0/1334810347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7108105950 msgr2=0x7f710819c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f710d000640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7108105950 0x7f710819c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f710d000640 1 -- 172.21.15.32:0/1334810347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71081a2240 con 0x7f710810b290 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f710da00640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7108105950 0x7f710819c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f710d000640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710810b290 0x7f710819c870 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f70f400d950 tx=0x7f70f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f7106000640 1 -- 172.21.15.32:0/1334810347 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4017070 con 0x7f710810b290 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f710ee00640 1 -- 172.21.15.32:0/1334810347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71081a2530 con 0x7f710810b290 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f7106000640 1 -- 172.21.15.32:0/1334810347 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70f4009c30 con 0x7f710810b290 2024-08-31T07:23:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f7106000640 1 -- 172.21.15.32:0/1334810347 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f40047a0 con 0x7f710810b290 2024-08-31T07:23:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.891+0000 7f710ee00640 1 -- 172.21.15.32:0/1334810347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71081a8c90 con 0x7f710810b290 2024-08-31T07:23:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.894+0000 7f711700e740 1 Processor -- start 2024-08-31T07:23:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f711700e740 1 -- start start 2024-08-31T07:23:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f711700e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 0x55d85d0eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f711700e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd1f1b0 0x55d85d0edfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f711700e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 0x55d85d0ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f711700e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d85d0647b0 con 0x55d85cd1f1b0 2024-08-31T07:23:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f711700e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d85d053d40 con 0x55d85cd27390 2024-08-31T07:23:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f711700e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d85d057cf0 con 0x55d85cd48070 2024-08-31T07:23:34.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f710da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd1f1b0 0x55d85d0edfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f710da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd1f1b0 0x55d85d0edfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59026/0 (socket says 172.21.15.32:59026) 2024-08-31T07:23:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f710e400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 0x55d85d0eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.895+0000 7f710da00640 1 -- 172.21.15.32:0/385331915 learned_addr learned my addr 172.21.15.32:0/385331915 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.896+0000 7f710e400640 1 -- 172.21.15.32:0/385331915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 msgr2=0x55d85d0ee4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:23:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.896+0000 7f710e400640 1 --2- 172.21.15.32:0/385331915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 0x55d85d0ee4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.896+0000 7f710e400640 1 -- 172.21.15.32:0/385331915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd1f1b0 msgr2=0x55d85d0edfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.896+0000 7f710e400640 1 --2- 172.21.15.32:0/385331915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd1f1b0 0x55d85d0edfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.896+0000 7f710e400640 1 -- 172.21.15.32:0/385331915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d85cdb3650 con 0x55d85cd48070 2024-08-31T07:23:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.896+0000 7f7106000640 1 -- 172.21.15.32:0/1334810347 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f70f4004020 con 0x7f710810b290 2024-08-31T07:23:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.896+0000 7f710e400640 1 --2- 172.21.15.32:0/385331915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 0x55d85d0eda70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7100009f00 tx=0x7f7100011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c173c2962ac038 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.897+0000 7f70faa00640 1 -- 172.21.15.32:0/385331915 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100004d20 con 0x55d85cd48070 2024-08-31T07:23:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.897+0000 7f70faa00640 1 -- 172.21.15.32:0/385331915 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7100004050 con 0x55d85cd48070 2024-08-31T07:23:34.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.897+0000 7f70faa00640 1 -- 172.21.15.32:0/385331915 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100005280 con 0x55d85cd48070 2024-08-31T07:23:34.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.897+0000 7f711700e740 1 -- 172.21.15.32:0/385331915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 msgr2=0x55d85d0eda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:34.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.897+0000 7f711700e740 1 --2- 172.21.15.32:0/385331915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 0x55d85d0eda70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7100009f00 tx=0x7f7100011810 comp rx=0 tx=0).stop 2024-08-31T07:23:34.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.898+0000 7f711700e740 1 -- 172.21.15.32:0/385331915 shutdown_connections 2024-08-31T07:23:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.898+0000 7f711700e740 1 --2- 172.21.15.32:0/385331915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 0x55d85d0ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.898+0000 7f711700e740 1 --2- 172.21.15.32:0/385331915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd1f1b0 0x55d85d0edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.898+0000 7f711700e740 1 --2- 172.21.15.32:0/385331915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 0x55d85d0eda70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.898+0000 7f711700e740 1 -- 172.21.15.32:0/385331915 >> 172.21.15.32:0/385331915 conn(0x55d85cd3c460 msgr2=0x55d85d0eb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.898+0000 7f711700e740 1 -- 172.21.15.32:0/385331915 shutdown_connections 2024-08-31T07:23:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.898+0000 7f7106000640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70f00b4c50 0x7f70f00b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.898+0000 7f711700e740 1 -- 172.21.15.32:0/385331915 wait complete. 2024-08-31T07:23:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.898+0000 7f7106000640 1 -- 172.21.15.32:0/1334810347 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f70f40e5080 con 0x7f710810b290 2024-08-31T07:23:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.898+0000 7f710da00640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70f00b4c50 0x7f70f00b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.899+0000 7f711700e740 1 -- start start 2024-08-31T07:23:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.899+0000 7f711700e740 1 Processor -- start 2024-08-31T07:23:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.899+0000 7f710da00640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70f00b4c50 0x7f70f00b7160 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0096c0 tx=0x7f70fc032ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f711700e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd1f1b0 0x55d85d102510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f711700e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d102af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f711700e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 0x55d85d107b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f711700e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d85ceb6700 con 0x55d85cd27390 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f711700e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d85cfdacf0 con 0x55d85cd1f1b0 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f711700e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d85d0fc250 con 0x55d85cd48070 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f711700e740 10 client.0 _ll_register_callbacks cb 0x55d85d000470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd1f1b0 0x55d85d102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d102af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59042/0 (socket says 172.21.15.32:59042) 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd1f1b0 0x55d85d102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53582/0 (socket says 172.21.15.32:53582) 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710d000640 1 -- 172.21.15.32:0/64422017 learned_addr learned my addr 172.21.15.32:0/64422017 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710e400640 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 0x55d85d107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710d000640 1 -- 172.21.15.32:0/64422017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 msgr2=0x55d85d107b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710d000640 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 0x55d85d107b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710d000640 1 -- 172.21.15.32:0/64422017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd1f1b0 msgr2=0x55d85d102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:34.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710d000640 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd1f1b0 0x55d85d102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.900+0000 7f710d000640 1 -- 172.21.15.32:0/64422017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d85d01b8b0 con 0x55d85cd27390 2024-08-31T07:23:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.901+0000 7f710d000640 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d102af0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f70f40ab7f0 tx=0x7f70f40e2dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23dca03043ea17a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.901+0000 7f70ef400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:23:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.901+0000 7f70ef400640 1 -- 172.21.15.32:0/64422017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f40ab370 con 0x55d85cd27390 2024-08-31T07:23:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.901+0000 7f711700e740 1 -- 172.21.15.32:0/64422017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7100004050 con 0x55d85cd27390 2024-08-31T07:23:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.901+0000 7f70ef400640 1 -- 172.21.15.32:0/64422017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f70f40acb90 con 0x55d85cd27390 2024-08-31T07:23:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.901+0000 7f70ef400640 1 -- 172.21.15.32:0/64422017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f70f40abf60 con 0x55d85cd27390 2024-08-31T07:23:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.901+0000 7f711700e740 10 client.17778 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.901+0000 7f711700e740 1 -- 172.21.15.32:0/64422017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d85d110ab0 con 0x55d85cd27390 2024-08-31T07:23:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.902+0000 7f70ef400640 1 -- 172.21.15.32:0/64422017 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7100004050 con 0x55d85cd27390 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.902+0000 7f70ef400640 1 -- 172.21.15.32:0/64422017 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f70f40a71d0 con 0x55d85cd27390 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.902+0000 7f711700e740 10 client.17778 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.902+0000 7f711700e740 10 client.17778 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.902+0000 7f711700e740 20 client.17778 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.902+0000 7f711700e740 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d85cd069f0 0x55d85d113f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.902+0000 7f711700e740 4 client.17778 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.902+0000 7f711700e740 1 -- 172.21.15.32:0/64422017 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d85d066700 con 0x55d85cd069f0 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.902+0000 7f710da00640 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d85cd069f0 0x55d85d113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.903+0000 7f710da00640 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d85cd069f0 0x55d85d113f20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ed4ad1a7e106380 server_cookie=59b3b8506545db61 in_seq=0 out_seq=0 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.903+0000 7f70ef400640 10 client.17778 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:23:34.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f70ef400640 1 -- 172.21.15.32:0/64422017 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d85d066700 con 0x55d85cd069f0 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f70ef400640 10 client.17778 handle_command_reply: tid=0 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 client.17778 shutdown 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 -- 172.21.15.32:0/64422017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 msgr2=0x55d85d102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d102af0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f70f40ab7f0 tx=0x7f70f40e2dc0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 -- 172.21.15.32:0/64422017 shutdown_connections 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd48070 0x55d85d107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d102af0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d85cd069f0 0x55d85d113f20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 --2- 172.21.15.32:0/64422017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd1f1b0 0x55d85d102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 -- 172.21.15.32:0/64422017 >> 172.21.15.32:0/64422017 conn(0x55d85cd3c460 msgr2=0x55d85d0f0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f70ef400640 0 client.17778 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 -- 172.21.15.32:0/64422017 shutdown_connections 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.904+0000 7f711700e740 1 -- 172.21.15.32:0/64422017 wait complete. 2024-08-31T07:23:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.905+0000 7f711700e740 20 client.17778 trim_cache size 0 max 16384 2024-08-31T07:23:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.919+0000 7f711700e740 1 Processor -- start 2024-08-31T07:23:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.919+0000 7f711700e740 1 -- start start 2024-08-31T07:23:34.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.919+0000 7f711700e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 0x55d85d0eb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.919+0000 7f711700e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d1048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.919+0000 7f711700e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd48070 0x55d85d104e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.919+0000 7f711700e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71081ad070 con 0x55d85cd27390 2024-08-31T07:23:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.919+0000 7f711700e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71081aebe0 con 0x55d85cd48070 2024-08-31T07:23:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.919+0000 7f711700e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f71081ad740 con 0x55d85cd1f1b0 2024-08-31T07:23:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.919+0000 7f710d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d1048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.920+0000 7f710e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd48070 0x55d85d104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.920+0000 7f710d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d1048d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59058/0 (socket says 172.21.15.32:59058) 2024-08-31T07:23:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.920+0000 7f710d000640 1 -- 172.21.15.32:0/1498809221 learned_addr learned my addr 172.21.15.32:0/1498809221 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.920+0000 7f710da00640 1 --2- 172.21.15.32:0/1498809221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 0x55d85d0eb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.920+0000 7f710e400640 1 -- 172.21.15.32:0/1498809221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 msgr2=0x55d85d0eb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.920+0000 7f710e400640 1 --2- 172.21.15.32:0/1498809221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 0x55d85d0eb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.920+0000 7f710e400640 1 -- 172.21.15.32:0/1498809221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 msgr2=0x55d85d1048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.920+0000 7f710e400640 1 --2- 172.21.15.32:0/1498809221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d1048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.920+0000 7f710e400640 1 -- 172.21.15.32:0/1498809221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d85d0f0300 con 0x55d85cd48070 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.920+0000 7f710e400640 1 --2- 172.21.15.32:0/1498809221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd48070 0x55d85d104e10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f710001b040 tx=0x7f7100009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936008be3e3e6548 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f70fa000640 1 -- 172.21.15.32:0/1498809221 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710002a070 con 0x55d85cd48070 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f70fa000640 1 -- 172.21.15.32:0/1498809221 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7100003d20 con 0x55d85cd48070 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f70fa000640 1 -- 172.21.15.32:0/1498809221 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000248a0 con 0x55d85cd48070 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f711700e740 1 -- 172.21.15.32:0/1498809221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd48070 msgr2=0x55d85d104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f711700e740 1 --2- 172.21.15.32:0/1498809221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd48070 0x55d85d104e10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f710001b040 tx=0x7f7100009d00 comp rx=0 tx=0).stop 2024-08-31T07:23:34.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f711700e740 1 -- 172.21.15.32:0/1498809221 shutdown_connections 2024-08-31T07:23:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f711700e740 1 --2- 172.21.15.32:0/1498809221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd48070 0x55d85d104e10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f711700e740 1 --2- 172.21.15.32:0/1498809221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd27390 0x55d85d1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f711700e740 1 --2- 172.21.15.32:0/1498809221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 0x55d85d0eb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f711700e740 1 -- 172.21.15.32:0/1498809221 >> 172.21.15.32:0/1498809221 conn(0x55d85cd3c460 msgr2=0x55d85d0eb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.921+0000 7f711700e740 1 -- 172.21.15.32:0/1498809221 shutdown_connections 2024-08-31T07:23:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.922+0000 7f711700e740 1 -- 172.21.15.32:0/1498809221 wait complete. 2024-08-31T07:23:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.922+0000 7f711700e740 1 -- start start 2024-08-31T07:23:34.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.922+0000 7f711700e740 1 Processor -- start 2024-08-31T07:23:34.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f711700e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 0x55d85d1073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f711700e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 0x55d85d105900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f711700e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd48070 0x55d85d105e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f711700e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71081ae4e0 con 0x55d85cd48070 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f711700e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71081ade10 con 0x55d85cd27390 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f711700e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f71081ab070 con 0x55d85cd1f1b0 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f711700e740 10 client.0 _ll_register_callbacks cb 0x7f70f40056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 0x55d85d1073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 0x55d85d105900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 0x55d85d1073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37856/0 (socket says 172.21.15.32:37856) 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 0x55d85d105900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53606/0 (socket says 172.21.15.32:53606) 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710da00640 1 -- 172.21.15.32:0/1164245092 learned_addr learned my addr 172.21.15.32:0/1164245092 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710e400640 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd48070 0x55d85d105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710d000640 1 -- 172.21.15.32:0/1164245092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 msgr2=0x55d85d1073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:34.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710d000640 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 0x55d85d1073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710d000640 1 -- 172.21.15.32:0/1164245092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd48070 msgr2=0x55d85d105e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710d000640 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd48070 0x55d85d105e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710d000640 1 -- 172.21.15.32:0/1164245092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d85d10ed80 con 0x55d85cd27390 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710da00640 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 0x55d85d1073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.923+0000 7f710e400640 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd48070 0x55d85d105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.924+0000 7f710d000640 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 0x55d85d105900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f70f40e5410 tx=0x7f70f40acda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1de9c2a546c606 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.924+0000 7f70ef400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.924+0000 7f70ef400640 1 -- 172.21.15.32:0/1164245092 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4009de0 con 0x55d85cd27390 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.924+0000 7f70ef400640 1 -- 172.21.15.32:0/1164245092 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f70f40aacd0 con 0x55d85cd27390 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.924+0000 7f711700e740 1 -- 172.21.15.32:0/1164245092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7100003d20 con 0x55d85cd27390 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.924+0000 7f70ef400640 1 -- 172.21.15.32:0/1164245092 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f70f40c2050 con 0x55d85cd27390 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.924+0000 7f711700e740 10 client.27782 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.924+0000 7f711700e740 1 -- 172.21.15.32:0/1164245092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d85d17f380 con 0x55d85cd27390 2024-08-31T07:23:34.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.924+0000 7f70ef400640 1 -- 172.21.15.32:0/1164245092 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7100003d20 con 0x55d85cd27390 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.925+0000 7f70ef400640 1 -- 172.21.15.32:0/1164245092 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f70f40b1260 con 0x55d85cd27390 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.925+0000 7f711700e740 10 client.27782 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.925+0000 7f711700e740 10 client.27782 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.925+0000 7f711700e740 20 client.27782 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.925+0000 7f711700e740 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d85ccdbda0 0x55d85d181b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.925+0000 7f711700e740 4 client.27782 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.925+0000 7f711700e740 1 -- 172.21.15.32:0/1164245092 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d85d0621a0 con 0x55d85ccdbda0 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.925+0000 7f710da00640 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d85ccdbda0 0x55d85d181b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f710da00640 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d85ccdbda0 0x55d85d181b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24207c7b111794e9 server_cookie=ffefae3821731e7f in_seq=0 out_seq=0 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f70ef400640 10 client.27782 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f70ef400640 1 -- 172.21.15.32:0/1164245092 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d85d0621a0 con 0x55d85ccdbda0 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f70ef400640 10 client.27782 handle_command_reply: tid=0 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f711700e740 1 client.27782 shutdown 2024-08-31T07:23:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f711700e740 1 -- 172.21.15.32:0/1164245092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 msgr2=0x55d85d105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f711700e740 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 0x55d85d105900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f70f40e5410 tx=0x7f70f40acda0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f711700e740 1 -- 172.21.15.32:0/1164245092 shutdown_connections 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f711700e740 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d85cd48070 0x55d85d105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f711700e740 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d85ccdbda0 0x55d85d181b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f711700e740 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d85cd27390 0x55d85d105900 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f711700e740 1 --2- 172.21.15.32:0/1164245092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d85cd1f1b0 0x55d85d1073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f70ef400640 0 client.27782 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.926+0000 7f711700e740 1 -- 172.21.15.32:0/1164245092 >> 172.21.15.32:0/1164245092 conn(0x55d85cd3c460 msgr2=0x55d85d1037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.927+0000 7f711700e740 1 -- 172.21.15.32:0/1164245092 shutdown_connections 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.927+0000 7f711700e740 1 -- 172.21.15.32:0/1164245092 wait complete. 2024-08-31T07:23:34.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.927+0000 7f711700e740 20 client.27782 trim_cache size 0 max 16384 2024-08-31T07:23:34.933 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:23:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.927+0000 7f710ee00640 1 -- 172.21.15.32:0/1334810347 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70f00b4c50 msgr2=0x7f70f00b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.927+0000 7f710ee00640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70f00b4c50 0x7f70f00b7160 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0096c0 tx=0x7f70fc032ed0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.927+0000 7f710ee00640 1 -- 172.21.15.32:0/1334810347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710810b290 msgr2=0x7f710819c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.927+0000 7f710ee00640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710810b290 0x7f710819c870 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f70f400d950 tx=0x7f70f400de10 comp rx=0 tx=0).stop 2024-08-31T07:23:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.927+0000 7f710ee00640 1 -- 172.21.15.32:0/1334810347 shutdown_connections 2024-08-31T07:23:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.927+0000 7f710ee00640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70f00b4c50 0x7f70f00b7160 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.928+0000 7f710ee00640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f710810bbe0 0x7f71081a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.928+0000 7f710ee00640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710810b290 0x7f710819c870 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.928+0000 7f710ee00640 1 --2- 172.21.15.32:0/1334810347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7108105950 0x7f710819c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.928+0000 7f710ee00640 1 -- 172.21.15.32:0/1334810347 >> 172.21.15.32:0/1334810347 conn(0x7f7108101420 msgr2=0x7f7108103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.928+0000 7f710ee00640 1 -- 172.21.15.32:0/1334810347 shutdown_connections 2024-08-31T07:23:34.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:34.928+0000 7f710ee00640 1 -- 172.21.15.32:0/1334810347 wait complete. 2024-08-31T07:23:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:34 smithi084 ceph-mon[22387]: pgmap v702: 97 pgs: 97 active+clean; 269 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-08-31T07:23:34.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:34 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:23:35.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:34 smithi073 ceph-mon[21995]: pgmap v702: 97 pgs: 97 active+clean; 269 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-08-31T07:23:35.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:34 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:23:35.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:23:35.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.94 MB/sec execute 444 sec latency 4246.142 ms 2024-08-31T07:23:36.165 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:36.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.93 MB/sec execute 445 sec latency 5246.179 ms 2024-08-31T07:23:36.841 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:36 smithi032 ceph-mon[17252]: pgmap v703: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-08-31T07:23:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.922+0000 7fddd3c00640 1 -- 172.21.15.32:0/4029062308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddcc10b460 msgr2=0x7fddcc10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.922+0000 7fddd3c00640 1 --2- 172.21.15.32:0/4029062308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddcc10b460 0x7fddcc10b870 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009900 tx=0x7fddb802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.922+0000 7fddd3c00640 1 -- 172.21.15.32:0/4029062308 shutdown_connections 2024-08-31T07:23:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.922+0000 7fddd3c00640 1 --2- 172.21.15.32:0/4029062308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddcc10bdb0 0x7fddcc1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.922+0000 7fddd3c00640 1 --2- 172.21.15.32:0/4029062308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddcc10b460 0x7fddcc10b870 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.922+0000 7fddd3c00640 1 --2- 172.21.15.32:0/4029062308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddcc105b20 0x7fddcc105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.922+0000 7fddd3c00640 1 -- 172.21.15.32:0/4029062308 >> 172.21.15.32:0/4029062308 conn(0x7fddcc1015b0 msgr2=0x7fddcc103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.922+0000 7fddd3c00640 1 -- 172.21.15.32:0/4029062308 shutdown_connections 2024-08-31T07:23:36.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.922+0000 7fddd3c00640 1 -- 172.21.15.32:0/4029062308 wait complete. 2024-08-31T07:23:36.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.923+0000 7fddd3c00640 1 Processor -- start 2024-08-31T07:23:36.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd3c00640 1 -- start start 2024-08-31T07:23:36.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd3c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddcc105b20 0x7fddcc19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd3c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddcc10b460 0x7fddcc19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd3c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddcc10bdb0 0x7fddcc1a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd3c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc115a70 con 0x7fddcc10bdb0 2024-08-31T07:23:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd3c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc1158f0 con 0x7fddcc10b460 2024-08-31T07:23:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd3c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc115bf0 con 0x7fddcc105b20 2024-08-31T07:23:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddcc105b20 0x7fddcc19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddcc10bdb0 0x7fddcc1a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddcc105b20 0x7fddcc19c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57400/0 (socket says 172.21.15.32:57400) 2024-08-31T07:23:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddd1200640 1 -- 172.21.15.32:0/2771368561 learned_addr learned my addr 172.21.15.32:0/2771368561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.924+0000 7fddcbe00640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddcc10b460 0x7fddcc19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.925+0000 7fddd1200640 1 -- 172.21.15.32:0/2771368561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddcc10b460 msgr2=0x7fddcc19ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.925+0000 7fddd1200640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddcc10b460 0x7fddcc19ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.925+0000 7fddd1200640 1 -- 172.21.15.32:0/2771368561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddcc10bdb0 msgr2=0x7fddcc1a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.925+0000 7fddd1200640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddcc10bdb0 0x7fddcc1a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.925+0000 7fddd1200640 1 -- 172.21.15.32:0/2771368561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc1a23d0 con 0x7fddcc105b20 2024-08-31T07:23:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.925+0000 7fddd1200640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddcc105b20 0x7fddcc19c460 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fddc000e9f0 tx=0x7fddc000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.925+0000 7fddc9600640 1 -- 172.21.15.32:0/2771368561 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc000cd80 con 0x7fddcc105b20 2024-08-31T07:23:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.925+0000 7fddc9600640 1 -- 172.21.15.32:0/2771368561 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc00054d0 con 0x7fddcc105b20 2024-08-31T07:23:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.925+0000 7fddd3c00640 1 -- 172.21.15.32:0/2771368561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddcc1a26c0 con 0x7fddcc105b20 2024-08-31T07:23:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.926+0000 7fddc9600640 1 -- 172.21.15.32:0/2771368561 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0010690 con 0x7fddcc105b20 2024-08-31T07:23:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.926+0000 7fddd3c00640 1 -- 172.21.15.32:0/2771368561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddcc1a8e20 con 0x7fddcc105b20 2024-08-31T07:23:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddda87e740 1 Processor -- start 2024-08-31T07:23:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddda87e740 1 -- start start 2024-08-31T07:23:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddda87e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79cc9070 0x559f7a06e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddda87e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca01b0 0x559f7a06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddda87e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 0x559f7a06f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddda87e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559f79fe57b0 con 0x559f79cc9070 2024-08-31T07:23:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddda87e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559f79fd4d40 con 0x559f79ca01b0 2024-08-31T07:23:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddda87e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559f79fd8cf0 con 0x559f79ca8390 2024-08-31T07:23:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddd1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79cc9070 0x559f7a06e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddc9600640 1 -- 172.21.15.32:0/2771368561 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fddc0004e90 con 0x7fddcc105b20 2024-08-31T07:23:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddd1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79cc9070 0x559f7a06e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50692/0 (socket says 172.21.15.32:50692) 2024-08-31T07:23:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.929+0000 7fddd1c00640 1 -- 172.21.15.32:0/3699117697 learned_addr learned my addr 172.21.15.32:0/3699117697 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.930+0000 7fddcbe00640 1 --2- 172.21.15.32:0/3699117697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 0x559f7a06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.930+0000 7fddd1200640 1 --2- 172.21.15.32:0/3699117697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca01b0 0x559f7a06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.930+0000 7fddd1c00640 1 -- 172.21.15.32:0/3699117697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 msgr2=0x559f7a06f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.930+0000 7fddd1c00640 1 --2- 172.21.15.32:0/3699117697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 0x559f7a06f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.930+0000 7fddd1c00640 1 -- 172.21.15.32:0/3699117697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca01b0 msgr2=0x559f7a06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.930+0000 7fddd1c00640 1 --2- 172.21.15.32:0/3699117697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca01b0 0x559f7a06eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.930+0000 7fddd1c00640 1 -- 172.21.15.32:0/3699117697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f79d34650 con 0x559f79cc9070 2024-08-31T07:23:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.930+0000 7fddd1200640 1 --2- 172.21.15.32:0/3699117697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca01b0 0x559f7a06eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:23:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.930+0000 7fddd1c00640 1 --2- 172.21.15.32:0/3699117697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79cc9070 0x559f7a06e990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009fd0 tx=0x7fddc40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347fa98300a5c204 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:36.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.931+0000 7fddbe000640 1 -- 172.21.15.32:0/3699117697 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4004d20 con 0x559f79cc9070 2024-08-31T07:23:36.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.931+0000 7fddbe000640 1 -- 172.21.15.32:0/3699117697 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc4004050 con 0x559f79cc9070 2024-08-31T07:23:36.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.931+0000 7fddbe000640 1 -- 172.21.15.32:0/3699117697 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4005280 con 0x559f79cc9070 2024-08-31T07:23:36.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.931+0000 7fddc9600640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fddb40b4ca0 0x7fddb40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddc9600640 1 -- 172.21.15.32:0/2771368561 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fddc00e59a0 con 0x7fddcc105b20 2024-08-31T07:23:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddda87e740 1 -- 172.21.15.32:0/3699117697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79cc9070 msgr2=0x559f7a06e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddda87e740 1 --2- 172.21.15.32:0/3699117697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79cc9070 0x559f7a06e990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009fd0 tx=0x7fddc40118d0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddd1200640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fddb40b4ca0 0x7fddb40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddda87e740 1 -- 172.21.15.32:0/3699117697 shutdown_connections 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddda87e740 1 --2- 172.21.15.32:0/3699117697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 0x559f7a06f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddda87e740 1 --2- 172.21.15.32:0/3699117697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca01b0 0x559f7a06eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddda87e740 1 --2- 172.21.15.32:0/3699117697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79cc9070 0x559f7a06e990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddda87e740 1 -- 172.21.15.32:0/3699117697 >> 172.21.15.32:0/3699117697 conn(0x559f79cbd460 msgr2=0x559f7a06c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddda87e740 1 -- 172.21.15.32:0/3699117697 shutdown_connections 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddda87e740 1 -- 172.21.15.32:0/3699117697 wait complete. 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.932+0000 7fddd1200640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fddb40b4ca0 0x7fddb40b71b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x559f7a06d610 tx=0x7fddc0004a90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.933+0000 7fddda87e740 1 -- start start 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.933+0000 7fddda87e740 1 Processor -- start 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.933+0000 7fddda87e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a07a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.933+0000 7fddda87e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca8390 0x559f7a07aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.933+0000 7fddda87e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79cc9070 0x559f7a086460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.933+0000 7fddda87e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559f79e37700 con 0x559f79ca01b0 2024-08-31T07:23:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddda87e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559f79f5bcf0 con 0x559f79ca8390 2024-08-31T07:23:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddda87e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559f7a07d170 con 0x559f79cc9070 2024-08-31T07:23:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddda87e740 10 client.0 _ll_register_callbacks cb 0x559f79f81470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddd1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a07a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddcbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca8390 0x559f7a07aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddd1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a07a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50706/0 (socket says 172.21.15.32:50706) 2024-08-31T07:23:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddd1c00640 1 -- 172.21.15.32:0/1600739387 learned_addr learned my addr 172.21.15.32:0/1600739387 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddd1c00640 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79cc9070 0x559f7a086460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddd1c00640 1 -- 172.21.15.32:0/1600739387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79cc9070 msgr2=0x559f7a086460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddd1c00640 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79cc9070 0x559f7a086460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddd1c00640 1 -- 172.21.15.32:0/1600739387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca8390 msgr2=0x559f7a07aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddd1c00640 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca8390 0x559f7a07aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.934+0000 7fddd1c00640 1 -- 172.21.15.32:0/1600739387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f79f9c8b0 con 0x559f79ca01b0 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.935+0000 7fddd1c00640 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a07a530 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fddc4004950 tx=0x7fddc4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ead93ead0deb61c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.935+0000 7fddb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.935+0000 7fddb2a00640 1 -- 172.21.15.32:0/1600739387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4004c50 con 0x559f79ca01b0 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.935+0000 7fddb2a00640 1 -- 172.21.15.32:0/1600739387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fddc4004df0 con 0x559f79ca01b0 2024-08-31T07:23:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.935+0000 7fddda87e740 1 -- 172.21.15.32:0/1600739387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddc4004050 con 0x559f79ca01b0 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.935+0000 7fddb2a00640 1 -- 172.21.15.32:0/1600739387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fddc4005040 con 0x559f79ca01b0 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.935+0000 7fddda87e740 10 client.17793 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.935+0000 7fddda87e740 1 -- 172.21.15.32:0/1600739387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559f7a089070 con 0x559f79ca01b0 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.936+0000 7fddb2a00640 1 -- 172.21.15.32:0/1600739387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc4004050 con 0x559f79ca01b0 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.936+0000 7fddb2a00640 1 -- 172.21.15.32:0/1600739387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fddc400f740 con 0x559f79ca01b0 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.936+0000 7fddda87e740 10 client.17793 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.936+0000 7fddda87e740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.936+0000 7fddda87e740 20 client.17793 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.936+0000 7fddda87e740 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559f79c879f0 0x559f7a08c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.936+0000 7fddda87e740 4 client.17793 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.936+0000 7fddda87e740 1 -- 172.21.15.32:0/1600739387 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f79fe7700 con 0x559f79c879f0 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.936+0000 7fddcbe00640 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559f79c879f0 0x559f7a08c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.937+0000 7fddcbe00640 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559f79c879f0 0x559f7a08c480 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6966b79607d582a server_cookie=84200e8c98b86d2f in_seq=0 out_seq=0 2024-08-31T07:23:36.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.937+0000 7fddb2a00640 10 client.17793 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddb2a00640 1 -- 172.21.15.32:0/1600739387 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f79fe7700 con 0x559f79c879f0 2024-08-31T07:23:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddb2a00640 10 client.17793 handle_command_reply: tid=0 2024-08-31T07:23:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddda87e740 1 client.17793 shutdown 2024-08-31T07:23:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddda87e740 1 -- 172.21.15.32:0/1600739387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 msgr2=0x559f7a07a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddda87e740 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a07a530 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fddc4004950 tx=0x7fddc4005bd0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddda87e740 1 -- 172.21.15.32:0/1600739387 shutdown_connections 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddda87e740 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559f79c879f0 0x559f7a08c480 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddda87e740 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79cc9070 0x559f7a086460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddda87e740 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79ca8390 0x559f7a07aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddda87e740 1 --2- 172.21.15.32:0/1600739387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a07a530 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddda87e740 1 -- 172.21.15.32:0/1600739387 >> 172.21.15.32:0/1600739387 conn(0x559f79cbd460 msgr2=0x559f7a071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddb2a00640 0 client.17793 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.938+0000 7fddda87e740 1 -- 172.21.15.32:0/1600739387 shutdown_connections 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.939+0000 7fddda87e740 1 -- 172.21.15.32:0/1600739387 wait complete. 2024-08-31T07:23:36.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.939+0000 7fddda87e740 20 client.17793 trim_cache size 0 max 16384 2024-08-31T07:23:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.952+0000 7fddda87e740 1 Processor -- start 2024-08-31T07:23:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddda87e740 1 -- start start 2024-08-31T07:23:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddda87e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a0721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddda87e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 0x559f7a06be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddda87e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 0x559f7a06c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddda87e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc1b1600 con 0x559f79ca01b0 2024-08-31T07:23:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddda87e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc1b3170 con 0x559f79cc9070 2024-08-31T07:23:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddda87e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc1b1cd0 con 0x559f79ca8390 2024-08-31T07:23:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddd1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a0721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddd1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a0721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50714/0 (socket says 172.21.15.32:50714) 2024-08-31T07:23:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddd1c00640 1 -- 172.21.15.32:0/1617494006 learned_addr learned my addr 172.21.15.32:0/1617494006 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:36.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.953+0000 7fddcbe00640 1 --2- 172.21.15.32:0/1617494006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 0x559f7a06be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddd1c00640 1 --2- 172.21.15.32:0/1617494006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 0x559f7a06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddd1c00640 1 -- 172.21.15.32:0/1617494006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 msgr2=0x559f7a06be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddd1c00640 1 --2- 172.21.15.32:0/1617494006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 0x559f7a06be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddd1c00640 1 -- 172.21.15.32:0/1617494006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 msgr2=0x559f7a06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:36.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddd1c00640 1 --2- 172.21.15.32:0/1617494006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 0x559f7a06c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddd1c00640 1 -- 172.21.15.32:0/1617494006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7a07b5f0 con 0x559f79ca01b0 2024-08-31T07:23:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddcbe00640 1 --2- 172.21.15.32:0/1617494006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 0x559f7a06be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddd1c00640 1 --2- 172.21.15.32:0/1617494006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a0721a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009560 tx=0x7fddc40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995300a24b53bf4a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddbd600640 1 -- 172.21.15.32:0/1617494006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4003ea0 con 0x559f79ca01b0 2024-08-31T07:23:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddbd600640 1 -- 172.21.15.32:0/1617494006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc4004040 con 0x559f79ca01b0 2024-08-31T07:23:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.954+0000 7fddbd600640 1 -- 172.21.15.32:0/1617494006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4005440 con 0x559f79ca01b0 2024-08-31T07:23:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.955+0000 7fddda87e740 1 -- 172.21.15.32:0/1617494006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 msgr2=0x559f7a0721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.955+0000 7fddda87e740 1 --2- 172.21.15.32:0/1617494006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a0721a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009560 tx=0x7fddc40118d0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.955+0000 7fddda87e740 1 -- 172.21.15.32:0/1617494006 shutdown_connections 2024-08-31T07:23:36.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.955+0000 7fddda87e740 1 --2- 172.21.15.32:0/1617494006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 0x559f7a06c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.955+0000 7fddda87e740 1 --2- 172.21.15.32:0/1617494006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca8390 0x559f7a06be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.955+0000 7fddda87e740 1 --2- 172.21.15.32:0/1617494006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca01b0 0x559f7a0721a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.955+0000 7fddda87e740 1 -- 172.21.15.32:0/1617494006 >> 172.21.15.32:0/1617494006 conn(0x559f79cbd460 msgr2=0x559f7a071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.955+0000 7fddda87e740 1 -- 172.21.15.32:0/1617494006 shutdown_connections 2024-08-31T07:23:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.955+0000 7fddda87e740 1 -- 172.21.15.32:0/1617494006 wait complete. 2024-08-31T07:23:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.956+0000 7fddda87e740 1 -- start start 2024-08-31T07:23:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.956+0000 7fddda87e740 1 Processor -- start 2024-08-31T07:23:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddda87e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca01b0 0x559f7a07e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddda87e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca8390 0x559f7a07e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddda87e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 0x559f7a07ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddda87e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc1b2a70 con 0x559f79ca8390 2024-08-31T07:23:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddda87e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc1b23a0 con 0x559f79cc9070 2024-08-31T07:23:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddda87e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc1af600 con 0x559f79ca01b0 2024-08-31T07:23:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddda87e740 10 client.0 _ll_register_callbacks cb 0x7fddc4004200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca8390 0x559f7a07e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddd1c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca01b0 0x559f7a07e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddd1c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca01b0 0x559f7a07e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57436/0 (socket says 172.21.15.32:57436) 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddd1c00640 1 -- 172.21.15.32:0/564633058 learned_addr learned my addr 172.21.15.32:0/564633058 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca8390 0x559f7a07e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50726/0 (socket says 172.21.15.32:50726) 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddd1c00640 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 0x559f7a07ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddcbe00640 1 -- 172.21.15.32:0/564633058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca01b0 msgr2=0x559f7a07e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddcbe00640 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca01b0 0x559f7a07e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddcbe00640 1 -- 172.21.15.32:0/564633058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 msgr2=0x559f7a07ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddcbe00640 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 0x559f7a07ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddcbe00640 1 -- 172.21.15.32:0/564633058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7a0f5cb0 con 0x559f79ca8390 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.957+0000 7fddd1c00640 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 0x559f7a07ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.958+0000 7fddcbe00640 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca8390 0x559f7a07e8d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fddb802a910 tx=0x7fddb8032e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9a9f13901abc24 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.958+0000 7fddb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.958+0000 7fddb2a00640 1 -- 172.21.15.32:0/564633058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8035040 con 0x559f79ca8390 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.958+0000 7fddda87e740 1 -- 172.21.15.32:0/564633058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f79fe31a0 con 0x559f79ca8390 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.958+0000 7fddb2a00640 1 -- 172.21.15.32:0/564633058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fddb8005040 con 0x559f79ca8390 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.958+0000 7fddb2a00640 1 -- 172.21.15.32:0/564633058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fddb8055050 con 0x559f79ca8390 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.958+0000 7fddda87e740 10 client.17799 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.958+0000 7fddda87e740 1 -- 172.21.15.32:0/564633058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559f7a0f6fc0 con 0x559f79ca8390 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.958+0000 7fddb2a00640 1 -- 172.21.15.32:0/564633058 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559f79fe31a0 con 0x559f79ca8390 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.959+0000 7fddb2a00640 1 -- 172.21.15.32:0/564633058 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fddb80035f0 con 0x559f79ca8390 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.959+0000 7fddda87e740 10 client.17799 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.959+0000 7fddda87e740 10 client.17799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.959+0000 7fddda87e740 20 client.17799 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.959+0000 7fddda87e740 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559f79c5cda0 0x559f7a0f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.959+0000 7fddda87e740 4 client.17799 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:23:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.959+0000 7fddda87e740 1 -- 172.21.15.32:0/564633058 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f79fe08e0 con 0x559f79c5cda0 2024-08-31T07:23:36.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.959+0000 7fddd1c00640 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559f79c5cda0 0x559f7a0f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddd1c00640 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559f79c5cda0 0x559f7a0f97e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf499991f4f05bc server_cookie=95babb37d5ca2b51 in_seq=0 out_seq=0 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddb2a00640 10 client.17799 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddb2a00640 1 -- 172.21.15.32:0/564633058 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f79fe08e0 con 0x559f79c5cda0 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddb2a00640 10 client.17799 handle_command_reply: tid=0 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddda87e740 1 client.17799 shutdown 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddda87e740 1 -- 172.21.15.32:0/564633058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca8390 msgr2=0x559f7a07e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddda87e740 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca8390 0x559f7a07e8d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fddb802a910 tx=0x7fddb8032e00 comp rx=0 tx=0).stop 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddda87e740 1 -- 172.21.15.32:0/564633058 shutdown_connections 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddda87e740 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559f79c5cda0 0x559f7a0f97e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddda87e740 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f79cc9070 0x559f7a07ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddda87e740 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f79ca8390 0x559f7a07e8d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddda87e740 1 --2- 172.21.15.32:0/564633058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f79ca01b0 0x559f7a07e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddda87e740 1 -- 172.21.15.32:0/564633058 >> 172.21.15.32:0/564633058 conn(0x559f79cbd460 msgr2=0x559f7a078f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.960+0000 7fddb2a00640 0 client.17799 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.961+0000 7fddda87e740 1 -- 172.21.15.32:0/564633058 shutdown_connections 2024-08-31T07:23:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.961+0000 7fddda87e740 1 -- 172.21.15.32:0/564633058 wait complete. 2024-08-31T07:23:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.961+0000 7fddda87e740 20 client.17799 trim_cache size 0 max 16384 2024-08-31T07:23:36.965 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:23:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.961+0000 7fddd3c00640 1 -- 172.21.15.32:0/2771368561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fddb40b4ca0 msgr2=0x7fddb40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.961+0000 7fddd3c00640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fddb40b4ca0 0x7fddb40b71b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x559f7a06d610 tx=0x7fddc0004a90 comp rx=0 tx=0).stop 2024-08-31T07:23:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.961+0000 7fddd3c00640 1 -- 172.21.15.32:0/2771368561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddcc105b20 msgr2=0x7fddcc19c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.961+0000 7fddd3c00640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddcc105b20 0x7fddcc19c460 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fddc000e9f0 tx=0x7fddc000edf0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.961+0000 7fddd3c00640 1 -- 172.21.15.32:0/2771368561 shutdown_connections 2024-08-31T07:23:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.961+0000 7fddd3c00640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fddb40b4ca0 0x7fddb40b71b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.962+0000 7fddd3c00640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddcc10bdb0 0x7fddcc1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.962+0000 7fddd3c00640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddcc10b460 0x7fddcc19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.962+0000 7fddd3c00640 1 --2- 172.21.15.32:0/2771368561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddcc105b20 0x7fddcc19c460 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.962+0000 7fddd3c00640 1 -- 172.21.15.32:0/2771368561 >> 172.21.15.32:0/2771368561 conn(0x7fddcc1015b0 msgr2=0x7fddcc1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.962+0000 7fddd3c00640 1 -- 172.21.15.32:0/2771368561 shutdown_connections 2024-08-31T07:23:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:36.962+0000 7fddd3c00640 1 -- 172.21.15.32:0/2771368561 wait complete. 2024-08-31T07:23:36.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:36 smithi084 ceph-mon[22387]: pgmap v703: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-08-31T07:23:37.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:36 smithi073 ceph-mon[21995]: pgmap v703: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-08-31T07:23:37.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.93 MB/sec execute 446 sec latency 6246.258 ms 2024-08-31T07:23:38.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.92 MB/sec execute 447 sec latency 7246.388 ms 2024-08-31T07:23:38.965 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:23:38.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:23:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:38 smithi084 ceph-mon[22387]: pgmap v704: 97 pgs: 97 active+clean; 271 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-08-31T07:23:39.020 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:38 smithi032 ceph-mon[17252]: pgmap v704: 97 pgs: 97 active+clean; 271 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-08-31T07:23:39.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:38 smithi073 ceph-mon[21995]: pgmap v704: 97 pgs: 97 active+clean; 271 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-08-31T07:23:39.355 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:39.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.91 MB/sec execute 448 sec latency 8246.449 ms 2024-08-31T07:23:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.008+0000 7fc827000640 1 -- 172.21.15.32:0/1879190932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8201099c0 msgr2=0x7fc8201101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.008+0000 7fc827000640 1 --2- 172.21.15.32:0/1879190932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8201099c0 0x7fc8201101f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc818009ca0 tx=0x7fc81802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.008+0000 7fc827000640 1 -- 172.21.15.32:0/1879190932 shutdown_connections 2024-08-31T07:23:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.008+0000 7fc827000640 1 --2- 172.21.15.32:0/1879190932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8201099c0 0x7fc8201101f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.008+0000 7fc827000640 1 --2- 172.21.15.32:0/1879190932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc820109070 0x7fc820109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.008+0000 7fc827000640 1 --2- 172.21.15.32:0/1879190932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc820103730 0x7fc820103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.008+0000 7fc827000640 1 -- 172.21.15.32:0/1879190932 >> 172.21.15.32:0/1879190932 conn(0x7fc8200ff180 msgr2=0x7fc8201015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.008+0000 7fc827000640 1 -- 172.21.15.32:0/1879190932 shutdown_connections 2024-08-31T07:23:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.008+0000 7fc827000640 1 -- 172.21.15.32:0/1879190932 wait complete. 2024-08-31T07:23:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.009+0000 7fc827000640 1 Processor -- start 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.009+0000 7fc827000640 1 -- start start 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc827000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc820103730 0x7fc8200754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc827000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc820109070 0x7fc8200759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc827000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8201099c0 0x7fc820075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc827000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8201136c0 con 0x7fc820109070 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc827000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc820113540 con 0x7fc8201099c0 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc827000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc820113840 con 0x7fc820103730 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc825200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc820109070 0x7fc8200759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc825c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc820103730 0x7fc8200754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc825200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc820109070 0x7fc8200759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50740/0 (socket says 172.21.15.32:50740) 2024-08-31T07:23:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc825200640 1 -- 172.21.15.32:0/752923922 learned_addr learned my addr 172.21.15.32:0/752923922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc825c00640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc820103730 0x7fc8200754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57444/0 (socket says 172.21.15.32:57444) 2024-08-31T07:23:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.010+0000 7fc826600640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8201099c0 0x7fc820075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc825c00640 1 -- 172.21.15.32:0/752923922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8201099c0 msgr2=0x7fc820075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc825c00640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8201099c0 0x7fc820075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc825c00640 1 -- 172.21.15.32:0/752923922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc820109070 msgr2=0x7fc8200759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc825c00640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc820109070 0x7fc8200759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc825c00640 1 -- 172.21.15.32:0/752923922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8201a82f0 con 0x7fc820103730 2024-08-31T07:23:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc825200640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc820109070 0x7fc8200759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc825c00640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc820103730 0x7fc8200754b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc81400e9f0 tx=0x7fc81400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc81e000640 1 -- 172.21.15.32:0/752923922 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81400ccc0 con 0x7fc820103730 2024-08-31T07:23:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc827000640 1 -- 172.21.15.32:0/752923922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8201a85e0 con 0x7fc820103730 2024-08-31T07:23:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc81e000640 1 -- 172.21.15.32:0/752923922 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8140108e0 con 0x7fc820103730 2024-08-31T07:23:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc81e000640 1 -- 172.21.15.32:0/752923922 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814018880 con 0x7fc820103730 2024-08-31T07:23:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.011+0000 7fc827000640 1 -- 172.21.15.32:0/752923922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8201a8b70 con 0x7fc820103730 2024-08-31T07:23:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.014+0000 7fc82f292740 1 Processor -- start 2024-08-31T07:23:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.014+0000 7fc82f292740 1 -- start start 2024-08-31T07:23:40.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc82f292740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963c257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc82f292740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 0x558963c25d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc82f292740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 0x558963c26250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc82f292740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558963b9b750 con 0x558963856490 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc82f292740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558963b8ace0 con 0x55896387f350 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc82f292740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558963b8ec90 con 0x55896385e670 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc826600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963c257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc826600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963c257d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49034/0 (socket says 172.21.15.32:49034) 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc826600640 1 -- 172.21.15.32:0/3215067978 learned_addr learned my addr 172.21.15.32:0/3215067978 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc825200640 1 --2- 172.21.15.32:0/3215067978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 0x558963c26250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.015+0000 7fc81e000640 1 -- 172.21.15.32:0/752923922 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc814010430 con 0x7fc820103730 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.016+0000 7fc826600640 1 -- 172.21.15.32:0/3215067978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 msgr2=0x558963c26250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.016+0000 7fc826600640 1 --2- 172.21.15.32:0/3215067978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 0x558963c26250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.016+0000 7fc826600640 1 -- 172.21.15.32:0/3215067978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 msgr2=0x558963c25d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:23:40.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.016+0000 7fc826600640 1 --2- 172.21.15.32:0/3215067978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 0x558963c25d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.016+0000 7fc826600640 1 -- 172.21.15.32:0/3215067978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589638ea930 con 0x55896387f350 2024-08-31T07:23:40.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.016+0000 7fc826600640 1 --2- 172.21.15.32:0/3215067978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963c257d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc818009ca0 tx=0x7fc81802f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1da09ecec1b234e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:40.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.016+0000 7fc812a00640 1 -- 172.21.15.32:0/3215067978 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818031db0 con 0x55896387f350 2024-08-31T07:23:40.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.017+0000 7fc812a00640 1 -- 172.21.15.32:0/3215067978 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc818036030 con 0x55896387f350 2024-08-31T07:23:40.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.017+0000 7fc812a00640 1 -- 172.21.15.32:0/3215067978 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8180042a0 con 0x55896387f350 2024-08-31T07:23:40.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.017+0000 7fc81e000640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8080b4ca0 0x7fc8080b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.017+0000 7fc82f292740 1 -- 172.21.15.32:0/3215067978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 msgr2=0x558963c257d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.017+0000 7fc82f292740 1 --2- 172.21.15.32:0/3215067978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963c257d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc818009ca0 tx=0x7fc81802f310 comp rx=0 tx=0).stop 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.017+0000 7fc82f292740 1 -- 172.21.15.32:0/3215067978 shutdown_connections 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.017+0000 7fc82f292740 1 --2- 172.21.15.32:0/3215067978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 0x558963c26250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.018+0000 7fc82f292740 1 --2- 172.21.15.32:0/3215067978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 0x558963c25d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.018+0000 7fc82f292740 1 --2- 172.21.15.32:0/3215067978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963c257d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.018+0000 7fc82f292740 1 -- 172.21.15.32:0/3215067978 >> 172.21.15.32:0/3215067978 conn(0x558963873740 msgr2=0x558963c236c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.018+0000 7fc81e000640 1 -- 172.21.15.32:0/752923922 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fc8140e5840 con 0x7fc820103730 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.018+0000 7fc825c00640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8080b4ca0 0x7fc8080b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.018+0000 7fc82f292740 1 -- 172.21.15.32:0/3215067978 shutdown_connections 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.018+0000 7fc82f292740 1 -- 172.21.15.32:0/3215067978 wait complete. 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.018+0000 7fc825c00640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8080b4ca0 0x7fc8080b71b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x558963c24450 tx=0x7fc814009700 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:40.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.019+0000 7fc82f292740 1 -- start start 2024-08-31T07:23:40.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.019+0000 7fc82f292740 1 Processor -- start 2024-08-31T07:23:40.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.019+0000 7fc82f292740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558963856490 0x558963c31450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.019+0000 7fc82f292740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c31990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.019+0000 7fc82f292740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896387f350 0x558963c3d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.019+0000 7fc82f292740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5589639ed890 con 0x55896385e670 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.019+0000 7fc82f292740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558963b11cd0 con 0x558963856490 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.019+0000 7fc82f292740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558963c33dd0 con 0x55896387f350 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.019+0000 7fc82f292740 10 client.0 _ll_register_callbacks cb 0x558963b85ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc826600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558963856490 0x558963c31450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc826600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558963856490 0x558963c31450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49038/0 (socket says 172.21.15.32:49038) 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc826600640 1 -- 172.21.15.32:0/1851113872 learned_addr learned my addr 172.21.15.32:0/1851113872 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc825200640 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c31990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc826600640 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896387f350 0x558963c3d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc825200640 1 -- 172.21.15.32:0/1851113872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896387f350 msgr2=0x558963c3d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:40.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc825200640 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896387f350 0x558963c3d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc825200640 1 -- 172.21.15.32:0/1851113872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558963856490 msgr2=0x558963c31450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc825200640 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558963856490 0x558963c31450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc825200640 1 -- 172.21.15.32:0/1851113872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558963b5b1b0 con 0x55896385e670 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.020+0000 7fc826600640 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558963856490 0x558963c31450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.021+0000 7fc825200640 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c31990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc80c004400 tx=0x7fc80c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac455c954aa6a0c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.021+0000 7fc807400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.021+0000 7fc807400640 1 -- 172.21.15.32:0/1851113872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0129e0 con 0x55896385e670 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.021+0000 7fc807400640 1 -- 172.21.15.32:0/1851113872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fc80c012b80 con 0x55896385e670 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.021+0000 7fc82f292740 1 -- 172.21.15.32:0/1851113872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc818036030 con 0x55896385e670 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.021+0000 7fc807400640 1 -- 172.21.15.32:0/1851113872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc80c012d80 con 0x55896385e670 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.021+0000 7fc82f292740 10 client.17808 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:40.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.021+0000 7fc82f292740 1 -- 172.21.15.32:0/1851113872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558963c3fe90 con 0x55896385e670 2024-08-31T07:23:40.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.022+0000 7fc807400640 1 -- 172.21.15.32:0/1851113872 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc818036030 con 0x55896385e670 2024-08-31T07:23:40.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.022+0000 7fc807400640 1 -- 172.21.15.32:0/1851113872 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fc80c019070 con 0x55896385e670 2024-08-31T07:23:40.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.022+0000 7fc82f292740 10 client.17808 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:40.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.022+0000 7fc82f292740 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:23:40.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.022+0000 7fc82f292740 20 client.17808 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:40.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.022+0000 7fc82f292740 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55896383dcd0 0x558963c43460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.022+0000 7fc82f292740 4 client.17808 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:40.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.022+0000 7fc82f292740 1 -- 172.21.15.32:0/1851113872 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558963b9d6a0 con 0x55896383dcd0 2024-08-31T07:23:40.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.022+0000 7fc826600640 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55896383dcd0 0x558963c43460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.023+0000 7fc826600640 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55896383dcd0 0x558963c43460 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86485d5bd8b3243b server_cookie=79c01701fd4e71b5 in_seq=0 out_seq=0 2024-08-31T07:23:40.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.023+0000 7fc807400640 10 client.17808 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:40.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc807400640 1 -- 172.21.15.32:0/1851113872 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558963b9d6a0 con 0x55896383dcd0 2024-08-31T07:23:40.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc807400640 10 client.17808 handle_command_reply: tid=0 2024-08-31T07:23:40.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 client.17808 shutdown 2024-08-31T07:23:40.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 -- 172.21.15.32:0/1851113872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 msgr2=0x558963c31990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:40.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c31990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc80c004400 tx=0x7fc80c0039c0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 -- 172.21.15.32:0/1851113872 shutdown_connections 2024-08-31T07:23:40.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55896383dcd0 0x558963c43460 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896387f350 0x558963c3d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c31990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 --2- 172.21.15.32:0/1851113872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558963856490 0x558963c31450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 -- 172.21.15.32:0/1851113872 >> 172.21.15.32:0/1851113872 conn(0x558963873740 msgr2=0x558963c289b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc807400640 0 client.17808 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 -- 172.21.15.32:0/1851113872 shutdown_connections 2024-08-31T07:23:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 1 -- 172.21.15.32:0/1851113872 wait complete. 2024-08-31T07:23:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.024+0000 7fc82f292740 20 client.17808 trim_cache size 0 max 16384 2024-08-31T07:23:40.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.043+0000 7fc82f292740 1 Processor -- start 2024-08-31T07:23:40.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.043+0000 7fc82f292740 1 -- start start 2024-08-31T07:23:40.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc82f292740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 0x558963c32e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc82f292740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 0x558963c2a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc82f292740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963c2adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc82f292740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8201b1350 con 0x558963856490 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc82f292740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8201b2ec0 con 0x55896387f350 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc82f292740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8201b1a20 con 0x55896385e670 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc826600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 0x558963c32e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc825200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 0x558963c2a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc826600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 0x558963c32e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50764/0 (socket says 172.21.15.32:50764) 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc826600640 1 -- 172.21.15.32:0/2445175488 learned_addr learned my addr 172.21.15.32:0/2445175488 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc825200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 0x558963c2a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57470/0 (socket says 172.21.15.32:57470) 2024-08-31T07:23:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc826600640 1 --2- 172.21.15.32:0/2445175488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963c2adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc826600640 1 -- 172.21.15.32:0/2445175488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 msgr2=0x558963c2a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc826600640 1 --2- 172.21.15.32:0/2445175488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 0x558963c2a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc826600640 1 -- 172.21.15.32:0/2445175488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 msgr2=0x558963c2adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc826600640 1 --2- 172.21.15.32:0/2445175488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963c2adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.044+0000 7fc826600640 1 -- 172.21.15.32:0/2445175488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558963c28630 con 0x558963856490 2024-08-31T07:23:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.045+0000 7fc826600640 1 --2- 172.21.15.32:0/2445175488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 0x558963c32e80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc818003e00 tx=0x7fc818005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54799e8d274b5901 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.045+0000 7fc812000640 1 -- 172.21.15.32:0/2445175488 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818031630 con 0x558963856490 2024-08-31T07:23:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.045+0000 7fc812000640 1 -- 172.21.15.32:0/2445175488 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc818036070 con 0x558963856490 2024-08-31T07:23:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.045+0000 7fc812000640 1 -- 172.21.15.32:0/2445175488 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8180305c0 con 0x558963856490 2024-08-31T07:23:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.046+0000 7fc82f292740 1 -- 172.21.15.32:0/2445175488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 msgr2=0x558963c32e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.046+0000 7fc82f292740 1 --2- 172.21.15.32:0/2445175488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 0x558963c32e80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc818003e00 tx=0x7fc818005d80 comp rx=0 tx=0).stop 2024-08-31T07:23:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.046+0000 7fc82f292740 1 -- 172.21.15.32:0/2445175488 shutdown_connections 2024-08-31T07:23:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.046+0000 7fc82f292740 1 --2- 172.21.15.32:0/2445175488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963c2adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.046+0000 7fc82f292740 1 --2- 172.21.15.32:0/2445175488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55896385e670 0x558963c2a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.046+0000 7fc82f292740 1 --2- 172.21.15.32:0/2445175488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558963856490 0x558963c32e80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.046+0000 7fc82f292740 1 -- 172.21.15.32:0/2445175488 >> 172.21.15.32:0/2445175488 conn(0x558963873740 msgr2=0x558963c32770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.046+0000 7fc82f292740 1 -- 172.21.15.32:0/2445175488 shutdown_connections 2024-08-31T07:23:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.046+0000 7fc82f292740 1 -- 172.21.15.32:0/2445175488 wait complete. 2024-08-31T07:23:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.047+0000 7fc82f292740 1 -- start start 2024-08-31T07:23:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.047+0000 7fc82f292740 1 Processor -- start 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.047+0000 7fc82f292740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558963856490 0x558963c35930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.047+0000 7fc82f292740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c35f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.047+0000 7fc82f292740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963ca6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.047+0000 7fc82f292740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8201b27c0 con 0x55896385e670 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.047+0000 7fc82f292740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8201b20f0 con 0x55896387f350 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.047+0000 7fc82f292740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8201af350 con 0x558963856490 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.047+0000 7fc82f292740 10 client.0 _ll_register_callbacks cb 0x7fc80c00bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc825200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c35f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc825200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c35f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50766/0 (socket says 172.21.15.32:50766) 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc825200640 1 -- 172.21.15.32:0/1144843525 learned_addr learned my addr 172.21.15.32:0/1144843525 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:40.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc826600640 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963ca6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc826600640 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558963856490 0x558963c35930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc825200640 1 -- 172.21.15.32:0/1144843525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558963856490 msgr2=0x558963c35930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc825200640 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558963856490 0x558963c35930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc825200640 1 -- 172.21.15.32:0/1144843525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 msgr2=0x558963ca6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc825200640 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963ca6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc825200640 1 -- 172.21.15.32:0/1144843525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558963ca7460 con 0x55896385e670 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc826600640 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558963856490 0x558963c35930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc825200640 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c35f10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0105c0 tx=0x7fc80c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f251914748ef5fdd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.048+0000 7fc807400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.049+0000 7fc807400640 1 -- 172.21.15.32:0/1144843525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c028040 con 0x55896385e670 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.049+0000 7fc807400640 1 -- 172.21.15.32:0/1144843525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fc80c005460 con 0x55896385e670 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.049+0000 7fc82f292740 1 -- 172.21.15.32:0/1144843525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc818036070 con 0x55896385e670 2024-08-31T07:23:40.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.049+0000 7fc807400640 1 -- 172.21.15.32:0/1144843525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc80c005850 con 0x55896385e670 2024-08-31T07:23:40.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.049+0000 7fc82f292740 10 client.17814 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:40.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.049+0000 7fc82f292740 1 -- 172.21.15.32:0/1144843525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558963cae7f0 con 0x55896385e670 2024-08-31T07:23:40.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.049+0000 7fc807400640 1 -- 172.21.15.32:0/1144843525 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc818036070 con 0x55896385e670 2024-08-31T07:23:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.050+0000 7fc807400640 1 -- 172.21.15.32:0/1144843525 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fc80c02b070 con 0x55896385e670 2024-08-31T07:23:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.050+0000 7fc82f292740 10 client.17814 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.050+0000 7fc82f292740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:23:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.050+0000 7fc82f292740 20 client.17814 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.050+0000 7fc82f292740 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558963813080 0x558963cb1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.050+0000 7fc82f292740 4 client.17814 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:23:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.050+0000 7fc82f292740 1 -- 172.21.15.32:0/1144843525 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558963b99140 con 0x558963813080 2024-08-31T07:23:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.050+0000 7fc826600640 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558963813080 0x558963cb1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.051+0000 7fc826600640 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558963813080 0x558963cb1010 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de2c67a6f1fd848 server_cookie=5aa8b53d0983fe39 in_seq=0 out_seq=0 2024-08-31T07:23:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.051+0000 7fc807400640 10 client.17814 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.052+0000 7fc807400640 1 -- 172.21.15.32:0/1144843525 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558963b99140 con 0x558963813080 2024-08-31T07:23:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.052+0000 7fc807400640 10 client.17814 handle_command_reply: tid=0 2024-08-31T07:23:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.052+0000 7fc82f292740 1 client.17814 shutdown 2024-08-31T07:23:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 1 -- 172.21.15.32:0/1144843525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 msgr2=0x558963c35f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c35f10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0105c0 tx=0x7fc80c005270 comp rx=0 tx=0).stop 2024-08-31T07:23:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 1 -- 172.21.15.32:0/1144843525 shutdown_connections 2024-08-31T07:23:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558963813080 0x558963cb1010 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55896387f350 0x558963ca6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55896385e670 0x558963c35f10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 1 --2- 172.21.15.32:0/1144843525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558963856490 0x558963c35930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 1 -- 172.21.15.32:0/1144843525 >> 172.21.15.32:0/1144843525 conn(0x558963873740 msgr2=0x558963c32510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc807400640 0 client.17814 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 1 -- 172.21.15.32:0/1144843525 shutdown_connections 2024-08-31T07:23:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 1 -- 172.21.15.32:0/1144843525 wait complete. 2024-08-31T07:23:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.053+0000 7fc82f292740 20 client.17814 trim_cache size 0 max 16384 2024-08-31T07:23:40.056 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:23:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:23:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "c2f847a7-820f-48ef-90f2-3d4a5f290866", 2024-08-31T07:23:40.056 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:23:40.056 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:23:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 -- 172.21.15.32:0/752923922 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8080b4ca0 msgr2=0x7fc8080b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8080b4ca0 0x7fc8080b71b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x558963c24450 tx=0x7fc814009700 comp rx=0 tx=0).stop 2024-08-31T07:23:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 -- 172.21.15.32:0/752923922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc820103730 msgr2=0x7fc8200754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc820103730 0x7fc8200754b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc81400e9f0 tx=0x7fc81400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 -- 172.21.15.32:0/752923922 shutdown_connections 2024-08-31T07:23:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8080b4ca0 0x7fc8080b71b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8201099c0 0x7fc820075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc820109070 0x7fc8200759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 --2- 172.21.15.32:0/752923922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc820103730 0x7fc8200754b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 -- 172.21.15.32:0/752923922 >> 172.21.15.32:0/752923922 conn(0x7fc8200ff180 msgr2=0x7fc8201015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 -- 172.21.15.32:0/752923922 shutdown_connections 2024-08-31T07:23:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:40.054+0000 7fc827000640 1 -- 172.21.15.32:0/752923922 wait complete. 2024-08-31T07:23:40.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.91 MB/sec execute 449 sec latency 9246.536 ms 2024-08-31T07:23:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: pgmap v705: 97 pgs: 97 active+clean; 271 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 895 KiB/s wr, 105 op/s 2024-08-31T07:23:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:23:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:23:40.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:23:40.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:40.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:23:40.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:40.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:23:40.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:40.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:40.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:40 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.088 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: pgmap v705: 97 pgs: 97 active+clean; 271 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 895 KiB/s wr, 105 op/s 2024-08-31T07:23:41.088 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:40 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: pgmap v705: 97 pgs: 97 active+clean; 271 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 895 KiB/s wr, 105 op/s 2024-08-31T07:23:41.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:23:41.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:23:41.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:23:41.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:23:41.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:23:41.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:40 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:23:41.679 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:41.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.90 MB/sec execute 450 sec latency 10246.596 ms 2024-08-31T07:23:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:41 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:23:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:41 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:41 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:23:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:41 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:23:41.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:41 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:41 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:23:41.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:41 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:41.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:41 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:42.013 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:41 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:23:42.013 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:41 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:42.013 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:41 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:23:42.013 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:41 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:23:42.013 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:41 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:42.013 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:41 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:23:42.013 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:41 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:42.014 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:41 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:42.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:41 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:23:42.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:41 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:42.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:41 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:23:42.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:41 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:23:42.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:41 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:42.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:41 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:23:42.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:41 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:42.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:41 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:23:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.392+0000 7f49a2000640 1 -- 172.21.15.32:0/2296440171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499c109240 msgr2=0x7f499c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.392+0000 7f49a2000640 1 --2- 172.21.15.32:0/2296440171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499c109240 0x7f499c109650 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4984009900 tx=0x7f498402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.392+0000 7f49a2000640 1 -- 172.21.15.32:0/2296440171 shutdown_connections 2024-08-31T07:23:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.392+0000 7f49a2000640 1 --2- 172.21.15.32:0/2296440171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c109b90 0x7f499c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.392+0000 7f49a2000640 1 --2- 172.21.15.32:0/2296440171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499c109240 0x7f499c109650 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.392+0000 7f49a2000640 1 --2- 172.21.15.32:0/2296440171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f499c103900 0x7f499c103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.392+0000 7f49a2000640 1 -- 172.21.15.32:0/2296440171 >> 172.21.15.32:0/2296440171 conn(0x7f499c07b5f0 msgr2=0x7f499c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.392+0000 7f49a2000640 1 -- 172.21.15.32:0/2296440171 shutdown_connections 2024-08-31T07:23:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.392+0000 7f49a2000640 1 -- 172.21.15.32:0/2296440171 wait complete. 2024-08-31T07:23:42.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.393+0000 7f49a2000640 1 Processor -- start 2024-08-31T07:23:42.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.393+0000 7f49a2000640 1 -- start start 2024-08-31T07:23:42.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f49a2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f499c103900 0x7f499c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f49a2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499c109240 0x7f499c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f49a2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c109b90 0x7f499c19faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f49a2000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f499c113850 con 0x7f499c109b90 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f49a2000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1136d0 con 0x7f499c109240 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f49a2000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1139d0 con 0x7f499c103900 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f4992a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499c109240 0x7f499c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f4992a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499c109240 0x7f499c19a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49082/0 (socket says 172.21.15.32:49082) 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f4992a00640 1 -- 172.21.15.32:0/1547186824 learned_addr learned my addr 172.21.15.32:0/1547186824 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f499b400640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f499c103900 0x7f499c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.394+0000 7f499be00640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c109b90 0x7f499c19faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f4992a00640 1 -- 172.21.15.32:0/1547186824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f499c103900 msgr2=0x7f499c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f4992a00640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f499c103900 0x7f499c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f4992a00640 1 -- 172.21.15.32:0/1547186824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c109b90 msgr2=0x7f499c19faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f4992a00640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c109b90 0x7f499c19faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f4992a00640 1 -- 172.21.15.32:0/1547186824 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c1a01d0 con 0x7f499c109240 2024-08-31T07:23:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f4992a00640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499c109240 0x7f499c19a800 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f498402a910 tx=0x7f4984031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f4998c00640 1 -- 172.21.15.32:0/1547186824 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498403f030 con 0x7f499c109240 2024-08-31T07:23:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f4998c00640 1 -- 172.21.15.32:0/1547186824 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4984004490 con 0x7f499c109240 2024-08-31T07:23:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f49a2000640 1 -- 172.21.15.32:0/1547186824 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499c1a0460 con 0x7f499c109240 2024-08-31T07:23:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f4998c00640 1 -- 172.21.15.32:0/1547186824 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984035070 con 0x7f499c109240 2024-08-31T07:23:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.395+0000 7f49a2000640 1 -- 172.21.15.32:0/1547186824 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f499c1a6c20 con 0x7f499c109240 2024-08-31T07:23:42.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.398+0000 7f49a8c97740 1 Processor -- start 2024-08-31T07:23:42.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f49a8c97740 1 -- start start 2024-08-31T07:23:42.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 0x563199c8f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199c8fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199c902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f49a8c97740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563199c06770 con 0x5631998c9390 2024-08-31T07:23:42.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f49a8c97740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563199bf5d00 con 0x5631998c11b0 2024-08-31T07:23:42.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f49a8c97740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563199bf9cb0 con 0x5631998ea070 2024-08-31T07:23:42.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f4998c00640 1 -- 172.21.15.32:0/1547186824 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4984042030 con 0x7f499c109240 2024-08-31T07:23:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f499be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 0x563199c8f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f499be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 0x563199c8f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57502/0 (socket says 172.21.15.32:57502) 2024-08-31T07:23:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f499be00640 1 -- 172.21.15.32:0/3644881946 learned_addr learned my addr 172.21.15.32:0/3644881946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f4992a00640 1 --2- 172.21.15.32:0/3644881946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199c902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.399+0000 7f499b400640 1 --2- 172.21.15.32:0/3644881946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199c8fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.400+0000 7f499be00640 1 -- 172.21.15.32:0/3644881946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 msgr2=0x563199c8fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.400+0000 7f499be00640 1 --2- 172.21.15.32:0/3644881946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199c8fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.400+0000 7f499be00640 1 -- 172.21.15.32:0/3644881946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 msgr2=0x563199c902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.400+0000 7f499be00640 1 --2- 172.21.15.32:0/3644881946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199c902f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.400+0000 7f499be00640 1 -- 172.21.15.32:0/3644881946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563199955650 con 0x5631998ea070 2024-08-31T07:23:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.400+0000 7f499be00640 1 --2- 172.21.15.32:0/3644881946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 0x563199c8f870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4994009fd0 tx=0x7f49940118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39047ef348440aac server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.400+0000 7f498be00640 1 -- 172.21.15.32:0/3644881946 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994004d20 con 0x5631998ea070 2024-08-31T07:23:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.400+0000 7f498be00640 1 -- 172.21.15.32:0/3644881946 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4994004050 con 0x5631998ea070 2024-08-31T07:23:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.400+0000 7f498be00640 1 -- 172.21.15.32:0/3644881946 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994005260 con 0x5631998ea070 2024-08-31T07:23:42.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.401+0000 7f4998c00640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f49800b4d20 0x7f49800b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.401+0000 7f4998c00640 1 -- 172.21.15.32:0/1547186824 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f4984105830 con 0x7f499c109240 2024-08-31T07:23:42.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f499b400640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f49800b4d20 0x7f49800b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f49a8c97740 1 -- 172.21.15.32:0/3644881946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 msgr2=0x563199c8f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3644881946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 0x563199c8f870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4994009fd0 tx=0x7f49940118d0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f49a8c97740 1 -- 172.21.15.32:0/3644881946 shutdown_connections 2024-08-31T07:23:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3644881946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199c902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3644881946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199c8fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3644881946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 0x563199c8f870 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f49a8c97740 1 -- 172.21.15.32:0/3644881946 >> 172.21.15.32:0/3644881946 conn(0x5631998de460 msgr2=0x563199c8d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f49a8c97740 1 -- 172.21.15.32:0/3644881946 shutdown_connections 2024-08-31T07:23:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f49a8c97740 1 -- 172.21.15.32:0/3644881946 wait complete. 2024-08-31T07:23:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.402+0000 7f499b400640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f49800b4d20 0x7f49800b7230 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x563199c8e4f0 tx=0x7f498c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.403+0000 7f49a8c97740 1 -- start start 2024-08-31T07:23:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.403+0000 7f49a8c97740 1 Processor -- start 2024-08-31T07:23:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.403+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199c9ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.403+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199ca0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.403+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 0x563199ca5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.403+0000 7f49a8c97740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563199a58700 con 0x5631998c9390 2024-08-31T07:23:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.403+0000 7f49a8c97740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563199b7ccb0 con 0x5631998c11b0 2024-08-31T07:23:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.403+0000 7f49a8c97740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563199c9e050 con 0x5631998ea070 2024-08-31T07:23:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.403+0000 7f49a8c97740 10 client.0 _ll_register_callbacks cb 0x563199ba2430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f4992a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199ca0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f4992a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199ca0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50812/0 (socket says 172.21.15.32:50812) 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f4992a00640 1 -- 172.21.15.32:0/3991165273 learned_addr learned my addr 172.21.15.32:0/3991165273 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f499b400640 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199c9ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f499be00640 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 0x563199ca5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f499b400640 1 -- 172.21.15.32:0/3991165273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 msgr2=0x563199ca5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f499b400640 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 0x563199ca5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f499b400640 1 -- 172.21.15.32:0/3991165273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 msgr2=0x563199ca0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f499b400640 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199ca0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f499b400640 1 -- 172.21.15.32:0/3991165273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563199bbd870 con 0x5631998c11b0 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f4992a00640 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199ca0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f499b400640 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199c9ff30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f498c01e9d0 tx=0x7f498c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11aff0c0250e2b4b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.404+0000 7f4988c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:23:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.405+0000 7f4988c00640 1 -- 172.21.15.32:0/3991165273 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c012ea0 con 0x5631998c11b0 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.405+0000 7f49a8c97740 1 -- 172.21.15.32:0/3991165273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4994004050 con 0x5631998c11b0 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.405+0000 7f4988c00640 1 -- 172.21.15.32:0/3991165273 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f498c018030 con 0x5631998c11b0 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.405+0000 7f4988c00640 1 -- 172.21.15.32:0/3991165273 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f498c004020 con 0x5631998c11b0 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.405+0000 7f49a8c97740 10 client.27827 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.405+0000 7f49a8c97740 1 -- 172.21.15.32:0/3991165273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563199cae530 con 0x5631998c11b0 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.405+0000 7f4988c00640 1 -- 172.21.15.32:0/3991165273 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4994004050 con 0x5631998c11b0 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.406+0000 7f4988c00640 1 -- 172.21.15.32:0/3991165273 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f498c029420 con 0x5631998c11b0 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.406+0000 7f49a8c97740 10 client.27827 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.406+0000 7f49a8c97740 10 client.27827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.406+0000 7f49a8c97740 20 client.27827 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.406+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5631998a89f0 0x563199cb1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.406+0000 7f49a8c97740 4 client.27827 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.406+0000 7f49a8c97740 1 -- 172.21.15.32:0/3991165273 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563199c086c0 con 0x5631998a89f0 2024-08-31T07:23:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.406+0000 7f4992a00640 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5631998a89f0 0x563199cb1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.407+0000 7f4992a00640 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5631998a89f0 0x563199cb1aa0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf78178fecc8e4ed server_cookie=675195f31d526377 in_seq=0 out_seq=0 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.407+0000 7f4988c00640 10 client.27827 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f4988c00640 1 -- 172.21.15.32:0/3991165273 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563199c086c0 con 0x5631998a89f0 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f4988c00640 10 client.27827 handle_command_reply: tid=0 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 client.27827 shutdown 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 -- 172.21.15.32:0/3991165273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 msgr2=0x563199c9ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199c9ff30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f498c01e9d0 tx=0x7f498c004730 comp rx=0 tx=0).stop 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 -- 172.21.15.32:0/3991165273 shutdown_connections 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5631998a89f0 0x563199cb1aa0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998ea070 0x563199ca5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199ca0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3991165273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199c9ff30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f4988c00640 0 client.27827 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 -- 172.21.15.32:0/3991165273 >> 172.21.15.32:0/3991165273 conn(0x5631998de460 msgr2=0x563199c92a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 -- 172.21.15.32:0/3991165273 shutdown_connections 2024-08-31T07:23:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.408+0000 7f49a8c97740 1 -- 172.21.15.32:0/3991165273 wait complete. 2024-08-31T07:23:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.409+0000 7f49a8c97740 20 client.27827 trim_cache size 0 max 16384 2024-08-31T07:23:42.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.428+0000 7f49a8c97740 1 Processor -- start 2024-08-31T07:23:42.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.428+0000 7f49a8c97740 1 -- start start 2024-08-31T07:23:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.428+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199ca5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.428+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998c9390 0x563199ca5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.428+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998ea070 0x563199c9c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.428+0000 7f49a8c97740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1ad1c0 con 0x5631998ea070 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.428+0000 7f49a8c97740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1aed30 con 0x5631998c11b0 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.428+0000 7f49a8c97740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1ad890 con 0x5631998c9390 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.428+0000 7f499b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199ca5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.428+0000 7f499be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998ea070 0x563199c9c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.429+0000 7f499b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199ca5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49104/0 (socket says 172.21.15.32:49104) 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.429+0000 7f499b400640 1 -- 172.21.15.32:0/3757141816 learned_addr learned my addr 172.21.15.32:0/3757141816 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.429+0000 7f4992a00640 1 --2- 172.21.15.32:0/3757141816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998c9390 0x563199ca5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.429+0000 7f499b400640 1 -- 172.21.15.32:0/3757141816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998c9390 msgr2=0x563199ca5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.429+0000 7f499b400640 1 --2- 172.21.15.32:0/3757141816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998c9390 0x563199ca5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.429+0000 7f499b400640 1 -- 172.21.15.32:0/3757141816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998ea070 msgr2=0x563199c9c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.429+0000 7f499b400640 1 --2- 172.21.15.32:0/3757141816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998ea070 0x563199c9c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.429+0000 7f499b400640 1 -- 172.21.15.32:0/3757141816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563199c93980 con 0x5631998c11b0 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.429+0000 7f499b400640 1 --2- 172.21.15.32:0/3757141816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199ca5310 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f498c012cb0 tx=0x7f498c004660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ada72880bc11082 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.429+0000 7f498b400640 1 -- 172.21.15.32:0/3757141816 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c029b30 con 0x5631998c11b0 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.430+0000 7f498b400640 1 -- 172.21.15.32:0/3757141816 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f498c029420 con 0x5631998c11b0 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.430+0000 7f498b400640 1 -- 172.21.15.32:0/3757141816 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c003d70 con 0x5631998c11b0 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.430+0000 7f49a8c97740 1 -- 172.21.15.32:0/3757141816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 msgr2=0x563199ca5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.430+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3757141816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199ca5310 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f498c012cb0 tx=0x7f498c004660 comp rx=0 tx=0).stop 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.430+0000 7f49a8c97740 1 -- 172.21.15.32:0/3757141816 shutdown_connections 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.430+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3757141816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998ea070 0x563199c9c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.430+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3757141816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998c9390 0x563199ca5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.430+0000 7f49a8c97740 1 --2- 172.21.15.32:0/3757141816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998c11b0 0x563199ca5310 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.430+0000 7f49a8c97740 1 -- 172.21.15.32:0/3757141816 >> 172.21.15.32:0/3757141816 conn(0x5631998de460 msgr2=0x563199c9e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.430+0000 7f49a8c97740 1 -- 172.21.15.32:0/3757141816 shutdown_connections 2024-08-31T07:23:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.431+0000 7f49a8c97740 1 -- 172.21.15.32:0/3757141816 wait complete. 2024-08-31T07:23:42.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.431+0000 7f49a8c97740 1 -- start start 2024-08-31T07:23:42.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.431+0000 7f49a8c97740 1 Processor -- start 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998c11b0 0x563199cab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199ca5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f49a8c97740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998ea070 0x563199ca63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f49a8c97740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1ae630 con 0x5631998c9390 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f49a8c97740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1adf60 con 0x5631998ea070 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f49a8c97740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1ab1a0 con 0x5631998c11b0 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f49a8c97740 10 client.0 _ll_register_callbacks cb 0x7f498c01e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f4992a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199ca5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f499be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998ea070 0x563199ca63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f499be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998ea070 0x563199ca63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49112/0 (socket says 172.21.15.32:49112) 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f499be00640 1 -- 172.21.15.32:0/2457357553 learned_addr learned my addr 172.21.15.32:0/2457357553 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f499b400640 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998c11b0 0x563199cab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f499be00640 1 -- 172.21.15.32:0/2457357553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998c11b0 msgr2=0x563199cab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f499be00640 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998c11b0 0x563199cab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f499be00640 1 -- 172.21.15.32:0/2457357553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 msgr2=0x563199ca5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f499be00640 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199ca5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.432+0000 7f499be00640 1 -- 172.21.15.32:0/2457357553 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563199ca69f0 con 0x5631998ea070 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.433+0000 7f499be00640 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998ea070 0x563199ca63b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4994014040 tx=0x7f4994004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614b1ddc90002348 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.433+0000 7f4988c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.433+0000 7f4988c00640 1 -- 172.21.15.32:0/2457357553 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994009a50 con 0x5631998ea070 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.433+0000 7f49a8c97740 1 -- 172.21.15.32:0/2457357553 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498c029420 con 0x5631998ea070 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.433+0000 7f4988c00640 1 -- 172.21.15.32:0/2457357553 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f4994004050 con 0x5631998ea070 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.433+0000 7f4988c00640 1 -- 172.21.15.32:0/2457357553 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4994004250 con 0x5631998ea070 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.433+0000 7f49a8c97740 10 client.27833 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.434+0000 7f49a8c97740 1 -- 172.21.15.32:0/2457357553 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563199d1d260 con 0x5631998ea070 2024-08-31T07:23:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.434+0000 7f4988c00640 1 -- 172.21.15.32:0/2457357553 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f498c029420 con 0x5631998ea070 2024-08-31T07:23:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.434+0000 7f4988c00640 1 -- 172.21.15.32:0/2457357553 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f49940057a0 con 0x5631998ea070 2024-08-31T07:23:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.434+0000 7f49a8c97740 10 client.27833 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.434+0000 7f49a8c97740 10 client.27833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:23:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.434+0000 7f49a8c97740 20 client.27833 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.434+0000 7f49a8c97740 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56319987dda0 0x563199d1f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.434+0000 7f49a8c97740 4 client.27833 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:23:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.434+0000 7f49a8c97740 1 -- 172.21.15.32:0/2457357553 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563199c04160 con 0x56319987dda0 2024-08-31T07:23:42.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.434+0000 7f499b400640 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56319987dda0 0x563199d1f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:42.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.435+0000 7f499b400640 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56319987dda0 0x563199d1f750 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2345f0d3fc74c0c3 server_cookie=357901b9e0dd67ea in_seq=0 out_seq=0 2024-08-31T07:23:42.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.435+0000 7f4988c00640 10 client.27833 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.435+0000 7f4988c00640 1 -- 172.21.15.32:0/2457357553 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x563199c04160 con 0x56319987dda0 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.435+0000 7f4988c00640 10 client.27833 handle_command_reply: tid=0 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.435+0000 7f49a8c97740 1 client.27833 shutdown 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 1 -- 172.21.15.32:0/2457357553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998ea070 msgr2=0x563199ca63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998ea070 0x563199ca63b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4994014040 tx=0x7f4994004b80 comp rx=0 tx=0).stop 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 1 -- 172.21.15.32:0/2457357553 shutdown_connections 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56319987dda0 0x563199d1f750 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631998ea070 0x563199ca63b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631998c9390 0x563199ca5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 1 --2- 172.21.15.32:0/2457357553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631998c11b0 0x563199cab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f4988c00640 0 client.27833 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 1 -- 172.21.15.32:0/2457357553 >> 172.21.15.32:0/2457357553 conn(0x5631998de460 msgr2=0x563199c8bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 1 -- 172.21.15.32:0/2457357553 shutdown_connections 2024-08-31T07:23:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 1 -- 172.21.15.32:0/2457357553 wait complete. 2024-08-31T07:23:42.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.436+0000 7f49a8c97740 20 client.27833 trim_cache size 0 max 16384 2024-08-31T07:23:42.440 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:23:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:23:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:23:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:23:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "c2f847a7-820f-48ef-90f2-3d4a5f290866", 2024-08-31T07:23:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:23:42.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:23:42.441 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:23:42.445 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:23:42.448 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 -- 172.21.15.32:0/1547186824 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f49800b4d20 msgr2=0x7f49800b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f49800b4d20 0x7f49800b7230 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x563199c8e4f0 tx=0x7f498c005f90 comp rx=0 tx=0).stop 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 -- 172.21.15.32:0/1547186824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499c109240 msgr2=0x7f499c19a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499c109240 0x7f499c19a800 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f498402a910 tx=0x7f4984031a20 comp rx=0 tx=0).stop 2024-08-31T07:23:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 -- 172.21.15.32:0/1547186824 shutdown_connections 2024-08-31T07:23:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f49800b4d20 0x7f49800b7230 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499c109b90 0x7f499c19faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499c109240 0x7f499c19a800 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 --2- 172.21.15.32:0/1547186824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f499c103900 0x7f499c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 -- 172.21.15.32:0/1547186824 >> 172.21.15.32:0/1547186824 conn(0x7f499c07b5f0 msgr2=0x7f499c1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 -- 172.21.15.32:0/1547186824 shutdown_connections 2024-08-31T07:23:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:42.437+0000 7f49a2000640 1 -- 172.21.15.32:0/1547186824 wait complete. 2024-08-31T07:23:42.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.89 MB/sec execute 451 sec latency 11246.650 ms 2024-08-31T07:23:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:42 smithi084 ceph-mon[22387]: pgmap v706: 97 pgs: 97 active+clean; 272 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 977 KiB/s wr, 103 op/s 2024-08-31T07:23:43.086 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:42 smithi032 ceph-mon[17252]: pgmap v706: 97 pgs: 97 active+clean; 272 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 977 KiB/s wr, 103 op/s 2024-08-31T07:23:43.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:42 smithi073 ceph-mon[21995]: pgmap v706: 97 pgs: 97 active+clean; 272 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 977 KiB/s wr, 103 op/s 2024-08-31T07:23:43.597 INFO:tasks.cephfs.filesystem:scrub status for tag:c2f847a7-820f-48ef-90f2-3d4a5f290866 - {'path': '/', 'tag': 'c2f847a7-820f-48ef-90f2-3d4a5f290866', 'options': 'recursive,force'} 2024-08-31T07:23:43.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.89 MB/sec execute 452 sec latency 12246.717 ms 2024-08-31T07:23:44.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.88 MB/sec execute 453 sec latency 13246.857 ms 2024-08-31T07:23:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:44 smithi084 ceph-mon[22387]: pgmap v707: 97 pgs: 97 active+clean; 272 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 691 KiB/s wr, 62 op/s 2024-08-31T07:23:45.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:44 smithi032 ceph-mon[17252]: pgmap v707: 97 pgs: 97 active+clean; 272 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 691 KiB/s wr, 62 op/s 2024-08-31T07:23:45.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:44 smithi073 ceph-mon[21995]: pgmap v707: 97 pgs: 97 active+clean; 272 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 691 KiB/s wr, 62 op/s 2024-08-31T07:23:45.771 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.87 MB/sec execute 454 sec latency 14246.943 ms 2024-08-31T07:23:46.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.87 MB/sec execute 455 sec latency 15247.044 ms 2024-08-31T07:23:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:46 smithi084 ceph-mon[22387]: pgmap v708: 97 pgs: 97 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 706 KiB/s wr, 51 op/s 2024-08-31T07:23:47.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:46 smithi032 ceph-mon[17252]: pgmap v708: 97 pgs: 97 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 706 KiB/s wr, 51 op/s 2024-08-31T07:23:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:46 smithi073 ceph-mon[21995]: pgmap v708: 97 pgs: 97 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 706 KiB/s wr, 51 op/s 2024-08-31T07:23:47.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.86 MB/sec execute 456 sec latency 16247.146 ms 2024-08-31T07:23:48.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.86 MB/sec execute 457 sec latency 17247.307 ms 2024-08-31T07:23:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:48 smithi084 ceph-mon[22387]: pgmap v709: 97 pgs: 97 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 899 KiB/s wr, 51 op/s 2024-08-31T07:23:49.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:48 smithi032 ceph-mon[17252]: pgmap v709: 97 pgs: 97 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 899 KiB/s wr, 51 op/s 2024-08-31T07:23:49.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:48 smithi073 ceph-mon[21995]: pgmap v709: 97 pgs: 97 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 899 KiB/s wr, 51 op/s 2024-08-31T07:23:49.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.85 MB/sec execute 458 sec latency 18247.478 ms 2024-08-31T07:23:50.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:23:50.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.84 MB/sec execute 459 sec latency 19247.555 ms 2024-08-31T07:23:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:50 smithi084 ceph-mon[22387]: pgmap v710: 97 pgs: 97 active+clean; 272 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 771 KiB/s wr, 41 op/s 2024-08-31T07:23:51.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:50 smithi032 ceph-mon[17252]: pgmap v710: 97 pgs: 97 active+clean; 272 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 771 KiB/s wr, 41 op/s 2024-08-31T07:23:51.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:50 smithi073 ceph-mon[21995]: pgmap v710: 97 pgs: 97 active+clean; 272 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 771 KiB/s wr, 41 op/s 2024-08-31T07:23:51.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.679+0000 7f91ed800640 1 -- 172.21.15.32:0/3560562912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e80776e0 msgr2=0x7f91e8077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:51.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.679+0000 7f91ed800640 1 --2- 172.21.15.32:0/3560562912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e80776e0 0x7f91e8077af0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009900 tx=0x7f91d802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.679+0000 7f91ed800640 1 -- 172.21.15.32:0/3560562912 shutdown_connections 2024-08-31T07:23:51.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.679+0000 7f91ed800640 1 --2- 172.21.15.32:0/3560562912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e8114310 0x7f91e8116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.679+0000 7f91ed800640 1 --2- 172.21.15.32:0/3560562912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91e8078030 0x7f91e8113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.679+0000 7f91ed800640 1 --2- 172.21.15.32:0/3560562912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e80776e0 0x7f91e8077af0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.679+0000 7f91ed800640 1 -- 172.21.15.32:0/3560562912 >> 172.21.15.32:0/3560562912 conn(0x7f91e8101420 msgr2=0x7f91e8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:51.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.679+0000 7f91ed800640 1 -- 172.21.15.32:0/3560562912 shutdown_connections 2024-08-31T07:23:51.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.679+0000 7f91ed800640 1 -- 172.21.15.32:0/3560562912 wait complete. 2024-08-31T07:23:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.680+0000 7f91ed800640 1 Processor -- start 2024-08-31T07:23:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.680+0000 7f91ed800640 1 -- start start 2024-08-31T07:23:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91ed800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e80776e0 0x7f91e81a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91ed800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8078030 0x7f91e81a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91ed800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91e8114310 0x7f91e81a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91ed800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f91e8119c70 con 0x7f91e8078030 2024-08-31T07:23:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91ed800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f91e8119af0 con 0x7f91e8114310 2024-08-31T07:23:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91ed800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f91e8119df0 con 0x7f91e80776e0 2024-08-31T07:23:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91e7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8078030 0x7f91e81a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e80776e0 0x7f91e81a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e80776e0 0x7f91e81a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42000/0 (socket says 172.21.15.32:42000) 2024-08-31T07:23:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91e7e00640 1 -- 172.21.15.32:0/108726614 learned_addr learned my addr 172.21.15.32:0/108726614 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.681+0000 7f91ece00640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91e8114310 0x7f91e81a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91ece00640 1 -- 172.21.15.32:0/108726614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e80776e0 msgr2=0x7f91e81a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91ece00640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e80776e0 0x7f91e81a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91ece00640 1 -- 172.21.15.32:0/108726614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8078030 msgr2=0x7f91e81a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91ece00640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8078030 0x7f91e81a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91ece00640 1 -- 172.21.15.32:0/108726614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e81a66b0 con 0x7f91e8114310 2024-08-31T07:23:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91e7e00640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e80776e0 0x7f91e81a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91e7400640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8078030 0x7f91e81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91ece00640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91e8114310 0x7f91e81a5f80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f91e000e940 tx=0x7f91e000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91e4c00640 1 -- 172.21.15.32:0/108726614 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0004d20 con 0x7f91e8114310 2024-08-31T07:23:51.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91e4c00640 1 -- 172.21.15.32:0/108726614 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e0005020 con 0x7f91e8114310 2024-08-31T07:23:51.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91ed800640 1 -- 172.21.15.32:0/108726614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e81a69a0 con 0x7f91e8114310 2024-08-31T07:23:51.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91e4c00640 1 -- 172.21.15.32:0/108726614 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e00116d0 con 0x7f91e8114310 2024-08-31T07:23:51.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.682+0000 7f91ed800640 1 -- 172.21.15.32:0/108726614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e81ad100 con 0x7f91e8114310 2024-08-31T07:23:51.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.684+0000 7f91ed800640 1 -- 172.21.15.32:0/108726614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e8077af0 con 0x7f91e8114310 2024-08-31T07:23:51.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.686+0000 7f91e4c00640 1 -- 172.21.15.32:0/108726614 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f91e0005460 con 0x7f91e8114310 2024-08-31T07:23:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.688+0000 7f91e4c00640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91cc0b4ca0 0x7f91cc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:51.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.688+0000 7f91e4c00640 1 -- 172.21.15.32:0/108726614 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7f91e00e5950 con 0x7f91e8114310 2024-08-31T07:23:51.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.688+0000 7f91e7e00640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91cc0b4ca0 0x7f91cc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:51.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.689+0000 7f91e7e00640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91cc0b4ca0 0x7f91cc0b71b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f91d8031440 tx=0x7f91d80058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:51.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.690+0000 7f91e4c00640 1 -- 172.21.15.32:0/108726614 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f91e0016030 con 0x7f91e8114310 2024-08-31T07:23:51.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.84 MB/sec execute 460 sec latency 20247.610 ms 2024-08-31T07:23:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.926+0000 7f91ed800640 1 -- 172.21.15.32:0/108726614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91e81a1870 con 0x7f91e8114310 2024-08-31T07:23:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.927+0000 7f91e4c00640 1 -- 172.21.15.32:0/108726614 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11961 (secure 0 0 0) 0x7f91e81a1870 con 0x7f91e8114310 2024-08-31T07:23:51.928 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:23:51.928 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"btime":"2024-08-31T07:23:33:679601+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:23:33.380019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:23:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-08-31T07:23:51.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.929+0000 7f91ed800640 1 -- 172.21.15.32:0/108726614 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91cc0b4ca0 msgr2=0x7f91cc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.929+0000 7f91ed800640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91cc0b4ca0 0x7f91cc0b71b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f91d8031440 tx=0x7f91d80058d0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.929+0000 7f91ed800640 1 -- 172.21.15.32:0/108726614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91e8114310 msgr2=0x7f91e81a5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.929+0000 7f91ed800640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91e8114310 0x7f91e81a5f80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f91e000e940 tx=0x7f91e000ee00 comp rx=0 tx=0).stop 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.929+0000 7f91ed800640 1 -- 172.21.15.32:0/108726614 shutdown_connections 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.929+0000 7f91ed800640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f91cc0b4ca0 0x7f91cc0b71b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.930+0000 7f91ed800640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91e8114310 0x7f91e81a5f80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.930+0000 7f91ed800640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8078030 0x7f91e81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.930+0000 7f91ed800640 1 --2- 172.21.15.32:0/108726614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e80776e0 0x7f91e81a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.930+0000 7f91ed800640 1 -- 172.21.15.32:0/108726614 >> 172.21.15.32:0/108726614 conn(0x7f91e8101420 msgr2=0x7f91e8116400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.930+0000 7f91ed800640 1 -- 172.21.15.32:0/108726614 shutdown_connections 2024-08-31T07:23:51.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:51.930+0000 7f91ed800640 1 -- 172.21.15.32:0/108726614 wait complete. 2024-08-31T07:23:52.708 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 500e9d42 --release --await' 2024-08-31T07:23:52.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 500e9d42 --release --await 2024-08-31T07:23:52.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.83 MB/sec execute 461 sec latency 21247.661 ms 2024-08-31T07:23:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:52 smithi084 ceph-mon[22387]: pgmap v711: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 760 KiB/s wr, 42 op/s 2024-08-31T07:23:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:52 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/108726614' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:53.084 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:53.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:52 smithi032 ceph-mon[17252]: pgmap v711: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 760 KiB/s wr, 42 op/s 2024-08-31T07:23:53.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:52 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/108726614' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:53.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:52 smithi073 ceph-mon[21995]: pgmap v711: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 760 KiB/s wr, 42 op/s 2024-08-31T07:23:53.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:52 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/108726614' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.738+0000 7fa6d1c00640 1 -- 172.21.15.32:0/1326001829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6cc1099c0 msgr2=0x7fa6cc1101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.738+0000 7fa6d1c00640 1 --2- 172.21.15.32:0/1326001829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6cc1099c0 0x7fa6cc1101f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009ca0 tx=0x7fa6c402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.738+0000 7fa6d1c00640 1 -- 172.21.15.32:0/1326001829 shutdown_connections 2024-08-31T07:23:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.738+0000 7fa6d1c00640 1 --2- 172.21.15.32:0/1326001829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6cc1099c0 0x7fa6cc1101f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.738+0000 7fa6d1c00640 1 --2- 172.21.15.32:0/1326001829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc109070 0x7fa6cc109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.738+0000 7fa6d1c00640 1 --2- 172.21.15.32:0/1326001829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc103730 0x7fa6cc103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.738+0000 7fa6d1c00640 1 -- 172.21.15.32:0/1326001829 >> 172.21.15.32:0/1326001829 conn(0x7fa6cc0ff180 msgr2=0x7fa6cc1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.738+0000 7fa6d1c00640 1 -- 172.21.15.32:0/1326001829 shutdown_connections 2024-08-31T07:23:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.738+0000 7fa6d1c00640 1 -- 172.21.15.32:0/1326001829 wait complete. 2024-08-31T07:23:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.739+0000 7fa6d1c00640 1 Processor -- start 2024-08-31T07:23:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.739+0000 7fa6d1c00640 1 -- start start 2024-08-31T07:23:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6d1c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc103730 0x7fa6cc19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6d1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc109070 0x7fa6cc19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6d1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6cc1099c0 0x7fa6cc1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6d1c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc113670 con 0x7fa6cc1099c0 2024-08-31T07:23:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6d1c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc1134f0 con 0x7fa6cc109070 2024-08-31T07:23:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6d1c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc1137f0 con 0x7fa6cc103730 2024-08-31T07:23:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6cbe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc103730 0x7fa6cc19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6cbe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc103730 0x7fa6cc19c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42016/0 (socket says 172.21.15.32:42016) 2024-08-31T07:23:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6cbe00640 1 -- 172.21.15.32:0/3552526033 learned_addr learned my addr 172.21.15.32:0/3552526033 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6c3400640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc109070 0x7fa6cc19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6d1200640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6cc1099c0 0x7fa6cc1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6cbe00640 1 -- 172.21.15.32:0/3552526033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc109070 msgr2=0x7fa6cc19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6cbe00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc109070 0x7fa6cc19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6cbe00640 1 -- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6cc1099c0 msgr2=0x7fa6cc1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6cbe00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6cc1099c0 0x7fa6cc1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6cbe00640 1 -- 172.21.15.32:0/3552526033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc1a2190 con 0x7fa6cc103730 2024-08-31T07:23:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.740+0000 7fa6d1200640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6cc1099c0 0x7fa6cc1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:23:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.741+0000 7fa6cbe00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc103730 0x7fa6cc19c220 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00e9f0 tx=0x7fa6bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.741+0000 7fa6c9600640 1 -- 172.21.15.32:0/3552526033 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00cd80 con 0x7fa6cc103730 2024-08-31T07:23:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.741+0000 7fa6d1c00640 1 -- 172.21.15.32:0/3552526033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6cc1a2480 con 0x7fa6cc103730 2024-08-31T07:23:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.741+0000 7fa6c9600640 1 -- 172.21.15.32:0/3552526033 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6bc0108e0 con 0x7fa6cc103730 2024-08-31T07:23:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.741+0000 7fa6c9600640 1 -- 172.21.15.32:0/3552526033 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc018980 con 0x7fa6cc103730 2024-08-31T07:23:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.741+0000 7fa6d1c00640 1 -- 172.21.15.32:0/3552526033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6cc1a8be0 con 0x7fa6cc103730 2024-08-31T07:23:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.744+0000 7fa6d9e1d740 1 Processor -- start 2024-08-31T07:23:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6d9e1d740 1 -- start start 2024-08-31T07:23:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6c9600640 1 -- 172.21.15.32:0/3552526033 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa6bc010430 con 0x7fa6cc103730 2024-08-31T07:23:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45cc520 0x559cd4972090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 0x559cd49725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45ab840 0x559cd4972b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559cd48e8a90 con 0x559cd45ab840 2024-08-31T07:23:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559cd48d8020 con 0x559cd45a3660 2024-08-31T07:23:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559cd48dbfd0 con 0x559cd45cc520 2024-08-31T07:23:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6d1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45cc520 0x559cd4972090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6d1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45cc520 0x559cd4972090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42032/0 (socket says 172.21.15.32:42032) 2024-08-31T07:23:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6d1200640 1 -- 172.21.15.32:0/2320663171 learned_addr learned my addr 172.21.15.32:0/2320663171 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.745+0000 7fa6c3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45ab840 0x559cd4972b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.746+0000 7fa6cbe00640 1 --2- 172.21.15.32:0/2320663171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 0x559cd49725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.746+0000 7fa6d1200640 1 -- 172.21.15.32:0/2320663171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 msgr2=0x559cd49725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.746+0000 7fa6d1200640 1 --2- 172.21.15.32:0/2320663171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 0x559cd49725d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.746+0000 7fa6d1200640 1 -- 172.21.15.32:0/2320663171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45ab840 msgr2=0x559cd4972b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.746+0000 7fa6d1200640 1 --2- 172.21.15.32:0/2320663171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45ab840 0x559cd4972b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.746+0000 7fa6d1200640 1 -- 172.21.15.32:0/2320663171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cd4637b00 con 0x559cd45cc520 2024-08-31T07:23:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.746+0000 7fa6d1200640 1 --2- 172.21.15.32:0/2320663171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45cc520 0x559cd4972090 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009470 tx=0x7fa6c40094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52259d6ffe712d3c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.747+0000 7fa6c9600640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6b00b4ca0 0x7fa6b00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.747+0000 7fa6c1600640 1 -- 172.21.15.32:0/2320663171 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4031cf0 con 0x559cd45cc520 2024-08-31T07:23:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.747+0000 7fa6c1600640 1 -- 172.21.15.32:0/2320663171 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c4036040 con 0x559cd45cc520 2024-08-31T07:23:53.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.747+0000 7fa6c1600640 1 -- 172.21.15.32:0/2320663171 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40042a0 con 0x559cd45cc520 2024-08-31T07:23:53.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.747+0000 7fa6c9600640 1 -- 172.21.15.32:0/3552526033 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fa6bc0e5910 con 0x7fa6cc103730 2024-08-31T07:23:53.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.747+0000 7fa6cbe00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6b00b4ca0 0x7fa6b00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.748+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/2320663171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45cc520 msgr2=0x559cd4972090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.748+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/2320663171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45cc520 0x559cd4972090 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009470 tx=0x7fa6c40094a0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.748+0000 7fa6cbe00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6b00b4ca0 0x7fa6b00b71b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x559cd4970d10 tx=0x7fa6bc0eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.748+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/2320663171 shutdown_connections 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.748+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/2320663171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45ab840 0x559cd4972b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.748+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/2320663171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 0x559cd49725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.748+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/2320663171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45cc520 0x559cd4972090 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.748+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/2320663171 >> 172.21.15.32:0/2320663171 conn(0x559cd45c0910 msgr2=0x559cd496ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.748+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/2320663171 shutdown_connections 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.748+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/2320663171 wait complete. 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.749+0000 7fa6d9e1d740 1 -- start start 2024-08-31T07:23:53.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.749+0000 7fa6d9e1d740 1 Processor -- start 2024-08-31T07:23:53.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.749+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45a3660 0x559cd497d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 0x559cd497d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45cc520 0x559cd497de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559cd473abb0 con 0x559cd45a3660 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559cd485ef90 con 0x559cd45cc520 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559cd4980810 con 0x559cd45ab840 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6d9e1d740 10 client.0 _ll_register_callbacks cb 0x559cd4884710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6d1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45a3660 0x559cd497d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6c3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 0x559cd497d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6c3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 0x559cd497d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42044/0 (socket says 172.21.15.32:42044) 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6d1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45a3660 0x559cd497d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45012/0 (socket says 172.21.15.32:45012) 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6c3400640 1 -- 172.21.15.32:0/3641107182 learned_addr learned my addr 172.21.15.32:0/3641107182 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:53.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6d1200640 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45cc520 0x559cd497de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6c3400640 1 -- 172.21.15.32:0/3641107182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45cc520 msgr2=0x559cd497de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6c3400640 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45cc520 0x559cd497de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6c3400640 1 -- 172.21.15.32:0/3641107182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45a3660 msgr2=0x559cd497d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6c3400640 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45a3660 0x559cd497d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6c3400640 1 -- 172.21.15.32:0/3641107182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cd46ae960 con 0x559cd45ab840 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.750+0000 7fa6d1200640 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45cc520 0x559cd497de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.751+0000 7fa6c3400640 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 0x559cd497d910 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40126e0 tx=0x7fa6b4012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a2e2610eef69b6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.751+0000 7fa6ba000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.751+0000 7fa6ba000640 1 -- 172.21.15.32:0/3641107182 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4012190 con 0x559cd45ab840 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.751+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/3641107182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559cd4876e30 con 0x559cd45ab840 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.751+0000 7fa6ba000640 1 -- 172.21.15.32:0/3641107182 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fa6b400f920 con 0x559cd45ab840 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.751+0000 7fa6ba000640 1 -- 172.21.15.32:0/3641107182 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa6b400fb20 con 0x559cd45ab840 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.751+0000 7fa6d9e1d740 10 client.27859 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.751+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/3641107182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559cd498c280 con 0x559cd45ab840 2024-08-31T07:23:53.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.752+0000 7fa6ba000640 1 -- 172.21.15.32:0/3641107182 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559cd4876e30 con 0x559cd45ab840 2024-08-31T07:23:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.752+0000 7fa6ba000640 1 -- 172.21.15.32:0/3641107182 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fa6b4010c90 con 0x559cd45ab840 2024-08-31T07:23:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.752+0000 7fa6d9e1d740 10 client.27859 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.752+0000 7fa6d9e1d740 10 client.27859 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:23:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.752+0000 7fa6d9e1d740 20 client.27859 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.752+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559cd458aea0 0x559cd498f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.752+0000 7fa6d9e1d740 4 client.27859 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:53.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.752+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/3641107182 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cd48ea9e0 con 0x559cd458aea0 2024-08-31T07:23:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.752+0000 7fa6d1200640 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559cd458aea0 0x559cd498f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.753+0000 7fa6d1200640 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559cd458aea0 0x559cd498f7a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f01eb5a59fd7fac7 server_cookie=e5d097fe2c16a74a in_seq=0 out_seq=0 2024-08-31T07:23:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.753+0000 7fa6ba000640 10 client.27859 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6ba000640 1 -- 172.21.15.32:0/3641107182 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559cd48ea9e0 con 0x559cd458aea0 2024-08-31T07:23:53.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6ba000640 10 client.27859 handle_command_reply: tid=0 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6d9e1d740 1 client.27859 shutdown 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/3641107182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 msgr2=0x559cd497d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 0x559cd497d910 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40126e0 tx=0x7fa6b4012ba0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/3641107182 shutdown_connections 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559cd458aea0 0x559cd498f7a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45cc520 0x559cd497de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 0x559cd497d910 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/3641107182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45a3660 0x559cd497d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.754+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/3641107182 >> 172.21.15.32:0/3641107182 conn(0x559cd45c0910 msgr2=0x559cd4975220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.755+0000 7fa6ba000640 0 client.27859 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.755+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/3641107182 shutdown_connections 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.755+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/3641107182 wait complete. 2024-08-31T07:23:53.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.755+0000 7fa6d9e1d740 20 client.27859 trim_cache size 0 max 16384 2024-08-31T07:23:53.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345762 2.82 MB/sec execute 462 sec latency 22247.732 ms 2024-08-31T07:23:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.773+0000 7fa6d9e1d740 1 Processor -- start 2024-08-31T07:23:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.773+0000 7fa6d9e1d740 1 -- start start 2024-08-31T07:23:53.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd4989a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 0x559cd496ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 0x559cd496f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc1b1480 con 0x559cd45c0910 2024-08-31T07:23:53.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc1b2ff0 con 0x559cd45a3660 2024-08-31T07:23:53.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc1b1b50 con 0x559cd45ab840 2024-08-31T07:23:53.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6d1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd4989a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6c3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 0x559cd496ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6d1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd4989a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45028/0 (socket says 172.21.15.32:45028) 2024-08-31T07:23:53.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6d1200640 1 -- 172.21.15.32:0/814622595 learned_addr learned my addr 172.21.15.32:0/814622595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:53.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6d1200640 1 --2- 172.21.15.32:0/814622595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 0x559cd496f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.774+0000 7fa6c3400640 1 -- 172.21.15.32:0/814622595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 msgr2=0x559cd496f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.775+0000 7fa6c3400640 1 --2- 172.21.15.32:0/814622595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 0x559cd496f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.775+0000 7fa6c3400640 1 -- 172.21.15.32:0/814622595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 msgr2=0x559cd4989a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.775+0000 7fa6c3400640 1 --2- 172.21.15.32:0/814622595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd4989a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.775+0000 7fa6c3400640 1 -- 172.21.15.32:0/814622595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cd498c090 con 0x559cd45a3660 2024-08-31T07:23:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.775+0000 7fa6d1200640 1 --2- 172.21.15.32:0/814622595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd4989a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.775+0000 7fa6c3400640 1 --2- 172.21.15.32:0/814622595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 0x559cd496ec30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4000c00 tx=0x7fa6b400fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9618587d58543248 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.775+0000 7fa6c0c00640 1 -- 172.21.15.32:0/814622595 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4029020 con 0x559cd45a3660 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.775+0000 7fa6c0c00640 1 -- 172.21.15.32:0/814622595 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b4012290 con 0x559cd45a3660 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.775+0000 7fa6c0c00640 1 -- 172.21.15.32:0/814622595 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b401cc80 con 0x559cd45a3660 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.776+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/814622595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 msgr2=0x559cd496ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.776+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/814622595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 0x559cd496ec30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4000c00 tx=0x7fa6b400fab0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.776+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/814622595 shutdown_connections 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.776+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/814622595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45ab840 0x559cd496f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.776+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/814622595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45a3660 0x559cd496ec30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.776+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/814622595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd4989a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.776+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/814622595 >> 172.21.15.32:0/814622595 conn(0x559cd4973d00 msgr2=0x559cd496e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.776+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/814622595 shutdown_connections 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.776+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/814622595 wait complete. 2024-08-31T07:23:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.777+0000 7fa6d9e1d740 1 -- start start 2024-08-31T07:23:53.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.777+0000 7fa6d9e1d740 1 Processor -- start 2024-08-31T07:23:53.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.777+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45a3660 0x559cd4982520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.777+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45ab840 0x559cd4982b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.777+0000 7fa6d9e1d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd49f3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.777+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc1b28f0 con 0x559cd45c0910 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.777+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc1b2220 con 0x559cd45ab840 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.777+0000 7fa6d9e1d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc1af480 con 0x559cd45a3660 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.777+0000 7fa6d9e1d740 10 client.0 _ll_register_callbacks cb 0x7fa6b4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6d1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd49f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6c3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45ab840 0x559cd4982b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6d1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd49f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45036/0 (socket says 172.21.15.32:45036) 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6c3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45ab840 0x559cd4982b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55980/0 (socket says 172.21.15.32:55980) 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6d1200640 1 -- 172.21.15.32:0/60664767 learned_addr learned my addr 172.21.15.32:0/60664767 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6d1200640 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45a3660 0x559cd4982520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6c3400640 1 -- 172.21.15.32:0/60664767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45a3660 msgr2=0x559cd4982520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6c3400640 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45a3660 0x559cd4982520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6c3400640 1 -- 172.21.15.32:0/60664767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 msgr2=0x559cd49f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6c3400640 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd49f3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6c3400640 1 -- 172.21.15.32:0/60664767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cd49f38a0 con 0x559cd45ab840 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.778+0000 7fa6d1200640 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd49f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.779+0000 7fa6c3400640 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45ab840 0x559cd4982b00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x559cd485f7e0 tx=0x7fa6b400fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e465bd02154579b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.779+0000 7fa6ba000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.779+0000 7fa6ba000640 1 -- 172.21.15.32:0/60664767 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4029040 con 0x559cd45ab840 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.779+0000 7fa6ba000640 1 -- 172.21.15.32:0/60664767 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fa6b4018070 con 0x559cd45ab840 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.779+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/60664767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559cd48e6480 con 0x559cd45ab840 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.779+0000 7fa6ba000640 1 -- 172.21.15.32:0/60664767 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa6b4005070 con 0x559cd45ab840 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.779+0000 7fa6d9e1d740 10 client.27854 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.779+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/60664767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559cd49facd0 con 0x559cd45ab840 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.779+0000 7fa6ba000640 1 -- 172.21.15.32:0/60664767 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559cd48e6480 con 0x559cd45ab840 2024-08-31T07:23:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.780+0000 7fa6ba000640 1 -- 172.21.15.32:0/60664767 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fa6b4003b20 con 0x559cd45ab840 2024-08-31T07:23:53.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.780+0000 7fa6d9e1d740 10 client.27854 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:53.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.780+0000 7fa6d9e1d740 10 client.27854 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:23:53.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.780+0000 7fa6d9e1d740 20 client.27854 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:53.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.780+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559cd45cc520 0x559cd49fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:53.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.780+0000 7fa6d9e1d740 4 client.27854 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "500e9d42", "release": true, "await": true}] 2024-08-31T07:23:53.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.780+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/60664767 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "500e9d42", "release": true, "await": true}) -- 0x559cd48e3bc0 con 0x559cd45cc520 2024-08-31T07:23:53.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.780+0000 7fa6d1200640 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559cd45cc520 0x559cd49fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:53.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.781+0000 7fa6d1200640 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559cd45cc520 0x559cd49fd2f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e534a602323a4a0 server_cookie=1c19ed89e40ca65e in_seq=0 out_seq=0 2024-08-31T07:23:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.781+0000 7fa6ba000640 10 client.27854 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.800+0000 7fa6ba000640 1 -- 172.21.15.32:0/60664767 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559cd48e3bc0 con 0x559cd45cc520 2024-08-31T07:23:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.800+0000 7fa6ba000640 10 client.27854 handle_command_reply: tid=0 2024-08-31T07:23:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.800+0000 7fa6d9e1d740 1 client.27854 shutdown 2024-08-31T07:23:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.800+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/60664767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45ab840 msgr2=0x559cd4982b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.800+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45ab840 0x559cd4982b00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x559cd485f7e0 tx=0x7fa6b400fef0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.801+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/60664767 shutdown_connections 2024-08-31T07:23:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.801+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559cd45cc520 0x559cd49fd2f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.801+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559cd45c0910 0x559cd49f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.801+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd45ab840 0x559cd4982b00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.801+0000 7fa6d9e1d740 1 --2- 172.21.15.32:0/60664767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559cd45a3660 0x559cd4982520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.801+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/60664767 >> 172.21.15.32:0/60664767 conn(0x559cd4973d00 msgr2=0x559cd496eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.801+0000 7fa6ba000640 0 client.27854 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.801+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/60664767 shutdown_connections 2024-08-31T07:23:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.801+0000 7fa6d9e1d740 1 -- 172.21.15.32:0/60664767 wait complete. 2024-08-31T07:23:53.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.802+0000 7fa6d9e1d740 20 client.27854 trim_cache size 0 max 16384 2024-08-31T07:23:53.803 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:23:53.803 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 44, 2024-08-31T07:23:53.803 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:23:53.803 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 36, 2024-08-31T07:23:53.803 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:23:53.803 INFO:teuthology.orchestra.run.smithi032.stdout: "500e9d42": { 2024-08-31T07:23:53.803 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 36, 2024-08-31T07:23:53.803 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 145.2, 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:53.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:53.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:53.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:53.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:53.805 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:23:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 -- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6b00b4ca0 msgr2=0x7fa6b00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6b00b4ca0 0x7fa6b00b71b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x559cd4970d10 tx=0x7fa6bc0eb530 comp rx=0 tx=0).stop 2024-08-31T07:23:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 -- 172.21.15.32:0/3552526033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc103730 msgr2=0x7fa6cc19c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc103730 0x7fa6cc19c220 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00e9f0 tx=0x7fa6bc00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 -- 172.21.15.32:0/3552526033 shutdown_connections 2024-08-31T07:23:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6b00b4ca0 0x7fa6b00b71b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6cc1099c0 0x7fa6cc1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc109070 0x7fa6cc19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 --2- 172.21.15.32:0/3552526033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc103730 0x7fa6cc19c220 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 -- 172.21.15.32:0/3552526033 >> 172.21.15.32:0/3552526033 conn(0x7fa6cc0ff180 msgr2=0x7fa6cc1076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 -- 172.21.15.32:0/3552526033 shutdown_connections 2024-08-31T07:23:53.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:53.803+0000 7fa6d1c00640 1 -- 172.21.15.32:0/3552526033 wait complete. 2024-08-31T07:23:54.603 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '500e9d42', total seconds elapsed: 29.3 2024-08-31T07:23:54.604 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-08-31T07:23:54.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:23:54.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 347688 2.85 MB/sec execute 463 sec latency 22287.400 ms 2024-08-31T07:23:54.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:54 smithi032 ceph-mon[17252]: pgmap v712: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 639 KiB/s wr, 41 op/s 2024-08-31T07:23:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:54 smithi084 ceph-mon[22387]: pgmap v712: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 639 KiB/s wr, 41 op/s 2024-08-31T07:23:55.032 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:55.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:54 smithi073 ceph-mon[21995]: pgmap v712: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 639 KiB/s wr, 41 op/s 2024-08-31T07:23:55.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.731+0000 7ff0e2e00640 1 -- 172.21.15.32:0/702168536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0dc104770 msgr2=0x7ff0dc104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.731+0000 7ff0e2e00640 1 --2- 172.21.15.32:0/702168536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0dc104770 0x7ff0dc104b80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009860 tx=0x7ff0cc02a340 comp rx=0 tx=0).stop 2024-08-31T07:23:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.731+0000 7ff0e2e00640 1 -- 172.21.15.32:0/702168536 shutdown_connections 2024-08-31T07:23:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.731+0000 7ff0e2e00640 1 --2- 172.21.15.32:0/702168536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0dc10dbd0 0x7ff0dc1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.731+0000 7ff0e2e00640 1 --2- 172.21.15.32:0/702168536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0dc1050c0 0x7ff0dc10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.731+0000 7ff0e2e00640 1 --2- 172.21.15.32:0/702168536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0dc104770 0x7ff0dc104b80 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.731+0000 7ff0e2e00640 1 -- 172.21.15.32:0/702168536 >> 172.21.15.32:0/702168536 conn(0x7ff0dc0794e0 msgr2=0x7ff0dc0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.731+0000 7ff0e2e00640 1 -- 172.21.15.32:0/702168536 shutdown_connections 2024-08-31T07:23:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.731+0000 7ff0e2e00640 1 -- 172.21.15.32:0/702168536 wait complete. 2024-08-31T07:23:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.733+0000 7ff0e2e00640 1 Processor -- start 2024-08-31T07:23:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.733+0000 7ff0e2e00640 1 -- start start 2024-08-31T07:23:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.733+0000 7ff0e2e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0dc104770 0x7ff0dc1024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.733+0000 7ff0e2e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0dc1050c0 0x7ff0dc1029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.733+0000 7ff0e2e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0dc10dbd0 0x7ff0dc102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.733+0000 7ff0e2e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc113830 con 0x7ff0dc104770 2024-08-31T07:23:55.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.733+0000 7ff0e2e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1136b0 con 0x7ff0dc1050c0 2024-08-31T07:23:55.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.733+0000 7ff0e2e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1139b0 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.734+0000 7ff0e0e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0dc10dbd0 0x7ff0dc102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.734+0000 7ff0d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0dc1050c0 0x7ff0dc1029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.734+0000 7ff0e0e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0dc10dbd0 0x7ff0dc102f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42086/0 (socket says 172.21.15.32:42086) 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.734+0000 7ff0e0e00640 1 -- 172.21.15.32:0/266050602 learned_addr learned my addr 172.21.15.32:0/266050602 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.734+0000 7ff0dbe00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0dc104770 0x7ff0dc1024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.734+0000 7ff0e0e00640 1 -- 172.21.15.32:0/266050602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0dc1050c0 msgr2=0x7ff0dc1029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.734+0000 7ff0e0e00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0dc1050c0 0x7ff0dc1029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.734+0000 7ff0e0e00640 1 -- 172.21.15.32:0/266050602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0dc104770 msgr2=0x7ff0dc1024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.734+0000 7ff0e0e00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0dc104770 0x7ff0dc1024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.734+0000 7ff0e0e00640 1 -- 172.21.15.32:0/266050602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc1a8580 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.735+0000 7ff0e0e00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0dc10dbd0 0x7ff0dc102f30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400efc0 tx=0x7ff0d400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.735+0000 7ff0d9600640 1 -- 172.21.15.32:0/266050602 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4003970 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.735+0000 7ff0d9600640 1 -- 172.21.15.32:0/266050602 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d4004020 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.735+0000 7ff0d9600640 1 -- 172.21.15.32:0/266050602 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400b940 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.735+0000 7ff0e2e00640 1 -- 172.21.15.32:0/266050602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc1a8870 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.735+0000 7ff0e2e00640 1 -- 172.21.15.32:0/266050602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc1a8d30 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.737+0000 7ff0e2e00640 1 -- 172.21.15.32:0/266050602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0dc06bfe0 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.739+0000 7ff0d9600640 1 -- 172.21.15.32:0/266050602 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff0d401d020 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.740+0000 7ff0d9600640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0c00b4d20 0x7ff0c00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.740+0000 7ff0d9600640 1 -- 172.21.15.32:0/266050602 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7ff0d40e5380 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.740+0000 7ff0dbe00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0c00b4d20 0x7ff0c00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:55.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.741+0000 7ff0dbe00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0c00b4d20 0x7ff0c00b7230 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0303f0 tx=0x7ff0cc030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:55.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:55.744+0000 7ff0d9600640 1 -- 172.21.15.32:0/266050602 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff0d40ac160 con 0x7ff0dc10dbd0 2024-08-31T07:23:55.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 348603 2.84 MB/sec execute 464 sec latency 18.507 ms 2024-08-31T07:23:56.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.010+0000 7ff0e2e00640 1 -- 172.21.15.32:0/266050602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0dc1037b0 con 0x7ff0dc10dbd0 2024-08-31T07:23:56.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.011+0000 7ff0d9600640 1 -- 172.21.15.32:0/266050602 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11961 (secure 0 0 0) 0x7ff0d40aba60 con 0x7ff0dc10dbd0 2024-08-31T07:23:56.013 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:23:56.013 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"btime":"2024-08-31T07:23:33:679601+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:23:33.380019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:23:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-08-31T07:23:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 -- 172.21.15.32:0/266050602 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0c00b4d20 msgr2=0x7ff0c00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0c00b4d20 0x7ff0c00b7230 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0303f0 tx=0x7ff0cc030340 comp rx=0 tx=0).stop 2024-08-31T07:23:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 -- 172.21.15.32:0/266050602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0dc10dbd0 msgr2=0x7ff0dc102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0dc10dbd0 0x7ff0dc102f30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400efc0 tx=0x7ff0d400c800 comp rx=0 tx=0).stop 2024-08-31T07:23:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 -- 172.21.15.32:0/266050602 shutdown_connections 2024-08-31T07:23:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0c00b4d20 0x7ff0c00b7230 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0dc10dbd0 0x7ff0dc102f30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0dc1050c0 0x7ff0dc1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:56.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 --2- 172.21.15.32:0/266050602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0dc104770 0x7ff0dc1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:56.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 -- 172.21.15.32:0/266050602 >> 172.21.15.32:0/266050602 conn(0x7ff0dc0794e0 msgr2=0x7ff0dc0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:56.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 -- 172.21.15.32:0/266050602 shutdown_connections 2024-08-31T07:23:56.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:56.014+0000 7ff0e2e00640 1 -- 172.21.15.32:0/266050602 wait complete. 2024-08-31T07:23:56.771 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-08-31T07:23:56.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-08-31T07:23:56.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349278 2.85 MB/sec execute 465 sec latency 202.911 ms 2024-08-31T07:23:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:56 smithi084 ceph-mon[22387]: pgmap v713: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 533 KiB/s wr, 37 op/s 2024-08-31T07:23:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:56 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/266050602' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:57.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:56 smithi032 ceph-mon[17252]: pgmap v713: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 533 KiB/s wr, 37 op/s 2024-08-31T07:23:57.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:56 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/266050602' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:57.153 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:23:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:56 smithi073 ceph-mon[21995]: pgmap v713: 97 pgs: 97 active+clean; 270 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 533 KiB/s wr, 37 op/s 2024-08-31T07:23:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:56 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/266050602' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:23:57.772 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349302 2.84 MB/sec execute 466 sec latency 655.397 ms 2024-08-31T07:23:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.860+0000 7fb23e200640 1 -- 172.21.15.32:0/4127127795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23810b290 msgr2=0x7fb23810b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.860+0000 7fb23e200640 1 --2- 172.21.15.32:0/4127127795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23810b290 0x7fb23810b6a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb224009900 tx=0x7fb22402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.860+0000 7fb23e200640 1 -- 172.21.15.32:0/4127127795 shutdown_connections 2024-08-31T07:23:57.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.860+0000 7fb23e200640 1 --2- 172.21.15.32:0/4127127795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb23810bbe0 0x7fb238112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.860+0000 7fb23e200640 1 --2- 172.21.15.32:0/4127127795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23810b290 0x7fb23810b6a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.860+0000 7fb23e200640 1 --2- 172.21.15.32:0/4127127795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb238105950 0x7fb238105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.860+0000 7fb23e200640 1 -- 172.21.15.32:0/4127127795 >> 172.21.15.32:0/4127127795 conn(0x7fb2381013e0 msgr2=0x7fb238103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.860+0000 7fb23e200640 1 -- 172.21.15.32:0/4127127795 shutdown_connections 2024-08-31T07:23:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.860+0000 7fb23e200640 1 -- 172.21.15.32:0/4127127795 wait complete. 2024-08-31T07:23:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.861+0000 7fb23e200640 1 Processor -- start 2024-08-31T07:23:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.861+0000 7fb23e200640 1 -- start start 2024-08-31T07:23:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.861+0000 7fb23e200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb238105950 0x7fb23819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb23810b290 0x7fb23819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23e200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23810bbe0 0x7fb2381a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23e200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381158a0 con 0x7fb23810bbe0 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb238115720 con 0x7fb23810b290 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23e200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb238115a20 con 0x7fb238105950 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb238105950 0x7fb23819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb237e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb23810b290 0x7fb23819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb238105950 0x7fb23819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41854/0 (socket says 172.21.15.32:41854) 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23ce00640 1 -- 172.21.15.32:0/4229240344 learned_addr learned my addr 172.21.15.32:0/4229240344 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb237e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb23810b290 0x7fb23819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:56714/0 (socket says 172.21.15.32:56714) 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23d800640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23810bbe0 0x7fb2381a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23ce00640 1 -- 172.21.15.32:0/4229240344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb23810b290 msgr2=0x7fb23819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23ce00640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb23810b290 0x7fb23819c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23ce00640 1 -- 172.21.15.32:0/4229240344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23810bbe0 msgr2=0x7fb2381a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23ce00640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23810bbe0 0x7fb2381a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.862+0000 7fb23ce00640 1 -- 172.21.15.32:0/4229240344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2381a2200 con 0x7fb238105950 2024-08-31T07:23:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.863+0000 7fb23ce00640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb238105950 0x7fb23819c290 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00e9f0 tx=0x7fb22c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.863+0000 7fb235600640 1 -- 172.21.15.32:0/4229240344 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00cd80 con 0x7fb238105950 2024-08-31T07:23:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.863+0000 7fb23e200640 1 -- 172.21.15.32:0/4229240344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2381a24f0 con 0x7fb238105950 2024-08-31T07:23:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.863+0000 7fb235600640 1 -- 172.21.15.32:0/4229240344 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c0108e0 con 0x7fb238105950 2024-08-31T07:23:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.863+0000 7fb235600640 1 -- 172.21.15.32:0/4229240344 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c018980 con 0x7fb238105950 2024-08-31T07:23:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.863+0000 7fb23e200640 1 -- 172.21.15.32:0/4229240344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2381a8c50 con 0x7fb238105950 2024-08-31T07:23:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.867+0000 7fb246462740 1 Processor -- start 2024-08-31T07:23:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.867+0000 7fb246462740 1 -- start start 2024-08-31T07:23:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.868+0000 7fb246462740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f9355dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.868+0000 7fb246462740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9356310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.868+0000 7fb246462740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 0x5563f9356850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.868+0000 7fb246462740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5563f92cb450 con 0x5563f8f8eb70 2024-08-31T07:23:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.868+0000 7fb246462740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5563f92ba9e0 con 0x5563f8f86990 2024-08-31T07:23:57.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.868+0000 7fb246462740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5563f92be990 con 0x5563f8faf850 2024-08-31T07:23:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.868+0000 7fb23d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f9355dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.868+0000 7fb237e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 0x5563f9356850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.868+0000 7fb23d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f9355dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41870/0 (socket says 172.21.15.32:41870) 2024-08-31T07:23:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.868+0000 7fb23d800640 1 -- 172.21.15.32:0/2697087177 learned_addr learned my addr 172.21.15.32:0/2697087177 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:57.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.869+0000 7fb23d800640 1 -- 172.21.15.32:0/2697087177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 msgr2=0x5563f9356310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:23:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.869+0000 7fb23d800640 1 --2- 172.21.15.32:0/2697087177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9356310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.869+0000 7fb23d800640 1 -- 172.21.15.32:0/2697087177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 msgr2=0x5563f9356850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.869+0000 7fb23d800640 1 --2- 172.21.15.32:0/2697087177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 0x5563f9356850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.869+0000 7fb23d800640 1 -- 172.21.15.32:0/2697087177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f924dfa0 con 0x5563f8faf850 2024-08-31T07:23:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.869+0000 7fb235600640 1 -- 172.21.15.32:0/4229240344 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb22c010430 con 0x7fb238105950 2024-08-31T07:23:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.869+0000 7fb23d800640 1 --2- 172.21.15.32:0/2697087177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f9355dd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb230009fd0 tx=0x7fb2300118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2906945839d096d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.869+0000 7fb22a000640 1 -- 172.21.15.32:0/2697087177 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230004d20 con 0x5563f8faf850 2024-08-31T07:23:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.870+0000 7fb22a000640 1 -- 172.21.15.32:0/2697087177 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb230004050 con 0x5563f8faf850 2024-08-31T07:23:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.870+0000 7fb22a000640 1 -- 172.21.15.32:0/2697087177 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230005280 con 0x5563f8faf850 2024-08-31T07:23:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.870+0000 7fb246462740 1 -- 172.21.15.32:0/2697087177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 msgr2=0x5563f9355dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.870+0000 7fb246462740 1 --2- 172.21.15.32:0/2697087177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f9355dd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb230009fd0 tx=0x7fb2300118d0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.870+0000 7fb246462740 1 -- 172.21.15.32:0/2697087177 shutdown_connections 2024-08-31T07:23:57.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.870+0000 7fb246462740 1 --2- 172.21.15.32:0/2697087177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 0x5563f9356850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.870+0000 7fb246462740 1 --2- 172.21.15.32:0/2697087177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9356310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.870+0000 7fb246462740 1 --2- 172.21.15.32:0/2697087177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f9355dd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.870+0000 7fb246462740 1 -- 172.21.15.32:0/2697087177 >> 172.21.15.32:0/2697087177 conn(0x5563f8fa3c40 msgr2=0x5563f9353cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.871+0000 7fb246462740 1 -- 172.21.15.32:0/2697087177 shutdown_connections 2024-08-31T07:23:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.871+0000 7fb246462740 1 -- 172.21.15.32:0/2697087177 wait complete. 2024-08-31T07:23:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.871+0000 7fb235600640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2200b4ba0 0x7fb2200b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.871+0000 7fb246462740 1 -- start start 2024-08-31T07:23:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.871+0000 7fb235600640 1 -- 172.21.15.32:0/4229240344 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fb22c0e5800 con 0x7fb238105950 2024-08-31T07:23:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.871+0000 7fb23d800640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2200b4ba0 0x7fb2200b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.871+0000 7fb246462740 1 Processor -- start 2024-08-31T07:23:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb246462740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9361630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb246462740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 0x5563f9361c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb246462740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f936d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb246462740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5563f911d7b0 con 0x5563f8f8eb70 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb246462740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5563f9255eb0 con 0x5563f8f86990 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb246462740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5563f93645b0 con 0x5563f8faf850 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb246462740 10 client.0 _ll_register_callbacks cb 0x5563f92b80a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb23d800640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2200b4ba0 0x7fb2200b70b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb2300049d0 tx=0x7fb2300048d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb237e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 0x5563f9361c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb23ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9361630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb237e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 0x5563f9361c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46144/0 (socket says 172.21.15.32:46144) 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb23ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9361630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:56726/0 (socket says 172.21.15.32:56726) 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb23d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f936d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.872+0000 7fb237e00640 1 -- 172.21.15.32:0/1737581399 learned_addr learned my addr 172.21.15.32:0/1737581399 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:57.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.873+0000 7fb23ce00640 1 -- 172.21.15.32:0/1737581399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 msgr2=0x5563f936d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.873+0000 7fb23ce00640 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f936d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.873+0000 7fb23ce00640 1 -- 172.21.15.32:0/1737581399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 msgr2=0x5563f9361c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.873+0000 7fb23ce00640 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 0x5563f9361c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.873+0000 7fb23ce00640 1 -- 172.21.15.32:0/1737581399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f9260f40 con 0x5563f8f86990 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.873+0000 7fb23ce00640 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9361630 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0e2fa0 tx=0x7fb22c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=119feec46f69e608 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.873+0000 7fb21ea00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.873+0000 7fb21ea00640 1 -- 172.21.15.32:0/1737581399 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c0b5260 con 0x5563f8f86990 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.873+0000 7fb21ea00640 1 -- 172.21.15.32:0/1737581399 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fb22c0b0400 con 0x5563f8f86990 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.873+0000 7fb246462740 1 -- 172.21.15.32:0/1737581399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb230004050 con 0x5563f8f86990 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb21ea00640 1 -- 172.21.15.32:0/1737581399 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb22c0971a0 con 0x5563f8f86990 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb246462740 10 client.27869 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb246462740 1 -- 172.21.15.32:0/1737581399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5563f9029ba0 con 0x5563f8f86990 2024-08-31T07:23:57.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb21ea00640 1 -- 172.21.15.32:0/1737581399 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb230004050 con 0x5563f8f86990 2024-08-31T07:23:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb21ea00640 1 -- 172.21.15.32:0/1737581399 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fb22c0a7960 con 0x5563f8f86990 2024-08-31T07:23:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb246462740 10 client.27869 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb246462740 10 client.27869 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:23:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb246462740 20 client.27869 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb246462740 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5563f8f6e1d0 0x5563f93736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb246462740 4 client.27869 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:23:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.874+0000 7fb246462740 1 -- 172.21.15.32:0/1737581399 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563f92cd3a0 con 0x5563f8f6e1d0 2024-08-31T07:23:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.875+0000 7fb237e00640 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5563f8f6e1d0 0x5563f93736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.875+0000 7fb237e00640 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5563f8f6e1d0 0x5563f93736a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c8b5c84b0a7e7b server_cookie=50d0ad902524351e in_seq=0 out_seq=0 2024-08-31T07:23:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.875+0000 7fb21ea00640 10 client.27869 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb21ea00640 1 -- 172.21.15.32:0/1737581399 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563f92cd3a0 con 0x5563f8f6e1d0 2024-08-31T07:23:57.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb21ea00640 10 client.27869 handle_command_reply: tid=0 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb246462740 1 client.27869 shutdown 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb246462740 1 -- 172.21.15.32:0/1737581399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 msgr2=0x5563f9361630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb246462740 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9361630 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0e2fa0 tx=0x7fb22c010860 comp rx=0 tx=0).stop 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb246462740 1 -- 172.21.15.32:0/1737581399 shutdown_connections 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb246462740 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5563f8f6e1d0 0x5563f93736a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb246462740 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f936d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb246462740 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f8eb70 0x5563f9361c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb21ea00640 0 client.27869 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb246462740 1 --2- 172.21.15.32:0/1737581399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9361630 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb246462740 1 -- 172.21.15.32:0/1737581399 >> 172.21.15.32:0/1737581399 conn(0x5563f8fa3c40 msgr2=0x5563f9358f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.876+0000 7fb246462740 1 -- 172.21.15.32:0/1737581399 shutdown_connections 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.877+0000 7fb246462740 1 -- 172.21.15.32:0/1737581399 wait complete. 2024-08-31T07:23:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.877+0000 7fb246462740 20 client.27869 trim_cache size 0 max 16384 2024-08-31T07:23:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.895+0000 7fb246462740 1 Processor -- start 2024-08-31T07:23:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.895+0000 7fb246462740 1 -- start start 2024-08-31T07:23:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.895+0000 7fb246462740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9359890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.895+0000 7fb246462740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8f8eb70 0x5563f93535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.895+0000 7fb246462740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8faf850 0x5563f936d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.895+0000 7fb246462740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381b1460 con 0x5563f8faf850 2024-08-31T07:23:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.895+0000 7fb246462740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381b2fd0 con 0x5563f8f86990 2024-08-31T07:23:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.895+0000 7fb246462740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381b1b30 con 0x5563f8f8eb70 2024-08-31T07:23:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb237e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8f8eb70 0x5563f93535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb23d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8faf850 0x5563f936d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb237e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8f8eb70 0x5563f93535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41890/0 (socket says 172.21.15.32:41890) 2024-08-31T07:23:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb237e00640 1 -- 172.21.15.32:0/3311965961 learned_addr learned my addr 172.21.15.32:0/3311965961 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb23ce00640 1 --2- 172.21.15.32:0/3311965961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9359890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb237e00640 1 -- 172.21.15.32:0/3311965961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 msgr2=0x5563f9359890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:57.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb237e00640 1 --2- 172.21.15.32:0/3311965961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9359890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb237e00640 1 -- 172.21.15.32:0/3311965961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8faf850 msgr2=0x5563f936d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:57.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb237e00640 1 --2- 172.21.15.32:0/3311965961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8faf850 0x5563f936d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb237e00640 1 -- 172.21.15.32:0/3311965961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f9358c30 con 0x5563f8f8eb70 2024-08-31T07:23:57.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.896+0000 7fb237e00640 1 --2- 172.21.15.32:0/3311965961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8f8eb70 0x5563f93535e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb224006010 tx=0x7fb224031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bcd0d6af72506b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:57.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.897+0000 7fb229600640 1 -- 172.21.15.32:0/3311965961 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224035040 con 0x5563f8f8eb70 2024-08-31T07:23:57.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.897+0000 7fb229600640 1 -- 172.21.15.32:0/3311965961 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2240054f0 con 0x5563f8f8eb70 2024-08-31T07:23:57.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.897+0000 7fb229600640 1 -- 172.21.15.32:0/3311965961 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224004dd0 con 0x5563f8f8eb70 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.897+0000 7fb246462740 1 -- 172.21.15.32:0/3311965961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8f8eb70 msgr2=0x5563f93535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.897+0000 7fb246462740 1 --2- 172.21.15.32:0/3311965961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8f8eb70 0x5563f93535e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb224006010 tx=0x7fb224031400 comp rx=0 tx=0).stop 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.898+0000 7fb246462740 1 -- 172.21.15.32:0/3311965961 shutdown_connections 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.898+0000 7fb246462740 1 --2- 172.21.15.32:0/3311965961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8faf850 0x5563f936d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.898+0000 7fb246462740 1 --2- 172.21.15.32:0/3311965961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8f8eb70 0x5563f93535e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.898+0000 7fb246462740 1 --2- 172.21.15.32:0/3311965961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f86990 0x5563f9359890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.898+0000 7fb246462740 1 -- 172.21.15.32:0/3311965961 >> 172.21.15.32:0/3311965961 conn(0x5563f8fa3c40 msgr2=0x5563f935fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.898+0000 7fb246462740 1 -- 172.21.15.32:0/3311965961 shutdown_connections 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.898+0000 7fb246462740 1 -- 172.21.15.32:0/3311965961 wait complete. 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.898+0000 7fb246462740 1 -- start start 2024-08-31T07:23:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.898+0000 7fb246462740 1 Processor -- start 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb246462740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f86990 0x5563f9366950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb246462740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f8eb70 0x5563f9366e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb246462740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f93673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb246462740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381b28d0 con 0x5563f8f86990 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb246462740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381b2200 con 0x5563f8f8eb70 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb246462740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381af460 con 0x5563f8faf850 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb246462740 10 client.0 _ll_register_callbacks cb 0x7fb22c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb23d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f93673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb23d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f93673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41904/0 (socket says 172.21.15.32:41904) 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb23d800640 1 -- 172.21.15.32:0/3803516957 learned_addr learned my addr 172.21.15.32:0/3803516957 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:23:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb237e00640 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f8eb70 0x5563f9366e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.899+0000 7fb23ce00640 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f86990 0x5563f9366950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb237e00640 1 -- 172.21.15.32:0/3803516957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 msgr2=0x5563f93673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb237e00640 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f93673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb237e00640 1 -- 172.21.15.32:0/3803516957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f86990 msgr2=0x5563f9366950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb237e00640 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f86990 0x5563f9366950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb237e00640 1 -- 172.21.15.32:0/3803516957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f93de1d0 con 0x5563f8f8eb70 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb23ce00640 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f86990 0x5563f9366950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb23d800640 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f93673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb237e00640 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f8eb70 0x5563f9366e90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb22402a910 tx=0x7fb22402aaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29bf302ee6eca8b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb21ea00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb21ea00640 1 -- 172.21.15.32:0/3803516957 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224035040 con 0x5563f8f8eb70 2024-08-31T07:23:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb21ea00640 1 -- 172.21.15.32:0/3803516957 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8441+0+0 (secure 0 0 0) 0x7fb2240037f0 con 0x5563f8f8eb70 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.900+0000 7fb246462740 1 -- 172.21.15.32:0/3803516957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2240054f0 con 0x5563f8f8eb70 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb21ea00640 1 -- 172.21.15.32:0/3803516957 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb224003a10 con 0x5563f8f8eb70 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb246462740 10 client.27875 fetch_fsmap learned FSMap version 44 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb246462740 1 -- 172.21.15.32:0/3803516957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5563f93df430 con 0x5563f8f8eb70 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb21ea00640 1 -- 172.21.15.32:0/3803516957 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2240054f0 con 0x5563f8f8eb70 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb21ea00640 1 -- 172.21.15.32:0/3803516957 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fb2240051a0 con 0x5563f8f8eb70 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb246462740 10 client.27875 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb246462740 10 client.27875 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb246462740 20 client.27875 populate_metadata read hostname 'smithi032' 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb246462740 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5563f8f43580 0x5563f93e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb246462740 4 client.27875 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.901+0000 7fb246462740 1 -- 172.21.15.32:0/3803516957 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x5563f92c8e40 con 0x5563f8f43580 2024-08-31T07:23:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.902+0000 7fb23ce00640 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5563f8f43580 0x5563f93e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:23:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.902+0000 7fb23ce00640 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5563f8f43580 0x5563f93e1b50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed6b12d7eb9bc01b server_cookie=85efb2655f95951f in_seq=0 out_seq=0 2024-08-31T07:23:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.902+0000 7fb21ea00640 10 client.27875 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb21ea00640 1 -- 172.21.15.32:0/3803516957 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5563f92c8e40 con 0x5563f8f43580 2024-08-31T07:23:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb21ea00640 10 client.27875 handle_command_reply: tid=0 2024-08-31T07:23:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 client.27875 shutdown 2024-08-31T07:23:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 -- 172.21.15.32:0/3803516957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f8eb70 msgr2=0x5563f9366e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f8eb70 0x5563f9366e90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb22402a910 tx=0x7fb22402aaf0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 -- 172.21.15.32:0/3803516957 shutdown_connections 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5563f8f43580 0x5563f93e1b50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5563f8faf850 0x5563f93673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563f8f8eb70 0x5563f9366e90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 --2- 172.21.15.32:0/3803516957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5563f8f86990 0x5563f9366950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb21ea00640 0 client.27875 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 -- 172.21.15.32:0/3803516957 >> 172.21.15.32:0/3803516957 conn(0x5563f8fa3c40 msgr2=0x5563f9358ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 -- 172.21.15.32:0/3803516957 shutdown_connections 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.962+0000 7fb246462740 1 -- 172.21.15.32:0/3803516957 wait complete. 2024-08-31T07:23:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.963+0000 7fb246462740 20 client.27875 trim_cache size 0 max 16384 2024-08-31T07:23:57.965 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:23:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 44, 2024-08-31T07:23:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:23:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 38, 2024-08-31T07:23:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:23:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "8e2b4950": { 2024-08-31T07:23:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 38, 2024-08-31T07:23:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:23:57.965 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 145.2, 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:23:57.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:57.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:23:57.967 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:23:57.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.963+0000 7fb23e200640 1 -- 172.21.15.32:0/4229240344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2200b4ba0 msgr2=0x7fb2200b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.963+0000 7fb23e200640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2200b4ba0 0x7fb2200b70b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb2300049d0 tx=0x7fb2300048d0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.963+0000 7fb23e200640 1 -- 172.21.15.32:0/4229240344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb238105950 msgr2=0x7fb23819c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.963+0000 7fb23e200640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb238105950 0x7fb23819c290 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00e9f0 tx=0x7fb22c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.964+0000 7fb23e200640 1 -- 172.21.15.32:0/4229240344 shutdown_connections 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.964+0000 7fb23e200640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2200b4ba0 0x7fb2200b70b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.964+0000 7fb23e200640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb23810bbe0 0x7fb2381a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.964+0000 7fb23e200640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb23810b290 0x7fb23819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.964+0000 7fb23e200640 1 --2- 172.21.15.32:0/4229240344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb238105950 0x7fb23819c290 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.964+0000 7fb23e200640 1 -- 172.21.15.32:0/4229240344 >> 172.21.15.32:0/4229240344 conn(0x7fb2381013e0 msgr2=0x7fb238103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.964+0000 7fb23e200640 1 -- 172.21.15.32:0/4229240344 shutdown_connections 2024-08-31T07:23:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:23:57.964+0000 7fb23e200640 1 -- 172.21.15.32:0/4229240344 wait complete. 2024-08-31T07:23:58.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.84 MB/sec execute 467 sec latency 827.126 ms 2024-08-31T07:23:58.875 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8e2b4950', quiesce took 4.3 seconds. Will release after: 20.95 2024-08-31T07:23:58.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:23:58 smithi073 ceph-mon[21995]: pgmap v714: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 638 KiB/s wr, 63 op/s 2024-08-31T07:23:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:23:58 smithi084 ceph-mon[22387]: pgmap v714: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 638 KiB/s wr, 63 op/s 2024-08-31T07:23:59.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:23:58 smithi032 ceph-mon[17252]: pgmap v714: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 638 KiB/s wr, 63 op/s 2024-08-31T07:23:59.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.83 MB/sec execute 468 sec latency 1827.175 ms 2024-08-31T07:24:00.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.82 MB/sec execute 469 sec latency 2827.296 ms 2024-08-31T07:24:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:00 smithi084 ceph-mon[22387]: pgmap v715: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 463 KiB/s wr, 64 op/s 2024-08-31T07:24:01.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:00 smithi032 ceph-mon[17252]: pgmap v715: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 463 KiB/s wr, 64 op/s 2024-08-31T07:24:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:00 smithi073 ceph-mon[21995]: pgmap v715: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 463 KiB/s wr, 64 op/s 2024-08-31T07:24:01.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.82 MB/sec execute 470 sec latency 3827.352 ms 2024-08-31T07:24:01.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:24:01.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:24:01.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:24:01.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:24:01.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:01 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:24:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:24:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:24:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:24:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:24:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:01 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:24:02.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:24:02.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:24:02.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:24:02.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:24:02.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:01 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:24:02.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.81 MB/sec execute 471 sec latency 4827.461 ms 2024-08-31T07:24:03.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:02 smithi032 ceph-mon[17252]: pgmap v716: 97 pgs: 97 active+clean; 274 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 903 KiB/s wr, 98 op/s 2024-08-31T07:24:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:02 smithi073 ceph-mon[21995]: pgmap v716: 97 pgs: 97 active+clean; 274 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 903 KiB/s wr, 98 op/s 2024-08-31T07:24:03.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:02 smithi084 ceph-mon[22387]: pgmap v716: 97 pgs: 97 active+clean; 274 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 903 KiB/s wr, 98 op/s 2024-08-31T07:24:03.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.81 MB/sec execute 472 sec latency 5827.509 ms 2024-08-31T07:24:04.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.80 MB/sec execute 473 sec latency 6827.623 ms 2024-08-31T07:24:05.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:04 smithi032 ceph-mon[17252]: pgmap v717: 97 pgs: 97 active+clean; 275 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-08-31T07:24:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:04 smithi073 ceph-mon[21995]: pgmap v717: 97 pgs: 97 active+clean; 275 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-08-31T07:24:05.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:04 smithi084 ceph-mon[22387]: pgmap v717: 97 pgs: 97 active+clean; 275 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-08-31T07:24:05.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.79 MB/sec execute 474 sec latency 7827.759 ms 2024-08-31T07:24:06.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.79 MB/sec execute 475 sec latency 8827.846 ms 2024-08-31T07:24:07.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:06 smithi032 ceph-mon[17252]: pgmap v718: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-08-31T07:24:07.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:06 smithi073 ceph-mon[21995]: pgmap v718: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-08-31T07:24:07.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:06 smithi084 ceph-mon[22387]: pgmap v718: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-08-31T07:24:07.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.78 MB/sec execute 476 sec latency 9827.911 ms 2024-08-31T07:24:08.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:07 smithi032 ceph-mon[17252]: osdmap e91: 12 total, 12 up, 12 in 2024-08-31T07:24:08.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:07 smithi073 ceph-mon[21995]: osdmap e91: 12 total, 12 up, 12 in 2024-08-31T07:24:08.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:07 smithi084 ceph-mon[22387]: osdmap e91: 12 total, 12 up, 12 in 2024-08-31T07:24:08.773 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.78 MB/sec execute 477 sec latency 10827.987 ms 2024-08-31T07:24:09.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:08 smithi032 ceph-mon[17252]: pgmap v720: 97 pgs: 97 active+clean; 275 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-08-31T07:24:09.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:08 smithi073 ceph-mon[21995]: pgmap v720: 97 pgs: 97 active+clean; 275 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-08-31T07:24:09.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:08 smithi084 ceph-mon[22387]: pgmap v720: 97 pgs: 97 active+clean; 275 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-08-31T07:24:09.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.77 MB/sec execute 478 sec latency 11828.128 ms 2024-08-31T07:24:10.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.76 MB/sec execute 479 sec latency 12828.236 ms 2024-08-31T07:24:11.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:10 smithi032 ceph-mon[17252]: pgmap v721: 97 pgs: 97 active+clean; 274 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.0 MiB/s wr, 83 op/s 2024-08-31T07:24:11.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:10 smithi073 ceph-mon[21995]: pgmap v721: 97 pgs: 97 active+clean; 274 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.0 MiB/s wr, 83 op/s 2024-08-31T07:24:11.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:10 smithi084 ceph-mon[22387]: pgmap v721: 97 pgs: 97 active+clean; 274 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.0 MiB/s wr, 83 op/s 2024-08-31T07:24:11.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.76 MB/sec execute 480 sec latency 13828.289 ms 2024-08-31T07:24:12.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.75 MB/sec execute 481 sec latency 14828.372 ms 2024-08-31T07:24:13.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:12 smithi032 ceph-mon[17252]: pgmap v722: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 498 KiB/s wr, 41 op/s 2024-08-31T07:24:13.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:12 smithi073 ceph-mon[21995]: pgmap v722: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 498 KiB/s wr, 41 op/s 2024-08-31T07:24:13.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:12 smithi084 ceph-mon[22387]: pgmap v722: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 498 KiB/s wr, 41 op/s 2024-08-31T07:24:13.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:24:13.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.75 MB/sec execute 482 sec latency 15828.422 ms 2024-08-31T07:24:13.989 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.767+0000 7fb170600640 1 -- 172.21.15.32:0/3711524667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb16810b460 msgr2=0x7fb16810b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:14.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.767+0000 7fb170600640 1 --2- 172.21.15.32:0/3711524667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb16810b460 0x7fb16810b870 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb154009900 tx=0x7fb15402a410 comp rx=0 tx=0).stop 2024-08-31T07:24:14.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.767+0000 7fb170600640 1 -- 172.21.15.32:0/3711524667 shutdown_connections 2024-08-31T07:24:14.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.767+0000 7fb170600640 1 --2- 172.21.15.32:0/3711524667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb16810bdb0 0x7fb1681125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.767+0000 7fb170600640 1 --2- 172.21.15.32:0/3711524667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb16810b460 0x7fb16810b870 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.767+0000 7fb170600640 1 --2- 172.21.15.32:0/3711524667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb168105b20 0x7fb168105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.767+0000 7fb170600640 1 -- 172.21.15.32:0/3711524667 >> 172.21.15.32:0/3711524667 conn(0x7fb1681015b0 msgr2=0x7fb168103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.767+0000 7fb170600640 1 -- 172.21.15.32:0/3711524667 shutdown_connections 2024-08-31T07:24:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.767+0000 7fb170600640 1 -- 172.21.15.32:0/3711524667 wait complete. 2024-08-31T07:24:14.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.768+0000 7fb170600640 1 Processor -- start 2024-08-31T07:24:14.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb170600640 1 -- start start 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb170600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168105b20 0x7fb16819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb170600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb16810b460 0x7fb16819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb170600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb16810bdb0 0x7fb1681a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb170600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb168115a70 con 0x7fb16810bdb0 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb170600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681158f0 con 0x7fb168105b20 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb170600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb168115bf0 con 0x7fb16810b460 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168105b20 0x7fb16819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb16e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb16810bdb0 0x7fb1681a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168105b20 0x7fb16819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35468/0 (socket says 172.21.15.32:35468) 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb16dc00640 1 -- 172.21.15.32:0/3385577640 learned_addr learned my addr 172.21.15.32:0/3385577640 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:14.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.769+0000 7fb16d200640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb16810b460 0x7fb16819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.770+0000 7fb16e600640 1 -- 172.21.15.32:0/3385577640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb16810b460 msgr2=0x7fb16819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.770+0000 7fb16e600640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb16810b460 0x7fb16819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.770+0000 7fb16e600640 1 -- 172.21.15.32:0/3385577640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168105b20 msgr2=0x7fb16819c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.770+0000 7fb16e600640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168105b20 0x7fb16819c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.770+0000 7fb16e600640 1 -- 172.21.15.32:0/3385577640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1681a23d0 con 0x7fb16810bdb0 2024-08-31T07:24:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.770+0000 7fb16dc00640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168105b20 0x7fb16819c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.770+0000 7fb16e600640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb16810bdb0 0x7fb1681a1ca0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb16000da10 tx=0x7fb16000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.770+0000 7fb166000640 1 -- 172.21.15.32:0/3385577640 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16000bd70 con 0x7fb16810bdb0 2024-08-31T07:24:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.771+0000 7fb166000640 1 -- 172.21.15.32:0/3385577640 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb160003e20 con 0x7fb16810bdb0 2024-08-31T07:24:14.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.771+0000 7fb170600640 1 -- 172.21.15.32:0/3385577640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1681a26c0 con 0x7fb16810bdb0 2024-08-31T07:24:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.771+0000 7fb166000640 1 -- 172.21.15.32:0/3385577640 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1600106c0 con 0x7fb16810bdb0 2024-08-31T07:24:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.771+0000 7fb170600640 1 -- 172.21.15.32:0/3385577640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1681a8e20 con 0x7fb16810bdb0 2024-08-31T07:24:14.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.74 MB/sec execute 483 sec latency 16828.473 ms 2024-08-31T07:24:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.774+0000 7fb17726f740 1 Processor -- start 2024-08-31T07:24:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.775+0000 7fb166000640 1 -- 172.21.15.32:0/3385577640 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb160004270 con 0x7fb16810bdb0 2024-08-31T07:24:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.775+0000 7fb17726f740 1 -- start start 2024-08-31T07:24:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.775+0000 7fb17726f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bfba070 0x56553c35f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.775+0000 7fb17726f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf911b0 0x56553c35fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.775+0000 7fb17726f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf99390 0x56553c3602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.775+0000 7fb17726f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56553c2d6770 con 0x56553bf99390 2024-08-31T07:24:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.775+0000 7fb17726f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56553c2c5d00 con 0x56553bf911b0 2024-08-31T07:24:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.775+0000 7fb17726f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56553c2c9cb0 con 0x56553bfba070 2024-08-31T07:24:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf911b0 0x56553c35fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf911b0 0x56553c35fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35470/0 (socket says 172.21.15.32:35470) 2024-08-31T07:24:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16dc00640 1 -- 172.21.15.32:0/695034692 learned_addr learned my addr 172.21.15.32:0/695034692 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16d200640 1 --2- 172.21.15.32:0/695034692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bfba070 0x56553c35f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16d200640 1 --2- 172.21.15.32:0/695034692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf99390 0x56553c3602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16dc00640 1 -- 172.21.15.32:0/695034692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bfba070 msgr2=0x56553c35f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16dc00640 1 --2- 172.21.15.32:0/695034692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bfba070 0x56553c35f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16dc00640 1 -- 172.21.15.32:0/695034692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf99390 msgr2=0x56553c3602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16dc00640 1 --2- 172.21.15.32:0/695034692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf99390 0x56553c3602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16dc00640 1 -- 172.21.15.32:0/695034692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56553c025650 con 0x56553bf911b0 2024-08-31T07:24:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.776+0000 7fb16dc00640 1 --2- 172.21.15.32:0/695034692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf911b0 0x56553c35fdb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb15c011630 tx=0x7fb15c011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d624ec74cbafa4f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.777+0000 7fb166000640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1500b4ca0 0x7fb1500b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.777+0000 7fb166000640 1 -- 172.21.15.32:0/3385577640 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8485+0+0 (secure 0 0 0) 0x7fb1600e58f0 con 0x7fb16810bdb0 2024-08-31T07:24:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.777+0000 7fb16d200640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1500b4ca0 0x7fb1500b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.778+0000 7fb16d200640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1500b4ca0 0x7fb1500b71b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb154009230 tx=0x7fb154031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.778+0000 7fb15aa00640 1 -- 172.21.15.32:0/695034692 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c01e070 con 0x56553bf911b0 2024-08-31T07:24:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.778+0000 7fb15aa00640 1 -- 172.21.15.32:0/695034692 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb15c00fba0 con 0x56553bf911b0 2024-08-31T07:24:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.778+0000 7fb15aa00640 1 -- 172.21.15.32:0/695034692 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c019ca0 con 0x56553bf911b0 2024-08-31T07:24:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.779+0000 7fb17726f740 1 -- 172.21.15.32:0/695034692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf911b0 msgr2=0x56553c35fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.779+0000 7fb17726f740 1 --2- 172.21.15.32:0/695034692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf911b0 0x56553c35fdb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb15c011630 tx=0x7fb15c011af0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.779+0000 7fb17726f740 1 -- 172.21.15.32:0/695034692 shutdown_connections 2024-08-31T07:24:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.779+0000 7fb17726f740 1 --2- 172.21.15.32:0/695034692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf99390 0x56553c3602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.779+0000 7fb17726f740 1 --2- 172.21.15.32:0/695034692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf911b0 0x56553c35fdb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.779+0000 7fb17726f740 1 --2- 172.21.15.32:0/695034692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bfba070 0x56553c35f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.779+0000 7fb17726f740 1 -- 172.21.15.32:0/695034692 >> 172.21.15.32:0/695034692 conn(0x56553bfae460 msgr2=0x56553c35d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.779+0000 7fb17726f740 1 -- 172.21.15.32:0/695034692 shutdown_connections 2024-08-31T07:24:14.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.779+0000 7fb17726f740 1 -- 172.21.15.32:0/695034692 wait complete. 2024-08-31T07:24:14.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.780+0000 7fb17726f740 1 -- start start 2024-08-31T07:24:14.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.780+0000 7fb17726f740 1 Processor -- start 2024-08-31T07:24:14.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb17726f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf911b0 0x56553c36b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb17726f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c36b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb17726f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bfba070 0x56553c377370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb17726f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56553c128700 con 0x56553bfba070 2024-08-31T07:24:14.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb17726f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56553c24ccb0 con 0x56553bf99390 2024-08-31T07:24:14.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb17726f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56553c36e050 con 0x56553bf911b0 2024-08-31T07:24:14.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb17726f740 10 client.0 _ll_register_callbacks cb 0x56553c272430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:14.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb16d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c36b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf911b0 0x56553c36b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb16e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bfba070 0x56553c377370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb16d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c36b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35486/0 (socket says 172.21.15.32:35486) 2024-08-31T07:24:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf911b0 0x56553c36b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:57074/0 (socket says 172.21.15.32:57074) 2024-08-31T07:24:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb16d200640 1 -- 172.21.15.32:0/2305811475 learned_addr learned my addr 172.21.15.32:0/2305811475 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.781+0000 7fb16d200640 1 -- 172.21.15.32:0/2305811475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf911b0 msgr2=0x56553c36b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.782+0000 7fb16d200640 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf911b0 0x56553c36b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.782+0000 7fb16d200640 1 -- 172.21.15.32:0/2305811475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bfba070 msgr2=0x56553c377370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.782+0000 7fb16d200640 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bfba070 0x56553c377370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.782+0000 7fb16d200640 1 -- 172.21.15.32:0/2305811475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56553c28d870 con 0x56553bf99390 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.782+0000 7fb16dc00640 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf911b0 0x56553c36b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.782+0000 7fb16d200640 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c36b970 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb154009900 tx=0x7fb154000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de61f37567e8f510 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.782+0000 7fb14f400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.782+0000 7fb14f400640 1 -- 172.21.15.32:0/2305811475 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1540035f0 con 0x56553bf99390 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.782+0000 7fb17726f740 1 -- 172.21.15.32:0/2305811475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb15c00fba0 con 0x56553bf99390 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.782+0000 7fb14f400640 1 -- 172.21.15.32:0/2305811475 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8485+0+0 (secure 0 0 0) 0x7fb154003790 con 0x56553bf99390 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb14f400640 1 -- 172.21.15.32:0/2305811475 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb154003990 con 0x56553bf99390 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb17726f740 10 client.27887 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb17726f740 1 -- 172.21.15.32:0/2305811475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56553c37a0a0 con 0x56553bf99390 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb14f400640 1 -- 172.21.15.32:0/2305811475 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb15c00fba0 con 0x56553bf99390 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb14f400640 1 -- 172.21.15.32:0/2305811475 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fb154044030 con 0x56553bf99390 2024-08-31T07:24:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb17726f740 10 client.27887 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb17726f740 10 client.27887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:24:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb17726f740 20 client.27887 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb17726f740 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56553bf789f0 0x56553c37d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb17726f740 4 client.27887 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:24:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.783+0000 7fb17726f740 1 -- 172.21.15.32:0/2305811475 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56553c2d86c0 con 0x56553bf789f0 2024-08-31T07:24:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.784+0000 7fb16dc00640 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56553bf789f0 0x56553c37d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.784+0000 7fb16dc00640 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56553bf789f0 0x56553c37d6c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff82ecb1dafb9b6 server_cookie=b1f3e7ec4031d397 in_seq=0 out_seq=0 2024-08-31T07:24:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.784+0000 7fb14f400640 10 client.27887 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:14.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.785+0000 7fb14f400640 1 -- 172.21.15.32:0/2305811475 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56553c2d86c0 con 0x56553bf789f0 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.785+0000 7fb14f400640 10 client.27887 handle_command_reply: tid=0 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.785+0000 7fb17726f740 1 client.27887 shutdown 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 1 -- 172.21.15.32:0/2305811475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 msgr2=0x56553c36b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c36b970 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb154009900 tx=0x7fb154000bc0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 1 -- 172.21.15.32:0/2305811475 shutdown_connections 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56553bf789f0 0x56553c37d6c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bfba070 0x56553c377370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c36b970 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 1 --2- 172.21.15.32:0/2305811475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf911b0 0x56553c36b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb14f400640 0 client.27887 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 1 -- 172.21.15.32:0/2305811475 >> 172.21.15.32:0/2305811475 conn(0x56553bfae460 msgr2=0x56553c362a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:14.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 1 -- 172.21.15.32:0/2305811475 shutdown_connections 2024-08-31T07:24:14.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 1 -- 172.21.15.32:0/2305811475 wait complete. 2024-08-31T07:24:14.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.786+0000 7fb17726f740 20 client.27887 trim_cache size 0 max 16384 2024-08-31T07:24:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.804+0000 7fb17726f740 1 Processor -- start 2024-08-31T07:24:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.804+0000 7fb17726f740 1 -- start start 2024-08-31T07:24:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb17726f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 0x56553c36ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb17726f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c377270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb17726f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bfba070 0x56553c3777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb17726f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681b15b0 con 0x56553bf911b0 2024-08-31T07:24:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb17726f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681b3120 con 0x56553bf99390 2024-08-31T07:24:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb17726f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681b1c80 con 0x56553bfba070 2024-08-31T07:24:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb16d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c377270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 0x56553c36ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb16d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c377270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35500/0 (socket says 172.21.15.32:35500) 2024-08-31T07:24:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb16d200640 1 -- 172.21.15.32:0/2623288377 learned_addr learned my addr 172.21.15.32:0/2623288377 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb16e600640 1 --2- 172.21.15.32:0/2623288377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bfba070 0x56553c3777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb16d200640 1 -- 172.21.15.32:0/2623288377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bfba070 msgr2=0x56553c3777b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb16d200640 1 --2- 172.21.15.32:0/2623288377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bfba070 0x56553c3777b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb16d200640 1 -- 172.21.15.32:0/2623288377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 msgr2=0x56553c36ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb16d200640 1 --2- 172.21.15.32:0/2623288377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 0x56553c36ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.805+0000 7fb16d200640 1 -- 172.21.15.32:0/2623288377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56553c3626d0 con 0x56553bf99390 2024-08-31T07:24:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.806+0000 7fb16d200640 1 --2- 172.21.15.32:0/2623288377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c377270 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb1540059d0 tx=0x7fb154031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=716cdea57f68c8ee server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.806+0000 7fb15a000640 1 -- 172.21.15.32:0/2623288377 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15405e020 con 0x56553bf99390 2024-08-31T07:24:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.806+0000 7fb15a000640 1 -- 172.21.15.32:0/2623288377 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb15404c070 con 0x56553bf99390 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.806+0000 7fb15a000640 1 -- 172.21.15.32:0/2623288377 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15403c600 con 0x56553bf99390 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.807+0000 7fb17726f740 1 -- 172.21.15.32:0/2623288377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 msgr2=0x56553c377270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.807+0000 7fb17726f740 1 --2- 172.21.15.32:0/2623288377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c377270 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb1540059d0 tx=0x7fb154031b10 comp rx=0 tx=0).stop 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.807+0000 7fb17726f740 1 -- 172.21.15.32:0/2623288377 shutdown_connections 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.807+0000 7fb17726f740 1 --2- 172.21.15.32:0/2623288377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bfba070 0x56553c3777b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.807+0000 7fb17726f740 1 --2- 172.21.15.32:0/2623288377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bf99390 0x56553c377270 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.807+0000 7fb17726f740 1 --2- 172.21.15.32:0/2623288377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 0x56553c36ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.807+0000 7fb17726f740 1 -- 172.21.15.32:0/2623288377 >> 172.21.15.32:0/2623288377 conn(0x56553bfae460 msgr2=0x56553c36c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.807+0000 7fb17726f740 1 -- 172.21.15.32:0/2623288377 shutdown_connections 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.807+0000 7fb17726f740 1 -- 172.21.15.32:0/2623288377 wait complete. 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.807+0000 7fb17726f740 1 -- start start 2024-08-31T07:24:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb17726f740 1 Processor -- start 2024-08-31T07:24:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb17726f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 0x56553c3732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb17726f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf99390 0x56553c373820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb17726f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bfba070 0x56553c36e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb17726f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681b2a20 con 0x56553bf911b0 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb17726f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681b2350 con 0x56553bfba070 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb17726f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681af5b0 con 0x56553bf99390 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb17726f740 10 client.0 _ll_register_callbacks cb 0x7fb1540314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 0x56553c3732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb16dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 0x56553c3732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54366/0 (socket says 172.21.15.32:54366) 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.808+0000 7fb16dc00640 1 -- 172.21.15.32:0/552467161 learned_addr learned my addr 172.21.15.32:0/552467161 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.809+0000 7fb16e600640 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bfba070 0x56553c36e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.809+0000 7fb16d200640 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf99390 0x56553c373820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.809+0000 7fb16dc00640 1 -- 172.21.15.32:0/552467161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf99390 msgr2=0x56553c373820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.809+0000 7fb16dc00640 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf99390 0x56553c373820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.809+0000 7fb16dc00640 1 -- 172.21.15.32:0/552467161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bfba070 msgr2=0x56553c36e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.809+0000 7fb16dc00640 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bfba070 0x56553c36e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.809+0000 7fb16dc00640 1 -- 172.21.15.32:0/552467161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56553c373ff0 con 0x56553bf911b0 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.809+0000 7fb16dc00640 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 0x56553c3732e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00f3c0 tx=0x7fb15c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518f323eda1c1d78 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.809+0000 7fb14f400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.809+0000 7fb14f400640 1 -- 172.21.15.32:0/552467161 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c01e040 con 0x56553bf911b0 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.810+0000 7fb14f400640 1 -- 172.21.15.32:0/552467161 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8485+0+0 (secure 0 0 0) 0x7fb15c009de0 con 0x56553bf911b0 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.810+0000 7fb17726f740 1 -- 172.21.15.32:0/552467161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb15404c070 con 0x56553bf911b0 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.810+0000 7fb14f400640 1 -- 172.21.15.32:0/552467161 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb15c019590 con 0x56553bf911b0 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.810+0000 7fb17726f740 10 client.17898 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.810+0000 7fb17726f740 1 -- 172.21.15.32:0/552467161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56553c3e8620 con 0x56553bf911b0 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.810+0000 7fb14f400640 1 -- 172.21.15.32:0/552467161 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb15404c070 con 0x56553bf911b0 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.810+0000 7fb14f400640 1 -- 172.21.15.32:0/552467161 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fb15c034070 con 0x56553bf911b0 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.810+0000 7fb17726f740 10 client.17898 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.810+0000 7fb17726f740 10 client.17898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:24:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.810+0000 7fb17726f740 20 client.17898 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.811+0000 7fb17726f740 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56553bf4dda0 0x56553c3eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.811+0000 7fb17726f740 4 client.17898 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:24:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.811+0000 7fb17726f740 1 -- 172.21.15.32:0/552467161 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56553c2d4160 con 0x56553bf4dda0 2024-08-31T07:24:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.811+0000 7fb16d200640 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56553bf4dda0 0x56553c3eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.811+0000 7fb16d200640 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56553bf4dda0 0x56553c3eac80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c36ee6e7be3a8e5f server_cookie=3af6f9d36d3c316b in_seq=0 out_seq=0 2024-08-31T07:24:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.811+0000 7fb14f400640 10 client.17898 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb14f400640 1 -- 172.21.15.32:0/552467161 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x56553c2d4160 con 0x56553bf4dda0 2024-08-31T07:24:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb14f400640 10 client.17898 handle_command_reply: tid=0 2024-08-31T07:24:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 client.17898 shutdown 2024-08-31T07:24:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 -- 172.21.15.32:0/552467161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 msgr2=0x56553c3732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 0x56553c3732e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00f3c0 tx=0x7fb15c005db0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 -- 172.21.15.32:0/552467161 shutdown_connections 2024-08-31T07:24:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56553bf4dda0 0x56553c3eac80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56553bfba070 0x56553c36e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56553bf99390 0x56553c373820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 --2- 172.21.15.32:0/552467161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56553bf911b0 0x56553c3732e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 -- 172.21.15.32:0/552467161 >> 172.21.15.32:0/552467161 conn(0x56553bfae460 msgr2=0x56553c36c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb14f400640 0 client.17898 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 -- 172.21.15.32:0/552467161 shutdown_connections 2024-08-31T07:24:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.812+0000 7fb17726f740 1 -- 172.21.15.32:0/552467161 wait complete. 2024-08-31T07:24:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb17726f740 20 client.17898 trim_cache size 0 max 16384 2024-08-31T07:24:14.816 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:24:14.816 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:24:14.818 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.819 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:24:14.820 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.822 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:24:14.824 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 -- 172.21.15.32:0/3385577640 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1500b4ca0 msgr2=0x7fb1500b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1500b4ca0 0x7fb1500b71b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb154009230 tx=0x7fb154031400 comp rx=0 tx=0).stop 2024-08-31T07:24:14.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 -- 172.21.15.32:0/3385577640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb16810bdb0 msgr2=0x7fb1681a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb16810bdb0 0x7fb1681a1ca0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb16000da10 tx=0x7fb16000ded0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 -- 172.21.15.32:0/3385577640 shutdown_connections 2024-08-31T07:24:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1500b4ca0 0x7fb1500b71b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb16810bdb0 0x7fb1681a1ca0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb16810b460 0x7fb16819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 --2- 172.21.15.32:0/3385577640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168105b20 0x7fb16819c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 -- 172.21.15.32:0/3385577640 >> 172.21.15.32:0/3385577640 conn(0x7fb1681015b0 msgr2=0x7fb1681039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 -- 172.21.15.32:0/3385577640 shutdown_connections 2024-08-31T07:24:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:14.813+0000 7fb170600640 1 -- 172.21.15.32:0/3385577640 wait complete. 2024-08-31T07:24:14.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:14 smithi032 ceph-mon[17252]: pgmap v723: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 234 KiB/s wr, 30 op/s 2024-08-31T07:24:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:14 smithi073 ceph-mon[21995]: pgmap v723: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 234 KiB/s wr, 30 op/s 2024-08-31T07:24:15.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:14 smithi084 ceph-mon[22387]: pgmap v723: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 234 KiB/s wr, 30 op/s 2024-08-31T07:24:15.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.74 MB/sec execute 484 sec latency 17828.552 ms 2024-08-31T07:24:15.882 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:24:15.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:24:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:16.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.73 MB/sec execute 485 sec latency 18828.662 ms 2024-08-31T07:24:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.993+0000 7f5082000640 1 -- 172.21.15.32:0/830974577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c103750 msgr2=0x7f507c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.993+0000 7f5082000640 1 --2- 172.21.15.32:0/830974577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c103750 0x7f507c103b60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5070009900 tx=0x7f507002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:24:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.993+0000 7f5082000640 1 -- 172.21.15.32:0/830974577 shutdown_connections 2024-08-31T07:24:16.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.993+0000 7f5082000640 1 --2- 172.21.15.32:0/830974577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c1099e0 0x7f507c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:16.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.993+0000 7f5082000640 1 --2- 172.21.15.32:0/830974577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c109090 0x7f507c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:16.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.993+0000 7f5082000640 1 --2- 172.21.15.32:0/830974577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c103750 0x7f507c103b60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:16.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.993+0000 7f5082000640 1 -- 172.21.15.32:0/830974577 >> 172.21.15.32:0/830974577 conn(0x7f507c0ff1e0 msgr2=0x7f507c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:16.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.993+0000 7f5082000640 1 -- 172.21.15.32:0/830974577 shutdown_connections 2024-08-31T07:24:16.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.993+0000 7f5082000640 1 -- 172.21.15.32:0/830974577 wait complete. 2024-08-31T07:24:16.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.994+0000 7f5082000640 1 Processor -- start 2024-08-31T07:24:16.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.994+0000 7f5082000640 1 -- start start 2024-08-31T07:24:16.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5082000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103750 0x7f507c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:16.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5082000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109090 0x7f507c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:16.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5082000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c1099e0 0x7f507c19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:16.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5082000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f507c113690 con 0x7f507c109090 2024-08-31T07:24:16.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5082000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f507c113510 con 0x7f507c1099e0 2024-08-31T07:24:16.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5082000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f507c113810 con 0x7f507c103750 2024-08-31T07:24:16.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5080c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103750 0x7f507c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5080c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103750 0x7f507c19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44396/0 (socket says 172.21.15.32:44396) 2024-08-31T07:24:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5080c00640 1 -- 172.21.15.32:0/1028120263 learned_addr learned my addr 172.21.15.32:0/1028120263 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f507be00640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109090 0x7f507c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5081600640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c1099e0 0x7f507c19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5080c00640 1 -- 172.21.15.32:0/1028120263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c1099e0 msgr2=0x7f507c19f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5080c00640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c1099e0 0x7f507c19f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.995+0000 7f5080c00640 1 -- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109090 msgr2=0x7f507c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.996+0000 7f5080c00640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109090 0x7f507c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.996+0000 7f5080c00640 1 -- 172.21.15.32:0/1028120263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c1a0030 con 0x7f507c103750 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.996+0000 7f5081600640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c1099e0 0x7f507c19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.996+0000 7f507be00640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109090 0x7f507c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.996+0000 7f5080c00640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103750 0x7f507c19a0c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5070005940 tx=0x7f50700040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.996+0000 7f5079600640 1 -- 172.21.15.32:0/1028120263 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507003f030 con 0x7f507c103750 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.996+0000 7f5082000640 1 -- 172.21.15.32:0/1028120263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507c1a02c0 con 0x7f507c103750 2024-08-31T07:24:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.996+0000 7f5079600640 1 -- 172.21.15.32:0/1028120263 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50700044c0 con 0x7f507c103750 2024-08-31T07:24:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.996+0000 7f5082000640 1 -- 172.21.15.32:0/1028120263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507c075390 con 0x7f507c103750 2024-08-31T07:24:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.997+0000 7f5079600640 1 -- 172.21.15.32:0/1028120263 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070035070 con 0x7f507c103750 2024-08-31T07:24:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:16.997+0000 7f5082000640 1 -- 172.21.15.32:0/1028120263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f505c0052c0 con 0x7f507c103750 2024-08-31T07:24:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.000+0000 7f5079600640 1 -- 172.21.15.32:0/1028120263 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5070042030 con 0x7f507c103750 2024-08-31T07:24:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.002+0000 7f5079600640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50640b4d20 0x7f50640b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.002+0000 7f5079600640 1 -- 172.21.15.32:0/1028120263 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f5070105a20 con 0x7f507c103750 2024-08-31T07:24:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.002+0000 7f507be00640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50640b4d20 0x7f50640b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.003+0000 7f507be00640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50640b4d20 0x7f50640b7230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f507c108c40 tx=0x7f5068005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.004+0000 7f5079600640 1 -- 172.21.15.32:0/1028120263 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f50700cc800 con 0x7f507c103750 2024-08-31T07:24:17.044 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:16 smithi032 ceph-mon[17252]: pgmap v724: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 90 KiB/s wr, 23 op/s 2024-08-31T07:24:17.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:16 smithi073 ceph-mon[21995]: pgmap v724: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 90 KiB/s wr, 23 op/s 2024-08-31T07:24:17.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.237+0000 7f5082000640 1 -- 172.21.15.32:0/1028120263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f505c0055b0 con 0x7f507c103750 2024-08-31T07:24:17.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.238+0000 7f5079600640 1 -- 172.21.15.32:0/1028120263 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11961 (secure 0 0 0) 0x7f50700cc100 con 0x7f507c103750 2024-08-31T07:24:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:24:17.239 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"btime":"2024-08-31T07:23:33:679601+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:23:33.380019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:24:17.240 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-08-31T07:24:17.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:16 smithi084 ceph-mon[22387]: pgmap v724: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 90 KiB/s wr, 23 op/s 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 -- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50640b4d20 msgr2=0x7f50640b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50640b4d20 0x7f50640b7230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f507c108c40 tx=0x7f5068005f90 comp rx=0 tx=0).stop 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 -- 172.21.15.32:0/1028120263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103750 msgr2=0x7f507c19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103750 0x7f507c19a0c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5070005940 tx=0x7f50700040c0 comp rx=0 tx=0).stop 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 -- 172.21.15.32:0/1028120263 shutdown_connections 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50640b4d20 0x7f50640b7230 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c1099e0 0x7f507c19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109090 0x7f507c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 --2- 172.21.15.32:0/1028120263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103750 0x7f507c19a0c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 -- 172.21.15.32:0/1028120263 >> 172.21.15.32:0/1028120263 conn(0x7f507c0ff1e0 msgr2=0x7f507c101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 -- 172.21.15.32:0/1028120263 shutdown_connections 2024-08-31T07:24:17.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:17.241+0000 7f5082000640 1 -- 172.21.15.32:0/1028120263 wait complete. 2024-08-31T07:24:17.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.72 MB/sec execute 486 sec latency 19828.767 ms 2024-08-31T07:24:18.044 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:17 smithi032 ceph-mon[17252]: osdmap e92: 12 total, 12 up, 12 in 2024-08-31T07:24:18.045 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:17 smithi032 ceph-mon[17252]: pgmap v726: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-08-31T07:24:18.045 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:17 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1028120263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:24:18.083 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:24:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:17 smithi073 ceph-mon[21995]: osdmap e92: 12 total, 12 up, 12 in 2024-08-31T07:24:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:17 smithi073 ceph-mon[21995]: pgmap v726: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-08-31T07:24:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:17 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1028120263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:24:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:17 smithi084 ceph-mon[22387]: osdmap e92: 12 total, 12 up, 12 in 2024-08-31T07:24:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:17 smithi084 ceph-mon[22387]: pgmap v726: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-08-31T07:24:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:17 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1028120263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:24:18.470 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:18.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.72 MB/sec execute 487 sec latency 20828.844 ms 2024-08-31T07:24:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.146+0000 7f10fe800640 1 -- 172.21.15.32:0/1371944099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f8103730 msgr2=0x7f10f8103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.146+0000 7f10fe800640 1 --2- 172.21.15.32:0/1371944099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f8103730 0x7f10f8103b40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009860 tx=0x7f10ec02a340 comp rx=0 tx=0).stop 2024-08-31T07:24:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.146+0000 7f10fe800640 1 -- 172.21.15.32:0/1371944099 shutdown_connections 2024-08-31T07:24:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.146+0000 7f10fe800640 1 --2- 172.21.15.32:0/1371944099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f81099c0 0x7f10f81101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.146+0000 7f10fe800640 1 --2- 172.21.15.32:0/1371944099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f8109070 0x7f10f8109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.146+0000 7f10fe800640 1 --2- 172.21.15.32:0/1371944099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f8103730 0x7f10f8103b40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.146+0000 7f10fe800640 1 -- 172.21.15.32:0/1371944099 >> 172.21.15.32:0/1371944099 conn(0x7f10f80ff180 msgr2=0x7f10f81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.146+0000 7f10fe800640 1 -- 172.21.15.32:0/1371944099 shutdown_connections 2024-08-31T07:24:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.146+0000 7f10fe800640 1 -- 172.21.15.32:0/1371944099 wait complete. 2024-08-31T07:24:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.147+0000 7f10fe800640 1 Processor -- start 2024-08-31T07:24:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fe800640 1 -- start start 2024-08-31T07:24:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f8103730 0x7f10f819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fe800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f8109070 0x7f10f819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f81099c0 0x7f10f819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fe800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8113670 con 0x7f10f8103730 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fe800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81134f0 con 0x7f10f81099c0 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fe800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81137f0 con 0x7f10f8109070 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f8103730 0x7f10f819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f8103730 0x7f10f819a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49252/0 (socket says 172.21.15.32:49252) 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fd400640 1 -- 172.21.15.32:0/2534981939 learned_addr learned my addr 172.21.15.32:0/2534981939 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.148+0000 7f10fca00640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f8109070 0x7f10f819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.149+0000 7f10fde00640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f81099c0 0x7f10f819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.149+0000 7f10fd400640 1 -- 172.21.15.32:0/2534981939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f8109070 msgr2=0x7f10f819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.149+0000 7f10fd400640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f8109070 0x7f10f819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.149+0000 7f10fd400640 1 -- 172.21.15.32:0/2534981939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f81099c0 msgr2=0x7f10f819f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.149+0000 7f10fd400640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f81099c0 0x7f10f819f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.149+0000 7f10fd400640 1 -- 172.21.15.32:0/2534981939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f81a0010 con 0x7f10f8103730 2024-08-31T07:24:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.149+0000 7f10fd400640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f8103730 0x7f10f819a0a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f10ec035040 tx=0x7f10ec032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.150+0000 7f10f6000640 1 -- 172.21.15.32:0/2534981939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec037040 con 0x7f10f8103730 2024-08-31T07:24:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.150+0000 7f10fe800640 1 -- 172.21.15.32:0/2534981939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f81a02a0 con 0x7f10f8103730 2024-08-31T07:24:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.150+0000 7f10f6000640 1 -- 172.21.15.32:0/2534981939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ec041070 con 0x7f10f8103730 2024-08-31T07:24:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.150+0000 7f10f6000640 1 -- 172.21.15.32:0/2534981939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec003850 con 0x7f10f8103730 2024-08-31T07:24:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.150+0000 7f10fe800640 1 -- 172.21.15.32:0/2534981939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f81a6a60 con 0x7f10f8103730 2024-08-31T07:24:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.153+0000 7f1106bf6740 1 Processor -- start 2024-08-31T07:24:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.153+0000 7f1106bf6740 1 -- start start 2024-08-31T07:24:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.153+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d77a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.153+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 0x564675d77fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.153+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 0x564675d784f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.153+0000 7f1106bf6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564675cee7b0 con 0x5646759d2070 2024-08-31T07:24:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.153+0000 7f1106bf6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564675cddd40 con 0x5646759b1390 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.153+0000 7f1106bf6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564675ce1cf0 con 0x5646759a91b0 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d77a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d77a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49260/0 (socket says 172.21.15.32:49260) 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fde00640 1 -- 172.21.15.32:0/1847120610 learned_addr learned my addr 172.21.15.32:0/1847120610 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10f6000640 1 -- 172.21.15.32:0/2534981939 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f10ec0039f0 con 0x7f10f8103730 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fca00640 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 0x564675d784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fd400640 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 0x564675d77fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fde00640 1 -- 172.21.15.32:0/1847120610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 msgr2=0x564675d77fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fde00640 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 0x564675d77fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fde00640 1 -- 172.21.15.32:0/1847120610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 msgr2=0x564675d784f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fde00640 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 0x564675d784f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fde00640 1 -- 172.21.15.32:0/1847120610 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564675a3d650 con 0x5646759d2070 2024-08-31T07:24:19.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fca00640 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 0x564675d784f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:19.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.154+0000 7f10fd400640 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 0x564675d77fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:24:19.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.155+0000 7f10fde00640 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d77a70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009fd0 tx=0x7f10f00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fe94c355b775b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.155+0000 7f10ea600640 1 -- 172.21.15.32:0/1847120610 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0004d20 con 0x5646759d2070 2024-08-31T07:24:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.155+0000 7f10ea600640 1 -- 172.21.15.32:0/1847120610 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10f0004050 con 0x5646759d2070 2024-08-31T07:24:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.155+0000 7f10ea600640 1 -- 172.21.15.32:0/1847120610 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0005280 con 0x5646759d2070 2024-08-31T07:24:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.156+0000 7f10f6000640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f10e00b4ca0 0x7f10e00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.156+0000 7f10f6000640 1 -- 172.21.15.32:0/2534981939 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f10ec104f40 con 0x7f10f8103730 2024-08-31T07:24:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.156+0000 7f10fd400640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f10e00b4ca0 0x7f10e00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.156+0000 7f1106bf6740 1 -- 172.21.15.32:0/1847120610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 msgr2=0x564675d77a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.156+0000 7f1106bf6740 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d77a70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009fd0 tx=0x7f10f00118d0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.156+0000 7f1106bf6740 1 -- 172.21.15.32:0/1847120610 shutdown_connections 2024-08-31T07:24:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.156+0000 7f1106bf6740 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 0x564675d784f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.156+0000 7f1106bf6740 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 0x564675d77fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.156+0000 7f1106bf6740 1 --2- 172.21.15.32:0/1847120610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d77a70 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.156+0000 7f1106bf6740 1 -- 172.21.15.32:0/1847120610 >> 172.21.15.32:0/1847120610 conn(0x5646759c6460 msgr2=0x564675d75960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.157+0000 7f1106bf6740 1 -- 172.21.15.32:0/1847120610 shutdown_connections 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.157+0000 7f1106bf6740 1 -- 172.21.15.32:0/1847120610 wait complete. 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.157+0000 7f10fd400640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f10e00b4ca0 0x7f10e00b71b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x564675d766f0 tx=0x7f10ec0ee040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.157+0000 7f1106bf6740 1 -- start start 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.157+0000 7f1106bf6740 1 Processor -- start 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.158+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759a91b0 0x564675d879f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.158+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759b1390 0x564675d87f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.158+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d93910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.158+0000 7f1106bf6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564675b40700 con 0x5646759d2070 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.158+0000 7f1106bf6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564675c64cf0 con 0x5646759a91b0 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.158+0000 7f1106bf6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564675d8a580 con 0x5646759b1390 2024-08-31T07:24:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.158+0000 7f1106bf6740 10 client.0 _ll_register_callbacks cb 0x564675c8a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.158+0000 7f10fde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759a91b0 0x564675d879f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759b1390 0x564675d87f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d93910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759b1390 0x564675d87f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44424/0 (socket says 172.21.15.32:44424) 2024-08-31T07:24:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fca00640 1 -- 172.21.15.32:0/3924333465 learned_addr learned my addr 172.21.15.32:0/3924333465 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:19.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d93910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49268/0 (socket says 172.21.15.32:49268) 2024-08-31T07:24:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fca00640 1 -- 172.21.15.32:0/3924333465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759a91b0 msgr2=0x564675d879f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fca00640 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759a91b0 0x564675d879f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fca00640 1 -- 172.21.15.32:0/3924333465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 msgr2=0x564675d93910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fca00640 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d93910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fca00640 1 -- 172.21.15.32:0/3924333465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564675ca58b0 con 0x5646759b1390 2024-08-31T07:24:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10fca00640 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759b1390 0x564675d87f30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f10e40116a0 tx=0x7f10e4011b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a002be4ba82e55a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.159+0000 7f10dea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:24:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.160+0000 7f10dea00640 1 -- 172.21.15.32:0/3924333465 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e401e020 con 0x5646759b1390 2024-08-31T07:24:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.160+0000 7f1106bf6740 1 -- 172.21.15.32:0/3924333465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10f0004050 con 0x5646759b1390 2024-08-31T07:24:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.160+0000 7f10dea00640 1 -- 172.21.15.32:0/3924333465 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f10e400fbf0 con 0x5646759b1390 2024-08-31T07:24:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.160+0000 7f10dea00640 1 -- 172.21.15.32:0/3924333465 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f10e400fdf0 con 0x5646759b1390 2024-08-31T07:24:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.160+0000 7f1106bf6740 10 client.27922 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.160+0000 7f1106bf6740 1 -- 172.21.15.32:0/3924333465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564675d96600 con 0x5646759b1390 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.160+0000 7f10dea00640 1 -- 172.21.15.32:0/3924333465 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10f0004050 con 0x5646759b1390 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.160+0000 7f10dea00640 1 -- 172.21.15.32:0/3924333465 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f10e4019070 con 0x5646759b1390 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.161+0000 7f1106bf6740 10 client.27922 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.161+0000 7f1106bf6740 10 client.27922 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.161+0000 7f1106bf6740 20 client.27922 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.161+0000 7f1106bf6740 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5646759909f0 0x564675d99bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.161+0000 7f1106bf6740 4 client.27922 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.161+0000 7f1106bf6740 1 -- 172.21.15.32:0/3924333465 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564675cf0700 con 0x5646759909f0 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.161+0000 7f10fde00640 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5646759909f0 0x564675d99bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.161+0000 7f10fde00640 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5646759909f0 0x564675d99bc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=90d7bcc443f77e6b server_cookie=c73667d36cde9a78 in_seq=0 out_seq=0 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.161+0000 7f10dea00640 10 client.27922 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.162+0000 7f10dea00640 1 -- 172.21.15.32:0/3924333465 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564675cf0700 con 0x5646759909f0 2024-08-31T07:24:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.162+0000 7f10dea00640 10 client.27922 handle_command_reply: tid=0 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.162+0000 7f1106bf6740 1 client.27922 shutdown 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 1 -- 172.21.15.32:0/3924333465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759b1390 msgr2=0x564675d87f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759b1390 0x564675d87f30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f10e40116a0 tx=0x7f10e4011b60 comp rx=0 tx=0).stop 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 1 -- 172.21.15.32:0/3924333465 shutdown_connections 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5646759909f0 0x564675d99bc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d93910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759b1390 0x564675d87f30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 1 --2- 172.21.15.32:0/3924333465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759a91b0 0x564675d879f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f10dea00640 0 client.27922 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 1 -- 172.21.15.32:0/3924333465 >> 172.21.15.32:0/3924333465 conn(0x5646759c6460 msgr2=0x564675d73f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 1 -- 172.21.15.32:0/3924333465 shutdown_connections 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 1 -- 172.21.15.32:0/3924333465 wait complete. 2024-08-31T07:24:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.163+0000 7f1106bf6740 20 client.27922 trim_cache size 0 max 16384 2024-08-31T07:24:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.177+0000 7f1106bf6740 1 Processor -- start 2024-08-31T07:24:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.177+0000 7f1106bf6740 1 -- start start 2024-08-31T07:24:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.177+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 0x564675d745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.177+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 0x564675d89420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.177+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d93810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f1106bf6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81acfd0 con 0x5646759d2070 2024-08-31T07:24:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f1106bf6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81aeb40 con 0x5646759b1390 2024-08-31T07:24:19.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f1106bf6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81ad6a0 con 0x5646759a91b0 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f10fde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d93810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f10fca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 0x564675d89420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f10fde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d93810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49284/0 (socket says 172.21.15.32:49284) 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f10fde00640 1 -- 172.21.15.32:0/380606212 learned_addr learned my addr 172.21.15.32:0/380606212 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f10fde00640 1 --2- 172.21.15.32:0/380606212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 0x564675d745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f10fca00640 1 -- 172.21.15.32:0/380606212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 msgr2=0x564675d745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f10fca00640 1 --2- 172.21.15.32:0/380606212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 0x564675d745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f10fca00640 1 -- 172.21.15.32:0/380606212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 msgr2=0x564675d93810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f10fca00640 1 --2- 172.21.15.32:0/380606212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d93810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.178+0000 7f10fca00640 1 -- 172.21.15.32:0/380606212 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564675d762e0 con 0x5646759b1390 2024-08-31T07:24:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.179+0000 7f10fca00640 1 --2- 172.21.15.32:0/380606212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 0x564675d89420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f10e4001eb0 tx=0x7f10e400ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631a8c5a3b804dea server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.179+0000 7f10e9c00640 1 -- 172.21.15.32:0/380606212 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e401e070 con 0x5646759b1390 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.179+0000 7f10e9c00640 1 -- 172.21.15.32:0/380606212 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10e4022cd0 con 0x5646759b1390 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.179+0000 7f10e9c00640 1 -- 172.21.15.32:0/380606212 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4005280 con 0x5646759b1390 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.180+0000 7f1106bf6740 1 -- 172.21.15.32:0/380606212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 msgr2=0x564675d89420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.180+0000 7f1106bf6740 1 --2- 172.21.15.32:0/380606212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 0x564675d89420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f10e4001eb0 tx=0x7f10e400ff90 comp rx=0 tx=0).stop 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.180+0000 7f1106bf6740 1 -- 172.21.15.32:0/380606212 shutdown_connections 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.180+0000 7f1106bf6740 1 --2- 172.21.15.32:0/380606212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759d2070 0x564675d93810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.180+0000 7f1106bf6740 1 --2- 172.21.15.32:0/380606212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759b1390 0x564675d89420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.180+0000 7f1106bf6740 1 --2- 172.21.15.32:0/380606212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759a91b0 0x564675d745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.180+0000 7f1106bf6740 1 -- 172.21.15.32:0/380606212 >> 172.21.15.32:0/380606212 conn(0x5646759c6460 msgr2=0x564675d88d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.180+0000 7f1106bf6740 1 -- 172.21.15.32:0/380606212 shutdown_connections 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.180+0000 7f1106bf6740 1 -- 172.21.15.32:0/380606212 wait complete. 2024-08-31T07:24:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.180+0000 7f1106bf6740 1 -- start start 2024-08-31T07:24:19.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f1106bf6740 1 Processor -- start 2024-08-31T07:24:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759a91b0 0x564675d8b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759b1390 0x564675d8be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f1106bf6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759d2070 0x564675d8c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f1106bf6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81ae440 con 0x5646759b1390 2024-08-31T07:24:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f1106bf6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81add70 con 0x5646759a91b0 2024-08-31T07:24:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f1106bf6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81aafd0 con 0x5646759d2070 2024-08-31T07:24:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f1106bf6740 10 client.0 _ll_register_callbacks cb 0x7f10e4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f10fca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759b1390 0x564675d8be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f10fde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759a91b0 0x564675d8b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f10fca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759b1390 0x564675d8be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49296/0 (socket says 172.21.15.32:49296) 2024-08-31T07:24:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.181+0000 7f10fca00640 1 -- 172.21.15.32:0/1341794895 learned_addr learned my addr 172.21.15.32:0/1341794895 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.182+0000 7f10fde00640 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759d2070 0x564675d8c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.182+0000 7f10fca00640 1 -- 172.21.15.32:0/1341794895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759d2070 msgr2=0x564675d8c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.182+0000 7f10fca00640 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759d2070 0x564675d8c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.182+0000 7f10fca00640 1 -- 172.21.15.32:0/1341794895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759a91b0 msgr2=0x564675d8b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.182+0000 7f10fca00640 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759a91b0 0x564675d8b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.182+0000 7f10fca00640 1 -- 172.21.15.32:0/1341794895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564675e03680 con 0x5646759b1390 2024-08-31T07:24:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.182+0000 7f10fca00640 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759b1390 0x564675d8be70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f10e40059d0 tx=0x7f10e4022740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284c2fd0227deec2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.182+0000 7f10dea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f10dea00640 1 -- 172.21.15.32:0/1341794895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e401e070 con 0x5646759b1390 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f1106bf6740 1 -- 172.21.15.32:0/1341794895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10e4022cd0 con 0x5646759b1390 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f10dea00640 1 -- 172.21.15.32:0/1341794895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f10e4017050 con 0x5646759b1390 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f10dea00640 1 -- 172.21.15.32:0/1341794895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f10e400fbf0 con 0x5646759b1390 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f1106bf6740 10 client.17922 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f1106bf6740 1 -- 172.21.15.32:0/1341794895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564675e048e0 con 0x5646759b1390 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f10dea00640 1 -- 172.21.15.32:0/1341794895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10e4022cd0 con 0x5646759b1390 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f10dea00640 1 -- 172.21.15.32:0/1341794895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7f10e4019040 con 0x5646759b1390 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f1106bf6740 10 client.17922 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f1106bf6740 10 client.17922 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f1106bf6740 20 client.17922 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f1106bf6740 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x564675965da0 0x564675e06fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f1106bf6740 4 client.17922 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:24:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.183+0000 7f1106bf6740 1 -- 172.21.15.32:0/1341794895 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564675cec1a0 con 0x564675965da0 2024-08-31T07:24:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.184+0000 7f10fde00640 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x564675965da0 0x564675e06fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.184+0000 7f10fde00640 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x564675965da0 0x564675e06fb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=177fddea39857f0b server_cookie=6d2da72febc19c1c in_seq=0 out_seq=0 2024-08-31T07:24:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.184+0000 7f10dea00640 10 client.17922 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:24:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f10dea00640 1 -- 172.21.15.32:0/1341794895 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564675cec1a0 con 0x564675965da0 2024-08-31T07:24:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f10dea00640 10 client.17922 handle_command_reply: tid=0 2024-08-31T07:24:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 client.17922 shutdown 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 -- 172.21.15.32:0/1341794895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759b1390 msgr2=0x564675d8be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759b1390 0x564675d8be70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f10e40059d0 tx=0x7f10e4022740 comp rx=0 tx=0).stop 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 -- 172.21.15.32:0/1341794895 shutdown_connections 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x564675965da0 0x564675e06fb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646759d2070 0x564675d8c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5646759b1390 0x564675d8be70 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f10dea00640 0 client.17922 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 --2- 172.21.15.32:0/1341794895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5646759a91b0 0x564675d8b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 -- 172.21.15.32:0/1341794895 >> 172.21.15.32:0/1341794895 conn(0x5646759c6460 msgr2=0x564675d758d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 -- 172.21.15.32:0/1341794895 shutdown_connections 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.185+0000 7f1106bf6740 1 -- 172.21.15.32:0/1341794895 wait complete. 2024-08-31T07:24:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f1106bf6740 20 client.17922 trim_cache size 0 max 16384 2024-08-31T07:24:19.191 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:24:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f10fe800640 1 -- 172.21.15.32:0/2534981939 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f10e00b4ca0 msgr2=0x7f10e00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f10fe800640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f10e00b4ca0 0x7f10e00b71b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x564675d766f0 tx=0x7f10ec0ee040 comp rx=0 tx=0).stop 2024-08-31T07:24:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f10fe800640 1 -- 172.21.15.32:0/2534981939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f8103730 msgr2=0x7f10f819a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:19.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f10fe800640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f8103730 0x7f10f819a0a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f10ec035040 tx=0x7f10ec032d50 comp rx=0 tx=0).stop 2024-08-31T07:24:19.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f10fe800640 1 -- 172.21.15.32:0/2534981939 shutdown_connections 2024-08-31T07:24:19.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f10fe800640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f10e00b4ca0 0x7f10e00b71b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f10fe800640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10f81099c0 0x7f10f819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f10fe800640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f8109070 0x7f10f819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f10fe800640 1 --2- 172.21.15.32:0/2534981939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f8103730 0x7f10f819a0a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:19.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.186+0000 7f10fe800640 1 -- 172.21.15.32:0/2534981939 >> 172.21.15.32:0/2534981939 conn(0x7f10f80ff180 msgr2=0x7f10f81015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:19.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.187+0000 7f10fe800640 1 -- 172.21.15.32:0/2534981939 shutdown_connections 2024-08-31T07:24:19.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:19.187+0000 7f10fe800640 1 -- 172.21.15.32:0/2534981939 wait complete. 2024-08-31T07:24:19.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.71 MB/sec execute 488 sec latency 21828.915 ms 2024-08-31T07:24:19.826 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:24:20.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:24:20.173 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:20 smithi032 ceph-mon[17252]: pgmap v727: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 18 op/s 2024-08-31T07:24:20.224 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:20 smithi073 ceph-mon[21995]: pgmap v727: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 18 op/s 2024-08-31T07:24:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:20 smithi084 ceph-mon[22387]: pgmap v727: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 18 op/s 2024-08-31T07:24:20.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.71 MB/sec execute 489 sec latency 22828.962 ms 2024-08-31T07:24:20.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.907+0000 7f5abdc00640 1 -- 172.21.15.32:0/1817307568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ab8103900 msgr2=0x7f5ab8103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:20.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.907+0000 7f5abdc00640 1 --2- 172.21.15.32:0/1817307568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ab8103900 0x7f5ab8103d10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009900 tx=0x7f5aa802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:24:20.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.907+0000 7f5abdc00640 1 -- 172.21.15.32:0/1817307568 shutdown_connections 2024-08-31T07:24:20.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.907+0000 7f5abdc00640 1 --2- 172.21.15.32:0/1817307568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab8109b90 0x7f5ab81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:20.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.907+0000 7f5abdc00640 1 --2- 172.21.15.32:0/1817307568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab8109240 0x7f5ab8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:20.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.907+0000 7f5abdc00640 1 --2- 172.21.15.32:0/1817307568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ab8103900 0x7f5ab8103d10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:20.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.907+0000 7f5abdc00640 1 -- 172.21.15.32:0/1817307568 >> 172.21.15.32:0/1817307568 conn(0x7f5ab807b5f0 msgr2=0x7f5ab807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:20.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.907+0000 7f5abdc00640 1 -- 172.21.15.32:0/1817307568 shutdown_connections 2024-08-31T07:24:20.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.907+0000 7f5abdc00640 1 -- 172.21.15.32:0/1817307568 wait complete. 2024-08-31T07:24:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.908+0000 7f5abdc00640 1 Processor -- start 2024-08-31T07:24:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.909+0000 7f5abdc00640 1 -- start start 2024-08-31T07:24:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.909+0000 7f5abdc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ab8103900 0x7f5ab80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.909+0000 7f5abdc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab8109240 0x7f5ab80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.909+0000 7f5abdc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab8109b90 0x7f5ab8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.909+0000 7f5abdc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81138a0 con 0x7f5ab8109240 2024-08-31T07:24:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.909+0000 7f5abdc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8113720 con 0x7f5ab8109b90 2024-08-31T07:24:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.909+0000 7f5abdc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8113a20 con 0x7f5ab8103900 2024-08-31T07:24:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.909+0000 7f5ab6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab8109240 0x7f5ab80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.909+0000 7f5ab6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab8109240 0x7f5ab80759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49310/0 (socket says 172.21.15.32:49310) 2024-08-31T07:24:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.909+0000 7f5ab6000640 1 -- 172.21.15.32:0/2303920398 learned_addr learned my addr 172.21.15.32:0/2303920398 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.910+0000 7f5ab6a00640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ab8103900 0x7f5ab80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.910+0000 7f5ab7400640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab8109b90 0x7f5ab8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.910+0000 7f5ab7400640 1 -- 172.21.15.32:0/2303920398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ab8103900 msgr2=0x7f5ab80754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.910+0000 7f5ab7400640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ab8103900 0x7f5ab80754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.910+0000 7f5ab7400640 1 -- 172.21.15.32:0/2303920398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab8109240 msgr2=0x7f5ab80759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.910+0000 7f5ab7400640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab8109240 0x7f5ab80759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.910+0000 7f5ab7400640 1 -- 172.21.15.32:0/2303920398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab81a8650 con 0x7f5ab8109b90 2024-08-31T07:24:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.910+0000 7f5ab6a00640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ab8103900 0x7f5ab80754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.910+0000 7f5ab6000640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab8109240 0x7f5ab80759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.911+0000 7f5ab7400640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab8109b90 0x7f5ab8075f30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009fd0 tx=0x7f5ab000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.911+0000 7f5aaf400640 1 -- 172.21.15.32:0/2303920398 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000ce20 con 0x7f5ab8109b90 2024-08-31T07:24:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.911+0000 7f5aaf400640 1 -- 172.21.15.32:0/2303920398 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab0005020 con 0x7f5ab8109b90 2024-08-31T07:24:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.911+0000 7f5abdc00640 1 -- 172.21.15.32:0/2303920398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab81a8940 con 0x7f5ab8109b90 2024-08-31T07:24:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.911+0000 7f5aaf400640 1 -- 172.21.15.32:0/2303920398 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0010690 con 0x7f5ab8109b90 2024-08-31T07:24:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.911+0000 7f5abdc00640 1 -- 172.21.15.32:0/2303920398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab81a8e50 con 0x7f5ab8109b90 2024-08-31T07:24:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.912+0000 7f5abdc00640 1 -- 172.21.15.32:0/2303920398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab8103d10 con 0x7f5ab8109b90 2024-08-31T07:24:20.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.915+0000 7f5aaf400640 1 -- 172.21.15.32:0/2303920398 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5ab00054d0 con 0x7f5ab8109b90 2024-08-31T07:24:20.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.916+0000 7f5aaf400640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a9c0b4ca0 0x7f5a9c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:20.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.916+0000 7f5aaf400640 1 -- 172.21.15.32:0/2303920398 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f5ab00e5c10 con 0x7f5ab8109b90 2024-08-31T07:24:20.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.917+0000 7f5ab6a00640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a9c0b4ca0 0x7f5a9c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.917+0000 7f5ab6a00640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a9c0b4ca0 0x7f5a9c0b71b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005ab0 tx=0x7f5aa8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:20.918+0000 7f5aaf400640 1 -- 172.21.15.32:0/2303920398 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5ab00ac9f0 con 0x7f5ab8109b90 2024-08-31T07:24:20.927 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.161+0000 7f5abdc00640 1 -- 172.21.15.32:0/2303920398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ab80767b0 con 0x7f5ab8109b90 2024-08-31T07:24:21.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.162+0000 7f5aaf400640 1 -- 172.21.15.32:0/2303920398 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11961 (secure 0 0 0) 0x7f5ab00ac2f0 con 0x7f5ab8109b90 2024-08-31T07:24:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:24:21.163 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"btime":"2024-08-31T07:23:33:679601+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:23:33.380019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:24:21.164 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 44 2024-08-31T07:24:21.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.164+0000 7f5abdc00640 1 -- 172.21.15.32:0/2303920398 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a9c0b4ca0 msgr2=0x7f5a9c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.164+0000 7f5abdc00640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a9c0b4ca0 0x7f5a9c0b71b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005ab0 tx=0x7f5aa8005a20 comp rx=0 tx=0).stop 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.165+0000 7f5abdc00640 1 -- 172.21.15.32:0/2303920398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab8109b90 msgr2=0x7f5ab8075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.165+0000 7f5abdc00640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab8109b90 0x7f5ab8075f30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009fd0 tx=0x7f5ab000ef10 comp rx=0 tx=0).stop 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.165+0000 7f5abdc00640 1 -- 172.21.15.32:0/2303920398 shutdown_connections 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.165+0000 7f5abdc00640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5a9c0b4ca0 0x7f5a9c0b71b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.165+0000 7f5abdc00640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ab8109b90 0x7f5ab8075f30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.165+0000 7f5abdc00640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab8109240 0x7f5ab80759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.165+0000 7f5abdc00640 1 --2- 172.21.15.32:0/2303920398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ab8103900 0x7f5ab80754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.165+0000 7f5abdc00640 1 -- 172.21.15.32:0/2303920398 >> 172.21.15.32:0/2303920398 conn(0x7f5ab807b5f0 msgr2=0x7f5ab8107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.165+0000 7f5abdc00640 1 -- 172.21.15.32:0/2303920398 shutdown_connections 2024-08-31T07:24:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.165+0000 7f5abdc00640 1 -- 172.21.15.32:0/2303920398 wait complete. 2024-08-31T07:24:21.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.70 MB/sec execute 490 sec latency 23829.010 ms 2024-08-31T07:24:21.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.856+0000 7fbcefe00640 1 -- 172.21.15.32:0/4014682960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf01099e0 msgr2=0x7fbcf0110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.856+0000 7fbcefe00640 1 --2- 172.21.15.32:0/4014682960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf01099e0 0x7fbcf0110210 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009ca0 tx=0x7fbce802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.856+0000 7fbcefe00640 1 -- 172.21.15.32:0/4014682960 shutdown_connections 2024-08-31T07:24:21.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.856+0000 7fbcefe00640 1 --2- 172.21.15.32:0/4014682960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf01099e0 0x7fbcf0110210 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.856+0000 7fbcefe00640 1 --2- 172.21.15.32:0/4014682960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf0109090 0x7fbcf01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.856+0000 7fbcefe00640 1 --2- 172.21.15.32:0/4014682960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf0103750 0x7fbcf0103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.856+0000 7fbcefe00640 1 -- 172.21.15.32:0/4014682960 >> 172.21.15.32:0/4014682960 conn(0x7fbcf00ff1a0 msgr2=0x7fbcf01015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.856+0000 7fbcefe00640 1 -- 172.21.15.32:0/4014682960 shutdown_connections 2024-08-31T07:24:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.856+0000 7fbcefe00640 1 -- 172.21.15.32:0/4014682960 wait complete. 2024-08-31T07:24:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.857+0000 7fbcefe00640 1 Processor -- start 2024-08-31T07:24:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbcefe00640 1 -- start start 2024-08-31T07:24:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbcefe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf0103750 0x7fbcf019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbcefe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf0109090 0x7fbcf019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbcefe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf01099e0 0x7fbcf019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbcefe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0113690 con 0x7fbcf0103750 2024-08-31T07:24:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbcefe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0113510 con 0x7fbcf01099e0 2024-08-31T07:24:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbcefe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0113810 con 0x7fbcf0109090 2024-08-31T07:24:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbceea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf0103750 0x7fbcf019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbcef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf01099e0 0x7fbcf019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbceea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf0103750 0x7fbcf019a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49328/0 (socket says 172.21.15.32:49328) 2024-08-31T07:24:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbceea00640 1 -- 172.21.15.32:0/907896755 learned_addr learned my addr 172.21.15.32:0/907896755 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.858+0000 7fbcee000640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf0109090 0x7fbcf019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.859+0000 7fbceea00640 1 -- 172.21.15.32:0/907896755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf0109090 msgr2=0x7fbcf019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.859+0000 7fbceea00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf0109090 0x7fbcf019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.859+0000 7fbceea00640 1 -- 172.21.15.32:0/907896755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf01099e0 msgr2=0x7fbcf019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.859+0000 7fbceea00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf01099e0 0x7fbcf019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.859+0000 7fbceea00640 1 -- 172.21.15.32:0/907896755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf019ff70 con 0x7fbcf0103750 2024-08-31T07:24:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.859+0000 7fbcee000640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf0109090 0x7fbcf019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.859+0000 7fbceea00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf0103750 0x7fbcf019a000 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbce000e9f0 tx=0x7fbce000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:21.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.860+0000 7fbce7400640 1 -- 172.21.15.32:0/907896755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce000cd80 con 0x7fbcf0103750 2024-08-31T07:24:21.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.860+0000 7fbce7400640 1 -- 172.21.15.32:0/907896755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce00108e0 con 0x7fbcf0103750 2024-08-31T07:24:21.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.860+0000 7fbce7400640 1 -- 172.21.15.32:0/907896755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0018980 con 0x7fbcf0103750 2024-08-31T07:24:21.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.860+0000 7fbcefe00640 1 -- 172.21.15.32:0/907896755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf01a0260 con 0x7fbcf0103750 2024-08-31T07:24:21.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.860+0000 7fbcefe00640 1 -- 172.21.15.32:0/907896755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf0075390 con 0x7fbcf0103750 2024-08-31T07:24:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.863+0000 7fbcfc9f2740 1 Processor -- start 2024-08-31T07:24:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.864+0000 7fbcfc9f2740 1 -- start start 2024-08-31T07:24:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.864+0000 7fbce7400640 1 -- 172.21.15.32:0/907896755 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbce0010430 con 0x7fbcf0103750 2024-08-31T07:24:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.864+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb787a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.864+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3b91b0 0x556dbb787fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.864+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3c1390 0x556dbb7884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.864+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556dbb6fe7b0 con 0x556dbb3c1390 2024-08-31T07:24:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.864+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556dbb6edd40 con 0x556dbb3b91b0 2024-08-31T07:24:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.864+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556dbb6f1cf0 con 0x556dbb3e2070 2024-08-31T07:24:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.865+0000 7fbceea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3b91b0 0x556dbb787fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.865+0000 7fbcef400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb787a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.865+0000 7fbcef400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb787a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44486/0 (socket says 172.21.15.32:44486) 2024-08-31T07:24:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.865+0000 7fbcef400640 1 -- 172.21.15.32:0/990528971 learned_addr learned my addr 172.21.15.32:0/990528971 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.865+0000 7fbcee000640 1 --2- 172.21.15.32:0/990528971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3c1390 0x556dbb7884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.865+0000 7fbcef400640 1 -- 172.21.15.32:0/990528971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3b91b0 msgr2=0x556dbb787fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.865+0000 7fbcef400640 1 --2- 172.21.15.32:0/990528971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3b91b0 0x556dbb787fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.865+0000 7fbcef400640 1 -- 172.21.15.32:0/990528971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3c1390 msgr2=0x556dbb7884f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.865+0000 7fbcef400640 1 --2- 172.21.15.32:0/990528971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3c1390 0x556dbb7884f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.865+0000 7fbcef400640 1 -- 172.21.15.32:0/990528971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbb44d650 con 0x556dbb3e2070 2024-08-31T07:24:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.866+0000 7fbcee000640 1 --2- 172.21.15.32:0/990528971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3c1390 0x556dbb7884f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.866+0000 7fbce7400640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd40b4ca0 0x7fbcd40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.867+0000 7fbcef400640 1 --2- 172.21.15.32:0/990528971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb787a70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbce802f410 tx=0x7fbce802f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b58bdacb0b65e717 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.867+0000 7fbce7400640 1 -- 172.21.15.32:0/907896755 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7fbce00e5860 con 0x7fbcf0103750 2024-08-31T07:24:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.867+0000 7fbceea00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd40b4ca0 0x7fbcd40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.867+0000 7fbcdbe00640 1 -- 172.21.15.32:0/990528971 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce804c070 con 0x556dbb3e2070 2024-08-31T07:24:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.867+0000 7fbcdbe00640 1 -- 172.21.15.32:0/990528971 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce8036040 con 0x556dbb3e2070 2024-08-31T07:24:21.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.867+0000 7fbcdbe00640 1 -- 172.21.15.32:0/990528971 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce803c720 con 0x556dbb3e2070 2024-08-31T07:24:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.869+0000 7fbceea00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd40b4ca0 0x7fbcd40b71b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x556dbb7866f0 tx=0x7fbce00eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.869+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/990528971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 msgr2=0x556dbb787a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.869+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/990528971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb787a70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbce802f410 tx=0x7fbce802f440 comp rx=0 tx=0).stop 2024-08-31T07:24:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.869+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/990528971 shutdown_connections 2024-08-31T07:24:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.869+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/990528971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3c1390 0x556dbb7884f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.869+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/990528971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3b91b0 0x556dbb787fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.869+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/990528971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb787a70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.869+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/990528971 >> 172.21.15.32:0/990528971 conn(0x556dbb3d6460 msgr2=0x556dbb785960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.869+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/990528971 shutdown_connections 2024-08-31T07:24:21.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.869+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/990528971 wait complete. 2024-08-31T07:24:21.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.870+0000 7fbcfc9f2740 1 -- start start 2024-08-31T07:24:21.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.870+0000 7fbcfc9f2740 1 Processor -- start 2024-08-31T07:24:21.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.870+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb793690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.870+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3c1390 0x556dbb793c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.870+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3e2070 0x556dbb79f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.870+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556dbb550700 con 0x556dbb3b91b0 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.870+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556dbb674cf0 con 0x556dbb3e2070 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.870+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556dbb7961f0 con 0x556dbb3c1390 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.871+0000 7fbcfc9f2740 10 client.0 _ll_register_callbacks cb 0x556dbb69a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.871+0000 7fbcef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb793690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.871+0000 7fbcee000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3c1390 0x556dbb793c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.871+0000 7fbcef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb793690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49352/0 (socket says 172.21.15.32:49352) 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.871+0000 7fbcef400640 1 -- 172.21.15.32:0/973025992 learned_addr learned my addr 172.21.15.32:0/973025992 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:21.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.871+0000 7fbcef400640 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3e2070 0x556dbb79f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcef400640 1 -- 172.21.15.32:0/973025992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3c1390 msgr2=0x556dbb793c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcef400640 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3c1390 0x556dbb793c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcef400640 1 -- 172.21.15.32:0/973025992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 msgr2=0x556dbb793690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcef400640 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb793690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcef400640 1 -- 172.21.15.32:0/973025992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dbb6b58b0 con 0x556dbb3e2070 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcef400640 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3e2070 0x556dbb79f4e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009620 tx=0x7fbce802f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=289626b99138cb0d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcd8c00640 1 -- 172.21.15.32:0/973025992 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce804c070 con 0x556dbb3e2070 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/973025992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce8036040 con 0x556dbb3e2070 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcd8c00640 1 -- 172.21.15.32:0/973025992 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7fbce803c920 con 0x556dbb3e2070 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcd8c00640 1 -- 172.21.15.32:0/973025992 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbce803cb20 con 0x556dbb3e2070 2024-08-31T07:24:21.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcfc9f2740 10 client.27935 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.872+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/973025992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556dbb7a2280 con 0x556dbb3e2070 2024-08-31T07:24:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.873+0000 7fbcd8c00640 1 -- 172.21.15.32:0/973025992 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce8036040 con 0x556dbb3e2070 2024-08-31T07:24:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.873+0000 7fbcd8c00640 1 -- 172.21.15.32:0/973025992 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fbce80305c0 con 0x556dbb3e2070 2024-08-31T07:24:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.873+0000 7fbcfc9f2740 10 client.27935 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.873+0000 7fbcfc9f2740 10 client.27935 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:24:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.873+0000 7fbcfc9f2740 20 client.27935 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.873+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556dbb3a09f0 0x556dbb7a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.873+0000 7fbcfc9f2740 4 client.27935 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:24:21.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.873+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/973025992 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dbb700700 con 0x556dbb3a09f0 2024-08-31T07:24:21.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.874+0000 7fbcee000640 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556dbb3a09f0 0x556dbb7a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.874+0000 7fbcee000640 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556dbb3a09f0 0x556dbb7a55e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=346d4cf315fb5d9 server_cookie=cc4b5be31dd11aa9 in_seq=0 out_seq=0 2024-08-31T07:24:21.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.874+0000 7fbcd8c00640 10 client.27935 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:24:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcd8c00640 1 -- 172.21.15.32:0/973025992 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556dbb700700 con 0x556dbb3a09f0 2024-08-31T07:24:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcd8c00640 10 client.27935 handle_command_reply: tid=0 2024-08-31T07:24:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcfc9f2740 1 client.27935 shutdown 2024-08-31T07:24:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/973025992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3e2070 msgr2=0x556dbb79f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3e2070 0x556dbb79f4e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009620 tx=0x7fbce802f3a0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/973025992 shutdown_connections 2024-08-31T07:24:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556dbb3a09f0 0x556dbb7a55e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3e2070 0x556dbb79f4e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3c1390 0x556dbb793c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcd8c00640 0 client.27935 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:24:21.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/973025992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb793690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.875+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/973025992 >> 172.21.15.32:0/973025992 conn(0x556dbb3d6460 msgr2=0x556dbb78ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:21.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.876+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/973025992 shutdown_connections 2024-08-31T07:24:21.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.876+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/973025992 wait complete. 2024-08-31T07:24:21.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.876+0000 7fbcfc9f2740 20 client.27935 trim_cache size 0 max 16384 2024-08-31T07:24:21.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.890+0000 7fbcfc9f2740 1 Processor -- start 2024-08-31T07:24:21.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.890+0000 7fbcfc9f2740 1 -- start start 2024-08-31T07:24:21.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.890+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb78d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.890+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 0x556dbb78ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.890+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb7950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.890+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01acf60 con 0x556dbb3b91b0 2024-08-31T07:24:21.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.890+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01aead0 con 0x556dbb3c1390 2024-08-31T07:24:21.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.890+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01ad630 con 0x556dbb3e2070 2024-08-31T07:24:21.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb78d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb78d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49358/0 (socket says 172.21.15.32:49358) 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcef400640 1 -- 172.21.15.32:0/3603673372 learned_addr learned my addr 172.21.15.32:0/3603673372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcee000640 1 --2- 172.21.15.32:0/3603673372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 0x556dbb78ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcef400640 1 --2- 172.21.15.32:0/3603673372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb7950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcee000640 1 -- 172.21.15.32:0/3603673372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 msgr2=0x556dbb7950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcee000640 1 --2- 172.21.15.32:0/3603673372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb7950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcee000640 1 -- 172.21.15.32:0/3603673372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 msgr2=0x556dbb78d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcee000640 1 --2- 172.21.15.32:0/3603673372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb78d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcee000640 1 -- 172.21.15.32:0/3603673372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbb789d60 con 0x556dbb3c1390 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.891+0000 7fbcef400640 1 --2- 172.21.15.32:0/3603673372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb7950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.892+0000 7fbcee000640 1 --2- 172.21.15.32:0/3603673372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 0x556dbb78ec40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x556dbb6fe8e0 tx=0x7fbcdc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52996553df32a19b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.892+0000 7fbcdb400640 1 -- 172.21.15.32:0/3603673372 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc004050 con 0x556dbb3c1390 2024-08-31T07:24:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.892+0000 7fbcdb400640 1 -- 172.21.15.32:0/3603673372 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc00fd40 con 0x556dbb3c1390 2024-08-31T07:24:21.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.892+0000 7fbcdb400640 1 -- 172.21.15.32:0/3603673372 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc005420 con 0x556dbb3c1390 2024-08-31T07:24:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.893+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/3603673372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 msgr2=0x556dbb78ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.893+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/3603673372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 0x556dbb78ec40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x556dbb6fe8e0 tx=0x7fbcdc005f90 comp rx=0 tx=0).stop 2024-08-31T07:24:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.893+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/3603673372 shutdown_connections 2024-08-31T07:24:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.893+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/3603673372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb7950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.893+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/3603673372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 0x556dbb78ec40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.893+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/3603673372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb78d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.893+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/3603673372 >> 172.21.15.32:0/3603673372 conn(0x556dbb3d6460 msgr2=0x556dbb7949b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.893+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/3603673372 shutdown_connections 2024-08-31T07:24:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.893+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/3603673372 wait complete. 2024-08-31T07:24:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.893+0000 7fbcfc9f2740 1 -- start start 2024-08-31T07:24:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcfc9f2740 1 Processor -- start 2024-08-31T07:24:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb7a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 0x556dbb7a7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcfc9f2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb808da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01ae3d0 con 0x556dbb3b91b0 2024-08-31T07:24:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01add00 con 0x556dbb3c1390 2024-08-31T07:24:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcfc9f2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01aaf60 con 0x556dbb3e2070 2024-08-31T07:24:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcfc9f2740 10 client.0 _ll_register_callbacks cb 0x7fbce8031ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb7a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb7a6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49374/0 (socket says 172.21.15.32:49374) 2024-08-31T07:24:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.894+0000 7fbcef400640 1 -- 172.21.15.32:0/444240811 learned_addr learned my addr 172.21.15.32:0/444240811 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcee000640 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 0x556dbb7a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcef400640 1 -- 172.21.15.32:0/444240811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 msgr2=0x556dbb808da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcef400640 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb808da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcef400640 1 -- 172.21.15.32:0/444240811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 msgr2=0x556dbb7a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcef400640 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 0x556dbb7a7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcef400640 1 -- 172.21.15.32:0/444240811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dbb809410 con 0x556dbb3b91b0 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcef400640 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb7a6f60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbce8030a10 tx=0x7fbce8004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d50a606de474b66 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcd8c00640 1 -- 172.21.15.32:0/444240811 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce804c040 con 0x556dbb3b91b0 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcd8c00640 1 -- 172.21.15.32:0/444240811 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7fbce802f860 con 0x556dbb3b91b0 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.895+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/444240811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcdc00fd40 con 0x556dbb3b91b0 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcd8c00640 1 -- 172.21.15.32:0/444240811 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbce806f020 con 0x556dbb3b91b0 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcfc9f2740 10 client.17946 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/444240811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556dbb810730 con 0x556dbb3b91b0 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcd8c00640 1 -- 172.21.15.32:0/444240811 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc00fd40 con 0x556dbb3b91b0 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcd8c00640 1 -- 172.21.15.32:0/444240811 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fbce8047480 con 0x556dbb3b91b0 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcfc9f2740 10 client.17946 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcfc9f2740 10 client.17946 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcfc9f2740 20 client.17946 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556dbb375da0 0x556dbb812e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcfc9f2740 4 client.17946 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:24:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.896+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/444240811 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556dbb6fc1a0 con 0x556dbb375da0 2024-08-31T07:24:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.897+0000 7fbcee000640 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556dbb375da0 0x556dbb812e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.897+0000 7fbcee000640 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556dbb375da0 0x556dbb812e10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=830040bb7f34fa9e server_cookie=5ad674317f744e95 in_seq=0 out_seq=0 2024-08-31T07:24:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.897+0000 7fbcd8c00640 10 client.17946 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcd8c00640 1 -- 172.21.15.32:0/444240811 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556dbb6fc1a0 con 0x556dbb375da0 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcd8c00640 10 client.17946 handle_command_reply: tid=0 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 client.17946 shutdown 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/444240811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 msgr2=0x556dbb7a6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb7a6f60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbce8030a10 tx=0x7fbce8004a80 comp rx=0 tx=0).stop 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/444240811 shutdown_connections 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x556dbb375da0 0x556dbb812e10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556dbb3e2070 0x556dbb808da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556dbb3c1390 0x556dbb7a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 --2- 172.21.15.32:0/444240811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556dbb3b91b0 0x556dbb7a6f60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcd8c00640 0 client.17946 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:24:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/444240811 >> 172.21.15.32:0/444240811 conn(0x556dbb3d6460 msgr2=0x556dbb788a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/444240811 shutdown_connections 2024-08-31T07:24:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 1 -- 172.21.15.32:0/444240811 wait complete. 2024-08-31T07:24:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.898+0000 7fbcfc9f2740 20 client.17946 trim_cache size 0 max 16384 2024-08-31T07:24:21.902 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 -- 172.21.15.32:0/907896755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd40b4ca0 msgr2=0x7fbcd40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd40b4ca0 0x7fbcd40b71b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x556dbb7866f0 tx=0x7fbce00eb530 comp rx=0 tx=0).stop 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 -- 172.21.15.32:0/907896755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf0103750 msgr2=0x7fbcf019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf0103750 0x7fbcf019a000 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbce000e9f0 tx=0x7fbce000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 -- 172.21.15.32:0/907896755 shutdown_connections 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd40b4ca0 0x7fbcd40b71b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf01099e0 0x7fbcf019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcf0109090 0x7fbcf019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 --2- 172.21.15.32:0/907896755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf0103750 0x7fbcf019a000 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 -- 172.21.15.32:0/907896755 >> 172.21.15.32:0/907896755 conn(0x7fbcf00ff1a0 msgr2=0x7fbcf01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.899+0000 7fbcefe00640 1 -- 172.21.15.32:0/907896755 shutdown_connections 2024-08-31T07:24:21.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:21.900+0000 7fbcefe00640 1 -- 172.21.15.32:0/907896755 wait complete. 2024-08-31T07:24:22.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:22 smithi032 ceph-mon[17252]: pgmap v728: 97 pgs: 97 active+clean; 274 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 352 KiB/s wr, 23 op/s 2024-08-31T07:24:22.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:22 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2303920398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:24:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:22 smithi073 ceph-mon[21995]: pgmap v728: 97 pgs: 97 active+clean; 274 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 352 KiB/s wr, 23 op/s 2024-08-31T07:24:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:22 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2303920398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:24:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:22 smithi084 ceph-mon[22387]: pgmap v728: 97 pgs: 97 active+clean; 274 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 352 KiB/s wr, 23 op/s 2024-08-31T07:24:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:22 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2303920398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:24:22.774 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.70 MB/sec execute 491 sec latency 24829.084 ms 2024-08-31T07:24:22.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:24:23.363 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 8e2b4950 --release --await' 2024-08-31T07:24:23.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 8e2b4950 --release --await 2024-08-31T07:24:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:23.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.69 MB/sec execute 492 sec latency 25829.122 ms 2024-08-31T07:24:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.131+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3040989292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcec109250 msgr2=0x7fbcec109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.131+0000 7fbcf1a00640 1 --2- 172.21.15.32:0/3040989292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcec109250 0x7fbcec109660 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009900 tx=0x7fbcd802a400 comp rx=0 tx=0).stop 2024-08-31T07:24:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.131+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3040989292 shutdown_connections 2024-08-31T07:24:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.131+0000 7fbcf1a00640 1 --2- 172.21.15.32:0/3040989292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec109ba0 0x7fbcec1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.131+0000 7fbcf1a00640 1 --2- 172.21.15.32:0/3040989292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcec109250 0x7fbcec109660 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.131+0000 7fbcf1a00640 1 --2- 172.21.15.32:0/3040989292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcec103910 0x7fbcec103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.131+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3040989292 >> 172.21.15.32:0/3040989292 conn(0x7fbcec0ff380 msgr2=0x7fbcec1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.131+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3040989292 shutdown_connections 2024-08-31T07:24:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.131+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3040989292 wait complete. 2024-08-31T07:24:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.132+0000 7fbcf1a00640 1 Processor -- start 2024-08-31T07:24:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.132+0000 7fbcf1a00640 1 -- start start 2024-08-31T07:24:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbcf1a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec103910 0x7fbcec19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbcf1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcec109250 0x7fbcec19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbcf1a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcec109ba0 0x7fbcec1a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbcf1a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1138b0 con 0x7fbcec109250 2024-08-31T07:24:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbcf1a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec113730 con 0x7fbcec103910 2024-08-31T07:24:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbcf1a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec113a30 con 0x7fbcec109ba0 2024-08-31T07:24:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbceaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec103910 0x7fbcec19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbceaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec103910 0x7fbcec19c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53028/0 (socket says 172.21.15.32:53028) 2024-08-31T07:24:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbceaa00640 1 -- 172.21.15.32:0/3138863972 learned_addr learned my addr 172.21.15.32:0/3138863972 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbcea000640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcec109250 0x7fbcec19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.133+0000 7fbceb400640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcec109ba0 0x7fbcec1a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.134+0000 7fbceaa00640 1 -- 172.21.15.32:0/3138863972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcec109ba0 msgr2=0x7fbcec1a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.134+0000 7fbceaa00640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcec109ba0 0x7fbcec1a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.134+0000 7fbceaa00640 1 -- 172.21.15.32:0/3138863972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcec109250 msgr2=0x7fbcec19ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.134+0000 7fbceaa00640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcec109250 0x7fbcec19ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.134+0000 7fbceaa00640 1 -- 172.21.15.32:0/3138863972 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec1a23d0 con 0x7fbcec103910 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.134+0000 7fbceaa00640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec103910 0x7fbcec19c460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00e9f0 tx=0x7fbcdc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.134+0000 7fbce3400640 1 -- 172.21.15.32:0/3138863972 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc00cd80 con 0x7fbcec103910 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.135+0000 7fbce3400640 1 -- 172.21.15.32:0/3138863972 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc0108e0 con 0x7fbcec103910 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.135+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3138863972 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec1a26c0 con 0x7fbcec103910 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.135+0000 7fbce3400640 1 -- 172.21.15.32:0/3138863972 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc018980 con 0x7fbcec103910 2024-08-31T07:24:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.135+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3138863972 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec1a8e20 con 0x7fbcec103910 2024-08-31T07:24:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.138+0000 7fbcf86a5740 1 Processor -- start 2024-08-31T07:24:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.138+0000 7fbce3400640 1 -- 172.21.15.32:0/3138863972 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbcdc010430 con 0x7fbcec103910 2024-08-31T07:24:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.138+0000 7fbcf86a5740 1 -- start start 2024-08-31T07:24:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x56548071e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548071eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 0x56548071f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5654806957b0 con 0x565480358390 2024-08-31T07:24:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x565480684d40 con 0x5654803501b0 2024-08-31T07:24:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x565480688cf0 con 0x565480379070 2024-08-31T07:24:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbceb400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x56548071e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbceb400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x56548071e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44530/0 (socket says 172.21.15.32:44530) 2024-08-31T07:24:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbceb400640 1 -- 172.21.15.32:0/153311044 learned_addr learned my addr 172.21.15.32:0/153311044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbceaa00640 1 --2- 172.21.15.32:0/153311044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548071eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.139+0000 7fbcea000640 1 --2- 172.21.15.32:0/153311044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 0x56548071f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.140+0000 7fbceaa00640 1 -- 172.21.15.32:0/153311044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 msgr2=0x56548071e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.140+0000 7fbceaa00640 1 --2- 172.21.15.32:0/153311044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x56548071e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.140+0000 7fbceaa00640 1 -- 172.21.15.32:0/153311044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 msgr2=0x56548071f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.140+0000 7fbceaa00640 1 --2- 172.21.15.32:0/153311044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 0x56548071f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.140+0000 7fbceaa00640 1 -- 172.21.15.32:0/153311044 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654803e4650 con 0x5654803501b0 2024-08-31T07:24:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.140+0000 7fbce3400640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd00b4ca0 0x7fbcd00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.140+0000 7fbceaa00640 1 --2- 172.21.15.32:0/153311044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548071eed0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0e3100 tx=0x7fbcdc0ed690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d74c97706dbea54 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.141+0000 7fbcd7e00640 1 -- 172.21.15.32:0/153311044 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0f4070 con 0x5654803501b0 2024-08-31T07:24:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.141+0000 7fbce3400640 1 -- 172.21.15.32:0/3138863972 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7fbcdc0e5960 con 0x7fbcec103910 2024-08-31T07:24:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.141+0000 7fbceb400640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd00b4ca0 0x7fbcd00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.141+0000 7fbcd7e00640 1 -- 172.21.15.32:0/153311044 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc010720 con 0x5654803501b0 2024-08-31T07:24:24.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.141+0000 7fbcd7e00640 1 -- 172.21.15.32:0/153311044 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0ab9f0 con 0x5654803501b0 2024-08-31T07:24:24.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.142+0000 7fbcf86a5740 1 -- 172.21.15.32:0/153311044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 msgr2=0x56548071eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:24.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.142+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/153311044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548071eed0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0e3100 tx=0x7fbcdc0ed690 comp rx=0 tx=0).stop 2024-08-31T07:24:24.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.142+0000 7fbceb400640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd00b4ca0 0x7fbcd00b71b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbce4003bc0 tx=0x7fbce4009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:24.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.142+0000 7fbcf86a5740 1 -- 172.21.15.32:0/153311044 shutdown_connections 2024-08-31T07:24:24.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.142+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/153311044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 0x56548071f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.142+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/153311044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548071eed0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.142+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/153311044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x56548071e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.142+0000 7fbcf86a5740 1 -- 172.21.15.32:0/153311044 >> 172.21.15.32:0/153311044 conn(0x56548036d460 msgr2=0x56548071c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:24.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.142+0000 7fbcf86a5740 1 -- 172.21.15.32:0/153311044 shutdown_connections 2024-08-31T07:24:24.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.142+0000 7fbcf86a5740 1 -- 172.21.15.32:0/153311044 wait complete. 2024-08-31T07:24:24.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.143+0000 7fbcf86a5740 1 -- start start 2024-08-31T07:24:24.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.144+0000 7fbcf86a5740 1 Processor -- start 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.144+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548072a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.144+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 0x56548072aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.144+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x565480736430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.144+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5654804e7700 con 0x565480358390 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.144+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56548060bcf0 con 0x5654803501b0 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.144+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56548072d110 con 0x565480379070 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.144+0000 7fbcf86a5740 10 client.0 _ll_register_callbacks cb 0x565480631470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbceaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548072a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbceaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548072a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53052/0 (socket says 172.21.15.32:53052) 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbceaa00640 1 -- 172.21.15.32:0/518882946 learned_addr learned my addr 172.21.15.32:0/518882946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbceb400640 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x565480736430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbcea000640 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 0x56548072aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbceaa00640 1 -- 172.21.15.32:0/518882946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 msgr2=0x565480736430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:24.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbceaa00640 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x565480736430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbceaa00640 1 -- 172.21.15.32:0/518882946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 msgr2=0x56548072aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:24.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbceaa00640 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 0x56548072aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbceaa00640 1 -- 172.21.15.32:0/518882946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56548064c8b0 con 0x5654803501b0 2024-08-31T07:24:24.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbcea000640 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 0x56548072aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:24.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbceaa00640 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548072a4e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0e30d0 tx=0x7fbcdc018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152021e68ca31773 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:24.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.145+0000 7fbcd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:24:24.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.146+0000 7fbcd4c00640 1 -- 172.21.15.32:0/518882946 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0f4030 con 0x5654803501b0 2024-08-31T07:24:24.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.146+0000 7fbcd4c00640 1 -- 172.21.15.32:0/518882946 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7fbcdc0ebc80 con 0x5654803501b0 2024-08-31T07:24:24.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.146+0000 7fbcf86a5740 1 -- 172.21.15.32:0/518882946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565480697700 con 0x5654803501b0 2024-08-31T07:24:24.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.146+0000 7fbcd4c00640 1 -- 172.21.15.32:0/518882946 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbcdc0ab810 con 0x5654803501b0 2024-08-31T07:24:24.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.146+0000 7fbcf86a5740 10 client.27953 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:24.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.146+0000 7fbcf86a5740 1 -- 172.21.15.32:0/518882946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x565480739100 con 0x5654803501b0 2024-08-31T07:24:24.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.146+0000 7fbcd4c00640 1 -- 172.21.15.32:0/518882946 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565480697700 con 0x5654803501b0 2024-08-31T07:24:24.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.146+0000 7fbcd4c00640 1 -- 172.21.15.32:0/518882946 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fbcdc01abe0 con 0x5654803501b0 2024-08-31T07:24:24.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.146+0000 7fbcf86a5740 10 client.27953 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:24.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.146+0000 7fbcf86a5740 10 client.27953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:24:24.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.147+0000 7fbcf86a5740 20 client.27953 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:24.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.147+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5654803379f0 0x56548073c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.147+0000 7fbcf86a5740 4 client.27953 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:24:24.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.147+0000 7fbcf86a5740 1 -- 172.21.15.32:0/518882946 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654806931a0 con 0x5654803379f0 2024-08-31T07:24:24.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.147+0000 7fbcea000640 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5654803379f0 0x56548073c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.148+0000 7fbcea000640 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5654803379f0 0x56548073c460 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cd0f817235cd10a server_cookie=e775f299c3406c6d in_seq=0 out_seq=0 2024-08-31T07:24:24.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.148+0000 7fbcd4c00640 10 client.27953 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.148+0000 7fbcd4c00640 1 -- 172.21.15.32:0/518882946 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5654806931a0 con 0x5654803379f0 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcd4c00640 10 client.27953 handle_command_reply: tid=0 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 client.27953 shutdown 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 -- 172.21.15.32:0/518882946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 msgr2=0x56548072a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548072a4e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0e30d0 tx=0x7fbcdc018420 comp rx=0 tx=0).stop 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 -- 172.21.15.32:0/518882946 shutdown_connections 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5654803379f0 0x56548073c460 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x565480736430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480358390 0x56548072aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcd4c00640 0 client.27953 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/518882946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654803501b0 0x56548072a4e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 -- 172.21.15.32:0/518882946 >> 172.21.15.32:0/518882946 conn(0x56548036d460 msgr2=0x565480721b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 -- 172.21.15.32:0/518882946 shutdown_connections 2024-08-31T07:24:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 1 -- 172.21.15.32:0/518882946 wait complete. 2024-08-31T07:24:24.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.149+0000 7fbcf86a5740 20 client.27953 trim_cache size 0 max 16384 2024-08-31T07:24:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.164+0000 7fbcf86a5740 1 Processor -- start 2024-08-31T07:24:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.164+0000 7fbcf86a5740 1 -- start start 2024-08-31T07:24:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654803501b0 0x56548071c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 0x5654807363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x5654807368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b1700 con 0x5654803501b0 2024-08-31T07:24:24.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b3270 con 0x565480358390 2024-08-31T07:24:24.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b1dd0 con 0x565480379070 2024-08-31T07:24:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcea000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 0x5654807363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbceaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654803501b0 0x56548071c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcea000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 0x5654807363a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53062/0 (socket says 172.21.15.32:53062) 2024-08-31T07:24:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcea000640 1 -- 172.21.15.32:0/2705310124 learned_addr learned my addr 172.21.15.32:0/2705310124 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbceb400640 1 --2- 172.21.15.32:0/2705310124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x5654807368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcea000640 1 -- 172.21.15.32:0/2705310124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 msgr2=0x5654807368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcea000640 1 --2- 172.21.15.32:0/2705310124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x5654807368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcea000640 1 -- 172.21.15.32:0/2705310124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654803501b0 msgr2=0x56548071c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.165+0000 7fbcea000640 1 --2- 172.21.15.32:0/2705310124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654803501b0 0x56548071c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.166+0000 7fbcea000640 1 -- 172.21.15.32:0/2705310124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56548071ccc0 con 0x565480358390 2024-08-31T07:24:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.166+0000 7fbcea000640 1 --2- 172.21.15.32:0/2705310124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 0x5654807363a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8000c00 tx=0x7fbcd8005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3eef0de8856e5b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.166+0000 7fbcd7400640 1 -- 172.21.15.32:0/2705310124 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8048020 con 0x565480358390 2024-08-31T07:24:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.166+0000 7fbcd7400640 1 -- 172.21.15.32:0/2705310124 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd8005450 con 0x565480358390 2024-08-31T07:24:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.166+0000 7fbcd7400640 1 -- 172.21.15.32:0/2705310124 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8035040 con 0x565480358390 2024-08-31T07:24:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.167+0000 7fbcf86a5740 1 -- 172.21.15.32:0/2705310124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 msgr2=0x5654807363a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.167+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/2705310124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 0x5654807363a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8000c00 tx=0x7fbcd8005a20 comp rx=0 tx=0).stop 2024-08-31T07:24:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.167+0000 7fbcf86a5740 1 -- 172.21.15.32:0/2705310124 shutdown_connections 2024-08-31T07:24:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.167+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/2705310124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x565480379070 0x5654807368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.167+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/2705310124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 0x5654807363a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.167+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/2705310124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654803501b0 0x56548071c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.167+0000 7fbcf86a5740 1 -- 172.21.15.32:0/2705310124 >> 172.21.15.32:0/2705310124 conn(0x56548036d460 msgr2=0x56548071c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.168+0000 7fbcf86a5740 1 -- 172.21.15.32:0/2705310124 shutdown_connections 2024-08-31T07:24:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.168+0000 7fbcf86a5740 1 -- 172.21.15.32:0/2705310124 wait complete. 2024-08-31T07:24:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.168+0000 7fbcf86a5740 1 -- start start 2024-08-31T07:24:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.168+0000 7fbcf86a5740 1 Processor -- start 2024-08-31T07:24:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654803501b0 0x56548073b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 0x56548073bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbcf86a5740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480379070 0x56548073c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b2b70 con 0x565480379070 2024-08-31T07:24:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b24a0 con 0x565480358390 2024-08-31T07:24:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbcf86a5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1af700 con 0x5654803501b0 2024-08-31T07:24:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbcf86a5740 10 client.0 _ll_register_callbacks cb 0x7fbcdc0eba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbceaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654803501b0 0x56548073b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbceb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480379070 0x56548073c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbceaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654803501b0 0x56548073b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44560/0 (socket says 172.21.15.32:44560) 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbceaa00640 1 -- 172.21.15.32:0/797349194 learned_addr learned my addr 172.21.15.32:0/797349194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.169+0000 7fbcea000640 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 0x56548073bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbceaa00640 1 -- 172.21.15.32:0/797349194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 msgr2=0x56548073bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbceaa00640 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 0x56548073bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbceaa00640 1 -- 172.21.15.32:0/797349194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480379070 msgr2=0x56548073c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbceaa00640 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480379070 0x56548073c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbceaa00640 1 -- 172.21.15.32:0/797349194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654807a5d30 con 0x5654803501b0 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbceb400640 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480379070 0x56548073c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbceaa00640 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654803501b0 0x56548073b5d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0e30d0 tx=0x7fbcdc0eb550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=586dc0ccb9640390 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbcd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:24:24.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbcd4c00640 1 -- 172.21.15.32:0/797349194 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0f4070 con 0x5654803501b0 2024-08-31T07:24:24.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbcd4c00640 1 -- 172.21.15.32:0/797349194 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7fbcdc0a8690 con 0x5654803501b0 2024-08-31T07:24:24.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbcf86a5740 1 -- 172.21.15.32:0/797349194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd8005450 con 0x5654803501b0 2024-08-31T07:24:24.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.170+0000 7fbcd4c00640 1 -- 172.21.15.32:0/797349194 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbcdc097030 con 0x5654803501b0 2024-08-31T07:24:24.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.171+0000 7fbcf86a5740 10 client.27967 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:24.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.171+0000 7fbcf86a5740 1 -- 172.21.15.32:0/797349194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5654807a6fb0 con 0x5654803501b0 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.171+0000 7fbcd4c00640 1 -- 172.21.15.32:0/797349194 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd8005450 con 0x5654803501b0 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.171+0000 7fbcd4c00640 1 -- 172.21.15.32:0/797349194 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fbcdc0971d0 con 0x5654803501b0 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.171+0000 7fbcf86a5740 10 client.27967 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.171+0000 7fbcf86a5740 10 client.27967 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.171+0000 7fbcf86a5740 20 client.27967 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.171+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56548030cda0 0x5654807a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.171+0000 7fbcf86a5740 4 client.27967 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.171+0000 7fbcf86a5740 1 -- 172.21.15.32:0/797349194 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654806908e0 con 0x56548030cda0 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.172+0000 7fbcea000640 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56548030cda0 0x5654807a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.172+0000 7fbcea000640 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56548030cda0 0x5654807a9520 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c18a8ad9571c84bd server_cookie=3a177c6ecf7783f9 in_seq=0 out_seq=0 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.172+0000 7fbcd4c00640 10 client.27967 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.172+0000 7fbcd4c00640 1 -- 172.21.15.32:0/797349194 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654806908e0 con 0x56548030cda0 2024-08-31T07:24:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.172+0000 7fbcd4c00640 10 client.27967 handle_command_reply: tid=0 2024-08-31T07:24:24.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 client.27967 shutdown 2024-08-31T07:24:24.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 -- 172.21.15.32:0/797349194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654803501b0 msgr2=0x56548073b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:24.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654803501b0 0x56548073b5d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0e30d0 tx=0x7fbcdc0eb550 comp rx=0 tx=0).stop 2024-08-31T07:24:24.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 -- 172.21.15.32:0/797349194 shutdown_connections 2024-08-31T07:24:24.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56548030cda0 0x5654807a9520 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565480379070 0x56548073c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565480358390 0x56548073bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 --2- 172.21.15.32:0/797349194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654803501b0 0x56548073b5d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 -- 172.21.15.32:0/797349194 >> 172.21.15.32:0/797349194 conn(0x56548036d460 msgr2=0x56548071ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcd4c00640 0 client.27967 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:24:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 -- 172.21.15.32:0/797349194 shutdown_connections 2024-08-31T07:24:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 1 -- 172.21.15.32:0/797349194 wait complete. 2024-08-31T07:24:24.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.173+0000 7fbcf86a5740 20 client.27967 trim_cache size 0 max 16384 2024-08-31T07:24:24.177 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:24:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3138863972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd00b4ca0 msgr2=0x7fbcd00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd00b4ca0 0x7fbcd00b71b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbce4003bc0 tx=0x7fbce4009500 comp rx=0 tx=0).stop 2024-08-31T07:24:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3138863972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec103910 msgr2=0x7fbcec19c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec103910 0x7fbcec19c460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00e9f0 tx=0x7fbcdc00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3138863972 shutdown_connections 2024-08-31T07:24:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbcd00b4ca0 0x7fbcd00b71b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcec109ba0 0x7fbcec1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcec109250 0x7fbcec19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 --2- 172.21.15.32:0/3138863972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec103910 0x7fbcec19c460 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3138863972 >> 172.21.15.32:0/3138863972 conn(0x7fbcec0ff380 msgr2=0x7fbcec106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3138863972 shutdown_connections 2024-08-31T07:24:24.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:24.174+0000 7fbcf1a00640 1 -- 172.21.15.32:0/3138863972 wait complete. 2024-08-31T07:24:24.196 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:24 smithi032 ceph-mon[17252]: pgmap v729: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 425 KiB/s wr, 27 op/s 2024-08-31T07:24:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:24 smithi073 ceph-mon[21995]: pgmap v729: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 425 KiB/s wr, 27 op/s 2024-08-31T07:24:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:24 smithi084 ceph-mon[22387]: pgmap v729: 97 pgs: 97 active+clean; 273 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 425 KiB/s wr, 27 op/s 2024-08-31T07:24:24.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349305 2.69 MB/sec execute 493 sec latency 26829.177 ms 2024-08-31T07:24:25.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:24:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.622+0000 7fb25be00640 1 -- 172.21.15.32:0/1228147923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c077120 msgr2=0x7fb25c075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.622+0000 7fb25be00640 1 --2- 172.21.15.32:0/1228147923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c077120 0x7fb25c075550 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009860 tx=0x7fb24c02a340 comp rx=0 tx=0).stop 2024-08-31T07:24:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.622+0000 7fb25be00640 1 -- 172.21.15.32:0/1228147923 shutdown_connections 2024-08-31T07:24:25.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.622+0000 7fb25be00640 1 --2- 172.21.15.32:0/1228147923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c114020 0x7fb25c1164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.622+0000 7fb25be00640 1 --2- 172.21.15.32:0/1228147923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c075b70 0x7fb25c075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.622+0000 7fb25be00640 1 --2- 172.21.15.32:0/1228147923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c077120 0x7fb25c075550 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.622+0000 7fb25be00640 1 -- 172.21.15.32:0/1228147923 >> 172.21.15.32:0/1228147923 conn(0x7fb25c0ff1e0 msgr2=0x7fb25c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.622+0000 7fb25be00640 1 -- 172.21.15.32:0/1228147923 shutdown_connections 2024-08-31T07:24:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.622+0000 7fb25be00640 1 -- 172.21.15.32:0/1228147923 wait complete. 2024-08-31T07:24:25.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.624+0000 7fb25be00640 1 Processor -- start 2024-08-31T07:24:25.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.624+0000 7fb25be00640 1 -- start start 2024-08-31T07:24:25.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c075b70 0x7fb25c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c077120 0x7fb25c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c114020 0x7fb25c1a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c119c60 con 0x7fb25c114020 2024-08-31T07:24:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c119ae0 con 0x7fb25c075b70 2024-08-31T07:24:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c119de0 con 0x7fb25c077120 2024-08-31T07:24:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c077120 0x7fb25c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c075b70 0x7fb25c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c077120 0x7fb25c1a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44578/0 (socket says 172.21.15.32:44578) 2024-08-31T07:24:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25a000640 1 -- 172.21.15.32:0/872667214 learned_addr learned my addr 172.21.15.32:0/872667214 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.625+0000 7fb25b400640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c114020 0x7fb25c1a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.626+0000 7fb25a000640 1 -- 172.21.15.32:0/872667214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c075b70 msgr2=0x7fb25c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.626+0000 7fb25a000640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c075b70 0x7fb25c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.626+0000 7fb25a000640 1 -- 172.21.15.32:0/872667214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c114020 msgr2=0x7fb25c1a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.626+0000 7fb25a000640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c114020 0x7fb25c1a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.626+0000 7fb25a000640 1 -- 172.21.15.32:0/872667214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c1a6600 con 0x7fb25c077120 2024-08-31T07:24:25.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.626+0000 7fb25a000640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c077120 0x7fb25c1a0c30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb24800e9f0 tx=0x7fb24800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:25.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.626+0000 7fb260a00640 1 -- 172.21.15.32:0/872667214 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24800ccc0 con 0x7fb25c077120 2024-08-31T07:24:25.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.626+0000 7fb260a00640 1 -- 172.21.15.32:0/872667214 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2480108e0 con 0x7fb25c077120 2024-08-31T07:24:25.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.626+0000 7fb25be00640 1 -- 172.21.15.32:0/872667214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25c1a68f0 con 0x7fb25c077120 2024-08-31T07:24:25.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.626+0000 7fb260a00640 1 -- 172.21.15.32:0/872667214 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248018710 con 0x7fb25c077120 2024-08-31T07:24:25.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.627+0000 7fb25be00640 1 -- 172.21.15.32:0/872667214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25c1ad050 con 0x7fb25c077120 2024-08-31T07:24:25.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.630+0000 7fb260a00640 1 -- 172.21.15.32:0/872667214 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb248010430 con 0x7fb25c077120 2024-08-31T07:24:25.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.631+0000 7fb268aac740 1 Processor -- start 2024-08-31T07:24:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.631+0000 7fb268aac740 1 -- start start 2024-08-31T07:24:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb268aac740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe34090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb268aac740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb268aac740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 0x555d7fe34b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb268aac740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555d7fdaaa90 con 0x555d7fa65660 2024-08-31T07:24:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb268aac740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555d7fd9a020 con 0x555d7fa6d840 2024-08-31T07:24:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb268aac740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555d7fd9dfd0 con 0x555d7fa8e520 2024-08-31T07:24:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb25b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe34090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb25a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 0x555d7fe34b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb25b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe34090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44584/0 (socket says 172.21.15.32:44584) 2024-08-31T07:24:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb25b400640 1 -- 172.21.15.32:0/233547725 learned_addr learned my addr 172.21.15.32:0/233547725 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.632+0000 7fb25aa00640 1 --2- 172.21.15.32:0/233547725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.633+0000 7fb25b400640 1 -- 172.21.15.32:0/233547725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 msgr2=0x555d7fe34b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.633+0000 7fb25b400640 1 --2- 172.21.15.32:0/233547725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 0x555d7fe34b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.633+0000 7fb25b400640 1 -- 172.21.15.32:0/233547725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 msgr2=0x555d7fe345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.633+0000 7fb25b400640 1 --2- 172.21.15.32:0/233547725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.633+0000 7fb25b400640 1 -- 172.21.15.32:0/233547725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d7faf9b00 con 0x555d7fa8e520 2024-08-31T07:24:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.633+0000 7fb260a00640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2400b4d20 0x7fb2400b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.633+0000 7fb260a00640 1 -- 172.21.15.32:0/872667214 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7fb2480e5740 con 0x7fb25c077120 2024-08-31T07:24:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.633+0000 7fb25b400640 1 --2- 172.21.15.32:0/233547725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe34090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb254011440 tx=0x7fb254011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bf821b30efac245 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.633+0000 7fb25aa00640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2400b4d20 0x7fb2400b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.633+0000 7fb247e00640 1 -- 172.21.15.32:0/233547725 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254003c30 con 0x555d7fa8e520 2024-08-31T07:24:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.634+0000 7fb247e00640 1 -- 172.21.15.32:0/233547725 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb254003dd0 con 0x555d7fa8e520 2024-08-31T07:24:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.634+0000 7fb247e00640 1 -- 172.21.15.32:0/233547725 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25400fae0 con 0x555d7fa8e520 2024-08-31T07:24:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.635+0000 7fb268aac740 1 -- 172.21.15.32:0/233547725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 msgr2=0x555d7fe34090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.635+0000 7fb25aa00640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2400b4d20 0x7fb2400b7230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb24c02a930 tx=0x7fb24c02c040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.635+0000 7fb268aac740 1 --2- 172.21.15.32:0/233547725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe34090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb254011440 tx=0x7fb254011900 comp rx=0 tx=0).stop 2024-08-31T07:24:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.635+0000 7fb268aac740 1 -- 172.21.15.32:0/233547725 shutdown_connections 2024-08-31T07:24:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.635+0000 7fb268aac740 1 --2- 172.21.15.32:0/233547725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 0x555d7fe34b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.635+0000 7fb268aac740 1 --2- 172.21.15.32:0/233547725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.635+0000 7fb268aac740 1 --2- 172.21.15.32:0/233547725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe34090 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.635+0000 7fb268aac740 1 -- 172.21.15.32:0/233547725 >> 172.21.15.32:0/233547725 conn(0x555d7fa82910 msgr2=0x555d7fe31f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.635+0000 7fb268aac740 1 -- 172.21.15.32:0/233547725 shutdown_connections 2024-08-31T07:24:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.635+0000 7fb268aac740 1 -- 172.21.15.32:0/233547725 wait complete. 2024-08-31T07:24:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.636+0000 7fb268aac740 1 -- start start 2024-08-31T07:24:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.636+0000 7fb268aac740 1 Processor -- start 2024-08-31T07:24:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.636+0000 7fb268aac740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa65660 0x555d7fe3f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.636+0000 7fb268aac740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 0x555d7fe3f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.636+0000 7fb268aac740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa8e520 0x555d7fe3fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.636+0000 7fb268aac740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555d7fbfcbb0 con 0x555d7fa8e520 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.636+0000 7fb268aac740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555d7fd20f90 con 0x555d7fa65660 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.636+0000 7fb268aac740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555d7fe42870 con 0x555d7fa6d840 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.636+0000 7fb268aac740 10 client.0 _ll_register_callbacks cb 0x555d7fd46710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.637+0000 7fb25b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa8e520 0x555d7fe3fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.637+0000 7fb25b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa8e520 0x555d7fe3fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49460/0 (socket says 172.21.15.32:49460) 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.637+0000 7fb25b400640 1 -- 172.21.15.32:0/517407625 learned_addr learned my addr 172.21.15.32:0/517407625 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.637+0000 7fb25aa00640 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa65660 0x555d7fe3f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.637+0000 7fb25a000640 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 0x555d7fe3f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.637+0000 7fb25aa00640 1 -- 172.21.15.32:0/517407625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 msgr2=0x555d7fe3f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.637+0000 7fb25aa00640 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 0x555d7fe3f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.637+0000 7fb25aa00640 1 -- 172.21.15.32:0/517407625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa8e520 msgr2=0x555d7fe3fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.637+0000 7fb25aa00640 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa8e520 0x555d7fe3fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.637+0000 7fb25aa00640 1 -- 172.21.15.32:0/517407625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d7fb70960 con 0x555d7fa65660 2024-08-31T07:24:25.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.638+0000 7fb25aa00640 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa65660 0x555d7fe3f450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb24c005940 tx=0x7fb24c02a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eb175b32e894eb6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:25.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.638+0000 7fb244c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:24:25.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.638+0000 7fb244c00640 1 -- 172.21.15.32:0/517407625 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c037040 con 0x555d7fa65660 2024-08-31T07:24:25.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.638+0000 7fb244c00640 1 -- 172.21.15.32:0/517407625 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7fb24c0035f0 con 0x555d7fa65660 2024-08-31T07:24:25.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.638+0000 7fb268aac740 1 -- 172.21.15.32:0/517407625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d7fd38e30 con 0x555d7fa65660 2024-08-31T07:24:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.638+0000 7fb244c00640 1 -- 172.21.15.32:0/517407625 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb24c057050 con 0x555d7fa65660 2024-08-31T07:24:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.638+0000 7fb268aac740 10 client.27971 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.638+0000 7fb268aac740 1 -- 172.21.15.32:0/517407625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555d7fe4e290 con 0x555d7fa65660 2024-08-31T07:24:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.639+0000 7fb244c00640 1 -- 172.21.15.32:0/517407625 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555d7fd38e30 con 0x555d7fa65660 2024-08-31T07:24:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.639+0000 7fb244c00640 1 -- 172.21.15.32:0/517407625 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fb24c03e7c0 con 0x555d7fa65660 2024-08-31T07:24:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.639+0000 7fb268aac740 10 client.27971 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.639+0000 7fb268aac740 10 client.27971 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:24:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.639+0000 7fb268aac740 20 client.27971 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.639+0000 7fb268aac740 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555d7fa4cea0 0x555d7fe51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.639+0000 7fb268aac740 4 client.27971 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:24:25.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.639+0000 7fb268aac740 1 -- 172.21.15.32:0/517407625 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d7fdac9e0 con 0x555d7fa4cea0 2024-08-31T07:24:25.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.640+0000 7fb25a000640 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555d7fa4cea0 0x555d7fe51840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.640+0000 7fb25a000640 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555d7fa4cea0 0x555d7fe51840 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16aa5323b5f0937 server_cookie=9ea9e3344c61f8f8 in_seq=0 out_seq=0 2024-08-31T07:24:25.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.640+0000 7fb244c00640 10 client.27971 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb244c00640 1 -- 172.21.15.32:0/517407625 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d7fdac9e0 con 0x555d7fa4cea0 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb244c00640 10 client.27971 handle_command_reply: tid=0 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb268aac740 1 client.27971 shutdown 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb268aac740 1 -- 172.21.15.32:0/517407625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa65660 msgr2=0x555d7fe3f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb268aac740 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa65660 0x555d7fe3f450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb24c005940 tx=0x7fb24c02a340 comp rx=0 tx=0).stop 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb268aac740 1 -- 172.21.15.32:0/517407625 shutdown_connections 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb268aac740 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555d7fa4cea0 0x555d7fe51840 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb268aac740 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa8e520 0x555d7fe3fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb268aac740 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 0x555d7fe3f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb268aac740 1 --2- 172.21.15.32:0/517407625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa65660 0x555d7fe3f450 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb268aac740 1 -- 172.21.15.32:0/517407625 >> 172.21.15.32:0/517407625 conn(0x555d7fa82910 msgr2=0x555d7fe37270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb244c00640 0 client.27971 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.641+0000 7fb268aac740 1 -- 172.21.15.32:0/517407625 shutdown_connections 2024-08-31T07:24:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.642+0000 7fb268aac740 1 -- 172.21.15.32:0/517407625 wait complete. 2024-08-31T07:24:25.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.642+0000 7fb268aac740 20 client.27971 trim_cache size 0 max 16384 2024-08-31T07:24:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.661+0000 7fb268aac740 1 Processor -- start 2024-08-31T07:24:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.661+0000 7fb268aac740 1 -- start start 2024-08-31T07:24:25.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.661+0000 7fb268aac740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe4ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.661+0000 7fb268aac740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 0x555d7fe3fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.661+0000 7fb268aac740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe3c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.661+0000 7fb268aac740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b57e0 con 0x555d7fa65660 2024-08-31T07:24:25.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.661+0000 7fb268aac740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b7350 con 0x555d7fa6d840 2024-08-31T07:24:25.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.661+0000 7fb268aac740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b5eb0 con 0x555d7fa8e520 2024-08-31T07:24:25.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.662+0000 7fb25a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 0x555d7fe3fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.662+0000 7fb25b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe3c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.662+0000 7fb25b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe3c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44598/0 (socket says 172.21.15.32:44598) 2024-08-31T07:24:25.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.662+0000 7fb25b400640 1 -- 172.21.15.32:0/1134945756 learned_addr learned my addr 172.21.15.32:0/1134945756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:25.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.662+0000 7fb25aa00640 1 --2- 172.21.15.32:0/1134945756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe4ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.662+0000 7fb25b400640 1 -- 172.21.15.32:0/1134945756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 msgr2=0x555d7fe3fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:25.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.662+0000 7fb25b400640 1 --2- 172.21.15.32:0/1134945756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 0x555d7fe3fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.662+0000 7fb25b400640 1 -- 172.21.15.32:0/1134945756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 msgr2=0x555d7fe4ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:25.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.663+0000 7fb25b400640 1 --2- 172.21.15.32:0/1134945756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe4ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.663+0000 7fb25b400640 1 -- 172.21.15.32:0/1134945756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d7fa4d0c0 con 0x555d7fa8e520 2024-08-31T07:24:25.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.663+0000 7fb25a000640 1 --2- 172.21.15.32:0/1134945756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 0x555d7fe3fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:25.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.663+0000 7fb25aa00640 1 --2- 172.21.15.32:0/1134945756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe4ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.664+0000 7fb25b400640 1 --2- 172.21.15.32:0/1134945756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe3c640 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb254014040 tx=0x7fb25400f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f57e5d84a06fe5b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.664+0000 7fb247400640 1 -- 172.21.15.32:0/1134945756 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254028070 con 0x555d7fa8e520 2024-08-31T07:24:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.664+0000 7fb247400640 1 -- 172.21.15.32:0/1134945756 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb25400f6f0 con 0x555d7fa8e520 2024-08-31T07:24:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.664+0000 7fb247400640 1 -- 172.21.15.32:0/1134945756 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2540052a0 con 0x555d7fa8e520 2024-08-31T07:24:25.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.665+0000 7fb268aac740 1 -- 172.21.15.32:0/1134945756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 msgr2=0x555d7fe3c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:25.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.665+0000 7fb268aac740 1 --2- 172.21.15.32:0/1134945756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe3c640 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb254014040 tx=0x7fb25400f680 comp rx=0 tx=0).stop 2024-08-31T07:24:25.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.665+0000 7fb268aac740 1 -- 172.21.15.32:0/1134945756 shutdown_connections 2024-08-31T07:24:25.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.665+0000 7fb268aac740 1 --2- 172.21.15.32:0/1134945756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa8e520 0x555d7fe3c640 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.665+0000 7fb268aac740 1 --2- 172.21.15.32:0/1134945756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa6d840 0x555d7fe3fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.665+0000 7fb268aac740 1 --2- 172.21.15.32:0/1134945756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe4ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.665+0000 7fb268aac740 1 -- 172.21.15.32:0/1134945756 >> 172.21.15.32:0/1134945756 conn(0x555d7fa82910 msgr2=0x555d7fe3f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.666+0000 7fb268aac740 1 -- 172.21.15.32:0/1134945756 shutdown_connections 2024-08-31T07:24:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.666+0000 7fb268aac740 1 -- 172.21.15.32:0/1134945756 wait complete. 2024-08-31T07:24:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.666+0000 7fb268aac740 1 -- start start 2024-08-31T07:24:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.666+0000 7fb268aac740 1 Processor -- start 2024-08-31T07:24:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb268aac740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb268aac740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 0x555d7fe44860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb268aac740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa8e520 0x555d7febb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb268aac740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b6c50 con 0x555d7fa65660 2024-08-31T07:24:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb268aac740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b6580 con 0x555d7fa8e520 2024-08-31T07:24:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb268aac740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b37e0 con 0x555d7fa6d840 2024-08-31T07:24:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb268aac740 10 client.0 _ll_register_callbacks cb 0x7fb24c030e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb25a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 0x555d7fe44860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb25a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 0x555d7fe44860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44602/0 (socket says 172.21.15.32:44602) 2024-08-31T07:24:25.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb25a000640 1 -- 172.21.15.32:0/745237636 learned_addr learned my addr 172.21.15.32:0/745237636 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:25.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb25aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.667+0000 7fb25b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa8e520 0x555d7febb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb25a000640 1 -- 172.21.15.32:0/745237636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa8e520 msgr2=0x555d7febb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb25a000640 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa8e520 0x555d7febb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb25a000640 1 -- 172.21.15.32:0/745237636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 msgr2=0x555d7fe442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb25a000640 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb25a000640 1 -- 172.21.15.32:0/745237636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d7febb840 con 0x555d7fa6d840 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb25a000640 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 0x555d7fe44860 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb2480a7ea0 tx=0x7fb2480abbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37267a3121a72108 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb244c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb244c00640 1 -- 172.21.15.32:0/745237636 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2480b0690 con 0x555d7fa6d840 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb244c00640 1 -- 172.21.15.32:0/745237636 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7fb2480a80b0 con 0x555d7fa6d840 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb268aac740 1 -- 172.21.15.32:0/745237636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb25400f6f0 con 0x555d7fa6d840 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb244c00640 1 -- 172.21.15.32:0/745237636 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb2480b5140 con 0x555d7fa6d840 2024-08-31T07:24:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb268aac740 10 client.27985 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.668+0000 7fb268aac740 1 -- 172.21.15.32:0/745237636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555d7febcb60 con 0x555d7fa6d840 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.669+0000 7fb244c00640 1 -- 172.21.15.32:0/745237636 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb25400f6f0 con 0x555d7fa6d840 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.669+0000 7fb244c00640 1 -- 172.21.15.32:0/745237636 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7fb2480930b0 con 0x555d7fa6d840 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.669+0000 7fb268aac740 10 client.27985 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.669+0000 7fb268aac740 10 client.27985 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.669+0000 7fb268aac740 20 client.27985 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.669+0000 7fb268aac740 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555d7fa22250 0x555d7febf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.669+0000 7fb268aac740 4 client.27985 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8e2b4950", "release": true, "await": true}] 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.669+0000 7fb268aac740 1 -- 172.21.15.32:0/745237636 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8e2b4950", "release": true, "await": true}) -- 0x555d7fda8480 con 0x555d7fa22250 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.670+0000 7fb25aa00640 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555d7fa22250 0x555d7febf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.670+0000 7fb25aa00640 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555d7fa22250 0x555d7febf330 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf902274a14ea682 server_cookie=879652029d6d52d8 in_seq=0 out_seq=0 2024-08-31T07:24:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.670+0000 7fb244c00640 10 client.27985 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.677+0000 7fb244c00640 1 -- 172.21.15.32:0/745237636 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555d7fda8480 con 0x555d7fa22250 2024-08-31T07:24:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb244c00640 10 client.27985 handle_command_reply: tid=0 2024-08-31T07:24:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb268aac740 1 client.27985 shutdown 2024-08-31T07:24:25.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb268aac740 1 -- 172.21.15.32:0/745237636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 msgr2=0x555d7fe44860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:25.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb268aac740 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 0x555d7fe44860 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb2480a7ea0 tx=0x7fb2480abbf0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb268aac740 1 -- 172.21.15.32:0/745237636 shutdown_connections 2024-08-31T07:24:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb268aac740 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555d7fa22250 0x555d7febf330 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb268aac740 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d7fa8e520 0x555d7febb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb268aac740 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555d7fa6d840 0x555d7fe44860 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb268aac740 1 --2- 172.21.15.32:0/745237636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d7fa65660 0x555d7fe442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb244c00640 0 client.27985 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.678+0000 7fb268aac740 1 -- 172.21.15.32:0/745237636 >> 172.21.15.32:0/745237636 conn(0x555d7fa82910 msgr2=0x555d7fe3fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.679+0000 7fb268aac740 1 -- 172.21.15.32:0/745237636 shutdown_connections 2024-08-31T07:24:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.679+0000 7fb268aac740 1 -- 172.21.15.32:0/745237636 wait complete. 2024-08-31T07:24:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.679+0000 7fb268aac740 20 client.27985 trim_cache size 0 max 16384 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 44, 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 40, 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "8e2b4950": { 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 40, 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:25.681 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 145.2, 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:24:25.682 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:24:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 -- 172.21.15.32:0/872667214 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2400b4d20 msgr2=0x7fb2400b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2400b4d20 0x7fb2400b7230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb24c02a930 tx=0x7fb24c02c040 comp rx=0 tx=0).stop 2024-08-31T07:24:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 -- 172.21.15.32:0/872667214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c077120 msgr2=0x7fb25c1a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c077120 0x7fb25c1a0c30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb24800e9f0 tx=0x7fb24800edf0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 -- 172.21.15.32:0/872667214 shutdown_connections 2024-08-31T07:24:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb2400b4d20 0x7fb2400b7230 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb25c114020 0x7fb25c1a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb25c077120 0x7fb25c1a0c30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 --2- 172.21.15.32:0/872667214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb25c075b70 0x7fb25c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 -- 172.21.15.32:0/872667214 >> 172.21.15.32:0/872667214 conn(0x7fb25c0ff1e0 msgr2=0x7fb25c10ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:25.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 -- 172.21.15.32:0/872667214 shutdown_connections 2024-08-31T07:24:25.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:25.680+0000 7fb25be00640 1 -- 172.21.15.32:0/872667214 wait complete. 2024-08-31T07:24:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:25.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349476 2.68 MB/sec execute 494 sec latency 27735.220 ms 2024-08-31T07:24:26.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:26 smithi032 ceph-mon[17252]: pgmap v730: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 426 KiB/s wr, 28 op/s 2024-08-31T07:24:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:26 smithi073 ceph-mon[21995]: pgmap v730: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 426 KiB/s wr, 28 op/s 2024-08-31T07:24:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:26 smithi084 ceph-mon[22387]: pgmap v730: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 426 KiB/s wr, 28 op/s 2024-08-31T07:24:26.621 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8e2b4950', total seconds elapsed: 32.0 2024-08-31T07:24:26.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 351371 2.70 MB/sec execute 495 sec latency 14.186 ms 2024-08-31T07:24:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.148+0000 7efc9cc00640 1 -- 172.21.15.32:0/2256375915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9810fe00 msgr2=0x7efc98112220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.148+0000 7efc9cc00640 1 --2- 172.21.15.32:0/2256375915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9810fe00 0x7efc98112220 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efc90009be0 tx=0x7efc9002a750 comp rx=0 tx=0).stop 2024-08-31T07:24:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.148+0000 7efc9cc00640 1 -- 172.21.15.32:0/2256375915 shutdown_connections 2024-08-31T07:24:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.148+0000 7efc9cc00640 1 --2- 172.21.15.32:0/2256375915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9810fe00 0x7efc98112220 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.148+0000 7efc9cc00640 1 --2- 172.21.15.32:0/2256375915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc98102ba0 0x7efc9810f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.148+0000 7efc9cc00640 1 --2- 172.21.15.32:0/2256375915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98102250 0x7efc98102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.148+0000 7efc9cc00640 1 -- 172.21.15.32:0/2256375915 >> 172.21.15.32:0/2256375915 conn(0x7efc980fe0d0 msgr2=0x7efc98100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.148+0000 7efc9cc00640 1 -- 172.21.15.32:0/2256375915 shutdown_connections 2024-08-31T07:24:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.148+0000 7efc9cc00640 1 -- 172.21.15.32:0/2256375915 wait complete. 2024-08-31T07:24:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.150+0000 7efc9cc00640 1 Processor -- start 2024-08-31T07:24:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.150+0000 7efc9cc00640 1 -- start start 2024-08-31T07:24:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.150+0000 7efc9cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98102250 0x7efc981082a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.150+0000 7efc9cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98102ba0 0x7efc98104580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.150+0000 7efc9cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc9810fe00 0x7efc98104ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.150+0000 7efc9cc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc98115870 con 0x7efc98102ba0 2024-08-31T07:24:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.150+0000 7efc9cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc981156f0 con 0x7efc98102250 2024-08-31T07:24:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.150+0000 7efc9cc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc981159f0 con 0x7efc9810fe00 2024-08-31T07:24:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc97400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98102250 0x7efc981082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc97400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98102250 0x7efc981082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58878/0 (socket says 172.21.15.32:58878) 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc97400640 1 -- 172.21.15.32:0/4107294564 learned_addr learned my addr 172.21.15.32:0/4107294564 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc96a00640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98102ba0 0x7efc98104580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc97e00640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc9810fe00 0x7efc98104ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc97400640 1 -- 172.21.15.32:0/4107294564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc9810fe00 msgr2=0x7efc98104ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc97400640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc9810fe00 0x7efc98104ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc97400640 1 -- 172.21.15.32:0/4107294564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98102ba0 msgr2=0x7efc98104580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc97400640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98102ba0 0x7efc98104580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc97400640 1 -- 172.21.15.32:0/4107294564 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc981ac750 con 0x7efc98102250 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.151+0000 7efc97e00640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc9810fe00 0x7efc98104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.152+0000 7efc97400640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98102250 0x7efc981082a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efc8800ea60 tx=0x7efc8800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:27.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.152+0000 7efc8fe00640 1 -- 172.21.15.32:0/4107294564 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8800cde0 con 0x7efc98102250 2024-08-31T07:24:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.152+0000 7efc9cc00640 1 -- 172.21.15.32:0/4107294564 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc981ac980 con 0x7efc98102250 2024-08-31T07:24:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.152+0000 7efc8fe00640 1 -- 172.21.15.32:0/4107294564 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc880035f0 con 0x7efc98102250 2024-08-31T07:24:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.152+0000 7efc8fe00640 1 -- 172.21.15.32:0/4107294564 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88010690 con 0x7efc98102250 2024-08-31T07:24:27.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.152+0000 7efc9cc00640 1 -- 172.21.15.32:0/4107294564 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc981acf10 con 0x7efc98102250 2024-08-31T07:24:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.155+0000 7efca4ecf740 1 Processor -- start 2024-08-31T07:24:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.155+0000 7efca4ecf740 1 -- start start 2024-08-31T07:24:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.155+0000 7efc8fe00640 1 -- 172.21.15.32:0/4107294564 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7efc880037a0 con 0x7efc98102250 2024-08-31T07:24:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af58070 0x55876b2fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af2f1b0 0x55876b2fe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b2fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efca4ecf740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55876b2747b0 con 0x55876af2f1b0 2024-08-31T07:24:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efca4ecf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55876b263d40 con 0x55876af37390 2024-08-31T07:24:27.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efca4ecf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55876b267cf0 con 0x55876af58070 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efc97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af58070 0x55876b2fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efc97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af58070 0x55876b2fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36630/0 (socket says 172.21.15.32:36630) 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efc97e00640 1 -- 172.21.15.32:0/4140297899 learned_addr learned my addr 172.21.15.32:0/4140297899 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efc97400640 1 --2- 172.21.15.32:0/4140297899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af2f1b0 0x55876b2fe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.156+0000 7efc96a00640 1 --2- 172.21.15.32:0/4140297899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b2fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.157+0000 7efc97e00640 1 -- 172.21.15.32:0/4140297899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 msgr2=0x55876b2fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.157+0000 7efc97e00640 1 --2- 172.21.15.32:0/4140297899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b2fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.157+0000 7efc97e00640 1 -- 172.21.15.32:0/4140297899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af2f1b0 msgr2=0x55876b2fe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.157+0000 7efc97e00640 1 --2- 172.21.15.32:0/4140297899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af2f1b0 0x55876b2fe0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.157+0000 7efc97e00640 1 -- 172.21.15.32:0/4140297899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876afc3650 con 0x55876af58070 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.157+0000 7efc96a00640 1 --2- 172.21.15.32:0/4140297899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b2fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.157+0000 7efc97e00640 1 --2- 172.21.15.32:0/4140297899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af58070 0x55876b2fdb60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efc9002ad40 tx=0x7efc90030420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2034c787515c665b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.158+0000 7efc83e00640 1 -- 172.21.15.32:0/4140297899 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90040030 con 0x55876af58070 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.158+0000 7efc83e00640 1 -- 172.21.15.32:0/4140297899 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc90003e20 con 0x55876af58070 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.158+0000 7efc83e00640 1 -- 172.21.15.32:0/4140297899 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90036040 con 0x55876af58070 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.158+0000 7efc8fe00640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc7c0b4d20 0x7efc7c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.158+0000 7efc8fe00640 1 -- 172.21.15.32:0/4107294564 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7efc880e58e0 con 0x7efc98102250 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.158+0000 7efc97400640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc7c0b4d20 0x7efc7c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.159+0000 7efca4ecf740 1 -- 172.21.15.32:0/4140297899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af58070 msgr2=0x55876b2fdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.159+0000 7efca4ecf740 1 --2- 172.21.15.32:0/4140297899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af58070 0x55876b2fdb60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efc9002ad40 tx=0x7efc90030420 comp rx=0 tx=0).stop 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.159+0000 7efca4ecf740 1 -- 172.21.15.32:0/4140297899 shutdown_connections 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.159+0000 7efca4ecf740 1 --2- 172.21.15.32:0/4140297899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b2fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.159+0000 7efca4ecf740 1 --2- 172.21.15.32:0/4140297899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af2f1b0 0x55876b2fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.159+0000 7efca4ecf740 1 --2- 172.21.15.32:0/4140297899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af58070 0x55876b2fdb60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.159+0000 7efca4ecf740 1 -- 172.21.15.32:0/4140297899 >> 172.21.15.32:0/4140297899 conn(0x55876af4c460 msgr2=0x55876b2fba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:27.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.159+0000 7efc97400640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc7c0b4d20 0x7efc7c0b7230 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55876b2fc7e0 tx=0x7efc880eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:27.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.159+0000 7efca4ecf740 1 -- 172.21.15.32:0/4140297899 shutdown_connections 2024-08-31T07:24:27.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.159+0000 7efca4ecf740 1 -- 172.21.15.32:0/4140297899 wait complete. 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.160+0000 7efca4ecf740 1 -- start start 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.160+0000 7efca4ecf740 1 Processor -- start 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.160+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af2f1b0 0x55876b309780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.160+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af37390 0x55876b309cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.160+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b315600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.160+0000 7efca4ecf740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55876b0c6700 con 0x55876af58070 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.160+0000 7efca4ecf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55876b1eacf0 con 0x55876af2f1b0 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.160+0000 7efca4ecf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55876b30c2e0 con 0x55876af37390 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efca4ecf740 10 client.0 _ll_register_callbacks cb 0x55876b210470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efc96a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af37390 0x55876b309cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efc97e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af2f1b0 0x55876b309780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efc96a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af37390 0x55876b309cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36632/0 (socket says 172.21.15.32:36632) 2024-08-31T07:24:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efc96a00640 1 -- 172.21.15.32:0/4010702708 learned_addr learned my addr 172.21.15.32:0/4010702708 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efc97e00640 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b315600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efc96a00640 1 -- 172.21.15.32:0/4010702708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af2f1b0 msgr2=0x55876b309780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efc96a00640 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af2f1b0 0x55876b309780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efc96a00640 1 -- 172.21.15.32:0/4010702708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 msgr2=0x55876b315600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efc96a00640 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b315600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.161+0000 7efc96a00640 1 -- 172.21.15.32:0/4010702708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876b22b8b0 con 0x55876af37390 2024-08-31T07:24:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.162+0000 7efc96a00640 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af37390 0x55876b309cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efc84012740 tx=0x7efc84012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ce6f36ac952dfb3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.162+0000 7efc80c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:24:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.162+0000 7efc80c00640 1 -- 172.21.15.32:0/4010702708 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8400f920 con 0x55876af37390 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.162+0000 7efca4ecf740 1 -- 172.21.15.32:0/4010702708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc90003e20 con 0x55876af37390 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.162+0000 7efc80c00640 1 -- 172.21.15.32:0/4010702708 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7efc8400fac0 con 0x55876af37390 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.162+0000 7efc80c00640 1 -- 172.21.15.32:0/4010702708 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7efc8400fcc0 con 0x55876af37390 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.162+0000 7efca4ecf740 10 client.27997 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.162+0000 7efca4ecf740 1 -- 172.21.15.32:0/4010702708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55876b3183e0 con 0x55876af37390 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.162+0000 7efc80c00640 1 -- 172.21.15.32:0/4010702708 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc90003e20 con 0x55876af37390 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.163+0000 7efc80c00640 1 -- 172.21.15.32:0/4010702708 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7efc84010430 con 0x55876af37390 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.163+0000 7efca4ecf740 10 client.27997 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.163+0000 7efca4ecf740 10 client.27997 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.163+0000 7efca4ecf740 20 client.27997 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.163+0000 7efca4ecf740 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55876af169f0 0x55876b31b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.163+0000 7efca4ecf740 4 client.27997 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.163+0000 7efca4ecf740 1 -- 172.21.15.32:0/4010702708 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55876b276700 con 0x55876af169f0 2024-08-31T07:24:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.163+0000 7efc97e00640 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55876af169f0 0x55876b31b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.164+0000 7efc97e00640 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55876af169f0 0x55876b31b8d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efd91c91ee64c05d server_cookie=caf15f7eada6b6b9 in_seq=0 out_seq=0 2024-08-31T07:24:27.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.164+0000 7efc80c00640 10 client.27997 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:24:27.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.164+0000 7efc80c00640 1 -- 172.21.15.32:0/4010702708 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55876b276700 con 0x55876af169f0 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.164+0000 7efc80c00640 10 client.27997 handle_command_reply: tid=0 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.164+0000 7efca4ecf740 1 client.27997 shutdown 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 1 -- 172.21.15.32:0/4010702708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af37390 msgr2=0x55876b309cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af37390 0x55876b309cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efc84012740 tx=0x7efc84012c00 comp rx=0 tx=0).stop 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 1 -- 172.21.15.32:0/4010702708 shutdown_connections 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b315600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af37390 0x55876b309cc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55876af169f0 0x55876b31b8d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 1 --2- 172.21.15.32:0/4010702708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af2f1b0 0x55876b309780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efc80c00640 0 client.27997 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 1 -- 172.21.15.32:0/4010702708 >> 172.21.15.32:0/4010702708 conn(0x55876af4c460 msgr2=0x55876b300cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 1 -- 172.21.15.32:0/4010702708 shutdown_connections 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 1 -- 172.21.15.32:0/4010702708 wait complete. 2024-08-31T07:24:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.165+0000 7efca4ecf740 20 client.27997 trim_cache size 0 max 16384 2024-08-31T07:24:27.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.180+0000 7efca4ecf740 1 Processor -- start 2024-08-31T07:24:27.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.180+0000 7efca4ecf740 1 -- start start 2024-08-31T07:24:27.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.180+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af2f1b0 0x55876b315500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.180+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b315a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.180+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b302c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.180+0000 7efca4ecf740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b57c0 con 0x55876af58070 2024-08-31T07:24:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.180+0000 7efca4ecf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b7330 con 0x55876af37390 2024-08-31T07:24:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.180+0000 7efca4ecf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b5e90 con 0x55876af2f1b0 2024-08-31T07:24:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc96a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b315a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af2f1b0 0x55876b315500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc96a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b315a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58896/0 (socket says 172.21.15.32:58896) 2024-08-31T07:24:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc96a00640 1 -- 172.21.15.32:0/1326275205 learned_addr learned my addr 172.21.15.32:0/1326275205 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:27.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc97e00640 1 --2- 172.21.15.32:0/1326275205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b302c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc96a00640 1 -- 172.21.15.32:0/1326275205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af2f1b0 msgr2=0x55876b315500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc96a00640 1 --2- 172.21.15.32:0/1326275205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af2f1b0 0x55876b315500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc96a00640 1 -- 172.21.15.32:0/1326275205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 msgr2=0x55876b302c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc96a00640 1 --2- 172.21.15.32:0/1326275205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b302c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc96a00640 1 -- 172.21.15.32:0/1326275205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876b3009c0 con 0x55876af37390 2024-08-31T07:24:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc97e00640 1 --2- 172.21.15.32:0/1326275205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b302c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.181+0000 7efc96a00640 1 --2- 172.21.15.32:0/1326275205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b315a40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efc84009530 tx=0x7efc84011f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d195ec84cd61d4eb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.182+0000 7efc83400640 1 -- 172.21.15.32:0/1326275205 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8400f920 con 0x55876af37390 2024-08-31T07:24:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.182+0000 7efc83400640 1 -- 172.21.15.32:0/1326275205 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc84012e50 con 0x55876af37390 2024-08-31T07:24:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.182+0000 7efc83400640 1 -- 172.21.15.32:0/1326275205 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840054f0 con 0x55876af37390 2024-08-31T07:24:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.183+0000 7efca4ecf740 1 -- 172.21.15.32:0/1326275205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 msgr2=0x55876b315a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.183+0000 7efca4ecf740 1 --2- 172.21.15.32:0/1326275205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b315a40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efc84009530 tx=0x7efc84011f20 comp rx=0 tx=0).stop 2024-08-31T07:24:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.183+0000 7efca4ecf740 1 -- 172.21.15.32:0/1326275205 shutdown_connections 2024-08-31T07:24:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.183+0000 7efca4ecf740 1 --2- 172.21.15.32:0/1326275205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b302c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.183+0000 7efca4ecf740 1 --2- 172.21.15.32:0/1326275205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b315a40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.183+0000 7efca4ecf740 1 --2- 172.21.15.32:0/1326275205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af2f1b0 0x55876b315500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.183+0000 7efca4ecf740 1 -- 172.21.15.32:0/1326275205 >> 172.21.15.32:0/1326275205 conn(0x55876af4c460 msgr2=0x55876b307e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.183+0000 7efca4ecf740 1 -- 172.21.15.32:0/1326275205 shutdown_connections 2024-08-31T07:24:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.183+0000 7efca4ecf740 1 -- 172.21.15.32:0/1326275205 wait complete. 2024-08-31T07:24:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.183+0000 7efca4ecf740 1 -- start start 2024-08-31T07:24:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.184+0000 7efca4ecf740 1 Processor -- start 2024-08-31T07:24:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.184+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af2f1b0 0x55876b311bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.184+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b30e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.184+0000 7efca4ecf740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b30e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.184+0000 7efca4ecf740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b6c30 con 0x55876af58070 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.184+0000 7efca4ecf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b6560 con 0x55876af37390 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.184+0000 7efca4ecf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b37c0 con 0x55876af2f1b0 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.184+0000 7efca4ecf740 10 client.0 _ll_register_callbacks cb 0x7efc84009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.185+0000 7efc96a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b30e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.185+0000 7efc97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af2f1b0 0x55876b311bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.185+0000 7efc96a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b30e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58904/0 (socket says 172.21.15.32:58904) 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.185+0000 7efc96a00640 1 -- 172.21.15.32:0/1196936697 learned_addr learned my addr 172.21.15.32:0/1196936697 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.185+0000 7efc97e00640 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b30e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.185+0000 7efc96a00640 1 -- 172.21.15.32:0/1196936697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af2f1b0 msgr2=0x55876b311bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.185+0000 7efc96a00640 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af2f1b0 0x55876b311bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.185+0000 7efc96a00640 1 -- 172.21.15.32:0/1196936697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 msgr2=0x55876b30e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.185+0000 7efc96a00640 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b30e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.185+0000 7efc96a00640 1 -- 172.21.15.32:0/1196936697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876b30ebe0 con 0x55876af37390 2024-08-31T07:24:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.186+0000 7efc96a00640 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b30e090 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efc84010970 tx=0x7efc84005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9cb4c69bce12069 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.186+0000 7efc80c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:24:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.186+0000 7efc80c00640 1 -- 172.21.15.32:0/1196936697 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8400fb20 con 0x55876af37390 2024-08-31T07:24:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.186+0000 7efc80c00640 1 -- 172.21.15.32:0/1196936697 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7efc84004020 con 0x55876af37390 2024-08-31T07:24:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.186+0000 7efca4ecf740 1 -- 172.21.15.32:0/1196936697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc84012e50 con 0x55876af37390 2024-08-31T07:24:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.186+0000 7efc80c00640 1 -- 172.21.15.32:0/1196936697 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7efc84004240 con 0x55876af37390 2024-08-31T07:24:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.186+0000 7efca4ecf740 10 client.27995 fetch_fsmap learned FSMap version 44 2024-08-31T07:24:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.186+0000 7efca4ecf740 1 -- 172.21.15.32:0/1196936697 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55876b387050 con 0x55876af37390 2024-08-31T07:24:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.186+0000 7efc80c00640 1 -- 172.21.15.32:0/1196936697 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc84012e50 con 0x55876af37390 2024-08-31T07:24:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.187+0000 7efc80c00640 1 -- 172.21.15.32:0/1196936697 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8366+0+0 (secure 0 0 0) 0x7efc84035070 con 0x55876af37390 2024-08-31T07:24:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.187+0000 7efca4ecf740 10 client.27995 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T07:24:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.187+0000 7efca4ecf740 10 client.27995 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:24:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.187+0000 7efca4ecf740 20 client.27995 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.187+0000 7efca4ecf740 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55876aeebda0 0x55876b3897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.187+0000 7efca4ecf740 4 client.27995 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.187+0000 7efca4ecf740 1 -- 172.21.15.32:0/1196936697 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55876b2721a0 con 0x55876aeebda0 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.187+0000 7efc97e00640 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55876aeebda0 0x55876b3897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efc97e00640 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55876aeebda0 0x55876b3897a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ba1a73c1550661d server_cookie=e31f0f370bc74601 in_seq=0 out_seq=0 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efc80c00640 10 client.27995 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efc80c00640 1 -- 172.21.15.32:0/1196936697 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55876b2721a0 con 0x55876aeebda0 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efc80c00640 10 client.27995 handle_command_reply: tid=0 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efca4ecf740 1 client.27995 shutdown 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efca4ecf740 1 -- 172.21.15.32:0/1196936697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 msgr2=0x55876b30e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efca4ecf740 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b30e090 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efc84010970 tx=0x7efc84005880 comp rx=0 tx=0).stop 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efca4ecf740 1 -- 172.21.15.32:0/1196936697 shutdown_connections 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efca4ecf740 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55876af58070 0x55876b30e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efca4ecf740 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55876aeebda0 0x55876b3897a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efca4ecf740 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55876af37390 0x55876b30e090 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efca4ecf740 1 --2- 172.21.15.32:0/1196936697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55876af2f1b0 0x55876b311bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efca4ecf740 1 -- 172.21.15.32:0/1196936697 >> 172.21.15.32:0/1196936697 conn(0x55876af4c460 msgr2=0x55876b307bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:27.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.188+0000 7efc80c00640 0 client.27995 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:24:27.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.189+0000 7efca4ecf740 1 -- 172.21.15.32:0/1196936697 shutdown_connections 2024-08-31T07:24:27.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.189+0000 7efca4ecf740 1 -- 172.21.15.32:0/1196936697 wait complete. 2024-08-31T07:24:27.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.189+0000 7efca4ecf740 20 client.27995 trim_cache size 0 max 16384 2024-08-31T07:24:27.192 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 -- 172.21.15.32:0/4107294564 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc7c0b4d20 msgr2=0x7efc7c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc7c0b4d20 0x7efc7c0b7230 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55876b2fc7e0 tx=0x7efc880eb530 comp rx=0 tx=0).stop 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 -- 172.21.15.32:0/4107294564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98102250 msgr2=0x7efc981082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98102250 0x7efc981082a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efc8800ea60 tx=0x7efc8800ef20 comp rx=0 tx=0).stop 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 -- 172.21.15.32:0/4107294564 shutdown_connections 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc7c0b4d20 0x7efc7c0b7230 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc9810fe00 0x7efc98104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98102ba0 0x7efc98104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 --2- 172.21.15.32:0/4107294564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98102250 0x7efc981082a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 -- 172.21.15.32:0/4107294564 >> 172.21.15.32:0/4107294564 conn(0x7efc980fe0d0 msgr2=0x7efc98109910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 -- 172.21.15.32:0/4107294564 shutdown_connections 2024-08-31T07:24:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:27.190+0000 7efc9cc00640 1 -- 172.21.15.32:0/4107294564 wait complete. 2024-08-31T07:24:27.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 352225 2.70 MB/sec execute 496 sec latency 61.632 ms 2024-08-31T07:24:28.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:24:28.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:28 smithi032 ceph-mon[17252]: pgmap v731: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 464 KiB/s wr, 35 op/s 2024-08-31T07:24:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:28 smithi073 ceph-mon[21995]: pgmap v731: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 464 KiB/s wr, 35 op/s 2024-08-31T07:24:28.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:28 smithi084 ceph-mon[22387]: pgmap v731: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 464 KiB/s wr, 35 op/s 2024-08-31T07:24:28.537 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:28.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 352850 2.70 MB/sec execute 497 sec latency 556.954 ms 2024-08-31T07:24:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.308+0000 7f1be7000640 1 -- 172.21.15.32:0/2442114601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be0103920 msgr2=0x7f1be0103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.308+0000 7f1be7000640 1 --2- 172.21.15.32:0/2442114601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be0103920 0x7f1be0103d30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009860 tx=0x7f1bd002a340 comp rx=0 tx=0).stop 2024-08-31T07:24:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.308+0000 7f1be7000640 1 -- 172.21.15.32:0/2442114601 shutdown_connections 2024-08-31T07:24:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.308+0000 7f1be7000640 1 --2- 172.21.15.32:0/2442114601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1be0109bb0 0x7f1be01103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.309+0000 7f1be7000640 1 --2- 172.21.15.32:0/2442114601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be0109260 0x7f1be0109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.309+0000 7f1be7000640 1 --2- 172.21.15.32:0/2442114601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be0103920 0x7f1be0103d30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.309+0000 7f1be7000640 1 -- 172.21.15.32:0/2442114601 >> 172.21.15.32:0/2442114601 conn(0x7f1be00ff3b0 msgr2=0x7f1be0101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.309+0000 7f1be7000640 1 -- 172.21.15.32:0/2442114601 shutdown_connections 2024-08-31T07:24:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.309+0000 7f1be7000640 1 -- 172.21.15.32:0/2442114601 wait complete. 2024-08-31T07:24:29.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.310+0000 7f1be7000640 1 Processor -- start 2024-08-31T07:24:29.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.310+0000 7f1be7000640 1 -- start start 2024-08-31T07:24:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.310+0000 7f1be7000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be0103920 0x7f1be019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.310+0000 7f1be7000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1be0109260 0x7f1be019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.310+0000 7f1be7000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be0109bb0 0x7f1be019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.310+0000 7f1be7000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0113870 con 0x7f1be0103920 2024-08-31T07:24:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.310+0000 7f1be7000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01136f0 con 0x7f1be0109260 2024-08-31T07:24:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.310+0000 7f1be7000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01139f0 con 0x7f1be0109bb0 2024-08-31T07:24:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be0103920 0x7f1be019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be0103920 0x7f1be019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37770/0 (socket says 172.21.15.32:37770) 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1bdfe00640 1 -- 172.21.15.32:0/745875844 learned_addr learned my addr 172.21.15.32:0/745875844 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1be5000640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be0109bb0 0x7f1be019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1bd7e00640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1be0109260 0x7f1be019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1be5000640 1 -- 172.21.15.32:0/745875844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1be0109260 msgr2=0x7f1be019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1be5000640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1be0109260 0x7f1be019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1be5000640 1 -- 172.21.15.32:0/745875844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be0103920 msgr2=0x7f1be019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1be5000640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be0103920 0x7f1be019a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1be5000640 1 -- 172.21.15.32:0/745875844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be01a0230 con 0x7f1be0109bb0 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.311+0000 7f1be5000640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be0109bb0 0x7f1be019fb00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009fd0 tx=0x7f1bd800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.312+0000 7f1bdd600640 1 -- 172.21.15.32:0/745875844 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd800cd60 con 0x7f1be0109bb0 2024-08-31T07:24:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.312+0000 7f1bdd600640 1 -- 172.21.15.32:0/745875844 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd8005020 con 0x7f1be0109bb0 2024-08-31T07:24:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.312+0000 7f1be7000640 1 -- 172.21.15.32:0/745875844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be01a0520 con 0x7f1be0109bb0 2024-08-31T07:24:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.312+0000 7f1bdd600640 1 -- 172.21.15.32:0/745875844 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8010690 con 0x7f1be0109bb0 2024-08-31T07:24:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.312+0000 7f1be7000640 1 -- 172.21.15.32:0/745875844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be0075390 con 0x7f1be0109bb0 2024-08-31T07:24:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.315+0000 7f1bedbd6740 1 Processor -- start 2024-08-31T07:24:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.315+0000 7f1bdd600640 1 -- 172.21.15.32:0/745875844 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1bd80054d0 con 0x7f1be0109bb0 2024-08-31T07:24:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.316+0000 7f1bedbd6740 1 -- start start 2024-08-31T07:24:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.316+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345a9d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.316+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345a9ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.316+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 0x558345a9e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.316+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558345a147b0 con 0x5583456d7390 2024-08-31T07:24:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.316+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558345a03d40 con 0x5583456cf1b0 2024-08-31T07:24:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.316+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558345a07cf0 con 0x5583456f8070 2024-08-31T07:24:29.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.316+0000 7f1bd7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345a9d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.316+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345a9ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.317+0000 7f1bd7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345a9d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36684/0 (socket says 172.21.15.32:36684) 2024-08-31T07:24:29.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.317+0000 7f1bd7e00640 1 -- 172.21.15.32:0/3897870933 learned_addr learned my addr 172.21.15.32:0/3897870933 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:29.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.317+0000 7f1bd7e00640 1 --2- 172.21.15.32:0/3897870933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 0x558345a9e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.317+0000 7f1bdfe00640 1 -- 172.21.15.32:0/3897870933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 msgr2=0x558345a9d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.317+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/3897870933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345a9d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.317+0000 7f1bdfe00640 1 -- 172.21.15.32:0/3897870933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 msgr2=0x558345a9e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.317+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/3897870933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 0x558345a9e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.317+0000 7f1bdfe00640 1 -- 172.21.15.32:0/3897870933 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558345763650 con 0x5583456cf1b0 2024-08-31T07:24:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.317+0000 7f1bd7e00640 1 --2- 172.21.15.32:0/3897870933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345a9d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.318+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/3897870933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345a9ded0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0000c00 tx=0x7f1bd0009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2964ae5e7c7cacd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.318+0000 7f1bdd600640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1bc40b4ca0 0x7f1bc40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.318+0000 7f1bdd600640 1 -- 172.21.15.32:0/745875844 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f1bd80e5b80 con 0x7f1be0109bb0 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.318+0000 7f1bd5600640 1 -- 172.21.15.32:0/3897870933 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0049070 con 0x5583456cf1b0 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.318+0000 7f1bd7e00640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1bc40b4ca0 0x7f1bc40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.318+0000 7f1bd5600640 1 -- 172.21.15.32:0/3897870933 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd0030c90 con 0x5583456cf1b0 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.318+0000 7f1bd5600640 1 -- 172.21.15.32:0/3897870933 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0037040 con 0x5583456cf1b0 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.319+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3897870933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 msgr2=0x558345a9ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.319+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/3897870933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345a9ded0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0000c00 tx=0x7f1bd0009490 comp rx=0 tx=0).stop 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.319+0000 7f1bd7e00640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1bc40b4ca0 0x7f1bc40b71b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009fd0 tx=0x7f1bc8009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.319+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3897870933 shutdown_connections 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.319+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/3897870933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 0x558345a9e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.319+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/3897870933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345a9ded0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.319+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/3897870933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345a9d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.319+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3897870933 >> 172.21.15.32:0/3897870933 conn(0x5583456ec460 msgr2=0x558345a9b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.319+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3897870933 shutdown_connections 2024-08-31T07:24:29.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.319+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3897870933 wait complete. 2024-08-31T07:24:29.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.320+0000 7f1bedbd6740 1 -- start start 2024-08-31T07:24:29.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.320+0000 7f1bedbd6740 1 Processor -- start 2024-08-31T07:24:29.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456cf1b0 0x558345aa9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456d7390 0x558345aa9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345ab5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558345866700 con 0x5583456cf1b0 2024-08-31T07:24:29.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55834598acf0 con 0x5583456d7390 2024-08-31T07:24:29.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558345aac110 con 0x5583456f8070 2024-08-31T07:24:29.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bedbd6740 10 client.0 _ll_register_callbacks cb 0x5583459b0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:29.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456cf1b0 0x558345aa9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bd7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456d7390 0x558345aa9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456cf1b0 0x558345aa9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37788/0 (socket says 172.21.15.32:37788) 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bd7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456d7390 0x558345aa9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58942/0 (socket says 172.21.15.32:58942) 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bdfe00640 1 -- 172.21.15.32:0/2489528830 learned_addr learned my addr 172.21.15.32:0/2489528830 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1be5000640 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345ab5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bdfe00640 1 -- 172.21.15.32:0/2489528830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 msgr2=0x558345ab5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345ab5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bdfe00640 1 -- 172.21.15.32:0/2489528830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456d7390 msgr2=0x558345aa9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456d7390 0x558345aa9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.321+0000 7f1bdfe00640 1 -- 172.21.15.32:0/2489528830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583459cb8b0 con 0x5583456cf1b0 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.322+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456cf1b0 0x558345aa9590 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0004270 tx=0x7f1bd00042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f047f2b94604deba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.322+0000 7f1bce000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:24:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.322+0000 7f1bce000640 1 -- 172.21.15.32:0/2489528830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0049030 con 0x5583456cf1b0 2024-08-31T07:24:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.322+0000 7f1bedbd6740 1 -- 172.21.15.32:0/2489528830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd0030c90 con 0x5583456cf1b0 2024-08-31T07:24:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.322+0000 7f1bce000640 1 -- 172.21.15.32:0/2489528830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f1bd0041040 con 0x5583456cf1b0 2024-08-31T07:24:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.323+0000 7f1bce000640 1 -- 172.21.15.32:0/2489528830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1bd0037040 con 0x5583456cf1b0 2024-08-31T07:24:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.323+0000 7f1bedbd6740 10 client.18012 fetch_fsmap learned FSMap version 45 2024-08-31T07:24:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.323+0000 7f1bedbd6740 1 -- 172.21.15.32:0/2489528830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558345ab8170 con 0x5583456cf1b0 2024-08-31T07:24:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.324+0000 7f1bce000640 1 -- 172.21.15.32:0/2489528830 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd0030c90 con 0x5583456cf1b0 2024-08-31T07:24:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.324+0000 7f1bce000640 1 -- 172.21.15.32:0/2489528830 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f1bd00035f0 con 0x5583456cf1b0 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.324+0000 7f1bedbd6740 10 client.18012 fetch_fsmap finished waiting for FSMap version 45 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.324+0000 7f1bedbd6740 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.324+0000 7f1bedbd6740 20 client.18012 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.324+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5583456b69f0 0x558345abb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.324+0000 7f1bedbd6740 4 client.18012 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.324+0000 7f1bedbd6740 1 -- 172.21.15.32:0/2489528830 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558345a16700 con 0x5583456b69f0 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.324+0000 7f1bd7e00640 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5583456b69f0 0x558345abb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.325+0000 7f1bd7e00640 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5583456b69f0 0x558345abb4d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeaaf940fafc16bd server_cookie=a25a7688b8d0d0ae in_seq=0 out_seq=0 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.325+0000 7f1bce000640 10 client.18012 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bce000640 1 -- 172.21.15.32:0/2489528830 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558345a16700 con 0x5583456b69f0 2024-08-31T07:24:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bce000640 10 client.18012 handle_command_reply: tid=0 2024-08-31T07:24:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 client.18012 shutdown 2024-08-31T07:24:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 -- 172.21.15.32:0/2489528830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456cf1b0 msgr2=0x558345aa9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456cf1b0 0x558345aa9590 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0004270 tx=0x7f1bd00042a0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 -- 172.21.15.32:0/2489528830 shutdown_connections 2024-08-31T07:24:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5583456b69f0 0x558345abb4d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345ab5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456d7390 0x558345aa9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/2489528830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456cf1b0 0x558345aa9590 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 -- 172.21.15.32:0/2489528830 >> 172.21.15.32:0/2489528830 conn(0x5583456ec460 msgr2=0x558345aa0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bce000640 0 client.18012 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 -- 172.21.15.32:0/2489528830 shutdown_connections 2024-08-31T07:24:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.326+0000 7f1bedbd6740 1 -- 172.21.15.32:0/2489528830 wait complete. 2024-08-31T07:24:29.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.327+0000 7f1bedbd6740 20 client.18012 trim_cache size 0 max 16384 2024-08-31T07:24:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bedbd6740 1 Processor -- start 2024-08-31T07:24:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bedbd6740 1 -- start start 2024-08-31T07:24:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345a9f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 0x558345a9ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345a9b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01ad130 con 0x5583456d7390 2024-08-31T07:24:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01aecd0 con 0x5583456cf1b0 2024-08-31T07:24:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01ad800 con 0x5583456f8070 2024-08-31T07:24:29.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bd7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 0x558345a9ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345a9f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.341+0000 7f1bd7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 0x558345a9ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37804/0 (socket says 172.21.15.32:37804) 2024-08-31T07:24:29.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1bd7e00640 1 -- 172.21.15.32:0/321952643 learned_addr learned my addr 172.21.15.32:0/321952643 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:29.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1be5000640 1 --2- 172.21.15.32:0/321952643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345a9b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1bdfe00640 1 -- 172.21.15.32:0/321952643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 msgr2=0x558345a9b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:29.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/321952643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345a9b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1bdfe00640 1 -- 172.21.15.32:0/321952643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 msgr2=0x558345a9ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:29.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/321952643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 0x558345a9ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1bdfe00640 1 -- 172.21.15.32:0/321952643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558345aaa650 con 0x5583456cf1b0 2024-08-31T07:24:29.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/321952643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345a9f0d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0005eb0 tx=0x7f1bd0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0e4db0e9e437683 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:29.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1bd4c00640 1 -- 172.21.15.32:0/321952643 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0049070 con 0x5583456cf1b0 2024-08-31T07:24:29.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1bd4c00640 1 -- 172.21.15.32:0/321952643 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd004b070 con 0x5583456cf1b0 2024-08-31T07:24:29.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.342+0000 7f1bd4c00640 1 -- 172.21.15.32:0/321952643 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0004310 con 0x5583456cf1b0 2024-08-31T07:24:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.343+0000 7f1bedbd6740 1 -- 172.21.15.32:0/321952643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 msgr2=0x558345a9f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.343+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/321952643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345a9f0d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0005eb0 tx=0x7f1bd0000bc0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.343+0000 7f1bedbd6740 1 -- 172.21.15.32:0/321952643 shutdown_connections 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.343+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/321952643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456f8070 0x558345a9b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.343+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/321952643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456d7390 0x558345a9ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.343+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/321952643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345a9f0d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.343+0000 7f1bedbd6740 1 -- 172.21.15.32:0/321952643 >> 172.21.15.32:0/321952643 conn(0x5583456ec460 msgr2=0x558345a9e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.344+0000 7f1bedbd6740 1 -- 172.21.15.32:0/321952643 shutdown_connections 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.344+0000 7f1bedbd6740 1 -- 172.21.15.32:0/321952643 wait complete. 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.344+0000 7f1bedbd6740 1 -- start start 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.344+0000 7f1bedbd6740 1 Processor -- start 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345abd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456d7390 0x558345abd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bedbd6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456f8070 0x558345b1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01ae5d0 con 0x5583456f8070 2024-08-31T07:24:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01aded0 con 0x5583456cf1b0 2024-08-31T07:24:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bedbd6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01ab130 con 0x5583456d7390 2024-08-31T07:24:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bedbd6740 10 client.0 _ll_register_callbacks cb 0x7f1bc8009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bd7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456d7390 0x558345abd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bd7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456d7390 0x558345abd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36712/0 (socket says 172.21.15.32:36712) 2024-08-31T07:24:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bd7e00640 1 -- 172.21.15.32:0/3274526400 learned_addr learned my addr 172.21.15.32:0/3274526400 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345abd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1be5000640 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456f8070 0x558345b1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bd7e00640 1 -- 172.21.15.32:0/3274526400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 msgr2=0x558345abd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bd7e00640 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345abd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bd7e00640 1 -- 172.21.15.32:0/3274526400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456f8070 msgr2=0x558345b1ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bd7e00640 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456f8070 0x558345b1ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1bd7e00640 1 -- 172.21.15.32:0/3274526400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558345b1f4c0 con 0x5583456d7390 2024-08-31T07:24:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.345+0000 7f1be5000640 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456f8070 0x558345b1ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.346+0000 7f1bd7e00640 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456d7390 0x558345abd6c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009530 tx=0x7f1bc8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898eff206fe5b98a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.346+0000 7f1bce000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:24:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.346+0000 7f1bce000640 1 -- 172.21.15.32:0/3274526400 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8039060 con 0x5583456d7390 2024-08-31T07:24:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.346+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3274526400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd004b070 con 0x5583456d7390 2024-08-31T07:24:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.346+0000 7f1bce000640 1 -- 172.21.15.32:0/3274526400 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f1bc8034440 con 0x5583456d7390 2024-08-31T07:24:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bce000640 1 -- 172.21.15.32:0/3274526400 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1bc8034640 con 0x5583456d7390 2024-08-31T07:24:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bedbd6740 10 client.28021 fetch_fsmap learned FSMap version 45 2024-08-31T07:24:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3274526400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558345b26990 con 0x5583456d7390 2024-08-31T07:24:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bce000640 1 -- 172.21.15.32:0/3274526400 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd004b070 con 0x5583456d7390 2024-08-31T07:24:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bce000640 1 -- 172.21.15.32:0/3274526400 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f1bc8022710 con 0x5583456d7390 2024-08-31T07:24:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bedbd6740 10 client.28021 fetch_fsmap finished waiting for FSMap version 45 2024-08-31T07:24:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bedbd6740 10 client.28021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:24:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bedbd6740 20 client.28021 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55834568bda0 0x558345b29190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bedbd6740 4 client.28021 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:24:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.347+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3274526400 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558345a121a0 con 0x55834568bda0 2024-08-31T07:24:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.348+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55834568bda0 0x558345b29190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.348+0000 7f1bdfe00640 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55834568bda0 0x558345b29190 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19104aab94ebb36f server_cookie=4f71349264a181ef in_seq=0 out_seq=0 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.348+0000 7f1bce000640 10 client.28021 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.348+0000 7f1bce000640 1 -- 172.21.15.32:0/3274526400 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558345a121a0 con 0x55834568bda0 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.348+0000 7f1bce000640 10 client.28021 handle_command_reply: tid=0 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.348+0000 7f1bedbd6740 1 client.28021 shutdown 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3274526400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456d7390 msgr2=0x558345abd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456d7390 0x558345abd6c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009530 tx=0x7f1bc8004730 comp rx=0 tx=0).stop 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3274526400 shutdown_connections 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55834568bda0 0x558345b29190 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5583456f8070 0x558345b1ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5583456d7390 0x558345abd6c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bce000640 0 client.28021 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 1 --2- 172.21.15.32:0/3274526400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583456cf1b0 0x558345abd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3274526400 >> 172.21.15.32:0/3274526400 conn(0x5583456ec460 msgr2=0x558345a9f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3274526400 shutdown_connections 2024-08-31T07:24:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 1 -- 172.21.15.32:0/3274526400 wait complete. 2024-08-31T07:24:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.349+0000 7f1bedbd6740 20 client.28021 trim_cache size 0 max 16384 2024-08-31T07:24:29.354 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:24:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 -- 172.21.15.32:0/745875844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1bc40b4ca0 msgr2=0x7f1bc40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1bc40b4ca0 0x7f1bc40b71b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009fd0 tx=0x7f1bc8009260 comp rx=0 tx=0).stop 2024-08-31T07:24:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 -- 172.21.15.32:0/745875844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be0109bb0 msgr2=0x7f1be019fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be0109bb0 0x7f1be019fb00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009fd0 tx=0x7f1bd800ee50 comp rx=0 tx=0).stop 2024-08-31T07:24:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 -- 172.21.15.32:0/745875844 shutdown_connections 2024-08-31T07:24:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1bc40b4ca0 0x7f1bc40b71b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be0109bb0 0x7f1be019fb00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1be0109260 0x7f1be019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 --2- 172.21.15.32:0/745875844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be0103920 0x7f1be019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 -- 172.21.15.32:0/745875844 >> 172.21.15.32:0/745875844 conn(0x7f1be00ff3b0 msgr2=0x7f1be01088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 -- 172.21.15.32:0/745875844 shutdown_connections 2024-08-31T07:24:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:29.350+0000 7f1be7000640 1 -- 172.21.15.32:0/745875844 wait complete. 2024-08-31T07:24:29.373 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:29 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:24:29.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:29 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:24:29.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:29 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:24:29.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 352850 2.70 MB/sec execute 498 sec latency 1556.998 ms 2024-08-31T07:24:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:30 smithi073 ceph-mon[21995]: pgmap v732: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 425 KiB/s wr, 41 op/s 2024-08-31T07:24:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:30 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:24:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:24:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:24:30.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:24:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:30 smithi084 ceph-mon[22387]: pgmap v732: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 425 KiB/s wr, 41 op/s 2024-08-31T07:24:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:30 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:24:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:24:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:24:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:24:30.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:30 smithi032 ceph-mon[17252]: pgmap v732: 97 pgs: 97 active+clean; 272 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 425 KiB/s wr, 41 op/s 2024-08-31T07:24:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:30 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:24:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:24:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:24:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:24:30.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 354171 2.71 MB/sec execute 499 sec latency 1907.231 ms 2024-08-31T07:24:31.174 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:24:31.174 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:24:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:31.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 354646 2.71 MB/sec execute 500 sec latency 838.622 ms 2024-08-31T07:24:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.263+0000 7f4dd7e00640 1 -- 172.21.15.32:0/2701307507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd8075b70 msgr2=0x7f4dd8075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.263+0000 7f4dd7e00640 1 --2- 172.21.15.32:0/2701307507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd8075b70 0x7f4dd8075f80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009900 tx=0x7f4dc402a400 comp rx=0 tx=0).stop 2024-08-31T07:24:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.264+0000 7f4dd7e00640 1 -- 172.21.15.32:0/2701307507 shutdown_connections 2024-08-31T07:24:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.264+0000 7f4dd7e00640 1 --2- 172.21.15.32:0/2701307507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd8114020 0x7f4dd81164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.264+0000 7f4dd7e00640 1 --2- 172.21.15.32:0/2701307507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd8075b70 0x7f4dd8075f80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.264+0000 7f4dd7e00640 1 --2- 172.21.15.32:0/2701307507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dd8077120 0x7f4dd8075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.264+0000 7f4dd7e00640 1 -- 172.21.15.32:0/2701307507 >> 172.21.15.32:0/2701307507 conn(0x7f4dd80ff1a0 msgr2=0x7f4dd81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.264+0000 7f4dd7e00640 1 -- 172.21.15.32:0/2701307507 shutdown_connections 2024-08-31T07:24:32.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.264+0000 7f4dd7e00640 1 -- 172.21.15.32:0/2701307507 wait complete. 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.265+0000 7f4dd7e00640 1 Processor -- start 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.265+0000 7f4dd7e00640 1 -- start start 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.265+0000 7f4dd7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dd8075b70 0x7f4dd81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.265+0000 7f4dd7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd8077120 0x7f4dd81a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.265+0000 7f4dd7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd8114020 0x7f4dd81a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.265+0000 7f4dd7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd8119c60 con 0x7f4dd8114020 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.265+0000 7f4dd7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd8119ae0 con 0x7f4dd8077120 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.265+0000 7f4dd7e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd8119de0 con 0x7f4dd8075b70 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.266+0000 7f4dd6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd8077120 0x7f4dd81a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.266+0000 7f4dd6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd8077120 0x7f4dd81a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58974/0 (socket says 172.21.15.32:58974) 2024-08-31T07:24:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.266+0000 7f4dd6000640 1 -- 172.21.15.32:0/3572094946 learned_addr learned my addr 172.21.15.32:0/3572094946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:32.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.266+0000 7f4dd7400640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd8114020 0x7f4dd81a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.266+0000 7f4dd6a00640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dd8075b70 0x7f4dd81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.266+0000 7f4dd6000640 1 -- 172.21.15.32:0/3572094946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dd8075b70 msgr2=0x7f4dd81a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:32.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.266+0000 7f4dd6000640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dd8075b70 0x7f4dd81a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.266+0000 7f4dd6000640 1 -- 172.21.15.32:0/3572094946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd8114020 msgr2=0x7f4dd81a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:32.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.266+0000 7f4dd6000640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd8114020 0x7f4dd81a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.266+0000 7f4dd6000640 1 -- 172.21.15.32:0/3572094946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd81a6620 con 0x7f4dd8077120 2024-08-31T07:24:32.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.267+0000 7f4dd6000640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd8077120 0x7f4dd81a0c50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4032fd0 tx=0x7f4dc4031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:32.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.267+0000 7f4dcf400640 1 -- 172.21.15.32:0/3572094946 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc403f030 con 0x7f4dd8077120 2024-08-31T07:24:32.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.267+0000 7f4dd7e00640 1 -- 172.21.15.32:0/3572094946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd81a68b0 con 0x7f4dd8077120 2024-08-31T07:24:32.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.267+0000 7f4dcf400640 1 -- 172.21.15.32:0/3572094946 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc40043d0 con 0x7f4dd8077120 2024-08-31T07:24:32.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.267+0000 7f4dcf400640 1 -- 172.21.15.32:0/3572094946 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4035070 con 0x7f4dd8077120 2024-08-31T07:24:32.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.267+0000 7f4dd7e00640 1 -- 172.21.15.32:0/3572094946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd81ad070 con 0x7f4dd8077120 2024-08-31T07:24:32.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.270+0000 7f4de47dc740 1 Processor -- start 2024-08-31T07:24:32.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4de47dc740 1 -- start start 2024-08-31T07:24:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 0x56285affe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac2f490 0x56285affed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac37670 0x56285afff250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4de47dc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56285af74750 con 0x56285ac58350 2024-08-31T07:24:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4de47dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56285af63ce0 con 0x56285ac2f490 2024-08-31T07:24:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4de47dc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56285af67c90 con 0x56285ac37670 2024-08-31T07:24:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4dcf400640 1 -- 172.21.15.32:0/3572094946 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4dc4042030 con 0x7f4dd8077120 2024-08-31T07:24:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4dd7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 0x56285affe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac2f490 0x56285affed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac2f490 0x56285affed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58984/0 (socket says 172.21.15.32:58984) 2024-08-31T07:24:32.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.271+0000 7f4dd6a00640 1 -- 172.21.15.32:0/2686591169 learned_addr learned my addr 172.21.15.32:0/2686591169 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.272+0000 7f4dd6000640 1 --2- 172.21.15.32:0/2686591169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac37670 0x56285afff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.272+0000 7f4dd6a00640 1 -- 172.21.15.32:0/2686591169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac37670 msgr2=0x56285afff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.272+0000 7f4dd6a00640 1 --2- 172.21.15.32:0/2686591169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac37670 0x56285afff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.272+0000 7f4dd6a00640 1 -- 172.21.15.32:0/2686591169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 msgr2=0x56285affe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.272+0000 7f4dd6a00640 1 --2- 172.21.15.32:0/2686591169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 0x56285affe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.272+0000 7f4dd6a00640 1 -- 172.21.15.32:0/2686591169 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56285acc3930 con 0x56285ac2f490 2024-08-31T07:24:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.272+0000 7f4dd6000640 1 --2- 172.21.15.32:0/2686591169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac37670 0x56285afff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.272+0000 7f4dd6a00640 1 --2- 172.21.15.32:0/2686591169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac2f490 0x56285affed10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009cf0 tx=0x7f4dc8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbcbc5860efe7f26 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.273+0000 7f4dc3e00640 1 -- 172.21.15.32:0/2686591169 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8004230 con 0x56285ac2f490 2024-08-31T07:24:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.273+0000 7f4dc3e00640 1 -- 172.21.15.32:0/2686591169 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc80043d0 con 0x56285ac2f490 2024-08-31T07:24:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.273+0000 7f4dc3e00640 1 -- 172.21.15.32:0/2686591169 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8019880 con 0x56285ac2f490 2024-08-31T07:24:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.273+0000 7f4dcf400640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4dbc0b4ca0 0x7f4dbc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.273+0000 7f4de47dc740 1 -- 172.21.15.32:0/2686591169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac2f490 msgr2=0x56285affed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.273+0000 7f4de47dc740 1 --2- 172.21.15.32:0/2686591169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac2f490 0x56285affed10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009cf0 tx=0x7f4dc8011810 comp rx=0 tx=0).stop 2024-08-31T07:24:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.273+0000 7f4dcf400640 1 -- 172.21.15.32:0/3572094946 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f4dc4105850 con 0x7f4dd8077120 2024-08-31T07:24:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.274+0000 7f4dd7400640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4dbc0b4ca0 0x7f4dbc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.274+0000 7f4de47dc740 1 -- 172.21.15.32:0/2686591169 shutdown_connections 2024-08-31T07:24:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.274+0000 7f4de47dc740 1 --2- 172.21.15.32:0/2686591169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac37670 0x56285afff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.274+0000 7f4de47dc740 1 --2- 172.21.15.32:0/2686591169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac2f490 0x56285affed10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.274+0000 7f4de47dc740 1 --2- 172.21.15.32:0/2686591169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 0x56285affe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.274+0000 7f4de47dc740 1 -- 172.21.15.32:0/2686591169 >> 172.21.15.32:0/2686591169 conn(0x56285ac4c740 msgr2=0x56285affc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.274+0000 7f4de47dc740 1 -- 172.21.15.32:0/2686591169 shutdown_connections 2024-08-31T07:24:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.274+0000 7f4de47dc740 1 -- 172.21.15.32:0/2686591169 wait complete. 2024-08-31T07:24:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.274+0000 7f4dd7400640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4dbc0b4ca0 0x7f4dbc0b71b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0005fd0 tx=0x7f4dd0009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.274+0000 7f4de47dc740 1 -- start start 2024-08-31T07:24:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.275+0000 7f4de47dc740 1 Processor -- start 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.275+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac2f490 0x56285b00a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.275+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 0x56285b00a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.275+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac58350 0x56285b0162a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.275+0000 7f4de47dc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56285adc6890 con 0x56285ac2f490 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.275+0000 7f4de47dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56285aeeacd0 con 0x56285ac37670 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.275+0000 7f4de47dc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56285b00cfb0 con 0x56285ac58350 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.275+0000 7f4de47dc740 10 client.0 _ll_register_callbacks cb 0x56285af5eef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac2f490 0x56285b00a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac2f490 0x56285b00a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37850/0 (socket says 172.21.15.32:37850) 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd6a00640 1 -- 172.21.15.32:0/1222645573 learned_addr learned my addr 172.21.15.32:0/1222645573 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd7400640 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac58350 0x56285b0162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd6000640 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 0x56285b00a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd6a00640 1 -- 172.21.15.32:0/1222645573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac58350 msgr2=0x56285b0162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:32.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd6a00640 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac58350 0x56285b0162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd6a00640 1 -- 172.21.15.32:0/1222645573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 msgr2=0x56285b00a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:32.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd6a00640 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 0x56285b00a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd6a00640 1 -- 172.21.15.32:0/1222645573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285af341b0 con 0x56285ac2f490 2024-08-31T07:24:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.276+0000 7f4dd6a00640 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac2f490 0x56285b00a400 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8019520 tx=0x7f4dc8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d22f19f0df5962 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.277+0000 7f4dc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:24:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.277+0000 7f4dc0c00640 1 -- 172.21.15.32:0/1222645573 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8027070 con 0x56285ac2f490 2024-08-31T07:24:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.277+0000 7f4dc0c00640 1 -- 172.21.15.32:0/1222645573 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f4dc8005240 con 0x56285ac2f490 2024-08-31T07:24:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.277+0000 7f4de47dc740 1 -- 172.21.15.32:0/1222645573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc80043d0 con 0x56285ac2f490 2024-08-31T07:24:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.277+0000 7f4dc0c00640 1 -- 172.21.15.32:0/1222645573 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4dc8033050 con 0x56285ac2f490 2024-08-31T07:24:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.277+0000 7f4de47dc740 10 client.18030 fetch_fsmap learned FSMap version 45 2024-08-31T07:24:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.277+0000 7f4de47dc740 1 -- 172.21.15.32:0/1222645573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56285b018fd0 con 0x56285ac2f490 2024-08-31T07:24:32.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.278+0000 7f4dc0c00640 1 -- 172.21.15.32:0/1222645573 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc80043d0 con 0x56285ac2f490 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.278+0000 7f4dc0c00640 1 -- 172.21.15.32:0/1222645573 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f4dc8004b70 con 0x56285ac2f490 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.278+0000 7f4de47dc740 10 client.18030 fetch_fsmap finished waiting for FSMap version 45 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.278+0000 7f4de47dc740 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.278+0000 7f4de47dc740 20 client.18030 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.278+0000 7f4de47dc740 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56285ac16cd0 0x56285b01c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.278+0000 7f4de47dc740 4 client.18030 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.278+0000 7f4de47dc740 1 -- 172.21.15.32:0/1222645573 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56285af766a0 con 0x56285ac16cd0 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.278+0000 7f4dd6000640 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56285ac16cd0 0x56285b01c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.279+0000 7f4dd6000640 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56285ac16cd0 0x56285b01c390 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67efcf2c42571993 server_cookie=70d25a0f40e5b2f4 in_seq=0 out_seq=0 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.279+0000 7f4dc0c00640 10 client.18030 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4dc0c00640 1 -- 172.21.15.32:0/1222645573 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56285af766a0 con 0x56285ac16cd0 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4dc0c00640 10 client.18030 handle_command_reply: tid=0 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 client.18030 shutdown 2024-08-31T07:24:32.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 -- 172.21.15.32:0/1222645573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac2f490 msgr2=0x56285b00a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac2f490 0x56285b00a400 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8019520 tx=0x7f4dc8003820 comp rx=0 tx=0).stop 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 -- 172.21.15.32:0/1222645573 shutdown_connections 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56285ac16cd0 0x56285b01c390 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac58350 0x56285b0162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 0x56285b00a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 --2- 172.21.15.32:0/1222645573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac2f490 0x56285b00a400 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 -- 172.21.15.32:0/1222645573 >> 172.21.15.32:0/1222645573 conn(0x56285ac4c740 msgr2=0x56285b0019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4dc0c00640 0 client.18030 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 -- 172.21.15.32:0/1222645573 shutdown_connections 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 1 -- 172.21.15.32:0/1222645573 wait complete. 2024-08-31T07:24:32.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.280+0000 7f4de47dc740 20 client.18030 trim_cache size 0 max 16384 2024-08-31T07:24:32.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.299+0000 7f4de47dc740 1 Processor -- start 2024-08-31T07:24:32.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.299+0000 7f4de47dc740 1 -- start start 2024-08-31T07:24:32.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 0x56285affff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac37670 0x56285b0038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac58350 0x56285b003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4de47dc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b5820 con 0x56285ac37670 2024-08-31T07:24:32.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4de47dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b73c0 con 0x56285ac58350 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4de47dc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b5ef0 con 0x56285ac2f490 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4dd6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac37670 0x56285b0038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4dd6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac37670 0x56285b0038b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37854/0 (socket says 172.21.15.32:37854) 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4dd6000640 1 -- 172.21.15.32:0/1777598412 learned_addr learned my addr 172.21.15.32:0/1777598412 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4dd7400640 1 --2- 172.21.15.32:0/1777598412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac58350 0x56285b003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4dd6a00640 1 --2- 172.21.15.32:0/1777598412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 0x56285affff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4dd7400640 1 -- 172.21.15.32:0/1777598412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 msgr2=0x56285affff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4dd7400640 1 --2- 172.21.15.32:0/1777598412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 0x56285affff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4dd7400640 1 -- 172.21.15.32:0/1777598412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac37670 msgr2=0x56285b0038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4dd7400640 1 --2- 172.21.15.32:0/1777598412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac37670 0x56285b0038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.300+0000 7f4dd7400640 1 -- 172.21.15.32:0/1777598412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56285b01c700 con 0x56285ac58350 2024-08-31T07:24:32.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.301+0000 7f4dd7400640 1 --2- 172.21.15.32:0/1777598412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac58350 0x56285b003df0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0012ea0 tx=0x7f4dd001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fd1dddf1735f096 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:32.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.301+0000 7f4dc3400640 1 -- 172.21.15.32:0/1777598412 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd002e070 con 0x56285ac58350 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.301+0000 7f4dc3400640 1 -- 172.21.15.32:0/1777598412 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd000fb10 con 0x56285ac58350 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.301+0000 7f4dc3400640 1 -- 172.21.15.32:0/1777598412 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd001eaf0 con 0x56285ac58350 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.302+0000 7f4de47dc740 1 -- 172.21.15.32:0/1777598412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac58350 msgr2=0x56285b003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.302+0000 7f4de47dc740 1 --2- 172.21.15.32:0/1777598412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac58350 0x56285b003df0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0012ea0 tx=0x7f4dd001b000 comp rx=0 tx=0).stop 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.302+0000 7f4de47dc740 1 -- 172.21.15.32:0/1777598412 shutdown_connections 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.302+0000 7f4de47dc740 1 --2- 172.21.15.32:0/1777598412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac58350 0x56285b003df0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.302+0000 7f4de47dc740 1 --2- 172.21.15.32:0/1777598412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac37670 0x56285b0038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.302+0000 7f4de47dc740 1 --2- 172.21.15.32:0/1777598412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 0x56285affff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.302+0000 7f4de47dc740 1 -- 172.21.15.32:0/1777598412 >> 172.21.15.32:0/1777598412 conn(0x56285ac4c740 msgr2=0x56285afff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.302+0000 7f4de47dc740 1 -- 172.21.15.32:0/1777598412 shutdown_connections 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.303+0000 7f4de47dc740 1 -- 172.21.15.32:0/1777598412 wait complete. 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.303+0000 7f4de47dc740 1 -- start start 2024-08-31T07:24:32.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.303+0000 7f4de47dc740 1 Processor -- start 2024-08-31T07:24:32.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 0x56285b00f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 0x56285b00f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4de47dc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 0x56285b00fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4de47dc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b6cc0 con 0x56285ac58350 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4de47dc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b65c0 con 0x56285ac37670 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4de47dc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b3820 con 0x56285ac2f490 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4de47dc740 10 client.0 _ll_register_callbacks cb 0x7f4dd0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 0x56285b00f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4dd6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 0x56285b00f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36784/0 (socket says 172.21.15.32:36784) 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4dd6a00640 1 -- 172.21.15.32:0/2789597341 learned_addr learned my addr 172.21.15.32:0/2789597341 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4dd6000640 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 0x56285b00f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.304+0000 7f4dd7400640 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 0x56285b00fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4dd6000640 1 -- 172.21.15.32:0/2789597341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 msgr2=0x56285b00f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4dd6000640 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 0x56285b00f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4dd6000640 1 -- 172.21.15.32:0/2789597341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 msgr2=0x56285b00fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4dd6000640 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 0x56285b00fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4dd6000640 1 -- 172.21.15.32:0/2789597341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285b086d50 con 0x56285ac37670 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4dd7400640 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 0x56285b00fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4dd6000640 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 0x56285b00f6b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40c91a0 tx=0x7f4dc402a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98ad982cf70f2a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4dc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4dc0c00640 1 -- 172.21.15.32:0/2789597341 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4049040 con 0x56285ac37670 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4dc0c00640 1 -- 172.21.15.32:0/2789597341 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f4dc4103210 con 0x56285ac37670 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.305+0000 7f4de47dc740 1 -- 172.21.15.32:0/2789597341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd000fb10 con 0x56285ac37670 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4dc0c00640 1 -- 172.21.15.32:0/2789597341 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4dc4103410 con 0x56285ac37670 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4de47dc740 10 client.28031 fetch_fsmap learned FSMap version 45 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4de47dc740 1 -- 172.21.15.32:0/2789597341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56285b087fa0 con 0x56285ac37670 2024-08-31T07:24:32.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4dc0c00640 1 -- 172.21.15.32:0/2789597341 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd000fb10 con 0x56285ac37670 2024-08-31T07:24:32.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4dc0c00640 1 -- 172.21.15.32:0/2789597341 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f4dc40d0410 con 0x56285ac37670 2024-08-31T07:24:32.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4de47dc740 10 client.28031 fetch_fsmap finished waiting for FSMap version 45 2024-08-31T07:24:32.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4de47dc740 10 client.28031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:24:32.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4de47dc740 20 client.28031 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:32.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4de47dc740 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56285abec080 0x56285b08a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:32.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4de47dc740 4 client.28031 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:24:32.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.306+0000 7f4de47dc740 1 -- 172.21.15.32:0/2789597341 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56285af72140 con 0x56285abec080 2024-08-31T07:24:32.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.307+0000 7f4dd6a00640 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56285abec080 0x56285b08a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:32.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.307+0000 7f4dd6a00640 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56285abec080 0x56285b08a600 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5dbefef9d5eb80e server_cookie=6ef019e36ea8a04d in_seq=0 out_seq=0 2024-08-31T07:24:32.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.308+0000 7f4dc0c00640 10 client.28031 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:32.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.310+0000 7f4dc0c00640 1 -- 172.21.15.32:0/2789597341 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56285af72140 con 0x56285abec080 2024-08-31T07:24:32.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.310+0000 7f4dc0c00640 10 client.28031 handle_command_reply: tid=0 2024-08-31T07:24:32.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.310+0000 7f4de47dc740 1 client.28031 shutdown 2024-08-31T07:24:32.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.310+0000 7f4de47dc740 1 -- 172.21.15.32:0/2789597341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 msgr2=0x56285b00f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:32.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.310+0000 7f4de47dc740 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 0x56285b00f6b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40c91a0 tx=0x7f4dc402a400 comp rx=0 tx=0).stop 2024-08-31T07:24:32.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.311+0000 7f4de47dc740 1 -- 172.21.15.32:0/2789597341 shutdown_connections 2024-08-31T07:24:32.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.311+0000 7f4de47dc740 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56285abec080 0x56285b08a600 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.311+0000 7f4de47dc740 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56285ac58350 0x56285b00fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.311+0000 7f4de47dc740 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56285ac37670 0x56285b00f6b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.311+0000 7f4de47dc740 1 --2- 172.21.15.32:0/2789597341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285ac2f490 0x56285b00f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.311+0000 7f4de47dc740 1 -- 172.21.15.32:0/2789597341 >> 172.21.15.32:0/2789597341 conn(0x56285ac4c740 msgr2=0x56285b0001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.311+0000 7f4dc0c00640 0 client.28031 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.311+0000 7f4de47dc740 1 -- 172.21.15.32:0/2789597341 shutdown_connections 2024-08-31T07:24:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.311+0000 7f4de47dc740 1 -- 172.21.15.32:0/2789597341 wait complete. 2024-08-31T07:24:32.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.311+0000 7f4de47dc740 20 client.28031 trim_cache size 0 max 16384 2024-08-31T07:24:32.313 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:24:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:24:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "96b7006d-efb1-4838-bb2b-531e42fd1874", 2024-08-31T07:24:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:24:32.313 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:24:32.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.312+0000 7f4dd7e00640 1 -- 172.21.15.32:0/3572094946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4dbc0b4ca0 msgr2=0x7f4dbc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:32.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.312+0000 7f4dd7e00640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4dbc0b4ca0 0x7f4dbc0b71b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0005fd0 tx=0x7f4dd0009500 comp rx=0 tx=0).stop 2024-08-31T07:24:32.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.312+0000 7f4dd7e00640 1 -- 172.21.15.32:0/3572094946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd8077120 msgr2=0x7f4dd81a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:32.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.312+0000 7f4dd7e00640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd8077120 0x7f4dd81a0c50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4032fd0 tx=0x7f4dc4031a20 comp rx=0 tx=0).stop 2024-08-31T07:24:32.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.313+0000 7f4dd7e00640 1 -- 172.21.15.32:0/3572094946 shutdown_connections 2024-08-31T07:24:32.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.313+0000 7f4dd7e00640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4dbc0b4ca0 0x7f4dbc0b71b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.313+0000 7f4dd7e00640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4dd8114020 0x7f4dd81a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.313+0000 7f4dd7e00640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd8077120 0x7f4dd81a0c50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.313+0000 7f4dd7e00640 1 --2- 172.21.15.32:0/3572094946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dd8075b70 0x7f4dd81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:32.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.313+0000 7f4dd7e00640 1 -- 172.21.15.32:0/3572094946 >> 172.21.15.32:0/3572094946 conn(0x7f4dd80ff1a0 msgr2=0x7f4dd810ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:32.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.314+0000 7f4dd7e00640 1 -- 172.21.15.32:0/3572094946 shutdown_connections 2024-08-31T07:24:32.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:32.315+0000 7f4dd7e00640 1 -- 172.21.15.32:0/3572094946 wait complete. 2024-08-31T07:24:32.348 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:32 smithi032 ceph-mon[17252]: pgmap v733: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 720 KiB/s wr, 69 op/s 2024-08-31T07:24:32.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:32 smithi073 ceph-mon[21995]: pgmap v733: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 720 KiB/s wr, 69 op/s 2024-08-31T07:24:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:32 smithi084 ceph-mon[22387]: pgmap v733: 97 pgs: 97 active+clean; 276 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 720 KiB/s wr, 69 op/s 2024-08-31T07:24:32.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 354646 2.71 MB/sec execute 501 sec latency 1838.768 ms 2024-08-31T07:24:33.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:24:33.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:24:33.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:24:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:24:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:24:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:24:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:24:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:24:33.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:33 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:24:33.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:24:33.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:33 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:24:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:24:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:24:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:24:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:24:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:24:33.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:24:33.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:33 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:33.656 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:24:33.775 INFO:tasks.workunit.client.0.smithi032.stdout: 1 354646 2.70 MB/sec execute 502 sec latency 2839.000 ms 2024-08-31T07:24:34.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.361+0000 7f8cdaa00640 1 -- 172.21.15.32:0/3129516236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cd410b290 msgr2=0x7f8cd410b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:34.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.361+0000 7f8cdaa00640 1 --2- 172.21.15.32:0/3129516236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cd410b290 0x7f8cd410b6a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009900 tx=0x7f8cc002a410 comp rx=0 tx=0).stop 2024-08-31T07:24:34.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.361+0000 7f8cdaa00640 1 -- 172.21.15.32:0/3129516236 shutdown_connections 2024-08-31T07:24:34.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.361+0000 7f8cdaa00640 1 --2- 172.21.15.32:0/3129516236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cd410bbe0 0x7f8cd4112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.361+0000 7f8cdaa00640 1 --2- 172.21.15.32:0/3129516236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cd410b290 0x7f8cd410b6a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.361+0000 7f8cdaa00640 1 --2- 172.21.15.32:0/3129516236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd4105950 0x7f8cd4105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.361+0000 7f8cdaa00640 1 -- 172.21.15.32:0/3129516236 >> 172.21.15.32:0/3129516236 conn(0x7f8cd4101420 msgr2=0x7f8cd4103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:34.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.361+0000 7f8cdaa00640 1 -- 172.21.15.32:0/3129516236 shutdown_connections 2024-08-31T07:24:34.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.362+0000 7f8cdaa00640 1 -- 172.21.15.32:0/3129516236 wait complete. 2024-08-31T07:24:34.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.362+0000 7f8cdaa00640 1 Processor -- start 2024-08-31T07:24:34.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.363+0000 7f8cdaa00640 1 -- start start 2024-08-31T07:24:34.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.363+0000 7f8cdaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cd4105950 0x7f8cd419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.363+0000 7f8cdaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cd410b290 0x7f8cd419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.363+0000 7f8cdaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd410bbe0 0x7f8cd41a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.363+0000 7f8cdaa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41158a0 con 0x7f8cd4105950 2024-08-31T07:24:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.363+0000 7f8cdaa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4115720 con 0x7f8cd410bbe0 2024-08-31T07:24:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.363+0000 7f8cdaa00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4115a20 con 0x7f8cd410b290 2024-08-31T07:24:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.363+0000 7f8cd9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cd4105950 0x7f8cd419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.363+0000 7f8cd9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cd4105950 0x7f8cd419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37892/0 (socket says 172.21.15.32:37892) 2024-08-31T07:24:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.363+0000 7f8cd9600640 1 -- 172.21.15.32:0/2660502754 learned_addr learned my addr 172.21.15.32:0/2660502754 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.364+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cd410b290 0x7f8cd419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.364+0000 7f8cda000640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd410bbe0 0x7f8cd41a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.364+0000 7f8cd9600640 1 -- 172.21.15.32:0/2660502754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cd410b290 msgr2=0x7f8cd419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:34.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.364+0000 7f8cd9600640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cd410b290 0x7f8cd419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.364+0000 7f8cd9600640 1 -- 172.21.15.32:0/2660502754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd410bbe0 msgr2=0x7f8cd41a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:34.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.364+0000 7f8cd9600640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd410bbe0 0x7f8cd41a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.364+0000 7f8cd9600640 1 -- 172.21.15.32:0/2660502754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd41a2220 con 0x7f8cd4105950 2024-08-31T07:24:34.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.364+0000 7f8cd9600640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cd4105950 0x7f8cd419c2b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800e9f0 tx=0x7f8cc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:34.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.365+0000 7f8cd2000640 1 -- 172.21.15.32:0/2660502754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800cd80 con 0x7f8cd4105950 2024-08-31T07:24:34.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.365+0000 7f8cd2000640 1 -- 172.21.15.32:0/2660502754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc80108e0 con 0x7f8cd4105950 2024-08-31T07:24:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.365+0000 7f8cdaa00640 1 -- 172.21.15.32:0/2660502754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd41a2510 con 0x7f8cd4105950 2024-08-31T07:24:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.365+0000 7f8cd2000640 1 -- 172.21.15.32:0/2660502754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8018980 con 0x7f8cd4105950 2024-08-31T07:24:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.365+0000 7f8cdaa00640 1 -- 172.21.15.32:0/2660502754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd41a8c70 con 0x7f8cd4105950 2024-08-31T07:24:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.368+0000 7f8ce2c05740 1 Processor -- start 2024-08-31T07:24:34.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.368+0000 7f8ce2c05740 1 -- start start 2024-08-31T07:24:34.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.368+0000 7f8cd2000640 1 -- 172.21.15.32:0/2660502754 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8cc8010430 con 0x7f8cd4105950 2024-08-31T07:24:34.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.368+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.368+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe11b0 0x5609c23b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.368+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe9390 0x5609c23b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.368+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5609c2326770 con 0x5609c200a070 2024-08-31T07:24:34.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5609c2315d00 con 0x5609c1fe11b0 2024-08-31T07:24:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5609c2319cb0 con 0x5609c1fe9390 2024-08-31T07:24:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cda000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cda000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37900/0 (socket says 172.21.15.32:37900) 2024-08-31T07:24:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cda000640 1 -- 172.21.15.32:0/1232357420 learned_addr learned my addr 172.21.15.32:0/1232357420 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cd9600640 1 --2- 172.21.15.32:0/1232357420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe11b0 0x5609c23b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/1232357420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe9390 0x5609c23b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cda000640 1 -- 172.21.15.32:0/1232357420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe9390 msgr2=0x5609c23b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cda000640 1 --2- 172.21.15.32:0/1232357420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe9390 0x5609c23b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cda000640 1 -- 172.21.15.32:0/1232357420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe11b0 msgr2=0x5609c23b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cda000640 1 --2- 172.21.15.32:0/1232357420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe11b0 0x5609c23b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cda000640 1 -- 172.21.15.32:0/1232357420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c2075650 con 0x5609c200a070 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.369+0000 7f8cd9600640 1 --2- 172.21.15.32:0/1232357420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe11b0 0x5609c23b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.370+0000 7f8cda000640 1 --2- 172.21.15.32:0/1232357420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23afb20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0039f0 tx=0x7f8ccc003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d25b3b13814dee server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.370+0000 7f8cc6600640 1 -- 172.21.15.32:0/1232357420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc01a470 con 0x5609c200a070 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.370+0000 7f8cc6600640 1 -- 172.21.15.32:0/1232357420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ccc015070 con 0x5609c200a070 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.370+0000 7f8cd2000640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8cbc0b4d20 0x7f8cbc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.370+0000 7f8cc6600640 1 -- 172.21.15.32:0/1232357420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc0107e0 con 0x5609c200a070 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.370+0000 7f8cd2000640 1 -- 172.21.15.32:0/2660502754 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f8cc80e57e0 con 0x7f8cd4105950 2024-08-31T07:24:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8cd9600640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8cbc0b4d20 0x7f8cbc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8ce2c05740 1 -- 172.21.15.32:0/1232357420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 msgr2=0x5609c23afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/1232357420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23afb20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0039f0 tx=0x7f8ccc003eb0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8ce2c05740 1 -- 172.21.15.32:0/1232357420 shutdown_connections 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/1232357420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe9390 0x5609c23b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/1232357420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe11b0 0x5609c23b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/1232357420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23afb20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8ce2c05740 1 -- 172.21.15.32:0/1232357420 >> 172.21.15.32:0/1232357420 conn(0x5609c1ffe460 msgr2=0x5609c23ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8cd9600640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8cbc0b4d20 0x7f8cbc0b7230 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x5609c23ae7a0 tx=0x7f8cc80eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8ce2c05740 1 -- 172.21.15.32:0/1232357420 shutdown_connections 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.371+0000 7f8ce2c05740 1 -- 172.21.15.32:0/1232357420 wait complete. 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.372+0000 7f8ce2c05740 1 -- start start 2024-08-31T07:24:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.372+0000 7f8ce2c05740 1 Processor -- start 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23bbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5609c2178700 con 0x5609c200a070 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5609c229ccb0 con 0x5609c1fe9390 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5609c23be350 con 0x5609c1fe11b0 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8ce2c05740 10 client.0 _ll_register_callbacks cb 0x5609c22c2430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cd8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23bbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cda000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cd8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23bbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59052/0 (socket says 172.21.15.32:59052) 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cda000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cd8c00640 1 -- 172.21.15.32:0/3956709993 learned_addr learned my addr 172.21.15.32:0/3956709993 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cd8c00640 1 -- 172.21.15.32:0/3956709993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 msgr2=0x5609c23bb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cd8c00640 1 -- 172.21.15.32:0/3956709993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 msgr2=0x5609c23c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cd8c00640 1 -- 172.21.15.32:0/3956709993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c22dd870 con 0x5609c1fe9390 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.373+0000 7f8cda000640 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.374+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23bbc00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009900 tx=0x7f8cc0032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84298268f1bdc076 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.374+0000 7f8cbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.374+0000 7f8cbaa00640 1 -- 172.21.15.32:0/3956709993 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0035040 con 0x5609c1fe9390 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.374+0000 7f8ce2c05740 1 -- 172.21.15.32:0/3956709993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ccc015070 con 0x5609c1fe9390 2024-08-31T07:24:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.374+0000 7f8cbaa00640 1 -- 172.21.15.32:0/3956709993 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f8cc0031760 con 0x5609c1fe9390 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.374+0000 7f8cbaa00640 1 -- 172.21.15.32:0/3956709993 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8cc0004040 con 0x5609c1fe9390 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.375+0000 7f8ce2c05740 10 client.28043 fetch_fsmap learned FSMap version 45 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.375+0000 7f8ce2c05740 1 -- 172.21.15.32:0/3956709993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5609c23ca330 con 0x5609c1fe9390 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.375+0000 7f8cbaa00640 1 -- 172.21.15.32:0/3956709993 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ccc015070 con 0x5609c1fe9390 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.375+0000 7f8cbaa00640 1 -- 172.21.15.32:0/3956709993 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f8cc0004750 con 0x5609c1fe9390 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.375+0000 7f8ce2c05740 10 client.28043 fetch_fsmap finished waiting for FSMap version 45 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.375+0000 7f8ce2c05740 10 client.28043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.375+0000 7f8ce2c05740 20 client.28043 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.375+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5609c1fc89f0 0x5609c23cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.375+0000 7f8ce2c05740 4 client.28043 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.375+0000 7f8ce2c05740 1 -- 172.21.15.32:0/3956709993 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609c23286c0 con 0x5609c1fc89f0 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.376+0000 7f8cda000640 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5609c1fc89f0 0x5609c23cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.376+0000 7f8cda000640 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5609c1fc89f0 0x5609c23cd7f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1e18a0194d410b5 server_cookie=f229c06f5d8e7944 in_seq=0 out_seq=0 2024-08-31T07:24:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.376+0000 7f8cbaa00640 10 client.28043 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.377+0000 7f8cbaa00640 1 -- 172.21.15.32:0/3956709993 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5609c23286c0 con 0x5609c1fc89f0 2024-08-31T07:24:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.377+0000 7f8cbaa00640 10 client.28043 handle_command_reply: tid=0 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.377+0000 7f8ce2c05740 1 client.28043 shutdown 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 1 -- 172.21.15.32:0/3956709993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 msgr2=0x5609c23bbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23bbc00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009900 tx=0x7f8cc0032e90 comp rx=0 tx=0).stop 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 1 -- 172.21.15.32:0/3956709993 shutdown_connections 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5609c1fc89f0 0x5609c23cd7f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23bbc00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/3956709993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8cbaa00640 0 client.28043 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 1 -- 172.21.15.32:0/3956709993 >> 172.21.15.32:0/3956709993 conn(0x5609c1ffe460 msgr2=0x5609c23b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 1 -- 172.21.15.32:0/3956709993 shutdown_connections 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 1 -- 172.21.15.32:0/3956709993 wait complete. 2024-08-31T07:24:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.378+0000 7f8ce2c05740 20 client.28043 trim_cache size 0 max 16384 2024-08-31T07:24:34.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.396+0000 7f8ce2c05740 1 Processor -- start 2024-08-31T07:24:34.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.396+0000 7f8ce2c05740 1 -- start start 2024-08-31T07:24:34.397 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:34 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:24:34.397 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:34 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:34.397 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:34 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:24:34.398 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:34 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:34.398 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:34 smithi032 ceph-mon[17252]: pgmap v734: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 538 KiB/s wr, 72 op/s 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23ad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41b1390 con 0x5609c200a070 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41b2f30 con 0x5609c1fe9390 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41b1a60 con 0x5609c1fe11b0 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8cd8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8cd8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59066/0 (socket says 172.21.15.32:59066) 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8cd8c00640 1 -- 172.21.15.32:0/2620412922 learned_addr learned my addr 172.21.15.32:0/2620412922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8cda000640 1 --2- 172.21.15.32:0/2620412922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8cda000640 1 --2- 172.21.15.32:0/2620412922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8cd8c00640 1 -- 172.21.15.32:0/2620412922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 msgr2=0x5609c23ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.397+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/2620412922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.398+0000 7f8cd8c00640 1 -- 172.21.15.32:0/2620412922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 msgr2=0x5609c23c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.398+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/2620412922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.398+0000 7f8cd8c00640 1 -- 172.21.15.32:0/2620412922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c23ba420 con 0x5609c1fe9390 2024-08-31T07:24:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.398+0000 7f8cda000640 1 --2- 172.21.15.32:0/2620412922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:24:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.398+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/2620412922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23b1260 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8cc002a930 tx=0x7f8cc0004900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f9722041830b04 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.398+0000 7f8cc5c00640 1 -- 172.21.15.32:0/2620412922 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0035070 con 0x5609c1fe9390 2024-08-31T07:24:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.398+0000 7f8cc5c00640 1 -- 172.21.15.32:0/2620412922 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc00043e0 con 0x5609c1fe9390 2024-08-31T07:24:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.398+0000 7f8cc5c00640 1 -- 172.21.15.32:0/2620412922 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0000e60 con 0x5609c1fe9390 2024-08-31T07:24:34.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.399+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2620412922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 msgr2=0x5609c23b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:34.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.399+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/2620412922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23b1260 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8cc002a930 tx=0x7f8cc0004900 comp rx=0 tx=0).stop 2024-08-31T07:24:34.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.399+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2620412922 shutdown_connections 2024-08-31T07:24:34.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.399+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/2620412922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c23c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.399+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/2620412922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23b1260 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.399+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/2620412922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.399+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2620412922 >> 172.21.15.32:0/2620412922 conn(0x5609c1ffe460 msgr2=0x5609c23b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:34.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.400+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2620412922 shutdown_connections 2024-08-31T07:24:34.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.400+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2620412922 wait complete. 2024-08-31T07:24:34.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.400+0000 7f8ce2c05740 1 -- start start 2024-08-31T07:24:34.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.400+0000 7f8ce2c05740 1 Processor -- start 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23c0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8ce2c05740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c2439a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41b2830 con 0x5609c200a070 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41b2130 con 0x5609c1fe9390 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8ce2c05740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41af390 con 0x5609c1fe11b0 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8ce2c05740 10 client.0 _ll_register_callbacks cb 0x7f8cc0032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8cda000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8cd8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8cda000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36826/0 (socket says 172.21.15.32:36826) 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8cda000640 1 -- 172.21.15.32:0/2029420380 learned_addr learned my addr 172.21.15.32:0/2029420380 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:24:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.401+0000 7f8cd8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23c0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59082/0 (socket says 172.21.15.32:59082) 2024-08-31T07:24:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.402+0000 7f8cda000640 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c2439a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.402+0000 7f8cd8c00640 1 -- 172.21.15.32:0/2029420380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 msgr2=0x5609c23bff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.402+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.402+0000 7f8cd8c00640 1 -- 172.21.15.32:0/2029420380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 msgr2=0x5609c2439a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:24:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.402+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c2439a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.402+0000 7f8cd8c00640 1 -- 172.21.15.32:0/2029420380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c243a0c0 con 0x5609c1fe9390 2024-08-31T07:24:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.402+0000 7f8cda000640 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:24:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.402+0000 7f8cd8c00640 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23c0520 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0004cc0 tx=0x7f8cc00039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d69c84202342f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:24:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.402+0000 7f8cbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:24:34.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.403+0000 7f8cbaa00640 1 -- 172.21.15.32:0/2029420380 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0035030 con 0x5609c1fe9390 2024-08-31T07:24:34.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.403+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2029420380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc00043e0 con 0x5609c1fe9390 2024-08-31T07:24:34.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.403+0000 7f8cbaa00640 1 -- 172.21.15.32:0/2029420380 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8441+0+0 (secure 0 0 0) 0x7f8cc0003d50 con 0x5609c1fe9390 2024-08-31T07:24:34.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.403+0000 7f8cbaa00640 1 -- 172.21.15.32:0/2029420380 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8cc0067020 con 0x5609c1fe9390 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.403+0000 7f8ce2c05740 10 client.28049 fetch_fsmap learned FSMap version 45 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.403+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2029420380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5609c2441360 con 0x5609c1fe9390 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.403+0000 7f8cbaa00640 1 -- 172.21.15.32:0/2029420380 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc00043e0 con 0x5609c1fe9390 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.404+0000 7f8cbaa00640 1 -- 172.21.15.32:0/2029420380 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f8cc00051c0 con 0x5609c1fe9390 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.404+0000 7f8ce2c05740 10 client.28049 fetch_fsmap finished waiting for FSMap version 45 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.404+0000 7f8ce2c05740 10 client.28049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.404+0000 7f8ce2c05740 20 client.28049 populate_metadata read hostname 'smithi032' 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.404+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5609c1f9dda0 0x5609c2443a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.404+0000 7f8ce2c05740 4 client.28049 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.404+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2029420380 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609c2324160 con 0x5609c1f9dda0 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.404+0000 7f8cda000640 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5609c1f9dda0 0x5609c2443a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8cda000640 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5609c1f9dda0 0x5609c2443a10 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d8f18f0854b9b2 server_cookie=3d508e387210c806 in_seq=0 out_seq=0 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8cbaa00640 10 client.28049 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8cbaa00640 1 -- 172.21.15.32:0/2029420380 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4552 (crc 0 0 0) 0x5609c2324160 con 0x5609c1f9dda0 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8cbaa00640 10 client.28049 handle_command_reply: tid=0 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8ce2c05740 1 client.28049 shutdown 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2029420380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 msgr2=0x5609c23c0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23c0520 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0004cc0 tx=0x7f8cc00039b0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2029420380 shutdown_connections 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5609c1f9dda0 0x5609c2443a10 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609c200a070 0x5609c2439a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609c1fe9390 0x5609c23c0520 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8ce2c05740 1 --2- 172.21.15.32:0/2029420380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609c1fe11b0 0x5609c23bff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8cbaa00640 0 client.28049 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2029420380 >> 172.21.15.32:0/2029420380 conn(0x5609c1ffe460 msgr2=0x5609c23b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.405+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2029420380 shutdown_connections 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8ce2c05740 1 -- 172.21.15.32:0/2029420380 wait complete. 2024-08-31T07:24:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8ce2c05740 20 client.28049 trim_cache size 0 max 16384 2024-08-31T07:24:34.409 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:24:34.409 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "scrub active (1 inodes in the stack)", 2024-08-31T07:24:34.409 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:24:34.409 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:24:34.409 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "96b7006d-efb1-4838-bb2b-531e42fd1874", 2024-08-31T07:24:34.409 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:24:34.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:24:34.410 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:24:34.411 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.413 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.414 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:24:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.417 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:24:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8cdaa00640 1 -- 172.21.15.32:0/2660502754 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8cbc0b4d20 msgr2=0x7f8cbc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8cdaa00640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8cbc0b4d20 0x7f8cbc0b7230 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x5609c23ae7a0 tx=0x7f8cc80eb530 comp rx=0 tx=0).stop 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8cdaa00640 1 -- 172.21.15.32:0/2660502754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cd4105950 msgr2=0x7f8cd419c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8cdaa00640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cd4105950 0x7f8cd419c2b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800e9f0 tx=0x7f8cc800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8cdaa00640 1 -- 172.21.15.32:0/2660502754 shutdown_connections 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8cdaa00640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8cbc0b4d20 0x7f8cbc0b7230 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8cdaa00640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd410bbe0 0x7f8cd41a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8cdaa00640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cd410b290 0x7f8cd419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8cdaa00640 1 --2- 172.21.15.32:0/2660502754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cd4105950 0x7f8cd419c2b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.406+0000 7f8cdaa00640 1 -- 172.21.15.32:0/2660502754 >> 172.21.15.32:0/2660502754 conn(0x7f8cd4101420 msgr2=0x7f8cd4103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.407+0000 7f8cdaa00640 1 -- 172.21.15.32:0/2660502754 shutdown_connections 2024-08-31T07:24:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:24:34.407+0000 7f8cdaa00640 1 -- 172.21.15.32:0/2660502754 wait complete. 2024-08-31T07:24:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:34 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:24:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:34 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:34 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:24:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:34 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:34.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:34 smithi073 ceph-mon[21995]: pgmap v734: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 538 KiB/s wr, 72 op/s 2024-08-31T07:24:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:34 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:24:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:34 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:34 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:24:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:34 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:34.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:34 smithi084 ceph-mon[22387]: pgmap v734: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 538 KiB/s wr, 72 op/s 2024-08-31T07:24:34.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 354646 2.69 MB/sec execute 503 sec latency 3839.125 ms 2024-08-31T07:24:35.316 INFO:tasks.cephfs.filesystem:scrub status for tag:96b7006d-efb1-4838-bb2b-531e42fd1874 - {'path': '/', 'tag': '96b7006d-efb1-4838-bb2b-531e42fd1874', 'options': 'recursive,force'} 2024-08-31T07:24:35.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 354646 2.69 MB/sec execute 504 sec latency 4839.194 ms 2024-08-31T07:24:36.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:36 smithi073 ceph-mon[21995]: pgmap v735: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 496 KiB/s wr, 72 op/s 2024-08-31T07:24:36.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:36 smithi084 ceph-mon[22387]: pgmap v735: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 496 KiB/s wr, 72 op/s 2024-08-31T07:24:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:36 smithi032 ceph-mon[17252]: pgmap v735: 97 pgs: 97 active+clean; 277 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 496 KiB/s wr, 72 op/s 2024-08-31T07:24:36.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 354646 2.68 MB/sec execute 505 sec latency 5839.316 ms 2024-08-31T07:24:37.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 354646 2.68 MB/sec execute 506 sec latency 6839.469 ms 2024-08-31T07:24:38.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:38 smithi073 ceph-mon[21995]: pgmap v736: 97 pgs: 97 active+clean; 279 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 686 KiB/s wr, 90 op/s 2024-08-31T07:24:38.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:38 smithi084 ceph-mon[22387]: pgmap v736: 97 pgs: 97 active+clean; 279 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 686 KiB/s wr, 90 op/s 2024-08-31T07:24:38.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:38 smithi032 ceph-mon[17252]: pgmap v736: 97 pgs: 97 active+clean; 279 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 686 KiB/s wr, 90 op/s 2024-08-31T07:24:38.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 354996 2.68 MB/sec execute 507 sec latency 7457.246 ms 2024-08-31T07:24:39.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:39 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:24:39.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:39 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:39.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:39 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:39.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:39 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:24:39.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:39 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:39.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:39 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:39.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:39 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:24:39.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:39 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:39 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:24:39.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 356231 2.68 MB/sec execute 508 sec latency 17.432 ms 2024-08-31T07:24:40.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:40 smithi073 ceph-mon[21995]: pgmap v737: 97 pgs: 97 active+clean; 279 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 667 KiB/s wr, 84 op/s 2024-08-31T07:24:40.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:40 smithi084 ceph-mon[22387]: pgmap v737: 97 pgs: 97 active+clean; 279 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 667 KiB/s wr, 84 op/s 2024-08-31T07:24:40.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:40 smithi032 ceph-mon[17252]: pgmap v737: 97 pgs: 97 active+clean; 279 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 667 KiB/s wr, 84 op/s 2024-08-31T07:24:40.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 357691 2.69 MB/sec execute 509 sec latency 234.808 ms 2024-08-31T07:24:41.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 358847 2.70 MB/sec execute 510 sec latency 17.881 ms 2024-08-31T07:24:42.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:42 smithi073 ceph-mon[21995]: pgmap v738: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 823 KiB/s wr, 88 op/s 2024-08-31T07:24:42.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:42 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:24:42.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:42 smithi084 ceph-mon[22387]: pgmap v738: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 823 KiB/s wr, 88 op/s 2024-08-31T07:24:42.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:42 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:24:42.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:42 smithi032 ceph-mon[17252]: pgmap v738: 97 pgs: 97 active+clean; 281 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 823 KiB/s wr, 88 op/s 2024-08-31T07:24:42.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:42 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:24:42.776 INFO:tasks.workunit.client.0.smithi032.stdout: 1 359908 2.70 MB/sec execute 511 sec latency 84.356 ms 2024-08-31T07:24:43.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 359941 2.70 MB/sec execute 512 sec latency 976.849 ms 2024-08-31T07:24:44.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:44 smithi073 ceph-mon[21995]: pgmap v739: 97 pgs: 97 active+clean; 283 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 681 KiB/s wr, 75 op/s 2024-08-31T07:24:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:44 smithi084 ceph-mon[22387]: pgmap v739: 97 pgs: 97 active+clean; 283 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 681 KiB/s wr, 75 op/s 2024-08-31T07:24:44.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:44 smithi032 ceph-mon[17252]: pgmap v739: 97 pgs: 97 active+clean; 283 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 681 KiB/s wr, 75 op/s 2024-08-31T07:24:44.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 359941 2.69 MB/sec execute 513 sec latency 1976.993 ms 2024-08-31T07:24:45.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 361597 2.71 MB/sec execute 514 sec latency 2174.816 ms 2024-08-31T07:24:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:46 smithi073 ceph-mon[21995]: pgmap v740: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 736 KiB/s wr, 92 op/s 2024-08-31T07:24:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:46 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:24:46.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:46 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:24:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:46 smithi084 ceph-mon[22387]: pgmap v740: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 736 KiB/s wr, 92 op/s 2024-08-31T07:24:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:46 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:24:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:46 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:24:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:46 smithi032 ceph-mon[17252]: pgmap v740: 97 pgs: 97 active+clean; 284 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 736 KiB/s wr, 92 op/s 2024-08-31T07:24:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:46 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:24:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:46 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:24:46.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 362665 2.71 MB/sec execute 515 sec latency 17.830 ms 2024-08-31T07:24:47.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:47 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:24:47.432 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:47 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:24:47.433 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:47 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:24:47.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:47 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:24:47.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:47 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:24:47.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:47 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:24:47.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:47 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:24:47.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:47 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:24:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:47 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:24:47.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 363483 2.71 MB/sec execute 516 sec latency 268.313 ms 2024-08-31T07:24:48.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:48 smithi073 ceph-mon[21995]: pgmap v741: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-08-31T07:24:48.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:48 smithi084 ceph-mon[22387]: pgmap v741: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-08-31T07:24:48.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:48 smithi032 ceph-mon[17252]: pgmap v741: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-08-31T07:24:48.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 363483 2.71 MB/sec execute 517 sec latency 1268.422 ms 2024-08-31T07:24:49.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 363483 2.70 MB/sec execute 518 sec latency 2268.549 ms 2024-08-31T07:24:50.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:50 smithi073 ceph-mon[21995]: pgmap v742: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 912 KiB/s wr, 112 op/s 2024-08-31T07:24:50.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:50 smithi084 ceph-mon[22387]: pgmap v742: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 912 KiB/s wr, 112 op/s 2024-08-31T07:24:50.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:50 smithi032 ceph-mon[17252]: pgmap v742: 97 pgs: 97 active+clean; 288 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 912 KiB/s wr, 112 op/s 2024-08-31T07:24:50.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 364796 2.71 MB/sec execute 519 sec latency 2617.996 ms 2024-08-31T07:24:51.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 365935 2.72 MB/sec execute 520 sec latency 16.441 ms 2024-08-31T07:24:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:52 smithi073 ceph-mon[21995]: pgmap v743: 97 pgs: 97 active+clean; 291 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-08-31T07:24:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:52 smithi084 ceph-mon[22387]: pgmap v743: 97 pgs: 97 active+clean; 291 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-08-31T07:24:52.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:52 smithi032 ceph-mon[17252]: pgmap v743: 97 pgs: 97 active+clean; 291 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-08-31T07:24:52.777 INFO:tasks.workunit.client.0.smithi032.stdout: 1 367032 2.73 MB/sec execute 521 sec latency 85.999 ms 2024-08-31T07:24:53.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 367032 2.72 MB/sec execute 522 sec latency 1086.111 ms 2024-08-31T07:24:54.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:54 smithi073 ceph-mon[21995]: pgmap v744: 97 pgs: 97 active+clean; 292 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 984 KiB/s wr, 127 op/s 2024-08-31T07:24:54.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:54 smithi084 ceph-mon[22387]: pgmap v744: 97 pgs: 97 active+clean; 292 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 984 KiB/s wr, 127 op/s 2024-08-31T07:24:54.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:54 smithi032 ceph-mon[17252]: pgmap v744: 97 pgs: 97 active+clean; 292 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 984 KiB/s wr, 127 op/s 2024-08-31T07:24:54.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 367032 2.71 MB/sec execute 523 sec latency 2086.270 ms 2024-08-31T07:24:55.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 368330 2.72 MB/sec execute 524 sec latency 2433.164 ms 2024-08-31T07:24:56.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:56 smithi073 ceph-mon[21995]: pgmap v745: 97 pgs: 97 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-08-31T07:24:56.481 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:56 smithi032 ceph-mon[17252]: pgmap v745: 97 pgs: 97 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-08-31T07:24:56.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:56 smithi084 ceph-mon[22387]: pgmap v745: 97 pgs: 97 active+clean; 295 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-08-31T07:24:56.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 369325 2.73 MB/sec execute 525 sec latency 142.252 ms 2024-08-31T07:24:57.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 370448 2.74 MB/sec execute 526 sec latency 19.685 ms 2024-08-31T07:24:58.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:24:58 smithi084 ceph-mon[22387]: pgmap v746: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-08-31T07:24:58.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:24:58 smithi032 ceph-mon[17252]: pgmap v746: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-08-31T07:24:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:24:58 smithi073 ceph-mon[21995]: pgmap v746: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-08-31T07:24:58.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 370575 2.73 MB/sec execute 527 sec latency 901.120 ms 2024-08-31T07:24:59.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 370575 2.73 MB/sec execute 528 sec latency 1901.221 ms 2024-08-31T07:25:00.493 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:00 smithi084 ceph-mon[22387]: pgmap v747: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 923 KiB/s wr, 106 op/s 2024-08-31T07:25:00.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:00 smithi032 ceph-mon[17252]: pgmap v747: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 923 KiB/s wr, 106 op/s 2024-08-31T07:25:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:00 smithi073 ceph-mon[21995]: pgmap v747: 97 pgs: 97 active+clean; 297 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 923 KiB/s wr, 106 op/s 2024-08-31T07:25:00.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 371376 2.72 MB/sec execute 529 sec latency 2248.997 ms 2024-08-31T07:25:01.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 372152 2.73 MB/sec execute 530 sec latency 263.737 ms 2024-08-31T07:25:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:02 smithi084 ceph-mon[22387]: pgmap v748: 97 pgs: 97 active+clean; 300 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-08-31T07:25:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:02 smithi084 ceph-mon[22387]: osdmap e93: 12 total, 12 up, 12 in 2024-08-31T07:25:02.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:02 smithi032 ceph-mon[17252]: pgmap v748: 97 pgs: 97 active+clean; 300 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-08-31T07:25:02.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:02 smithi032 ceph-mon[17252]: osdmap e93: 12 total, 12 up, 12 in 2024-08-31T07:25:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:02 smithi073 ceph-mon[21995]: pgmap v748: 97 pgs: 97 active+clean; 300 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-08-31T07:25:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:02 smithi073 ceph-mon[21995]: osdmap e93: 12 total, 12 up, 12 in 2024-08-31T07:25:02.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 373051 2.74 MB/sec execute 531 sec latency 372.902 ms 2024-08-31T07:25:03.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 374123 2.74 MB/sec execute 532 sec latency 259.034 ms 2024-08-31T07:25:04.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:04 smithi084 ceph-mon[22387]: pgmap v750: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-08-31T07:25:04.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:04 smithi032 ceph-mon[17252]: pgmap v750: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-08-31T07:25:04.766 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:04 smithi073 ceph-mon[21995]: pgmap v750: 97 pgs: 97 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-08-31T07:25:04.778 INFO:tasks.workunit.client.0.smithi032.stdout: 1 374123 2.74 MB/sec execute 533 sec latency 1259.156 ms 2024-08-31T07:25:05.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:25:05.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:05 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:25:05.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:05 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:25:05.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:05 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:25:05.729 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:05.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 375363 2.74 MB/sec execute 534 sec latency 1604.813 ms 2024-08-31T07:25:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.412+0000 7ff09f200640 1 -- 172.21.15.32:0/586831806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a01099e0 msgr2=0x7ff0a0110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.412+0000 7ff09f200640 1 --2- 172.21.15.32:0/586831806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a01099e0 0x7ff0a0110210 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff098009ca0 tx=0x7ff09802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.412+0000 7ff09f200640 1 -- 172.21.15.32:0/586831806 shutdown_connections 2024-08-31T07:25:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.412+0000 7ff09f200640 1 --2- 172.21.15.32:0/586831806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a01099e0 0x7ff0a0110210 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.412+0000 7ff09f200640 1 --2- 172.21.15.32:0/586831806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a0109090 0x7ff0a01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.412+0000 7ff09f200640 1 --2- 172.21.15.32:0/586831806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a0103750 0x7ff0a0103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.412+0000 7ff09f200640 1 -- 172.21.15.32:0/586831806 >> 172.21.15.32:0/586831806 conn(0x7ff0a00ff1a0 msgr2=0x7ff0a01015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.413+0000 7ff09f200640 1 -- 172.21.15.32:0/586831806 shutdown_connections 2024-08-31T07:25:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.413+0000 7ff09f200640 1 -- 172.21.15.32:0/586831806 wait complete. 2024-08-31T07:25:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.414+0000 7ff09f200640 1 Processor -- start 2024-08-31T07:25:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.414+0000 7ff09f200640 1 -- start start 2024-08-31T07:25:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.414+0000 7ff09f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0103750 0x7ff0a019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.414+0000 7ff09f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a0109090 0x7ff0a019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.414+0000 7ff09f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01099e0 0x7ff0a019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.414+0000 7ff09f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a0113690 con 0x7ff0a0109090 2024-08-31T07:25:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.414+0000 7ff09f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a0113510 con 0x7ff0a0103750 2024-08-31T07:25:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.414+0000 7ff09f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a0113810 con 0x7ff0a01099e0 2024-08-31T07:25:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.414+0000 7ff097e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a0109090 0x7ff0a019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff09de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0103750 0x7ff0a019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff097e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a0109090 0x7ff0a019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52146/0 (socket says 172.21.15.32:52146) 2024-08-31T07:25:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff097e00640 1 -- 172.21.15.32:0/2199139321 learned_addr learned my addr 172.21.15.32:0/2199139321 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff09e800640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01099e0 0x7ff0a019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff09de00640 1 -- 172.21.15.32:0/2199139321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01099e0 msgr2=0x7ff0a019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff09de00640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01099e0 0x7ff0a019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff09de00640 1 -- 172.21.15.32:0/2199139321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a0109090 msgr2=0x7ff0a019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff09de00640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a0109090 0x7ff0a019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff09de00640 1 -- 172.21.15.32:0/2199139321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a019ff90 con 0x7ff0a0103750 2024-08-31T07:25:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff097e00640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a0109090 0x7ff0a019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.415+0000 7ff09de00640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0103750 0x7ff0a019a020 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff09000e9f0 tx=0x7ff09000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.416+0000 7ff096a00640 1 -- 172.21.15.32:0/2199139321 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09000ccc0 con 0x7ff0a0103750 2024-08-31T07:25:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.416+0000 7ff096a00640 1 -- 172.21.15.32:0/2199139321 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0900108e0 con 0x7ff0a0103750 2024-08-31T07:25:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.416+0000 7ff096a00640 1 -- 172.21.15.32:0/2199139321 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090018880 con 0x7ff0a0103750 2024-08-31T07:25:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.416+0000 7ff09f200640 1 -- 172.21.15.32:0/2199139321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a01a0280 con 0x7ff0a0103750 2024-08-31T07:25:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.416+0000 7ff09f200640 1 -- 172.21.15.32:0/2199139321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a01a69e0 con 0x7ff0a0103750 2024-08-31T07:25:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.419+0000 7ff0abc79740 1 Processor -- start 2024-08-31T07:25:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.419+0000 7ff0abc79740 1 -- start start 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.419+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3002e070 0x55ae303d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.419+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae300051b0 0x55ae303d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.419+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.419+0000 7ff0abc79740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ae3034a770 con 0x55ae3002e070 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.419+0000 7ff0abc79740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ae30339d00 con 0x55ae3000d390 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.419+0000 7ff0abc79740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ae3033dcb0 con 0x55ae300051b0 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff096a00640 1 -- 172.21.15.32:0/2199139321 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff090010430 con 0x7ff0a0103750 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff097e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff09e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3002e070 0x55ae303d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff097e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42046/0 (socket says 172.21.15.32:42046) 2024-08-31T07:25:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff097e00640 1 -- 172.21.15.32:0/1886270595 learned_addr learned my addr 172.21.15.32:0/1886270595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff09de00640 1 --2- 172.21.15.32:0/1886270595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae300051b0 0x55ae303d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff097e00640 1 -- 172.21.15.32:0/1886270595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae300051b0 msgr2=0x55ae303d4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff097e00640 1 --2- 172.21.15.32:0/1886270595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae300051b0 0x55ae303d4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff097e00640 1 -- 172.21.15.32:0/1886270595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3002e070 msgr2=0x55ae303d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff097e00640 1 --2- 172.21.15.32:0/1886270595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3002e070 0x55ae303d3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.420+0000 7ff097e00640 1 -- 172.21.15.32:0/1886270595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae30099650 con 0x55ae3000d390 2024-08-31T07:25:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.421+0000 7ff097e00640 1 --2- 172.21.15.32:0/1886270595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303d45a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff088011540 tx=0x7ff088011940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9ca712d6c725f21 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:06.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.421+0000 7ff08f400640 1 -- 172.21.15.32:0/1886270595 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08801e070 con 0x55ae3000d390 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.421+0000 7ff08f400640 1 -- 172.21.15.32:0/1886270595 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08800fa70 con 0x55ae3000d390 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.421+0000 7ff08f400640 1 -- 172.21.15.32:0/1886270595 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088019bc0 con 0x55ae3000d390 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.421+0000 7ff096a00640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0840b4ca0 0x7ff0840b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff096a00640 1 -- 172.21.15.32:0/2199139321 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8485+0+0 (secure 0 0 0) 0x7ff0900e5830 con 0x7ff0a0103750 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff0abc79740 1 -- 172.21.15.32:0/1886270595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 msgr2=0x55ae303d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff0abc79740 1 --2- 172.21.15.32:0/1886270595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303d45a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff088011540 tx=0x7ff088011940 comp rx=0 tx=0).stop 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff09e800640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0840b4ca0 0x7ff0840b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff0abc79740 1 -- 172.21.15.32:0/1886270595 shutdown_connections 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff0abc79740 1 --2- 172.21.15.32:0/1886270595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303d45a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff0abc79740 1 --2- 172.21.15.32:0/1886270595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae300051b0 0x55ae303d4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff0abc79740 1 --2- 172.21.15.32:0/1886270595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3002e070 0x55ae303d3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff0abc79740 1 -- 172.21.15.32:0/1886270595 >> 172.21.15.32:0/1886270595 conn(0x55ae30022460 msgr2=0x55ae303d1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff0abc79740 1 -- 172.21.15.32:0/1886270595 shutdown_connections 2024-08-31T07:25:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff0abc79740 1 -- 172.21.15.32:0/1886270595 wait complete. 2024-08-31T07:25:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.422+0000 7ff09e800640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0840b4ca0 0x7ff0840b71b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff09802f500 tx=0x7ff09802f310 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.423+0000 7ff0abc79740 1 -- start start 2024-08-31T07:25:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.423+0000 7ff0abc79740 1 Processor -- start 2024-08-31T07:25:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.423+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae303df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.423+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.423+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3002e070 0x55ae303eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.423+0000 7ff0abc79740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ae3019c700 con 0x55ae300051b0 2024-08-31T07:25:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.423+0000 7ff0abc79740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ae302c0cb0 con 0x55ae3000d390 2024-08-31T07:25:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.423+0000 7ff0abc79740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ae303e2300 con 0x55ae3002e070 2024-08-31T07:25:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff0abc79740 10 client.0 _ll_register_callbacks cb 0x55ae302e6430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff09de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae303df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff097e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff09de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae303df770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52170/0 (socket says 172.21.15.32:52170) 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff097e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303dfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42048/0 (socket says 172.21.15.32:42048) 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff09de00640 1 -- 172.21.15.32:0/1941250968 learned_addr learned my addr 172.21.15.32:0/1941250968 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff09e800640 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3002e070 0x55ae303eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff097e00640 1 -- 172.21.15.32:0/1941250968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3002e070 msgr2=0x55ae303eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff097e00640 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3002e070 0x55ae303eb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff097e00640 1 -- 172.21.15.32:0/1941250968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 msgr2=0x55ae303df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff097e00640 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae303df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff097e00640 1 -- 172.21.15.32:0/1941250968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae30301870 con 0x55ae3000d390 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.424+0000 7ff09de00640 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae303df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.425+0000 7ff097e00640 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303dfd30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff088009760 tx=0x7ff088019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d61f9d9112c82b0a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.425+0000 7ff083e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:25:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.425+0000 7ff083e00640 1 -- 172.21.15.32:0/1941250968 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08801e070 con 0x55ae3000d390 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.425+0000 7ff0abc79740 1 -- 172.21.15.32:0/1941250968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08800fa70 con 0x55ae3000d390 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.425+0000 7ff083e00640 1 -- 172.21.15.32:0/1941250968 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8485+0+0 (secure 0 0 0) 0x7ff08800f7d0 con 0x55ae3000d390 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.425+0000 7ff083e00640 1 -- 172.21.15.32:0/1941250968 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff088004040 con 0x55ae3000d390 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.425+0000 7ff0abc79740 10 client.28061 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.425+0000 7ff0abc79740 1 -- 172.21.15.32:0/1941250968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ae303ee3a0 con 0x55ae3000d390 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.426+0000 7ff083e00640 1 -- 172.21.15.32:0/1941250968 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08800fa70 con 0x55ae3000d390 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.426+0000 7ff083e00640 1 -- 172.21.15.32:0/1941250968 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7ff088003610 con 0x55ae3000d390 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.426+0000 7ff0abc79740 10 client.28061 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.426+0000 7ff0abc79740 10 client.28061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.426+0000 7ff0abc79740 20 client.28061 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.426+0000 7ff0abc79740 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ae2ffec9f0 0x55ae303f1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.426+0000 7ff0abc79740 4 client.28061 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.426+0000 7ff0abc79740 1 -- 172.21.15.32:0/1941250968 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae3034c6c0 con 0x55ae2ffec9f0 2024-08-31T07:25:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.426+0000 7ff09de00640 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ae2ffec9f0 0x55ae303f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.427+0000 7ff09de00640 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ae2ffec9f0 0x55ae303f1760 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc5e8242cffd422 server_cookie=183f6ffef8a6373c in_seq=0 out_seq=0 2024-08-31T07:25:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.427+0000 7ff083e00640 10 client.28061 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.427+0000 7ff083e00640 1 -- 172.21.15.32:0/1941250968 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ae3034c6c0 con 0x55ae2ffec9f0 2024-08-31T07:25:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.427+0000 7ff083e00640 10 client.28061 handle_command_reply: tid=0 2024-08-31T07:25:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.427+0000 7ff0abc79740 1 client.28061 shutdown 2024-08-31T07:25:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 1 -- 172.21.15.32:0/1941250968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 msgr2=0x55ae303dfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303dfd30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff088009760 tx=0x7ff088019640 comp rx=0 tx=0).stop 2024-08-31T07:25:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 1 -- 172.21.15.32:0/1941250968 shutdown_connections 2024-08-31T07:25:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ae2ffec9f0 0x55ae303f1760 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3002e070 0x55ae303eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff083e00640 0 client.28061 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3000d390 0x55ae303dfd30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 1 --2- 172.21.15.32:0/1941250968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae303df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 1 -- 172.21.15.32:0/1941250968 >> 172.21.15.32:0/1941250968 conn(0x55ae30022460 msgr2=0x55ae303d6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 1 -- 172.21.15.32:0/1941250968 shutdown_connections 2024-08-31T07:25:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 1 -- 172.21.15.32:0/1941250968 wait complete. 2024-08-31T07:25:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.428+0000 7ff0abc79740 20 client.28061 trim_cache size 0 max 16384 2024-08-31T07:25:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff0abc79740 1 Processor -- start 2024-08-31T07:25:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff0abc79740 1 -- start start 2024-08-31T07:25:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae300051b0 0x55ae303eb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3000d390 0x55ae303e7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3002e070 0x55ae303e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff0abc79740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01af230 con 0x55ae3000d390 2024-08-31T07:25:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff0abc79740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01b0e90 con 0x55ae300051b0 2024-08-31T07:25:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff0abc79740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01af900 con 0x55ae3002e070 2024-08-31T07:25:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff097e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3000d390 0x55ae303e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff097e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3000d390 0x55ae303e7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52174/0 (socket says 172.21.15.32:52174) 2024-08-31T07:25:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.447+0000 7ff097e00640 1 -- 172.21.15.32:0/3395186499 learned_addr learned my addr 172.21.15.32:0/3395186499 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.448+0000 7ff09e800640 1 --2- 172.21.15.32:0/3395186499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3002e070 0x55ae303e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.448+0000 7ff09de00640 1 --2- 172.21.15.32:0/3395186499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae300051b0 0x55ae303eb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.448+0000 7ff097e00640 1 -- 172.21.15.32:0/3395186499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3002e070 msgr2=0x55ae303e7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.448+0000 7ff097e00640 1 --2- 172.21.15.32:0/3395186499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3002e070 0x55ae303e7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.448+0000 7ff097e00640 1 -- 172.21.15.32:0/3395186499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae300051b0 msgr2=0x55ae303eb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.448+0000 7ff097e00640 1 --2- 172.21.15.32:0/3395186499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae300051b0 0x55ae303eb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.448+0000 7ff097e00640 1 -- 172.21.15.32:0/3395186499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae303d5e10 con 0x55ae3000d390 2024-08-31T07:25:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.448+0000 7ff097e00640 1 --2- 172.21.15.32:0/3395186499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3000d390 0x55ae303e7930 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff0880057c0 tx=0x7ff0880092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7303f954499a85 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.448+0000 7ff08ea00640 1 -- 172.21.15.32:0/3395186499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08801e040 con 0x55ae3000d390 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff08ea00640 1 -- 172.21.15.32:0/3395186499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08801f040 con 0x55ae3000d390 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff08ea00640 1 -- 172.21.15.32:0/3395186499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088005260 con 0x55ae3000d390 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff0abc79740 1 -- 172.21.15.32:0/3395186499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3000d390 msgr2=0x55ae303e7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff0abc79740 1 --2- 172.21.15.32:0/3395186499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3000d390 0x55ae303e7930 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff0880057c0 tx=0x7ff0880092a0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff0abc79740 1 -- 172.21.15.32:0/3395186499 shutdown_connections 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff0abc79740 1 --2- 172.21.15.32:0/3395186499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3002e070 0x55ae303e7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff0abc79740 1 --2- 172.21.15.32:0/3395186499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae3000d390 0x55ae303e7930 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff0abc79740 1 --2- 172.21.15.32:0/3395186499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae300051b0 0x55ae303eb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff0abc79740 1 -- 172.21.15.32:0/3395186499 >> 172.21.15.32:0/3395186499 conn(0x55ae30022460 msgr2=0x55ae303df770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff0abc79740 1 -- 172.21.15.32:0/3395186499 shutdown_connections 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.449+0000 7ff0abc79740 1 -- 172.21.15.32:0/3395186499 wait complete. 2024-08-31T07:25:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.450+0000 7ff0abc79740 1 -- start start 2024-08-31T07:25:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.450+0000 7ff0abc79740 1 Processor -- start 2024-08-31T07:25:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.450+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae304541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.450+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3000d390 0x55ae30454730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff0abc79740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3002e070 0x55ae3045f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff0abc79740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01b0700 con 0x55ae300051b0 2024-08-31T07:25:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff0abc79740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01affd0 con 0x55ae3002e070 2024-08-31T07:25:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff0abc79740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01ad230 con 0x55ae3000d390 2024-08-31T07:25:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff0abc79740 10 client.0 _ll_register_callbacks cb 0x7ff0880056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:06.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff09de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae304541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff097e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3000d390 0x55ae30454730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff09e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3002e070 0x55ae3045f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff09de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae304541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52186/0 (socket says 172.21.15.32:52186) 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff09de00640 1 -- 172.21.15.32:0/2624881915 learned_addr learned my addr 172.21.15.32:0/2624881915 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff09e800640 1 -- 172.21.15.32:0/2624881915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3000d390 msgr2=0x55ae30454730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff09e800640 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3000d390 0x55ae30454730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff09e800640 1 -- 172.21.15.32:0/2624881915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 msgr2=0x55ae304541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff09e800640 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae304541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff09e800640 1 -- 172.21.15.32:0/2624881915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3045f5b0 con 0x55ae3002e070 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.451+0000 7ff09de00640 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae304541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.452+0000 7ff09e800640 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3002e070 0x55ae3045f070 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff098006010 tx=0x7ff0980095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f27b56afb506a817 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.452+0000 7ff083e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.452+0000 7ff083e00640 1 -- 172.21.15.32:0/2624881915 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098058060 con 0x55ae3002e070 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.452+0000 7ff083e00640 1 -- 172.21.15.32:0/2624881915 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8485+0+0 (secure 0 0 0) 0x7ff098004040 con 0x55ae3002e070 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.452+0000 7ff0abc79740 1 -- 172.21.15.32:0/2624881915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08801f040 con 0x55ae3002e070 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.452+0000 7ff083e00640 1 -- 172.21.15.32:0/2624881915 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff098004240 con 0x55ae3002e070 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.452+0000 7ff0abc79740 10 client.28067 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.452+0000 7ff0abc79740 1 -- 172.21.15.32:0/2624881915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ae30460780 con 0x55ae3002e070 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.452+0000 7ff083e00640 1 -- 172.21.15.32:0/2624881915 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08801f040 con 0x55ae3002e070 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.453+0000 7ff083e00640 1 -- 172.21.15.32:0/2624881915 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7ff09802fc30 con 0x55ae3002e070 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.453+0000 7ff0abc79740 10 client.28067 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.453+0000 7ff0abc79740 10 client.28067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.453+0000 7ff0abc79740 20 client.28067 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.453+0000 7ff0abc79740 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ae2ffc1da0 0x55ae30463030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:06.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.453+0000 7ff0abc79740 4 client.28067 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.453+0000 7ff0abc79740 1 -- 172.21.15.32:0/2624881915 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae30348160 con 0x55ae2ffc1da0 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.453+0000 7ff09de00640 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ae2ffc1da0 0x55ae30463030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff09de00640 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ae2ffc1da0 0x55ae30463030 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=500f4b12f7d7c536 server_cookie=5efc7900a6e7e552 in_seq=0 out_seq=0 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff083e00640 10 client.28067 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff083e00640 1 -- 172.21.15.32:0/2624881915 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x55ae30348160 con 0x55ae2ffc1da0 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff083e00640 10 client.28067 handle_command_reply: tid=0 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff0abc79740 1 client.28067 shutdown 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff0abc79740 1 -- 172.21.15.32:0/2624881915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3002e070 msgr2=0x55ae3045f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff0abc79740 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3002e070 0x55ae3045f070 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff098006010 tx=0x7ff0980095c0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff0abc79740 1 -- 172.21.15.32:0/2624881915 shutdown_connections 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff0abc79740 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55ae2ffc1da0 0x55ae30463030 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff0abc79740 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae3002e070 0x55ae3045f070 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff0abc79740 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ae3000d390 0x55ae30454730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff0abc79740 1 --2- 172.21.15.32:0/2624881915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ae300051b0 0x55ae304541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff0abc79740 1 -- 172.21.15.32:0/2624881915 >> 172.21.15.32:0/2624881915 conn(0x55ae30022460 msgr2=0x55ae303d8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:06.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff083e00640 0 client.28067 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:06.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.454+0000 7ff0abc79740 1 -- 172.21.15.32:0/2624881915 shutdown_connections 2024-08-31T07:25:06.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff0abc79740 1 -- 172.21.15.32:0/2624881915 wait complete. 2024-08-31T07:25:06.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff0abc79740 20 client.28067 trim_cache size 0 max 16384 2024-08-31T07:25:06.459 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:06 smithi032 ceph-mon[17252]: pgmap v751: 97 pgs: 97 active+clean; 303 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.460 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.461 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.462 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:25:06.463 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:25:06.464 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.466 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.467 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:25:06.468 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff09f200640 1 -- 172.21.15.32:0/2199139321 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0840b4ca0 msgr2=0x7ff0840b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff09f200640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0840b4ca0 0x7ff0840b71b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff09802f500 tx=0x7ff09802f310 comp rx=0 tx=0).stop 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff09f200640 1 -- 172.21.15.32:0/2199139321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0103750 msgr2=0x7ff0a019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff09f200640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0103750 0x7ff0a019a020 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff09000e9f0 tx=0x7ff09000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff09f200640 1 -- 172.21.15.32:0/2199139321 shutdown_connections 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff09f200640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0840b4ca0 0x7ff0840b71b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff09f200640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a01099e0 0x7ff0a019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff09f200640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a0109090 0x7ff0a019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff09f200640 1 --2- 172.21.15.32:0/2199139321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0a0103750 0x7ff0a019a020 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.455+0000 7ff09f200640 1 -- 172.21.15.32:0/2199139321 >> 172.21.15.32:0/2199139321 conn(0x7ff0a00ff1a0 msgr2=0x7ff0a01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.456+0000 7ff09f200640 1 -- 172.21.15.32:0/2199139321 shutdown_connections 2024-08-31T07:25:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:06.456+0000 7ff09f200640 1 -- 172.21.15.32:0/2199139321 wait complete. 2024-08-31T07:25:06.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:06 smithi084 ceph-mon[22387]: pgmap v751: 97 pgs: 97 active+clean; 303 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2024-08-31T07:25:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:06 smithi073 ceph-mon[21995]: pgmap v751: 97 pgs: 97 active+clean; 303 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2024-08-31T07:25:06.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 375983 2.75 MB/sec execute 535 sec latency 734.402 ms 2024-08-31T07:25:07.492 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:25:07.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:25:07.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 376093 2.74 MB/sec execute 536 sec latency 1589.936 ms 2024-08-31T07:25:07.880 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:08.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:08 smithi032 ceph-mon[17252]: pgmap v752: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-08-31T07:25:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:08 smithi084 ceph-mon[22387]: pgmap v752: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-08-31T07:25:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.538+0000 7f8bb5e00640 1 -- 172.21.15.32:0/1056962381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb010bbe0 msgr2=0x7f8bb0112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.538+0000 7f8bb5e00640 1 --2- 172.21.15.32:0/1056962381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb010bbe0 0x7f8bb0112410 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009ca0 tx=0x7f8ba802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.538+0000 7f8bb5e00640 1 -- 172.21.15.32:0/1056962381 shutdown_connections 2024-08-31T07:25:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.538+0000 7f8bb5e00640 1 --2- 172.21.15.32:0/1056962381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb010bbe0 0x7f8bb0112410 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.538+0000 7f8bb5e00640 1 --2- 172.21.15.32:0/1056962381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb010b290 0x7f8bb010b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.538+0000 7f8bb5e00640 1 --2- 172.21.15.32:0/1056962381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb0105950 0x7f8bb0105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.538+0000 7f8bb5e00640 1 -- 172.21.15.32:0/1056962381 >> 172.21.15.32:0/1056962381 conn(0x7f8bb0101420 msgr2=0x7f8bb0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.538+0000 7f8bb5e00640 1 -- 172.21.15.32:0/1056962381 shutdown_connections 2024-08-31T07:25:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.538+0000 7f8bb5e00640 1 -- 172.21.15.32:0/1056962381 wait complete. 2024-08-31T07:25:08.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.539+0000 7f8bb5e00640 1 Processor -- start 2024-08-31T07:25:08.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.539+0000 7f8bb5e00640 1 -- start start 2024-08-31T07:25:08.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bb5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105950 0x7f8bb019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:08.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bb5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb010b290 0x7f8bb019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:08.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bb5e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb010bbe0 0x7f8bb01a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:08.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bb5e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01158a0 con 0x7f8bb0105950 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bb5e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0115720 con 0x7f8bb010b290 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bb5e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0115a20 con 0x7f8bb010bbe0 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bb4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105950 0x7f8bb019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bb4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105950 0x7f8bb019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52206/0 (socket says 172.21.15.32:52206) 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bb4a00640 1 -- 172.21.15.32:0/2243235641 learned_addr learned my addr 172.21.15.32:0/2243235641 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bafe00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb010b290 0x7f8bb019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.540+0000 7f8bb5400640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb010bbe0 0x7f8bb01a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bb4a00640 1 -- 172.21.15.32:0/2243235641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb010bbe0 msgr2=0x7f8bb01a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bb4a00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb010bbe0 0x7f8bb01a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bb4a00640 1 -- 172.21.15.32:0/2243235641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb010b290 msgr2=0x7f8bb019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bb4a00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb010b290 0x7f8bb019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bb4a00640 1 -- 172.21.15.32:0/2243235641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb01a2200 con 0x7f8bb0105950 2024-08-31T07:25:08.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bafe00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb010b290 0x7f8bb019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:08.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bb4a00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105950 0x7f8bb019c290 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400e9f0 tx=0x7f8ba400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:08.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bad600640 1 -- 172.21.15.32:0/2243235641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400cd80 con 0x7f8bb0105950 2024-08-31T07:25:08.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bb5e00640 1 -- 172.21.15.32:0/2243235641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb01a24f0 con 0x7f8bb0105950 2024-08-31T07:25:08.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bad600640 1 -- 172.21.15.32:0/2243235641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba40054d0 con 0x7f8bb0105950 2024-08-31T07:25:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bad600640 1 -- 172.21.15.32:0/2243235641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4010690 con 0x7f8bb0105950 2024-08-31T07:25:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.541+0000 7f8bb5e00640 1 -- 172.21.15.32:0/2243235641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb01a8c50 con 0x7f8bb0105950 2024-08-31T07:25:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.543+0000 7f8bb5e00640 1 -- 172.21.15.32:0/2243235641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb0105d60 con 0x7f8bb0105950 2024-08-31T07:25:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.545+0000 7f8bad600640 1 -- 172.21.15.32:0/2243235641 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8ba4004e30 con 0x7f8bb0105950 2024-08-31T07:25:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.546+0000 7f8bad600640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8b980b4d20 0x7f8b980b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.547+0000 7f8bad600640 1 -- 172.21.15.32:0/2243235641 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8485+0+0 (secure 0 0 0) 0x7f8ba40e5b00 con 0x7f8bb0105950 2024-08-31T07:25:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.547+0000 7f8bafe00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8b980b4d20 0x7f8b980b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.548+0000 7f8bafe00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8b980b4d20 0x7f8b980b7230 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0107bb0 tx=0x7f8b9c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:08.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.549+0000 7f8bad600640 1 -- 172.21.15.32:0/2243235641 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8ba4056130 con 0x7f8bb0105950 2024-08-31T07:25:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:08 smithi073 ceph-mon[21995]: pgmap v752: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-08-31T07:25:08.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 376093 2.74 MB/sec execute 537 sec latency 1090.045 ms 2024-08-31T07:25:08.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.782+0000 7f8bb5e00640 1 -- 172.21.15.32:0/2243235641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bb019d3c0 con 0x7f8bb0105950 2024-08-31T07:25:08.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.782+0000 7f8bad600640 1 -- 172.21.15.32:0/2243235641 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11970 (secure 0 0 0) 0x7f8ba40ac8b0 con 0x7f8bb0105950 2024-08-31T07:25:08.784 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:25:08.784 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"btime":"2024-08-31T07:25:04:192234+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:25:03.382644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:25:08.784 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2024-08-31T07:25:08.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.785+0000 7f8bb5e00640 1 -- 172.21.15.32:0/2243235641 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8b980b4d20 msgr2=0x7f8b980b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:08.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.785+0000 7f8bb5e00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8b980b4d20 0x7f8b980b7230 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0107bb0 tx=0x7f8b9c005f90 comp rx=0 tx=0).stop 2024-08-31T07:25:08.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.785+0000 7f8bb5e00640 1 -- 172.21.15.32:0/2243235641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105950 msgr2=0x7f8bb019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.785+0000 7f8bb5e00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105950 0x7f8bb019c290 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400e9f0 tx=0x7f8ba400edf0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.785+0000 7f8bb5e00640 1 -- 172.21.15.32:0/2243235641 shutdown_connections 2024-08-31T07:25:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.785+0000 7f8bb5e00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8b980b4d20 0x7f8b980b7230 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.785+0000 7f8bb5e00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb010bbe0 0x7f8bb01a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.785+0000 7f8bb5e00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb010b290 0x7f8bb019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.785+0000 7f8bb5e00640 1 --2- 172.21.15.32:0/2243235641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb0105950 0x7f8bb019c290 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.785+0000 7f8bb5e00640 1 -- 172.21.15.32:0/2243235641 >> 172.21.15.32:0/2243235641 conn(0x7f8bb0101420 msgr2=0x7f8bb0103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.786+0000 7f8bb5e00640 1 -- 172.21.15.32:0/2243235641 shutdown_connections 2024-08-31T07:25:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:08.786+0000 7f8bb5e00640 1 -- 172.21.15.32:0/2243235641 wait complete. 2024-08-31T07:25:09.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:09 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2243235641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:25:09.497 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:09 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2243235641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:25:09.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:25:09.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:09 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2243235641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:25:09.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 376093 2.73 MB/sec execute 538 sec latency 2090.107 ms 2024-08-31T07:25:09.918 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:10.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:10 smithi032 ceph-mon[17252]: pgmap v753: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 178 op/s 2024-08-31T07:25:10.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:10 smithi084 ceph-mon[22387]: pgmap v753: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 178 op/s 2024-08-31T07:25:10.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.600+0000 7ffb54000640 1 -- 172.21.15.32:0/4038648388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4c103900 msgr2=0x7ffb4c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:10.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.600+0000 7ffb54000640 1 --2- 172.21.15.32:0/4038648388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4c103900 0x7ffb4c103d10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009900 tx=0x7ffb4002a400 comp rx=0 tx=0).stop 2024-08-31T07:25:10.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.600+0000 7ffb54000640 1 -- 172.21.15.32:0/4038648388 shutdown_connections 2024-08-31T07:25:10.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.600+0000 7ffb54000640 1 --2- 172.21.15.32:0/4038648388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb4c109b90 0x7ffb4c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.600+0000 7ffb54000640 1 --2- 172.21.15.32:0/4038648388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c109240 0x7ffb4c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.600+0000 7ffb54000640 1 --2- 172.21.15.32:0/4038648388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4c103900 0x7ffb4c103d10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.600+0000 7ffb54000640 1 -- 172.21.15.32:0/4038648388 >> 172.21.15.32:0/4038648388 conn(0x7ffb4c07b5f0 msgr2=0x7ffb4c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:10.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.600+0000 7ffb54000640 1 -- 172.21.15.32:0/4038648388 shutdown_connections 2024-08-31T07:25:10.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.600+0000 7ffb54000640 1 -- 172.21.15.32:0/4038648388 wait complete. 2024-08-31T07:25:10.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.601+0000 7ffb54000640 1 Processor -- start 2024-08-31T07:25:10.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb54000640 1 -- start start 2024-08-31T07:25:10.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb54000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4c103900 0x7ffb4c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb54000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb4c109240 0x7ffb4c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb54000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c109b90 0x7ffb4c1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb54000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c1138a0 con 0x7ffb4c109b90 2024-08-31T07:25:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb54000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c113720 con 0x7ffb4c103900 2024-08-31T07:25:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb54000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c113a20 con 0x7ffb4c109240 2024-08-31T07:25:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb51600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4c103900 0x7ffb4c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb50c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb4c109240 0x7ffb4c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb51600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4c103900 0x7ffb4c19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42096/0 (socket says 172.21.15.32:42096) 2024-08-31T07:25:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb51600640 1 -- 172.21.15.32:0/242905595 learned_addr learned my addr 172.21.15.32:0/242905595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.602+0000 7ffb52000640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c109b90 0x7ffb4c1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.603+0000 7ffb51600640 1 -- 172.21.15.32:0/242905595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb4c109240 msgr2=0x7ffb4c19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.603+0000 7ffb51600640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb4c109240 0x7ffb4c19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.603+0000 7ffb51600640 1 -- 172.21.15.32:0/242905595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c109b90 msgr2=0x7ffb4c1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.603+0000 7ffb51600640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c109b90 0x7ffb4c1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.603+0000 7ffb51600640 1 -- 172.21.15.32:0/242905595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c1a2420 con 0x7ffb4c103900 2024-08-31T07:25:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.603+0000 7ffb51600640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4c103900 0x7ffb4c19c4b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ffb40005ab0 tx=0x7ffb40003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.603+0000 7ffb4a000640 1 -- 172.21.15.32:0/242905595 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4003f070 con 0x7ffb4c103900 2024-08-31T07:25:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.603+0000 7ffb4a000640 1 -- 172.21.15.32:0/242905595 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb40031b60 con 0x7ffb4c103900 2024-08-31T07:25:10.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.603+0000 7ffb4a000640 1 -- 172.21.15.32:0/242905595 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40035030 con 0x7ffb4c103900 2024-08-31T07:25:10.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.603+0000 7ffb54000640 1 -- 172.21.15.32:0/242905595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4c1a26b0 con 0x7ffb4c103900 2024-08-31T07:25:10.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.604+0000 7ffb54000640 1 -- 172.21.15.32:0/242905595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4c1a8e70 con 0x7ffb4c103900 2024-08-31T07:25:10.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.607+0000 7ffb5ac7e740 1 Processor -- start 2024-08-31T07:25:10.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.607+0000 7ffb5ac7e740 1 -- start start 2024-08-31T07:25:10.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.607+0000 7ffb4a000640 1 -- 172.21.15.32:0/242905595 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ffb40042070 con 0x7ffb4c103900 2024-08-31T07:25:10.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb17797df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb17798330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55eb1770e7b0 con 0x55eb173d1390 2024-08-31T07:25:10.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55eb176fdd40 con 0x55eb173f2070 2024-08-31T07:25:10.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55eb17701cf0 con 0x55eb173c91b0 2024-08-31T07:25:10.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb52000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb52000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42110/0 (socket says 172.21.15.32:42110) 2024-08-31T07:25:10.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb51600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb17797df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb52000640 1 -- 172.21.15.32:0/3140655552 learned_addr learned my addr 172.21.15.32:0/3140655552 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb50c00640 1 --2- 172.21.15.32:0/3140655552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb17798330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb52000640 1 -- 172.21.15.32:0/3140655552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 msgr2=0x55eb17797df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb52000640 1 --2- 172.21.15.32:0/3140655552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb17797df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb52000640 1 -- 172.21.15.32:0/3140655552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 msgr2=0x55eb17798330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb52000640 1 --2- 172.21.15.32:0/3140655552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb17798330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb52000640 1 -- 172.21.15.32:0/3140655552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1745d650 con 0x55eb173f2070 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb51600640 1 --2- 172.21.15.32:0/3140655552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb17797df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.608+0000 7ffb50c00640 1 --2- 172.21.15.32:0/3140655552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb17798330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.609+0000 7ffb52000640 1 --2- 172.21.15.32:0/3140655552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177978b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009f60 tx=0x7ffb44009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79c54d8b29ead52 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.609+0000 7ffb3e600640 1 -- 172.21.15.32:0/3140655552 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4401e070 con 0x55eb173f2070 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.609+0000 7ffb3e600640 1 -- 172.21.15.32:0/3140655552 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb44003c30 con 0x55eb173f2070 2024-08-31T07:25:10.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.609+0000 7ffb3e600640 1 -- 172.21.15.32:0/3140655552 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4400f820 con 0x55eb173f2070 2024-08-31T07:25:10.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.609+0000 7ffb4a000640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb340b4d20 0x7ffb340b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.610+0000 7ffb4a000640 1 -- 172.21.15.32:0/242905595 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8485+0+0 (secure 0 0 0) 0x7ffb40105ba0 con 0x7ffb4c103900 2024-08-31T07:25:10.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.610+0000 7ffb51600640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb340b4d20 0x7ffb340b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.610+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/3140655552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 msgr2=0x55eb177978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:10.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.610+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/3140655552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177978b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009f60 tx=0x7ffb44009f90 comp rx=0 tx=0).stop 2024-08-31T07:25:10.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.610+0000 7ffb51600640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb340b4d20 0x7ffb340b7230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55eb17796530 tx=0x7ffb40109650 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.611+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/3140655552 shutdown_connections 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.611+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/3140655552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb17798330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.611+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/3140655552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb17797df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.611+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/3140655552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177978b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.611+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/3140655552 >> 172.21.15.32:0/3140655552 conn(0x55eb173e6460 msgr2=0x55eb177957a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.611+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/3140655552 shutdown_connections 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.611+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/3140655552 wait complete. 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.611+0000 7ffb5ac7e740 1 -- start start 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb5ac7e740 1 Processor -- start 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177a3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb177a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55eb17560700 con 0x55eb173d1390 2024-08-31T07:25:10.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55eb17684cf0 con 0x55eb173f2070 2024-08-31T07:25:10.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55eb177a6090 con 0x55eb173c91b0 2024-08-31T07:25:10.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb5ac7e740 10 client.0 _ll_register_callbacks cb 0x55eb176aa470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:10.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb52000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb52000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177a3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37952/0 (socket says 172.21.15.32:37952) 2024-08-31T07:25:10.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb52000640 1 -- 172.21.15.32:0/2111712328 learned_addr learned my addr 172.21.15.32:0/2111712328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:10.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb50c00640 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb177a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.612+0000 7ffb52000640 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.613+0000 7ffb52000640 1 -- 172.21.15.32:0/2111712328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 msgr2=0x55eb177af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.613+0000 7ffb52000640 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.613+0000 7ffb52000640 1 -- 172.21.15.32:0/2111712328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 msgr2=0x55eb177a39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.613+0000 7ffb52000640 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb177a39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.613+0000 7ffb52000640 1 -- 172.21.15.32:0/2111712328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb176c58b0 con 0x55eb173c91b0 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.613+0000 7ffb52000640 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177a3460 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009aa0 tx=0x7ffb4400f480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=144bfcd5c058ee55 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.613+0000 7ffb32a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.613+0000 7ffb32a00640 1 -- 172.21.15.32:0/2111712328 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4401e030 con 0x55eb173c91b0 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.613+0000 7ffb32a00640 1 -- 172.21.15.32:0/2111712328 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8485+0+0 (secure 0 0 0) 0x7ffb4400f560 con 0x55eb173c91b0 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.613+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/2111712328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb44003c30 con 0x55eb173c91b0 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb32a00640 1 -- 172.21.15.32:0/2111712328 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ffb4400f760 con 0x55eb173c91b0 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb5ac7e740 10 client.28090 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/2111712328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55eb177b2100 con 0x55eb173c91b0 2024-08-31T07:25:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb32a00640 1 -- 172.21.15.32:0/2111712328 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb44003c30 con 0x55eb173c91b0 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb32a00640 1 -- 172.21.15.32:0/2111712328 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7ffb440197e0 con 0x55eb173c91b0 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb5ac7e740 10 client.28090 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb5ac7e740 10 client.28090 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb5ac7e740 20 client.28090 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eb173b09f0 0x55eb177b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb5ac7e740 4 client.28090 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.614+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/2111712328 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb17710700 con 0x55eb173b09f0 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.615+0000 7ffb50c00640 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eb173b09f0 0x55eb177b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.615+0000 7ffb50c00640 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eb173b09f0 0x55eb177b5570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=edd26e206d323cfa server_cookie=bfa10fe2cfb2925f in_seq=0 out_seq=0 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.615+0000 7ffb32a00640 10 client.28090 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb32a00640 1 -- 172.21.15.32:0/2111712328 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eb17710700 con 0x55eb173b09f0 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb32a00640 10 client.28090 handle_command_reply: tid=0 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 client.28090 shutdown 2024-08-31T07:25:10.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/2111712328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 msgr2=0x55eb177a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177a3460 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009aa0 tx=0x7ffb4400f480 comp rx=0 tx=0).stop 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/2111712328 shutdown_connections 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eb173b09f0 0x55eb177b5570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb177a39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb32a00640 0 client.28090 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/2111712328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177a3460 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/2111712328 >> 172.21.15.32:0/2111712328 conn(0x55eb173e6460 msgr2=0x55eb1779aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/2111712328 shutdown_connections 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.616+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/2111712328 wait complete. 2024-08-31T07:25:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.617+0000 7ffb5ac7e740 20 client.28090 trim_cache size 0 max 16384 2024-08-31T07:25:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.632+0000 7ffb5ac7e740 1 Processor -- start 2024-08-31T07:25:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.632+0000 7ffb5ac7e740 1 -- start start 2024-08-31T07:25:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173d1390 0x55eb177a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173f2070 0x55eb177af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c1b1670 con 0x55eb173f2070 2024-08-31T07:25:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c1b32d0 con 0x55eb173d1390 2024-08-31T07:25:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c1b1d40 con 0x55eb173c91b0 2024-08-31T07:25:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb50c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173d1390 0x55eb177a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb50c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173d1390 0x55eb177a4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42128/0 (socket says 172.21.15.32:42128) 2024-08-31T07:25:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb50c00640 1 -- 172.21.15.32:0/1438825101 learned_addr learned my addr 172.21.15.32:0/1438825101 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb52000640 1 --2- 172.21.15.32:0/1438825101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.633+0000 7ffb52000640 1 --2- 172.21.15.32:0/1438825101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173f2070 0x55eb177af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.634+0000 7ffb50c00640 1 -- 172.21.15.32:0/1438825101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 msgr2=0x55eb177947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.634+0000 7ffb50c00640 1 --2- 172.21.15.32:0/1438825101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.634+0000 7ffb50c00640 1 -- 172.21.15.32:0/1438825101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173f2070 msgr2=0x55eb177af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.634+0000 7ffb50c00640 1 --2- 172.21.15.32:0/1438825101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173f2070 0x55eb177af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.634+0000 7ffb50c00640 1 -- 172.21.15.32:0/1438825101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1779a710 con 0x55eb173d1390 2024-08-31T07:25:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.634+0000 7ffb50c00640 1 --2- 172.21.15.32:0/1438825101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173d1390 0x55eb177a4e90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009c20 tx=0x7ffb38005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af292e1eb483b12d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.634+0000 7ffb3dc00640 1 -- 172.21.15.32:0/1438825101 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb38004310 con 0x55eb173d1390 2024-08-31T07:25:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.634+0000 7ffb3dc00640 1 -- 172.21.15.32:0/1438825101 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb380044b0 con 0x55eb173d1390 2024-08-31T07:25:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.634+0000 7ffb3dc00640 1 -- 172.21.15.32:0/1438825101 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb38025990 con 0x55eb173d1390 2024-08-31T07:25:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.635+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1438825101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173d1390 msgr2=0x55eb177a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.635+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/1438825101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173d1390 0x55eb177a4e90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009c20 tx=0x7ffb38005f90 comp rx=0 tx=0).stop 2024-08-31T07:25:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.635+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1438825101 shutdown_connections 2024-08-31T07:25:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.635+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/1438825101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173f2070 0x55eb177af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.635+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/1438825101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173d1390 0x55eb177a4e90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.635+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/1438825101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.635+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1438825101 >> 172.21.15.32:0/1438825101 conn(0x55eb173e6460 msgr2=0x55eb177a4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.636+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1438825101 shutdown_connections 2024-08-31T07:25:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.636+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1438825101 wait complete. 2024-08-31T07:25:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.636+0000 7ffb5ac7e740 1 -- start start 2024-08-31T07:25:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.636+0000 7ffb5ac7e740 1 Processor -- start 2024-08-31T07:25:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb177a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb5ac7e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177a7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c1b2b40 con 0x55eb173d1390 2024-08-31T07:25:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c1b2410 con 0x55eb173f2070 2024-08-31T07:25:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb5ac7e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c1af670 con 0x55eb173c91b0 2024-08-31T07:25:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb5ac7e740 10 client.0 _ll_register_callbacks cb 0x7ffb44004d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb52000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb50c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb177a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb52000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177abbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37980/0 (socket says 172.21.15.32:37980) 2024-08-31T07:25:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb52000640 1 -- 172.21.15.32:0/1480317164 learned_addr learned my addr 172.21.15.32:0/1480317164 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.637+0000 7ffb52000640 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.638+0000 7ffb50c00640 1 -- 172.21.15.32:0/1480317164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 msgr2=0x55eb177abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.638+0000 7ffb50c00640 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.638+0000 7ffb50c00640 1 -- 172.21.15.32:0/1480317164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 msgr2=0x55eb177a7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:10.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.638+0000 7ffb50c00640 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177a7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.638+0000 7ffb50c00640 1 -- 172.21.15.32:0/1480317164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb177a7dd0 con 0x55eb173d1390 2024-08-31T07:25:10.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.638+0000 7ffb50c00640 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb177a7070 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55eb176b2910 tx=0x7ffb380043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9040950fbef4647d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:10.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.638+0000 7ffb32a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:25:10.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.638+0000 7ffb32a00640 1 -- 172.21.15.32:0/1480317164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3802a070 con 0x55eb173d1390 2024-08-31T07:25:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.638+0000 7ffb32a00640 1 -- 172.21.15.32:0/1480317164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8485+0+0 (secure 0 0 0) 0x7ffb38005230 con 0x55eb173d1390 2024-08-31T07:25:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.638+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1480317164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb380044b0 con 0x55eb173d1390 2024-08-31T07:25:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.639+0000 7ffb32a00640 1 -- 172.21.15.32:0/1480317164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ffb38005470 con 0x55eb173d1390 2024-08-31T07:25:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.639+0000 7ffb5ac7e740 10 client.18093 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.639+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1480317164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55eb1781fd70 con 0x55eb173d1390 2024-08-31T07:25:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.639+0000 7ffb32a00640 1 -- 172.21.15.32:0/1480317164 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb380044b0 con 0x55eb173d1390 2024-08-31T07:25:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.639+0000 7ffb32a00640 1 -- 172.21.15.32:0/1480317164 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7ffb38035020 con 0x55eb173d1390 2024-08-31T07:25:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.639+0000 7ffb5ac7e740 10 client.18093 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.639+0000 7ffb5ac7e740 10 client.18093 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:25:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.639+0000 7ffb5ac7e740 20 client.18093 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.640+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eb17385da0 0x55eb17822460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:10.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.640+0000 7ffb5ac7e740 4 client.18093 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:25:10.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.640+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1480317164 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb1770c1a0 con 0x55eb17385da0 2024-08-31T07:25:10.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.640+0000 7ffb52000640 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eb17385da0 0x55eb17822460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:10.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.640+0000 7ffb52000640 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eb17385da0 0x55eb17822460 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5fb3ab411a936e33 server_cookie=ff77331cd7a07590 in_seq=0 out_seq=0 2024-08-31T07:25:10.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.640+0000 7ffb32a00640 10 client.18093 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:25:10.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.640+0000 7ffb32a00640 1 -- 172.21.15.32:0/1480317164 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb1770c1a0 con 0x55eb17385da0 2024-08-31T07:25:10.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.640+0000 7ffb32a00640 10 client.18093 handle_command_reply: tid=0 2024-08-31T07:25:10.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.640+0000 7ffb5ac7e740 1 client.18093 shutdown 2024-08-31T07:25:10.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1480317164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 msgr2=0x55eb177a7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:10.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb177a7070 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55eb176b2910 tx=0x7ffb380043b0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1480317164 shutdown_connections 2024-08-31T07:25:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55eb17385da0 0x55eb17822460 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb173f2070 0x55eb177a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb173d1390 0x55eb177a7070 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 1 --2- 172.21.15.32:0/1480317164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb173c91b0 0x55eb177abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1480317164 >> 172.21.15.32:0/1480317164 conn(0x55eb173e6460 msgr2=0x55eb17794570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:10.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb32a00640 0 client.18093 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:25:10.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1480317164 shutdown_connections 2024-08-31T07:25:10.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 1 -- 172.21.15.32:0/1480317164 wait complete. 2024-08-31T07:25:10.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.641+0000 7ffb5ac7e740 20 client.18093 trim_cache size 0 max 16384 2024-08-31T07:25:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:25:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 -- 172.21.15.32:0/242905595 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb340b4d20 msgr2=0x7ffb340b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb340b4d20 0x7ffb340b7230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55eb17796530 tx=0x7ffb40109650 comp rx=0 tx=0).stop 2024-08-31T07:25:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 -- 172.21.15.32:0/242905595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4c103900 msgr2=0x7ffb4c19c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4c103900 0x7ffb4c19c4b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ffb40005ab0 tx=0x7ffb40003c10 comp rx=0 tx=0).stop 2024-08-31T07:25:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 -- 172.21.15.32:0/242905595 shutdown_connections 2024-08-31T07:25:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffb340b4d20 0x7ffb340b7230 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4c109b90 0x7ffb4c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb4c109240 0x7ffb4c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 --2- 172.21.15.32:0/242905595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb4c103900 0x7ffb4c19c4b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 -- 172.21.15.32:0/242905595 >> 172.21.15.32:0/242905595 conn(0x7ffb4c07b5f0 msgr2=0x7ffb4c1070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 -- 172.21.15.32:0/242905595 shutdown_connections 2024-08-31T07:25:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:10.642+0000 7ffb54000640 1 -- 172.21.15.32:0/242905595 wait complete. 2024-08-31T07:25:10.678 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:10 smithi073 ceph-mon[21995]: pgmap v753: 97 pgs: 97 active+clean; 304 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 178 op/s 2024-08-31T07:25:10.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 376764 2.73 MB/sec execute 539 sec latency 2281.158 ms 2024-08-31T07:25:11.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:25:11.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 377666 2.74 MB/sec execute 540 sec latency 364.886 ms 2024-08-31T07:25:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:12.472 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:12 smithi032 ceph-mon[17252]: pgmap v754: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-08-31T07:25:12.472 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:12 smithi032 ceph-mon[17252]: osdmap e94: 12 total, 12 up, 12 in 2024-08-31T07:25:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:12 smithi084 ceph-mon[22387]: pgmap v754: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-08-31T07:25:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:12 smithi084 ceph-mon[22387]: osdmap e94: 12 total, 12 up, 12 in 2024-08-31T07:25:12.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:12 smithi073 ceph-mon[21995]: pgmap v754: 97 pgs: 97 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-08-31T07:25:12.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:12 smithi073 ceph-mon[21995]: osdmap e94: 12 total, 12 up, 12 in 2024-08-31T07:25:12.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 377666 2.73 MB/sec execute 541 sec latency 1364.944 ms 2024-08-31T07:25:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.791+0000 7fc465200640 1 -- 172.21.15.32:0/3251892731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc460103900 msgr2=0x7fc460103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.791+0000 7fc465200640 1 --2- 172.21.15.32:0/3251892731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc460103900 0x7fc460103d10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc450009900 tx=0x7fc45002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.791+0000 7fc465200640 1 -- 172.21.15.32:0/3251892731 shutdown_connections 2024-08-31T07:25:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.791+0000 7fc465200640 1 --2- 172.21.15.32:0/3251892731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc460109b90 0x7fc4601103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.791+0000 7fc465200640 1 --2- 172.21.15.32:0/3251892731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc460109240 0x7fc460109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.791+0000 7fc465200640 1 --2- 172.21.15.32:0/3251892731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc460103900 0x7fc460103d10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.791+0000 7fc465200640 1 -- 172.21.15.32:0/3251892731 >> 172.21.15.32:0/3251892731 conn(0x7fc46007b5f0 msgr2=0x7fc46007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.791+0000 7fc465200640 1 -- 172.21.15.32:0/3251892731 shutdown_connections 2024-08-31T07:25:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.791+0000 7fc465200640 1 -- 172.21.15.32:0/3251892731 wait complete. 2024-08-31T07:25:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.792+0000 7fc465200640 1 Processor -- start 2024-08-31T07:25:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.792+0000 7fc465200640 1 -- start start 2024-08-31T07:25:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc465200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc460103900 0x7fc46019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc465200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc460109240 0x7fc46019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc465200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc460109b90 0x7fc46019fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc465200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc460113850 con 0x7fc460109b90 2024-08-31T07:25:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc465200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601136d0 con 0x7fc460109240 2024-08-31T07:25:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc465200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601139d0 con 0x7fc460103900 2024-08-31T07:25:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc45e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc460109b90 0x7fc46019fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc45e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc460109b90 0x7fc46019fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52282/0 (socket says 172.21.15.32:52282) 2024-08-31T07:25:12.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc45e800640 1 -- 172.21.15.32:0/3042687301 learned_addr learned my addr 172.21.15.32:0/3042687301 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:12.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc455400640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc460109240 0x7fc46019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.793+0000 7fc45de00640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc460103900 0x7fc46019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.794+0000 7fc45e800640 1 -- 172.21.15.32:0/3042687301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc460103900 msgr2=0x7fc46019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.794+0000 7fc45e800640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc460103900 0x7fc46019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.794+0000 7fc45e800640 1 -- 172.21.15.32:0/3042687301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc460109240 msgr2=0x7fc46019a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.794+0000 7fc45e800640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc460109240 0x7fc46019a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.794+0000 7fc45e800640 1 -- 172.21.15.32:0/3042687301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4601a01f0 con 0x7fc460109b90 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.794+0000 7fc455400640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc460109240 0x7fc46019a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.794+0000 7fc45e800640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc460109b90 0x7fc46019fac0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc45800eb10 tx=0x7fc45800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.794+0000 7fc457400640 1 -- 172.21.15.32:0/3042687301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458004b20 con 0x7fc460109b90 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.794+0000 7fc457400640 1 -- 172.21.15.32:0/3042687301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc458004cc0 con 0x7fc460109b90 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.795+0000 7fc465200640 1 -- 172.21.15.32:0/3042687301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4601a04e0 con 0x7fc460109b90 2024-08-31T07:25:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.795+0000 7fc457400640 1 -- 172.21.15.32:0/3042687301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458010670 con 0x7fc460109b90 2024-08-31T07:25:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.795+0000 7fc465200640 1 -- 172.21.15.32:0/3042687301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4601a6c40 con 0x7fc460109b90 2024-08-31T07:25:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.798+0000 7fc46c023740 1 Processor -- start 2024-08-31T07:25:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.798+0000 7fc457400640 1 -- 172.21.15.32:0/3042687301 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc458005020 con 0x7fc460109b90 2024-08-31T07:25:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.798+0000 7fc46c023740 1 -- start start 2024-08-31T07:25:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc46c023740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef93c070 0x5614efce1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc46c023740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef9131b0 0x5614efce1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc46c023740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef91b390 0x5614efce2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc46c023740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5614efc587b0 con 0x5614ef9131b0 2024-08-31T07:25:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc46c023740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5614efc47d40 con 0x5614ef91b390 2024-08-31T07:25:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc46c023740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5614efc4bcf0 con 0x5614ef93c070 2024-08-31T07:25:12.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc45de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef9131b0 0x5614efce1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc45de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef9131b0 0x5614efce1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52298/0 (socket says 172.21.15.32:52298) 2024-08-31T07:25:12.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc45de00640 1 -- 172.21.15.32:0/694208938 learned_addr learned my addr 172.21.15.32:0/694208938 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:12.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc455400640 1 --2- 172.21.15.32:0/694208938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef93c070 0x5614efce1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.799+0000 7fc455400640 1 --2- 172.21.15.32:0/694208938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef91b390 0x5614efce2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc45de00640 1 -- 172.21.15.32:0/694208938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef93c070 msgr2=0x5614efce1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc45de00640 1 --2- 172.21.15.32:0/694208938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef93c070 0x5614efce1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc45de00640 1 -- 172.21.15.32:0/694208938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef91b390 msgr2=0x5614efce2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc45de00640 1 --2- 172.21.15.32:0/694208938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef91b390 0x5614efce2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc45de00640 1 -- 172.21.15.32:0/694208938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ef9a7650 con 0x5614ef9131b0 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc455400640 1 --2- 172.21.15.32:0/694208938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef91b390 0x5614efce2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc457400640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc4440b4d20 0x7fc4440b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc45de00640 1 --2- 172.21.15.32:0/694208938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef9131b0 0x5614efce1ed0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc450033000 tx=0x7fc450032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9fef8049cdcfdc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc457400640 1 -- 172.21.15.32:0/3042687301 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7fc4580e5aa0 con 0x7fc460109b90 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc455400640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc4440b4d20 0x7fc4440b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.800+0000 7fc44f400640 1 -- 172.21.15.32:0/694208938 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450035040 con 0x5614ef9131b0 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.801+0000 7fc44f400640 1 -- 172.21.15.32:0/694208938 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc450004030 con 0x5614ef9131b0 2024-08-31T07:25:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.801+0000 7fc44f400640 1 -- 172.21.15.32:0/694208938 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4500055e0 con 0x5614ef9131b0 2024-08-31T07:25:12.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.801+0000 7fc46c023740 1 -- 172.21.15.32:0/694208938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef9131b0 msgr2=0x5614efce1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:12.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.801+0000 7fc455400640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc4440b4d20 0x7fc4440b7230 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc4480058c0 tx=0x7fc448013040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:12.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.801+0000 7fc46c023740 1 --2- 172.21.15.32:0/694208938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef9131b0 0x5614efce1ed0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc450033000 tx=0x7fc450032b90 comp rx=0 tx=0).stop 2024-08-31T07:25:12.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.802+0000 7fc46c023740 1 -- 172.21.15.32:0/694208938 shutdown_connections 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.802+0000 7fc46c023740 1 --2- 172.21.15.32:0/694208938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef91b390 0x5614efce2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.802+0000 7fc46c023740 1 --2- 172.21.15.32:0/694208938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef9131b0 0x5614efce1ed0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.802+0000 7fc46c023740 1 --2- 172.21.15.32:0/694208938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef93c070 0x5614efce1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.802+0000 7fc46c023740 1 -- 172.21.15.32:0/694208938 >> 172.21.15.32:0/694208938 conn(0x5614ef930460 msgr2=0x5614efcdf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.802+0000 7fc46c023740 1 -- 172.21.15.32:0/694208938 shutdown_connections 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.802+0000 7fc46c023740 1 -- 172.21.15.32:0/694208938 wait complete. 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.802+0000 7fc46c023740 1 -- start start 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.802+0000 7fc46c023740 1 Processor -- start 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc46c023740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcf1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc46c023740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 0x5614efcf1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc46c023740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 0x5614efcfd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc46c023740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5614efaaa700 con 0x5614ef91b390 2024-08-31T07:25:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc46c023740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5614efbcecf0 con 0x5614ef93c070 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc46c023740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5614efcf4440 con 0x5614ef9131b0 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc46c023740 10 client.0 _ll_register_callbacks cb 0x5614efbf4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc45de00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcf1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc455400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 0x5614efcf1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc45de00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcf1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38012/0 (socket says 172.21.15.32:38012) 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc45de00640 1 -- 172.21.15.32:0/3773023988 learned_addr learned my addr 172.21.15.32:0/3773023988 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.803+0000 7fc45e800640 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 0x5614efcfd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.804+0000 7fc45de00640 1 -- 172.21.15.32:0/3773023988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 msgr2=0x5614efcfd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.804+0000 7fc45de00640 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 0x5614efcfd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.804+0000 7fc45de00640 1 -- 172.21.15.32:0/3773023988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 msgr2=0x5614efcf1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.804+0000 7fc45de00640 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 0x5614efcf1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.804+0000 7fc45de00640 1 -- 172.21.15.32:0/3773023988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614efc0f8b0 con 0x5614ef9131b0 2024-08-31T07:25:12.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.804+0000 7fc45e800640 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 0x5614efcfd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.804+0000 7fc45de00640 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcf1900 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc45002a910 tx=0x7fc450031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aefa4f12aaf08a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.804+0000 7fc443e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.804+0000 7fc443e00640 1 -- 172.21.15.32:0/3773023988 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450035070 con 0x5614ef9131b0 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.804+0000 7fc46c023740 1 -- 172.21.15.32:0/3773023988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc450004030 con 0x5614ef9131b0 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.805+0000 7fc443e00640 1 -- 172.21.15.32:0/3773023988 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7fc450005590 con 0x5614ef9131b0 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.805+0000 7fc443e00640 1 -- 172.21.15.32:0/3773023988 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc450005db0 con 0x5614ef9131b0 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.805+0000 7fc46c023740 10 client.28108 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.805+0000 7fc46c023740 1 -- 172.21.15.32:0/3773023988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5614efd00470 con 0x5614ef9131b0 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.805+0000 7fc443e00640 1 -- 172.21.15.32:0/3773023988 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc450004030 con 0x5614ef9131b0 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.805+0000 7fc443e00640 1 -- 172.21.15.32:0/3773023988 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7fc450004b90 con 0x5614ef9131b0 2024-08-31T07:25:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.805+0000 7fc46c023740 10 client.28108 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.805+0000 7fc46c023740 10 client.28108 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.805+0000 7fc46c023740 20 client.28108 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.806+0000 7fc46c023740 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5614ef8fa9f0 0x5614efd037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.806+0000 7fc46c023740 4 client.28108 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.806+0000 7fc46c023740 1 -- 172.21.15.32:0/3773023988 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614efc5a700 con 0x5614ef8fa9f0 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.806+0000 7fc455400640 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5614ef8fa9f0 0x5614efd037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.807+0000 7fc455400640 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5614ef8fa9f0 0x5614efd037d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c8108f49ce1f6573 server_cookie=3b4eca11762e1db1 in_seq=0 out_seq=0 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.807+0000 7fc443e00640 10 client.28108 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.807+0000 7fc443e00640 1 -- 172.21.15.32:0/3773023988 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5614efc5a700 con 0x5614ef8fa9f0 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.807+0000 7fc443e00640 10 client.28108 handle_command_reply: tid=0 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.807+0000 7fc46c023740 1 client.28108 shutdown 2024-08-31T07:25:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 1 -- 172.21.15.32:0/3773023988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 msgr2=0x5614efcf1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcf1900 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc45002a910 tx=0x7fc450031400 comp rx=0 tx=0).stop 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 1 -- 172.21.15.32:0/3773023988 shutdown_connections 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5614ef8fa9f0 0x5614efd037d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 0x5614efcfd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 0x5614efcf1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 1 --2- 172.21.15.32:0/3773023988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcf1900 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 1 -- 172.21.15.32:0/3773023988 >> 172.21.15.32:0/3773023988 conn(0x5614ef930460 msgr2=0x5614efce4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc443e00640 0 client.28108 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 1 -- 172.21.15.32:0/3773023988 shutdown_connections 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 1 -- 172.21.15.32:0/3773023988 wait complete. 2024-08-31T07:25:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.808+0000 7fc46c023740 20 client.28108 trim_cache size 0 max 16384 2024-08-31T07:25:12.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.822+0000 7fc46c023740 1 Processor -- start 2024-08-31T07:25:12.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.822+0000 7fc46c023740 1 -- start start 2024-08-31T07:25:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc46c023740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcdee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc46c023740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 0x5614efcdf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc46c023740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 0x5614efcee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc46c023740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601ad130 con 0x5614ef91b390 2024-08-31T07:25:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc46c023740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601aed90 con 0x5614ef93c070 2024-08-31T07:25:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc46c023740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601ad800 con 0x5614ef9131b0 2024-08-31T07:25:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc455400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 0x5614efcdf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc455400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 0x5614efcdf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52318/0 (socket says 172.21.15.32:52318) 2024-08-31T07:25:12.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc455400640 1 -- 172.21.15.32:0/1649802921 learned_addr learned my addr 172.21.15.32:0/1649802921 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:12.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc45de00640 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcdee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.823+0000 7fc45e800640 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 0x5614efcee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.824+0000 7fc455400640 1 -- 172.21.15.32:0/1649802921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 msgr2=0x5614efcdee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.824+0000 7fc455400640 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcdee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.824+0000 7fc455400640 1 -- 172.21.15.32:0/1649802921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 msgr2=0x5614efcee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.824+0000 7fc455400640 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 0x5614efcee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.824+0000 7fc455400640 1 -- 172.21.15.32:0/1649802921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614efced270 con 0x5614ef91b390 2024-08-31T07:25:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.824+0000 7fc45de00640 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcdee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.824+0000 7fc45e800640 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 0x5614efcee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.824+0000 7fc455400640 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 0x5614efcdf370 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc46006c500 tx=0x7fc448005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849e2464c2584620 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.824+0000 7fc44ea00640 1 -- 172.21.15.32:0/1649802921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448037070 con 0x5614ef91b390 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.825+0000 7fc44ea00640 1 -- 172.21.15.32:0/1649802921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc44801fdb0 con 0x5614ef91b390 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.825+0000 7fc44ea00640 1 -- 172.21.15.32:0/1649802921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448032d90 con 0x5614ef91b390 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.825+0000 7fc46c023740 1 -- 172.21.15.32:0/1649802921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 msgr2=0x5614efcdf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.825+0000 7fc46c023740 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 0x5614efcdf370 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc46006c500 tx=0x7fc448005850 comp rx=0 tx=0).stop 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.825+0000 7fc46c023740 1 -- 172.21.15.32:0/1649802921 shutdown_connections 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.825+0000 7fc46c023740 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef93c070 0x5614efcee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.825+0000 7fc46c023740 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef91b390 0x5614efcdf370 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.825+0000 7fc46c023740 1 --2- 172.21.15.32:0/1649802921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef9131b0 0x5614efcdee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.825+0000 7fc46c023740 1 -- 172.21.15.32:0/1649802921 >> 172.21.15.32:0/1649802921 conn(0x5614ef930460 msgr2=0x5614efce3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.826+0000 7fc46c023740 1 -- 172.21.15.32:0/1649802921 shutdown_connections 2024-08-31T07:25:12.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.826+0000 7fc46c023740 1 -- 172.21.15.32:0/1649802921 wait complete. 2024-08-31T07:25:12.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.826+0000 7fc46c023740 1 -- start start 2024-08-31T07:25:12.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.826+0000 7fc46c023740 1 Processor -- start 2024-08-31T07:25:12.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc46c023740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef9131b0 0x5614efd05330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc46c023740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef91b390 0x5614efd058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc46c023740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef93c070 0x5614efd65f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc46c023740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601ae600 con 0x5614ef93c070 2024-08-31T07:25:12.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc46c023740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601aded0 con 0x5614ef9131b0 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc46c023740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601ab110 con 0x5614ef91b390 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc46c023740 10 client.0 _ll_register_callbacks cb 0x7fc450005200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc45de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef9131b0 0x5614efd05330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc45e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef93c070 0x5614efd65f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc45de00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef9131b0 0x5614efd05330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42190/0 (socket says 172.21.15.32:42190) 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc45e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef93c070 0x5614efd65f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52328/0 (socket says 172.21.15.32:52328) 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc455400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef91b390 0x5614efd058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc45de00640 1 -- 172.21.15.32:0/267273088 learned_addr learned my addr 172.21.15.32:0/267273088 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc45de00640 1 -- 172.21.15.32:0/267273088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef91b390 msgr2=0x5614efd058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc45de00640 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef91b390 0x5614efd058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc45de00640 1 -- 172.21.15.32:0/267273088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef93c070 msgr2=0x5614efd65f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.827+0000 7fc45de00640 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef93c070 0x5614efd65f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.828+0000 7fc45de00640 1 -- 172.21.15.32:0/267273088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614efd665f0 con 0x5614ef9131b0 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.828+0000 7fc45e800640 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef93c070 0x5614efd65f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.828+0000 7fc45de00640 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef9131b0 0x5614efd05330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc450004f60 tx=0x7fc450032f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=185841c3b32aa948 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.828+0000 7fc443e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.828+0000 7fc443e00640 1 -- 172.21.15.32:0/267273088 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450035040 con 0x5614ef9131b0 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.828+0000 7fc46c023740 1 -- 172.21.15.32:0/267273088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44801fdb0 con 0x5614ef9131b0 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.828+0000 7fc443e00640 1 -- 172.21.15.32:0/267273088 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7fc450031de0 con 0x5614ef9131b0 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc443e00640 1 -- 172.21.15.32:0/267273088 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc450068050 con 0x5614ef9131b0 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc46c023740 10 client.28109 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc46c023740 1 -- 172.21.15.32:0/267273088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5614efd6eaf0 con 0x5614ef9131b0 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc443e00640 1 -- 172.21.15.32:0/267273088 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc44801fdb0 con 0x5614ef9131b0 2024-08-31T07:25:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc443e00640 1 -- 172.21.15.32:0/267273088 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7fc450060020 con 0x5614ef9131b0 2024-08-31T07:25:12.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc46c023740 10 client.28109 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:12.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc46c023740 10 client.28109 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:25:12.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc46c023740 20 client.28109 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:12.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc46c023740 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5614ef8cfda0 0x5614efd71180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:12.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc46c023740 4 client.28109 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:25:12.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc46c023740 1 -- 172.21.15.32:0/267273088 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614efc561a0 con 0x5614ef8cfda0 2024-08-31T07:25:12.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.829+0000 7fc455400640 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5614ef8cfda0 0x5614efd71180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:12.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.830+0000 7fc455400640 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5614ef8cfda0 0x5614efd71180 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1e06b34c1bb05a11 server_cookie=59769f798fe8f6e in_seq=0 out_seq=0 2024-08-31T07:25:12.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.830+0000 7fc443e00640 10 client.28109 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:25:12.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.830+0000 7fc443e00640 1 -- 172.21.15.32:0/267273088 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614efc561a0 con 0x5614ef8cfda0 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.830+0000 7fc443e00640 10 client.28109 handle_command_reply: tid=0 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.830+0000 7fc46c023740 1 client.28109 shutdown 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 1 -- 172.21.15.32:0/267273088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef9131b0 msgr2=0x5614efd05330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef9131b0 0x5614efd05330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc450004f60 tx=0x7fc450032f40 comp rx=0 tx=0).stop 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 1 -- 172.21.15.32:0/267273088 shutdown_connections 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5614ef8cfda0 0x5614efd71180 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614ef93c070 0x5614efd65f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5614ef91b390 0x5614efd058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 1 --2- 172.21.15.32:0/267273088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5614ef9131b0 0x5614efd05330 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 1 -- 172.21.15.32:0/267273088 >> 172.21.15.32:0/267273088 conn(0x5614ef930460 msgr2=0x5614efce3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc443e00640 0 client.28109 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 1 -- 172.21.15.32:0/267273088 shutdown_connections 2024-08-31T07:25:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 1 -- 172.21.15.32:0/267273088 wait complete. 2024-08-31T07:25:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.831+0000 7fc46c023740 20 client.28109 trim_cache size 0 max 16384 2024-08-31T07:25:12.836 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 -- 172.21.15.32:0/3042687301 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc4440b4d20 msgr2=0x7fc4440b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc4440b4d20 0x7fc4440b7230 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc4480058c0 tx=0x7fc448013040 comp rx=0 tx=0).stop 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 -- 172.21.15.32:0/3042687301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc460109b90 msgr2=0x7fc46019fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc460109b90 0x7fc46019fac0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc45800eb10 tx=0x7fc45800c2d0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 -- 172.21.15.32:0/3042687301 shutdown_connections 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc4440b4d20 0x7fc4440b7230 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc460109b90 0x7fc46019fac0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc460109240 0x7fc46019a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 --2- 172.21.15.32:0/3042687301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc460103900 0x7fc46019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 -- 172.21.15.32:0/3042687301 >> 172.21.15.32:0/3042687301 conn(0x7fc46007b5f0 msgr2=0x7fc460107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 -- 172.21.15.32:0/3042687301 shutdown_connections 2024-08-31T07:25:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:12.832+0000 7fc465200640 1 -- 172.21.15.32:0/3042687301 wait complete. 2024-08-31T07:25:13.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:25:13.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 377666 2.73 MB/sec execute 542 sec latency 2365.007 ms 2024-08-31T07:25:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:14.485 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:14 smithi032 ceph-mon[17252]: pgmap v756: 97 pgs: 97 active+clean; 307 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-08-31T07:25:14.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:14 smithi084 ceph-mon[22387]: pgmap v756: 97 pgs: 97 active+clean; 307 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-08-31T07:25:14.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:14 smithi073 ceph-mon[21995]: pgmap v756: 97 pgs: 97 active+clean; 307 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-08-31T07:25:14.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 377666 2.72 MB/sec execute 543 sec latency 3365.076 ms 2024-08-31T07:25:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.796+0000 7f46a3e00640 1 -- 172.21.15.32:0/1720553383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4102b30 msgr2=0x7f46a410f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.796+0000 7f46a3e00640 1 --2- 172.21.15.32:0/1720553383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4102b30 0x7f46a410f850 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f468c009900 tx=0x7f468c02a480 comp rx=0 tx=0).stop 2024-08-31T07:25:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.796+0000 7f46a3e00640 1 -- 172.21.15.32:0/1720553383 shutdown_connections 2024-08-31T07:25:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.796+0000 7f46a3e00640 1 --2- 172.21.15.32:0/1720553383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a410fd90 0x7f46a41121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.796+0000 7f46a3e00640 1 --2- 172.21.15.32:0/1720553383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a4102b30 0x7f46a410f850 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.796+0000 7f46a3e00640 1 --2- 172.21.15.32:0/1720553383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46a41021e0 0x7f46a41025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.796+0000 7f46a3e00640 1 -- 172.21.15.32:0/1720553383 >> 172.21.15.32:0/1720553383 conn(0x7f46a40fe060 msgr2=0x7f46a41004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.796+0000 7f46a3e00640 1 -- 172.21.15.32:0/1720553383 shutdown_connections 2024-08-31T07:25:14.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.796+0000 7f46a3e00640 1 -- 172.21.15.32:0/1720553383 wait complete. 2024-08-31T07:25:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.797+0000 7f46a3e00640 1 Processor -- start 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.797+0000 7f46a3e00640 1 -- start start 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f46a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a41021e0 0x7f46a419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f46a3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a4102b30 0x7f46a419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f46a3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46a410fd90 0x7f46a41a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f46a3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4115840 con 0x7f46a41021e0 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f46a3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46a41156c0 con 0x7f46a4102b30 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f46a3e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f46a41159c0 con 0x7f46a410fd90 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f46a2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a41021e0 0x7f46a419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f469be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a4102b30 0x7f46a419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f46a2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a41021e0 0x7f46a419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52336/0 (socket says 172.21.15.32:52336) 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f46a2a00640 1 -- 172.21.15.32:0/1154149624 learned_addr learned my addr 172.21.15.32:0/1154149624 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:14.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.798+0000 7f46a3400640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46a410fd90 0x7f46a41a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.799+0000 7f46a2a00640 1 -- 172.21.15.32:0/1154149624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46a410fd90 msgr2=0x7f46a41a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.799+0000 7f46a2a00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46a410fd90 0x7f46a41a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.799+0000 7f46a2a00640 1 -- 172.21.15.32:0/1154149624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a4102b30 msgr2=0x7f46a419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.799+0000 7f46a2a00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a4102b30 0x7f46a419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.799+0000 7f46a2a00640 1 -- 172.21.15.32:0/1154149624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a41a21c0 con 0x7f46a41021e0 2024-08-31T07:25:14.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.799+0000 7f46a2a00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a41021e0 0x7f46a419c250 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f469400e9f0 tx=0x7f469400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.799+0000 7f469b400640 1 -- 172.21.15.32:0/1154149624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469400cd80 con 0x7f46a41021e0 2024-08-31T07:25:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.799+0000 7f46a3e00640 1 -- 172.21.15.32:0/1154149624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a41a24b0 con 0x7f46a41021e0 2024-08-31T07:25:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.799+0000 7f469b400640 1 -- 172.21.15.32:0/1154149624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46940054d0 con 0x7f46a41021e0 2024-08-31T07:25:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.799+0000 7f469b400640 1 -- 172.21.15.32:0/1154149624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694010690 con 0x7f46a41021e0 2024-08-31T07:25:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.800+0000 7f46a3e00640 1 -- 172.21.15.32:0/1154149624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a41a8c10 con 0x7f46a41021e0 2024-08-31T07:25:14.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.803+0000 7f46b09bc740 1 Processor -- start 2024-08-31T07:25:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.803+0000 7f46b09bc740 1 -- start start 2024-08-31T07:25:14.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.803+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50cc8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.803+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50cc90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.803+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 0x558f50cc95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f46b09bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558f50c3f7b0 con 0x558f50923070 2024-08-31T07:25:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f46b09bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558f50c2ed40 con 0x558f50902390 2024-08-31T07:25:14.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f46b09bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558f50c32cf0 con 0x558f508fa1b0 2024-08-31T07:25:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f469b400640 1 -- 172.21.15.32:0/1154149624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4694004e90 con 0x7f46a41021e0 2024-08-31T07:25:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f46a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50cc8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f46a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50cc8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52342/0 (socket says 172.21.15.32:52342) 2024-08-31T07:25:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f46a3400640 1 -- 172.21.15.32:0/4058921025 learned_addr learned my addr 172.21.15.32:0/4058921025 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f469be00640 1 --2- 172.21.15.32:0/4058921025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 0x558f50cc95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f46a2a00640 1 --2- 172.21.15.32:0/4058921025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50cc90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f46a3400640 1 -- 172.21.15.32:0/4058921025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 msgr2=0x558f50cc90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.804+0000 7f46a3400640 1 --2- 172.21.15.32:0/4058921025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50cc90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.805+0000 7f46a3400640 1 -- 172.21.15.32:0/4058921025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 msgr2=0x558f50cc95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.805+0000 7f46a3400640 1 --2- 172.21.15.32:0/4058921025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 0x558f50cc95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.805+0000 7f46a3400640 1 -- 172.21.15.32:0/4058921025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f5098e650 con 0x558f50923070 2024-08-31T07:25:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.805+0000 7f469be00640 1 --2- 172.21.15.32:0/4058921025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 0x558f50cc95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:14.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.805+0000 7f46a3400640 1 --2- 172.21.15.32:0/4058921025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50cc8b60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f469c009fd0 tx=0x7f469c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77734051013bcd2f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.805+0000 7f4693e00640 1 -- 172.21.15.32:0/4058921025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c004d20 con 0x558f50923070 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.805+0000 7f4693e00640 1 -- 172.21.15.32:0/4058921025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c004050 con 0x558f50923070 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.805+0000 7f4693e00640 1 -- 172.21.15.32:0/4058921025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c005220 con 0x558f50923070 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.806+0000 7f469b400640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46880b4ca0 0x7f46880b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.806+0000 7f469b400640 1 -- 172.21.15.32:0/1154149624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f46940e5940 con 0x7f46a41021e0 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.806+0000 7f46a2a00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46880b4ca0 0x7f46880b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.806+0000 7f46b09bc740 1 -- 172.21.15.32:0/4058921025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 msgr2=0x558f50cc8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.806+0000 7f46b09bc740 1 --2- 172.21.15.32:0/4058921025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50cc8b60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f469c009fd0 tx=0x7f469c0118d0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.806+0000 7f46b09bc740 1 -- 172.21.15.32:0/4058921025 shutdown_connections 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.806+0000 7f46b09bc740 1 --2- 172.21.15.32:0/4058921025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 0x558f50cc95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.806+0000 7f46b09bc740 1 --2- 172.21.15.32:0/4058921025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50cc90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.806+0000 7f46b09bc740 1 --2- 172.21.15.32:0/4058921025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50cc8b60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.806+0000 7f46b09bc740 1 -- 172.21.15.32:0/4058921025 >> 172.21.15.32:0/4058921025 conn(0x558f50917460 msgr2=0x558f50cc6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.807+0000 7f46a2a00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46880b4ca0 0x7f46880b71b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f46940e3190 tx=0x7f4694004a90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.807+0000 7f46b09bc740 1 -- 172.21.15.32:0/4058921025 shutdown_connections 2024-08-31T07:25:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.807+0000 7f46b09bc740 1 -- 172.21.15.32:0/4058921025 wait complete. 2024-08-31T07:25:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.807+0000 7f46b09bc740 1 -- start start 2024-08-31T07:25:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.807+0000 7f46b09bc740 1 Processor -- start 2024-08-31T07:25:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.808+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f508fa1b0 0x558f50cd46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.808+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f50902390 0x558f50cd4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.808+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50ce0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.808+0000 7f46b09bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558f50a91700 con 0x558f50923070 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.808+0000 7f46b09bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558f50bb5cf0 con 0x558f508fa1b0 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.808+0000 7f46b09bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558f50cd7340 con 0x558f50902390 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.808+0000 7f46b09bc740 10 client.0 _ll_register_callbacks cb 0x558f50bdb470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.808+0000 7f46a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50ce0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.808+0000 7f469be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f50902390 0x558f50cd4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f46a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50ce0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52348/0 (socket says 172.21.15.32:52348) 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f469be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f50902390 0x558f50cd4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38066/0 (socket says 172.21.15.32:38066) 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f46a3400640 1 -- 172.21.15.32:0/3704194119 learned_addr learned my addr 172.21.15.32:0/3704194119 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f46a3400640 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f508fa1b0 0x558f50cd46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f469be00640 1 -- 172.21.15.32:0/3704194119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f508fa1b0 msgr2=0x558f50cd46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f469be00640 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f508fa1b0 0x558f50cd46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f469be00640 1 -- 172.21.15.32:0/3704194119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 msgr2=0x558f50ce0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f469be00640 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50ce0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f469be00640 1 -- 172.21.15.32:0/3704194119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f50bf68b0 con 0x558f50902390 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f46a3400640 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50ce0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f469be00640 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f50902390 0x558f50cd4c30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f468c02aa70 tx=0x7f468c004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6740cb0c00a387a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.809+0000 7f4690c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.810+0000 7f4690c00640 1 -- 172.21.15.32:0/3704194119 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c035040 con 0x558f50902390 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.810+0000 7f46b09bc740 1 -- 172.21.15.32:0/3704194119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f469c004050 con 0x558f50902390 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.810+0000 7f4690c00640 1 -- 172.21.15.32:0/3704194119 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f468c005510 con 0x558f50902390 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.810+0000 7f4690c00640 1 -- 172.21.15.32:0/3704194119 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f468c004770 con 0x558f50902390 2024-08-31T07:25:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.810+0000 7f46b09bc740 10 client.28126 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.810+0000 7f46b09bc740 1 -- 172.21.15.32:0/3704194119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558f50ce3370 con 0x558f50902390 2024-08-31T07:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.811+0000 7f4690c00640 1 -- 172.21.15.32:0/3704194119 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c004050 con 0x558f50902390 2024-08-31T07:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.811+0000 7f4690c00640 1 -- 172.21.15.32:0/3704194119 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f468c003a10 con 0x558f50902390 2024-08-31T07:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.811+0000 7f46b09bc740 10 client.28126 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.811+0000 7f46b09bc740 10 client.28126 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.811+0000 7f46b09bc740 20 client.28126 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.811+0000 7f46b09bc740 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558f508e19f0 0x558f50ce67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.811+0000 7f46b09bc740 4 client.28126 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.811+0000 7f46b09bc740 1 -- 172.21.15.32:0/3704194119 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f50c41700 con 0x558f508e19f0 2024-08-31T07:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.812+0000 7f46a3400640 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558f508e19f0 0x558f50ce67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.812+0000 7f46a3400640 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558f508e19f0 0x558f50ce67c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb1bfa63873aac17 server_cookie=6578b447192621da in_seq=0 out_seq=0 2024-08-31T07:25:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.812+0000 7f4690c00640 10 client.28126 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:25:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.813+0000 7f4690c00640 1 -- 172.21.15.32:0/3704194119 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f50c41700 con 0x558f508e19f0 2024-08-31T07:25:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.813+0000 7f4690c00640 10 client.28126 handle_command_reply: tid=0 2024-08-31T07:25:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.813+0000 7f46b09bc740 1 client.28126 shutdown 2024-08-31T07:25:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 1 -- 172.21.15.32:0/3704194119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f50902390 msgr2=0x558f50cd4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f50902390 0x558f50cd4c30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f468c02aa70 tx=0x7f468c004420 comp rx=0 tx=0).stop 2024-08-31T07:25:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 1 -- 172.21.15.32:0/3704194119 shutdown_connections 2024-08-31T07:25:14.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558f508e19f0 0x558f50ce67c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50ce0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f50902390 0x558f50cd4c30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 1 --2- 172.21.15.32:0/3704194119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f508fa1b0 0x558f50cd46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 1 -- 172.21.15.32:0/3704194119 >> 172.21.15.32:0/3704194119 conn(0x558f50917460 msgr2=0x558f50ccbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f4690c00640 0 client.28126 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:25:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 1 -- 172.21.15.32:0/3704194119 shutdown_connections 2024-08-31T07:25:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 1 -- 172.21.15.32:0/3704194119 wait complete. 2024-08-31T07:25:14.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.814+0000 7f46b09bc740 20 client.28126 trim_cache size 0 max 16384 2024-08-31T07:25:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.828+0000 7f46b09bc740 1 Processor -- start 2024-08-31T07:25:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.828+0000 7f46b09bc740 1 -- start start 2024-08-31T07:25:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50ce0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 0x558f50ce0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50cd16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f46b09bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46a41b13f0 con 0x558f50923070 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f46b09bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46a41b3050 con 0x558f50902390 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f46b09bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f46a41b1ac0 con 0x558f508fa1b0 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f46a3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50ce0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f469be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 0x558f50ce0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f46a3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50ce0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38076/0 (socket says 172.21.15.32:38076) 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f46a3400640 1 -- 172.21.15.32:0/962681943 learned_addr learned my addr 172.21.15.32:0/962681943 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.829+0000 7f46a3400640 1 --2- 172.21.15.32:0/962681943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50cd16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.830+0000 7f46a3400640 1 -- 172.21.15.32:0/962681943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 msgr2=0x558f50ce0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.830+0000 7f46a3400640 1 --2- 172.21.15.32:0/962681943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 0x558f50ce0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.830+0000 7f46a3400640 1 -- 172.21.15.32:0/962681943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 msgr2=0x558f50cd16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:14.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.830+0000 7f46a3400640 1 --2- 172.21.15.32:0/962681943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50cd16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.830+0000 7f46a3400640 1 -- 172.21.15.32:0/962681943 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f50cc2200 con 0x558f508fa1b0 2024-08-31T07:25:14.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.830+0000 7f46a3400640 1 --2- 172.21.15.32:0/962681943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50ce0530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f469c005ff0 tx=0x7f469c003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a175e72f8164d50a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.830+0000 7f4693400640 1 -- 172.21.15.32:0/962681943 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c004050 con 0x558f508fa1b0 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.830+0000 7f4693400640 1 -- 172.21.15.32:0/962681943 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c004780 con 0x558f508fa1b0 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.830+0000 7f4693400640 1 -- 172.21.15.32:0/962681943 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c00faa0 con 0x558f508fa1b0 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.831+0000 7f46b09bc740 1 -- 172.21.15.32:0/962681943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 msgr2=0x558f50ce0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.831+0000 7f46b09bc740 1 --2- 172.21.15.32:0/962681943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50ce0530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f469c005ff0 tx=0x7f469c003f50 comp rx=0 tx=0).stop 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.831+0000 7f46b09bc740 1 -- 172.21.15.32:0/962681943 shutdown_connections 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.831+0000 7f46b09bc740 1 --2- 172.21.15.32:0/962681943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50923070 0x558f50cd16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.831+0000 7f46b09bc740 1 --2- 172.21.15.32:0/962681943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50902390 0x558f50ce0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.831+0000 7f46b09bc740 1 --2- 172.21.15.32:0/962681943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50ce0530 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.831+0000 7f46b09bc740 1 -- 172.21.15.32:0/962681943 >> 172.21.15.32:0/962681943 conn(0x558f50917460 msgr2=0x558f50cd2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.831+0000 7f46b09bc740 1 -- 172.21.15.32:0/962681943 shutdown_connections 2024-08-31T07:25:14.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.831+0000 7f46b09bc740 1 -- 172.21.15.32:0/962681943 wait complete. 2024-08-31T07:25:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.832+0000 7f46b09bc740 1 -- start start 2024-08-31T07:25:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.832+0000 7f46b09bc740 1 Processor -- start 2024-08-31T07:25:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.832+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50cd8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.832+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50902390 0x558f50cd94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.832+0000 7f46b09bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50923070 0x558f50d4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f46b09bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46a41b28c0 con 0x558f50902390 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f46b09bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46a41b2190 con 0x558f50923070 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f46b09bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f46a41af3f0 con 0x558f508fa1b0 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f46b09bc740 10 client.0 _ll_register_callbacks cb 0x7f468c0040b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f469be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50902390 0x558f50cd94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f469be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50902390 0x558f50cd94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52354/0 (socket says 172.21.15.32:52354) 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f469be00640 1 -- 172.21.15.32:0/1909083813 learned_addr learned my addr 172.21.15.32:0/1909083813 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f46a3400640 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50923070 0x558f50d4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f46a3400640 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50cd8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f469be00640 1 -- 172.21.15.32:0/1909083813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 msgr2=0x558f50cd8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f469be00640 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50cd8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f469be00640 1 -- 172.21.15.32:0/1909083813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50923070 msgr2=0x558f50d4a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f469be00640 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50923070 0x558f50d4a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.833+0000 7f469be00640 1 -- 172.21.15.32:0/1909083813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f50d4a7a0 con 0x558f50902390 2024-08-31T07:25:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.834+0000 7f469be00640 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50902390 0x558f50cd94b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f468c005f70 tx=0x7f468c005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2401da4d5ba9d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.834+0000 7f4690c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:25:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.834+0000 7f4690c00640 1 -- 172.21.15.32:0/1909083813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c035040 con 0x558f50902390 2024-08-31T07:25:14.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.834+0000 7f46b09bc740 1 -- 172.21.15.32:0/1909083813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f469c004780 con 0x558f50902390 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.834+0000 7f4690c00640 1 -- 172.21.15.32:0/1909083813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f468c031d20 con 0x558f50902390 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.834+0000 7f4690c00640 1 -- 172.21.15.32:0/1909083813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f468c060050 con 0x558f50902390 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.834+0000 7f46b09bc740 10 client.18129 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.835+0000 7f46b09bc740 1 -- 172.21.15.32:0/1909083813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558f50d51ba0 con 0x558f50902390 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.835+0000 7f4690c00640 1 -- 172.21.15.32:0/1909083813 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c004780 con 0x558f50902390 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.835+0000 7f4690c00640 1 -- 172.21.15.32:0/1909083813 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f468c058020 con 0x558f50902390 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.835+0000 7f46b09bc740 10 client.18129 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.835+0000 7f46b09bc740 10 client.18129 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.835+0000 7f46b09bc740 20 client.18129 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.835+0000 7f46b09bc740 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558f508b6da0 0x558f50d54380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.835+0000 7f46b09bc740 4 client.18129 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.835+0000 7f46b09bc740 1 -- 172.21.15.32:0/1909083813 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f50c3d1a0 con 0x558f508b6da0 2024-08-31T07:25:14.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.836+0000 7f46a3400640 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558f508b6da0 0x558f50d54380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.836+0000 7f46a3400640 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558f508b6da0 0x558f50d54380 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97de6fbfb97df4d3 server_cookie=8e4d21449a5408cf in_seq=0 out_seq=0 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.836+0000 7f4690c00640 10 client.18129 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.836+0000 7f4690c00640 1 -- 172.21.15.32:0/1909083813 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f50c3d1a0 con 0x558f508b6da0 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.836+0000 7f4690c00640 10 client.18129 handle_command_reply: tid=0 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.836+0000 7f46b09bc740 1 client.18129 shutdown 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 1 -- 172.21.15.32:0/1909083813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50902390 msgr2=0x558f50cd94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50902390 0x558f50cd94b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f468c005f70 tx=0x7f468c005540 comp rx=0 tx=0).stop 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 1 -- 172.21.15.32:0/1909083813 shutdown_connections 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558f508b6da0 0x558f50d54380 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f50923070 0x558f50d4a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f50902390 0x558f50cd94b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 1 --2- 172.21.15.32:0/1909083813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558f508fa1b0 0x558f50cd8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 1 -- 172.21.15.32:0/1909083813 >> 172.21.15.32:0/1909083813 conn(0x558f50917460 msgr2=0x558f50cc9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:14.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f4690c00640 0 client.18129 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:25:14.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 1 -- 172.21.15.32:0/1909083813 shutdown_connections 2024-08-31T07:25:14.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 1 -- 172.21.15.32:0/1909083813 wait complete. 2024-08-31T07:25:14.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.837+0000 7f46b09bc740 20 client.18129 trim_cache size 0 max 16384 2024-08-31T07:25:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:25:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 -- 172.21.15.32:0/1154149624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46880b4ca0 msgr2=0x7f46880b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46880b4ca0 0x7f46880b71b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f46940e3190 tx=0x7f4694004a90 comp rx=0 tx=0).stop 2024-08-31T07:25:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 -- 172.21.15.32:0/1154149624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a41021e0 msgr2=0x7f46a419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a41021e0 0x7f46a419c250 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f469400e9f0 tx=0x7f469400edf0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 -- 172.21.15.32:0/1154149624 shutdown_connections 2024-08-31T07:25:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f46880b4ca0 0x7f46880b71b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46a410fd90 0x7f46a41a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a4102b30 0x7f46a419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 --2- 172.21.15.32:0/1154149624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a41021e0 0x7f46a419c250 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:14.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 -- 172.21.15.32:0/1154149624 >> 172.21.15.32:0/1154149624 conn(0x7f46a40fe060 msgr2=0x7f46a4111ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:14.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 -- 172.21.15.32:0/1154149624 shutdown_connections 2024-08-31T07:25:14.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:14.838+0000 7f46a3e00640 1 -- 172.21.15.32:0/1154149624 wait complete. 2024-08-31T07:25:15.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:25:15.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 379007 2.73 MB/sec execute 544 sec latency 3714.905 ms 2024-08-31T07:25:16.160 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:16.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:16 smithi084 ceph-mon[22387]: pgmap v757: 97 pgs: 97 active+clean; 309 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-08-31T07:25:16.499 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:16 smithi032 ceph-mon[17252]: pgmap v757: 97 pgs: 97 active+clean; 309 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-08-31T07:25:16.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:16 smithi073 ceph-mon[21995]: pgmap v757: 97 pgs: 97 active+clean; 309 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-08-31T07:25:16.779 INFO:tasks.workunit.client.0.smithi032.stdout: 1 380168 2.74 MB/sec execute 545 sec latency 23.048 ms 2024-08-31T07:25:16.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.887+0000 7f9ee3200640 1 -- 172.21.15.32:0/3188570581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4104eb0 msgr2=0x7f9ee410d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:16.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.887+0000 7f9ee3200640 1 --2- 172.21.15.32:0/3188570581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4104eb0 0x7f9ee410d400 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009900 tx=0x7f9ed002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.887+0000 7f9ee3200640 1 -- 172.21.15.32:0/3188570581 shutdown_connections 2024-08-31T07:25:16.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.887+0000 7f9ee3200640 1 --2- 172.21.15.32:0/3188570581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee410d9c0 0x7f9ee410feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.887+0000 7f9ee3200640 1 --2- 172.21.15.32:0/3188570581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4104eb0 0x7f9ee410d400 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.887+0000 7f9ee3200640 1 --2- 172.21.15.32:0/3188570581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ee4104560 0x7f9ee4104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.887+0000 7f9ee3200640 1 -- 172.21.15.32:0/3188570581 >> 172.21.15.32:0/3188570581 conn(0x7f9ee40fe060 msgr2=0x7f9ee41004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:16.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.888+0000 7f9ee3200640 1 -- 172.21.15.32:0/3188570581 shutdown_connections 2024-08-31T07:25:16.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.888+0000 7f9ee3200640 1 -- 172.21.15.32:0/3188570581 wait complete. 2024-08-31T07:25:16.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.889+0000 7f9ee3200640 1 Processor -- start 2024-08-31T07:25:16.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.889+0000 7f9ee3200640 1 -- start start 2024-08-31T07:25:16.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4104560 0x7f9ee419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4104eb0 0x7f9ee419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ee410d9c0 0x7f9ee41a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4113660 con 0x7f9ee410d9c0 2024-08-31T07:25:16.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41134e0 con 0x7f9ee4104eb0 2024-08-31T07:25:16.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41137e0 con 0x7f9ee4104560 2024-08-31T07:25:16.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4104eb0 0x7f9ee419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4104eb0 0x7f9ee419c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52286/0 (socket says 172.21.15.32:52286) 2024-08-31T07:25:16.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee1400640 1 -- 172.21.15.32:0/1951787852 learned_addr learned my addr 172.21.15.32:0/1951787852 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4104560 0x7f9ee419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.890+0000 7f9ee2800640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ee410d9c0 0x7f9ee41a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.891+0000 7f9ee1e00640 1 -- 172.21.15.32:0/1951787852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4104eb0 msgr2=0x7f9ee419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.891+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4104eb0 0x7f9ee419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.891+0000 7f9ee1e00640 1 -- 172.21.15.32:0/1951787852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ee410d9c0 msgr2=0x7f9ee41a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.891+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ee410d9c0 0x7f9ee41a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.891+0000 7f9ee1e00640 1 -- 172.21.15.32:0/1951787852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee41a22a0 con 0x7f9ee4104560 2024-08-31T07:25:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.891+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4104560 0x7f9ee419c330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e9f0 tx=0x7f9ed400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.891+0000 7f9edaa00640 1 -- 172.21.15.32:0/1951787852 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400cd80 con 0x7f9ee4104560 2024-08-31T07:25:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.891+0000 7f9edaa00640 1 -- 172.21.15.32:0/1951787852 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed40035f0 con 0x7f9ee4104560 2024-08-31T07:25:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.891+0000 7f9ee3200640 1 -- 172.21.15.32:0/1951787852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee41a2590 con 0x7f9ee4104560 2024-08-31T07:25:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.891+0000 7f9edaa00640 1 -- 172.21.15.32:0/1951787852 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4010670 con 0x7f9ee4104560 2024-08-31T07:25:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.892+0000 7f9ee3200640 1 -- 172.21.15.32:0/1951787852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a8cf0 con 0x7f9ee4104560 2024-08-31T07:25:16.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.896+0000 7f9eefe0b740 1 Processor -- start 2024-08-31T07:25:16.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.896+0000 7f9eefe0b740 1 -- start start 2024-08-31T07:25:16.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.896+0000 7f9edaa00640 1 -- 172.21.15.32:0/1951787852 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9ed40037a0 con 0x7f9ee4104560 2024-08-31T07:25:16.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.896+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041af070 0x561204554a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.896+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 0x561204554fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.896+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x5612045554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.896+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5612044cb7b0 con 0x5612041af070 2024-08-31T07:25:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.896+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5612044bad40 con 0x5612041861b0 2024-08-31T07:25:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.896+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5612044becf0 con 0x56120418e390 2024-08-31T07:25:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.897+0000 7f9ee1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 0x561204554fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.897+0000 7f9ee1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 0x561204554fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52294/0 (socket says 172.21.15.32:52294) 2024-08-31T07:25:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.897+0000 7f9ee1e00640 1 -- 172.21.15.32:0/3998575691 learned_addr learned my addr 172.21.15.32:0/3998575691 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.897+0000 7f9ee2800640 1 --2- 172.21.15.32:0/3998575691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041af070 0x561204554a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.897+0000 7f9ee1400640 1 --2- 172.21.15.32:0/3998575691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x5612045554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.897+0000 7f9ee1400640 1 -- 172.21.15.32:0/3998575691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 msgr2=0x561204554fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.897+0000 7f9ee1400640 1 --2- 172.21.15.32:0/3998575691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 0x561204554fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.897+0000 7f9ee1400640 1 -- 172.21.15.32:0/3998575691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041af070 msgr2=0x561204554a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.897+0000 7f9ee1400640 1 --2- 172.21.15.32:0/3998575691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041af070 0x561204554a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.897+0000 7f9ee1400640 1 -- 172.21.15.32:0/3998575691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56120421a650 con 0x56120418e390 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.898+0000 7f9ee1400640 1 --2- 172.21.15.32:0/3998575691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x5612045554f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005ab0 tx=0x7f9ed0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=586022a03d838525 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.898+0000 7f9ecf400640 1 -- 172.21.15.32:0/3998575691 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0040070 con 0x56120418e390 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.898+0000 7f9ecf400640 1 -- 172.21.15.32:0/3998575691 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed00054d0 con 0x56120418e390 2024-08-31T07:25:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.898+0000 7f9ecf400640 1 -- 172.21.15.32:0/3998575691 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0035040 con 0x56120418e390 2024-08-31T07:25:16.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.899+0000 7f9edaa00640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ec80b4ca0 0x7f9ec80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.899+0000 7f9edaa00640 1 -- 172.21.15.32:0/1951787852 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f9ed40e57e0 con 0x7f9ee4104560 2024-08-31T07:25:16.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.899+0000 7f9ee2800640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ec80b4ca0 0x7f9ec80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.899+0000 7f9eefe0b740 1 -- 172.21.15.32:0/3998575691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 msgr2=0x5612045554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:16.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.899+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/3998575691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x5612045554f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005ab0 tx=0x7f9ed0004040 comp rx=0 tx=0).stop 2024-08-31T07:25:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.900+0000 7f9eefe0b740 1 -- 172.21.15.32:0/3998575691 shutdown_connections 2024-08-31T07:25:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.900+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/3998575691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x5612045554f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.900+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/3998575691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 0x561204554fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.900+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/3998575691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041af070 0x561204554a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.900+0000 7f9eefe0b740 1 -- 172.21.15.32:0/3998575691 >> 172.21.15.32:0/3998575691 conn(0x5612041a3460 msgr2=0x561204552960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.900+0000 7f9eefe0b740 1 -- 172.21.15.32:0/3998575691 shutdown_connections 2024-08-31T07:25:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.900+0000 7f9eefe0b740 1 -- 172.21.15.32:0/3998575691 wait complete. 2024-08-31T07:25:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.901+0000 7f9eefe0b740 1 -- start start 2024-08-31T07:25:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.901+0000 7f9ee2800640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ec80b4ca0 0x7f9ec80b71b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9edc005fd0 tx=0x7f9edc009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.901+0000 7f9eefe0b740 1 Processor -- start 2024-08-31T07:25:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041861b0 0x561204568de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56120418e390 0x561204569320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612041af070 0x561204574c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56120431d700 con 0x5612041861b0 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561204441cf0 con 0x56120418e390 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56120456b920 con 0x5612041af070 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9eefe0b740 10 client.0 _ll_register_callbacks cb 0x561204467470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9ee2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612041af070 0x561204574c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9ee2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612041af070 0x561204574c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35630/0 (socket says 172.21.15.32:35630) 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9ee2800640 1 -- 172.21.15.32:0/1888118917 learned_addr learned my addr 172.21.15.32:0/1888118917 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9ee1400640 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56120418e390 0x561204569320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.902+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041861b0 0x561204568de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.903+0000 7f9ee1400640 1 -- 172.21.15.32:0/1888118917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612041af070 msgr2=0x561204574c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.903+0000 7f9ee1400640 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612041af070 0x561204574c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.903+0000 7f9ee1400640 1 -- 172.21.15.32:0/1888118917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041861b0 msgr2=0x561204568de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:16.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.903+0000 7f9ee1400640 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041861b0 0x561204568de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.903+0000 7f9ee1400640 1 -- 172.21.15.32:0/1888118917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612044828b0 con 0x56120418e390 2024-08-31T07:25:16.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.903+0000 7f9ee1400640 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56120418e390 0x561204569320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51c2331482479be3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:16.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.903+0000 7f9ec3e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:25:16.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.903+0000 7f9ec3e00640 1 -- 172.21.15.32:0/1888118917 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0051060 con 0x56120418e390 2024-08-31T07:25:16.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.903+0000 7f9eefe0b740 1 -- 172.21.15.32:0/1888118917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed00054d0 con 0x56120418e390 2024-08-31T07:25:16.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.903+0000 7f9ec3e00640 1 -- 172.21.15.32:0/1888118917 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f9ed0004970 con 0x56120418e390 2024-08-31T07:25:16.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9ec3e00640 1 -- 172.21.15.32:0/1888118917 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9ed0035070 con 0x56120418e390 2024-08-31T07:25:16.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9eefe0b740 10 client.28139 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9eefe0b740 1 -- 172.21.15.32:0/1888118917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561204577920 con 0x56120418e390 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9ec3e00640 1 -- 172.21.15.32:0/1888118917 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed00054d0 con 0x56120418e390 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9ec3e00640 1 -- 172.21.15.32:0/1888118917 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f9ed0003bd0 con 0x56120418e390 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9eefe0b740 10 client.28139 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9eefe0b740 10 client.28139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9eefe0b740 20 client.28139 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56120416d9f0 0x56120457ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9eefe0b740 4 client.28139 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.904+0000 7f9eefe0b740 1 -- 172.21.15.32:0/1888118917 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612044cd700 con 0x56120416d9f0 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.905+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56120416d9f0 0x56120457ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.905+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56120416d9f0 0x56120457ac80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a3986604464f86b server_cookie=bf3f33cd1fc9f344 in_seq=0 out_seq=0 2024-08-31T07:25:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.905+0000 7f9ec3e00640 10 client.28139 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:25:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.906+0000 7f9ec3e00640 1 -- 172.21.15.32:0/1888118917 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f9ed4085e50 con 0x56120416d9f0 2024-08-31T07:25:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.906+0000 7f9ec3e00640 10 client.28139 handle_command_reply: tid=0 2024-08-31T07:25:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.906+0000 7f9eefe0b740 1 client.28139 shutdown 2024-08-31T07:25:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 1 -- 172.21.15.32:0/1888118917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56120418e390 msgr2=0x561204569320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56120418e390 0x561204569320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 1 -- 172.21.15.32:0/1888118917 shutdown_connections 2024-08-31T07:25:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612041af070 0x561204574c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56120416d9f0 0x56120457ac80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56120418e390 0x561204569320 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/1888118917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041861b0 0x561204568de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 1 -- 172.21.15.32:0/1888118917 >> 172.21.15.32:0/1888118917 conn(0x5612041a3460 msgr2=0x561204550f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9ec3e00640 0 client.28139 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:25:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 1 -- 172.21.15.32:0/1888118917 shutdown_connections 2024-08-31T07:25:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 1 -- 172.21.15.32:0/1888118917 wait complete. 2024-08-31T07:25:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.907+0000 7f9eefe0b740 20 client.28139 trim_cache size 0 max 16384 2024-08-31T07:25:16.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.924+0000 7f9eefe0b740 1 Processor -- start 2024-08-31T07:25:16.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.924+0000 7f9eefe0b740 1 -- start start 2024-08-31T07:25:16.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.924+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 0x561204551650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.924+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x561204574bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.924+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041af070 0x561204575100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ad0c0 con 0x5612041af070 2024-08-31T07:25:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41aed20 con 0x5612041861b0 2024-08-31T07:25:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ad790 con 0x56120418e390 2024-08-31T07:25:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9ee1400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x561204574bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9ee1400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x561204574bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35644/0 (socket says 172.21.15.32:35644) 2024-08-31T07:25:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9ee1400640 1 -- 172.21.15.32:0/2791912164 learned_addr learned my addr 172.21.15.32:0/2791912164 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/2791912164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 0x561204551650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9ee2800640 1 --2- 172.21.15.32:0/2791912164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041af070 0x561204575100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9ee1e00640 1 -- 172.21.15.32:0/2791912164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 msgr2=0x561204574bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/2791912164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x561204574bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9ee1e00640 1 -- 172.21.15.32:0/2791912164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041af070 msgr2=0x561204575100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/2791912164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041af070 0x561204575100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.925+0000 7f9ee1e00640 1 -- 172.21.15.32:0/2791912164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561204569ea0 con 0x5612041861b0 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.926+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/2791912164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 0x561204551650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40a9130 tx=0x7f9ed40a7d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d668b6c20fcd867f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.926+0000 7f9ecea00640 1 -- 172.21.15.32:0/2791912164 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40b0730 con 0x5612041861b0 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.926+0000 7f9ecea00640 1 -- 172.21.15.32:0/2791912164 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5612044cd700 con 0x5612041861b0 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.926+0000 7f9ecea00640 1 -- 172.21.15.32:0/2791912164 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40e31a0 con 0x5612041861b0 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.927+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2791912164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 msgr2=0x561204551650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.927+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/2791912164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 0x561204551650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40a9130 tx=0x7f9ed40a7d80 comp rx=0 tx=0).stop 2024-08-31T07:25:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.927+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2791912164 shutdown_connections 2024-08-31T07:25:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.927+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/2791912164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041af070 0x561204575100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.927+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/2791912164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x561204574bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.927+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/2791912164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041861b0 0x561204551650 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.927+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2791912164 >> 172.21.15.32:0/2791912164 conn(0x5612041a3460 msgr2=0x561204550f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.927+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2791912164 shutdown_connections 2024-08-31T07:25:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.928+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2791912164 wait complete. 2024-08-31T07:25:16.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.928+0000 7f9eefe0b740 1 -- start start 2024-08-31T07:25:16.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.928+0000 7f9eefe0b740 1 Processor -- start 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041861b0 0x56120457e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x56120457aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9eefe0b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041af070 0x56120457b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ae590 con 0x5612041861b0 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ade60 con 0x5612041af070 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9eefe0b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ab120 con 0x56120418e390 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9eefe0b740 10 client.0 _ll_register_callbacks cb 0x7f9ed0005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9ee1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041861b0 0x56120457e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9ee2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041af070 0x56120457b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9ee1400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x56120457aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9ee2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041af070 0x56120457b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52316/0 (socket says 172.21.15.32:52316) 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.929+0000 7f9ee2800640 1 -- 172.21.15.32:0/2426156800 learned_addr learned my addr 172.21.15.32:0/2426156800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.930+0000 7f9ee2800640 1 -- 172.21.15.32:0/2426156800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 msgr2=0x56120457aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.930+0000 7f9ee2800640 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x56120457aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.930+0000 7f9ee2800640 1 -- 172.21.15.32:0/2426156800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041861b0 msgr2=0x56120457e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.930+0000 7f9ee2800640 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041861b0 0x56120457e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.930+0000 7f9ee2800640 1 -- 172.21.15.32:0/2426156800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56120457b7f0 con 0x5612041af070 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.930+0000 7f9ee2800640 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041af070 0x56120457b010 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9edc004280 tx=0x7f9edc016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9fc719bf11e770f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.930+0000 7f9ec3e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.930+0000 7f9ec3e00640 1 -- 172.21.15.32:0/2426156800 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc02d060 con 0x5612041af070 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2426156800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612044cd700 con 0x5612041af070 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.930+0000 7f9ec3e00640 1 -- 172.21.15.32:0/2426156800 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f9edc004800 con 0x5612041af070 2024-08-31T07:25:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9ec3e00640 1 -- 172.21.15.32:0/2426156800 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9edc00fc10 con 0x5612041af070 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9eefe0b740 10 client.28145 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2426156800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5612045e6450 con 0x5612041af070 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9ec3e00640 1 -- 172.21.15.32:0/2426156800 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5612044cd700 con 0x5612041af070 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9ec3e00640 1 -- 172.21.15.32:0/2426156800 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f9edc0055b0 con 0x5612041af070 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9eefe0b740 10 client.28145 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9eefe0b740 10 client.28145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9eefe0b740 20 client.28145 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561204142da0 0x5612045e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9eefe0b740 4 client.28145 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.931+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2426156800 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561204564790 con 0x561204142da0 2024-08-31T07:25:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.932+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561204142da0 0x5612045e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.932+0000 7f9ee1e00640 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561204142da0 0x5612045e8c40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2910a4d463291059 server_cookie=bea8040b16a5de95 in_seq=0 out_seq=0 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.932+0000 7f9ec3e00640 10 client.28145 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9ec3e00640 1 -- 172.21.15.32:0/2426156800 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561204564790 con 0x561204142da0 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9ec3e00640 10 client.28145 handle_command_reply: tid=0 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 client.28145 shutdown 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2426156800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041af070 msgr2=0x56120457b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041af070 0x56120457b010 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9edc004280 tx=0x7f9edc016000 comp rx=0 tx=0).stop 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2426156800 shutdown_connections 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x561204142da0 0x5612045e8c40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5612041af070 0x56120457b010 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56120418e390 0x56120457aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 --2- 172.21.15.32:0/2426156800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5612041861b0 0x56120457e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9ec3e00640 0 client.28145 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:25:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2426156800 >> 172.21.15.32:0/2426156800 conn(0x5612041a3460 msgr2=0x561204555aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2426156800 shutdown_connections 2024-08-31T07:25:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 1 -- 172.21.15.32:0/2426156800 wait complete. 2024-08-31T07:25:16.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.933+0000 7f9eefe0b740 20 client.28145 trim_cache size 0 max 16384 2024-08-31T07:25:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:25:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 -- 172.21.15.32:0/1951787852 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ec80b4ca0 msgr2=0x7f9ec80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ec80b4ca0 0x7f9ec80b71b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9edc005fd0 tx=0x7f9edc009500 comp rx=0 tx=0).stop 2024-08-31T07:25:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 -- 172.21.15.32:0/1951787852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4104560 msgr2=0x7f9ee419c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4104560 0x7f9ee419c330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e9f0 tx=0x7f9ed400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 -- 172.21.15.32:0/1951787852 shutdown_connections 2024-08-31T07:25:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9ec80b4ca0 0x7f9ec80b71b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ee410d9c0 0x7f9ee41a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ee4104eb0 0x7f9ee419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 --2- 172.21.15.32:0/1951787852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ee4104560 0x7f9ee419c330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 -- 172.21.15.32:0/1951787852 >> 172.21.15.32:0/1951787852 conn(0x7f9ee40fe060 msgr2=0x7f9ee410ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.934+0000 7f9ee3200640 1 -- 172.21.15.32:0/1951787852 shutdown_connections 2024-08-31T07:25:16.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:16.935+0000 7f9ee3200640 1 -- 172.21.15.32:0/1951787852 wait complete. 2024-08-31T07:25:17.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 380801 2.73 MB/sec execute 546 sec latency 470.200 ms 2024-08-31T07:25:18.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:25:18.394 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:18 smithi032 ceph-mon[17252]: pgmap v758: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 979 KiB/s wr, 87 op/s 2024-08-31T07:25:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:18 smithi084 ceph-mon[22387]: pgmap v758: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 979 KiB/s wr, 87 op/s 2024-08-31T07:25:18.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:18 smithi073 ceph-mon[21995]: pgmap v758: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 979 KiB/s wr, 87 op/s 2024-08-31T07:25:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:18.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 381255 2.74 MB/sec execute 547 sec latency 687.117 ms 2024-08-31T07:25:19.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.420+0000 7f4917a00640 1 -- 172.21.15.32:0/3863460501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f491010bde0 msgr2=0x7f4910112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:19.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.420+0000 7f4917a00640 1 --2- 172.21.15.32:0/3863460501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f491010bde0 0x7f4910112610 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4908009ca0 tx=0x7f490802a8b0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.420+0000 7f4917a00640 1 -- 172.21.15.32:0/3863460501 shutdown_connections 2024-08-31T07:25:19.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.420+0000 7f4917a00640 1 --2- 172.21.15.32:0/3863460501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f491010bde0 0x7f4910112610 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.420+0000 7f4917a00640 1 --2- 172.21.15.32:0/3863460501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f491010b490 0x7f491010b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.420+0000 7f4917a00640 1 --2- 172.21.15.32:0/3863460501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4910105b50 0x7f4910105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.420+0000 7f4917a00640 1 -- 172.21.15.32:0/3863460501 >> 172.21.15.32:0/3863460501 conn(0x7f4910101580 msgr2=0x7f49101039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.420+0000 7f4917a00640 1 -- 172.21.15.32:0/3863460501 shutdown_connections 2024-08-31T07:25:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.420+0000 7f4917a00640 1 -- 172.21.15.32:0/3863460501 wait complete. 2024-08-31T07:25:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.421+0000 7f4917a00640 1 Processor -- start 2024-08-31T07:25:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.422+0000 7f4917a00640 1 -- start start 2024-08-31T07:25:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.422+0000 7f4917a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4910105b50 0x7f491019c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.422+0000 7f4917a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f491010b490 0x7f491019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.422+0000 7f4917a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f491010bde0 0x7f49101a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.422+0000 7f4917a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4910115aa0 con 0x7f491010b490 2024-08-31T07:25:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.422+0000 7f4917a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4910115920 con 0x7f491010bde0 2024-08-31T07:25:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.422+0000 7f4917a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4910115c20 con 0x7f4910105b50 2024-08-31T07:25:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.422+0000 7f4907e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f491010b490 0x7f491019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4915000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4910105b50 0x7f491019c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4915000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4910105b50 0x7f491019c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35672/0 (socket says 172.21.15.32:35672) 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4907e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f491010b490 0x7f491019ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38896/0 (socket says 172.21.15.32:38896) 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4915000640 1 -- 172.21.15.32:0/1194439720 learned_addr learned my addr 172.21.15.32:0/1194439720 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4915a00640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f491010bde0 0x7f49101a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4915000640 1 -- 172.21.15.32:0/1194439720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f491010bde0 msgr2=0x7f49101a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4915000640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f491010bde0 0x7f49101a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4915000640 1 -- 172.21.15.32:0/1194439720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f491010b490 msgr2=0x7f491019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4915000640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f491010b490 0x7f491019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4915000640 1 -- 172.21.15.32:0/1194439720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49101a23e0 con 0x7f4910105b50 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.423+0000 7f4915000640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4910105b50 0x7f491019c470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f490000e9f0 tx=0x7f490000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.424+0000 7f490e000640 1 -- 172.21.15.32:0/1194439720 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490000cd80 con 0x7f4910105b50 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.424+0000 7f4917a00640 1 -- 172.21.15.32:0/1194439720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49101a26d0 con 0x7f4910105b50 2024-08-31T07:25:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.424+0000 7f490e000640 1 -- 172.21.15.32:0/1194439720 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49000054d0 con 0x7f4910105b50 2024-08-31T07:25:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.424+0000 7f490e000640 1 -- 172.21.15.32:0/1194439720 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900010430 con 0x7f4910105b50 2024-08-31T07:25:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.424+0000 7f4917a00640 1 -- 172.21.15.32:0/1194439720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49101a8e30 con 0x7f4910105b50 2024-08-31T07:25:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.427+0000 7f491e658740 1 Processor -- start 2024-08-31T07:25:19.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.427+0000 7f491e658740 1 -- start start 2024-08-31T07:25:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f491e658740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x564526989990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f491e658740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265bb1b0 0x564526989ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f491e658740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265c3390 0x56452698a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f491e658740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5645269007b0 con 0x5645265e4070 2024-08-31T07:25:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f491e658740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5645268efd40 con 0x5645265bb1b0 2024-08-31T07:25:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f491e658740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5645268f3cf0 con 0x5645265c3390 2024-08-31T07:25:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f490e000640 1 -- 172.21.15.32:0/1194439720 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f49000046c0 con 0x7f4910105b50 2024-08-31T07:25:19.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f4915a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x564526989990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f4915a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x564526989990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38904/0 (socket says 172.21.15.32:38904) 2024-08-31T07:25:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f4907e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265c3390 0x56452698a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f4915a00640 1 -- 172.21.15.32:0/2783079551 learned_addr learned my addr 172.21.15.32:0/2783079551 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.428+0000 7f4915000640 1 --2- 172.21.15.32:0/2783079551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265bb1b0 0x564526989ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.429+0000 7f4915a00640 1 -- 172.21.15.32:0/2783079551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265c3390 msgr2=0x56452698a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:19.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.429+0000 7f4915a00640 1 --2- 172.21.15.32:0/2783079551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265c3390 0x56452698a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.429+0000 7f4915a00640 1 -- 172.21.15.32:0/2783079551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265bb1b0 msgr2=0x564526989ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:19.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.429+0000 7f4915a00640 1 --2- 172.21.15.32:0/2783079551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265bb1b0 0x564526989ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.429+0000 7f4915a00640 1 -- 172.21.15.32:0/2783079551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56452664f650 con 0x5645265e4070 2024-08-31T07:25:19.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.429+0000 7f4915a00640 1 --2- 172.21.15.32:0/2783079551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x564526989990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4908009800 tx=0x7f4908009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e943e3852b1094da server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:19.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.429+0000 7f4906000640 1 -- 172.21.15.32:0/2783079551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49080305a0 con 0x5645265e4070 2024-08-31T07:25:19.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.430+0000 7f4906000640 1 -- 172.21.15.32:0/2783079551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4908036030 con 0x5645265e4070 2024-08-31T07:25:19.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.430+0000 7f4906000640 1 -- 172.21.15.32:0/2783079551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908004240 con 0x5645265e4070 2024-08-31T07:25:19.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.430+0000 7f490e000640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f40b4d20 0x7f48f40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.430+0000 7f490e000640 1 -- 172.21.15.32:0/1194439720 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f49000e5870 con 0x7f4910105b50 2024-08-31T07:25:19.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.430+0000 7f4915000640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f40b4d20 0x7f48f40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.430+0000 7f491e658740 1 -- 172.21.15.32:0/2783079551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 msgr2=0x564526989990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.430+0000 7f491e658740 1 --2- 172.21.15.32:0/2783079551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x564526989990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4908009800 tx=0x7f4908009830 comp rx=0 tx=0).stop 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.431+0000 7f491e658740 1 -- 172.21.15.32:0/2783079551 shutdown_connections 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.431+0000 7f491e658740 1 --2- 172.21.15.32:0/2783079551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265c3390 0x56452698a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.431+0000 7f491e658740 1 --2- 172.21.15.32:0/2783079551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265bb1b0 0x564526989ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.431+0000 7f491e658740 1 --2- 172.21.15.32:0/2783079551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x564526989990 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.431+0000 7f491e658740 1 -- 172.21.15.32:0/2783079551 >> 172.21.15.32:0/2783079551 conn(0x5645265d8460 msgr2=0x564526987880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.431+0000 7f4915000640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f40b4d20 0x7f48f40b7230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x564526988610 tx=0x7f4900009720 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.431+0000 7f491e658740 1 -- 172.21.15.32:0/2783079551 shutdown_connections 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.431+0000 7f491e658740 1 -- 172.21.15.32:0/2783079551 wait complete. 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.432+0000 7f491e658740 1 -- start start 2024-08-31T07:25:19.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.432+0000 7f491e658740 1 Processor -- start 2024-08-31T07:25:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.432+0000 7f491e658740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x5645269955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.432+0000 7f491e658740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x564526995af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.432+0000 7f491e658740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x5645269a1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.432+0000 7f491e658740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564526752700 con 0x5645265e4070 2024-08-31T07:25:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.432+0000 7f491e658740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564526876cf0 con 0x5645265c3390 2024-08-31T07:25:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.432+0000 7f491e658740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564526998110 con 0x5645265bb1b0 2024-08-31T07:25:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f491e658740 10 client.0 _ll_register_callbacks cb 0x56452689c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f4907e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x564526995af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f4907e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x564526995af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52344/0 (socket says 172.21.15.32:52344) 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f4907e00640 1 -- 172.21.15.32:0/1108326757 learned_addr learned my addr 172.21.15.32:0/1108326757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f4915a00640 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x5645269a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f4915a00640 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x5645269955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f4907e00640 1 -- 172.21.15.32:0/1108326757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 msgr2=0x5645269955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f4907e00640 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x5645269955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f4907e00640 1 -- 172.21.15.32:0/1108326757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 msgr2=0x5645269a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f4907e00640 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x5645269a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.433+0000 7f4907e00640 1 -- 172.21.15.32:0/1108326757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645268b78b0 con 0x5645265c3390 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.434+0000 7f4907e00640 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x564526995af0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f48f80126e0 tx=0x7f48f8012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e117aa6e2d1146d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.434+0000 7f48fea00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.434+0000 7f48fea00640 1 -- 172.21.15.32:0/1108326757 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8012190 con 0x5645265c3390 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.434+0000 7f491e658740 1 -- 172.21.15.32:0/1108326757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4908036030 con 0x5645265c3390 2024-08-31T07:25:19.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.434+0000 7f48fea00640 1 -- 172.21.15.32:0/1108326757 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f48f800f920 con 0x5645265c3390 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.434+0000 7f48fea00640 1 -- 172.21.15.32:0/1108326757 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f48f800fb20 con 0x5645265c3390 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.434+0000 7f491e658740 10 client.28157 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.434+0000 7f491e658740 1 -- 172.21.15.32:0/1108326757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5645269a4210 con 0x5645265c3390 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.434+0000 7f48fea00640 1 -- 172.21.15.32:0/1108326757 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4908036030 con 0x5645265c3390 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.435+0000 7f48fea00640 1 -- 172.21.15.32:0/1108326757 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f48f8010c90 con 0x5645265c3390 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.435+0000 7f491e658740 10 client.28157 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.435+0000 7f491e658740 10 client.28157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.435+0000 7f491e658740 20 client.28157 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.435+0000 7f491e658740 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5645265a29f0 0x5645269a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.435+0000 7f491e658740 4 client.28157 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.435+0000 7f491e658740 1 -- 172.21.15.32:0/1108326757 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564526902700 con 0x5645265a29f0 2024-08-31T07:25:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.435+0000 7f4915a00640 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5645265a29f0 0x5645269a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.436+0000 7f4915a00640 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5645265a29f0 0x5645269a78a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b93b9f018d9c8620 server_cookie=e3b3f866ea280a1 in_seq=0 out_seq=0 2024-08-31T07:25:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.436+0000 7f48fea00640 10 client.28157 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f48fea00640 1 -- 172.21.15.32:0/1108326757 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564526902700 con 0x5645265a29f0 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f48fea00640 10 client.28157 handle_command_reply: tid=0 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 client.28157 shutdown 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 -- 172.21.15.32:0/1108326757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 msgr2=0x564526995af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x564526995af0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f48f80126e0 tx=0x7f48f8012ba0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 -- 172.21.15.32:0/1108326757 shutdown_connections 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5645265a29f0 0x5645269a78a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x5645269a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x564526995af0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 --2- 172.21.15.32:0/1108326757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x5645269955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 -- 172.21.15.32:0/1108326757 >> 172.21.15.32:0/1108326757 conn(0x5645265d8460 msgr2=0x56452698cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f48fea00640 0 client.28157 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 -- 172.21.15.32:0/1108326757 shutdown_connections 2024-08-31T07:25:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 1 -- 172.21.15.32:0/1108326757 wait complete. 2024-08-31T07:25:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.437+0000 7f491e658740 20 client.28157 trim_cache size 0 max 16384 2024-08-31T07:25:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.451+0000 7f491e658740 1 Processor -- start 2024-08-31T07:25:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.451+0000 7f491e658740 1 -- start start 2024-08-31T07:25:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f491e658740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x56452698f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f491e658740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x5645269a1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f491e658740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x5645269a1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f491e658740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b1540 con 0x5645265e4070 2024-08-31T07:25:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f491e658740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b31a0 con 0x5645265c3390 2024-08-31T07:25:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f491e658740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b1c10 con 0x5645265bb1b0 2024-08-31T07:25:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f4907e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x5645269a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f4907e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x5645269a1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52354/0 (socket says 172.21.15.32:52354) 2024-08-31T07:25:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f4907e00640 1 -- 172.21.15.32:0/1212804229 learned_addr learned my addr 172.21.15.32:0/1212804229 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f4915a00640 1 --2- 172.21.15.32:0/1212804229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x5645269a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.452+0000 7f4915a00640 1 --2- 172.21.15.32:0/1212804229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x56452698f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.453+0000 7f4907e00640 1 -- 172.21.15.32:0/1212804229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 msgr2=0x56452698f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.453+0000 7f4907e00640 1 --2- 172.21.15.32:0/1212804229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x56452698f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.453+0000 7f4907e00640 1 -- 172.21.15.32:0/1212804229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 msgr2=0x5645269a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.453+0000 7f4907e00640 1 --2- 172.21.15.32:0/1212804229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x5645269a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.453+0000 7f4907e00640 1 -- 172.21.15.32:0/1212804229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564526981a80 con 0x5645265c3390 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.453+0000 7f4915a00640 1 --2- 172.21.15.32:0/1212804229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x5645269a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.453+0000 7f4907e00640 1 --2- 172.21.15.32:0/1212804229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x5645269a1330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f48f8012240 tx=0x7f48f800fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89995a84bfa8507c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.453+0000 7f4905600640 1 -- 172.21.15.32:0/1212804229 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8029070 con 0x5645265c3390 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.453+0000 7f4905600640 1 -- 172.21.15.32:0/1212804229 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48f800fcc0 con 0x5645265c3390 2024-08-31T07:25:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.453+0000 7f4905600640 1 -- 172.21.15.32:0/1212804229 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f801cd60 con 0x5645265c3390 2024-08-31T07:25:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.454+0000 7f491e658740 1 -- 172.21.15.32:0/1212804229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 msgr2=0x5645269a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.454+0000 7f491e658740 1 --2- 172.21.15.32:0/1212804229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x5645269a1330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f48f8012240 tx=0x7f48f800fab0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.454+0000 7f491e658740 1 -- 172.21.15.32:0/1212804229 shutdown_connections 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.454+0000 7f491e658740 1 --2- 172.21.15.32:0/1212804229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265e4070 0x5645269a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.454+0000 7f491e658740 1 --2- 172.21.15.32:0/1212804229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265c3390 0x5645269a1330 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.454+0000 7f491e658740 1 --2- 172.21.15.32:0/1212804229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x56452698f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.454+0000 7f491e658740 1 -- 172.21.15.32:0/1212804229 >> 172.21.15.32:0/1212804229 conn(0x5645265d8460 msgr2=0x56452698eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.454+0000 7f491e658740 1 -- 172.21.15.32:0/1212804229 shutdown_connections 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.454+0000 7f491e658740 1 -- 172.21.15.32:0/1212804229 wait complete. 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.454+0000 7f491e658740 1 -- start start 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.455+0000 7f491e658740 1 Processor -- start 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.455+0000 7f491e658740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x56452699dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.455+0000 7f491e658740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265c3390 0x564526999f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.455+0000 7f491e658740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265e4070 0x56452699a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.455+0000 7f491e658740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b2a10 con 0x5645265c3390 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.455+0000 7f491e658740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b22e0 con 0x5645265e4070 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.455+0000 7f491e658740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f49101af540 con 0x5645265bb1b0 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.455+0000 7f491e658740 10 client.0 _ll_register_callbacks cb 0x7f48f8009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.455+0000 7f4907e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265c3390 0x564526999f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f4915a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x56452699dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f4907e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265c3390 0x564526999f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38922/0 (socket says 172.21.15.32:38922) 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f4907e00640 1 -- 172.21.15.32:0/460298199 learned_addr learned my addr 172.21.15.32:0/460298199 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f4915a00640 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265e4070 0x56452699a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f4907e00640 1 -- 172.21.15.32:0/460298199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 msgr2=0x56452699dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f4907e00640 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x56452699dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f4907e00640 1 -- 172.21.15.32:0/460298199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265e4070 msgr2=0x56452699a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f4907e00640 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265e4070 0x56452699a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f4907e00640 1 -- 172.21.15.32:0/460298199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56452699ab70 con 0x5645265c3390 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f4907e00640 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265c3390 0x564526999f70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f48f801c610 tx=0x7f48f801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b367dad88104d14c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.456+0000 7f48fea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.457+0000 7f48fea00640 1 -- 172.21.15.32:0/460298199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8031070 con 0x5645265c3390 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.457+0000 7f491e658740 1 -- 172.21.15.32:0/460298199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f800fcc0 con 0x5645265c3390 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.457+0000 7f48fea00640 1 -- 172.21.15.32:0/460298199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f48f8005250 con 0x5645265c3390 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.457+0000 7f48fea00640 1 -- 172.21.15.32:0/460298199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f48f803d050 con 0x5645265c3390 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.457+0000 7f491e658740 10 client.18165 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.457+0000 7f491e658740 1 -- 172.21.15.32:0/460298199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564526a13120 con 0x5645265c3390 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.457+0000 7f48fea00640 1 -- 172.21.15.32:0/460298199 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48f800fcc0 con 0x5645265c3390 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.457+0000 7f48fea00640 1 -- 172.21.15.32:0/460298199 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f48f8003bc0 con 0x5645265c3390 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.458+0000 7f491e658740 10 client.18165 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.458+0000 7f491e658740 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:25:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.458+0000 7f491e658740 20 client.18165 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.458+0000 7f491e658740 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564526577da0 0x564526a15860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.458+0000 7f491e658740 4 client.18165 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:25:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.458+0000 7f491e658740 1 -- 172.21.15.32:0/460298199 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645268fe1a0 con 0x564526577da0 2024-08-31T07:25:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.458+0000 7f4915a00640 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564526577da0 0x564526a15860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.458+0000 7f4915a00640 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564526577da0 0x564526a15860 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d43535949266f0 server_cookie=b32968fb7ab3d02a in_seq=0 out_seq=0 2024-08-31T07:25:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.458+0000 7f48fea00640 10 client.18165 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f48fea00640 1 -- 172.21.15.32:0/460298199 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645268fe1a0 con 0x564526577da0 2024-08-31T07:25:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f48fea00640 10 client.18165 handle_command_reply: tid=0 2024-08-31T07:25:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 client.18165 shutdown 2024-08-31T07:25:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 -- 172.21.15.32:0/460298199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265c3390 msgr2=0x564526999f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265c3390 0x564526999f70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f48f801c610 tx=0x7f48f801c910 comp rx=0 tx=0).stop 2024-08-31T07:25:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 -- 172.21.15.32:0/460298199 shutdown_connections 2024-08-31T07:25:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564526577da0 0x564526a15860 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645265e4070 0x56452699a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5645265c3390 0x564526999f70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 --2- 172.21.15.32:0/460298199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5645265bb1b0 0x56452699dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 -- 172.21.15.32:0/460298199 >> 172.21.15.32:0/460298199 conn(0x5645265d8460 msgr2=0x56452698a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f48fea00640 0 client.18165 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 -- 172.21.15.32:0/460298199 shutdown_connections 2024-08-31T07:25:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 1 -- 172.21.15.32:0/460298199 wait complete. 2024-08-31T07:25:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.459+0000 7f491e658740 20 client.18165 trim_cache size 0 max 16384 2024-08-31T07:25:19.464 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:25:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 -- 172.21.15.32:0/1194439720 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f40b4d20 msgr2=0x7f48f40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f40b4d20 0x7f48f40b7230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x564526988610 tx=0x7f4900009720 comp rx=0 tx=0).stop 2024-08-31T07:25:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 -- 172.21.15.32:0/1194439720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4910105b50 msgr2=0x7f491019c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4910105b50 0x7f491019c470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f490000e9f0 tx=0x7f490000edf0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 -- 172.21.15.32:0/1194439720 shutdown_connections 2024-08-31T07:25:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f48f40b4d20 0x7f48f40b7230 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f491010bde0 0x7f49101a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f491010b490 0x7f491019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 --2- 172.21.15.32:0/1194439720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4910105b50 0x7f491019c470 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 -- 172.21.15.32:0/1194439720 >> 172.21.15.32:0/1194439720 conn(0x7f4910101580 msgr2=0x7f491010aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 -- 172.21.15.32:0/1194439720 shutdown_connections 2024-08-31T07:25:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:19.460+0000 7f4917a00640 1 -- 172.21.15.32:0/1194439720 wait complete. 2024-08-31T07:25:19.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 381255 2.73 MB/sec execute 548 sec latency 1687.227 ms 2024-08-31T07:25:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:20 smithi032 ceph-mon[17252]: pgmap v759: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-08-31T07:25:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:20 smithi084 ceph-mon[22387]: pgmap v759: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-08-31T07:25:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:20 smithi073 ceph-mon[21995]: pgmap v759: 97 pgs: 97 active+clean; 311 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-08-31T07:25:20.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 382573 2.74 MB/sec execute 549 sec latency 2031.847 ms 2024-08-31T07:25:21.539 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:25:21.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:25:21.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 383856 2.75 MB/sec execute 550 sec latency 22.637 ms 2024-08-31T07:25:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:22 smithi084 ceph-mon[22387]: pgmap v760: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-08-31T07:25:22.497 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:22 smithi032 ceph-mon[17252]: pgmap v760: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-08-31T07:25:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:22 smithi073 ceph-mon[21995]: pgmap v760: 97 pgs: 97 active+clean; 312 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-08-31T07:25:22.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 384552 2.75 MB/sec execute 551 sec latency 339.340 ms 2024-08-31T07:25:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.096+0000 7f3b4b000640 1 -- 172.21.15.32:0/2345706930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44105b30 msgr2=0x7f3b44105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.096+0000 7f3b4b000640 1 --2- 172.21.15.32:0/2345706930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44105b30 0x7f3b44105f40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009900 tx=0x7f3b3402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.096+0000 7f3b4b000640 1 -- 172.21.15.32:0/2345706930 shutdown_connections 2024-08-31T07:25:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.096+0000 7f3b4b000640 1 --2- 172.21.15.32:0/2345706930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b4410bdc0 0x7f3b441125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.096+0000 7f3b4b000640 1 --2- 172.21.15.32:0/2345706930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b4410b470 0x7f3b4410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.096+0000 7f3b4b000640 1 --2- 172.21.15.32:0/2345706930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b44105b30 0x7f3b44105f40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.096+0000 7f3b4b000640 1 -- 172.21.15.32:0/2345706930 >> 172.21.15.32:0/2345706930 conn(0x7f3b44101600 msgr2=0x7f3b44103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.096+0000 7f3b4b000640 1 -- 172.21.15.32:0/2345706930 shutdown_connections 2024-08-31T07:25:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.096+0000 7f3b4b000640 1 -- 172.21.15.32:0/2345706930 wait complete. 2024-08-31T07:25:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.097+0000 7f3b4b000640 1 Processor -- start 2024-08-31T07:25:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.097+0000 7f3b4b000640 1 -- start start 2024-08-31T07:25:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.097+0000 7f3b4b000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b44105b30 0x7f3b4419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.097+0000 7f3b4b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b4410b470 0x7f3b4419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.097+0000 7f3b4b000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b4410bdc0 0x7f3b441a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.097+0000 7f3b4b000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44115a80 con 0x7f3b44105b30 2024-08-31T07:25:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.097+0000 7f3b4b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44115900 con 0x7f3b4410b470 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.097+0000 7f3b4b000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44115c00 con 0x7f3b4410bdc0 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b43e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b44105b30 0x7f3b4419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b49000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b4410bdc0 0x7f3b441a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b43e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b44105b30 0x7f3b4419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38936/0 (socket says 172.21.15.32:38936) 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b4410b470 0x7f3b4419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b43e00640 1 -- 172.21.15.32:0/180689962 learned_addr learned my addr 172.21.15.32:0/180689962 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b43e00640 1 -- 172.21.15.32:0/180689962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b4410bdc0 msgr2=0x7f3b441a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b43e00640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b4410bdc0 0x7f3b441a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b43e00640 1 -- 172.21.15.32:0/180689962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b4410b470 msgr2=0x7f3b4419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b43e00640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b4410b470 0x7f3b4419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b43e00640 1 -- 172.21.15.32:0/180689962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b441a23a0 con 0x7f3b44105b30 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.098+0000 7f3b3be00640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b4410b470 0x7f3b4419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.099+0000 7f3b43e00640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b44105b30 0x7f3b4419c430 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3b34005940 tx=0x7f3b340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.099+0000 7f3b41600640 1 -- 172.21.15.32:0/180689962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3403f030 con 0x7f3b44105b30 2024-08-31T07:25:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.099+0000 7f3b4b000640 1 -- 172.21.15.32:0/180689962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b441a2630 con 0x7f3b44105b30 2024-08-31T07:25:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.099+0000 7f3b41600640 1 -- 172.21.15.32:0/180689962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b340044c0 con 0x7f3b44105b30 2024-08-31T07:25:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.099+0000 7f3b4b000640 1 -- 172.21.15.32:0/180689962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b441a8df0 con 0x7f3b44105b30 2024-08-31T07:25:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.099+0000 7f3b41600640 1 -- 172.21.15.32:0/180689962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34049060 con 0x7f3b44105b30 2024-08-31T07:25:23.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.102+0000 7f3b51e19740 1 Processor -- start 2024-08-31T07:25:23.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b51e19740 1 -- start start 2024-08-31T07:25:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b41600640 1 -- 172.21.15.32:0/180689962 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3b34042030 con 0x7f3b44105b30 2024-08-31T07:25:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0a9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0aa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b51e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556f9b01f750 con 0x556f9ad03350 2024-08-31T07:25:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b51e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556f9b00ece0 con 0x556f9ace2670 2024-08-31T07:25:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b51e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556f9b012c90 con 0x556f9acda490 2024-08-31T07:25:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b49000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b49000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0a9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38952/0 (socket says 172.21.15.32:38952) 2024-08-31T07:25:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b49000640 1 -- 172.21.15.32:0/4048380988 learned_addr learned my addr 172.21.15.32:0/4048380988 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.103+0000 7f3b43e00640 1 --2- 172.21.15.32:0/4048380988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.104+0000 7f3b3be00640 1 --2- 172.21.15.32:0/4048380988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.104+0000 7f3b49000640 1 -- 172.21.15.32:0/4048380988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 msgr2=0x556f9b0a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.104+0000 7f3b49000640 1 --2- 172.21.15.32:0/4048380988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.104+0000 7f3b49000640 1 -- 172.21.15.32:0/4048380988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 msgr2=0x556f9b0aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.104+0000 7f3b49000640 1 --2- 172.21.15.32:0/4048380988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.104+0000 7f3b49000640 1 -- 172.21.15.32:0/4048380988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f9ad6e930 con 0x556f9ad03350 2024-08-31T07:25:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.104+0000 7f3b49000640 1 --2- 172.21.15.32:0/4048380988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0a9600 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009fd0 tx=0x7f3b3c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992b30a6e063edee server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.105+0000 7f3b39600640 1 -- 172.21.15.32:0/4048380988 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c004d20 con 0x556f9ad03350 2024-08-31T07:25:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.105+0000 7f3b39600640 1 -- 172.21.15.32:0/4048380988 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b3c004050 con 0x556f9ad03350 2024-08-31T07:25:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.105+0000 7f3b41600640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3b280b4ca0 0x7f3b280b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.105+0000 7f3b39600640 1 -- 172.21.15.32:0/4048380988 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c005280 con 0x556f9ad03350 2024-08-31T07:25:23.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.105+0000 7f3b41600640 1 -- 172.21.15.32:0/180689962 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f3b34105ba0 con 0x7f3b44105b30 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.105+0000 7f3b43e00640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3b280b4ca0 0x7f3b280b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.106+0000 7f3b51e19740 1 -- 172.21.15.32:0/4048380988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 msgr2=0x556f9b0a9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.106+0000 7f3b43e00640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3b280b4ca0 0x7f3b280b71b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3b341033f0 tx=0x7f3b341096f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.106+0000 7f3b51e19740 1 --2- 172.21.15.32:0/4048380988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0a9600 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009fd0 tx=0x7f3b3c0118d0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.106+0000 7f3b51e19740 1 -- 172.21.15.32:0/4048380988 shutdown_connections 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.106+0000 7f3b51e19740 1 --2- 172.21.15.32:0/4048380988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.106+0000 7f3b51e19740 1 --2- 172.21.15.32:0/4048380988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.106+0000 7f3b51e19740 1 --2- 172.21.15.32:0/4048380988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0a9600 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.106+0000 7f3b51e19740 1 -- 172.21.15.32:0/4048380988 >> 172.21.15.32:0/4048380988 conn(0x556f9acf7740 msgr2=0x556f9b0a74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.106+0000 7f3b51e19740 1 -- 172.21.15.32:0/4048380988 shutdown_connections 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.106+0000 7f3b51e19740 1 -- 172.21.15.32:0/4048380988 wait complete. 2024-08-31T07:25:23.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.107+0000 7f3b51e19740 1 -- start start 2024-08-31T07:25:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.107+0000 7f3b51e19740 1 Processor -- start 2024-08-31T07:25:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b51e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556f9ae71890 con 0x556f9ad03350 2024-08-31T07:25:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b51e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556f9af95cd0 con 0x556f9ace2670 2024-08-31T07:25:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b51e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556f9b0b7de0 con 0x556f9acda490 2024-08-31T07:25:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b51e19740 10 client.0 _ll_register_callbacks cb 0x556f9b009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:23.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52390/0 (socket says 172.21.15.32:52390) 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b3be00640 1 -- 172.21.15.32:0/3160896455 learned_addr learned my addr 172.21.15.32:0/3160896455 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b49000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b49000640 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b3be00640 1 -- 172.21.15.32:0/3160896455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 msgr2=0x556f9b0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b3be00640 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b3be00640 1 -- 172.21.15.32:0/3160896455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 msgr2=0x556f9b0c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b3be00640 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0c10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b3be00640 1 -- 172.21.15.32:0/3160896455 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f9afdf1b0 con 0x556f9ace2670 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.108+0000 7f3b49000640 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.109+0000 7f3b3be00640 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0b5690 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c012d20 tx=0x7f3b2c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1ebb190ea092d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.109+0000 7f3b32000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:25:23.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.109+0000 7f3b32000640 1 -- 172.21.15.32:0/3160896455 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c012110 con 0x556f9ace2670 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.109+0000 7f3b51e19740 1 -- 172.21.15.32:0/3160896455 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b3c004050 con 0x556f9ace2670 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.109+0000 7f3b32000640 1 -- 172.21.15.32:0/3160896455 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f3b2c004050 con 0x556f9ace2670 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.109+0000 7f3b32000640 1 -- 172.21.15.32:0/3160896455 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3b2c0122b0 con 0x556f9ace2670 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.110+0000 7f3b51e19740 10 client.28175 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.110+0000 7f3b51e19740 1 -- 172.21.15.32:0/3160896455 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556f9b0c3e50 con 0x556f9ace2670 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.110+0000 7f3b32000640 1 -- 172.21.15.32:0/3160896455 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b3c004050 con 0x556f9ace2670 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.110+0000 7f3b32000640 1 -- 172.21.15.32:0/3160896455 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f3b2c026480 con 0x556f9ace2670 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.110+0000 7f3b51e19740 10 client.28175 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.110+0000 7f3b51e19740 10 client.28175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.110+0000 7f3b51e19740 20 client.28175 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.110+0000 7f3b51e19740 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556f9acc1cd0 0x556f9b0c7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.110+0000 7f3b51e19740 4 client.28175 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:25:23.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.110+0000 7f3b51e19740 1 -- 172.21.15.32:0/3160896455 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f9b0216a0 con 0x556f9acc1cd0 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.111+0000 7f3b49000640 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556f9acc1cd0 0x556f9b0c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.111+0000 7f3b49000640 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556f9acc1cd0 0x556f9b0c7390 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cb7dca9e3122cd8 server_cookie=96aa5072b4688055 in_seq=0 out_seq=0 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.111+0000 7f3b32000640 10 client.28175 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b32000640 1 -- 172.21.15.32:0/3160896455 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f9b0216a0 con 0x556f9acc1cd0 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b32000640 10 client.28175 handle_command_reply: tid=0 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 client.28175 shutdown 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 -- 172.21.15.32:0/3160896455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 msgr2=0x556f9b0b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0b5690 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c012d20 tx=0x7f3b2c00f790 comp rx=0 tx=0).stop 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 -- 172.21.15.32:0/3160896455 shutdown_connections 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556f9acc1cd0 0x556f9b0c7390 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0c10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0b5690 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 --2- 172.21.15.32:0/3160896455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b32000640 0 client.28175 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:23.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 -- 172.21.15.32:0/3160896455 >> 172.21.15.32:0/3160896455 conn(0x556f9acf7740 msgr2=0x556f9b0ac7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:23.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 -- 172.21.15.32:0/3160896455 shutdown_connections 2024-08-31T07:25:23.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 1 -- 172.21.15.32:0/3160896455 wait complete. 2024-08-31T07:25:23.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.112+0000 7f3b51e19740 20 client.28175 trim_cache size 0 max 16384 2024-08-31T07:25:23.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.131+0000 7f3b51e19740 1 Processor -- start 2024-08-31T07:25:23.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.131+0000 7f3b51e19740 1 -- start start 2024-08-31T07:25:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.131+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.131+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.131+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0c1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.131+0000 7f3b51e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441b15d0 con 0x556f9ad03350 2024-08-31T07:25:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.131+0000 7f3b51e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441b3230 con 0x556f9ace2670 2024-08-31T07:25:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.131+0000 7f3b51e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441b1ca0 con 0x556f9acda490 2024-08-31T07:25:23.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b49000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b49000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0c1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38974/0 (socket says 172.21.15.32:38974) 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b49000640 1 -- 172.21.15.32:0/1705537329 learned_addr learned my addr 172.21.15.32:0/1705537329 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b3be00640 1 --2- 172.21.15.32:0/1705537329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b49000640 1 --2- 172.21.15.32:0/1705537329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b3be00640 1 -- 172.21.15.32:0/1705537329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 msgr2=0x556f9b0b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b3be00640 1 --2- 172.21.15.32:0/1705537329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b3be00640 1 -- 172.21.15.32:0/1705537329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 msgr2=0x556f9b0c1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b3be00640 1 --2- 172.21.15.32:0/1705537329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0c1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b3be00640 1 -- 172.21.15.32:0/1705537329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f9b0b0ac0 con 0x556f9ace2670 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b49000640 1 --2- 172.21.15.32:0/1705537329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:23.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.132+0000 7f3b3be00640 1 --2- 172.21.15.32:0/1705537329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0c0fd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c026a90 tx=0x7f3b2c00fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7ee59a922cb77f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:23.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.133+0000 7f3b38c00640 1 -- 172.21.15.32:0/1705537329 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c026b10 con 0x556f9ace2670 2024-08-31T07:25:23.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.133+0000 7f3b38c00640 1 -- 172.21.15.32:0/1705537329 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b2c026cb0 con 0x556f9ace2670 2024-08-31T07:25:23.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.133+0000 7f3b38c00640 1 -- 172.21.15.32:0/1705537329 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0039e0 con 0x556f9ace2670 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.134+0000 7f3b51e19740 1 -- 172.21.15.32:0/1705537329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 msgr2=0x556f9b0c0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.134+0000 7f3b51e19740 1 --2- 172.21.15.32:0/1705537329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0c0fd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c026a90 tx=0x7f3b2c00fc90 comp rx=0 tx=0).stop 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.134+0000 7f3b51e19740 1 -- 172.21.15.32:0/1705537329 shutdown_connections 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.134+0000 7f3b51e19740 1 --2- 172.21.15.32:0/1705537329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9ad03350 0x556f9b0c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.134+0000 7f3b51e19740 1 --2- 172.21.15.32:0/1705537329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0c0fd0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.134+0000 7f3b51e19740 1 --2- 172.21.15.32:0/1705537329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9acda490 0x556f9b0b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.134+0000 7f3b51e19740 1 -- 172.21.15.32:0/1705537329 >> 172.21.15.32:0/1705537329 conn(0x556f9acf7740 msgr2=0x556f9b0aaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.134+0000 7f3b51e19740 1 -- 172.21.15.32:0/1705537329 shutdown_connections 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.134+0000 7f3b51e19740 1 -- 172.21.15.32:0/1705537329 wait complete. 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.134+0000 7f3b51e19740 1 -- start start 2024-08-31T07:25:23.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b51e19740 1 Processor -- start 2024-08-31T07:25:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9acda490 0x556f9b0b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0ba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b51e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9ad03350 0x556f9b12ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b51e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441b2aa0 con 0x556f9acda490 2024-08-31T07:25:23.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b51e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441b2370 con 0x556f9ace2670 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b51e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441af5d0 con 0x556f9ad03350 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b51e19740 10 client.0 _ll_register_callbacks cb 0x7f3b2c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0ba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b49000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9acda490 0x556f9b0b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0ba120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52406/0 (socket says 172.21.15.32:52406) 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.135+0000 7f3b3be00640 1 -- 172.21.15.32:0/893840412 learned_addr learned my addr 172.21.15.32:0/893840412 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.136+0000 7f3b49000640 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9ad03350 0x556f9b12ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.136+0000 7f3b3be00640 1 -- 172.21.15.32:0/893840412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9ad03350 msgr2=0x556f9b12ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.136+0000 7f3b3be00640 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9ad03350 0x556f9b12ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.136+0000 7f3b3be00640 1 -- 172.21.15.32:0/893840412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9acda490 msgr2=0x556f9b0b9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.136+0000 7f3b3be00640 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9acda490 0x556f9b0b9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.136+0000 7f3b3be00640 1 -- 172.21.15.32:0/893840412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f9b12b490 con 0x556f9ace2670 2024-08-31T07:25:23.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.136+0000 7f3b49000640 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9acda490 0x556f9b0b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.136+0000 7f3b3be00640 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0ba120 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0269c0 tx=0x7f3b2c0266b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b7eeaa9c4c5e588 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.136+0000 7f3b32000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.136+0000 7f3b32000640 1 -- 172.21.15.32:0/893840412 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c034070 con 0x556f9ace2670 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b32000640 1 -- 172.21.15.32:0/893840412 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f3b2c005320 con 0x556f9ace2670 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b51e19740 1 -- 172.21.15.32:0/893840412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b2c026cb0 con 0x556f9ace2670 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b32000640 1 -- 172.21.15.32:0/893840412 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3b2c03f050 con 0x556f9ace2670 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b51e19740 10 client.28181 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b51e19740 1 -- 172.21.15.32:0/893840412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556f9b1328a0 con 0x556f9ace2670 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b32000640 1 -- 172.21.15.32:0/893840412 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b2c026cb0 con 0x556f9ace2670 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b32000640 1 -- 172.21.15.32:0/893840412 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f3b2c030500 con 0x556f9ace2670 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b51e19740 10 client.28181 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b51e19740 10 client.28181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b51e19740 20 client.28181 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b51e19740 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556f9ac97080 0x556f9b1350e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b51e19740 4 client.28181 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:25:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.137+0000 7f3b51e19740 1 -- 172.21.15.32:0/893840412 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556f9b01d140 con 0x556f9ac97080 2024-08-31T07:25:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.138+0000 7f3b49000640 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556f9ac97080 0x556f9b1350e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.138+0000 7f3b49000640 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556f9ac97080 0x556f9b1350e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=119873d419574afb server_cookie=cafb41b93f460b2c in_seq=0 out_seq=0 2024-08-31T07:25:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.138+0000 7f3b32000640 10 client.28181 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b32000640 1 -- 172.21.15.32:0/893840412 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556f9b01d140 con 0x556f9ac97080 2024-08-31T07:25:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b32000640 10 client.28181 handle_command_reply: tid=0 2024-08-31T07:25:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b51e19740 1 client.28181 shutdown 2024-08-31T07:25:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b51e19740 1 -- 172.21.15.32:0/893840412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 msgr2=0x556f9b0ba120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b51e19740 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0ba120 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0269c0 tx=0x7f3b2c0266b0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b51e19740 1 -- 172.21.15.32:0/893840412 shutdown_connections 2024-08-31T07:25:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b51e19740 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556f9ac97080 0x556f9b1350e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b51e19740 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556f9ad03350 0x556f9b12ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b51e19740 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f9ace2670 0x556f9b0ba120 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b51e19740 1 --2- 172.21.15.32:0/893840412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556f9acda490 0x556f9b0b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b51e19740 1 -- 172.21.15.32:0/893840412 >> 172.21.15.32:0/893840412 conn(0x556f9acf7740 msgr2=0x556f9b0aaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.140+0000 7f3b32000640 0 client.28181 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.141+0000 7f3b51e19740 1 -- 172.21.15.32:0/893840412 shutdown_connections 2024-08-31T07:25:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.141+0000 7f3b51e19740 1 -- 172.21.15.32:0/893840412 wait complete. 2024-08-31T07:25:23.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.141+0000 7f3b51e19740 20 client.28181 trim_cache size 0 max 16384 2024-08-31T07:25:23.143 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:25:23.143 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:25:23.144 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5", 2024-08-31T07:25:23.144 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:25:23.144 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:25:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.141+0000 7f3b4b000640 1 -- 172.21.15.32:0/180689962 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3b280b4ca0 msgr2=0x7f3b280b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.141+0000 7f3b4b000640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3b280b4ca0 0x7f3b280b71b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3b341033f0 tx=0x7f3b341096f0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.141+0000 7f3b4b000640 1 -- 172.21.15.32:0/180689962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b44105b30 msgr2=0x7f3b4419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.141+0000 7f3b4b000640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b44105b30 0x7f3b4419c430 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3b34005940 tx=0x7f3b340040c0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.142+0000 7f3b4b000640 1 -- 172.21.15.32:0/180689962 shutdown_connections 2024-08-31T07:25:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.142+0000 7f3b4b000640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3b280b4ca0 0x7f3b280b71b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.142+0000 7f3b4b000640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b4410bdc0 0x7f3b441a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.142+0000 7f3b4b000640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b4410b470 0x7f3b4419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.142+0000 7f3b4b000640 1 --2- 172.21.15.32:0/180689962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b44105b30 0x7f3b4419c430 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.142+0000 7f3b4b000640 1 -- 172.21.15.32:0/180689962 >> 172.21.15.32:0/180689962 conn(0x7f3b44101600 msgr2=0x7f3b44103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.142+0000 7f3b4b000640 1 -- 172.21.15.32:0/180689962 shutdown_connections 2024-08-31T07:25:23.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:23.142+0000 7f3b4b000640 1 -- 172.21.15.32:0/180689962 wait complete. 2024-08-31T07:25:23.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:23 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:25:23.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:23 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:23.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:23 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:25:23.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:23 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:23.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:23 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:25:23.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:23 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:23.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 385879 2.76 MB/sec execute 552 sec latency 179.132 ms 2024-08-31T07:25:24.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:25:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: pgmap v761: 97 pgs: 97 active+clean; 315 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-08-31T07:25:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:25:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:25:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:25:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:25:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:25:24.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:25:24.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:24 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.489 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: pgmap v761: 97 pgs: 97 active+clean; 315 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-08-31T07:25:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:25:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:25:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:25:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:25:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:25:24.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:25:24.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:24 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: pgmap v761: 97 pgs: 97 active+clean; 315 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-08-31T07:25:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:25:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:25:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:25:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:25:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:25:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:25:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:24 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:25:24.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 387213 2.77 MB/sec execute 553 sec latency 16.368 ms 2024-08-31T07:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.210+0000 7f4236800640 1 -- 172.21.15.32:0/546163191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4230109090 msgr2=0x7f42301094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.210+0000 7f4236800640 1 --2- 172.21.15.32:0/546163191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4230109090 0x7f42301094a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f421c009900 tx=0x7f421c02a410 comp rx=0 tx=0).stop 2024-08-31T07:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.210+0000 7f4236800640 1 -- 172.21.15.32:0/546163191 shutdown_connections 2024-08-31T07:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.210+0000 7f4236800640 1 --2- 172.21.15.32:0/546163191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42301099e0 0x7f4230110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.210+0000 7f4236800640 1 --2- 172.21.15.32:0/546163191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4230109090 0x7f42301094a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.210+0000 7f4236800640 1 --2- 172.21.15.32:0/546163191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4230103750 0x7f4230103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.210+0000 7f4236800640 1 -- 172.21.15.32:0/546163191 >> 172.21.15.32:0/546163191 conn(0x7f42300ff1e0 msgr2=0x7f4230101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.211+0000 7f4236800640 1 -- 172.21.15.32:0/546163191 shutdown_connections 2024-08-31T07:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.211+0000 7f4236800640 1 -- 172.21.15.32:0/546163191 wait complete. 2024-08-31T07:25:25.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.212+0000 7f4236800640 1 Processor -- start 2024-08-31T07:25:25.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.212+0000 7f4236800640 1 -- start start 2024-08-31T07:25:25.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.212+0000 7f4236800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4230103750 0x7f423019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.212+0000 7f4236800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230109090 0x7f423019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.212+0000 7f4236800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42301099e0 0x7f42301a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.212+0000 7f4236800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4230113670 con 0x7f4230103750 2024-08-31T07:25:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.212+0000 7f4236800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42301134f0 con 0x7f4230109090 2024-08-31T07:25:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.212+0000 7f4236800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f42301137f0 con 0x7f42301099e0 2024-08-31T07:25:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4235400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4230103750 0x7f423019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4234a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230109090 0x7f423019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4234a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230109090 0x7f423019c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52424/0 (socket says 172.21.15.32:52424) 2024-08-31T07:25:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4234a00640 1 -- 172.21.15.32:0/324306865 learned_addr learned my addr 172.21.15.32:0/324306865 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4235e00640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42301099e0 0x7f42301a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4234a00640 1 -- 172.21.15.32:0/324306865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42301099e0 msgr2=0x7f42301a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4234a00640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42301099e0 0x7f42301a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4234a00640 1 -- 172.21.15.32:0/324306865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4230103750 msgr2=0x7f423019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4234a00640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4230103750 0x7f423019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4234a00640 1 -- 172.21.15.32:0/324306865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42301a2230 con 0x7f4230109090 2024-08-31T07:25:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4235400640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4230103750 0x7f423019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.213+0000 7f4235e00640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42301099e0 0x7f42301a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:25:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.214+0000 7f4234a00640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230109090 0x7f423019c860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f421c02a930 tx=0x7f421c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.214+0000 7f422e000640 1 -- 172.21.15.32:0/324306865 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c03f030 con 0x7f4230109090 2024-08-31T07:25:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.214+0000 7f4236800640 1 -- 172.21.15.32:0/324306865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42301a24c0 con 0x7f4230109090 2024-08-31T07:25:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.214+0000 7f422e000640 1 -- 172.21.15.32:0/324306865 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f421c004490 con 0x7f4230109090 2024-08-31T07:25:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.214+0000 7f422e000640 1 -- 172.21.15.32:0/324306865 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c035070 con 0x7f4230109090 2024-08-31T07:25:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.214+0000 7f4236800640 1 -- 172.21.15.32:0/324306865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42301a8c80 con 0x7f4230109090 2024-08-31T07:25:25.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.218+0000 7f423ebcd740 1 Processor -- start 2024-08-31T07:25:25.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.219+0000 7f423ebcd740 1 -- start start 2024-08-31T07:25:25.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.219+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c11b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.219+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 0x560ee4c12060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.219+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee484b390 0x560ee4c125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.219+0000 7f423ebcd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560ee4b88770 con 0x560ee484b390 2024-08-31T07:25:25.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.219+0000 7f423ebcd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560ee4b77d00 con 0x560ee486c070 2024-08-31T07:25:25.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.219+0000 7f423ebcd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560ee4b7bcb0 con 0x560ee48431b0 2024-08-31T07:25:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4235e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c11b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4235e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c11b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52436/0 (socket says 172.21.15.32:52436) 2024-08-31T07:25:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4235e00640 1 -- 172.21.15.32:0/416155629 learned_addr learned my addr 172.21.15.32:0/416155629 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4235400640 1 --2- 172.21.15.32:0/416155629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 0x560ee4c12060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4235e00640 1 -- 172.21.15.32:0/416155629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 msgr2=0x560ee4c12060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4235e00640 1 --2- 172.21.15.32:0/416155629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 0x560ee4c12060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4235e00640 1 -- 172.21.15.32:0/416155629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee484b390 msgr2=0x560ee4c125a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:25:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4235e00640 1 --2- 172.21.15.32:0/416155629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee484b390 0x560ee4c125a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4235e00640 1 -- 172.21.15.32:0/416155629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee48d7650 con 0x560ee486c070 2024-08-31T07:25:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f422e000640 1 -- 172.21.15.32:0/324306865 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f421c042030 con 0x7f4230109090 2024-08-31T07:25:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4235e00640 1 --2- 172.21.15.32:0/416155629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c11b20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4228009fd0 tx=0x7f42280108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4877fcafd8fe2a29 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.220+0000 7f4222600640 1 -- 172.21.15.32:0/416155629 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228011760 con 0x560ee486c070 2024-08-31T07:25:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.221+0000 7f4222600640 1 -- 172.21.15.32:0/416155629 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4228011db0 con 0x560ee486c070 2024-08-31T07:25:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.221+0000 7f4222600640 1 -- 172.21.15.32:0/416155629 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422800e3c0 con 0x560ee486c070 2024-08-31T07:25:25.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.221+0000 7f423ebcd740 1 -- 172.21.15.32:0/416155629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 msgr2=0x560ee4c11b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:25.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.221+0000 7f423ebcd740 1 --2- 172.21.15.32:0/416155629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c11b20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4228009fd0 tx=0x7f42280108e0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.221+0000 7f423ebcd740 1 -- 172.21.15.32:0/416155629 shutdown_connections 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.221+0000 7f423ebcd740 1 --2- 172.21.15.32:0/416155629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee484b390 0x560ee4c125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.221+0000 7f423ebcd740 1 --2- 172.21.15.32:0/416155629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 0x560ee4c12060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.221+0000 7f423ebcd740 1 --2- 172.21.15.32:0/416155629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c11b20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.221+0000 7f423ebcd740 1 -- 172.21.15.32:0/416155629 >> 172.21.15.32:0/416155629 conn(0x560ee4860460 msgr2=0x560ee4c0fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.222+0000 7f423ebcd740 1 -- 172.21.15.32:0/416155629 shutdown_connections 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.222+0000 7f423ebcd740 1 -- 172.21.15.32:0/416155629 wait complete. 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.222+0000 7f422e000640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42180b4c30 0x7f42180b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.222+0000 7f422e000640 1 -- 172.21.15.32:0/324306865 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f421c1054d0 con 0x7f4230109090 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.222+0000 7f4235400640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42180b4c30 0x7f42180b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.222+0000 7f423ebcd740 1 -- start start 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.223+0000 7f423ebcd740 1 Processor -- start 2024-08-31T07:25:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.223+0000 7f4235400640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42180b4c30 0x7f42180b7140 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x560ee4c107a0 tx=0x7f4224005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.223+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee48431b0 0x560ee4c1d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.223+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee484b390 0x560ee4c1dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.223+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c29650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.223+0000 7f423ebcd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560ee49da700 con 0x560ee48431b0 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.223+0000 7f423ebcd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560ee4afecb0 con 0x560ee486c070 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.223+0000 7f423ebcd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560ee4c20300 con 0x560ee484b390 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.223+0000 7f423ebcd740 10 client.0 _ll_register_callbacks cb 0x560ee4b24430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.223+0000 7f4235e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c29650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4235400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee48431b0 0x560ee4c1d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4234a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee484b390 0x560ee4c1dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4235e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c29650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52452/0 (socket says 172.21.15.32:52452) 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4235e00640 1 -- 172.21.15.32:0/812213520 learned_addr learned my addr 172.21.15.32:0/812213520 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4235e00640 1 -- 172.21.15.32:0/812213520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee484b390 msgr2=0x560ee4c1dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4235e00640 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee484b390 0x560ee4c1dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4235e00640 1 -- 172.21.15.32:0/812213520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee48431b0 msgr2=0x560ee4c1d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4235e00640 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee48431b0 0x560ee4c1d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4235e00640 1 -- 172.21.15.32:0/812213520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee4b3f870 con 0x560ee486c070 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4235e00640 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c29650 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f422801bb00 tx=0x7f4228004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be8bfb36b620bbb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.224+0000 7f4216a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.225+0000 7f4216a00640 1 -- 172.21.15.32:0/812213520 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422801b840 con 0x560ee486c070 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.225+0000 7f4216a00640 1 -- 172.21.15.32:0/812213520 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f4228011560 con 0x560ee486c070 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.225+0000 7f423ebcd740 1 -- 172.21.15.32:0/812213520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4228011db0 con 0x560ee486c070 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.225+0000 7f4216a00640 1 -- 172.21.15.32:0/812213520 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4228033050 con 0x560ee486c070 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.225+0000 7f423ebcd740 10 client.28193 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.225+0000 7f423ebcd740 1 -- 172.21.15.32:0/812213520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560ee4c2c420 con 0x560ee486c070 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.225+0000 7f4216a00640 1 -- 172.21.15.32:0/812213520 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4228011db0 con 0x560ee486c070 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.226+0000 7f4216a00640 1 -- 172.21.15.32:0/812213520 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f42280055e0 con 0x560ee486c070 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.226+0000 7f423ebcd740 10 client.28193 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.226+0000 7f423ebcd740 10 client.28193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:25:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.226+0000 7f423ebcd740 20 client.28193 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.226+0000 7f423ebcd740 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ee482a9f0 0x560ee4c2f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.226+0000 7f423ebcd740 4 client.28193 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:25:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.226+0000 7f423ebcd740 1 -- 172.21.15.32:0/812213520 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ee4b8a6c0 con 0x560ee482a9f0 2024-08-31T07:25:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.226+0000 7f4235400640 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ee482a9f0 0x560ee4c2f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.227+0000 7f4235400640 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ee482a9f0 0x560ee4c2f830 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bfabc0624b26e34 server_cookie=f75818a81418e869 in_seq=0 out_seq=0 2024-08-31T07:25:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.227+0000 7f4216a00640 10 client.28193 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.227+0000 7f4216a00640 1 -- 172.21.15.32:0/812213520 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560ee4b8a6c0 con 0x560ee482a9f0 2024-08-31T07:25:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.227+0000 7f4216a00640 10 client.28193 handle_command_reply: tid=0 2024-08-31T07:25:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 client.28193 shutdown 2024-08-31T07:25:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 -- 172.21.15.32:0/812213520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 msgr2=0x560ee4c29650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c29650 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f422801bb00 tx=0x7f4228004cf0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 -- 172.21.15.32:0/812213520 shutdown_connections 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ee482a9f0 0x560ee4c2f830 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c29650 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee484b390 0x560ee4c1dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 --2- 172.21.15.32:0/812213520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee48431b0 0x560ee4c1d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 -- 172.21.15.32:0/812213520 >> 172.21.15.32:0/812213520 conn(0x560ee4860460 msgr2=0x560ee4c14cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f4216a00640 0 client.28193 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 -- 172.21.15.32:0/812213520 shutdown_connections 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 1 -- 172.21.15.32:0/812213520 wait complete. 2024-08-31T07:25:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.228+0000 7f423ebcd740 20 client.28193 trim_cache size 0 max 16384 2024-08-31T07:25:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.248+0000 7f423ebcd740 1 Processor -- start 2024-08-31T07:25:25.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.248+0000 7f423ebcd740 1 -- start start 2024-08-31T07:25:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 0x560ee4c17340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee484b390 0x560ee4c17990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c0efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f423ebcd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f42301b1390 con 0x560ee484b390 2024-08-31T07:25:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f423ebcd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42301b2ff0 con 0x560ee486c070 2024-08-31T07:25:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f423ebcd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f42301b1a60 con 0x560ee48431b0 2024-08-31T07:25:25.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f4235400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 0x560ee4c17340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f4235e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c0efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f4235400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 0x560ee4c17340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35828/0 (socket says 172.21.15.32:35828) 2024-08-31T07:25:25.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f4235400640 1 -- 172.21.15.32:0/3635121715 learned_addr learned my addr 172.21.15.32:0/3635121715 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:25.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f4234a00640 1 --2- 172.21.15.32:0/3635121715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee484b390 0x560ee4c17990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.249+0000 7f4235400640 1 -- 172.21.15.32:0/3635121715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 msgr2=0x560ee4c0efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:25.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.250+0000 7f4235400640 1 --2- 172.21.15.32:0/3635121715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c0efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.250+0000 7f4235400640 1 -- 172.21.15.32:0/3635121715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee484b390 msgr2=0x560ee4c17990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:25.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.250+0000 7f4235400640 1 --2- 172.21.15.32:0/3635121715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee484b390 0x560ee4c17990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.250+0000 7f4235400640 1 -- 172.21.15.32:0/3635121715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee4c1c4e0 con 0x560ee48431b0 2024-08-31T07:25:25.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.250+0000 7f4235400640 1 --2- 172.21.15.32:0/3635121715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 0x560ee4c17340 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x560ee4c2fda0 tx=0x7f422401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d01c87f8c28fc2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:25.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.250+0000 7f4221c00640 1 -- 172.21.15.32:0/3635121715 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224005020 con 0x560ee48431b0 2024-08-31T07:25:25.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.250+0000 7f4221c00640 1 -- 172.21.15.32:0/3635121715 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42240047a0 con 0x560ee48431b0 2024-08-31T07:25:25.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.250+0000 7f4221c00640 1 -- 172.21.15.32:0/3635121715 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422402b6d0 con 0x560ee48431b0 2024-08-31T07:25:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.251+0000 7f423ebcd740 1 -- 172.21.15.32:0/3635121715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 msgr2=0x560ee4c17340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.251+0000 7f423ebcd740 1 --2- 172.21.15.32:0/3635121715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 0x560ee4c17340 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x560ee4c2fda0 tx=0x7f422401a000 comp rx=0 tx=0).stop 2024-08-31T07:25:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.251+0000 7f423ebcd740 1 -- 172.21.15.32:0/3635121715 shutdown_connections 2024-08-31T07:25:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.251+0000 7f423ebcd740 1 --2- 172.21.15.32:0/3635121715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee486c070 0x560ee4c0efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.251+0000 7f423ebcd740 1 --2- 172.21.15.32:0/3635121715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee484b390 0x560ee4c17990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.251+0000 7f423ebcd740 1 --2- 172.21.15.32:0/3635121715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee48431b0 0x560ee4c17340 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.251+0000 7f423ebcd740 1 -- 172.21.15.32:0/3635121715 >> 172.21.15.32:0/3635121715 conn(0x560ee4860460 msgr2=0x560ee4c16c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.252+0000 7f423ebcd740 1 -- 172.21.15.32:0/3635121715 shutdown_connections 2024-08-31T07:25:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.252+0000 7f423ebcd740 1 -- 172.21.15.32:0/3635121715 wait complete. 2024-08-31T07:25:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.252+0000 7f423ebcd740 1 -- start start 2024-08-31T07:25:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f423ebcd740 1 Processor -- start 2024-08-31T07:25:25.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee48431b0 0x560ee4c215f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee484b390 0x560ee4c21b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f423ebcd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee486c070 0x560ee4c22070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f423ebcd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f42301b2860 con 0x560ee486c070 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f423ebcd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42301b2130 con 0x560ee48431b0 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f423ebcd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f42301af390 con 0x560ee484b390 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f423ebcd740 10 client.0 _ll_register_callbacks cb 0x7f4228009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f4235400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee48431b0 0x560ee4c215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f4235400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee48431b0 0x560ee4c215f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52468/0 (socket says 172.21.15.32:52468) 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.253+0000 7f4235400640 1 -- 172.21.15.32:0/2940073061 learned_addr learned my addr 172.21.15.32:0/2940073061 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.254+0000 7f4235e00640 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee486c070 0x560ee4c22070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.254+0000 7f4234a00640 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee484b390 0x560ee4c21b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.254+0000 7f4235e00640 1 -- 172.21.15.32:0/2940073061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee484b390 msgr2=0x560ee4c21b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.254+0000 7f4235e00640 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee484b390 0x560ee4c21b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.254+0000 7f4235e00640 1 -- 172.21.15.32:0/2940073061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee48431b0 msgr2=0x560ee4c215f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.254+0000 7f4235e00640 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee48431b0 0x560ee4c215f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.254+0000 7f4235e00640 1 -- 172.21.15.32:0/2940073061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee4c99080 con 0x560ee486c070 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.254+0000 7f4235400640 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee48431b0 0x560ee4c215f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.254+0000 7f4235e00640 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee486c070 0x560ee4c22070 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f42280059e0 tx=0x7f422801b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0a610a20a9d085 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.254+0000 7f4216a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.255+0000 7f4216a00640 1 -- 172.21.15.32:0/2940073061 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422801bdd0 con 0x560ee486c070 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.255+0000 7f4216a00640 1 -- 172.21.15.32:0/2940073061 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f4228005250 con 0x560ee486c070 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.255+0000 7f423ebcd740 1 -- 172.21.15.32:0/2940073061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42240047a0 con 0x560ee486c070 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.255+0000 7f4216a00640 1 -- 172.21.15.32:0/2940073061 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4228045020 con 0x560ee486c070 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.255+0000 7f423ebcd740 10 client.18198 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:25.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.255+0000 7f423ebcd740 1 -- 172.21.15.32:0/2940073061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560ee4c9a2f0 con 0x560ee486c070 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.255+0000 7f4216a00640 1 -- 172.21.15.32:0/2940073061 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42240047a0 con 0x560ee486c070 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.255+0000 7f4216a00640 1 -- 172.21.15.32:0/2940073061 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f4228011560 con 0x560ee486c070 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.256+0000 7f423ebcd740 10 client.18198 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.256+0000 7f423ebcd740 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.256+0000 7f423ebcd740 20 client.18198 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.256+0000 7f423ebcd740 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ee47ffda0 0x560ee4c9ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.256+0000 7f423ebcd740 4 client.18198 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.256+0000 7f423ebcd740 1 -- 172.21.15.32:0/2940073061 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560ee4b86160 con 0x560ee47ffda0 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.256+0000 7f4235400640 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ee47ffda0 0x560ee4c9ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.256+0000 7f4235400640 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ee47ffda0 0x560ee4c9ca20 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25714195c74d1f6b server_cookie=c92cfd83acbef15b in_seq=0 out_seq=0 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.256+0000 7f4216a00640 10 client.18198 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f4216a00640 1 -- 172.21.15.32:0/2940073061 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x560ee4b86160 con 0x560ee47ffda0 2024-08-31T07:25:25.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f4216a00640 10 client.18198 handle_command_reply: tid=0 2024-08-31T07:25:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 client.18198 shutdown 2024-08-31T07:25:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 -- 172.21.15.32:0/2940073061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee486c070 msgr2=0x560ee4c22070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee486c070 0x560ee4c22070 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f42280059e0 tx=0x7f422801b8f0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 -- 172.21.15.32:0/2940073061 shutdown_connections 2024-08-31T07:25:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560ee47ffda0 0x560ee4c9ca20 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ee486c070 0x560ee4c22070 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560ee484b390 0x560ee4c21b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f4216a00640 0 client.18198 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 --2- 172.21.15.32:0/2940073061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ee48431b0 0x560ee4c215f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 -- 172.21.15.32:0/2940073061 >> 172.21.15.32:0/2940073061 conn(0x560ee4860460 msgr2=0x560ee4c12b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 -- 172.21.15.32:0/2940073061 shutdown_connections 2024-08-31T07:25:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.257+0000 7f423ebcd740 1 -- 172.21.15.32:0/2940073061 wait complete. 2024-08-31T07:25:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.258+0000 7f423ebcd740 20 client.18198 trim_cache size 0 max 16384 2024-08-31T07:25:25.261 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5", 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.262 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.263 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:25:25.263 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:25:25.263 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:25:25.266 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.267 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:25:25.270 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:25:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:25:25.271 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:25:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:25:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:25.272 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:25.273 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:25.273 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:25.273 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:25.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:25:25.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:25:25.273 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:25:25.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.258+0000 7f4236800640 1 -- 172.21.15.32:0/324306865 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42180b4c30 msgr2=0x7f42180b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:25.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.258+0000 7f4236800640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42180b4c30 0x7f42180b7140 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x560ee4c107a0 tx=0x7f4224005f90 comp rx=0 tx=0).stop 2024-08-31T07:25:25.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.258+0000 7f4236800640 1 -- 172.21.15.32:0/324306865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230109090 msgr2=0x7f423019c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:25.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.258+0000 7f4236800640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230109090 0x7f423019c860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f421c02a930 tx=0x7f421c031a20 comp rx=0 tx=0).stop 2024-08-31T07:25:25.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.258+0000 7f4236800640 1 -- 172.21.15.32:0/324306865 shutdown_connections 2024-08-31T07:25:25.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.258+0000 7f4236800640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f42180b4c30 0x7f42180b7140 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.258+0000 7f4236800640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42301099e0 0x7f42301a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.258+0000 7f4236800640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230109090 0x7f423019c860 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.258+0000 7f4236800640 1 --2- 172.21.15.32:0/324306865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4230103750 0x7f423019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:25.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.259+0000 7f4236800640 1 -- 172.21.15.32:0/324306865 >> 172.21.15.32:0/324306865 conn(0x7f42300ff1e0 msgr2=0x7f423010e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:25.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.259+0000 7f4236800640 1 -- 172.21.15.32:0/324306865 shutdown_connections 2024-08-31T07:25:25.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:25.259+0000 7f4236800640 1 -- 172.21.15.32:0/324306865 wait complete. 2024-08-31T07:25:25.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 388325 2.77 MB/sec execute 554 sec latency 69.531 ms 2024-08-31T07:25:26.172 INFO:tasks.cephfs.filesystem:scrub status for tag:e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5 - {'path': '/', 'tag': 'e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5', 'options': 'recursive,force'} 2024-08-31T07:25:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:26 smithi032 ceph-mon[17252]: pgmap v762: 97 pgs: 97 active+clean; 317 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-08-31T07:25:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:26 smithi073 ceph-mon[21995]: pgmap v762: 97 pgs: 97 active+clean; 317 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-08-31T07:25:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:26 smithi084 ceph-mon[22387]: pgmap v762: 97 pgs: 97 active+clean; 317 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-08-31T07:25:26.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 388368 2.77 MB/sec execute 555 sec latency 958.147 ms 2024-08-31T07:25:27.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 388368 2.76 MB/sec execute 556 sec latency 1958.233 ms 2024-08-31T07:25:28.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:28 smithi032 ceph-mon[17252]: pgmap v763: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-08-31T07:25:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:28 smithi073 ceph-mon[21995]: pgmap v763: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-08-31T07:25:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:28 smithi084 ceph-mon[22387]: pgmap v763: 97 pgs: 97 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-08-31T07:25:28.780 INFO:tasks.workunit.client.0.smithi032.stdout: 1 388937 2.76 MB/sec execute 557 sec latency 2575.036 ms 2024-08-31T07:25:29.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 388937 2.75 MB/sec execute 558 sec latency 1161.870 ms 2024-08-31T07:25:30.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:30 smithi032 ceph-mon[17252]: pgmap v764: 97 pgs: 97 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-08-31T07:25:30.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:30 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:25:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:30 smithi073 ceph-mon[21995]: pgmap v764: 97 pgs: 97 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-08-31T07:25:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:30 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:25:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:30 smithi084 ceph-mon[22387]: pgmap v764: 97 pgs: 97 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-08-31T07:25:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:30 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:25:30.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 388937 2.75 MB/sec execute 559 sec latency 2162.012 ms 2024-08-31T07:25:31.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:31 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:25:31.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:31 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:25:31.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:31 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:25:31.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:31 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:25:31.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:31 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:25:31.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:31 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:25:31.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:31 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:25:31.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:31 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:25:31.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:31 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:25:31.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 388937 2.74 MB/sec execute 560 sec latency 3162.173 ms 2024-08-31T07:25:32.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 388972 2.74 MB/sec execute 561 sec latency 3706.805 ms 2024-08-31T07:25:32.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:32 smithi032 ceph-mon[17252]: pgmap v765: 97 pgs: 97 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-08-31T07:25:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:32 smithi073 ceph-mon[21995]: pgmap v765: 97 pgs: 97 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-08-31T07:25:32.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:32 smithi084 ceph-mon[22387]: pgmap v765: 97 pgs: 97 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-08-31T07:25:33.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 390474 2.75 MB/sec execute 562 sec latency 429.387 ms 2024-08-31T07:25:34.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 391441 2.75 MB/sec execute 563 sec latency 30.732 ms 2024-08-31T07:25:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:34 smithi073 ceph-mon[21995]: pgmap v766: 97 pgs: 97 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.6 MiB/s wr, 153 op/s 2024-08-31T07:25:34.971 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:34 smithi032 ceph-mon[17252]: pgmap v766: 97 pgs: 97 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.6 MiB/s wr, 153 op/s 2024-08-31T07:25:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:34 smithi084 ceph-mon[22387]: pgmap v766: 97 pgs: 97 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.6 MiB/s wr, 153 op/s 2024-08-31T07:25:35.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 391911 2.75 MB/sec execute 564 sec latency 530.972 ms 2024-08-31T07:25:36.781 INFO:tasks.workunit.client.0.smithi032.stdout: 1 391911 2.75 MB/sec execute 565 sec latency 1531.114 ms 2024-08-31T07:25:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:36 smithi073 ceph-mon[21995]: pgmap v767: 97 pgs: 97 active+clean; 323 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-08-31T07:25:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:36 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:25:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:36 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:25:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:36 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:25:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:36 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:25:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:36 smithi084 ceph-mon[22387]: pgmap v767: 97 pgs: 97 active+clean; 323 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-08-31T07:25:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:36 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:25:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:36 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:25:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:36 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:25:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:36 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:25:37.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:36 smithi032 ceph-mon[17252]: pgmap v767: 97 pgs: 97 active+clean; 323 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-08-31T07:25:37.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:36 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:25:37.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:36 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:25:37.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:36 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:25:37.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:36 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:25:37.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 391911 2.75 MB/sec execute 566 sec latency 2531.201 ms 2024-08-31T07:25:37.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:37 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:25:37.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:37 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:25:38.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:37 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:25:38.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 391911 2.74 MB/sec execute 567 sec latency 3531.321 ms 2024-08-31T07:25:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:38 smithi073 ceph-mon[21995]: pgmap v768: 97 pgs: 97 active+clean; 328 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-08-31T07:25:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:38 smithi084 ceph-mon[22387]: pgmap v768: 97 pgs: 97 active+clean; 328 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-08-31T07:25:39.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:38 smithi032 ceph-mon[17252]: pgmap v768: 97 pgs: 97 active+clean; 328 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-08-31T07:25:39.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 391911 2.74 MB/sec execute 568 sec latency 4531.459 ms 2024-08-31T07:25:40.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 392842 2.74 MB/sec execute 569 sec latency 4878.303 ms 2024-08-31T07:25:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:40 smithi073 ceph-mon[21995]: pgmap v769: 97 pgs: 97 active+clean; 328 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-08-31T07:25:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:40 smithi084 ceph-mon[22387]: pgmap v769: 97 pgs: 97 active+clean; 328 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-08-31T07:25:41.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:40 smithi032 ceph-mon[17252]: pgmap v769: 97 pgs: 97 active+clean; 328 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-08-31T07:25:41.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 394229 2.75 MB/sec execute 570 sec latency 152.842 ms 2024-08-31T07:25:42.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:42 smithi032 ceph-mon[17252]: pgmap v770: 97 pgs: 97 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-08-31T07:25:42.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 395060 2.75 MB/sec execute 571 sec latency 373.548 ms 2024-08-31T07:25:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:42 smithi073 ceph-mon[21995]: pgmap v770: 97 pgs: 97 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-08-31T07:25:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:42 smithi084 ceph-mon[22387]: pgmap v770: 97 pgs: 97 active+clean; 329 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-08-31T07:25:43.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 395459 2.75 MB/sec execute 572 sec latency 495.621 ms 2024-08-31T07:25:44.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 395459 2.74 MB/sec execute 573 sec latency 1482.775 ms 2024-08-31T07:25:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:44 smithi073 ceph-mon[21995]: pgmap v771: 97 pgs: 97 active+clean; 330 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-08-31T07:25:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:44 smithi084 ceph-mon[22387]: pgmap v771: 97 pgs: 97 active+clean; 330 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-08-31T07:25:45.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:44 smithi032 ceph-mon[17252]: pgmap v771: 97 pgs: 97 active+clean; 330 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-08-31T07:25:45.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 396740 2.75 MB/sec execute 574 sec latency 1829.932 ms 2024-08-31T07:25:46.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 397759 2.76 MB/sec execute 575 sec latency 181.437 ms 2024-08-31T07:25:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:46 smithi073 ceph-mon[21995]: pgmap v772: 97 pgs: 97 active+clean; 330 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-08-31T07:25:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:46 smithi084 ceph-mon[22387]: pgmap v772: 97 pgs: 97 active+clean; 330 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-08-31T07:25:47.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:46 smithi032 ceph-mon[17252]: pgmap v772: 97 pgs: 97 active+clean; 330 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-08-31T07:25:47.782 INFO:tasks.workunit.client.0.smithi032.stdout: 1 398734 2.76 MB/sec execute 576 sec latency 242.175 ms 2024-08-31T07:25:48.783 INFO:tasks.workunit.client.0.smithi032.stdout: 1 399001 2.76 MB/sec execute 577 sec latency 633.255 ms 2024-08-31T07:25:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:48 smithi073 ceph-mon[21995]: pgmap v773: 97 pgs: 97 active+clean; 334 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-08-31T07:25:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:48 smithi084 ceph-mon[22387]: pgmap v773: 97 pgs: 97 active+clean; 334 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-08-31T07:25:49.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:48 smithi032 ceph-mon[17252]: pgmap v773: 97 pgs: 97 active+clean; 334 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-08-31T07:25:49.783 INFO:tasks.workunit.client.0.smithi032.stdout: 1 399001 2.75 MB/sec execute 578 sec latency 1633.338 ms 2024-08-31T07:25:50.783 INFO:tasks.workunit.client.0.smithi032.stdout: 1 400183 2.76 MB/sec execute 579 sec latency 1976.314 ms 2024-08-31T07:25:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:50 smithi073 ceph-mon[21995]: pgmap v774: 97 pgs: 97 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-08-31T07:25:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:50 smithi084 ceph-mon[22387]: pgmap v774: 97 pgs: 97 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-08-31T07:25:51.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:50 smithi032 ceph-mon[17252]: pgmap v774: 97 pgs: 97 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-08-31T07:25:51.783 INFO:tasks.workunit.client.0.smithi032.stdout: 1 401337 2.77 MB/sec execute 580 sec latency 35.000 ms 2024-08-31T07:25:52.783 INFO:tasks.workunit.client.0.smithi032.stdout: 1 402243 2.77 MB/sec execute 581 sec latency 295.036 ms 2024-08-31T07:25:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:52 smithi073 ceph-mon[21995]: pgmap v775: 97 pgs: 97 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-08-31T07:25:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:52 smithi084 ceph-mon[22387]: pgmap v775: 97 pgs: 97 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-08-31T07:25:53.061 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:52 smithi032 ceph-mon[17252]: pgmap v775: 97 pgs: 97 active+clean; 336 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-08-31T07:25:53.783 INFO:tasks.workunit.client.0.smithi032.stdout: 1 402551 2.77 MB/sec execute 582 sec latency 646.125 ms 2024-08-31T07:25:54.783 INFO:tasks.workunit.client.0.smithi032.stdout: 1 402551 2.76 MB/sec execute 583 sec latency 1163.033 ms 2024-08-31T07:25:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:54 smithi073 ceph-mon[21995]: pgmap v776: 97 pgs: 97 active+clean; 337 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-08-31T07:25:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:54 smithi084 ceph-mon[22387]: pgmap v776: 97 pgs: 97 active+clean; 337 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-08-31T07:25:55.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:54 smithi032 ceph-mon[17252]: pgmap v776: 97 pgs: 97 active+clean; 337 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-08-31T07:25:55.783 INFO:tasks.workunit.client.0.smithi032.stdout: 1 403839 2.77 MB/sec execute 584 sec latency 1509.227 ms 2024-08-31T07:25:56.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:25:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:56.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 404831 2.78 MB/sec execute 585 sec latency 199.439 ms 2024-08-31T07:25:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:56 smithi073 ceph-mon[21995]: pgmap v777: 97 pgs: 97 active+clean; 341 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-08-31T07:25:56.951 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:56 smithi032 ceph-mon[17252]: pgmap v777: 97 pgs: 97 active+clean; 341 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-08-31T07:25:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:56 smithi084 ceph-mon[22387]: pgmap v777: 97 pgs: 97 active+clean; 341 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-08-31T07:25:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.295+0000 7f17e6400640 1 -- 172.21.15.32:0/888184598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102200 msgr2=0x7f17e0102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.295+0000 7f17e6400640 1 --2- 172.21.15.32:0/888184598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102200 0x7f17e0102610 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009900 tx=0x7f17d402a480 comp rx=0 tx=0).stop 2024-08-31T07:25:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.296+0000 7f17e6400640 1 -- 172.21.15.32:0/888184598 shutdown_connections 2024-08-31T07:25:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.296+0000 7f17e6400640 1 --2- 172.21.15.32:0/888184598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e010fdb0 0x7f17e01121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.296+0000 7f17e6400640 1 --2- 172.21.15.32:0/888184598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e0102b50 0x7f17e010f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.296+0000 7f17e6400640 1 --2- 172.21.15.32:0/888184598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102200 0x7f17e0102610 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.296+0000 7f17e6400640 1 -- 172.21.15.32:0/888184598 >> 172.21.15.32:0/888184598 conn(0x7f17e00fe060 msgr2=0x7f17e01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.296+0000 7f17e6400640 1 -- 172.21.15.32:0/888184598 shutdown_connections 2024-08-31T07:25:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.296+0000 7f17e6400640 1 -- 172.21.15.32:0/888184598 wait complete. 2024-08-31T07:25:57.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.297+0000 7f17e6400640 1 Processor -- start 2024-08-31T07:25:57.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.297+0000 7f17e6400640 1 -- start start 2024-08-31T07:25:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.297+0000 7f17e6400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e0102200 0x7f17e019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.297+0000 7f17e6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b50 0x7f17e019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e6400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e010fdb0 0x7f17e01a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e6400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0115830 con 0x7f17e0102b50 2024-08-31T07:25:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e6400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01156b0 con 0x7f17e0102200 2024-08-31T07:25:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e6400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01159b0 con 0x7f17e010fdb0 2024-08-31T07:25:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e0102200 0x7f17e019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e0102200 0x7f17e019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55808/0 (socket says 172.21.15.32:55808) 2024-08-31T07:25:57.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e5000640 1 -- 172.21.15.32:0/109991565 learned_addr learned my addr 172.21.15.32:0/109991565 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17dfe00640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b50 0x7f17e019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e5a00640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e010fdb0 0x7f17e01a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e5000640 1 -- 172.21.15.32:0/109991565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e010fdb0 msgr2=0x7f17e01a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e5000640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e010fdb0 0x7f17e01a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e5000640 1 -- 172.21.15.32:0/109991565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b50 msgr2=0x7f17e019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e5000640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b50 0x7f17e019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e5000640 1 -- 172.21.15.32:0/109991565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e01a2210 con 0x7f17e0102200 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.298+0000 7f17e5a00640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e010fdb0 0x7f17e01a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.299+0000 7f17e5000640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e0102200 0x7f17e019c2a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f17d4005ab0 tx=0x7f17d4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.299+0000 7f17dd600640 1 -- 172.21.15.32:0/109991565 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d403f070 con 0x7f17e0102200 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.299+0000 7f17dd600640 1 -- 172.21.15.32:0/109991565 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d4031b60 con 0x7f17e0102200 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.299+0000 7f17e6400640 1 -- 172.21.15.32:0/109991565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e01a24a0 con 0x7f17e0102200 2024-08-31T07:25:57.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.299+0000 7f17dd600640 1 -- 172.21.15.32:0/109991565 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4035030 con 0x7f17e0102200 2024-08-31T07:25:57.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.299+0000 7f17e6400640 1 -- 172.21.15.32:0/109991565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e01a8c60 con 0x7f17e0102200 2024-08-31T07:25:57.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.303+0000 7f17dd600640 1 -- 172.21.15.32:0/109991565 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f17d4042070 con 0x7f17e0102200 2024-08-31T07:25:57.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.303+0000 7f17ee7c8740 1 Processor -- start 2024-08-31T07:25:57.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17ee7c8740 1 -- start start 2024-08-31T07:25:57.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586e9070 0x563b58a8eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c01b0 0x563b58a8f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c8390 0x563b58a8f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563b58a05770 con 0x563b586e9070 2024-08-31T07:25:57.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563b589f4d00 con 0x563b586c8390 2024-08-31T07:25:57.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563b589f8cb0 con 0x563b586c01b0 2024-08-31T07:25:57.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17dfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c8390 0x563b58a8f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17e5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586e9070 0x563b58a8eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17e5000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c01b0 0x563b58a8f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17dfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c8390 0x563b58a8f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55820/0 (socket says 172.21.15.32:55820) 2024-08-31T07:25:57.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.304+0000 7f17dfe00640 1 -- 172.21.15.32:0/1813368868 learned_addr learned my addr 172.21.15.32:0/1813368868 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:57.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.305+0000 7f17dfe00640 1 -- 172.21.15.32:0/1813368868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c01b0 msgr2=0x563b58a8f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:57.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.305+0000 7f17dfe00640 1 --2- 172.21.15.32:0/1813368868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c01b0 0x563b58a8f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.305+0000 7f17dfe00640 1 -- 172.21.15.32:0/1813368868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586e9070 msgr2=0x563b58a8eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.305+0000 7f17dfe00640 1 --2- 172.21.15.32:0/1813368868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586e9070 0x563b58a8eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.305+0000 7f17dfe00640 1 -- 172.21.15.32:0/1813368868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b58754650 con 0x563b586c8390 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.305+0000 7f17dfe00640 1 --2- 172.21.15.32:0/1813368868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c8390 0x563b58a8f5a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009c10 tx=0x7f17cc0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=980c8a225e5303a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.305+0000 7f17dd600640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c80b4d20 0x7f17c80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.305+0000 7f17d2000640 1 -- 172.21.15.32:0/1813368868 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc012450 con 0x563b586c8390 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.306+0000 7f17d2000640 1 -- 172.21.15.32:0/1813368868 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc012aa0 con 0x563b586c8390 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.306+0000 7f17d2000640 1 -- 172.21.15.32:0/1813368868 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00fb00 con 0x563b586c8390 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.306+0000 7f17dd600640 1 -- 172.21.15.32:0/109991565 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f17d4105930 con 0x7f17e0102200 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.306+0000 7f17e5a00640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c80b4d20 0x7f17c80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.306+0000 7f17ee7c8740 1 -- 172.21.15.32:0/1813368868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c8390 msgr2=0x563b58a8f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.306+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/1813368868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c8390 0x563b58a8f5a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009c10 tx=0x7f17cc0117b0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.307+0000 7f17e5a00640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c80b4d20 0x7f17c80b7230 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f17d8005fd0 tx=0x7f17d8009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:57.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.307+0000 7f17ee7c8740 1 -- 172.21.15.32:0/1813368868 shutdown_connections 2024-08-31T07:25:57.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.307+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/1813368868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c8390 0x563b58a8f5a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.307+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/1813368868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c01b0 0x563b58a8f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.307+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/1813368868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586e9070 0x563b58a8eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.307+0000 7f17ee7c8740 1 -- 172.21.15.32:0/1813368868 >> 172.21.15.32:0/1813368868 conn(0x563b586dd460 msgr2=0x563b58a8ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:57.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.307+0000 7f17ee7c8740 1 -- 172.21.15.32:0/1813368868 shutdown_connections 2024-08-31T07:25:57.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.307+0000 7f17ee7c8740 1 -- 172.21.15.32:0/1813368868 wait complete. 2024-08-31T07:25:57.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.307+0000 7f17ee7c8740 1 -- start start 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.308+0000 7f17ee7c8740 1 Processor -- start 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.308+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c01b0 0x563b58a9a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.308+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a9acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.308+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586e9070 0x563b58aa65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.308+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563b58857700 con 0x563b586e9070 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.308+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563b5897bcb0 con 0x563b586c01b0 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.308+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563b58a9d300 con 0x563b586c8390 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.308+0000 7f17ee7c8740 10 client.0 _ll_register_callbacks cb 0x563b589a1430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.308+0000 7f17e5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c01b0 0x563b58a9a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17e5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c01b0 0x563b58a9a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55834/0 (socket says 172.21.15.32:55834) 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17e5000640 1 -- 172.21.15.32:0/3214282041 learned_addr learned my addr 172.21.15.32:0/3214282041 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:57.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17dfe00640 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a9acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17e5a00640 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586e9070 0x563b58aa65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17e5000640 1 -- 172.21.15.32:0/3214282041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 msgr2=0x563b58a9acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17e5000640 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a9acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17e5000640 1 -- 172.21.15.32:0/3214282041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586e9070 msgr2=0x563b58aa65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17e5000640 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586e9070 0x563b58aa65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17e5000640 1 -- 172.21.15.32:0/3214282041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b589bc870 con 0x563b586c01b0 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17e5000640 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c01b0 0x563b58a9a750 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f17d4031440 tx=0x7f17d4009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c11bc43d078eec1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.309+0000 7f17c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.310+0000 7f17c6a00640 1 -- 172.21.15.32:0/3214282041 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4049070 con 0x563b586c01b0 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.310+0000 7f17c6a00640 1 -- 172.21.15.32:0/3214282041 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f17d403b460 con 0x563b586c01b0 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.310+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3214282041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17cc012aa0 con 0x563b586c01b0 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.310+0000 7f17c6a00640 1 -- 172.21.15.32:0/3214282041 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f17d40b1e60 con 0x563b586c01b0 2024-08-31T07:25:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.310+0000 7f17ee7c8740 10 client.28211 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:57.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.310+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3214282041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x563b58aa9390 con 0x563b586c01b0 2024-08-31T07:25:57.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.310+0000 7f17c6a00640 1 -- 172.21.15.32:0/3214282041 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc012aa0 con 0x563b586c01b0 2024-08-31T07:25:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.310+0000 7f17c6a00640 1 -- 172.21.15.32:0/3214282041 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f17d404c060 con 0x563b586c01b0 2024-08-31T07:25:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.310+0000 7f17ee7c8740 10 client.28211 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.310+0000 7f17ee7c8740 10 client.28211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:25:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.311+0000 7f17ee7c8740 20 client.28211 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.311+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563b586a79f0 0x563b58aaca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.311+0000 7f17ee7c8740 4 client.28211 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.311+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3214282041 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b58a076c0 con 0x563b586a79f0 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.311+0000 7f17dfe00640 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563b586a79f0 0x563b58aaca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.311+0000 7f17dfe00640 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563b586a79f0 0x563b58aaca00 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba4cfa6b3e99b01 server_cookie=c6108ad1b7d4b0b6 in_seq=0 out_seq=0 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.311+0000 7f17c6a00640 10 client.28211 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.312+0000 7f17c6a00640 1 -- 172.21.15.32:0/3214282041 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b58a076c0 con 0x563b586a79f0 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.312+0000 7f17c6a00640 10 client.28211 handle_command_reply: tid=0 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.312+0000 7f17ee7c8740 1 client.28211 shutdown 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3214282041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c01b0 msgr2=0x563b58a9a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c01b0 0x563b58a9a750 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f17d4031440 tx=0x7f17d4009620 comp rx=0 tx=0).stop 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3214282041 shutdown_connections 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563b586a79f0 0x563b58aaca00 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586e9070 0x563b58aa65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a9acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3214282041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586c01b0 0x563b58a9a750 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17c6a00640 0 client.28211 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:57.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3214282041 >> 172.21.15.32:0/3214282041 conn(0x563b586dd460 msgr2=0x563b58a91d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:57.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3214282041 shutdown_connections 2024-08-31T07:25:57.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3214282041 wait complete. 2024-08-31T07:25:57.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.313+0000 7f17ee7c8740 20 client.28211 trim_cache size 0 max 16384 2024-08-31T07:25:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.333+0000 7f17ee7c8740 1 Processor -- start 2024-08-31T07:25:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.333+0000 7f17ee7c8740 1 -- start start 2024-08-31T07:25:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586c01b0 0x563b58aa64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a87630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586e9070 0x563b58a93c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01b13e0 con 0x563b586c01b0 2024-08-31T07:25:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01b3040 con 0x563b586e9070 2024-08-31T07:25:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01b1ab0 con 0x563b586c8390 2024-08-31T07:25:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17dfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a87630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17e5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586e9070 0x563b58a93c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17dfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a87630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60978/0 (socket says 172.21.15.32:60978) 2024-08-31T07:25:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17e5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586c01b0 0x563b58aa64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17dfe00640 1 -- 172.21.15.32:0/414663448 learned_addr learned my addr 172.21.15.32:0/414663448 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17dfe00640 1 -- 172.21.15.32:0/414663448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586e9070 msgr2=0x563b58a93c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17dfe00640 1 --2- 172.21.15.32:0/414663448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586e9070 0x563b58a93c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17dfe00640 1 -- 172.21.15.32:0/414663448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586c01b0 msgr2=0x563b58aa64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.334+0000 7f17dfe00640 1 --2- 172.21.15.32:0/414663448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586c01b0 0x563b58aa64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.335+0000 7f17dfe00640 1 -- 172.21.15.32:0/414663448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b58a95de0 con 0x563b586c8390 2024-08-31T07:25:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.335+0000 7f17dfe00640 1 --2- 172.21.15.32:0/414663448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a87630 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f17cc01c040 tx=0x7f17cc00f660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=847bb3ee18d7203c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.335+0000 7f17d1600640 1 -- 172.21.15.32:0/414663448 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc009ea0 con 0x563b586c8390 2024-08-31T07:25:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.335+0000 7f17d1600640 1 -- 172.21.15.32:0/414663448 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc00f4b0 con 0x563b586c8390 2024-08-31T07:25:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.335+0000 7f17d1600640 1 -- 172.21.15.32:0/414663448 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00fa70 con 0x563b586c8390 2024-08-31T07:25:57.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.336+0000 7f17ee7c8740 1 -- 172.21.15.32:0/414663448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 msgr2=0x563b58a87630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:57.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.336+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/414663448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a87630 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f17cc01c040 tx=0x7f17cc00f660 comp rx=0 tx=0).stop 2024-08-31T07:25:57.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.336+0000 7f17ee7c8740 1 -- 172.21.15.32:0/414663448 shutdown_connections 2024-08-31T07:25:57.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.336+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/414663448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586e9070 0x563b58a93c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.336+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/414663448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a87630 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.336+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/414663448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586c01b0 0x563b58aa64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.336+0000 7f17ee7c8740 1 -- 172.21.15.32:0/414663448 >> 172.21.15.32:0/414663448 conn(0x563b586dd460 msgr2=0x563b58a98dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:57.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.336+0000 7f17ee7c8740 1 -- 172.21.15.32:0/414663448 shutdown_connections 2024-08-31T07:25:57.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.336+0000 7f17ee7c8740 1 -- 172.21.15.32:0/414663448 wait complete. 2024-08-31T07:25:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.337+0000 7f17ee7c8740 1 -- start start 2024-08-31T07:25:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.337+0000 7f17ee7c8740 1 Processor -- start 2024-08-31T07:25:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.337+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586c01b0 0x563b58a9eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.337+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a9f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.337+0000 7f17ee7c8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586e9070 0x563b58b10390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.337+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01b28b0 con 0x563b586c01b0 2024-08-31T07:25:57.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.337+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01b2180 con 0x563b586e9070 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.337+0000 7f17ee7c8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01af3e0 con 0x563b586c8390 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17ee7c8740 10 client.0 _ll_register_callbacks cb 0x7f17d4009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17dfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a9f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17dfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a9f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:60984/0 (socket says 172.21.15.32:60984) 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17dfe00640 1 -- 172.21.15.32:0/3114961105 learned_addr learned my addr 172.21.15.32:0/3114961105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17e5000640 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586c01b0 0x563b58a9eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17e5a00640 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586e9070 0x563b58b10390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17dfe00640 1 -- 172.21.15.32:0/3114961105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586e9070 msgr2=0x563b58b10390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17dfe00640 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586e9070 0x563b58b10390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17dfe00640 1 -- 172.21.15.32:0/3114961105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586c01b0 msgr2=0x563b58a9eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17dfe00640 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586c01b0 0x563b58a9eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.338+0000 7f17dfe00640 1 -- 172.21.15.32:0/3114961105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b58b10a00 con 0x563b586c8390 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.339+0000 7f17dfe00640 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a9f470 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009450 tx=0x7f17cc009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=568b634458181eac server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:25:57.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.339+0000 7f17c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:25:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.339+0000 7f17c6a00640 1 -- 172.21.15.32:0/3114961105 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc031070 con 0x563b586c8390 2024-08-31T07:25:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.339+0000 7f17c6a00640 1 -- 172.21.15.32:0/3114961105 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f17cc005450 con 0x563b586c8390 2024-08-31T07:25:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.339+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3114961105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17cc00f4b0 con 0x563b586c8390 2024-08-31T07:25:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.339+0000 7f17c6a00640 1 -- 172.21.15.32:0/3114961105 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f17cc043050 con 0x563b586c8390 2024-08-31T07:25:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17ee7c8740 10 client.28219 fetch_fsmap learned FSMap version 47 2024-08-31T07:25:57.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3114961105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x563b58b17de0 con 0x563b586c8390 2024-08-31T07:25:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17c6a00640 1 -- 172.21.15.32:0/3114961105 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc00f4b0 con 0x563b586c8390 2024-08-31T07:25:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17c6a00640 1 -- 172.21.15.32:0/3114961105 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f17cc004900 con 0x563b586c8390 2024-08-31T07:25:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17ee7c8740 10 client.28219 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:25:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17ee7c8740 10 client.28219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:25:57.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17ee7c8740 20 client.28219 populate_metadata read hostname 'smithi032' 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563b5867cda0 0x563b58b1a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17ee7c8740 4 client.28219 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3114961105 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b58a03160 con 0x563b5867cda0 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.340+0000 7f17e5000640 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563b5867cda0 0x563b58b1a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.341+0000 7f17e5000640 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563b5867cda0 0x563b58b1a390 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7fcacdbfb8cdfb server_cookie=64bed10ac424061 in_seq=0 out_seq=0 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.341+0000 7f17c6a00640 10 client.28219 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.341+0000 7f17c6a00640 1 -- 172.21.15.32:0/3114961105 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x563b58a03160 con 0x563b5867cda0 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.341+0000 7f17c6a00640 10 client.28219 handle_command_reply: tid=0 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.341+0000 7f17ee7c8740 1 client.28219 shutdown 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.341+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3114961105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 msgr2=0x563b58a9f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.341+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a9f470 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009450 tx=0x7f17cc009830 comp rx=0 tx=0).stop 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.341+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3114961105 shutdown_connections 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.341+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563b5867cda0 0x563b58b1a390 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.341+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b586e9070 0x563b58b10390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.342+0000 7f17c6a00640 0 client.28219 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:25:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.342+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b586c8390 0x563b58a9f470 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.342+0000 7f17ee7c8740 1 --2- 172.21.15.32:0/3114961105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563b586c01b0 0x563b58a9eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.342+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3114961105 >> 172.21.15.32:0/3114961105 conn(0x563b586dd460 msgr2=0x563b58a93c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.342+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3114961105 shutdown_connections 2024-08-31T07:25:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.342+0000 7f17ee7c8740 1 -- 172.21.15.32:0/3114961105 wait complete. 2024-08-31T07:25:57.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.342+0000 7f17ee7c8740 20 client.28219 trim_cache size 0 max 16384 2024-08-31T07:25:57.345 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:25:57.346 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.348 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:25:57.349 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.350 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.352 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.353 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:25:57.354 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:25:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:25:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:25:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:25:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:25:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:25:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:25:57.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:25:57.355 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:25:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.342+0000 7f17e6400640 1 -- 172.21.15.32:0/109991565 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c80b4d20 msgr2=0x7f17c80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:57.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.342+0000 7f17e6400640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c80b4d20 0x7f17c80b7230 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f17d8005fd0 tx=0x7f17d8009500 comp rx=0 tx=0).stop 2024-08-31T07:25:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.343+0000 7f17e6400640 1 -- 172.21.15.32:0/109991565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e0102200 msgr2=0x7f17e019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:25:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.343+0000 7f17e6400640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e0102200 0x7f17e019c2a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f17d4005ab0 tx=0x7f17d4003c10 comp rx=0 tx=0).stop 2024-08-31T07:25:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.343+0000 7f17e6400640 1 -- 172.21.15.32:0/109991565 shutdown_connections 2024-08-31T07:25:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.343+0000 7f17e6400640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c80b4d20 0x7f17c80b7230 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.343+0000 7f17e6400640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e010fdb0 0x7f17e01a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.343+0000 7f17e6400640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b50 0x7f17e019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.343+0000 7f17e6400640 1 --2- 172.21.15.32:0/109991565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e0102200 0x7f17e019c2a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:25:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.343+0000 7f17e6400640 1 -- 172.21.15.32:0/109991565 >> 172.21.15.32:0/109991565 conn(0x7f17e00fe060 msgr2=0x7f17e0100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:25:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.343+0000 7f17e6400640 1 -- 172.21.15.32:0/109991565 shutdown_connections 2024-08-31T07:25:57.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:25:57.343+0000 7f17e6400640 1 -- 172.21.15.32:0/109991565 wait complete. 2024-08-31T07:25:57.783 INFO:tasks.workunit.client.0.smithi032.stdout: 1 405825 2.78 MB/sec execute 586 sec latency 180.731 ms 2024-08-31T07:25:58.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 406092 2.78 MB/sec execute 587 sec latency 665.275 ms 2024-08-31T07:25:58.947 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:25:58 smithi032 ceph-mon[17252]: pgmap v778: 97 pgs: 97 active+clean; 342 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.8 MiB/s wr, 199 op/s 2024-08-31T07:25:58.950 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:25:58.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:25:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:25:58 smithi084 ceph-mon[22387]: pgmap v778: 97 pgs: 97 active+clean; 342 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.8 MiB/s wr, 199 op/s 2024-08-31T07:25:59.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:25:58 smithi073 ceph-mon[21995]: pgmap v778: 97 pgs: 97 active+clean; 342 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.8 MiB/s wr, 199 op/s 2024-08-31T07:25:59.332 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:25:59.785 INFO:tasks.workunit.client.0.smithi032.stdout: 1 406092 2.77 MB/sec execute 588 sec latency 1079.627 ms 2024-08-31T07:26:00.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.021+0000 7fbe75e00640 1 -- 172.21.15.32:0/1726523128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe70105960 msgr2=0x7fbe70105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:00.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.021+0000 7fbe75e00640 1 --2- 172.21.15.32:0/1726523128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe70105960 0x7fbe70105d70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009900 tx=0x7fbe6402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.021+0000 7fbe75e00640 1 -- 172.21.15.32:0/1726523128 shutdown_connections 2024-08-31T07:26:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.021+0000 7fbe75e00640 1 --2- 172.21.15.32:0/1726523128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7010bbf0 0x7fbe70112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.021+0000 7fbe75e00640 1 --2- 172.21.15.32:0/1726523128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe7010b2a0 0x7fbe7010b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.021+0000 7fbe75e00640 1 --2- 172.21.15.32:0/1726523128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe70105960 0x7fbe70105d70 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.021+0000 7fbe75e00640 1 -- 172.21.15.32:0/1726523128 >> 172.21.15.32:0/1726523128 conn(0x7fbe701013d0 msgr2=0x7fbe70103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.021+0000 7fbe75e00640 1 -- 172.21.15.32:0/1726523128 shutdown_connections 2024-08-31T07:26:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.021+0000 7fbe75e00640 1 -- 172.21.15.32:0/1726523128 wait complete. 2024-08-31T07:26:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.022+0000 7fbe75e00640 1 Processor -- start 2024-08-31T07:26:00.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.022+0000 7fbe75e00640 1 -- start start 2024-08-31T07:26:00.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.023+0000 7fbe75e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe70105960 0x7fbe7019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:00.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.023+0000 7fbe75e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7010b2a0 0x7fbe7019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:00.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.023+0000 7fbe75e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7010bbf0 0x7fbe701a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:00.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.023+0000 7fbe75e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701158b0 con 0x7fbe7010bbf0 2024-08-31T07:26:00.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.023+0000 7fbe75e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70115730 con 0x7fbe70105960 2024-08-31T07:26:00.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.023+0000 7fbe75e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70115a30 con 0x7fbe7010b2a0 2024-08-31T07:26:00.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.023+0000 7fbe74a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe70105960 0x7fbe7019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:00.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.023+0000 7fbe6fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7010b2a0 0x7fbe7019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:00.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.023+0000 7fbe74a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe70105960 0x7fbe7019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55868/0 (socket says 172.21.15.32:55868) 2024-08-31T07:26:00.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.023+0000 7fbe74a00640 1 -- 172.21.15.32:0/3980497037 learned_addr learned my addr 172.21.15.32:0/3980497037 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:00.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.024+0000 7fbe74a00640 1 -- 172.21.15.32:0/3980497037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7010b2a0 msgr2=0x7fbe7019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:00.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.024+0000 7fbe74a00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7010b2a0 0x7fbe7019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.024+0000 7fbe74a00640 1 -- 172.21.15.32:0/3980497037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7010bbf0 msgr2=0x7fbe701a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:00.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.024+0000 7fbe74a00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7010bbf0 0x7fbe701a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.024+0000 7fbe74a00640 1 -- 172.21.15.32:0/3980497037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe701a2210 con 0x7fbe70105960 2024-08-31T07:26:00.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.024+0000 7fbe74a00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe70105960 0x7fbe7019c2a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005940 tx=0x7fbe640040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:00.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.025+0000 7fbe6d600640 1 -- 172.21.15.32:0/3980497037 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6403f030 con 0x7fbe70105960 2024-08-31T07:26:00.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.025+0000 7fbe6d600640 1 -- 172.21.15.32:0/3980497037 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe640044c0 con 0x7fbe70105960 2024-08-31T07:26:00.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.025+0000 7fbe6d600640 1 -- 172.21.15.32:0/3980497037 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64035070 con 0x7fbe70105960 2024-08-31T07:26:00.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.025+0000 7fbe75e00640 1 -- 172.21.15.32:0/3980497037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe701a24a0 con 0x7fbe70105960 2024-08-31T07:26:00.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.025+0000 7fbe75e00640 1 -- 172.21.15.32:0/3980497037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe701a8c60 con 0x7fbe70105960 2024-08-31T07:26:00.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.026+0000 7fbe75e00640 1 -- 172.21.15.32:0/3980497037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe70106be0 con 0x7fbe70105960 2024-08-31T07:26:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.029+0000 7fbe6d600640 1 -- 172.21.15.32:0/3980497037 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbe64042030 con 0x7fbe70105960 2024-08-31T07:26:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.030+0000 7fbe6d600640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbe580b4d20 0x7fbe580b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.031+0000 7fbe6d600640 1 -- 172.21.15.32:0/3980497037 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7fbe64105ae0 con 0x7fbe70105960 2024-08-31T07:26:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.031+0000 7fbe6fe00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbe580b4d20 0x7fbe580b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.032+0000 7fbe6fe00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbe580b4d20 0x7fbe580b7230 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe7010ae00 tx=0x7fbe5c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:00.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.032+0000 7fbe6d600640 1 -- 172.21.15.32:0/3980497037 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fbe640cc8c0 con 0x7fbe70105960 2024-08-31T07:26:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.319+0000 7fbe75e00640 1 -- 172.21.15.32:0/3980497037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe70105d70 con 0x7fbe70105960 2024-08-31T07:26:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.320+0000 7fbe6d600640 1 -- 172.21.15.32:0/3980497037 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11970 (secure 0 0 0) 0x7fbe640cc1c0 con 0x7fbe70105960 2024-08-31T07:26:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:26:00.321 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"btime":"2024-08-31T07:25:04:192234+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:25:03.382644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:26:00.322 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2024-08-31T07:26:00.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.323+0000 7fbe75e00640 1 -- 172.21.15.32:0/3980497037 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbe580b4d20 msgr2=0x7fbe580b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:00.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.323+0000 7fbe75e00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbe580b4d20 0x7fbe580b7230 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe7010ae00 tx=0x7fbe5c005f90 comp rx=0 tx=0).stop 2024-08-31T07:26:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.323+0000 7fbe75e00640 1 -- 172.21.15.32:0/3980497037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe70105960 msgr2=0x7fbe7019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.323+0000 7fbe75e00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe70105960 0x7fbe7019c2a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005940 tx=0x7fbe640040c0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.324+0000 7fbe75e00640 1 -- 172.21.15.32:0/3980497037 shutdown_connections 2024-08-31T07:26:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.324+0000 7fbe75e00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbe580b4d20 0x7fbe580b7230 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.324+0000 7fbe75e00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7010bbf0 0x7fbe701a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.324+0000 7fbe75e00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe7010b2a0 0x7fbe7019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.324+0000 7fbe75e00640 1 --2- 172.21.15.32:0/3980497037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe70105960 0x7fbe7019c2a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.324+0000 7fbe75e00640 1 -- 172.21.15.32:0/3980497037 >> 172.21.15.32:0/3980497037 conn(0x7fbe701013d0 msgr2=0x7fbe701037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.324+0000 7fbe75e00640 1 -- 172.21.15.32:0/3980497037 shutdown_connections 2024-08-31T07:26:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:00.324+0000 7fbe75e00640 1 -- 172.21.15.32:0/3980497037 wait complete. 2024-08-31T07:26:00.407 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.7 2024-08-31T07:26:00.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:26:00.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 407277 2.78 MB/sec execute 589 sec latency 1271.118 ms 2024-08-31T07:26:00.993 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:00 smithi084 ceph-mon[22387]: pgmap v779: 97 pgs: 97 active+clean; 346 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-31T07:26:00.993 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:00 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3980497037' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:26:01.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:00 smithi032 ceph-mon[17252]: pgmap v779: 97 pgs: 97 active+clean; 346 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-31T07:26:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:00 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3980497037' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:26:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:00 smithi073 ceph-mon[21995]: pgmap v779: 97 pgs: 97 active+clean; 346 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-31T07:26:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:00 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3980497037' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:26:01.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:26:01.619 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:01.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 407277 2.77 MB/sec execute 590 sec latency 1084.541 ms 2024-08-31T07:26:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.311+0000 7f8149c00640 1 -- 172.21.15.32:0/3836308206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81441050c0 msgr2=0x7f814410d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.311+0000 7f8149c00640 1 --2- 172.21.15.32:0/3836308206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81441050c0 0x7f814410d610 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f812c009900 tx=0x7f812c02a410 comp rx=0 tx=0).stop 2024-08-31T07:26:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.311+0000 7f8149c00640 1 -- 172.21.15.32:0/3836308206 shutdown_connections 2024-08-31T07:26:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.311+0000 7f8149c00640 1 --2- 172.21.15.32:0/3836308206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f814410dbd0 0x7f81441100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.311+0000 7f8149c00640 1 --2- 172.21.15.32:0/3836308206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81441050c0 0x7f814410d610 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.311+0000 7f8149c00640 1 --2- 172.21.15.32:0/3836308206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8144104770 0x7f8144104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.311+0000 7f8149c00640 1 -- 172.21.15.32:0/3836308206 >> 172.21.15.32:0/3836308206 conn(0x7f81440794e0 msgr2=0x7f81440798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.311+0000 7f8149c00640 1 -- 172.21.15.32:0/3836308206 shutdown_connections 2024-08-31T07:26:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.311+0000 7f8149c00640 1 -- 172.21.15.32:0/3836308206 wait complete. 2024-08-31T07:26:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.313+0000 7f8149c00640 1 Processor -- start 2024-08-31T07:26:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.313+0000 7f8149c00640 1 -- start start 2024-08-31T07:26:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.313+0000 7f8149c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8144104770 0x7f814419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.313+0000 7f8149c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81441050c0 0x7f814419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.313+0000 7f8149c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f814410dbd0 0x7f814419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.313+0000 7f8149c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8144113810 con 0x7f814410dbd0 2024-08-31T07:26:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.313+0000 7f8149c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8144113690 con 0x7f81441050c0 2024-08-31T07:26:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.313+0000 7f8149c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8144113990 con 0x7f8144104770 2024-08-31T07:26:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.314+0000 7f813be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81441050c0 0x7f814419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.314+0000 7f8142a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8144104770 0x7f814419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.314+0000 7f813be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81441050c0 0x7f814419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55896/0 (socket says 172.21.15.32:55896) 2024-08-31T07:26:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.314+0000 7f813be00640 1 -- 172.21.15.32:0/978405129 learned_addr learned my addr 172.21.15.32:0/978405129 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.314+0000 7f8143400640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f814410dbd0 0x7f814419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.314+0000 7f8142a00640 1 -- 172.21.15.32:0/978405129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81441050c0 msgr2=0x7f814419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.314+0000 7f8142a00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81441050c0 0x7f814419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.314+0000 7f8142a00640 1 -- 172.21.15.32:0/978405129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f814410dbd0 msgr2=0x7f814419fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.314+0000 7f8142a00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f814410dbd0 0x7f814419fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.314+0000 7f8142a00640 1 -- 172.21.15.32:0/978405129 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81441a0210 con 0x7f8144104770 2024-08-31T07:26:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.315+0000 7f8142a00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8144104770 0x7f814419a2a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f813400e9c0 tx=0x7f813400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.315+0000 7f813b400640 1 -- 172.21.15.32:0/978405129 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813400ccc0 con 0x7f8144104770 2024-08-31T07:26:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.315+0000 7f813b400640 1 -- 172.21.15.32:0/978405129 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8134003610 con 0x7f8144104770 2024-08-31T07:26:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.315+0000 7f8149c00640 1 -- 172.21.15.32:0/978405129 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81441a0500 con 0x7f8144104770 2024-08-31T07:26:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.315+0000 7f813b400640 1 -- 172.21.15.32:0/978405129 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813400ce60 con 0x7f8144104770 2024-08-31T07:26:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.315+0000 7f8149c00640 1 -- 172.21.15.32:0/978405129 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81441a6c60 con 0x7f8144104770 2024-08-31T07:26:02.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.316+0000 7f8149c00640 1 -- 172.21.15.32:0/978405129 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81200052c0 con 0x7f8144104770 2024-08-31T07:26:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.320+0000 7f813b400640 1 -- 172.21.15.32:0/978405129 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8134004000 con 0x7f8144104770 2024-08-31T07:26:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.322+0000 7f813b400640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f81280b4ca0 0x7f81280b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.322+0000 7f813b400640 1 -- 172.21.15.32:0/978405129 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f81340e5720 con 0x7f8144104770 2024-08-31T07:26:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.323+0000 7f813be00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f81280b4ca0 0x7f81280b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.323+0000 7f813b400640 1 -- 172.21.15.32:0/978405129 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f81340ac500 con 0x7f8144104770 2024-08-31T07:26:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.323+0000 7f813be00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f81280b4ca0 0x7f81280b71b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f81441021a0 tx=0x7f812c009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:02.351 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:02.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.579+0000 7f8149c00640 1 -- 172.21.15.32:0/978405129 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81200055b0 con 0x7f8144104770 2024-08-31T07:26:02.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.580+0000 7f813b400640 1 -- 172.21.15.32:0/978405129 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11970 (secure 0 0 0) 0x7f81340042f0 con 0x7f8144104770 2024-08-31T07:26:02.581 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:26:02.581 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"btime":"2024-08-31T07:25:04:192234+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:25:03.382644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:26:02.581 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2024-08-31T07:26:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.582+0000 7f8149c00640 1 -- 172.21.15.32:0/978405129 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f81280b4ca0 msgr2=0x7f81280b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.582+0000 7f8149c00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f81280b4ca0 0x7f81280b71b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f81441021a0 tx=0x7f812c009560 comp rx=0 tx=0).stop 2024-08-31T07:26:02.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.582+0000 7f8149c00640 1 -- 172.21.15.32:0/978405129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8144104770 msgr2=0x7f814419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.582+0000 7f8149c00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8144104770 0x7f814419a2a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f813400e9c0 tx=0x7f813400ee80 comp rx=0 tx=0).stop 2024-08-31T07:26:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.583+0000 7f8149c00640 1 -- 172.21.15.32:0/978405129 shutdown_connections 2024-08-31T07:26:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.583+0000 7f8149c00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f81280b4ca0 0x7f81280b71b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.583+0000 7f8149c00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f814410dbd0 0x7f814419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.583+0000 7f8149c00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81441050c0 0x7f814419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.583+0000 7f8149c00640 1 --2- 172.21.15.32:0/978405129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8144104770 0x7f814419a2a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.583+0000 7f8149c00640 1 -- 172.21.15.32:0/978405129 >> 172.21.15.32:0/978405129 conn(0x7f81440794e0 msgr2=0x7f8144100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.583+0000 7f8149c00640 1 -- 172.21.15.32:0/978405129 shutdown_connections 2024-08-31T07:26:02.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:02.583+0000 7f8149c00640 1 -- 172.21.15.32:0/978405129 wait complete. 2024-08-31T07:26:02.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 407277 2.77 MB/sec execute 591 sec latency 2084.587 ms 2024-08-31T07:26:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:02 smithi032 ceph-mon[17252]: pgmap v780: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-08-31T07:26:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:02 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/978405129' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:26:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:02 smithi084 ceph-mon[22387]: pgmap v780: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-08-31T07:26:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:02 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/978405129' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:26:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:02 smithi073 ceph-mon[21995]: pgmap v780: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-08-31T07:26:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:02 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/978405129' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:26:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.251+0000 7f71cbe00640 1 -- 172.21.15.32:0/249046664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc109090 msgr2=0x7f71cc1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.251+0000 7f71cbe00640 1 --2- 172.21.15.32:0/249046664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc109090 0x7f71cc1094a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009900 tx=0x7f71b802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.251+0000 7f71cbe00640 1 -- 172.21.15.32:0/249046664 shutdown_connections 2024-08-31T07:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.251+0000 7f71cbe00640 1 --2- 172.21.15.32:0/249046664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1099e0 0x7f71cc110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.251+0000 7f71cbe00640 1 --2- 172.21.15.32:0/249046664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc109090 0x7f71cc1094a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.251+0000 7f71cbe00640 1 --2- 172.21.15.32:0/249046664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71cc103750 0x7f71cc103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.251+0000 7f71cbe00640 1 -- 172.21.15.32:0/249046664 >> 172.21.15.32:0/249046664 conn(0x7f71cc0ff1e0 msgr2=0x7f71cc101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.251+0000 7f71cbe00640 1 -- 172.21.15.32:0/249046664 shutdown_connections 2024-08-31T07:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.251+0000 7f71cbe00640 1 -- 172.21.15.32:0/249046664 wait complete. 2024-08-31T07:26:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.252+0000 7f71cbe00640 1 Processor -- start 2024-08-31T07:26:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71cbe00640 1 -- start start 2024-08-31T07:26:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71cbe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71cc103750 0x7f71cc19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71cbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc109090 0x7f71cc19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71cbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1099e0 0x7f71cc1a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71cbe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc075f20 con 0x7f71cc109090 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71cbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc075da0 con 0x7f71cc1099e0 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71cbe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc0760a0 con 0x7f71cc103750 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71ca000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc109090 0x7f71cc19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71ca000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc109090 0x7f71cc19c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54276/0 (socket says 172.21.15.32:54276) 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71ca000640 1 -- 172.21.15.32:0/2657006272 learned_addr learned my addr 172.21.15.32:0/2657006272 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71caa00640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71cc103750 0x7f71cc19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.253+0000 7f71cb400640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1099e0 0x7f71cc1a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.254+0000 7f71ca000640 1 -- 172.21.15.32:0/2657006272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71cc103750 msgr2=0x7f71cc19c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.254+0000 7f71ca000640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71cc103750 0x7f71cc19c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.254+0000 7f71ca000640 1 -- 172.21.15.32:0/2657006272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1099e0 msgr2=0x7f71cc1a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.254+0000 7f71ca000640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1099e0 0x7f71cc1a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.254+0000 7f71ca000640 1 -- 172.21.15.32:0/2657006272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc1a2300 con 0x7f71cc109090 2024-08-31T07:26:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.254+0000 7f71caa00640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71cc103750 0x7f71cc19c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.254+0000 7f71ca000640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc109090 0x7f71cc19c930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f71b8006010 tx=0x7f71b8004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.254+0000 7f71c3400640 1 -- 172.21.15.32:0/2657006272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b803f030 con 0x7f71cc109090 2024-08-31T07:26:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.255+0000 7f71c3400640 1 -- 172.21.15.32:0/2657006272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b8003aa0 con 0x7f71cc109090 2024-08-31T07:26:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.255+0000 7f71cbe00640 1 -- 172.21.15.32:0/2657006272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc1a2590 con 0x7f71cc109090 2024-08-31T07:26:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.255+0000 7f71c3400640 1 -- 172.21.15.32:0/2657006272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8035070 con 0x7f71cc109090 2024-08-31T07:26:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.255+0000 7f71cbe00640 1 -- 172.21.15.32:0/2657006272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc1a8d50 con 0x7f71cc109090 2024-08-31T07:26:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.258+0000 7f71d87bd740 1 Processor -- start 2024-08-31T07:26:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.258+0000 7f71d87bd740 1 -- start start 2024-08-31T07:26:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.258+0000 7f71c3400640 1 -- 172.21.15.32:0/2657006272 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f71b8042030 con 0x7f71cc109090 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad4ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 0x557edad4b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71d87bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557edacc17b0 con 0x557eda9a5070 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71d87bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557edacb0d40 con 0x557eda984390 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71d87bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557edacb4cf0 con 0x557eda97c1b0 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71cb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad4ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71caa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71ca000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 0x557edad4b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71caa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:32826/0 (socket says 172.21.15.32:32826) 2024-08-31T07:26:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71caa00640 1 -- 172.21.15.32:0/3519347105 learned_addr learned my addr 172.21.15.32:0/3519347105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71caa00640 1 -- 172.21.15.32:0/3519347105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 msgr2=0x557edad4b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71caa00640 1 --2- 172.21.15.32:0/3519347105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 0x557edad4b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71caa00640 1 -- 172.21.15.32:0/3519347105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 msgr2=0x557edad4ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71caa00640 1 --2- 172.21.15.32:0/3519347105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad4ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71caa00640 1 -- 172.21.15.32:0/3519347105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557edaa10650 con 0x557eda97c1b0 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.259+0000 7f71ca000640 1 --2- 172.21.15.32:0/3519347105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 0x557edad4b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.260+0000 7f71caa00640 1 --2- 172.21.15.32:0/3519347105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4b0a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0115c0 tx=0x7f71bc011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc366f7a42cbbdc1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.260+0000 7f71b7e00640 1 -- 172.21.15.32:0/3519347105 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00f640 con 0x557eda97c1b0 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.260+0000 7f71b7e00640 1 -- 172.21.15.32:0/3519347105 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc00fc90 con 0x557eda97c1b0 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.260+0000 7f71b7e00640 1 -- 172.21.15.32:0/3519347105 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc019c80 con 0x557eda97c1b0 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71c3400640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71b00b4ca0 0x7f71b00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71c3400640 1 -- 172.21.15.32:0/2657006272 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f71b8105970 con 0x7f71cc109090 2024-08-31T07:26:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71cb400640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71b00b4ca0 0x7f71b00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71d87bd740 1 -- 172.21.15.32:0/3519347105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 msgr2=0x557edad4b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71d87bd740 1 --2- 172.21.15.32:0/3519347105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4b0a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0115c0 tx=0x7f71bc011a80 comp rx=0 tx=0).stop 2024-08-31T07:26:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71d87bd740 1 -- 172.21.15.32:0/3519347105 shutdown_connections 2024-08-31T07:26:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71d87bd740 1 --2- 172.21.15.32:0/3519347105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 0x557edad4b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71d87bd740 1 --2- 172.21.15.32:0/3519347105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4b0a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71d87bd740 1 --2- 172.21.15.32:0/3519347105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad4ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71d87bd740 1 -- 172.21.15.32:0/3519347105 >> 172.21.15.32:0/3519347105 conn(0x557eda999460 msgr2=0x557edad48a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.261+0000 7f71d87bd740 1 -- 172.21.15.32:0/3519347105 shutdown_connections 2024-08-31T07:26:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.262+0000 7f71d87bd740 1 -- 172.21.15.32:0/3519347105 wait complete. 2024-08-31T07:26:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.262+0000 7f71cb400640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71b00b4ca0 0x7f71b00b71b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f71c4005fd0 tx=0x7f71c40096a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.262+0000 7f71d87bd740 1 -- start start 2024-08-31T07:26:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.262+0000 7f71d87bd740 1 Processor -- start 2024-08-31T07:26:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda984390 0x557edad56c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad62690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71d87bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557edab13700 con 0x557eda9a5070 2024-08-31T07:26:03.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71d87bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557edac37cf0 con 0x557eda97c1b0 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71d87bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557edad59340 con 0x557eda984390 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71d87bd740 10 client.0 _ll_register_callbacks cb 0x557edac5d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71caa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71caa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55944/0 (socket says 172.21.15.32:55944) 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71caa00640 1 -- 172.21.15.32:0/1981840649 learned_addr learned my addr 172.21.15.32:0/1981840649 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71ca000640 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda984390 0x557edad56c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.263+0000 7f71cb400640 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad62690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.264+0000 7f71caa00640 1 -- 172.21.15.32:0/1981840649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda984390 msgr2=0x557edad56c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.264+0000 7f71caa00640 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda984390 0x557edad56c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.264+0000 7f71caa00640 1 -- 172.21.15.32:0/1981840649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 msgr2=0x557edad62690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.264+0000 7f71caa00640 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad62690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.264+0000 7f71caa00640 1 -- 172.21.15.32:0/1981840649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557edac788b0 con 0x557eda97c1b0 2024-08-31T07:26:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.265+0000 7f71caa00640 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad566f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f71bc005750 tx=0x7f71bc005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9b386030b7b118a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.265+0000 7f71b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:26:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.265+0000 7f71b4c00640 1 -- 172.21.15.32:0/1981840649 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc026020 con 0x557eda97c1b0 2024-08-31T07:26:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.265+0000 7f71b4c00640 1 -- 172.21.15.32:0/1981840649 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f71bc019c30 con 0x557eda97c1b0 2024-08-31T07:26:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.265+0000 7f71d87bd740 1 -- 172.21.15.32:0/1981840649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71bc00fc90 con 0x557eda97c1b0 2024-08-31T07:26:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.265+0000 7f71b4c00640 1 -- 172.21.15.32:0/1981840649 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f71bc031050 con 0x557eda97c1b0 2024-08-31T07:26:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.266+0000 7f71d87bd740 10 client.28241 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.266+0000 7f71d87bd740 1 -- 172.21.15.32:0/1981840649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557edad65330 con 0x557eda97c1b0 2024-08-31T07:26:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.266+0000 7f71b4c00640 1 -- 172.21.15.32:0/1981840649 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc00fc90 con 0x557eda97c1b0 2024-08-31T07:26:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.266+0000 7f71b4c00640 1 -- 172.21.15.32:0/1981840649 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f71bc0035f0 con 0x557eda97c1b0 2024-08-31T07:26:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.266+0000 7f71d87bd740 10 client.28241 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.266+0000 7f71d87bd740 10 client.28241 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:26:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.266+0000 7f71d87bd740 20 client.28241 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.266+0000 7f71d87bd740 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557eda9639f0 0x557edad687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.266+0000 7f71d87bd740 4 client.28241 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.266+0000 7f71d87bd740 1 -- 172.21.15.32:0/1981840649 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557edacc3700 con 0x557eda9639f0 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.267+0000 7f71ca000640 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557eda9639f0 0x557edad687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.267+0000 7f71ca000640 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557eda9639f0 0x557edad687a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=85edbcf2de6b0a84 server_cookie=211701fd56abaf5f in_seq=0 out_seq=0 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.267+0000 7f71b4c00640 10 client.28241 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.268+0000 7f71b4c00640 1 -- 172.21.15.32:0/1981840649 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557edacc3700 con 0x557eda9639f0 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.268+0000 7f71b4c00640 10 client.28241 handle_command_reply: tid=0 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 client.28241 shutdown 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 -- 172.21.15.32:0/1981840649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 msgr2=0x557edad566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad566f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f71bc005750 tx=0x7f71bc005780 comp rx=0 tx=0).stop 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 -- 172.21.15.32:0/1981840649 shutdown_connections 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557eda9639f0 0x557edad687a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad62690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda984390 0x557edad56c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 --2- 172.21.15.32:0/1981840649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad566f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71b4c00640 0 client.28241 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:26:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 -- 172.21.15.32:0/1981840649 >> 172.21.15.32:0/1981840649 conn(0x557eda999460 msgr2=0x557edad4dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 -- 172.21.15.32:0/1981840649 shutdown_connections 2024-08-31T07:26:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 1 -- 172.21.15.32:0/1981840649 wait complete. 2024-08-31T07:26:03.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.269+0000 7f71d87bd740 20 client.28241 trim_cache size 0 max 16384 2024-08-31T07:26:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.283+0000 7f71d87bd740 1 Processor -- start 2024-08-31T07:26:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.283+0000 7f71d87bd740 1 -- start start 2024-08-31T07:26:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 0x557edad48000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad48540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71d87bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1b1410 con 0x557eda9a5070 2024-08-31T07:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71d87bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1b3070 con 0x557eda984390 2024-08-31T07:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71d87bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1b1ae0 con 0x557eda97c1b0 2024-08-31T07:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71caa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71cb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad48540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71caa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:32840/0 (socket says 172.21.15.32:32840) 2024-08-31T07:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71caa00640 1 -- 172.21.15.32:0/1846927835 learned_addr learned my addr 172.21.15.32:0/1846927835 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71ca000640 1 --2- 172.21.15.32:0/1846927835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 0x557edad48000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71caa00640 1 -- 172.21.15.32:0/1846927835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 msgr2=0x557edad48000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71caa00640 1 --2- 172.21.15.32:0/1846927835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 0x557edad48000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71caa00640 1 -- 172.21.15.32:0/1846927835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 msgr2=0x557edad48540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71caa00640 1 --2- 172.21.15.32:0/1846927835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad48540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.284+0000 7f71caa00640 1 -- 172.21.15.32:0/1846927835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557edad63070 con 0x557eda97c1b0 2024-08-31T07:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.285+0000 7f71caa00640 1 --2- 172.21.15.32:0/1846927835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4e370 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0115c0 tx=0x7f71bc003780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535e29b94895a72 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.285+0000 7f71b7400640 1 -- 172.21.15.32:0/1846927835 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc026040 con 0x557eda97c1b0 2024-08-31T07:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.285+0000 7f71b7400640 1 -- 172.21.15.32:0/1846927835 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc00fe30 con 0x557eda97c1b0 2024-08-31T07:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.285+0000 7f71b7400640 1 -- 172.21.15.32:0/1846927835 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc022480 con 0x557eda97c1b0 2024-08-31T07:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.286+0000 7f71d87bd740 1 -- 172.21.15.32:0/1846927835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 msgr2=0x557edad4e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.286+0000 7f71d87bd740 1 --2- 172.21.15.32:0/1846927835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4e370 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0115c0 tx=0x7f71bc003780 comp rx=0 tx=0).stop 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.286+0000 7f71d87bd740 1 -- 172.21.15.32:0/1846927835 shutdown_connections 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.286+0000 7f71d87bd740 1 --2- 172.21.15.32:0/1846927835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edad48540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.286+0000 7f71d87bd740 1 --2- 172.21.15.32:0/1846927835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda984390 0x557edad48000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.286+0000 7f71d87bd740 1 --2- 172.21.15.32:0/1846927835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda97c1b0 0x557edad4e370 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.286+0000 7f71d87bd740 1 -- 172.21.15.32:0/1846927835 >> 172.21.15.32:0/1846927835 conn(0x557eda999460 msgr2=0x557edad4dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.286+0000 7f71d87bd740 1 -- 172.21.15.32:0/1846927835 shutdown_connections 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.286+0000 7f71d87bd740 1 -- 172.21.15.32:0/1846927835 wait complete. 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.287+0000 7f71d87bd740 1 -- start start 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.287+0000 7f71d87bd740 1 Processor -- start 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad5ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda984390 0x557edad5b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71d87bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edadd2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71d87bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1b28e0 con 0x557eda9a5070 2024-08-31T07:26:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71d87bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1b21b0 con 0x557eda97c1b0 2024-08-31T07:26:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71d87bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1af410 con 0x557eda984390 2024-08-31T07:26:03.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71d87bd740 10 client.0 _ll_register_callbacks cb 0x7f71bc011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71cb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edadd2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71caa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad5ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71cb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edadd2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54302/0 (socket says 172.21.15.32:54302) 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71cb400640 1 -- 172.21.15.32:0/601503613 learned_addr learned my addr 172.21.15.32:0/601503613 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71ca000640 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda984390 0x557edad5b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71caa00640 1 -- 172.21.15.32:0/601503613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda984390 msgr2=0x557edad5b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71caa00640 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda984390 0x557edad5b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71caa00640 1 -- 172.21.15.32:0/601503613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 msgr2=0x557edadd2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71caa00640 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edadd2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.288+0000 7f71caa00640 1 -- 172.21.15.32:0/601503613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557edadd26f0 con 0x557eda97c1b0 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.289+0000 7f71caa00640 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad5ad60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f71bc003bb0 tx=0x7f71bc0055d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b183d9d2c896b345 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.289+0000 7f71b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.289+0000 7f71b4c00640 1 -- 172.21.15.32:0/601503613 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00fc30 con 0x557eda97c1b0 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.289+0000 7f71b4c00640 1 -- 172.21.15.32:0/601503613 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f71bc00f850 con 0x557eda97c1b0 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.289+0000 7f71d87bd740 1 -- 172.21.15.32:0/601503613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71bc00fe30 con 0x557eda97c1b0 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.289+0000 7f71b4c00640 1 -- 172.21.15.32:0/601503613 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f71bc022480 con 0x557eda97c1b0 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.289+0000 7f71d87bd740 10 client.28247 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.289+0000 7f71d87bd740 1 -- 172.21.15.32:0/601503613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557edadd3970 con 0x557eda97c1b0 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.289+0000 7f71b4c00640 1 -- 172.21.15.32:0/601503613 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc00fe30 con 0x557eda97c1b0 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.290+0000 7f71b4c00640 1 -- 172.21.15.32:0/601503613 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f71bc022ad0 con 0x557eda97c1b0 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.290+0000 7f71d87bd740 10 client.28247 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.290+0000 7f71d87bd740 10 client.28247 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.290+0000 7f71d87bd740 20 client.28247 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.290+0000 7f71d87bd740 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557eda938da0 0x557edadd6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:03.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.290+0000 7f71d87bd740 4 client.28247 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:26:03.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.290+0000 7f71d87bd740 1 -- 172.21.15.32:0/601503613 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557edacbf1a0 con 0x557eda938da0 2024-08-31T07:26:03.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.290+0000 7f71ca000640 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557eda938da0 0x557edadd6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:03.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71ca000640 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557eda938da0 0x557edadd6010 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3dae8e702784d73 server_cookie=c76a1559ed9a970e in_seq=0 out_seq=0 2024-08-31T07:26:03.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71b4c00640 10 client.28247 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:26:03.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71b4c00640 1 -- 172.21.15.32:0/601503613 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557edacbf1a0 con 0x557eda938da0 2024-08-31T07:26:03.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71b4c00640 10 client.28247 handle_command_reply: tid=0 2024-08-31T07:26:03.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71d87bd740 1 client.28247 shutdown 2024-08-31T07:26:03.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71d87bd740 1 -- 172.21.15.32:0/601503613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 msgr2=0x557edad5ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:03.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71d87bd740 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad5ad60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f71bc003bb0 tx=0x7f71bc0055d0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71d87bd740 1 -- 172.21.15.32:0/601503613 shutdown_connections 2024-08-31T07:26:03.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71d87bd740 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x557eda938da0 0x557edadd6010 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71d87bd740 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557eda9a5070 0x557edadd2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71d87bd740 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eda984390 0x557edad5b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71d87bd740 1 --2- 172.21.15.32:0/601503613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557eda97c1b0 0x557edad5ad60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.291+0000 7f71b4c00640 0 client.28247 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:26:03.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.292+0000 7f71d87bd740 1 -- 172.21.15.32:0/601503613 >> 172.21.15.32:0/601503613 conn(0x557eda999460 msgr2=0x557edad4dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:03.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.292+0000 7f71d87bd740 1 -- 172.21.15.32:0/601503613 shutdown_connections 2024-08-31T07:26:03.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.292+0000 7f71d87bd740 1 -- 172.21.15.32:0/601503613 wait complete. 2024-08-31T07:26:03.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.292+0000 7f71d87bd740 20 client.28247 trim_cache size 0 max 16384 2024-08-31T07:26:03.297 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:26:03.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.292+0000 7f71cbe00640 1 -- 172.21.15.32:0/2657006272 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71b00b4ca0 msgr2=0x7f71b00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:03.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.292+0000 7f71cbe00640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71b00b4ca0 0x7f71b00b71b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f71c4005fd0 tx=0x7f71c40096a0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.292+0000 7f71cbe00640 1 -- 172.21.15.32:0/2657006272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc109090 msgr2=0x7f71cc19c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:03.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.292+0000 7f71cbe00640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc109090 0x7f71cc19c930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f71b8006010 tx=0x7f71b8004060 comp rx=0 tx=0).stop 2024-08-31T07:26:03.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.293+0000 7f71cbe00640 1 -- 172.21.15.32:0/2657006272 shutdown_connections 2024-08-31T07:26:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.293+0000 7f71cbe00640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71b00b4ca0 0x7f71b00b71b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.293+0000 7f71cbe00640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71cc1099e0 0x7f71cc1a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.293+0000 7f71cbe00640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71cc109090 0x7f71cc19c930 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.293+0000 7f71cbe00640 1 --2- 172.21.15.32:0/2657006272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71cc103750 0x7f71cc19c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.293+0000 7f71cbe00640 1 -- 172.21.15.32:0/2657006272 >> 172.21.15.32:0/2657006272 conn(0x7f71cc0ff1e0 msgr2=0x7f71cc105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.293+0000 7f71cbe00640 1 -- 172.21.15.32:0/2657006272 shutdown_connections 2024-08-31T07:26:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:03.293+0000 7f71cbe00640 1 -- 172.21.15.32:0/2657006272 wait complete. 2024-08-31T07:26:03.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 407277 2.76 MB/sec execute 592 sec latency 3084.665 ms 2024-08-31T07:26:03.853 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 134.7 --await-for 45.0' 2024-08-31T07:26:03.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 134.7 --await-for 45.0 2024-08-31T07:26:04.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 407277 2.76 MB/sec execute 593 sec latency 4084.785 ms 2024-08-31T07:26:04.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:26:04.983 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:04 smithi032 ceph-mon[17252]: pgmap v781: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-08-31T07:26:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:04 smithi084 ceph-mon[22387]: pgmap v781: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-08-31T07:26:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:04 smithi073 ceph-mon[21995]: pgmap v781: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-08-31T07:26:05.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 408224 2.77 MB/sec execute 594 sec latency 4430.722 ms 2024-08-31T07:26:05.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.855+0000 7f8ffee00640 1 -- 172.21.15.32:0/2563206599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ff80754f0 msgr2=0x7f8ff8075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:05.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.855+0000 7f8ffee00640 1 --2- 172.21.15.32:0/2563206599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ff80754f0 0x7f8ff8075900 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009900 tx=0x7f8fec02a410 comp rx=0 tx=0).stop 2024-08-31T07:26:05.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.855+0000 7f8ffee00640 1 -- 172.21.15.32:0/2563206599 shutdown_connections 2024-08-31T07:26:05.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.855+0000 7f8ffee00640 1 --2- 172.21.15.32:0/2563206599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff810fd50 0x7f8ff8112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.855+0000 7f8ffee00640 1 --2- 172.21.15.32:0/2563206599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ff8075e40 0x7f8ff810f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.855+0000 7f8ffee00640 1 --2- 172.21.15.32:0/2563206599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ff80754f0 0x7f8ff8075900 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.855+0000 7f8ffee00640 1 -- 172.21.15.32:0/2563206599 >> 172.21.15.32:0/2563206599 conn(0x7f8ff80ff220 msgr2=0x7f8ff8101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:05.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.855+0000 7f8ffee00640 1 -- 172.21.15.32:0/2563206599 shutdown_connections 2024-08-31T07:26:05.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.855+0000 7f8ffee00640 1 -- 172.21.15.32:0/2563206599 wait complete. 2024-08-31T07:26:05.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.856+0000 7f8ffee00640 1 Processor -- start 2024-08-31T07:26:05.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.857+0000 7f8ffee00640 1 -- start start 2024-08-31T07:26:05.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.857+0000 7f8ffee00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ff80754f0 0x7f8ff819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.857+0000 7f8ffee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff8075e40 0x7f8ff819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.857+0000 7f8ffee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ff810fd50 0x7f8ff81a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.857+0000 7f8ffee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81158c0 con 0x7f8ff8075e40 2024-08-31T07:26:05.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.857+0000 7f8ffee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8115740 con 0x7f8ff810fd50 2024-08-31T07:26:05.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.857+0000 7f8ffee00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8115a40 con 0x7f8ff80754f0 2024-08-31T07:26:05.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffda00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ff80754f0 0x7f8ff819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffe400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ff810fd50 0x7f8ff81a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffda00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ff80754f0 0x7f8ff819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38270/0 (socket says 172.21.15.32:38270) 2024-08-31T07:26:05.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffda00640 1 -- 172.21.15.32:0/1194426982 learned_addr learned my addr 172.21.15.32:0/1194426982 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:05.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffd000640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff8075e40 0x7f8ff819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffda00640 1 -- 172.21.15.32:0/1194426982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ff810fd50 msgr2=0x7f8ff81a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:05.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffda00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ff810fd50 0x7f8ff81a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffda00640 1 -- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff8075e40 msgr2=0x7f8ff819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:05.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffda00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff8075e40 0x7f8ff819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffda00640 1 -- 172.21.15.32:0/1194426982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff81a2220 con 0x7f8ff80754f0 2024-08-31T07:26:05.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffe400640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ff810fd50 0x7f8ff81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:05.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffd000640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff8075e40 0x7f8ff819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:26:05.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.858+0000 7f8ffda00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ff80754f0 0x7f8ff819c2b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8fec005940 tx=0x7f8fec0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:05.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.859+0000 7f8ff6000640 1 -- 172.21.15.32:0/1194426982 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec03f030 con 0x7f8ff80754f0 2024-08-31T07:26:05.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.859+0000 7f8ffee00640 1 -- 172.21.15.32:0/1194426982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff81a24b0 con 0x7f8ff80754f0 2024-08-31T07:26:05.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.859+0000 7f8ff6000640 1 -- 172.21.15.32:0/1194426982 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fec0044c0 con 0x7f8ff80754f0 2024-08-31T07:26:05.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.859+0000 7f8ff6000640 1 -- 172.21.15.32:0/1194426982 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec035070 con 0x7f8ff80754f0 2024-08-31T07:26:05.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.859+0000 7f8ffee00640 1 -- 172.21.15.32:0/1194426982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff81a8c70 con 0x7f8ff80754f0 2024-08-31T07:26:05.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.864+0000 7f9007014740 1 Processor -- start 2024-08-31T07:26:05.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.864+0000 7f9007014740 1 -- start start 2024-08-31T07:26:05.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.864+0000 7f9007014740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a0f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.864+0000 7f9007014740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 0x559fa1a0f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.864+0000 7f9007014740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a0fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f9007014740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559fa1984530 con 0x559fa163f980 2024-08-31T07:26:05.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f9007014740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559fa1973ac0 con 0x559fa1647b60 2024-08-31T07:26:05.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f9007014740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559fa1977a70 con 0x559fa1668840 2024-08-31T07:26:05.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f8ffe400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a0f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f8ffe400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a0f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38284/0 (socket says 172.21.15.32:38284) 2024-08-31T07:26:05.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f8ffe400640 1 -- 172.21.15.32:0/36775098 learned_addr learned my addr 172.21.15.32:0/36775098 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:05.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f8ffd000640 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a0fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f8ff6000640 1 -- 172.21.15.32:0/1194426982 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8fec042030 con 0x7f8ff80754f0 2024-08-31T07:26:05.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f8ffe400640 1 -- 172.21.15.32:0/36775098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 msgr2=0x559fa1a0fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:05.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f8ffe400640 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a0fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.865+0000 7f8ffe400640 1 -- 172.21.15.32:0/36775098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 msgr2=0x559fa1a0f810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:26:05.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.866+0000 7f8ffda00640 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 0x559fa1a0f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.866+0000 7f8ffe400640 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 0x559fa1a0f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.866+0000 7f8ffe400640 1 -- 172.21.15.32:0/36775098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa1926880 con 0x559fa1668840 2024-08-31T07:26:05.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.866+0000 7f8ffe400640 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a0f2d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00117e0 tx=0x7f8ff0011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a55f8015ac43aaa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:05.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.867+0000 7f8ffd000640 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a0fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:05.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.867+0000 7f8feaa00640 1 -- 172.21.15.32:0/36775098 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0011310 con 0x559fa1668840 2024-08-31T07:26:05.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.867+0000 7f8feaa00640 1 -- 172.21.15.32:0/36775098 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff0003e20 con 0x559fa1668840 2024-08-31T07:26:05.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.867+0000 7f8feaa00640 1 -- 172.21.15.32:0/36775098 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0004530 con 0x559fa1668840 2024-08-31T07:26:05.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.867+0000 7f8ffda00640 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 0x559fa1a0f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:26:05.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.868+0000 7f8ff6000640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8fe00b4d90 0x7f8fe00b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.868+0000 7f9007014740 1 -- 172.21.15.32:0/36775098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 msgr2=0x559fa1a0f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:05.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.868+0000 7f9007014740 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a0f2d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00117e0 tx=0x7f8ff0011ca0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.868+0000 7f8ff6000640 1 -- 172.21.15.32:0/1194426982 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f8fec105560 con 0x7f8ff80754f0 2024-08-31T07:26:05.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.868+0000 7f9007014740 1 -- 172.21.15.32:0/36775098 shutdown_connections 2024-08-31T07:26:05.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.868+0000 7f9007014740 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a0fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.868+0000 7f9007014740 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 0x559fa1a0f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.868+0000 7f9007014740 1 --2- 172.21.15.32:0/36775098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a0f2d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.868+0000 7f9007014740 1 -- 172.21.15.32:0/36775098 >> 172.21.15.32:0/36775098 conn(0x559fa165c990 msgr2=0x559fa1a0d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:05.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.868+0000 7f9007014740 1 -- 172.21.15.32:0/36775098 shutdown_connections 2024-08-31T07:26:05.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.869+0000 7f8ffda00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8fe00b4d90 0x7f8fe00b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.869+0000 7f9007014740 1 -- 172.21.15.32:0/36775098 wait complete. 2024-08-31T07:26:05.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.869+0000 7f9007014740 1 -- start start 2024-08-31T07:26:05.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.869+0000 7f8ffda00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8fe00b4d90 0x7f8fe00b72a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x559fa1a0df50 tx=0x7f8fec02a410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:05.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.869+0000 7f9007014740 1 Processor -- start 2024-08-31T07:26:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.870+0000 7f9007014740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 0x559fa1a1aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.870+0000 7f9007014740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a1b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.870+0000 7f9007014740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a26da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.870+0000 7f9007014740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559fa17d67a0 con 0x559fa163f980 2024-08-31T07:26:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.870+0000 7f9007014740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559fa18fa7f0 con 0x559fa1647b60 2024-08-31T07:26:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.870+0000 7f9007014740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559fa1a1dab0 con 0x559fa1668840 2024-08-31T07:26:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.870+0000 7f9007014740 10 client.0 _ll_register_callbacks cb 0x559fa191fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.871+0000 7f8ffd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a1b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.871+0000 7f8ffe400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 0x559fa1a1aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.871+0000 7f8ffd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a1b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54276/0 (socket says 172.21.15.32:54276) 2024-08-31T07:26:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.871+0000 7f8ffd000640 1 -- 172.21.15.32:0/37449121 learned_addr learned my addr 172.21.15.32:0/37449121 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.871+0000 7f8ffe400640 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a26da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.871+0000 7f8ffd000640 1 -- 172.21.15.32:0/37449121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 msgr2=0x559fa1a26da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.871+0000 7f8ffd000640 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a26da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.871+0000 7f8ffd000640 1 -- 172.21.15.32:0/37449121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 msgr2=0x559fa1a1aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.871+0000 7f8ffd000640 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 0x559fa1a1aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.871+0000 7f8ffd000640 1 -- 172.21.15.32:0/37449121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa16e2b90 con 0x559fa1647b60 2024-08-31T07:26:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.872+0000 7f8ffd000640 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a1b490 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4012730 tx=0x7f8fe4012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695b799ec3aef7a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.872+0000 7f8fdf400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:26:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.872+0000 7f8fdf400640 1 -- 172.21.15.32:0/37449121 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4011b30 con 0x559fa1647b60 2024-08-31T07:26:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.872+0000 7f8fdf400640 1 -- 172.21.15.32:0/37449121 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f8fe4011cd0 con 0x559fa1647b60 2024-08-31T07:26:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.872+0000 7f9007014740 1 -- 172.21.15.32:0/37449121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ff0003e20 con 0x559fa1647b60 2024-08-31T07:26:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.872+0000 7f8fdf400640 1 -- 172.21.15.32:0/37449121 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8fe4005020 con 0x559fa1647b60 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.872+0000 7f9007014740 10 client.28259 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.872+0000 7f9007014740 1 -- 172.21.15.32:0/37449121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559fa1a29b10 con 0x559fa1647b60 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.873+0000 7f8fdf400640 1 -- 172.21.15.32:0/37449121 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff0003e20 con 0x559fa1647b60 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.873+0000 7f8fdf400640 1 -- 172.21.15.32:0/37449121 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f8fe401c470 con 0x559fa1647b60 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.873+0000 7f9007014740 10 client.28259 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.873+0000 7f9007014740 10 client.28259 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.873+0000 7f9007014740 20 client.28259 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.873+0000 7f9007014740 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559fa16271c0 0x559fa1a2d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.873+0000 7f9007014740 4 client.28259 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.873+0000 7f9007014740 1 -- 172.21.15.32:0/37449121 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fa1986480 con 0x559fa16271c0 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.873+0000 7f8ffe400640 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559fa16271c0 0x559fa1a2d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.874+0000 7f8ffe400640 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559fa16271c0 0x559fa1a2d0a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3eed408998d5fe5 server_cookie=b58c5a3dccc29756 in_seq=0 out_seq=0 2024-08-31T07:26:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.874+0000 7f8fdf400640 10 client.28259 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f8fdf400640 1 -- 172.21.15.32:0/37449121 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559fa1986480 con 0x559fa16271c0 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f8fdf400640 10 client.28259 handle_command_reply: tid=0 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f9007014740 1 client.28259 shutdown 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f9007014740 1 -- 172.21.15.32:0/37449121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 msgr2=0x559fa1a1b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f9007014740 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a1b490 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4012730 tx=0x7f8fe4012bf0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f9007014740 1 -- 172.21.15.32:0/37449121 shutdown_connections 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f9007014740 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559fa16271c0 0x559fa1a2d0a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f9007014740 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a26da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f9007014740 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a1b490 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f9007014740 1 --2- 172.21.15.32:0/37449121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa163f980 0x559fa1a1aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f9007014740 1 -- 172.21.15.32:0/37449121 >> 172.21.15.32:0/37449121 conn(0x559fa165c990 msgr2=0x559fa1a12460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.875+0000 7f8fdf400640 0 client.28259 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.876+0000 7f9007014740 1 -- 172.21.15.32:0/37449121 shutdown_connections 2024-08-31T07:26:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.876+0000 7f9007014740 1 -- 172.21.15.32:0/37449121 wait complete. 2024-08-31T07:26:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.876+0000 7f9007014740 20 client.28259 trim_cache size 0 max 16384 2024-08-31T07:26:05.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.897+0000 7f9007014740 1 Processor -- start 2024-08-31T07:26:05.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.898+0000 7f9007014740 1 -- start start 2024-08-31T07:26:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.898+0000 7f9007014740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa163f980 0x559fa1a10cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.898+0000 7f9007014740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1647b60 0x559fa1a1aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.898+0000 7f9007014740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a14550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.898+0000 7f9007014740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81b13f0 con 0x559fa1647b60 2024-08-31T07:26:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.898+0000 7f9007014740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81b3050 con 0x559fa163f980 2024-08-31T07:26:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.898+0000 7f9007014740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81b1ac0 con 0x559fa1668840 2024-08-31T07:26:05.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.898+0000 7f8ffe400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa163f980 0x559fa1a10cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.898+0000 7f8ffd000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1647b60 0x559fa1a1aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.899+0000 7f8ffe400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa163f980 0x559fa1a10cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54288/0 (socket says 172.21.15.32:54288) 2024-08-31T07:26:05.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.899+0000 7f8ffe400640 1 -- 172.21.15.32:0/2826639530 learned_addr learned my addr 172.21.15.32:0/2826639530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.899+0000 7f8ffe400640 1 --2- 172.21.15.32:0/2826639530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a14550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.899+0000 7f8ffe400640 1 -- 172.21.15.32:0/2826639530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 msgr2=0x559fa1a14550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.899+0000 7f8ffe400640 1 --2- 172.21.15.32:0/2826639530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a14550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.899+0000 7f8ffe400640 1 -- 172.21.15.32:0/2826639530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1647b60 msgr2=0x559fa1a1aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.899+0000 7f8ffe400640 1 --2- 172.21.15.32:0/2826639530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1647b60 0x559fa1a1aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.899+0000 7f8ffe400640 1 -- 172.21.15.32:0/2826639530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa1a1c8f0 con 0x559fa163f980 2024-08-31T07:26:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.899+0000 7f8ffe400640 1 --2- 172.21.15.32:0/2826639530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa163f980 0x559fa1a10cc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0014040 tx=0x7f8ff0003b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=188f93b39215e73b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.900+0000 7f8fea000640 1 -- 172.21.15.32:0/2826639530 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0022040 con 0x559fa163f980 2024-08-31T07:26:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.900+0000 7f8fea000640 1 -- 172.21.15.32:0/2826639530 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff0005720 con 0x559fa163f980 2024-08-31T07:26:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.900+0000 7f8fea000640 1 -- 172.21.15.32:0/2826639530 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff002b800 con 0x559fa163f980 2024-08-31T07:26:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.901+0000 7f9007014740 1 -- 172.21.15.32:0/2826639530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa163f980 msgr2=0x559fa1a10cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.901+0000 7f9007014740 1 --2- 172.21.15.32:0/2826639530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa163f980 0x559fa1a10cc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0014040 tx=0x7f8ff0003b50 comp rx=0 tx=0).stop 2024-08-31T07:26:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.901+0000 7f9007014740 1 -- 172.21.15.32:0/2826639530 shutdown_connections 2024-08-31T07:26:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.901+0000 7f9007014740 1 --2- 172.21.15.32:0/2826639530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa1668840 0x559fa1a14550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.901+0000 7f9007014740 1 --2- 172.21.15.32:0/2826639530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1647b60 0x559fa1a1aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.901+0000 7f9007014740 1 --2- 172.21.15.32:0/2826639530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa163f980 0x559fa1a10cc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.901+0000 7f9007014740 1 -- 172.21.15.32:0/2826639530 >> 172.21.15.32:0/2826639530 conn(0x559fa165c990 msgr2=0x559fa1a15e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.901+0000 7f9007014740 1 -- 172.21.15.32:0/2826639530 shutdown_connections 2024-08-31T07:26:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.901+0000 7f9007014740 1 -- 172.21.15.32:0/2826639530 wait complete. 2024-08-31T07:26:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.902+0000 7f9007014740 1 -- start start 2024-08-31T07:26:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.902+0000 7f9007014740 1 Processor -- start 2024-08-31T07:26:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f9007014740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa163f980 0x559fa1a22fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f9007014740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a23580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f9007014740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1668840 0x559fa1a1dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f9007014740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81b28c0 con 0x559fa1668840 2024-08-31T07:26:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f9007014740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81b2190 con 0x559fa1647b60 2024-08-31T07:26:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f9007014740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81af3f0 con 0x559fa163f980 2024-08-31T07:26:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f9007014740 10 client.0 _ll_register_callbacks cb 0x7f8fe4009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f8ffe400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa163f980 0x559fa1a22fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f8ffe400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa163f980 0x559fa1a22fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38298/0 (socket says 172.21.15.32:38298) 2024-08-31T07:26:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f8ffe400640 1 -- 172.21.15.32:0/464699960 learned_addr learned my addr 172.21.15.32:0/464699960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.903+0000 7f8ffd000640 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a23580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.904+0000 7f8ffe400640 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1668840 0x559fa1a1dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.904+0000 7f8ffd000640 1 -- 172.21.15.32:0/464699960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa163f980 msgr2=0x559fa1a22fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.904+0000 7f8ffd000640 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa163f980 0x559fa1a22fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.904+0000 7f8ffd000640 1 -- 172.21.15.32:0/464699960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1668840 msgr2=0x559fa1a1dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.904+0000 7f8ffd000640 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1668840 0x559fa1a1dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.904+0000 7f8ffd000640 1 -- 172.21.15.32:0/464699960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa1a1e470 con 0x559fa1647b60 2024-08-31T07:26:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.904+0000 7f8ffe400640 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1668840 0x559fa1a1dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:26:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.905+0000 7f8fdf400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:26:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.905+0000 7f8ffd000640 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a23580 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8fe401ca30 tx=0x7f8fe4011d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efaa0463c76838e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.905+0000 7f8fdf400640 1 -- 172.21.15.32:0/464699960 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe401ccc0 con 0x559fa1647b60 2024-08-31T07:26:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.906+0000 7f8fdf400640 1 -- 172.21.15.32:0/464699960 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8441+0+0 (secure 0 0 0) 0x7f8fe40051c0 con 0x559fa1647b60 2024-08-31T07:26:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.906+0000 7f9007014740 1 -- 172.21.15.32:0/464699960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ff0005720 con 0x559fa1647b60 2024-08-31T07:26:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.906+0000 7f8fdf400640 1 -- 172.21.15.32:0/464699960 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8fe401ce60 con 0x559fa1647b60 2024-08-31T07:26:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.906+0000 7f9007014740 10 client.28265 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.906+0000 7f9007014740 1 -- 172.21.15.32:0/464699960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559fa1a98460 con 0x559fa1647b60 2024-08-31T07:26:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.906+0000 7f8fdf400640 1 -- 172.21.15.32:0/464699960 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff0005720 con 0x559fa1647b60 2024-08-31T07:26:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.907+0000 7f8fdf400640 1 -- 172.21.15.32:0/464699960 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f8fe40047f0 con 0x559fa1647b60 2024-08-31T07:26:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.907+0000 7f9007014740 10 client.28265 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.907+0000 7f9007014740 10 client.28265 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:26:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.907+0000 7f9007014740 20 client.28265 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.907+0000 7f9007014740 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559fa15fc570 0x559fa1a9acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:05.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.907+0000 7f9007014740 4 client.28265 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.7, "await_for": 45.0}] 2024-08-31T07:26:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.907+0000 7f9007014740 1 -- 172.21.15.32:0/464699960 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.7, "await_for": 45.0}) -- 0x559fa1981f20 con 0x559fa15fc570 2024-08-31T07:26:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.908+0000 7f8ffe400640 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559fa15fc570 0x559fa1a9acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.908+0000 7f8ffe400640 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559fa15fc570 0x559fa1a9acd0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12c10c54354ac84 server_cookie=bef8a1706e7e0d19 in_seq=0 out_seq=0 2024-08-31T07:26:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.908+0000 7f8fdf400640 10 client.28265 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:05.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.988+0000 7f8fdf400640 1 -- 172.21.15.32:0/464699960 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559fa1981f20 con 0x559fa15fc570 2024-08-31T07:26:05.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.988+0000 7f8fdf400640 10 client.28265 handle_command_reply: tid=0 2024-08-31T07:26:05.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.988+0000 7f9007014740 1 client.28265 shutdown 2024-08-31T07:26:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.988+0000 7f9007014740 1 -- 172.21.15.32:0/464699960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 msgr2=0x559fa1a23580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.988+0000 7f9007014740 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a23580 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8fe401ca30 tx=0x7f8fe4011d60 comp rx=0 tx=0).stop 2024-08-31T07:26:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.988+0000 7f9007014740 1 -- 172.21.15.32:0/464699960 shutdown_connections 2024-08-31T07:26:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.989+0000 7f9007014740 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559fa15fc570 0x559fa1a9acd0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.989+0000 7f9007014740 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fa1668840 0x559fa1a1dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.989+0000 7f9007014740 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa1647b60 0x559fa1a23580 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.989+0000 7f9007014740 1 --2- 172.21.15.32:0/464699960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559fa163f980 0x559fa1a22fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.989+0000 7f8fdf400640 0 client.28265 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.989+0000 7f9007014740 1 -- 172.21.15.32:0/464699960 >> 172.21.15.32:0/464699960 conn(0x559fa165c990 msgr2=0x559fa1a15e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:05.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.989+0000 7f9007014740 1 -- 172.21.15.32:0/464699960 shutdown_connections 2024-08-31T07:26:05.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.989+0000 7f9007014740 1 -- 172.21.15.32:0/464699960 wait complete. 2024-08-31T07:26:05.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.990+0000 7f9007014740 20 client.28265 trim_cache size 0 max 16384 2024-08-31T07:26:05.991 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:26:05.991 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 47, 2024-08-31T07:26:05.991 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:26:05.991 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 42, 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "63a6ed69": { 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 42, 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 134.7, 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:26:05.992 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:26:05.993 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:26:05.993 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:26:05.993 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:26:05.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:05.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:05.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:05.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:05.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:05.993 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:26:05.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 -- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8fe00b4d90 msgr2=0x7f8fe00b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:05.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8fe00b4d90 0x7f8fe00b72a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x559fa1a0df50 tx=0x7f8fec02a410 comp rx=0 tx=0).stop 2024-08-31T07:26:05.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 -- 172.21.15.32:0/1194426982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ff80754f0 msgr2=0x7f8ff819c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:05.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ff80754f0 0x7f8ff819c2b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8fec005940 tx=0x7f8fec0040c0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 -- 172.21.15.32:0/1194426982 shutdown_connections 2024-08-31T07:26:05.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8fe00b4d90 0x7f8fe00b72a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ff810fd50 0x7f8ff81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff8075e40 0x7f8ff819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 --2- 172.21.15.32:0/1194426982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ff80754f0 0x7f8ff819c2b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:05.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 -- 172.21.15.32:0/1194426982 >> 172.21.15.32:0/1194426982 conn(0x7f8ff80ff220 msgr2=0x7f8ff81098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:05.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.991+0000 7f8ffee00640 1 -- 172.21.15.32:0/1194426982 shutdown_connections 2024-08-31T07:26:05.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:05.992+0000 7f8ffee00640 1 -- 172.21.15.32:0/1194426982 wait complete. 2024-08-31T07:26:06.784 INFO:tasks.workunit.client.0.smithi032.stdout: 1 408304 2.76 MB/sec execute 595 sec latency 876.014 ms 2024-08-31T07:26:06.877 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '63a6ed69', quiesce took 6.5 seconds. Will release after: 8.2 2024-08-31T07:26:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:06 smithi084 ceph-mon[22387]: pgmap v782: 97 pgs: 97 active+clean; 348 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-08-31T07:26:07.049 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:06 smithi032 ceph-mon[17252]: pgmap v782: 97 pgs: 97 active+clean; 348 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-08-31T07:26:07.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:06 smithi073 ceph-mon[21995]: pgmap v782: 97 pgs: 97 active+clean; 348 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-08-31T07:26:07.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.551+0000 7f4b10800640 1 -- 172.21.15.32:0/963220443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b08109240 msgr2=0x7f4b08109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:07.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.551+0000 7f4b10800640 1 --2- 172.21.15.32:0/963220443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b08109240 0x7f4b08109650 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009900 tx=0x7f4af802a410 comp rx=0 tx=0).stop 2024-08-31T07:26:07.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.551+0000 7f4b10800640 1 -- 172.21.15.32:0/963220443 shutdown_connections 2024-08-31T07:26:07.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.551+0000 7f4b10800640 1 --2- 172.21.15.32:0/963220443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b08109b90 0x7f4b081103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.551+0000 7f4b10800640 1 --2- 172.21.15.32:0/963220443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b08109240 0x7f4b08109650 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.551+0000 7f4b10800640 1 --2- 172.21.15.32:0/963220443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b08103900 0x7f4b08103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.551+0000 7f4b10800640 1 -- 172.21.15.32:0/963220443 >> 172.21.15.32:0/963220443 conn(0x7f4b0807b5f0 msgr2=0x7f4b0807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.551+0000 7f4b10800640 1 -- 172.21.15.32:0/963220443 shutdown_connections 2024-08-31T07:26:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.551+0000 7f4b10800640 1 -- 172.21.15.32:0/963220443 wait complete. 2024-08-31T07:26:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.552+0000 7f4b10800640 1 Processor -- start 2024-08-31T07:26:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b10800640 1 -- start start 2024-08-31T07:26:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b10800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b08103900 0x7f4b0819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b10800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b08109240 0x7f4b0819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b10800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b08109b90 0x7f4b0819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b10800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081138a0 con 0x7f4b08109240 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b10800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08113720 con 0x7f4b08109b90 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b10800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08113a20 con 0x7f4b08103900 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b0d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b08109240 0x7f4b0819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b0d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b08109240 0x7f4b0819a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54640/0 (socket says 172.21.15.32:54640) 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b0d400640 1 -- 172.21.15.32:0/3579097179 learned_addr learned my addr 172.21.15.32:0/3579097179 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b0e800640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b08109b90 0x7f4b0819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.553+0000 7f4b0de00640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b08103900 0x7f4b0819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.554+0000 7f4b0d400640 1 -- 172.21.15.32:0/3579097179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b08103900 msgr2=0x7f4b0819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.554+0000 7f4b0d400640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b08103900 0x7f4b0819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.554+0000 7f4b0d400640 1 -- 172.21.15.32:0/3579097179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b08109b90 msgr2=0x7f4b0819fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.554+0000 7f4b0d400640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b08109b90 0x7f4b0819fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.554+0000 7f4b0d400640 1 -- 172.21.15.32:0/3579097179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b081a01a0 con 0x7f4b08109240 2024-08-31T07:26:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.554+0000 7f4b0e800640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b08109b90 0x7f4b0819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.554+0000 7f4b0d400640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b08109240 0x7f4b0819a7d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4af8005ab0 tx=0x7f4af8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.555+0000 7f4b02a00640 1 -- 172.21.15.32:0/3579097179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af803f070 con 0x7f4b08109240 2024-08-31T07:26:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.555+0000 7f4b02a00640 1 -- 172.21.15.32:0/3579097179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4af8031b50 con 0x7f4b08109240 2024-08-31T07:26:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.555+0000 7f4b02a00640 1 -- 172.21.15.32:0/3579097179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8035030 con 0x7f4b08109240 2024-08-31T07:26:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.555+0000 7f4b10800640 1 -- 172.21.15.32:0/3579097179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b081a0430 con 0x7f4b08109240 2024-08-31T07:26:07.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.555+0000 7f4b10800640 1 -- 172.21.15.32:0/3579097179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b08075390 con 0x7f4b08109240 2024-08-31T07:26:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.558+0000 7f4b17467740 1 Processor -- start 2024-08-31T07:26:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.558+0000 7f4b17467740 1 -- start start 2024-08-31T07:26:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.558+0000 7f4b02a00640 1 -- 172.21.15.32:0/3579097179 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4af8042070 con 0x7f4b08109240 2024-08-31T07:26:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b17467740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 0x55ee9849a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b17467740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee9849aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b17467740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 0x55ee9849b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b17467740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ee984117b0 con 0x55ee980cc1b0 2024-08-31T07:26:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b17467740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ee98400d40 con 0x55ee980d4390 2024-08-31T07:26:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b17467740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ee98404cf0 con 0x55ee980f5070 2024-08-31T07:26:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b0e800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 0x55ee9849a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b0de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee9849aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b0e800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 0x55ee9849a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38322/0 (socket says 172.21.15.32:38322) 2024-08-31T07:26:07.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b0e800640 1 -- 172.21.15.32:0/3691837623 learned_addr learned my addr 172.21.15.32:0/3691837623 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b0d400640 1 --2- 172.21.15.32:0/3691837623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 0x55ee9849b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b0e800640 1 -- 172.21.15.32:0/3691837623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 msgr2=0x55ee9849b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.559+0000 7f4b0e800640 1 --2- 172.21.15.32:0/3691837623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 0x55ee9849b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.560+0000 7f4b0e800640 1 -- 172.21.15.32:0/3691837623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 msgr2=0x55ee9849aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.560+0000 7f4b0e800640 1 --2- 172.21.15.32:0/3691837623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee9849aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.560+0000 7f4b0e800640 1 -- 172.21.15.32:0/3691837623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee98160650 con 0x55ee980f5070 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.560+0000 7f4b0de00640 1 --2- 172.21.15.32:0/3691837623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee9849aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.560+0000 7f4b0e800640 1 --2- 172.21.15.32:0/3691837623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 0x55ee9849a990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4b040117e0 tx=0x7f4b04011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e40a4a8d2f37b21 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.560+0000 7f4af7000640 1 -- 172.21.15.32:0/3691837623 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04011310 con 0x55ee980f5070 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.560+0000 7f4af7000640 1 -- 172.21.15.32:0/3691837623 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b04003e20 con 0x55ee980f5070 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.560+0000 7f4af7000640 1 -- 172.21.15.32:0/3691837623 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b040044f0 con 0x55ee980f5070 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.560+0000 7f4b02a00640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4af00b4d20 0x7f4af00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.561+0000 7f4b02a00640 1 -- 172.21.15.32:0/3579097179 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f4af8105a90 con 0x7f4b08109240 2024-08-31T07:26:07.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.561+0000 7f4b0de00640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4af00b4d20 0x7f4af00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.561+0000 7f4b17467740 1 -- 172.21.15.32:0/3691837623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 msgr2=0x55ee9849a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.561+0000 7f4b17467740 1 --2- 172.21.15.32:0/3691837623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 0x55ee9849a990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4b040117e0 tx=0x7f4b04011ca0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.561+0000 7f4b17467740 1 -- 172.21.15.32:0/3691837623 shutdown_connections 2024-08-31T07:26:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.561+0000 7f4b17467740 1 --2- 172.21.15.32:0/3691837623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 0x55ee9849b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.561+0000 7f4b17467740 1 --2- 172.21.15.32:0/3691837623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee9849aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.561+0000 7f4b17467740 1 --2- 172.21.15.32:0/3691837623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 0x55ee9849a990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.561+0000 7f4b17467740 1 -- 172.21.15.32:0/3691837623 >> 172.21.15.32:0/3691837623 conn(0x55ee980e9460 msgr2=0x55ee98498880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.561+0000 7f4b0de00640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4af00b4d20 0x7f4af00b7230 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55ee98499610 tx=0x7f4afc012040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.562+0000 7f4b17467740 1 -- 172.21.15.32:0/3691837623 shutdown_connections 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.562+0000 7f4b17467740 1 -- 172.21.15.32:0/3691837623 wait complete. 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.562+0000 7f4b17467740 1 -- start start 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.562+0000 7f4b17467740 1 Processor -- start 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b17467740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980cc1b0 0x55ee984a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b17467740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980d4390 0x55ee984a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b17467740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 0x55ee984b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b17467740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ee98263700 con 0x55ee980d4390 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b17467740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ee98387cf0 con 0x55ee980f5070 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b17467740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ee984a9170 con 0x55ee980cc1b0 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b17467740 10 client.0 _ll_register_callbacks cb 0x55ee983ad470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b0de00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980cc1b0 0x55ee984a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b0d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980d4390 0x55ee984a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b0de00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980cc1b0 0x55ee984a65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38332/0 (socket says 172.21.15.32:38332) 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b0de00640 1 -- 172.21.15.32:0/2758616223 learned_addr learned my addr 172.21.15.32:0/2758616223 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.563+0000 7f4b0e800640 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 0x55ee984b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.564+0000 7f4b0d400640 1 -- 172.21.15.32:0/2758616223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980cc1b0 msgr2=0x55ee984a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.564+0000 7f4b0d400640 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980cc1b0 0x55ee984a65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.564+0000 7f4b0d400640 1 -- 172.21.15.32:0/2758616223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 msgr2=0x55ee984b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.564+0000 7f4b0d400640 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 0x55ee984b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.564+0000 7f4b0d400640 1 -- 172.21.15.32:0/2758616223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee983c88b0 con 0x55ee980d4390 2024-08-31T07:26:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.564+0000 7f4b0d400640 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980d4390 0x55ee984a6b30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4af81031a0 tx=0x7f4af8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faed4d49f72908ea server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.564+0000 7f4aeb400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:26:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.564+0000 7f4aeb400640 1 -- 172.21.15.32:0/2758616223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8049030 con 0x55ee980d4390 2024-08-31T07:26:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.564+0000 7f4aeb400640 1 -- 172.21.15.32:0/2758616223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f4af8103420 con 0x55ee980d4390 2024-08-31T07:26:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.564+0000 7f4b17467740 1 -- 172.21.15.32:0/2758616223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b04003e20 con 0x55ee980d4390 2024-08-31T07:26:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4aeb400640 1 -- 172.21.15.32:0/2758616223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4af810e070 con 0x55ee980d4390 2024-08-31T07:26:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4b17467740 10 client.18267 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4b17467740 1 -- 172.21.15.32:0/2758616223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ee984b51e0 con 0x55ee980d4390 2024-08-31T07:26:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4aeb400640 1 -- 172.21.15.32:0/2758616223 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b04003e20 con 0x55ee980d4390 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4aeb400640 1 -- 172.21.15.32:0/2758616223 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f4af80cc450 con 0x55ee980d4390 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4b17467740 10 client.18267 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4b17467740 10 client.18267 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4b17467740 20 client.18267 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4b17467740 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ee980b39f0 0x55ee984b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4b17467740 4 client.18267 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.565+0000 7f4b17467740 1 -- 172.21.15.32:0/2758616223 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee98413700 con 0x55ee980b39f0 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.566+0000 7f4b0de00640 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ee980b39f0 0x55ee984b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.566+0000 7f4b0de00640 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ee980b39f0 0x55ee984b86d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c04ec783e6e7323 server_cookie=fd0d08b0d13aebde in_seq=0 out_seq=0 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.566+0000 7f4aeb400640 10 client.18267 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:26:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.567+0000 7f4aeb400640 1 -- 172.21.15.32:0/2758616223 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee98413700 con 0x55ee980b39f0 2024-08-31T07:26:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.567+0000 7f4aeb400640 10 client.18267 handle_command_reply: tid=0 2024-08-31T07:26:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.567+0000 7f4b17467740 1 client.18267 shutdown 2024-08-31T07:26:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.567+0000 7f4b17467740 1 -- 172.21.15.32:0/2758616223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980d4390 msgr2=0x55ee984a6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.567+0000 7f4b17467740 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980d4390 0x55ee984a6b30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4af81031a0 tx=0x7f4af8009560 comp rx=0 tx=0).stop 2024-08-31T07:26:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.567+0000 7f4b17467740 1 -- 172.21.15.32:0/2758616223 shutdown_connections 2024-08-31T07:26:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.567+0000 7f4b17467740 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ee980b39f0 0x55ee984b86d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.568+0000 7f4b17467740 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 0x55ee984b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.568+0000 7f4b17467740 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980d4390 0x55ee984a6b30 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.568+0000 7f4b17467740 1 --2- 172.21.15.32:0/2758616223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980cc1b0 0x55ee984a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.568+0000 7f4b17467740 1 -- 172.21.15.32:0/2758616223 >> 172.21.15.32:0/2758616223 conn(0x55ee980e9460 msgr2=0x55ee9849db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.568+0000 7f4aeb400640 0 client.18267 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:26:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.568+0000 7f4b17467740 1 -- 172.21.15.32:0/2758616223 shutdown_connections 2024-08-31T07:26:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.568+0000 7f4b17467740 1 -- 172.21.15.32:0/2758616223 wait complete. 2024-08-31T07:26:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.568+0000 7f4b17467740 20 client.18267 trim_cache size 0 max 16384 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.582+0000 7f4b17467740 1 Processor -- start 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.582+0000 7f4b17467740 1 -- start start 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.582+0000 7f4b17467740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.582+0000 7f4b17467740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980d4390 0x55ee9849faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.582+0000 7f4b17467740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 0x55ee9849ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.582+0000 7f4b17467740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081ad230 con 0x55ee980cc1b0 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.582+0000 7f4b17467740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081aee90 con 0x55ee980f5070 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.582+0000 7f4b17467740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081ad900 con 0x55ee980d4390 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.582+0000 7f4b0de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54666/0 (socket says 172.21.15.32:54666) 2024-08-31T07:26:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0de00640 1 -- 172.21.15.32:0/3657429603 learned_addr learned my addr 172.21.15.32:0/3657429603 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0d400640 1 --2- 172.21.15.32:0/3657429603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980d4390 0x55ee9849faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0e800640 1 --2- 172.21.15.32:0/3657429603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 0x55ee9849ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0de00640 1 -- 172.21.15.32:0/3657429603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980d4390 msgr2=0x55ee9849faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0de00640 1 --2- 172.21.15.32:0/3657429603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980d4390 0x55ee9849faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0de00640 1 -- 172.21.15.32:0/3657429603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 msgr2=0x55ee9849ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0de00640 1 --2- 172.21.15.32:0/3657429603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 0x55ee9849ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0de00640 1 -- 172.21.15.32:0/3657429603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee984a7690 con 0x55ee980cc1b0 2024-08-31T07:26:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0e800640 1 --2- 172.21.15.32:0/3657429603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 0x55ee9849ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:26:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4b0de00640 1 --2- 172.21.15.32:0/3657429603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984b2390 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009f80 tx=0x7f4afc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b137b594beb21d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.583+0000 7f4af6600640 1 -- 172.21.15.32:0/3657429603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc004250 con 0x55ee980cc1b0 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.584+0000 7f4af6600640 1 -- 172.21.15.32:0/3657429603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4afc021040 con 0x55ee980cc1b0 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.584+0000 7f4af6600640 1 -- 172.21.15.32:0/3657429603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc02a780 con 0x55ee980cc1b0 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.584+0000 7f4b17467740 1 -- 172.21.15.32:0/3657429603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 msgr2=0x55ee984b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.584+0000 7f4b17467740 1 --2- 172.21.15.32:0/3657429603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984b2390 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009f80 tx=0x7f4afc01a000 comp rx=0 tx=0).stop 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.584+0000 7f4b17467740 1 -- 172.21.15.32:0/3657429603 shutdown_connections 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.584+0000 7f4b17467740 1 --2- 172.21.15.32:0/3657429603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980f5070 0x55ee9849ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.584+0000 7f4b17467740 1 --2- 172.21.15.32:0/3657429603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980d4390 0x55ee9849faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.584+0000 7f4b17467740 1 --2- 172.21.15.32:0/3657429603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984b2390 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.584+0000 7f4b17467740 1 -- 172.21.15.32:0/3657429603 >> 172.21.15.32:0/3657429603 conn(0x55ee980e9460 msgr2=0x55ee984a4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.585+0000 7f4b17467740 1 -- 172.21.15.32:0/3657429603 shutdown_connections 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.585+0000 7f4b17467740 1 -- 172.21.15.32:0/3657429603 wait complete. 2024-08-31T07:26:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.585+0000 7f4b17467740 1 -- start start 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.585+0000 7f4b17467740 1 Processor -- start 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b17467740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984aae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b17467740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 0x55ee984ab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b17467740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 0x55ee9851c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b17467740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081ae700 con 0x55ee980cc1b0 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b17467740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081adfd0 con 0x55ee980d4390 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b17467740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081ab230 con 0x55ee980f5070 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b17467740 10 client.0 _ll_register_callbacks cb 0x7f4af8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b0de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b0de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54668/0 (socket says 172.21.15.32:54668) 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b0de00640 1 -- 172.21.15.32:0/100578540 learned_addr learned my addr 172.21.15.32:0/100578540 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b0d400640 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 0x55ee984ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.586+0000 7f4b0e800640 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 0x55ee9851c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4b0d400640 1 -- 172.21.15.32:0/100578540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 msgr2=0x55ee9851c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4b0d400640 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 0x55ee9851c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4b0d400640 1 -- 172.21.15.32:0/100578540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 msgr2=0x55ee984aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4b0d400640 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4b0d400640 1 -- 172.21.15.32:0/100578540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee9851c800 con 0x55ee980d4390 2024-08-31T07:26:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4b0de00640 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4b0d400640 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 0x55ee984ab400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4af8006010 tx=0x7f4af8031f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f5d3eb7480d6ff6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4aeb400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:26:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4aeb400640 1 -- 172.21.15.32:0/100578540 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8049040 con 0x55ee980d4390 2024-08-31T07:26:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4aeb400640 1 -- 172.21.15.32:0/100578540 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f4af80d51e0 con 0x55ee980d4390 2024-08-31T07:26:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.587+0000 7f4b17467740 1 -- 172.21.15.32:0/100578540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4afc021040 con 0x55ee980d4390 2024-08-31T07:26:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4aeb400640 1 -- 172.21.15.32:0/100578540 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4af80cb960 con 0x55ee980d4390 2024-08-31T07:26:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4b17467740 10 client.28283 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4b17467740 1 -- 172.21.15.32:0/100578540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ee9851c800 con 0x55ee980d4390 2024-08-31T07:26:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4aeb400640 1 -- 172.21.15.32:0/100578540 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4afc021040 con 0x55ee980d4390 2024-08-31T07:26:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4aeb400640 1 -- 172.21.15.32:0/100578540 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f4af803bb90 con 0x55ee980d4390 2024-08-31T07:26:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4b17467740 10 client.28283 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4b17467740 10 client.28283 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:26:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4b17467740 20 client.28283 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4b17467740 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ee98088da0 0x55ee98525fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4b17467740 4 client.28283 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:26:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.588+0000 7f4b17467740 1 -- 172.21.15.32:0/100578540 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee984a6ec0 con 0x55ee98088da0 2024-08-31T07:26:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.589+0000 7f4b0de00640 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ee98088da0 0x55ee98525fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.589+0000 7f4aeb400640 10 client.28283 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.589+0000 7f4b0de00640 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ee98088da0 0x55ee98525fa0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9888239b41954574 server_cookie=7d9898926430daec in_seq=0 out_seq=0 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4aeb400640 1 -- 172.21.15.32:0/100578540 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee984a6ec0 con 0x55ee98088da0 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4aeb400640 10 client.28283 handle_command_reply: tid=0 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 client.28283 shutdown 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 -- 172.21.15.32:0/100578540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 msgr2=0x55ee984ab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 0x55ee984ab400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4af8006010 tx=0x7f4af8031f70 comp rx=0 tx=0).stop 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 -- 172.21.15.32:0/100578540 shutdown_connections 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ee98088da0 0x55ee98525fa0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee980f5070 0x55ee9851c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee980d4390 0x55ee984ab400 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 --2- 172.21.15.32:0/100578540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee980cc1b0 0x55ee984aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4aeb400640 0 client.28283 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:26:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 -- 172.21.15.32:0/100578540 >> 172.21.15.32:0/100578540 conn(0x55ee980e9460 msgr2=0x55ee9849faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 -- 172.21.15.32:0/100578540 shutdown_connections 2024-08-31T07:26:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.590+0000 7f4b17467740 1 -- 172.21.15.32:0/100578540 wait complete. 2024-08-31T07:26:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.591+0000 7f4b17467740 20 client.28283 trim_cache size 0 max 16384 2024-08-31T07:26:07.595 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:26:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.591+0000 7f4b10800640 1 -- 172.21.15.32:0/3579097179 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4af00b4d20 msgr2=0x7f4af00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.591+0000 7f4b10800640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4af00b4d20 0x7f4af00b7230 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55ee98499610 tx=0x7f4afc012040 comp rx=0 tx=0).stop 2024-08-31T07:26:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.591+0000 7f4b10800640 1 -- 172.21.15.32:0/3579097179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b08109240 msgr2=0x7f4b0819a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.591+0000 7f4b10800640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b08109240 0x7f4b0819a7d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4af8005ab0 tx=0x7f4af8003b90 comp rx=0 tx=0).stop 2024-08-31T07:26:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.591+0000 7f4b10800640 1 -- 172.21.15.32:0/3579097179 shutdown_connections 2024-08-31T07:26:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.591+0000 7f4b10800640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4af00b4d20 0x7f4af00b7230 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.591+0000 7f4b10800640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b08109b90 0x7f4b0819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.591+0000 7f4b10800640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b08109240 0x7f4b0819a7d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.591+0000 7f4b10800640 1 --2- 172.21.15.32:0/3579097179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b08103900 0x7f4b0819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.592+0000 7f4b10800640 1 -- 172.21.15.32:0/3579097179 >> 172.21.15.32:0/3579097179 conn(0x7f4b0807b5f0 msgr2=0x7f4b081052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.592+0000 7f4b10800640 1 -- 172.21.15.32:0/3579097179 shutdown_connections 2024-08-31T07:26:07.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:07.592+0000 7f4b10800640 1 -- 172.21.15.32:0/3579097179 wait complete. 2024-08-31T07:26:07.785 INFO:tasks.workunit.client.0.smithi032.stdout: 1 408304 2.76 MB/sec execute 596 sec latency 1876.096 ms 2024-08-31T07:26:07.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:07 smithi084 ceph-mon[22387]: osdmap e95: 12 total, 12 up, 12 in 2024-08-31T07:26:08.018 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:07 smithi032 ceph-mon[17252]: osdmap e95: 12 total, 12 up, 12 in 2024-08-31T07:26:08.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:07 smithi073 ceph-mon[21995]: osdmap e95: 12 total, 12 up, 12 in 2024-08-31T07:26:08.785 INFO:tasks.workunit.client.0.smithi032.stdout: 1 408304 2.75 MB/sec execute 597 sec latency 2876.192 ms 2024-08-31T07:26:08.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:26:08.971 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:08 smithi032 ceph-mon[17252]: pgmap v784: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-08-31T07:26:08.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:08 smithi084 ceph-mon[22387]: pgmap v784: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-08-31T07:26:09.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:08 smithi073 ceph-mon[21995]: pgmap v784: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-08-31T07:26:09.306 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:09.785 INFO:tasks.workunit.client.0.smithi032.stdout: 1 408304 2.75 MB/sec execute 598 sec latency 3876.348 ms 2024-08-31T07:26:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.993+0000 7f0b81a00640 1 -- 172.21.15.32:0/4205754945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b7c078030 msgr2=0x7f0b7c113dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.993+0000 7f0b81a00640 1 --2- 172.21.15.32:0/4205754945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b7c078030 0x7f0b7c113dd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009900 tx=0x7f0b6802a410 comp rx=0 tx=0).stop 2024-08-31T07:26:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.993+0000 7f0b81a00640 1 -- 172.21.15.32:0/4205754945 shutdown_connections 2024-08-31T07:26:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.994+0000 7f0b81a00640 1 --2- 172.21.15.32:0/4205754945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b7c114310 0x7f0b7c116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.994+0000 7f0b81a00640 1 --2- 172.21.15.32:0/4205754945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b7c078030 0x7f0b7c113dd0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.994+0000 7f0b81a00640 1 --2- 172.21.15.32:0/4205754945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c0776e0 0x7f0b7c077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.994+0000 7f0b81a00640 1 -- 172.21.15.32:0/4205754945 >> 172.21.15.32:0/4205754945 conn(0x7f0b7c101420 msgr2=0x7f0b7c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.994+0000 7f0b81a00640 1 -- 172.21.15.32:0/4205754945 shutdown_connections 2024-08-31T07:26:09.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.994+0000 7f0b81a00640 1 -- 172.21.15.32:0/4205754945 wait complete. 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.995+0000 7f0b81a00640 1 Processor -- start 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.995+0000 7f0b81a00640 1 -- start start 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.995+0000 7f0b81a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b7c0776e0 0x7f0b7c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.995+0000 7f0b81a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b7c078030 0x7f0b7c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.995+0000 7f0b81a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c114310 0x7f0b7c1a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.995+0000 7f0b81a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c119cc0 con 0x7f0b7c078030 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.995+0000 7f0b81a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c119b40 con 0x7f0b7c0776e0 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.995+0000 7f0b81a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c119e40 con 0x7f0b7c114310 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.995+0000 7f0b7b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b7c078030 0x7f0b7c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.996+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b7c0776e0 0x7f0b7c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.996+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b7c0776e0 0x7f0b7c1a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54356/0 (socket says 172.21.15.32:54356) 2024-08-31T07:26:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.996+0000 7f0b7be00640 1 -- 172.21.15.32:0/430343054 learned_addr learned my addr 172.21.15.32:0/430343054 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.996+0000 7f0b81000640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c114310 0x7f0b7c1a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.996+0000 7f0b7be00640 1 -- 172.21.15.32:0/430343054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c114310 msgr2=0x7f0b7c1a5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.996+0000 7f0b7be00640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c114310 0x7f0b7c1a5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.996+0000 7f0b7be00640 1 -- 172.21.15.32:0/430343054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b7c078030 msgr2=0x7f0b7c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.996+0000 7f0b7be00640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b7c078030 0x7f0b7c1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.996+0000 7f0b7be00640 1 -- 172.21.15.32:0/430343054 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c1a6680 con 0x7f0b7c0776e0 2024-08-31T07:26:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.996+0000 7f0b7be00640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b7c0776e0 0x7f0b7c1a0710 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00e9f0 tx=0x7f0b6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.997+0000 7f0b78c00640 1 -- 172.21.15.32:0/430343054 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00cd80 con 0x7f0b7c0776e0 2024-08-31T07:26:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.997+0000 7f0b81a00640 1 -- 172.21.15.32:0/430343054 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c1a6970 con 0x7f0b7c0776e0 2024-08-31T07:26:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.997+0000 7f0b78c00640 1 -- 172.21.15.32:0/430343054 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b6c0108e0 con 0x7f0b7c0776e0 2024-08-31T07:26:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.997+0000 7f0b78c00640 1 -- 172.21.15.32:0/430343054 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c018980 con 0x7f0b7c0776e0 2024-08-31T07:26:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:09.997+0000 7f0b81a00640 1 -- 172.21.15.32:0/430343054 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c1ad0d0 con 0x7f0b7c0776e0 2024-08-31T07:26:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.000+0000 7f0b89c5b740 1 Processor -- start 2024-08-31T07:26:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.000+0000 7f0b89c5b740 1 -- start start 2024-08-31T07:26:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.000+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb71070 0x558ebbf16b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.000+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.000+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 0x558ebbf175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.000+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558ebbe8d7b0 con 0x558ebbb50390 2024-08-31T07:26:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.000+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558ebbe7cd40 con 0x558ebbb481b0 2024-08-31T07:26:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.000+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558ebbe80cf0 con 0x558ebbb71070 2024-08-31T07:26:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b7b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 0x558ebbf175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b7b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 0x558ebbf175e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54690/0 (socket says 172.21.15.32:54690) 2024-08-31T07:26:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b7b400640 1 -- 172.21.15.32:0/2525225409 learned_addr learned my addr 172.21.15.32:0/2525225409 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b78c00640 1 -- 172.21.15.32:0/430343054 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0b6c010430 con 0x7f0b7c0776e0 2024-08-31T07:26:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b81000640 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb71070 0x558ebbf16b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b7be00640 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b7b400640 1 -- 172.21.15.32:0/2525225409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb71070 msgr2=0x558ebbf16b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b7b400640 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb71070 0x558ebbf16b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b7b400640 1 -- 172.21.15.32:0/2525225409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 msgr2=0x558ebbf170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b7b400640 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b7b400640 1 -- 172.21.15.32:0/2525225409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ebbbdc650 con 0x558ebbb50390 2024-08-31T07:26:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b81000640 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb71070 0x558ebbf16b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.001+0000 7f0b7be00640 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:26:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.002+0000 7f0b7b400640 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 0x558ebbf175e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0b68005ab0 tx=0x7f0b68004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce5e7695fe343a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.002+0000 7f0b71600640 1 -- 172.21.15.32:0/2525225409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68035040 con 0x558ebbb50390 2024-08-31T07:26:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.002+0000 7f0b71600640 1 -- 172.21.15.32:0/2525225409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b680054f0 con 0x558ebbb50390 2024-08-31T07:26:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.002+0000 7f0b71600640 1 -- 172.21.15.32:0/2525225409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b680035f0 con 0x558ebbb50390 2024-08-31T07:26:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b78c00640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0b600b4ca0 0x7f0b600b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b78c00640 1 -- 172.21.15.32:0/430343054 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f0b6c0e5830 con 0x7f0b7c0776e0 2024-08-31T07:26:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b81000640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0b600b4ca0 0x7f0b600b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b89c5b740 1 -- 172.21.15.32:0/2525225409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 msgr2=0x558ebbf175e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 0x558ebbf175e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0b68005ab0 tx=0x7f0b68004040 comp rx=0 tx=0).stop 2024-08-31T07:26:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b89c5b740 1 -- 172.21.15.32:0/2525225409 shutdown_connections 2024-08-31T07:26:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 0x558ebbf175e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/2525225409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb71070 0x558ebbf16b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b89c5b740 1 -- 172.21.15.32:0/2525225409 >> 172.21.15.32:0/2525225409 conn(0x558ebbb65460 msgr2=0x558ebbf14a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.003+0000 7f0b89c5b740 1 -- 172.21.15.32:0/2525225409 shutdown_connections 2024-08-31T07:26:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.004+0000 7f0b89c5b740 1 -- 172.21.15.32:0/2525225409 wait complete. 2024-08-31T07:26:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.004+0000 7f0b81000640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0b600b4ca0 0x7f0b600b71b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009d40 tx=0x7f0b74009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.004+0000 7f0b89c5b740 1 -- start start 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.004+0000 7f0b89c5b740 1 Processor -- start 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb481b0 0x558ebbf226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 0x558ebbf22bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb71070 0x558ebbf2e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558ebbcdf700 con 0x558ebbb50390 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558ebbe03cf0 con 0x558ebbb71070 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558ebbf25330 con 0x558ebbb481b0 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b89c5b740 10 client.0 _ll_register_callbacks cb 0x558ebbe29470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb481b0 0x558ebbf226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b7b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 0x558ebbf22bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb481b0 0x558ebbf226b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38370/0 (socket says 172.21.15.32:38370) 2024-08-31T07:26:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b7be00640 1 -- 172.21.15.32:0/1107486511 learned_addr learned my addr 172.21.15.32:0/1107486511 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.005+0000 7f0b81000640 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb71070 0x558ebbf2e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.006+0000 7f0b7be00640 1 -- 172.21.15.32:0/1107486511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb71070 msgr2=0x558ebbf2e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.006+0000 7f0b7be00640 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb71070 0x558ebbf2e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.006+0000 7f0b7be00640 1 -- 172.21.15.32:0/1107486511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 msgr2=0x558ebbf22bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.006+0000 7f0b7be00640 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 0x558ebbf22bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.006+0000 7f0b7be00640 1 -- 172.21.15.32:0/1107486511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ebbe448b0 con 0x558ebbb481b0 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.006+0000 7f0b7be00640 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb481b0 0x558ebbf226b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c6a0 tx=0x7f0b6c0ab820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56bbf51540c03848 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.006+0000 7f0b66000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.006+0000 7f0b66000640 1 -- 172.21.15.32:0/1107486511 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0e3430 con 0x558ebbb481b0 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.006+0000 7f0b66000640 1 -- 172.21.15.32:0/1107486511 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f0b6c0b0440 con 0x558ebbb481b0 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.006+0000 7f0b89c5b740 1 -- 172.21.15.32:0/1107486511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b680054f0 con 0x558ebbb481b0 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b66000640 1 -- 172.21.15.32:0/1107486511 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0b6c0b07b0 con 0x558ebbb481b0 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b89c5b740 10 client.28297 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b89c5b740 1 -- 172.21.15.32:0/1107486511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558ebbf31260 con 0x558ebbb481b0 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b66000640 1 -- 172.21.15.32:0/1107486511 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b680054f0 con 0x558ebbb481b0 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b66000640 1 -- 172.21.15.32:0/1107486511 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f0b6c018b20 con 0x558ebbb481b0 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b89c5b740 10 client.28297 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b89c5b740 10 client.28297 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:26:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b89c5b740 20 client.28297 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558ebbb2f9f0 0x558ebbf346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b89c5b740 4 client.28297 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:26:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.007+0000 7f0b89c5b740 1 -- 172.21.15.32:0/1107486511 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ebbe8f700 con 0x558ebbb2f9f0 2024-08-31T07:26:10.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.008+0000 7f0b7b400640 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558ebbb2f9f0 0x558ebbf346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.008+0000 7f0b7b400640 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558ebbb2f9f0 0x558ebbf346d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51301cee96b7c251 server_cookie=67bba169ffd7994a in_seq=0 out_seq=0 2024-08-31T07:26:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.008+0000 7f0b66000640 10 client.28297 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:26:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b66000640 1 -- 172.21.15.32:0/1107486511 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ebbe8f700 con 0x558ebbb2f9f0 2024-08-31T07:26:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b66000640 10 client.28297 handle_command_reply: tid=0 2024-08-31T07:26:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b89c5b740 1 client.28297 shutdown 2024-08-31T07:26:10.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b89c5b740 1 -- 172.21.15.32:0/1107486511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb481b0 msgr2=0x558ebbf226b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb481b0 0x558ebbf226b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c6a0 tx=0x7f0b6c0ab820 comp rx=0 tx=0).stop 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b89c5b740 1 -- 172.21.15.32:0/1107486511 shutdown_connections 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558ebbb2f9f0 0x558ebbf346d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb71070 0x558ebbf2e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb50390 0x558ebbf22bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/1107486511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb481b0 0x558ebbf226b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b66000640 0 client.28297 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.009+0000 7f0b89c5b740 1 -- 172.21.15.32:0/1107486511 >> 172.21.15.32:0/1107486511 conn(0x558ebbb65460 msgr2=0x558ebbf17b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.010+0000 7f0b89c5b740 1 -- 172.21.15.32:0/1107486511 shutdown_connections 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.010+0000 7f0b89c5b740 1 -- 172.21.15.32:0/1107486511 wait complete. 2024-08-31T07:26:10.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.010+0000 7f0b89c5b740 20 client.28297 trim_cache size 0 max 16384 2024-08-31T07:26:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.025+0000 7f0b89c5b740 1 Processor -- start 2024-08-31T07:26:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.025+0000 7f0b89c5b740 1 -- start start 2024-08-31T07:26:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.025+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf2e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.025+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 0x558ebbf24390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.025+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb71070 0x558ebbf13700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.025+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1b58f0 con 0x558ebbb71070 2024-08-31T07:26:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.025+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1b7550 con 0x558ebbb481b0 2024-08-31T07:26:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.025+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1b5fc0 con 0x558ebbb50390 2024-08-31T07:26:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.025+0000 7f0b81000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb71070 0x558ebbf13700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.025+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf2e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b7b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 0x558ebbf24390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf2e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54384/0 (socket says 172.21.15.32:54384) 2024-08-31T07:26:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b81000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb71070 0x558ebbf13700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54708/0 (socket says 172.21.15.32:54708) 2024-08-31T07:26:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b7be00640 1 -- 172.21.15.32:0/2561478995 learned_addr learned my addr 172.21.15.32:0/2561478995 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b7be00640 1 -- 172.21.15.32:0/2561478995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 msgr2=0x558ebbf24390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b7be00640 1 --2- 172.21.15.32:0/2561478995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 0x558ebbf24390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b7be00640 1 -- 172.21.15.32:0/2561478995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb71070 msgr2=0x558ebbf13700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b7be00640 1 --2- 172.21.15.32:0/2561478995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb71070 0x558ebbf13700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b7be00640 1 -- 172.21.15.32:0/2561478995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ebbf193f0 con 0x558ebbb481b0 2024-08-31T07:26:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b7be00640 1 --2- 172.21.15.32:0/2561478995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf2e520 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c018fd0 tx=0x7f0b6c018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=801fc86d4cc1c3e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b70c00640 1 -- 172.21.15.32:0/2561478995 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0a7960 con 0x558ebbb481b0 2024-08-31T07:26:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b70c00640 1 -- 172.21.15.32:0/2561478995 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b6c0b5260 con 0x558ebbb481b0 2024-08-31T07:26:10.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.026+0000 7f0b70c00640 1 -- 172.21.15.32:0/2561478995 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c097110 con 0x558ebbb481b0 2024-08-31T07:26:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.027+0000 7f0b89c5b740 1 -- 172.21.15.32:0/2561478995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 msgr2=0x558ebbf2e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.027+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/2561478995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf2e520 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c018fd0 tx=0x7f0b6c018420 comp rx=0 tx=0).stop 2024-08-31T07:26:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.027+0000 7f0b89c5b740 1 -- 172.21.15.32:0/2561478995 shutdown_connections 2024-08-31T07:26:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.027+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/2561478995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb71070 0x558ebbf13700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.027+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/2561478995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 0x558ebbf24390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.027+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/2561478995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb481b0 0x558ebbf2e520 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.027+0000 7f0b89c5b740 1 -- 172.21.15.32:0/2561478995 >> 172.21.15.32:0/2561478995 conn(0x558ebbb65460 msgr2=0x558ebbf12ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.027+0000 7f0b89c5b740 1 -- 172.21.15.32:0/2561478995 shutdown_connections 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.027+0000 7f0b89c5b740 1 -- 172.21.15.32:0/2561478995 wait complete. 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.028+0000 7f0b89c5b740 1 -- start start 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.028+0000 7f0b89c5b740 1 Processor -- start 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb481b0 0x558ebbf2aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 0x558ebbf26fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b89c5b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb71070 0x558ebbf274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1b6dc0 con 0x558ebbb481b0 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1b6690 con 0x558ebbb71070 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b89c5b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1b38f0 con 0x558ebbb50390 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b89c5b740 10 client.0 _ll_register_callbacks cb 0x7f0b6c0ab9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb481b0 0x558ebbf2aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b7b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 0x558ebbf26fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b81000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb71070 0x558ebbf274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb481b0 0x558ebbf2aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54722/0 (socket says 172.21.15.32:54722) 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b7be00640 1 -- 172.21.15.32:0/479103066 learned_addr learned my addr 172.21.15.32:0/479103066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b7b400640 1 -- 172.21.15.32:0/479103066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb71070 msgr2=0x558ebbf274f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b7b400640 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb71070 0x558ebbf274f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b7b400640 1 -- 172.21.15.32:0/479103066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb481b0 msgr2=0x558ebbf2aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b7b400640 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb481b0 0x558ebbf2aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.029+0000 7f0b7b400640 1 -- 172.21.15.32:0/479103066 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ebbf27a30 con 0x558ebbb50390 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.030+0000 7f0b7b400640 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 0x558ebbf26fb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0b6802a930 tx=0x7f0b68004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a0dbaebaea03f9a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.030+0000 7f0b66000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.030+0000 7f0b66000640 1 -- 172.21.15.32:0/479103066 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6805c060 con 0x558ebbb50390 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.030+0000 7f0b66000640 1 -- 172.21.15.32:0/479103066 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f0b68003a00 con 0x558ebbb50390 2024-08-31T07:26:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.030+0000 7f0b89c5b740 1 -- 172.21.15.32:0/479103066 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b6c0b5260 con 0x558ebbb50390 2024-08-31T07:26:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.030+0000 7f0b66000640 1 -- 172.21.15.32:0/479103066 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0b68035040 con 0x558ebbb50390 2024-08-31T07:26:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.030+0000 7f0b89c5b740 10 client.28303 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.030+0000 7f0b89c5b740 1 -- 172.21.15.32:0/479103066 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558ebbf9fd30 con 0x558ebbb50390 2024-08-31T07:26:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.031+0000 7f0b66000640 1 -- 172.21.15.32:0/479103066 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b6c0b5260 con 0x558ebbb50390 2024-08-31T07:26:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.031+0000 7f0b66000640 1 -- 172.21.15.32:0/479103066 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f0b68031b50 con 0x558ebbb50390 2024-08-31T07:26:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.031+0000 7f0b89c5b740 10 client.28303 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.031+0000 7f0b89c5b740 10 client.28303 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:26:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.031+0000 7f0b89c5b740 20 client.28303 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.031+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558ebbb04da0 0x558ebbfa20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.031+0000 7f0b89c5b740 4 client.28303 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.031+0000 7f0b89c5b740 1 -- 172.21.15.32:0/479103066 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ebbe8b1a0 con 0x558ebbb04da0 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.032+0000 7f0b7be00640 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558ebbb04da0 0x558ebbfa20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.032+0000 7f0b7be00640 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558ebbb04da0 0x558ebbfa20d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3831c765b7a579d server_cookie=e171ce58678066d2 in_seq=0 out_seq=0 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.032+0000 7f0b66000640 10 client.28303 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.032+0000 7f0b66000640 1 -- 172.21.15.32:0/479103066 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ebbe8b1a0 con 0x558ebbb04da0 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.032+0000 7f0b66000640 10 client.28303 handle_command_reply: tid=0 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.032+0000 7f0b89c5b740 1 client.28303 shutdown 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 1 -- 172.21.15.32:0/479103066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 msgr2=0x558ebbf26fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 0x558ebbf26fb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0b6802a930 tx=0x7f0b68004ab0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 1 -- 172.21.15.32:0/479103066 shutdown_connections 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x558ebbb04da0 0x558ebbfa20d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ebbb71070 0x558ebbf274f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ebbb50390 0x558ebbf26fb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 1 --2- 172.21.15.32:0/479103066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ebbb481b0 0x558ebbf2aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b66000640 0 client.28303 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:26:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 1 -- 172.21.15.32:0/479103066 >> 172.21.15.32:0/479103066 conn(0x558ebbb65460 msgr2=0x558ebbf12ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 1 -- 172.21.15.32:0/479103066 shutdown_connections 2024-08-31T07:26:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 1 -- 172.21.15.32:0/479103066 wait complete. 2024-08-31T07:26:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.033+0000 7f0b89c5b740 20 client.28303 trim_cache size 0 max 16384 2024-08-31T07:26:10.037 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:26:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 -- 172.21.15.32:0/430343054 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0b600b4ca0 msgr2=0x7f0b600b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0b600b4ca0 0x7f0b600b71b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009d40 tx=0x7f0b74009500 comp rx=0 tx=0).stop 2024-08-31T07:26:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 -- 172.21.15.32:0/430343054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b7c0776e0 msgr2=0x7f0b7c1a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b7c0776e0 0x7f0b7c1a0710 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00e9f0 tx=0x7f0b6c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 -- 172.21.15.32:0/430343054 shutdown_connections 2024-08-31T07:26:10.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0b600b4ca0 0x7f0b600b71b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b7c114310 0x7f0b7c1a5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b7c078030 0x7f0b7c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 --2- 172.21.15.32:0/430343054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b7c0776e0 0x7f0b7c1a0710 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 -- 172.21.15.32:0/430343054 >> 172.21.15.32:0/430343054 conn(0x7f0b7c101420 msgr2=0x7f0b7c1163c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 -- 172.21.15.32:0/430343054 shutdown_connections 2024-08-31T07:26:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:10.034+0000 7f0b81a00640 1 -- 172.21.15.32:0/430343054 wait complete. 2024-08-31T07:26:10.785 INFO:tasks.workunit.client.0.smithi032.stdout: 1 408304 2.74 MB/sec execute 599 sec latency 4876.459 ms 2024-08-31T07:26:10.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:10 smithi032 ceph-mon[17252]: pgmap v785: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 878 KiB/s wr, 108 op/s 2024-08-31T07:26:10.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:26:10.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:10 smithi084 ceph-mon[22387]: pgmap v785: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 878 KiB/s wr, 108 op/s 2024-08-31T07:26:11.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:10 smithi073 ceph-mon[21995]: pgmap v785: 97 pgs: 97 active+clean; 347 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 878 KiB/s wr, 108 op/s 2024-08-31T07:26:11.329 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:11.785 INFO:tasks.workunit.client.0.smithi032.stdout: 1 cleanup 600 sec 2024-08-31T07:26:11.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.970+0000 7f18f5000640 1 -- 172.21.15.32:0/1499755457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f0075550 msgr2=0x7f18f0075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.970+0000 7f18f5000640 1 --2- 172.21.15.32:0/1499755457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f0075550 0x7f18f0075960 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009900 tx=0x7f18e002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.970+0000 7f18f5000640 1 -- 172.21.15.32:0/1499755457 shutdown_connections 2024-08-31T07:26:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.970+0000 7f18f5000640 1 --2- 172.21.15.32:0/1499755457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18f010fcb0 0x7f18f01120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.970+0000 7f18f5000640 1 --2- 172.21.15.32:0/1499755457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18f0075ea0 0x7f18f010f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.970+0000 7f18f5000640 1 --2- 172.21.15.32:0/1499755457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f0075550 0x7f18f0075960 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.970+0000 7f18f5000640 1 -- 172.21.15.32:0/1499755457 >> 172.21.15.32:0/1499755457 conn(0x7f18f00ff1e0 msgr2=0x7f18f0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.970+0000 7f18f5000640 1 -- 172.21.15.32:0/1499755457 shutdown_connections 2024-08-31T07:26:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.971+0000 7f18f5000640 1 -- 172.21.15.32:0/1499755457 wait complete. 2024-08-31T07:26:11.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.971+0000 7f18f5000640 1 Processor -- start 2024-08-31T07:26:11.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18f5000640 1 -- start start 2024-08-31T07:26:11.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18f5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18f0075550 0x7f18f019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18f5000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18f0075ea0 0x7f18f019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18f5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f010fcb0 0x7f18f01a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18f5000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01158b0 con 0x7f18f010fcb0 2024-08-31T07:26:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18f5000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0115730 con 0x7f18f0075550 2024-08-31T07:26:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18f5000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0115a30 con 0x7f18f0075ea0 2024-08-31T07:26:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18f0075550 0x7f18f019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18f0075550 0x7f18f019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54402/0 (socket says 172.21.15.32:54402) 2024-08-31T07:26:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18ef400640 1 -- 172.21.15.32:0/1549988826 learned_addr learned my addr 172.21.15.32:0/1549988826 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.972+0000 7f18efe00640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f010fcb0 0x7f18f01a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:11.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.973+0000 7f18eea00640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18f0075ea0 0x7f18f019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:11.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.973+0000 7f18ef400640 1 -- 172.21.15.32:0/1549988826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18f0075ea0 msgr2=0x7f18f019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:11.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.973+0000 7f18ef400640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18f0075ea0 0x7f18f019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.973+0000 7f18ef400640 1 -- 172.21.15.32:0/1549988826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f010fcb0 msgr2=0x7f18f01a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:11.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.973+0000 7f18ef400640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f010fcb0 0x7f18f01a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.973+0000 7f18ef400640 1 -- 172.21.15.32:0/1549988826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f01a2210 con 0x7f18f0075550 2024-08-31T07:26:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.973+0000 7f18ef400640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18f0075550 0x7f18f019c2a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005ab0 tx=0x7f18e0003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.974+0000 7f18e7e00640 1 -- 172.21.15.32:0/1549988826 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e003f070 con 0x7f18f0075550 2024-08-31T07:26:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.974+0000 7f18f5000640 1 -- 172.21.15.32:0/1549988826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f01a24a0 con 0x7f18f0075550 2024-08-31T07:26:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.974+0000 7f18f5000640 1 -- 172.21.15.32:0/1549988826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f01a8c60 con 0x7f18f0075550 2024-08-31T07:26:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.974+0000 7f18e7e00640 1 -- 172.21.15.32:0/1549988826 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e00054d0 con 0x7f18f0075550 2024-08-31T07:26:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.974+0000 7f18e7e00640 1 -- 172.21.15.32:0/1549988826 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0049060 con 0x7f18f0075550 2024-08-31T07:26:11.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.977+0000 7f18fd1e4740 1 Processor -- start 2024-08-31T07:26:11.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.977+0000 7f18fd1e4740 1 -- start start 2024-08-31T07:26:11.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.977+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccdc070 0x55ea1d081b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.977+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccb31b0 0x55ea1d0820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.977+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 0x55ea1d0825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.977+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ea1cff87b0 con 0x55ea1ccb31b0 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.977+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ea1cfe7d40 con 0x55ea1ccdc070 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.977+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ea1cfebcf0 con 0x55ea1ccbb390 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.978+0000 7f18efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccdc070 0x55ea1d081b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.978+0000 7f18efe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccdc070 0x55ea1d081b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54404/0 (socket says 172.21.15.32:54404) 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.978+0000 7f18efe00640 1 -- 172.21.15.32:0/1730949369 learned_addr learned my addr 172.21.15.32:0/1730949369 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.978+0000 7f18eea00640 1 --2- 172.21.15.32:0/1730949369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 0x55ea1d0825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.978+0000 7f18e7e00640 1 -- 172.21.15.32:0/1549988826 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f18e0042070 con 0x7f18f0075550 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.978+0000 7f18efe00640 1 -- 172.21.15.32:0/1730949369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 msgr2=0x55ea1d0825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.978+0000 7f18efe00640 1 --2- 172.21.15.32:0/1730949369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 0x55ea1d0825e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.978+0000 7f18efe00640 1 -- 172.21.15.32:0/1730949369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccb31b0 msgr2=0x55ea1d0820a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.978+0000 7f18efe00640 1 --2- 172.21.15.32:0/1730949369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccb31b0 0x55ea1d0820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.978+0000 7f18efe00640 1 -- 172.21.15.32:0/1730949369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1cd47650 con 0x55ea1ccdc070 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.979+0000 7f18efe00640 1 --2- 172.21.15.32:0/1730949369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccdc070 0x55ea1d081b60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f18e80039f0 tx=0x7f18e8003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c1baf532aba8240 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.979+0000 7f18e4c00640 1 -- 172.21.15.32:0/1730949369 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e801a470 con 0x55ea1ccdc070 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.979+0000 7f18e4c00640 1 -- 172.21.15.32:0/1730949369 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e8015070 con 0x55ea1ccdc070 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.979+0000 7f18e4c00640 1 -- 172.21.15.32:0/1730949369 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e80107e0 con 0x55ea1ccdc070 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18e7e00640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18d40b4d20 0x7f18d40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1730949369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccdc070 msgr2=0x55ea1d081b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/1730949369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccdc070 0x55ea1d081b60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f18e80039f0 tx=0x7f18e8003eb0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18e7e00640 1 -- 172.21.15.32:0/1549988826 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f18e01057b0 con 0x7f18f0075550 2024-08-31T07:26:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1730949369 shutdown_connections 2024-08-31T07:26:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/1730949369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 0x55ea1d0825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/1730949369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccb31b0 0x55ea1d0820a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/1730949369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccdc070 0x55ea1d081b60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18ef400640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18d40b4d20 0x7f18d40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1730949369 >> 172.21.15.32:0/1730949369 conn(0x55ea1ccd0460 msgr2=0x55ea1d07fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1730949369 shutdown_connections 2024-08-31T07:26:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.980+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1730949369 wait complete. 2024-08-31T07:26:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.981+0000 7f18fd1e4740 1 -- start start 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.981+0000 7f18ef400640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18d40b4d20 0x7f18d40b7230 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x55ea1d0807e0 tx=0x7f18e0004a00 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.981+0000 7f18fd1e4740 1 Processor -- start 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccb31b0 0x55ea1d08d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 0x55ea1d08dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d0996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ea1ce4a700 con 0x55ea1ccdc070 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ea1cf6ecf0 con 0x55ea1ccb31b0 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ea1d090340 con 0x55ea1ccbb390 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18fd1e4740 10 client.0 _ll_register_callbacks cb 0x55ea1cf94470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18eea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 0x55ea1d08dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d0996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18eea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 0x55ea1d08dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38420/0 (socket says 172.21.15.32:38420) 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18eea00640 1 -- 172.21.15.32:0/2909573874 learned_addr learned my addr 172.21.15.32:0/2909573874 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18efe00640 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccb31b0 0x55ea1d08d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18eea00640 1 -- 172.21.15.32:0/2909573874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccb31b0 msgr2=0x55ea1d08d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18eea00640 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccb31b0 0x55ea1d08d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18eea00640 1 -- 172.21.15.32:0/2909573874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 msgr2=0x55ea1d0996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18eea00640 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d0996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.982+0000 7f18eea00640 1 -- 172.21.15.32:0/2909573874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1cfaf8b0 con 0x55ea1ccbb390 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.983+0000 7f18eea00640 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 0x55ea1d08dc40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f18dc012700 tx=0x7f18dc012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70754f1f8967b934 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.983+0000 7f18d9600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.983+0000 7f18d9600640 1 -- 172.21.15.32:0/2909573874 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc004050 con 0x55ea1ccbb390 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.983+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2909573874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e8015070 con 0x55ea1ccbb390 2024-08-31T07:26:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.983+0000 7f18d9600640 1 -- 172.21.15.32:0/2909573874 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f18dc0041f0 con 0x55ea1ccbb390 2024-08-31T07:26:11.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.983+0000 7f18d9600640 1 -- 172.21.15.32:0/2909573874 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f18dc0043f0 con 0x55ea1ccbb390 2024-08-31T07:26:11.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.983+0000 7f18fd1e4740 10 client.28315 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:11.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.983+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2909573874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ea1d09c3f0 con 0x55ea1ccbb390 2024-08-31T07:26:11.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.984+0000 7f18d9600640 1 -- 172.21.15.32:0/2909573874 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e8015070 con 0x55ea1ccbb390 2024-08-31T07:26:11.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.984+0000 7f18d9600640 1 -- 172.21.15.32:0/2909573874 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f18dc003b40 con 0x55ea1ccbb390 2024-08-31T07:26:11.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.984+0000 7f18fd1e4740 10 client.28315 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.984+0000 7f18fd1e4740 10 client.28315 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.984+0000 7f18fd1e4740 20 client.28315 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.984+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ea1cc9a9f0 0x55ea1d09f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.984+0000 7f18fd1e4740 4 client.28315 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.984+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2909573874 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea1cffa700 con 0x55ea1cc9a9f0 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.984+0000 7f18efe00640 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ea1cc9a9f0 0x55ea1d09f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.985+0000 7f18efe00640 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ea1cc9a9f0 0x55ea1d09f9e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b635cc3f331450f3 server_cookie=649f3daa92994c3 in_seq=0 out_seq=0 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.985+0000 7f18d9600640 10 client.28315 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18d9600640 1 -- 172.21.15.32:0/2909573874 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ea1cffa700 con 0x55ea1cc9a9f0 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18d9600640 10 client.28315 handle_command_reply: tid=0 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 client.28315 shutdown 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2909573874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 msgr2=0x55ea1d08dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 0x55ea1d08dc40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f18dc012700 tx=0x7f18dc012bc0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2909573874 shutdown_connections 2024-08-31T07:26:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d0996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccbb390 0x55ea1d08dc40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ea1cc9a9f0 0x55ea1d09f9e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/2909573874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccb31b0 0x55ea1d08d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2909573874 >> 172.21.15.32:0/2909573874 conn(0x55ea1ccd0460 msgr2=0x55ea1d084cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18d9600640 0 client.28315 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:26:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2909573874 shutdown_connections 2024-08-31T07:26:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2909573874 wait complete. 2024-08-31T07:26:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:11.986+0000 7f18fd1e4740 20 client.28315 trim_cache size 0 max 16384 2024-08-31T07:26:12.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.001+0000 7f18fd1e4740 1 Processor -- start 2024-08-31T07:26:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.001+0000 7f18fd1e4740 1 -- start start 2024-08-31T07:26:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.001+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccb31b0 0x55ea1d0995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.001+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d099b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.001+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d086c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.001+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18d000e740 con 0x55ea1ccdc070 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.001+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f18d00103a0 con 0x55ea1ccbb390 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.001+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f18d000ee10 con 0x55ea1ccb31b0 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18eea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d099b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18efe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccb31b0 0x55ea1d0995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18eea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d099b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54426/0 (socket says 172.21.15.32:54426) 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18eea00640 1 -- 172.21.15.32:0/2153518868 learned_addr learned my addr 172.21.15.32:0/2153518868 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18efe00640 1 --2- 172.21.15.32:0/2153518868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d086c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18eea00640 1 -- 172.21.15.32:0/2153518868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccb31b0 msgr2=0x55ea1d0995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18eea00640 1 --2- 172.21.15.32:0/2153518868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccb31b0 0x55ea1d0995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18eea00640 1 -- 172.21.15.32:0/2153518868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 msgr2=0x55ea1d086c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18eea00640 1 --2- 172.21.15.32:0/2153518868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d086c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18eea00640 1 -- 172.21.15.32:0/2153518868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1d0890b0 con 0x55ea1ccbb390 2024-08-31T07:26:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18efe00640 1 --2- 172.21.15.32:0/2153518868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d086c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:26:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.002+0000 7f18eea00640 1 --2- 172.21.15.32:0/2153518868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d099b00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f18dc003f60 tx=0x7f18dc010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac74759171d62ea5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.003+0000 7f18dbe00640 1 -- 172.21.15.32:0/2153518868 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc029040 con 0x55ea1ccbb390 2024-08-31T07:26:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.003+0000 7f18dbe00640 1 -- 172.21.15.32:0/2153518868 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18dc004060 con 0x55ea1ccbb390 2024-08-31T07:26:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.003+0000 7f18dbe00640 1 -- 172.21.15.32:0/2153518868 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00f790 con 0x55ea1ccbb390 2024-08-31T07:26:12.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.004+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2153518868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 msgr2=0x55ea1d099b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:12.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.004+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/2153518868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d099b00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f18dc003f60 tx=0x7f18dc010830 comp rx=0 tx=0).stop 2024-08-31T07:26:12.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.004+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2153518868 shutdown_connections 2024-08-31T07:26:12.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.004+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/2153518868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d086c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.004+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/2153518868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d099b00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.004+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/2153518868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccb31b0 0x55ea1d0995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.004+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2153518868 >> 172.21.15.32:0/2153518868 conn(0x55ea1ccd0460 msgr2=0x55ea1d08bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.004+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2153518868 shutdown_connections 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.004+0000 7f18fd1e4740 1 -- 172.21.15.32:0/2153518868 wait complete. 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.004+0000 7f18fd1e4740 1 -- start start 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.005+0000 7f18fd1e4740 1 Processor -- start 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.005+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccb31b0 0x55ea1d091e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.005+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d092460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.005+0000 7f18fd1e4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d109380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.005+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18d000fc10 con 0x55ea1ccdc070 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.005+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f18d000f4e0 con 0x55ea1ccbb390 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.005+0000 7f18fd1e4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f18d000c740 con 0x55ea1ccb31b0 2024-08-31T07:26:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.005+0000 7f18fd1e4740 10 client.0 _ll_register_callbacks cb 0x7f18dc009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18efe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccb31b0 0x55ea1d091e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d109380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54780/0 (socket says 172.21.15.32:54780) 2024-08-31T07:26:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18efe00640 1 -- 172.21.15.32:0/1233925424 learned_addr learned my addr 172.21.15.32:0/1233925424 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18eea00640 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d092460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:12.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18eea00640 1 -- 172.21.15.32:0/1233925424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccb31b0 msgr2=0x55ea1d091e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18eea00640 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccb31b0 0x55ea1d091e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18eea00640 1 -- 172.21.15.32:0/1233925424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 msgr2=0x55ea1d109380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18eea00640 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d109380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18eea00640 1 -- 172.21.15.32:0/1233925424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1d1099f0 con 0x55ea1ccbb390 2024-08-31T07:26:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.006+0000 7f18efe00640 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.007+0000 7f18eea00640 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d092460 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55ea1d09a7f0 tx=0x7f18dc010620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d07d40a05c4a363e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.007+0000 7f18d9600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:26:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.007+0000 7f18d9600640 1 -- 172.21.15.32:0/1233925424 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc029070 con 0x55ea1ccbb390 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.007+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1233925424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18dc004060 con 0x55ea1ccbb390 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.007+0000 7f18d9600640 1 -- 172.21.15.32:0/1233925424 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f18dc010d20 con 0x55ea1ccbb390 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.007+0000 7f18d9600640 1 -- 172.21.15.32:0/1233925424 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f18dc00f790 con 0x55ea1ccbb390 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.007+0000 7f18fd1e4740 10 client.28316 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.007+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1233925424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ea1d10ac70 con 0x55ea1ccbb390 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.008+0000 7f18d9600640 1 -- 172.21.15.32:0/1233925424 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18dc004060 con 0x55ea1ccbb390 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.008+0000 7f18d9600640 1 -- 172.21.15.32:0/1233925424 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f18dc036060 con 0x55ea1ccbb390 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.008+0000 7f18fd1e4740 10 client.28316 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.008+0000 7f18fd1e4740 10 client.28316 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.008+0000 7f18fd1e4740 20 client.28316 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.008+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ea1cc6fda0 0x55ea1d10d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.008+0000 7f18fd1e4740 4 client.28316 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.008+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1233925424 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea1cff61a0 con 0x55ea1cc6fda0 2024-08-31T07:26:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.008+0000 7f18efe00640 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ea1cc6fda0 0x55ea1d10d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.009+0000 7f18efe00640 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ea1cc6fda0 0x55ea1d10d4a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92133bb891210e36 server_cookie=2b241d14735294fa in_seq=0 out_seq=0 2024-08-31T07:26:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.009+0000 7f18d9600640 10 client.28316 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:26:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.009+0000 7f18d9600640 1 -- 172.21.15.32:0/1233925424 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea1cff61a0 con 0x55ea1cc6fda0 2024-08-31T07:26:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.009+0000 7f18d9600640 10 client.28316 handle_command_reply: tid=0 2024-08-31T07:26:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.009+0000 7f18fd1e4740 1 client.28316 shutdown 2024-08-31T07:26:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.009+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1233925424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 msgr2=0x55ea1d092460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.009+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d092460 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55ea1d09a7f0 tx=0x7f18dc010620 comp rx=0 tx=0).stop 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.010+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1233925424 shutdown_connections 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.010+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea1ccdc070 0x55ea1d109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.010+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ea1cc6fda0 0x55ea1d10d4a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.010+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1ccbb390 0x55ea1d092460 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.010+0000 7f18fd1e4740 1 --2- 172.21.15.32:0/1233925424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1ccb31b0 0x55ea1d091e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.010+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1233925424 >> 172.21.15.32:0/1233925424 conn(0x55ea1ccd0460 msgr2=0x55ea1d086c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.010+0000 7f18d9600640 0 client.28316 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.010+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1233925424 shutdown_connections 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.010+0000 7f18fd1e4740 1 -- 172.21.15.32:0/1233925424 wait complete. 2024-08-31T07:26:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.010+0000 7f18fd1e4740 20 client.28316 trim_cache size 0 max 16384 2024-08-31T07:26:12.015 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:26:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 -- 172.21.15.32:0/1549988826 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18d40b4d20 msgr2=0x7f18d40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18d40b4d20 0x7f18d40b7230 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x55ea1d0807e0 tx=0x7f18e0004a00 comp rx=0 tx=0).stop 2024-08-31T07:26:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 -- 172.21.15.32:0/1549988826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18f0075550 msgr2=0x7f18f019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18f0075550 0x7f18f019c2a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005ab0 tx=0x7f18e0003c80 comp rx=0 tx=0).stop 2024-08-31T07:26:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 -- 172.21.15.32:0/1549988826 shutdown_connections 2024-08-31T07:26:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18d40b4d20 0x7f18d40b7230 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f010fcb0 0x7f18f01a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18f0075ea0 0x7f18f019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 --2- 172.21.15.32:0/1549988826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18f0075550 0x7f18f019c2a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 -- 172.21.15.32:0/1549988826 >> 172.21.15.32:0/1549988826 conn(0x7f18f00ff1e0 msgr2=0x7f18f0109850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 -- 172.21.15.32:0/1549988826 shutdown_connections 2024-08-31T07:26:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:12.011+0000 7f18f5000640 1 -- 172.21.15.32:0/1549988826 wait complete. 2024-08-31T07:26:12.785 INFO:tasks.workunit.client.0.smithi032.stdout: 1 cleanup 601 sec 2024-08-31T07:26:12.790 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:26:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:12 smithi084 ceph-mon[22387]: pgmap v786: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 986 KiB/s wr, 103 op/s 2024-08-31T07:26:13.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:12 smithi032 ceph-mon[17252]: pgmap v786: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 986 KiB/s wr, 103 op/s 2024-08-31T07:26:13.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:12 smithi073 ceph-mon[21995]: pgmap v786: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 986 KiB/s wr, 103 op/s 2024-08-31T07:26:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:13.785 INFO:tasks.workunit.client.0.smithi032.stdout: 1 cleanup 602 sec 2024-08-31T07:26:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.004+0000 7f083b800640 1 -- 172.21.15.32:0/2507417272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08340754f0 msgr2=0x7f0834075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.004+0000 7f083b800640 1 --2- 172.21.15.32:0/2507417272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08340754f0 0x7f0834075900 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0828009900 tx=0x7f082802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.005+0000 7f083b800640 1 -- 172.21.15.32:0/2507417272 shutdown_connections 2024-08-31T07:26:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.005+0000 7f083b800640 1 --2- 172.21.15.32:0/2507417272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083410fec0 0x7f08341122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.005+0000 7f083b800640 1 --2- 172.21.15.32:0/2507417272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0834075e40 0x7f083410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.005+0000 7f083b800640 1 --2- 172.21.15.32:0/2507417272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08340754f0 0x7f0834075900 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.005+0000 7f083b800640 1 -- 172.21.15.32:0/2507417272 >> 172.21.15.32:0/2507417272 conn(0x7f083407b5f0 msgr2=0x7f083407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.005+0000 7f083b800640 1 -- 172.21.15.32:0/2507417272 shutdown_connections 2024-08-31T07:26:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.005+0000 7f083b800640 1 -- 172.21.15.32:0/2507417272 wait complete. 2024-08-31T07:26:14.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.006+0000 7f083b800640 1 Processor -- start 2024-08-31T07:26:14.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.006+0000 7f083b800640 1 -- start start 2024-08-31T07:26:14.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.006+0000 7f083b800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08340754f0 0x7f083419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.006+0000 7f083b800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0834075e40 0x7f083419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.006+0000 7f083b800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083410fec0 0x7f08341a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.006+0000 7f083b800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0834115a80 con 0x7f083410fec0 2024-08-31T07:26:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.006+0000 7f083b800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0834115900 con 0x7f0834075e40 2024-08-31T07:26:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.006+0000 7f083b800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0834115c00 con 0x7f08340754f0 2024-08-31T07:26:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0839800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083410fec0 0x7f08341a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0839800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083410fec0 0x7f08341a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54790/0 (socket says 172.21.15.32:54790) 2024-08-31T07:26:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0839800640 1 -- 172.21.15.32:0/2476734411 learned_addr learned my addr 172.21.15.32:0/2476734411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0838e00640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08340754f0 0x7f083419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0833e00640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0834075e40 0x7f083419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0838e00640 1 -- 172.21.15.32:0/2476734411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0834075e40 msgr2=0x7f083419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0838e00640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0834075e40 0x7f083419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0838e00640 1 -- 172.21.15.32:0/2476734411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083410fec0 msgr2=0x7f08341a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0838e00640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083410fec0 0x7f08341a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0838e00640 1 -- 172.21.15.32:0/2476734411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08341a2440 con 0x7f08340754f0 2024-08-31T07:26:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.007+0000 7f0839800640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083410fec0 0x7f08341a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.008+0000 7f0838e00640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08340754f0 0x7f083419c4d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f082802a9e0 tx=0x7f0828003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.008+0000 7f0831600640 1 -- 172.21.15.32:0/2476734411 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082803f070 con 0x7f08340754f0 2024-08-31T07:26:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.008+0000 7f0831600640 1 -- 172.21.15.32:0/2476734411 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0828031b50 con 0x7f08340754f0 2024-08-31T07:26:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.008+0000 7f083b800640 1 -- 172.21.15.32:0/2476734411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08341a26d0 con 0x7f08340754f0 2024-08-31T07:26:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.008+0000 7f083b800640 1 -- 172.21.15.32:0/2476734411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08341a8e90 con 0x7f08340754f0 2024-08-31T07:26:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.008+0000 7f0831600640 1 -- 172.21.15.32:0/2476734411 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828035030 con 0x7f08340754f0 2024-08-31T07:26:14.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.011+0000 7f08424d8740 1 Processor -- start 2024-08-31T07:26:14.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.011+0000 7f08424d8740 1 -- start start 2024-08-31T07:26:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f08424d8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 0x558303d2d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f08424d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55830395f1b0 0x558303d2ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f08424d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d2e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f08424d8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558303ca47b0 con 0x558303988070 2024-08-31T07:26:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f08424d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558303c93d40 con 0x558303967390 2024-08-31T07:26:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f08424d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558303c97cf0 con 0x55830395f1b0 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f0839800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 0x558303d2d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f0839800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 0x558303d2d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54800/0 (socket says 172.21.15.32:54800) 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f0839800640 1 -- 172.21.15.32:0/402934075 learned_addr learned my addr 172.21.15.32:0/402934075 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f0833e00640 1 --2- 172.21.15.32:0/402934075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d2e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f0831600640 1 -- 172.21.15.32:0/2476734411 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0828042070 con 0x7f08340754f0 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f0839800640 1 -- 172.21.15.32:0/402934075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55830395f1b0 msgr2=0x558303d2ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f0839800640 1 --2- 172.21.15.32:0/402934075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55830395f1b0 0x558303d2ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.012+0000 7f0839800640 1 -- 172.21.15.32:0/402934075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 msgr2=0x558303d2e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.013+0000 7f0839800640 1 --2- 172.21.15.32:0/402934075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d2e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.013+0000 7f0839800640 1 -- 172.21.15.32:0/402934075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583039f3650 con 0x558303988070 2024-08-31T07:26:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.013+0000 7f0833e00640 1 --2- 172.21.15.32:0/402934075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d2e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.013+0000 7f0839800640 1 --2- 172.21.15.32:0/402934075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 0x558303d2d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f082c00e830 tx=0x7f082c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1747937b5b292dbb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.013+0000 7f0826000640 1 -- 172.21.15.32:0/402934075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c004d20 con 0x558303988070 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.013+0000 7f0826000640 1 -- 172.21.15.32:0/402934075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f082c011e50 con 0x558303988070 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.013+0000 7f0826000640 1 -- 172.21.15.32:0/402934075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c00f3c0 con 0x558303988070 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.014+0000 7f08424d8740 1 -- 172.21.15.32:0/402934075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 msgr2=0x558303d2d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.014+0000 7f08424d8740 1 --2- 172.21.15.32:0/402934075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 0x558303d2d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f082c00e830 tx=0x7f082c00ecf0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.014+0000 7f0831600640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f081c0b4d20 0x7f081c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.014+0000 7f08424d8740 1 -- 172.21.15.32:0/402934075 shutdown_connections 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.014+0000 7f08424d8740 1 --2- 172.21.15.32:0/402934075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d2e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.014+0000 7f0831600640 1 -- 172.21.15.32:0/2476734411 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f0828105a40 con 0x7f08340754f0 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.014+0000 7f08424d8740 1 --2- 172.21.15.32:0/402934075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55830395f1b0 0x558303d2ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.014+0000 7f08424d8740 1 --2- 172.21.15.32:0/402934075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 0x558303d2d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.015+0000 7f08424d8740 1 -- 172.21.15.32:0/402934075 >> 172.21.15.32:0/402934075 conn(0x55830397c460 msgr2=0x558303d2b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.015+0000 7f0839800640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f081c0b4d20 0x7f081c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.015+0000 7f08424d8740 1 -- 172.21.15.32:0/402934075 shutdown_connections 2024-08-31T07:26:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.015+0000 7f08424d8740 1 -- 172.21.15.32:0/402934075 wait complete. 2024-08-31T07:26:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.016+0000 7f0839800640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f081c0b4d20 0x7f081c0b7230 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f082c005650 tx=0x7f082c0054b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.016+0000 7f08424d8740 1 -- start start 2024-08-31T07:26:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.016+0000 7f08424d8740 1 Processor -- start 2024-08-31T07:26:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.016+0000 7f08424d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55830395f1b0 0x558303d39640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.016+0000 7f08424d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d39c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.016+0000 7f08424d8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 0x558303d454c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.016+0000 7f08424d8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558303af6700 con 0x558303988070 2024-08-31T07:26:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.016+0000 7f08424d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558303c1acf0 con 0x558303967390 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.016+0000 7f08424d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558303d3c170 con 0x55830395f1b0 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f08424d8740 10 client.0 _ll_register_callbacks cb 0x558303c40470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f0838e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55830395f1b0 0x558303d39640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f0839800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 0x558303d454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f0838e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55830395f1b0 0x558303d39640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38460/0 (socket says 172.21.15.32:38460) 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f0838e00640 1 -- 172.21.15.32:0/3834774599 learned_addr learned my addr 172.21.15.32:0/3834774599 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f0833e00640 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d39c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f0833e00640 1 -- 172.21.15.32:0/3834774599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55830395f1b0 msgr2=0x558303d39640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f0833e00640 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55830395f1b0 0x558303d39640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f0833e00640 1 -- 172.21.15.32:0/3834774599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 msgr2=0x558303d454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f0833e00640 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 0x558303d454c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.017+0000 7f0833e00640 1 -- 172.21.15.32:0/3834774599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558303c5b8b0 con 0x558303967390 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.018+0000 7f0833e00640 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d39c00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f083406c790 tx=0x7f0820012aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649a34937d7c5c93 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.018+0000 7f081aa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:26:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.018+0000 7f081aa00640 1 -- 172.21.15.32:0/3834774599 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820004030 con 0x558303967390 2024-08-31T07:26:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.018+0000 7f08424d8740 1 -- 172.21.15.32:0/3834774599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f082c011e50 con 0x558303967390 2024-08-31T07:26:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.018+0000 7f081aa00640 1 -- 172.21.15.32:0/3834774599 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f08200041d0 con 0x558303967390 2024-08-31T07:26:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.018+0000 7f081aa00640 1 -- 172.21.15.32:0/3834774599 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f08200043d0 con 0x558303967390 2024-08-31T07:26:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.018+0000 7f08424d8740 10 client.28328 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.018+0000 7f08424d8740 1 -- 172.21.15.32:0/3834774599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558303d48320 con 0x558303967390 2024-08-31T07:26:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.018+0000 7f081aa00640 1 -- 172.21.15.32:0/3834774599 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f082c011e50 con 0x558303967390 2024-08-31T07:26:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.019+0000 7f081aa00640 1 -- 172.21.15.32:0/3834774599 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f0820023020 con 0x558303967390 2024-08-31T07:26:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.019+0000 7f08424d8740 10 client.28328 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.019+0000 7f08424d8740 10 client.28328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:26:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.019+0000 7f08424d8740 20 client.28328 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.019+0000 7f08424d8740 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5583039469f0 0x558303d4b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.019+0000 7f08424d8740 4 client.28328 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:26:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.019+0000 7f08424d8740 1 -- 172.21.15.32:0/3834774599 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558303ca6700 con 0x5583039469f0 2024-08-31T07:26:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.019+0000 7f0838e00640 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5583039469f0 0x558303d4b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.020+0000 7f0838e00640 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5583039469f0 0x558303d4b780 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c38799e0e8f6045 server_cookie=b4b7076165b1cd3e in_seq=0 out_seq=0 2024-08-31T07:26:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.020+0000 7f081aa00640 10 client.28328 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:14.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f081aa00640 1 -- 172.21.15.32:0/3834774599 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558303ca6700 con 0x5583039469f0 2024-08-31T07:26:14.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f081aa00640 10 client.28328 handle_command_reply: tid=0 2024-08-31T07:26:14.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 client.28328 shutdown 2024-08-31T07:26:14.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 -- 172.21.15.32:0/3834774599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 msgr2=0x558303d39c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:14.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d39c00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f083406c790 tx=0x7f0820012aa0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 -- 172.21.15.32:0/3834774599 shutdown_connections 2024-08-31T07:26:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5583039469f0 0x558303d4b780 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558303988070 0x558303d454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d39c00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f081aa00640 0 client.28328 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 --2- 172.21.15.32:0/3834774599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55830395f1b0 0x558303d39640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 -- 172.21.15.32:0/3834774599 >> 172.21.15.32:0/3834774599 conn(0x55830397c460 msgr2=0x558303d30b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 -- 172.21.15.32:0/3834774599 shutdown_connections 2024-08-31T07:26:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 1 -- 172.21.15.32:0/3834774599 wait complete. 2024-08-31T07:26:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.021+0000 7f08424d8740 20 client.28328 trim_cache size 0 max 16384 2024-08-31T07:26:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.035+0000 7f08424d8740 1 Processor -- start 2024-08-31T07:26:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f08424d8740 1 -- start start 2024-08-31T07:26:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f08424d8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d39d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f08424d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d32aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f08424d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 0x558303d32fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f08424d8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08341b16b0 con 0x55830395f1b0 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f08424d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f08341b3310 con 0x558303967390 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f08424d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f08341b1d80 con 0x558303988070 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f0838e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d39d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f0833e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d32aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f0839800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 0x558303d32fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f0838e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d39d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54822/0 (socket says 172.21.15.32:54822) 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.036+0000 7f0838e00640 1 -- 172.21.15.32:0/238512002 learned_addr learned my addr 172.21.15.32:0/238512002 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.037+0000 7f0838e00640 1 -- 172.21.15.32:0/238512002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 msgr2=0x558303d32fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.037+0000 7f0838e00640 1 --2- 172.21.15.32:0/238512002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 0x558303d32fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.037+0000 7f0838e00640 1 -- 172.21.15.32:0/238512002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 msgr2=0x558303d32aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:14.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.037+0000 7f0838e00640 1 --2- 172.21.15.32:0/238512002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d32aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.037+0000 7f0838e00640 1 -- 172.21.15.32:0/238512002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558303d2fc80 con 0x55830395f1b0 2024-08-31T07:26:14.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.037+0000 7f0838e00640 1 --2- 172.21.15.32:0/238512002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d39d50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c00 tx=0x7f08280ef040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9994da4c10be670c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:14.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.037+0000 7f0825600640 1 -- 172.21.15.32:0/238512002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828049040 con 0x55830395f1b0 2024-08-31T07:26:14.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.037+0000 7f0825600640 1 -- 172.21.15.32:0/238512002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08280cc1a0 con 0x55830395f1b0 2024-08-31T07:26:14.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.037+0000 7f0825600640 1 -- 172.21.15.32:0/238512002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08280c85c0 con 0x55830395f1b0 2024-08-31T07:26:14.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.038+0000 7f08424d8740 1 -- 172.21.15.32:0/238512002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 msgr2=0x558303d39d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:14.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.038+0000 7f08424d8740 1 --2- 172.21.15.32:0/238512002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d39d50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c00 tx=0x7f08280ef040 comp rx=0 tx=0).stop 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.038+0000 7f08424d8740 1 -- 172.21.15.32:0/238512002 shutdown_connections 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.038+0000 7f08424d8740 1 --2- 172.21.15.32:0/238512002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 0x558303d32fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.038+0000 7f08424d8740 1 --2- 172.21.15.32:0/238512002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d32aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.038+0000 7f08424d8740 1 --2- 172.21.15.32:0/238512002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d39d50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.038+0000 7f08424d8740 1 -- 172.21.15.32:0/238512002 >> 172.21.15.32:0/238512002 conn(0x55830397c460 msgr2=0x558303d39640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.038+0000 7f08424d8740 1 -- 172.21.15.32:0/238512002 shutdown_connections 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.038+0000 7f08424d8740 1 -- 172.21.15.32:0/238512002 wait complete. 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.039+0000 7f08424d8740 1 -- start start 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.039+0000 7f08424d8740 1 Processor -- start 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f08424d8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d3dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f08424d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d3e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f08424d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 0x558303daf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f08424d8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08341b2b80 con 0x55830395f1b0 2024-08-31T07:26:14.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f08424d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f08341b2450 con 0x558303967390 2024-08-31T07:26:14.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f08424d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f08341af6b0 con 0x558303988070 2024-08-31T07:26:14.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f08424d8740 10 client.0 _ll_register_callbacks cb 0x7f0820009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0838e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d3dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0833e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d3e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0839800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 0x558303daf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0833e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d3e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:54482/0 (socket says 172.21.15.32:54482) 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0839800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 0x558303daf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:38470/0 (socket says 172.21.15.32:38470) 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0839800640 1 -- 172.21.15.32:0/3307378005 learned_addr learned my addr 172.21.15.32:0/3307378005 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0838e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d3dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54832/0 (socket says 172.21.15.32:54832) 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0839800640 1 -- 172.21.15.32:0/3307378005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 msgr2=0x558303d3e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0839800640 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d3e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0839800640 1 -- 172.21.15.32:0/3307378005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 msgr2=0x558303d3dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0839800640 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d3dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0839800640 1 -- 172.21.15.32:0/3307378005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558303daf830 con 0x558303988070 2024-08-31T07:26:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0833e00640 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d3e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.040+0000 7f0838e00640 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d3dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.041+0000 7f0839800640 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 0x558303daf1c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f082c00e7d0 tx=0x7f082c00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31795074613d9539 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.041+0000 7f081aa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.041+0000 7f081aa00640 1 -- 172.21.15.32:0/3307378005 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c037030 con 0x558303988070 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.041+0000 7f08424d8740 1 -- 172.21.15.32:0/3307378005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08280cc1a0 con 0x558303988070 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.041+0000 7f081aa00640 1 -- 172.21.15.32:0/3307378005 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f082c026690 con 0x558303988070 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.041+0000 7f081aa00640 1 -- 172.21.15.32:0/3307378005 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f082c026890 con 0x558303988070 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.041+0000 7f08424d8740 10 client.28336 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.041+0000 7f08424d8740 1 -- 172.21.15.32:0/3307378005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558303db6b50 con 0x558303988070 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.042+0000 7f081aa00640 1 -- 172.21.15.32:0/3307378005 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08280cc1a0 con 0x558303988070 2024-08-31T07:26:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.042+0000 7f081aa00640 1 -- 172.21.15.32:0/3307378005 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7f082c043020 con 0x558303988070 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.042+0000 7f08424d8740 10 client.28336 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.042+0000 7f08424d8740 10 client.28336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.042+0000 7f08424d8740 20 client.28336 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.042+0000 7f08424d8740 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55830391bda0 0x558303db91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.042+0000 7f08424d8740 4 client.28336 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.042+0000 7f08424d8740 1 -- 172.21.15.32:0/3307378005 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558303ca21a0 con 0x55830391bda0 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.042+0000 7f0838e00640 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55830391bda0 0x558303db91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f0838e00640 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55830391bda0 0x558303db91a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afb00a53de2eb9af server_cookie=1bf03d1f02b8e645 in_seq=0 out_seq=0 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f081aa00640 10 client.28336 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f081aa00640 1 -- 172.21.15.32:0/3307378005 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558303ca21a0 con 0x55830391bda0 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f081aa00640 10 client.28336 handle_command_reply: tid=0 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f08424d8740 1 client.28336 shutdown 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f08424d8740 1 -- 172.21.15.32:0/3307378005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 msgr2=0x558303daf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f08424d8740 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 0x558303daf1c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f082c00e7d0 tx=0x7f082c00ecf0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f08424d8740 1 -- 172.21.15.32:0/3307378005 shutdown_connections 2024-08-31T07:26:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f08424d8740 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55830391bda0 0x558303db91a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f08424d8740 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558303988070 0x558303daf1c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f08424d8740 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558303967390 0x558303d3e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f08424d8740 1 --2- 172.21.15.32:0/3307378005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55830395f1b0 0x558303d3dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f081aa00640 0 client.28336 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.043+0000 7f08424d8740 1 -- 172.21.15.32:0/3307378005 >> 172.21.15.32:0/3307378005 conn(0x55830397c460 msgr2=0x558303d32aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.044+0000 7f08424d8740 1 -- 172.21.15.32:0/3307378005 shutdown_connections 2024-08-31T07:26:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.044+0000 7f08424d8740 1 -- 172.21.15.32:0/3307378005 wait complete. 2024-08-31T07:26:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.044+0000 7f08424d8740 20 client.28336 trim_cache size 0 max 16384 2024-08-31T07:26:14.049 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:26:14.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.044+0000 7f083b800640 1 -- 172.21.15.32:0/2476734411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f081c0b4d20 msgr2=0x7f081c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:14.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.044+0000 7f083b800640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f081c0b4d20 0x7f081c0b7230 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f082c005650 tx=0x7f082c0054b0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.044+0000 7f083b800640 1 -- 172.21.15.32:0/2476734411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08340754f0 msgr2=0x7f083419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:14.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.044+0000 7f083b800640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08340754f0 0x7f083419c4d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f082802a9e0 tx=0x7f0828003b90 comp rx=0 tx=0).stop 2024-08-31T07:26:14.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.045+0000 7f083b800640 1 -- 172.21.15.32:0/2476734411 shutdown_connections 2024-08-31T07:26:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.045+0000 7f083b800640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f081c0b4d20 0x7f081c0b7230 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.045+0000 7f083b800640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f083410fec0 0x7f08341a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.045+0000 7f083b800640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0834075e40 0x7f083419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.045+0000 7f083b800640 1 --2- 172.21.15.32:0/2476734411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08340754f0 0x7f083419c4d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.045+0000 7f083b800640 1 -- 172.21.15.32:0/2476734411 >> 172.21.15.32:0/2476734411 conn(0x7f083407b5f0 msgr2=0x7f08341099f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.045+0000 7f083b800640 1 -- 172.21.15.32:0/2476734411 shutdown_connections 2024-08-31T07:26:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:14.045+0000 7f083b800640 1 -- 172.21.15.32:0/2476734411 wait complete. 2024-08-31T07:26:14.786 INFO:tasks.workunit.client.0.smithi032.stdout: 1 cleanup 603 sec 2024-08-31T07:26:14.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:14 smithi032 ceph-mon[17252]: pgmap v787: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 908 KiB/s wr, 96 op/s 2024-08-31T07:26:14.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:14 smithi084 ceph-mon[22387]: pgmap v787: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 908 KiB/s wr, 96 op/s 2024-08-31T07:26:15.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:26:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:14 smithi073 ceph-mon[21995]: pgmap v787: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 908 KiB/s wr, 96 op/s 2024-08-31T07:26:15.465 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:15.785 INFO:tasks.workunit.client.0.smithi032.stdout: 1 cleanup 604 sec 2024-08-31T07:26:16.037 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:26:16.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:26:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.164+0000 7f4daca00640 1 -- 172.21.15.32:0/3918727045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4da810fcb0 msgr2=0x7f4da81120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.164+0000 7f4daca00640 1 --2- 172.21.15.32:0/3918727045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4da810fcb0 0x7f4da81120f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009ca0 tx=0x7f4da002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:26:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.164+0000 7f4daca00640 1 -- 172.21.15.32:0/3918727045 shutdown_connections 2024-08-31T07:26:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.164+0000 7f4daca00640 1 --2- 172.21.15.32:0/3918727045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4da810fcb0 0x7f4da81120f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.164+0000 7f4daca00640 1 --2- 172.21.15.32:0/3918727045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4da8075ea0 0x7f4da810f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:16.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.164+0000 7f4daca00640 1 --2- 172.21.15.32:0/3918727045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da8075550 0x7f4da8075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.164+0000 7f4daca00640 1 -- 172.21.15.32:0/3918727045 >> 172.21.15.32:0/3918727045 conn(0x7f4da80ff1e0 msgr2=0x7f4da8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.164+0000 7f4daca00640 1 -- 172.21.15.32:0/3918727045 shutdown_connections 2024-08-31T07:26:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.164+0000 7f4daca00640 1 -- 172.21.15.32:0/3918727045 wait complete. 2024-08-31T07:26:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.166+0000 7f4daca00640 1 Processor -- start 2024-08-31T07:26:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.166+0000 7f4daca00640 1 -- start start 2024-08-31T07:26:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4daca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da8075550 0x7f4da819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4daca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4da8075ea0 0x7f4da819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4daca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4da810fcb0 0x7f4da81a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4daca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8115860 con 0x7f4da810fcb0 2024-08-31T07:26:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4daca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81156e0 con 0x7f4da8075ea0 2024-08-31T07:26:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4daca00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81159e0 con 0x7f4da8075550 2024-08-31T07:26:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4da6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4da8075ea0 0x7f4da819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4da7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da8075550 0x7f4da819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4da6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4da8075ea0 0x7f4da819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52328/0 (socket says 172.21.15.32:52328) 2024-08-31T07:26:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4da6a00640 1 -- 172.21.15.32:0/3332980512 learned_addr learned my addr 172.21.15.32:0/3332980512 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4da7e00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4da810fcb0 0x7f4da81a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4da6a00640 1 -- 172.21.15.32:0/3332980512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da8075550 msgr2=0x7f4da819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4da6a00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da8075550 0x7f4da819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4da6a00640 1 -- 172.21.15.32:0/3332980512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4da810fcb0 msgr2=0x7f4da81a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4da6a00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4da810fcb0 0x7f4da81a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.167+0000 7f4da6a00640 1 -- 172.21.15.32:0/3332980512 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da81a21e0 con 0x7f4da8075ea0 2024-08-31T07:26:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.168+0000 7f4da7400640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da8075550 0x7f4da819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.168+0000 7f4da6a00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4da8075ea0 0x7f4da819c810 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400d950 tx=0x7f4d9400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.168+0000 7f4d9fe00640 1 -- 172.21.15.32:0/3332980512 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d94017020 con 0x7f4da8075ea0 2024-08-31T07:26:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.168+0000 7f4daca00640 1 -- 172.21.15.32:0/3332980512 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da81a24d0 con 0x7f4da8075ea0 2024-08-31T07:26:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.168+0000 7f4d9fe00640 1 -- 172.21.15.32:0/3332980512 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d94009c30 con 0x7f4da8075ea0 2024-08-31T07:26:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.168+0000 7f4d9fe00640 1 -- 172.21.15.32:0/3332980512 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9400f6b0 con 0x7f4da8075ea0 2024-08-31T07:26:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.168+0000 7f4daca00640 1 -- 172.21.15.32:0/3332980512 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da81a8c30 con 0x7f4da8075ea0 2024-08-31T07:26:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.169+0000 7f4daca00640 1 -- 172.21.15.32:0/3332980512 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da8075960 con 0x7f4da8075ea0 2024-08-31T07:26:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.173+0000 7f4d9fe00640 1 -- 172.21.15.32:0/3332980512 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4d9401a020 con 0x7f4da8075ea0 2024-08-31T07:26:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.174+0000 7f4d9fe00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d8c0b4ca0 0x7f4d8c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.175+0000 7f4d9fe00640 1 -- 172.21.15.32:0/3332980512 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8485+0+0 (secure 0 0 0) 0x7f4d940e5420 con 0x7f4da8075ea0 2024-08-31T07:26:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.175+0000 7f4da7400640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d8c0b4ca0 0x7f4d8c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.175+0000 7f4da7400640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d8c0b4ca0 0x7f4d8c0b71b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4d98005fd0 tx=0x7f4d9800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.175+0000 7f4d9fe00640 1 -- 172.21.15.32:0/3332980512 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4d94009de0 con 0x7f4da8075ea0 2024-08-31T07:26:16.430 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:16.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.432+0000 7f4daca00640 1 -- 172.21.15.32:0/3332980512 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4da819d3a0 con 0x7f4da8075ea0 2024-08-31T07:26:16.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.433+0000 7f4d9fe00640 1 -- 172.21.15.32:0/3332980512 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11970 (secure 0 0 0) 0x7f4da819d3a0 con 0x7f4da8075ea0 2024-08-31T07:26:16.435 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:26:16.435 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"btime":"2024-08-31T07:25:04:192234+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:25:03.382644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:26:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 -- 172.21.15.32:0/3332980512 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d8c0b4ca0 msgr2=0x7f4d8c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d8c0b4ca0 0x7f4d8c0b71b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4d98005fd0 tx=0x7f4d9800b390 comp rx=0 tx=0).stop 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 -- 172.21.15.32:0/3332980512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4da8075ea0 msgr2=0x7f4da819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4da8075ea0 0x7f4da819c810 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400d950 tx=0x7f4d9400de10 comp rx=0 tx=0).stop 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 -- 172.21.15.32:0/3332980512 shutdown_connections 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4d8c0b4ca0 0x7f4d8c0b71b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4da810fcb0 0x7f4da81a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4da8075ea0 0x7f4da819c810 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 --2- 172.21.15.32:0/3332980512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da8075550 0x7f4da819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 -- 172.21.15.32:0/3332980512 >> 172.21.15.32:0/3332980512 conn(0x7f4da80ff1e0 msgr2=0x7f4da81097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:16.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 -- 172.21.15.32:0/3332980512 shutdown_connections 2024-08-31T07:26:16.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:16.436+0000 7f4daca00640 1 -- 172.21.15.32:0/3332980512 wait complete. 2024-08-31T07:26:16.786 INFO:tasks.workunit.client.0.smithi032.stdout: 1 cleanup 605 sec 2024-08-31T07:26:16.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:16 smithi084 ceph-mon[22387]: pgmap v788: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 902 KiB/s wr, 80 op/s 2024-08-31T07:26:16.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:16 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3332980512' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:26:17.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:16 smithi032 ceph-mon[17252]: pgmap v788: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 902 KiB/s wr, 80 op/s 2024-08-31T07:26:17.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:16 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3332980512' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:26:17.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:16 smithi073 ceph-mon[21995]: pgmap v788: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 902 KiB/s wr, 80 op/s 2024-08-31T07:26:17.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:16 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3332980512' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:26:17.326 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 63a6ed69 --release --await' 2024-08-31T07:26:17.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 63a6ed69 --release --await 2024-08-31T07:26:17.786 INFO:tasks.workunit.client.0.smithi032.stdout: 1 cleanup 606 sec 2024-08-31T07:26:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.942+0000 7fb175e00640 1 -- 172.21.15.32:0/1071525914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb170075e40 msgr2=0x7fb17010f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.942+0000 7fb175e00640 1 --2- 172.21.15.32:0/1071525914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb170075e40 0x7fb17010f7b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009900 tx=0x7fb15c02a400 comp rx=0 tx=0).stop 2024-08-31T07:26:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.942+0000 7fb175e00640 1 -- 172.21.15.32:0/1071525914 shutdown_connections 2024-08-31T07:26:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.942+0000 7fb175e00640 1 --2- 172.21.15.32:0/1071525914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb17010fcf0 0x7fb170112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.942+0000 7fb175e00640 1 --2- 172.21.15.32:0/1071525914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb170075e40 0x7fb17010f7b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.942+0000 7fb175e00640 1 --2- 172.21.15.32:0/1071525914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1700754f0 0x7fb170075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.942+0000 7fb175e00640 1 -- 172.21.15.32:0/1071525914 >> 172.21.15.32:0/1071525914 conn(0x7fb1700ff180 msgr2=0x7fb1701015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.942+0000 7fb175e00640 1 -- 172.21.15.32:0/1071525914 shutdown_connections 2024-08-31T07:26:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.942+0000 7fb175e00640 1 -- 172.21.15.32:0/1071525914 wait complete. 2024-08-31T07:26:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.943+0000 7fb175e00640 1 Processor -- start 2024-08-31T07:26:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.943+0000 7fb175e00640 1 -- start start 2024-08-31T07:26:17.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb175e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700754f0 0x7fb17019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb175e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb170075e40 0x7fb17019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb175e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb17010fcf0 0x7fb1701a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb175e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb170115860 con 0x7fb17010fcf0 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb175e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701156e0 con 0x7fb1700754f0 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb175e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701159e0 con 0x7fb170075e40 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb175400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb17010fcf0 0x7fb1701a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb174a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700754f0 0x7fb17019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb170075e40 0x7fb17019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb174a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700754f0 0x7fb17019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52350/0 (socket says 172.21.15.32:52350) 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb174a00640 1 -- 172.21.15.32:0/475627910 learned_addr learned my addr 172.21.15.32:0/475627910 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb174a00640 1 -- 172.21.15.32:0/475627910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb170075e40 msgr2=0x7fb17019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb174a00640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb170075e40 0x7fb17019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb174a00640 1 -- 172.21.15.32:0/475627910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb17010fcf0 msgr2=0x7fb1701a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb174a00640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb17010fcf0 0x7fb1701a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb174a00640 1 -- 172.21.15.32:0/475627910 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1701a2200 con 0x7fb1700754f0 2024-08-31T07:26:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.944+0000 7fb175400640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb17010fcf0 0x7fb1701a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.945+0000 7fb174a00640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700754f0 0x7fb17019c290 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb16400e9f0 tx=0x7fb16400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.945+0000 7fb16d600640 1 -- 172.21.15.32:0/475627910 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16400ccc0 con 0x7fb1700754f0 2024-08-31T07:26:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.945+0000 7fb16d600640 1 -- 172.21.15.32:0/475627910 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb16400ce60 con 0x7fb1700754f0 2024-08-31T07:26:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.945+0000 7fb175e00640 1 -- 172.21.15.32:0/475627910 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1701a24f0 con 0x7fb1700754f0 2024-08-31T07:26:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.945+0000 7fb16d600640 1 -- 172.21.15.32:0/475627910 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164010430 con 0x7fb1700754f0 2024-08-31T07:26:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.945+0000 7fb175e00640 1 -- 172.21.15.32:0/475627910 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1701a8c50 con 0x7fb1700754f0 2024-08-31T07:26:17.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.948+0000 7fb17e082740 1 Processor -- start 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb17e082740 1 -- start start 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb16d600640 1 -- 172.21.15.32:0/475627910 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb1640046c0 con 0x7fb1700754f0 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb17e082740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1d3350 0x561d4c5797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb17e082740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1aa490 0x561d4c579d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb17e082740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1b2670 0x561d4c57a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb17e082740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561d4c4ef750 con 0x561d4c1b2670 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb17e082740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561d4c4dece0 con 0x561d4c1d3350 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb17e082740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561d4c4e2c90 con 0x561d4c1aa490 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1b2670 0x561d4c57a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb174a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1aa490 0x561d4c579d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1b2670 0x561d4c57a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49238/0 (socket says 172.21.15.32:49238) 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.949+0000 7fb16fe00640 1 -- 172.21.15.32:0/1540074663 learned_addr learned my addr 172.21.15.32:0/1540074663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.950+0000 7fb175400640 1 --2- 172.21.15.32:0/1540074663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1d3350 0x561d4c5797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.950+0000 7fb16fe00640 1 -- 172.21.15.32:0/1540074663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1aa490 msgr2=0x561d4c579d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.950+0000 7fb16fe00640 1 --2- 172.21.15.32:0/1540074663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1aa490 0x561d4c579d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.950+0000 7fb16fe00640 1 -- 172.21.15.32:0/1540074663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1d3350 msgr2=0x561d4c5797d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.950+0000 7fb16fe00640 1 --2- 172.21.15.32:0/1540074663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1d3350 0x561d4c5797d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.950+0000 7fb16fe00640 1 -- 172.21.15.32:0/1540074663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4c23e930 con 0x561d4c1b2670 2024-08-31T07:26:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.950+0000 7fb175400640 1 --2- 172.21.15.32:0/1540074663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1d3350 0x561d4c5797d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.950+0000 7fb16fe00640 1 --2- 172.21.15.32:0/1540074663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1b2670 0x561d4c57a250 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009900 tx=0x7fb15c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6f4c9e7864a174 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:17.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.951+0000 7fb161c00640 1 -- 172.21.15.32:0/1540074663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c040070 con 0x561d4c1b2670 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.951+0000 7fb161c00640 1 -- 172.21.15.32:0/1540074663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb15c004030 con 0x561d4c1b2670 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.951+0000 7fb161c00640 1 -- 172.21.15.32:0/1540074663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c035040 con 0x561d4c1b2670 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb17e082740 1 -- 172.21.15.32:0/1540074663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1b2670 msgr2=0x561d4c57a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb17e082740 1 --2- 172.21.15.32:0/1540074663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1b2670 0x561d4c57a250 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009900 tx=0x7fb15c000bc0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb16d600640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1580b4ca0 0x7fb1580b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb17e082740 1 -- 172.21.15.32:0/1540074663 shutdown_connections 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb17e082740 1 --2- 172.21.15.32:0/1540074663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1b2670 0x561d4c57a250 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb17e082740 1 --2- 172.21.15.32:0/1540074663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1aa490 0x561d4c579d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb17e082740 1 --2- 172.21.15.32:0/1540074663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1d3350 0x561d4c5797d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb17e082740 1 -- 172.21.15.32:0/1540074663 >> 172.21.15.32:0/1540074663 conn(0x561d4c1c7740 msgr2=0x561d4c5776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb16d600640 1 -- 172.21.15.32:0/475627910 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7fb1640e5930 con 0x7fb1700754f0 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb175400640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1580b4ca0 0x7fb1580b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb17e082740 1 -- 172.21.15.32:0/1540074663 shutdown_connections 2024-08-31T07:26:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.952+0000 7fb17e082740 1 -- 172.21.15.32:0/1540074663 wait complete. 2024-08-31T07:26:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.953+0000 7fb17e082740 1 -- start start 2024-08-31T07:26:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.953+0000 7fb175400640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1580b4ca0 0x7fb1580b71b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb168005fd0 tx=0x7fb168009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.953+0000 7fb17e082740 1 Processor -- start 2024-08-31T07:26:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.953+0000 7fb17e082740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1aa490 0x561d4c585350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.953+0000 7fb17e082740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 0x561d4c585890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb17e082740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1d3350 0x561d4c591270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb17e082740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x561d4c341890 con 0x561d4c1aa490 2024-08-31T07:26:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb17e082740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561d4c465cd0 con 0x561d4c1b2670 2024-08-31T07:26:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb17e082740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561d4c587f50 con 0x561d4c1d3350 2024-08-31T07:26:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb174a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1aa490 0x561d4c585350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb17e082740 10 client.0 _ll_register_callbacks cb 0x561d4c4d9ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb174a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1aa490 0x561d4c585350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49248/0 (socket says 172.21.15.32:49248) 2024-08-31T07:26:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb174a00640 1 -- 172.21.15.32:0/3594256815 learned_addr learned my addr 172.21.15.32:0/3594256815 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb175400640 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1d3350 0x561d4c591270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb16fe00640 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 0x561d4c585890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb174a00640 1 -- 172.21.15.32:0/3594256815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1d3350 msgr2=0x561d4c591270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb174a00640 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1d3350 0x561d4c591270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb174a00640 1 -- 172.21.15.32:0/3594256815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 msgr2=0x561d4c585890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb174a00640 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 0x561d4c585890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.954+0000 7fb174a00640 1 -- 172.21.15.32:0/3594256815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d4c4af1b0 con 0x561d4c1aa490 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.955+0000 7fb174a00640 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1aa490 0x561d4c585350 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb16400c6a0 tx=0x7fb1640ac630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec93ffe11ebbded8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.955+0000 7fb156000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.955+0000 7fb156000640 1 -- 172.21.15.32:0/3594256815 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640e7e60 con 0x561d4c1aa490 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.955+0000 7fb156000640 1 -- 172.21.15.32:0/3594256815 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7fb1640a8600 con 0x561d4c1aa490 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.955+0000 7fb17e082740 1 -- 172.21.15.32:0/3594256815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb15c004030 con 0x561d4c1aa490 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.955+0000 7fb156000640 1 -- 172.21.15.32:0/3594256815 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb1640abde0 con 0x561d4c1aa490 2024-08-31T07:26:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.955+0000 7fb17e082740 10 client.18342 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.955+0000 7fb17e082740 1 -- 172.21.15.32:0/3594256815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561d4c593fc0 con 0x561d4c1aa490 2024-08-31T07:26:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.955+0000 7fb156000640 1 -- 172.21.15.32:0/3594256815 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb15c004030 con 0x561d4c1aa490 2024-08-31T07:26:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.956+0000 7fb156000640 1 -- 172.21.15.32:0/3594256815 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7fb164097190 con 0x561d4c1aa490 2024-08-31T07:26:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.956+0000 7fb17e082740 10 client.18342 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.956+0000 7fb17e082740 10 client.18342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:26:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.956+0000 7fb17e082740 20 client.18342 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.956+0000 7fb17e082740 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561d4c191cd0 0x561d4c597660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.956+0000 7fb17e082740 4 client.18342 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:26:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.956+0000 7fb17e082740 1 -- 172.21.15.32:0/3594256815 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d4c4f16a0 con 0x561d4c191cd0 2024-08-31T07:26:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.956+0000 7fb16fe00640 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561d4c191cd0 0x561d4c597660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.957+0000 7fb16fe00640 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561d4c191cd0 0x561d4c597660 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=885f4ab52ec88ebf server_cookie=6e55beb9c563c78a in_seq=0 out_seq=0 2024-08-31T07:26:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.957+0000 7fb156000640 10 client.18342 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb156000640 1 -- 172.21.15.32:0/3594256815 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d4c4f16a0 con 0x561d4c191cd0 2024-08-31T07:26:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb156000640 10 client.18342 handle_command_reply: tid=0 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 client.18342 shutdown 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 -- 172.21.15.32:0/3594256815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1aa490 msgr2=0x561d4c585350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1aa490 0x561d4c585350 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb16400c6a0 tx=0x7fb1640ac630 comp rx=0 tx=0).stop 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 -- 172.21.15.32:0/3594256815 shutdown_connections 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561d4c191cd0 0x561d4c597660 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1d3350 0x561d4c591270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 0x561d4c585890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 --2- 172.21.15.32:0/3594256815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1aa490 0x561d4c585350 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb156000640 0 client.18342 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 -- 172.21.15.32:0/3594256815 >> 172.21.15.32:0/3594256815 conn(0x561d4c1c7740 msgr2=0x561d4c57c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 -- 172.21.15.32:0/3594256815 shutdown_connections 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 1 -- 172.21.15.32:0/3594256815 wait complete. 2024-08-31T07:26:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.958+0000 7fb17e082740 20 client.18342 trim_cache size 0 max 16384 2024-08-31T07:26:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.978+0000 7fb17e082740 1 Processor -- start 2024-08-31T07:26:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.978+0000 7fb17e082740 1 -- start start 2024-08-31T07:26:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.978+0000 7fb17e082740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1aa490 0x561d4c580900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.978+0000 7fb17e082740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1b2670 0x561d4c57af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.978+0000 7fb17e082740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 0x561d4c591170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.978+0000 7fb17e082740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701b1340 con 0x561d4c1d3350 2024-08-31T07:26:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.978+0000 7fb17e082740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701b2fa0 con 0x561d4c1aa490 2024-08-31T07:26:17.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.978+0000 7fb17e082740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701b1a10 con 0x561d4c1b2670 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb174a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1aa490 0x561d4c580900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb174a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1aa490 0x561d4c580900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52376/0 (socket says 172.21.15.32:52376) 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb174a00640 1 -- 172.21.15.32:0/3088318561 learned_addr learned my addr 172.21.15.32:0/3088318561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb16fe00640 1 --2- 172.21.15.32:0/3088318561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1b2670 0x561d4c57af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb175400640 1 --2- 172.21.15.32:0/3088318561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 0x561d4c591170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb174a00640 1 -- 172.21.15.32:0/3088318561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1b2670 msgr2=0x561d4c57af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb174a00640 1 --2- 172.21.15.32:0/3088318561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1b2670 0x561d4c57af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb174a00640 1 -- 172.21.15.32:0/3088318561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 msgr2=0x561d4c591170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb174a00640 1 --2- 172.21.15.32:0/3088318561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 0x561d4c591170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb174a00640 1 -- 172.21.15.32:0/3088318561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4c5840b0 con 0x561d4c1aa490 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.979+0000 7fb174a00640 1 --2- 172.21.15.32:0/3088318561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1aa490 0x561d4c580900 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb1640a9280 tx=0x7fb164009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84be405839edf524 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.980+0000 7fb161200640 1 -- 172.21.15.32:0/3088318561 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640c6070 con 0x561d4c1aa490 2024-08-31T07:26:17.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.980+0000 7fb161200640 1 -- 172.21.15.32:0/3088318561 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1640abb20 con 0x561d4c1aa490 2024-08-31T07:26:17.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.980+0000 7fb161200640 1 -- 172.21.15.32:0/3088318561 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640ab780 con 0x561d4c1aa490 2024-08-31T07:26:17.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.981+0000 7fb17e082740 1 -- 172.21.15.32:0/3088318561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1aa490 msgr2=0x561d4c580900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:17.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.981+0000 7fb17e082740 1 --2- 172.21.15.32:0/3088318561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1aa490 0x561d4c580900 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb1640a9280 tx=0x7fb164009330 comp rx=0 tx=0).stop 2024-08-31T07:26:17.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.981+0000 7fb17e082740 1 -- 172.21.15.32:0/3088318561 shutdown_connections 2024-08-31T07:26:17.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.981+0000 7fb17e082740 1 --2- 172.21.15.32:0/3088318561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 0x561d4c591170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.981+0000 7fb17e082740 1 --2- 172.21.15.32:0/3088318561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1b2670 0x561d4c57af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.981+0000 7fb17e082740 1 --2- 172.21.15.32:0/3088318561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1aa490 0x561d4c580900 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.981+0000 7fb17e082740 1 -- 172.21.15.32:0/3088318561 >> 172.21.15.32:0/3088318561 conn(0x561d4c1c7740 msgr2=0x561d4c57a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:17.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.981+0000 7fb17e082740 1 -- 172.21.15.32:0/3088318561 shutdown_connections 2024-08-31T07:26:17.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.981+0000 7fb17e082740 1 -- 172.21.15.32:0/3088318561 wait complete. 2024-08-31T07:26:17.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.981+0000 7fb17e082740 1 -- start start 2024-08-31T07:26:17.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.982+0000 7fb17e082740 1 Processor -- start 2024-08-31T07:26:17.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.982+0000 7fb17e082740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1aa490 0x561d4c589b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.982+0000 7fb17e082740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 0x561d4c58a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.982+0000 7fb17e082740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 0x561d4c5fb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.982+0000 7fb17e082740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701b2810 con 0x561d4c1d3350 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.982+0000 7fb17e082740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701b20e0 con 0x561d4c1b2670 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.982+0000 7fb17e082740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701af340 con 0x561d4c1aa490 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.982+0000 7fb17e082740 10 client.0 _ll_register_callbacks cb 0x7fb1640eb220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb175400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 0x561d4c5fb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 0x561d4c58a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 0x561d4c58a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52388/0 (socket says 172.21.15.32:52388) 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb175400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 0x561d4c5fb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49264/0 (socket says 172.21.15.32:49264) 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb16fe00640 1 -- 172.21.15.32:0/554142082 learned_addr learned my addr 172.21.15.32:0/554142082 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb174a00640 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1aa490 0x561d4c589b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb16fe00640 1 -- 172.21.15.32:0/554142082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1aa490 msgr2=0x561d4c589b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb16fe00640 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1aa490 0x561d4c589b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb16fe00640 1 -- 172.21.15.32:0/554142082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 msgr2=0x561d4c5fb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:17.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb16fe00640 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 0x561d4c5fb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb16fe00640 1 -- 172.21.15.32:0/554142082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d4c5fb740 con 0x561d4c1b2670 2024-08-31T07:26:17.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb16fe00640 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 0x561d4c58a150 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb15c031a60 tx=0x7fb15c005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6d7c3668392e76 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:17.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.983+0000 7fb156000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:26:17.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.984+0000 7fb156000640 1 -- 172.21.15.32:0/554142082 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c040040 con 0x561d4c1b2670 2024-08-31T07:26:17.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.984+0000 7fb156000640 1 -- 172.21.15.32:0/554142082 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7fb15c031d10 con 0x561d4c1b2670 2024-08-31T07:26:17.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.984+0000 7fb17e082740 1 -- 172.21.15.32:0/554142082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1640abb20 con 0x561d4c1b2670 2024-08-31T07:26:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.984+0000 7fb156000640 1 -- 172.21.15.32:0/554142082 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb15c035040 con 0x561d4c1b2670 2024-08-31T07:26:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.984+0000 7fb17e082740 10 client.28358 fetch_fsmap learned FSMap version 47 2024-08-31T07:26:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.984+0000 7fb17e082740 1 -- 172.21.15.32:0/554142082 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561d4c602aa0 con 0x561d4c1b2670 2024-08-31T07:26:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.985+0000 7fb156000640 1 -- 172.21.15.32:0/554142082 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1640abb20 con 0x561d4c1b2670 2024-08-31T07:26:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.985+0000 7fb156000640 1 -- 172.21.15.32:0/554142082 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7fb15c055030 con 0x561d4c1b2670 2024-08-31T07:26:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.985+0000 7fb17e082740 10 client.28358 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T07:26:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.985+0000 7fb17e082740 10 client.28358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:26:17.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.985+0000 7fb17e082740 20 client.28358 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.985+0000 7fb17e082740 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561d4c167080 0x561d4c604f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.985+0000 7fb17e082740 4 client.28358 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:26:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.985+0000 7fb17e082740 1 -- 172.21.15.32:0/554142082 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561d4c4ed140 con 0x561d4c167080 2024-08-31T07:26:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.985+0000 7fb174a00640 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561d4c167080 0x561d4c604f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.986+0000 7fb174a00640 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561d4c167080 0x561d4c604f00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce37b30f1cbdd35a server_cookie=e979ffdceb38a0b in_seq=0 out_seq=0 2024-08-31T07:26:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.986+0000 7fb156000640 10 client.28358 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb156000640 1 -- 172.21.15.32:0/554142082 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561d4c4ed140 con 0x561d4c167080 2024-08-31T07:26:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb156000640 10 client.28358 handle_command_reply: tid=0 2024-08-31T07:26:17.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb17e082740 1 client.28358 shutdown 2024-08-31T07:26:17.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb17e082740 1 -- 172.21.15.32:0/554142082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 msgr2=0x561d4c58a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:17.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb17e082740 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 0x561d4c58a150 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb15c031a60 tx=0x7fb15c005870 comp rx=0 tx=0).stop 2024-08-31T07:26:17.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb17e082740 1 -- 172.21.15.32:0/554142082 shutdown_connections 2024-08-31T07:26:17.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb17e082740 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x561d4c167080 0x561d4c604f00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb17e082740 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561d4c1d3350 0x561d4c5fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb17e082740 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d4c1b2670 0x561d4c58a150 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb17e082740 1 --2- 172.21.15.32:0/554142082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d4c1aa490 0x561d4c589b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb17e082740 1 -- 172.21.15.32:0/554142082 >> 172.21.15.32:0/554142082 conn(0x561d4c1c7740 msgr2=0x561d4c586600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:17.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.988+0000 7fb156000640 0 client.28358 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:17.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.989+0000 7fb17e082740 1 -- 172.21.15.32:0/554142082 shutdown_connections 2024-08-31T07:26:17.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.989+0000 7fb17e082740 1 -- 172.21.15.32:0/554142082 wait complete. 2024-08-31T07:26:17.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.989+0000 7fb17e082740 20 client.28358 trim_cache size 0 max 16384 2024-08-31T07:26:17.991 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:26:17.991 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:26:17.991 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "ce8b7231-1fea-412d-a295-8119d9478cfa", 2024-08-31T07:26:17.991 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:26:17.991 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:26:17.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 -- 172.21.15.32:0/475627910 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1580b4ca0 msgr2=0x7fb1580b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:17.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1580b4ca0 0x7fb1580b71b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb168005fd0 tx=0x7fb168009500 comp rx=0 tx=0).stop 2024-08-31T07:26:17.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 -- 172.21.15.32:0/475627910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700754f0 msgr2=0x7fb17019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:17.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700754f0 0x7fb17019c290 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb16400e9f0 tx=0x7fb16400edf0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 -- 172.21.15.32:0/475627910 shutdown_connections 2024-08-31T07:26:17.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb1580b4ca0 0x7fb1580b71b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb17010fcf0 0x7fb1701a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb170075e40 0x7fb17019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 --2- 172.21.15.32:0/475627910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700754f0 0x7fb17019c290 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:17.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 -- 172.21.15.32:0/475627910 >> 172.21.15.32:0/475627910 conn(0x7fb1700ff180 msgr2=0x7fb1701098f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:17.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 -- 172.21.15.32:0/475627910 shutdown_connections 2024-08-31T07:26:17.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:17.990+0000 7fb175e00640 1 -- 172.21.15.32:0/475627910 wait complete. 2024-08-31T07:26:18.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:17 smithi032 ceph-mon[17252]: osdmap e96: 12 total, 12 up, 12 in 2024-08-31T07:26:18.017 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:17 smithi032 ceph-mon[17252]: pgmap v790: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 871 KiB/s wr, 59 op/s 2024-08-31T07:26:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:17 smithi073 ceph-mon[21995]: osdmap e96: 12 total, 12 up, 12 in 2024-08-31T07:26:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:17 smithi073 ceph-mon[21995]: pgmap v790: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 871 KiB/s wr, 59 op/s 2024-08-31T07:26:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:17 smithi084 ceph-mon[22387]: osdmap e96: 12 total, 12 up, 12 in 2024-08-31T07:26:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:17 smithi084 ceph-mon[22387]: pgmap v790: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 871 KiB/s wr, 59 op/s 2024-08-31T07:26:18.689 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:18.786 INFO:tasks.workunit.client.0.smithi032.stdout: 1 cleanup 607 sec 2024-08-31T07:26:18.996 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:26:18.996 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:18.996 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:26:18.996 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:26:18.996 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:26:18.996 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:26:18.996 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:26:18.997 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:18.997 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:26:18.997 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:18.997 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:18.997 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:18.997 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:18.997 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:18 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:26:19.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:18 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:18 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:26:19.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:26:19.786 INFO:tasks.workunit.client.0.smithi032.stdout: 1 cleanup 608 sec 2024-08-31T07:26:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.809+0000 7f98a0c00640 1 -- 172.21.15.32:0/3647317931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c1088a0 msgr2=0x7f989c10f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.809+0000 7f98a0c00640 1 --2- 172.21.15.32:0/3647317931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c1088a0 0x7f989c10f0d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9894009ca0 tx=0x7f989402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.809+0000 7f98a0c00640 1 -- 172.21.15.32:0/3647317931 shutdown_connections 2024-08-31T07:26:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.809+0000 7f98a0c00640 1 --2- 172.21.15.32:0/3647317931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c1088a0 0x7f989c10f0d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.809+0000 7f98a0c00640 1 --2- 172.21.15.32:0/3647317931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c107f50 0x7f989c108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.809+0000 7f98a0c00640 1 --2- 172.21.15.32:0/3647317931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c102610 0x7f989c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.809+0000 7f98a0c00640 1 -- 172.21.15.32:0/3647317931 >> 172.21.15.32:0/3647317931 conn(0x7f989c0fe060 msgr2=0x7f989c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.810+0000 7f98a0c00640 1 -- 172.21.15.32:0/3647317931 shutdown_connections 2024-08-31T07:26:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.810+0000 7f98a0c00640 1 -- 172.21.15.32:0/3647317931 wait complete. 2024-08-31T07:26:19.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.811+0000 7f98a0c00640 1 Processor -- start 2024-08-31T07:26:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.811+0000 7f98a0c00640 1 -- start start 2024-08-31T07:26:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.811+0000 7f98a0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c102610 0x7f989c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.811+0000 7f98a0c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c107f50 0x7f989c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.811+0000 7f98a0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c1088a0 0x7f989c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.811+0000 7f98a0c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f989c112570 con 0x7f989c1088a0 2024-08-31T07:26:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.811+0000 7f98a0c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1123f0 con 0x7f989c102610 2024-08-31T07:26:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.811+0000 7f98a0c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1126f0 con 0x7f989c107f50 2024-08-31T07:26:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c102610 0x7f989c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c107f50 0x7f989c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c102610 0x7f989c19c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52410/0 (socket says 172.21.15.32:52410) 2024-08-31T07:26:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989b400640 1 -- 172.21.15.32:0/434434290 learned_addr learned my addr 172.21.15.32:0/434434290 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989be00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c1088a0 0x7f989c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989aa00640 1 -- 172.21.15.32:0/434434290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c102610 msgr2=0x7f989c19c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989aa00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c102610 0x7f989c19c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989aa00640 1 -- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c1088a0 msgr2=0x7f989c1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989aa00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c1088a0 0x7f989c1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989aa00640 1 -- 172.21.15.32:0/434434290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989c1a2190 con 0x7f989c107f50 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989b400640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c102610 0x7f989c19c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.812+0000 7f989be00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c1088a0 0x7f989c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.813+0000 7f989aa00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c107f50 0x7f989c19c7c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f989c06a830 tx=0x7f988800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.813+0000 7f9893e00640 1 -- 172.21.15.32:0/434434290 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988800cd00 con 0x7f989c107f50 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.813+0000 7f98a0c00640 1 -- 172.21.15.32:0/434434290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989c1a2480 con 0x7f989c107f50 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.813+0000 7f98a0c00640 1 -- 172.21.15.32:0/434434290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f989c1a8be0 con 0x7f989c107f50 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.813+0000 7f9893e00640 1 -- 172.21.15.32:0/434434290 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98880035f0 con 0x7f989c107f50 2024-08-31T07:26:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.813+0000 7f9893e00640 1 -- 172.21.15.32:0/434434290 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888010690 con 0x7f989c107f50 2024-08-31T07:26:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.817+0000 7f98a8fcc740 1 Processor -- start 2024-08-31T07:26:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.817+0000 7f98a8fcc740 1 -- start start 2024-08-31T07:26:19.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.817+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c32141090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.817+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 0x564c321415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.817+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c32141b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.817+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564c320b7a90 con 0x564c31d7a840 2024-08-31T07:26:19.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.817+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564c320a7020 con 0x564c31d9b520 2024-08-31T07:26:19.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.817+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564c320aafd0 con 0x564c31d72660 2024-08-31T07:26:19.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f989be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c32141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f9893e00640 1 -- 172.21.15.32:0/434434290 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f98880037a0 con 0x7f989c107f50 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f989be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c32141090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52426/0 (socket says 172.21.15.32:52426) 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f989be00640 1 -- 172.21.15.32:0/2943141437 learned_addr learned my addr 172.21.15.32:0/2943141437 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f989b400640 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 0x564c321415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f989aa00640 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c32141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f989b400640 1 -- 172.21.15.32:0/2943141437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 msgr2=0x564c32141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f989b400640 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c32141090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f989b400640 1 -- 172.21.15.32:0/2943141437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 msgr2=0x564c32141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f989b400640 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c32141b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.818+0000 7f989b400640 1 -- 172.21.15.32:0/2943141437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c31e06b00 con 0x564c31d72660 2024-08-31T07:26:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.819+0000 7f989aa00640 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c32141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:26:19.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.819+0000 7f989be00640 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c32141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:19.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.819+0000 7f989b400640 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 0x564c321415d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f988c011510 tx=0x7f988c0119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e13ea5b42ae9f65 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.819+0000 7f9887e00640 1 -- 172.21.15.32:0/2943141437 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c01e070 con 0x564c31d72660 2024-08-31T07:26:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.819+0000 7f9887e00640 1 -- 172.21.15.32:0/2943141437 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f988c00fa70 con 0x564c31d72660 2024-08-31T07:26:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.819+0000 7f9887e00640 1 -- 172.21.15.32:0/2943141437 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c019b00 con 0x564c31d72660 2024-08-31T07:26:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.820+0000 7f9893e00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f98800b4d20 0x7f98800b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.820+0000 7f9893e00640 1 -- 172.21.15.32:0/434434290 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7f98880e5820 con 0x7f989c107f50 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.820+0000 7f989be00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f98800b4d20 0x7f98800b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.820+0000 7f98a8fcc740 1 -- 172.21.15.32:0/2943141437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 msgr2=0x564c321415d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.820+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 0x564c321415d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f988c011510 tx=0x7f988c0119d0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.820+0000 7f98a8fcc740 1 -- 172.21.15.32:0/2943141437 shutdown_connections 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.820+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c32141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.820+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 0x564c321415d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.820+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/2943141437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c32141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.820+0000 7f98a8fcc740 1 -- 172.21.15.32:0/2943141437 >> 172.21.15.32:0/2943141437 conn(0x564c31d8f910 msgr2=0x564c3213ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.821+0000 7f98a8fcc740 1 -- 172.21.15.32:0/2943141437 shutdown_connections 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.821+0000 7f98a8fcc740 1 -- 172.21.15.32:0/2943141437 wait complete. 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.821+0000 7f989be00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f98800b4d20 0x7f98800b7230 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f989402f490 tx=0x7f9894005c30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.821+0000 7f98a8fcc740 1 -- start start 2024-08-31T07:26:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.822+0000 7f98a8fcc740 1 Processor -- start 2024-08-31T07:26:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.822+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 0x564c3214c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.822+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c3214c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.822+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 0x564c3214cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.822+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564c31f09bb0 con 0x564c31d7a840 2024-08-31T07:26:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.822+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564c3202df90 con 0x564c31d72660 2024-08-31T07:26:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.822+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564c3214f870 con 0x564c31d9b520 2024-08-31T07:26:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.822+0000 7f98a8fcc740 10 client.0 _ll_register_callbacks cb 0x564c32053710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.823+0000 7f989aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c3214c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.823+0000 7f989b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 0x564c3214c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.823+0000 7f989aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c3214c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49298/0 (socket says 172.21.15.32:49298) 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.823+0000 7f989aa00640 1 -- 172.21.15.32:0/1508981399 learned_addr learned my addr 172.21.15.32:0/1508981399 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.823+0000 7f989be00640 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 0x564c3214cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.823+0000 7f989aa00640 1 -- 172.21.15.32:0/1508981399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 msgr2=0x564c3214cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.823+0000 7f989aa00640 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 0x564c3214cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.823+0000 7f989aa00640 1 -- 172.21.15.32:0/1508981399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 msgr2=0x564c3214c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.823+0000 7f989aa00640 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 0x564c3214c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.823+0000 7f989aa00640 1 -- 172.21.15.32:0/1508981399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c31e7d960 con 0x564c31d7a840 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.824+0000 7f989aa00640 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c3214c9c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9888009230 tx=0x7f9888003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16640f01ac9ab533 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.824+0000 7f9884c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.824+0000 7f9884c00640 1 -- 172.21.15.32:0/1508981399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98880e33f0 con 0x564c31d7a840 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.824+0000 7f9884c00640 1 -- 172.21.15.32:0/1508981399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7f98880a84f0 con 0x564c31d7a840 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.824+0000 7f98a8fcc740 1 -- 172.21.15.32:0/1508981399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564c32045e30 con 0x564c31d7a840 2024-08-31T07:26:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.824+0000 7f9884c00640 1 -- 172.21.15.32:0/1508981399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f98880abf00 con 0x564c31d7a840 2024-08-31T07:26:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.824+0000 7f98a8fcc740 10 client.18357 fetch_fsmap learned FSMap version 48 2024-08-31T07:26:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.824+0000 7f98a8fcc740 1 -- 172.21.15.32:0/1508981399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564c3215b290 con 0x564c31d7a840 2024-08-31T07:26:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.825+0000 7f9884c00640 1 -- 172.21.15.32:0/1508981399 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564c32045e30 con 0x564c31d7a840 2024-08-31T07:26:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.825+0000 7f9884c00640 1 -- 172.21.15.32:0/1508981399 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 48) ==== 8382+0+0 (secure 0 0 0) 0x7f9888093170 con 0x564c31d7a840 2024-08-31T07:26:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.825+0000 7f98a8fcc740 10 client.18357 fetch_fsmap finished waiting for FSMap version 48 2024-08-31T07:26:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.825+0000 7f98a8fcc740 10 client.18357 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:26:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.825+0000 7f98a8fcc740 20 client.18357 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.825+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564c31d59ea0 0x564c3215e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.825+0000 7f98a8fcc740 4 client.18357 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.825+0000 7f98a8fcc740 1 -- 172.21.15.32:0/1508981399 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c320b99e0 con 0x564c31d59ea0 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.826+0000 7f989b400640 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564c31d59ea0 0x564c3215e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.826+0000 7f989b400640 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564c31d59ea0 0x564c3215e900 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785b1ebb6b129866 server_cookie=4ddae4b0844dc2fa in_seq=0 out_seq=0 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.826+0000 7f9884c00640 10 client.18357 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.827+0000 7f9884c00640 1 -- 172.21.15.32:0/1508981399 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564c320b99e0 con 0x564c31d59ea0 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.827+0000 7f9884c00640 10 client.18357 handle_command_reply: tid=0 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.827+0000 7f98a8fcc740 1 client.18357 shutdown 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.827+0000 7f98a8fcc740 1 -- 172.21.15.32:0/1508981399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 msgr2=0x564c3214c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.827+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c3214c9c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9888009230 tx=0x7f9888003cd0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.828+0000 7f98a8fcc740 1 -- 172.21.15.32:0/1508981399 shutdown_connections 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.828+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564c31d59ea0 0x564c3215e900 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.828+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 0x564c3214cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.828+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d7a840 0x564c3214c9c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.828+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/1508981399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 0x564c3214c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.828+0000 7f98a8fcc740 1 -- 172.21.15.32:0/1508981399 >> 172.21.15.32:0/1508981399 conn(0x564c31d8f910 msgr2=0x564c32144270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.828+0000 7f9884c00640 0 client.18357 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.828+0000 7f98a8fcc740 1 -- 172.21.15.32:0/1508981399 shutdown_connections 2024-08-31T07:26:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.828+0000 7f98a8fcc740 1 -- 172.21.15.32:0/1508981399 wait complete. 2024-08-31T07:26:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.828+0000 7f98a8fcc740 20 client.18357 trim_cache size 0 max 16384 2024-08-31T07:26:19.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.847+0000 7f98a8fcc740 1 Processor -- start 2024-08-31T07:26:19.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.847+0000 7f98a8fcc740 1 -- start start 2024-08-31T07:26:19.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.847+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 0x564c32142a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.847+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c3214e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.847+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 0x564c3213e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.847+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1b13d0 con 0x564c31d2f250 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.847+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1b3000 con 0x564c31d9b520 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.847+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1b1aa0 con 0x564c31d72660 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c3214e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c3214e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52448/0 (socket says 172.21.15.32:52448) 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989aa00640 1 -- 172.21.15.32:0/3578773665 learned_addr learned my addr 172.21.15.32:0/3578773665 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989b400640 1 --2- 172.21.15.32:0/3578773665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 0x564c32142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989be00640 1 --2- 172.21.15.32:0/3578773665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 0x564c3213e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989aa00640 1 -- 172.21.15.32:0/3578773665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 msgr2=0x564c32142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:19.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989aa00640 1 --2- 172.21.15.32:0/3578773665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 0x564c32142a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989aa00640 1 -- 172.21.15.32:0/3578773665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 msgr2=0x564c3213e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:19.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989aa00640 1 --2- 172.21.15.32:0/3578773665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 0x564c3213e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989aa00640 1 -- 172.21.15.32:0/3578773665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c3214ebc0 con 0x564c31d9b520 2024-08-31T07:26:19.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.848+0000 7f989aa00640 1 --2- 172.21.15.32:0/3578773665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c3214e0b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f98880abe90 tx=0x7f98880abcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a85c44a361ce3ba6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:19.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.849+0000 7f9887400640 1 -- 172.21.15.32:0/3578773665 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98880e33f0 con 0x564c31d9b520 2024-08-31T07:26:19.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.849+0000 7f9887400640 1 -- 172.21.15.32:0/3578773665 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98880b5220 con 0x564c31d9b520 2024-08-31T07:26:19.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.849+0000 7f9887400640 1 -- 172.21.15.32:0/3578773665 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98880b3bc0 con 0x564c31d9b520 2024-08-31T07:26:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.850+0000 7f98a8fcc740 1 -- 172.21.15.32:0/3578773665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 msgr2=0x564c3214e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.850+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/3578773665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c3214e0b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f98880abe90 tx=0x7f98880abcd0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.850+0000 7f98a8fcc740 1 -- 172.21.15.32:0/3578773665 shutdown_connections 2024-08-31T07:26:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.850+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/3578773665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 0x564c3213e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.850+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/3578773665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d9b520 0x564c3214e0b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.850+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/3578773665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d72660 0x564c32142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.850+0000 7f98a8fcc740 1 -- 172.21.15.32:0/3578773665 >> 172.21.15.32:0/3578773665 conn(0x564c31d8f910 msgr2=0x564c3214d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.850+0000 7f98a8fcc740 1 -- 172.21.15.32:0/3578773665 shutdown_connections 2024-08-31T07:26:19.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.850+0000 7f98a8fcc740 1 -- 172.21.15.32:0/3578773665 wait complete. 2024-08-31T07:26:19.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.851+0000 7f98a8fcc740 1 -- start start 2024-08-31T07:26:19.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.851+0000 7f98a8fcc740 1 Processor -- start 2024-08-31T07:26:19.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.851+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 0x564c321540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.851+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 0x564c32154600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.851+0000 7f98a8fcc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 0x564c32154b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.851+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1b2870 con 0x564c31d2f250 2024-08-31T07:26:19.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.851+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1b2170 con 0x564c31d72660 2024-08-31T07:26:19.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.851+0000 7f98a8fcc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1af3d0 con 0x564c31d9b520 2024-08-31T07:26:19.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.851+0000 7f98a8fcc740 10 client.0 _ll_register_callbacks cb 0x7f98880ab670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.852+0000 7f989aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 0x564c32154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.852+0000 7f989aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 0x564c32154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52450/0 (socket says 172.21.15.32:52450) 2024-08-31T07:26:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.852+0000 7f989aa00640 1 -- 172.21.15.32:0/4018682953 learned_addr learned my addr 172.21.15.32:0/4018682953 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.852+0000 7f989be00640 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 0x564c32154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.852+0000 7f989b400640 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 0x564c321540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.852+0000 7f989aa00640 1 -- 172.21.15.32:0/4018682953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 msgr2=0x564c32154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.853+0000 7f989aa00640 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 0x564c32154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.853+0000 7f989aa00640 1 -- 172.21.15.32:0/4018682953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 msgr2=0x564c321540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:19.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.853+0000 7f989aa00640 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 0x564c321540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.853+0000 7f989aa00640 1 -- 172.21.15.32:0/4018682953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c321d0b60 con 0x564c31d72660 2024-08-31T07:26:19.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.853+0000 7f989b400640 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 0x564c321540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:26:19.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.853+0000 7f989be00640 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 0x564c32154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:19.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.853+0000 7f989aa00640 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 0x564c32154600 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9888009230 tx=0x7f98880e5aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed5678f0dae60e40 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:19.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.853+0000 7f9884c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:26:19.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.857+0000 7f9884c00640 1 -- 172.21.15.32:0/4018682953 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98880b0770 con 0x564c31d72660 2024-08-31T07:26:19.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.857+0000 7f9884c00640 1 -- 172.21.15.32:0/4018682953 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7f98880b3bc0 con 0x564c31d72660 2024-08-31T07:26:19.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.857+0000 7f98a8fcc740 1 -- 172.21.15.32:0/4018682953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98880b5220 con 0x564c31d72660 2024-08-31T07:26:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.863+0000 7f9884c00640 1 -- 172.21.15.32:0/4018682953 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f98880b3dc0 con 0x564c31d72660 2024-08-31T07:26:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.863+0000 7f98a8fcc740 10 client.28376 fetch_fsmap learned FSMap version 48 2024-08-31T07:26:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.863+0000 7f98a8fcc740 1 -- 172.21.15.32:0/4018682953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564c321d1e20 con 0x564c31d72660 2024-08-31T07:26:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.864+0000 7f9884c00640 1 -- 172.21.15.32:0/4018682953 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98880b5220 con 0x564c31d72660 2024-08-31T07:26:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.864+0000 7f9884c00640 1 -- 172.21.15.32:0/4018682953 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8382+0+0 (secure 0 0 0) 0x7f98880a7a10 con 0x564c31d72660 2024-08-31T07:26:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.864+0000 7f98a8fcc740 10 client.28376 fetch_fsmap finished waiting for FSMap version 48 2024-08-31T07:26:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.864+0000 7f98a8fcc740 10 client.28376 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:26:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.864+0000 7f98a8fcc740 20 client.28376 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.864+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564c321d2380 0x564c321d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.864+0000 7f98a8fcc740 4 client.28376 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "63a6ed69", "release": true, "await": true}] 2024-08-31T07:26:19.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.864+0000 7f98a8fcc740 1 -- 172.21.15.32:0/4018682953 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "63a6ed69", "release": true, "await": true}) -- 0x564c320b5480 con 0x564c321d2380 2024-08-31T07:26:19.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.865+0000 7f989b400640 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564c321d2380 0x564c321d47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:19.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.865+0000 7f989b400640 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564c321d2380 0x564c321d47a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba2c3120edece2b2 server_cookie=36a12ce84492b00b in_seq=0 out_seq=0 2024-08-31T07:26:19.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.865+0000 7f9884c00640 10 client.28376 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:19.886 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.892+0000 7f9884c00640 1 -- 172.21.15.32:0/4018682953 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564c320b5480 con 0x564c321d2380 2024-08-31T07:26:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.892+0000 7f9884c00640 10 client.28376 handle_command_reply: tid=0 2024-08-31T07:26:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.892+0000 7f98a8fcc740 1 client.28376 shutdown 2024-08-31T07:26:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f98a8fcc740 1 -- 172.21.15.32:0/4018682953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 msgr2=0x564c32154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 0x564c32154600 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9888009230 tx=0x7f98880e5aa0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f98a8fcc740 1 -- 172.21.15.32:0/4018682953 shutdown_connections 2024-08-31T07:26:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564c321d2380 0x564c321d47a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c31d9b520 0x564c32154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c31d72660 0x564c32154600 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f98a8fcc740 1 --2- 172.21.15.32:0/4018682953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c31d2f250 0x564c321540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f98a8fcc740 1 -- 172.21.15.32:0/4018682953 >> 172.21.15.32:0/4018682953 conn(0x564c31d8f910 msgr2=0x564c32142800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f9884c00640 0 client.28376 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:19.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f98a8fcc740 1 -- 172.21.15.32:0/4018682953 shutdown_connections 2024-08-31T07:26:19.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.893+0000 7f98a8fcc740 1 -- 172.21.15.32:0/4018682953 wait complete. 2024-08-31T07:26:19.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.894+0000 7f98a8fcc740 20 client.28376 trim_cache size 0 max 16384 2024-08-31T07:26:19.896 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:26:19.896 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 48, 2024-08-31T07:26:19.896 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:26:19.896 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 44, 2024-08-31T07:26:19.896 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:26:19.896 INFO:teuthology.orchestra.run.smithi032.stdout: "63a6ed69": { 2024-08-31T07:26:19.896 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 44, 2024-08-31T07:26:19.896 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:26:19.896 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:26:19.897 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:26:19.897 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:26:19.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:19.897 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:26:19.897 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 134.7, 2024-08-31T07:26:19.897 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:26:19.897 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:26:19.897 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:26:19.897 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:26:19.897 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:26:19.898 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:26:19.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:19.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:19.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:19.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:19.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:19.898 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:26:19.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.895+0000 7f98a0c00640 1 -- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f98800b4d20 msgr2=0x7f98800b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.895+0000 7f98a0c00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f98800b4d20 0x7f98800b7230 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f989402f490 tx=0x7f9894005c30 comp rx=0 tx=0).stop 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.895+0000 7f98a0c00640 1 -- 172.21.15.32:0/434434290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c107f50 msgr2=0x7f989c19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.895+0000 7f98a0c00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c107f50 0x7f989c19c7c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f989c06a830 tx=0x7f988800ee50 comp rx=0 tx=0).stop 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.896+0000 7f98a0c00640 1 -- 172.21.15.32:0/434434290 shutdown_connections 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.896+0000 7f98a0c00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f98800b4d20 0x7f98800b7230 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.896+0000 7f98a0c00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c1088a0 0x7f989c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.896+0000 7f98a0c00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c107f50 0x7f989c19c7c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.896+0000 7f98a0c00640 1 --2- 172.21.15.32:0/434434290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f989c102610 0x7f989c19c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.896+0000 7f98a0c00640 1 -- 172.21.15.32:0/434434290 >> 172.21.15.32:0/434434290 conn(0x7f989c0fe060 msgr2=0x7f989c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.896+0000 7f98a0c00640 1 -- 172.21.15.32:0/434434290 shutdown_connections 2024-08-31T07:26:19.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:19.896+0000 7f98a0c00640 1 -- 172.21.15.32:0/434434290 wait complete. 2024-08-31T07:26:20.038 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:19 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:20.039 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:19 smithi032 ceph-mon[17252]: pgmap v791: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 841 KiB/s wr, 53 op/s 2024-08-31T07:26:20.141 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:19 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:20.141 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:19 smithi084 ceph-mon[22387]: pgmap v791: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 841 KiB/s wr, 53 op/s 2024-08-31T07:26:20.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:19 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:20.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:19 smithi073 ceph-mon[21995]: pgmap v791: 97 pgs: 97 active+clean; 349 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 841 KiB/s wr, 53 op/s 2024-08-31T07:26:20.217 INFO:tasks.workunit.client.0.smithi032.stdout: 0 cleanup 609 sec 2024-08-31T07:26:20.217 INFO:tasks.workunit.client.0.smithi032.stdout: 2024-08-31T07:26:20.217 INFO:tasks.workunit.client.0.smithi032.stdout: Operation Count AvgLat MaxLat 2024-08-31T07:26:20.217 INFO:tasks.workunit.client.0.smithi032.stdout: ---------------------------------------- 2024-08-31T07:26:20.217 INFO:tasks.workunit.client.0.smithi032.stdout: NTCreateX 52669 7.058 32962.432 2024-08-31T07:26:20.217 INFO:tasks.workunit.client.0.smithi032.stdout: Close 38702 0.005 0.308 2024-08-31T07:26:20.217 INFO:tasks.workunit.client.0.smithi032.stdout: Rename 2236 49.365 4878.287 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: Unlink 10617 2.231 4430.709 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: Qpathinfo 47767 1.546 7457.231 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: Qfileinfo 8340 0.079 483.801 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: Qfsinfo 8760 0.307 272.153 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: Sfileinfo 4296 0.052 7.172 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: Find 18465 0.899 9909.096 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: WriteX 26052 0.121 47.994 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: ReadX 82745 0.090 495.611 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: LockX 172 0.769 2.466 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: UnlockX 172 0.669 1.915 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: Flush 3692 3.489 1523.834 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout: 2024-08-31T07:26:20.218 INFO:tasks.workunit.client.0.smithi032.stdout:Throughput 2.74461 MB/sec 1 clients 1 procs max_latency=32962.439 ms 2024-08-31T07:26:20.219 INFO:tasks.workunit.client.0.smithi032.stderr:+ dbench 10 2024-08-31T07:26:20.219 INFO:tasks.workunit.client.0.smithi032.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-08-31T07:26:20.219 INFO:tasks.workunit.client.0.smithi032.stdout: 2024-08-31T07:26:20.219 INFO:tasks.workunit.client.0.smithi032.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-08-31T07:26:20.222 INFO:tasks.workunit.client.0.smithi032.stdout:9 of 10 processes prepared for launch 0 sec 2024-08-31T07:26:20.322 INFO:tasks.workunit.client.0.smithi032.stdout:10 of 10 processes prepared for launch 0 sec 2024-08-31T07:26:20.322 INFO:tasks.workunit.client.0.smithi032.stdout:releasing clients 2024-08-31T07:26:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.960+0000 7f7550c00640 1 -- 172.21.15.32:0/977910994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c105b20 msgr2=0x7f754c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.960+0000 7f7550c00640 1 --2- 172.21.15.32:0/977910994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c105b20 0x7f754c105f30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f753c009860 tx=0x7f753c02a340 comp rx=0 tx=0).stop 2024-08-31T07:26:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.961+0000 7f7550c00640 1 -- 172.21.15.32:0/977910994 shutdown_connections 2024-08-31T07:26:20.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.961+0000 7f7550c00640 1 --2- 172.21.15.32:0/977910994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754c10bdb0 0x7f754c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.961+0000 7f7550c00640 1 --2- 172.21.15.32:0/977910994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f754c10b460 0x7f754c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.961+0000 7f7550c00640 1 --2- 172.21.15.32:0/977910994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c105b20 0x7f754c105f30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.961+0000 7f7550c00640 1 -- 172.21.15.32:0/977910994 >> 172.21.15.32:0/977910994 conn(0x7f754c1015b0 msgr2=0x7f754c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.961+0000 7f7550c00640 1 -- 172.21.15.32:0/977910994 shutdown_connections 2024-08-31T07:26:20.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.961+0000 7f7550c00640 1 -- 172.21.15.32:0/977910994 wait complete. 2024-08-31T07:26:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.962+0000 7f7550c00640 1 Processor -- start 2024-08-31T07:26:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.962+0000 7f7550c00640 1 -- start start 2024-08-31T07:26:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7550c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c105b20 0x7f754c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7550c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f754c10b460 0x7f754c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7550c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754c10bdb0 0x7f754c1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7550c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f754c115a70 con 0x7f754c105b20 2024-08-31T07:26:20.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7550c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1158f0 con 0x7f754c10b460 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7550c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f754c115bf0 con 0x7f754c10bdb0 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7541400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f754c10b460 0x7f754c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7549e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c105b20 0x7f754c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7541400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f754c10b460 0x7f754c19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52470/0 (socket says 172.21.15.32:52470) 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7541400640 1 -- 172.21.15.32:0/929042746 learned_addr learned my addr 172.21.15.32:0/929042746 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f754a800640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754c10bdb0 0x7f754c1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7541400640 1 -- 172.21.15.32:0/929042746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754c10bdb0 msgr2=0x7f754c1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7541400640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754c10bdb0 0x7f754c1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7541400640 1 -- 172.21.15.32:0/929042746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c105b20 msgr2=0x7f754c19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7541400640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c105b20 0x7f754c19c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.963+0000 7f7541400640 1 -- 172.21.15.32:0/929042746 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c1a2450 con 0x7f754c10b460 2024-08-31T07:26:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.964+0000 7f7541400640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f754c10b460 0x7f754c19ca80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f753400e970 tx=0x7f753400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.964+0000 7f7543400640 1 -- 172.21.15.32:0/929042746 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753400cd80 con 0x7f754c10b460 2024-08-31T07:26:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.964+0000 7f7550c00640 1 -- 172.21.15.32:0/929042746 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c1a2740 con 0x7f754c10b460 2024-08-31T07:26:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.964+0000 7f7543400640 1 -- 172.21.15.32:0/929042746 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75340035f0 con 0x7f754c10b460 2024-08-31T07:26:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.964+0000 7f7543400640 1 -- 172.21.15.32:0/929042746 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534010690 con 0x7f754c10b460 2024-08-31T07:26:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.964+0000 7f7550c00640 1 -- 172.21.15.32:0/929042746 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c1a8ea0 con 0x7f754c10b460 2024-08-31T07:26:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.968+0000 7f7543400640 1 -- 172.21.15.32:0/929042746 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f75340108f0 con 0x7f754c10b460 2024-08-31T07:26:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.968+0000 7f7557870740 1 Processor -- start 2024-08-31T07:26:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.969+0000 7f7557870740 1 -- start start 2024-08-31T07:26:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.969+0000 7f7557870740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554ca070 0x56395586f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.969+0000 7f7557870740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 0x56395586fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.969+0000 7f7557870740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x5639558703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.969+0000 7f7557870740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5639557e6770 con 0x5639554a11b0 2024-08-31T07:26:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.969+0000 7f7557870740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5639557d5d00 con 0x5639554a9390 2024-08-31T07:26:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.969+0000 7f7557870740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5639557d9cb0 con 0x5639554ca070 2024-08-31T07:26:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.969+0000 7f754a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554ca070 0x56395586f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f7541400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x5639558703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f754a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554ca070 0x56395586f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:35886/0 (socket says 172.21.15.32:35886) 2024-08-31T07:26:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f754a800640 1 -- 172.21.15.32:0/751167295 learned_addr learned my addr 172.21.15.32:0/751167295 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f7549e00640 1 --2- 172.21.15.32:0/751167295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 0x56395586fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f7541400640 1 -- 172.21.15.32:0/751167295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554ca070 msgr2=0x56395586f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f7541400640 1 --2- 172.21.15.32:0/751167295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554ca070 0x56395586f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f7541400640 1 -- 172.21.15.32:0/751167295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 msgr2=0x56395586fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f7541400640 1 --2- 172.21.15.32:0/751167295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 0x56395586fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f7541400640 1 -- 172.21.15.32:0/751167295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563955535650 con 0x5639554a9390 2024-08-31T07:26:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f7549e00640 1 --2- 172.21.15.32:0/751167295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 0x56395586fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:26:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.970+0000 7f7541400640 1 --2- 172.21.15.32:0/751167295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x5639558703d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f75340e2ea0 tx=0x7f75340ebb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3ef88c2563c799b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.971+0000 7f7543400640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f75300b4d20 0x7f75300b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.971+0000 7f753b000640 1 -- 172.21.15.32:0/751167295 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75340f4070 con 0x5639554a9390 2024-08-31T07:26:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.971+0000 7f7543400640 1 -- 172.21.15.32:0/929042746 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7f75340e5700 con 0x7f754c10b460 2024-08-31T07:26:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.971+0000 7f753b000640 1 -- 172.21.15.32:0/751167295 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f753407f9b0 con 0x5639554a9390 2024-08-31T07:26:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.971+0000 7f754a800640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f75300b4d20 0x7f75300b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.971+0000 7f753b000640 1 -- 172.21.15.32:0/751167295 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75340abc10 con 0x5639554a9390 2024-08-31T07:26:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.972+0000 7f7557870740 1 -- 172.21.15.32:0/751167295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 msgr2=0x5639558703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.972+0000 7f754a800640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f75300b4d20 0x7f75300b7230 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7544005c40 tx=0x7f7544010040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.972+0000 7f7557870740 1 --2- 172.21.15.32:0/751167295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x5639558703d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f75340e2ea0 tx=0x7f75340ebb40 comp rx=0 tx=0).stop 2024-08-31T07:26:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.972+0000 7f7557870740 1 -- 172.21.15.32:0/751167295 shutdown_connections 2024-08-31T07:26:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.972+0000 7f7557870740 1 --2- 172.21.15.32:0/751167295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x5639558703d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.972+0000 7f7557870740 1 --2- 172.21.15.32:0/751167295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 0x56395586fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.972+0000 7f7557870740 1 --2- 172.21.15.32:0/751167295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554ca070 0x56395586f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.972+0000 7f7557870740 1 -- 172.21.15.32:0/751167295 >> 172.21.15.32:0/751167295 conn(0x5639554be460 msgr2=0x56395586d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.972+0000 7f7557870740 1 -- 172.21.15.32:0/751167295 shutdown_connections 2024-08-31T07:26:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.972+0000 7f7557870740 1 -- 172.21.15.32:0/751167295 wait complete. 2024-08-31T07:26:20.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.973+0000 7f7557870740 1 -- start start 2024-08-31T07:26:20.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.973+0000 7f7557870740 1 Processor -- start 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7557870740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 0x56395587b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7557870740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x56395587ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7557870740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554ca070 0x563955887420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7557870740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563955638700 con 0x5639554ca070 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7557870740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56395575ccb0 con 0x5639554a9390 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7557870740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56395587e130 con 0x5639554a11b0 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7557870740 10 client.0 _ll_register_callbacks cb 0x563955782430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7541400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x56395587ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7549e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 0x56395587b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7541400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x56395587ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52488/0 (socket says 172.21.15.32:52488) 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.974+0000 7f7541400640 1 -- 172.21.15.32:0/1490419377 learned_addr learned my addr 172.21.15.32:0/1490419377 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.975+0000 7f754a800640 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554ca070 0x563955887420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.975+0000 7f7541400640 1 -- 172.21.15.32:0/1490419377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 msgr2=0x56395587b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.975+0000 7f7541400640 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 0x56395587b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.975+0000 7f7541400640 1 -- 172.21.15.32:0/1490419377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554ca070 msgr2=0x563955887420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.975+0000 7f7541400640 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554ca070 0x563955887420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.975+0000 7f7541400640 1 -- 172.21.15.32:0/1490419377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56395579d870 con 0x5639554a9390 2024-08-31T07:26:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.975+0000 7f7549e00640 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 0x56395587b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.975+0000 7f7541400640 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x56395587ba20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f75340098c0 tx=0x7f75340a7d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62ce646b2d0d7586 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.975+0000 7f752f400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.975+0000 7f752f400640 1 -- 172.21.15.32:0/1490419377 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75340f4030 con 0x5639554a9390 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.976+0000 7f7557870740 1 -- 172.21.15.32:0/1490419377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f753407f9b0 con 0x5639554a9390 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.976+0000 7f752f400640 1 -- 172.21.15.32:0/1490419377 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7f75340a80d0 con 0x5639554a9390 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.976+0000 7f752f400640 1 -- 172.21.15.32:0/1490419377 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f75340ae030 con 0x5639554a9390 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.976+0000 7f7557870740 10 client.28388 fetch_fsmap learned FSMap version 48 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.976+0000 7f7557870740 1 -- 172.21.15.32:0/1490419377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56395588a110 con 0x5639554a9390 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.976+0000 7f752f400640 1 -- 172.21.15.32:0/1490419377 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f753407f9b0 con 0x5639554a9390 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.976+0000 7f752f400640 1 -- 172.21.15.32:0/1490419377 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8382+0+0 (secure 0 0 0) 0x7f7534066ea0 con 0x5639554a9390 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.977+0000 7f7557870740 10 client.28388 fetch_fsmap finished waiting for FSMap version 48 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.977+0000 7f7557870740 10 client.28388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:26:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.977+0000 7f7557870740 20 client.28388 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:20.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.977+0000 7f7557870740 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5639554889f0 0x56395588d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:20.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.977+0000 7f7557870740 4 client.28388 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:26:20.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.977+0000 7f7557870740 1 -- 172.21.15.32:0/1490419377 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639557e86c0 con 0x5639554889f0 2024-08-31T07:26:20.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.977+0000 7f7549e00640 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5639554889f0 0x56395588d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:20.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.978+0000 7f7549e00640 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5639554889f0 0x56395588d470 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e247dd262ef76eb server_cookie=acfdb9cad76d5451 in_seq=0 out_seq=0 2024-08-31T07:26:20.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.978+0000 7f752f400640 10 client.28388 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f752f400640 1 -- 172.21.15.32:0/1490419377 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5639557e86c0 con 0x5639554889f0 2024-08-31T07:26:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f752f400640 10 client.28388 handle_command_reply: tid=0 2024-08-31T07:26:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 client.28388 shutdown 2024-08-31T07:26:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 -- 172.21.15.32:0/1490419377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 msgr2=0x56395587ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x56395587ba20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f75340098c0 tx=0x7f75340a7d00 comp rx=0 tx=0).stop 2024-08-31T07:26:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 -- 172.21.15.32:0/1490419377 shutdown_connections 2024-08-31T07:26:20.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5639554889f0 0x56395588d470 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554ca070 0x563955887420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554a9390 0x56395587ba20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 --2- 172.21.15.32:0/1490419377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 0x56395587b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:20.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f752f400640 0 client.28388 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:20.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 -- 172.21.15.32:0/1490419377 >> 172.21.15.32:0/1490419377 conn(0x5639554be460 msgr2=0x563955870980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:20.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 -- 172.21.15.32:0/1490419377 shutdown_connections 2024-08-31T07:26:20.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.980+0000 7f7557870740 1 -- 172.21.15.32:0/1490419377 wait complete. 2024-08-31T07:26:20.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:20.981+0000 7f7557870740 20 client.28388 trim_cache size 0 max 16384 2024-08-31T07:26:21.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f7557870740 1 Processor -- start 2024-08-31T07:26:21.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f7557870740 1 -- start start 2024-08-31T07:26:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f7557870740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 0x563955871340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f7557870740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a9390 0x563955875a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f7557870740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395587cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f7557870740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1b1610 con 0x5639554a9390 2024-08-31T07:26:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f7557870740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1b3240 con 0x5639554ca070 2024-08-31T07:26:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f7557870740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1b1ce0 con 0x5639554a11b0 2024-08-31T07:26:21.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f754a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395587cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:21.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f7541400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a9390 0x563955875a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:21.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.000+0000 7f7549e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 0x563955871340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:21.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.001+0000 7f754a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395587cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52500/0 (socket says 172.21.15.32:52500) 2024-08-31T07:26:21.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.001+0000 7f754a800640 1 -- 172.21.15.32:0/1064098084 learned_addr learned my addr 172.21.15.32:0/1064098084 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:21.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.001+0000 7f7549e00640 1 -- 172.21.15.32:0/1064098084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 msgr2=0x56395587cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:21.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.001+0000 7f7549e00640 1 --2- 172.21.15.32:0/1064098084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395587cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.001+0000 7f7549e00640 1 -- 172.21.15.32:0/1064098084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a9390 msgr2=0x563955875a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:21.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.001+0000 7f7549e00640 1 --2- 172.21.15.32:0/1064098084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a9390 0x563955875a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.001+0000 7f7549e00640 1 -- 172.21.15.32:0/1064098084 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56395587c5a0 con 0x5639554a11b0 2024-08-31T07:26:21.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.001+0000 7f7549e00640 1 --2- 172.21.15.32:0/1064098084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 0x563955871340 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f753c005940 tx=0x7f753c0058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d339395f570239e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:21.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.002+0000 7f753a600640 1 -- 172.21.15.32:0/1064098084 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c04a020 con 0x5639554a11b0 2024-08-31T07:26:21.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.002+0000 7f753a600640 1 -- 172.21.15.32:0/1064098084 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f753c030900 con 0x5639554a11b0 2024-08-31T07:26:21.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.002+0000 7f753a600640 1 -- 172.21.15.32:0/1064098084 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c037040 con 0x5639554a11b0 2024-08-31T07:26:21.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.003+0000 7f7557870740 1 -- 172.21.15.32:0/1064098084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 msgr2=0x563955871340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:21.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.003+0000 7f7557870740 1 --2- 172.21.15.32:0/1064098084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 0x563955871340 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f753c005940 tx=0x7f753c0058d0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.003+0000 7f7557870740 1 -- 172.21.15.32:0/1064098084 shutdown_connections 2024-08-31T07:26:21.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.003+0000 7f7557870740 1 --2- 172.21.15.32:0/1064098084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395587cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.003+0000 7f7557870740 1 --2- 172.21.15.32:0/1064098084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a9390 0x563955875a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.003+0000 7f7557870740 1 --2- 172.21.15.32:0/1064098084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a11b0 0x563955871340 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.003+0000 7f7557870740 1 -- 172.21.15.32:0/1064098084 >> 172.21.15.32:0/1064098084 conn(0x5639554be460 msgr2=0x56395587c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:21.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.003+0000 7f7557870740 1 -- 172.21.15.32:0/1064098084 shutdown_connections 2024-08-31T07:26:21.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.003+0000 7f7557870740 1 -- 172.21.15.32:0/1064098084 wait complete. 2024-08-31T07:26:21.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.004+0000 7f7557870740 1 -- start start 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.004+0000 7f7557870740 1 Processor -- start 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7557870740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 0x563955890b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7557870740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a9390 0x56395588f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7557870740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395588f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7557870740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1b2ab0 con 0x5639554a11b0 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7557870740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1b23b0 con 0x5639554ca070 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7557870740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1af610 con 0x5639554a9390 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7557870740 10 client.0 _ll_register_callbacks cb 0x7f75340e5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f754a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395588f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f754a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395588f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52504/0 (socket says 172.21.15.32:52504) 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f754a800640 1 -- 172.21.15.32:0/1685921431 learned_addr learned my addr 172.21.15.32:0/1685921431 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7549e00640 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 0x563955890b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7541400640 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a9390 0x56395588f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:21.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7541400640 1 -- 172.21.15.32:0/1685921431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 msgr2=0x56395588f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:21.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7541400640 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395588f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7541400640 1 -- 172.21.15.32:0/1685921431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 msgr2=0x563955890b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7541400640 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 0x563955890b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7541400640 1 -- 172.21.15.32:0/1685921431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56395588fbe0 con 0x5639554a9390 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f7549e00640 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 0x563955890b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.005+0000 7f754a800640 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395588f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.006+0000 7f7541400640 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a9390 0x56395588f030 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f75340a90b0 tx=0x7f75340b04f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32fa18b836f2466c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.006+0000 7f752f400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.006+0000 7f752f400640 1 -- 172.21.15.32:0/1685921431 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75340f4070 con 0x5639554a9390 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.006+0000 7f752f400640 1 -- 172.21.15.32:0/1685921431 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7f75340a8430 con 0x5639554a9390 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.006+0000 7f7557870740 1 -- 172.21.15.32:0/1685921431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f753c030900 con 0x5639554a9390 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.006+0000 7f752f400640 1 -- 172.21.15.32:0/1685921431 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7534065350 con 0x5639554a9390 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.006+0000 7f7557870740 10 client.28396 fetch_fsmap learned FSMap version 48 2024-08-31T07:26:21.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.006+0000 7f7557870740 1 -- 172.21.15.32:0/1685921431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5639558f8a10 con 0x5639554a9390 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.007+0000 7f752f400640 1 -- 172.21.15.32:0/1685921431 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f753c030900 con 0x5639554a9390 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.007+0000 7f752f400640 1 -- 172.21.15.32:0/1685921431 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 48) ==== 8382+0+0 (secure 0 0 0) 0x7f75340930d0 con 0x5639554a9390 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.007+0000 7f7557870740 10 client.28396 fetch_fsmap finished waiting for FSMap version 48 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.007+0000 7f7557870740 10 client.28396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.007+0000 7f7557870740 20 client.28396 populate_metadata read hostname 'smithi032' 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.007+0000 7f7557870740 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56395545dda0 0x5639558faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.007+0000 7f7557870740 4 client.28396 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.007+0000 7f7557870740 1 -- 172.21.15.32:0/1685921431 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5639557e4160 con 0x56395545dda0 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.007+0000 7f7549e00640 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56395545dda0 0x5639558faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.008+0000 7f7549e00640 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56395545dda0 0x5639558faf60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe15a7d00725c8e3 server_cookie=d1991c266fad3e1d in_seq=0 out_seq=0 2024-08-31T07:26:21.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.008+0000 7f752f400640 10 client.28396 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f752f400640 1 -- 172.21.15.32:0/1685921431 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5114 (crc 0 0 0) 0x5639557e4160 con 0x56395545dda0 2024-08-31T07:26:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f752f400640 10 client.28396 handle_command_reply: tid=0 2024-08-31T07:26:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 client.28396 shutdown 2024-08-31T07:26:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 -- 172.21.15.32:0/1685921431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a9390 msgr2=0x56395588f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a9390 0x56395588f030 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f75340a90b0 tx=0x7f75340b04f0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 -- 172.21.15.32:0/1685921431 shutdown_connections 2024-08-31T07:26:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56395545dda0 0x5639558faf60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5639554ca070 0x56395588f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5639554a9390 0x56395588f030 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 --2- 172.21.15.32:0/1685921431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639554a11b0 0x563955890b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 -- 172.21.15.32:0/1685921431 >> 172.21.15.32:0/1685921431 conn(0x5639554be460 msgr2=0x56395587d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f752f400640 0 client.28396 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:26:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 -- 172.21.15.32:0/1685921431 shutdown_connections 2024-08-31T07:26:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 1 -- 172.21.15.32:0/1685921431 wait complete. 2024-08-31T07:26:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.012+0000 7f7557870740 20 client.28396 trim_cache size 0 max 16384 2024-08-31T07:26:21.015 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "ce8b7231-1fea-412d-a295-8119d9478cfa", 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:26:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:26:21.026 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.013+0000 7f7550c00640 1 -- 172.21.15.32:0/929042746 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f75300b4d20 msgr2=0x7f75300b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.013+0000 7f7550c00640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f75300b4d20 0x7f75300b7230 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7544005c40 tx=0x7f7544010040 comp rx=0 tx=0).stop 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.013+0000 7f7550c00640 1 -- 172.21.15.32:0/929042746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f754c10b460 msgr2=0x7f754c19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.013+0000 7f7550c00640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f754c10b460 0x7f754c19ca80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f753400e970 tx=0x7f753400ee30 comp rx=0 tx=0).stop 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.013+0000 7f7550c00640 1 -- 172.21.15.32:0/929042746 shutdown_connections 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.013+0000 7f7550c00640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f75300b4d20 0x7f75300b7230 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.013+0000 7f7550c00640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f754c10bdb0 0x7f754c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.013+0000 7f7550c00640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f754c10b460 0x7f754c19ca80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.013+0000 7f7550c00640 1 --2- 172.21.15.32:0/929042746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f754c105b20 0x7f754c19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.013+0000 7f7550c00640 1 -- 172.21.15.32:0/929042746 >> 172.21.15.32:0/929042746 conn(0x7f754c1015b0 msgr2=0x7f754c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:26:21.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.014+0000 7f7550c00640 1 -- 172.21.15.32:0/929042746 shutdown_connections 2024-08-31T07:26:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:26:21.014+0000 7f7550c00640 1 -- 172.21.15.32:0/929042746 wait complete. 2024-08-31T07:26:21.322 INFO:tasks.workunit.client.0.smithi032.stdout: 10 398 141.36 MB/sec warmup 1 sec latency 44.922 ms 2024-08-31T07:26:21.419 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '63a6ed69', total seconds elapsed: 21.0 2024-08-31T07:26:22.322 INFO:tasks.workunit.client.0.smithi032.stdout: 10 899 105.25 MB/sec warmup 2 sec latency 293.802 ms 2024-08-31T07:26:22.353 INFO:tasks.cephfs.filesystem:scrub status for tag:ce8b7231-1fea-412d-a295-8119d9478cfa - {'path': '/', 'tag': 'ce8b7231-1fea-412d-a295-8119d9478cfa', 'options': 'recursive,force'} 2024-08-31T07:26:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:22 smithi073 ceph-mon[21995]: pgmap v792: 97 pgs: 97 active+clean; 350 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 735 KiB/s wr, 39 op/s 2024-08-31T07:26:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:22 smithi084 ceph-mon[22387]: pgmap v792: 97 pgs: 97 active+clean; 350 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 735 KiB/s wr, 39 op/s 2024-08-31T07:26:22.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:22 smithi032 ceph-mon[17252]: pgmap v792: 97 pgs: 97 active+clean; 350 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 735 KiB/s wr, 39 op/s 2024-08-31T07:26:23.322 INFO:tasks.workunit.client.0.smithi032.stdout: 10 1559 83.46 MB/sec warmup 3 sec latency 1293.846 ms 2024-08-31T07:26:24.322 INFO:tasks.workunit.client.0.smithi032.stdout: 10 2114 65.72 MB/sec warmup 4 sec latency 1943.311 ms 2024-08-31T07:26:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:24 smithi073 ceph-mon[21995]: pgmap v793: 97 pgs: 97 active+clean; 351 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 969 KiB/s wr, 57 op/s 2024-08-31T07:26:24.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:24 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:24 smithi084 ceph-mon[22387]: pgmap v793: 97 pgs: 97 active+clean; 351 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 969 KiB/s wr, 57 op/s 2024-08-31T07:26:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:24 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:24.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:24 smithi032 ceph-mon[17252]: pgmap v793: 97 pgs: 97 active+clean; 351 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 969 KiB/s wr, 57 op/s 2024-08-31T07:26:24.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:24 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:25.322 INFO:tasks.workunit.client.0.smithi032.stdout: 10 2740 64.45 MB/sec warmup 5 sec latency 89.996 ms 2024-08-31T07:26:26.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 3471 62.62 MB/sec warmup 6 sec latency 115.820 ms 2024-08-31T07:26:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:26 smithi073 ceph-mon[21995]: pgmap v794: 97 pgs: 97 active+clean; 355 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-31T07:26:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:26 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:26 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:26:26.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:26 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:26:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:26 smithi084 ceph-mon[22387]: pgmap v794: 97 pgs: 97 active+clean; 355 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-31T07:26:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:26 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:26 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:26:26.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:26 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:26:26.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:26 smithi032 ceph-mon[17252]: pgmap v794: 97 pgs: 97 active+clean; 355 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-31T07:26:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:26 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:26 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:26:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:26 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:26:27.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 4130 56.88 MB/sec warmup 7 sec latency 683.547 ms 2024-08-31T07:26:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:27 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:26:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:27 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:26:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:27 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:26:27.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:27 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:27.444 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:27 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:26:27.444 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:27 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:26:27.444 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:27 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:26:27.444 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:27 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:27 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:26:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:27 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:26:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:27 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:26:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:27 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:28.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 5339 57.65 MB/sec warmup 8 sec latency 1683.604 ms 2024-08-31T07:26:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:28 smithi073 ceph-mon[21995]: pgmap v795: 97 pgs: 97 active+clean; 367 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2024-08-31T07:26:28.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:28 smithi084 ceph-mon[22387]: pgmap v795: 97 pgs: 97 active+clean; 367 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2024-08-31T07:26:28.505 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:28 smithi032 ceph-mon[17252]: pgmap v795: 97 pgs: 97 active+clean; 367 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 194 op/s 2024-08-31T07:26:29.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 5978 54.58 MB/sec warmup 9 sec latency 2683.666 ms 2024-08-31T07:26:29.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:29 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:29.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:29 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:29.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:29 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:30.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 7265 55.73 MB/sec warmup 10 sec latency 3325.882 ms 2024-08-31T07:26:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:30 smithi084 ceph-mon[22387]: pgmap v796: 97 pgs: 97 active+clean; 373 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-08-31T07:26:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:30 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:30 smithi032 ceph-mon[17252]: pgmap v796: 97 pgs: 97 active+clean; 373 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-08-31T07:26:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:30 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:30 smithi073 ceph-mon[21995]: pgmap v796: 97 pgs: 97 active+clean; 373 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-08-31T07:26:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:30 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:31.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 8329 57.66 MB/sec warmup 11 sec latency 99.986 ms 2024-08-31T07:26:31.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:31 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:31.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:31 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:26:31.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:31 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:31.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:31 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:26:31.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:31 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:31.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:31 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:26:32.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 9087 54.54 MB/sec warmup 12 sec latency 148.279 ms 2024-08-31T07:26:32.457 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:32 smithi032 ceph-mon[17252]: pgmap v797: 97 pgs: 97 active+clean; 380 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 255 op/s 2024-08-31T07:26:32.457 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:32 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:32.457 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:32 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:26:32.457 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:32 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:26:32.457 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:32 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:26:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:32 smithi084 ceph-mon[22387]: pgmap v797: 97 pgs: 97 active+clean; 380 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 255 op/s 2024-08-31T07:26:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:32 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:32 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:26:32.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:32 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:26:32.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:32 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:26:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:32 smithi073 ceph-mon[21995]: pgmap v797: 97 pgs: 97 active+clean; 380 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 255 op/s 2024-08-31T07:26:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:32 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:26:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:32 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:26:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:32 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:26:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:32 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:26:33.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 10836 58.27 MB/sec warmup 13 sec latency 38.255 ms 2024-08-31T07:26:34.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 11805 58.68 MB/sec warmup 14 sec latency 173.050 ms 2024-08-31T07:26:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:34 smithi084 ceph-mon[22387]: pgmap v798: 97 pgs: 97 active+clean; 397 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-08-31T07:26:34.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:34 smithi032 ceph-mon[17252]: pgmap v798: 97 pgs: 97 active+clean; 397 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-08-31T07:26:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:34 smithi073 ceph-mon[21995]: pgmap v798: 97 pgs: 97 active+clean; 397 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 389 op/s 2024-08-31T07:26:35.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 12844 57.34 MB/sec warmup 15 sec latency 73.097 ms 2024-08-31T07:26:36.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 14364 59.41 MB/sec warmup 16 sec latency 78.258 ms 2024-08-31T07:26:36.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:36 smithi084 ceph-mon[22387]: pgmap v799: 97 pgs: 97 active+clean; 408 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 477 op/s 2024-08-31T07:26:36.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:36 smithi032 ceph-mon[17252]: pgmap v799: 97 pgs: 97 active+clean; 408 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 477 op/s 2024-08-31T07:26:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:36 smithi073 ceph-mon[21995]: pgmap v799: 97 pgs: 97 active+clean; 408 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 477 op/s 2024-08-31T07:26:37.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 15358 59.05 MB/sec warmup 17 sec latency 62.820 ms 2024-08-31T07:26:38.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 16567 58.83 MB/sec warmup 18 sec latency 107.088 ms 2024-08-31T07:26:38.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:38 smithi084 ceph-mon[22387]: pgmap v800: 97 pgs: 97 active+clean; 425 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.5 MiB/s wr, 585 op/s 2024-08-31T07:26:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:38 smithi032 ceph-mon[17252]: pgmap v800: 97 pgs: 97 active+clean; 425 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.5 MiB/s wr, 585 op/s 2024-08-31T07:26:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:38 smithi073 ceph-mon[21995]: pgmap v800: 97 pgs: 97 active+clean; 425 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.5 MiB/s wr, 585 op/s 2024-08-31T07:26:39.323 INFO:tasks.workunit.client.0.smithi032.stdout: 10 17877 60.06 MB/sec warmup 19 sec latency 93.099 ms 2024-08-31T07:26:40.324 INFO:tasks.workunit.client.0.smithi032.stdout: 10 18791 59.65 MB/sec warmup 20 sec latency 60.013 ms 2024-08-31T07:26:40.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:40 smithi084 ceph-mon[22387]: pgmap v801: 97 pgs: 97 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.1 MiB/s wr, 589 op/s 2024-08-31T07:26:40.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:40 smithi032 ceph-mon[17252]: pgmap v801: 97 pgs: 97 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.1 MiB/s wr, 589 op/s 2024-08-31T07:26:40.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:40 smithi073 ceph-mon[21995]: pgmap v801: 97 pgs: 97 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.1 MiB/s wr, 589 op/s 2024-08-31T07:26:41.324 INFO:tasks.workunit.client.0.smithi032.stdout: 10 19946 59.30 MB/sec warmup 21 sec latency 48.175 ms 2024-08-31T07:26:42.324 INFO:tasks.workunit.client.0.smithi032.stdout: 10 21516 60.77 MB/sec warmup 22 sec latency 59.545 ms 2024-08-31T07:26:42.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:42 smithi084 ceph-mon[22387]: pgmap v802: 97 pgs: 97 active+clean; 472 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 678 op/s 2024-08-31T07:26:42.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:42 smithi032 ceph-mon[17252]: pgmap v802: 97 pgs: 97 active+clean; 472 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 678 op/s 2024-08-31T07:26:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:42 smithi073 ceph-mon[21995]: pgmap v802: 97 pgs: 97 active+clean; 472 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 678 op/s 2024-08-31T07:26:43.324 INFO:tasks.workunit.client.0.smithi032.stdout: 10 22501 60.40 MB/sec warmup 23 sec latency 74.864 ms 2024-08-31T07:26:44.324 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23193 59.32 MB/sec warmup 24 sec latency 198.887 ms 2024-08-31T07:26:44.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:44 smithi084 ceph-mon[22387]: pgmap v803: 97 pgs: 97 active+clean; 489 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 732 op/s 2024-08-31T07:26:44.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:44 smithi032 ceph-mon[17252]: pgmap v803: 97 pgs: 97 active+clean; 489 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 732 op/s 2024-08-31T07:26:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:44 smithi073 ceph-mon[21995]: pgmap v803: 97 pgs: 97 active+clean; 489 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 732 op/s 2024-08-31T07:26:45.324 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23257 57.02 MB/sec warmup 25 sec latency 688.366 ms 2024-08-31T07:26:46.279 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:46 smithi032 ceph-mon[17252]: pgmap v804: 97 pgs: 97 active+clean; 521 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 12 MiB/s wr, 683 op/s 2024-08-31T07:26:46.324 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23261 54.83 MB/sec warmup 26 sec latency 1688.437 ms 2024-08-31T07:26:46.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:46 smithi084 ceph-mon[22387]: pgmap v804: 97 pgs: 97 active+clean; 521 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 12 MiB/s wr, 683 op/s 2024-08-31T07:26:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:46 smithi073 ceph-mon[21995]: pgmap v804: 97 pgs: 97 active+clean; 521 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 12 MiB/s wr, 683 op/s 2024-08-31T07:26:47.326 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23261 52.79 MB/sec warmup 27 sec latency 2690.789 ms 2024-08-31T07:26:48.326 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23261 50.91 MB/sec warmup 28 sec latency 3690.832 ms 2024-08-31T07:26:48.506 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:48 smithi032 ceph-mon[17252]: pgmap v805: 97 pgs: 97 active+clean; 686 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 25 MiB/s wr, 864 op/s 2024-08-31T07:26:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:48 smithi073 ceph-mon[21995]: pgmap v805: 97 pgs: 97 active+clean; 686 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 25 MiB/s wr, 864 op/s 2024-08-31T07:26:48.720 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 35.0 2024-08-31T07:26:48.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:26:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:48 smithi084 ceph-mon[22387]: pgmap v805: 97 pgs: 97 active+clean; 686 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 25 MiB/s wr, 864 op/s 2024-08-31T07:26:49.164 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:26:49.326 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23261 49.15 MB/sec warmup 29 sec latency 4690.882 ms 2024-08-31T07:26:50.326 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23261 47.51 MB/sec warmup 30 sec latency 5690.949 ms 2024-08-31T07:26:50.520 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:50 smithi032 ceph-mon[17252]: pgmap v806: 97 pgs: 97 active+clean; 736 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 28 MiB/s wr, 788 op/s 2024-08-31T07:26:50.541 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:50 smithi084 ceph-mon[22387]: pgmap v806: 97 pgs: 97 active+clean; 736 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 28 MiB/s wr, 788 op/s 2024-08-31T07:26:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:50 smithi073 ceph-mon[21995]: pgmap v806: 97 pgs: 97 active+clean; 736 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 28 MiB/s wr, 788 op/s 2024-08-31T07:26:51.326 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23262 45.98 MB/sec warmup 31 sec latency 6691.010 ms 2024-08-31T07:26:52.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23262 44.55 MB/sec warmup 32 sec latency 7691.062 ms 2024-08-31T07:26:52.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:26:52.360 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:52 smithi032 ceph-mon[17252]: pgmap v807: 97 pgs: 97 active+clean; 776 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 33 MiB/s wr, 797 op/s 2024-08-31T07:26:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:52 smithi073 ceph-mon[21995]: pgmap v807: 97 pgs: 97 active+clean; 776 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 33 MiB/s wr, 797 op/s 2024-08-31T07:26:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:52 smithi084 ceph-mon[22387]: pgmap v807: 97 pgs: 97 active+clean; 776 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 33 MiB/s wr, 797 op/s 2024-08-31T07:26:53.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23262 43.20 MB/sec warmup 33 sec latency 8691.139 ms 2024-08-31T07:26:54.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23262 41.92 MB/sec warmup 34 sec latency 9691.186 ms 2024-08-31T07:26:54.502 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:54 smithi032 ceph-mon[17252]: pgmap v808: 97 pgs: 97 active+clean; 758 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 29 MiB/s wr, 685 op/s 2024-08-31T07:26:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:54 smithi073 ceph-mon[21995]: pgmap v808: 97 pgs: 97 active+clean; 758 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 29 MiB/s wr, 685 op/s 2024-08-31T07:26:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:54 smithi084 ceph-mon[22387]: pgmap v808: 97 pgs: 97 active+clean; 758 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 29 MiB/s wr, 685 op/s 2024-08-31T07:26:55.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23281 40.74 MB/sec warmup 35 sec latency 10691.254 ms 2024-08-31T07:26:56.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23322 39.65 MB/sec warmup 36 sec latency 11691.317 ms 2024-08-31T07:26:56.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:56 smithi032 ceph-mon[17252]: pgmap v809: 97 pgs: 97 active+clean; 735 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 28 MiB/s wr, 620 op/s 2024-08-31T07:26:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:56 smithi073 ceph-mon[21995]: pgmap v809: 97 pgs: 97 active+clean; 735 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 28 MiB/s wr, 620 op/s 2024-08-31T07:26:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:56 smithi084 ceph-mon[22387]: pgmap v809: 97 pgs: 97 active+clean; 735 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 28 MiB/s wr, 620 op/s 2024-08-31T07:26:57.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23329 38.58 MB/sec warmup 37 sec latency 12691.374 ms 2024-08-31T07:26:58.071 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:26:57 smithi032 ceph-mon[17252]: pgmap v810: 97 pgs: 97 active+clean; 717 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 560 op/s 2024-08-31T07:26:58.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:26:57 smithi073 ceph-mon[21995]: pgmap v810: 97 pgs: 97 active+clean; 717 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 560 op/s 2024-08-31T07:26:58.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:26:57 smithi084 ceph-mon[22387]: pgmap v810: 97 pgs: 97 active+clean; 717 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 560 op/s 2024-08-31T07:26:58.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23381 37.75 MB/sec warmup 38 sec latency 13691.429 ms 2024-08-31T07:26:59.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23381 36.79 MB/sec warmup 39 sec latency 14691.498 ms 2024-08-31T07:27:00.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23381 35.87 MB/sec warmup 40 sec latency 15691.561 ms 2024-08-31T07:27:00.396 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:00 smithi032 ceph-mon[17252]: pgmap v811: 97 pgs: 97 active+clean; 708 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 11 MiB/s wr, 288 op/s 2024-08-31T07:27:00.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:00 smithi073 ceph-mon[21995]: pgmap v811: 97 pgs: 97 active+clean; 708 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 11 MiB/s wr, 288 op/s 2024-08-31T07:27:00.492 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:00 smithi084 ceph-mon[22387]: pgmap v811: 97 pgs: 97 active+clean; 708 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 11 MiB/s wr, 288 op/s 2024-08-31T07:27:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- 172.21.15.32:0/1529080750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c0754f0 msgr2=0x7f2b6c075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:00.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 --2- 172.21.15.32:0/1529080750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c0754f0 0x7f2b6c075900 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009900 tx=0x7f2b6002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- 172.21.15.32:0/1529080750 shutdown_connections 2024-08-31T07:27:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 --2- 172.21.15.32:0/1529080750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10fee0 0x7f2b6c112300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 --2- 172.21.15.32:0/1529080750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c075e40 0x7f2b6c10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 --2- 172.21.15.32:0/1529080750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c0754f0 0x7f2b6c075900 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- 172.21.15.32:0/1529080750 >> 172.21.15.32:0/1529080750 conn(0x7f2b6c0ff330 msgr2=0x7f2b6c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- 172.21.15.32:0/1529080750 shutdown_connections 2024-08-31T07:27:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- 172.21.15.32:0/1529080750 wait complete. 2024-08-31T07:27:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 Processor -- start 2024-08-31T07:27:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- start start 2024-08-31T07:27:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c0754f0 0x7f2b6c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c075e40 0x7f2b6c19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10fee0 0x7f2b6c1a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c115a50 con 0x7f2b6c075e40 2024-08-31T07:27:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1158d0 con 0x7f2b6c0754f0 2024-08-31T07:27:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c115bd0 con 0x7f2b6c10fee0 2024-08-31T07:27:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c0754f0 0x7f2b6c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c075e40 0x7f2b6c19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c0754f0 0x7f2b6c19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:43824/0 (socket says 172.21.15.32:43824) 2024-08-31T07:27:00.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71000640 1 -- 172.21.15.32:0/450799411 learned_addr learned my addr 172.21.15.32:0/450799411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:00.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71a00640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10fee0 0x7f2b6c1a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:00.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71000640 1 -- 172.21.15.32:0/450799411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10fee0 msgr2=0x7f2b6c1a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:00.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71000640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10fee0 0x7f2b6c1a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71000640 1 -- 172.21.15.32:0/450799411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c075e40 msgr2=0x7f2b6c19c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:00.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71000640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c075e40 0x7f2b6c19c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71000640 1 -- 172.21.15.32:0/450799411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c1a2390 con 0x7f2b6c0754f0 2024-08-31T07:27:00.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71a00640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10fee0 0x7f2b6c1a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:27:00.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b71000640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c0754f0 0x7f2b6c19c420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2b60005940 tx=0x7f2b600040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:00.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b69600640 1 -- 172.21.15.32:0/450799411 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6003f030 con 0x7f2b6c0754f0 2024-08-31T07:27:00.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b69600640 1 -- 172.21.15.32:0/450799411 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b600044c0 con 0x7f2b6c0754f0 2024-08-31T07:27:00.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- 172.21.15.32:0/450799411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c1a2620 con 0x7f2b6c0754f0 2024-08-31T07:27:00.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b69600640 1 -- 172.21.15.32:0/450799411 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60035070 con 0x7f2b6c0754f0 2024-08-31T07:27:00.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.508+0000 7f2b73a00640 1 -- 172.21.15.32:0/450799411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c1a8de0 con 0x7f2b6c0754f0 2024-08-31T07:27:00.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.514+0000 7f2b73a00640 1 -- 172.21.15.32:0/450799411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6c075900 con 0x7f2b6c0754f0 2024-08-31T07:27:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.514+0000 7f2b69600640 1 -- 172.21.15.32:0/450799411 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2b60042030 con 0x7f2b6c0754f0 2024-08-31T07:27:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.514+0000 7f2b69600640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b540b4d20 0x7f2b540b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.514+0000 7f2b69600640 1 -- 172.21.15.32:0/450799411 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7f2b60105ae0 con 0x7f2b6c0754f0 2024-08-31T07:27:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.514+0000 7f2b6be00640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b540b4d20 0x7f2b540b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.519+0000 7f2b6be00640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b540b4d20 0x7f2b540b7230 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2b580095f0 tx=0x7f2b58005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:00.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.519+0000 7f2b69600640 1 -- 172.21.15.32:0/450799411 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2b600cc8c0 con 0x7f2b6c0754f0 2024-08-31T07:27:00.709 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:00.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.768+0000 7f2b73a00640 1 -- 172.21.15.32:0/450799411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b6c19d550 con 0x7f2b6c0754f0 2024-08-31T07:27:00.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.769+0000 7f2b69600640 1 -- 172.21.15.32:0/450799411 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11979 (secure 0 0 0) 0x7f2b600cc1c0 con 0x7f2b6c0754f0 2024-08-31T07:27:00.770 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:27:00.770 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":55,"btime":"2024-08-31T07:26:31:188885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:26:30.959002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:27:00.770 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 55 2024-08-31T07:27:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 -- 172.21.15.32:0/450799411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b540b4d20 msgr2=0x7f2b540b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b540b4d20 0x7f2b540b7230 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2b580095f0 tx=0x7f2b58005f90 comp rx=0 tx=0).stop 2024-08-31T07:27:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 -- 172.21.15.32:0/450799411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c0754f0 msgr2=0x7f2b6c19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c0754f0 0x7f2b6c19c420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2b60005940 tx=0x7f2b600040c0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 -- 172.21.15.32:0/450799411 shutdown_connections 2024-08-31T07:27:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2b540b4d20 0x7f2b540b7230 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b6c10fee0 0x7f2b6c1a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c075e40 0x7f2b6c19c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 --2- 172.21.15.32:0/450799411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b6c0754f0 0x7f2b6c19c420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 -- 172.21.15.32:0/450799411 >> 172.21.15.32:0/450799411 conn(0x7f2b6c0ff330 msgr2=0x7f2b6c0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 -- 172.21.15.32:0/450799411 shutdown_connections 2024-08-31T07:27:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:00.771+0000 7f2b73a00640 1 -- 172.21.15.32:0/450799411 wait complete. 2024-08-31T07:27:01.259 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:01 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/450799411' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:27:01.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23381 34.99 MB/sec warmup 41 sec latency 16691.624 ms 2024-08-31T07:27:01.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:01 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/450799411' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:27:01.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:01 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/450799411' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:27:02.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23381 34.16 MB/sec warmup 42 sec latency 17691.687 ms 2024-08-31T07:27:02.393 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:02 smithi032 ceph-mon[17252]: pgmap v812: 97 pgs: 97 active+clean; 694 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 5.9 MiB/s wr, 262 op/s 2024-08-31T07:27:02.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:02 smithi073 ceph-mon[21995]: pgmap v812: 97 pgs: 97 active+clean; 694 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 5.9 MiB/s wr, 262 op/s 2024-08-31T07:27:02.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:02 smithi084 ceph-mon[22387]: pgmap v812: 97 pgs: 97 active+clean; 694 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 5.9 MiB/s wr, 262 op/s 2024-08-31T07:27:03.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23381 33.36 MB/sec warmup 43 sec latency 18691.737 ms 2024-08-31T07:27:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.098+0000 7f6998400640 1 -- 172.21.15.32:0/3583059811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6990109260 msgr2=0x7f6990109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.098+0000 7f6998400640 1 --2- 172.21.15.32:0/3583059811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6990109260 0x7f6990109670 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f697c009900 tx=0x7f697c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 -- 172.21.15.32:0/3583059811 shutdown_connections 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 --2- 172.21.15.32:0/3583059811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6990109bb0 0x7f69901103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 --2- 172.21.15.32:0/3583059811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6990109260 0x7f6990109670 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 --2- 172.21.15.32:0/3583059811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6990103920 0x7f6990103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 -- 172.21.15.32:0/3583059811 >> 172.21.15.32:0/3583059811 conn(0x7f69900ff370 msgr2=0x7f69901017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 -- 172.21.15.32:0/3583059811 shutdown_connections 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 -- 172.21.15.32:0/3583059811 wait complete. 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 Processor -- start 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 -- start start 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6990103920 0x7f69900754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6990109260 0x7f69900759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6990109bb0 0x7f6990075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6990113870 con 0x7f6990109260 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69901136f0 con 0x7f6990103920 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69901139f0 con 0x7f6990109bb0 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6995000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6990109260 0x7f69900759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6995000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6990109260 0x7f69900759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35200/0 (socket says 172.21.15.32:35200) 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6995000640 1 -- 172.21.15.32:0/3427023833 learned_addr learned my addr 172.21.15.32:0/3427023833 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6995a00640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6990103920 0x7f69900754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6996400640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6990109bb0 0x7f6990075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6995a00640 1 -- 172.21.15.32:0/3427023833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6990109bb0 msgr2=0x7f6990075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6995a00640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6990109bb0 0x7f6990075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6995a00640 1 -- 172.21.15.32:0/3427023833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6990109260 msgr2=0x7f69900759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6995a00640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6990109260 0x7f69900759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6995a00640 1 -- 172.21.15.32:0/3427023833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69901a84c0 con 0x7f6990103920 2024-08-31T07:27:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6995a00640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6990103920 0x7f69900754b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6984005fd0 tx=0x7f698400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f698e000640 1 -- 172.21.15.32:0/3427023833 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984009c30 con 0x7f6990103920 2024-08-31T07:27:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f698e000640 1 -- 172.21.15.32:0/3427023833 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6984009dd0 con 0x7f6990103920 2024-08-31T07:27:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f698e000640 1 -- 172.21.15.32:0/3427023833 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984003e20 con 0x7f6990103920 2024-08-31T07:27:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 -- 172.21.15.32:0/3427023833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69901a87b0 con 0x7f6990103920 2024-08-31T07:27:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.099+0000 7f6998400640 1 -- 172.21.15.32:0/3427023833 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69901a8d40 con 0x7f6990103920 2024-08-31T07:27:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f699f0a2740 1 Processor -- start 2024-08-31T07:27:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f699f0a2740 1 -- start start 2024-08-31T07:27:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 0x5578adace950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 0x5578adacee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adacf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f699f0a2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5578ada45770 con 0x5578ad708390 2024-08-31T07:27:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f699f0a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5578ada34d00 con 0x5578ad729070 2024-08-31T07:27:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f699f0a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5578ada38cb0 con 0x5578ad7001b0 2024-08-31T07:27:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6995000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adacf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6996400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 0x5578adace950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6996400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 0x5578adace950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:43848/0 (socket says 172.21.15.32:43848) 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6996400640 1 -- 172.21.15.32:0/939056698 learned_addr learned my addr 172.21.15.32:0/939056698 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6996400640 1 -- 172.21.15.32:0/939056698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 msgr2=0x5578adacee90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6996400640 1 --2- 172.21.15.32:0/939056698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 0x5578adacee90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6996400640 1 -- 172.21.15.32:0/939056698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 msgr2=0x5578adacf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6996400640 1 --2- 172.21.15.32:0/939056698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adacf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6996400640 1 -- 172.21.15.32:0/939056698 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ad794650 con 0x5578ad729070 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6996400640 1 --2- 172.21.15.32:0/939056698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 0x5578adace950 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6988009f50 tx=0x7f69880118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fcf0b5f04ac3eec server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f698e000640 1 -- 172.21.15.32:0/3427023833 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6984003fe0 con 0x7f6990103920 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6982a00640 1 -- 172.21.15.32:0/939056698 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988004b20 con 0x5578ad729070 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6982a00640 1 -- 172.21.15.32:0/939056698 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6988004cc0 con 0x5578ad729070 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f6982a00640 1 -- 172.21.15.32:0/939056698 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880053e0 con 0x5578ad729070 2024-08-31T07:27:03.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f699f0a2740 1 -- 172.21.15.32:0/939056698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 msgr2=0x5578adace950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.105+0000 7f699f0a2740 1 --2- 172.21.15.32:0/939056698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 0x5578adace950 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6988009f50 tx=0x7f69880118d0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- 172.21.15.32:0/939056698 shutdown_connections 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 --2- 172.21.15.32:0/939056698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adacf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 --2- 172.21.15.32:0/939056698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 0x5578adacee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 --2- 172.21.15.32:0/939056698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 0x5578adace950 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- 172.21.15.32:0/939056698 >> 172.21.15.32:0/939056698 conn(0x5578ad71d460 msgr2=0x5578adacc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- 172.21.15.32:0/939056698 shutdown_connections 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- 172.21.15.32:0/939056698 wait complete. 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- start start 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 Processor -- start 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f698e000640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69780b4c30 0x7f69780b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f698e000640 1 -- 172.21.15.32:0/3427023833 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7f69840e5060 con 0x7f6990103920 2024-08-31T07:27:03.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6996400640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69780b4c30 0x7f69780b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 0x5578adada5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adadabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 0x5578adae0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5578ad897700 con 0x5578ad708390 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5578ad9bbcb0 con 0x5578ad729070 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5578adadd130 con 0x5578ad7001b0 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 10 client.0 _ll_register_callbacks cb 0x5578ad9e1430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 0x5578adada5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 0x5578adada5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42454/0 (socket says 172.21.15.32:42454) 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995a00640 1 -- 172.21.15.32:0/2209384504 learned_addr learned my addr 172.21.15.32:0/2209384504 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6996400640 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 0x5578adae0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995a00640 1 -- 172.21.15.32:0/2209384504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 msgr2=0x5578adae0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995a00640 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 0x5578adae0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995a00640 1 -- 172.21.15.32:0/2209384504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 msgr2=0x5578adadabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995000640 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adadabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995a00640 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adadabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995a00640 1 -- 172.21.15.32:0/2209384504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ad9fc870 con 0x5578ad7001b0 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6996400640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69780b4c30 0x7f69780b7140 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f698801b040 tx=0x7f698800ff40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995000640 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adadabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995a00640 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 0x5578adada5d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6984009610 tx=0x7f6984003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2716b3b86a88083a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6977400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6977400640 1 -- 172.21.15.32:0/2209384504 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984020040 con 0x5578ad7001b0 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6977400640 1 -- 172.21.15.32:0/2209384504 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7f69840a7a80 con 0x5578ad7001b0 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- 172.21.15.32:0/2209384504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6988004cc0 con 0x5578ad7001b0 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6977400640 1 -- 172.21.15.32:0/2209384504 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f69840b4a60 con 0x5578ad7001b0 2024-08-31T07:27:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 10 client.28411 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- 172.21.15.32:0/2209384504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5578adae9230 con 0x5578ad7001b0 2024-08-31T07:27:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6977400640 1 -- 172.21.15.32:0/2209384504 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6988004cc0 con 0x5578ad7001b0 2024-08-31T07:27:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6977400640 1 -- 172.21.15.32:0/2209384504 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f69840aa4c0 con 0x5578ad7001b0 2024-08-31T07:27:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 10 client.28411 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 10 client.28411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:27:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 20 client.28411 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5578ad6e79f0 0x5578adaec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 4 client.28411 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:27:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f699f0a2740 1 -- 172.21.15.32:0/2209384504 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578ada476c0 con 0x5578ad6e79f0 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.110+0000 7f6995000640 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5578ad6e79f0 0x5578adaec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.169+0000 7f6977400640 10 client.28411 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.169+0000 7f6995000640 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5578ad6e79f0 0x5578adaec6a0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca6824b1ccebbc7 server_cookie=9b2e33ebffef30f0 in_seq=0 out_seq=0 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f6977400640 1 -- 172.21.15.32:0/2209384504 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5578ada476c0 con 0x5578ad6e79f0 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f6977400640 10 client.28411 handle_command_reply: tid=0 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 client.28411 shutdown 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 -- 172.21.15.32:0/2209384504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 msgr2=0x5578adada5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 0x5578adada5d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6984009610 tx=0x7f6984003820 comp rx=0 tx=0).stop 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 -- 172.21.15.32:0/2209384504 shutdown_connections 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5578ad6e79f0 0x5578adaec6a0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad729070 0x5578adae0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adadabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 --2- 172.21.15.32:0/2209384504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad7001b0 0x5578adada5d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 -- 172.21.15.32:0/2209384504 >> 172.21.15.32:0/2209384504 conn(0x5578ad71d460 msgr2=0x5578adad1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f6977400640 0 client.28411 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 -- 172.21.15.32:0/2209384504 shutdown_connections 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 1 -- 172.21.15.32:0/2209384504 wait complete. 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.217+0000 7f699f0a2740 20 client.28411 trim_cache size 0 max 16384 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 Processor -- start 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- start start 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad71d460 0x5578adae0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad7001b0 0x5578adad0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad708390 0x5578adad3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69901b15d0 con 0x5578ad7001b0 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69901b32f0 con 0x5578ad71d460 2024-08-31T07:27:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69901b1cd0 con 0x5578ad708390 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad7001b0 0x5578adad0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad7001b0 0x5578adad0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35222/0 (socket says 172.21.15.32:35222) 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995000640 1 -- 172.21.15.32:0/3405537930 learned_addr learned my addr 172.21.15.32:0/3405537930 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995a00640 1 --2- 172.21.15.32:0/3405537930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad71d460 0x5578adae0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6996400640 1 --2- 172.21.15.32:0/3405537930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad708390 0x5578adad3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995a00640 1 -- 172.21.15.32:0/3405537930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad708390 msgr2=0x5578adad3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995a00640 1 --2- 172.21.15.32:0/3405537930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad708390 0x5578adad3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995a00640 1 -- 172.21.15.32:0/3405537930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad7001b0 msgr2=0x5578adad0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995a00640 1 --2- 172.21.15.32:0/3405537930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad7001b0 0x5578adad0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995a00640 1 -- 172.21.15.32:0/3405537930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578adad17b0 con 0x5578ad71d460 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995000640 1 --2- 172.21.15.32:0/3405537930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad7001b0 0x5578adad0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995a00640 1 --2- 172.21.15.32:0/3405537930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad71d460 0x5578adae0390 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f698400bfa0 tx=0x7f6984005d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67b06f19c1dbdc79 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6982000640 1 -- 172.21.15.32:0/3405537930 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984020070 con 0x5578ad71d460 2024-08-31T07:27:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6982000640 1 -- 172.21.15.32:0/3405537930 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69840d2040 con 0x5578ad71d460 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6982000640 1 -- 172.21.15.32:0/3405537930 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69840aa4c0 con 0x5578ad71d460 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- 172.21.15.32:0/3405537930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad71d460 msgr2=0x5578adae0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 --2- 172.21.15.32:0/3405537930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad71d460 0x5578adae0390 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f698400bfa0 tx=0x7f6984005d40 comp rx=0 tx=0).stop 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- 172.21.15.32:0/3405537930 shutdown_connections 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 --2- 172.21.15.32:0/3405537930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad708390 0x5578adad3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 --2- 172.21.15.32:0/3405537930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad7001b0 0x5578adad0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 --2- 172.21.15.32:0/3405537930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad71d460 0x5578adae0390 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- 172.21.15.32:0/3405537930 >> 172.21.15.32:0/3405537930 conn(0x5578adacc880 msgr2=0x5578adacf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- 172.21.15.32:0/3405537930 shutdown_connections 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- 172.21.15.32:0/3405537930 wait complete. 2024-08-31T07:27:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- start start 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 Processor -- start 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad7001b0 0x5578adae6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adae6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad71d460 0x5578adae0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69901b2b60 con 0x5578ad708390 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69901b2430 con 0x5578ad7001b0 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69901af5d0 con 0x5578ad71d460 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f699f0a2740 10 client.0 _ll_register_callbacks cb 0x7f69840aaf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6996400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad71d460 0x5578adae0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adae6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6996400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad71d460 0x5578adae0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42474/0 (socket says 172.21.15.32:42474) 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6996400640 1 -- 172.21.15.32:0/3389245780 learned_addr learned my addr 172.21.15.32:0/3389245780 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995a00640 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad7001b0 0x5578adae6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995000640 1 -- 172.21.15.32:0/3389245780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad71d460 msgr2=0x5578adae0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995000640 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad71d460 0x5578adae0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.241+0000 7f6995000640 1 -- 172.21.15.32:0/3389245780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad7001b0 msgr2=0x5578adae6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6995000640 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad7001b0 0x5578adae6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6995000640 1 -- 172.21.15.32:0/3389245780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578adae1730 con 0x5578ad708390 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6996400640 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad71d460 0x5578adae0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6995a00640 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad7001b0 0x5578adae6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6995000640 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adae6e30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x5578adacf8c0 tx=0x7f697c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92eef4e540669d1a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6977400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6977400640 1 -- 172.21.15.32:0/3389245780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c035040 con 0x5578ad708390 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f699f0a2740 1 -- 172.21.15.32:0/3389245780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69840d2040 con 0x5578ad708390 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6977400640 1 -- 172.21.15.32:0/3389245780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8441+0+0 (secure 0 0 0) 0x7f697c005220 con 0x5578ad708390 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6977400640 1 -- 172.21.15.32:0/3389245780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f697c005420 con 0x5578ad708390 2024-08-31T07:27:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f699f0a2740 10 client.18402 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f699f0a2740 1 -- 172.21.15.32:0/3389245780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5578adb57ac0 con 0x5578ad708390 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6977400640 1 -- 172.21.15.32:0/3389245780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69840d2040 con 0x5578ad708390 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f6977400640 1 -- 172.21.15.32:0/3389245780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f697c004b20 con 0x5578ad708390 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f699f0a2740 10 client.18402 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f699f0a2740 10 client.18402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f699f0a2740 20 client.18402 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f699f0a2740 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5578ad729070 0x5578adb5a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f699f0a2740 4 client.18402 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.246+0000 7f699f0a2740 1 -- 172.21.15.32:0/3389245780 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578ada43160 con 0x5578ad729070 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.252+0000 7f6995a00640 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5578ad729070 0x5578adb5a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.311+0000 7f6995a00640 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5578ad729070 0x5578adb5a320 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d0cba7de94ec36 server_cookie=3a84c491daf84787 in_seq=0 out_seq=0 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.311+0000 7f6977400640 10 client.18402 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6977400640 1 -- 172.21.15.32:0/3389245780 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x5578ada43160 con 0x5578ad729070 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6977400640 10 client.18402 handle_command_reply: tid=0 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 client.18402 shutdown 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 -- 172.21.15.32:0/3389245780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 msgr2=0x5578adae6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adae6e30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x5578adacf8c0 tx=0x7f697c000bc0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 -- 172.21.15.32:0/3389245780 shutdown_connections 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5578ad729070 0x5578adb5a320 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5578ad71d460 0x5578adae0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6977400640 0 client.18402 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5578ad708390 0x5578adae6e30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 --2- 172.21.15.32:0/3389245780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5578ad7001b0 0x5578adae6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 -- 172.21.15.32:0/3389245780 >> 172.21.15.32:0/3389245780 conn(0x5578adacc880 msgr2=0x5578adad1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 -- 172.21.15.32:0/3389245780 shutdown_connections 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 1 -- 172.21.15.32:0/3389245780 wait complete. 2024-08-31T07:27:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f699f0a2740 20 client.18402 trim_cache size 0 max 16384 2024-08-31T07:27:03.439 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:27:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:27:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:27:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:27:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:27:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:27:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:27:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:27:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:27:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:27:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.448 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:03.449 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:03.450 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:03.450 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:03.450 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:03.450 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:27:03.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 -- 172.21.15.32:0/3427023833 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69780b4c30 msgr2=0x7f69780b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:03.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69780b4c30 0x7f69780b7140 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f698801b040 tx=0x7f698800ff40 comp rx=0 tx=0).stop 2024-08-31T07:27:03.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 -- 172.21.15.32:0/3427023833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6990103920 msgr2=0x7f69900754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:03.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6990103920 0x7f69900754b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6984005fd0 tx=0x7f698400dde0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 -- 172.21.15.32:0/3427023833 shutdown_connections 2024-08-31T07:27:03.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f69780b4c30 0x7f69780b7140 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6990109bb0 0x7f6990075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6990109260 0x7f69900759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 --2- 172.21.15.32:0/3427023833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6990103920 0x7f69900754b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:03.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 -- 172.21.15.32:0/3427023833 >> 172.21.15.32:0/3427023833 conn(0x7f69900ff370 msgr2=0x7f6990101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:03.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 -- 172.21.15.32:0/3427023833 shutdown_connections 2024-08-31T07:27:03.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:03.323+0000 7f6998400640 1 -- 172.21.15.32:0/3427023833 wait complete. 2024-08-31T07:27:04.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23381 32.61 MB/sec warmup 44 sec latency 19692.190 ms 2024-08-31T07:27:04.348 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:04 smithi032 ceph-mon[17252]: pgmap v813: 97 pgs: 97 active+clean; 687 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 936 KiB/s wr, 183 op/s 2024-08-31T07:27:04.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:04 smithi073 ceph-mon[21995]: pgmap v813: 97 pgs: 97 active+clean; 687 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 936 KiB/s wr, 183 op/s 2024-08-31T07:27:04.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:04 smithi084 ceph-mon[22387]: pgmap v813: 97 pgs: 97 active+clean; 687 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 936 KiB/s wr, 183 op/s 2024-08-31T07:27:05.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23381 31.88 MB/sec warmup 45 sec latency 20692.255 ms 2024-08-31T07:27:06.255 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:06 smithi032 ceph-mon[17252]: pgmap v814: 97 pgs: 97 active+clean; 685 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 797 KiB/s wr, 156 op/s 2024-08-31T07:27:06.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23382 31.19 MB/sec warmup 46 sec latency 21692.314 ms 2024-08-31T07:27:06.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:06 smithi073 ceph-mon[21995]: pgmap v814: 97 pgs: 97 active+clean; 685 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 797 KiB/s wr, 156 op/s 2024-08-31T07:27:06.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:06 smithi084 ceph-mon[22387]: pgmap v814: 97 pgs: 97 active+clean; 685 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 797 KiB/s wr, 156 op/s 2024-08-31T07:27:07.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23382 30.53 MB/sec warmup 47 sec latency 22692.351 ms 2024-08-31T07:27:07.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:07 smithi073 ceph-mon[21995]: osdmap e97: 12 total, 12 up, 12 in 2024-08-31T07:27:07.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:07 smithi084 ceph-mon[22387]: osdmap e97: 12 total, 12 up, 12 in 2024-08-31T07:27:07.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:07 smithi032 ceph-mon[17252]: osdmap e97: 12 total, 12 up, 12 in 2024-08-31T07:27:08.081 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 155.0 --await-for 45.0' 2024-08-31T07:27:08.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 155.0 --await-for 45.0 2024-08-31T07:27:08.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:08 smithi032 ceph-mon[17252]: pgmap v816: 97 pgs: 97 active+clean; 682 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 705 KiB/s wr, 120 op/s 2024-08-31T07:27:08.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23382 29.89 MB/sec warmup 48 sec latency 23692.414 ms 2024-08-31T07:27:08.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:08 smithi073 ceph-mon[21995]: pgmap v816: 97 pgs: 97 active+clean; 682 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 705 KiB/s wr, 120 op/s 2024-08-31T07:27:08.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:08 smithi084 ceph-mon[22387]: pgmap v816: 97 pgs: 97 active+clean; 682 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 705 KiB/s wr, 120 op/s 2024-08-31T07:27:09.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23382 29.28 MB/sec warmup 49 sec latency 24692.469 ms 2024-08-31T07:27:09.403 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:27:09.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:27:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:10.291 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:10 smithi032 ceph-mon[17252]: pgmap v817: 97 pgs: 97 active+clean; 680 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 693 KiB/s wr, 106 op/s 2024-08-31T07:27:10.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23382 28.69 MB/sec warmup 50 sec latency 25692.525 ms 2024-08-31T07:27:10.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:10 smithi073 ceph-mon[21995]: pgmap v817: 97 pgs: 97 active+clean; 680 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 693 KiB/s wr, 106 op/s 2024-08-31T07:27:10.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:10 smithi084 ceph-mon[22387]: pgmap v817: 97 pgs: 97 active+clean; 680 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 693 KiB/s wr, 106 op/s 2024-08-31T07:27:11.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.766+0000 7f2831400640 1 -- 172.21.15.32:0/3239525637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c1090a0 msgr2=0x7f282c1094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:11.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.766+0000 7f2831400640 1 --2- 172.21.15.32:0/3239525637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c1090a0 0x7f282c1094b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2818009900 tx=0x7f281802a420 comp rx=0 tx=0).stop 2024-08-31T07:27:11.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.769+0000 7f2831400640 1 -- 172.21.15.32:0/3239525637 shutdown_connections 2024-08-31T07:27:11.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.769+0000 7f2831400640 1 --2- 172.21.15.32:0/3239525637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c1099f0 0x7f282c110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.769+0000 7f2831400640 1 --2- 172.21.15.32:0/3239525637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c1090a0 0x7f282c1094b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.769+0000 7f2831400640 1 --2- 172.21.15.32:0/3239525637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c103760 0x7f282c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.769+0000 7f2831400640 1 -- 172.21.15.32:0/3239525637 >> 172.21.15.32:0/3239525637 conn(0x7f282c0ff1d0 msgr2=0x7f282c101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.769+0000 7f2831400640 1 -- 172.21.15.32:0/3239525637 shutdown_connections 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.769+0000 7f2831400640 1 -- 172.21.15.32:0/3239525637 wait complete. 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.770+0000 7f2831400640 1 Processor -- start 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.770+0000 7f2831400640 1 -- start start 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.770+0000 7f2831400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c103760 0x7f282c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.770+0000 7f2831400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c1090a0 0x7f282c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.770+0000 7f2831400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c1099f0 0x7f282c1a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.770+0000 7f2831400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f282c113740 con 0x7f282c103760 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.770+0000 7f2831400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1135c0 con 0x7f282c1099f0 2024-08-31T07:27:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.770+0000 7f2831400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1138c0 con 0x7f282c1090a0 2024-08-31T07:27:11.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f282be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c103760 0x7f282c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f282b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c1090a0 0x7f282c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f282be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c103760 0x7f282c19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38708/0 (socket says 172.21.15.32:38708) 2024-08-31T07:27:11.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f282be00640 1 -- 172.21.15.32:0/2991008163 learned_addr learned my addr 172.21.15.32:0/2991008163 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:11.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f2830a00640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c1099f0 0x7f282c1a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f282b400640 1 -- 172.21.15.32:0/2991008163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c1099f0 msgr2=0x7f282c1a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:11.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f282b400640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c1099f0 0x7f282c1a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f282b400640 1 -- 172.21.15.32:0/2991008163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c103760 msgr2=0x7f282c19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:11.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f282b400640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c103760 0x7f282c19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f282b400640 1 -- 172.21.15.32:0/2991008163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c1a2360 con 0x7f282c1090a0 2024-08-31T07:27:11.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f282b400640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c1090a0 0x7f282c19c990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2818009230 tx=0x7f2818003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:11.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f2828c00640 1 -- 172.21.15.32:0/2991008163 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281803f070 con 0x7f282c1090a0 2024-08-31T07:27:11.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f2831400640 1 -- 172.21.15.32:0/2991008163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c1a25f0 con 0x7f282c1090a0 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f2828c00640 1 -- 172.21.15.32:0/2991008163 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2818031b60 con 0x7f282c1090a0 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f2828c00640 1 -- 172.21.15.32:0/2991008163 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818035030 con 0x7f282c1090a0 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f2831400640 1 -- 172.21.15.32:0/2991008163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c1a8db0 con 0x7f282c1090a0 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.771+0000 7f2839621740 1 Processor -- start 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- start start 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc7758c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771bc980 0x55dc7758c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc7758cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dc77501530 con 0x55dc771c4b60 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dc774f0ac0 con 0x55dc771bc980 2024-08-31T07:27:11.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dc774f4a70 con 0x55dc771e5840 2024-08-31T07:27:11.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2830a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc7758c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2830a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc7758c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33894/0 (socket says 172.21.15.32:33894) 2024-08-31T07:27:11.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f282be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771bc980 0x55dc7758c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2830a00640 1 -- 172.21.15.32:0/476514380 learned_addr learned my addr 172.21.15.32:0/476514380 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:11.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2830a00640 1 -- 172.21.15.32:0/476514380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771bc980 msgr2=0x55dc7758c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:11.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2830a00640 1 --2- 172.21.15.32:0/476514380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771bc980 0x55dc7758c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2830a00640 1 -- 172.21.15.32:0/476514380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 msgr2=0x55dc7758cd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:27:11.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2830a00640 1 --2- 172.21.15.32:0/476514380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc7758cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2830a00640 1 -- 172.21.15.32:0/476514380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc774a3880 con 0x55dc771e5840 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2828c00640 1 -- 172.21.15.32:0/2991008163 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2818042070 con 0x7f282c1090a0 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2830a00640 1 --2- 172.21.15.32:0/476514380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc7758c2d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2824009fd0 tx=0x7f28240118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc7e81f3f914df31 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2821200640 1 -- 172.21.15.32:0/476514380 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824004d20 con 0x55dc771e5840 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2821200640 1 -- 172.21.15.32:0/476514380 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2824004050 con 0x55dc771e5840 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2821200640 1 -- 172.21.15.32:0/476514380 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824005280 con 0x55dc771e5840 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- 172.21.15.32:0/476514380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 msgr2=0x55dc7758c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 --2- 172.21.15.32:0/476514380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc7758c2d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2824009fd0 tx=0x7f28240118d0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- 172.21.15.32:0/476514380 shutdown_connections 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 --2- 172.21.15.32:0/476514380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc7758cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 --2- 172.21.15.32:0/476514380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771bc980 0x55dc7758c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 --2- 172.21.15.32:0/476514380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc7758c2d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- 172.21.15.32:0/476514380 >> 172.21.15.32:0/476514380 conn(0x55dc771d9990 msgr2=0x55dc7758a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- 172.21.15.32:0/476514380 shutdown_connections 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- 172.21.15.32:0/476514380 wait complete. 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- start start 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2828c00640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4ba0 0x7f28100b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 Processor -- start 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2828c00640 1 -- 172.21.15.32:0/2991008163 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8485+0+0 (secure 0 0 0) 0x7f2818105b30 con 0x7f282c1090a0 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2830a00640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4ba0 0x7f28100b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771bc980 0x55dc77597f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc77598510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771e5840 0x55dc775a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dc773537a0 con 0x55dc771c4b60 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dc774777f0 con 0x55dc771e5840 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dc7759aab0 con 0x55dc771bc980 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f2839621740 10 client.0 _ll_register_callbacks cb 0x55dc7749ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f282be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771bc980 0x55dc77597f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f282be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771bc980 0x55dc77597f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33906/0 (socket says 172.21.15.32:33906) 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.776+0000 7f282be00640 1 -- 172.21.15.32:0/2832118523 learned_addr learned my addr 172.21.15.32:0/2832118523 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2830a00640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4ba0 0x7f28100b70b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2824004910 tx=0x7f2824000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2830a00640 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771e5840 0x55dc775a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f282b400640 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc77598510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f282be00640 1 -- 172.21.15.32:0/2832118523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771e5840 msgr2=0x55dc775a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f282be00640 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771e5840 0x55dc775a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f282be00640 1 -- 172.21.15.32:0/2832118523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 msgr2=0x55dc77598510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f282be00640 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc77598510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f282be00640 1 -- 172.21.15.32:0/2832118523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7725fb90 con 0x55dc771bc980 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f282b400640 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc77598510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f282be00640 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771bc980 0x55dc77597f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f281c0115e0 tx=0x7f281c011aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3fa14af9ee322be server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2815600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2815600640 1 -- 172.21.15.32:0/2832118523 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c00f750 con 0x55dc771bc980 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2815600640 1 -- 172.21.15.32:0/2832118523 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8485+0+0 (secure 0 0 0) 0x7f281c00f8f0 con 0x55dc771bc980 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 -- 172.21.15.32:0/2832118523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2824004050 con 0x55dc771bc980 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2815600640 1 -- 172.21.15.32:0/2832118523 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f281c00faf0 con 0x55dc771bc980 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 10 client.28429 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:11.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 -- 172.21.15.32:0/2832118523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55dc775a6bb0 con 0x55dc771bc980 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2815600640 1 -- 172.21.15.32:0/2832118523 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2824004050 con 0x55dc771bc980 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2815600640 1 -- 172.21.15.32:0/2832118523 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f281c01d400 con 0x55dc771bc980 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 10 client.28429 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 10 client.28429 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 20 client.28429 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dc771a41c0 0x55dc775aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 4 client.28429 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 -- 172.21.15.32:0/2832118523 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc77503480 con 0x55dc771a41c0 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f282b400640 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dc771a41c0 0x55dc775aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f282b400640 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dc771a41c0 0x55dc775aa0d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3fdd77c0ca8175f server_cookie=10af29bbf4fd971c in_seq=0 out_seq=0 2024-08-31T07:27:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2815600640 10 client.28429 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2815600640 1 -- 172.21.15.32:0/2832118523 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc77503480 con 0x55dc771a41c0 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2815600640 10 client.28429 handle_command_reply: tid=0 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 client.28429 shutdown 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 -- 172.21.15.32:0/2832118523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771bc980 msgr2=0x55dc77597f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771bc980 0x55dc77597f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f281c0115e0 tx=0x7f281c011aa0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 -- 172.21.15.32:0/2832118523 shutdown_connections 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dc771a41c0 0x55dc775aa0d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771e5840 0x55dc775a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc77598510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2815600640 0 client.28429 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 --2- 172.21.15.32:0/2832118523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771bc980 0x55dc77597f30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 -- 172.21.15.32:0/2832118523 >> 172.21.15.32:0/2832118523 conn(0x55dc771d9990 msgr2=0x55dc7758f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 -- 172.21.15.32:0/2832118523 shutdown_connections 2024-08-31T07:27:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 1 -- 172.21.15.32:0/2832118523 wait complete. 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.781+0000 7f2839621740 20 client.28429 trim_cache size 0 max 16384 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f2839621740 1 Processor -- start 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f2839621740 1 -- start start 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f2839621740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771c4b60 0x55dc775a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f2839621740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc77591550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f2839621740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc77179570 0x55dc77591a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f2839621740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1b1560 con 0x55dc77179570 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f2839621740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1b3280 con 0x55dc771c4b60 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f2839621740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1b1c60 con 0x55dc771e5840 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f282be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771c4b60 0x55dc775a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f282b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc77591550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f2830a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc77179570 0x55dc77591a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f282b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc77591550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33922/0 (socket says 172.21.15.32:33922) 2024-08-31T07:27:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f282be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771c4b60 0x55dc775a3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:39350/0 (socket says 172.21.15.32:39350) 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f282b400640 1 -- 172.21.15.32:0/3242449921 learned_addr learned my addr 172.21.15.32:0/3242449921 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f282b400640 1 -- 172.21.15.32:0/3242449921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771c4b60 msgr2=0x55dc775a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.799+0000 7f282b400640 1 --2- 172.21.15.32:0/3242449921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771c4b60 0x55dc775a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.805+0000 7f282b400640 1 -- 172.21.15.32:0/3242449921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc77179570 msgr2=0x55dc77591a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.805+0000 7f282b400640 1 --2- 172.21.15.32:0/3242449921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc77179570 0x55dc77591a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.805+0000 7f282b400640 1 -- 172.21.15.32:0/3242449921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc77593570 con 0x55dc771e5840 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.805+0000 7f282be00640 1 --2- 172.21.15.32:0/3242449921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771c4b60 0x55dc775a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f282b400640 1 --2- 172.21.15.32:0/3242449921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc77591550 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2818000c00 tx=0x7f2818009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbc43cca71d8a3f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2817e00640 1 -- 172.21.15.32:0/3242449921 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818049040 con 0x55dc771e5840 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2817e00640 1 -- 172.21.15.32:0/3242449921 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28180d5500 con 0x55dc771e5840 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2817e00640 1 -- 172.21.15.32:0/3242449921 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28180d5070 con 0x55dc771e5840 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 -- 172.21.15.32:0/3242449921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 msgr2=0x55dc77591550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 --2- 172.21.15.32:0/3242449921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc77591550 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2818000c00 tx=0x7f2818009260 comp rx=0 tx=0).stop 2024-08-31T07:27:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 -- 172.21.15.32:0/3242449921 shutdown_connections 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 --2- 172.21.15.32:0/3242449921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc77179570 0x55dc77591a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 --2- 172.21.15.32:0/3242449921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc77591550 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 --2- 172.21.15.32:0/3242449921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc771c4b60 0x55dc775a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 -- 172.21.15.32:0/3242449921 >> 172.21.15.32:0/3242449921 conn(0x55dc771d9990 msgr2=0x55dc775965d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 -- 172.21.15.32:0/3242449921 shutdown_connections 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 -- 172.21.15.32:0/3242449921 wait complete. 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 -- start start 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 Processor -- start 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc77179570 0x55dc775a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc775a0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc7759ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1b2af0 con 0x55dc771c4b60 2024-08-31T07:27:11.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1b23c0 con 0x55dc77179570 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1af560 con 0x55dc771e5840 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2839621740 10 client.0 _ll_register_callbacks cb 0x7f281c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f282be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc77179570 0x55dc775a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f282be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc77179570 0x55dc775a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:39352/0 (socket says 172.21.15.32:39352) 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f282be00640 1 -- 172.21.15.32:0/410601387 learned_addr learned my addr 172.21.15.32:0/410601387 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f282b400640 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc775a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f2830a00640 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc7759ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f282b400640 1 -- 172.21.15.32:0/410601387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 msgr2=0x55dc7759ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f282b400640 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc7759ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f282b400640 1 -- 172.21.15.32:0/410601387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc77179570 msgr2=0x55dc775a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.810+0000 7f282b400640 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc77179570 0x55dc775a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f282b400640 1 -- 172.21.15.32:0/410601387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7759b620 con 0x55dc771c4b60 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f282be00640 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc77179570 0x55dc775a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:11.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f282b400640 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc775a0730 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2818000c00 tx=0x7f28180c8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c13e73b4d15aebf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2815600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:27:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2815600640 1 -- 172.21.15.32:0/410601387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818049050 con 0x55dc771c4b60 2024-08-31T07:27:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2815600640 1 -- 172.21.15.32:0/410601387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8485+0+0 (secure 0 0 0) 0x7f28180d5070 con 0x55dc771c4b60 2024-08-31T07:27:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2839621740 1 -- 172.21.15.32:0/410601387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28180d5500 con 0x55dc771c4b60 2024-08-31T07:27:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2815600640 1 -- 172.21.15.32:0/410601387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f28180d5270 con 0x55dc771c4b60 2024-08-31T07:27:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2839621740 10 client.18417 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2839621740 1 -- 172.21.15.32:0/410601387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55dc77615690 con 0x55dc771c4b60 2024-08-31T07:27:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2815600640 1 -- 172.21.15.32:0/410601387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28180d5500 con 0x55dc771c4b60 2024-08-31T07:27:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2815600640 1 -- 172.21.15.32:0/410601387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f28180c7c00 con 0x55dc771c4b60 2024-08-31T07:27:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2839621740 10 client.18417 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2839621740 10 client.18417 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:27:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2839621740 20 client.18417 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2839621740 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dc77615e20 0x55dc77618240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2839621740 4 client.18417 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 155.0, "await_for": 45.0}] 2024-08-31T07:27:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2839621740 1 -- 172.21.15.32:0/410601387 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 155.0, "await_for": 45.0}) -- 0x55dc774fef20 con 0x55dc77615e20 2024-08-31T07:27:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f282be00640 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dc77615e20 0x55dc77618240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f282be00640 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dc77615e20 0x55dc77618240 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b669c16adbae07 server_cookie=66dfd70847bef214 in_seq=0 out_seq=0 2024-08-31T07:27:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:10.815+0000 7f2815600640 10 client.18417 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:11.231 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:11.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23382 28.13 MB/sec warmup 51 sec latency 26692.577 ms 2024-08-31T07:27:12.310 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:12 smithi032 ceph-mon[17252]: pgmap v818: 97 pgs: 97 active+clean; 673 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 476 KiB/s wr, 62 op/s 2024-08-31T07:27:12.310 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:12 smithi032 ceph-mon[17252]: osdmap e98: 12 total, 12 up, 12 in 2024-08-31T07:27:12.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23382 27.59 MB/sec warmup 52 sec latency 27692.646 ms 2024-08-31T07:27:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:12 smithi084 ceph-mon[22387]: pgmap v818: 97 pgs: 97 active+clean; 673 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 476 KiB/s wr, 62 op/s 2024-08-31T07:27:12.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:12 smithi084 ceph-mon[22387]: osdmap e98: 12 total, 12 up, 12 in 2024-08-31T07:27:12.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:12 smithi073 ceph-mon[21995]: pgmap v818: 97 pgs: 97 active+clean; 673 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 476 KiB/s wr, 62 op/s 2024-08-31T07:27:12.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:12 smithi073 ceph-mon[21995]: osdmap e98: 12 total, 12 up, 12 in 2024-08-31T07:27:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.928+0000 7f51bf200640 1 -- 172.21.15.32:0/560957193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b80754f0 msgr2=0x7f51b8075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.928+0000 7f51bf200640 1 --2- 172.21.15.32:0/560957193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b80754f0 0x7f51b8075900 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009900 tx=0x7f51a802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 -- 172.21.15.32:0/560957193 shutdown_connections 2024-08-31T07:27:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 --2- 172.21.15.32:0/560957193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51b810fec0 0x7f51b81122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 --2- 172.21.15.32:0/560957193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b8075e40 0x7f51b810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 --2- 172.21.15.32:0/560957193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b80754f0 0x7f51b8075900 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 -- 172.21.15.32:0/560957193 >> 172.21.15.32:0/560957193 conn(0x7f51b807b5f0 msgr2=0x7f51b807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 -- 172.21.15.32:0/560957193 shutdown_connections 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 -- 172.21.15.32:0/560957193 wait complete. 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 Processor -- start 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 -- start start 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b80754f0 0x7f51b819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b8075e40 0x7f51b819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51b810fec0 0x7f51b81a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8115a30 con 0x7f51b80754f0 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51b81158b0 con 0x7f51b810fec0 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bf200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8115bb0 con 0x7f51b8075e40 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b80754f0 0x7f51b819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b80754f0 0x7f51b819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38738/0 (socket says 172.21.15.32:38738) 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7e00640 1 -- 172.21.15.32:0/3814703681 learned_addr learned my addr 172.21.15.32:0/3814703681 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7400640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b8075e40 0x7f51b819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51bd200640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51b810fec0 0x7f51b81a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7400640 1 -- 172.21.15.32:0/3814703681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51b810fec0 msgr2=0x7f51b81a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7400640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51b810fec0 0x7f51b81a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7400640 1 -- 172.21.15.32:0/3814703681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b80754f0 msgr2=0x7f51b819c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7400640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b80754f0 0x7f51b819c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7400640 1 -- 172.21.15.32:0/3814703681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b81a2370 con 0x7f51b8075e40 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7e00640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b80754f0 0x7f51b819c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.930+0000 7f51b7400640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b8075e40 0x7f51b819c9a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f51b8077570 tx=0x7f51a400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.933+0000 7f51b4c00640 1 -- 172.21.15.32:0/3814703681 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a400cc40 con 0x7f51b8075e40 2024-08-31T07:27:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.933+0000 7f51b4c00640 1 -- 172.21.15.32:0/3814703681 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a400cde0 con 0x7f51b8075e40 2024-08-31T07:27:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.933+0000 7f51b4c00640 1 -- 172.21.15.32:0/3814703681 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4018780 con 0x7f51b8075e40 2024-08-31T07:27:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.933+0000 7f51bf200640 1 -- 172.21.15.32:0/3814703681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b81a2660 con 0x7f51b8075e40 2024-08-31T07:27:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.933+0000 7f51bf200640 1 -- 172.21.15.32:0/3814703681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b81a8dc0 con 0x7f51b8075e40 2024-08-31T07:27:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.933+0000 7f51bf200640 1 -- 172.21.15.32:0/3814703681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b8075900 con 0x7f51b8075e40 2024-08-31T07:27:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.933+0000 7f51b4c00640 1 -- 172.21.15.32:0/3814703681 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f51a4010a90 con 0x7f51b8075e40 2024-08-31T07:27:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.933+0000 7f51b4c00640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f519c0b4d20 0x7f519c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.933+0000 7f51b4c00640 1 -- 172.21.15.32:0/3814703681 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f51a40e5ce0 con 0x7f51b8075e40 2024-08-31T07:27:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.933+0000 7f51b7e00640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f519c0b4d20 0x7f519c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.938+0000 7f51b7e00640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f519c0b4d20 0x7f519c0b7230 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f51a8031440 tx=0x7f51a80058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:12.938+0000 7f51b4c00640 1 -- 172.21.15.32:0/3814703681 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f51a40acac0 con 0x7f51b8075e40 2024-08-31T07:27:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.176+0000 7f51bf200640 1 -- 172.21.15.32:0/3814703681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51b81a9060 con 0x7f51b8075e40 2024-08-31T07:27:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.176+0000 7f51b4c00640 1 -- 172.21.15.32:0/3814703681 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11979 (secure 0 0 0) 0x7f51a40ac3c0 con 0x7f51b8075e40 2024-08-31T07:27:13.178 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:27:13.178 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":55,"btime":"2024-08-31T07:26:31:188885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:26:30.959002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:27:13.178 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 55 2024-08-31T07:27:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 -- 172.21.15.32:0/3814703681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f519c0b4d20 msgr2=0x7f519c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f519c0b4d20 0x7f519c0b7230 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f51a8031440 tx=0x7f51a80058d0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 -- 172.21.15.32:0/3814703681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b8075e40 msgr2=0x7f51b819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b8075e40 0x7f51b819c9a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f51b8077570 tx=0x7f51a400ee50 comp rx=0 tx=0).stop 2024-08-31T07:27:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 -- 172.21.15.32:0/3814703681 shutdown_connections 2024-08-31T07:27:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f519c0b4d20 0x7f519c0b7230 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51b810fec0 0x7f51b81a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51b8075e40 0x7f51b819c9a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 --2- 172.21.15.32:0/3814703681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b80754f0 0x7f51b819c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 -- 172.21.15.32:0/3814703681 >> 172.21.15.32:0/3814703681 conn(0x7f51b807b5f0 msgr2=0x7f51b8101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 -- 172.21.15.32:0/3814703681 shutdown_connections 2024-08-31T07:27:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:13.180+0000 7f51bf200640 1 -- 172.21.15.32:0/3814703681 wait complete. 2024-08-31T07:27:13.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23382 27.07 MB/sec warmup 53 sec latency 28692.685 ms 2024-08-31T07:27:13.436 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:13 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3814703681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:27:13.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:13 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3814703681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:27:13.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:13 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3814703681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:27:14.290 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:14 smithi032 ceph-mon[17252]: pgmap v820: 97 pgs: 97 active+clean; 678 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 886 KiB/s wr, 54 op/s 2024-08-31T07:27:14.331 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23382 26.57 MB/sec warmup 54 sec latency 29692.729 ms 2024-08-31T07:27:14.494 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:14 smithi084 ceph-mon[22387]: pgmap v820: 97 pgs: 97 active+clean; 678 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 886 KiB/s wr, 54 op/s 2024-08-31T07:27:14.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:14 smithi073 ceph-mon[21995]: pgmap v820: 97 pgs: 97 active+clean; 678 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 886 KiB/s wr, 54 op/s 2024-08-31T07:27:15.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:27:15.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23382 26.09 MB/sec warmup 55 sec latency 30692.768 ms 2024-08-31T07:27:15.399 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:16.249 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:16 smithi032 ceph-mon[17252]: pgmap v821: 97 pgs: 97 active+clean; 679 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.4 MiB/s wr, 73 op/s 2024-08-31T07:27:16.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23383 25.62 MB/sec warmup 56 sec latency 31692.813 ms 2024-08-31T07:27:16.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:16 smithi084 ceph-mon[22387]: pgmap v821: 97 pgs: 97 active+clean; 679 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.4 MiB/s wr, 73 op/s 2024-08-31T07:27:16.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.575+0000 7ff69dc00640 1 -- 172.21.15.32:0/1052929784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff698103900 msgr2=0x7ff698103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.575+0000 7ff69dc00640 1 --2- 172.21.15.32:0/1052929784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff698103900 0x7ff698103d10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff688009900 tx=0x7ff68802a420 comp rx=0 tx=0).stop 2024-08-31T07:27:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.575+0000 7ff69dc00640 1 -- 172.21.15.32:0/1052929784 shutdown_connections 2024-08-31T07:27:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.575+0000 7ff69dc00640 1 --2- 172.21.15.32:0/1052929784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff698109b90 0x7ff6981103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.575+0000 7ff69dc00640 1 --2- 172.21.15.32:0/1052929784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff698109240 0x7ff698109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.575+0000 7ff69dc00640 1 --2- 172.21.15.32:0/1052929784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff698103900 0x7ff698103d10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.575+0000 7ff69dc00640 1 -- 172.21.15.32:0/1052929784 >> 172.21.15.32:0/1052929784 conn(0x7ff69807b5f0 msgr2=0x7ff69807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.575+0000 7ff69dc00640 1 -- 172.21.15.32:0/1052929784 shutdown_connections 2024-08-31T07:27:16.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.575+0000 7ff69dc00640 1 -- 172.21.15.32:0/1052929784 wait complete. 2024-08-31T07:27:16.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.576+0000 7ff69dc00640 1 Processor -- start 2024-08-31T07:27:16.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.576+0000 7ff69dc00640 1 -- start start 2024-08-31T07:27:16.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff69dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff698103900 0x7ff69819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff69dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff698109240 0x7ff69819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff69dc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff698109b90 0x7ff6981a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff69dc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981138e0 con 0x7ff698103900 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff69dc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff698113760 con 0x7ff698109240 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff69dc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff698113a60 con 0x7ff698109b90 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff696a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff698103900 0x7ff69819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff696000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff698109240 0x7ff69819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff696a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff698103900 0x7ff69819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41262/0 (socket says 172.21.15.32:41262) 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff696a00640 1 -- 172.21.15.32:0/331646235 learned_addr learned my addr 172.21.15.32:0/331646235 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff696000640 1 -- 172.21.15.32:0/331646235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff698109b90 msgr2=0x7ff6981a1cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff696000640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff698109b90 0x7ff6981a1cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff696000640 1 -- 172.21.15.32:0/331646235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff698103900 msgr2=0x7ff69819c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff696000640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff698103900 0x7ff69819c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.577+0000 7ff696000640 1 -- 172.21.15.32:0/331646235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6981a2420 con 0x7ff698109240 2024-08-31T07:27:16.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.578+0000 7ff696000640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff698109240 0x7ff69819ca50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff69806c790 tx=0x7ff68400b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.578+0000 7ff68f400640 1 -- 172.21.15.32:0/331646235 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684015020 con 0x7ff698109240 2024-08-31T07:27:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.578+0000 7ff68f400640 1 -- 172.21.15.32:0/331646235 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff684011ce0 con 0x7ff698109240 2024-08-31T07:27:16.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.578+0000 7ff69dc00640 1 -- 172.21.15.32:0/331646235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6981a2710 con 0x7ff698109240 2024-08-31T07:27:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.578+0000 7ff68f400640 1 -- 172.21.15.32:0/331646235 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684010da0 con 0x7ff698109240 2024-08-31T07:27:16.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.578+0000 7ff69dc00640 1 -- 172.21.15.32:0/331646235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6981a8ef0 con 0x7ff698109240 2024-08-31T07:27:16.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.582+0000 7ff6a482c740 1 Processor -- start 2024-08-31T07:27:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.582+0000 7ff6a482c740 1 -- start start 2024-08-31T07:27:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.582+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a950070 0x55ce1acf5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.582+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a9271b0 0x55ce1acf5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.582+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1acf6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.582+0000 7ff6a482c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ce1ac6c7b0 con 0x55ce1a92f390 2024-08-31T07:27:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.582+0000 7ff6a482c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ce1ac5bd40 con 0x55ce1a950070 2024-08-31T07:27:16.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.582+0000 7ff6a482c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ce1ac5fcf0 con 0x55ce1a9271b0 2024-08-31T07:27:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff697400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a950070 0x55ce1acf5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff697400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a950070 0x55ce1acf5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:40352/0 (socket says 172.21.15.32:40352) 2024-08-31T07:27:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff697400640 1 -- 172.21.15.32:0/3030546341 learned_addr learned my addr 172.21.15.32:0/3030546341 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:16.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff696a00640 1 --2- 172.21.15.32:0/3030546341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a9271b0 0x55ce1acf5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff697400640 1 -- 172.21.15.32:0/3030546341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a9271b0 msgr2=0x55ce1acf5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff697400640 1 --2- 172.21.15.32:0/3030546341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a9271b0 0x55ce1acf5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff697400640 1 -- 172.21.15.32:0/3030546341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 msgr2=0x55ce1acf6410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:27:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff697400640 1 --2- 172.21.15.32:0/3030546341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1acf6410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff697400640 1 -- 172.21.15.32:0/3030546341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1a9bb650 con 0x55ce1a950070 2024-08-31T07:27:16.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff68f400640 1 -- 172.21.15.32:0/331646235 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff684018020 con 0x7ff698109240 2024-08-31T07:27:16.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.583+0000 7ff697400640 1 --2- 172.21.15.32:0/3030546341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a950070 0x55ce1acf5990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff690009e30 tx=0x7ff69000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba17a182e27d84f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:16.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.584+0000 7ff683e00640 1 -- 172.21.15.32:0/3030546341 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690004d20 con 0x55ce1a950070 2024-08-31T07:27:16.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.584+0000 7ff683e00640 1 -- 172.21.15.32:0/3030546341 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff69000ee70 con 0x55ce1a950070 2024-08-31T07:27:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.584+0000 7ff683e00640 1 -- 172.21.15.32:0/3030546341 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690010760 con 0x55ce1a950070 2024-08-31T07:27:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.584+0000 7ff6a482c740 1 -- 172.21.15.32:0/3030546341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a950070 msgr2=0x55ce1acf5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.584+0000 7ff6a482c740 1 --2- 172.21.15.32:0/3030546341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a950070 0x55ce1acf5990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff690009e30 tx=0x7ff69000e720 comp rx=0 tx=0).stop 2024-08-31T07:27:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.585+0000 7ff6a482c740 1 -- 172.21.15.32:0/3030546341 shutdown_connections 2024-08-31T07:27:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.585+0000 7ff6a482c740 1 --2- 172.21.15.32:0/3030546341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1acf6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.585+0000 7ff6a482c740 1 --2- 172.21.15.32:0/3030546341 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a9271b0 0x55ce1acf5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.585+0000 7ff6a482c740 1 --2- 172.21.15.32:0/3030546341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a950070 0x55ce1acf5990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.585+0000 7ff6a482c740 1 -- 172.21.15.32:0/3030546341 >> 172.21.15.32:0/3030546341 conn(0x55ce1a944460 msgr2=0x55ce1acf3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:16.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.585+0000 7ff6a482c740 1 -- 172.21.15.32:0/3030546341 shutdown_connections 2024-08-31T07:27:16.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.585+0000 7ff6a482c740 1 -- 172.21.15.32:0/3030546341 wait complete. 2024-08-31T07:27:16.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.585+0000 7ff68f400640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff67c0b4c30 0x7ff67c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.586+0000 7ff68f400640 1 -- 172.21.15.32:0/331646235 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7ff6840e4ba0 con 0x7ff698109240 2024-08-31T07:27:16.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.586+0000 7ff6a482c740 1 -- start start 2024-08-31T07:27:16.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.586+0000 7ff696a00640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff67c0b4c30 0x7ff67c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.586+0000 7ff6a482c740 1 Processor -- start 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad01580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a92f390 0x55ce1ad01b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a950070 0x55ce1ad0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff6a482c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ce1aabe700 con 0x55ce1a950070 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff6a482c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ce1abe2cf0 con 0x55ce1a9271b0 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff6a482c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ce1ad04170 con 0x55ce1a92f390 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff696a00640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff67c0b4c30 0x7ff67c0b7140 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55ce1acf4610 tx=0x7ff688005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff6a482c740 10 client.0 _ll_register_callbacks cb 0x55ce1ac08470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff696a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad01580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff696a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad01580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:40362/0 (socket says 172.21.15.32:40362) 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff696a00640 1 -- 172.21.15.32:0/2328310805 learned_addr learned my addr 172.21.15.32:0/2328310805 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.587+0000 7ff697400640 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a950070 0x55ce1ad0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.588+0000 7ff696a00640 1 -- 172.21.15.32:0/2328310805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a92f390 msgr2=0x55ce1ad01b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:27:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.588+0000 7ff696a00640 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a92f390 0x55ce1ad01b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.588+0000 7ff696a00640 1 -- 172.21.15.32:0/2328310805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a950070 msgr2=0x55ce1ad0d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.588+0000 7ff696a00640 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a950070 0x55ce1ad0d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.588+0000 7ff696a00640 1 -- 172.21.15.32:0/2328310805 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1ac238b0 con 0x55ce1a9271b0 2024-08-31T07:27:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.588+0000 7ff696a00640 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad01580 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff688006010 tx=0x7ff688009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73cde441d58db65 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:16.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.588+0000 7ff680c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:27:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.589+0000 7ff680c00640 1 -- 172.21.15.32:0/2328310805 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688035040 con 0x55ce1a9271b0 2024-08-31T07:27:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.589+0000 7ff680c00640 1 -- 172.21.15.32:0/2328310805 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7ff6880318b0 con 0x55ce1a9271b0 2024-08-31T07:27:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.589+0000 7ff6a482c740 1 -- 172.21.15.32:0/2328310805 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff69000ee70 con 0x55ce1a9271b0 2024-08-31T07:27:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.589+0000 7ff680c00640 1 -- 172.21.15.32:0/2328310805 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff688031ab0 con 0x55ce1a9271b0 2024-08-31T07:27:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.589+0000 7ff6a482c740 10 client.28451 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.589+0000 7ff6a482c740 1 -- 172.21.15.32:0/2328310805 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ce1ad10200 con 0x55ce1a9271b0 2024-08-31T07:27:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.590+0000 7ff680c00640 1 -- 172.21.15.32:0/2328310805 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff69000ee70 con 0x55ce1a9271b0 2024-08-31T07:27:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.590+0000 7ff680c00640 1 -- 172.21.15.32:0/2328310805 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7ff68804f020 con 0x55ce1a9271b0 2024-08-31T07:27:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.590+0000 7ff6a482c740 10 client.28451 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:16.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.590+0000 7ff6a482c740 10 client.28451 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:27:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.590+0000 7ff6a482c740 20 client.28451 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.590+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55ce1a90e9f0 0x55ce1ad13780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.590+0000 7ff6a482c740 4 client.28451 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:27:16.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.590+0000 7ff6a482c740 1 -- 172.21.15.32:0/2328310805 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce1ac6e700 con 0x55ce1a90e9f0 2024-08-31T07:27:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.613+0000 7ff696000640 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55ce1a90e9f0 0x55ce1ad13780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.614+0000 7ff696000640 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55ce1a90e9f0 0x55ce1ad13780 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b01b29ddce497601 server_cookie=ccdea87c28978a7a in_seq=0 out_seq=0 2024-08-31T07:27:16.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.614+0000 7ff680c00640 10 client.28451 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:27:16.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff680c00640 1 -- 172.21.15.32:0/2328310805 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce1ac6e700 con 0x55ce1a90e9f0 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff680c00640 10 client.28451 handle_command_reply: tid=0 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff6a482c740 1 client.28451 shutdown 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff6a482c740 1 -- 172.21.15.32:0/2328310805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 msgr2=0x55ce1ad01580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad01580 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff688006010 tx=0x7ff688009260 comp rx=0 tx=0).stop 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff6a482c740 1 -- 172.21.15.32:0/2328310805 shutdown_connections 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55ce1a90e9f0 0x55ce1ad13780 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a950070 0x55ce1ad0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a92f390 0x55ce1ad01b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2328310805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad01580 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff6a482c740 1 -- 172.21.15.32:0/2328310805 >> 172.21.15.32:0/2328310805 conn(0x55ce1a944460 msgr2=0x55ce1acf8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:16.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff680c00640 0 client.28451 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:27:16.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.637+0000 7ff6a482c740 1 -- 172.21.15.32:0/2328310805 shutdown_connections 2024-08-31T07:27:16.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.638+0000 7ff6a482c740 1 -- 172.21.15.32:0/2328310805 wait complete. 2024-08-31T07:27:16.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.638+0000 7ff6a482c740 20 client.28451 trim_cache size 0 max 16384 2024-08-31T07:27:16.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.652+0000 7ff6a482c740 1 Processor -- start 2024-08-31T07:27:16.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.652+0000 7ff6a482c740 1 -- start start 2024-08-31T07:27:16.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.652+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad0d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.652+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1acffc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.652+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a950070 0x55ce1ad00160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.652+0000 7ff6a482c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981b15d0 con 0x55ce1a92f390 2024-08-31T07:27:16.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.652+0000 7ff6a482c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981b32f0 con 0x55ce1a9271b0 2024-08-31T07:27:16.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.652+0000 7ff6a482c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981b1cd0 con 0x55ce1a950070 2024-08-31T07:27:16.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff696a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad0d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff696a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad0d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:40368/0 (socket says 172.21.15.32:40368) 2024-08-31T07:27:16.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff696a00640 1 -- 172.21.15.32:0/1439981499 learned_addr learned my addr 172.21.15.32:0/1439981499 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:16.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff697400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a950070 0x55ce1ad00160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff696000640 1 --2- 172.21.15.32:0/1439981499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1acffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff696a00640 1 -- 172.21.15.32:0/1439981499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a950070 msgr2=0x55ce1ad00160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:16.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff696a00640 1 --2- 172.21.15.32:0/1439981499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a950070 0x55ce1ad00160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff696a00640 1 -- 172.21.15.32:0/1439981499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 msgr2=0x55ce1acffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff696a00640 1 --2- 172.21.15.32:0/1439981499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1acffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff696a00640 1 -- 172.21.15.32:0/1439981499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1ad02640 con 0x55ce1a9271b0 2024-08-31T07:27:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.653+0000 7ff696000640 1 --2- 172.21.15.32:0/1439981499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1acffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:27:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.660+0000 7ff696a00640 1 --2- 172.21.15.32:0/1439981499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad0d390 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff688046ee0 tx=0x7ff68802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2b3ce77aaa39989 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.661+0000 7ff683400640 1 -- 172.21.15.32:0/1439981499 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688031d10 con 0x55ce1a9271b0 2024-08-31T07:27:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.661+0000 7ff683400640 1 -- 172.21.15.32:0/1439981499 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff68803dce0 con 0x55ce1a9271b0 2024-08-31T07:27:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.661+0000 7ff683400640 1 -- 172.21.15.32:0/1439981499 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688046880 con 0x55ce1a9271b0 2024-08-31T07:27:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.662+0000 7ff6a482c740 1 -- 172.21.15.32:0/1439981499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 msgr2=0x55ce1ad0d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.662+0000 7ff6a482c740 1 --2- 172.21.15.32:0/1439981499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad0d390 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff688046ee0 tx=0x7ff68802a420 comp rx=0 tx=0).stop 2024-08-31T07:27:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.662+0000 7ff6a482c740 1 -- 172.21.15.32:0/1439981499 shutdown_connections 2024-08-31T07:27:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.662+0000 7ff6a482c740 1 --2- 172.21.15.32:0/1439981499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a950070 0x55ce1ad00160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.662+0000 7ff6a482c740 1 --2- 172.21.15.32:0/1439981499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1acffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.662+0000 7ff6a482c740 1 --2- 172.21.15.32:0/1439981499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad0d390 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.662+0000 7ff6a482c740 1 -- 172.21.15.32:0/1439981499 >> 172.21.15.32:0/1439981499 conn(0x55ce1a944460 msgr2=0x55ce1acfc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.662+0000 7ff6a482c740 1 -- 172.21.15.32:0/1439981499 shutdown_connections 2024-08-31T07:27:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.662+0000 7ff6a482c740 1 -- 172.21.15.32:0/1439981499 wait complete. 2024-08-31T07:27:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.662+0000 7ff6a482c740 1 -- start start 2024-08-31T07:27:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.663+0000 7ff6a482c740 1 Processor -- start 2024-08-31T07:27:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.663+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad05540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.663+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1ad05a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.663+0000 7ff6a482c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a950070 0x55ce1ad05fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.663+0000 7ff6a482c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981b2b60 con 0x55ce1a92f390 2024-08-31T07:27:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.663+0000 7ff6a482c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981b2430 con 0x55ce1a9271b0 2024-08-31T07:27:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.663+0000 7ff6a482c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981af5d0 con 0x55ce1a950070 2024-08-31T07:27:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.663+0000 7ff6a482c740 10 client.0 _ll_register_callbacks cb 0x7ff6880045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.664+0000 7ff696000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1ad05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.664+0000 7ff696000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1ad05a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41290/0 (socket says 172.21.15.32:41290) 2024-08-31T07:27:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.664+0000 7ff696000640 1 -- 172.21.15.32:0/2089823682 learned_addr learned my addr 172.21.15.32:0/2089823682 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.664+0000 7ff696a00640 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad05540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.664+0000 7ff696a00640 1 -- 172.21.15.32:0/2089823682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a950070 msgr2=0x55ce1ad05fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:27:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.664+0000 7ff696a00640 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a950070 0x55ce1ad05fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.664+0000 7ff696a00640 1 -- 172.21.15.32:0/2089823682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 msgr2=0x55ce1ad05a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.664+0000 7ff696a00640 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1ad05a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.664+0000 7ff696a00640 1 -- 172.21.15.32:0/2089823682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1ad7d270 con 0x55ce1a9271b0 2024-08-31T07:27:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.664+0000 7ff696000640 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1ad05a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:16.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.672+0000 7ff696a00640 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad05540 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff688046ee0 tx=0x7ff688031f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31bca9e36b793c94 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:16.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.672+0000 7ff680c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:27:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.672+0000 7ff680c00640 1 -- 172.21.15.32:0/2089823682 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68803da40 con 0x55ce1a9271b0 2024-08-31T07:27:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.673+0000 7ff680c00640 1 -- 172.21.15.32:0/2089823682 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7ff688046580 con 0x55ce1a9271b0 2024-08-31T07:27:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.673+0000 7ff6a482c740 1 -- 172.21.15.32:0/2089823682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff68803dce0 con 0x55ce1a9271b0 2024-08-31T07:27:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.673+0000 7ff680c00640 1 -- 172.21.15.32:0/2089823682 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff688072050 con 0x55ce1a9271b0 2024-08-31T07:27:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.673+0000 7ff6a482c740 10 client.28457 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.673+0000 7ff6a482c740 1 -- 172.21.15.32:0/2089823682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ce1ad7e4f0 con 0x55ce1a9271b0 2024-08-31T07:27:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.673+0000 7ff680c00640 1 -- 172.21.15.32:0/2089823682 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff68803dce0 con 0x55ce1a9271b0 2024-08-31T07:27:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.673+0000 7ff680c00640 1 -- 172.21.15.32:0/2089823682 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7ff68804f070 con 0x55ce1a9271b0 2024-08-31T07:27:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.674+0000 7ff6a482c740 10 client.28457 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.674+0000 7ff6a482c740 10 client.28457 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:27:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.674+0000 7ff6a482c740 20 client.28457 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.674+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55ce1a8e3da0 0x55ce1ad80bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.674+0000 7ff6a482c740 4 client.28457 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:27:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.674+0000 7ff6a482c740 1 -- 172.21.15.32:0/2089823682 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce1ac6a1a0 con 0x55ce1a8e3da0 2024-08-31T07:27:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.674+0000 7ff696000640 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55ce1a8e3da0 0x55ce1ad80bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:16.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff680c00640 10 client.28457 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff696000640 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55ce1a8e3da0 0x55ce1ad80bc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10be8c0058b90c79 server_cookie=70f77ab2cb54ccdd in_seq=0 out_seq=0 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff680c00640 1 -- 172.21.15.32:0/2089823682 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce1ac6a1a0 con 0x55ce1a8e3da0 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff680c00640 10 client.28457 handle_command_reply: tid=0 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff6a482c740 1 client.28457 shutdown 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff6a482c740 1 -- 172.21.15.32:0/2089823682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 msgr2=0x55ce1ad05540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad05540 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff688046ee0 tx=0x7ff688031f40 comp rx=0 tx=0).stop 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff6a482c740 1 -- 172.21.15.32:0/2089823682 shutdown_connections 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55ce1a8e3da0 0x55ce1ad80bc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce1a950070 0x55ce1ad05fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ce1a92f390 0x55ce1ad05a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff6a482c740 1 --2- 172.21.15.32:0/2089823682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce1a9271b0 0x55ce1ad05540 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff6a482c740 1 -- 172.21.15.32:0/2089823682 >> 172.21.15.32:0/2089823682 conn(0x55ce1a944460 msgr2=0x55ce1acffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.675+0000 7ff680c00640 0 client.28457 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:27:16.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.676+0000 7ff6a482c740 1 -- 172.21.15.32:0/2089823682 shutdown_connections 2024-08-31T07:27:16.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.676+0000 7ff6a482c740 1 -- 172.21.15.32:0/2089823682 wait complete. 2024-08-31T07:27:16.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.676+0000 7ff6a482c740 20 client.28457 trim_cache size 0 max 16384 2024-08-31T07:27:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 -- 172.21.15.32:0/331646235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff67c0b4c30 msgr2=0x7ff67c0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff67c0b4c30 0x7ff67c0b7140 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55ce1acf4610 tx=0x7ff688005a20 comp rx=0 tx=0).stop 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 -- 172.21.15.32:0/331646235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff698109240 msgr2=0x7ff69819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff698109240 0x7ff69819ca50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff69806c790 tx=0x7ff68400b850 comp rx=0 tx=0).stop 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 -- 172.21.15.32:0/331646235 shutdown_connections 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff67c0b4c30 0x7ff67c0b7140 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff698109b90 0x7ff6981a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff698109240 0x7ff69819ca50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 --2- 172.21.15.32:0/331646235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff698103900 0x7ff69819c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 -- 172.21.15.32:0/331646235 >> 172.21.15.32:0/331646235 conn(0x7ff69807b5f0 msgr2=0x7ff698107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 -- 172.21.15.32:0/331646235 shutdown_connections 2024-08-31T07:27:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:16.677+0000 7ff69dc00640 1 -- 172.21.15.32:0/331646235 wait complete. 2024-08-31T07:27:16.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:16 smithi073 ceph-mon[21995]: pgmap v821: 97 pgs: 97 active+clean; 679 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.4 MiB/s wr, 73 op/s 2024-08-31T07:27:17.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23387 25.17 MB/sec warmup 57 sec latency 32507.647 ms 2024-08-31T07:27:18.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23387 24.74 MB/sec warmup 58 sec latency 33507.692 ms 2024-08-31T07:27:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:18 smithi084 ceph-mon[22387]: pgmap v822: 97 pgs: 97 active+clean; 686 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 3.5 MiB/s wr, 140 op/s 2024-08-31T07:27:18.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:18 smithi032 ceph-mon[17252]: pgmap v822: 97 pgs: 97 active+clean; 686 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 3.5 MiB/s wr, 140 op/s 2024-08-31T07:27:18.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:18 smithi073 ceph-mon[21995]: pgmap v822: 97 pgs: 97 active+clean; 686 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 3.5 MiB/s wr, 140 op/s 2024-08-31T07:27:19.328 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23387 24.32 MB/sec warmup 59 sec latency 34507.750 ms 2024-08-31T07:27:20.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23387 23.91 MB/sec warmup 60 sec latency 35507.810 ms 2024-08-31T07:27:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:20 smithi084 ceph-mon[22387]: pgmap v823: 97 pgs: 97 active+clean; 686 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 4.1 MiB/s wr, 161 op/s 2024-08-31T07:27:20.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:20 smithi032 ceph-mon[17252]: pgmap v823: 97 pgs: 97 active+clean; 686 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 4.1 MiB/s wr, 161 op/s 2024-08-31T07:27:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:20 smithi073 ceph-mon[21995]: pgmap v823: 97 pgs: 97 active+clean; 686 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 4.1 MiB/s wr, 161 op/s 2024-08-31T07:27:21.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23387 23.52 MB/sec warmup 61 sec latency 36507.853 ms 2024-08-31T07:27:22.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23387 23.14 MB/sec warmup 62 sec latency 37507.913 ms 2024-08-31T07:27:22.486 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:22 smithi032 ceph-mon[17252]: pgmap v824: 97 pgs: 97 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 5.6 MiB/s wr, 204 op/s 2024-08-31T07:27:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:22 smithi084 ceph-mon[22387]: pgmap v824: 97 pgs: 97 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 5.6 MiB/s wr, 204 op/s 2024-08-31T07:27:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:22 smithi073 ceph-mon[21995]: pgmap v824: 97 pgs: 97 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 5.6 MiB/s wr, 204 op/s 2024-08-31T07:27:23.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:27:23.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23387 22.77 MB/sec warmup 63 sec latency 38152.440 ms 2024-08-31T07:27:23.631 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2815600640 1 -- 172.21.15.32:0/410601387 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55dc774fef20 con 0x55dc77615e20 2024-08-31T07:27:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2815600640 10 client.18417 handle_command_reply: tid=0 2024-08-31T07:27:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2839621740 1 client.18417 shutdown 2024-08-31T07:27:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2839621740 1 -- 172.21.15.32:0/410601387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 msgr2=0x55dc775a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2839621740 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc775a0730 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2818000c00 tx=0x7f28180c8750 comp rx=0 tx=0).stop 2024-08-31T07:27:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2839621740 1 -- 172.21.15.32:0/410601387 shutdown_connections 2024-08-31T07:27:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2839621740 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dc77615e20 0x55dc77618240 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2839621740 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc771e5840 0x55dc7759ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2839621740 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dc771c4b60 0x55dc775a0730 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2839621740 1 --2- 172.21.15.32:0/410601387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc77179570 0x55dc775a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2815600640 0 client.18417 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.669+0000 7f2839621740 1 -- 172.21.15.32:0/410601387 >> 172.21.15.32:0/410601387 conn(0x55dc771d9990 msgr2=0x55dc7758d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.670+0000 7f2839621740 1 -- 172.21.15.32:0/410601387 shutdown_connections 2024-08-31T07:27:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.670+0000 7f2839621740 1 -- 172.21.15.32:0/410601387 wait complete. 2024-08-31T07:27:23.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.670+0000 7f2839621740 20 client.18417 trim_cache size 0 max 16384 2024-08-31T07:27:23.673 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:27:23.673 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 55, 2024-08-31T07:27:23.673 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:27:23.673 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 46, 2024-08-31T07:27:23.673 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:27:23.673 INFO:teuthology.orchestra.run.smithi032.stdout: "f9a618dc": { 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 46, 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 155.0, 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:27:23.674 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:27:23.675 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:27:23.675 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:27:23.675 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:27:23.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:23.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:23.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:23.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:23.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:23.676 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:27:23.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.671+0000 7f2831400640 1 -- 172.21.15.32:0/2991008163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4ba0 msgr2=0x7f28100b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.671+0000 7f2831400640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4ba0 0x7f28100b70b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2824004910 tx=0x7f2824000f40 comp rx=0 tx=0).stop 2024-08-31T07:27:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.671+0000 7f2831400640 1 -- 172.21.15.32:0/2991008163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c1090a0 msgr2=0x7f282c19c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.671+0000 7f2831400640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c1090a0 0x7f282c19c990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2818009230 tx=0x7f2818003c10 comp rx=0 tx=0).stop 2024-08-31T07:27:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.672+0000 7f2831400640 1 -- 172.21.15.32:0/2991008163 shutdown_connections 2024-08-31T07:27:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.672+0000 7f2831400640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f28100b4ba0 0x7f28100b70b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.672+0000 7f2831400640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f282c1099f0 0x7f282c1a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.672+0000 7f2831400640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f282c1090a0 0x7f282c19c990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.672+0000 7f2831400640 1 --2- 172.21.15.32:0/2991008163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f282c103760 0x7f282c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.672+0000 7f2831400640 1 -- 172.21.15.32:0/2991008163 >> 172.21.15.32:0/2991008163 conn(0x7f282c0ff1d0 msgr2=0x7f282c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:23.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.672+0000 7f2831400640 1 -- 172.21.15.32:0/2991008163 shutdown_connections 2024-08-31T07:27:23.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:23.672+0000 7f2831400640 1 -- 172.21.15.32:0/2991008163 wait complete. 2024-08-31T07:27:24.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23394 22.42 MB/sec warmup 64 sec latency 10717.173 ms 2024-08-31T07:27:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:24 smithi084 ceph-mon[22387]: pgmap v825: 97 pgs: 97 active+clean; 686 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 4.5 MiB/s wr, 189 op/s 2024-08-31T07:27:24.539 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:24 smithi032 ceph-mon[17252]: pgmap v825: 97 pgs: 97 active+clean; 686 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 4.5 MiB/s wr, 189 op/s 2024-08-31T07:27:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.627+0000 7fa58c200640 1 -- 172.21.15.32:0/1639308268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5840776e0 msgr2=0x7fa584077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.627+0000 7fa58c200640 1 --2- 172.21.15.32:0/1639308268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5840776e0 0x7fa584077af0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa578009900 tx=0x7fa57802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.627+0000 7fa58c200640 1 -- 172.21.15.32:0/1639308268 shutdown_connections 2024-08-31T07:27:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.627+0000 7fa58c200640 1 --2- 172.21.15.32:0/1639308268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5841144e0 0x7fa584116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.627+0000 7fa58c200640 1 --2- 172.21.15.32:0/1639308268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa584078030 0x7fa584113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.627+0000 7fa58c200640 1 --2- 172.21.15.32:0/1639308268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5840776e0 0x7fa584077af0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.627+0000 7fa58c200640 1 -- 172.21.15.32:0/1639308268 >> 172.21.15.32:0/1639308268 conn(0x7fa5841015b0 msgr2=0x7fa584103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.627+0000 7fa58c200640 1 -- 172.21.15.32:0/1639308268 shutdown_connections 2024-08-31T07:27:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.627+0000 7fa58c200640 1 -- 172.21.15.32:0/1639308268 wait complete. 2024-08-31T07:27:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.628+0000 7fa58c200640 1 Processor -- start 2024-08-31T07:27:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.628+0000 7fa58c200640 1 -- start start 2024-08-31T07:27:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa58c200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5840776e0 0x7fa5841a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa58c200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa584078030 0x7fa5841a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa58c200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5841144e0 0x7fa5841a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa58c200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa584119e40 con 0x7fa5841144e0 2024-08-31T07:27:24.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa58c200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa584119cc0 con 0x7fa5840776e0 2024-08-31T07:27:24.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa58c200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa584119fc0 con 0x7fa584078030 2024-08-31T07:27:24.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa589800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5840776e0 0x7fa5841a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa589800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5840776e0 0x7fa5841a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:40398/0 (socket says 172.21.15.32:40398) 2024-08-31T07:27:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa589800640 1 -- 172.21.15.32:0/3238214718 learned_addr learned my addr 172.21.15.32:0/3238214718 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa588e00640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa584078030 0x7fa5841a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.629+0000 7fa58a200640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5841144e0 0x7fa5841a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa589800640 1 -- 172.21.15.32:0/3238214718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa584078030 msgr2=0x7fa5841a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa589800640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa584078030 0x7fa5841a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa589800640 1 -- 172.21.15.32:0/3238214718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5841144e0 msgr2=0x7fa5841a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa589800640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5841144e0 0x7fa5841a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa589800640 1 -- 172.21.15.32:0/3238214718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5841a67a0 con 0x7fa5840776e0 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa589800640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5840776e0 0x7fa5841a0830 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa578005940 tx=0x7fa5780040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa582000640 1 -- 172.21.15.32:0/3238214718 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57803f030 con 0x7fa5840776e0 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa58c200640 1 -- 172.21.15.32:0/3238214718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5841a6a30 con 0x7fa5840776e0 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa582000640 1 -- 172.21.15.32:0/3238214718 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5780044c0 con 0x7fa5840776e0 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa582000640 1 -- 172.21.15.32:0/3238214718 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578035070 con 0x7fa5840776e0 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.630+0000 7fa58c200640 1 -- 172.21.15.32:0/3238214718 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5841ad1f0 con 0x7fa5840776e0 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.633+0000 7fa592e19740 1 Processor -- start 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa592e19740 1 -- start start 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa592e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa592e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdeef1b0 0x562cfe2bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa592e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdef7390 0x562cfe2be410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa592e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562cfe2347b0 con 0x562cfdeef1b0 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa592e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562cfe223d40 con 0x562cfdf18070 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa592e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562cfe227cf0 con 0x562cfdef7390 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa58a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa582000640 1 -- 172.21.15.32:0/3238214718 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa578042030 con 0x7fa5840776e0 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa58a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2bd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:40400/0 (socket says 172.21.15.32:40400) 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa58a200640 1 -- 172.21.15.32:0/3479839055 learned_addr learned my addr 172.21.15.32:0/3479839055 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa589800640 1 --2- 172.21.15.32:0/3479839055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdeef1b0 0x562cfe2bded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.634+0000 7fa588e00640 1 --2- 172.21.15.32:0/3479839055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdef7390 0x562cfe2be410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.635+0000 7fa58a200640 1 -- 172.21.15.32:0/3479839055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdef7390 msgr2=0x562cfe2be410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.635+0000 7fa58a200640 1 --2- 172.21.15.32:0/3479839055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdef7390 0x562cfe2be410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.635+0000 7fa58a200640 1 -- 172.21.15.32:0/3479839055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdeef1b0 msgr2=0x562cfe2bded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.635+0000 7fa58a200640 1 --2- 172.21.15.32:0/3479839055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdeef1b0 0x562cfe2bded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.635+0000 7fa58a200640 1 -- 172.21.15.32:0/3479839055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfdf83650 con 0x562cfdf18070 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.635+0000 7fa58a200640 1 --2- 172.21.15.32:0/3479839055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2bd990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009fd0 tx=0x7fa57c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93eba3a6d6de268c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.635+0000 7fa576a00640 1 -- 172.21.15.32:0/3479839055 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c004d20 con 0x562cfdf18070 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.635+0000 7fa576a00640 1 -- 172.21.15.32:0/3479839055 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa57c004050 con 0x562cfdf18070 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.635+0000 7fa576a00640 1 -- 172.21.15.32:0/3479839055 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c005260 con 0x562cfdf18070 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.636+0000 7fa582000640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa56c0b4da0 0x7fa56c0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.636+0000 7fa592e19740 1 -- 172.21.15.32:0/3479839055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 msgr2=0x562cfe2bd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.636+0000 7fa592e19740 1 --2- 172.21.15.32:0/3479839055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2bd990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009fd0 tx=0x7fa57c0118d0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.636+0000 7fa582000640 1 -- 172.21.15.32:0/3238214718 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7fa578105850 con 0x7fa5840776e0 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.636+0000 7fa589800640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa56c0b4da0 0x7fa56c0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.636+0000 7fa592e19740 1 -- 172.21.15.32:0/3479839055 shutdown_connections 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.636+0000 7fa592e19740 1 --2- 172.21.15.32:0/3479839055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdef7390 0x562cfe2be410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.636+0000 7fa592e19740 1 --2- 172.21.15.32:0/3479839055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdeef1b0 0x562cfe2bded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.636+0000 7fa592e19740 1 --2- 172.21.15.32:0/3479839055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2bd990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.636+0000 7fa592e19740 1 -- 172.21.15.32:0/3479839055 >> 172.21.15.32:0/3479839055 conn(0x562cfdf0c460 msgr2=0x562cfe2bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.637+0000 7fa592e19740 1 -- 172.21.15.32:0/3479839055 shutdown_connections 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.637+0000 7fa592e19740 1 -- 172.21.15.32:0/3479839055 wait complete. 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.637+0000 7fa589800640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa56c0b4da0 0x7fa56c0b72b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x562cfe2bc610 tx=0x7fa57803b4f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.637+0000 7fa592e19740 1 -- start start 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.637+0000 7fa592e19740 1 Processor -- start 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa592e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa592e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa592e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdf18070 0x562cfe2d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa592e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562cfe086700 con 0x562cfdef7390 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa592e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562cfe1aacf0 con 0x562cfdeef1b0 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa592e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562cfe2cc170 con 0x562cfdf18070 2024-08-31T07:27:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa592e19740 10 client.0 _ll_register_callbacks cb 0x562cfe1d0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa58a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa588e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa58a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:40414/0 (socket says 172.21.15.32:40414) 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.638+0000 7fa58a200640 1 -- 172.21.15.32:0/2256785619 learned_addr learned my addr 172.21.15.32:0/2256785619 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.639+0000 7fa58a200640 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdf18070 0x562cfe2d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.639+0000 7fa588e00640 1 -- 172.21.15.32:0/2256785619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdf18070 msgr2=0x562cfe2d5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.639+0000 7fa588e00640 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdf18070 0x562cfe2d5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.639+0000 7fa588e00640 1 -- 172.21.15.32:0/2256785619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 msgr2=0x562cfe2c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.639+0000 7fa588e00640 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.639+0000 7fa588e00640 1 -- 172.21.15.32:0/2256785619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfe1eb8b0 con 0x562cfdef7390 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.639+0000 7fa58a200640 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.639+0000 7fa588e00640 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c9b70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa58406c5b0 tx=0x7fa570012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae42883b6af493a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.639+0000 7fa56b400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.640+0000 7fa56b400640 1 -- 172.21.15.32:0/2256785619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570004050 con 0x562cfdef7390 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.640+0000 7fa56b400640 1 -- 172.21.15.32:0/2256785619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7fa5700041f0 con 0x562cfdef7390 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.640+0000 7fa592e19740 1 -- 172.21.15.32:0/2256785619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa57c004050 con 0x562cfdef7390 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.640+0000 7fa56b400640 1 -- 172.21.15.32:0/2256785619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa5700043f0 con 0x562cfdef7390 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.640+0000 7fa592e19740 10 client.18450 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.640+0000 7fa592e19740 1 -- 172.21.15.32:0/2256785619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562cfe2d8220 con 0x562cfdef7390 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.640+0000 7fa56b400640 1 -- 172.21.15.32:0/2256785619 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa57c004050 con 0x562cfdef7390 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.640+0000 7fa56b400640 1 -- 172.21.15.32:0/2256785619 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7fa570003b10 con 0x562cfdef7390 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.641+0000 7fa592e19740 10 client.18450 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.641+0000 7fa592e19740 10 client.18450 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.641+0000 7fa592e19740 20 client.18450 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.641+0000 7fa592e19740 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562cfded69f0 0x562cfe2db6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.641+0000 7fa592e19740 4 client.18450 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.641+0000 7fa592e19740 1 -- 172.21.15.32:0/2256785619 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cfe236700 con 0x562cfded69f0 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.641+0000 7fa58a200640 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562cfded69f0 0x562cfe2db6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.641+0000 7fa58a200640 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562cfded69f0 0x562cfe2db6b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf3ab0664f3e4279 server_cookie=a7fed82e784d0274 in_seq=0 out_seq=0 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.641+0000 7fa56b400640 10 client.18450 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.642+0000 7fa56b400640 1 -- 172.21.15.32:0/2256785619 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562cfe236700 con 0x562cfded69f0 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.642+0000 7fa56b400640 10 client.18450 handle_command_reply: tid=0 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.642+0000 7fa592e19740 1 client.18450 shutdown 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 1 -- 172.21.15.32:0/2256785619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 msgr2=0x562cfe2c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c9b70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa58406c5b0 tx=0x7fa570012b60 comp rx=0 tx=0).stop 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 1 -- 172.21.15.32:0/2256785619 shutdown_connections 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562cfded69f0 0x562cfe2db6b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdf18070 0x562cfe2d5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c9b70 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 1 --2- 172.21.15.32:0/2256785619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 1 -- 172.21.15.32:0/2256785619 >> 172.21.15.32:0/2256785619 conn(0x562cfdf0c460 msgr2=0x562cfe2c0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa56b400640 0 client.18450 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:27:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 1 -- 172.21.15.32:0/2256785619 shutdown_connections 2024-08-31T07:27:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 1 -- 172.21.15.32:0/2256785619 wait complete. 2024-08-31T07:27:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.643+0000 7fa592e19740 20 client.18450 trim_cache size 0 max 16384 2024-08-31T07:27:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.657+0000 7fa592e19740 1 Processor -- start 2024-08-31T07:27:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.657+0000 7fa592e19740 1 -- start start 2024-08-31T07:27:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa592e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdeef1b0 0x562cfe2c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa592e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa592e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa592e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841b5990 con 0x562cfdef7390 2024-08-31T07:27:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa592e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841b76b0 con 0x562cfdf18070 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa592e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841b6090 con 0x562cfdeef1b0 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa588e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa588e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41340/0 (socket says 172.21.15.32:41340) 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa588e00640 1 -- 172.21.15.32:0/393188502 learned_addr learned my addr 172.21.15.32:0/393188502 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa58a200640 1 --2- 172.21.15.32:0/393188502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdeef1b0 0x562cfe2c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa58a200640 1 --2- 172.21.15.32:0/393188502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa588e00640 1 -- 172.21.15.32:0/393188502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdeef1b0 msgr2=0x562cfe2c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.658+0000 7fa588e00640 1 --2- 172.21.15.32:0/393188502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdeef1b0 0x562cfe2c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.659+0000 7fa588e00640 1 -- 172.21.15.32:0/393188502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 msgr2=0x562cfe2c2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.659+0000 7fa588e00640 1 --2- 172.21.15.32:0/393188502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2c2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.659+0000 7fa588e00640 1 -- 172.21.15.32:0/393188502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfe2bfc80 con 0x562cfdef7390 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.659+0000 7fa58a200640 1 --2- 172.21.15.32:0/393188502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.659+0000 7fa588e00640 1 --2- 172.21.15.32:0/393188502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c81f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa570000c00 tx=0x7fa570026880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a5675d65b71e92 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.659+0000 7fa576000640 1 -- 172.21.15.32:0/393188502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570003fe0 con 0x562cfdef7390 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.659+0000 7fa576000640 1 -- 172.21.15.32:0/393188502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa570026e50 con 0x562cfdef7390 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.659+0000 7fa576000640 1 -- 172.21.15.32:0/393188502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570005040 con 0x562cfdef7390 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.660+0000 7fa592e19740 1 -- 172.21.15.32:0/393188502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 msgr2=0x562cfe2c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.660+0000 7fa592e19740 1 --2- 172.21.15.32:0/393188502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c81f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa570000c00 tx=0x7fa570026880 comp rx=0 tx=0).stop 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.660+0000 7fa592e19740 1 -- 172.21.15.32:0/393188502 shutdown_connections 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.660+0000 7fa592e19740 1 --2- 172.21.15.32:0/393188502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdf18070 0x562cfe2c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.660+0000 7fa592e19740 1 --2- 172.21.15.32:0/393188502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2c81f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.660+0000 7fa592e19740 1 --2- 172.21.15.32:0/393188502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdeef1b0 0x562cfe2c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.660+0000 7fa592e19740 1 -- 172.21.15.32:0/393188502 >> 172.21.15.32:0/393188502 conn(0x562cfdf0c460 msgr2=0x562cfe2c4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.660+0000 7fa592e19740 1 -- 172.21.15.32:0/393188502 shutdown_connections 2024-08-31T07:27:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.660+0000 7fa592e19740 1 -- 172.21.15.32:0/393188502 wait complete. 2024-08-31T07:27:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.661+0000 7fa592e19740 1 -- start start 2024-08-31T07:27:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.661+0000 7fa592e19740 1 Processor -- start 2024-08-31T07:27:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.661+0000 7fa592e19740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa592e19740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa592e19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdf18070 0x562cfe2cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa592e19740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841b6f20 con 0x562cfdef7390 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa592e19740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841b67f0 con 0x562cfdeef1b0 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa592e19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841b3990 con 0x562cfdf18070 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa592e19740 10 client.0 _ll_register_callbacks cb 0x7fa570009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa58a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa588e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa58a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:40426/0 (socket says 172.21.15.32:40426) 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa58a200640 1 -- 172.21.15.32:0/2386313790 learned_addr learned my addr 172.21.15.32:0/2386313790 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa58a200640 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdf18070 0x562cfe2cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa58a200640 1 -- 172.21.15.32:0/2386313790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdf18070 msgr2=0x562cfe2cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa58a200640 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdf18070 0x562cfe2cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa58a200640 1 -- 172.21.15.32:0/2386313790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 msgr2=0x562cfe2cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:24.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa58a200640 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa58a200640 1 -- 172.21.15.32:0/2386313790 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfe2cd8c0 con 0x562cfdeef1b0 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.662+0000 7fa588e00640 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.663+0000 7fa58a200640 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2d1990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009fa0 tx=0x7fa570009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce978db94595928 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.663+0000 7fa56b400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.663+0000 7fa56b400640 1 -- 172.21.15.32:0/2386313790 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c004530 con 0x562cfdeef1b0 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.663+0000 7fa592e19740 1 -- 172.21.15.32:0/2386313790 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa570026e50 con 0x562cfdeef1b0 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.663+0000 7fa56b400640 1 -- 172.21.15.32:0/2386313790 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7fa57c01f040 con 0x562cfdeef1b0 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.663+0000 7fa56b400640 1 -- 172.21.15.32:0/2386313790 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa57c0046d0 con 0x562cfdeef1b0 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.663+0000 7fa592e19740 10 client.28475 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.663+0000 7fa592e19740 1 -- 172.21.15.32:0/2386313790 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562cfe345da0 con 0x562cfdeef1b0 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.664+0000 7fa56b400640 1 -- 172.21.15.32:0/2386313790 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa570026e50 con 0x562cfdeef1b0 2024-08-31T07:27:24.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.664+0000 7fa56b400640 1 -- 172.21.15.32:0/2386313790 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7fa57c028450 con 0x562cfdeef1b0 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.664+0000 7fa592e19740 10 client.28475 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.664+0000 7fa592e19740 10 client.28475 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.664+0000 7fa592e19740 20 client.28475 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.664+0000 7fa592e19740 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562cfdeabda0 0x562cfe348440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.664+0000 7fa592e19740 4 client.28475 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.664+0000 7fa592e19740 1 -- 172.21.15.32:0/2386313790 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cfe2321a0 con 0x562cfdeabda0 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.664+0000 7fa588e00640 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562cfdeabda0 0x562cfe348440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa588e00640 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562cfdeabda0 0x562cfe348440 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c4b8fea787e509d server_cookie=9b50e424ca234c8e in_seq=0 out_seq=0 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa56b400640 10 client.28475 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa56b400640 1 -- 172.21.15.32:0/2386313790 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cfe2321a0 con 0x562cfdeabda0 2024-08-31T07:27:24.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa56b400640 10 client.28475 handle_command_reply: tid=0 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa592e19740 1 client.28475 shutdown 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa592e19740 1 -- 172.21.15.32:0/2386313790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 msgr2=0x562cfe2d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa592e19740 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2d1990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009fa0 tx=0x7fa570009ca0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa592e19740 1 -- 172.21.15.32:0/2386313790 shutdown_connections 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa592e19740 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x562cfdeabda0 0x562cfe348440 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa592e19740 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562cfdf18070 0x562cfe2cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa592e19740 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562cfdef7390 0x562cfe2cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa56b400640 0 client.28475 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa592e19740 1 --2- 172.21.15.32:0/2386313790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cfdeef1b0 0x562cfe2d1990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.665+0000 7fa592e19740 1 -- 172.21.15.32:0/2386313790 >> 172.21.15.32:0/2386313790 conn(0x562cfdf0c460 msgr2=0x562cfe2c4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.666+0000 7fa592e19740 1 -- 172.21.15.32:0/2386313790 shutdown_connections 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.666+0000 7fa592e19740 1 -- 172.21.15.32:0/2386313790 wait complete. 2024-08-31T07:27:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.666+0000 7fa592e19740 20 client.28475 trim_cache size 0 max 16384 2024-08-31T07:27:24.672 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.666+0000 7fa58c200640 1 -- 172.21.15.32:0/3238214718 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa56c0b4da0 msgr2=0x7fa56c0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.666+0000 7fa58c200640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa56c0b4da0 0x7fa56c0b72b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x562cfe2bc610 tx=0x7fa57803b4f0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.666+0000 7fa58c200640 1 -- 172.21.15.32:0/3238214718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5840776e0 msgr2=0x7fa5841a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.666+0000 7fa58c200640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5840776e0 0x7fa5841a0830 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa578005940 tx=0x7fa5780040c0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.667+0000 7fa58c200640 1 -- 172.21.15.32:0/3238214718 shutdown_connections 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.667+0000 7fa58c200640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa56c0b4da0 0x7fa56c0b72b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.667+0000 7fa58c200640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5841144e0 0x7fa5841a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.667+0000 7fa58c200640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa584078030 0x7fa5841a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.667+0000 7fa58c200640 1 --2- 172.21.15.32:0/3238214718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5840776e0 0x7fa5841a0830 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.667+0000 7fa58c200640 1 -- 172.21.15.32:0/3238214718 >> 172.21.15.32:0/3238214718 conn(0x7fa5841015b0 msgr2=0x7fa584116610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.667+0000 7fa58c200640 1 -- 172.21.15.32:0/3238214718 shutdown_connections 2024-08-31T07:27:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:24.667+0000 7fa58c200640 1 -- 172.21.15.32:0/3238214718 wait complete. 2024-08-31T07:27:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:24 smithi073 ceph-mon[21995]: pgmap v825: 97 pgs: 97 active+clean; 686 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 4.5 MiB/s wr, 189 op/s 2024-08-31T07:27:25.062 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f9a618dc', quiesce took 36.3 seconds. Will release after: -1.2999999999999972 2024-08-31T07:27:25.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:27:25.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23394 22.07 MB/sec warmup 65 sec latency 11717.255 ms 2024-08-31T07:27:25.837 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:27:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:26.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23394 21.74 MB/sec warmup 66 sec latency 12717.383 ms 2024-08-31T07:27:26.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:26 smithi032 ceph-mon[17252]: pgmap v826: 97 pgs: 97 active+clean; 683 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 4.3 MiB/s wr, 181 op/s 2024-08-31T07:27:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:26 smithi084 ceph-mon[22387]: pgmap v826: 97 pgs: 97 active+clean; 683 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 4.3 MiB/s wr, 181 op/s 2024-08-31T07:27:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.585+0000 7f05c7a00640 1 -- 172.21.15.32:0/4100163172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c0075550 msgr2=0x7f05c0075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.585+0000 7f05c7a00640 1 --2- 172.21.15.32:0/4100163172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c0075550 0x7f05c0075960 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009900 tx=0x7f05b402a410 comp rx=0 tx=0).stop 2024-08-31T07:27:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.585+0000 7f05c7a00640 1 -- 172.21.15.32:0/4100163172 shutdown_connections 2024-08-31T07:27:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.585+0000 7f05c7a00640 1 --2- 172.21.15.32:0/4100163172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05c010fe70 0x7f05c01122b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.585+0000 7f05c7a00640 1 --2- 172.21.15.32:0/4100163172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c0075ea0 0x7f05c010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.585+0000 7f05c7a00640 1 --2- 172.21.15.32:0/4100163172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c0075550 0x7f05c0075960 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.586+0000 7f05c7a00640 1 -- 172.21.15.32:0/4100163172 >> 172.21.15.32:0/4100163172 conn(0x7f05c00ff380 msgr2=0x7f05c01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.586+0000 7f05c7a00640 1 -- 172.21.15.32:0/4100163172 shutdown_connections 2024-08-31T07:27:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.586+0000 7f05c7a00640 1 -- 172.21.15.32:0/4100163172 wait complete. 2024-08-31T07:27:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.587+0000 7f05c7a00640 1 Processor -- start 2024-08-31T07:27:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.587+0000 7f05c7a00640 1 -- start start 2024-08-31T07:27:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.587+0000 7f05c7a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c0075550 0x7f05c019c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.587+0000 7f05c7a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c0075ea0 0x7f05c019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.587+0000 7f05c7a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05c010fe70 0x7f05c01a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.587+0000 7f05c7a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0115a20 con 0x7f05c0075ea0 2024-08-31T07:27:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.587+0000 7f05c7a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01158a0 con 0x7f05c0075550 2024-08-31T07:27:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.587+0000 7f05c7a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0115ba0 con 0x7f05c010fe70 2024-08-31T07:27:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.588+0000 7f05bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c0075ea0 0x7f05c019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.588+0000 7f05c5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c0075550 0x7f05c019c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.588+0000 7f05bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c0075ea0 0x7f05c019ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34702/0 (socket says 172.21.15.32:34702) 2024-08-31T07:27:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.588+0000 7f05bfe00640 1 -- 172.21.15.32:0/2930141530 learned_addr learned my addr 172.21.15.32:0/2930141530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.588+0000 7f05c5a00640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05c010fe70 0x7f05c01a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:26.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.588+0000 7f05c5000640 1 -- 172.21.15.32:0/2930141530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05c010fe70 msgr2=0x7f05c01a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:26.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.588+0000 7f05c5000640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05c010fe70 0x7f05c01a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:26.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.588+0000 7f05c5000640 1 -- 172.21.15.32:0/2930141530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c0075ea0 msgr2=0x7f05c019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:26.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.588+0000 7f05c5000640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c0075ea0 0x7f05c019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:26.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.588+0000 7f05c5000640 1 -- 172.21.15.32:0/2930141530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c01a23e0 con 0x7f05c0075550 2024-08-31T07:27:26.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.589+0000 7f05c5000640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c0075550 0x7f05c019c470 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f05b402a930 tx=0x7f05b4031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.589+0000 7f05bd600640 1 -- 172.21.15.32:0/2930141530 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b403f030 con 0x7f05c0075550 2024-08-31T07:27:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.589+0000 7f05bd600640 1 -- 172.21.15.32:0/2930141530 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b4004490 con 0x7f05c0075550 2024-08-31T07:27:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.589+0000 7f05c7a00640 1 -- 172.21.15.32:0/2930141530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c01a2670 con 0x7f05c0075550 2024-08-31T07:27:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.589+0000 7f05bd600640 1 -- 172.21.15.32:0/2930141530 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4035070 con 0x7f05c0075550 2024-08-31T07:27:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.589+0000 7f05c7a00640 1 -- 172.21.15.32:0/2930141530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c01a8e30 con 0x7f05c0075550 2024-08-31T07:27:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.590+0000 7f05c7a00640 1 -- 172.21.15.32:0/2930141530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c0076a60 con 0x7f05c0075550 2024-08-31T07:27:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.593+0000 7f05bd600640 1 -- 172.21.15.32:0/2930141530 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f05b4042030 con 0x7f05c0075550 2024-08-31T07:27:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.594+0000 7f05bd600640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05a80b4ca0 0x7f05a80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.595+0000 7f05bd600640 1 -- 172.21.15.32:0/2930141530 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f05b4105a00 con 0x7f05c0075550 2024-08-31T07:27:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.595+0000 7f05bfe00640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05a80b4ca0 0x7f05a80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.596+0000 7f05bfe00640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05a80b4ca0 0x7f05a80b71b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f05c0100490 tx=0x7f05ac005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:26.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.596+0000 7f05bd600640 1 -- 172.21.15.32:0/2930141530 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f05b40cc190 con 0x7f05c0075550 2024-08-31T07:27:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:26 smithi073 ceph-mon[21995]: pgmap v826: 97 pgs: 97 active+clean; 683 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 4.3 MiB/s wr, 181 op/s 2024-08-31T07:27:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.835+0000 7f05c7a00640 1 -- 172.21.15.32:0/2930141530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05c01a90d0 con 0x7f05c0075550 2024-08-31T07:27:26.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.836+0000 7f05bd600640 1 -- 172.21.15.32:0/2930141530 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11979 (secure 0 0 0) 0x7f05b40d0950 con 0x7f05c0075550 2024-08-31T07:27:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:27:26.838 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":55,"btime":"2024-08-31T07:26:31:188885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:26:30.959002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:27:26.839 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 55 2024-08-31T07:27:26.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.838+0000 7f05c7a00640 1 -- 172.21.15.32:0/2930141530 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05a80b4ca0 msgr2=0x7f05a80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:26.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.838+0000 7f05c7a00640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05a80b4ca0 0x7f05a80b71b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f05c0100490 tx=0x7f05ac005f90 comp rx=0 tx=0).stop 2024-08-31T07:27:26.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.838+0000 7f05c7a00640 1 -- 172.21.15.32:0/2930141530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c0075550 msgr2=0x7f05c019c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:26.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.838+0000 7f05c7a00640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c0075550 0x7f05c019c470 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f05b402a930 tx=0x7f05b4031a20 comp rx=0 tx=0).stop 2024-08-31T07:27:26.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.838+0000 7f05c7a00640 1 -- 172.21.15.32:0/2930141530 shutdown_connections 2024-08-31T07:27:26.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.838+0000 7f05c7a00640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05a80b4ca0 0x7f05a80b71b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:26.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.838+0000 7f05c7a00640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05c010fe70 0x7f05c01a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:26.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.838+0000 7f05c7a00640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05c0075ea0 0x7f05c019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:26.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.838+0000 7f05c7a00640 1 --2- 172.21.15.32:0/2930141530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c0075550 0x7f05c019c470 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:26.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.839+0000 7f05c7a00640 1 -- 172.21.15.32:0/2930141530 >> 172.21.15.32:0/2930141530 conn(0x7f05c00ff380 msgr2=0x7f05c0100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:26.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.839+0000 7f05c7a00640 1 -- 172.21.15.32:0/2930141530 shutdown_connections 2024-08-31T07:27:26.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:26.839+0000 7f05c7a00640 1 -- 172.21.15.32:0/2930141530 wait complete. 2024-08-31T07:27:27.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23394 21.41 MB/sec warmup 67 sec latency 13717.431 ms 2024-08-31T07:27:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.436+0000 7fca28a00640 1 -- 172.21.15.32:0/3016449485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca2410bbe0 msgr2=0x7fca24112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.436+0000 7fca28a00640 1 --2- 172.21.15.32:0/3016449485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca2410bbe0 0x7fca24112410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009ca0 tx=0x7fca1c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.437+0000 7fca28a00640 1 -- 172.21.15.32:0/3016449485 shutdown_connections 2024-08-31T07:27:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.437+0000 7fca28a00640 1 --2- 172.21.15.32:0/3016449485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca2410bbe0 0x7fca24112410 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.437+0000 7fca28a00640 1 --2- 172.21.15.32:0/3016449485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca2410b290 0x7fca2410b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.437+0000 7fca28a00640 1 --2- 172.21.15.32:0/3016449485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca24105950 0x7fca24105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.437+0000 7fca28a00640 1 -- 172.21.15.32:0/3016449485 >> 172.21.15.32:0/3016449485 conn(0x7fca24101420 msgr2=0x7fca24103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.437+0000 7fca28a00640 1 -- 172.21.15.32:0/3016449485 shutdown_connections 2024-08-31T07:27:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.437+0000 7fca28a00640 1 -- 172.21.15.32:0/3016449485 wait complete. 2024-08-31T07:27:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.438+0000 7fca28a00640 1 Processor -- start 2024-08-31T07:27:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.438+0000 7fca28a00640 1 -- start start 2024-08-31T07:27:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.438+0000 7fca28a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca24105950 0x7fca2419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.438+0000 7fca28a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca2410b290 0x7fca2419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.438+0000 7fca28a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca2410bbe0 0x7fca241a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.438+0000 7fca28a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca241158f0 con 0x7fca2410b290 2024-08-31T07:27:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.438+0000 7fca28a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fca24115770 con 0x7fca24105950 2024-08-31T07:27:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.438+0000 7fca28a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fca24115a70 con 0x7fca2410bbe0 2024-08-31T07:27:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca22a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca2410b290 0x7fca2419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca23400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca24105950 0x7fca2419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca22a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca2410b290 0x7fca2419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34724/0 (socket says 172.21.15.32:34724) 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca22a00640 1 -- 172.21.15.32:0/1405898877 learned_addr learned my addr 172.21.15.32:0/1405898877 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca23e00640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca2410bbe0 0x7fca241a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca23400640 1 -- 172.21.15.32:0/1405898877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca2410bbe0 msgr2=0x7fca241a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca23400640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca2410bbe0 0x7fca241a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca23400640 1 -- 172.21.15.32:0/1405898877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca2410b290 msgr2=0x7fca2419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca23400640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca2410b290 0x7fca2419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca23400640 1 -- 172.21.15.32:0/1405898877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca241a21f0 con 0x7fca24105950 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.439+0000 7fca22a00640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca2410b290 0x7fca2419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.440+0000 7fca23400640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca24105950 0x7fca2419c280 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fca1400e9f0 tx=0x7fca1400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.440+0000 7fca1be00640 1 -- 172.21.15.32:0/1405898877 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1400ccc0 con 0x7fca24105950 2024-08-31T07:27:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.440+0000 7fca1be00640 1 -- 172.21.15.32:0/1405898877 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca14005020 con 0x7fca24105950 2024-08-31T07:27:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.440+0000 7fca28a00640 1 -- 172.21.15.32:0/1405898877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca241a24e0 con 0x7fca24105950 2024-08-31T07:27:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.440+0000 7fca1be00640 1 -- 172.21.15.32:0/1405898877 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14004a00 con 0x7fca24105950 2024-08-31T07:27:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.440+0000 7fca28a00640 1 -- 172.21.15.32:0/1405898877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca241a8c40 con 0x7fca24105950 2024-08-31T07:27:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.443+0000 7fca30cd4740 1 Processor -- start 2024-08-31T07:27:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.443+0000 7fca30cd4740 1 -- start start 2024-08-31T07:27:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.443+0000 7fca1be00640 1 -- 172.21.15.32:0/1405898877 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fca14004ba0 con 0x7fca24105950 2024-08-31T07:27:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f623104a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d361b0 0x55f623104fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f6231054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca30cd4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f62307b7b0 con 0x55f622d3e390 2024-08-31T07:27:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca30cd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f62306ad40 con 0x55f622d5f070 2024-08-31T07:27:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca30cd4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f62306ecf0 con 0x55f622d361b0 2024-08-31T07:27:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca23e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f623104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca23e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f623104a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50650/0 (socket says 172.21.15.32:50650) 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca23e00640 1 -- 172.21.15.32:0/985697793 learned_addr learned my addr 172.21.15.32:0/985697793 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca23400640 1 --2- 172.21.15.32:0/985697793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d361b0 0x55f623104fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca22a00640 1 --2- 172.21.15.32:0/985697793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f6231054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca23e00640 1 -- 172.21.15.32:0/985697793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d361b0 msgr2=0x55f623104fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca23e00640 1 --2- 172.21.15.32:0/985697793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d361b0 0x55f623104fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca23e00640 1 -- 172.21.15.32:0/985697793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 msgr2=0x55f6231054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca23e00640 1 --2- 172.21.15.32:0/985697793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f6231054f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca23e00640 1 -- 172.21.15.32:0/985697793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f622dca650 con 0x55f622d5f070 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.444+0000 7fca22a00640 1 --2- 172.21.15.32:0/985697793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f6231054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:27:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.445+0000 7fca23e00640 1 --2- 172.21.15.32:0/985697793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f623104a70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009ca0 tx=0x7fca1c009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cac43d2588073dd1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.445+0000 7fca0fe00640 1 -- 172.21.15.32:0/985697793 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c031cf0 con 0x55f622d5f070 2024-08-31T07:27:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.445+0000 7fca0fe00640 1 -- 172.21.15.32:0/985697793 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca1c036030 con 0x55f622d5f070 2024-08-31T07:27:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.445+0000 7fca0fe00640 1 -- 172.21.15.32:0/985697793 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0042a0 con 0x55f622d5f070 2024-08-31T07:27:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.445+0000 7fca1be00640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fca080b4ca0 0x7fca080b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca1be00640 1 -- 172.21.15.32:0/1405898877 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7fca140e54b0 con 0x7fca24105950 2024-08-31T07:27:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca23400640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fca080b4ca0 0x7fca080b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca30cd4740 1 -- 172.21.15.32:0/985697793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 msgr2=0x55f623104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca30cd4740 1 --2- 172.21.15.32:0/985697793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f623104a70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009ca0 tx=0x7fca1c009430 comp rx=0 tx=0).stop 2024-08-31T07:27:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca30cd4740 1 -- 172.21.15.32:0/985697793 shutdown_connections 2024-08-31T07:27:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca30cd4740 1 --2- 172.21.15.32:0/985697793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f6231054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca30cd4740 1 --2- 172.21.15.32:0/985697793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d361b0 0x55f623104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca30cd4740 1 --2- 172.21.15.32:0/985697793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f623104a70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca30cd4740 1 -- 172.21.15.32:0/985697793 >> 172.21.15.32:0/985697793 conn(0x55f622d53460 msgr2=0x55f623102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca30cd4740 1 -- 172.21.15.32:0/985697793 shutdown_connections 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.446+0000 7fca30cd4740 1 -- 172.21.15.32:0/985697793 wait complete. 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.447+0000 7fca23400640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fca080b4ca0 0x7fca080b71b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fca140e8fd0 tx=0x7fca140e8f60 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.447+0000 7fca30cd4740 1 -- start start 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.447+0000 7fca30cd4740 1 Processor -- start 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.448+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d361b0 0x55f6231106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.448+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 0x55f623110cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.448+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f62311c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.448+0000 7fca30cd4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f622ecd700 con 0x55f622d361b0 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.448+0000 7fca30cd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f622ff1cf0 con 0x55f622d5f070 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.448+0000 7fca30cd4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f623113070 con 0x55f622d3e390 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.448+0000 7fca30cd4740 10 client.0 _ll_register_callbacks cb 0x55f623017470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.448+0000 7fca23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d361b0 0x55f6231106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.448+0000 7fca23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d361b0 0x55f6231106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34734/0 (socket says 172.21.15.32:34734) 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.448+0000 7fca23e00640 1 -- 172.21.15.32:0/3224619614 learned_addr learned my addr 172.21.15.32:0/3224619614 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.449+0000 7fca22a00640 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 0x55f623110cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.449+0000 7fca23e00640 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f62311c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.449+0000 7fca23e00640 1 -- 172.21.15.32:0/3224619614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 msgr2=0x55f623110cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.449+0000 7fca23e00640 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 0x55f623110cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.449+0000 7fca23e00640 1 -- 172.21.15.32:0/3224619614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 msgr2=0x55f62311c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.449+0000 7fca23e00640 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f62311c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.449+0000 7fca23e00640 1 -- 172.21.15.32:0/3224619614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6230328b0 con 0x55f622d361b0 2024-08-31T07:27:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.449+0000 7fca23e00640 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d361b0 0x55f6231106d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009620 tx=0x7fca1c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906642509180bc6e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.449+0000 7fca0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:27:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.449+0000 7fca0cc00640 1 -- 172.21.15.32:0/3224619614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c058060 con 0x55f622d361b0 2024-08-31T07:27:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.450+0000 7fca30cd4740 1 -- 172.21.15.32:0/3224619614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca1c036030 con 0x55f622d361b0 2024-08-31T07:27:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.450+0000 7fca0cc00640 1 -- 172.21.15.32:0/3224619614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7fca1c02f610 con 0x55f622d361b0 2024-08-31T07:27:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.450+0000 7fca0cc00640 1 -- 172.21.15.32:0/3224619614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fca1c0318e0 con 0x55f622d361b0 2024-08-31T07:27:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.450+0000 7fca30cd4740 10 client.18474 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.450+0000 7fca30cd4740 1 -- 172.21.15.32:0/3224619614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f62311f130 con 0x55f622d361b0 2024-08-31T07:27:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.450+0000 7fca0cc00640 1 -- 172.21.15.32:0/3224619614 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca1c036030 con 0x55f622d361b0 2024-08-31T07:27:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.450+0000 7fca0cc00640 1 -- 172.21.15.32:0/3224619614 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7fca1c043070 con 0x55f622d361b0 2024-08-31T07:27:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.450+0000 7fca30cd4740 10 client.18474 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.450+0000 7fca30cd4740 10 client.18474 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:27:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.451+0000 7fca30cd4740 20 client.18474 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.451+0000 7fca30cd4740 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f622d1d9f0 0x55f623122490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.451+0000 7fca30cd4740 4 client.18474 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:27:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.451+0000 7fca30cd4740 1 -- 172.21.15.32:0/3224619614 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f62307d700 con 0x55f622d1d9f0 2024-08-31T07:27:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.451+0000 7fca22a00640 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f622d1d9f0 0x55f623122490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.451+0000 7fca22a00640 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f622d1d9f0 0x55f623122490 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d2ae5d562041661 server_cookie=645128610720fe2f in_seq=0 out_seq=0 2024-08-31T07:27:27.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.451+0000 7fca0cc00640 10 client.18474 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:27:27.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.452+0000 7fca0cc00640 1 -- 172.21.15.32:0/3224619614 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f62307d700 con 0x55f622d1d9f0 2024-08-31T07:27:27.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.452+0000 7fca0cc00640 10 client.18474 handle_command_reply: tid=0 2024-08-31T07:27:27.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.452+0000 7fca30cd4740 1 client.18474 shutdown 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.452+0000 7fca30cd4740 1 -- 172.21.15.32:0/3224619614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d361b0 msgr2=0x55f6231106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.452+0000 7fca30cd4740 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d361b0 0x55f6231106d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009620 tx=0x7fca1c003d80 comp rx=0 tx=0).stop 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.453+0000 7fca30cd4740 1 -- 172.21.15.32:0/3224619614 shutdown_connections 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.453+0000 7fca30cd4740 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f622d1d9f0 0x55f623122490 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.453+0000 7fca30cd4740 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f62311c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.453+0000 7fca30cd4740 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 0x55f623110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.453+0000 7fca30cd4740 1 --2- 172.21.15.32:0/3224619614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d361b0 0x55f6231106d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.453+0000 7fca30cd4740 1 -- 172.21.15.32:0/3224619614 >> 172.21.15.32:0/3224619614 conn(0x55f622d53460 msgr2=0x55f623107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.453+0000 7fca0cc00640 0 client.18474 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.453+0000 7fca30cd4740 1 -- 172.21.15.32:0/3224619614 shutdown_connections 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.453+0000 7fca30cd4740 1 -- 172.21.15.32:0/3224619614 wait complete. 2024-08-31T07:27:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.453+0000 7fca30cd4740 20 client.18474 trim_cache size 0 max 16384 2024-08-31T07:27:27.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.467+0000 7fca30cd4740 1 Processor -- start 2024-08-31T07:27:27.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.467+0000 7fca30cd4740 1 -- start start 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.467+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d361b0 0x55f623109b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 0x55f62310a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d5f070 0x55f62311c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca30cd4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca241b12e0 con 0x55f622d5f070 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca30cd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fca241b3000 con 0x55f622d361b0 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca30cd4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fca241b19e0 con 0x55f622d3e390 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca22a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 0x55f62310a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca22a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 0x55f62310a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:39176/0 (socket says 172.21.15.32:39176) 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca22a00640 1 -- 172.21.15.32:0/270749832 learned_addr learned my addr 172.21.15.32:0/270749832 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca23e00640 1 --2- 172.21.15.32:0/270749832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d361b0 0x55f623109b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca23e00640 1 --2- 172.21.15.32:0/270749832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d5f070 0x55f62311c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca23e00640 1 -- 172.21.15.32:0/270749832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 msgr2=0x55f62310a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca23e00640 1 --2- 172.21.15.32:0/270749832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 0x55f62310a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca23e00640 1 -- 172.21.15.32:0/270749832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d361b0 msgr2=0x55f623109b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca23e00640 1 --2- 172.21.15.32:0/270749832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d361b0 0x55f623109b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca23e00640 1 -- 172.21.15.32:0/270749832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f623106d60 con 0x55f622d5f070 2024-08-31T07:27:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.468+0000 7fca22a00640 1 --2- 172.21.15.32:0/270749832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 0x55f62310a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:27.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.469+0000 7fca23e00640 1 --2- 172.21.15.32:0/270749832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d5f070 0x55f62311c260 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fca1c031fd0 tx=0x7fca1c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b1e4f9af999a83 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:27.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.469+0000 7fca0f400640 1 -- 172.21.15.32:0/270749832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c058070 con 0x55f622d5f070 2024-08-31T07:27:27.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.469+0000 7fca0f400640 1 -- 172.21.15.32:0/270749832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca1c036040 con 0x55f622d5f070 2024-08-31T07:27:27.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.469+0000 7fca0f400640 1 -- 172.21.15.32:0/270749832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c02f870 con 0x55f622d5f070 2024-08-31T07:27:27.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.470+0000 7fca30cd4740 1 -- 172.21.15.32:0/270749832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d5f070 msgr2=0x55f62311c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:27.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.470+0000 7fca30cd4740 1 --2- 172.21.15.32:0/270749832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d5f070 0x55f62311c260 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fca1c031fd0 tx=0x7fca1c000f40 comp rx=0 tx=0).stop 2024-08-31T07:27:27.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.470+0000 7fca30cd4740 1 -- 172.21.15.32:0/270749832 shutdown_connections 2024-08-31T07:27:27.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.470+0000 7fca30cd4740 1 --2- 172.21.15.32:0/270749832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d5f070 0x55f62311c260 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.470+0000 7fca30cd4740 1 --2- 172.21.15.32:0/270749832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d3e390 0x55f62310a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.470+0000 7fca30cd4740 1 --2- 172.21.15.32:0/270749832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d361b0 0x55f623109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.470+0000 7fca30cd4740 1 -- 172.21.15.32:0/270749832 >> 172.21.15.32:0/270749832 conn(0x55f622d53460 msgr2=0x55f62310ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.470+0000 7fca30cd4740 1 -- 172.21.15.32:0/270749832 shutdown_connections 2024-08-31T07:27:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.470+0000 7fca30cd4740 1 -- 172.21.15.32:0/270749832 wait complete. 2024-08-31T07:27:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.470+0000 7fca30cd4740 1 -- start start 2024-08-31T07:27:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.471+0000 7fca30cd4740 1 Processor -- start 2024-08-31T07:27:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.471+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d361b0 0x55f6231149a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.471+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f623114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.471+0000 7fca30cd4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f62318bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.471+0000 7fca30cd4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca241b2870 con 0x55f622d3e390 2024-08-31T07:27:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.471+0000 7fca30cd4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fca241b2140 con 0x55f622d5f070 2024-08-31T07:27:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.471+0000 7fca30cd4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fca241af2e0 con 0x55f622d361b0 2024-08-31T07:27:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.471+0000 7fca30cd4740 10 client.0 _ll_register_callbacks cb 0x7fca1c031ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.471+0000 7fca22a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f623114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca22a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f623114ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34758/0 (socket says 172.21.15.32:34758) 2024-08-31T07:27:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca23e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d361b0 0x55f6231149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca22a00640 1 -- 172.21.15.32:0/2667525212 learned_addr learned my addr 172.21.15.32:0/2667525212 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:27.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca23e00640 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f62318bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca22a00640 1 -- 172.21.15.32:0/2667525212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d361b0 msgr2=0x55f6231149a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:27.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca22a00640 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d361b0 0x55f6231149a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca22a00640 1 -- 172.21.15.32:0/2667525212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 msgr2=0x55f62318bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:27.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca22a00640 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f62318bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca22a00640 1 -- 172.21.15.32:0/2667525212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62318c1d0 con 0x55f622d3e390 2024-08-31T07:27:27.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca22a00640 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f623114ee0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x55f623122a30 tx=0x7fca10012c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2450c241d2f5694 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:27.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.472+0000 7fca0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.473+0000 7fca0cc00640 1 -- 172.21.15.32:0/2667525212 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10028030 con 0x55f622d3e390 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.473+0000 7fca0cc00640 1 -- 172.21.15.32:0/2667525212 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7fca10010a10 con 0x55f622d3e390 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.473+0000 7fca30cd4740 1 -- 172.21.15.32:0/2667525212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca1c036040 con 0x55f622d3e390 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.473+0000 7fca0cc00640 1 -- 172.21.15.32:0/2667525212 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fca10032050 con 0x55f622d3e390 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.473+0000 7fca30cd4740 10 client.18480 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.473+0000 7fca30cd4740 1 -- 172.21.15.32:0/2667525212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55f62318d480 con 0x55f622d3e390 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.473+0000 7fca0cc00640 1 -- 172.21.15.32:0/2667525212 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca1c036040 con 0x55f622d3e390 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.474+0000 7fca0cc00640 1 -- 172.21.15.32:0/2667525212 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7fca1002d400 con 0x55f622d3e390 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.474+0000 7fca30cd4740 10 client.18480 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.474+0000 7fca30cd4740 10 client.18480 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.474+0000 7fca30cd4740 20 client.18480 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.474+0000 7fca30cd4740 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f622cf2da0 0x55f62318fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.474+0000 7fca30cd4740 4 client.18480 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:27:27.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.474+0000 7fca30cd4740 1 -- 172.21.15.32:0/2667525212 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6230791a0 con 0x55f622cf2da0 2024-08-31T07:27:27.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.474+0000 7fca23e00640 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f622cf2da0 0x55f62318fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:27.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca23e00640 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f622cf2da0 0x55f62318fd30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=defe8b5136e26f49 server_cookie=8adb681ce9b5495c in_seq=0 out_seq=0 2024-08-31T07:27:27.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca0cc00640 10 client.18480 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:27:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca0cc00640 1 -- 172.21.15.32:0/2667525212 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6230791a0 con 0x55f622cf2da0 2024-08-31T07:27:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca0cc00640 10 client.18480 handle_command_reply: tid=0 2024-08-31T07:27:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca30cd4740 1 client.18480 shutdown 2024-08-31T07:27:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca30cd4740 1 -- 172.21.15.32:0/2667525212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 msgr2=0x55f623114ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca30cd4740 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f623114ee0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x55f623122a30 tx=0x7fca10012c20 comp rx=0 tx=0).stop 2024-08-31T07:27:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca30cd4740 1 -- 172.21.15.32:0/2667525212 shutdown_connections 2024-08-31T07:27:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca30cd4740 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f622cf2da0 0x55f62318fd30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca30cd4740 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f622d5f070 0x55f62318bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca30cd4740 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f622d3e390 0x55f623114ee0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca30cd4740 1 --2- 172.21.15.32:0/2667525212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f622d361b0 0x55f6231149a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca30cd4740 1 -- 172.21.15.32:0/2667525212 >> 172.21.15.32:0/2667525212 conn(0x55f622d53460 msgr2=0x55f623109b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.475+0000 7fca0cc00640 0 client.18480 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:27:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.476+0000 7fca30cd4740 1 -- 172.21.15.32:0/2667525212 shutdown_connections 2024-08-31T07:27:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.476+0000 7fca30cd4740 1 -- 172.21.15.32:0/2667525212 wait complete. 2024-08-31T07:27:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.476+0000 7fca30cd4740 20 client.18480 trim_cache size 0 max 16384 2024-08-31T07:27:27.482 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:27:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.476+0000 7fca28a00640 1 -- 172.21.15.32:0/1405898877 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fca080b4ca0 msgr2=0x7fca080b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.476+0000 7fca28a00640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fca080b4ca0 0x7fca080b71b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fca140e8fd0 tx=0x7fca140e8f60 comp rx=0 tx=0).stop 2024-08-31T07:27:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.476+0000 7fca28a00640 1 -- 172.21.15.32:0/1405898877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca24105950 msgr2=0x7fca2419c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.476+0000 7fca28a00640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca24105950 0x7fca2419c280 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fca1400e9f0 tx=0x7fca1400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.476+0000 7fca28a00640 1 -- 172.21.15.32:0/1405898877 shutdown_connections 2024-08-31T07:27:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.477+0000 7fca28a00640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fca080b4ca0 0x7fca080b71b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.477+0000 7fca28a00640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca2410bbe0 0x7fca241a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.477+0000 7fca28a00640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca2410b290 0x7fca2419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.477+0000 7fca28a00640 1 --2- 172.21.15.32:0/1405898877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca24105950 0x7fca2419c280 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.477+0000 7fca28a00640 1 -- 172.21.15.32:0/1405898877 >> 172.21.15.32:0/1405898877 conn(0x7fca24101420 msgr2=0x7fca241088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.477+0000 7fca28a00640 1 -- 172.21.15.32:0/1405898877 shutdown_connections 2024-08-31T07:27:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:27.477+0000 7fca28a00640 1 -- 172.21.15.32:0/1405898877 wait complete. 2024-08-31T07:27:27.499 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:27 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2930141530' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:27:27.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:27 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2930141530' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:27:27.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:27 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2930141530' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:27:28.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23394 21.10 MB/sec warmup 68 sec latency 14717.549 ms 2024-08-31T07:27:28.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:27:28.580 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:28 smithi073 ceph-mon[21995]: pgmap v827: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.9 MiB/s wr, 175 op/s 2024-08-31T07:27:28.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:28 smithi032 ceph-mon[17252]: pgmap v827: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.9 MiB/s wr, 175 op/s 2024-08-31T07:27:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:28 smithi084 ceph-mon[22387]: pgmap v827: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.9 MiB/s wr, 175 op/s 2024-08-31T07:27:28.793 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id f9a618dc --release --await' 2024-08-31T07:27:28.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id f9a618dc --release --await 2024-08-31T07:27:28.894 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:29.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23394 20.79 MB/sec warmup 69 sec latency 15717.700 ms 2024-08-31T07:27:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.616+0000 7f1461200640 1 -- 172.21.15.32:0/111836729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c10bbe0 msgr2=0x7f145c112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.616+0000 7f1461200640 1 --2- 172.21.15.32:0/111836729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c10bbe0 0x7f145c112410 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1454009ca0 tx=0x7f145402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.616+0000 7f1461200640 1 -- 172.21.15.32:0/111836729 shutdown_connections 2024-08-31T07:27:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.616+0000 7f1461200640 1 --2- 172.21.15.32:0/111836729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c10bbe0 0x7f145c112410 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.616+0000 7f1461200640 1 --2- 172.21.15.32:0/111836729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145c10b290 0x7f145c10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.616+0000 7f1461200640 1 --2- 172.21.15.32:0/111836729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f145c105950 0x7f145c105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.616+0000 7f1461200640 1 -- 172.21.15.32:0/111836729 >> 172.21.15.32:0/111836729 conn(0x7f145c101420 msgr2=0x7f145c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.616+0000 7f1461200640 1 -- 172.21.15.32:0/111836729 shutdown_connections 2024-08-31T07:27:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.616+0000 7f1461200640 1 -- 172.21.15.32:0/111836729 wait complete. 2024-08-31T07:27:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.618+0000 7f1461200640 1 Processor -- start 2024-08-31T07:27:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.618+0000 7f1461200640 1 -- start start 2024-08-31T07:27:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.618+0000 7f1461200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c105950 0x7f145c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.618+0000 7f1461200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f145c10b290 0x7f145c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.618+0000 7f1461200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145c10bbe0 0x7f145c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.618+0000 7f1461200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1158f0 con 0x7f145c105950 2024-08-31T07:27:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.618+0000 7f1461200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f145c115770 con 0x7f145c10b290 2024-08-31T07:27:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.618+0000 7f1461200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f145c115a70 con 0x7f145c10bbe0 2024-08-31T07:27:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.619+0000 7f145b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c105950 0x7f145c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.619+0000 7f145b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c105950 0x7f145c19c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34774/0 (socket says 172.21.15.32:34774) 2024-08-31T07:27:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.619+0000 7f145b400640 1 -- 172.21.15.32:0/1774711471 learned_addr learned my addr 172.21.15.32:0/1774711471 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.619+0000 7f145be00640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145c10bbe0 0x7f145c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.619+0000 7f145aa00640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f145c10b290 0x7f145c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.619+0000 7f145b400640 1 -- 172.21.15.32:0/1774711471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145c10bbe0 msgr2=0x7f145c1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.619+0000 7f145b400640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145c10bbe0 0x7f145c1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.619+0000 7f145b400640 1 -- 172.21.15.32:0/1774711471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f145c10b290 msgr2=0x7f145c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.619+0000 7f145b400640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f145c10b290 0x7f145c19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.619+0000 7f145b400640 1 -- 172.21.15.32:0/1774711471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c1a2190 con 0x7f145c105950 2024-08-31T07:27:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.620+0000 7f145b400640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c105950 0x7f145c19c220 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f144c00ea60 tx=0x7f144c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:29.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.620+0000 7f1453e00640 1 -- 172.21.15.32:0/1774711471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c00cde0 con 0x7f145c105950 2024-08-31T07:27:29.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.620+0000 7f1453e00640 1 -- 172.21.15.32:0/1774711471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f144c0035f0 con 0x7f145c105950 2024-08-31T07:27:29.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.620+0000 7f1461200640 1 -- 172.21.15.32:0/1774711471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145c1a2480 con 0x7f145c105950 2024-08-31T07:27:29.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.620+0000 7f1453e00640 1 -- 172.21.15.32:0/1774711471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c010690 con 0x7f145c105950 2024-08-31T07:27:29.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.620+0000 7f1461200640 1 -- 172.21.15.32:0/1774711471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145c1a8be0 con 0x7f145c105950 2024-08-31T07:27:29.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.624+0000 7f146942b740 1 Processor -- start 2024-08-31T07:27:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.624+0000 7f146942b740 1 -- start start 2024-08-31T07:27:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.624+0000 7f1453e00640 1 -- 172.21.15.32:0/1774711471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f144c0037a0 con 0x7f145c105950 2024-08-31T07:27:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.624+0000 7f146942b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 0x56078577da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.624+0000 7f146942b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853af1b0 0x56078577dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.624+0000 7f146942b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853b7390 0x56078577e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.624+0000 7f146942b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5607856f47b0 con 0x5607853b7390 2024-08-31T07:27:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.624+0000 7f146942b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5607856e3d40 con 0x5607853d8070 2024-08-31T07:27:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.624+0000 7f146942b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5607856e7cf0 con 0x5607853af1b0 2024-08-31T07:27:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.625+0000 7f145be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 0x56078577da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.625+0000 7f145be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 0x56078577da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50694/0 (socket says 172.21.15.32:50694) 2024-08-31T07:27:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.625+0000 7f145be00640 1 -- 172.21.15.32:0/2007448294 learned_addr learned my addr 172.21.15.32:0/2007448294 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.625+0000 7f145aa00640 1 --2- 172.21.15.32:0/2007448294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853b7390 0x56078577e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.625+0000 7f145b400640 1 --2- 172.21.15.32:0/2007448294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853af1b0 0x56078577dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.625+0000 7f145aa00640 1 -- 172.21.15.32:0/2007448294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853af1b0 msgr2=0x56078577dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.625+0000 7f145aa00640 1 --2- 172.21.15.32:0/2007448294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853af1b0 0x56078577dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.625+0000 7f145aa00640 1 -- 172.21.15.32:0/2007448294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 msgr2=0x56078577da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.625+0000 7f145aa00640 1 --2- 172.21.15.32:0/2007448294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 0x56078577da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.625+0000 7f145aa00640 1 -- 172.21.15.32:0/2007448294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560785443650 con 0x5607853b7390 2024-08-31T07:27:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.626+0000 7f145aa00640 1 --2- 172.21.15.32:0/2007448294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853b7390 0x56078577e4f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1448005d00 tx=0x7f14480107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cf3d0228216489 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.626+0000 7f1450c00640 1 -- 172.21.15.32:0/2007448294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144801c480 con 0x5607853b7390 2024-08-31T07:27:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.626+0000 7f1450c00640 1 -- 172.21.15.32:0/2007448294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f144801cad0 con 0x5607853b7390 2024-08-31T07:27:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.626+0000 7f1450c00640 1 -- 172.21.15.32:0/2007448294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448011ae0 con 0x5607853b7390 2024-08-31T07:27:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.627+0000 7f1453e00640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14400b4ca0 0x7f14400b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.627+0000 7f1453e00640 1 -- 172.21.15.32:0/1774711471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f144c0e5830 con 0x7f145c105950 2024-08-31T07:27:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.627+0000 7f145be00640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14400b4ca0 0x7f14400b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.627+0000 7f146942b740 1 -- 172.21.15.32:0/2007448294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853b7390 msgr2=0x56078577e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.628+0000 7f146942b740 1 --2- 172.21.15.32:0/2007448294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853b7390 0x56078577e4f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1448005d00 tx=0x7f14480107a0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.628+0000 7f145be00640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14400b4ca0 0x7f14400b71b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1454009a60 tx=0x7f1454005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.628+0000 7f146942b740 1 -- 172.21.15.32:0/2007448294 shutdown_connections 2024-08-31T07:27:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.628+0000 7f146942b740 1 --2- 172.21.15.32:0/2007448294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853b7390 0x56078577e4f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.628+0000 7f146942b740 1 --2- 172.21.15.32:0/2007448294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853af1b0 0x56078577dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.628+0000 7f146942b740 1 --2- 172.21.15.32:0/2007448294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 0x56078577da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.628+0000 7f146942b740 1 -- 172.21.15.32:0/2007448294 >> 172.21.15.32:0/2007448294 conn(0x5607853cc460 msgr2=0x56078577b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:29.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.628+0000 7f146942b740 1 -- 172.21.15.32:0/2007448294 shutdown_connections 2024-08-31T07:27:29.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.628+0000 7f146942b740 1 -- 172.21.15.32:0/2007448294 wait complete. 2024-08-31T07:27:29.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.629+0000 7f146942b740 1 -- start start 2024-08-31T07:27:29.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.629+0000 7f146942b740 1 Processor -- start 2024-08-31T07:27:29.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f146942b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 0x560785789660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f146942b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x560785789be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f146942b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x560785795540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f146942b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560785546700 con 0x5607853d8070 2024-08-31T07:27:29.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f146942b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56078566acf0 con 0x5607853af1b0 2024-08-31T07:27:29.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f146942b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56078578c250 con 0x5607853b7390 2024-08-31T07:27:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f146942b740 10 client.0 _ll_register_callbacks cb 0x560785690470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f145be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x560785795540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f145b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 0x560785789660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f145be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x560785795540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34786/0 (socket says 172.21.15.32:34786) 2024-08-31T07:27:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f145be00640 1 -- 172.21.15.32:0/3985698887 learned_addr learned my addr 172.21.15.32:0/3985698887 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f145aa00640 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x560785789be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f145b400640 1 -- 172.21.15.32:0/3985698887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 msgr2=0x560785789be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f145b400640 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x560785789be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f145b400640 1 -- 172.21.15.32:0/3985698887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 msgr2=0x560785795540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f145b400640 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x560785795540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.630+0000 7f145b400640 1 -- 172.21.15.32:0/3985698887 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607856ab8b0 con 0x5607853af1b0 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.631+0000 7f145b400640 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 0x560785789660 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f144c0e2fd0 tx=0x7f144c003d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86bf9258fb462e7d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.631+0000 7f1445600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.631+0000 7f1445600640 1 -- 172.21.15.32:0/3985698887 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c0ab8c0 con 0x5607853af1b0 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.631+0000 7f1445600640 1 -- 172.21.15.32:0/3985698887 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f144c0a8200 con 0x5607853af1b0 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.631+0000 7f146942b740 1 -- 172.21.15.32:0/3985698887 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f144801cad0 con 0x5607853af1b0 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.631+0000 7f1445600640 1 -- 172.21.15.32:0/3985698887 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f144c0ab6e0 con 0x5607853af1b0 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.631+0000 7f146942b740 10 client.28511 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.631+0000 7f146942b740 1 -- 172.21.15.32:0/3985698887 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5607857a9320 con 0x5607853af1b0 2024-08-31T07:27:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.632+0000 7f1445600640 1 -- 172.21.15.32:0/3985698887 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f144801cad0 con 0x5607853af1b0 2024-08-31T07:27:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.632+0000 7f1445600640 1 -- 172.21.15.32:0/3985698887 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f144c0af060 con 0x5607853af1b0 2024-08-31T07:27:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.632+0000 7f146942b740 10 client.28511 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.632+0000 7f146942b740 10 client.28511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:27:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.632+0000 7f146942b740 20 client.28511 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.632+0000 7f146942b740 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5607853969f0 0x5607857ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.632+0000 7f146942b740 4 client.28511 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:27:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.632+0000 7f146942b740 1 -- 172.21.15.32:0/3985698887 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607856f6700 con 0x5607853969f0 2024-08-31T07:27:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.633+0000 7f145aa00640 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5607853969f0 0x5607857ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.634+0000 7f1445600640 10 client.28511 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.634+0000 7f145aa00640 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5607853969f0 0x5607857ac880 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=431ca4638e67bab3 server_cookie=173a948a30f22904 in_seq=0 out_seq=0 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.634+0000 7f1445600640 1 -- 172.21.15.32:0/3985698887 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5607856f6700 con 0x5607853969f0 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.634+0000 7f1445600640 10 client.28511 handle_command_reply: tid=0 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 client.28511 shutdown 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 -- 172.21.15.32:0/3985698887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 msgr2=0x560785789660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 0x560785789660 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f144c0e2fd0 tx=0x7f144c003d90 comp rx=0 tx=0).stop 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 -- 172.21.15.32:0/3985698887 shutdown_connections 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x560785795540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x560785789be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5607853969f0 0x5607857ac880 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 --2- 172.21.15.32:0/3985698887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 0x560785789660 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 -- 172.21.15.32:0/3985698887 >> 172.21.15.32:0/3985698887 conn(0x5607853cc460 msgr2=0x560785780c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f1445600640 0 client.28511 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:27:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 -- 172.21.15.32:0/3985698887 shutdown_connections 2024-08-31T07:27:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 1 -- 172.21.15.32:0/3985698887 wait complete. 2024-08-31T07:27:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.635+0000 7f146942b740 20 client.28511 trim_cache size 0 max 16384 2024-08-31T07:27:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.650+0000 7f146942b740 1 Processor -- start 2024-08-31T07:27:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.650+0000 7f146942b740 1 -- start start 2024-08-31T07:27:29.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.650+0000 7f146942b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 0x560785781530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.650+0000 7f146942b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078577f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.650+0000 7f146942b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x56078577af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.650+0000 7f146942b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f145c075160 con 0x5607853d8070 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.650+0000 7f146942b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1aa900 con 0x5607853af1b0 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.650+0000 7f146942b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1a92e0 con 0x5607853b7390 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x56078577af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x56078577af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34788/0 (socket says 172.21.15.32:34788) 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145be00640 1 -- 172.21.15.32:0/751491506 learned_addr learned my addr 172.21.15.32:0/751491506 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145aa00640 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078577f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145b400640 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 0x560785781530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145b400640 1 -- 172.21.15.32:0/751491506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 msgr2=0x56078577f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145b400640 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078577f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145b400640 1 -- 172.21.15.32:0/751491506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 msgr2=0x56078577af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145b400640 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x56078577af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145b400640 1 -- 172.21.15.32:0/751491506 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56078577e880 con 0x5607853af1b0 2024-08-31T07:27:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145aa00640 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078577f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.651+0000 7f145be00640 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x56078577af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.652+0000 7f145b400640 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 0x560785781530 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f144c01f000 tx=0x7f144c003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=970cbddf2edd382 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.652+0000 7f1447e00640 1 -- 172.21.15.32:0/751491506 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c0ab8c0 con 0x5607853af1b0 2024-08-31T07:27:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.652+0000 7f1447e00640 1 -- 172.21.15.32:0/751491506 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f144c0aba80 con 0x5607853af1b0 2024-08-31T07:27:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.652+0000 7f1447e00640 1 -- 172.21.15.32:0/751491506 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c0abf70 con 0x5607853af1b0 2024-08-31T07:27:29.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.653+0000 7f146942b740 1 -- 172.21.15.32:0/751491506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 msgr2=0x560785781530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.653+0000 7f146942b740 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 0x560785781530 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f144c01f000 tx=0x7f144c003e90 comp rx=0 tx=0).stop 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.653+0000 7f146942b740 1 -- 172.21.15.32:0/751491506 shutdown_connections 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.653+0000 7f146942b740 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853d8070 0x56078577af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.653+0000 7f146942b740 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078577f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.653+0000 7f146942b740 1 --2- 172.21.15.32:0/751491506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853af1b0 0x560785781530 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.653+0000 7f146942b740 1 -- 172.21.15.32:0/751491506 >> 172.21.15.32:0/751491506 conn(0x5607853cc460 msgr2=0x56078577eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.653+0000 7f146942b740 1 -- 172.21.15.32:0/751491506 shutdown_connections 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.653+0000 7f146942b740 1 -- 172.21.15.32:0/751491506 wait complete. 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.654+0000 7f146942b740 1 -- start start 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.654+0000 7f146942b740 1 Processor -- start 2024-08-31T07:27:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.654+0000 7f146942b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853af1b0 0x56078578d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f146942b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078578dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f146942b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 0x560785816240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f146942b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1aa170 con 0x5607853af1b0 2024-08-31T07:27:29.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f146942b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1a9a40 con 0x5607853d8070 2024-08-31T07:27:29.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f146942b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f145c073160 con 0x5607853b7390 2024-08-31T07:27:29.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f146942b740 10 client.0 _ll_register_callbacks cb 0x7f144c0eb240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:29.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f145aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078578dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f145be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 0x560785816240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f145aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078578dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:39226/0 (socket says 172.21.15.32:39226) 2024-08-31T07:27:29.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f145aa00640 1 -- 172.21.15.32:0/564054237 learned_addr learned my addr 172.21.15.32:0/564054237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:29.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f145b400640 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853af1b0 0x56078578d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f145aa00640 1 -- 172.21.15.32:0/564054237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 msgr2=0x560785816240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f145aa00640 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 0x560785816240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f145aa00640 1 -- 172.21.15.32:0/564054237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853af1b0 msgr2=0x56078578d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.655+0000 7f145aa00640 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853af1b0 0x56078578d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.656+0000 7f145aa00640 1 -- 172.21.15.32:0/564054237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56078578e450 con 0x5607853b7390 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.656+0000 7f145aa00640 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078578dc40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1448011710 tx=0x7f1448005c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eedd0f282aaba0b6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.656+0000 7f1445600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.656+0000 7f145b400640 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853af1b0 0x56078578d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.656+0000 7f1445600640 1 -- 172.21.15.32:0/564054237 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448005450 con 0x5607853b7390 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.656+0000 7f146942b740 1 -- 172.21.15.32:0/564054237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f144c0aba80 con 0x5607853b7390 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.656+0000 7f1445600640 1 -- 172.21.15.32:0/564054237 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f14480055f0 con 0x5607853b7390 2024-08-31T07:27:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f1445600640 1 -- 172.21.15.32:0/564054237 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f144800b710 con 0x5607853b7390 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f146942b740 10 client.28507 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f146942b740 1 -- 172.21.15.32:0/564054237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5607858177c0 con 0x5607853b7390 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f1445600640 1 -- 172.21.15.32:0/564054237 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f144c0aba80 con 0x5607853b7390 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f1445600640 1 -- 172.21.15.32:0/564054237 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f1448004ab0 con 0x5607853b7390 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f146942b740 10 client.28507 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f146942b740 10 client.28507 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f146942b740 20 client.28507 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f146942b740 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56078536bda0 0x560785819d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f146942b740 4 client.28507 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.657+0000 7f146942b740 1 -- 172.21.15.32:0/564054237 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607856f21a0 con 0x56078536bda0 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.658+0000 7f145b400640 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56078536bda0 0x560785819d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.658+0000 7f145b400640 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56078536bda0 0x560785819d00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cc5f282dc51e481 server_cookie=2d5f9258858448ec in_seq=0 out_seq=0 2024-08-31T07:27:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.658+0000 7f1445600640 10 client.28507 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:27:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f1445600640 1 -- 172.21.15.32:0/564054237 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607856f21a0 con 0x56078536bda0 2024-08-31T07:27:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f1445600640 10 client.28507 handle_command_reply: tid=0 2024-08-31T07:27:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 client.28507 shutdown 2024-08-31T07:27:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 -- 172.21.15.32:0/564054237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 msgr2=0x56078578dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078578dc40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1448011710 tx=0x7f1448005c60 comp rx=0 tx=0).stop 2024-08-31T07:27:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 -- 172.21.15.32:0/564054237 shutdown_connections 2024-08-31T07:27:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56078536bda0 0x560785819d00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607853d8070 0x560785816240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607853b7390 0x56078578dc40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 --2- 172.21.15.32:0/564054237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5607853af1b0 0x56078578d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 -- 172.21.15.32:0/564054237 >> 172.21.15.32:0/564054237 conn(0x5607853cc460 msgr2=0x560785781300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:29.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f1445600640 0 client.28507 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:27:29.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 -- 172.21.15.32:0/564054237 shutdown_connections 2024-08-31T07:27:29.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 1 -- 172.21.15.32:0/564054237 wait complete. 2024-08-31T07:27:29.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.659+0000 7f146942b740 20 client.28507 trim_cache size 0 max 16384 2024-08-31T07:27:29.666 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:27:29.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.660+0000 7f1461200640 1 -- 172.21.15.32:0/1774711471 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14400b4ca0 msgr2=0x7f14400b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:29.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.660+0000 7f1461200640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14400b4ca0 0x7f14400b71b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1454009a60 tx=0x7f1454005d80 comp rx=0 tx=0).stop 2024-08-31T07:27:29.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.660+0000 7f1461200640 1 -- 172.21.15.32:0/1774711471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c105950 msgr2=0x7f145c19c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:29.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.660+0000 7f1461200640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c105950 0x7f145c19c220 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f144c00ea60 tx=0x7f144c00ef20 comp rx=0 tx=0).stop 2024-08-31T07:27:29.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.660+0000 7f1461200640 1 -- 172.21.15.32:0/1774711471 shutdown_connections 2024-08-31T07:27:29.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.660+0000 7f1461200640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f14400b4ca0 0x7f14400b71b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.660+0000 7f1461200640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f145c10bbe0 0x7f145c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.660+0000 7f1461200640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f145c10b290 0x7f145c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.660+0000 7f1461200640 1 --2- 172.21.15.32:0/1774711471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c105950 0x7f145c19c220 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:29.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.660+0000 7f1461200640 1 -- 172.21.15.32:0/1774711471 >> 172.21.15.32:0/1774711471 conn(0x7f145c101420 msgr2=0x7f145c10a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:29.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.661+0000 7f1461200640 1 -- 172.21.15.32:0/1774711471 shutdown_connections 2024-08-31T07:27:29.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:29.661+0000 7f1461200640 1 -- 172.21.15.32:0/1774711471 wait complete. 2024-08-31T07:27:30.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23394 20.50 MB/sec warmup 70 sec latency 16717.807 ms 2024-08-31T07:27:30.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:30 smithi032 ceph-mon[17252]: pgmap v828: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-08-31T07:27:30.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:27:30.638 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:30 smithi073 ceph-mon[21995]: pgmap v828: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-08-31T07:27:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:30 smithi084 ceph-mon[22387]: pgmap v828: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-08-31T07:27:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.326+0000 7fd7ecc00640 1 -- 172.21.15.32:0/3258339942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e8103900 msgr2=0x7fd7e8103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.326+0000 7fd7ecc00640 1 --2- 172.21.15.32:0/3258339942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e8103900 0x7fd7e8103d10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009900 tx=0x7fd7d802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.327+0000 7fd7ecc00640 1 -- 172.21.15.32:0/3258339942 shutdown_connections 2024-08-31T07:27:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.327+0000 7fd7ecc00640 1 --2- 172.21.15.32:0/3258339942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e8109b90 0x7fd7e81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.327+0000 7fd7ecc00640 1 --2- 172.21.15.32:0/3258339942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e8109240 0x7fd7e8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.327+0000 7fd7ecc00640 1 --2- 172.21.15.32:0/3258339942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e8103900 0x7fd7e8103d10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.327+0000 7fd7ecc00640 1 -- 172.21.15.32:0/3258339942 >> 172.21.15.32:0/3258339942 conn(0x7fd7e807b5f0 msgr2=0x7fd7e807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.327+0000 7fd7ecc00640 1 -- 172.21.15.32:0/3258339942 shutdown_connections 2024-08-31T07:27:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.327+0000 7fd7ecc00640 1 -- 172.21.15.32:0/3258339942 wait complete. 2024-08-31T07:27:31.330 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23394 20.21 MB/sec warmup 71 sec latency 17717.859 ms 2024-08-31T07:27:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.328+0000 7fd7ecc00640 1 Processor -- start 2024-08-31T07:27:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.328+0000 7fd7ecc00640 1 -- start start 2024-08-31T07:27:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.328+0000 7fd7ecc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e8103900 0x7fd7e80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.328+0000 7fd7ecc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e8109240 0x7fd7e80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.329+0000 7fd7ecc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e8109b90 0x7fd7e8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.329+0000 7fd7ecc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e8113850 con 0x7fd7e8103900 2024-08-31T07:27:31.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.329+0000 7fd7ecc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e81136d0 con 0x7fd7e8109b90 2024-08-31T07:27:31.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.329+0000 7fd7ecc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e81139d0 con 0x7fd7e8109240 2024-08-31T07:27:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.329+0000 7fd7e5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e8103900 0x7fd7e80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.329+0000 7fd7e5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e8103900 0x7fd7e80754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34808/0 (socket says 172.21.15.32:34808) 2024-08-31T07:27:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.329+0000 7fd7e5e00640 1 -- 172.21.15.32:0/823631311 learned_addr learned my addr 172.21.15.32:0/823631311 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.329+0000 7fd7e6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e8109b90 0x7fd7e8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.329+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e8109240 0x7fd7e80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7dfe00640 1 -- 172.21.15.32:0/823631311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e8109b90 msgr2=0x7fd7e8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e8109b90 0x7fd7e8075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7dfe00640 1 -- 172.21.15.32:0/823631311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e8103900 msgr2=0x7fd7e80754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e8103900 0x7fd7e80754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7dfe00640 1 -- 172.21.15.32:0/823631311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e81a8500 con 0x7fd7e8109240 2024-08-31T07:27:31.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e8103900 0x7fd7e80754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:31.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7e6800640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e8109b90 0x7fd7e8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:31.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e8109240 0x7fd7e80759f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd7e806c790 tx=0x7fd7d000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:31.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7dea00640 1 -- 172.21.15.32:0/823631311 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d000cc40 con 0x7fd7e8109240 2024-08-31T07:27:31.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7dea00640 1 -- 172.21.15.32:0/823631311 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d000cde0 con 0x7fd7e8109240 2024-08-31T07:27:31.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7dea00640 1 -- 172.21.15.32:0/823631311 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0004a00 con 0x7fd7e8109240 2024-08-31T07:27:31.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.330+0000 7fd7ecc00640 1 -- 172.21.15.32:0/823631311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e81a87f0 con 0x7fd7e8109240 2024-08-31T07:27:31.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.331+0000 7fd7ecc00640 1 -- 172.21.15.32:0/823631311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e81a8d30 con 0x7fd7e8109240 2024-08-31T07:27:31.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.334+0000 7fd7f3827740 1 Processor -- start 2024-08-31T07:27:31.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.334+0000 7fd7f3827740 1 -- start start 2024-08-31T07:27:31.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.334+0000 7fd7dea00640 1 -- 172.21.15.32:0/823631311 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd7d0004ba0 con 0x7fd7e8109240 2024-08-31T07:27:31.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.334+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bfba520 0x555a1c35fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.334+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bf91660 0x555a1c360430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.334+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf99840 0x555a1c360970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.334+0000 7fd7f3827740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555a1c2d6a90 con 0x555a1bf99840 2024-08-31T07:27:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.334+0000 7fd7f3827740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555a1c2c6020 con 0x555a1bfba520 2024-08-31T07:27:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.334+0000 7fd7f3827740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555a1c2c9fd0 con 0x555a1bf91660 2024-08-31T07:27:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf99840 0x555a1c360970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7e6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bfba520 0x555a1c35fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf99840 0x555a1c360970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34810/0 (socket says 172.21.15.32:34810) 2024-08-31T07:27:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7dfe00640 1 -- 172.21.15.32:0/1335597855 learned_addr learned my addr 172.21.15.32:0/1335597855 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:31.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/1335597855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bf91660 0x555a1c360430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7dfe00640 1 -- 172.21.15.32:0/1335597855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bf91660 msgr2=0x555a1c360430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:31.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/1335597855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bf91660 0x555a1c360430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7dfe00640 1 -- 172.21.15.32:0/1335597855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bfba520 msgr2=0x555a1c35fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/1335597855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bfba520 0x555a1c35fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7dfe00640 1 -- 172.21.15.32:0/1335597855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1c025b00 con 0x555a1bf99840 2024-08-31T07:27:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.335+0000 7fd7e6800640 1 --2- 172.21.15.32:0/1335597855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bfba520 0x555a1c35fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.336+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/1335597855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf99840 0x555a1c360970 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00e2af0 tx=0x7fd7d00eb680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7f22f4e9a75208 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.336+0000 7fd7d7000640 1 -- 172.21.15.32:0/1335597855 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00f9070 con 0x555a1bf99840 2024-08-31T07:27:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.336+0000 7fd7d7000640 1 -- 172.21.15.32:0/1335597855 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d00e7580 con 0x555a1bf99840 2024-08-31T07:27:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.336+0000 7fd7d7000640 1 -- 172.21.15.32:0/1335597855 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00f1810 con 0x555a1bf99840 2024-08-31T07:27:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.337+0000 7fd7dea00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd7cc0b4d20 0x7fd7cc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.337+0000 7fd7dea00640 1 -- 172.21.15.32:0/823631311 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7fd7d00e5470 con 0x7fd7e8109240 2024-08-31T07:27:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.337+0000 7fd7f3827740 1 -- 172.21.15.32:0/1335597855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf99840 msgr2=0x555a1c360970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.337+0000 7fd7f3827740 1 --2- 172.21.15.32:0/1335597855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf99840 0x555a1c360970 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00e2af0 tx=0x7fd7d00eb680 comp rx=0 tx=0).stop 2024-08-31T07:27:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.338+0000 7fd7e6800640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd7cc0b4d20 0x7fd7cc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.338+0000 7fd7f3827740 1 -- 172.21.15.32:0/1335597855 shutdown_connections 2024-08-31T07:27:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.338+0000 7fd7f3827740 1 --2- 172.21.15.32:0/1335597855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf99840 0x555a1c360970 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.338+0000 7fd7f3827740 1 --2- 172.21.15.32:0/1335597855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bf91660 0x555a1c360430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.338+0000 7fd7f3827740 1 --2- 172.21.15.32:0/1335597855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bfba520 0x555a1c35fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.338+0000 7fd7f3827740 1 -- 172.21.15.32:0/1335597855 >> 172.21.15.32:0/1335597855 conn(0x555a1bfae910 msgr2=0x555a1c35dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.338+0000 7fd7f3827740 1 -- 172.21.15.32:0/1335597855 shutdown_connections 2024-08-31T07:27:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.338+0000 7fd7f3827740 1 -- 172.21.15.32:0/1335597855 wait complete. 2024-08-31T07:27:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.339+0000 7fd7e6800640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd7cc0b4d20 0x7fd7cc0b7230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009fd0 tx=0x7fd7e0009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.339+0000 7fd7f3827740 1 -- start start 2024-08-31T07:27:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.339+0000 7fd7f3827740 1 Processor -- start 2024-08-31T07:27:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.340+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bf91660 0x555a1c36b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.340+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c36b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.340+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bfba520 0x555a1c36bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.340+0000 7fd7f3827740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555a1c128bb0 con 0x555a1bfba520 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.340+0000 7fd7f3827740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555a1c24cf90 con 0x555a1bf99840 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.340+0000 7fd7f3827740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555a1c36e6d0 con 0x555a1bf91660 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.340+0000 7fd7f3827740 10 client.0 _ll_register_callbacks cb 0x555a1c272710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.340+0000 7fd7e6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bfba520 0x555a1c36bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.340+0000 7fd7e6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bfba520 0x555a1c36bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34826/0 (socket says 172.21.15.32:34826) 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.340+0000 7fd7e6800640 1 -- 172.21.15.32:0/608713491 learned_addr learned my addr 172.21.15.32:0/608713491 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.341+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bf91660 0x555a1c36b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.341+0000 7fd7e6800640 1 -- 172.21.15.32:0/608713491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bf91660 msgr2=0x555a1c36b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.341+0000 7fd7e6800640 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bf91660 0x555a1c36b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.341+0000 7fd7e6800640 1 -- 172.21.15.32:0/608713491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 msgr2=0x555a1c36b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.341+0000 7fd7e6800640 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c36b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.341+0000 7fd7e6800640 1 -- 172.21.15.32:0/608713491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1c09c960 con 0x555a1bfba520 2024-08-31T07:27:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.341+0000 7fd7e6800640 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bfba520 0x555a1c36bd80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0011b10 tx=0x7fd7e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14887b33c67f8fbf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.341+0000 7fd7cb400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:27:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.342+0000 7fd7cb400640 1 -- 172.21.15.32:0/608713491 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e002d060 con 0x555a1bfba520 2024-08-31T07:27:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.342+0000 7fd7cb400640 1 -- 172.21.15.32:0/608713491 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7fd7e0017030 con 0x555a1bfba520 2024-08-31T07:27:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.342+0000 7fd7f3827740 1 -- 172.21.15.32:0/608713491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555a1c264e30 con 0x555a1bfba520 2024-08-31T07:27:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.342+0000 7fd7cb400640 1 -- 172.21.15.32:0/608713491 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd7e001d6a0 con 0x555a1bfba520 2024-08-31T07:27:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.342+0000 7fd7f3827740 10 client.18507 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.342+0000 7fd7f3827740 1 -- 172.21.15.32:0/608713491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555a1c37a190 con 0x555a1bfba520 2024-08-31T07:27:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.343+0000 7fd7cb400640 1 -- 172.21.15.32:0/608713491 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555a1c264e30 con 0x555a1bfba520 2024-08-31T07:27:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.343+0000 7fd7cb400640 1 -- 172.21.15.32:0/608713491 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7fd7e00289d0 con 0x555a1bfba520 2024-08-31T07:27:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.343+0000 7fd7f3827740 10 client.18507 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.343+0000 7fd7f3827740 10 client.18507 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:27:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.343+0000 7fd7f3827740 20 client.18507 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.343+0000 7fd7f3827740 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a1bf78ea0 0x555a1c37d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.343+0000 7fd7f3827740 4 client.18507 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:27:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.343+0000 7fd7f3827740 1 -- 172.21.15.32:0/608713491 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a1c2d89e0 con 0x555a1bf78ea0 2024-08-31T07:27:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.344+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a1bf78ea0 0x555a1c37d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.345+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a1bf78ea0 0x555a1c37d7e0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a4a9a6d86b423e server_cookie=e75561f38fd1271 in_seq=0 out_seq=0 2024-08-31T07:27:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.345+0000 7fd7cb400640 10 client.18507 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7cb400640 1 -- 172.21.15.32:0/608713491 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a1c2d89e0 con 0x555a1bf78ea0 2024-08-31T07:27:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7cb400640 10 client.18507 handle_command_reply: tid=0 2024-08-31T07:27:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 client.18507 shutdown 2024-08-31T07:27:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 -- 172.21.15.32:0/608713491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bfba520 msgr2=0x555a1c36bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bfba520 0x555a1c36bd80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0011b10 tx=0x7fd7e0004ab0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 -- 172.21.15.32:0/608713491 shutdown_connections 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a1bf78ea0 0x555a1c37d7e0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bfba520 0x555a1c36bd80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c36b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 --2- 172.21.15.32:0/608713491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bf91660 0x555a1c36b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 -- 172.21.15.32:0/608713491 >> 172.21.15.32:0/608713491 conn(0x555a1bfae910 msgr2=0x555a1c3630d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7cb400640 0 client.18507 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 -- 172.21.15.32:0/608713491 shutdown_connections 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.346+0000 7fd7f3827740 1 -- 172.21.15.32:0/608713491 wait complete. 2024-08-31T07:27:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.347+0000 7fd7f3827740 20 client.18507 trim_cache size 0 max 16384 2024-08-31T07:27:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7f3827740 1 Processor -- start 2024-08-31T07:27:31.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7f3827740 1 -- start start 2024-08-31T07:27:31.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c36bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c3778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 0x555a1c361630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7f3827740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e81b1530 con 0x555a1bf91660 2024-08-31T07:27:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7f3827740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e81b3250 con 0x555a1bf99840 2024-08-31T07:27:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7f3827740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e81b1c30 con 0x555a1bfba520 2024-08-31T07:27:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7e5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c36bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7e5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c36bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34842/0 (socket says 172.21.15.32:34842) 2024-08-31T07:27:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.366+0000 7fd7e5e00640 1 -- 172.21.15.32:0/2008896304 learned_addr learned my addr 172.21.15.32:0/2008896304 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.367+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/2008896304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c3778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.367+0000 7fd7e6800640 1 --2- 172.21.15.32:0/2008896304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 0x555a1c361630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.367+0000 7fd7e5e00640 1 -- 172.21.15.32:0/2008896304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 msgr2=0x555a1c361630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.367+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/2008896304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 0x555a1c361630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.367+0000 7fd7e5e00640 1 -- 172.21.15.32:0/2008896304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 msgr2=0x555a1c3778c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.367+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/2008896304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c3778c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.367+0000 7fd7e5e00640 1 -- 172.21.15.32:0/2008896304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1c3621e0 con 0x555a1bf91660 2024-08-31T07:27:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.367+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/2008896304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c36bc30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8005f70 tx=0x7fd7d80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462f83979a13dee2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.368+0000 7fd7d6600640 1 -- 172.21.15.32:0/2008896304 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8035040 con 0x555a1bf91660 2024-08-31T07:27:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.368+0000 7fd7d6600640 1 -- 172.21.15.32:0/2008896304 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d8049070 con 0x555a1bf91660 2024-08-31T07:27:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.368+0000 7fd7d6600640 1 -- 172.21.15.32:0/2008896304 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8003e20 con 0x555a1bf91660 2024-08-31T07:27:31.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.369+0000 7fd7f3827740 1 -- 172.21.15.32:0/2008896304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 msgr2=0x555a1c36bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:31.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.369+0000 7fd7f3827740 1 --2- 172.21.15.32:0/2008896304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c36bc30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8005f70 tx=0x7fd7d80058d0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.369+0000 7fd7f3827740 1 -- 172.21.15.32:0/2008896304 shutdown_connections 2024-08-31T07:27:31.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.369+0000 7fd7f3827740 1 --2- 172.21.15.32:0/2008896304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 0x555a1c361630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.369+0000 7fd7f3827740 1 --2- 172.21.15.32:0/2008896304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c3778c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.369+0000 7fd7f3827740 1 --2- 172.21.15.32:0/2008896304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c36bc30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.369+0000 7fd7f3827740 1 -- 172.21.15.32:0/2008896304 >> 172.21.15.32:0/2008896304 conn(0x555a1bfae910 msgr2=0x555a1c360f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.369+0000 7fd7f3827740 1 -- 172.21.15.32:0/2008896304 shutdown_connections 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.369+0000 7fd7f3827740 1 -- 172.21.15.32:0/2008896304 wait complete. 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.370+0000 7fd7f3827740 1 -- start start 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.370+0000 7fd7f3827740 1 Processor -- start 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.370+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c370180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.370+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c370760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.370+0000 7fd7f3827740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 0x555a1c3e71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.370+0000 7fd7f3827740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e81b2ac0 con 0x555a1bf91660 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.370+0000 7fd7f3827740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e81b2390 con 0x555a1bf99840 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.370+0000 7fd7f3827740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e81af530 con 0x555a1bfba520 2024-08-31T07:27:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.370+0000 7fd7f3827740 10 client.0 _ll_register_callbacks cb 0x7fd7e00119d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c370180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c370180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34850/0 (socket says 172.21.15.32:34850) 2024-08-31T07:27:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e5e00640 1 -- 172.21.15.32:0/1088128862 learned_addr learned my addr 172.21.15.32:0/1088128862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c370760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e6800640 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 0x555a1c3e71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e5e00640 1 -- 172.21.15.32:0/1088128862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 msgr2=0x555a1c3e71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 0x555a1c3e71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e5e00640 1 -- 172.21.15.32:0/1088128862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 msgr2=0x555a1c370760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c370760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e5e00640 1 -- 172.21.15.32:0/1088128862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1c3e7810 con 0x555a1bf91660 2024-08-31T07:27:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e6800640 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 0x555a1c3e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:27:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c370760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7e5e00640 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c370180 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8031440 tx=0x7fd7d8032750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9fab3e70cdd9c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.371+0000 7fd7cb400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:27:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.372+0000 7fd7cb400640 1 -- 172.21.15.32:0/1088128862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8035070 con 0x555a1bf91660 2024-08-31T07:27:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.372+0000 7fd7cb400640 1 -- 172.21.15.32:0/1088128862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7fd7d80035f0 con 0x555a1bf91660 2024-08-31T07:27:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.372+0000 7fd7f3827740 1 -- 172.21.15.32:0/1088128862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d8049070 con 0x555a1bf91660 2024-08-31T07:27:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.372+0000 7fd7cb400640 1 -- 172.21.15.32:0/1088128862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd7d805b020 con 0x555a1bf91660 2024-08-31T07:27:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.372+0000 7fd7f3827740 10 client.18513 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.372+0000 7fd7f3827740 1 -- 172.21.15.32:0/1088128862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555a1c3e8b00 con 0x555a1bf91660 2024-08-31T07:27:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.373+0000 7fd7cb400640 1 -- 172.21.15.32:0/1088128862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d8049070 con 0x555a1bf91660 2024-08-31T07:27:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.373+0000 7fd7cb400640 1 -- 172.21.15.32:0/1088128862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7fd7d8005470 con 0x555a1bf91660 2024-08-31T07:27:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.373+0000 7fd7f3827740 10 client.18513 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.373+0000 7fd7f3827740 10 client.18513 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:27:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.373+0000 7fd7f3827740 20 client.18513 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.373+0000 7fd7f3827740 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a1bf4e250 0x555a1c3eb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.373+0000 7fd7f3827740 4 client.18513 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f9a618dc", "release": true, "await": true}] 2024-08-31T07:27:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.373+0000 7fd7f3827740 1 -- 172.21.15.32:0/1088128862 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f9a618dc", "release": true, "await": true}) -- 0x555a1c2d4480 con 0x555a1bf4e250 2024-08-31T07:27:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.373+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a1bf4e250 0x555a1c3eb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.374+0000 7fd7dfe00640 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a1bf4e250 0x555a1c3eb300 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e087a40450421ab server_cookie=c2c46a8c4f4fc8bd in_seq=0 out_seq=0 2024-08-31T07:27:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.374+0000 7fd7cb400640 10 client.18513 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:31.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.400+0000 7fd7cb400640 1 -- 172.21.15.32:0/1088128862 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555a1c2d4480 con 0x555a1bf4e250 2024-08-31T07:27:31.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.400+0000 7fd7cb400640 10 client.18513 handle_command_reply: tid=0 2024-08-31T07:27:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.400+0000 7fd7f3827740 1 client.18513 shutdown 2024-08-31T07:27:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.400+0000 7fd7f3827740 1 -- 172.21.15.32:0/1088128862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 msgr2=0x555a1c370180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.400+0000 7fd7f3827740 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c370180 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8031440 tx=0x7fd7d8032750 comp rx=0 tx=0).stop 2024-08-31T07:27:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.400+0000 7fd7f3827740 1 -- 172.21.15.32:0/1088128862 shutdown_connections 2024-08-31T07:27:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.400+0000 7fd7f3827740 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a1bf4e250 0x555a1c3eb300 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.400+0000 7fd7f3827740 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a1bfba520 0x555a1c3e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.400+0000 7fd7f3827740 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a1bf99840 0x555a1c370760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.400+0000 7fd7f3827740 1 --2- 172.21.15.32:0/1088128862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a1bf91660 0x555a1c370180 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.401+0000 7fd7f3827740 1 -- 172.21.15.32:0/1088128862 >> 172.21.15.32:0/1088128862 conn(0x555a1bfae910 msgr2=0x555a1c36ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.401+0000 7fd7cb400640 0 client.18513 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.401+0000 7fd7f3827740 1 -- 172.21.15.32:0/1088128862 shutdown_connections 2024-08-31T07:27:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.401+0000 7fd7f3827740 1 -- 172.21.15.32:0/1088128862 wait complete. 2024-08-31T07:27:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.402+0000 7fd7f3827740 20 client.18513 trim_cache size 0 max 16384 2024-08-31T07:27:31.404 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:27:31.405 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 55, 2024-08-31T07:27:31.405 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:27:31.405 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 48, 2024-08-31T07:27:31.405 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:27:31.405 INFO:teuthology.orchestra.run.smithi032.stdout: "f9a618dc": { 2024-08-31T07:27:31.405 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 48, 2024-08-31T07:27:31.405 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:27:31.405 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:27:31.405 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:27:31.405 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:27:31.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:31.406 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:27:31.406 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 155.0, 2024-08-31T07:27:31.406 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:27:31.406 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:27:31.406 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:27:31.406 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:27:31.406 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:27:31.406 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:27:31.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:31.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:31.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:31.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:31.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:31.407 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:27:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.403+0000 7fd7ecc00640 1 -- 172.21.15.32:0/823631311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd7cc0b4d20 msgr2=0x7fd7cc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.403+0000 7fd7ecc00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd7cc0b4d20 0x7fd7cc0b7230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009fd0 tx=0x7fd7e0009500 comp rx=0 tx=0).stop 2024-08-31T07:27:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.403+0000 7fd7ecc00640 1 -- 172.21.15.32:0/823631311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e8109240 msgr2=0x7fd7e80759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.403+0000 7fd7ecc00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e8109240 0x7fd7e80759f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd7e806c790 tx=0x7fd7d000ee50 comp rx=0 tx=0).stop 2024-08-31T07:27:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.405+0000 7fd7ecc00640 1 -- 172.21.15.32:0/823631311 shutdown_connections 2024-08-31T07:27:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.405+0000 7fd7ecc00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd7cc0b4d20 0x7fd7cc0b7230 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.405+0000 7fd7ecc00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e8109b90 0x7fd7e8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.405+0000 7fd7ecc00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e8109240 0x7fd7e80759f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.405+0000 7fd7ecc00640 1 --2- 172.21.15.32:0/823631311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7e8103900 0x7fd7e80754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.405+0000 7fd7ecc00640 1 -- 172.21.15.32:0/823631311 >> 172.21.15.32:0/823631311 conn(0x7fd7e807b5f0 msgr2=0x7fd7e81072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.406+0000 7fd7ecc00640 1 -- 172.21.15.32:0/823631311 shutdown_connections 2024-08-31T07:27:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:31.406+0000 7fd7ecc00640 1 -- 172.21.15.32:0/823631311 wait complete. 2024-08-31T07:27:32.329 INFO:tasks.workunit.client.0.smithi032.stdout: 10 24452 20.61 MB/sec warmup 72 sec latency 17790.845 ms 2024-08-31T07:27:32.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:32 smithi032 ceph-mon[17252]: pgmap v829: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-08-31T07:27:32.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:32 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:27:32.652 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f9a618dc', total seconds elapsed: 43.9 2024-08-31T07:27:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:32 smithi073 ceph-mon[21995]: pgmap v829: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-08-31T07:27:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:32 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:27:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:32 smithi084 ceph-mon[22387]: pgmap v829: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.6 MiB/s wr, 95 op/s 2024-08-31T07:27:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:32 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:27:33.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.058+0000 7f3977a00640 1 -- 172.21.15.32:0/2365868349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3970102540 msgr2=0x7f3970102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:33.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.059+0000 7f3977a00640 1 --2- 172.21.15.32:0/2365868349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3970102540 0x7f3970102950 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3968009c10 tx=0x7f396802a6f0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.060+0000 7f3977a00640 1 -- 172.21.15.32:0/2365868349 shutdown_connections 2024-08-31T07:27:33.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.060+0000 7f3977a00640 1 --2- 172.21.15.32:0/2365868349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3970107980 0x7f3970107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.060+0000 7f3977a00640 1 --2- 172.21.15.32:0/2365868349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3970102e90 0x7f39701073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.060+0000 7f3977a00640 1 --2- 172.21.15.32:0/2365868349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3970102540 0x7f3970102950 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.060+0000 7f3977a00640 1 -- 172.21.15.32:0/2365868349 >> 172.21.15.32:0/2365868349 conn(0x7f39700794d0 msgr2=0x7f39700798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:33.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.060+0000 7f3977a00640 1 -- 172.21.15.32:0/2365868349 shutdown_connections 2024-08-31T07:27:33.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.060+0000 7f3977a00640 1 -- 172.21.15.32:0/2365868349 wait complete. 2024-08-31T07:27:33.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.062+0000 7f3977a00640 1 Processor -- start 2024-08-31T07:27:33.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.063+0000 7f3977a00640 1 -- start start 2024-08-31T07:27:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.063+0000 7f3977a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3970102540 0x7f39701a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.063+0000 7f3977a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3970102e90 0x7f39701a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.063+0000 7f3977a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3970107980 0x7f39701a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.063+0000 7f3977a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3970119e20 con 0x7f3970107980 2024-08-31T07:27:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.063+0000 7f3977a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3970119ca0 con 0x7f3970102540 2024-08-31T07:27:33.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.063+0000 7f3977a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3970119fa0 con 0x7f3970102e90 2024-08-31T07:27:33.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.064+0000 7f396fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3970102e90 0x7f39701a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.064+0000 7f3975000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3970102540 0x7f39701a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.065+0000 7f396fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3970102e90 0x7f39701a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:39300/0 (socket says 172.21.15.32:39300) 2024-08-31T07:27:33.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.065+0000 7f396fe00640 1 -- 172.21.15.32:0/1777985607 learned_addr learned my addr 172.21.15.32:0/1777985607 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:33.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.065+0000 7f3975000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3970102540 0x7f39701a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50792/0 (socket says 172.21.15.32:50792) 2024-08-31T07:27:33.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.064+0000 7f3975a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3970107980 0x7f39701a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.066+0000 7f3975000640 1 -- 172.21.15.32:0/1777985607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3970102e90 msgr2=0x7f39701a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.066+0000 7f3975000640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3970102e90 0x7f39701a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.066+0000 7f3975000640 1 -- 172.21.15.32:0/1777985607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3970107980 msgr2=0x7f39701a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.066+0000 7f3975000640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3970107980 0x7f39701a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.066+0000 7f3975000640 1 -- 172.21.15.32:0/1777985607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39701a6800 con 0x7f3970102540 2024-08-31T07:27:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.066+0000 7f396fe00640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3970102e90 0x7f39701a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:27:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.067+0000 7f3975000640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3970102540 0x7f39701a0890 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3968034040 tx=0x7f3968003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.067+0000 7f396d600640 1 -- 172.21.15.32:0/1777985607 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968041020 con 0x7f3970102540 2024-08-31T07:27:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.067+0000 7f396d600640 1 -- 172.21.15.32:0/1777985607 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3968030970 con 0x7f3970102540 2024-08-31T07:27:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.067+0000 7f396d600640 1 -- 172.21.15.32:0/1777985607 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968036040 con 0x7f3970102540 2024-08-31T07:27:33.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.069+0000 7f3977a00640 1 -- 172.21.15.32:0/1777985607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39701a6a90 con 0x7f3970102540 2024-08-31T07:27:33.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.069+0000 7f3977a00640 1 -- 172.21.15.32:0/1777985607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39701a6ef0 con 0x7f3970102540 2024-08-31T07:27:33.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.074+0000 7f397e606740 1 Processor -- start 2024-08-31T07:27:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.075+0000 7f397e606740 1 -- start start 2024-08-31T07:27:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.075+0000 7f397e606740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e234990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.075+0000 7f397e606740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de661b0 0x560b9e234ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.075+0000 7f397e606740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de6e390 0x560b9e235410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.075+0000 7f397e606740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560b9e1ab7b0 con 0x560b9de6e390 2024-08-31T07:27:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.075+0000 7f397e606740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560b9e19ad40 con 0x560b9de661b0 2024-08-31T07:27:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.075+0000 7f397e606740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560b9e19ecf0 con 0x560b9de8f070 2024-08-31T07:27:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.076+0000 7f3975a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e234990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.076+0000 7f3975a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e234990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:39312/0 (socket says 172.21.15.32:39312) 2024-08-31T07:27:33.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.076+0000 7f3975a00640 1 -- 172.21.15.32:0/3255846221 learned_addr learned my addr 172.21.15.32:0/3255846221 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.076+0000 7f396fe00640 1 --2- 172.21.15.32:0/3255846221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de6e390 0x560b9e235410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.076+0000 7f3975a00640 1 -- 172.21.15.32:0/3255846221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de661b0 msgr2=0x560b9e234ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:27:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.076+0000 7f3975a00640 1 --2- 172.21.15.32:0/3255846221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de661b0 0x560b9e234ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.076+0000 7f3975a00640 1 -- 172.21.15.32:0/3255846221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de6e390 msgr2=0x560b9e235410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.076+0000 7f3975a00640 1 --2- 172.21.15.32:0/3255846221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de6e390 0x560b9e235410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.076+0000 7f3975a00640 1 -- 172.21.15.32:0/3255846221 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9defa650 con 0x560b9de8f070 2024-08-31T07:27:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.077+0000 7f3975a00640 1 --2- 172.21.15.32:0/3255846221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e234990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f396000b700 tx=0x7f396000bbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca36dbb886768f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.078+0000 7f3966000640 1 -- 172.21.15.32:0/3255846221 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960003dd0 con 0x560b9de8f070 2024-08-31T07:27:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.079+0000 7f3966000640 1 -- 172.21.15.32:0/3255846221 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3960015070 con 0x560b9de8f070 2024-08-31T07:27:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.079+0000 7f3966000640 1 -- 172.21.15.32:0/3255846221 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39600106a0 con 0x560b9de8f070 2024-08-31T07:27:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.080+0000 7f397e606740 1 -- 172.21.15.32:0/3255846221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 msgr2=0x560b9e234990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.080+0000 7f397e606740 1 --2- 172.21.15.32:0/3255846221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e234990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f396000b700 tx=0x7f396000bbc0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.080+0000 7f397e606740 1 -- 172.21.15.32:0/3255846221 shutdown_connections 2024-08-31T07:27:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.080+0000 7f397e606740 1 --2- 172.21.15.32:0/3255846221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de6e390 0x560b9e235410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.080+0000 7f397e606740 1 --2- 172.21.15.32:0/3255846221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de661b0 0x560b9e234ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.080+0000 7f397e606740 1 --2- 172.21.15.32:0/3255846221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e234990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.080+0000 7f397e606740 1 -- 172.21.15.32:0/3255846221 >> 172.21.15.32:0/3255846221 conn(0x560b9de83460 msgr2=0x560b9e232880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.083+0000 7f396d600640 1 -- 172.21.15.32:0/1777985607 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3968044020 con 0x7f3970102540 2024-08-31T07:27:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.084+0000 7f397e606740 1 -- 172.21.15.32:0/3255846221 shutdown_connections 2024-08-31T07:27:33.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.084+0000 7f397e606740 1 -- 172.21.15.32:0/3255846221 wait complete. 2024-08-31T07:27:33.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.085+0000 7f397e606740 1 -- start start 2024-08-31T07:27:33.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.085+0000 7f396d600640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39540b4c30 0x7f39540b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.085+0000 7f397e606740 1 Processor -- start 2024-08-31T07:27:33.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.085+0000 7f396d600640 1 -- 172.21.15.32:0/1777985607 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f3968105c10 con 0x7f3970102540 2024-08-31T07:27:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.086+0000 7f3975a00640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39540b4c30 0x7f39540b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.087+0000 7f397e606740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 0x560b9e2405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.087+0000 7f397e606740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e240b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.087+0000 7f397e606740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 0x560b9e25d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.087+0000 7f397e606740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560b9dffd700 con 0x560b9de4d9f0 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.087+0000 7f397e606740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560b9e121cf0 con 0x560b9de6e390 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.087+0000 7f397e606740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560b9e254180 con 0x560b9de8f070 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.087+0000 7f397e606740 10 client.0 _ll_register_callbacks cb 0x560b9e147470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.088+0000 7f3975000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 0x560b9e2405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.088+0000 7f396fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e240b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.088+0000 7f3975000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 0x560b9e2405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50794/0 (socket says 172.21.15.32:50794) 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.088+0000 7f3975000640 1 -- 172.21.15.32:0/1651248478 learned_addr learned my addr 172.21.15.32:0/1651248478 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.088+0000 7f396fe00640 1 -- 172.21.15.32:0/1651248478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 msgr2=0x560b9e2405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.088+0000 7f396fe00640 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 0x560b9e2405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.088+0000 7f396fe00640 1 -- 172.21.15.32:0/1651248478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 msgr2=0x560b9e25d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.088+0000 7f396fe00640 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 0x560b9e25d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.088+0000 7f396fe00640 1 -- 172.21.15.32:0/1651248478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9e1628b0 con 0x560b9de8f070 2024-08-31T07:27:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.089+0000 7f396fe00640 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e240b20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3970102260 tx=0x7f395c0039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6425b7003fb761b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.089+0000 7f395aa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:27:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.089+0000 7f395aa00640 1 -- 172.21.15.32:0/1651248478 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c005220 con 0x560b9de8f070 2024-08-31T07:27:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.089+0000 7f395aa00640 1 -- 172.21.15.32:0/1651248478 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f395c0053c0 con 0x560b9de8f070 2024-08-31T07:27:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.089+0000 7f397e606740 1 -- 172.21.15.32:0/1651248478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3960015070 con 0x560b9de8f070 2024-08-31T07:27:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.090+0000 7f3975a00640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39540b4c30 0x7f39540b7140 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3960011d00 tx=0x7f3960000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.090+0000 7f395aa00640 1 -- 172.21.15.32:0/1651248478 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f395c0055c0 con 0x560b9de8f070 2024-08-31T07:27:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.090+0000 7f397e606740 10 client.28534 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.090+0000 7f397e606740 1 -- 172.21.15.32:0/1651248478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560b9e260270 con 0x560b9de8f070 2024-08-31T07:27:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.090+0000 7f395aa00640 1 -- 172.21.15.32:0/1651248478 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3960015070 con 0x560b9de8f070 2024-08-31T07:27:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.091+0000 7f395aa00640 1 -- 172.21.15.32:0/1651248478 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f395c019030 con 0x560b9de8f070 2024-08-31T07:27:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.091+0000 7f397e606740 10 client.28534 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.091+0000 7f397e606740 10 client.28534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:27:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.091+0000 7f397e606740 20 client.28534 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.091+0000 7f397e606740 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560b9de22da0 0x560b9e263800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.091+0000 7f397e606740 4 client.28534 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:27:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.091+0000 7f397e606740 1 -- 172.21.15.32:0/1651248478 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b9e1ad700 con 0x560b9de22da0 2024-08-31T07:27:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.092+0000 7f3975000640 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560b9de22da0 0x560b9e263800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.093+0000 7f3975000640 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560b9de22da0 0x560b9e263800 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4680d5844ea9944 server_cookie=b0e98ca4267d5610 in_seq=0 out_seq=0 2024-08-31T07:27:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.093+0000 7f395aa00640 10 client.28534 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f395aa00640 1 -- 172.21.15.32:0/1651248478 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560b9e1ad700 con 0x560b9de22da0 2024-08-31T07:27:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f395aa00640 10 client.28534 handle_command_reply: tid=0 2024-08-31T07:27:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f397e606740 1 client.28534 shutdown 2024-08-31T07:27:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f397e606740 1 -- 172.21.15.32:0/1651248478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 msgr2=0x560b9e240b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f397e606740 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e240b20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3970102260 tx=0x7f395c0039c0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f397e606740 1 -- 172.21.15.32:0/1651248478 shutdown_connections 2024-08-31T07:27:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f397e606740 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560b9de22da0 0x560b9e263800 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f397e606740 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 0x560b9e25d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f397e606740 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e240b20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f397e606740 1 --2- 172.21.15.32:0/1651248478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 0x560b9e2405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f395aa00640 0 client.28534 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.094+0000 7f397e606740 1 -- 172.21.15.32:0/1651248478 >> 172.21.15.32:0/1651248478 conn(0x560b9de83460 msgr2=0x560b9e237b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.095+0000 7f397e606740 1 -- 172.21.15.32:0/1651248478 shutdown_connections 2024-08-31T07:27:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.095+0000 7f397e606740 1 -- 172.21.15.32:0/1651248478 wait complete. 2024-08-31T07:27:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.095+0000 7f397e606740 20 client.28534 trim_cache size 0 max 16384 2024-08-31T07:27:33.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.113+0000 7f397e606740 1 Processor -- start 2024-08-31T07:27:33.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.114+0000 7f397e606740 1 -- start start 2024-08-31T07:27:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f397e606740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de4d9f0 0x560b9e25d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f397e606740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de6e390 0x560b9e240580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f397e606740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de8f070 0x560b9e240ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f397e606740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f39701ad370 con 0x560b9de8f070 2024-08-31T07:27:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f397e606740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39701aef50 con 0x560b9de4d9f0 2024-08-31T07:27:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f397e606740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f39701ad930 con 0x560b9de6e390 2024-08-31T07:27:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f3975000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de4d9f0 0x560b9e25d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f3975000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de4d9f0 0x560b9e25d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50796/0 (socket says 172.21.15.32:50796) 2024-08-31T07:27:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f3975000640 1 -- 172.21.15.32:0/3719522411 learned_addr learned my addr 172.21.15.32:0/3719522411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f3975a00640 1 --2- 172.21.15.32:0/3719522411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de8f070 0x560b9e240ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.115+0000 7f396fe00640 1 --2- 172.21.15.32:0/3719522411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de6e390 0x560b9e240580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.116+0000 7f396fe00640 1 -- 172.21.15.32:0/3719522411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de4d9f0 msgr2=0x560b9e25d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.116+0000 7f396fe00640 1 --2- 172.21.15.32:0/3719522411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de4d9f0 0x560b9e25d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.116+0000 7f396fe00640 1 -- 172.21.15.32:0/3719522411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de8f070 msgr2=0x560b9e240ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.116+0000 7f396fe00640 1 --2- 172.21.15.32:0/3719522411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de8f070 0x560b9e240ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.116+0000 7f396fe00640 1 -- 172.21.15.32:0/3719522411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9e23bf30 con 0x560b9de6e390 2024-08-31T07:27:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.117+0000 7f396fe00640 1 --2- 172.21.15.32:0/3719522411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de6e390 0x560b9e240580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f395c00fbe0 tx=0x7f395c00f930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ed7a5e9c43d14ee server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.119+0000 7f3965600640 1 -- 172.21.15.32:0/3719522411 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c02a070 con 0x560b9de6e390 2024-08-31T07:27:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.119+0000 7f3965600640 1 -- 172.21.15.32:0/3719522411 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f395c00fe60 con 0x560b9de6e390 2024-08-31T07:27:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.119+0000 7f3965600640 1 -- 172.21.15.32:0/3719522411 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0116a0 con 0x560b9de6e390 2024-08-31T07:27:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.121+0000 7f397e606740 1 -- 172.21.15.32:0/3719522411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de6e390 msgr2=0x560b9e240580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.121+0000 7f397e606740 1 --2- 172.21.15.32:0/3719522411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de6e390 0x560b9e240580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f395c00fbe0 tx=0x7f395c00f930 comp rx=0 tx=0).stop 2024-08-31T07:27:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.121+0000 7f397e606740 1 -- 172.21.15.32:0/3719522411 shutdown_connections 2024-08-31T07:27:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.121+0000 7f397e606740 1 --2- 172.21.15.32:0/3719522411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de8f070 0x560b9e240ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.121+0000 7f397e606740 1 --2- 172.21.15.32:0/3719522411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de6e390 0x560b9e240580 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.121+0000 7f397e606740 1 --2- 172.21.15.32:0/3719522411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de4d9f0 0x560b9e25d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.121+0000 7f397e606740 1 -- 172.21.15.32:0/3719522411 >> 172.21.15.32:0/3719522411 conn(0x560b9de83460 msgr2=0x560b9e23ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.122+0000 7f397e606740 1 -- 172.21.15.32:0/3719522411 shutdown_connections 2024-08-31T07:27:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.123+0000 7f397e606740 1 -- 172.21.15.32:0/3719522411 wait complete. 2024-08-31T07:27:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.123+0000 7f397e606740 1 -- start start 2024-08-31T07:27:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.124+0000 7f397e606740 1 Processor -- start 2024-08-31T07:27:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.125+0000 7f397e606740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 0x560b9e255ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.125+0000 7f397e606740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 0x560b9e2564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.125+0000 7f397e606740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e2c7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.125+0000 7f3975000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 0x560b9e255ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.126+0000 7f3975000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 0x560b9e255ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34920/0 (socket says 172.21.15.32:34920) 2024-08-31T07:27:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.126+0000 7f397e606740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f39701ae7c0 con 0x560b9de4d9f0 2024-08-31T07:27:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.126+0000 7f397e606740 1 -- 172.21.15.32:0/3716132892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f39701ae090 con 0x560b9de6e390 2024-08-31T07:27:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.126+0000 7f3975000640 1 -- 172.21.15.32:0/3716132892 learned_addr learned my addr 172.21.15.32:0/3716132892 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.126+0000 7f397e606740 1 -- 172.21.15.32:0/3716132892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f39701a19c0 con 0x560b9de8f070 2024-08-31T07:27:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.126+0000 7f397e606740 10 client.0 _ll_register_callbacks cb 0x7f395c009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.126+0000 7f396fe00640 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 0x560b9e2564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.127+0000 7f396fe00640 1 -- 172.21.15.32:0/3716132892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 msgr2=0x560b9e2c7240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:27:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.127+0000 7f396fe00640 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e2c7240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.127+0000 7f396fe00640 1 -- 172.21.15.32:0/3716132892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 msgr2=0x560b9e255ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.127+0000 7f396fe00640 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 0x560b9e255ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.127+0000 7f396fe00640 1 -- 172.21.15.32:0/3716132892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9e2c78b0 con 0x560b9de6e390 2024-08-31T07:27:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.127+0000 7f3975000640 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 0x560b9e255ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:27:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.129+0000 7f396fe00640 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 0x560b9e2564d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f395c00fb10 tx=0x7f395c024b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc03234c13fb79b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.129+0000 7f395aa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:27:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.131+0000 7f395aa00640 1 -- 172.21.15.32:0/3716132892 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c02a070 con 0x560b9de6e390 2024-08-31T07:27:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.131+0000 7f397e606740 1 -- 172.21.15.32:0/3716132892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f395c00fe60 con 0x560b9de6e390 2024-08-31T07:27:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.131+0000 7f395aa00640 1 -- 172.21.15.32:0/3716132892 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f395c033040 con 0x560b9de6e390 2024-08-31T07:27:33.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.133+0000 7f395aa00640 1 -- 172.21.15.32:0/3716132892 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f395c02a070 con 0x560b9de6e390 2024-08-31T07:27:33.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.133+0000 7f397e606740 10 client.28547 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:33.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.133+0000 7f397e606740 1 -- 172.21.15.32:0/3716132892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560b9e2cec80 con 0x560b9de6e390 2024-08-31T07:27:33.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.133+0000 7f395aa00640 1 -- 172.21.15.32:0/3716132892 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f395c00fe60 con 0x560b9de6e390 2024-08-31T07:27:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.135+0000 7f395aa00640 1 -- 172.21.15.32:0/3716132892 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f395c019070 con 0x560b9de6e390 2024-08-31T07:27:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.135+0000 7f397e606740 10 client.28547 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.135+0000 7f397e606740 10 client.28547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:27:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.136+0000 7f397e606740 20 client.28547 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:33.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.136+0000 7f397e606740 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560b9e2cf400 0x560b9e2d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:33.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.136+0000 7f397e606740 4 client.28547 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:27:33.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.136+0000 7f397e606740 1 -- 172.21.15.32:0/3716132892 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b9e1a91a0 con 0x560b9e2cf400 2024-08-31T07:27:33.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.137+0000 7f3975000640 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560b9e2cf400 0x560b9e2d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:33.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.137+0000 7f3975000640 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560b9e2cf400 0x560b9e2d1820 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929d953b84ecf7a2 server_cookie=b23e20ae50b181f7 in_seq=0 out_seq=0 2024-08-31T07:27:33.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.137+0000 7f395aa00640 10 client.28547 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:33.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.138+0000 7f395aa00640 1 -- 172.21.15.32:0/3716132892 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b9e1a91a0 con 0x560b9e2cf400 2024-08-31T07:27:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.138+0000 7f395aa00640 10 client.28547 handle_command_reply: tid=0 2024-08-31T07:27:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.138+0000 7f397e606740 1 client.28547 shutdown 2024-08-31T07:27:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.138+0000 7f397e606740 1 -- 172.21.15.32:0/3716132892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 msgr2=0x560b9e2564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.138+0000 7f397e606740 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 0x560b9e2564d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f395c00fb10 tx=0x7f395c024b70 comp rx=0 tx=0).stop 2024-08-31T07:27:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.139+0000 7f397e606740 1 -- 172.21.15.32:0/3716132892 shutdown_connections 2024-08-31T07:27:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.139+0000 7f397e606740 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560b9e2cf400 0x560b9e2d1820 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.139+0000 7f397e606740 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560b9de8f070 0x560b9e2c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.139+0000 7f397e606740 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b9de6e390 0x560b9e2564d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.139+0000 7f397e606740 1 --2- 172.21.15.32:0/3716132892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560b9de4d9f0 0x560b9e255ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.139+0000 7f395aa00640 0 client.28547 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.139+0000 7f397e606740 1 -- 172.21.15.32:0/3716132892 >> 172.21.15.32:0/3716132892 conn(0x560b9de83460 msgr2=0x560b9e240580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.139+0000 7f397e606740 1 -- 172.21.15.32:0/3716132892 shutdown_connections 2024-08-31T07:27:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.139+0000 7f397e606740 1 -- 172.21.15.32:0/3716132892 wait complete. 2024-08-31T07:27:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.140+0000 7f397e606740 20 client.28547 trim_cache size 0 max 16384 2024-08-31T07:27:33.141 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:27:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.141+0000 7f3977a00640 1 -- 172.21.15.32:0/1777985607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39540b4c30 msgr2=0x7f39540b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.141+0000 7f3977a00640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39540b4c30 0x7f39540b7140 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3960011d00 tx=0x7f3960000bc0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.141+0000 7f3977a00640 1 -- 172.21.15.32:0/1777985607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3970102540 msgr2=0x7f39701a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.141+0000 7f3977a00640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3970102540 0x7f39701a0890 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3968034040 tx=0x7f3968003c10 comp rx=0 tx=0).stop 2024-08-31T07:27:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.141+0000 7f3977a00640 1 -- 172.21.15.32:0/1777985607 shutdown_connections 2024-08-31T07:27:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.141+0000 7f3977a00640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f39540b4c30 0x7f39540b7140 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.141+0000 7f3977a00640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3970107980 0x7f39701a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.141+0000 7f3977a00640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3970102e90 0x7f39701a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.141+0000 7f3977a00640 1 --2- 172.21.15.32:0/1777985607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3970102540 0x7f39701a0890 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.141+0000 7f3977a00640 1 -- 172.21.15.32:0/1777985607 >> 172.21.15.32:0/1777985607 conn(0x7f39700794d0 msgr2=0x7f3970105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.142+0000 7f3977a00640 1 -- 172.21.15.32:0/1777985607 shutdown_connections 2024-08-31T07:27:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:33.142+0000 7f3977a00640 1 -- 172.21.15.32:0/1777985607 wait complete. 2024-08-31T07:27:33.330 INFO:tasks.workunit.client.0.smithi032.stdout: 10 25500 21.34 MB/sec warmup 73 sec latency 40.622 ms 2024-08-31T07:27:34.333 INFO:tasks.workunit.client.0.smithi032.stdout: 10 26230 21.44 MB/sec warmup 74 sec latency 65.187 ms 2024-08-31T07:27:34.521 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:34 smithi032 ceph-mon[17252]: pgmap v830: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 253 KiB/s wr, 48 op/s 2024-08-31T07:27:34.521 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:27:34.521 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:27:34.521 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:27:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:34 smithi073 ceph-mon[21995]: pgmap v830: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 253 KiB/s wr, 48 op/s 2024-08-31T07:27:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:27:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:27:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:27:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:34 smithi084 ceph-mon[22387]: pgmap v830: 97 pgs: 97 active+clean; 677 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 253 KiB/s wr, 48 op/s 2024-08-31T07:27:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:27:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:27:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:27:35.333 INFO:tasks.workunit.client.0.smithi032.stdout: 10 27538 21.99 MB/sec warmup 75 sec latency 65.607 ms 2024-08-31T07:27:35.339 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:27:35.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:27:35.948 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:36.333 INFO:tasks.workunit.client.0.smithi032.stdout: 10 28916 22.88 MB/sec warmup 76 sec latency 38.998 ms 2024-08-31T07:27:36.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:36 smithi032 ceph-mon[17252]: pgmap v831: 97 pgs: 97 active+clean; 683 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 819 KiB/s wr, 99 op/s 2024-08-31T07:27:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:36 smithi073 ceph-mon[21995]: pgmap v831: 97 pgs: 97 active+clean; 683 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 819 KiB/s wr, 99 op/s 2024-08-31T07:27:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:36 smithi084 ceph-mon[22387]: pgmap v831: 97 pgs: 97 active+clean; 683 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 819 KiB/s wr, 99 op/s 2024-08-31T07:27:37.334 INFO:tasks.workunit.client.0.smithi032.stdout: 10 29726 23.05 MB/sec warmup 77 sec latency 51.146 ms 2024-08-31T07:27:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.394+0000 7f548e600640 1 -- 172.21.15.32:0/2381886087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f548810b470 msgr2=0x7f548810b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.394+0000 7f548e600640 1 --2- 172.21.15.32:0/2381886087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f548810b470 0x7f548810b880 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5474009900 tx=0x7f547402a440 comp rx=0 tx=0).stop 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.395+0000 7f548e600640 1 -- 172.21.15.32:0/2381886087 shutdown_connections 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.395+0000 7f548e600640 1 --2- 172.21.15.32:0/2381886087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548810bdc0 0x7f54881125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.395+0000 7f548e600640 1 --2- 172.21.15.32:0/2381886087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f548810b470 0x7f548810b880 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.395+0000 7f548e600640 1 --2- 172.21.15.32:0/2381886087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488105b30 0x7f5488105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.395+0000 7f548e600640 1 -- 172.21.15.32:0/2381886087 >> 172.21.15.32:0/2381886087 conn(0x7f5488101600 msgr2=0x7f5488103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.396+0000 7f548e600640 1 -- 172.21.15.32:0/2381886087 shutdown_connections 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.396+0000 7f548e600640 1 -- 172.21.15.32:0/2381886087 wait complete. 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.398+0000 7f548e600640 1 Processor -- start 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.398+0000 7f548e600640 1 -- start start 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.401+0000 7f548e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488105b30 0x7f548810fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.402+0000 7f548e600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548810b470 0x7f548810ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.402+0000 7f5487400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488105b30 0x7f548810fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.402+0000 7f5487400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488105b30 0x7f548810fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49866/0 (socket says 172.21.15.32:49866) 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.403+0000 7f5487400640 1 -- 172.21.15.32:0/1333646273 learned_addr learned my addr 172.21.15.32:0/1333646273 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.403+0000 7f5486a00640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548810b470 0x7f548810ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.403+0000 7f548e600640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f548810bdc0 0x7f5488110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.403+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5488115ad0 con 0x7f5488105b30 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.403+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5488115950 con 0x7f548810bdc0 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.403+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5488115c50 con 0x7f548810b470 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.404+0000 7f5487e00640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f548810bdc0 0x7f5488110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.404+0000 7f5487400640 1 -- 172.21.15.32:0/1333646273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548810b470 msgr2=0x7f548810ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.404+0000 7f5487400640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548810b470 0x7f548810ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.404+0000 7f5487400640 1 -- 172.21.15.32:0/1333646273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f548810bdc0 msgr2=0x7f5488110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.404+0000 7f5487400640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f548810bdc0 0x7f5488110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.404+0000 7f5487400640 1 -- 172.21.15.32:0/1333646273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54881ac940 con 0x7f5488105b30 2024-08-31T07:27:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.404+0000 7f5487e00640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f548810bdc0 0x7f5488110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:27:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.405+0000 7f5487400640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488105b30 0x7f548810fa90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f547800e9f0 tx=0x7f547800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.405+0000 7f547fe00640 1 -- 172.21.15.32:0/1333646273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547800cd80 con 0x7f5488105b30 2024-08-31T07:27:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.405+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54881acc30 con 0x7f5488105b30 2024-08-31T07:27:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.405+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54881ad190 con 0x7f5488105b30 2024-08-31T07:27:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.406+0000 7f547fe00640 1 -- 172.21.15.32:0/1333646273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54780108e0 con 0x7f5488105b30 2024-08-31T07:27:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.406+0000 7f547fe00640 1 -- 172.21.15.32:0/1333646273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478018980 con 0x7f5488105b30 2024-08-31T07:27:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.412+0000 7f547fe00640 1 -- 172.21.15.32:0/1333646273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5478018b20 con 0x7f5488105b30 2024-08-31T07:27:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.413+0000 7f549528b740 1 Processor -- start 2024-08-31T07:27:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.414+0000 7f549528b740 1 -- start start 2024-08-31T07:27:37.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.415+0000 7f547fe00640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f546c0b4ca0 0x7f546c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.415+0000 7f547fe00640 1 -- 172.21.15.32:0/1333646273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f54780e56a0 con 0x7f5488105b30 2024-08-31T07:27:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.416+0000 7f5487e00640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f546c0b4ca0 0x7f546c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.416+0000 7f549528b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b40c350 0x55a03b7b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.416+0000 7f549528b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.416+0000 7f549528b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.416+0000 7f549528b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a03b728750 con 0x55a03b3eb670 2024-08-31T07:27:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.416+0000 7f549528b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a03b717ce0 con 0x55a03b40c350 2024-08-31T07:27:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.416+0000 7f549528b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a03b71bc90 con 0x55a03b3e3490 2024-08-31T07:27:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.417+0000 7f5487400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b40c350 0x55a03b7b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.417+0000 7f5487400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b40c350 0x55a03b7b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35890/0 (socket says 172.21.15.32:35890) 2024-08-31T07:27:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.417+0000 7f5487400640 1 -- 172.21.15.32:0/4178817361 learned_addr learned my addr 172.21.15.32:0/4178817361 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.417+0000 7f5486a00640 1 --2- 172.21.15.32:0/4178817361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.417+0000 7f5487400640 1 -- 172.21.15.32:0/4178817361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 msgr2=0x55a03b7b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.417+0000 7f5487400640 1 --2- 172.21.15.32:0/4178817361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.417+0000 7f5487400640 1 -- 172.21.15.32:0/4178817361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 msgr2=0x55a03b7b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.417+0000 7f5487400640 1 --2- 172.21.15.32:0/4178817361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.417+0000 7f5487400640 1 -- 172.21.15.32:0/4178817361 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03b477930 con 0x55a03b40c350 2024-08-31T07:27:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.418+0000 7f5487400640 1 --2- 172.21.15.32:0/4178817361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b40c350 0x55a03b7b2600 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f54780abd70 tx=0x7f54780ac620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e828a6c1e91e9d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.418+0000 7f547cc00640 1 -- 172.21.15.32:0/4178817361 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54780e30c0 con 0x55a03b40c350 2024-08-31T07:27:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.419+0000 7f547cc00640 1 -- 172.21.15.32:0/4178817361 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54780a8070 con 0x55a03b40c350 2024-08-31T07:27:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.419+0000 7f547cc00640 1 -- 172.21.15.32:0/4178817361 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54780ab040 con 0x55a03b40c350 2024-08-31T07:27:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.420+0000 7f549528b740 1 -- 172.21.15.32:0/4178817361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b40c350 msgr2=0x55a03b7b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.420+0000 7f549528b740 1 --2- 172.21.15.32:0/4178817361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b40c350 0x55a03b7b2600 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f54780abd70 tx=0x7f54780ac620 comp rx=0 tx=0).stop 2024-08-31T07:27:37.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.421+0000 7f549528b740 1 -- 172.21.15.32:0/4178817361 shutdown_connections 2024-08-31T07:27:37.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.421+0000 7f549528b740 1 --2- 172.21.15.32:0/4178817361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.421+0000 7f549528b740 1 --2- 172.21.15.32:0/4178817361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.421+0000 7f549528b740 1 --2- 172.21.15.32:0/4178817361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b40c350 0x55a03b7b2600 secure :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f54780abd70 tx=0x7f54780ac620 comp rx=0 tx=0).stop 2024-08-31T07:27:37.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.421+0000 7f549528b740 1 -- 172.21.15.32:0/4178817361 >> 172.21.15.32:0/4178817361 conn(0x55a03b400740 msgr2=0x55a03b7b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:37.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.421+0000 7f5487e00640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f546c0b4ca0 0x7f546c0b71b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5480005fd0 tx=0x7f5480009b30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.422+0000 7f549528b740 1 -- 172.21.15.32:0/4178817361 shutdown_connections 2024-08-31T07:27:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.422+0000 7f549528b740 1 -- 172.21.15.32:0/4178817361 wait complete. 2024-08-31T07:27:37.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.424+0000 7f549528b740 1 -- start start 2024-08-31T07:27:37.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.424+0000 7f549528b740 1 Processor -- start 2024-08-31T07:27:37.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.425+0000 7f549528b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.425+0000 7f549528b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7be870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.425+0000 7f549528b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.425+0000 7f549528b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a03b57a890 con 0x55a03b3eb670 2024-08-31T07:27:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.425+0000 7f549528b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a03b69ecd0 con 0x55a03b3cacd0 2024-08-31T07:27:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.425+0000 7f549528b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a03b7c0d80 con 0x55a03b3e3490 2024-08-31T07:27:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.426+0000 7f549528b740 10 client.0 _ll_register_callbacks cb 0x55a03b712ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.427+0000 7f5487e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.427+0000 7f5486a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7be870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.427+0000 7f5486a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7be870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49884/0 (socket says 172.21.15.32:49884) 2024-08-31T07:27:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.427+0000 7f5486a00640 1 -- 172.21.15.32:0/3023143519 learned_addr learned my addr 172.21.15.32:0/3023143519 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.427+0000 7f5487400640 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.428+0000 7f5487400640 1 -- 172.21.15.32:0/3023143519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 msgr2=0x55a03b7ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.428+0000 7f5487400640 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.429+0000 7f5487400640 1 -- 172.21.15.32:0/3023143519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 msgr2=0x55a03b7be870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.429+0000 7f5487400640 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7be870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.429+0000 7f5487400640 1 -- 172.21.15.32:0/3023143519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03b6e81b0 con 0x55a03b3e3490 2024-08-31T07:27:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.429+0000 7f5487e00640 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.429+0000 7f5487400640 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7be2b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f54780abd70 tx=0x7f54780b0440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=711927b11096c86b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.429+0000 7f5471600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:27:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.430+0000 7f5471600640 1 -- 172.21.15.32:0/3023143519 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478010430 con 0x55a03b3e3490 2024-08-31T07:27:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.430+0000 7f5471600640 1 -- 172.21.15.32:0/3023143519 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f54780105d0 con 0x55a03b3e3490 2024-08-31T07:27:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.430+0000 7f549528b740 1 -- 172.21.15.32:0/3023143519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54780a8070 con 0x55a03b3e3490 2024-08-31T07:27:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.431+0000 7f5471600640 1 -- 172.21.15.32:0/3023143519 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f54780ab040 con 0x55a03b3e3490 2024-08-31T07:27:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.431+0000 7f549528b740 10 client.28549 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.431+0000 7f549528b740 1 -- 172.21.15.32:0/3023143519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a03b7cce50 con 0x55a03b3e3490 2024-08-31T07:27:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.431+0000 7f5471600640 1 -- 172.21.15.32:0/3023143519 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54780a8070 con 0x55a03b3e3490 2024-08-31T07:27:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.432+0000 7f5471600640 1 -- 172.21.15.32:0/3023143519 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f54780aaab0 con 0x55a03b3e3490 2024-08-31T07:27:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.432+0000 7f549528b740 10 client.28549 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.432+0000 7f549528b740 10 client.28549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:27:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.432+0000 7f549528b740 20 client.28549 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.432+0000 7f549528b740 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a03b3a0080 0x55a03b7d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.432+0000 7f549528b740 4 client.28549 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:27:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.432+0000 7f549528b740 1 -- 172.21.15.32:0/3023143519 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a03b72a6a0 con 0x55a03b3a0080 2024-08-31T07:27:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.432+0000 7f5486a00640 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a03b3a0080 0x55a03b7d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.433+0000 7f5471600640 10 client.28549 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:37.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.433+0000 7f5486a00640 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a03b3a0080 0x55a03b7d01b0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f9b6622a853766 server_cookie=380cf0d552fb9b02 in_seq=0 out_seq=0 2024-08-31T07:27:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.437+0000 7f5471600640 1 -- 172.21.15.32:0/3023143519 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a03b72a6a0 con 0x55a03b3a0080 2024-08-31T07:27:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.437+0000 7f5471600640 10 client.28549 handle_command_reply: tid=0 2024-08-31T07:27:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.437+0000 7f549528b740 1 client.28549 shutdown 2024-08-31T07:27:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.438+0000 7f549528b740 1 -- 172.21.15.32:0/3023143519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 msgr2=0x55a03b7be2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.438+0000 7f549528b740 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7be2b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f54780abd70 tx=0x7f54780b0440 comp rx=0 tx=0).stop 2024-08-31T07:27:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.438+0000 7f549528b740 1 -- 172.21.15.32:0/3023143519 shutdown_connections 2024-08-31T07:27:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.438+0000 7f549528b740 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a03b3a0080 0x55a03b7d01b0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.438+0000 7f5471600640 0 client.28549 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.438+0000 7f549528b740 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.438+0000 7f549528b740 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7be870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.438+0000 7f549528b740 1 --2- 172.21.15.32:0/3023143519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7be2b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.438+0000 7f549528b740 1 -- 172.21.15.32:0/3023143519 >> 172.21.15.32:0/3023143519 conn(0x55a03b400740 msgr2=0x55a03b7b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.439+0000 7f549528b740 1 -- 172.21.15.32:0/3023143519 shutdown_connections 2024-08-31T07:27:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.439+0000 7f549528b740 1 -- 172.21.15.32:0/3023143519 wait complete. 2024-08-31T07:27:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.440+0000 7f549528b740 20 client.28549 trim_cache size 0 max 16384 2024-08-31T07:27:37.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.465+0000 7f549528b740 1 Processor -- start 2024-08-31T07:27:37.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.465+0000 7f549528b740 1 -- start start 2024-08-31T07:27:37.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f549528b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f549528b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f549528b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f549528b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f546800e660 con 0x55a03b3eb670 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f549528b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5468010380 con 0x55a03b3cacd0 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f549528b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f546800ed60 con 0x55a03b3e3490 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f5486a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f5487400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f5487400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35904/0 (socket says 172.21.15.32:35904) 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f5487400640 1 -- 172.21.15.32:0/2552849222 learned_addr learned my addr 172.21.15.32:0/2552849222 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f5486a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37104/0 (socket says 172.21.15.32:37104) 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.466+0000 7f5487e00640 1 --2- 172.21.15.32:0/2552849222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.467+0000 7f5487400640 1 -- 172.21.15.32:0/2552849222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 msgr2=0x55a03b7b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:37.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.467+0000 7f5487400640 1 --2- 172.21.15.32:0/2552849222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.467+0000 7f5487400640 1 -- 172.21.15.32:0/2552849222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 msgr2=0x55a03b7afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:37.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.467+0000 7f5487400640 1 --2- 172.21.15.32:0/2552849222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.467+0000 7f5487400640 1 -- 172.21.15.32:0/2552849222 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03b7b48f0 con 0x55a03b3cacd0 2024-08-31T07:27:37.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.467+0000 7f5486a00640 1 --2- 172.21.15.32:0/2552849222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:27:37.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.467+0000 7f5487400640 1 --2- 172.21.15.32:0/2552849222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7b3630 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f547800c6a0 tx=0x7f54780abce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f06e6e61826b9571 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:37.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.467+0000 7f5473e00640 1 -- 172.21.15.32:0/2552849222 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54780a7ca0 con 0x55a03b3cacd0 2024-08-31T07:27:37.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.468+0000 7f5473e00640 1 -- 172.21.15.32:0/2552849222 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54780a83d0 con 0x55a03b3cacd0 2024-08-31T07:27:37.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.468+0000 7f5473e00640 1 -- 172.21.15.32:0/2552849222 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54780aad10 con 0x55a03b3cacd0 2024-08-31T07:27:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.469+0000 7f549528b740 1 -- 172.21.15.32:0/2552849222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 msgr2=0x55a03b7b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.469+0000 7f549528b740 1 --2- 172.21.15.32:0/2552849222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7b3630 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f547800c6a0 tx=0x7f54780abce0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.469+0000 7f549528b740 1 -- 172.21.15.32:0/2552849222 shutdown_connections 2024-08-31T07:27:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.469+0000 7f549528b740 1 --2- 172.21.15.32:0/2552849222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b7afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.469+0000 7f549528b740 1 --2- 172.21.15.32:0/2552849222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3e3490 0x55a03b7b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.469+0000 7f549528b740 1 --2- 172.21.15.32:0/2552849222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3cacd0 0x55a03b7b3630 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.469+0000 7f549528b740 1 -- 172.21.15.32:0/2552849222 >> 172.21.15.32:0/2552849222 conn(0x55a03b400740 msgr2=0x55a03b7bc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.469+0000 7f549528b740 1 -- 172.21.15.32:0/2552849222 shutdown_connections 2024-08-31T07:27:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.470+0000 7f549528b740 1 -- 172.21.15.32:0/2552849222 wait complete. 2024-08-31T07:27:37.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.470+0000 7f549528b740 1 -- start start 2024-08-31T07:27:37.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.471+0000 7f549528b740 1 Processor -- start 2024-08-31T07:27:37.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.471+0000 7f549528b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3cacd0 0x55a03b7cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.471+0000 7f549528b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3e3490 0x55a03b7d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.471+0000 7f549528b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b8339d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.472+0000 7f549528b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f546800fbf0 con 0x55a03b3eb670 2024-08-31T07:27:37.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.472+0000 7f549528b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f546800f4c0 con 0x55a03b3e3490 2024-08-31T07:27:37.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.472+0000 7f549528b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f546800c660 con 0x55a03b3cacd0 2024-08-31T07:27:37.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.472+0000 7f5487400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3cacd0 0x55a03b7cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.472+0000 7f5487400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3cacd0 0x55a03b7cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37110/0 (socket says 172.21.15.32:37110) 2024-08-31T07:27:37.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.472+0000 7f5487400640 1 -- 172.21.15.32:0/700456334 learned_addr learned my addr 172.21.15.32:0/700456334 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:37.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.472+0000 7f5487e00640 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b8339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.472+0000 7f5486a00640 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3e3490 0x55a03b7d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.473+0000 7f549528b740 10 client.0 _ll_register_callbacks cb 0x7f54780ab9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.473+0000 7f5487400640 1 -- 172.21.15.32:0/700456334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3e3490 msgr2=0x55a03b7d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.473+0000 7f5487400640 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3e3490 0x55a03b7d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.473+0000 7f5487400640 1 -- 172.21.15.32:0/700456334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 msgr2=0x55a03b8339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.473+0000 7f5487400640 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b8339d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.473+0000 7f5487400640 1 -- 172.21.15.32:0/700456334 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03b834040 con 0x55a03b3cacd0 2024-08-31T07:27:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.474+0000 7f5487400640 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3cacd0 0x55a03b7cfb80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f54780abc30 tx=0x7f5478010b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb511969a57563f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:37.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.474+0000 7f5471600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:27:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.474+0000 7f5471600640 1 -- 172.21.15.32:0/700456334 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54780a7ca0 con 0x55a03b3cacd0 2024-08-31T07:27:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.474+0000 7f5471600640 1 -- 172.21.15.32:0/700456334 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f54780c6b90 con 0x55a03b3cacd0 2024-08-31T07:27:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.474+0000 7f549528b740 1 -- 172.21.15.32:0/700456334 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54780a83d0 con 0x55a03b3cacd0 2024-08-31T07:27:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f5471600640 1 -- 172.21.15.32:0/700456334 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f54780ca020 con 0x55a03b3cacd0 2024-08-31T07:27:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f549528b740 10 client.28555 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f549528b740 1 -- 172.21.15.32:0/700456334 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a03b834040 con 0x55a03b3cacd0 2024-08-31T07:27:37.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f5471600640 1 -- 172.21.15.32:0/700456334 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54780a83d0 con 0x55a03b3cacd0 2024-08-31T07:27:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f5471600640 1 -- 172.21.15.32:0/700456334 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f54780e5080 con 0x55a03b3cacd0 2024-08-31T07:27:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f549528b740 10 client.28555 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f549528b740 10 client.28555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:27:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f549528b740 20 client.28555 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f549528b740 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a03b83b680 0x55a03b83daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f549528b740 4 client.28555 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:27:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.475+0000 7f549528b740 1 -- 172.21.15.32:0/700456334 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a03b726140 con 0x55a03b83b680 2024-08-31T07:27:37.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.476+0000 7f5486a00640 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a03b83b680 0x55a03b83daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:37.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.477+0000 7f5486a00640 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a03b83b680 0x55a03b83daa0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96079425bf02aa6c server_cookie=27a6c7adce642c9f in_seq=0 out_seq=0 2024-08-31T07:27:37.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.477+0000 7f5471600640 10 client.28555 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:37.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.479+0000 7f5471600640 1 -- 172.21.15.32:0/700456334 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a03b726140 con 0x55a03b83b680 2024-08-31T07:27:37.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.479+0000 7f5471600640 10 client.28555 handle_command_reply: tid=0 2024-08-31T07:27:37.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.479+0000 7f549528b740 1 client.28555 shutdown 2024-08-31T07:27:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.479+0000 7f549528b740 1 -- 172.21.15.32:0/700456334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3cacd0 msgr2=0x55a03b7cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.479+0000 7f549528b740 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3cacd0 0x55a03b7cfb80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f54780abc30 tx=0x7f5478010b30 comp rx=0 tx=0).stop 2024-08-31T07:27:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.480+0000 7f549528b740 1 -- 172.21.15.32:0/700456334 shutdown_connections 2024-08-31T07:27:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.480+0000 7f549528b740 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a03b83b680 0x55a03b83daa0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.480+0000 7f549528b740 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a03b3eb670 0x55a03b8339d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.480+0000 7f549528b740 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a03b3e3490 0x55a03b7d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.480+0000 7f549528b740 1 --2- 172.21.15.32:0/700456334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a03b3cacd0 0x55a03b7cfb80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.480+0000 7f549528b740 1 -- 172.21.15.32:0/700456334 >> 172.21.15.32:0/700456334 conn(0x55a03b400740 msgr2=0x55a03b7afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:37.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.480+0000 7f5471600640 0 client.28555 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:37.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.480+0000 7f549528b740 1 -- 172.21.15.32:0/700456334 shutdown_connections 2024-08-31T07:27:37.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.480+0000 7f549528b740 1 -- 172.21.15.32:0/700456334 wait complete. 2024-08-31T07:27:37.482 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:27:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:27:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "d4942a56-ab98-4c7e-922a-46e26db5d6ed", 2024-08-31T07:27:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:27:37.482 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:27:37.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.480+0000 7f549528b740 20 client.28555 trim_cache size 0 max 16384 2024-08-31T07:27:37.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.481+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f546c0b4ca0 msgr2=0x7f546c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:37.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.481+0000 7f548e600640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f546c0b4ca0 0x7f546c0b71b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5480005fd0 tx=0x7f5480009b30 comp rx=0 tx=0).stop 2024-08-31T07:27:37.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.481+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488105b30 msgr2=0x7f548810fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:37.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.481+0000 7f548e600640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488105b30 0x7f548810fa90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f547800e9f0 tx=0x7f547800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.482+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 shutdown_connections 2024-08-31T07:27:37.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.482+0000 7f548e600640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f546c0b4ca0 0x7f546c0b71b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.482+0000 7f548e600640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f548810bdc0 0x7f5488110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.482+0000 7f548e600640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548810b470 0x7f548810ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.482+0000 7f548e600640 1 --2- 172.21.15.32:0/1333646273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488105b30 0x7f548810fa90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:37.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.482+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 >> 172.21.15.32:0/1333646273 conn(0x7f5488101600 msgr2=0x7f5488108aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:37.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.483+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 shutdown_connections 2024-08-31T07:27:37.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:37.483+0000 7f548e600640 1 -- 172.21.15.32:0/1333646273 wait complete. 2024-08-31T07:27:38.334 INFO:tasks.workunit.client.0.smithi032.stdout: 10 30508 23.27 MB/sec warmup 78 sec latency 237.030 ms 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: pgmap v832: 97 pgs: 97 active+clean; 708 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:38 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: pgmap v832: 97 pgs: 97 active+clean; 708 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-08-31T07:27:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:27:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:27:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:27:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:27:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:27:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:27:38.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:38 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: pgmap v832: 97 pgs: 97 active+clean; 708 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-08-31T07:27:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:27:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:27:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:27:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:27:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:27:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:27:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:38 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:39.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:27:39.334 INFO:tasks.workunit.client.0.smithi032.stdout: 10 31874 24.02 MB/sec warmup 79 sec latency 37.724 ms 2024-08-31T07:27:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:39 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:39.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:39 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:39.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:39 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:39.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:39 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:39.679 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:27:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:39 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:39 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:27:40.334 INFO:tasks.workunit.client.0.smithi032.stdout: 10 32686 24.24 MB/sec warmup 80 sec latency 68.373 ms 2024-08-31T07:27:40.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:40 smithi032 ceph-mon[17252]: pgmap v833: 97 pgs: 97 active+clean; 710 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-08-31T07:27:40.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.585+0000 7f80f5000640 1 -- 172.21.15.32:0/4132490221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80f01101e0 msgr2=0x7f80f0116a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:40.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.585+0000 7f80f5000640 1 --2- 172.21.15.32:0/4132490221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80f01101e0 0x7f80f0116a10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009be0 tx=0x7f80e802a6c0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.585+0000 7f80f5000640 1 -- 172.21.15.32:0/4132490221 shutdown_connections 2024-08-31T07:27:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.585+0000 7f80f5000640 1 --2- 172.21.15.32:0/4132490221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80f01101e0 0x7f80f0116a10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.585+0000 7f80f5000640 1 --2- 172.21.15.32:0/4132490221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f010f890 0x7f80f010fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.585+0000 7f80f5000640 1 --2- 172.21.15.32:0/4132490221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80f0109f50 0x7f80f010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.585+0000 7f80f5000640 1 -- 172.21.15.32:0/4132490221 >> 172.21.15.32:0/4132490221 conn(0x7f80f0077e80 msgr2=0x7f80f0107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.585+0000 7f80f5000640 1 -- 172.21.15.32:0/4132490221 shutdown_connections 2024-08-31T07:27:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.585+0000 7f80f5000640 1 -- 172.21.15.32:0/4132490221 wait complete. 2024-08-31T07:27:40.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.588+0000 7f80f5000640 1 Processor -- start 2024-08-31T07:27:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.589+0000 7f80f5000640 1 -- start start 2024-08-31T07:27:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.590+0000 7f80f5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f0109f50 0x7f80f01a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.590+0000 7f80f5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80f010f890 0x7f80f01a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.590+0000 7f80f5000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80f01101e0 0x7f80f01a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.590+0000 7f80f5000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0119ea0 con 0x7f80f0109f50 2024-08-31T07:27:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.590+0000 7f80f5000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0119d20 con 0x7f80f010f890 2024-08-31T07:27:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.590+0000 7f80f5000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f80f011a020 con 0x7f80f01101e0 2024-08-31T07:27:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.591+0000 7f80ee800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80f01101e0 0x7f80f01a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.591+0000 7f80ee800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80f01101e0 0x7f80f01a6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37122/0 (socket says 172.21.15.32:37122) 2024-08-31T07:27:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.591+0000 7f80ee800640 1 -- 172.21.15.32:0/1325624071 learned_addr learned my addr 172.21.15.32:0/1325624071 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.592+0000 7f80ed400640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80f010f890 0x7f80f01a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.592+0000 7f80ee800640 1 -- 172.21.15.32:0/1325624071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80f010f890 msgr2=0x7f80f01a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.592+0000 7f80ee800640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80f010f890 0x7f80f01a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.592+0000 7f80ee800640 1 -- 172.21.15.32:0/1325624071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f0109f50 msgr2=0x7f80f01a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.592+0000 7f80ee800640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f0109f50 0x7f80f01a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.592+0000 7f80ee800640 1 -- 172.21.15.32:0/1325624071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f01a6780 con 0x7f80f01101e0 2024-08-31T07:27:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.592+0000 7f80ed400640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80f010f890 0x7f80f01a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:27:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.593+0000 7f80ee800640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80f01101e0 0x7f80f01a6110 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f80e8003cc0 tx=0x7f80e8003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.594+0000 7f80e6a00640 1 -- 172.21.15.32:0/1325624071 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8040020 con 0x7f80f01101e0 2024-08-31T07:27:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.594+0000 7f80e6a00640 1 -- 172.21.15.32:0/1325624071 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e8003e10 con 0x7f80f01101e0 2024-08-31T07:27:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.594+0000 7f80e6a00640 1 -- 172.21.15.32:0/1325624071 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8036040 con 0x7f80f01101e0 2024-08-31T07:27:40.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.596+0000 7f80f5000640 1 -- 172.21.15.32:0/1325624071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f01a6a10 con 0x7f80f01101e0 2024-08-31T07:27:40.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.597+0000 7f80f5000640 1 -- 172.21.15.32:0/1325624071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f01a6ea0 con 0x7f80f01101e0 2024-08-31T07:27:40.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.604+0000 7f80e6a00640 1 -- 172.21.15.32:0/1325624071 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f80e8043020 con 0x7f80f01101e0 2024-08-31T07:27:40.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.607+0000 7f80fbc5e740 1 Processor -- start 2024-08-31T07:27:40.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.608+0000 7f80e6a00640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f80d40b4e10 0x7f80d40b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.608+0000 7f80fbc5e740 1 -- start start 2024-08-31T07:27:40.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.608+0000 7f80e6a00640 1 -- 172.21.15.32:0/1325624071 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f80e81060f0 con 0x7f80f01101e0 2024-08-31T07:27:40.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.609+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.609+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f281b0 0x560f662f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.609+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f30390 0x560f662f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.609+0000 7f80ed400640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f80d40b4e10 0x7f80d40b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.610+0000 7f80ede00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.610+0000 7f80ede00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35942/0 (socket says 172.21.15.32:35942) 2024-08-31T07:27:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.610+0000 7f80ede00640 1 -- 172.21.15.32:0/1623094266 learned_addr learned my addr 172.21.15.32:0/1623094266 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.610+0000 7f80ee800640 1 --2- 172.21.15.32:0/1623094266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f30390 0x560f662f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.610+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1623094266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560f6626d770 con 0x560f65f281b0 2024-08-31T07:27:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.610+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1623094266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560f6625cd00 con 0x560f65f51070 2024-08-31T07:27:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.610+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1623094266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560f66260cb0 con 0x560f65f30390 2024-08-31T07:27:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.612+0000 7f80ed400640 1 --2- 172.21.15.32:0/1623094266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f281b0 0x560f662f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.613+0000 7f80ed400640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f80d40b4e10 0x7f80d40b7320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f80f006ae80 tx=0x7f80e000aab0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.614+0000 7f80ee800640 1 -- 172.21.15.32:0/1623094266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 msgr2=0x560f662f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.614+0000 7f80ee800640 1 --2- 172.21.15.32:0/1623094266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.614+0000 7f80ee800640 1 -- 172.21.15.32:0/1623094266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f281b0 msgr2=0x560f662f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.614+0000 7f80ee800640 1 --2- 172.21.15.32:0/1623094266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f281b0 0x560f662f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.615+0000 7f80ee800640 1 -- 172.21.15.32:0/1623094266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f65fbc650 con 0x560f65f30390 2024-08-31T07:27:40.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.616+0000 7f80ee800640 1 --2- 172.21.15.32:0/1623094266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f30390 0x560f662f73d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f80e8103830 tx=0x7f80e80cc230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eebcc3b21ab86e79 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:40.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.617+0000 7f80db400640 1 -- 172.21.15.32:0/1623094266 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80d5420 con 0x560f65f30390 2024-08-31T07:27:40.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.617+0000 7f80db400640 1 -- 172.21.15.32:0/1623094266 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e80cc7d0 con 0x560f65f30390 2024-08-31T07:27:40.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.617+0000 7f80db400640 1 -- 172.21.15.32:0/1623094266 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80d5660 con 0x560f65f30390 2024-08-31T07:27:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.619+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1623094266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f30390 msgr2=0x560f662f73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.619+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/1623094266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f30390 0x560f662f73d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f80e8103830 tx=0x7f80e80cc230 comp rx=0 tx=0).stop 2024-08-31T07:27:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.619+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1623094266 shutdown_connections 2024-08-31T07:27:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.619+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/1623094266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f30390 0x560f662f73d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.619+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/1623094266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f281b0 0x560f662f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.619+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/1623094266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.619+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1623094266 >> 172.21.15.32:0/1623094266 conn(0x560f65f45460 msgr2=0x560f662f4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.620+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1623094266 shutdown_connections 2024-08-31T07:27:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.620+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1623094266 wait complete. 2024-08-31T07:27:40.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.622+0000 7f80fbc5e740 1 -- start start 2024-08-31T07:27:40.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.622+0000 7f80fbc5e740 1 Processor -- start 2024-08-31T07:27:40.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f281b0 0x560f663025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f30390 0x560f66302b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f51070 0x560f663080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80fbc5e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560f660bf700 con 0x560f65f30390 2024-08-31T07:27:40.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80fbc5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560f661e3cb0 con 0x560f65f281b0 2024-08-31T07:27:40.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80fbc5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560f66304ef0 con 0x560f65f51070 2024-08-31T07:27:40.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80fbc5e740 10 client.0 _ll_register_callbacks cb 0x560f66209430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:40.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80ede00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f281b0 0x560f663025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80ede00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f281b0 0x560f663025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35946/0 (socket says 172.21.15.32:35946) 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80ede00640 1 -- 172.21.15.32:0/2701078624 learned_addr learned my addr 172.21.15.32:0/2701078624 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.625+0000 7f80ee800640 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f51070 0x560f663080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.626+0000 7f80ede00640 1 -- 172.21.15.32:0/2701078624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f51070 msgr2=0x560f663080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.626+0000 7f80ede00640 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f51070 0x560f663080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.626+0000 7f80ede00640 1 -- 172.21.15.32:0/2701078624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f30390 msgr2=0x560f66302b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.626+0000 7f80ede00640 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f30390 0x560f66302b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.626+0000 7f80ede00640 1 -- 172.21.15.32:0/2701078624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f66224870 con 0x560f65f281b0 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.626+0000 7f80ede00640 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f281b0 0x560f663025b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0116f0 tx=0x7f80dc011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98d3ba7e85a50c5f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.626+0000 7f80cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.627+0000 7f80cfe00640 1 -- 172.21.15.32:0/2701078624 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00fd30 con 0x560f65f281b0 2024-08-31T07:27:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.627+0000 7f80cfe00640 1 -- 172.21.15.32:0/2701078624 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f80dc005020 con 0x560f65f281b0 2024-08-31T07:27:40.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.629+0000 7f80fbc5e740 1 -- 172.21.15.32:0/2701078624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e80cc7d0 con 0x560f65f281b0 2024-08-31T07:27:40.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.629+0000 7f80cfe00640 1 -- 172.21.15.32:0/2701078624 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f80dc005220 con 0x560f65f281b0 2024-08-31T07:27:40.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.629+0000 7f80fbc5e740 10 client.28577 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:40.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.630+0000 7f80fbc5e740 1 -- 172.21.15.32:0/2701078624 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560f66321fe0 con 0x560f65f281b0 2024-08-31T07:27:40.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.630+0000 7f80cfe00640 1 -- 172.21.15.32:0/2701078624 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e80cc7d0 con 0x560f65f281b0 2024-08-31T07:27:40.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.631+0000 7f80cfe00640 1 -- 172.21.15.32:0/2701078624 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f80dc019430 con 0x560f65f281b0 2024-08-31T07:27:40.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.632+0000 7f80fbc5e740 10 client.28577 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:40.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.632+0000 7f80fbc5e740 10 client.28577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:27:40.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.632+0000 7f80fbc5e740 20 client.28577 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:40.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.632+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560f65f0f9f0 0x560f663255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.632+0000 7f80fbc5e740 4 client.28577 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:27:40.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.632+0000 7f80fbc5e740 1 -- 172.21.15.32:0/2701078624 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f6626f6c0 con 0x560f65f0f9f0 2024-08-31T07:27:40.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.632+0000 7f80ed400640 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560f65f0f9f0 0x560f663255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.633+0000 7f80cfe00640 10 client.28577 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:40.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.634+0000 7f80ed400640 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560f65f0f9f0 0x560f663255b0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9accf0d39e7c839d server_cookie=82f5a4de8e61d5b1 in_seq=0 out_seq=0 2024-08-31T07:27:40.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.635+0000 7f80cfe00640 1 -- 172.21.15.32:0/2701078624 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f6626f6c0 con 0x560f65f0f9f0 2024-08-31T07:27:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.635+0000 7f80cfe00640 10 client.28577 handle_command_reply: tid=0 2024-08-31T07:27:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 client.28577 shutdown 2024-08-31T07:27:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 -- 172.21.15.32:0/2701078624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f281b0 msgr2=0x560f663025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f281b0 0x560f663025b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0116f0 tx=0x7f80dc011bb0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 -- 172.21.15.32:0/2701078624 shutdown_connections 2024-08-31T07:27:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560f65f0f9f0 0x560f663255b0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f51070 0x560f663080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f30390 0x560f66302b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/2701078624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f281b0 0x560f663025b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 -- 172.21.15.32:0/2701078624 >> 172.21.15.32:0/2701078624 conn(0x560f65f45460 msgr2=0x560f662f7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:40.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80cfe00640 0 client.28577 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:40.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 -- 172.21.15.32:0/2701078624 shutdown_connections 2024-08-31T07:27:40.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.636+0000 7f80fbc5e740 1 -- 172.21.15.32:0/2701078624 wait complete. 2024-08-31T07:27:40.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.637+0000 7f80fbc5e740 20 client.28577 trim_cache size 0 max 16384 2024-08-31T07:27:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.674+0000 7f80fbc5e740 1 Processor -- start 2024-08-31T07:27:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.674+0000 7f80fbc5e740 1 -- start start 2024-08-31T07:27:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.674+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f662fc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.674+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f30390 0x560f662f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.674+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.674+0000 7f80fbc5e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01ad290 con 0x560f65f30390 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.674+0000 7f80fbc5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01aef80 con 0x560f65f51070 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.674+0000 7f80fbc5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01ad960 con 0x560f65f281b0 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.675+0000 7f80ee800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.675+0000 7f80ee800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35962/0 (socket says 172.21.15.32:35962) 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.675+0000 7f80ee800640 1 -- 172.21.15.32:0/1347668784 learned_addr learned my addr 172.21.15.32:0/1347668784 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.675+0000 7f80ede00640 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f662fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.676+0000 7f80ee800640 1 -- 172.21.15.32:0/1347668784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 msgr2=0x560f662fc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.676+0000 7f80ee800640 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f662fc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.676+0000 7f80ee800640 1 -- 172.21.15.32:0/1347668784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f30390 msgr2=0x560f662f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.676+0000 7f80ed400640 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f30390 0x560f662f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.676+0000 7f80ee800640 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f30390 0x560f662f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.676+0000 7f80ee800640 1 -- 172.21.15.32:0/1347668784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f66301310 con 0x560f65f51070 2024-08-31T07:27:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.676+0000 7f80ede00640 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f662fc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:27:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:40 smithi073 ceph-mon[21995]: pgmap v833: 97 pgs: 97 active+clean; 710 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-08-31T07:27:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.677+0000 7f80ed400640 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f30390 0x560f662f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:27:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.677+0000 7f80ee800640 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f4330 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f80e8004af0 tx=0x7f80e802a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611c90eb306bd0c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.678+0000 7f80daa00640 1 -- 172.21.15.32:0/1347668784 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80d5420 con 0x560f65f51070 2024-08-31T07:27:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.678+0000 7f80daa00640 1 -- 172.21.15.32:0/1347668784 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e80d5af0 con 0x560f65f51070 2024-08-31T07:27:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.679+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1347668784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 msgr2=0x560f662f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.679+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f4330 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f80e8004af0 tx=0x7f80e802a6c0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.679+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1347668784 shutdown_connections 2024-08-31T07:27:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.679+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f51070 0x560f662f4330 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.679+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f30390 0x560f662f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.679+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/1347668784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f662fc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.679+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1347668784 >> 172.21.15.32:0/1347668784 conn(0x560f65f45460 msgr2=0x560f662fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.680+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1347668784 shutdown_connections 2024-08-31T07:27:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.680+0000 7f80fbc5e740 1 -- 172.21.15.32:0/1347668784 wait complete. 2024-08-31T07:27:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.681+0000 7f80fbc5e740 1 -- start start 2024-08-31T07:27:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.681+0000 7f80fbc5e740 1 Processor -- start 2024-08-31T07:27:40.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.682+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f6630e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.682+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f30390 0x560f6630aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.682+0000 7f80fbc5e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f51070 0x560f6630afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.682+0000 7f80fbc5e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01ae7f0 con 0x560f65f51070 2024-08-31T07:27:40.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.682+0000 7f80fbc5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01ae0c0 con 0x560f65f30390 2024-08-31T07:27:40.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.682+0000 7f80fbc5e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0082150 con 0x560f65f281b0 2024-08-31T07:27:40.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.682+0000 7f80fbc5e740 10 client.0 _ll_register_callbacks cb 0x7f80dc009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:27:40.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.683+0000 7f80ede00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f6630e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.683+0000 7f80ed400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f30390 0x560f6630aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.683+0000 7f80ede00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f6630e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37154/0 (socket says 172.21.15.32:37154) 2024-08-31T07:27:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.683+0000 7f80ede00640 1 -- 172.21.15.32:0/3081317128 learned_addr learned my addr 172.21.15.32:0/3081317128 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:27:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.683+0000 7f80ee800640 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f51070 0x560f6630afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.683+0000 7f80ede00640 1 -- 172.21.15.32:0/3081317128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f30390 msgr2=0x560f6630aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.683+0000 7f80ede00640 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f30390 0x560f6630aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.683+0000 7f80ede00640 1 -- 172.21.15.32:0/3081317128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f51070 msgr2=0x560f6630afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:27:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.683+0000 7f80ede00640 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f51070 0x560f6630afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.684+0000 7f80ede00640 1 -- 172.21.15.32:0/3081317128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f6630b610 con 0x560f65f281b0 2024-08-31T07:27:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.684+0000 7f80ede00640 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f6630e550 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f80dc019850 tx=0x7f80dc019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdab5700810f3763 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:27:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.684+0000 7f80cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:27:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.684+0000 7f80cfe00640 1 -- 172.21.15.32:0/3081317128 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc031020 con 0x560f65f281b0 2024-08-31T07:27:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.684+0000 7f80cfe00640 1 -- 172.21.15.32:0/3081317128 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8441+0+0 (secure 0 0 0) 0x7f80dc00fb30 con 0x560f65f281b0 2024-08-31T07:27:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.685+0000 7f80fbc5e740 1 -- 172.21.15.32:0/3081317128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e80d5af0 con 0x560f65f281b0 2024-08-31T07:27:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.685+0000 7f80cfe00640 1 -- 172.21.15.32:0/3081317128 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f80dc005220 con 0x560f65f281b0 2024-08-31T07:27:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.685+0000 7f80fbc5e740 10 client.28573 fetch_fsmap learned FSMap version 55 2024-08-31T07:27:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.685+0000 7f80fbc5e740 1 -- 172.21.15.32:0/3081317128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x560f66390d40 con 0x560f65f281b0 2024-08-31T07:27:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.686+0000 7f80cfe00640 1 -- 172.21.15.32:0/3081317128 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e80d5af0 con 0x560f65f281b0 2024-08-31T07:27:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.686+0000 7f80cfe00640 1 -- 172.21.15.32:0/3081317128 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7f80dc0039a0 con 0x560f65f281b0 2024-08-31T07:27:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.686+0000 7f80fbc5e740 10 client.28573 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.686+0000 7f80fbc5e740 10 client.28573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.686+0000 7f80fbc5e740 20 client.28573 populate_metadata read hostname 'smithi032' 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.686+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560f65ee4da0 0x560f66393640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.686+0000 7f80fbc5e740 4 client.28573 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.686+0000 7f80fbc5e740 1 -- 172.21.15.32:0/3081317128 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f6626b160 con 0x560f65ee4da0 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.687+0000 7f80ed400640 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560f65ee4da0 0x560f66393640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.687+0000 7f80ed400640 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560f65ee4da0 0x560f66393640 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188d47576458c0a1 server_cookie=36a2a62486096f64 in_seq=0 out_seq=0 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.687+0000 7f80cfe00640 10 client.28573 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80cfe00640 1 -- 172.21.15.32:0/3081317128 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x560f6626b160 con 0x560f65ee4da0 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80cfe00640 10 client.28573 handle_command_reply: tid=0 2024-08-31T07:27:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80fbc5e740 1 client.28573 shutdown 2024-08-31T07:27:40.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80fbc5e740 1 -- 172.21.15.32:0/3081317128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 msgr2=0x560f6630e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:40.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f6630e550 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f80dc019850 tx=0x7f80dc019660 comp rx=0 tx=0).stop 2024-08-31T07:27:40.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80fbc5e740 1 -- 172.21.15.32:0/3081317128 shutdown_connections 2024-08-31T07:27:40.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560f65ee4da0 0x560f66393640 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f65f51070 0x560f6630afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560f65f30390 0x560f6630aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80fbc5e740 1 --2- 172.21.15.32:0/3081317128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560f65f281b0 0x560f6630e550 secure :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f80dc019850 tx=0x7f80dc019660 comp rx=0 tx=0).stop 2024-08-31T07:27:40.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80fbc5e740 1 -- 172.21.15.32:0/3081317128 >> 172.21.15.32:0/3081317128 conn(0x560f65f45460 msgr2=0x560f662f7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:40.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.688+0000 7f80cfe00640 0 client.28573 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:27:40.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.689+0000 7f80fbc5e740 1 -- 172.21.15.32:0/3081317128 shutdown_connections 2024-08-31T07:27:40.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.689+0000 7f80fbc5e740 1 -- 172.21.15.32:0/3081317128 wait complete. 2024-08-31T07:27:40.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.689+0000 7f80fbc5e740 20 client.28573 trim_cache size 0 max 16384 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "d4942a56-ab98-4c7e-922a-46e26db5d6ed", 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:27:40.693 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.694 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.695 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.696 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.697 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:27:40.698 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.699 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.700 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.701 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:27:40.702 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:27:40.704 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:27:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.691+0000 7f80f5000640 1 -- 172.21.15.32:0/1325624071 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f80d40b4e10 msgr2=0x7f80d40b7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.691+0000 7f80f5000640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f80d40b4e10 0x7f80d40b7320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f80f006ae80 tx=0x7f80e000aab0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.691+0000 7f80f5000640 1 -- 172.21.15.32:0/1325624071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80f01101e0 msgr2=0x7f80f01a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:27:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.691+0000 7f80f5000640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80f01101e0 0x7f80f01a6110 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f80e8003cc0 tx=0x7f80e8003cf0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.691+0000 7f80f5000640 1 -- 172.21.15.32:0/1325624071 shutdown_connections 2024-08-31T07:27:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.691+0000 7f80f5000640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f80d40b4e10 0x7f80d40b7320 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.691+0000 7f80f5000640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80f01101e0 0x7f80f01a6110 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.691+0000 7f80f5000640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80f010f890 0x7f80f01a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.691+0000 7f80f5000640 1 --2- 172.21.15.32:0/1325624071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80f0109f50 0x7f80f01a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:27:40.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.691+0000 7f80f5000640 1 -- 172.21.15.32:0/1325624071 >> 172.21.15.32:0/1325624071 conn(0x7f80f0077e80 msgr2=0x7f80f01082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:27:40.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.692+0000 7f80f5000640 1 -- 172.21.15.32:0/1325624071 shutdown_connections 2024-08-31T07:27:40.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:27:40.692+0000 7f80f5000640 1 -- 172.21.15.32:0/1325624071 wait complete. 2024-08-31T07:27:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:40 smithi084 ceph-mon[22387]: pgmap v833: 97 pgs: 97 active+clean; 710 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-08-31T07:27:41.334 INFO:tasks.workunit.client.0.smithi032.stdout: 10 33651 24.46 MB/sec warmup 81 sec latency 63.323 ms 2024-08-31T07:27:42.334 INFO:tasks.workunit.client.0.smithi032.stdout: 10 35062 25.10 MB/sec warmup 82 sec latency 74.521 ms 2024-08-31T07:27:42.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:42 smithi032 ceph-mon[17252]: pgmap v834: 97 pgs: 97 active+clean; 733 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-08-31T07:27:42.656 INFO:tasks.cephfs.filesystem:scrub status for tag:d4942a56-ab98-4c7e-922a-46e26db5d6ed - {'path': '/', 'tag': 'd4942a56-ab98-4c7e-922a-46e26db5d6ed', 'options': 'recursive,force'} 2024-08-31T07:27:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:42 smithi073 ceph-mon[21995]: pgmap v834: 97 pgs: 97 active+clean; 733 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-08-31T07:27:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:42 smithi084 ceph-mon[22387]: pgmap v834: 97 pgs: 97 active+clean; 733 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-08-31T07:27:43.335 INFO:tasks.workunit.client.0.smithi032.stdout: 10 36120 25.64 MB/sec warmup 83 sec latency 41.395 ms 2024-08-31T07:27:44.335 INFO:tasks.workunit.client.0.smithi032.stdout: 10 37080 25.80 MB/sec warmup 84 sec latency 75.113 ms 2024-08-31T07:27:44.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:44 smithi032 ceph-mon[17252]: pgmap v835: 97 pgs: 97 active+clean; 742 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-08-31T07:27:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:44 smithi073 ceph-mon[21995]: pgmap v835: 97 pgs: 97 active+clean; 742 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-08-31T07:27:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:44 smithi084 ceph-mon[22387]: pgmap v835: 97 pgs: 97 active+clean; 742 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-08-31T07:27:45.335 INFO:tasks.workunit.client.0.smithi032.stdout: 10 38317 26.35 MB/sec warmup 85 sec latency 83.604 ms 2024-08-31T07:27:46.335 INFO:tasks.workunit.client.0.smithi032.stdout: 10 39386 26.76 MB/sec warmup 86 sec latency 146.784 ms 2024-08-31T07:27:46.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:46 smithi032 ceph-mon[17252]: pgmap v836: 97 pgs: 97 active+clean; 760 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 7.8 MiB/s wr, 730 op/s 2024-08-31T07:27:46.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:46 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:27:46.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:46 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:27:46.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:46 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:27:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:46 smithi073 ceph-mon[21995]: pgmap v836: 97 pgs: 97 active+clean; 760 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 7.8 MiB/s wr, 730 op/s 2024-08-31T07:27:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:46 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:27:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:46 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:27:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:46 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:27:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:46 smithi084 ceph-mon[22387]: pgmap v836: 97 pgs: 97 active+clean; 760 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 7.8 MiB/s wr, 730 op/s 2024-08-31T07:27:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:46 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:27:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:46 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:27:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:46 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:27:47.335 INFO:tasks.workunit.client.0.smithi032.stdout: 10 40421 26.95 MB/sec warmup 87 sec latency 63.779 ms 2024-08-31T07:27:47.418 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:47 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:27:47.418 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:47 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:27:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:47 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:27:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:47 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:27:47.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:47 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:27:47.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:47 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:27:48.337 INFO:tasks.workunit.client.0.smithi032.stdout: 10 41529 27.37 MB/sec warmup 88 sec latency 72.339 ms 2024-08-31T07:27:48.411 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:48 smithi032 ceph-mon[17252]: pgmap v837: 97 pgs: 97 active+clean; 782 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.5 MiB/s wr, 843 op/s 2024-08-31T07:27:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:48 smithi073 ceph-mon[21995]: pgmap v837: 97 pgs: 97 active+clean; 782 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.5 MiB/s wr, 843 op/s 2024-08-31T07:27:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:48 smithi084 ceph-mon[22387]: pgmap v837: 97 pgs: 97 active+clean; 782 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.5 MiB/s wr, 843 op/s 2024-08-31T07:27:49.335 INFO:tasks.workunit.client.0.smithi032.stdout: 10 42509 27.74 MB/sec warmup 89 sec latency 48.857 ms 2024-08-31T07:27:50.335 INFO:tasks.workunit.client.0.smithi032.stdout: 10 43426 27.91 MB/sec warmup 90 sec latency 53.830 ms 2024-08-31T07:27:50.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:50 smithi032 ceph-mon[17252]: pgmap v838: 97 pgs: 97 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 681 op/s 2024-08-31T07:27:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:50 smithi073 ceph-mon[21995]: pgmap v838: 97 pgs: 97 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 681 op/s 2024-08-31T07:27:50.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:50 smithi084 ceph-mon[22387]: pgmap v838: 97 pgs: 97 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 681 op/s 2024-08-31T07:27:51.335 INFO:tasks.workunit.client.0.smithi032.stdout: 10 44528 28.27 MB/sec warmup 91 sec latency 120.859 ms 2024-08-31T07:27:52.335 INFO:tasks.workunit.client.0.smithi032.stdout: 10 45608 28.57 MB/sec warmup 92 sec latency 52.077 ms 2024-08-31T07:27:52.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:52 smithi032 ceph-mon[17252]: pgmap v839: 97 pgs: 97 active+clean; 810 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.6 MiB/s wr, 813 op/s 2024-08-31T07:27:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:52 smithi073 ceph-mon[21995]: pgmap v839: 97 pgs: 97 active+clean; 810 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.6 MiB/s wr, 813 op/s 2024-08-31T07:27:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:52 smithi084 ceph-mon[22387]: pgmap v839: 97 pgs: 97 active+clean; 810 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.6 MiB/s wr, 813 op/s 2024-08-31T07:27:53.336 INFO:tasks.workunit.client.0.smithi032.stdout: 10 46260 28.64 MB/sec warmup 93 sec latency 326.741 ms 2024-08-31T07:27:54.327 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:54 smithi032 ceph-mon[17252]: pgmap v840: 97 pgs: 97 active+clean; 823 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.8 MiB/s wr, 733 op/s 2024-08-31T07:27:54.337 INFO:tasks.workunit.client.0.smithi032.stdout: 10 47536 29.03 MB/sec warmup 94 sec latency 48.658 ms 2024-08-31T07:27:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:54 smithi073 ceph-mon[21995]: pgmap v840: 97 pgs: 97 active+clean; 823 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.8 MiB/s wr, 733 op/s 2024-08-31T07:27:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:54 smithi084 ceph-mon[22387]: pgmap v840: 97 pgs: 97 active+clean; 823 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.8 MiB/s wr, 733 op/s 2024-08-31T07:27:55.336 INFO:tasks.workunit.client.0.smithi032.stdout: 10 48648 29.47 MB/sec warmup 95 sec latency 48.457 ms 2024-08-31T07:27:56.340 INFO:tasks.workunit.client.0.smithi032.stdout: 10 49563 29.60 MB/sec warmup 96 sec latency 88.054 ms 2024-08-31T07:27:56.597 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:56 smithi032 ceph-mon[17252]: pgmap v841: 97 pgs: 97 active+clean; 837 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.2 MiB/s wr, 752 op/s 2024-08-31T07:27:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:56 smithi073 ceph-mon[21995]: pgmap v841: 97 pgs: 97 active+clean; 837 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.2 MiB/s wr, 752 op/s 2024-08-31T07:27:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:56 smithi084 ceph-mon[22387]: pgmap v841: 97 pgs: 97 active+clean; 837 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.2 MiB/s wr, 752 op/s 2024-08-31T07:27:57.345 INFO:tasks.workunit.client.0.smithi032.stdout: 10 49896 29.48 MB/sec warmup 97 sec latency 789.078 ms 2024-08-31T07:27:58.341 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50023 29.30 MB/sec warmup 98 sec latency 1215.750 ms 2024-08-31T07:27:58.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:27:58 smithi032 ceph-mon[17252]: pgmap v842: 97 pgs: 97 active+clean; 866 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 827 op/s 2024-08-31T07:27:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:27:58 smithi073 ceph-mon[21995]: pgmap v842: 97 pgs: 97 active+clean; 866 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 827 op/s 2024-08-31T07:27:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:27:58 smithi084 ceph-mon[22387]: pgmap v842: 97 pgs: 97 active+clean; 866 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 827 op/s 2024-08-31T07:27:59.341 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50023 29.01 MB/sec warmup 99 sec latency 2215.823 ms 2024-08-31T07:28:00.341 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50023 28.72 MB/sec warmup 100 sec latency 3215.866 ms 2024-08-31T07:28:00.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:00 smithi032 ceph-mon[17252]: pgmap v843: 97 pgs: 97 active+clean; 870 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 712 op/s 2024-08-31T07:28:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:00 smithi073 ceph-mon[21995]: pgmap v843: 97 pgs: 97 active+clean; 870 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 712 op/s 2024-08-31T07:28:00.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:00 smithi084 ceph-mon[22387]: pgmap v843: 97 pgs: 97 active+clean; 870 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 712 op/s 2024-08-31T07:28:01.341 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50023 28.43 MB/sec warmup 101 sec latency 4215.907 ms 2024-08-31T07:28:02.341 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50027 28.15 MB/sec warmup 102 sec latency 5124.607 ms 2024-08-31T07:28:02.452 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:02 smithi032 ceph-mon[17252]: pgmap v844: 97 pgs: 97 active+clean; 877 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 829 op/s 2024-08-31T07:28:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:02 smithi073 ceph-mon[21995]: pgmap v844: 97 pgs: 97 active+clean; 877 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 829 op/s 2024-08-31T07:28:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:02 smithi084 ceph-mon[22387]: pgmap v844: 97 pgs: 97 active+clean; 877 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 829 op/s 2024-08-31T07:28:03.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50039 27.88 MB/sec warmup 103 sec latency 5169.961 ms 2024-08-31T07:28:04.341 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50040 27.62 MB/sec warmup 104 sec latency 2129.889 ms 2024-08-31T07:28:04.505 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:04 smithi032 ceph-mon[17252]: pgmap v845: 97 pgs: 97 active+clean; 869 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 734 op/s 2024-08-31T07:28:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:04 smithi073 ceph-mon[21995]: pgmap v845: 97 pgs: 97 active+clean; 869 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 734 op/s 2024-08-31T07:28:04.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:04 smithi084 ceph-mon[22387]: pgmap v845: 97 pgs: 97 active+clean; 869 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 734 op/s 2024-08-31T07:28:05.341 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50040 27.35 MB/sec warmup 105 sec latency 2942.983 ms 2024-08-31T07:28:06.341 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50040 27.09 MB/sec warmup 106 sec latency 3943.037 ms 2024-08-31T07:28:06.512 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:06 smithi032 ceph-mon[17252]: pgmap v846: 97 pgs: 97 active+clean; 926 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 721 op/s 2024-08-31T07:28:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:06 smithi073 ceph-mon[21995]: pgmap v846: 97 pgs: 97 active+clean; 926 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 721 op/s 2024-08-31T07:28:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:06 smithi084 ceph-mon[22387]: pgmap v846: 97 pgs: 97 active+clean; 926 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 721 op/s 2024-08-31T07:28:07.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50040 26.84 MB/sec warmup 107 sec latency 4818.268 ms 2024-08-31T07:28:07.643 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:07 smithi032 ceph-mon[17252]: osdmap e99: 12 total, 12 up, 12 in 2024-08-31T07:28:07.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:07 smithi073 ceph-mon[21995]: osdmap e99: 12 total, 12 up, 12 in 2024-08-31T07:28:07.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:07 smithi084 ceph-mon[22387]: osdmap e99: 12 total, 12 up, 12 in 2024-08-31T07:28:08.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50040 26.59 MB/sec warmup 108 sec latency 5818.320 ms 2024-08-31T07:28:08.493 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:08 smithi032 ceph-mon[17252]: pgmap v848: 97 pgs: 97 active+clean; 973 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 24 MiB/s wr, 720 op/s 2024-08-31T07:28:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:08 smithi073 ceph-mon[21995]: pgmap v848: 97 pgs: 97 active+clean; 973 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 24 MiB/s wr, 720 op/s 2024-08-31T07:28:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:08 smithi084 ceph-mon[22387]: pgmap v848: 97 pgs: 97 active+clean; 973 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 24 MiB/s wr, 720 op/s 2024-08-31T07:28:09.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 26.35 MB/sec warmup 109 sec latency 6818.479 ms 2024-08-31T07:28:10.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 26.11 MB/sec warmup 110 sec latency 7818.545 ms 2024-08-31T07:28:10.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:10 smithi084 ceph-mon[22387]: pgmap v849: 97 pgs: 97 active+clean; 947 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 22 MiB/s wr, 692 op/s 2024-08-31T07:28:10.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:10 smithi032 ceph-mon[17252]: pgmap v849: 97 pgs: 97 active+clean; 947 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 22 MiB/s wr, 692 op/s 2024-08-31T07:28:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:10 smithi073 ceph-mon[21995]: pgmap v849: 97 pgs: 97 active+clean; 947 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 22 MiB/s wr, 692 op/s 2024-08-31T07:28:11.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 25.87 MB/sec warmup 111 sec latency 8818.602 ms 2024-08-31T07:28:12.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 25.64 MB/sec warmup 112 sec latency 9818.667 ms 2024-08-31T07:28:12.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:28:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:12 smithi084 ceph-mon[22387]: pgmap v850: 97 pgs: 97 active+clean; 898 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 572 op/s 2024-08-31T07:28:12.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:12 smithi032 ceph-mon[17252]: pgmap v850: 97 pgs: 97 active+clean; 898 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 572 op/s 2024-08-31T07:28:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:12 smithi073 ceph-mon[21995]: pgmap v850: 97 pgs: 97 active+clean; 898 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 572 op/s 2024-08-31T07:28:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:13.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 25.42 MB/sec warmup 113 sec latency 10818.729 ms 2024-08-31T07:28:14.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 25.19 MB/sec warmup 114 sec latency 11818.790 ms 2024-08-31T07:28:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:14 smithi084 ceph-mon[22387]: pgmap v851: 97 pgs: 97 active+clean; 888 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 521 op/s 2024-08-31T07:28:14.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:14 smithi032 ceph-mon[17252]: pgmap v851: 97 pgs: 97 active+clean; 888 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 521 op/s 2024-08-31T07:28:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:14 smithi073 ceph-mon[21995]: pgmap v851: 97 pgs: 97 active+clean; 888 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 521 op/s 2024-08-31T07:28:15.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 24.97 MB/sec warmup 115 sec latency 12818.854 ms 2024-08-31T07:28:16.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 24.76 MB/sec warmup 116 sec latency 13818.898 ms 2024-08-31T07:28:16.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:16 smithi084 ceph-mon[22387]: pgmap v852: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 404 op/s 2024-08-31T07:28:16.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:16 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:16.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:16 smithi032 ceph-mon[17252]: pgmap v852: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 404 op/s 2024-08-31T07:28:16.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:16 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:16 smithi073 ceph-mon[21995]: pgmap v852: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 404 op/s 2024-08-31T07:28:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:16 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:17.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 24.55 MB/sec warmup 117 sec latency 14818.943 ms 2024-08-31T07:28:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:17 smithi073 ceph-mon[21995]: osdmap e100: 12 total, 12 up, 12 in 2024-08-31T07:28:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:17 smithi073 ceph-mon[21995]: pgmap v854: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 654 KiB/s wr, 177 op/s 2024-08-31T07:28:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:17 smithi084 ceph-mon[22387]: osdmap e100: 12 total, 12 up, 12 in 2024-08-31T07:28:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:17 smithi084 ceph-mon[22387]: pgmap v854: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 654 KiB/s wr, 177 op/s 2024-08-31T07:28:18.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 24.34 MB/sec warmup 118 sec latency 15818.989 ms 2024-08-31T07:28:18.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:17 smithi032 ceph-mon[17252]: osdmap e100: 12 total, 12 up, 12 in 2024-08-31T07:28:18.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:17 smithi032 ceph-mon[17252]: pgmap v854: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 654 KiB/s wr, 177 op/s 2024-08-31T07:28:19.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:18 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:19.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:18 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:19.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 24.14 MB/sec warmup 119 sec latency 16819.034 ms 2024-08-31T07:28:19.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:18 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:20.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:19 smithi073 ceph-mon[21995]: pgmap v855: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 512 KiB/s wr, 133 op/s 2024-08-31T07:28:20.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:19 smithi084 ceph-mon[22387]: pgmap v855: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 512 KiB/s wr, 133 op/s 2024-08-31T07:28:20.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:19 smithi032 ceph-mon[17252]: pgmap v855: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 512 KiB/s wr, 133 op/s 2024-08-31T07:28:21.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 1 sec latency 18819.106 ms 2024-08-31T07:28:22.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 2 sec latency 19819.150 ms 2024-08-31T07:28:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:22 smithi073 ceph-mon[21995]: pgmap v856: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 85 KiB/s wr, 41 op/s 2024-08-31T07:28:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:22 smithi084 ceph-mon[22387]: pgmap v856: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 85 KiB/s wr, 41 op/s 2024-08-31T07:28:22.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:22 smithi032 ceph-mon[17252]: pgmap v856: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 85 KiB/s wr, 41 op/s 2024-08-31T07:28:23.342 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 3 sec latency 20819.208 ms 2024-08-31T07:28:24.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 4 sec latency 21819.268 ms 2024-08-31T07:28:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:24 smithi084 ceph-mon[22387]: pgmap v857: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-08-31T07:28:24.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:24 smithi032 ceph-mon[17252]: pgmap v857: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-08-31T07:28:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:24 smithi073 ceph-mon[21995]: pgmap v857: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-08-31T07:28:25.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 5 sec latency 22819.330 ms 2024-08-31T07:28:25.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:25 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:25.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:25 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:25.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:25 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:26.328 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:26 smithi032 ceph-mon[17252]: pgmap v858: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-08-31T07:28:26.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 6 sec latency 23819.390 ms 2024-08-31T07:28:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 -- 172.21.15.32:0/1707960051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0c4102560 msgr2=0x7fb0c4102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 --2- 172.21.15.32:0/1707960051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0c4102560 0x7fb0c4102970 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009930 tx=0x7fb0b402a420 comp rx=0 tx=0).stop 2024-08-31T07:28:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 -- 172.21.15.32:0/1707960051 shutdown_connections 2024-08-31T07:28:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 --2- 172.21.15.32:0/1707960051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c41079a0 0x7fb0c4107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 --2- 172.21.15.32:0/1707960051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c4102eb0 0x7fb0c41073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 --2- 172.21.15.32:0/1707960051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0c4102560 0x7fb0c4102970 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 -- 172.21.15.32:0/1707960051 >> 172.21.15.32:0/1707960051 conn(0x7fb0c40794e0 msgr2=0x7fb0c40798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:26.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 -- 172.21.15.32:0/1707960051 shutdown_connections 2024-08-31T07:28:26.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 -- 172.21.15.32:0/1707960051 wait complete. 2024-08-31T07:28:26.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 Processor -- start 2024-08-31T07:28:26.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 -- start start 2024-08-31T07:28:26.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c4102560 0x7fb0c41a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0c4102eb0 0x7fb0c41a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c41079a0 0x7fb0c41a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c4119e60 con 0x7fb0c4102eb0 2024-08-31T07:28:26.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c4119ce0 con 0x7fb0c41079a0 2024-08-31T07:28:26.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c4119fe0 con 0x7fb0c4102560 2024-08-31T07:28:26.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c4102560 0x7fb0c41a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0c4102eb0 0x7fb0c41a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c4102560 0x7fb0c41a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47790/0 (socket says 172.21.15.32:47790) 2024-08-31T07:28:26.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c3400640 1 -- 172.21.15.32:0/2310793688 learned_addr learned my addr 172.21.15.32:0/2310793688 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:26.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c3400640 1 -- 172.21.15.32:0/2310793688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c41079a0 msgr2=0x7fb0c41a6110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:28:26.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c3400640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c41079a0 0x7fb0c41a6110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c3400640 1 -- 172.21.15.32:0/2310793688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0c4102eb0 msgr2=0x7fb0c41a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:26.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c3400640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0c4102eb0 0x7fb0c41a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c3400640 1 -- 172.21.15.32:0/2310793688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c41a6840 con 0x7fb0c4102560 2024-08-31T07:28:26.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0c4102eb0 0x7fb0c41a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:26.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0c3400640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c4102560 0x7fb0c41a08d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4004770 tx=0x7fb0b4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:26.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0bbe00640 1 -- 172.21.15.32:0/2310793688 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b403f030 con 0x7fb0c4102560 2024-08-31T07:28:26.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0bbe00640 1 -- 172.21.15.32:0/2310793688 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b4003aa0 con 0x7fb0c4102560 2024-08-31T07:28:26.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0ca800640 1 -- 172.21.15.32:0/2310793688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c41a6ad0 con 0x7fb0c4102560 2024-08-31T07:28:26.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.460+0000 7fb0bbe00640 1 -- 172.21.15.32:0/2310793688 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4035070 con 0x7fb0c4102560 2024-08-31T07:28:26.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.466+0000 7fb0ca800640 1 -- 172.21.15.32:0/2310793688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c41ad290 con 0x7fb0c4102560 2024-08-31T07:28:26.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 Processor -- start 2024-08-31T07:28:26.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 -- start start 2024-08-31T07:28:26.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e370950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e370e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 0x557b6e3713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557b6e2e7770 con 0x557b6dfcb070 2024-08-31T07:28:26.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557b6e2d6d00 con 0x557b6dfa21b0 2024-08-31T07:28:26.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557b6e2dacb0 con 0x557b6dfaa390 2024-08-31T07:28:26.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e370950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e370950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34670/0 (socket says 172.21.15.32:34670) 2024-08-31T07:28:26.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c3e00640 1 -- 172.21.15.32:0/1318292189 learned_addr learned my addr 172.21.15.32:0/1318292189 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/1318292189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 0x557b6e3713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c3e00640 1 -- 172.21.15.32:0/1318292189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 msgr2=0x557b6e3713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c3e00640 1 --2- 172.21.15.32:0/1318292189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 0x557b6e3713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c3e00640 1 -- 172.21.15.32:0/1318292189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 msgr2=0x557b6e370e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:28:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c3e00640 1 --2- 172.21.15.32:0/1318292189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e370e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c3e00640 1 -- 172.21.15.32:0/1318292189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6e036650 con 0x557b6dfcb070 2024-08-31T07:28:26.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0bbe00640 1 -- 172.21.15.32:0/2310793688 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb0b4042030 con 0x7fb0c4102560 2024-08-31T07:28:26.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c3e00640 1 --2- 172.21.15.32:0/1318292189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e370950 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009e30 tx=0x7fb0bc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b46f75453146b9e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:26.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0b8c00640 1 -- 172.21.15.32:0/1318292189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc004d20 con 0x557b6dfcb070 2024-08-31T07:28:26.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0b8c00640 1 -- 172.21.15.32:0/1318292189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0bc00ee70 con 0x557b6dfcb070 2024-08-31T07:28:26.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0b8c00640 1 -- 172.21.15.32:0/1318292189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc010760 con 0x557b6dfcb070 2024-08-31T07:28:26.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 -- 172.21.15.32:0/1318292189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 msgr2=0x557b6e370950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:26.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 --2- 172.21.15.32:0/1318292189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e370950 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009e30 tx=0x7fb0bc00e720 comp rx=0 tx=0).stop 2024-08-31T07:28:26.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 -- 172.21.15.32:0/1318292189 shutdown_connections 2024-08-31T07:28:26.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 --2- 172.21.15.32:0/1318292189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 0x557b6e3713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 --2- 172.21.15.32:0/1318292189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e370e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 --2- 172.21.15.32:0/1318292189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e370950 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 -- 172.21.15.32:0/1318292189 >> 172.21.15.32:0/1318292189 conn(0x557b6dfbf460 msgr2=0x557b6e36e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:26.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0bbe00640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0a80b4d20 0x7fb0a80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 -- 172.21.15.32:0/1318292189 shutdown_connections 2024-08-31T07:28:26.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0bbe00640 1 -- 172.21.15.32:0/2310793688 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fb0b4105930 con 0x7fb0c4102560 2024-08-31T07:28:26.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0d1411740 1 -- 172.21.15.32:0/1318292189 wait complete. 2024-08-31T07:28:26.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.468+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0a80b4d20 0x7fb0a80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 -- start start 2024-08-31T07:28:26.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 Processor -- start 2024-08-31T07:28:26.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0a80b4d20 0x7fb0a80b7230 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4079a80 tx=0x7fb0b0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:26.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e37c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 0x557b6e37cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e3823f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557b6e139700 con 0x557b6dfcb070 2024-08-31T07:28:26.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557b6e25dcb0 con 0x557b6dfa21b0 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557b6e37f130 con 0x557b6dfaa390 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 10 client.0 _ll_register_callbacks cb 0x557b6e283430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 0x557b6e37cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e37c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 0x557b6e37cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47804/0 (socket says 172.21.15.32:47804) 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e37c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:51024/0 (socket says 172.21.15.32:51024) 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c2a00640 1 -- 172.21.15.32:0/941701708 learned_addr learned my addr 172.21.15.32:0/941701708 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c3e00640 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e3823f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c3400640 1 -- 172.21.15.32:0/941701708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 msgr2=0x557b6e37cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c3400640 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 0x557b6e37cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c3400640 1 -- 172.21.15.32:0/941701708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 msgr2=0x557b6e3823f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c3400640 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e3823f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c3400640 1 -- 172.21.15.32:0/941701708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6e29e870 con 0x557b6dfa21b0 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 0x557b6e37cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c3400640 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e37c650 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4103070 tx=0x7fb0b40c7f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb42ea1f3f294d6a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0ad600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0ad600640 1 -- 172.21.15.32:0/941701708 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4049070 con 0x557b6dfa21b0 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 -- 172.21.15.32:0/941701708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0bc00ee70 con 0x557b6dfa21b0 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0ad600640 1 -- 172.21.15.32:0/941701708 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fb0b40c8600 con 0x557b6dfa21b0 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0ad600640 1 -- 172.21.15.32:0/941701708 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fb0b4103500 con 0x557b6dfa21b0 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 10 client.28589 fetch_fsmap learned FSMap version 58 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 -- 172.21.15.32:0/941701708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557b6e38b220 con 0x557b6dfa21b0 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0ad600640 1 -- 172.21.15.32:0/941701708 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0bc00ee70 con 0x557b6dfa21b0 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0ad600640 1 -- 172.21.15.32:0/941701708 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fb0b40d4ea0 con 0x557b6dfa21b0 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 10 client.28589 fetch_fsmap finished waiting for FSMap version 58 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 10 client.28589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:28:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 20 client.28589 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b6df899f0 0x557b6e38e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 4 client.28589 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0d1411740 1 -- 172.21.15.32:0/941701708 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b6e2e96c0 con 0x557b6df899f0 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b6df899f0 0x557b6e38e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b6df899f0 0x557b6e38e6e0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac5097bf8cf599cc server_cookie=1596eab5f7bb5268 in_seq=0 out_seq=0 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.474+0000 7fb0ad600640 10 client.28589 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0ad600640 1 -- 172.21.15.32:0/941701708 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b6e2e96c0 con 0x557b6df899f0 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0ad600640 10 client.28589 handle_command_reply: tid=0 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 client.28589 shutdown 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 -- 172.21.15.32:0/941701708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 msgr2=0x557b6e37c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e37c650 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4103070 tx=0x7fb0b40c7f90 comp rx=0 tx=0).stop 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 -- 172.21.15.32:0/941701708 shutdown_connections 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b6df899f0 0x557b6e38e6e0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e3823f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfaa390 0x557b6e37cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 --2- 172.21.15.32:0/941701708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e37c650 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0ad600640 0 client.28589 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 -- 172.21.15.32:0/941701708 >> 172.21.15.32:0/941701708 conn(0x557b6dfbf460 msgr2=0x557b6e373b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 -- 172.21.15.32:0/941701708 shutdown_connections 2024-08-31T07:28:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 1 -- 172.21.15.32:0/941701708 wait complete. 2024-08-31T07:28:26.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.479+0000 7fb0d1411740 20 client.28589 trim_cache size 0 max 16384 2024-08-31T07:28:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 Processor -- start 2024-08-31T07:28:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 -- start start 2024-08-31T07:28:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfa21b0 0x557b6e37e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfaa390 0x557b6e375a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e375f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41b59f0 con 0x557b6dfcb070 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41b7680 con 0x557b6dfaa390 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41b60f0 con 0x557b6dfa21b0 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfaa390 0x557b6e375a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e375f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfaa390 0x557b6e375a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:51036/0 (socket says 172.21.15.32:51036) 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c2a00640 1 -- 172.21.15.32:0/2042946661 learned_addr learned my addr 172.21.15.32:0/2042946661 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c3400640 1 --2- 172.21.15.32:0/2042946661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfa21b0 0x557b6e37e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c2a00640 1 -- 172.21.15.32:0/2042946661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfa21b0 msgr2=0x557b6e37e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/2042946661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfa21b0 0x557b6e37e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c2a00640 1 -- 172.21.15.32:0/2042946661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 msgr2=0x557b6e375f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:26.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/2042946661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e375f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c2a00640 1 -- 172.21.15.32:0/2042946661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6e37c200 con 0x557b6dfaa390 2024-08-31T07:28:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/2042946661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfaa390 0x557b6e375a30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x557b6e38ee10 tx=0x7fb0b001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1286f0ce6878a43 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0afe00640 1 -- 172.21.15.32:0/2042946661 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0003ec0 con 0x557b6dfaa390 2024-08-31T07:28:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0afe00640 1 -- 172.21.15.32:0/2042946661 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b001fd20 con 0x557b6dfaa390 2024-08-31T07:28:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0afe00640 1 -- 172.21.15.32:0/2042946661 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0004a00 con 0x557b6dfaa390 2024-08-31T07:28:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 -- 172.21.15.32:0/2042946661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfaa390 msgr2=0x557b6e375a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 --2- 172.21.15.32:0/2042946661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfaa390 0x557b6e375a30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x557b6e38ee10 tx=0x7fb0b001a000 comp rx=0 tx=0).stop 2024-08-31T07:28:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 -- 172.21.15.32:0/2042946661 shutdown_connections 2024-08-31T07:28:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 --2- 172.21.15.32:0/2042946661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfcb070 0x557b6e375f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 --2- 172.21.15.32:0/2042946661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfaa390 0x557b6e375a30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 --2- 172.21.15.32:0/2042946661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfa21b0 0x557b6e37e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 -- 172.21.15.32:0/2042946661 >> 172.21.15.32:0/2042946661 conn(0x557b6dfbf460 msgr2=0x557b6e37d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 -- 172.21.15.32:0/2042946661 shutdown_connections 2024-08-31T07:28:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.498+0000 7fb0d1411740 1 -- 172.21.15.32:0/2042946661 wait complete. 2024-08-31T07:28:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.503+0000 7fb0d1411740 1 -- start start 2024-08-31T07:28:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.503+0000 7fb0d1411740 1 Processor -- start 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.503+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e384960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.503+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfaa390 0x557b6e384ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.503+0000 7fb0d1411740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfcb070 0x557b6e3f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.503+0000 7fb0d1411740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41b6ef0 con 0x557b6dfaa390 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.503+0000 7fb0d1411740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41b67f0 con 0x557b6dfa21b0 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.503+0000 7fb0d1411740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c41b39f0 con 0x557b6dfcb070 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.503+0000 7fb0d1411740 10 client.0 _ll_register_callbacks cb 0x7fb0b4003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e384960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e384960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:51040/0 (socket says 172.21.15.32:51040) 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c3400640 1 -- 172.21.15.32:0/3078842616 learned_addr learned my addr 172.21.15.32:0/3078842616 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c3e00640 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfcb070 0x557b6e3f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfaa390 0x557b6e384ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c3400640 1 -- 172.21.15.32:0/3078842616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfcb070 msgr2=0x557b6e3f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c3400640 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfcb070 0x557b6e3f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c3400640 1 -- 172.21.15.32:0/3078842616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfaa390 msgr2=0x557b6e384ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c3400640 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfaa390 0x557b6e384ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c3400640 1 -- 172.21.15.32:0/3078842616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6e385790 con 0x557b6dfa21b0 2024-08-31T07:28:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c3400640 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e384960 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40c8090 tx=0x7fb0b4003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bef5683fc74bb391 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ad600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:28:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ad600640 1 -- 172.21.15.32:0/3078842616 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4049040 con 0x557b6dfa21b0 2024-08-31T07:28:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ad600640 1 -- 172.21.15.32:0/3078842616 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fb0b407a020 con 0x557b6dfa21b0 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 -- 172.21.15.32:0/3078842616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b001fd20 con 0x557b6dfa21b0 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ad600640 1 -- 172.21.15.32:0/3078842616 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fb0b40cbb20 con 0x557b6dfa21b0 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 10 client.28595 fetch_fsmap learned FSMap version 58 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 -- 172.21.15.32:0/3078842616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557b6e3f96d0 con 0x557b6dfa21b0 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ad600640 1 -- 172.21.15.32:0/3078842616 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b001fd20 con 0x557b6dfa21b0 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ad600640 1 -- 172.21.15.32:0/3078842616 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) ==== 8394+0+0 (secure 0 0 0) 0x7fb0b40ed330 con 0x557b6dfa21b0 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 10 client.28595 fetch_fsmap finished waiting for FSMap version 58 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 10 client.28595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 20 client.28595 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b6df5eda0 0x557b6e3fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 4 client.28595 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:28:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 -- 172.21.15.32:0/3078842616 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b6e2e5160 con 0x557b6df5eda0 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b6df5eda0 0x557b6e3fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0c2a00640 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b6df5eda0 0x557b6e3fbd70 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62587570b1ac3124 server_cookie=bb653da0f9d000f3 in_seq=0 out_seq=0 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ad600640 10 client.28595 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ad600640 1 -- 172.21.15.32:0/3078842616 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x557b6e2e5160 con 0x557b6df5eda0 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ad600640 10 client.28595 handle_command_reply: tid=0 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 client.28595 shutdown 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 -- 172.21.15.32:0/3078842616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 msgr2=0x557b6e384960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e384960 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40c8090 tx=0x7fb0b4003f50 comp rx=0 tx=0).stop 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 -- 172.21.15.32:0/3078842616 shutdown_connections 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b6df5eda0 0x557b6e3fbd70 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b6dfcb070 0x557b6e3f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b6dfaa390 0x557b6e384ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 --2- 172.21.15.32:0/3078842616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b6dfa21b0 0x557b6e384960 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 -- 172.21.15.32:0/3078842616 >> 172.21.15.32:0/3078842616 conn(0x557b6dfbf460 msgr2=0x557b6e37e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:26.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ad600640 0 client.28595 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:26.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 -- 172.21.15.32:0/3078842616 shutdown_connections 2024-08-31T07:28:26.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 1 -- 172.21.15.32:0/3078842616 wait complete. 2024-08-31T07:28:26.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0d1411740 20 client.28595 trim_cache size 0 max 16384 2024-08-31T07:28:26.511 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.513 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.515 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:28:26.515 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:28:26.515 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.515 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.515 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.515 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.515 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.515 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.516 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.517 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.517 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.517 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:28:26.517 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:28:26.517 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.517 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:28:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:28:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:28:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.521 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.522 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:28:26.523 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:28:26.524 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:26.524 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:26.524 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:26.524 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:26.524 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:26.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:28:26.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:28:26.524 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:28:26.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ca800640 1 -- 172.21.15.32:0/2310793688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0a80b4d20 msgr2=0x7fb0a80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:26.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ca800640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0a80b4d20 0x7fb0a80b7230 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4079a80 tx=0x7fb0b0005f90 comp rx=0 tx=0).stop 2024-08-31T07:28:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ca800640 1 -- 172.21.15.32:0/2310793688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c4102560 msgr2=0x7fb0c41a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.504+0000 7fb0ca800640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c4102560 0x7fb0c41a08d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4004770 tx=0x7fb0b4004040 comp rx=0 tx=0).stop 2024-08-31T07:28:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.510+0000 7fb0ca800640 1 -- 172.21.15.32:0/2310793688 shutdown_connections 2024-08-31T07:28:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.510+0000 7fb0ca800640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0a80b4d20 0x7fb0a80b7230 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.510+0000 7fb0ca800640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c41079a0 0x7fb0c41a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.510+0000 7fb0ca800640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0c4102eb0 0x7fb0c41a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.510+0000 7fb0ca800640 1 --2- 172.21.15.32:0/2310793688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0c4102560 0x7fb0c41a08d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.510+0000 7fb0ca800640 1 -- 172.21.15.32:0/2310793688 >> 172.21.15.32:0/2310793688 conn(0x7fb0c40794e0 msgr2=0x7fb0c41053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.510+0000 7fb0ca800640 1 -- 172.21.15.32:0/2310793688 shutdown_connections 2024-08-31T07:28:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:26.510+0000 7fb0ca800640 1 -- 172.21.15.32:0/2310793688 wait complete. 2024-08-31T07:28:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:26 smithi073 ceph-mon[21995]: pgmap v858: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-08-31T07:28:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:26 smithi084 ceph-mon[22387]: pgmap v858: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-08-31T07:28:27.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 7 sec latency 24819.444 ms 2024-08-31T07:28:27.639 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:28:27.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:28:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:28.288 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:28 smithi032 ceph-mon[17252]: pgmap v859: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-08-31T07:28:28.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 8 sec latency 25819.507 ms 2024-08-31T07:28:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:28 smithi073 ceph-mon[21995]: pgmap v859: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-08-31T07:28:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:28 smithi084 ceph-mon[22387]: pgmap v859: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-08-31T07:28:28.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.853+0000 7f4ba6c00640 1 -- 172.21.15.32:0/1050618620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba01144f0 msgr2=0x7f4ba0116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:28.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.853+0000 7f4ba6c00640 1 --2- 172.21.15.32:0/1050618620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba01144f0 0x7f4ba0116910 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009ca0 tx=0x7f4b9802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:28:28.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.853+0000 7f4ba6c00640 1 -- 172.21.15.32:0/1050618620 shutdown_connections 2024-08-31T07:28:28.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.853+0000 7f4ba6c00640 1 --2- 172.21.15.32:0/1050618620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba01144f0 0x7f4ba0116910 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:28.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.853+0000 7f4ba6c00640 1 --2- 172.21.15.32:0/1050618620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ba0078030 0x7f4ba0113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:28.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.853+0000 7f4ba6c00640 1 --2- 172.21.15.32:0/1050618620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ba00776e0 0x7f4ba0077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:28.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.853+0000 7f4ba6c00640 1 -- 172.21.15.32:0/1050618620 >> 172.21.15.32:0/1050618620 conn(0x7f4ba0101600 msgr2=0x7f4ba0103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:28.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.853+0000 7f4ba6c00640 1 -- 172.21.15.32:0/1050618620 shutdown_connections 2024-08-31T07:28:28.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.853+0000 7f4ba6c00640 1 -- 172.21.15.32:0/1050618620 wait complete. 2024-08-31T07:28:28.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.855+0000 7f4ba6c00640 1 Processor -- start 2024-08-31T07:28:28.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.855+0000 7f4ba6c00640 1 -- start start 2024-08-31T07:28:28.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba6c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ba00776e0 0x7f4ba01a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:28.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba6c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ba0078030 0x7f4ba01a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:28.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba6c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba01144f0 0x7f4ba01a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:28.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba6c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0119ec0 con 0x7f4ba01144f0 2024-08-31T07:28:28.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba6c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0119d40 con 0x7f4ba0078030 2024-08-31T07:28:28.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba6c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba011a040 con 0x7f4ba00776e0 2024-08-31T07:28:28.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4b9fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ba00776e0 0x7f4ba01a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:28.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4b9f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ba0078030 0x7f4ba01a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:28.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4b9fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ba00776e0 0x7f4ba01a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47830/0 (socket says 172.21.15.32:47830) 2024-08-31T07:28:28.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4b9fe00640 1 -- 172.21.15.32:0/2771814421 learned_addr learned my addr 172.21.15.32:0/2771814421 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:28.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4b9f400640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ba0078030 0x7f4ba01a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:51050/0 (socket says 172.21.15.32:51050) 2024-08-31T07:28:28.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba4c00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba01144f0 0x7f4ba01a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba4c00640 1 -- 172.21.15.32:0/2771814421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ba00776e0 msgr2=0x7f4ba01a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba4c00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ba00776e0 0x7f4ba01a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba4c00640 1 -- 172.21.15.32:0/2771814421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ba0078030 msgr2=0x7f4ba01a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba4c00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ba0078030 0x7f4ba01a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba4c00640 1 -- 172.21.15.32:0/2771814421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba01a67a0 con 0x7f4ba01144f0 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4b9f400640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ba0078030 0x7f4ba01a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba4c00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba01144f0 0x7f4ba01a6070 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4b98005e30 tx=0x7f4b980317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4b9cc00640 1 -- 172.21.15.32:0/2771814421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98003b70 con 0x7f4ba01144f0 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4b9cc00640 1 -- 172.21.15.32:0/2771814421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b98003d10 con 0x7f4ba01144f0 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba6c00640 1 -- 172.21.15.32:0/2771814421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba01a6a30 con 0x7f4ba01144f0 2024-08-31T07:28:28.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4b9cc00640 1 -- 172.21.15.32:0/2771814421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9802f970 con 0x7f4ba01144f0 2024-08-31T07:28:28.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba6c00640 1 -- 172.21.15.32:0/2771814421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba01ad1f0 con 0x7f4ba01144f0 2024-08-31T07:28:28.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.856+0000 7f4ba6c00640 1 -- 172.21.15.32:0/2771814421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba0077af0 con 0x7f4ba01144f0 2024-08-31T07:28:28.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.862+0000 7f4b9cc00640 1 -- 172.21.15.32:0/2771814421 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4b9802fb10 con 0x7f4ba01144f0 2024-08-31T07:28:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.862+0000 7f4b9cc00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b840b4ca0 0x7f4b840b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.862+0000 7f4b9cc00640 1 -- 172.21.15.32:0/2771814421 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7f4b981066e0 con 0x7f4ba01144f0 2024-08-31T07:28:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.862+0000 7f4b9fe00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b840b4ca0 0x7f4b840b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:28.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.862+0000 7f4b9fe00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b840b4ca0 0x7f4b840b71b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009cd0 tx=0x7f4b90005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:28.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:28.866+0000 7f4b9cc00640 1 -- 172.21.15.32:0/2771814421 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4b980cd4c0 con 0x7f4ba01144f0 2024-08-31T07:28:29.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.101+0000 7f4ba6c00640 1 -- 172.21.15.32:0/2771814421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ba010f6b0 con 0x7f4ba01144f0 2024-08-31T07:28:29.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.102+0000 7f4b9cc00640 1 -- 172.21.15.32:0/2771814421 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11971 (secure 0 0 0) 0x7f4b980ccdc0 con 0x7f4ba01144f0 2024-08-31T07:28:29.103 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:28:29.103 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":58,"btime":"2024-08-31T07:28:24:176777+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:28:23.399641+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:28:29.103 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 58 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 -- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b840b4ca0 msgr2=0x7f4b840b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b840b4ca0 0x7f4b840b71b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009cd0 tx=0x7f4b90005f90 comp rx=0 tx=0).stop 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 -- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba01144f0 msgr2=0x7f4ba01a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba01144f0 0x7f4ba01a6070 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4b98005e30 tx=0x7f4b980317c0 comp rx=0 tx=0).stop 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 -- 172.21.15.32:0/2771814421 shutdown_connections 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b840b4ca0 0x7f4b840b71b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba01144f0 0x7f4ba01a6070 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ba0078030 0x7f4ba01a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 --2- 172.21.15.32:0/2771814421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ba00776e0 0x7f4ba01a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 -- 172.21.15.32:0/2771814421 >> 172.21.15.32:0/2771814421 conn(0x7f4ba0101600 msgr2=0x7f4ba01164b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 -- 172.21.15.32:0/2771814421 shutdown_connections 2024-08-31T07:28:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:29.105+0000 7f4ba6c00640 1 -- 172.21.15.32:0/2771814421 wait complete. 2024-08-31T07:28:29.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 9 sec latency 26819.569 ms 2024-08-31T07:28:29.362 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:29 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2771814421' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:28:29.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:29 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2771814421' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:28:29.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:29 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2771814421' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:28:30.142 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:28:30.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 10 sec latency 27819.634 ms 2024-08-31T07:28:30.506 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:30 smithi032 ceph-mon[17252]: pgmap v860: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-08-31T07:28:30.540 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:30 smithi073 ceph-mon[21995]: pgmap v860: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-08-31T07:28:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:30 smithi084 ceph-mon[22387]: pgmap v860: 97 pgs: 97 active+clean; 883 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-08-31T07:28:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- 172.21.15.32:0/308703416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a98075e40 msgr2=0x7f4a9810f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 --2- 172.21.15.32:0/308703416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a98075e40 0x7f4a9810f980 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009900 tx=0x7f4a8402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- 172.21.15.32:0/308703416 shutdown_connections 2024-08-31T07:28:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 --2- 172.21.15.32:0/308703416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a9810fec0 0x7f4a981122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 --2- 172.21.15.32:0/308703416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a98075e40 0x7f4a9810f980 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 --2- 172.21.15.32:0/308703416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a980754f0 0x7f4a98075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- 172.21.15.32:0/308703416 >> 172.21.15.32:0/308703416 conn(0x7f4a9807b5f0 msgr2=0x7f4a9807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- 172.21.15.32:0/308703416 shutdown_connections 2024-08-31T07:28:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- 172.21.15.32:0/308703416 wait complete. 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 Processor -- start 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- start start 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a980754f0 0x7f4a9819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a98075e40 0x7f4a9819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810fec0 0x7f4a981a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4a98115a80 con 0x7f4a9810fec0 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a98115900 con 0x7f4a980754f0 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a98115c00 con 0x7f4a98075e40 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a980754f0 0x7f4a9819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a980754f0 0x7f4a9819c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:51072/0 (socket says 172.21.15.32:51072) 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a97e00640 1 -- 172.21.15.32:0/1140126710 learned_addr learned my addr 172.21.15.32:0/1140126710 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a97400640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a98075e40 0x7f4a9819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9d200640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810fec0 0x7f4a981a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a97400640 1 -- 172.21.15.32:0/1140126710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a980754f0 msgr2=0x7f4a9819c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a97400640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a980754f0 0x7f4a9819c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a97400640 1 -- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810fec0 msgr2=0x7f4a981a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a97400640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810fec0 0x7f4a981a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a97400640 1 -- 172.21.15.32:0/1140126710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a981a23e0 con 0x7f4a98075e40 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9d200640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810fec0 0x7f4a981a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a97400640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a98075e40 0x7f4a9819ca10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009230 tx=0x7f4a84003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a94c00640 1 -- 172.21.15.32:0/1140126710 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8403f070 con 0x7f4a98075e40 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a94c00640 1 -- 172.21.15.32:0/1140126710 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a84031b50 con 0x7f4a98075e40 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- 172.21.15.32:0/1140126710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a981a2670 con 0x7f4a98075e40 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a94c00640 1 -- 172.21.15.32:0/1140126710 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84035030 con 0x7f4a98075e40 2024-08-31T07:28:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.300+0000 7f4a9f200640 1 -- 172.21.15.32:0/1140126710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a981a8e30 con 0x7f4a98075e40 2024-08-31T07:28:31.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4aa6015740 1 Processor -- start 2024-08-31T07:28:31.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a94c00640 1 -- 172.21.15.32:0/1140126710 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4a84042070 con 0x7f4a98075e40 2024-08-31T07:28:31.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4aa6015740 1 -- start start 2024-08-31T07:28:31.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 0x5593801f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 0x5593801f9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4aa6015740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55938016f7b0 con 0x55937fe2a1b0 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4aa6015740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55938015ed40 con 0x55937fe32390 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4aa6015740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559380162cf0 con 0x55937fe53070 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a9d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 0x5593801f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a9d200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 0x5593801f88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47842/0 (socket says 172.21.15.32:47842) 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801f8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34740/0 (socket says 172.21.15.32:34740) 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a97e00640 1 -- 172.21.15.32:0/2113550153 learned_addr learned my addr 172.21.15.32:0/2113550153 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a97400640 1 --2- 172.21.15.32:0/2113550153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 0x5593801f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a9d200640 1 -- 172.21.15.32:0/2113550153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 msgr2=0x5593801f9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a9d200640 1 --2- 172.21.15.32:0/2113550153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 0x5593801f9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a9d200640 1 -- 172.21.15.32:0/2113550153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 msgr2=0x5593801f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a9d200640 1 --2- 172.21.15.32:0/2113550153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a9d200640 1 -- 172.21.15.32:0/2113550153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937febe650 con 0x55937fe53070 2024-08-31T07:28:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a9d200640 1 --2- 172.21.15.32:0/2113550153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 0x5593801f88b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4a90005fd0 tx=0x7f4a90011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3991f02853031f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a94c00640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 0x7f4a7c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a8d600640 1 -- 172.21.15.32:0/2113550153 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90004d20 con 0x55937fe53070 2024-08-31T07:28:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a8d600640 1 -- 172.21.15.32:0/2113550153 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a9000f9a0 con 0x55937fe53070 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a8d600640 1 -- 172.21.15.32:0/2113550153 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90004470 con 0x55937fe53070 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a94c00640 1 -- 172.21.15.32:0/1140126710 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7f4a84105a90 con 0x7f4a98075e40 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.306+0000 7f4a97e00640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 0x7f4a7c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 -- 172.21.15.32:0/2113550153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 msgr2=0x5593801f88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 --2- 172.21.15.32:0/2113550153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 0x5593801f88b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4a90005fd0 tx=0x7f4a90011bf0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 -- 172.21.15.32:0/2113550153 shutdown_connections 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 --2- 172.21.15.32:0/2113550153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 0x5593801f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 --2- 172.21.15.32:0/2113550153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 --2- 172.21.15.32:0/2113550153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 0x5593801f88b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 -- 172.21.15.32:0/2113550153 >> 172.21.15.32:0/2113550153 conn(0x55937fe47460 msgr2=0x5593801f67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4a97e00640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 0x7f4a7c0b7230 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x5593801f7530 tx=0x7f4a88005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 -- 172.21.15.32:0/2113550153 shutdown_connections 2024-08-31T07:28:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 -- 172.21.15.32:0/2113550153 wait complete. 2024-08-31T07:28:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 -- start start 2024-08-31T07:28:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.311+0000 7f4aa6015740 1 Processor -- start 2024-08-31T07:28:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe2a1b0 0x559380204450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380204990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe53070 0x559380210380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4aa6015740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55937ffc1700 con 0x55937fe53070 2024-08-31T07:28:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4aa6015740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5593800e5cf0 con 0x55937fe2a1b0 2024-08-31T07:28:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4aa6015740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559380207090 con 0x55937fe32390 2024-08-31T07:28:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4aa6015740 10 client.0 _ll_register_callbacks cb 0x55938010b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4a97400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380204990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4a97400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380204990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47854/0 (socket says 172.21.15.32:47854) 2024-08-31T07:28:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4a97400640 1 -- 172.21.15.32:0/172535519 learned_addr learned my addr 172.21.15.32:0/172535519 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4a9d200640 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe53070 0x559380210380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4a97400640 1 -- 172.21.15.32:0/172535519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe2a1b0 msgr2=0x559380204450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:28:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4a97400640 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe2a1b0 0x559380204450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4a97400640 1 -- 172.21.15.32:0/172535519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe53070 msgr2=0x559380210380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4a97400640 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe53070 0x559380210380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4a97400640 1 -- 172.21.15.32:0/172535519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593801268b0 con 0x55937fe32390 2024-08-31T07:28:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.312+0000 7f4a9d200640 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe53070 0x559380210380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:28:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a97400640 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380204990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4a84006010 tx=0x7f4a84003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e530a3bc23f10b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a82000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a82000640 1 -- 172.21.15.32:0/172535519 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84049070 con 0x55937fe32390 2024-08-31T07:28:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a82000640 1 -- 172.21.15.32:0/172535519 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7f4a840d5210 con 0x55937fe32390 2024-08-31T07:28:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 -- 172.21.15.32:0/172535519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a9000f9a0 con 0x55937fe32390 2024-08-31T07:28:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a82000640 1 -- 172.21.15.32:0/172535519 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f4a840cbac0 con 0x55937fe32390 2024-08-31T07:28:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 10 client.28609 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 -- 172.21.15.32:0/172535519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5593802130a0 con 0x55937fe32390 2024-08-31T07:28:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a82000640 1 -- 172.21.15.32:0/172535519 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a9000f9a0 con 0x55937fe32390 2024-08-31T07:28:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a82000640 1 -- 172.21.15.32:0/172535519 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7f4a8403bb40 con 0x55937fe32390 2024-08-31T07:28:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 10 client.28609 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 10 client.28609 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:28:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 20 client.28609 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55937fe119f0 0x559380216620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 4 client.28609 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:28:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 -- 172.21.15.32:0/172535519 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559380171700 con 0x55937fe119f0 2024-08-31T07:28:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a97e00640 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55937fe119f0 0x559380216620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a97e00640 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55937fe119f0 0x559380216620 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=50097d347dd1675f server_cookie=a5532c3ad9af6f2e in_seq=0 out_seq=0 2024-08-31T07:28:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a82000640 10 client.28609 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:28:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a82000640 1 -- 172.21.15.32:0/172535519 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559380171700 con 0x55937fe119f0 2024-08-31T07:28:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a82000640 10 client.28609 handle_command_reply: tid=0 2024-08-31T07:28:31.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 client.28609 shutdown 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 -- 172.21.15.32:0/172535519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 msgr2=0x559380204990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380204990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4a84006010 tx=0x7f4a84003f90 comp rx=0 tx=0).stop 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 -- 172.21.15.32:0/172535519 shutdown_connections 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55937fe119f0 0x559380216620 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe53070 0x559380210380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380204990 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 --2- 172.21.15.32:0/172535519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe2a1b0 0x559380204450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4a82000640 0 client.28609 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 -- 172.21.15.32:0/172535519 >> 172.21.15.32:0/172535519 conn(0x55937fe47460 msgr2=0x5593801fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 -- 172.21.15.32:0/172535519 shutdown_connections 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 1 -- 172.21.15.32:0/172535519 wait complete. 2024-08-31T07:28:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.324+0000 7f4aa6015740 20 client.28609 trim_cache size 0 max 16384 2024-08-31T07:28:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 Processor -- start 2024-08-31T07:28:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- start start 2024-08-31T07:28:31.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 11 sec latency 28819.678 ms 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801fe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 0x559380210280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 0x5593801f6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b1590 con 0x55937fe2a1b0 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b31f0 con 0x55937fe32390 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b1c60 con 0x55937fe53070 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801fe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801fe0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34766/0 (socket says 172.21.15.32:34766) 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97e00640 1 -- 172.21.15.32:0/3963412512 learned_addr learned my addr 172.21.15.32:0/3963412512 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97400640 1 --2- 172.21.15.32:0/3963412512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 0x559380210280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a9d200640 1 --2- 172.21.15.32:0/3963412512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 0x5593801f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97e00640 1 -- 172.21.15.32:0/3963412512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 msgr2=0x5593801f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97e00640 1 --2- 172.21.15.32:0/3963412512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 0x5593801f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97e00640 1 -- 172.21.15.32:0/3963412512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 msgr2=0x559380210280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97e00640 1 --2- 172.21.15.32:0/3963412512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 0x559380210280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97e00640 1 -- 172.21.15.32:0/3963412512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593801fb710 con 0x55937fe2a1b0 2024-08-31T07:28:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97400640 1 --2- 172.21.15.32:0/3963412512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 0x559380210280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97e00640 1 --2- 172.21.15.32:0/3963412512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801fe0d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4a88013010 tx=0x7f4a8801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d798364536383c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a8cc00640 1 -- 172.21.15.32:0/3963412512 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880043d0 con 0x55937fe2a1b0 2024-08-31T07:28:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a8cc00640 1 -- 172.21.15.32:0/3963412512 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a88005020 con 0x55937fe2a1b0 2024-08-31T07:28:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a8cc00640 1 -- 172.21.15.32:0/3963412512 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88004a70 con 0x55937fe2a1b0 2024-08-31T07:28:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- 172.21.15.32:0/3963412512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 msgr2=0x5593801fe0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 --2- 172.21.15.32:0/3963412512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801fe0d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4a88013010 tx=0x7f4a8801a000 comp rx=0 tx=0).stop 2024-08-31T07:28:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- 172.21.15.32:0/3963412512 shutdown_connections 2024-08-31T07:28:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 --2- 172.21.15.32:0/3963412512 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe53070 0x5593801f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 --2- 172.21.15.32:0/3963412512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe32390 0x559380210280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 --2- 172.21.15.32:0/3963412512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x5593801fe0d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- 172.21.15.32:0/3963412512 >> 172.21.15.32:0/3963412512 conn(0x55937fe47460 msgr2=0x5593801fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- 172.21.15.32:0/3963412512 shutdown_connections 2024-08-31T07:28:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- 172.21.15.32:0/3963412512 wait complete. 2024-08-31T07:28:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- start start 2024-08-31T07:28:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 Processor -- start 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x55938020c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380208c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe53070 0x559380209180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b2a60 con 0x55937fe2a1b0 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b2360 con 0x55937fe53070 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981af590 con 0x55937fe32390 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4aa6015740 10 client.0 _ll_register_callbacks cb 0x7f4a840c8630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.342+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x55938020c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x55938020c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34770/0 (socket says 172.21.15.32:34770) 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a97e00640 1 -- 172.21.15.32:0/1613957302 learned_addr learned my addr 172.21.15.32:0/1613957302 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a97400640 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380208c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a97400640 1 -- 172.21.15.32:0/1613957302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe53070 msgr2=0x559380209180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:28:31.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a97400640 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe53070 0x559380209180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a97400640 1 -- 172.21.15.32:0/1613957302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 msgr2=0x55938020c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:31.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a97400640 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x55938020c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a97400640 1 -- 172.21.15.32:0/1613957302 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593802096c0 con 0x55937fe32390 2024-08-31T07:28:31.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a97400640 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380208c40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4a840c81f0 tx=0x7f4a840cc8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2e3556f0a7143c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:31.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a82000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:31.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a82000640 1 -- 172.21.15.32:0/1613957302 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84049070 con 0x55937fe32390 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a82000640 1 -- 172.21.15.32:0/1613957302 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7f4a840318b0 con 0x55937fe32390 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4aa6015740 1 -- 172.21.15.32:0/1613957302 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a88005020 con 0x55937fe32390 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4a82000640 1 -- 172.21.15.32:0/1613957302 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f4a840b7020 con 0x55937fe32390 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4aa6015740 10 client.28615 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.347+0000 7f4aa6015740 1 -- 172.21.15.32:0/1613957302 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x559380281bf0 con 0x55937fe32390 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a82000640 1 -- 172.21.15.32:0/1613957302 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a88005020 con 0x55937fe32390 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a82000640 1 -- 172.21.15.32:0/1613957302 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7f4a840f1020 con 0x55937fe32390 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 10 client.28615 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 10 client.28615 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 20 client.28615 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55937fde6da0 0x559380284220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 4 client.28615 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 -- 172.21.15.32:0/1613957302 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55938016d1a0 con 0x55937fde6da0 2024-08-31T07:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a97e00640 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55937fde6da0 0x559380284220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a97e00640 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55937fde6da0 0x559380284220 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3640870433bf0fc0 server_cookie=12e80b99b5037d90 in_seq=0 out_seq=0 2024-08-31T07:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a82000640 10 client.28615 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a82000640 1 -- 172.21.15.32:0/1613957302 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55938016d1a0 con 0x55937fde6da0 2024-08-31T07:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a82000640 10 client.28615 handle_command_reply: tid=0 2024-08-31T07:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 client.28615 shutdown 2024-08-31T07:28:31.353 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:31 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 -- 172.21.15.32:0/1613957302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 msgr2=0x559380208c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380208c40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4a840c81f0 tx=0x7f4a840cc8c0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 -- 172.21.15.32:0/1613957302 shutdown_connections 2024-08-31T07:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55937fde6da0 0x559380284220 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55937fe53070 0x559380209180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55937fe32390 0x559380208c40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 --2- 172.21.15.32:0/1613957302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55937fe2a1b0 0x55938020c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 -- 172.21.15.32:0/1613957302 >> 172.21.15.32:0/1613957302 conn(0x55937fe47460 msgr2=0x5593801f6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a82000640 0 client.28615 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:28:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 -- 172.21.15.32:0/1613957302 shutdown_connections 2024-08-31T07:28:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 1 -- 172.21.15.32:0/1613957302 wait complete. 2024-08-31T07:28:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4aa6015740 20 client.28615 trim_cache size 0 max 16384 2024-08-31T07:28:31.355 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 -- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 msgr2=0x7f4a7c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 0x7f4a7c0b7230 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x5593801f7530 tx=0x7f4a88005f90 comp rx=0 tx=0).stop 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 -- 172.21.15.32:0/1140126710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a98075e40 msgr2=0x7f4a9819ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a98075e40 0x7f4a9819ca10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009230 tx=0x7f4a84003b90 comp rx=0 tx=0).stop 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 -- 172.21.15.32:0/1140126710 shutdown_connections 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4a7c0b4d20 0x7f4a7c0b7230 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a9810fec0 0x7f4a981a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a98075e40 0x7f4a9819ca10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 --2- 172.21.15.32:0/1140126710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a980754f0 0x7f4a9819c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 -- 172.21.15.32:0/1140126710 >> 172.21.15.32:0/1140126710 conn(0x7f4a9807b5f0 msgr2=0x7f4a98109a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:31.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 -- 172.21.15.32:0/1140126710 shutdown_connections 2024-08-31T07:28:31.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:31.348+0000 7f4a9f200640 1 -- 172.21.15.32:0/1140126710 wait complete. 2024-08-31T07:28:31.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:31 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:31.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:31 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:32.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 12 sec latency 29819.736 ms 2024-08-31T07:28:32.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:32 smithi032 ceph-mon[17252]: pgmap v861: 97 pgs: 97 active+clean; 882 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-08-31T07:28:32.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:28:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:32 smithi073 ceph-mon[21995]: pgmap v861: 97 pgs: 97 active+clean; 882 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-08-31T07:28:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:32 smithi084 ceph-mon[22387]: pgmap v861: 97 pgs: 97 active+clean; 882 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-08-31T07:28:32.810 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:33.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50041 0.00 MB/sec execute 13 sec latency 30819.797 ms 2024-08-31T07:28:33.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.815+0000 7f70b3e00640 1 -- 172.21.15.32:0/294838622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b4077120 msgr2=0x7f70b4075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:33.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.815+0000 7f70b3e00640 1 --2- 172.21.15.32:0/294838622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b4077120 0x7f70b4075550 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009900 tx=0x7f70a402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 -- 172.21.15.32:0/294838622 shutdown_connections 2024-08-31T07:28:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 --2- 172.21.15.32:0/294838622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b4114020 0x7f70b41164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 --2- 172.21.15.32:0/294838622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70b4075b70 0x7f70b4075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 --2- 172.21.15.32:0/294838622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b4077120 0x7f70b4075550 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 -- 172.21.15.32:0/294838622 >> 172.21.15.32:0/294838622 conn(0x7f70b40ff1e0 msgr2=0x7f70b4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 -- 172.21.15.32:0/294838622 shutdown_connections 2024-08-31T07:28:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 -- 172.21.15.32:0/294838622 wait complete. 2024-08-31T07:28:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 Processor -- start 2024-08-31T07:28:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 -- start start 2024-08-31T07:28:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b4075b70 0x7f70b41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70b4077120 0x7f70b41a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b4114020 0x7f70b41a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4119c60 con 0x7f70b4075b70 2024-08-31T07:28:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4119ae0 con 0x7f70b4077120 2024-08-31T07:28:33.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4119de0 con 0x7f70b4114020 2024-08-31T07:28:33.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b4075b70 0x7f70b41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b4075b70 0x7f70b41a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34786/0 (socket says 172.21.15.32:34786) 2024-08-31T07:28:33.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b2a00640 1 -- 172.21.15.32:0/744643790 learned_addr learned my addr 172.21.15.32:0/744643790 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:33.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3400640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b4114020 0x7f70b41a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b2a00640 1 -- 172.21.15.32:0/744643790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b4114020 msgr2=0x7f70b41a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b2a00640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b4114020 0x7f70b41a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b2a00640 1 -- 172.21.15.32:0/744643790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70b4077120 msgr2=0x7f70b41a0c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:28:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b2a00640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70b4077120 0x7f70b41a0c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b2a00640 1 -- 172.21.15.32:0/744643790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b41a65e0 con 0x7f70b4075b70 2024-08-31T07:28:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b2a00640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b4075b70 0x7f70b41a0670 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f70a4005940 tx=0x7f70a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70ab400640 1 -- 172.21.15.32:0/744643790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a403f030 con 0x7f70b4075b70 2024-08-31T07:28:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70ab400640 1 -- 172.21.15.32:0/744643790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a4003aa0 con 0x7f70b4075b70 2024-08-31T07:28:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 -- 172.21.15.32:0/744643790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b41a6870 con 0x7f70b4075b70 2024-08-31T07:28:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70ab400640 1 -- 172.21.15.32:0/744643790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4035070 con 0x7f70b4075b70 2024-08-31T07:28:33.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.817+0000 7f70b3e00640 1 -- 172.21.15.32:0/744643790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b41ad030 con 0x7f70b4075b70 2024-08-31T07:28:33.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70c03e9740 1 Processor -- start 2024-08-31T07:28:33.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70c03e9740 1 -- start start 2024-08-31T07:28:33.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d6899b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64cb1b0 0x5627d689a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64d3390 0x5627d689a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70c03e9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5627d68107b0 con 0x5627d64d3390 2024-08-31T07:28:33.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70c03e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5627d67ffd40 con 0x5627d64f4070 2024-08-31T07:28:33.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70c03e9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5627d6803cf0 con 0x5627d64cb1b0 2024-08-31T07:28:33.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70ab400640 1 -- 172.21.15.32:0/744643790 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f70a4042030 con 0x7f70b4075b70 2024-08-31T07:28:33.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70b2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64d3390 0x5627d689a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70b3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d6899b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70b3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d6899b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:51130/0 (socket says 172.21.15.32:51130) 2024-08-31T07:28:33.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70b3400640 1 -- 172.21.15.32:0/3181067928 learned_addr learned my addr 172.21.15.32:0/3181067928 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:33.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.823+0000 7f70b2a00640 1 --2- 172.21.15.32:0/3181067928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64cb1b0 0x5627d689a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 -- 172.21.15.32:0/3181067928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64cb1b0 msgr2=0x5627d689a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:33.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 --2- 172.21.15.32:0/3181067928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64cb1b0 0x5627d689a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 -- 172.21.15.32:0/3181067928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 msgr2=0x5627d6899b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:33.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 --2- 172.21.15.32:0/3181067928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d6899b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 -- 172.21.15.32:0/3181067928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d655f650 con 0x5627d64d3390 2024-08-31T07:28:33.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b3400640 1 --2- 172.21.15.32:0/3181067928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d6899b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:33.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 --2- 172.21.15.32:0/3181067928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64d3390 0x5627d689a5e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f70b406b730 tx=0x7f70a0009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab9bd3de5881d0c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:33.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f709f400640 1 -- 172.21.15.32:0/3181067928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a000ecb0 con 0x5627d64d3390 2024-08-31T07:28:33.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f709f400640 1 -- 172.21.15.32:0/3181067928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a000ee50 con 0x5627d64d3390 2024-08-31T07:28:33.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f709f400640 1 -- 172.21.15.32:0/3181067928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0004a00 con 0x5627d64d3390 2024-08-31T07:28:33.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70ab400640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70980b4da0 0x7f70980b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 -- 172.21.15.32:0/3181067928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64d3390 msgr2=0x5627d689a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:33.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 --2- 172.21.15.32:0/3181067928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64d3390 0x5627d689a5e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f70b406b730 tx=0x7f70a0009f30 comp rx=0 tx=0).stop 2024-08-31T07:28:33.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70ab400640 1 -- 172.21.15.32:0/744643790 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7f70a4105ae0 con 0x7f70b4075b70 2024-08-31T07:28:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b3400640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70980b4da0 0x7f70980b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 -- 172.21.15.32:0/3181067928 shutdown_connections 2024-08-31T07:28:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 --2- 172.21.15.32:0/3181067928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64d3390 0x5627d689a5e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 --2- 172.21.15.32:0/3181067928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64cb1b0 0x5627d689a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 --2- 172.21.15.32:0/3181067928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d6899b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 -- 172.21.15.32:0/3181067928 >> 172.21.15.32:0/3181067928 conn(0x5627d64e8460 msgr2=0x5627d6897a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 -- 172.21.15.32:0/3181067928 shutdown_connections 2024-08-31T07:28:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 -- 172.21.15.32:0/3181067928 wait complete. 2024-08-31T07:28:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b3400640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70980b4da0 0x7f70980b72b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009ca0 tx=0x7f70ac0097e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:33.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 -- start start 2024-08-31T07:28:33.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 Processor -- start 2024-08-31T07:28:33.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 0x5627d68a5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d68b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5627d6662700 con 0x5627d64cb1b0 2024-08-31T07:28:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5627d6786cf0 con 0x5627d64f4070 2024-08-31T07:28:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5627d68a8340 con 0x5627d64d3390 2024-08-31T07:28:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70c03e9740 10 client.0 _ll_register_callbacks cb 0x5627d67ac470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68a5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47924/0 (socket says 172.21.15.32:47924) 2024-08-31T07:28:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 -- 172.21.15.32:0/321083414 learned_addr learned my addr 172.21.15.32:0/321083414 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2a00640 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 0x5627d68a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b3400640 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d68b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 -- 172.21.15.32:0/321083414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 msgr2=0x5627d68b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:33.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d68b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 -- 172.21.15.32:0/321083414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 msgr2=0x5627d68a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 0x5627d68a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 -- 172.21.15.32:0/321083414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d67c78b0 con 0x5627d64d3390 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f70b2000640 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68a5ca0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f70a00055c0 tx=0x7f70a00052d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4d3a49732736e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.826+0000 7f7093e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f7093e00640 1 -- 172.21.15.32:0/321083414 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0004460 con 0x5627d64d3390 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 -- 172.21.15.32:0/321083414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a000ee50 con 0x5627d64d3390 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f7093e00640 1 -- 172.21.15.32:0/321083414 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7f70a00046c0 con 0x5627d64d3390 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f7093e00640 1 -- 172.21.15.32:0/321083414 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f70a00048c0 con 0x5627d64d3390 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 10 client.28627 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 -- 172.21.15.32:0/321083414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5627d68b43b0 con 0x5627d64d3390 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f7093e00640 1 -- 172.21.15.32:0/321083414 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a000ee50 con 0x5627d64d3390 2024-08-31T07:28:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f7093e00640 1 -- 172.21.15.32:0/321083414 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7f70a0003c90 con 0x5627d64d3390 2024-08-31T07:28:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 10 client.28627 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 10 client.28627 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:28:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 20 client.28627 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5627d64b29f0 0x5627d68b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 4 client.28627 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:28:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 -- 172.21.15.32:0/321083414 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627d6812700 con 0x5627d64b29f0 2024-08-31T07:28:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70b2a00640 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5627d64b29f0 0x5627d68b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f7093e00640 10 client.28627 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:28:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70b2a00640 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5627d64b29f0 0x5627d68b7770 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e2c76eb39b6ac3e server_cookie=bf2ba6d018fdbb0e in_seq=0 out_seq=0 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f7093e00640 1 -- 172.21.15.32:0/321083414 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627d6812700 con 0x5627d64b29f0 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f7093e00640 10 client.28627 handle_command_reply: tid=0 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 client.28627 shutdown 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 -- 172.21.15.32:0/321083414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 msgr2=0x5627d68a5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68a5ca0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f70a00055c0 tx=0x7f70a00052d0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 -- 172.21.15.32:0/321083414 shutdown_connections 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5627d64b29f0 0x5627d68b7770 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d68b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68a5ca0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 --2- 172.21.15.32:0/321083414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 0x5627d68a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f7093e00640 0 client.28627 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 -- 172.21.15.32:0/321083414 >> 172.21.15.32:0/321083414 conn(0x5627d64e8460 msgr2=0x5627d689ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 -- 172.21.15.32:0/321083414 shutdown_connections 2024-08-31T07:28:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 1 -- 172.21.15.32:0/321083414 wait complete. 2024-08-31T07:28:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.832+0000 7f70c03e9740 20 client.28627 trim_cache size 0 max 16384 2024-08-31T07:28:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 Processor -- start 2024-08-31T07:28:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 -- start start 2024-08-31T07:28:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64cb1b0 0x5627d689d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64d3390 0x5627d68b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64f4070 0x5627d689b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41b5810 con 0x5627d64f4070 2024-08-31T07:28:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41b7470 con 0x5627d64d3390 2024-08-31T07:28:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41b5ee0 con 0x5627d64cb1b0 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64f4070 0x5627d689b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64d3390 0x5627d68b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64f4070 0x5627d689b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34822/0 (socket says 172.21.15.32:34822) 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b3400640 1 -- 172.21.15.32:0/1570553415 learned_addr learned my addr 172.21.15.32:0/1570553415 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b2a00640 1 --2- 172.21.15.32:0/1570553415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64cb1b0 0x5627d689d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b2000640 1 -- 172.21.15.32:0/1570553415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64cb1b0 msgr2=0x5627d689d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b2000640 1 --2- 172.21.15.32:0/1570553415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64cb1b0 0x5627d689d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b2000640 1 -- 172.21.15.32:0/1570553415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64f4070 msgr2=0x5627d689b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b2000640 1 --2- 172.21.15.32:0/1570553415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64f4070 0x5627d689b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b2000640 1 -- 172.21.15.32:0/1570553415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d689be50 con 0x5627d64d3390 2024-08-31T07:28:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70b2000640 1 --2- 172.21.15.32:0/1570553415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64d3390 0x5627d68b15d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009a70 tx=0x7f70a0009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ec1a9f75e21346 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f709ea00640 1 -- 172.21.15.32:0/1570553415 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a000ee30 con 0x5627d64d3390 2024-08-31T07:28:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f709ea00640 1 -- 172.21.15.32:0/1570553415 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a00096e0 con 0x5627d64d3390 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f709ea00640 1 -- 172.21.15.32:0/1570553415 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a00044c0 con 0x5627d64d3390 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 -- 172.21.15.32:0/1570553415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64d3390 msgr2=0x5627d68b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 --2- 172.21.15.32:0/1570553415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64d3390 0x5627d68b15d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009a70 tx=0x7f70a0009530 comp rx=0 tx=0).stop 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 -- 172.21.15.32:0/1570553415 shutdown_connections 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 --2- 172.21.15.32:0/1570553415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64f4070 0x5627d689b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 --2- 172.21.15.32:0/1570553415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64d3390 0x5627d68b15d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 --2- 172.21.15.32:0/1570553415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64cb1b0 0x5627d689d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 -- 172.21.15.32:0/1570553415 >> 172.21.15.32:0/1570553415 conn(0x5627d64e8460 msgr2=0x5627d68a6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 -- 172.21.15.32:0/1570553415 shutdown_connections 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 -- 172.21.15.32:0/1570553415 wait complete. 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 -- start start 2024-08-31T07:28:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 Processor -- start 2024-08-31T07:28:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 0x5627d68b9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68b9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.849+0000 7f70c03e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d6920ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70c03e9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41b6ce0 con 0x5627d64cb1b0 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70c03e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41b65e0 con 0x5627d64f4070 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70c03e9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41b3810 con 0x5627d64d3390 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70c03e9740 10 client.0 _ll_register_callbacks cb 0x7f70a00099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 0x5627d68b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68b9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47940/0 (socket says 172.21.15.32:47940) 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b2000640 1 -- 172.21.15.32:0/1833792536 learned_addr learned my addr 172.21.15.32:0/1833792536 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b3400640 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d6920ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b2000640 1 -- 172.21.15.32:0/1833792536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 msgr2=0x5627d6920ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b2000640 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d6920ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b2000640 1 -- 172.21.15.32:0/1833792536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 msgr2=0x5627d68b9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b2000640 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 0x5627d68b9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b2000640 1 -- 172.21.15.32:0/1833792536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d6921560 con 0x5627d64d3390 2024-08-31T07:28:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.855+0000 7f70b2a00640 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 0x5627d68b9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:33.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b2000640 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68b9600 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f70a00059b0 tx=0x7f70a002ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fb62a06822a2d9e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:33.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f7093e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:33.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f7093e00640 1 -- 172.21.15.32:0/1833792536 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a002e720 con 0x5627d64d3390 2024-08-31T07:28:33.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f7093e00640 1 -- 172.21.15.32:0/1833792536 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7f70a002e8c0 con 0x5627d64d3390 2024-08-31T07:28:33.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 -- 172.21.15.32:0/1833792536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a00096e0 con 0x5627d64d3390 2024-08-31T07:28:33.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f7093e00640 1 -- 172.21.15.32:0/1833792536 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f70a0044020 con 0x5627d64d3390 2024-08-31T07:28:33.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 10 client.28633 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:33.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 -- 172.21.15.32:0/1833792536 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5627d69227e0 con 0x5627d64d3390 2024-08-31T07:28:33.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f7093e00640 1 -- 172.21.15.32:0/1833792536 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a00096e0 con 0x5627d64d3390 2024-08-31T07:28:33.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f7093e00640 1 -- 172.21.15.32:0/1833792536 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7f70a00035f0 con 0x5627d64d3390 2024-08-31T07:28:33.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 10 client.28633 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:33.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 10 client.28633 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:28:33.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 20 client.28633 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:33.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5627d6487da0 0x5627d6924f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:33.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 4 client.28633 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:28:33.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 -- 172.21.15.32:0/1833792536 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627d680e1a0 con 0x5627d6487da0 2024-08-31T07:28:33.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b2a00640 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5627d6487da0 0x5627d6924f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:33.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b2a00640 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5627d6487da0 0x5627d6924f60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c7db38dff37c6e3e server_cookie=e271bfc92a090583 in_seq=0 out_seq=0 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f7093e00640 10 client.28633 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f7093e00640 1 -- 172.21.15.32:0/1833792536 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627d680e1a0 con 0x5627d6487da0 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f7093e00640 10 client.28633 handle_command_reply: tid=0 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 client.28633 shutdown 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 -- 172.21.15.32:0/1833792536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 msgr2=0x5627d68b9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68b9600 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f70a00059b0 tx=0x7f70a002ec30 comp rx=0 tx=0).stop 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 -- 172.21.15.32:0/1833792536 shutdown_connections 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5627d6487da0 0x5627d6924f60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5627d64f4070 0x5627d6920ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5627d64d3390 0x5627d68b9600 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 --2- 172.21.15.32:0/1833792536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5627d64cb1b0 0x5627d68b9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 -- 172.21.15.32:0/1833792536 >> 172.21.15.32:0/1833792536 conn(0x5627d64e8460 msgr2=0x5627d689b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:33.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f7093e00640 0 client.28633 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:28:33.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 -- 172.21.15.32:0/1833792536 shutdown_connections 2024-08-31T07:28:33.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 1 -- 172.21.15.32:0/1833792536 wait complete. 2024-08-31T07:28:33.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70c03e9740 20 client.28633 trim_cache size 0 max 16384 2024-08-31T07:28:33.868 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:28:33.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 -- 172.21.15.32:0/744643790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70980b4da0 msgr2=0x7f70980b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:33.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70980b4da0 0x7f70980b72b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009ca0 tx=0x7f70ac0097e0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 -- 172.21.15.32:0/744643790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b4075b70 msgr2=0x7f70b41a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:33.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b4075b70 0x7f70b41a0670 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f70a4005940 tx=0x7f70a4004270 comp rx=0 tx=0).stop 2024-08-31T07:28:33.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 -- 172.21.15.32:0/744643790 shutdown_connections 2024-08-31T07:28:33.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f70980b4da0 0x7f70980b72b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70b4114020 0x7f70b41a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70b4077120 0x7f70b41a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 --2- 172.21.15.32:0/744643790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70b4075b70 0x7f70b41a0670 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:33.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 -- 172.21.15.32:0/744643790 >> 172.21.15.32:0/744643790 conn(0x7f70b40ff1e0 msgr2=0x7f70b410ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:33.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 -- 172.21.15.32:0/744643790 shutdown_connections 2024-08-31T07:28:33.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:33.861+0000 7f70b3e00640 1 -- 172.21.15.32:0/744643790 wait complete. 2024-08-31T07:28:34.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50048 0.04 MB/sec execute 14 sec latency 31771.137 ms 2024-08-31T07:28:34.455 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:34 smithi032 ceph-mon[17252]: pgmap v862: 97 pgs: 97 active+clean; 877 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 22 op/s 2024-08-31T07:28:34.455 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:34 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:28:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:34 smithi073 ceph-mon[21995]: pgmap v862: 97 pgs: 97 active+clean; 877 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 22 op/s 2024-08-31T07:28:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:34 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:28:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:34 smithi084 ceph-mon[22387]: pgmap v862: 97 pgs: 97 active+clean; 877 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 22 op/s 2024-08-31T07:28:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:34 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:28:35.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50199 0.80 MB/sec execute 15 sec latency 32771.181 ms 2024-08-31T07:28:35.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:28:35.727 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:36.343 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50199 0.75 MB/sec execute 16 sec latency 33771.227 ms 2024-08-31T07:28:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.403+0000 7fa348a00640 1 -- 172.21.15.32:0/3853475891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa344104770 msgr2=0x7fa344104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.403+0000 7fa348a00640 1 --2- 172.21.15.32:0/3853475891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa344104770 0x7fa344104b80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa334009860 tx=0x7fa33402a340 comp rx=0 tx=0).stop 2024-08-31T07:28:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.403+0000 7fa348a00640 1 -- 172.21.15.32:0/3853475891 shutdown_connections 2024-08-31T07:28:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.403+0000 7fa348a00640 1 --2- 172.21.15.32:0/3853475891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa34410dbd0 0x7fa3441100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.403+0000 7fa348a00640 1 --2- 172.21.15.32:0/3853475891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3441050c0 0x7fa34410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.403+0000 7fa348a00640 1 --2- 172.21.15.32:0/3853475891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa344104770 0x7fa344104b80 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.403+0000 7fa348a00640 1 -- 172.21.15.32:0/3853475891 >> 172.21.15.32:0/3853475891 conn(0x7fa3440794e0 msgr2=0x7fa3440798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.403+0000 7fa348a00640 1 -- 172.21.15.32:0/3853475891 shutdown_connections 2024-08-31T07:28:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.403+0000 7fa348a00640 1 -- 172.21.15.32:0/3853475891 wait complete. 2024-08-31T07:28:36.406 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:36 smithi032 ceph-mon[17252]: pgmap v863: 97 pgs: 97 active+clean; 868 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 291 KiB/s wr, 35 op/s 2024-08-31T07:28:36.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.403+0000 7fa348a00640 1 Processor -- start 2024-08-31T07:28:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa348a00640 1 -- start start 2024-08-31T07:28:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa348a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa344104770 0x7fa34419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa348a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3441050c0 0x7fa34419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa348a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa34410dbd0 0x7fa3441a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa348a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa344113850 con 0x7fa3441050c0 2024-08-31T07:28:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa348a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441136d0 con 0x7fa344104770 2024-08-31T07:28:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa348a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441139d0 con 0x7fa34410dbd0 2024-08-31T07:28:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa341e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa344104770 0x7fa34419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa341e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa344104770 0x7fa34419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59370/0 (socket says 172.21.15.32:59370) 2024-08-31T07:28:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa341e00640 1 -- 172.21.15.32:0/3604845722 learned_addr learned my addr 172.21.15.32:0/3604845722 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:36.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa341400640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3441050c0 0x7fa34419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa341e00640 1 -- 172.21.15.32:0/3604845722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa34410dbd0 msgr2=0x7fa3441a1cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:28:36.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa341e00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa34410dbd0 0x7fa3441a1cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa341e00640 1 -- 172.21.15.32:0/3604845722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3441050c0 msgr2=0x7fa34419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:36.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa341e00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3441050c0 0x7fa34419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa341e00640 1 -- 172.21.15.32:0/3604845722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3441a23f0 con 0x7fa344104770 2024-08-31T07:28:36.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa341e00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa344104770 0x7fa34419c480 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa334035040 tx=0x7fa334032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:36.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa33aa00640 1 -- 172.21.15.32:0/3604845722 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334037040 con 0x7fa344104770 2024-08-31T07:28:36.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa33aa00640 1 -- 172.21.15.32:0/3604845722 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa334041070 con 0x7fa344104770 2024-08-31T07:28:36.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa33aa00640 1 -- 172.21.15.32:0/3604845722 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334003850 con 0x7fa344104770 2024-08-31T07:28:36.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa348a00640 1 -- 172.21.15.32:0/3604845722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3441a2680 con 0x7fa344104770 2024-08-31T07:28:36.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.405+0000 7fa348a00640 1 -- 172.21.15.32:0/3604845722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3441a2ab0 con 0x7fa344104770 2024-08-31T07:28:36.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa34f6ae740 1 Processor -- start 2024-08-31T07:28:36.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa34f6ae740 1 -- start start 2024-08-31T07:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c26070 0x557ea4fcb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4bfd1b0 0x557ea4fcbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c05390 0x557ea4fcc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557ea4f427b0 con 0x557ea4c26070 2024-08-31T07:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557ea4f31d40 con 0x557ea4c05390 2024-08-31T07:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557ea4f35cf0 con 0x557ea4bfd1b0 2024-08-31T07:28:36.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa342800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c26070 0x557ea4fcb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa342800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c26070 0x557ea4fcb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33390/0 (socket says 172.21.15.32:33390) 2024-08-31T07:28:36.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa342800640 1 -- 172.21.15.32:0/1149812112 learned_addr learned my addr 172.21.15.32:0/1149812112 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:36.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa342800640 1 -- 172.21.15.32:0/1149812112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4bfd1b0 msgr2=0x557ea4fcbed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:28:36.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa342800640 1 --2- 172.21.15.32:0/1149812112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4bfd1b0 0x557ea4fcbed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa342800640 1 -- 172.21.15.32:0/1149812112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c05390 msgr2=0x557ea4fcc410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:28:36.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa342800640 1 --2- 172.21.15.32:0/1149812112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c05390 0x557ea4fcc410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa342800640 1 -- 172.21.15.32:0/1149812112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea4c91650 con 0x557ea4c26070 2024-08-31T07:28:36.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa33aa00640 1 -- 172.21.15.32:0/3604845722 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa334004050 con 0x7fa344104770 2024-08-31T07:28:36.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa342800640 1 --2- 172.21.15.32:0/1149812112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c26070 0x557ea4fcb990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009e30 tx=0x7fa33c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607a96b49e8f98e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:36.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa32f000640 1 -- 172.21.15.32:0/1149812112 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c004d20 con 0x557ea4c26070 2024-08-31T07:28:36.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa32f000640 1 -- 172.21.15.32:0/1149812112 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c00ee70 con 0x557ea4c26070 2024-08-31T07:28:36.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.411+0000 7fa32f000640 1 -- 172.21.15.32:0/1149812112 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c010740 con 0x557ea4c26070 2024-08-31T07:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- 172.21.15.32:0/1149812112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c26070 msgr2=0x557ea4fcb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/1149812112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c26070 0x557ea4fcb990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009e30 tx=0x7fa33c00e720 comp rx=0 tx=0).stop 2024-08-31T07:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa33aa00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3280b4ca0 0x7fa3280b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa33aa00640 1 -- 172.21.15.32:0/3604845722 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fa334104da0 con 0x7fa344104770 2024-08-31T07:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- 172.21.15.32:0/1149812112 shutdown_connections 2024-08-31T07:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/1149812112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c05390 0x557ea4fcc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/1149812112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4bfd1b0 0x557ea4fcbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/1149812112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c26070 0x557ea4fcb990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341e00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3280b4ca0 0x7fa3280b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- 172.21.15.32:0/1149812112 >> 172.21.15.32:0/1149812112 conn(0x557ea4c1a460 msgr2=0x557ea4fc9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- 172.21.15.32:0/1149812112 shutdown_connections 2024-08-31T07:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- 172.21.15.32:0/1149812112 wait complete. 2024-08-31T07:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341e00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3280b4ca0 0x7fa3280b71b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x557ea4fca610 tx=0x7fa3340303b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- start start 2024-08-31T07:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 Processor -- start 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4bfd1b0 0x557ea4fd74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c05390 0x557ea4fd7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c26070 0x557ea4fe3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557ea4d94700 con 0x557ea4c05390 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557ea4eb8cf0 con 0x557ea4bfd1b0 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557ea4fda170 con 0x557ea4c26070 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 10 client.0 _ll_register_callbacks cb 0x557ea4ede470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c05390 0x557ea4fd7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa342800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4bfd1b0 0x557ea4fd74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c05390 0x557ea4fd7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33400/0 (socket says 172.21.15.32:33400) 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341400640 1 -- 172.21.15.32:0/3807568233 learned_addr learned my addr 172.21.15.32:0/3807568233 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa342800640 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c26070 0x557ea4fe3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341400640 1 -- 172.21.15.32:0/3807568233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c26070 msgr2=0x557ea4fe3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341400640 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c26070 0x557ea4fe3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341400640 1 -- 172.21.15.32:0/3807568233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4bfd1b0 msgr2=0x557ea4fd74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341400640 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4bfd1b0 0x557ea4fd74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341400640 1 -- 172.21.15.32:0/3807568233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea4ef98b0 con 0x557ea4c05390 2024-08-31T07:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa342800640 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c26070 0x557ea4fe3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa341400640 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c05390 0x557ea4fd7a10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa3300116c0 tx=0x7fa330011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbe65fcfb3d57b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa323400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:28:36.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa323400640 1 -- 172.21.15.32:0/3807568233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33001e060 con 0x557ea4c05390 2024-08-31T07:28:36.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- 172.21.15.32:0/3807568233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa33c00ee70 con 0x557ea4c05390 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa323400640 1 -- 172.21.15.32:0/3807568233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fa33000f840 con 0x557ea4c05390 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa323400640 1 -- 172.21.15.32:0/3807568233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa33000fa40 con 0x557ea4c05390 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 10 client.18621 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 1 -- 172.21.15.32:0/3807568233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x557ea4fe6190 con 0x557ea4c05390 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa323400640 1 -- 172.21.15.32:0/3807568233 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c00ee70 con 0x557ea4c05390 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa323400640 1 -- 172.21.15.32:0/3807568233 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fa330019410 con 0x557ea4c05390 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 10 client.18621 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.413+0000 7fa34f6ae740 10 client.18621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 20 client.18621 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557ea4be49f0 0x557ea4fe9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 4 client.18621 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 -- 172.21.15.32:0/3807568233 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ea4f44700 con 0x557ea4be49f0 2024-08-31T07:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa342800640 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557ea4be49f0 0x557ea4fe9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa342800640 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557ea4be49f0 0x557ea4fe9620 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25879095b82cf582 server_cookie=661868d380b4389b in_seq=0 out_seq=0 2024-08-31T07:28:36.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa323400640 10 client.18621 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:28:36.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa323400640 1 -- 172.21.15.32:0/3807568233 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ea4f44700 con 0x557ea4be49f0 2024-08-31T07:28:36.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa323400640 10 client.18621 handle_command_reply: tid=0 2024-08-31T07:28:36.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 client.18621 shutdown 2024-08-31T07:28:36.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 -- 172.21.15.32:0/3807568233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c05390 msgr2=0x557ea4fd7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:36.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c05390 0x557ea4fd7a10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa3300116c0 tx=0x7fa330011ac0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 -- 172.21.15.32:0/3807568233 shutdown_connections 2024-08-31T07:28:36.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557ea4be49f0 0x557ea4fe9620 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c26070 0x557ea4fe3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4c05390 0x557ea4fd7a10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/3807568233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4bfd1b0 0x557ea4fd74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 -- 172.21.15.32:0/3807568233 >> 172.21.15.32:0/3807568233 conn(0x557ea4c1a460 msgr2=0x557ea4fceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa323400640 0 client.18621 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:28:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 -- 172.21.15.32:0/3807568233 shutdown_connections 2024-08-31T07:28:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 1 -- 172.21.15.32:0/3807568233 wait complete. 2024-08-31T07:28:36.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.419+0000 7fa34f6ae740 20 client.18621 trim_cache size 0 max 16384 2024-08-31T07:28:36.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.431+0000 7fa34f6ae740 1 Processor -- start 2024-08-31T07:28:36.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- start start 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fc8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 0x557ea4fc9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c26070 0x557ea4fd44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441b1620 con 0x557ea4bfd1b0 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441b3280 con 0x557ea4c26070 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441b1cf0 con 0x557ea4c05390 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fc8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa341400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 0x557ea4fc9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fc8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33412/0 (socket says 172.21.15.32:33412) 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 -- 172.21.15.32:0/774249041 learned_addr learned my addr 172.21.15.32:0/774249041 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 --2- 172.21.15.32:0/774249041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c26070 0x557ea4fd44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa341400640 1 -- 172.21.15.32:0/774249041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c26070 msgr2=0x557ea4fd44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:36.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa341400640 1 --2- 172.21.15.32:0/774249041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c26070 0x557ea4fd44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa341400640 1 -- 172.21.15.32:0/774249041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 msgr2=0x557ea4fc8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:36.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa341400640 1 --2- 172.21.15.32:0/774249041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fc8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa341400640 1 -- 172.21.15.32:0/774249041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea4be4c20 con 0x557ea4c05390 2024-08-31T07:28:36.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa341400640 1 --2- 172.21.15.32:0/774249041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 0x557ea4fc9370 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa330009230 tx=0x7fa330009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b300259c514cb98a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:36.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa32e600640 1 -- 172.21.15.32:0/774249041 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330019410 con 0x557ea4c05390 2024-08-31T07:28:36.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa32e600640 1 -- 172.21.15.32:0/774249041 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3300195b0 con 0x557ea4c05390 2024-08-31T07:28:36.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa32e600640 1 -- 172.21.15.32:0/774249041 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330005290 con 0x557ea4c05390 2024-08-31T07:28:36.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- 172.21.15.32:0/774249041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 msgr2=0x557ea4fc9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:36.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/774249041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 0x557ea4fc9370 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa330009230 tx=0x7fa330009c30 comp rx=0 tx=0).stop 2024-08-31T07:28:36.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- 172.21.15.32:0/774249041 shutdown_connections 2024-08-31T07:28:36.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/774249041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c26070 0x557ea4fd44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/774249041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 0x557ea4fc9370 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/774249041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fc8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- 172.21.15.32:0/774249041 >> 172.21.15.32:0/774249041 conn(0x557ea4c1a460 msgr2=0x557ea4fd5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:36.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- 172.21.15.32:0/774249041 shutdown_connections 2024-08-31T07:28:36.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- 172.21.15.32:0/774249041 wait complete. 2024-08-31T07:28:36.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- start start 2024-08-31T07:28:36.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 Processor -- start 2024-08-31T07:28:36.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fdba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 0x557ea4fdbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c26070 0x557ea5053020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441b2af0 con 0x557ea4bfd1b0 2024-08-31T07:28:36.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441b23f0 con 0x557ea4c26070 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441af620 con 0x557ea4c05390 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa34f6ae740 10 client.0 _ll_register_callbacks cb 0x7fa330009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fdba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fdba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33422/0 (socket says 172.21.15.32:33422) 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 -- 172.21.15.32:0/657073914 learned_addr learned my addr 172.21.15.32:0/657073914 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa341400640 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 0x557ea4fdbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 -- 172.21.15.32:0/657073914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 msgr2=0x557ea4fdbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 0x557ea4fdbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 -- 172.21.15.32:0/657073914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c26070 msgr2=0x557ea5053020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c26070 0x557ea5053020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.436+0000 7fa342800640 1 -- 172.21.15.32:0/657073914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea5053560 con 0x557ea4bfd1b0 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa342800640 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fdba90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009e30 tx=0x7fa33c005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35288bba9d398d3c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:36.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa323400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:28:36.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa323400640 1 -- 172.21.15.32:0/657073914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0106e0 con 0x557ea4bfd1b0 2024-08-31T07:28:36.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa323400640 1 -- 172.21.15.32:0/657073914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fa33c01d040 con 0x557ea4bfd1b0 2024-08-31T07:28:36.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 -- 172.21.15.32:0/657073914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3300195b0 con 0x557ea4bfd1b0 2024-08-31T07:28:36.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa323400640 1 -- 172.21.15.32:0/657073914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa33c03a030 con 0x557ea4bfd1b0 2024-08-31T07:28:36.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 10 client.18627 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:36.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 -- 172.21.15.32:0/657073914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x557ea50547a0 con 0x557ea4bfd1b0 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa323400640 1 -- 172.21.15.32:0/657073914 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3300195b0 con 0x557ea4bfd1b0 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa323400640 1 -- 172.21.15.32:0/657073914 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fa33c004ff0 con 0x557ea4bfd1b0 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 10 client.18627 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 10 client.18627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 20 client.18627 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557ea4bb9da0 0x557ea5056e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 4 client.18627 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 -- 172.21.15.32:0/657073914 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ea4f401a0 con 0x557ea4bb9da0 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa341400640 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557ea4bb9da0 0x557ea5056e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa341400640 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557ea4bb9da0 0x557ea5056e30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a88edf3f558c7f4c server_cookie=eab0d60dd4aa8b45 in_seq=0 out_seq=0 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa323400640 10 client.18627 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:28:36.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa323400640 1 -- 172.21.15.32:0/657073914 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ea4f401a0 con 0x557ea4bb9da0 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa323400640 10 client.18627 handle_command_reply: tid=0 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 client.18627 shutdown 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 -- 172.21.15.32:0/657073914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 msgr2=0x557ea4fdba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fdba90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009e30 tx=0x7fa33c005980 comp rx=0 tx=0).stop 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 -- 172.21.15.32:0/657073914 shutdown_connections 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x557ea4bb9da0 0x557ea5056e30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ea4c26070 0x557ea5053020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ea4c05390 0x557ea4fdbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 --2- 172.21.15.32:0/657073914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557ea4bfd1b0 0x557ea4fdba90 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 -- 172.21.15.32:0/657073914 >> 172.21.15.32:0/657073914 conn(0x557ea4c1a460 msgr2=0x557ea4fcea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa323400640 0 client.18627 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 -- 172.21.15.32:0/657073914 shutdown_connections 2024-08-31T07:28:36.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 1 -- 172.21.15.32:0/657073914 wait complete. 2024-08-31T07:28:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa34f6ae740 20 client.18627 trim_cache size 0 max 16384 2024-08-31T07:28:36.448 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 -- 172.21.15.32:0/3604845722 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3280b4ca0 msgr2=0x7fa3280b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3280b4ca0 0x7fa3280b71b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x557ea4fca610 tx=0x7fa3340303b0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 -- 172.21.15.32:0/3604845722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa344104770 msgr2=0x7fa34419c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa344104770 0x7fa34419c480 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa334035040 tx=0x7fa334032d50 comp rx=0 tx=0).stop 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 -- 172.21.15.32:0/3604845722 shutdown_connections 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3280b4ca0 0x7fa3280b71b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa34410dbd0 0x7fa3441a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3441050c0 0x7fa34419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 --2- 172.21.15.32:0/3604845722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa344104770 0x7fa34419c480 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 -- 172.21.15.32:0/3604845722 >> 172.21.15.32:0/3604845722 conn(0x7fa3440794e0 msgr2=0x7fa34410cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 -- 172.21.15.32:0/3604845722 shutdown_connections 2024-08-31T07:28:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:36.441+0000 7fa348a00640 1 -- 172.21.15.32:0/3604845722 wait complete. 2024-08-31T07:28:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:36 smithi073 ceph-mon[21995]: pgmap v863: 97 pgs: 97 active+clean; 868 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 291 KiB/s wr, 35 op/s 2024-08-31T07:28:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:36 smithi084 ceph-mon[22387]: pgmap v863: 97 pgs: 97 active+clean; 868 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 291 KiB/s wr, 35 op/s 2024-08-31T07:28:37.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50200 0.70 MB/sec execute 17 sec latency 34771.529 ms 2024-08-31T07:28:37.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:28:37.653 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-31T07:28:37.654 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.25 2024-08-31T07:28:37.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:28:37.937 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:38.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50269 0.84 MB/sec execute 18 sec latency 35353.908 ms 2024-08-31T07:28:38.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:38 smithi032 ceph-mon[17252]: pgmap v864: 97 pgs: 97 active+clean; 863 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 519 KiB/s wr, 58 op/s 2024-08-31T07:28:38.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:28:38.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:28:38.574 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:38 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:28:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:38 smithi073 ceph-mon[21995]: pgmap v864: 97 pgs: 97 active+clean; 863 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 519 KiB/s wr, 58 op/s 2024-08-31T07:28:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:28:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:28:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:38 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:28:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 -- 172.21.15.32:0/2667388166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c075b70 msgr2=0x7fc86c075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 --2- 172.21.15.32:0/2667388166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c075b70 0x7fc86c075f80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc858009860 tx=0x7fc85802a340 comp rx=0 tx=0).stop 2024-08-31T07:28:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 -- 172.21.15.32:0/2667388166 shutdown_connections 2024-08-31T07:28:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 --2- 172.21.15.32:0/2667388166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c114020 0x7fc86c1164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 --2- 172.21.15.32:0/2667388166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c075b70 0x7fc86c075f80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 --2- 172.21.15.32:0/2667388166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c077120 0x7fc86c075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 -- 172.21.15.32:0/2667388166 >> 172.21.15.32:0/2667388166 conn(0x7fc86c0ff1e0 msgr2=0x7fc86c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 -- 172.21.15.32:0/2667388166 shutdown_connections 2024-08-31T07:28:38.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 -- 172.21.15.32:0/2667388166 wait complete. 2024-08-31T07:28:38.719 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:38.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 Processor -- start 2024-08-31T07:28:38.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 -- start start 2024-08-31T07:28:38.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c075b70 0x7fc86c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c077120 0x7fc86c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c114020 0x7fc86c06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c119c60 con 0x7fc86c114020 2024-08-31T07:28:38.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c075b70 0x7fc86c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c075b70 0x7fc86c074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59416/0 (socket says 172.21.15.32:59416) 2024-08-31T07:28:38.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86b400640 1 -- 172.21.15.32:0/2100368437 learned_addr learned my addr 172.21.15.32:0/2100368437 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:38.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c119ae0 con 0x7fc86c075b70 2024-08-31T07:28:38.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc870a00640 1 -- 172.21.15.32:0/2100368437 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c119de0 con 0x7fc86c077120 2024-08-31T07:28:38.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86aa00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c077120 0x7fc86c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86be00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c114020 0x7fc86c06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86b400640 1 -- 172.21.15.32:0/2100368437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c077120 msgr2=0x7fc86c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:38.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86b400640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c077120 0x7fc86c06df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86b400640 1 -- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c114020 msgr2=0x7fc86c06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86b400640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c114020 0x7fc86c06e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86b400640 1 -- 172.21.15.32:0/2100368437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86c073910 con 0x7fc86c075b70 2024-08-31T07:28:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86be00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c114020 0x7fc86c06e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:28:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.715+0000 7fc86b400640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c075b70 0x7fc86c074ca0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00d770 tx=0x7fc85c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.721+0000 7fc863e00640 1 -- 172.21.15.32:0/2100368437 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c017070 con 0x7fc86c075b70 2024-08-31T07:28:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.721+0000 7fc863e00640 1 -- 172.21.15.32:0/2100368437 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc85c0054d0 con 0x7fc86c075b70 2024-08-31T07:28:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.721+0000 7fc863e00640 1 -- 172.21.15.32:0/2100368437 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c004d50 con 0x7fc86c075b70 2024-08-31T07:28:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.721+0000 7fc870a00640 1 -- 172.21.15.32:0/2100368437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86c073ba0 con 0x7fc86c075b70 2024-08-31T07:28:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.721+0000 7fc870a00640 1 -- 172.21.15.32:0/2100368437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86c074060 con 0x7fc86c075b70 2024-08-31T07:28:38.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.721+0000 7fc863e00640 1 -- 172.21.15.32:0/2100368437 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc85c009c30 con 0x7fc86c075b70 2024-08-31T07:28:38.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.726+0000 7fc878c15740 1 Processor -- start 2024-08-31T07:28:38.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.726+0000 7fc878c15740 1 -- start start 2024-08-31T07:28:38.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.726+0000 7fc878c15740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688605e070 0x556886403b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.726+0000 7fc878c15740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 0x5568864040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.726+0000 7fc878c15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688603d390 0x5568864045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.726+0000 7fc878c15740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55688637a7b0 con 0x5568860351b0 2024-08-31T07:28:38.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.726+0000 7fc878c15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556886369d40 con 0x55688603d390 2024-08-31T07:28:38.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.726+0000 7fc878c15740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55688636dcf0 con 0x55688605e070 2024-08-31T07:28:38.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc863e00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8500b4d20 0x7fc8500b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 0x5568864040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 0x5568864040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33454/0 (socket says 172.21.15.32:33454) 2024-08-31T07:28:38.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86b400640 1 -- 172.21.15.32:0/1880325539 learned_addr learned my addr 172.21.15.32:0/1880325539 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:38.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86aa00640 1 --2- 172.21.15.32:0/1880325539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688603d390 0x5568864045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86be00640 1 --2- 172.21.15.32:0/1880325539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688605e070 0x556886403b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc863e00640 1 -- 172.21.15.32:0/2100368437 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fc85c0e4ee0 con 0x7fc86c075b70 2024-08-31T07:28:38.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86be00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8500b4d20 0x7fc8500b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86be00640 1 -- 172.21.15.32:0/1880325539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688603d390 msgr2=0x5568864045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:38.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86be00640 1 --2- 172.21.15.32:0/1880325539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688603d390 0x5568864045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86be00640 1 -- 172.21.15.32:0/1880325539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 msgr2=0x5568864040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:38.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86be00640 1 --2- 172.21.15.32:0/1880325539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 0x5568864040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86be00640 1 -- 172.21.15.32:0/1880325539 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568860c9650 con 0x55688605e070 2024-08-31T07:28:38.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86be00640 1 --2- 172.21.15.32:0/1880325539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688605e070 0x556886403b60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc864005fd0 tx=0x7fc864010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b146c66c7616ecc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:38.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc86be00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8500b4d20 0x7fc8500b7230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc864010c40 tx=0x7fc86401d000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:38.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc857e00640 1 -- 172.21.15.32:0/1880325539 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8640128b0 con 0x55688605e070 2024-08-31T07:28:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc857e00640 1 -- 172.21.15.32:0/1880325539 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc864009da0 con 0x55688605e070 2024-08-31T07:28:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc857e00640 1 -- 172.21.15.32:0/1880325539 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864005420 con 0x55688605e070 2024-08-31T07:28:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 -- 172.21.15.32:0/1880325539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688605e070 msgr2=0x556886403b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 --2- 172.21.15.32:0/1880325539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688605e070 0x556886403b60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc864005fd0 tx=0x7fc864010740 comp rx=0 tx=0).stop 2024-08-31T07:28:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 -- 172.21.15.32:0/1880325539 shutdown_connections 2024-08-31T07:28:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 --2- 172.21.15.32:0/1880325539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688603d390 0x5568864045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 --2- 172.21.15.32:0/1880325539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 0x5568864040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 --2- 172.21.15.32:0/1880325539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688605e070 0x556886403b60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 -- 172.21.15.32:0/1880325539 >> 172.21.15.32:0/1880325539 conn(0x556886052460 msgr2=0x556886401a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 -- 172.21.15.32:0/1880325539 shutdown_connections 2024-08-31T07:28:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 -- 172.21.15.32:0/1880325539 wait complete. 2024-08-31T07:28:38.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 -- start start 2024-08-31T07:28:38.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.727+0000 7fc878c15740 1 Processor -- start 2024-08-31T07:28:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568860351b0 0x55688640f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x55688640fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 0x5568864155a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5568861cc700 con 0x55688605e070 2024-08-31T07:28:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5568862f0cf0 con 0x5568860351b0 2024-08-31T07:28:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5568864122e0 con 0x55688603d390 2024-08-31T07:28:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 10 client.0 _ll_register_callbacks cb 0x556886316470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x55688640fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568860351b0 0x55688640f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x55688640fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46132/0 (socket says 172.21.15.32:46132) 2024-08-31T07:28:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86aa00640 1 -- 172.21.15.32:0/537464367 learned_addr learned my addr 172.21.15.32:0/537464367 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86be00640 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 0x5568864155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86aa00640 1 -- 172.21.15.32:0/537464367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568860351b0 msgr2=0x55688640f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86aa00640 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568860351b0 0x55688640f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86aa00640 1 -- 172.21.15.32:0/537464367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 msgr2=0x5568864155a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86aa00640 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 0x5568864155a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86aa00640 1 -- 172.21.15.32:0/537464367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568863318b0 con 0x55688603d390 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86b400640 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568860351b0 0x55688640f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86be00640 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 0x5568864155a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86aa00640 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x55688640fde0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc85802a930 tx=0x7fc858004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f63ce81e586fbe6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc854c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc854c00640 1 -- 172.21.15.32:0/537464367 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858041070 con 0x55688603d390 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 1 -- 172.21.15.32:0/537464367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc864009da0 con 0x55688603d390 2024-08-31T07:28:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc854c00640 1 -- 172.21.15.32:0/537464367 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fc858030450 con 0x55688603d390 2024-08-31T07:28:38.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc854c00640 1 -- 172.21.15.32:0/537464367 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fc85804d050 con 0x55688603d390 2024-08-31T07:28:38.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 10 client.28657 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:38.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 1 -- 172.21.15.32:0/537464367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55688641e450 con 0x55688603d390 2024-08-31T07:28:38.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc854c00640 1 -- 172.21.15.32:0/537464367 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc864009da0 con 0x55688603d390 2024-08-31T07:28:38.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc854c00640 1 -- 172.21.15.32:0/537464367 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fc858032440 con 0x55688603d390 2024-08-31T07:28:38.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 10 client.28657 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:38.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 10 client.28657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:28:38.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 20 client.28657 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55688601c9f0 0x5568864218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 4 client.28657 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc878c15740 1 -- 172.21.15.32:0/537464367 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55688637c700 con 0x55688601c9f0 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86b400640 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55688601c9f0 0x5568864218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc86b400640 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55688601c9f0 0x5568864218e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29a2350e0a0a2e0d server_cookie=7c9a798590d87acb in_seq=0 out_seq=0 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc854c00640 10 client.28657 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc854c00640 1 -- 172.21.15.32:0/537464367 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55688637c700 con 0x55688601c9f0 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.733+0000 7fc854c00640 10 client.28657 handle_command_reply: tid=0 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 client.28657 shutdown 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 -- 172.21.15.32:0/537464367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 msgr2=0x55688640fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x55688640fde0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc85802a930 tx=0x7fc858004270 comp rx=0 tx=0).stop 2024-08-31T07:28:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 -- 172.21.15.32:0/537464367 shutdown_connections 2024-08-31T07:28:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 0x5568864155a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x55688640fde0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55688601c9f0 0x5568864218e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 --2- 172.21.15.32:0/537464367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568860351b0 0x55688640f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc854c00640 0 client.28657 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:28:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 -- 172.21.15.32:0/537464367 >> 172.21.15.32:0/537464367 conn(0x556886052460 msgr2=0x556886406d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 -- 172.21.15.32:0/537464367 shutdown_connections 2024-08-31T07:28:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 1 -- 172.21.15.32:0/537464367 wait complete. 2024-08-31T07:28:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.738+0000 7fc878c15740 20 client.28657 trim_cache size 0 max 16384 2024-08-31T07:28:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:38 smithi084 ceph-mon[22387]: pgmap v864: 97 pgs: 97 active+clean; 863 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 519 KiB/s wr, 58 op/s 2024-08-31T07:28:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T07:28:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:28:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:28:38.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:38 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:28:38.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc878c15740 1 Processor -- start 2024-08-31T07:28:38.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc878c15740 1 -- start start 2024-08-31T07:28:38.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc878c15740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568860351b0 0x556886409380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc878c15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688603d390 0x556886415510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc878c15740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 0x556886415a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc878c15740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1be050 con 0x55688605e070 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc878c15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1bfcb0 con 0x55688603d390 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc878c15740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1be720 con 0x5568860351b0 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc86b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568860351b0 0x556886409380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc86b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568860351b0 0x556886409380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46138/0 (socket says 172.21.15.32:46138) 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc86b400640 1 -- 172.21.15.32:0/1324804308 learned_addr learned my addr 172.21.15.32:0/1324804308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc86aa00640 1 --2- 172.21.15.32:0/1324804308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688603d390 0x556886415510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc86be00640 1 --2- 172.21.15.32:0/1324804308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 0x556886415a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc86b400640 1 -- 172.21.15.32:0/1324804308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688603d390 msgr2=0x556886415510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc86b400640 1 --2- 172.21.15.32:0/1324804308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688603d390 0x556886415510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc86b400640 1 -- 172.21.15.32:0/1324804308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 msgr2=0x556886415a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:38.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc86b400640 1 --2- 172.21.15.32:0/1324804308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 0x556886415a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.757+0000 7fc86b400640 1 -- 172.21.15.32:0/1324804308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568864108e0 con 0x5568860351b0 2024-08-31T07:28:38.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.762+0000 7fc86b400640 1 --2- 172.21.15.32:0/1324804308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568860351b0 0x556886409380 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009610 tx=0x7fc85c0a74c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535ff014b1973f8b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:38.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.762+0000 7fc857400640 1 -- 172.21.15.32:0/1324804308 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c020040 con 0x5568860351b0 2024-08-31T07:28:38.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.762+0000 7fc857400640 1 -- 172.21.15.32:0/1324804308 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc85c0ab5c0 con 0x5568860351b0 2024-08-31T07:28:38.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.762+0000 7fc857400640 1 -- 172.21.15.32:0/1324804308 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c05b360 con 0x5568860351b0 2024-08-31T07:28:38.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- 172.21.15.32:0/1324804308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568860351b0 msgr2=0x556886409380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:38.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 --2- 172.21.15.32:0/1324804308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568860351b0 0x556886409380 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009610 tx=0x7fc85c0a74c0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- 172.21.15.32:0/1324804308 shutdown_connections 2024-08-31T07:28:38.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 --2- 172.21.15.32:0/1324804308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55688605e070 0x556886415a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 --2- 172.21.15.32:0/1324804308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688603d390 0x556886415510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 --2- 172.21.15.32:0/1324804308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568860351b0 0x556886409380 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- 172.21.15.32:0/1324804308 >> 172.21.15.32:0/1324804308 conn(0x556886052460 msgr2=0x556886408c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:38.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- 172.21.15.32:0/1324804308 shutdown_connections 2024-08-31T07:28:38.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- 172.21.15.32:0/1324804308 wait complete. 2024-08-31T07:28:38.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- start start 2024-08-31T07:28:38.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 Processor -- start 2024-08-31T07:28:38.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 0x55688641bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x556886418500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688605e070 0x556886418a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1bf520 con 0x5568860351b0 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1bee20 con 0x55688605e070 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1bc050 con 0x55688603d390 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688605e070 0x556886418a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688605e070 0x556886418a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59446/0 (socket says 172.21.15.32:59446) 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86be00640 1 -- 172.21.15.32:0/887412242 learned_addr learned my addr 172.21.15.32:0/887412242 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86aa00640 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x556886418500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 10 client.0 _ll_register_callbacks cb 0x7fc858009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:38.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86be00640 1 -- 172.21.15.32:0/887412242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 msgr2=0x556886418500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86be00640 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x556886418500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86b400640 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 0x55688641bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86be00640 1 -- 172.21.15.32:0/887412242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 msgr2=0x55688641bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86be00640 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 0x55688641bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86be00640 1 -- 172.21.15.32:0/887412242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556886419030 con 0x55688605e070 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86aa00640 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x556886418500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86b400640 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 0x55688641bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc86be00640 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688605e070 0x556886418a40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc864009990 tx=0x7fc864009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c969b695523c55 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc854c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc854c00640 1 -- 172.21.15.32:0/887412242 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86401b040 con 0x55688605e070 2024-08-31T07:28:38.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc854c00640 1 -- 172.21.15.32:0/887412242 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fc8640049c0 con 0x55688605e070 2024-08-31T07:28:38.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- 172.21.15.32:0/887412242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc85c0ab5c0 con 0x55688605e070 2024-08-31T07:28:38.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc854c00640 1 -- 172.21.15.32:0/887412242 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fc86403d050 con 0x55688605e070 2024-08-31T07:28:38.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 10 client.28658 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:38.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- 172.21.15.32:0/887412242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55688648d0a0 con 0x55688605e070 2024-08-31T07:28:38.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc854c00640 1 -- 172.21.15.32:0/887412242 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc85c0ab5c0 con 0x55688605e070 2024-08-31T07:28:38.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc854c00640 1 -- 172.21.15.32:0/887412242 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fc8640056f0 con 0x55688605e070 2024-08-31T07:28:38.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 10 client.28658 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:38.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 10 client.28658 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 20 client.28658 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x556885ff1da0 0x55688648f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 4 client.28658 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.763+0000 7fc878c15740 1 -- 172.21.15.32:0/887412242 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568863781a0 con 0x556885ff1da0 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc86b400640 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x556885ff1da0 0x55688648f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc86b400640 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x556885ff1da0 0x55688648f570 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0b6b8de156dcca0 server_cookie=3ad5412621fc4847 in_seq=0 out_seq=0 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc854c00640 10 client.28658 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc854c00640 1 -- 172.21.15.32:0/887412242 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568863781a0 con 0x556885ff1da0 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc854c00640 10 client.28658 handle_command_reply: tid=0 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 client.28658 shutdown 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 -- 172.21.15.32:0/887412242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688605e070 msgr2=0x556886418a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:38.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688605e070 0x556886418a40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc864009990 tx=0x7fc864009570 comp rx=0 tx=0).stop 2024-08-31T07:28:38.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 -- 172.21.15.32:0/887412242 shutdown_connections 2024-08-31T07:28:38.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x556885ff1da0 0x55688648f570 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55688605e070 0x556886418a40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc854c00640 0 client.28658 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:28:38.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55688603d390 0x556886418500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 --2- 172.21.15.32:0/887412242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5568860351b0 0x55688641bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 -- 172.21.15.32:0/887412242 >> 172.21.15.32:0/887412242 conn(0x556886052460 msgr2=0x556886410ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:38.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 -- 172.21.15.32:0/887412242 shutdown_connections 2024-08-31T07:28:38.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 1 -- 172.21.15.32:0/887412242 wait complete. 2024-08-31T07:28:38.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc878c15740 20 client.28658 trim_cache size 0 max 16384 2024-08-31T07:28:38.775 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:28:38.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc870a00640 1 -- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8500b4d20 msgr2=0x7fc8500b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:38.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc870a00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8500b4d20 0x7fc8500b7230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc864010c40 tx=0x7fc86401d000 comp rx=0 tx=0).stop 2024-08-31T07:28:38.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc870a00640 1 -- 172.21.15.32:0/2100368437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c075b70 msgr2=0x7fc86c074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:38.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc870a00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c075b70 0x7fc86c074ca0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00d770 tx=0x7fc85c00dc30 comp rx=0 tx=0).stop 2024-08-31T07:28:38.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc870a00640 1 -- 172.21.15.32:0/2100368437 shutdown_connections 2024-08-31T07:28:38.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc870a00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8500b4d20 0x7fc8500b7230 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc870a00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c114020 0x7fc86c06e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc870a00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c077120 0x7fc86c06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc870a00640 1 --2- 172.21.15.32:0/2100368437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c075b70 0x7fc86c074ca0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:38.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.769+0000 7fc870a00640 1 -- 172.21.15.32:0/2100368437 >> 172.21.15.32:0/2100368437 conn(0x7fc86c0ff1e0 msgr2=0x7fc86c10ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:38.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.774+0000 7fc870a00640 1 -- 172.21.15.32:0/2100368437 shutdown_connections 2024-08-31T07:28:38.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:38.774+0000 7fc870a00640 1 -- 172.21.15.32:0/2100368437 wait complete. 2024-08-31T07:28:39.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50301 0.81 MB/sec execute 19 sec latency 36014.770 ms 2024-08-31T07:28:39.383 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:39 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi032 to 177.5M 2024-08-31T07:28:39.383 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:39 smithi032 ceph-mon[17252]: Unable to set osd_memory_target on smithi032 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T07:28:39.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:39 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi032 to 177.5M 2024-08-31T07:28:39.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:39 smithi073 ceph-mon[21995]: Unable to set osd_memory_target on smithi032 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T07:28:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:39 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi032 to 177.5M 2024-08-31T07:28:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:39 smithi084 ceph-mon[22387]: Unable to set osd_memory_target on smithi032 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T07:28:40.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:28:40.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.813+0000 7fb7c6600640 1 -- 172.21.15.32:0/973327014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0104560 msgr2=0x7fb7c0104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:40.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.813+0000 7fb7c6600640 1 --2- 172.21.15.32:0/973327014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0104560 0x7fb7c0104970 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009900 tx=0x7fb7b402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.813+0000 7fb7c6600640 1 -- 172.21.15.32:0/973327014 shutdown_connections 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.813+0000 7fb7c6600640 1 --2- 172.21.15.32:0/973327014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7c010d9c0 0x7fb7c010feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.813+0000 7fb7c6600640 1 --2- 172.21.15.32:0/973327014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c0104eb0 0x7fb7c010d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.813+0000 7fb7c6600640 1 --2- 172.21.15.32:0/973327014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0104560 0x7fb7c0104970 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.813+0000 7fb7c6600640 1 -- 172.21.15.32:0/973327014 >> 172.21.15.32:0/973327014 conn(0x7fb7c00fe060 msgr2=0x7fb7c01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.813+0000 7fb7c6600640 1 -- 172.21.15.32:0/973327014 shutdown_connections 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.813+0000 7fb7c6600640 1 -- 172.21.15.32:0/973327014 wait complete. 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 Processor -- start 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 -- start start 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0104560 0x7fb7c019c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7c0104eb0 0x7fb7c019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c010d9c0 0x7fb7c01a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c01136a0 con 0x7fb7c0104560 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0113520 con 0x7fb7c010d9c0 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0113820 con 0x7fb7c0104eb0 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0104560 0x7fb7c019c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0104560 0x7fb7c019c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33488/0 (socket says 172.21.15.32:33488) 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c5200640 1 -- 172.21.15.32:0/1467975944 learned_addr learned my addr 172.21.15.32:0/1467975944 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7bfe00640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7c0104eb0 0x7fb7c019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c5c00640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c010d9c0 0x7fb7c01a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7bfe00640 1 -- 172.21.15.32:0/1467975944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c010d9c0 msgr2=0x7fb7c01a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7bfe00640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c010d9c0 0x7fb7c01a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7bfe00640 1 -- 172.21.15.32:0/1467975944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0104560 msgr2=0x7fb7c019c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7bfe00640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0104560 0x7fb7c019c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7bfe00640 1 -- 172.21.15.32:0/1467975944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c01a2260 con 0x7fb7c0104eb0 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c5200640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0104560 0x7fb7c019c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7bfe00640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7c0104eb0 0x7fb7c019c890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb7c006a830 tx=0x7fb7ac00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7bd600640 1 -- 172.21.15.32:0/1467975944 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00cc40 con 0x7fb7c0104eb0 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 -- 172.21.15.32:0/1467975944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c01a2550 con 0x7fb7c0104eb0 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7bd600640 1 -- 172.21.15.32:0/1467975944 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ac00cde0 con 0x7fb7c0104eb0 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7bd600640 1 -- 172.21.15.32:0/1467975944 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac018760 con 0x7fb7c0104eb0 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 -- 172.21.15.32:0/1467975944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c01a8cb0 con 0x7fb7c0104eb0 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.814+0000 7fb7c6600640 1 -- 172.21.15.32:0/1467975944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c0104970 con 0x7fb7c0104eb0 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.820+0000 7fb7bd600640 1 -- 172.21.15.32:0/1467975944 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb7ac010a90 con 0x7fb7c0104eb0 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.820+0000 7fb7bd600640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7a80b4ca0 0x7fb7a80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.820+0000 7fb7bd600640 1 -- 172.21.15.32:0/1467975944 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fb7ac0e5c80 con 0x7fb7c0104eb0 2024-08-31T07:28:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.820+0000 7fb7c5200640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7a80b4ca0 0x7fb7a80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:40.214 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:28:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.820+0000 7fb7bd600640 1 -- 172.21.15.32:0/1467975944 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb7ac0ac410 con 0x7fb7c0104eb0 2024-08-31T07:28:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:39.820+0000 7fb7c5200640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7a80b4ca0 0x7fb7a80b71b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40096c0 tx=0x7fb7b4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.066+0000 7fb7c6600640 1 -- 172.21.15.32:0/1467975944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7c01a8ee0 con 0x7fb7c0104eb0 2024-08-31T07:28:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.071+0000 7fb7bd600640 1 -- 172.21.15.32:0/1467975944 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11970 (secure 0 0 0) 0x7fb7ac0b0bd0 con 0x7fb7c0104eb0 2024-08-31T07:28:40.215 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":59,"btime":"2024-08-31T07:28:30:260014+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:28:29.968519+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:28:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 59 2024-08-31T07:28:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 -- 172.21.15.32:0/1467975944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7a80b4ca0 msgr2=0x7fb7a80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7a80b4ca0 0x7fb7a80b71b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40096c0 tx=0x7fb7b4005a20 comp rx=0 tx=0).stop 2024-08-31T07:28:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 -- 172.21.15.32:0/1467975944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7c0104eb0 msgr2=0x7fb7c019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7c0104eb0 0x7fb7c019c890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb7c006a830 tx=0x7fb7ac00ee50 comp rx=0 tx=0).stop 2024-08-31T07:28:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 -- 172.21.15.32:0/1467975944 shutdown_connections 2024-08-31T07:28:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb7a80b4ca0 0x7fb7a80b71b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c010d9c0 0x7fb7c01a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7c0104eb0 0x7fb7c019c890 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:40.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 --2- 172.21.15.32:0/1467975944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c0104560 0x7fb7c019c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:40.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 -- 172.21.15.32:0/1467975944 >> 172.21.15.32:0/1467975944 conn(0x7fb7c00fe060 msgr2=0x7fb7c0102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:40.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 -- 172.21.15.32:0/1467975944 shutdown_connections 2024-08-31T07:28:40.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:40.074+0000 7fb7c6600640 1 -- 172.21.15.32:0/1467975944 wait complete. 2024-08-31T07:28:40.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50301 0.77 MB/sec execute 20 sec latency 37014.827 ms 2024-08-31T07:28:40.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:40 smithi032 ceph-mon[17252]: pgmap v865: 97 pgs: 97 active+clean; 863 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 530 KiB/s wr, 56 op/s 2024-08-31T07:28:40.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:40 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1467975944' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:28:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:40 smithi073 ceph-mon[21995]: pgmap v865: 97 pgs: 97 active+clean; 863 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 530 KiB/s wr, 56 op/s 2024-08-31T07:28:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:40 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1467975944' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:28:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:40 smithi084 ceph-mon[22387]: pgmap v865: 97 pgs: 97 active+clean; 863 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 530 KiB/s wr, 56 op/s 2024-08-31T07:28:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:40 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1467975944' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:28:41.240 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 141.25 --await-for 45.0' 2024-08-31T07:28:41.241 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 141.25 --await-for 45.0 2024-08-31T07:28:41.274 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:41.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50301 0.73 MB/sec execute 21 sec latency 38014.898 ms 2024-08-31T07:28:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.310+0000 7fb39a600640 1 -- 172.21.15.32:0/934640386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb394102540 msgr2=0x7fb394102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.310+0000 7fb39a600640 1 --2- 172.21.15.32:0/934640386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb394102540 0x7fb394102950 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb384009900 tx=0x7fb38402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.310+0000 7fb39a600640 1 -- 172.21.15.32:0/934640386 shutdown_connections 2024-08-31T07:28:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.310+0000 7fb39a600640 1 --2- 172.21.15.32:0/934640386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394107980 0x7fb394107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.310+0000 7fb39a600640 1 --2- 172.21.15.32:0/934640386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb394102e90 0x7fb3941073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.310+0000 7fb39a600640 1 --2- 172.21.15.32:0/934640386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb394102540 0x7fb394102950 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.310+0000 7fb39a600640 1 -- 172.21.15.32:0/934640386 >> 172.21.15.32:0/934640386 conn(0x7fb3940794d0 msgr2=0x7fb3940798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.311+0000 7fb39a600640 1 -- 172.21.15.32:0/934640386 shutdown_connections 2024-08-31T07:28:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.311+0000 7fb39a600640 1 -- 172.21.15.32:0/934640386 wait complete. 2024-08-31T07:28:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.313+0000 7fb39a600640 1 Processor -- start 2024-08-31T07:28:42.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.315+0000 7fb39a600640 1 -- start start 2024-08-31T07:28:42.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.315+0000 7fb39a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb394102540 0x7fb3941a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.315+0000 7fb39a600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb394102e90 0x7fb3941a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.315+0000 7fb39a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394107980 0x7fb3941a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.315+0000 7fb39a600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb394119e40 con 0x7fb394107980 2024-08-31T07:28:42.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.315+0000 7fb39a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb394119cc0 con 0x7fb394102540 2024-08-31T07:28:42.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.315+0000 7fb39a600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb394119fc0 con 0x7fb394102e90 2024-08-31T07:28:42.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.316+0000 7fb393e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394107980 0x7fb3941a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.316+0000 7fb393400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb394102540 0x7fb3941a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.317+0000 7fb393e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394107980 0x7fb3941a60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33510/0 (socket says 172.21.15.32:33510) 2024-08-31T07:28:42.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.317+0000 7fb393400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb394102540 0x7fb3941a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59464/0 (socket says 172.21.15.32:59464) 2024-08-31T07:28:42.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.317+0000 7fb393400640 1 -- 172.21.15.32:0/3577579755 learned_addr learned my addr 172.21.15.32:0/3577579755 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:42.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.317+0000 7fb392a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb394102e90 0x7fb3941a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.317+0000 7fb392a00640 1 -- 172.21.15.32:0/3577579755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb394102540 msgr2=0x7fb3941a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:42.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.317+0000 7fb392a00640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb394102540 0x7fb3941a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.317+0000 7fb392a00640 1 -- 172.21.15.32:0/3577579755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394107980 msgr2=0x7fb3941a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:42.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.318+0000 7fb392a00640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394107980 0x7fb3941a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.318+0000 7fb392a00640 1 -- 172.21.15.32:0/3577579755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3941a67b0 con 0x7fb394102e90 2024-08-31T07:28:42.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.318+0000 7fb393e00640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394107980 0x7fb3941a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:28:42.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.318+0000 7fb392a00640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb394102e90 0x7fb3941a0e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb38000e9c0 tx=0x7fb38000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:42.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.319+0000 7fb38be00640 1 -- 172.21.15.32:0/3577579755 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38000ccc0 con 0x7fb394102e90 2024-08-31T07:28:42.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.319+0000 7fb38be00640 1 -- 172.21.15.32:0/3577579755 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb38000ce60 con 0x7fb394102e90 2024-08-31T07:28:42.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.319+0000 7fb39a600640 1 -- 172.21.15.32:0/3577579755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3941a6aa0 con 0x7fb394102e90 2024-08-31T07:28:42.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.320+0000 7fb38be00640 1 -- 172.21.15.32:0/3577579755 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380004a00 con 0x7fb394102e90 2024-08-31T07:28:42.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.320+0000 7fb39a600640 1 -- 172.21.15.32:0/3577579755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39406cb80 con 0x7fb394102e90 2024-08-31T07:28:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.328+0000 7fb3a140a740 1 Processor -- start 2024-08-31T07:28:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.328+0000 7fb3a140a740 1 -- start start 2024-08-31T07:28:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.328+0000 7fb38be00640 1 -- 172.21.15.32:0/3577579755 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb380004ba0 con 0x7fb394102e90 2024-08-31T07:28:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.329+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd3111a070 0x55bd314bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.329+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310f11b0 0x55bd314bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.329+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f9390 0x55bd314c0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.329+0000 7fb3a140a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bd314367b0 con 0x55bd3111a070 2024-08-31T07:28:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.329+0000 7fb3a140a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bd31425d40 con 0x55bd310f11b0 2024-08-31T07:28:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.329+0000 7fb3a140a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bd31429cf0 con 0x55bd310f9390 2024-08-31T07:28:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.330+0000 7fb393e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd3111a070 0x55bd314bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.330+0000 7fb393400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310f11b0 0x55bd314bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.330+0000 7fb393400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310f11b0 0x55bd314bfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59466/0 (socket says 172.21.15.32:59466) 2024-08-31T07:28:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.330+0000 7fb393400640 1 -- 172.21.15.32:0/1613263267 learned_addr learned my addr 172.21.15.32:0/1613263267 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.330+0000 7fb392a00640 1 --2- 172.21.15.32:0/1613263267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f9390 0x55bd314c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.330+0000 7fb393400640 1 -- 172.21.15.32:0/1613263267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f9390 msgr2=0x55bd314c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.330+0000 7fb393400640 1 --2- 172.21.15.32:0/1613263267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f9390 0x55bd314c0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.330+0000 7fb393400640 1 -- 172.21.15.32:0/1613263267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd3111a070 msgr2=0x55bd314bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.330+0000 7fb393400640 1 --2- 172.21.15.32:0/1613263267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd3111a070 0x55bd314bf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.330+0000 7fb393400640 1 -- 172.21.15.32:0/1613263267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd31185650 con 0x55bd310f11b0 2024-08-31T07:28:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.331+0000 7fb393400640 1 --2- 172.21.15.32:0/1613263267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310f11b0 0x55bd314bfed0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb384005f70 tx=0x7fb384031b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac63638cb0ae6a21 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.332+0000 7fb388c00640 1 -- 172.21.15.32:0/1613263267 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384040070 con 0x55bd310f11b0 2024-08-31T07:28:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.332+0000 7fb388c00640 1 -- 172.21.15.32:0/1613263267 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb384004030 con 0x55bd310f11b0 2024-08-31T07:28:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.332+0000 7fb388c00640 1 -- 172.21.15.32:0/1613263267 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384035040 con 0x55bd310f11b0 2024-08-31T07:28:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.333+0000 7fb3a140a740 1 -- 172.21.15.32:0/1613263267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310f11b0 msgr2=0x55bd314bfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.333+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1613263267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310f11b0 0x55bd314bfed0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb384005f70 tx=0x7fb384031b20 comp rx=0 tx=0).stop 2024-08-31T07:28:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.333+0000 7fb3a140a740 1 -- 172.21.15.32:0/1613263267 shutdown_connections 2024-08-31T07:28:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.333+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1613263267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f9390 0x55bd314c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.334+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1613263267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310f11b0 0x55bd314bfed0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.334+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1613263267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd3111a070 0x55bd314bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.334+0000 7fb3a140a740 1 -- 172.21.15.32:0/1613263267 >> 172.21.15.32:0/1613263267 conn(0x55bd3110e460 msgr2=0x55bd314bd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.334+0000 7fb3a140a740 1 -- 172.21.15.32:0/1613263267 shutdown_connections 2024-08-31T07:28:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.335+0000 7fb3a140a740 1 -- 172.21.15.32:0/1613263267 wait complete. 2024-08-31T07:28:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.335+0000 7fb38be00640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3780b4c40 0x7fb3780b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.336+0000 7fb38be00640 1 -- 172.21.15.32:0/3577579755 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fb3800e5580 con 0x7fb394102e90 2024-08-31T07:28:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.336+0000 7fb3a140a740 1 -- start start 2024-08-31T07:28:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.336+0000 7fb3a140a740 1 Processor -- start 2024-08-31T07:28:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.338+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 0x55bd314cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.338+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 0x55bd314cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.338+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 0x55bd314e2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.339+0000 7fb3a140a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bd31288700 con 0x55bd310d89f0 2024-08-31T07:28:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.339+0000 7fb3a140a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bd313accf0 con 0x55bd3111a070 2024-08-31T07:28:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.339+0000 7fb3a140a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bd314df090 con 0x55bd310f11b0 2024-08-31T07:28:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.339+0000 7fb392a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 0x55bd314cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.339+0000 7fb392a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 0x55bd314cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59480/0 (socket says 172.21.15.32:59480) 2024-08-31T07:28:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.339+0000 7fb392a00640 1 -- 172.21.15.32:0/2438975080 learned_addr learned my addr 172.21.15.32:0/2438975080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.339+0000 7fb3a140a740 10 client.0 _ll_register_callbacks cb 0x55bd313d2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.339+0000 7fb393400640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3780b4c40 0x7fb3780b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.339+0000 7fb393400640 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 0x55bd314cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.340+0000 7fb392a00640 1 -- 172.21.15.32:0/2438975080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 msgr2=0x55bd314cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.340+0000 7fb392a00640 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 0x55bd314cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.340+0000 7fb392a00640 1 -- 172.21.15.32:0/2438975080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 msgr2=0x55bd314e2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.340+0000 7fb392a00640 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 0x55bd314e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.340+0000 7fb392a00640 1 -- 172.21.15.32:0/2438975080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd313ed8b0 con 0x55bd3111a070 2024-08-31T07:28:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.340+0000 7fb392a00640 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 0x55bd314cbc70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb3800053e0 tx=0x7fb3800940e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1e6a51bd32b7fb5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.340+0000 7fb37d600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:28:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.340+0000 7fb393400640 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 0x55bd314cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:28:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.340+0000 7fb37d600640 1 -- 172.21.15.32:0/2438975080 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38001c040 con 0x55bd3111a070 2024-08-31T07:28:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.341+0000 7fb3a140a740 1 -- 172.21.15.32:0/2438975080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb384004030 con 0x55bd3111a070 2024-08-31T07:28:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.341+0000 7fb37d600640 1 -- 172.21.15.32:0/2438975080 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fb3800b4d00 con 0x55bd3111a070 2024-08-31T07:28:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.341+0000 7fb37d600640 1 -- 172.21.15.32:0/2438975080 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb3800abc80 con 0x55bd3111a070 2024-08-31T07:28:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.341+0000 7fb393400640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3780b4c40 0x7fb3780b7150 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb3840096c0 tx=0x7fb384005160 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.342+0000 7fb3a140a740 10 client.28676 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.342+0000 7fb3a140a740 1 -- 172.21.15.32:0/2438975080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55bd314eb290 con 0x55bd3111a070 2024-08-31T07:28:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.342+0000 7fb37d600640 1 -- 172.21.15.32:0/2438975080 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb384004030 con 0x55bd3111a070 2024-08-31T07:28:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.342+0000 7fb37d600640 1 -- 172.21.15.32:0/2438975080 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fb3800b0080 con 0x55bd3111a070 2024-08-31T07:28:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.342+0000 7fb3a140a740 10 client.28676 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:42.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50596 1.12 MB/sec execute 22 sec latency 38290.490 ms 2024-08-31T07:28:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.342+0000 7fb3a140a740 10 client.28676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:28:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.342+0000 7fb3a140a740 20 client.28676 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.342+0000 7fb3a140a740 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55bd310adda0 0x55bd314ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.343+0000 7fb3a140a740 4 client.28676 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:28:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.343+0000 7fb3a140a740 1 -- 172.21.15.32:0/2438975080 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd31438700 con 0x55bd310adda0 2024-08-31T07:28:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.344+0000 7fb393400640 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55bd310adda0 0x55bd314ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.345+0000 7fb393400640 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55bd310adda0 0x55bd314ee800 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b1e505ada15f5d server_cookie=1cc7f2e4a317c45f in_seq=0 out_seq=0 2024-08-31T07:28:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.345+0000 7fb37d600640 10 client.28676 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.347+0000 7fb37d600640 1 -- 172.21.15.32:0/2438975080 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd31438700 con 0x55bd310adda0 2024-08-31T07:28:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.347+0000 7fb37d600640 10 client.28676 handle_command_reply: tid=0 2024-08-31T07:28:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.348+0000 7fb3a140a740 1 client.28676 shutdown 2024-08-31T07:28:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.349+0000 7fb3a140a740 1 -- 172.21.15.32:0/2438975080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 msgr2=0x55bd314cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.349+0000 7fb3a140a740 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 0x55bd314cbc70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb3800053e0 tx=0x7fb3800940e0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.349+0000 7fb3a140a740 1 -- 172.21.15.32:0/2438975080 shutdown_connections 2024-08-31T07:28:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.349+0000 7fb3a140a740 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55bd310adda0 0x55bd314ee800 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.349+0000 7fb3a140a740 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 0x55bd314e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.349+0000 7fb3a140a740 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 0x55bd314cbc70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.349+0000 7fb3a140a740 1 --2- 172.21.15.32:0/2438975080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 0x55bd314cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.349+0000 7fb3a140a740 1 -- 172.21.15.32:0/2438975080 >> 172.21.15.32:0/2438975080 conn(0x55bd3110e460 msgr2=0x55bd314c2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.349+0000 7fb37d600640 0 client.28676 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.350+0000 7fb3a140a740 1 -- 172.21.15.32:0/2438975080 shutdown_connections 2024-08-31T07:28:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.350+0000 7fb3a140a740 1 -- 172.21.15.32:0/2438975080 wait complete. 2024-08-31T07:28:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.351+0000 7fb3a140a740 20 client.28676 trim_cache size 0 max 16384 2024-08-31T07:28:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.378+0000 7fb3a140a740 1 Processor -- start 2024-08-31T07:28:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.378+0000 7fb3a140a740 1 -- start start 2024-08-31T07:28:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.379+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310d89f0 0x55bd314e23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.379+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310f11b0 0x55bd314e9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.379+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd3111a070 0x55bd314bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.379+0000 7fb3a140a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb394109bf0 con 0x55bd310f11b0 2024-08-31T07:28:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.379+0000 7fb3a140a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410b850 con 0x55bd310d89f0 2024-08-31T07:28:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.379+0000 7fb3a140a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410a2c0 con 0x55bd3111a070 2024-08-31T07:28:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.379+0000 7fb393e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd3111a070 0x55bd314bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.379+0000 7fb393e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd3111a070 0x55bd314bd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46204/0 (socket says 172.21.15.32:46204) 2024-08-31T07:28:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.379+0000 7fb393e00640 1 -- 172.21.15.32:0/1798299374 learned_addr learned my addr 172.21.15.32:0/1798299374 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.379+0000 7fb393400640 1 --2- 172.21.15.32:0/1798299374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310d89f0 0x55bd314e23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.380+0000 7fb392a00640 1 --2- 172.21.15.32:0/1798299374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310f11b0 0x55bd314e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.380+0000 7fb393e00640 1 -- 172.21.15.32:0/1798299374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310d89f0 msgr2=0x55bd314e23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.380+0000 7fb393e00640 1 --2- 172.21.15.32:0/1798299374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310d89f0 0x55bd314e23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.380+0000 7fb393e00640 1 -- 172.21.15.32:0/1798299374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310f11b0 msgr2=0x55bd314e9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.380+0000 7fb393e00640 1 --2- 172.21.15.32:0/1798299374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310f11b0 0x55bd314e9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.380+0000 7fb393e00640 1 -- 172.21.15.32:0/1798299374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd314c9990 con 0x55bd3111a070 2024-08-31T07:28:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.380+0000 7fb393400640 1 --2- 172.21.15.32:0/1798299374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310d89f0 0x55bd314e23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.381+0000 7fb393e00640 1 --2- 172.21.15.32:0/1798299374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd3111a070 0x55bd314bd780 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0124d0 tx=0x7fb38c012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a75b1bc5ffb3ff8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.381+0000 7fb37fe00640 1 -- 172.21.15.32:0/1798299374 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c01f070 con 0x55bd3111a070 2024-08-31T07:28:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.381+0000 7fb37fe00640 1 -- 172.21.15.32:0/1798299374 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb38c0109a0 con 0x55bd3111a070 2024-08-31T07:28:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.381+0000 7fb37fe00640 1 -- 172.21.15.32:0/1798299374 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c01aa70 con 0x55bd3111a070 2024-08-31T07:28:42.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.383+0000 7fb3a140a740 1 -- 172.21.15.32:0/1798299374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd3111a070 msgr2=0x55bd314bd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:42.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.383+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1798299374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd3111a070 0x55bd314bd780 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0124d0 tx=0x7fb38c012990 comp rx=0 tx=0).stop 2024-08-31T07:28:42.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.383+0000 7fb3a140a740 1 -- 172.21.15.32:0/1798299374 shutdown_connections 2024-08-31T07:28:42.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.383+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1798299374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd3111a070 0x55bd314bd780 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.383+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1798299374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310f11b0 0x55bd314e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.383+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1798299374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd310d89f0 0x55bd314e23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.383+0000 7fb3a140a740 1 -- 172.21.15.32:0/1798299374 >> 172.21.15.32:0/1798299374 conn(0x55bd3110e460 msgr2=0x55bd314bd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:42.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.383+0000 7fb3a140a740 1 -- 172.21.15.32:0/1798299374 shutdown_connections 2024-08-31T07:28:42.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.383+0000 7fb3a140a740 1 -- 172.21.15.32:0/1798299374 wait complete. 2024-08-31T07:28:42.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.384+0000 7fb3a140a740 1 -- start start 2024-08-31T07:28:42.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.384+0000 7fb3a140a740 1 Processor -- start 2024-08-31T07:28:42.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.386+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 0x55bd314e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.386+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 0x55bd314e5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.386+0000 7fb3a140a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 0x55bd315581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.386+0000 7fb3a140a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410b0c0 con 0x55bd310d89f0 2024-08-31T07:28:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.386+0000 7fb3a140a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410a9c0 con 0x55bd3111a070 2024-08-31T07:28:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.386+0000 7fb3a140a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3941073c0 con 0x55bd310f11b0 2024-08-31T07:28:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.386+0000 7fb3a140a740 10 client.0 _ll_register_callbacks cb 0x7fb380005710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.386+0000 7fb393400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 0x55bd314e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.386+0000 7fb393400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 0x55bd314e4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33550/0 (socket says 172.21.15.32:33550) 2024-08-31T07:28:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.386+0000 7fb393400640 1 -- 172.21.15.32:0/1251420369 learned_addr learned my addr 172.21.15.32:0/1251420369 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.387+0000 7fb392a00640 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 0x55bd314e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.387+0000 7fb393e00640 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 0x55bd315581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.387+0000 7fb392a00640 1 -- 172.21.15.32:0/1251420369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 msgr2=0x55bd315581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.387+0000 7fb392a00640 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 0x55bd315581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.387+0000 7fb392a00640 1 -- 172.21.15.32:0/1251420369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 msgr2=0x55bd314e4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.387+0000 7fb392a00640 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 0x55bd314e4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.387+0000 7fb392a00640 1 -- 172.21.15.32:0/1251420369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd31558700 con 0x55bd310f11b0 2024-08-31T07:28:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.387+0000 7fb393400640 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 0x55bd314e4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.388+0000 7fb392a00640 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 0x55bd314e5400 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb3800b0440 tx=0x7fb380005120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8fdf33dace478fe server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.388+0000 7fb37d600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.388+0000 7fb37d600640 1 -- 172.21.15.32:0/1251420369 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38001c070 con 0x55bd310f11b0 2024-08-31T07:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.388+0000 7fb37d600640 1 -- 172.21.15.32:0/1251420369 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fb3800a82e0 con 0x55bd310f11b0 2024-08-31T07:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.388+0000 7fb3a140a740 1 -- 172.21.15.32:0/1251420369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb38c0109a0 con 0x55bd310f11b0 2024-08-31T07:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.388+0000 7fb37d600640 1 -- 172.21.15.32:0/1251420369 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb3800b1a20 con 0x55bd310f11b0 2024-08-31T07:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.389+0000 7fb3a140a740 10 client.28681 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.389+0000 7fb3a140a740 1 -- 172.21.15.32:0/1251420369 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55bd315598f0 con 0x55bd310f11b0 2024-08-31T07:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.389+0000 7fb37d600640 1 -- 172.21.15.32:0/1251420369 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb38c0109a0 con 0x55bd310f11b0 2024-08-31T07:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.389+0000 7fb37d600640 1 -- 172.21.15.32:0/1251420369 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fb380096060 con 0x55bd310f11b0 2024-08-31T07:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.389+0000 7fb3a140a740 10 client.28681 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.389+0000 7fb3a140a740 10 client.28681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:28:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.390+0000 7fb3a140a740 20 client.28681 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.390+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55bd3155a150 0x55bd3155c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.390+0000 7fb3a140a740 4 client.28681 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:28:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.390+0000 7fb3a140a740 1 -- 172.21.15.32:0/1251420369 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd314341a0 con 0x55bd3155a150 2024-08-31T07:28:42.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.391+0000 7fb393400640 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55bd3155a150 0x55bd3155c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:42.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.391+0000 7fb393400640 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55bd3155a150 0x55bd3155c570 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cde7dd832b78ff1 server_cookie=447cd96e2f5b70a5 in_seq=0 out_seq=0 2024-08-31T07:28:42.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.391+0000 7fb37d600640 10 client.28681 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.392+0000 7fb37d600640 1 -- 172.21.15.32:0/1251420369 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd314341a0 con 0x55bd3155a150 2024-08-31T07:28:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.392+0000 7fb37d600640 10 client.28681 handle_command_reply: tid=0 2024-08-31T07:28:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.392+0000 7fb3a140a740 1 client.28681 shutdown 2024-08-31T07:28:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 1 -- 172.21.15.32:0/1251420369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 msgr2=0x55bd314e5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 0x55bd314e5400 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb3800b0440 tx=0x7fb380005120 comp rx=0 tx=0).stop 2024-08-31T07:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 1 -- 172.21.15.32:0/1251420369 shutdown_connections 2024-08-31T07:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55bd3155a150 0x55bd3155c570 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bd3111a070 0x55bd315581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bd310f11b0 0x55bd314e5400 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 1 --2- 172.21.15.32:0/1251420369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd310d89f0 0x55bd314e4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 1 -- 172.21.15.32:0/1251420369 >> 172.21.15.32:0/1251420369 conn(0x55bd3110e460 msgr2=0x55bd314bbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb37d600640 0 client.28681 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:42.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 1 -- 172.21.15.32:0/1251420369 shutdown_connections 2024-08-31T07:28:42.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 1 -- 172.21.15.32:0/1251420369 wait complete. 2024-08-31T07:28:42.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.393+0000 7fb3a140a740 20 client.28681 trim_cache size 0 max 16384 2024-08-31T07:28:42.395 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:28:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.395+0000 7fb39a600640 1 -- 172.21.15.32:0/3577579755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3780b4c40 msgr2=0x7fb3780b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.395+0000 7fb39a600640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3780b4c40 0x7fb3780b7150 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb3840096c0 tx=0x7fb384005160 comp rx=0 tx=0).stop 2024-08-31T07:28:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.395+0000 7fb39a600640 1 -- 172.21.15.32:0/3577579755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb394102e90 msgr2=0x7fb3941a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.395+0000 7fb39a600640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb394102e90 0x7fb3941a0e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb38000e9c0 tx=0x7fb38000ee80 comp rx=0 tx=0).stop 2024-08-31T07:28:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.396+0000 7fb39a600640 1 -- 172.21.15.32:0/3577579755 shutdown_connections 2024-08-31T07:28:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.396+0000 7fb39a600640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb3780b4c40 0x7fb3780b7150 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.396+0000 7fb39a600640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394107980 0x7fb3941a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.396+0000 7fb39a600640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb394102e90 0x7fb3941a0e10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.396+0000 7fb39a600640 1 --2- 172.21.15.32:0/3577579755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb394102540 0x7fb3941a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.396+0000 7fb39a600640 1 -- 172.21.15.32:0/3577579755 >> 172.21.15.32:0/3577579755 conn(0x7fb3940794d0 msgr2=0x7fb394105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:42.398 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:42 smithi032 ceph-mon[17252]: pgmap v866: 97 pgs: 97 active+clean; 864 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 616 KiB/s wr, 66 op/s 2024-08-31T07:28:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.396+0000 7fb39a600640 1 -- 172.21.15.32:0/3577579755 shutdown_connections 2024-08-31T07:28:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:42.396+0000 7fb39a600640 1 -- 172.21.15.32:0/3577579755 wait complete. 2024-08-31T07:28:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:42 smithi073 ceph-mon[21995]: pgmap v866: 97 pgs: 97 active+clean; 864 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 616 KiB/s wr, 66 op/s 2024-08-31T07:28:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:42 smithi084 ceph-mon[22387]: pgmap v866: 97 pgs: 97 active+clean; 864 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 616 KiB/s wr, 66 op/s 2024-08-31T07:28:43.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 51900 4.27 MB/sec execute 23 sec latency 42.312 ms 2024-08-31T07:28:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.190+0000 7fa8fa400640 1 -- 172.21.15.32:0/2422758728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f4104770 msgr2=0x7fa8f4104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.190+0000 7fa8fa400640 1 --2- 172.21.15.32:0/2422758728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f4104770 0x7fa8f4104b80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009900 tx=0x7fa8e402a400 comp rx=0 tx=0).stop 2024-08-31T07:28:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.191+0000 7fa8fa400640 1 -- 172.21.15.32:0/2422758728 shutdown_connections 2024-08-31T07:28:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.191+0000 7fa8fa400640 1 --2- 172.21.15.32:0/2422758728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f410dbd0 0x7fa8f41100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.191+0000 7fa8fa400640 1 --2- 172.21.15.32:0/2422758728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8f41050c0 0x7fa8f410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.191+0000 7fa8fa400640 1 --2- 172.21.15.32:0/2422758728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f4104770 0x7fa8f4104b80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.191+0000 7fa8fa400640 1 -- 172.21.15.32:0/2422758728 >> 172.21.15.32:0/2422758728 conn(0x7fa8f4079500 msgr2=0x7fa8f4079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.191+0000 7fa8fa400640 1 -- 172.21.15.32:0/2422758728 shutdown_connections 2024-08-31T07:28:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.191+0000 7fa8fa400640 1 -- 172.21.15.32:0/2422758728 wait complete. 2024-08-31T07:28:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.193+0000 7fa8fa400640 1 Processor -- start 2024-08-31T07:28:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.193+0000 7fa8fa400640 1 -- start start 2024-08-31T07:28:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8fa400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f4104770 0x7fa8f4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8fa400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8f41050c0 0x7fa8f406df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8fa400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f410dbd0 0x7fa8f406e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8fa400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4113830 con 0x7fa8f4104770 2024-08-31T07:28:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8fa400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41136b0 con 0x7fa8f410dbd0 2024-08-31T07:28:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8fa400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41139b0 con 0x7fa8f41050c0 2024-08-31T07:28:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f410dbd0 0x7fa8f406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f410dbd0 0x7fa8f406e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59510/0 (socket says 172.21.15.32:59510) 2024-08-31T07:28:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8f2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8f41050c0 0x7fa8f406df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8f2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8f41050c0 0x7fa8f406df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46222/0 (socket says 172.21.15.32:46222) 2024-08-31T07:28:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.194+0000 7fa8f2a00640 1 -- 172.21.15.32:0/1487764558 learned_addr learned my addr 172.21.15.32:0/1487764558 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.195+0000 7fa8f3400640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f4104770 0x7fa8f4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.196+0000 7fa8f3400640 1 -- 172.21.15.32:0/1487764558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8f41050c0 msgr2=0x7fa8f406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.196+0000 7fa8f3400640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8f41050c0 0x7fa8f406df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.196+0000 7fa8f3400640 1 -- 172.21.15.32:0/1487764558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f410dbd0 msgr2=0x7fa8f406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.196+0000 7fa8f3400640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f410dbd0 0x7fa8f406e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.196+0000 7fa8f3400640 1 -- 172.21.15.32:0/1487764558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f406eb60 con 0x7fa8f4104770 2024-08-31T07:28:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.196+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8f41050c0 0x7fa8f406df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:28:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.196+0000 7fa8f3e00640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f410dbd0 0x7fa8f406e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:28:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.196+0000 7fa8f3400640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f4104770 0x7fa8f4074c60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4006010 tx=0x7fa8e4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.197+0000 7fa8ebe00640 1 -- 172.21.15.32:0/1487764558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e403f070 con 0x7fa8f4104770 2024-08-31T07:28:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.197+0000 7fa8fa400640 1 -- 172.21.15.32:0/1487764558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f40738d0 con 0x7fa8f4104770 2024-08-31T07:28:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.197+0000 7fa8ebe00640 1 -- 172.21.15.32:0/1487764558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e4003ca0 con 0x7fa8f4104770 2024-08-31T07:28:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.197+0000 7fa8fa400640 1 -- 172.21.15.32:0/1487764558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f4073e00 con 0x7fa8f4104770 2024-08-31T07:28:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.198+0000 7fa8ebe00640 1 -- 172.21.15.32:0/1487764558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e403f070 con 0x7fa8f4104770 2024-08-31T07:28:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.203+0000 7fa901208740 1 Processor -- start 2024-08-31T07:28:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.204+0000 7fa901208740 1 -- start start 2024-08-31T07:28:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.205+0000 7fa901208740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e34850 0x563c391dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.205+0000 7fa901208740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391db150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.205+0000 7fa8ebe00640 1 -- 172.21.15.32:0/1487764558 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa8e403b460 con 0x7fa8f4104770 2024-08-31T07:28:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.205+0000 7fa901208740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 0x563c391db690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.205+0000 7fa901208740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563c39150450 con 0x563c38e13b70 2024-08-31T07:28:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.205+0000 7fa901208740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563c3913f9e0 con 0x563c38e0b990 2024-08-31T07:28:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.205+0000 7fa901208740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563c39143990 con 0x563c38e34850 2024-08-31T07:28:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.206+0000 7fa8f3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391db150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.206+0000 7fa8f3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391db150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59524/0 (socket says 172.21.15.32:59524) 2024-08-31T07:28:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.206+0000 7fa8f3400640 1 -- 172.21.15.32:0/2023104670 learned_addr learned my addr 172.21.15.32:0/2023104670 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.206+0000 7fa8f3e00640 1 --2- 172.21.15.32:0/2023104670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e34850 0x563c391dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.206+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/2023104670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 0x563c391db690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.207+0000 7fa8f3400640 1 -- 172.21.15.32:0/2023104670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e34850 msgr2=0x563c391dac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.207+0000 7fa8f3400640 1 --2- 172.21.15.32:0/2023104670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e34850 0x563c391dac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.207+0000 7fa8f3400640 1 -- 172.21.15.32:0/2023104670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 msgr2=0x563c391db690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.207+0000 7fa8f3400640 1 --2- 172.21.15.32:0/2023104670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 0x563c391db690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.207+0000 7fa8f3400640 1 -- 172.21.15.32:0/2023104670 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c390d2fa0 con 0x563c38e0b990 2024-08-31T07:28:44.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.207+0000 7fa8f3400640 1 --2- 172.21.15.32:0/2023104670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391db150 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4102650 tx=0x7fa8e4108f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=691c68646ba34e9d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:44.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.208+0000 7fa8e8c00640 1 -- 172.21.15.32:0/2023104670 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e410b7f0 con 0x563c38e0b990 2024-08-31T07:28:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.208+0000 7fa8e8c00640 1 -- 172.21.15.32:0/2023104670 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e410be40 con 0x563c38e0b990 2024-08-31T07:28:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.208+0000 7fa8e8c00640 1 -- 172.21.15.32:0/2023104670 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e41153d0 con 0x563c38e0b990 2024-08-31T07:28:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.208+0000 7fa8ebe00640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8d80b50a0 0x7fa8d80b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.208+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/2023104670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 0x563c391db690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:28:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.208+0000 7fa8ebe00640 1 -- 172.21.15.32:0/1487764558 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fa8e40035f0 con 0x7fa8f4104770 2024-08-31T07:28:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.208+0000 7fa8f3e00640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8d80b50a0 0x7fa8d80b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.210+0000 7fa901208740 1 -- 172.21.15.32:0/2023104670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 msgr2=0x563c391db150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.210+0000 7fa901208740 1 --2- 172.21.15.32:0/2023104670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391db150 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4102650 tx=0x7fa8e4108f90 comp rx=0 tx=0).stop 2024-08-31T07:28:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.210+0000 7fa901208740 1 -- 172.21.15.32:0/2023104670 shutdown_connections 2024-08-31T07:28:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.210+0000 7fa901208740 1 --2- 172.21.15.32:0/2023104670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 0x563c391db690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.210+0000 7fa901208740 1 --2- 172.21.15.32:0/2023104670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391db150 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.210+0000 7fa901208740 1 --2- 172.21.15.32:0/2023104670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e34850 0x563c391dac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.210+0000 7fa901208740 1 -- 172.21.15.32:0/2023104670 >> 172.21.15.32:0/2023104670 conn(0x563c38e28c40 msgr2=0x563c391d8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:44.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.211+0000 7fa901208740 1 -- 172.21.15.32:0/2023104670 shutdown_connections 2024-08-31T07:28:44.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.211+0000 7fa8f3e00640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8d80b50a0 0x7fa8d80b75b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec005fd0 tx=0x7fa8ec008630 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:44.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.211+0000 7fa901208740 1 -- 172.21.15.32:0/2023104670 wait complete. 2024-08-31T07:28:44.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.212+0000 7fa901208740 1 -- start start 2024-08-31T07:28:44.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.212+0000 7fa901208740 1 Processor -- start 2024-08-31T07:28:44.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.213+0000 7fa901208740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391e6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.213+0000 7fa901208740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 0x563c391e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.213+0000 7fa901208740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e34850 0x563c391f2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.213+0000 7fa901208740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563c38fa27b0 con 0x563c38e34850 2024-08-31T07:28:44.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.213+0000 7fa901208740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563c390daeb0 con 0x563c38e0b990 2024-08-31T07:28:44.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.213+0000 7fa901208740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563c3913f9e0 con 0x563c38e13b70 2024-08-31T07:28:44.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.213+0000 7fa901208740 10 client.0 _ll_register_callbacks cb 0x563c3913d0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:44.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.214+0000 7fa8f3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391e6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.214+0000 7fa8f3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391e6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59532/0 (socket says 172.21.15.32:59532) 2024-08-31T07:28:44.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.214+0000 7fa8f3400640 1 -- 172.21.15.32:0/3121524586 learned_addr learned my addr 172.21.15.32:0/3121524586 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:44.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.214+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 0x563c391e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.214+0000 7fa8f3e00640 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e34850 0x563c391f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.215+0000 7fa8f3400640 1 -- 172.21.15.32:0/3121524586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 msgr2=0x563c391e68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:44.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.215+0000 7fa8f3400640 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 0x563c391e68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.215+0000 7fa8f3400640 1 -- 172.21.15.32:0/3121524586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e34850 msgr2=0x563c391f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:44.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.215+0000 7fa8f3400640 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e34850 0x563c391f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.215+0000 7fa8f3400640 1 -- 172.21.15.32:0/3121524586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c390e5f40 con 0x563c38e0b990 2024-08-31T07:28:44.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.215+0000 7fa8f3400640 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391e6370 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40c7700 tx=0x7fa8e41085a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf596058ac268842 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:44.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.215+0000 7fa8dd600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:28:44.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.216+0000 7fa8dd600640 1 -- 172.21.15.32:0/3121524586 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4035070 con 0x563c38e0b990 2024-08-31T07:28:44.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.216+0000 7fa8dd600640 1 -- 172.21.15.32:0/3121524586 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fa8e40c7c70 con 0x563c38e0b990 2024-08-31T07:28:44.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.216+0000 7fa901208740 1 -- 172.21.15.32:0/3121524586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563c391523a0 con 0x563c38e0b990 2024-08-31T07:28:44.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.217+0000 7fa8dd600640 1 -- 172.21.15.32:0/3121524586 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa8e403f030 con 0x563c38e0b990 2024-08-31T07:28:44.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.217+0000 7fa901208740 10 client.28691 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:44.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.217+0000 7fa901208740 1 -- 172.21.15.32:0/3121524586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x563c38eaeba0 con 0x563c38e0b990 2024-08-31T07:28:44.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.217+0000 7fa8dd600640 1 -- 172.21.15.32:0/3121524586 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563c391523a0 con 0x563c38e0b990 2024-08-31T07:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.217+0000 7fa8dd600640 1 -- 172.21.15.32:0/3121524586 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fa8e4108020 con 0x563c38e0b990 2024-08-31T07:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.218+0000 7fa901208740 10 client.28691 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.218+0000 7fa901208740 10 client.28691 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.218+0000 7fa901208740 20 client.28691 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.218+0000 7fa901208740 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c38df31d0 0x563c391f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.218+0000 7fa901208740 4 client.28691 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.218+0000 7fa901208740 1 -- 172.21.15.32:0/3121524586 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c3914de40 con 0x563c38df31d0 2024-08-31T07:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.218+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c38df31d0 0x563c391f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.220+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c38df31d0 0x563c391f7e50 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94db960f6545b337 server_cookie=52bb79fc599ed3f3 in_seq=0 out_seq=0 2024-08-31T07:28:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.220+0000 7fa8dd600640 10 client.28691 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa8dd600640 1 -- 172.21.15.32:0/3121524586 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c3914de40 con 0x563c38df31d0 2024-08-31T07:28:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa8dd600640 10 client.28691 handle_command_reply: tid=0 2024-08-31T07:28:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 client.28691 shutdown 2024-08-31T07:28:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 -- 172.21.15.32:0/3121524586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 msgr2=0x563c391e6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391e6370 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40c7700 tx=0x7fa8e41085a0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 -- 172.21.15.32:0/3121524586 shutdown_connections 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c38df31d0 0x563c391f7e50 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e34850 0x563c391f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 0x563c391e68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 --2- 172.21.15.32:0/3121524586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e0b990 0x563c391e6370 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 -- 172.21.15.32:0/3121524586 >> 172.21.15.32:0/3121524586 conn(0x563c38e28c40 msgr2=0x563c391ddda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa8dd600640 0 client.28691 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 -- 172.21.15.32:0/3121524586 shutdown_connections 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.221+0000 7fa901208740 1 -- 172.21.15.32:0/3121524586 wait complete. 2024-08-31T07:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.222+0000 7fa901208740 20 client.28691 trim_cache size 0 max 16384 2024-08-31T07:28:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.250+0000 7fa901208740 1 Processor -- start 2024-08-31T07:28:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.250+0000 7fa901208740 1 -- start start 2024-08-31T07:28:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.251+0000 7fa901208740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e0b990 0x563c391e6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.251+0000 7fa901208740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 0x563c391dc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.251+0000 7fa901208740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 0x563c391f2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.251+0000 7fa901208740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41be380 con 0x563c38e13b70 2024-08-31T07:28:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.251+0000 7fa901208740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41bffe0 con 0x563c38e34850 2024-08-31T07:28:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.251+0000 7fa901208740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41bea50 con 0x563c38e0b990 2024-08-31T07:28:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.251+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 0x563c391f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.252+0000 7fa8f3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e0b990 0x563c391e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.252+0000 7fa8f3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e0b990 0x563c391e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46248/0 (socket says 172.21.15.32:46248) 2024-08-31T07:28:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.252+0000 7fa8f3400640 1 -- 172.21.15.32:0/451332072 learned_addr learned my addr 172.21.15.32:0/451332072 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.252+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/451332072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 0x563c391dc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.254+0000 7fa8f3400640 1 -- 172.21.15.32:0/451332072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 msgr2=0x563c391f2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.255+0000 7fa8f3400640 1 --2- 172.21.15.32:0/451332072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 0x563c391f2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.255+0000 7fa8f3400640 1 -- 172.21.15.32:0/451332072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 msgr2=0x563c391dc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.255+0000 7fa8f3400640 1 --2- 172.21.15.32:0/451332072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 0x563c391dc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.255+0000 7fa8f3400640 1 -- 172.21.15.32:0/451332072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c391e7630 con 0x563c38e0b990 2024-08-31T07:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.255+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/451332072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 0x563c391dc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:28:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.256+0000 7fa8f3400640 1 --2- 172.21.15.32:0/451332072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e0b990 0x563c391e6790 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40cb610 tx=0x7fa8e403b750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=887779169581db87 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.256+0000 7fa8dfe00640 1 -- 172.21.15.32:0/451332072 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4035020 con 0x563c38e0b990 2024-08-31T07:28:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.256+0000 7fa8dfe00640 1 -- 172.21.15.32:0/451332072 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e4108b90 con 0x563c38e0b990 2024-08-31T07:28:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.256+0000 7fa8dfe00640 1 -- 172.21.15.32:0/451332072 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40cb190 con 0x563c38e0b990 2024-08-31T07:28:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.257+0000 7fa901208740 1 -- 172.21.15.32:0/451332072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e0b990 msgr2=0x563c391e6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.257+0000 7fa901208740 1 --2- 172.21.15.32:0/451332072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e0b990 0x563c391e6790 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40cb610 tx=0x7fa8e403b750 comp rx=0 tx=0).stop 2024-08-31T07:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.258+0000 7fa901208740 1 -- 172.21.15.32:0/451332072 shutdown_connections 2024-08-31T07:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.258+0000 7fa901208740 1 --2- 172.21.15.32:0/451332072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 0x563c391f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.258+0000 7fa901208740 1 --2- 172.21.15.32:0/451332072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e13b70 0x563c391dc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.258+0000 7fa901208740 1 --2- 172.21.15.32:0/451332072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e0b990 0x563c391e6790 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.258+0000 7fa901208740 1 -- 172.21.15.32:0/451332072 >> 172.21.15.32:0/451332072 conn(0x563c38e28c40 msgr2=0x563c391dbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.258+0000 7fa901208740 1 -- 172.21.15.32:0/451332072 shutdown_connections 2024-08-31T07:28:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.258+0000 7fa901208740 1 -- 172.21.15.32:0/451332072 wait complete. 2024-08-31T07:28:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.259+0000 7fa901208740 1 -- start start 2024-08-31T07:28:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.259+0000 7fa901208740 1 Processor -- start 2024-08-31T07:28:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.260+0000 7fa901208740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e0b990 0x563c391fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.260+0000 7fa901208740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 0x563c391f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.261+0000 7fa901208740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 0x563c391f7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.261+0000 7fa901208740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41bf850 con 0x563c38e0b990 2024-08-31T07:28:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.261+0000 7fa901208740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41bf150 con 0x563c38e34850 2024-08-31T07:28:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.261+0000 7fa901208740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41bc470 con 0x563c38e13b70 2024-08-31T07:28:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.261+0000 7fa901208740 10 client.0 _ll_register_callbacks cb 0x7fa8ec0084d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.261+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 0x563c391f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.261+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 0x563c391f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59558/0 (socket says 172.21.15.32:59558) 2024-08-31T07:28:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.261+0000 7fa8f3e00640 1 -- 172.21.15.32:0/2187082370 learned_addr learned my addr 172.21.15.32:0/2187082370 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.261+0000 7fa8f3400640 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e0b990 0x563c391fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.261+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 0x563c391f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.262+0000 7fa8f2a00640 1 -- 172.21.15.32:0/2187082370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 msgr2=0x563c391f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.262+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 0x563c391f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.262+0000 7fa8f2a00640 1 -- 172.21.15.32:0/2187082370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e0b990 msgr2=0x563c391fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.262+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e0b990 0x563c391fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.262+0000 7fa8f2a00640 1 -- 172.21.15.32:0/2187082370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c392615c0 con 0x563c38e13b70 2024-08-31T07:28:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.262+0000 7fa8f3400640 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e0b990 0x563c391fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:28:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.262+0000 7fa8f2a00640 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 0x563c391f6e40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x563c390db000 tx=0x7fa8e0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82636e61923c1df9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.262+0000 7fa8dd600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.263+0000 7fa8dd600640 1 -- 172.21.15.32:0/2187082370 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e001e600 con 0x563c38e13b70 2024-08-31T07:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.263+0000 7fa8dd600640 1 -- 172.21.15.32:0/2187082370 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fa8e001e7a0 con 0x563c38e13b70 2024-08-31T07:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.263+0000 7fa901208740 1 -- 172.21.15.32:0/2187082370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e4108b90 con 0x563c38e13b70 2024-08-31T07:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.263+0000 7fa8dd600640 1 -- 172.21.15.32:0/2187082370 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa8e001e9c0 con 0x563c38e13b70 2024-08-31T07:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.263+0000 7fa901208740 10 client.28696 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.263+0000 7fa901208740 1 -- 172.21.15.32:0/2187082370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x563c39262790 con 0x563c38e13b70 2024-08-31T07:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.264+0000 7fa8dd600640 1 -- 172.21.15.32:0/2187082370 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e4108b90 con 0x563c38e13b70 2024-08-31T07:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.264+0000 7fa8dd600640 1 -- 172.21.15.32:0/2187082370 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fa8e0023420 con 0x563c38e13b70 2024-08-31T07:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.264+0000 7fa901208740 10 client.28696 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.264+0000 7fa901208740 10 client.28696 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.264+0000 7fa901208740 20 client.28696 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.264+0000 7fa901208740 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c38dc8580 0x563c39265080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.264+0000 7fa901208740 4 client.28696 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.25, "await_for": 45.0}] 2024-08-31T07:28:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.264+0000 7fa901208740 1 -- 172.21.15.32:0/2187082370 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.25, "await_for": 45.0}) -- 0x563c3914b580 con 0x563c38dc8580 2024-08-31T07:28:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.264+0000 7fa8f3400640 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c38dc8580 0x563c39265080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.266+0000 7fa8f3400640 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c38dc8580 0x563c39265080 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9e8208b65c48ce4 server_cookie=3a5239f8ef6fec19 in_seq=0 out_seq=0 2024-08-31T07:28:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.266+0000 7fa8dd600640 10 client.28696 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:44.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52824 6.17 MB/sec execute 24 sec latency 74.650 ms 2024-08-31T07:28:44.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:44 smithi032 ceph-mon[17252]: pgmap v867: 97 pgs: 97 active+clean; 866 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 757 KiB/s wr, 68 op/s 2024-08-31T07:28:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:44 smithi073 ceph-mon[21995]: pgmap v867: 97 pgs: 97 active+clean; 866 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 757 KiB/s wr, 68 op/s 2024-08-31T07:28:44.695 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:28:44.696 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:28:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:44 smithi084 ceph-mon[22387]: pgmap v867: 97 pgs: 97 active+clean; 866 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 757 KiB/s wr, 68 op/s 2024-08-31T07:28:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa8dd600640 1 -- 172.21.15.32:0/2187082370 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563c3914b580 con 0x563c38dc8580 2024-08-31T07:28:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa8dd600640 10 client.28696 handle_command_reply: tid=0 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 client.28696 shutdown 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 -- 172.21.15.32:0/2187082370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 msgr2=0x563c391f6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 0x563c391f6e40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x563c390db000 tx=0x7fa8e0009f90 comp rx=0 tx=0).stop 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 -- 172.21.15.32:0/2187082370 shutdown_connections 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c38dc8580 0x563c39265080 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c38e34850 0x563c391f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c38e13b70 0x563c391f6e40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 --2- 172.21.15.32:0/2187082370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c38e0b990 0x563c391fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 -- 172.21.15.32:0/2187082370 >> 172.21.15.32:0/2187082370 conn(0x563c38e28c40 msgr2=0x563c391d7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa8dd600640 0 client.28696 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 -- 172.21.15.32:0/2187082370 shutdown_connections 2024-08-31T07:28:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.810+0000 7fa901208740 1 -- 172.21.15.32:0/2187082370 wait complete. 2024-08-31T07:28:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.811+0000 7fa901208740 20 client.28696 trim_cache size 0 max 16384 2024-08-31T07:28:44.813 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:28:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 59, 2024-08-31T07:28:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:28:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 50, 2024-08-31T07:28:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:28:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "a2a13203": { 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 50, 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 141.2, 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:28:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:28:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:28:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:28:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:28:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:28:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:28:44.815 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.811+0000 7fa8fa400640 1 -- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8d80b50a0 msgr2=0x7fa8d80b75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.811+0000 7fa8fa400640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8d80b50a0 0x7fa8d80b75b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec005fd0 tx=0x7fa8ec008630 comp rx=0 tx=0).stop 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.811+0000 7fa8fa400640 1 -- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f4104770 msgr2=0x7fa8f4074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.811+0000 7fa8fa400640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f4104770 0x7fa8f4074c60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4006010 tx=0x7fa8e4004060 comp rx=0 tx=0).stop 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.812+0000 7fa8fa400640 1 -- 172.21.15.32:0/1487764558 shutdown_connections 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.812+0000 7fa8fa400640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa8d80b50a0 0x7fa8d80b75b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.812+0000 7fa8fa400640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f410dbd0 0x7fa8f406e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.812+0000 7fa8fa400640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8f41050c0 0x7fa8f406df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.812+0000 7fa8fa400640 1 --2- 172.21.15.32:0/1487764558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f4104770 0x7fa8f4074c60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.812+0000 7fa8fa400640 1 -- 172.21.15.32:0/1487764558 >> 172.21.15.32:0/1487764558 conn(0x7fa8f4079500 msgr2=0x7fa8f40ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.812+0000 7fa8fa400640 1 -- 172.21.15.32:0/1487764558 shutdown_connections 2024-08-31T07:28:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:44.812+0000 7fa8fa400640 1 -- 172.21.15.32:0/1487764558 wait complete. 2024-08-31T07:28:45.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 5.92 MB/sec execute 25 sec latency 1061.319 ms 2024-08-31T07:28:45.606 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a2a13203', quiesce took 8.0 seconds. Will release after: 13.25 2024-08-31T07:28:45.633 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:46.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.329+0000 7fe6b3200640 1 -- 172.21.15.32:0/551908010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b40754f0 msgr2=0x7fe6b4075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:46.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.329+0000 7fe6b3200640 1 --2- 172.21.15.32:0/551908010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b40754f0 0x7fe6b4075900 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009900 tx=0x7fe6a402a410 comp rx=0 tx=0).stop 2024-08-31T07:28:46.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.329+0000 7fe6b3200640 1 -- 172.21.15.32:0/551908010 shutdown_connections 2024-08-31T07:28:46.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.329+0000 7fe6b3200640 1 --2- 172.21.15.32:0/551908010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6b410fcf0 0x7fe6b4112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.329+0000 7fe6b3200640 1 --2- 172.21.15.32:0/551908010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b4075e40 0x7fe6b410f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.329+0000 7fe6b3200640 1 --2- 172.21.15.32:0/551908010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b40754f0 0x7fe6b4075900 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.329+0000 7fe6b3200640 1 -- 172.21.15.32:0/551908010 >> 172.21.15.32:0/551908010 conn(0x7fe6b40ff180 msgr2=0x7fe6b41015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:46.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.329+0000 7fe6b3200640 1 -- 172.21.15.32:0/551908010 shutdown_connections 2024-08-31T07:28:46.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.329+0000 7fe6b3200640 1 -- 172.21.15.32:0/551908010 wait complete. 2024-08-31T07:28:46.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.330+0000 7fe6b3200640 1 Processor -- start 2024-08-31T07:28:46.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.330+0000 7fe6b3200640 1 -- start start 2024-08-31T07:28:46.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b40754f0 0x7fe6b419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6b4075e40 0x7fe6b419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b410fcf0 0x7fe6b41a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b41158b0 con 0x7fe6b40754f0 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b4115730 con 0x7fe6b4075e40 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b4115a30 con 0x7fe6b410fcf0 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b40754f0 0x7fe6b419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6b4075e40 0x7fe6b419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6b4075e40 0x7fe6b419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45358/0 (socket says 172.21.15.32:45358) 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b1400640 1 -- 172.21.15.32:0/4291623089 learned_addr learned my addr 172.21.15.32:0/4291623089 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b40754f0 0x7fe6b419c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58268/0 (socket says 172.21.15.32:58268) 2024-08-31T07:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.331+0000 7fe6b2800640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b410fcf0 0x7fe6b41a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.332+0000 7fe6b1400640 1 -- 172.21.15.32:0/4291623089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b410fcf0 msgr2=0x7fe6b41a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:46.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.332+0000 7fe6b1400640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b410fcf0 0x7fe6b41a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.332+0000 7fe6b1400640 1 -- 172.21.15.32:0/4291623089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b40754f0 msgr2=0x7fe6b419c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:46.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.332+0000 7fe6b1400640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b40754f0 0x7fe6b419c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.332+0000 7fe6b1400640 1 -- 172.21.15.32:0/4291623089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b41a21d0 con 0x7fe6b4075e40 2024-08-31T07:28:46.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.332+0000 7fe6b1400640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6b4075e40 0x7fe6b419c800 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000e9f0 tx=0x7fe6a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:46.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.332+0000 7fe6aaa00640 1 -- 172.21.15.32:0/4291623089 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a000ccc0 con 0x7fe6b4075e40 2024-08-31T07:28:46.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.332+0000 7fe6b3200640 1 -- 172.21.15.32:0/4291623089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b41a24c0 con 0x7fe6b4075e40 2024-08-31T07:28:46.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.333+0000 7fe6b3200640 1 -- 172.21.15.32:0/4291623089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b41a8c20 con 0x7fe6b4075e40 2024-08-31T07:28:46.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.333+0000 7fe6aaa00640 1 -- 172.21.15.32:0/4291623089 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6a00108e0 con 0x7fe6b4075e40 2024-08-31T07:28:46.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.333+0000 7fe6aaa00640 1 -- 172.21.15.32:0/4291623089 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00187a0 con 0x7fe6b4075e40 2024-08-31T07:28:46.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.335+0000 7fe6bfc1b740 1 Processor -- start 2024-08-31T07:28:46.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6bfc1b740 1 -- start start 2024-08-31T07:28:46.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 0x5601270ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d1d490 0x5601270ecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d25670 0x5601270ed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560127062750 con 0x560126d46350 2024-08-31T07:28:46.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560127051ce0 con 0x560126d25670 2024-08-31T07:28:46.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560127055c90 con 0x560126d1d490 2024-08-31T07:28:46.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6aaa00640 1 -- 172.21.15.32:0/4291623089 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe6a0010430 con 0x7fe6b4075e40 2024-08-31T07:28:46.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6b2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 0x5601270ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6b2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 0x5601270ec7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58278/0 (socket says 172.21.15.32:58278) 2024-08-31T07:28:46.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.336+0000 7fe6b2800640 1 -- 172.21.15.32:0/597835492 learned_addr learned my addr 172.21.15.32:0/597835492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:46.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.337+0000 7fe6b1e00640 1 --2- 172.21.15.32:0/597835492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d1d490 0x5601270ecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.337+0000 7fe6b1400640 1 --2- 172.21.15.32:0/597835492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d25670 0x5601270ed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.337+0000 7fe6b2800640 1 -- 172.21.15.32:0/597835492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d1d490 msgr2=0x5601270ecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.337+0000 7fe6b2800640 1 --2- 172.21.15.32:0/597835492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d1d490 0x5601270ecd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.337+0000 7fe6b2800640 1 -- 172.21.15.32:0/597835492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d25670 msgr2=0x5601270ed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.337+0000 7fe6b2800640 1 --2- 172.21.15.32:0/597835492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d25670 0x5601270ed250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.337+0000 7fe6b2800640 1 -- 172.21.15.32:0/597835492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560126db1930 con 0x560126d46350 2024-08-31T07:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.337+0000 7fe6b2800640 1 --2- 172.21.15.32:0/597835492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 0x5601270ec7d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009fd0 tx=0x7fe6ac0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928772f8aafec72d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.338+0000 7fe69f400640 1 -- 172.21.15.32:0/597835492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac004d20 con 0x560126d46350 2024-08-31T07:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.338+0000 7fe69f400640 1 -- 172.21.15.32:0/597835492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6ac004050 con 0x560126d46350 2024-08-31T07:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.338+0000 7fe69f400640 1 -- 172.21.15.32:0/597835492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac005220 con 0x560126d46350 2024-08-31T07:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.338+0000 7fe6aaa00640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6980b4d20 0x7fe6980b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.338+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/597835492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 msgr2=0x5601270ec7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.338+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/597835492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 0x5601270ec7d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009fd0 tx=0x7fe6ac0118d0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.339+0000 7fe6aaa00640 1 -- 172.21.15.32:0/4291623089 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fe6a00e5850 con 0x7fe6b4075e40 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.339+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/597835492 shutdown_connections 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.339+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/597835492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d25670 0x5601270ed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.339+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/597835492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d1d490 0x5601270ecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.339+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/597835492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 0x5601270ec7d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.339+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/597835492 >> 172.21.15.32:0/597835492 conn(0x560126d3a740 msgr2=0x5601270ea6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.339+0000 7fe6b1e00640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6980b4d20 0x7fe6980b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.339+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/597835492 shutdown_connections 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.339+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/597835492 wait complete. 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.340+0000 7fe6b1e00640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6980b4d20 0x7fe6980b7230 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4005ab0 tx=0x7fe6a4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.340+0000 7fe6bfc1b740 1 -- start start 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.340+0000 7fe6bfc1b740 1 Processor -- start 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.340+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 0x5601270f8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.340+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 0x5601270f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 0x5601271042d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x560126eb4890 con 0x560126d46350 2024-08-31T07:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560126fd8cd0 con 0x560126d1d490 2024-08-31T07:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5601270fafb0 con 0x560126d25670 2024-08-31T07:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6bfc1b740 10 client.0 _ll_register_callbacks cb 0x56012704cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6b1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 0x5601270f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6b1400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 0x5601270f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6b1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 0x5601270f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:45376/0 (socket says 172.21.15.32:45376) 2024-08-31T07:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6b1e00640 1 -- 172.21.15.32:0/2517092692 learned_addr learned my addr 172.21.15.32:0/2517092692 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6b2800640 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 0x5601271042d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6b1400640 1 -- 172.21.15.32:0/2517092692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 msgr2=0x5601270f8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6b1400640 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 0x5601270f8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6b1400640 1 -- 172.21.15.32:0/2517092692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 msgr2=0x5601271042d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6b1400640 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 0x5601271042d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.341+0000 7fe6b1400640 1 -- 172.21.15.32:0/2517092692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601270221b0 con 0x560126d25670 2024-08-31T07:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.342+0000 7fe6b1400640 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 0x5601270f88c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00e2f90 tx=0x7fe6a00abe70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95b10fb1e039048b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.342+0000 7fe693e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.342+0000 7fe693e00640 1 -- 172.21.15.32:0/2517092692 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00abaf0 con 0x560126d25670 2024-08-31T07:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.342+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/2517092692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6ac004050 con 0x560126d25670 2024-08-31T07:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.342+0000 7fe693e00640 1 -- 172.21.15.32:0/2517092692 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fe6a00e3210 con 0x560126d25670 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.342+0000 7fe693e00640 1 -- 172.21.15.32:0/2517092692 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe6a00e3410 con 0x560126d25670 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.342+0000 7fe6bfc1b740 10 client.28708 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.342+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/2517092692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560127106ff0 con 0x560126d25670 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.342+0000 7fe693e00640 1 -- 172.21.15.32:0/2517092692 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6ac004050 con 0x560126d25670 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.343+0000 7fe693e00640 1 -- 172.21.15.32:0/2517092692 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fe6a00a7950 con 0x560126d25670 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.343+0000 7fe6bfc1b740 10 client.28708 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.343+0000 7fe6bfc1b740 10 client.28708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.343+0000 7fe6bfc1b740 20 client.28708 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.343+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560126d04cd0 0x56012710a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.343+0000 7fe6bfc1b740 4 client.28708 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.343+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/2517092692 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601270646a0 con 0x560126d04cd0 2024-08-31T07:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.343+0000 7fe6b1e00640 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560126d04cd0 0x56012710a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.347 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 5.70 MB/sec execute 26 sec latency 2061.382 ms 2024-08-31T07:28:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.344+0000 7fe6b1e00640 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560126d04cd0 0x56012710a4d0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=debdb75e77afd1c8 server_cookie=cc6e78af9fb4136a in_seq=0 out_seq=0 2024-08-31T07:28:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.344+0000 7fe693e00640 10 client.28708 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe693e00640 1 -- 172.21.15.32:0/2517092692 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601270646a0 con 0x560126d04cd0 2024-08-31T07:28:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe693e00640 10 client.28708 handle_command_reply: tid=0 2024-08-31T07:28:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 client.28708 shutdown 2024-08-31T07:28:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/2517092692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 msgr2=0x5601270f88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 0x5601270f88c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00e2f90 tx=0x7fe6a00abe70 comp rx=0 tx=0).stop 2024-08-31T07:28:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/2517092692 shutdown_connections 2024-08-31T07:28:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560126d04cd0 0x56012710a4d0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d46350 0x5601271042d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 0x5601270f88c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/2517092692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 0x5601270f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/2517092692 >> 172.21.15.32:0/2517092692 conn(0x560126d3a740 msgr2=0x5601270ef9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe693e00640 0 client.28708 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/2517092692 shutdown_connections 2024-08-31T07:28:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.345+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/2517092692 wait complete. 2024-08-31T07:28:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.346+0000 7fe6bfc1b740 20 client.28708 trim_cache size 0 max 16384 2024-08-31T07:28:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.364+0000 7fe6bfc1b740 1 Processor -- start 2024-08-31T07:28:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.364+0000 7fe6bfc1b740 1 -- start start 2024-08-31T07:28:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 0x5601271041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d25670 0x5601270edf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d46350 0x5601270f5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b41b1450 con 0x560126d25670 2024-08-31T07:28:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b41b30b0 con 0x560126d1d490 2024-08-31T07:28:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b41b1b20 con 0x560126d46350 2024-08-31T07:28:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6b1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d25670 0x5601270edf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6b2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d46350 0x5601270f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6b1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 0x5601271041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6b1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d25670 0x5601270edf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58302/0 (socket says 172.21.15.32:58302) 2024-08-31T07:28:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.365+0000 7fe6b1400640 1 -- 172.21.15.32:0/4223586074 learned_addr learned my addr 172.21.15.32:0/4223586074 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.366+0000 7fe6b2800640 1 -- 172.21.15.32:0/4223586074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 msgr2=0x5601271041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.366+0000 7fe6b2800640 1 --2- 172.21.15.32:0/4223586074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 0x5601271041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.366+0000 7fe6b2800640 1 -- 172.21.15.32:0/4223586074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d25670 msgr2=0x5601270edf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.366+0000 7fe6b2800640 1 --2- 172.21.15.32:0/4223586074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d25670 0x5601270edf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.366+0000 7fe6b2800640 1 -- 172.21.15.32:0/4223586074 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601270ed610 con 0x560126d46350 2024-08-31T07:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.366+0000 7fe6b1e00640 1 --2- 172.21.15.32:0/4223586074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 0x5601271041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.366+0000 7fe6b1400640 1 --2- 172.21.15.32:0/4223586074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d25670 0x5601270edf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.366+0000 7fe6b2800640 1 --2- 172.21.15.32:0/4223586074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d46350 0x5601270f5370 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac003fd0 tx=0x7fe6ac009cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28b9d9b2e022411 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.367+0000 7fe69ea00640 1 -- 172.21.15.32:0/4223586074 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac003cf0 con 0x560126d46350 2024-08-31T07:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.367+0000 7fe69ea00640 1 -- 172.21.15.32:0/4223586074 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6ac004050 con 0x560126d46350 2024-08-31T07:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.367+0000 7fe69ea00640 1 -- 172.21.15.32:0/4223586074 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0052a0 con 0x560126d46350 2024-08-31T07:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.367+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/4223586074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d46350 msgr2=0x5601270f5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.367+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/4223586074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d46350 0x5601270f5370 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac003fd0 tx=0x7fe6ac009cc0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.368+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/4223586074 shutdown_connections 2024-08-31T07:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.368+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/4223586074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d46350 0x5601270f5370 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.368+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/4223586074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d25670 0x5601270edf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.368+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/4223586074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d1d490 0x5601271041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.368+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/4223586074 >> 172.21.15.32:0/4223586074 conn(0x560126d3a740 msgr2=0x5601270ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.368+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/4223586074 shutdown_connections 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.368+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/4223586074 wait complete. 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.368+0000 7fe6bfc1b740 1 -- start start 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.368+0000 7fe6bfc1b740 1 Processor -- start 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d1d490 0x560127100c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 0x5601270fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6bfc1b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d46350 0x5601270fd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b41b2920 con 0x560126d1d490 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b41b2220 con 0x560126d46350 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6bfc1b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b41af450 con 0x560126d25670 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6bfc1b740 10 client.0 _ll_register_callbacks cb 0x7fe6a00eb220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d1d490 0x560127100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d1d490 0x560127100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58308/0 (socket says 172.21.15.32:58308) 2024-08-31T07:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6b1e00640 1 -- 172.21.15.32:0/763112222 learned_addr learned my addr 172.21.15.32:0/763112222 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6b1400640 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 0x5601270fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.369+0000 7fe6b2800640 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d46350 0x5601270fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe6b1400640 1 -- 172.21.15.32:0/763112222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d46350 msgr2=0x5601270fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe6b1400640 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d46350 0x5601270fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe6b1400640 1 -- 172.21.15.32:0/763112222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d1d490 msgr2=0x560127100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe6b1400640 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d1d490 0x560127100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe6b1400640 1 -- 172.21.15.32:0/763112222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560127174f30 con 0x560126d25670 2024-08-31T07:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe6b1e00640 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d1d490 0x560127100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe6b1400640 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 0x5601270fd130 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x560126fd9260 tx=0x7fe6a0010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7314af0ad9ccc5c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe693e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe693e00640 1 -- 172.21.15.32:0/763112222 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0093110 con 0x560126d25670 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe693e00640 1 -- 172.21.15.32:0/763112222 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fe6a00971c0 con 0x560126d25670 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/763112222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6ac004050 con 0x560126d25670 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe693e00640 1 -- 172.21.15.32:0/763112222 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe6a00b07a0 con 0x560126d25670 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe6bfc1b740 10 client.28714 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.370+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/763112222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5601271760c0 con 0x560126d25670 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.371+0000 7fe693e00640 1 -- 172.21.15.32:0/763112222 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6ac004050 con 0x560126d25670 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.371+0000 7fe693e00640 1 -- 172.21.15.32:0/763112222 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fe6a00e3210 con 0x560126d25670 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.371+0000 7fe6bfc1b740 10 client.28714 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.371+0000 7fe6bfc1b740 10 client.28714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.371+0000 7fe6bfc1b740 20 client.28714 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.371+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560126cda080 0x560127178770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.371+0000 7fe6bfc1b740 4 client.28714 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.371+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/763112222 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560127060140 con 0x560126cda080 2024-08-31T07:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.371+0000 7fe6b1e00640 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560126cda080 0x560127178770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.372+0000 7fe6b1e00640 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560126cda080 0x560127178770 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=900e96d91b6cd0a server_cookie=163d4b87cb5ec5c7 in_seq=0 out_seq=0 2024-08-31T07:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.372+0000 7fe693e00640 10 client.28714 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.373+0000 7fe693e00640 1 -- 172.21.15.32:0/763112222 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560127060140 con 0x560126cda080 2024-08-31T07:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.373+0000 7fe693e00640 10 client.28714 handle_command_reply: tid=0 2024-08-31T07:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.373+0000 7fe6bfc1b740 1 client.28714 shutdown 2024-08-31T07:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/763112222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 msgr2=0x5601270fd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 0x5601270fd130 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x560126fd9260 tx=0x7fe6a0010800 comp rx=0 tx=0).stop 2024-08-31T07:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/763112222 shutdown_connections 2024-08-31T07:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x560126cda080 0x560127178770 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560126d46350 0x5601270fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560126d25670 0x5601270fd130 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 1 --2- 172.21.15.32:0/763112222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560126d1d490 0x560127100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe693e00640 0 client.28714 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/763112222 >> 172.21.15.32:0/763112222 conn(0x560126d3a740 msgr2=0x5601270f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/763112222 shutdown_connections 2024-08-31T07:28:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 1 -- 172.21.15.32:0/763112222 wait complete. 2024-08-31T07:28:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.374+0000 7fe6bfc1b740 20 client.28714 trim_cache size 0 max 16384 2024-08-31T07:28:46.377 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:28:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:28:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "05113edd-ca51-44d5-a01f-804676bde5ad", 2024-08-31T07:28:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:28:46.378 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:28:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 -- 172.21.15.32:0/4291623089 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6980b4d20 msgr2=0x7fe6980b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6980b4d20 0x7fe6980b7230 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4005ab0 tx=0x7fe6a4005a20 comp rx=0 tx=0).stop 2024-08-31T07:28:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 -- 172.21.15.32:0/4291623089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6b4075e40 msgr2=0x7fe6b419c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6b4075e40 0x7fe6b419c800 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000e9f0 tx=0x7fe6a000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 -- 172.21.15.32:0/4291623089 shutdown_connections 2024-08-31T07:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe6980b4d20 0x7fe6980b7230 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6b410fcf0 0x7fe6b41a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6b4075e40 0x7fe6b419c800 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 --2- 172.21.15.32:0/4291623089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b40754f0 0x7fe6b419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 -- 172.21.15.32:0/4291623089 >> 172.21.15.32:0/4291623089 conn(0x7fe6b40ff180 msgr2=0x7fe6b4109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 -- 172.21.15.32:0/4291623089 shutdown_connections 2024-08-31T07:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:46.375+0000 7fe6b3200640 1 -- 172.21.15.32:0/4291623089 wait complete. 2024-08-31T07:28:46.408 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:46 smithi032 ceph-mon[17252]: pgmap v868: 97 pgs: 97 active+clean; 873 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-08-31T07:28:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:46 smithi073 ceph-mon[21995]: pgmap v868: 97 pgs: 97 active+clean; 873 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-08-31T07:28:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:46 smithi084 ceph-mon[22387]: pgmap v868: 97 pgs: 97 active+clean; 873 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-08-31T07:28:47.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:28:47.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 5.49 MB/sec execute 27 sec latency 3061.441 ms 2024-08-31T07:28:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:28:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:28:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:28:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:28:47.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:28:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:28:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:47 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:28:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:28:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:28:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:28:47.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:28:47.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:28:47.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:47 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:47.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:28:47.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:28:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:28:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:28:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:28:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:28:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:47.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:47 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:28:48.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 5.29 MB/sec execute 28 sec latency 4061.503 ms 2024-08-31T07:28:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.344+0000 7fda9e800640 1 -- 172.21.15.32:0/2227944834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda98109070 msgr2=0x7fda98109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.344+0000 7fda9e800640 1 --2- 172.21.15.32:0/2227944834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda98109070 0x7fda98109480 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fda84009900 tx=0x7fda8402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.344+0000 7fda9e800640 1 -- 172.21.15.32:0/2227944834 shutdown_connections 2024-08-31T07:28:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.344+0000 7fda9e800640 1 --2- 172.21.15.32:0/2227944834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda981099c0 0x7fda981101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.344+0000 7fda9e800640 1 --2- 172.21.15.32:0/2227944834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda98109070 0x7fda98109480 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.344+0000 7fda9e800640 1 --2- 172.21.15.32:0/2227944834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda98103730 0x7fda98103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.344+0000 7fda9e800640 1 -- 172.21.15.32:0/2227944834 >> 172.21.15.32:0/2227944834 conn(0x7fda980ff180 msgr2=0x7fda981015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.344+0000 7fda9e800640 1 -- 172.21.15.32:0/2227944834 shutdown_connections 2024-08-31T07:28:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.344+0000 7fda9e800640 1 -- 172.21.15.32:0/2227944834 wait complete. 2024-08-31T07:28:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.346+0000 7fda9e800640 1 Processor -- start 2024-08-31T07:28:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.346+0000 7fda9e800640 1 -- start start 2024-08-31T07:28:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.346+0000 7fda9e800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda98103730 0x7fda9819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.346+0000 7fda9e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda98109070 0x7fda9819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.346+0000 7fda9e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda981099c0 0x7fda981a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.346+0000 7fda9e800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda98113650 con 0x7fda98109070 2024-08-31T07:28:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.346+0000 7fda9e800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda981134d0 con 0x7fda981099c0 2024-08-31T07:28:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.346+0000 7fda9e800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fda981137d0 con 0x7fda98103730 2024-08-31T07:28:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.346+0000 7fda9ca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda98109070 0x7fda9819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.346+0000 7fda9ca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda98109070 0x7fda9819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58324/0 (socket says 172.21.15.32:58324) 2024-08-31T07:28:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.347+0000 7fda9ca00640 1 -- 172.21.15.32:0/1543165080 learned_addr learned my addr 172.21.15.32:0/1543165080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.347+0000 7fda9d400640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda98103730 0x7fda9819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.347+0000 7fda9de00640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda981099c0 0x7fda981a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.347+0000 7fda9ca00640 1 -- 172.21.15.32:0/1543165080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda98103730 msgr2=0x7fda9819c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.347+0000 7fda9ca00640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda98103730 0x7fda9819c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.347+0000 7fda9ca00640 1 -- 172.21.15.32:0/1543165080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda981099c0 msgr2=0x7fda981a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.347+0000 7fda9ca00640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda981099c0 0x7fda981a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.347+0000 7fda9ca00640 1 -- 172.21.15.32:0/1543165080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda981a22d0 con 0x7fda98109070 2024-08-31T07:28:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.347+0000 7fda9d400640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda98103730 0x7fda9819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.347+0000 7fda9ca00640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda98109070 0x7fda9819c900 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fda8402a910 tx=0x7fda84031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.348+0000 7fda96000640 1 -- 172.21.15.32:0/1543165080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8403f030 con 0x7fda98109070 2024-08-31T07:28:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.348+0000 7fda96000640 1 -- 172.21.15.32:0/1543165080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda84004490 con 0x7fda98109070 2024-08-31T07:28:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.348+0000 7fda9e800640 1 -- 172.21.15.32:0/1543165080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda981a2560 con 0x7fda98109070 2024-08-31T07:28:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.348+0000 7fda96000640 1 -- 172.21.15.32:0/1543165080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84035070 con 0x7fda98109070 2024-08-31T07:28:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.348+0000 7fda9e800640 1 -- 172.21.15.32:0/1543165080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda981a8d20 con 0x7fda98109070 2024-08-31T07:28:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.351+0000 7fdaa6a83740 1 Processor -- start 2024-08-31T07:28:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.351+0000 7fdaa6a83740 1 -- start start 2024-08-31T07:28:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.351+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 0x55cf5b8edb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.351+0000 7fda96000640 1 -- 172.21.15.32:0/1543165080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fda84042030 con 0x7fda98109070 2024-08-31T07:28:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8ee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b527390 0x55cf5b8ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55cf5b864770 con 0x55cf5b548070 2024-08-31T07:28:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cf5b853d00 con 0x55cf5b527390 2024-08-31T07:28:48.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cf5b857cb0 con 0x55cf5b51f1b0 2024-08-31T07:28:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fda9de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 0x55cf5b8edb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fda9ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b527390 0x55cf5b8ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fda9de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 0x55cf5b8edb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58334/0 (socket says 172.21.15.32:58334) 2024-08-31T07:28:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fda9de00640 1 -- 172.21.15.32:0/2770772561 learned_addr learned my addr 172.21.15.32:0/2770772561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fda9d400640 1 --2- 172.21.15.32:0/2770772561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fda9de00640 1 -- 172.21.15.32:0/2770772561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 msgr2=0x55cf5b8ee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fda9de00640 1 --2- 172.21.15.32:0/2770772561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8ee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fda9de00640 1 -- 172.21.15.32:0/2770772561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b527390 msgr2=0x55cf5b8ee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fda9de00640 1 --2- 172.21.15.32:0/2770772561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b527390 0x55cf5b8ee5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.352+0000 7fda9de00640 1 -- 172.21.15.32:0/2770772561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5b5b3650 con 0x55cf5b548070 2024-08-31T07:28:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.353+0000 7fda9de00640 1 --2- 172.21.15.32:0/2770772561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 0x55cf5b8edb20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fda90009f50 tx=0x7fda900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f5d38e26521c54 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.353+0000 7fda8a600640 1 -- 172.21.15.32:0/2770772561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90004d20 con 0x55cf5b548070 2024-08-31T07:28:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.353+0000 7fda8a600640 1 -- 172.21.15.32:0/2770772561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda90004050 con 0x55cf5b548070 2024-08-31T07:28:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.353+0000 7fda8a600640 1 -- 172.21.15.32:0/2770772561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90005280 con 0x55cf5b548070 2024-08-31T07:28:48.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.354+0000 7fda96000640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda800b4ca0 0x7fda800b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.354+0000 7fda96000640 1 -- 172.21.15.32:0/1543165080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fda841057e0 con 0x7fda98109070 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.354+0000 7fda9d400640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda800b4ca0 0x7fda800b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.354+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2770772561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 msgr2=0x55cf5b8edb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.354+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/2770772561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 0x55cf5b8edb20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fda90009f50 tx=0x7fda900118d0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.354+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2770772561 shutdown_connections 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.354+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/2770772561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b527390 0x55cf5b8ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.354+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/2770772561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.354+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/2770772561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 0x55cf5b8edb20 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.354+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2770772561 >> 172.21.15.32:0/2770772561 conn(0x55cf5b53c460 msgr2=0x55cf5b8eba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.355+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2770772561 shutdown_connections 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.355+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2770772561 wait complete. 2024-08-31T07:28:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.355+0000 7fda9d400640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda800b4ca0 0x7fda800b71b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55cf5b8ec7a0 tx=0x7fda8c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.355+0000 7fdaa6a83740 1 -- start start 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.355+0000 7fdaa6a83740 1 Processor -- start 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 0x55cf5b8f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 0x55cf5b905680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55cf5b6b6700 con 0x55cf5b51f1b0 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cf5b7dacb0 con 0x55cf5b548070 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cf5b8fc300 con 0x55cf5b527390 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fdaa6a83740 10 client.0 _ll_register_callbacks cb 0x55cf5b800430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fda9d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fda9ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 0x55cf5b8f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fda9d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8f96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58344/0 (socket says 172.21.15.32:58344) 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.356+0000 7fda9d400640 1 -- 172.21.15.32:0/2994290363 learned_addr learned my addr 172.21.15.32:0/2994290363 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda9de00640 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 0x55cf5b905680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda9ca00640 1 -- 172.21.15.32:0/2994290363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 msgr2=0x55cf5b905680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda9ca00640 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 0x55cf5b905680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda9ca00640 1 -- 172.21.15.32:0/2994290363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 msgr2=0x55cf5b8f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda9ca00640 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8f96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda9ca00640 1 -- 172.21.15.32:0/2994290363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5b81b870 con 0x55cf5b527390 2024-08-31T07:28:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda9d400640 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda9de00640 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 0x55cf5b905680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:28:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda9ca00640 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 0x55cf5b8f9c20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fda84102f80 tx=0x7fda8402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e78f092b24dec server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda7ea00640 1 -- 172.21.15.32:0/2994290363 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84049070 con 0x55cf5b527390 2024-08-31T07:28:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2994290363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda90004050 con 0x55cf5b527390 2024-08-31T07:28:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.357+0000 7fda7ea00640 1 -- 172.21.15.32:0/2994290363 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fda840d4db0 con 0x55cf5b527390 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fda7ea00640 1 -- 172.21.15.32:0/2994290363 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fda840cbda0 con 0x55cf5b527390 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fdaa6a83740 10 client.28726 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2994290363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cf5b908350 con 0x55cf5b527390 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fda7ea00640 1 -- 172.21.15.32:0/2994290363 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda90004050 con 0x55cf5b527390 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fda7ea00640 1 -- 172.21.15.32:0/2994290363 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fda840b71f0 con 0x55cf5b527390 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fdaa6a83740 10 client.28726 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fdaa6a83740 10 client.28726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fdaa6a83740 20 client.28726 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cf5b5069f0 0x55cf5b90b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fdaa6a83740 4 client.28726 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.358+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2994290363 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf5b8666c0 con 0x55cf5b5069f0 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.359+0000 7fda9d400640 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cf5b5069f0 0x55cf5b90b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.359+0000 7fda7ea00640 10 client.28726 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.359+0000 7fda9d400640 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cf5b5069f0 0x55cf5b90b910 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d9c8be18ad7307c server_cookie=8161f04a8448ebf in_seq=0 out_seq=0 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.360+0000 7fda7ea00640 1 -- 172.21.15.32:0/2994290363 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cf5b8666c0 con 0x55cf5b5069f0 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.360+0000 7fda7ea00640 10 client.28726 handle_command_reply: tid=0 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.360+0000 7fdaa6a83740 1 client.28726 shutdown 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2994290363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 msgr2=0x55cf5b8f9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 0x55cf5b8f9c20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fda84102f80 tx=0x7fda8402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2994290363 shutdown_connections 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cf5b5069f0 0x55cf5b90b910 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 0x55cf5b905680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 0x55cf5b8f9c20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/2994290363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fda7ea00640 0 client.28726 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2994290363 >> 172.21.15.32:0/2994290363 conn(0x55cf5b53c460 msgr2=0x55cf5b8f0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:48.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2994290363 shutdown_connections 2024-08-31T07:28:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 1 -- 172.21.15.32:0/2994290363 wait complete. 2024-08-31T07:28:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.361+0000 7fdaa6a83740 20 client.28726 trim_cache size 0 max 16384 2024-08-31T07:28:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.379+0000 7fdaa6a83740 1 Processor -- start 2024-08-31T07:28:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fdaa6a83740 1 -- start start 2024-08-31T07:28:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8ef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b527390 0x55cf5b905580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 0x55cf5b905ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda981b13f0 con 0x55cf5b548070 2024-08-31T07:28:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda981b3050 con 0x55cf5b527390 2024-08-31T07:28:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fda981b1ac0 con 0x55cf5b51f1b0 2024-08-31T07:28:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fda9de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 0x55cf5b905ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fda9d400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fda9d400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8ef260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42864/0 (socket says 172.21.15.32:42864) 2024-08-31T07:28:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fda9d400640 1 -- 172.21.15.32:0/1984371054 learned_addr learned my addr 172.21.15.32:0/1984371054 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.380+0000 7fda9ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b527390 0x55cf5b905580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.381+0000 7fda9d400640 1 -- 172.21.15.32:0/1984371054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b527390 msgr2=0x55cf5b905580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.381+0000 7fda9d400640 1 --2- 172.21.15.32:0/1984371054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b527390 0x55cf5b905580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.381+0000 7fda9d400640 1 -- 172.21.15.32:0/1984371054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 msgr2=0x55cf5b905ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.381+0000 7fda9d400640 1 --2- 172.21.15.32:0/1984371054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 0x55cf5b905ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.381+0000 7fda9d400640 1 -- 172.21.15.32:0/1984371054 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5b906000 con 0x55cf5b51f1b0 2024-08-31T07:28:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.381+0000 7fda9d400640 1 --2- 172.21.15.32:0/1984371054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8ef260 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fda8c012ea0 tx=0x7fda8c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4396d4d7b8c0d8d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.381+0000 7fda89c00640 1 -- 172.21.15.32:0/1984371054 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c005020 con 0x55cf5b51f1b0 2024-08-31T07:28:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.381+0000 7fda89c00640 1 -- 172.21.15.32:0/1984371054 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8c0047a0 con 0x55cf5b51f1b0 2024-08-31T07:28:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.381+0000 7fda89c00640 1 -- 172.21.15.32:0/1984371054 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00f880 con 0x55cf5b51f1b0 2024-08-31T07:28:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.382+0000 7fdaa6a83740 1 -- 172.21.15.32:0/1984371054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 msgr2=0x55cf5b8ef260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.382+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/1984371054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8ef260 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fda8c012ea0 tx=0x7fda8c01a000 comp rx=0 tx=0).stop 2024-08-31T07:28:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.382+0000 7fdaa6a83740 1 -- 172.21.15.32:0/1984371054 shutdown_connections 2024-08-31T07:28:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.382+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/1984371054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b548070 0x55cf5b905ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.382+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/1984371054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b527390 0x55cf5b905580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.382+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/1984371054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8ef260 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.382+0000 7fdaa6a83740 1 -- 172.21.15.32:0/1984371054 >> 172.21.15.32:0/1984371054 conn(0x55cf5b53c460 msgr2=0x55cf5b8eeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.382+0000 7fdaa6a83740 1 -- 172.21.15.32:0/1984371054 shutdown_connections 2024-08-31T07:28:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.382+0000 7fdaa6a83740 1 -- 172.21.15.32:0/1984371054 wait complete. 2024-08-31T07:28:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.383+0000 7fdaa6a83740 1 -- start start 2024-08-31T07:28:48.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.383+0000 7fdaa6a83740 1 Processor -- start 2024-08-31T07:28:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 0x55cf5b8fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fdaa6a83740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 0x55cf5b96f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda981b28c0 con 0x55cf5b51f1b0 2024-08-31T07:28:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda981b21c0 con 0x55cf5b548070 2024-08-31T07:28:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fdaa6a83740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fda981af3f0 con 0x55cf5b527390 2024-08-31T07:28:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fdaa6a83740 10 client.0 _ll_register_callbacks cb 0x7fda8403b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:28:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fda9ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 0x55cf5b8fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fda9d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fda9d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8fdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58370/0 (socket says 172.21.15.32:58370) 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fda9d400640 1 -- 172.21.15.32:0/3674095008 learned_addr learned my addr 172.21.15.32:0/3674095008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.384+0000 7fda9de00640 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 0x55cf5b96f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda9ca00640 1 -- 172.21.15.32:0/3674095008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 msgr2=0x55cf5b96f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda9ca00640 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 0x55cf5b96f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda9ca00640 1 -- 172.21.15.32:0/3674095008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 msgr2=0x55cf5b8fdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda9ca00640 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8fdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda9ca00640 1 -- 172.21.15.32:0/3674095008 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5b96f910 con 0x55cf5b527390 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda9d400640 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda9ca00640 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 0x55cf5b8fe580 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fda84006010 tx=0x7fda840b7420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=756b190ebec9ac06 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda7ea00640 1 -- 172.21.15.32:0/3674095008 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84049070 con 0x55cf5b527390 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda7ea00640 1 -- 172.21.15.32:0/3674095008 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fda840c85c0 con 0x55cf5b527390 2024-08-31T07:28:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fdaa6a83740 1 -- 172.21.15.32:0/3674095008 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda8c0047a0 con 0x55cf5b527390 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.385+0000 7fda7ea00640 1 -- 172.21.15.32:0/3674095008 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fda840cb740 con 0x55cf5b527390 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fdaa6a83740 10 client.28732 fetch_fsmap learned FSMap version 59 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fdaa6a83740 1 -- 172.21.15.32:0/3674095008 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cf5b96f910 con 0x55cf5b527390 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fda7ea00640 1 -- 172.21.15.32:0/3674095008 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8c0047a0 con 0x55cf5b527390 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fda7ea00640 1 -- 172.21.15.32:0/3674095008 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 59) ==== 8390+0+0 (secure 0 0 0) 0x7fda840c7940 con 0x55cf5b527390 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fdaa6a83740 10 client.28732 fetch_fsmap finished waiting for FSMap version 59 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fdaa6a83740 10 client.28732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fdaa6a83740 20 client.28732 populate_metadata read hostname 'smithi032' 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cf5b4dbda0 0x55cf5b979260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fdaa6a83740 4 client.28732 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fdaa6a83740 1 -- 172.21.15.32:0/3674095008 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf5b862160 con 0x55cf5b4dbda0 2024-08-31T07:28:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.386+0000 7fda9d400640 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cf5b4dbda0 0x55cf5b979260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:28:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.387+0000 7fda9d400640 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cf5b4dbda0 0x55cf5b979260 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d61bae060e23c5e server_cookie=8a46d908398c2db1 in_seq=0 out_seq=0 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.387+0000 7fda7ea00640 10 client.28732 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.387+0000 7fda7ea00640 1 -- 172.21.15.32:0/3674095008 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x55cf5b862160 con 0x55cf5b4dbda0 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.387+0000 7fda7ea00640 10 client.28732 handle_command_reply: tid=0 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.387+0000 7fdaa6a83740 1 client.28732 shutdown 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 1 -- 172.21.15.32:0/3674095008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 msgr2=0x55cf5b8fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 0x55cf5b8fe580 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fda84006010 tx=0x7fda840b7420 comp rx=0 tx=0).stop 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 1 -- 172.21.15.32:0/3674095008 shutdown_connections 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cf5b4dbda0 0x55cf5b979260 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf5b548070 0x55cf5b96f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fda7ea00640 0 client.28732 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf5b527390 0x55cf5b8fe580 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 1 --2- 172.21.15.32:0/3674095008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cf5b51f1b0 0x55cf5b8fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 1 -- 172.21.15.32:0/3674095008 >> 172.21.15.32:0/3674095008 conn(0x55cf5b53c460 msgr2=0x55cf5b8f0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 1 -- 172.21.15.32:0/3674095008 shutdown_connections 2024-08-31T07:28:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 1 -- 172.21.15.32:0/3674095008 wait complete. 2024-08-31T07:28:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.388+0000 7fdaa6a83740 20 client.28732 trim_cache size 0 max 16384 2024-08-31T07:28:48.392 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:28:48.392 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:28:48.392 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:28:48.392 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:28:48.392 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "05113edd-ca51-44d5-a01f-804676bde5ad", 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.393 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.394 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.395 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:28:48.396 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.398 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:28:48.400 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.402 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:28:48.403 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:28:48.404 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:28:48.404 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:28:48.404 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:28:48.404 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:28:48.404 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:28:48.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 -- 172.21.15.32:0/1543165080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda800b4ca0 msgr2=0x7fda800b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:48.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda800b4ca0 0x7fda800b71b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55cf5b8ec7a0 tx=0x7fda8c005f90 comp rx=0 tx=0).stop 2024-08-31T07:28:48.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 -- 172.21.15.32:0/1543165080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda98109070 msgr2=0x7fda9819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:28:48.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda98109070 0x7fda9819c900 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fda8402a910 tx=0x7fda84031a20 comp rx=0 tx=0).stop 2024-08-31T07:28:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 -- 172.21.15.32:0/1543165080 shutdown_connections 2024-08-31T07:28:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda800b4ca0 0x7fda800b71b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda981099c0 0x7fda981a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda98109070 0x7fda9819c900 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 --2- 172.21.15.32:0/1543165080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda98103730 0x7fda9819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:28:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 -- 172.21.15.32:0/1543165080 >> 172.21.15.32:0/1543165080 conn(0x7fda980ff180 msgr2=0x7fda98108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:28:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 -- 172.21.15.32:0/1543165080 shutdown_connections 2024-08-31T07:28:48.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:28:48.389+0000 7fda9e800640 1 -- 172.21.15.32:0/1543165080 wait complete. 2024-08-31T07:28:48.414 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:48 smithi032 ceph-mon[17252]: pgmap v869: 97 pgs: 97 active+clean; 884 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-08-31T07:28:48.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:48 smithi073 ceph-mon[21995]: pgmap v869: 97 pgs: 97 active+clean; 884 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-08-31T07:28:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:48 smithi084 ceph-mon[22387]: pgmap v869: 97 pgs: 97 active+clean; 884 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-08-31T07:28:49.344 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 5.11 MB/sec execute 29 sec latency 5061.570 ms 2024-08-31T07:28:49.367 INFO:tasks.cephfs.filesystem:scrub status for tag:05113edd-ca51-44d5-a01f-804676bde5ad - {'path': '/', 'tag': '05113edd-ca51-44d5-a01f-804676bde5ad', 'options': 'recursive,force'} 2024-08-31T07:28:50.345 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 4.94 MB/sec execute 30 sec latency 6061.645 ms 2024-08-31T07:28:50.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:50 smithi032 ceph-mon[17252]: pgmap v870: 97 pgs: 97 active+clean; 887 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-08-31T07:28:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:50 smithi073 ceph-mon[21995]: pgmap v870: 97 pgs: 97 active+clean; 887 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-08-31T07:28:50.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:50 smithi084 ceph-mon[22387]: pgmap v870: 97 pgs: 97 active+clean; 887 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-08-31T07:28:51.345 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 4.78 MB/sec execute 31 sec latency 7061.747 ms 2024-08-31T07:28:52.345 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 4.63 MB/sec execute 32 sec latency 8061.801 ms 2024-08-31T07:28:52.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:52 smithi032 ceph-mon[17252]: pgmap v871: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 214 op/s 2024-08-31T07:28:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:52 smithi073 ceph-mon[21995]: pgmap v871: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 214 op/s 2024-08-31T07:28:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:52 smithi084 ceph-mon[22387]: pgmap v871: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 214 op/s 2024-08-31T07:28:53.345 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 4.49 MB/sec execute 33 sec latency 9061.872 ms 2024-08-31T07:28:54.345 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 4.36 MB/sec execute 34 sec latency 10061.927 ms 2024-08-31T07:28:54.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:54 smithi032 ceph-mon[17252]: pgmap v872: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-08-31T07:28:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:54 smithi073 ceph-mon[21995]: pgmap v872: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-08-31T07:28:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:54 smithi084 ceph-mon[22387]: pgmap v872: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-08-31T07:28:55.345 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 4.23 MB/sec execute 35 sec latency 11062.046 ms 2024-08-31T07:28:55.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:55 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:55.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:55 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:55.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:55 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:56.345 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 4.11 MB/sec execute 36 sec latency 12062.208 ms 2024-08-31T07:28:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:56 smithi073 ceph-mon[21995]: pgmap v873: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-08-31T07:28:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:56 smithi084 ceph-mon[22387]: pgmap v873: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-08-31T07:28:56.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:56 smithi032 ceph-mon[17252]: pgmap v873: 97 pgs: 97 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-08-31T07:28:57.346 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 4.00 MB/sec execute 37 sec latency 13062.372 ms 2024-08-31T07:28:58.346 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 3.90 MB/sec execute 38 sec latency 14062.534 ms 2024-08-31T07:28:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:58 smithi073 ceph-mon[21995]: pgmap v874: 97 pgs: 97 active+clean; 891 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 151 op/s 2024-08-31T07:28:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:58 smithi084 ceph-mon[22387]: pgmap v874: 97 pgs: 97 active+clean; 891 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 151 op/s 2024-08-31T07:28:58.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:28:58.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:58 smithi032 ceph-mon[17252]: pgmap v874: 97 pgs: 97 active+clean; 891 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 151 op/s 2024-08-31T07:28:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:28:59.345 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 3.80 MB/sec execute 39 sec latency 15062.619 ms 2024-08-31T07:28:59.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:28:59 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:59.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:28:59 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:28:59.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:28:59 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:00.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.005+0000 7f2a61e00640 1 -- 172.21.15.32:0/3474543500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c077120 msgr2=0x7f2a5c075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.005+0000 7f2a61e00640 1 --2- 172.21.15.32:0/3474543500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c077120 0x7f2a5c075550 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009900 tx=0x7f2a5002a410 comp rx=0 tx=0).stop 2024-08-31T07:29:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.005+0000 7f2a61e00640 1 -- 172.21.15.32:0/3474543500 shutdown_connections 2024-08-31T07:29:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.005+0000 7f2a61e00640 1 --2- 172.21.15.32:0/3474543500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a5c114020 0x7f2a5c1164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.005+0000 7f2a61e00640 1 --2- 172.21.15.32:0/3474543500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a5c075b70 0x7f2a5c075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.005+0000 7f2a61e00640 1 --2- 172.21.15.32:0/3474543500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c077120 0x7f2a5c075550 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.005+0000 7f2a61e00640 1 -- 172.21.15.32:0/3474543500 >> 172.21.15.32:0/3474543500 conn(0x7f2a5c0ff1e0 msgr2=0x7f2a5c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.005+0000 7f2a61e00640 1 -- 172.21.15.32:0/3474543500 shutdown_connections 2024-08-31T07:29:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.005+0000 7f2a61e00640 1 -- 172.21.15.32:0/3474543500 wait complete. 2024-08-31T07:29:00.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.006+0000 7f2a61e00640 1 Processor -- start 2024-08-31T07:29:00.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.006+0000 7f2a61e00640 1 -- start start 2024-08-31T07:29:00.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.007+0000 7f2a61e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a5c075b70 0x7f2a5c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:00.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.007+0000 7f2a61e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c077120 0x7f2a5c1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.007+0000 7f2a61e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a5c114020 0x7f2a5c1a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.007+0000 7f2a61e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c119c60 con 0x7f2a5c077120 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.007+0000 7f2a61e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c119ae0 con 0x7f2a5c114020 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.007+0000 7f2a61e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c119de0 con 0x7f2a5c075b70 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.007+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c077120 0x7f2a5c1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.007+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c077120 0x7f2a5c1a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59450/0 (socket says 172.21.15.32:59450) 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.007+0000 7f2a5be00640 1 -- 172.21.15.32:0/1939112835 learned_addr learned my addr 172.21.15.32:0/1939112835 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.007+0000 7f2a60a00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a5c075b70 0x7f2a5c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a61400640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a5c114020 0x7f2a5c1a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a5be00640 1 -- 172.21.15.32:0/1939112835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a5c075b70 msgr2=0x7f2a5c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a5be00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a5c075b70 0x7f2a5c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a5be00640 1 -- 172.21.15.32:0/1939112835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a5c114020 msgr2=0x7f2a5c1a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:00.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a5be00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a5c114020 0x7f2a5c1a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:00.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a5be00640 1 -- 172.21.15.32:0/1939112835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c1a65e0 con 0x7f2a5c077120 2024-08-31T07:29:00.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a61400640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a5c114020 0x7f2a5c1a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:29:00.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a5be00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c077120 0x7f2a5c1a0c10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800d770 tx=0x7f2a4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:00.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a59600640 1 -- 172.21.15.32:0/1939112835 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48017070 con 0x7f2a5c077120 2024-08-31T07:29:00.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a61e00640 1 -- 172.21.15.32:0/1939112835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c1a68d0 con 0x7f2a5c077120 2024-08-31T07:29:00.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.008+0000 7f2a59600640 1 -- 172.21.15.32:0/1939112835 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a48009c30 con 0x7f2a5c077120 2024-08-31T07:29:00.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.009+0000 7f2a59600640 1 -- 172.21.15.32:0/1939112835 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4800f620 con 0x7f2a5c077120 2024-08-31T07:29:00.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.009+0000 7f2a61e00640 1 -- 172.21.15.32:0/1939112835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c1ad030 con 0x7f2a5c077120 2024-08-31T07:29:00.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.010+0000 7f2a61e00640 1 -- 172.21.15.32:0/1939112835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c075f80 con 0x7f2a5c077120 2024-08-31T07:29:00.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.013+0000 7f2a59600640 1 -- 172.21.15.32:0/1939112835 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2a4801a070 con 0x7f2a5c077120 2024-08-31T07:29:00.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.014+0000 7f2a59600640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a440b4ca0 0x7f2a440b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:00.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.015+0000 7f2a59600640 1 -- 172.21.15.32:0/1939112835 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7f2a480e4ff0 con 0x7f2a5c077120 2024-08-31T07:29:00.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.015+0000 7f2a60a00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a440b4ca0 0x7f2a440b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:00.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.016+0000 7f2a60a00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a440b4ca0 0x7f2a440b71b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2a50031440 tx=0x7f2a50009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.016+0000 7f2a59600640 1 -- 172.21.15.32:0/1939112835 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2a48009de0 con 0x7f2a5c077120 2024-08-31T07:29:00.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.256+0000 7f2a61e00640 1 -- 172.21.15.32:0/1939112835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a5c112d80 con 0x7f2a5c077120 2024-08-31T07:29:00.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.257+0000 7f2a59600640 1 -- 172.21.15.32:0/1939112835 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11968 (secure 0 0 0) 0x7f2a480abdf0 con 0x7f2a5c077120 2024-08-31T07:29:00.258 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:29:00.258 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":61,"btime":"2024-08-31T07:28:58:360883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:28:57.601720+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:29:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 61 2024-08-31T07:29:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 -- 172.21.15.32:0/1939112835 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a440b4ca0 msgr2=0x7f2a440b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a440b4ca0 0x7f2a440b71b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2a50031440 tx=0x7f2a50009560 comp rx=0 tx=0).stop 2024-08-31T07:29:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 -- 172.21.15.32:0/1939112835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c077120 msgr2=0x7f2a5c1a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c077120 0x7f2a5c1a0c10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800d770 tx=0x7f2a4800dc30 comp rx=0 tx=0).stop 2024-08-31T07:29:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 -- 172.21.15.32:0/1939112835 shutdown_connections 2024-08-31T07:29:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2a440b4ca0 0x7f2a440b71b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a5c114020 0x7f2a5c1a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5c077120 0x7f2a5c1a0c10 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 --2- 172.21.15.32:0/1939112835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a5c075b70 0x7f2a5c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 -- 172.21.15.32:0/1939112835 >> 172.21.15.32:0/1939112835 conn(0x7f2a5c0ff1e0 msgr2=0x7f2a5c10ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 -- 172.21.15.32:0/1939112835 shutdown_connections 2024-08-31T07:29:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:00.260+0000 7f2a61e00640 1 -- 172.21.15.32:0/1939112835 wait complete. 2024-08-31T07:29:00.346 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 3.70 MB/sec execute 40 sec latency 16062.755 ms 2024-08-31T07:29:00.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:00 smithi032 ceph-mon[17252]: pgmap v875: 97 pgs: 97 active+clean; 893 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 827 KiB/s wr, 58 op/s 2024-08-31T07:29:00.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:00 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1939112835' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:00 smithi073 ceph-mon[21995]: pgmap v875: 97 pgs: 97 active+clean; 893 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 827 KiB/s wr, 58 op/s 2024-08-31T07:29:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:00 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1939112835' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:00.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:00 smithi084 ceph-mon[22387]: pgmap v875: 97 pgs: 97 active+clean; 893 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 827 KiB/s wr, 58 op/s 2024-08-31T07:29:00.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:00 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1939112835' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:01.193 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id a2a13203 --release --await' 2024-08-31T07:29:01.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id a2a13203 --release --await 2024-08-31T07:29:01.346 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52829 3.61 MB/sec execute 41 sec latency 17062.840 ms 2024-08-31T07:29:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:02.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.253+0000 7fd155a00640 1 -- 172.21.15.32:0/1997816966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd150109070 msgr2=0x7fd150109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:02.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.253+0000 7fd155a00640 1 --2- 172.21.15.32:0/1997816966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd150109070 0x7fd150109480 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd138009900 tx=0x7fd13802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.253+0000 7fd155a00640 1 -- 172.21.15.32:0/1997816966 shutdown_connections 2024-08-31T07:29:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.253+0000 7fd155a00640 1 --2- 172.21.15.32:0/1997816966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1501099c0 0x7fd1501101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.253+0000 7fd155a00640 1 --2- 172.21.15.32:0/1997816966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd150109070 0x7fd150109480 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.253+0000 7fd155a00640 1 --2- 172.21.15.32:0/1997816966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd150103730 0x7fd150103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.253+0000 7fd155a00640 1 -- 172.21.15.32:0/1997816966 >> 172.21.15.32:0/1997816966 conn(0x7fd1500ff180 msgr2=0x7fd1501015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.253+0000 7fd155a00640 1 -- 172.21.15.32:0/1997816966 shutdown_connections 2024-08-31T07:29:02.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.253+0000 7fd155a00640 1 -- 172.21.15.32:0/1997816966 wait complete. 2024-08-31T07:29:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.254+0000 7fd155a00640 1 Processor -- start 2024-08-31T07:29:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.254+0000 7fd155a00640 1 -- start start 2024-08-31T07:29:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd155a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd150103730 0x7fd15019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd155a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd150109070 0x7fd15019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd155a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1501099c0 0x7fd1501a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd155a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd150113630 con 0x7fd150103730 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd155a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501134b0 con 0x7fd1501099c0 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd155a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501137b0 con 0x7fd150109070 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd14fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd150103730 0x7fd15019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd14fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd150103730 0x7fd15019c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59468/0 (socket says 172.21.15.32:59468) 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd147400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd150109070 0x7fd15019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd14fe00640 1 -- 172.21.15.32:0/1508949223 learned_addr learned my addr 172.21.15.32:0/1508949223 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.255+0000 7fd155000640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1501099c0 0x7fd1501a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.256+0000 7fd14fe00640 1 -- 172.21.15.32:0/1508949223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd150109070 msgr2=0x7fd15019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.256+0000 7fd14fe00640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd150109070 0x7fd15019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.256+0000 7fd14fe00640 1 -- 172.21.15.32:0/1508949223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1501099c0 msgr2=0x7fd1501a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:02.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.256+0000 7fd14fe00640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1501099c0 0x7fd1501a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.256+0000 7fd14fe00640 1 -- 172.21.15.32:0/1508949223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1501a2230 con 0x7fd150103730 2024-08-31T07:29:02.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.256+0000 7fd147400640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd150109070 0x7fd15019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:02.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.256+0000 7fd14fe00640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd150103730 0x7fd15019c2c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd14000d950 tx=0x7fd14000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:02.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.256+0000 7fd14d600640 1 -- 172.21.15.32:0/1508949223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140017030 con 0x7fd150103730 2024-08-31T07:29:02.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.257+0000 7fd14d600640 1 -- 172.21.15.32:0/1508949223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd140004020 con 0x7fd150103730 2024-08-31T07:29:02.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.257+0000 7fd155a00640 1 -- 172.21.15.32:0/1508949223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1501a2520 con 0x7fd150103730 2024-08-31T07:29:02.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.257+0000 7fd14d600640 1 -- 172.21.15.32:0/1508949223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1400047a0 con 0x7fd150103730 2024-08-31T07:29:02.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.257+0000 7fd155a00640 1 -- 172.21.15.32:0/1508949223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1501a8c80 con 0x7fd150103730 2024-08-31T07:29:02.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.260+0000 7fd15dc40740 1 Processor -- start 2024-08-31T07:29:02.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.260+0000 7fd15dc40740 1 -- start start 2024-08-31T07:29:02.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.260+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 0x56276b17d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.260+0000 7fd14d600640 1 -- 172.21.15.32:0/1508949223 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd140009a30 con 0x7fd150103730 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.260+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 0x56276b17d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.260+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b17db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.260+0000 7fd15dc40740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56276b0f3a90 con 0x56276adb6840 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.260+0000 7fd15dc40740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56276b0e3020 con 0x56276adae660 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.260+0000 7fd15dc40740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56276b0e6fd0 con 0x56276add7520 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd14fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 0x56276b17d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd155000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 0x56276b17d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd14fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 0x56276b17d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48864/0 (socket says 172.21.15.32:48864) 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd14fe00640 1 -- 172.21.15.32:0/3643206533 learned_addr learned my addr 172.21.15.32:0/3643206533 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:02.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd155000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 0x56276b17d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:56606/0 (socket says 172.21.15.32:56606) 2024-08-31T07:29:02.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd147400640 1 --2- 172.21.15.32:0/3643206533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b17db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd14fe00640 1 -- 172.21.15.32:0/3643206533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 msgr2=0x56276b17d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd14fe00640 1 --2- 172.21.15.32:0/3643206533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 0x56276b17d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd14fe00640 1 -- 172.21.15.32:0/3643206533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 msgr2=0x56276b17db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd14fe00640 1 --2- 172.21.15.32:0/3643206533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b17db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd14fe00640 1 -- 172.21.15.32:0/3643206533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56276ae42b00 con 0x56276adae660 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.261+0000 7fd155000640 1 --2- 172.21.15.32:0/3643206533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 0x56276b17d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.262+0000 7fd14fe00640 1 --2- 172.21.15.32:0/3643206533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 0x56276b17d5d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd1400e25a0 tx=0x7fd1400ed4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a25a20fe0cf23725 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.262+0000 7fd145600640 1 -- 172.21.15.32:0/3643206533 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1400f8070 con 0x56276adae660 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.262+0000 7fd145600640 1 -- 172.21.15.32:0/3643206533 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1400f3ab0 con 0x56276adae660 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.262+0000 7fd145600640 1 -- 172.21.15.32:0/3643206533 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1400f28a0 con 0x56276adae660 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.262+0000 7fd14d600640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1340b4d20 0x7fd1340b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd14d600640 1 -- 172.21.15.32:0/1508949223 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fd1400e4f20 con 0x7fd150103730 2024-08-31T07:29:02.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd155000640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1340b4d20 0x7fd1340b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd15dc40740 1 -- 172.21.15.32:0/3643206533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 msgr2=0x56276b17d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd15dc40740 1 --2- 172.21.15.32:0/3643206533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 0x56276b17d5d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd1400e25a0 tx=0x7fd1400ed4b0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd15dc40740 1 -- 172.21.15.32:0/3643206533 shutdown_connections 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd15dc40740 1 --2- 172.21.15.32:0/3643206533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b17db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd15dc40740 1 --2- 172.21.15.32:0/3643206533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 0x56276b17d5d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd15dc40740 1 --2- 172.21.15.32:0/3643206533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 0x56276b17d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd15dc40740 1 -- 172.21.15.32:0/3643206533 >> 172.21.15.32:0/3643206533 conn(0x56276adcb910 msgr2=0x56276b17af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd155000640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1340b4d20 0x7fd1340b7230 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd148005fd0 tx=0x7fd148009600 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd15dc40740 1 -- 172.21.15.32:0/3643206533 shutdown_connections 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.263+0000 7fd15dc40740 1 -- 172.21.15.32:0/3643206533 wait complete. 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.264+0000 7fd15dc40740 1 -- start start 2024-08-31T07:29:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.264+0000 7fd15dc40740 1 Processor -- start 2024-08-31T07:29:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adae660 0x56276b188440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adb6840 0x56276b188980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 0x56276b188ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd15dc40740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56276af45bb0 con 0x56276adae660 2024-08-31T07:29:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd15dc40740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56276b069f90 con 0x56276add7520 2024-08-31T07:29:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd15dc40740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56276b18b870 con 0x56276adb6840 2024-08-31T07:29:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd15dc40740 10 client.0 _ll_register_callbacks cb 0x56276b08f710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd14fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adae660 0x56276b188440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd14fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adae660 0x56276b188440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59490/0 (socket says 172.21.15.32:59490) 2024-08-31T07:29:02.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd14fe00640 1 -- 172.21.15.32:0/1935479133 learned_addr learned my addr 172.21.15.32:0/1935479133 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:02.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd147400640 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adb6840 0x56276b188980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.265+0000 7fd155000640 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 0x56276b188ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd14fe00640 1 -- 172.21.15.32:0/1935479133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adb6840 msgr2=0x56276b188980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd14fe00640 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adb6840 0x56276b188980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd14fe00640 1 -- 172.21.15.32:0/1935479133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 msgr2=0x56276b188ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd14fe00640 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 0x56276b188ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd14fe00640 1 -- 172.21.15.32:0/1935479133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56276aeb9960 con 0x56276adae660 2024-08-31T07:29:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd147400640 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adb6840 0x56276b188980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd14fe00640 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adae660 0x56276b188440 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd1400e51c0 tx=0x7fd140009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781de67e0598dde0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd13e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:29:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd13e000640 1 -- 172.21.15.32:0/1935479133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1400a74c0 con 0x56276adae660 2024-08-31T07:29:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd13e000640 1 -- 172.21.15.32:0/1935479133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fd1400a7890 con 0x56276adae660 2024-08-31T07:29:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd15dc40740 1 -- 172.21.15.32:0/1935479133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56276b081e30 con 0x56276adae660 2024-08-31T07:29:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.266+0000 7fd13e000640 1 -- 172.21.15.32:0/1935479133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd1400aad70 con 0x56276adae660 2024-08-31T07:29:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.267+0000 7fd15dc40740 10 client.18720 fetch_fsmap learned FSMap version 62 2024-08-31T07:29:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.267+0000 7fd15dc40740 1 -- 172.21.15.32:0/1935479133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x56276b1972a0 con 0x56276adae660 2024-08-31T07:29:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.267+0000 7fd13e000640 1 -- 172.21.15.32:0/1935479133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56276b081e30 con 0x56276adae660 2024-08-31T07:29:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.267+0000 7fd13e000640 1 -- 172.21.15.32:0/1935479133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 62) ==== 8374+0+0 (secure 0 0 0) 0x7fd1400e28e0 con 0x56276adae660 2024-08-31T07:29:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.267+0000 7fd15dc40740 10 client.18720 fetch_fsmap finished waiting for FSMap version 62 2024-08-31T07:29:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.267+0000 7fd15dc40740 10 client.18720 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:29:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.267+0000 7fd15dc40740 20 client.18720 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.267+0000 7fd15dc40740 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56276ad95ea0 0x56276b19a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.267+0000 7fd15dc40740 4 client.18720 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:02.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.267+0000 7fd15dc40740 1 -- 172.21.15.32:0/1935479133 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56276b0f59e0 con 0x56276ad95ea0 2024-08-31T07:29:02.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.268+0000 7fd147400640 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56276ad95ea0 0x56276b19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.268+0000 7fd147400640 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56276ad95ea0 0x56276b19a600 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c4021673ba3661d server_cookie=400049002571623f in_seq=0 out_seq=0 2024-08-31T07:29:02.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.268+0000 7fd13e000640 10 client.18720 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd13e000640 1 -- 172.21.15.32:0/1935479133 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56276b0f59e0 con 0x56276ad95ea0 2024-08-31T07:29:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd13e000640 10 client.18720 handle_command_reply: tid=0 2024-08-31T07:29:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd15dc40740 1 client.18720 shutdown 2024-08-31T07:29:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd15dc40740 1 -- 172.21.15.32:0/1935479133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adae660 msgr2=0x56276b188440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd15dc40740 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adae660 0x56276b188440 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd1400e51c0 tx=0x7fd140009f50 comp rx=0 tx=0).stop 2024-08-31T07:29:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd15dc40740 1 -- 172.21.15.32:0/1935479133 shutdown_connections 2024-08-31T07:29:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd15dc40740 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56276ad95ea0 0x56276b19a600 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd15dc40740 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 0x56276b188ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd15dc40740 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adb6840 0x56276b188980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd15dc40740 1 --2- 172.21.15.32:0/1935479133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adae660 0x56276b188440 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd15dc40740 1 -- 172.21.15.32:0/1935479133 >> 172.21.15.32:0/1935479133 conn(0x56276adcb910 msgr2=0x56276b180220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.269+0000 7fd13e000640 0 client.18720 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.270+0000 7fd15dc40740 1 -- 172.21.15.32:0/1935479133 shutdown_connections 2024-08-31T07:29:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.270+0000 7fd15dc40740 1 -- 172.21.15.32:0/1935479133 wait complete. 2024-08-31T07:29:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.270+0000 7fd15dc40740 20 client.18720 trim_cache size 0 max 16384 2024-08-31T07:29:02.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.288+0000 7fd15dc40740 1 Processor -- start 2024-08-31T07:29:02.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.288+0000 7fd15dc40740 1 -- start start 2024-08-31T07:29:02.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.288+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adae660 0x56276b194b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.288+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b188d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.288+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 0x56276b17e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.288+0000 7fd15dc40740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501b1460 con 0x56276adb6840 2024-08-31T07:29:02.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.288+0000 7fd15dc40740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501b3030 con 0x56276add7520 2024-08-31T07:29:02.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.288+0000 7fd15dc40740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501b1b30 con 0x56276adae660 2024-08-31T07:29:02.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd147400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b188d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd155000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 0x56276b17e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd147400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b188d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59504/0 (socket says 172.21.15.32:59504) 2024-08-31T07:29:02.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd147400640 1 -- 172.21.15.32:0/1198615282 learned_addr learned my addr 172.21.15.32:0/1198615282 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:02.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd14fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adae660 0x56276b194b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd155000640 1 -- 172.21.15.32:0/1198615282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adae660 msgr2=0x56276b194b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:02.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd155000640 1 --2- 172.21.15.32:0/1198615282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adae660 0x56276b194b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd155000640 1 -- 172.21.15.32:0/1198615282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 msgr2=0x56276b188d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:02.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd155000640 1 --2- 172.21.15.32:0/1198615282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b188d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd155000640 1 -- 172.21.15.32:0/1198615282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56276b1894e0 con 0x56276add7520 2024-08-31T07:29:02.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.289+0000 7fd147400640 1 --2- 172.21.15.32:0/1198615282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b188d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:02.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.290+0000 7fd155000640 1 --2- 172.21.15.32:0/1198615282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 0x56276b17e7d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd1480117e0 tx=0x7fd148016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2006b41ab01a434f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:02.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.290+0000 7fd144c00640 1 -- 172.21.15.32:0/1198615282 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148011e90 con 0x56276add7520 2024-08-31T07:29:02.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.290+0000 7fd144c00640 1 -- 172.21.15.32:0/1198615282 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd148003e20 con 0x56276add7520 2024-08-31T07:29:02.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.290+0000 7fd144c00640 1 -- 172.21.15.32:0/1198615282 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148028670 con 0x56276add7520 2024-08-31T07:29:02.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.291+0000 7fd15dc40740 1 -- 172.21.15.32:0/1198615282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 msgr2=0x56276b17e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:02.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.291+0000 7fd15dc40740 1 --2- 172.21.15.32:0/1198615282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 0x56276b17e7d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd1480117e0 tx=0x7fd148016000 comp rx=0 tx=0).stop 2024-08-31T07:29:02.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.291+0000 7fd15dc40740 1 -- 172.21.15.32:0/1198615282 shutdown_connections 2024-08-31T07:29:02.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.291+0000 7fd15dc40740 1 --2- 172.21.15.32:0/1198615282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276add7520 0x56276b17e7d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.291+0000 7fd15dc40740 1 --2- 172.21.15.32:0/1198615282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b188d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.291+0000 7fd15dc40740 1 --2- 172.21.15.32:0/1198615282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276adae660 0x56276b194b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.291+0000 7fd15dc40740 1 -- 172.21.15.32:0/1198615282 >> 172.21.15.32:0/1198615282 conn(0x56276adcb910 msgr2=0x56276b17e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:02.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.291+0000 7fd15dc40740 1 -- 172.21.15.32:0/1198615282 shutdown_connections 2024-08-31T07:29:02.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.291+0000 7fd15dc40740 1 -- 172.21.15.32:0/1198615282 wait complete. 2024-08-31T07:29:02.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.292+0000 7fd15dc40740 1 -- start start 2024-08-31T07:29:02.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.292+0000 7fd15dc40740 1 Processor -- start 2024-08-31T07:29:02.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.292+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 0x56276b19b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.292+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.292+0000 7fd15dc40740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 0x56276b19c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.292+0000 7fd15dc40740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501b2900 con 0x56276adb6840 2024-08-31T07:29:02.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.292+0000 7fd15dc40740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501b2230 con 0x56276adae660 2024-08-31T07:29:02.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.292+0000 7fd15dc40740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1501af460 con 0x56276add7520 2024-08-31T07:29:02.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.292+0000 7fd15dc40740 10 client.0 _ll_register_callbacks cb 0x7fd148009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:02.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd147400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd147400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b19bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59514/0 (socket says 172.21.15.32:59514) 2024-08-31T07:29:02.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd147400640 1 -- 172.21.15.32:0/956948105 learned_addr learned my addr 172.21.15.32:0/956948105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:02.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd14fe00640 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 0x56276b19b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd155000640 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 0x56276b19c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd147400640 1 -- 172.21.15.32:0/956948105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 msgr2=0x56276b19c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd147400640 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 0x56276b19c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd147400640 1 -- 172.21.15.32:0/956948105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 msgr2=0x56276b19b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd147400640 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 0x56276b19b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd147400640 1 -- 172.21.15.32:0/956948105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56276b203d80 con 0x56276adb6840 2024-08-31T07:29:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd147400640 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b19bb30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd138009900 tx=0x7fd138004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ad42decf4e28be server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.293+0000 7fd13e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:29:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.294+0000 7fd13e000640 1 -- 172.21.15.32:0/956948105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138035040 con 0x56276adb6840 2024-08-31T07:29:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.294+0000 7fd13e000640 1 -- 172.21.15.32:0/956948105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7fd138031a80 con 0x56276adb6840 2024-08-31T07:29:02.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.294+0000 7fd15dc40740 1 -- 172.21.15.32:0/956948105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd148003e20 con 0x56276adb6840 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.294+0000 7fd13e000640 1 -- 172.21.15.32:0/956948105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fd138052020 con 0x56276adb6840 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.294+0000 7fd15dc40740 10 client.18726 fetch_fsmap learned FSMap version 62 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.294+0000 7fd15dc40740 1 -- 172.21.15.32:0/956948105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x56276b205010 con 0x56276adb6840 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.294+0000 7fd13e000640 1 -- 172.21.15.32:0/956948105 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd148003e20 con 0x56276adb6840 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.295+0000 7fd13e000640 1 -- 172.21.15.32:0/956948105 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 62) ==== 8374+0+0 (secure 0 0 0) 0x7fd138004800 con 0x56276adb6840 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.295+0000 7fd15dc40740 10 client.18726 fetch_fsmap finished waiting for FSMap version 62 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.295+0000 7fd15dc40740 10 client.18726 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.295+0000 7fd15dc40740 20 client.18726 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.295+0000 7fd15dc40740 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56276ad6b250 0x56276b207810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.295+0000 7fd15dc40740 4 client.18726 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a2a13203", "release": true, "await": true}] 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.295+0000 7fd15dc40740 1 -- 172.21.15.32:0/956948105 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a2a13203", "release": true, "await": true}) -- 0x56276b0f1480 con 0x56276ad6b250 2024-08-31T07:29:02.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.295+0000 7fd14fe00640 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56276ad6b250 0x56276b207810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:02.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.295+0000 7fd14fe00640 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56276ad6b250 0x56276b207810 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b6b71686853e2e server_cookie=121548489e2452a8 in_seq=0 out_seq=0 2024-08-31T07:29:02.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.295+0000 7fd13e000640 10 client.18726 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:02.346 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52830 3.53 MB/sec execute 42 sec latency 18062.888 ms 2024-08-31T07:29:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.391+0000 7fd13e000640 1 -- 172.21.15.32:0/956948105 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56276b0f1480 con 0x56276ad6b250 2024-08-31T07:29:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.391+0000 7fd13e000640 10 client.18726 handle_command_reply: tid=0 2024-08-31T07:29:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.391+0000 7fd15dc40740 1 client.18726 shutdown 2024-08-31T07:29:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.391+0000 7fd15dc40740 1 -- 172.21.15.32:0/956948105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 msgr2=0x56276b19bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.391+0000 7fd15dc40740 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b19bb30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd138009900 tx=0x7fd138004130 comp rx=0 tx=0).stop 2024-08-31T07:29:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.392+0000 7fd15dc40740 1 -- 172.21.15.32:0/956948105 shutdown_connections 2024-08-31T07:29:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.392+0000 7fd15dc40740 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56276ad6b250 0x56276b207810 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.392+0000 7fd15dc40740 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56276add7520 0x56276b19c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.392+0000 7fd15dc40740 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56276adb6840 0x56276b19bb30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.392+0000 7fd15dc40740 1 --2- 172.21.15.32:0/956948105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56276adae660 0x56276b19b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.392+0000 7fd13e000640 0 client.18726 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.392+0000 7fd15dc40740 1 -- 172.21.15.32:0/956948105 >> 172.21.15.32:0/956948105 conn(0x56276adcb910 msgr2=0x56276b188a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.392+0000 7fd15dc40740 1 -- 172.21.15.32:0/956948105 shutdown_connections 2024-08-31T07:29:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.392+0000 7fd15dc40740 1 -- 172.21.15.32:0/956948105 wait complete. 2024-08-31T07:29:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.392+0000 7fd15dc40740 20 client.18726 trim_cache size 0 max 16384 2024-08-31T07:29:02.394 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 62, 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 52, 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "a2a13203": { 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 52, 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 141.2, 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:02.396 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:29:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.393+0000 7fd155a00640 1 -- 172.21.15.32:0/1508949223 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1340b4d20 msgr2=0x7fd1340b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.393+0000 7fd155a00640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1340b4d20 0x7fd1340b7230 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd148005fd0 tx=0x7fd148009600 comp rx=0 tx=0).stop 2024-08-31T07:29:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.393+0000 7fd155a00640 1 -- 172.21.15.32:0/1508949223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd150103730 msgr2=0x7fd15019c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.393+0000 7fd155a00640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd150103730 0x7fd15019c2c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd14000d950 tx=0x7fd14000de10 comp rx=0 tx=0).stop 2024-08-31T07:29:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.394+0000 7fd155a00640 1 -- 172.21.15.32:0/1508949223 shutdown_connections 2024-08-31T07:29:02.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.394+0000 7fd155a00640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd1340b4d20 0x7fd1340b7230 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.394+0000 7fd155a00640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1501099c0 0x7fd1501a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.394+0000 7fd155a00640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd150109070 0x7fd15019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.394+0000 7fd155a00640 1 --2- 172.21.15.32:0/1508949223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd150103730 0x7fd15019c2c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:02.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.394+0000 7fd155a00640 1 -- 172.21.15.32:0/1508949223 >> 172.21.15.32:0/1508949223 conn(0x7fd1500ff180 msgr2=0x7fd1501087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:02.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.394+0000 7fd155a00640 1 -- 172.21.15.32:0/1508949223 shutdown_connections 2024-08-31T07:29:02.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:02.394+0000 7fd155a00640 1 -- 172.21.15.32:0/1508949223 wait complete. 2024-08-31T07:29:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:02 smithi073 ceph-mon[21995]: pgmap v876: 97 pgs: 97 active+clean; 896 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 804 KiB/s wr, 48 op/s 2024-08-31T07:29:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:02 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:02 smithi084 ceph-mon[22387]: pgmap v876: 97 pgs: 97 active+clean; 896 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 804 KiB/s wr, 48 op/s 2024-08-31T07:29:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:02 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:02 smithi032 ceph-mon[17252]: pgmap v876: 97 pgs: 97 active+clean; 896 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 804 KiB/s wr, 48 op/s 2024-08-31T07:29:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:02 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:03.346 INFO:tasks.workunit.client.0.smithi032.stdout: 10 53639 4.33 MB/sec execute 43 sec latency 18298.525 ms 2024-08-31T07:29:03.400 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a2a13203', total seconds elapsed: 25.7 2024-08-31T07:29:03.400 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.25 2024-08-31T07:29:03.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:29:04.013 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:04.346 INFO:tasks.workunit.client.0.smithi032.stdout: 10 54833 5.85 MB/sec execute 44 sec latency 53.497 ms 2024-08-31T07:29:04.447 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:04 smithi032 ceph-mon[17252]: pgmap v877: 97 pgs: 97 active+clean; 897 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 844 KiB/s wr, 42 op/s 2024-08-31T07:29:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:04 smithi073 ceph-mon[21995]: pgmap v877: 97 pgs: 97 active+clean; 897 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 844 KiB/s wr, 42 op/s 2024-08-31T07:29:04.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:04 smithi084 ceph-mon[22387]: pgmap v877: 97 pgs: 97 active+clean; 897 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 844 KiB/s wr, 42 op/s 2024-08-31T07:29:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.150+0000 7f04e3e00640 1 -- 172.21.15.32:0/223810299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04e410ad30 msgr2=0x7f04e4115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.150+0000 7f04e3e00640 1 --2- 172.21.15.32:0/223810299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04e410ad30 0x7f04e4115d70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009900 tx=0x7f04d002a400 comp rx=0 tx=0).stop 2024-08-31T07:29:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.151+0000 7f04e3e00640 1 -- 172.21.15.32:0/223810299 shutdown_connections 2024-08-31T07:29:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.151+0000 7f04e3e00640 1 --2- 172.21.15.32:0/223810299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e41162b0 0x7f04e4118730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.151+0000 7f04e3e00640 1 --2- 172.21.15.32:0/223810299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04e410ad30 0x7f04e4115d70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.151+0000 7f04e3e00640 1 --2- 172.21.15.32:0/223810299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e410a3e0 0x7f04e410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.151+0000 7f04e3e00640 1 -- 172.21.15.32:0/223810299 >> 172.21.15.32:0/223810299 conn(0x7f04e406e790 msgr2=0x7f04e406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.151+0000 7f04e3e00640 1 -- 172.21.15.32:0/223810299 shutdown_connections 2024-08-31T07:29:05.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.151+0000 7f04e3e00640 1 -- 172.21.15.32:0/223810299 wait complete. 2024-08-31T07:29:05.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.153+0000 7f04e3e00640 1 Processor -- start 2024-08-31T07:29:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.154+0000 7f04e3e00640 1 -- start start 2024-08-31T07:29:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.154+0000 7f04e3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e410a3e0 0x7f04e41a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.154+0000 7f04e3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e410ad30 0x7f04e41aa070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.154+0000 7f04e3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04e41162b0 0x7f04e41b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.154+0000 7f04e3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f04e411be50 con 0x7f04e410ad30 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.154+0000 7f04e3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f04e411bcd0 con 0x7f04e41162b0 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.154+0000 7f04e3e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f04e411bfd0 con 0x7f04e410a3e0 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.155+0000 7f04e3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04e41162b0 0x7f04e41b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.155+0000 7f04e2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e410ad30 0x7f04e41aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.155+0000 7f04e3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04e41162b0 0x7f04e41b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48926/0 (socket says 172.21.15.32:48926) 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.155+0000 7f04e3400640 1 -- 172.21.15.32:0/1139043592 learned_addr learned my addr 172.21.15.32:0/1139043592 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.155+0000 7f04e3400640 1 -- 172.21.15.32:0/1139043592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e410a3e0 msgr2=0x7f04e41a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.155+0000 7f04e3400640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e410a3e0 0x7f04e41a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.155+0000 7f04e3400640 1 -- 172.21.15.32:0/1139043592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e410ad30 msgr2=0x7f04e41aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.156+0000 7f04e3400640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e410ad30 0x7f04e41aa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.156+0000 7f04e3400640 1 -- 172.21.15.32:0/1139043592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e41aa910 con 0x7f04e41162b0 2024-08-31T07:29:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.156+0000 7f04e2000640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e410ad30 0x7f04e41aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.156+0000 7f04e3400640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04e41162b0 0x7f04e41b4bb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f04d400d750 tx=0x7f04d400dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.158+0000 7f04e8a00640 1 -- 172.21.15.32:0/1139043592 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4017070 con 0x7f04e41162b0 2024-08-31T07:29:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.158+0000 7f04e8a00640 1 -- 172.21.15.32:0/1139043592 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d40054d0 con 0x7f04e41162b0 2024-08-31T07:29:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.158+0000 7f04e8a00640 1 -- 172.21.15.32:0/1139043592 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4004c60 con 0x7f04e41162b0 2024-08-31T07:29:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.158+0000 7f04e3e00640 1 -- 172.21.15.32:0/1139043592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e41b51b0 con 0x7f04e41162b0 2024-08-31T07:29:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.158+0000 7f04e3e00640 1 -- 172.21.15.32:0/1139043592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41b5670 con 0x7f04e41162b0 2024-08-31T07:29:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.160+0000 7f04e3e00640 1 -- 172.21.15.32:0/1139043592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e410b8f0 con 0x7f04e41162b0 2024-08-31T07:29:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.167+0000 7f04e8a00640 1 -- 172.21.15.32:0/1139043592 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f04d4009bd0 con 0x7f04e41162b0 2024-08-31T07:29:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.168+0000 7f04e8a00640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f04c80b4e70 0x7f04c80b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.168+0000 7f04e8a00640 1 -- 172.21.15.32:0/1139043592 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8441+0+0 (secure 0 0 0) 0x7f04d40e4e70 con 0x7f04e41162b0 2024-08-31T07:29:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.171+0000 7f04e2a00640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f04c80b4e70 0x7f04c80b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.171+0000 7f04e8a00640 1 -- 172.21.15.32:0/1139043592 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f04d40abd50 con 0x7f04e41162b0 2024-08-31T07:29:05.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.172+0000 7f04e2a00640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f04c80b4e70 0x7f04c80b7380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f04e4112d00 tx=0x7f04dc009710 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:05.346 INFO:tasks.workunit.client.0.smithi032.stdout: 10 55778 7.00 MB/sec execute 45 sec latency 179.298 ms 2024-08-31T07:29:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.597+0000 7f04e3e00640 1 -- 172.21.15.32:0/1139043592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04e410a7f0 con 0x7f04e41162b0 2024-08-31T07:29:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.598+0000 7f04e8a00640 1 -- 172.21.15.32:0/1139043592 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11963 (secure 0 0 0) 0x7f04d40ab650 con 0x7f04e41162b0 2024-08-31T07:29:05.601 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:29:05.601 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":63,"btime":"2024-08-31T07:29:05:393597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":63,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:29:05.122632+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:29:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 63 2024-08-31T07:29:05.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.601+0000 7f04e3e00640 1 -- 172.21.15.32:0/1139043592 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f04c80b4e70 msgr2=0x7f04c80b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:05.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.601+0000 7f04e3e00640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f04c80b4e70 0x7f04c80b7380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f04e4112d00 tx=0x7f04dc009710 comp rx=0 tx=0).stop 2024-08-31T07:29:05.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.601+0000 7f04e3e00640 1 -- 172.21.15.32:0/1139043592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04e41162b0 msgr2=0x7f04e41b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:05.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.601+0000 7f04e3e00640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04e41162b0 0x7f04e41b4bb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f04d400d750 tx=0x7f04d400dc10 comp rx=0 tx=0).stop 2024-08-31T07:29:05.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.602+0000 7f04e3e00640 1 -- 172.21.15.32:0/1139043592 shutdown_connections 2024-08-31T07:29:05.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.602+0000 7f04e3e00640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f04c80b4e70 0x7f04c80b7380 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:05.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.602+0000 7f04e3e00640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04e41162b0 0x7f04e41b4bb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:05.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.602+0000 7f04e3e00640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e410ad30 0x7f04e41aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:05.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.602+0000 7f04e3e00640 1 --2- 172.21.15.32:0/1139043592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04e410a3e0 0x7f04e41a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:05.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.602+0000 7f04e3e00640 1 -- 172.21.15.32:0/1139043592 >> 172.21.15.32:0/1139043592 conn(0x7f04e406e790 msgr2=0x7f04e40739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:05.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.603+0000 7f04e3e00640 1 -- 172.21.15.32:0/1139043592 shutdown_connections 2024-08-31T07:29:05.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:05.603+0000 7f04e3e00640 1 -- 172.21.15.32:0/1139043592 wait complete. 2024-08-31T07:29:06.346 INFO:tasks.workunit.client.0.smithi032.stdout: 10 56594 7.57 MB/sec execute 46 sec latency 123.001 ms 2024-08-31T07:29:06.553 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 141.25 --await-for 45.0' 2024-08-31T07:29:06.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 141.25 --await-for 45.0 2024-08-31T07:29:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:06 smithi073 ceph-mon[21995]: pgmap v878: 97 pgs: 97 active+clean; 908 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-08-31T07:29:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:06 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:06 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1139043592' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:06 smithi084 ceph-mon[22387]: pgmap v878: 97 pgs: 97 active+clean; 908 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-08-31T07:29:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:06 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:06 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1139043592' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:06.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:06 smithi032 ceph-mon[17252]: pgmap v878: 97 pgs: 97 active+clean; 908 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 83 op/s 2024-08-31T07:29:06.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:06 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:06.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:06 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1139043592' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:07.197 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:07.346 INFO:tasks.workunit.client.0.smithi032.stdout: 10 57868 8.84 MB/sec execute 47 sec latency 44.002 ms 2024-08-31T07:29:07.622 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:07 smithi032 ceph-mon[17252]: osdmap e101: 12 total, 12 up, 12 in 2024-08-31T07:29:07.622 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:07 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:07.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:07 smithi073 ceph-mon[21995]: osdmap e101: 12 total, 12 up, 12 in 2024-08-31T07:29:07.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:07 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:07.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:07 smithi084 ceph-mon[22387]: osdmap e101: 12 total, 12 up, 12 in 2024-08-31T07:29:07.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:07 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.099+0000 7f17dfe00640 1 -- 172.21.15.32:0/4188564390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102210 msgr2=0x7f17e0102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.099+0000 7f17dfe00640 1 --2- 172.21.15.32:0/4188564390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102210 0x7f17e0102620 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009900 tx=0x7f17d002a490 comp rx=0 tx=0).stop 2024-08-31T07:29:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.099+0000 7f17dfe00640 1 -- 172.21.15.32:0/4188564390 shutdown_connections 2024-08-31T07:29:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.099+0000 7f17dfe00640 1 --2- 172.21.15.32:0/4188564390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e010fdc0 0x7f17e01121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.099+0000 7f17dfe00640 1 --2- 172.21.15.32:0/4188564390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e0102b60 0x7f17e010f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.099+0000 7f17dfe00640 1 --2- 172.21.15.32:0/4188564390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102210 0x7f17e0102620 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.099+0000 7f17dfe00640 1 -- 172.21.15.32:0/4188564390 >> 172.21.15.32:0/4188564390 conn(0x7f17e00fe0d0 msgr2=0x7f17e0100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.101+0000 7f17dfe00640 1 -- 172.21.15.32:0/4188564390 shutdown_connections 2024-08-31T07:29:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.101+0000 7f17dfe00640 1 -- 172.21.15.32:0/4188564390 wait complete. 2024-08-31T07:29:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.102+0000 7f17dfe00640 1 Processor -- start 2024-08-31T07:29:08.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.103+0000 7f17dfe00640 1 -- start start 2024-08-31T07:29:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.103+0000 7f17dfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102210 0x7f17e019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.103+0000 7f17dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b60 0x7f17e019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.103+0000 7f17dfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e010fdc0 0x7f17e01a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.103+0000 7f17dfe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01158b0 con 0x7f17e0102b60 2024-08-31T07:29:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.103+0000 7f17dfe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0115730 con 0x7f17e010fdc0 2024-08-31T07:29:08.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.103+0000 7f17dfe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0115a30 con 0x7f17e0102210 2024-08-31T07:29:08.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.103+0000 7f17de000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b60 0x7f17e019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.104+0000 7f17dea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102210 0x7f17e019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.104+0000 7f17dea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102210 0x7f17e019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58668/0 (socket says 172.21.15.32:58668) 2024-08-31T07:29:08.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.104+0000 7f17dea00640 1 -- 172.21.15.32:0/2949201451 learned_addr learned my addr 172.21.15.32:0/2949201451 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:08.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.104+0000 7f17df400640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e010fdc0 0x7f17e01a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.105+0000 7f17dea00640 1 -- 172.21.15.32:0/2949201451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e010fdc0 msgr2=0x7f17e01a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.105+0000 7f17dea00640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e010fdc0 0x7f17e01a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.105+0000 7f17dea00640 1 -- 172.21.15.32:0/2949201451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b60 msgr2=0x7f17e019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.105+0000 7f17dea00640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b60 0x7f17e019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.105+0000 7f17dea00640 1 -- 172.21.15.32:0/2949201451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e01a2190 con 0x7f17e0102210 2024-08-31T07:29:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.105+0000 7f17de000640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b60 0x7f17e019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:08.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.105+0000 7f17dea00640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102210 0x7f17e019c2e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17d0005ab0 tx=0x7f17d0003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:08.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.106+0000 7f17d7400640 1 -- 172.21.15.32:0/2949201451 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d003f070 con 0x7f17e0102210 2024-08-31T07:29:08.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.106+0000 7f17d7400640 1 -- 172.21.15.32:0/2949201451 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d0031d60 con 0x7f17e0102210 2024-08-31T07:29:08.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.106+0000 7f17d7400640 1 -- 172.21.15.32:0/2949201451 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0035030 con 0x7f17e0102210 2024-08-31T07:29:08.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.107+0000 7f17dfe00640 1 -- 172.21.15.32:0/2949201451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e01a2420 con 0x7f17e0102210 2024-08-31T07:29:08.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.107+0000 7f17dfe00640 1 -- 172.21.15.32:0/2949201451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e01a28b0 con 0x7f17e0102210 2024-08-31T07:29:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.111+0000 7f17ec6bd740 1 Processor -- start 2024-08-31T07:29:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.112+0000 7f17ec6bd740 1 -- start start 2024-08-31T07:29:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.112+0000 7f17d7400640 1 -- 172.21.15.32:0/2949201451 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f17d0042070 con 0x7f17e0102210 2024-08-31T07:29:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.112+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cdf01850 0x55b8ce2b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.112+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cded8990 0x55b8ce2b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.113+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.113+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b8ce21d450 con 0x55b8cdf01850 2024-08-31T07:29:08.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.113+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b8ce20c9e0 con 0x55b8cded8990 2024-08-31T07:29:08.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.113+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b8ce210990 con 0x55b8cdee0b70 2024-08-31T07:29:08.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.113+0000 7f17de000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.113+0000 7f17de000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58682/0 (socket says 172.21.15.32:58682) 2024-08-31T07:29:08.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.113+0000 7f17de000640 1 -- 172.21.15.32:0/237969460 learned_addr learned my addr 172.21.15.32:0/237969460 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:08.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.113+0000 7f17df400640 1 --2- 172.21.15.32:0/237969460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cdf01850 0x55b8ce2b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.115+0000 7f17d7400640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c40b4e10 0x7f17c40b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.115+0000 7f17d7400640 1 -- 172.21.15.32:0/2949201451 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8485+0+0 (secure 0 0 0) 0x7f17d00c8960 con 0x7f17e0102210 2024-08-31T07:29:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.115+0000 7f17de000640 1 -- 172.21.15.32:0/237969460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cded8990 msgr2=0x55b8ce2b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.115+0000 7f17de000640 1 --2- 172.21.15.32:0/237969460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cded8990 0x55b8ce2b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.115+0000 7f17de000640 1 -- 172.21.15.32:0/237969460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cdf01850 msgr2=0x55b8ce2b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.115+0000 7f17de000640 1 --2- 172.21.15.32:0/237969460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cdf01850 0x55b8ce2b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.115+0000 7f17de000640 1 -- 172.21.15.32:0/237969460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ce19ffa0 con 0x55b8cdee0b70 2024-08-31T07:29:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.116+0000 7f17df400640 1 --2- 172.21.15.32:0/237969460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cdf01850 0x55b8ce2b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.116+0000 7f17df400640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c40b4e10 0x7f17c40b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.117+0000 7f17df400640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c40b4e10 0x7f17c40b7320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f17d8005fd0 tx=0x7f17d8004a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.118+0000 7f17de000640 1 --2- 172.21.15.32:0/237969460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2b1060 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00f470 tx=0x7f17cc00f930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b89343bab6531d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.119+0000 7f17cbe00640 1 -- 172.21.15.32:0/237969460 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc005220 con 0x55b8cdee0b70 2024-08-31T07:29:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.119+0000 7f17cbe00640 1 -- 172.21.15.32:0/237969460 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc004c50 con 0x55b8cdee0b70 2024-08-31T07:29:08.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.119+0000 7f17cbe00640 1 -- 172.21.15.32:0/237969460 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc010c80 con 0x55b8cdee0b70 2024-08-31T07:29:08.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.120+0000 7f17ec6bd740 1 -- 172.21.15.32:0/237969460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 msgr2=0x55b8ce2b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:08.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.120+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/237969460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2b1060 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00f470 tx=0x7f17cc00f930 comp rx=0 tx=0).stop 2024-08-31T07:29:08.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.120+0000 7f17ec6bd740 1 -- 172.21.15.32:0/237969460 shutdown_connections 2024-08-31T07:29:08.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.120+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/237969460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2b1060 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.120+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/237969460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cded8990 0x55b8ce2b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.120+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/237969460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cdf01850 0x55b8ce2b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.120+0000 7f17ec6bd740 1 -- 172.21.15.32:0/237969460 >> 172.21.15.32:0/237969460 conn(0x55b8cdef5c40 msgr2=0x55b8ce2ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:08.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.120+0000 7f17ec6bd740 1 -- 172.21.15.32:0/237969460 shutdown_connections 2024-08-31T07:29:08.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.121+0000 7f17ec6bd740 1 -- 172.21.15.32:0/237969460 wait complete. 2024-08-31T07:29:08.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.121+0000 7f17ec6bd740 1 -- start start 2024-08-31T07:29:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.122+0000 7f17ec6bd740 1 Processor -- start 2024-08-31T07:29:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cded8990 0x55b8ce2bbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2bc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cdf01850 0x55b8ce2c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b8ce06f7b0 con 0x55b8cded8990 2024-08-31T07:29:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b8ce1a7eb0 con 0x55b8cdf01850 2024-08-31T07:29:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b8ce2bedc0 con 0x55b8cdee0b70 2024-08-31T07:29:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17ec6bd740 10 client.0 _ll_register_callbacks cb 0x55b8ce20a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17df400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cdf01850 0x55b8ce2c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17de000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2bc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17de000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2bc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58696/0 (socket says 172.21.15.32:58696) 2024-08-31T07:29:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.123+0000 7f17de000640 1 -- 172.21.15.32:0/2591241432 learned_addr learned my addr 172.21.15.32:0/2591241432 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.124+0000 7f17de000640 1 -- 172.21.15.32:0/2591241432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cdf01850 msgr2=0x55b8ce2c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.124+0000 7f17de000640 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cdf01850 0x55b8ce2c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.124+0000 7f17de000640 1 -- 172.21.15.32:0/2591241432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cded8990 msgr2=0x55b8ce2bbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.124+0000 7f17de000640 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cded8990 0x55b8ce2bbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.124+0000 7f17de000640 1 -- 172.21.15.32:0/2591241432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ce1b2f40 con 0x55b8cdee0b70 2024-08-31T07:29:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.125+0000 7f17de000640 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2bc320 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f17cc01a040 tx=0x7f17cc005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7291fd3752c8564f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.125+0000 7f17c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:29:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.125+0000 7f17c8c00640 1 -- 172.21.15.32:0/2591241432 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc027070 con 0x55b8cdee0b70 2024-08-31T07:29:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.125+0000 7f17c8c00640 1 -- 172.21.15.32:0/2591241432 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8485+0+0 (secure 0 0 0) 0x7f17cc010b90 con 0x55b8cdee0b70 2024-08-31T07:29:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.125+0000 7f17ec6bd740 1 -- 172.21.15.32:0/2591241432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17cc004c50 con 0x55b8cdee0b70 2024-08-31T07:29:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.126+0000 7f17c8c00640 1 -- 172.21.15.32:0/2591241432 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f17cc033050 con 0x55b8cdee0b70 2024-08-31T07:29:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.126+0000 7f17ec6bd740 10 client.28771 fetch_fsmap learned FSMap version 64 2024-08-31T07:29:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.126+0000 7f17ec6bd740 1 -- 172.21.15.32:0/2591241432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b8cdf7bba0 con 0x55b8cdee0b70 2024-08-31T07:29:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.127+0000 7f17c8c00640 1 -- 172.21.15.32:0/2591241432 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17cc004c50 con 0x55b8cdee0b70 2024-08-31T07:29:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.127+0000 7f17c8c00640 1 -- 172.21.15.32:0/2591241432 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 64) ==== 8362+0+0 (secure 0 0 0) 0x7f17cc0185a0 con 0x55b8cdee0b70 2024-08-31T07:29:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.127+0000 7f17ec6bd740 10 client.28771 fetch_fsmap finished waiting for FSMap version 64 2024-08-31T07:29:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.127+0000 7f17ec6bd740 10 client.28771 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:29:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.127+0000 7f17ec6bd740 20 client.28771 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.127+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b8cdec01d0 0x55b8ce2deeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.127+0000 7f17ec6bd740 4 client.28771 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.127+0000 7f17ec6bd740 1 -- 172.21.15.32:0/2591241432 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8ce21f3a0 con 0x55b8cdec01d0 2024-08-31T07:29:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.128+0000 7f17dea00640 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b8cdec01d0 0x55b8ce2deeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.129+0000 7f17dea00640 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b8cdec01d0 0x55b8ce2deeb0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb66c0c5c080c61 server_cookie=7a743aecdfc2155c in_seq=0 out_seq=0 2024-08-31T07:29:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.129+0000 7f17c8c00640 10 client.28771 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.132+0000 7f17c8c00640 1 -- 172.21.15.32:0/2591241432 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b8ce21f3a0 con 0x55b8cdec01d0 2024-08-31T07:29:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.132+0000 7f17c8c00640 10 client.28771 handle_command_reply: tid=0 2024-08-31T07:29:08.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.132+0000 7f17ec6bd740 1 client.28771 shutdown 2024-08-31T07:29:08.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.133+0000 7f17ec6bd740 1 -- 172.21.15.32:0/2591241432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 msgr2=0x55b8ce2bc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:08.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.133+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2bc320 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f17cc01a040 tx=0x7f17cc005870 comp rx=0 tx=0).stop 2024-08-31T07:29:08.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.133+0000 7f17ec6bd740 1 -- 172.21.15.32:0/2591241432 shutdown_connections 2024-08-31T07:29:08.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.133+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b8cdec01d0 0x55b8ce2deeb0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.133+0000 7f17c8c00640 0 client.28771 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:08.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.133+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cdf01850 0x55b8ce2c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.133+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdee0b70 0x55b8ce2bc320 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.133+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/2591241432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cded8990 0x55b8ce2bbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.133+0000 7f17ec6bd740 1 -- 172.21.15.32:0/2591241432 >> 172.21.15.32:0/2591241432 conn(0x55b8cdef5c40 msgr2=0x55b8ce2b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.134+0000 7f17ec6bd740 1 -- 172.21.15.32:0/2591241432 shutdown_connections 2024-08-31T07:29:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.135+0000 7f17ec6bd740 1 -- 172.21.15.32:0/2591241432 wait complete. 2024-08-31T07:29:08.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.135+0000 7f17ec6bd740 20 client.28771 trim_cache size 0 max 16384 2024-08-31T07:29:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.172+0000 7f17ec6bd740 1 Processor -- start 2024-08-31T07:29:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.172+0000 7f17ec6bd740 1 -- start start 2024-08-31T07:29:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.173+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cded8990 0x55b8ce2c8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.173+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cdf01850 0x55b8ce2b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.173+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cde95580 0x55b8ce2b94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.173+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0070f70 con 0x55b8cded8990 2024-08-31T07:29:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.173+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0072ab0 con 0x55b8cdf01850 2024-08-31T07:29:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.173+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0071640 con 0x55b8cde95580 2024-08-31T07:29:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.173+0000 7f17df400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cde95580 0x55b8ce2b94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.173+0000 7f17df400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cde95580 0x55b8ce2b94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58708/0 (socket says 172.21.15.32:58708) 2024-08-31T07:29:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.173+0000 7f17df400640 1 -- 172.21.15.32:0/1717601959 learned_addr learned my addr 172.21.15.32:0/1717601959 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.173+0000 7f17de000640 1 --2- 172.21.15.32:0/1717601959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cdf01850 0x55b8ce2b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.174+0000 7f17df400640 1 -- 172.21.15.32:0/1717601959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cdf01850 msgr2=0x55b8ce2b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.174+0000 7f17df400640 1 --2- 172.21.15.32:0/1717601959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cdf01850 0x55b8ce2b8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.174+0000 7f17df400640 1 -- 172.21.15.32:0/1717601959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cded8990 msgr2=0x55b8ce2c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.174+0000 7f17df400640 1 --2- 172.21.15.32:0/1717601959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cded8990 0x55b8ce2c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.174+0000 7f17df400640 1 -- 172.21.15.32:0/1717601959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ce2a10c0 con 0x55b8cde95580 2024-08-31T07:29:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.174+0000 7f17df400640 1 --2- 172.21.15.32:0/1717601959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cde95580 0x55b8ce2b94f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f17d8013420 tx=0x7f17d8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c28a365cf5c86446 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.175+0000 7f17cb400640 1 -- 172.21.15.32:0/1717601959 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8011800 con 0x55b8cde95580 2024-08-31T07:29:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.175+0000 7f17cb400640 1 -- 172.21.15.32:0/1717601959 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d8011e50 con 0x55b8cde95580 2024-08-31T07:29:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.175+0000 7f17cb400640 1 -- 172.21.15.32:0/1717601959 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d801e440 con 0x55b8cde95580 2024-08-31T07:29:08.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.176+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1717601959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cde95580 msgr2=0x55b8ce2b94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:08.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.176+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/1717601959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cde95580 0x55b8ce2b94f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f17d8013420 tx=0x7f17d8016000 comp rx=0 tx=0).stop 2024-08-31T07:29:08.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.177+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1717601959 shutdown_connections 2024-08-31T07:29:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.177+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/1717601959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cde95580 0x55b8ce2b94f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.177+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/1717601959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cdf01850 0x55b8ce2b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.177+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/1717601959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cded8990 0x55b8ce2c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.177+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1717601959 >> 172.21.15.32:0/1717601959 conn(0x55b8cdef5c40 msgr2=0x55b8ce2ae460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.177+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1717601959 shutdown_connections 2024-08-31T07:29:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.177+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1717601959 wait complete. 2024-08-31T07:29:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.177+0000 7f17ec6bd740 1 -- start start 2024-08-31T07:29:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.178+0000 7f17ec6bd740 1 Processor -- start 2024-08-31T07:29:08.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.178+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cde95580 0x55b8ce2e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.178+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cded8990 0x55b8ce2e1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.178+0000 7f17ec6bd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdf01850 0x55b8ce3405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.178+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17e00723e0 con 0x55b8cde95580 2024-08-31T07:29:08.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.178+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0071d10 con 0x55b8cded8990 2024-08-31T07:29:08.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.178+0000 7f17ec6bd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f17e006ef70 con 0x55b8cdf01850 2024-08-31T07:29:08.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.178+0000 7f17ec6bd740 10 client.0 _ll_register_callbacks cb 0x7f17d8009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:08.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.179+0000 7f17df400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdf01850 0x55b8ce3405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.179+0000 7f17de000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cded8990 0x55b8ce2e1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.179+0000 7f17df400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdf01850 0x55b8ce3405e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:58714/0 (socket says 172.21.15.32:58714) 2024-08-31T07:29:08.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.179+0000 7f17df400640 1 -- 172.21.15.32:0/1298423309 learned_addr learned my addr 172.21.15.32:0/1298423309 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:08.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.179+0000 7f17dea00640 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cde95580 0x55b8ce2e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.180+0000 7f17df400640 1 -- 172.21.15.32:0/1298423309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cded8990 msgr2=0x55b8ce2e1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:08.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.180+0000 7f17df400640 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cded8990 0x55b8ce2e1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.180+0000 7f17df400640 1 -- 172.21.15.32:0/1298423309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cde95580 msgr2=0x55b8ce2e0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:08.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.180+0000 7f17df400640 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cde95580 0x55b8ce2e0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.180+0000 7f17df400640 1 -- 172.21.15.32:0/1298423309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ce340c50 con 0x55b8cdf01850 2024-08-31T07:29:08.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.180+0000 7f17de000640 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cded8990 0x55b8ce2e1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:29:08.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.180+0000 7f17dea00640 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cde95580 0x55b8ce2e0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:08.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.181+0000 7f17df400640 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdf01850 0x55b8ce3405e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f17d8020e30 tx=0x7f17d8020e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492b2a0d3d002068 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:08.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.181+0000 7f17c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:29:08.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.181+0000 7f17c8c00640 1 -- 172.21.15.32:0/1298423309 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8037060 con 0x55b8cdf01850 2024-08-31T07:29:08.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.181+0000 7f17c8c00640 1 -- 172.21.15.32:0/1298423309 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8485+0+0 (secure 0 0 0) 0x7f17d8011600 con 0x55b8cdf01850 2024-08-31T07:29:08.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.181+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1298423309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d8011e50 con 0x55b8cdf01850 2024-08-31T07:29:08.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.182+0000 7f17c8c00640 1 -- 172.21.15.32:0/1298423309 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f17d8011800 con 0x55b8cdf01850 2024-08-31T07:29:08.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.182+0000 7f17ec6bd740 10 client.28777 fetch_fsmap learned FSMap version 64 2024-08-31T07:29:08.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.182+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1298423309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b8ce34a210 con 0x55b8cdf01850 2024-08-31T07:29:08.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.182+0000 7f17c8c00640 1 -- 172.21.15.32:0/1298423309 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d8011e50 con 0x55b8cdf01850 2024-08-31T07:29:08.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.183+0000 7f17c8c00640 1 -- 172.21.15.32:0/1298423309 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 64) ==== 8362+0+0 (secure 0 0 0) 0x7f17d8005410 con 0x55b8cdf01850 2024-08-31T07:29:08.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.183+0000 7f17ec6bd740 10 client.28777 fetch_fsmap finished waiting for FSMap version 64 2024-08-31T07:29:08.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.183+0000 7f17ec6bd740 10 client.28777 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:29:08.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.183+0000 7f17ec6bd740 20 client.28777 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:08.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.184+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b8ce34a900 0x55b8ce34cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:08.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.184+0000 7f17ec6bd740 4 client.28777 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.25, "await_for": 45.0}] 2024-08-31T07:29:08.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.184+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1298423309 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.25, "await_for": 45.0}) -- 0x55b8ce21ae40 con 0x55b8ce34a900 2024-08-31T07:29:08.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.184+0000 7f17dea00640 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b8ce34a900 0x55b8ce34cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:08.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.185+0000 7f17dea00640 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b8ce34a900 0x55b8ce34cd20 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd0f8fdc3d3593a server_cookie=f1da42b8993f7d0c in_seq=0 out_seq=0 2024-08-31T07:29:08.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.185+0000 7f17c8c00640 10 client.28777 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:08.347 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58905 10.10 MB/sec execute 48 sec latency 158.946 ms 2024-08-31T07:29:08.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:08 smithi032 ceph-mon[17252]: pgmap v880: 97 pgs: 97 active+clean; 935 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.7 MiB/s wr, 273 op/s 2024-08-31T07:29:08.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.607+0000 7f17c8c00640 1 -- 172.21.15.32:0/1298423309 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b8ce21ae40 con 0x55b8ce34a900 2024-08-31T07:29:08.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.607+0000 7f17c8c00640 10 client.28777 handle_command_reply: tid=0 2024-08-31T07:29:08.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.608+0000 7f17ec6bd740 1 client.28777 shutdown 2024-08-31T07:29:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.608+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1298423309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdf01850 msgr2=0x55b8ce3405e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.608+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdf01850 0x55b8ce3405e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f17d8020e30 tx=0x7f17d8020e60 comp rx=0 tx=0).stop 2024-08-31T07:29:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.609+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1298423309 shutdown_connections 2024-08-31T07:29:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.609+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b8ce34a900 0x55b8ce34cd20 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.609+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b8cdf01850 0x55b8ce3405e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.609+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b8cded8990 0x55b8ce2e1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.609+0000 7f17ec6bd740 1 --2- 172.21.15.32:0/1298423309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8cde95580 0x55b8ce2e0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.609+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1298423309 >> 172.21.15.32:0/1298423309 conn(0x55b8cdef5c40 msgr2=0x55b8ce2b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.609+0000 7f17c8c00640 0 client.28777 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.609+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1298423309 shutdown_connections 2024-08-31T07:29:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.609+0000 7f17ec6bd740 1 -- 172.21.15.32:0/1298423309 wait complete. 2024-08-31T07:29:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.609+0000 7f17ec6bd740 20 client.28777 trim_cache size 0 max 16384 2024-08-31T07:29:08.612 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:29:08.612 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 64, 2024-08-31T07:29:08.612 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:29:08.612 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 54, 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "71721071": { 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 54, 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 141.2, 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:29:08.613 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:29:08.614 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:29:08.614 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:29:08.614 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:29:08.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:08.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:08.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:08.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:08.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:08.614 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.610+0000 7f17dfe00640 1 -- 172.21.15.32:0/2949201451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c40b4e10 msgr2=0x7f17c40b7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.610+0000 7f17dfe00640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c40b4e10 0x7f17c40b7320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f17d8005fd0 tx=0x7f17d8004a40 comp rx=0 tx=0).stop 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.610+0000 7f17dfe00640 1 -- 172.21.15.32:0/2949201451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102210 msgr2=0x7f17e019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.610+0000 7f17dfe00640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102210 0x7f17e019c2e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17d0005ab0 tx=0x7f17d0003cd0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.611+0000 7f17dfe00640 1 -- 172.21.15.32:0/2949201451 shutdown_connections 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.611+0000 7f17dfe00640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f17c40b4e10 0x7f17c40b7320 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.611+0000 7f17dfe00640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e010fdc0 0x7f17e01a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.611+0000 7f17dfe00640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17e0102b60 0x7f17e019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.611+0000 7f17dfe00640 1 --2- 172.21.15.32:0/2949201451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17e0102210 0x7f17e019c2e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.611+0000 7f17dfe00640 1 -- 172.21.15.32:0/2949201451 >> 172.21.15.32:0/2949201451 conn(0x7f17e00fe0d0 msgr2=0x7f17e0111f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.611+0000 7f17dfe00640 1 -- 172.21.15.32:0/2949201451 shutdown_connections 2024-08-31T07:29:08.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:08.611+0000 7f17dfe00640 1 -- 172.21.15.32:0/2949201451 wait complete. 2024-08-31T07:29:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:08 smithi073 ceph-mon[21995]: pgmap v880: 97 pgs: 97 active+clean; 935 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.7 MiB/s wr, 273 op/s 2024-08-31T07:29:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:08 smithi084 ceph-mon[22387]: pgmap v880: 97 pgs: 97 active+clean; 935 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.7 MiB/s wr, 273 op/s 2024-08-31T07:29:09.347 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 9.90 MB/sec execute 49 sec latency 1133.029 ms 2024-08-31T07:29:10.347 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 9.70 MB/sec execute 50 sec latency 2133.099 ms 2024-08-31T07:29:10.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:10 smithi032 ceph-mon[17252]: pgmap v881: 97 pgs: 97 active+clean; 951 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.4 MiB/s wr, 404 op/s 2024-08-31T07:29:10.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:10 smithi084 ceph-mon[22387]: pgmap v881: 97 pgs: 97 active+clean; 951 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.4 MiB/s wr, 404 op/s 2024-08-31T07:29:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:10 smithi073 ceph-mon[21995]: pgmap v881: 97 pgs: 97 active+clean; 951 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.4 MiB/s wr, 404 op/s 2024-08-31T07:29:11.035 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '71721071', quiesce took 7.6 seconds. Will release after: 13.65 2024-08-31T07:29:11.347 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 9.51 MB/sec execute 51 sec latency 3133.226 ms 2024-08-31T07:29:11.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:11 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:11.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:11 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:29:11.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:11 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:29:11.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:11 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:29:11.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:11 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:11.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:11 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:29:11.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:11 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:29:11.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:11 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:29:11.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:11 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:11.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:11 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:29:11.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:11 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:29:11.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:11 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:29:12.347 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 9.33 MB/sec execute 52 sec latency 4133.379 ms 2024-08-31T07:29:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:12 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:29:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:12 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:29:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:12 smithi084 ceph-mon[22387]: pgmap v882: 97 pgs: 97 active+clean; 961 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.3 MiB/s wr, 488 op/s 2024-08-31T07:29:12.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:12 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:29:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:12 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:29:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:12 smithi032 ceph-mon[17252]: pgmap v882: 97 pgs: 97 active+clean; 961 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.3 MiB/s wr, 488 op/s 2024-08-31T07:29:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:12 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:29:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:12 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:29:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:12 smithi073 ceph-mon[21995]: pgmap v882: 97 pgs: 97 active+clean; 961 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.3 MiB/s wr, 488 op/s 2024-08-31T07:29:13.347 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 9.15 MB/sec execute 53 sec latency 5133.532 ms 2024-08-31T07:29:14.347 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 8.98 MB/sec execute 54 sec latency 6133.660 ms 2024-08-31T07:29:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:14 smithi084 ceph-mon[22387]: pgmap v883: 97 pgs: 97 active+clean; 966 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 530 op/s 2024-08-31T07:29:14.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:14 smithi032 ceph-mon[17252]: pgmap v883: 97 pgs: 97 active+clean; 966 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 530 op/s 2024-08-31T07:29:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:14 smithi073 ceph-mon[21995]: pgmap v883: 97 pgs: 97 active+clean; 966 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 530 op/s 2024-08-31T07:29:15.348 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 8.82 MB/sec execute 55 sec latency 7133.802 ms 2024-08-31T07:29:16.348 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 8.66 MB/sec execute 56 sec latency 8133.893 ms 2024-08-31T07:29:16.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:16 smithi084 ceph-mon[22387]: pgmap v884: 97 pgs: 97 active+clean; 966 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.8 MiB/s wr, 489 op/s 2024-08-31T07:29:16.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:16 smithi032 ceph-mon[17252]: pgmap v884: 97 pgs: 97 active+clean; 966 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.8 MiB/s wr, 489 op/s 2024-08-31T07:29:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:16 smithi073 ceph-mon[21995]: pgmap v884: 97 pgs: 97 active+clean; 966 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.8 MiB/s wr, 489 op/s 2024-08-31T07:29:17.348 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 8.51 MB/sec execute 57 sec latency 9133.957 ms 2024-08-31T07:29:18.168 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:17 smithi032 ceph-mon[17252]: osdmap e102: 12 total, 12 up, 12 in 2024-08-31T07:29:18.168 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:17 smithi032 ceph-mon[17252]: pgmap v886: 97 pgs: 97 active+clean; 972 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 326 op/s 2024-08-31T07:29:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:17 smithi073 ceph-mon[21995]: osdmap e102: 12 total, 12 up, 12 in 2024-08-31T07:29:18.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:17 smithi073 ceph-mon[21995]: pgmap v886: 97 pgs: 97 active+clean; 972 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 326 op/s 2024-08-31T07:29:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:17 smithi084 ceph-mon[22387]: osdmap e102: 12 total, 12 up, 12 in 2024-08-31T07:29:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:17 smithi084 ceph-mon[22387]: pgmap v886: 97 pgs: 97 active+clean; 972 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 326 op/s 2024-08-31T07:29:18.348 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 8.36 MB/sec execute 58 sec latency 10134.018 ms 2024-08-31T07:29:19.348 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 8.22 MB/sec execute 59 sec latency 11134.125 ms 2024-08-31T07:29:19.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:29:19.756 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:20.348 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 8.08 MB/sec execute 60 sec latency 12134.191 ms 2024-08-31T07:29:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.419+0000 7f5749800640 1 -- 172.21.15.32:0/2827862778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f574410b290 msgr2=0x7f574410b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.419+0000 7f5749800640 1 --2- 172.21.15.32:0/2827862778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f574410b290 0x7f574410b6a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5730009860 tx=0x7f573002a340 comp rx=0 tx=0).stop 2024-08-31T07:29:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.419+0000 7f5749800640 1 -- 172.21.15.32:0/2827862778 shutdown_connections 2024-08-31T07:29:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.419+0000 7f5749800640 1 --2- 172.21.15.32:0/2827862778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574410bbe0 0x7f5744112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.419+0000 7f5749800640 1 --2- 172.21.15.32:0/2827862778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f574410b290 0x7f574410b6a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.419+0000 7f5749800640 1 --2- 172.21.15.32:0/2827862778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5744105950 0x7f5744105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.419+0000 7f5749800640 1 -- 172.21.15.32:0/2827862778 >> 172.21.15.32:0/2827862778 conn(0x7f5744101420 msgr2=0x7f5744103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.419+0000 7f5749800640 1 -- 172.21.15.32:0/2827862778 shutdown_connections 2024-08-31T07:29:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.419+0000 7f5749800640 1 -- 172.21.15.32:0/2827862778 wait complete. 2024-08-31T07:29:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.420+0000 7f5749800640 1 Processor -- start 2024-08-31T07:29:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.420+0000 7f5749800640 1 -- start start 2024-08-31T07:29:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5749800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5744105950 0x7f574419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5749800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f574410b290 0x7f574419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5749800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f574410bbe0 0x7f57441a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5749800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f57441158a0 con 0x7f5744105950 2024-08-31T07:29:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5749800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5744115720 con 0x7f574410bbe0 2024-08-31T07:29:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5749800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5744115a20 con 0x7f574410b290 2024-08-31T07:29:20.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5743e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5744105950 0x7f574419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5743e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5744105950 0x7f574419c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54072/0 (socket says 172.21.15.32:54072) 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5743e00640 1 -- 172.21.15.32:0/2490977710 learned_addr learned my addr 172.21.15.32:0/2490977710 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5748e00640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f574410bbe0 0x7f57441a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.421+0000 7f5743400640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f574410b290 0x7f574419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5743e00640 1 -- 172.21.15.32:0/2490977710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f574410b290 msgr2=0x7f574419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5743e00640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f574410b290 0x7f574419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5743e00640 1 -- 172.21.15.32:0/2490977710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f574410bbe0 msgr2=0x7f57441a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5743e00640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f574410bbe0 0x7f57441a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5743e00640 1 -- 172.21.15.32:0/2490977710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57441a2240 con 0x7f5744105950 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5748e00640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f574410bbe0 0x7f57441a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5743e00640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5744105950 0x7f574419c2d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f573400e9c0 tx=0x7f573400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5740c00640 1 -- 172.21.15.32:0/2490977710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573400cd80 con 0x7f5744105950 2024-08-31T07:29:20.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5740c00640 1 -- 172.21.15.32:0/2490977710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57340035f0 con 0x7f5744105950 2024-08-31T07:29:20.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5749800640 1 -- 172.21.15.32:0/2490977710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57441a2530 con 0x7f5744105950 2024-08-31T07:29:20.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5740c00640 1 -- 172.21.15.32:0/2490977710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734010690 con 0x7f5744105950 2024-08-31T07:29:20.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.422+0000 7f5749800640 1 -- 172.21.15.32:0/2490977710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57441a8c90 con 0x7f5744105950 2024-08-31T07:29:20.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.426+0000 7f57519bc740 1 Processor -- start 2024-08-31T07:29:20.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.426+0000 7f57519bc740 1 -- start start 2024-08-31T07:29:20.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.426+0000 7f57519bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d44009070 0x559d443aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.426+0000 7f5740c00640 1 -- 172.21.15.32:0/2490977710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5734004000 con 0x7f5744105950 2024-08-31T07:29:20.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.426+0000 7f57519bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe01b0 0x559d443af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.426+0000 7f57519bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.426+0000 7f57519bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559d44325770 con 0x559d43fe8390 2024-08-31T07:29:20.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.426+0000 7f57519bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559d44314d00 con 0x559d44009070 2024-08-31T07:29:20.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.426+0000 7f57519bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559d44318cb0 con 0x559d43fe01b0 2024-08-31T07:29:20.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5743400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5748e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d44009070 0x559d443aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5748e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d44009070 0x559d443aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48274/0 (socket says 172.21.15.32:48274) 2024-08-31T07:29:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5748e00640 1 -- 172.21.15.32:0/138332182 learned_addr learned my addr 172.21.15.32:0/138332182 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5743e00640 1 --2- 172.21.15.32:0/138332182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe01b0 0x559d443af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5743400640 1 -- 172.21.15.32:0/138332182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe01b0 msgr2=0x559d443af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5743400640 1 --2- 172.21.15.32:0/138332182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe01b0 0x559d443af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5743400640 1 -- 172.21.15.32:0/138332182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d44009070 msgr2=0x559d443aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5743400640 1 --2- 172.21.15.32:0/138332182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d44009070 0x559d443aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5743400640 1 -- 172.21.15.32:0/138332182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d44074650 con 0x559d43fe8390 2024-08-31T07:29:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.427+0000 7f5748e00640 1 --2- 172.21.15.32:0/138332182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d44009070 0x559d443aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:20.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.428+0000 7f5743400640 1 --2- 172.21.15.32:0/138332182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443af5a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5730009780 tx=0x7f5730005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a18c864b9b6ff36 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:20.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.428+0000 7f5739600640 1 -- 172.21.15.32:0/138332182 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730041020 con 0x559d43fe8390 2024-08-31T07:29:20.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.428+0000 7f5739600640 1 -- 172.21.15.32:0/138332182 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5730030450 con 0x559d43fe8390 2024-08-31T07:29:20.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.428+0000 7f5739600640 1 -- 172.21.15.32:0/138332182 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730037040 con 0x559d43fe8390 2024-08-31T07:29:20.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.428+0000 7f5740c00640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f57280b4ca0 0x7f57280b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f5740c00640 1 -- 172.21.15.32:0/2490977710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f57340e5800 con 0x7f5744105950 2024-08-31T07:29:20.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f5748e00640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f57280b4ca0 0x7f57280b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f57519bc740 1 -- 172.21.15.32:0/138332182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 msgr2=0x559d443af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:20.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f57519bc740 1 --2- 172.21.15.32:0/138332182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443af5a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5730009780 tx=0x7f5730005d10 comp rx=0 tx=0).stop 2024-08-31T07:29:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f57519bc740 1 -- 172.21.15.32:0/138332182 shutdown_connections 2024-08-31T07:29:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f57519bc740 1 --2- 172.21.15.32:0/138332182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443af5a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f57519bc740 1 --2- 172.21.15.32:0/138332182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe01b0 0x559d443af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f57519bc740 1 --2- 172.21.15.32:0/138332182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d44009070 0x559d443aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f57519bc740 1 -- 172.21.15.32:0/138332182 >> 172.21.15.32:0/138332182 conn(0x559d43ffd460 msgr2=0x559d443aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f57519bc740 1 -- 172.21.15.32:0/138332182 shutdown_connections 2024-08-31T07:29:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f5748e00640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f57280b4ca0 0x7f57280b71b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f573c012c30 tx=0x7f573c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.429+0000 7f57519bc740 1 -- 172.21.15.32:0/138332182 wait complete. 2024-08-31T07:29:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.430+0000 7f57519bc740 1 -- start start 2024-08-31T07:29:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.430+0000 7f57519bc740 1 Processor -- start 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f57519bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443ba710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f57519bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 0x559d443bac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f57519bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d44009070 0x559d443c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f57519bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559d44177700 con 0x559d44009070 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f57519bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559d4429bcb0 con 0x559d43fe01b0 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f57519bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559d443bd2a0 con 0x559d43fe8390 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f57519bc740 10 client.0 _ll_register_callbacks cb 0x559d442c1430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f5743e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f5748e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d44009070 0x559d443c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f5743e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443ba710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48280/0 (socket says 172.21.15.32:48280) 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f5743e00640 1 -- 172.21.15.32:0/3089888233 learned_addr learned my addr 172.21.15.32:0/3089888233 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:20.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.431+0000 7f5743400640 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 0x559d443bac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.432+0000 7f5743e00640 1 -- 172.21.15.32:0/3089888233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 msgr2=0x559d443bac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:20.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.432+0000 7f5743e00640 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 0x559d443bac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.432+0000 7f5743e00640 1 -- 172.21.15.32:0/3089888233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d44009070 msgr2=0x559d443c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:20.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.432+0000 7f5743e00640 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d44009070 0x559d443c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.432+0000 7f5743e00640 1 -- 172.21.15.32:0/3089888233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d442dc870 con 0x559d43fe01b0 2024-08-31T07:29:20.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.432+0000 7f5743e00640 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443ba710 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f57340abbe0 tx=0x7f57340092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e986c57779613c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:20.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.432+0000 7f572e000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:29:20.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.432+0000 7f572e000640 1 -- 172.21.15.32:0/3089888233 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57340ab650 con 0x559d43fe01b0 2024-08-31T07:29:20.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.432+0000 7f572e000640 1 -- 172.21.15.32:0/3089888233 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f57340094f0 con 0x559d43fe01b0 2024-08-31T07:29:20.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.432+0000 7f57519bc740 1 -- 172.21.15.32:0/3089888233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5730030450 con 0x559d43fe01b0 2024-08-31T07:29:20.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f572e000640 1 -- 172.21.15.32:0/3089888233 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f57340042f0 con 0x559d43fe01b0 2024-08-31T07:29:20.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f57519bc740 10 client.28781 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:20.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f57519bc740 1 -- 172.21.15.32:0/3089888233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x559d443c9360 con 0x559d43fe01b0 2024-08-31T07:29:20.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f572e000640 1 -- 172.21.15.32:0/3089888233 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5730030450 con 0x559d43fe01b0 2024-08-31T07:29:20.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f572e000640 1 -- 172.21.15.32:0/3089888233 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f57340b0020 con 0x559d43fe01b0 2024-08-31T07:29:20.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f57519bc740 10 client.28781 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:20.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f57519bc740 10 client.28781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:29:20.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f57519bc740 20 client.28781 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f57519bc740 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559d43fc79f0 0x559d443cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f57519bc740 4 client.28781 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.433+0000 7f57519bc740 1 -- 172.21.15.32:0/3089888233 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d443276c0 con 0x559d43fc79f0 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.434+0000 7f5743400640 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559d43fc79f0 0x559d443cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.434+0000 7f5743400640 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559d43fc79f0 0x559d443cc8d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88288572980281ce server_cookie=d36af9b9dab9c71e in_seq=0 out_seq=0 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.434+0000 7f572e000640 10 client.28781 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f572e000640 1 -- 172.21.15.32:0/3089888233 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d443276c0 con 0x559d43fc79f0 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f572e000640 10 client.28781 handle_command_reply: tid=0 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f57519bc740 1 client.28781 shutdown 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f57519bc740 1 -- 172.21.15.32:0/3089888233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 msgr2=0x559d443ba710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f57519bc740 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443ba710 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f57340abbe0 tx=0x7f57340092d0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f57519bc740 1 -- 172.21.15.32:0/3089888233 shutdown_connections 2024-08-31T07:29:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f57519bc740 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559d43fc79f0 0x559d443cc8d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f57519bc740 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d44009070 0x559d443c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f57519bc740 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 0x559d443bac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f57519bc740 1 --2- 172.21.15.32:0/3089888233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443ba710 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f57519bc740 1 -- 172.21.15.32:0/3089888233 >> 172.21.15.32:0/3089888233 conn(0x559d43ffd460 msgr2=0x559d443b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.435+0000 7f572e000640 0 client.28781 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.436+0000 7f57519bc740 1 -- 172.21.15.32:0/3089888233 shutdown_connections 2024-08-31T07:29:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.436+0000 7f57519bc740 1 -- 172.21.15.32:0/3089888233 wait complete. 2024-08-31T07:29:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.436+0000 7f57519bc740 20 client.28781 trim_cache size 0 max 16384 2024-08-31T07:29:20.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.454+0000 7f57519bc740 1 Processor -- start 2024-08-31T07:29:20.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.454+0000 7f57519bc740 1 -- start start 2024-08-31T07:29:20.456 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:20 smithi032 ceph-mon[17252]: pgmap v887: 97 pgs: 97 active+clean; 976 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-08-31T07:29:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f57519bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f57519bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 0x559d443c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f57519bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d44009070 0x559d443b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f57519bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f57441b13e0 con 0x559d44009070 2024-08-31T07:29:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f57519bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f57441b2f50 con 0x559d43fe01b0 2024-08-31T07:29:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f57519bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f57441b1ae0 con 0x559d43fe8390 2024-08-31T07:29:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5743e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5743400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 0x559d443c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5743e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443b5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48296/0 (socket says 172.21.15.32:48296) 2024-08-31T07:29:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5743e00640 1 -- 172.21.15.32:0/2597332803 learned_addr learned my addr 172.21.15.32:0/2597332803 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5748e00640 1 --2- 172.21.15.32:0/2597332803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d44009070 0x559d443b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5743400640 1 -- 172.21.15.32:0/2597332803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 msgr2=0x559d443b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5743400640 1 --2- 172.21.15.32:0/2597332803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5743400640 1 -- 172.21.15.32:0/2597332803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d44009070 msgr2=0x559d443b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5743400640 1 --2- 172.21.15.32:0/2597332803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d44009070 0x559d443b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5743400640 1 -- 172.21.15.32:0/2597332803 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d443bb0f0 con 0x559d43fe8390 2024-08-31T07:29:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.455+0000 7f5743e00640 1 --2- 172.21.15.32:0/2597332803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.456+0000 7f5743400640 1 --2- 172.21.15.32:0/2597332803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 0x559d443c6490 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5730003650 tx=0x7f573002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e2e80164501ead server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.456+0000 7f5738c00640 1 -- 172.21.15.32:0/2597332803 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730059070 con 0x559d43fe8390 2024-08-31T07:29:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.456+0000 7f5738c00640 1 -- 172.21.15.32:0/2597332803 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5730030650 con 0x559d43fe8390 2024-08-31T07:29:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.456+0000 7f5738c00640 1 -- 172.21.15.32:0/2597332803 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730041040 con 0x559d43fe8390 2024-08-31T07:29:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.457+0000 7f57519bc740 1 -- 172.21.15.32:0/2597332803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 msgr2=0x559d443c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.457+0000 7f57519bc740 1 --2- 172.21.15.32:0/2597332803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 0x559d443c6490 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5730003650 tx=0x7f573002a340 comp rx=0 tx=0).stop 2024-08-31T07:29:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.457+0000 7f57519bc740 1 -- 172.21.15.32:0/2597332803 shutdown_connections 2024-08-31T07:29:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.457+0000 7f57519bc740 1 --2- 172.21.15.32:0/2597332803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d44009070 0x559d443b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.457+0000 7f57519bc740 1 --2- 172.21.15.32:0/2597332803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d43fe8390 0x559d443c6490 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.457+0000 7f57519bc740 1 --2- 172.21.15.32:0/2597332803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.457+0000 7f57519bc740 1 -- 172.21.15.32:0/2597332803 >> 172.21.15.32:0/2597332803 conn(0x559d43ffd460 msgr2=0x559d443afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.458+0000 7f57519bc740 1 -- 172.21.15.32:0/2597332803 shutdown_connections 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.458+0000 7f57519bc740 1 -- 172.21.15.32:0/2597332803 wait complete. 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.458+0000 7f57519bc740 1 -- start start 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.458+0000 7f57519bc740 1 Processor -- start 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f57519bc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443bee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f57519bc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443bf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f57519bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d44009070 0x559d44430260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f57519bc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f57441b2880 con 0x559d43fe8390 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f57519bc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f57441b21b0 con 0x559d43fe01b0 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f57519bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f57441af3e0 con 0x559d44009070 2024-08-31T07:29:20.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f57519bc740 10 client.0 _ll_register_callbacks cb 0x7f57340046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f5743400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443bf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f5743e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f5743400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443bf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54096/0 (socket says 172.21.15.32:54096) 2024-08-31T07:29:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f5743400640 1 -- 172.21.15.32:0/3035900196 learned_addr learned my addr 172.21.15.32:0/3035900196 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.459+0000 7f5748e00640 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d44009070 0x559d44430260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.460+0000 7f5743400640 1 -- 172.21.15.32:0/3035900196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d44009070 msgr2=0x559d44430260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.460+0000 7f5743400640 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d44009070 0x559d44430260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.460+0000 7f5743400640 1 -- 172.21.15.32:0/3035900196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 msgr2=0x559d443bee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.460+0000 7f5743400640 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443bee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.460+0000 7f5743400640 1 -- 172.21.15.32:0/3035900196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d444308d0 con 0x559d43fe8390 2024-08-31T07:29:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.460+0000 7f5743e00640 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443bee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.460+0000 7f5743400640 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443bf410 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f573002a860 tx=0x7f5730000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9d04677ea79eca server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.460+0000 7f572e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:29:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.460+0000 7f572e000640 1 -- 172.21.15.32:0/3035900196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730059070 con 0x559d43fe8390 2024-08-31T07:29:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.461+0000 7f572e000640 1 -- 172.21.15.32:0/3035900196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f573003d480 con 0x559d43fe8390 2024-08-31T07:29:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.461+0000 7f57519bc740 1 -- 172.21.15.32:0/3035900196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5730030650 con 0x559d43fe8390 2024-08-31T07:29:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.461+0000 7f572e000640 1 -- 172.21.15.32:0/3035900196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f5730037040 con 0x559d43fe8390 2024-08-31T07:29:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.461+0000 7f57519bc740 10 client.18762 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.461+0000 7f57519bc740 1 -- 172.21.15.32:0/3035900196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x559d44437cb0 con 0x559d43fe8390 2024-08-31T07:29:20.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.461+0000 7f572e000640 1 -- 172.21.15.32:0/3035900196 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5730030650 con 0x559d43fe8390 2024-08-31T07:29:20.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.462+0000 7f572e000640 1 -- 172.21.15.32:0/3035900196 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f57300326b0 con 0x559d43fe8390 2024-08-31T07:29:20.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.462+0000 7f57519bc740 10 client.18762 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:20.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.462+0000 7f57519bc740 10 client.18762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:29:20.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.462+0000 7f57519bc740 20 client.18762 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:20.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.462+0000 7f57519bc740 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559d43f9cda0 0x559d44439f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:20.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.462+0000 7f57519bc740 4 client.18762 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:29:20.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.462+0000 7f57519bc740 1 -- 172.21.15.32:0/3035900196 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d44323160 con 0x559d43f9cda0 2024-08-31T07:29:20.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.462+0000 7f5743e00640 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559d43f9cda0 0x559d44439f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f5743e00640 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559d43f9cda0 0x559d44439f40 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8dd3367214b5be server_cookie=5cc2a2a3e97e5527 in_seq=0 out_seq=0 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f572e000640 10 client.18762 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f572e000640 1 -- 172.21.15.32:0/3035900196 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x559d44323160 con 0x559d43f9cda0 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f572e000640 10 client.18762 handle_command_reply: tid=0 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f57519bc740 1 client.18762 shutdown 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f57519bc740 1 -- 172.21.15.32:0/3035900196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 msgr2=0x559d443bf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f57519bc740 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443bf410 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f573002a860 tx=0x7f5730000bc0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f57519bc740 1 -- 172.21.15.32:0/3035900196 shutdown_connections 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f57519bc740 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x559d43f9cda0 0x559d44439f40 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f57519bc740 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d44009070 0x559d44430260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f572e000640 0 client.18762 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f57519bc740 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d43fe8390 0x559d443bf410 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f57519bc740 1 --2- 172.21.15.32:0/3035900196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559d43fe01b0 0x559d443bee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.463+0000 7f57519bc740 1 -- 172.21.15.32:0/3035900196 >> 172.21.15.32:0/3035900196 conn(0x559d43ffd460 msgr2=0x559d443bb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:20.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.464+0000 7f57519bc740 1 -- 172.21.15.32:0/3035900196 shutdown_connections 2024-08-31T07:29:20.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.464+0000 7f57519bc740 1 -- 172.21.15.32:0/3035900196 wait complete. 2024-08-31T07:29:20.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.464+0000 7f57519bc740 20 client.18762 trim_cache size 0 max 16384 2024-08-31T07:29:20.469 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:29:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:29:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:29:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:29:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:29:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:29:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:29:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:29:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:29:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 -- 172.21.15.32:0/2490977710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f57280b4ca0 msgr2=0x7f57280b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f57280b4ca0 0x7f57280b71b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f573c012c30 tx=0x7f573c009500 comp rx=0 tx=0).stop 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 -- 172.21.15.32:0/2490977710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5744105950 msgr2=0x7f574419c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5744105950 0x7f574419c2d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f573400e9c0 tx=0x7f573400edc0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 -- 172.21.15.32:0/2490977710 shutdown_connections 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f57280b4ca0 0x7f57280b71b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f574410bbe0 0x7f57441a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f574410b290 0x7f574419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 --2- 172.21.15.32:0/2490977710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5744105950 0x7f574419c2d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 -- 172.21.15.32:0/2490977710 >> 172.21.15.32:0/2490977710 conn(0x7f5744101420 msgr2=0x7f5744103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 -- 172.21.15.32:0/2490977710 shutdown_connections 2024-08-31T07:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:20.465+0000 7f5749800640 1 -- 172.21.15.32:0/2490977710 wait complete. 2024-08-31T07:29:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:20 smithi084 ceph-mon[22387]: pgmap v887: 97 pgs: 97 active+clean; 976 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-08-31T07:29:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:20 smithi073 ceph-mon[21995]: pgmap v887: 97 pgs: 97 active+clean; 976 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-08-31T07:29:21.348 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 7.95 MB/sec execute 61 sec latency 13134.356 ms 2024-08-31T07:29:21.431 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:29:21.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:29:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:22.288 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:22 smithi032 ceph-mon[17252]: pgmap v888: 97 pgs: 97 active+clean; 980 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-08-31T07:29:22.348 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 7.82 MB/sec execute 62 sec latency 14134.449 ms 2024-08-31T07:29:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.466+0000 7f99b2a00640 1 -- 172.21.15.32:0/522803921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac1099e0 msgr2=0x7f99ac110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.466+0000 7f99b2a00640 1 --2- 172.21.15.32:0/522803921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac1099e0 0x7f99ac110210 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009ca0 tx=0x7f99a402a8c0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.466+0000 7f99b2a00640 1 -- 172.21.15.32:0/522803921 shutdown_connections 2024-08-31T07:29:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.466+0000 7f99b2a00640 1 --2- 172.21.15.32:0/522803921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac1099e0 0x7f99ac110210 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.466+0000 7f99b2a00640 1 --2- 172.21.15.32:0/522803921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99ac109090 0x7f99ac1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.466+0000 7f99b2a00640 1 --2- 172.21.15.32:0/522803921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ac103750 0x7f99ac103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.466+0000 7f99b2a00640 1 -- 172.21.15.32:0/522803921 >> 172.21.15.32:0/522803921 conn(0x7f99ac0ff1e0 msgr2=0x7f99ac101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.467+0000 7f99b2a00640 1 -- 172.21.15.32:0/522803921 shutdown_connections 2024-08-31T07:29:22.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.467+0000 7f99b2a00640 1 -- 172.21.15.32:0/522803921 wait complete. 2024-08-31T07:29:22.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.467+0000 7f99b2a00640 1 Processor -- start 2024-08-31T07:29:22.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.468+0000 7f99b2a00640 1 -- start start 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.468+0000 7f99b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac103750 0x7f99ac19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.468+0000 7f99b2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99ac109090 0x7f99ac19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.468+0000 7f99b2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ac1099e0 0x7f99ac1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.468+0000 7f99b2a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac113700 con 0x7f99ac103750 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.468+0000 7f99b2a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac113580 con 0x7f99ac1099e0 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.468+0000 7f99b2a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac113880 con 0x7f99ac109090 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.468+0000 7f99b1600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac103750 0x7f99ac19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.468+0000 7f99b1600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac103750 0x7f99ac19c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54110/0 (socket says 172.21.15.32:54110) 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.468+0000 7f99b1600640 1 -- 172.21.15.32:0/797970139 learned_addr learned my addr 172.21.15.32:0/797970139 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.469+0000 7f99b2000640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ac1099e0 0x7f99ac1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.469+0000 7f99b0c00640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99ac109090 0x7f99ac19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.469+0000 7f99b1600640 1 -- 172.21.15.32:0/797970139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99ac109090 msgr2=0x7f99ac19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.469+0000 7f99b1600640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99ac109090 0x7f99ac19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.469+0000 7f99b1600640 1 -- 172.21.15.32:0/797970139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ac1099e0 msgr2=0x7f99ac1a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.469+0000 7f99b1600640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ac1099e0 0x7f99ac1a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.469+0000 7f99b1600640 1 -- 172.21.15.32:0/797970139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ac1a2200 con 0x7f99ac103750 2024-08-31T07:29:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.469+0000 7f99b1600640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac103750 0x7f99ac19c290 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e9f0 tx=0x7f99a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.470+0000 7f99aa000640 1 -- 172.21.15.32:0/797970139 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a000cd80 con 0x7f99ac103750 2024-08-31T07:29:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.470+0000 7f99b2a00640 1 -- 172.21.15.32:0/797970139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ac1a24f0 con 0x7f99ac103750 2024-08-31T07:29:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.470+0000 7f99aa000640 1 -- 172.21.15.32:0/797970139 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a00108e0 con 0x7f99ac103750 2024-08-31T07:29:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.470+0000 7f99aa000640 1 -- 172.21.15.32:0/797970139 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0018980 con 0x7f99ac103750 2024-08-31T07:29:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.470+0000 7f99b2a00640 1 -- 172.21.15.32:0/797970139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ac1a8c50 con 0x7f99ac103750 2024-08-31T07:29:22.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.471+0000 7f99b2a00640 1 -- 172.21.15.32:0/797970139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99ac103b60 con 0x7f99ac103750 2024-08-31T07:29:22.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.474+0000 7f99aa000640 1 -- 172.21.15.32:0/797970139 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f99a0010430 con 0x7f99ac103750 2024-08-31T07:29:22.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.475+0000 7f99aa000640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99940b4ca0 0x7f99940b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:22.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.475+0000 7f99aa000640 1 -- 172.21.15.32:0/797970139 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f99a00e5d50 con 0x7f99ac103750 2024-08-31T07:29:22.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.475+0000 7f99b0c00640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99940b4ca0 0x7f99940b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:22.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.476+0000 7f99b0c00640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99940b4ca0 0x7f99940b71b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f99ac109860 tx=0x7f9998005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:22.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.477+0000 7f99aa000640 1 -- 172.21.15.32:0/797970139 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f99a00ac4e0 con 0x7f99ac103750 2024-08-31T07:29:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:22 smithi084 ceph-mon[22387]: pgmap v888: 97 pgs: 97 active+clean; 980 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-08-31T07:29:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:22 smithi073 ceph-mon[21995]: pgmap v888: 97 pgs: 97 active+clean; 980 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-08-31T07:29:22.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.710+0000 7f99b2a00640 1 -- 172.21.15.32:0/797970139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99ac19d3c0 con 0x7f99ac103750 2024-08-31T07:29:22.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.711+0000 7f99aa000640 1 -- 172.21.15.32:0/797970139 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11961 (secure 0 0 0) 0x7f99a00b5070 con 0x7f99ac103750 2024-08-31T07:29:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:29:22.712 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"btime":"2024-08-31T07:29:10:429665+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":65,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:29:09.971782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:29:22.713 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 65 2024-08-31T07:29:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.713+0000 7f99b2a00640 1 -- 172.21.15.32:0/797970139 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99940b4ca0 msgr2=0x7f99940b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.713+0000 7f99b2a00640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99940b4ca0 0x7f99940b71b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f99ac109860 tx=0x7f9998005f90 comp rx=0 tx=0).stop 2024-08-31T07:29:22.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.713+0000 7f99b2a00640 1 -- 172.21.15.32:0/797970139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac103750 msgr2=0x7f99ac19c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.713+0000 7f99b2a00640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac103750 0x7f99ac19c290 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e9f0 tx=0x7f99a000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.714+0000 7f99b2a00640 1 -- 172.21.15.32:0/797970139 shutdown_connections 2024-08-31T07:29:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.714+0000 7f99b2a00640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99940b4ca0 0x7f99940b71b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.714+0000 7f99b2a00640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99ac1099e0 0x7f99ac1a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.714+0000 7f99b2a00640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99ac109090 0x7f99ac19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.714+0000 7f99b2a00640 1 --2- 172.21.15.32:0/797970139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac103750 0x7f99ac19c290 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.714+0000 7f99b2a00640 1 -- 172.21.15.32:0/797970139 >> 172.21.15.32:0/797970139 conn(0x7f99ac0ff1e0 msgr2=0x7f99ac1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.714+0000 7f99b2a00640 1 -- 172.21.15.32:0/797970139 shutdown_connections 2024-08-31T07:29:22.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:22.714+0000 7f99b2a00640 1 -- 172.21.15.32:0/797970139 wait complete. 2024-08-31T07:29:23.349 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 7.70 MB/sec execute 63 sec latency 15134.500 ms 2024-08-31T07:29:23.419 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:23 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/797970139' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:23.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:23 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/797970139' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:23.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:23 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/797970139' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:23.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:29:24.093 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:24.349 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 7.58 MB/sec execute 64 sec latency 16134.888 ms 2024-08-31T07:29:24.473 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:24 smithi032 ceph-mon[17252]: pgmap v889: 97 pgs: 97 active+clean; 979 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.3 MiB/s wr, 90 op/s 2024-08-31T07:29:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:24 smithi084 ceph-mon[22387]: pgmap v889: 97 pgs: 97 active+clean; 979 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.3 MiB/s wr, 90 op/s 2024-08-31T07:29:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:24 smithi073 ceph-mon[21995]: pgmap v889: 97 pgs: 97 active+clean; 979 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.3 MiB/s wr, 90 op/s 2024-08-31T07:29:24.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:29:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.973+0000 7f9277200640 1 -- 172.21.15.32:0/916387871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927810fcb0 msgr2=0x7f92781120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.973+0000 7f9277200640 1 --2- 172.21.15.32:0/916387871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927810fcb0 0x7f92781120f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9270009ca0 tx=0x7f927002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.973+0000 7f9277200640 1 -- 172.21.15.32:0/916387871 shutdown_connections 2024-08-31T07:29:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.973+0000 7f9277200640 1 --2- 172.21.15.32:0/916387871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927810fcb0 0x7f92781120f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.973+0000 7f9277200640 1 --2- 172.21.15.32:0/916387871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9278075ea0 0x7f927810f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.973+0000 7f9277200640 1 --2- 172.21.15.32:0/916387871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9278075550 0x7f9278075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.973+0000 7f9277200640 1 -- 172.21.15.32:0/916387871 >> 172.21.15.32:0/916387871 conn(0x7f92780ff1e0 msgr2=0x7f9278101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.973+0000 7f9277200640 1 -- 172.21.15.32:0/916387871 shutdown_connections 2024-08-31T07:29:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.974+0000 7f9277200640 1 -- 172.21.15.32:0/916387871 wait complete. 2024-08-31T07:29:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.975+0000 7f9277200640 1 Processor -- start 2024-08-31T07:29:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.975+0000 7f9277200640 1 -- start start 2024-08-31T07:29:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9277200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9278075550 0x7f927819c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9277200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9278075ea0 0x7f927819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9277200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927810fcb0 0x7f92781a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9277200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92781158b0 con 0x7f927810fcb0 2024-08-31T07:29:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9277200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9278115730 con 0x7f9278075ea0 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9277200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9278115a30 con 0x7f9278075550 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9275400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9278075ea0 0x7f927819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9275400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9278075ea0 0x7f927819c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:48348/0 (socket says 172.21.15.32:48348) 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9275400640 1 -- 172.21.15.32:0/3231351330 learned_addr learned my addr 172.21.15.32:0/3231351330 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9275e00640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9278075550 0x7f927819c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9276800640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927810fcb0 0x7f92781a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9275400640 1 -- 172.21.15.32:0/3231351330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9278075550 msgr2=0x7f927819c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9275400640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9278075550 0x7f927819c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9275400640 1 -- 172.21.15.32:0/3231351330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927810fcb0 msgr2=0x7f92781a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.976+0000 7f9275400640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927810fcb0 0x7f92781a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.977+0000 7f9275400640 1 -- 172.21.15.32:0/3231351330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92781a21b0 con 0x7f9278075ea0 2024-08-31T07:29:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.977+0000 7f9275400640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9278075ea0 0x7f927819c7e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f926400e9c0 tx=0x7f926400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.977+0000 7f926ea00640 1 -- 172.21.15.32:0/3231351330 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926400cd80 con 0x7f9278075ea0 2024-08-31T07:29:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.977+0000 7f926ea00640 1 -- 172.21.15.32:0/3231351330 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92640035f0 con 0x7f9278075ea0 2024-08-31T07:29:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.977+0000 7f9277200640 1 -- 172.21.15.32:0/3231351330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92781a24a0 con 0x7f9278075ea0 2024-08-31T07:29:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.977+0000 7f926ea00640 1 -- 172.21.15.32:0/3231351330 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264010690 con 0x7f9278075ea0 2024-08-31T07:29:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.977+0000 7f9277200640 1 -- 172.21.15.32:0/3231351330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92781a8c00 con 0x7f9278075ea0 2024-08-31T07:29:24.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.981+0000 7f92838a4740 1 Processor -- start 2024-08-31T07:29:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.981+0000 7f92838a4740 1 -- start start 2024-08-31T07:29:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.981+0000 7f926ea00640 1 -- 172.21.15.32:0/3231351330 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f92640108f0 con 0x7f9278075ea0 2024-08-31T07:29:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.981+0000 7f92838a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x559182886b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.981+0000 7f92838a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824b81b0 0x5591828870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f92838a4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 0x5591828875e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f92838a4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5591827fd7b0 con 0x5591824c0390 2024-08-31T07:29:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f92838a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5591827ecd40 con 0x5591824b81b0 2024-08-31T07:29:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f92838a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5591827f0cf0 con 0x5591824e1070 2024-08-31T07:29:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9276800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x559182886b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9275e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824b81b0 0x5591828870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9276800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x559182886b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:39508/0 (socket says 172.21.15.32:39508) 2024-08-31T07:29:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9276800640 1 -- 172.21.15.32:0/2097833672 learned_addr learned my addr 172.21.15.32:0/2097833672 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9275400640 1 --2- 172.21.15.32:0/2097833672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 0x5591828875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9276800640 1 -- 172.21.15.32:0/2097833672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824b81b0 msgr2=0x5591828870a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9276800640 1 --2- 172.21.15.32:0/2097833672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824b81b0 0x5591828870a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9276800640 1 -- 172.21.15.32:0/2097833672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 msgr2=0x5591828875e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9276800640 1 --2- 172.21.15.32:0/2097833672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 0x5591828875e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9276800640 1 -- 172.21.15.32:0/2097833672 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55918254c650 con 0x5591824e1070 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.982+0000 7f9275400640 1 --2- 172.21.15.32:0/2097833672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 0x5591828875e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.983+0000 7f9276800640 1 --2- 172.21.15.32:0/2097833672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x559182886b60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9270005e30 tx=0x7f9270009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=798b16014e78e5da server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.983+0000 7f9263000640 1 -- 172.21.15.32:0/2097833672 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270031cf0 con 0x5591824e1070 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.983+0000 7f926ea00640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f925c0b4ca0 0x7f925c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.983+0000 7f9263000640 1 -- 172.21.15.32:0/2097833672 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9270036040 con 0x5591824e1070 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.983+0000 7f9263000640 1 -- 172.21.15.32:0/2097833672 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700042a0 con 0x5591824e1070 2024-08-31T07:29:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.983+0000 7f926ea00640 1 -- 172.21.15.32:0/3231351330 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f92640e59f0 con 0x7f9278075ea0 2024-08-31T07:29:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.984+0000 7f9275e00640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f925c0b4ca0 0x7f925c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.984+0000 7f92838a4740 1 -- 172.21.15.32:0/2097833672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 msgr2=0x559182886b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.984+0000 7f92838a4740 1 --2- 172.21.15.32:0/2097833672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x559182886b60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9270005e30 tx=0x7f9270009460 comp rx=0 tx=0).stop 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.984+0000 7f92838a4740 1 -- 172.21.15.32:0/2097833672 shutdown_connections 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.984+0000 7f92838a4740 1 --2- 172.21.15.32:0/2097833672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 0x5591828875e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.984+0000 7f92838a4740 1 --2- 172.21.15.32:0/2097833672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824b81b0 0x5591828870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.984+0000 7f92838a4740 1 --2- 172.21.15.32:0/2097833672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x559182886b60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.984+0000 7f92838a4740 1 -- 172.21.15.32:0/2097833672 >> 172.21.15.32:0/2097833672 conn(0x5591824d5460 msgr2=0x559182884a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.985+0000 7f92838a4740 1 -- 172.21.15.32:0/2097833672 shutdown_connections 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.985+0000 7f92838a4740 1 -- 172.21.15.32:0/2097833672 wait complete. 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.985+0000 7f92838a4740 1 -- start start 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.985+0000 7f9275e00640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f925c0b4ca0 0x7f925c0b71b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x5591828857e0 tx=0x7f9268005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.985+0000 7f92838a4740 1 Processor -- start 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f92838a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824b81b0 0x559182892720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f92838a4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 0x559182892c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f92838a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918289e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f92838a4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55918264f700 con 0x5591824c0390 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f92838a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559182773cf0 con 0x5591824b81b0 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f92838a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5591828952e0 con 0x5591824e1070 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f92838a4740 10 client.0 _ll_register_callbacks cb 0x559182799470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f9276800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918289e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f9276800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918289e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:39522/0 (socket says 172.21.15.32:39522) 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f9276800640 1 -- 172.21.15.32:0/760021105 learned_addr learned my addr 172.21.15.32:0/760021105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.986+0000 7f9275400640 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 0x559182892c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.987+0000 7f9275e00640 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824b81b0 0x559182892720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.987+0000 7f9276800640 1 -- 172.21.15.32:0/760021105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824b81b0 msgr2=0x559182892720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.987+0000 7f9276800640 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824b81b0 0x559182892720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.987+0000 7f9276800640 1 -- 172.21.15.32:0/760021105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 msgr2=0x559182892c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.987+0000 7f9276800640 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 0x559182892c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.987+0000 7f9276800640 1 -- 172.21.15.32:0/760021105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591827b48b0 con 0x5591824e1070 2024-08-31T07:29:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.987+0000 7f9275400640 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 0x559182892c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:29:24.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.987+0000 7f9276800640 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918289e5d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9270003c30 tx=0x7f9270009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f41d8a2dae430630 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.987+0000 7f9257400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.987+0000 7f9257400640 1 -- 172.21.15.32:0/760021105 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270052020 con 0x5591824e1070 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.988+0000 7f92838a4740 1 -- 172.21.15.32:0/760021105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9270036040 con 0x5591824e1070 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.988+0000 7f9257400640 1 -- 172.21.15.32:0/760021105 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f9270031af0 con 0x5591824e1070 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.988+0000 7f9257400640 1 -- 172.21.15.32:0/760021105 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f927005d050 con 0x5591824e1070 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.988+0000 7f92838a4740 10 client.28813 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.988+0000 7f92838a4740 1 -- 172.21.15.32:0/760021105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5591828a1270 con 0x5591824e1070 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.988+0000 7f9257400640 1 -- 172.21.15.32:0/760021105 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9270036040 con 0x5591824e1070 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.989+0000 7f9257400640 1 -- 172.21.15.32:0/760021105 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f927004d030 con 0x5591824e1070 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.989+0000 7f92838a4740 10 client.28813 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.989+0000 7f92838a4740 10 client.28813 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.989+0000 7f92838a4740 20 client.28813 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:24.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.989+0000 7f92838a4740 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55918249f9f0 0x5591828a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.989+0000 7f92838a4740 4 client.28813 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.989+0000 7f92838a4740 1 -- 172.21.15.32:0/760021105 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591827ff700 con 0x55918249f9f0 2024-08-31T07:29:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.989+0000 7f9275e00640 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55918249f9f0 0x5591828a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.990+0000 7f9275e00640 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55918249f9f0 0x5591828a45d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a2646f1655fdbd9 server_cookie=4421b00eeed59c2c in_seq=0 out_seq=0 2024-08-31T07:29:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.990+0000 7f9257400640 10 client.28813 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:29:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.991+0000 7f9257400640 1 -- 172.21.15.32:0/760021105 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591827ff700 con 0x55918249f9f0 2024-08-31T07:29:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.991+0000 7f9257400640 10 client.28813 handle_command_reply: tid=0 2024-08-31T07:29:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.991+0000 7f92838a4740 1 client.28813 shutdown 2024-08-31T07:29:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.991+0000 7f92838a4740 1 -- 172.21.15.32:0/760021105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 msgr2=0x55918289e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.991+0000 7f92838a4740 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918289e5d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9270003c30 tx=0x7f9270009c50 comp rx=0 tx=0).stop 2024-08-31T07:29:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.992+0000 7f92838a4740 1 -- 172.21.15.32:0/760021105 shutdown_connections 2024-08-31T07:29:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.992+0000 7f92838a4740 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55918249f9f0 0x5591828a45d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.992+0000 7f92838a4740 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918289e5d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.992+0000 7f9257400640 0 client.28813 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:29:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.992+0000 7f92838a4740 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824c0390 0x559182892c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.992+0000 7f92838a4740 1 --2- 172.21.15.32:0/760021105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824b81b0 0x559182892720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.992+0000 7f92838a4740 1 -- 172.21.15.32:0/760021105 >> 172.21.15.32:0/760021105 conn(0x5591824d5460 msgr2=0x559182889cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.992+0000 7f92838a4740 1 -- 172.21.15.32:0/760021105 shutdown_connections 2024-08-31T07:29:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.992+0000 7f92838a4740 1 -- 172.21.15.32:0/760021105 wait complete. 2024-08-31T07:29:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:24.992+0000 7f92838a4740 20 client.28813 trim_cache size 0 max 16384 2024-08-31T07:29:25.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.008+0000 7f92838a4740 1 Processor -- start 2024-08-31T07:29:25.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.008+0000 7f92838a4740 1 -- start start 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.008+0000 7f92838a4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x559182884000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.008+0000 7f92838a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824c0390 0x559182884540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.008+0000 7f92838a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918288f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.008+0000 7f92838a4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b1450 con 0x5591824b81b0 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.008+0000 7f92838a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b2fc0 con 0x5591824c0390 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.008+0000 7f92838a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b1b50 con 0x5591824e1070 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9275e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x559182884000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9275e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x559182884000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54168/0 (socket says 172.21.15.32:54168) 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9275e00640 1 -- 172.21.15.32:0/730306945 learned_addr learned my addr 172.21.15.32:0/730306945 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9275400640 1 --2- 172.21.15.32:0/730306945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824c0390 0x559182884540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9276800640 1 --2- 172.21.15.32:0/730306945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918288f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:25.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9276800640 1 -- 172.21.15.32:0/730306945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824c0390 msgr2=0x559182884540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:25.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9276800640 1 --2- 172.21.15.32:0/730306945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824c0390 0x559182884540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9276800640 1 -- 172.21.15.32:0/730306945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 msgr2=0x559182884000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:25.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9276800640 1 --2- 172.21.15.32:0/730306945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x559182884000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9276800640 1 -- 172.21.15.32:0/730306945 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591828937e0 con 0x5591824e1070 2024-08-31T07:29:25.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.009+0000 7f9275e00640 1 --2- 172.21.15.32:0/730306945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x559182884000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:25.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.010+0000 7f9276800640 1 --2- 172.21.15.32:0/730306945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918288f730 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f927002fbb0 tx=0x7f927002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6568eeb2827c075d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:25.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.010+0000 7f9262600640 1 -- 172.21.15.32:0/730306945 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270052040 con 0x5591824e1070 2024-08-31T07:29:25.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.010+0000 7f9262600640 1 -- 172.21.15.32:0/730306945 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9270036040 con 0x5591824e1070 2024-08-31T07:29:25.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.010+0000 7f9262600640 1 -- 172.21.15.32:0/730306945 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927004a040 con 0x5591824e1070 2024-08-31T07:29:25.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.011+0000 7f92838a4740 1 -- 172.21.15.32:0/730306945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 msgr2=0x55918288f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:25.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.011+0000 7f92838a4740 1 --2- 172.21.15.32:0/730306945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918288f730 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f927002fbb0 tx=0x7f927002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.011+0000 7f92838a4740 1 -- 172.21.15.32:0/730306945 shutdown_connections 2024-08-31T07:29:25.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.011+0000 7f92838a4740 1 --2- 172.21.15.32:0/730306945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824e1070 0x55918288f730 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.011+0000 7f92838a4740 1 --2- 172.21.15.32:0/730306945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824c0390 0x559182884540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.011+0000 7f92838a4740 1 --2- 172.21.15.32:0/730306945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x559182884000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.011+0000 7f92838a4740 1 -- 172.21.15.32:0/730306945 >> 172.21.15.32:0/730306945 conn(0x5591824d5460 msgr2=0x559182890da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:25.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.011+0000 7f92838a4740 1 -- 172.21.15.32:0/730306945 shutdown_connections 2024-08-31T07:29:25.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.011+0000 7f92838a4740 1 -- 172.21.15.32:0/730306945 wait complete. 2024-08-31T07:29:25.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.012+0000 7f92838a4740 1 -- start start 2024-08-31T07:29:25.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.012+0000 7f92838a4740 1 Processor -- start 2024-08-31T07:29:25.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f92838a4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x5591828a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:25.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f92838a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824c0390 0x5591828a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f92838a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824e1070 0x559182907d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f92838a4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b28f0 con 0x5591824b81b0 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f92838a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b2220 con 0x5591824e1070 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f92838a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f92781af450 con 0x5591824c0390 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f92838a4740 10 client.0 _ll_register_callbacks cb 0x7f9270009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f9275e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x5591828a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f9275e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x5591828a5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54170/0 (socket says 172.21.15.32:54170) 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f9275400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824c0390 0x5591828a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f9275e00640 1 -- 172.21.15.32:0/1455022200 learned_addr learned my addr 172.21.15.32:0/1455022200 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.013+0000 7f9276800640 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824e1070 0x559182907d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.014+0000 7f9275e00640 1 -- 172.21.15.32:0/1455022200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824c0390 msgr2=0x5591828a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.014+0000 7f9275e00640 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824c0390 0x5591828a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.014+0000 7f9275e00640 1 -- 172.21.15.32:0/1455022200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824e1070 msgr2=0x559182907d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.014+0000 7f9275e00640 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824e1070 0x559182907d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.014+0000 7f9275e00640 1 -- 172.21.15.32:0/1455022200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559182908390 con 0x5591824b81b0 2024-08-31T07:29:25.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.014+0000 7f9276800640 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824e1070 0x559182907d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:25.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.014+0000 7f9275e00640 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x5591828a5f10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9268004630 tx=0x7f92680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a1274a43b1e7d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:25.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.014+0000 7f9257400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:29:25.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.015+0000 7f9257400640 1 -- 172.21.15.32:0/1455022200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268003b70 con 0x5591824b81b0 2024-08-31T07:29:25.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.015+0000 7f9257400640 1 -- 172.21.15.32:0/1455022200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f9268003d10 con 0x5591824b81b0 2024-08-31T07:29:25.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.015+0000 7f92838a4740 1 -- 172.21.15.32:0/1455022200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9270036040 con 0x5591824b81b0 2024-08-31T07:29:25.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.015+0000 7f9257400640 1 -- 172.21.15.32:0/1455022200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f9268003f10 con 0x5591824b81b0 2024-08-31T07:29:25.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.015+0000 7f92838a4740 10 client.18783 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:25.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.015+0000 7f92838a4740 1 -- 172.21.15.32:0/1455022200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55918290f720 con 0x5591824b81b0 2024-08-31T07:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.015+0000 7f9257400640 1 -- 172.21.15.32:0/1455022200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9270036040 con 0x5591824b81b0 2024-08-31T07:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.016+0000 7f9257400640 1 -- 172.21.15.32:0/1455022200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f9268031440 con 0x5591824b81b0 2024-08-31T07:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.016+0000 7f92838a4740 10 client.18783 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.016+0000 7f92838a4740 10 client.18783 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.016+0000 7f92838a4740 20 client.18783 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.016+0000 7f92838a4740 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x559182474da0 0x559182911f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.016+0000 7f92838a4740 4 client.18783 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.016+0000 7f92838a4740 1 -- 172.21.15.32:0/1455022200 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591827fb1a0 con 0x559182474da0 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.016+0000 7f9275400640 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x559182474da0 0x559182911f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.017+0000 7f9275400640 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x559182474da0 0x559182911f40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c756f814a0199063 server_cookie=221fdc32aae42466 in_seq=0 out_seq=0 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.017+0000 7f9257400640 10 client.18783 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.017+0000 7f9257400640 1 -- 172.21.15.32:0/1455022200 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591827fb1a0 con 0x559182474da0 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.017+0000 7f9257400640 10 client.18783 handle_command_reply: tid=0 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.017+0000 7f92838a4740 1 client.18783 shutdown 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.017+0000 7f92838a4740 1 -- 172.21.15.32:0/1455022200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 msgr2=0x5591828a5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.017+0000 7f92838a4740 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x5591828a5f10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9268004630 tx=0x7f92680046c0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.018+0000 7f92838a4740 1 -- 172.21.15.32:0/1455022200 shutdown_connections 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.018+0000 7f92838a4740 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x559182474da0 0x559182911f40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.018+0000 7f92838a4740 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591824e1070 0x559182907d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.018+0000 7f92838a4740 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591824c0390 0x5591828a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.018+0000 7f92838a4740 1 --2- 172.21.15.32:0/1455022200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591824b81b0 0x5591828a5f10 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.018+0000 7f92838a4740 1 -- 172.21.15.32:0/1455022200 >> 172.21.15.32:0/1455022200 conn(0x5591824d5460 msgr2=0x55918288b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.018+0000 7f9257400640 0 client.18783 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.018+0000 7f92838a4740 1 -- 172.21.15.32:0/1455022200 shutdown_connections 2024-08-31T07:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.018+0000 7f92838a4740 1 -- 172.21.15.32:0/1455022200 wait complete. 2024-08-31T07:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.018+0000 7f92838a4740 20 client.18783 trim_cache size 0 max 16384 2024-08-31T07:29:25.022 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 -- 172.21.15.32:0/3231351330 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f925c0b4ca0 msgr2=0x7f925c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f925c0b4ca0 0x7f925c0b71b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x5591828857e0 tx=0x7f9268005f90 comp rx=0 tx=0).stop 2024-08-31T07:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 -- 172.21.15.32:0/3231351330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9278075ea0 msgr2=0x7f927819c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9278075ea0 0x7f927819c7e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f926400e9c0 tx=0x7f926400ee80 comp rx=0 tx=0).stop 2024-08-31T07:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 -- 172.21.15.32:0/3231351330 shutdown_connections 2024-08-31T07:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f925c0b4ca0 0x7f925c0b71b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927810fcb0 0x7f92781a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9278075ea0 0x7f927819c7e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 --2- 172.21.15.32:0/3231351330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9278075550 0x7f927819c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 -- 172.21.15.32:0/3231351330 >> 172.21.15.32:0/3231351330 conn(0x7f92780ff1e0 msgr2=0x7f92781097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 -- 172.21.15.32:0/3231351330 shutdown_connections 2024-08-31T07:29:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:25.019+0000 7f9277200640 1 -- 172.21.15.32:0/3231351330 wait complete. 2024-08-31T07:29:25.320 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:25.349 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 7.46 MB/sec execute 65 sec latency 17134.964 ms 2024-08-31T07:29:26.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:29:26.349 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 7.35 MB/sec execute 66 sec latency 18135.030 ms 2024-08-31T07:29:26.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:26 smithi032 ceph-mon[17252]: pgmap v890: 97 pgs: 97 active+clean; 977 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 2.3 MiB/s wr, 92 op/s 2024-08-31T07:29:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:26 smithi084 ceph-mon[22387]: pgmap v890: 97 pgs: 97 active+clean; 977 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 2.3 MiB/s wr, 92 op/s 2024-08-31T07:29:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:26 smithi073 ceph-mon[21995]: pgmap v890: 97 pgs: 97 active+clean; 977 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 2.3 MiB/s wr, 92 op/s 2024-08-31T07:29:26.777 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.796+0000 7f6659400640 1 -- 172.21.15.32:0/3338999888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6654103900 msgr2=0x7f6654103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.796+0000 7f6659400640 1 --2- 172.21.15.32:0/3338999888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6654103900 0x7f6654103d10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6644009900 tx=0x7f664402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:29:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.797+0000 7f6659400640 1 -- 172.21.15.32:0/3338999888 shutdown_connections 2024-08-31T07:29:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.797+0000 7f6659400640 1 --2- 172.21.15.32:0/3338999888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6654109b90 0x7f66541103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.797+0000 7f6659400640 1 --2- 172.21.15.32:0/3338999888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6654109240 0x7f6654109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.797+0000 7f6659400640 1 --2- 172.21.15.32:0/3338999888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6654103900 0x7f6654103d10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.797+0000 7f6659400640 1 -- 172.21.15.32:0/3338999888 >> 172.21.15.32:0/3338999888 conn(0x7f665407b5f0 msgr2=0x7f665407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.797+0000 7f6659400640 1 -- 172.21.15.32:0/3338999888 shutdown_connections 2024-08-31T07:29:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.797+0000 7f6659400640 1 -- 172.21.15.32:0/3338999888 wait complete. 2024-08-31T07:29:26.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.798+0000 7f6659400640 1 Processor -- start 2024-08-31T07:29:26.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.798+0000 7f6659400640 1 -- start start 2024-08-31T07:29:26.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.798+0000 7f6659400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6654103900 0x7f665419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:26.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.798+0000 7f6659400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6654109240 0x7f665419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:26.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.798+0000 7f6659400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6654109b90 0x7f665419fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:26.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.798+0000 7f6659400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6654113850 con 0x7f6654109b90 2024-08-31T07:29:26.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.798+0000 7f6659400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f66541136d0 con 0x7f6654109240 2024-08-31T07:29:26.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.798+0000 7f6659400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f66541139d0 con 0x7f6654103900 2024-08-31T07:29:26.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6651e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6654109240 0x7f665419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:26.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6652800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6654103900 0x7f665419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:26.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6651e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6654109240 0x7f665419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50680/0 (socket says 172.21.15.32:50680) 2024-08-31T07:29:26.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6652800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6654103900 0x7f665419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40568/0 (socket says 172.21.15.32:40568) 2024-08-31T07:29:26.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6651e00640 1 -- 172.21.15.32:0/4063431212 learned_addr learned my addr 172.21.15.32:0/4063431212 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:26.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6653200640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6654109b90 0x7f665419fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:26.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6652800640 1 -- 172.21.15.32:0/4063431212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6654109240 msgr2=0x7f665419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:26.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6652800640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6654109240 0x7f665419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:26.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6652800640 1 -- 172.21.15.32:0/4063431212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6654109b90 msgr2=0x7f665419fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:26.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6652800640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6654109b90 0x7f665419fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:26.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.799+0000 7f6652800640 1 -- 172.21.15.32:0/4063431212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66541a0170 con 0x7f6654103900 2024-08-31T07:29:26.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.800+0000 7f6652800640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6654103900 0x7f665419a200 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f664402a910 tx=0x7f6644031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:26.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.800+0000 7f664b400640 1 -- 172.21.15.32:0/4063431212 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664403f030 con 0x7f6654103900 2024-08-31T07:29:26.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.800+0000 7f6659400640 1 -- 172.21.15.32:0/4063431212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66541a0400 con 0x7f6654103900 2024-08-31T07:29:26.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.800+0000 7f664b400640 1 -- 172.21.15.32:0/4063431212 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66440043d0 con 0x7f6654103900 2024-08-31T07:29:26.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.800+0000 7f664b400640 1 -- 172.21.15.32:0/4063431212 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644035070 con 0x7f6654103900 2024-08-31T07:29:26.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.800+0000 7f6659400640 1 -- 172.21.15.32:0/4063431212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6654075390 con 0x7f6654103900 2024-08-31T07:29:26.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.801+0000 7f6659400640 1 -- 172.21.15.32:0/4063431212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6654104b80 con 0x7f6654103900 2024-08-31T07:29:26.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.804+0000 7f664b400640 1 -- 172.21.15.32:0/4063431212 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6644042030 con 0x7f6654103900 2024-08-31T07:29:26.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.805+0000 7f664b400640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66380b4ca0 0x7f66380b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:26.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.806+0000 7f664b400640 1 -- 172.21.15.32:0/4063431212 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f66441059e0 con 0x7f6654103900 2024-08-31T07:29:26.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.806+0000 7f6651e00640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66380b4ca0 0x7f66380b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:26.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.807+0000 7f6651e00640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66380b4ca0 0x7f66380b71b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6654106260 tx=0x7f6640012040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:26.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:26.807+0000 7f664b400640 1 -- 172.21.15.32:0/4063431212 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f66440cc7c0 con 0x7f6654103900 2024-08-31T07:29:27.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.045+0000 7f6659400640 1 -- 172.21.15.32:0/4063431212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6654103d10 con 0x7f6654103900 2024-08-31T07:29:27.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.046+0000 7f664b400640 1 -- 172.21.15.32:0/4063431212 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11961 (secure 0 0 0) 0x7f66440cc0c0 con 0x7f6654103900 2024-08-31T07:29:27.048 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:29:27.048 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"btime":"2024-08-31T07:29:10:429665+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":65,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:29:09.971782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:29:27.048 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 65 2024-08-31T07:29:27.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 -- 172.21.15.32:0/4063431212 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66380b4ca0 msgr2=0x7f66380b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66380b4ca0 0x7f66380b71b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6654106260 tx=0x7f6640012040 comp rx=0 tx=0).stop 2024-08-31T07:29:27.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 -- 172.21.15.32:0/4063431212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6654103900 msgr2=0x7f665419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6654103900 0x7f665419a200 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f664402a910 tx=0x7f6644031a20 comp rx=0 tx=0).stop 2024-08-31T07:29:27.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 -- 172.21.15.32:0/4063431212 shutdown_connections 2024-08-31T07:29:27.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f66380b4ca0 0x7f66380b71b0 secure :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6654106260 tx=0x7f6640012040 comp rx=0 tx=0).stop 2024-08-31T07:29:27.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6654109b90 0x7f665419fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6654109240 0x7f665419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 --2- 172.21.15.32:0/4063431212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6654103900 0x7f665419a200 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 -- 172.21.15.32:0/4063431212 >> 172.21.15.32:0/4063431212 conn(0x7f665407b5f0 msgr2=0x7f66541072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:27.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 -- 172.21.15.32:0/4063431212 shutdown_connections 2024-08-31T07:29:27.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.049+0000 7f6659400640 1 -- 172.21.15.32:0/4063431212 wait complete. 2024-08-31T07:29:27.349 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 7.24 MB/sec execute 67 sec latency 19135.163 ms 2024-08-31T07:29:27.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:27 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4063431212' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:27.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:27 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4063431212' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.615+0000 7fe1d5600640 1 -- 172.21.15.32:0/4220926513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1d01099e0 msgr2=0x7fe1d0110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.615+0000 7fe1d5600640 1 --2- 172.21.15.32:0/4220926513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1d01099e0 0x7fe1d0110210 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009ca0 tx=0x7fe1c802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.615+0000 7fe1d5600640 1 -- 172.21.15.32:0/4220926513 shutdown_connections 2024-08-31T07:29:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.615+0000 7fe1d5600640 1 --2- 172.21.15.32:0/4220926513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1d01099e0 0x7fe1d0110210 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.615+0000 7fe1d5600640 1 --2- 172.21.15.32:0/4220926513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d0109090 0x7fe1d01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.615+0000 7fe1d5600640 1 --2- 172.21.15.32:0/4220926513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1d0103750 0x7fe1d0103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.615+0000 7fe1d5600640 1 -- 172.21.15.32:0/4220926513 >> 172.21.15.32:0/4220926513 conn(0x7fe1d00ff1a0 msgr2=0x7fe1d01015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.615+0000 7fe1d5600640 1 -- 172.21.15.32:0/4220926513 shutdown_connections 2024-08-31T07:29:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.615+0000 7fe1d5600640 1 -- 172.21.15.32:0/4220926513 wait complete. 2024-08-31T07:29:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.617+0000 7fe1d5600640 1 Processor -- start 2024-08-31T07:29:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.617+0000 7fe1d5600640 1 -- start start 2024-08-31T07:29:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.617+0000 7fe1d5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1d0103750 0x7fe1d019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.617+0000 7fe1d5600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1d0109090 0x7fe1d019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.617+0000 7fe1d5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d01099e0 0x7fe1d019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.617+0000 7fe1d5600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d0113690 con 0x7fe1d0103750 2024-08-31T07:29:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.617+0000 7fe1d5600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d0113510 con 0x7fe1d01099e0 2024-08-31T07:29:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.617+0000 7fe1d5600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d0113810 con 0x7fe1d0109090 2024-08-31T07:29:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1cf400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1d0109090 0x7fe1d019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1cfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1d0103750 0x7fe1d019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1d4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d01099e0 0x7fe1d019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1cf400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1d0109090 0x7fe1d019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40582/0 (socket says 172.21.15.32:40582) 2024-08-31T07:29:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1cf400640 1 -- 172.21.15.32:0/1658030311 learned_addr learned my addr 172.21.15.32:0/1658030311 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1cf400640 1 -- 172.21.15.32:0/1658030311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d01099e0 msgr2=0x7fe1d019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1cf400640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d01099e0 0x7fe1d019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1cf400640 1 -- 172.21.15.32:0/1658030311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1d0103750 msgr2=0x7fe1d019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1cf400640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1d0103750 0x7fe1d019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1cf400640 1 -- 172.21.15.32:0/1658030311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d019ff70 con 0x7fe1d0109090 2024-08-31T07:29:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.618+0000 7fe1cf400640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1d0109090 0x7fe1d019a5a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00e9f0 tx=0x7fe1bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.619+0000 7fe1ccc00640 1 -- 172.21.15.32:0/1658030311 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00ccc0 con 0x7fe1d0109090 2024-08-31T07:29:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.619+0000 7fe1d5600640 1 -- 172.21.15.32:0/1658030311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d01a0260 con 0x7fe1d0109090 2024-08-31T07:29:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.619+0000 7fe1ccc00640 1 -- 172.21.15.32:0/1658030311 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1bc0108e0 con 0x7fe1d0109090 2024-08-31T07:29:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.619+0000 7fe1ccc00640 1 -- 172.21.15.32:0/1658030311 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc018860 con 0x7fe1d0109090 2024-08-31T07:29:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.619+0000 7fe1d5600640 1 -- 172.21.15.32:0/1658030311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d01a69c0 con 0x7fe1d0109090 2024-08-31T07:29:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.622+0000 7fe1dd83e740 1 Processor -- start 2024-08-31T07:29:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.622+0000 7fe1dd83e740 1 -- start start 2024-08-31T07:29:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.622+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 0x55a8e03afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.622+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe11b0 0x55a8e03b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.622+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe9390 0x55a8e03b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.622+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a8e03267b0 con 0x55a8dffe11b0 2024-08-31T07:29:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.622+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a8e0315d40 con 0x55a8e000a070 2024-08-31T07:29:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.622+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a8e0319cf0 con 0x55a8dffe9390 2024-08-31T07:29:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1d4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 0x55a8e03afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1cfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe11b0 0x55a8e03b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1d4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 0x55a8e03afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50712/0 (socket says 172.21.15.32:50712) 2024-08-31T07:29:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1d4c00640 1 -- 172.21.15.32:0/892319670 learned_addr learned my addr 172.21.15.32:0/892319670 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1cfe00640 1 -- 172.21.15.32:0/892319670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe9390 msgr2=0x55a8e03b05e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:29:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/892319670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe9390 0x55a8e03b05e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1cfe00640 1 -- 172.21.15.32:0/892319670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 msgr2=0x55a8e03afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/892319670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 0x55a8e03afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1cfe00640 1 -- 172.21.15.32:0/892319670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e0075650 con 0x55a8dffe11b0 2024-08-31T07:29:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1ccc00640 1 -- 172.21.15.32:0/1658030311 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe1bc010430 con 0x7fe1d0109090 2024-08-31T07:29:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.623+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/892319670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe11b0 0x55a8e03b00a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009c20 tx=0x7fe1c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baba3f46c83a6c40 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.624+0000 7fe1c5200640 1 -- 172.21.15.32:0/892319670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0009dd0 con 0x55a8dffe11b0 2024-08-31T07:29:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.624+0000 7fe1c5200640 1 -- 172.21.15.32:0/892319670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c0005020 con 0x55a8dffe11b0 2024-08-31T07:29:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.624+0000 7fe1c5200640 1 -- 172.21.15.32:0/892319670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0004b20 con 0x55a8dffe11b0 2024-08-31T07:29:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.625+0000 7fe1dd83e740 1 -- 172.21.15.32:0/892319670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe11b0 msgr2=0x55a8e03b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.625+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/892319670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe11b0 0x55a8e03b00a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009c20 tx=0x7fe1c0011810 comp rx=0 tx=0).stop 2024-08-31T07:29:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.625+0000 7fe1dd83e740 1 -- 172.21.15.32:0/892319670 shutdown_connections 2024-08-31T07:29:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.625+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/892319670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe9390 0x55a8e03b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.625+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/892319670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe11b0 0x55a8e03b00a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.625+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/892319670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 0x55a8e03afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.625+0000 7fe1dd83e740 1 -- 172.21.15.32:0/892319670 >> 172.21.15.32:0/892319670 conn(0x55a8dfffe460 msgr2=0x55a8e03ada50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.625+0000 7fe1dd83e740 1 -- 172.21.15.32:0/892319670 shutdown_connections 2024-08-31T07:29:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.625+0000 7fe1dd83e740 1 -- 172.21.15.32:0/892319670 wait complete. 2024-08-31T07:29:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.626+0000 7fe1dd83e740 1 -- start start 2024-08-31T07:29:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.626+0000 7fe1ccc00640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1b40b4bc0 0x7fe1b40b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.626+0000 7fe1dd83e740 1 Processor -- start 2024-08-31T07:29:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.626+0000 7fe1ccc00640 1 -- 172.21.15.32:0/1658030311 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7fe1bc0e57e0 con 0x7fe1d0109090 2024-08-31T07:29:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.626+0000 7fe1d4c00640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1b40b4bc0 0x7fe1b40b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 0x55a8e03c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a8e0178700 con 0x55a8e000a070 2024-08-31T07:29:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a8e029ccf0 con 0x55a8dffe9390 2024-08-31T07:29:27.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a8e03be340 con 0x55a8dffe11b0 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1dd83e740 10 client.0 _ll_register_callbacks cb 0x55a8e02c2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1d4c00640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1b40b4bc0 0x7fe1b40b70d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80094e0 tx=0x7fe1c8005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1cf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1cf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50726/0 (socket says 172.21.15.32:50726) 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1cf400640 1 -- 172.21.15.32:0/1993823765 learned_addr learned my addr 172.21.15.32:0/1993823765 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.627+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1cf400640 1 -- 172.21.15.32:0/1993823765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 msgr2=0x55a8e03c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1d4c00640 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 0x55a8e03c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1cf400640 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1cf400640 1 -- 172.21.15.32:0/1993823765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 msgr2=0x55a8e03c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1cf400640 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 0x55a8e03c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1cf400640 1 -- 172.21.15.32:0/1993823765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e02dd8b0 con 0x55a8dffe9390 2024-08-31T07:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1d4c00640 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 0x55a8e03c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1cf400640 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c08d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0a7ff0 tx=0x7fe1bc0aba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be40ceeba8a16490 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1b9600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1b9600640 1 -- 172.21.15.32:0/1993823765 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0b51e0 con 0x55a8dffe9390 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1b9600640 1 -- 172.21.15.32:0/1993823765 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7fe1bc0b0340 con 0x55a8dffe9390 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.628+0000 7fe1dd83e740 1 -- 172.21.15.32:0/1993823765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c0005020 con 0x55a8dffe9390 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1b9600640 1 -- 172.21.15.32:0/1993823765 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe1bc097170 con 0x55a8dffe9390 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1dd83e740 10 client.28829 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1dd83e740 1 -- 172.21.15.32:0/1993823765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55a8e03ce840 con 0x55a8dffe9390 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1b9600640 1 -- 172.21.15.32:0/1993823765 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c0005020 con 0x55a8dffe9390 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1b9600640 1 -- 172.21.15.32:0/1993823765 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7fe1bc0a78e0 con 0x55a8dffe9390 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1dd83e740 10 client.28829 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1dd83e740 10 client.28829 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1dd83e740 20 client.28829 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a8dffc89f0 0x55a8e03d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1dd83e740 4 client.28829 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.629+0000 7fe1dd83e740 1 -- 172.21.15.32:0/1993823765 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8e0328700 con 0x55a8dffc89f0 2024-08-31T07:29:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.630+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a8dffc89f0 0x55a8e03d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.630+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a8dffc89f0 0x55a8e03d1d60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=940c0e4ff6d17687 server_cookie=5702144cb6fea51b in_seq=0 out_seq=0 2024-08-31T07:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.630+0000 7fe1b9600640 10 client.28829 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1b9600640 1 -- 172.21.15.32:0/1993823765 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8e0328700 con 0x55a8dffc89f0 2024-08-31T07:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1b9600640 10 client.28829 handle_command_reply: tid=0 2024-08-31T07:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1dd83e740 1 client.28829 shutdown 2024-08-31T07:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1dd83e740 1 -- 172.21.15.32:0/1993823765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 msgr2=0x55a8e03c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c08d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0a7ff0 tx=0x7fe1bc0aba80 comp rx=0 tx=0).stop 2024-08-31T07:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1dd83e740 1 -- 172.21.15.32:0/1993823765 shutdown_connections 2024-08-31T07:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a8dffc89f0 0x55a8e03d1d60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 0x55a8e03c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c08d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1b9600640 0 client.28829 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/1993823765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.631+0000 7fe1dd83e740 1 -- 172.21.15.32:0/1993823765 >> 172.21.15.32:0/1993823765 conn(0x55a8dfffe460 msgr2=0x55a8e03b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.632+0000 7fe1dd83e740 1 -- 172.21.15.32:0/1993823765 shutdown_connections 2024-08-31T07:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.632+0000 7fe1dd83e740 1 -- 172.21.15.32:0/1993823765 wait complete. 2024-08-31T07:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.632+0000 7fe1dd83e740 20 client.28829 trim_cache size 0 max 16384 2024-08-31T07:29:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.646+0000 7fe1dd83e740 1 Processor -- start 2024-08-31T07:29:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.646+0000 7fe1dd83e740 1 -- start start 2024-08-31T07:29:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.646+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.646+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.646+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 0x55a8e03c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.646+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01ad070 con 0x55a8e000a070 2024-08-31T07:29:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01aebe0 con 0x55a8dffe9390 2024-08-31T07:29:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01ad770 con 0x55a8dffe11b0 2024-08-31T07:29:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1cf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1d4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 0x55a8e03c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1cf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50736/0 (socket says 172.21.15.32:50736) 2024-08-31T07:29:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1cf400640 1 -- 172.21.15.32:0/88649564 learned_addr learned my addr 172.21.15.32:0/88649564 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/88649564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1cfe00640 1 -- 172.21.15.32:0/88649564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 msgr2=0x55a8e03c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/88649564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1cfe00640 1 -- 172.21.15.32:0/88649564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 msgr2=0x55a8e03c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/88649564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 0x55a8e03c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.647+0000 7fe1cfe00640 1 -- 172.21.15.32:0/88649564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e03b23f0 con 0x55a8dffe11b0 2024-08-31T07:29:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.648+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/88649564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c1d20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0000c00 tx=0x7fe1c0009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fddcfaa37657b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.648+0000 7fe1bbe00640 1 -- 172.21.15.32:0/88649564 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00055d0 con 0x55a8dffe11b0 2024-08-31T07:29:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.648+0000 7fe1bbe00640 1 -- 172.21.15.32:0/88649564 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c0003aa0 con 0x55a8dffe11b0 2024-08-31T07:29:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.648+0000 7fe1bbe00640 1 -- 172.21.15.32:0/88649564 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0024700 con 0x55a8dffe11b0 2024-08-31T07:29:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.649+0000 7fe1dd83e740 1 -- 172.21.15.32:0/88649564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 msgr2=0x55a8e03c1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.649+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/88649564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c1d20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0000c00 tx=0x7fe1c0009680 comp rx=0 tx=0).stop 2024-08-31T07:29:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.649+0000 7fe1dd83e740 1 -- 172.21.15.32:0/88649564 shutdown_connections 2024-08-31T07:29:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.649+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/88649564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8e000a070 0x55a8e03c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.649+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/88649564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8dffe9390 0x55a8e03c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.649+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/88649564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c1d20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.649+0000 7fe1dd83e740 1 -- 172.21.15.32:0/88649564 >> 172.21.15.32:0/88649564 conn(0x55a8dfffe460 msgr2=0x55a8e03c1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.649+0000 7fe1dd83e740 1 -- 172.21.15.32:0/88649564 shutdown_connections 2024-08-31T07:29:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.649+0000 7fe1dd83e740 1 -- 172.21.15.32:0/88649564 wait complete. 2024-08-31T07:29:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.650+0000 7fe1dd83e740 1 -- start start 2024-08-31T07:29:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.650+0000 7fe1dd83e740 1 Processor -- start 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.650+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.650+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe9390 0x55a8e03c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.650+0000 7fe1dd83e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 0x55a8e03cb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.650+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01ae510 con 0x55a8dffe9390 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.650+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01ade40 con 0x55a8e000a070 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.650+0000 7fe1dd83e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01ab070 con 0x55a8dffe11b0 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.650+0000 7fe1dd83e740 10 client.0 _ll_register_callbacks cb 0x7fe1bc0ab7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe9390 0x55a8e03c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe9390 0x55a8e03c1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60516/0 (socket says 172.21.15.32:60516) 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cf400640 1 -- 172.21.15.32:0/3988861916 learned_addr learned my addr 172.21.15.32:0/3988861916 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40608/0 (socket says 172.21.15.32:40608) 2024-08-31T07:29:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1d4c00640 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 0x55a8e03cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cfe00640 1 -- 172.21.15.32:0/3988861916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 msgr2=0x55a8e03cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 0x55a8e03cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cfe00640 1 -- 172.21.15.32:0/3988861916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe9390 msgr2=0x55a8e03c1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe9390 0x55a8e03c1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cfe00640 1 -- 172.21.15.32:0/3988861916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e03cbf10 con 0x55a8dffe11b0 2024-08-31T07:29:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1cfe00640 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c15d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009730 tx=0x7fe1c0003800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=613e5d2485b2460b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.651+0000 7fe1b9600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:29:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.652+0000 7fe1b9600640 1 -- 172.21.15.32:0/3988861916 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00055d0 con 0x55a8dffe11b0 2024-08-31T07:29:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.652+0000 7fe1dd83e740 1 -- 172.21.15.32:0/3988861916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c0003aa0 con 0x55a8dffe11b0 2024-08-31T07:29:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.652+0000 7fe1b9600640 1 -- 172.21.15.32:0/3988861916 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7fe1c0016070 con 0x55a8dffe11b0 2024-08-31T07:29:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.652+0000 7fe1b9600640 1 -- 172.21.15.32:0/3988861916 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe1c0005250 con 0x55a8dffe11b0 2024-08-31T07:29:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.652+0000 7fe1dd83e740 10 client.28840 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.652+0000 7fe1dd83e740 1 -- 172.21.15.32:0/3988861916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55a8e043cb80 con 0x55a8dffe11b0 2024-08-31T07:29:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.652+0000 7fe1b9600640 1 -- 172.21.15.32:0/3988861916 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c0003aa0 con 0x55a8dffe11b0 2024-08-31T07:29:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.652+0000 7fe1b9600640 1 -- 172.21.15.32:0/3988861916 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7fe1c0034070 con 0x55a8dffe11b0 2024-08-31T07:29:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.653+0000 7fe1dd83e740 10 client.28840 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.653+0000 7fe1dd83e740 10 client.28840 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.653+0000 7fe1dd83e740 20 client.28840 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.653+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a8dff9dda0 0x55a8e043f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.653+0000 7fe1dd83e740 4 client.28840 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.653+0000 7fe1dd83e740 1 -- 172.21.15.32:0/3988861916 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8e03241a0 con 0x55a8dff9dda0 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.653+0000 7fe1cf400640 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a8dff9dda0 0x55a8e043f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.653+0000 7fe1cf400640 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a8dff9dda0 0x55a8e043f150 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1392220972b7c server_cookie=4c9fb4a75f9d441d in_seq=0 out_seq=0 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.653+0000 7fe1b9600640 10 client.28840 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1b9600640 1 -- 172.21.15.32:0/3988861916 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8e03241a0 con 0x55a8dff9dda0 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1b9600640 10 client.28840 handle_command_reply: tid=0 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 client.28840 shutdown 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 -- 172.21.15.32:0/3988861916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 msgr2=0x55a8e03c15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c15d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009730 tx=0x7fe1c0003800 comp rx=0 tx=0).stop 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 -- 172.21.15.32:0/3988861916 shutdown_connections 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a8dff9dda0 0x55a8e043f150 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a8e000a070 0x55a8e03cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a8dffe9390 0x55a8e03c1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 --2- 172.21.15.32:0/3988861916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a8dffe11b0 0x55a8e03c15d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1b9600640 0 client.28840 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 -- 172.21.15.32:0/3988861916 >> 172.21.15.32:0/3988861916 conn(0x55a8dfffe460 msgr2=0x55a8e03b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 -- 172.21.15.32:0/3988861916 shutdown_connections 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 1 -- 172.21.15.32:0/3988861916 wait complete. 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.654+0000 7fe1dd83e740 20 client.28840 trim_cache size 0 max 16384 2024-08-31T07:29:27.658 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:29:27.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 -- 172.21.15.32:0/1658030311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1b40b4bc0 msgr2=0x7fe1b40b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1b40b4bc0 0x7fe1b40b70d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80094e0 tx=0x7fe1c8005fd0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 -- 172.21.15.32:0/1658030311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1d0109090 msgr2=0x7fe1d019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:27.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1d0109090 0x7fe1d019a5a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00e9f0 tx=0x7fe1bc00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 -- 172.21.15.32:0/1658030311 shutdown_connections 2024-08-31T07:29:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe1b40b4bc0 0x7fe1b40b70d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d01099e0 0x7fe1d019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1d0109090 0x7fe1d019a5a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 --2- 172.21.15.32:0/1658030311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1d0103750 0x7fe1d019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 -- 172.21.15.32:0/1658030311 >> 172.21.15.32:0/1658030311 conn(0x7fe1d00ff1a0 msgr2=0x7fe1d01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 -- 172.21.15.32:0/1658030311 shutdown_connections 2024-08-31T07:29:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:27.655+0000 7fe1d5600640 1 -- 172.21.15.32:0/1658030311 wait complete. 2024-08-31T07:29:27.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:27 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4063431212' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:29:28.349 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 7.13 MB/sec execute 68 sec latency 20135.282 ms 2024-08-31T07:29:28.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:28 smithi084 ceph-mon[22387]: pgmap v891: 97 pgs: 97 active+clean; 975 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.8 MiB/s wr, 83 op/s 2024-08-31T07:29:28.544 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:28 smithi032 ceph-mon[17252]: pgmap v891: 97 pgs: 97 active+clean; 975 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.8 MiB/s wr, 83 op/s 2024-08-31T07:29:28.591 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:28 smithi073 ceph-mon[21995]: pgmap v891: 97 pgs: 97 active+clean; 975 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.8 MiB/s wr, 83 op/s 2024-08-31T07:29:28.593 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 71721071 --release --await' 2024-08-31T07:29:28.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 71721071 --release --await 2024-08-31T07:29:29.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:29:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:29.349 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58914 7.03 MB/sec execute 69 sec latency 21135.400 ms 2024-08-31T07:29:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.032+0000 7f5bef200640 1 -- 172.21.15.32:0/970891261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be8103750 msgr2=0x7f5be8103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.032+0000 7f5bef200640 1 --2- 172.21.15.32:0/970891261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be8103750 0x7f5be8103b60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009900 tx=0x7f5bdc02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.032+0000 7f5bef200640 1 -- 172.21.15.32:0/970891261 shutdown_connections 2024-08-31T07:29:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.032+0000 7f5bef200640 1 --2- 172.21.15.32:0/970891261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5be81099e0 0x7f5be8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.032+0000 7f5bef200640 1 --2- 172.21.15.32:0/970891261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be8109090 0x7f5be81094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.032+0000 7f5bef200640 1 --2- 172.21.15.32:0/970891261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be8103750 0x7f5be8103b60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.032+0000 7f5bef200640 1 -- 172.21.15.32:0/970891261 >> 172.21.15.32:0/970891261 conn(0x7f5be80ff1e0 msgr2=0x7f5be8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.032+0000 7f5bef200640 1 -- 172.21.15.32:0/970891261 shutdown_connections 2024-08-31T07:29:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.032+0000 7f5bef200640 1 -- 172.21.15.32:0/970891261 wait complete. 2024-08-31T07:29:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.033+0000 7f5bef200640 1 Processor -- start 2024-08-31T07:29:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.033+0000 7f5bef200640 1 -- start start 2024-08-31T07:29:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.034+0000 7f5bef200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5be8103750 0x7f5be819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.034+0000 7f5bef200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be8109090 0x7f5be819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.034+0000 7f5bef200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be81099e0 0x7f5be81a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.034+0000 7f5bef200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8113690 con 0x7f5be8103750 2024-08-31T07:29:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.034+0000 7f5bef200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8113510 con 0x7f5be8109090 2024-08-31T07:29:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.034+0000 7f5bef200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8113810 con 0x7f5be81099e0 2024-08-31T07:29:30.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.034+0000 7f5bede00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5be8103750 0x7f5be819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.034+0000 7f5bede00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5be8103750 0x7f5be819c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60534/0 (socket says 172.21.15.32:60534) 2024-08-31T07:29:30.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.034+0000 7f5bede00640 1 -- 172.21.15.32:0/2869772457 learned_addr learned my addr 172.21.15.32:0/2869772457 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:30.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.034+0000 7f5bed400640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be8109090 0x7f5be819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5bede00640 1 -- 172.21.15.32:0/2869772457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be81099e0 msgr2=0x7f5be81a1b20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:29:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5bede00640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be81099e0 0x7f5be81a1b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5bede00640 1 -- 172.21.15.32:0/2869772457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be8109090 msgr2=0x7f5be819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5bede00640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be8109090 0x7f5be819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5bede00640 1 -- 172.21.15.32:0/2869772457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be81a2250 con 0x7f5be8103750 2024-08-31T07:29:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5bed400640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be8109090 0x7f5be819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5bede00640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5be8103750 0x7f5be819c2e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005940 tx=0x7f5bdc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5be2a00640 1 -- 172.21.15.32:0/2869772457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc03f030 con 0x7f5be8103750 2024-08-31T07:29:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5be2a00640 1 -- 172.21.15.32:0/2869772457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bdc0044c0 con 0x7f5be8103750 2024-08-31T07:29:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5be2a00640 1 -- 172.21.15.32:0/2869772457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc035070 con 0x7f5be8103750 2024-08-31T07:29:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.035+0000 7f5bef200640 1 -- 172.21.15.32:0/2869772457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be81a24e0 con 0x7f5be8103750 2024-08-31T07:29:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.036+0000 7f5bef200640 1 -- 172.21.15.32:0/2869772457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be81a8ca0 con 0x7f5be8103750 2024-08-31T07:29:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.039+0000 7f5bf747b740 1 Processor -- start 2024-08-31T07:29:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.040+0000 7f5bf747b740 1 -- start start 2024-08-31T07:29:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.040+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0daf9520 0x557a0dea78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.040+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 0x557a0dea7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.040+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0dad8840 0x557a0dea8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.040+0000 7f5bf747b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557a0de15a90 con 0x557a0daf9520 2024-08-31T07:29:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.040+0000 7f5bf747b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a0de05020 con 0x557a0dad0660 2024-08-31T07:29:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.040+0000 7f5bf747b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557a0de08fd0 con 0x557a0dad8840 2024-08-31T07:29:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.040+0000 7f5be2a00640 1 -- 172.21.15.32:0/2869772457 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5bdc042030 con 0x7f5be8103750 2024-08-31T07:29:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.040+0000 7f5bed400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0dad8840 0x557a0dea8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.041+0000 7f5bee800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0daf9520 0x557a0dea78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.041+0000 7f5bed400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0dad8840 0x557a0dea8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40632/0 (socket says 172.21.15.32:40632) 2024-08-31T07:29:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.041+0000 7f5bed400640 1 -- 172.21.15.32:0/3722336705 learned_addr learned my addr 172.21.15.32:0/3722336705 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.041+0000 7f5bede00640 1 --2- 172.21.15.32:0/3722336705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 0x557a0dea7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.041+0000 7f5bed400640 1 -- 172.21.15.32:0/3722336705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 msgr2=0x557a0dea7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.041+0000 7f5bed400640 1 --2- 172.21.15.32:0/3722336705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 0x557a0dea7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.041+0000 7f5bed400640 1 -- 172.21.15.32:0/3722336705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0daf9520 msgr2=0x557a0dea78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.041+0000 7f5bed400640 1 --2- 172.21.15.32:0/3722336705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0daf9520 0x557a0dea78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.041+0000 7f5bed400640 1 -- 172.21.15.32:0/3722336705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a0db64b00 con 0x557a0dad8840 2024-08-31T07:29:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.041+0000 7f5bed400640 1 --2- 172.21.15.32:0/3722336705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0dad8840 0x557a0dea8320 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8011470 tx=0x7f5bd8011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e45115377d61385 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.042+0000 7f5bd7000640 1 -- 172.21.15.32:0/3722336705 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd801e070 con 0x557a0dad8840 2024-08-31T07:29:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.042+0000 7f5bd7000640 1 -- 172.21.15.32:0/3722336705 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bd80035f0 con 0x557a0dad8840 2024-08-31T07:29:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.042+0000 7f5bd7000640 1 -- 172.21.15.32:0/3722336705 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800f860 con 0x557a0dad8840 2024-08-31T07:29:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.042+0000 7f5be2a00640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 0x7f5bd00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.043+0000 7f5be2a00640 1 -- 172.21.15.32:0/2869772457 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f5bdc105620 con 0x7f5be8103750 2024-08-31T07:29:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.043+0000 7f5bee800640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 0x7f5bd00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.043+0000 7f5bf747b740 1 -- 172.21.15.32:0/3722336705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0dad8840 msgr2=0x557a0dea8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:30.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.043+0000 7f5bf747b740 1 --2- 172.21.15.32:0/3722336705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0dad8840 0x557a0dea8320 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8011470 tx=0x7f5bd8011930 comp rx=0 tx=0).stop 2024-08-31T07:29:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.043+0000 7f5bf747b740 1 -- 172.21.15.32:0/3722336705 shutdown_connections 2024-08-31T07:29:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.043+0000 7f5bf747b740 1 --2- 172.21.15.32:0/3722336705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0dad8840 0x557a0dea8320 secure :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8011470 tx=0x7f5bd8011930 comp rx=0 tx=0).stop 2024-08-31T07:29:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.043+0000 7f5bf747b740 1 --2- 172.21.15.32:0/3722336705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 0x557a0dea7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.043+0000 7f5bf747b740 1 --2- 172.21.15.32:0/3722336705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0daf9520 0x557a0dea78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.043+0000 7f5bf747b740 1 -- 172.21.15.32:0/3722336705 >> 172.21.15.32:0/3722336705 conn(0x557a0daed910 msgr2=0x557a0dea5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.044+0000 7f5bf747b740 1 -- 172.21.15.32:0/3722336705 shutdown_connections 2024-08-31T07:29:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.044+0000 7f5bf747b740 1 -- 172.21.15.32:0/3722336705 wait complete. 2024-08-31T07:29:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.044+0000 7f5bee800640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 0x7f5bd00b71b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5be4005fd0 tx=0x7f5be4009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.044+0000 7f5bf747b740 1 -- start start 2024-08-31T07:29:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.045+0000 7f5bf747b740 1 Processor -- start 2024-08-31T07:29:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.045+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 0x557a0deb2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.045+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deb3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.045+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 0x557a0deb3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.045+0000 7f5bf747b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557a0dc67bb0 con 0x557a0dad0660 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.045+0000 7f5bf747b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557a0dd8bf90 con 0x557a0dab7ea0 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.045+0000 7f5bf747b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557a0deb6080 con 0x557a0daf9520 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.045+0000 7f5bf747b740 10 client.0 _ll_register_callbacks cb 0x557a0ddb1710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bed400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deb3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bede00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 0x557a0deb2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bed400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deb3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40640/0 (socket says 172.21.15.32:40640) 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bed400640 1 -- 172.21.15.32:0/595931153 learned_addr learned my addr 172.21.15.32:0/595931153 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bee800640 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 0x557a0deb3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bed400640 1 -- 172.21.15.32:0/595931153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 msgr2=0x557a0deb3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:30.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bed400640 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 0x557a0deb3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bed400640 1 -- 172.21.15.32:0/595931153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 msgr2=0x557a0deb2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bed400640 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 0x557a0deb2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bed400640 1 -- 172.21.15.32:0/595931153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a0dbdb960 con 0x557a0daf9520 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.046+0000 7f5bede00640 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 0x557a0deb2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.047+0000 7f5bed400640 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deb3250 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8011e30 tx=0x7f5bd8003d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17a0df4e26c6957 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.047+0000 7f5bcb400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.047+0000 7f5bcb400640 1 -- 172.21.15.32:0/595931153 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd801e040 con 0x557a0daf9520 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.047+0000 7f5bf747b740 1 -- 172.21.15.32:0/595931153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557a0dda3e30 con 0x557a0daf9520 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.047+0000 7f5bcb400640 1 -- 172.21.15.32:0/595931153 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f5bd80035f0 con 0x557a0daf9520 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.048+0000 7f5bcb400640 1 -- 172.21.15.32:0/595931153 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f5bd8032050 con 0x557a0daf9520 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.048+0000 7f5bf747b740 10 client.28849 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:30.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.048+0000 7f5bf747b740 1 -- 172.21.15.32:0/595931153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x557a0dec1b10 con 0x557a0daf9520 2024-08-31T07:29:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.048+0000 7f5bcb400640 1 -- 172.21.15.32:0/595931153 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557a0dda3e30 con 0x557a0daf9520 2024-08-31T07:29:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.048+0000 7f5bcb400640 1 -- 172.21.15.32:0/595931153 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f5bd8021030 con 0x557a0daf9520 2024-08-31T07:29:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.049+0000 7f5bf747b740 10 client.28849 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.049+0000 7f5bf747b740 10 client.28849 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:29:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.049+0000 7f5bf747b740 20 client.28849 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.049+0000 7f5bf747b740 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557a0da8d250 0x557a0dec4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.049+0000 7f5bf747b740 4 client.28849 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.049+0000 7f5bf747b740 1 -- 172.21.15.32:0/595931153 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a0de179e0 con 0x557a0da8d250 2024-08-31T07:29:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.049+0000 7f5bede00640 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557a0da8d250 0x557a0dec4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.050+0000 7f5bede00640 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557a0da8d250 0x557a0dec4e70 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d9e1ac47768b785 server_cookie=71047c36f6ddc912 in_seq=0 out_seq=0 2024-08-31T07:29:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.050+0000 7f5bcb400640 10 client.28849 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.050+0000 7f5bcb400640 1 -- 172.21.15.32:0/595931153 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a0de179e0 con 0x557a0da8d250 2024-08-31T07:29:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.050+0000 7f5bcb400640 10 client.28849 handle_command_reply: tid=0 2024-08-31T07:29:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 client.28849 shutdown 2024-08-31T07:29:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 -- 172.21.15.32:0/595931153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 msgr2=0x557a0deb3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deb3250 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8011e30 tx=0x7f5bd8003d90 comp rx=0 tx=0).stop 2024-08-31T07:29:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 -- 172.21.15.32:0/595931153 shutdown_connections 2024-08-31T07:29:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557a0da8d250 0x557a0dec4e70 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 0x557a0deb3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deb3250 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 --2- 172.21.15.32:0/595931153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 0x557a0deb2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bcb400640 0 client.28849 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 -- 172.21.15.32:0/595931153 >> 172.21.15.32:0/595931153 conn(0x557a0daed910 msgr2=0x557a0deaaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 -- 172.21.15.32:0/595931153 shutdown_connections 2024-08-31T07:29:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 1 -- 172.21.15.32:0/595931153 wait complete. 2024-08-31T07:29:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.051+0000 7f5bf747b740 20 client.28849 trim_cache size 0 max 16384 2024-08-31T07:29:30.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.070+0000 7f5bf747b740 1 Processor -- start 2024-08-31T07:29:30.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.070+0000 7f5bf747b740 1 -- start start 2024-08-31T07:29:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.070+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dab7ea0 0x557a0deb4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.070+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 0x557a0debf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.070+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deaff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.070+0000 7f5bf747b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5be81acf90 con 0x557a0dab7ea0 2024-08-31T07:29:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.070+0000 7f5bf747b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5be81aeb00 con 0x557a0dad0660 2024-08-31T07:29:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.070+0000 7f5bf747b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5be81ad690 con 0x557a0daf9520 2024-08-31T07:29:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.071+0000 7f5bede00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dab7ea0 0x557a0deb4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.071+0000 7f5bed400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 0x557a0debf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.071+0000 7f5bede00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dab7ea0 0x557a0deb4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60562/0 (socket says 172.21.15.32:60562) 2024-08-31T07:29:30.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.071+0000 7f5bede00640 1 -- 172.21.15.32:0/623088378 learned_addr learned my addr 172.21.15.32:0/623088378 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:30.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.071+0000 7f5bee800640 1 --2- 172.21.15.32:0/623088378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deaff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.071+0000 7f5bed400640 1 -- 172.21.15.32:0/623088378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 msgr2=0x557a0deaff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:30.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.071+0000 7f5bed400640 1 --2- 172.21.15.32:0/623088378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deaff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.071+0000 7f5bed400640 1 -- 172.21.15.32:0/623088378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dab7ea0 msgr2=0x557a0deb4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:30.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.071+0000 7f5bed400640 1 --2- 172.21.15.32:0/623088378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dab7ea0 0x557a0deb4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.071+0000 7f5bed400640 1 -- 172.21.15.32:0/623088378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a0dea5ca0 con 0x557a0dad0660 2024-08-31T07:29:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.072+0000 7f5bed400640 1 --2- 172.21.15.32:0/623088378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 0x557a0debf2a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009f30 tx=0x7f5bd8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=374e50b30fefa8c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.072+0000 7f5bd6600640 1 -- 172.21.15.32:0/623088378 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd801e070 con 0x557a0dad0660 2024-08-31T07:29:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.072+0000 7f5bd6600640 1 -- 172.21.15.32:0/623088378 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bd8029070 con 0x557a0dad0660 2024-08-31T07:29:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.072+0000 7f5bd6600640 1 -- 172.21.15.32:0/623088378 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8004dd0 con 0x557a0dad0660 2024-08-31T07:29:30.074 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.073+0000 7f5bf747b740 1 -- 172.21.15.32:0/623088378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 msgr2=0x557a0debf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.073+0000 7f5bf747b740 1 --2- 172.21.15.32:0/623088378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 0x557a0debf2a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009f30 tx=0x7f5bd8000bc0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.073+0000 7f5bf747b740 1 -- 172.21.15.32:0/623088378 shutdown_connections 2024-08-31T07:29:30.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.073+0000 7f5bf747b740 1 --2- 172.21.15.32:0/623088378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deaff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.073+0000 7f5bf747b740 1 --2- 172.21.15.32:0/623088378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dad0660 0x557a0debf2a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.073+0000 7f5bf747b740 1 --2- 172.21.15.32:0/623088378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dab7ea0 0x557a0deb4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.073+0000 7f5bf747b740 1 -- 172.21.15.32:0/623088378 >> 172.21.15.32:0/623088378 conn(0x557a0daed910 msgr2=0x557a0deb2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.073+0000 7f5bf747b740 1 -- 172.21.15.32:0/623088378 shutdown_connections 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.073+0000 7f5bf747b740 1 -- 172.21.15.32:0/623088378 wait complete. 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.074+0000 7f5bf747b740 1 -- start start 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.074+0000 7f5bf747b740 1 Processor -- start 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 0x557a0debbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 0x557a0deb7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bf747b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deb7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bf747b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5be81ae430 con 0x557a0dad0660 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bf747b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5be81add60 con 0x557a0dab7ea0 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bf747b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5be81aaff0 con 0x557a0daf9520 2024-08-31T07:29:30.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bf747b740 10 client.0 _ll_register_callbacks cb 0x7f5bd8009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:30.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bede00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 0x557a0debbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bee800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deb7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bed400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 0x557a0deb7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bede00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 0x557a0debbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50784/0 (socket says 172.21.15.32:50784) 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bede00640 1 -- 172.21.15.32:0/2040645967 learned_addr learned my addr 172.21.15.32:0/2040645967 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bede00640 1 -- 172.21.15.32:0/2040645967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 msgr2=0x557a0deb7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.075+0000 7f5bede00640 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deb7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.076+0000 7f5bede00640 1 -- 172.21.15.32:0/2040645967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 msgr2=0x557a0deb7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.076+0000 7f5bede00640 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 0x557a0deb7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.076+0000 7f5bede00640 1 -- 172.21.15.32:0/2040645967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a0df2e7f0 con 0x557a0dab7ea0 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.076+0000 7f5bed400640 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 0x557a0deb7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.076+0000 7f5bede00640 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 0x557a0debbd20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0cbc90 tx=0x7f5bdc0cc400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3568aff002a31a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.076+0000 7f5bcb400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.076+0000 7f5bcb400640 1 -- 172.21.15.32:0/2040645967 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc049040 con 0x557a0dab7ea0 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.076+0000 7f5bcb400640 1 -- 172.21.15.32:0/2040645967 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f5bdc0c7f90 con 0x557a0dab7ea0 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.076+0000 7f5bf747b740 1 -- 172.21.15.32:0/2040645967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bd8029070 con 0x557a0dab7ea0 2024-08-31T07:29:30.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bcb400640 1 -- 172.21.15.32:0/2040645967 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f5bdc0b7020 con 0x557a0dab7ea0 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bf747b740 10 client.28853 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bf747b740 1 -- 172.21.15.32:0/2040645967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x557a0df2f9b0 con 0x557a0dab7ea0 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bcb400640 1 -- 172.21.15.32:0/2040645967 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bd8029070 con 0x557a0dab7ea0 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bcb400640 1 -- 172.21.15.32:0/2040645967 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f5bdc04c060 con 0x557a0dab7ea0 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bf747b740 10 client.28853 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bf747b740 10 client.28853 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bf747b740 20 client.28853 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bf747b740 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557a0df300b0 0x557a0df324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bf747b740 4 client.28853 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "71721071", "release": true, "await": true}] 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.077+0000 7f5bf747b740 1 -- 172.21.15.32:0/2040645967 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "71721071", "release": true, "await": true}) -- 0x557a0de13480 con 0x557a0df300b0 2024-08-31T07:29:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.078+0000 7f5bed400640 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557a0df300b0 0x557a0df324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:30.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.078+0000 7f5bed400640 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557a0df300b0 0x557a0df324d0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c63c6c74b8fa5301 server_cookie=c2c3f8ea150909e9 in_seq=0 out_seq=0 2024-08-31T07:29:30.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.078+0000 7f5bcb400640 10 client.28853 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:30.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.118+0000 7f5bcb400640 1 -- 172.21.15.32:0/2040645967 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557a0de13480 con 0x557a0df300b0 2024-08-31T07:29:30.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.118+0000 7f5bcb400640 10 client.28853 handle_command_reply: tid=0 2024-08-31T07:29:30.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.118+0000 7f5bf747b740 1 client.28853 shutdown 2024-08-31T07:29:30.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.118+0000 7f5bf747b740 1 -- 172.21.15.32:0/2040645967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 msgr2=0x557a0debbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:30.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.118+0000 7f5bf747b740 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 0x557a0debbd20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0cbc90 tx=0x7f5bdc0cc400 comp rx=0 tx=0).stop 2024-08-31T07:29:30.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.119+0000 7f5bf747b740 1 -- 172.21.15.32:0/2040645967 shutdown_connections 2024-08-31T07:29:30.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.119+0000 7f5bf747b740 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557a0df300b0 0x557a0df324d0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.119+0000 7f5bf747b740 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557a0daf9520 0x557a0deb7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.119+0000 7f5bf747b740 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a0dad0660 0x557a0deb7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.119+0000 7f5bf747b740 1 --2- 172.21.15.32:0/2040645967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a0dab7ea0 0x557a0debbd20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.119+0000 7f5bf747b740 1 -- 172.21.15.32:0/2040645967 >> 172.21.15.32:0/2040645967 conn(0x557a0daed910 msgr2=0x557a0deb2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:30.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.119+0000 7f5bcb400640 0 client.28853 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:30.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.119+0000 7f5bf747b740 1 -- 172.21.15.32:0/2040645967 shutdown_connections 2024-08-31T07:29:30.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.120+0000 7f5bf747b740 1 -- 172.21.15.32:0/2040645967 wait complete. 2024-08-31T07:29:30.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.120+0000 7f5bf747b740 20 client.28853 trim_cache size 0 max 16384 2024-08-31T07:29:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:29:30.123 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 65, 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 56, 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "71721071": { 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 56, 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 141.2, 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:29:30.124 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:29:30.125 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:29:30.125 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:29:30.125 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:29:30.125 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:29:30.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:30.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:30.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:30.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:30.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:30.125 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.121+0000 7f5bef200640 1 -- 172.21.15.32:0/2869772457 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 msgr2=0x7f5bd00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.121+0000 7f5bef200640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 0x7f5bd00b71b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5be4005fd0 tx=0x7f5be4009500 comp rx=0 tx=0).stop 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.121+0000 7f5bef200640 1 -- 172.21.15.32:0/2869772457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5be8103750 msgr2=0x7f5be819c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.121+0000 7f5bef200640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5be8103750 0x7f5be819c2e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005940 tx=0x7f5bdc0040c0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.122+0000 7f5bef200640 1 -- 172.21.15.32:0/2869772457 shutdown_connections 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.122+0000 7f5bef200640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5bd00b4ca0 0x7f5bd00b71b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.122+0000 7f5bef200640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be81099e0 0x7f5be81a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.122+0000 7f5bef200640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be8109090 0x7f5be819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.122+0000 7f5bef200640 1 --2- 172.21.15.32:0/2869772457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5be8103750 0x7f5be819c2e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.122+0000 7f5bef200640 1 -- 172.21.15.32:0/2869772457 >> 172.21.15.32:0/2869772457 conn(0x7f5be80ff1e0 msgr2=0x7f5be810e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.122+0000 7f5bef200640 1 -- 172.21.15.32:0/2869772457 shutdown_connections 2024-08-31T07:29:30.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:30.122+0000 7f5bef200640 1 -- 172.21.15.32:0/2869772457 wait complete. 2024-08-31T07:29:30.261 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:30 smithi032 ceph-mon[17252]: pgmap v892: 97 pgs: 97 active+clean; 975 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.4 MiB/s wr, 68 op/s 2024-08-31T07:29:30.349 INFO:tasks.workunit.client.0.smithi032.stdout: 10 59104 7.08 MB/sec execute 70 sec latency 21887.065 ms 2024-08-31T07:29:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:30 smithi084 ceph-mon[22387]: pgmap v892: 97 pgs: 97 active+clean; 975 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.4 MiB/s wr, 68 op/s 2024-08-31T07:29:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:30 smithi073 ceph-mon[21995]: pgmap v892: 97 pgs: 97 active+clean; 975 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.4 MiB/s wr, 68 op/s 2024-08-31T07:29:31.350 INFO:tasks.workunit.client.0.smithi032.stdout: 10 60016 7.47 MB/sec execute 71 sec latency 52.265 ms 2024-08-31T07:29:31.615 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '71721071', total seconds elapsed: 28.2 2024-08-31T07:29:31.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.341+0000 7f013be00640 1 -- 172.21.15.32:0/832141465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f013c103750 msgr2=0x7f013c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.341+0000 7f013be00640 1 --2- 172.21.15.32:0/832141465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f013c103750 0x7f013c103b60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f012c009900 tx=0x7f012c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.341+0000 7f013be00640 1 -- 172.21.15.32:0/832141465 shutdown_connections 2024-08-31T07:29:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.341+0000 7f013be00640 1 --2- 172.21.15.32:0/832141465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013c1099e0 0x7f013c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.341+0000 7f013be00640 1 --2- 172.21.15.32:0/832141465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c109090 0x7f013c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.341+0000 7f013be00640 1 --2- 172.21.15.32:0/832141465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f013c103750 0x7f013c103b60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.341+0000 7f013be00640 1 -- 172.21.15.32:0/832141465 >> 172.21.15.32:0/832141465 conn(0x7f013c0ff1e0 msgr2=0x7f013c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.341+0000 7f013be00640 1 -- 172.21.15.32:0/832141465 shutdown_connections 2024-08-31T07:29:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.341+0000 7f013be00640 1 -- 172.21.15.32:0/832141465 wait complete. 2024-08-31T07:29:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.343+0000 7f013be00640 1 Processor -- start 2024-08-31T07:29:31.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.343+0000 7f013be00640 1 -- start start 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.343+0000 7f013be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f013c103750 0x7f013c19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c109090 0x7f013c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013c1099e0 0x7f013c1a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1136e0 con 0x7f013c109090 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f013c113560 con 0x7f013c103750 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f013c113860 con 0x7f013c1099e0 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f013c103750 0x7f013c19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f013c103750 0x7f013c19c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50810/0 (socket says 172.21.15.32:50810) 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013aa00640 1 -- 172.21.15.32:0/1535009815 learned_addr learned my addr 172.21.15.32:0/1535009815 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013a000640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c109090 0x7f013c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013b400640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013c1099e0 0x7f013c1a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013aa00640 1 -- 172.21.15.32:0/1535009815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013c1099e0 msgr2=0x7f013c1a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013aa00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013c1099e0 0x7f013c1a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013aa00640 1 -- 172.21.15.32:0/1535009815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c109090 msgr2=0x7f013c19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013aa00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c109090 0x7f013c19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.344+0000 7f013aa00640 1 -- 172.21.15.32:0/1535009815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c1a2240 con 0x7f013c103750 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.345+0000 7f013aa00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f013c103750 0x7f013c19c2d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f012c005ab0 tx=0x7f012c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.345+0000 7f0133400640 1 -- 172.21.15.32:0/1535009815 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c03f070 con 0x7f013c103750 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.346+0000 7f0133400640 1 -- 172.21.15.32:0/1535009815 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012c031b60 con 0x7f013c103750 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.346+0000 7f013be00640 1 -- 172.21.15.32:0/1535009815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c1a24d0 con 0x7f013c103750 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.346+0000 7f0133400640 1 -- 172.21.15.32:0/1535009815 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c035030 con 0x7f013c103750 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.346+0000 7f013be00640 1 -- 172.21.15.32:0/1535009815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c1a8c90 con 0x7f013c103750 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.352+0000 7f0148434740 1 Processor -- start 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.353+0000 7f0148434740 1 -- start start 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.353+0000 7f0148434740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc70cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.353+0000 7f0148434740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 0x555bbc70d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.353+0000 7f0148434740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc346390 0x555bbc70d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.353+0000 7f0148434740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555bbc6837b0 con 0x555bbc346390 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.353+0000 7f0148434740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555bbc672d40 con 0x555bbc33e1b0 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.353+0000 7f0148434740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555bbc676cf0 con 0x555bbc367070 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.354+0000 7f013b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc70cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.354+0000 7f013b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc70cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40698/0 (socket says 172.21.15.32:40698) 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.354+0000 7f013b400640 1 -- 172.21.15.32:0/3680164822 learned_addr learned my addr 172.21.15.32:0/3680164822 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.354+0000 7f013a000640 1 --2- 172.21.15.32:0/3680164822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc346390 0x555bbc70d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.354+0000 7f013b400640 1 -- 172.21.15.32:0/3680164822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 msgr2=0x555bbc70d0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.354+0000 7f013b400640 1 --2- 172.21.15.32:0/3680164822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 0x555bbc70d0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.354+0000 7f013b400640 1 -- 172.21.15.32:0/3680164822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc346390 msgr2=0x555bbc70d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:31.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.354+0000 7f0133400640 1 -- 172.21.15.32:0/1535009815 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f012c042070 con 0x7f013c103750 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.354+0000 7f013b400640 1 --2- 172.21.15.32:0/3680164822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc346390 0x555bbc70d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.355+0000 7f013b400640 1 -- 172.21.15.32:0/3680164822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bbc3d2650 con 0x555bbc367070 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.355+0000 7f013a000640 1 --2- 172.21.15.32:0/3680164822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc346390 0x555bbc70d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.355+0000 7f013b400640 1 --2- 172.21.15.32:0/3680164822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc70cb60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0134009fd0 tx=0x7f01340118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8b3de1d4ee6090 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.356+0000 7f0127e00640 1 -- 172.21.15.32:0/3680164822 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134004d20 con 0x555bbc367070 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.356+0000 7f0127e00640 1 -- 172.21.15.32:0/3680164822 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0134004050 con 0x555bbc367070 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.356+0000 7f0127e00640 1 -- 172.21.15.32:0/3680164822 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134005280 con 0x555bbc367070 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.357+0000 7f0133400640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01200b4d20 0x7f01200b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.357+0000 7f0133400640 1 -- 172.21.15.32:0/1535009815 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f012c105af0 con 0x7f013c103750 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.358+0000 7f0148434740 1 -- 172.21.15.32:0/3680164822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 msgr2=0x555bbc70cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.358+0000 7f0148434740 1 --2- 172.21.15.32:0/3680164822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc70cb60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0134009fd0 tx=0x7f01340118d0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.358+0000 7f0148434740 1 -- 172.21.15.32:0/3680164822 shutdown_connections 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.358+0000 7f0148434740 1 --2- 172.21.15.32:0/3680164822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc346390 0x555bbc70d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.358+0000 7f0148434740 1 --2- 172.21.15.32:0/3680164822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 0x555bbc70d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.358+0000 7f0148434740 1 --2- 172.21.15.32:0/3680164822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc70cb60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.358+0000 7f0148434740 1 -- 172.21.15.32:0/3680164822 >> 172.21.15.32:0/3680164822 conn(0x555bbc35b460 msgr2=0x555bbc70aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.358+0000 7f013aa00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01200b4d20 0x7f01200b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.359+0000 7f0148434740 1 -- 172.21.15.32:0/3680164822 shutdown_connections 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.360+0000 7f013aa00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01200b4d20 0x7f01200b7230 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x555bbc70b7e0 tx=0x7f012c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.360+0000 7f0148434740 1 -- 172.21.15.32:0/3680164822 wait complete. 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.361+0000 7f0148434740 1 -- start start 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.361+0000 7f0148434740 1 Processor -- start 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.362+0000 7f0148434740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc33e1b0 0x555bbc720f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.362+0000 7f0148434740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc346390 0x555bbc721460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.362+0000 7f0148434740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc72ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.362+0000 7f0148434740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555bbc4d5700 con 0x555bbc33e1b0 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.362+0000 7f0148434740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555bbc5f9cf0 con 0x555bbc346390 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.362+0000 7f0148434740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555bbc723a70 con 0x555bbc367070 2024-08-31T07:29:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.362+0000 7f0148434740 10 client.0 _ll_register_callbacks cb 0x555bbc61f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.363+0000 7f013a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc346390 0x555bbc721460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.363+0000 7f013a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc346390 0x555bbc721460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50816/0 (socket says 172.21.15.32:50816) 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.363+0000 7f013a000640 1 -- 172.21.15.32:0/1699349213 learned_addr learned my addr 172.21.15.32:0/1699349213 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.363+0000 7f013b400640 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc72ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.364+0000 7f013a000640 1 -- 172.21.15.32:0/1699349213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 msgr2=0x555bbc72ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.364+0000 7f013a000640 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc72ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.364+0000 7f013a000640 1 -- 172.21.15.32:0/1699349213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc33e1b0 msgr2=0x555bbc720f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.364+0000 7f013a000640 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc33e1b0 0x555bbc720f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.364+0000 7f013a000640 1 -- 172.21.15.32:0/1699349213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bbc63a8b0 con 0x555bbc346390 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.364+0000 7f013b400640 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc72ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.364+0000 7f013a000640 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc346390 0x555bbc721460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0128012700 tx=0x7f0128012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0390bee819c3693 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.364+0000 7f0124c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:29:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.364+0000 7f0124c00640 1 -- 172.21.15.32:0/1699349213 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128004050 con 0x555bbc346390 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.364+0000 7f0124c00640 1 -- 172.21.15.32:0/1699349213 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f01280041f0 con 0x555bbc346390 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.365+0000 7f0148434740 1 -- 172.21.15.32:0/1699349213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0134004050 con 0x555bbc346390 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.365+0000 7f0124c00640 1 -- 172.21.15.32:0/1699349213 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f01280043f0 con 0x555bbc346390 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.365+0000 7f0148434740 10 client.28862 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.365+0000 7f0148434740 1 -- 172.21.15.32:0/1699349213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x555bbc72fb80 con 0x555bbc346390 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.366+0000 7f0124c00640 1 -- 172.21.15.32:0/1699349213 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0134004050 con 0x555bbc346390 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.366+0000 7f0124c00640 1 -- 172.21.15.32:0/1699349213 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f0128003b90 con 0x555bbc346390 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.367+0000 7f0148434740 10 client.28862 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.367+0000 7f0148434740 10 client.28862 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.367+0000 7f0148434740 20 client.28862 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.367+0000 7f0148434740 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x555bbc3259f0 0x555bbc733010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.367+0000 7f0148434740 4 client.28862 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.367+0000 7f0148434740 1 -- 172.21.15.32:0/1699349213 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bbc685700 con 0x555bbc3259f0 2024-08-31T07:29:31.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.367+0000 7f013b400640 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x555bbc3259f0 0x555bbc733010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.368+0000 7f013b400640 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x555bbc3259f0 0x555bbc733010 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22713476e2927461 server_cookie=18a4a0675dabaf59 in_seq=0 out_seq=0 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.368+0000 7f0124c00640 10 client.28862 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.369+0000 7f0124c00640 1 -- 172.21.15.32:0/1699349213 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555bbc685700 con 0x555bbc3259f0 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.369+0000 7f0124c00640 10 client.28862 handle_command_reply: tid=0 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.369+0000 7f0148434740 1 client.28862 shutdown 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.370+0000 7f0148434740 1 -- 172.21.15.32:0/1699349213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc346390 msgr2=0x555bbc721460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.370+0000 7f0148434740 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc346390 0x555bbc721460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0128012700 tx=0x7f0128012bc0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.370+0000 7f0148434740 1 -- 172.21.15.32:0/1699349213 shutdown_connections 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.370+0000 7f0148434740 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x555bbc3259f0 0x555bbc733010 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.370+0000 7f0148434740 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc72ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.370+0000 7f0148434740 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc346390 0x555bbc721460 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.370+0000 7f0148434740 1 --2- 172.21.15.32:0/1699349213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc33e1b0 0x555bbc720f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.370+0000 7f0148434740 1 -- 172.21.15.32:0/1699349213 >> 172.21.15.32:0/1699349213 conn(0x555bbc35b460 msgr2=0x555bbc708ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.370+0000 7f0124c00640 0 client.28862 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.370+0000 7f0148434740 1 -- 172.21.15.32:0/1699349213 shutdown_connections 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.371+0000 7f0148434740 1 -- 172.21.15.32:0/1699349213 wait complete. 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.371+0000 7f0148434740 20 client.28862 trim_cache size 0 max 16384 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.397+0000 7f0148434740 1 Processor -- start 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.398+0000 7f0148434740 1 -- start start 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.398+0000 7f0148434740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc33e1b0 0x555bbc71be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.398+0000 7f0148434740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc346390 0x555bbc71c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.398+0000 7f0148434740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc367070 0x555bbc71f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.398+0000 7f0148434740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1ad0b0 con 0x555bbc33e1b0 2024-08-31T07:29:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.398+0000 7f0148434740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1aec20 con 0x555bbc367070 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.398+0000 7f0148434740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1ad7b0 con 0x555bbc346390 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.399+0000 7f013a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc346390 0x555bbc71c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.399+0000 7f013a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc346390 0x555bbc71c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40714/0 (socket says 172.21.15.32:40714) 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.399+0000 7f013b400640 1 --2- 172.21.15.32:0/2837908003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc367070 0x555bbc71f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.399+0000 7f013a000640 1 -- 172.21.15.32:0/2837908003 learned_addr learned my addr 172.21.15.32:0/2837908003 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.400+0000 7f013b400640 1 --2- 172.21.15.32:0/2837908003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc33e1b0 0x555bbc71be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.400+0000 7f013a000640 1 -- 172.21.15.32:0/2837908003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc367070 msgr2=0x555bbc71f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.400+0000 7f013a000640 1 --2- 172.21.15.32:0/2837908003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc367070 0x555bbc71f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.400+0000 7f013a000640 1 -- 172.21.15.32:0/2837908003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc33e1b0 msgr2=0x555bbc71be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.400+0000 7f013a000640 1 --2- 172.21.15.32:0/2837908003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc33e1b0 0x555bbc71be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.400+0000 7f013a000640 1 -- 172.21.15.32:0/2837908003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bbc70f970 con 0x555bbc346390 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.400+0000 7f013a000640 1 --2- 172.21.15.32:0/2837908003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc346390 0x555bbc71c3a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f01280126d0 tx=0x7f012800f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b80fbadb414913a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.401+0000 7f0127400640 1 -- 172.21.15.32:0/2837908003 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012802c070 con 0x555bbc346390 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.401+0000 7f0127400640 1 -- 172.21.15.32:0/2837908003 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0128010650 con 0x555bbc346390 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.401+0000 7f0127400640 1 -- 172.21.15.32:0/2837908003 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128010e00 con 0x555bbc346390 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.402+0000 7f0148434740 1 -- 172.21.15.32:0/2837908003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc346390 msgr2=0x555bbc71c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.402+0000 7f0148434740 1 --2- 172.21.15.32:0/2837908003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc346390 0x555bbc71c3a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f01280126d0 tx=0x7f012800f810 comp rx=0 tx=0).stop 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.403+0000 7f0148434740 1 -- 172.21.15.32:0/2837908003 shutdown_connections 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.403+0000 7f0148434740 1 --2- 172.21.15.32:0/2837908003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc367070 0x555bbc71f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.403+0000 7f0148434740 1 --2- 172.21.15.32:0/2837908003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc346390 0x555bbc71c3a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.403+0000 7f0148434740 1 --2- 172.21.15.32:0/2837908003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc33e1b0 0x555bbc71be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.403+0000 7f0148434740 1 -- 172.21.15.32:0/2837908003 >> 172.21.15.32:0/2837908003 conn(0x555bbc35b460 msgr2=0x555bbc720f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.404+0000 7f0148434740 1 -- 172.21.15.32:0/2837908003 shutdown_connections 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.404+0000 7f0148434740 1 -- 172.21.15.32:0/2837908003 wait complete. 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.405+0000 7f0148434740 1 -- start start 2024-08-31T07:29:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.405+0000 7f0148434740 1 Processor -- start 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.405+0000 7f0148434740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 0x555bbc729720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.405+0000 7f0148434740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc346390 0x555bbc71de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.405+0000 7f0148434740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc71e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.405+0000 7f0148434740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1ae550 con 0x555bbc346390 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.405+0000 7f0148434740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1ade80 con 0x555bbc33e1b0 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.405+0000 7f0148434740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1ab110 con 0x555bbc367070 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.406+0000 7f0148434740 10 client.0 _ll_register_callbacks cb 0x7f0128009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.406+0000 7f013b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 0x555bbc729720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.406+0000 7f013b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 0x555bbc729720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50826/0 (socket says 172.21.15.32:50826) 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.406+0000 7f013b400640 1 -- 172.21.15.32:0/3077443688 learned_addr learned my addr 172.21.15.32:0/3077443688 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.406+0000 7f013b400640 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc71e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.407+0000 7f013b400640 1 -- 172.21.15.32:0/3077443688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 msgr2=0x555bbc71e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.407+0000 7f013b400640 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc71e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.407+0000 7f013b400640 1 -- 172.21.15.32:0/3077443688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc346390 msgr2=0x555bbc71de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.407+0000 7f013b400640 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc346390 0x555bbc71de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.407+0000 7f013b400640 1 -- 172.21.15.32:0/3077443688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bbc728360 con 0x555bbc33e1b0 2024-08-31T07:29:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.407+0000 7f013b400640 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 0x555bbc729720 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0134011ed0 tx=0x7f0134009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5616aaa57be1633e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.407+0000 7f0124c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:29:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.407+0000 7f0124c00640 1 -- 172.21.15.32:0/3077443688 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01340049c0 con 0x555bbc33e1b0 2024-08-31T07:29:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.407+0000 7f0124c00640 1 -- 172.21.15.32:0/3077443688 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f013401f040 con 0x555bbc33e1b0 2024-08-31T07:29:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.408+0000 7f0148434740 1 -- 172.21.15.32:0/3077443688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0128010650 con 0x555bbc33e1b0 2024-08-31T07:29:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.408+0000 7f0124c00640 1 -- 172.21.15.32:0/3077443688 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f0134004000 con 0x555bbc33e1b0 2024-08-31T07:29:31.792 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:29:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.409+0000 7f0148434740 10 client.28868 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.409+0000 7f0148434740 1 -- 172.21.15.32:0/3077443688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x555bbc79db50 con 0x555bbc33e1b0 2024-08-31T07:29:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.409+0000 7f0124c00640 1 -- 172.21.15.32:0/3077443688 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0128010650 con 0x555bbc33e1b0 2024-08-31T07:29:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.409+0000 7f0124c00640 1 -- 172.21.15.32:0/3077443688 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f013400f990 con 0x555bbc33e1b0 2024-08-31T07:29:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.410+0000 7f0148434740 10 client.28868 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.410+0000 7f0148434740 10 client.28868 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:29:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.410+0000 7f0148434740 20 client.28868 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:31.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.410+0000 7f0148434740 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x555bbc2fada0 0x555bbc7a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.410+0000 7f0148434740 4 client.28868 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.410+0000 7f0148434740 1 -- 172.21.15.32:0/3077443688 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bbc6811a0 con 0x555bbc2fada0 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.410+0000 7f013a000640 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x555bbc2fada0 0x555bbc7a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.411+0000 7f0124c00640 10 client.28868 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.411+0000 7f013a000640 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x555bbc2fada0 0x555bbc7a00e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93b22c90de5177f5 server_cookie=7c48ad169d969561 in_seq=0 out_seq=0 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.413+0000 7f0124c00640 1 -- 172.21.15.32:0/3077443688 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bbc6811a0 con 0x555bbc2fada0 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.413+0000 7f0124c00640 10 client.28868 handle_command_reply: tid=0 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.413+0000 7f0148434740 1 client.28868 shutdown 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 1 -- 172.21.15.32:0/3077443688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 msgr2=0x555bbc729720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 0x555bbc729720 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0134011ed0 tx=0x7f0134009500 comp rx=0 tx=0).stop 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 1 -- 172.21.15.32:0/3077443688 shutdown_connections 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x555bbc2fada0 0x555bbc7a00e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555bbc367070 0x555bbc71e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555bbc346390 0x555bbc71de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 1 --2- 172.21.15.32:0/3077443688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bbc33e1b0 0x555bbc729720 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 1 -- 172.21.15.32:0/3077443688 >> 172.21.15.32:0/3077443688 conn(0x555bbc35b460 msgr2=0x555bbc720f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0124c00640 0 client.28868 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 1 -- 172.21.15.32:0/3077443688 shutdown_connections 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 1 -- 172.21.15.32:0/3077443688 wait complete. 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.414+0000 7f0148434740 20 client.28868 trim_cache size 0 max 16384 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 -- 172.21.15.32:0/1535009815 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01200b4d20 msgr2=0x7f01200b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01200b4d20 0x7f01200b7230 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x555bbc70b7e0 tx=0x7f012c009260 comp rx=0 tx=0).stop 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 -- 172.21.15.32:0/1535009815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f013c103750 msgr2=0x7f013c19c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f013c103750 0x7f013c19c2d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f012c005ab0 tx=0x7f012c003c10 comp rx=0 tx=0).stop 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 -- 172.21.15.32:0/1535009815 shutdown_connections 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f01200b4d20 0x7f01200b7230 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013c1099e0 0x7f013c1a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f013c109090 0x7f013c19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 --2- 172.21.15.32:0/1535009815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f013c103750 0x7f013c19c2d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:31.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 -- 172.21.15.32:0/1535009815 >> 172.21.15.32:0/1535009815 conn(0x7f013c0ff1e0 msgr2=0x7f013c108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:31.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.416+0000 7f013be00640 1 -- 172.21.15.32:0/1535009815 shutdown_connections 2024-08-31T07:29:31.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:31.417+0000 7f013be00640 1 -- 172.21.15.32:0/1535009815 wait complete. 2024-08-31T07:29:32.349 INFO:tasks.workunit.client.0.smithi032.stdout: 10 61321 8.32 MB/sec execute 72 sec latency 47.217 ms 2024-08-31T07:29:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:32 smithi084 ceph-mon[22387]: pgmap v893: 97 pgs: 97 active+clean; 975 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.1 MiB/s wr, 59 op/s 2024-08-31T07:29:32.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:32 smithi032 ceph-mon[17252]: pgmap v893: 97 pgs: 97 active+clean; 975 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.1 MiB/s wr, 59 op/s 2024-08-31T07:29:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:32 smithi073 ceph-mon[21995]: pgmap v893: 97 pgs: 97 active+clean; 975 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.1 MiB/s wr, 59 op/s 2024-08-31T07:29:32.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:29:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:33.350 INFO:tasks.workunit.client.0.smithi032.stdout: 10 62508 9.35 MB/sec execute 73 sec latency 57.586 ms 2024-08-31T07:29:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.179+0000 7f90d3e00640 1 -- 172.21.15.32:0/1184098654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90d40754d0 msgr2=0x7f90d40758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.179+0000 7f90d3e00640 1 --2- 172.21.15.32:0/1184098654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90d40754d0 0x7f90d40758e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009900 tx=0x7f90c402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.180+0000 7f90d3e00640 1 -- 172.21.15.32:0/1184098654 shutdown_connections 2024-08-31T07:29:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.180+0000 7f90d3e00640 1 --2- 172.21.15.32:0/1184098654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90d410fd30 0x7f90d4112150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.180+0000 7f90d3e00640 1 --2- 172.21.15.32:0/1184098654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d4075e20 0x7f90d410f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.180+0000 7f90d3e00640 1 --2- 172.21.15.32:0/1184098654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90d40754d0 0x7f90d40758e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.180+0000 7f90d3e00640 1 -- 172.21.15.32:0/1184098654 >> 172.21.15.32:0/1184098654 conn(0x7f90d40ff1a0 msgr2=0x7f90d41015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.180+0000 7f90d3e00640 1 -- 172.21.15.32:0/1184098654 shutdown_connections 2024-08-31T07:29:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.181+0000 7f90d3e00640 1 -- 172.21.15.32:0/1184098654 wait complete. 2024-08-31T07:29:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.182+0000 7f90d3e00640 1 Processor -- start 2024-08-31T07:29:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.182+0000 7f90d3e00640 1 -- start start 2024-08-31T07:29:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.183+0000 7f90d3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90d40754d0 0x7f90d419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.183+0000 7f90d3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90d4075e20 0x7f90d419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.183+0000 7f90d3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d410fd30 0x7f90d41a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.183+0000 7f90d3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4115890 con 0x7f90d410fd30 2024-08-31T07:29:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.183+0000 7f90d3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4115710 con 0x7f90d40754d0 2024-08-31T07:29:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.183+0000 7f90d3e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4115a10 con 0x7f90d4075e20 2024-08-31T07:29:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.183+0000 7f90d2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90d40754d0 0x7f90d419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.184+0000 7f90d2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90d40754d0 0x7f90d419c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50854/0 (socket says 172.21.15.32:50854) 2024-08-31T07:29:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.184+0000 7f90d2a00640 1 -- 172.21.15.32:0/3553476900 learned_addr learned my addr 172.21.15.32:0/3553476900 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.184+0000 7f90d3400640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d410fd30 0x7f90d41a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.184+0000 7f90d2a00640 1 -- 172.21.15.32:0/3553476900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90d4075e20 msgr2=0x7f90d419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.184+0000 7f90d2a00640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90d4075e20 0x7f90d419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.184+0000 7f90d2a00640 1 -- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d410fd30 msgr2=0x7f90d41a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.184+0000 7f90d2a00640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d410fd30 0x7f90d41a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.184+0000 7f90d2a00640 1 -- 172.21.15.32:0/3553476900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d41a21d0 con 0x7f90d40754d0 2024-08-31T07:29:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.184+0000 7f90d3400640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d410fd30 0x7f90d41a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:29:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.185+0000 7f90d2a00640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90d40754d0 0x7f90d419c260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f90c4005940 tx=0x7f90c40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.186+0000 7f90d8a00640 1 -- 172.21.15.32:0/3553476900 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c403f030 con 0x7f90d40754d0 2024-08-31T07:29:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.186+0000 7f90d8a00640 1 -- 172.21.15.32:0/3553476900 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c40044c0 con 0x7f90d40754d0 2024-08-31T07:29:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.186+0000 7f90d8a00640 1 -- 172.21.15.32:0/3553476900 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4035070 con 0x7f90d40754d0 2024-08-31T07:29:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.186+0000 7f90d3e00640 1 -- 172.21.15.32:0/3553476900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d41a2460 con 0x7f90d40754d0 2024-08-31T07:29:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.187+0000 7f90d3e00640 1 -- 172.21.15.32:0/3553476900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d41a8c20 con 0x7f90d40754d0 2024-08-31T07:29:34.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.192+0000 7f90e0a50740 1 Processor -- start 2024-08-31T07:29:34.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.193+0000 7f90d8a00640 1 -- 172.21.15.32:0/3553476900 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f90c4042030 con 0x7f90d40754d0 2024-08-31T07:29:34.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.194+0000 7f90e0a50740 1 -- start start 2024-08-31T07:29:34.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.195+0000 7f90d8a00640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f90b80b4d20 0x7f90b80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.195+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d53070 0x55dcb50f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.195+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d2a1b0 0x55dcb50f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.195+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.196+0000 7f90d8a00640 1 -- 172.21.15.32:0/3553476900 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f90c4105590 con 0x7f90d40754d0 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.196+0000 7f90e0a50740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dcb506f7b0 con 0x55dcb4d2a1b0 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.196+0000 7f90e0a50740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dcb505ed40 con 0x55dcb4d53070 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.196+0000 7f90e0a50740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dcb5062cf0 con 0x55dcb4d32390 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.196+0000 7f90d3400640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f90b80b4d20 0x7f90b80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.196+0000 7f90d2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d53070 0x55dcb50f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.197+0000 7f90d3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.197+0000 7f90d3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50f95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40754/0 (socket says 172.21.15.32:40754) 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.197+0000 7f90d3400640 1 -- 172.21.15.32:0/2837912064 learned_addr learned my addr 172.21.15.32:0/2837912064 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.197+0000 7f90d2a00640 1 -- 172.21.15.32:0/2837912064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 msgr2=0x55dcb50f95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.197+0000 7f90d2a00640 1 --2- 172.21.15.32:0/2837912064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50f95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.197+0000 7f90d2a00640 1 -- 172.21.15.32:0/2837912064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d2a1b0 msgr2=0x55dcb50f90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:34.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.198+0000 7f90d2000640 1 --2- 172.21.15.32:0/2837912064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d2a1b0 0x55dcb50f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.198+0000 7f90d2a00640 1 --2- 172.21.15.32:0/2837912064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d2a1b0 0x55dcb50f90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.198+0000 7f90d2a00640 1 -- 172.21.15.32:0/2837912064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcb4dbe650 con 0x55dcb4d53070 2024-08-31T07:29:34.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.198+0000 7f90d3400640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f90b80b4d20 0x7f90b80b7230 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f90cc014cf0 tx=0x7f90cc009760 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:34.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.199+0000 7f90d2000640 1 --2- 172.21.15.32:0/2837912064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d2a1b0 0x55dcb50f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:29:34.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.199+0000 7f90d2a00640 1 --2- 172.21.15.32:0/2837912064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d53070 0x55dcb50f8b60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f90c4102d30 tx=0x7f90c40cb440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ed54d13901aeba8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:34.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.199+0000 7f90bfe00640 1 -- 172.21.15.32:0/2837912064 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40cc3d0 con 0x55dcb4d53070 2024-08-31T07:29:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.200+0000 7f90bfe00640 1 -- 172.21.15.32:0/2837912064 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c40cbcd0 con 0x55dcb4d53070 2024-08-31T07:29:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.200+0000 7f90bfe00640 1 -- 172.21.15.32:0/2837912064 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40c7f60 con 0x55dcb4d53070 2024-08-31T07:29:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.201+0000 7f90e0a50740 1 -- 172.21.15.32:0/2837912064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d53070 msgr2=0x55dcb50f8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.201+0000 7f90e0a50740 1 --2- 172.21.15.32:0/2837912064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d53070 0x55dcb50f8b60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f90c4102d30 tx=0x7f90c40cb440 comp rx=0 tx=0).stop 2024-08-31T07:29:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.202+0000 7f90e0a50740 1 -- 172.21.15.32:0/2837912064 shutdown_connections 2024-08-31T07:29:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.202+0000 7f90e0a50740 1 --2- 172.21.15.32:0/2837912064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50f95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.202+0000 7f90e0a50740 1 --2- 172.21.15.32:0/2837912064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d2a1b0 0x55dcb50f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.202+0000 7f90e0a50740 1 --2- 172.21.15.32:0/2837912064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d53070 0x55dcb50f8b60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.202+0000 7f90e0a50740 1 -- 172.21.15.32:0/2837912064 >> 172.21.15.32:0/2837912064 conn(0x55dcb4d47460 msgr2=0x55dcb50f6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:34.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.202+0000 7f90e0a50740 1 -- 172.21.15.32:0/2837912064 shutdown_connections 2024-08-31T07:29:34.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.202+0000 7f90e0a50740 1 -- 172.21.15.32:0/2837912064 wait complete. 2024-08-31T07:29:34.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.203+0000 7f90e0a50740 1 -- start start 2024-08-31T07:29:34.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.203+0000 7f90e0a50740 1 Processor -- start 2024-08-31T07:29:34.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.204+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 0x55dcb51047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.204+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb5104d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.204+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.204+0000 7f90e0a50740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dcb4ec1700 con 0x55dcb4d53070 2024-08-31T07:29:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.204+0000 7f90e0a50740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dcb4fe5cf0 con 0x55dcb4d2a1b0 2024-08-31T07:29:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.204+0000 7f90e0a50740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dcb5107100 con 0x55dcb4d32390 2024-08-31T07:29:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.204+0000 7f90e0a50740 10 client.0 _ll_register_callbacks cb 0x55dcb500b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.205+0000 7f90d3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.205+0000 7f90d2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 0x55dcb51047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.205+0000 7f90d2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 0x55dcb51047e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50866/0 (socket says 172.21.15.32:50866) 2024-08-31T07:29:34.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.205+0000 7f90d2a00640 1 -- 172.21.15.32:0/361278079 learned_addr learned my addr 172.21.15.32:0/361278079 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:34.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.205+0000 7f90d2000640 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb5104d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.206+0000 7f90d3400640 1 -- 172.21.15.32:0/361278079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 msgr2=0x55dcb5104d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:34.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.206+0000 7f90d3400640 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb5104d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.206+0000 7f90d3400640 1 -- 172.21.15.32:0/361278079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 msgr2=0x55dcb51047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:34.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.206+0000 7f90d3400640 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 0x55dcb51047e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.206+0000 7f90d3400640 1 -- 172.21.15.32:0/361278079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcb50268b0 con 0x55dcb4d53070 2024-08-31T07:29:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.207+0000 7f90d3400640 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118b70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009810 tx=0x7f90cc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9818b2491a614ba8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.207+0000 7f90bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:29:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.207+0000 7f90bcc00640 1 -- 172.21.15.32:0/361278079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc026020 con 0x55dcb4d53070 2024-08-31T07:29:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.207+0000 7f90bcc00640 1 -- 172.21.15.32:0/361278079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f90cc004580 con 0x55dcb4d53070 2024-08-31T07:29:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.208+0000 7f90e0a50740 1 -- 172.21.15.32:0/361278079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c40cbcd0 con 0x55dcb4d53070 2024-08-31T07:29:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.208+0000 7f90bcc00640 1 -- 172.21.15.32:0/361278079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f90cc004780 con 0x55dcb4d53070 2024-08-31T07:29:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.208+0000 7f90e0a50740 10 client.18837 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.208+0000 7f90e0a50740 1 -- 172.21.15.32:0/361278079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55dcb511b870 con 0x55dcb4d53070 2024-08-31T07:29:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.209+0000 7f90bcc00640 1 -- 172.21.15.32:0/361278079 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c40cbcd0 con 0x55dcb4d53070 2024-08-31T07:29:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.211+0000 7f90bcc00640 1 -- 172.21.15.32:0/361278079 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f90cc012450 con 0x55dcb4d53070 2024-08-31T07:29:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.211+0000 7f90e0a50740 10 client.18837 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.211+0000 7f90e0a50740 10 client.18837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:29:34.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.211+0000 7f90e0a50740 20 client.18837 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:34.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.211+0000 7f90e0a50740 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55dcb4d119f0 0x55dcb511ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.211+0000 7f90e0a50740 4 client.18837 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.211+0000 7f90e0a50740 1 -- 172.21.15.32:0/361278079 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcb5071700 con 0x55dcb4d119f0 2024-08-31T07:29:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.211+0000 7f90d2a00640 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55dcb4d119f0 0x55dcb511ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.213+0000 7f90d2a00640 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55dcb4d119f0 0x55dcb511ed40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e9f5c55e65113ec server_cookie=f63ba1a730f93fa1 in_seq=0 out_seq=0 2024-08-31T07:29:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.213+0000 7f90bcc00640 10 client.18837 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:29:34.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.214+0000 7f90bcc00640 1 -- 172.21.15.32:0/361278079 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dcb5071700 con 0x55dcb4d119f0 2024-08-31T07:29:34.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.214+0000 7f90bcc00640 10 client.18837 handle_command_reply: tid=0 2024-08-31T07:29:34.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.214+0000 7f90e0a50740 1 client.18837 shutdown 2024-08-31T07:29:34.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.215+0000 7f90e0a50740 1 -- 172.21.15.32:0/361278079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 msgr2=0x55dcb5118b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:34.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.215+0000 7f90e0a50740 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118b70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009810 tx=0x7f90cc004ab0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.215+0000 7f90e0a50740 1 -- 172.21.15.32:0/361278079 shutdown_connections 2024-08-31T07:29:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.215+0000 7f90e0a50740 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118b70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.215+0000 7f90e0a50740 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb5104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.215+0000 7f90e0a50740 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55dcb4d119f0 0x55dcb511ed40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.215+0000 7f90e0a50740 1 --2- 172.21.15.32:0/361278079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 0x55dcb51047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.215+0000 7f90e0a50740 1 -- 172.21.15.32:0/361278079 >> 172.21.15.32:0/361278079 conn(0x55dcb4d47460 msgr2=0x55dcb50fbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.215+0000 7f90bcc00640 0 client.18837 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:29:34.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.216+0000 7f90e0a50740 1 -- 172.21.15.32:0/361278079 shutdown_connections 2024-08-31T07:29:34.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.216+0000 7f90e0a50740 1 -- 172.21.15.32:0/361278079 wait complete. 2024-08-31T07:29:34.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.216+0000 7f90e0a50740 20 client.18837 trim_cache size 0 max 16384 2024-08-31T07:29:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.241+0000 7f90e0a50740 1 Processor -- start 2024-08-31T07:29:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.241+0000 7f90e0a50740 1 -- start start 2024-08-31T07:29:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.241+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 0x55dcb51064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.241+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.241+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.241+0000 7f90e0a50740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41ad160 con 0x55dcb4d53070 2024-08-31T07:29:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.241+0000 7f90e0a50740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41aecd0 con 0x55dcb4d2a1b0 2024-08-31T07:29:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.241+0000 7f90e0a50740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41ad860 con 0x55dcb4d32390 2024-08-31T07:29:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.242+0000 7f90d3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.242+0000 7f90d2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 0x55dcb51064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.242+0000 7f90d2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.242+0000 7f90d2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50fa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40780/0 (socket says 172.21.15.32:40780) 2024-08-31T07:29:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.242+0000 7f90d2000640 1 -- 172.21.15.32:0/2137949720 learned_addr learned my addr 172.21.15.32:0/2137949720 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.242+0000 7f90d3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60656/0 (socket says 172.21.15.32:60656) 2024-08-31T07:29:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.243+0000 7f90d2000640 1 -- 172.21.15.32:0/2137949720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 msgr2=0x55dcb51064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.243+0000 7f90d2000640 1 --2- 172.21.15.32:0/2137949720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 0x55dcb51064c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.243+0000 7f90d2000640 1 -- 172.21.15.32:0/2137949720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 msgr2=0x55dcb5118af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.243+0000 7f90d2000640 1 --2- 172.21.15.32:0/2137949720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.243+0000 7f90d2000640 1 -- 172.21.15.32:0/2137949720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcb511f0b0 con 0x55dcb4d32390 2024-08-31T07:29:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.243+0000 7f90d3400640 1 --2- 172.21.15.32:0/2137949720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:29:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.243+0000 7f90d2000640 1 --2- 172.21.15.32:0/2137949720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50fa2a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009b70 tx=0x7f90c00127c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae0336ac71ead30c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.244+0000 7f90bf400640 1 -- 172.21.15.32:0/2137949720 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0009e60 con 0x55dcb4d32390 2024-08-31T07:29:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.244+0000 7f90bf400640 1 -- 172.21.15.32:0/2137949720 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c001a890 con 0x55dcb4d32390 2024-08-31T07:29:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.244+0000 7f90bf400640 1 -- 172.21.15.32:0/2137949720 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c000e9a0 con 0x55dcb4d32390 2024-08-31T07:29:34.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.245+0000 7f90e0a50740 1 -- 172.21.15.32:0/2137949720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 msgr2=0x55dcb50fa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:34.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.245+0000 7f90e0a50740 1 --2- 172.21.15.32:0/2137949720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50fa2a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009b70 tx=0x7f90c00127c0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.246+0000 7f90e0a50740 1 -- 172.21.15.32:0/2137949720 shutdown_connections 2024-08-31T07:29:34.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.246+0000 7f90e0a50740 1 --2- 172.21.15.32:0/2137949720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb5118af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.246+0000 7f90e0a50740 1 --2- 172.21.15.32:0/2137949720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d32390 0x55dcb50fa2a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.246+0000 7f90e0a50740 1 --2- 172.21.15.32:0/2137949720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d2a1b0 0x55dcb51064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.246+0000 7f90e0a50740 1 -- 172.21.15.32:0/2137949720 >> 172.21.15.32:0/2137949720 conn(0x55dcb4d47460 msgr2=0x55dcb50f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:34.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.246+0000 7f90e0a50740 1 -- 172.21.15.32:0/2137949720 shutdown_connections 2024-08-31T07:29:34.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.246+0000 7f90e0a50740 1 -- 172.21.15.32:0/2137949720 wait complete. 2024-08-31T07:29:34.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.247+0000 7f90e0a50740 1 -- start start 2024-08-31T07:29:34.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.247+0000 7f90e0a50740 1 Processor -- start 2024-08-31T07:29:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d2a1b0 0x55dcb5111310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d32390 0x55dcb5111870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90e0a50740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb51886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90e0a50740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41ae600 con 0x55dcb4d53070 2024-08-31T07:29:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90e0a50740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41adf30 con 0x55dcb4d32390 2024-08-31T07:29:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90e0a50740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41ab1a0 con 0x55dcb4d2a1b0 2024-08-31T07:29:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90e0a50740 10 client.0 _ll_register_callbacks cb 0x7f90cc014de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90d2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d2a1b0 0x55dcb5111310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90d2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d2a1b0 0x55dcb5111310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:40792/0 (socket says 172.21.15.32:40792) 2024-08-31T07:29:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90d2a00640 1 -- 172.21.15.32:0/3614861898 learned_addr learned my addr 172.21.15.32:0/3614861898 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.248+0000 7f90d2000640 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d32390 0x55dcb5111870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.249+0000 7f90d3400640 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb51886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.249+0000 7f90d2000640 1 -- 172.21.15.32:0/3614861898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d2a1b0 msgr2=0x55dcb5111310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.249+0000 7f90d2000640 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d2a1b0 0x55dcb5111310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.249+0000 7f90d2000640 1 -- 172.21.15.32:0/3614861898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 msgr2=0x55dcb51886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.249+0000 7f90d2000640 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb51886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.249+0000 7f90d2000640 1 -- 172.21.15.32:0/3614861898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcb5188ca0 con 0x55dcb4d32390 2024-08-31T07:29:34.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.249+0000 7f90d3400640 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb51886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:29:34.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.249+0000 7f90d2000640 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d32390 0x55dcb5111870 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e5c0 tx=0x7f90c001a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab520c6982144feb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:34.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.250+0000 7f90bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:29:34.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.250+0000 7f90bcc00640 1 -- 172.21.15.32:0/3614861898 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0028020 con 0x55dcb4d32390 2024-08-31T07:29:34.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.250+0000 7f90bcc00640 1 -- 172.21.15.32:0/3614861898 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f90c001aa40 con 0x55dcb4d32390 2024-08-31T07:29:34.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.251+0000 7f90e0a50740 1 -- 172.21.15.32:0/3614861898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c001a890 con 0x55dcb4d32390 2024-08-31T07:29:34.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.251+0000 7f90bcc00640 1 -- 172.21.15.32:0/3614861898 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f90c001ac40 con 0x55dcb4d32390 2024-08-31T07:29:34.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.251+0000 7f90e0a50740 10 client.28886 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:34.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.251+0000 7f90e0a50740 1 -- 172.21.15.32:0/3614861898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55dcb5189f20 con 0x55dcb4d32390 2024-08-31T07:29:34.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.252+0000 7f90bcc00640 1 -- 172.21.15.32:0/3614861898 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c001a890 con 0x55dcb4d32390 2024-08-31T07:29:34.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.252+0000 7f90bcc00640 1 -- 172.21.15.32:0/3614861898 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f90c0019070 con 0x55dcb4d32390 2024-08-31T07:29:34.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.252+0000 7f90e0a50740 10 client.28886 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.252+0000 7f90e0a50740 10 client.28886 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:29:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.252+0000 7f90e0a50740 20 client.28886 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.252+0000 7f90e0a50740 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55dcb4ce6da0 0x55dcb518c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.252+0000 7f90e0a50740 4 client.28886 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:29:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.252+0000 7f90e0a50740 1 -- 172.21.15.32:0/3614861898 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcb506d1a0 con 0x55dcb4ce6da0 2024-08-31T07:29:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.252+0000 7f90d2a00640 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55dcb4ce6da0 0x55dcb518c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:34.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.255+0000 7f90d2a00640 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55dcb4ce6da0 0x55dcb518c630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=101859c31cf90d87 server_cookie=f5e556a36fa48420 in_seq=0 out_seq=0 2024-08-31T07:29:34.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.255+0000 7f90bcc00640 10 client.28886 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:29:34.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.255+0000 7f90bcc00640 1 -- 172.21.15.32:0/3614861898 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcb506d1a0 con 0x55dcb4ce6da0 2024-08-31T07:29:34.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.255+0000 7f90bcc00640 10 client.28886 handle_command_reply: tid=0 2024-08-31T07:29:34.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.255+0000 7f90e0a50740 1 client.28886 shutdown 2024-08-31T07:29:34.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.256+0000 7f90e0a50740 1 -- 172.21.15.32:0/3614861898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d32390 msgr2=0x55dcb5111870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:34.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.256+0000 7f90e0a50740 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d32390 0x55dcb5111870 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e5c0 tx=0x7f90c001a490 comp rx=0 tx=0).stop 2024-08-31T07:29:34.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.256+0000 7f90e0a50740 1 -- 172.21.15.32:0/3614861898 shutdown_connections 2024-08-31T07:29:34.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.256+0000 7f90e0a50740 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcb4d53070 0x55dcb51886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.256+0000 7f90e0a50740 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55dcb4ce6da0 0x55dcb518c630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.256+0000 7f90e0a50740 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb4d32390 0x55dcb5111870 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.256+0000 7f90e0a50740 1 --2- 172.21.15.32:0/3614861898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dcb4d2a1b0 0x55dcb5111310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.256+0000 7f90e0a50740 1 -- 172.21.15.32:0/3614861898 >> 172.21.15.32:0/3614861898 conn(0x55dcb4d47460 msgr2=0x55dcb50fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:34.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.256+0000 7f90bcc00640 0 client.28886 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:29:34.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.257+0000 7f90e0a50740 1 -- 172.21.15.32:0/3614861898 shutdown_connections 2024-08-31T07:29:34.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.257+0000 7f90e0a50740 1 -- 172.21.15.32:0/3614861898 wait complete. 2024-08-31T07:29:34.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.257+0000 7f90e0a50740 20 client.28886 trim_cache size 0 max 16384 2024-08-31T07:29:34.258 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:29:34.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.258+0000 7f90d3e00640 1 -- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f90b80b4d20 msgr2=0x7f90b80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:34.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.258+0000 7f90d3e00640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f90b80b4d20 0x7f90b80b7230 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f90cc014cf0 tx=0x7f90cc009760 comp rx=0 tx=0).stop 2024-08-31T07:29:34.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.258+0000 7f90d3e00640 1 -- 172.21.15.32:0/3553476900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90d40754d0 msgr2=0x7f90d419c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:34.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.258+0000 7f90d3e00640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90d40754d0 0x7f90d419c260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f90c4005940 tx=0x7f90c40040c0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.270+0000 7f90d3e00640 1 -- 172.21.15.32:0/3553476900 shutdown_connections 2024-08-31T07:29:34.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.270+0000 7f90d3e00640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f90b80b4d20 0x7f90b80b7230 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.270+0000 7f90d3e00640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d410fd30 0x7f90d41a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.270+0000 7f90d3e00640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90d4075e20 0x7f90d419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.270+0000 7f90d3e00640 1 --2- 172.21.15.32:0/3553476900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90d40754d0 0x7f90d419c260 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:34.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.270+0000 7f90d3e00640 1 -- 172.21.15.32:0/3553476900 >> 172.21.15.32:0/3553476900 conn(0x7f90d40ff1a0 msgr2=0x7f90d4101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:34.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.273+0000 7f90d3e00640 1 -- 172.21.15.32:0/3553476900 shutdown_connections 2024-08-31T07:29:34.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:34.273+0000 7f90d3e00640 1 -- 172.21.15.32:0/3553476900 wait complete. 2024-08-31T07:29:34.292 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:34 smithi032 ceph-mon[17252]: pgmap v894: 97 pgs: 97 active+clean; 982 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1010 KiB/s wr, 84 op/s 2024-08-31T07:29:34.350 INFO:tasks.workunit.client.0.smithi032.stdout: 10 63321 9.61 MB/sec execute 74 sec latency 117.508 ms 2024-08-31T07:29:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:34 smithi084 ceph-mon[22387]: pgmap v894: 97 pgs: 97 active+clean; 982 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1010 KiB/s wr, 84 op/s 2024-08-31T07:29:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:34 smithi073 ceph-mon[21995]: pgmap v894: 97 pgs: 97 active+clean; 982 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1010 KiB/s wr, 84 op/s 2024-08-31T07:29:35.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:29:35.350 INFO:tasks.workunit.client.0.smithi032.stdout: 10 64633 10.40 MB/sec execute 75 sec latency 42.536 ms 2024-08-31T07:29:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:36.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65243 10.76 MB/sec execute 76 sec latency 479.926 ms 2024-08-31T07:29:36.471 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:36 smithi032 ceph-mon[17252]: pgmap v895: 97 pgs: 97 active+clean; 995 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-08-31T07:29:36.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:36 smithi084 ceph-mon[22387]: pgmap v895: 97 pgs: 97 active+clean; 995 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-08-31T07:29:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:36 smithi073 ceph-mon[21995]: pgmap v895: 97 pgs: 97 active+clean; 995 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-08-31T07:29:37.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65273 10.67 MB/sec execute 77 sec latency 1413.631 ms 2024-08-31T07:29:38.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65273 10.53 MB/sec execute 78 sec latency 2413.694 ms 2024-08-31T07:29:38.572 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:38 smithi032 ceph-mon[17252]: pgmap v896: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.2 MiB/s wr, 374 op/s 2024-08-31T07:29:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:38 smithi073 ceph-mon[21995]: pgmap v896: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.2 MiB/s wr, 374 op/s 2024-08-31T07:29:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:38 smithi084 ceph-mon[22387]: pgmap v896: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.2 MiB/s wr, 374 op/s 2024-08-31T07:29:38.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.984+0000 7f6f0fe00640 1 -- 172.21.15.32:0/100508539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10103750 msgr2=0x7f6f10103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:38.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.984+0000 7f6f0fe00640 1 --2- 172.21.15.32:0/100508539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10103750 0x7f6f10103b60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009900 tx=0x7f6f0002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:29:38.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.984+0000 7f6f0fe00640 1 -- 172.21.15.32:0/100508539 shutdown_connections 2024-08-31T07:29:38.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.984+0000 7f6f0fe00640 1 --2- 172.21.15.32:0/100508539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f101099e0 0x7f6f10110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:38.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.984+0000 7f6f0fe00640 1 --2- 172.21.15.32:0/100508539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f10109090 0x7f6f101094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:38.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.984+0000 7f6f0fe00640 1 --2- 172.21.15.32:0/100508539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10103750 0x7f6f10103b60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:38.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.984+0000 7f6f0fe00640 1 -- 172.21.15.32:0/100508539 >> 172.21.15.32:0/100508539 conn(0x7f6f100ff1e0 msgr2=0x7f6f10101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:38.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.984+0000 7f6f0fe00640 1 -- 172.21.15.32:0/100508539 shutdown_connections 2024-08-31T07:29:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.985+0000 7f6f0fe00640 1 -- 172.21.15.32:0/100508539 wait complete. 2024-08-31T07:29:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.985+0000 7f6f0fe00640 1 Processor -- start 2024-08-31T07:29:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0fe00640 1 -- start start 2024-08-31T07:29:38.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f10103750 0x7f6f1019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:38.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10109090 0x7f6f1019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:38.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f101099e0 0x7f6f1019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:38.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101136b0 con 0x7f6f10109090 2024-08-31T07:29:38.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10113530 con 0x7f6f101099e0 2024-08-31T07:29:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10113830 con 0x7f6f10103750 2024-08-31T07:29:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f101099e0 0x7f6f1019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10109090 0x7f6f1019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f101099e0 0x7f6f1019f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55082/0 (socket says 172.21.15.32:55082) 2024-08-31T07:29:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.986+0000 7f6f0f400640 1 -- 172.21.15.32:0/3185225956 learned_addr learned my addr 172.21.15.32:0/3185225956 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.987+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f10103750 0x7f6f1019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.987+0000 7f6f0e000640 1 -- 172.21.15.32:0/3185225956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f10103750 msgr2=0x7f6f1019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.987+0000 7f6f0e000640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f10103750 0x7f6f1019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.987+0000 7f6f0e000640 1 -- 172.21.15.32:0/3185225956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f101099e0 msgr2=0x7f6f1019f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.987+0000 7f6f0e000640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f101099e0 0x7f6f1019f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.987+0000 7f6f0e000640 1 -- 172.21.15.32:0/3185225956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f101a0070 con 0x7f6f10109090 2024-08-31T07:29:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.987+0000 7f6f0e000640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10109090 0x7f6f1019a6a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6f1006b730 tx=0x7f6efc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.988+0000 7f6f07400640 1 -- 172.21.15.32:0/3185225956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc00cd00 con 0x7f6f10109090 2024-08-31T07:29:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.988+0000 7f6f07400640 1 -- 172.21.15.32:0/3185225956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6efc0035f0 con 0x7f6f10109090 2024-08-31T07:29:38.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.988+0000 7f6f0fe00640 1 -- 172.21.15.32:0/3185225956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f101a0360 con 0x7f6f10109090 2024-08-31T07:29:38.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.988+0000 7f6f07400640 1 -- 172.21.15.32:0/3185225956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc010690 con 0x7f6f10109090 2024-08-31T07:29:38.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.988+0000 7f6f0fe00640 1 -- 172.21.15.32:0/3185225956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f101a6ac0 con 0x7f6f10109090 2024-08-31T07:29:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.991+0000 7f6f1c3de740 1 Processor -- start 2024-08-31T07:29:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.991+0000 7f6f1c3de740 1 -- start start 2024-08-31T07:29:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.991+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 0x563c502cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.991+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 0x563c502d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.991+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563c502427b0 con 0x563c4ff26070 2024-08-31T07:29:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563c50231d40 con 0x563c4fefd1b0 2024-08-31T07:29:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563c50235cf0 con 0x563c4ff05390 2024-08-31T07:29:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f07400640 1 -- 172.21.15.32:0/3185225956 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6efc0037a0 con 0x7f6f10109090 2024-08-31T07:29:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f0f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 0x563c502cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f0f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 0x563c502cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57496/0 (socket says 172.21.15.32:57496) 2024-08-31T07:29:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f0f400640 1 -- 172.21.15.32:0/2473629482 learned_addr learned my addr 172.21.15.32:0/2473629482 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f0e000640 1 --2- 172.21.15.32:0/2473629482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f0f400640 1 -- 172.21.15.32:0/2473629482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 msgr2=0x563c502d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f0f400640 1 --2- 172.21.15.32:0/2473629482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f0f400640 1 -- 172.21.15.32:0/2473629482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 msgr2=0x563c502d04b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:29:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f0f400640 1 --2- 172.21.15.32:0/2473629482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 0x563c502d04b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.992+0000 7f6f0f400640 1 -- 172.21.15.32:0/2473629482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c4ff91650 con 0x563c4ff26070 2024-08-31T07:29:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.993+0000 7f6f0f400640 1 --2- 172.21.15.32:0/2473629482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 0x563c502cff70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6f08011570 tx=0x7f6f08011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417f0ab8f136cc9c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.993+0000 7f6efbe00640 1 -- 172.21.15.32:0/2473629482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08004b20 con 0x563c4ff26070 2024-08-31T07:29:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.993+0000 7f6efbe00640 1 -- 172.21.15.32:0/2473629482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f08004cc0 con 0x563c4ff26070 2024-08-31T07:29:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.993+0000 7f6efbe00640 1 -- 172.21.15.32:0/2473629482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08005420 con 0x563c4ff26070 2024-08-31T07:29:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f07400640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6ef40b4d20 0x7f6ef40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f07400640 1 -- 172.21.15.32:0/3185225956 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f6efc0e57f0 con 0x7f6f10109090 2024-08-31T07:29:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6ef40b4d20 0x7f6ef40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f1c3de740 1 -- 172.21.15.32:0/2473629482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 msgr2=0x563c502cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/2473629482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 0x563c502cff70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6f08011570 tx=0x7f6f08011a30 comp rx=0 tx=0).stop 2024-08-31T07:29:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f1c3de740 1 -- 172.21.15.32:0/2473629482 shutdown_connections 2024-08-31T07:29:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/2473629482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/2473629482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 0x563c502d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/2473629482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 0x563c502cff70 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f1c3de740 1 -- 172.21.15.32:0/2473629482 >> 172.21.15.32:0/2473629482 conn(0x563c4ff1a460 msgr2=0x563c502cde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f1c3de740 1 -- 172.21.15.32:0/2473629482 shutdown_connections 2024-08-31T07:29:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.994+0000 7f6f1c3de740 1 -- 172.21.15.32:0/2473629482 wait complete. 2024-08-31T07:29:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.995+0000 7f6f1c3de740 1 -- start start 2024-08-31T07:29:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.995+0000 7f6f1c3de740 1 Processor -- start 2024-08-31T07:29:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6ef40b4d20 0x7f6ef40b7230 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6f00031440 tx=0x7f6f00009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502dbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff26070 0x563c502e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563c50094700 con 0x563c4fefd1b0 2024-08-31T07:29:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563c501b8cf0 con 0x563c4ff26070 2024-08-31T07:29:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563c502de750 con 0x563c4ff05390 2024-08-31T07:29:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f1c3de740 10 client.0 _ll_register_callbacks cb 0x563c501de470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502dbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502dbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57506/0 (socket says 172.21.15.32:57506) 2024-08-31T07:29:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f0ea00640 1 -- 172.21.15.32:0/3110642717 learned_addr learned my addr 172.21.15.32:0/3110642717 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.996+0000 7f6f0e000640 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6f0f400640 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff26070 0x563c502e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6f0e000640 1 -- 172.21.15.32:0/3110642717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff26070 msgr2=0x563c502e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6f0e000640 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff26070 0x563c502e7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6f0e000640 1 -- 172.21.15.32:0/3110642717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 msgr2=0x563c502dbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6f0e000640 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502dbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6f0e000640 1 -- 172.21.15.32:0/3110642717 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c501f98b0 con 0x563c4ff05390 2024-08-31T07:29:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6f0e000640 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502dc040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0e2f30 tx=0x7f6efc0a7eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8325156162ab231a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6ef8c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:29:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3110642717 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc009460 con 0x563c4ff05390 2024-08-31T07:29:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3110642717 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f08004cc0 con 0x563c4ff05390 2024-08-31T07:29:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.997+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3110642717 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f6efc0a8160 con 0x563c4ff05390 2024-08-31T07:29:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3110642717 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f6efc0ab9e0 con 0x563c4ff05390 2024-08-31T07:29:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6f1c3de740 10 client.28900 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3110642717 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x563c502ea790 con 0x563c4ff05390 2024-08-31T07:29:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3110642717 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f08004cc0 con 0x563c4ff05390 2024-08-31T07:29:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3110642717 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f6efc0b0330 con 0x563c4ff05390 2024-08-31T07:29:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6f1c3de740 10 client.28900 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6f1c3de740 10 client.28900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:29:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6f1c3de740 20 client.28900 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c4fee49f0 0x563c502edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6f1c3de740 4 client.28900 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.998+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3110642717 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c50244700 con 0x563c4fee49f0 2024-08-31T07:29:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.999+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c4fee49f0 0x563c502edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.999+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c4fee49f0 0x563c502edc50 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7fe577a164c254f server_cookie=a9620c75a2ba7566 in_seq=0 out_seq=0 2024-08-31T07:29:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:38.999+0000 7f6ef8c00640 10 client.28900 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3110642717 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c50244700 con 0x563c4fee49f0 2024-08-31T07:29:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6ef8c00640 10 client.28900 handle_command_reply: tid=0 2024-08-31T07:29:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6f1c3de740 1 client.28900 shutdown 2024-08-31T07:29:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3110642717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 msgr2=0x563c502dc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502dc040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0e2f30 tx=0x7f6efc0a7eb0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3110642717 shutdown_connections 2024-08-31T07:29:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c4fee49f0 0x563c502edc50 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff26070 0x563c502e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502dc040 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3110642717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502dbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3110642717 >> 172.21.15.32:0/3110642717 conn(0x563c4ff1a460 msgr2=0x563c502d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.000+0000 7f6ef8c00640 0 client.28900 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.001+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3110642717 shutdown_connections 2024-08-31T07:29:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.001+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3110642717 wait complete. 2024-08-31T07:29:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.001+0000 7f6f1c3de740 20 client.28900 trim_cache size 0 max 16384 2024-08-31T07:29:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.015+0000 7f6f1c3de740 1 Processor -- start 2024-08-31T07:29:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.015+0000 7f6f1c3de740 1 -- start start 2024-08-31T07:29:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.015+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.015+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff05390 0x563c502d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff26070 0x563c502e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ad020 con 0x563c4fefd1b0 2024-08-31T07:29:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101aeb90 con 0x563c4ff05390 2024-08-31T07:29:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ad720 con 0x563c4ff26070 2024-08-31T07:29:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff05390 0x563c502d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57520/0 (socket says 172.21.15.32:57520) 2024-08-31T07:29:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0ea00640 1 -- 172.21.15.32:0/1014340427 learned_addr learned my addr 172.21.15.32:0/1014340427 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0f400640 1 --2- 172.21.15.32:0/1014340427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff26070 0x563c502e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0ea00640 1 -- 172.21.15.32:0/1014340427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff26070 msgr2=0x563c502e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/1014340427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff26070 0x563c502e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0ea00640 1 -- 172.21.15.32:0/1014340427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff05390 msgr2=0x563c502d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/1014340427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff05390 0x563c502d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0ea00640 1 -- 172.21.15.32:0/1014340427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c502c50a0 con 0x563c4fefd1b0 2024-08-31T07:29:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.016+0000 7f6f0e000640 1 --2- 172.21.15.32:0/1014340427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff05390 0x563c502d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.017+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/1014340427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502c89b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009230 tx=0x7f6f00040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57bb56f439e4eac server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.017+0000 7f6efb400640 1 -- 172.21.15.32:0/1014340427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00035040 con 0x563c4fefd1b0 2024-08-31T07:29:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.017+0000 7f6efb400640 1 -- 172.21.15.32:0/1014340427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f00057070 con 0x563c4fefd1b0 2024-08-31T07:29:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.017+0000 7f6efb400640 1 -- 172.21.15.32:0/1014340427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00004750 con 0x563c4fefd1b0 2024-08-31T07:29:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.018+0000 7f6f1c3de740 1 -- 172.21.15.32:0/1014340427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 msgr2=0x563c502c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.018+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/1014340427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502c89b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009230 tx=0x7f6f00040000 comp rx=0 tx=0).stop 2024-08-31T07:29:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.018+0000 7f6f1c3de740 1 -- 172.21.15.32:0/1014340427 shutdown_connections 2024-08-31T07:29:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.018+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/1014340427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff26070 0x563c502e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.018+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/1014340427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4ff05390 0x563c502d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.018+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/1014340427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4fefd1b0 0x563c502c89b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.018+0000 7f6f1c3de740 1 -- 172.21.15.32:0/1014340427 >> 172.21.15.32:0/1014340427 conn(0x563c4ff1a460 msgr2=0x563c502d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.018+0000 7f6f1c3de740 1 -- 172.21.15.32:0/1014340427 shutdown_connections 2024-08-31T07:29:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.018+0000 7f6f1c3de740 1 -- 172.21.15.32:0/1014340427 wait complete. 2024-08-31T07:29:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.019+0000 7f6f1c3de740 1 -- start start 2024-08-31T07:29:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.019+0000 7f6f1c3de740 1 Processor -- start 2024-08-31T07:29:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.019+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 0x563c502e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.019+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.019+0000 7f6f1c3de740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 0x563c502dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.019+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ae4c0 con 0x563c4ff26070 2024-08-31T07:29:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.019+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101addf0 con 0x563c4fefd1b0 2024-08-31T07:29:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.019+0000 7f6f1c3de740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ab020 con 0x563c4ff05390 2024-08-31T07:29:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f1c3de740 10 client.0 _ll_register_callbacks cb 0x563c501de470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f0ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 0x563c502e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f0ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 0x563c502e43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55106/0 (socket says 172.21.15.32:55106) 2024-08-31T07:29:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f0ea00640 1 -- 172.21.15.32:0/3966668670 learned_addr learned my addr 172.21.15.32:0/3966668670 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f0e000640 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f0f400640 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 0x563c502dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f0ea00640 1 -- 172.21.15.32:0/3966668670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 msgr2=0x563c502df8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502df8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f0ea00640 1 -- 172.21.15.32:0/3966668670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 msgr2=0x563c502dffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 0x563c502dffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.020+0000 7f6f0ea00640 1 -- 172.21.15.32:0/3966668670 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c50357610 con 0x563c4fefd1b0 2024-08-31T07:29:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.021+0000 7f6f0ea00640 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 0x563c502e43b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6f00005f70 tx=0x7f6f00032d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3a99a7599148220 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:39.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.021+0000 7f6ef8c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:29:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.021+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3966668670 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00060060 con 0x563c4fefd1b0 2024-08-31T07:29:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.021+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3966668670 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f00057070 con 0x563c4fefd1b0 2024-08-31T07:29:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.021+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3966668670 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f6f00004450 con 0x563c4fefd1b0 2024-08-31T07:29:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.021+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3966668670 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f6f00035040 con 0x563c4fefd1b0 2024-08-31T07:29:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.021+0000 7f6f1c3de740 10 client.28901 fetch_fsmap learned FSMap version 65 2024-08-31T07:29:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.021+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3966668670 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x563c50358760 con 0x563c4fefd1b0 2024-08-31T07:29:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.021+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3966668670 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f00057070 con 0x563c4fefd1b0 2024-08-31T07:29:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.022+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3966668670 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) ==== 8366+0+0 (secure 0 0 0) 0x7f6f000035f0 con 0x563c4fefd1b0 2024-08-31T07:29:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.022+0000 7f6f1c3de740 10 client.28901 fetch_fsmap finished waiting for FSMap version 65 2024-08-31T07:29:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.022+0000 7f6f1c3de740 10 client.28901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:29:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.022+0000 7f6f1c3de740 20 client.28901 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.022+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c4feb9da0 0x563c5035aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.022+0000 7f6f1c3de740 4 client.28901 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:29:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.022+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3966668670 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c502401a0 con 0x563c4feb9da0 2024-08-31T07:29:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.022+0000 7f6f0e000640 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c4feb9da0 0x563c5035aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f0e000640 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c4feb9da0 0x563c5035aeb0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc1d9ffa29708e20 server_cookie=2f554d40d5314ba4 in_seq=0 out_seq=0 2024-08-31T07:29:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6ef8c00640 10 client.28901 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6ef8c00640 1 -- 172.21.15.32:0/3966668670 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c502401a0 con 0x563c4feb9da0 2024-08-31T07:29:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6ef8c00640 10 client.28901 handle_command_reply: tid=0 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 client.28901 shutdown 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3966668670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 msgr2=0x563c502e43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 0x563c502e43b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6f00005f70 tx=0x7f6f00032d70 comp rx=0 tx=0).stop 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3966668670 shutdown_connections 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x563c4feb9da0 0x563c5035aeb0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c4ff26070 0x563c502dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563c4ff05390 0x563c502df8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6ef8c00640 0 client.28901 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 --2- 172.21.15.32:0/3966668670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563c4fefd1b0 0x563c502e43b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3966668670 >> 172.21.15.32:0/3966668670 conn(0x563c4ff1a460 msgr2=0x563c502dd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3966668670 shutdown_connections 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 1 -- 172.21.15.32:0/3966668670 wait complete. 2024-08-31T07:29:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.023+0000 7f6f1c3de740 20 client.28901 trim_cache size 0 max 16384 2024-08-31T07:29:39.036 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:29:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.024+0000 7f6f0fe00640 1 -- 172.21.15.32:0/3185225956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6ef40b4d20 msgr2=0x7f6ef40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.024+0000 7f6f0fe00640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6ef40b4d20 0x7f6ef40b7230 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6f00031440 tx=0x7f6f00009560 comp rx=0 tx=0).stop 2024-08-31T07:29:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.024+0000 7f6f0fe00640 1 -- 172.21.15.32:0/3185225956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10109090 msgr2=0x7f6f1019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.024+0000 7f6f0fe00640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10109090 0x7f6f1019a6a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6f1006b730 tx=0x7f6efc00ee50 comp rx=0 tx=0).stop 2024-08-31T07:29:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.024+0000 7f6f0fe00640 1 -- 172.21.15.32:0/3185225956 shutdown_connections 2024-08-31T07:29:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.024+0000 7f6f0fe00640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f6ef40b4d20 0x7f6ef40b7230 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.024+0000 7f6f0fe00640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f101099e0 0x7f6f1019f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.024+0000 7f6f0fe00640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f10109090 0x7f6f1019a6a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.024+0000 7f6f0fe00640 1 --2- 172.21.15.32:0/3185225956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f10103750 0x7f6f1019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.024+0000 7f6f0fe00640 1 -- 172.21.15.32:0/3185225956 >> 172.21.15.32:0/3185225956 conn(0x7f6f100ff1e0 msgr2=0x7f6f10101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.025+0000 7f6f0fe00640 1 -- 172.21.15.32:0/3185225956 shutdown_connections 2024-08-31T07:29:39.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:39.025+0000 7f6f0fe00640 1 -- 172.21.15.32:0/3185225956 wait complete. 2024-08-31T07:29:39.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65274 10.40 MB/sec execute 79 sec latency 3413.760 ms 2024-08-31T07:29:39.421 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:29:39.421 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.421 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi084 to 176.0M 2024-08-31T07:29:39.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: Adjusting osd_memory_target on smithi073 to 176.0M 2024-08-31T07:29:39.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: Unable to set osd_memory_target on smithi084 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-08-31T07:29:39.422 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:39 smithi032 ceph-mon[17252]: Unable to set osd_memory_target on smithi073 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-08-31T07:29:39.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:29:39.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi084 to 176.0M 2024-08-31T07:29:39.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: Adjusting osd_memory_target on smithi073 to 176.0M 2024-08-31T07:29:39.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: Unable to set osd_memory_target on smithi084 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-08-31T07:29:39.682 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:39 smithi073 ceph-mon[21995]: Unable to set osd_memory_target on smithi073 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-08-31T07:29:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:29:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T07:29:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi084 to 176.0M 2024-08-31T07:29:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: Adjusting osd_memory_target on smithi073 to 176.0M 2024-08-31T07:29:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: Unable to set osd_memory_target on smithi084 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-08-31T07:29:39.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:39 smithi084 ceph-mon[22387]: Unable to set osd_memory_target on smithi073 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-08-31T07:29:40.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65285 10.27 MB/sec execute 80 sec latency 4413.800 ms 2024-08-31T07:29:40.509 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:40 smithi032 ceph-mon[17252]: pgmap v897: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 399 op/s 2024-08-31T07:29:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:40 smithi073 ceph-mon[21995]: pgmap v897: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 399 op/s 2024-08-31T07:29:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:40 smithi084 ceph-mon[22387]: pgmap v897: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 399 op/s 2024-08-31T07:29:41.002 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:29:41.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:29:41.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65322 10.18 MB/sec execute 81 sec latency 5413.852 ms 2024-08-31T07:29:41.354 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:41 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:41.354 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:29:41.354 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:29:41.354 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:29:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:41 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:29:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:29:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:29:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:41 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:29:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:29:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:29:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:29:42.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65322 10.05 MB/sec execute 82 sec latency 6413.901 ms 2024-08-31T07:29:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.624+0000 7f34aae00640 1 -- 172.21.15.32:0/3313520531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a4105b30 msgr2=0x7f34a4105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.624+0000 7f34aae00640 1 --2- 172.21.15.32:0/3313520531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a4105b30 0x7f34a4105f40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3494009860 tx=0x7f349402a340 comp rx=0 tx=0).stop 2024-08-31T07:29:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.625+0000 7f34aae00640 1 -- 172.21.15.32:0/3313520531 shutdown_connections 2024-08-31T07:29:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.625+0000 7f34aae00640 1 --2- 172.21.15.32:0/3313520531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34a410bdc0 0x7f34a41125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.625+0000 7f34aae00640 1 --2- 172.21.15.32:0/3313520531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a410b470 0x7f34a410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.625+0000 7f34aae00640 1 --2- 172.21.15.32:0/3313520531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a4105b30 0x7f34a4105f40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.625+0000 7f34aae00640 1 -- 172.21.15.32:0/3313520531 >> 172.21.15.32:0/3313520531 conn(0x7f34a4101600 msgr2=0x7f34a4103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.625+0000 7f34aae00640 1 -- 172.21.15.32:0/3313520531 shutdown_connections 2024-08-31T07:29:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.625+0000 7f34aae00640 1 -- 172.21.15.32:0/3313520531 wait complete. 2024-08-31T07:29:42.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.626+0000 7f34aae00640 1 Processor -- start 2024-08-31T07:29:42.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.626+0000 7f34aae00640 1 -- start start 2024-08-31T07:29:42.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34aae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4105b30 0x7f34a419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34aae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a410b470 0x7f34a419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34aae00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34a410bdc0 0x7f34a41a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34aae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4115a80 con 0x7f34a410b470 2024-08-31T07:29:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34aae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4115900 con 0x7f34a4105b30 2024-08-31T07:29:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34aae00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4115c00 con 0x7f34a410bdc0 2024-08-31T07:29:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a410b470 0x7f34a419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4105b30 0x7f34a419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4105b30 0x7f34a419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55128/0 (socket says 172.21.15.32:55128) 2024-08-31T07:29:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a410b470 0x7f34a419ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57540/0 (socket says 172.21.15.32:57540) 2024-08-31T07:29:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.627+0000 7f34a3e00640 1 -- 172.21.15.32:0/293937533 learned_addr learned my addr 172.21.15.32:0/293937533 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34a3e00640 1 -- 172.21.15.32:0/293937533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34a410bdc0 msgr2=0x7f34a41a1cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:29:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34a3e00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34a410bdc0 0x7f34a41a1cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34a3e00640 1 -- 172.21.15.32:0/293937533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a410b470 msgr2=0x7f34a419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34a3e00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a410b470 0x7f34a419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34a3e00640 1 -- 172.21.15.32:0/293937533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a41a23e0 con 0x7f34a4105b30 2024-08-31T07:29:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34a3400640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a410b470 0x7f34a419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:42.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34a3e00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4105b30 0x7f34a419c470 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3494035040 tx=0x7f3494032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:42.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34a0c00640 1 -- 172.21.15.32:0/293937533 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494037040 con 0x7f34a4105b30 2024-08-31T07:29:42.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34aae00640 1 -- 172.21.15.32:0/293937533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a41a2670 con 0x7f34a4105b30 2024-08-31T07:29:42.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34a0c00640 1 -- 172.21.15.32:0/293937533 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3494041070 con 0x7f34a4105b30 2024-08-31T07:29:42.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.628+0000 7f34a0c00640 1 -- 172.21.15.32:0/293937533 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494003850 con 0x7f34a4105b30 2024-08-31T07:29:42.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.629+0000 7f34aae00640 1 -- 172.21.15.32:0/293937533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a41a2af0 con 0x7f34a4105b30 2024-08-31T07:29:42.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.632+0000 7f34a0c00640 1 -- 172.21.15.32:0/293937533 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3494004050 con 0x7f34a4105b30 2024-08-31T07:29:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.632+0000 7f34b1a41740 1 Processor -- start 2024-08-31T07:29:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.633+0000 7f34b1a41740 1 -- start start 2024-08-31T07:29:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.633+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c49350 0x556631fef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.633+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c20490 0x556631fefb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.633+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c28670 0x556631ff0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.633+0000 7f34b1a41740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556631f65750 con 0x556631c28670 2024-08-31T07:29:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.633+0000 7f34b1a41740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556631f54ce0 con 0x556631c20490 2024-08-31T07:29:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.633+0000 7f34b1a41740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556631f58c90 con 0x556631c49350 2024-08-31T07:29:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.634+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c20490 0x556631fefb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.634+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c20490 0x556631fefb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55138/0 (socket says 172.21.15.32:55138) 2024-08-31T07:29:42.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.634+0000 7f34a3e00640 1 -- 172.21.15.32:0/147177763 learned_addr learned my addr 172.21.15.32:0/147177763 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.634+0000 7f34a3400640 1 --2- 172.21.15.32:0/147177763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c28670 0x556631ff0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.634+0000 7f34a3e00640 1 -- 172.21.15.32:0/147177763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c49350 msgr2=0x556631fef600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:29:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.634+0000 7f34a3e00640 1 --2- 172.21.15.32:0/147177763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c49350 0x556631fef600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.634+0000 7f34a3e00640 1 -- 172.21.15.32:0/147177763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c28670 msgr2=0x556631ff0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:42.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.634+0000 7f34a3e00640 1 --2- 172.21.15.32:0/147177763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c28670 0x556631ff0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.634+0000 7f34a3e00640 1 -- 172.21.15.32:0/147177763 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556631cb4930 con 0x556631c20490 2024-08-31T07:29:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.634+0000 7f34a0c00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34880b4ca0 0x7f34880b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.635+0000 7f34a0c00640 1 -- 172.21.15.32:0/293937533 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f34941050b0 con 0x7f34a4105b30 2024-08-31T07:29:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.635+0000 7f34a3e00640 1 --2- 172.21.15.32:0/147177763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c20490 0x556631fefb40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3494102730 tx=0x7f349410d1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61434f608f14ff68 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.635+0000 7f34a8e00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34880b4ca0 0x7f34880b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.635+0000 7f3499600640 1 -- 172.21.15.32:0/147177763 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34940c7650 con 0x556631c20490 2024-08-31T07:29:42.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.635+0000 7f3499600640 1 -- 172.21.15.32:0/147177763 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f349408d030 con 0x556631c20490 2024-08-31T07:29:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.635+0000 7f3499600640 1 -- 172.21.15.32:0/147177763 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34940caf00 con 0x556631c20490 2024-08-31T07:29:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.636+0000 7f34b1a41740 1 -- 172.21.15.32:0/147177763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c20490 msgr2=0x556631fefb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.636+0000 7f34a8e00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34880b4ca0 0x7f34880b71b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f349c005c40 tx=0x7f349c005bd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.636+0000 7f34b1a41740 1 --2- 172.21.15.32:0/147177763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c20490 0x556631fefb40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3494102730 tx=0x7f349410d1a0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.636+0000 7f34b1a41740 1 -- 172.21.15.32:0/147177763 shutdown_connections 2024-08-31T07:29:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.636+0000 7f34b1a41740 1 --2- 172.21.15.32:0/147177763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c28670 0x556631ff0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.636+0000 7f34b1a41740 1 --2- 172.21.15.32:0/147177763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c20490 0x556631fefb40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.636+0000 7f34b1a41740 1 --2- 172.21.15.32:0/147177763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c49350 0x556631fef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.636+0000 7f34b1a41740 1 -- 172.21.15.32:0/147177763 >> 172.21.15.32:0/147177763 conn(0x556631c3d740 msgr2=0x556631fed4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:42.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.636+0000 7f34b1a41740 1 -- 172.21.15.32:0/147177763 shutdown_connections 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.637+0000 7f34b1a41740 1 -- 172.21.15.32:0/147177763 wait complete. 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.637+0000 7f34b1a41740 1 -- start start 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.637+0000 7f34b1a41740 1 Processor -- start 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.638+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c20490 0x556631ffb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.638+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c28670 0x556631ffb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.638+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 0x556632006d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.638+0000 7f34b1a41740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556631db7890 con 0x556631c20490 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.638+0000 7f34b1a41740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556631edbcd0 con 0x556631c49350 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.638+0000 7f34b1a41740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556631ffdba0 con 0x556631c28670 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.638+0000 7f34b1a41740 10 client.0 _ll_register_callbacks cb 0x556631f4fef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.638+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c20490 0x556631ffb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.639+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c20490 0x556631ffb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57556/0 (socket says 172.21.15.32:57556) 2024-08-31T07:29:42.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.639+0000 7f34a3e00640 1 -- 172.21.15.32:0/2643791486 learned_addr learned my addr 172.21.15.32:0/2643791486 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.639+0000 7f34a8e00640 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 0x556632006d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.639+0000 7f34a3e00640 1 -- 172.21.15.32:0/2643791486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c28670 msgr2=0x556631ffb770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.639+0000 7f34a3e00640 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c28670 0x556631ffb770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.639+0000 7f34a3e00640 1 -- 172.21.15.32:0/2643791486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 msgr2=0x556632006d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.639+0000 7f34a3e00640 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 0x556632006d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.639+0000 7f34a3e00640 1 -- 172.21.15.32:0/2643791486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556631f251b0 con 0x556631c20490 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.639+0000 7f34a3e00640 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c20490 0x556631ffb1f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f34940306c0 tx=0x7f34940c7cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26298293c5d1e394 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.639+0000 7f348e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.640+0000 7f348e000640 1 -- 172.21.15.32:0/2643791486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34940cb0e0 con 0x556631c20490 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.640+0000 7f348e000640 1 -- 172.21.15.32:0/2643791486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f34940cb9f0 con 0x556631c20490 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.640+0000 7f34b1a41740 1 -- 172.21.15.32:0/2643791486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f349408d030 con 0x556631c20490 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.640+0000 7f348e000640 1 -- 172.21.15.32:0/2643791486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f34940cb560 con 0x556631c20490 2024-08-31T07:29:42.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.640+0000 7f34b1a41740 10 client.18873 fetch_fsmap learned FSMap version 66 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.640+0000 7f34b1a41740 1 -- 172.21.15.32:0/2643791486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x556632009af0 con 0x556631c20490 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.640+0000 7f348e000640 1 -- 172.21.15.32:0/2643791486 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f349408d030 con 0x556631c20490 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.640+0000 7f348e000640 1 -- 172.21.15.32:0/2643791486 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f34940d4620 con 0x556631c20490 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.641+0000 7f34b1a41740 10 client.18873 fetch_fsmap finished waiting for FSMap version 66 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.641+0000 7f34b1a41740 10 client.18873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.641+0000 7f34b1a41740 20 client.18873 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.641+0000 7f34b1a41740 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556631c07cd0 0x55663200cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.641+0000 7f34b1a41740 4 client.18873 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.641+0000 7f34b1a41740 1 -- 172.21.15.32:0/2643791486 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556631f676a0 con 0x556631c07cd0 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.641+0000 7f34a3400640 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556631c07cd0 0x55663200cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.642+0000 7f34a3400640 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556631c07cd0 0x55663200cc80 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e4722cd278cbac server_cookie=8d28babea557d2e7 in_seq=0 out_seq=0 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.642+0000 7f348e000640 10 client.18873 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:42.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.642+0000 7f348e000640 1 -- 172.21.15.32:0/2643791486 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556631f676a0 con 0x556631c07cd0 2024-08-31T07:29:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.642+0000 7f348e000640 10 client.18873 handle_command_reply: tid=0 2024-08-31T07:29:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 client.18873 shutdown 2024-08-31T07:29:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 -- 172.21.15.32:0/2643791486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c20490 msgr2=0x556631ffb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c20490 0x556631ffb1f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f34940306c0 tx=0x7f34940c7cd0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 -- 172.21.15.32:0/2643791486 shutdown_connections 2024-08-31T07:29:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556631c07cd0 0x55663200cc80 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 0x556632006d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c28670 0x556631ffb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 --2- 172.21.15.32:0/2643791486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631c20490 0x556631ffb1f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f348e000640 0 client.18873 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 -- 172.21.15.32:0/2643791486 >> 172.21.15.32:0/2643791486 conn(0x556631c3d740 msgr2=0x556631ff0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 -- 172.21.15.32:0/2643791486 shutdown_connections 2024-08-31T07:29:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 1 -- 172.21.15.32:0/2643791486 wait complete. 2024-08-31T07:29:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.643+0000 7f34b1a41740 20 client.18873 trim_cache size 0 max 16384 2024-08-31T07:29:42.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.675+0000 7f34b1a41740 1 Processor -- start 2024-08-31T07:29:42.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.675+0000 7f34b1a41740 1 -- start start 2024-08-31T07:29:42.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.675+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c28670 0x556631ff67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.675+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 0x556631ffced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.675+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556631ff5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.675+0000 7f34b1a41740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f34a41b1620 con 0x556631bdd080 2024-08-31T07:29:42.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.675+0000 7f34b1a41740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f34a41b31c0 con 0x556631c49350 2024-08-31T07:29:42.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.675+0000 7f34b1a41740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f34a41b1d20 con 0x556631c28670 2024-08-31T07:29:42.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.676+0000 7f34a8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556631ff5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.676+0000 7f34a8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556631ff5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57562/0 (socket says 172.21.15.32:57562) 2024-08-31T07:29:42.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.676+0000 7f34a8e00640 1 -- 172.21.15.32:0/2708912851 learned_addr learned my addr 172.21.15.32:0/2708912851 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:42.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.676+0000 7f34a3400640 1 --2- 172.21.15.32:0/2708912851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 0x556631ffced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.676+0000 7f34a8e00640 1 -- 172.21.15.32:0/2708912851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c28670 msgr2=0x556631ff67a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:29:42.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.676+0000 7f34a8e00640 1 --2- 172.21.15.32:0/2708912851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c28670 0x556631ff67a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.676+0000 7f34a8e00640 1 -- 172.21.15.32:0/2708912851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 msgr2=0x556631ffced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:42.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.676+0000 7f34a8e00640 1 --2- 172.21.15.32:0/2708912851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 0x556631ffced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.676+0000 7f34a8e00640 1 -- 172.21.15.32:0/2708912851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556631ff18f0 con 0x556631bdd080 2024-08-31T07:29:42.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.676+0000 7f34a3400640 1 --2- 172.21.15.32:0/2708912851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 0x556631ffced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:42.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.677+0000 7f34a8e00640 1 --2- 172.21.15.32:0/2708912851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556631ff5720 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f349c0114c0 tx=0x7f349c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca30cbc26a62cd9c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.677+0000 7f3498c00640 1 -- 172.21.15.32:0/2708912851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c027600 con 0x556631bdd080 2024-08-31T07:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.677+0000 7f3498c00640 1 -- 172.21.15.32:0/2708912851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f349c017030 con 0x556631bdd080 2024-08-31T07:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.677+0000 7f3498c00640 1 -- 172.21.15.32:0/2708912851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c01fb60 con 0x556631bdd080 2024-08-31T07:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.678+0000 7f34b1a41740 1 -- 172.21.15.32:0/2708912851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 msgr2=0x556631ff5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.678+0000 7f34b1a41740 1 --2- 172.21.15.32:0/2708912851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556631ff5720 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f349c0114c0 tx=0x7f349c011000 comp rx=0 tx=0).stop 2024-08-31T07:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.678+0000 7f34b1a41740 1 -- 172.21.15.32:0/2708912851 shutdown_connections 2024-08-31T07:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.678+0000 7f34b1a41740 1 --2- 172.21.15.32:0/2708912851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556631ff5720 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.678+0000 7f34b1a41740 1 --2- 172.21.15.32:0/2708912851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c49350 0x556631ffced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.678+0000 7f34b1a41740 1 --2- 172.21.15.32:0/2708912851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c28670 0x556631ff67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.678+0000 7f34b1a41740 1 -- 172.21.15.32:0/2708912851 >> 172.21.15.32:0/2708912851 conn(0x556631c3d740 msgr2=0x556631ff9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:42.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.678+0000 7f34b1a41740 1 -- 172.21.15.32:0/2708912851 shutdown_connections 2024-08-31T07:29:42.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.678+0000 7f34b1a41740 1 -- 172.21.15.32:0/2708912851 wait complete. 2024-08-31T07:29:42.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.679+0000 7f34b1a41740 1 -- start start 2024-08-31T07:29:42.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.679+0000 7f34b1a41740 1 Processor -- start 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556632003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c28670 0x556631fffca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34b1a41740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c49350 0x5566320003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34b1a41740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f34a41b2af0 con 0x556631bdd080 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34b1a41740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f34a41b2420 con 0x556631c28670 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34b1a41740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f34a41af620 con 0x556631c49350 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34b1a41740 10 client.0 _ll_register_callbacks cb 0x7f3490009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556632003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34a3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c28670 0x556631fffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34a3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c28670 0x556631fffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55168/0 (socket says 172.21.15.32:55168) 2024-08-31T07:29:42.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34a3400640 1 -- 172.21.15.32:0/668522551 learned_addr learned my addr 172.21.15.32:0/668522551 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:42.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556632003780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57574/0 (socket says 172.21.15.32:57574) 2024-08-31T07:29:42.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.680+0000 7f34a8e00640 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c49350 0x5566320003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.681+0000 7f34a8e00640 1 -- 172.21.15.32:0/668522551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c28670 msgr2=0x556631fffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:42.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.681+0000 7f34a8e00640 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c28670 0x556631fffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.681+0000 7f34a8e00640 1 -- 172.21.15.32:0/668522551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 msgr2=0x556632003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:42.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.681+0000 7f34a8e00640 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556632003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.681+0000 7f34a8e00640 1 -- 172.21.15.32:0/668522551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566320774f0 con 0x556631c49350 2024-08-31T07:29:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.681+0000 7f34a8e00640 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c49350 0x5566320003b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f349c0099c0 tx=0x7f349c01fd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab4c476c0c9ba61e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.681+0000 7f348e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:29:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.681+0000 7f348e000640 1 -- 172.21.15.32:0/668522551 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c035060 con 0x556631c49350 2024-08-31T07:29:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.681+0000 7f348e000640 1 -- 172.21.15.32:0/668522551 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7f349c004a40 con 0x556631c49350 2024-08-31T07:29:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.681+0000 7f34b1a41740 1 -- 172.21.15.32:0/668522551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f349c017030 con 0x556631c49350 2024-08-31T07:29:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.682+0000 7f348e000640 1 -- 172.21.15.32:0/668522551 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f349c004c40 con 0x556631c49350 2024-08-31T07:29:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.682+0000 7f34b1a41740 10 client.28912 fetch_fsmap learned FSMap version 66 2024-08-31T07:29:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.682+0000 7f34b1a41740 1 -- 172.21.15.32:0/668522551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5566320786b0 con 0x556631c49350 2024-08-31T07:29:42.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.682+0000 7f348e000640 1 -- 172.21.15.32:0/668522551 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f349c017030 con 0x556631c49350 2024-08-31T07:29:42.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.683+0000 7f348e000640 1 -- 172.21.15.32:0/668522551 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7f349c00f040 con 0x556631c49350 2024-08-31T07:29:42.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.683+0000 7f34b1a41740 10 client.28912 fetch_fsmap finished waiting for FSMap version 66 2024-08-31T07:29:42.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.683+0000 7f34b1a41740 10 client.28912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:29:42.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.683+0000 7f34b1a41740 20 client.28912 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:42.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.683+0000 7f34b1a41740 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556632078d50 0x55663207b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:42.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.683+0000 7f34b1a41740 4 client.28912 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.683+0000 7f34b1a41740 1 -- 172.21.15.32:0/668522551 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556631f63140 con 0x556632078d50 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.683+0000 7f34a3e00640 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556632078d50 0x55663207b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.684+0000 7f34a3e00640 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556632078d50 0x55663207b170 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f7698bd0872ea45 server_cookie=f057c155e150ad87 in_seq=0 out_seq=0 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.684+0000 7f348e000640 10 client.28912 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.685+0000 7f348e000640 1 -- 172.21.15.32:0/668522551 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556631f63140 con 0x556632078d50 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.685+0000 7f348e000640 10 client.28912 handle_command_reply: tid=0 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.685+0000 7f34b1a41740 1 client.28912 shutdown 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 1 -- 172.21.15.32:0/668522551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c49350 msgr2=0x5566320003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c49350 0x5566320003b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f349c0099c0 tx=0x7f349c01fd40 comp rx=0 tx=0).stop 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 1 -- 172.21.15.32:0/668522551 shutdown_connections 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x556632078d50 0x55663207b170 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556631c49350 0x5566320003b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556631c28670 0x556631fffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 1 --2- 172.21.15.32:0/668522551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556631bdd080 0x556632003780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 1 -- 172.21.15.32:0/668522551 >> 172.21.15.32:0/668522551 conn(0x556631c3d740 msgr2=0x556631ffcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f348e000640 0 client.28912 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 1 -- 172.21.15.32:0/668522551 shutdown_connections 2024-08-31T07:29:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 1 -- 172.21.15.32:0/668522551 wait complete. 2024-08-31T07:29:42.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.686+0000 7f34b1a41740 20 client.28912 trim_cache size 0 max 16384 2024-08-31T07:29:42.690 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:29:42.690 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:29:42.690 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "23179d43-8143-476f-8da3-09d7ff58e6e1", 2024-08-31T07:29:42.690 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:29:42.691 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:29:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 -- 172.21.15.32:0/293937533 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34880b4ca0 msgr2=0x7f34880b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34880b4ca0 0x7f34880b71b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f349c005c40 tx=0x7f349c005bd0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 -- 172.21.15.32:0/293937533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4105b30 msgr2=0x7f34a419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4105b30 0x7f34a419c470 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3494035040 tx=0x7f3494032dc0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 -- 172.21.15.32:0/293937533 shutdown_connections 2024-08-31T07:29:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f34880b4ca0 0x7f34880b71b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34a410bdc0 0x7f34a41a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34a410b470 0x7f34a419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 --2- 172.21.15.32:0/293937533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4105b30 0x7f34a419c470 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 -- 172.21.15.32:0/293937533 >> 172.21.15.32:0/293937533 conn(0x7f34a4101600 msgr2=0x7f34a410ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.687+0000 7f34aae00640 1 -- 172.21.15.32:0/293937533 shutdown_connections 2024-08-31T07:29:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:42.688+0000 7f34aae00640 1 -- 172.21.15.32:0/293937533 wait complete. 2024-08-31T07:29:42.695 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:42 smithi032 ceph-mon[17252]: pgmap v898: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.7 MiB/s wr, 469 op/s 2024-08-31T07:29:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:42 smithi073 ceph-mon[21995]: pgmap v898: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.7 MiB/s wr, 469 op/s 2024-08-31T07:29:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:42 smithi084 ceph-mon[22387]: pgmap v898: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.7 MiB/s wr, 469 op/s 2024-08-31T07:29:43.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65322 9.93 MB/sec execute 83 sec latency 7413.941 ms 2024-08-31T07:29:43.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:29:43.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:43 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:29:43.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:43 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.802 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:43 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:29:43.802 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:43 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:29:43.802 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:43 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:29:43.802 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:43 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:29:43.802 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:43 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:29:43.802 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:43 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.802 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:43 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.802 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:43 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:43 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:29:43.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:43 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:43 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:29:43.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:43 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:29:43.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:43 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:29:43.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:43 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:29:43.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:43 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:29:43.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:43 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:43 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:43 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:43 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:29:43.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:43 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:43 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:29:43.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:43 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:29:43.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:43 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:29:43.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:43 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:29:43.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:43 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:29:43.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:43 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:43 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:43.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:43 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:44.083 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:29:44.354 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65323 9.81 MB/sec execute 84 sec latency 8413.982 ms 2024-08-31T07:29:44.663 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:44 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:44.663 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:44 smithi032 ceph-mon[17252]: pgmap v899: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.2 MiB/s wr, 495 op/s 2024-08-31T07:29:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:44 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:44 smithi073 ceph-mon[21995]: pgmap v899: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.2 MiB/s wr, 495 op/s 2024-08-31T07:29:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.952+0000 7fc1df000640 1 -- 172.21.15.32:0/3581771147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d8075ea0 msgr2=0x7fc1d810f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.952+0000 7fc1df000640 1 --2- 172.21.15.32:0/3581771147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d8075ea0 0x7fc1d810f770 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009900 tx=0x7fc1c402a420 comp rx=0 tx=0).stop 2024-08-31T07:29:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.953+0000 7fc1df000640 1 -- 172.21.15.32:0/3581771147 shutdown_connections 2024-08-31T07:29:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.953+0000 7fc1df000640 1 --2- 172.21.15.32:0/3581771147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d810fcb0 0x7fc1d81120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.953+0000 7fc1df000640 1 --2- 172.21.15.32:0/3581771147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d8075ea0 0x7fc1d810f770 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.953+0000 7fc1df000640 1 --2- 172.21.15.32:0/3581771147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d8075550 0x7fc1d8075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.953+0000 7fc1df000640 1 -- 172.21.15.32:0/3581771147 >> 172.21.15.32:0/3581771147 conn(0x7fc1d80ff1e0 msgr2=0x7fc1d8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:44.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.953+0000 7fc1df000640 1 -- 172.21.15.32:0/3581771147 shutdown_connections 2024-08-31T07:29:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.955+0000 7fc1df000640 1 -- 172.21.15.32:0/3581771147 wait complete. 2024-08-31T07:29:44.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.956+0000 7fc1df000640 1 Processor -- start 2024-08-31T07:29:44.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.956+0000 7fc1df000640 1 -- start start 2024-08-31T07:29:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.957+0000 7fc1df000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d8075550 0x7fc1d819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.957+0000 7fc1df000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d8075ea0 0x7fc1d819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.957+0000 7fc1df000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d810fcb0 0x7fc1d81a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.957+0000 7fc1df000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8115910 con 0x7fc1d8075550 2024-08-31T07:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.957+0000 7fc1df000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8115790 con 0x7fc1d810fcb0 2024-08-31T07:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.957+0000 7fc1df000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8115a90 con 0x7fc1d8075ea0 2024-08-31T07:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.957+0000 7fc1de600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d810fcb0 0x7fc1d81a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.957+0000 7fc1de600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d810fcb0 0x7fc1d81a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55182/0 (socket says 172.21.15.32:55182) 2024-08-31T07:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.957+0000 7fc1de600640 1 -- 172.21.15.32:0/1101586411 learned_addr learned my addr 172.21.15.32:0/1101586411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.958+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d8075550 0x7fc1d819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.958+0000 7fc1dd200640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d8075ea0 0x7fc1d819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.958+0000 7fc1ddc00640 1 -- 172.21.15.32:0/1101586411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d8075ea0 msgr2=0x7fc1d819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.958+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d8075ea0 0x7fc1d819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.958+0000 7fc1ddc00640 1 -- 172.21.15.32:0/1101586411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d810fcb0 msgr2=0x7fc1d81a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.958+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d810fcb0 0x7fc1d81a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.959+0000 7fc1ddc00640 1 -- 172.21.15.32:0/1101586411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d81a2250 con 0x7fc1d8075550 2024-08-31T07:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.959+0000 7fc1dd200640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d8075ea0 0x7fc1d819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.959+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d8075550 0x7fc1d819c2e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00e9f0 tx=0x7fc1cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.961+0000 7fc1d6000640 1 -- 172.21.15.32:0/1101586411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00ccc0 con 0x7fc1d8075550 2024-08-31T07:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.961+0000 7fc1d6000640 1 -- 172.21.15.32:0/1101586411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1cc0108e0 con 0x7fc1d8075550 2024-08-31T07:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.961+0000 7fc1d6000640 1 -- 172.21.15.32:0/1101586411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc018880 con 0x7fc1d8075550 2024-08-31T07:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.961+0000 7fc1df000640 1 -- 172.21.15.32:0/1101586411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d81a2540 con 0x7fc1d8075550 2024-08-31T07:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.961+0000 7fc1df000640 1 -- 172.21.15.32:0/1101586411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d81a8ca0 con 0x7fc1d8075550 2024-08-31T07:29:44.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.965+0000 7fc1e73eb740 1 Processor -- start 2024-08-31T07:29:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.965+0000 7fc1e73eb740 1 -- start start 2024-08-31T07:29:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.965+0000 7fc1d6000640 1 -- 172.21.15.32:0/1101586411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc1cc010430 con 0x7fc1d8075550 2024-08-31T07:29:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 0x562576efcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b2e1b0 0x562576efd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b36390 0x562576efd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562576e73770 con 0x562576b2e1b0 2024-08-31T07:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562576e62d00 con 0x562576b36390 2024-08-31T07:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562576e66cb0 con 0x562576b57070 2024-08-31T07:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1de600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 0x562576efcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1de600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 0x562576efcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:49102/0 (socket says 172.21.15.32:49102) 2024-08-31T07:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/3271817314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b2e1b0 0x562576efd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1dd200640 1 --2- 172.21.15.32:0/3271817314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b36390 0x562576efd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.966+0000 7fc1de600640 1 -- 172.21.15.32:0/3271817314 learned_addr learned my addr 172.21.15.32:0/3271817314 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.967+0000 7fc1ddc00640 1 -- 172.21.15.32:0/3271817314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 msgr2=0x562576efcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.967+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/3271817314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 0x562576efcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.967+0000 7fc1ddc00640 1 -- 172.21.15.32:0/3271817314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b36390 msgr2=0x562576efd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.967+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/3271817314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b36390 0x562576efd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.967+0000 7fc1ddc00640 1 -- 172.21.15.32:0/3271817314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562576bc2650 con 0x562576b2e1b0 2024-08-31T07:29:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.968+0000 7fc1d6000640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc1c00b4d20 0x7fc1c00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.968+0000 7fc1d6000640 1 -- 172.21.15.32:0/1101586411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7fc1cc0e58a0 con 0x7fc1d8075550 2024-08-31T07:29:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.968+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/3271817314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b2e1b0 0x562576efd060 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0e2fe0 tx=0x7fc1cc0ed670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d8ef5cf1359946 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.968+0000 7fc1caa00640 1 -- 172.21.15.32:0/3271817314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0a7e40 con 0x562576b2e1b0 2024-08-31T07:29:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.968+0000 7fc1caa00640 1 -- 172.21.15.32:0/3271817314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1cc010720 con 0x562576b2e1b0 2024-08-31T07:29:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.969+0000 7fc1caa00640 1 -- 172.21.15.32:0/3271817314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0ab6f0 con 0x562576b2e1b0 2024-08-31T07:29:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.970+0000 7fc1e73eb740 1 -- 172.21.15.32:0/3271817314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b2e1b0 msgr2=0x562576efd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.970+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/3271817314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b2e1b0 0x562576efd060 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0e2fe0 tx=0x7fc1cc0ed670 comp rx=0 tx=0).stop 2024-08-31T07:29:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.970+0000 7fc1de600640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc1c00b4d20 0x7fc1c00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.970+0000 7fc1e73eb740 1 -- 172.21.15.32:0/3271817314 shutdown_connections 2024-08-31T07:29:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.970+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/3271817314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b36390 0x562576efd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.970+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/3271817314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b2e1b0 0x562576efd060 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.970+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/3271817314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 0x562576efcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.970+0000 7fc1e73eb740 1 -- 172.21.15.32:0/3271817314 >> 172.21.15.32:0/3271817314 conn(0x562576b4b460 msgr2=0x562576efaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.970+0000 7fc1e73eb740 1 -- 172.21.15.32:0/3271817314 shutdown_connections 2024-08-31T07:29:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.970+0000 7fc1e73eb740 1 -- 172.21.15.32:0/3271817314 wait complete. 2024-08-31T07:29:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.971+0000 7fc1de600640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc1c00b4d20 0x7fc1c00b7230 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00094c0 tx=0x7fc1d0009970 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.971+0000 7fc1e73eb740 1 -- start start 2024-08-31T07:29:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.971+0000 7fc1e73eb740 1 Processor -- start 2024-08-31T07:29:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.972+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 0x562576f08780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.972+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b36390 0x562576f08d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.972+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 0x562576f14620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.972+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562576cc5700 con 0x562576b36390 2024-08-31T07:29:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.972+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562576de9cb0 con 0x562576b2e1b0 2024-08-31T07:29:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.972+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562576f0b300 con 0x562576b57070 2024-08-31T07:29:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.972+0000 7fc1e73eb740 10 client.0 _ll_register_callbacks cb 0x562576e0f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.973+0000 7fc1dd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b36390 0x562576f08d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.973+0000 7fc1dd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b36390 0x562576f08d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57610/0 (socket says 172.21.15.32:57610) 2024-08-31T07:29:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.973+0000 7fc1dd200640 1 -- 172.21.15.32:0/371102389 learned_addr learned my addr 172.21.15.32:0/371102389 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.973+0000 7fc1de600640 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 0x562576f14620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.974+0000 7fc1de600640 1 -- 172.21.15.32:0/371102389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 msgr2=0x562576f08780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:29:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.974+0000 7fc1de600640 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 0x562576f08780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.974+0000 7fc1de600640 1 -- 172.21.15.32:0/371102389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b36390 msgr2=0x562576f08d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.974+0000 7fc1de600640 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b36390 0x562576f08d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.974+0000 7fc1de600640 1 -- 172.21.15.32:0/371102389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562576e2a870 con 0x562576b57070 2024-08-31T07:29:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.974+0000 7fc1dd200640 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b36390 0x562576f08d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:29:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.975+0000 7fc1de600640 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 0x562576f14620 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009520 tx=0x7fc1d0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc665878d4f1467 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.975+0000 7fc1bf400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:29:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.975+0000 7fc1bf400640 1 -- 172.21.15.32:0/371102389 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0027060 con 0x562576b57070 2024-08-31T07:29:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.975+0000 7fc1bf400640 1 -- 172.21.15.32:0/371102389 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7fc1d0015cc0 con 0x562576b57070 2024-08-31T07:29:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.975+0000 7fc1e73eb740 1 -- 172.21.15.32:0/371102389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562576e756c0 con 0x562576b57070 2024-08-31T07:29:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.976+0000 7fc1bf400640 1 -- 172.21.15.32:0/371102389 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fc1d002c460 con 0x562576b57070 2024-08-31T07:29:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.976+0000 7fc1e73eb740 10 client.28924 fetch_fsmap learned FSMap version 66 2024-08-31T07:29:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.976+0000 7fc1e73eb740 1 -- 172.21.15.32:0/371102389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x562576f17440 con 0x562576b57070 2024-08-31T07:29:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.976+0000 7fc1bf400640 1 -- 172.21.15.32:0/371102389 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562576e756c0 con 0x562576b57070 2024-08-31T07:29:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.977+0000 7fc1bf400640 1 -- 172.21.15.32:0/371102389 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fc1d0016070 con 0x562576b57070 2024-08-31T07:29:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.977+0000 7fc1e73eb740 10 client.28924 fetch_fsmap finished waiting for FSMap version 66 2024-08-31T07:29:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.977+0000 7fc1e73eb740 10 client.28924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:29:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.977+0000 7fc1e73eb740 20 client.28924 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.977+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562576b159f0 0x562576f1a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.977+0000 7fc1e73eb740 4 client.28924 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:29:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.977+0000 7fc1e73eb740 1 -- 172.21.15.32:0/371102389 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562576e71160 con 0x562576b159f0 2024-08-31T07:29:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:44 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:44 smithi084 ceph-mon[22387]: pgmap v899: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.2 MiB/s wr, 495 op/s 2024-08-31T07:29:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:44.994+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562576b159f0 0x562576f1a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.077+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562576b159f0 0x562576f1a9a0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2a62359483cd38 server_cookie=ae4f93941b8824c5 in_seq=0 out_seq=0 2024-08-31T07:29:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.077+0000 7fc1bf400640 10 client.28924 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1bf400640 1 -- 172.21.15.32:0/371102389 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562576e71160 con 0x562576b159f0 2024-08-31T07:29:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1bf400640 10 client.28924 handle_command_reply: tid=0 2024-08-31T07:29:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1e73eb740 1 client.28924 shutdown 2024-08-31T07:29:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1e73eb740 1 -- 172.21.15.32:0/371102389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 msgr2=0x562576f14620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 0x562576f14620 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009520 tx=0x7fc1d0004ad0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1e73eb740 1 -- 172.21.15.32:0/371102389 shutdown_connections 2024-08-31T07:29:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562576b159f0 0x562576f1a9a0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b57070 0x562576f14620 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b36390 0x562576f08d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1bf400640 0 client.28924 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/371102389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 0x562576f08780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.089+0000 7fc1e73eb740 1 -- 172.21.15.32:0/371102389 >> 172.21.15.32:0/371102389 conn(0x562576b4b460 msgr2=0x562576effcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.090+0000 7fc1e73eb740 1 -- 172.21.15.32:0/371102389 shutdown_connections 2024-08-31T07:29:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.090+0000 7fc1e73eb740 1 -- 172.21.15.32:0/371102389 wait complete. 2024-08-31T07:29:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.090+0000 7fc1e73eb740 20 client.28924 trim_cache size 0 max 16384 2024-08-31T07:29:45.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.122+0000 7fc1e73eb740 1 Processor -- start 2024-08-31T07:29:45.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.122+0000 7fc1e73eb740 1 -- start start 2024-08-31T07:29:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.123+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b2e1b0 0x562576f01da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.123+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b36390 0x562576f022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.123+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 0x562576ef9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.123+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81b1420 con 0x562576b57070 2024-08-31T07:29:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.123+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81b2fc0 con 0x562576b36390 2024-08-31T07:29:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.123+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81b1b20 con 0x562576b2e1b0 2024-08-31T07:29:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.123+0000 7fc1ddc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b2e1b0 0x562576f01da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:45.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.123+0000 7fc1ddc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b2e1b0 0x562576f01da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:49124/0 (socket says 172.21.15.32:49124) 2024-08-31T07:29:45.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.123+0000 7fc1ddc00640 1 -- 172.21.15.32:0/1399382677 learned_addr learned my addr 172.21.15.32:0/1399382677 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:45.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.124+0000 7fc1ddc00640 1 -- 172.21.15.32:0/1399382677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b36390 msgr2=0x562576f022e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:29:45.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.124+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/1399382677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b36390 0x562576f022e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.124+0000 7fc1ddc00640 1 -- 172.21.15.32:0/1399382677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 msgr2=0x562576ef9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:45.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.124+0000 7fc1de600640 1 --2- 172.21.15.32:0/1399382677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 0x562576ef9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:45.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.124+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/1399382677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 0x562576ef9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.124+0000 7fc1ddc00640 1 -- 172.21.15.32:0/1399382677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562576efee10 con 0x562576b2e1b0 2024-08-31T07:29:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.124+0000 7fc1de600640 1 --2- 172.21.15.32:0/1399382677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 0x562576ef9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:29:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.125+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/1399382677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b2e1b0 0x562576f01da0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0abbb0 tx=0x7fc1cc00ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef43c4cfd7511bb0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.125+0000 7fc1ca000640 1 -- 172.21.15.32:0/1399382677 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0ab8d0 con 0x562576b2e1b0 2024-08-31T07:29:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.125+0000 7fc1ca000640 1 -- 172.21.15.32:0/1399382677 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1cc0ede70 con 0x562576b2e1b0 2024-08-31T07:29:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.125+0000 7fc1ca000640 1 -- 172.21.15.32:0/1399382677 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0abd50 con 0x562576b2e1b0 2024-08-31T07:29:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.126+0000 7fc1e73eb740 1 -- 172.21.15.32:0/1399382677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b2e1b0 msgr2=0x562576f01da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:45.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.126+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/1399382677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b2e1b0 0x562576f01da0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0abbb0 tx=0x7fc1cc00ce60 comp rx=0 tx=0).stop 2024-08-31T07:29:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.126+0000 7fc1e73eb740 1 -- 172.21.15.32:0/1399382677 shutdown_connections 2024-08-31T07:29:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.126+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/1399382677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 0x562576ef9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.126+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/1399382677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b36390 0x562576f022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.126+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/1399382677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b2e1b0 0x562576f01da0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.126+0000 7fc1e73eb740 1 -- 172.21.15.32:0/1399382677 >> 172.21.15.32:0/1399382677 conn(0x562576b4b460 msgr2=0x562576f06e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.127+0000 7fc1e73eb740 1 -- 172.21.15.32:0/1399382677 shutdown_connections 2024-08-31T07:29:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.127+0000 7fc1e73eb740 1 -- 172.21.15.32:0/1399382677 wait complete. 2024-08-31T07:29:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.127+0000 7fc1e73eb740 1 -- start start 2024-08-31T07:29:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1e73eb740 1 Processor -- start 2024-08-31T07:29:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 0x562576f10590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b36390 0x562576f10ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1e73eb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 0x562576f0b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81b28f0 con 0x562576b57070 2024-08-31T07:29:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81b2220 con 0x562576b2e1b0 2024-08-31T07:29:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1e73eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81af420 con 0x562576b36390 2024-08-31T07:29:45.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1e73eb740 10 client.0 _ll_register_callbacks cb 0x7fc1d0017770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:29:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1ddc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 0x562576f10590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1ddc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 0x562576f10590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55218/0 (socket says 172.21.15.32:55218) 2024-08-31T07:29:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1ddc00640 1 -- 172.21.15.32:0/2040664816 learned_addr learned my addr 172.21.15.32:0/2040664816 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:29:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.128+0000 7fc1de600640 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 0x562576f0b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.129+0000 7fc1dd200640 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b36390 0x562576f10ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.129+0000 7fc1ddc00640 1 -- 172.21.15.32:0/2040664816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b36390 msgr2=0x562576f10ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.129+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b36390 0x562576f10ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.129+0000 7fc1ddc00640 1 -- 172.21.15.32:0/2040664816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 msgr2=0x562576f0b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:29:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.129+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 0x562576f0b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.129+0000 7fc1ddc00640 1 -- 172.21.15.32:0/2040664816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562576f11010 con 0x562576b2e1b0 2024-08-31T07:29:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.148+0000 7fc1ddc00640 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 0x562576f10590 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0e30c0 tx=0x7fc1cc0abf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df438fb01c2401d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:29:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.148+0000 7fc1bf400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:29:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.148+0000 7fc1bf400640 1 -- 172.21.15.32:0/2040664816 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0abd50 con 0x562576b2e1b0 2024-08-31T07:29:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.148+0000 7fc1bf400640 1 -- 172.21.15.32:0/2040664816 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8441+0+0 (secure 0 0 0) 0x7fc1cc0eb6d0 con 0x562576b2e1b0 2024-08-31T07:29:45.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.148+0000 7fc1e73eb740 1 -- 172.21.15.32:0/2040664816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1cc0ede70 con 0x562576b2e1b0 2024-08-31T07:29:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.148+0000 7fc1bf400640 1 -- 172.21.15.32:0/2040664816 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fc1cc0ab8d0 con 0x562576b2e1b0 2024-08-31T07:29:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.148+0000 7fc1e73eb740 10 client.28928 fetch_fsmap learned FSMap version 66 2024-08-31T07:29:45.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.148+0000 7fc1e73eb740 1 -- 172.21.15.32:0/2040664816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x562576f8ddb0 con 0x562576b2e1b0 2024-08-31T07:29:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.149+0000 7fc1bf400640 1 -- 172.21.15.32:0/2040664816 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1cc0ede70 con 0x562576b2e1b0 2024-08-31T07:29:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.149+0000 7fc1bf400640 1 -- 172.21.15.32:0/2040664816 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) ==== 8370+0+0 (secure 0 0 0) 0x7fc1cc0a8210 con 0x562576b2e1b0 2024-08-31T07:29:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.149+0000 7fc1e73eb740 10 client.28928 fetch_fsmap finished waiting for FSMap version 66 2024-08-31T07:29:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.149+0000 7fc1e73eb740 10 client.28928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:29:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.149+0000 7fc1e73eb740 20 client.28928 populate_metadata read hostname 'smithi032' 2024-08-31T07:29:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.149+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562576aeada0 0x562576f8ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:29:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.149+0000 7fc1e73eb740 4 client.28928 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:29:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.149+0000 7fc1e73eb740 1 -- 172.21.15.32:0/2040664816 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562576e6e8a0 con 0x562576aeada0 2024-08-31T07:29:45.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.149+0000 7fc1dd200640 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562576aeada0 0x562576f8ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1dd200640 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562576aeada0 0x562576f8ffe0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e84b18139330214 server_cookie=7dfc0baee4d7b70f in_seq=0 out_seq=0 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1bf400640 10 client.28928 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1bf400640 1 -- 172.21.15.32:0/2040664816 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5972 (crc 0 0 0) 0x562576e6e8a0 con 0x562576aeada0 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1bf400640 10 client.28928 handle_command_reply: tid=0 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1e73eb740 1 client.28928 shutdown 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1e73eb740 1 -- 172.21.15.32:0/2040664816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 msgr2=0x562576f10590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 0x562576f10590 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0e30c0 tx=0x7fc1cc0abf80 comp rx=0 tx=0).stop 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1e73eb740 1 -- 172.21.15.32:0/2040664816 shutdown_connections 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562576aeada0 0x562576f8ffe0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562576b57070 0x562576f0b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562576b36390 0x562576f10ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1bf400640 0 client.28928 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:29:45.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1e73eb740 1 --2- 172.21.15.32:0/2040664816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562576b2e1b0 0x562576f10590 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.150+0000 7fc1e73eb740 1 -- 172.21.15.32:0/2040664816 >> 172.21.15.32:0/2040664816 conn(0x562576b4b460 msgr2=0x562576effc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:45.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.151+0000 7fc1e73eb740 1 -- 172.21.15.32:0/2040664816 shutdown_connections 2024-08-31T07:29:45.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.151+0000 7fc1e73eb740 1 -- 172.21.15.32:0/2040664816 wait complete. 2024-08-31T07:29:45.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.151+0000 7fc1e73eb740 20 client.28928 trim_cache size 0 max 16384 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "scrub active (0 inodes in the stack)", 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "23179d43-8143-476f-8da3-09d7ff58e6e1", 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:29:45.155 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.156 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:29:45.156 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:29:45.156 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:29:45.157 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:29:45.161 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.162 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:29:45.163 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.164 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.165 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:29:45.166 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:29:45.167 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.167 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.167 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.167 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.167 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:29:45.167 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:29:45.167 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:29:45.167 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:29:45.167 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.151+0000 7fc1df000640 1 -- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc1c00b4d20 msgr2=0x7fc1c00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.151+0000 7fc1df000640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc1c00b4d20 0x7fc1c00b7230 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00094c0 tx=0x7fc1d0009970 comp rx=0 tx=0).stop 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.151+0000 7fc1df000640 1 -- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d8075550 msgr2=0x7fc1d819c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.151+0000 7fc1df000640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d8075550 0x7fc1d819c2e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00e9f0 tx=0x7fc1cc00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.152+0000 7fc1df000640 1 -- 172.21.15.32:0/1101586411 shutdown_connections 2024-08-31T07:29:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.152+0000 7fc1df000640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc1c00b4d20 0x7fc1c00b7230 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.152+0000 7fc1df000640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d810fcb0 0x7fc1d81a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.152+0000 7fc1df000640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1d8075ea0 0x7fc1d819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.152+0000 7fc1df000640 1 --2- 172.21.15.32:0/1101586411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1d8075550 0x7fc1d819c2e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:29:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.152+0000 7fc1df000640 1 -- 172.21.15.32:0/1101586411 >> 172.21.15.32:0/1101586411 conn(0x7fc1d80ff1e0 msgr2=0x7fc1d8100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:29:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.152+0000 7fc1df000640 1 -- 172.21.15.32:0/1101586411 shutdown_connections 2024-08-31T07:29:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:29:45.152+0000 7fc1df000640 1 -- 172.21.15.32:0/1101586411 wait complete. 2024-08-31T07:29:45.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65362 9.75 MB/sec execute 85 sec latency 9414.022 ms 2024-08-31T07:29:46.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65416 9.65 MB/sec execute 86 sec latency 10128.460 ms 2024-08-31T07:29:46.429 INFO:tasks.cephfs.filesystem:scrub status for tag:23179d43-8143-476f-8da3-09d7ff58e6e1 - {'path': '/', 'tag': '23179d43-8143-476f-8da3-09d7ff58e6e1', 'options': 'recursive,force'} 2024-08-31T07:29:46.698 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:46 smithi032 ceph-mon[17252]: pgmap v900: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 476 op/s 2024-08-31T07:29:46.698 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:46 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:46 smithi073 ceph-mon[21995]: pgmap v900: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 476 op/s 2024-08-31T07:29:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:46 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:46 smithi084 ceph-mon[22387]: pgmap v900: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 476 op/s 2024-08-31T07:29:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:46 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:29:47.353 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65539 9.60 MB/sec execute 87 sec latency 6295.354 ms 2024-08-31T07:29:48.354 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65797 9.66 MB/sec execute 88 sec latency 7015.999 ms 2024-08-31T07:29:48.606 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:48 smithi032 ceph-mon[17252]: pgmap v901: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.7 MiB/s wr, 436 op/s 2024-08-31T07:29:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:48 smithi073 ceph-mon[21995]: pgmap v901: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.7 MiB/s wr, 436 op/s 2024-08-31T07:29:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:48 smithi084 ceph-mon[22387]: pgmap v901: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.7 MiB/s wr, 436 op/s 2024-08-31T07:29:49.354 INFO:tasks.workunit.client.0.smithi032.stdout: 10 66790 10.05 MB/sec execute 89 sec latency 422.488 ms 2024-08-31T07:29:50.354 INFO:tasks.workunit.client.0.smithi032.stdout: 10 67924 10.76 MB/sec execute 90 sec latency 41.563 ms 2024-08-31T07:29:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:50 smithi073 ceph-mon[21995]: pgmap v902: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 245 op/s 2024-08-31T07:29:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:50 smithi084 ceph-mon[22387]: pgmap v902: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 245 op/s 2024-08-31T07:29:51.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:50 smithi032 ceph-mon[17252]: pgmap v902: 97 pgs: 97 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 245 op/s 2024-08-31T07:29:51.354 INFO:tasks.workunit.client.0.smithi032.stdout: 10 68915 11.14 MB/sec execute 91 sec latency 48.968 ms 2024-08-31T07:29:52.354 INFO:tasks.workunit.client.0.smithi032.stdout: 10 70106 11.73 MB/sec execute 92 sec latency 52.831 ms 2024-08-31T07:29:52.904 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:52 smithi032 ceph-mon[17252]: pgmap v903: 97 pgs: 97 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 326 op/s 2024-08-31T07:29:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:52 smithi073 ceph-mon[21995]: pgmap v903: 97 pgs: 97 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 326 op/s 2024-08-31T07:29:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:52 smithi084 ceph-mon[22387]: pgmap v903: 97 pgs: 97 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 326 op/s 2024-08-31T07:29:53.354 INFO:tasks.workunit.client.0.smithi032.stdout: 10 71205 12.27 MB/sec execute 93 sec latency 69.485 ms 2024-08-31T07:29:54.354 INFO:tasks.workunit.client.0.smithi032.stdout: 10 72010 12.52 MB/sec execute 94 sec latency 95.758 ms 2024-08-31T07:29:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:54 smithi073 ceph-mon[21995]: pgmap v904: 97 pgs: 97 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.3 MiB/s wr, 339 op/s 2024-08-31T07:29:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:54 smithi084 ceph-mon[22387]: pgmap v904: 97 pgs: 97 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.3 MiB/s wr, 339 op/s 2024-08-31T07:29:55.035 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:54 smithi032 ceph-mon[17252]: pgmap v904: 97 pgs: 97 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.3 MiB/s wr, 339 op/s 2024-08-31T07:29:55.354 INFO:tasks.workunit.client.0.smithi032.stdout: 10 73259 13.13 MB/sec execute 95 sec latency 105.977 ms 2024-08-31T07:29:56.355 INFO:tasks.workunit.client.0.smithi032.stdout: 10 74367 13.68 MB/sec execute 96 sec latency 39.446 ms 2024-08-31T07:29:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:56 smithi073 ceph-mon[21995]: pgmap v905: 97 pgs: 97 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 468 op/s 2024-08-31T07:29:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:29:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:29:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:29:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:56 smithi084 ceph-mon[22387]: pgmap v905: 97 pgs: 97 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 468 op/s 2024-08-31T07:29:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:29:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:29:56.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:29:57.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:56 smithi032 ceph-mon[17252]: pgmap v905: 97 pgs: 97 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 468 op/s 2024-08-31T07:29:57.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:29:57.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:29:57.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:29:57.355 INFO:tasks.workunit.client.0.smithi032.stdout: 10 75304 14.04 MB/sec execute 97 sec latency 48.886 ms 2024-08-31T07:29:57.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:57 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:29:57.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:57 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:29:57.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:57 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:29:57.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:57 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:29:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:57 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:29:58.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:57 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:29:58.355 INFO:tasks.workunit.client.0.smithi032.stdout: 10 76534 14.53 MB/sec execute 98 sec latency 36.180 ms 2024-08-31T07:29:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:29:58 smithi073 ceph-mon[21995]: pgmap v906: 97 pgs: 97 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 636 op/s 2024-08-31T07:29:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:29:58 smithi084 ceph-mon[22387]: pgmap v906: 97 pgs: 97 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 636 op/s 2024-08-31T07:29:59.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:29:58 smithi032 ceph-mon[17252]: pgmap v906: 97 pgs: 97 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 636 op/s 2024-08-31T07:29:59.355 INFO:tasks.workunit.client.0.smithi032.stdout: 10 77712 15.12 MB/sec execute 99 sec latency 89.645 ms 2024-08-31T07:30:00.355 INFO:tasks.workunit.client.0.smithi032.stdout: 10 78554 15.32 MB/sec execute 100 sec latency 73.379 ms 2024-08-31T07:30:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:00 smithi073 ceph-mon[21995]: pgmap v907: 97 pgs: 97 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.7 MiB/s wr, 627 op/s 2024-08-31T07:30:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:00 smithi073 ceph-mon[21995]: overall HEALTH_OK 2024-08-31T07:30:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:00 smithi084 ceph-mon[22387]: pgmap v907: 97 pgs: 97 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.7 MiB/s wr, 627 op/s 2024-08-31T07:30:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:00 smithi084 ceph-mon[22387]: overall HEALTH_OK 2024-08-31T07:30:01.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:00 smithi032 ceph-mon[17252]: pgmap v907: 97 pgs: 97 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.7 MiB/s wr, 627 op/s 2024-08-31T07:30:01.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:00 smithi032 ceph-mon[17252]: overall HEALTH_OK 2024-08-31T07:30:01.355 INFO:tasks.workunit.client.0.smithi032.stdout: 10 79016 15.41 MB/sec execute 101 sec latency 564.237 ms 2024-08-31T07:30:02.365 INFO:tasks.workunit.client.0.smithi032.stdout: 10 79016 15.26 MB/sec execute 102 sec latency 1574.604 ms 2024-08-31T07:30:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:02 smithi073 ceph-mon[21995]: pgmap v908: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 769 op/s 2024-08-31T07:30:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:02 smithi084 ceph-mon[22387]: pgmap v908: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 769 op/s 2024-08-31T07:30:03.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:02 smithi032 ceph-mon[17252]: pgmap v908: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 769 op/s 2024-08-31T07:30:03.365 INFO:tasks.workunit.client.0.smithi032.stdout: 10 79070 15.14 MB/sec execute 103 sec latency 2564.694 ms 2024-08-31T07:30:04.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 80027 15.49 MB/sec execute 104 sec latency 2809.111 ms 2024-08-31T07:30:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:04 smithi073 ceph-mon[21995]: pgmap v909: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 782 op/s 2024-08-31T07:30:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:04 smithi084 ceph-mon[22387]: pgmap v909: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 782 op/s 2024-08-31T07:30:05.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:04 smithi032 ceph-mon[17252]: pgmap v909: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 782 op/s 2024-08-31T07:30:05.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 81027 15.90 MB/sec execute 105 sec latency 125.989 ms 2024-08-31T07:30:06.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82007 16.22 MB/sec execute 106 sec latency 56.424 ms 2024-08-31T07:30:06.914 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:06 smithi032 ceph-mon[17252]: pgmap v910: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 789 op/s 2024-08-31T07:30:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:06 smithi073 ceph-mon[21995]: pgmap v910: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 789 op/s 2024-08-31T07:30:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:06 smithi084 ceph-mon[22387]: pgmap v910: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 789 op/s 2024-08-31T07:30:07.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82208 16.17 MB/sec execute 107 sec latency 747.519 ms 2024-08-31T07:30:07.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:07 smithi032 ceph-mon[17252]: osdmap e103: 12 total, 12 up, 12 in 2024-08-31T07:30:07.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:07 smithi084 ceph-mon[22387]: osdmap e103: 12 total, 12 up, 12 in 2024-08-31T07:30:08.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:07 smithi073 ceph-mon[21995]: osdmap e103: 12 total, 12 up, 12 in 2024-08-31T07:30:08.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82208 16.02 MB/sec execute 108 sec latency 1747.571 ms 2024-08-31T07:30:08.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:08 smithi032 ceph-mon[17252]: pgmap v912: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 691 op/s 2024-08-31T07:30:08.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:08 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:30:08.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:08 smithi084 ceph-mon[22387]: pgmap v912: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 691 op/s 2024-08-31T07:30:08.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:08 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:30:09.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:08 smithi073 ceph-mon[21995]: pgmap v912: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 691 op/s 2024-08-31T07:30:09.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:08 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:30:09.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82209 15.87 MB/sec execute 109 sec latency 2586.395 ms 2024-08-31T07:30:10.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82209 15.73 MB/sec execute 110 sec latency 3586.447 ms 2024-08-31T07:30:10.917 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:10 smithi032 ceph-mon[17252]: pgmap v913: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.7 MiB/s wr, 682 op/s 2024-08-31T07:30:10.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:10 smithi084 ceph-mon[22387]: pgmap v913: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.7 MiB/s wr, 682 op/s 2024-08-31T07:30:11.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:10 smithi073 ceph-mon[21995]: pgmap v913: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.7 MiB/s wr, 682 op/s 2024-08-31T07:30:11.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82209 15.58 MB/sec execute 111 sec latency 4586.498 ms 2024-08-31T07:30:12.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82216 15.45 MB/sec execute 112 sec latency 5586.553 ms 2024-08-31T07:30:12.919 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:12 smithi032 ceph-mon[17252]: pgmap v914: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 626 op/s 2024-08-31T07:30:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:12 smithi084 ceph-mon[22387]: pgmap v914: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 626 op/s 2024-08-31T07:30:13.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:12 smithi073 ceph-mon[21995]: pgmap v914: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 626 op/s 2024-08-31T07:30:13.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82217 15.31 MB/sec execute 113 sec latency 6586.604 ms 2024-08-31T07:30:14.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82296 15.24 MB/sec execute 114 sec latency 7586.654 ms 2024-08-31T07:30:14.822 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:14 smithi032 ceph-mon[17252]: pgmap v915: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.9 MiB/s wr, 504 op/s 2024-08-31T07:30:14.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:14 smithi084 ceph-mon[22387]: pgmap v915: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.9 MiB/s wr, 504 op/s 2024-08-31T07:30:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:14 smithi073 ceph-mon[21995]: pgmap v915: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.9 MiB/s wr, 504 op/s 2024-08-31T07:30:15.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82534 15.22 MB/sec execute 115 sec latency 8586.702 ms 2024-08-31T07:30:16.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 83422 15.49 MB/sec execute 116 sec latency 9586.767 ms 2024-08-31T07:30:16.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:30:16.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:16 smithi084 ceph-mon[22387]: pgmap v916: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.6 MiB/s wr, 413 op/s 2024-08-31T07:30:16.991 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:16 smithi032 ceph-mon[17252]: pgmap v916: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.6 MiB/s wr, 413 op/s 2024-08-31T07:30:17.163 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:17.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:16 smithi073 ceph-mon[21995]: pgmap v916: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.6 MiB/s wr, 413 op/s 2024-08-31T07:30:17.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 83857 15.55 MB/sec execute 117 sec latency 10586.865 ms 2024-08-31T07:30:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:17 smithi084 ceph-mon[22387]: osdmap e104: 12 total, 12 up, 12 in 2024-08-31T07:30:18.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:17 smithi084 ceph-mon[22387]: pgmap v918: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.4 MiB/s wr, 329 op/s 2024-08-31T07:30:18.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:17 smithi032 ceph-mon[17252]: osdmap e104: 12 total, 12 up, 12 in 2024-08-31T07:30:18.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:17 smithi032 ceph-mon[17252]: pgmap v918: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.4 MiB/s wr, 329 op/s 2024-08-31T07:30:18.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84251 15.61 MB/sec execute 118 sec latency 11499.716 ms 2024-08-31T07:30:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:17 smithi073 ceph-mon[21995]: osdmap e104: 12 total, 12 up, 12 in 2024-08-31T07:30:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:17 smithi073 ceph-mon[21995]: pgmap v918: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.4 MiB/s wr, 329 op/s 2024-08-31T07:30:19.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84356 15.56 MB/sec execute 119 sec latency 12499.760 ms 2024-08-31T07:30:20.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84358 15.43 MB/sec execute 120 sec latency 13499.805 ms 2024-08-31T07:30:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:20 smithi084 ceph-mon[22387]: pgmap v919: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 336 op/s 2024-08-31T07:30:20.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:20 smithi032 ceph-mon[17252]: pgmap v919: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 336 op/s 2024-08-31T07:30:20.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:20 smithi073 ceph-mon[21995]: pgmap v919: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 336 op/s 2024-08-31T07:30:21.366 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84454 15.38 MB/sec execute 121 sec latency 14499.847 ms 2024-08-31T07:30:22.335 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:22 smithi032 ceph-mon[17252]: pgmap v920: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 334 op/s 2024-08-31T07:30:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.351+0000 7f936a000640 1 -- 172.21.15.32:0/2769675043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93640776e0 msgr2=0x7f9364077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.351+0000 7f936a000640 1 --2- 172.21.15.32:0/2769675043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93640776e0 0x7f9364077af0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9354009900 tx=0x7f935402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.351+0000 7f936a000640 1 -- 172.21.15.32:0/2769675043 shutdown_connections 2024-08-31T07:30:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.351+0000 7f936a000640 1 --2- 172.21.15.32:0/2769675043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9364114320 0x7f9364116740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.351+0000 7f936a000640 1 --2- 172.21.15.32:0/2769675043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9364078030 0x7f9364113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.351+0000 7f936a000640 1 --2- 172.21.15.32:0/2769675043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93640776e0 0x7f9364077af0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.351+0000 7f936a000640 1 -- 172.21.15.32:0/2769675043 >> 172.21.15.32:0/2769675043 conn(0x7f93641013d0 msgr2=0x7f9364103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.351+0000 7f936a000640 1 -- 172.21.15.32:0/2769675043 shutdown_connections 2024-08-31T07:30:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.352+0000 7f936a000640 1 -- 172.21.15.32:0/2769675043 wait complete. 2024-08-31T07:30:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.353+0000 7f936a000640 1 Processor -- start 2024-08-31T07:30:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.353+0000 7f936a000640 1 -- start start 2024-08-31T07:30:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.354+0000 7f936a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93640776e0 0x7f93641a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.354+0000 7f936a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9364078030 0x7f93641a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.354+0000 7f936a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9364114320 0x7f93641a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.354+0000 7f936a000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9364119d50 con 0x7f9364114320 2024-08-31T07:30:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.354+0000 7f936a000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9364119bd0 con 0x7f9364078030 2024-08-31T07:30:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.354+0000 7f936a000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9364119ed0 con 0x7f93640776e0 2024-08-31T07:30:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.354+0000 7f9368c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93640776e0 0x7f93641a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.354+0000 7f9368c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93640776e0 0x7f93641a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:32904/0 (socket says 172.21.15.32:32904) 2024-08-31T07:30:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.354+0000 7f9368c00640 1 -- 172.21.15.32:0/2093241531 learned_addr learned my addr 172.21.15.32:0/2093241531 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.355+0000 7f9368c00640 1 -- 172.21.15.32:0/2093241531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9364078030 msgr2=0x7f93641a0ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:30:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.355+0000 7f9363e00640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9364078030 0x7f93641a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.355+0000 7f9368c00640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9364078030 0x7f93641a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.355+0000 7f9368c00640 1 -- 172.21.15.32:0/2093241531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9364114320 msgr2=0x7f93641a5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.355+0000 7f9368c00640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9364114320 0x7f93641a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.355+0000 7f9368c00640 1 -- 172.21.15.32:0/2093241531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93641a66b0 con 0x7f93640776e0 2024-08-31T07:30:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.355+0000 7f9368c00640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93640776e0 0x7f93641a0740 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9354005ab0 tx=0x7f9354003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:22.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.358+0000 7f9361600640 1 -- 172.21.15.32:0/2093241531 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935403f070 con 0x7f93640776e0 2024-08-31T07:30:22.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.359+0000 7f9361600640 1 -- 172.21.15.32:0/2093241531 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9354031d60 con 0x7f93640776e0 2024-08-31T07:30:22.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.359+0000 7f936a000640 1 -- 172.21.15.32:0/2093241531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93641a6940 con 0x7f93640776e0 2024-08-31T07:30:22.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.359+0000 7f9361600640 1 -- 172.21.15.32:0/2093241531 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354035030 con 0x7f93640776e0 2024-08-31T07:30:22.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.359+0000 7f936a000640 1 -- 172.21.15.32:0/2093241531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93641ad100 con 0x7f93640776e0 2024-08-31T07:30:22.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.364+0000 7f9372239740 1 Processor -- start 2024-08-31T07:30:22.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.365+0000 7f9372239740 1 -- start start 2024-08-31T07:30:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.365+0000 7f9372239740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65aab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.366+0000 7f9372239740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.366+0000 7f9372239740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.366+0000 7f9372239740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a5d6521770 con 0x55a5d6205070 2024-08-31T07:30:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.366+0000 7f9372239740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a5d6510d00 con 0x55a5d61dc1b0 2024-08-31T07:30:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.366+0000 7f9372239740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a5d6514cb0 con 0x55a5d61e4390 2024-08-31T07:30:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.366+0000 7f9369600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.366+0000 7f9369600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65aab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34634/0 (socket says 172.21.15.32:34634) 2024-08-31T07:30:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.366+0000 7f9369600640 1 -- 172.21.15.32:0/693211102 learned_addr learned my addr 172.21.15.32:0/693211102 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.367+0000 7f9369600640 1 -- 172.21.15.32:0/693211102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 msgr2=0x55a5d65ab5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:30:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.367+0000 7f9363e00640 1 --2- 172.21.15.32:0/693211102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.367+0000 7f9369600640 1 --2- 172.21.15.32:0/693211102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65ab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.367+0000 7f9369600640 1 -- 172.21.15.32:0/693211102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 msgr2=0x55a5d65ab060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:30:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.367+0000 7f9369600640 1 --2- 172.21.15.32:0/693211102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ab060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.367+0000 7f9369600640 1 -- 172.21.15.32:0/693211102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d6270650 con 0x55a5d6205070 2024-08-31T07:30:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.368+0000 7f9369600640 1 --2- 172.21.15.32:0/693211102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65aab20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f935c009f60 tx=0x7f935c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0c8c3ed1683a71 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.368+0000 7f9359c00640 1 -- 172.21.15.32:0/693211102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c00bca0 con 0x55a5d6205070 2024-08-31T07:30:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.368+0000 7f9359c00640 1 -- 172.21.15.32:0/693211102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c017070 con 0x55a5d6205070 2024-08-31T07:30:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.368+0000 7f9359c00640 1 -- 172.21.15.32:0/693211102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c011750 con 0x55a5d6205070 2024-08-31T07:30:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.370+0000 7f9372239740 1 -- 172.21.15.32:0/693211102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 msgr2=0x55a5d65aab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.370+0000 7f9372239740 1 --2- 172.21.15.32:0/693211102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65aab20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f935c009f60 tx=0x7f935c009f90 comp rx=0 tx=0).stop 2024-08-31T07:30:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.370+0000 7f9372239740 1 -- 172.21.15.32:0/693211102 shutdown_connections 2024-08-31T07:30:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.370+0000 7f9372239740 1 --2- 172.21.15.32:0/693211102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65ab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.370+0000 7f9372239740 1 --2- 172.21.15.32:0/693211102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.370+0000 7f9372239740 1 --2- 172.21.15.32:0/693211102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65aab20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.371+0000 7f9372239740 1 -- 172.21.15.32:0/693211102 >> 172.21.15.32:0/693211102 conn(0x55a5d61f9460 msgr2=0x55a5d65a8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:22.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.371+0000 7f9372239740 1 -- 172.21.15.32:0/693211102 shutdown_connections 2024-08-31T07:30:22.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.371+0000 7f9372239740 1 -- 172.21.15.32:0/693211102 wait complete. 2024-08-31T07:30:22.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.372+0000 7f9372239740 1 -- start start 2024-08-31T07:30:22.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.373+0000 7f9372239740 1 Processor -- start 2024-08-31T07:30:22.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.374+0000 7f9372239740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61dc1b0 0x55a5d65b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.374+0000 7f9372239740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61e4390 0x55a5d65b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.374+0000 7f9372239740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.374+0000 7f9372239740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a5d6373700 con 0x55a5d6205070 2024-08-31T07:30:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.374+0000 7f9372239740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a5d6497cb0 con 0x55a5d61e4390 2024-08-31T07:30:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.375+0000 7f9372239740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a5d65b9300 con 0x55a5d61dc1b0 2024-08-31T07:30:22.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.375+0000 7f9372239740 10 client.0 _ll_register_callbacks cb 0x55a5d64bd430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:22.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.375+0000 7f9369600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61dc1b0 0x55a5d65b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.375+0000 7f9369600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61dc1b0 0x55a5d65b6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:32924/0 (socket says 172.21.15.32:32924) 2024-08-31T07:30:22.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.375+0000 7f9369600640 1 -- 172.21.15.32:0/3836794768 learned_addr learned my addr 172.21.15.32:0/3836794768 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:22.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.376+0000 7f9363e00640 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.377+0000 7f9369600640 1 -- 172.21.15.32:0/3836794768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61e4390 msgr2=0x55a5d65b6de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:30:22.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.377+0000 7f9369600640 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61e4390 0x55a5d65b6de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.377+0000 7f9369600640 1 -- 172.21.15.32:0/3836794768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 msgr2=0x55a5d65c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:22.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.378+0000 7f9369600640 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.378+0000 7f9369600640 1 -- 172.21.15.32:0/3836794768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d64d8870 con 0x55a5d61dc1b0 2024-08-31T07:30:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.378+0000 7f9363e00640 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.379+0000 7f9361600640 1 -- 172.21.15.32:0/2093241531 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9354042070 con 0x7f93640776e0 2024-08-31T07:30:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.379+0000 7f9369600640 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61dc1b0 0x55a5d65b6800 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f935c012ec0 tx=0x7f935c012c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b3e123603443963 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.379+0000 7f934e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:30:22.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.380+0000 7f934e000640 1 -- 172.21.15.32:0/3836794768 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c022030 con 0x55a5d61dc1b0 2024-08-31T07:30:22.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.380+0000 7f934e000640 1 -- 172.21.15.32:0/3836794768 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f935c00f040 con 0x55a5d61dc1b0 2024-08-31T07:30:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.380+0000 7f9372239740 1 -- 172.21.15.32:0/3836794768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f935c017070 con 0x55a5d61dc1b0 2024-08-31T07:30:22.382 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84546 15.30 MB/sec execute 122 sec latency 15515.088 ms 2024-08-31T07:30:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.381+0000 7f934e000640 1 -- 172.21.15.32:0/3836794768 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f935c01b630 con 0x55a5d61dc1b0 2024-08-31T07:30:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.381+0000 7f9372239740 10 client.28939 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.381+0000 7f9372239740 1 -- 172.21.15.32:0/3836794768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55a5d65c53c0 con 0x55a5d61dc1b0 2024-08-31T07:30:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.382+0000 7f9361600640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93480b4cb0 0x7f93480b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.382+0000 7f9361600640 1 -- 172.21.15.32:0/2093241531 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f9354105cd0 con 0x7f93640776e0 2024-08-31T07:30:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.382+0000 7f934e000640 1 -- 172.21.15.32:0/3836794768 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c017070 con 0x55a5d61dc1b0 2024-08-31T07:30:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.383+0000 7f934e000640 1 -- 172.21.15.32:0/3836794768 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f935c011070 con 0x55a5d61dc1b0 2024-08-31T07:30:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.383+0000 7f9372239740 10 client.28939 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.383+0000 7f9372239740 10 client.28939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:30:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.383+0000 7f9372239740 20 client.28939 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.383+0000 7f9372239740 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a5d61c39f0 0x55a5d65c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.383+0000 7f9372239740 4 client.28939 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:30:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.383+0000 7f9372239740 1 -- 172.21.15.32:0/3836794768 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5d65236c0 con 0x55a5d61c39f0 2024-08-31T07:30:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.383+0000 7f9368c00640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93480b4cb0 0x7f93480b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.384+0000 7f9363e00640 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a5d61c39f0 0x55a5d65c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.385+0000 7f9368c00640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93480b4cb0 0x7f93480b71c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f93540c8430 tx=0x7f9354031f10 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.385+0000 7f934e000640 10 client.28939 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.385+0000 7f9363e00640 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a5d61c39f0 0x55a5d65c8720 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e412c6a78a002e server_cookie=ad957caf2c0656d4 in_seq=0 out_seq=0 2024-08-31T07:30:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.386+0000 7f934e000640 1 -- 172.21.15.32:0/3836794768 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5d65236c0 con 0x55a5d61c39f0 2024-08-31T07:30:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.386+0000 7f934e000640 10 client.28939 handle_command_reply: tid=0 2024-08-31T07:30:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.386+0000 7f9372239740 1 client.28939 shutdown 2024-08-31T07:30:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 1 -- 172.21.15.32:0/3836794768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61dc1b0 msgr2=0x55a5d65b6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61dc1b0 0x55a5d65b6800 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f935c012ec0 tx=0x7f935c012c70 comp rx=0 tx=0).stop 2024-08-31T07:30:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 1 -- 172.21.15.32:0/3836794768 shutdown_connections 2024-08-31T07:30:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a5d61c39f0 0x55a5d65c8720 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61e4390 0x55a5d65b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 1 --2- 172.21.15.32:0/3836794768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61dc1b0 0x55a5d65b6800 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 1 -- 172.21.15.32:0/3836794768 >> 172.21.15.32:0/3836794768 conn(0x55a5d61f9460 msgr2=0x55a5d65add00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:22.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f934e000640 0 client.28939 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:22.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 1 -- 172.21.15.32:0/3836794768 shutdown_connections 2024-08-31T07:30:22.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 1 -- 172.21.15.32:0/3836794768 wait complete. 2024-08-31T07:30:22.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.387+0000 7f9372239740 20 client.28939 trim_cache size 0 max 16384 2024-08-31T07:30:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.420+0000 7f9372239740 1 Processor -- start 2024-08-31T07:30:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.420+0000 7f9372239740 1 -- start start 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9372239740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9372239740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9372239740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9372239740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93641b58d0 con 0x55a5d6205070 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9372239740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93641b7470 con 0x55a5d61dc1b0 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9372239740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93641b5fd0 con 0x55a5d61e4390 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9369600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9363e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9369600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ac260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49144/0 (socket says 172.21.15.32:49144) 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9363e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65c24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:32928/0 (socket says 172.21.15.32:32928) 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9369600640 1 -- 172.21.15.32:0/1583022625 learned_addr learned my addr 172.21.15.32:0/1583022625 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9369600640 1 --2- 172.21.15.32:0/1583022625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9369600640 1 -- 172.21.15.32:0/1583022625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 msgr2=0x55a5d65c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:22.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9369600640 1 --2- 172.21.15.32:0/1583022625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65c24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9369600640 1 -- 172.21.15.32:0/1583022625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 msgr2=0x55a5d65c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:22.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9369600640 1 --2- 172.21.15.32:0/1583022625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65c2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.421+0000 7f9369600640 1 -- 172.21.15.32:0/1583022625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d65a2c50 con 0x55a5d61dc1b0 2024-08-31T07:30:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.422+0000 7f9369600640 1 --2- 172.21.15.32:0/1583022625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ac260 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f935c007160 tx=0x7f935c012930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4dbb47f94e221a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.422+0000 7f9359200640 1 -- 172.21.15.32:0/1583022625 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c01f040 con 0x55a5d61dc1b0 2024-08-31T07:30:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.422+0000 7f9359200640 1 -- 172.21.15.32:0/1583022625 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c017040 con 0x55a5d61dc1b0 2024-08-31T07:30:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.422+0000 7f9359200640 1 -- 172.21.15.32:0/1583022625 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c022040 con 0x55a5d61dc1b0 2024-08-31T07:30:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.424+0000 7f9372239740 1 -- 172.21.15.32:0/1583022625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 msgr2=0x55a5d65ac260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.424+0000 7f9372239740 1 --2- 172.21.15.32:0/1583022625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ac260 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f935c007160 tx=0x7f935c012930 comp rx=0 tx=0).stop 2024-08-31T07:30:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.424+0000 7f9372239740 1 -- 172.21.15.32:0/1583022625 shutdown_connections 2024-08-31T07:30:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.424+0000 7f9372239740 1 --2- 172.21.15.32:0/1583022625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.424+0000 7f9372239740 1 --2- 172.21.15.32:0/1583022625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.424+0000 7f9372239740 1 --2- 172.21.15.32:0/1583022625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ac260 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.424+0000 7f9372239740 1 -- 172.21.15.32:0/1583022625 >> 172.21.15.32:0/1583022625 conn(0x55a5d61f9460 msgr2=0x55a5d65abb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.425+0000 7f9372239740 1 -- 172.21.15.32:0/1583022625 shutdown_connections 2024-08-31T07:30:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.425+0000 7f9372239740 1 -- 172.21.15.32:0/1583022625 wait complete. 2024-08-31T07:30:22.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.426+0000 7f9372239740 1 -- start start 2024-08-31T07:30:22.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.426+0000 7f9372239740 1 Processor -- start 2024-08-31T07:30:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9372239740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9372239740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65baa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9372239740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9372239740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93641b6da0 con 0x55a5d6205070 2024-08-31T07:30:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9372239740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93641b66d0 con 0x55a5d61dc1b0 2024-08-31T07:30:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9372239740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93641b3920 con 0x55a5d61e4390 2024-08-31T07:30:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9372239740 10 client.0 _ll_register_callbacks cb 0x7f935c012880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9363e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65baa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9369600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9363e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65baa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:32936/0 (socket says 172.21.15.32:32936) 2024-08-31T07:30:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.427+0000 7f9363e00640 1 -- 172.21.15.32:0/153588737 learned_addr learned my addr 172.21.15.32:0/153588737 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.428+0000 7f9369600640 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.428+0000 7f9363e00640 1 -- 172.21.15.32:0/153588737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 msgr2=0x55a5d65ba500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.428+0000 7f9363e00640 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ba500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.428+0000 7f9363e00640 1 -- 172.21.15.32:0/153588737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 msgr2=0x55a5d65baf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.428+0000 7f9363e00640 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65baf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.428+0000 7f9363e00640 1 -- 172.21.15.32:0/153588737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d6631ec0 con 0x55a5d61e4390 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.428+0000 7f9369600640 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65baf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.428+0000 7f9363e00640 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65baa40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f936406b6d0 tx=0x7f9350011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7fa92dc67981325 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.428+0000 7f934e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.429+0000 7f934e000640 1 -- 172.21.15.32:0/153588737 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935001b870 con 0x55a5d61e4390 2024-08-31T07:30:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.429+0000 7f934e000640 1 -- 172.21.15.32:0/153588737 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f935001ba10 con 0x55a5d61e4390 2024-08-31T07:30:22.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.429+0000 7f9372239740 1 -- 172.21.15.32:0/153588737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f935c017040 con 0x55a5d61e4390 2024-08-31T07:30:22.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.429+0000 7f934e000640 1 -- 172.21.15.32:0/153588737 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f9350027020 con 0x55a5d61e4390 2024-08-31T07:30:22.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.429+0000 7f9372239740 10 client.28945 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:22.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.429+0000 7f9372239740 1 -- 172.21.15.32:0/153588737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55a5d66331b0 con 0x55a5d61e4390 2024-08-31T07:30:22.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.429+0000 7f934e000640 1 -- 172.21.15.32:0/153588737 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c017040 con 0x55a5d61e4390 2024-08-31T07:30:22.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.430+0000 7f934e000640 1 -- 172.21.15.32:0/153588737 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f9350023400 con 0x55a5d61e4390 2024-08-31T07:30:22.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.430+0000 7f9372239740 10 client.28945 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.430+0000 7f9372239740 10 client.28945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.430+0000 7f9372239740 20 client.28945 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.430+0000 7f9372239740 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a5d6198da0 0x55a5d6635980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.430+0000 7f9372239740 4 client.28945 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.430+0000 7f9372239740 1 -- 172.21.15.32:0/153588737 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5d651f160 con 0x55a5d6198da0 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.430+0000 7f9369600640 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a5d6198da0 0x55a5d6635980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f934e000640 10 client.28945 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f9369600640 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a5d6198da0 0x55a5d6635980 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=648ae54a237cd4c4 server_cookie=1a9d2b609ea189fe in_seq=0 out_seq=0 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f934e000640 1 -- 172.21.15.32:0/153588737 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5770 (crc 0 0 0) 0x55a5d651f160 con 0x55a5d6198da0 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f934e000640 10 client.28945 handle_command_reply: tid=0 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f9372239740 1 client.28945 shutdown 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f9372239740 1 -- 172.21.15.32:0/153588737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 msgr2=0x55a5d65baa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f9372239740 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65baa40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f936406b6d0 tx=0x7f9350011810 comp rx=0 tx=0).stop 2024-08-31T07:30:22.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f9372239740 1 -- 172.21.15.32:0/153588737 shutdown_connections 2024-08-31T07:30:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f9372239740 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a5d6198da0 0x55a5d6635980 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f9372239740 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a5d6205070 0x55a5d65baf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f9372239740 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a5d61e4390 0x55a5d65baa40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f9372239740 1 --2- 172.21.15.32:0/153588737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5d61dc1b0 0x55a5d65ba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f934e000640 0 client.28945 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.431+0000 7f9372239740 1 -- 172.21.15.32:0/153588737 >> 172.21.15.32:0/153588737 conn(0x55a5d61f9460 msgr2=0x55a5d65abb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.432+0000 7f9372239740 1 -- 172.21.15.32:0/153588737 shutdown_connections 2024-08-31T07:30:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.432+0000 7f9372239740 1 -- 172.21.15.32:0/153588737 wait complete. 2024-08-31T07:30:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.432+0000 7f9372239740 20 client.28945 trim_cache size 0 max 16384 2024-08-31T07:30:22.435 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:30:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:30:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:30:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:30:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:30:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:30:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:30:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:30:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.432+0000 7f936a000640 1 -- 172.21.15.32:0/2093241531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93480b4cb0 msgr2=0x7f93480b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.432+0000 7f936a000640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93480b4cb0 0x7f93480b71c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f93540c8430 tx=0x7f9354031f10 comp rx=0 tx=0).stop 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.432+0000 7f936a000640 1 -- 172.21.15.32:0/2093241531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93640776e0 msgr2=0x7f93641a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.432+0000 7f936a000640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93640776e0 0x7f93641a0740 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9354005ab0 tx=0x7f9354003cd0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.433+0000 7f936a000640 1 -- 172.21.15.32:0/2093241531 shutdown_connections 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.433+0000 7f936a000640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f93480b4cb0 0x7f93480b71c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.433+0000 7f936a000640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9364114320 0x7f93641a5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.433+0000 7f936a000640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9364078030 0x7f93641a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.433+0000 7f936a000640 1 --2- 172.21.15.32:0/2093241531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93640776e0 0x7f93641a0740 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.433+0000 7f936a000640 1 -- 172.21.15.32:0/2093241531 >> 172.21.15.32:0/2093241531 conn(0x7f93641013d0 msgr2=0x7f93641163a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.433+0000 7f936a000640 1 -- 172.21.15.32:0/2093241531 shutdown_connections 2024-08-31T07:30:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:22.433+0000 7f936a000640 1 -- 172.21.15.32:0/2093241531 wait complete. 2024-08-31T07:30:22.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:22 smithi084 ceph-mon[22387]: pgmap v920: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 334 op/s 2024-08-31T07:30:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:22 smithi073 ceph-mon[21995]: pgmap v920: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 334 op/s 2024-08-31T07:30:23.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84546 15.18 MB/sec execute 123 sec latency 16522.112 ms 2024-08-31T07:30:23.824 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:30:23.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:30:24.215 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:24.316 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:24 smithi032 ceph-mon[17252]: pgmap v921: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 350 op/s 2024-08-31T07:30:24.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84546 15.06 MB/sec execute 124 sec latency 17522.192 ms 2024-08-31T07:30:24.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:24 smithi084 ceph-mon[22387]: pgmap v921: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 350 op/s 2024-08-31T07:30:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:24 smithi073 ceph-mon[21995]: pgmap v921: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 350 op/s 2024-08-31T07:30:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.127+0000 7f2f13400640 1 -- 172.21.15.32:0/2291165034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c109ba0 msgr2=0x7f2f0c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.127+0000 7f2f13400640 1 --2- 172.21.15.32:0/2291165034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c109ba0 0x7f2f0c1103d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009900 tx=0x7f2ef802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:30:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.127+0000 7f2f13400640 1 -- 172.21.15.32:0/2291165034 shutdown_connections 2024-08-31T07:30:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.127+0000 7f2f13400640 1 --2- 172.21.15.32:0/2291165034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c109ba0 0x7f2f0c1103d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.127+0000 7f2f13400640 1 --2- 172.21.15.32:0/2291165034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c109250 0x7f2f0c109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.127+0000 7f2f13400640 1 --2- 172.21.15.32:0/2291165034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c103960 0x7f2f0c103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.127+0000 7f2f13400640 1 -- 172.21.15.32:0/2291165034 >> 172.21.15.32:0/2291165034 conn(0x7f2f0c07b6b0 msgr2=0x7f2f0c07bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.127+0000 7f2f13400640 1 -- 172.21.15.32:0/2291165034 shutdown_connections 2024-08-31T07:30:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.127+0000 7f2f13400640 1 -- 172.21.15.32:0/2291165034 wait complete. 2024-08-31T07:30:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f13400640 1 Processor -- start 2024-08-31T07:30:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f13400640 1 -- start start 2024-08-31T07:30:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f13400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c103960 0x7f2f0c1a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f13400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c109250 0x7f2f0c1a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f13400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c109ba0 0x7f2f0c1ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f13400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c06d440 con 0x7f2f0c109ba0 2024-08-31T07:30:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f13400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c06d2c0 con 0x7f2f0c109250 2024-08-31T07:30:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f13400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c06d5c0 con 0x7f2f0c103960 2024-08-31T07:30:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f10a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c103960 0x7f2f0c1a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f10a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c103960 0x7f2f0c1a9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:32968/0 (socket says 172.21.15.32:32968) 2024-08-31T07:30:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f10a00640 1 -- 172.21.15.32:0/2542475323 learned_addr learned my addr 172.21.15.32:0/2542475323 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f0be00640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c109250 0x7f2f0c1a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f11400640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c109ba0 0x7f2f0c1ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f10a00640 1 -- 172.21.15.32:0/2542475323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c109250 msgr2=0x7f2f0c1a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f10a00640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c109250 0x7f2f0c1a95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f10a00640 1 -- 172.21.15.32:0/2542475323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c109ba0 msgr2=0x7f2f0c1ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f10a00640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c109ba0 0x7f2f0c1ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f10a00640 1 -- 172.21.15.32:0/2542475323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c1aef90 con 0x7f2f0c103960 2024-08-31T07:30:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f10a00640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c103960 0x7f2f0c1a9020 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400cce0 tx=0x7f2f0400a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f09600640 1 -- 172.21.15.32:0/2542475323 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04004b20 con 0x7f2f0c103960 2024-08-31T07:30:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f09600640 1 -- 172.21.15.32:0/2542475323 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f04004cc0 con 0x7f2f0c103960 2024-08-31T07:30:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.133+0000 7f2f09600640 1 -- 172.21.15.32:0/2542475323 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400f7c0 con 0x7f2f0c103960 2024-08-31T07:30:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.138+0000 7f2f13400640 1 -- 172.21.15.32:0/2542475323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0c1af280 con 0x7f2f0c103960 2024-08-31T07:30:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.138+0000 7f2f13400640 1 -- 172.21.15.32:0/2542475323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0c1b59e0 con 0x7f2f0c103960 2024-08-31T07:30:25.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.138+0000 7f2f13400640 1 -- 172.21.15.32:0/2542475323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0c103d70 con 0x7f2f0c103960 2024-08-31T07:30:25.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.145+0000 7f2f09600640 1 -- 172.21.15.32:0/2542475323 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2f04005560 con 0x7f2f0c103960 2024-08-31T07:30:25.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.145+0000 7f2f09600640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4dd0 0x7f2ef00b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:25.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.145+0000 7f2f09600640 1 -- 172.21.15.32:0/2542475323 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f2f040e55a0 con 0x7f2f0c103960 2024-08-31T07:30:25.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.150+0000 7f2f09600640 1 -- 172.21.15.32:0/2542475323 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2f04014030 con 0x7f2f0c103960 2024-08-31T07:30:25.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.150+0000 7f2f0be00640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4dd0 0x7f2ef00b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:25.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.151+0000 7f2f0be00640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4dd0 0x7f2ef00b72e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c07bbe0 tx=0x7f2efc00b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:25.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84546 14.94 MB/sec execute 125 sec latency 18522.242 ms 2024-08-31T07:30:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.407+0000 7f2f13400640 1 -- 172.21.15.32:0/2542475323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f0c1aa150 con 0x7f2f0c103960 2024-08-31T07:30:25.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.407+0000 7f2f09600640 1 -- 172.21.15.32:0/2542475323 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+11963 (secure 0 0 0) 0x7f2f040ac400 con 0x7f2f0c103960 2024-08-31T07:30:25.409 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:30:25.409 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"btime":"2024-08-31T07:30:07:717453+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":67,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:30:07.599620+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:30:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 67 2024-08-31T07:30:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 -- 172.21.15.32:0/2542475323 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4dd0 msgr2=0x7f2ef00b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4dd0 0x7f2ef00b72e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c07bbe0 tx=0x7f2efc00b410 comp rx=0 tx=0).stop 2024-08-31T07:30:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 -- 172.21.15.32:0/2542475323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c103960 msgr2=0x7f2f0c1a9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c103960 0x7f2f0c1a9020 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400cce0 tx=0x7f2f0400a620 comp rx=0 tx=0).stop 2024-08-31T07:30:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 -- 172.21.15.32:0/2542475323 shutdown_connections 2024-08-31T07:30:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4dd0 0x7f2ef00b72e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c109ba0 0x7f2f0c1ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c109250 0x7f2f0c1a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 --2- 172.21.15.32:0/2542475323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c103960 0x7f2f0c1a9020 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 -- 172.21.15.32:0/2542475323 >> 172.21.15.32:0/2542475323 conn(0x7f2f0c07b6b0 msgr2=0x7f2f0c076410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 -- 172.21.15.32:0/2542475323 shutdown_connections 2024-08-31T07:30:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:25.410+0000 7f2f13400640 1 -- 172.21.15.32:0/2542475323 wait complete. 2024-08-31T07:30:26.258 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:26 smithi032 ceph-mon[17252]: pgmap v922: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 436 op/s 2024-08-31T07:30:26.259 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:26 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2542475323' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:30:26.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 14.82 MB/sec execute 126 sec latency 19522.315 ms 2024-08-31T07:30:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:26 smithi084 ceph-mon[22387]: pgmap v922: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 436 op/s 2024-08-31T07:30:26.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:26 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2542475323' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:30:26.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:30:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:26 smithi073 ceph-mon[21995]: pgmap v922: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 436 op/s 2024-08-31T07:30:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:26 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2542475323' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:30:27.036 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:27.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 14.70 MB/sec execute 127 sec latency 20522.378 ms 2024-08-31T07:30:27.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 -- 172.21.15.32:0/3245602451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4324075b70 msgr2=0x7f4324075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:27.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 --2- 172.21.15.32:0/3245602451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4324075b70 0x7f4324075f80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4310009900 tx=0x7f431002a410 comp rx=0 tx=0).stop 2024-08-31T07:30:27.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 -- 172.21.15.32:0/3245602451 shutdown_connections 2024-08-31T07:30:27.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 --2- 172.21.15.32:0/3245602451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4324114020 0x7f43241164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 --2- 172.21.15.32:0/3245602451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4324075b70 0x7f4324075f80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 --2- 172.21.15.32:0/3245602451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4324077120 0x7f4324075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 -- 172.21.15.32:0/3245602451 >> 172.21.15.32:0/3245602451 conn(0x7f43240ff1e0 msgr2=0x7f4324101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:27.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 -- 172.21.15.32:0/3245602451 shutdown_connections 2024-08-31T07:30:27.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 -- 172.21.15.32:0/3245602451 wait complete. 2024-08-31T07:30:27.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 Processor -- start 2024-08-31T07:30:27.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 -- start start 2024-08-31T07:30:27.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4324075b70 0x7f43241a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4324077120 0x7f43241a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4324114020 0x7f43241a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4324119cb0 con 0x7f4324114020 2024-08-31T07:30:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4324119b30 con 0x7f4324077120 2024-08-31T07:30:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4324119e30 con 0x7f4324075b70 2024-08-31T07:30:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4323400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4324075b70 0x7f43241a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4322a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4324077120 0x7f43241a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4323400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4324075b70 0x7f43241a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47382/0 (socket says 172.21.15.32:47382) 2024-08-31T07:30:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4323400640 1 -- 172.21.15.32:0/3039993510 learned_addr learned my addr 172.21.15.32:0/3039993510 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4323e00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4324114020 0x7f43241a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4322a00640 1 -- 172.21.15.32:0/3039993510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4324075b70 msgr2=0x7f43241a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4322a00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4324075b70 0x7f43241a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4322a00640 1 -- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4324114020 msgr2=0x7f43241a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4322a00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4324114020 0x7f43241a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4322a00640 1 -- 172.21.15.32:0/3039993510 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43241a65f0 con 0x7f4324077120 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4323e00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4324114020 0x7f43241a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4322a00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4324077120 0x7f43241a0c20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f431002a930 tx=0x7f4310003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f431be00640 1 -- 172.21.15.32:0/3039993510 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431003f070 con 0x7f4324077120 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f431be00640 1 -- 172.21.15.32:0/3039993510 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310031b60 con 0x7f4324077120 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f431be00640 1 -- 172.21.15.32:0/3039993510 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310035030 con 0x7f4324077120 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.960+0000 7f4328a00640 1 -- 172.21.15.32:0/3039993510 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43241a6880 con 0x7f4324077120 2024-08-31T07:30:27.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4328a00640 1 -- 172.21.15.32:0/3039993510 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43241a6ce0 con 0x7f4324077120 2024-08-31T07:30:27.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f43307fd740 1 Processor -- start 2024-08-31T07:30:27.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f43307fd740 1 -- start start 2024-08-31T07:30:27.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f43307fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a35a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f43307fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a35fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f43307fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f43307fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d3549ac7b0 con 0x55d354690070 2024-08-31T07:30:27.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f43307fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d35499bd40 con 0x55d3546671b0 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f43307fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d35499fcf0 con 0x55d35466f390 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4323400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a35fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4323400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a35fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47460/0 (socket says 172.21.15.32:47460) 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4323400640 1 -- 172.21.15.32:0/1942715458 learned_addr learned my addr 172.21.15.32:0/1942715458 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f431be00640 1 -- 172.21.15.32:0/3039993510 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4310042070 con 0x7f4324077120 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4323e00640 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a35a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4322a00640 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4323400640 1 -- 172.21.15.32:0/1942715458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 msgr2=0x55d354a364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4323400640 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a364f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4323400640 1 -- 172.21.15.32:0/1942715458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 msgr2=0x55d354a35a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4323400640 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a35a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.966+0000 7f4323400640 1 -- 172.21.15.32:0/1942715458 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3546fb650 con 0x55d3546671b0 2024-08-31T07:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4322a00640 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4323e00640 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a35a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4323400640 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a35fb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4314009e10 tx=0x7f4314011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e14dea9dff9bba6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f430fe00640 1 -- 172.21.15.32:0/1942715458 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431401e070 con 0x55d3546671b0 2024-08-31T07:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f430fe00640 1 -- 172.21.15.32:0/1942715458 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4314019ae0 con 0x55d3546671b0 2024-08-31T07:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f430fe00640 1 -- 172.21.15.32:0/1942715458 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431400fb10 con 0x55d3546671b0 2024-08-31T07:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f431be00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43080b4d20 0x7f43080b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 -- 172.21.15.32:0/1942715458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 msgr2=0x55d354a35fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a35fb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4314009e10 tx=0x7f4314011810 comp rx=0 tx=0).stop 2024-08-31T07:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 -- 172.21.15.32:0/1942715458 shutdown_connections 2024-08-31T07:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a35fb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 --2- 172.21.15.32:0/1942715458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a35a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 -- 172.21.15.32:0/1942715458 >> 172.21.15.32:0/1942715458 conn(0x55d354684460 msgr2=0x55d354a33960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f431be00640 1 -- 172.21.15.32:0/3039993510 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f4310105790 con 0x7f4324077120 2024-08-31T07:30:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 -- 172.21.15.32:0/1942715458 shutdown_connections 2024-08-31T07:30:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4323e00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43080b4d20 0x7f43080b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 -- 172.21.15.32:0/1942715458 wait complete. 2024-08-31T07:30:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4323e00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43080b4d20 0x7f43080b7230 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f431c005fd0 tx=0x7f431c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 -- start start 2024-08-31T07:30:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 Processor -- start 2024-08-31T07:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a41be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a4d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d3547fe700 con 0x55d354690070 2024-08-31T07:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d354922cf0 con 0x55d3546671b0 2024-08-31T07:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d354a44250 con 0x55d35466f390 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f43307fd740 10 client.0 _ll_register_callbacks cb 0x55d354948470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4323400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4323400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47464/0 (socket says 172.21.15.32:47464) 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4323400640 1 -- 172.21.15.32:0/2237097304 learned_addr learned my addr 172.21.15.32:0/2237097304 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4322a00640 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a41be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4323e00640 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a4d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4322a00640 1 -- 172.21.15.32:0/2237097304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 msgr2=0x55d354a416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4322a00640 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4322a00640 1 -- 172.21.15.32:0/2237097304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 msgr2=0x55d354a4d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4322a00640 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a4d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4322a00640 1 -- 172.21.15.32:0/2237097304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3549638b0 con 0x55d35466f390 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.971+0000 7f4323e00640 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a4d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f4322a00640 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a41be0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4310102ed0 tx=0x7f43100cb880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44c83daed9736d17 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f430cc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f430cc00640 1 -- 172.21.15.32:0/2237097304 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310049070 con 0x55d35466f390 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f430cc00640 1 -- 172.21.15.32:0/2237097304 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f43100c8100 con 0x55d35466f390 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f43307fd740 1 -- 172.21.15.32:0/2237097304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4314019ae0 con 0x55d35466f390 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f430cc00640 1 -- 172.21.15.32:0/2237097304 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f43100cbe70 con 0x55d35466f390 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f43307fd740 10 client.28957 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f43307fd740 1 -- 172.21.15.32:0/2237097304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55d354a50150 con 0x55d35466f390 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f430cc00640 1 -- 172.21.15.32:0/2237097304 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4314019ae0 con 0x55d35466f390 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f430cc00640 1 -- 172.21.15.32:0/2237097304 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f4310103150 con 0x55d35466f390 2024-08-31T07:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f43307fd740 10 client.28957 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f43307fd740 10 client.28957 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:30:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f43307fd740 20 client.28957 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f43307fd740 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d35464e9f0 0x55d354a53570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f43307fd740 4 client.28957 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:30:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f43307fd740 1 -- 172.21.15.32:0/2237097304 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3549ae700 con 0x55d35464e9f0 2024-08-31T07:30:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f4323400640 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d35464e9f0 0x55d354a53570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f4323400640 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d35464e9f0 0x55d354a53570 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea805346f3b92a33 server_cookie=e78d9e686703035c in_seq=0 out_seq=0 2024-08-31T07:30:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.976+0000 7f430cc00640 10 client.28957 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:30:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f430cc00640 1 -- 172.21.15.32:0/2237097304 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3549ae700 con 0x55d35464e9f0 2024-08-31T07:30:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f430cc00640 10 client.28957 handle_command_reply: tid=0 2024-08-31T07:30:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 client.28957 shutdown 2024-08-31T07:30:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 -- 172.21.15.32:0/2237097304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 msgr2=0x55d354a41be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a41be0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4310102ed0 tx=0x7f43100cb880 comp rx=0 tx=0).stop 2024-08-31T07:30:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 -- 172.21.15.32:0/2237097304 shutdown_connections 2024-08-31T07:30:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d35464e9f0 0x55d354a53570 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a4d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a41be0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 --2- 172.21.15.32:0/2237097304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 -- 172.21.15.32:0/2237097304 >> 172.21.15.32:0/2237097304 conn(0x55d354684460 msgr2=0x55d354a38c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f430cc00640 0 client.28957 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:30:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 -- 172.21.15.32:0/2237097304 shutdown_connections 2024-08-31T07:30:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 1 -- 172.21.15.32:0/2237097304 wait complete. 2024-08-31T07:30:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:27.978+0000 7f43307fd740 20 client.28957 trim_cache size 0 max 16384 2024-08-31T07:30:28.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.001+0000 7f43307fd740 1 Processor -- start 2024-08-31T07:30:28.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.001+0000 7f43307fd740 1 -- start start 2024-08-31T07:30:28.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.001+0000 7f43307fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a4d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:28.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.001+0000 7f43307fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:28.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.001+0000 7f43307fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a3acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:28.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.001+0000 7f43307fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f43241b57c0 con 0x55d354690070 2024-08-31T07:30:28.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.001+0000 7f43307fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f43241b7360 con 0x55d3546671b0 2024-08-31T07:30:28.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.001+0000 7f43307fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f43241b5ec0 con 0x55d35466f390 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4322a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4322a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47408/0 (socket says 172.21.15.32:47408) 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4322a00640 1 -- 172.21.15.32:0/3515276684 learned_addr learned my addr 172.21.15.32:0/3515276684 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323e00640 1 --2- 172.21.15.32:0/3515276684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a3acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323400640 1 --2- 172.21.15.32:0/3515276684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a4d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4322a00640 1 -- 172.21.15.32:0/3515276684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 msgr2=0x55d354a4d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4322a00640 1 --2- 172.21.15.32:0/3515276684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a4d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4322a00640 1 -- 172.21.15.32:0/3515276684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 msgr2=0x55d354a3acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4322a00640 1 --2- 172.21.15.32:0/3515276684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a3acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4322a00640 1 -- 172.21.15.32:0/3515276684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d354a3f9a0 con 0x55d35466f390 2024-08-31T07:30:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4322a00640 1 --2- 172.21.15.32:0/3515276684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a430d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f43100c90e0 tx=0x7f43100cb8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d02afb4fab09b97f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:28.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f430f400640 1 -- 172.21.15.32:0/3515276684 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310049030 con 0x55d35466f390 2024-08-31T07:30:28.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f430f400640 1 -- 172.21.15.32:0/3515276684 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43100d06e0 con 0x55d35466f390 2024-08-31T07:30:28.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f430f400640 1 -- 172.21.15.32:0/3515276684 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310103150 con 0x55d35466f390 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- 172.21.15.32:0/3515276684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 msgr2=0x55d354a430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 --2- 172.21.15.32:0/3515276684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a430d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f43100c90e0 tx=0x7f43100cb8f0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- 172.21.15.32:0/3515276684 shutdown_connections 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 --2- 172.21.15.32:0/3515276684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a3acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 --2- 172.21.15.32:0/3515276684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d35466f390 0x55d354a430d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 --2- 172.21.15.32:0/3515276684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3546671b0 0x55d354a4d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- 172.21.15.32:0/3515276684 >> 172.21.15.32:0/3515276684 conn(0x55d354684460 msgr2=0x55d354a429c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- 172.21.15.32:0/3515276684 shutdown_connections 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- 172.21.15.32:0/3515276684 wait complete. 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- start start 2024-08-31T07:30:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 Processor -- start 2024-08-31T07:30:28.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d3546671b0 0x55d354a494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:28.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d35466f390 0x55d354a499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a44460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f43241b6c90 con 0x55d354690070 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f43241b65c0 con 0x55d35466f390 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f43241b37c0 con 0x55d3546671b0 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 10 client.0 _ll_register_callbacks cb 0x7f43100cbcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d3546671b0 0x55d354a494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d3546671b0 0x55d354a494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47422/0 (socket says 172.21.15.32:47422) 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323400640 1 -- 172.21.15.32:0/229457482 learned_addr learned my addr 172.21.15.32:0/229457482 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4322a00640 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d35466f390 0x55d354a499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323e00640 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a44460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323400640 1 -- 172.21.15.32:0/229457482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d35466f390 msgr2=0x55d354a499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323400640 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d35466f390 0x55d354a499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323400640 1 -- 172.21.15.32:0/229457482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 msgr2=0x55d354a44460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323400640 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a44460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323400640 1 -- 172.21.15.32:0/229457482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d354a49f30 con 0x55d3546671b0 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323e00640 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a44460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f4323400640 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d3546671b0 0x55d354a494b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4314014040 tx=0x7f4314005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ca16b15807b8bc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f430cc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f430cc00640 1 -- 172.21.15.32:0/229457482 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431401e040 con 0x55d3546671b0 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- 172.21.15.32:0/229457482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43100d06e0 con 0x55d3546671b0 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f430cc00640 1 -- 172.21.15.32:0/229457482 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f4314005540 con 0x55d3546671b0 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f430cc00640 1 -- 172.21.15.32:0/229457482 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f431400f880 con 0x55d3546671b0 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 10 client.28963 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- 172.21.15.32:0/229457482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55d354abe290 con 0x55d3546671b0 2024-08-31T07:30:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f430cc00640 1 -- 172.21.15.32:0/229457482 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43100d06e0 con 0x55d3546671b0 2024-08-31T07:30:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f430cc00640 1 -- 172.21.15.32:0/229457482 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f4314003b80 con 0x55d3546671b0 2024-08-31T07:30:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 10 client.28963 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 10 client.28963 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:30:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 20 client.28963 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d354623da0 0x55d354ac0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 4 client.28963 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:30:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.002+0000 7f43307fd740 1 -- 172.21.15.32:0/229457482 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3549aa1a0 con 0x55d354623da0 2024-08-31T07:30:28.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4322a00640 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d354623da0 0x55d354ac0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:28.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4322a00640 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d354623da0 0x55d354ac0860 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb29dbcaa59588eb server_cookie=1d15af3880eeef51 in_seq=0 out_seq=0 2024-08-31T07:30:28.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f430cc00640 10 client.28963 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:30:28.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f430cc00640 1 -- 172.21.15.32:0/229457482 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3549aa1a0 con 0x55d354623da0 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f430cc00640 10 client.28963 handle_command_reply: tid=0 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 client.28963 shutdown 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 -- 172.21.15.32:0/229457482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d3546671b0 msgr2=0x55d354a494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d3546671b0 0x55d354a494b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4314014040 tx=0x7f4314005c50 comp rx=0 tx=0).stop 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 -- 172.21.15.32:0/229457482 shutdown_connections 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d354623da0 0x55d354ac0860 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d354690070 0x55d354a44460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f430cc00640 0 client.28963 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d35466f390 0x55d354a499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 --2- 172.21.15.32:0/229457482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d3546671b0 0x55d354a494b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 -- 172.21.15.32:0/229457482 >> 172.21.15.32:0/229457482 conn(0x55d354684460 msgr2=0x55d354a38b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 -- 172.21.15.32:0/229457482 shutdown_connections 2024-08-31T07:30:28.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 1 -- 172.21.15.32:0/229457482 wait complete. 2024-08-31T07:30:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f43307fd740 20 client.28963 trim_cache size 0 max 16384 2024-08-31T07:30:28.014 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:30:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 -- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43080b4d20 msgr2=0x7f43080b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43080b4d20 0x7f43080b7230 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f431c005fd0 tx=0x7f431c009500 comp rx=0 tx=0).stop 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 -- 172.21.15.32:0/3039993510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4324077120 msgr2=0x7f43241a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4324077120 0x7f43241a0c20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f431002a930 tx=0x7f4310003c10 comp rx=0 tx=0).stop 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 -- 172.21.15.32:0/3039993510 shutdown_connections 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f43080b4d20 0x7f43080b7230 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4324114020 0x7f43241a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4324077120 0x7f43241a0c20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 --2- 172.21.15.32:0/3039993510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4324075b70 0x7f43241a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 -- 172.21.15.32:0/3039993510 >> 172.21.15.32:0/3039993510 conn(0x7f43240ff1e0 msgr2=0x7f432410ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 -- 172.21.15.32:0/3039993510 shutdown_connections 2024-08-31T07:30:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:28.008+0000 7f4328a00640 1 -- 172.21.15.32:0/3039993510 wait complete. 2024-08-31T07:30:28.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 14.59 MB/sec execute 128 sec latency 21522.435 ms 2024-08-31T07:30:28.481 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:28 smithi073 ceph-mon[21995]: pgmap v923: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 17 MiB/s wr, 556 op/s 2024-08-31T07:30:28.485 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:28 smithi032 ceph-mon[17252]: pgmap v923: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 17 MiB/s wr, 556 op/s 2024-08-31T07:30:28.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:28 smithi084 ceph-mon[22387]: pgmap v923: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 17 MiB/s wr, 556 op/s 2024-08-31T07:30:28.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:30:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:29.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 14.47 MB/sec execute 129 sec latency 22522.489 ms 2024-08-31T07:30:30.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- 172.21.15.32:0/4159319340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa67c077120 msgr2=0x7fa67c075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 --2- 172.21.15.32:0/4159319340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa67c077120 0x7fa67c075550 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009900 tx=0x7fa66c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- 172.21.15.32:0/4159319340 shutdown_connections 2024-08-31T07:30:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 --2- 172.21.15.32:0/4159319340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa67c114020 0x7fa67c1164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 --2- 172.21.15.32:0/4159319340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c075b70 0x7fa67c075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 --2- 172.21.15.32:0/4159319340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa67c077120 0x7fa67c075550 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- 172.21.15.32:0/4159319340 >> 172.21.15.32:0/4159319340 conn(0x7fa67c0ff1e0 msgr2=0x7fa67c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- 172.21.15.32:0/4159319340 shutdown_connections 2024-08-31T07:30:30.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- 172.21.15.32:0/4159319340 wait complete. 2024-08-31T07:30:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 Processor -- start 2024-08-31T07:30:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- start start 2024-08-31T07:30:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa67c075b70 0x7fa67c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa67c077120 0x7fa67c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c114020 0x7fa67c1a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c119c60 con 0x7fa67c075b70 2024-08-31T07:30:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c119ae0 con 0x7fa67c077120 2024-08-31T07:30:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c119de0 con 0x7fa67c114020 2024-08-31T07:30:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa67c075b70 0x7fa67c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa67c077120 0x7fa67c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c114020 0x7fa67c1a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa67c077120 0x7fa67c1a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47496/0 (socket says 172.21.15.32:47496) 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67aa00640 1 -- 172.21.15.32:0/1553695196 learned_addr learned my addr 172.21.15.32:0/1553695196 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67be00640 1 -- 172.21.15.32:0/1553695196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa67c077120 msgr2=0x7fa67c1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67be00640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa67c077120 0x7fa67c1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67be00640 1 -- 172.21.15.32:0/1553695196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa67c075b70 msgr2=0x7fa67c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67be00640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa67c075b70 0x7fa67c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67be00640 1 -- 172.21.15.32:0/1553695196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c1a6600 con 0x7fa67c114020 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67b400640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa67c075b70 0x7fa67c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa67be00640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c114020 0x7fa67c1a5ed0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa674009fd0 tx=0x7fa67400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa673e00640 1 -- 172.21.15.32:0/1553695196 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67400ce20 con 0x7fa67c114020 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- 172.21.15.32:0/1553695196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c1a68f0 con 0x7fa67c114020 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa673e00640 1 -- 172.21.15.32:0/1553695196 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa674005020 con 0x7fa67c114020 2024-08-31T07:30:30.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa673e00640 1 -- 172.21.15.32:0/1553695196 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674010690 con 0x7fa67c114020 2024-08-31T07:30:30.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.151+0000 7fa681000640 1 -- 172.21.15.32:0/1553695196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c1ad050 con 0x7fa67c114020 2024-08-31T07:30:30.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa68922b740 1 Processor -- start 2024-08-31T07:30:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa673e00640 1 -- 172.21.15.32:0/1553695196 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa6740054d0 con 0x7fa67c114020 2024-08-31T07:30:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa68922b740 1 -- start start 2024-08-31T07:30:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa68922b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc996c070 0x55ebc9d11a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa68922b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d11fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa68922b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc994b390 0x55ebc9d124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa68922b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ebc9c887b0 con 0x55ebc996c070 2024-08-31T07:30:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa68922b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ebc9c77d40 con 0x55ebc994b390 2024-08-31T07:30:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa68922b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ebc9c7bcf0 con 0x55ebc99431b0 2024-08-31T07:30:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa67aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc994b390 0x55ebc9d124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa67aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc994b390 0x55ebc9d124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47504/0 (socket says 172.21.15.32:47504) 2024-08-31T07:30:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa67aa00640 1 -- 172.21.15.32:0/691520435 learned_addr learned my addr 172.21.15.32:0/691520435 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.156+0000 7fa67b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d11fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67aa00640 1 --2- 172.21.15.32:0/691520435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc996c070 0x55ebc9d11a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67b400640 1 -- 172.21.15.32:0/691520435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc994b390 msgr2=0x55ebc9d124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67b400640 1 --2- 172.21.15.32:0/691520435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc994b390 0x55ebc9d124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67b400640 1 -- 172.21.15.32:0/691520435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc996c070 msgr2=0x55ebc9d11a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67b400640 1 --2- 172.21.15.32:0/691520435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc996c070 0x55ebc9d11a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67b400640 1 -- 172.21.15.32:0/691520435 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebc99d7650 con 0x55ebc99431b0 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67aa00640 1 --2- 172.21.15.32:0/691520435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc996c070 0x55ebc9d11a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa673e00640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6600b4ca0 0x7fa6600b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa673e00640 1 -- 172.21.15.32:0/1553695196 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fa6740e5c00 con 0x7fa67c114020 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67b400640 1 --2- 172.21.15.32:0/691520435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d11fb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009230 tx=0x7fa66c005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e7b34dcdbb7d3e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67aa00640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6600b4ca0 0x7fa6600b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa670c00640 1 -- 172.21.15.32:0/691520435 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c040070 con 0x55ebc99431b0 2024-08-31T07:30:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa670c00640 1 -- 172.21.15.32:0/691520435 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa66c031720 con 0x55ebc99431b0 2024-08-31T07:30:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa670c00640 1 -- 172.21.15.32:0/691520435 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c035040 con 0x55ebc99431b0 2024-08-31T07:30:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 -- 172.21.15.32:0/691520435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 msgr2=0x55ebc9d11fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 --2- 172.21.15.32:0/691520435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d11fb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009230 tx=0x7fa66c005e20 comp rx=0 tx=0).stop 2024-08-31T07:30:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67aa00640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6600b4ca0 0x7fa6600b71b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa6680097f0 tx=0x7fa668013040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 -- 172.21.15.32:0/691520435 shutdown_connections 2024-08-31T07:30:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 --2- 172.21.15.32:0/691520435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc994b390 0x55ebc9d124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 --2- 172.21.15.32:0/691520435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d11fb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 --2- 172.21.15.32:0/691520435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc996c070 0x55ebc9d11a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 -- 172.21.15.32:0/691520435 >> 172.21.15.32:0/691520435 conn(0x55ebc9960460 msgr2=0x55ebc9d0f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 -- 172.21.15.32:0/691520435 shutdown_connections 2024-08-31T07:30:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 -- 172.21.15.32:0/691520435 wait complete. 2024-08-31T07:30:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 -- start start 2024-08-31T07:30:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 Processor -- start 2024-08-31T07:30:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d1d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d1db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 0x55ebc9d294e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ebc9ada700 con 0x55ebc994b390 2024-08-31T07:30:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ebc9bfecf0 con 0x55ebc996c070 2024-08-31T07:30:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ebc9d201f0 con 0x55ebc99431b0 2024-08-31T07:30:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa68922b740 10 client.0 _ll_register_callbacks cb 0x55ebc9c24470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d1d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d1d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47460/0 (socket says 172.21.15.32:47460) 2024-08-31T07:30:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.162+0000 7fa67b400640 1 -- 172.21.15.32:0/3251612416 learned_addr learned my addr 172.21.15.32:0/3251612416 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67be00640 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 0x55ebc9d294e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67aa00640 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d1db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67b400640 1 -- 172.21.15.32:0/3251612416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 msgr2=0x55ebc9d294e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67b400640 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 0x55ebc9d294e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67b400640 1 -- 172.21.15.32:0/3251612416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 msgr2=0x55ebc9d1db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67b400640 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d1db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67b400640 1 -- 172.21.15.32:0/3251612416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebc9c3f8b0 con 0x55ebc99431b0 2024-08-31T07:30:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67be00640 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 0x55ebc9d294e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67b400640 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d1d630 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa66c031a50 tx=0x7fa66c009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5739c643c7297bba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa665600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:30:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa665600640 1 -- 172.21.15.32:0/3251612416 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c04a070 con 0x55ebc99431b0 2024-08-31T07:30:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa665600640 1 -- 172.21.15.32:0/3251612416 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fa66c004450 con 0x55ebc99431b0 2024-08-31T07:30:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa68922b740 1 -- 172.21.15.32:0/3251612416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa66c031720 con 0x55ebc99431b0 2024-08-31T07:30:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa665600640 1 -- 172.21.15.32:0/3251612416 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fa66c035040 con 0x55ebc99431b0 2024-08-31T07:30:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa68922b740 10 client.28975 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa68922b740 1 -- 172.21.15.32:0/3251612416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55ebc9d2c180 con 0x55ebc99431b0 2024-08-31T07:30:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa665600640 1 -- 172.21.15.32:0/3251612416 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa66c031720 con 0x55ebc99431b0 2024-08-31T07:30:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa665600640 1 -- 172.21.15.32:0/3251612416 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fa66c03c620 con 0x55ebc99431b0 2024-08-31T07:30:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa68922b740 10 client.28975 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa68922b740 10 client.28975 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:30:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa68922b740 20 client.28975 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa68922b740 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ebc992a9f0 0x55ebc9d2f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa68922b740 4 client.28975 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:30:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa68922b740 1 -- 172.21.15.32:0/3251612416 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebc9c8a700 con 0x55ebc992a9f0 2024-08-31T07:30:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67aa00640 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ebc992a9f0 0x55ebc9d2f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa67aa00640 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ebc992a9f0 0x55ebc9d2f4e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5afb886f0387a102 server_cookie=568303880caefca6 in_seq=0 out_seq=0 2024-08-31T07:30:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.167+0000 7fa665600640 10 client.28975 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:30:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa665600640 1 -- 172.21.15.32:0/3251612416 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ebc9c8a700 con 0x55ebc992a9f0 2024-08-31T07:30:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa665600640 10 client.28975 handle_command_reply: tid=0 2024-08-31T07:30:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 client.28975 shutdown 2024-08-31T07:30:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 -- 172.21.15.32:0/3251612416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 msgr2=0x55ebc9d1d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d1d630 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa66c031a50 tx=0x7fa66c009520 comp rx=0 tx=0).stop 2024-08-31T07:30:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 -- 172.21.15.32:0/3251612416 shutdown_connections 2024-08-31T07:30:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ebc992a9f0 0x55ebc9d2f4e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 0x55ebc9d294e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d1db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 --2- 172.21.15.32:0/3251612416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d1d630 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 -- 172.21.15.32:0/3251612416 >> 172.21.15.32:0/3251612416 conn(0x55ebc9960460 msgr2=0x55ebc9d14c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa665600640 0 client.28975 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:30:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 -- 172.21.15.32:0/3251612416 shutdown_connections 2024-08-31T07:30:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 1 -- 172.21.15.32:0/3251612416 wait complete. 2024-08-31T07:30:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.172+0000 7fa68922b740 20 client.28975 trim_cache size 0 max 16384 2024-08-31T07:30:30.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.191+0000 7fa68922b740 1 Processor -- start 2024-08-31T07:30:30.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.191+0000 7fa68922b740 1 -- start start 2024-08-31T07:30:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.191+0000 7fa68922b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d16b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 0x55ebc9d0ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b5740 con 0x55ebc994b390 2024-08-31T07:30:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b72e0 con 0x55ebc996c070 2024-08-31T07:30:30.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b5e40 con 0x55ebc99431b0 2024-08-31T07:30:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d16b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d16b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47476/0 (socket says 172.21.15.32:47476) 2024-08-31T07:30:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 -- 172.21.15.32:0/1527759571 learned_addr learned my addr 172.21.15.32:0/1527759571 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67be00640 1 --2- 172.21.15.32:0/1527759571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 0x55ebc9d0ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67aa00640 1 --2- 172.21.15.32:0/1527759571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 -- 172.21.15.32:0/1527759571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 msgr2=0x55ebc9d0ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 --2- 172.21.15.32:0/1527759571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 0x55ebc9d0ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 -- 172.21.15.32:0/1527759571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 msgr2=0x55ebc9d170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 --2- 172.21.15.32:0/1527759571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 -- 172.21.15.32:0/1527759571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebc9d1dfd0 con 0x55ebc99431b0 2024-08-31T07:30:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 --2- 172.21.15.32:0/1527759571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d16b80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa66c005f70 tx=0x7fa66c005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78b7d1d7f585f482 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa667e00640 1 -- 172.21.15.32:0/1527759571 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c040040 con 0x55ebc99431b0 2024-08-31T07:30:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa667e00640 1 -- 172.21.15.32:0/1527759571 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa66c03cbd0 con 0x55ebc99431b0 2024-08-31T07:30:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa667e00640 1 -- 172.21.15.32:0/1527759571 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c003850 con 0x55ebc99431b0 2024-08-31T07:30:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- 172.21.15.32:0/1527759571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 msgr2=0x55ebc9d16b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 --2- 172.21.15.32:0/1527759571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d16b80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa66c005f70 tx=0x7fa66c005e20 comp rx=0 tx=0).stop 2024-08-31T07:30:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- 172.21.15.32:0/1527759571 shutdown_connections 2024-08-31T07:30:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 --2- 172.21.15.32:0/1527759571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc996c070 0x55ebc9d0ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 --2- 172.21.15.32:0/1527759571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 --2- 172.21.15.32:0/1527759571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc99431b0 0x55ebc9d16b80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- 172.21.15.32:0/1527759571 >> 172.21.15.32:0/1527759571 conn(0x55ebc9960460 msgr2=0x55ebc9d1bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- 172.21.15.32:0/1527759571 shutdown_connections 2024-08-31T07:30:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- 172.21.15.32:0/1527759571 wait complete. 2024-08-31T07:30:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- start start 2024-08-31T07:30:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 Processor -- start 2024-08-31T07:30:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc99431b0 0x55ebc9d32eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d30380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc996c070 0x55ebc9d30a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b6c10 con 0x55ebc994b390 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b6540 con 0x55ebc99431b0 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b3740 con 0x55ebc996c070 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa68922b740 10 client.0 _ll_register_callbacks cb 0x7fa66c0318d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d30380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc99431b0 0x55ebc9d32eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d30380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52670/0 (socket says 172.21.15.32:52670) 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67aa00640 1 -- 172.21.15.32:0/3324134845 learned_addr learned my addr 172.21.15.32:0/3324134845 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc99431b0 0x55ebc9d32eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47532/0 (socket says 172.21.15.32:47532) 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67be00640 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc996c070 0x55ebc9d30a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 -- 172.21.15.32:0/3324134845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc996c070 msgr2=0x55ebc9d30a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc996c070 0x55ebc9d30a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 -- 172.21.15.32:0/3324134845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 msgr2=0x55ebc9d30380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d30380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.197+0000 7fa67b400640 1 -- 172.21.15.32:0/3324134845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebc9d98c60 con 0x55ebc99431b0 2024-08-31T07:30:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.202+0000 7fa67b400640 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc99431b0 0x55ebc9d32eb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa66c02a910 tx=0x7fa66c0046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ddb0890f6e7c00 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.202+0000 7fa665600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:30:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.202+0000 7fa665600640 1 -- 172.21.15.32:0/3324134845 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c040070 con 0x55ebc99431b0 2024-08-31T07:30:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.202+0000 7fa68922b740 1 -- 172.21.15.32:0/3324134845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa66c03cbd0 con 0x55ebc99431b0 2024-08-31T07:30:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.202+0000 7fa665600640 1 -- 172.21.15.32:0/3324134845 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fa66c031a50 con 0x55ebc99431b0 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.202+0000 7fa665600640 1 -- 172.21.15.32:0/3324134845 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fa66c0044a0 con 0x55ebc99431b0 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.202+0000 7fa68922b740 10 client.28976 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.202+0000 7fa68922b740 1 -- 172.21.15.32:0/3324134845 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55ebc9d99d50 con 0x55ebc99431b0 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa665600640 1 -- 172.21.15.32:0/3324134845 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa66c03cbd0 con 0x55ebc99431b0 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa665600640 1 -- 172.21.15.32:0/3324134845 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fa66c003cd0 con 0x55ebc99431b0 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 10 client.28976 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 10 client.28976 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 20 client.28976 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ebc98ffda0 0x55ebc9d9c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 4 client.28976 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 -- 172.21.15.32:0/3324134845 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebc9c861a0 con 0x55ebc98ffda0 2024-08-31T07:30:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa67aa00640 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ebc98ffda0 0x55ebc9d9c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa67aa00640 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ebc98ffda0 0x55ebc9d9c3c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6944b76a97ba3882 server_cookie=24731ec272051469 in_seq=0 out_seq=0 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa665600640 10 client.28976 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa665600640 1 -- 172.21.15.32:0/3324134845 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebc9c861a0 con 0x55ebc98ffda0 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa665600640 10 client.28976 handle_command_reply: tid=0 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 client.28976 shutdown 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 -- 172.21.15.32:0/3324134845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc99431b0 msgr2=0x55ebc9d32eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc99431b0 0x55ebc9d32eb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa66c02a910 tx=0x7fa66c0046e0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 -- 172.21.15.32:0/3324134845 shutdown_connections 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55ebc98ffda0 0x55ebc9d9c3c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ebc996c070 0x55ebc9d30a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ebc994b390 0x55ebc9d30380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa665600640 0 client.28976 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:30:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 --2- 172.21.15.32:0/3324134845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ebc99431b0 0x55ebc9d32eb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 -- 172.21.15.32:0/3324134845 >> 172.21.15.32:0/3324134845 conn(0x55ebc9960460 msgr2=0x55ebc9d16b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 -- 172.21.15.32:0/3324134845 shutdown_connections 2024-08-31T07:30:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 1 -- 172.21.15.32:0/3324134845 wait complete. 2024-08-31T07:30:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa68922b740 20 client.28976 trim_cache size 0 max 16384 2024-08-31T07:30:30.209 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 -- 172.21.15.32:0/1553695196 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6600b4ca0 msgr2=0x7fa6600b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6600b4ca0 0x7fa6600b71b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa6680097f0 tx=0x7fa668013040 comp rx=0 tx=0).stop 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 -- 172.21.15.32:0/1553695196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c114020 msgr2=0x7fa67c1a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c114020 0x7fa67c1a5ed0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa674009fd0 tx=0x7fa67400ef10 comp rx=0 tx=0).stop 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 -- 172.21.15.32:0/1553695196 shutdown_connections 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa6600b4ca0 0x7fa6600b71b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa67c114020 0x7fa67c1a5ed0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa67c077120 0x7fa67c1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 --2- 172.21.15.32:0/1553695196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa67c075b70 0x7fa67c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 -- 172.21.15.32:0/1553695196 >> 172.21.15.32:0/1553695196 conn(0x7fa67c0ff1e0 msgr2=0x7fa67c10ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 -- 172.21.15.32:0/1553695196 shutdown_connections 2024-08-31T07:30:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:30.203+0000 7fa681000640 1 -- 172.21.15.32:0/1553695196 wait complete. 2024-08-31T07:30:30.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 14.36 MB/sec execute 130 sec latency 23522.556 ms 2024-08-31T07:30:30.416 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:30 smithi032 ceph-mon[17252]: pgmap v924: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 483 op/s 2024-08-31T07:30:30.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:30 smithi084 ceph-mon[22387]: pgmap v924: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 483 op/s 2024-08-31T07:30:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:30 smithi073 ceph-mon[21995]: pgmap v924: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 483 op/s 2024-08-31T07:30:31.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:30:31.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 14.25 MB/sec execute 131 sec latency 24522.598 ms 2024-08-31T07:30:31.588 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:32.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:32 smithi032 ceph-mon[17252]: pgmap v925: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 498 op/s 2024-08-31T07:30:32.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 14.15 MB/sec execute 132 sec latency 25522.644 ms 2024-08-31T07:30:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1738976726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0b410bbc0 msgr2=0x7fb0b41123f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 --2- 172.21.15.32:0/1738976726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0b410bbc0 0x7fb0b41123f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009ca0 tx=0x7fb0ac02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1738976726 shutdown_connections 2024-08-31T07:30:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 --2- 172.21.15.32:0/1738976726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0b410bbc0 0x7fb0b41123f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 --2- 172.21.15.32:0/1738976726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0b410b270 0x7fb0b410b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 --2- 172.21.15.32:0/1738976726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0b41059c0 0x7fb0b4105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1738976726 >> 172.21.15.32:0/1738976726 conn(0x7fb0b4101430 msgr2=0x7fb0b4103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1738976726 shutdown_connections 2024-08-31T07:30:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1738976726 wait complete. 2024-08-31T07:30:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 Processor -- start 2024-08-31T07:30:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 -- start start 2024-08-31T07:30:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0b41059c0 0x7fb0b419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0b410b270 0x7fb0b419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0b410bbc0 0x7fb0b41a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4115880 con 0x7fb0b41059c0 2024-08-31T07:30:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4115700 con 0x7fb0b410bbc0 2024-08-31T07:30:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4115a00 con 0x7fb0b410b270 2024-08-31T07:30:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0b41059c0 0x7fb0b419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0b41059c0 0x7fb0b419c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52688/0 (socket says 172.21.15.32:52688) 2024-08-31T07:30:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b2000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0b410b270 0x7fb0b419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b2a00640 1 -- 172.21.15.32:0/1156304236 learned_addr learned my addr 172.21.15.32:0/1156304236 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b3400640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0b410bbc0 0x7fb0b41a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b2000640 1 -- 172.21.15.32:0/1156304236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0b410bbc0 msgr2=0x7fb0b41a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b2000640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0b410bbc0 0x7fb0b41a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b2000640 1 -- 172.21.15.32:0/1156304236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0b41059c0 msgr2=0x7fb0b419c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b2000640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0b41059c0 0x7fb0b419c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b2000640 1 -- 172.21.15.32:0/1156304236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b41a2180 con 0x7fb0b410b270 2024-08-31T07:30:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.430+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0b41059c0 0x7fb0b419c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.435+0000 7fb0b2000640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0b410b270 0x7fb0b419c7b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb0b406b730 tx=0x7fb0a000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.435+0000 7fb0ab400640 1 -- 172.21.15.32:0/1156304236 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a000cc40 con 0x7fb0b410b270 2024-08-31T07:30:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.435+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1156304236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b41a2470 con 0x7fb0b410b270 2024-08-31T07:30:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.435+0000 7fb0ab400640 1 -- 172.21.15.32:0/1156304236 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a000cde0 con 0x7fb0b410b270 2024-08-31T07:30:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.435+0000 7fb0ab400640 1 -- 172.21.15.32:0/1156304236 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0004a00 con 0x7fb0b410b270 2024-08-31T07:30:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.435+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1156304236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b41a8bd0 con 0x7fb0b410b270 2024-08-31T07:30:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.435+0000 7fb0c0288740 1 Processor -- start 2024-08-31T07:30:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.435+0000 7fb0c0288740 1 -- start start 2024-08-31T07:30:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0ab400640 1 -- 172.21.15.32:0/1156304236 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb0a0004ba0 con 0x7fb0b410b270 2024-08-31T07:30:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e492b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0c41b0 0x55e70e4930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e4935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e70e4097b0 con 0x55e70e0cc390 2024-08-31T07:30:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e70e3f8d40 con 0x55e70e0c41b0 2024-08-31T07:30:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e70e3fccf0 con 0x55e70e0ed070 2024-08-31T07:30:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e492b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e492b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47506/0 (socket says 172.21.15.32:47506) 2024-08-31T07:30:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b3400640 1 -- 172.21.15.32:0/166564119 learned_addr learned my addr 172.21.15.32:0/166564119 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/166564119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0c41b0 0x55e70e4930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b2000640 1 --2- 172.21.15.32:0/166564119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e4935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b3400640 1 -- 172.21.15.32:0/166564119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0c41b0 msgr2=0x55e70e4930a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b3400640 1 --2- 172.21.15.32:0/166564119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0c41b0 0x55e70e4930a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b3400640 1 -- 172.21.15.32:0/166564119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 msgr2=0x55e70e4935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b3400640 1 --2- 172.21.15.32:0/166564119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e4935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b3400640 1 -- 172.21.15.32:0/166564119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e70e158650 con 0x55e70e0ed070 2024-08-31T07:30:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b3400640 1 --2- 172.21.15.32:0/166564119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e492b60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac005cc0 tx=0x7fb0ac02f420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18aa6e4757b2d21f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb09f400640 1 -- 172.21.15.32:0/166564119 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac031cf0 con 0x55e70e0ed070 2024-08-31T07:30:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb09f400640 1 -- 172.21.15.32:0/166564119 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0ac036040 con 0x55e70e0ed070 2024-08-31T07:30:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb09f400640 1 -- 172.21.15.32:0/166564119 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac004040 con 0x55e70e0ed070 2024-08-31T07:30:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0ab400640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0980b4ca0 0x7fb0980b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0ab400640 1 -- 172.21.15.32:0/1156304236 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fb0a00e5550 con 0x7fb0b410b270 2024-08-31T07:30:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0980b4ca0 0x7fb0980b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- 172.21.15.32:0/166564119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 msgr2=0x55e70e492b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 --2- 172.21.15.32:0/166564119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e492b60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac005cc0 tx=0x7fb0ac02f420 comp rx=0 tx=0).stop 2024-08-31T07:30:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- 172.21.15.32:0/166564119 shutdown_connections 2024-08-31T07:30:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 --2- 172.21.15.32:0/166564119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e4935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 --2- 172.21.15.32:0/166564119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0c41b0 0x55e70e4930a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 --2- 172.21.15.32:0/166564119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e492b60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- 172.21.15.32:0/166564119 >> 172.21.15.32:0/166564119 conn(0x55e70e0e1460 msgr2=0x55e70e490a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- 172.21.15.32:0/166564119 shutdown_connections 2024-08-31T07:30:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- 172.21.15.32:0/166564119 wait complete. 2024-08-31T07:30:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0980b4ca0 0x7fb0980b71b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55e70e4917e0 tx=0x7fb0a4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- start start 2024-08-31T07:30:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 Processor -- start 2024-08-31T07:30:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0c41b0 0x55e70e49e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0cc390 0x55e70e49eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e4aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e70e25b700 con 0x55e70e0c41b0 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e70e37fcf0 con 0x55e70e0cc390 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e70e4a12e0 con 0x55e70e0ed070 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0c0288740 10 client.0 _ll_register_callbacks cb 0x55e70e3a5470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0c41b0 0x55e70e49e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0c41b0 0x55e70e49e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52714/0 (socket says 172.21.15.32:52714) 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b2a00640 1 -- 172.21.15.32:0/2153236672 learned_addr learned my addr 172.21.15.32:0/2153236672 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.439+0000 7fb0b2000640 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0cc390 0x55e70e49eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb0b3400640 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e4aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb0b2000640 1 -- 172.21.15.32:0/2153236672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 msgr2=0x55e70e4aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb0b2000640 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e4aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb0b2000640 1 -- 172.21.15.32:0/2153236672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0c41b0 msgr2=0x55e70e49e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb0b2000640 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0c41b0 0x55e70e49e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb0b2000640 1 -- 172.21.15.32:0/2153236672 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e70e3c08b0 con 0x55e70e0cc390 2024-08-31T07:30:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb0b3400640 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e4aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb0b2000640 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0cc390 0x55e70e49eca0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00e2c90 tx=0x7fb0a00a7bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31a6c687b1cd6fe5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb093e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:30:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb093e00640 1 -- 172.21.15.32:0/2153236672 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00a73c0 con 0x55e70e0cc390 2024-08-31T07:30:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb093e00640 1 -- 172.21.15.32:0/2153236672 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fb0a00b4ac0 con 0x55e70e0cc390 2024-08-31T07:30:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.445+0000 7fb0c0288740 1 -- 172.21.15.32:0/2153236672 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0ac036040 con 0x55e70e0cc390 2024-08-31T07:30:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb093e00640 1 -- 172.21.15.32:0/2153236672 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fb0a00e3120 con 0x55e70e0cc390 2024-08-31T07:30:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 10 client.28988 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 -- 172.21.15.32:0/2153236672 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55e70e4ad380 con 0x55e70e0cc390 2024-08-31T07:30:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb093e00640 1 -- 172.21.15.32:0/2153236672 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0ac036040 con 0x55e70e0cc390 2024-08-31T07:30:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb093e00640 1 -- 172.21.15.32:0/2153236672 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fb0a00b00d0 con 0x55e70e0cc390 2024-08-31T07:30:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 10 client.28988 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 10 client.28988 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:30:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 20 client.28988 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55e70e0ab9f0 0x55e70e4b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 4 client.28988 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:30:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 -- 172.21.15.32:0/2153236672 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e70e40b700 con 0x55e70e0ab9f0 2024-08-31T07:30:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55e70e0ab9f0 0x55e70e4b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55e70e0ab9f0 0x55e70e4b0920 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4fdab98182283c71 server_cookie=590b362a79f6c9b8 in_seq=0 out_seq=0 2024-08-31T07:30:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb093e00640 10 client.28988 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:30:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb093e00640 1 -- 172.21.15.32:0/2153236672 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e70e40b700 con 0x55e70e0ab9f0 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb093e00640 10 client.28988 handle_command_reply: tid=0 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 client.28988 shutdown 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 -- 172.21.15.32:0/2153236672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0cc390 msgr2=0x55e70e49eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0cc390 0x55e70e49eca0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00e2c90 tx=0x7fb0a00a7bb0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 -- 172.21.15.32:0/2153236672 shutdown_connections 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55e70e0ab9f0 0x55e70e4b0920 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0ed070 0x55e70e4aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0cc390 0x55e70e49eca0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2153236672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0c41b0 0x55e70e49e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 -- 172.21.15.32:0/2153236672 >> 172.21.15.32:0/2153236672 conn(0x55e70e0e1460 msgr2=0x55e70e495cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb093e00640 0 client.28988 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:30:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 -- 172.21.15.32:0/2153236672 shutdown_connections 2024-08-31T07:30:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 1 -- 172.21.15.32:0/2153236672 wait complete. 2024-08-31T07:30:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.446+0000 7fb0c0288740 20 client.28988 trim_cache size 0 max 16384 2024-08-31T07:30:32.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0c0288740 1 Processor -- start 2024-08-31T07:30:32.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0c0288740 1 -- start start 2024-08-31T07:30:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 0x55e70e497c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 0x55e70e4981b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e490000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0c0288740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41b1340 con 0x55e70e0cc390 2024-08-31T07:30:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0c0288740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41b2ee0 con 0x55e70e0e1460 2024-08-31T07:30:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0c0288740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41b1a40 con 0x55e70e0c41b0 2024-08-31T07:30:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0b2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 0x55e70e497c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0b3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e490000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0b2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 0x55e70e497c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47558/0 (socket says 172.21.15.32:47558) 2024-08-31T07:30:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0b2a00640 1 -- 172.21.15.32:0/2954021679 learned_addr learned my addr 172.21.15.32:0/2954021679 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0b2000640 1 --2- 172.21.15.32:0/2954021679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 0x55e70e4981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0b2a00640 1 -- 172.21.15.32:0/2954021679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 msgr2=0x55e70e4981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/2954021679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 0x55e70e4981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0b2a00640 1 -- 172.21.15.32:0/2954021679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 msgr2=0x55e70e490000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.472+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/2954021679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e490000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2a00640 1 -- 172.21.15.32:0/2954021679 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e70e49a0b0 con 0x55e70e0e1460 2024-08-31T07:30:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/2954021679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 0x55e70e497c70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40124c0 tx=0x7fb0a401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=316c090d91dd3c65 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb09ea00640 1 -- 172.21.15.32:0/2954021679 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4004030 con 0x55e70e0e1460 2024-08-31T07:30:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb09ea00640 1 -- 172.21.15.32:0/2954021679 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a40035f0 con 0x55e70e0e1460 2024-08-31T07:30:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb09ea00640 1 -- 172.21.15.32:0/2954021679 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a402b6d0 con 0x55e70e0e1460 2024-08-31T07:30:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- 172.21.15.32:0/2954021679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 msgr2=0x55e70e497c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2954021679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 0x55e70e497c70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40124c0 tx=0x7fb0a401a000 comp rx=0 tx=0).stop 2024-08-31T07:30:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- 172.21.15.32:0/2954021679 shutdown_connections 2024-08-31T07:30:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2954021679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e490000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2954021679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 0x55e70e4981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2954021679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 0x55e70e497c70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- 172.21.15.32:0/2954021679 >> 172.21.15.32:0/2954021679 conn(0x55e70e4b0c10 msgr2=0x55e70e49cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- 172.21.15.32:0/2954021679 shutdown_connections 2024-08-31T07:30:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- 172.21.15.32:0/2954021679 wait complete. 2024-08-31T07:30:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- start start 2024-08-31T07:30:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 Processor -- start 2024-08-31T07:30:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 0x55e70e4a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e4a3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 0x55e70e514340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41b2810 con 0x55e70e0cc390 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41b2140 con 0x55e70e0e1460 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41af340 con 0x55e70e0c41b0 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 10 client.0 _ll_register_callbacks cb 0x7fb0a0004f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e4a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 0x55e70e4a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 0x55e70e4a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47534/0 (socket says 172.21.15.32:47534) 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2a00640 1 -- 172.21.15.32:0/2826191828 learned_addr learned my addr 172.21.15.32:0/2826191828 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b3400640 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 0x55e70e514340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2000640 1 -- 172.21.15.32:0/2826191828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 msgr2=0x55e70e4a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2000640 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 0x55e70e4a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2000640 1 -- 172.21.15.32:0/2826191828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 msgr2=0x55e70e514340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:32.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2000640 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 0x55e70e514340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2000640 1 -- 172.21.15.32:0/2826191828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e70e5149b0 con 0x55e70e0cc390 2024-08-31T07:30:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0b2000640 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e4a3520 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00a7cb0 tx=0x7fb0a00053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842c88e2feee0dbd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb093e00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:30:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb093e00640 1 -- 172.21.15.32:0/2826191828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00a73c0 con 0x55e70e0cc390 2024-08-31T07:30:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb093e00640 1 -- 172.21.15.32:0/2826191828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fb0a00e5020 con 0x55e70e0cc390 2024-08-31T07:30:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- 172.21.15.32:0/2826191828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a40035f0 con 0x55e70e0cc390 2024-08-31T07:30:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb093e00640 1 -- 172.21.15.32:0/2826191828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fb0a00ab580 con 0x55e70e0cc390 2024-08-31T07:30:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 10 client.18951 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.474+0000 7fb0c0288740 1 -- 172.21.15.32:0/2826191828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55e70e51bda0 con 0x55e70e0cc390 2024-08-31T07:30:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb093e00640 1 -- 172.21.15.32:0/2826191828 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a40035f0 con 0x55e70e0cc390 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb093e00640 1 -- 172.21.15.32:0/2826191828 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fb0a00a75a0 con 0x55e70e0cc390 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 10 client.18951 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 10 client.18951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 20 client.18951 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55e70e0ed070 0x55e70e51e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 4 client.18951 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 -- 172.21.15.32:0/2826191828 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e70e4071a0 con 0x55e70e0ed070 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55e70e0ed070 0x55e70e51e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b2a00640 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55e70e0ed070 0x55e70e51e400 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3346b3d78077fab0 server_cookie=db605ccd395b6d6b in_seq=0 out_seq=0 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb093e00640 10 client.18951 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb093e00640 1 -- 172.21.15.32:0/2826191828 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e70e4071a0 con 0x55e70e0ed070 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb093e00640 10 client.18951 handle_command_reply: tid=0 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 client.18951 shutdown 2024-08-31T07:30:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 -- 172.21.15.32:0/2826191828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 msgr2=0x55e70e4a3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e4a3520 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00a7cb0 tx=0x7fb0a00053f0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 -- 172.21.15.32:0/2826191828 shutdown_connections 2024-08-31T07:30:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55e70e0ed070 0x55e70e51e400 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e70e0e1460 0x55e70e514340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e70e0cc390 0x55e70e4a3520 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 --2- 172.21.15.32:0/2826191828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e70e0c41b0 0x55e70e4a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 -- 172.21.15.32:0/2826191828 >> 172.21.15.32:0/2826191828 conn(0x55e70e4b0c10 msgr2=0x55e70e497c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb093e00640 0 client.18951 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:30:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 -- 172.21.15.32:0/2826191828 shutdown_connections 2024-08-31T07:30:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 1 -- 172.21.15.32:0/2826191828 wait complete. 2024-08-31T07:30:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0c0288740 20 client.18951 trim_cache size 0 max 16384 2024-08-31T07:30:32.484 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:30:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1156304236 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0980b4ca0 msgr2=0x7fb0980b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0980b4ca0 0x7fb0980b71b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55e70e4917e0 tx=0x7fb0a4005f90 comp rx=0 tx=0).stop 2024-08-31T07:30:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1156304236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0b410b270 msgr2=0x7fb0b419c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0b410b270 0x7fb0b419c7b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb0b406b730 tx=0x7fb0a000ee50 comp rx=0 tx=0).stop 2024-08-31T07:30:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1156304236 shutdown_connections 2024-08-31T07:30:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb0980b4ca0 0x7fb0980b71b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0b410bbc0 0x7fb0b41a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0b410b270 0x7fb0b419c7b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 --2- 172.21.15.32:0/1156304236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0b41059c0 0x7fb0b419c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1156304236 >> 172.21.15.32:0/1156304236 conn(0x7fb0b4101430 msgr2=0x7fb0b410a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1156304236 shutdown_connections 2024-08-31T07:30:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:32.479+0000 7fb0b3e00640 1 -- 172.21.15.32:0/1156304236 wait complete. 2024-08-31T07:30:32.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:32 smithi084 ceph-mon[22387]: pgmap v925: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 498 op/s 2024-08-31T07:30:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:32 smithi073 ceph-mon[21995]: pgmap v925: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 498 op/s 2024-08-31T07:30:33.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 14.04 MB/sec execute 133 sec latency 26522.688 ms 2024-08-31T07:30:33.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:30:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:34.311 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:34 smithi032 ceph-mon[17252]: pgmap v926: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 402 op/s 2024-08-31T07:30:34.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 13.93 MB/sec execute 134 sec latency 27522.729 ms 2024-08-31T07:30:34.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:34 smithi084 ceph-mon[22387]: pgmap v926: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 402 op/s 2024-08-31T07:30:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:34 smithi073 ceph-mon[21995]: pgmap v926: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 402 op/s 2024-08-31T07:30:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- 172.21.15.32:0/98359129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf88075b70 msgr2=0x7fbf88075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 --2- 172.21.15.32:0/98359129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf88075b70 0x7fbf88075f80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009860 tx=0x7fbf7402a340 comp rx=0 tx=0).stop 2024-08-31T07:30:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- 172.21.15.32:0/98359129 shutdown_connections 2024-08-31T07:30:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 --2- 172.21.15.32:0/98359129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf88114020 0x7fbf881164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 --2- 172.21.15.32:0/98359129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf88075b70 0x7fbf88075f80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 --2- 172.21.15.32:0/98359129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf88077120 0x7fbf88075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- 172.21.15.32:0/98359129 >> 172.21.15.32:0/98359129 conn(0x7fbf880ff1e0 msgr2=0x7fbf88101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- 172.21.15.32:0/98359129 shutdown_connections 2024-08-31T07:30:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- 172.21.15.32:0/98359129 wait complete. 2024-08-31T07:30:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 Processor -- start 2024-08-31T07:30:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- start start 2024-08-31T07:30:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf88075b70 0x7fbf881a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf88077120 0x7fbf881a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf88114020 0x7fbf881a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88119c30 con 0x7fbf88114020 2024-08-31T07:30:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88119ab0 con 0x7fbf88075b70 2024-08-31T07:30:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88119db0 con 0x7fbf88077120 2024-08-31T07:30:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf86a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf88075b70 0x7fbf881a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf88114020 0x7fbf881a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf86000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf88077120 0x7fbf881a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf86a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf88075b70 0x7fbf881a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47592/0 (socket says 172.21.15.32:47592) 2024-08-31T07:30:34.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf86a00640 1 -- 172.21.15.32:0/3514769411 learned_addr learned my addr 172.21.15.32:0/3514769411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:34.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf86a00640 1 -- 172.21.15.32:0/3514769411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf88077120 msgr2=0x7fbf881a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf86a00640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf88077120 0x7fbf881a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf86a00640 1 -- 172.21.15.32:0/3514769411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf88114020 msgr2=0x7fbf881a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf86a00640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf88114020 0x7fbf881a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf86a00640 1 -- 172.21.15.32:0/3514769411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf881a65f0 con 0x7fbf88075b70 2024-08-31T07:30:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf86a00640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf88075b70 0x7fbf881a0680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800e9f0 tx=0x7fbf7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf7f400640 1 -- 172.21.15.32:0/3514769411 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7800cd80 con 0x7fbf88075b70 2024-08-31T07:30:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- 172.21.15.32:0/3514769411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf881a68e0 con 0x7fbf88075b70 2024-08-31T07:30:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf7f400640 1 -- 172.21.15.32:0/3514769411 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf780108e0 con 0x7fbf88075b70 2024-08-31T07:30:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.793+0000 7fbf87e00640 1 -- 172.21.15.32:0/3514769411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf881ad040 con 0x7fbf88075b70 2024-08-31T07:30:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf7f400640 1 -- 172.21.15.32:0/3514769411 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78018980 con 0x7fbf88075b70 2024-08-31T07:30:34.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf949f1740 1 Processor -- start 2024-08-31T07:30:34.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf949f1740 1 -- start start 2024-08-31T07:30:34.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634405a8070 0x56344094db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344057f1b0 0x56344094e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 0x56344094e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf949f1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5634408c47b0 con 0x5634405a8070 2024-08-31T07:30:34.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf949f1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5634408b3d40 con 0x563440587390 2024-08-31T07:30:34.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf949f1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5634408b7cf0 con 0x56344057f1b0 2024-08-31T07:30:34.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf87400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634405a8070 0x56344094db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf87400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634405a8070 0x56344094db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52746/0 (socket says 172.21.15.32:52746) 2024-08-31T07:30:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf87400640 1 -- 172.21.15.32:0/2663990479 learned_addr learned my addr 172.21.15.32:0/2663990479 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf86000640 1 --2- 172.21.15.32:0/2663990479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 0x56344094e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf87400640 1 -- 172.21.15.32:0/2663990479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344057f1b0 msgr2=0x56344094e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:30:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf87400640 1 --2- 172.21.15.32:0/2663990479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344057f1b0 0x56344094e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf87400640 1 -- 172.21.15.32:0/2663990479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 msgr2=0x56344094e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf87400640 1 --2- 172.21.15.32:0/2663990479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 0x56344094e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf87400640 1 -- 172.21.15.32:0/2663990479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563440613650 con 0x5634405a8070 2024-08-31T07:30:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.798+0000 7fbf86000640 1 --2- 172.21.15.32:0/2663990479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 0x56344094e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.803+0000 7fbf87400640 1 --2- 172.21.15.32:0/2663990479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634405a8070 0x56344094db60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000e450 tx=0x7fbf800095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65acf0a9e325fe08 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:34.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.803+0000 7fbf7f400640 1 -- 172.21.15.32:0/3514769411 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbf78010430 con 0x7fbf88075b70 2024-08-31T07:30:34.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.803+0000 7fbf73e00640 1 -- 172.21.15.32:0/2663990479 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8000d910 con 0x5634405a8070 2024-08-31T07:30:34.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.803+0000 7fbf73e00640 1 -- 172.21.15.32:0/2663990479 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8001c070 con 0x5634405a8070 2024-08-31T07:30:34.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.803+0000 7fbf73e00640 1 -- 172.21.15.32:0/2663990479 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80016380 con 0x5634405a8070 2024-08-31T07:30:34.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- 172.21.15.32:0/2663990479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634405a8070 msgr2=0x56344094db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:34.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 --2- 172.21.15.32:0/2663990479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634405a8070 0x56344094db60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000e450 tx=0x7fbf800095c0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- 172.21.15.32:0/2663990479 shutdown_connections 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 --2- 172.21.15.32:0/2663990479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 0x56344094e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 --2- 172.21.15.32:0/2663990479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56344057f1b0 0x56344094e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 --2- 172.21.15.32:0/2663990479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634405a8070 0x56344094db60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- 172.21.15.32:0/2663990479 >> 172.21.15.32:0/2663990479 conn(0x56344059c460 msgr2=0x56344094ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- 172.21.15.32:0/2663990479 shutdown_connections 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- 172.21.15.32:0/2663990479 wait complete. 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf7f400640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf6c0b4bb0 0x7fbf6c0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf7f400640 1 -- 172.21.15.32:0/3514769411 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fbf780e5690 con 0x7fbf88075b70 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf86000640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf6c0b4bb0 0x7fbf6c0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- start start 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 Processor -- start 2024-08-31T07:30:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344057f1b0 0x563440959720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563440587390 0x563440959c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 0x5634409656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563440716700 con 0x563440587390 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56344083acf0 con 0x56344057f1b0 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56344095c340 con 0x5634405a8070 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 10 client.0 _ll_register_callbacks cb 0x563440860470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf86a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344057f1b0 0x563440959720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf86a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344057f1b0 0x563440959720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47600/0 (socket says 172.21.15.32:47600) 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf86000640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf6c0b4bb0 0x7fbf6c0b70c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009830 tx=0x7fbf740058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf86a00640 1 -- 172.21.15.32:0/1354678616 learned_addr learned my addr 172.21.15.32:0/1354678616 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf87400640 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 0x5634409656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf86000640 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563440587390 0x563440959c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf87400640 1 -- 172.21.15.32:0/1354678616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344057f1b0 msgr2=0x563440959720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf87400640 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344057f1b0 0x563440959720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf87400640 1 -- 172.21.15.32:0/1354678616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563440587390 msgr2=0x563440959c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf87400640 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563440587390 0x563440959c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf87400640 1 -- 172.21.15.32:0/1354678616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344087b8b0 con 0x5634405a8070 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf86a00640 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344057f1b0 0x563440959720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf86000640 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563440587390 0x563440959c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf87400640 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 0x5634409656c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbf80017a30 tx=0x7fbf80017dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a002476f6b5c87f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf70c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf70c00640 1 -- 172.21.15.32:0/1354678616 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80017620 con 0x5634405a8070 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- 172.21.15.32:0/1354678616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8001c070 con 0x5634405a8070 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf70c00640 1 -- 172.21.15.32:0/1354678616 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fbf800177c0 con 0x5634405a8070 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf70c00640 1 -- 172.21.15.32:0/1354678616 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fbf8002f020 con 0x5634405a8070 2024-08-31T07:30:34.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 10 client.29005 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf949f1740 1 -- 172.21.15.32:0/1354678616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x563440968390 con 0x5634405a8070 2024-08-31T07:30:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.805+0000 7fbf70c00640 1 -- 172.21.15.32:0/1354678616 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8001c070 con 0x5634405a8070 2024-08-31T07:30:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf70c00640 1 -- 172.21.15.32:0/1354678616 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fbf8000b040 con 0x5634405a8070 2024-08-31T07:30:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 10 client.29005 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 10 client.29005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:30:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 20 client.29005 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5634405669f0 0x56344096b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 4 client.29005 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:30:34.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 -- 172.21.15.32:0/1354678616 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634408c6700 con 0x5634405669f0 2024-08-31T07:30:34.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf86a00640 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5634405669f0 0x56344096b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf86a00640 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5634405669f0 0x56344096b750 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b4beac4bd161031 server_cookie=523bdc2d66fa489a in_seq=0 out_seq=0 2024-08-31T07:30:34.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf70c00640 10 client.29005 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf70c00640 1 -- 172.21.15.32:0/1354678616 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5634408c6700 con 0x5634405669f0 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf70c00640 10 client.29005 handle_command_reply: tid=0 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 client.29005 shutdown 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 -- 172.21.15.32:0/1354678616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 msgr2=0x5634409656c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 0x5634409656c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbf80017a30 tx=0x7fbf80017dc0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 -- 172.21.15.32:0/1354678616 shutdown_connections 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 0x5634409656c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563440587390 0x563440959c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5634405669f0 0x56344096b750 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 --2- 172.21.15.32:0/1354678616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344057f1b0 0x563440959720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 -- 172.21.15.32:0/1354678616 >> 172.21.15.32:0/1354678616 conn(0x56344059c460 msgr2=0x563440950cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf70c00640 0 client.29005 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 -- 172.21.15.32:0/1354678616 shutdown_connections 2024-08-31T07:30:34.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 1 -- 172.21.15.32:0/1354678616 wait complete. 2024-08-31T07:30:34.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.810+0000 7fbf949f1740 20 client.29005 trim_cache size 0 max 16384 2024-08-31T07:30:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.823+0000 7fbf949f1740 1 Processor -- start 2024-08-31T07:30:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.823+0000 7fbf949f1740 1 -- start start 2024-08-31T07:30:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.823+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440965630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.823+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563440587390 0x56344094bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.823+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634405a8070 0x563440959720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.823+0000 7fbf949f1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6800e6c0 con 0x56344057f1b0 2024-08-31T07:30:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.823+0000 7fbf949f1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbf68010260 con 0x5634405a8070 2024-08-31T07:30:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.823+0000 7fbf949f1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6800edc0 con 0x563440587390 2024-08-31T07:30:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.823+0000 7fbf86a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440965630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf87400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634405a8070 0x563440959720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf86a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440965630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52774/0 (socket says 172.21.15.32:52774) 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf86a00640 1 -- 172.21.15.32:0/3440679083 learned_addr learned my addr 172.21.15.32:0/3440679083 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf86000640 1 --2- 172.21.15.32:0/3440679083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563440587390 0x56344094bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf87400640 1 -- 172.21.15.32:0/3440679083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563440587390 msgr2=0x56344094bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf87400640 1 --2- 172.21.15.32:0/3440679083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563440587390 0x56344094bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf87400640 1 -- 172.21.15.32:0/3440679083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 msgr2=0x563440965630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf87400640 1 --2- 172.21.15.32:0/3440679083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440965630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf87400640 1 -- 172.21.15.32:0/3440679083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563440950140 con 0x5634405a8070 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf86a00640 1 --2- 172.21.15.32:0/3440679083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440965630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf87400640 1 --2- 172.21.15.32:0/3440679083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634405a8070 0x563440959720 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbf80005b40 tx=0x7fbf800098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93b0079044c3844e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.829+0000 7fbf73400640 1 -- 172.21.15.32:0/3440679083 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80017e30 con 0x5634405a8070 2024-08-31T07:30:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf73400640 1 -- 172.21.15.32:0/3440679083 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8001c070 con 0x5634405a8070 2024-08-31T07:30:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf73400640 1 -- 172.21.15.32:0/3440679083 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf800050b0 con 0x5634405a8070 2024-08-31T07:30:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- 172.21.15.32:0/3440679083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634405a8070 msgr2=0x563440959720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 --2- 172.21.15.32:0/3440679083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634405a8070 0x563440959720 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbf80005b40 tx=0x7fbf800098c0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- 172.21.15.32:0/3440679083 shutdown_connections 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 --2- 172.21.15.32:0/3440679083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634405a8070 0x563440959720 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 --2- 172.21.15.32:0/3440679083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563440587390 0x56344094bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 --2- 172.21.15.32:0/3440679083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440965630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- 172.21.15.32:0/3440679083 >> 172.21.15.32:0/3440679083 conn(0x56344059c460 msgr2=0x56344094b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- 172.21.15.32:0/3440679083 shutdown_connections 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- 172.21.15.32:0/3440679083 wait complete. 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- start start 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 Processor -- start 2024-08-31T07:30:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440970180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 0x56344096b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 0x56344096bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6800fb90 con 0x56344057f1b0 2024-08-31T07:30:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6800f4c0 con 0x563440587390 2024-08-31T07:30:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6800c6c0 con 0x5634405a8070 2024-08-31T07:30:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 10 client.0 _ll_register_callbacks cb 0x7fbf80009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440970180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440970180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52778/0 (socket says 172.21.15.32:52778) 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86a00640 1 -- 172.21.15.32:0/123491444 learned_addr learned my addr 172.21.15.32:0/123491444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86000640 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 0x56344096b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf87400640 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 0x56344096bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86a00640 1 -- 172.21.15.32:0/123491444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 msgr2=0x56344096bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86a00640 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 0x56344096bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86a00640 1 -- 172.21.15.32:0/123491444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 msgr2=0x56344096b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86a00640 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 0x56344096b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86a00640 1 -- 172.21.15.32:0/123491444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344096c390 con 0x56344057f1b0 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86a00640 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440970180 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800e9c0 tx=0x7fbf780ab6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e270a4271ad255 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf70c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf70c00640 1 -- 172.21.15.32:0/123491444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf780930d0 con 0x56344057f1b0 2024-08-31T07:30:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf70c00640 1 -- 172.21.15.32:0/123491444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fbf780ab080 con 0x56344057f1b0 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- 172.21.15.32:0/123491444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8001c070 con 0x56344057f1b0 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf70c00640 1 -- 172.21.15.32:0/123491444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fbf78093270 con 0x56344057f1b0 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 10 client.18969 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- 172.21.15.32:0/123491444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5634409d5e90 con 0x56344057f1b0 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf70c00640 1 -- 172.21.15.32:0/123491444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8001c070 con 0x56344057f1b0 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf70c00640 1 -- 172.21.15.32:0/123491444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fbf780a7820 con 0x56344057f1b0 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 10 client.18969 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 10 client.18969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 20 client.18969 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56344053bda0 0x5634409d8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 4 client.18969 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:30:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf949f1740 1 -- 172.21.15.32:0/123491444 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634408c21a0 con 0x56344053bda0 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.830+0000 7fbf86000640 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56344053bda0 0x5634409d8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf86000640 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56344053bda0 0x5634409d8600 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47404b3d7f1787f6 server_cookie=20c4f40741cec3ef in_seq=0 out_seq=0 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf70c00640 10 client.18969 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf70c00640 1 -- 172.21.15.32:0/123491444 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634408c21a0 con 0x56344053bda0 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf70c00640 10 client.18969 handle_command_reply: tid=0 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 client.18969 shutdown 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 -- 172.21.15.32:0/123491444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 msgr2=0x563440970180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440970180 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800e9c0 tx=0x7fbf780ab6b0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 -- 172.21.15.32:0/123491444 shutdown_connections 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5634405a8070 0x56344096bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56344053bda0 0x5634409d8600 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563440587390 0x56344096b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 --2- 172.21.15.32:0/123491444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56344057f1b0 0x563440970180 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 -- 172.21.15.32:0/123491444 >> 172.21.15.32:0/123491444 conn(0x56344059c460 msgr2=0x563440959720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf70c00640 0 client.18969 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:30:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 -- 172.21.15.32:0/123491444 shutdown_connections 2024-08-31T07:30:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 1 -- 172.21.15.32:0/123491444 wait complete. 2024-08-31T07:30:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf949f1740 20 client.18969 trim_cache size 0 max 16384 2024-08-31T07:30:34.840 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:30:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 -- 172.21.15.32:0/3514769411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf6c0b4bb0 msgr2=0x7fbf6c0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf6c0b4bb0 0x7fbf6c0b70c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009830 tx=0x7fbf740058d0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 -- 172.21.15.32:0/3514769411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf88075b70 msgr2=0x7fbf881a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf88075b70 0x7fbf881a0680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800e9f0 tx=0x7fbf7800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 -- 172.21.15.32:0/3514769411 shutdown_connections 2024-08-31T07:30:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf6c0b4bb0 0x7fbf6c0b70c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf88114020 0x7fbf881a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf88077120 0x7fbf881a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 --2- 172.21.15.32:0/3514769411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf88075b70 0x7fbf881a0680 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 -- 172.21.15.32:0/3514769411 >> 172.21.15.32:0/3514769411 conn(0x7fbf880ff1e0 msgr2=0x7fbf8810ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 -- 172.21.15.32:0/3514769411 shutdown_connections 2024-08-31T07:30:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:34.836+0000 7fbf87e00640 1 -- 172.21.15.32:0/3514769411 wait complete. 2024-08-31T07:30:35.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 13.83 MB/sec execute 135 sec latency 28522.769 ms 2024-08-31T07:30:35.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:30:36.119 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:36.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84547 13.73 MB/sec execute 136 sec latency 29522.812 ms 2024-08-31T07:30:36.500 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:36 smithi032 ceph-mon[17252]: pgmap v927: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 371 op/s 2024-08-31T07:30:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:36 smithi073 ceph-mon[21995]: pgmap v927: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 371 op/s 2024-08-31T07:30:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:36 smithi084 ceph-mon[22387]: pgmap v927: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 371 op/s 2024-08-31T07:30:36.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.928+0000 7efca3200640 1 -- 172.21.15.32:0/4245763473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4114020 msgr2=0x7efca41164f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:36.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.928+0000 7efca3200640 1 --2- 172.21.15.32:0/4245763473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4114020 0x7efca41164f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009ca0 tx=0x7efc9c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.928+0000 7efca3200640 1 -- 172.21.15.32:0/4245763473 shutdown_connections 2024-08-31T07:30:36.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.928+0000 7efca3200640 1 --2- 172.21.15.32:0/4245763473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4114020 0x7efca41164f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.928+0000 7efca3200640 1 --2- 172.21.15.32:0/4245763473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efca4075b70 0x7efca4075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.928+0000 7efca3200640 1 --2- 172.21.15.32:0/4245763473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efca4077120 0x7efca4075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.928+0000 7efca3200640 1 -- 172.21.15.32:0/4245763473 >> 172.21.15.32:0/4245763473 conn(0x7efca40ff1e0 msgr2=0x7efca4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.928+0000 7efca3200640 1 -- 172.21.15.32:0/4245763473 shutdown_connections 2024-08-31T07:30:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.928+0000 7efca3200640 1 -- 172.21.15.32:0/4245763473 wait complete. 2024-08-31T07:30:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.933+0000 7efca3200640 1 Processor -- start 2024-08-31T07:30:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.933+0000 7efca3200640 1 -- start start 2024-08-31T07:30:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.933+0000 7efca3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4075b70 0x7efca41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.933+0000 7efca3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efca4077120 0x7efca41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.933+0000 7efca3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efca4114020 0x7efca41a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.933+0000 7efca3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efca4119c60 con 0x7efca4075b70 2024-08-31T07:30:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.933+0000 7efca3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efca4119ae0 con 0x7efca4077120 2024-08-31T07:30:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.933+0000 7efca3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efca4119de0 con 0x7efca4114020 2024-08-31T07:30:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efca4114020 0x7efca41a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efca4077120 0x7efca41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efca4114020 0x7efca41a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52954/0 (socket says 172.21.15.32:52954) 2024-08-31T07:30:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca2800640 1 -- 172.21.15.32:0/3534733237 learned_addr learned my addr 172.21.15.32:0/3534733237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca1e00640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4075b70 0x7efca41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca1400640 1 -- 172.21.15.32:0/3534733237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efca4114020 msgr2=0x7efca41a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca1400640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efca4114020 0x7efca41a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca1400640 1 -- 172.21.15.32:0/3534733237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4075b70 msgr2=0x7efca41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca1400640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4075b70 0x7efca41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca1400640 1 -- 172.21.15.32:0/3534733237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca41a6580 con 0x7efca4077120 2024-08-31T07:30:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca1400640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efca4077120 0x7efca41a0bb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e9c0 tx=0x7efc9000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efc9aa00640 1 -- 172.21.15.32:0/3534733237 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9000ccc0 con 0x7efca4077120 2024-08-31T07:30:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca3200640 1 -- 172.21.15.32:0/3534733237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca41a6870 con 0x7efca4077120 2024-08-31T07:30:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efc9aa00640 1 -- 172.21.15.32:0/3534733237 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc90005020 con 0x7efca4077120 2024-08-31T07:30:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efc9aa00640 1 -- 172.21.15.32:0/3534733237 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90004a00 con 0x7efca4077120 2024-08-31T07:30:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.935+0000 7efca3200640 1 -- 172.21.15.32:0/3534733237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca41acfd0 con 0x7efca4077120 2024-08-31T07:30:36.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 Processor -- start 2024-08-31T07:30:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 -- start start 2024-08-31T07:30:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 0x55b5a3ad5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ad60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a370f390 0x55b5a3ad65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b5a3a4c7b0 con 0x55b5a37071b0 2024-08-31T07:30:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b5a3a3bd40 con 0x55b5a3730070 2024-08-31T07:30:36.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b5a3a3fcf0 con 0x55b5a370f390 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 0x55b5a3ad5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 0x55b5a3ad5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47136/0 (socket says 172.21.15.32:47136) 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca2800640 1 -- 172.21.15.32:0/242725074 learned_addr learned my addr 172.21.15.32:0/242725074 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca1e00640 1 --2- 172.21.15.32:0/242725074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ad60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca2800640 1 -- 172.21.15.32:0/242725074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a370f390 msgr2=0x55b5a3ad65e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca2800640 1 --2- 172.21.15.32:0/242725074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a370f390 0x55b5a3ad65e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca2800640 1 -- 172.21.15.32:0/242725074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 msgr2=0x55b5a3ad60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca2800640 1 --2- 172.21.15.32:0/242725074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ad60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca2800640 1 -- 172.21.15.32:0/242725074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a379b650 con 0x55b5a3730070 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efc9aa00640 1 -- 172.21.15.32:0/3534733237 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7efc900051d0 con 0x7efca4077120 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca2800640 1 --2- 172.21.15.32:0/242725074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 0x55b5a3ad5b60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7efc9c005e30 tx=0x7efc9c0094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=851fe4e14d59a022 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:36.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efc8f000640 1 -- 172.21.15.32:0/242725074 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c031cf0 con 0x55b5a3730070 2024-08-31T07:30:36.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efc8f000640 1 -- 172.21.15.32:0/242725074 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9c036040 con 0x55b5a3730070 2024-08-31T07:30:36.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efc8f000640 1 -- 172.21.15.32:0/242725074 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0042a0 con 0x55b5a3730070 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 -- 172.21.15.32:0/242725074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 msgr2=0x55b5a3ad5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 --2- 172.21.15.32:0/242725074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 0x55b5a3ad5b60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7efc9c005e30 tx=0x7efc9c0094a0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 -- 172.21.15.32:0/242725074 shutdown_connections 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 --2- 172.21.15.32:0/242725074 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a370f390 0x55b5a3ad65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 --2- 172.21.15.32:0/242725074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ad60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 --2- 172.21.15.32:0/242725074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 0x55b5a3ad5b60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 -- 172.21.15.32:0/242725074 >> 172.21.15.32:0/242725074 conn(0x55b5a3724460 msgr2=0x55b5a3ad3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efc9aa00640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc880b4d20 0x7efc880b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efc9aa00640 1 -- 172.21.15.32:0/3534733237 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7efc900e5540 con 0x7efca4077120 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 -- 172.21.15.32:0/242725074 shutdown_connections 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efca1e00640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc880b4d20 0x7efc880b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.940+0000 7efcaf7be740 1 -- 172.21.15.32:0/242725074 wait complete. 2024-08-31T07:30:36.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efcaf7be740 1 -- start start 2024-08-31T07:30:36.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efcaf7be740 1 Processor -- start 2024-08-31T07:30:36.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efca1e00640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc880b4d20 0x7efc880b7230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55b5a3ad47e0 tx=0x7efc94005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ae1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a370f390 0x55b5a3ae1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 0x55b5a3aed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efcaf7be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b5a389e700 con 0x55b5a37071b0 2024-08-31T07:30:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efcaf7be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b5a39c2cf0 con 0x55b5a3730070 2024-08-31T07:30:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efcaf7be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b5a3ae42e0 con 0x55b5a370f390 2024-08-31T07:30:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efcaf7be740 10 client.0 _ll_register_callbacks cb 0x55b5a39e8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efca1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ae1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efca1400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a370f390 0x55b5a3ae1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efca1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ae1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38854/0 (socket says 172.21.15.32:38854) 2024-08-31T07:30:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.945+0000 7efca1e00640 1 -- 172.21.15.32:0/641769677 learned_addr learned my addr 172.21.15.32:0/641769677 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efca2800640 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 0x55b5a3aed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efca1400640 1 -- 172.21.15.32:0/641769677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 msgr2=0x55b5a3aed5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:36.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efca1400640 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 0x55b5a3aed5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efca1400640 1 -- 172.21.15.32:0/641769677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 msgr2=0x55b5a3ae1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efca1400640 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ae1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efca1400640 1 -- 172.21.15.32:0/641769677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a3a038b0 con 0x55b5a370f390 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efca1400640 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a370f390 0x55b5a3ae1dd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efc900e2c80 tx=0x7efc900ab7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c831f7c1f03d41eb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efc83400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efc83400640 1 -- 172.21.15.32:0/641769677 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900b0070 con 0x55b5a370f390 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efc83400640 1 -- 172.21.15.32:0/641769677 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7efc900b0210 con 0x55b5a370f390 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 -- 172.21.15.32:0/641769677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc9c036040 con 0x55b5a370f390 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efc83400640 1 -- 172.21.15.32:0/641769677 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7efc9001c070 con 0x55b5a370f390 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 10 client.29020 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 -- 172.21.15.32:0/641769677 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55b5a3af03a0 con 0x55b5a370f390 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efc83400640 1 -- 172.21.15.32:0/641769677 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9c036040 con 0x55b5a370f390 2024-08-31T07:30:36.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efc83400640 1 -- 172.21.15.32:0/641769677 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7efc900b4ab0 con 0x55b5a370f390 2024-08-31T07:30:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 10 client.29020 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 10 client.29020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:30:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 20 client.29020 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b5a36ee9f0 0x55b5a3af3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 4 client.29020 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:30:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 -- 172.21.15.32:0/641769677 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5a3a4e700 con 0x55b5a36ee9f0 2024-08-31T07:30:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efca1e00640 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b5a36ee9f0 0x55b5a3af3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efca1e00640 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b5a36ee9f0 0x55b5a3af3870 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda4855be1d0667e server_cookie=f2dd8d25934b592 in_seq=0 out_seq=0 2024-08-31T07:30:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efc83400640 10 client.29020 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efc83400640 1 -- 172.21.15.32:0/641769677 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b5a3a4e700 con 0x55b5a36ee9f0 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efc83400640 10 client.29020 handle_command_reply: tid=0 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 client.29020 shutdown 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 -- 172.21.15.32:0/641769677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a370f390 msgr2=0x55b5a3ae1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a370f390 0x55b5a3ae1dd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efc900e2c80 tx=0x7efc900ab7f0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 -- 172.21.15.32:0/641769677 shutdown_connections 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b5a36ee9f0 0x55b5a3af3870 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a3730070 0x55b5a3aed5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a370f390 0x55b5a3ae1dd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efc83400640 0 client.29020 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 --2- 172.21.15.32:0/641769677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ae1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 -- 172.21.15.32:0/641769677 >> 172.21.15.32:0/641769677 conn(0x55b5a3724460 msgr2=0x55b5a3ad8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 -- 172.21.15.32:0/641769677 shutdown_connections 2024-08-31T07:30:36.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 1 -- 172.21.15.32:0/641769677 wait complete. 2024-08-31T07:30:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.947+0000 7efcaf7be740 20 client.29020 trim_cache size 0 max 16384 2024-08-31T07:30:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.970+0000 7efcaf7be740 1 Processor -- start 2024-08-31T07:30:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.970+0000 7efcaf7be740 1 -- start start 2024-08-31T07:30:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.970+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3adade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.970+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 0x55b5a3adb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 0x55b5a3ad33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efca41b57c0 con 0x55b5a37071b0 2024-08-31T07:30:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efca41b7360 con 0x55b5a370f390 2024-08-31T07:30:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efca41b5ec0 con 0x55b5a3730070 2024-08-31T07:30:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3adade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3adade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38858/0 (socket says 172.21.15.32:38858) 2024-08-31T07:30:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 -- 172.21.15.32:0/1886954372 learned_addr learned my addr 172.21.15.32:0/1886954372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 0x55b5a3adb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca2800640 1 --2- 172.21.15.32:0/1886954372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 0x55b5a3ad33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 -- 172.21.15.32:0/1886954372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 msgr2=0x55b5a3ad33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 --2- 172.21.15.32:0/1886954372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 0x55b5a3ad33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 -- 172.21.15.32:0/1886954372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 msgr2=0x55b5a3adb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 --2- 172.21.15.32:0/1886954372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 0x55b5a3adb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 -- 172.21.15.32:0/1886954372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a3ad7e50 con 0x55b5a37071b0 2024-08-31T07:30:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 --2- 172.21.15.32:0/1886954372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3adade0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x55b5a3af3ec0 tx=0x7efc94022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa71b17492d9ca9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efc8e600640 1 -- 172.21.15.32:0/1886954372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9402b470 con 0x55b5a37071b0 2024-08-31T07:30:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efc8e600640 1 -- 172.21.15.32:0/1886954372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9402bac0 con 0x55b5a37071b0 2024-08-31T07:30:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efc8e600640 1 -- 172.21.15.32:0/1886954372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9401fa70 con 0x55b5a37071b0 2024-08-31T07:30:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- 172.21.15.32:0/1886954372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 msgr2=0x55b5a3adade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 --2- 172.21.15.32:0/1886954372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3adade0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x55b5a3af3ec0 tx=0x7efc94022040 comp rx=0 tx=0).stop 2024-08-31T07:30:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- 172.21.15.32:0/1886954372 shutdown_connections 2024-08-31T07:30:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 --2- 172.21.15.32:0/1886954372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 0x55b5a3ad33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 --2- 172.21.15.32:0/1886954372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 0x55b5a3adb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 --2- 172.21.15.32:0/1886954372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3adade0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- 172.21.15.32:0/1886954372 >> 172.21.15.32:0/1886954372 conn(0x55b5a3724460 msgr2=0x55b5a3adc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:36.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- 172.21.15.32:0/1886954372 shutdown_connections 2024-08-31T07:30:36.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- 172.21.15.32:0/1886954372 wait complete. 2024-08-31T07:30:36.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- start start 2024-08-31T07:30:36.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 Processor -- start 2024-08-31T07:30:36.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ae9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 0x55b5a3ae44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 0x55b5a3ae4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efca41b6c90 con 0x55b5a37071b0 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efca41b65c0 con 0x55b5a370f390 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efca41b37c0 con 0x55b5a3730070 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efcaf7be740 10 client.0 _ll_register_callbacks cb 0x7efc900ab420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 0x55b5a3ae44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 0x55b5a3ae44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47174/0 (socket says 172.21.15.32:47174) 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1400640 1 -- 172.21.15.32:0/1907624233 learned_addr learned my addr 172.21.15.32:0/1907624233 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ae9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca2800640 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 0x55b5a3ae4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1400640 1 -- 172.21.15.32:0/1907624233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 msgr2=0x55b5a3ae4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1400640 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 0x55b5a3ae4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1400640 1 -- 172.21.15.32:0/1907624233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 msgr2=0x55b5a3ae9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1400640 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ae9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1400640 1 -- 172.21.15.32:0/1907624233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a3ae5070 con 0x55b5a370f390 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca2800640 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 0x55b5a3ae4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.976+0000 7efca1e00640 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ae9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.981+0000 7efca1400640 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 0x55b5a3ae44f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efc900b4e20 tx=0x7efc90009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce65714f9251d3ae server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.981+0000 7efc83400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.981+0000 7efc83400640 1 -- 172.21.15.32:0/1907624233 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900b0040 con 0x55b5a370f390 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.981+0000 7efc83400640 1 -- 172.21.15.32:0/1907624233 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7efc900a8210 con 0x55b5a370f390 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.981+0000 7efcaf7be740 1 -- 172.21.15.32:0/1907624233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc9402bac0 con 0x55b5a370f390 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.981+0000 7efc83400640 1 -- 172.21.15.32:0/1907624233 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7efc900a73b0 con 0x55b5a370f390 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.981+0000 7efcaf7be740 10 client.29030 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.981+0000 7efcaf7be740 1 -- 172.21.15.32:0/1907624233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55b5a3b5ef00 con 0x55b5a370f390 2024-08-31T07:30:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.981+0000 7efc83400640 1 -- 172.21.15.32:0/1907624233 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9402bac0 con 0x55b5a370f390 2024-08-31T07:30:36.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efc83400640 1 -- 172.21.15.32:0/1907624233 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7efc900e5140 con 0x55b5a370f390 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 10 client.29030 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 10 client.29030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 20 client.29030 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b5a36c3da0 0x55b5a3b61560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 4 client.29030 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 -- 172.21.15.32:0/1907624233 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5a3a4a1a0 con 0x55b5a36c3da0 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca1e00640 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b5a36c3da0 0x55b5a3b61560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca1e00640 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b5a36c3da0 0x55b5a3b61560 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e03e56b9386974a1 server_cookie=95d7b97fb44a843f in_seq=0 out_seq=0 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efc83400640 10 client.29030 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efc83400640 1 -- 172.21.15.32:0/1907624233 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5a3a4a1a0 con 0x55b5a36c3da0 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efc83400640 10 client.29030 handle_command_reply: tid=0 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 client.29030 shutdown 2024-08-31T07:30:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 -- 172.21.15.32:0/1907624233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 msgr2=0x55b5a3ae44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 0x55b5a3ae44f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efc900b4e20 tx=0x7efc90009f60 comp rx=0 tx=0).stop 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 -- 172.21.15.32:0/1907624233 shutdown_connections 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b5a36c3da0 0x55b5a3b61560 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b5a3730070 0x55b5a3ae4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5a370f390 0x55b5a3ae44f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 --2- 172.21.15.32:0/1907624233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b5a37071b0 0x55b5a3ae9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 -- 172.21.15.32:0/1907624233 >> 172.21.15.32:0/1907624233 conn(0x55b5a3724460 msgr2=0x55b5a3ae1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efc83400640 0 client.29030 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 -- 172.21.15.32:0/1907624233 shutdown_connections 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 1 -- 172.21.15.32:0/1907624233 wait complete. 2024-08-31T07:30:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efcaf7be740 20 client.29030 trim_cache size 0 max 16384 2024-08-31T07:30:36.988 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:30:36.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 -- 172.21.15.32:0/3534733237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc880b4d20 msgr2=0x7efc880b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:36.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc880b4d20 0x7efc880b7230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55b5a3ad47e0 tx=0x7efc94005f90 comp rx=0 tx=0).stop 2024-08-31T07:30:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 -- 172.21.15.32:0/3534733237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efca4077120 msgr2=0x7efca41a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efca4077120 0x7efca41a0bb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e9c0 tx=0x7efc9000ee80 comp rx=0 tx=0).stop 2024-08-31T07:30:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 -- 172.21.15.32:0/3534733237 shutdown_connections 2024-08-31T07:30:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efc880b4d20 0x7efc880b7230 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efca4114020 0x7efca41a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efca4077120 0x7efca41a0bb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 --2- 172.21.15.32:0/3534733237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4075b70 0x7efca41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 -- 172.21.15.32:0/3534733237 >> 172.21.15.32:0/3534733237 conn(0x7efca40ff1e0 msgr2=0x7efca410ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 -- 172.21.15.32:0/3534733237 shutdown_connections 2024-08-31T07:30:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:36.982+0000 7efca3200640 1 -- 172.21.15.32:0/3534733237 wait complete. 2024-08-31T07:30:37.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84627 13.67 MB/sec execute 137 sec latency 30522.852 ms 2024-08-31T07:30:38.389 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84627 13.57 MB/sec execute 138 sec latency 31522.911 ms 2024-08-31T07:30:38.430 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:38 smithi032 ceph-mon[17252]: pgmap v928: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.7 MiB/s wr, 297 op/s 2024-08-31T07:30:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:38 smithi073 ceph-mon[21995]: pgmap v928: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.7 MiB/s wr, 297 op/s 2024-08-31T07:30:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:38 smithi084 ceph-mon[22387]: pgmap v928: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.7 MiB/s wr, 297 op/s 2024-08-31T07:30:39.171 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:30:39.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:30:39.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84699 13.48 MB/sec execute 139 sec latency 32522.967 ms 2024-08-31T07:30:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:40.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84699 13.39 MB/sec execute 140 sec latency 33523.025 ms 2024-08-31T07:30:40.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:40 smithi032 ceph-mon[17252]: pgmap v929: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 911 KiB/s wr, 143 op/s 2024-08-31T07:30:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:40 smithi073 ceph-mon[21995]: pgmap v929: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 911 KiB/s wr, 143 op/s 2024-08-31T07:30:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:40 smithi084 ceph-mon[22387]: pgmap v929: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 911 KiB/s wr, 143 op/s 2024-08-31T07:30:41.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84699 13.29 MB/sec execute 141 sec latency 34523.081 ms 2024-08-31T07:30:41.396 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:41 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:30:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.526+0000 7fe824600640 1 -- 172.21.15.32:0/3270969833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe81c105b30 msgr2=0x7fe81c105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.526+0000 7fe824600640 1 --2- 172.21.15.32:0/3270969833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe81c105b30 0x7fe81c105f40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe810009900 tx=0x7fe81002a410 comp rx=0 tx=0).stop 2024-08-31T07:30:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.528+0000 7fe824600640 1 -- 172.21.15.32:0/3270969833 shutdown_connections 2024-08-31T07:30:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.528+0000 7fe824600640 1 --2- 172.21.15.32:0/3270969833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe81c10bdc0 0x7fe81c1125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.528+0000 7fe824600640 1 --2- 172.21.15.32:0/3270969833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe81c10b470 0x7fe81c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.528+0000 7fe824600640 1 --2- 172.21.15.32:0/3270969833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe81c105b30 0x7fe81c105f40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.528+0000 7fe824600640 1 -- 172.21.15.32:0/3270969833 >> 172.21.15.32:0/3270969833 conn(0x7fe81c101600 msgr2=0x7fe81c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.528+0000 7fe824600640 1 -- 172.21.15.32:0/3270969833 shutdown_connections 2024-08-31T07:30:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.528+0000 7fe824600640 1 -- 172.21.15.32:0/3270969833 wait complete. 2024-08-31T07:30:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.529+0000 7fe824600640 1 Processor -- start 2024-08-31T07:30:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.529+0000 7fe824600640 1 -- start start 2024-08-31T07:30:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe824600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe81c105b30 0x7fe81c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe824600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe81c10b470 0x7fe81c19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe824600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe81c10bdc0 0x7fe81c1a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe824600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c115a80 con 0x7fe81c10b470 2024-08-31T07:30:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe824600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c115900 con 0x7fe81c10bdc0 2024-08-31T07:30:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe824600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c115c00 con 0x7fe81c105b30 2024-08-31T07:30:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe821200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe81c10b470 0x7fe81c19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe822600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe81c10bdc0 0x7fe81c1a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe821200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe81c10b470 0x7fe81c19ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38892/0 (socket says 172.21.15.32:38892) 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe821200640 1 -- 172.21.15.32:0/520851731 learned_addr learned my addr 172.21.15.32:0/520851731 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe821c00640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe81c105b30 0x7fe81c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe821200640 1 -- 172.21.15.32:0/520851731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe81c105b30 msgr2=0x7fe81c19c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe821200640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe81c105b30 0x7fe81c19c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe821200640 1 -- 172.21.15.32:0/520851731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe81c10bdc0 msgr2=0x7fe81c1a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe821200640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe81c10bdc0 0x7fe81c1a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe821200640 1 -- 172.21.15.32:0/520851731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c1a23e0 con 0x7fe81c10b470 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe822600640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe81c10bdc0 0x7fe81c1a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.530+0000 7fe821c00640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe81c105b30 0x7fe81c19c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.531+0000 7fe821200640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe81c10b470 0x7fe81c19ca10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe80800e970 tx=0x7fe80800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.531+0000 7fe81a000640 1 -- 172.21.15.32:0/520851731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80800cd80 con 0x7fe81c10b470 2024-08-31T07:30:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.531+0000 7fe81a000640 1 -- 172.21.15.32:0/520851731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8080035f0 con 0x7fe81c10b470 2024-08-31T07:30:41.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.531+0000 7fe824600640 1 -- 172.21.15.32:0/520851731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81c1a26d0 con 0x7fe81c10b470 2024-08-31T07:30:41.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.531+0000 7fe81a000640 1 -- 172.21.15.32:0/520851731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808010690 con 0x7fe81c10b470 2024-08-31T07:30:41.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.531+0000 7fe824600640 1 -- 172.21.15.32:0/520851731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81c1a8e30 con 0x7fe81c10b470 2024-08-31T07:30:41.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe82b3ca740 1 Processor -- start 2024-08-31T07:30:41.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe81a000640 1 -- 172.21.15.32:0/520851731 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe808003aa0 con 0x7fe81c10b470 2024-08-31T07:30:41.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe82b3ca740 1 -- start start 2024-08-31T07:30:41.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 0x55acc62b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5ee6490 0x55acc62b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5eee670 0x55acc62b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55acc622b750 con 0x55acc5eee670 2024-08-31T07:30:41.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55acc621ace0 con 0x55acc5f0f350 2024-08-31T07:30:41.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55acc621ec90 con 0x55acc5ee6490 2024-08-31T07:30:41.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe821c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5ee6490 0x55acc62b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe821200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5eee670 0x55acc62b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe822600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 0x55acc62b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe821c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5ee6490 0x55acc62b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52998/0 (socket says 172.21.15.32:52998) 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe822600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 0x55acc62b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47188/0 (socket says 172.21.15.32:47188) 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe821c00640 1 -- 172.21.15.32:0/3345042008 learned_addr learned my addr 172.21.15.32:0/3345042008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe821c00640 1 -- 172.21.15.32:0/3345042008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 msgr2=0x55acc62b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe821c00640 1 --2- 172.21.15.32:0/3345042008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 0x55acc62b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe821c00640 1 -- 172.21.15.32:0/3345042008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5eee670 msgr2=0x55acc62b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe821c00640 1 --2- 172.21.15.32:0/3345042008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5eee670 0x55acc62b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe821c00640 1 -- 172.21.15.32:0/3345042008 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc5f7a930 con 0x55acc5ee6490 2024-08-31T07:30:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.535+0000 7fe821c00640 1 --2- 172.21.15.32:0/3345042008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5ee6490 0x55acc62b5b40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe810009900 tx=0x7fe810005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e86197ca1ce5fee4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe80ea00640 1 -- 172.21.15.32:0/3345042008 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810040070 con 0x55acc5ee6490 2024-08-31T07:30:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe80ea00640 1 -- 172.21.15.32:0/3345042008 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe810031720 con 0x55acc5ee6490 2024-08-31T07:30:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe80ea00640 1 -- 172.21.15.32:0/3345042008 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810035040 con 0x55acc5ee6490 2024-08-31T07:30:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe81a000640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8040b4ca0 0x7fe8040b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe81a000640 1 -- 172.21.15.32:0/520851731 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fe8080e57a0 con 0x7fe81c10b470 2024-08-31T07:30:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe822600640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8040b4ca0 0x7fe8040b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3345042008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5ee6490 msgr2=0x55acc62b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/3345042008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5ee6490 0x55acc62b5b40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe810009900 tx=0x7fe810005e20 comp rx=0 tx=0).stop 2024-08-31T07:30:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3345042008 shutdown_connections 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/3345042008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5eee670 0x55acc62b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/3345042008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5ee6490 0x55acc62b5b40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/3345042008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 0x55acc62b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3345042008 >> 172.21.15.32:0/3345042008 conn(0x55acc5f03740 msgr2=0x55acc62b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe822600640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8040b4ca0 0x7fe8040b71b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe814009fd0 tx=0x7fe814009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3345042008 shutdown_connections 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3345042008 wait complete. 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- start start 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 Processor -- start 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5ee6490 0x55acc62c1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62c1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 0x55acc62cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55acc607d890 con 0x55acc5ee6490 2024-08-31T07:30:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55acc61a1cd0 con 0x55acc5f0f350 2024-08-31T07:30:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55acc62c3d80 con 0x55acc5eee670 2024-08-31T07:30:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 10 client.0 _ll_register_callbacks cb 0x55acc6215ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe821200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe821200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62c1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:53004/0 (socket says 172.21.15.32:53004) 2024-08-31T07:30:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe821200640 1 -- 172.21.15.32:0/2551939927 learned_addr learned my addr 172.21.15.32:0/2551939927 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe822600640 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 0x55acc62cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe821c00640 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5ee6490 0x55acc62c1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe822600640 1 -- 172.21.15.32:0/2551939927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 msgr2=0x55acc62c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe822600640 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62c1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe822600640 1 -- 172.21.15.32:0/2551939927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5ee6490 msgr2=0x55acc62c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe822600640 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5ee6490 0x55acc62c1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe822600640 1 -- 172.21.15.32:0/2551939927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc61eb1b0 con 0x55acc5f0f350 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe821200640 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe822600640 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 0x55acc62cd0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe814005fd0 tx=0x7fe814004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767ae6722374b4ba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe803400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe803400640 1 -- 172.21.15.32:0/2551939927 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140044d0 con 0x55acc5f0f350 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- 172.21.15.32:0/2551939927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe810031720 con 0x55acc5f0f350 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe803400640 1 -- 172.21.15.32:0/2551939927 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fe814018030 con 0x55acc5f0f350 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe803400640 1 -- 172.21.15.32:0/2551939927 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fe814004670 con 0x55acc5f0f350 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 10 client.29042 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- 172.21.15.32:0/2551939927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55acc62cfd70 con 0x55acc5f0f350 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe803400640 1 -- 172.21.15.32:0/2551939927 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe810031720 con 0x55acc5f0f350 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe803400640 1 -- 172.21.15.32:0/2551939927 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fe814028420 con 0x55acc5f0f350 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 10 client.29042 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 10 client.29042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 20 client.29042 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55acc5ecdcd0 0x55acc62d3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 4 client.29042 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe82b3ca740 1 -- 172.21.15.32:0/2551939927 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acc622d6a0 con 0x55acc5ecdcd0 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.537+0000 7fe821c00640 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55acc5ecdcd0 0x55acc62d3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe821c00640 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55acc5ecdcd0 0x55acc62d3380 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db44180f3a025462 server_cookie=52b9ea5b8cabec67 in_seq=0 out_seq=0 2024-08-31T07:30:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe803400640 10 client.29042 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe803400640 1 -- 172.21.15.32:0/2551939927 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55acc622d6a0 con 0x55acc5ecdcd0 2024-08-31T07:30:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe803400640 10 client.29042 handle_command_reply: tid=0 2024-08-31T07:30:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 client.29042 shutdown 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 -- 172.21.15.32:0/2551939927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 msgr2=0x55acc62cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 0x55acc62cd0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe814005fd0 tx=0x7fe814004ab0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 -- 172.21.15.32:0/2551939927 shutdown_connections 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55acc5ecdcd0 0x55acc62d3380 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5f0f350 0x55acc62cd0a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/2551939927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5ee6490 0x55acc62c1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe803400640 0 client.29042 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 -- 172.21.15.32:0/2551939927 >> 172.21.15.32:0/2551939927 conn(0x55acc5f03740 msgr2=0x55acc62b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 -- 172.21.15.32:0/2551939927 shutdown_connections 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 1 -- 172.21.15.32:0/2551939927 wait complete. 2024-08-31T07:30:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.543+0000 7fe82b3ca740 20 client.29042 trim_cache size 0 max 16384 2024-08-31T07:30:41.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe82b3ca740 1 Processor -- start 2024-08-31T07:30:41.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe82b3ca740 1 -- start start 2024-08-31T07:30:41.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5ee6490 0x55acc62c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5f0f350 0x55acc62cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c1b1620 con 0x55acc5f0f350 2024-08-31T07:30:41.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c1b31c0 con 0x55acc5ee6490 2024-08-31T07:30:41.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c1b1d20 con 0x55acc5eee670 2024-08-31T07:30:41.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5ee6490 0x55acc62c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5ee6490 0x55acc62c2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47202/0 (socket says 172.21.15.32:47202) 2024-08-31T07:30:41.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821c00640 1 -- 172.21.15.32:0/4204975351 learned_addr learned my addr 172.21.15.32:0/4204975351 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:41.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe822600640 1 --2- 172.21.15.32:0/4204975351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5f0f350 0x55acc62cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821c00640 1 -- 172.21.15.32:0/4204975351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 msgr2=0x55acc62ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:41.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821c00640 1 --2- 172.21.15.32:0/4204975351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821c00640 1 -- 172.21.15.32:0/4204975351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5f0f350 msgr2=0x55acc62cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821c00640 1 --2- 172.21.15.32:0/4204975351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5f0f350 0x55acc62cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821c00640 1 -- 172.21.15.32:0/4204975351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc62c21c0 con 0x55acc5ee6490 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821200640 1 --2- 172.21.15.32:0/4204975351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe821c00640 1 --2- 172.21.15.32:0/4204975351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5ee6490 0x55acc62c2b30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe810033000 tx=0x7fe810005530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=831e83fe8462c8f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe80e000640 1 -- 172.21.15.32:0/4204975351 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810035040 con 0x55acc5ee6490 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe80e000640 1 -- 172.21.15.32:0/4204975351 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe810031760 con 0x55acc5ee6490 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe80e000640 1 -- 172.21.15.32:0/4204975351 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810040040 con 0x55acc5ee6490 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe82b3ca740 1 -- 172.21.15.32:0/4204975351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5ee6490 msgr2=0x55acc62c2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.561+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/4204975351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5ee6490 0x55acc62c2b30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe810033000 tx=0x7fe810005530 comp rx=0 tx=0).stop 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- 172.21.15.32:0/4204975351 shutdown_connections 2024-08-31T07:30:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/4204975351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5f0f350 0x55acc62cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/4204975351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5eee670 0x55acc62ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/4204975351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5ee6490 0x55acc62c2b30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- 172.21.15.32:0/4204975351 >> 172.21.15.32:0/4204975351 conn(0x55acc5f03740 msgr2=0x55acc62c2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- 172.21.15.32:0/4204975351 shutdown_connections 2024-08-31T07:30:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- 172.21.15.32:0/4204975351 wait complete. 2024-08-31T07:30:41.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- start start 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 Processor -- start 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5ee6490 0x55acc62c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5eee670 0x55acc62c6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5f0f350 0x55acc6336e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c1b2af0 con 0x55acc5ee6490 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c1b2420 con 0x55acc5eee670 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c1af620 con 0x55acc5f0f350 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 10 client.0 _ll_register_callbacks cb 0x7fe8140126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5eee670 0x55acc62c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5eee670 0x55acc62c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47206/0 (socket says 172.21.15.32:47206) 2024-08-31T07:30:41.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821200640 1 -- 172.21.15.32:0/3922282745 learned_addr learned my addr 172.21.15.32:0/3922282745 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821c00640 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5ee6490 0x55acc62c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe822600640 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5f0f350 0x55acc6336e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821200640 1 -- 172.21.15.32:0/3922282745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5f0f350 msgr2=0x55acc6336e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821200640 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5f0f350 0x55acc6336e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821200640 1 -- 172.21.15.32:0/3922282745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5ee6490 msgr2=0x55acc62c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821200640 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5ee6490 0x55acc62c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821200640 1 -- 172.21.15.32:0/3922282745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc6337480 con 0x55acc5eee670 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821200640 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5eee670 0x55acc62c6090 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe8080058c0 tx=0x7fe8080ac3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477896df5e11095e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe803400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe803400640 1 -- 172.21.15.32:0/3922282745 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8080ab650 con 0x55acc5eee670 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3922282745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe810031760 con 0x55acc5eee670 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe803400640 1 -- 172.21.15.32:0/3922282745 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7fe8080a84d0 con 0x55acc5eee670 2024-08-31T07:30:41.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe803400640 1 -- 172.21.15.32:0/3922282745 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fe8080ab9f0 con 0x55acc5eee670 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 10 client.29048 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3922282745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55acc633e910 con 0x55acc5eee670 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe803400640 1 -- 172.21.15.32:0/3922282745 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe810031760 con 0x55acc5eee670 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe803400640 1 -- 172.21.15.32:0/3922282745 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fe8080b0390 con 0x55acc5eee670 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 10 client.29048 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 10 client.29048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 20 client.29048 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55acc5ea3080 0x55acc6341090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 4 client.29048 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3922282745 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55acc6229140 con 0x55acc5ea3080 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821c00640 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55acc5ea3080 0x55acc6341090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:41.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe803400640 10 client.29048 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:41.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.566+0000 7fe821c00640 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55acc5ea3080 0x55acc6341090 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f166731536da90 server_cookie=606720882376aa71 in_seq=0 out_seq=0 2024-08-31T07:30:41.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.571+0000 7fe803400640 1 -- 172.21.15.32:0/3922282745 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55acc6229140 con 0x55acc5ea3080 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.571+0000 7fe803400640 10 client.29048 handle_command_reply: tid=0 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.571+0000 7fe82b3ca740 1 client.29048 shutdown 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.571+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3922282745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5eee670 msgr2=0x55acc62c6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.571+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5eee670 0x55acc62c6090 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe8080058c0 tx=0x7fe8080ac3c0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.571+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3922282745 shutdown_connections 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.571+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55acc5ea3080 0x55acc6341090 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc5f0f350 0x55acc6336e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe803400640 0 client.29048 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acc5eee670 0x55acc62c6090 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe82b3ca740 1 --2- 172.21.15.32:0/3922282745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acc5ee6490 0x55acc62c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3922282745 >> 172.21.15.32:0/3922282745 conn(0x55acc5f03740 msgr2=0x55acc62b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3922282745 shutdown_connections 2024-08-31T07:30:41.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe82b3ca740 1 -- 172.21.15.32:0/3922282745 wait complete. 2024-08-31T07:30:41.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe82b3ca740 20 client.29048 trim_cache size 0 max 16384 2024-08-31T07:30:41.576 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:30:41.576 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:30:41.576 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "d52d6274-7572-465a-9504-ef90242d910a", 2024-08-31T07:30:41.576 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:30:41.576 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 -- 172.21.15.32:0/520851731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8040b4ca0 msgr2=0x7fe8040b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8040b4ca0 0x7fe8040b71b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe814009fd0 tx=0x7fe814009500 comp rx=0 tx=0).stop 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 -- 172.21.15.32:0/520851731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe81c10b470 msgr2=0x7fe81c19ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe81c10b470 0x7fe81c19ca10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe80800e970 tx=0x7fe80800ee30 comp rx=0 tx=0).stop 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 -- 172.21.15.32:0/520851731 shutdown_connections 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fe8040b4ca0 0x7fe8040b71b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe81c10bdc0 0x7fe81c1a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe81c10b470 0x7fe81c19ca10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 --2- 172.21.15.32:0/520851731 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe81c105b30 0x7fe81c19c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 -- 172.21.15.32:0/520851731 >> 172.21.15.32:0/520851731 conn(0x7fe81c101600 msgr2=0x7fe81c10ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 -- 172.21.15.32:0/520851731 shutdown_connections 2024-08-31T07:30:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:41.573+0000 7fe824600640 1 -- 172.21.15.32:0/520851731 wait complete. 2024-08-31T07:30:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:41 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:30:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:41 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:30:42.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84699 13.20 MB/sec execute 142 sec latency 35523.152 ms 2024-08-31T07:30:42.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:42 smithi032 ceph-mon[17252]: pgmap v930: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 614 KiB/s wr, 120 op/s 2024-08-31T07:30:42.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:42 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:30:42.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:42 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:42.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:42 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:30:42.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:42 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:30:42.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:42 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:30:42.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:42 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:30:42.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:42 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:42.511 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:42 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:42 smithi073 ceph-mon[21995]: pgmap v930: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 614 KiB/s wr, 120 op/s 2024-08-31T07:30:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:42 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:30:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:42 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:42 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:30:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:42 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:30:42.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:42 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:30:42.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:42 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:30:42.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:42 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:42.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:42 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:42.718 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:30:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:42 smithi084 ceph-mon[22387]: pgmap v930: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 614 KiB/s wr, 120 op/s 2024-08-31T07:30:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:42 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:30:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:42 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:42 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:30:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:42 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:30:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:42 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:30:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:42 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:30:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:42 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:42.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:42 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:43.353 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:43 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:30:43.353 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:43 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:43.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84699 13.11 MB/sec execute 143 sec latency 36523.222 ms 2024-08-31T07:30:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:43 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:30:43.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:43 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:43.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:43 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:30:43.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:43 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.175+0000 7f3cd4a00640 1 -- 172.21.15.32:0/416772239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cd0104640 msgr2=0x7f3cd0104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.175+0000 7f3cd4a00640 1 --2- 172.21.15.32:0/416772239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cd0104640 0x7f3cd0104a50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009900 tx=0x7f3cc002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.175+0000 7f3cd4a00640 1 -- 172.21.15.32:0/416772239 shutdown_connections 2024-08-31T07:30:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.175+0000 7f3cd4a00640 1 --2- 172.21.15.32:0/416772239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd010dbf0 0x7f3cd01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.175+0000 7f3cd4a00640 1 --2- 172.21.15.32:0/416772239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cd0105020 0x7f3cd010d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.175+0000 7f3cd4a00640 1 --2- 172.21.15.32:0/416772239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cd0104640 0x7f3cd0104a50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.175+0000 7f3cd4a00640 1 -- 172.21.15.32:0/416772239 >> 172.21.15.32:0/416772239 conn(0x7f3cd00794c0 msgr2=0x7f3cd00798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.175+0000 7f3cd4a00640 1 -- 172.21.15.32:0/416772239 shutdown_connections 2024-08-31T07:30:44.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.175+0000 7f3cd4a00640 1 -- 172.21.15.32:0/416772239 wait complete. 2024-08-31T07:30:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cd4a00640 1 Processor -- start 2024-08-31T07:30:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cd4a00640 1 -- start start 2024-08-31T07:30:44.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cd4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd0104640 0x7f3cd006ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cd4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cd0105020 0x7f3cd006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cd4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cd010dbf0 0x7f3cd00726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cd4a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01138a0 con 0x7f3cd0104640 2024-08-31T07:30:44.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cd4a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0113720 con 0x7f3cd0105020 2024-08-31T07:30:44.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cd4a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0113a20 con 0x7f3cd010dbf0 2024-08-31T07:30:44.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3ccd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cd0105020 0x7f3cd006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3ccd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cd0105020 0x7f3cd006d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47220/0 (socket says 172.21.15.32:47220) 2024-08-31T07:30:44.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3ccd400640 1 -- 172.21.15.32:0/2588814966 learned_addr learned my addr 172.21.15.32:0/2588814966 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:44.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3ccde00640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd0104640 0x7f3cd006ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3ccd400640 1 -- 172.21.15.32:0/2588814966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cd010dbf0 msgr2=0x7f3cd00726b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:30:44.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3ccd400640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cd010dbf0 0x7f3cd00726b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3ccd400640 1 -- 172.21.15.32:0/2588814966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd0104640 msgr2=0x7f3cd006ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:44.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3ccd400640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd0104640 0x7f3cd006ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3ccd400640 1 -- 172.21.15.32:0/2588814966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd0072de0 con 0x7f3cd0105020 2024-08-31T07:30:44.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3ccd400640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cd0105020 0x7f3cd006d3f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00e9c0 tx=0x7f3cbc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:44.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cc6a00640 1 -- 172.21.15.32:0/2588814966 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00ccc0 con 0x7f3cd0105020 2024-08-31T07:30:44.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cc6a00640 1 -- 172.21.15.32:0/2588814966 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cbc0054d0 con 0x7f3cd0105020 2024-08-31T07:30:44.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cc6a00640 1 -- 172.21.15.32:0/2588814966 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc010430 con 0x7f3cd0105020 2024-08-31T07:30:44.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cd4a00640 1 -- 172.21.15.32:0/2588814966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd0073070 con 0x7f3cd0105020 2024-08-31T07:30:44.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.181+0000 7f3cd4a00640 1 -- 172.21.15.32:0/2588814966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd0073820 con 0x7f3cd0105020 2024-08-31T07:30:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cdb7dd740 1 Processor -- start 2024-08-31T07:30:44.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cdb7dd740 1 -- start start 2024-08-31T07:30:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cc6a00640 1 -- 172.21.15.32:0/2588814966 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3cbc005020 con 0x7f3cd0105020 2024-08-31T07:30:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d961870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5931b0 0x55e24d961db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d9622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e24d8d8770 con 0x55e24d59b390 2024-08-31T07:30:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e24d8c7d00 con 0x55e24d5931b0 2024-08-31T07:30:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e24d8cbcb0 con 0x55e24d5bc070 2024-08-31T07:30:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cce800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d961870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cce800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d961870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:53064/0 (socket says 172.21.15.32:53064) 2024-08-31T07:30:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cce800640 1 -- 172.21.15.32:0/2549511199 learned_addr learned my addr 172.21.15.32:0/2549511199 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3ccd400640 1 --2- 172.21.15.32:0/2549511199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d9622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cce800640 1 -- 172.21.15.32:0/2549511199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5931b0 msgr2=0x55e24d961db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:30:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cce800640 1 --2- 172.21.15.32:0/2549511199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5931b0 0x55e24d961db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cce800640 1 -- 172.21.15.32:0/2549511199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 msgr2=0x55e24d9622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:44.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cce800640 1 --2- 172.21.15.32:0/2549511199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d9622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cce800640 1 -- 172.21.15.32:0/2549511199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24d627650 con 0x55e24d5bc070 2024-08-31T07:30:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cce800640 1 --2- 172.21.15.32:0/2549511199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d961870 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009d60 tx=0x7f3cc800d6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3dbd5cc1bf68a5c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cbb000640 1 -- 172.21.15.32:0/2549511199 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8003970 con 0x55e24d5bc070 2024-08-31T07:30:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.186+0000 7f3cbb000640 1 -- 172.21.15.32:0/2549511199 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc8004020 con 0x55e24d5bc070 2024-08-31T07:30:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cbb000640 1 -- 172.21.15.32:0/2549511199 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80107f0 con 0x55e24d5bc070 2024-08-31T07:30:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cc6a00640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cb40b4df0 0x7f3cb40b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cc6a00640 1 -- 172.21.15.32:0/2588814966 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f3cbc0e5830 con 0x7f3cd0105020 2024-08-31T07:30:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/2549511199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 msgr2=0x55e24d961870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/2549511199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d961870 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009d60 tx=0x7f3cc800d6b0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccde00640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cb40b4df0 0x7f3cb40b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/2549511199 shutdown_connections 2024-08-31T07:30:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/2549511199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d9622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/2549511199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5931b0 0x55e24d961db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/2549511199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d961870 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/2549511199 >> 172.21.15.32:0/2549511199 conn(0x55e24d5b0460 msgr2=0x55e24d95f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/2549511199 shutdown_connections 2024-08-31T07:30:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/2549511199 wait complete. 2024-08-31T07:30:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 -- start start 2024-08-31T07:30:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 Processor -- start 2024-08-31T07:30:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccde00640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cb40b4df0 0x7f3cb40b7300 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0005ab0 tx=0x7f3cc0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5931b0 0x55e24d96d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d96d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d981b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e24d72a700 con 0x55e24d59b390 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e24d84ecb0 con 0x55e24d5931b0 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e24d970050 con 0x55e24d5bc070 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cdb7dd740 10 client.0 _ll_register_callbacks cb 0x55e24d874430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5931b0 0x55e24d96d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d96d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5931b0 0x55e24d96d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47222/0 (socket says 172.21.15.32:47222) 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccde00640 1 -- 172.21.15.32:0/3531140643 learned_addr learned my addr 172.21.15.32:0/3531140643 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cce800640 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d981b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccd400640 1 -- 172.21.15.32:0/3531140643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 msgr2=0x55e24d981b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccd400640 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d981b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccd400640 1 -- 172.21.15.32:0/3531140643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5931b0 msgr2=0x55e24d96d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccd400640 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5931b0 0x55e24d96d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccd400640 1 -- 172.21.15.32:0/3531140643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e24d88f870 con 0x55e24d59b390 2024-08-31T07:30:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3cce800640 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d981b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3ccd400640 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d96d8e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0e2f70 tx=0x7f3cbc0ab740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c173cac62dcbbb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.191+0000 7f3caf400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:30:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.196+0000 7f3caf400640 1 -- 172.21.15.32:0/3531140643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc005680 con 0x55e24d59b390 2024-08-31T07:30:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.196+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3531140643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc8004020 con 0x55e24d59b390 2024-08-31T07:30:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.196+0000 7f3caf400640 1 -- 172.21.15.32:0/3531140643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f3cbc0b4fb0 con 0x55e24d59b390 2024-08-31T07:30:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.196+0000 7f3caf400640 1 -- 172.21.15.32:0/3531140643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f3cbc0aba20 con 0x55e24d59b390 2024-08-31T07:30:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.196+0000 7f3cdb7dd740 10 client.19017 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.196+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3531140643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55e24d9848d0 con 0x55e24d59b390 2024-08-31T07:30:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3caf400640 1 -- 172.21.15.32:0/3531140643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc8004020 con 0x55e24d59b390 2024-08-31T07:30:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3caf400640 1 -- 172.21.15.32:0/3531140643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f3cbc0b0390 con 0x55e24d59b390 2024-08-31T07:30:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 10 client.19017 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 10 client.19017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:30:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 20 client.19017 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e24d57a9f0 0x55e24d987de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 4 client.19017 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:30:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3531140643 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e24d8da6c0 con 0x55e24d57a9f0 2024-08-31T07:30:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3ccde00640 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e24d57a9f0 0x55e24d987de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3ccde00640 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e24d57a9f0 0x55e24d987de0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e261cceb7ded6c server_cookie=3d20d16bf6a78f38 in_seq=0 out_seq=0 2024-08-31T07:30:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3caf400640 10 client.19017 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:44.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3caf400640 1 -- 172.21.15.32:0/3531140643 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e24d8da6c0 con 0x55e24d57a9f0 2024-08-31T07:30:44.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3caf400640 10 client.19017 handle_command_reply: tid=0 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 client.19017 shutdown 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3531140643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 msgr2=0x55e24d96d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d96d8e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0e2f70 tx=0x7f3cbc0ab740 comp rx=0 tx=0).stop 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3531140643 shutdown_connections 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e24d57a9f0 0x55e24d987de0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5bc070 0x55e24d981b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d96d8e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3531140643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5931b0 0x55e24d96d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3531140643 >> 172.21.15.32:0/3531140643 conn(0x55e24d5b0460 msgr2=0x55e24d964a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3caf400640 0 client.19017 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3531140643 shutdown_connections 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3531140643 wait complete. 2024-08-31T07:30:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.198+0000 7f3cdb7dd740 20 client.19017 trim_cache size 0 max 16384 2024-08-31T07:30:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cdb7dd740 1 Processor -- start 2024-08-31T07:30:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cdb7dd740 1 -- start start 2024-08-31T07:30:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5931b0 0x55e24d95e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d962fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 0x55e24d981b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01b5900 con 0x55e24d59b390 2024-08-31T07:30:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01b74a0 con 0x55e24d5bc070 2024-08-31T07:30:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01b6000 con 0x55e24d5931b0 2024-08-31T07:30:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3ccde00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5931b0 0x55e24d95e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3ccd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d962fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3ccde00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5931b0 0x55e24d95e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:53082/0 (socket says 172.21.15.32:53082) 2024-08-31T07:30:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3ccde00640 1 -- 172.21.15.32:0/2630987925 learned_addr learned my addr 172.21.15.32:0/2630987925 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3ccde00640 1 -- 172.21.15.32:0/2630987925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 msgr2=0x55e24d981b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:30:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3ccde00640 1 --2- 172.21.15.32:0/2630987925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 0x55e24d981b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3ccde00640 1 -- 172.21.15.32:0/2630987925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 msgr2=0x55e24d962fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3ccde00640 1 --2- 172.21.15.32:0/2630987925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d962fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3ccde00640 1 -- 172.21.15.32:0/2630987925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24d96e240 con 0x55e24d5931b0 2024-08-31T07:30:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3ccde00640 1 --2- 172.21.15.32:0/2630987925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5931b0 0x55e24d95e760 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3cc002a9e0 tx=0x7f3cc0040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d3f29a4e69ebb3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cba600640 1 -- 172.21.15.32:0/2630987925 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0035040 con 0x55e24d5931b0 2024-08-31T07:30:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cba600640 1 -- 172.21.15.32:0/2630987925 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc0057070 con 0x55e24d5931b0 2024-08-31T07:30:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.221+0000 7f3cba600640 1 -- 172.21.15.32:0/2630987925 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0003ad0 con 0x55e24d5931b0 2024-08-31T07:30:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/2630987925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5931b0 msgr2=0x55e24d95e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/2630987925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5931b0 0x55e24d95e760 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3cc002a9e0 tx=0x7f3cc0040000 comp rx=0 tx=0).stop 2024-08-31T07:30:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/2630987925 shutdown_connections 2024-08-31T07:30:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/2630987925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 0x55e24d981b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/2630987925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d59b390 0x55e24d962fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/2630987925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d5931b0 0x55e24d95e760 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/2630987925 >> 172.21.15.32:0/2630987925 conn(0x55e24d5b0460 msgr2=0x55e24d9628a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/2630987925 shutdown_connections 2024-08-31T07:30:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/2630987925 wait complete. 2024-08-31T07:30:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- start start 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 Processor -- start 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d5931b0 0x55e24d97a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d59b390 0x55e24d97a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 0x55e24d9f1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01b6dd0 con 0x55e24d5931b0 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01b6700 con 0x55e24d5bc070 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01b3920 con 0x55e24d59b390 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 10 client.0 _ll_register_callbacks cb 0x7f3cbc0ab860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 0x55e24d9f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 0x55e24d9f1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47232/0 (socket says 172.21.15.32:47232) 2024-08-31T07:30:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cce800640 1 -- 172.21.15.32:0/3009678990 learned_addr learned my addr 172.21.15.32:0/3009678990 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3ccd400640 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d59b390 0x55e24d97a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cce800640 1 -- 172.21.15.32:0/3009678990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d59b390 msgr2=0x55e24d97a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cce800640 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d59b390 0x55e24d97a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3ccde00640 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d5931b0 0x55e24d97a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cce800640 1 -- 172.21.15.32:0/3009678990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d5931b0 msgr2=0x55e24d97a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cce800640 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d5931b0 0x55e24d97a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cce800640 1 -- 172.21.15.32:0/3009678990 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e24d9f1cb0 con 0x55e24d5bc070 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3ccde00640 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d5931b0 0x55e24d97a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cce800640 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 0x55e24d9f1770 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009d60 tx=0x7f3cc8004f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd8dc73157edbf07 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3caf400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3caf400640 1 -- 172.21.15.32:0/3009678990 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc802a060 con 0x55e24d5bc070 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3caf400640 1 -- 172.21.15.32:0/3009678990 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f3cc8005240 con 0x55e24d5bc070 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3009678990 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc0057070 con 0x55e24d5bc070 2024-08-31T07:30:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3caf400640 1 -- 172.21.15.32:0/3009678990 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f3cc8005440 con 0x55e24d5bc070 2024-08-31T07:30:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 10 client.29060 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3009678990 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55e24d9f2f00 con 0x55e24d5bc070 2024-08-31T07:30:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.227+0000 7f3caf400640 1 -- 172.21.15.32:0/3009678990 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc0057070 con 0x55e24d5bc070 2024-08-31T07:30:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.232+0000 7f3caf400640 1 -- 172.21.15.32:0/3009678990 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f3cc8025aa0 con 0x55e24d5bc070 2024-08-31T07:30:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.232+0000 7f3cdb7dd740 10 client.29060 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.232+0000 7f3cdb7dd740 10 client.29060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:30:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.232+0000 7f3cdb7dd740 20 client.29060 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.232+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e24d54fda0 0x55e24d9f5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.232+0000 7f3cdb7dd740 4 client.29060 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:30:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.232+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3009678990 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e24d8d6160 con 0x55e24d54fda0 2024-08-31T07:30:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.232+0000 7f3ccde00640 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e24d54fda0 0x55e24d9f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3caf400640 10 client.29060 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3ccde00640 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e24d54fda0 0x55e24d9f5430 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73f42cc777b517b server_cookie=a0a215e96013424f in_seq=0 out_seq=0 2024-08-31T07:30:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3caf400640 1 -- 172.21.15.32:0/3009678990 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6250 (crc 0 0 0) 0x55e24d8d6160 con 0x55e24d54fda0 2024-08-31T07:30:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3caf400640 10 client.29060 handle_command_reply: tid=0 2024-08-31T07:30:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 client.29060 shutdown 2024-08-31T07:30:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3009678990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 msgr2=0x55e24d9f1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 0x55e24d9f1770 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009d60 tx=0x7f3cc8004f20 comp rx=0 tx=0).stop 2024-08-31T07:30:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3009678990 shutdown_connections 2024-08-31T07:30:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e24d54fda0 0x55e24d9f5430 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e24d5bc070 0x55e24d9f1770 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3caf400640 0 client.29060 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e24d59b390 0x55e24d97a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 --2- 172.21.15.32:0/3009678990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e24d5931b0 0x55e24d97a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3009678990 >> 172.21.15.32:0/3009678990 conn(0x55e24d5b0460 msgr2=0x55e24d95e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3009678990 shutdown_connections 2024-08-31T07:30:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 1 -- 172.21.15.32:0/3009678990 wait complete. 2024-08-31T07:30:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cdb7dd740 20 client.29060 trim_cache size 0 max 16384 2024-08-31T07:30:44.236 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:30:44.236 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "d52d6274-7572-465a-9504-ef90242d910a", 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.237 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.238 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.240 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.241 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:30:44.242 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:30:44.243 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.244 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.245 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.246 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.247 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:30:44.248 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:30:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:30:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:30:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:30:44.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:30:44.249 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:30:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 -- 172.21.15.32:0/2588814966 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cb40b4df0 msgr2=0x7f3cb40b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cb40b4df0 0x7f3cb40b7300 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0005ab0 tx=0x7f3cc0005a20 comp rx=0 tx=0).stop 2024-08-31T07:30:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 -- 172.21.15.32:0/2588814966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cd0105020 msgr2=0x7f3cd006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cd0105020 0x7f3cd006d3f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00e9c0 tx=0x7f3cbc00ee80 comp rx=0 tx=0).stop 2024-08-31T07:30:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 -- 172.21.15.32:0/2588814966 shutdown_connections 2024-08-31T07:30:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f3cb40b4df0 0x7f3cb40b7300 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cd010dbf0 0x7f3cd00726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cd0105020 0x7f3cd006d3f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 --2- 172.21.15.32:0/2588814966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd0104640 0x7f3cd006ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 -- 172.21.15.32:0/2588814966 >> 172.21.15.32:0/2588814966 conn(0x7f3cd00794c0 msgr2=0x7f3cd01005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.233+0000 7f3cd4a00640 1 -- 172.21.15.32:0/2588814966 shutdown_connections 2024-08-31T07:30:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:44.238+0000 7f3cd4a00640 1 -- 172.21.15.32:0/2588814966 wait complete. 2024-08-31T07:30:44.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84806 13.05 MB/sec execute 144 sec latency 36865.989 ms 2024-08-31T07:30:44.447 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:44 smithi032 ceph-mon[17252]: pgmap v931: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 484 KiB/s wr, 70 op/s 2024-08-31T07:30:44.447 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:30:44.447 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:30:44.447 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:30:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:44 smithi073 ceph-mon[21995]: pgmap v931: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 484 KiB/s wr, 70 op/s 2024-08-31T07:30:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:30:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:30:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:30:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:44 smithi084 ceph-mon[22387]: pgmap v931: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 484 KiB/s wr, 70 op/s 2024-08-31T07:30:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:30:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:30:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:30:45.205 INFO:tasks.cephfs.filesystem:scrub status for tag:d52d6274-7572-465a-9504-ef90242d910a - {'path': '/', 'tag': 'd52d6274-7572-465a-9504-ef90242d910a', 'options': 'recursive,force'} 2024-08-31T07:30:45.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84835 12.96 MB/sec execute 145 sec latency 27263.871 ms 2024-08-31T07:30:46.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 85566 13.08 MB/sec execute 146 sec latency 27320.424 ms 2024-08-31T07:30:46.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:46 smithi032 ceph-mon[17252]: pgmap v932: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 426 KiB/s wr, 52 op/s 2024-08-31T07:30:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:46 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:46 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:46 smithi073 ceph-mon[21995]: pgmap v932: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 426 KiB/s wr, 52 op/s 2024-08-31T07:30:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:46 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:46 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:46 smithi084 ceph-mon[22387]: pgmap v932: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 426 KiB/s wr, 52 op/s 2024-08-31T07:30:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:46 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:46 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:30:47.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 86595 13.38 MB/sec execute 147 sec latency 56.071 ms 2024-08-31T07:30:48.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 87458 13.65 MB/sec execute 148 sec latency 86.392 ms 2024-08-31T07:30:48.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:48 smithi032 ceph-mon[17252]: pgmap v933: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 868 KiB/s wr, 94 op/s 2024-08-31T07:30:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:48 smithi073 ceph-mon[21995]: pgmap v933: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 868 KiB/s wr, 94 op/s 2024-08-31T07:30:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:48 smithi084 ceph-mon[22387]: pgmap v933: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 868 KiB/s wr, 94 op/s 2024-08-31T07:30:49.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 88259 13.84 MB/sec execute 149 sec latency 308.528 ms 2024-08-31T07:30:50.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 88858 13.93 MB/sec execute 150 sec latency 774.431 ms 2024-08-31T07:30:50.541 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:50 smithi084 ceph-mon[22387]: pgmap v934: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-08-31T07:30:50.545 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 31.7 2024-08-31T07:30:50.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:30:50.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:50 smithi032 ceph-mon[17252]: pgmap v934: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-08-31T07:30:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:50 smithi073 ceph-mon[21995]: pgmap v934: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-08-31T07:30:51.097 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:51.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 89662 14.13 MB/sec execute 151 sec latency 814.177 ms 2024-08-31T07:30:52.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 90108 14.21 MB/sec execute 152 sec latency 1814.230 ms 2024-08-31T07:30:52.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:52 smithi032 ceph-mon[17252]: pgmap v935: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 295 op/s 2024-08-31T07:30:52.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:52 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1913993740' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:30:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:52 smithi073 ceph-mon[21995]: pgmap v935: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 295 op/s 2024-08-31T07:30:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:52 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1913993740' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:30:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:52 smithi084 ceph-mon[22387]: pgmap v935: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 295 op/s 2024-08-31T07:30:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:52 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1913993740' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:30:53.391 INFO:tasks.workunit.client.0.smithi032.stdout: 10 90548 14.25 MB/sec execute 153 sec latency 2814.285 ms 2024-08-31T07:30:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.882+0000 7f7ef6400640 1 -- 172.21.15.32:0/3165828088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0077120 msgr2=0x7f7ef0075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.882+0000 7f7ef6400640 1 --2- 172.21.15.32:0/3165828088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0077120 0x7f7ef0075550 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009900 tx=0x7f7ee002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.882+0000 7f7ef6400640 1 -- 172.21.15.32:0/3165828088 shutdown_connections 2024-08-31T07:30:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.882+0000 7f7ef6400640 1 --2- 172.21.15.32:0/3165828088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef01141d0 0x7f7ef01166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.882+0000 7f7ef6400640 1 --2- 172.21.15.32:0/3165828088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0075b70 0x7f7ef0075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.882+0000 7f7ef6400640 1 --2- 172.21.15.32:0/3165828088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0077120 0x7f7ef0075550 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.882+0000 7f7ef6400640 1 -- 172.21.15.32:0/3165828088 >> 172.21.15.32:0/3165828088 conn(0x7f7ef007b5f0 msgr2=0x7f7ef007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.882+0000 7f7ef6400640 1 -- 172.21.15.32:0/3165828088 shutdown_connections 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.882+0000 7f7ef6400640 1 -- 172.21.15.32:0/3165828088 wait complete. 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.883+0000 7f7ef6400640 1 Processor -- start 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.884+0000 7f7ef6400640 1 -- start start 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.884+0000 7f7ef6400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0075b70 0x7f7ef01a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.884+0000 7f7ef6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0077120 0x7f7ef01a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.884+0000 7f7ef6400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef01141d0 0x7f7ef01a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.884+0000 7f7ef6400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0119e10 con 0x7f7ef0077120 2024-08-31T07:30:53.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.884+0000 7f7ef6400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0119c90 con 0x7f7ef0075b70 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.884+0000 7f7ef6400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0119f90 con 0x7f7ef01141d0 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0075b70 0x7f7ef01a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eefe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef01141d0 0x7f7ef01a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eeea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0077120 0x7f7ef01a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0075b70 0x7f7ef01a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:39574/0 (socket says 172.21.15.32:39574) 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eefe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef01141d0 0x7f7ef01a60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:48446/0 (socket says 172.21.15.32:48446) 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eef400640 1 -- 172.21.15.32:0/1913993740 learned_addr learned my addr 172.21.15.32:0/1913993740 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eef400640 1 -- 172.21.15.32:0/1913993740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef01141d0 msgr2=0x7f7ef01a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eef400640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef01141d0 0x7f7ef01a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eef400640 1 -- 172.21.15.32:0/1913993740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0077120 msgr2=0x7f7ef01a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eef400640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0077120 0x7f7ef01a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eef400640 1 -- 172.21.15.32:0/1913993740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef01a6810 con 0x7f7ef0075b70 2024-08-31T07:30:53.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.885+0000 7f7eef400640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0075b70 0x7f7ef01a08a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0005940 tx=0x7f7ee00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.886+0000 7f7ee7e00640 1 -- 172.21.15.32:0/1913993740 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee003f030 con 0x7f7ef0075b70 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.886+0000 7f7ee7e00640 1 -- 172.21.15.32:0/1913993740 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee00044c0 con 0x7f7ef0075b70 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.886+0000 7f7ef6400640 1 -- 172.21.15.32:0/1913993740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef01a6aa0 con 0x7f7ef0075b70 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.886+0000 7f7ee7e00640 1 -- 172.21.15.32:0/1913993740 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0035070 con 0x7f7ef0075b70 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.886+0000 7f7ef6400640 1 -- 172.21.15.32:0/1913993740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01ad260 con 0x7f7ef0075b70 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.887+0000 7f7ef6400640 1 -- 172.21.15.32:0/1913993740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef0075f80 con 0x7f7ef0075b70 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.890+0000 7f7ee7e00640 1 -- 172.21.15.32:0/1913993740 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7ee0042030 con 0x7f7ef0075b70 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.891+0000 7f7ee7e00640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ed40b4d20 0x7f7ed40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.891+0000 7f7ee7e00640 1 -- 172.21.15.32:0/1913993740 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f7ee0105a90 con 0x7f7ef0075b70 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.891+0000 7f7eeea00640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ed40b4d20 0x7f7ed40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.892+0000 7f7eeea00640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ed40b4d20 0x7f7ed40b7230 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0115790 tx=0x7f7edc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:51.893+0000 7f7ee7e00640 1 -- 172.21.15.32:0/1913993740 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7ee00cc870 con 0x7f7ef0075b70 2024-08-31T07:30:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.130+0000 7f7ef6400640 1 -- 172.21.15.32:0/1913993740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ef01a19d0 con 0x7f7ef0075b70 2024-08-31T07:30:53.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.131+0000 7f7ee7e00640 1 -- 172.21.15.32:0/1913993740 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+11963 (secure 0 0 0) 0x7f7ee00cc170 con 0x7f7ef0075b70 2024-08-31T07:30:53.574 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:30:53.574 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"btime":"2024-08-31T07:30:07:717453+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":67,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:30:07.599620+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:30:53.574 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 67 2024-08-31T07:30:53.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 -- 172.21.15.32:0/1913993740 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ed40b4d20 msgr2=0x7f7ed40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:53.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ed40b4d20 0x7f7ed40b7230 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0115790 tx=0x7f7edc005f90 comp rx=0 tx=0).stop 2024-08-31T07:30:53.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 -- 172.21.15.32:0/1913993740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0075b70 msgr2=0x7f7ef01a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:53.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0075b70 0x7f7ef01a08a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0005940 tx=0x7f7ee00040c0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 -- 172.21.15.32:0/1913993740 shutdown_connections 2024-08-31T07:30:53.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7ed40b4d20 0x7f7ed40b7230 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef01141d0 0x7f7ef01a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef0077120 0x7f7ef01a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 --2- 172.21.15.32:0/1913993740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0075b70 0x7f7ef01a08a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:53.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 -- 172.21.15.32:0/1913993740 >> 172.21.15.32:0/1913993740 conn(0x7f7ef007b5f0 msgr2=0x7f7ef0112f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:53.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 -- 172.21.15.32:0/1913993740 shutdown_connections 2024-08-31T07:30:53.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:52.133+0000 7f7ef6400640 1 -- 172.21.15.32:0/1913993740 wait complete. 2024-08-31T07:30:54.390 INFO:tasks.workunit.client.0.smithi032.stdout: 10 91277 14.44 MB/sec execute 154 sec latency 2985.768 ms 2024-08-31T07:30:54.542 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:54 smithi032 ceph-mon[17252]: pgmap v936: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 350 op/s 2024-08-31T07:30:54.623 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 151.7 --await-for 45.0' 2024-08-31T07:30:54.624 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 151.7 --await-for 45.0 2024-08-31T07:30:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:54 smithi073 ceph-mon[21995]: pgmap v936: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 350 op/s 2024-08-31T07:30:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:54 smithi084 ceph-mon[22387]: pgmap v936: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 350 op/s 2024-08-31T07:30:55.249 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:30:55.391 INFO:tasks.workunit.client.0.smithi032.stdout: 10 91867 14.51 MB/sec execute 155 sec latency 1555.188 ms 2024-08-31T07:30:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.178+0000 7f05a2a00640 1 -- 172.21.15.32:0/1094843294 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c032340 con 0x7f05a4104590 2024-08-31T07:30:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.179+0000 7f05a9c00640 1 -- 172.21.15.32:0/1094843294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a4104590 msgr2=0x7f05a41049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.179+0000 7f05a9c00640 1 --2- 172.21.15.32:0/1094843294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a4104590 0x7f05a41049a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f059c009ca0 tx=0x7f059c02e990 comp rx=0 tx=0).stop 2024-08-31T07:30:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.180+0000 7f05a9c00640 1 -- 172.21.15.32:0/1094843294 shutdown_connections 2024-08-31T07:30:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.180+0000 7f05a9c00640 1 --2- 172.21.15.32:0/1094843294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a410d9f0 0x7f05a410fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.180+0000 7f05a9c00640 1 --2- 172.21.15.32:0/1094843294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a4104ee0 0x7f05a410d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.180+0000 7f05a9c00640 1 --2- 172.21.15.32:0/1094843294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a4104590 0x7f05a41049a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.180+0000 7f05a9c00640 1 -- 172.21.15.32:0/1094843294 >> 172.21.15.32:0/1094843294 conn(0x7f05a40fe0d0 msgr2=0x7f05a4100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.180+0000 7f05a9c00640 1 -- 172.21.15.32:0/1094843294 shutdown_connections 2024-08-31T07:30:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.181+0000 7f05a9c00640 1 -- 172.21.15.32:0/1094843294 wait complete. 2024-08-31T07:30:56.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.182+0000 7f05a9c00640 1 Processor -- start 2024-08-31T07:30:56.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.182+0000 7f05a9c00640 1 -- start start 2024-08-31T07:30:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.183+0000 7f05a9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a4104ee0 0x7f05a41a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.183+0000 7f05a9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a410d9f0 0x7f05a41a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.183+0000 7f05a9c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a41a5f80 0x7f05a41a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.183+0000 7f05a9c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4119ca0 con 0x7f05a4104ee0 2024-08-31T07:30:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.183+0000 7f05a9c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4119b20 con 0x7f05a410d9f0 2024-08-31T07:30:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.183+0000 7f05a9c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4119e20 con 0x7f05a41a5f80 2024-08-31T07:30:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.183+0000 7f05a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a4104ee0 0x7f05a41a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.183+0000 7f05a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a4104ee0 0x7f05a41a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52214/0 (socket says 172.21.15.32:52214) 2024-08-31T07:30:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.183+0000 7f05a3e00640 1 -- 172.21.15.32:0/772854174 learned_addr learned my addr 172.21.15.32:0/772854174 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.183+0000 7f05a3400640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a410d9f0 0x7f05a41a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.184+0000 7f05a3e00640 1 -- 172.21.15.32:0/772854174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a41a5f80 msgr2=0x7f05a41a6390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:30:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.184+0000 7f05a3e00640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a41a5f80 0x7f05a41a6390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.184+0000 7f05a3e00640 1 -- 172.21.15.32:0/772854174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a410d9f0 msgr2=0x7f05a41a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.184+0000 7f05a3e00640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a410d9f0 0x7f05a41a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.184+0000 7f05a3e00640 1 -- 172.21.15.32:0/772854174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a41a6b60 con 0x7f05a4104ee0 2024-08-31T07:30:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.184+0000 7f05a3400640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a410d9f0 0x7f05a41a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.184+0000 7f05a3e00640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a4104ee0 0x7f05a41a0800 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f059c000f80 tx=0x7f059c0348c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.184+0000 7f05a0c00640 1 -- 172.21.15.32:0/772854174 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c041070 con 0x7f05a4104ee0 2024-08-31T07:30:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.185+0000 7f05a0c00640 1 -- 172.21.15.32:0/772854174 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059c033e80 con 0x7f05a4104ee0 2024-08-31T07:30:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.185+0000 7f05a0c00640 1 -- 172.21.15.32:0/772854174 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c038030 con 0x7f05a4104ee0 2024-08-31T07:30:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.186+0000 7f05a9c00640 1 -- 172.21.15.32:0/772854174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a41ad100 con 0x7f05a4104ee0 2024-08-31T07:30:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.186+0000 7f05a9c00640 1 -- 172.21.15.32:0/772854174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a41ad4a0 con 0x7f05a4104ee0 2024-08-31T07:30:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.190+0000 7f05b1e3a740 1 Processor -- start 2024-08-31T07:30:56.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.192+0000 7f05b1e3a740 1 -- start start 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb82a840 0x55a9bbbd12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05a0c00640 1 -- 172.21.15.32:0/772854174 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f059c044070 con 0x7f05a4104ee0 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbd1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb809b60 0x55a9bbbd1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05b1e3a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a9bbb46530 con 0x55a9bb809b60 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05b1e3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a9bbb35ac0 con 0x55a9bb801980 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05b1e3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a9bbb39a70 con 0x55a9bb82a840 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05a9200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb82a840 0x55a9bbbd12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05a3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbd1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05a9200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb82a840 0x55a9bbbd12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:53584/0 (socket says 172.21.15.32:53584) 2024-08-31T07:30:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.193+0000 7f05a9200640 1 -- 172.21.15.32:0/1570772566 learned_addr learned my addr 172.21.15.32:0/1570772566 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.194+0000 7f05a3e00640 1 -- 172.21.15.32:0/1570772566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb82a840 msgr2=0x55a9bbbd12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.194+0000 7f05a3e00640 1 --2- 172.21.15.32:0/1570772566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb82a840 0x55a9bbbd12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.194+0000 7f05a3e00640 1 -- 172.21.15.32:0/1570772566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb809b60 msgr2=0x55a9bbbd1d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:30:56.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.195+0000 7f05a3400640 1 --2- 172.21.15.32:0/1570772566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb809b60 0x55a9bbbd1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.195+0000 7f05a0c00640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05880b4d10 0x7f05880b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.196+0000 7f05a0c00640 1 -- 172.21.15.32:0/772854174 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f059c106070 con 0x7f05a4104ee0 2024-08-31T07:30:56.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.196+0000 7f05a9200640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05880b4d10 0x7f05880b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.197+0000 7f05a3e00640 1 --2- 172.21.15.32:0/1570772566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb809b60 0x55a9bbbd1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.197+0000 7f05a3e00640 1 -- 172.21.15.32:0/1570772566 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9bbae8880 con 0x55a9bb801980 2024-08-31T07:30:56.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.197+0000 7f05a3400640 1 --2- 172.21.15.32:0/1570772566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb809b60 0x55a9bbbd1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:56.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.197+0000 7f05a9200640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05880b4d10 0x7f05880b7220 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0594009ba0 tx=0x7f0594005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:56.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.198+0000 7f05a3e00640 1 --2- 172.21.15.32:0/1570772566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbd1810 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f059c1037b0 tx=0x7f059c02e990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7de34e25b5528f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.198+0000 7f0599600640 1 -- 172.21.15.32:0/1570772566 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c04b030 con 0x55a9bb801980 2024-08-31T07:30:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.198+0000 7f0599600640 1 -- 172.21.15.32:0/1570772566 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059c0d53c0 con 0x55a9bb801980 2024-08-31T07:30:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.198+0000 7f0599600640 1 -- 172.21.15.32:0/1570772566 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0cc260 con 0x55a9bb801980 2024-08-31T07:30:56.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.199+0000 7f05b1e3a740 1 -- 172.21.15.32:0/1570772566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 msgr2=0x55a9bbbd1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:56.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.199+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/1570772566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbd1810 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f059c1037b0 tx=0x7f059c02e990 comp rx=0 tx=0).stop 2024-08-31T07:30:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.200+0000 7f05b1e3a740 1 -- 172.21.15.32:0/1570772566 shutdown_connections 2024-08-31T07:30:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.200+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/1570772566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb809b60 0x55a9bbbd1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.200+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/1570772566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbd1810 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.200+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/1570772566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb82a840 0x55a9bbbd12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.200+0000 7f05b1e3a740 1 -- 172.21.15.32:0/1570772566 >> 172.21.15.32:0/1570772566 conn(0x55a9bb81e990 msgr2=0x55a9bbbcf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.200+0000 7f05b1e3a740 1 -- 172.21.15.32:0/1570772566 shutdown_connections 2024-08-31T07:30:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.200+0000 7f05b1e3a740 1 -- 172.21.15.32:0/1570772566 wait complete. 2024-08-31T07:30:56.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.201+0000 7f05b1e3a740 1 -- start start 2024-08-31T07:30:56.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.201+0000 7f05b1e3a740 1 Processor -- start 2024-08-31T07:30:56.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.202+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbdce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.202+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.203+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb82a840 0x55a9bbbe89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.203+0000 7f05a3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbdce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.203+0000 7f05a3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbdce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59994/0 (socket says 172.21.15.32:59994) 2024-08-31T07:30:56.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.203+0000 7f05a3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.203+0000 7f05a3400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:53600/0 (socket says 172.21.15.32:53600) 2024-08-31T07:30:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.203+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a9bb9987a0 con 0x55a9bb82a840 2024-08-31T07:30:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.203+0000 7f05a3e00640 1 -- 172.21.15.32:0/2712211444 learned_addr learned my addr 172.21.15.32:0/2712211444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.203+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a9bbabc7f0 con 0x55a9bb801980 2024-08-31T07:30:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.203+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a9bbb35ac0 con 0x55a9bb809b60 2024-08-31T07:30:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.203+0000 7f05b1e3a740 10 client.0 _ll_register_callbacks cb 0x55a9bbae1e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.204+0000 7f05a3400640 1 -- 172.21.15.32:0/2712211444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 msgr2=0x55a9bbbdce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.204+0000 7f05a3400640 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbdce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.204+0000 7f05a3400640 1 -- 172.21.15.32:0/2712211444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb82a840 msgr2=0x55a9bbbe89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.204+0000 7f05a3400640 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb82a840 0x55a9bbbe89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.204+0000 7f05a3400640 1 -- 172.21.15.32:0/2712211444 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9bb8a4b90 con 0x55a9bb809b60 2024-08-31T07:30:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.204+0000 7f05a3e00640 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbdce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:30:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.205+0000 7f05a3400640 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdd380 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0590005d00 tx=0x7f0590011b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d513c036b3151ff4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.205+0000 7f058e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:30:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.205+0000 7f058e000640 1 -- 172.21.15.32:0/2712211444 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590018070 con 0x55a9bb809b60 2024-08-31T07:30:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.206+0000 7f058e000640 1 -- 172.21.15.32:0/2712211444 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f059000fb10 con 0x55a9bb809b60 2024-08-31T07:30:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.206+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f059c0d53c0 con 0x55a9bb809b60 2024-08-31T07:30:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.206+0000 7f058e000640 1 -- 172.21.15.32:0/2712211444 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f059000fd50 con 0x55a9bb809b60 2024-08-31T07:30:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.206+0000 7f05b1e3a740 10 client.29071 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.206+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55a9bbbeb640 con 0x55a9bb809b60 2024-08-31T07:30:56.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.207+0000 7f058e000640 1 -- 172.21.15.32:0/2712211444 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059c0d53c0 con 0x55a9bb809b60 2024-08-31T07:30:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.207+0000 7f058e000640 1 -- 172.21.15.32:0/2712211444 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f059001c460 con 0x55a9bb809b60 2024-08-31T07:30:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.208+0000 7f05b1e3a740 10 client.29071 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.208+0000 7f05b1e3a740 10 client.29071 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:30:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.208+0000 7f05b1e3a740 20 client.29071 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.208+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a9bb7e91c0 0x55a9bbbeec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.208+0000 7f05b1e3a740 4 client.29071 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:30:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.208+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9bbb48480 con 0x55a9bb7e91c0 2024-08-31T07:30:56.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.208+0000 7f05a3e00640 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a9bb7e91c0 0x55a9bbbeec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.210+0000 7f05a3e00640 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a9bb7e91c0 0x55a9bbbeec80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbcddf02c2a41fc3 server_cookie=d5eae2f6880d2aab in_seq=0 out_seq=0 2024-08-31T07:30:56.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.210+0000 7f058e000640 10 client.29071 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.211+0000 7f058e000640 1 -- 172.21.15.32:0/2712211444 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9bbb48480 con 0x55a9bb7e91c0 2024-08-31T07:30:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.211+0000 7f058e000640 10 client.29071 handle_command_reply: tid=0 2024-08-31T07:30:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.211+0000 7f05b1e3a740 1 client.29071 shutdown 2024-08-31T07:30:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.211+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 msgr2=0x55a9bbbdd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.212+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdd380 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0590005d00 tx=0x7f0590011b40 comp rx=0 tx=0).stop 2024-08-31T07:30:56.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.212+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 shutdown_connections 2024-08-31T07:30:56.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.212+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a9bb7e91c0 0x55a9bbbeec80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.212+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb82a840 0x55a9bbbe89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.212+0000 7f058e000640 0 client.29071 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.212+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdd380 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.212+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/2712211444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbdce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.213+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 >> 172.21.15.32:0/2712211444 conn(0x55a9bb81e990 msgr2=0x55a9bbbd4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.213+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 shutdown_connections 2024-08-31T07:30:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.213+0000 7f05b1e3a740 1 -- 172.21.15.32:0/2712211444 wait complete. 2024-08-31T07:30:56.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.213+0000 7f05b1e3a740 20 client.29071 trim_cache size 0 max 16384 2024-08-31T07:30:56.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.246+0000 7f05b1e3a740 1 Processor -- start 2024-08-31T07:30:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.246+0000 7f05b1e3a740 1 -- start start 2024-08-31T07:30:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.246+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb801980 0x55a9bbbcefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.246+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.246+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb82a840 0x55a9bbbe88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.246+0000 7f05b1e3a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41b5dc0 con 0x55a9bb801980 2024-08-31T07:30:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.246+0000 7f05b1e3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41b7960 con 0x55a9bb82a840 2024-08-31T07:30:56.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.246+0000 7f05b1e3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41b64c0 con 0x55a9bb809b60 2024-08-31T07:30:56.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.247+0000 7f05a9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb82a840 0x55a9bbbe88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.247+0000 7f05a9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb82a840 0x55a9bbbe88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59998/0 (socket says 172.21.15.32:59998) 2024-08-31T07:30:56.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.247+0000 7f05a9200640 1 -- 172.21.15.32:0/3207193128 learned_addr learned my addr 172.21.15.32:0/3207193128 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:56.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.247+0000 7f05a3e00640 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb801980 0x55a9bbbcefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.248+0000 7f05a9200640 1 -- 172.21.15.32:0/3207193128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 msgr2=0x55a9bbbdeb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:30:56.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.248+0000 7f05a3400640 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.248+0000 7f05a9200640 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.248+0000 7f05a9200640 1 -- 172.21.15.32:0/3207193128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb801980 msgr2=0x55a9bbbcefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:56.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.248+0000 7f05a9200640 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb801980 0x55a9bbbcefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.248+0000 7f05a9200640 1 -- 172.21.15.32:0/3207193128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9bbbef010 con 0x55a9bb82a840 2024-08-31T07:30:56.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.248+0000 7f05a3e00640 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb801980 0x55a9bbbcefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:30:56.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.248+0000 7f05a3400640 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:30:56.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.248+0000 7f05a9200640 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb82a840 0x55a9bbbe88e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0594011e20 tx=0x7f0594014070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a2cc61cda536e11 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:56.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.249+0000 7f0598c00640 1 -- 172.21.15.32:0/3207193128 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059401f420 con 0x55a9bb82a840 2024-08-31T07:30:56.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.249+0000 7f0598c00640 1 -- 172.21.15.32:0/3207193128 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059401fa70 con 0x55a9bb82a840 2024-08-31T07:30:56.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.249+0000 7f0598c00640 1 -- 172.21.15.32:0/3207193128 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594027ae0 con 0x55a9bb82a840 2024-08-31T07:30:56.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.250+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3207193128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb82a840 msgr2=0x55a9bbbe88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:56.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.250+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb82a840 0x55a9bbbe88e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0594011e20 tx=0x7f0594014070 comp rx=0 tx=0).stop 2024-08-31T07:30:56.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.250+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3207193128 shutdown_connections 2024-08-31T07:30:56.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.251+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb82a840 0x55a9bbbe88e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.251+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbdeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.251+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/3207193128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb801980 0x55a9bbbcefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.251+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3207193128 >> 172.21.15.32:0/3207193128 conn(0x55a9bb81e990 msgr2=0x55a9bbbcde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:56.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.251+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3207193128 shutdown_connections 2024-08-31T07:30:56.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.251+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3207193128 wait complete. 2024-08-31T07:30:56.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.252+0000 7f05b1e3a740 1 -- start start 2024-08-31T07:30:56.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.253+0000 7f05b1e3a740 1 Processor -- start 2024-08-31T07:30:56.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.254+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbe1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.255+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbe17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.255+0000 7f05b1e3a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb82a840 0x55a9bbc52610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.255+0000 7f05b1e3a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41b7290 con 0x55a9bb82a840 2024-08-31T07:30:56.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.255+0000 7f05b1e3a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41b6bc0 con 0x55a9bb801980 2024-08-31T07:30:56.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.255+0000 7f05b1e3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41b3dc0 con 0x55a9bb809b60 2024-08-31T07:30:56.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.256+0000 7f05a9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb82a840 0x55a9bbc52610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.256+0000 7f05a9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb82a840 0x55a9bbc52610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52260/0 (socket says 172.21.15.32:52260) 2024-08-31T07:30:56.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.256+0000 7f05a9200640 1 -- 172.21.15.32:0/3199872204 learned_addr learned my addr 172.21.15.32:0/3199872204 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:30:56.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.256+0000 7f05a3e00640 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbe1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.256+0000 7f05a3400640 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbe17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.257+0000 7f05a3e00640 1 -- 172.21.15.32:0/3199872204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 msgr2=0x55a9bbbe17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:56.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.257+0000 7f05a3e00640 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbe17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.257+0000 7f05a3e00640 1 -- 172.21.15.32:0/3199872204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb82a840 msgr2=0x55a9bbc52610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:30:56.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.257+0000 7f05a3e00640 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb82a840 0x55a9bbc52610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.257+0000 7f05b1e3a740 10 client.0 _ll_register_callbacks cb 0x7f059000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:30:56.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.257+0000 7f05a3e00640 1 -- 172.21.15.32:0/3199872204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9bbc52c80 con 0x55a9bb801980 2024-08-31T07:30:56.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.257+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3199872204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f059401fa70 con 0x55a9bb801980 2024-08-31T07:30:56.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.257+0000 7f05a3e00640 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbe1230 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f059c103780 tx=0x7f059c0c8730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2663c56526da68df server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:30:56.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.258+0000 7f058e000640 10 client.29084 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:30:56.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.258+0000 7f058e000640 1 -- 172.21.15.32:0/3199872204 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c04b040 con 0x55a9bb801980 2024-08-31T07:30:56.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.258+0000 7f058e000640 1 -- 172.21.15.32:0/3199872204 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8441+0+0 (secure 0 0 0) 0x7f059c0333e0 con 0x55a9bb801980 2024-08-31T07:30:56.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.258+0000 7f058e000640 1 -- 172.21.15.32:0/3199872204 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f059c0cbec0 con 0x55a9bb801980 2024-08-31T07:30:56.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.258+0000 7f05b1e3a740 10 client.29084 fetch_fsmap learned FSMap version 67 2024-08-31T07:30:56.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.258+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3199872204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=67}) -- 0x55a9bbc59dd0 con 0x55a9bb801980 2024-08-31T07:30:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.259+0000 7f058e000640 1 -- 172.21.15.32:0/3199872204 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f059c04e060 con 0x55a9bb801980 2024-08-31T07:30:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.259+0000 7f05b1e3a740 10 client.29084 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:30:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.259+0000 7f05b1e3a740 10 client.29084 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:30:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.259+0000 7f05b1e3a740 20 client.29084 populate_metadata read hostname 'smithi032' 2024-08-31T07:30:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.259+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a9bb7be570 0x55a9bbc5c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:30:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.260+0000 7f05b1e3a740 4 client.29084 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 151.7, "await_for": 45.0}] 2024-08-31T07:30:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.260+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3199872204 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 151.7, "await_for": 45.0}) -- 0x55a9bbb43f20 con 0x55a9bb7be570 2024-08-31T07:30:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.260+0000 7f05a3400640 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a9bb7be570 0x55a9bbc5c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:30:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.261+0000 7f05a3400640 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a9bb7be570 0x55a9bbc5c240 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ad62a3059b4b7a server_cookie=d8070215881b08cb in_seq=0 out_seq=0 2024-08-31T07:30:56.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.261+0000 7f058e000640 10 client.29084 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:56.391 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92537 14.67 MB/sec execute 156 sec latency 2555.251 ms 2024-08-31T07:30:56.507 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:56 smithi032 ceph-mon[17252]: pgmap v937: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.9 MiB/s wr, 417 op/s 2024-08-31T07:30:56.507 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:30:56.507 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:30:56.507 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:30:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:56 smithi073 ceph-mon[21995]: pgmap v937: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.9 MiB/s wr, 417 op/s 2024-08-31T07:30:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:30:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:30:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:30:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f058e000640 1 -- 172.21.15.32:0/3199872204 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a9bbb43f20 con 0x55a9bb7be570 2024-08-31T07:30:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f058e000640 10 client.29084 handle_command_reply: tid=0 2024-08-31T07:30:56.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 client.29084 shutdown 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3199872204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 msgr2=0x55a9bbbe1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbe1230 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f059c103780 tx=0x7f059c0c8730 comp rx=0 tx=0).stop 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3199872204 shutdown_connections 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a9bb7be570 0x55a9bbc5c240 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a9bb82a840 0x55a9bbc52610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a9bb809b60 0x55a9bbbe17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 --2- 172.21.15.32:0/3199872204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9bb801980 0x55a9bbbe1230 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f058e000640 0 client.29084 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3199872204 >> 172.21.15.32:0/3199872204 conn(0x55a9bb81e990 msgr2=0x55a9bbbd2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3199872204 shutdown_connections 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.691+0000 7f05b1e3a740 1 -- 172.21.15.32:0/3199872204 wait complete. 2024-08-31T07:30:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.692+0000 7f05b1e3a740 20 client.29084 trim_cache size 0 max 16384 2024-08-31T07:30:56.694 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:30:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 67, 2024-08-31T07:30:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:30:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 58, 2024-08-31T07:30:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:30:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "fd6c69c8": { 2024-08-31T07:30:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 58, 2024-08-31T07:30:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:30:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:30:56.694 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 151.7, 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:30:56.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:30:56.696 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:30:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.692+0000 7f05a9c00640 1 -- 172.21.15.32:0/772854174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05880b4d10 msgr2=0x7f05880b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.692+0000 7f05a9c00640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05880b4d10 0x7f05880b7220 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0594009ba0 tx=0x7f0594005f90 comp rx=0 tx=0).stop 2024-08-31T07:30:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.693+0000 7f05a9c00640 1 -- 172.21.15.32:0/772854174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a4104ee0 msgr2=0x7f05a41a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:30:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.693+0000 7f05a9c00640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a4104ee0 0x7f05a41a0800 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f059c000f80 tx=0x7f059c0348c0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.693+0000 7f05a9c00640 1 -- 172.21.15.32:0/772854174 shutdown_connections 2024-08-31T07:30:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.693+0000 7f05a9c00640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05880b4d10 0x7f05880b7220 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.693+0000 7f05a9c00640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a41a5f80 0x7f05a41a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.693+0000 7f05a9c00640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a410d9f0 0x7f05a41a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.693+0000 7f05a9c00640 1 --2- 172.21.15.32:0/772854174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a4104ee0 0x7f05a41a0800 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:30:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.693+0000 7f05a9c00640 1 -- 172.21.15.32:0/772854174 >> 172.21.15.32:0/772854174 conn(0x7f05a40fe0d0 msgr2=0x7f05a40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:30:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.693+0000 7f05a9c00640 1 -- 172.21.15.32:0/772854174 shutdown_connections 2024-08-31T07:30:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:30:56.693+0000 7f05a9c00640 1 -- 172.21.15.32:0/772854174 wait complete. 2024-08-31T07:30:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:56 smithi084 ceph-mon[22387]: pgmap v937: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.9 MiB/s wr, 417 op/s 2024-08-31T07:30:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:30:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:30:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:30:57.391 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 14.57 MB/sec execute 157 sec latency 2743.425 ms 2024-08-31T07:30:57.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:57 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:30:57.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:57 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:30:57.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:57 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:30:57.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:57 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:30:57.708 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fd6c69c8', quiesce took 7.2 seconds. Will release after: 24.5 2024-08-31T07:30:57.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:57 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:30:57.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:57 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:30:58.391 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 14.48 MB/sec execute 158 sec latency 2120.461 ms 2024-08-31T07:30:58.580 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:30:58 smithi073 ceph-mon[21995]: pgmap v938: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.4 MiB/s wr, 542 op/s 2024-08-31T07:30:58.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:30:58 smithi032 ceph-mon[17252]: pgmap v938: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.4 MiB/s wr, 542 op/s 2024-08-31T07:30:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:30:58 smithi084 ceph-mon[22387]: pgmap v938: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.4 MiB/s wr, 542 op/s 2024-08-31T07:30:59.392 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 14.39 MB/sec execute 159 sec latency 3121.358 ms 2024-08-31T07:31:00.392 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 14.30 MB/sec execute 160 sec latency 4121.423 ms 2024-08-31T07:31:00.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:00 smithi032 ceph-mon[17252]: pgmap v939: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.5 MiB/s wr, 536 op/s 2024-08-31T07:31:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:00 smithi073 ceph-mon[21995]: pgmap v939: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.5 MiB/s wr, 536 op/s 2024-08-31T07:31:00.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:00 smithi084 ceph-mon[22387]: pgmap v939: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.5 MiB/s wr, 536 op/s 2024-08-31T07:31:01.392 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 14.21 MB/sec execute 161 sec latency 5121.469 ms 2024-08-31T07:31:02.392 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 14.12 MB/sec execute 162 sec latency 6121.512 ms 2024-08-31T07:31:02.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:02 smithi032 ceph-mon[17252]: pgmap v940: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.4 MiB/s wr, 539 op/s 2024-08-31T07:31:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:02 smithi073 ceph-mon[21995]: pgmap v940: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.4 MiB/s wr, 539 op/s 2024-08-31T07:31:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:02 smithi084 ceph-mon[22387]: pgmap v940: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.4 MiB/s wr, 539 op/s 2024-08-31T07:31:03.392 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 14.04 MB/sec execute 163 sec latency 7121.561 ms 2024-08-31T07:31:04.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.95 MB/sec execute 164 sec latency 8121.629 ms 2024-08-31T07:31:04.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:04 smithi032 ceph-mon[17252]: pgmap v941: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.4 MiB/s wr, 402 op/s 2024-08-31T07:31:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:04 smithi073 ceph-mon[21995]: pgmap v941: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.4 MiB/s wr, 402 op/s 2024-08-31T07:31:04.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:04 smithi084 ceph-mon[22387]: pgmap v941: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.4 MiB/s wr, 402 op/s 2024-08-31T07:31:05.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.87 MB/sec execute 165 sec latency 9121.677 ms 2024-08-31T07:31:06.392 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.78 MB/sec execute 166 sec latency 10121.737 ms 2024-08-31T07:31:06.529 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:06 smithi032 ceph-mon[17252]: pgmap v942: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 366 op/s 2024-08-31T07:31:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:06 smithi073 ceph-mon[21995]: pgmap v942: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 366 op/s 2024-08-31T07:31:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:06 smithi084 ceph-mon[22387]: pgmap v942: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 366 op/s 2024-08-31T07:31:07.377 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:07 smithi032 ceph-mon[17252]: osdmap e105: 12 total, 12 up, 12 in 2024-08-31T07:31:07.392 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.70 MB/sec execute 167 sec latency 11121.800 ms 2024-08-31T07:31:07.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:07 smithi073 ceph-mon[21995]: osdmap e105: 12 total, 12 up, 12 in 2024-08-31T07:31:07.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:07 smithi084 ceph-mon[22387]: osdmap e105: 12 total, 12 up, 12 in 2024-08-31T07:31:08.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.62 MB/sec execute 168 sec latency 12121.862 ms 2024-08-31T07:31:08.532 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:08 smithi032 ceph-mon[17252]: pgmap v944: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 287 op/s 2024-08-31T07:31:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:08 smithi073 ceph-mon[21995]: pgmap v944: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 287 op/s 2024-08-31T07:31:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:08 smithi084 ceph-mon[22387]: pgmap v944: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 287 op/s 2024-08-31T07:31:09.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.54 MB/sec execute 169 sec latency 13121.929 ms 2024-08-31T07:31:10.381 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:10 smithi032 ceph-mon[17252]: pgmap v945: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 259 op/s 2024-08-31T07:31:10.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.46 MB/sec execute 170 sec latency 14121.974 ms 2024-08-31T07:31:10.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:10 smithi073 ceph-mon[21995]: pgmap v945: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 259 op/s 2024-08-31T07:31:10.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:10 smithi084 ceph-mon[22387]: pgmap v945: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 259 op/s 2024-08-31T07:31:11.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.38 MB/sec execute 171 sec latency 15122.036 ms 2024-08-31T07:31:12.381 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:12 smithi032 ceph-mon[17252]: pgmap v946: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 6.4 MiB/s wr, 242 op/s 2024-08-31T07:31:12.381 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:12 smithi032 ceph-mon[17252]: osdmap e106: 12 total, 12 up, 12 in 2024-08-31T07:31:12.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.30 MB/sec execute 172 sec latency 16122.098 ms 2024-08-31T07:31:12.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:12 smithi073 ceph-mon[21995]: pgmap v946: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 6.4 MiB/s wr, 242 op/s 2024-08-31T07:31:12.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:12 smithi073 ceph-mon[21995]: osdmap e106: 12 total, 12 up, 12 in 2024-08-31T07:31:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:12 smithi084 ceph-mon[22387]: pgmap v946: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 6.4 MiB/s wr, 242 op/s 2024-08-31T07:31:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:12 smithi084 ceph-mon[22387]: osdmap e106: 12 total, 12 up, 12 in 2024-08-31T07:31:13.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.23 MB/sec execute 173 sec latency 17122.161 ms 2024-08-31T07:31:14.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.15 MB/sec execute 174 sec latency 18122.277 ms 2024-08-31T07:31:14.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:14 smithi032 ceph-mon[17252]: pgmap v948: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 6.1 MiB/s wr, 228 op/s 2024-08-31T07:31:14.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:14 smithi073 ceph-mon[21995]: pgmap v948: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 6.1 MiB/s wr, 228 op/s 2024-08-31T07:31:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:14 smithi084 ceph-mon[22387]: pgmap v948: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 6.1 MiB/s wr, 228 op/s 2024-08-31T07:31:15.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:31:15.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.08 MB/sec execute 175 sec latency 19122.365 ms 2024-08-31T07:31:15.598 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.286+0000 7fd5fba00640 1 -- 172.21.15.32:0/2084481290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f4109240 msgr2=0x7fd5f4109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.286+0000 7fd5fba00640 1 --2- 172.21.15.32:0/2084481290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f4109240 0x7fd5f4109650 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009900 tx=0x7fd5e002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.287+0000 7fd5fba00640 1 -- 172.21.15.32:0/2084481290 shutdown_connections 2024-08-31T07:31:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.287+0000 7fd5fba00640 1 --2- 172.21.15.32:0/2084481290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f4109b90 0x7fd5f41103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.287+0000 7fd5fba00640 1 --2- 172.21.15.32:0/2084481290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f4109240 0x7fd5f4109650 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.287+0000 7fd5fba00640 1 --2- 172.21.15.32:0/2084481290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f4103900 0x7fd5f4103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.287+0000 7fd5fba00640 1 -- 172.21.15.32:0/2084481290 >> 172.21.15.32:0/2084481290 conn(0x7fd5f407b5f0 msgr2=0x7fd5f407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:16.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.287+0000 7fd5fba00640 1 -- 172.21.15.32:0/2084481290 shutdown_connections 2024-08-31T07:31:16.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.287+0000 7fd5fba00640 1 -- 172.21.15.32:0/2084481290 wait complete. 2024-08-31T07:31:16.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.288+0000 7fd5fba00640 1 Processor -- start 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.288+0000 7fd5fba00640 1 -- start start 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.288+0000 7fd5fba00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f4103900 0x7fd5f40754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.288+0000 7fd5fba00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f4109240 0x7fd5f40759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.288+0000 7fd5fba00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f4109b90 0x7fd5f4075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.288+0000 7fd5fba00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f4113850 con 0x7fd5f4109b90 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.288+0000 7fd5fba00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f41136d0 con 0x7fd5f4103900 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.288+0000 7fd5fba00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f41139d0 con 0x7fd5f4109240 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.289+0000 7fd5f9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f4103900 0x7fd5f40754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.289+0000 7fd5f9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f4109b90 0x7fd5f4075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.289+0000 7fd5f9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f4103900 0x7fd5f40754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47264/0 (socket says 172.21.15.32:47264) 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.289+0000 7fd5f9000640 1 -- 172.21.15.32:0/3916754183 learned_addr learned my addr 172.21.15.32:0/3916754183 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.289+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f4109240 0x7fd5f40759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.289+0000 7fd5f9a00640 1 -- 172.21.15.32:0/3916754183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f4109240 msgr2=0x7fd5f40759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.289+0000 7fd5f9a00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f4109240 0x7fd5f40759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.289+0000 7fd5f9a00640 1 -- 172.21.15.32:0/3916754183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f4103900 msgr2=0x7fd5f40754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.289+0000 7fd5f9a00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f4103900 0x7fd5f40754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.289+0000 7fd5f9a00640 1 -- 172.21.15.32:0/3916754183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f41a8520 con 0x7fd5f4109b90 2024-08-31T07:31:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.290+0000 7fd5f9a00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f4109b90 0x7fd5f4075f30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d6b0 tx=0x7fd5ec00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.290+0000 7fd5f1600640 1 -- 172.21.15.32:0/3916754183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec017070 con 0x7fd5f4109b90 2024-08-31T07:31:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.290+0000 7fd5f1600640 1 -- 172.21.15.32:0/3916754183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ec009cd0 con 0x7fd5f4109b90 2024-08-31T07:31:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.290+0000 7fd5fba00640 1 -- 172.21.15.32:0/3916754183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f41a8810 con 0x7fd5f4109b90 2024-08-31T07:31:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.290+0000 7fd5f1600640 1 -- 172.21.15.32:0/3916754183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec003cc0 con 0x7fd5f4109b90 2024-08-31T07:31:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.290+0000 7fd5fba00640 1 -- 172.21.15.32:0/3916754183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f41a8cd0 con 0x7fd5f4109b90 2024-08-31T07:31:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.293+0000 7fd6025d4740 1 Processor -- start 2024-08-31T07:31:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.293+0000 7fd5f1600640 1 -- 172.21.15.32:0/3916754183 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd5ec009820 con 0x7fd5f4109b90 2024-08-31T07:31:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.294+0000 7fd6025d4740 1 -- start start 2024-08-31T07:31:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.294+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad736950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.294+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 0x55f6ad736e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.294+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad7373d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.294+0000 7fd6025d4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f6ad6ad770 con 0x55f6ad391070 2024-08-31T07:31:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.294+0000 7fd6025d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f6ad69cd00 con 0x55f6ad370390 2024-08-31T07:31:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.294+0000 7fd6025d4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f6ad6a0cb0 con 0x55f6ad3681b0 2024-08-31T07:31:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.294+0000 7fd5f3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad736950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.294+0000 7fd5f3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad736950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53198/0 (socket says 172.21.15.32:53198) 2024-08-31T07:31:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.294+0000 7fd5f3e00640 1 -- 172.21.15.32:0/459291942 learned_addr learned my addr 172.21.15.32:0/459291942 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.295+0000 7fd5f9000640 1 --2- 172.21.15.32:0/459291942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 0x55f6ad736e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.295+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/459291942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad7373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.295+0000 7fd5f9000640 1 -- 172.21.15.32:0/459291942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 msgr2=0x55f6ad7373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.295+0000 7fd5f9000640 1 --2- 172.21.15.32:0/459291942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad7373d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.295+0000 7fd5f9000640 1 -- 172.21.15.32:0/459291942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 msgr2=0x55f6ad736950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.295+0000 7fd5f9000640 1 --2- 172.21.15.32:0/459291942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad736950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.295+0000 7fd5f9000640 1 -- 172.21.15.32:0/459291942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6ad3fc650 con 0x55f6ad3681b0 2024-08-31T07:31:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.295+0000 7fd5f9000640 1 --2- 172.21.15.32:0/459291942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 0x55f6ad736e90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80104a0 tx=0x7fd5e8010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dff542d9be1e573b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.295+0000 7fd5f1600640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5dc0b4ca0 0x7fd5dc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.296+0000 7fd5f1600640 1 -- 172.21.15.32:0/3916754183 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fd5ec0e4d30 con 0x7fd5f4109b90 2024-08-31T07:31:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.296+0000 7fd5e6000640 1 -- 172.21.15.32:0/459291942 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8011700 con 0x55f6ad3681b0 2024-08-31T07:31:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.296+0000 7fd5e6000640 1 -- 172.21.15.32:0/459291942 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e8011d50 con 0x55f6ad3681b0 2024-08-31T07:31:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.296+0000 7fd5e6000640 1 -- 172.21.15.32:0/459291942 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800ede0 con 0x55f6ad3681b0 2024-08-31T07:31:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.296+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5dc0b4ca0 0x7fd5dc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd6025d4740 1 -- 172.21.15.32:0/459291942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 msgr2=0x55f6ad736e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd6025d4740 1 --2- 172.21.15.32:0/459291942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 0x55f6ad736e90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80104a0 tx=0x7fd5e8010960 comp rx=0 tx=0).stop 2024-08-31T07:31:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5dc0b4ca0 0x7fd5dc0b71b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00096c0 tx=0x7fd5e0041040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd6025d4740 1 -- 172.21.15.32:0/459291942 shutdown_connections 2024-08-31T07:31:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd6025d4740 1 --2- 172.21.15.32:0/459291942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad7373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd6025d4740 1 --2- 172.21.15.32:0/459291942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 0x55f6ad736e90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd6025d4740 1 --2- 172.21.15.32:0/459291942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad736950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd6025d4740 1 -- 172.21.15.32:0/459291942 >> 172.21.15.32:0/459291942 conn(0x55f6ad385460 msgr2=0x55f6ad734840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd6025d4740 1 -- 172.21.15.32:0/459291942 shutdown_connections 2024-08-31T07:31:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd6025d4740 1 -- 172.21.15.32:0/459291942 wait complete. 2024-08-31T07:31:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.297+0000 7fd6025d4740 1 -- start start 2024-08-31T07:31:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.298+0000 7fd6025d4740 1 Processor -- start 2024-08-31T07:31:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.298+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad3681b0 0x55f6ad742480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.298+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad370390 0x55f6ad7429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.298+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad74e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.298+0000 7fd6025d4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f6ad4ff700 con 0x55f6ad391070 2024-08-31T07:31:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.298+0000 7fd6025d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f6ad623cb0 con 0x55f6ad3681b0 2024-08-31T07:31:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.298+0000 7fd6025d4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f6ad745130 con 0x55f6ad370390 2024-08-31T07:31:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.298+0000 7fd6025d4740 10 client.0 _ll_register_callbacks cb 0x55f6ad649430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad3681b0 0x55f6ad742480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad3681b0 0x55f6ad742480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47278/0 (socket says 172.21.15.32:47278) 2024-08-31T07:31:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f9000640 1 -- 172.21.15.32:0/2133789852 learned_addr learned my addr 172.21.15.32:0/2133789852 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f9a00640 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad74e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad370390 0x55f6ad7429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f9000640 1 -- 172.21.15.32:0/2133789852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad370390 msgr2=0x55f6ad7429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f9000640 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad370390 0x55f6ad7429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f9000640 1 -- 172.21.15.32:0/2133789852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 msgr2=0x55f6ad74e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f9000640 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad74e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f9000640 1 -- 172.21.15.32:0/2133789852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6ad664870 con 0x55f6ad3681b0 2024-08-31T07:31:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5f9000640 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad3681b0 0x55f6ad742480 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8000f80 tx=0x7fd5e800e870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d1c92046918912 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.299+0000 7fd5daa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:31:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.300+0000 7fd5daa00640 1 -- 172.21.15.32:0/2133789852 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8027020 con 0x55f6ad3681b0 2024-08-31T07:31:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.300+0000 7fd6025d4740 1 -- 172.21.15.32:0/2133789852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e8011d50 con 0x55f6ad3681b0 2024-08-31T07:31:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.300+0000 7fd5daa00640 1 -- 172.21.15.32:0/2133789852 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fd5e800e570 con 0x55f6ad3681b0 2024-08-31T07:31:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.300+0000 7fd5daa00640 1 -- 172.21.15.32:0/2133789852 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd5e8032050 con 0x55f6ad3681b0 2024-08-31T07:31:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.300+0000 7fd6025d4740 10 client.29096 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.300+0000 7fd6025d4740 1 -- 172.21.15.32:0/2133789852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55f6ad7510c0 con 0x55f6ad3681b0 2024-08-31T07:31:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.300+0000 7fd5daa00640 1 -- 172.21.15.32:0/2133789852 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e8011d50 con 0x55f6ad3681b0 2024-08-31T07:31:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.301+0000 7fd5daa00640 1 -- 172.21.15.32:0/2133789852 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fd5e8011760 con 0x55f6ad3681b0 2024-08-31T07:31:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.301+0000 7fd6025d4740 10 client.29096 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.301+0000 7fd6025d4740 10 client.29096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:31:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.301+0000 7fd6025d4740 20 client.29096 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.301+0000 7fd6025d4740 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6ad34f9f0 0x55f6ad754480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.301+0000 7fd6025d4740 4 client.29096 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:31:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.301+0000 7fd6025d4740 1 -- 172.21.15.32:0/2133789852 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6ad6af6c0 con 0x55f6ad34f9f0 2024-08-31T07:31:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.301+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6ad34f9f0 0x55f6ad754480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.302+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6ad34f9f0 0x55f6ad754480 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d87dbf4cf0d629 server_cookie=9a3a247355780da1 in_seq=0 out_seq=0 2024-08-31T07:31:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.302+0000 7fd5daa00640 10 client.29096 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd5daa00640 1 -- 172.21.15.32:0/2133789852 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f6ad6af6c0 con 0x55f6ad34f9f0 2024-08-31T07:31:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd5daa00640 10 client.29096 handle_command_reply: tid=0 2024-08-31T07:31:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd6025d4740 1 client.29096 shutdown 2024-08-31T07:31:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd6025d4740 1 -- 172.21.15.32:0/2133789852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad3681b0 msgr2=0x55f6ad742480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd6025d4740 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad3681b0 0x55f6ad742480 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8000f80 tx=0x7fd5e800e870 comp rx=0 tx=0).stop 2024-08-31T07:31:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd6025d4740 1 -- 172.21.15.32:0/2133789852 shutdown_connections 2024-08-31T07:31:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd6025d4740 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6ad34f9f0 0x55f6ad754480 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd6025d4740 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad74e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd6025d4740 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad370390 0x55f6ad7429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd6025d4740 1 --2- 172.21.15.32:0/2133789852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad3681b0 0x55f6ad742480 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd6025d4740 1 -- 172.21.15.32:0/2133789852 >> 172.21.15.32:0/2133789852 conn(0x55f6ad385460 msgr2=0x55f6ad739b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.303+0000 7fd5daa00640 0 client.29096 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.304+0000 7fd6025d4740 1 -- 172.21.15.32:0/2133789852 shutdown_connections 2024-08-31T07:31:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.304+0000 7fd6025d4740 1 -- 172.21.15.32:0/2133789852 wait complete. 2024-08-31T07:31:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.304+0000 7fd6025d4740 20 client.29096 trim_cache size 0 max 16384 2024-08-31T07:31:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.322+0000 7fd6025d4740 1 Processor -- start 2024-08-31T07:31:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.322+0000 7fd6025d4740 1 -- start start 2024-08-31T07:31:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad3681b0 0x55f6ad7337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad738090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad391070 0x55f6ad74e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd6025d4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f41b1660 con 0x55f6ad3681b0 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd6025d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f41b3200 con 0x55f6ad370390 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd6025d4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f41b1d60 con 0x55f6ad391070 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd5f9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad3681b0 0x55f6ad7337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd5f9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad3681b0 0x55f6ad7337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53220/0 (socket says 172.21.15.32:53220) 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd5f9000640 1 -- 172.21.15.32:0/130779705 learned_addr learned my addr 172.21.15.32:0/130779705 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/130779705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad738090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.323+0000 7fd5f9a00640 1 --2- 172.21.15.32:0/130779705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad391070 0x55f6ad74e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.324+0000 7fd5f9000640 1 -- 172.21.15.32:0/130779705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad391070 msgr2=0x55f6ad74e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.324+0000 7fd5f9000640 1 --2- 172.21.15.32:0/130779705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad391070 0x55f6ad74e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.324+0000 7fd5f9000640 1 -- 172.21.15.32:0/130779705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 msgr2=0x55f6ad738090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.324+0000 7fd5f9000640 1 --2- 172.21.15.32:0/130779705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad738090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.324+0000 7fd5f9000640 1 -- 172.21.15.32:0/130779705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6ad743540 con 0x55f6ad3681b0 2024-08-31T07:31:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.324+0000 7fd5f9a00640 1 --2- 172.21.15.32:0/130779705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad391070 0x55f6ad74e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:31:16.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.324+0000 7fd5f9000640 1 --2- 172.21.15.32:0/130779705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad3681b0 0x55f6ad7337a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8011010 tx=0x7fd5e8010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae734da9ee153146 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.324+0000 7fd5e5600640 1 -- 172.21.15.32:0/130779705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8027040 con 0x55f6ad3681b0 2024-08-31T07:31:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.324+0000 7fd5e5600640 1 -- 172.21.15.32:0/130779705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e800e570 con 0x55f6ad3681b0 2024-08-31T07:31:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.324+0000 7fd5e5600640 1 -- 172.21.15.32:0/130779705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8011720 con 0x55f6ad3681b0 2024-08-31T07:31:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.325+0000 7fd6025d4740 1 -- 172.21.15.32:0/130779705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad3681b0 msgr2=0x55f6ad7337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.325+0000 7fd6025d4740 1 --2- 172.21.15.32:0/130779705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad3681b0 0x55f6ad7337a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8011010 tx=0x7fd5e8010e60 comp rx=0 tx=0).stop 2024-08-31T07:31:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.325+0000 7fd6025d4740 1 -- 172.21.15.32:0/130779705 shutdown_connections 2024-08-31T07:31:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.325+0000 7fd6025d4740 1 --2- 172.21.15.32:0/130779705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad391070 0x55f6ad74e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.325+0000 7fd6025d4740 1 --2- 172.21.15.32:0/130779705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad738090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.325+0000 7fd6025d4740 1 --2- 172.21.15.32:0/130779705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad3681b0 0x55f6ad7337a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.325+0000 7fd6025d4740 1 -- 172.21.15.32:0/130779705 >> 172.21.15.32:0/130779705 conn(0x55f6ad385460 msgr2=0x55f6ad737980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.325+0000 7fd6025d4740 1 -- 172.21.15.32:0/130779705 shutdown_connections 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.325+0000 7fd6025d4740 1 -- 172.21.15.32:0/130779705 wait complete. 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.326+0000 7fd6025d4740 1 -- start start 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.326+0000 7fd6025d4740 1 Processor -- start 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.326+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 0x55f6ad746a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.326+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad746f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.326+0000 7fd6025d4740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad7bdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.326+0000 7fd6025d4740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f41b2b30 con 0x55f6ad391070 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.326+0000 7fd6025d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f41b2460 con 0x55f6ad370390 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.326+0000 7fd6025d4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f41af660 con 0x55f6ad3681b0 2024-08-31T07:31:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.326+0000 7fd6025d4740 10 client.0 _ll_register_callbacks cb 0x7fd5e8009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad7bdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 0x55f6ad746a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad7bdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53228/0 (socket says 172.21.15.32:53228) 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9a00640 1 -- 172.21.15.32:0/1287340142 learned_addr learned my addr 172.21.15.32:0/1287340142 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad746f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9000640 1 -- 172.21.15.32:0/1287340142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 msgr2=0x55f6ad746f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9000640 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad746f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9000640 1 -- 172.21.15.32:0/1287340142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 msgr2=0x55f6ad7bdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9000640 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad7bdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9000640 1 -- 172.21.15.32:0/1287340142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6ad7be160 con 0x55f6ad3681b0 2024-08-31T07:31:16.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9a00640 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad7bdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5f9000640 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 0x55f6ad746a50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8002230 tx=0x7fd5e800ea10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b35d0bcc75f1923 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.327+0000 7fd5daa00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.328+0000 7fd5daa00640 1 -- 172.21.15.32:0/1287340142 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e803b070 con 0x55f6ad3681b0 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.328+0000 7fd5daa00640 1 -- 172.21.15.32:0/1287340142 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fd5e800ed20 con 0x55f6ad3681b0 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.328+0000 7fd6025d4740 1 -- 172.21.15.32:0/1287340142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e800e570 con 0x55f6ad3681b0 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.328+0000 7fd5daa00640 1 -- 172.21.15.32:0/1287340142 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd5e8044050 con 0x55f6ad3681b0 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.328+0000 7fd6025d4740 10 client.29092 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.328+0000 7fd6025d4740 1 -- 172.21.15.32:0/1287340142 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55f6ad7bf350 con 0x55f6ad3681b0 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.328+0000 7fd5daa00640 1 -- 172.21.15.32:0/1287340142 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e800e570 con 0x55f6ad3681b0 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.328+0000 7fd5daa00640 1 -- 172.21.15.32:0/1287340142 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fd5e803e020 con 0x55f6ad3681b0 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.329+0000 7fd6025d4740 10 client.29092 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.329+0000 7fd6025d4740 10 client.29092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.329+0000 7fd6025d4740 20 client.29092 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:16.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.329+0000 7fd6025d4740 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6ad324da0 0x55f6ad7c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.329+0000 7fd6025d4740 4 client.29092 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:31:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.329+0000 7fd6025d4740 1 -- 172.21.15.32:0/1287340142 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6ad6ab160 con 0x55f6ad324da0 2024-08-31T07:31:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.329+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6ad324da0 0x55f6ad7c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd5f3e00640 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6ad324da0 0x55f6ad7c1a50 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78426662bdc5690 server_cookie=b162c79752b2fb70 in_seq=0 out_seq=0 2024-08-31T07:31:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd5daa00640 10 client.29092 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd5daa00640 1 -- 172.21.15.32:0/1287340142 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6054 (crc 0 0 0) 0x55f6ad6ab160 con 0x55f6ad324da0 2024-08-31T07:31:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd5daa00640 10 client.29092 handle_command_reply: tid=0 2024-08-31T07:31:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd6025d4740 1 client.29092 shutdown 2024-08-31T07:31:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd6025d4740 1 -- 172.21.15.32:0/1287340142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 msgr2=0x55f6ad746a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd6025d4740 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 0x55f6ad746a50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8002230 tx=0x7fd5e800ea10 comp rx=0 tx=0).stop 2024-08-31T07:31:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd6025d4740 1 -- 172.21.15.32:0/1287340142 shutdown_connections 2024-08-31T07:31:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd6025d4740 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f6ad324da0 0x55f6ad7c1a50 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd6025d4740 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ad391070 0x55f6ad7bdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd6025d4740 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6ad370390 0x55f6ad746f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd6025d4740 1 --2- 172.21.15.32:0/1287340142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6ad3681b0 0x55f6ad746a50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd5daa00640 0 client.29092 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.330+0000 7fd6025d4740 1 -- 172.21.15.32:0/1287340142 >> 172.21.15.32:0/1287340142 conn(0x55f6ad385460 msgr2=0x55f6ad733570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.331+0000 7fd6025d4740 1 -- 172.21.15.32:0/1287340142 shutdown_connections 2024-08-31T07:31:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.331+0000 7fd6025d4740 1 -- 172.21.15.32:0/1287340142 wait complete. 2024-08-31T07:31:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.331+0000 7fd6025d4740 20 client.29092 trim_cache size 0 max 16384 2024-08-31T07:31:16.335 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:31:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:31:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:31:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:31:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.331+0000 7fd5fba00640 1 -- 172.21.15.32:0/3916754183 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5dc0b4ca0 msgr2=0x7fd5dc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.331+0000 7fd5fba00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5dc0b4ca0 0x7fd5dc0b71b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00096c0 tx=0x7fd5e0041040 comp rx=0 tx=0).stop 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.331+0000 7fd5fba00640 1 -- 172.21.15.32:0/3916754183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f4109b90 msgr2=0x7fd5f4075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.331+0000 7fd5fba00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f4109b90 0x7fd5f4075f30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d6b0 tx=0x7fd5ec00db70 comp rx=0 tx=0).stop 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.332+0000 7fd5fba00640 1 -- 172.21.15.32:0/3916754183 shutdown_connections 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.332+0000 7fd5fba00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5dc0b4ca0 0x7fd5dc0b71b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.332+0000 7fd5fba00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f4109b90 0x7fd5f4075f30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.332+0000 7fd5fba00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f4109240 0x7fd5f40759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.332+0000 7fd5fba00640 1 --2- 172.21.15.32:0/3916754183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f4103900 0x7fd5f40754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.332+0000 7fd5fba00640 1 -- 172.21.15.32:0/3916754183 >> 172.21.15.32:0/3916754183 conn(0x7fd5f407b5f0 msgr2=0x7fd5f4105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.332+0000 7fd5fba00640 1 -- 172.21.15.32:0/3916754183 shutdown_connections 2024-08-31T07:31:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:16.332+0000 7fd5fba00640 1 -- 172.21.15.32:0/3916754183 wait complete. 2024-08-31T07:31:16.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:16 smithi032 ceph-mon[17252]: pgmap v949: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.8 MiB/s wr, 156 op/s 2024-08-31T07:31:16.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 13.00 MB/sec execute 176 sec latency 20122.446 ms 2024-08-31T07:31:16.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:16 smithi073 ceph-mon[21995]: pgmap v949: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.8 MiB/s wr, 156 op/s 2024-08-31T07:31:16.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:16 smithi084 ceph-mon[22387]: pgmap v949: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.8 MiB/s wr, 156 op/s 2024-08-31T07:31:17.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.93 MB/sec execute 177 sec latency 21122.514 ms 2024-08-31T07:31:17.395 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:31:17.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:31:17.774 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:18.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.86 MB/sec execute 178 sec latency 22122.559 ms 2024-08-31T07:31:18.443 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:18 smithi032 ceph-mon[17252]: pgmap v950: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.2 MiB/s wr, 117 op/s 2024-08-31T07:31:18.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.504+0000 7fc872e00640 1 -- 172.21.15.32:0/3818090598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c103750 msgr2=0x7fc86c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:18.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.504+0000 7fc872e00640 1 --2- 172.21.15.32:0/3818090598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c103750 0x7fc86c103b60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc860009900 tx=0x7fc86002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:31:18.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.504+0000 7fc872e00640 1 -- 172.21.15.32:0/3818090598 shutdown_connections 2024-08-31T07:31:18.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.504+0000 7fc872e00640 1 --2- 172.21.15.32:0/3818090598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c1099e0 0x7fc86c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:18.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.504+0000 7fc872e00640 1 --2- 172.21.15.32:0/3818090598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c109090 0x7fc86c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:18.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.504+0000 7fc872e00640 1 --2- 172.21.15.32:0/3818090598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c103750 0x7fc86c103b60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:18.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.504+0000 7fc872e00640 1 -- 172.21.15.32:0/3818090598 >> 172.21.15.32:0/3818090598 conn(0x7fc86c0ff1e0 msgr2=0x7fc86c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:18.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.504+0000 7fc872e00640 1 -- 172.21.15.32:0/3818090598 shutdown_connections 2024-08-31T07:31:18.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.504+0000 7fc872e00640 1 -- 172.21.15.32:0/3818090598 wait complete. 2024-08-31T07:31:18.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.505+0000 7fc872e00640 1 Processor -- start 2024-08-31T07:31:18.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc872e00640 1 -- start start 2024-08-31T07:31:18.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc872e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c103750 0x7fc86c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:18.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc872e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c109090 0x7fc86c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:18.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc872e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c1099e0 0x7fc86c1a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:18.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc872e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c113690 con 0x7fc86c1099e0 2024-08-31T07:31:18.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc872e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c113510 con 0x7fc86c109090 2024-08-31T07:31:18.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc872e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c113810 con 0x7fc86c103750 2024-08-31T07:31:18.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc872400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c1099e0 0x7fc86c1a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:18.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc871a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c103750 0x7fc86c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:18.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc872400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c1099e0 0x7fc86c1a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53252/0 (socket says 172.21.15.32:53252) 2024-08-31T07:31:18.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc871000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c109090 0x7fc86c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:18.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.506+0000 7fc872400640 1 -- 172.21.15.32:0/1966158438 learned_addr learned my addr 172.21.15.32:0/1966158438 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:18.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.507+0000 7fc871a00640 1 -- 172.21.15.32:0/1966158438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c109090 msgr2=0x7fc86c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:18.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.507+0000 7fc871a00640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c109090 0x7fc86c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:18.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.507+0000 7fc871a00640 1 -- 172.21.15.32:0/1966158438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c1099e0 msgr2=0x7fc86c1a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:18.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.507+0000 7fc871a00640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c1099e0 0x7fc86c1a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:18.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.507+0000 7fc871a00640 1 -- 172.21.15.32:0/1966158438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86c1a2230 con 0x7fc86c103750 2024-08-31T07:31:18.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.507+0000 7fc871a00640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c103750 0x7fc86c19c2c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc86002a910 tx=0x7fc860031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:18.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.507+0000 7fc86a000640 1 -- 172.21.15.32:0/1966158438 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86003f030 con 0x7fc86c103750 2024-08-31T07:31:18.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.508+0000 7fc872e00640 1 -- 172.21.15.32:0/1966158438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86c1a24c0 con 0x7fc86c103750 2024-08-31T07:31:18.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.508+0000 7fc86a000640 1 -- 172.21.15.32:0/1966158438 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc860004490 con 0x7fc86c103750 2024-08-31T07:31:18.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.508+0000 7fc86a000640 1 -- 172.21.15.32:0/1966158438 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860035070 con 0x7fc86c103750 2024-08-31T07:31:18.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.508+0000 7fc872e00640 1 -- 172.21.15.32:0/1966158438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86c1a8c80 con 0x7fc86c103750 2024-08-31T07:31:18.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.509+0000 7fc872e00640 1 -- 172.21.15.32:0/1966158438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86c103b60 con 0x7fc86c103750 2024-08-31T07:31:18.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.512+0000 7fc86a000640 1 -- 172.21.15.32:0/1966158438 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc860042030 con 0x7fc86c103750 2024-08-31T07:31:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.513+0000 7fc86a000640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8540b4d20 0x7fc8540b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.513+0000 7fc86a000640 1 -- 172.21.15.32:0/1966158438 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fc860105930 con 0x7fc86c103750 2024-08-31T07:31:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.513+0000 7fc871000640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8540b4d20 0x7fc8540b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.514+0000 7fc871000640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8540b4d20 0x7fc8540b7230 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc86c108c40 tx=0x7fc858005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:18.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.515+0000 7fc86a000640 1 -- 172.21.15.32:0/1966158438 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc8600cc0c0 con 0x7fc86c103750 2024-08-31T07:31:18.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:18 smithi073 ceph-mon[21995]: pgmap v950: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.2 MiB/s wr, 117 op/s 2024-08-31T07:31:18.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:18 smithi084 ceph-mon[22387]: pgmap v950: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.2 MiB/s wr, 117 op/s 2024-08-31T07:31:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.751+0000 7fc872e00640 1 -- 172.21.15.32:0/1966158438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc86c1a8f70 con 0x7fc86c103750 2024-08-31T07:31:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.752+0000 7fc86a000640 1 -- 172.21.15.32:0/1966158438 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+11963 (secure 0 0 0) 0x7fc8600d0880 con 0x7fc86c103750 2024-08-31T07:31:18.753 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:31:18.753 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"btime":"2024-08-31T07:30:07:717453+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":67,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:30:07.599620+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:31:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 67 2024-08-31T07:31:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 -- 172.21.15.32:0/1966158438 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8540b4d20 msgr2=0x7fc8540b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8540b4d20 0x7fc8540b7230 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc86c108c40 tx=0x7fc858005f90 comp rx=0 tx=0).stop 2024-08-31T07:31:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 -- 172.21.15.32:0/1966158438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c103750 msgr2=0x7fc86c19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c103750 0x7fc86c19c2c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc86002a910 tx=0x7fc860031a20 comp rx=0 tx=0).stop 2024-08-31T07:31:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 -- 172.21.15.32:0/1966158438 shutdown_connections 2024-08-31T07:31:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc8540b4d20 0x7fc8540b7230 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c1099e0 0x7fc86c1a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc86c109090 0x7fc86c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 --2- 172.21.15.32:0/1966158438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc86c103750 0x7fc86c19c2c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 -- 172.21.15.32:0/1966158438 >> 172.21.15.32:0/1966158438 conn(0x7fc86c0ff1e0 msgr2=0x7fc86c10e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 -- 172.21.15.32:0/1966158438 shutdown_connections 2024-08-31T07:31:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:18.755+0000 7fc872e00640 1 -- 172.21.15.32:0/1966158438 wait complete. 2024-08-31T07:31:19.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.78 MB/sec execute 179 sec latency 23122.606 ms 2024-08-31T07:31:19.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:19 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1966158438' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:31:19.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:19 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1966158438' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:31:19.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:19 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1966158438' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:31:19.757 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:31:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:20.393 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.71 MB/sec execute 180 sec latency 24122.651 ms 2024-08-31T07:31:20.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:20 smithi032 ceph-mon[17252]: pgmap v951: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-08-31T07:31:20.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:20 smithi084 ceph-mon[22387]: pgmap v951: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-08-31T07:31:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:20 smithi073 ceph-mon[21995]: pgmap v951: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-08-31T07:31:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.876+0000 7f6901e00640 1 -- 172.21.15.32:0/1313869313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f40318a0 con 0x7f6904103900 2024-08-31T07:31:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.877+0000 7f6909400640 1 -- 172.21.15.32:0/1313869313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6904103900 msgr2=0x7f6904103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.877+0000 7f6909400640 1 --2- 172.21.15.32:0/1313869313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6904103900 0x7f6904103d10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009900 tx=0x7f68f402a410 comp rx=0 tx=0).stop 2024-08-31T07:31:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.877+0000 7f6909400640 1 -- 172.21.15.32:0/1313869313 shutdown_connections 2024-08-31T07:31:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.877+0000 7f6909400640 1 --2- 172.21.15.32:0/1313869313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6904109b90 0x7f69041103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.877+0000 7f6909400640 1 --2- 172.21.15.32:0/1313869313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6904109240 0x7f6904109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.877+0000 7f6909400640 1 --2- 172.21.15.32:0/1313869313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6904103900 0x7f6904103d10 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.877+0000 7f6909400640 1 -- 172.21.15.32:0/1313869313 >> 172.21.15.32:0/1313869313 conn(0x7f690407b5f0 msgr2=0x7f690407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:20.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.877+0000 7f6909400640 1 -- 172.21.15.32:0/1313869313 shutdown_connections 2024-08-31T07:31:20.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.877+0000 7f6909400640 1 -- 172.21.15.32:0/1313869313 wait complete. 2024-08-31T07:31:20.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.878+0000 7f6909400640 1 Processor -- start 2024-08-31T07:31:20.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.878+0000 7f6909400640 1 -- start start 2024-08-31T07:31:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f6909400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6904103900 0x7f690419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f6909400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6904109240 0x7f690419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f6909400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6904109b90 0x7f690419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f6909400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6904113850 con 0x7f6904103900 2024-08-31T07:31:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f6909400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69041136d0 con 0x7f6904109240 2024-08-31T07:31:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f6909400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69041139d0 con 0x7f6904109b90 2024-08-31T07:31:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f6902800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6904103900 0x7f690419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f6902800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6904103900 0x7f690419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53256/0 (socket says 172.21.15.32:53256) 2024-08-31T07:31:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f6902800640 1 -- 172.21.15.32:0/2753706913 learned_addr learned my addr 172.21.15.32:0/2753706913 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f68f9e00640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6904109240 0x7f690419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.879+0000 7f6903200640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6904109b90 0x7f690419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.880+0000 7f6902800640 1 -- 172.21.15.32:0/2753706913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6904109b90 msgr2=0x7f690419fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.880+0000 7f6902800640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6904109b90 0x7f690419fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.880+0000 7f6902800640 1 -- 172.21.15.32:0/2753706913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6904109240 msgr2=0x7f690419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.880+0000 7f6902800640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6904109240 0x7f690419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.880+0000 7f6902800640 1 -- 172.21.15.32:0/2753706913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69041a0210 con 0x7f6904103900 2024-08-31T07:31:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.880+0000 7f6903200640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6904109b90 0x7f690419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.880+0000 7f6902800640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6904103900 0x7f690419a2a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005940 tx=0x7f68f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.881+0000 7f68fbe00640 1 -- 172.21.15.32:0/2753706913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f403f030 con 0x7f6904103900 2024-08-31T07:31:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.881+0000 7f68fbe00640 1 -- 172.21.15.32:0/2753706913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f4003aa0 con 0x7f6904103900 2024-08-31T07:31:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.881+0000 7f6909400640 1 -- 172.21.15.32:0/2753706913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69041a04a0 con 0x7f6904103900 2024-08-31T07:31:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.881+0000 7f68fbe00640 1 -- 172.21.15.32:0/2753706913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4035070 con 0x7f6904103900 2024-08-31T07:31:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.881+0000 7f6909400640 1 -- 172.21.15.32:0/2753706913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69041a6c60 con 0x7f6904103900 2024-08-31T07:31:20.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.884+0000 7f6910039740 1 Processor -- start 2024-08-31T07:31:20.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.884+0000 7f6910039740 1 -- start start 2024-08-31T07:31:20.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.884+0000 7f68fbe00640 1 -- 172.21.15.32:0/2753706913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f68f4042030 con 0x7f6904103900 2024-08-31T07:31:20.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6910039740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5df070 0x558b0d988cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6910039740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5b61b0 0x558b0d989200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6910039740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5be390 0x558b0d989740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6910039740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558b0d8fb7b0 con 0x558b0d5b61b0 2024-08-31T07:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6910039740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558b0d8ead40 con 0x558b0d5be390 2024-08-31T07:31:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6910039740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558b0d8eecf0 con 0x558b0d5df070 2024-08-31T07:31:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6903200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5df070 0x558b0d988cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6903200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5df070 0x558b0d988cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41418/0 (socket says 172.21.15.32:41418) 2024-08-31T07:31:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6903200640 1 -- 172.21.15.32:0/4250544285 learned_addr learned my addr 172.21.15.32:0/4250544285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f68f9e00640 1 --2- 172.21.15.32:0/4250544285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5be390 0x558b0d989740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6902800640 1 --2- 172.21.15.32:0/4250544285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5b61b0 0x558b0d989200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6903200640 1 -- 172.21.15.32:0/4250544285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5be390 msgr2=0x558b0d989740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6903200640 1 --2- 172.21.15.32:0/4250544285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5be390 0x558b0d989740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6903200640 1 -- 172.21.15.32:0/4250544285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5b61b0 msgr2=0x558b0d989200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6903200640 1 --2- 172.21.15.32:0/4250544285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5b61b0 0x558b0d989200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.885+0000 7f6903200640 1 -- 172.21.15.32:0/4250544285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b0d64a650 con 0x558b0d5df070 2024-08-31T07:31:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.886+0000 7f6903200640 1 --2- 172.21.15.32:0/4250544285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5df070 0x558b0d988cc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0107a0 tx=0x7f68fc010c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c55ed7ec95ebcace server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.886+0000 7f68f3400640 1 -- 172.21.15.32:0/4250544285 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc003b70 con 0x558b0d5df070 2024-08-31T07:31:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.886+0000 7f68f3400640 1 -- 172.21.15.32:0/4250544285 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68fc010e50 con 0x558b0d5df070 2024-08-31T07:31:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.886+0000 7f68f3400640 1 -- 172.21.15.32:0/4250544285 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0046c0 con 0x558b0d5df070 2024-08-31T07:31:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f68fbe00640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68e80b4d20 0x7f68e80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f6902800640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68e80b4d20 0x7f68e80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f68fbe00640 1 -- 172.21.15.32:0/2753706913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f68f41058a0 con 0x7f6904103900 2024-08-31T07:31:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f6910039740 1 -- 172.21.15.32:0/4250544285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5df070 msgr2=0x558b0d988cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f6910039740 1 --2- 172.21.15.32:0/4250544285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5df070 0x558b0d988cc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0107a0 tx=0x7f68fc010c60 comp rx=0 tx=0).stop 2024-08-31T07:31:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f6910039740 1 -- 172.21.15.32:0/4250544285 shutdown_connections 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f6910039740 1 --2- 172.21.15.32:0/4250544285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5be390 0x558b0d989740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f6910039740 1 --2- 172.21.15.32:0/4250544285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5b61b0 0x558b0d989200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f6910039740 1 --2- 172.21.15.32:0/4250544285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5df070 0x558b0d988cc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f6910039740 1 -- 172.21.15.32:0/4250544285 >> 172.21.15.32:0/4250544285 conn(0x558b0d5d3460 msgr2=0x558b0d986bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f6910039740 1 -- 172.21.15.32:0/4250544285 shutdown_connections 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.887+0000 7f6910039740 1 -- 172.21.15.32:0/4250544285 wait complete. 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.888+0000 7f6902800640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68e80b4d20 0x7f68e80b7230 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f68f41030f0 tx=0x7f68f402a410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.888+0000 7f6910039740 1 -- start start 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.888+0000 7f6910039740 1 Processor -- start 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f6910039740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5b61b0 0x558b0d9948a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f6910039740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d994e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f6910039740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5df070 0x558b0d9a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f6910039740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558b0d74d700 con 0x558b0d5df070 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f6910039740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558b0d871cf0 con 0x558b0d5b61b0 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f6910039740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558b0d9974a0 con 0x558b0d5be390 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f6910039740 10 client.0 _ll_register_callbacks cb 0x558b0d897470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f68f9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d994e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f6903200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5df070 0x558b0d9a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f68f9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d994e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41426/0 (socket says 172.21.15.32:41426) 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f68f9e00640 1 -- 172.21.15.32:0/1990372808 learned_addr learned my addr 172.21.15.32:0/1990372808 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.889+0000 7f6903200640 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5b61b0 0x558b0d9948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f68f9e00640 1 -- 172.21.15.32:0/1990372808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5b61b0 msgr2=0x558b0d9948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f68f9e00640 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5b61b0 0x558b0d9948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f68f9e00640 1 -- 172.21.15.32:0/1990372808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5df070 msgr2=0x558b0d9a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f68f9e00640 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5df070 0x558b0d9a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f68f9e00640 1 -- 172.21.15.32:0/1990372808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b0d8b28b0 con 0x558b0d5be390 2024-08-31T07:31:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f6903200640 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5b61b0 0x558b0d9948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:31:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f68f9e00640 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d994e00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f68ec012700 tx=0x7f68ec012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4443068407b6bac6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f68e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:31:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f68e7e00640 1 -- 172.21.15.32:0/1990372808 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00f920 con 0x558b0d5be390 2024-08-31T07:31:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f6910039740 1 -- 172.21.15.32:0/1990372808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68fc010e50 con 0x558b0d5be390 2024-08-31T07:31:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.890+0000 7f68e7e00640 1 -- 172.21.15.32:0/1990372808 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f68ec00fac0 con 0x558b0d5be390 2024-08-31T07:31:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f68e7e00640 1 -- 172.21.15.32:0/1990372808 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f68ec00fcc0 con 0x558b0d5be390 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f6910039740 10 client.29110 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f6910039740 1 -- 172.21.15.32:0/1990372808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x558b0d9a3590 con 0x558b0d5be390 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f68e7e00640 1 -- 172.21.15.32:0/1990372808 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68fc010e50 con 0x558b0d5be390 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f68e7e00640 1 -- 172.21.15.32:0/1990372808 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f68ec01c420 con 0x558b0d5be390 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f6910039740 10 client.29110 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f6910039740 10 client.29110 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f6910039740 20 client.29110 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f6910039740 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558b0d59d9f0 0x558b0d9a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f6910039740 4 client.29110 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.891+0000 7f6910039740 1 -- 172.21.15.32:0/1990372808 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b0d8fd700 con 0x558b0d59d9f0 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.892+0000 7f6903200640 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558b0d59d9f0 0x558b0d9a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.892+0000 7f6903200640 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558b0d59d9f0 0x558b0d9a6ac0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=94027ca00e82988 server_cookie=70d4d2ff857f7856 in_seq=0 out_seq=0 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.892+0000 7f68e7e00640 10 client.29110 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:31:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f68e7e00640 1 -- 172.21.15.32:0/1990372808 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b0d8fd700 con 0x558b0d59d9f0 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f68e7e00640 10 client.29110 handle_command_reply: tid=0 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f6910039740 1 client.29110 shutdown 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f6910039740 1 -- 172.21.15.32:0/1990372808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 msgr2=0x558b0d994e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f6910039740 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d994e00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f68ec012700 tx=0x7f68ec012bc0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f6910039740 1 -- 172.21.15.32:0/1990372808 shutdown_connections 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f6910039740 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558b0d59d9f0 0x558b0d9a6ac0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f6910039740 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5df070 0x558b0d9a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f6910039740 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d994e00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f68e7e00640 0 client.29110 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f6910039740 1 --2- 172.21.15.32:0/1990372808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5b61b0 0x558b0d9948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.893+0000 7f6910039740 1 -- 172.21.15.32:0/1990372808 >> 172.21.15.32:0/1990372808 conn(0x558b0d5d3460 msgr2=0x558b0d980de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.894+0000 7f6910039740 1 -- 172.21.15.32:0/1990372808 shutdown_connections 2024-08-31T07:31:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.894+0000 7f6910039740 1 -- 172.21.15.32:0/1990372808 wait complete. 2024-08-31T07:31:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.894+0000 7f6910039740 20 client.29110 trim_cache size 0 max 16384 2024-08-31T07:31:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.908+0000 7f6910039740 1 Processor -- start 2024-08-31T07:31:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.908+0000 7f6910039740 1 -- start start 2024-08-31T07:31:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.908+0000 7f6910039740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5b61b0 0x558b0d992f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.908+0000 7f6910039740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5be390 0x558b0d993460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.908+0000 7f6910039740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 0x558b0d980de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6910039740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ad2a0 con 0x558b0d5be390 2024-08-31T07:31:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6910039740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69041aee40 con 0x558b0d5df070 2024-08-31T07:31:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6910039740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ad9a0 con 0x558b0d5b61b0 2024-08-31T07:31:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6903200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5b61b0 0x558b0d992f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6903200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5b61b0 0x558b0d992f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41432/0 (socket says 172.21.15.32:41432) 2024-08-31T07:31:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6903200640 1 -- 172.21.15.32:0/2885017754 learned_addr learned my addr 172.21.15.32:0/2885017754 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6903200640 1 --2- 172.21.15.32:0/2885017754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 0x558b0d980de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f68f9e00640 1 --2- 172.21.15.32:0/2885017754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5be390 0x558b0d993460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6903200640 1 -- 172.21.15.32:0/2885017754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 msgr2=0x558b0d980de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6903200640 1 --2- 172.21.15.32:0/2885017754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 0x558b0d980de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6903200640 1 -- 172.21.15.32:0/2885017754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5be390 msgr2=0x558b0d993460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6903200640 1 --2- 172.21.15.32:0/2885017754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5be390 0x558b0d993460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.909+0000 7f6903200640 1 -- 172.21.15.32:0/2885017754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b0d995960 con 0x558b0d5b61b0 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.910+0000 7f6903200640 1 --2- 172.21.15.32:0/2885017754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5b61b0 0x558b0d992f20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f68fc010ca0 tx=0x7f68fc0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c72b0d7f1a5d407f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.910+0000 7f68f2a00640 1 -- 172.21.15.32:0/2885017754 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc024040 con 0x558b0d5b61b0 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.910+0000 7f68f2a00640 1 -- 172.21.15.32:0/2885017754 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68fc010e50 con 0x558b0d5b61b0 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.910+0000 7f68f2a00640 1 -- 172.21.15.32:0/2885017754 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc003bd0 con 0x558b0d5b61b0 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.911+0000 7f6910039740 1 -- 172.21.15.32:0/2885017754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5b61b0 msgr2=0x558b0d992f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.911+0000 7f6910039740 1 --2- 172.21.15.32:0/2885017754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5b61b0 0x558b0d992f20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f68fc010ca0 tx=0x7f68fc0096a0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.911+0000 7f6910039740 1 -- 172.21.15.32:0/2885017754 shutdown_connections 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.911+0000 7f6910039740 1 --2- 172.21.15.32:0/2885017754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 0x558b0d980de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.911+0000 7f6910039740 1 --2- 172.21.15.32:0/2885017754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5be390 0x558b0d993460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.911+0000 7f6910039740 1 --2- 172.21.15.32:0/2885017754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5b61b0 0x558b0d992f20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.911+0000 7f6910039740 1 -- 172.21.15.32:0/2885017754 >> 172.21.15.32:0/2885017754 conn(0x558b0d5d3460 msgr2=0x558b0d98f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.911+0000 7f6910039740 1 -- 172.21.15.32:0/2885017754 shutdown_connections 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.911+0000 7f6910039740 1 -- 172.21.15.32:0/2885017754 wait complete. 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.911+0000 7f6910039740 1 -- start start 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f6910039740 1 Processor -- start 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f6910039740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5b61b0 0x558b0d99c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f6910039740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d99cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f6910039740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 0x558b0d9976b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f6910039740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ae770 con 0x558b0d5b61b0 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f6910039740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ae0a0 con 0x558b0d5df070 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f6910039740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69041ab2a0 con 0x558b0d5be390 2024-08-31T07:31:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f6910039740 10 client.0 _ll_register_callbacks cb 0x7f68ec009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:20.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f6903200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5b61b0 0x558b0d99c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f68f9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d99cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f68f9e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d99cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41442/0 (socket says 172.21.15.32:41442) 2024-08-31T07:31:20.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.912+0000 7f68f9e00640 1 -- 172.21.15.32:0/1486343565 learned_addr learned my addr 172.21.15.32:0/1486343565 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:20.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.913+0000 7f68f9e00640 1 -- 172.21.15.32:0/1486343565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 msgr2=0x558b0d9976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:20.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.913+0000 7f6903200640 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 0x558b0d9976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.913+0000 7f68f9e00640 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 0x558b0d9976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.913+0000 7f68f9e00640 1 -- 172.21.15.32:0/1486343565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5b61b0 msgr2=0x558b0d99c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.913+0000 7f68f9e00640 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5b61b0 0x558b0d99c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.913+0000 7f68f9e00640 1 -- 172.21.15.32:0/1486343565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b0d997bf0 con 0x558b0d5be390 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.913+0000 7f6903200640 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 0x558b0d9976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.913+0000 7f68f9e00640 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d99cc40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f68ec01c890 tx=0x7f68ec01c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=460185ec03f9c8a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.913+0000 7f68e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.913+0000 7f68e7e00640 1 -- 172.21.15.32:0/1486343565 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec029020 con 0x558b0d5be390 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.914+0000 7f68e7e00640 1 -- 172.21.15.32:0/1486343565 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f68ec01cc90 con 0x558b0d5be390 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.914+0000 7f6910039740 1 -- 172.21.15.32:0/1486343565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68fc010e50 con 0x558b0d5be390 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.914+0000 7f68e7e00640 1 -- 172.21.15.32:0/1486343565 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f68ec034050 con 0x558b0d5be390 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.914+0000 7f6910039740 10 client.29116 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.914+0000 7f6910039740 1 -- 172.21.15.32:0/1486343565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x558b0da11ac0 con 0x558b0d5be390 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.914+0000 7f68e7e00640 1 -- 172.21.15.32:0/1486343565 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68fc010e50 con 0x558b0d5be390 2024-08-31T07:31:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.914+0000 7f68e7e00640 1 -- 172.21.15.32:0/1486343565 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f68ec0039d0 con 0x558b0d5be390 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.915+0000 7f6910039740 10 client.29116 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.915+0000 7f6910039740 10 client.29116 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.915+0000 7f6910039740 20 client.29116 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.915+0000 7f6910039740 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558b0d572da0 0x558b0da142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.915+0000 7f6910039740 4 client.29116 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.915+0000 7f6910039740 1 -- 172.21.15.32:0/1486343565 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b0d8f91a0 con 0x558b0d572da0 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.915+0000 7f6903200640 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558b0d572da0 0x558b0da142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.915+0000 7f6903200640 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558b0d572da0 0x558b0da142a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=567a5c9d9cc4bd54 server_cookie=ddbbb9672828cdf in_seq=0 out_seq=0 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.915+0000 7f68e7e00640 10 client.29116 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f68e7e00640 1 -- 172.21.15.32:0/1486343565 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b0d8f91a0 con 0x558b0d572da0 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f68e7e00640 10 client.29116 handle_command_reply: tid=0 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 client.29116 shutdown 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 -- 172.21.15.32:0/1486343565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 msgr2=0x558b0d99cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d99cc40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f68ec01c890 tx=0x7f68ec01c6d0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 -- 172.21.15.32:0/1486343565 shutdown_connections 2024-08-31T07:31:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x558b0d572da0 0x558b0da142a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b0d5df070 0x558b0d9976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b0d5be390 0x558b0d99cc40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 --2- 172.21.15.32:0/1486343565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b0d5b61b0 0x558b0d99c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f68e7e00640 0 client.29116 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:31:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 -- 172.21.15.32:0/1486343565 >> 172.21.15.32:0/1486343565 conn(0x558b0d5d3460 msgr2=0x558b0d98f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 -- 172.21.15.32:0/1486343565 shutdown_connections 2024-08-31T07:31:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 1 -- 172.21.15.32:0/1486343565 wait complete. 2024-08-31T07:31:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.916+0000 7f6910039740 20 client.29116 trim_cache size 0 max 16384 2024-08-31T07:31:20.921 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:31:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 -- 172.21.15.32:0/2753706913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68e80b4d20 msgr2=0x7f68e80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68e80b4d20 0x7f68e80b7230 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f68f41030f0 tx=0x7f68f402a410 comp rx=0 tx=0).stop 2024-08-31T07:31:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 -- 172.21.15.32:0/2753706913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6904103900 msgr2=0x7f690419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6904103900 0x7f690419a2a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005940 tx=0x7f68f4004270 comp rx=0 tx=0).stop 2024-08-31T07:31:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 -- 172.21.15.32:0/2753706913 shutdown_connections 2024-08-31T07:31:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f68e80b4d20 0x7f68e80b7230 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6904109b90 0x7f690419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6904109240 0x7f690419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 --2- 172.21.15.32:0/2753706913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6904103900 0x7f690419a2a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 -- 172.21.15.32:0/2753706913 >> 172.21.15.32:0/2753706913 conn(0x7f690407b5f0 msgr2=0x7f6904107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 -- 172.21.15.32:0/2753706913 shutdown_connections 2024-08-31T07:31:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:20.917+0000 7f6909400640 1 -- 172.21.15.32:0/2753706913 wait complete. 2024-08-31T07:31:21.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.64 MB/sec execute 181 sec latency 25122.781 ms 2024-08-31T07:31:22.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:31:22.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:31:22.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.57 MB/sec execute 182 sec latency 26122.864 ms 2024-08-31T07:31:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:22.536 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:22 smithi032 ceph-mon[17252]: pgmap v952: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 372 KiB/s wr, 59 op/s 2024-08-31T07:31:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:22 smithi073 ceph-mon[21995]: pgmap v952: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 372 KiB/s wr, 59 op/s 2024-08-31T07:31:22.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:22 smithi084 ceph-mon[22387]: pgmap v952: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 372 KiB/s wr, 59 op/s 2024-08-31T07:31:23.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.077+0000 7fdfd4a00640 1 -- 172.21.15.32:0/282380242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd01099e0 msgr2=0x7fdfd0110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:23.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.077+0000 7fdfd4a00640 1 --2- 172.21.15.32:0/282380242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd01099e0 0x7fdfd0110210 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009ca0 tx=0x7fdfc802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.077+0000 7fdfd4a00640 1 -- 172.21.15.32:0/282380242 shutdown_connections 2024-08-31T07:31:23.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.077+0000 7fdfd4a00640 1 --2- 172.21.15.32:0/282380242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd01099e0 0x7fdfd0110210 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.077+0000 7fdfd4a00640 1 --2- 172.21.15.32:0/282380242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfd0109090 0x7fdfd01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.077+0000 7fdfd4a00640 1 --2- 172.21.15.32:0/282380242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfd0103750 0x7fdfd0103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.077+0000 7fdfd4a00640 1 -- 172.21.15.32:0/282380242 >> 172.21.15.32:0/282380242 conn(0x7fdfd00ff1e0 msgr2=0x7fdfd0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.077+0000 7fdfd4a00640 1 -- 172.21.15.32:0/282380242 shutdown_connections 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.077+0000 7fdfd4a00640 1 -- 172.21.15.32:0/282380242 wait complete. 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.078+0000 7fdfd4a00640 1 Processor -- start 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfd4a00640 1 -- start start 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfd4a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfd0103750 0x7fdfd019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfd4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0109090 0x7fdfd019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfd4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfd01099e0 0x7fdfd019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfd4a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01136b0 con 0x7fdfd0109090 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfd4a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0113530 con 0x7fdfd01099e0 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfd4a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0113830 con 0x7fdfd0103750 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfc7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0109090 0x7fdfd019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfc7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0109090 0x7fdfd019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53318/0 (socket says 172.21.15.32:53318) 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfc7e00640 1 -- 172.21.15.32:0/2634341418 learned_addr learned my addr 172.21.15.32:0/2634341418 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfcf400640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfd0103750 0x7fdfd019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.079+0000 7fdfcfe00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfd01099e0 0x7fdfd019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.080+0000 7fdfcf400640 1 -- 172.21.15.32:0/2634341418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfd01099e0 msgr2=0x7fdfd019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.080+0000 7fdfcf400640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfd01099e0 0x7fdfd019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.080+0000 7fdfcf400640 1 -- 172.21.15.32:0/2634341418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0109090 msgr2=0x7fdfd019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.080+0000 7fdfcf400640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0109090 0x7fdfd019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.080+0000 7fdfcf400640 1 -- 172.21.15.32:0/2634341418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd01a0010 con 0x7fdfd0103750 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.080+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0109090 0x7fdfd019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.080+0000 7fdfcf400640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfd0103750 0x7fdfd019a0a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000e9f0 tx=0x7fdfc000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.081+0000 7fdfccc00640 1 -- 172.21.15.32:0/2634341418 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000ccc0 con 0x7fdfd0103750 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.081+0000 7fdfd4a00640 1 -- 172.21.15.32:0/2634341418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd01a0300 con 0x7fdfd0103750 2024-08-31T07:31:23.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.081+0000 7fdfccc00640 1 -- 172.21.15.32:0/2634341418 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc0005020 con 0x7fdfd0103750 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.081+0000 7fdfccc00640 1 -- 172.21.15.32:0/2634341418 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0004a00 con 0x7fdfd0103750 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.081+0000 7fdfd4a00640 1 -- 172.21.15.32:0/2634341418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd01a6a60 con 0x7fdfd0103750 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.084+0000 7fdfdcc45740 1 Processor -- start 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.084+0000 7fdfdcc45740 1 -- start start 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.084+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f43e070 0x55757f7e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.084+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f4151b0 0x55757f7e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.084+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f41d390 0x55757f7e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.084+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55757f75a7b0 con 0x55757f4151b0 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.084+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55757f749d40 con 0x55757f41d390 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.084+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55757f74dcf0 con 0x55757f43e070 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfc7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f41d390 0x55757f7e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfcfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f43e070 0x55757f7e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfc7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f41d390 0x55757f7e45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47380/0 (socket says 172.21.15.32:47380) 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfc7e00640 1 -- 172.21.15.32:0/3076448627 learned_addr learned my addr 172.21.15.32:0/3076448627 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfc7e00640 1 -- 172.21.15.32:0/3076448627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f43e070 msgr2=0x55757f7e3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/3076448627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f43e070 0x55757f7e3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfc7e00640 1 -- 172.21.15.32:0/3076448627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f4151b0 msgr2=0x55757f7e40a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/3076448627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f4151b0 0x55757f7e40a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfc7e00640 1 -- 172.21.15.32:0/3076448627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757f4a9650 con 0x55757f41d390 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfcfe00640 1 --2- 172.21.15.32:0/3076448627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f43e070 0x55757f7e3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfccc00640 1 -- 172.21.15.32:0/2634341418 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fdfc0004ba0 con 0x7fdfd0103750 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.085+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/3076448627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f41d390 0x55757f7e45e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80114c0 tx=0x7fdfb8011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04c0f7fc1ee7190 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.086+0000 7fdfbfe00640 1 -- 172.21.15.32:0/3076448627 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb800f5c0 con 0x55757f41d390 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.086+0000 7fdfbfe00640 1 -- 172.21.15.32:0/3076448627 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb800fc10 con 0x55757f41d390 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.086+0000 7fdfbfe00640 1 -- 172.21.15.32:0/3076448627 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8019c30 con 0x55757f41d390 2024-08-31T07:31:23.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3076448627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f41d390 msgr2=0x55757f7e45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3076448627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f41d390 0x55757f7e45e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80114c0 tx=0x7fdfb8011980 comp rx=0 tx=0).stop 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3076448627 shutdown_connections 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3076448627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f41d390 0x55757f7e45e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3076448627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f4151b0 0x55757f7e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3076448627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f43e070 0x55757f7e3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3076448627 >> 172.21.15.32:0/3076448627 conn(0x55757f432460 msgr2=0x55757f7e1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3076448627 shutdown_connections 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3076448627 wait complete. 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfccc00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdfb40b4bb0 0x7fdfb40b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.087+0000 7fdfccc00640 1 -- 172.21.15.32:0/2634341418 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fdfc00e54b0 con 0x7fdfd0103750 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.088+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdfb40b4bb0 0x7fdfb40b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.088+0000 7fdfdcc45740 1 -- start start 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.088+0000 7fdfdcc45740 1 Processor -- start 2024-08-31T07:31:23.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.088+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdfb40b4bb0 0x7fdfb40b70c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdfb801b040 tx=0x7fdfb8009720 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 0x55757f7f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 0x55757f7f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 0x55757f7ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55757f5ac700 con 0x55757f43e070 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55757f6d0cf0 con 0x55757f4151b0 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55757f7f6670 con 0x55757f41d390 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfdcc45740 10 client.0 _ll_register_callbacks cb 0x55757f6f6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfcf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 0x55757f7f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfcfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 0x55757f7ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfc7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 0x55757f7f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfcf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 0x55757f7f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47396/0 (socket says 172.21.15.32:47396) 2024-08-31T07:31:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.089+0000 7fdfcf400640 1 -- 172.21.15.32:0/3670358914 learned_addr learned my addr 172.21.15.32:0/3670358914 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.090+0000 7fdfc7e00640 1 -- 172.21.15.32:0/3670358914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 msgr2=0x55757f7f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.090+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 0x55757f7f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.090+0000 7fdfc7e00640 1 -- 172.21.15.32:0/3670358914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 msgr2=0x55757f7ff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.090+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 0x55757f7ff9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.090+0000 7fdfc7e00640 1 -- 172.21.15.32:0/3670358914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757f7118b0 con 0x55757f41d390 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.090+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 0x55757f7f3f80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8011e80 tx=0x7fdfb8019770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ade083d6266377d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.090+0000 7fdfbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.090+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3670358914 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80044d0 con 0x55757f41d390 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.090+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3670358914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb800fc10 con 0x55757f41d390 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.090+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3670358914 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fdfb8016040 con 0x55757f41d390 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3670358914 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fdfb803d050 con 0x55757f41d390 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfdcc45740 10 client.29128 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3670358914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55757f802620 con 0x55757f41d390 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3670358914 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb800fc10 con 0x55757f41d390 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3670358914 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fdfb8025510 con 0x55757f41d390 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfdcc45740 10 client.29128 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfdcc45740 10 client.29128 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfdcc45740 20 client.29128 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55757f3fc9f0 0x55757f805980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfdcc45740 4 client.29128 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.091+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3670358914 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55757f75c700 con 0x55757f3fc9f0 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.092+0000 7fdfcf400640 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55757f3fc9f0 0x55757f805980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.092+0000 7fdfcf400640 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55757f3fc9f0 0x55757f805980 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d316cf8330cb6de1 server_cookie=678b9295da5c922b in_seq=0 out_seq=0 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.092+0000 7fdfbcc00640 10 client.29128 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3670358914 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55757f75c700 con 0x55757f3fc9f0 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfbcc00640 10 client.29128 handle_command_reply: tid=0 2024-08-31T07:31:23.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfdcc45740 1 client.29128 shutdown 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3670358914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 msgr2=0x55757f7f3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 0x55757f7f3f80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8011e80 tx=0x7fdfb8019770 comp rx=0 tx=0).stop 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3670358914 shutdown_connections 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55757f3fc9f0 0x55757f805980 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 0x55757f7ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 0x55757f7f3f80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfbcc00640 0 client.29128 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3670358914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 0x55757f7f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.093+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3670358914 >> 172.21.15.32:0/3670358914 conn(0x55757f432460 msgr2=0x55757f7e6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.094+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3670358914 shutdown_connections 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.094+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3670358914 wait complete. 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.094+0000 7fdfdcc45740 20 client.29128 trim_cache size 0 max 16384 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.108+0000 7fdfdcc45740 1 Processor -- start 2024-08-31T07:31:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.108+0000 7fdfdcc45740 1 -- start start 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.108+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f4151b0 0x55757f7e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.108+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f41d390 0x55757f7f5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.108+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f43e070 0x55757f7e1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.108+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ad060 con 0x55757f41d390 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.108+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01aec00 con 0x55757f43e070 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.108+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ad760 con 0x55757f4151b0 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfc7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f41d390 0x55757f7f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfc7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f41d390 0x55757f7f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53328/0 (socket says 172.21.15.32:53328) 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfc7e00640 1 -- 172.21.15.32:0/1418996417 learned_addr learned my addr 172.21.15.32:0/1418996417 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfcfe00640 1 --2- 172.21.15.32:0/1418996417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f43e070 0x55757f7e1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfcf400640 1 --2- 172.21.15.32:0/1418996417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f4151b0 0x55757f7e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfc7e00640 1 -- 172.21.15.32:0/1418996417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f4151b0 msgr2=0x55757f7e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/1418996417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f4151b0 0x55757f7e75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfc7e00640 1 -- 172.21.15.32:0/1418996417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f43e070 msgr2=0x55757f7e1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/1418996417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f43e070 0x55757f7e1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfc7e00640 1 -- 172.21.15.32:0/1418996417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757f7f1d60 con 0x55757f41d390 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.109+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/1418996417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f41d390 0x55757f7f5470 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8019740 tx=0x7fdfb8009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc499c59984d14f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.110+0000 7fdfbf400640 1 -- 172.21.15.32:0/1418996417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8046070 con 0x55757f41d390 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.110+0000 7fdfbf400640 1 -- 172.21.15.32:0/1418996417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb8034070 con 0x55757f41d390 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.110+0000 7fdfbf400640 1 -- 172.21.15.32:0/1418996417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8025480 con 0x55757f41d390 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.110+0000 7fdfdcc45740 1 -- 172.21.15.32:0/1418996417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f41d390 msgr2=0x55757f7f5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.110+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/1418996417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f41d390 0x55757f7f5470 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8019740 tx=0x7fdfb8009d40 comp rx=0 tx=0).stop 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.111+0000 7fdfdcc45740 1 -- 172.21.15.32:0/1418996417 shutdown_connections 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.111+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/1418996417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f43e070 0x55757f7e1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.111+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/1418996417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f41d390 0x55757f7f5470 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.111+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/1418996417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f4151b0 0x55757f7e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.111+0000 7fdfdcc45740 1 -- 172.21.15.32:0/1418996417 >> 172.21.15.32:0/1418996417 conn(0x55757f432460 msgr2=0x55757f7f4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.111+0000 7fdfdcc45740 1 -- 172.21.15.32:0/1418996417 shutdown_connections 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.111+0000 7fdfdcc45740 1 -- 172.21.15.32:0/1418996417 wait complete. 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.111+0000 7fdfdcc45740 1 -- start start 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.111+0000 7fdfdcc45740 1 Processor -- start 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 0x55757f7fc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 0x55757f7f7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfdcc45740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 0x55757f7f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ae530 con 0x55757f43e070 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ade60 con 0x55757f4151b0 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfdcc45740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01ab060 con 0x55757f41d390 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfdcc45740 10 client.0 _ll_register_callbacks cb 0x7fdfb8009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfcfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 0x55757f7f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfcf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 0x55757f7fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfcfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 0x55757f7f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53332/0 (socket says 172.21.15.32:53332) 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfcfe00640 1 -- 172.21.15.32:0/3278784257 learned_addr learned my addr 172.21.15.32:0/3278784257 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.112+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 0x55757f7f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfcfe00640 1 -- 172.21.15.32:0/3278784257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 msgr2=0x55757f7f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfcfe00640 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 0x55757f7f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfcfe00640 1 -- 172.21.15.32:0/3278784257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 msgr2=0x55757f7fc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfcfe00640 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 0x55757f7fc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfcfe00640 1 -- 172.21.15.32:0/3278784257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757f86f260 con 0x55757f43e070 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfcf400640 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 0x55757f7fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfc7e00640 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 0x55757f7f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfcfe00640 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 0x55757f7f7f40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8031680 tx=0x7fdfc80316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3f1256faa0a175 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3278784257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8003b70 con 0x55757f43e070 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3278784257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb8034070 con 0x55757f43e070 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.113+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3278784257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fdfc8003d10 con 0x55757f43e070 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3278784257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fdfc8004040 con 0x55757f43e070 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfdcc45740 10 client.19095 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3278784257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55757f870430 con 0x55757f43e070 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3278784257 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb8034070 con 0x55757f43e070 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3278784257 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fdfc8050030 con 0x55757f43e070 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfdcc45740 10 client.19095 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfdcc45740 10 client.19095 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfdcc45740 20 client.19095 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55757f3d1da0 0x55757f872bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfdcc45740 4 client.19095 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:31:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.114+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3278784257 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55757f7581a0 con 0x55757f3d1da0 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.115+0000 7fdfcf400640 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55757f3d1da0 0x55757f872bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.115+0000 7fdfcf400640 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55757f3d1da0 0x55757f872bf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21e2bba951925fe8 server_cookie=e4b2a62206ab288d in_seq=0 out_seq=0 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.115+0000 7fdfbcc00640 10 client.19095 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.115+0000 7fdfbcc00640 1 -- 172.21.15.32:0/3278784257 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55757f7581a0 con 0x55757f3d1da0 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.115+0000 7fdfbcc00640 10 client.19095 handle_command_reply: tid=0 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.115+0000 7fdfdcc45740 1 client.19095 shutdown 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3278784257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 msgr2=0x55757f7f7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 0x55757f7f7f40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8031680 tx=0x7fdfc80316b0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3278784257 shutdown_connections 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55757f3d1da0 0x55757f872bf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55757f43e070 0x55757f7f7f40 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55757f41d390 0x55757f7f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 1 --2- 172.21.15.32:0/3278784257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55757f4151b0 0x55757f7fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfbcc00640 0 client.19095 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:31:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3278784257 >> 172.21.15.32:0/3278784257 conn(0x55757f432460 msgr2=0x55757f7e5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3278784257 shutdown_connections 2024-08-31T07:31:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 1 -- 172.21.15.32:0/3278784257 wait complete. 2024-08-31T07:31:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.116+0000 7fdfdcc45740 20 client.19095 trim_cache size 0 max 16384 2024-08-31T07:31:23.174 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 -- 172.21.15.32:0/2634341418 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdfb40b4bb0 msgr2=0x7fdfb40b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdfb40b4bb0 0x7fdfb40b70c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdfb801b040 tx=0x7fdfb8009720 comp rx=0 tx=0).stop 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 -- 172.21.15.32:0/2634341418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfd0103750 msgr2=0x7fdfd019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfd0103750 0x7fdfd019a0a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000e9f0 tx=0x7fdfc000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 -- 172.21.15.32:0/2634341418 shutdown_connections 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fdfb40b4bb0 0x7fdfb40b70c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfd01099e0 0x7fdfd019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfd0109090 0x7fdfd019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 --2- 172.21.15.32:0/2634341418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfd0103750 0x7fdfd019a0a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 -- 172.21.15.32:0/2634341418 >> 172.21.15.32:0/2634341418 conn(0x7fdfd00ff1e0 msgr2=0x7fdfd0101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 -- 172.21.15.32:0/2634341418 shutdown_connections 2024-08-31T07:31:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:23.117+0000 7fdfd4a00640 1 -- 172.21.15.32:0/2634341418 wait complete. 2024-08-31T07:31:23.395 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.50 MB/sec execute 183 sec latency 27123.009 ms 2024-08-31T07:31:24.013 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:24.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:31:24.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.44 MB/sec execute 184 sec latency 28123.094 ms 2024-08-31T07:31:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:24 smithi073 ceph-mon[21995]: pgmap v953: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 153 KiB/s wr, 44 op/s 2024-08-31T07:31:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:24 smithi084 ceph-mon[22387]: pgmap v953: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 153 KiB/s wr, 44 op/s 2024-08-31T07:31:24.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:24 smithi032 ceph-mon[17252]: pgmap v953: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 153 KiB/s wr, 44 op/s 2024-08-31T07:31:25.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.37 MB/sec execute 185 sec latency 29123.223 ms 2024-08-31T07:31:25.758 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.779+0000 7fcf62e00640 1 -- 172.21.15.32:0/1596616414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5c103920 msgr2=0x7fcf5c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.779+0000 7fcf62e00640 1 --2- 172.21.15.32:0/1596616414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5c103920 0x7fcf5c103d30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009900 tx=0x7fcf4c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:31:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.779+0000 7fcf62e00640 1 -- 172.21.15.32:0/1596616414 shutdown_connections 2024-08-31T07:31:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.779+0000 7fcf62e00640 1 --2- 172.21.15.32:0/1596616414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c109bb0 0x7fcf5c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.779+0000 7fcf62e00640 1 --2- 172.21.15.32:0/1596616414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c109260 0x7fcf5c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.779+0000 7fcf62e00640 1 --2- 172.21.15.32:0/1596616414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5c103920 0x7fcf5c103d30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.779+0000 7fcf62e00640 1 -- 172.21.15.32:0/1596616414 >> 172.21.15.32:0/1596616414 conn(0x7fcf5c0ff330 msgr2=0x7fcf5c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.779+0000 7fcf62e00640 1 -- 172.21.15.32:0/1596616414 shutdown_connections 2024-08-31T07:31:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.780+0000 7fcf62e00640 1 -- 172.21.15.32:0/1596616414 wait complete. 2024-08-31T07:31:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.780+0000 7fcf62e00640 1 Processor -- start 2024-08-31T07:31:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.781+0000 7fcf62e00640 1 -- start start 2024-08-31T07:31:25.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.781+0000 7fcf62e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c103920 0x7fcf5c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:25.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.781+0000 7fcf62e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5c109260 0x7fcf5c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:25.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.781+0000 7fcf62e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c109bb0 0x7fcf5c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:25.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.781+0000 7fcf62e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1138c0 con 0x7fcf5c109260 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.781+0000 7fcf62e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c113740 con 0x7fcf5c109bb0 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.781+0000 7fcf62e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c113a40 con 0x7fcf5c103920 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.781+0000 7fcf5b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5c109260 0x7fcf5c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.781+0000 7fcf5b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5c109260 0x7fcf5c0759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53354/0 (socket says 172.21.15.32:53354) 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.781+0000 7fcf5b400640 1 -- 172.21.15.32:0/261779688 learned_addr learned my addr 172.21.15.32:0/261779688 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.782+0000 7fcf5be00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c103920 0x7fcf5c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.782+0000 7fcf60e00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c109bb0 0x7fcf5c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.782+0000 7fcf5b400640 1 -- 172.21.15.32:0/261779688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c103920 msgr2=0x7fcf5c0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.782+0000 7fcf5b400640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c103920 0x7fcf5c0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.782+0000 7fcf5b400640 1 -- 172.21.15.32:0/261779688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c109bb0 msgr2=0x7fcf5c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.782+0000 7fcf5b400640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c109bb0 0x7fcf5c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.782+0000 7fcf5b400640 1 -- 172.21.15.32:0/261779688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c1a8550 con 0x7fcf5c109260 2024-08-31T07:31:25.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.782+0000 7fcf60e00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c109bb0 0x7fcf5c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:25.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.782+0000 7fcf5b400640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5c109260 0x7fcf5c0759f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800e9f0 tx=0x7fcf4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:25.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.783+0000 7fcf58c00640 1 -- 172.21.15.32:0/261779688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4800ccc0 con 0x7fcf5c109260 2024-08-31T07:31:25.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.783+0000 7fcf62e00640 1 -- 172.21.15.32:0/261779688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5c1a8840 con 0x7fcf5c109260 2024-08-31T07:31:25.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.783+0000 7fcf58c00640 1 -- 172.21.15.32:0/261779688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf480108e0 con 0x7fcf5c109260 2024-08-31T07:31:25.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.783+0000 7fcf58c00640 1 -- 172.21.15.32:0/261779688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48018860 con 0x7fcf5c109260 2024-08-31T07:31:25.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.783+0000 7fcf62e00640 1 -- 172.21.15.32:0/261779688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5c1a8d50 con 0x7fcf5c109260 2024-08-31T07:31:25.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.784+0000 7fcf62e00640 1 -- 172.21.15.32:0/261779688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf5c103d30 con 0x7fcf5c109260 2024-08-31T07:31:25.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.787+0000 7fcf58c00640 1 -- 172.21.15.32:0/261779688 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcf48010430 con 0x7fcf5c109260 2024-08-31T07:31:25.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.790+0000 7fcf58c00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf400b4ca0 0x7fcf400b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:25.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.790+0000 7fcf58c00640 1 -- 172.21.15.32:0/261779688 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fcf480e57c0 con 0x7fcf5c109260 2024-08-31T07:31:25.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.790+0000 7fcf5be00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf400b4ca0 0x7fcf400b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:25.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.791+0000 7fcf5be00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf400b4ca0 0x7fcf400b71b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005ab0 tx=0x7fcf4c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:25.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:25.794+0000 7fcf58c00640 1 -- 172.21.15.32:0/261779688 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcf480abfb0 con 0x7fcf5c109260 2024-08-31T07:31:26.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.031+0000 7fcf62e00640 1 -- 172.21.15.32:0/261779688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf5c02e430 con 0x7fcf5c109260 2024-08-31T07:31:26.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.032+0000 7fcf58c00640 1 -- 172.21.15.32:0/261779688 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+11963 (secure 0 0 0) 0x7fcf480b0770 con 0x7fcf5c109260 2024-08-31T07:31:26.033 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:31:26.033 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"btime":"2024-08-31T07:30:07:717453+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":67,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:30:07.599620+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:31:26.033 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 67 2024-08-31T07:31:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 -- 172.21.15.32:0/261779688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf400b4ca0 msgr2=0x7fcf400b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf400b4ca0 0x7fcf400b71b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005ab0 tx=0x7fcf4c005a20 comp rx=0 tx=0).stop 2024-08-31T07:31:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 -- 172.21.15.32:0/261779688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5c109260 msgr2=0x7fcf5c0759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5c109260 0x7fcf5c0759f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800e9f0 tx=0x7fcf4800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 -- 172.21.15.32:0/261779688 shutdown_connections 2024-08-31T07:31:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcf400b4ca0 0x7fcf400b71b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c109bb0 0x7fcf5c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5c109260 0x7fcf5c0759f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 --2- 172.21.15.32:0/261779688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf5c103920 0x7fcf5c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 -- 172.21.15.32:0/261779688 >> 172.21.15.32:0/261779688 conn(0x7fcf5c0ff330 msgr2=0x7fcf5c1027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 -- 172.21.15.32:0/261779688 shutdown_connections 2024-08-31T07:31:26.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.034+0000 7fcf62e00640 1 -- 172.21.15.32:0/261779688 wait complete. 2024-08-31T07:31:26.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.30 MB/sec execute 186 sec latency 30123.304 ms 2024-08-31T07:31:26.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.564+0000 7ff05e400640 1 -- 172.21.15.32:0/3817227594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058109090 msgr2=0x7ff0581094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.564+0000 7ff05e400640 1 --2- 172.21.15.32:0/3817227594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058109090 0x7ff0581094a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff044009900 tx=0x7ff04402a410 comp rx=0 tx=0).stop 2024-08-31T07:31:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.564+0000 7ff05e400640 1 -- 172.21.15.32:0/3817227594 shutdown_connections 2024-08-31T07:31:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.564+0000 7ff05e400640 1 --2- 172.21.15.32:0/3817227594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0581099e0 0x7ff058110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.564+0000 7ff05e400640 1 --2- 172.21.15.32:0/3817227594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058109090 0x7ff0581094a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.564+0000 7ff05e400640 1 --2- 172.21.15.32:0/3817227594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff058103750 0x7ff058103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.564+0000 7ff05e400640 1 -- 172.21.15.32:0/3817227594 >> 172.21.15.32:0/3817227594 conn(0x7ff0580ff1e0 msgr2=0x7ff058101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.565+0000 7ff05e400640 1 -- 172.21.15.32:0/3817227594 shutdown_connections 2024-08-31T07:31:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.565+0000 7ff05e400640 1 -- 172.21.15.32:0/3817227594 wait complete. 2024-08-31T07:31:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.566+0000 7ff05e400640 1 Processor -- start 2024-08-31T07:31:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.566+0000 7ff05e400640 1 -- start start 2024-08-31T07:31:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.566+0000 7ff05e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058103750 0x7ff0580754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.566+0000 7ff05e400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff058109090 0x7ff0580759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.566+0000 7ff05e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0581099e0 0x7ff058075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.566+0000 7ff05e400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0581136c0 con 0x7ff058103750 2024-08-31T07:31:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.566+0000 7ff05e400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff058113540 con 0x7ff0581099e0 2024-08-31T07:31:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.566+0000 7ff05e400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff058113840 con 0x7ff058109090 2024-08-31T07:31:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff05d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058103750 0x7ff0580754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff05d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058103750 0x7ff0580754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37772/0 (socket says 172.21.15.32:37772) 2024-08-31T07:31:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff05d000640 1 -- 172.21.15.32:0/2123598528 learned_addr learned my addr 172.21.15.32:0/2123598528 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff057e00640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff058109090 0x7ff0580759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff05da00640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0581099e0 0x7ff058075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff05d000640 1 -- 172.21.15.32:0/2123598528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff058109090 msgr2=0x7ff0580759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff05d000640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff058109090 0x7ff0580759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff05d000640 1 -- 172.21.15.32:0/2123598528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0581099e0 msgr2=0x7ff058075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff05d000640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0581099e0 0x7ff058075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff05d000640 1 -- 172.21.15.32:0/2123598528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0581a83d0 con 0x7ff058103750 2024-08-31T07:31:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.567+0000 7ff05da00640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0581099e0 0x7ff058075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.568+0000 7ff05d000640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058103750 0x7ff0580754b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00eb90 tx=0x7ff04c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:26.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.568+0000 7ff055600640 1 -- 172.21.15.32:0/2123598528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c004d20 con 0x7ff058103750 2024-08-31T07:31:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.568+0000 7ff055600640 1 -- 172.21.15.32:0/2123598528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04c00ee90 con 0x7ff058103750 2024-08-31T07:31:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.568+0000 7ff055600640 1 -- 172.21.15.32:0/2123598528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c010670 con 0x7ff058103750 2024-08-31T07:31:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.568+0000 7ff05e400640 1 -- 172.21.15.32:0/2123598528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0581a86c0 con 0x7ff058103750 2024-08-31T07:31:26.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.568+0000 7ff05e400640 1 -- 172.21.15.32:0/2123598528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0581a8b80 con 0x7ff058103750 2024-08-31T07:31:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.571+0000 7ff0666a8740 1 Processor -- start 2024-08-31T07:31:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.571+0000 7ff0666a8740 1 -- start start 2024-08-31T07:31:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 0x56164abf2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a8241b0 0x56164abf30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164abf35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff0666a8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56164ab697b0 con 0x56164a82c390 2024-08-31T07:31:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff0666a8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56164ab58d40 con 0x56164a8241b0 2024-08-31T07:31:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff0666a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56164ab5ccf0 con 0x56164a84d070 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff057e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164abf35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff05da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 0x56164abf2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff057e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164abf35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37782/0 (socket says 172.21.15.32:37782) 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff057e00640 1 -- 172.21.15.32:0/512546105 learned_addr learned my addr 172.21.15.32:0/512546105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.572+0000 7ff055600640 1 -- 172.21.15.32:0/2123598528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff04c005020 con 0x7ff058103750 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.573+0000 7ff05da00640 1 -- 172.21.15.32:0/512546105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a8241b0 msgr2=0x56164abf30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.573+0000 7ff05da00640 1 --2- 172.21.15.32:0/512546105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a8241b0 0x56164abf30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.573+0000 7ff05da00640 1 -- 172.21.15.32:0/512546105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 msgr2=0x56164abf35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.573+0000 7ff05da00640 1 --2- 172.21.15.32:0/512546105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164abf35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.573+0000 7ff05da00640 1 -- 172.21.15.32:0/512546105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164a8b8650 con 0x56164a84d070 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.573+0000 7ff057e00640 1 --2- 172.21.15.32:0/512546105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164abf35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.573+0000 7ff05da00640 1 --2- 172.21.15.32:0/512546105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 0x56164abf2b60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff050009f50 tx=0x7ff0500118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=566dec1e3b9b3b92 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.573+0000 7ff04a000640 1 -- 172.21.15.32:0/512546105 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05001e070 con 0x56164a84d070 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.574+0000 7ff04a000640 1 -- 172.21.15.32:0/512546105 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff050004310 con 0x56164a84d070 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.574+0000 7ff04a000640 1 -- 172.21.15.32:0/512546105 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05000f840 con 0x56164a84d070 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.574+0000 7ff055600640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0400b4d20 0x7ff0400b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.574+0000 7ff0666a8740 1 -- 172.21.15.32:0/512546105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 msgr2=0x56164abf2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.574+0000 7ff0666a8740 1 --2- 172.21.15.32:0/512546105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 0x56164abf2b60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff050009f50 tx=0x7ff0500118d0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.574+0000 7ff055600640 1 -- 172.21.15.32:0/2123598528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7ff04c0e5ac0 con 0x7ff058103750 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.575+0000 7ff05d000640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0400b4d20 0x7ff0400b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.575+0000 7ff0666a8740 1 -- 172.21.15.32:0/512546105 shutdown_connections 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.575+0000 7ff0666a8740 1 --2- 172.21.15.32:0/512546105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164abf35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.575+0000 7ff0666a8740 1 --2- 172.21.15.32:0/512546105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a8241b0 0x56164abf30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.575+0000 7ff0666a8740 1 --2- 172.21.15.32:0/512546105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 0x56164abf2b60 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.575+0000 7ff0666a8740 1 -- 172.21.15.32:0/512546105 >> 172.21.15.32:0/512546105 conn(0x56164a841460 msgr2=0x56164abf0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.575+0000 7ff0666a8740 1 -- 172.21.15.32:0/512546105 shutdown_connections 2024-08-31T07:31:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.575+0000 7ff0666a8740 1 -- 172.21.15.32:0/512546105 wait complete. 2024-08-31T07:31:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.575+0000 7ff05d000640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0400b4d20 0x7ff0400b7230 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x56164abf17e0 tx=0x7ff04c009200 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.575+0000 7ff0666a8740 1 -- start start 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.576+0000 7ff0666a8740 1 Processor -- start 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.576+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 0x56164abfe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.576+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a82c390 0x56164abfebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.576+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a84d070 0x56164ac0a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.576+0000 7ff0666a8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56164a9bb700 con 0x56164a8241b0 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.576+0000 7ff0666a8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56164aadfcf0 con 0x56164a84d070 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.576+0000 7ff0666a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56164ac01340 con 0x56164a82c390 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.576+0000 7ff0666a8740 10 client.0 _ll_register_callbacks cb 0x56164ab05470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff05da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 0x56164abfe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff057e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a82c390 0x56164abfebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff057e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a82c390 0x56164abfebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42008/0 (socket says 172.21.15.32:42008) 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff057e00640 1 -- 172.21.15.32:0/2116270285 learned_addr learned my addr 172.21.15.32:0/2116270285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff05da00640 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a84d070 0x56164ac0a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff057e00640 1 -- 172.21.15.32:0/2116270285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a84d070 msgr2=0x56164ac0a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff057e00640 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a84d070 0x56164ac0a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff057e00640 1 -- 172.21.15.32:0/2116270285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 msgr2=0x56164abfe6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff057e00640 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 0x56164abfe6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff057e00640 1 -- 172.21.15.32:0/2116270285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164ab208b0 con 0x56164a82c390 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff057e00640 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a82c390 0x56164abfebe0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff04402aa00 tx=0x7ff0440043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf70a4fa7afeb178 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.577+0000 7ff03ea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.578+0000 7ff03ea00640 1 -- 172.21.15.32:0/2116270285 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044035040 con 0x56164a82c390 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.578+0000 7ff03ea00640 1 -- 172.21.15.32:0/2116270285 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7ff044005450 con 0x56164a82c390 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.578+0000 7ff0666a8740 1 -- 172.21.15.32:0/2116270285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff050004310 con 0x56164a82c390 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.578+0000 7ff03ea00640 1 -- 172.21.15.32:0/2116270285 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff044005650 con 0x56164a82c390 2024-08-31T07:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.578+0000 7ff0666a8740 10 client.29152 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.578+0000 7ff0666a8740 1 -- 172.21.15.32:0/2116270285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x56164ac0d330 con 0x56164a82c390 2024-08-31T07:31:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.579+0000 7ff03ea00640 1 -- 172.21.15.32:0/2116270285 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff050004310 con 0x56164a82c390 2024-08-31T07:31:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.579+0000 7ff03ea00640 1 -- 172.21.15.32:0/2116270285 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7ff044003b50 con 0x56164a82c390 2024-08-31T07:31:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.579+0000 7ff0666a8740 10 client.29152 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.579+0000 7ff0666a8740 10 client.29152 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:31:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.579+0000 7ff0666a8740 20 client.29152 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.579+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56164a80b9f0 0x56164ac107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.579+0000 7ff0666a8740 4 client.29152 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:31:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.579+0000 7ff0666a8740 1 -- 172.21.15.32:0/2116270285 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56164ab6b700 con 0x56164a80b9f0 2024-08-31T07:31:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.579+0000 7ff05da00640 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56164a80b9f0 0x56164ac107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.580+0000 7ff03ea00640 10 client.29152 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:31:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.580+0000 7ff05da00640 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56164a80b9f0 0x56164ac107b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d1c6ecb36eaeb4f server_cookie=56399f87e94c145b in_seq=0 out_seq=0 2024-08-31T07:31:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.581+0000 7ff03ea00640 1 -- 172.21.15.32:0/2116270285 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56164ab6b700 con 0x56164a80b9f0 2024-08-31T07:31:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.581+0000 7ff03ea00640 10 client.29152 handle_command_reply: tid=0 2024-08-31T07:31:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.581+0000 7ff0666a8740 1 client.29152 shutdown 2024-08-31T07:31:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 1 -- 172.21.15.32:0/2116270285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a82c390 msgr2=0x56164abfebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a82c390 0x56164abfebe0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff04402aa00 tx=0x7ff0440043b0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 1 -- 172.21.15.32:0/2116270285 shutdown_connections 2024-08-31T07:31:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56164a80b9f0 0x56164ac107b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a84d070 0x56164ac0a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a82c390 0x56164abfebe0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff03ea00640 0 client.29152 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:31:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2116270285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 0x56164abfe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 1 -- 172.21.15.32:0/2116270285 >> 172.21.15.32:0/2116270285 conn(0x56164a841460 msgr2=0x56164abf5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 1 -- 172.21.15.32:0/2116270285 shutdown_connections 2024-08-31T07:31:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 1 -- 172.21.15.32:0/2116270285 wait complete. 2024-08-31T07:31:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.582+0000 7ff0666a8740 20 client.29152 trim_cache size 0 max 16384 2024-08-31T07:31:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.596+0000 7ff0666a8740 1 Processor -- start 2024-08-31T07:31:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.596+0000 7ff0666a8740 1 -- start start 2024-08-31T07:31:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.596+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 0x56164abf6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.596+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a82c390 0x56164abf42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.596+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 0x56164ac0a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.596+0000 7ff0666a8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0581b1410 con 0x56164a8241b0 2024-08-31T07:31:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.596+0000 7ff0666a8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff0581b2fb0 con 0x56164a82c390 2024-08-31T07:31:26.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.596+0000 7ff0666a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0581b1b10 con 0x56164a84d070 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff05da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 0x56164abf6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff05da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 0x56164abf6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37792/0 (socket says 172.21.15.32:37792) 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff05da00640 1 -- 172.21.15.32:0/558265309 learned_addr learned my addr 172.21.15.32:0/558265309 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff057e00640 1 --2- 172.21.15.32:0/558265309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a82c390 0x56164abf42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff05da00640 1 --2- 172.21.15.32:0/558265309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 0x56164ac0a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff057e00640 1 -- 172.21.15.32:0/558265309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 msgr2=0x56164ac0a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff057e00640 1 --2- 172.21.15.32:0/558265309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 0x56164ac0a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff057e00640 1 -- 172.21.15.32:0/558265309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 msgr2=0x56164abf6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff057e00640 1 --2- 172.21.15.32:0/558265309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 0x56164abf6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff057e00640 1 -- 172.21.15.32:0/558265309 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164abf4e80 con 0x56164a82c390 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.597+0000 7ff05da00640 1 --2- 172.21.15.32:0/558265309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 0x56164abf6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.598+0000 7ff057e00640 1 --2- 172.21.15.32:0/558265309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a82c390 0x56164abf42a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff04402a930 tx=0x7ff04402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a6384680d86e160 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.598+0000 7ff049600640 1 -- 172.21.15.32:0/558265309 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044035070 con 0x56164a82c390 2024-08-31T07:31:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.598+0000 7ff049600640 1 -- 172.21.15.32:0/558265309 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff044041070 con 0x56164a82c390 2024-08-31T07:31:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.598+0000 7ff049600640 1 -- 172.21.15.32:0/558265309 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044003b30 con 0x56164a82c390 2024-08-31T07:31:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.599+0000 7ff0666a8740 1 -- 172.21.15.32:0/558265309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a82c390 msgr2=0x56164abf42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.599+0000 7ff0666a8740 1 --2- 172.21.15.32:0/558265309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a82c390 0x56164abf42a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff04402a930 tx=0x7ff04402a410 comp rx=0 tx=0).stop 2024-08-31T07:31:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.599+0000 7ff0666a8740 1 -- 172.21.15.32:0/558265309 shutdown_connections 2024-08-31T07:31:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.599+0000 7ff0666a8740 1 --2- 172.21.15.32:0/558265309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a84d070 0x56164ac0a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.599+0000 7ff0666a8740 1 --2- 172.21.15.32:0/558265309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a82c390 0x56164abf42a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.599+0000 7ff0666a8740 1 --2- 172.21.15.32:0/558265309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a8241b0 0x56164abf6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.599+0000 7ff0666a8740 1 -- 172.21.15.32:0/558265309 >> 172.21.15.32:0/558265309 conn(0x56164a841460 msgr2=0x56164abf3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.599+0000 7ff0666a8740 1 -- 172.21.15.32:0/558265309 shutdown_connections 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.599+0000 7ff0666a8740 1 -- 172.21.15.32:0/558265309 wait complete. 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.599+0000 7ff0666a8740 1 -- start start 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff0666a8740 1 Processor -- start 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a8241b0 0x56164ac06ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164ac03390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff0666a8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a84d070 0x56164ac039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff0666a8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0581b28e0 con 0x56164a82c390 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff0666a8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff0581b2210 con 0x56164a84d070 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff0666a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0581af410 con 0x56164a8241b0 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff0666a8740 10 client.0 _ll_register_callbacks cb 0x7ff044004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff057e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164ac03390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff057e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164ac03390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37802/0 (socket says 172.21.15.32:37802) 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff057e00640 1 -- 172.21.15.32:0/2343567460 learned_addr learned my addr 172.21.15.32:0/2343567460 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.600+0000 7ff05da00640 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a84d070 0x56164ac039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.601+0000 7ff05da00640 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a8241b0 0x56164ac06ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.601+0000 7ff057e00640 1 -- 172.21.15.32:0/2343567460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a8241b0 msgr2=0x56164ac06ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.601+0000 7ff057e00640 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a8241b0 0x56164ac06ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.601+0000 7ff057e00640 1 -- 172.21.15.32:0/2343567460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a84d070 msgr2=0x56164ac039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.601+0000 7ff057e00640 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a84d070 0x56164ac039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.601+0000 7ff057e00640 1 -- 172.21.15.32:0/2343567460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164ac04150 con 0x56164a82c390 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.601+0000 7ff05da00640 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a8241b0 0x56164ac06ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.601+0000 7ff057e00640 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164ac03390 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff044000c00 tx=0x7ff044005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a93b53984709519 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.601+0000 7ff03ea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:31:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff03ea00640 1 -- 172.21.15.32:0/2343567460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044035070 con 0x56164a82c390 2024-08-31T07:31:26.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff03ea00640 1 -- 172.21.15.32:0/2343567460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7ff044003cc0 con 0x56164a82c390 2024-08-31T07:31:26.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff0666a8740 1 -- 172.21.15.32:0/2343567460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff044041070 con 0x56164a82c390 2024-08-31T07:31:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff03ea00640 1 -- 172.21.15.32:0/2343567460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff0440318c0 con 0x56164a82c390 2024-08-31T07:31:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff0666a8740 10 client.19119 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff0666a8740 1 -- 172.21.15.32:0/2343567460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x56164ac7c1c0 con 0x56164a82c390 2024-08-31T07:31:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff03ea00640 1 -- 172.21.15.32:0/2343567460 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff044041070 con 0x56164a82c390 2024-08-31T07:31:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff03ea00640 1 -- 172.21.15.32:0/2343567460 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7ff044046470 con 0x56164a82c390 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff0666a8740 10 client.19119 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff0666a8740 10 client.19119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.602+0000 7ff0666a8740 20 client.19119 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.603+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56164a7e0da0 0x56164ac7e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.603+0000 7ff0666a8740 4 client.19119 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.603+0000 7ff0666a8740 1 -- 172.21.15.32:0/2343567460 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56164abf0fb0 con 0x56164a7e0da0 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.603+0000 7ff05da00640 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56164a7e0da0 0x56164ac7e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.603+0000 7ff05da00640 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56164a7e0da0 0x56164ac7e860 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf693ffc302d8605 server_cookie=1ad5e672b6c1d36c in_seq=0 out_seq=0 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.603+0000 7ff03ea00640 10 client.19119 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff03ea00640 1 -- 172.21.15.32:0/2343567460 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56164abf0fb0 con 0x56164a7e0da0 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff03ea00640 10 client.19119 handle_command_reply: tid=0 2024-08-31T07:31:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 client.19119 shutdown 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 -- 172.21.15.32:0/2343567460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 msgr2=0x56164ac03390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164ac03390 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff044000c00 tx=0x7ff044005f30 comp rx=0 tx=0).stop 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 -- 172.21.15.32:0/2343567460 shutdown_connections 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56164a7e0da0 0x56164ac7e860 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56164a84d070 0x56164ac039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56164a82c390 0x56164ac03390 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 --2- 172.21.15.32:0/2343567460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56164a8241b0 0x56164ac06ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 -- 172.21.15.32:0/2343567460 >> 172.21.15.32:0/2343567460 conn(0x56164a841460 msgr2=0x56164abf63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff03ea00640 0 client.19119 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 -- 172.21.15.32:0/2343567460 shutdown_connections 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 1 -- 172.21.15.32:0/2343567460 wait complete. 2024-08-31T07:31:26.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.604+0000 7ff0666a8740 20 client.19119 trim_cache size 0 max 16384 2024-08-31T07:31:26.609 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:31:26.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 -- 172.21.15.32:0/2123598528 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0400b4d20 msgr2=0x7ff0400b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0400b4d20 0x7ff0400b7230 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x56164abf17e0 tx=0x7ff04c009200 comp rx=0 tx=0).stop 2024-08-31T07:31:26.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 -- 172.21.15.32:0/2123598528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058103750 msgr2=0x7ff0580754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:26.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058103750 0x7ff0580754b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00eb90 tx=0x7ff04c00c6a0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 -- 172.21.15.32:0/2123598528 shutdown_connections 2024-08-31T07:31:26.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff0400b4d20 0x7ff0400b7230 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0581099e0 0x7ff058075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff058109090 0x7ff0580759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 --2- 172.21.15.32:0/2123598528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff058103750 0x7ff0580754b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:26.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 -- 172.21.15.32:0/2123598528 >> 172.21.15.32:0/2123598528 conn(0x7ff0580ff1e0 msgr2=0x7ff058101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:26.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.605+0000 7ff05e400640 1 -- 172.21.15.32:0/2123598528 shutdown_connections 2024-08-31T07:31:26.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:26.606+0000 7ff05e400640 1 -- 172.21.15.32:0/2123598528 wait complete. 2024-08-31T07:31:26.643 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:26 smithi032 ceph-mon[17252]: pgmap v954: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 162 KiB/s wr, 39 op/s 2024-08-31T07:31:26.644 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:26 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/261779688' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:31:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:26 smithi073 ceph-mon[21995]: pgmap v954: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 162 KiB/s wr, 39 op/s 2024-08-31T07:31:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:26 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/261779688' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:31:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:26 smithi084 ceph-mon[22387]: pgmap v954: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 162 KiB/s wr, 39 op/s 2024-08-31T07:31:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:26 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/261779688' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:31:27.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.24 MB/sec execute 187 sec latency 31123.411 ms 2024-08-31T07:31:27.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:31:28.001 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id fd6c69c8 --release --await' 2024-08-31T07:31:28.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id fd6c69c8 --release --await 2024-08-31T07:31:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:28.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.17 MB/sec execute 188 sec latency 32123.489 ms 2024-08-31T07:31:28.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:28 smithi032 ceph-mon[17252]: pgmap v955: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 136 KiB/s wr, 31 op/s 2024-08-31T07:31:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:28 smithi073 ceph-mon[21995]: pgmap v955: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 136 KiB/s wr, 31 op/s 2024-08-31T07:31:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:28 smithi084 ceph-mon[22387]: pgmap v955: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 136 KiB/s wr, 31 op/s 2024-08-31T07:31:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.888+0000 7f9b1be00640 1 -- 172.21.15.32:0/2868813167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14075e40 msgr2=0x7f9b1410f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.888+0000 7f9b1be00640 1 --2- 172.21.15.32:0/2868813167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14075e40 0x7f9b1410f9b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009900 tx=0x7f9b0002a400 comp rx=0 tx=0).stop 2024-08-31T07:31:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.888+0000 7f9b1be00640 1 -- 172.21.15.32:0/2868813167 shutdown_connections 2024-08-31T07:31:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.888+0000 7f9b1be00640 1 --2- 172.21.15.32:0/2868813167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b1410fef0 0x7f9b14112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.888+0000 7f9b1be00640 1 --2- 172.21.15.32:0/2868813167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14075e40 0x7f9b1410f9b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.888+0000 7f9b1be00640 1 --2- 172.21.15.32:0/2868813167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b140754f0 0x7f9b14075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.888+0000 7f9b1be00640 1 -- 172.21.15.32:0/2868813167 >> 172.21.15.32:0/2868813167 conn(0x7f9b140ff380 msgr2=0x7f9b141017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:28.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.888+0000 7f9b1be00640 1 -- 172.21.15.32:0/2868813167 shutdown_connections 2024-08-31T07:31:28.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.888+0000 7f9b1be00640 1 -- 172.21.15.32:0/2868813167 wait complete. 2024-08-31T07:31:28.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.889+0000 7f9b1be00640 1 Processor -- start 2024-08-31T07:31:28.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.889+0000 7f9b1be00640 1 -- start start 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b1be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b140754f0 0x7f9b1419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b1be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b14075e40 0x7f9b1419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b1be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1410fef0 0x7f9b141a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b1be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b14115ab0 con 0x7f9b1410fef0 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b1be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9b14115930 con 0x7f9b140754f0 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b1be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b14115c30 con 0x7f9b14075e40 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b18a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b14075e40 0x7f9b1419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b19400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b140754f0 0x7f9b1419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b19e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1410fef0 0x7f9b141a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b19400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b140754f0 0x7f9b1419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57284/0 (socket says 172.21.15.32:57284) 2024-08-31T07:31:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.890+0000 7f9b19400640 1 -- 172.21.15.32:0/73677663 learned_addr learned my addr 172.21.15.32:0/73677663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.891+0000 7f9b19400640 1 -- 172.21.15.32:0/73677663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b14075e40 msgr2=0x7f9b1419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.891+0000 7f9b19400640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b14075e40 0x7f9b1419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.891+0000 7f9b19400640 1 -- 172.21.15.32:0/73677663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1410fef0 msgr2=0x7f9b141a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.891+0000 7f9b19400640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1410fef0 0x7f9b141a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.891+0000 7f9b19400640 1 -- 172.21.15.32:0/73677663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b141a23d0 con 0x7f9b140754f0 2024-08-31T07:31:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.891+0000 7f9b19e00640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1410fef0 0x7f9b141a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:28.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.891+0000 7f9b19400640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b140754f0 0x7f9b1419c460 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800e9f0 tx=0x7f9b0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:28.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.891+0000 7f9b12000640 1 -- 172.21.15.32:0/73677663 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0800cd80 con 0x7f9b140754f0 2024-08-31T07:31:28.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.891+0000 7f9b12000640 1 -- 172.21.15.32:0/73677663 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b080108e0 con 0x7f9b140754f0 2024-08-31T07:31:28.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.891+0000 7f9b12000640 1 -- 172.21.15.32:0/73677663 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08018980 con 0x7f9b140754f0 2024-08-31T07:31:28.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.892+0000 7f9b1be00640 1 -- 172.21.15.32:0/73677663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b141a26c0 con 0x7f9b140754f0 2024-08-31T07:31:28.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.892+0000 7f9b1be00640 1 -- 172.21.15.32:0/73677663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b141a8e20 con 0x7f9b140754f0 2024-08-31T07:31:28.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.895+0000 7f9b22be9740 1 Processor -- start 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.895+0000 7f9b22be9740 1 -- start start 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.895+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c122070 0x55ee0c4c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.895+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.895+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c4c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.895+0000 7f9b22be9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ee0c43e7b0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.895+0000 7f9b22be9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ee0c42dd40 con 0x55ee0c122070 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b22be9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ee0c431cf0 con 0x55ee0c101390 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b19e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c122070 0x55ee0c4c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b18a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c4c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b19e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c122070 0x55ee0c4c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57294/0 (socket says 172.21.15.32:57294) 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b18a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c4c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42058/0 (socket says 172.21.15.32:42058) 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b19e00640 1 -- 172.21.15.32:0/4075059376 learned_addr learned my addr 172.21.15.32:0/4075059376 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:28.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b12000640 1 -- 172.21.15.32:0/73677663 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9b08010430 con 0x7f9b140754f0 2024-08-31T07:31:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b18a00640 1 -- 172.21.15.32:0/4075059376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c122070 msgr2=0x55ee0c4c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:28.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b18a00640 1 --2- 172.21.15.32:0/4075059376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c122070 0x55ee0c4c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b18a00640 1 -- 172.21.15.32:0/4075059376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c0f91b0 msgr2=0x55ee0c4c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:28.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b18a00640 1 --2- 172.21.15.32:0/4075059376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.896+0000 7f9b18a00640 1 -- 172.21.15.32:0/4075059376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0c18d650 con 0x55ee0c101390 2024-08-31T07:31:28.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.897+0000 7f9b19e00640 1 --2- 172.21.15.32:0/4075059376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c122070 0x55ee0c4c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:28.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.897+0000 7f9b18a00640 1 --2- 172.21.15.32:0/4075059376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c4c8330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009230 tx=0x7f9b00004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9d51477213e1c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:28.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.897+0000 7f9b06600640 1 -- 172.21.15.32:0/4075059376 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00035040 con 0x55ee0c101390 2024-08-31T07:31:28.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.897+0000 7f9b06600640 1 -- 172.21.15.32:0/4075059376 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b000054f0 con 0x55ee0c101390 2024-08-31T07:31:28.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.897+0000 7f9b06600640 1 -- 172.21.15.32:0/4075059376 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000035f0 con 0x55ee0c101390 2024-08-31T07:31:28.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.898+0000 7f9b12000640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9afc0b4ca0 0x7f9afc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b12000640 1 -- 172.21.15.32:0/73677663 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f9b080e5860 con 0x7f9b140754f0 2024-08-31T07:31:28.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b22be9740 1 -- 172.21.15.32:0/4075059376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 msgr2=0x55ee0c4c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:28.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b22be9740 1 --2- 172.21.15.32:0/4075059376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c4c8330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009230 tx=0x7f9b00004040 comp rx=0 tx=0).stop 2024-08-31T07:31:28.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b19e00640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9afc0b4ca0 0x7f9afc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b22be9740 1 -- 172.21.15.32:0/4075059376 shutdown_connections 2024-08-31T07:31:28.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b22be9740 1 --2- 172.21.15.32:0/4075059376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c4c8330 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b22be9740 1 --2- 172.21.15.32:0/4075059376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b22be9740 1 --2- 172.21.15.32:0/4075059376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c122070 0x55ee0c4c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b22be9740 1 -- 172.21.15.32:0/4075059376 >> 172.21.15.32:0/4075059376 conn(0x55ee0c116460 msgr2=0x55ee0c4c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:28.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b22be9740 1 -- 172.21.15.32:0/4075059376 shutdown_connections 2024-08-31T07:31:28.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.899+0000 7f9b22be9740 1 -- 172.21.15.32:0/4075059376 wait complete. 2024-08-31T07:31:28.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.900+0000 7f9b19e00640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9afc0b4ca0 0x7f9afc0b71b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009fd0 tx=0x7f9b0c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:28.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.900+0000 7f9b22be9740 1 -- start start 2024-08-31T07:31:28.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.900+0000 7f9b22be9740 1 Processor -- start 2024-08-31T07:31:28.902 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 0x55ee0c4d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c122070 0x55ee0c4d9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b22be9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ee0c290700 con 0x55ee0c101390 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b22be9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ee0c3b4cf0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b22be9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ee0c4cdf00 con 0x55ee0c122070 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b22be9740 10 client.0 _ll_register_callbacks cb 0x55ee0c3da470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b19400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b19400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4d3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57302/0 (socket says 172.21.15.32:57302) 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b19400640 1 -- 172.21.15.32:0/3745103611 learned_addr learned my addr 172.21.15.32:0/3745103611 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b19e00640 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c122070 0x55ee0c4d9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.901+0000 7f9b18a00640 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 0x55ee0c4d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.902+0000 7f9b19400640 1 -- 172.21.15.32:0/3745103611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c122070 msgr2=0x55ee0c4d9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.902+0000 7f9b19400640 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c122070 0x55ee0c4d9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.902+0000 7f9b19400640 1 -- 172.21.15.32:0/3745103611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 msgr2=0x55ee0c4d40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.902+0000 7f9b19400640 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 0x55ee0c4d40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.902+0000 7f9b19400640 1 -- 172.21.15.32:0/3745103611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0c3f58b0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.902+0000 7f9b19400640 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4d3b00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800e9c0 tx=0x7f9b080107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13a534cab7c5797c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.902+0000 7f9afaa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.902+0000 7f9afaa00640 1 -- 172.21.15.32:0/3745103611 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080e3430 con 0x55ee0c0f91b0 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.902+0000 7f9b22be9740 1 -- 172.21.15.32:0/3745103611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b000054f0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.902+0000 7f9afaa00640 1 -- 172.21.15.32:0/3745103611 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f9b080b0410 con 0x55ee0c0f91b0 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9afaa00640 1 -- 172.21.15.32:0/3745103611 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f9b080b07b0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9b22be9740 10 client.29174 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9b22be9740 1 -- 172.21.15.32:0/3745103611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55ee0c4f3100 con 0x55ee0c0f91b0 2024-08-31T07:31:28.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9afaa00640 1 -- 172.21.15.32:0/3745103611 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b000054f0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9afaa00640 1 -- 172.21.15.32:0/3745103611 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f9b08018b20 con 0x55ee0c0f91b0 2024-08-31T07:31:28.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9b22be9740 10 client.29174 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:28.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9b22be9740 10 client.29174 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:31:28.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9b22be9740 20 client.29174 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:28.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9b22be9740 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ee0c0e09f0 0x55ee0c4f6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9b22be9740 4 client.29174 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:31:28.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.903+0000 7f9b22be9740 1 -- 172.21.15.32:0/3745103611 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee0c440700 con 0x55ee0c0e09f0 2024-08-31T07:31:28.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.904+0000 7f9b18a00640 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ee0c0e09f0 0x55ee0c4f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.904+0000 7f9b18a00640 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ee0c0e09f0 0x55ee0c4f6570 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=842b6f096e6206b7 server_cookie=157300140c48996f in_seq=0 out_seq=0 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.904+0000 7f9afaa00640 10 client.29174 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.905+0000 7f9afaa00640 1 -- 172.21.15.32:0/3745103611 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee0c440700 con 0x55ee0c0e09f0 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.905+0000 7f9afaa00640 10 client.29174 handle_command_reply: tid=0 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.905+0000 7f9b22be9740 1 client.29174 shutdown 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 1 -- 172.21.15.32:0/3745103611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 msgr2=0x55ee0c4d3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4d3b00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800e9c0 tx=0x7f9b080107e0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 1 -- 172.21.15.32:0/3745103611 shutdown_connections 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c122070 0x55ee0c4d9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 0x55ee0c4d40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ee0c0e09f0 0x55ee0c4f6570 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 1 --2- 172.21.15.32:0/3745103611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4d3b00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 1 -- 172.21.15.32:0/3745103611 >> 172.21.15.32:0/3745103611 conn(0x55ee0c116460 msgr2=0x55ee0c4c3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9afaa00640 0 client.29174 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:31:28.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 1 -- 172.21.15.32:0/3745103611 shutdown_connections 2024-08-31T07:31:28.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 1 -- 172.21.15.32:0/3745103611 wait complete. 2024-08-31T07:31:28.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.906+0000 7f9b22be9740 20 client.29174 trim_cache size 0 max 16384 2024-08-31T07:31:28.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.920+0000 7f9b22be9740 1 Processor -- start 2024-08-31T07:31:28.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.920+0000 7f9b22be9740 1 -- start start 2024-08-31T07:31:28.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 0x55ee0c4c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c122070 0x55ee0c4c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b22be9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b141ad1c0 con 0x55ee0c101390 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b22be9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9b141aed60 con 0x55ee0c0f91b0 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b22be9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b141ad8c0 con 0x55ee0c122070 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b18a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 0x55ee0c4c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b19400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b19400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57316/0 (socket says 172.21.15.32:57316) 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b18a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 0x55ee0c4c3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37848/0 (socket says 172.21.15.32:37848) 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b19400640 1 -- 172.21.15.32:0/712489649 learned_addr learned my addr 172.21.15.32:0/712489649 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.921+0000 7f9b19e00640 1 --2- 172.21.15.32:0/712489649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c122070 0x55ee0c4c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.922+0000 7f9b19400640 1 -- 172.21.15.32:0/712489649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c122070 msgr2=0x55ee0c4c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.922+0000 7f9b19400640 1 --2- 172.21.15.32:0/712489649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c122070 0x55ee0c4c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.922+0000 7f9b19400640 1 -- 172.21.15.32:0/712489649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 msgr2=0x55ee0c4c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:28.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.922+0000 7f9b19400640 1 --2- 172.21.15.32:0/712489649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 0x55ee0c4c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.922+0000 7f9b19400640 1 -- 172.21.15.32:0/712489649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0c4d9f20 con 0x55ee0c0f91b0 2024-08-31T07:31:28.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.922+0000 7f9b18a00640 1 --2- 172.21.15.32:0/712489649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 0x55ee0c4c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.922+0000 7f9b19400640 1 --2- 172.21.15.32:0/712489649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4cd080 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9b080a9260 tx=0x7f9b08000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b18bada5cc41fb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.922+0000 7f9b05c00640 1 -- 172.21.15.32:0/712489649 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080b5260 con 0x55ee0c0f91b0 2024-08-31T07:31:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.922+0000 7f9b05c00640 1 -- 172.21.15.32:0/712489649 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b080971f0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.922+0000 7f9b05c00640 1 -- 172.21.15.32:0/712489649 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080ab250 con 0x55ee0c0f91b0 2024-08-31T07:31:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.923+0000 7f9b22be9740 1 -- 172.21.15.32:0/712489649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 msgr2=0x55ee0c4cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.923+0000 7f9b22be9740 1 --2- 172.21.15.32:0/712489649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4cd080 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9b080a9260 tx=0x7f9b08000bc0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.923+0000 7f9b22be9740 1 -- 172.21.15.32:0/712489649 shutdown_connections 2024-08-31T07:31:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.923+0000 7f9b22be9740 1 --2- 172.21.15.32:0/712489649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c122070 0x55ee0c4c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.923+0000 7f9b22be9740 1 --2- 172.21.15.32:0/712489649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c101390 0x55ee0c4c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.923+0000 7f9b22be9740 1 --2- 172.21.15.32:0/712489649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c4cd080 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.923+0000 7f9b22be9740 1 -- 172.21.15.32:0/712489649 >> 172.21.15.32:0/712489649 conn(0x55ee0c116460 msgr2=0x55ee0c4cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:28.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.924+0000 7f9b22be9740 1 -- 172.21.15.32:0/712489649 shutdown_connections 2024-08-31T07:31:28.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.924+0000 7f9b22be9740 1 -- 172.21.15.32:0/712489649 wait complete. 2024-08-31T07:31:28.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.924+0000 7f9b22be9740 1 -- start start 2024-08-31T07:31:28.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.924+0000 7f9b22be9740 1 Processor -- start 2024-08-31T07:31:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c504560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c504b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b22be9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c122070 0x55ee0c559ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b22be9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b141ae690 con 0x55ee0c122070 2024-08-31T07:31:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b22be9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9b141adfc0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b22be9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b141ab1e0 con 0x55ee0c101390 2024-08-31T07:31:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b22be9740 10 client.0 _ll_register_callbacks cb 0x7f9b080ab760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b19400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c504560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b19400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c504560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57318/0 (socket says 172.21.15.32:57318) 2024-08-31T07:31:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b19400640 1 -- 172.21.15.32:0/4231405958 learned_addr learned my addr 172.21.15.32:0/4231405958 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b19e00640 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c122070 0x55ee0c559ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.925+0000 7f9b18a00640 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c504b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9b19400640 1 -- 172.21.15.32:0/4231405958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 msgr2=0x55ee0c504b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9b19400640 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c504b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9b19400640 1 -- 172.21.15.32:0/4231405958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c122070 msgr2=0x55ee0c559ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9b19400640 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c122070 0x55ee0c559ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9b19400640 1 -- 172.21.15.32:0/4231405958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0c55a550 con 0x55ee0c0f91b0 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9b18a00640 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c504b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9b19e00640 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c122070 0x55ee0c559ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9b19400640 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c504560 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9b080097b0 tx=0x7f9b080107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12007a17729da569 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9afaa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9afaa00640 1 -- 172.21.15.32:0/4231405958 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080973a0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9afaa00640 1 -- 172.21.15.32:0/4231405958 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f9b080ab250 con 0x55ee0c0f91b0 2024-08-31T07:31:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.926+0000 7f9b22be9740 1 -- 172.21.15.32:0/4231405958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b080971f0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9afaa00640 1 -- 172.21.15.32:0/4231405958 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f9b080ab450 con 0x55ee0c0f91b0 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9b22be9740 10 client.29180 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9b22be9740 1 -- 172.21.15.32:0/4231405958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55ee0c561850 con 0x55ee0c0f91b0 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9afaa00640 1 -- 172.21.15.32:0/4231405958 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b080971f0 con 0x55ee0c0f91b0 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9afaa00640 1 -- 172.21.15.32:0/4231405958 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f9b080cd150 con 0x55ee0c0f91b0 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9b22be9740 10 client.29180 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9b22be9740 10 client.29180 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9b22be9740 20 client.29180 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9b22be9740 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ee0c0b5da0 0x55ee0c563d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9b22be9740 4 client.29180 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:31:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.927+0000 7f9b22be9740 1 -- 172.21.15.32:0/4231405958 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee0c43c1a0 con 0x55ee0c0b5da0 2024-08-31T07:31:28.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.928+0000 7f9b18a00640 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ee0c0b5da0 0x55ee0c563d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:28.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.928+0000 7f9b18a00640 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ee0c0b5da0 0x55ee0c563d40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=466725a2221415a4 server_cookie=81655eccc6d25ce2 in_seq=0 out_seq=0 2024-08-31T07:31:28.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.928+0000 7f9afaa00640 10 client.29180 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.928+0000 7f9afaa00640 1 -- 172.21.15.32:0/4231405958 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee0c43c1a0 con 0x55ee0c0b5da0 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.928+0000 7f9afaa00640 10 client.29180 handle_command_reply: tid=0 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.928+0000 7f9b22be9740 1 client.29180 shutdown 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 1 -- 172.21.15.32:0/4231405958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 msgr2=0x55ee0c504560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c504560 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9b080097b0 tx=0x7f9b080107e0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 1 -- 172.21.15.32:0/4231405958 shutdown_connections 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee0c122070 0x55ee0c559ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ee0c101390 0x55ee0c504b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55ee0c0b5da0 0x55ee0c563d40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 1 --2- 172.21.15.32:0/4231405958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee0c0f91b0 0x55ee0c504560 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 1 -- 172.21.15.32:0/4231405958 >> 172.21.15.32:0/4231405958 conn(0x55ee0c116460 msgr2=0x55ee0c4d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9afaa00640 0 client.29180 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 1 -- 172.21.15.32:0/4231405958 shutdown_connections 2024-08-31T07:31:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 1 -- 172.21.15.32:0/4231405958 wait complete. 2024-08-31T07:31:28.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.929+0000 7f9b22be9740 20 client.29180 trim_cache size 0 max 16384 2024-08-31T07:31:28.933 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:31:28.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.930+0000 7f9b1be00640 1 -- 172.21.15.32:0/73677663 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9afc0b4ca0 msgr2=0x7f9afc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.930+0000 7f9b1be00640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9afc0b4ca0 0x7f9afc0b71b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009fd0 tx=0x7f9b0c009500 comp rx=0 tx=0).stop 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.930+0000 7f9b1be00640 1 -- 172.21.15.32:0/73677663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b140754f0 msgr2=0x7f9b1419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.930+0000 7f9b1be00640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b140754f0 0x7f9b1419c460 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800e9f0 tx=0x7f9b0800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.930+0000 7f9b1be00640 1 -- 172.21.15.32:0/73677663 shutdown_connections 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.930+0000 7f9b1be00640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9afc0b4ca0 0x7f9afc0b71b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.930+0000 7f9b1be00640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b1410fef0 0x7f9b141a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.930+0000 7f9b1be00640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b14075e40 0x7f9b1419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.930+0000 7f9b1be00640 1 --2- 172.21.15.32:0/73677663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b140754f0 0x7f9b1419c460 secure :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800e9f0 tx=0x7f9b0800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.930+0000 7f9b1be00640 1 -- 172.21.15.32:0/73677663 >> 172.21.15.32:0/73677663 conn(0x7f9b140ff380 msgr2=0x7f9b14101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.931+0000 7f9b1be00640 1 -- 172.21.15.32:0/73677663 shutdown_connections 2024-08-31T07:31:28.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:28.931+0000 7f9b1be00640 1 -- 172.21.15.32:0/73677663 wait complete. 2024-08-31T07:31:29.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.11 MB/sec execute 189 sec latency 33123.591 ms 2024-08-31T07:31:29.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:31:30.385 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:30 smithi032 ceph-mon[17252]: pgmap v956: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-08-31T07:31:30.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92552 12.04 MB/sec execute 190 sec latency 34123.677 ms 2024-08-31T07:31:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.426+0000 7f2bc5600640 1 -- 172.21.15.32:0/3143524831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc010bbf0 msgr2=0x7f2bc0112420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.426+0000 7f2bc5600640 1 --2- 172.21.15.32:0/3143524831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc010bbf0 0x7f2bc0112420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009be0 tx=0x7f2bb802a6c0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.426+0000 7f2bc5600640 1 -- 172.21.15.32:0/3143524831 shutdown_connections 2024-08-31T07:31:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.426+0000 7f2bc5600640 1 --2- 172.21.15.32:0/3143524831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc010bbf0 0x7f2bc0112420 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.426+0000 7f2bc5600640 1 --2- 172.21.15.32:0/3143524831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc010b2a0 0x7f2bc010b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.426+0000 7f2bc5600640 1 --2- 172.21.15.32:0/3143524831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc0105960 0x7f2bc0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.426+0000 7f2bc5600640 1 -- 172.21.15.32:0/3143524831 >> 172.21.15.32:0/3143524831 conn(0x7f2bc01013d0 msgr2=0x7f2bc0103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:30.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.426+0000 7f2bc5600640 1 -- 172.21.15.32:0/3143524831 shutdown_connections 2024-08-31T07:31:30.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.426+0000 7f2bc5600640 1 -- 172.21.15.32:0/3143524831 wait complete. 2024-08-31T07:31:30.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.427+0000 7f2bc5600640 1 Processor -- start 2024-08-31T07:31:30.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.428+0000 7f2bc5600640 1 -- start start 2024-08-31T07:31:30.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.428+0000 7f2bc5600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc0105960 0x7f2bc019c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.428+0000 7f2bc5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc010b2a0 0x7f2bc019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.428+0000 7f2bc5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc010bbf0 0x7f2bc01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.428+0000 7f2bc5600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01158b0 con 0x7f2bc010b2a0 2024-08-31T07:31:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.428+0000 7f2bc5600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0115730 con 0x7f2bc010bbf0 2024-08-31T07:31:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.428+0000 7f2bc5600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0115a30 con 0x7f2bc0105960 2024-08-31T07:31:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.428+0000 7f2bbf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc010b2a0 0x7f2bc019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.428+0000 7f2bbf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc010b2a0 0x7f2bc019c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37870/0 (socket says 172.21.15.32:37870) 2024-08-31T07:31:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.429+0000 7f2bbf400640 1 -- 172.21.15.32:0/1478711101 learned_addr learned my addr 172.21.15.32:0/1478711101 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.429+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc0105960 0x7f2bc019c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.429+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc010bbf0 0x7f2bc01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.429+0000 7f2bbfe00640 1 -- 172.21.15.32:0/1478711101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc010bbf0 msgr2=0x7f2bc01a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.429+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc010bbf0 0x7f2bc01a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.429+0000 7f2bbfe00640 1 -- 172.21.15.32:0/1478711101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc010b2a0 msgr2=0x7f2bc019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.429+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc010b2a0 0x7f2bc019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.429+0000 7f2bbfe00640 1 -- 172.21.15.32:0/1478711101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc01a2190 con 0x7f2bc0105960 2024-08-31T07:31:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.429+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc0105960 0x7f2bc019c220 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0005fd0 tx=0x7f2bb000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:30.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.430+0000 7f2bbcc00640 1 -- 172.21.15.32:0/1478711101 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000cd00 con 0x7f2bc0105960 2024-08-31T07:31:30.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.430+0000 7f2bc5600640 1 -- 172.21.15.32:0/1478711101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc01a2480 con 0x7f2bc0105960 2024-08-31T07:31:30.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.430+0000 7f2bbcc00640 1 -- 172.21.15.32:0/1478711101 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb00035f0 con 0x7f2bc0105960 2024-08-31T07:31:30.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.430+0000 7f2bbcc00640 1 -- 172.21.15.32:0/1478711101 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0010690 con 0x7f2bc0105960 2024-08-31T07:31:30.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.430+0000 7f2bc5600640 1 -- 172.21.15.32:0/1478711101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc01a8be0 con 0x7f2bc0105960 2024-08-31T07:31:30.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.433+0000 7f2bcd7d9740 1 Processor -- start 2024-08-31T07:31:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.433+0000 7f2bcd7d9740 1 -- start start 2024-08-31T07:31:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.433+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.433+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9aa85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 0x5597f9aa8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bbcc00640 1 -- 172.21.15.32:0/1478711101 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2bb0010830 con 0x7f2bc0105960 2024-08-31T07:31:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5597f9a1ea90 con 0x5597f96e1840 2024-08-31T07:31:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5597f9a0e020 con 0x5597f9702520 2024-08-31T07:31:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5597f9a11fd0 con 0x5597f96d9660 2024-08-31T07:31:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57354/0 (socket says 172.21.15.32:57354) 2024-08-31T07:31:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bc4c00640 1 -- 172.21.15.32:0/1498821389 learned_addr learned my addr 172.21.15.32:0/1498821389 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/1498821389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9aa85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bbf400640 1 --2- 172.21.15.32:0/1498821389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 0x5597f9aa8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.434+0000 7f2bc4c00640 1 -- 172.21.15.32:0/1498821389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 msgr2=0x5597f9aa85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.435+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/1498821389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9aa85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.435+0000 7f2bc4c00640 1 -- 172.21.15.32:0/1498821389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 msgr2=0x5597f9aa8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.435+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/1498821389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 0x5597f9aa8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.435+0000 7f2bc4c00640 1 -- 172.21.15.32:0/1498821389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597f976db00 con 0x5597f9702520 2024-08-31T07:31:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.435+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/1498821389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9aa85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:31:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.435+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/1498821389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa8090 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2bb802acb0 tx=0x7f2bb8030420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=220f1d4f145e8c12 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.435+0000 7f2bb5200640 1 -- 172.21.15.32:0/1498821389 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8041030 con 0x5597f9702520 2024-08-31T07:31:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.436+0000 7f2bb5200640 1 -- 172.21.15.32:0/1498821389 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb8003e20 con 0x5597f9702520 2024-08-31T07:31:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.436+0000 7f2bb5200640 1 -- 172.21.15.32:0/1498821389 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8036040 con 0x5597f9702520 2024-08-31T07:31:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.436+0000 7f2bbcc00640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ba40b4ca0 0x7f2ba40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.436+0000 7f2bbcc00640 1 -- 172.21.15.32:0/1478711101 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f2bb00e57a0 con 0x7f2bc0105960 2024-08-31T07:31:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.436+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ba40b4ca0 0x7f2ba40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/1498821389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 msgr2=0x5597f9aa8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/1498821389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa8090 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2bb802acb0 tx=0x7f2bb8030420 comp rx=0 tx=0).stop 2024-08-31T07:31:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ba40b4ca0 0x7f2ba40b71b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x5597f9aa6d10 tx=0x7f2bb0000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:30.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/1498821389 shutdown_connections 2024-08-31T07:31:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/1498821389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 0x5597f9aa8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/1498821389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9aa85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/1498821389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa8090 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/1498821389 >> 172.21.15.32:0/1498821389 conn(0x5597f96f6910 msgr2=0x5597f9aa5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/1498821389 shutdown_connections 2024-08-31T07:31:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/1498821389 wait complete. 2024-08-31T07:31:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.437+0000 7f2bcd7d9740 1 -- start start 2024-08-31T07:31:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.438+0000 7f2bcd7d9740 1 Processor -- start 2024-08-31T07:31:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.438+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96d9660 0x5597f9ab5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.438+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96e1840 0x5597f9ab3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.438+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9ab3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.438+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5597f9870bb0 con 0x5597f96d9660 2024-08-31T07:31:30.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.438+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5597f9994f90 con 0x5597f9702520 2024-08-31T07:31:30.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.438+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5597f9ab6810 con 0x5597f96e1840 2024-08-31T07:31:30.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.438+0000 7f2bcd7d9740 10 client.0 _ll_register_callbacks cb 0x5597f99ba710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:30.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96d9660 0x5597f9ab5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96d9660 0x5597f9ab5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37894/0 (socket says 172.21.15.32:37894) 2024-08-31T07:31:30.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bc4c00640 1 -- 172.21.15.32:0/507866335 learned_addr learned my addr 172.21.15.32:0/507866335 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:30.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bbf400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96e1840 0x5597f9ab3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9ab3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bbf400640 1 -- 172.21.15.32:0/507866335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 msgr2=0x5597f9ab3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:30.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bbf400640 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9ab3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bbf400640 1 -- 172.21.15.32:0/507866335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96d9660 msgr2=0x5597f9ab5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:30.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bbf400640 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96d9660 0x5597f9ab5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bbf400640 1 -- 172.21.15.32:0/507866335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597f97e4960 con 0x5597f96e1840 2024-08-31T07:31:30.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.439+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96d9660 0x5597f9ab5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:30.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.440+0000 7f2bbf400640 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96e1840 0x5597f9ab3410 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0104c80 tx=0x7f2bac012bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70f1d02263698d92 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:30.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.440+0000 7f2ba9600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:31:30.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.440+0000 7f2ba9600640 1 -- 172.21.15.32:0/507866335 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00f8f0 con 0x5597f96e1840 2024-08-31T07:31:30.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.440+0000 7f2ba9600640 1 -- 172.21.15.32:0/507866335 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f2bac00fa90 con 0x5597f96e1840 2024-08-31T07:31:30.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.440+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/507866335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5597f99ace30 con 0x5597f96e1840 2024-08-31T07:31:30.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.440+0000 7f2ba9600640 1 -- 172.21.15.32:0/507866335 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f2bac00fc90 con 0x5597f96e1840 2024-08-31T07:31:30.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.440+0000 7f2bcd7d9740 10 client.29188 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:30.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.440+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/507866335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5597f9ac21d0 con 0x5597f96e1840 2024-08-31T07:31:30.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.441+0000 7f2ba9600640 1 -- 172.21.15.32:0/507866335 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5597f99ace30 con 0x5597f96e1840 2024-08-31T07:31:30.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.441+0000 7f2ba9600640 1 -- 172.21.15.32:0/507866335 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f2bac010430 con 0x5597f96e1840 2024-08-31T07:31:30.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.441+0000 7f2bcd7d9740 10 client.29188 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:30.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.441+0000 7f2bcd7d9740 10 client.29188 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:31:30.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.441+0000 7f2bcd7d9740 20 client.29188 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:30.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.441+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597f96c0ea0 0x5597f9ac5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.441+0000 7f2bcd7d9740 4 client.29188 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:31:30.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.441+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/507866335 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597f9a209e0 con 0x5597f96c0ea0 2024-08-31T07:31:30.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.441+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597f96c0ea0 0x5597f9ac5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.442+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597f96c0ea0 0x5597f9ac5730 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6938e53a88a20e server_cookie=108cd47cd972fe1 in_seq=0 out_seq=0 2024-08-31T07:31:30.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.442+0000 7f2ba9600640 10 client.29188 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:30.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2ba9600640 1 -- 172.21.15.32:0/507866335 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597f9a209e0 con 0x5597f96c0ea0 2024-08-31T07:31:30.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2ba9600640 10 client.29188 handle_command_reply: tid=0 2024-08-31T07:31:30.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 client.29188 shutdown 2024-08-31T07:31:30.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/507866335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96e1840 msgr2=0x5597f9ab3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:30.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96e1840 0x5597f9ab3410 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0104c80 tx=0x7f2bac012bd0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/507866335 shutdown_connections 2024-08-31T07:31:30.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597f96c0ea0 0x5597f9ac5730 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9ab3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96e1840 0x5597f9ab3410 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/507866335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96d9660 0x5597f9ab5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/507866335 >> 172.21.15.32:0/507866335 conn(0x5597f96f6910 msgr2=0x5597f9aab270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:30.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2ba9600640 0 client.29188 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:30.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/507866335 shutdown_connections 2024-08-31T07:31:30.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.443+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/507866335 wait complete. 2024-08-31T07:31:30.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.444+0000 7f2bcd7d9740 20 client.29188 trim_cache size 0 max 16384 2024-08-31T07:31:30.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.464+0000 7f2bcd7d9740 1 Processor -- start 2024-08-31T07:31:30.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.465+0000 7f2bcd7d9740 1 -- start start 2024-08-31T07:31:30.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.465+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9ab5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.465+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 0x5597f9aa97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.465+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.465+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01b13a0 con 0x5597f96e1840 2024-08-31T07:31:30.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.465+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01b2f40 con 0x5597f9702520 2024-08-31T07:31:30.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.465+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01b1aa0 con 0x5597f96d9660 2024-08-31T07:31:30.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.465+0000 7f2bbf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 0x5597f9aa97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.465+0000 7f2bbf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 0x5597f9aa97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37898/0 (socket says 172.21.15.32:37898) 2024-08-31T07:31:30.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.465+0000 7f2bbf400640 1 -- 172.21.15.32:0/811807094 learned_addr learned my addr 172.21.15.32:0/811807094 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:30.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.466+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/811807094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9ab5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.466+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/811807094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.466+0000 7f2bbf400640 1 -- 172.21.15.32:0/811807094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 msgr2=0x5597f9ab5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.466+0000 7f2bbf400640 1 --2- 172.21.15.32:0/811807094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9ab5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.466+0000 7f2bbf400640 1 -- 172.21.15.32:0/811807094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 msgr2=0x5597f9aa5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.466+0000 7f2bbf400640 1 --2- 172.21.15.32:0/811807094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.466+0000 7f2bbf400640 1 -- 172.21.15.32:0/811807094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597f9ab6750 con 0x5597f96e1840 2024-08-31T07:31:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.466+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/811807094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:31:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.466+0000 7f2bbf400640 1 --2- 172.21.15.32:0/811807094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 0x5597f9aa97d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0123e0 tx=0x7f2bac012410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0362f914a316ab server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:30.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.467+0000 7f2babe00640 1 -- 172.21.15.32:0/811807094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00f8f0 con 0x5597f96e1840 2024-08-31T07:31:30.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.467+0000 7f2babe00640 1 -- 172.21.15.32:0/811807094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bac00fa90 con 0x5597f96e1840 2024-08-31T07:31:30.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.467+0000 7f2babe00640 1 -- 172.21.15.32:0/811807094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac01cc10 con 0x5597f96e1840 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.468+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/811807094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 msgr2=0x5597f9aa97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.468+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/811807094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 0x5597f9aa97d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0123e0 tx=0x7f2bac012410 comp rx=0 tx=0).stop 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.468+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/811807094 shutdown_connections 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.468+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/811807094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f9702520 0x5597f9aa5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.468+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/811807094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f96e1840 0x5597f9aa97d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.468+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/811807094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9ab5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.468+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/811807094 >> 172.21.15.32:0/811807094 conn(0x5597f96f6910 msgr2=0x5597f9aa90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.468+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/811807094 shutdown_connections 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.468+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/811807094 wait complete. 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.468+0000 7f2bcd7d9740 1 -- start start 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.469+0000 7f2bcd7d9740 1 Processor -- start 2024-08-31T07:31:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.469+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9b29400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.469+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f96e1840 0x5597f9b299c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.469+0000 7f2bcd7d9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f9702520 0x5597f9ab8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.469+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01b2870 con 0x5597f9702520 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.469+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01b21a0 con 0x5597f96e1840 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.469+0000 7f2bcd7d9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01af3a0 con 0x5597f96d9660 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.469+0000 7f2bcd7d9740 10 client.0 _ll_register_callbacks cb 0x7f2bac009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9b29400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bbf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f96e1840 0x5597f9b299c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9b29400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42108/0 (socket says 172.21.15.32:42108) 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bc4c00640 1 -- 172.21.15.32:0/2682666511 learned_addr learned my addr 172.21.15.32:0/2682666511 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f9702520 0x5597f9ab8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bbf400640 1 -- 172.21.15.32:0/2682666511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 msgr2=0x5597f9b29400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bbf400640 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9b29400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bbf400640 1 -- 172.21.15.32:0/2682666511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f9702520 msgr2=0x5597f9ab8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:30.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bbf400640 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f9702520 0x5597f9ab8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bbf400640 1 -- 172.21.15.32:0/2682666511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597f9ab88a0 con 0x5597f96e1840 2024-08-31T07:31:30.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.470+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9b29400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.471+0000 7f2bbf400640 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f96e1840 0x5597f9b299c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2bac010850 tx=0x7f2bac01d730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3d543de9d98966 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.471+0000 7f2ba9600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:31:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.471+0000 7f2ba9600640 1 -- 172.21.15.32:0/2682666511 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00f8f0 con 0x5597f96e1840 2024-08-31T07:31:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.471+0000 7f2ba9600640 1 -- 172.21.15.32:0/2682666511 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f2bac01db30 con 0x5597f96e1840 2024-08-31T07:31:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.471+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/2682666511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bac00fa90 con 0x5597f96e1840 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.471+0000 7f2ba9600640 1 -- 172.21.15.32:0/2682666511 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f2bac03c020 con 0x5597f96e1840 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.471+0000 7f2bcd7d9740 10 client.29198 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.471+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/2682666511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5597f9b30b60 con 0x5597f96e1840 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.472+0000 7f2ba9600640 1 -- 172.21.15.32:0/2682666511 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bac00fa90 con 0x5597f96e1840 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.472+0000 7f2ba9600640 1 -- 172.21.15.32:0/2682666511 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f2bac0418e0 con 0x5597f96e1840 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.472+0000 7f2bcd7d9740 10 client.29198 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.472+0000 7f2bcd7d9740 10 client.29198 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.472+0000 7f2bcd7d9740 20 client.29198 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.472+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597f9696250 0x5597f9b33230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.472+0000 7f2bcd7d9740 4 client.29198 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fd6c69c8", "release": true, "await": true}] 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.472+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/2682666511 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fd6c69c8", "release": true, "await": true}) -- 0x5597f9a1c480 con 0x5597f9696250 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.472+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597f9696250 0x5597f9b33230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.473+0000 7f2ba9600640 10 client.29198 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.473+0000 7f2bc4c00640 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597f9696250 0x5597f9b33230 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2942dd55a49768d6 server_cookie=49a277f66c5b22e2 in_seq=0 out_seq=0 2024-08-31T07:31:30.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.501+0000 7f2ba9600640 1 -- 172.21.15.32:0/2682666511 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5597f9a1c480 con 0x5597f9696250 2024-08-31T07:31:30.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.501+0000 7f2ba9600640 10 client.29198 handle_command_reply: tid=0 2024-08-31T07:31:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.501+0000 7f2bcd7d9740 1 client.29198 shutdown 2024-08-31T07:31:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.501+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/2682666511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f96e1840 msgr2=0x5597f9b299c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.501+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f96e1840 0x5597f9b299c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2bac010850 tx=0x7f2bac01d730 comp rx=0 tx=0).stop 2024-08-31T07:31:30.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/2682666511 shutdown_connections 2024-08-31T07:31:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597f9696250 0x5597f9b33230 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597f9702520 0x5597f9ab8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597f96e1840 0x5597f9b299c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2ba9600640 0 client.29198 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bcd7d9740 1 --2- 172.21.15.32:0/2682666511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597f96d9660 0x5597f9b29400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/2682666511 >> 172.21.15.32:0/2682666511 conn(0x5597f96f6910 msgr2=0x5597f9ab5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/2682666511 shutdown_connections 2024-08-31T07:31:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bcd7d9740 1 -- 172.21.15.32:0/2682666511 wait complete. 2024-08-31T07:31:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bcd7d9740 20 client.29198 trim_cache size 0 max 16384 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 67, 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 60, 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "fd6c69c8": { 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 60, 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:31:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 151.7, 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:31:30.506 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:31:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 -- 172.21.15.32:0/1478711101 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ba40b4ca0 msgr2=0x7f2ba40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ba40b4ca0 0x7f2ba40b71b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x5597f9aa6d10 tx=0x7f2bb0000bc0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 -- 172.21.15.32:0/1478711101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc0105960 msgr2=0x7f2bc019c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc0105960 0x7f2bc019c220 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0005fd0 tx=0x7f2bb000ee50 comp rx=0 tx=0).stop 2024-08-31T07:31:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 -- 172.21.15.32:0/1478711101 shutdown_connections 2024-08-31T07:31:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ba40b4ca0 0x7f2ba40b71b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc010bbf0 0x7f2bc01a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc010b2a0 0x7f2bc019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 --2- 172.21.15.32:0/1478711101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc0105960 0x7f2bc019c220 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 -- 172.21.15.32:0/1478711101 >> 172.21.15.32:0/1478711101 conn(0x7f2bc01013d0 msgr2=0x7f2bc010a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 -- 172.21.15.32:0/1478711101 shutdown_connections 2024-08-31T07:31:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:30.502+0000 7f2bc5600640 1 -- 172.21.15.32:0/1478711101 wait complete. 2024-08-31T07:31:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:30 smithi073 ceph-mon[21995]: pgmap v956: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-08-31T07:31:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:30 smithi084 ceph-mon[22387]: pgmap v956: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-08-31T07:31:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.374+0000 7fd5f8600640 1 -- 172.21.15.32:0/2109196519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f0102cc0 msgr2=0x7fd5f010f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.374+0000 7fd5f8600640 1 --2- 172.21.15.32:0/2109196519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f0102cc0 0x7fd5f010f9e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009900 tx=0x7fd5dc02a480 comp rx=0 tx=0).stop 2024-08-31T07:31:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.374+0000 7fd5f8600640 1 -- 172.21.15.32:0/2109196519 shutdown_connections 2024-08-31T07:31:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.374+0000 7fd5f8600640 1 --2- 172.21.15.32:0/2109196519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f010ff20 0x7fd5f0112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.374+0000 7fd5f8600640 1 --2- 172.21.15.32:0/2109196519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f0102cc0 0x7fd5f010f9e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.374+0000 7fd5f8600640 1 --2- 172.21.15.32:0/2109196519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f0102370 0x7fd5f0102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.374+0000 7fd5f8600640 1 -- 172.21.15.32:0/2109196519 >> 172.21.15.32:0/2109196519 conn(0x7fd5f00794d0 msgr2=0x7fd5f00798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.375+0000 7fd5f8600640 1 -- 172.21.15.32:0/2109196519 shutdown_connections 2024-08-31T07:31:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.375+0000 7fd5f8600640 1 -- 172.21.15.32:0/2109196519 wait complete. 2024-08-31T07:31:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.376+0000 7fd5f8600640 1 Processor -- start 2024-08-31T07:31:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.377+0000 7fd5f8600640 1 -- start start 2024-08-31T07:31:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.377+0000 7fd5f8600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f0102370 0x7fd5f01083c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.377+0000 7fd5f8600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0102cc0 0x7fd5f01046a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.377+0000 7fd5f8600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f010ff20 0x7fd5f0104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.377+0000 7fd5f8600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f0115a60 con 0x7fd5f0102cc0 2024-08-31T07:31:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.377+0000 7fd5f8600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01158e0 con 0x7fd5f010ff20 2024-08-31T07:31:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.377+0000 7fd5f8600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f0115be0 con 0x7fd5f0102370 2024-08-31T07:31:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.378+0000 7fd5f5c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f0102370 0x7fd5f01083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.378+0000 7fd5f6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f010ff20 0x7fd5f0104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.378+0000 7fd5f5c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f0102370 0x7fd5f01083c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42114/0 (socket says 172.21.15.32:42114) 2024-08-31T07:31:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.378+0000 7fd5f5c00640 1 -- 172.21.15.32:0/3818237929 learned_addr learned my addr 172.21.15.32:0/3818237929 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.378+0000 7fd5f5200640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0102cc0 0x7fd5f01046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.378+0000 7fd5f6600640 1 -- 172.21.15.32:0/3818237929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f0102370 msgr2=0x7fd5f01083c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.378+0000 7fd5f6600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f0102370 0x7fd5f01083c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.378+0000 7fd5f6600640 1 -- 172.21.15.32:0/3818237929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0102cc0 msgr2=0x7fd5f01046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.378+0000 7fd5f6600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0102cc0 0x7fd5f01046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.378+0000 7fd5f6600640 1 -- 172.21.15.32:0/3818237929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f01ac9c0 con 0x7fd5f010ff20 2024-08-31T07:31:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.379+0000 7fd5f6600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f010ff20 0x7fd5f0104dc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800ca30 tx=0x7fd5e800cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.379+0000 7fd5ee000640 1 -- 172.21.15.32:0/3818237929 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8007d90 con 0x7fd5f010ff20 2024-08-31T07:31:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.379+0000 7fd5ee000640 1 -- 172.21.15.32:0/3818237929 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e8005020 con 0x7fd5f010ff20 2024-08-31T07:31:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.379+0000 7fd5ee000640 1 -- 172.21.15.32:0/3818237929 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800f690 con 0x7fd5f010ff20 2024-08-31T07:31:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.379+0000 7fd5f8600640 1 -- 172.21.15.32:0/3818237929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f01acbf0 con 0x7fd5f010ff20 2024-08-31T07:31:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.380+0000 7fd5f8600640 1 -- 172.21.15.32:0/3818237929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f01ad0b0 con 0x7fd5f010ff20 2024-08-31T07:31:31.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.385+0000 7fd5ee000640 1 -- 172.21.15.32:0/3818237929 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd5e80051d0 con 0x7fd5f010ff20 2024-08-31T07:31:31.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.386+0000 7fd5ff29f740 1 Processor -- start 2024-08-31T07:31:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.386+0000 7fd5ff29f740 1 -- start start 2024-08-31T07:31:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.386+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5621070 0x55dac59c6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.386+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.386+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5600390 0x55dac59c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.386+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dac593d7b0 con 0x55dac55f81b0 2024-08-31T07:31:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.386+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dac592cd40 con 0x55dac5621070 2024-08-31T07:31:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.386+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dac5930cf0 con 0x55dac5600390 2024-08-31T07:31:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.387+0000 7fd5ee000640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5d80b4d20 0x7fd5d80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.387+0000 7fd5f5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5621070 0x55dac59c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.387+0000 7fd5f5200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5600390 0x55dac59c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.388+0000 7fd5f6600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5d80b4d20 0x7fd5d80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.388+0000 7fd5f5200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5600390 0x55dac59c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42130/0 (socket says 172.21.15.32:42130) 2024-08-31T07:31:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.388+0000 7fd5f5200640 1 -- 172.21.15.32:0/1348240988 learned_addr learned my addr 172.21.15.32:0/1348240988 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.388+0000 7fd5ee000640 1 -- 172.21.15.32:0/3818237929 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fd5e80e57b0 con 0x7fd5f010ff20 2024-08-31T07:31:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.389+0000 7fd5f5200640 1 -- 172.21.15.32:0/1348240988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5621070 msgr2=0x55dac59c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.389+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1348240988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5621070 0x55dac59c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.389+0000 7fd5f5200640 1 -- 172.21.15.32:0/1348240988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 msgr2=0x55dac59c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.389+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1348240988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.389+0000 7fd5f5200640 1 -- 172.21.15.32:0/1348240988 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dac568c650 con 0x55dac5600390 2024-08-31T07:31:31.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.389+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1348240988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5600390 0x55dac59c7410 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009230 tx=0x7fd5dc02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c655506c7cd420cb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:31.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.390+0000 7fd5f6600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5d80b4d20 0x7fd5d80b7230 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80abe20 tx=0x7fd5e80abc60 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:31.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.390+0000 7fd5e2a00640 1 -- 172.21.15.32:0/1348240988 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc035030 con 0x55dac5600390 2024-08-31T07:31:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.391+0000 7fd5e2a00640 1 -- 172.21.15.32:0/1348240988 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5dc005630 con 0x55dac5600390 2024-08-31T07:31:31.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.391+0000 7fd5e2a00640 1 -- 172.21.15.32:0/1348240988 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc035030 con 0x55dac5600390 2024-08-31T07:31:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.392+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1348240988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5600390 msgr2=0x55dac59c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.392+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1348240988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5600390 0x55dac59c7410 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009230 tx=0x7fd5dc02a480 comp rx=0 tx=0).stop 2024-08-31T07:31:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.392+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1348240988 shutdown_connections 2024-08-31T07:31:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.392+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1348240988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5600390 0x55dac59c7410 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.392+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1348240988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.392+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1348240988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5621070 0x55dac59c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.392+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1348240988 >> 172.21.15.32:0/1348240988 conn(0x55dac5615460 msgr2=0x55dac59c4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.393+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1348240988 shutdown_connections 2024-08-31T07:31:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.393+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1348240988 wait complete. 2024-08-31T07:31:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.393+0000 7fd5ff29f740 1 -- start start 2024-08-31T07:31:31.395 INFO:tasks.workunit.client.0.smithi032.stdout: 10 93122 12.14 MB/sec execute 191 sec latency 34231.237 ms 2024-08-31T07:31:31.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.394+0000 7fd5ff29f740 1 Processor -- start 2024-08-31T07:31:31.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.395+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac55f81b0 0x55dac59d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.395+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac5600390 0x55dac59d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.395+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5621070 0x55dac59de430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.395+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55dac578f700 con 0x55dac5600390 2024-08-31T07:31:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.395+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dac58b3cf0 con 0x55dac5621070 2024-08-31T07:31:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.395+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dac59d5110 con 0x55dac55f81b0 2024-08-31T07:31:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.395+0000 7fd5ff29f740 10 client.0 _ll_register_callbacks cb 0x55dac58d9470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:31.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.396+0000 7fd5f5200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac55f81b0 0x55dac59d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.396+0000 7fd5f5200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac55f81b0 0x55dac59d25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42144/0 (socket says 172.21.15.32:42144) 2024-08-31T07:31:31.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.396+0000 7fd5f5200640 1 -- 172.21.15.32:0/1766587449 learned_addr learned my addr 172.21.15.32:0/1766587449 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:31.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.396+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac5600390 0x55dac59d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.396+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5621070 0x55dac59de430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.397+0000 7fd5f5200640 1 -- 172.21.15.32:0/1766587449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5621070 msgr2=0x55dac59de430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.397+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5621070 0x55dac59de430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.397+0000 7fd5f5200640 1 -- 172.21.15.32:0/1766587449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac5600390 msgr2=0x55dac59d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.397+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac5600390 0x55dac59d2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.397+0000 7fd5f5200640 1 -- 172.21.15.32:0/1766587449 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dac58f48b0 con 0x55dac55f81b0 2024-08-31T07:31:31.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.397+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac55f81b0 0x55dac59d25d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc005a80 tx=0x7fd5dc004c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41826ce52d6df2fd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:31.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.397+0000 7fd5d7400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:31:31.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.397+0000 7fd5d7400640 1 -- 172.21.15.32:0/1766587449 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc035040 con 0x55dac55f81b0 2024-08-31T07:31:31.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.398+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1766587449 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dac593f700 con 0x55dac55f81b0 2024-08-31T07:31:31.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.398+0000 7fd5d7400640 1 -- 172.21.15.32:0/1766587449 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fd5dc044070 con 0x55dac55f81b0 2024-08-31T07:31:31.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.398+0000 7fd5d7400640 1 -- 172.21.15.32:0/1766587449 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd5dc005410 con 0x55dac55f81b0 2024-08-31T07:31:31.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.398+0000 7fd5ff29f740 10 client.29206 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.398+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1766587449 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55dac59e1180 con 0x55dac55f81b0 2024-08-31T07:31:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.398+0000 7fd5d7400640 1 -- 172.21.15.32:0/1766587449 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dac593f700 con 0x55dac55f81b0 2024-08-31T07:31:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.399+0000 7fd5d7400640 1 -- 172.21.15.32:0/1766587449 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fd5dc049040 con 0x55dac55f81b0 2024-08-31T07:31:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.399+0000 7fd5ff29f740 10 client.29206 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.399+0000 7fd5ff29f740 10 client.29206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:31:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.399+0000 7fd5ff29f740 20 client.29206 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.399+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dac55df9f0 0x55dac59e44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.399+0000 7fd5ff29f740 4 client.29206 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:31:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.399+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1766587449 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dac593b1a0 con 0x55dac55df9f0 2024-08-31T07:31:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.401+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dac55df9f0 0x55dac59e44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.401+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dac55df9f0 0x55dac59e44e0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ee8cb07cf3192bc server_cookie=edf260c94f9adcf5 in_seq=0 out_seq=0 2024-08-31T07:31:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.401+0000 7fd5d7400640 10 client.29206 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:31.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.406+0000 7fd5d7400640 1 -- 172.21.15.32:0/1766587449 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dac593b1a0 con 0x55dac55df9f0 2024-08-31T07:31:31.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.406+0000 7fd5d7400640 10 client.29206 handle_command_reply: tid=0 2024-08-31T07:31:31.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.406+0000 7fd5ff29f740 1 client.29206 shutdown 2024-08-31T07:31:31.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.407+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1766587449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac55f81b0 msgr2=0x55dac59d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.407+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac55f81b0 0x55dac59d25d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc005a80 tx=0x7fd5dc004c00 comp rx=0 tx=0).stop 2024-08-31T07:31:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.407+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1766587449 shutdown_connections 2024-08-31T07:31:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.407+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dac55df9f0 0x55dac59e44e0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.407+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5621070 0x55dac59de430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.407+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac5600390 0x55dac59d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.407+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1766587449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac55f81b0 0x55dac59d25d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.407+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1766587449 >> 172.21.15.32:0/1766587449 conn(0x55dac5615460 msgr2=0x55dac59c9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:31.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.407+0000 7fd5d7400640 0 client.29206 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.407+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1766587449 shutdown_connections 2024-08-31T07:31:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.408+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1766587449 wait complete. 2024-08-31T07:31:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.408+0000 7fd5ff29f740 20 client.29206 trim_cache size 0 max 16384 2024-08-31T07:31:31.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5ff29f740 1 Processor -- start 2024-08-31T07:31:31.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5ff29f740 1 -- start start 2024-08-31T07:31:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5600390 0x55dac59d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 0x55dac59de3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01b59a0 con 0x55dac55f81b0 2024-08-31T07:31:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01b7540 con 0x55dac5600390 2024-08-31T07:31:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01b60a0 con 0x55dac5621070 2024-08-31T07:31:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5f5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5600390 0x55dac59d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5f5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5600390 0x55dac59d4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57442/0 (socket says 172.21.15.32:57442) 2024-08-31T07:31:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5f5c00640 1 -- 172.21.15.32:0/2787669093 learned_addr learned my addr 172.21.15.32:0/2787669093 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5f5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5f5c00640 1 -- 172.21.15.32:0/2787669093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 msgr2=0x55dac59de3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/2787669093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 0x55dac59de3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5f5c00640 1 -- 172.21.15.32:0/2787669093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 msgr2=0x55dac59cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/2787669093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.432+0000 7fd5f5c00640 1 -- 172.21.15.32:0/2787669093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dac59c8c80 con 0x55dac5600390 2024-08-31T07:31:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/2787669093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5600390 0x55dac59d4000 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4004700 tx=0x7fd5e4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ce4971c736fa69 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5e2000640 1 -- 172.21.15.32:0/2787669093 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4010e30 con 0x55dac5600390 2024-08-31T07:31:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5e2000640 1 -- 172.21.15.32:0/2787669093 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e40035f0 con 0x55dac5600390 2024-08-31T07:31:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5e2000640 1 -- 172.21.15.32:0/2787669093 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e401d6d0 con 0x55dac5600390 2024-08-31T07:31:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 -- 172.21.15.32:0/2787669093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5600390 msgr2=0x55dac59d4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/2787669093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5600390 0x55dac59d4000 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4004700 tx=0x7fd5e4000bc0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 -- 172.21.15.32:0/2787669093 shutdown_connections 2024-08-31T07:31:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/2787669093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 0x55dac59de3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/2787669093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac5600390 0x55dac59d4000 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/2787669093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 -- 172.21.15.32:0/2787669093 >> 172.21.15.32:0/2787669093 conn(0x55dac5615460 msgr2=0x55dac59d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 -- 172.21.15.32:0/2787669093 shutdown_connections 2024-08-31T07:31:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 -- 172.21.15.32:0/2787669093 wait complete. 2024-08-31T07:31:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 -- start start 2024-08-31T07:31:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 Processor -- start 2024-08-31T07:31:31.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.438+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 0x55dac59e6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5ff29f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac55b4da0 0x55dac5a4dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01b6e70 con 0x55dac55f81b0 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01b67a0 con 0x55dac55b4da0 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5ff29f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01b39a0 con 0x55dac5621070 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5ff29f740 10 client.0 _ll_register_callbacks cb 0x7fd5dc0042e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 0x55dac59e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 0x55dac59e6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42164/0 (socket says 172.21.15.32:42164) 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5c00640 1 -- 172.21.15.32:0/1884937675 learned_addr learned my addr 172.21.15.32:0/1884937675 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac55b4da0 0x55dac5a4dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59e5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5c00640 1 -- 172.21.15.32:0/1884937675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac55b4da0 msgr2=0x55dac5a4dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac55b4da0 0x55dac5a4dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5c00640 1 -- 172.21.15.32:0/1884937675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 msgr2=0x55dac59e5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59e5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5c00640 1 -- 172.21.15.32:0/1884937675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dac5a4e2d0 con 0x55dac5621070 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 0x55dac59e6360 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009df0 tx=0x7fd5e4003f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6066f8a88f0655e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5d7400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5d7400640 1 -- 172.21.15.32:0/1884937675 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4004430 con 0x55dac5621070 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.443+0000 7fd5d7400640 1 -- 172.21.15.32:0/1884937675 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fd5e4005030 con 0x55dac5621070 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1884937675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e40035f0 con 0x55dac5621070 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5d7400640 1 -- 172.21.15.32:0/1884937675 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd5e4034050 con 0x55dac5621070 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5ff29f740 10 client.29209 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1884937675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55dac5a4f550 con 0x55dac5621070 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5d7400640 1 -- 172.21.15.32:0/1884937675 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e40035f0 con 0x55dac5621070 2024-08-31T07:31:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5d7400640 1 -- 172.21.15.32:0/1884937675 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7fd5e4004730 con 0x55dac5621070 2024-08-31T07:31:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5ff29f740 10 client.29209 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5ff29f740 10 client.29209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:31:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5ff29f740 20 client.29209 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dac5a4fbf0 0x55dac5a52010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5ff29f740 4 client.29209 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:31:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1884937675 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dac59388e0 con 0x55dac5a4fbf0 2024-08-31T07:31:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dac5a4fbf0 0x55dac5a52010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5f5200640 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dac5a4fbf0 0x55dac5a52010 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8058ae287afdd84 server_cookie=d3435de48a887abe in_seq=0 out_seq=0 2024-08-31T07:31:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.446+0000 7fd5d7400640 10 client.29209 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.451+0000 7fd5d7400640 1 -- 172.21.15.32:0/1884937675 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dac59388e0 con 0x55dac5a4fbf0 2024-08-31T07:31:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.451+0000 7fd5d7400640 10 client.29209 handle_command_reply: tid=0 2024-08-31T07:31:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.451+0000 7fd5ff29f740 1 client.29209 shutdown 2024-08-31T07:31:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.453+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1884937675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 msgr2=0x55dac59e6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.453+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 0x55dac59e6360 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009df0 tx=0x7fd5e4003f70 comp rx=0 tx=0).stop 2024-08-31T07:31:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.453+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1884937675 shutdown_connections 2024-08-31T07:31:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.453+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55dac5a4fbf0 0x55dac5a52010 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.453+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dac55b4da0 0x55dac5a4dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.453+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dac5621070 0x55dac59e6360 secure :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009df0 tx=0x7fd5e4003f70 comp rx=0 tx=0).stop 2024-08-31T07:31:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.453+0000 7fd5ff29f740 1 --2- 172.21.15.32:0/1884937675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dac55f81b0 0x55dac59e5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.453+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1884937675 >> 172.21.15.32:0/1884937675 conn(0x55dac5615460 msgr2=0x55dac59c8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.453+0000 7fd5d7400640 0 client.29209 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.454+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1884937675 shutdown_connections 2024-08-31T07:31:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.454+0000 7fd5ff29f740 1 -- 172.21.15.32:0/1884937675 wait complete. 2024-08-31T07:31:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.455+0000 7fd5ff29f740 20 client.29209 trim_cache size 0 max 16384 2024-08-31T07:31:31.456 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:31:31.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 -- 172.21.15.32:0/3818237929 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5d80b4d20 msgr2=0x7fd5d80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:31.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5d80b4d20 0x7fd5d80b7230 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80abe20 tx=0x7fd5e80abc60 comp rx=0 tx=0).stop 2024-08-31T07:31:31.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 -- 172.21.15.32:0/3818237929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f010ff20 msgr2=0x7fd5f0104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:31.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f010ff20 0x7fd5f0104dc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800ca30 tx=0x7fd5e800cef0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 -- 172.21.15.32:0/3818237929 shutdown_connections 2024-08-31T07:31:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd5d80b4d20 0x7fd5d80b7230 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5f010ff20 0x7fd5f0104dc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0102cc0 0x7fd5f01046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 --2- 172.21.15.32:0/3818237929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5f0102370 0x7fd5f01083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 -- 172.21.15.32:0/3818237929 >> 172.21.15.32:0/3818237929 conn(0x7fd5f00794d0 msgr2=0x7fd5f0109ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.456+0000 7fd5f8600640 1 -- 172.21.15.32:0/3818237929 shutdown_connections 2024-08-31T07:31:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:31.457+0000 7fd5f8600640 1 -- 172.21.15.32:0/3818237929 wait complete. 2024-08-31T07:31:31.860 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fd6c69c8', total seconds elapsed: 41.3 2024-08-31T07:31:32.394 INFO:tasks.workunit.client.0.smithi032.stdout: 10 93921 12.29 MB/sec execute 192 sec latency 1214.197 ms 2024-08-31T07:31:32.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:32 smithi032 ceph-mon[17252]: pgmap v957: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 17 op/s 2024-08-31T07:31:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:32 smithi073 ceph-mon[21995]: pgmap v957: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 17 op/s 2024-08-31T07:31:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:32 smithi084 ceph-mon[22387]: pgmap v957: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 17 op/s 2024-08-31T07:31:33.395 INFO:tasks.workunit.client.0.smithi032.stdout: 10 94670 12.46 MB/sec execute 193 sec latency 2214.311 ms 2024-08-31T07:31:33.715 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:31:33.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:31:34.298 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:34.395 INFO:tasks.workunit.client.0.smithi032.stdout: 10 95387 12.58 MB/sec execute 194 sec latency 3214.369 ms 2024-08-31T07:31:34.658 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:34 smithi032 ceph-mon[17252]: pgmap v958: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 564 KiB/s wr, 51 op/s 2024-08-31T07:31:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:34 smithi073 ceph-mon[21995]: pgmap v958: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 564 KiB/s wr, 51 op/s 2024-08-31T07:31:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:34 smithi084 ceph-mon[22387]: pgmap v958: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 564 KiB/s wr, 51 op/s 2024-08-31T07:31:35.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.107+0000 7f5c25400640 1 -- 172.21.15.32:0/165468925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c2010bbf0 msgr2=0x7f5c20112420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:35.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.107+0000 7f5c25400640 1 --2- 172.21.15.32:0/165468925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c2010bbf0 0x7f5c20112420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009be0 tx=0x7f5c1002a6c0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.107+0000 7f5c25400640 1 -- 172.21.15.32:0/165468925 shutdown_connections 2024-08-31T07:31:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.107+0000 7f5c25400640 1 --2- 172.21.15.32:0/165468925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c2010bbf0 0x7f5c20112420 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.107+0000 7f5c25400640 1 --2- 172.21.15.32:0/165468925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c2010b2a0 0x7f5c2010b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.107+0000 7f5c25400640 1 --2- 172.21.15.32:0/165468925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c20105960 0x7f5c20105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.107+0000 7f5c25400640 1 -- 172.21.15.32:0/165468925 >> 172.21.15.32:0/165468925 conn(0x7f5c201013d0 msgr2=0x7f5c20103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.107+0000 7f5c25400640 1 -- 172.21.15.32:0/165468925 shutdown_connections 2024-08-31T07:31:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.107+0000 7f5c25400640 1 -- 172.21.15.32:0/165468925 wait complete. 2024-08-31T07:31:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.109+0000 7f5c25400640 1 Processor -- start 2024-08-31T07:31:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.109+0000 7f5c25400640 1 -- start start 2024-08-31T07:31:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.109+0000 7f5c25400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c20105960 0x7f5c201113a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.110+0000 7f5c25400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c2010b2a0 0x7f5c20111940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.110+0000 7f5c25400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2010bbf0 0x7f5c201a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.110+0000 7f5c25400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20078740 con 0x7f5c2010b2a0 2024-08-31T07:31:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.110+0000 7f5c25400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5c200785c0 con 0x7f5c2010bbf0 2024-08-31T07:31:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.110+0000 7f5c25400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5c200788c0 con 0x7f5c20105960 2024-08-31T07:31:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.110+0000 7f5c1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c2010b2a0 0x7f5c20111940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.110+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c20105960 0x7f5c201113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.110+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c20105960 0x7f5c201113a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42184/0 (socket says 172.21.15.32:42184) 2024-08-31T07:31:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.110+0000 7f5c1fe00640 1 -- 172.21.15.32:0/1309673354 learned_addr learned my addr 172.21.15.32:0/1309673354 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.110+0000 7f5c24a00640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2010bbf0 0x7f5c201a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.111+0000 7f5c1fe00640 1 -- 172.21.15.32:0/1309673354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2010bbf0 msgr2=0x7f5c201a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.111+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2010bbf0 0x7f5c201a3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.111+0000 7f5c1fe00640 1 -- 172.21.15.32:0/1309673354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c2010b2a0 msgr2=0x7f5c20111940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.111+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c2010b2a0 0x7f5c20111940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.111+0000 7f5c1fe00640 1 -- 172.21.15.32:0/1309673354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c201a43f0 con 0x7f5c20105960 2024-08-31T07:31:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.111+0000 7f5c1f400640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c2010b2a0 0x7f5c20111940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:31:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.112+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c20105960 0x7f5c201113a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800e9e0 tx=0x7f5c1800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.112+0000 7f5c1cc00640 1 -- 172.21.15.32:0/1309673354 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800cd50 con 0x7f5c20105960 2024-08-31T07:31:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.112+0000 7f5c1cc00640 1 -- 172.21.15.32:0/1309673354 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c180035f0 con 0x7f5c20105960 2024-08-31T07:31:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.112+0000 7f5c25400640 1 -- 172.21.15.32:0/1309673354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c201a46e0 con 0x7f5c20105960 2024-08-31T07:31:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.112+0000 7f5c25400640 1 -- 172.21.15.32:0/1309673354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c201ad030 con 0x7f5c20105960 2024-08-31T07:31:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.112+0000 7f5c1cc00640 1 -- 172.21.15.32:0/1309673354 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18010690 con 0x7f5c20105960 2024-08-31T07:31:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.117+0000 7f5c2d7c2740 1 Processor -- start 2024-08-31T07:31:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.117+0000 7f5c2d7c2740 1 -- start start 2024-08-31T07:31:35.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.118+0000 7f5c1cc00640 1 -- 172.21.15.32:0/1309673354 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5c18010830 con 0x7f5c20105960 2024-08-31T07:31:35.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.118+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34f18350 0x55cb352be7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.118+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352bed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.118+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352bf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.118+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55cb35234750 con 0x55cb34ef7670 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.118+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cb35223ce0 con 0x55cb34f18350 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.118+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cb35227c90 con 0x55cb34eef490 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352bed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352bf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c24a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34f18350 0x55cb352be7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c24a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34f18350 0x55cb352be7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57470/0 (socket says 172.21.15.32:57470) 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c24a00640 1 -- 172.21.15.32:0/889600197 learned_addr learned my addr 172.21.15.32:0/889600197 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352bf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37984/0 (socket says 172.21.15.32:37984) 2024-08-31T07:31:35.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/889600197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352bed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42188/0 (socket says 172.21.15.32:42188) 2024-08-31T07:31:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c1fe00640 1 -- 172.21.15.32:0/889600197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34f18350 msgr2=0x55cb352be7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/889600197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34f18350 0x55cb352be7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c1fe00640 1 -- 172.21.15.32:0/889600197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 msgr2=0x55cb352bf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.119+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/889600197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352bf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.120+0000 7f5c1fe00640 1 -- 172.21.15.32:0/889600197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb34f83930 con 0x55cb34eef490 2024-08-31T07:31:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.120+0000 7f5c24a00640 1 --2- 172.21.15.32:0/889600197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34f18350 0x55cb352be7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.120+0000 7f5c1f400640 1 --2- 172.21.15.32:0/889600197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352bf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.120+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/889600197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352bed10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5c180e2f60 tx=0x7f5c180ebf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e021fd1e2e5d3cb6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.120+0000 7f5c1cc00640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4d50 0x7f5c040b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.120+0000 7f5c1cc00640 1 -- 172.21.15.32:0/1309673354 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f5c180e57c0 con 0x7f5c20105960 2024-08-31T07:31:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.120+0000 7f5c15200640 1 -- 172.21.15.32:0/889600197 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c180f4070 con 0x55cb34eef490 2024-08-31T07:31:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.121+0000 7f5c15200640 1 -- 172.21.15.32:0/889600197 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c180aba10 con 0x55cb34eef490 2024-08-31T07:31:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.121+0000 7f5c15200640 1 -- 172.21.15.32:0/889600197 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c180ab850 con 0x55cb34eef490 2024-08-31T07:31:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.121+0000 7f5c24a00640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4d50 0x7f5c040b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.122+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/889600197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 msgr2=0x55cb352bed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.122+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/889600197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352bed10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5c180e2f60 tx=0x7f5c180ebf10 comp rx=0 tx=0).stop 2024-08-31T07:31:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.122+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/889600197 shutdown_connections 2024-08-31T07:31:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.122+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/889600197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352bf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.122+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/889600197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352bed10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.122+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/889600197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34f18350 0x55cb352be7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.122+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/889600197 >> 172.21.15.32:0/889600197 conn(0x55cb34f0c740 msgr2=0x55cb352bc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.123+0000 7f5c24a00640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4d50 0x7f5c040b7260 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5c100099a0 tx=0x7f5c10032f00 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.123+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/889600197 shutdown_connections 2024-08-31T07:31:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.123+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/889600197 wait complete. 2024-08-31T07:31:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.123+0000 7f5c2d7c2740 1 -- start start 2024-08-31T07:31:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.124+0000 7f5c2d7c2740 1 Processor -- start 2024-08-31T07:31:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.124+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.124+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 0x55cb352ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.124+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34f18350 0x55cb352d62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.125+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55cb35086890 con 0x55cb34f18350 2024-08-31T07:31:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.125+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cb351aacd0 con 0x55cb34ef7670 2024-08-31T07:31:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.125+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cb352ccfb0 con 0x55cb34eef490 2024-08-31T07:31:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.125+0000 7f5c2d7c2740 10 client.0 _ll_register_callbacks cb 0x55cb3521eef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.125+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.125+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42196/0 (socket says 172.21.15.32:42196) 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.125+0000 7f5c1fe00640 1 -- 172.21.15.32:0/4177538808 learned_addr learned my addr 172.21.15.32:0/4177538808 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.125+0000 7f5c1f400640 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 0x55cb352ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.125+0000 7f5c24a00640 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34f18350 0x55cb352d62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.126+0000 7f5c1f400640 1 -- 172.21.15.32:0/4177538808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 msgr2=0x55cb352ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.126+0000 7f5c1f400640 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.126+0000 7f5c1f400640 1 -- 172.21.15.32:0/4177538808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34f18350 msgr2=0x55cb352d62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.126+0000 7f5c1f400640 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34f18350 0x55cb352d62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.126+0000 7f5c1f400640 1 -- 172.21.15.32:0/4177538808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb351f41b0 con 0x55cb34ef7670 2024-08-31T07:31:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.126+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:31:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.126+0000 7f5c1f400640 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 0x55cb352ca8e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5c2006b4a0 tx=0x7f5c0c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b967727787a130e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.126+0000 7f5c09600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:31:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.127+0000 7f5c09600640 1 -- 172.21.15.32:0/4177538808 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c011b80 con 0x55cb34ef7670 2024-08-31T07:31:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.127+0000 7f5c09600640 1 -- 172.21.15.32:0/4177538808 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f5c0c011d20 con 0x55cb34ef7670 2024-08-31T07:31:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.127+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4177538808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c180aba10 con 0x55cb34ef7670 2024-08-31T07:31:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.127+0000 7f5c09600640 1 -- 172.21.15.32:0/4177538808 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f5c0c004030 con 0x55cb34ef7670 2024-08-31T07:31:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.127+0000 7f5c2d7c2740 10 client.29228 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.127+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4177538808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55cb352d9030 con 0x55cb34ef7670 2024-08-31T07:31:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.128+0000 7f5c09600640 1 -- 172.21.15.32:0/4177538808 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c180aba10 con 0x55cb34ef7670 2024-08-31T07:31:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.128+0000 7f5c09600640 1 -- 172.21.15.32:0/4177538808 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f5c0c01c460 con 0x55cb34ef7670 2024-08-31T07:31:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.129+0000 7f5c2d7c2740 10 client.29228 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.129+0000 7f5c2d7c2740 10 client.29228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:31:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.129+0000 7f5c2d7c2740 20 client.29228 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.129+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cb34ed6cd0 0x55cb352dc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.129+0000 7f5c2d7c2740 4 client.29228 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:31:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.129+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4177538808 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb352366a0 con 0x55cb34ed6cd0 2024-08-31T07:31:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.130+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cb34ed6cd0 0x55cb352dc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.131+0000 7f5c09600640 10 client.29228 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.131+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cb34ed6cd0 0x55cb352dc550 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=988cd1bc280a7e1 server_cookie=a8bfaaf1dbe3b897 in_seq=0 out_seq=0 2024-08-31T07:31:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.133+0000 7f5c09600640 1 -- 172.21.15.32:0/4177538808 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb352366a0 con 0x55cb34ed6cd0 2024-08-31T07:31:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.133+0000 7f5c09600640 10 client.29228 handle_command_reply: tid=0 2024-08-31T07:31:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.134+0000 7f5c2d7c2740 1 client.29228 shutdown 2024-08-31T07:31:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.134+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4177538808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 msgr2=0x55cb352ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.134+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 0x55cb352ca8e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5c2006b4a0 tx=0x7f5c0c012b60 comp rx=0 tx=0).stop 2024-08-31T07:31:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.134+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4177538808 shutdown_connections 2024-08-31T07:31:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.134+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cb34ed6cd0 0x55cb352dc550 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.134+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34f18350 0x55cb352d62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.134+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 0x55cb352ca8e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.134+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/4177538808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.134+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4177538808 >> 172.21.15.32:0/4177538808 conn(0x55cb34f0c740 msgr2=0x55cb352c19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.134+0000 7f5c09600640 0 client.29228 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.135+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4177538808 shutdown_connections 2024-08-31T07:31:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.135+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4177538808 wait complete. 2024-08-31T07:31:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.135+0000 7f5c2d7c2740 20 client.29228 trim_cache size 0 max 16384 2024-08-31T07:31:35.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.161+0000 7f5c2d7c2740 1 Processor -- start 2024-08-31T07:31:35.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.161+0000 7f5c2d7c2740 1 -- start start 2024-08-31T07:31:35.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34eef490 0x55cb352c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34f18350 0x55cb352c7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b58f0 con 0x55cb34ef7670 2024-08-31T07:31:35.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b7490 con 0x55cb34eef490 2024-08-31T07:31:35.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b5ff0 con 0x55cb34f18350 2024-08-31T07:31:35.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34eef490 0x55cb352c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34eef490 0x55cb352c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:57498/0 (socket says 172.21.15.32:57498) 2024-08-31T07:31:35.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c1fe00640 1 -- 172.21.15.32:0/4249169559 learned_addr learned my addr 172.21.15.32:0/4249169559 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.162+0000 7f5c1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37998/0 (socket says 172.21.15.32:37998) 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.163+0000 7f5c24a00640 1 --2- 172.21.15.32:0/4249169559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34f18350 0x55cb352c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.163+0000 7f5c1fe00640 1 -- 172.21.15.32:0/4249169559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34f18350 msgr2=0x55cb352c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.163+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/4249169559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34f18350 0x55cb352c7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.163+0000 7f5c1fe00640 1 -- 172.21.15.32:0/4249169559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 msgr2=0x55cb352cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.163+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/4249169559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.163+0000 7f5c1fe00640 1 -- 172.21.15.32:0/4249169559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb352dc8c0 con 0x55cb34eef490 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.163+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/4249169559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34eef490 0x55cb352c48f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800e9b0 tx=0x7f5c18009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52488570e540c0aa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.164+0000 7f5c0be00640 1 -- 172.21.15.32:0/4249169559 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c180f4070 con 0x55cb34eef490 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.164+0000 7f5c0be00640 1 -- 172.21.15.32:0/4249169559 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c180aba10 con 0x55cb34eef490 2024-08-31T07:31:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.164+0000 7f5c0be00640 1 -- 172.21.15.32:0/4249169559 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c180ab670 con 0x55cb34eef490 2024-08-31T07:31:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.165+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4249169559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34eef490 msgr2=0x55cb352c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.165+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/4249169559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34eef490 0x55cb352c48f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800e9b0 tx=0x7f5c18009310 comp rx=0 tx=0).stop 2024-08-31T07:31:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.165+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4249169559 shutdown_connections 2024-08-31T07:31:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.165+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/4249169559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34f18350 0x55cb352c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.165+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/4249169559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34ef7670 0x55cb352cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.165+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/4249169559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34eef490 0x55cb352c48f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.165+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4249169559 >> 172.21.15.32:0/4249169559 conn(0x55cb34f0c740 msgr2=0x55cb352cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.165+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4249169559 shutdown_connections 2024-08-31T07:31:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.165+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/4249169559 wait complete. 2024-08-31T07:31:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.166+0000 7f5c2d7c2740 1 -- start start 2024-08-31T07:31:35.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.166+0000 7f5c2d7c2740 1 Processor -- start 2024-08-31T07:31:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 0x55cb352e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c2d7c2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34f18350 0x55cb352e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b6dc0 con 0x55cb34f18350 2024-08-31T07:31:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b66f0 con 0x55cb34ef7670 2024-08-31T07:31:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c2d7c2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201b38f0 con 0x55cb34eef490 2024-08-31T07:31:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c2d7c2740 10 client.0 _ll_register_callbacks cb 0x7f5c0c009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:35.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c1fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352e0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42210/0 (socket says 172.21.15.32:42210) 2024-08-31T07:31:35.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c1fe00640 1 -- 172.21.15.32:0/2958939788 learned_addr learned my addr 172.21.15.32:0/2958939788 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:35.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c1f400640 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 0x55cb352e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.167+0000 7f5c24a00640 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34f18350 0x55cb352e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.168+0000 7f5c1fe00640 1 -- 172.21.15.32:0/2958939788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 msgr2=0x55cb352e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.168+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 0x55cb352e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.168+0000 7f5c1fe00640 1 -- 172.21.15.32:0/2958939788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34f18350 msgr2=0x55cb352e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.168+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34f18350 0x55cb352e1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.168+0000 7f5c1fe00640 1 -- 172.21.15.32:0/2958939788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb35346e00 con 0x55cb34eef490 2024-08-31T07:31:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.168+0000 7f5c1fe00640 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352e0b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5c180e3560 tx=0x7f5c180eb730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9102750f2c9ee39f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.168+0000 7f5c09600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:31:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.168+0000 7f5c09600640 1 -- 172.21.15.32:0/2958939788 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c180f4070 con 0x55cb34eef490 2024-08-31T07:31:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.169+0000 7f5c09600640 1 -- 172.21.15.32:0/2958939788 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f5c180037a0 con 0x55cb34eef490 2024-08-31T07:31:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.169+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/2958939788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c180aba10 con 0x55cb34eef490 2024-08-31T07:31:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.169+0000 7f5c09600640 1 -- 172.21.15.32:0/2958939788 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f5c180ab670 con 0x55cb34eef490 2024-08-31T07:31:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.169+0000 7f5c2d7c2740 10 client.29227 fetch_fsmap learned FSMap version 67 2024-08-31T07:31:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.169+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/2958939788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55cb35348090 con 0x55cb34eef490 2024-08-31T07:31:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.170+0000 7f5c09600640 1 -- 172.21.15.32:0/2958939788 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c180aba10 con 0x55cb34eef490 2024-08-31T07:31:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.170+0000 7f5c09600640 1 -- 172.21.15.32:0/2958939788 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7f5c180e31e0 con 0x55cb34eef490 2024-08-31T07:31:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.170+0000 7f5c2d7c2740 10 client.29227 fetch_fsmap finished waiting for FSMap version 67 2024-08-31T07:31:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.170+0000 7f5c2d7c2740 10 client.29227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:31:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.170+0000 7f5c2d7c2740 20 client.29227 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.170+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cb34eac080 0x55cb3534a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.170+0000 7f5c2d7c2740 4 client.29227 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:31:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.170+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/2958939788 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cb35232140 con 0x55cb34eac080 2024-08-31T07:31:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.172+0000 7f5c1f400640 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cb34eac080 0x55cb3534a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.172+0000 7f5c1f400640 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cb34eac080 0x55cb3534a270 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c3c666d1206fd6 server_cookie=96554234158824c7 in_seq=0 out_seq=0 2024-08-31T07:31:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.173+0000 7f5c09600640 10 client.29227 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.176+0000 7f5c09600640 1 -- 172.21.15.32:0/2958939788 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cb35232140 con 0x55cb34eac080 2024-08-31T07:31:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.176+0000 7f5c09600640 10 client.29227 handle_command_reply: tid=0 2024-08-31T07:31:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.176+0000 7f5c2d7c2740 1 client.29227 shutdown 2024-08-31T07:31:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/2958939788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 msgr2=0x55cb352e0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352e0b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5c180e3560 tx=0x7f5c180eb730 comp rx=0 tx=0).stop 2024-08-31T07:31:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/2958939788 shutdown_connections 2024-08-31T07:31:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55cb34eac080 0x55cb3534a270 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cb34f18350 0x55cb352e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb34ef7670 0x55cb352e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c2d7c2740 1 --2- 172.21.15.32:0/2958939788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb34eef490 0x55cb352e0b80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c09600640 0 client.29227 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/2958939788 >> 172.21.15.32:0/2958939788 conn(0x55cb34f0c740 msgr2=0x55cb352c01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/2958939788 shutdown_connections 2024-08-31T07:31:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.177+0000 7f5c2d7c2740 1 -- 172.21.15.32:0/2958939788 wait complete. 2024-08-31T07:31:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.178+0000 7f5c2d7c2740 20 client.29227 trim_cache size 0 max 16384 2024-08-31T07:31:35.179 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:31:35.179 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:31:35.180 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3", 2024-08-31T07:31:35.180 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:31:35.180 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:31:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.179+0000 7f5c25400640 1 -- 172.21.15.32:0/1309673354 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4d50 msgr2=0x7f5c040b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.179+0000 7f5c25400640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4d50 0x7f5c040b7260 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5c100099a0 tx=0x7f5c10032f00 comp rx=0 tx=0).stop 2024-08-31T07:31:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.179+0000 7f5c25400640 1 -- 172.21.15.32:0/1309673354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c20105960 msgr2=0x7f5c201113a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.179+0000 7f5c25400640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c20105960 0x7f5c201113a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800e9e0 tx=0x7f5c1800eea0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.179+0000 7f5c25400640 1 -- 172.21.15.32:0/1309673354 shutdown_connections 2024-08-31T07:31:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.179+0000 7f5c25400640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4d50 0x7f5c040b7260 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.179+0000 7f5c25400640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c2010bbf0 0x7f5c201a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.179+0000 7f5c25400640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c2010b2a0 0x7f5c20111940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.179+0000 7f5c25400640 1 --2- 172.21.15.32:0/1309673354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c20105960 0x7f5c201113a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.179+0000 7f5c25400640 1 -- 172.21.15.32:0/1309673354 >> 172.21.15.32:0/1309673354 conn(0x7f5c201013d0 msgr2=0x7f5c2007aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.180+0000 7f5c25400640 1 -- 172.21.15.32:0/1309673354 shutdown_connections 2024-08-31T07:31:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:35.180+0000 7f5c25400640 1 -- 172.21.15.32:0/1309673354 wait complete. 2024-08-31T07:31:35.395 INFO:tasks.workunit.client.0.smithi032.stdout: 10 95979 12.68 MB/sec execute 195 sec latency 4214.421 ms 2024-08-31T07:31:35.475 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:35 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:31:35.476 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:35 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:35.476 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:35 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:31:35.476 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:35 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:31:35.476 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:35 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:31:35.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:35 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:31:35.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:35 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:35.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:35 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:31:35.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:35 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:31:35.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:35 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:31:35.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:35 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:31:35.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:35 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:35.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:35 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:31:35.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:35 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:31:35.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:35 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:31:36.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:31:36.395 INFO:tasks.workunit.client.0.smithi032.stdout: 10 96442 12.74 MB/sec execute 196 sec latency 5214.471 ms 2024-08-31T07:31:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:36 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:31:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:36 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:31:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:36 smithi073 ceph-mon[21995]: pgmap v959: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-08-31T07:31:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:36 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:36 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:31:36.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:36 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:36 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:36.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:36 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:36 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:36 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:36 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:31:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:36 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:31:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:36 smithi084 ceph-mon[22387]: pgmap v959: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-08-31T07:31:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:36 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:36 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:31:36.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:36 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:36 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:36.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:36 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:36 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:36 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.761 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:31:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:36 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:31:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:36 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:31:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:36 smithi032 ceph-mon[17252]: pgmap v959: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-08-31T07:31:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:36 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:36 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:31:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:36 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:36 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:36.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:36 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:36 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:36.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:36 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:37.395 INFO:tasks.workunit.client.0.smithi032.stdout: 10 97207 12.88 MB/sec execute 197 sec latency 6214.533 ms 2024-08-31T07:31:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.636+0000 7f885da00640 1 -- 172.21.15.32:0/2835550525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8858105960 msgr2=0x7f8858105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.636+0000 7f885da00640 1 --2- 172.21.15.32:0/2835550525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8858105960 0x7f8858105d70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8848009900 tx=0x7f884802a410 comp rx=0 tx=0).stop 2024-08-31T07:31:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.637+0000 7f885da00640 1 -- 172.21.15.32:0/2835550525 shutdown_connections 2024-08-31T07:31:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.637+0000 7f885da00640 1 --2- 172.21.15.32:0/2835550525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885810bbf0 0x7f8858112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.637+0000 7f885da00640 1 --2- 172.21.15.32:0/2835550525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f885810b2a0 0x7f885810b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.637+0000 7f885da00640 1 --2- 172.21.15.32:0/2835550525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8858105960 0x7f8858105d70 secure :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8848009900 tx=0x7f884802a410 comp rx=0 tx=0).stop 2024-08-31T07:31:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.637+0000 7f885da00640 1 -- 172.21.15.32:0/2835550525 >> 172.21.15.32:0/2835550525 conn(0x7f88581013d0 msgr2=0x7f8858103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.638+0000 7f885da00640 1 -- 172.21.15.32:0/2835550525 shutdown_connections 2024-08-31T07:31:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.638+0000 7f885da00640 1 -- 172.21.15.32:0/2835550525 wait complete. 2024-08-31T07:31:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.639+0000 7f885da00640 1 Processor -- start 2024-08-31T07:31:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.640+0000 7f885da00640 1 -- start start 2024-08-31T07:31:37.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.640+0000 7f885da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885810b2a0 0x7f8858112340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.640+0000 7f885da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f885810bbf0 0x7f8858110580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.641+0000 7f885da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8858110b60 0x7f88581ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.641+0000 7f885da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8858077a50 con 0x7f885810bbf0 2024-08-31T07:31:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.641+0000 7f885da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f88580778d0 con 0x7f8858110b60 2024-08-31T07:31:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.641+0000 7f885da00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8858077bd0 con 0x7f885810b2a0 2024-08-31T07:31:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.641+0000 7f885d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8858110b60 0x7f88581ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.641+0000 7f885d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8858110b60 0x7f88581ac9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53096/0 (socket says 172.21.15.32:53096) 2024-08-31T07:31:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.641+0000 7f885d000640 1 -- 172.21.15.32:0/1618522707 learned_addr learned my addr 172.21.15.32:0/1618522707 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.641+0000 7f8857e00640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885810b2a0 0x7f8858112340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.641+0000 7f885d000640 1 -- 172.21.15.32:0/1618522707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885810b2a0 msgr2=0x7f8858112340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.641+0000 7f885d000640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885810b2a0 0x7f8858112340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.642+0000 7f885d000640 1 -- 172.21.15.32:0/1618522707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f885810bbf0 msgr2=0x7f8858110580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.642+0000 7f885d000640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f885810bbf0 0x7f8858110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.642+0000 7f885d000640 1 -- 172.21.15.32:0/1618522707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8858111340 con 0x7f8858110b60 2024-08-31T07:31:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.642+0000 7f8857e00640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885810b2a0 0x7f8858112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:31:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.642+0000 7f885d000640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8858110b60 0x7f88581ac9e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f885000eb10 tx=0x7f885000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.642+0000 7f8854c00640 1 -- 172.21.15.32:0/1618522707 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850004b20 con 0x7f8858110b60 2024-08-31T07:31:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.642+0000 7f8854c00640 1 -- 172.21.15.32:0/1618522707 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8850004cc0 con 0x7f8858110b60 2024-08-31T07:31:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.642+0000 7f8854c00640 1 -- 172.21.15.32:0/1618522707 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850010670 con 0x7f8858110b60 2024-08-31T07:31:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.642+0000 7f885da00640 1 -- 172.21.15.32:0/1618522707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88581acfe0 con 0x7f8858110b60 2024-08-31T07:31:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.642+0000 7f885da00640 1 -- 172.21.15.32:0/1618522707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88581ad3e0 con 0x7f8858110b60 2024-08-31T07:31:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.646+0000 7f8865dbe740 1 Processor -- start 2024-08-31T07:31:37.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.648+0000 7f8854c00640 1 -- 172.21.15.32:0/1618522707 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8850010830 con 0x7f8858110b60 2024-08-31T07:31:37.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.648+0000 7f8865dbe740 1 -- start start 2024-08-31T07:31:37.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.649+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 0x55c93a293b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.649+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 0x55c93a294060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.649+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 0x55c93a2945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.649+0000 7f8865dbe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c93a20a770 con 0x55c939ec51b0 2024-08-31T07:31:37.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.649+0000 7f8865dbe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c93a1f9d00 con 0x55c939ecd390 2024-08-31T07:31:37.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.649+0000 7f8865dbe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c93a1fdcb0 con 0x55c939eee070 2024-08-31T07:31:37.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.650+0000 7f8857e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 0x55c93a294060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.650+0000 7f8857e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 0x55c93a294060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52574/0 (socket says 172.21.15.32:52574) 2024-08-31T07:31:37.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.650+0000 7f8857e00640 1 -- 172.21.15.32:0/3184537164 learned_addr learned my addr 172.21.15.32:0/3184537164 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:37.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.651+0000 7f8854c00640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f883c0b4e70 0x7f883c0b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.651+0000 7f8857e00640 1 -- 172.21.15.32:0/3184537164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 msgr2=0x55c93a293b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:31:37.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.651+0000 7f8857400640 1 --2- 172.21.15.32:0/3184537164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 0x55c93a293b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.651+0000 7f8854c00640 1 -- 172.21.15.32:0/1618522707 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f88500e5850 con 0x7f8858110b60 2024-08-31T07:31:37.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.651+0000 7f8857e00640 1 --2- 172.21.15.32:0/3184537164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 0x55c93a293b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.651+0000 7f8857e00640 1 -- 172.21.15.32:0/3184537164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 msgr2=0x55c93a2945a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:31:37.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.651+0000 7f8857e00640 1 --2- 172.21.15.32:0/3184537164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 0x55c93a2945a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.651+0000 7f8857e00640 1 -- 172.21.15.32:0/3184537164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c939f59650 con 0x55c939ec51b0 2024-08-31T07:31:37.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.651+0000 7f8857400640 1 --2- 172.21.15.32:0/3184537164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 0x55c93a293b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:31:37.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.652+0000 7f885d000640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f883c0b4e70 0x7f883c0b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.652+0000 7f8857e00640 1 --2- 172.21.15.32:0/3184537164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 0x55c93a294060 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8848006010 tx=0x7f88480042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808790e886592c17 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:37.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.653+0000 7f885d000640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f883c0b4e70 0x7f883c0b7380 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f88500049d0 tx=0x7f8850005a80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:37.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.653+0000 7f884d600640 1 -- 172.21.15.32:0/3184537164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884803f020 con 0x55c939ec51b0 2024-08-31T07:31:37.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.653+0000 7f884d600640 1 -- 172.21.15.32:0/3184537164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88480044d0 con 0x55c939ec51b0 2024-08-31T07:31:37.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.653+0000 7f884d600640 1 -- 172.21.15.32:0/3184537164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848035070 con 0x55c939ec51b0 2024-08-31T07:31:37.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.654+0000 7f8865dbe740 1 -- 172.21.15.32:0/3184537164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 msgr2=0x55c93a294060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:37.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.654+0000 7f8865dbe740 1 --2- 172.21.15.32:0/3184537164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 0x55c93a294060 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8848006010 tx=0x7f88480042e0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.655+0000 7f8865dbe740 1 -- 172.21.15.32:0/3184537164 shutdown_connections 2024-08-31T07:31:37.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.655+0000 7f8865dbe740 1 --2- 172.21.15.32:0/3184537164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 0x55c93a2945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.655+0000 7f8865dbe740 1 --2- 172.21.15.32:0/3184537164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 0x55c93a294060 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.655+0000 7f8865dbe740 1 --2- 172.21.15.32:0/3184537164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 0x55c93a293b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.655+0000 7f8865dbe740 1 -- 172.21.15.32:0/3184537164 >> 172.21.15.32:0/3184537164 conn(0x55c939ee2460 msgr2=0x55c93a291a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:37.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.656+0000 7f8865dbe740 1 -- 172.21.15.32:0/3184537164 shutdown_connections 2024-08-31T07:31:37.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.656+0000 7f8865dbe740 1 -- 172.21.15.32:0/3184537164 wait complete. 2024-08-31T07:31:37.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.657+0000 7f8865dbe740 1 -- start start 2024-08-31T07:31:37.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.657+0000 7f8865dbe740 1 Processor -- start 2024-08-31T07:31:37.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.658+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ec51b0 0x55c93a29f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.658+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939ecd390 0x55c93a29fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.658+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939eee070 0x55c93a2ab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.658+0000 7f8865dbe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c93a05c700 con 0x55c939eee070 2024-08-31T07:31:37.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.658+0000 7f8865dbe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c93a180cb0 con 0x55c939ec51b0 2024-08-31T07:31:37.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.658+0000 7f8865dbe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c93a2a22a0 con 0x55c939ecd390 2024-08-31T07:31:37.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.658+0000 7f8865dbe740 10 client.0 _ll_register_callbacks cb 0x55c93a1a6430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:37.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.659+0000 7f8857400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ec51b0 0x55c93a29f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.659+0000 7f8857400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ec51b0 0x55c93a29f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53120/0 (socket says 172.21.15.32:53120) 2024-08-31T07:31:37.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.659+0000 7f8857400640 1 -- 172.21.15.32:0/1148085184 learned_addr learned my addr 172.21.15.32:0/1148085184 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:37.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.659+0000 7f8857400640 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939eee070 0x55c93a2ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.659+0000 7f8857400640 1 -- 172.21.15.32:0/1148085184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939ecd390 msgr2=0x55c93a29fcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:31:37.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.659+0000 7f8857400640 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939ecd390 0x55c93a29fcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.659+0000 7f8857400640 1 -- 172.21.15.32:0/1148085184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939eee070 msgr2=0x55c93a2ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:37.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.659+0000 7f8857400640 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939eee070 0x55c93a2ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.659+0000 7f8857400640 1 -- 172.21.15.32:0/1148085184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c93a1c1870 con 0x55c939ec51b0 2024-08-31T07:31:37.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.660+0000 7f8857400640 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ec51b0 0x55c93a29f770 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8844013cc0 tx=0x7f884400b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5c9c8488dd46a3d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:37.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.660+0000 7f8842000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:31:37.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.660+0000 7f8842000640 1 -- 172.21.15.32:0/1148085184 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884401e430 con 0x55c939ec51b0 2024-08-31T07:31:37.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.661+0000 7f8842000640 1 -- 172.21.15.32:0/1148085184 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f884401e5d0 con 0x55c939ec51b0 2024-08-31T07:31:37.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.661+0000 7f8865dbe740 1 -- 172.21.15.32:0/1148085184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88480044d0 con 0x55c939ec51b0 2024-08-31T07:31:37.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.661+0000 7f8842000640 1 -- 172.21.15.32:0/1148085184 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f884401e7d0 con 0x55c939ec51b0 2024-08-31T07:31:37.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.661+0000 7f8865dbe740 10 client.29243 fetch_fsmap learned FSMap version 68 2024-08-31T07:31:37.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.661+0000 7f8865dbe740 1 -- 172.21.15.32:0/1148085184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55c93a2ae2f0 con 0x55c939ec51b0 2024-08-31T07:31:37.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.662+0000 7f8842000640 1 -- 172.21.15.32:0/1148085184 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88480044d0 con 0x55c939ec51b0 2024-08-31T07:31:37.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.663+0000 7f8842000640 1 -- 172.21.15.32:0/1148085184 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 68) ==== 8386+0+0 (secure 0 0 0) 0x7f884400f040 con 0x55c939ec51b0 2024-08-31T07:31:37.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.663+0000 7f8865dbe740 10 client.29243 fetch_fsmap finished waiting for FSMap version 68 2024-08-31T07:31:37.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.663+0000 7f8865dbe740 10 client.29243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:31:37.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.663+0000 7f8865dbe740 20 client.29243 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:37.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.663+0000 7f8865dbe740 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c939eac9f0 0x55c93a2b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.663+0000 7f8865dbe740 4 client.29243 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:31:37.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.663+0000 7f8865dbe740 1 -- 172.21.15.32:0/1148085184 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c93a20c6c0 con 0x55c939eac9f0 2024-08-31T07:31:37.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.664+0000 7f8857e00640 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c939eac9f0 0x55c93a2b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.665+0000 7f8857e00640 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c939eac9f0 0x55c93a2b1770 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c21a8418ad46102c server_cookie=44587c04890b7381 in_seq=0 out_seq=0 2024-08-31T07:31:37.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.665+0000 7f8842000640 10 client.29243 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:37.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.670+0000 7f8842000640 1 -- 172.21.15.32:0/1148085184 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c93a20c6c0 con 0x55c939eac9f0 2024-08-31T07:31:37.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.670+0000 7f8842000640 10 client.29243 handle_command_reply: tid=0 2024-08-31T07:31:37.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.670+0000 7f8865dbe740 1 client.29243 shutdown 2024-08-31T07:31:37.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8865dbe740 1 -- 172.21.15.32:0/1148085184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ec51b0 msgr2=0x55c93a29f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:37.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8865dbe740 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ec51b0 0x55c93a29f770 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8844013cc0 tx=0x7f884400b400 comp rx=0 tx=0).stop 2024-08-31T07:31:37.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8865dbe740 1 -- 172.21.15.32:0/1148085184 shutdown_connections 2024-08-31T07:31:37.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8865dbe740 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c939eac9f0 0x55c93a2b1770 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8865dbe740 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939eee070 0x55c93a2ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8865dbe740 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939ecd390 0x55c93a29fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8865dbe740 1 --2- 172.21.15.32:0/1148085184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ec51b0 0x55c93a29f770 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8842000640 0 client.29243 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:37.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8865dbe740 1 -- 172.21.15.32:0/1148085184 >> 172.21.15.32:0/1148085184 conn(0x55c939ee2460 msgr2=0x55c93a296cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:37.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8865dbe740 1 -- 172.21.15.32:0/1148085184 shutdown_connections 2024-08-31T07:31:37.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.672+0000 7f8865dbe740 1 -- 172.21.15.32:0/1148085184 wait complete. 2024-08-31T07:31:37.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.673+0000 7f8865dbe740 20 client.29243 trim_cache size 0 max 16384 2024-08-31T07:31:37.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.704+0000 7f8865dbe740 1 Processor -- start 2024-08-31T07:31:37.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.704+0000 7f8865dbe740 1 -- start start 2024-08-31T07:31:37.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.705+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939ec51b0 0x55c93a299340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.705+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ecd390 0x55c93a290fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.705+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939eee070 0x55c93a291500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.705+0000 7f8865dbe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f883800e6b0 con 0x55c939ecd390 2024-08-31T07:31:37.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.705+0000 7f8865dbe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f88380102e0 con 0x55c939eee070 2024-08-31T07:31:37.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.705+0000 7f8865dbe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f883800edb0 con 0x55c939ec51b0 2024-08-31T07:31:37.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.706+0000 7f8857400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939ec51b0 0x55c93a299340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.706+0000 7f8857400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939ec51b0 0x55c93a299340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47966/0 (socket says 172.21.15.32:47966) 2024-08-31T07:31:37.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.706+0000 7f8857400640 1 -- 172.21.15.32:0/1830629658 learned_addr learned my addr 172.21.15.32:0/1830629658 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.706+0000 7f8857400640 1 --2- 172.21.15.32:0/1830629658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939eee070 0x55c93a291500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.706+0000 7f8857e00640 1 --2- 172.21.15.32:0/1830629658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ecd390 0x55c93a290fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.707+0000 7f8857400640 1 -- 172.21.15.32:0/1830629658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939ec51b0 msgr2=0x55c93a299340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.707+0000 7f8857400640 1 --2- 172.21.15.32:0/1830629658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939ec51b0 0x55c93a299340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.707+0000 7f8857400640 1 -- 172.21.15.32:0/1830629658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ecd390 msgr2=0x55c93a290fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.707+0000 7f8857400640 1 --2- 172.21.15.32:0/1830629658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ecd390 0x55c93a290fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.707+0000 7f8857400640 1 -- 172.21.15.32:0/1830629658 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c93a2a0550 con 0x55c939eee070 2024-08-31T07:31:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.707+0000 7f8857400640 1 --2- 172.21.15.32:0/1830629658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939eee070 0x55c93a291500 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8844013840 tx=0x7f88440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce2ee3b7982b593 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.708+0000 7f884cc00640 1 -- 172.21.15.32:0/1830629658 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884401e630 con 0x55c939eee070 2024-08-31T07:31:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.708+0000 7f884cc00640 1 -- 172.21.15.32:0/1830629658 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884402f070 con 0x55c939eee070 2024-08-31T07:31:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.708+0000 7f884cc00640 1 -- 172.21.15.32:0/1830629658 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8844005020 con 0x55c939eee070 2024-08-31T07:31:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.709+0000 7f8865dbe740 1 -- 172.21.15.32:0/1830629658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939eee070 msgr2=0x55c93a291500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.709+0000 7f8865dbe740 1 --2- 172.21.15.32:0/1830629658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939eee070 0x55c93a291500 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8844013840 tx=0x7f88440046c0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.710+0000 7f8865dbe740 1 -- 172.21.15.32:0/1830629658 shutdown_connections 2024-08-31T07:31:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.710+0000 7f8865dbe740 1 --2- 172.21.15.32:0/1830629658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939eee070 0x55c93a291500 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.710+0000 7f8865dbe740 1 --2- 172.21.15.32:0/1830629658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ecd390 0x55c93a290fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.710+0000 7f8865dbe740 1 --2- 172.21.15.32:0/1830629658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939ec51b0 0x55c93a299340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.710+0000 7f8865dbe740 1 -- 172.21.15.32:0/1830629658 >> 172.21.15.32:0/1830629658 conn(0x55c939ee2460 msgr2=0x55c93a298c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.710+0000 7f8865dbe740 1 -- 172.21.15.32:0/1830629658 shutdown_connections 2024-08-31T07:31:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.710+0000 7f8865dbe740 1 -- 172.21.15.32:0/1830629658 wait complete. 2024-08-31T07:31:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.711+0000 7f8865dbe740 1 -- start start 2024-08-31T07:31:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.711+0000 7f8865dbe740 1 Processor -- start 2024-08-31T07:31:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.712+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 0x55c93a2a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.712+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 0x55c93a2a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.712+0000 7f8865dbe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 0x55c93a31c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.712+0000 7f8865dbe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f883800fc10 con 0x55c939ec51b0 2024-08-31T07:31:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.712+0000 7f8865dbe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f883800f540 con 0x55c939ecd390 2024-08-31T07:31:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.712+0000 7f8865dbe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f883800c6b0 con 0x55c939eee070 2024-08-31T07:31:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.712+0000 7f8865dbe740 10 client.0 _ll_register_callbacks cb 0x7f8844013900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:31:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.712+0000 7f8857e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 0x55c93a2a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.713+0000 7f8857e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 0x55c93a2a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53136/0 (socket says 172.21.15.32:53136) 2024-08-31T07:31:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.713+0000 7f8857e00640 1 -- 172.21.15.32:0/2761787499 learned_addr learned my addr 172.21.15.32:0/2761787499 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:31:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.713+0000 7f8857400640 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 0x55c93a31c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.713+0000 7f8857e00640 1 -- 172.21.15.32:0/2761787499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 msgr2=0x55c93a31c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.713+0000 7f8857e00640 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 0x55c93a31c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.713+0000 7f8857e00640 1 -- 172.21.15.32:0/2761787499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 msgr2=0x55c93a2a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:31:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.713+0000 7f8857e00640 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 0x55c93a2a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.713+0000 7f8857e00640 1 -- 172.21.15.32:0/2761787499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c93a2a5540 con 0x55c939ecd390 2024-08-31T07:31:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.714+0000 7f8857e00640 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 0x55c93a2a4d30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8848005ab0 tx=0x7f8848005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=766987f1bb13e3a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:31:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.714+0000 7f8842000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:31:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.714+0000 7f8842000640 1 -- 172.21.15.32:0/2761787499 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884803f040 con 0x55c939ecd390 2024-08-31T07:31:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.715+0000 7f8842000640 1 -- 172.21.15.32:0/2761787499 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7f88480314a0 con 0x55c939ecd390 2024-08-31T07:31:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.715+0000 7f8865dbe740 1 -- 172.21.15.32:0/2761787499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f884402f070 con 0x55c939ecd390 2024-08-31T07:31:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.715+0000 7f8842000640 1 -- 172.21.15.32:0/2761787499 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f8848059030 con 0x55c939ecd390 2024-08-31T07:31:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.715+0000 7f8865dbe740 10 client.29249 fetch_fsmap learned FSMap version 68 2024-08-31T07:31:37.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.715+0000 7f8865dbe740 1 -- 172.21.15.32:0/2761787499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55c93a31d6f0 con 0x55c939ecd390 2024-08-31T07:31:37.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.717+0000 7f8842000640 1 -- 172.21.15.32:0/2761787499 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884402f070 con 0x55c939ecd390 2024-08-31T07:31:37.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.718+0000 7f8842000640 1 -- 172.21.15.32:0/2761787499 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 68) ==== 8386+0+0 (secure 0 0 0) 0x7f884803b510 con 0x55c939ecd390 2024-08-31T07:31:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.718+0000 7f8865dbe740 10 client.29249 fetch_fsmap finished waiting for FSMap version 68 2024-08-31T07:31:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.718+0000 7f8865dbe740 10 client.29249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:31:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.718+0000 7f8865dbe740 20 client.29249 populate_metadata read hostname 'smithi032' 2024-08-31T07:31:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.718+0000 7f8865dbe740 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c939e81da0 0x55c93a31fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:31:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.718+0000 7f8865dbe740 4 client.29249 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:31:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.718+0000 7f8865dbe740 1 -- 172.21.15.32:0/2761787499 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c93a208160 con 0x55c939e81da0 2024-08-31T07:31:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.718+0000 7f8857400640 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c939e81da0 0x55c93a31fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:31:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.719+0000 7f8842000640 10 client.29249 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.719+0000 7f8857400640 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c939e81da0 0x55c93a31fb50 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f21c8ee89a26eb server_cookie=2e9f6f52900e1878 in_seq=0 out_seq=0 2024-08-31T07:31:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.720+0000 7f8842000640 1 -- 172.21.15.32:0/2761787499 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6534 (crc 0 0 0) 0x55c93a208160 con 0x55c939e81da0 2024-08-31T07:31:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.720+0000 7f8842000640 10 client.29249 handle_command_reply: tid=0 2024-08-31T07:31:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.720+0000 7f8865dbe740 1 client.29249 shutdown 2024-08-31T07:31:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.721+0000 7f8865dbe740 1 -- 172.21.15.32:0/2761787499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 msgr2=0x55c93a2a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.721+0000 7f8865dbe740 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 0x55c93a2a4d30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8848005ab0 tx=0x7f8848005ec0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.721+0000 7f8865dbe740 1 -- 172.21.15.32:0/2761787499 shutdown_connections 2024-08-31T07:31:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.721+0000 7f8865dbe740 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c939e81da0 0x55c93a31fb50 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.721+0000 7f8865dbe740 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c939eee070 0x55c93a31c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.721+0000 7f8865dbe740 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c939ecd390 0x55c93a2a4d30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.721+0000 7f8865dbe740 1 --2- 172.21.15.32:0/2761787499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c939ec51b0 0x55c93a2a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.721+0000 7f8865dbe740 1 -- 172.21.15.32:0/2761787499 >> 172.21.15.32:0/2761787499 conn(0x55c939ee2460 msgr2=0x55c93a294ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.721+0000 7f8842000640 0 client.29249 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:31:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.722+0000 7f8865dbe740 1 -- 172.21.15.32:0/2761787499 shutdown_connections 2024-08-31T07:31:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.722+0000 7f8865dbe740 1 -- 172.21.15.32:0/2761787499 wait complete. 2024-08-31T07:31:37.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.722+0000 7f8865dbe740 20 client.29249 trim_cache size 0 max 16384 2024-08-31T07:31:37.726 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:31:37.726 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:31:37.726 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3", 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.729 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:31:37.731 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:31:37.733 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.737 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:31:37.740 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:31:37.741 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:31:37.741 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:31:37.741 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:31:37.741 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:31:37.741 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:31:37.741 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:31:37.741 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:31:37.741 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:31:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.723+0000 7f885da00640 1 -- 172.21.15.32:0/1618522707 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f883c0b4e70 msgr2=0x7f883c0b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.723+0000 7f885da00640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f883c0b4e70 0x7f883c0b7380 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f88500049d0 tx=0x7f8850005a80 comp rx=0 tx=0).stop 2024-08-31T07:31:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.723+0000 7f885da00640 1 -- 172.21.15.32:0/1618522707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8858110b60 msgr2=0x7f88581ac9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:31:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.723+0000 7f885da00640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8858110b60 0x7f88581ac9e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f885000eb10 tx=0x7f885000c2d0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.723+0000 7f885da00640 1 -- 172.21.15.32:0/1618522707 shutdown_connections 2024-08-31T07:31:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.723+0000 7f885da00640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f883c0b4e70 0x7f883c0b7380 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.723+0000 7f885da00640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8858110b60 0x7f88581ac9e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.723+0000 7f885da00640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f885810bbf0 0x7f8858110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.723+0000 7f885da00640 1 --2- 172.21.15.32:0/1618522707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885810b2a0 0x7f8858112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:31:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.723+0000 7f885da00640 1 -- 172.21.15.32:0/1618522707 >> 172.21.15.32:0/1618522707 conn(0x7f88581013d0 msgr2=0x7f885807a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:31:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.724+0000 7f885da00640 1 -- 172.21.15.32:0/1618522707 shutdown_connections 2024-08-31T07:31:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:31:37.724+0000 7f885da00640 1 -- 172.21.15.32:0/1618522707 wait complete. 2024-08-31T07:31:38.395 INFO:tasks.workunit.client.0.smithi032.stdout: 10 97897 13.00 MB/sec execute 198 sec latency 7214.613 ms 2024-08-31T07:31:38.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:38 smithi032 ceph-mon[17252]: pgmap v960: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-08-31T07:31:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:38 smithi073 ceph-mon[21995]: pgmap v960: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-08-31T07:31:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:38 smithi084 ceph-mon[22387]: pgmap v960: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-08-31T07:31:39.029 INFO:tasks.cephfs.filesystem:scrub status for tag:fb535dd7-5c86-4ca9-bd4e-7017aaa318b3 - {'path': '/', 'tag': 'fb535dd7-5c86-4ca9-bd4e-7017aaa318b3', 'options': 'recursive,force'} 2024-08-31T07:31:39.395 INFO:tasks.workunit.client.0.smithi032.stdout: 10 98588 13.15 MB/sec execute 199 sec latency 8214.666 ms 2024-08-31T07:31:40.396 INFO:tasks.workunit.client.0.smithi032.stdout: 10 99243 13.23 MB/sec execute 200 sec latency 9215.233 ms 2024-08-31T07:31:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:40 smithi073 ceph-mon[21995]: pgmap v961: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-08-31T07:31:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:40 smithi084 ceph-mon[22387]: pgmap v961: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-08-31T07:31:40.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:40 smithi032 ceph-mon[17252]: pgmap v961: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-08-31T07:31:41.396 INFO:tasks.workunit.client.0.smithi032.stdout: 10 99839 13.35 MB/sec execute 201 sec latency 10215.293 ms 2024-08-31T07:31:42.396 INFO:tasks.workunit.client.0.smithi032.stdout: 10 100607 13.50 MB/sec execute 202 sec latency 11215.350 ms 2024-08-31T07:31:42.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:42 smithi032 ceph-mon[17252]: pgmap v962: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.4 MiB/s wr, 390 op/s 2024-08-31T07:31:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:42 smithi073 ceph-mon[21995]: pgmap v962: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.4 MiB/s wr, 390 op/s 2024-08-31T07:31:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:42 smithi084 ceph-mon[22387]: pgmap v962: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.4 MiB/s wr, 390 op/s 2024-08-31T07:31:43.396 INFO:tasks.workunit.client.0.smithi032.stdout: 10 101220 13.54 MB/sec execute 203 sec latency 12215.407 ms 2024-08-31T07:31:44.396 INFO:tasks.workunit.client.0.smithi032.stdout: 10 101875 13.68 MB/sec execute 204 sec latency 13215.451 ms 2024-08-31T07:31:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:44 smithi073 ceph-mon[21995]: pgmap v963: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 496 op/s 2024-08-31T07:31:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:44 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:31:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:44 smithi084 ceph-mon[22387]: pgmap v963: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 496 op/s 2024-08-31T07:31:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:44 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:31:44.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:44 smithi032 ceph-mon[17252]: pgmap v963: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 496 op/s 2024-08-31T07:31:44.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:44 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:31:45.396 INFO:tasks.workunit.client.0.smithi032.stdout: 10 102577 13.80 MB/sec execute 205 sec latency 14025.032 ms 2024-08-31T07:31:45.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:31:45.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:31:45.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:31:45.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:45 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:45.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:31:45.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:31:45.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:31:45.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:45 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:45.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:31:45.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:31:45.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:31:45.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:45 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:46.396 INFO:tasks.workunit.client.0.smithi032.stdout: 10 103323 13.94 MB/sec execute 206 sec latency 191.288 ms 2024-08-31T07:31:47.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:46 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:31:47.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:46 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:47.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:46 smithi032 ceph-mon[17252]: pgmap v964: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 533 op/s 2024-08-31T07:31:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:46 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:31:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:46 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:47.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:46 smithi073 ceph-mon[21995]: pgmap v964: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 533 op/s 2024-08-31T07:31:47.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:46 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:31:47.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:46 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:31:47.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:46 smithi084 ceph-mon[22387]: pgmap v964: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 533 op/s 2024-08-31T07:31:47.396 INFO:tasks.workunit.client.0.smithi032.stdout: 10 103986 14.01 MB/sec execute 207 sec latency 74.316 ms 2024-08-31T07:31:48.396 INFO:tasks.workunit.client.0.smithi032.stdout: 10 104790 14.18 MB/sec execute 208 sec latency 82.699 ms 2024-08-31T07:31:49.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:48 smithi032 ceph-mon[17252]: pgmap v965: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.3 MiB/s wr, 647 op/s 2024-08-31T07:31:49.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:48 smithi073 ceph-mon[21995]: pgmap v965: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.3 MiB/s wr, 647 op/s 2024-08-31T07:31:49.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:48 smithi084 ceph-mon[22387]: pgmap v965: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.3 MiB/s wr, 647 op/s 2024-08-31T07:31:49.397 INFO:tasks.workunit.client.0.smithi032.stdout: 10 105561 14.29 MB/sec execute 209 sec latency 74.895 ms 2024-08-31T07:31:50.397 INFO:tasks.workunit.client.0.smithi032.stdout: 10 106429 14.48 MB/sec execute 210 sec latency 80.988 ms 2024-08-31T07:31:51.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:50 smithi032 ceph-mon[17252]: pgmap v966: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.3 MiB/s wr, 480 op/s 2024-08-31T07:31:51.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:50 smithi073 ceph-mon[21995]: pgmap v966: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.3 MiB/s wr, 480 op/s 2024-08-31T07:31:51.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:50 smithi084 ceph-mon[22387]: pgmap v966: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.3 MiB/s wr, 480 op/s 2024-08-31T07:31:51.397 INFO:tasks.workunit.client.0.smithi032.stdout: 10 107117 14.55 MB/sec execute 211 sec latency 99.988 ms 2024-08-31T07:31:52.397 INFO:tasks.workunit.client.0.smithi032.stdout: 10 107951 14.71 MB/sec execute 212 sec latency 95.052 ms 2024-08-31T07:31:53.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:52 smithi032 ceph-mon[17252]: pgmap v967: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 621 op/s 2024-08-31T07:31:53.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:52 smithi073 ceph-mon[21995]: pgmap v967: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 621 op/s 2024-08-31T07:31:53.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:52 smithi084 ceph-mon[22387]: pgmap v967: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 621 op/s 2024-08-31T07:31:53.398 INFO:tasks.workunit.client.0.smithi032.stdout: 10 108783 14.87 MB/sec execute 213 sec latency 70.324 ms 2024-08-31T07:31:54.398 INFO:tasks.workunit.client.0.smithi032.stdout: 10 109562 14.98 MB/sec execute 214 sec latency 98.820 ms 2024-08-31T07:31:55.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:54 smithi073 ceph-mon[21995]: pgmap v968: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.3 MiB/s wr, 609 op/s 2024-08-31T07:31:55.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:54 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:55.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:54 smithi084 ceph-mon[22387]: pgmap v968: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.3 MiB/s wr, 609 op/s 2024-08-31T07:31:55.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:54 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:55.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:54 smithi032 ceph-mon[17252]: pgmap v968: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.3 MiB/s wr, 609 op/s 2024-08-31T07:31:55.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:54 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:55.398 INFO:tasks.workunit.client.0.smithi032.stdout: 10 110177 15.04 MB/sec execute 215 sec latency 151.585 ms 2024-08-31T07:31:56.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 110894 15.16 MB/sec execute 216 sec latency 243.640 ms 2024-08-31T07:31:57.004 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:56 smithi032 ceph-mon[17252]: pgmap v969: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.8 MiB/s wr, 582 op/s 2024-08-31T07:31:57.004 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:56 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:57.005 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:31:57.005 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:31:57.005 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:31:57.005 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:31:57.005 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:56 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:31:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:56 smithi073 ceph-mon[21995]: pgmap v969: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.8 MiB/s wr, 582 op/s 2024-08-31T07:31:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:56 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:31:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:31:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:31:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:31:57.180 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:56 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:31:57.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:56 smithi084 ceph-mon[22387]: pgmap v969: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.8 MiB/s wr, 582 op/s 2024-08-31T07:31:57.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:56 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:57.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:31:57.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:31:57.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:31:57.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:31:57.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:56 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:31:57.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 110896 15.09 MB/sec execute 217 sec latency 1243.699 ms 2024-08-31T07:31:58.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:57 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:58.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:31:57 smithi073 ceph-mon[21995]: pgmap v970: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 733 op/s 2024-08-31T07:31:58.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:57 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:58.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:31:57 smithi084 ceph-mon[22387]: pgmap v970: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 733 op/s 2024-08-31T07:31:58.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:57 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:31:58.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:31:57 smithi032 ceph-mon[17252]: pgmap v970: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 733 op/s 2024-08-31T07:31:58.398 INFO:tasks.workunit.client.0.smithi032.stdout: 10 110928 15.03 MB/sec execute 218 sec latency 2243.749 ms 2024-08-31T07:31:59.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111023 14.99 MB/sec execute 219 sec latency 3243.802 ms 2024-08-31T07:32:00.332 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:00 smithi032 ceph-mon[17252]: pgmap v971: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.6 MiB/s wr, 580 op/s 2024-08-31T07:32:00.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111025 14.92 MB/sec execute 220 sec latency 4243.852 ms 2024-08-31T07:32:00.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:00 smithi073 ceph-mon[21995]: pgmap v971: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.6 MiB/s wr, 580 op/s 2024-08-31T07:32:00.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:00 smithi084 ceph-mon[22387]: pgmap v971: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.6 MiB/s wr, 580 op/s 2024-08-31T07:32:01.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111040 14.87 MB/sec execute 221 sec latency 5243.907 ms 2024-08-31T07:32:02.329 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:02 smithi032 ceph-mon[17252]: pgmap v972: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 628 op/s 2024-08-31T07:32:02.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.80 MB/sec execute 222 sec latency 6242.509 ms 2024-08-31T07:32:02.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:02 smithi073 ceph-mon[21995]: pgmap v972: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 628 op/s 2024-08-31T07:32:02.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:02 smithi084 ceph-mon[22387]: pgmap v972: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 628 op/s 2024-08-31T07:32:03.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.73 MB/sec execute 223 sec latency 7242.572 ms 2024-08-31T07:32:04.306 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:04 smithi032 ceph-mon[17252]: pgmap v973: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 515 op/s 2024-08-31T07:32:04.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.67 MB/sec execute 224 sec latency 8242.625 ms 2024-08-31T07:32:04.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:04 smithi073 ceph-mon[21995]: pgmap v973: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 515 op/s 2024-08-31T07:32:04.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:04 smithi084 ceph-mon[22387]: pgmap v973: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 515 op/s 2024-08-31T07:32:05.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.60 MB/sec execute 225 sec latency 9242.678 ms 2024-08-31T07:32:06.320 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:06 smithi032 ceph-mon[17252]: pgmap v974: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 550 op/s 2024-08-31T07:32:06.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.54 MB/sec execute 226 sec latency 10242.732 ms 2024-08-31T07:32:06.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:06 smithi073 ceph-mon[21995]: pgmap v974: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 550 op/s 2024-08-31T07:32:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:06 smithi084 ceph-mon[22387]: pgmap v974: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 550 op/s 2024-08-31T07:32:07.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.47 MB/sec execute 227 sec latency 11242.785 ms 2024-08-31T07:32:08.287 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:08 smithi032 ceph-mon[17252]: pgmap v975: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 18 MiB/s wr, 643 op/s 2024-08-31T07:32:08.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.41 MB/sec execute 228 sec latency 12242.839 ms 2024-08-31T07:32:08.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:08 smithi073 ceph-mon[21995]: pgmap v975: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 18 MiB/s wr, 643 op/s 2024-08-31T07:32:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:08 smithi084 ceph-mon[22387]: pgmap v975: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 18 MiB/s wr, 643 op/s 2024-08-31T07:32:09.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:32:09.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.35 MB/sec execute 229 sec latency 13242.889 ms 2024-08-31T07:32:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 -- 172.21.15.32:0/512801729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee04075b70 msgr2=0x7fee04075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 --2- 172.21.15.32:0/512801729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee04075b70 0x7fee04075f80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009860 tx=0x7fedf002a340 comp rx=0 tx=0).stop 2024-08-31T07:32:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 -- 172.21.15.32:0/512801729 shutdown_connections 2024-08-31T07:32:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 --2- 172.21.15.32:0/512801729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee04114020 0x7fee041164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 --2- 172.21.15.32:0/512801729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee04075b70 0x7fee04075f80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 --2- 172.21.15.32:0/512801729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee04077120 0x7fee04075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 -- 172.21.15.32:0/512801729 >> 172.21.15.32:0/512801729 conn(0x7fee040ff1e0 msgr2=0x7fee04101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:10.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 -- 172.21.15.32:0/512801729 shutdown_connections 2024-08-31T07:32:10.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 -- 172.21.15.32:0/512801729 wait complete. 2024-08-31T07:32:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 Processor -- start 2024-08-31T07:32:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.255+0000 7fee03e00640 1 -- start start 2024-08-31T07:32:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee04075b70 0x7fee041a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee04077120 0x7fee041a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee04114020 0x7fee041a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fee04119c30 con 0x7fee04114020 2024-08-31T07:32:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee04119ab0 con 0x7fee04077120 2024-08-31T07:32:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fee04119db0 con 0x7fee04075b70 2024-08-31T07:32:10.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee04114020 0x7fee041a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee04114020 0x7fee041a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52044/0 (socket says 172.21.15.32:52044) 2024-08-31T07:32:10.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03400640 1 -- 172.21.15.32:0/181979069 learned_addr learned my addr 172.21.15.32:0/181979069 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:10.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee02000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee04077120 0x7fee041a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03400640 1 -- 172.21.15.32:0/181979069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee04075b70 msgr2=0x7fee041a0620 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:32:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03400640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee04075b70 0x7fee041a0620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03400640 1 -- 172.21.15.32:0/181979069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee04077120 msgr2=0x7fee041a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03400640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee04077120 0x7fee041a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03400640 1 -- 172.21.15.32:0/181979069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee041a6590 con 0x7fee04114020 2024-08-31T07:32:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee02000640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee04077120 0x7fee041a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03400640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee04114020 0x7fee041a5e60 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009fd0 tx=0x7fedfc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fedfb400640 1 -- 172.21.15.32:0/181979069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc004b20 con 0x7fee04114020 2024-08-31T07:32:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fedfb400640 1 -- 172.21.15.32:0/181979069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedfc003e20 con 0x7fee04114020 2024-08-31T07:32:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03e00640 1 -- 172.21.15.32:0/181979069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee041a6880 con 0x7fee04114020 2024-08-31T07:32:10.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fedfb400640 1 -- 172.21.15.32:0/181979069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc004820 con 0x7fee04114020 2024-08-31T07:32:10.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.260+0000 7fee03e00640 1 -- 172.21.15.32:0/181979069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee041acfe0 con 0x7fee04114020 2024-08-31T07:32:10.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee105f6740 1 Processor -- start 2024-08-31T07:32:10.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fedfb400640 1 -- 172.21.15.32:0/181979069 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fedfc01d020 con 0x7fee04114020 2024-08-31T07:32:10.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee105f6740 1 -- start start 2024-08-31T07:32:10.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee105f6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5496070 0x5597a583bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee105f6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a583c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee105f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5475390 0x5597a583c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee105f6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5597a57b2770 con 0x5597a5496070 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee105f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5597a57a1d00 con 0x5597a5475390 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee105f6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5597a57a5cb0 con 0x5597a546d1b0 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5496070 0x5597a583bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a583c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5496070 0x5597a583bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52054/0 (socket says 172.21.15.32:52054) 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02000640 1 -- 172.21.15.32:0/419616023 learned_addr learned my addr 172.21.15.32:0/419616023 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02a00640 1 -- 172.21.15.32:0/419616023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5475390 msgr2=0x5597a583c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02a00640 1 --2- 172.21.15.32:0/419616023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5475390 0x5597a583c5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02a00640 1 -- 172.21.15.32:0/419616023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5496070 msgr2=0x5597a583bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:10.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02a00640 1 --2- 172.21.15.32:0/419616023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5496070 0x5597a583bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02a00640 1 -- 172.21.15.32:0/419616023 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a5501650 con 0x5597a546d1b0 2024-08-31T07:32:10.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02a00640 1 --2- 172.21.15.32:0/419616023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a583c060 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009ad0 tx=0x7fedf4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c6c52bcc5ce8ec server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fedfb400640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fede80b4ca0 0x7fede80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fedfb400640 1 -- 172.21.15.32:0/181979069 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fedfc0e5600 con 0x7fee04114020 2024-08-31T07:32:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fedefe00640 1 -- 172.21.15.32:0/419616023 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf400ed10 con 0x5597a546d1b0 2024-08-31T07:32:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fedefe00640 1 -- 172.21.15.32:0/419616023 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4005020 con 0x5597a546d1b0 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fedefe00640 1 -- 172.21.15.32:0/419616023 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4004a00 con 0x5597a546d1b0 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.265+0000 7fee02000640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fede80b4ca0 0x7fede80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- 172.21.15.32:0/419616023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 msgr2=0x5597a583c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02000640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fede80b4ca0 0x7fede80b71b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fedf00057c0 tx=0x7fedf002c040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 --2- 172.21.15.32:0/419616023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a583c060 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009ad0 tx=0x7fedf4009f90 comp rx=0 tx=0).stop 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- 172.21.15.32:0/419616023 shutdown_connections 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 --2- 172.21.15.32:0/419616023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5475390 0x5597a583c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 --2- 172.21.15.32:0/419616023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a583c060 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 --2- 172.21.15.32:0/419616023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5496070 0x5597a583bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- 172.21.15.32:0/419616023 >> 172.21.15.32:0/419616023 conn(0x5597a548a460 msgr2=0x5597a5839a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- 172.21.15.32:0/419616023 shutdown_connections 2024-08-31T07:32:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- 172.21.15.32:0/419616023 wait complete. 2024-08-31T07:32:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- start start 2024-08-31T07:32:10.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 Processor -- start 2024-08-31T07:32:10.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a546d1b0 0x5597a58476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a5475390 0x5597a5847c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a5853650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5597a5604700 con 0x5597a546d1b0 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5597a5728cb0 con 0x5597a5496070 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5597a584a300 con 0x5597a5475390 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 10 client.0 _ll_register_callbacks cb 0x5597a574e430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a546d1b0 0x5597a58476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a546d1b0 0x5597a58476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52064/0 (socket says 172.21.15.32:52064) 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02a00640 1 -- 172.21.15.32:0/3172297829 learned_addr learned my addr 172.21.15.32:0/3172297829 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02000640 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a5475390 0x5597a5847c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee03400640 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a5853650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02a00640 1 -- 172.21.15.32:0/3172297829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a5475390 msgr2=0x5597a5847c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02a00640 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a5475390 0x5597a5847c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02a00640 1 -- 172.21.15.32:0/3172297829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 msgr2=0x5597a5853650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02a00640 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a5853650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02a00640 1 -- 172.21.15.32:0/3172297829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a5769870 con 0x5597a546d1b0 2024-08-31T07:32:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02000640 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a5475390 0x5597a5847c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee02a00640 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a546d1b0 0x5597a58476c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fedf4000c00 tx=0x7fedf4005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41550e43e8e3f39f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fedecc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:32:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fedecc00640 1 -- 172.21.15.32:0/3172297829 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4004ad0 con 0x5597a546d1b0 2024-08-31T07:32:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fedecc00640 1 -- 172.21.15.32:0/3172297829 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fedf4004c70 con 0x5597a546d1b0 2024-08-31T07:32:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- 172.21.15.32:0/3172297829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf4005020 con 0x5597a546d1b0 2024-08-31T07:32:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fedecc00640 1 -- 172.21.15.32:0/3172297829 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fedf4018470 con 0x5597a546d1b0 2024-08-31T07:32:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 10 client.19206 fetch_fsmap learned FSMap version 71 2024-08-31T07:32:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 1 -- 172.21.15.32:0/3172297829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5597a58562f0 con 0x5597a546d1b0 2024-08-31T07:32:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fedecc00640 1 -- 172.21.15.32:0/3172297829 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4005020 con 0x5597a546d1b0 2024-08-31T07:32:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fedecc00640 1 -- 172.21.15.32:0/3172297829 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7fedf40104e0 con 0x5597a546d1b0 2024-08-31T07:32:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 10 client.19206 fetch_fsmap finished waiting for FSMap version 71 2024-08-31T07:32:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 10 client.19206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:32:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.270+0000 7fee105f6740 20 client.19206 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.275+0000 7fee105f6740 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597a54549f0 0x5597a5859700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.275+0000 7fee105f6740 4 client.19206 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.275+0000 7fee105f6740 1 -- 172.21.15.32:0/3172297829 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597a57b46c0 con 0x5597a54549f0 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.275+0000 7fee02000640 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597a54549f0 0x5597a5859700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.275+0000 7fee02000640 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597a54549f0 0x5597a5859700 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3f24ceb816a8af0 server_cookie=6bdc72297c25f361 in_seq=0 out_seq=0 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.275+0000 7fedecc00640 10 client.19206 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.275+0000 7fedecc00640 1 -- 172.21.15.32:0/3172297829 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597a57b46c0 con 0x5597a54549f0 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.275+0000 7fedecc00640 10 client.19206 handle_command_reply: tid=0 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.275+0000 7fee105f6740 1 client.19206 shutdown 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 1 -- 172.21.15.32:0/3172297829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a546d1b0 msgr2=0x5597a58476c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a546d1b0 0x5597a58476c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fedf4000c00 tx=0x7fedf4005460 comp rx=0 tx=0).stop 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 1 -- 172.21.15.32:0/3172297829 shutdown_connections 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597a54549f0 0x5597a5859700 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a5853650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a5475390 0x5597a5847c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 1 --2- 172.21.15.32:0/3172297829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a546d1b0 0x5597a58476c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 1 -- 172.21.15.32:0/3172297829 >> 172.21.15.32:0/3172297829 conn(0x5597a548a460 msgr2=0x5597a583ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:10.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fedecc00640 0 client.19206 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:10.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 1 -- 172.21.15.32:0/3172297829 shutdown_connections 2024-08-31T07:32:10.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 1 -- 172.21.15.32:0/3172297829 wait complete. 2024-08-31T07:32:10.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.277+0000 7fee105f6740 20 client.19206 trim_cache size 0 max 16384 2024-08-31T07:32:10.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.299+0000 7fee105f6740 1 Processor -- start 2024-08-31T07:32:10.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.299+0000 7fee105f6740 1 -- start start 2024-08-31T07:32:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.299+0000 7fee105f6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a58478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.299+0000 7fee105f6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 0x5597a583f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.299+0000 7fee105f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a5853550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.299+0000 7fee105f6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b5720 con 0x5597a5475390 2024-08-31T07:32:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.299+0000 7fee105f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b7380 con 0x5597a5496070 2024-08-31T07:32:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.299+0000 7fee105f6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b5df0 con 0x5597a546d1b0 2024-08-31T07:32:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a58478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a58478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45224/0 (socket says 172.21.15.32:45224) 2024-08-31T07:32:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02a00640 1 -- 172.21.15.32:0/2602202891 learned_addr learned my addr 172.21.15.32:0/2602202891 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:10.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02000640 1 --2- 172.21.15.32:0/2602202891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 0x5597a583f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee03400640 1 --2- 172.21.15.32:0/2602202891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a5853550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02a00640 1 -- 172.21.15.32:0/2602202891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 msgr2=0x5597a5853550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02a00640 1 --2- 172.21.15.32:0/2602202891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a5853550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02a00640 1 -- 172.21.15.32:0/2602202891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 msgr2=0x5597a583f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02a00640 1 --2- 172.21.15.32:0/2602202891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 0x5597a583f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02a00640 1 -- 172.21.15.32:0/2602202891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a5846420 con 0x5597a546d1b0 2024-08-31T07:32:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02000640 1 --2- 172.21.15.32:0/2602202891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 0x5597a583f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:32:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02a00640 1 --2- 172.21.15.32:0/2602202891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a58478d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fedf40098e0 tx=0x7fedf40095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7689700f981414d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fedef400640 1 -- 172.21.15.32:0/2602202891 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4031070 con 0x5597a546d1b0 2024-08-31T07:32:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fedef400640 1 -- 172.21.15.32:0/2602202891 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4010d70 con 0x5597a546d1b0 2024-08-31T07:32:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fedef400640 1 -- 172.21.15.32:0/2602202891 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4004b30 con 0x5597a546d1b0 2024-08-31T07:32:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 -- 172.21.15.32:0/2602202891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 msgr2=0x5597a58478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 --2- 172.21.15.32:0/2602202891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a58478d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fedf40098e0 tx=0x7fedf40095b0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 -- 172.21.15.32:0/2602202891 shutdown_connections 2024-08-31T07:32:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 --2- 172.21.15.32:0/2602202891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a5853550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 --2- 172.21.15.32:0/2602202891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 0x5597a583f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 --2- 172.21.15.32:0/2602202891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a58478d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 -- 172.21.15.32:0/2602202891 >> 172.21.15.32:0/2602202891 conn(0x5597a548a460 msgr2=0x5597a583ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 -- 172.21.15.32:0/2602202891 shutdown_connections 2024-08-31T07:32:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 -- 172.21.15.32:0/2602202891 wait complete. 2024-08-31T07:32:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 -- start start 2024-08-31T07:32:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 Processor -- start 2024-08-31T07:32:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a584b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 0x5597a584be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a58c3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b6c80 con 0x5597a5475390 2024-08-31T07:32:10.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b6580 con 0x5597a5496070 2024-08-31T07:32:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b3720 con 0x5597a546d1b0 2024-08-31T07:32:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee105f6740 10 client.0 _ll_register_callbacks cb 0x7fedf4009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 0x5597a584be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 0x5597a584be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52090/0 (socket says 172.21.15.32:52090) 2024-08-31T07:32:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a584b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee02000640 1 -- 172.21.15.32:0/772431163 learned_addr learned my addr 172.21.15.32:0/772431163 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.300+0000 7fee03400640 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a58c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee02a00640 1 -- 172.21.15.32:0/772431163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 msgr2=0x5597a58c3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee02a00640 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a58c3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee02a00640 1 -- 172.21.15.32:0/772431163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 msgr2=0x5597a584be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee02a00640 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 0x5597a584be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee02a00640 1 -- 172.21.15.32:0/772431163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a5839c60 con 0x5597a546d1b0 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee02a00640 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a584b8d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fedf4010af0 tx=0x7fedf4005710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d72f4b99e75f15bb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fedecc00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fedecc00640 1 -- 172.21.15.32:0/772431163 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4031040 con 0x5597a546d1b0 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fedecc00640 1 -- 172.21.15.32:0/772431163 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7fedf4005460 con 0x5597a546d1b0 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 -- 172.21.15.32:0/772431163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf4010d70 con 0x5597a546d1b0 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fedecc00640 1 -- 172.21.15.32:0/772431163 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fedf4044050 con 0x5597a546d1b0 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 10 client.29251 fetch_fsmap learned FSMap version 71 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 -- 172.21.15.32:0/772431163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5597a58c43d0 con 0x5597a546d1b0 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fedecc00640 1 -- 172.21.15.32:0/772431163 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4010d70 con 0x5597a546d1b0 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fedecc00640 1 -- 172.21.15.32:0/772431163 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7fedf4003780 con 0x5597a546d1b0 2024-08-31T07:32:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 10 client.29251 fetch_fsmap finished waiting for FSMap version 71 2024-08-31T07:32:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 10 client.29251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:32:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 20 client.29251 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597a5429da0 0x5597a58c6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 4 client.29251 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:32:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 -- 172.21.15.32:0/772431163 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597a57b0160 con 0x5597a5429da0 2024-08-31T07:32:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee02000640 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597a5429da0 0x5597a58c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee02000640 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597a5429da0 0x5597a58c6940 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49e1676c33157b99 server_cookie=df726fff2667940c in_seq=0 out_seq=0 2024-08-31T07:32:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fedecc00640 10 client.29251 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fedecc00640 1 -- 172.21.15.32:0/772431163 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6338 (crc 0 0 0) 0x5597a57b0160 con 0x5597a5429da0 2024-08-31T07:32:10.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fedecc00640 10 client.29251 handle_command_reply: tid=0 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 client.29251 shutdown 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 -- 172.21.15.32:0/772431163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 msgr2=0x5597a584b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a584b8d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fedf4010af0 tx=0x7fedf4005710 comp rx=0 tx=0).stop 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 -- 172.21.15.32:0/772431163 shutdown_connections 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5597a5429da0 0x5597a58c6940 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597a5496070 0x5597a58c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5597a5475390 0x5597a584be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 --2- 172.21.15.32:0/772431163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597a546d1b0 0x5597a584b8d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 -- 172.21.15.32:0/772431163 >> 172.21.15.32:0/772431163 conn(0x5597a548a460 msgr2=0x5597a583cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fedecc00640 0 client.29251 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 -- 172.21.15.32:0/772431163 shutdown_connections 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 1 -- 172.21.15.32:0/772431163 wait complete. 2024-08-31T07:32:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee105f6740 20 client.29251 trim_cache size 0 max 16384 2024-08-31T07:32:10.314 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:32:10.314 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:32:10.314 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:32:10.314 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:32:10.314 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:32:10.314 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:32:10.314 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.315 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.316 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.317 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.318 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:32:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:32:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:32:10.323 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:32:10.327 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.306+0000 7fee03e00640 1 -- 172.21.15.32:0/181979069 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fede80b4ca0 msgr2=0x7fede80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fede80b4ca0 0x7fede80b71b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fedf00057c0 tx=0x7fedf002c040 comp rx=0 tx=0).stop 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 -- 172.21.15.32:0/181979069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee04114020 msgr2=0x7fee041a5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee04114020 0x7fee041a5e60 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009fd0 tx=0x7fedfc00c800 comp rx=0 tx=0).stop 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 -- 172.21.15.32:0/181979069 shutdown_connections 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fede80b4ca0 0x7fede80b71b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee04114020 0x7fee041a5e60 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee04077120 0x7fee041a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 --2- 172.21.15.32:0/181979069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee04075b70 0x7fee041a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 -- 172.21.15.32:0/181979069 >> 172.21.15.32:0/181979069 conn(0x7fee040ff1e0 msgr2=0x7fee0410ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 -- 172.21.15.32:0/181979069 shutdown_connections 2024-08-31T07:32:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:10.311+0000 7fee03e00640 1 -- 172.21.15.32:0/181979069 wait complete. 2024-08-31T07:32:10.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.28 MB/sec execute 230 sec latency 14242.941 ms 2024-08-31T07:32:10.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:10 smithi032 ceph-mon[17252]: pgmap v976: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 438 op/s 2024-08-31T07:32:10.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:10 smithi073 ceph-mon[21995]: pgmap v976: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 438 op/s 2024-08-31T07:32:10.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:10 smithi084 ceph-mon[22387]: pgmap v976: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 438 op/s 2024-08-31T07:32:11.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.22 MB/sec execute 231 sec latency 15242.994 ms 2024-08-31T07:32:11.503 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:32:11.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:32:11.904 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:12.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.16 MB/sec execute 232 sec latency 16243.044 ms 2024-08-31T07:32:12.563 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:12 smithi032 ceph-mon[17252]: pgmap v977: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 449 op/s 2024-08-31T07:32:12.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:12 smithi073 ceph-mon[21995]: pgmap v977: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 449 op/s 2024-08-31T07:32:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:12 smithi084 ceph-mon[22387]: pgmap v977: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 449 op/s 2024-08-31T07:32:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.805+0000 7efd42800640 1 -- 172.21.15.32:0/2403721125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd3c075b70 msgr2=0x7efd3c075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.805+0000 7efd42800640 1 --2- 172.21.15.32:0/2403721125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd3c075b70 0x7efd3c075f80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7efd28009900 tx=0x7efd2802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:32:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.805+0000 7efd42800640 1 -- 172.21.15.32:0/2403721125 shutdown_connections 2024-08-31T07:32:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.805+0000 7efd42800640 1 --2- 172.21.15.32:0/2403721125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c114020 0x7efd3c1164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.805+0000 7efd42800640 1 --2- 172.21.15.32:0/2403721125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd3c075b70 0x7efd3c075f80 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.805+0000 7efd42800640 1 --2- 172.21.15.32:0/2403721125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd3c077120 0x7efd3c075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.805+0000 7efd42800640 1 -- 172.21.15.32:0/2403721125 >> 172.21.15.32:0/2403721125 conn(0x7efd3c0ff1e0 msgr2=0x7efd3c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.805+0000 7efd42800640 1 -- 172.21.15.32:0/2403721125 shutdown_connections 2024-08-31T07:32:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.805+0000 7efd42800640 1 -- 172.21.15.32:0/2403721125 wait complete. 2024-08-31T07:32:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd42800640 1 Processor -- start 2024-08-31T07:32:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd42800640 1 -- start start 2024-08-31T07:32:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd42800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c075b70 0x7efd3c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd42800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd3c077120 0x7efd3c1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd42800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd3c114020 0x7efd3c1a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd42800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c119c60 con 0x7efd3c077120 2024-08-31T07:32:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd42800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c119ae0 con 0x7efd3c075b70 2024-08-31T07:32:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd42800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c119de0 con 0x7efd3c114020 2024-08-31T07:32:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd41400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c075b70 0x7efd3c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd40a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd3c077120 0x7efd3c1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd41400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c075b70 0x7efd3c1a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55700/0 (socket says 172.21.15.32:55700) 2024-08-31T07:32:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd41400640 1 -- 172.21.15.32:0/1693404475 learned_addr learned my addr 172.21.15.32:0/1693404475 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd41e00640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd3c114020 0x7efd3c1a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd41400640 1 -- 172.21.15.32:0/1693404475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd3c114020 msgr2=0x7efd3c1a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd41400640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd3c114020 0x7efd3c1a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd41400640 1 -- 172.21.15.32:0/1693404475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd3c077120 msgr2=0x7efd3c1a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd41400640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd3c077120 0x7efd3c1a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd41400640 1 -- 172.21.15.32:0/1693404475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c1a66c0 con 0x7efd3c075b70 2024-08-31T07:32:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.810+0000 7efd41e00640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd3c114020 0x7efd3c1a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:32:12.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.821+0000 7efd41400640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c075b70 0x7efd3c1a0750 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efd3000e9f0 tx=0x7efd3000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:12.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.821+0000 7efd3a000640 1 -- 172.21.15.32:0/1693404475 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3000cd80 con 0x7efd3c075b70 2024-08-31T07:32:12.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.821+0000 7efd3a000640 1 -- 172.21.15.32:0/1693404475 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd300054d0 con 0x7efd3c075b70 2024-08-31T07:32:12.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.821+0000 7efd3a000640 1 -- 172.21.15.32:0/1693404475 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30010690 con 0x7efd3c075b70 2024-08-31T07:32:12.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.821+0000 7efd42800640 1 -- 172.21.15.32:0/1693404475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c1a69b0 con 0x7efd3c075b70 2024-08-31T07:32:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.821+0000 7efd42800640 1 -- 172.21.15.32:0/1693404475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c1ad110 con 0x7efd3c075b70 2024-08-31T07:32:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.821+0000 7efd42800640 1 -- 172.21.15.32:0/1693404475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c076a60 con 0x7efd3c075b70 2024-08-31T07:32:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.826+0000 7efd3a000640 1 -- 172.21.15.32:0/1693404475 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7efd30004e50 con 0x7efd3c075b70 2024-08-31T07:32:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.826+0000 7efd3a000640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efd240b4ca0 0x7efd240b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.826+0000 7efd3a000640 1 -- 172.21.15.32:0/1693404475 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8441+0+0 (secure 0 0 0) 0x7efd300e5c00 con 0x7efd3c075b70 2024-08-31T07:32:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.831+0000 7efd40a00640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efd240b4ca0 0x7efd240b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.831+0000 7efd3a000640 1 -- 172.21.15.32:0/1693404475 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efd300a8570 con 0x7efd3c075b70 2024-08-31T07:32:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:12.831+0000 7efd40a00640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efd240b4ca0 0x7efd240b71b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efd28000c00 tx=0x7efd28031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.078+0000 7efd42800640 1 -- 172.21.15.32:0/1693404475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd3c075f80 con 0x7efd3c075b70 2024-08-31T07:32:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.079+0000 7efd3a000640 1 -- 172.21.15.32:0/1693404475 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+11970 (secure 0 0 0) 0x7efd3c075f80 con 0x7efd3c075b70 2024-08-31T07:32:13.080 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:32:13.081 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"btime":"2024-08-31T07:31:56:879790+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:31:55.975354+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:32:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 71 2024-08-31T07:32:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 -- 172.21.15.32:0/1693404475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efd240b4ca0 msgr2=0x7efd240b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efd240b4ca0 0x7efd240b71b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efd28000c00 tx=0x7efd28031400 comp rx=0 tx=0).stop 2024-08-31T07:32:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 -- 172.21.15.32:0/1693404475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c075b70 msgr2=0x7efd3c1a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c075b70 0x7efd3c1a0750 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efd3000e9f0 tx=0x7efd3000edf0 comp rx=0 tx=0).stop 2024-08-31T07:32:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 -- 172.21.15.32:0/1693404475 shutdown_connections 2024-08-31T07:32:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7efd240b4ca0 0x7efd240b71b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd3c114020 0x7efd3c1a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd3c077120 0x7efd3c1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 --2- 172.21.15.32:0/1693404475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c075b70 0x7efd3c1a0750 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 -- 172.21.15.32:0/1693404475 >> 172.21.15.32:0/1693404475 conn(0x7efd3c0ff1e0 msgr2=0x7efd3c10ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 -- 172.21.15.32:0/1693404475 shutdown_connections 2024-08-31T07:32:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:13.082+0000 7efd42800640 1 -- 172.21.15.32:0/1693404475 wait complete. 2024-08-31T07:32:13.399 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111042 14.10 MB/sec execute 233 sec latency 17243.092 ms 2024-08-31T07:32:13.400 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:13 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1693404475' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:32:13.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:13 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1693404475' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:32:13.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:13 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1693404475' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:32:14.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:32:14.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111044 14.04 MB/sec execute 234 sec latency 18243.142 ms 2024-08-31T07:32:14.511 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:14.553 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:14 smithi032 ceph-mon[17252]: pgmap v978: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 412 op/s 2024-08-31T07:32:14.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:14 smithi073 ceph-mon[21995]: pgmap v978: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 412 op/s 2024-08-31T07:32:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:14 smithi084 ceph-mon[22387]: pgmap v978: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 412 op/s 2024-08-31T07:32:15.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111077 13.98 MB/sec execute 235 sec latency 19243.194 ms 2024-08-31T07:32:16.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111088 13.93 MB/sec execute 236 sec latency 20243.242 ms 2024-08-31T07:32:16.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:16 smithi032 ceph-mon[17252]: pgmap v979: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 380 op/s 2024-08-31T07:32:16.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:16 smithi073 ceph-mon[21995]: pgmap v979: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 380 op/s 2024-08-31T07:32:16.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:16 smithi084 ceph-mon[22387]: pgmap v979: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 380 op/s 2024-08-31T07:32:17.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111103 13.87 MB/sec execute 237 sec latency 21243.301 ms 2024-08-31T07:32:17.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:17 smithi073 ceph-mon[21995]: osdmap e107: 12 total, 12 up, 12 in 2024-08-31T07:32:17.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:17 smithi084 ceph-mon[22387]: osdmap e107: 12 total, 12 up, 12 in 2024-08-31T07:32:17.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:17 smithi032 ceph-mon[17252]: osdmap e107: 12 total, 12 up, 12 in 2024-08-31T07:32:18.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111124 13.83 MB/sec execute 238 sec latency 22243.360 ms 2024-08-31T07:32:18.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:18 smithi084 ceph-mon[22387]: pgmap v981: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 158 op/s 2024-08-31T07:32:18.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:18 smithi032 ceph-mon[17252]: pgmap v981: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 158 op/s 2024-08-31T07:32:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:18 smithi073 ceph-mon[21995]: pgmap v981: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 158 op/s 2024-08-31T07:32:19.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111124 13.77 MB/sec execute 239 sec latency 23243.419 ms 2024-08-31T07:32:20.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111124 13.71 MB/sec execute 240 sec latency 24243.460 ms 2024-08-31T07:32:20.466 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:20 smithi032 ceph-mon[17252]: pgmap v982: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-08-31T07:32:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.546+0000 7ffbd3200640 1 -- 172.21.15.32:0/715735292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc0754d0 msgr2=0x7ffbcc0758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.546+0000 7ffbd3200640 1 --2- 172.21.15.32:0/715735292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc0754d0 0x7ffbcc0758e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009900 tx=0x7ffbbc02a410 comp rx=0 tx=0).stop 2024-08-31T07:32:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 -- 172.21.15.32:0/715735292 shutdown_connections 2024-08-31T07:32:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 --2- 172.21.15.32:0/715735292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbcc10fee0 0x7ffbcc112300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 --2- 172.21.15.32:0/715735292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc075e20 0x7ffbcc10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 --2- 172.21.15.32:0/715735292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc0754d0 0x7ffbcc0758e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 -- 172.21.15.32:0/715735292 >> 172.21.15.32:0/715735292 conn(0x7ffbcc07b5f0 msgr2=0x7ffbcc07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 -- 172.21.15.32:0/715735292 shutdown_connections 2024-08-31T07:32:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 -- 172.21.15.32:0/715735292 wait complete. 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 Processor -- start 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 -- start start 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0754d0 0x7ffbcc19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc075e20 0x7ffbcc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbcc10fee0 0x7ffbcc1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc115a40 con 0x7ffbcc075e20 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1158c0 con 0x7ffbcc0754d0 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc115bc0 con 0x7ffbcc10fee0 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbcb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc075e20 0x7ffbcc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbcbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0754d0 0x7ffbcc19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbcb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc075e20 0x7ffbcc19c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40588/0 (socket says 172.21.15.32:40588) 2024-08-31T07:32:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbcb400640 1 -- 172.21.15.32:0/873608266 learned_addr learned my addr 172.21.15.32:0/873608266 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd1200640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbcc10fee0 0x7ffbcc1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbcb400640 1 -- 172.21.15.32:0/873608266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbcc10fee0 msgr2=0x7ffbcc1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbcb400640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbcc10fee0 0x7ffbcc1a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbcb400640 1 -- 172.21.15.32:0/873608266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0754d0 msgr2=0x7ffbcc19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbcb400640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0754d0 0x7ffbcc19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbcb400640 1 -- 172.21.15.32:0/873608266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc1a23c0 con 0x7ffbcc075e20 2024-08-31T07:32:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbcb400640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc075e20 0x7ffbcc19c9f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800d950 tx=0x7ffbb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbc8c00640 1 -- 172.21.15.32:0/873608266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8017070 con 0x7ffbcc075e20 2024-08-31T07:32:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbc8c00640 1 -- 172.21.15.32:0/873608266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb8009c30 con 0x7ffbcc075e20 2024-08-31T07:32:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 -- 172.21.15.32:0/873608266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc1a26b0 con 0x7ffbcc075e20 2024-08-31T07:32:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbc8c00640 1 -- 172.21.15.32:0/873608266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80047a0 con 0x7ffbcc075e20 2024-08-31T07:32:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.548+0000 7ffbd3200640 1 -- 172.21.15.32:0/873608266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc1a8e10 con 0x7ffbcc075e20 2024-08-31T07:32:20.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd9e52740 1 Processor -- start 2024-08-31T07:32:20.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd9e52740 1 -- start start 2024-08-31T07:32:20.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917249b070 0x559172840990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172840ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917247a390 0x559172841410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5591727b77b0 con 0x5591724721b0 2024-08-31T07:32:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5591727a6d40 con 0x55917249b070 2024-08-31T07:32:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5591727aacf0 con 0x55917247a390 2024-08-31T07:32:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172840ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917249b070 0x559172840990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917249b070 0x559172840990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:39676/0 (socket says 172.21.15.32:39676) 2024-08-31T07:32:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd1200640 1 -- 172.21.15.32:0/2262552735 learned_addr learned my addr 172.21.15.32:0/2262552735 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbc8c00640 1 -- 172.21.15.32:0/873608266 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ffbb8004020 con 0x7ffbcc075e20 2024-08-31T07:32:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbcbe00640 1 -- 172.21.15.32:0/2262552735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917247a390 msgr2=0x559172841410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbcb400640 1 --2- 172.21.15.32:0/2262552735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917247a390 0x559172841410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/2262552735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917247a390 0x559172841410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbcbe00640 1 -- 172.21.15.32:0/2262552735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917249b070 msgr2=0x559172840990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/2262552735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917249b070 0x559172840990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbcbe00640 1 -- 172.21.15.32:0/2262552735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559172506650 con 0x5591724721b0 2024-08-31T07:32:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd1200640 1 --2- 172.21.15.32:0/2262552735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917249b070 0x559172840990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbcb400640 1 --2- 172.21.15.32:0/2262552735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917247a390 0x559172841410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:32:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/2262552735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172840ed0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009230 tx=0x7ffbbc032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7459aaf38468cb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbc1600640 1 -- 172.21.15.32:0/2262552735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc040070 con 0x5591724721b0 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbc1600640 1 -- 172.21.15.32:0/2262552735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbbc004030 con 0x5591724721b0 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbc1600640 1 -- 172.21.15.32:0/2262552735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc035040 con 0x5591724721b0 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbc8c00640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffbb00b4d20 0x7ffbb00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd9e52740 1 -- 172.21.15.32:0/2262552735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 msgr2=0x559172840ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/2262552735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172840ed0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009230 tx=0x7ffbbc032c40 comp rx=0 tx=0).stop 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.553+0000 7ffbc8c00640 1 -- 172.21.15.32:0/873608266 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8485+0+0 (secure 0 0 0) 0x7ffbb80e5160 con 0x7ffbcc075e20 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd9e52740 1 -- 172.21.15.32:0/2262552735 shutdown_connections 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/2262552735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917247a390 0x559172841410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/2262552735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172840ed0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/2262552735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917249b070 0x559172840990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd9e52740 1 -- 172.21.15.32:0/2262552735 >> 172.21.15.32:0/2262552735 conn(0x55917248f460 msgr2=0x55917283e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd1200640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffbb00b4d20 0x7ffbb00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd9e52740 1 -- 172.21.15.32:0/2262552735 shutdown_connections 2024-08-31T07:32:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd9e52740 1 -- 172.21.15.32:0/2262552735 wait complete. 2024-08-31T07:32:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd1200640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffbb00b4d20 0x7ffbb00b7230 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4005fd0 tx=0x7ffbc4009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd9e52740 1 -- start start 2024-08-31T07:32:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.558+0000 7ffbd9e52740 1 Processor -- start 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591724721b0 0x55917284c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917247a390 0x55917284ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917249b070 0x559172858400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559172609700 con 0x55917249b070 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55917272dcf0 con 0x55917247a390 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55917284f110 con 0x5591724721b0 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 10 client.0 _ll_register_callbacks cb 0x559172753470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbcbe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591724721b0 0x55917284c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbcb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917247a390 0x55917284ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbcbe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591724721b0 0x55917284c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33432/0 (socket says 172.21.15.32:33432) 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbcbe00640 1 -- 172.21.15.32:0/568778626 learned_addr learned my addr 172.21.15.32:0/568778626 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd1200640 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917249b070 0x559172858400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbcbe00640 1 -- 172.21.15.32:0/568778626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917247a390 msgr2=0x55917284ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917247a390 0x55917284ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbcbe00640 1 -- 172.21.15.32:0/568778626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917249b070 msgr2=0x559172858400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917249b070 0x559172858400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbcbe00640 1 -- 172.21.15.32:0/568778626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917276e8b0 con 0x5591724721b0 2024-08-31T07:32:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591724721b0 0x55917284c550 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc005ec0 tx=0x7ffbbc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7cee99d235e3bd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbb6000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:32:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbb6000640 1 -- 172.21.15.32:0/568778626 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc035070 con 0x5591724721b0 2024-08-31T07:32:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 1 -- 172.21.15.32:0/568778626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbbc004030 con 0x5591724721b0 2024-08-31T07:32:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbb6000640 1 -- 172.21.15.32:0/568778626 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8485+0+0 (secure 0 0 0) 0x7ffbbc005540 con 0x5591724721b0 2024-08-31T07:32:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbb6000640 1 -- 172.21.15.32:0/568778626 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7ffbbc040040 con 0x5591724721b0 2024-08-31T07:32:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 10 client.29266 fetch_fsmap learned FSMap version 71 2024-08-31T07:32:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 1 -- 172.21.15.32:0/568778626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55917285b040 con 0x5591724721b0 2024-08-31T07:32:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbb6000640 1 -- 172.21.15.32:0/568778626 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbbc004030 con 0x5591724721b0 2024-08-31T07:32:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbb6000640 1 -- 172.21.15.32:0/568778626 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7ffbbc054020 con 0x5591724721b0 2024-08-31T07:32:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 10 client.29266 fetch_fsmap finished waiting for FSMap version 71 2024-08-31T07:32:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 10 client.29266 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:32:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 20 client.29266 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5591724599f0 0x55917285e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 4 client.29266 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:32:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.560+0000 7ffbd9e52740 1 -- 172.21.15.32:0/568778626 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591727b9700 con 0x5591724599f0 2024-08-31T07:32:20.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.572+0000 7ffbcb400640 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5591724599f0 0x55917285e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.630+0000 7ffbcb400640 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5591724599f0 0x55917285e3a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=18d15fe750998a87 server_cookie=64b5268a4f17716f in_seq=0 out_seq=0 2024-08-31T07:32:20.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.630+0000 7ffbb6000640 10 client.29266 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:32:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbb6000640 1 -- 172.21.15.32:0/568778626 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591727b9700 con 0x5591724599f0 2024-08-31T07:32:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbb6000640 10 client.29266 handle_command_reply: tid=0 2024-08-31T07:32:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 client.29266 shutdown 2024-08-31T07:32:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 -- 172.21.15.32:0/568778626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591724721b0 msgr2=0x55917284c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591724721b0 0x55917284c550 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc005ec0 tx=0x7ffbbc009260 comp rx=0 tx=0).stop 2024-08-31T07:32:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 -- 172.21.15.32:0/568778626 shutdown_connections 2024-08-31T07:32:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5591724599f0 0x55917285e3a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917249b070 0x559172858400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917247a390 0x55917284ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/568778626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5591724721b0 0x55917284c550 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 -- 172.21.15.32:0/568778626 >> 172.21.15.32:0/568778626 conn(0x55917248f460 msgr2=0x559172843b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbb6000640 0 client.29266 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:32:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 -- 172.21.15.32:0/568778626 shutdown_connections 2024-08-31T07:32:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 1 -- 172.21.15.32:0/568778626 wait complete. 2024-08-31T07:32:20.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.655+0000 7ffbd9e52740 20 client.29266 trim_cache size 0 max 16384 2024-08-31T07:32:20.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbd9e52740 1 Processor -- start 2024-08-31T07:32:20.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbd9e52740 1 -- start start 2024-08-31T07:32:20.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172844560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917247a390 0x55917283de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917249b070 0x55917283e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1b16c0 con 0x5591724721b0 2024-08-31T07:32:20.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1b3320 con 0x55917247a390 2024-08-31T07:32:20.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1b1d90 con 0x55917249b070 2024-08-31T07:32:20.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172844560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172844560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40616/0 (socket says 172.21.15.32:40616) 2024-08-31T07:32:20.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbcbe00640 1 -- 172.21.15.32:0/2959025872 learned_addr learned my addr 172.21.15.32:0/2959025872 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:20.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbcb400640 1 --2- 172.21.15.32:0/2959025872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917247a390 0x55917283de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbd1200640 1 --2- 172.21.15.32:0/2959025872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917249b070 0x55917283e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbcbe00640 1 -- 172.21.15.32:0/2959025872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917249b070 msgr2=0x55917283e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:20.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/2959025872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917249b070 0x55917283e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbcbe00640 1 -- 172.21.15.32:0/2959025872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917247a390 msgr2=0x55917283de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/2959025872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917247a390 0x55917283de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbcbe00640 1 -- 172.21.15.32:0/2959025872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559172842c80 con 0x5591724721b0 2024-08-31T07:32:20.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.666+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/2959025872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172844560 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc001eb0 tx=0x7ffbbc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2670e1dae0f5dd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:20.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbc0c00640 1 -- 172.21.15.32:0/2959025872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc035040 con 0x5591724721b0 2024-08-31T07:32:20.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbc0c00640 1 -- 172.21.15.32:0/2959025872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbbc005610 con 0x5591724721b0 2024-08-31T07:32:20.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbc0c00640 1 -- 172.21.15.32:0/2959025872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc031cf0 con 0x5591724721b0 2024-08-31T07:32:20.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- 172.21.15.32:0/2959025872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 msgr2=0x559172844560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:20.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/2959025872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172844560 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc001eb0 tx=0x7ffbbc031400 comp rx=0 tx=0).stop 2024-08-31T07:32:20.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- 172.21.15.32:0/2959025872 shutdown_connections 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/2959025872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917249b070 0x55917283e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/2959025872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55917247a390 0x55917283de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/2959025872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5591724721b0 0x559172844560 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- 172.21.15.32:0/2959025872 >> 172.21.15.32:0/2959025872 conn(0x55917248f460 msgr2=0x559172842380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- 172.21.15.32:0/2959025872 shutdown_connections 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- 172.21.15.32:0/2959025872 wait complete. 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- start start 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 Processor -- start 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591724721b0 0x559172861ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917247a390 0x55917285f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917249b070 0x55917285f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1b2c20 con 0x55917247a390 2024-08-31T07:32:20.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1b2520 con 0x5591724721b0 2024-08-31T07:32:20.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1af6c0 con 0x55917249b070 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 10 client.0 _ll_register_callbacks cb 0x7ffbbc005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbcb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917247a390 0x55917285f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbcb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917247a390 0x55917285f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40632/0 (socket says 172.21.15.32:40632) 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbcb400640 1 -- 172.21.15.32:0/181120080 learned_addr learned my addr 172.21.15.32:0/181120080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591724721b0 0x559172861ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd1200640 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917249b070 0x55917285f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbcb400640 1 -- 172.21.15.32:0/181120080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917249b070 msgr2=0x55917285f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbcb400640 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917249b070 0x55917285f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbcb400640 1 -- 172.21.15.32:0/181120080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591724721b0 msgr2=0x559172861ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbcb400640 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591724721b0 0x559172861ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbcb400640 1 -- 172.21.15.32:0/181120080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917285ffc0 con 0x55917247a390 2024-08-31T07:32:20.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbcb400640 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917247a390 0x55917285f1b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8004770 tx=0x7ffbb80e7620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877d65215501750f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbb6000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbb6000640 1 -- 172.21.15.32:0/181120080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80abf10 con 0x55917247a390 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbb6000640 1 -- 172.21.15.32:0/181120080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8485+0+0 (secure 0 0 0) 0x7ffbb80e2af0 con 0x55917247a390 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- 172.21.15.32:0/181120080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbbc005610 con 0x55917247a390 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbb6000640 1 -- 172.21.15.32:0/181120080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7ffbb80e2cf0 con 0x55917247a390 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 10 client.19236 fetch_fsmap learned FSMap version 71 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- 172.21.15.32:0/181120080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5591728c8be0 con 0x55917247a390 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbb6000640 1 -- 172.21.15.32:0/181120080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbbc005610 con 0x55917247a390 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbb6000640 1 -- 172.21.15.32:0/181120080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7ffbb80e70b0 con 0x55917247a390 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 10 client.19236 fetch_fsmap finished waiting for FSMap version 71 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 10 client.19236 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 20 client.19236 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55917242eda0 0x5591728cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:20.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 4 client.19236 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:32:20.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.671+0000 7ffbd9e52740 1 -- 172.21.15.32:0/181120080 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591727b51a0 con 0x55917242eda0 2024-08-31T07:32:20.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55917242eda0 0x5591728cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:20.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbcbe00640 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55917242eda0 0x5591728cb350 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5c83c1ca2a7511a server_cookie=ddfbf2f6982c1497 in_seq=0 out_seq=0 2024-08-31T07:32:20.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbb6000640 10 client.19236 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:32:20.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbb6000640 1 -- 172.21.15.32:0/181120080 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591727b51a0 con 0x55917242eda0 2024-08-31T07:32:20.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbb6000640 10 client.19236 handle_command_reply: tid=0 2024-08-31T07:32:20.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 client.19236 shutdown 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 -- 172.21.15.32:0/181120080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917247a390 msgr2=0x55917285f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917247a390 0x55917285f1b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8004770 tx=0x7ffbb80e7620 comp rx=0 tx=0).stop 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 -- 172.21.15.32:0/181120080 shutdown_connections 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55917242eda0 0x5591728cb350 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55917249b070 0x55917285f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55917247a390 0x55917285f1b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 --2- 172.21.15.32:0/181120080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5591724721b0 0x559172861ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbb6000640 0 client.19236 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 -- 172.21.15.32:0/181120080 >> 172.21.15.32:0/181120080 conn(0x55917248f460 msgr2=0x559172842150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 -- 172.21.15.32:0/181120080 shutdown_connections 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 1 -- 172.21.15.32:0/181120080 wait complete. 2024-08-31T07:32:20.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.676+0000 7ffbd9e52740 20 client.19236 trim_cache size 0 max 16384 2024-08-31T07:32:20.684 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:32:20.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 -- 172.21.15.32:0/873608266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffbb00b4d20 msgr2=0x7ffbb00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:20.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffbb00b4d20 0x7ffbb00b7230 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4005fd0 tx=0x7ffbc4009500 comp rx=0 tx=0).stop 2024-08-31T07:32:20.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 -- 172.21.15.32:0/873608266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc075e20 msgr2=0x7ffbcc19c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:20.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc075e20 0x7ffbcc19c9f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800d950 tx=0x7ffbb800de10 comp rx=0 tx=0).stop 2024-08-31T07:32:20.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 -- 172.21.15.32:0/873608266 shutdown_connections 2024-08-31T07:32:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ffbb00b4d20 0x7ffbb00b7230 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbcc10fee0 0x7ffbcc1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc075e20 0x7ffbcc19c9f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 --2- 172.21.15.32:0/873608266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0754d0 0x7ffbcc19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 -- 172.21.15.32:0/873608266 >> 172.21.15.32:0/873608266 conn(0x7ffbcc07b5f0 msgr2=0x7ffbcc109a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 -- 172.21.15.32:0/873608266 shutdown_connections 2024-08-31T07:32:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:20.678+0000 7ffbd3200640 1 -- 172.21.15.32:0/873608266 wait complete. 2024-08-31T07:32:20.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:20 smithi084 ceph-mon[22387]: pgmap v982: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-08-31T07:32:20.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:20 smithi073 ceph-mon[21995]: pgmap v982: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-08-31T07:32:21.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111124 13.65 MB/sec execute 241 sec latency 25243.523 ms 2024-08-31T07:32:22.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111124 13.60 MB/sec execute 242 sec latency 26243.563 ms 2024-08-31T07:32:22.473 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:22 smithi032 ceph-mon[17252]: pgmap v983: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 794 KiB/s wr, 122 op/s 2024-08-31T07:32:22.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:22 smithi084 ceph-mon[22387]: pgmap v983: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 794 KiB/s wr, 122 op/s 2024-08-31T07:32:22.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:22 smithi073 ceph-mon[21995]: pgmap v983: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 794 KiB/s wr, 122 op/s 2024-08-31T07:32:23.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111134 13.54 MB/sec execute 243 sec latency 27019.336 ms 2024-08-31T07:32:23.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:32:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:24.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111134 13.49 MB/sec execute 244 sec latency 28019.394 ms 2024-08-31T07:32:24.504 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:24 smithi032 ceph-mon[17252]: pgmap v984: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 399 KiB/s wr, 95 op/s 2024-08-31T07:32:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:24 smithi084 ceph-mon[22387]: pgmap v984: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 399 KiB/s wr, 95 op/s 2024-08-31T07:32:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- 172.21.15.32:0/984663507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d44103750 msgr2=0x7f2d44103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 --2- 172.21.15.32:0/984663507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d44103750 0x7f2d44103b60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009900 tx=0x7f2d3802a400 comp rx=0 tx=0).stop 2024-08-31T07:32:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- 172.21.15.32:0/984663507 shutdown_connections 2024-08-31T07:32:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 --2- 172.21.15.32:0/984663507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d441099e0 0x7f2d44110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 --2- 172.21.15.32:0/984663507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d44109090 0x7f2d441094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 --2- 172.21.15.32:0/984663507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d44103750 0x7f2d44103b60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- 172.21.15.32:0/984663507 >> 172.21.15.32:0/984663507 conn(0x7f2d440ff1e0 msgr2=0x7f2d44101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:24.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- 172.21.15.32:0/984663507 shutdown_connections 2024-08-31T07:32:24.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- 172.21.15.32:0/984663507 wait complete. 2024-08-31T07:32:24.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 Processor -- start 2024-08-31T07:32:24.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- start start 2024-08-31T07:32:24.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d44103750 0x7f2d440754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d44109090 0x7f2d440759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d441099e0 0x7f2d44075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44113690 con 0x7f2d44109090 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44113510 con 0x7f2d441099e0 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44113810 con 0x7f2d44103750 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d49400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d44109090 0x7f2d440759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d49400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d44109090 0x7f2d440759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40644/0 (socket says 172.21.15.32:40644) 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d49400640 1 -- 172.21.15.32:0/271327094 learned_addr learned my addr 172.21.15.32:0/271327094 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4a800640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d441099e0 0x7f2d44075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d49400640 1 -- 172.21.15.32:0/271327094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d44103750 msgr2=0x7f2d440754b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d49400640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d44103750 0x7f2d440754b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d49400640 1 -- 172.21.15.32:0/271327094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d441099e0 msgr2=0x7f2d44075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:24.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d49400640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d441099e0 0x7f2d44075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d49400640 1 -- 172.21.15.32:0/271327094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d441a8380 con 0x7f2d44109090 2024-08-31T07:32:24.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d49400640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d44109090 0x7f2d440759f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400e9f0 tx=0x7f2d3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:24.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d3ea00640 1 -- 172.21.15.32:0/271327094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400ccc0 con 0x7f2d44109090 2024-08-31T07:32:24.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d3ea00640 1 -- 172.21.15.32:0/271327094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d340108e0 con 0x7f2d44109090 2024-08-31T07:32:24.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- 172.21.15.32:0/271327094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d441a8670 con 0x7f2d44109090 2024-08-31T07:32:24.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d3ea00640 1 -- 172.21.15.32:0/271327094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34018860 con 0x7f2d44109090 2024-08-31T07:32:24.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.859+0000 7f2d4b200640 1 -- 172.21.15.32:0/271327094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d441a8c00 con 0x7f2d44109090 2024-08-31T07:32:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d52ffd740 1 Processor -- start 2024-08-31T07:32:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d52ffd740 1 -- start start 2024-08-31T07:32:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761442070 0x55a7617e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 0x55a7617e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761421390 0x55a7617e85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a76175e7b0 con 0x55a761442070 2024-08-31T07:32:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a76174dd40 con 0x55a761421390 2024-08-31T07:32:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a761751cf0 con 0x55a7614191b0 2024-08-31T07:32:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d4a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761442070 0x55a7617e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d4a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761442070 0x55a7617e7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40658/0 (socket says 172.21.15.32:40658) 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.864+0000 7f2d4a800640 1 -- 172.21.15.32:0/293055840 learned_addr learned my addr 172.21.15.32:0/293055840 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d49e00640 1 --2- 172.21.15.32:0/293055840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 0x55a7617e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d3ea00640 1 -- 172.21.15.32:0/271327094 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2d34010430 con 0x7f2d44109090 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d4a800640 1 -- 172.21.15.32:0/293055840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 msgr2=0x55a7617e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d4a800640 1 --2- 172.21.15.32:0/293055840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 0x55a7617e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d4a800640 1 -- 172.21.15.32:0/293055840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761421390 msgr2=0x55a7617e85e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d4a800640 1 --2- 172.21.15.32:0/293055840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761421390 0x55a7617e85e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d4a800640 1 -- 172.21.15.32:0/293055840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7614ad650 con 0x55a761442070 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d4a800640 1 --2- 172.21.15.32:0/293055840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761442070 0x55a7617e7b60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009fd0 tx=0x7f2d400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7bb2f38854b3c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d33000640 1 -- 172.21.15.32:0/293055840 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40004d20 con 0x55a761442070 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d33000640 1 -- 172.21.15.32:0/293055840 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d40004050 con 0x55a761442070 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.869+0000 7f2d33000640 1 -- 172.21.15.32:0/293055840 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40005280 con 0x55a761442070 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- 172.21.15.32:0/293055840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761442070 msgr2=0x55a7617e7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:24.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/293055840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761442070 0x55a7617e7b60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009fd0 tx=0x7f2d400118d0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- 172.21.15.32:0/293055840 shutdown_connections 2024-08-31T07:32:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/293055840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761421390 0x55a7617e85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/293055840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 0x55a7617e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/293055840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761442070 0x55a7617e7b60 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- 172.21.15.32:0/293055840 >> 172.21.15.32:0/293055840 conn(0x55a761436460 msgr2=0x55a7617e5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- 172.21.15.32:0/293055840 shutdown_connections 2024-08-31T07:32:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- 172.21.15.32:0/293055840 wait complete. 2024-08-31T07:32:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d3ea00640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d2c0b4c30 0x7f2d2c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d3ea00640 1 -- 172.21.15.32:0/271327094 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8485+0+0 (secure 0 0 0) 0x7f2d340e5850 con 0x7f2d44109090 2024-08-31T07:32:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d2c0b4c30 0x7f2d2c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- start start 2024-08-31T07:32:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 Processor -- start 2024-08-31T07:32:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d2c0b4c30 0x7f2d2c0b7140 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2d38000c00 tx=0x7f2d38031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a761421390 0x55a7617f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761442070 0x55a7617ff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a7615b0700 con 0x55a7614191b0 2024-08-31T07:32:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a7616d4cf0 con 0x55a761442070 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a7617f6340 con 0x55a761421390 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 10 client.0 _ll_register_callbacks cb 0x55a7616fa470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617f37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40666/0 (socket says 172.21.15.32:40666) 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 -- 172.21.15.32:0/3540622399 learned_addr learned my addr 172.21.15.32:0/3540622399 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d4a800640 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761442070 0x55a7617ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 -- 172.21.15.32:0/3540622399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a761421390 msgr2=0x55a7617f3d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a761421390 0x55a7617f3d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 -- 172.21.15.32:0/3540622399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761442070 msgr2=0x55a7617ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761442070 0x55a7617ff660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 -- 172.21.15.32:0/3540622399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7617158b0 con 0x55a7614191b0 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d49e00640 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617f37b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2d38033000 tx=0x7f2d38009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52ec951f08e8274 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d27400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d27400640 1 -- 172.21.15.32:0/3540622399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38042070 con 0x55a7614191b0 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d27400640 1 -- 172.21.15.32:0/3540622399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8485+0+0 (secure 0 0 0) 0x7f2d38005240 con 0x55a7614191b0 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- 172.21.15.32:0/3540622399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d40004050 con 0x55a7614191b0 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d27400640 1 -- 172.21.15.32:0/3540622399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f2d38057050 con 0x55a7614191b0 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 10 client.19248 fetch_fsmap learned FSMap version 71 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- 172.21.15.32:0/3540622399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55a7618023d0 con 0x55a7614191b0 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d27400640 1 -- 172.21.15.32:0/3540622399 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d40004050 con 0x55a7614191b0 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d27400640 1 -- 172.21.15.32:0/3540622399 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7f2d3803a040 con 0x55a7614191b0 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 10 client.19248 fetch_fsmap finished waiting for FSMap version 71 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 10 client.19248 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 20 client.19248 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7614009f0 0x55a761805a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 4 client.19248 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:32:24.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.871+0000 7f2d52ffd740 1 -- 172.21.15.32:0/3540622399 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a761760700 con 0x55a7614009f0 2024-08-31T07:32:24.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d49400640 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7614009f0 0x55a761805a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d49400640 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7614009f0 0x55a761805a10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7387e0796d26ce5f server_cookie=261c0a56dd65eae4 in_seq=0 out_seq=0 2024-08-31T07:32:24.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d27400640 10 client.19248 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:32:24.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d27400640 1 -- 172.21.15.32:0/3540622399 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a761760700 con 0x55a7614009f0 2024-08-31T07:32:24.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d27400640 10 client.19248 handle_command_reply: tid=0 2024-08-31T07:32:24.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 client.19248 shutdown 2024-08-31T07:32:24.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 -- 172.21.15.32:0/3540622399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 msgr2=0x55a7617f37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:24.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617f37b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2d38033000 tx=0x7f2d38009260 comp rx=0 tx=0).stop 2024-08-31T07:32:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 -- 172.21.15.32:0/3540622399 shutdown_connections 2024-08-31T07:32:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7614009f0 0x55a761805a10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761442070 0x55a7617ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a761421390 0x55a7617f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/3540622399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617f37b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 -- 172.21.15.32:0/3540622399 >> 172.21.15.32:0/3540622399 conn(0x55a761436460 msgr2=0x55a7617ead40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d27400640 0 client.19248 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:32:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 -- 172.21.15.32:0/3540622399 shutdown_connections 2024-08-31T07:32:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 1 -- 172.21.15.32:0/3540622399 wait complete. 2024-08-31T07:32:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.876+0000 7f2d52ffd740 20 client.19248 trim_cache size 0 max 16384 2024-08-31T07:32:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.888+0000 7f2d52ffd740 1 Processor -- start 2024-08-31T07:32:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.888+0000 7f2d52ffd740 1 -- start start 2024-08-31T07:32:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.888+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617ff560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.888+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761421390 0x55a7617f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.888+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a761442070 0x55a7617ecde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.888+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441b14e0 con 0x55a7614191b0 2024-08-31T07:32:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.888+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441b3140 con 0x55a761421390 2024-08-31T07:32:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.888+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441b1bb0 con 0x55a761442070 2024-08-31T07:32:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.888+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617ff560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40682/0 (socket says 172.21.15.32:40682) 2024-08-31T07:32:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d49e00640 1 -- 172.21.15.32:0/2511144249 learned_addr learned my addr 172.21.15.32:0/2511144249 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d49400640 1 --2- 172.21.15.32:0/2511144249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761421390 0x55a7617f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d4a800640 1 --2- 172.21.15.32:0/2511144249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a761442070 0x55a7617ecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d49e00640 1 -- 172.21.15.32:0/2511144249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a761442070 msgr2=0x55a7617ecde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d49e00640 1 --2- 172.21.15.32:0/2511144249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a761442070 0x55a7617ecde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d49e00640 1 -- 172.21.15.32:0/2511144249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761421390 msgr2=0x55a7617f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d49e00640 1 --2- 172.21.15.32:0/2511144249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761421390 0x55a7617f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d49e00640 1 -- 172.21.15.32:0/2511144249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7617ea3f0 con 0x55a7614191b0 2024-08-31T07:32:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d4a800640 1 --2- 172.21.15.32:0/2511144249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a761442070 0x55a7617ecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:32:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.893+0000 7f2d49e00640 1 --2- 172.21.15.32:0/2511144249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617ff560 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2d380037e0 tx=0x7f2d3803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b382d37fd631489 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d32600640 1 -- 172.21.15.32:0/2511144249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38042040 con 0x55a7614191b0 2024-08-31T07:32:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d32600640 1 -- 172.21.15.32:0/2511144249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d38060070 con 0x55a7614191b0 2024-08-31T07:32:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d32600640 1 -- 172.21.15.32:0/2511144249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38035040 con 0x55a7614191b0 2024-08-31T07:32:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- 172.21.15.32:0/2511144249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 msgr2=0x55a7617ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/2511144249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617ff560 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2d380037e0 tx=0x7f2d3803c000 comp rx=0 tx=0).stop 2024-08-31T07:32:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- 172.21.15.32:0/2511144249 shutdown_connections 2024-08-31T07:32:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/2511144249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a761442070 0x55a7617ecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/2511144249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761421390 0x55a7617f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/2511144249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a7614191b0 0x55a7617ff560 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- 172.21.15.32:0/2511144249 >> 172.21.15.32:0/2511144249 conn(0x55a761436460 msgr2=0x55a7617f4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- 172.21.15.32:0/2511144249 shutdown_connections 2024-08-31T07:32:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- 172.21.15.32:0/2511144249 wait complete. 2024-08-31T07:32:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- start start 2024-08-31T07:32:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 Processor -- start 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 0x55a7617fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761421390 0x55a7617fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761442070 0x55a7617f6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441b2a40 con 0x55a761421390 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441b2340 con 0x55a761442070 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441af4e0 con 0x55a7614191b0 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 10 client.0 _ll_register_callbacks cb 0x7f2d38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 0x55a7617fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761421390 0x55a7617fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 0x55a7617fb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33504/0 (socket says 172.21.15.32:33504) 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49e00640 1 -- 172.21.15.32:0/165428792 learned_addr learned my addr 172.21.15.32:0/165428792 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d4a800640 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761442070 0x55a7617f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49e00640 1 -- 172.21.15.32:0/165428792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761442070 msgr2=0x55a7617f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49e00640 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761442070 0x55a7617f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49e00640 1 -- 172.21.15.32:0/165428792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761421390 msgr2=0x55a7617fbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49e00640 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761421390 0x55a7617fbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49e00640 1 -- 172.21.15.32:0/165428792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7617f6e00 con 0x55a7614191b0 2024-08-31T07:32:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49e00640 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 0x55a7617fb960 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2d38033000 tx=0x7f2d38003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a48bff43343b1eb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d27400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:32:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d27400640 1 -- 172.21.15.32:0/165428792 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38035040 con 0x55a7614191b0 2024-08-31T07:32:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- 172.21.15.32:0/165428792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d38060070 con 0x55a7614191b0 2024-08-31T07:32:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d27400640 1 -- 172.21.15.32:0/165428792 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8485+0+0 (secure 0 0 0) 0x7f2d38005410 con 0x55a7614191b0 2024-08-31T07:32:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d27400640 1 -- 172.21.15.32:0/165428792 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f2d38042040 con 0x55a7614191b0 2024-08-31T07:32:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 10 client.29284 fetch_fsmap learned FSMap version 71 2024-08-31T07:32:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- 172.21.15.32:0/165428792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55a761870ed0 con 0x55a7614191b0 2024-08-31T07:32:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d27400640 1 -- 172.21.15.32:0/165428792 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d38060070 con 0x55a7614191b0 2024-08-31T07:32:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d27400640 1 -- 172.21.15.32:0/165428792 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7f2d3803a070 con 0x55a7614191b0 2024-08-31T07:32:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 10 client.29284 fetch_fsmap finished waiting for FSMap version 71 2024-08-31T07:32:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 10 client.29284 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:32:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 20 client.29284 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7613d5da0 0x55a761873470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 4 client.29284 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d52ffd740 1 -- 172.21.15.32:0/165428792 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a76175c1a0 con 0x55a7613d5da0 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49400640 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7613d5da0 0x55a761873470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d49400640 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7613d5da0 0x55a761873470 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=faf54ddd11c46c70 server_cookie=b60fc574b29cbac3 in_seq=0 out_seq=0 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.894+0000 7f2d27400640 10 client.29284 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d27400640 1 -- 172.21.15.32:0/165428792 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a76175c1a0 con 0x55a7613d5da0 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d27400640 10 client.29284 handle_command_reply: tid=0 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 client.29284 shutdown 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 -- 172.21.15.32:0/165428792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 msgr2=0x55a7617fb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 0x55a7617fb960 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2d38033000 tx=0x7f2d38003c10 comp rx=0 tx=0).stop 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 -- 172.21.15.32:0/165428792 shutdown_connections 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55a7613d5da0 0x55a761873470 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a761442070 0x55a7617f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a761421390 0x55a7617fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 --2- 172.21.15.32:0/165428792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7614191b0 0x55a7617fb960 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d27400640 0 client.29284 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:32:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 -- 172.21.15.32:0/165428792 >> 172.21.15.32:0/165428792 conn(0x55a761436460 msgr2=0x55a7617e8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 -- 172.21.15.32:0/165428792 shutdown_connections 2024-08-31T07:32:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 1 -- 172.21.15.32:0/165428792 wait complete. 2024-08-31T07:32:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d52ffd740 20 client.29284 trim_cache size 0 max 16384 2024-08-31T07:32:24.904 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:32:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 -- 172.21.15.32:0/271327094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d2c0b4c30 msgr2=0x7f2d2c0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d2c0b4c30 0x7f2d2c0b7140 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2d38000c00 tx=0x7f2d38031400 comp rx=0 tx=0).stop 2024-08-31T07:32:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 -- 172.21.15.32:0/271327094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d44109090 msgr2=0x7f2d440759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d44109090 0x7f2d440759f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400e9f0 tx=0x7f2d3400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 -- 172.21.15.32:0/271327094 shutdown_connections 2024-08-31T07:32:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2d2c0b4c30 0x7f2d2c0b7140 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d441099e0 0x7f2d44075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d44109090 0x7f2d440759f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 --2- 172.21.15.32:0/271327094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d44103750 0x7f2d440754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 -- 172.21.15.32:0/271327094 >> 172.21.15.32:0/271327094 conn(0x7f2d440ff1e0 msgr2=0x7f2d44101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 -- 172.21.15.32:0/271327094 shutdown_connections 2024-08-31T07:32:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:24.900+0000 7f2d4b200640 1 -- 172.21.15.32:0/271327094 wait complete. 2024-08-31T07:32:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:24 smithi073 ceph-mon[21995]: pgmap v984: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 399 KiB/s wr, 95 op/s 2024-08-31T07:32:25.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111212 13.45 MB/sec execute 245 sec latency 28845.828 ms 2024-08-31T07:32:26.388 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:32:26.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111537 13.48 MB/sec execute 246 sec latency 25044.612 ms 2024-08-31T07:32:26.714 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:26 smithi032 ceph-mon[17252]: pgmap v985: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 397 KiB/s wr, 79 op/s 2024-08-31T07:32:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:26 smithi084 ceph-mon[22387]: pgmap v985: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 397 KiB/s wr, 79 op/s 2024-08-31T07:32:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:26 smithi073 ceph-mon[21995]: pgmap v985: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 397 KiB/s wr, 79 op/s 2024-08-31T07:32:27.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111717 13.46 MB/sec execute 247 sec latency 26044.662 ms 2024-08-31T07:32:28.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:27 smithi032 ceph-mon[17252]: osdmap e108: 12 total, 12 up, 12 in 2024-08-31T07:32:28.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:27 smithi032 ceph-mon[17252]: pgmap v987: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 448 KiB/s wr, 62 op/s 2024-08-31T07:32:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.136+0000 7f091d800640 1 -- 172.21.15.32:0/2391562312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0918077120 msgr2=0x7f0918075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.136+0000 7f091d800640 1 --2- 172.21.15.32:0/2391562312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0918077120 0x7f0918075550 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0908009900 tx=0x7f090802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.136+0000 7f091d800640 1 -- 172.21.15.32:0/2391562312 shutdown_connections 2024-08-31T07:32:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.136+0000 7f091d800640 1 --2- 172.21.15.32:0/2391562312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0918114020 0x7f09181164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.136+0000 7f091d800640 1 --2- 172.21.15.32:0/2391562312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0918075b70 0x7f0918075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.136+0000 7f091d800640 1 --2- 172.21.15.32:0/2391562312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0918077120 0x7f0918075550 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.136+0000 7f091d800640 1 -- 172.21.15.32:0/2391562312 >> 172.21.15.32:0/2391562312 conn(0x7f09180ff1e0 msgr2=0x7f0918101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.136+0000 7f091d800640 1 -- 172.21.15.32:0/2391562312 shutdown_connections 2024-08-31T07:32:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.136+0000 7f091d800640 1 -- 172.21.15.32:0/2391562312 wait complete. 2024-08-31T07:32:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.138+0000 7f091d800640 1 Processor -- start 2024-08-31T07:32:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.138+0000 7f091d800640 1 -- start start 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f091d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0918075b70 0x7f09181a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f091d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0918077120 0x7f09181a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f091d800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0918114020 0x7f09181a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f091d800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0918119cb0 con 0x7f0918077120 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f091d800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0918119b30 con 0x7f0918075b70 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f091d800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0918119e30 con 0x7f0918114020 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f0917400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0918077120 0x7f09181a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f0917e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0918075b70 0x7f09181a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f0917400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0918077120 0x7f09181a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60642/0 (socket says 172.21.15.32:60642) 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f0917400640 1 -- 172.21.15.32:0/4287665937 learned_addr learned my addr 172.21.15.32:0/4287665937 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.139+0000 7f091ce00640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0918114020 0x7f09181a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f0917400640 1 -- 172.21.15.32:0/4287665937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0918114020 msgr2=0x7f09181a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:28.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f0917400640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0918114020 0x7f09181a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f0917400640 1 -- 172.21.15.32:0/4287665937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0918075b70 msgr2=0x7f09181a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:28.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f0917400640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0918075b70 0x7f09181a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f0917400640 1 -- 172.21.15.32:0/4287665937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09181a65f0 con 0x7f0918077120 2024-08-31T07:32:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f0917e00640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0918075b70 0x7f09181a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f091ce00640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0918114020 0x7f09181a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f0917400640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0918077120 0x7f09181a0c20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f090400e9c0 tx=0x7f090400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f0914c00640 1 -- 172.21.15.32:0/4287665937 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090400cd80 con 0x7f0918077120 2024-08-31T07:32:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f091d800640 1 -- 172.21.15.32:0/4287665937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09181a68e0 con 0x7f0918077120 2024-08-31T07:32:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f0914c00640 1 -- 172.21.15.32:0/4287665937 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09040054d0 con 0x7f0918077120 2024-08-31T07:32:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f0914c00640 1 -- 172.21.15.32:0/4287665937 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904010430 con 0x7f0918077120 2024-08-31T07:32:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.140+0000 7f091d800640 1 -- 172.21.15.32:0/4287665937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09181ad040 con 0x7f0918077120 2024-08-31T07:32:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.144+0000 7f0925bc2740 1 Processor -- start 2024-08-31T07:32:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.144+0000 7f0925bc2740 1 -- start start 2024-08-31T07:32:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.144+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77a0070 0x5556f7b45b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.144+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77771b0 0x5556f7b460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.144+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f777f390 0x5556f7b465e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.144+0000 7f0925bc2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5556f7abc7b0 con 0x5556f777f390 2024-08-31T07:32:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.144+0000 7f0925bc2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5556f7aabd40 con 0x5556f77771b0 2024-08-31T07:32:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.144+0000 7f0925bc2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5556f7aafcf0 con 0x5556f77a0070 2024-08-31T07:32:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f0914c00640 1 -- 172.21.15.32:0/4287665937 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0904004fd0 con 0x7f0918077120 2024-08-31T07:32:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f091ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77a0070 0x5556f7b45b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f091ce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77a0070 0x5556f7b45b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46898/0 (socket says 172.21.15.32:46898) 2024-08-31T07:32:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f091ce00640 1 -- 172.21.15.32:0/742002933 learned_addr learned my addr 172.21.15.32:0/742002933 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f0917400640 1 --2- 172.21.15.32:0/742002933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f777f390 0x5556f7b465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f091ce00640 1 -- 172.21.15.32:0/742002933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77771b0 msgr2=0x5556f7b460a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:32:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f091ce00640 1 --2- 172.21.15.32:0/742002933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77771b0 0x5556f7b460a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f091ce00640 1 -- 172.21.15.32:0/742002933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f777f390 msgr2=0x5556f7b465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f091ce00640 1 --2- 172.21.15.32:0/742002933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f777f390 0x5556f7b465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f091ce00640 1 -- 172.21.15.32:0/742002933 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f780b650 con 0x5556f77a0070 2024-08-31T07:32:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f0917400640 1 --2- 172.21.15.32:0/742002933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f777f390 0x5556f7b465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:32:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f091ce00640 1 --2- 172.21.15.32:0/742002933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77a0070 0x5556f7b45b60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f09100117e0 tx=0x7f0910011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae492b71d265ab9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f090d600640 1 -- 172.21.15.32:0/742002933 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910011310 con 0x5556f77a0070 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f090d600640 1 -- 172.21.15.32:0/742002933 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0910003e20 con 0x5556f77a0070 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f090d600640 1 -- 172.21.15.32:0/742002933 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100044f0 con 0x5556f77a0070 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f0914c00640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f08fc0b4ca0 0x7f08fc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f0914c00640 1 -- 172.21.15.32:0/4287665937 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f09040e5960 con 0x7f0918077120 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f0925bc2740 1 -- 172.21.15.32:0/742002933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77a0070 msgr2=0x5556f7b45b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f0925bc2740 1 --2- 172.21.15.32:0/742002933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77a0070 0x5556f7b45b60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f09100117e0 tx=0x7f0910011ca0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.145+0000 7f0917e00640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f08fc0b4ca0 0x7f08fc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- 172.21.15.32:0/742002933 shutdown_connections 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 --2- 172.21.15.32:0/742002933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f777f390 0x5556f7b465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 --2- 172.21.15.32:0/742002933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77771b0 0x5556f7b460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 --2- 172.21.15.32:0/742002933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77a0070 0x5556f7b45b60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- 172.21.15.32:0/742002933 >> 172.21.15.32:0/742002933 conn(0x5556f7794460 msgr2=0x5556f7b43a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- 172.21.15.32:0/742002933 shutdown_connections 2024-08-31T07:32:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- 172.21.15.32:0/742002933 wait complete. 2024-08-31T07:32:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917e00640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f08fc0b4ca0 0x7f08fc0b71b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0908005ab0 tx=0x7f0908005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- start start 2024-08-31T07:32:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 Processor -- start 2024-08-31T07:32:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77771b0 0x5556f7b516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f777f390 0x5556f7b51c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77a0070 0x5556f7b5d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5556f790e700 con 0x5556f77a0070 2024-08-31T07:32:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5556f7a32cf0 con 0x5556f777f390 2024-08-31T07:32:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5556f7b54340 con 0x5556f77771b0 2024-08-31T07:32:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 10 client.0 _ll_register_callbacks cb 0x5556f7a58470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77771b0 0x5556f7b516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77771b0 0x5556f7b516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46912/0 (socket says 172.21.15.32:46912) 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917e00640 1 -- 172.21.15.32:0/3609168698 learned_addr learned my addr 172.21.15.32:0/3609168698 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f091ce00640 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77a0070 0x5556f7b5d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917400640 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f777f390 0x5556f7b51c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917e00640 1 -- 172.21.15.32:0/3609168698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f777f390 msgr2=0x5556f7b51c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917e00640 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f777f390 0x5556f7b51c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917e00640 1 -- 172.21.15.32:0/3609168698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77a0070 msgr2=0x5556f7b5d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917e00640 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77a0070 0x5556f7b5d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917e00640 1 -- 172.21.15.32:0/3609168698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f7a738b0 con 0x5556f77771b0 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917e00640 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77771b0 0x5556f7b516c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f09080096c0 tx=0x7f0908009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0822dd9a31b5635 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:28.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0902000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0902000640 1 -- 172.21.15.32:0/3609168698 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090804d020 con 0x5556f77771b0 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0902000640 1 -- 172.21.15.32:0/3609168698 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f0908004b80 con 0x5556f77771b0 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- 172.21.15.32:0/3609168698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0910003e20 con 0x5556f77771b0 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0902000640 1 -- 172.21.15.32:0/3609168698 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f0908058050 con 0x5556f77771b0 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 10 client.29296 fetch_fsmap learned FSMap version 71 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- 172.21.15.32:0/3609168698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5556f7b60270 con 0x5556f77771b0 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0902000640 1 -- 172.21.15.32:0/3609168698 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0910003e20 con 0x5556f77771b0 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0902000640 1 -- 172.21.15.32:0/3609168698 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7f09080325d0 con 0x5556f77771b0 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 10 client.29296 fetch_fsmap finished waiting for FSMap version 71 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 10 client.29296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 20 client.29296 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5556f775e9f0 0x5556f7b63870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 4 client.29296 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:32:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 -- 172.21.15.32:0/3609168698 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556f7abe700 con 0x5556f775e9f0 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917400640 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5556f775e9f0 0x5556f7b63870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0917400640 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5556f775e9f0 0x5556f7b63870 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97b7e1d0f3b7a095 server_cookie=96fd60c5cdc5b1a8 in_seq=0 out_seq=0 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0902000640 10 client.29296 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0902000640 1 -- 172.21.15.32:0/3609168698 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5556f7abe700 con 0x5556f775e9f0 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0902000640 10 client.29296 handle_command_reply: tid=0 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.148+0000 7f0925bc2740 1 client.29296 shutdown 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 1 -- 172.21.15.32:0/3609168698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77771b0 msgr2=0x5556f7b516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77771b0 0x5556f7b516c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f09080096c0 tx=0x7f0908009260 comp rx=0 tx=0).stop 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 1 -- 172.21.15.32:0/3609168698 shutdown_connections 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5556f775e9f0 0x5556f7b63870 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77a0070 0x5556f7b5d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f777f390 0x5556f7b51c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 1 --2- 172.21.15.32:0/3609168698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f77771b0 0x5556f7b516c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0902000640 0 client.29296 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:32:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 1 -- 172.21.15.32:0/3609168698 >> 172.21.15.32:0/3609168698 conn(0x5556f7794460 msgr2=0x5556f7b48cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 1 -- 172.21.15.32:0/3609168698 shutdown_connections 2024-08-31T07:32:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 1 -- 172.21.15.32:0/3609168698 wait complete. 2024-08-31T07:32:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.153+0000 7f0925bc2740 20 client.29296 trim_cache size 0 max 16384 2024-08-31T07:32:28.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0925bc2740 1 Processor -- start 2024-08-31T07:32:28.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0925bc2740 1 -- start start 2024-08-31T07:32:28.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 0x5556f7b47550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 0x5556f7b5d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0925bc2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09181b5800 con 0x5556f77771b0 2024-08-31T07:32:28.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0925bc2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f09181b7460 con 0x5556f77a0070 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0925bc2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f09181b5ed0 con 0x5556f777f390 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0917e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 0x5556f7b47550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0917400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0917e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 0x5556f7b47550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60674/0 (socket says 172.21.15.32:60674) 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0917e00640 1 -- 172.21.15.32:0/3000695936 learned_addr learned my addr 172.21.15.32:0/3000695936 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f091ce00640 1 --2- 172.21.15.32:0/3000695936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 0x5556f7b5d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0917400640 1 -- 172.21.15.32:0/3000695936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 msgr2=0x5556f7b5d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0917400640 1 --2- 172.21.15.32:0/3000695936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 0x5556f7b5d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0917400640 1 -- 172.21.15.32:0/3000695936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 msgr2=0x5556f7b47550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0917400640 1 --2- 172.21.15.32:0/3000695936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 0x5556f7b47550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0917400640 1 -- 172.21.15.32:0/3000695936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f7b52780 con 0x5556f777f390 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f091ce00640 1 --2- 172.21.15.32:0/3000695936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 0x5556f7b5d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:32:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f0917400640 1 --2- 172.21.15.32:0/3000695936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b530f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f09040e30a0 tx=0x7f09040049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc780fde0ad69265 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f090cc00640 1 -- 172.21.15.32:0/3000695936 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09040abb50 con 0x5556f777f390 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f090cc00640 1 -- 172.21.15.32:0/3000695936 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09040d1070 con 0x5556f777f390 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.165+0000 7f090cc00640 1 -- 172.21.15.32:0/3000695936 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09040e3320 con 0x5556f777f390 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 -- 172.21.15.32:0/3000695936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 msgr2=0x5556f7b530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 --2- 172.21.15.32:0/3000695936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b530f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f09040e30a0 tx=0x7f09040049b0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 -- 172.21.15.32:0/3000695936 shutdown_connections 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 --2- 172.21.15.32:0/3000695936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 0x5556f7b5d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 --2- 172.21.15.32:0/3000695936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b530f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 --2- 172.21.15.32:0/3000695936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 0x5556f7b47550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 -- 172.21.15.32:0/3000695936 >> 172.21.15.32:0/3000695936 conn(0x5556f7794460 msgr2=0x5556f7b529e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 -- 172.21.15.32:0/3000695936 shutdown_connections 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 -- 172.21.15.32:0/3000695936 wait complete. 2024-08-31T07:32:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 -- start start 2024-08-31T07:32:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.170+0000 7f0925bc2740 1 Processor -- start 2024-08-31T07:32:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 0x5556f7b55c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 0x5556f7bcd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09181b6d60 con 0x5556f77771b0 2024-08-31T07:32:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f09181b6660 con 0x5556f77a0070 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f09181b3800 con 0x5556f777f390 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 10 client.0 _ll_register_callbacks cb 0x7f0908009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 0x5556f7b55c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46928/0 (socket says 172.21.15.32:46928) 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917400640 1 -- 172.21.15.32:0/4288470894 learned_addr learned my addr 172.21.15.32:0/4288470894 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091ce00640 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 0x5556f7bcd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917400640 1 -- 172.21.15.32:0/4288470894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 msgr2=0x5556f7bcd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917400640 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 0x5556f7bcd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917400640 1 -- 172.21.15.32:0/4288470894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 msgr2=0x5556f7b55c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917400640 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 0x5556f7b55c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917400640 1 -- 172.21.15.32:0/4288470894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f7bcd770 con 0x5556f777f390 2024-08-31T07:32:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917400640 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b561a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f09040abfd0 tx=0x7f09040a8500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db6aede7a4a9ee9f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0902000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0902000640 1 -- 172.21.15.32:0/4288470894 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09040e3530 con 0x5556f777f390 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0902000640 1 -- 172.21.15.32:0/4288470894 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f09040ac2a0 con 0x5556f777f390 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- 172.21.15.32:0/4288470894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09040d1070 con 0x5556f777f390 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0902000640 1 -- 172.21.15.32:0/4288470894 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f09040ac740 con 0x5556f777f390 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 10 client.29302 fetch_fsmap learned FSMap version 71 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- 172.21.15.32:0/4288470894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5556f7bce950 con 0x5556f777f390 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0902000640 1 -- 172.21.15.32:0/4288470894 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09040d1070 con 0x5556f777f390 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0902000640 1 -- 172.21.15.32:0/4288470894 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7f09040a79b0 con 0x5556f777f390 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 10 client.29302 fetch_fsmap finished waiting for FSMap version 71 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 10 client.29302 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 20 client.29302 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5556f7733da0 0x5556f7bd1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:28.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 4 client.29302 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:32:28.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- 172.21.15.32:0/4288470894 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556f7aba1a0 con 0x5556f7733da0 2024-08-31T07:32:28.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917e00640 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5556f7733da0 0x5556f7bd1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:28.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0917e00640 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5556f7733da0 0x5556f7bd1140 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14edef0f4e1c601a server_cookie=abef9f84d1e7e900 in_seq=0 out_seq=0 2024-08-31T07:32:28.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0902000640 10 client.29302 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:32:28.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0902000640 1 -- 172.21.15.32:0/4288470894 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556f7aba1a0 con 0x5556f7733da0 2024-08-31T07:32:28.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0902000640 10 client.29302 handle_command_reply: tid=0 2024-08-31T07:32:28.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 client.29302 shutdown 2024-08-31T07:32:28.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- 172.21.15.32:0/4288470894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 msgr2=0x5556f7b561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:28.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b561a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f09040abfd0 tx=0x7f09040a8500 comp rx=0 tx=0).stop 2024-08-31T07:32:28.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- 172.21.15.32:0/4288470894 shutdown_connections 2024-08-31T07:32:28.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5556f7733da0 0x5556f7bd1140 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5556f77a0070 0x5556f7bcd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556f777f390 0x5556f7b561a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 --2- 172.21.15.32:0/4288470894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556f77771b0 0x5556f7b55c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- 172.21.15.32:0/4288470894 >> 172.21.15.32:0/4288470894 conn(0x5556f7794460 msgr2=0x5556f7b47320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0902000640 0 client.29302 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:32:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- 172.21.15.32:0/4288470894 shutdown_connections 2024-08-31T07:32:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 1 -- 172.21.15.32:0/4288470894 wait complete. 2024-08-31T07:32:28.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f0925bc2740 20 client.29302 trim_cache size 0 max 16384 2024-08-31T07:32:28.179 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:32:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 -- 172.21.15.32:0/4287665937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f08fc0b4ca0 msgr2=0x7f08fc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f08fc0b4ca0 0x7f08fc0b71b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0908005ab0 tx=0x7f0908005a20 comp rx=0 tx=0).stop 2024-08-31T07:32:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 -- 172.21.15.32:0/4287665937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0918077120 msgr2=0x7f09181a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0918077120 0x7f09181a0c20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f090400e9c0 tx=0x7f090400ee80 comp rx=0 tx=0).stop 2024-08-31T07:32:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 -- 172.21.15.32:0/4287665937 shutdown_connections 2024-08-31T07:32:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f08fc0b4ca0 0x7f08fc0b71b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0918114020 0x7f09181a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0918077120 0x7f09181a0c20 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 --2- 172.21.15.32:0/4287665937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0918075b70 0x7f09181a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 -- 172.21.15.32:0/4287665937 >> 172.21.15.32:0/4287665937 conn(0x7f09180ff1e0 msgr2=0x7f091810ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 -- 172.21.15.32:0/4287665937 shutdown_connections 2024-08-31T07:32:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:28.172+0000 7f091d800640 1 -- 172.21.15.32:0/4287665937 wait complete. 2024-08-31T07:32:28.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:27 smithi084 ceph-mon[22387]: osdmap e108: 12 total, 12 up, 12 in 2024-08-31T07:32:28.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:27 smithi084 ceph-mon[22387]: pgmap v987: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 448 KiB/s wr, 62 op/s 2024-08-31T07:32:28.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111717 13.41 MB/sec execute 248 sec latency 27044.708 ms 2024-08-31T07:32:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:27 smithi073 ceph-mon[21995]: osdmap e108: 12 total, 12 up, 12 in 2024-08-31T07:32:28.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:27 smithi073 ceph-mon[21995]: pgmap v987: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 448 KiB/s wr, 62 op/s 2024-08-31T07:32:29.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:32:29.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 112115 13.44 MB/sec execute 249 sec latency 27292.941 ms 2024-08-31T07:32:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:30.400 INFO:tasks.workunit.client.0.smithi032.stdout: 10 112955 13.53 MB/sec execute 250 sec latency 72.438 ms 2024-08-31T07:32:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:30 smithi032 ceph-mon[17252]: pgmap v988: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 489 KiB/s wr, 63 op/s 2024-08-31T07:32:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:30 smithi073 ceph-mon[21995]: pgmap v988: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 489 KiB/s wr, 63 op/s 2024-08-31T07:32:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.677+0000 7f0f35c00640 1 -- 172.21.15.32:0/1866427971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f30075ea0 msgr2=0x7f0f3010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.677+0000 7f0f35c00640 1 --2- 172.21.15.32:0/1866427971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f30075ea0 0x7f0f3010f930 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c013150 tx=0x7f0f1c032760 comp rx=0 tx=0).stop 2024-08-31T07:32:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.677+0000 7f0f35c00640 1 -- 172.21.15.32:0/1866427971 shutdown_connections 2024-08-31T07:32:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.677+0000 7f0f35c00640 1 --2- 172.21.15.32:0/1866427971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f3010fe70 0x7f0f301122b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.677+0000 7f0f35c00640 1 --2- 172.21.15.32:0/1866427971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f30075ea0 0x7f0f3010f930 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.677+0000 7f0f35c00640 1 --2- 172.21.15.32:0/1866427971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30075550 0x7f0f30075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.677+0000 7f0f35c00640 1 -- 172.21.15.32:0/1866427971 >> 172.21.15.32:0/1866427971 conn(0x7f0f300ff380 msgr2=0x7f0f301017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.677+0000 7f0f35c00640 1 -- 172.21.15.32:0/1866427971 shutdown_connections 2024-08-31T07:32:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.677+0000 7f0f35c00640 1 -- 172.21.15.32:0/1866427971 wait complete. 2024-08-31T07:32:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.679+0000 7f0f35c00640 1 Processor -- start 2024-08-31T07:32:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.679+0000 7f0f35c00640 1 -- start start 2024-08-31T07:32:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.680+0000 7f0f35c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f30075550 0x7f0f301980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.680+0000 7f0f35c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30075ea0 0x7f0f30198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.680+0000 7f0f35c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f3010fe70 0x7f0f3019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.680+0000 7f0f35c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30105cf0 con 0x7f0f3010fe70 2024-08-31T07:32:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.680+0000 7f0f35c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30105b70 con 0x7f0f30075550 2024-08-31T07:32:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.680+0000 7f0f35c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30105e70 con 0x7f0f30075ea0 2024-08-31T07:32:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.681+0000 7f0f2f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f3010fe70 0x7f0f3019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.681+0000 7f0f2ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f30075550 0x7f0f301980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.681+0000 7f0f2ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f30075550 0x7f0f301980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46402/0 (socket says 172.21.15.32:46402) 2024-08-31T07:32:30.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.681+0000 7f0f2ea00640 1 -- 172.21.15.32:0/3919638504 learned_addr learned my addr 172.21.15.32:0/3919638504 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.681+0000 7f0f2e000640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30075ea0 0x7f0f30198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.682+0000 7f0f2e000640 1 -- 172.21.15.32:0/3919638504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f30075550 msgr2=0x7f0f301980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.682+0000 7f0f2e000640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f30075550 0x7f0f301980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.682+0000 7f0f2e000640 1 -- 172.21.15.32:0/3919638504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f3010fe70 msgr2=0x7f0f3019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.682+0000 7f0f2e000640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f3010fe70 0x7f0f3019d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.682+0000 7f0f2e000640 1 -- 172.21.15.32:0/3919638504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3019def0 con 0x7f0f30075ea0 2024-08-31T07:32:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.683+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f30075550 0x7f0f301980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.684+0000 7f0f2e000640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30075ea0 0x7f0f30198660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c013120 tx=0x7f0f1c011370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.684+0000 7f0f27400640 1 -- 172.21.15.32:0/3919638504 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c047070 con 0x7f0f30075ea0 2024-08-31T07:32:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.684+0000 7f0f35c00640 1 -- 172.21.15.32:0/3919638504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3019e180 con 0x7f0f30075ea0 2024-08-31T07:32:30.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.684+0000 7f0f35c00640 1 -- 172.21.15.32:0/3919638504 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f301a4940 con 0x7f0f30075ea0 2024-08-31T07:32:30.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.685+0000 7f0f27400640 1 -- 172.21.15.32:0/3919638504 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f1c004d40 con 0x7f0f30075ea0 2024-08-31T07:32:30.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.685+0000 7f0f27400640 1 -- 172.21.15.32:0/3919638504 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c0038e0 con 0x7f0f30075ea0 2024-08-31T07:32:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.690+0000 7f0f3c522740 1 Processor -- start 2024-08-31T07:32:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.690+0000 7f0f3c522740 1 -- start start 2024-08-31T07:32:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.691+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d026a070 0x5625d060f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.691+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 0x5625d060fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.691+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 0x5625d0610330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.691+0000 7f0f3c522740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5625d05867b0 con 0x5625d02411b0 2024-08-31T07:32:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.691+0000 7f0f3c522740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5625d0575d40 con 0x5625d026a070 2024-08-31T07:32:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.691+0000 7f0f3c522740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5625d0579cf0 con 0x5625d0249390 2024-08-31T07:32:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.691+0000 7f0f2f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d026a070 0x5625d060f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.691+0000 7f0f2f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d026a070 0x5625d060f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46416/0 (socket says 172.21.15.32:46416) 2024-08-31T07:32:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.691+0000 7f0f2f400640 1 -- 172.21.15.32:0/584153059 learned_addr learned my addr 172.21.15.32:0/584153059 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.691+0000 7f0f27400640 1 -- 172.21.15.32:0/3919638504 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0f1c0046c0 con 0x7f0f30075ea0 2024-08-31T07:32:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.692+0000 7f0f2f400640 1 -- 172.21.15.32:0/584153059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 msgr2=0x5625d0610330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:32:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.692+0000 7f0f2f400640 1 --2- 172.21.15.32:0/584153059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 0x5625d0610330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.692+0000 7f0f2f400640 1 -- 172.21.15.32:0/584153059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 msgr2=0x5625d060fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:32:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.693+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/584153059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 0x5625d060fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.693+0000 7f0f2f400640 1 --2- 172.21.15.32:0/584153059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 0x5625d060fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.693+0000 7f0f2f400640 1 -- 172.21.15.32:0/584153059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625d02d5650 con 0x5625d026a070 2024-08-31T07:32:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.693+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/584153059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 0x5625d060fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:32:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.694+0000 7f0f27400640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0f140b4dd0 0x7f0f140b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.694+0000 7f0f27400640 1 -- 172.21.15.32:0/3919638504 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f0f1c10d630 con 0x7f0f30075ea0 2024-08-31T07:32:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.695+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0f140b4dd0 0x7f0f140b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.696+0000 7f0f2f400640 1 --2- 172.21.15.32:0/584153059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d026a070 0x5625d060f8b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0f28010370 tx=0x7f0f28010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7df5744283736c2b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:30.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.696+0000 7f0f1be00640 1 -- 172.21.15.32:0/584153059 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28016070 con 0x5625d026a070 2024-08-31T07:32:30.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.697+0000 7f0f1be00640 1 -- 172.21.15.32:0/584153059 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f28004d20 con 0x5625d026a070 2024-08-31T07:32:30.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.697+0000 7f0f1be00640 1 -- 172.21.15.32:0/584153059 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28005420 con 0x5625d026a070 2024-08-31T07:32:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.699+0000 7f0f3c522740 1 -- 172.21.15.32:0/584153059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d026a070 msgr2=0x5625d060f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.699+0000 7f0f3c522740 1 --2- 172.21.15.32:0/584153059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d026a070 0x5625d060f8b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0f28010370 tx=0x7f0f28010830 comp rx=0 tx=0).stop 2024-08-31T07:32:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.699+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0f140b4dd0 0x7f0f140b72e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x5625d060e530 tx=0x7f0f20005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.699+0000 7f0f3c522740 1 -- 172.21.15.32:0/584153059 shutdown_connections 2024-08-31T07:32:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.699+0000 7f0f3c522740 1 --2- 172.21.15.32:0/584153059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 0x5625d0610330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.699+0000 7f0f3c522740 1 --2- 172.21.15.32:0/584153059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 0x5625d060fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.699+0000 7f0f3c522740 1 --2- 172.21.15.32:0/584153059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d026a070 0x5625d060f8b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.699+0000 7f0f3c522740 1 -- 172.21.15.32:0/584153059 >> 172.21.15.32:0/584153059 conn(0x5625d025e460 msgr2=0x5625d060d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.700+0000 7f0f3c522740 1 -- 172.21.15.32:0/584153059 shutdown_connections 2024-08-31T07:32:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.700+0000 7f0f3c522740 1 -- 172.21.15.32:0/584153059 wait complete. 2024-08-31T07:32:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.701+0000 7f0f3c522740 1 -- start start 2024-08-31T07:32:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.701+0000 7f0f3c522740 1 Processor -- start 2024-08-31T07:32:30.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.702+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d02411b0 0x5625d061b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.702+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d0249390 0x5625d061bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.702+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d026a070 0x5625d0627410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.703+0000 7f0f3c522740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5625d03d8700 con 0x5625d026a070 2024-08-31T07:32:30.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.703+0000 7f0f3c522740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5625d04fccf0 con 0x5625d0249390 2024-08-31T07:32:30.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.703+0000 7f0f3c522740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5625d061e090 con 0x5625d02411b0 2024-08-31T07:32:30.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.703+0000 7f0f3c522740 10 client.0 _ll_register_callbacks cb 0x5625d0522470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.704+0000 7f0f2f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d026a070 0x5625d0627410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.704+0000 7f0f2ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d02411b0 0x5625d061b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.704+0000 7f0f2e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d0249390 0x5625d061bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.704+0000 7f0f2ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d02411b0 0x5625d061b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46988/0 (socket says 172.21.15.32:46988) 2024-08-31T07:32:30.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.704+0000 7f0f2ea00640 1 -- 172.21.15.32:0/1767974594 learned_addr learned my addr 172.21.15.32:0/1767974594 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.705+0000 7f0f2e000640 1 -- 172.21.15.32:0/1767974594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d02411b0 msgr2=0x5625d061b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.705+0000 7f0f2e000640 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d02411b0 0x5625d061b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.705+0000 7f0f2e000640 1 -- 172.21.15.32:0/1767974594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d026a070 msgr2=0x5625d0627410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.705+0000 7f0f2e000640 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d026a070 0x5625d0627410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.705+0000 7f0f2e000640 1 -- 172.21.15.32:0/1767974594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625d053d8b0 con 0x5625d0249390 2024-08-31T07:32:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.705+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d02411b0 0x5625d061b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.706+0000 7f0f2e000640 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d0249390 0x5625d061bb10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c10ad70 tx=0x7f0f1c004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f37368226da98d54 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.706+0000 7f0f18c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:32:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.707+0000 7f0f18c00640 1 -- 172.21.15.32:0/1767974594 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c050040 con 0x5625d0249390 2024-08-31T07:32:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.707+0000 7f0f18c00640 1 -- 172.21.15.32:0/1767974594 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f0f1c0d81c0 con 0x5625d0249390 2024-08-31T07:32:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.707+0000 7f0f3c522740 1 -- 172.21.15.32:0/1767974594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f28004d20 con 0x5625d0249390 2024-08-31T07:32:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.708+0000 7f0f18c00640 1 -- 172.21.15.32:0/1767974594 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f0f1c0bf0d0 con 0x5625d0249390 2024-08-31T07:32:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.708+0000 7f0f3c522740 10 client.29330 fetch_fsmap learned FSMap version 71 2024-08-31T07:32:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.708+0000 7f0f3c522740 1 -- 172.21.15.32:0/1767974594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5625d062a130 con 0x5625d0249390 2024-08-31T07:32:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.709+0000 7f0f18c00640 1 -- 172.21.15.32:0/1767974594 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f28004d20 con 0x5625d0249390 2024-08-31T07:32:30.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.711+0000 7f0f18c00640 1 -- 172.21.15.32:0/1767974594 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7f0f1c053060 con 0x5625d0249390 2024-08-31T07:32:30.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.711+0000 7f0f3c522740 10 client.29330 fetch_fsmap finished waiting for FSMap version 71 2024-08-31T07:32:30.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.711+0000 7f0f3c522740 10 client.29330 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:32:30.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.711+0000 7f0f3c522740 20 client.29330 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:30.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.711+0000 7f0f3c522740 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5625d02289f0 0x5625d062d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.711+0000 7f0f3c522740 4 client.29330 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:32:30.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.711+0000 7f0f3c522740 1 -- 172.21.15.32:0/1767974594 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625d0588700 con 0x5625d02289f0 2024-08-31T07:32:30.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.712+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5625d02289f0 0x5625d062d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.714+0000 7f0f18c00640 10 client.29330 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:32:30.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.714+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5625d02289f0 0x5625d062d6d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da9ee0807ed21762 server_cookie=b316d310729d3929 in_seq=0 out_seq=0 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.716+0000 7f0f18c00640 1 -- 172.21.15.32:0/1767974594 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625d0588700 con 0x5625d02289f0 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.716+0000 7f0f18c00640 10 client.29330 handle_command_reply: tid=0 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.716+0000 7f0f3c522740 1 client.29330 shutdown 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.716+0000 7f0f3c522740 1 -- 172.21.15.32:0/1767974594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d0249390 msgr2=0x5625d061bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.716+0000 7f0f3c522740 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d0249390 0x5625d061bb10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c10ad70 tx=0x7f0f1c004be0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.717+0000 7f0f3c522740 1 -- 172.21.15.32:0/1767974594 shutdown_connections 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.717+0000 7f0f3c522740 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d026a070 0x5625d0627410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.717+0000 7f0f3c522740 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5625d02289f0 0x5625d062d6d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.717+0000 7f0f3c522740 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d0249390 0x5625d061bb10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.717+0000 7f0f3c522740 1 --2- 172.21.15.32:0/1767974594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d02411b0 0x5625d061b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.717+0000 7f0f3c522740 1 -- 172.21.15.32:0/1767974594 >> 172.21.15.32:0/1767974594 conn(0x5625d025e460 msgr2=0x5625d0612a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:30.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.718+0000 7f0f18c00640 0 client.29330 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:32:30.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.718+0000 7f0f3c522740 1 -- 172.21.15.32:0/1767974594 shutdown_connections 2024-08-31T07:32:30.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.718+0000 7f0f3c522740 1 -- 172.21.15.32:0/1767974594 wait complete. 2024-08-31T07:32:30.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.719+0000 7f0f3c522740 20 client.29330 trim_cache size 0 max 16384 2024-08-31T07:32:30.733 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 32.8 2024-08-31T07:32:30.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:32:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:30 smithi084 ceph-mon[22387]: pgmap v988: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 489 KiB/s wr, 63 op/s 2024-08-31T07:32:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.747+0000 7f0f3c522740 1 Processor -- start 2024-08-31T07:32:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.747+0000 7f0f3c522740 1 -- start start 2024-08-31T07:32:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.748+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 0x5625d06149c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.748+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d0249390 0x5625d0614f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.748+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d026a070 0x5625d060d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.748+0000 7f0f3c522740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1000e6a0 con 0x5625d02411b0 2024-08-31T07:32:30.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.748+0000 7f0f3c522740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10010300 con 0x5625d0249390 2024-08-31T07:32:30.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.748+0000 7f0f3c522740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1000ed70 con 0x5625d026a070 2024-08-31T07:32:30.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.748+0000 7f0f2f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d026a070 0x5625d060d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.748+0000 7f0f2f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d026a070 0x5625d060d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46992/0 (socket says 172.21.15.32:46992) 2024-08-31T07:32:30.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.748+0000 7f0f2f400640 1 -- 172.21.15.32:0/377849403 learned_addr learned my addr 172.21.15.32:0/377849403 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.749+0000 7f0f2f400640 1 -- 172.21.15.32:0/377849403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d0249390 msgr2=0x5625d0614f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.749+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/377849403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 0x5625d06149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.749+0000 7f0f2f400640 1 --2- 172.21.15.32:0/377849403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d0249390 0x5625d0614f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.749+0000 7f0f2f400640 1 -- 172.21.15.32:0/377849403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 msgr2=0x5625d06149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:30.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.749+0000 7f0f2f400640 1 --2- 172.21.15.32:0/377849403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 0x5625d06149c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.749+0000 7f0f2f400640 1 -- 172.21.15.32:0/377849403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625d061c610 con 0x5625d026a070 2024-08-31T07:32:30.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.750+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/377849403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 0x5625d06149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:32:30.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.751+0000 7f0f2f400640 1 --2- 172.21.15.32:0/377849403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d026a070 0x5625d060d0c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009f40 tx=0x7f0f28003bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df80822f5dc9319 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:30.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.751+0000 7f0f1b400640 1 -- 172.21.15.32:0/377849403 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28029070 con 0x5625d026a070 2024-08-31T07:32:30.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.752+0000 7f0f1b400640 1 -- 172.21.15.32:0/377849403 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f28005560 con 0x5625d026a070 2024-08-31T07:32:30.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.752+0000 7f0f1b400640 1 -- 172.21.15.32:0/377849403 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28016070 con 0x5625d026a070 2024-08-31T07:32:30.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.753+0000 7f0f3c522740 1 -- 172.21.15.32:0/377849403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d026a070 msgr2=0x5625d060d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:30.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.753+0000 7f0f3c522740 1 --2- 172.21.15.32:0/377849403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d026a070 0x5625d060d0c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009f40 tx=0x7f0f28003bb0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.754+0000 7f0f3c522740 1 -- 172.21.15.32:0/377849403 shutdown_connections 2024-08-31T07:32:30.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.754+0000 7f0f3c522740 1 --2- 172.21.15.32:0/377849403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d026a070 0x5625d060d0c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.754+0000 7f0f3c522740 1 --2- 172.21.15.32:0/377849403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d0249390 0x5625d0614f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.754+0000 7f0f3c522740 1 --2- 172.21.15.32:0/377849403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d02411b0 0x5625d06149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.754+0000 7f0f3c522740 1 -- 172.21.15.32:0/377849403 >> 172.21.15.32:0/377849403 conn(0x5625d025e460 msgr2=0x5625d0619bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:30.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.754+0000 7f0f3c522740 1 -- 172.21.15.32:0/377849403 shutdown_connections 2024-08-31T07:32:30.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.754+0000 7f0f3c522740 1 -- 172.21.15.32:0/377849403 wait complete. 2024-08-31T07:32:30.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.755+0000 7f0f3c522740 1 -- start start 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.755+0000 7f0f3c522740 1 Processor -- start 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.757+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d02411b0 0x5625d0623730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.757+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 0x5625d0623cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.757+0000 7f0f3c522740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d026a070 0x5625d061e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.757+0000 7f0f3c522740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1000fc00 con 0x5625d026a070 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.757+0000 7f0f3c522740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1000f500 con 0x5625d02411b0 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.757+0000 7f0f3c522740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1000c6a0 con 0x5625d0249390 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.757+0000 7f0f3c522740 10 client.0 _ll_register_callbacks cb 0x7f0f1c011de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 0x5625d0623cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d026a070 0x5625d061e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d02411b0 0x5625d0623730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 0x5625d0623cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47006/0 (socket says 172.21.15.32:47006) 2024-08-31T07:32:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2e000640 1 -- 172.21.15.32:0/2923549930 learned_addr learned my addr 172.21.15.32:0/2923549930 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d02411b0 0x5625d0623730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46442/0 (socket says 172.21.15.32:46442) 2024-08-31T07:32:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2ea00640 1 -- 172.21.15.32:0/2923549930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 msgr2=0x5625d0623cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 0x5625d0623cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2ea00640 1 -- 172.21.15.32:0/2923549930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d026a070 msgr2=0x5625d061e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d026a070 0x5625d061e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2ea00640 1 -- 172.21.15.32:0/2923549930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625d061ebb0 con 0x5625d02411b0 2024-08-31T07:32:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.758+0000 7f0f2e000640 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 0x5625d0623cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.760+0000 7f0f2ea00640 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d02411b0 0x5625d0623730 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009fd0 tx=0x7f0f200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=845ce61a4c9c24d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.760+0000 7f0f18c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:32:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.762+0000 7f0f18c00640 1 -- 172.21.15.32:0/2923549930 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2002a840 con 0x5625d02411b0 2024-08-31T07:32:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.762+0000 7f0f18c00640 1 -- 172.21.15.32:0/2923549930 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f0f2002a9e0 con 0x5625d02411b0 2024-08-31T07:32:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.762+0000 7f0f3c522740 1 -- 172.21.15.32:0/2923549930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f28005560 con 0x5625d02411b0 2024-08-31T07:32:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.763+0000 7f0f18c00640 1 -- 172.21.15.32:0/2923549930 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f0f20036020 con 0x5625d02411b0 2024-08-31T07:32:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.763+0000 7f0f3c522740 10 client.29333 fetch_fsmap learned FSMap version 71 2024-08-31T07:32:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.763+0000 7f0f3c522740 1 -- 172.21.15.32:0/2923549930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5625d0698b60 con 0x5625d02411b0 2024-08-31T07:32:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.764+0000 7f0f18c00640 1 -- 172.21.15.32:0/2923549930 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f28005560 con 0x5625d02411b0 2024-08-31T07:32:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.765+0000 7f0f18c00640 1 -- 172.21.15.32:0/2923549930 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7f0f1c0bb050 con 0x5625d02411b0 2024-08-31T07:32:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.766+0000 7f0f3c522740 10 client.29333 fetch_fsmap finished waiting for FSMap version 71 2024-08-31T07:32:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.766+0000 7f0f3c522740 10 client.29333 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:32:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.766+0000 7f0f3c522740 20 client.29333 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.766+0000 7f0f3c522740 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5625d01fdda0 0x5625d069b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.766+0000 7f0f3c522740 4 client.29333 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:32:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.766+0000 7f0f3c522740 1 -- 172.21.15.32:0/2923549930 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625d05841a0 con 0x5625d01fdda0 2024-08-31T07:32:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.767+0000 7f0f2e000640 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5625d01fdda0 0x5625d069b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:30.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.768+0000 7f0f2e000640 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5625d01fdda0 0x5625d069b2a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff19b787665d3b79 server_cookie=7f244d2553fa4800 in_seq=0 out_seq=0 2024-08-31T07:32:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.769+0000 7f0f18c00640 10 client.29333 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:32:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.770+0000 7f0f18c00640 1 -- 172.21.15.32:0/2923549930 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625d05841a0 con 0x5625d01fdda0 2024-08-31T07:32:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.770+0000 7f0f18c00640 10 client.29333 handle_command_reply: tid=0 2024-08-31T07:32:30.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.770+0000 7f0f3c522740 1 client.29333 shutdown 2024-08-31T07:32:30.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.771+0000 7f0f3c522740 1 -- 172.21.15.32:0/2923549930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d02411b0 msgr2=0x5625d0623730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:30.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.771+0000 7f0f3c522740 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d02411b0 0x5625d0623730 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009fd0 tx=0x7f0f200046c0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.771+0000 7f0f3c522740 1 -- 172.21.15.32:0/2923549930 shutdown_connections 2024-08-31T07:32:30.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.772+0000 7f0f3c522740 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5625d026a070 0x5625d061e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.772+0000 7f0f3c522740 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625d0249390 0x5625d0623cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.772+0000 7f0f3c522740 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5625d01fdda0 0x5625d069b2a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.772+0000 7f0f3c522740 1 --2- 172.21.15.32:0/2923549930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5625d02411b0 0x5625d0623730 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.772+0000 7f0f3c522740 1 -- 172.21.15.32:0/2923549930 >> 172.21.15.32:0/2923549930 conn(0x5625d025e460 msgr2=0x5625d060d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:30.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.772+0000 7f0f18c00640 0 client.29333 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:32:30.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.772+0000 7f0f3c522740 1 -- 172.21.15.32:0/2923549930 shutdown_connections 2024-08-31T07:32:30.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.772+0000 7f0f3c522740 1 -- 172.21.15.32:0/2923549930 wait complete. 2024-08-31T07:32:30.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.773+0000 7f0f3c522740 20 client.29333 trim_cache size 0 max 16384 2024-08-31T07:32:30.775 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:32:30.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.774+0000 7f0f35c00640 1 -- 172.21.15.32:0/3919638504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0f140b4dd0 msgr2=0x7f0f140b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:30.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.774+0000 7f0f35c00640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0f140b4dd0 0x7f0f140b72e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x5625d060e530 tx=0x7f0f20005f90 comp rx=0 tx=0).stop 2024-08-31T07:32:30.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.774+0000 7f0f35c00640 1 -- 172.21.15.32:0/3919638504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30075ea0 msgr2=0x7f0f30198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:30.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.774+0000 7f0f35c00640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30075ea0 0x7f0f30198660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c013120 tx=0x7f0f1c011370 comp rx=0 tx=0).stop 2024-08-31T07:32:30.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.774+0000 7f0f35c00640 1 -- 172.21.15.32:0/3919638504 shutdown_connections 2024-08-31T07:32:30.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.774+0000 7f0f35c00640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f0f140b4dd0 0x7f0f140b72e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.774+0000 7f0f35c00640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f3010fe70 0x7f0f3019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.774+0000 7f0f35c00640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f30075ea0 0x7f0f30198660 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.774+0000 7f0f35c00640 1 --2- 172.21.15.32:0/3919638504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f30075550 0x7f0f301980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:30.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.774+0000 7f0f35c00640 1 -- 172.21.15.32:0/3919638504 >> 172.21.15.32:0/3919638504 conn(0x7f0f300ff380 msgr2=0x7f0f301085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:30.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.775+0000 7f0f35c00640 1 -- 172.21.15.32:0/3919638504 shutdown_connections 2024-08-31T07:32:30.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:30.775+0000 7f0f35c00640 1 -- 172.21.15.32:0/3919638504 wait complete. 2024-08-31T07:32:31.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 113774 13.70 MB/sec execute 251 sec latency 88.781 ms 2024-08-31T07:32:32.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 114363 13.75 MB/sec execute 252 sec latency 1083.601 ms 2024-08-31T07:32:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:32 smithi032 ceph-mon[17252]: pgmap v989: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 920 KiB/s wr, 81 op/s 2024-08-31T07:32:32.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:32 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:32:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:32 smithi073 ceph-mon[21995]: pgmap v989: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 920 KiB/s wr, 81 op/s 2024-08-31T07:32:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:32 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:32:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:32 smithi084 ceph-mon[22387]: pgmap v989: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 920 KiB/s wr, 81 op/s 2024-08-31T07:32:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:32 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:32:33.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 114895 13.81 MB/sec execute 253 sec latency 2083.652 ms 2024-08-31T07:32:33.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:32:33.671 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:34.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:34 smithi032 ceph-mon[17252]: pgmap v990: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-08-31T07:32:34.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 115493 13.86 MB/sec execute 254 sec latency 3083.711 ms 2024-08-31T07:32:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.521+0000 7fec44e00640 1 -- 172.21.15.32:0/2797867819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec401088a0 msgr2=0x7fec4006ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.521+0000 7fec44e00640 1 --2- 172.21.15.32:0/2797867819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec401088a0 0x7fec4006ac00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fec38009be0 tx=0x7fec3802a6c0 comp rx=0 tx=0).stop 2024-08-31T07:32:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.521+0000 7fec44e00640 1 -- 172.21.15.32:0/2797867819 shutdown_connections 2024-08-31T07:32:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.521+0000 7fec44e00640 1 --2- 172.21.15.32:0/2797867819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec401088a0 0x7fec4006ac00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.521+0000 7fec44e00640 1 --2- 172.21.15.32:0/2797867819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec40107f50 0x7fec40108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.521+0000 7fec44e00640 1 --2- 172.21.15.32:0/2797867819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40102610 0x7fec40102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.521+0000 7fec44e00640 1 -- 172.21.15.32:0/2797867819 >> 172.21.15.32:0/2797867819 conn(0x7fec400fe060 msgr2=0x7fec401004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.521+0000 7fec44e00640 1 -- 172.21.15.32:0/2797867819 shutdown_connections 2024-08-31T07:32:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.521+0000 7fec44e00640 1 -- 172.21.15.32:0/2797867819 wait complete. 2024-08-31T07:32:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.523+0000 7fec44e00640 1 Processor -- start 2024-08-31T07:32:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.523+0000 7fec44e00640 1 -- start start 2024-08-31T07:32:34.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.524+0000 7fec44e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40102610 0x7fec40112680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:34.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.524+0000 7fec44e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec40107f50 0x7fec40112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:34.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.524+0000 7fec44e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec401088a0 0x7fec40113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:34.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.524+0000 7fec44e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec401158c0 con 0x7fec40107f50 2024-08-31T07:32:34.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.524+0000 7fec44e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fec40115740 con 0x7fec401088a0 2024-08-31T07:32:34.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.524+0000 7fec44e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fec40115a40 con 0x7fec40102610 2024-08-31T07:32:34.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.524+0000 7fec3f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40102610 0x7fec40112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:34.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.524+0000 7fec3f400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40102610 0x7fec40112680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47014/0 (socket says 172.21.15.32:47014) 2024-08-31T07:32:34.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.524+0000 7fec3f400640 1 -- 172.21.15.32:0/2373166383 learned_addr learned my addr 172.21.15.32:0/2373166383 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:34.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.525+0000 7fec3ea00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec40107f50 0x7fec40112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.525+0000 7fec3fe00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec401088a0 0x7fec40113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.525+0000 7fec3f400640 1 -- 172.21.15.32:0/2373166383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec401088a0 msgr2=0x7fec40113100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.525+0000 7fec3f400640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec401088a0 0x7fec40113100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.525+0000 7fec3f400640 1 -- 172.21.15.32:0/2373166383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec40107f50 msgr2=0x7fec40112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.525+0000 7fec3f400640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec40107f50 0x7fec40112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.525+0000 7fec3f400640 1 -- 172.21.15.32:0/2373166383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec401ac780 con 0x7fec40102610 2024-08-31T07:32:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.526+0000 7fec3fe00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec401088a0 0x7fec40113100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:32:34.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.527+0000 7fec3f400640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40102610 0x7fec40112680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec3000e9f0 tx=0x7fec3000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:34.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.528+0000 7fec37e00640 1 -- 172.21.15.32:0/2373166383 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3000ccc0 con 0x7fec40102610 2024-08-31T07:32:34.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.528+0000 7fec37e00640 1 -- 172.21.15.32:0/2373166383 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec300054d0 con 0x7fec40102610 2024-08-31T07:32:34.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.528+0000 7fec44e00640 1 -- 172.21.15.32:0/2373166383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec401aca70 con 0x7fec40102610 2024-08-31T07:32:34.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.528+0000 7fec44e00640 1 -- 172.21.15.32:0/2373166383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec401acfb0 con 0x7fec40102610 2024-08-31T07:32:34.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.528+0000 7fec37e00640 1 -- 172.21.15.32:0/2373166383 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30010430 con 0x7fec40102610 2024-08-31T07:32:34.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.531+0000 7fec44e00640 1 -- 172.21.15.32:0/2373166383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec40108360 con 0x7fec40102610 2024-08-31T07:32:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.535+0000 7fec37e00640 1 -- 172.21.15.32:0/2373166383 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fec300046c0 con 0x7fec40102610 2024-08-31T07:32:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.537+0000 7fec37e00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fec240b4df0 0x7fec240b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.537+0000 7fec37e00640 1 -- 172.21.15.32:0/2373166383 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7fec300e5720 con 0x7fec40102610 2024-08-31T07:32:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.538+0000 7fec3ea00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fec240b4df0 0x7fec240b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:34.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.538+0000 7fec3ea00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fec240b4df0 0x7fec240b7300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fec40102df0 tx=0x7fec2c00b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:34.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:34.540+0000 7fec37e00640 1 -- 172.21.15.32:0/2373166383 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fec300ac560 con 0x7fec40102610 2024-08-31T07:32:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:34 smithi073 ceph-mon[21995]: pgmap v990: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-08-31T07:32:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:34 smithi084 ceph-mon[22387]: pgmap v990: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-08-31T07:32:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.003+0000 7fec44e00640 1 -- 172.21.15.32:0/2373166383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec401ad400 con 0x7fec40102610 2024-08-31T07:32:35.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.004+0000 7fec37e00640 1 -- 172.21.15.32:0/2373166383 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+11972 (secure 0 0 0) 0x7fec300abe60 con 0x7fec40102610 2024-08-31T07:32:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:32:35.007 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"btime":"2024-08-31T07:32:31:268746+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":72,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:32:30.973488+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:32:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 72 2024-08-31T07:32:35.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.008+0000 7fec44e00640 1 -- 172.21.15.32:0/2373166383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fec240b4df0 msgr2=0x7fec240b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.008+0000 7fec44e00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fec240b4df0 0x7fec240b7300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fec40102df0 tx=0x7fec2c00b410 comp rx=0 tx=0).stop 2024-08-31T07:32:35.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.008+0000 7fec44e00640 1 -- 172.21.15.32:0/2373166383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40102610 msgr2=0x7fec40112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.008+0000 7fec44e00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40102610 0x7fec40112680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec3000e9f0 tx=0x7fec3000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.009+0000 7fec44e00640 1 -- 172.21.15.32:0/2373166383 shutdown_connections 2024-08-31T07:32:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.009+0000 7fec44e00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fec240b4df0 0x7fec240b7300 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.009+0000 7fec44e00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec401088a0 0x7fec40113100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.009+0000 7fec44e00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec40107f50 0x7fec40112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.009+0000 7fec44e00640 1 --2- 172.21.15.32:0/2373166383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40102610 0x7fec40112680 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.009+0000 7fec44e00640 1 -- 172.21.15.32:0/2373166383 >> 172.21.15.32:0/2373166383 conn(0x7fec400fe060 msgr2=0x7fec40103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.009+0000 7fec44e00640 1 -- 172.21.15.32:0/2373166383 shutdown_connections 2024-08-31T07:32:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.010+0000 7fec44e00640 1 -- 172.21.15.32:0/2373166383 wait complete. 2024-08-31T07:32:35.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 116201 13.97 MB/sec execute 255 sec latency 4083.838 ms 2024-08-31T07:32:35.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.434+0000 7f13f3a00640 1 -- 172.21.15.32:0/2876567746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13ec075ea0 msgr2=0x7f13ec10f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.434+0000 7f13f3a00640 1 --2- 172.21.15.32:0/2876567746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13ec075ea0 0x7f13ec10f940 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009860 tx=0x7f13d802a360 comp rx=0 tx=0).stop 2024-08-31T07:32:35.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.434+0000 7f13f3a00640 1 -- 172.21.15.32:0/2876567746 shutdown_connections 2024-08-31T07:32:35.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.434+0000 7f13f3a00640 1 --2- 172.21.15.32:0/2876567746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13ec10fe80 0x7f13ec1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.434+0000 7f13f3a00640 1 --2- 172.21.15.32:0/2876567746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13ec075ea0 0x7f13ec10f940 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.434+0000 7f13f3a00640 1 --2- 172.21.15.32:0/2876567746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13ec075550 0x7f13ec075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.434+0000 7f13f3a00640 1 -- 172.21.15.32:0/2876567746 >> 172.21.15.32:0/2876567746 conn(0x7f13ec0ff330 msgr2=0x7f13ec101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:35.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.435+0000 7f13f3a00640 1 -- 172.21.15.32:0/2876567746 shutdown_connections 2024-08-31T07:32:35.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.435+0000 7f13f3a00640 1 -- 172.21.15.32:0/2876567746 wait complete. 2024-08-31T07:32:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.437+0000 7f13f3a00640 1 Processor -- start 2024-08-31T07:32:35.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.438+0000 7f13f3a00640 1 -- start start 2024-08-31T07:32:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.438+0000 7f13f3a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13ec075550 0x7f13ec19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.438+0000 7f13f3a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13ec075ea0 0x7f13ec19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13f3a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13ec10fe80 0x7f13ec1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13f3a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec115a40 con 0x7f13ec10fe80 2024-08-31T07:32:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13f3a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1158c0 con 0x7f13ec075ea0 2024-08-31T07:32:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13f3a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec115bc0 con 0x7f13ec075550 2024-08-31T07:32:35.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13f1000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13ec075550 0x7f13ec19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13f1000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13ec075550 0x7f13ec19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47038/0 (socket says 172.21.15.32:47038) 2024-08-31T07:32:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13ebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13ec075ea0 0x7f13ec19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13f1000640 1 -- 172.21.15.32:0/2621029487 learned_addr learned my addr 172.21.15.32:0/2621029487 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13f1a00640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13ec10fe80 0x7f13ec1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13ebe00640 1 -- 172.21.15.32:0/2621029487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13ec075550 msgr2=0x7f13ec19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13ebe00640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13ec075550 0x7f13ec19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13ebe00640 1 -- 172.21.15.32:0/2621029487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13ec10fe80 msgr2=0x7f13ec1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13ebe00640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13ec10fe80 0x7f13ec1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.439+0000 7f13ebe00640 1 -- 172.21.15.32:0/2621029487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec1a2420 con 0x7f13ec075ea0 2024-08-31T07:32:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.441+0000 7f13ebe00640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13ec075ea0 0x7f13ec19ca50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009780 tx=0x7f13d8032cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.441+0000 7f13e9600640 1 -- 172.21.15.32:0/2621029487 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8040020 con 0x7f13ec075ea0 2024-08-31T07:32:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.441+0000 7f13f3a00640 1 -- 172.21.15.32:0/2621029487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ec1a26b0 con 0x7f13ec075ea0 2024-08-31T07:32:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.441+0000 7f13e9600640 1 -- 172.21.15.32:0/2621029487 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13d8038040 con 0x7f13ec075ea0 2024-08-31T07:32:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.441+0000 7f13e9600640 1 -- 172.21.15.32:0/2621029487 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8037070 con 0x7f13ec075ea0 2024-08-31T07:32:35.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.441+0000 7f13f3a00640 1 -- 172.21.15.32:0/2621029487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ec1a8e70 con 0x7f13ec075ea0 2024-08-31T07:32:35.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.445+0000 7f13fa61f740 1 Processor -- start 2024-08-31T07:32:35.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.445+0000 7f13fa61f740 1 -- start start 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.446+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 0x5621226221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.446+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56212224b1b0 0x5621226226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.446+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562122253390 0x562122622c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.446+0000 7f13fa61f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5621225907b0 con 0x562122253390 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.446+0000 7f13fa61f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56212257fd40 con 0x562122274070 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.446+0000 7f13fa61f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562122583cf0 con 0x56212224b1b0 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.447+0000 7f13f1a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 0x5621226221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.447+0000 7f13f1a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 0x5621226221a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46482/0 (socket says 172.21.15.32:46482) 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.447+0000 7f13f1a00640 1 -- 172.21.15.32:0/500621254 learned_addr learned my addr 172.21.15.32:0/500621254 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.447+0000 7f13f1000640 1 --2- 172.21.15.32:0/500621254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56212224b1b0 0x5621226226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.447+0000 7f13e9600640 1 -- 172.21.15.32:0/2621029487 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f13d8043020 con 0x7f13ec075ea0 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.447+0000 7f13f1000640 1 -- 172.21.15.32:0/500621254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 msgr2=0x5621226221a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:35.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.447+0000 7f13f1000640 1 --2- 172.21.15.32:0/500621254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 0x5621226221a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.447+0000 7f13f1000640 1 -- 172.21.15.32:0/500621254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562122253390 msgr2=0x562122622c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:32:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.447+0000 7f13f1000640 1 --2- 172.21.15.32:0/500621254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562122253390 0x562122622c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.447+0000 7f13f1000640 1 -- 172.21.15.32:0/500621254 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621222df650 con 0x56212224b1b0 2024-08-31T07:32:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.448+0000 7f13f1000640 1 --2- 172.21.15.32:0/500621254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56212224b1b0 0x5621226226e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009c80 tx=0x7f13e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=845b71e58db9b026 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.449+0000 7f13de000640 1 -- 172.21.15.32:0/500621254 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0004230 con 0x56212224b1b0 2024-08-31T07:32:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.449+0000 7f13de000640 1 -- 172.21.15.32:0/500621254 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e00043d0 con 0x56212224b1b0 2024-08-31T07:32:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.449+0000 7f13de000640 1 -- 172.21.15.32:0/500621254 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0004d80 con 0x56212224b1b0 2024-08-31T07:32:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.450+0000 7f13fa61f740 1 -- 172.21.15.32:0/500621254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56212224b1b0 msgr2=0x5621226226e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.450+0000 7f13fa61f740 1 --2- 172.21.15.32:0/500621254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56212224b1b0 0x5621226226e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009c80 tx=0x7f13e0011810 comp rx=0 tx=0).stop 2024-08-31T07:32:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.450+0000 7f13fa61f740 1 -- 172.21.15.32:0/500621254 shutdown_connections 2024-08-31T07:32:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.450+0000 7f13fa61f740 1 --2- 172.21.15.32:0/500621254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562122253390 0x562122622c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.450+0000 7f13fa61f740 1 --2- 172.21.15.32:0/500621254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56212224b1b0 0x5621226226e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.450+0000 7f13fa61f740 1 --2- 172.21.15.32:0/500621254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 0x5621226221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.450+0000 7f13fa61f740 1 -- 172.21.15.32:0/500621254 >> 172.21.15.32:0/500621254 conn(0x562122268460 msgr2=0x562122620090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.451+0000 7f13e9600640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f13d40b4d50 0x7f13d40b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.451+0000 7f13fa61f740 1 -- 172.21.15.32:0/500621254 shutdown_connections 2024-08-31T07:32:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.451+0000 7f13e9600640 1 -- 172.21.15.32:0/2621029487 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f13d8105420 con 0x7f13ec075ea0 2024-08-31T07:32:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.452+0000 7f13fa61f740 1 -- 172.21.15.32:0/500621254 wait complete. 2024-08-31T07:32:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.452+0000 7f13fa61f740 1 -- start start 2024-08-31T07:32:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.453+0000 7f13f1000640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f13d40b4d50 0x7f13d40b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.453+0000 7f13fa61f740 1 Processor -- start 2024-08-31T07:32:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 0x56212262ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122253390 0x56212262e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122639c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13fa61f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5621223e2700 con 0x56212224b1b0 2024-08-31T07:32:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13fa61f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562122506cf0 con 0x562122253390 2024-08-31T07:32:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13fa61f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562122630980 con 0x562122274070 2024-08-31T07:32:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13fa61f740 10 client.0 _ll_register_callbacks cb 0x56212252c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13f1a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122639c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13f1a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122639c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47052/0 (socket says 172.21.15.32:47052) 2024-08-31T07:32:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13f1a00640 1 -- 172.21.15.32:0/222226725 learned_addr learned my addr 172.21.15.32:0/222226725 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:35.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13f1000640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f13d40b4d50 0x7f13d40b7260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009620 tx=0x7f13e00048e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:35.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.454+0000 7f13f1000640 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 0x56212262ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.455+0000 7f13f1a00640 1 -- 172.21.15.32:0/222226725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122253390 msgr2=0x56212262e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:35.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.455+0000 7f13f1a00640 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122253390 0x56212262e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.455+0000 7f13f1a00640 1 -- 172.21.15.32:0/222226725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 msgr2=0x56212262ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.455+0000 7f13f1a00640 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 0x56212262ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.455+0000 7f13f1a00640 1 -- 172.21.15.32:0/222226725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621225478b0 con 0x562122274070 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.455+0000 7f13f1000640 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 0x56212262ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.456+0000 7f13f1a00640 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122639c70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f13e400ff40 tx=0x7f13e400d7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1b7593c71ad2898 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.456+0000 7f13d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.456+0000 7f13d2a00640 1 -- 172.21.15.32:0/222226725 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e4004b20 con 0x562122274070 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.456+0000 7f13d2a00640 1 -- 172.21.15.32:0/222226725 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f13e4004cc0 con 0x562122274070 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.456+0000 7f13fa61f740 1 -- 172.21.15.32:0/222226725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e00043d0 con 0x562122274070 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.456+0000 7f13d2a00640 1 -- 172.21.15.32:0/222226725 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f13e4005020 con 0x562122274070 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.456+0000 7f13fa61f740 10 client.29335 fetch_fsmap learned FSMap version 72 2024-08-31T07:32:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.456+0000 7f13fa61f740 1 -- 172.21.15.32:0/222226725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x562122619970 con 0x562122274070 2024-08-31T07:32:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.457+0000 7f13d2a00640 1 -- 172.21.15.32:0/222226725 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e00043d0 con 0x562122274070 2024-08-31T07:32:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.457+0000 7f13d2a00640 1 -- 172.21.15.32:0/222226725 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 72) ==== 8394+0+0 (secure 0 0 0) 0x7f13e400b040 con 0x562122274070 2024-08-31T07:32:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.457+0000 7f13fa61f740 10 client.29335 fetch_fsmap finished waiting for FSMap version 72 2024-08-31T07:32:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.457+0000 7f13fa61f740 10 client.29335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:32:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.457+0000 7f13fa61f740 20 client.29335 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.457+0000 7f13fa61f740 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5621222329f0 0x56212261cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.457+0000 7f13fa61f740 4 client.29335 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:32:35.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.457+0000 7f13fa61f740 1 -- 172.21.15.32:0/222226725 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562122592700 con 0x5621222329f0 2024-08-31T07:32:35.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.458+0000 7f13f1000640 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5621222329f0 0x56212261cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.459+0000 7f13d2a00640 10 client.29335 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:35.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.459+0000 7f13f1000640 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5621222329f0 0x56212261cfe0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39d9d61d31c0de2 server_cookie=7751f12a6c1a386c in_seq=0 out_seq=0 2024-08-31T07:32:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13d2a00640 1 -- 172.21.15.32:0/222226725 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562122592700 con 0x5621222329f0 2024-08-31T07:32:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13d2a00640 10 client.29335 handle_command_reply: tid=0 2024-08-31T07:32:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13fa61f740 1 client.29335 shutdown 2024-08-31T07:32:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13fa61f740 1 -- 172.21.15.32:0/222226725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 msgr2=0x562122639c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13fa61f740 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122639c70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f13e400ff40 tx=0x7f13e400d7b0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13fa61f740 1 -- 172.21.15.32:0/222226725 shutdown_connections 2024-08-31T07:32:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13fa61f740 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5621222329f0 0x56212261cfe0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13fa61f740 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122639c70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13fa61f740 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122253390 0x56212262e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13fa61f740 1 --2- 172.21.15.32:0/222226725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 0x56212262ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13fa61f740 1 -- 172.21.15.32:0/222226725 >> 172.21.15.32:0/222226725 conn(0x562122268460 msgr2=0x562122625380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.462+0000 7f13d2a00640 0 client.29335 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.463+0000 7f13fa61f740 1 -- 172.21.15.32:0/222226725 shutdown_connections 2024-08-31T07:32:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.463+0000 7f13fa61f740 1 -- 172.21.15.32:0/222226725 wait complete. 2024-08-31T07:32:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.463+0000 7f13fa61f740 20 client.29335 trim_cache size 0 max 16384 2024-08-31T07:32:35.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.485+0000 7f13fa61f740 1 Processor -- start 2024-08-31T07:32:35.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.485+0000 7f13fa61f740 1 -- start start 2024-08-31T07:32:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.486+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 0x562122627420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.486+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122253390 0x562122627960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.486+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 0x56212262c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.486+0000 7f13fa61f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1ad370 con 0x56212224b1b0 2024-08-31T07:32:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.486+0000 7f13fa61f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1af000 con 0x562122274070 2024-08-31T07:32:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.486+0000 7f13fa61f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1ada40 con 0x562122253390 2024-08-31T07:32:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.486+0000 7f13ebe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122253390 0x562122627960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.486+0000 7f13ebe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122253390 0x562122627960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:47060/0 (socket says 172.21.15.32:47060) 2024-08-31T07:32:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.486+0000 7f13ebe00640 1 -- 172.21.15.32:0/1262546062 learned_addr learned my addr 172.21.15.32:0/1262546062 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.486+0000 7f13f1a00640 1 --2- 172.21.15.32:0/1262546062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 0x56212262c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.487+0000 7f13f1000640 1 --2- 172.21.15.32:0/1262546062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 0x562122627420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.487+0000 7f13f1a00640 1 -- 172.21.15.32:0/1262546062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122253390 msgr2=0x562122627960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.487+0000 7f13f1a00640 1 --2- 172.21.15.32:0/1262546062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122253390 0x562122627960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.487+0000 7f13f1a00640 1 -- 172.21.15.32:0/1262546062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 msgr2=0x562122627420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.487+0000 7f13f1a00640 1 --2- 172.21.15.32:0/1262546062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 0x562122627420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.487+0000 7f13f1a00640 1 -- 172.21.15.32:0/1262546062 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562122639fb0 con 0x562122274070 2024-08-31T07:32:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.488+0000 7f13f1a00640 1 --2- 172.21.15.32:0/1262546062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 0x56212262c460 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f13e4002230 tx=0x7f13e4000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56060da376bbfa99 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.488+0000 7f13dd600640 1 -- 172.21.15.32:0/1262546062 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e4005020 con 0x562122274070 2024-08-31T07:32:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.488+0000 7f13dd600640 1 -- 172.21.15.32:0/1262546062 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e4022940 con 0x562122274070 2024-08-31T07:32:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.488+0000 7f13dd600640 1 -- 172.21.15.32:0/1262546062 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e40059b0 con 0x562122274070 2024-08-31T07:32:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.489+0000 7f13fa61f740 1 -- 172.21.15.32:0/1262546062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 msgr2=0x56212262c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.489+0000 7f13fa61f740 1 --2- 172.21.15.32:0/1262546062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 0x56212262c460 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f13e4002230 tx=0x7f13e4000f40 comp rx=0 tx=0).stop 2024-08-31T07:32:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.490+0000 7f13fa61f740 1 -- 172.21.15.32:0/1262546062 shutdown_connections 2024-08-31T07:32:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.490+0000 7f13fa61f740 1 --2- 172.21.15.32:0/1262546062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562122274070 0x56212262c460 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.490+0000 7f13fa61f740 1 --2- 172.21.15.32:0/1262546062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122253390 0x562122627960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.490+0000 7f13fa61f740 1 --2- 172.21.15.32:0/1262546062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56212224b1b0 0x562122627420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.490+0000 7f13fa61f740 1 -- 172.21.15.32:0/1262546062 >> 172.21.15.32:0/1262546062 conn(0x562122268460 msgr2=0x562122628d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.490+0000 7f13fa61f740 1 -- 172.21.15.32:0/1262546062 shutdown_connections 2024-08-31T07:32:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.491+0000 7f13fa61f740 1 -- 172.21.15.32:0/1262546062 wait complete. 2024-08-31T07:32:35.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.492+0000 7f13fa61f740 1 -- start start 2024-08-31T07:32:35.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.492+0000 7f13fa61f740 1 Processor -- start 2024-08-31T07:32:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.493+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56212224b1b0 0x562122636560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.493+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562122253390 0x562122650020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.493+0000 7f13fa61f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122650730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.493+0000 7f13fa61f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1ae8d0 con 0x562122253390 2024-08-31T07:32:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.493+0000 7f13fa61f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1ae1d0 con 0x56212224b1b0 2024-08-31T07:32:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.493+0000 7f13fa61f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1ab2f0 con 0x562122274070 2024-08-31T07:32:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.493+0000 7f13fa61f740 10 client.0 _ll_register_callbacks cb 0x7f13e400fb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.494+0000 7f13f1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56212224b1b0 0x562122636560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.494+0000 7f13f1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56212224b1b0 0x562122636560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:46512/0 (socket says 172.21.15.32:46512) 2024-08-31T07:32:35.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.494+0000 7f13f1000640 1 -- 172.21.15.32:0/3643773870 learned_addr learned my addr 172.21.15.32:0/3643773870 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:35.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.495+0000 7f13f1000640 1 -- 172.21.15.32:0/3643773870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 msgr2=0x562122650730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:32:35.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.495+0000 7f13f1a00640 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122650730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.495+0000 7f13f1000640 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122650730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.495+0000 7f13f1000640 1 -- 172.21.15.32:0/3643773870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562122253390 msgr2=0x562122650020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.495+0000 7f13ebe00640 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562122253390 0x562122650020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.495+0000 7f13f1000640 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562122253390 0x562122650020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.495+0000 7f13f1000640 1 -- 172.21.15.32:0/3643773870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621226351a0 con 0x56212224b1b0 2024-08-31T07:32:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.495+0000 7f13f1a00640 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122650730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:32:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.495+0000 7f13ebe00640 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562122253390 0x562122650020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:32:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.496+0000 7f13f1000640 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56212224b1b0 0x562122636560 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f13e00048b0 tx=0x7f13e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0c4ec333dee683 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.496+0000 7f13d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:32:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.496+0000 7f13d2a00640 1 -- 172.21.15.32:0/3643773870 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e002c040 con 0x56212224b1b0 2024-08-31T07:32:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.496+0000 7f13d2a00640 1 -- 172.21.15.32:0/3643773870 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f13e0025c70 con 0x56212224b1b0 2024-08-31T07:32:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.497+0000 7f13fa61f740 1 -- 172.21.15.32:0/3643773870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e4022940 con 0x56212224b1b0 2024-08-31T07:32:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.497+0000 7f13d2a00640 1 -- 172.21.15.32:0/3643773870 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f13e001e050 con 0x56212224b1b0 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.497+0000 7f13fa61f740 10 client.29351 fetch_fsmap learned FSMap version 72 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.497+0000 7f13fa61f740 1 -- 172.21.15.32:0/3643773870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x5621226c4540 con 0x56212224b1b0 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.498+0000 7f13d2a00640 1 -- 172.21.15.32:0/3643773870 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e4022940 con 0x56212224b1b0 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.498+0000 7f13d2a00640 1 -- 172.21.15.32:0/3643773870 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 72) ==== 8394+0+0 (secure 0 0 0) 0x7f13e0018070 con 0x56212224b1b0 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.498+0000 7f13fa61f740 10 client.29351 fetch_fsmap finished waiting for FSMap version 72 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.498+0000 7f13fa61f740 10 client.29351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.498+0000 7f13fa61f740 20 client.29351 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.498+0000 7f13fa61f740 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562122207da0 0x5621226c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.499+0000 7f13fa61f740 4 client.29351 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.499+0000 7f13fa61f740 1 -- 172.21.15.32:0/3643773870 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56212258e1a0 con 0x562122207da0 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.499+0000 7f13ebe00640 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562122207da0 0x5621226c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.500+0000 7f13ebe00640 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562122207da0 0x5621226c6bf0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c11eaad285bbf7 server_cookie=93d35849e147c691 in_seq=0 out_seq=0 2024-08-31T07:32:35.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.500+0000 7f13d2a00640 10 client.29351 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:35.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.503+0000 7f13d2a00640 1 -- 172.21.15.32:0/3643773870 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56212258e1a0 con 0x562122207da0 2024-08-31T07:32:35.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.503+0000 7f13d2a00640 10 client.29351 handle_command_reply: tid=0 2024-08-31T07:32:35.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.503+0000 7f13fa61f740 1 client.29351 shutdown 2024-08-31T07:32:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.504+0000 7f13fa61f740 1 -- 172.21.15.32:0/3643773870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56212224b1b0 msgr2=0x562122636560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.504+0000 7f13fa61f740 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56212224b1b0 0x562122636560 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f13e00048b0 tx=0x7f13e0011810 comp rx=0 tx=0).stop 2024-08-31T07:32:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.504+0000 7f13fa61f740 1 -- 172.21.15.32:0/3643773870 shutdown_connections 2024-08-31T07:32:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.504+0000 7f13fa61f740 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562122207da0 0x5621226c6bf0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.504+0000 7f13fa61f740 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562122274070 0x562122650730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.504+0000 7f13fa61f740 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562122253390 0x562122650020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.504+0000 7f13fa61f740 1 --2- 172.21.15.32:0/3643773870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56212224b1b0 0x562122636560 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.504+0000 7f13fa61f740 1 -- 172.21.15.32:0/3643773870 >> 172.21.15.32:0/3643773870 conn(0x562122268460 msgr2=0x56212262c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.504+0000 7f13d2a00640 0 client.29351 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.505+0000 7f13fa61f740 1 -- 172.21.15.32:0/3643773870 shutdown_connections 2024-08-31T07:32:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.505+0000 7f13fa61f740 1 -- 172.21.15.32:0/3643773870 wait complete. 2024-08-31T07:32:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.505+0000 7f13fa61f740 20 client.29351 trim_cache size 0 max 16384 2024-08-31T07:32:35.508 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:32:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.506+0000 7f13f3a00640 1 -- 172.21.15.32:0/2621029487 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f13d40b4d50 msgr2=0x7f13d40b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.506+0000 7f13f3a00640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f13d40b4d50 0x7f13d40b7260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009620 tx=0x7f13e00048e0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.506+0000 7f13f3a00640 1 -- 172.21.15.32:0/2621029487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13ec075ea0 msgr2=0x7f13ec19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:35.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.506+0000 7f13f3a00640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13ec075ea0 0x7f13ec19ca50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009780 tx=0x7f13d8032cd0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.507+0000 7f13f3a00640 1 -- 172.21.15.32:0/2621029487 shutdown_connections 2024-08-31T07:32:35.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.507+0000 7f13f3a00640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f13d40b4d50 0x7f13d40b7260 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.507+0000 7f13f3a00640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13ec10fe80 0x7f13ec1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.507+0000 7f13f3a00640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13ec075ea0 0x7f13ec19ca50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.507+0000 7f13f3a00640 1 --2- 172.21.15.32:0/2621029487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13ec075550 0x7f13ec19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:35.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.507+0000 7f13f3a00640 1 -- 172.21.15.32:0/2621029487 >> 172.21.15.32:0/2621029487 conn(0x7f13ec0ff330 msgr2=0x7f13ec112150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.507+0000 7f13f3a00640 1 -- 172.21.15.32:0/2621029487 shutdown_connections 2024-08-31T07:32:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:35.507+0000 7f13f3a00640 1 -- 172.21.15.32:0/2621029487 wait complete. 2024-08-31T07:32:35.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:35 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2373166383' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:32:35.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:35 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2373166383' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:32:35.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:35 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2373166383' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:32:36.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 116946 14.09 MB/sec execute 256 sec latency 5083.898 ms 2024-08-31T07:32:36.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:36 smithi032 ceph-mon[17252]: pgmap v991: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.2 MiB/s wr, 342 op/s 2024-08-31T07:32:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:36 smithi073 ceph-mon[21995]: pgmap v991: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.2 MiB/s wr, 342 op/s 2024-08-31T07:32:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:36 smithi084 ceph-mon[22387]: pgmap v991: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.2 MiB/s wr, 342 op/s 2024-08-31T07:32:36.952 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 152.8 --await-for 45.0' 2024-08-31T07:32:36.953 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 152.8 --await-for 45.0 2024-08-31T07:32:37.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 117503 14.13 MB/sec execute 257 sec latency 6083.953 ms 2024-08-31T07:32:37.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:37 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:32:37.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:37 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:32:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:37 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:32:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:38.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:38 smithi032 ceph-mon[17252]: pgmap v992: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.8 MiB/s wr, 385 op/s 2024-08-31T07:32:38.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 118334 14.25 MB/sec execute 258 sec latency 7084.010 ms 2024-08-31T07:32:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:38 smithi073 ceph-mon[21995]: pgmap v992: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.8 MiB/s wr, 385 op/s 2024-08-31T07:32:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:38 smithi084 ceph-mon[22387]: pgmap v992: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.8 MiB/s wr, 385 op/s 2024-08-31T07:32:39.071 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:32:39.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:32:39.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119111 14.39 MB/sec execute 259 sec latency 8084.068 ms 2024-08-31T07:32:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.155+0000 7fc8bb600640 1 -- 172.21.15.32:0/2464387398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b4102560 msgr2=0x7fc8b4102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.155+0000 7fc8bb600640 1 --2- 172.21.15.32:0/2464387398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b4102560 0x7fc8b4102970 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009930 tx=0x7fc8a802a440 comp rx=0 tx=0).stop 2024-08-31T07:32:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.156+0000 7fc8bb600640 1 -- 172.21.15.32:0/2464387398 shutdown_connections 2024-08-31T07:32:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.156+0000 7fc8bb600640 1 --2- 172.21.15.32:0/2464387398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b41079a0 0x7fc8b4107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.156+0000 7fc8bb600640 1 --2- 172.21.15.32:0/2464387398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b4102eb0 0x7fc8b41073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.156+0000 7fc8bb600640 1 --2- 172.21.15.32:0/2464387398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b4102560 0x7fc8b4102970 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.156+0000 7fc8bb600640 1 -- 172.21.15.32:0/2464387398 >> 172.21.15.32:0/2464387398 conn(0x7fc8b4079500 msgr2=0x7fc8b4079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.156+0000 7fc8bb600640 1 -- 172.21.15.32:0/2464387398 shutdown_connections 2024-08-31T07:32:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.156+0000 7fc8bb600640 1 -- 172.21.15.32:0/2464387398 wait complete. 2024-08-31T07:32:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.158+0000 7fc8bb600640 1 Processor -- start 2024-08-31T07:32:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.158+0000 7fc8bb600640 1 -- start start 2024-08-31T07:32:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.158+0000 7fc8bb600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4102560 0x7fc8b4116ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.158+0000 7fc8bb600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b4102eb0 0x7fc8b410fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.158+0000 7fc8bb600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b41079a0 0x7fc8b4110380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.158+0000 7fc8bb600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b4119eb0 con 0x7fc8b4102560 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.158+0000 7fc8bb600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b4119d30 con 0x7fc8b4102eb0 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.158+0000 7fc8bb600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b411a030 con 0x7fc8b41079a0 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.159+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b4102eb0 0x7fc8b410fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.159+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b4102eb0 0x7fc8b410fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38548/0 (socket says 172.21.15.32:38548) 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.159+0000 7fc8b3e00640 1 -- 172.21.15.32:0/3462503033 learned_addr learned my addr 172.21.15.32:0/3462503033 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.159+0000 7fc8b9600640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b41079a0 0x7fc8b4110380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.159+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4102560 0x7fc8b4116ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.159+0000 7fc8b3e00640 1 -- 172.21.15.32:0/3462503033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b41079a0 msgr2=0x7fc8b4110380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.159+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b41079a0 0x7fc8b4110380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.159+0000 7fc8b3e00640 1 -- 172.21.15.32:0/3462503033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4102560 msgr2=0x7fc8b4116ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.159+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4102560 0x7fc8b4116ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.159+0000 7fc8b3e00640 1 -- 172.21.15.32:0/3462503033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b4110ba0 con 0x7fc8b4102eb0 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.160+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b4102eb0 0x7fc8b410fd90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e9c0 tx=0x7fc8a000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:40.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.160+0000 7fc8b1600640 1 -- 172.21.15.32:0/3462503033 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a000ccc0 con 0x7fc8b4102eb0 2024-08-31T07:32:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.160+0000 7fc8b1600640 1 -- 172.21.15.32:0/3462503033 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a00054d0 con 0x7fc8b4102eb0 2024-08-31T07:32:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.160+0000 7fc8b1600640 1 -- 172.21.15.32:0/3462503033 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0010430 con 0x7fc8b4102eb0 2024-08-31T07:32:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.160+0000 7fc8bb600640 1 -- 172.21.15.32:0/3462503033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b4115850 con 0x7fc8b4102eb0 2024-08-31T07:32:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.160+0000 7fc8bb600640 1 -- 172.21.15.32:0/3462503033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b4115d10 con 0x7fc8b4102eb0 2024-08-31T07:32:40.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.164+0000 7fc8b1600640 1 -- 172.21.15.32:0/3462503033 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc8a0005020 con 0x7fc8b4102eb0 2024-08-31T07:32:40.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.166+0000 7fc8b1600640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc89c0b4d20 0x7fc89c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.166+0000 7fc8c241a740 1 Processor -- start 2024-08-31T07:32:40.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.167+0000 7fc8b1600640 1 -- 172.21.15.32:0/3462503033 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7fc8a00e5980 con 0x7fc8b4102eb0 2024-08-31T07:32:40.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.167+0000 7fc8c241a740 1 -- start start 2024-08-31T07:32:40.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.167+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc89c0b4d20 0x7fc89c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.167+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeaff110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.167+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae72f980 0x55a0aeaff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.167+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeaffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.167+0000 7fc8c241a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a0aea74530 con 0x55a0ae72f980 2024-08-31T07:32:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.167+0000 7fc8c241a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a0aea63ac0 con 0x55a0ae737b60 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.167+0000 7fc8c241a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0aea67a70 con 0x55a0ae758840 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b9600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeaffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b8c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeaff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b9600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeaffb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38554/0 (socket says 172.21.15.32:38554) 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b9600640 1 -- 172.21.15.32:0/817641658 learned_addr learned my addr 172.21.15.32:0/817641658 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc89c0b4d20 0x7fc89c0b7230 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009900 tx=0x7fc8a8000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b8c00640 1 -- 172.21.15.32:0/817641658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 msgr2=0x55a0aeaffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/817641658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeaffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b8c00640 1 -- 172.21.15.32:0/817641658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae72f980 msgr2=0x55a0aeaff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/817641658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae72f980 0x55a0aeaff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b8c00640 1 -- 172.21.15.32:0/817641658 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0aea16880 con 0x55a0ae758840 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.168+0000 7fc8b9600640 1 --2- 172.21.15.32:0/817641658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeaffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.169+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/817641658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeaff110 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009230 tx=0x7fc8a8044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=243433662d64b5eb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.169+0000 7fc8a5c00640 1 -- 172.21.15.32:0/817641658 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8035050 con 0x55a0ae758840 2024-08-31T07:32:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.169+0000 7fc8a5c00640 1 -- 172.21.15.32:0/817641658 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a8004d70 con 0x55a0ae758840 2024-08-31T07:32:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.169+0000 7fc8a5c00640 1 -- 172.21.15.32:0/817641658 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a80324b0 con 0x55a0ae758840 2024-08-31T07:32:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.170+0000 7fc8c241a740 1 -- 172.21.15.32:0/817641658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 msgr2=0x55a0aeaff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.170+0000 7fc8c241a740 1 --2- 172.21.15.32:0/817641658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeaff110 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009230 tx=0x7fc8a8044000 comp rx=0 tx=0).stop 2024-08-31T07:32:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.170+0000 7fc8c241a740 1 -- 172.21.15.32:0/817641658 shutdown_connections 2024-08-31T07:32:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.170+0000 7fc8c241a740 1 --2- 172.21.15.32:0/817641658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeaffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.170+0000 7fc8c241a740 1 --2- 172.21.15.32:0/817641658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae72f980 0x55a0aeaff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.170+0000 7fc8c241a740 1 --2- 172.21.15.32:0/817641658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeaff110 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.170+0000 7fc8c241a740 1 -- 172.21.15.32:0/817641658 >> 172.21.15.32:0/817641658 conn(0x55a0ae74c990 msgr2=0x55a0aeafd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.170+0000 7fc8c241a740 1 -- 172.21.15.32:0/817641658 shutdown_connections 2024-08-31T07:32:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.171+0000 7fc8c241a740 1 -- 172.21.15.32:0/817641658 wait complete. 2024-08-31T07:32:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.171+0000 7fc8c241a740 1 -- start start 2024-08-31T07:32:40.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.172+0000 7fc8c241a740 1 Processor -- start 2024-08-31T07:32:40.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.172+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb0adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.172+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae737b60 0x55a0aeb0b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.172+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeb10b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.172+0000 7fc8c241a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a0ae8c67a0 con 0x55a0ae737b60 2024-08-31T07:32:40.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.172+0000 7fc8c241a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a0ae9ea7f0 con 0x55a0ae72f980 2024-08-31T07:32:40.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.172+0000 7fc8c241a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0aeb0d890 con 0x55a0ae758840 2024-08-31T07:32:40.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb0adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae737b60 0x55a0aeb0b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb0adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38566/0 (socket says 172.21.15.32:38566) 2024-08-31T07:32:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b8c00640 1 -- 172.21.15.32:0/2408430029 learned_addr learned my addr 172.21.15.32:0/2408430029 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.172+0000 7fc8c241a740 10 client.0 _ll_register_callbacks cb 0x55a0aea0fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b9600640 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeb10b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b8c00640 1 -- 172.21.15.32:0/2408430029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 msgr2=0x55a0aeb10b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeb10b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b8c00640 1 -- 172.21.15.32:0/2408430029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae737b60 msgr2=0x55a0aeb0b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae737b60 0x55a0aeb0b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b8c00640 1 -- 172.21.15.32:0/2408430029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ae7d2b90 con 0x55a0ae72f980 2024-08-31T07:32:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb0adb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005740 tx=0x7fc8a802a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5079eb5baa8830 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.173+0000 7fc89a000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:32:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.174+0000 7fc89a000640 1 -- 172.21.15.32:0/2408430029 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8057070 con 0x55a0ae72f980 2024-08-31T07:32:40.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.174+0000 7fc89a000640 1 -- 172.21.15.32:0/2408430029 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7fc8a80046c0 con 0x55a0ae72f980 2024-08-31T07:32:40.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.174+0000 7fc8c241a740 1 -- 172.21.15.32:0/2408430029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8a8004d70 con 0x55a0ae72f980 2024-08-31T07:32:40.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.174+0000 7fc89a000640 1 -- 172.21.15.32:0/2408430029 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fc8a8061050 con 0x55a0ae72f980 2024-08-31T07:32:40.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.174+0000 7fc8c241a740 10 client.29363 fetch_fsmap learned FSMap version 73 2024-08-31T07:32:40.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.174+0000 7fc8c241a740 1 -- 172.21.15.32:0/2408430029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55a0aeb19960 con 0x55a0ae72f980 2024-08-31T07:32:40.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.175+0000 7fc89a000640 1 -- 172.21.15.32:0/2408430029 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a8004d70 con 0x55a0ae72f980 2024-08-31T07:32:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.175+0000 7fc89a000640 1 -- 172.21.15.32:0/2408430029 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 73) ==== 8398+0+0 (secure 0 0 0) 0x7fc8a8048450 con 0x55a0ae72f980 2024-08-31T07:32:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.175+0000 7fc8c241a740 10 client.29363 fetch_fsmap finished waiting for FSMap version 73 2024-08-31T07:32:40.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.175+0000 7fc8c241a740 10 client.29363 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:32:40.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.175+0000 7fc8c241a740 20 client.29363 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:40.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.175+0000 7fc8c241a740 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0ae7171c0 0x55a0aeb1ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.175+0000 7fc8c241a740 4 client.29363 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:32:40.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.175+0000 7fc8c241a740 1 -- 172.21.15.32:0/2408430029 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0aea76480 con 0x55a0ae7171c0 2024-08-31T07:32:40.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.175+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0ae7171c0 0x55a0aeb1ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.176+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0ae7171c0 0x55a0aeb1ccc0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9a5195285b72a8 server_cookie=2c930db22284bc79 in_seq=0 out_seq=0 2024-08-31T07:32:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.176+0000 7fc89a000640 10 client.29363 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.176+0000 7fc89a000640 1 -- 172.21.15.32:0/2408430029 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a0aea76480 con 0x55a0ae7171c0 2024-08-31T07:32:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.176+0000 7fc89a000640 10 client.29363 handle_command_reply: tid=0 2024-08-31T07:32:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.177+0000 7fc8c241a740 1 client.29363 shutdown 2024-08-31T07:32:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.177+0000 7fc8c241a740 1 -- 172.21.15.32:0/2408430029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 msgr2=0x55a0aeb0adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.177+0000 7fc8c241a740 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb0adb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005740 tx=0x7fc8a802a440 comp rx=0 tx=0).stop 2024-08-31T07:32:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.177+0000 7fc8c241a740 1 -- 172.21.15.32:0/2408430029 shutdown_connections 2024-08-31T07:32:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.177+0000 7fc8c241a740 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0ae7171c0 0x55a0aeb1ccc0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.177+0000 7fc8c241a740 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeb10b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.177+0000 7fc8c241a740 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae737b60 0x55a0aeb0b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.177+0000 7fc8c241a740 1 --2- 172.21.15.32:0/2408430029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb0adb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.177+0000 7fc8c241a740 1 -- 172.21.15.32:0/2408430029 >> 172.21.15.32:0/2408430029 conn(0x55a0ae74c990 msgr2=0x55a0aeafb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.178+0000 7fc89a000640 0 client.29363 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.178+0000 7fc8c241a740 1 -- 172.21.15.32:0/2408430029 shutdown_connections 2024-08-31T07:32:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.178+0000 7fc8c241a740 1 -- 172.21.15.32:0/2408430029 wait complete. 2024-08-31T07:32:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.178+0000 7fc8c241a740 20 client.29363 trim_cache size 0 max 16384 2024-08-31T07:32:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.196+0000 7fc8c241a740 1 Processor -- start 2024-08-31T07:32:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.196+0000 7fc8c241a740 1 -- start start 2024-08-31T07:32:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae72f980 0x55a0aeafb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeafbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeb04220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8c241a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41be230 con 0x55a0ae72f980 2024-08-31T07:32:40.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8c241a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41bfef0 con 0x55a0ae737b60 2024-08-31T07:32:40.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8c241a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41be900 con 0x55a0ae758840 2024-08-31T07:32:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeafbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeafbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38580/0 (socket says 172.21.15.32:38580) 2024-08-31T07:32:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8b9600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeb04220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8b3e00640 1 -- 172.21.15.32:0/1578571591 learned_addr learned my addr 172.21.15.32:0/1578571591 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.197+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/1578571591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae72f980 0x55a0aeafb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.198+0000 7fc8b3e00640 1 -- 172.21.15.32:0/1578571591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 msgr2=0x55a0aeb04220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.198+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/1578571591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeb04220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.198+0000 7fc8b3e00640 1 -- 172.21.15.32:0/1578571591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae72f980 msgr2=0x55a0aeafb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.198+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/1578571591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae72f980 0x55a0aeafb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.198+0000 7fc8b3e00640 1 -- 172.21.15.32:0/1578571591 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0aeb090b0 con 0x55a0ae737b60 2024-08-31T07:32:40.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.198+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/1578571591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeafbd20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00abd60 tx=0x7fc8a0004920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4960a7baabc63d46 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:40.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.198+0000 7fc8a5200640 1 -- 172.21.15.32:0/1578571591 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0097020 con 0x55a0ae737b60 2024-08-31T07:32:40.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.198+0000 7fc8a5200640 1 -- 172.21.15.32:0/1578571591 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a0091e50 con 0x55a0ae737b60 2024-08-31T07:32:40.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.198+0000 7fc8a5200640 1 -- 172.21.15.32:0/1578571591 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a00b5380 con 0x55a0ae737b60 2024-08-31T07:32:40.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.199+0000 7fc8c241a740 1 -- 172.21.15.32:0/1578571591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 msgr2=0x55a0aeafbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.199+0000 7fc8c241a740 1 --2- 172.21.15.32:0/1578571591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeafbd20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00abd60 tx=0x7fc8a0004920 comp rx=0 tx=0).stop 2024-08-31T07:32:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.199+0000 7fc8c241a740 1 -- 172.21.15.32:0/1578571591 shutdown_connections 2024-08-31T07:32:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.199+0000 7fc8c241a740 1 --2- 172.21.15.32:0/1578571591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae758840 0x55a0aeb04220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.199+0000 7fc8c241a740 1 --2- 172.21.15.32:0/1578571591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae737b60 0x55a0aeafbd20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.199+0000 7fc8c241a740 1 --2- 172.21.15.32:0/1578571591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae72f980 0x55a0aeafb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.199+0000 7fc8c241a740 1 -- 172.21.15.32:0/1578571591 >> 172.21.15.32:0/1578571591 conn(0x55a0ae74c990 msgr2=0x55a0aeb09430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:40.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.200+0000 7fc8c241a740 1 -- 172.21.15.32:0/1578571591 shutdown_connections 2024-08-31T07:32:40.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.200+0000 7fc8c241a740 1 -- 172.21.15.32:0/1578571591 wait complete. 2024-08-31T07:32:40.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.200+0000 7fc8c241a740 1 -- start start 2024-08-31T07:32:40.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.200+0000 7fc8c241a740 1 Processor -- start 2024-08-31T07:32:40.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb130a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae737b60 0x55a0aeb135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8c241a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae758840 0x55a0aeb86430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8c241a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41bf790 con 0x55a0ae758840 2024-08-31T07:32:40.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8c241a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41bf090 con 0x55a0ae72f980 2024-08-31T07:32:40.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8c241a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41bc230 con 0x55a0ae737b60 2024-08-31T07:32:40.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8c241a740 10 client.0 _ll_register_callbacks cb 0x7fc8a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:40.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8b8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8b8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb130a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38590/0 (socket says 172.21.15.32:38590) 2024-08-31T07:32:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8b8c00640 1 -- 172.21.15.32:0/572798993 learned_addr learned my addr 172.21.15.32:0/572798993 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8b9600640 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae758840 0x55a0aeb86430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae737b60 0x55a0aeb135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8b8c00640 1 -- 172.21.15.32:0/572798993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae737b60 msgr2=0x55a0aeb135e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae737b60 0x55a0aeb135e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8b8c00640 1 -- 172.21.15.32:0/572798993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae758840 msgr2=0x55a0aeb86430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae758840 0x55a0aeb86430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.201+0000 7fc8b8c00640 1 -- 172.21.15.32:0/572798993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0aeb86970 con 0x55a0ae72f980 2024-08-31T07:32:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.202+0000 7fc8b8c00640 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb130a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005ab0 tx=0x7fc8a8003ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0c6e62001443b7b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.202+0000 7fc89a000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:32:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.202+0000 7fc89a000640 1 -- 172.21.15.32:0/572798993 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a806b060 con 0x55a0ae72f980 2024-08-31T07:32:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.202+0000 7fc89a000640 1 -- 172.21.15.32:0/572798993 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7fc8a80046c0 con 0x55a0ae72f980 2024-08-31T07:32:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.202+0000 7fc8c241a740 1 -- 172.21.15.32:0/572798993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8a0091e50 con 0x55a0ae72f980 2024-08-31T07:32:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.202+0000 7fc89a000640 1 -- 172.21.15.32:0/572798993 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fc8a8057070 con 0x55a0ae72f980 2024-08-31T07:32:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.202+0000 7fc8c241a740 10 client.29369 fetch_fsmap learned FSMap version 73 2024-08-31T07:32:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.202+0000 7fc8c241a740 1 -- 172.21.15.32:0/572798993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55a0aeb87b30 con 0x55a0ae72f980 2024-08-31T07:32:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.203+0000 7fc89a000640 1 -- 172.21.15.32:0/572798993 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a0091e50 con 0x55a0ae72f980 2024-08-31T07:32:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.203+0000 7fc89a000640 1 -- 172.21.15.32:0/572798993 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 73) ==== 8398+0+0 (secure 0 0 0) 0x7fc8a80329d0 con 0x55a0ae72f980 2024-08-31T07:32:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.203+0000 7fc8c241a740 10 client.29369 fetch_fsmap finished waiting for FSMap version 73 2024-08-31T07:32:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.203+0000 7fc8c241a740 10 client.29369 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:32:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.203+0000 7fc8c241a740 20 client.29369 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.203+0000 7fc8c241a740 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0ae6ec570 0x55a0aeb8a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.203+0000 7fc8c241a740 4 client.29369 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 152.8, "await_for": 45.0}] 2024-08-31T07:32:40.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.203+0000 7fc8c241a740 1 -- 172.21.15.32:0/572798993 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 152.8, "await_for": 45.0}) -- 0x55a0aea71f20 con 0x55a0ae6ec570 2024-08-31T07:32:40.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.203+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0ae6ec570 0x55a0aeb8a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:40.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.204+0000 7fc8b3e00640 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0ae6ec570 0x55a0aeb8a3c0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d21ddab60469bc server_cookie=e27e216903921dd0 in_seq=0 out_seq=0 2024-08-31T07:32:40.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.204+0000 7fc89a000640 10 client.29369 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:40.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119593 14.42 MB/sec execute 260 sec latency 9084.168 ms 2024-08-31T07:32:40.413 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:40 smithi032 ceph-mon[17252]: pgmap v993: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.9 MiB/s wr, 388 op/s 2024-08-31T07:32:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:40 smithi073 ceph-mon[21995]: pgmap v993: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.9 MiB/s wr, 388 op/s 2024-08-31T07:32:40.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc89a000640 1 -- 172.21.15.32:0/572798993 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a0aea71f20 con 0x55a0ae6ec570 2024-08-31T07:32:40.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc89a000640 10 client.29369 handle_command_reply: tid=0 2024-08-31T07:32:40.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc8c241a740 1 client.29369 shutdown 2024-08-31T07:32:40.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc8c241a740 1 -- 172.21.15.32:0/572798993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 msgr2=0x55a0aeb130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:40.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc8c241a740 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb130a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005ab0 tx=0x7fc8a8003ff0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc8c241a740 1 -- 172.21.15.32:0/572798993 shutdown_connections 2024-08-31T07:32:40.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc8c241a740 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a0ae6ec570 0x55a0aeb8a3c0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc8c241a740 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a0ae758840 0x55a0aeb86430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc8c241a740 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0ae737b60 0x55a0aeb135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc8c241a740 1 --2- 172.21.15.32:0/572798993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0ae72f980 0x55a0aeb130a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc8c241a740 1 -- 172.21.15.32:0/572798993 >> 172.21.15.32:0/572798993 conn(0x55a0ae74c990 msgr2=0x55a0aeb093a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:40.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.728+0000 7fc89a000640 0 client.29369 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:40.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.729+0000 7fc8c241a740 1 -- 172.21.15.32:0/572798993 shutdown_connections 2024-08-31T07:32:40.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.729+0000 7fc8c241a740 1 -- 172.21.15.32:0/572798993 wait complete. 2024-08-31T07:32:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.729+0000 7fc8c241a740 20 client.29369 trim_cache size 0 max 16384 2024-08-31T07:32:40.731 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:32:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 73, 2024-08-31T07:32:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:32:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 62, 2024-08-31T07:32:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:32:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "a5d91dba": { 2024-08-31T07:32:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 62, 2024-08-31T07:32:40.731 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 152.8, 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:32:40.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:32:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:32:40.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:32:40.733 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:32:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 -- 172.21.15.32:0/3462503033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc89c0b4d20 msgr2=0x7fc89c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc89c0b4d20 0x7fc89c0b7230 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009900 tx=0x7fc8a8000bc0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 -- 172.21.15.32:0/3462503033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b4102eb0 msgr2=0x7fc8b410fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:40.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b4102eb0 0x7fc8b410fd90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e9c0 tx=0x7fc8a000ee80 comp rx=0 tx=0).stop 2024-08-31T07:32:40.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 -- 172.21.15.32:0/3462503033 shutdown_connections 2024-08-31T07:32:40.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc89c0b4d20 0x7fc89c0b7230 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b41079a0 0x7fc8b4110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b4102eb0 0x7fc8b410fd90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 --2- 172.21.15.32:0/3462503033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8b4102560 0x7fc8b4116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:40.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 -- 172.21.15.32:0/3462503033 >> 172.21.15.32:0/3462503033 conn(0x7fc8b4079500 msgr2=0x7fc8b41053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:40.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 -- 172.21.15.32:0/3462503033 shutdown_connections 2024-08-31T07:32:40.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:40.730+0000 7fc8bb600640 1 -- 172.21.15.32:0/3462503033 wait complete. 2024-08-31T07:32:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:40 smithi084 ceph-mon[22387]: pgmap v993: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 5.9 MiB/s wr, 388 op/s 2024-08-31T07:32:41.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 14.37 MB/sec execute 261 sec latency 10084.242 ms 2024-08-31T07:32:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.123+0000 7f9933c00640 1 -- 172.21.15.32:0/167499651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f992c10b470 msgr2=0x7f992c10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.123+0000 7f9933c00640 1 --2- 172.21.15.32:0/167499651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f992c10b470 0x7f992c10b880 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9918009900 tx=0x7f991802a400 comp rx=0 tx=0).stop 2024-08-31T07:32:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.123+0000 7f9933c00640 1 -- 172.21.15.32:0/167499651 shutdown_connections 2024-08-31T07:32:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.123+0000 7f9933c00640 1 --2- 172.21.15.32:0/167499651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f992c10bdc0 0x7f992c1125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.123+0000 7f9933c00640 1 --2- 172.21.15.32:0/167499651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f992c10b470 0x7f992c10b880 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.123+0000 7f9933c00640 1 --2- 172.21.15.32:0/167499651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f992c105b30 0x7f992c105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.123+0000 7f9933c00640 1 -- 172.21.15.32:0/167499651 >> 172.21.15.32:0/167499651 conn(0x7f992c101600 msgr2=0x7f992c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:41.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.124+0000 7f9933c00640 1 -- 172.21.15.32:0/167499651 shutdown_connections 2024-08-31T07:32:41.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.124+0000 7f9933c00640 1 -- 172.21.15.32:0/167499651 wait complete. 2024-08-31T07:32:41.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.124+0000 7f9933c00640 1 Processor -- start 2024-08-31T07:32:41.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9933c00640 1 -- start start 2024-08-31T07:32:41.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9933c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f992c105b30 0x7f992c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9933c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f992c10b470 0x7f992c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9933c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f992c10bdc0 0x7f992c1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9933c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f992c115a80 con 0x7f992c105b30 2024-08-31T07:32:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9933c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f992c115900 con 0x7f992c10b470 2024-08-31T07:32:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9933c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f992c115c00 con 0x7f992c10bdc0 2024-08-31T07:32:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9931200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f992c105b30 0x7f992c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9931200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f992c105b30 0x7f992c19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51416/0 (socket says 172.21.15.32:51416) 2024-08-31T07:32:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9931200640 1 -- 172.21.15.32:0/2684784017 learned_addr learned my addr 172.21.15.32:0/2684784017 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f992be00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f992c10b470 0x7f992c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.125+0000 7f9931c00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f992c10bdc0 0x7f992c1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.126+0000 7f9931200640 1 -- 172.21.15.32:0/2684784017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f992c10bdc0 msgr2=0x7f992c1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.126+0000 7f9931200640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f992c10bdc0 0x7f992c1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.126+0000 7f9931200640 1 -- 172.21.15.32:0/2684784017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f992c10b470 msgr2=0x7f992c19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.126+0000 7f9931200640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f992c10b470 0x7f992c19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.126+0000 7f9931200640 1 -- 172.21.15.32:0/2684784017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c1a2420 con 0x7f992c105b30 2024-08-31T07:32:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.126+0000 7f9931c00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f992c10bdc0 0x7f992c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.126+0000 7f9931200640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f992c105b30 0x7f992c19c4b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f992000ea60 tx=0x7f992000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:41.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.126+0000 7f9929600640 1 -- 172.21.15.32:0/2684784017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992000cd20 con 0x7f992c105b30 2024-08-31T07:32:41.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.126+0000 7f9933c00640 1 -- 172.21.15.32:0/2684784017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992c1a2710 con 0x7f992c105b30 2024-08-31T07:32:41.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.127+0000 7f9929600640 1 -- 172.21.15.32:0/2684784017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99200035f0 con 0x7f992c105b30 2024-08-31T07:32:41.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.127+0000 7f9929600640 1 -- 172.21.15.32:0/2684784017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920005220 con 0x7f992c105b30 2024-08-31T07:32:41.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.127+0000 7f9933c00640 1 -- 172.21.15.32:0/2684784017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992c1a8e70 con 0x7f992c105b30 2024-08-31T07:32:41.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.130+0000 7f993aa07740 1 Processor -- start 2024-08-31T07:32:41.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.130+0000 7f993aa07740 1 -- start start 2024-08-31T07:32:41.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.130+0000 7f993aa07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26baa350 0x562d26f50600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.130+0000 7f993aa07740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b81490 0x562d26f50b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.130+0000 7f993aa07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b89670 0x562d26f51080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.130+0000 7f9929600640 1 -- 172.21.15.32:0/2684784017 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f99200037a0 con 0x7f992c105b30 2024-08-31T07:32:41.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.130+0000 7f993aa07740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562d26ec6750 con 0x562d26b81490 2024-08-31T07:32:41.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.130+0000 7f993aa07740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562d26eb5ce0 con 0x562d26b89670 2024-08-31T07:32:41.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.130+0000 7f993aa07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562d26eb9c90 con 0x562d26baa350 2024-08-31T07:32:41.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f9931200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b81490 0x562d26f50b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f9931c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26baa350 0x562d26f50600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f992be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b89670 0x562d26f51080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f992be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b89670 0x562d26f51080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:38622/0 (socket says 172.21.15.32:38622) 2024-08-31T07:32:41.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f992be00640 1 -- 172.21.15.32:0/4014928010 learned_addr learned my addr 172.21.15.32:0/4014928010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:41.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f992be00640 1 -- 172.21.15.32:0/4014928010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26baa350 msgr2=0x562d26f50600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:41.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f992be00640 1 --2- 172.21.15.32:0/4014928010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26baa350 0x562d26f50600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f992be00640 1 -- 172.21.15.32:0/4014928010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b81490 msgr2=0x562d26f50b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:41.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f992be00640 1 --2- 172.21.15.32:0/4014928010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b81490 0x562d26f50b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f992be00640 1 -- 172.21.15.32:0/4014928010 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d26c15930 con 0x562d26b89670 2024-08-31T07:32:41.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.131+0000 7f992be00640 1 --2- 172.21.15.32:0/4014928010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b89670 0x562d26f51080 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9918009900 tx=0x7f9918000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fda82c046141889 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:41.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.132+0000 7f991e000640 1 -- 172.21.15.32:0/4014928010 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918040070 con 0x562d26b89670 2024-08-31T07:32:41.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.132+0000 7f991e000640 1 -- 172.21.15.32:0/4014928010 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9918004030 con 0x562d26b89670 2024-08-31T07:32:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.132+0000 7f991e000640 1 -- 172.21.15.32:0/4014928010 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918035040 con 0x562d26b89670 2024-08-31T07:32:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.132+0000 7f9929600640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99140b4ca0 0x7f99140b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f993aa07740 1 -- 172.21.15.32:0/4014928010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b89670 msgr2=0x562d26f51080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f993aa07740 1 --2- 172.21.15.32:0/4014928010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b89670 0x562d26f51080 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9918009900 tx=0x7f9918000bc0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f9929600640 1 -- 172.21.15.32:0/2684784017 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f99200e57a0 con 0x7f992c105b30 2024-08-31T07:32:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f993aa07740 1 -- 172.21.15.32:0/4014928010 shutdown_connections 2024-08-31T07:32:41.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f993aa07740 1 --2- 172.21.15.32:0/4014928010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b89670 0x562d26f51080 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f993aa07740 1 --2- 172.21.15.32:0/4014928010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b81490 0x562d26f50b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f993aa07740 1 --2- 172.21.15.32:0/4014928010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26baa350 0x562d26f50600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f9931c00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99140b4ca0 0x7f99140b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f993aa07740 1 -- 172.21.15.32:0/4014928010 >> 172.21.15.32:0/4014928010 conn(0x562d26b9e740 msgr2=0x562d26f4e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:41.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f993aa07740 1 -- 172.21.15.32:0/4014928010 shutdown_connections 2024-08-31T07:32:41.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.133+0000 7f993aa07740 1 -- 172.21.15.32:0/4014928010 wait complete. 2024-08-31T07:32:41.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.134+0000 7f9931c00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99140b4ca0 0x7f99140b71b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9924005fd0 tx=0x7f9924009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:41.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.134+0000 7f993aa07740 1 -- start start 2024-08-31T07:32:41.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.134+0000 7f993aa07740 1 Processor -- start 2024-08-31T07:32:41.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f993aa07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f5c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f993aa07740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f5c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f993aa07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26baa350 0x562d26f68070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f993aa07740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562d26d18890 con 0x562d26b89670 2024-08-31T07:32:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f993aa07740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562d26e3ccd0 con 0x562d26baa350 2024-08-31T07:32:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f993aa07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562d26f5ed80 con 0x562d26b81490 2024-08-31T07:32:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f993aa07740 10 client.0 _ll_register_callbacks cb 0x562d26eb0ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:41.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f9931200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f5c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f992be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f5c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f9931200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f5c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42334/0 (socket says 172.21.15.32:42334) 2024-08-31T07:32:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f9931200640 1 -- 172.21.15.32:0/4087158043 learned_addr learned my addr 172.21.15.32:0/4087158043 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f9931c00640 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26baa350 0x562d26f68070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f9931200640 1 -- 172.21.15.32:0/4087158043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26baa350 msgr2=0x562d26f68070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f9931200640 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26baa350 0x562d26f68070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f9931200640 1 -- 172.21.15.32:0/4087158043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 msgr2=0x562d26f5c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f9931200640 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f5c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.135+0000 7f9931200640 1 -- 172.21.15.32:0/4087158043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d26e861b0 con 0x562d26b81490 2024-08-31T07:32:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f9931200640 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f5c150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f99200abe10 tx=0x7f9920009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c8c31411dbc3760 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f9912a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:32:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f9912a00640 1 -- 172.21.15.32:0/4087158043 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99200ab7d0 con 0x562d26b81490 2024-08-31T07:32:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f9912a00640 1 -- 172.21.15.32:0/4087158043 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f99200b0350 con 0x562d26b81490 2024-08-31T07:32:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f993aa07740 1 -- 172.21.15.32:0/4087158043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9918004030 con 0x562d26b81490 2024-08-31T07:32:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f9912a00640 1 -- 172.21.15.32:0/4087158043 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f99200ab5f0 con 0x562d26b81490 2024-08-31T07:32:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f993aa07740 10 client.29368 fetch_fsmap learned FSMap version 73 2024-08-31T07:32:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f993aa07740 1 -- 172.21.15.32:0/4087158043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x562d26f6ad40 con 0x562d26b81490 2024-08-31T07:32:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f9912a00640 1 -- 172.21.15.32:0/4087158043 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9918004030 con 0x562d26b81490 2024-08-31T07:32:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f9912a00640 1 -- 172.21.15.32:0/4087158043 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 73) ==== 8398+0+0 (secure 0 0 0) 0x7f99200e3160 con 0x562d26b81490 2024-08-31T07:32:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f993aa07740 10 client.29368 fetch_fsmap finished waiting for FSMap version 73 2024-08-31T07:32:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.136+0000 7f993aa07740 10 client.29368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:32:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.137+0000 7f993aa07740 20 client.29368 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.137+0000 7f993aa07740 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d26b68cd0 0x562d26f6e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.137+0000 7f993aa07740 4 client.29368 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:32:41.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.137+0000 7f993aa07740 1 -- 172.21.15.32:0/4087158043 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d26ec86a0 con 0x562d26b68cd0 2024-08-31T07:32:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.137+0000 7f992be00640 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d26b68cd0 0x562d26f6e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.137+0000 7f992be00640 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d26b68cd0 0x562d26f6e320 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=315a7c4f91ddb1 server_cookie=3c81afea5a829aa6 in_seq=0 out_seq=0 2024-08-31T07:32:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.137+0000 7f9912a00640 10 client.29368 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f9912a00640 1 -- 172.21.15.32:0/4087158043 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d26ec86a0 con 0x562d26b68cd0 2024-08-31T07:32:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f9912a00640 10 client.29368 handle_command_reply: tid=0 2024-08-31T07:32:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f993aa07740 1 client.29368 shutdown 2024-08-31T07:32:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f993aa07740 1 -- 172.21.15.32:0/4087158043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 msgr2=0x562d26f5c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f993aa07740 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f5c150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f99200abe10 tx=0x7f9920009490 comp rx=0 tx=0).stop 2024-08-31T07:32:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f993aa07740 1 -- 172.21.15.32:0/4087158043 shutdown_connections 2024-08-31T07:32:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f993aa07740 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d26b68cd0 0x562d26f6e320 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f993aa07740 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26baa350 0x562d26f68070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f9912a00640 0 client.29368 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f993aa07740 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f5c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f993aa07740 1 --2- 172.21.15.32:0/4087158043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f5c150 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.138+0000 7f993aa07740 1 -- 172.21.15.32:0/4087158043 >> 172.21.15.32:0/4087158043 conn(0x562d26b9e740 msgr2=0x562d26f53790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.139+0000 7f993aa07740 1 -- 172.21.15.32:0/4087158043 shutdown_connections 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.139+0000 7f993aa07740 1 -- 172.21.15.32:0/4087158043 wait complete. 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.139+0000 7f993aa07740 20 client.29368 trim_cache size 0 max 16384 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.157+0000 7f993aa07740 1 Processor -- start 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f993aa07740 1 -- start start 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f993aa07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b9e740 0x562d26f5df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f993aa07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b81490 0x562d26f56720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f993aa07740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f51ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:41 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:32:41.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:41 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:41.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:41 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:32:41.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:41 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:32:41.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:41 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:32:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f993aa07740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1b1630 con 0x562d26b89670 2024-08-31T07:32:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f993aa07740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1b32f0 con 0x562d26b81490 2024-08-31T07:32:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f993aa07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1b1d00 con 0x562d26b9e740 2024-08-31T07:32:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f9931200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b9e740 0x562d26f5df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f9931c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f51ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f9931200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b9e740 0x562d26f5df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:42348/0 (socket says 172.21.15.32:42348) 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f9931200640 1 -- 172.21.15.32:0/3455208069 learned_addr learned my addr 172.21.15.32:0/3455208069 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.158+0000 7f992be00640 1 --2- 172.21.15.32:0/3455208069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b81490 0x562d26f56720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.159+0000 7f9931200640 1 -- 172.21.15.32:0/3455208069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b81490 msgr2=0x562d26f56720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.159+0000 7f9931200640 1 --2- 172.21.15.32:0/3455208069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b81490 0x562d26f56720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.159+0000 7f9931200640 1 -- 172.21.15.32:0/3455208069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 msgr2=0x562d26f51ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.159+0000 7f9931200640 1 --2- 172.21.15.32:0/3455208069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f51ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.159+0000 7f9931200640 1 -- 172.21.15.32:0/3455208069 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d26f528f0 con 0x562d26b9e740 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.159+0000 7f9931200640 1 --2- 172.21.15.32:0/3455208069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b9e740 0x562d26f5df40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f99200eb190 tx=0x7f9920000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d3729710c5cb57 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.159+0000 7f991d600640 1 -- 172.21.15.32:0/3455208069 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99200b51f0 con 0x562d26b9e740 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.159+0000 7f991d600640 1 -- 172.21.15.32:0/3455208069 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9920097020 con 0x562d26b9e740 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.159+0000 7f991d600640 1 -- 172.21.15.32:0/3455208069 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99200a7d40 con 0x562d26b9e740 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.160+0000 7f993aa07740 1 -- 172.21.15.32:0/3455208069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b9e740 msgr2=0x562d26f5df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.160+0000 7f993aa07740 1 --2- 172.21.15.32:0/3455208069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b9e740 0x562d26f5df40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f99200eb190 tx=0x7f9920000bc0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.160+0000 7f993aa07740 1 -- 172.21.15.32:0/3455208069 shutdown_connections 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.160+0000 7f993aa07740 1 --2- 172.21.15.32:0/3455208069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f51ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.160+0000 7f993aa07740 1 --2- 172.21.15.32:0/3455208069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b81490 0x562d26f56720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.160+0000 7f993aa07740 1 --2- 172.21.15.32:0/3455208069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b9e740 0x562d26f5df40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.160+0000 7f993aa07740 1 -- 172.21.15.32:0/3455208069 >> 172.21.15.32:0/3455208069 conn(0x562d26f4e540 msgr2=0x562d26f5a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.160+0000 7f993aa07740 1 -- 172.21.15.32:0/3455208069 shutdown_connections 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.160+0000 7f993aa07740 1 -- 172.21.15.32:0/3455208069 wait complete. 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.161+0000 7f993aa07740 1 -- start start 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.161+0000 7f993aa07740 1 Processor -- start 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.161+0000 7f993aa07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f60f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.161+0000 7f993aa07740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f61440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.161+0000 7f993aa07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b9e740 0x562d26f61980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.161+0000 7f993aa07740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1b2b90 con 0x562d26b89670 2024-08-31T07:32:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.161+0000 7f993aa07740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1b2490 con 0x562d26b9e740 2024-08-31T07:32:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.161+0000 7f993aa07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1af630 con 0x562d26b81490 2024-08-31T07:32:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.161+0000 7f993aa07740 10 client.0 _ll_register_callbacks cb 0x7f992000ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f992be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f61440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f9931200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f60f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f992be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f61440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51452/0 (socket says 172.21.15.32:51452) 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f992be00640 1 -- 172.21.15.32:0/2588152261 learned_addr learned my addr 172.21.15.32:0/2588152261 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f9931c00640 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b9e740 0x562d26f61980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f992be00640 1 -- 172.21.15.32:0/2588152261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 msgr2=0x562d26f60f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f992be00640 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f60f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f992be00640 1 -- 172.21.15.32:0/2588152261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b9e740 msgr2=0x562d26f61980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f992be00640 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b9e740 0x562d26f61980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f992be00640 1 -- 172.21.15.32:0/2588152261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d26fd8cf0 con 0x562d26b89670 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f9931200640 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f60f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f9931c00640 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b9e740 0x562d26f61980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f992be00640 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f61440 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9918031a90 tx=0x7f991802a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d6909e880a5319 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.162+0000 7f9912a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:32:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.163+0000 7f9912a00640 1 -- 172.21.15.32:0/2588152261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991805a060 con 0x562d26b89670 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.163+0000 7f993aa07740 1 -- 172.21.15.32:0/2588152261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9920097020 con 0x562d26b89670 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.163+0000 7f9912a00640 1 -- 172.21.15.32:0/2588152261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7f9918004240 con 0x562d26b89670 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.163+0000 7f9912a00640 1 -- 172.21.15.32:0/2588152261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f9918035040 con 0x562d26b89670 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.163+0000 7f993aa07740 10 client.19329 fetch_fsmap learned FSMap version 73 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.163+0000 7f993aa07740 1 -- 172.21.15.32:0/2588152261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x562d26fd9f50 con 0x562d26b89670 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.163+0000 7f9912a00640 1 -- 172.21.15.32:0/2588152261 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9920097020 con 0x562d26b89670 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.164+0000 7f9912a00640 1 -- 172.21.15.32:0/2588152261 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 73) ==== 8398+0+0 (secure 0 0 0) 0x7f9918004990 con 0x562d26b89670 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.164+0000 7f993aa07740 10 client.19329 fetch_fsmap finished waiting for FSMap version 73 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.164+0000 7f993aa07740 10 client.19329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.164+0000 7f993aa07740 20 client.19329 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.164+0000 7f993aa07740 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d26baa350 0x562d26fdc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.164+0000 7f993aa07740 4 client.19329 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:32:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.164+0000 7f993aa07740 1 -- 172.21.15.32:0/2588152261 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562d26ec4140 con 0x562d26baa350 2024-08-31T07:32:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.164+0000 7f9931200640 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d26baa350 0x562d26fdc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.164+0000 7f9931200640 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d26baa350 0x562d26fdc360 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222bd1c94d052573 server_cookie=eb30efa786cbc04d in_seq=0 out_seq=0 2024-08-31T07:32:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.164+0000 7f9912a00640 10 client.19329 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f9912a00640 1 -- 172.21.15.32:0/2588152261 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562d26ec4140 con 0x562d26baa350 2024-08-31T07:32:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f9912a00640 10 client.19329 handle_command_reply: tid=0 2024-08-31T07:32:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f993aa07740 1 client.19329 shutdown 2024-08-31T07:32:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f993aa07740 1 -- 172.21.15.32:0/2588152261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 msgr2=0x562d26f61440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f993aa07740 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f61440 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9918031a90 tx=0x7f991802a9a0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f993aa07740 1 -- 172.21.15.32:0/2588152261 shutdown_connections 2024-08-31T07:32:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f993aa07740 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x562d26baa350 0x562d26fdc360 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f993aa07740 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562d26b9e740 0x562d26f61980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f993aa07740 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562d26b89670 0x562d26f61440 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f993aa07740 1 --2- 172.21.15.32:0/2588152261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d26b81490 0x562d26f60f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f9912a00640 0 client.19329 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.165+0000 7f993aa07740 1 -- 172.21.15.32:0/2588152261 >> 172.21.15.32:0/2588152261 conn(0x562d26f4e540 msgr2=0x562d26f5a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:41.477 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:32:41.477 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:32:41.477 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "5476ca92-3649-4ebb-868c-09cfdc2c594c", 2024-08-31T07:32:41.477 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:32:41.477 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:32:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.166+0000 7f993aa07740 1 -- 172.21.15.32:0/2588152261 shutdown_connections 2024-08-31T07:32:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.166+0000 7f993aa07740 1 -- 172.21.15.32:0/2588152261 wait complete. 2024-08-31T07:32:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.166+0000 7f993aa07740 20 client.19329 trim_cache size 0 max 16384 2024-08-31T07:32:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.166+0000 7f9933c00640 1 -- 172.21.15.32:0/2684784017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99140b4ca0 msgr2=0x7f99140b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.166+0000 7f9933c00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99140b4ca0 0x7f99140b71b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9924005fd0 tx=0x7f9924009500 comp rx=0 tx=0).stop 2024-08-31T07:32:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.166+0000 7f9933c00640 1 -- 172.21.15.32:0/2684784017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f992c105b30 msgr2=0x7f992c19c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.167+0000 7f9933c00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f992c105b30 0x7f992c19c4b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f992000ea60 tx=0x7f992000ee60 comp rx=0 tx=0).stop 2024-08-31T07:32:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.167+0000 7f9933c00640 1 -- 172.21.15.32:0/2684784017 shutdown_connections 2024-08-31T07:32:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.167+0000 7f9933c00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f99140b4ca0 0x7f99140b71b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.167+0000 7f9933c00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f992c10bdc0 0x7f992c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.167+0000 7f9933c00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f992c10b470 0x7f992c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.167+0000 7f9933c00640 1 --2- 172.21.15.32:0/2684784017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f992c105b30 0x7f992c19c4b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.167+0000 7f9933c00640 1 -- 172.21.15.32:0/2684784017 >> 172.21.15.32:0/2684784017 conn(0x7f992c101600 msgr2=0x7f992c103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.167+0000 7f9933c00640 1 -- 172.21.15.32:0/2684784017 shutdown_connections 2024-08-31T07:32:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:41.167+0000 7f9933c00640 1 -- 172.21.15.32:0/2684784017 wait complete. 2024-08-31T07:32:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:41 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:32:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:41 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:41 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:32:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:41 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:32:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:41 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:32:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:41 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:32:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:41 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:41.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:41 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:32:41.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:41 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:32:41.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:41 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:32:42.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:42 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:32:42.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:42 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:32:42.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:42 smithi032 ceph-mon[17252]: pgmap v994: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 482 op/s 2024-08-31T07:32:42.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:42 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:42 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:42 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:42 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.401 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 14.31 MB/sec execute 262 sec latency 11084.363 ms 2024-08-31T07:32:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:42 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:32:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:42 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:32:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:42 smithi073 ceph-mon[21995]: pgmap v994: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 482 op/s 2024-08-31T07:32:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:42 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:42 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:42 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:42 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:42 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:32:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:42 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:32:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:42 smithi084 ceph-mon[22387]: pgmap v994: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 482 op/s 2024-08-31T07:32:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:42 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:42 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:42 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:42 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:32:42.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:32:43.410 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 14.26 MB/sec execute 263 sec latency 12093.156 ms 2024-08-31T07:32:43.771 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a5d91dba', quiesce took 13.0 seconds. Will release after: 19.799999999999997 2024-08-31T07:32:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:32:44.410 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 14.21 MB/sec execute 264 sec latency 13093.227 ms 2024-08-31T07:32:44.485 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:44 smithi032 ceph-mon[17252]: pgmap v995: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 507 op/s 2024-08-31T07:32:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:44 smithi073 ceph-mon[21995]: pgmap v995: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 507 op/s 2024-08-31T07:32:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:44 smithi084 ceph-mon[22387]: pgmap v995: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 507 op/s 2024-08-31T07:32:45.410 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 14.15 MB/sec execute 265 sec latency 14093.282 ms 2024-08-31T07:32:45.635 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:45 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:32:45.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:45 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:32:45.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:45 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:32:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 -- 172.21.15.32:0/2621861439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a810d9c0 msgr2=0x7fd8a810feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/2621861439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a810d9c0 0x7fd8a810feb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009ca0 tx=0x7fd8a002a8d0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 -- 172.21.15.32:0/2621861439 shutdown_connections 2024-08-31T07:32:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/2621861439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a810d9c0 0x7fd8a810feb0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/2621861439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8104eb0 0x7fd8a810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/2621861439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a8104560 0x7fd8a8104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 -- 172.21.15.32:0/2621861439 >> 172.21.15.32:0/2621861439 conn(0x7fd8a80fe060 msgr2=0x7fd8a81004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 -- 172.21.15.32:0/2621861439 shutdown_connections 2024-08-31T07:32:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 -- 172.21.15.32:0/2621861439 wait complete. 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 Processor -- start 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 -- start start 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8104560 0x7fd8a81022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8104eb0 0x7fd8a81027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a810d9c0 0x7fd8a8102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81136c0 con 0x7fd8a8104560 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8113540 con 0x7fd8a810d9c0 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8113840 con 0x7fd8a8104eb0 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8104eb0 0x7fd8a81027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8104eb0 0x7fd8a81027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50934/0 (socket says 172.21.15.32:50934) 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6000640 1 -- 172.21.15.32:0/738535031 learned_addr learned my addr 172.21.15.32:0/738535031 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a7400640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a810d9c0 0x7fd8a8102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6000640 1 -- 172.21.15.32:0/738535031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a810d9c0 msgr2=0x7fd8a8102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6000640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a810d9c0 0x7fd8a8102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6000640 1 -- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8104560 msgr2=0x7fd8a81022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6a00640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8104560 0x7fd8a81022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6000640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8104560 0x7fd8a81022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6000640 1 -- 172.21.15.32:0/738535031 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a81a8330 con 0x7fd8a8104eb0 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6a00640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8104560 0x7fd8a81022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.952+0000 7fd8a6000640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8104eb0 0x7fd8a81027e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd89400e970 tx=0x7fd89400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.957+0000 7fd89f400640 1 -- 172.21.15.32:0/738535031 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89400cd80 con 0x7fd8a8104eb0 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.957+0000 7fd8a7e00640 1 -- 172.21.15.32:0/738535031 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a81a8620 con 0x7fd8a8104eb0 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.957+0000 7fd89f400640 1 -- 172.21.15.32:0/738535031 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8940035f0 con 0x7fd8a8104eb0 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.957+0000 7fd89f400640 1 -- 172.21.15.32:0/738535031 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894010690 con 0x7fd8a8104eb0 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.957+0000 7fd8a7e00640 1 -- 172.21.15.32:0/738535031 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a81a8c30 con 0x7fd8a8104eb0 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8b42a1740 1 Processor -- start 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8b42a1740 1 -- start start 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d328f070 0x5562d3634b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d3635060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 0x5562d36355a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5562d35ab770 con 0x5562d326e390 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5562d359ad00 con 0x5562d32661b0 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5562d359ecb0 con 0x5562d328f070 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8a6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d3635060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8a6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d3635060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47956/0 (socket says 172.21.15.32:47956) 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8a6a00640 1 -- 172.21.15.32:0/733567308 learned_addr learned my addr 172.21.15.32:0/733567308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8a6000640 1 --2- 172.21.15.32:0/733567308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 0x5562d36355a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8a6000640 1 -- 172.21.15.32:0/733567308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d328f070 msgr2=0x5562d3634b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8a6000640 1 --2- 172.21.15.32:0/733567308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d328f070 0x5562d3634b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8a6000640 1 -- 172.21.15.32:0/733567308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 msgr2=0x5562d3635060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8a6000640 1 --2- 172.21.15.32:0/733567308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d3635060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8a6000640 1 -- 172.21.15.32:0/733567308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562d32fa650 con 0x5562d326e390 2024-08-31T07:32:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd8a6000640 1 --2- 172.21.15.32:0/733567308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 0x5562d36355a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd8940098c0 tx=0x7fd894003950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becfe51a0a579a14 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd893400640 1 -- 172.21.15.32:0/733567308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894005050 con 0x5562d326e390 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd893400640 1 -- 172.21.15.32:0/733567308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8940108f0 con 0x5562d326e390 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.958+0000 7fd893400640 1 -- 172.21.15.32:0/733567308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894004c40 con 0x5562d326e390 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- 172.21.15.32:0/733567308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 msgr2=0x5562d36355a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/733567308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 0x5562d36355a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd8940098c0 tx=0x7fd894003950 comp rx=0 tx=0).stop 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- 172.21.15.32:0/733567308 shutdown_connections 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/733567308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 0x5562d36355a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/733567308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d3635060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/733567308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d328f070 0x5562d3634b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- 172.21.15.32:0/733567308 >> 172.21.15.32:0/733567308 conn(0x5562d3283460 msgr2=0x5562d3632a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- 172.21.15.32:0/733567308 shutdown_connections 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- 172.21.15.32:0/733567308 wait complete. 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- start start 2024-08-31T07:32:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 Processor -- start 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d32661b0 0x5562d3640810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 0x5562d3640db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d328f070 0x5562d364edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5562d33fd700 con 0x5562d326e390 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5562d3521cb0 con 0x5562d328f070 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5562d364bb10 con 0x5562d32661b0 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 10 client.0 _ll_register_callbacks cb 0x5562d3547430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8a7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d32661b0 0x5562d3640810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8a7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d32661b0 0x5562d3640810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50958/0 (socket says 172.21.15.32:50958) 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8a7400640 1 -- 172.21.15.32:0/2770713453 learned_addr learned my addr 172.21.15.32:0/2770713453 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8a6a00640 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 0x5562d3640db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8a7400640 1 -- 172.21.15.32:0/2770713453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d328f070 msgr2=0x5562d364edd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:32:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d328f070 0x5562d364edd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8a7400640 1 -- 172.21.15.32:0/2770713453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 msgr2=0x5562d3640db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 0x5562d3640db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8a7400640 1 -- 172.21.15.32:0/2770713453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562d3562870 con 0x5562d32661b0 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d32661b0 0x5562d3640810 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009a30 tx=0x7fd8a0009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18a67e50d2c4515a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd887e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd887e00640 1 -- 172.21.15.32:0/2770713453 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0031660 con 0x5562d32661b0 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd887e00640 1 -- 172.21.15.32:0/2770713453 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7fd8a0031800 con 0x5562d32661b0 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2770713453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8940108f0 con 0x5562d32661b0 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd887e00640 1 -- 172.21.15.32:0/2770713453 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fd8a0003970 con 0x5562d32661b0 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 10 client.29383 fetch_fsmap learned FSMap version 73 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2770713453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5562d3657c50 con 0x5562d32661b0 2024-08-31T07:32:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd887e00640 1 -- 172.21.15.32:0/2770713453 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8940108f0 con 0x5562d32661b0 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd887e00640 1 -- 172.21.15.32:0/2770713453 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 73) ==== 8398+0+0 (secure 0 0 0) 0x7fd8a00305d0 con 0x5562d32661b0 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 10 client.29383 fetch_fsmap finished waiting for FSMap version 73 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 10 client.29383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 20 client.29383 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5562d324d9f0 0x5562d365b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 4 client.29383 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.964+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2770713453 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562d35ad6c0 con 0x5562d324d9f0 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.982+0000 7fd8a6a00640 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5562d324d9f0 0x5562d365b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.982+0000 7fd89f400640 1 -- 172.21.15.32:0/738535031 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd894005050 con 0x7fd8a8104eb0 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.982+0000 7fd89f400640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4bb0 0x7fd88c0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.987+0000 7fd89f400640 1 -- 172.21.15.32:0/738535031 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7fd8940e6d10 con 0x7fd8a8104eb0 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.987+0000 7fd8a6000640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4bb0 0x7fd88c0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:45.987+0000 7fd8a6000640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4bb0 0x7fd88c0b70c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd8940aa830 tx=0x7fd894011fc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.029+0000 7fd8a6a00640 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5562d324d9f0 0x5562d365b1b0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccbeae68c2614636 server_cookie=23e496f659542d39 in_seq=0 out_seq=0 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.029+0000 7fd887e00640 10 client.29383 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd887e00640 1 -- 172.21.15.32:0/2770713453 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5562d35ad6c0 con 0x5562d324d9f0 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd887e00640 10 client.29383 handle_command_reply: tid=0 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 client.29383 shutdown 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2770713453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d32661b0 msgr2=0x5562d3640810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d32661b0 0x5562d3640810 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009a30 tx=0x7fd8a0009490 comp rx=0 tx=0).stop 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2770713453 shutdown_connections 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5562d324d9f0 0x5562d365b1b0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d328f070 0x5562d364edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d326e390 0x5562d3640db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2770713453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d32661b0 0x5562d3640810 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd887e00640 0 client.29383 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2770713453 >> 172.21.15.32:0/2770713453 conn(0x5562d3283460 msgr2=0x5562d3637d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2770713453 shutdown_connections 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2770713453 wait complete. 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.041+0000 7fd8b42a1740 20 client.29383 trim_cache size 0 max 16384 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.059+0000 7fd8b42a1740 1 Processor -- start 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.059+0000 7fd8b42a1740 1 -- start start 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d32661b0 0x5562d3639c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d326e390 0x5562d363a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d328f070 0x5562d363d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81ad210 con 0x5562d32661b0 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81aeed0 con 0x5562d326e390 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81ad8e0 con 0x5562d328f070 2024-08-31T07:32:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d32661b0 0x5562d3639c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d32661b0 0x5562d3639c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54880/0 (socket says 172.21.15.32:54880) 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8a7400640 1 -- 172.21.15.32:0/2212119084 learned_addr learned my addr 172.21.15.32:0/2212119084 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2212119084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d328f070 0x5562d363d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8a7400640 1 -- 172.21.15.32:0/2212119084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d328f070 msgr2=0x5562d363d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2212119084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d328f070 0x5562d363d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8a7400640 1 -- 172.21.15.32:0/2212119084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d326e390 msgr2=0x5562d363a170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2212119084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d326e390 0x5562d363a170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.064+0000 7fd8a7400640 1 -- 172.21.15.32:0/2212119084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562d3641210 con 0x5562d32661b0 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2212119084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d32661b0 0x5562d3639c30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd8a002adf0 tx=0x7fd8a002a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddf37e05c1e68b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd892a00640 1 -- 172.21.15.32:0/2212119084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0030e10 con 0x5562d32661b0 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd892a00640 1 -- 172.21.15.32:0/2212119084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a0036040 con 0x5562d32661b0 2024-08-31T07:32:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd892a00640 1 -- 172.21.15.32:0/2212119084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a002fbb0 con 0x5562d32661b0 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2212119084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d32661b0 msgr2=0x5562d3639c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2212119084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d32661b0 0x5562d3639c30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd8a002adf0 tx=0x7fd8a002a8d0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2212119084 shutdown_connections 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2212119084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d328f070 0x5562d363d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2212119084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d326e390 0x5562d363a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2212119084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d32661b0 0x5562d3639c30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2212119084 >> 172.21.15.32:0/2212119084 conn(0x5562d3283460 msgr2=0x5562d3632350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2212119084 shutdown_connections 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2212119084 wait complete. 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 -- start start 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 Processor -- start 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d36559f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d326e390 0x5562d36546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d328f070 0x5562d3654ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81ae770 con 0x5562d328f070 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81ae070 con 0x5562d32661b0 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81ab250 con 0x5562d326e390 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8b42a1740 10 client.0 _ll_register_callbacks cb 0x7fd8a002f320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d36559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d36559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47978/0 (socket says 172.21.15.32:47978) 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 -- 172.21.15.32:0/2003142265 learned_addr learned my addr 172.21.15.32:0/2003142265 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d328f070 0x5562d3654ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 -- 172.21.15.32:0/2003142265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d326e390 msgr2=0x5562d36546a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d326e390 0x5562d36546a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 -- 172.21.15.32:0/2003142265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d328f070 msgr2=0x5562d3654ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d328f070 0x5562d3654ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 -- 172.21.15.32:0/2003142265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562d3655400 con 0x5562d32661b0 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd8a7400640 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d36559f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0030840 tx=0x7fd8a002f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=978ff3d0796292c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.065+0000 7fd887e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd887e00640 1 -- 172.21.15.32:0/2003142265 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0067040 con 0x5562d32661b0 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd887e00640 1 -- 172.21.15.32:0/2003142265 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7fd8a0030e10 con 0x5562d32661b0 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2003142265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8a0036040 con 0x5562d32661b0 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd887e00640 1 -- 172.21.15.32:0/2003142265 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fd8a0073050 con 0x5562d32661b0 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd8b42a1740 10 client.29396 fetch_fsmap learned FSMap version 73 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2003142265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5562d36c5c00 con 0x5562d32661b0 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd887e00640 1 -- 172.21.15.32:0/2003142265 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a0036040 con 0x5562d32661b0 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd887e00640 1 -- 172.21.15.32:0/2003142265 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 73) ==== 8398+0+0 (secure 0 0 0) 0x7fd8a002f950 con 0x5562d32661b0 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd8b42a1740 10 client.29396 fetch_fsmap finished waiting for FSMap version 73 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd8b42a1740 10 client.29396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:32:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd8b42a1740 20 client.29396 populate_metadata read hostname 'smithi032' 2024-08-31T07:32:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5562d3222da0 0x5562d36c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:32:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd8b42a1740 4 client.29396 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:32:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.071+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2003142265 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562d3632f70 con 0x5562d3222da0 2024-08-31T07:32:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.077+0000 7fd8a6a00640 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5562d3222da0 0x5562d36c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:32:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd887e00640 10 client.29396 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a6a00640 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5562d3222da0 0x5562d36c8200 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8f2d4bef91ae00 server_cookie=b6659b92b7fdf9c7 in_seq=0 out_seq=0 2024-08-31T07:32:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd887e00640 1 -- 172.21.15.32:0/2003142265 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6818 (crc 0 0 0) 0x5562d3632f70 con 0x5562d3222da0 2024-08-31T07:32:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd887e00640 10 client.29396 handle_command_reply: tid=0 2024-08-31T07:32:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 client.29396 shutdown 2024-08-31T07:32:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2003142265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 msgr2=0x5562d36559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d36559f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0030840 tx=0x7fd8a002f730 comp rx=0 tx=0).stop 2024-08-31T07:32:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2003142265 shutdown_connections 2024-08-31T07:32:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5562d3222da0 0x5562d36c8200 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562d328f070 0x5562d3654ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5562d326e390 0x5562d36546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 --2- 172.21.15.32:0/2003142265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562d32661b0 0x5562d36559f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2003142265 >> 172.21.15.32:0/2003142265 conn(0x5562d3283460 msgr2=0x5562d3639c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd887e00640 0 client.29396 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:32:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2003142265 shutdown_connections 2024-08-31T07:32:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 1 -- 172.21.15.32:0/2003142265 wait complete. 2024-08-31T07:32:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8b42a1740 20 client.29396 trim_cache size 0 max 16384 2024-08-31T07:32:46.130 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "5476ca92-3649-4ebb-868c-09cfdc2c594c", 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:32:41.165270+0000", 2024-08-31T07:32:46.131 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:32:46.134 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.135 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.136 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:32:46.137 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.139 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.140 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.141 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.143 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:32:46.144 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:32:46.145 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:32:46.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:32:46.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:32:46.145 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:32:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 -- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4bb0 msgr2=0x7fd88c0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4bb0 0x7fd88c0b70c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd8940aa830 tx=0x7fd894011fc0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 -- 172.21.15.32:0/738535031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8104eb0 msgr2=0x7fd8a81027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:32:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8104eb0 0x7fd8a81027e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd89400e970 tx=0x7fd89400ee30 comp rx=0 tx=0).stop 2024-08-31T07:32:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 -- 172.21.15.32:0/738535031 shutdown_connections 2024-08-31T07:32:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4bb0 0x7fd88c0b70c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a810d9c0 0x7fd8a8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8104eb0 0x7fd8a81027e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/738535031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8104560 0x7fd8a81022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:32:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 -- 172.21.15.32:0/738535031 >> 172.21.15.32:0/738535031 conn(0x7fd8a80fe060 msgr2=0x7fd8a8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:32:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 -- 172.21.15.32:0/738535031 shutdown_connections 2024-08-31T07:32:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:32:46.124+0000 7fd8a7e00640 1 -- 172.21.15.32:0/738535031 wait complete. 2024-08-31T07:32:46.410 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 14.10 MB/sec execute 266 sec latency 15093.333 ms 2024-08-31T07:32:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:46 smithi073 ceph-mon[21995]: pgmap v996: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 525 op/s 2024-08-31T07:32:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:46 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.703 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:46 smithi032 ceph-mon[17252]: pgmap v996: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 525 op/s 2024-08-31T07:32:46.703 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.703 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.703 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.703 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.704 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.704 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:46 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:46 smithi084 ceph-mon[22387]: pgmap v996: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 525 op/s 2024-08-31T07:32:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:46 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:47.410 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 14.05 MB/sec execute 267 sec latency 16093.382 ms 2024-08-31T07:32:48.410 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.99 MB/sec execute 268 sec latency 17093.439 ms 2024-08-31T07:32:48.735 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:48 smithi032 ceph-mon[17252]: pgmap v997: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.1 MiB/s wr, 419 op/s 2024-08-31T07:32:48.735 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:48.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:48 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:48 smithi084 ceph-mon[22387]: pgmap v997: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.1 MiB/s wr, 419 op/s 2024-08-31T07:32:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:48 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:48 smithi073 ceph-mon[21995]: pgmap v997: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.1 MiB/s wr, 419 op/s 2024-08-31T07:32:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:48 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:48.940 INFO:tasks.cephfs.filesystem:scrub status for tag:5476ca92-3649-4ebb-868c-09cfdc2c594c - {'path': '/', 'tag': '5476ca92-3649-4ebb-868c-09cfdc2c594c', 'options': 'recursive,force'} 2024-08-31T07:32:49.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.94 MB/sec execute 269 sec latency 18093.498 ms 2024-08-31T07:32:49.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:49 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:32:49.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:49 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:49 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:49 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:49 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:49 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:32:49.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:49 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:49 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:49 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:49 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:49 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:32:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:49 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:49 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:49 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:49 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:50.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.89 MB/sec execute 270 sec latency 19093.903 ms 2024-08-31T07:32:50.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:50 smithi032 ceph-mon[17252]: pgmap v998: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.8 MiB/s wr, 376 op/s 2024-08-31T07:32:50.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:50.690 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:50 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:50 smithi073 ceph-mon[21995]: pgmap v998: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.8 MiB/s wr, 376 op/s 2024-08-31T07:32:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:50 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:50 smithi084 ceph-mon[22387]: pgmap v998: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.8 MiB/s wr, 376 op/s 2024-08-31T07:32:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:50.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:50 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:51.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.84 MB/sec execute 271 sec latency 20093.969 ms 2024-08-31T07:32:52.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.79 MB/sec execute 272 sec latency 21094.031 ms 2024-08-31T07:32:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:52 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:52 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:52.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:52 smithi073 ceph-mon[21995]: pgmap v999: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.2 MiB/s wr, 347 op/s 2024-08-31T07:32:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:52 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:52 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:52.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:52 smithi084 ceph-mon[22387]: pgmap v999: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.2 MiB/s wr, 347 op/s 2024-08-31T07:32:52.587 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:52 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:52.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:52 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:52.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:52 smithi032 ceph-mon[17252]: pgmap v999: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.2 MiB/s wr, 347 op/s 2024-08-31T07:32:53.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.74 MB/sec execute 273 sec latency 22094.092 ms 2024-08-31T07:32:54.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.69 MB/sec execute 274 sec latency 23094.140 ms 2024-08-31T07:32:54.601 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:54 smithi032 ceph-mon[17252]: pgmap v1000: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.4 MiB/s wr, 273 op/s 2024-08-31T07:32:54.601 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:54 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:54.601 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:54 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:54.601 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:54 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:32:54.601 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:54 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:32:54.601 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:54 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:54 smithi073 ceph-mon[21995]: pgmap v1000: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.4 MiB/s wr, 273 op/s 2024-08-31T07:32:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:54 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:54 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:54 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:32:54.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:54 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:32:54.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:54 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:54 smithi084 ceph-mon[22387]: pgmap v1000: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.4 MiB/s wr, 273 op/s 2024-08-31T07:32:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:54 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:54.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:54 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:54.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:54 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:32:54.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:54 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:32:54.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:54 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:32:55.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.64 MB/sec execute 275 sec latency 24094.207 ms 2024-08-31T07:32:56.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.59 MB/sec execute 276 sec latency 25094.268 ms 2024-08-31T07:32:56.837 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:56 smithi032 ceph-mon[17252]: pgmap v1001: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 232 op/s 2024-08-31T07:32:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:56 smithi073 ceph-mon[21995]: pgmap v1001: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 232 op/s 2024-08-31T07:32:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:56 smithi084 ceph-mon[22387]: pgmap v1001: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 232 op/s 2024-08-31T07:32:57.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.54 MB/sec execute 277 sec latency 26094.308 ms 2024-08-31T07:32:58.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.49 MB/sec execute 278 sec latency 27094.355 ms 2024-08-31T07:32:58.776 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:32:58 smithi032 ceph-mon[17252]: pgmap v1002: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 5.3 MiB/s wr, 173 op/s 2024-08-31T07:32:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:32:58 smithi073 ceph-mon[21995]: pgmap v1002: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 5.3 MiB/s wr, 173 op/s 2024-08-31T07:32:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:32:58 smithi084 ceph-mon[22387]: pgmap v1002: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 5.3 MiB/s wr, 173 op/s 2024-08-31T07:32:59.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.44 MB/sec execute 279 sec latency 28094.422 ms 2024-08-31T07:33:00.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.39 MB/sec execute 280 sec latency 29094.468 ms 2024-08-31T07:33:00.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:00 smithi032 ceph-mon[17252]: pgmap v1003: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.8 MiB/s wr, 116 op/s 2024-08-31T07:33:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:00 smithi073 ceph-mon[21995]: pgmap v1003: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.8 MiB/s wr, 116 op/s 2024-08-31T07:33:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:00 smithi084 ceph-mon[22387]: pgmap v1003: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.8 MiB/s wr, 116 op/s 2024-08-31T07:33:01.411 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.35 MB/sec execute 281 sec latency 30094.595 ms 2024-08-31T07:33:02.412 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.30 MB/sec execute 282 sec latency 31094.688 ms 2024-08-31T07:33:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:02 smithi032 ceph-mon[17252]: pgmap v1004: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-08-31T07:33:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:02 smithi073 ceph-mon[21995]: pgmap v1004: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-08-31T07:33:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:02 smithi084 ceph-mon[22387]: pgmap v1004: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.3 MiB/s wr, 129 op/s 2024-08-31T07:33:03.412 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.25 MB/sec execute 283 sec latency 32094.836 ms 2024-08-31T07:33:03.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:33:03.963 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:33:04.412 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.20 MB/sec execute 284 sec latency 33094.988 ms 2024-08-31T07:33:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.623+0000 7fd39ae00640 1 -- 172.21.15.32:0/496639561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd394105b20 msgr2=0x7fd394105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.623+0000 7fd39ae00640 1 --2- 172.21.15.32:0/496639561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd394105b20 0x7fd394105f30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd384009900 tx=0x7fd38402a410 comp rx=0 tx=0).stop 2024-08-31T07:33:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.623+0000 7fd39ae00640 1 -- 172.21.15.32:0/496639561 shutdown_connections 2024-08-31T07:33:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.623+0000 7fd39ae00640 1 --2- 172.21.15.32:0/496639561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd39410bdb0 0x7fd3941125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.623+0000 7fd39ae00640 1 --2- 172.21.15.32:0/496639561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd39410b460 0x7fd39410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.623+0000 7fd39ae00640 1 --2- 172.21.15.32:0/496639561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd394105b20 0x7fd394105f30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.623+0000 7fd39ae00640 1 -- 172.21.15.32:0/496639561 >> 172.21.15.32:0/496639561 conn(0x7fd394101570 msgr2=0x7fd3941039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.623+0000 7fd39ae00640 1 -- 172.21.15.32:0/496639561 shutdown_connections 2024-08-31T07:33:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.623+0000 7fd39ae00640 1 -- 172.21.15.32:0/496639561 wait complete. 2024-08-31T07:33:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.624+0000 7fd39ae00640 1 Processor -- start 2024-08-31T07:33:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.624+0000 7fd39ae00640 1 -- start start 2024-08-31T07:33:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.624+0000 7fd39ae00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd394105b20 0x7fd39419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.624+0000 7fd39ae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd39410b460 0x7fd39419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.624+0000 7fd39ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd39410bdb0 0x7fd3941a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd39ae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd394115a70 con 0x7fd39410bdb0 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd39ae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd3941158f0 con 0x7fd39410b460 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd39ae00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd394115bf0 con 0x7fd394105b20 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd393e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd394105b20 0x7fd39419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd393400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd39410b460 0x7fd39419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd393e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd394105b20 0x7fd39419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33808/0 (socket says 172.21.15.32:33808) 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd393e00640 1 -- 172.21.15.32:0/2036140141 learned_addr learned my addr 172.21.15.32:0/2036140141 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd398e00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd39410bdb0 0x7fd3941a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd393e00640 1 -- 172.21.15.32:0/2036140141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd39410b460 msgr2=0x7fd39419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd393e00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd39410b460 0x7fd39419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd393e00640 1 -- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd39410bdb0 msgr2=0x7fd3941a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd393e00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd39410bdb0 0x7fd3941a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd393e00640 1 -- 172.21.15.32:0/2036140141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3941a2430 con 0x7fd394105b20 2024-08-31T07:33:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd393400640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd39410b460 0x7fd39419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:33:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.625+0000 7fd398e00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd39410bdb0 0x7fd3941a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:33:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.626+0000 7fd393e00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd394105b20 0x7fd39419c4c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd384005940 tx=0x7fd384004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.626+0000 7fd390c00640 1 -- 172.21.15.32:0/2036140141 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38403f030 con 0x7fd394105b20 2024-08-31T07:33:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.626+0000 7fd390c00640 1 -- 172.21.15.32:0/2036140141 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd384003aa0 con 0x7fd394105b20 2024-08-31T07:33:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.626+0000 7fd390c00640 1 -- 172.21.15.32:0/2036140141 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384035070 con 0x7fd394105b20 2024-08-31T07:33:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.626+0000 7fd39ae00640 1 -- 172.21.15.32:0/2036140141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3941a26c0 con 0x7fd394105b20 2024-08-31T07:33:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.626+0000 7fd39ae00640 1 -- 172.21.15.32:0/2036140141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3941a8e80 con 0x7fd394105b20 2024-08-31T07:33:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.628+0000 7fd39ae00640 1 -- 172.21.15.32:0/2036140141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd394106da0 con 0x7fd394105b20 2024-08-31T07:33:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.630+0000 7fd390c00640 1 -- 172.21.15.32:0/2036140141 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd384042030 con 0x7fd394105b20 2024-08-31T07:33:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.631+0000 7fd390c00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd3780b4d20 0x7fd3780b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.631+0000 7fd390c00640 1 -- 172.21.15.32:0/2036140141 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8441+0+0 (secure 0 0 0) 0x7fd384105b00 con 0x7fd394105b20 2024-08-31T07:33:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.631+0000 7fd393400640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd3780b4d20 0x7fd3780b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.632+0000 7fd393400640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd3780b4d20 0x7fd3780b7230 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd3800095f0 tx=0x7fd380005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.635+0000 7fd390c00640 1 -- 172.21.15.32:0/2036140141 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd3840cc290 con 0x7fd394105b20 2024-08-31T07:33:04.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:04 smithi032 ceph-mon[17252]: pgmap v1005: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-08-31T07:33:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.870+0000 7fd39ae00640 1 -- 172.21.15.32:0/2036140141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd39410dbb0 con 0x7fd394105b20 2024-08-31T07:33:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.872+0000 7fd390c00640 1 -- 172.21.15.32:0/2036140141 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+11973 (secure 0 0 0) 0x7fd3840d0a50 con 0x7fd394105b20 2024-08-31T07:33:04.873 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:33:04.873 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":74,"btime":"2024-08-31T07:32:48:457856+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":74,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:32:47.609437+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:33:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 74 2024-08-31T07:33:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 -- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd3780b4d20 msgr2=0x7fd3780b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd3780b4d20 0x7fd3780b7230 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd3800095f0 tx=0x7fd380005f90 comp rx=0 tx=0).stop 2024-08-31T07:33:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 -- 172.21.15.32:0/2036140141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd394105b20 msgr2=0x7fd39419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd394105b20 0x7fd39419c4c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd384005940 tx=0x7fd384004270 comp rx=0 tx=0).stop 2024-08-31T07:33:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 -- 172.21.15.32:0/2036140141 shutdown_connections 2024-08-31T07:33:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd3780b4d20 0x7fd3780b7230 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd39410bdb0 0x7fd3941a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd39410b460 0x7fd39419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 --2- 172.21.15.32:0/2036140141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd394105b20 0x7fd39419c4c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 -- 172.21.15.32:0/2036140141 >> 172.21.15.32:0/2036140141 conn(0x7fd394101570 msgr2=0x7fd394103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 -- 172.21.15.32:0/2036140141 shutdown_connections 2024-08-31T07:33:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:04.874+0000 7fd39ae00640 1 -- 172.21.15.32:0/2036140141 wait complete. 2024-08-31T07:33:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:04 smithi073 ceph-mon[21995]: pgmap v1005: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-08-31T07:33:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:04 smithi084 ceph-mon[22387]: pgmap v1005: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-08-31T07:33:05.412 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.16 MB/sec execute 285 sec latency 34095.123 ms 2024-08-31T07:33:05.698 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id a5d91dba --release --await' 2024-08-31T07:33:05.699 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id a5d91dba --release --await 2024-08-31T07:33:05.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:05 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2036140141' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:33:05.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:05 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2036140141' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:33:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:33:06.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:05 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2036140141' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:33:06.412 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.11 MB/sec execute 286 sec latency 35095.272 ms 2024-08-31T07:33:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:06 smithi073 ceph-mon[21995]: pgmap v1006: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 493 KiB/s wr, 97 op/s 2024-08-31T07:33:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:06 smithi084 ceph-mon[22387]: pgmap v1006: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 493 KiB/s wr, 97 op/s 2024-08-31T07:33:07.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:06 smithi032 ceph-mon[17252]: pgmap v1006: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 493 KiB/s wr, 97 op/s 2024-08-31T07:33:07.412 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119610 13.07 MB/sec execute 287 sec latency 36095.344 ms 2024-08-31T07:33:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.423+0000 7f513a200640 1 -- 172.21.15.32:0/645504712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51341069c0 msgr2=0x7f5134110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.423+0000 7f513a200640 1 --2- 172.21.15.32:0/645504712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51341069c0 0x7f5134110410 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f512c009ca0 tx=0x7f512c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.423+0000 7f513a200640 1 -- 172.21.15.32:0/645504712 shutdown_connections 2024-08-31T07:33:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.423+0000 7f513a200640 1 --2- 172.21.15.32:0/645504712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51341069c0 0x7f5134110410 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.423+0000 7f513a200640 1 --2- 172.21.15.32:0/645504712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5134106070 0x7f5134106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.423+0000 7f513a200640 1 --2- 172.21.15.32:0/645504712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513406baf0 0x7f513406bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.423+0000 7f513a200640 1 -- 172.21.15.32:0/645504712 >> 172.21.15.32:0/645504712 conn(0x7f5134079740 msgr2=0x7f5134079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.423+0000 7f513a200640 1 -- 172.21.15.32:0/645504712 shutdown_connections 2024-08-31T07:33:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.424+0000 7f513a200640 1 -- 172.21.15.32:0/645504712 wait complete. 2024-08-31T07:33:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.425+0000 7f513a200640 1 Processor -- start 2024-08-31T07:33:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.425+0000 7f513a200640 1 -- start start 2024-08-31T07:33:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.425+0000 7f513a200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f513406baf0 0x7f513419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.425+0000 7f513a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134106070 0x7f513419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.425+0000 7f513a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51341069c0 0x7f51341a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.425+0000 7f513a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f51341138b0 con 0x7f51341069c0 2024-08-31T07:33:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.425+0000 7f513a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5134113730 con 0x7f5134106070 2024-08-31T07:33:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.425+0000 7f513a200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5134113a30 con 0x7f513406baf0 2024-08-31T07:33:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f5133400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f513406baf0 0x7f513419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f5133e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51341069c0 0x7f51341a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f512aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134106070 0x7f513419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f512aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134106070 0x7f513419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:56192/0 (socket says 172.21.15.32:56192) 2024-08-31T07:33:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f512aa00640 1 -- 172.21.15.32:0/667218061 learned_addr learned my addr 172.21.15.32:0/667218061 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f5133400640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f513406baf0 0x7f513419c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:55306/0 (socket says 172.21.15.32:55306) 2024-08-31T07:33:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f5133400640 1 -- 172.21.15.32:0/667218061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134106070 msgr2=0x7f513419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f5133400640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134106070 0x7f513419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f5133400640 1 -- 172.21.15.32:0/667218061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51341069c0 msgr2=0x7f51341a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f5133400640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51341069c0 0x7f51341a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f5133400640 1 -- 172.21.15.32:0/667218061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51341a2410 con 0x7f513406baf0 2024-08-31T07:33:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f512aa00640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134106070 0x7f513419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:33:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.426+0000 7f5133e00640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51341069c0 0x7f51341a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:33:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.427+0000 7f5133400640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f513406baf0 0x7f513419c4a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f512400e9f0 tx=0x7f512400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.427+0000 7f5130c00640 1 -- 172.21.15.32:0/667218061 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512400ccc0 con 0x7f513406baf0 2024-08-31T07:33:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.427+0000 7f5130c00640 1 -- 172.21.15.32:0/667218061 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51240108e0 con 0x7f513406baf0 2024-08-31T07:33:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.427+0000 7f5130c00640 1 -- 172.21.15.32:0/667218061 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124018880 con 0x7f513406baf0 2024-08-31T07:33:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.427+0000 7f513a200640 1 -- 172.21.15.32:0/667218061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51341a2700 con 0x7f513406baf0 2024-08-31T07:33:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.427+0000 7f513a200640 1 -- 172.21.15.32:0/667218061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51341a8e60 con 0x7f513406baf0 2024-08-31T07:33:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.430+0000 7f5140dd9740 1 Processor -- start 2024-08-31T07:33:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.430+0000 7f5140dd9740 1 -- start start 2024-08-31T07:33:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29f1520 0x5596b2d96ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29c8660 0x5596b2d97430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 0x5596b2d97970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f5140dd9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5596b2d0da90 con 0x5596b29c8660 2024-08-31T07:33:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f5140dd9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5596b2cfd020 con 0x5596b29f1520 2024-08-31T07:33:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f5140dd9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5596b2d00fd0 con 0x5596b29d0840 2024-08-31T07:33:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f5133e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29f1520 0x5596b2d96ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f512aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 0x5596b2d97970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f5133e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29f1520 0x5596b2d96ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:56206/0 (socket says 172.21.15.32:56206) 2024-08-31T07:33:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f5133e00640 1 -- 172.21.15.32:0/4047316936 learned_addr learned my addr 172.21.15.32:0/4047316936 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f512aa00640 1 -- 172.21.15.32:0/4047316936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29f1520 msgr2=0x5596b2d96ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f512aa00640 1 --2- 172.21.15.32:0/4047316936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29f1520 0x5596b2d96ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f512aa00640 1 -- 172.21.15.32:0/4047316936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29c8660 msgr2=0x5596b2d97430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:33:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f512aa00640 1 --2- 172.21.15.32:0/4047316936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29c8660 0x5596b2d97430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.431+0000 7f512aa00640 1 -- 172.21.15.32:0/4047316936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596b2a5cb00 con 0x5596b29d0840 2024-08-31T07:33:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.432+0000 7f5130c00640 1 -- 172.21.15.32:0/667218061 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5124010430 con 0x7f513406baf0 2024-08-31T07:33:07.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.432+0000 7f512aa00640 1 --2- 172.21.15.32:0/4047316936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 0x5596b2d97970 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f513406ae50 tx=0x7f511c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb1aef7cfe5991f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.432+0000 7f5123e00640 1 -- 172.21.15.32:0/4047316936 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c01e070 con 0x5596b29d0840 2024-08-31T07:33:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.432+0000 7f5123e00640 1 -- 172.21.15.32:0/4047316936 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f511c00fa50 con 0x5596b29d0840 2024-08-31T07:33:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.432+0000 7f5123e00640 1 -- 172.21.15.32:0/4047316936 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c019b20 con 0x5596b29d0840 2024-08-31T07:33:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.433+0000 7f5140dd9740 1 -- 172.21.15.32:0/4047316936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 msgr2=0x5596b2d97970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.433+0000 7f5140dd9740 1 --2- 172.21.15.32:0/4047316936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 0x5596b2d97970 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f513406ae50 tx=0x7f511c0118d0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.433+0000 7f5140dd9740 1 -- 172.21.15.32:0/4047316936 shutdown_connections 2024-08-31T07:33:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.433+0000 7f5140dd9740 1 --2- 172.21.15.32:0/4047316936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 0x5596b2d97970 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.433+0000 7f5140dd9740 1 --2- 172.21.15.32:0/4047316936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29c8660 0x5596b2d97430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.433+0000 7f5140dd9740 1 --2- 172.21.15.32:0/4047316936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29f1520 0x5596b2d96ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.433+0000 7f5140dd9740 1 -- 172.21.15.32:0/4047316936 >> 172.21.15.32:0/4047316936 conn(0x5596b29e5910 msgr2=0x5596b2d94de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.433+0000 7f5140dd9740 1 -- 172.21.15.32:0/4047316936 shutdown_connections 2024-08-31T07:33:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.433+0000 7f5140dd9740 1 -- 172.21.15.32:0/4047316936 wait complete. 2024-08-31T07:33:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.434+0000 7f5140dd9740 1 -- start start 2024-08-31T07:33:07.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.434+0000 7f5130c00640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51180b4bb0 0x7f51180b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.434+0000 7f5140dd9740 1 Processor -- start 2024-08-31T07:33:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.434+0000 7f5130c00640 1 -- 172.21.15.32:0/667218061 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8485+0+0 (secure 0 0 0) 0x7f51240e5890 con 0x7f513406baf0 2024-08-31T07:33:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.434+0000 7f5133e00640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51180b4bb0 0x7f51180b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 0x5596b2da2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29f1520 0x5596b2da2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5140dd9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5596b2b5fbb0 con 0x5596b29f1520 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5140dd9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5596b2c83f90 con 0x5596b29c8660 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5140dd9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5596b2da56d0 con 0x5596b29d0840 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5140dd9740 10 client.0 _ll_register_callbacks cb 0x5596b2ca9710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5133e00640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51180b4bb0 0x7f51180b70c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f512c0094e0 tx=0x7f512c005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5133e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29f1520 0x5596b2da2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5133400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5133e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29f1520 0x5596b2da2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33694/0 (socket says 172.21.15.32:33694) 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f5133e00640 1 -- 172.21.15.32:0/397024010 learned_addr learned my addr 172.21.15.32:0/397024010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.435+0000 7f512aa00640 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 0x5596b2da2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.436+0000 7f5133400640 1 -- 172.21.15.32:0/397024010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 msgr2=0x5596b2da2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.436+0000 7f5133400640 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 0x5596b2da2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.436+0000 7f5133400640 1 -- 172.21.15.32:0/397024010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29f1520 msgr2=0x5596b2da2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.436+0000 7f5133400640 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29f1520 0x5596b2da2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.436+0000 7f5133400640 1 -- 172.21.15.32:0/397024010 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596b2ad3960 con 0x5596b29c8660 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.436+0000 7f5133400640 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2330 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f51240e2fa0 tx=0x7f5124010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e4da6b0fec8e19 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.436+0000 7f5120c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.436+0000 7f5120c00640 1 -- 172.21.15.32:0/397024010 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51240b07b0 con 0x5596b29c8660 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.436+0000 7f5120c00640 1 -- 172.21.15.32:0/397024010 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8485+0+0 (secure 0 0 0) 0x7f51240b3b30 con 0x5596b29c8660 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.436+0000 7f5140dd9740 1 -- 172.21.15.32:0/397024010 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596b2c9be30 con 0x5596b29c8660 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5120c00640 1 -- 172.21.15.32:0/397024010 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f51240b5260 con 0x5596b29c8660 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5140dd9740 10 client.29414 fetch_fsmap learned FSMap version 74 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5140dd9740 1 -- 172.21.15.32:0/397024010 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5596b2db1100 con 0x5596b29c8660 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5120c00640 1 -- 172.21.15.32:0/397024010 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5596b2c9be30 con 0x5596b29c8660 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5120c00640 1 -- 172.21.15.32:0/397024010 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8394+0+0 (secure 0 0 0) 0x7f51240971e0 con 0x5596b29c8660 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5140dd9740 10 client.29414 fetch_fsmap finished waiting for FSMap version 74 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5140dd9740 10 client.29414 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5140dd9740 20 client.29414 populate_metadata read hostname 'smithi032' 2024-08-31T07:33:07.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5140dd9740 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596b29afea0 0x5596b2db4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5140dd9740 4 client.29414 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:33:07.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.437+0000 7f5140dd9740 1 -- 172.21.15.32:0/397024010 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596b2d0f9e0 con 0x5596b29afea0 2024-08-31T07:33:07.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.438+0000 7f512aa00640 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596b29afea0 0x5596b2db4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.438+0000 7f512aa00640 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596b29afea0 0x5596b2db4660 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8496582b4633d2 server_cookie=436d4741690e278c in_seq=0 out_seq=0 2024-08-31T07:33:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.438+0000 7f5120c00640 10 client.29414 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.439+0000 7f5120c00640 1 -- 172.21.15.32:0/397024010 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596b2d0f9e0 con 0x5596b29afea0 2024-08-31T07:33:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.439+0000 7f5120c00640 10 client.29414 handle_command_reply: tid=0 2024-08-31T07:33:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.439+0000 7f5140dd9740 1 client.29414 shutdown 2024-08-31T07:33:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.439+0000 7f5140dd9740 1 -- 172.21.15.32:0/397024010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 msgr2=0x5596b2da2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.439+0000 7f5140dd9740 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2330 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f51240e2fa0 tx=0x7f5124010860 comp rx=0 tx=0).stop 2024-08-31T07:33:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.440+0000 7f5140dd9740 1 -- 172.21.15.32:0/397024010 shutdown_connections 2024-08-31T07:33:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.440+0000 7f5140dd9740 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596b29afea0 0x5596b2db4660 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.440+0000 7f5140dd9740 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29f1520 0x5596b2da2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.440+0000 7f5140dd9740 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29d0840 0x5596b2da2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.440+0000 7f5140dd9740 1 --2- 172.21.15.32:0/397024010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2330 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.440+0000 7f5140dd9740 1 -- 172.21.15.32:0/397024010 >> 172.21.15.32:0/397024010 conn(0x5596b29e5910 msgr2=0x5596b2d9a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.440+0000 7f5120c00640 0 client.29414 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.440+0000 7f5140dd9740 1 -- 172.21.15.32:0/397024010 shutdown_connections 2024-08-31T07:33:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.440+0000 7f5140dd9740 1 -- 172.21.15.32:0/397024010 wait complete. 2024-08-31T07:33:07.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.440+0000 7f5140dd9740 20 client.29414 trim_cache size 0 max 16384 2024-08-31T07:33:07.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.458+0000 7f5140dd9740 1 Processor -- start 2024-08-31T07:33:07.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.458+0000 7f5140dd9740 1 -- start start 2024-08-31T07:33:07.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.458+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.458+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2dae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29f1520 0x5596b2d9f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5140dd9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f51341b16b0 con 0x5596b29d0840 2024-08-31T07:33:07.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5140dd9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51341b3370 con 0x5596b29c8660 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5140dd9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f51341b1d80 con 0x5596b29f1520 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f512aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2dae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5133400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5133400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:56220/0 (socket says 172.21.15.32:56220) 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f512aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2dae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33708/0 (socket says 172.21.15.32:33708) 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5133400640 1 -- 172.21.15.32:0/991240149 learned_addr learned my addr 172.21.15.32:0/991240149 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5133e00640 1 --2- 172.21.15.32:0/991240149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29f1520 0x5596b2d9f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5133400640 1 -- 172.21.15.32:0/991240149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29f1520 msgr2=0x5596b2d9f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5133400640 1 --2- 172.21.15.32:0/991240149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29f1520 0x5596b2d9f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5133400640 1 -- 172.21.15.32:0/991240149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 msgr2=0x5596b2dae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5133400640 1 --2- 172.21.15.32:0/991240149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2dae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.459+0000 7f5133400640 1 -- 172.21.15.32:0/991240149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596b2d99d50 con 0x5596b29c8660 2024-08-31T07:33:07.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.460+0000 7f5133400640 1 --2- 172.21.15.32:0/991240149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2c60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5124000c00 tx=0x7f51240e3080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f39eb39fa1ce56a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.460+0000 7f5123400640 1 -- 172.21.15.32:0/991240149 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51240b07b0 con 0x5596b29c8660 2024-08-31T07:33:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.460+0000 7f5123400640 1 -- 172.21.15.32:0/991240149 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51240aaea0 con 0x5596b29c8660 2024-08-31T07:33:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.460+0000 7f5123400640 1 -- 172.21.15.32:0/991240149 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51240930e0 con 0x5596b29c8660 2024-08-31T07:33:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.461+0000 7f5140dd9740 1 -- 172.21.15.32:0/991240149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 msgr2=0x5596b2da2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.461+0000 7f5140dd9740 1 --2- 172.21.15.32:0/991240149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2c60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5124000c00 tx=0x7f51240e3080 comp rx=0 tx=0).stop 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.461+0000 7f5140dd9740 1 -- 172.21.15.32:0/991240149 shutdown_connections 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.461+0000 7f5140dd9740 1 --2- 172.21.15.32:0/991240149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29f1520 0x5596b2d9f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.461+0000 7f5140dd9740 1 --2- 172.21.15.32:0/991240149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2dae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.461+0000 7f5140dd9740 1 --2- 172.21.15.32:0/991240149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2da2c60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.461+0000 7f5140dd9740 1 -- 172.21.15.32:0/991240149 >> 172.21.15.32:0/991240149 conn(0x5596b29e5910 msgr2=0x5596b2d9a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.461+0000 7f5140dd9740 1 -- 172.21.15.32:0/991240149 shutdown_connections 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.461+0000 7f5140dd9740 1 -- 172.21.15.32:0/991240149 wait complete. 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.461+0000 7f5140dd9740 1 -- start start 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f5140dd9740 1 Processor -- start 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2dab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2da67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f5140dd9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29f1520 0x5596b2da6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f5140dd9740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f51341b2c10 con 0x5596b29d0840 2024-08-31T07:33:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f5140dd9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51341b2510 con 0x5596b29c8660 2024-08-31T07:33:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f5140dd9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f51341af6b0 con 0x5596b29f1520 2024-08-31T07:33:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f5140dd9740 10 client.0 _ll_register_callbacks cb 0x7f51240ab7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:33:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f512aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2da67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f512aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2da67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33718/0 (socket says 172.21.15.32:33718) 2024-08-31T07:33:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f512aa00640 1 -- 172.21.15.32:0/1865219447 learned_addr learned my addr 172.21.15.32:0/1865219447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f5133400640 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2dab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.462+0000 7f5133e00640 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29f1520 0x5596b2da6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.463+0000 7f512aa00640 1 -- 172.21.15.32:0/1865219447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29f1520 msgr2=0x5596b2da6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.463+0000 7f512aa00640 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29f1520 0x5596b2da6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.463+0000 7f512aa00640 1 -- 172.21.15.32:0/1865219447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 msgr2=0x5596b2dab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.463+0000 7f512aa00640 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2dab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.463+0000 7f512aa00640 1 -- 172.21.15.32:0/1865219447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596b2e1e020 con 0x5596b29d0840 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.463+0000 7f512aa00640 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2da67d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f511c000c00 tx=0x7f511c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf36dbd8785ebb1f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.463+0000 7f5120c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.463+0000 7f5120c00640 1 -- 172.21.15.32:0/1865219447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c031060 con 0x5596b29d0840 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.464+0000 7f5120c00640 1 -- 172.21.15.32:0/1865219447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8485+0+0 (secure 0 0 0) 0x7f511c019a80 con 0x5596b29d0840 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.464+0000 7f5140dd9740 1 -- 172.21.15.32:0/1865219447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51240aaea0 con 0x5596b29d0840 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.464+0000 7f5120c00640 1 -- 172.21.15.32:0/1865219447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f511c01e040 con 0x5596b29d0840 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.464+0000 7f5140dd9740 10 client.19365 fetch_fsmap learned FSMap version 74 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.464+0000 7f5140dd9740 1 -- 172.21.15.32:0/1865219447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5596b2e1f180 con 0x5596b29d0840 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.464+0000 7f5120c00640 1 -- 172.21.15.32:0/1865219447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51240aaea0 con 0x5596b29d0840 2024-08-31T07:33:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.464+0000 7f5120c00640 1 -- 172.21.15.32:0/1865219447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 74) ==== 8394+0+0 (secure 0 0 0) 0x7f511c004710 con 0x5596b29d0840 2024-08-31T07:33:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.464+0000 7f5140dd9740 10 client.19365 fetch_fsmap finished waiting for FSMap version 74 2024-08-31T07:33:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.464+0000 7f5140dd9740 10 client.19365 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:33:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.464+0000 7f5140dd9740 20 client.19365 populate_metadata read hostname 'smithi032' 2024-08-31T07:33:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.465+0000 7f5140dd9740 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596b2985250 0x5596b2e21740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.465+0000 7f5140dd9740 4 client.19365 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a5d91dba", "release": true, "await": true}] 2024-08-31T07:33:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.465+0000 7f5140dd9740 1 -- 172.21.15.32:0/1865219447 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a5d91dba", "release": true, "await": true}) -- 0x5596b2d0b480 con 0x5596b2985250 2024-08-31T07:33:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.465+0000 7f5133400640 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596b2985250 0x5596b2e21740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.465+0000 7f5133400640 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596b2985250 0x5596b2e21740 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02c9e0641c5c348 server_cookie=175a2285dfb8e657 in_seq=0 out_seq=0 2024-08-31T07:33:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.465+0000 7f5120c00640 10 client.19365 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.553+0000 7f5120c00640 1 -- 172.21.15.32:0/1865219447 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5596b2d0b480 con 0x5596b2985250 2024-08-31T07:33:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.553+0000 7f5120c00640 10 client.19365 handle_command_reply: tid=0 2024-08-31T07:33:07.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.553+0000 7f5140dd9740 1 client.19365 shutdown 2024-08-31T07:33:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.554+0000 7f5140dd9740 1 -- 172.21.15.32:0/1865219447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 msgr2=0x5596b2da67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.554+0000 7f5140dd9740 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2da67d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f511c000c00 tx=0x7f511c005d30 comp rx=0 tx=0).stop 2024-08-31T07:33:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.554+0000 7f5140dd9740 1 -- 172.21.15.32:0/1865219447 shutdown_connections 2024-08-31T07:33:07.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.554+0000 7f5140dd9740 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596b2985250 0x5596b2e21740 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.554+0000 7f5140dd9740 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596b29f1520 0x5596b2da6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.554+0000 7f5140dd9740 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596b29d0840 0x5596b2da67d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.554+0000 7f5140dd9740 1 --2- 172.21.15.32:0/1865219447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596b29c8660 0x5596b2dab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.554+0000 7f5120c00640 0 client.19365 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.554+0000 7f5140dd9740 1 -- 172.21.15.32:0/1865219447 >> 172.21.15.32:0/1865219447 conn(0x5596b29e5910 msgr2=0x5596b2d97f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.554+0000 7f5140dd9740 1 -- 172.21.15.32:0/1865219447 shutdown_connections 2024-08-31T07:33:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.555+0000 7f5140dd9740 1 -- 172.21.15.32:0/1865219447 wait complete. 2024-08-31T07:33:07.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.555+0000 7f5140dd9740 20 client.19365 trim_cache size 0 max 16384 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 74, 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 64, 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "a5d91dba": { 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 64, 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 152.8, 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:33:07.558 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:33:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 -- 172.21.15.32:0/667218061 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51180b4bb0 msgr2=0x7f51180b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51180b4bb0 0x7f51180b70c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f512c0094e0 tx=0x7f512c005d80 comp rx=0 tx=0).stop 2024-08-31T07:33:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 -- 172.21.15.32:0/667218061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f513406baf0 msgr2=0x7f513419c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:07.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f513406baf0 0x7f513419c4a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f512400e9f0 tx=0x7f512400eeb0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 -- 172.21.15.32:0/667218061 shutdown_connections 2024-08-31T07:33:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f51180b4bb0 0x7f51180b70c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51341069c0 0x7f51341a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5134106070 0x7f513419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 --2- 172.21.15.32:0/667218061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f513406baf0 0x7f513419c4a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 -- 172.21.15.32:0/667218061 >> 172.21.15.32:0/667218061 conn(0x7f5134079740 msgr2=0x7f5134109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 -- 172.21.15.32:0/667218061 shutdown_connections 2024-08-31T07:33:07.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:07.556+0000 7f513a200640 1 -- 172.21.15.32:0/667218061 wait complete. 2024-08-31T07:33:07.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:07 smithi032 ceph-mon[17252]: osdmap e109: 12 total, 12 up, 12 in 2024-08-31T07:33:07.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:07 smithi073 ceph-mon[21995]: osdmap e109: 12 total, 12 up, 12 in 2024-08-31T07:33:07.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:07 smithi084 ceph-mon[22387]: osdmap e109: 12 total, 12 up, 12 in 2024-08-31T07:33:08.412 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119872 13.07 MB/sec execute 288 sec latency 36937.098 ms 2024-08-31T07:33:08.645 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a5d91dba', total seconds elapsed: 37.9 2024-08-31T07:33:08.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:08 smithi032 ceph-mon[17252]: pgmap v1008: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 488 KiB/s wr, 84 op/s 2024-08-31T07:33:08.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:08 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:33:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:08 smithi073 ceph-mon[21995]: pgmap v1008: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 488 KiB/s wr, 84 op/s 2024-08-31T07:33:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:08 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:33:08.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:08 smithi084 ceph-mon[22387]: pgmap v1008: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 488 KiB/s wr, 84 op/s 2024-08-31T07:33:08.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:08 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:33:09.412 INFO:tasks.workunit.client.0.smithi032.stdout: 10 120531 13.13 MB/sec execute 289 sec latency 242.621 ms 2024-08-31T07:33:09.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:09 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:33:09.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:09 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:33:09.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:09 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:33:09.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:09 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:33:10.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:09 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:33:10.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:09 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:33:10.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 121235 13.22 MB/sec execute 290 sec latency 180.231 ms 2024-08-31T07:33:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:10 smithi073 ceph-mon[21995]: pgmap v1009: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 596 KiB/s wr, 85 op/s 2024-08-31T07:33:10.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:10 smithi084 ceph-mon[22387]: pgmap v1009: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 596 KiB/s wr, 85 op/s 2024-08-31T07:33:11.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:10 smithi032 ceph-mon[17252]: pgmap v1009: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 596 KiB/s wr, 85 op/s 2024-08-31T07:33:11.412 INFO:tasks.workunit.client.0.smithi032.stdout: 10 122146 13.37 MB/sec execute 291 sec latency 66.306 ms 2024-08-31T07:33:12.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 122894 13.46 MB/sec execute 292 sec latency 67.194 ms 2024-08-31T07:33:12.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:12 smithi032 ceph-mon[17252]: pgmap v1010: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-08-31T07:33:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:12 smithi073 ceph-mon[21995]: pgmap v1010: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-08-31T07:33:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:12 smithi084 ceph-mon[22387]: pgmap v1010: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-08-31T07:33:13.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 123578 13.52 MB/sec execute 293 sec latency 117.930 ms 2024-08-31T07:33:14.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 124284 13.60 MB/sec execute 294 sec latency 162.567 ms 2024-08-31T07:33:14.649 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:14 smithi032 ceph-mon[17252]: pgmap v1011: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-08-31T07:33:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:14 smithi073 ceph-mon[21995]: pgmap v1011: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-08-31T07:33:14.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:14 smithi084 ceph-mon[22387]: pgmap v1011: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-08-31T07:33:15.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 125199 13.71 MB/sec execute 295 sec latency 128.800 ms 2024-08-31T07:33:16.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 126129 13.87 MB/sec execute 296 sec latency 105.314 ms 2024-08-31T07:33:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:16 smithi073 ceph-mon[21995]: pgmap v1012: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-08-31T07:33:16.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:16 smithi084 ceph-mon[22387]: pgmap v1012: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-08-31T07:33:17.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:16 smithi032 ceph-mon[17252]: pgmap v1012: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.2 MiB/s wr, 300 op/s 2024-08-31T07:33:17.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 126883 13.95 MB/sec execute 297 sec latency 90.477 ms 2024-08-31T07:33:18.147 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:18 smithi032 ceph-mon[17252]: osdmap e110: 12 total, 12 up, 12 in 2024-08-31T07:33:18.148 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:18 smithi032 ceph-mon[17252]: pgmap v1014: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 8.3 MiB/s wr, 528 op/s 2024-08-31T07:33:18.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127355 13.98 MB/sec execute 298 sec latency 319.682 ms 2024-08-31T07:33:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:18 smithi073 ceph-mon[21995]: osdmap e110: 12 total, 12 up, 12 in 2024-08-31T07:33:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:18 smithi073 ceph-mon[21995]: pgmap v1014: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 8.3 MiB/s wr, 528 op/s 2024-08-31T07:33:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:18 smithi084 ceph-mon[22387]: osdmap e110: 12 total, 12 up, 12 in 2024-08-31T07:33:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:18 smithi084 ceph-mon[22387]: pgmap v1014: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 8.3 MiB/s wr, 528 op/s 2024-08-31T07:33:18.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:33:19.333 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:33:19.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127359 13.94 MB/sec execute 299 sec latency 1319.723 ms 2024-08-31T07:33:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.243+0000 7f3069e00640 1 -- 172.21.15.32:0/2597105039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306410dbd0 msgr2=0x7f30641100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.243+0000 7f3069e00640 1 --2- 172.21.15.32:0/2597105039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306410dbd0 0x7f30641100c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f305c009ca0 tx=0x7f305c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.243+0000 7f3069e00640 1 -- 172.21.15.32:0/2597105039 shutdown_connections 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.243+0000 7f3069e00640 1 --2- 172.21.15.32:0/2597105039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306410dbd0 0x7f30641100c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.243+0000 7f3069e00640 1 --2- 172.21.15.32:0/2597105039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30641050c0 0x7f306410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.243+0000 7f3069e00640 1 --2- 172.21.15.32:0/2597105039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3064104770 0x7f3064104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.243+0000 7f3069e00640 1 -- 172.21.15.32:0/2597105039 >> 172.21.15.32:0/2597105039 conn(0x7f30640794e0 msgr2=0x7f30640798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.243+0000 7f3069e00640 1 -- 172.21.15.32:0/2597105039 shutdown_connections 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.243+0000 7f3069e00640 1 -- 172.21.15.32:0/2597105039 wait complete. 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.245+0000 7f3069e00640 1 Processor -- start 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.245+0000 7f3069e00640 1 -- start start 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.245+0000 7f3069e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3064104770 0x7f30641024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.245+0000 7f3069e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30641050c0 0x7f30641029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.245+0000 7f3069e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306410dbd0 0x7f3064102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.245+0000 7f3069e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3064113830 con 0x7f306410dbd0 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.245+0000 7f3069e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f30641136b0 con 0x7f30641050c0 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.245+0000 7f3069e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f30641139b0 con 0x7f3064104770 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3063400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306410dbd0 0x7f3064102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3062a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3064104770 0x7f30641024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3063400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306410dbd0 0x7f3064102f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58388/0 (socket says 172.21.15.32:58388) 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3063400640 1 -- 172.21.15.32:0/931706201 learned_addr learned my addr 172.21.15.32:0/931706201 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3062000640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30641050c0 0x7f30641029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3062a00640 1 -- 172.21.15.32:0/931706201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30641050c0 msgr2=0x7f30641029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3062a00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30641050c0 0x7f30641029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3062a00640 1 -- 172.21.15.32:0/931706201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306410dbd0 msgr2=0x7f3064102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3062a00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306410dbd0 0x7f3064102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3062a00640 1 -- 172.21.15.32:0/931706201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30641a8520 con 0x7f3064104770 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3063400640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306410dbd0 0x7f3064102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.246+0000 7f3062a00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3064104770 0x7f30641024b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f305400e9c0 tx=0x7f305400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.247+0000 7f3068a00640 1 -- 172.21.15.32:0/931706201 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305400ccc0 con 0x7f3064104770 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.247+0000 7f3069e00640 1 -- 172.21.15.32:0/931706201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30641a8810 con 0x7f3064104770 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.247+0000 7f3068a00640 1 -- 172.21.15.32:0/931706201 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30540044d0 con 0x7f3064104770 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.247+0000 7f3068a00640 1 -- 172.21.15.32:0/931706201 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054003e60 con 0x7f3064104770 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.247+0000 7f3069e00640 1 -- 172.21.15.32:0/931706201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30641a8da0 con 0x7f3064104770 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.251+0000 7f3068a00640 1 -- 172.21.15.32:0/931706201 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f30540035f0 con 0x7f3064104770 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.251+0000 7f3070a31740 1 Processor -- start 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.251+0000 7f3070a31740 1 -- start start 2024-08-31T07:33:20.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.251+0000 7f3070a31740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2aea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.251+0000 7f3070a31740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2aeadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.251+0000 7f3070a31740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2aeb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3070a31740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f1f2a61770 con 0x55f1f2724390 2024-08-31T07:33:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3070a31740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f1f2a50d00 con 0x55f1f271c1b0 2024-08-31T07:33:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3070a31740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f1f2a54cb0 con 0x55f1f2745070 2024-08-31T07:33:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3063400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2aea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3062a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2aeadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3063400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2aea870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46588/0 (socket says 172.21.15.32:46588) 2024-08-31T07:33:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3063400640 1 -- 172.21.15.32:0/4165137907 learned_addr learned my addr 172.21.15.32:0/4165137907 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3062000640 1 --2- 172.21.15.32:0/4165137907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2aeb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3063400640 1 -- 172.21.15.32:0/4165137907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 msgr2=0x55f1f2aeadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3063400640 1 --2- 172.21.15.32:0/4165137907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2aeadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3063400640 1 -- 172.21.15.32:0/4165137907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 msgr2=0x55f1f2aeb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3063400640 1 --2- 172.21.15.32:0/4165137907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2aeb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.252+0000 7f3063400640 1 -- 172.21.15.32:0/4165137907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f27b0650 con 0x55f1f2745070 2024-08-31T07:33:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.253+0000 7f3068a00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30480b4d20 0x7f30480b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.253+0000 7f3063400640 1 --2- 172.21.15.32:0/4165137907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2aea870 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f305c0094b0 tx=0x7f305c02f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c5eff71d0621db server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.253+0000 7f3068a00640 1 -- 172.21.15.32:0/931706201 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8441+0+0 (secure 0 0 0) 0x7f30540e5910 con 0x7f3064104770 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.253+0000 7f3062a00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30480b4d20 0x7f30480b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.253+0000 7f304fe00640 1 -- 172.21.15.32:0/4165137907 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c031450 con 0x55f1f2745070 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.253+0000 7f304fe00640 1 -- 172.21.15.32:0/4165137907 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f305c036040 con 0x55f1f2745070 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.253+0000 7f304fe00640 1 -- 172.21.15.32:0/4165137907 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c004500 con 0x55f1f2745070 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.254+0000 7f3070a31740 1 -- 172.21.15.32:0/4165137907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 msgr2=0x55f1f2aea870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.254+0000 7f3062a00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30480b4d20 0x7f30480b7230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55f1f2ae94f0 tx=0x7f30540eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.254+0000 7f3070a31740 1 --2- 172.21.15.32:0/4165137907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2aea870 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f305c0094b0 tx=0x7f305c02f310 comp rx=0 tx=0).stop 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.254+0000 7f3070a31740 1 -- 172.21.15.32:0/4165137907 shutdown_connections 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.254+0000 7f3070a31740 1 --2- 172.21.15.32:0/4165137907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2aeb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.254+0000 7f3070a31740 1 --2- 172.21.15.32:0/4165137907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2aeadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.254+0000 7f3070a31740 1 --2- 172.21.15.32:0/4165137907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2aea870 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.254+0000 7f3070a31740 1 -- 172.21.15.32:0/4165137907 >> 172.21.15.32:0/4165137907 conn(0x55f1f2739460 msgr2=0x55f1f2ae8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.254+0000 7f3070a31740 1 -- 172.21.15.32:0/4165137907 shutdown_connections 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.254+0000 7f3070a31740 1 -- 172.21.15.32:0/4165137907 wait complete. 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.255+0000 7f3070a31740 1 -- start start 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.255+0000 7f3070a31740 1 Processor -- start 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.255+0000 7f3070a31740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f271c1b0 0x55f1f2af6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.255+0000 7f3070a31740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f2724390 0x55f1f2af69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3070a31740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b02160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3070a31740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f1f28b3700 con 0x55f1f271c1b0 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3070a31740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f1f29d7cb0 con 0x55f1f2724390 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3070a31740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f1f2af8e70 con 0x55f1f2745070 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3070a31740 10 client.0 _ll_register_callbacks cb 0x55f1f29fd430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3062000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f2724390 0x55f1f2af69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3063400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f271c1b0 0x55f1f2af6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3062000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f2724390 0x55f1f2af69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:40456/0 (socket says 172.21.15.32:40456) 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3062000640 1 -- 172.21.15.32:0/2396154075 learned_addr learned my addr 172.21.15.32:0/2396154075 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3063400640 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b02160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3062000640 1 -- 172.21.15.32:0/2396154075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 msgr2=0x55f1f2b02160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3062000640 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b02160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3062000640 1 -- 172.21.15.32:0/2396154075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f271c1b0 msgr2=0x55f1f2af6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3062000640 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f271c1b0 0x55f1f2af6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.256+0000 7f3062000640 1 -- 172.21.15.32:0/2396154075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f2a18870 con 0x55f1f2724390 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.257+0000 7f3062000640 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f2724390 0x55f1f2af69e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f30500126e0 tx=0x7f3050012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78dfb4f16a43a68a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.257+0000 7f304cc00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.257+0000 7f304cc00640 1 -- 172.21.15.32:0/2396154075 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050012190 con 0x55f1f2724390 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.257+0000 7f3070a31740 1 -- 172.21.15.32:0/2396154075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f305c036040 con 0x55f1f2724390 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.257+0000 7f304cc00640 1 -- 172.21.15.32:0/2396154075 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8441+0+0 (secure 0 0 0) 0x7f305000f920 con 0x55f1f2724390 2024-08-31T07:33:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.257+0000 7f304cc00640 1 -- 172.21.15.32:0/2396154075 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f305000fb20 con 0x55f1f2724390 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.257+0000 7f3070a31740 10 client.29432 fetch_fsmap learned FSMap version 74 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.257+0000 7f3070a31740 1 -- 172.21.15.32:0/2396154075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55f1f2b04f80 con 0x55f1f2724390 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.257+0000 7f304cc00640 1 -- 172.21.15.32:0/2396154075 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f305c036040 con 0x55f1f2724390 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.258+0000 7f304cc00640 1 -- 172.21.15.32:0/2396154075 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8394+0+0 (secure 0 0 0) 0x7f3050010c90 con 0x55f1f2724390 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.258+0000 7f3070a31740 10 client.29432 fetch_fsmap finished waiting for FSMap version 74 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.258+0000 7f3070a31740 10 client.29432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.258+0000 7f3070a31740 20 client.29432 populate_metadata read hostname 'smithi032' 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.258+0000 7f3070a31740 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f1f27039f0 0x55f1f2b08520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.258+0000 7f3070a31740 4 client.29432 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.258+0000 7f3070a31740 1 -- 172.21.15.32:0/2396154075 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1f2a636c0 con 0x55f1f27039f0 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.258+0000 7f3063400640 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f1f27039f0 0x55f1f2b08520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.259+0000 7f3063400640 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f1f27039f0 0x55f1f2b08520 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53a011fb15acc66 server_cookie=ca643e474cdbb59d in_seq=0 out_seq=0 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.259+0000 7f304cc00640 10 client.29432 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f304cc00640 1 -- 172.21.15.32:0/2396154075 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1f2a636c0 con 0x55f1f27039f0 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f304cc00640 10 client.29432 handle_command_reply: tid=0 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 client.29432 shutdown 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 -- 172.21.15.32:0/2396154075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f2724390 msgr2=0x55f1f2af69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f2724390 0x55f1f2af69e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f30500126e0 tx=0x7f3050012ba0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 -- 172.21.15.32:0/2396154075 shutdown_connections 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f1f27039f0 0x55f1f2b08520 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b02160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f2724390 0x55f1f2af69e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 --2- 172.21.15.32:0/2396154075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f271c1b0 0x55f1f2af6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 -- 172.21.15.32:0/2396154075 >> 172.21.15.32:0/2396154075 conn(0x55f1f2739460 msgr2=0x55f1f2aeda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f304cc00640 0 client.29432 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 -- 172.21.15.32:0/2396154075 shutdown_connections 2024-08-31T07:33:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 1 -- 172.21.15.32:0/2396154075 wait complete. 2024-08-31T07:33:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.260+0000 7f3070a31740 20 client.29432 trim_cache size 0 max 16384 2024-08-31T07:33:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3070a31740 1 Processor -- start 2024-08-31T07:33:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3070a31740 1 -- start start 2024-08-31T07:33:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3070a31740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2aebfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3070a31740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2b02060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3070a31740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b025a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3070a31740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f30641b1580 con 0x55f1f2724390 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3070a31740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f30641b3240 con 0x55f1f271c1b0 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3070a31740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f30641b1c50 con 0x55f1f2745070 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3062000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2b02060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3063400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3063400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:46602/0 (socket says 172.21.15.32:46602) 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3063400640 1 -- 172.21.15.32:0/1197562021 learned_addr learned my addr 172.21.15.32:0/1197562021 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.279+0000 7f3063400640 1 --2- 172.21.15.32:0/1197562021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2aebfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.280+0000 7f3063400640 1 -- 172.21.15.32:0/1197562021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 msgr2=0x55f1f2aebfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.280+0000 7f3063400640 1 --2- 172.21.15.32:0/1197562021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2aebfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.280+0000 7f3063400640 1 -- 172.21.15.32:0/1197562021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 msgr2=0x55f1f2b02060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.280+0000 7f3063400640 1 --2- 172.21.15.32:0/1197562021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2b02060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.280+0000 7f3063400640 1 -- 172.21.15.32:0/1197562021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f2af7520 con 0x55f1f2745070 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.280+0000 7f3063400640 1 --2- 172.21.15.32:0/1197562021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b025a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f305c004150 tx=0x7f305c0040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23032dc2bf5668ef server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.280+0000 7f304f400640 1 -- 172.21.15.32:0/1197562021 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c0315c0 con 0x55f1f2745070 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.280+0000 7f304f400640 1 -- 172.21.15.32:0/1197562021 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f305c036070 con 0x55f1f2745070 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.280+0000 7f304f400640 1 -- 172.21.15.32:0/1197562021 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c004470 con 0x55f1f2745070 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.281+0000 7f3070a31740 1 -- 172.21.15.32:0/1197562021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 msgr2=0x55f1f2b025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.281+0000 7f3070a31740 1 --2- 172.21.15.32:0/1197562021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b025a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f305c004150 tx=0x7f305c0040e0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.281+0000 7f3070a31740 1 -- 172.21.15.32:0/1197562021 shutdown_connections 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.281+0000 7f3070a31740 1 --2- 172.21.15.32:0/1197562021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b025a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.281+0000 7f3070a31740 1 --2- 172.21.15.32:0/1197562021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2b02060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.281+0000 7f3070a31740 1 --2- 172.21.15.32:0/1197562021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2aebfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.281+0000 7f3070a31740 1 -- 172.21.15.32:0/1197562021 >> 172.21.15.32:0/1197562021 conn(0x55f1f2739460 msgr2=0x55f1f2aeb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.281+0000 7f3070a31740 1 -- 172.21.15.32:0/1197562021 shutdown_connections 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.281+0000 7f3070a31740 1 -- 172.21.15.32:0/1197562021 wait complete. 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.282+0000 7f3070a31740 1 -- start start 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.282+0000 7f3070a31740 1 Processor -- start 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3070a31740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2afa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3070a31740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2afa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3070a31740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b71f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3070a31740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f30641b2ae0 con 0x55f1f2724390 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3070a31740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f30641b23e0 con 0x55f1f271c1b0 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3070a31740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f30641af580 con 0x55f1f2745070 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3070a31740 10 client.0 _ll_register_callbacks cb 0x7f3050009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:33:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3063400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2afa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3063400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2afa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:40468/0 (socket says 172.21.15.32:40468) 2024-08-31T07:33:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3063400640 1 -- 172.21.15.32:0/365473654 learned_addr learned my addr 172.21.15.32:0/365473654 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3062000640 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2afa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.283+0000 7f3063400640 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b71f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.284+0000 7f3062000640 1 -- 172.21.15.32:0/365473654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 msgr2=0x55f1f2b71f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.284+0000 7f3062000640 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b71f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.284+0000 7f3062000640 1 -- 172.21.15.32:0/365473654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 msgr2=0x55f1f2afa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.284+0000 7f3062000640 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2afa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.284+0000 7f3062000640 1 -- 172.21.15.32:0/365473654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f2afb1d0 con 0x55f1f2724390 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.284+0000 7f3063400640 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2afa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.284+0000 7f3062000640 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2afa900 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3050009e00 tx=0x7f30500123c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87974aad9cf4b547 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.284+0000 7f304cc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.286+0000 7f304cc00640 1 -- 172.21.15.32:0/365473654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050029070 con 0x55f1f2724390 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.286+0000 7f304cc00640 1 -- 172.21.15.32:0/365473654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8441+0+0 (secure 0 0 0) 0x7f3050005390 con 0x55f1f2724390 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.286+0000 7f3070a31740 1 -- 172.21.15.32:0/365473654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f305c036070 con 0x55f1f2724390 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f304cc00640 1 -- 172.21.15.32:0/365473654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f3050035050 con 0x55f1f2724390 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f3070a31740 10 client.19383 fetch_fsmap learned FSMap version 75 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f3070a31740 1 -- 172.21.15.32:0/365473654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55f1f2b734d0 con 0x55f1f2724390 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f304cc00640 1 -- 172.21.15.32:0/365473654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f305c036070 con 0x55f1f2724390 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f304cc00640 1 -- 172.21.15.32:0/365473654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 75) ==== 8398+0+0 (secure 0 0 0) 0x7f30500035f0 con 0x55f1f2724390 2024-08-31T07:33:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f3070a31740 10 client.19383 fetch_fsmap finished waiting for FSMap version 75 2024-08-31T07:33:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f3070a31740 10 client.19383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:33:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f3070a31740 20 client.19383 populate_metadata read hostname 'smithi032' 2024-08-31T07:33:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f3070a31740 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f1f26d8da0 0x55f1f2b75d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f3070a31740 4 client.19383 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:33:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.287+0000 7f3070a31740 1 -- 172.21.15.32:0/365473654 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1f2a5f160 con 0x55f1f26d8da0 2024-08-31T07:33:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.288+0000 7f3063400640 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f1f26d8da0 0x55f1f2b75d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.288+0000 7f3063400640 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f1f26d8da0 0x55f1f2b75d50 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7497065cfbeb2db8 server_cookie=ca1579be67850c58 in_seq=0 out_seq=0 2024-08-31T07:33:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.288+0000 7f304cc00640 10 client.19383 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f304cc00640 1 -- 172.21.15.32:0/365473654 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6818 (crc 0 0 0) 0x55f1f2a5f160 con 0x55f1f26d8da0 2024-08-31T07:33:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f304cc00640 10 client.19383 handle_command_reply: tid=0 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "5476ca92-3649-4ebb-868c-09cfdc2c594c", 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:33:20.318 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:32:41.165270+0000", 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.319 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.320 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.321 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:33:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.324 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:33:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:33:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.325 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:33:20.326 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.327 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:33:20.328 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.329 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.330 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:33:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:33:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.331 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.332 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.332 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:33:20.332 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:33:20.332 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:33:20.333 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:33:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:20.334 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:33:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:33:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:20.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:33:20.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:33:20.336 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f3070a31740 1 client.19383 shutdown 2024-08-31T07:33:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f3070a31740 1 -- 172.21.15.32:0/365473654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 msgr2=0x55f1f2afa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f3070a31740 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2afa900 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3050009e00 tx=0x7f30500123c0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f3070a31740 1 -- 172.21.15.32:0/365473654 shutdown_connections 2024-08-31T07:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f3070a31740 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f1f26d8da0 0x55f1f2b75d50 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f3070a31740 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f1f2745070 0x55f1f2b71f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f3070a31740 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f1f2724390 0x55f1f2afa900 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f3070a31740 1 --2- 172.21.15.32:0/365473654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1f271c1b0 0x55f1f2afa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.289+0000 7f3070a31740 1 -- 172.21.15.32:0/365473654 >> 172.21.15.32:0/365473654 conn(0x55f1f2739460 msgr2=0x55f1f2ae6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.290+0000 7f304cc00640 0 client.19383 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.290+0000 7f3070a31740 1 -- 172.21.15.32:0/365473654 shutdown_connections 2024-08-31T07:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.290+0000 7f3070a31740 1 -- 172.21.15.32:0/365473654 wait complete. 2024-08-31T07:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.290+0000 7f3070a31740 20 client.19383 trim_cache size 0 max 16384 2024-08-31T07:33:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 -- 172.21.15.32:0/931706201 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30480b4d20 msgr2=0x7f30480b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:20.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30480b4d20 0x7f30480b7230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55f1f2ae94f0 tx=0x7f30540eb530 comp rx=0 tx=0).stop 2024-08-31T07:33:20.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 -- 172.21.15.32:0/931706201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3064104770 msgr2=0x7f30641024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:20.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3064104770 0x7f30641024b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f305400e9c0 tx=0x7f305400ee80 comp rx=0 tx=0).stop 2024-08-31T07:33:20.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 -- 172.21.15.32:0/931706201 shutdown_connections 2024-08-31T07:33:20.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f30480b4d20 0x7f30480b7230 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f306410dbd0 0x7f3064102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30641050c0 0x7f30641029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 --2- 172.21.15.32:0/931706201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3064104770 0x7f30641024b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:20.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 -- 172.21.15.32:0/931706201 >> 172.21.15.32:0/931706201 conn(0x7f30640794e0 msgr2=0x7f30640ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:20.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 -- 172.21.15.32:0/931706201 shutdown_connections 2024-08-31T07:33:20.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:20.291+0000 7f3069e00640 1 -- 172.21.15.32:0/931706201 wait complete. 2024-08-31T07:33:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:20 smithi032 ceph-mon[17252]: pgmap v1015: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.3 MiB/s wr, 572 op/s 2024-08-31T07:33:20.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127359 13.89 MB/sec execute 300 sec latency 2319.770 ms 2024-08-31T07:33:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:20 smithi073 ceph-mon[21995]: pgmap v1015: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.3 MiB/s wr, 572 op/s 2024-08-31T07:33:20.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:20 smithi084 ceph-mon[22387]: pgmap v1015: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.3 MiB/s wr, 572 op/s 2024-08-31T07:33:21.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127359 13.84 MB/sec execute 301 sec latency 3319.813 ms 2024-08-31T07:33:21.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:21 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:21.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:21 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:33:21.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:21 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:21.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:21 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:33:21.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:21 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:21.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:21 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:33:22.413 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127384 13.81 MB/sec execute 302 sec latency 4319.852 ms 2024-08-31T07:33:22.566 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:22 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:33:22.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:22 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:33:22.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:22 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:33:22.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:22 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:33:22.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:22 smithi032 ceph-mon[17252]: pgmap v1016: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 12 MiB/s wr, 628 op/s 2024-08-31T07:33:22.613 INFO:tasks.cephfs.filesystem:scrub status for tag:5476ca92-3649-4ebb-868c-09cfdc2c594c - {'path': '/', 'tag': '5476ca92-3649-4ebb-868c-09cfdc2c594c', 'options': 'recursive,force'} 2024-08-31T07:33:22.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:22 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:33:22.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:22 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:33:22.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:22 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:33:22.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:22 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:33:22.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:22 smithi084 ceph-mon[22387]: pgmap v1016: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 12 MiB/s wr, 628 op/s 2024-08-31T07:33:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:22 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:33:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:22 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:33:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:22 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:33:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:22 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:33:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:22 smithi073 ceph-mon[21995]: pgmap v1016: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 12 MiB/s wr, 628 op/s 2024-08-31T07:33:23.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127588 13.80 MB/sec execute 303 sec latency 4929.211 ms 2024-08-31T07:33:24.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127589 13.75 MB/sec execute 304 sec latency 1218.191 ms 2024-08-31T07:33:24.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:24 smithi032 ceph-mon[17252]: pgmap v1017: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 588 op/s 2024-08-31T07:33:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:24 smithi073 ceph-mon[21995]: pgmap v1017: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 588 op/s 2024-08-31T07:33:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:24 smithi084 ceph-mon[22387]: pgmap v1017: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 588 op/s 2024-08-31T07:33:25.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127589 13.71 MB/sec execute 305 sec latency 2218.258 ms 2024-08-31T07:33:26.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127589 13.66 MB/sec execute 306 sec latency 3218.322 ms 2024-08-31T07:33:26.517 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:26 smithi032 ceph-mon[17252]: pgmap v1018: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 537 op/s 2024-08-31T07:33:26.517 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:26 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:26 smithi073 ceph-mon[21995]: pgmap v1018: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 537 op/s 2024-08-31T07:33:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:26 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:26 smithi084 ceph-mon[22387]: pgmap v1018: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 537 op/s 2024-08-31T07:33:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:26 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:27.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127589 13.62 MB/sec execute 307 sec latency 4218.388 ms 2024-08-31T07:33:28.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127589 13.57 MB/sec execute 308 sec latency 5218.449 ms 2024-08-31T07:33:28.490 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:28 smithi032 ceph-mon[17252]: pgmap v1019: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 422 op/s 2024-08-31T07:33:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:28 smithi073 ceph-mon[21995]: pgmap v1019: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 422 op/s 2024-08-31T07:33:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:28 smithi084 ceph-mon[22387]: pgmap v1019: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 422 op/s 2024-08-31T07:33:29.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127592 13.53 MB/sec execute 309 sec latency 6218.500 ms 2024-08-31T07:33:30.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127592 13.49 MB/sec execute 310 sec latency 7218.553 ms 2024-08-31T07:33:30.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:30 smithi032 ceph-mon[17252]: pgmap v1020: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.1 MiB/s wr, 325 op/s 2024-08-31T07:33:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:30 smithi073 ceph-mon[21995]: pgmap v1020: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.1 MiB/s wr, 325 op/s 2024-08-31T07:33:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:30 smithi084 ceph-mon[22387]: pgmap v1020: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.1 MiB/s wr, 325 op/s 2024-08-31T07:33:31.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127597 13.44 MB/sec execute 311 sec latency 8218.606 ms 2024-08-31T07:33:32.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127597 13.40 MB/sec execute 312 sec latency 9198.904 ms 2024-08-31T07:33:32.515 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:32 smithi032 ceph-mon[17252]: pgmap v1021: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.3 MiB/s wr, 296 op/s 2024-08-31T07:33:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:32 smithi073 ceph-mon[21995]: pgmap v1021: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.3 MiB/s wr, 296 op/s 2024-08-31T07:33:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:32 smithi084 ceph-mon[22387]: pgmap v1021: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.3 MiB/s wr, 296 op/s 2024-08-31T07:33:33.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127811 13.39 MB/sec execute 313 sec latency 10173.184 ms 2024-08-31T07:33:34.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 128023 13.37 MB/sec execute 314 sec latency 6940.219 ms 2024-08-31T07:33:34.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:34 smithi032 ceph-mon[17252]: pgmap v1022: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 202 op/s 2024-08-31T07:33:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:34 smithi073 ceph-mon[21995]: pgmap v1022: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 202 op/s 2024-08-31T07:33:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:34 smithi084 ceph-mon[22387]: pgmap v1022: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 202 op/s 2024-08-31T07:33:35.415 INFO:tasks.workunit.client.0.smithi032.stdout: 10 128605 13.42 MB/sec execute 315 sec latency 7520.951 ms 2024-08-31T07:33:36.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 129522 13.53 MB/sec execute 316 sec latency 156.051 ms 2024-08-31T07:33:36.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:36 smithi032 ceph-mon[17252]: pgmap v1023: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 197 op/s 2024-08-31T07:33:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:36 smithi073 ceph-mon[21995]: pgmap v1023: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 197 op/s 2024-08-31T07:33:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:36 smithi084 ceph-mon[22387]: pgmap v1023: 97 pgs: 97 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 197 op/s 2024-08-31T07:33:37.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 130378 13.68 MB/sec execute 317 sec latency 188.112 ms 2024-08-31T07:33:38.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 131049 13.75 MB/sec execute 318 sec latency 91.770 ms 2024-08-31T07:33:38.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:38 smithi032 ceph-mon[17252]: pgmap v1024: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-08-31T07:33:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:38 smithi073 ceph-mon[21995]: pgmap v1024: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-08-31T07:33:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:38 smithi084 ceph-mon[22387]: pgmap v1024: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-08-31T07:33:39.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 131602 13.78 MB/sec execute 319 sec latency 157.180 ms 2024-08-31T07:33:40.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 132293 13.84 MB/sec execute 320 sec latency 260.218 ms 2024-08-31T07:33:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:40 smithi073 ceph-mon[21995]: pgmap v1025: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 218 op/s 2024-08-31T07:33:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:40 smithi084 ceph-mon[22387]: pgmap v1025: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 218 op/s 2024-08-31T07:33:40.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:40 smithi032 ceph-mon[17252]: pgmap v1025: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 218 op/s 2024-08-31T07:33:41.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 133178 13.95 MB/sec execute 321 sec latency 113.322 ms 2024-08-31T07:33:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:41 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:41 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:41.743 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:41 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:42.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 134080 14.07 MB/sec execute 322 sec latency 80.519 ms 2024-08-31T07:33:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:42 smithi073 ceph-mon[21995]: pgmap v1026: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-08-31T07:33:42.708 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:42 smithi032 ceph-mon[17252]: pgmap v1026: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-08-31T07:33:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:42 smithi084 ceph-mon[22387]: pgmap v1026: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-08-31T07:33:43.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 134617 14.12 MB/sec execute 323 sec latency 230.416 ms 2024-08-31T07:33:44.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 135395 14.19 MB/sec execute 324 sec latency 76.553 ms 2024-08-31T07:33:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:44 smithi073 ceph-mon[21995]: pgmap v1027: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.8 MiB/s wr, 385 op/s 2024-08-31T07:33:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:44 smithi084 ceph-mon[22387]: pgmap v1027: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.8 MiB/s wr, 385 op/s 2024-08-31T07:33:44.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:44 smithi032 ceph-mon[17252]: pgmap v1027: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.8 MiB/s wr, 385 op/s 2024-08-31T07:33:45.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 136140 14.28 MB/sec execute 325 sec latency 99.225 ms 2024-08-31T07:33:46.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 136841 14.33 MB/sec execute 326 sec latency 226.171 ms 2024-08-31T07:33:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:46 smithi032 ceph-mon[17252]: pgmap v1028: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 475 op/s 2024-08-31T07:33:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:46 smithi073 ceph-mon[21995]: pgmap v1028: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 475 op/s 2024-08-31T07:33:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:46 smithi084 ceph-mon[22387]: pgmap v1028: 97 pgs: 97 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 475 op/s 2024-08-31T07:33:47.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 137581 14.42 MB/sec execute 327 sec latency 232.920 ms 2024-08-31T07:33:48.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 138367 14.52 MB/sec execute 328 sec latency 105.932 ms 2024-08-31T07:33:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:48 smithi032 ceph-mon[17252]: pgmap v1029: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.7 MiB/s wr, 607 op/s 2024-08-31T07:33:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:48 smithi073 ceph-mon[21995]: pgmap v1029: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.7 MiB/s wr, 607 op/s 2024-08-31T07:33:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:48 smithi084 ceph-mon[22387]: pgmap v1029: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.7 MiB/s wr, 607 op/s 2024-08-31T07:33:49.416 INFO:tasks.workunit.client.0.smithi032.stdout: 10 139035 14.58 MB/sec execute 329 sec latency 83.938 ms 2024-08-31T07:33:50.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 139719 14.64 MB/sec execute 330 sec latency 237.428 ms 2024-08-31T07:33:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:50 smithi073 ceph-mon[21995]: pgmap v1030: 97 pgs: 97 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.8 MiB/s wr, 534 op/s 2024-08-31T07:33:50.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:50 smithi084 ceph-mon[22387]: pgmap v1030: 97 pgs: 97 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.8 MiB/s wr, 534 op/s 2024-08-31T07:33:50.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:50 smithi032 ceph-mon[17252]: pgmap v1030: 97 pgs: 97 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.8 MiB/s wr, 534 op/s 2024-08-31T07:33:51.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 140428 14.72 MB/sec execute 331 sec latency 165.771 ms 2024-08-31T07:33:52.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 141107 14.77 MB/sec execute 332 sec latency 177.958 ms 2024-08-31T07:33:52.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:33:52.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:52 smithi032 ceph-mon[17252]: pgmap v1031: 97 pgs: 97 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.9 MiB/s wr, 606 op/s 2024-08-31T07:33:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:52 smithi073 ceph-mon[21995]: pgmap v1031: 97 pgs: 97 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.9 MiB/s wr, 606 op/s 2024-08-31T07:33:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:52 smithi084 ceph-mon[22387]: pgmap v1031: 97 pgs: 97 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.9 MiB/s wr, 606 op/s 2024-08-31T07:33:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:33:53.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 141661 14.81 MB/sec execute 333 sec latency 449.959 ms 2024-08-31T07:33:54.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142090 14.83 MB/sec execute 334 sec latency 484.921 ms 2024-08-31T07:33:54.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:54 smithi032 ceph-mon[17252]: pgmap v1032: 97 pgs: 97 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.8 MiB/s wr, 593 op/s 2024-08-31T07:33:54.524 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:54 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:33:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:54 smithi073 ceph-mon[21995]: pgmap v1032: 97 pgs: 97 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.8 MiB/s wr, 593 op/s 2024-08-31T07:33:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:54 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:33:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:54 smithi084 ceph-mon[22387]: pgmap v1032: 97 pgs: 97 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.8 MiB/s wr, 593 op/s 2024-08-31T07:33:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:54 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:33:55.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142091 14.79 MB/sec execute 335 sec latency 1468.227 ms 2024-08-31T07:33:56.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142092 14.74 MB/sec execute 336 sec latency 2468.290 ms 2024-08-31T07:33:56.467 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:56 smithi032 ceph-mon[17252]: pgmap v1033: 97 pgs: 97 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 606 op/s 2024-08-31T07:33:56.467 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:56 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:56 smithi084 ceph-mon[22387]: pgmap v1033: 97 pgs: 97 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 606 op/s 2024-08-31T07:33:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:56 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:56 smithi073 ceph-mon[21995]: pgmap v1033: 97 pgs: 97 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 606 op/s 2024-08-31T07:33:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:56 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:33:57.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142147 14.72 MB/sec execute 337 sec latency 3319.925 ms 2024-08-31T07:33:57.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:33:57.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:33:57.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:33:57.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:33:57.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:33:57.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:33:57.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:33:57.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:33:57.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:33:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.899+0000 7f8c36200640 1 -- 172.21.15.32:0/1236818950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c3010e140 msgr2=0x7f8c3010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.899+0000 7f8c36200640 1 --2- 172.21.15.32:0/1236818950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c3010e140 0x7f8c3010e550 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009900 tx=0x7f8c1c02a410 comp rx=0 tx=0).stop 2024-08-31T07:33:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.900+0000 7f8c36200640 1 -- 172.21.15.32:0/1236818950 shutdown_connections 2024-08-31T07:33:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.900+0000 7f8c36200640 1 --2- 172.21.15.32:0/1236818950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c301143d0 0x7f8c3011ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.900+0000 7f8c36200640 1 --2- 172.21.15.32:0/1236818950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c30113a80 0x7f8c30113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.900+0000 7f8c36200640 1 --2- 172.21.15.32:0/1236818950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c3010e140 0x7f8c3010e550 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.900+0000 7f8c36200640 1 -- 172.21.15.32:0/1236818950 >> 172.21.15.32:0/1236818950 conn(0x7f8c3006e830 msgr2=0x7f8c3010c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.901+0000 7f8c36200640 1 -- 172.21.15.32:0/1236818950 shutdown_connections 2024-08-31T07:33:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.901+0000 7f8c36200640 1 -- 172.21.15.32:0/1236818950 wait complete. 2024-08-31T07:33:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.902+0000 7f8c36200640 1 Processor -- start 2024-08-31T07:33:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.903+0000 7f8c36200640 1 -- start start 2024-08-31T07:33:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c36200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c3010e140 0x7f8c30118040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c36200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c30113a80 0x7f8c30118580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c36200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c301143d0 0x7f8c30118ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c34e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c3010e140 0x7f8c30118040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c34e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c3010e140 0x7f8c30118040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43608/0 (socket says 172.21.15.32:43608) 2024-08-31T07:33:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c35800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c301143d0 0x7f8c30118ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c35800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c301143d0 0x7f8c30118ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53538/0 (socket says 172.21.15.32:53538) 2024-08-31T07:33:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c2fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c30113a80 0x7f8c30118580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c2fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c30113a80 0x7f8c30118580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51778/0 (socket says 172.21.15.32:51778) 2024-08-31T07:33:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c36200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3011e0d0 con 0x7f8c3010e140 2024-08-31T07:33:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c36200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3011df50 con 0x7f8c301143d0 2024-08-31T07:33:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c36200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3011e250 con 0x7f8c30113a80 2024-08-31T07:33:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.904+0000 7f8c34e00640 1 -- 172.21.15.32:0/4080249317 learned_addr learned my addr 172.21.15.32:0/4080249317 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.905+0000 7f8c2fe00640 1 -- 172.21.15.32:0/4080249317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c301143d0 msgr2=0x7f8c30118ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.905+0000 7f8c2fe00640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c301143d0 0x7f8c30118ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.905+0000 7f8c2fe00640 1 -- 172.21.15.32:0/4080249317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c3010e140 msgr2=0x7f8c30118040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.905+0000 7f8c2fe00640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c3010e140 0x7f8c30118040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.905+0000 7f8c2fe00640 1 -- 172.21.15.32:0/4080249317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c301aa720 con 0x7f8c30113a80 2024-08-31T07:33:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.906+0000 7f8c34e00640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c3010e140 0x7f8c30118040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:33:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.906+0000 7f8c2fe00640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c30113a80 0x7f8c30118580 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8c30074ac0 tx=0x7f8c2400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.906+0000 7f8c2d600640 1 -- 172.21.15.32:0/4080249317 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2400cc40 con 0x7f8c30113a80 2024-08-31T07:33:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.906+0000 7f8c36200640 1 -- 172.21.15.32:0/4080249317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c301aaa10 con 0x7f8c30113a80 2024-08-31T07:33:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.906+0000 7f8c36200640 1 -- 172.21.15.32:0/4080249317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c301aafa0 con 0x7f8c30113a80 2024-08-31T07:33:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.906+0000 7f8c2d600640 1 -- 172.21.15.32:0/4080249317 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c2400cde0 con 0x7f8c30113a80 2024-08-31T07:33:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.906+0000 7f8c2d600640 1 -- 172.21.15.32:0/4080249317 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c240187a0 con 0x7f8c30113a80 2024-08-31T07:33:57.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.911+0000 7f8c3e3ef740 1 Processor -- start 2024-08-31T07:33:57.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.911+0000 7f8c3e3ef740 1 -- start start 2024-08-31T07:33:57.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 0x5654a95e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a92141b0 0x5654a95e3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a921c390 0x5654a95e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5654a9559770 con 0x5654a92141b0 2024-08-31T07:33:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5654a9548d00 con 0x5654a921c390 2024-08-31T07:33:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5654a954ccb0 con 0x5654a923d070 2024-08-31T07:33:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c2d600640 1 -- 172.21.15.32:0/4080249317 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8c24018940 con 0x7f8c30113a80 2024-08-31T07:33:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c35800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 0x5654a95e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c35800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 0x5654a95e2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51792/0 (socket says 172.21.15.32:51792) 2024-08-31T07:33:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c35800640 1 -- 172.21.15.32:0/3716544922 learned_addr learned my addr 172.21.15.32:0/3716544922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c2fe00640 1 --2- 172.21.15.32:0/3716544922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a921c390 0x5654a95e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.912+0000 7f8c34e00640 1 --2- 172.21.15.32:0/3716544922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a92141b0 0x5654a95e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.913+0000 7f8c35800640 1 -- 172.21.15.32:0/3716544922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a921c390 msgr2=0x5654a95e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.913+0000 7f8c35800640 1 --2- 172.21.15.32:0/3716544922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a921c390 0x5654a95e35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.913+0000 7f8c35800640 1 -- 172.21.15.32:0/3716544922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a92141b0 msgr2=0x5654a95e3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.913+0000 7f8c35800640 1 --2- 172.21.15.32:0/3716544922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a92141b0 0x5654a95e3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.913+0000 7f8c35800640 1 -- 172.21.15.32:0/3716544922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a92a8650 con 0x5654a923d070 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.913+0000 7f8c34e00640 1 --2- 172.21.15.32:0/3716544922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a92141b0 0x5654a95e3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.913+0000 7f8c35800640 1 --2- 172.21.15.32:0/3716544922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 0x5654a95e2b20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009fd0 tx=0x7f8c280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6180cafa171a4686 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.914+0000 7f8c22000640 1 -- 172.21.15.32:0/3716544922 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28004d20 con 0x5654a923d070 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.914+0000 7f8c22000640 1 -- 172.21.15.32:0/3716544922 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c28004050 con 0x5654a923d070 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.914+0000 7f8c22000640 1 -- 172.21.15.32:0/3716544922 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28005280 con 0x5654a923d070 2024-08-31T07:33:57.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.914+0000 7f8c2d600640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8c180b4d20 0x7f8c180b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.914+0000 7f8c2d600640 1 -- 172.21.15.32:0/4080249317 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8441+0+0 (secure 0 0 0) 0x7f8c240e5730 con 0x7f8c30113a80 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.914+0000 7f8c34e00640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8c180b4d20 0x7f8c180b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.915+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3716544922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 msgr2=0x5654a95e2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.915+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3716544922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 0x5654a95e2b20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009fd0 tx=0x7f8c280118d0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.915+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3716544922 shutdown_connections 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.915+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3716544922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a921c390 0x5654a95e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.915+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3716544922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a92141b0 0x5654a95e3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.915+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3716544922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 0x5654a95e2b20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.915+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3716544922 >> 172.21.15.32:0/3716544922 conn(0x5654a9231460 msgr2=0x5654a95e0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.915+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3716544922 shutdown_connections 2024-08-31T07:33:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.915+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3716544922 wait complete. 2024-08-31T07:33:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.915+0000 7f8c34e00640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8c180b4d20 0x7f8c180b7230 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0096c0 tx=0x7f8c1c004540 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.916+0000 7f8c3e3ef740 1 -- start start 2024-08-31T07:33:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.916+0000 7f8c3e3ef740 1 Processor -- start 2024-08-31T07:33:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.916+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a92141b0 0x5654a95ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a921c390 0x5654a95eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 0x5654a95fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5654a93ab700 con 0x5654a923d070 2024-08-31T07:33:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5654a94cfcb0 con 0x5654a921c390 2024-08-31T07:33:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5654a95f1300 con 0x5654a92141b0 2024-08-31T07:33:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c3e3ef740 10 client.0 _ll_register_callbacks cb 0x5654a94f5430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:33:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c34e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a92141b0 0x5654a95ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c34e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a92141b0 0x5654a95ee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51806/0 (socket says 172.21.15.32:51806) 2024-08-31T07:33:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c34e00640 1 -- 172.21.15.32:0/3750862519 learned_addr learned my addr 172.21.15.32:0/3750862519 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c35800640 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 0x5654a95fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c34e00640 1 -- 172.21.15.32:0/3750862519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a921c390 msgr2=0x5654a95eebc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:33:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c34e00640 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a921c390 0x5654a95eebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c34e00640 1 -- 172.21.15.32:0/3750862519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 msgr2=0x5654a95fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c34e00640 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 0x5654a95fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.917+0000 7f8c34e00640 1 -- 172.21.15.32:0/3750862519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a9510870 con 0x5654a92141b0 2024-08-31T07:33:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.918+0000 7f8c34e00640 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a92141b0 0x5654a95ee680 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c006010 tx=0x7f8c1c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=755c101b88dc7b13 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.918+0000 7f8c16a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:33:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.918+0000 7f8c16a00640 1 -- 172.21.15.32:0/3750862519 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c04c070 con 0x5654a92141b0 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.918+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3750862519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c28004050 con 0x5654a92141b0 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.918+0000 7f8c16a00640 1 -- 172.21.15.32:0/3750862519 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8441+0+0 (secure 0 0 0) 0x7f8c1c004b90 con 0x5654a92141b0 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.918+0000 7f8c16a00640 1 -- 172.21.15.32:0/3750862519 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f8c1c058050 con 0x5654a92141b0 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.918+0000 7f8c3e3ef740 10 client.29440 fetch_fsmap learned FSMap version 78 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.918+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3750862519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x5654a95fd370 con 0x5654a92141b0 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.919+0000 7f8c16a00640 1 -- 172.21.15.32:0/3750862519 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c28004050 con 0x5654a92141b0 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.919+0000 7f8c16a00640 1 -- 172.21.15.32:0/3750862519 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 78) ==== 8402+0+0 (secure 0 0 0) 0x7f8c1c032540 con 0x5654a92141b0 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.919+0000 7f8c3e3ef740 10 client.29440 fetch_fsmap finished waiting for FSMap version 78 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.919+0000 7f8c3e3ef740 10 client.29440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.919+0000 7f8c3e3ef740 20 client.29440 populate_metadata read hostname 'smithi032' 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.919+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5654a91fb9f0 0x5654a9600950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.919+0000 7f8c3e3ef740 4 client.29440 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:33:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.919+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3750862519 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654a955b6c0 con 0x5654a91fb9f0 2024-08-31T07:33:57.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.932+0000 7f8c2fe00640 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5654a91fb9f0 0x5654a9600950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.967+0000 7f8c2fe00640 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5654a91fb9f0 0x5654a9600950 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96279c5faca5095 server_cookie=e107fc6122785000 in_seq=0 out_seq=0 2024-08-31T07:33:57.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.967+0000 7f8c16a00640 10 client.29440 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:57.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.968+0000 7f8c16a00640 1 -- 172.21.15.32:0/3750862519 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5654a955b6c0 con 0x5654a91fb9f0 2024-08-31T07:33:57.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.968+0000 7f8c16a00640 10 client.29440 handle_command_reply: tid=0 2024-08-31T07:33:57.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.968+0000 7f8c3e3ef740 1 client.29440 shutdown 2024-08-31T07:33:57.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3750862519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a92141b0 msgr2=0x5654a95ee680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:57.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a92141b0 0x5654a95ee680 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c006010 tx=0x7f8c1c009260 comp rx=0 tx=0).stop 2024-08-31T07:33:57.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3750862519 shutdown_connections 2024-08-31T07:33:57.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5654a91fb9f0 0x5654a9600950 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 0x5654a95fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a921c390 0x5654a95eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3750862519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a92141b0 0x5654a95ee680 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c16a00640 0 client.29440 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:57.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3750862519 >> 172.21.15.32:0/3750862519 conn(0x5654a9231460 msgr2=0x5654a95e5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:57.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3750862519 shutdown_connections 2024-08-31T07:33:57.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3750862519 wait complete. 2024-08-31T07:33:57.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.969+0000 7f8c3e3ef740 20 client.29440 trim_cache size 0 max 16384 2024-08-31T07:33:57.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.988+0000 7f8c3e3ef740 1 Processor -- start 2024-08-31T07:33:57.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.988+0000 7f8c3e3ef740 1 -- start start 2024-08-31T07:33:57.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.988+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 0x5654a95fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.988+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a921c390 0x5654a95faac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.988+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 0x5654a95eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.988+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1000e6c0 con 0x5654a923d070 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.988+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8c100103e0 con 0x5654a92141b0 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.988+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1000ee50 con 0x5654a921c390 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c35800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 0x5654a95eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c34e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 0x5654a95fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c35800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 0x5654a95eb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43628/0 (socket says 172.21.15.32:43628) 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c35800640 1 -- 172.21.15.32:0/3960365952 learned_addr learned my addr 172.21.15.32:0/3960365952 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c2fe00640 1 --2- 172.21.15.32:0/3960365952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a921c390 0x5654a95faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c34e00640 1 -- 172.21.15.32:0/3960365952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a921c390 msgr2=0x5654a95faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c34e00640 1 --2- 172.21.15.32:0/3960365952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a921c390 0x5654a95faac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c34e00640 1 -- 172.21.15.32:0/3960365952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 msgr2=0x5654a95eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c34e00640 1 --2- 172.21.15.32:0/3960365952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 0x5654a95eb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c34e00640 1 -- 172.21.15.32:0/3960365952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a95e4e10 con 0x5654a92141b0 2024-08-31T07:33:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.989+0000 7f8c34e00640 1 --2- 172.21.15.32:0/3960365952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 0x5654a95fa580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c032960 tx=0x7f8c1c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fdb49e58c3587d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.990+0000 7f8c21600640 1 -- 172.21.15.32:0/3960365952 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c04c070 con 0x5654a92141b0 2024-08-31T07:33:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.990+0000 7f8c21600640 1 -- 172.21.15.32:0/3960365952 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c1c061070 con 0x5654a92141b0 2024-08-31T07:33:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.990+0000 7f8c21600640 1 -- 172.21.15.32:0/3960365952 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c035050 con 0x5654a92141b0 2024-08-31T07:33:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.991+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3960365952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 msgr2=0x5654a95fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.991+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3960365952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 0x5654a95fa580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c032960 tx=0x7f8c1c004610 comp rx=0 tx=0).stop 2024-08-31T07:33:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.991+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3960365952 shutdown_connections 2024-08-31T07:33:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.991+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3960365952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a923d070 0x5654a95eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.991+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3960365952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a921c390 0x5654a95faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.991+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3960365952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 0x5654a95fa580 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.991+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3960365952 >> 172.21.15.32:0/3960365952 conn(0x5654a9231460 msgr2=0x5654a95ecd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.991+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3960365952 shutdown_connections 2024-08-31T07:33:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.991+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3960365952 wait complete. 2024-08-31T07:33:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.991+0000 7f8c3e3ef740 1 -- start start 2024-08-31T07:33:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.992+0000 7f8c3e3ef740 1 Processor -- start 2024-08-31T07:33:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.992+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 0x5654a95f2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.992+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a921c390 0x5654a95f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.992+0000 7f8c3e3ef740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 0x5654a95f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.992+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1000fc80 con 0x5654a921c390 2024-08-31T07:33:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.992+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1000f580 con 0x5654a92141b0 2024-08-31T07:33:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.992+0000 7f8c3e3ef740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1000c6c0 con 0x5654a923d070 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.992+0000 7f8c3e3ef740 10 client.0 _ll_register_callbacks cb 0x7f8c1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c34e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 0x5654a95f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c2fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a921c390 0x5654a95f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c34e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 0x5654a95f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53572/0 (socket says 172.21.15.32:53572) 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c34e00640 1 -- 172.21.15.32:0/3843896645 learned_addr learned my addr 172.21.15.32:0/3843896645 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c35800640 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 0x5654a95f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c2fe00640 1 -- 172.21.15.32:0/3843896645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 msgr2=0x5654a95f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c2fe00640 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 0x5654a95f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c2fe00640 1 -- 172.21.15.32:0/3843896645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 msgr2=0x5654a95f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c2fe00640 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 0x5654a95f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c2fe00640 1 -- 172.21.15.32:0/3843896645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a966a400 con 0x5654a921c390 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c2fe00640 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a921c390 0x5654a95f2ba0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8c240e2ed0 tx=0x7f8c2400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdce551293307271 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:33:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.993+0000 7f8c16a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:33:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.994+0000 7f8c16a00640 1 -- 172.21.15.32:0/3843896645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c240ab5e0 con 0x5654a921c390 2024-08-31T07:33:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.994+0000 7f8c16a00640 1 -- 172.21.15.32:0/3843896645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8441+0+0 (secure 0 0 0) 0x7f8c240a8460 con 0x5654a921c390 2024-08-31T07:33:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.994+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3843896645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c1c061070 con 0x5654a921c390 2024-08-31T07:33:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.994+0000 7f8c16a00640 1 -- 172.21.15.32:0/3843896645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f8c240ab980 con 0x5654a921c390 2024-08-31T07:33:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.994+0000 7f8c3e3ef740 10 client.19401 fetch_fsmap learned FSMap version 78 2024-08-31T07:33:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.994+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3843896645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x5654a966b6f0 con 0x5654a921c390 2024-08-31T07:33:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.994+0000 7f8c16a00640 1 -- 172.21.15.32:0/3843896645 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c1c061070 con 0x5654a921c390 2024-08-31T07:33:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.995+0000 7f8c16a00640 1 -- 172.21.15.32:0/3843896645 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 78) ==== 8402+0+0 (secure 0 0 0) 0x7f8c240b0310 con 0x5654a921c390 2024-08-31T07:33:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.995+0000 7f8c3e3ef740 10 client.19401 fetch_fsmap finished waiting for FSMap version 78 2024-08-31T07:33:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.995+0000 7f8c3e3ef740 10 client.19401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:33:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.995+0000 7f8c3e3ef740 20 client.19401 populate_metadata read hostname 'smithi032' 2024-08-31T07:33:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.995+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5654a91d0da0 0x5654a966def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:33:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.995+0000 7f8c3e3ef740 4 client.19401 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:33:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.995+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3843896645 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5654a9557160 con 0x5654a91d0da0 2024-08-31T07:33:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.995+0000 7f8c34e00640 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5654a91d0da0 0x5654a966def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:33:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.996+0000 7f8c34e00640 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5654a91d0da0 0x5654a966def0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe67fa40bf329f6 server_cookie=e4a83774ad0461db in_seq=0 out_seq=0 2024-08-31T07:33:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.996+0000 7f8c16a00640 10 client.19401 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.996+0000 7f8c16a00640 1 -- 172.21.15.32:0/3843896645 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6622 (crc 0 0 0) 0x5654a9557160 con 0x5654a91d0da0 2024-08-31T07:33:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.996+0000 7f8c16a00640 10 client.19401 handle_command_reply: tid=0 2024-08-31T07:33:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.996+0000 7f8c3e3ef740 1 client.19401 shutdown 2024-08-31T07:33:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.996+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3843896645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a921c390 msgr2=0x5654a95f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.996+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a921c390 0x5654a95f2ba0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8c240e2ed0 tx=0x7f8c2400cf90 comp rx=0 tx=0).stop 2024-08-31T07:33:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.997+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3843896645 shutdown_connections 2024-08-31T07:33:58.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.997+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5654a91d0da0 0x5654a966def0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:58.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.997+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5654a923d070 0x5654a95f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:58.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.997+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5654a921c390 0x5654a95f2ba0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:58.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.997+0000 7f8c16a00640 0 client.19401 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:33:58.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.997+0000 7f8c3e3ef740 1 --2- 172.21.15.32:0/3843896645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654a92141b0 0x5654a95f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:58.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.997+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3843896645 >> 172.21.15.32:0/3843896645 conn(0x5654a9231460 msgr2=0x5654a95e5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:58.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.997+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3843896645 shutdown_connections 2024-08-31T07:33:58.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.997+0000 7f8c3e3ef740 1 -- 172.21.15.32:0/3843896645 wait complete. 2024-08-31T07:33:58.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.997+0000 7f8c3e3ef740 20 client.19401 trim_cache size 0 max 16384 2024-08-31T07:33:58.001 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:33:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:33:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:33:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:33:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:33:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:32:41.165270+0000", 2024-08-31T07:33:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:33:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:33:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:33:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:33:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:33:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 -- 172.21.15.32:0/4080249317 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8c180b4d20 msgr2=0x7f8c180b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8c180b4d20 0x7f8c180b7230 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0096c0 tx=0x7f8c1c004540 comp rx=0 tx=0).stop 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 -- 172.21.15.32:0/4080249317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c30113a80 msgr2=0x7f8c30118580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:33:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c30113a80 0x7f8c30118580 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8c30074ac0 tx=0x7f8c2400ee50 comp rx=0 tx=0).stop 2024-08-31T07:33:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 -- 172.21.15.32:0/4080249317 shutdown_connections 2024-08-31T07:33:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8c180b4d20 0x7f8c180b7230 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c301143d0 0x7f8c30118ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c30113a80 0x7f8c30118580 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 --2- 172.21.15.32:0/4080249317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c3010e140 0x7f8c30118040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:33:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 -- 172.21.15.32:0/4080249317 >> 172.21.15.32:0/4080249317 conn(0x7f8c3006e830 msgr2=0x7f8c301130f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:33:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 -- 172.21.15.32:0/4080249317 shutdown_connections 2024-08-31T07:33:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:33:57.998+0000 7f8c36200640 1 -- 172.21.15.32:0/4080249317 wait complete. 2024-08-31T07:33:58.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142276 14.70 MB/sec execute 338 sec latency 4126.583 ms 2024-08-31T07:33:59.109 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:33:58 smithi032 ceph-mon[17252]: pgmap v1034: 97 pgs: 97 active+clean; 1.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 605 op/s 2024-08-31T07:33:59.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:33:58 smithi073 ceph-mon[21995]: pgmap v1034: 97 pgs: 97 active+clean; 1.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 605 op/s 2024-08-31T07:33:59.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:33:58 smithi084 ceph-mon[22387]: pgmap v1034: 97 pgs: 97 active+clean; 1.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 605 op/s 2024-08-31T07:33:59.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142541 14.70 MB/sec execute 339 sec latency 3986.517 ms 2024-08-31T07:34:00.373 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:34:00.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:34:00.417 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142587 14.66 MB/sec execute 340 sec latency 1382.666 ms 2024-08-31T07:34:00.842 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:01.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:00 smithi032 ceph-mon[17252]: pgmap v1035: 97 pgs: 97 active+clean; 1.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 462 op/s 2024-08-31T07:34:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:00 smithi073 ceph-mon[21995]: pgmap v1035: 97 pgs: 97 active+clean; 1.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 462 op/s 2024-08-31T07:34:01.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:00 smithi084 ceph-mon[22387]: pgmap v1035: 97 pgs: 97 active+clean; 1.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 462 op/s 2024-08-31T07:34:01.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142639 14.63 MB/sec execute 341 sec latency 2382.715 ms 2024-08-31T07:34:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.768+0000 7fcbb6e00640 1 -- 172.21.15.32:0/2819198738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb0103900 msgr2=0x7fcbb0103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.768+0000 7fcbb6e00640 1 --2- 172.21.15.32:0/2819198738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb0103900 0x7fcbb0103d10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009900 tx=0x7fcba002a410 comp rx=0 tx=0).stop 2024-08-31T07:34:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.768+0000 7fcbb6e00640 1 -- 172.21.15.32:0/2819198738 shutdown_connections 2024-08-31T07:34:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.768+0000 7fcbb6e00640 1 --2- 172.21.15.32:0/2819198738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb0109b90 0x7fcbb01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.768+0000 7fcbb6e00640 1 --2- 172.21.15.32:0/2819198738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb0109240 0x7fcbb0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.768+0000 7fcbb6e00640 1 --2- 172.21.15.32:0/2819198738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb0103900 0x7fcbb0103d10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.768+0000 7fcbb6e00640 1 -- 172.21.15.32:0/2819198738 >> 172.21.15.32:0/2819198738 conn(0x7fcbb007b5f0 msgr2=0x7fcbb007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.768+0000 7fcbb6e00640 1 -- 172.21.15.32:0/2819198738 shutdown_connections 2024-08-31T07:34:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.768+0000 7fcbb6e00640 1 -- 172.21.15.32:0/2819198738 wait complete. 2024-08-31T07:34:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.770+0000 7fcbb6e00640 1 Processor -- start 2024-08-31T07:34:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.770+0000 7fcbb6e00640 1 -- start start 2024-08-31T07:34:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.770+0000 7fcbb6e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb0103900 0x7fcbb019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.770+0000 7fcbb6e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb0109240 0x7fcbb019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.770+0000 7fcbb6e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb0109b90 0x7fcbb019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.770+0000 7fcbb6e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01138a0 con 0x7fcbb0103900 2024-08-31T07:34:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.770+0000 7fcbb6e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113720 con 0x7fcbb0109b90 2024-08-31T07:34:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.770+0000 7fcbb6e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113a20 con 0x7fcbb0109240 2024-08-31T07:34:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb0103900 0x7fcbb019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb0109240 0x7fcbb019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbb4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb0109b90 0x7fcbb019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb0103900 0x7fcbb019a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43646/0 (socket says 172.21.15.32:43646) 2024-08-31T07:34:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbafe00640 1 -- 172.21.15.32:0/1007349810 learned_addr learned my addr 172.21.15.32:0/1007349810 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbaf400640 1 -- 172.21.15.32:0/1007349810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb0109b90 msgr2=0x7fcbb019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbaf400640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb0109b90 0x7fcbb019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbaf400640 1 -- 172.21.15.32:0/1007349810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb0103900 msgr2=0x7fcbb019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbaf400640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb0103900 0x7fcbb019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbaf400640 1 -- 172.21.15.32:0/1007349810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb01a01e0 con 0x7fcbb0109240 2024-08-31T07:34:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.771+0000 7fcbaf400640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb0109240 0x7fcbb019a810 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcbb006c790 tx=0x7fcb9c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.772+0000 7fcbacc00640 1 -- 172.21.15.32:0/1007349810 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c00cc40 con 0x7fcbb0109240 2024-08-31T07:34:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.772+0000 7fcbacc00640 1 -- 172.21.15.32:0/1007349810 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb9c00cde0 con 0x7fcbb0109240 2024-08-31T07:34:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.772+0000 7fcbacc00640 1 -- 172.21.15.32:0/1007349810 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c018780 con 0x7fcbb0109240 2024-08-31T07:34:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.772+0000 7fcbb6e00640 1 -- 172.21.15.32:0/1007349810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb01a04d0 con 0x7fcbb0109240 2024-08-31T07:34:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.772+0000 7fcbb6e00640 1 -- 172.21.15.32:0/1007349810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a6c30 con 0x7fcbb0109240 2024-08-31T07:34:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.773+0000 7fcbb6e00640 1 -- 172.21.15.32:0/1007349810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb0103d10 con 0x7fcbb0109240 2024-08-31T07:34:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.777+0000 7fcbacc00640 1 -- 172.21.15.32:0/1007349810 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcb9c010a90 con 0x7fcbb0109240 2024-08-31T07:34:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.778+0000 7fcbacc00640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcb940b4ca0 0x7fcb940b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.778+0000 7fcbacc00640 1 -- 172.21.15.32:0/1007349810 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8441+0+0 (secure 0 0 0) 0x7fcb9c0e5cb0 con 0x7fcbb0109240 2024-08-31T07:34:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.778+0000 7fcbafe00640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcb940b4ca0 0x7fcb940b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.779+0000 7fcbacc00640 1 -- 172.21.15.32:0/1007349810 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcb9c0ac440 con 0x7fcbb0109240 2024-08-31T07:34:01.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:01.779+0000 7fcbafe00640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcb940b4ca0 0x7fcb940b71b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcba00096c0 tx=0x7fcba0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.042+0000 7fcbb6e00640 1 -- 172.21.15.32:0/1007349810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbb010b990 con 0x7fcbb0109240 2024-08-31T07:34:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.044+0000 7fcbacc00640 1 -- 172.21.15.32:0/1007349810 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+11978 (secure 0 0 0) 0x7fcb9c0b0c00 con 0x7fcbb0109240 2024-08-31T07:34:02.045 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:34:02.045 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"btime":"2024-08-31T07:34:00:841849+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":79,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:34:00.136133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:34:02.046 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 79 2024-08-31T07:34:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.048+0000 7fcbb6e00640 1 -- 172.21.15.32:0/1007349810 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcb940b4ca0 msgr2=0x7fcb940b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.048+0000 7fcbb6e00640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcb940b4ca0 0x7fcb940b71b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcba00096c0 tx=0x7fcba0005a20 comp rx=0 tx=0).stop 2024-08-31T07:34:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.048+0000 7fcbb6e00640 1 -- 172.21.15.32:0/1007349810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb0109240 msgr2=0x7fcbb019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.048+0000 7fcbb6e00640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb0109240 0x7fcbb019a810 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcbb006c790 tx=0x7fcb9c00ee50 comp rx=0 tx=0).stop 2024-08-31T07:34:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.048+0000 7fcbb6e00640 1 -- 172.21.15.32:0/1007349810 shutdown_connections 2024-08-31T07:34:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.048+0000 7fcbb6e00640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcb940b4ca0 0x7fcb940b71b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.048+0000 7fcbb6e00640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb0109b90 0x7fcbb019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.048+0000 7fcbb6e00640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb0109240 0x7fcbb019a810 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.048+0000 7fcbb6e00640 1 --2- 172.21.15.32:0/1007349810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb0103900 0x7fcbb019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.048+0000 7fcbb6e00640 1 -- 172.21.15.32:0/1007349810 >> 172.21.15.32:0/1007349810 conn(0x7fcbb007b5f0 msgr2=0x7fcbb01072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.049+0000 7fcbb6e00640 1 -- 172.21.15.32:0/1007349810 shutdown_connections 2024-08-31T07:34:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:02.049+0000 7fcbb6e00640 1 -- 172.21.15.32:0/1007349810 wait complete. 2024-08-31T07:34:02.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:01 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:02.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:01 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:02.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:01 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:02.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142639 14.58 MB/sec execute 342 sec latency 3382.994 ms 2024-08-31T07:34:02.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:02 smithi032 ceph-mon[17252]: pgmap v1036: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 554 op/s 2024-08-31T07:34:02.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:02 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1007349810' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:34:03.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:34:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:02 smithi073 ceph-mon[21995]: pgmap v1036: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 554 op/s 2024-08-31T07:34:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:02 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1007349810' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:34:03.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:02 smithi084 ceph-mon[22387]: pgmap v1036: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 554 op/s 2024-08-31T07:34:03.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:02 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1007349810' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:34:03.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142639 14.54 MB/sec execute 343 sec latency 4383.046 ms 2024-08-31T07:34:03.559 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:04.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142639 14.50 MB/sec execute 344 sec latency 5383.111 ms 2024-08-31T07:34:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.721+0000 7f7127400640 1 -- 172.21.15.32:0/2392289944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120109240 msgr2=0x7f7120109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.721+0000 7f7127400640 1 --2- 172.21.15.32:0/2392289944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120109240 0x7f7120109650 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f710c009860 tx=0x7f710c02a340 comp rx=0 tx=0).stop 2024-08-31T07:34:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.721+0000 7f7127400640 1 -- 172.21.15.32:0/2392289944 shutdown_connections 2024-08-31T07:34:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.721+0000 7f7127400640 1 --2- 172.21.15.32:0/2392289944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7120109b90 0x7f71201103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.721+0000 7f7127400640 1 --2- 172.21.15.32:0/2392289944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120109240 0x7f7120109650 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.721+0000 7f7127400640 1 --2- 172.21.15.32:0/2392289944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120103900 0x7f7120103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.721+0000 7f7127400640 1 -- 172.21.15.32:0/2392289944 >> 172.21.15.32:0/2392289944 conn(0x7f712007b5f0 msgr2=0x7f712007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.721+0000 7f7127400640 1 -- 172.21.15.32:0/2392289944 shutdown_connections 2024-08-31T07:34:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.721+0000 7f7127400640 1 -- 172.21.15.32:0/2392289944 wait complete. 2024-08-31T07:34:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.722+0000 7f7127400640 1 Processor -- start 2024-08-31T07:34:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.722+0000 7f7127400640 1 -- start start 2024-08-31T07:34:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f7127400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120103900 0x7f712019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f7127400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120109240 0x7f712019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f7127400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7120109b90 0x7f71201a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f7127400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7120113850 con 0x7f7120109240 2024-08-31T07:34:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f7127400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71201136d0 con 0x7f7120103900 2024-08-31T07:34:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f7127400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f71201139d0 con 0x7f7120109b90 2024-08-31T07:34:04.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f711fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120109240 0x7f712019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f7124a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120103900 0x7f712019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f7124a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120103900 0x7f712019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53608/0 (socket says 172.21.15.32:53608) 2024-08-31T07:34:04.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f7124a00640 1 -- 172.21.15.32:0/3196010869 learned_addr learned my addr 172.21.15.32:0/3196010869 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:04.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.723+0000 7f7125400640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7120109b90 0x7f71201a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.724+0000 7f7124a00640 1 -- 172.21.15.32:0/3196010869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7120109b90 msgr2=0x7f71201a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.724+0000 7f7124a00640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7120109b90 0x7f71201a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.724+0000 7f7124a00640 1 -- 172.21.15.32:0/3196010869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120109240 msgr2=0x7f712019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.724+0000 7f7124a00640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120109240 0x7f712019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.724+0000 7f7124a00640 1 -- 172.21.15.32:0/3196010869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71201a23b0 con 0x7f7120103900 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.724+0000 7f7124a00640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120103900 0x7f712019c440 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f711400e9c0 tx=0x7f711400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.724+0000 7f711d600640 1 -- 172.21.15.32:0/3196010869 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711400cde0 con 0x7f7120103900 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.725+0000 7f711d600640 1 -- 172.21.15.32:0/3196010869 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71140035f0 con 0x7f7120103900 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.725+0000 7f7127400640 1 -- 172.21.15.32:0/3196010869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71201a26a0 con 0x7f7120103900 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.725+0000 7f7127400640 1 -- 172.21.15.32:0/3196010869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71201a8e00 con 0x7f7120103900 2024-08-31T07:34:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.725+0000 7f711d600640 1 -- 172.21.15.32:0/3196010869 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114010690 con 0x7f7120103900 2024-08-31T07:34:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.729+0000 7f712e0c3740 1 Processor -- start 2024-08-31T07:34:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.729+0000 7f712e0c3740 1 -- start start 2024-08-31T07:34:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.730+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32ba3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.730+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327d51b0 0x555f32ba3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.730+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327dd390 0x555f32ba4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.730+0000 7f712e0c3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555f32b1a7b0 con 0x555f327dd390 2024-08-31T07:34:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.730+0000 7f712e0c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555f32b09d40 con 0x555f327d51b0 2024-08-31T07:34:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.730+0000 7f712e0c3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555f32b0dcf0 con 0x555f327fe070 2024-08-31T07:34:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.730+0000 7f711d600640 1 -- 172.21.15.32:0/3196010869 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7114003fe0 con 0x7f7120103900 2024-08-31T07:34:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.730+0000 7f7125400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32ba3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.730+0000 7f7125400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32ba3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51864/0 (socket says 172.21.15.32:51864) 2024-08-31T07:34:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.730+0000 7f7125400640 1 -- 172.21.15.32:0/1087915652 learned_addr learned my addr 172.21.15.32:0/1087915652 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.731+0000 7f711fe00640 1 --2- 172.21.15.32:0/1087915652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327dd390 0x555f32ba4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.731+0000 7f7125400640 1 -- 172.21.15.32:0/1087915652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327d51b0 msgr2=0x555f32ba3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.731+0000 7f7125400640 1 --2- 172.21.15.32:0/1087915652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327d51b0 0x555f32ba3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.731+0000 7f7125400640 1 -- 172.21.15.32:0/1087915652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327dd390 msgr2=0x555f32ba4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.731+0000 7f7125400640 1 --2- 172.21.15.32:0/1087915652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327dd390 0x555f32ba4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.731+0000 7f7125400640 1 -- 172.21.15.32:0/1087915652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f32869650 con 0x555f327fe070 2024-08-31T07:34:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.731+0000 7f7125400640 1 --2- 172.21.15.32:0/1087915652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32ba3990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7118009fd0 tx=0x7f71180118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bd8f64b52c19b9b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.732+0000 7f7111c00640 1 -- 172.21.15.32:0/1087915652 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118004d20 con 0x555f327fe070 2024-08-31T07:34:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.732+0000 7f7111c00640 1 -- 172.21.15.32:0/1087915652 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7118004050 con 0x555f327fe070 2024-08-31T07:34:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.732+0000 7f7111c00640 1 -- 172.21.15.32:0/1087915652 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118005280 con 0x555f327fe070 2024-08-31T07:34:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.732+0000 7f712e0c3740 1 -- 172.21.15.32:0/1087915652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 msgr2=0x555f32ba3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.732+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1087915652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32ba3990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7118009fd0 tx=0x7f71180118d0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.733+0000 7f712e0c3740 1 -- 172.21.15.32:0/1087915652 shutdown_connections 2024-08-31T07:34:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.733+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1087915652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327dd390 0x555f32ba4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.733+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1087915652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327d51b0 0x555f32ba3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.733+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1087915652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32ba3990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.733+0000 7f712e0c3740 1 -- 172.21.15.32:0/1087915652 >> 172.21.15.32:0/1087915652 conn(0x555f327f2460 msgr2=0x555f32ba1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.733+0000 7f712e0c3740 1 -- 172.21.15.32:0/1087915652 shutdown_connections 2024-08-31T07:34:04.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.733+0000 7f712e0c3740 1 -- 172.21.15.32:0/1087915652 wait complete. 2024-08-31T07:34:04.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.733+0000 7f711d600640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71080b4c30 0x7f71080b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.734+0000 7f711d600640 1 -- 172.21.15.32:0/3196010869 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8441+0+0 (secure 0 0 0) 0x7f71140e5640 con 0x7f7120103900 2024-08-31T07:34:04.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.734+0000 7f711fe00640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71080b4c30 0x7f71080b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.734+0000 7f712e0c3740 1 -- start start 2024-08-31T07:34:04.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.734+0000 7f712e0c3740 1 Processor -- start 2024-08-31T07:34:04.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f711fe00640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71080b4c30 0x7f71080b7140 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f710c005940 tx=0x7f710c0058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:04.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32baf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bafae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32bbb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f712e0c3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555f3296c700 con 0x555f327d51b0 2024-08-31T07:34:04.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f712e0c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555f32a90cf0 con 0x555f327dd390 2024-08-31T07:34:04.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f712e0c3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555f32bb2170 con 0x555f327fe070 2024-08-31T07:34:04.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f712e0c3740 10 client.0 _ll_register_callbacks cb 0x555f32ab6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:04.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f7124a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32baf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f7124a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32baf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43676/0 (socket says 172.21.15.32:43676) 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f7124a00640 1 -- 172.21.15.32:0/369378214 learned_addr learned my addr 172.21.15.32:0/369378214 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f7125400640 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32bbb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.735+0000 7f711fe00640 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bafae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.736+0000 7f711fe00640 1 -- 172.21.15.32:0/369378214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 msgr2=0x555f32bbb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.736+0000 7f711fe00640 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32bbb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.736+0000 7f711fe00640 1 -- 172.21.15.32:0/369378214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 msgr2=0x555f32baf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.736+0000 7f711fe00640 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32baf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.736+0000 7f711fe00640 1 -- 172.21.15.32:0/369378214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f32ad18b0 con 0x555f327dd390 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.736+0000 7f7125400640 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32bbb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.736+0000 7f711fe00640 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bafae0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f710c02a860 tx=0x7f710c02a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53cdbcd4811aaf7c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.736+0000 7f7106000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.737+0000 7f7106000640 1 -- 172.21.15.32:0/369378214 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c046040 con 0x555f327dd390 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.737+0000 7f7106000640 1 -- 172.21.15.32:0/369378214 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8441+0+0 (secure 0 0 0) 0x7f710c0035f0 con 0x555f327dd390 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.737+0000 7f712e0c3740 1 -- 172.21.15.32:0/369378214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7118004050 con 0x555f327dd390 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.737+0000 7f7106000640 1 -- 172.21.15.32:0/369378214 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f710c037040 con 0x555f327dd390 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.737+0000 7f712e0c3740 10 client.29468 fetch_fsmap learned FSMap version 79 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.737+0000 7f712e0c3740 1 -- 172.21.15.32:0/369378214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x555f32bcf220 con 0x555f327dd390 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.738+0000 7f7106000640 1 -- 172.21.15.32:0/369378214 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7118004050 con 0x555f327dd390 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.738+0000 7f7106000640 1 -- 172.21.15.32:0/369378214 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 79) ==== 8406+0+0 (secure 0 0 0) 0x7f710c03d910 con 0x555f327dd390 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.738+0000 7f712e0c3740 10 client.29468 fetch_fsmap finished waiting for FSMap version 79 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.738+0000 7f712e0c3740 10 client.29468 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.738+0000 7f712e0c3740 20 client.29468 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:04.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.738+0000 7f712e0c3740 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555f327bc9f0 0x555f32bd2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.738+0000 7f712e0c3740 4 client.29468 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:34:04.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.738+0000 7f712e0c3740 1 -- 172.21.15.32:0/369378214 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f32b1c700 con 0x555f327bc9f0 2024-08-31T07:34:04.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.738+0000 7f7124a00640 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555f327bc9f0 0x555f32bd2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.739+0000 7f7124a00640 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555f327bc9f0 0x555f32bd2810 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f7d9dbf02090a39 server_cookie=bb1c7d1daa4865a in_seq=0 out_seq=0 2024-08-31T07:34:04.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.739+0000 7f7106000640 10 client.29468 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:34:04.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f7106000640 1 -- 172.21.15.32:0/369378214 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555f32b1c700 con 0x555f327bc9f0 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f7106000640 10 client.29468 handle_command_reply: tid=0 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 client.29468 shutdown 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 -- 172.21.15.32:0/369378214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 msgr2=0x555f32bafae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bafae0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f710c02a860 tx=0x7f710c02a340 comp rx=0 tx=0).stop 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 -- 172.21.15.32:0/369378214 shutdown_connections 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555f327bc9f0 0x555f32bd2810 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32bbb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bafae0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 --2- 172.21.15.32:0/369378214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32baf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 -- 172.21.15.32:0/369378214 >> 172.21.15.32:0/369378214 conn(0x555f327f2460 msgr2=0x555f32ba6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f7106000640 0 client.29468 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:34:04.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 -- 172.21.15.32:0/369378214 shutdown_connections 2024-08-31T07:34:04.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.740+0000 7f712e0c3740 1 -- 172.21.15.32:0/369378214 wait complete. 2024-08-31T07:34:04.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.741+0000 7f712e0c3740 20 client.29468 trim_cache size 0 max 16384 2024-08-31T07:34:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.764+0000 7f712e0c3740 1 Processor -- start 2024-08-31T07:34:04.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.765+0000 7f712e0c3740 1 -- start start 2024-08-31T07:34:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.765+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32bbb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.765+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32ba91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.765+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32bac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.765+0000 7f712e0c3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f710000e680 con 0x555f327d51b0 2024-08-31T07:34:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.765+0000 7f712e0c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71000103d0 con 0x555f327dd390 2024-08-31T07:34:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.765+0000 7f712e0c3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f710000ee10 con 0x555f327fe070 2024-08-31T07:34:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.765+0000 7f7124a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32bbb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.766+0000 7f7124a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32bbb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43682/0 (socket says 172.21.15.32:43682) 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.766+0000 7f7124a00640 1 -- 172.21.15.32:0/1911443616 learned_addr learned my addr 172.21.15.32:0/1911443616 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.766+0000 7f711fe00640 1 --2- 172.21.15.32:0/1911443616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32ba91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.766+0000 7f7125400640 1 --2- 172.21.15.32:0/1911443616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32bac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.766+0000 7f711fe00640 1 -- 172.21.15.32:0/1911443616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 msgr2=0x555f32bac5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.766+0000 7f711fe00640 1 --2- 172.21.15.32:0/1911443616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32bac5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.766+0000 7f711fe00640 1 -- 172.21.15.32:0/1911443616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 msgr2=0x555f32bbb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.766+0000 7f711fe00640 1 --2- 172.21.15.32:0/1911443616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32bbb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.766+0000 7f711fe00640 1 -- 172.21.15.32:0/1911443616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f327bcc20 con 0x555f327dd390 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.766+0000 7f7124a00640 1 --2- 172.21.15.32:0/1911443616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32bbb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.767+0000 7f711fe00640 1 --2- 172.21.15.32:0/1911443616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32ba91b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f710c004650 tx=0x7f710c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0eaaebe1b4575c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.767+0000 7f7111200640 1 -- 172.21.15.32:0/1911443616 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c046070 con 0x555f327dd390 2024-08-31T07:34:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.767+0000 7f7111200640 1 -- 172.21.15.32:0/1911443616 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710c030d00 con 0x555f327dd390 2024-08-31T07:34:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.767+0000 7f7111200640 1 -- 172.21.15.32:0/1911443616 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c032b50 con 0x555f327dd390 2024-08-31T07:34:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.768+0000 7f712e0c3740 1 -- 172.21.15.32:0/1911443616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 msgr2=0x555f32ba91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.768+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1911443616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32ba91b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f710c004650 tx=0x7f710c005fd0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.768+0000 7f712e0c3740 1 -- 172.21.15.32:0/1911443616 shutdown_connections 2024-08-31T07:34:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.768+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1911443616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327fe070 0x555f32bac5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.768+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1911443616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32ba91b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.768+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1911443616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327d51b0 0x555f32bbb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.768+0000 7f712e0c3740 1 -- 172.21.15.32:0/1911443616 >> 172.21.15.32:0/1911443616 conn(0x555f327f2460 msgr2=0x555f32ba8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.768+0000 7f712e0c3740 1 -- 172.21.15.32:0/1911443616 shutdown_connections 2024-08-31T07:34:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.768+0000 7f712e0c3740 1 -- 172.21.15.32:0/1911443616 wait complete. 2024-08-31T07:34:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.769+0000 7f712e0c3740 1 -- start start 2024-08-31T07:34:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.769+0000 7f712e0c3740 1 Processor -- start 2024-08-31T07:34:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.769+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327d51b0 0x555f32bb3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.769+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bb38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.769+0000 7f712e0c3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327fe070 0x555f32bb3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.769+0000 7f712e0c3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f710000fc70 con 0x555f327fe070 2024-08-31T07:34:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.769+0000 7f712e0c3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f710000f570 con 0x555f327dd390 2024-08-31T07:34:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.769+0000 7f712e0c3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f710000c680 con 0x555f327d51b0 2024-08-31T07:34:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f712e0c3740 10 client.0 _ll_register_callbacks cb 0x7f710c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f711fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bb38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f711fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bb38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:53650/0 (socket says 172.21.15.32:53650) 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f711fe00640 1 -- 172.21.15.32:0/1603837126 learned_addr learned my addr 172.21.15.32:0/1603837126 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f7124a00640 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327d51b0 0x555f32bb3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f7125400640 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327fe070 0x555f32bb3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f711fe00640 1 -- 172.21.15.32:0/1603837126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327d51b0 msgr2=0x555f32bb3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f711fe00640 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327d51b0 0x555f32bb3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f711fe00640 1 -- 172.21.15.32:0/1603837126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327fe070 msgr2=0x555f32bb3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f711fe00640 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327fe070 0x555f32bb3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f711fe00640 1 -- 172.21.15.32:0/1603837126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f32c3c1d0 con 0x555f327dd390 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.770+0000 7f7124a00640 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327d51b0 0x555f32bb3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.771+0000 7f711fe00640 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bb38d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f710c030c50 tx=0x7f710c032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bff656308b6bf0d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.771+0000 7f7106000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:34:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.771+0000 7f7106000640 1 -- 172.21.15.32:0/1603837126 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c046040 con 0x555f327dd390 2024-08-31T07:34:04.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.771+0000 7f712e0c3740 1 -- 172.21.15.32:0/1603837126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f710c030d00 con 0x555f327dd390 2024-08-31T07:34:04.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.771+0000 7f7106000640 1 -- 172.21.15.32:0/1603837126 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8441+0+0 (secure 0 0 0) 0x7f710c03d650 con 0x555f327dd390 2024-08-31T07:34:04.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.771+0000 7f7106000640 1 -- 172.21.15.32:0/1603837126 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f710c03d850 con 0x555f327dd390 2024-08-31T07:34:04.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.771+0000 7f712e0c3740 10 client.29474 fetch_fsmap learned FSMap version 79 2024-08-31T07:34:04.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.771+0000 7f712e0c3740 1 -- 172.21.15.32:0/1603837126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x555f32c3d3b0 con 0x555f327dd390 2024-08-31T07:34:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.772+0000 7f7106000640 1 -- 172.21.15.32:0/1603837126 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710c030d00 con 0x555f327dd390 2024-08-31T07:34:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.772+0000 7f7106000640 1 -- 172.21.15.32:0/1603837126 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 79) ==== 8406+0+0 (secure 0 0 0) 0x7f710c063030 con 0x555f327dd390 2024-08-31T07:34:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.772+0000 7f712e0c3740 10 client.29474 fetch_fsmap finished waiting for FSMap version 79 2024-08-31T07:34:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.772+0000 7f712e0c3740 10 client.29474 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:34:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.772+0000 7f712e0c3740 20 client.29474 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.772+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555f32791da0 0x555f32c3faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.772+0000 7f712e0c3740 4 client.29474 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:34:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.772+0000 7f712e0c3740 1 -- 172.21.15.32:0/1603837126 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f32b181a0 con 0x555f32791da0 2024-08-31T07:34:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.772+0000 7f7124a00640 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555f32791da0 0x555f32c3faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f7124a00640 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555f32791da0 0x555f32c3faf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa424486c03cb6dd server_cookie=5ef81ba8e9e0e1dd in_seq=0 out_seq=0 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f7106000640 10 client.29474 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f7106000640 1 -- 172.21.15.32:0/1603837126 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f32b181a0 con 0x555f32791da0 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f7106000640 10 client.29474 handle_command_reply: tid=0 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f712e0c3740 1 client.29474 shutdown 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f712e0c3740 1 -- 172.21.15.32:0/1603837126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 msgr2=0x555f32bb38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bb38d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f710c030c50 tx=0x7f710c032e90 comp rx=0 tx=0).stop 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f712e0c3740 1 -- 172.21.15.32:0/1603837126 shutdown_connections 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x555f32791da0 0x555f32c3faf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555f327fe070 0x555f32bb3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f327dd390 0x555f32bb38d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f7106000640 0 client.29474 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:34:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f712e0c3740 1 --2- 172.21.15.32:0/1603837126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555f327d51b0 0x555f32bb3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.773+0000 7f712e0c3740 1 -- 172.21.15.32:0/1603837126 >> 172.21.15.32:0/1603837126 conn(0x555f327f2460 msgr2=0x555f32ba49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.774+0000 7f712e0c3740 1 -- 172.21.15.32:0/1603837126 shutdown_connections 2024-08-31T07:34:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.774+0000 7f712e0c3740 1 -- 172.21.15.32:0/1603837126 wait complete. 2024-08-31T07:34:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.774+0000 7f712e0c3740 20 client.29474 trim_cache size 0 max 16384 2024-08-31T07:34:04.778 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:34:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 -- 172.21.15.32:0/3196010869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71080b4c30 msgr2=0x7f71080b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71080b4c30 0x7f71080b7140 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f710c005940 tx=0x7f710c0058d0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 -- 172.21.15.32:0/3196010869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120103900 msgr2=0x7f712019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120103900 0x7f712019c440 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f711400e9c0 tx=0x7f711400ee80 comp rx=0 tx=0).stop 2024-08-31T07:34:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 -- 172.21.15.32:0/3196010869 shutdown_connections 2024-08-31T07:34:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71080b4c30 0x7f71080b7140 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7120109b90 0x7f71201a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120109240 0x7f712019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 --2- 172.21.15.32:0/3196010869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120103900 0x7f712019c440 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 -- 172.21.15.32:0/3196010869 >> 172.21.15.32:0/3196010869 conn(0x7f712007b5f0 msgr2=0x7f7120107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 -- 172.21.15.32:0/3196010869 shutdown_connections 2024-08-31T07:34:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:04.775+0000 7f7127400640 1 -- 172.21.15.32:0/3196010869 wait complete. 2024-08-31T07:34:05.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:04 smithi032 ceph-mon[17252]: pgmap v1037: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 505 op/s 2024-08-31T07:34:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:04 smithi073 ceph-mon[21995]: pgmap v1037: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 505 op/s 2024-08-31T07:34:05.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:04 smithi084 ceph-mon[22387]: pgmap v1037: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 505 op/s 2024-08-31T07:34:05.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142682 14.47 MB/sec execute 345 sec latency 6383.176 ms 2024-08-31T07:34:06.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142682 14.43 MB/sec execute 346 sec latency 7383.231 ms 2024-08-31T07:34:07.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:06 smithi073 ceph-mon[21995]: pgmap v1038: 97 pgs: 97 active+clean; 1.9 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 15 MiB/s wr, 542 op/s 2024-08-31T07:34:07.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:06 smithi084 ceph-mon[22387]: pgmap v1038: 97 pgs: 97 active+clean; 1.9 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 15 MiB/s wr, 542 op/s 2024-08-31T07:34:07.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:06 smithi032 ceph-mon[17252]: pgmap v1038: 97 pgs: 97 active+clean; 1.9 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 15 MiB/s wr, 542 op/s 2024-08-31T07:34:07.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142682 14.39 MB/sec execute 347 sec latency 8383.293 ms 2024-08-31T07:34:08.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:07 smithi073 ceph-mon[21995]: osdmap e111: 12 total, 12 up, 12 in 2024-08-31T07:34:08.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:07 smithi084 ceph-mon[22387]: osdmap e111: 12 total, 12 up, 12 in 2024-08-31T07:34:08.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:07 smithi032 ceph-mon[17252]: osdmap e111: 12 total, 12 up, 12 in 2024-08-31T07:34:08.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142682 14.34 MB/sec execute 348 sec latency 9383.354 ms 2024-08-31T07:34:09.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:08 smithi073 ceph-mon[21995]: pgmap v1040: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 443 op/s 2024-08-31T07:34:09.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:08 smithi084 ceph-mon[22387]: pgmap v1040: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 443 op/s 2024-08-31T07:34:09.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:08 smithi032 ceph-mon[17252]: pgmap v1040: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 443 op/s 2024-08-31T07:34:09.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142682 14.30 MB/sec execute 349 sec latency 10383.413 ms 2024-08-31T07:34:09.561 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.1 2024-08-31T07:34:09.561 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:34:10.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142682 14.26 MB/sec execute 350 sec latency 11383.471 ms 2024-08-31T07:34:10.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:34:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:10.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:10 smithi032 ceph-mon[17252]: pgmap v1041: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 428 op/s 2024-08-31T07:34:11.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:10 smithi073 ceph-mon[21995]: pgmap v1041: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 428 op/s 2024-08-31T07:34:11.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:10 smithi084 ceph-mon[22387]: pgmap v1041: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 428 op/s 2024-08-31T07:34:11.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142682 14.22 MB/sec execute 351 sec latency 12383.542 ms 2024-08-31T07:34:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.703+0000 7f8e4b800640 1 -- 172.21.15.32:0/4099474109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e44109260 msgr2=0x7f8e44109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.703+0000 7f8e4b800640 1 --2- 172.21.15.32:0/4099474109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e44109260 0x7f8e44109670 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009900 tx=0x7f8e3002a410 comp rx=0 tx=0).stop 2024-08-31T07:34:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.703+0000 7f8e4b800640 1 -- 172.21.15.32:0/4099474109 shutdown_connections 2024-08-31T07:34:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.703+0000 7f8e4b800640 1 --2- 172.21.15.32:0/4099474109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e44109bb0 0x7f8e441103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.703+0000 7f8e4b800640 1 --2- 172.21.15.32:0/4099474109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e44109260 0x7f8e44109670 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.703+0000 7f8e4b800640 1 --2- 172.21.15.32:0/4099474109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e44103920 0x7f8e44103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.703+0000 7f8e4b800640 1 -- 172.21.15.32:0/4099474109 >> 172.21.15.32:0/4099474109 conn(0x7f8e440ff3b0 msgr2=0x7f8e44101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.703+0000 7f8e4b800640 1 -- 172.21.15.32:0/4099474109 shutdown_connections 2024-08-31T07:34:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.703+0000 7f8e4b800640 1 -- 172.21.15.32:0/4099474109 wait complete. 2024-08-31T07:34:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.704+0000 7f8e4b800640 1 Processor -- start 2024-08-31T07:34:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.705+0000 7f8e4b800640 1 -- start start 2024-08-31T07:34:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.705+0000 7f8e4b800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e44103920 0x7f8e4419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.705+0000 7f8e4b800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e44109260 0x7f8e4419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.705+0000 7f8e4b800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e44109bb0 0x7f8e441a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.705+0000 7f8e4b800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44113870 con 0x7f8e44109bb0 2024-08-31T07:34:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.705+0000 7f8e4b800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8e441136f0 con 0x7f8e44103920 2024-08-31T07:34:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.705+0000 7f8e4b800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8e441139f0 con 0x7f8e44109260 2024-08-31T07:34:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.705+0000 7f8e43e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e44109260 0x7f8e4419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.706+0000 7f8e48e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e44103920 0x7f8e4419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.706+0000 7f8e43e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e44109260 0x7f8e4419c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:43584/0 (socket says 172.21.15.32:43584) 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.706+0000 7f8e43e00640 1 -- 172.21.15.32:0/4163370531 learned_addr learned my addr 172.21.15.32:0/4163370531 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.706+0000 7f8e49800640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e44109bb0 0x7f8e441a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.706+0000 7f8e48e00640 1 -- 172.21.15.32:0/4163370531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e44109260 msgr2=0x7f8e4419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.706+0000 7f8e48e00640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e44109260 0x7f8e4419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.706+0000 7f8e48e00640 1 -- 172.21.15.32:0/4163370531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e44109bb0 msgr2=0x7f8e441a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.706+0000 7f8e48e00640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e44109bb0 0x7f8e441a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.706+0000 7f8e48e00640 1 -- 172.21.15.32:0/4163370531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e441a2390 con 0x7f8e44103920 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.706+0000 7f8e48e00640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e44103920 0x7f8e4419c420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800e9f0 tx=0x7f8e3800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.707+0000 7f8e41600640 1 -- 172.21.15.32:0/4163370531 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3800cd80 con 0x7f8e44103920 2024-08-31T07:34:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.707+0000 7f8e41600640 1 -- 172.21.15.32:0/4163370531 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e380035f0 con 0x7f8e44103920 2024-08-31T07:34:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.707+0000 7f8e4b800640 1 -- 172.21.15.32:0/4163370531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e441a2680 con 0x7f8e44103920 2024-08-31T07:34:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.707+0000 7f8e4b800640 1 -- 172.21.15.32:0/4163370531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e441a8de0 con 0x7f8e44103920 2024-08-31T07:34:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.707+0000 7f8e41600640 1 -- 172.21.15.32:0/4163370531 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38010670 con 0x7f8e44103920 2024-08-31T07:34:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.708+0000 7f8e4b800640 1 -- 172.21.15.32:0/4163370531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e240052c0 con 0x7f8e44103920 2024-08-31T07:34:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.712+0000 7f8e41600640 1 -- 172.21.15.32:0/4163370531 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8e380037a0 con 0x7f8e44103920 2024-08-31T07:34:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.714+0000 7f8e41600640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e2c0b4ca0 0x7f8e2c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.714+0000 7f8e41600640 1 -- 172.21.15.32:0/4163370531 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8485+0+0 (secure 0 0 0) 0x7f8e380e5800 con 0x7f8e44103920 2024-08-31T07:34:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.714+0000 7f8e43e00640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e2c0b4ca0 0x7f8e2c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.715+0000 7f8e43e00640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e2c0b4ca0 0x7f8e2c0b71b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8e44108d30 tx=0x7f8e30009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.715+0000 7f8e41600640 1 -- 172.21.15.32:0/4163370531 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8e38056130 con 0x7f8e44103920 2024-08-31T07:34:11.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.964+0000 7f8e4b800640 1 -- 172.21.15.32:0/4163370531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e240055b0 con 0x7f8e44103920 2024-08-31T07:34:11.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.965+0000 7f8e41600640 1 -- 172.21.15.32:0/4163370531 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+11978 (secure 0 0 0) 0x7f8e380ac5b0 con 0x7f8e44103920 2024-08-31T07:34:11.967 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:34:11.967 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"btime":"2024-08-31T07:34:00:841849+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":79,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:34:00.136133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:34:11.967 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 79 2024-08-31T07:34:11.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 -- 172.21.15.32:0/4163370531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e2c0b4ca0 msgr2=0x7f8e2c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:11.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e2c0b4ca0 0x7f8e2c0b71b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8e44108d30 tx=0x7f8e30009560 comp rx=0 tx=0).stop 2024-08-31T07:34:11.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 -- 172.21.15.32:0/4163370531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e44103920 msgr2=0x7f8e4419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:11.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e44103920 0x7f8e4419c420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800e9f0 tx=0x7f8e3800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:34:11.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 -- 172.21.15.32:0/4163370531 shutdown_connections 2024-08-31T07:34:11.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e2c0b4ca0 0x7f8e2c0b71b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:11.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e44109bb0 0x7f8e441a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:11.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e44109260 0x7f8e4419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:11.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 --2- 172.21.15.32:0/4163370531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e44103920 0x7f8e4419c420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:11.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 -- 172.21.15.32:0/4163370531 >> 172.21.15.32:0/4163370531 conn(0x7f8e440ff3b0 msgr2=0x7f8e4410e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:11.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 -- 172.21.15.32:0/4163370531 shutdown_connections 2024-08-31T07:34:11.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:11.968+0000 7f8e4b800640 1 -- 172.21.15.32:0/4163370531 wait complete. 2024-08-31T07:34:12.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142682 14.18 MB/sec execute 352 sec latency 13383.586 ms 2024-08-31T07:34:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.545+0000 7fce50000640 1 -- 172.21.15.32:0/3035228480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce481144e0 msgr2=0x7fce48116900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.545+0000 7fce50000640 1 --2- 172.21.15.32:0/3035228480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce481144e0 0x7fce48116900 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fce40009ca0 tx=0x7fce4002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.545+0000 7fce50000640 1 -- 172.21.15.32:0/3035228480 shutdown_connections 2024-08-31T07:34:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.545+0000 7fce50000640 1 --2- 172.21.15.32:0/3035228480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce481144e0 0x7fce48116900 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.545+0000 7fce50000640 1 --2- 172.21.15.32:0/3035228480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce48078030 0x7fce48113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.545+0000 7fce50000640 1 --2- 172.21.15.32:0/3035228480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480776e0 0x7fce48077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.545+0000 7fce50000640 1 -- 172.21.15.32:0/3035228480 >> 172.21.15.32:0/3035228480 conn(0x7fce481015b0 msgr2=0x7fce48103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:12.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.545+0000 7fce50000640 1 -- 172.21.15.32:0/3035228480 shutdown_connections 2024-08-31T07:34:12.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.545+0000 7fce50000640 1 -- 172.21.15.32:0/3035228480 wait complete. 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.546+0000 7fce50000640 1 Processor -- start 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.546+0000 7fce50000640 1 -- start start 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce50000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480776e0 0x7fce481a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce50000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce48078030 0x7fce481a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce50000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481144e0 0x7fce481a6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce50000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce48119e40 con 0x7fce48078030 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce50000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce48119cc0 con 0x7fce481144e0 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce50000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce48119fc0 con 0x7fce480776e0 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce4d600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480776e0 0x7fce481a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce4d600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480776e0 0x7fce481a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:43602/0 (socket says 172.21.15.32:43602) 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce4d600640 1 -- 172.21.15.32:0/642366058 learned_addr learned my addr 172.21.15.32:0/642366058 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:12.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce4cc00640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce48078030 0x7fce481a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce4e000640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481144e0 0x7fce481a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce4d600640 1 -- 172.21.15.32:0/642366058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481144e0 msgr2=0x7fce481a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:12.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce4d600640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481144e0 0x7fce481a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce4d600640 1 -- 172.21.15.32:0/642366058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce48078030 msgr2=0x7fce481a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:12.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.547+0000 7fce4d600640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce48078030 0x7fce481a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.548+0000 7fce4d600640 1 -- 172.21.15.32:0/642366058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce481a6760 con 0x7fce480776e0 2024-08-31T07:34:12.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.548+0000 7fce4d600640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480776e0 0x7fce481a07f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00e9f0 tx=0x7fce3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:12.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.548+0000 7fce46000640 1 -- 172.21.15.32:0/642366058 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00cd80 con 0x7fce480776e0 2024-08-31T07:34:12.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.548+0000 7fce50000640 1 -- 172.21.15.32:0/642366058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce481a6a50 con 0x7fce480776e0 2024-08-31T07:34:12.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.548+0000 7fce46000640 1 -- 172.21.15.32:0/642366058 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c0108e0 con 0x7fce480776e0 2024-08-31T07:34:12.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.548+0000 7fce46000640 1 -- 172.21.15.32:0/642366058 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c018980 con 0x7fce480776e0 2024-08-31T07:34:12.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.548+0000 7fce50000640 1 -- 172.21.15.32:0/642366058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce481ad1b0 con 0x7fce480776e0 2024-08-31T07:34:12.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.551+0000 7fce56bfc740 1 Processor -- start 2024-08-31T07:34:12.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce56bfc740 1 -- start start 2024-08-31T07:34:12.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 0x5611c968c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 0x5611c968cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c968d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce56bfc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5611c96037b0 con 0x5611c92e7070 2024-08-31T07:34:12.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce56bfc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5611c95f2d40 con 0x5611c92c6390 2024-08-31T07:34:12.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce56bfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5611c95f6cf0 con 0x5611c92be1b0 2024-08-31T07:34:12.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce4e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 0x5611c968c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce4e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 0x5611c968c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41150/0 (socket says 172.21.15.32:41150) 2024-08-31T07:34:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce4e000640 1 -- 172.21.15.32:0/2037866773 learned_addr learned my addr 172.21.15.32:0/2037866773 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.552+0000 7fce4cc00640 1 --2- 172.21.15.32:0/2037866773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c968d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.553+0000 7fce4cc00640 1 -- 172.21.15.32:0/2037866773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 msgr2=0x5611c968cdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.553+0000 7fce4cc00640 1 --2- 172.21.15.32:0/2037866773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 0x5611c968cdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.553+0000 7fce4cc00640 1 -- 172.21.15.32:0/2037866773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 msgr2=0x5611c968c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.553+0000 7fce4cc00640 1 --2- 172.21.15.32:0/2037866773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 0x5611c968c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.553+0000 7fce4cc00640 1 -- 172.21.15.32:0/2037866773 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c9352650 con 0x5611c92c6390 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.553+0000 7fce4e000640 1 --2- 172.21.15.32:0/2037866773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 0x5611c968c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.553+0000 7fce46000640 1 -- 172.21.15.32:0/642366058 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fce3c010430 con 0x7fce480776e0 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.553+0000 7fce4cc00640 1 --2- 172.21.15.32:0/2037866773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c968d330 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fce4806c5b0 tx=0x7fce34011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=335f3d7c15fef1d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.553+0000 7fce3a600640 1 -- 172.21.15.32:0/2037866773 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34004050 con 0x5611c92c6390 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.553+0000 7fce3a600640 1 -- 172.21.15.32:0/2037866773 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce340054d0 con 0x5611c92c6390 2024-08-31T07:34:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.554+0000 7fce3a600640 1 -- 172.21.15.32:0/2037866773 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34004c60 con 0x5611c92c6390 2024-08-31T07:34:12.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.554+0000 7fce56bfc740 1 -- 172.21.15.32:0/2037866773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 msgr2=0x5611c968d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:12.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.554+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2037866773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c968d330 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fce4806c5b0 tx=0x7fce34011810 comp rx=0 tx=0).stop 2024-08-31T07:34:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.554+0000 7fce56bfc740 1 -- 172.21.15.32:0/2037866773 shutdown_connections 2024-08-31T07:34:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.554+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2037866773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c968d330 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.554+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2037866773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 0x5611c968cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.554+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2037866773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 0x5611c968c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.554+0000 7fce56bfc740 1 -- 172.21.15.32:0/2037866773 >> 172.21.15.32:0/2037866773 conn(0x5611c92db460 msgr2=0x5611c968a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.555+0000 7fce46000640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 0x7fce300b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.555+0000 7fce46000640 1 -- 172.21.15.32:0/642366058 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8485+0+0 (secure 0 0 0) 0x7fce3c0e5940 con 0x7fce480776e0 2024-08-31T07:34:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.555+0000 7fce4cc00640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 0x7fce300b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.555+0000 7fce56bfc740 1 -- 172.21.15.32:0/2037866773 shutdown_connections 2024-08-31T07:34:12.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.555+0000 7fce56bfc740 1 -- 172.21.15.32:0/2037866773 wait complete. 2024-08-31T07:34:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.556+0000 7fce56bfc740 1 -- start start 2024-08-31T07:34:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.556+0000 7fce4cc00640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 0x7fce300b7230 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fce340057c0 tx=0x7fce34004840 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.556+0000 7fce56bfc740 1 Processor -- start 2024-08-31T07:34:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 0x5611c9698520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c9698b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 0x5611c96a4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce56bfc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5611c9455700 con 0x5611c92e7070 2024-08-31T07:34:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce56bfc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5611c9579cf0 con 0x5611c92c6390 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce56bfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5611c969b090 con 0x5611c92be1b0 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce56bfc740 10 client.0 _ll_register_callbacks cb 0x5611c959f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce4d600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 0x5611c9698520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce4d600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 0x5611c9698520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:43604/0 (socket says 172.21.15.32:43604) 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce4d600640 1 -- 172.21.15.32:0/2200619125 learned_addr learned my addr 172.21.15.32:0/2200619125 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce4e000640 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 0x5611c96a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce4cc00640 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c9698b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce4d600640 1 -- 172.21.15.32:0/2200619125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 msgr2=0x5611c9698b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce4d600640 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c9698b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce4d600640 1 -- 172.21.15.32:0/2200619125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 msgr2=0x5611c96a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce4d600640 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 0x5611c96a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.557+0000 7fce4d600640 1 -- 172.21.15.32:0/2200619125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c95ba8b0 con 0x5611c92be1b0 2024-08-31T07:34:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.558+0000 7fce4d600640 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 0x5611c9698520 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0e30b0 tx=0x7fce3c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebe15a808077aebc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.558+0000 7fce2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:34:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.558+0000 7fce2ea00640 1 -- 172.21.15.32:0/2200619125 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0e34e0 con 0x5611c92be1b0 2024-08-31T07:34:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.558+0000 7fce2ea00640 1 -- 172.21.15.32:0/2200619125 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8485+0+0 (secure 0 0 0) 0x7fce3c0b0470 con 0x5611c92be1b0 2024-08-31T07:34:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.558+0000 7fce56bfc740 1 -- 172.21.15.32:0/2200619125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce340054d0 con 0x5611c92be1b0 2024-08-31T07:34:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.558+0000 7fce2ea00640 1 -- 172.21.15.32:0/2200619125 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fce3c0b0860 con 0x5611c92be1b0 2024-08-31T07:34:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.558+0000 7fce56bfc740 10 client.29485 fetch_fsmap learned FSMap version 79 2024-08-31T07:34:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.558+0000 7fce56bfc740 1 -- 172.21.15.32:0/2200619125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x5611c96a7150 con 0x5611c92be1b0 2024-08-31T07:34:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.558+0000 7fce2ea00640 1 -- 172.21.15.32:0/2200619125 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce340054d0 con 0x5611c92be1b0 2024-08-31T07:34:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.559+0000 7fce2ea00640 1 -- 172.21.15.32:0/2200619125 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 79) ==== 8406+0+0 (secure 0 0 0) 0x7fce3c018b20 con 0x5611c92be1b0 2024-08-31T07:34:12.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.559+0000 7fce56bfc740 10 client.29485 fetch_fsmap finished waiting for FSMap version 79 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.559+0000 7fce56bfc740 10 client.29485 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.559+0000 7fce56bfc740 20 client.29485 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.559+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5611c92a59f0 0x5611c96aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.559+0000 7fce56bfc740 4 client.29485 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.559+0000 7fce56bfc740 1 -- 172.21.15.32:0/2200619125 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611c9605700 con 0x5611c92a59f0 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.559+0000 7fce4cc00640 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5611c92a59f0 0x5611c96aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.560+0000 7fce4cc00640 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5611c92a59f0 0x5611c96aa5c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=735950b98c2200af server_cookie=ff1bc40b735c65a3 in_seq=0 out_seq=0 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.560+0000 7fce2ea00640 10 client.29485 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce2ea00640 1 -- 172.21.15.32:0/2200619125 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5611c9605700 con 0x5611c92a59f0 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce2ea00640 10 client.29485 handle_command_reply: tid=0 2024-08-31T07:34:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce56bfc740 1 client.29485 shutdown 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce56bfc740 1 -- 172.21.15.32:0/2200619125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 msgr2=0x5611c9698520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 0x5611c9698520 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0e30b0 tx=0x7fce3c010860 comp rx=0 tx=0).stop 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce56bfc740 1 -- 172.21.15.32:0/2200619125 shutdown_connections 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5611c92a59f0 0x5611c96aa5c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92e7070 0x5611c96a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c9698b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce2ea00640 0 client.29485 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2200619125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92be1b0 0x5611c9698520 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.561+0000 7fce56bfc740 1 -- 172.21.15.32:0/2200619125 >> 172.21.15.32:0/2200619125 conn(0x5611c92db460 msgr2=0x5611c968fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.562+0000 7fce56bfc740 1 -- 172.21.15.32:0/2200619125 shutdown_connections 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.562+0000 7fce56bfc740 1 -- 172.21.15.32:0/2200619125 wait complete. 2024-08-31T07:34:12.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.562+0000 7fce56bfc740 20 client.29485 trim_cache size 0 max 16384 2024-08-31T07:34:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.576+0000 7fce56bfc740 1 Processor -- start 2024-08-31T07:34:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.576+0000 7fce56bfc740 1 -- start start 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.576+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c9693ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.576+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c96a4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.576+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92e7070 0x5611c968a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.576+0000 7fce56bfc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b5a40 con 0x5611c92be1b0 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.576+0000 7fce56bfc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b7790 con 0x5611c92c6390 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.576+0000 7fce56bfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b61d0 con 0x5611c92e7070 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c9693ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c9693ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41156/0 (socket says 172.21.15.32:41156) 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4d600640 1 -- 172.21.15.32:0/2997248492 learned_addr learned my addr 172.21.15.32:0/2997248492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4e000640 1 --2- 172.21.15.32:0/2997248492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92e7070 0x5611c968a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4cc00640 1 --2- 172.21.15.32:0/2997248492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c96a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4d600640 1 -- 172.21.15.32:0/2997248492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92e7070 msgr2=0x5611c968a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4d600640 1 --2- 172.21.15.32:0/2997248492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92e7070 0x5611c968a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4d600640 1 -- 172.21.15.32:0/2997248492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 msgr2=0x5611c96a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4d600640 1 --2- 172.21.15.32:0/2997248492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c96a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4d600640 1 -- 172.21.15.32:0/2997248492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c96995e0 con 0x5611c92be1b0 2024-08-31T07:34:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.577+0000 7fce4d600640 1 --2- 172.21.15.32:0/2997248492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c9693ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0abcf0 tx=0x7fce3c0a7960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29eec2394c67fdc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.578+0000 7fce39c00640 1 -- 172.21.15.32:0/2997248492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0a7a70 con 0x5611c92be1b0 2024-08-31T07:34:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.578+0000 7fce39c00640 1 -- 172.21.15.32:0/2997248492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c0b5310 con 0x5611c92be1b0 2024-08-31T07:34:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.578+0000 7fce39c00640 1 -- 172.21.15.32:0/2997248492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0930e0 con 0x5611c92be1b0 2024-08-31T07:34:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.578+0000 7fce56bfc740 1 -- 172.21.15.32:0/2997248492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 msgr2=0x5611c9693ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.578+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2997248492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c9693ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0abcf0 tx=0x7fce3c0a7960 comp rx=0 tx=0).stop 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.579+0000 7fce56bfc740 1 -- 172.21.15.32:0/2997248492 shutdown_connections 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.579+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2997248492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92e7070 0x5611c968a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.579+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2997248492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c96a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.579+0000 7fce56bfc740 1 --2- 172.21.15.32:0/2997248492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c9693ad0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.579+0000 7fce56bfc740 1 -- 172.21.15.32:0/2997248492 >> 172.21.15.32:0/2997248492 conn(0x5611c92db460 msgr2=0x5611c968a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.579+0000 7fce56bfc740 1 -- 172.21.15.32:0/2997248492 shutdown_connections 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.579+0000 7fce56bfc740 1 -- 172.21.15.32:0/2997248492 wait complete. 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.579+0000 7fce56bfc740 1 -- start start 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.579+0000 7fce56bfc740 1 Processor -- start 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c969c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c969c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce56bfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92e7070 0x5611c969cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce56bfc740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b7030 con 0x5611c92be1b0 2024-08-31T07:34:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce56bfc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b6930 con 0x5611c92c6390 2024-08-31T07:34:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce56bfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b3a40 con 0x5611c92e7070 2024-08-31T07:34:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce56bfc740 10 client.0 _ll_register_callbacks cb 0x7fce3c0ab760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce4d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c969c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce4d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c969c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41160/0 (socket says 172.21.15.32:41160) 2024-08-31T07:34:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce4d600640 1 -- 172.21.15.32:0/3702319268 learned_addr learned my addr 172.21.15.32:0/3702319268 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.580+0000 7fce4cc00640 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c969c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.581+0000 7fce4e000640 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92e7070 0x5611c969cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.581+0000 7fce4d600640 1 -- 172.21.15.32:0/3702319268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92e7070 msgr2=0x5611c969cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.581+0000 7fce4d600640 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92e7070 0x5611c969cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.581+0000 7fce4d600640 1 -- 172.21.15.32:0/3702319268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 msgr2=0x5611c969c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.581+0000 7fce4d600640 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c969c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.581+0000 7fce4d600640 1 -- 172.21.15.32:0/3702319268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c9714180 con 0x5611c92be1b0 2024-08-31T07:34:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.581+0000 7fce4cc00640 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c969c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.581+0000 7fce4d600640 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c969c4b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0abea0 tx=0x7fce3c0abbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82779aa5eab7b5b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.581+0000 7fce2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.581+0000 7fce2ea00640 1 -- 172.21.15.32:0/3702319268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0ab250 con 0x5611c92be1b0 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.582+0000 7fce2ea00640 1 -- 172.21.15.32:0/3702319268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8485+0+0 (secure 0 0 0) 0x7fce3c0a85e0 con 0x5611c92be1b0 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.582+0000 7fce56bfc740 1 -- 172.21.15.32:0/3702319268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce3c0b5310 con 0x5611c92be1b0 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.582+0000 7fce2ea00640 1 -- 172.21.15.32:0/3702319268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fce3c0ab3f0 con 0x5611c92be1b0 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.582+0000 7fce56bfc740 10 client.19449 fetch_fsmap learned FSMap version 79 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.582+0000 7fce56bfc740 1 -- 172.21.15.32:0/3702319268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x5611c9715400 con 0x5611c92be1b0 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.582+0000 7fce2ea00640 1 -- 172.21.15.32:0/3702319268 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c0b5310 con 0x5611c92be1b0 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.582+0000 7fce2ea00640 1 -- 172.21.15.32:0/3702319268 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 79) ==== 8406+0+0 (secure 0 0 0) 0x7fce3c0cd050 con 0x5611c92be1b0 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.582+0000 7fce56bfc740 10 client.19449 fetch_fsmap finished waiting for FSMap version 79 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.582+0000 7fce56bfc740 10 client.19449 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.583+0000 7fce56bfc740 20 client.19449 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.583+0000 7fce56bfc740 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5611c927ada0 0x5611c9717820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.583+0000 7fce56bfc740 4 client.19449 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.583+0000 7fce56bfc740 1 -- 172.21.15.32:0/3702319268 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611c96011a0 con 0x5611c927ada0 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.583+0000 7fce4cc00640 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5611c927ada0 0x5611c9717820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.583+0000 7fce4cc00640 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5611c927ada0 0x5611c9717820 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2f01cd9802bae974 server_cookie=2f2e83a5f0d0b3ae in_seq=0 out_seq=0 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.583+0000 7fce2ea00640 10 client.19449 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce2ea00640 1 -- 172.21.15.32:0/3702319268 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611c96011a0 con 0x5611c927ada0 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce2ea00640 10 client.19449 handle_command_reply: tid=0 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 client.19449 shutdown 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 -- 172.21.15.32:0/3702319268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 msgr2=0x5611c969c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c969c4b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0abea0 tx=0x7fce3c0abbc0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 -- 172.21.15.32:0/3702319268 shutdown_connections 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x5611c927ada0 0x5611c9717820 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5611c92e7070 0x5611c969cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611c92c6390 0x5611c969c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 --2- 172.21.15.32:0/3702319268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5611c92be1b0 0x5611c969c4b0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 -- 172.21.15.32:0/3702319268 >> 172.21.15.32:0/3702319268 conn(0x5611c92db460 msgr2=0x5611c9688de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce2ea00640 0 client.19449 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:34:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 -- 172.21.15.32:0/3702319268 shutdown_connections 2024-08-31T07:34:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 1 -- 172.21.15.32:0/3702319268 wait complete. 2024-08-31T07:34:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.584+0000 7fce56bfc740 20 client.19449 trim_cache size 0 max 16384 2024-08-31T07:34:12.589 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:34:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 -- 172.21.15.32:0/642366058 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 msgr2=0x7fce300b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 0x7fce300b7230 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fce340057c0 tx=0x7fce34004840 comp rx=0 tx=0).stop 2024-08-31T07:34:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 -- 172.21.15.32:0/642366058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480776e0 msgr2=0x7fce481a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480776e0 0x7fce481a07f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00e9f0 tx=0x7fce3c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 -- 172.21.15.32:0/642366058 shutdown_connections 2024-08-31T07:34:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce300b4d20 0x7fce300b7230 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481144e0 0x7fce481a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce48078030 0x7fce481a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 --2- 172.21.15.32:0/642366058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce480776e0 0x7fce481a07f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 -- 172.21.15.32:0/642366058 >> 172.21.15.32:0/642366058 conn(0x7fce481015b0 msgr2=0x7fce48116510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 -- 172.21.15.32:0/642366058 shutdown_connections 2024-08-31T07:34:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:12.585+0000 7fce50000640 1 -- 172.21.15.32:0/642366058 wait complete. 2024-08-31T07:34:13.141 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:12 smithi032 ceph-mon[17252]: pgmap v1042: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 305 op/s 2024-08-31T07:34:13.141 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:12 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4163370531' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:34:13.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:12 smithi073 ceph-mon[21995]: pgmap v1042: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 305 op/s 2024-08-31T07:34:13.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:12 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4163370531' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:34:13.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:12 smithi084 ceph-mon[22387]: pgmap v1042: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 305 op/s 2024-08-31T07:34:13.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:12 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4163370531' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:34:13.320 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 134.1 --await-for 45.0' 2024-08-31T07:34:13.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 134.1 --await-for 45.0 2024-08-31T07:34:13.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142682 14.14 MB/sec execute 353 sec latency 14383.625 ms 2024-08-31T07:34:13.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:34:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:14.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142688 14.10 MB/sec execute 354 sec latency 15379.088 ms 2024-08-31T07:34:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:14.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.842+0000 7fa33e000640 1 -- 172.21.15.32:0/3709515670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa338104ee0 msgr2=0x7fa33810d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:14.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.842+0000 7fa33e000640 1 --2- 172.21.15.32:0/3709515670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa338104ee0 0x7fa33810d430 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa324009900 tx=0x7fa32402a410 comp rx=0 tx=0).stop 2024-08-31T07:34:14.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.842+0000 7fa33e000640 1 -- 172.21.15.32:0/3709515670 shutdown_connections 2024-08-31T07:34:14.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.842+0000 7fa33e000640 1 --2- 172.21.15.32:0/3709515670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa33810d9f0 0x7fa33810fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.842+0000 7fa33e000640 1 --2- 172.21.15.32:0/3709515670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa338104ee0 0x7fa33810d430 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.842+0000 7fa33e000640 1 --2- 172.21.15.32:0/3709515670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa338104590 0x7fa3381049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.842+0000 7fa33e000640 1 -- 172.21.15.32:0/3709515670 >> 172.21.15.32:0/3709515670 conn(0x7fa3380fe0d0 msgr2=0x7fa338100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:14.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.842+0000 7fa33e000640 1 -- 172.21.15.32:0/3709515670 shutdown_connections 2024-08-31T07:34:14.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.842+0000 7fa33e000640 1 -- 172.21.15.32:0/3709515670 wait complete. 2024-08-31T07:34:14.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.843+0000 7fa33e000640 1 Processor -- start 2024-08-31T07:34:14.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.843+0000 7fa33e000640 1 -- start start 2024-08-31T07:34:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa33e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa338104590 0x7fa3381022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa33e000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa338104ee0 0x7fa338102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa33e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa33810d9f0 0x7fa338102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa33e000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa338113670 con 0x7fa338104590 2024-08-31T07:34:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa33e000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa3381134f0 con 0x7fa33810d9f0 2024-08-31T07:34:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa33e000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3381137f0 con 0x7fa338104ee0 2024-08-31T07:34:14.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa33cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa338104590 0x7fa3381022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa33cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa338104590 0x7fa3381022d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41178/0 (socket says 172.21.15.32:41178) 2024-08-31T07:34:14.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa33cc00640 1 -- 172.21.15.32:0/3091088066 learned_addr learned my addr 172.21.15.32:0/3091088066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:14.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa337e00640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa338104ee0 0x7fa338102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.844+0000 7fa33d600640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa33810d9f0 0x7fa338102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.845+0000 7fa33cc00640 1 -- 172.21.15.32:0/3091088066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa338104ee0 msgr2=0x7fa338102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:14.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.845+0000 7fa33cc00640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa338104ee0 0x7fa338102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.845+0000 7fa33cc00640 1 -- 172.21.15.32:0/3091088066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa33810d9f0 msgr2=0x7fa338102d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:14.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.845+0000 7fa33cc00640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa33810d9f0 0x7fa338102d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.845+0000 7fa33cc00640 1 -- 172.21.15.32:0/3091088066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3381a8340 con 0x7fa338104590 2024-08-31T07:34:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.845+0000 7fa33d600640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa33810d9f0 0x7fa338102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:34:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.845+0000 7fa33cc00640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa338104590 0x7fa3381022d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00e9f0 tx=0x7fa32c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.846+0000 7fa335600640 1 -- 172.21.15.32:0/3091088066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c00cd80 con 0x7fa338104590 2024-08-31T07:34:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.846+0000 7fa335600640 1 -- 172.21.15.32:0/3091088066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa32c0108e0 con 0x7fa338104590 2024-08-31T07:34:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.846+0000 7fa33e000640 1 -- 172.21.15.32:0/3091088066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3381a8630 con 0x7fa338104590 2024-08-31T07:34:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.846+0000 7fa33e000640 1 -- 172.21.15.32:0/3091088066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3381a8b40 con 0x7fa338104590 2024-08-31T07:34:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.846+0000 7fa335600640 1 -- 172.21.15.32:0/3091088066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c018980 con 0x7fa338104590 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.850+0000 7fa3461e3740 1 Processor -- start 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.850+0000 7fa3461e3740 1 -- start start 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 0x555a992c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f00b60 0x555a992c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa3461e3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555a9923d530 con 0x555a98ef8980 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa3461e3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555a9922cac0 con 0x555a98f00b60 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa3461e3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555a99230a70 con 0x555a98f21840 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa337e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f00b60 0x555a992c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa337e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f00b60 0x555a992c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59218/0 (socket says 172.21.15.32:59218) 2024-08-31T07:34:14.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa337e00640 1 -- 172.21.15.32:0/2576084139 learned_addr learned my addr 172.21.15.32:0/2576084139 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa33d600640 1 --2- 172.21.15.32:0/2576084139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 0x555a992c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa337e00640 1 -- 172.21.15.32:0/2576084139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 msgr2=0x555a992c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa337e00640 1 --2- 172.21.15.32:0/2576084139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 0x555a992c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa337e00640 1 -- 172.21.15.32:0/2576084139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 msgr2=0x555a992c8810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.851+0000 7fa337e00640 1 --2- 172.21.15.32:0/2576084139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992c8810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.852+0000 7fa337e00640 1 -- 172.21.15.32:0/2576084139 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a991df880 con 0x555a98f00b60 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.852+0000 7fa335600640 1 -- 172.21.15.32:0/3091088066 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa32c010430 con 0x7fa338104590 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.852+0000 7fa337e00640 1 --2- 172.21.15.32:0/2576084139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f00b60 0x555a992c8d50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa324005ab0 tx=0x7fa324004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f501dfa0cbfd48 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.852+0000 7fa329c00640 1 -- 172.21.15.32:0/2576084139 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324035040 con 0x555a98f00b60 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.852+0000 7fa329c00640 1 -- 172.21.15.32:0/2576084139 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3240054f0 con 0x555a98f00b60 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.852+0000 7fa329c00640 1 -- 172.21.15.32:0/2576084139 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3240035f0 con 0x555a98f00b60 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.853+0000 7fa335600640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3200b4d10 0x7fa3200b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.853+0000 7fa335600640 1 -- 172.21.15.32:0/3091088066 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8485+0+0 (secure 0 0 0) 0x7fa32c0e5750 con 0x7fa338104590 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.854+0000 7fa3461e3740 1 -- 172.21.15.32:0/2576084139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f00b60 msgr2=0x555a992c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.854+0000 7fa3461e3740 1 --2- 172.21.15.32:0/2576084139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f00b60 0x555a992c8d50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa324005ab0 tx=0x7fa324004040 comp rx=0 tx=0).stop 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.854+0000 7fa33d600640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3200b4d10 0x7fa3200b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.854+0000 7fa3461e3740 1 -- 172.21.15.32:0/2576084139 shutdown_connections 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.854+0000 7fa3461e3740 1 --2- 172.21.15.32:0/2576084139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f00b60 0x555a992c8d50 secure :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa324005ab0 tx=0x7fa324004040 comp rx=0 tx=0).stop 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.854+0000 7fa3461e3740 1 --2- 172.21.15.32:0/2576084139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.854+0000 7fa3461e3740 1 --2- 172.21.15.32:0/2576084139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 0x555a992c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.854+0000 7fa3461e3740 1 -- 172.21.15.32:0/2576084139 >> 172.21.15.32:0/2576084139 conn(0x555a98f15990 msgr2=0x555a992c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.854+0000 7fa3461e3740 1 -- 172.21.15.32:0/2576084139 shutdown_connections 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.854+0000 7fa3461e3740 1 -- 172.21.15.32:0/2576084139 wait complete. 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.855+0000 7fa3461e3740 1 -- start start 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.855+0000 7fa33d600640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3200b4d10 0x7fa3200b7220 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa330005fd0 tx=0x7fa330009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:14.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.855+0000 7fa3461e3740 1 Processor -- start 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992dced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 0x555a992dd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98ee01c0 0x555a992e24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa3461e3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555a9908f7a0 con 0x555a98ef8980 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa3461e3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555a991b37f0 con 0x555a98ee01c0 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa3461e3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555a992d6a50 con 0x555a98f21840 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa3461e3740 10 client.0 _ll_register_callbacks cb 0x555a991d8e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa33cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992dced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa33cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992dced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41194/0 (socket says 172.21.15.32:41194) 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa33cc00640 1 -- 172.21.15.32:0/1109239587 learned_addr learned my addr 172.21.15.32:0/1109239587 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa33d600640 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98ee01c0 0x555a992e24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.856+0000 7fa337e00640 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 0x555a992dd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.857+0000 7fa337e00640 1 -- 172.21.15.32:0/1109239587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98ee01c0 msgr2=0x555a992e24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.857+0000 7fa337e00640 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98ee01c0 0x555a992e24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.857+0000 7fa337e00640 1 -- 172.21.15.32:0/1109239587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 msgr2=0x555a992dced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.857+0000 7fa337e00640 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992dced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.857+0000 7fa337e00640 1 -- 172.21.15.32:0/1109239587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a98f9bb90 con 0x555a98f21840 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.857+0000 7fa33cc00640 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992dced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.857+0000 7fa337e00640 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 0x555a992dd4b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa324038000 tx=0x7fa324004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c722e567eb7de4d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.857+0000 7fa31e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.858+0000 7fa31e000640 1 -- 172.21.15.32:0/1109239587 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324052060 con 0x555a98f21840 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.858+0000 7fa31e000640 1 -- 172.21.15.32:0/1109239587 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8485+0+0 (secure 0 0 0) 0x7fa324004ca0 con 0x555a98f21840 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.858+0000 7fa3461e3740 1 -- 172.21.15.32:0/1109239587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3240054f0 con 0x555a98f21840 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.858+0000 7fa31e000640 1 -- 172.21.15.32:0/1109239587 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fa324035070 con 0x555a98f21840 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.858+0000 7fa3461e3740 10 client.29500 fetch_fsmap learned FSMap version 79 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.858+0000 7fa3461e3740 1 -- 172.21.15.32:0/1109239587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x555a992eb420 con 0x555a98f21840 2024-08-31T07:34:14.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.859+0000 7fa31e000640 1 -- 172.21.15.32:0/1109239587 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3240054f0 con 0x555a98f21840 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.859+0000 7fa31e000640 1 -- 172.21.15.32:0/1109239587 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 79) ==== 8406+0+0 (secure 0 0 0) 0x7fa324003ba0 con 0x555a98f21840 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.859+0000 7fa3461e3740 10 client.29500 fetch_fsmap finished waiting for FSMap version 79 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.859+0000 7fa3461e3740 10 client.29500 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.859+0000 7fa3461e3740 20 client.29500 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.859+0000 7fa3461e3740 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a98eb5570 0x555a992ee780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.859+0000 7fa3461e3740 4 client.29500 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.859+0000 7fa3461e3740 1 -- 172.21.15.32:0/1109239587 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a9923f480 con 0x555a98eb5570 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.860+0000 7fa33cc00640 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a98eb5570 0x555a992ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.860+0000 7fa33cc00640 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a98eb5570 0x555a992ee780 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f02ef3d5747fe84 server_cookie=dd408f9cfe2a4dd2 in_seq=0 out_seq=0 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.860+0000 7fa31e000640 10 client.29500 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.861+0000 7fa31e000640 1 -- 172.21.15.32:0/1109239587 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a9923f480 con 0x555a98eb5570 2024-08-31T07:34:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.861+0000 7fa31e000640 10 client.29500 handle_command_reply: tid=0 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.861+0000 7fa3461e3740 1 client.29500 shutdown 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.862+0000 7fa3461e3740 1 -- 172.21.15.32:0/1109239587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 msgr2=0x555a992dd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.862+0000 7fa3461e3740 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 0x555a992dd4b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa324038000 tx=0x7fa324004980 comp rx=0 tx=0).stop 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.862+0000 7fa3461e3740 1 -- 172.21.15.32:0/1109239587 shutdown_connections 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.862+0000 7fa3461e3740 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a98eb5570 0x555a992ee780 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.862+0000 7fa3461e3740 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98ee01c0 0x555a992e24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.862+0000 7fa3461e3740 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98f21840 0x555a992dd4b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.862+0000 7fa3461e3740 1 --2- 172.21.15.32:0/1109239587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992dced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.862+0000 7fa3461e3740 1 -- 172.21.15.32:0/1109239587 >> 172.21.15.32:0/1109239587 conn(0x555a98f15990 msgr2=0x555a992cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.862+0000 7fa31e000640 0 client.29500 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.862+0000 7fa3461e3740 1 -- 172.21.15.32:0/1109239587 shutdown_connections 2024-08-31T07:34:14.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.863+0000 7fa3461e3740 1 -- 172.21.15.32:0/1109239587 wait complete. 2024-08-31T07:34:14.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.863+0000 7fa3461e3740 20 client.29500 trim_cache size 0 max 16384 2024-08-31T07:34:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.882+0000 7fa3461e3740 1 Processor -- start 2024-08-31T07:34:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.882+0000 7fa3461e3740 1 -- start start 2024-08-31T07:34:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.882+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 0x555a992d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.882+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992d6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.882+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 0x555a992d7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa3461e3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa31800e750 con 0x555a98ef8980 2024-08-31T07:34:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa3461e3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa3180104a0 con 0x555a98f21840 2024-08-31T07:34:14.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa3461e3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa31800eee0 con 0x555a98ee01c0 2024-08-31T07:34:14.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa337e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992d6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa33cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 0x555a992d5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa33d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 0x555a992d7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa337e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992d6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41206/0 (socket says 172.21.15.32:41206) 2024-08-31T07:34:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa337e00640 1 -- 172.21.15.32:0/3658778289 learned_addr learned my addr 172.21.15.32:0/3658778289 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa33d600640 1 -- 172.21.15.32:0/3658778289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 msgr2=0x555a992d5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa33d600640 1 --2- 172.21.15.32:0/3658778289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 0x555a992d5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa33d600640 1 -- 172.21.15.32:0/3658778289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 msgr2=0x555a992d6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa33d600640 1 --2- 172.21.15.32:0/3658778289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992d6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.883+0000 7fa33d600640 1 -- 172.21.15.32:0/3658778289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a992dbc30 con 0x555a98f21840 2024-08-31T07:34:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.884+0000 7fa337e00640 1 --2- 172.21.15.32:0/3658778289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992d6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:14.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.884+0000 7fa33d600640 1 --2- 172.21.15.32:0/3658778289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 0x555a992d7230 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa330012010 tx=0x7fa330014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d10b9be95f8aa4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:14.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.886+0000 7fa329200640 1 -- 172.21.15.32:0/3658778289 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330020b80 con 0x555a98f21840 2024-08-31T07:34:14.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.886+0000 7fa329200640 1 -- 172.21.15.32:0/3658778289 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa330020d20 con 0x555a98f21840 2024-08-31T07:34:14.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.886+0000 7fa329200640 1 -- 172.21.15.32:0/3658778289 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33001b600 con 0x555a98f21840 2024-08-31T07:34:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.887+0000 7fa3461e3740 1 -- 172.21.15.32:0/3658778289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 msgr2=0x555a992d7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:14.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.887+0000 7fa3461e3740 1 --2- 172.21.15.32:0/3658778289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 0x555a992d7230 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa330012010 tx=0x7fa330014000 comp rx=0 tx=0).stop 2024-08-31T07:34:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.887+0000 7fa3461e3740 1 -- 172.21.15.32:0/3658778289 shutdown_connections 2024-08-31T07:34:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.887+0000 7fa3461e3740 1 --2- 172.21.15.32:0/3658778289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 0x555a992d7230 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.887+0000 7fa3461e3740 1 --2- 172.21.15.32:0/3658778289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992d6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.887+0000 7fa3461e3740 1 --2- 172.21.15.32:0/3658778289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 0x555a992d5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.887+0000 7fa3461e3740 1 -- 172.21.15.32:0/3658778289 >> 172.21.15.32:0/3658778289 conn(0x555a98f15990 msgr2=0x555a992c5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.889+0000 7fa3461e3740 1 -- 172.21.15.32:0/3658778289 shutdown_connections 2024-08-31T07:34:14.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.889+0000 7fa3461e3740 1 -- 172.21.15.32:0/3658778289 wait complete. 2024-08-31T07:34:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.889+0000 7fa3461e3740 1 -- start start 2024-08-31T07:34:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.889+0000 7fa3461e3740 1 Processor -- start 2024-08-31T07:34:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.890+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 0x555a992e3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.890+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.890+0000 7fa3461e3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 0x555a992e8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.890+0000 7fa3461e3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa31800fd40 con 0x555a98ef8980 2024-08-31T07:34:14.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.890+0000 7fa3461e3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa31800f640 con 0x555a98f21840 2024-08-31T07:34:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.890+0000 7fa3461e3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa31800c750 con 0x555a98ee01c0 2024-08-31T07:34:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.890+0000 7fa3461e3740 10 client.0 _ll_register_callbacks cb 0x7fa3240056a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.890+0000 7fa33cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 0x555a992e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.890+0000 7fa337e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.891+0000 7fa337e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992e3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41210/0 (socket says 172.21.15.32:41210) 2024-08-31T07:34:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.891+0000 7fa33cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 0x555a992e3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:43674/0 (socket says 172.21.15.32:43674) 2024-08-31T07:34:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.891+0000 7fa337e00640 1 -- 172.21.15.32:0/3362868273 learned_addr learned my addr 172.21.15.32:0/3362868273 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.891+0000 7fa33d600640 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 0x555a992e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.891+0000 7fa337e00640 1 -- 172.21.15.32:0/3362868273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 msgr2=0x555a992e3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.891+0000 7fa337e00640 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 0x555a992e3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.891+0000 7fa337e00640 1 -- 172.21.15.32:0/3362868273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 msgr2=0x555a992e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.891+0000 7fa337e00640 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 0x555a992e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.891+0000 7fa337e00640 1 -- 172.21.15.32:0/3362868273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a992e8bc0 con 0x555a98ef8980 2024-08-31T07:34:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.891+0000 7fa337e00640 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992e3d00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa3240096c0 tx=0x7fa324005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fdd66c581d4b39 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.892+0000 7fa31e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:34:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.892+0000 7fa31e000640 1 -- 172.21.15.32:0/3362868273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324052040 con 0x555a98ef8980 2024-08-31T07:34:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.892+0000 7fa3461e3740 1 -- 172.21.15.32:0/3362868273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa330020d20 con 0x555a98ef8980 2024-08-31T07:34:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.892+0000 7fa31e000640 1 -- 172.21.15.32:0/3362868273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8485+0+0 (secure 0 0 0) 0x7fa324003800 con 0x555a98ef8980 2024-08-31T07:34:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.892+0000 7fa31e000640 1 -- 172.21.15.32:0/3362868273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fa324004010 con 0x555a98ef8980 2024-08-31T07:34:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.892+0000 7fa3461e3740 10 client.19464 fetch_fsmap learned FSMap version 79 2024-08-31T07:34:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.892+0000 7fa3461e3740 1 -- 172.21.15.32:0/3362868273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x555a993593e0 con 0x555a98ef8980 2024-08-31T07:34:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.893+0000 7fa31e000640 1 -- 172.21.15.32:0/3362868273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa330020d20 con 0x555a98ef8980 2024-08-31T07:34:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.893+0000 7fa31e000640 1 -- 172.21.15.32:0/3362868273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 79) ==== 8406+0+0 (secure 0 0 0) 0x7fa324005070 con 0x555a98ef8980 2024-08-31T07:34:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.893+0000 7fa3461e3740 10 client.19464 fetch_fsmap finished waiting for FSMap version 79 2024-08-31T07:34:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.893+0000 7fa3461e3740 10 client.19464 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:34:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.893+0000 7fa3461e3740 20 client.19464 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.893+0000 7fa3461e3740 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a99359c30 0x555a9935c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.893+0000 7fa3461e3740 4 client.19464 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.1, "await_for": 45.0}] 2024-08-31T07:34:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.893+0000 7fa3461e3740 1 -- 172.21.15.32:0/3362868273 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.1, "await_for": 45.0}) -- 0x555a9923af20 con 0x555a99359c30 2024-08-31T07:34:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.894+0000 7fa33cc00640 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a99359c30 0x555a9935c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.894+0000 7fa33cc00640 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a99359c30 0x555a9935c050 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f3b052abab5e7b6 server_cookie=a2b7aed507923cab in_seq=0 out_seq=0 2024-08-31T07:34:14.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:14.894+0000 7fa31e000640 10 client.19464 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:15.106 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:14 smithi032 ceph-mon[17252]: pgmap v1043: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 212 op/s 2024-08-31T07:34:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:14 smithi073 ceph-mon[21995]: pgmap v1043: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 212 op/s 2024-08-31T07:34:15.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:14 smithi084 ceph-mon[22387]: pgmap v1043: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 212 op/s 2024-08-31T07:34:15.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142701 14.06 MB/sec execute 355 sec latency 16342.896 ms 2024-08-31T07:34:16.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142701 14.02 MB/sec execute 356 sec latency 17342.947 ms 2024-08-31T07:34:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.940+0000 7f9728000640 1 -- 172.21.15.32:0/1429314593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9720105b30 msgr2=0x7f9720105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.940+0000 7f9728000640 1 --2- 172.21.15.32:0/1429314593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9720105b30 0x7f9720105f40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9714009900 tx=0x7f971402a400 comp rx=0 tx=0).stop 2024-08-31T07:34:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.940+0000 7f9728000640 1 -- 172.21.15.32:0/1429314593 shutdown_connections 2024-08-31T07:34:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.940+0000 7f9728000640 1 --2- 172.21.15.32:0/1429314593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f972010bdc0 0x7f97201125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.940+0000 7f9728000640 1 --2- 172.21.15.32:0/1429314593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972010b470 0x7f972010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.940+0000 7f9728000640 1 --2- 172.21.15.32:0/1429314593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9720105b30 0x7f9720105f40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.940+0000 7f9728000640 1 -- 172.21.15.32:0/1429314593 >> 172.21.15.32:0/1429314593 conn(0x7f9720101600 msgr2=0x7f9720103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:16.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.940+0000 7f9728000640 1 -- 172.21.15.32:0/1429314593 shutdown_connections 2024-08-31T07:34:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.940+0000 7f9728000640 1 -- 172.21.15.32:0/1429314593 wait complete. 2024-08-31T07:34:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.941+0000 7f9728000640 1 Processor -- start 2024-08-31T07:34:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.941+0000 7f9728000640 1 -- start start 2024-08-31T07:34:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9728000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720105b30 0x7f972019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9728000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f972010b470 0x7f972019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9728000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f972010bdc0 0x7f97201a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9728000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9720115ad0 con 0x7f9720105b30 2024-08-31T07:34:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9728000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9720115950 con 0x7f972010bdc0 2024-08-31T07:34:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9728000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9720115c50 con 0x7f972010b470 2024-08-31T07:34:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9724c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f972010b470 0x7f972019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9725600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720105b30 0x7f972019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9724c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f972010b470 0x7f972019c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50410/0 (socket says 172.21.15.32:50410) 2024-08-31T07:34:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9724c00640 1 -- 172.21.15.32:0/2208033790 learned_addr learned my addr 172.21.15.32:0/2208033790 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9726000640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f972010bdc0 0x7f97201a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9724c00640 1 -- 172.21.15.32:0/2208033790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f972010bdc0 msgr2=0x7f97201a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9724c00640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f972010bdc0 0x7f97201a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9724c00640 1 -- 172.21.15.32:0/2208033790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720105b30 msgr2=0x7f972019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9724c00640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720105b30 0x7f972019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.942+0000 7f9724c00640 1 -- 172.21.15.32:0/2208033790 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97201a23b0 con 0x7f972010b470 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.943+0000 7f9724c00640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f972010b470 0x7f972019c9e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f970c00e970 tx=0x7f970c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.943+0000 7f971e000640 1 -- 172.21.15.32:0/2208033790 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c00cd80 con 0x7f972010b470 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.943+0000 7f971e000640 1 -- 172.21.15.32:0/2208033790 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f970c0035f0 con 0x7f972010b470 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.943+0000 7f9728000640 1 -- 172.21.15.32:0/2208033790 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97201a26a0 con 0x7f972010b470 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.943+0000 7f971e000640 1 -- 172.21.15.32:0/2208033790 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c010690 con 0x7f972010b470 2024-08-31T07:34:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.943+0000 7f9728000640 1 -- 172.21.15.32:0/2208033790 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97201a8e00 con 0x7f972010b470 2024-08-31T07:34:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.946+0000 7f972ec58740 1 Processor -- start 2024-08-31T07:34:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f972ec58740 1 -- start start 2024-08-31T07:34:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f972ec58740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3fdc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f972ec58740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fdced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f972ec58740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c16390 0x55d7c3fdd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f972ec58740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d7c3f537b0 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f972ec58740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d7c3f42d40 con 0x55d7c3c16390 2024-08-31T07:34:16.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f972ec58740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d7c3f46cf0 con 0x55d7c3c37070 2024-08-31T07:34:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f9725600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fdced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f9726000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3fdc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f9726000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3fdc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50424/0 (socket says 172.21.15.32:50424) 2024-08-31T07:34:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f9725600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fdced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40980/0 (socket says 172.21.15.32:40980) 2024-08-31T07:34:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.947+0000 7f9726000640 1 -- 172.21.15.32:0/1446160626 learned_addr learned my addr 172.21.15.32:0/1446160626 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f971e000640 1 -- 172.21.15.32:0/2208033790 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f970c003aa0 con 0x7f972010b470 2024-08-31T07:34:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f9726000640 1 -- 172.21.15.32:0/1446160626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c16390 msgr2=0x55d7c3fdd410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:34:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f9726000640 1 --2- 172.21.15.32:0/1446160626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c16390 0x55d7c3fdd410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f9726000640 1 -- 172.21.15.32:0/1446160626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c0e1b0 msgr2=0x55d7c3fdced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f9726000640 1 --2- 172.21.15.32:0/1446160626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fdced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f9726000640 1 -- 172.21.15.32:0/1446160626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c3ca2650 con 0x55d7c3c37070 2024-08-31T07:34:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f9725600640 1 --2- 172.21.15.32:0/1446160626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fdced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f9726000640 1 --2- 172.21.15.32:0/1446160626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3fdc990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9718009fd0 tx=0x7f97180118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725ae9b393e025ee server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f9712600640 1 -- 172.21.15.32:0/1446160626 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718004d20 con 0x55d7c3c37070 2024-08-31T07:34:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f9712600640 1 -- 172.21.15.32:0/1446160626 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9718004050 con 0x55d7c3c37070 2024-08-31T07:34:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.948+0000 7f9712600640 1 -- 172.21.15.32:0/1446160626 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718005280 con 0x55d7c3c37070 2024-08-31T07:34:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.949+0000 7f972ec58740 1 -- 172.21.15.32:0/1446160626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 msgr2=0x55d7c3fdc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.949+0000 7f972ec58740 1 --2- 172.21.15.32:0/1446160626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3fdc990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9718009fd0 tx=0x7f97180118d0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.949+0000 7f972ec58740 1 -- 172.21.15.32:0/1446160626 shutdown_connections 2024-08-31T07:34:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.949+0000 7f972ec58740 1 --2- 172.21.15.32:0/1446160626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c16390 0x55d7c3fdd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.949+0000 7f972ec58740 1 --2- 172.21.15.32:0/1446160626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fdced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.949+0000 7f972ec58740 1 --2- 172.21.15.32:0/1446160626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3fdc990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.949+0000 7f972ec58740 1 -- 172.21.15.32:0/1446160626 >> 172.21.15.32:0/1446160626 conn(0x55d7c3c2b460 msgr2=0x55d7c3fda880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.949+0000 7f972ec58740 1 -- 172.21.15.32:0/1446160626 shutdown_connections 2024-08-31T07:34:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.950+0000 7f972ec58740 1 -- 172.21.15.32:0/1446160626 wait complete. 2024-08-31T07:34:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.950+0000 7f971e000640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97080b4bb0 0x7f97080b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.950+0000 7f971e000640 1 -- 172.21.15.32:0/2208033790 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8485+0+0 (secure 0 0 0) 0x7f970c0e57d0 con 0x7f972010b470 2024-08-31T07:34:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.950+0000 7f9725600640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97080b4bb0 0x7f97080b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.950+0000 7f972ec58740 1 -- start start 2024-08-31T07:34:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.950+0000 7f972ec58740 1 Processor -- start 2024-08-31T07:34:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.951+0000 7f9725600640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97080b4bb0 0x7f97080b70c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9714005ab0 tx=0x7f9714005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.951+0000 7f972ec58740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.951+0000 7f972ec58740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3fe8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.951+0000 7f972ec58740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c37070 0x55d7c3ff4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.951+0000 7f972ec58740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d7c3da5700 con 0x55d7c3c16390 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.951+0000 7f972ec58740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d7c3ec9cf0 con 0x55d7c3c37070 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.951+0000 7f972ec58740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d7c3feb170 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.951+0000 7f972ec58740 10 client.0 _ll_register_callbacks cb 0x55d7c3eef470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9725600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9725600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50438/0 (socket says 172.21.15.32:50438) 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9725600640 1 -- 172.21.15.32:0/999146176 learned_addr learned my addr 172.21.15.32:0/999146176 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9726000640 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c37070 0x55d7c3ff4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9724c00640 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3fe8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9725600640 1 -- 172.21.15.32:0/999146176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c37070 msgr2=0x55d7c3ff4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9725600640 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c37070 0x55d7c3ff4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9725600640 1 -- 172.21.15.32:0/999146176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 msgr2=0x55d7c3fe8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9725600640 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3fe8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9725600640 1 -- 172.21.15.32:0/999146176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c3f0a8b0 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.952+0000 7f9726000640 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c37070 0x55d7c3ff4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.953+0000 7f9725600640 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe8580 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9714033000 tx=0x7f9714009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff8ba10351c52dc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.953+0000 7f9706a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:34:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.953+0000 7f9706a00640 1 -- 172.21.15.32:0/999146176 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971404e070 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.953+0000 7f9706a00640 1 -- 172.21.15.32:0/999146176 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8485+0+0 (secure 0 0 0) 0x7f9714004b80 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.953+0000 7f972ec58740 1 -- 172.21.15.32:0/999146176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9718004050 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.953+0000 7f9706a00640 1 -- 172.21.15.32:0/999146176 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f971405a050 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.953+0000 7f972ec58740 10 client.29518 fetch_fsmap learned FSMap version 80 2024-08-31T07:34:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.953+0000 7f972ec58740 1 -- 172.21.15.32:0/999146176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55d7c3ff71d0 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.954+0000 7f9706a00640 1 -- 172.21.15.32:0/999146176 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9718004050 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.954+0000 7f9706a00640 1 -- 172.21.15.32:0/999146176 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 80) ==== 8402+0+0 (secure 0 0 0) 0x7f9714032540 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.954+0000 7f972ec58740 10 client.29518 fetch_fsmap finished waiting for FSMap version 80 2024-08-31T07:34:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.954+0000 7f972ec58740 10 client.29518 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:34:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.954+0000 7f972ec58740 20 client.29518 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.954+0000 7f972ec58740 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55d7c3bf59f0 0x55d7c3ffa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.954+0000 7f972ec58740 4 client.29518 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:34:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.954+0000 7f972ec58740 1 -- 172.21.15.32:0/999146176 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7c3f55700 con 0x55d7c3bf59f0 2024-08-31T07:34:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.954+0000 7f9724c00640 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55d7c3bf59f0 0x55d7c3ffa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.955+0000 7f9724c00640 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55d7c3bf59f0 0x55d7c3ffa710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa77dec4bda90700 server_cookie=ea27b7850267ddc4 in_seq=0 out_seq=0 2024-08-31T07:34:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.955+0000 7f9706a00640 10 client.29518 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:34:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f9706a00640 1 -- 172.21.15.32:0/999146176 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7c3f55700 con 0x55d7c3bf59f0 2024-08-31T07:34:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f9706a00640 10 client.29518 handle_command_reply: tid=0 2024-08-31T07:34:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 client.29518 shutdown 2024-08-31T07:34:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 -- 172.21.15.32:0/999146176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c0e1b0 msgr2=0x55d7c3fe8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe8580 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9714033000 tx=0x7f9714009260 comp rx=0 tx=0).stop 2024-08-31T07:34:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 -- 172.21.15.32:0/999146176 shutdown_connections 2024-08-31T07:34:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55d7c3bf59f0 0x55d7c3ffa710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c37070 0x55d7c3ff4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3fe8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 --2- 172.21.15.32:0/999146176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe8580 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 -- 172.21.15.32:0/999146176 >> 172.21.15.32:0/999146176 conn(0x55d7c3c2b460 msgr2=0x55d7c3fdfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f9706a00640 0 client.29518 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:34:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 -- 172.21.15.32:0/999146176 shutdown_connections 2024-08-31T07:34:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.956+0000 7f972ec58740 1 -- 172.21.15.32:0/999146176 wait complete. 2024-08-31T07:34:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.957+0000 7f972ec58740 20 client.29518 trim_cache size 0 max 16384 2024-08-31T07:34:16.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f972ec58740 1 Processor -- start 2024-08-31T07:34:16.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f972ec58740 1 -- start start 2024-08-31T07:34:16.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f972ec58740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f972ec58740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3fe1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f972ec58740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3fe2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f972ec58740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f97201b16d0 con 0x55d7c3c16390 2024-08-31T07:34:16.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f972ec58740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f97201b33f0 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f972ec58740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f97201b1e60 con 0x55d7c3c37070 2024-08-31T07:34:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f9724c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3fe1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f9725600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f9725600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42010/0 (socket says 172.21.15.32:42010) 2024-08-31T07:34:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.971+0000 7f9725600640 1 -- 172.21.15.32:0/3083716683 learned_addr learned my addr 172.21.15.32:0/3083716683 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.972+0000 7f9726000640 1 --2- 172.21.15.32:0/3083716683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3fe2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.972+0000 7f9725600640 1 -- 172.21.15.32:0/3083716683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 msgr2=0x55d7c3fe2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.972+0000 7f9725600640 1 --2- 172.21.15.32:0/3083716683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3fe2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.972+0000 7f9725600640 1 -- 172.21.15.32:0/3083716683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 msgr2=0x55d7c3fe1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.972+0000 7f9725600640 1 --2- 172.21.15.32:0/3083716683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3fe1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.972+0000 7f9725600640 1 -- 172.21.15.32:0/3083716683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c3fe9640 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.972+0000 7f9725600640 1 --2- 172.21.15.32:0/3083716683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe9fb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f97140328e0 tx=0x7f9714032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4661b394d7243537 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.973+0000 7f9711c00640 1 -- 172.21.15.32:0/3083716683 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714062070 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.973+0000 7f9711c00640 1 -- 172.21.15.32:0/3083716683 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97140326b0 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.973+0000 7f9711c00640 1 -- 172.21.15.32:0/3083716683 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971404e050 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.973+0000 7f972ec58740 1 -- 172.21.15.32:0/3083716683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 msgr2=0x55d7c3fe9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.973+0000 7f972ec58740 1 --2- 172.21.15.32:0/3083716683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe9fb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f97140328e0 tx=0x7f9714032ce0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.973+0000 7f972ec58740 1 -- 172.21.15.32:0/3083716683 shutdown_connections 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.973+0000 7f972ec58740 1 --2- 172.21.15.32:0/3083716683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3fe2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.973+0000 7f972ec58740 1 --2- 172.21.15.32:0/3083716683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3fe1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.973+0000 7f972ec58740 1 --2- 172.21.15.32:0/3083716683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe9fb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.973+0000 7f972ec58740 1 -- 172.21.15.32:0/3083716683 >> 172.21.15.32:0/3083716683 conn(0x55d7c3c2b460 msgr2=0x55d7c3fe98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.974+0000 7f972ec58740 1 -- 172.21.15.32:0/3083716683 shutdown_connections 2024-08-31T07:34:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.974+0000 7f972ec58740 1 -- 172.21.15.32:0/3083716683 wait complete. 2024-08-31T07:34:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.974+0000 7f972ec58740 1 -- start start 2024-08-31T07:34:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.974+0000 7f972ec58740 1 Processor -- start 2024-08-31T07:34:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f972ec58740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f972ec58740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3ff03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f972ec58740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3ff0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f972ec58740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f97201b2c90 con 0x55d7c3c16390 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f972ec58740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f97201b2590 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f972ec58740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f97201af6d0 con 0x55d7c3c37070 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f972ec58740 10 client.0 _ll_register_callbacks cb 0x7f9714004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f9724c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3ff03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f9725600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f9724c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3ff03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41000/0 (socket says 172.21.15.32:41000) 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f9724c00640 1 -- 172.21.15.32:0/3756271617 learned_addr learned my addr 172.21.15.32:0/3756271617 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f9725600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42018/0 (socket says 172.21.15.32:42018) 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.975+0000 7f9726000640 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3ff0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.976+0000 7f9725600640 1 -- 172.21.15.32:0/3756271617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 msgr2=0x55d7c3ff0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.976+0000 7f9725600640 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3ff0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.976+0000 7f9725600640 1 -- 172.21.15.32:0/3756271617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 msgr2=0x55d7c3ff03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.976+0000 7f9725600640 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3ff03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.976+0000 7f9725600640 1 -- 172.21.15.32:0/3756271617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c3ff0ff0 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.976+0000 7f9724c00640 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3ff03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.976+0000 7f9725600640 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe5d50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9714032960 tx=0x7f9714009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8a88ec101c8c0fb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.976+0000 7f9706a00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.976+0000 7f9706a00640 1 -- 172.21.15.32:0/3756271617 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971406b020 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f972ec58740 1 -- 172.21.15.32:0/3756271617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97140326b0 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f9706a00640 1 -- 172.21.15.32:0/3756271617 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8485+0+0 (secure 0 0 0) 0x7f9714004b90 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f9706a00640 1 -- 172.21.15.32:0/3756271617 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f9714076050 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f972ec58740 10 client.29531 fetch_fsmap learned FSMap version 80 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f972ec58740 1 -- 172.21.15.32:0/3756271617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55d7c4065120 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f9706a00640 1 -- 172.21.15.32:0/3756271617 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97140326b0 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f9706a00640 1 -- 172.21.15.32:0/3756271617 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 80) ==== 8402+0+0 (secure 0 0 0) 0x7f971404a650 con 0x55d7c3c0e1b0 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f972ec58740 10 client.29531 fetch_fsmap finished waiting for FSMap version 80 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f972ec58740 10 client.29531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f972ec58740 20 client.29531 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f972ec58740 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55d7c3bcada0 0x55d7c40677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f972ec58740 4 client.29531 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.977+0000 7f972ec58740 1 -- 172.21.15.32:0/3756271617 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7c3f511a0 con 0x55d7c3bcada0 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.978+0000 7f9724c00640 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55d7c3bcada0 0x55d7c40677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.978+0000 7f9724c00640 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55d7c3bcada0 0x55d7c40677a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ec5ed938dca7e89 server_cookie=bfeaed04b134e47f in_seq=0 out_seq=0 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.978+0000 7f9706a00640 10 client.29531 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f9706a00640 1 -- 172.21.15.32:0/3756271617 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7c3f511a0 con 0x55d7c3bcada0 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f9706a00640 10 client.29531 handle_command_reply: tid=0 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 client.29531 shutdown 2024-08-31T07:34:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 -- 172.21.15.32:0/3756271617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 msgr2=0x55d7c3fe5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe5d50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9714032960 tx=0x7f9714009260 comp rx=0 tx=0).stop 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 -- 172.21.15.32:0/3756271617 shutdown_connections 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55d7c3bcada0 0x55d7c40677a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7c3c37070 0x55d7c3ff0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7c3c16390 0x55d7c3ff03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 --2- 172.21.15.32:0/3756271617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7c3c0e1b0 0x55d7c3fe5d50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 -- 172.21.15.32:0/3756271617 >> 172.21.15.32:0/3756271617 conn(0x55d7c3c2b460 msgr2=0x55d7c3fdd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f9706a00640 0 client.29531 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 -- 172.21.15.32:0/3756271617 shutdown_connections 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 1 -- 172.21.15.32:0/3756271617 wait complete. 2024-08-31T07:34:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.979+0000 7f972ec58740 20 client.29531 trim_cache size 0 max 16384 2024-08-31T07:34:16.986 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:34:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 -- 172.21.15.32:0/2208033790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97080b4bb0 msgr2=0x7f97080b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97080b4bb0 0x7f97080b70c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9714005ab0 tx=0x7f9714005a20 comp rx=0 tx=0).stop 2024-08-31T07:34:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 -- 172.21.15.32:0/2208033790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f972010b470 msgr2=0x7f972019c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f972010b470 0x7f972019c9e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f970c00e970 tx=0x7f970c00ee30 comp rx=0 tx=0).stop 2024-08-31T07:34:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 -- 172.21.15.32:0/2208033790 shutdown_connections 2024-08-31T07:34:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f97080b4bb0 0x7f97080b70c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f972010bdc0 0x7f97201a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f972010b470 0x7f972019c9e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 --2- 172.21.15.32:0/2208033790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9720105b30 0x7f972019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 -- 172.21.15.32:0/2208033790 >> 172.21.15.32:0/2208033790 conn(0x7f9720101600 msgr2=0x7f972010aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 -- 172.21.15.32:0/2208033790 shutdown_connections 2024-08-31T07:34:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:16.980+0000 7f9728000640 1 -- 172.21.15.32:0/2208033790 wait complete. 2024-08-31T07:34:17.012 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:16 smithi032 ceph-mon[17252]: pgmap v1044: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 454 KiB/s wr, 82 op/s 2024-08-31T07:34:17.013 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:16 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:17.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:16 smithi073 ceph-mon[21995]: pgmap v1044: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 454 KiB/s wr, 82 op/s 2024-08-31T07:34:17.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:16 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:17.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:16 smithi084 ceph-mon[22387]: pgmap v1044: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 454 KiB/s wr, 82 op/s 2024-08-31T07:34:17.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:16 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:17.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142701 13.98 MB/sec execute 357 sec latency 18342.996 ms 2024-08-31T07:34:17.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:34:18.097 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:18 smithi032 ceph-mon[17252]: osdmap e112: 12 total, 12 up, 12 in 2024-08-31T07:34:18.097 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:18 smithi032 ceph-mon[17252]: pgmap v1046: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-08-31T07:34:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:18.418 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142701 13.94 MB/sec execute 358 sec latency 19343.041 ms 2024-08-31T07:34:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:18 smithi073 ceph-mon[21995]: osdmap e112: 12 total, 12 up, 12 in 2024-08-31T07:34:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:18 smithi073 ceph-mon[21995]: pgmap v1046: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-08-31T07:34:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:18 smithi084 ceph-mon[22387]: osdmap e112: 12 total, 12 up, 12 in 2024-08-31T07:34:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:18 smithi084 ceph-mon[22387]: pgmap v1046: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-08-31T07:34:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa31e000640 1 -- 172.21.15.32:0/3362868273 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555a9923af20 con 0x555a99359c30 2024-08-31T07:34:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa31e000640 10 client.19464 handle_command_reply: tid=0 2024-08-31T07:34:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa3461e3740 1 client.19464 shutdown 2024-08-31T07:34:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa3461e3740 1 -- 172.21.15.32:0/3362868273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 msgr2=0x555a992e3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa3461e3740 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992e3d00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa3240096c0 tx=0x7fa324005a20 comp rx=0 tx=0).stop 2024-08-31T07:34:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa3461e3740 1 -- 172.21.15.32:0/3362868273 shutdown_connections 2024-08-31T07:34:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa3461e3740 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555a99359c30 0x555a9935c050 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa3461e3740 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555a98f21840 0x555a992e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa3461e3740 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a98ef8980 0x555a992e3d00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa3461e3740 1 --2- 172.21.15.32:0/3362868273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555a98ee01c0 0x555a992e3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa3461e3740 1 -- 172.21.15.32:0/3362868273 >> 172.21.15.32:0/3362868273 conn(0x555a98f15990 msgr2=0x555a992d6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.017+0000 7fa31e000640 0 client.19464 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.018+0000 7fa3461e3740 1 -- 172.21.15.32:0/3362868273 shutdown_connections 2024-08-31T07:34:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.018+0000 7fa3461e3740 1 -- 172.21.15.32:0/3362868273 wait complete. 2024-08-31T07:34:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.018+0000 7fa3461e3740 20 client.19464 trim_cache size 0 max 16384 2024-08-31T07:34:19.022 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:34:19.022 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 80, 2024-08-31T07:34:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:34:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 66, 2024-08-31T07:34:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:34:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "e3f2fbef": { 2024-08-31T07:34:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 66, 2024-08-31T07:34:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:34:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:34:19.023 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:34:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:34:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:34:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 134.1, 2024-08-31T07:34:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:34:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:34:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:34:19.024 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:34:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:34:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:34:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:19.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:19.025 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 -- 172.21.15.32:0/3091088066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3200b4d10 msgr2=0x7fa3200b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3200b4d10 0x7fa3200b7220 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa330005fd0 tx=0x7fa330009500 comp rx=0 tx=0).stop 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 -- 172.21.15.32:0/3091088066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa338104590 msgr2=0x7fa3381022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa338104590 0x7fa3381022d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00e9f0 tx=0x7fa32c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 -- 172.21.15.32:0/3091088066 shutdown_connections 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3200b4d10 0x7fa3200b7220 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa33810d9f0 0x7fa338102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa338104ee0 0x7fa338102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 --2- 172.21.15.32:0/3091088066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa338104590 0x7fa3381022d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 -- 172.21.15.32:0/3091088066 >> 172.21.15.32:0/3091088066 conn(0x7fa3380fe0d0 msgr2=0x7fa338107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 -- 172.21.15.32:0/3091088066 shutdown_connections 2024-08-31T07:34:19.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.019+0000 7fa33e000640 1 -- 172.21.15.32:0/3091088066 wait complete. 2024-08-31T07:34:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.173+0000 7f35a2600640 1 -- 172.21.15.32:0/1273996391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c10b290 msgr2=0x7f359c10b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.173+0000 7f35a2600640 1 --2- 172.21.15.32:0/1273996391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c10b290 0x7f359c10b6a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3588009900 tx=0x7f358802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.173+0000 7f35a2600640 1 -- 172.21.15.32:0/1273996391 shutdown_connections 2024-08-31T07:34:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.173+0000 7f35a2600640 1 --2- 172.21.15.32:0/1273996391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f359c10bbe0 0x7f359c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.173+0000 7f35a2600640 1 --2- 172.21.15.32:0/1273996391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c10b290 0x7f359c10b6a0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.173+0000 7f35a2600640 1 --2- 172.21.15.32:0/1273996391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f359c105950 0x7f359c105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.173+0000 7f35a2600640 1 -- 172.21.15.32:0/1273996391 >> 172.21.15.32:0/1273996391 conn(0x7f359c101420 msgr2=0x7f359c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.173+0000 7f35a2600640 1 -- 172.21.15.32:0/1273996391 shutdown_connections 2024-08-31T07:34:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.174+0000 7f35a2600640 1 -- 172.21.15.32:0/1273996391 wait complete. 2024-08-31T07:34:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.175+0000 7f35a2600640 1 Processor -- start 2024-08-31T07:34:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.175+0000 7f35a2600640 1 -- start start 2024-08-31T07:34:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.175+0000 7f35a2600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c105950 0x7f359c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.175+0000 7f35a2600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f359c10b290 0x7f359c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.175+0000 7f35a2600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f359c10bbe0 0x7f359c1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.175+0000 7f35a2600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1158a0 con 0x7f359c105950 2024-08-31T07:34:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.175+0000 7f35a2600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f359c115720 con 0x7f359c10b290 2024-08-31T07:34:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.175+0000 7f35a2600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f359c115a20 con 0x7f359c10bbe0 2024-08-31T07:34:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f35a1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c105950 0x7f359c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f359be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f359c10b290 0x7f359c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f35a1c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f359c10bbe0 0x7f359c1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f35a1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c105950 0x7f359c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41024/0 (socket says 172.21.15.32:41024) 2024-08-31T07:34:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f35a1200640 1 -- 172.21.15.32:0/2068526255 learned_addr learned my addr 172.21.15.32:0/2068526255 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:19.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f35a1c00640 1 -- 172.21.15.32:0/2068526255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f359c10b290 msgr2=0x7f359c19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f35a1c00640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f359c10b290 0x7f359c19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f35a1c00640 1 -- 172.21.15.32:0/2068526255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c105950 msgr2=0x7f359c19c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f35a1c00640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c105950 0x7f359c19c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f35a1c00640 1 -- 172.21.15.32:0/2068526255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c1a2220 con 0x7f359c10bbe0 2024-08-31T07:34:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.176+0000 7f35a1c00640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f359c10bbe0 0x7f359c1a1af0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3594009fd0 tx=0x7f359400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.177+0000 7f3599600640 1 -- 172.21.15.32:0/2068526255 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400ce20 con 0x7f359c10bbe0 2024-08-31T07:34:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.177+0000 7f3599600640 1 -- 172.21.15.32:0/2068526255 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594005020 con 0x7f359c10bbe0 2024-08-31T07:34:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.177+0000 7f3599600640 1 -- 172.21.15.32:0/2068526255 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594010690 con 0x7f359c10bbe0 2024-08-31T07:34:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.177+0000 7f35a2600640 1 -- 172.21.15.32:0/2068526255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c1a2510 con 0x7f359c10bbe0 2024-08-31T07:34:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.177+0000 7f35a2600640 1 -- 172.21.15.32:0/2068526255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c1a8c70 con 0x7f359c10bbe0 2024-08-31T07:34:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.180+0000 7f35aa855740 1 Processor -- start 2024-08-31T07:34:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.180+0000 7f3599600640 1 -- 172.21.15.32:0/2068526255 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f35940054d0 con 0x7f359c10bbe0 2024-08-31T07:34:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f35aa855740 1 -- start start 2024-08-31T07:34:19.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f35aa855740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462af5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f35aa855740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462af60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f35aa855740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746272f390 0x557462af65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f35aa855740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557462a6c7b0 con 0x55746272f390 2024-08-31T07:34:19.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f35aa855740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557462a5bd40 con 0x557462750070 2024-08-31T07:34:19.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f35aa855740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557462a5fcf0 con 0x5574627271b0 2024-08-31T07:34:19.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f35a1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462af60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f35a1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462af60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50482/0 (socket says 172.21.15.32:50482) 2024-08-31T07:34:19.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f35a1200640 1 -- 172.21.15.32:0/2517903742 learned_addr learned my addr 172.21.15.32:0/2517903742 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f359be00640 1 --2- 172.21.15.32:0/2517903742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746272f390 0x557462af65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.181+0000 7f359be00640 1 --2- 172.21.15.32:0/2517903742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462af5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.182+0000 7f35a1200640 1 -- 172.21.15.32:0/2517903742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 msgr2=0x557462af5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.182+0000 7f35a1200640 1 --2- 172.21.15.32:0/2517903742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462af5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.182+0000 7f35a1200640 1 -- 172.21.15.32:0/2517903742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746272f390 msgr2=0x557462af65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.182+0000 7f35a1200640 1 --2- 172.21.15.32:0/2517903742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746272f390 0x557462af65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.182+0000 7f35a1200640 1 -- 172.21.15.32:0/2517903742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574627bb650 con 0x5574627271b0 2024-08-31T07:34:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.182+0000 7f35a1200640 1 --2- 172.21.15.32:0/2517903742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462af60a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3590009e10 tx=0x7f3590011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=643da534855d4dbe server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.182+0000 7f3599600640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35840b4ca0 0x7f35840b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.182+0000 7f358e000640 1 -- 172.21.15.32:0/2517903742 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359001e070 con 0x5574627271b0 2024-08-31T07:34:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.182+0000 7f358e000640 1 -- 172.21.15.32:0/2517903742 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3590019ae0 con 0x5574627271b0 2024-08-31T07:34:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.182+0000 7f358e000640 1 -- 172.21.15.32:0/2517903742 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359000fba0 con 0x5574627271b0 2024-08-31T07:34:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.183+0000 7f3599600640 1 -- 172.21.15.32:0/2068526255 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f35940e5c00 con 0x7f359c10bbe0 2024-08-31T07:34:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.183+0000 7f359be00640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35840b4ca0 0x7f35840b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.183+0000 7f35aa855740 1 -- 172.21.15.32:0/2517903742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 msgr2=0x557462af60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.183+0000 7f35aa855740 1 --2- 172.21.15.32:0/2517903742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462af60a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3590009e10 tx=0x7f3590011810 comp rx=0 tx=0).stop 2024-08-31T07:34:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.183+0000 7f359be00640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35840b4ca0 0x7f35840b71b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3588009230 tx=0x7f3588000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.183+0000 7f35aa855740 1 -- 172.21.15.32:0/2517903742 shutdown_connections 2024-08-31T07:34:19.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.183+0000 7f35aa855740 1 --2- 172.21.15.32:0/2517903742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746272f390 0x557462af65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.183+0000 7f35aa855740 1 --2- 172.21.15.32:0/2517903742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462af60a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.184+0000 7f35aa855740 1 --2- 172.21.15.32:0/2517903742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462af5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.184+0000 7f35aa855740 1 -- 172.21.15.32:0/2517903742 >> 172.21.15.32:0/2517903742 conn(0x557462744460 msgr2=0x557462af3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.184+0000 7f35aa855740 1 -- 172.21.15.32:0/2517903742 shutdown_connections 2024-08-31T07:34:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.184+0000 7f35aa855740 1 -- 172.21.15.32:0/2517903742 wait complete. 2024-08-31T07:34:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.184+0000 7f35aa855740 1 -- start start 2024-08-31T07:34:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.184+0000 7f35aa855740 1 Processor -- start 2024-08-31T07:34:19.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f35aa855740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574627271b0 0x557462b01700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f35aa855740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b01c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f35aa855740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557462750070 0x557462b0d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f35aa855740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5574628be700 con 0x557462750070 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f35aa855740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5574629e2cf0 con 0x5574627271b0 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f35aa855740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557462b04340 con 0x55746272f390 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f35aa855740 10 client.0 _ll_register_callbacks cb 0x557462a08470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f359be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b01c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f35a1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574627271b0 0x557462b01700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f359be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b01c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50484/0 (socket says 172.21.15.32:50484) 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f359be00640 1 -- 172.21.15.32:0/789051866 learned_addr learned my addr 172.21.15.32:0/789051866 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.185+0000 7f35a1c00640 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557462750070 0x557462b0d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.186+0000 7f359be00640 1 -- 172.21.15.32:0/789051866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574627271b0 msgr2=0x557462b01700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:19.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.186+0000 7f359be00640 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574627271b0 0x557462b01700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.186+0000 7f359be00640 1 -- 172.21.15.32:0/789051866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557462750070 msgr2=0x557462b0d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.186+0000 7f359be00640 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557462750070 0x557462b0d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.186+0000 7f359be00640 1 -- 172.21.15.32:0/789051866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557462a238b0 con 0x55746272f390 2024-08-31T07:34:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.186+0000 7f359be00640 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b01c40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3588031720 tx=0x7f358802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e351879a6df8bbd2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.186+0000 7f3582a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:34:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.187+0000 7f3582a00640 1 -- 172.21.15.32:0/789051866 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588035070 con 0x55746272f390 2024-08-31T07:34:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.187+0000 7f3582a00640 1 -- 172.21.15.32:0/789051866 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f3588005450 con 0x55746272f390 2024-08-31T07:34:19.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.187+0000 7f35aa855740 1 -- 172.21.15.32:0/789051866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3590019ae0 con 0x55746272f390 2024-08-31T07:34:19.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.187+0000 7f3582a00640 1 -- 172.21.15.32:0/789051866 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f3588005650 con 0x55746272f390 2024-08-31T07:34:19.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.187+0000 7f35aa855740 10 client.29536 fetch_fsmap learned FSMap version 81 2024-08-31T07:34:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.187+0000 7f35aa855740 1 -- 172.21.15.32:0/789051866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x557462b103b0 con 0x55746272f390 2024-08-31T07:34:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.187+0000 7f3582a00640 1 -- 172.21.15.32:0/789051866 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3590019ae0 con 0x55746272f390 2024-08-31T07:34:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.187+0000 7f3582a00640 1 -- 172.21.15.32:0/789051866 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 81) ==== 8398+0+0 (secure 0 0 0) 0x7f3588044030 con 0x55746272f390 2024-08-31T07:34:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.188+0000 7f35aa855740 10 client.29536 fetch_fsmap finished waiting for FSMap version 81 2024-08-31T07:34:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.188+0000 7f35aa855740 10 client.29536 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:34:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.188+0000 7f35aa855740 20 client.29536 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:19.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.188+0000 7f35aa855740 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55746270e9f0 0x557462b13960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.188+0000 7f35aa855740 4 client.29536 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:34:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.188+0000 7f35aa855740 1 -- 172.21.15.32:0/789051866 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557462a6e700 con 0x55746270e9f0 2024-08-31T07:34:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.188+0000 7f35a1200640 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55746270e9f0 0x557462b13960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.189+0000 7f35a1200640 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55746270e9f0 0x557462b13960 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c816c34efc893f91 server_cookie=ebfb7172e1c06870 in_seq=0 out_seq=0 2024-08-31T07:34:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.189+0000 7f3582a00640 10 client.29536 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:34:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.189+0000 7f3582a00640 1 -- 172.21.15.32:0/789051866 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557462a6e700 con 0x55746270e9f0 2024-08-31T07:34:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.189+0000 7f3582a00640 10 client.29536 handle_command_reply: tid=0 2024-08-31T07:34:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.189+0000 7f35aa855740 1 client.29536 shutdown 2024-08-31T07:34:19.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 1 -- 172.21.15.32:0/789051866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 msgr2=0x557462b01c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b01c40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3588031720 tx=0x7f358802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 1 -- 172.21.15.32:0/789051866 shutdown_connections 2024-08-31T07:34:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557462750070 0x557462b0d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b01c40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55746270e9f0 0x557462b13960 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 1 --2- 172.21.15.32:0/789051866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574627271b0 0x557462b01700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 1 -- 172.21.15.32:0/789051866 >> 172.21.15.32:0/789051866 conn(0x557462744460 msgr2=0x557462af8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:19.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f3582a00640 0 client.29536 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:34:19.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 1 -- 172.21.15.32:0/789051866 shutdown_connections 2024-08-31T07:34:19.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 1 -- 172.21.15.32:0/789051866 wait complete. 2024-08-31T07:34:19.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.190+0000 7f35aa855740 20 client.29536 trim_cache size 0 max 16384 2024-08-31T07:34:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.204+0000 7f35aa855740 1 Processor -- start 2024-08-31T07:34:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.204+0000 7f35aa855740 1 -- start start 2024-08-31T07:34:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f35aa855740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574627271b0 0x557462af72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f35aa855740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b0d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f35aa855740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462b0dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f35aa855740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1b1380 con 0x5574627271b0 2024-08-31T07:34:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f35aa855740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1b3070 con 0x557462750070 2024-08-31T07:34:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f35aa855740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1b1b10 con 0x55746272f390 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f359be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b0d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f35a1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574627271b0 0x557462af72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f359be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b0d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50486/0 (socket says 172.21.15.32:50486) 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f359be00640 1 -- 172.21.15.32:0/2417773766 learned_addr learned my addr 172.21.15.32:0/2417773766 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f35a1c00640 1 --2- 172.21.15.32:0/2417773766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462b0dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f359be00640 1 -- 172.21.15.32:0/2417773766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 msgr2=0x557462b0dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f359be00640 1 --2- 172.21.15.32:0/2417773766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462b0dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f359be00640 1 -- 172.21.15.32:0/2417773766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574627271b0 msgr2=0x557462af72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f359be00640 1 --2- 172.21.15.32:0/2417773766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574627271b0 0x557462af72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.205+0000 7f359be00640 1 -- 172.21.15.32:0/2417773766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557462af89c0 con 0x55746272f390 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.206+0000 7f359be00640 1 --2- 172.21.15.32:0/2417773766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b0d590 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f358802a910 tx=0x7f3588031d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16b2c9a30cdd0b0d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.206+0000 7f358d600640 1 -- 172.21.15.32:0/2417773766 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358805e070 con 0x55746272f390 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.206+0000 7f358d600640 1 -- 172.21.15.32:0/2417773766 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f358804b070 con 0x55746272f390 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.206+0000 7f358d600640 1 -- 172.21.15.32:0/2417773766 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588035040 con 0x55746272f390 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.207+0000 7f35aa855740 1 -- 172.21.15.32:0/2417773766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 msgr2=0x557462b0d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.207+0000 7f35aa855740 1 --2- 172.21.15.32:0/2417773766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b0d590 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f358802a910 tx=0x7f3588031d50 comp rx=0 tx=0).stop 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.207+0000 7f35aa855740 1 -- 172.21.15.32:0/2417773766 shutdown_connections 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.207+0000 7f35aa855740 1 --2- 172.21.15.32:0/2417773766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462b0dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.207+0000 7f35aa855740 1 --2- 172.21.15.32:0/2417773766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55746272f390 0x557462b0d590 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.207+0000 7f35aa855740 1 --2- 172.21.15.32:0/2417773766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5574627271b0 0x557462af72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.207+0000 7f35aa855740 1 -- 172.21.15.32:0/2417773766 >> 172.21.15.32:0/2417773766 conn(0x557462744460 msgr2=0x557462af6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.207+0000 7f35aa855740 1 -- 172.21.15.32:0/2417773766 shutdown_connections 2024-08-31T07:34:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.207+0000 7f35aa855740 1 -- 172.21.15.32:0/2417773766 wait complete. 2024-08-31T07:34:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.208+0000 7f35aa855740 1 -- start start 2024-08-31T07:34:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.208+0000 7f35aa855740 1 Processor -- start 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.208+0000 7f35aa855740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462b09ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.208+0000 7f35aa855740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746272f390 0x557462b05fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.208+0000 7f35aa855740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462b06520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.208+0000 7f35aa855740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1b2910 con 0x55746272f390 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.208+0000 7f35aa855740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1b2240 con 0x557462750070 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.208+0000 7f35aa855740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1af380 con 0x5574627271b0 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35aa855740 10 client.0 _ll_register_callbacks cb 0x7f3588009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35a1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462b09ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35a1200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462b09ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50498/0 (socket says 172.21.15.32:50498) 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35a1200640 1 -- 172.21.15.32:0/298493495 learned_addr learned my addr 172.21.15.32:0/298493495 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f359be00640 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746272f390 0x557462b05fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35a1c00640 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462b06520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35a1200640 1 -- 172.21.15.32:0/298493495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 msgr2=0x557462b06520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35a1200640 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462b06520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35a1200640 1 -- 172.21.15.32:0/298493495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746272f390 msgr2=0x557462b05fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35a1200640 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746272f390 0x557462b05fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35a1200640 1 -- 172.21.15.32:0/298493495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557462b06a60 con 0x5574627271b0 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.209+0000 7f35a1c00640 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462b06520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.210+0000 7f35a1200640 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462b09ad0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3590009bd0 tx=0x7f3590005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb71e31176216b9d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.210+0000 7f3582a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.210+0000 7f3582a00640 1 -- 172.21.15.32:0/298493495 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359001e040 con 0x5574627271b0 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.210+0000 7f35aa855740 1 -- 172.21.15.32:0/298493495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358804b070 con 0x5574627271b0 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.210+0000 7f3582a00640 1 -- 172.21.15.32:0/298493495 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f3590005040 con 0x5574627271b0 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.210+0000 7f3582a00640 1 -- 172.21.15.32:0/298493495 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f3590005240 con 0x5574627271b0 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.210+0000 7f35aa855740 10 client.29542 fetch_fsmap learned FSMap version 81 2024-08-31T07:34:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.210+0000 7f35aa855740 1 -- 172.21.15.32:0/298493495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x557462b7efe0 con 0x5574627271b0 2024-08-31T07:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.210+0000 7f3582a00640 1 -- 172.21.15.32:0/298493495 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f358804b070 con 0x5574627271b0 2024-08-31T07:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.211+0000 7f3582a00640 1 -- 172.21.15.32:0/298493495 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 81) ==== 8398+0+0 (secure 0 0 0) 0x7f359000f9b0 con 0x5574627271b0 2024-08-31T07:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.211+0000 7f35aa855740 10 client.29542 fetch_fsmap finished waiting for FSMap version 81 2024-08-31T07:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.211+0000 7f35aa855740 10 client.29542 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.211+0000 7f35aa855740 20 client.29542 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.211+0000 7f35aa855740 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5574626e3da0 0x557462b81630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.211+0000 7f35aa855740 4 client.29542 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.211+0000 7f35aa855740 1 -- 172.21.15.32:0/298493495 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557462a6a1a0 con 0x5574626e3da0 2024-08-31T07:34:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.211+0000 7f359be00640 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5574626e3da0 0x557462b81630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f359be00640 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5574626e3da0 0x557462b81630 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc3015fe412b096a server_cookie=b7d2da6108b58cc9 in_seq=0 out_seq=0 2024-08-31T07:34:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f3582a00640 10 client.29542 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:34:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f3582a00640 1 -- 172.21.15.32:0/298493495 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557462a6a1a0 con 0x5574626e3da0 2024-08-31T07:34:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f3582a00640 10 client.29542 handle_command_reply: tid=0 2024-08-31T07:34:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f35aa855740 1 client.29542 shutdown 2024-08-31T07:34:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f35aa855740 1 -- 172.21.15.32:0/298493495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 msgr2=0x557462b09ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f35aa855740 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462b09ad0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3590009bd0 tx=0x7f3590005d20 comp rx=0 tx=0).stop 2024-08-31T07:34:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f35aa855740 1 -- 172.21.15.32:0/298493495 shutdown_connections 2024-08-31T07:34:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f35aa855740 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5574626e3da0 0x557462b81630 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f35aa855740 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557462750070 0x557462b06520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f35aa855740 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55746272f390 0x557462b05fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f35aa855740 1 --2- 172.21.15.32:0/298493495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574627271b0 0x557462b09ad0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f3582a00640 0 client.29542 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:34:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.212+0000 7f35aa855740 1 -- 172.21.15.32:0/298493495 >> 172.21.15.32:0/298493495 conn(0x557462744460 msgr2=0x557462b029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.213+0000 7f35aa855740 1 -- 172.21.15.32:0/298493495 shutdown_connections 2024-08-31T07:34:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.213+0000 7f35aa855740 1 -- 172.21.15.32:0/298493495 wait complete. 2024-08-31T07:34:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.213+0000 7f35aa855740 20 client.29542 trim_cache size 0 max 16384 2024-08-31T07:34:19.221 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:34:19.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.213+0000 7f35a2600640 1 -- 172.21.15.32:0/2068526255 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35840b4ca0 msgr2=0x7f35840b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.213+0000 7f35a2600640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35840b4ca0 0x7f35840b71b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3588009230 tx=0x7f3588000bc0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.213+0000 7f35a2600640 1 -- 172.21.15.32:0/2068526255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f359c10bbe0 msgr2=0x7f359c1a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.213+0000 7f35a2600640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f359c10bbe0 0x7f359c1a1af0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3594009fd0 tx=0x7f359400ef10 comp rx=0 tx=0).stop 2024-08-31T07:34:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.214+0000 7f35a2600640 1 -- 172.21.15.32:0/2068526255 shutdown_connections 2024-08-31T07:34:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.214+0000 7f35a2600640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f35840b4ca0 0x7f35840b71b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.214+0000 7f35a2600640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f359c10bbe0 0x7f359c1a1af0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.214+0000 7f35a2600640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f359c10b290 0x7f359c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.214+0000 7f35a2600640 1 --2- 172.21.15.32:0/2068526255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c105950 0x7f359c19c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:19.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.214+0000 7f35a2600640 1 -- 172.21.15.32:0/2068526255 >> 172.21.15.32:0/2068526255 conn(0x7f359c101420 msgr2=0x7f359c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:19.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.214+0000 7f35a2600640 1 -- 172.21.15.32:0/2068526255 shutdown_connections 2024-08-31T07:34:19.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:19.214+0000 7f35a2600640 1 -- 172.21.15.32:0/2068526255 wait complete. 2024-08-31T07:34:19.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.91 MB/sec execute 359 sec latency 19906.147 ms 2024-08-31T07:34:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:20 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:20 smithi032 ceph-mon[17252]: pgmap v1047: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 883 KiB/s wr, 74 op/s 2024-08-31T07:34:20.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.87 MB/sec execute 360 sec latency 5503.795 ms 2024-08-31T07:34:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:20 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:20 smithi073 ceph-mon[21995]: pgmap v1047: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 883 KiB/s wr, 74 op/s 2024-08-31T07:34:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:20 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:20 smithi084 ceph-mon[22387]: pgmap v1047: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 883 KiB/s wr, 74 op/s 2024-08-31T07:34:20.569 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e3f2fbef', quiesce took 11.0 seconds. Will release after: 3.0999999999999996 2024-08-31T07:34:20.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:34:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:21.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.83 MB/sec execute 361 sec latency 6503.913 ms 2024-08-31T07:34:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.986+0000 7f2bc7e00640 1 -- 172.21.15.32:0/798469138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc810bbe0 msgr2=0x7f2bc8112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.986+0000 7f2bc7e00640 1 --2- 172.21.15.32:0/798469138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc810bbe0 0x7f2bc8112410 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009ca0 tx=0x7f2bc002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:34:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.986+0000 7f2bc7e00640 1 -- 172.21.15.32:0/798469138 shutdown_connections 2024-08-31T07:34:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.986+0000 7f2bc7e00640 1 --2- 172.21.15.32:0/798469138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc810bbe0 0x7f2bc8112410 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.986+0000 7f2bc7e00640 1 --2- 172.21.15.32:0/798469138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc810b290 0x7f2bc810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.986+0000 7f2bc7e00640 1 --2- 172.21.15.32:0/798469138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc8105950 0x7f2bc8105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:21.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.986+0000 7f2bc7e00640 1 -- 172.21.15.32:0/798469138 >> 172.21.15.32:0/798469138 conn(0x7f2bc8101420 msgr2=0x7f2bc8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:21.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.986+0000 7f2bc7e00640 1 -- 172.21.15.32:0/798469138 shutdown_connections 2024-08-31T07:34:21.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.986+0000 7f2bc7e00640 1 -- 172.21.15.32:0/798469138 wait complete. 2024-08-31T07:34:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.987+0000 7f2bc7e00640 1 Processor -- start 2024-08-31T07:34:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.987+0000 7f2bc7e00640 1 -- start start 2024-08-31T07:34:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc8105950 0x7f2bc819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc810b290 0x7f2bc819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc810bbe0 0x7f2bc81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81158a0 con 0x7f2bc8105950 2024-08-31T07:34:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8115720 con 0x7f2bc810b290 2024-08-31T07:34:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc7e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8115a20 con 0x7f2bc810bbe0 2024-08-31T07:34:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc8105950 0x7f2bc819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc8105950 0x7f2bc819c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41068/0 (socket says 172.21.15.32:41068) 2024-08-31T07:34:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc6a00640 1 -- 172.21.15.32:0/3131914335 learned_addr learned my addr 172.21.15.32:0/3131914335 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc810b290 0x7f2bc819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc7400640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc810bbe0 0x7f2bc81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc6a00640 1 -- 172.21.15.32:0/3131914335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc810bbe0 msgr2=0x7f2bc81a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc810bbe0 0x7f2bc81a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc6a00640 1 -- 172.21.15.32:0/3131914335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc810b290 msgr2=0x7f2bc819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc810b290 0x7f2bc819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.988+0000 7f2bc6a00640 1 -- 172.21.15.32:0/3131914335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc81a2180 con 0x7f2bc8105950 2024-08-31T07:34:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.989+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc8105950 0x7f2bc819c210 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800d950 tx=0x7f2bb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.989+0000 7f2bbf400640 1 -- 172.21.15.32:0/3131914335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8017030 con 0x7f2bc8105950 2024-08-31T07:34:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.989+0000 7f2bbf400640 1 -- 172.21.15.32:0/3131914335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb8004020 con 0x7f2bc8105950 2024-08-31T07:34:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.989+0000 7f2bbf400640 1 -- 172.21.15.32:0/3131914335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80047a0 con 0x7f2bc8105950 2024-08-31T07:34:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.989+0000 7f2bc7e00640 1 -- 172.21.15.32:0/3131914335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc81a2470 con 0x7f2bc8105950 2024-08-31T07:34:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.989+0000 7f2bc7e00640 1 -- 172.21.15.32:0/3131914335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc81a8bd0 con 0x7f2bc8105950 2024-08-31T07:34:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.992+0000 7f2bd4257740 1 Processor -- start 2024-08-31T07:34:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.993+0000 7f2bd4257740 1 -- start start 2024-08-31T07:34:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.993+0000 7f2bbf400640 1 -- 172.21.15.32:0/3131914335 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2bb8009a30 con 0x7f2bc8105950 2024-08-31T07:34:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.993+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 0x55e4c9727b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.993+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c97280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.993+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9361390 0x55e4c97285e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.993+0000 7f2bd4257740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e4c969e7b0 con 0x55e4c9361390 2024-08-31T07:34:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.993+0000 7f2bd4257740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e4c968dd40 con 0x55e4c93591b0 2024-08-31T07:34:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.993+0000 7f2bd4257740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e4c9691cf0 con 0x55e4c9382070 2024-08-31T07:34:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.993+0000 7f2bc6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c97280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bc7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 0x55e4c9727b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bc6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c97280a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42118/0 (socket says 172.21.15.32:42118) 2024-08-31T07:34:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bc7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 0x55e4c9727b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50518/0 (socket says 172.21.15.32:50518) 2024-08-31T07:34:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bc6a00640 1 -- 172.21.15.32:0/1273430008 learned_addr learned my addr 172.21.15.32:0/1273430008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/1273430008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9361390 0x55e4c97285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bc6a00640 1 -- 172.21.15.32:0/1273430008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 msgr2=0x55e4c9727b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/1273430008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 0x55e4c9727b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bc6a00640 1 -- 172.21.15.32:0/1273430008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9361390 msgr2=0x55e4c97285e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/1273430008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9361390 0x55e4c97285e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bc6a00640 1 -- 172.21.15.32:0/1273430008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4c93ed650 con 0x55e4c93591b0 2024-08-31T07:34:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.994+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/1273430008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c97280a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80e25f0 tx=0x7f2bb80ed4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ecf9a7fc43e749 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.995+0000 7f2bb7400640 1 -- 172.21.15.32:0/1273430008 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80f8070 con 0x55e4c93591b0 2024-08-31T07:34:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.995+0000 7f2bb7400640 1 -- 172.21.15.32:0/1273430008 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb80f3ab0 con 0x55e4c93591b0 2024-08-31T07:34:21.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.995+0000 7f2bb7400640 1 -- 172.21.15.32:0/1273430008 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80f29f0 con 0x55e4c93591b0 2024-08-31T07:34:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.995+0000 7f2bbf400640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bac0b4ca0 0x7f2bac0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.995+0000 7f2bbf400640 1 -- 172.21.15.32:0/3131914335 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f2bb80e4f70 con 0x7f2bc8105950 2024-08-31T07:34:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.995+0000 7f2bc7400640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bac0b4ca0 0x7f2bac0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.996+0000 7f2bd4257740 1 -- 172.21.15.32:0/1273430008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 msgr2=0x55e4c97280a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.996+0000 7f2bd4257740 1 --2- 172.21.15.32:0/1273430008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c97280a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80e25f0 tx=0x7f2bb80ed4b0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.996+0000 7f2bd4257740 1 -- 172.21.15.32:0/1273430008 shutdown_connections 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.996+0000 7f2bd4257740 1 --2- 172.21.15.32:0/1273430008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9361390 0x55e4c97285e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.996+0000 7f2bd4257740 1 --2- 172.21.15.32:0/1273430008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c97280a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.996+0000 7f2bd4257740 1 --2- 172.21.15.32:0/1273430008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 0x55e4c9727b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.996+0000 7f2bd4257740 1 -- 172.21.15.32:0/1273430008 >> 172.21.15.32:0/1273430008 conn(0x55e4c9376460 msgr2=0x55e4c9725a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.996+0000 7f2bd4257740 1 -- 172.21.15.32:0/1273430008 shutdown_connections 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.996+0000 7f2bd4257740 1 -- 172.21.15.32:0/1273430008 wait complete. 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.996+0000 7f2bc7400640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bac0b4ca0 0x7f2bac0b71b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2bc002f500 tx=0x7f2bc002f310 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.997+0000 7f2bd4257740 1 -- start start 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.997+0000 7f2bd4257740 1 Processor -- start 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c93591b0 0x55e4c9733790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9361390 0x55e4c9733d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9382070 0x55e4c973f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bd4257740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e4c94f0700 con 0x55e4c9382070 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bd4257740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e4c9614cf0 con 0x55e4c9361390 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bd4257740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e4c9736340 con 0x55e4c93591b0 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bd4257740 10 client.0 _ll_register_callbacks cb 0x55e4c963a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bc6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c93591b0 0x55e4c9733790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bbfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9361390 0x55e4c9733d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bc6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c93591b0 0x55e4c9733790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50532/0 (socket says 172.21.15.32:50532) 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bc6a00640 1 -- 172.21.15.32:0/2096917057 learned_addr learned my addr 172.21.15.32:0/2096917057 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bc7400640 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9382070 0x55e4c973f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bc6a00640 1 -- 172.21.15.32:0/2096917057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9361390 msgr2=0x55e4c9733d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9361390 0x55e4c9733d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bc6a00640 1 -- 172.21.15.32:0/2096917057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9382070 msgr2=0x55e4c973f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9382070 0x55e4c973f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.998+0000 7f2bc6a00640 1 -- 172.21.15.32:0/2096917057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4c96558b0 con 0x55e4c93591b0 2024-08-31T07:34:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.999+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c93591b0 0x55e4c9733790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80a88c0 tx=0x7f2bb8009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6f4f1e79a9a1bdd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.999+0000 7f2babe00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.999+0000 7f2babe00640 1 -- 172.21.15.32:0/2096917057 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80a7510 con 0x55e4c93591b0 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.999+0000 7f2babe00640 1 -- 172.21.15.32:0/2096917057 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f2bb80a7c40 con 0x55e4c93591b0 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.999+0000 7f2bd4257740 1 -- 172.21.15.32:0/2096917057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb80f3ab0 con 0x55e4c93591b0 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.999+0000 7f2babe00640 1 -- 172.21.15.32:0/2096917057 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2bb80aadc0 con 0x55e4c93591b0 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.999+0000 7f2bd4257740 10 client.29554 fetch_fsmap learned FSMap version 82 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.999+0000 7f2bd4257740 1 -- 172.21.15.32:0/2096917057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x55e4c9742400 con 0x55e4c93591b0 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:21.999+0000 7f2babe00640 1 -- 172.21.15.32:0/2096917057 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb80f3ab0 con 0x55e4c93591b0 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.000+0000 7f2babe00640 1 -- 172.21.15.32:0/2096917057 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 82) ==== 8394+0+0 (secure 0 0 0) 0x7f2bb80a78e0 con 0x55e4c93591b0 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.000+0000 7f2bd4257740 10 client.29554 fetch_fsmap finished waiting for FSMap version 82 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.000+0000 7f2bd4257740 10 client.29554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.000+0000 7f2bd4257740 20 client.29554 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.000+0000 7f2bd4257740 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e4c93409f0 0x55e4c9745760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.000+0000 7f2bd4257740 4 client.29554 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:34:22.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.000+0000 7f2bd4257740 1 -- 172.21.15.32:0/2096917057 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4c96a0700 con 0x55e4c93409f0 2024-08-31T07:34:22.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.000+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e4c93409f0 0x55e4c9745760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.001+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e4c93409f0 0x55e4c9745760 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd2e6934ef445e28 server_cookie=af3f436607c96e6f in_seq=0 out_seq=0 2024-08-31T07:34:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.001+0000 7f2babe00640 10 client.29554 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2babe00640 1 -- 172.21.15.32:0/2096917057 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e4c96a0700 con 0x55e4c93409f0 2024-08-31T07:34:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2babe00640 10 client.29554 handle_command_reply: tid=0 2024-08-31T07:34:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 client.29554 shutdown 2024-08-31T07:34:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 -- 172.21.15.32:0/2096917057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c93591b0 msgr2=0x55e4c9733790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c93591b0 0x55e4c9733790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80a88c0 tx=0x7f2bb8009f50 comp rx=0 tx=0).stop 2024-08-31T07:34:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 -- 172.21.15.32:0/2096917057 shutdown_connections 2024-08-31T07:34:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e4c93409f0 0x55e4c9745760 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9382070 0x55e4c973f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9361390 0x55e4c9733d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 --2- 172.21.15.32:0/2096917057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c93591b0 0x55e4c9733790 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2babe00640 0 client.29554 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 -- 172.21.15.32:0/2096917057 >> 172.21.15.32:0/2096917057 conn(0x55e4c9376460 msgr2=0x55e4c972acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 -- 172.21.15.32:0/2096917057 shutdown_connections 2024-08-31T07:34:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.002+0000 7f2bd4257740 1 -- 172.21.15.32:0/2096917057 wait complete. 2024-08-31T07:34:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.003+0000 7f2bd4257740 20 client.29554 trim_cache size 0 max 16384 2024-08-31T07:34:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bd4257740 1 Processor -- start 2024-08-31T07:34:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bd4257740 1 -- start start 2024-08-31T07:34:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c93591b0 0x55e4c972d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9361390 0x55e4c9725000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9382070 0x55e4c9725540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bd4257740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81b1420 con 0x55e4c93591b0 2024-08-31T07:34:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bd4257740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81b30e0 con 0x55e4c9382070 2024-08-31T07:34:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bd4257740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81b1bb0 con 0x55e4c9361390 2024-08-31T07:34:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bc6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c93591b0 0x55e4c972d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bc7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9382070 0x55e4c9725540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bc6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c93591b0 0x55e4c972d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41080/0 (socket says 172.21.15.32:41080) 2024-08-31T07:34:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.017+0000 7f2bc6a00640 1 -- 172.21.15.32:0/4208069831 learned_addr learned my addr 172.21.15.32:0/4208069831 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.018+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/4208069831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9361390 0x55e4c9725000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.018+0000 7f2bc6a00640 1 -- 172.21.15.32:0/4208069831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9361390 msgr2=0x55e4c9725000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.018+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/4208069831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9361390 0x55e4c9725000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.018+0000 7f2bc6a00640 1 -- 172.21.15.32:0/4208069831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9382070 msgr2=0x55e4c9725540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.018+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/4208069831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9382070 0x55e4c9725540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.018+0000 7f2bc6a00640 1 -- 172.21.15.32:0/4208069831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4c97267e0 con 0x55e4c93591b0 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.018+0000 7f2bc7400640 1 --2- 172.21.15.32:0/4208069831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9382070 0x55e4c9725540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.018+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/4208069831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c93591b0 0x55e4c972d380 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8005fd0 tx=0x7f2bb80043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398a20f3d0cb20da server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.018+0000 7f2bb6a00640 1 -- 172.21.15.32:0/4208069831 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80aafa0 con 0x55e4c93591b0 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.018+0000 7f2bb6a00640 1 -- 172.21.15.32:0/4208069831 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb80b42b0 con 0x55e4c93591b0 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.019+0000 7f2bb6a00640 1 -- 172.21.15.32:0/4208069831 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80ab420 con 0x55e4c93591b0 2024-08-31T07:34:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.019+0000 7f2bd4257740 1 -- 172.21.15.32:0/4208069831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c93591b0 msgr2=0x55e4c972d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:22.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.019+0000 7f2bd4257740 1 --2- 172.21.15.32:0/4208069831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c93591b0 0x55e4c972d380 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8005fd0 tx=0x7f2bb80043e0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.019+0000 7f2bd4257740 1 -- 172.21.15.32:0/4208069831 shutdown_connections 2024-08-31T07:34:22.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.019+0000 7f2bd4257740 1 --2- 172.21.15.32:0/4208069831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c9382070 0x55e4c9725540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.019+0000 7f2bd4257740 1 --2- 172.21.15.32:0/4208069831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9361390 0x55e4c9725000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.019+0000 7f2bd4257740 1 --2- 172.21.15.32:0/4208069831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c93591b0 0x55e4c972d380 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.019+0000 7f2bd4257740 1 -- 172.21.15.32:0/4208069831 >> 172.21.15.32:0/4208069831 conn(0x55e4c9376460 msgr2=0x55e4c972cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:22.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.019+0000 7f2bd4257740 1 -- 172.21.15.32:0/4208069831 shutdown_connections 2024-08-31T07:34:22.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.019+0000 7f2bd4257740 1 -- 172.21.15.32:0/4208069831 wait complete. 2024-08-31T07:34:22.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.020+0000 7f2bd4257740 1 -- start start 2024-08-31T07:34:22.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.020+0000 7f2bd4257740 1 Processor -- start 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.020+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c9748ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.020+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9361390 0x55e4c97471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.020+0000 7f2bd4257740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 0x55e4c9747700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bd4257740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81b29b0 con 0x55e4c9361390 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bd4257740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81b22e0 con 0x55e4c93591b0 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bd4257740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc81af420 con 0x55e4c9382070 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bd4257740 10 client.0 _ll_register_callbacks cb 0x7f2bb80e8d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bc6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c9748ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bbfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9361390 0x55e4c97471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bc6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c9748ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42148/0 (socket says 172.21.15.32:42148) 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bc6a00640 1 -- 172.21.15.32:0/3949059651 learned_addr learned my addr 172.21.15.32:0/3949059651 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:22.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bc7400640 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 0x55e4c9747700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bc6a00640 1 -- 172.21.15.32:0/3949059651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 msgr2=0x55e4c9747700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:22.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 0x55e4c9747700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bc6a00640 1 -- 172.21.15.32:0/3949059651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9361390 msgr2=0x55e4c97471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:22.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9361390 0x55e4c97471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.021+0000 7f2bc6a00640 1 -- 172.21.15.32:0/3949059651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4c9735cd0 con 0x55e4c93591b0 2024-08-31T07:34:22.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.022+0000 7f2bc6a00640 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c9748ce0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80abf00 tx=0x7f2bb80e8a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=626e13d4c7f92622 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:22.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.022+0000 7f2babe00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:34:22.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.022+0000 7f2babe00640 1 -- 172.21.15.32:0/3949059651 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80ab420 con 0x55e4c93591b0 2024-08-31T07:34:22.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.022+0000 7f2babe00640 1 -- 172.21.15.32:0/3949059651 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f2bb80a7c40 con 0x55e4c93591b0 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.022+0000 7f2bd4257740 1 -- 172.21.15.32:0/3949059651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e4c969c1a0 con 0x55e4c93591b0 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.022+0000 7f2babe00640 1 -- 172.21.15.32:0/3949059651 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2bb80e2b40 con 0x55e4c93591b0 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.022+0000 7f2bd4257740 10 client.29564 fetch_fsmap learned FSMap version 82 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.022+0000 7f2bd4257740 1 -- 172.21.15.32:0/3949059651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x55e4c9735cd0 con 0x55e4c93591b0 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.022+0000 7f2babe00640 1 -- 172.21.15.32:0/3949059651 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e4c969c1a0 con 0x55e4c93591b0 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.023+0000 7f2babe00640 1 -- 172.21.15.32:0/3949059651 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 82) ==== 8394+0+0 (secure 0 0 0) 0x7f2bb80b4570 con 0x55e4c93591b0 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.023+0000 7f2bd4257740 10 client.29564 fetch_fsmap finished waiting for FSMap version 82 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.023+0000 7f2bd4257740 10 client.29564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.023+0000 7f2bd4257740 20 client.29564 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.023+0000 7f2bd4257740 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e4c9315da0 0x55e4c97b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.023+0000 7f2bd4257740 4 client.29564 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.023+0000 7f2bd4257740 1 -- 172.21.15.32:0/3949059651 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4c96998e0 con 0x55e4c9315da0 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.023+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e4c9315da0 0x55e4c97b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bbfe00640 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e4c9315da0 0x55e4c97b2d50 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a8accc8131eeb9 server_cookie=831c487edf480218 in_seq=0 out_seq=0 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2babe00640 10 client.29564 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2babe00640 1 -- 172.21.15.32:0/3949059651 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4c96998e0 con 0x55e4c9315da0 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2babe00640 10 client.29564 handle_command_reply: tid=0 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 client.29564 shutdown 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 -- 172.21.15.32:0/3949059651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 msgr2=0x55e4c9748ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c9748ce0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80abf00 tx=0x7f2bb80e8a20 comp rx=0 tx=0).stop 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 -- 172.21.15.32:0/3949059651 shutdown_connections 2024-08-31T07:34:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e4c9315da0 0x55e4c97b2d50 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e4c9382070 0x55e4c9747700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2babe00640 0 client.29564 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4c9361390 0x55e4c97471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 --2- 172.21.15.32:0/3949059651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e4c93591b0 0x55e4c9748ce0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 -- 172.21.15.32:0/3949059651 >> 172.21.15.32:0/3949059651 conn(0x55e4c9376460 msgr2=0x55e4c9728a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 -- 172.21.15.32:0/3949059651 shutdown_connections 2024-08-31T07:34:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.024+0000 7f2bd4257740 1 -- 172.21.15.32:0/3949059651 wait complete. 2024-08-31T07:34:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.025+0000 7f2bd4257740 20 client.29564 trim_cache size 0 max 16384 2024-08-31T07:34:22.030 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:34:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.025+0000 7f2bc7e00640 1 -- 172.21.15.32:0/3131914335 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bac0b4ca0 msgr2=0x7f2bac0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.025+0000 7f2bc7e00640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bac0b4ca0 0x7f2bac0b71b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2bc002f500 tx=0x7f2bc002f310 comp rx=0 tx=0).stop 2024-08-31T07:34:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.025+0000 7f2bc7e00640 1 -- 172.21.15.32:0/3131914335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc8105950 msgr2=0x7f2bc819c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.025+0000 7f2bc7e00640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc8105950 0x7f2bc819c210 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800d950 tx=0x7f2bb800de10 comp rx=0 tx=0).stop 2024-08-31T07:34:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.026+0000 7f2bc7e00640 1 -- 172.21.15.32:0/3131914335 shutdown_connections 2024-08-31T07:34:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.026+0000 7f2bc7e00640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2bac0b4ca0 0x7f2bac0b71b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.026+0000 7f2bc7e00640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bc810bbe0 0x7f2bc81a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.026+0000 7f2bc7e00640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bc810b290 0x7f2bc819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.026+0000 7f2bc7e00640 1 --2- 172.21.15.32:0/3131914335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bc8105950 0x7f2bc819c210 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.026+0000 7f2bc7e00640 1 -- 172.21.15.32:0/3131914335 >> 172.21.15.32:0/3131914335 conn(0x7f2bc8101420 msgr2=0x7f2bc8108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.026+0000 7f2bc7e00640 1 -- 172.21.15.32:0/3131914335 shutdown_connections 2024-08-31T07:34:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:22.026+0000 7f2bc7e00640 1 -- 172.21.15.32:0/3131914335 wait complete. 2024-08-31T07:34:22.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:22 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:22.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:22 smithi032 ceph-mon[17252]: pgmap v1048: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-08-31T07:34:22.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.79 MB/sec execute 362 sec latency 7503.976 ms 2024-08-31T07:34:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:22 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:22.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:22 smithi073 ceph-mon[21995]: pgmap v1048: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-08-31T07:34:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:22 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:22.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:22 smithi084 ceph-mon[22387]: pgmap v1048: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-08-31T07:34:23.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.75 MB/sec execute 363 sec latency 8504.114 ms 2024-08-31T07:34:23.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:34:24.043 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:34:24.043 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:34:24.062 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:24.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:24 smithi032 ceph-mon[17252]: pgmap v1049: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-08-31T07:34:24.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.71 MB/sec execute 364 sec latency 9504.205 ms 2024-08-31T07:34:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:24 smithi073 ceph-mon[21995]: pgmap v1049: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-08-31T07:34:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:24 smithi084 ceph-mon[22387]: pgmap v1049: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-08-31T07:34:24.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.744+0000 7f52dee00640 1 -- 172.21.15.32:0/1681252875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d8105b20 msgr2=0x7f52d8105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:24.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.744+0000 7f52dee00640 1 --2- 172.21.15.32:0/1681252875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d8105b20 0x7f52d8105f30 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009900 tx=0x7f52c802a410 comp rx=0 tx=0).stop 2024-08-31T07:34:24.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.744+0000 7f52dee00640 1 -- 172.21.15.32:0/1681252875 shutdown_connections 2024-08-31T07:34:24.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.744+0000 7f52dee00640 1 --2- 172.21.15.32:0/1681252875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d810bdb0 0x7f52d81125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:24.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.744+0000 7f52dee00640 1 --2- 172.21.15.32:0/1681252875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52d810b460 0x7f52d810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:24.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.744+0000 7f52dee00640 1 --2- 172.21.15.32:0/1681252875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d8105b20 0x7f52d8105f30 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:24.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.744+0000 7f52dee00640 1 -- 172.21.15.32:0/1681252875 >> 172.21.15.32:0/1681252875 conn(0x7f52d81015b0 msgr2=0x7f52d8103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:24.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.744+0000 7f52dee00640 1 -- 172.21.15.32:0/1681252875 shutdown_connections 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.745+0000 7f52dee00640 1 -- 172.21.15.32:0/1681252875 wait complete. 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.746+0000 7f52dee00640 1 Processor -- start 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.746+0000 7f52dee00640 1 -- start start 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.746+0000 7f52dee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d8105b20 0x7f52d819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.746+0000 7f52dee00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52d810b460 0x7f52d819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.746+0000 7f52dee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d810bdb0 0x7f52d81a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.746+0000 7f52dee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8115a70 con 0x7f52d810bdb0 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.746+0000 7f52dee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81158f0 con 0x7f52d8105b20 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.746+0000 7f52dee00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8115bf0 con 0x7f52d810b460 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52d7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d8105b20 0x7f52d819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52d7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d8105b20 0x7f52d819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:42164/0 (socket says 172.21.15.32:42164) 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52d7e00640 1 -- 172.21.15.32:0/2182839109 learned_addr learned my addr 172.21.15.32:0/2182839109 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52dce00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d810bdb0 0x7f52d81a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52cf400640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52d810b460 0x7f52d819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52d7e00640 1 -- 172.21.15.32:0/2182839109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52d810b460 msgr2=0x7f52d819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52d7e00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52d810b460 0x7f52d819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52d7e00640 1 -- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d810bdb0 msgr2=0x7f52d81a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52d7e00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d810bdb0 0x7f52d81a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52d7e00640 1 -- 172.21.15.32:0/2182839109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d81a23d0 con 0x7f52d8105b20 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.747+0000 7f52dce00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d810bdb0 0x7f52d81a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.748+0000 7f52d7e00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d8105b20 0x7f52d819c460 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005940 tx=0x7f52c80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.748+0000 7f52d5600640 1 -- 172.21.15.32:0/2182839109 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c803f030 con 0x7f52d8105b20 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.748+0000 7f52dee00640 1 -- 172.21.15.32:0/2182839109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d81a2660 con 0x7f52d8105b20 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.748+0000 7f52d5600640 1 -- 172.21.15.32:0/2182839109 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52c80044c0 con 0x7f52d8105b20 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.748+0000 7f52d5600640 1 -- 172.21.15.32:0/2182839109 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8035070 con 0x7f52d8105b20 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.748+0000 7f52dee00640 1 -- 172.21.15.32:0/2182839109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d81a8e20 con 0x7f52d8105b20 2024-08-31T07:34:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.749+0000 7f52dee00640 1 -- 172.21.15.32:0/2182839109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d8105f30 con 0x7f52d8105b20 2024-08-31T07:34:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.752+0000 7f52d5600640 1 -- 172.21.15.32:0/2182839109 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f52c8042030 con 0x7f52d8105b20 2024-08-31T07:34:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.753+0000 7f52d5600640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52bc0b4d20 0x7f52bc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.753+0000 7f52d5600640 1 -- 172.21.15.32:0/2182839109 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f52c8105a90 con 0x7f52d8105b20 2024-08-31T07:34:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.754+0000 7f52cf400640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52bc0b4d20 0x7f52bc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.754+0000 7f52cf400640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52bc0b4d20 0x7f52bc0b7230 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f52d810afa0 tx=0x7f52c0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.755+0000 7f52d5600640 1 -- 172.21.15.32:0/2182839109 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f52c80cc220 con 0x7f52d8105b20 2024-08-31T07:34:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.988+0000 7f52dee00640 1 -- 172.21.15.32:0/2182839109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52d819d590 con 0x7f52d8105b20 2024-08-31T07:34:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.989+0000 7f52d5600640 1 -- 172.21.15.32:0/2182839109 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+11973 (secure 0 0 0) 0x7f52d819d590 con 0x7f52d8105b20 2024-08-31T07:34:24.990 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:34:24.991 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"btime":"2024-08-31T07:34:21:040714+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":82,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:34:20.977968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:34:24.991 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 82 2024-08-31T07:34:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 -- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52bc0b4d20 msgr2=0x7f52bc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52bc0b4d20 0x7f52bc0b7230 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f52d810afa0 tx=0x7f52c0005f90 comp rx=0 tx=0).stop 2024-08-31T07:34:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 -- 172.21.15.32:0/2182839109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d8105b20 msgr2=0x7f52d819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d8105b20 0x7f52d819c460 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005940 tx=0x7f52c80040c0 comp rx=0 tx=0).stop 2024-08-31T07:34:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 -- 172.21.15.32:0/2182839109 shutdown_connections 2024-08-31T07:34:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f52bc0b4d20 0x7f52bc0b7230 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:24.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d810bdb0 0x7f52d81a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52d810b460 0x7f52d819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 --2- 172.21.15.32:0/2182839109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52d8105b20 0x7f52d819c460 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 -- 172.21.15.32:0/2182839109 >> 172.21.15.32:0/2182839109 conn(0x7f52d81015b0 msgr2=0x7f52d81039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 -- 172.21.15.32:0/2182839109 shutdown_connections 2024-08-31T07:34:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:24.992+0000 7f52dee00640 1 -- 172.21.15.32:0/2182839109 wait complete. 2024-08-31T07:34:25.010 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:25.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.68 MB/sec execute 365 sec latency 10504.269 ms 2024-08-31T07:34:25.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:25 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2182839109' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:34:25.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:25 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2182839109' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:34:25.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:25 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2182839109' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:34:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.122+0000 7f0777200640 1 -- 172.21.15.32:0/2850253962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0770109f50 msgr2=0x7f077010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.122+0000 7f0777200640 1 --2- 172.21.15.32:0/2850253962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0770109f50 0x7f077010a360 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0760009900 tx=0x7f076002a410 comp rx=0 tx=0).stop 2024-08-31T07:34:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.122+0000 7f0777200640 1 -- 172.21.15.32:0/2850253962 shutdown_connections 2024-08-31T07:34:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.122+0000 7f0777200640 1 --2- 172.21.15.32:0/2850253962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07701101e0 0x7f0770116a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.122+0000 7f0777200640 1 --2- 172.21.15.32:0/2850253962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f077010f890 0x7f077010fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.122+0000 7f0777200640 1 --2- 172.21.15.32:0/2850253962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0770109f50 0x7f077010a360 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.122+0000 7f0777200640 1 -- 172.21.15.32:0/2850253962 >> 172.21.15.32:0/2850253962 conn(0x7f0770077e80 msgr2=0x7f0770107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.122+0000 7f0777200640 1 -- 172.21.15.32:0/2850253962 shutdown_connections 2024-08-31T07:34:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.122+0000 7f0777200640 1 -- 172.21.15.32:0/2850253962 wait complete. 2024-08-31T07:34:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.123+0000 7f0777200640 1 Processor -- start 2024-08-31T07:34:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.123+0000 7f0777200640 1 -- start start 2024-08-31T07:34:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f0777200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0770109f50 0x7f07701a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f0777200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f077010f890 0x7f07701a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f0777200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07701101e0 0x7f07701a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f0777200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0770119ef0 con 0x7f07701101e0 2024-08-31T07:34:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f0777200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0770119d70 con 0x7f0770109f50 2024-08-31T07:34:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f0777200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f077011a070 con 0x7f077010f890 2024-08-31T07:34:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f0775200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07701101e0 0x7f07701a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f076fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0770109f50 0x7f07701a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f0775200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07701101e0 0x7f07701a6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43804/0 (socket says 172.21.15.32:43804) 2024-08-31T07:34:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f0767e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f077010f890 0x7f07701a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f0775200640 1 -- 172.21.15.32:0/2062873148 learned_addr learned my addr 172.21.15.32:0/2062873148 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.124+0000 7f076fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0770109f50 0x7f07701a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34000/0 (socket says 172.21.15.32:34000) 2024-08-31T07:34:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.125+0000 7f076fe00640 1 -- 172.21.15.32:0/2062873148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f077010f890 msgr2=0x7f07701a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.125+0000 7f076fe00640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f077010f890 0x7f07701a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.125+0000 7f076fe00640 1 -- 172.21.15.32:0/2062873148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07701101e0 msgr2=0x7f07701a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.125+0000 7f076fe00640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07701101e0 0x7f07701a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.125+0000 7f076fe00640 1 -- 172.21.15.32:0/2062873148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07701a6830 con 0x7f0770109f50 2024-08-31T07:34:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.125+0000 7f0767e00640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f077010f890 0x7f07701a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.125+0000 7f076fe00640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0770109f50 0x7f07701a08c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f076002aa00 tx=0x7f0760003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.126+0000 7f076d600640 1 -- 172.21.15.32:0/2062873148 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076003f070 con 0x7f0770109f50 2024-08-31T07:34:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.126+0000 7f0777200640 1 -- 172.21.15.32:0/2062873148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07701a6ac0 con 0x7f0770109f50 2024-08-31T07:34:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.126+0000 7f076d600640 1 -- 172.21.15.32:0/2062873148 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0760031b50 con 0x7f0770109f50 2024-08-31T07:34:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.126+0000 7f076d600640 1 -- 172.21.15.32:0/2062873148 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760035030 con 0x7f0770109f50 2024-08-31T07:34:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.126+0000 7f0777200640 1 -- 172.21.15.32:0/2062873148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07701ad280 con 0x7f0770109f50 2024-08-31T07:34:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.130+0000 7f077deb2740 1 Processor -- start 2024-08-31T07:34:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.130+0000 7f077deb2740 1 -- start start 2024-08-31T07:34:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.130+0000 7f077deb2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c33350 0x55eb23fd9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.130+0000 7f077deb2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fd9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.130+0000 7f077deb2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c12670 0x55eb23fda080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.130+0000 7f077deb2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55eb23f4f750 con 0x55eb23c0a490 2024-08-31T07:34:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.130+0000 7f077deb2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55eb23f3ece0 con 0x55eb23c33350 2024-08-31T07:34:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.130+0000 7f077deb2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55eb23f42c90 con 0x55eb23c12670 2024-08-31T07:34:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.131+0000 7f0775200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c33350 0x55eb23fd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.131+0000 7f0775200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c33350 0x55eb23fd9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34006/0 (socket says 172.21.15.32:34006) 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.131+0000 7f0775200640 1 -- 172.21.15.32:0/3952910707 learned_addr learned my addr 172.21.15.32:0/3952910707 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.131+0000 7f0767e00640 1 --2- 172.21.15.32:0/3952910707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c12670 0x55eb23fda080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.131+0000 7f0775200640 1 -- 172.21.15.32:0/3952910707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c12670 msgr2=0x55eb23fda080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.131+0000 7f0775200640 1 --2- 172.21.15.32:0/3952910707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c12670 0x55eb23fda080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.131+0000 7f0775200640 1 -- 172.21.15.32:0/3952910707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 msgr2=0x55eb23fd9b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.131+0000 7f0775200640 1 --2- 172.21.15.32:0/3952910707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fd9b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.131+0000 7f0775200640 1 -- 172.21.15.32:0/3952910707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb23c9e930 con 0x55eb23c33350 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.131+0000 7f076d600640 1 -- 172.21.15.32:0/2062873148 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0760042070 con 0x7f0770109f50 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.132+0000 7f0775200640 1 --2- 172.21.15.32:0/3952910707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c33350 0x55eb23fd9600 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f07680104d0 tx=0x7f0768010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d9fc3313111d562 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.132+0000 7f0765600640 1 -- 172.21.15.32:0/3952910707 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768011760 con 0x55eb23c33350 2024-08-31T07:34:26.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.132+0000 7f0765600640 1 -- 172.21.15.32:0/3952910707 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0768011db0 con 0x55eb23c33350 2024-08-31T07:34:26.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.132+0000 7f0765600640 1 -- 172.21.15.32:0/3952910707 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07680193d0 con 0x55eb23c33350 2024-08-31T07:34:26.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.133+0000 7f077deb2740 1 -- 172.21.15.32:0/3952910707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c33350 msgr2=0x55eb23fd9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.133+0000 7f077deb2740 1 --2- 172.21.15.32:0/3952910707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c33350 0x55eb23fd9600 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f07680104d0 tx=0x7f0768010990 comp rx=0 tx=0).stop 2024-08-31T07:34:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.133+0000 7f076d600640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f07540b4ca0 0x7f07540b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.133+0000 7f076d600640 1 -- 172.21.15.32:0/2062873148 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f0760105b00 con 0x7f0770109f50 2024-08-31T07:34:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.133+0000 7f077deb2740 1 -- 172.21.15.32:0/3952910707 shutdown_connections 2024-08-31T07:34:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.133+0000 7f077deb2740 1 --2- 172.21.15.32:0/3952910707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c12670 0x55eb23fda080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.133+0000 7f077deb2740 1 --2- 172.21.15.32:0/3952910707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fd9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.133+0000 7f077deb2740 1 --2- 172.21.15.32:0/3952910707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c33350 0x55eb23fd9600 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.133+0000 7f077deb2740 1 -- 172.21.15.32:0/3952910707 >> 172.21.15.32:0/3952910707 conn(0x55eb23c27740 msgr2=0x55eb23fd74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.134+0000 7f0775200640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f07540b4ca0 0x7f07540b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.134+0000 7f077deb2740 1 -- 172.21.15.32:0/3952910707 shutdown_connections 2024-08-31T07:34:26.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.134+0000 7f077deb2740 1 -- 172.21.15.32:0/3952910707 wait complete. 2024-08-31T07:34:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.134+0000 7f0775200640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f07540b4ca0 0x7f07540b71b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0768009e90 tx=0x7f0768011f60 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.135+0000 7f077deb2740 1 -- start start 2024-08-31T07:34:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.135+0000 7f077deb2740 1 Processor -- start 2024-08-31T07:34:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.135+0000 7f077deb2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c0a490 0x55eb23fe51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.135+0000 7f077deb2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c12670 0x55eb23fe5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.135+0000 7f077deb2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c33350 0x55eb23ff1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f077deb2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55eb23da1890 con 0x55eb23c33350 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f077deb2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55eb23ec5cd0 con 0x55eb23c0a490 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f077deb2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55eb23fe7de0 con 0x55eb23c12670 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f077deb2740 10 client.0 _ll_register_callbacks cb 0x55eb23f39ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f076fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c0a490 0x55eb23fe51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f076fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c0a490 0x55eb23fe51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34016/0 (socket says 172.21.15.32:34016) 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f076fe00640 1 -- 172.21.15.32:0/2371070248 learned_addr learned my addr 172.21.15.32:0/2371070248 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f0775200640 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c33350 0x55eb23ff1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f0767e00640 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c12670 0x55eb23fe5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f076fe00640 1 -- 172.21.15.32:0/2371070248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c12670 msgr2=0x55eb23fe5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f076fe00640 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c12670 0x55eb23fe5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f076fe00640 1 -- 172.21.15.32:0/2371070248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c33350 msgr2=0x55eb23ff1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f076fe00640 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c33350 0x55eb23ff1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.136+0000 7f076fe00640 1 -- 172.21.15.32:0/2371070248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb23f0f1b0 con 0x55eb23c0a490 2024-08-31T07:34:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.137+0000 7f076fe00640 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c0a490 0x55eb23fe51f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f07601032a0 tx=0x7f076003b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc329d1dcd0b9980 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:26.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.137+0000 7f075e000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:34:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.137+0000 7f075e000640 1 -- 172.21.15.32:0/2371070248 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760049030 con 0x55eb23c0a490 2024-08-31T07:34:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.137+0000 7f075e000640 1 -- 172.21.15.32:0/2371070248 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f07600c8520 con 0x55eb23c0a490 2024-08-31T07:34:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.137+0000 7f077deb2740 1 -- 172.21.15.32:0/2371070248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0768011db0 con 0x55eb23c0a490 2024-08-31T07:34:26.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.137+0000 7f075e000640 1 -- 172.21.15.32:0/2371070248 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f07600cbda0 con 0x55eb23c0a490 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.137+0000 7f077deb2740 10 client.29582 fetch_fsmap learned FSMap version 82 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.137+0000 7f077deb2740 1 -- 172.21.15.32:0/2371070248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x55eb23ff3df0 con 0x55eb23c0a490 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.137+0000 7f075e000640 1 -- 172.21.15.32:0/2371070248 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0768011db0 con 0x55eb23c0a490 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.138+0000 7f075e000640 1 -- 172.21.15.32:0/2371070248 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 82) ==== 8394+0+0 (secure 0 0 0) 0x7f07600d5070 con 0x55eb23c0a490 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.138+0000 7f077deb2740 10 client.29582 fetch_fsmap finished waiting for FSMap version 82 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.138+0000 7f077deb2740 10 client.29582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.138+0000 7f077deb2740 20 client.29582 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.138+0000 7f077deb2740 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55eb23bf1cd0 0x55eb23ff72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.138+0000 7f077deb2740 4 client.29582 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.138+0000 7f077deb2740 1 -- 172.21.15.32:0/2371070248 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb23f516a0 con 0x55eb23bf1cd0 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.138+0000 7f0767e00640 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55eb23bf1cd0 0x55eb23ff72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.139+0000 7f0767e00640 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55eb23bf1cd0 0x55eb23ff72a0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1124810122912086 server_cookie=9bb0280b256be111 in_seq=0 out_seq=0 2024-08-31T07:34:26.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.139+0000 7f075e000640 10 client.29582 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f075e000640 1 -- 172.21.15.32:0/2371070248 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eb23f516a0 con 0x55eb23bf1cd0 2024-08-31T07:34:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f075e000640 10 client.29582 handle_command_reply: tid=0 2024-08-31T07:34:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 client.29582 shutdown 2024-08-31T07:34:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 -- 172.21.15.32:0/2371070248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c0a490 msgr2=0x55eb23fe51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c0a490 0x55eb23fe51f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f07601032a0 tx=0x7f076003b690 comp rx=0 tx=0).stop 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 -- 172.21.15.32:0/2371070248 shutdown_connections 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55eb23bf1cd0 0x55eb23ff72a0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c33350 0x55eb23ff1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c12670 0x55eb23fe5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 --2- 172.21.15.32:0/2371070248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c0a490 0x55eb23fe51f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f075e000640 0 client.29582 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 -- 172.21.15.32:0/2371070248 >> 172.21.15.32:0/2371070248 conn(0x55eb23c27740 msgr2=0x55eb23fdc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 -- 172.21.15.32:0/2371070248 shutdown_connections 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.140+0000 7f077deb2740 1 -- 172.21.15.32:0/2371070248 wait complete. 2024-08-31T07:34:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.141+0000 7f077deb2740 20 client.29582 trim_cache size 0 max 16384 2024-08-31T07:34:26.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.159+0000 7f077deb2740 1 Processor -- start 2024-08-31T07:34:26.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.159+0000 7f077deb2740 1 -- start start 2024-08-31T07:34:26.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f077deb2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fdce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f077deb2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 0x55eb23fde880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f077deb2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 0x55eb23fdedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f077deb2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b5940 con 0x55eb23c0a490 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f077deb2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b7600 con 0x55eb23c12670 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f077deb2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b60d0 con 0x55eb23c33350 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f076fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fdce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f076fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fdce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43820/0 (socket says 172.21.15.32:43820) 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f076fe00640 1 -- 172.21.15.32:0/806248681 learned_addr learned my addr 172.21.15.32:0/806248681 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f0767e00640 1 --2- 172.21.15.32:0/806248681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 0x55eb23fde880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.160+0000 7f0775200640 1 --2- 172.21.15.32:0/806248681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 0x55eb23fdedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.161+0000 7f0767e00640 1 -- 172.21.15.32:0/806248681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 msgr2=0x55eb23fdedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.161+0000 7f0767e00640 1 --2- 172.21.15.32:0/806248681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 0x55eb23fdedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.161+0000 7f0767e00640 1 -- 172.21.15.32:0/806248681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 msgr2=0x55eb23fdce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:26.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.161+0000 7f0767e00640 1 --2- 172.21.15.32:0/806248681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fdce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.161+0000 7f0767e00640 1 -- 172.21.15.32:0/806248681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb23fe3f50 con 0x55eb23c12670 2024-08-31T07:34:26.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.161+0000 7f076fe00640 1 --2- 172.21.15.32:0/806248681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fdce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:26.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.161+0000 7f0767e00640 1 --2- 172.21.15.32:0/806248681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 0x55eb23fde880 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x55eb23ff79e0 tx=0x7f0758005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee29f75fd1bb682f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.161+0000 7f0764c00640 1 -- 172.21.15.32:0/806248681 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758004220 con 0x55eb23c12670 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.161+0000 7f0764c00640 1 -- 172.21.15.32:0/806248681 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07580043c0 con 0x55eb23c12670 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.161+0000 7f0764c00640 1 -- 172.21.15.32:0/806248681 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758003c60 con 0x55eb23c12670 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.162+0000 7f077deb2740 1 -- 172.21.15.32:0/806248681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 msgr2=0x55eb23fde880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.162+0000 7f077deb2740 1 --2- 172.21.15.32:0/806248681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 0x55eb23fde880 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x55eb23ff79e0 tx=0x7f0758005f90 comp rx=0 tx=0).stop 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.162+0000 7f077deb2740 1 -- 172.21.15.32:0/806248681 shutdown_connections 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.162+0000 7f077deb2740 1 --2- 172.21.15.32:0/806248681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 0x55eb23fdedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.162+0000 7f077deb2740 1 --2- 172.21.15.32:0/806248681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 0x55eb23fde880 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.162+0000 7f077deb2740 1 --2- 172.21.15.32:0/806248681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fdce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.162+0000 7f077deb2740 1 -- 172.21.15.32:0/806248681 >> 172.21.15.32:0/806248681 conn(0x55eb23c27740 msgr2=0x55eb23fdc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:26.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.162+0000 7f077deb2740 1 -- 172.21.15.32:0/806248681 shutdown_connections 2024-08-31T07:34:26.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.163+0000 7f077deb2740 1 -- 172.21.15.32:0/806248681 wait complete. 2024-08-31T07:34:26.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.163+0000 7f077deb2740 1 -- start start 2024-08-31T07:34:26.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.163+0000 7f077deb2740 1 Processor -- start 2024-08-31T07:34:26.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f077deb2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f077deb2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 0x55eb23fed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f077deb2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 0x55eb23fe8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f077deb2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b6ed0 con 0x55eb23c0a490 2024-08-31T07:34:26.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f077deb2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b6800 con 0x55eb23c12670 2024-08-31T07:34:26.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f077deb2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f07701b3940 con 0x55eb23c33350 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f077deb2740 10 client.0 _ll_register_callbacks cb 0x7f0760005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f0775200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 0x55eb23fe8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f076fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f0775200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 0x55eb23fe8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33992/0 (socket says 172.21.15.32:33992) 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f0775200640 1 -- 172.21.15.32:0/2793091305 learned_addr learned my addr 172.21.15.32:0/2793091305 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.164+0000 7f0767e00640 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 0x55eb23fed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.165+0000 7f0775200640 1 -- 172.21.15.32:0/2793091305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 msgr2=0x55eb23fed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.165+0000 7f0775200640 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 0x55eb23fed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.165+0000 7f0775200640 1 -- 172.21.15.32:0/2793091305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 msgr2=0x55eb23fed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.165+0000 7f0775200640 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.165+0000 7f0775200640 1 -- 172.21.15.32:0/2793091305 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb23fe88c0 con 0x55eb23c33350 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.165+0000 7f0775200640 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 0x55eb23fe8130 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f076800ebe0 tx=0x7f076800ea70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45ca44732d7ff06d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.165+0000 7f075e000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:34:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.165+0000 7f075e000640 1 -- 172.21.15.32:0/2793091305 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768003970 con 0x55eb23c33350 2024-08-31T07:34:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.165+0000 7f077deb2740 1 -- 172.21.15.32:0/2793091305 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07580043c0 con 0x55eb23c33350 2024-08-31T07:34:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.165+0000 7f075e000640 1 -- 172.21.15.32:0/2793091305 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f076802e050 con 0x55eb23c33350 2024-08-31T07:34:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f075e000640 1 -- 172.21.15.32:0/2793091305 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f0768003b10 con 0x55eb23c33350 2024-08-31T07:34:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f077deb2740 10 client.29584 fetch_fsmap learned FSMap version 82 2024-08-31T07:34:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f077deb2740 1 -- 172.21.15.32:0/2793091305 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x55eb24062800 con 0x55eb23c33350 2024-08-31T07:34:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f075e000640 1 -- 172.21.15.32:0/2793091305 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07580043c0 con 0x55eb23c33350 2024-08-31T07:34:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f075e000640 1 -- 172.21.15.32:0/2793091305 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 82) ==== 8394+0+0 (secure 0 0 0) 0x7f0768011560 con 0x55eb23c33350 2024-08-31T07:34:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f077deb2740 10 client.29584 fetch_fsmap finished waiting for FSMap version 82 2024-08-31T07:34:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f077deb2740 10 client.29584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:34:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f077deb2740 20 client.29584 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f077deb2740 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55eb23bc7080 0x55eb24064d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f077deb2740 4 client.29584 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:34:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.166+0000 7f077deb2740 1 -- 172.21.15.32:0/2793091305 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eb23f4d140 con 0x55eb23bc7080 2024-08-31T07:34:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.167+0000 7f076fe00640 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55eb23bc7080 0x55eb24064d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.167+0000 7f076fe00640 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55eb23bc7080 0x55eb24064d90 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd1c4b8e44dae3ed server_cookie=35e81e885cb9678d in_seq=0 out_seq=0 2024-08-31T07:34:26.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.167+0000 7f075e000640 10 client.29584 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.169+0000 7f075e000640 1 -- 172.21.15.32:0/2793091305 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eb23f4d140 con 0x55eb23bc7080 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.169+0000 7f075e000640 10 client.29584 handle_command_reply: tid=0 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.169+0000 7f077deb2740 1 client.29584 shutdown 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.169+0000 7f077deb2740 1 -- 172.21.15.32:0/2793091305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 msgr2=0x55eb23fe8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.169+0000 7f077deb2740 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 0x55eb23fe8130 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f076800ebe0 tx=0x7f076800ea70 comp rx=0 tx=0).stop 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.170+0000 7f077deb2740 1 -- 172.21.15.32:0/2793091305 shutdown_connections 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.170+0000 7f077deb2740 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55eb23bc7080 0x55eb24064d90 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.170+0000 7f077deb2740 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eb23c33350 0x55eb23fe8130 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.170+0000 7f077deb2740 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eb23c12670 0x55eb23fed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.170+0000 7f077deb2740 1 --2- 172.21.15.32:0/2793091305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb23c0a490 0x55eb23fed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.170+0000 7f075e000640 0 client.29584 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.170+0000 7f077deb2740 1 -- 172.21.15.32:0/2793091305 >> 172.21.15.32:0/2793091305 conn(0x55eb23c27740 msgr2=0x55eb23fd6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.170+0000 7f077deb2740 1 -- 172.21.15.32:0/2793091305 shutdown_connections 2024-08-31T07:34:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.170+0000 7f077deb2740 1 -- 172.21.15.32:0/2793091305 wait complete. 2024-08-31T07:34:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.170+0000 7f077deb2740 20 client.29584 trim_cache size 0 max 16384 2024-08-31T07:34:26.172 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:34:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:34:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "d83ecd52-a6c2-49e8-bdaa-039848b73de4", 2024-08-31T07:34:26.172 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:34:26.172 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:34:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 -- 172.21.15.32:0/2062873148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f07540b4ca0 msgr2=0x7f07540b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f07540b4ca0 0x7f07540b71b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0768009e90 tx=0x7f0768011f60 comp rx=0 tx=0).stop 2024-08-31T07:34:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 -- 172.21.15.32:0/2062873148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0770109f50 msgr2=0x7f07701a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0770109f50 0x7f07701a08c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f076002aa00 tx=0x7f0760003b90 comp rx=0 tx=0).stop 2024-08-31T07:34:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 -- 172.21.15.32:0/2062873148 shutdown_connections 2024-08-31T07:34:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f07540b4ca0 0x7f07540b71b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07701101e0 0x7f07701a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f077010f890 0x7f07701a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 --2- 172.21.15.32:0/2062873148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0770109f50 0x7f07701a08c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 -- 172.21.15.32:0/2062873148 >> 172.21.15.32:0/2062873148 conn(0x7f0770077e80 msgr2=0x7f0770078650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 -- 172.21.15.32:0/2062873148 shutdown_connections 2024-08-31T07:34:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:26.171+0000 7f0777200640 1 -- 172.21.15.32:0/2062873148 wait complete. 2024-08-31T07:34:26.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.64 MB/sec execute 366 sec latency 11504.332 ms 2024-08-31T07:34:26.457 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:26 smithi032 ceph-mon[17252]: pgmap v1050: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-08-31T07:34:26.457 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:26 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:34:26.457 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:26 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:26.458 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:26 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:34:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:26 smithi073 ceph-mon[21995]: pgmap v1050: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-08-31T07:34:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:26 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:34:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:26 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:26 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:34:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:26 smithi084 ceph-mon[22387]: pgmap v1050: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-08-31T07:34:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:26 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:34:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:26 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:26 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:34:27.161 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id e3f2fbef --release --await' 2024-08-31T07:34:27.161 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id e3f2fbef --release --await 2024-08-31T07:34:27.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.60 MB/sec execute 367 sec latency 12504.422 ms 2024-08-31T07:34:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:27 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:34:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:27 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:34:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:27 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:34:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:27 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:34:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:27 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:27 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:27 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:27 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:27 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:27 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:34:27.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:27 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:34:27.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:27 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:34:27.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:27 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:34:27.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:27 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:27 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:27 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:27 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:27 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:27 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:34:27.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:27 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:34:27.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:27 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:34:27.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:27 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:34:27.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:27 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:27 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:27 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:27 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:27.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:27 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:34:28.356 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:34:28.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.57 MB/sec execute 368 sec latency 13504.474 ms 2024-08-31T07:34:28.531 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:28.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:28 smithi032 ceph-mon[17252]: pgmap v1051: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 633 KiB/s wr, 62 op/s 2024-08-31T07:34:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:28 smithi073 ceph-mon[21995]: pgmap v1051: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 633 KiB/s wr, 62 op/s 2024-08-31T07:34:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:28 smithi084 ceph-mon[22387]: pgmap v1051: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 633 KiB/s wr, 62 op/s 2024-08-31T07:34:29.272 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.299+0000 7ff82c600640 1 -- 172.21.15.32:0/2720300852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff824103920 msgr2=0x7ff824103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.299+0000 7ff82c600640 1 --2- 172.21.15.32:0/2720300852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff824103920 0x7ff824103d30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff818009900 tx=0x7ff81802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.299+0000 7ff82c600640 1 -- 172.21.15.32:0/2720300852 shutdown_connections 2024-08-31T07:34:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.299+0000 7ff82c600640 1 --2- 172.21.15.32:0/2720300852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff824109bb0 0x7ff8241103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.299+0000 7ff82c600640 1 --2- 172.21.15.32:0/2720300852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff824109260 0x7ff824109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.299+0000 7ff82c600640 1 --2- 172.21.15.32:0/2720300852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff824103920 0x7ff824103d30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.299+0000 7ff82c600640 1 -- 172.21.15.32:0/2720300852 >> 172.21.15.32:0/2720300852 conn(0x7ff8240ff3b0 msgr2=0x7ff824101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.299+0000 7ff82c600640 1 -- 172.21.15.32:0/2720300852 shutdown_connections 2024-08-31T07:34:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.299+0000 7ff82c600640 1 -- 172.21.15.32:0/2720300852 wait complete. 2024-08-31T07:34:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.300+0000 7ff82c600640 1 Processor -- start 2024-08-31T07:34:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.300+0000 7ff82c600640 1 -- start start 2024-08-31T07:34:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff82c600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff824103920 0x7ff82419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff82c600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff824109260 0x7ff82419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff82c600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff824109bb0 0x7ff8241a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff82c600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff824113870 con 0x7ff824109bb0 2024-08-31T07:34:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff82c600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241136f0 con 0x7ff824109260 2024-08-31T07:34:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff82c600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241139f0 con 0x7ff824103920 2024-08-31T07:34:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff829c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff824103920 0x7ff82419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff82a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff824109bb0 0x7ff8241a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff829c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff824103920 0x7ff82419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34000/0 (socket says 172.21.15.32:34000) 2024-08-31T07:34:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff829c00640 1 -- 172.21.15.32:0/926048333 learned_addr learned my addr 172.21.15.32:0/926048333 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.301+0000 7ff829200640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff824109260 0x7ff82419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.302+0000 7ff829c00640 1 -- 172.21.15.32:0/926048333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff824109260 msgr2=0x7ff82419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.302+0000 7ff829c00640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff824109260 0x7ff82419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.302+0000 7ff829c00640 1 -- 172.21.15.32:0/926048333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff824109bb0 msgr2=0x7ff8241a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.302+0000 7ff829c00640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff824109bb0 0x7ff8241a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.302+0000 7ff829c00640 1 -- 172.21.15.32:0/926048333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8241a23d0 con 0x7ff824103920 2024-08-31T07:34:29.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.302+0000 7ff82a600640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff824109bb0 0x7ff8241a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:29.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.302+0000 7ff829200640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff824109260 0x7ff82419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:29.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.302+0000 7ff829c00640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff824103920 0x7ff82419c460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff818005940 tx=0x7ff8180040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.303+0000 7ff822000640 1 -- 172.21.15.32:0/926048333 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81803f030 con 0x7ff824103920 2024-08-31T07:34:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.303+0000 7ff822000640 1 -- 172.21.15.32:0/926048333 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8180044c0 con 0x7ff824103920 2024-08-31T07:34:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.303+0000 7ff82c600640 1 -- 172.21.15.32:0/926048333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8241a2660 con 0x7ff824103920 2024-08-31T07:34:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.303+0000 7ff822000640 1 -- 172.21.15.32:0/926048333 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818035070 con 0x7ff824103920 2024-08-31T07:34:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.303+0000 7ff82c600640 1 -- 172.21.15.32:0/926048333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241a8e20 con 0x7ff824103920 2024-08-31T07:34:29.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.306+0000 7ff8331fe740 1 Processor -- start 2024-08-31T07:34:29.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.307+0000 7ff8331fe740 1 -- start start 2024-08-31T07:34:29.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.307+0000 7ff822000640 1 -- 172.21.15.32:0/926048333 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff818042030 con 0x7ff824103920 2024-08-31T07:34:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.307+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.307+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8250fe660 0x55f8254cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.307+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 0x55f8254cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.307+0000 7ff8331fe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f825443a90 con 0x55f825127520 2024-08-31T07:34:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.307+0000 7ff8331fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f825433020 con 0x55f825106840 2024-08-31T07:34:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.307+0000 7ff8331fe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f825436fd0 con 0x55f8250fe660 2024-08-31T07:34:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff829c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8250fe660 0x55f8254cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff829c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8250fe660 0x55f8254cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34012/0 (socket says 172.21.15.32:34012) 2024-08-31T07:34:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff829c00640 1 -- 172.21.15.32:0/949936860 learned_addr learned my addr 172.21.15.32:0/949936860 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff829200640 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 0x55f8254cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff82a600640 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff829c00640 1 -- 172.21.15.32:0/949936860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 msgr2=0x55f8254cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff829c00640 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 0x55f8254cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff829c00640 1 -- 172.21.15.32:0/949936860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 msgr2=0x55f8254ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff829c00640 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff829c00640 1 -- 172.21.15.32:0/949936860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f825192b00 con 0x55f8250fe660 2024-08-31T07:34:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.308+0000 7ff82a600640 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:34:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.309+0000 7ff829200640 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 0x55f8254cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:34:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.309+0000 7ff829c00640 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8250fe660 0x55f8254cd430 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff818102ec0 tx=0x7ff81810bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603f45c5b2ffe68f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.309+0000 7ff822000640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff80c0b4d20 0x7ff80c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.309+0000 7ff816a00640 1 -- 172.21.15.32:0/949936860 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818114070 con 0x55f8250fe660 2024-08-31T07:34:29.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.309+0000 7ff816a00640 1 -- 172.21.15.32:0/949936860 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8180cbae0 con 0x55f8250fe660 2024-08-31T07:34:29.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.309+0000 7ff816a00640 1 -- 172.21.15.32:0/949936860 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818090c30 con 0x55f8250fe660 2024-08-31T07:34:29.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.310+0000 7ff822000640 1 -- 172.21.15.32:0/926048333 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7ff818105840 con 0x7ff824103920 2024-08-31T07:34:29.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.310+0000 7ff82a600640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff80c0b4d20 0x7ff80c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.311+0000 7ff8331fe740 1 -- 172.21.15.32:0/949936860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8250fe660 msgr2=0x55f8254cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.311+0000 7ff8331fe740 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8250fe660 0x55f8254cd430 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff818102ec0 tx=0x7ff81810bcb0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.311+0000 7ff82a600640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff80c0b4d20 0x7ff80c0b7230 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff81c005fd0 tx=0x7ff81c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.311+0000 7ff8331fe740 1 -- 172.21.15.32:0/949936860 shutdown_connections 2024-08-31T07:34:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.311+0000 7ff8331fe740 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 0x55f8254cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.311+0000 7ff8331fe740 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8250fe660 0x55f8254cd430 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.311+0000 7ff8331fe740 1 --2- 172.21.15.32:0/949936860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.311+0000 7ff8331fe740 1 -- 172.21.15.32:0/949936860 >> 172.21.15.32:0/949936860 conn(0x55f82511b910 msgr2=0x55f8254cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.311+0000 7ff8331fe740 1 -- 172.21.15.32:0/949936860 shutdown_connections 2024-08-31T07:34:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.311+0000 7ff8331fe740 1 -- 172.21.15.32:0/949936860 wait complete. 2024-08-31T07:34:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.312+0000 7ff8331fe740 1 -- start start 2024-08-31T07:34:29.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.312+0000 7ff8331fe740 1 Processor -- start 2024-08-31T07:34:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.313+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 0x55f8254d8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.313+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 0x55f8254d8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.313+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254d8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.313+0000 7ff8331fe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f825295bb0 con 0x55f825127520 2024-08-31T07:34:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.313+0000 7ff8331fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f8253b9f90 con 0x55f8250fe660 2024-08-31T07:34:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.313+0000 7ff8331fe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f8254db490 con 0x55f825106840 2024-08-31T07:34:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.313+0000 7ff8331fe740 10 client.0 _ll_register_callbacks cb 0x55f8253df710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.313+0000 7ff82a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254d8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff829c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 0x55f8254d8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff829200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 0x55f8254d8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff829c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 0x55f8254d8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34052/0 (socket says 172.21.15.32:34052) 2024-08-31T07:34:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff829c00640 1 -- 172.21.15.32:0/1929346995 learned_addr learned my addr 172.21.15.32:0/1929346995 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff829c00640 1 -- 172.21.15.32:0/1929346995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 msgr2=0x55f8254d8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff829c00640 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 0x55f8254d8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff829c00640 1 -- 172.21.15.32:0/1929346995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 msgr2=0x55f8254d8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff829c00640 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254d8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff829c00640 1 -- 172.21.15.32:0/1929346995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f825209960 con 0x55f8250fe660 2024-08-31T07:34:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff829c00640 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 0x55f8254d8210 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff8180cbf60 tx=0x7ff8180a2850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a9dd2fc14969fdb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.314+0000 7ff80b400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:34:29.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.315+0000 7ff80b400640 1 -- 172.21.15.32:0/1929346995 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8180a14b0 con 0x55f8250fe660 2024-08-31T07:34:29.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.315+0000 7ff80b400640 1 -- 172.21.15.32:0/1929346995 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7ff8180fffe0 con 0x55f8250fe660 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.315+0000 7ff8331fe740 1 -- 172.21.15.32:0/1929346995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f8253d1e30 con 0x55f8250fe660 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.315+0000 7ff80b400640 1 -- 172.21.15.32:0/1929346995 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7ff818114030 con 0x55f8250fe660 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.315+0000 7ff8331fe740 10 client.29597 fetch_fsmap learned FSMap version 82 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.315+0000 7ff8331fe740 1 -- 172.21.15.32:0/1929346995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x55f8254e6c50 con 0x55f8250fe660 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.315+0000 7ff80b400640 1 -- 172.21.15.32:0/1929346995 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f8253d1e30 con 0x55f8250fe660 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.316+0000 7ff80b400640 1 -- 172.21.15.32:0/1929346995 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 82) ==== 8394+0+0 (secure 0 0 0) 0x7ff8180c8260 con 0x55f8250fe660 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.316+0000 7ff8331fe740 10 client.29597 fetch_fsmap finished waiting for FSMap version 82 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.316+0000 7ff8331fe740 10 client.29597 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.316+0000 7ff8331fe740 20 client.29597 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.316+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f8250e5ea0 0x55f8254e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.316+0000 7ff8331fe740 4 client.29597 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:34:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.316+0000 7ff8331fe740 1 -- 172.21.15.32:0/1929346995 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8254459e0 con 0x55f8250e5ea0 2024-08-31T07:34:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.316+0000 7ff829200640 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f8250e5ea0 0x55f8254e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.317+0000 7ff829200640 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f8250e5ea0 0x55f8254e9fb0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277074faf25eed4f server_cookie=48bd8cae05c90582 in_seq=0 out_seq=0 2024-08-31T07:34:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.317+0000 7ff80b400640 10 client.29597 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff80b400640 1 -- 172.21.15.32:0/1929346995 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8254459e0 con 0x55f8250e5ea0 2024-08-31T07:34:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff80b400640 10 client.29597 handle_command_reply: tid=0 2024-08-31T07:34:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff8331fe740 1 client.29597 shutdown 2024-08-31T07:34:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff8331fe740 1 -- 172.21.15.32:0/1929346995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 msgr2=0x55f8254d8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 0x55f8254d8210 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff8180cbf60 tx=0x7ff8180a2850 comp rx=0 tx=0).stop 2024-08-31T07:34:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff8331fe740 1 -- 172.21.15.32:0/1929346995 shutdown_connections 2024-08-31T07:34:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f8250e5ea0 0x55f8254e9fb0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254d8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 0x55f8254d8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1929346995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 0x55f8254d8210 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff8331fe740 1 -- 172.21.15.32:0/1929346995 >> 172.21.15.32:0/1929346995 conn(0x55f82511b910 msgr2=0x55f8254d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:29.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff80b400640 0 client.29597 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:29.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.318+0000 7ff8331fe740 1 -- 172.21.15.32:0/1929346995 shutdown_connections 2024-08-31T07:34:29.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.319+0000 7ff8331fe740 1 -- 172.21.15.32:0/1929346995 wait complete. 2024-08-31T07:34:29.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.319+0000 7ff8331fe740 20 client.29597 trim_cache size 0 max 16384 2024-08-31T07:34:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff8331fe740 1 Processor -- start 2024-08-31T07:34:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff8331fe740 1 -- start start 2024-08-31T07:34:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 0x55f8254ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 0x55f8254ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff8331fe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b14e0 con 0x55f825127520 2024-08-31T07:34:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff8331fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b31a0 con 0x55f8250fe660 2024-08-31T07:34:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff8331fe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b1c70 con 0x55f825106840 2024-08-31T07:34:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff829200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 0x55f8254ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff829c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 0x55f8254ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff829200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 0x55f8254ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34034/0 (socket says 172.21.15.32:34034) 2024-08-31T07:34:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff829200640 1 -- 172.21.15.32:0/583650671 learned_addr learned my addr 172.21.15.32:0/583650671 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.347+0000 7ff82a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.348+0000 7ff829200640 1 -- 172.21.15.32:0/583650671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 msgr2=0x55f8254ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.348+0000 7ff829200640 1 --2- 172.21.15.32:0/583650671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 0x55f8254ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.348+0000 7ff829200640 1 -- 172.21.15.32:0/583650671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 msgr2=0x55f8254ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.348+0000 7ff829200640 1 --2- 172.21.15.32:0/583650671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.348+0000 7ff829200640 1 -- 172.21.15.32:0/583650671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8254d92b0 con 0x55f825106840 2024-08-31T07:34:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.348+0000 7ff82a600640 1 --2- 172.21.15.32:0/583650671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:34:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.348+0000 7ff829200640 1 --2- 172.21.15.32:0/583650671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 0x55f8254ca3d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff810009560 tx=0x7ff810013580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a04b3228d082477 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.348+0000 7ff816000640 1 -- 172.21.15.32:0/583650671 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810004050 con 0x55f825106840 2024-08-31T07:34:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.349+0000 7ff816000640 1 -- 172.21.15.32:0/583650671 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8100054d0 con 0x55f825106840 2024-08-31T07:34:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.349+0000 7ff816000640 1 -- 172.21.15.32:0/583650671 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810004bc0 con 0x55f825106840 2024-08-31T07:34:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.349+0000 7ff8331fe740 1 -- 172.21.15.32:0/583650671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 msgr2=0x55f8254ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.349+0000 7ff8331fe740 1 --2- 172.21.15.32:0/583650671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 0x55f8254ca3d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff810009560 tx=0x7ff810013580 comp rx=0 tx=0).stop 2024-08-31T07:34:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.350+0000 7ff8331fe740 1 -- 172.21.15.32:0/583650671 shutdown_connections 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.350+0000 7ff8331fe740 1 --2- 172.21.15.32:0/583650671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f825127520 0x55f8254ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.350+0000 7ff8331fe740 1 --2- 172.21.15.32:0/583650671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825106840 0x55f8254ca3d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.350+0000 7ff8331fe740 1 --2- 172.21.15.32:0/583650671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8250fe660 0x55f8254ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.350+0000 7ff8331fe740 1 -- 172.21.15.32:0/583650671 >> 172.21.15.32:0/583650671 conn(0x55f82511b910 msgr2=0x55f8254cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.350+0000 7ff8331fe740 1 -- 172.21.15.32:0/583650671 shutdown_connections 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.350+0000 7ff8331fe740 1 -- 172.21.15.32:0/583650671 wait complete. 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.350+0000 7ff8331fe740 1 -- start start 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.350+0000 7ff8331fe740 1 Processor -- start 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f8250fe660 0x55f8254dc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 0x55f8254dcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff8331fe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825127520 0x55f8254dcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff8331fe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b2a70 con 0x55f8250fe660 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff8331fe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b23a0 con 0x55f825106840 2024-08-31T07:34:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff8331fe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241af4e0 con 0x55f825127520 2024-08-31T07:34:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff8331fe740 10 client.0 _ll_register_callbacks cb 0x7ff818005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff82a600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825127520 0x55f8254dcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff829c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f8250fe660 0x55f8254dc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff82a600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825127520 0x55f8254dcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34050/0 (socket says 172.21.15.32:34050) 2024-08-31T07:34:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff82a600640 1 -- 172.21.15.32:0/1571588294 learned_addr learned my addr 172.21.15.32:0/1571588294 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.351+0000 7ff829200640 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 0x55f8254dcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.352+0000 7ff82a600640 1 -- 172.21.15.32:0/1571588294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 msgr2=0x55f8254dcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.352+0000 7ff82a600640 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 0x55f8254dcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.352+0000 7ff82a600640 1 -- 172.21.15.32:0/1571588294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f8250fe660 msgr2=0x55f8254dc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.352+0000 7ff82a600640 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f8250fe660 0x55f8254dc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.352+0000 7ff82a600640 1 -- 172.21.15.32:0/1571588294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f825553930 con 0x55f825127520 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.352+0000 7ff829c00640 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f8250fe660 0x55f8254dc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.352+0000 7ff82a600640 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825127520 0x55f8254dcfe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff81c003bf0 tx=0x7ff81c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ef4df0821203ed server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.352+0000 7ff80b400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.352+0000 7ff80b400640 1 -- 172.21.15.32:0/1571588294 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004800 con 0x55f825127520 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.352+0000 7ff8331fe740 1 -- 172.21.15.32:0/1571588294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8100054d0 con 0x55f825127520 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff80b400640 1 -- 172.21.15.32:0/1571588294 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7ff81c020630 con 0x55f825127520 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff80b400640 1 -- 172.21.15.32:0/1571588294 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7ff81c020830 con 0x55f825127520 2024-08-31T07:34:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff8331fe740 10 client.29602 fetch_fsmap learned FSMap version 82 2024-08-31T07:34:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff8331fe740 1 -- 172.21.15.32:0/1571588294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x55f825554ca0 con 0x55f825127520 2024-08-31T07:34:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff80b400640 1 -- 172.21.15.32:0/1571588294 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8100054d0 con 0x55f825127520 2024-08-31T07:34:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff80b400640 1 -- 172.21.15.32:0/1571588294 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 82) ==== 8394+0+0 (secure 0 0 0) 0x7ff81c034b00 con 0x55f825127520 2024-08-31T07:34:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff8331fe740 10 client.29602 fetch_fsmap finished waiting for FSMap version 82 2024-08-31T07:34:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff8331fe740 10 client.29602 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:34:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff8331fe740 20 client.29602 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f8250bb250 0x55f8255574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff8331fe740 4 client.29602 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e3f2fbef", "release": true, "await": true}] 2024-08-31T07:34:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.353+0000 7ff8331fe740 1 -- 172.21.15.32:0/1571588294 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e3f2fbef", "release": true, "await": true}) -- 0x55f825441480 con 0x55f8250bb250 2024-08-31T07:34:29.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.354+0000 7ff829c00640 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f8250bb250 0x55f8255574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:29.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.354+0000 7ff829c00640 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f8250bb250 0x55f8255574d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c224ac4de2c8ce88 server_cookie=4bf90beae7180775 in_seq=0 out_seq=0 2024-08-31T07:34:29.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.354+0000 7ff80b400640 10 client.29602 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:29.419 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142704 13.53 MB/sec execute 369 sec latency 14504.524 ms 2024-08-31T07:34:29.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.446+0000 7ff80b400640 1 -- 172.21.15.32:0/1571588294 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f825441480 con 0x55f8250bb250 2024-08-31T07:34:29.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.446+0000 7ff80b400640 10 client.29602 handle_command_reply: tid=0 2024-08-31T07:34:29.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.446+0000 7ff8331fe740 1 client.29602 shutdown 2024-08-31T07:34:29.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 1 -- 172.21.15.32:0/1571588294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825127520 msgr2=0x55f8254dcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:29.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825127520 0x55f8254dcfe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff81c003bf0 tx=0x7ff81c004a40 comp rx=0 tx=0).stop 2024-08-31T07:34:29.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 1 -- 172.21.15.32:0/1571588294 shutdown_connections 2024-08-31T07:34:29.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55f8250bb250 0x55f8255574d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f825127520 0x55f8254dcfe0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f825106840 0x55f8254dcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 1 --2- 172.21.15.32:0/1571588294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f8250fe660 0x55f8254dc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff80b400640 0 client.29602 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:29.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 1 -- 172.21.15.32:0/1571588294 >> 172.21.15.32:0/1571588294 conn(0x55f82511b910 msgr2=0x55f8254cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:29.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 1 -- 172.21.15.32:0/1571588294 shutdown_connections 2024-08-31T07:34:29.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 1 -- 172.21.15.32:0/1571588294 wait complete. 2024-08-31T07:34:29.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.447+0000 7ff8331fe740 20 client.29602 trim_cache size 0 max 16384 2024-08-31T07:34:29.450 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:34:29.450 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 82, 2024-08-31T07:34:29.450 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:34:29.450 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 68, 2024-08-31T07:34:29.450 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "e3f2fbef": { 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 68, 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 134.1, 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:34:29.451 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:34:29.452 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:34:29.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:29.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:29.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:29.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:29.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:29.452 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:34:29.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.449+0000 7ff82c600640 1 -- 172.21.15.32:0/926048333 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff80c0b4d20 msgr2=0x7ff80c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.449+0000 7ff82c600640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff80c0b4d20 0x7ff80c0b7230 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff81c005fd0 tx=0x7ff81c009500 comp rx=0 tx=0).stop 2024-08-31T07:34:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.449+0000 7ff82c600640 1 -- 172.21.15.32:0/926048333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff824103920 msgr2=0x7ff82419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.449+0000 7ff82c600640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff824103920 0x7ff82419c460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff818005940 tx=0x7ff8180040c0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.449+0000 7ff82c600640 1 -- 172.21.15.32:0/926048333 shutdown_connections 2024-08-31T07:34:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.449+0000 7ff82c600640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff80c0b4d20 0x7ff80c0b7230 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.449+0000 7ff82c600640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff824109bb0 0x7ff8241a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.449+0000 7ff82c600640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff824109260 0x7ff82419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.449+0000 7ff82c600640 1 --2- 172.21.15.32:0/926048333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff824103920 0x7ff82419c460 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.449+0000 7ff82c600640 1 -- 172.21.15.32:0/926048333 >> 172.21.15.32:0/926048333 conn(0x7ff8240ff3b0 msgr2=0x7ff8241088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:29.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.450+0000 7ff82c600640 1 -- 172.21.15.32:0/926048333 shutdown_connections 2024-08-31T07:34:29.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:29.450+0000 7ff82c600640 1 -- 172.21.15.32:0/926048333 wait complete. 2024-08-31T07:34:30.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 -- 172.21.15.32:0/3865241777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4064110090 msgr2=0x7f40641104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:30.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 --2- 172.21.15.32:0/3865241777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4064110090 0x7f40641104a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4054009860 tx=0x7f405402a340 comp rx=0 tx=0).stop 2024-08-31T07:34:30.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 -- 172.21.15.32:0/3865241777 shutdown_connections 2024-08-31T07:34:30.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 --2- 172.21.15.32:0/3865241777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40641109e0 0x7f4064117210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 --2- 172.21.15.32:0/3865241777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4064110090 0x7f40641104a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 --2- 172.21.15.32:0/3865241777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406410ace0 0x7f406410b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 -- 172.21.15.32:0/3865241777 >> 172.21.15.32:0/3865241777 conn(0x7f4064071090 msgr2=0x7f4064073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:30.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 -- 172.21.15.32:0/3865241777 shutdown_connections 2024-08-31T07:34:30.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 -- 172.21.15.32:0/3865241777 wait complete. 2024-08-31T07:34:30.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 Processor -- start 2024-08-31T07:34:30.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 -- start start 2024-08-31T07:34:30.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406410ace0 0x7f40641a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4064110090 0x7f40641a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40641109e0 0x7f40641a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4064119e50 con 0x7f40641109e0 2024-08-31T07:34:30.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4064119cd0 con 0x7f406410ace0 2024-08-31T07:34:30.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f406bc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4064119fd0 con 0x7f4064110090 2024-08-31T07:34:30.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f4069200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406410ace0 0x7f40641a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f4069200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406410ace0 0x7f40641a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34082/0 (socket says 172.21.15.32:34082) 2024-08-31T07:34:30.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f4069200640 1 -- 172.21.15.32:0/2014593162 learned_addr learned my addr 172.21.15.32:0/2014593162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:30.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f4063e00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4064110090 0x7f40641a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.307+0000 7f4069c00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40641109e0 0x7f40641a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.313+0000 7f4063e00640 1 -- 172.21.15.32:0/2014593162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406410ace0 msgr2=0x7f40641a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.313+0000 7f4063e00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406410ace0 0x7f40641a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.313+0000 7f4063e00640 1 -- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40641109e0 msgr2=0x7f40641a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.313+0000 7f4063e00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40641109e0 0x7f40641a82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.313+0000 7f4063e00640 1 -- 172.21.15.32:0/2014593162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40641a89d0 con 0x7f4064110090 2024-08-31T07:34:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.313+0000 7f4069c00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40641109e0 0x7f40641a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:34:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4063e00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4064110090 0x7f40641a3000 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4054009780 tx=0x7f4054032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4061600640 1 -- 172.21.15.32:0/2014593162 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054037040 con 0x7f4064110090 2024-08-31T07:34:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f406bc00640 1 -- 172.21.15.32:0/2014593162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40641a8c60 con 0x7f4064110090 2024-08-31T07:34:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f406bc00640 1 -- 172.21.15.32:0/2014593162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4064071bc0 con 0x7f4064110090 2024-08-31T07:34:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4061600640 1 -- 172.21.15.32:0/2014593162 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4054041030 con 0x7f4064110090 2024-08-31T07:34:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4061600640 1 -- 172.21.15.32:0/2014593162 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405403d670 con 0x7f4064110090 2024-08-31T07:34:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4072815740 1 Processor -- start 2024-08-31T07:34:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4072815740 1 -- start start 2024-08-31T07:34:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4072815740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fd96170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4072815740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9bf1b0 0x557f3fd966b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4072815740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fd96bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4072815740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557f3fd04770 con 0x557f3f9c7390 2024-08-31T07:34:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4072815740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557f3fcf3d00 con 0x557f3f9e8070 2024-08-31T07:34:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4072815740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557f3fcf7cb0 con 0x557f3f9bf1b0 2024-08-31T07:34:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.319+0000 7f4061600640 1 -- 172.21.15.32:0/2014593162 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4054004050 con 0x7f4064110090 2024-08-31T07:34:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fd96170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fd96170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34094/0 (socket says 172.21.15.32:34094) 2024-08-31T07:34:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069c00640 1 -- 172.21.15.32:0/2650935804 learned_addr learned my addr 172.21.15.32:0/2650935804 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069200640 1 --2- 172.21.15.32:0/2650935804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9bf1b0 0x557f3fd966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4063e00640 1 --2- 172.21.15.32:0/2650935804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fd96bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069c00640 1 -- 172.21.15.32:0/2650935804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9bf1b0 msgr2=0x557f3fd966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069c00640 1 --2- 172.21.15.32:0/2650935804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9bf1b0 0x557f3fd966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069c00640 1 -- 172.21.15.32:0/2650935804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 msgr2=0x557f3fd96bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069c00640 1 --2- 172.21.15.32:0/2650935804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fd96bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069c00640 1 -- 172.21.15.32:0/2650935804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f3fa53650 con 0x557f3f9e8070 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069c00640 1 --2- 172.21.15.32:0/2650935804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fd96170 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4050009cc0 tx=0x7f40500117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28a77941f3768014 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f405a000640 1 -- 172.21.15.32:0/2650935804 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050012450 con 0x557f3f9e8070 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f405a000640 1 -- 172.21.15.32:0/2650935804 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4050012aa0 con 0x557f3f9e8070 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f405a000640 1 -- 172.21.15.32:0/2650935804 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405000fbb0 con 0x557f3f9e8070 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4061600640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40480b4d20 0x7f40480b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4061600640 1 -- 172.21.15.32:0/2014593162 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f4054104350 con 0x7f4064110090 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069200640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40480b4d20 0x7f40480b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 -- 172.21.15.32:0/2650935804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 msgr2=0x557f3fd96170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 --2- 172.21.15.32:0/2650935804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fd96170 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4050009cc0 tx=0x7f40500117b0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 -- 172.21.15.32:0/2650935804 shutdown_connections 2024-08-31T07:34:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 --2- 172.21.15.32:0/2650935804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fd96bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4069200640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40480b4d20 0x7f40480b7230 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x557f3fd94df0 tx=0x7f405c00a6d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 --2- 172.21.15.32:0/2650935804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9bf1b0 0x557f3fd966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 --2- 172.21.15.32:0/2650935804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fd96170 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 -- 172.21.15.32:0/2650935804 >> 172.21.15.32:0/2650935804 conn(0x557f3f9dc460 msgr2=0x557f3fd94060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 -- 172.21.15.32:0/2650935804 shutdown_connections 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 -- 172.21.15.32:0/2650935804 wait complete. 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 -- start start 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.325+0000 7f4072815740 1 Processor -- start 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4072815740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9bf1b0 0x557f3fda1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4072815740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fda23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4072815740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fdadc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4072815740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557f3fb56700 con 0x557f3f9c7390 2024-08-31T07:34:30.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4072815740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557f3fc7acb0 con 0x557f3f9e8070 2024-08-31T07:34:30.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4072815740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557f3fda4950 con 0x557f3f9bf1b0 2024-08-31T07:34:30.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4072815740 10 client.0 _ll_register_callbacks cb 0x557f3fca0430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:30.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4069c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fdadc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4069c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fdadc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:34104/0 (socket says 172.21.15.32:34104) 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4069c00640 1 -- 172.21.15.32:0/2218250734 learned_addr learned my addr 172.21.15.32:0/2218250734 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4069200640 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9bf1b0 0x557f3fda1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4063e00640 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fda23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4069c00640 1 -- 172.21.15.32:0/2218250734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9bf1b0 msgr2=0x557f3fda1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4069c00640 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9bf1b0 0x557f3fda1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4069c00640 1 -- 172.21.15.32:0/2218250734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 msgr2=0x557f3fda23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4069c00640 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fda23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4069c00640 1 -- 172.21.15.32:0/2218250734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3fcbb870 con 0x557f3f9e8070 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4069c00640 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fdadc40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4050009c90 tx=0x7f405000f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff5efb9f6c5d8fa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f404ea00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f404ea00640 1 -- 172.21.15.32:0/2218250734 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050029020 con 0x557f3f9e8070 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f404ea00640 1 -- 172.21.15.32:0/2218250734 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f405000fb40 con 0x557f3f9e8070 2024-08-31T07:34:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4072815740 1 -- 172.21.15.32:0/2218250734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4050012aa0 con 0x557f3f9e8070 2024-08-31T07:34:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f404ea00640 1 -- 172.21.15.32:0/2218250734 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f405000fd40 con 0x557f3f9e8070 2024-08-31T07:34:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4072815740 10 client.29615 fetch_fsmap learned FSMap version 82 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f4072815740 1 -- 172.21.15.32:0/2218250734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x557f3fdb09f0 con 0x557f3f9e8070 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f404ea00640 1 -- 172.21.15.32:0/2218250734 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4050012aa0 con 0x557f3f9e8070 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.330+0000 7f404ea00640 1 -- 172.21.15.32:0/2218250734 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 82) ==== 8394+0+0 (secure 0 0 0) 0x7f4050004880 con 0x557f3f9e8070 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 10 client.29615 fetch_fsmap finished waiting for FSMap version 82 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 10 client.29615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 20 client.29615 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557f3f9a69f0 0x557f3fdb3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 4 client.29615 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 -- 172.21.15.32:0/2218250734 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f3fd066c0 con 0x557f3f9a69f0 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4069200640 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557f3f9a69f0 0x557f3fdb3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4069200640 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557f3f9a69f0 0x557f3fdb3db0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa87fec0e2c73de server_cookie=19f1257ce5e42965 in_seq=0 out_seq=0 2024-08-31T07:34:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f404ea00640 10 client.29615 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f404ea00640 1 -- 172.21.15.32:0/2218250734 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f3fd066c0 con 0x557f3f9a69f0 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f404ea00640 10 client.29615 handle_command_reply: tid=0 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 client.29615 shutdown 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 -- 172.21.15.32:0/2218250734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 msgr2=0x557f3fdadc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fdadc40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4050009c90 tx=0x7f405000f460 comp rx=0 tx=0).stop 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 -- 172.21.15.32:0/2218250734 shutdown_connections 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557f3f9a69f0 0x557f3fdb3db0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9e8070 0x557f3fdadc40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fda23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 --2- 172.21.15.32:0/2218250734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9bf1b0 0x557f3fda1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 -- 172.21.15.32:0/2218250734 >> 172.21.15.32:0/2218250734 conn(0x557f3f9dc460 msgr2=0x557f3fd99350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f404ea00640 0 client.29615 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 -- 172.21.15.32:0/2218250734 shutdown_connections 2024-08-31T07:34:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 1 -- 172.21.15.32:0/2218250734 wait complete. 2024-08-31T07:34:30.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.335+0000 7f4072815740 20 client.29615 trim_cache size 0 max 16384 2024-08-31T07:34:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 Processor -- start 2024-08-31T07:34:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 -- start start 2024-08-31T07:34:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9bf1b0 0x557f3fdadbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fd9bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9e8070 0x557f3fd9b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b5990 con 0x557f3f9c7390 2024-08-31T07:34:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b7650 con 0x557f3f9bf1b0 2024-08-31T07:34:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b6120 con 0x557f3f9e8070 2024-08-31T07:34:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4063e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fd9bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4069c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9e8070 0x557f3fd9b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4063e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fd9bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43898/0 (socket says 172.21.15.32:43898) 2024-08-31T07:34:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4063e00640 1 -- 172.21.15.32:0/2108732978 learned_addr learned my addr 172.21.15.32:0/2108732978 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4063e00640 1 -- 172.21.15.32:0/2108732978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9e8070 msgr2=0x557f3fd9b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4063e00640 1 --2- 172.21.15.32:0/2108732978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9e8070 0x557f3fd9b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4063e00640 1 -- 172.21.15.32:0/2108732978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9bf1b0 msgr2=0x557f3fdadbb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:34:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4063e00640 1 --2- 172.21.15.32:0/2108732978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9bf1b0 0x557f3fdadbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4063e00640 1 -- 172.21.15.32:0/2108732978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f3fd9d480 con 0x557f3f9c7390 2024-08-31T07:34:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4063e00640 1 --2- 172.21.15.32:0/2108732978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fd9bfe0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4054009830 tx=0x7f40540303b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f3a3f07154deeb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4059600640 1 -- 172.21.15.32:0/2108732978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405404b040 con 0x557f3f9c7390 2024-08-31T07:34:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4059600640 1 -- 172.21.15.32:0/2108732978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40540328f0 con 0x557f3f9c7390 2024-08-31T07:34:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4059600640 1 -- 172.21.15.32:0/2108732978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540caa90 con 0x557f3f9c7390 2024-08-31T07:34:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 -- 172.21.15.32:0/2108732978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 msgr2=0x557f3fd9bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 --2- 172.21.15.32:0/2108732978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fd9bfe0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4054009830 tx=0x7f40540303b0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 -- 172.21.15.32:0/2108732978 shutdown_connections 2024-08-31T07:34:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 --2- 172.21.15.32:0/2108732978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9e8070 0x557f3fd9b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 --2- 172.21.15.32:0/2108732978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9c7390 0x557f3fd9bfe0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 --2- 172.21.15.32:0/2108732978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9bf1b0 0x557f3fdadbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 -- 172.21.15.32:0/2108732978 >> 172.21.15.32:0/2108732978 conn(0x557f3f9dc460 msgr2=0x557f3fda0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 -- 172.21.15.32:0/2108732978 shutdown_connections 2024-08-31T07:34:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 -- 172.21.15.32:0/2108732978 wait complete. 2024-08-31T07:34:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 -- start start 2024-08-31T07:34:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.359+0000 7f4072815740 1 Processor -- start 2024-08-31T07:34:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4072815740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9bf1b0 0x557f3fdb5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4072815740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9c7390 0x557f3fdb57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4072815740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9e8070 0x557f3fe1d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4072815740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b6f20 con 0x557f3f9e8070 2024-08-31T07:34:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4072815740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b6850 con 0x557f3f9bf1b0 2024-08-31T07:34:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4072815740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b3990 con 0x557f3f9c7390 2024-08-31T07:34:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4072815740 10 client.0 _ll_register_callbacks cb 0x7f40500128a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:34:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4063e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9c7390 0x557f3fdb57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4069200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9bf1b0 0x557f3fdb5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4063e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9c7390 0x557f3fdb57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:34094/0 (socket says 172.21.15.32:34094) 2024-08-31T07:34:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4063e00640 1 -- 172.21.15.32:0/2095395701 learned_addr learned my addr 172.21.15.32:0/2095395701 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:34:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4069c00640 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9e8070 0x557f3fe1d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4063e00640 1 -- 172.21.15.32:0/2095395701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9bf1b0 msgr2=0x557f3fdb5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4063e00640 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9bf1b0 0x557f3fdb5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4063e00640 1 -- 172.21.15.32:0/2095395701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9e8070 msgr2=0x557f3fe1d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:34:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4063e00640 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9e8070 0x557f3fe1d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4063e00640 1 -- 172.21.15.32:0/2095395701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3fe1da70 con 0x557f3f9c7390 2024-08-31T07:34:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4069c00640 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9e8070 0x557f3fe1d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:34:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f4063e00640 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9c7390 0x557f3fdb57c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f40540caa20 tx=0x7f4054005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcba408adce097f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:34:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f404ea00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:34:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f404ea00640 1 -- 172.21.15.32:0/2095395701 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540cf020 con 0x557f3f9c7390 2024-08-31T07:34:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.364+0000 7f404ea00640 1 -- 172.21.15.32:0/2095395701 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f4054101d70 con 0x557f3f9c7390 2024-08-31T07:34:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 -- 172.21.15.32:0/2095395701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40540328f0 con 0x557f3f9c7390 2024-08-31T07:34:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f404ea00640 1 -- 172.21.15.32:0/2095395701 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f405404b040 con 0x557f3f9c7390 2024-08-31T07:34:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 10 client.29620 fetch_fsmap learned FSMap version 82 2024-08-31T07:34:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 -- 172.21.15.32:0/2095395701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x557f3fe1ebd0 con 0x557f3f9c7390 2024-08-31T07:34:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f404ea00640 1 -- 172.21.15.32:0/2095395701 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40540328f0 con 0x557f3f9c7390 2024-08-31T07:34:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f404ea00640 1 -- 172.21.15.32:0/2095395701 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 82) ==== 8394+0+0 (secure 0 0 0) 0x7f40540d3920 con 0x557f3f9c7390 2024-08-31T07:34:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 10 client.29620 fetch_fsmap finished waiting for FSMap version 82 2024-08-31T07:34:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 10 client.29620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 20 client.29620 populate_metadata read hostname 'smithi032' 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557f3f97bda0 0x557f3fe21310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 4 client.29620 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 -- 172.21.15.32:0/2095395701 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f3fd02160 con 0x557f3f97bda0 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4069200640 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557f3f97bda0 0x557f3fe21310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f404ea00640 10 client.29620 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4069200640 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557f3f97bda0 0x557f3fe21310 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576ee0dd831fb8cf server_cookie=32e9af49cc4040ea in_seq=0 out_seq=0 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f404ea00640 1 -- 172.21.15.32:0/2095395701 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7102 (crc 0 0 0) 0x557f3fd02160 con 0x557f3f97bda0 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f404ea00640 10 client.29620 handle_command_reply: tid=0 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 client.29620 shutdown 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 -- 172.21.15.32:0/2095395701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9c7390 msgr2=0x557f3fdb57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9c7390 0x557f3fdb57c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f40540caa20 tx=0x7f4054005f90 comp rx=0 tx=0).stop 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 -- 172.21.15.32:0/2095395701 shutdown_connections 2024-08-31T07:34:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557f3f97bda0 0x557f3fe21310 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557f3f9e8070 0x557f3fe1d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f3f9c7390 0x557f3fdb57c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 --2- 172.21.15.32:0/2095395701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557f3f9bf1b0 0x557f3fdb5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f404ea00640 0 client.29620 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:34:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 -- 172.21.15.32:0/2095395701 >> 172.21.15.32:0/2095395701 conn(0x557f3f9dc460 msgr2=0x557f3fd97140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 -- 172.21.15.32:0/2095395701 shutdown_connections 2024-08-31T07:34:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 1 -- 172.21.15.32:0/2095395701 wait complete. 2024-08-31T07:34:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f4072815740 20 client.29620 trim_cache size 0 max 16384 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "d83ecd52-a6c2-49e8-bdaa-039848b73de4": { 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "d83ecd52-a6c2-49e8-bdaa-039848b73de4", 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "d83ecd52-a6c2-49e8-bdaa-039848b73de4": { 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:34:26.167734+0000", 2024-08-31T07:34:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:32:41.165270+0000", 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.375 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:34:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:34:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:34:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.377 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:34:30.378 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.379 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:34:30.380 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.381 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.382 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:34:30.383 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.384 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:34:30.387 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:34:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:34:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:34:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:34:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:34:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:34:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:30.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:34:30.388 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:34:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f406bc00640 1 -- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40480b4d20 msgr2=0x7f40480b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f406bc00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40480b4d20 0x7f40480b7230 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x557f3fd94df0 tx=0x7f405c00a6d0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f406bc00640 1 -- 172.21.15.32:0/2014593162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4064110090 msgr2=0x7f40641a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:34:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f406bc00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4064110090 0x7f40641a3000 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4054009780 tx=0x7f4054032d50 comp rx=0 tx=0).stop 2024-08-31T07:34:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f406bc00640 1 -- 172.21.15.32:0/2014593162 shutdown_connections 2024-08-31T07:34:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f406bc00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f40480b4d20 0x7f40480b7230 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f406bc00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40641109e0 0x7f40641a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f406bc00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4064110090 0x7f40641a3000 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f406bc00640 1 --2- 172.21.15.32:0/2014593162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406410ace0 0x7f40641a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:34:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.366+0000 7f406bc00640 1 -- 172.21.15.32:0/2014593162 >> 172.21.15.32:0/2014593162 conn(0x7f4064071090 msgr2=0x7f406410e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:34:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.372+0000 7f406bc00640 1 -- 172.21.15.32:0/2014593162 shutdown_connections 2024-08-31T07:34:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:34:30.372+0000 7f406bc00640 1 -- 172.21.15.32:0/2014593162 wait complete. 2024-08-31T07:34:30.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 143183 13.54 MB/sec execute 370 sec latency 14579.652 ms 2024-08-31T07:34:30.582 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:30 smithi032 ceph-mon[17252]: pgmap v1052: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 515 KiB/s wr, 50 op/s 2024-08-31T07:34:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:30 smithi073 ceph-mon[21995]: pgmap v1052: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 515 KiB/s wr, 50 op/s 2024-08-31T07:34:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:30 smithi084 ceph-mon[22387]: pgmap v1052: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 515 KiB/s wr, 50 op/s 2024-08-31T07:34:31.164 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e3f2fbef', total seconds elapsed: 21.6 2024-08-31T07:34:31.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 143889 13.62 MB/sec execute 371 sec latency 349.739 ms 2024-08-31T07:34:32.241 INFO:tasks.cephfs.filesystem:scrub status for tag:d83ecd52-a6c2-49e8-bdaa-039848b73de4 - {'path': '/', 'tag': 'd83ecd52-a6c2-49e8-bdaa-039848b73de4', 'options': 'recursive,force'} 2024-08-31T07:34:32.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 144691 13.73 MB/sec execute 372 sec latency 73.192 ms 2024-08-31T07:34:32.512 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:32 smithi032 ceph-mon[17252]: pgmap v1053: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 635 KiB/s wr, 61 op/s 2024-08-31T07:34:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:32 smithi073 ceph-mon[21995]: pgmap v1053: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 635 KiB/s wr, 61 op/s 2024-08-31T07:34:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:32 smithi084 ceph-mon[22387]: pgmap v1053: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 635 KiB/s wr, 61 op/s 2024-08-31T07:34:33.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 145230 13.76 MB/sec execute 373 sec latency 57.148 ms 2024-08-31T07:34:34.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 145745 13.79 MB/sec execute 374 sec latency 341.121 ms 2024-08-31T07:34:34.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:34 smithi032 ceph-mon[17252]: pgmap v1054: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-31T07:34:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:34 smithi073 ceph-mon[21995]: pgmap v1054: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-31T07:34:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:34 smithi084 ceph-mon[22387]: pgmap v1054: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-31T07:34:35.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 146478 13.85 MB/sec execute 375 sec latency 80.777 ms 2024-08-31T07:34:36.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 147263 13.92 MB/sec execute 376 sec latency 174.745 ms 2024-08-31T07:34:36.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:36 smithi032 ceph-mon[17252]: pgmap v1055: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-08-31T07:34:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:36 smithi073 ceph-mon[21995]: pgmap v1055: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-08-31T07:34:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:36 smithi084 ceph-mon[22387]: pgmap v1055: 97 pgs: 97 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-08-31T07:34:37.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 148085 14.03 MB/sec execute 377 sec latency 270.236 ms 2024-08-31T07:34:38.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 148773 14.10 MB/sec execute 378 sec latency 1270.293 ms 2024-08-31T07:34:38.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:38 smithi032 ceph-mon[17252]: pgmap v1056: 97 pgs: 97 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.6 MiB/s wr, 301 op/s 2024-08-31T07:34:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:38 smithi073 ceph-mon[21995]: pgmap v1056: 97 pgs: 97 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.6 MiB/s wr, 301 op/s 2024-08-31T07:34:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:38 smithi084 ceph-mon[22387]: pgmap v1056: 97 pgs: 97 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.6 MiB/s wr, 301 op/s 2024-08-31T07:34:39.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 149365 14.13 MB/sec execute 379 sec latency 2270.351 ms 2024-08-31T07:34:40.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 150027 14.19 MB/sec execute 380 sec latency 3270.415 ms 2024-08-31T07:34:40.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:40 smithi032 ceph-mon[17252]: pgmap v1057: 97 pgs: 97 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.0 MiB/s wr, 324 op/s 2024-08-31T07:34:40.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:40 smithi073 ceph-mon[21995]: pgmap v1057: 97 pgs: 97 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.0 MiB/s wr, 324 op/s 2024-08-31T07:34:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:40 smithi084 ceph-mon[22387]: pgmap v1057: 97 pgs: 97 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.0 MiB/s wr, 324 op/s 2024-08-31T07:34:41.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 150744 14.26 MB/sec execute 381 sec latency 4270.464 ms 2024-08-31T07:34:41.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:41 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:34:41.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:41 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:34:41.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:41 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:34:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:41 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:34:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:41 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:34:41.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:41 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:34:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:41 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:34:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:41 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:34:41.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:41 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:34:42.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 151303 14.30 MB/sec execute 382 sec latency 5270.520 ms 2024-08-31T07:34:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:42 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:34:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:42 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:34:42.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:42 smithi073 ceph-mon[21995]: pgmap v1058: 97 pgs: 97 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.0 MiB/s wr, 450 op/s 2024-08-31T07:34:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:42 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:34:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:42 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:34:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:42 smithi084 ceph-mon[22387]: pgmap v1058: 97 pgs: 97 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.0 MiB/s wr, 450 op/s 2024-08-31T07:34:42.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:42 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:34:42.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:42 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:34:42.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:42 smithi032 ceph-mon[17252]: pgmap v1058: 97 pgs: 97 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.0 MiB/s wr, 450 op/s 2024-08-31T07:34:43.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 151908 14.35 MB/sec execute 383 sec latency 6270.586 ms 2024-08-31T07:34:44.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 152532 14.40 MB/sec execute 384 sec latency 7270.644 ms 2024-08-31T07:34:44.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:44 smithi032 ceph-mon[17252]: pgmap v1059: 97 pgs: 97 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 536 op/s 2024-08-31T07:34:44.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:44 smithi073 ceph-mon[21995]: pgmap v1059: 97 pgs: 97 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 536 op/s 2024-08-31T07:34:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:44 smithi084 ceph-mon[22387]: pgmap v1059: 97 pgs: 97 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 536 op/s 2024-08-31T07:34:45.420 INFO:tasks.workunit.client.0.smithi032.stdout: 10 153105 14.43 MB/sec execute 385 sec latency 8093.256 ms 2024-08-31T07:34:46.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 153732 14.47 MB/sec execute 386 sec latency 7090.575 ms 2024-08-31T07:34:46.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:46 smithi032 ceph-mon[17252]: pgmap v1060: 97 pgs: 97 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.5 MiB/s wr, 582 op/s 2024-08-31T07:34:46.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:46 smithi073 ceph-mon[21995]: pgmap v1060: 97 pgs: 97 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.5 MiB/s wr, 582 op/s 2024-08-31T07:34:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:46 smithi084 ceph-mon[22387]: pgmap v1060: 97 pgs: 97 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.5 MiB/s wr, 582 op/s 2024-08-31T07:34:47.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 154486 14.55 MB/sec execute 387 sec latency 128.993 ms 2024-08-31T07:34:48.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 155211 14.60 MB/sec execute 388 sec latency 182.056 ms 2024-08-31T07:34:48.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:48 smithi073 ceph-mon[21995]: pgmap v1061: 97 pgs: 97 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 642 op/s 2024-08-31T07:34:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:48 smithi084 ceph-mon[22387]: pgmap v1061: 97 pgs: 97 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 642 op/s 2024-08-31T07:34:48.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:48 smithi032 ceph-mon[17252]: pgmap v1061: 97 pgs: 97 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 642 op/s 2024-08-31T07:34:49.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 156030 14.68 MB/sec execute 389 sec latency 70.386 ms 2024-08-31T07:34:50.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 156835 14.77 MB/sec execute 390 sec latency 86.790 ms 2024-08-31T07:34:50.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:50 smithi032 ceph-mon[17252]: pgmap v1062: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.2 MiB/s wr, 540 op/s 2024-08-31T07:34:50.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:50 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:50 smithi073 ceph-mon[21995]: pgmap v1062: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.2 MiB/s wr, 540 op/s 2024-08-31T07:34:50.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:50 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:50.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:50 smithi084 ceph-mon[22387]: pgmap v1062: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.2 MiB/s wr, 540 op/s 2024-08-31T07:34:50.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:50 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:51.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157133 14.77 MB/sec execute 391 sec latency 672.076 ms 2024-08-31T07:34:52.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157133 14.73 MB/sec execute 392 sec latency 1672.133 ms 2024-08-31T07:34:52.496 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:52 smithi032 ceph-mon[17252]: pgmap v1063: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-08-31T07:34:52.496 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:52 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:52 smithi073 ceph-mon[21995]: pgmap v1063: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-08-31T07:34:52.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:52 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:52 smithi084 ceph-mon[22387]: pgmap v1063: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 596 op/s 2024-08-31T07:34:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:52 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:53.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157152 14.70 MB/sec execute 393 sec latency 2672.199 ms 2024-08-31T07:34:54.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157170 14.66 MB/sec execute 394 sec latency 3672.262 ms 2024-08-31T07:34:54.484 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:54 smithi032 ceph-mon[17252]: pgmap v1064: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 566 op/s 2024-08-31T07:34:54.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:54 smithi073 ceph-mon[21995]: pgmap v1064: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 566 op/s 2024-08-31T07:34:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:54 smithi084 ceph-mon[22387]: pgmap v1064: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 566 op/s 2024-08-31T07:34:55.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157205 14.62 MB/sec execute 395 sec latency 4672.322 ms 2024-08-31T07:34:55.594 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:55 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:55.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:55 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:55.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:55 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:34:56.420 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:56 smithi032 ceph-mon[17252]: pgmap v1065: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-08-31T07:34:56.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157338 14.59 MB/sec execute 396 sec latency 5612.152 ms 2024-08-31T07:34:56.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:56 smithi073 ceph-mon[21995]: pgmap v1065: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-08-31T07:34:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:56 smithi084 ceph-mon[22387]: pgmap v1065: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-08-31T07:34:57.351 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-31T07:34:57.351 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-08-31T07:34:57.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:34:57.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157339 14.56 MB/sec execute 397 sec latency 6612.208 ms 2024-08-31T07:34:57.589 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:57 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:34:57.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:57 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:34:57.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:57 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:34:57.825 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:34:58.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157339 14.52 MB/sec execute 398 sec latency 7612.300 ms 2024-08-31T07:34:58.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:58 smithi032 ceph-mon[17252]: pgmap v1066: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 552 op/s 2024-08-31T07:34:58.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:34:58.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:34:58.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:34:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:34:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:58 smithi073 ceph-mon[21995]: pgmap v1066: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 552 op/s 2024-08-31T07:34:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:34:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:34:58.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:34:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:34:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:58 smithi084 ceph-mon[22387]: pgmap v1066: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 552 op/s 2024-08-31T07:34:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:34:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:34:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:34:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:34:59.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157342 14.48 MB/sec execute 399 sec latency 8612.352 ms 2024-08-31T07:35:00.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157342 14.45 MB/sec execute 400 sec latency 9612.396 ms 2024-08-31T07:35:00.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:00 smithi084 ceph-mon[22387]: pgmap v1067: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 469 op/s 2024-08-31T07:35:00.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:00 smithi032 ceph-mon[17252]: pgmap v1067: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 469 op/s 2024-08-31T07:35:00.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:00 smithi073 ceph-mon[21995]: pgmap v1067: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 469 op/s 2024-08-31T07:35:01.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157342 14.41 MB/sec execute 401 sec latency 10612.445 ms 2024-08-31T07:35:02.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:35:02.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157342 14.38 MB/sec execute 402 sec latency 11612.498 ms 2024-08-31T07:35:02.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:02 smithi032 ceph-mon[17252]: pgmap v1068: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 546 op/s 2024-08-31T07:35:02.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:02 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:02 smithi084 ceph-mon[22387]: pgmap v1068: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 546 op/s 2024-08-31T07:35:02.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:02 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:02 smithi073 ceph-mon[21995]: pgmap v1068: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 546 op/s 2024-08-31T07:35:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:02 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:03.421 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157344 14.34 MB/sec execute 403 sec latency 12612.554 ms 2024-08-31T07:35:03.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.470+0000 7f994a200640 1 -- 172.21.15.32:0/1534313853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f994410bbf0 msgr2=0x7f9944112420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:03.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.470+0000 7f994a200640 1 --2- 172.21.15.32:0/1534313853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f994410bbf0 0x7f9944112420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f993c009ca0 tx=0x7f993c02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:35:03.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.470+0000 7f994a200640 1 -- 172.21.15.32:0/1534313853 shutdown_connections 2024-08-31T07:35:03.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.470+0000 7f994a200640 1 --2- 172.21.15.32:0/1534313853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f994410bbf0 0x7f9944112420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:03.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.470+0000 7f994a200640 1 --2- 172.21.15.32:0/1534313853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994410b2a0 0x7f994410b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:03.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.470+0000 7f994a200640 1 --2- 172.21.15.32:0/1534313853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9944105960 0x7f9944105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.470+0000 7f994a200640 1 -- 172.21.15.32:0/1534313853 >> 172.21.15.32:0/1534313853 conn(0x7f99441013d0 msgr2=0x7f9944103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.470+0000 7f994a200640 1 -- 172.21.15.32:0/1534313853 shutdown_connections 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.470+0000 7f994a200640 1 -- 172.21.15.32:0/1534313853 wait complete. 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.472+0000 7f994a200640 1 Processor -- start 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.472+0000 7f994a200640 1 -- start start 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f994a200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9944105960 0x7f994419c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f994a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f994410b2a0 0x7f994419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f994a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994410bbf0 0x7f99441a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f994a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99441158b0 con 0x7f994410b2a0 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f994a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9944115730 con 0x7f994410bbf0 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f994a200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9944115a30 con 0x7f9944105960 2024-08-31T07:35:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f9943e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f994410b2a0 0x7f994419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f9943e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f994410b2a0 0x7f994419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37972/0 (socket says 172.21.15.32:37972) 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f9943e00640 1 -- 172.21.15.32:0/3577478715 learned_addr learned my addr 172.21.15.32:0/3577478715 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f9949800640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994410bbf0 0x7f99441a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f9948e00640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9944105960 0x7f994419c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f9949800640 1 -- 172.21.15.32:0/3577478715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9944105960 msgr2=0x7f994419c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f9949800640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9944105960 0x7f994419c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f9949800640 1 -- 172.21.15.32:0/3577478715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f994410b2a0 msgr2=0x7f994419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f9949800640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f994410b2a0 0x7f994419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.473+0000 7f9949800640 1 -- 172.21.15.32:0/3577478715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99441a2190 con 0x7f994410bbf0 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.474+0000 7f9943e00640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f994410b2a0 0x7f994419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.474+0000 7f9949800640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994410bbf0 0x7f99441a1a60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f993c02ade0 tx=0x7f993c031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.474+0000 7f9941600640 1 -- 172.21.15.32:0/3577478715 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c003b70 con 0x7f994410bbf0 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.474+0000 7f994a200640 1 -- 172.21.15.32:0/3577478715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99441a2420 con 0x7f994410bbf0 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.474+0000 7f9941600640 1 -- 172.21.15.32:0/3577478715 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993c031910 con 0x7f994410bbf0 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.474+0000 7f9941600640 1 -- 172.21.15.32:0/3577478715 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c02f950 con 0x7f994410bbf0 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.474+0000 7f994a200640 1 -- 172.21.15.32:0/3577478715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99441a8be0 con 0x7f994410bbf0 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.475+0000 7f994a200640 1 -- 172.21.15.32:0/3577478715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9944105d70 con 0x7f994410bbf0 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.479+0000 7f9941600640 1 -- 172.21.15.32:0/3577478715 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f993c004b40 con 0x7f994410bbf0 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.480+0000 7f9941600640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f992c0b4d20 0x7f992c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.481+0000 7f9941600640 1 -- 172.21.15.32:0/3577478715 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f993c1066a0 con 0x7f994410bbf0 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.481+0000 7f9948e00640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f992c0b4d20 0x7f992c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.481+0000 7f9941600640 1 -- 172.21.15.32:0/3577478715 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f993c0cd480 con 0x7f994410bbf0 2024-08-31T07:35:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.481+0000 7f9948e00640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f992c0b4d20 0x7f992c0b7230 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9938009bd0 tx=0x7f9938005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:03.571 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:03.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.734+0000 7f994a200640 1 -- 172.21.15.32:0/3577478715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f994419d350 con 0x7f994410bbf0 2024-08-31T07:35:03.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.735+0000 7f9941600640 1 -- 172.21.15.32:0/3577478715 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+11977 (secure 0 0 0) 0x7f994419d350 con 0x7f994410bbf0 2024-08-31T07:35:03.736 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:35:03.737 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"btime":"2024-08-31T07:35:01:444248+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":86,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:35:00.978669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:35:03.737 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 86 2024-08-31T07:35:03.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.737+0000 7f994a200640 1 -- 172.21.15.32:0/3577478715 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f992c0b4d20 msgr2=0x7f992c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.737+0000 7f994a200640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f992c0b4d20 0x7f992c0b7230 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9938009bd0 tx=0x7f9938005f90 comp rx=0 tx=0).stop 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.737+0000 7f994a200640 1 -- 172.21.15.32:0/3577478715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994410bbf0 msgr2=0x7f99441a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.737+0000 7f994a200640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994410bbf0 0x7f99441a1a60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f993c02ade0 tx=0x7f993c031660 comp rx=0 tx=0).stop 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.738+0000 7f994a200640 1 -- 172.21.15.32:0/3577478715 shutdown_connections 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.738+0000 7f994a200640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f992c0b4d20 0x7f992c0b7230 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.738+0000 7f994a200640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f994410bbf0 0x7f99441a1a60 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.738+0000 7f994a200640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f994410b2a0 0x7f994419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.738+0000 7f994a200640 1 --2- 172.21.15.32:0/3577478715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9944105960 0x7f994419c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.738+0000 7f994a200640 1 -- 172.21.15.32:0/3577478715 >> 172.21.15.32:0/3577478715 conn(0x7f99441013d0 msgr2=0x7f99441037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.738+0000 7f994a200640 1 -- 172.21.15.32:0/3577478715 shutdown_connections 2024-08-31T07:35:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:03.738+0000 7f994a200640 1 -- 172.21.15.32:0/3577478715 wait complete. 2024-08-31T07:35:04.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157348 14.31 MB/sec execute 404 sec latency 13612.594 ms 2024-08-31T07:35:04.707 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:04 smithi032 ceph-mon[17252]: pgmap v1069: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 496 op/s 2024-08-31T07:35:04.707 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:04 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3577478715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:04.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:04 smithi084 ceph-mon[22387]: pgmap v1069: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 496 op/s 2024-08-31T07:35:04.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:04 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3577478715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:04.805 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-08-31T07:35:04.805 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-08-31T07:35:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:04 smithi073 ceph-mon[21995]: pgmap v1069: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 496 op/s 2024-08-31T07:35:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:04 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3577478715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:05.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.162+0000 7fcce6a00640 1 -- 172.21.15.32:0/3107285838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcce0077120 msgr2=0x7fcce0075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:05.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.162+0000 7fcce6a00640 1 --2- 172.21.15.32:0/3107285838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcce0077120 0x7fcce0075550 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009900 tx=0x7fccd402a410 comp rx=0 tx=0).stop 2024-08-31T07:35:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.162+0000 7fcce6a00640 1 -- 172.21.15.32:0/3107285838 shutdown_connections 2024-08-31T07:35:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.162+0000 7fcce6a00640 1 --2- 172.21.15.32:0/3107285838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce0114020 0x7fcce01164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.162+0000 7fcce6a00640 1 --2- 172.21.15.32:0/3107285838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcce0075b70 0x7fcce0075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.162+0000 7fcce6a00640 1 --2- 172.21.15.32:0/3107285838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcce0077120 0x7fcce0075550 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.162+0000 7fcce6a00640 1 -- 172.21.15.32:0/3107285838 >> 172.21.15.32:0/3107285838 conn(0x7fcce00ff1e0 msgr2=0x7fcce0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.163+0000 7fcce6a00640 1 -- 172.21.15.32:0/3107285838 shutdown_connections 2024-08-31T07:35:05.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.163+0000 7fcce6a00640 1 -- 172.21.15.32:0/3107285838 wait complete. 2024-08-31T07:35:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.164+0000 7fcce6a00640 1 Processor -- start 2024-08-31T07:35:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.164+0000 7fcce6a00640 1 -- start start 2024-08-31T07:35:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcce0075b70 0x7fcce01a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce0077120 0x7fcce01a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce6a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcce0114020 0x7fcce01a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce6a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0119d10 con 0x7fcce0075b70 2024-08-31T07:35:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce6a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0119b90 con 0x7fcce0077120 2024-08-31T07:35:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce6a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0119e90 con 0x7fcce0114020 2024-08-31T07:35:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce6000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcce0114020 0x7fcce01a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce0077120 0x7fcce01a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce0077120 0x7fcce01a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47648/0 (socket says 172.21.15.32:47648) 2024-08-31T07:35:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce4c00640 1 -- 172.21.15.32:0/2711935085 learned_addr learned my addr 172.21.15.32:0/2711935085 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.165+0000 7fcce6000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcce0114020 0x7fcce01a5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52448/0 (socket says 172.21.15.32:52448) 2024-08-31T07:35:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.166+0000 7fcce5600640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcce0075b70 0x7fcce01a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.166+0000 7fcce4c00640 1 -- 172.21.15.32:0/2711935085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcce0114020 msgr2=0x7fcce01a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.166+0000 7fcce4c00640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcce0114020 0x7fcce01a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.166+0000 7fcce4c00640 1 -- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcce0075b70 msgr2=0x7fcce01a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.166+0000 7fcce4c00640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcce0075b70 0x7fcce01a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.166+0000 7fcce4c00640 1 -- 172.21.15.32:0/2711935085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce01a6670 con 0x7fcce0077120 2024-08-31T07:35:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.166+0000 7fcce5600640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcce0075b70 0x7fcce01a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.166+0000 7fcce4c00640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce0077120 0x7fcce01a0ca0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00b470 tx=0x7fcccc00b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.167+0000 7fccde000640 1 -- 172.21.15.32:0/2711935085 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc015020 con 0x7fcce0077120 2024-08-31T07:35:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.167+0000 7fcce6a00640 1 -- 172.21.15.32:0/2711935085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce01a6960 con 0x7fcce0077120 2024-08-31T07:35:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.167+0000 7fccde000640 1 -- 172.21.15.32:0/2711935085 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcccc011d40 con 0x7fcce0077120 2024-08-31T07:35:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.167+0000 7fccde000640 1 -- 172.21.15.32:0/2711935085 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc00f340 con 0x7fcce0077120 2024-08-31T07:35:05.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.167+0000 7fcce6a00640 1 -- 172.21.15.32:0/2711935085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce01ad0c0 con 0x7fcce0077120 2024-08-31T07:35:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.172+0000 7fcceec5d740 1 Processor -- start 2024-08-31T07:35:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.172+0000 7fccde000640 1 -- 172.21.15.32:0/2711935085 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcccc018020 con 0x7fcce0077120 2024-08-31T07:35:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.172+0000 7fcceec5d740 1 -- start start 2024-08-31T07:35:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.172+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fe3070 0x55e0a7388a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.172+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7388f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.172+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.172+0000 7fcceec5d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e0a72ff770 con 0x55e0a6fba1b0 2024-08-31T07:35:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.172+0000 7fcceec5d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e0a72eed00 con 0x55e0a6fe3070 2024-08-31T07:35:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.172+0000 7fcceec5d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e0a72f2cb0 con 0x55e0a6fc2390 2024-08-31T07:35:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.173+0000 7fcce4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.173+0000 7fcce4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73894b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52458/0 (socket says 172.21.15.32:52458) 2024-08-31T07:35:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.173+0000 7fcce4c00640 1 -- 172.21.15.32:0/180968561 learned_addr learned my addr 172.21.15.32:0/180968561 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:05.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.173+0000 7fcce5600640 1 --2- 172.21.15.32:0/180968561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7388f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.173+0000 7fcce4c00640 1 -- 172.21.15.32:0/180968561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fe3070 msgr2=0x55e0a7388a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:35:05.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.173+0000 7fcce4c00640 1 --2- 172.21.15.32:0/180968561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fe3070 0x55e0a7388a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.173+0000 7fcce4c00640 1 -- 172.21.15.32:0/180968561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 msgr2=0x55e0a7388f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:05.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.174+0000 7fcce4c00640 1 --2- 172.21.15.32:0/180968561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7388f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.174+0000 7fcce4c00640 1 -- 172.21.15.32:0/180968561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a704e650 con 0x55e0a6fc2390 2024-08-31T07:35:05.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.174+0000 7fcce5600640 1 --2- 172.21.15.32:0/180968561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7388f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:05.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.174+0000 7fcce4c00640 1 --2- 172.21.15.32:0/180968561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73894b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0e2660 tx=0x7fcccc0ed640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb4a929bec97961c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:05.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.174+0000 7fccd2600640 1 -- 172.21.15.32:0/180968561 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0f8070 con 0x55e0a6fc2390 2024-08-31T07:35:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.174+0000 7fccd2600640 1 -- 172.21.15.32:0/180968561 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcccc0ede40 con 0x55e0a6fc2390 2024-08-31T07:35:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.174+0000 7fccd2600640 1 -- 172.21.15.32:0/180968561 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0e58a0 con 0x55e0a6fc2390 2024-08-31T07:35:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.175+0000 7fccde000640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccc80b4ca0 0x7fccc80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.175+0000 7fccde000640 1 -- 172.21.15.32:0/2711935085 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7fcccc0e4f50 con 0x7fcce0077120 2024-08-31T07:35:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.175+0000 7fcceec5d740 1 -- 172.21.15.32:0/180968561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 msgr2=0x55e0a73894b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.175+0000 7fcceec5d740 1 --2- 172.21.15.32:0/180968561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73894b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0e2660 tx=0x7fcccc0ed640 comp rx=0 tx=0).stop 2024-08-31T07:35:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.176+0000 7fcceec5d740 1 -- 172.21.15.32:0/180968561 shutdown_connections 2024-08-31T07:35:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.176+0000 7fcceec5d740 1 --2- 172.21.15.32:0/180968561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73894b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.176+0000 7fcceec5d740 1 --2- 172.21.15.32:0/180968561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7388f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.176+0000 7fcceec5d740 1 --2- 172.21.15.32:0/180968561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fe3070 0x55e0a7388a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.176+0000 7fcceec5d740 1 -- 172.21.15.32:0/180968561 >> 172.21.15.32:0/180968561 conn(0x55e0a6fd7460 msgr2=0x55e0a7386920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.176+0000 7fcce6000640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccc80b4ca0 0x7fccc80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.177+0000 7fcceec5d740 1 -- 172.21.15.32:0/180968561 shutdown_connections 2024-08-31T07:35:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.177+0000 7fcceec5d740 1 -- 172.21.15.32:0/180968561 wait complete. 2024-08-31T07:35:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.177+0000 7fcce6000640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccc80b4ca0 0x7fccc80b71b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fccd8005fd0 tx=0x7fccd80096a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.178+0000 7fcceec5d740 1 -- start start 2024-08-31T07:35:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.178+0000 7fcceec5d740 1 Processor -- start 2024-08-31T07:35:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.179+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7394680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.179+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a7394c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.179+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fe3070 0x55e0a73a0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.179+0000 7fcceec5d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e0a7151700 con 0x55e0a6fba1b0 2024-08-31T07:35:05.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.179+0000 7fcceec5d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e0a7275cb0 con 0x55e0a6fe3070 2024-08-31T07:35:05.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.179+0000 7fcceec5d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e0a7397210 con 0x55e0a6fc2390 2024-08-31T07:35:05.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.179+0000 7fcceec5d740 10 client.0 _ll_register_callbacks cb 0x55e0a729b430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:05.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.180+0000 7fcce5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7394680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.180+0000 7fcce5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7394680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38018/0 (socket says 172.21.15.32:38018) 2024-08-31T07:35:05.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.180+0000 7fcce5600640 1 -- 172.21.15.32:0/3800312466 learned_addr learned my addr 172.21.15.32:0/3800312466 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:05.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.180+0000 7fcce6000640 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fe3070 0x55e0a73a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.180+0000 7fcce4c00640 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a7394c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.180+0000 7fcce5600640 1 -- 172.21.15.32:0/3800312466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 msgr2=0x55e0a7394c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:05.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.180+0000 7fcce5600640 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a7394c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.180+0000 7fcce5600640 1 -- 172.21.15.32:0/3800312466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fe3070 msgr2=0x55e0a73a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:05.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.180+0000 7fcce5600640 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fe3070 0x55e0a73a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.181+0000 7fcce5600640 1 -- 172.21.15.32:0/3800312466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a72b6870 con 0x55e0a6fba1b0 2024-08-31T07:35:05.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.181+0000 7fcce6000640 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fe3070 0x55e0a73a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:05.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.181+0000 7fcce5600640 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7394680 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009900 tx=0x7fccd4005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa22f1bbafd0e70f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:05.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.181+0000 7fccc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:35:05.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.182+0000 7fccc6a00640 1 -- 172.21.15.32:0/3800312466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4035030 con 0x55e0a6fba1b0 2024-08-31T07:35:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.182+0000 7fccc6a00640 1 -- 172.21.15.32:0/3800312466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7fccd40048c0 con 0x55e0a6fba1b0 2024-08-31T07:35:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.182+0000 7fcceec5d740 1 -- 172.21.15.32:0/3800312466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e0a73016c0 con 0x55e0a6fba1b0 2024-08-31T07:35:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.182+0000 7fccc6a00640 1 -- 172.21.15.32:0/3800312466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fccd4004ac0 con 0x55e0a6fba1b0 2024-08-31T07:35:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.182+0000 7fcceec5d740 10 client.19590 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.182+0000 7fcceec5d740 1 -- 172.21.15.32:0/3800312466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x55e0a73a3270 con 0x55e0a6fba1b0 2024-08-31T07:35:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.183+0000 7fccc6a00640 1 -- 172.21.15.32:0/3800312466 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e0a73016c0 con 0x55e0a6fba1b0 2024-08-31T07:35:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.183+0000 7fccc6a00640 1 -- 172.21.15.32:0/3800312466 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7fccd40314a0 con 0x55e0a6fba1b0 2024-08-31T07:35:05.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.184+0000 7fcceec5d740 10 client.19590 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:05.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.184+0000 7fcceec5d740 10 client.19590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:35:05.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.184+0000 7fcceec5d740 20 client.19590 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:05.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.184+0000 7fcceec5d740 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e0a6fa19f0 0x55e0a73a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.184+0000 7fcceec5d740 4 client.19590 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:05.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.184+0000 7fcceec5d740 1 -- 172.21.15.32:0/3800312466 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0a72fd160 con 0x55e0a6fa19f0 2024-08-31T07:35:05.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.184+0000 7fcce4c00640 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e0a6fa19f0 0x55e0a73a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.184+0000 7fcce4c00640 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e0a6fa19f0 0x55e0a73a6b90 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68dad2744945b4 server_cookie=bef14cdeac192683 in_seq=0 out_seq=0 2024-08-31T07:35:05.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.184+0000 7fccc6a00640 10 client.19590 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:05.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.185+0000 7fccc6a00640 1 -- 172.21.15.32:0/3800312466 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0a72fd160 con 0x55e0a6fa19f0 2024-08-31T07:35:05.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.185+0000 7fccc6a00640 10 client.19590 handle_command_reply: tid=0 2024-08-31T07:35:05.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.185+0000 7fcceec5d740 1 client.19590 shutdown 2024-08-31T07:35:05.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fcceec5d740 1 -- 172.21.15.32:0/3800312466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 msgr2=0x55e0a7394680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:05.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fcceec5d740 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7394680 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009900 tx=0x7fccd4005360 comp rx=0 tx=0).stop 2024-08-31T07:35:05.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fcceec5d740 1 -- 172.21.15.32:0/3800312466 shutdown_connections 2024-08-31T07:35:05.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fcceec5d740 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e0a6fa19f0 0x55e0a73a6b90 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fcceec5d740 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fe3070 0x55e0a73a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fcceec5d740 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a7394c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fcceec5d740 1 --2- 172.21.15.32:0/3800312466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7394680 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fccc6a00640 0 client.19590 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:05.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fcceec5d740 1 -- 172.21.15.32:0/3800312466 >> 172.21.15.32:0/3800312466 conn(0x55e0a6fd7460 msgr2=0x55e0a738bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:05.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fcceec5d740 1 -- 172.21.15.32:0/3800312466 shutdown_connections 2024-08-31T07:35:05.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.186+0000 7fcceec5d740 1 -- 172.21.15.32:0/3800312466 wait complete. 2024-08-31T07:35:05.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.187+0000 7fcceec5d740 20 client.19590 trim_cache size 0 max 16384 2024-08-31T07:35:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.220+0000 7fcceec5d740 1 Processor -- start 2024-08-31T07:35:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.220+0000 7fcceec5d740 1 -- start start 2024-08-31T07:35:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.221+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7386a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.221+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fc2390 0x55e0a73a0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.221+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fe3070 0x55e0a7385500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.221+0000 7fcceec5d740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b5780 con 0x55e0a6fba1b0 2024-08-31T07:35:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.221+0000 7fcceec5d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b74a0 con 0x55e0a6fc2390 2024-08-31T07:35:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.221+0000 7fcceec5d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b5f10 con 0x55e0a6fe3070 2024-08-31T07:35:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.221+0000 7fcce4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fc2390 0x55e0a73a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.221+0000 7fcce6000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fe3070 0x55e0a7385500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.222+0000 7fcce4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fc2390 0x55e0a73a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47670/0 (socket says 172.21.15.32:47670) 2024-08-31T07:35:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.222+0000 7fcce4c00640 1 -- 172.21.15.32:0/3050804233 learned_addr learned my addr 172.21.15.32:0/3050804233 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.222+0000 7fcce5600640 1 --2- 172.21.15.32:0/3050804233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7386a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.224+0000 7fcce4c00640 1 -- 172.21.15.32:0/3050804233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fe3070 msgr2=0x55e0a7385500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.224+0000 7fcce4c00640 1 --2- 172.21.15.32:0/3050804233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fe3070 0x55e0a7385500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.224+0000 7fcce4c00640 1 -- 172.21.15.32:0/3050804233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 msgr2=0x55e0a7386a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.224+0000 7fcce4c00640 1 --2- 172.21.15.32:0/3050804233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7386a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.224+0000 7fcce4c00640 1 -- 172.21.15.32:0/3050804233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a73933e0 con 0x55e0a6fc2390 2024-08-31T07:35:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.224+0000 7fcce6000640 1 --2- 172.21.15.32:0/3050804233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fe3070 0x55e0a7385500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.225+0000 7fcce5600640 1 --2- 172.21.15.32:0/3050804233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7386a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:35:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.224+0000 7fcce4c00640 1 --2- 172.21.15.32:0/3050804233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fc2390 0x55e0a73a0400 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0ce010 tx=0x7fcccc0f3f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a3865da106cad9f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.225+0000 7fccd1c00640 1 -- 172.21.15.32:0/3050804233 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0ab460 con 0x55e0a6fc2390 2024-08-31T07:35:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.225+0000 7fccd1c00640 1 -- 172.21.15.32:0/3050804233 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcccc010eb0 con 0x55e0a6fc2390 2024-08-31T07:35:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.225+0000 7fccd1c00640 1 -- 172.21.15.32:0/3050804233 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0aae00 con 0x55e0a6fc2390 2024-08-31T07:35:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.227+0000 7fcceec5d740 1 -- 172.21.15.32:0/3050804233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fc2390 msgr2=0x55e0a73a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.227+0000 7fcceec5d740 1 --2- 172.21.15.32:0/3050804233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fc2390 0x55e0a73a0400 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0ce010 tx=0x7fcccc0f3f10 comp rx=0 tx=0).stop 2024-08-31T07:35:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.228+0000 7fcceec5d740 1 -- 172.21.15.32:0/3050804233 shutdown_connections 2024-08-31T07:35:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.228+0000 7fcceec5d740 1 --2- 172.21.15.32:0/3050804233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fe3070 0x55e0a7385500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.228+0000 7fcceec5d740 1 --2- 172.21.15.32:0/3050804233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fc2390 0x55e0a73a0400 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.228+0000 7fcceec5d740 1 --2- 172.21.15.32:0/3050804233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fba1b0 0x55e0a7386a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.228+0000 7fcceec5d740 1 -- 172.21.15.32:0/3050804233 >> 172.21.15.32:0/3050804233 conn(0x55e0a6fd7460 msgr2=0x55e0a7386310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.229+0000 7fcceec5d740 1 -- 172.21.15.32:0/3050804233 shutdown_connections 2024-08-31T07:35:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.229+0000 7fcceec5d740 1 -- 172.21.15.32:0/3050804233 wait complete. 2024-08-31T07:35:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.229+0000 7fcceec5d740 1 -- start start 2024-08-31T07:35:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.230+0000 7fcceec5d740 1 Processor -- start 2024-08-31T07:35:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.231+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fba1b0 0x55e0a739ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.231+0000 7fcceec5d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73915f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.232+0000 7fcce5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fba1b0 0x55e0a739ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.232+0000 7fcce5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fba1b0 0x55e0a739ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:47672/0 (socket says 172.21.15.32:47672) 2024-08-31T07:35:05.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.232+0000 7fcce4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.232+0000 7fcce4c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73915f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52480/0 (socket says 172.21.15.32:52480) 2024-08-31T07:35:05.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.232+0000 7fcce5600640 1 -- 172.21.15.32:0/2353333684 learned_addr learned my addr 172.21.15.32:0/2353333684 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:05.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.232+0000 7fcceec5d740 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fe3070 0x55e0a7391d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.232+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b6dd0 con 0x55e0a6fe3070 2024-08-31T07:35:05.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.232+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b6640 con 0x55e0a6fba1b0 2024-08-31T07:35:05.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.232+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b3780 con 0x55e0a6fc2390 2024-08-31T07:35:05.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.232+0000 7fcceec5d740 10 client.0 _ll_register_callbacks cb 0x7fccd4004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.233+0000 7fcce4c00640 1 -- 172.21.15.32:0/2353333684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fba1b0 msgr2=0x55e0a739ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.233+0000 7fcce4c00640 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fba1b0 0x55e0a739ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.233+0000 7fcce4c00640 1 -- 172.21.15.32:0/2353333684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fe3070 msgr2=0x55e0a7391d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.233+0000 7fcce4c00640 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fe3070 0x55e0a7391d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.233+0000 7fcce4c00640 1 -- 172.21.15.32:0/2353333684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a73988a0 con 0x55e0a6fc2390 2024-08-31T07:35:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.234+0000 7fcce4c00640 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73915f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0ab390 tx=0x7fcccc009320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232caa2d85ff2108 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.234+0000 7fccc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:35:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.234+0000 7fccc6a00640 1 -- 172.21.15.32:0/2353333684 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc096070 con 0x55e0a6fc2390 2024-08-31T07:35:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.235+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e0a72fa8a0 con 0x55e0a6fc2390 2024-08-31T07:35:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.234+0000 7fccc6a00640 1 -- 172.21.15.32:0/2353333684 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7fcccc0e8350 con 0x55e0a6fc2390 2024-08-31T07:35:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.235+0000 7fccc6a00640 1 -- 172.21.15.32:0/2353333684 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fcccc0c4070 con 0x55e0a6fc2390 2024-08-31T07:35:05.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.236+0000 7fcceec5d740 10 client.29641 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:05.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.236+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x55e0a7411650 con 0x55e0a6fc2390 2024-08-31T07:35:05.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.237+0000 7fccc6a00640 1 -- 172.21.15.32:0/2353333684 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e0a72fa8a0 con 0x55e0a6fc2390 2024-08-31T07:35:05.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.237+0000 7fccc6a00640 1 -- 172.21.15.32:0/2353333684 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7fcccc0e2970 con 0x55e0a6fc2390 2024-08-31T07:35:05.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.238+0000 7fcceec5d740 10 client.29641 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:05.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.238+0000 7fcceec5d740 10 client.29641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:35:05.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.238+0000 7fcceec5d740 20 client.29641 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:05.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.238+0000 7fcceec5d740 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e0a6f76da0 0x55e0a7413880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:05.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.238+0000 7fcceec5d740 4 client.29641 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:35:05.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.238+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e0a7271160 con 0x55e0a6f76da0 2024-08-31T07:35:05.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.239+0000 7fcce5600640 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e0a6f76da0 0x55e0a7413880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:05.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.247+0000 7fcce5600640 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e0a6f76da0 0x55e0a7413880 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48cbf24567ef2b9 server_cookie=1dacab3cef8a16b9 in_seq=0 out_seq=0 2024-08-31T07:35:05.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.247+0000 7fccc6a00640 10 client.29641 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:05.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.247+0000 7fccc6a00640 1 -- 172.21.15.32:0/2353333684 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6906 (crc 0 0 0) 0x55e0a7271160 con 0x55e0a6f76da0 2024-08-31T07:35:05.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.247+0000 7fccc6a00640 10 client.29641 handle_command_reply: tid=0 2024-08-31T07:35:05.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.248+0000 7fcceec5d740 1 client.29641 shutdown 2024-08-31T07:35:05.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.249+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 msgr2=0x55e0a73915f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:05.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.249+0000 7fcceec5d740 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73915f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0ab390 tx=0x7fcccc009320 comp rx=0 tx=0).stop 2024-08-31T07:35:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.249+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 shutdown_connections 2024-08-31T07:35:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.249+0000 7fcceec5d740 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55e0a6f76da0 0x55e0a7413880 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.249+0000 7fcceec5d740 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0a6fe3070 0x55e0a7391d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.249+0000 7fcceec5d740 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e0a6fc2390 0x55e0a73915f0 secure :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0ab390 tx=0x7fcccc009320 comp rx=0 tx=0).stop 2024-08-31T07:35:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.249+0000 7fcceec5d740 1 --2- 172.21.15.32:0/2353333684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0a6fba1b0 0x55e0a739ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.249+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 >> 172.21.15.32:0/2353333684 conn(0x55e0a6fd7460 msgr2=0x55e0a7389a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.249+0000 7fccc6a00640 0 client.29641 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.250+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 shutdown_connections 2024-08-31T07:35:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.250+0000 7fcceec5d740 1 -- 172.21.15.32:0/2353333684 wait complete. 2024-08-31T07:35:05.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.250+0000 7fcceec5d740 20 client.29641 trim_cache size 0 max 16384 2024-08-31T07:35:05.255 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:35:05.255 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:35:05.255 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:35:05.255 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "d83ecd52-a6c2-49e8-bdaa-039848b73de4": { 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:34:26.167734+0000", 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:32:41.165270+0000", 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.256 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:35:05.257 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.258 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.259 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.260 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.261 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.262 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.263 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:35:05.264 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.265 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.267 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.268 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:05.269 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 -- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccc80b4ca0 msgr2=0x7fccc80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccc80b4ca0 0x7fccc80b71b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fccd8005fd0 tx=0x7fccd80096a0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 -- 172.21.15.32:0/2711935085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce0077120 msgr2=0x7fcce01a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce0077120 0x7fcce01a0ca0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00b470 tx=0x7fcccc00b930 comp rx=0 tx=0).stop 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 -- 172.21.15.32:0/2711935085 shutdown_connections 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fccc80b4ca0 0x7fccc80b71b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcce0114020 0x7fcce01a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce0077120 0x7fcce01a0ca0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 --2- 172.21.15.32:0/2711935085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcce0075b70 0x7fcce01a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 -- 172.21.15.32:0/2711935085 >> 172.21.15.32:0/2711935085 conn(0x7fcce00ff1e0 msgr2=0x7fcce010ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.252+0000 7fcce6a00640 1 -- 172.21.15.32:0/2711935085 shutdown_connections 2024-08-31T07:35:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:05.253+0000 7fcce6a00640 1 -- 172.21.15.32:0/2711935085 wait complete. 2024-08-31T07:35:05.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157348 14.27 MB/sec execute 405 sec latency 14612.658 ms 2024-08-31T07:35:06.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157348 14.24 MB/sec execute 406 sec latency 15612.714 ms 2024-08-31T07:35:06.546 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:35:06.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:35:06.551 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:06.600 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:06 smithi032 ceph-mon[17252]: pgmap v1070: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 406 op/s 2024-08-31T07:35:06.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:06 smithi084 ceph-mon[22387]: pgmap v1070: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 406 op/s 2024-08-31T07:35:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:06 smithi073 ceph-mon[21995]: pgmap v1070: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 406 op/s 2024-08-31T07:35:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 -- 172.21.15.32:0/1414876515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa418077120 msgr2=0x7fa418075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 --2- 172.21.15.32:0/1414876515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa418077120 0x7fa418075550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa408009900 tx=0x7fa40802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 -- 172.21.15.32:0/1414876515 shutdown_connections 2024-08-31T07:35:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 --2- 172.21.15.32:0/1414876515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4181141f0 0x7fa4181166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 --2- 172.21.15.32:0/1414876515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa418075b70 0x7fa418075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 --2- 172.21.15.32:0/1414876515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa418077120 0x7fa418075550 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 -- 172.21.15.32:0/1414876515 >> 172.21.15.32:0/1414876515 conn(0x7fa4180ff370 msgr2=0x7fa4181017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 -- 172.21.15.32:0/1414876515 shutdown_connections 2024-08-31T07:35:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 -- 172.21.15.32:0/1414876515 wait complete. 2024-08-31T07:35:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 Processor -- start 2024-08-31T07:35:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 -- start start 2024-08-31T07:35:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa418075b70 0x7fa4181a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa418077120 0x7fa4181a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4181141f0 0x7fa4181a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa418119e30 con 0x7fa4181141f0 2024-08-31T07:35:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa418119cb0 con 0x7fa418077120 2024-08-31T07:35:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa418119fb0 con 0x7fa418075b70 2024-08-31T07:35:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa415e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa418075b70 0x7fa4181a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa415e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa418075b70 0x7fa4181a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44182/0 (socket says 172.21.15.32:44182) 2024-08-31T07:35:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa415e00640 1 -- 172.21.15.32:0/237024494 learned_addr learned my addr 172.21.15.32:0/237024494 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa415400640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa418077120 0x7fa4181a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa416800640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4181141f0 0x7fa4181a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa415e00640 1 -- 172.21.15.32:0/237024494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa418077120 msgr2=0x7fa4181a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa415e00640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa418077120 0x7fa4181a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa415e00640 1 -- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4181141f0 msgr2=0x7fa4181a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa415e00640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4181141f0 0x7fa4181a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa415e00640 1 -- 172.21.15.32:0/237024494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4181a6750 con 0x7fa418075b70 2024-08-31T07:35:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa416800640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4181141f0 0x7fa4181a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa415e00640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa418075b70 0x7fa4181a07e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa408005940 tx=0x7fa4080040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa40ea00640 1 -- 172.21.15.32:0/237024494 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40803f030 con 0x7fa418075b70 2024-08-31T07:35:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa40ea00640 1 -- 172.21.15.32:0/237024494 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4080044c0 con 0x7fa418075b70 2024-08-31T07:35:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa41cc00640 1 -- 172.21.15.32:0/237024494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4181a69e0 con 0x7fa418075b70 2024-08-31T07:35:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.382+0000 7fa40ea00640 1 -- 172.21.15.32:0/237024494 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408035070 con 0x7fa418075b70 2024-08-31T07:35:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.387+0000 7fa41cc00640 1 -- 172.21.15.32:0/237024494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181ad1a0 con 0x7fa418075b70 2024-08-31T07:35:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.387+0000 7fa40ea00640 1 -- 172.21.15.32:0/237024494 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa408042030 con 0x7fa418075b70 2024-08-31T07:35:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.387+0000 7fa4239f8740 1 Processor -- start 2024-08-31T07:35:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa4239f8740 1 -- start start 2024-08-31T07:35:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080e2840 0x558908489110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 0x558908489650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908489b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa4239f8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5589083fe530 con 0x5589080b9980 2024-08-31T07:35:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa4239f8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5589083edac0 con 0x5589080e2840 2024-08-31T07:35:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa4239f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5589083f1a70 con 0x5589080c1b60 2024-08-31T07:35:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa415400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908489b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa416800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080e2840 0x558908489110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa415400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908489b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44198/0 (socket says 172.21.15.32:44198) 2024-08-31T07:35:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa415400640 1 -- 172.21.15.32:0/2894953201 learned_addr learned my addr 172.21.15.32:0/2894953201 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.392+0000 7fa415e00640 1 --2- 172.21.15.32:0/2894953201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 0x558908489650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa40ea00640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3fc0b4ca0 0x7fa3fc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 -- 172.21.15.32:0/2894953201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080e2840 msgr2=0x558908489110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 --2- 172.21.15.32:0/2894953201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080e2840 0x558908489110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 -- 172.21.15.32:0/2894953201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 msgr2=0x558908489650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 --2- 172.21.15.32:0/2894953201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 0x558908489650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 -- 172.21.15.32:0/2894953201 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589083a0880 con 0x5589080c1b60 2024-08-31T07:35:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa40ea00640 1 -- 172.21.15.32:0/237024494 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7fa4081055b0 con 0x7fa418075b70 2024-08-31T07:35:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 --2- 172.21.15.32:0/2894953201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908489b90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa404009c60 tx=0x7fa404011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc9899cd9ad3ae31 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa416800640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3fc0b4ca0 0x7fa3fc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa403000640 1 -- 172.21.15.32:0/2894953201 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404004230 con 0x5589080c1b60 2024-08-31T07:35:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa403000640 1 -- 172.21.15.32:0/2894953201 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa404009e10 con 0x5589080c1b60 2024-08-31T07:35:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa403000640 1 -- 172.21.15.32:0/2894953201 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404019800 con 0x5589080c1b60 2024-08-31T07:35:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 -- 172.21.15.32:0/2894953201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 msgr2=0x558908489b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 --2- 172.21.15.32:0/2894953201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908489b90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa404009c60 tx=0x7fa404011810 comp rx=0 tx=0).stop 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa416800640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3fc0b4ca0 0x7fa3fc0b71b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa410005fd0 tx=0x7fa410004a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 -- 172.21.15.32:0/2894953201 shutdown_connections 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 --2- 172.21.15.32:0/2894953201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908489b90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 --2- 172.21.15.32:0/2894953201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 0x558908489650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 --2- 172.21.15.32:0/2894953201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080e2840 0x558908489110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 -- 172.21.15.32:0/2894953201 >> 172.21.15.32:0/2894953201 conn(0x5589080d6990 msgr2=0x558908487000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 -- 172.21.15.32:0/2894953201 shutdown_connections 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 -- 172.21.15.32:0/2894953201 wait complete. 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 -- start start 2024-08-31T07:35:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 Processor -- start 2024-08-31T07:35:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080b9980 0x558908494cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x5589084951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 0x5589084a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5589082507a0 con 0x5589080e2840 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5589083747f0 con 0x5589080c1b60 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5589084978f0 con 0x5589080b9980 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa4239f8740 10 client.0 _ll_register_callbacks cb 0x558908399e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x5589084951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080b9980 0x558908494cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x5589084951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59708/0 (socket says 172.21.15.32:59708) 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080b9980 0x558908494cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44204/0 (socket says 172.21.15.32:44204) 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 -- 172.21.15.32:0/1294231012 learned_addr learned my addr 172.21.15.32:0/1294231012 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 -- 172.21.15.32:0/1294231012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080b9980 msgr2=0x558908494cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080b9980 0x558908494cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 -- 172.21.15.32:0/1294231012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 msgr2=0x5589084a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 0x5589084a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.394+0000 7fa415400640 1 -- 172.21.15.32:0/1294231012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890815cb90 con 0x5589080c1b60 2024-08-31T07:35:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa415400640 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x5589084951f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa404019420 tx=0x7fa404004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f51d15003c2b2135 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa3f7400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:35:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa3f7400640 1 -- 172.21.15.32:0/1294231012 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404028020 con 0x5589080c1b60 2024-08-31T07:35:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa3f7400640 1 -- 172.21.15.32:0/1294231012 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7fa4040037f0 con 0x5589080c1b60 2024-08-31T07:35:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 -- 172.21.15.32:0/1294231012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa404009e10 con 0x5589080c1b60 2024-08-31T07:35:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa3f7400640 1 -- 172.21.15.32:0/1294231012 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa404033050 con 0x5589080c1b60 2024-08-31T07:35:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 10 client.29651 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 -- 172.21.15.32:0/1294231012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x5589084a3880 con 0x5589080c1b60 2024-08-31T07:35:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa3f7400640 1 -- 172.21.15.32:0/1294231012 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa404009e10 con 0x5589080c1b60 2024-08-31T07:35:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa3f7400640 1 -- 172.21.15.32:0/1294231012 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7fa404018070 con 0x5589080c1b60 2024-08-31T07:35:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 10 client.29651 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 10 client.29651 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:35:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 20 client.29651 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589080a11c0 0x5589084a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 4 client.29651 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 -- 172.21.15.32:0/1294231012 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558908400480 con 0x5589080a11c0 2024-08-31T07:35:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa415e00640 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589080a11c0 0x5589084a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa415e00640 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589080a11c0 0x5589084a6c90 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396e361f7f8e81d0 server_cookie=79cd0e7b23b12762 in_seq=0 out_seq=0 2024-08-31T07:35:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa3f7400640 10 client.29651 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa3f7400640 1 -- 172.21.15.32:0/1294231012 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558908400480 con 0x5589080a11c0 2024-08-31T07:35:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa3f7400640 10 client.29651 handle_command_reply: tid=0 2024-08-31T07:35:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 client.29651 shutdown 2024-08-31T07:35:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 -- 172.21.15.32:0/1294231012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 msgr2=0x5589084951f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x5589084951f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa404019420 tx=0x7fa404004610 comp rx=0 tx=0).stop 2024-08-31T07:35:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 -- 172.21.15.32:0/1294231012 shutdown_connections 2024-08-31T07:35:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5589080a11c0 0x5589084a6c90 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa3f7400640 0 client.29651 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 0x5589084a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x5589084951f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 --2- 172.21.15.32:0/1294231012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080b9980 0x558908494cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.399+0000 7fa4239f8740 1 -- 172.21.15.32:0/1294231012 >> 172.21.15.32:0/1294231012 conn(0x5589080d6990 msgr2=0x55890848c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.404+0000 7fa4239f8740 1 -- 172.21.15.32:0/1294231012 shutdown_connections 2024-08-31T07:35:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.404+0000 7fa4239f8740 1 -- 172.21.15.32:0/1294231012 wait complete. 2024-08-31T07:35:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.404+0000 7fa4239f8740 20 client.29651 trim_cache size 0 max 16384 2024-08-31T07:35:07.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157415 14.21 MB/sec execute 407 sec latency 16612.763 ms 2024-08-31T07:35:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 Processor -- start 2024-08-31T07:35:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 -- start start 2024-08-31T07:35:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 0x558908496990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x55890848a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080e2840 0x5589084a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b5960 con 0x5589080b9980 2024-08-31T07:35:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b7680 con 0x5589080c1b60 2024-08-31T07:35:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b60f0 con 0x5589080e2840 2024-08-31T07:35:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa415e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 0x558908496990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa416800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080e2840 0x5589084a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa415e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 0x558908496990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56334/0 (socket says 172.21.15.32:56334) 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa415e00640 1 -- 172.21.15.32:0/608665552 learned_addr learned my addr 172.21.15.32:0/608665552 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa415400640 1 --2- 172.21.15.32:0/608665552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x55890848a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa415400640 1 -- 172.21.15.32:0/608665552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080e2840 msgr2=0x5589084a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa415400640 1 --2- 172.21.15.32:0/608665552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080e2840 0x5589084a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa415400640 1 -- 172.21.15.32:0/608665552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 msgr2=0x558908496990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa415400640 1 --2- 172.21.15.32:0/608665552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 0x558908496990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa415400640 1 -- 172.21.15.32:0/608665552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890848f750 con 0x5589080c1b60 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa415400640 1 --2- 172.21.15.32:0/608665552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x55890848a850 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa404004940 tx=0x7fa404012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e6f05961bf60f10 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa402600640 1 -- 172.21.15.32:0/608665552 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404028040 con 0x5589080c1b60 2024-08-31T07:35:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa402600640 1 -- 172.21.15.32:0/608665552 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa404004680 con 0x5589080c1b60 2024-08-31T07:35:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa402600640 1 -- 172.21.15.32:0/608665552 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40401f040 con 0x5589080c1b60 2024-08-31T07:35:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 -- 172.21.15.32:0/608665552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 msgr2=0x55890848a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 --2- 172.21.15.32:0/608665552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x55890848a850 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa404004940 tx=0x7fa404012f90 comp rx=0 tx=0).stop 2024-08-31T07:35:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 -- 172.21.15.32:0/608665552 shutdown_connections 2024-08-31T07:35:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 --2- 172.21.15.32:0/608665552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080e2840 0x5589084a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 --2- 172.21.15.32:0/608665552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080c1b60 0x55890848a850 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 --2- 172.21.15.32:0/608665552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080b9980 0x558908496990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 -- 172.21.15.32:0/608665552 >> 172.21.15.32:0/608665552 conn(0x5589080d6990 msgr2=0x55890848a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 -- 172.21.15.32:0/608665552 shutdown_connections 2024-08-31T07:35:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.423+0000 7fa4239f8740 1 -- 172.21.15.32:0/608665552 wait complete. 2024-08-31T07:35:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.428+0000 7fa4239f8740 1 -- start start 2024-08-31T07:35:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.428+0000 7fa4239f8740 1 Processor -- start 2024-08-31T07:35:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.428+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080b9980 0x558908499500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.428+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908499ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.428+0000 7fa4239f8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 0x55890850a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.428+0000 7fa4239f8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b6fb0 con 0x5589080e2840 2024-08-31T07:35:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.428+0000 7fa4239f8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b6820 con 0x5589080b9980 2024-08-31T07:35:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.428+0000 7fa4239f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181b3960 con 0x5589080c1b60 2024-08-31T07:35:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.428+0000 7fa4239f8740 10 client.0 _ll_register_callbacks cb 0x7fa404009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa415400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908499ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa416800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 0x55890850a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa415400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908499ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44220/0 (socket says 172.21.15.32:44220) 2024-08-31T07:35:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa415400640 1 -- 172.21.15.32:0/4282825045 learned_addr learned my addr 172.21.15.32:0/4282825045 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa416800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 0x55890850a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56340/0 (socket says 172.21.15.32:56340) 2024-08-31T07:35:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa416800640 1 -- 172.21.15.32:0/4282825045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 msgr2=0x558908499ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa416800640 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908499ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa416800640 1 -- 172.21.15.32:0/4282825045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080b9980 msgr2=0x558908499500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa416800640 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080b9980 0x558908499500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa416800640 1 -- 172.21.15.32:0/4282825045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890850ac60 con 0x5589080e2840 2024-08-31T07:35:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa416800640 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 0x55890850a5f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa410013fd0 tx=0x7fa410009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24d13e13b778be4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa3f7400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:35:07.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa3f7400640 1 -- 172.21.15.32:0/4282825045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410011910 con 0x5589080e2840 2024-08-31T07:35:07.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa3f7400640 1 -- 172.21.15.32:0/4282825045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7fa410011ab0 con 0x5589080e2840 2024-08-31T07:35:07.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa4239f8740 1 -- 172.21.15.32:0/4282825045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa404004680 con 0x5589080e2840 2024-08-31T07:35:07.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa3f7400640 1 -- 172.21.15.32:0/4282825045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa410011cb0 con 0x5589080e2840 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa4239f8740 10 client.19605 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa4239f8740 1 -- 172.21.15.32:0/4282825045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x558908512010 con 0x5589080e2840 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa3f7400640 1 -- 172.21.15.32:0/4282825045 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa404004680 con 0x5589080e2840 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa3f7400640 1 -- 172.21.15.32:0/4282825045 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7fa410026020 con 0x5589080e2840 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa4239f8740 10 client.19605 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa4239f8740 10 client.19605 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa4239f8740 20 client.19605 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa4239f8740 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558908076570 0x558908514870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa4239f8740 4 client.19605 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa4239f8740 1 -- 172.21.15.32:0/4282825045 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5589083fbf20 con 0x558908076570 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa415e00640 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558908076570 0x558908514870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa415e00640 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558908076570 0x558908514870 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5788258c7893207f server_cookie=fca8834a43e8e856 in_seq=0 out_seq=0 2024-08-31T07:35:07.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:07.430+0000 7fa3f7400640 10 client.19605 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 -- 172.21.15.32:0/2600840094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e8c105950 msgr2=0x7f1e8c105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 --2- 172.21.15.32:0/2600840094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e8c105950 0x7f1e8c105d60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009860 tx=0x7f1e7c02a340 comp rx=0 tx=0).stop 2024-08-31T07:35:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 -- 172.21.15.32:0/2600840094 shutdown_connections 2024-08-31T07:35:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 --2- 172.21.15.32:0/2600840094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c10bbe0 0x7f1e8c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:08.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 --2- 172.21.15.32:0/2600840094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e8c10b290 0x7f1e8c10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:08.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 --2- 172.21.15.32:0/2600840094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e8c105950 0x7f1e8c105d60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:08.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 -- 172.21.15.32:0/2600840094 >> 172.21.15.32:0/2600840094 conn(0x7f1e8c101420 msgr2=0x7f1e8c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:08.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 -- 172.21.15.32:0/2600840094 shutdown_connections 2024-08-31T07:35:08.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 -- 172.21.15.32:0/2600840094 wait complete. 2024-08-31T07:35:08.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 Processor -- start 2024-08-31T07:35:08.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 -- start start 2024-08-31T07:35:08.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e8c105950 0x7f1e8c07b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:08.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e8c10b290 0x7f1e8c0776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:08.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c10bbe0 0x7f1e8c077de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:08.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1158a0 con 0x7f1e8c10bbe0 2024-08-31T07:35:08.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c115720 con 0x7f1e8c10b290 2024-08-31T07:35:08.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.105+0000 7f1e8b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c115a20 con 0x7f1e8c105950 2024-08-31T07:35:08.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e89400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e8c10b290 0x7f1e8c0776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:08.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e89400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e8c10b290 0x7f1e8c0776c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59738/0 (socket says 172.21.15.32:59738) 2024-08-31T07:35:08.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e89400640 1 -- 172.21.15.32:0/1440958475 learned_addr learned my addr 172.21.15.32:0/1440958475 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e89400640 1 -- 172.21.15.32:0/1440958475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e8c105950 msgr2=0x7f1e8c07b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:35:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e89400640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e8c105950 0x7f1e8c07b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e89400640 1 -- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c10bbe0 msgr2=0x7f1e8c077de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T07:35:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e8a800640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c10bbe0 0x7f1e8c077de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e89400640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c10bbe0 0x7f1e8c077de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e89400640 1 -- 172.21.15.32:0/1440958475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c1ac800 con 0x7f1e8c10b290 2024-08-31T07:35:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e8a800640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c10bbe0 0x7f1e8c077de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:35:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.110+0000 7f1e89400640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e8c10b290 0x7f1e8c0776c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e9c0 tx=0x7f1e7400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.112+0000 7f1e82a00640 1 -- 172.21.15.32:0/1440958475 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400ccc0 con 0x7f1e8c10b290 2024-08-31T07:35:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.112+0000 7f1e82a00640 1 -- 172.21.15.32:0/1440958475 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e740108e0 con 0x7f1e8c10b290 2024-08-31T07:35:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.112+0000 7f1e82a00640 1 -- 172.21.15.32:0/1440958475 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e740187e0 con 0x7f1e8c10b290 2024-08-31T07:35:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.112+0000 7f1e8b200640 1 -- 172.21.15.32:0/1440958475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c1aca30 con 0x7f1e8c10b290 2024-08-31T07:35:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.112+0000 7f1e8b200640 1 -- 172.21.15.32:0/1440958475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c1acf20 con 0x7f1e8c10b290 2024-08-31T07:35:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.112+0000 7f1e8b200640 1 -- 172.21.15.32:0/1440958475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8c105d60 con 0x7f1e8c10b290 2024-08-31T07:35:08.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.112+0000 7f1e82a00640 1 -- 172.21.15.32:0/1440958475 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1e74010430 con 0x7f1e8c10b290 2024-08-31T07:35:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.118+0000 7f1e82a00640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e700b4d20 0x7f1e700b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.118+0000 7f1e82a00640 1 -- 172.21.15.32:0/1440958475 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f1e740e58c0 con 0x7f1e8c10b290 2024-08-31T07:35:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.118+0000 7f1e89e00640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e700b4d20 0x7f1e700b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.118+0000 7f1e89e00640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e700b4d20 0x7f1e700b7230 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c02a860 tx=0x7f1e7c000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:08.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.118+0000 7f1e82a00640 1 -- 172.21.15.32:0/1440958475 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1e740ac6a0 con 0x7f1e8c10b290 2024-08-31T07:35:08.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.407+0000 7f1e8b200640 1 -- 172.21.15.32:0/1440958475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e8c0789c0 con 0x7f1e8c10b290 2024-08-31T07:35:08.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.409+0000 7f1e82a00640 1 -- 172.21.15.32:0/1440958475 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+11977 (secure 0 0 0) 0x7f1e740abfa0 con 0x7f1e8c10b290 2024-08-31T07:35:08.410 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:35:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"btime":"2024-08-31T07:35:01:444248+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":86,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:35:00.978669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:35:08.411 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 86 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 -- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e700b4d20 msgr2=0x7f1e700b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e700b4d20 0x7f1e700b7230 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c02a860 tx=0x7f1e7c000bc0 comp rx=0 tx=0).stop 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 -- 172.21.15.32:0/1440958475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e8c10b290 msgr2=0x7f1e8c0776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e8c10b290 0x7f1e8c0776c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e9c0 tx=0x7f1e7400ee80 comp rx=0 tx=0).stop 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 -- 172.21.15.32:0/1440958475 shutdown_connections 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e700b4d20 0x7f1e700b7230 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c10bbe0 0x7f1e8c077de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e8c10b290 0x7f1e8c0776c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 --2- 172.21.15.32:0/1440958475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e8c105950 0x7f1e8c07b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 -- 172.21.15.32:0/1440958475 >> 172.21.15.32:0/1440958475 conn(0x7f1e8c101420 msgr2=0x7f1e8c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 -- 172.21.15.32:0/1440958475 shutdown_connections 2024-08-31T07:35:08.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:08.411+0000 7f1e8b200640 1 -- 172.21.15.32:0/1440958475 wait complete. 2024-08-31T07:35:08.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157416 14.18 MB/sec execute 408 sec latency 17612.838 ms 2024-08-31T07:35:08.502 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:08 smithi032 ceph-mon[17252]: pgmap v1071: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 313 op/s 2024-08-31T07:35:08.503 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:08 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1440958475' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:08 smithi084 ceph-mon[22387]: pgmap v1071: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 313 op/s 2024-08-31T07:35:08.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:08 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1440958475' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:08 smithi073 ceph-mon[21995]: pgmap v1071: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 313 op/s 2024-08-31T07:35:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:08 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1440958475' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:09.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:35:09.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157416 14.14 MB/sec execute 409 sec latency 18612.903 ms 2024-08-31T07:35:09.690 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- 172.21.15.32:0/1707611975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98103750 msgr2=0x7f1d98103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 --2- 172.21.15.32:0/1707611975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98103750 0x7f1d98103b60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009900 tx=0x7f1d8802a420 comp rx=0 tx=0).stop 2024-08-31T07:35:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- 172.21.15.32:0/1707611975 shutdown_connections 2024-08-31T07:35:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 --2- 172.21.15.32:0/1707611975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d981099e0 0x7f1d98110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 --2- 172.21.15.32:0/1707611975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d98109090 0x7f1d981094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 --2- 172.21.15.32:0/1707611975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98103750 0x7f1d98103b60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- 172.21.15.32:0/1707611975 >> 172.21.15.32:0/1707611975 conn(0x7f1d980ff1a0 msgr2=0x7f1d981015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- 172.21.15.32:0/1707611975 shutdown_connections 2024-08-31T07:35:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- 172.21.15.32:0/1707611975 wait complete. 2024-08-31T07:35:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 Processor -- start 2024-08-31T07:35:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- start start 2024-08-31T07:35:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98103750 0x7f1d9819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d98109090 0x7f1d9819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d981099e0 0x7f1d981a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981136e0 con 0x7f1d98103750 2024-08-31T07:35:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98113560 con 0x7f1d981099e0 2024-08-31T07:35:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98113860 con 0x7f1d98109090 2024-08-31T07:35:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d97400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98103750 0x7f1d9819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d97400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98103750 0x7f1d9819c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56374/0 (socket says 172.21.15.32:56374) 2024-08-31T07:35:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d97400640 1 -- 172.21.15.32:0/2143014315 learned_addr learned my addr 172.21.15.32:0/2143014315 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d98109090 0x7f1d9819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d8fe00640 1 -- 172.21.15.32:0/2143014315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d981099e0 msgr2=0x7f1d981a1b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T07:35:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d981099e0 0x7f1d981a1b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d8fe00640 1 -- 172.21.15.32:0/2143014315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98103750 msgr2=0x7f1d9819c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98103750 0x7f1d9819c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d8fe00640 1 -- 172.21.15.32:0/2143014315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d981a2260 con 0x7f1d98109090 2024-08-31T07:35:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d97400640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98103750 0x7f1d9819c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d98109090 0x7f1d9819c890 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1d9806b6d0 tx=0x7f1d8000b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d94c00640 1 -- 172.21.15.32:0/2143014315 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80015020 con 0x7f1d98109090 2024-08-31T07:35:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d94c00640 1 -- 172.21.15.32:0/2143014315 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d80011ce0 con 0x7f1d98109090 2024-08-31T07:35:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d94c00640 1 -- 172.21.15.32:0/2143014315 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80010da0 con 0x7f1d98109090 2024-08-31T07:35:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- 172.21.15.32:0/2143014315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d981a2550 con 0x7f1d98109090 2024-08-31T07:35:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.408+0000 7f1d9ca00640 1 -- 172.21.15.32:0/2143014315 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d981a8cb0 con 0x7f1d98109090 2024-08-31T07:35:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1da4dc8740 1 Processor -- start 2024-08-31T07:35:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1da4dc8740 1 -- start start 2024-08-31T07:35:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d94c00640 1 -- 172.21.15.32:0/2143014315 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1d80018020 con 0x7f1d98109090 2024-08-31T07:35:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bddb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194bde0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194bde5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558194b547b0 con 0x558194817390 2024-08-31T07:35:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558194b43d40 con 0x55819480f1b0 2024-08-31T07:35:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558194b47cf0 con 0x558194838070 2024-08-31T07:35:10.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bddb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bddb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44258/0 (socket says 172.21.15.32:44258) 2024-08-31T07:35:10.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d97e00640 1 -- 172.21.15.32:0/3263391482 learned_addr learned my addr 172.21.15.32:0/3263391482 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:10.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/3263391482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194bde5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d97e00640 1 -- 172.21.15.32:0/3263391482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 msgr2=0x558194bde0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:35:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d97e00640 1 --2- 172.21.15.32:0/3263391482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194bde0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d97e00640 1 -- 172.21.15.32:0/3263391482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 msgr2=0x558194bde5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d97e00640 1 --2- 172.21.15.32:0/3263391482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194bde5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d97e00640 1 -- 172.21.15.32:0/3263391482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581948a3650 con 0x558194838070 2024-08-31T07:35:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d97e00640 1 --2- 172.21.15.32:0/3263391482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bddb60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009e30 tx=0x7f1d9000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a95832b717a8b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d87e00640 1 -- 172.21.15.32:0/3263391482 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90004d20 con 0x558194838070 2024-08-31T07:35:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d87e00640 1 -- 172.21.15.32:0/3263391482 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9000ee70 con 0x558194838070 2024-08-31T07:35:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.414+0000 7f1d87e00640 1 -- 172.21.15.32:0/3263391482 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90010760 con 0x558194838070 2024-08-31T07:35:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1d94c00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d7c0b4d20 0x7f1d7c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1d94c00640 1 -- 172.21.15.32:0/2143014315 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f1d800e4c80 con 0x7f1d98109090 2024-08-31T07:35:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1d97400640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d7c0b4d20 0x7f1d7c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1da4dc8740 1 -- 172.21.15.32:0/3263391482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 msgr2=0x558194bddb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/3263391482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bddb60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009e30 tx=0x7f1d9000e720 comp rx=0 tx=0).stop 2024-08-31T07:35:10.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1da4dc8740 1 -- 172.21.15.32:0/3263391482 shutdown_connections 2024-08-31T07:35:10.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/3263391482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194bde5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/3263391482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194bde0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/3263391482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bddb60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1da4dc8740 1 -- 172.21.15.32:0/3263391482 >> 172.21.15.32:0/3263391482 conn(0x55819482c460 msgr2=0x558194bdba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:10.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.419+0000 7f1da4dc8740 1 -- 172.21.15.32:0/3263391482 shutdown_connections 2024-08-31T07:35:10.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 -- 172.21.15.32:0/3263391482 wait complete. 2024-08-31T07:35:10.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97400640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d7c0b4d20 0x7f1d7c0b7230 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1d88031440 tx=0x7f1d880058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:10.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157416 14.11 MB/sec execute 410 sec latency 19612.952 ms 2024-08-31T07:35:10.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 -- start start 2024-08-31T07:35:10.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 Processor -- start 2024-08-31T07:35:10.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194be9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194be9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bf5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5581949a6700 con 0x558194817390 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558194acacf0 con 0x55819480f1b0 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558194bec340 con 0x558194838070 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 10 client.0 _ll_register_callbacks cb 0x558194af0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194be9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bf5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194be9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59764/0 (socket says 172.21.15.32:59764) 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97400640 1 -- 172.21.15.32:0/1918730332 learned_addr learned my addr 172.21.15.32:0/1918730332 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194be9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97e00640 1 -- 172.21.15.32:0/1918730332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 msgr2=0x558194be9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97e00640 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194be9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97e00640 1 -- 172.21.15.32:0/1918730332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 msgr2=0x558194be9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97e00640 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194be9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97e00640 1 -- 172.21.15.32:0/1918730332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558194b0b8b0 con 0x558194838070 2024-08-31T07:35:10.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97e00640 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bf5630 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009e30 tx=0x7f1d900052d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bceca161b30aea7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d84c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:35:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d84c00640 1 -- 172.21.15.32:0/1918730332 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9001d040 con 0x558194838070 2024-08-31T07:35:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 -- 172.21.15.32:0/1918730332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9000ee70 con 0x558194838070 2024-08-31T07:35:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d84c00640 1 -- 172.21.15.32:0/1918730332 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f1d900056c0 con 0x558194838070 2024-08-31T07:35:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d84c00640 1 -- 172.21.15.32:0/1918730332 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f1d90003970 con 0x558194838070 2024-08-31T07:35:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 10 client.29671 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 -- 172.21.15.32:0/1918730332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x558194bf83a0 con 0x558194838070 2024-08-31T07:35:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d84c00640 1 -- 172.21.15.32:0/1918730332 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9000ee70 con 0x558194838070 2024-08-31T07:35:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d84c00640 1 -- 172.21.15.32:0/1918730332 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f1d90004680 con 0x558194838070 2024-08-31T07:35:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 10 client.29671 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 10 client.29671 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:35:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 20 client.29671 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5581947f69f0 0x558194bfb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 4 client.29671 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1da4dc8740 1 -- 172.21.15.32:0/1918730332 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558194b56700 con 0x5581947f69f0 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97400640 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5581947f69f0 0x558194bfb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d97400640 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5581947f69f0 0x558194bfb700 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee3b04809060d525 server_cookie=c018fba4d0014095 in_seq=0 out_seq=0 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.421+0000 7f1d84c00640 10 client.29671 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1d84c00640 1 -- 172.21.15.32:0/1918730332 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558194b56700 con 0x5581947f69f0 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1d84c00640 10 client.29671 handle_command_reply: tid=0 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 client.29671 shutdown 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 -- 172.21.15.32:0/1918730332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 msgr2=0x558194bf5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bf5630 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009e30 tx=0x7f1d900052d0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 -- 172.21.15.32:0/1918730332 shutdown_connections 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5581947f69f0 0x558194bfb700 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bf5630 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194be9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/1918730332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194be9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 -- 172.21.15.32:0/1918730332 >> 172.21.15.32:0/1918730332 conn(0x55819482c460 msgr2=0x558194be0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:10.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1d84c00640 0 client.29671 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:35:10.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 -- 172.21.15.32:0/1918730332 shutdown_connections 2024-08-31T07:35:10.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 1 -- 172.21.15.32:0/1918730332 wait complete. 2024-08-31T07:35:10.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.426+0000 7f1da4dc8740 20 client.29671 trim_cache size 0 max 16384 2024-08-31T07:35:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1da4dc8740 1 Processor -- start 2024-08-31T07:35:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1da4dc8740 1 -- start start 2024-08-31T07:35:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55819480f1b0 0x558194be1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558194817390 0x558194bf5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194838070 0x558194bf5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b1520 con 0x558194838070 2024-08-31T07:35:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b3240 con 0x558194817390 2024-08-31T07:35:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b1cb0 con 0x55819480f1b0 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d97400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55819480f1b0 0x558194be1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d97400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55819480f1b0 0x558194be1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44282/0 (socket says 172.21.15.32:44282) 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d8fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558194817390 0x558194bf5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d97400640 1 -- 172.21.15.32:0/2491488746 learned_addr learned my addr 172.21.15.32:0/2491488746 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d97e00640 1 --2- 172.21.15.32:0/2491488746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194838070 0x558194bf5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d8fe00640 1 -- 172.21.15.32:0/2491488746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55819480f1b0 msgr2=0x558194be1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/2491488746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55819480f1b0 0x558194be1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d8fe00640 1 -- 172.21.15.32:0/2491488746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194838070 msgr2=0x558194bf5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/2491488746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194838070 0x558194bf5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d8fe00640 1 -- 172.21.15.32:0/2491488746 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558194bfba70 con 0x558194817390 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.438+0000 7f1d97e00640 1 --2- 172.21.15.32:0/2491488746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194838070 0x558194bf5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/2491488746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558194817390 0x558194bf5530 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1d800e23c0 tx=0x7f1d800aacb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3274945bbd9eada6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d87400640 1 -- 172.21.15.32:0/2491488746 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d800aba60 con 0x558194817390 2024-08-31T07:35:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d87400640 1 -- 172.21.15.32:0/2491488746 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d800ab360 con 0x558194817390 2024-08-31T07:35:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d87400640 1 -- 172.21.15.32:0/2491488746 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d800a7780 con 0x558194817390 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2491488746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558194817390 msgr2=0x558194bf5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/2491488746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558194817390 0x558194bf5530 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1d800e23c0 tx=0x7f1d800aacb0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2491488746 shutdown_connections 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/2491488746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194838070 0x558194bf5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/2491488746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558194817390 0x558194bf5530 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/2491488746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55819480f1b0 0x558194be1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2491488746 >> 172.21.15.32:0/2491488746 conn(0x55819482c460 msgr2=0x558194be0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2491488746 shutdown_connections 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2491488746 wait complete. 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- start start 2024-08-31T07:35:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 Processor -- start 2024-08-31T07:35:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194bf17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194bf1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bec550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b2b70 con 0x558194817390 2024-08-31T07:35:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b23e0 con 0x55819480f1b0 2024-08-31T07:35:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981af520 con 0x558194838070 2024-08-31T07:35:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 10 client.0 _ll_register_callbacks cb 0x7f1d90004e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d8fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194bf1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d8fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194bf1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56408/0 (socket says 172.21.15.32:56408) 2024-08-31T07:35:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d8fe00640 1 -- 172.21.15.32:0/2336794625 learned_addr learned my addr 172.21.15.32:0/2336794625 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:10.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d8fe00640 1 -- 172.21.15.32:0/2336794625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 msgr2=0x558194bec550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:10.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bec550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d8fe00640 1 -- 172.21.15.32:0/2336794625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 msgr2=0x558194bf17a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:35:10.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194bf17a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d8fe00640 1 -- 172.21.15.32:0/2336794625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558194becb40 con 0x558194817390 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d97e00640 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d8fe00640 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194bf1ce0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000bd50 tx=0x7f1d800aade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a54ec99b1d9d5d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d84c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d84c00640 1 -- 172.21.15.32:0/2336794625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80096020 con 0x558194817390 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d84c00640 1 -- 172.21.15.32:0/2336794625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8441+0+0 (secure 0 0 0) 0x7f1d80011830 con 0x558194817390 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2336794625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d800ab360 con 0x558194817390 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d84c00640 1 -- 172.21.15.32:0/2336794625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f1d800c4030 con 0x558194817390 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 10 client.19626 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2336794625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x558194c66730 con 0x558194817390 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.443+0000 7f1d84c00640 1 -- 172.21.15.32:0/2336794625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d800ab360 con 0x558194817390 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.448+0000 7f1d84c00640 1 -- 172.21.15.32:0/2336794625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f1d800ec040 con 0x558194817390 2024-08-31T07:35:10.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.448+0000 7f1da4dc8740 10 client.19626 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.448+0000 7f1da4dc8740 10 client.19626 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:35:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.448+0000 7f1da4dc8740 20 client.19626 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.448+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5581947cbda0 0x558194c68f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.448+0000 7f1da4dc8740 4 client.19626 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:35:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.448+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2336794625 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558194b521a0 con 0x5581947cbda0 2024-08-31T07:35:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.468+0000 7f1d97400640 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5581947cbda0 0x558194c68f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:10.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.527+0000 7f1d97400640 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5581947cbda0 0x558194c68f70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6dcfb0910598813 server_cookie=65941889a55f24af in_seq=0 out_seq=0 2024-08-31T07:35:10.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.527+0000 7f1d84c00640 10 client.19626 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:35:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1d84c00640 1 -- 172.21.15.32:0/2336794625 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558194b521a0 con 0x5581947cbda0 2024-08-31T07:35:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1d84c00640 10 client.19626 handle_command_reply: tid=0 2024-08-31T07:35:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 client.19626 shutdown 2024-08-31T07:35:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2336794625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 msgr2=0x558194bf1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194bf1ce0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000bd50 tx=0x7f1d800aade0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2336794625 shutdown_connections 2024-08-31T07:35:10.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5581947cbda0 0x558194c68f70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558194838070 0x558194bec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558194817390 0x558194bf1ce0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 --2- 172.21.15.32:0/2336794625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55819480f1b0 0x558194bf17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1d84c00640 0 client.19626 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:35:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2336794625 >> 172.21.15.32:0/2336794625 conn(0x55819482c460 msgr2=0x558194bdf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2336794625 shutdown_connections 2024-08-31T07:35:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 1 -- 172.21.15.32:0/2336794625 wait complete. 2024-08-31T07:35:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.539+0000 7f1da4dc8740 20 client.19626 trim_cache size 0 max 16384 2024-08-31T07:35:10.542 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:35:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 -- 172.21.15.32:0/2143014315 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d7c0b4d20 msgr2=0x7f1d7c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d7c0b4d20 0x7f1d7c0b7230 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1d88031440 tx=0x7f1d880058d0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 -- 172.21.15.32:0/2143014315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d98109090 msgr2=0x7f1d9819c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d98109090 0x7f1d9819c890 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1d9806b6d0 tx=0x7f1d8000b850 comp rx=0 tx=0).stop 2024-08-31T07:35:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 -- 172.21.15.32:0/2143014315 shutdown_connections 2024-08-31T07:35:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1d7c0b4d20 0x7f1d7c0b7230 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d981099e0 0x7f1d981a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d98109090 0x7f1d9819c890 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 --2- 172.21.15.32:0/2143014315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d98103750 0x7f1d9819c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 -- 172.21.15.32:0/2143014315 >> 172.21.15.32:0/2143014315 conn(0x7f1d980ff1a0 msgr2=0x7f1d981087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 -- 172.21.15.32:0/2143014315 shutdown_connections 2024-08-31T07:35:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:10.540+0000 7f1d9ca00640 1 -- 172.21.15.32:0/2143014315 wait complete. 2024-08-31T07:35:10.661 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:10 smithi032 ceph-mon[17252]: pgmap v1072: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.5 MiB/s wr, 255 op/s 2024-08-31T07:35:10.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:10 smithi084 ceph-mon[22387]: pgmap v1072: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.5 MiB/s wr, 255 op/s 2024-08-31T07:35:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:10 smithi073 ceph-mon[21995]: pgmap v1072: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.5 MiB/s wr, 255 op/s 2024-08-31T07:35:11.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157419 14.08 MB/sec execute 411 sec latency 20613.017 ms 2024-08-31T07:35:11.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:35:12.042 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:12.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157422 14.04 MB/sec execute 412 sec latency 20841.202 ms 2024-08-31T07:35:12.675 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:12 smithi032 ceph-mon[17252]: pgmap v1073: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 243 op/s 2024-08-31T07:35:12.676 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:12 smithi032 ceph-mon[17252]: osdmap e113: 12 total, 12 up, 12 in 2024-08-31T07:35:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:12 smithi084 ceph-mon[22387]: pgmap v1073: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 243 op/s 2024-08-31T07:35:12.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:12 smithi084 ceph-mon[22387]: osdmap e113: 12 total, 12 up, 12 in 2024-08-31T07:35:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:12 smithi073 ceph-mon[21995]: pgmap v1073: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 243 op/s 2024-08-31T07:35:12.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:12 smithi073 ceph-mon[21995]: osdmap e113: 12 total, 12 up, 12 in 2024-08-31T07:35:12.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.934+0000 7f61dca00640 1 -- 172.21.15.32:0/3853909523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d8075ea0 msgr2=0x7f61d810f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:12.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.934+0000 7f61dca00640 1 --2- 172.21.15.32:0/3853909523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d8075ea0 0x7f61d810f770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009900 tx=0x7f61c002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.934+0000 7f61dca00640 1 -- 172.21.15.32:0/3853909523 shutdown_connections 2024-08-31T07:35:12.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.934+0000 7f61dca00640 1 --2- 172.21.15.32:0/3853909523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d810fcb0 0x7f61d81120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.934+0000 7f61dca00640 1 --2- 172.21.15.32:0/3853909523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d8075ea0 0x7f61d810f770 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.934+0000 7f61dca00640 1 --2- 172.21.15.32:0/3853909523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d8075550 0x7f61d8075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.934+0000 7f61dca00640 1 -- 172.21.15.32:0/3853909523 >> 172.21.15.32:0/3853909523 conn(0x7f61d80ff1e0 msgr2=0x7f61d8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:12.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.934+0000 7f61dca00640 1 -- 172.21.15.32:0/3853909523 shutdown_connections 2024-08-31T07:35:12.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.934+0000 7f61dca00640 1 -- 172.21.15.32:0/3853909523 wait complete. 2024-08-31T07:35:12.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.936+0000 7f61dca00640 1 Processor -- start 2024-08-31T07:35:12.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.936+0000 7f61dca00640 1 -- start start 2024-08-31T07:35:12.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61dca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d8075550 0x7f61d819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61dca00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d8075ea0 0x7f61d819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61dca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d810fcb0 0x7f61d81a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61dca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61d8115820 con 0x7f61d810fcb0 2024-08-31T07:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61dca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81156a0 con 0x7f61d8075550 2024-08-31T07:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61dca00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81159a0 con 0x7f61d8075ea0 2024-08-31T07:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61d6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d8075550 0x7f61d819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d8075ea0 0x7f61d819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61d6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d8075550 0x7f61d819c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59800/0 (socket says 172.21.15.32:59800) 2024-08-31T07:35:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61d6800640 1 -- 172.21.15.32:0/3896610217 learned_addr learned my addr 172.21.15.32:0/3896610217 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61d7200640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d810fcb0 0x7f61d81a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61cfe00640 1 -- 172.21.15.32:0/3896610217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d8075550 msgr2=0x7f61d819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61cfe00640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d8075550 0x7f61d819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61cfe00640 1 -- 172.21.15.32:0/3896610217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d810fcb0 msgr2=0x7f61d81a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61cfe00640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d810fcb0 0x7f61d81a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.937+0000 7f61cfe00640 1 -- 172.21.15.32:0/3896610217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d81a21c0 con 0x7f61d8075ea0 2024-08-31T07:35:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.946+0000 7f61cfe00640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d8075ea0 0x7f61d819c7f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f61c002a910 tx=0x7f61c0031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.946+0000 7f61cf400640 1 -- 172.21.15.32:0/3896610217 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c003f070 con 0x7f61d8075ea0 2024-08-31T07:35:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.946+0000 7f61cf400640 1 -- 172.21.15.32:0/3896610217 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c0004490 con 0x7f61d8075ea0 2024-08-31T07:35:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.946+0000 7f61dca00640 1 -- 172.21.15.32:0/3896610217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d81a2450 con 0x7f61d8075ea0 2024-08-31T07:35:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.946+0000 7f61cf400640 1 -- 172.21.15.32:0/3896610217 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0035070 con 0x7f61d8075ea0 2024-08-31T07:35:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.946+0000 7f61dca00640 1 -- 172.21.15.32:0/3896610217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d81a8c10 con 0x7f61d8075ea0 2024-08-31T07:35:12.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.950+0000 7f61e4213740 1 Processor -- start 2024-08-31T07:35:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.950+0000 7f61e4213740 1 -- start start 2024-08-31T07:35:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.950+0000 7f61cf400640 1 -- 172.21.15.32:0/3896610217 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f61c003b460 con 0x7f61d8075ea0 2024-08-31T07:35:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.950+0000 7f61e4213740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236cd06070 0x56236d0abb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.950+0000 7f61e4213740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 0x56236d0ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.950+0000 7f61e4213740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cce5390 0x56236d0ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.950+0000 7f61e4213740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56236d0227b0 con 0x56236cd06070 2024-08-31T07:35:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.950+0000 7f61e4213740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56236d011d40 con 0x56236cce5390 2024-08-31T07:35:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.950+0000 7f61e4213740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56236d015cf0 con 0x56236ccdd1b0 2024-08-31T07:35:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cce5390 0x56236d0ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61d6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 0x56236d0ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cce5390 0x56236d0ac5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59806/0 (socket says 172.21.15.32:59806) 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cfe00640 1 -- 172.21.15.32:0/368078759 learned_addr learned my addr 172.21.15.32:0/368078759 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61d7200640 1 --2- 172.21.15.32:0/368078759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236cd06070 0x56236d0abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cfe00640 1 -- 172.21.15.32:0/368078759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 msgr2=0x56236d0ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cfe00640 1 --2- 172.21.15.32:0/368078759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 0x56236d0ac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cfe00640 1 -- 172.21.15.32:0/368078759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236cd06070 msgr2=0x56236d0abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cfe00640 1 --2- 172.21.15.32:0/368078759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236cd06070 0x56236d0abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cfe00640 1 -- 172.21.15.32:0/368078759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236cd71650 con 0x56236cce5390 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61d6800640 1 --2- 172.21.15.32:0/368078759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 0x56236d0ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cfe00640 1 --2- 172.21.15.32:0/368078759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cce5390 0x56236d0ac5e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f61c0103020 tx=0x7f61c010bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4435461efa496e86 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61c7400640 1 -- 172.21.15.32:0/368078759 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0114070 con 0x56236cce5390 2024-08-31T07:35:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61c7400640 1 -- 172.21.15.32:0/368078759 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c010dc90 con 0x56236cce5390 2024-08-31T07:35:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61c7400640 1 -- 172.21.15.32:0/368078759 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c010f5e0 con 0x56236cce5390 2024-08-31T07:35:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cf400640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61bc0b4ca0 0x7f61bc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cf400640 1 -- 172.21.15.32:0/3896610217 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f61c01059d0 con 0x7f61d8075ea0 2024-08-31T07:35:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61d7200640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61bc0b4ca0 0x7f61bc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 -- 172.21.15.32:0/368078759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cce5390 msgr2=0x56236d0ac5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 --2- 172.21.15.32:0/368078759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cce5390 0x56236d0ac5e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f61c0103020 tx=0x7f61c010bc60 comp rx=0 tx=0).stop 2024-08-31T07:35:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 -- 172.21.15.32:0/368078759 shutdown_connections 2024-08-31T07:35:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 --2- 172.21.15.32:0/368078759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cce5390 0x56236d0ac5e0 secure :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f61c0103020 tx=0x7f61c010bc60 comp rx=0 tx=0).stop 2024-08-31T07:35:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 --2- 172.21.15.32:0/368078759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 0x56236d0ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 --2- 172.21.15.32:0/368078759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236cd06070 0x56236d0abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 -- 172.21.15.32:0/368078759 >> 172.21.15.32:0/368078759 conn(0x56236ccfa460 msgr2=0x56236d0a9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 -- 172.21.15.32:0/368078759 shutdown_connections 2024-08-31T07:35:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 -- 172.21.15.32:0/368078759 wait complete. 2024-08-31T07:35:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61d7200640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61bc0b4ca0 0x7f61bc0b71b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f61d0005fd0 tx=0x7f61d0009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 -- start start 2024-08-31T07:35:12.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 Processor -- start 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 0x56236d0bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cd06070 0x56236d0c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccc49f0 0x56236d0cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56236ce74700 con 0x56236ccc49f0 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56236cf98cf0 con 0x56236cd06070 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56236d0c2830 con 0x56236ccdd1b0 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61e4213740 10 client.0 _ll_register_callbacks cb 0x56236cfbe470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61d6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 0x56236d0bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61d6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 0x56236d0bffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44328/0 (socket says 172.21.15.32:44328) 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61d6800640 1 -- 172.21.15.32:0/1522283473 learned_addr learned my addr 172.21.15.32:0/1522283473 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61cfe00640 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cd06070 0x56236d0c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.951+0000 7f61d7200640 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccc49f0 0x56236d0cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.956+0000 7f61d7200640 1 -- 172.21.15.32:0/1522283473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 msgr2=0x56236d0bffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:12.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.956+0000 7f61d7200640 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 0x56236d0bffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.956+0000 7f61d7200640 1 -- 172.21.15.32:0/1522283473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cd06070 msgr2=0x56236d0c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:12.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.956+0000 7f61d7200640 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cd06070 0x56236d0c05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.956+0000 7f61d7200640 1 -- 172.21.15.32:0/1522283473 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236cfd98b0 con 0x56236ccc49f0 2024-08-31T07:35:12.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.956+0000 7f61cfe00640 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cd06070 0x56236d0c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:12.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.956+0000 7f61d7200640 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccc49f0 0x56236d0cbb20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f61d0011e90 tx=0x7f61d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b238b1db07693e4e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:12.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.956+0000 7f61bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:35:12.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61bbe00640 1 -- 172.21.15.32:0/1522283473 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0004b20 con 0x56236ccc49f0 2024-08-31T07:35:12.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 -- 172.21.15.32:0/1522283473 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c010dc90 con 0x56236ccc49f0 2024-08-31T07:35:12.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61bbe00640 1 -- 172.21.15.32:0/1522283473 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f61d0004cc0 con 0x56236ccc49f0 2024-08-31T07:35:12.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61bbe00640 1 -- 172.21.15.32:0/1522283473 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f61d000f870 con 0x56236ccc49f0 2024-08-31T07:35:12.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 10 client.19638 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:12.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 -- 172.21.15.32:0/1522283473 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x56236d0ce9a0 con 0x56236ccc49f0 2024-08-31T07:35:12.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61bbe00640 1 -- 172.21.15.32:0/1522283473 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c010dc90 con 0x56236ccc49f0 2024-08-31T07:35:12.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61bbe00640 1 -- 172.21.15.32:0/1522283473 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f61d002e070 con 0x56236ccc49f0 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 10 client.19638 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 10 client.19638 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 20 client.19638 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56236cc99da0 0x56236d0d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 4 client.19638 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 -- 172.21.15.32:0/1522283473 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56236d024700 con 0x56236cc99da0 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61d6800640 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56236cc99da0 0x56236d0d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61d6800640 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56236cc99da0 0x56236d0d1eb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=20889e1450629273 server_cookie=ddd5452ac20e7377 in_seq=0 out_seq=0 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61bbe00640 10 client.19638 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61bbe00640 1 -- 172.21.15.32:0/1522283473 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56236d024700 con 0x56236cc99da0 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61bbe00640 10 client.19638 handle_command_reply: tid=0 2024-08-31T07:35:12.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 client.19638 shutdown 2024-08-31T07:35:12.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 -- 172.21.15.32:0/1522283473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccc49f0 msgr2=0x56236d0cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:12.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccc49f0 0x56236d0cbb20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f61d0011e90 tx=0x7f61d0004ab0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 -- 172.21.15.32:0/1522283473 shutdown_connections 2024-08-31T07:35:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56236cc99da0 0x56236d0d1eb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccc49f0 0x56236d0cbb20 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236cd06070 0x56236d0c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 --2- 172.21.15.32:0/1522283473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236ccdd1b0 0x56236d0bffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 -- 172.21.15.32:0/1522283473 >> 172.21.15.32:0/1522283473 conn(0x56236ccfa460 msgr2=0x56236d0a7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61bbe00640 0 client.19638 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:35:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 -- 172.21.15.32:0/1522283473 shutdown_connections 2024-08-31T07:35:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 1 -- 172.21.15.32:0/1522283473 wait complete. 2024-08-31T07:35:12.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.957+0000 7f61e4213740 20 client.19638 trim_cache size 0 max 16384 2024-08-31T07:35:12.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 Processor -- start 2024-08-31T07:35:12.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 -- start start 2024-08-31T07:35:12.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0be660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 0x56236d0beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236cd06070 0x56236d0a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81ad0f0 con 0x56236ccdd1b0 2024-08-31T07:35:12.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81aee10 con 0x56236ccc49f0 2024-08-31T07:35:12.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81ad880 con 0x56236cd06070 2024-08-31T07:35:12.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 0x56236d0beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 0x56236d0beba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56466/0 (socket says 172.21.15.32:56466) 2024-08-31T07:35:12.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61cfe00640 1 -- 172.21.15.32:0/1094926012 learned_addr learned my addr 172.21.15.32:0/1094926012 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:12.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61d6800640 1 --2- 172.21.15.32:0/1094926012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0be660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61cfe00640 1 -- 172.21.15.32:0/1094926012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236cd06070 msgr2=0x56236d0a7ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:35:12.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61cfe00640 1 --2- 172.21.15.32:0/1094926012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236cd06070 0x56236d0a7ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61cfe00640 1 -- 172.21.15.32:0/1094926012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 msgr2=0x56236d0be660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:12.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61cfe00640 1 --2- 172.21.15.32:0/1094926012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0be660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61cfe00640 1 -- 172.21.15.32:0/1094926012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236d0a9180 con 0x56236ccdd1b0 2024-08-31T07:35:12.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61d6800640 1 --2- 172.21.15.32:0/1094926012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:12.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61cfe00640 1 --2- 172.21.15.32:0/1094926012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 0x56236d0beba0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f61c010d670 tx=0x7f61c0110d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3779cb125c1fe70d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:12.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61c6a00640 1 -- 172.21.15.32:0/1094926012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0114070 con 0x56236ccdd1b0 2024-08-31T07:35:12.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61c6a00640 1 -- 172.21.15.32:0/1094926012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c00cbc40 con 0x56236ccdd1b0 2024-08-31T07:35:12.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61c6a00640 1 -- 172.21.15.32:0/1094926012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c00cc130 con 0x56236ccdd1b0 2024-08-31T07:35:12.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 -- 172.21.15.32:0/1094926012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 msgr2=0x56236d0beba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:12.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 --2- 172.21.15.32:0/1094926012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 0x56236d0beba0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f61c010d670 tx=0x7f61c0110d40 comp rx=0 tx=0).stop 2024-08-31T07:35:12.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 -- 172.21.15.32:0/1094926012 shutdown_connections 2024-08-31T07:35:12.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 --2- 172.21.15.32:0/1094926012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236cd06070 0x56236d0a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 --2- 172.21.15.32:0/1094926012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 0x56236d0beba0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 --2- 172.21.15.32:0/1094926012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.974+0000 7f61e4213740 1 -- 172.21.15.32:0/1094926012 >> 172.21.15.32:0/1094926012 conn(0x56236ccfa460 msgr2=0x56236d0baf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:12.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 1 -- 172.21.15.32:0/1094926012 shutdown_connections 2024-08-31T07:35:12.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 1 -- 172.21.15.32:0/1094926012 wait complete. 2024-08-31T07:35:12.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 1 -- start start 2024-08-31T07:35:12.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 1 Processor -- start 2024-08-31T07:35:12.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 0x56236d0c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236cd06070 0x56236d0c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81ae740 con 0x56236ccdd1b0 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81adfb0 con 0x56236ccc49f0 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81ab130 con 0x56236cd06070 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.979+0000 7f61e4213740 10 client.0 _ll_register_callbacks cb 0x7f61d0003e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.981+0000 7f61d6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.981+0000 7f61d6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59838/0 (socket says 172.21.15.32:59838) 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.981+0000 7f61d6800640 1 -- 172.21.15.32:0/3314320793 learned_addr learned my addr 172.21.15.32:0/3314320793 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.981+0000 7f61cfe00640 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 0x56236d0c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.981+0000 7f61d6800640 1 -- 172.21.15.32:0/3314320793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236cd06070 msgr2=0x56236d0c2b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.981+0000 7f61d6800640 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236cd06070 0x56236d0c2b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.981+0000 7f61d6800640 1 -- 172.21.15.32:0/3314320793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 msgr2=0x56236d0c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:12.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.981+0000 7f61d6800640 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 0x56236d0c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.981+0000 7f61d6800640 1 -- 172.21.15.32:0/3314320793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236d0c32c0 con 0x56236ccc49f0 2024-08-31T07:35:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61d6800640 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0c7e40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f61c80125c0 tx=0x7f61c8012a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8acfd40ff999d68e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:35:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61bbe00640 1 -- 172.21.15.32:0/3314320793 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8004020 con 0x56236ccc49f0 2024-08-31T07:35:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61bbe00640 1 -- 172.21.15.32:0/3314320793 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f61c80041c0 con 0x56236ccc49f0 2024-08-31T07:35:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 -- 172.21.15.32:0/3314320793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c00cbc40 con 0x56236ccc49f0 2024-08-31T07:35:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61bbe00640 1 -- 172.21.15.32:0/3314320793 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f61c80043c0 con 0x56236ccc49f0 2024-08-31T07:35:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 10 client.29687 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 -- 172.21.15.32:0/3314320793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x56236d13d350 con 0x56236ccc49f0 2024-08-31T07:35:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61bbe00640 1 -- 172.21.15.32:0/3314320793 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c00cbc40 con 0x56236ccc49f0 2024-08-31T07:35:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61bbe00640 1 -- 172.21.15.32:0/3314320793 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f61c8005430 con 0x56236ccc49f0 2024-08-31T07:35:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 10 client.29687 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 10 client.29687 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:35:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 20 client.29687 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56236d13dab0 0x56236d13fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 4 client.29687 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:35:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 -- 172.21.15.32:0/3314320793 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56236d0201a0 con 0x56236d13dab0 2024-08-31T07:35:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61cfe00640 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56236d13dab0 0x56236d13fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61cfe00640 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56236d13dab0 0x56236d13fed0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=beafdf2a1e0c1293 server_cookie=b1322c07650b539e in_seq=0 out_seq=0 2024-08-31T07:35:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61bbe00640 10 client.29687 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:35:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61bbe00640 1 -- 172.21.15.32:0/3314320793 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56236d0201a0 con 0x56236d13dab0 2024-08-31T07:35:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61bbe00640 10 client.29687 handle_command_reply: tid=0 2024-08-31T07:35:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 client.29687 shutdown 2024-08-31T07:35:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 -- 172.21.15.32:0/3314320793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 msgr2=0x56236d0c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0c7e40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f61c80125c0 tx=0x7f61c8012a80 comp rx=0 tx=0).stop 2024-08-31T07:35:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 -- 172.21.15.32:0/3314320793 shutdown_connections 2024-08-31T07:35:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x56236d13dab0 0x56236d13fed0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56236cd06070 0x56236d0c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56236ccdd1b0 0x56236d0c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 --2- 172.21.15.32:0/3314320793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56236ccc49f0 0x56236d0c7e40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61bbe00640 0 client.29687 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:35:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 -- 172.21.15.32:0/3314320793 >> 172.21.15.32:0/3314320793 conn(0x56236ccfa460 msgr2=0x56236d0baf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 -- 172.21.15.32:0/3314320793 shutdown_connections 2024-08-31T07:35:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 1 -- 172.21.15.32:0/3314320793 wait complete. 2024-08-31T07:35:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61e4213740 20 client.29687 trim_cache size 0 max 16384 2024-08-31T07:35:12.999 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:35:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 -- 172.21.15.32:0/3896610217 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61bc0b4ca0 msgr2=0x7f61bc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61bc0b4ca0 0x7f61bc0b71b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f61d0005fd0 tx=0x7f61d0009500 comp rx=0 tx=0).stop 2024-08-31T07:35:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 -- 172.21.15.32:0/3896610217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d8075ea0 msgr2=0x7f61d819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d8075ea0 0x7f61d819c7f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f61c002a910 tx=0x7f61c0031a20 comp rx=0 tx=0).stop 2024-08-31T07:35:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 -- 172.21.15.32:0/3896610217 shutdown_connections 2024-08-31T07:35:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61bc0b4ca0 0x7f61bc0b71b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d810fcb0 0x7f61d81a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61d8075ea0 0x7f61d819c7f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 --2- 172.21.15.32:0/3896610217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d8075550 0x7f61d819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 -- 172.21.15.32:0/3896610217 >> 172.21.15.32:0/3896610217 conn(0x7f61d80ff1e0 msgr2=0x7f61d8109850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 -- 172.21.15.32:0/3896610217 shutdown_connections 2024-08-31T07:35:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:12.993+0000 7f61dca00640 1 -- 172.21.15.32:0/3896610217 wait complete. 2024-08-31T07:35:13.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157422 14.01 MB/sec execute 413 sec latency 17353.677 ms 2024-08-31T07:35:13.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:35:14.245 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:14.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157427 13.97 MB/sec execute 414 sec latency 18353.722 ms 2024-08-31T07:35:14.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.581+0000 7fa3f7400640 1 -- 172.21.15.32:0/4282825045 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5589083fbf20 con 0x558908076570 2024-08-31T07:35:14.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.581+0000 7fa3f7400640 10 client.19605 handle_command_reply: tid=0 2024-08-31T07:35:14.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.581+0000 7fa4239f8740 1 client.19605 shutdown 2024-08-31T07:35:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.581+0000 7fa4239f8740 1 -- 172.21.15.32:0/4282825045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 msgr2=0x55890850a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.581+0000 7fa4239f8740 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 0x55890850a5f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa410013fd0 tx=0x7fa410009500 comp rx=0 tx=0).stop 2024-08-31T07:35:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.582+0000 7fa4239f8740 1 -- 172.21.15.32:0/4282825045 shutdown_connections 2024-08-31T07:35:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.582+0000 7fa4239f8740 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x558908076570 0x558908514870 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.582+0000 7fa4239f8740 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589080e2840 0x55890850a5f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.582+0000 7fa4239f8740 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589080c1b60 0x558908499ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.582+0000 7fa4239f8740 1 --2- 172.21.15.32:0/4282825045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589080b9980 0x558908499500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.582+0000 7fa4239f8740 1 -- 172.21.15.32:0/4282825045 >> 172.21.15.32:0/4282825045 conn(0x5589080d6990 msgr2=0x558908496920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:14.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.582+0000 7fa3f7400640 0 client.19605 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:14.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.582+0000 7fa4239f8740 1 -- 172.21.15.32:0/4282825045 shutdown_connections 2024-08-31T07:35:14.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.582+0000 7fa4239f8740 1 -- 172.21.15.32:0/4282825045 wait complete. 2024-08-31T07:35:14.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.582+0000 7fa4239f8740 20 client.19605 trim_cache size 0 max 16384 2024-08-31T07:35:14.584 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:35:14.584 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 86, 2024-08-31T07:35:14.584 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:35:14.584 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 70, 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "fdae7e2f": { 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 70, 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 150.0, 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:35:14.585 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:35:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:35:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:35:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:14.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:14.586 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.583+0000 7fa41cc00640 1 -- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3fc0b4ca0 msgr2=0x7fa3fc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.583+0000 7fa41cc00640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3fc0b4ca0 0x7fa3fc0b71b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa410005fd0 tx=0x7fa410004a40 comp rx=0 tx=0).stop 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.583+0000 7fa41cc00640 1 -- 172.21.15.32:0/237024494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa418075b70 msgr2=0x7fa4181a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.583+0000 7fa41cc00640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa418075b70 0x7fa4181a07e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa408005940 tx=0x7fa4080040c0 comp rx=0 tx=0).stop 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.584+0000 7fa41cc00640 1 -- 172.21.15.32:0/237024494 shutdown_connections 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.584+0000 7fa41cc00640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa3fc0b4ca0 0x7fa3fc0b71b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.584+0000 7fa41cc00640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4181141f0 0x7fa4181a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.584+0000 7fa41cc00640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa418077120 0x7fa4181a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.584+0000 7fa41cc00640 1 --2- 172.21.15.32:0/237024494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa418075b70 0x7fa4181a07e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.584+0000 7fa41cc00640 1 -- 172.21.15.32:0/237024494 >> 172.21.15.32:0/237024494 conn(0x7fa4180ff370 msgr2=0x7fa4181100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.584+0000 7fa41cc00640 1 -- 172.21.15.32:0/237024494 shutdown_connections 2024-08-31T07:35:14.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:14.584+0000 7fa41cc00640 1 -- 172.21.15.32:0/237024494 wait complete. 2024-08-31T07:35:14.642 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:14 smithi032 ceph-mon[17252]: pgmap v1075: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 824 KiB/s wr, 120 op/s 2024-08-31T07:35:14.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:14 smithi084 ceph-mon[22387]: pgmap v1075: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 824 KiB/s wr, 120 op/s 2024-08-31T07:35:14.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:14 smithi073 ceph-mon[21995]: pgmap v1075: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 824 KiB/s wr, 120 op/s 2024-08-31T07:35:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.252+0000 7f477cc00640 1 -- 172.21.15.32:0/1633929825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47781099e0 msgr2=0x7f4778110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.252+0000 7f477cc00640 1 --2- 172.21.15.32:0/1633929825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47781099e0 0x7f4778110210 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4770009ca0 tx=0x7f477002a8b0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.252+0000 7f477cc00640 1 -- 172.21.15.32:0/1633929825 shutdown_connections 2024-08-31T07:35:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.252+0000 7f477cc00640 1 --2- 172.21.15.32:0/1633929825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47781099e0 0x7f4778110210 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.252+0000 7f477cc00640 1 --2- 172.21.15.32:0/1633929825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4778109090 0x7f47781094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.252+0000 7f477cc00640 1 --2- 172.21.15.32:0/1633929825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778103750 0x7f4778103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.252+0000 7f477cc00640 1 -- 172.21.15.32:0/1633929825 >> 172.21.15.32:0/1633929825 conn(0x7f47780ff1e0 msgr2=0x7f4778101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.252+0000 7f477cc00640 1 -- 172.21.15.32:0/1633929825 shutdown_connections 2024-08-31T07:35:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.253+0000 7f477cc00640 1 -- 172.21.15.32:0/1633929825 wait complete. 2024-08-31T07:35:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.254+0000 7f477cc00640 1 Processor -- start 2024-08-31T07:35:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.254+0000 7f477cc00640 1 -- start start 2024-08-31T07:35:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.254+0000 7f477cc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778103750 0x7f47780754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.254+0000 7f477cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4778109090 0x7f47780759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.254+0000 7f477cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47781099e0 0x7f4778075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.254+0000 7f477cc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4778113700 con 0x7f4778109090 2024-08-31T07:35:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.254+0000 7f477cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4778113580 con 0x7f47781099e0 2024-08-31T07:35:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.254+0000 7f477cc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4778113880 con 0x7f4778103750 2024-08-31T07:35:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778103750 0x7f47780754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4776a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4778109090 0x7f47780759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778103750 0x7f47780754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44352/0 (socket says 172.21.15.32:44352) 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777400640 1 -- 172.21.15.32:0/611441745 learned_addr learned my addr 172.21.15.32:0/611441745 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4776a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4778109090 0x7f47780759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56484/0 (socket says 172.21.15.32:56484) 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777e00640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47781099e0 0x7f4778075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777400640 1 -- 172.21.15.32:0/611441745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47781099e0 msgr2=0x7f4778075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777400640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47781099e0 0x7f4778075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777400640 1 -- 172.21.15.32:0/611441745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4778109090 msgr2=0x7f47780759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777400640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4778109090 0x7f47780759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777400640 1 -- 172.21.15.32:0/611441745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47781a83b0 con 0x7f4778103750 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777e00640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47781099e0 0x7f4778075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.255+0000 7f4777400640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778103750 0x7f47780754b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f476800e9f0 tx=0x7f476800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.256+0000 7f476fe00640 1 -- 172.21.15.32:0/611441745 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476800cd80 con 0x7f4778103750 2024-08-31T07:35:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.256+0000 7f477cc00640 1 -- 172.21.15.32:0/611441745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47781a86a0 con 0x7f4778103750 2024-08-31T07:35:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.256+0000 7f476fe00640 1 -- 172.21.15.32:0/611441745 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47680054d0 con 0x7f4778103750 2024-08-31T07:35:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.256+0000 7f476fe00640 1 -- 172.21.15.32:0/611441745 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768010690 con 0x7f4778103750 2024-08-31T07:35:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.256+0000 7f477cc00640 1 -- 172.21.15.32:0/611441745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781a8be0 con 0x7f4778103750 2024-08-31T07:35:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.259+0000 7f4784ed3740 1 Processor -- start 2024-08-31T07:35:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.259+0000 7f4784ed3740 1 -- start start 2024-08-31T07:35:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 0x55f3067ffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3064311b0 0x55f3067fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f306439390 0x55f3068004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4784ed3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f3067767b0 con 0x55f306439390 2024-08-31T07:35:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4784ed3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f306765d40 con 0x55f3064311b0 2024-08-31T07:35:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4784ed3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f306769cf0 con 0x55f30645a070 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4777e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 0x55f3067ffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4777e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 0x55f3067ffa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44362/0 (socket says 172.21.15.32:44362) 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4777e00640 1 -- 172.21.15.32:0/2659159934 learned_addr learned my addr 172.21.15.32:0/2659159934 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4776a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f306439390 0x55f3068004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f476fe00640 1 -- 172.21.15.32:0/611441745 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4768004e90 con 0x7f4778103750 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4777e00640 1 -- 172.21.15.32:0/2659159934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3064311b0 msgr2=0x55f3067fffb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4777e00640 1 --2- 172.21.15.32:0/2659159934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3064311b0 0x55f3067fffb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4777e00640 1 -- 172.21.15.32:0/2659159934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f306439390 msgr2=0x55f3068004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4777e00640 1 --2- 172.21.15.32:0/2659159934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f306439390 0x55f3068004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.260+0000 7f4777e00640 1 -- 172.21.15.32:0/2659159934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3064c5650 con 0x55f30645a070 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.261+0000 7f4777e00640 1 --2- 172.21.15.32:0/2659159934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 0x55f3067ffa70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4770009ca0 tx=0x7f4770030f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=439df9bd62773396 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.261+0000 7f4763e00640 1 -- 172.21.15.32:0/2659159934 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770030ac0 con 0x55f30645a070 2024-08-31T07:35:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.261+0000 7f4763e00640 1 -- 172.21.15.32:0/2659159934 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4770036030 con 0x55f30645a070 2024-08-31T07:35:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.261+0000 7f4763e00640 1 -- 172.21.15.32:0/2659159934 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770004630 con 0x55f30645a070 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f476fe00640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f475c0b4d20 0x7f475c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f4784ed3740 1 -- 172.21.15.32:0/2659159934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 msgr2=0x55f3067ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2659159934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 0x55f3067ffa70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4770009ca0 tx=0x7f4770030f90 comp rx=0 tx=0).stop 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f4784ed3740 1 -- 172.21.15.32:0/2659159934 shutdown_connections 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2659159934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f306439390 0x55f3068004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2659159934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3064311b0 0x55f3067fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2659159934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 0x55f3067ffa70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f4784ed3740 1 -- 172.21.15.32:0/2659159934 >> 172.21.15.32:0/2659159934 conn(0x55f30644e460 msgr2=0x55f3067fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f476fe00640 1 -- 172.21.15.32:0/611441745 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f47680e58e0 con 0x7f4778103750 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f4777400640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f475c0b4d20 0x7f475c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f4784ed3740 1 -- 172.21.15.32:0/2659159934 shutdown_connections 2024-08-31T07:35:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.262+0000 7f4784ed3740 1 -- 172.21.15.32:0/2659159934 wait complete. 2024-08-31T07:35:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.263+0000 7f4777400640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f475c0b4d20 0x7f475c0b7230 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x55f3067fe6f0 tx=0x7f4768004990 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.263+0000 7f4784ed3740 1 -- start start 2024-08-31T07:35:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.263+0000 7f4784ed3740 1 Processor -- start 2024-08-31T07:35:15.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f30680b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f306439390 0x55f30680bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 0x55f3068174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4784ed3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f3065c8700 con 0x55f3064311b0 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4784ed3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f3066eccf0 con 0x55f306439390 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4784ed3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f30680e1f0 con 0x55f30645a070 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4784ed3740 10 client.0 _ll_register_callbacks cb 0x55f306712470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4777e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f30680b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4776a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f306439390 0x55f30680bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4777e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f30680b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56508/0 (socket says 172.21.15.32:56508) 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4776a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f306439390 0x55f30680bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59878/0 (socket says 172.21.15.32:59878) 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.264+0000 7f4777e00640 1 -- 172.21.15.32:0/2929411355 learned_addr learned my addr 172.21.15.32:0/2929411355 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4777e00640 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 0x55f3068174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4776a00640 1 -- 172.21.15.32:0/2929411355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 msgr2=0x55f3068174e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4776a00640 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 0x55f3068174e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4776a00640 1 -- 172.21.15.32:0/2929411355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 msgr2=0x55f30680b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4776a00640 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f30680b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4776a00640 1 -- 172.21.15.32:0/2929411355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f30672d8b0 con 0x55f306439390 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4777e00640 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f30680b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4776a00640 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f306439390 0x55f30680bb00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f477806b4a0 tx=0x7f4764012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe6b6bb8835d861 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4760c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4760c00640 1 -- 172.21.15.32:0/2929411355 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764004030 con 0x55f306439390 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4760c00640 1 -- 172.21.15.32:0/2929411355 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f47640041d0 con 0x55f306439390 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.265+0000 7f4784ed3740 1 -- 172.21.15.32:0/2929411355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4770036030 con 0x55f306439390 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.266+0000 7f4760c00640 1 -- 172.21.15.32:0/2929411355 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f47640043d0 con 0x55f306439390 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.266+0000 7f4784ed3740 10 client.29696 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.266+0000 7f4784ed3740 1 -- 172.21.15.32:0/2929411355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x55f30681a260 con 0x55f306439390 2024-08-31T07:35:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.266+0000 7f4760c00640 1 -- 172.21.15.32:0/2929411355 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4770036030 con 0x55f306439390 2024-08-31T07:35:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.266+0000 7f4760c00640 1 -- 172.21.15.32:0/2929411355 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f4764023020 con 0x55f306439390 2024-08-31T07:35:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.267+0000 7f4784ed3740 10 client.29696 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.267+0000 7f4784ed3740 10 client.29696 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:35:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.267+0000 7f4784ed3740 20 client.29696 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.267+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f3064189f0 0x55f30681d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.267+0000 7f4784ed3740 4 client.29696 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.267+0000 7f4784ed3740 1 -- 172.21.15.32:0/2929411355 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f306778700 con 0x55f3064189f0 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.267+0000 7f4777e00640 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f3064189f0 0x55f30681d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.268+0000 7f4777e00640 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f3064189f0 0x55f30681d7d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf55895a3406272d server_cookie=dde7560785e828c6 in_seq=0 out_seq=0 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.268+0000 7f4760c00640 10 client.29696 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.268+0000 7f4760c00640 1 -- 172.21.15.32:0/2929411355 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f306778700 con 0x55f3064189f0 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.268+0000 7f4760c00640 10 client.29696 handle_command_reply: tid=0 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.268+0000 7f4784ed3740 1 client.29696 shutdown 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 1 -- 172.21.15.32:0/2929411355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f306439390 msgr2=0x55f30680bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f306439390 0x55f30680bb00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f477806b4a0 tx=0x7f4764012b60 comp rx=0 tx=0).stop 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 1 -- 172.21.15.32:0/2929411355 shutdown_connections 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f3064189f0 0x55f30681d7d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f30645a070 0x55f3068174e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f306439390 0x55f30680bb00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2929411355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f30680b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4760c00640 0 client.29696 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:35:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 1 -- 172.21.15.32:0/2929411355 >> 172.21.15.32:0/2929411355 conn(0x55f30644e460 msgr2=0x55f306802c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 1 -- 172.21.15.32:0/2929411355 shutdown_connections 2024-08-31T07:35:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 1 -- 172.21.15.32:0/2929411355 wait complete. 2024-08-31T07:35:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.269+0000 7f4784ed3740 20 client.29696 trim_cache size 0 max 16384 2024-08-31T07:35:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.283+0000 7f4784ed3740 1 Processor -- start 2024-08-31T07:35:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.283+0000 7f4784ed3740 1 -- start start 2024-08-31T07:35:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f306803530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f3068011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f30645a070 0x55f3067fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4784ed3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f47781b1430 con 0x55f3064311b0 2024-08-31T07:35:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4784ed3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47781b3150 con 0x55f30645a070 2024-08-31T07:35:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4784ed3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47781b1bc0 con 0x55f306439390 2024-08-31T07:35:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4776a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f3068011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4776a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f3068011b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44366/0 (socket says 172.21.15.32:44366) 2024-08-31T07:35:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4776a00640 1 -- 172.21.15.32:0/387643650 learned_addr learned my addr 172.21.15.32:0/387643650 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4777e00640 1 --2- 172.21.15.32:0/387643650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f306803530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4777e00640 1 --2- 172.21.15.32:0/387643650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f30645a070 0x55f3067fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4776a00640 1 -- 172.21.15.32:0/387643650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f30645a070 msgr2=0x55f3067fcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4776a00640 1 --2- 172.21.15.32:0/387643650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f30645a070 0x55f3067fcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4776a00640 1 -- 172.21.15.32:0/387643650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 msgr2=0x55f306803530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4776a00640 1 --2- 172.21.15.32:0/387643650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f306803530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4776a00640 1 -- 172.21.15.32:0/387643650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f30681a070 con 0x55f306439390 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.284+0000 7f4777e00640 1 --2- 172.21.15.32:0/387643650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f306803530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.285+0000 7f4776a00640 1 --2- 172.21.15.32:0/387643650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f3068011b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f476400a010 tx=0x7f4764003d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=166304cb24139c4b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.285+0000 7f4763400640 1 -- 172.21.15.32:0/387643650 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764004000 con 0x55f306439390 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.285+0000 7f4763400640 1 -- 172.21.15.32:0/387643650 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f476402a020 con 0x55f306439390 2024-08-31T07:35:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.285+0000 7f4763400640 1 -- 172.21.15.32:0/387643650 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764025460 con 0x55f306439390 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.286+0000 7f4784ed3740 1 -- 172.21.15.32:0/387643650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 msgr2=0x55f3068011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.286+0000 7f4784ed3740 1 --2- 172.21.15.32:0/387643650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f3068011b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f476400a010 tx=0x7f4764003d30 comp rx=0 tx=0).stop 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.286+0000 7f4784ed3740 1 -- 172.21.15.32:0/387643650 shutdown_connections 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.286+0000 7f4784ed3740 1 --2- 172.21.15.32:0/387643650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f30645a070 0x55f3067fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.286+0000 7f4784ed3740 1 --2- 172.21.15.32:0/387643650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f3068011b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.286+0000 7f4784ed3740 1 --2- 172.21.15.32:0/387643650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f3064311b0 0x55f306803530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.286+0000 7f4784ed3740 1 -- 172.21.15.32:0/387643650 >> 172.21.15.32:0/387643650 conn(0x55f30644e460 msgr2=0x55f306800aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.286+0000 7f4784ed3740 1 -- 172.21.15.32:0/387643650 shutdown_connections 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.286+0000 7f4784ed3740 1 -- 172.21.15.32:0/387643650 wait complete. 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.287+0000 7f4784ed3740 1 -- start start 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.287+0000 7f4784ed3740 1 Processor -- start 2024-08-31T07:35:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3064311b0 0x55f30680f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f30680fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4784ed3740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f30645a070 0x55f30680ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4784ed3740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f47781b2a80 con 0x55f30645a070 2024-08-31T07:35:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4784ed3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47781b22f0 con 0x55f3064311b0 2024-08-31T07:35:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4784ed3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47781af430 con 0x55f306439390 2024-08-31T07:35:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4784ed3740 10 client.0 _ll_register_callbacks cb 0x7f4764009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4776a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f30680fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4777e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3064311b0 0x55f30680f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4776a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f30680fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:44378/0 (socket says 172.21.15.32:44378) 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4776a00640 1 -- 172.21.15.32:0/2822365630 learned_addr learned my addr 172.21.15.32:0/2822365630 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.288+0000 7f4777e00640 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f30645a070 0x55f30680ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.289+0000 7f4776a00640 1 -- 172.21.15.32:0/2822365630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3064311b0 msgr2=0x55f30680f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.289+0000 7f4776a00640 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3064311b0 0x55f30680f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.289+0000 7f4776a00640 1 -- 172.21.15.32:0/2822365630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f30645a070 msgr2=0x55f30680ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.289+0000 7f4776a00640 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f30645a070 0x55f30680ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.289+0000 7f4776a00640 1 -- 172.21.15.32:0/2822365630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3068872d0 con 0x55f306439390 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.289+0000 7f4777e00640 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f30645a070 0x55f30680ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.289+0000 7f4776a00640 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f30680fa40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f47640126a0 tx=0x7f4764005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a5ff8067be5c65 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.289+0000 7f4760c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.289+0000 7f4760c00640 1 -- 172.21.15.32:0/2822365630 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47640053c0 con 0x55f306439390 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.289+0000 7f4760c00640 1 -- 172.21.15.32:0/2822365630 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f4764005560 con 0x55f306439390 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.290+0000 7f4784ed3740 1 -- 172.21.15.32:0/2822365630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f476402a020 con 0x55f306439390 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.290+0000 7f4760c00640 1 -- 172.21.15.32:0/2822365630 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f476403d050 con 0x55f306439390 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.290+0000 7f4784ed3740 10 client.29707 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.290+0000 7f4784ed3740 1 -- 172.21.15.32:0/2822365630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x55f306888530 con 0x55f306439390 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.290+0000 7f4760c00640 1 -- 172.21.15.32:0/2822365630 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f476402a020 con 0x55f306439390 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.290+0000 7f4760c00640 1 -- 172.21.15.32:0/2822365630 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f4764023070 con 0x55f306439390 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.290+0000 7f4784ed3740 10 client.29707 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.291+0000 7f4784ed3740 10 client.29707 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.291+0000 7f4784ed3740 20 client.29707 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.291+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f3063edda0 0x55f30688ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.291+0000 7f4784ed3740 4 client.29707 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:35:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.291+0000 7f4784ed3740 1 -- 172.21.15.32:0/2822365630 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3067741a0 con 0x55f3063edda0 2024-08-31T07:35:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.291+0000 7f4777e00640 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f3063edda0 0x55f30688ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.291+0000 7f4760c00640 10 client.29707 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.291+0000 7f4777e00640 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f3063edda0 0x55f30688ac50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b6e08a43c3d1807 server_cookie=a183c3821d24bc71 in_seq=0 out_seq=0 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4760c00640 1 -- 172.21.15.32:0/2822365630 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3067741a0 con 0x55f3063edda0 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4760c00640 10 client.29707 handle_command_reply: tid=0 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 client.29707 shutdown 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 -- 172.21.15.32:0/2822365630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 msgr2=0x55f30680fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f30680fa40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f47640126a0 tx=0x7f4764005040 comp rx=0 tx=0).stop 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 -- 172.21.15.32:0/2822365630 shutdown_connections 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x55f3063edda0 0x55f30688ac50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f30645a070 0x55f30680ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f306439390 0x55f30680fa40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 --2- 172.21.15.32:0/2822365630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3064311b0 0x55f30680f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 -- 172.21.15.32:0/2822365630 >> 172.21.15.32:0/2822365630 conn(0x55f30644e460 msgr2=0x55f306803300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4760c00640 0 client.29707 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:35:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 -- 172.21.15.32:0/2822365630 shutdown_connections 2024-08-31T07:35:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 1 -- 172.21.15.32:0/2822365630 wait complete. 2024-08-31T07:35:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.292+0000 7f4784ed3740 20 client.29707 trim_cache size 0 max 16384 2024-08-31T07:35:15.296 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:35:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 -- 172.21.15.32:0/611441745 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f475c0b4d20 msgr2=0x7f475c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f475c0b4d20 0x7f475c0b7230 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x55f3067fe6f0 tx=0x7f4768004990 comp rx=0 tx=0).stop 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 -- 172.21.15.32:0/611441745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778103750 msgr2=0x7f47780754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778103750 0x7f47780754b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f476800e9f0 tx=0x7f476800edf0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 -- 172.21.15.32:0/611441745 shutdown_connections 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f475c0b4d20 0x7f475c0b7230 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47781099e0 0x7f4778075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4778109090 0x7f47780759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 --2- 172.21.15.32:0/611441745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4778103750 0x7f47780754b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 -- 172.21.15.32:0/611441745 >> 172.21.15.32:0/611441745 conn(0x7f47780ff1e0 msgr2=0x7f47781086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.293+0000 7f477cc00640 1 -- 172.21.15.32:0/611441745 shutdown_connections 2024-08-31T07:35:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:15.294+0000 7f477cc00640 1 -- 172.21.15.32:0/611441745 wait complete. 2024-08-31T07:35:15.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.94 MB/sec execute 415 sec latency 18479.260 ms 2024-08-31T07:35:16.262 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fdae7e2f', quiesce took 18.9 seconds. Will release after: 11.100000000000001 2024-08-31T07:35:16.422 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.91 MB/sec execute 416 sec latency 12338.283 ms 2024-08-31T07:35:16.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:35:16.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:16 smithi084 ceph-mon[22387]: pgmap v1076: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-08-31T07:35:16.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:16 smithi032 ceph-mon[17252]: pgmap v1076: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-08-31T07:35:16.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:16 smithi073 ceph-mon[21995]: pgmap v1076: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-08-31T07:35:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:17.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.87 MB/sec execute 417 sec latency 13338.349 ms 2024-08-31T07:35:17.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.688+0000 7f2502c00640 1 -- 172.21.15.32:0/2689748481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24fc103730 msgr2=0x7f24fc103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:17.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.688+0000 7f2502c00640 1 --2- 172.21.15.32:0/2689748481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24fc103730 0x7f24fc103b40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009900 tx=0x7f24f002a410 comp rx=0 tx=0).stop 2024-08-31T07:35:17.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.688+0000 7f2502c00640 1 -- 172.21.15.32:0/2689748481 shutdown_connections 2024-08-31T07:35:17.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.688+0000 7f2502c00640 1 --2- 172.21.15.32:0/2689748481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24fc1099c0 0x7f24fc1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.688+0000 7f2502c00640 1 --2- 172.21.15.32:0/2689748481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24fc109070 0x7f24fc109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.688+0000 7f2502c00640 1 --2- 172.21.15.32:0/2689748481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24fc103730 0x7f24fc103b40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.688+0000 7f2502c00640 1 -- 172.21.15.32:0/2689748481 >> 172.21.15.32:0/2689748481 conn(0x7f24fc0ff180 msgr2=0x7f24fc1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:17.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.688+0000 7f2502c00640 1 -- 172.21.15.32:0/2689748481 shutdown_connections 2024-08-31T07:35:17.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.689+0000 7f2502c00640 1 -- 172.21.15.32:0/2689748481 wait complete. 2024-08-31T07:35:17.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.690+0000 7f2502c00640 1 Processor -- start 2024-08-31T07:35:17.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.690+0000 7f2502c00640 1 -- start start 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.690+0000 7f2502c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24fc103730 0x7f24fc19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.690+0000 7f2502c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24fc109070 0x7f24fc19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.690+0000 7f2502c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24fc1099c0 0x7f24fc1a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.690+0000 7f2502c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc1136e0 con 0x7f24fc103730 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.690+0000 7f2502c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc113560 con 0x7f24fc1099c0 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.690+0000 7f2502c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc113860 con 0x7f24fc109070 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2501800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24fc103730 0x7f24fc19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2501800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24fc103730 0x7f24fc19c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42410/0 (socket says 172.21.15.32:42410) 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2501800640 1 -- 172.21.15.32:0/2787321395 learned_addr learned my addr 172.21.15.32:0/2787321395 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2500e00640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24fc109070 0x7f24fc19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2502200640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24fc1099c0 0x7f24fc1a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2501800640 1 -- 172.21.15.32:0/2787321395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24fc109070 msgr2=0x7f24fc19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:17.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2501800640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24fc109070 0x7f24fc19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2501800640 1 -- 172.21.15.32:0/2787321395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24fc1099c0 msgr2=0x7f24fc1a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:17.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2501800640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24fc1099c0 0x7f24fc1a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2501800640 1 -- 172.21.15.32:0/2787321395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc1a2240 con 0x7f24fc103730 2024-08-31T07:35:17.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.691+0000 7f2500e00640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24fc109070 0x7f24fc19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:17.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.692+0000 7f2501800640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24fc103730 0x7f24fc19c2d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f24f0005ab0 tx=0x7f24f0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:17.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.692+0000 7f24fa000640 1 -- 172.21.15.32:0/2787321395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f003f070 con 0x7f24fc103730 2024-08-31T07:35:17.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.692+0000 7f24fa000640 1 -- 172.21.15.32:0/2787321395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f0031b60 con 0x7f24fc103730 2024-08-31T07:35:17.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.692+0000 7f2502c00640 1 -- 172.21.15.32:0/2787321395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24fc1a24d0 con 0x7f24fc103730 2024-08-31T07:35:17.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.692+0000 7f24fa000640 1 -- 172.21.15.32:0/2787321395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0035030 con 0x7f24fc103730 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.692+0000 7f2502c00640 1 -- 172.21.15.32:0/2787321395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24fc1a8c90 con 0x7f24fc103730 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.695+0000 7f250ae4f740 1 Processor -- start 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.695+0000 7f250ae4f740 1 -- start start 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f24fa000640 1 -- 172.21.15.32:0/2787321395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f24f0042070 con 0x7f24fc103730 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x564102811b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x5641028125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f250ae4f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5641027887b0 con 0x56410246c070 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f250ae4f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564102777d40 con 0x5641024431b0 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f250ae4f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56410277bcf0 con 0x56410244b390 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f2502200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x564102811b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f2501800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f2502200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x564102811b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42422/0 (socket says 172.21.15.32:42422) 2024-08-31T07:35:17.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f2501800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:49944/0 (socket says 172.21.15.32:49944) 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f2502200640 1 -- 172.21.15.32:0/2325323433 learned_addr learned my addr 172.21.15.32:0/2325323433 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.696+0000 7f2500e00640 1 --2- 172.21.15.32:0/2325323433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x5641028125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.697+0000 7f2502200640 1 -- 172.21.15.32:0/2325323433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 msgr2=0x5641028125e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.697+0000 7f2502200640 1 --2- 172.21.15.32:0/2325323433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x5641028125e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.697+0000 7f2502200640 1 -- 172.21.15.32:0/2325323433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 msgr2=0x5641028120a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.697+0000 7f2502200640 1 --2- 172.21.15.32:0/2325323433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028120a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.697+0000 7f2502200640 1 -- 172.21.15.32:0/2325323433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641024d7650 con 0x56410246c070 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.697+0000 7f2501800640 1 --2- 172.21.15.32:0/2325323433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.697+0000 7f2502200640 1 --2- 172.21.15.32:0/2325323433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x564102811b60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009fd0 tx=0x7f24f40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954c34f2a721ca48 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.697+0000 7f24eea00640 1 -- 172.21.15.32:0/2325323433 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4004d20 con 0x56410246c070 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.697+0000 7f24eea00640 1 -- 172.21.15.32:0/2325323433 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f4004050 con 0x56410246c070 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.697+0000 7f24eea00640 1 -- 172.21.15.32:0/2325323433 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4005280 con 0x56410246c070 2024-08-31T07:35:17.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.698+0000 7f24fa000640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f24e40b4ca0 0x7f24e40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.698+0000 7f24fa000640 1 -- 172.21.15.32:0/2787321395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f24f0105900 con 0x7f24fc103730 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.698+0000 7f2501800640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f24e40b4ca0 0x7f24e40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.698+0000 7f250ae4f740 1 -- 172.21.15.32:0/2325323433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 msgr2=0x564102811b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.698+0000 7f250ae4f740 1 --2- 172.21.15.32:0/2325323433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x564102811b60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009fd0 tx=0x7f24f40118d0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.698+0000 7f250ae4f740 1 -- 172.21.15.32:0/2325323433 shutdown_connections 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.698+0000 7f250ae4f740 1 --2- 172.21.15.32:0/2325323433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x5641028125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.698+0000 7f250ae4f740 1 --2- 172.21.15.32:0/2325323433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.698+0000 7f250ae4f740 1 --2- 172.21.15.32:0/2325323433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x564102811b60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.698+0000 7f250ae4f740 1 -- 172.21.15.32:0/2325323433 >> 172.21.15.32:0/2325323433 conn(0x564102460460 msgr2=0x56410280fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.699+0000 7f250ae4f740 1 -- 172.21.15.32:0/2325323433 shutdown_connections 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.699+0000 7f250ae4f740 1 -- 172.21.15.32:0/2325323433 wait complete. 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.699+0000 7f2501800640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f24e40b4ca0 0x7f24e40b71b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x5641028107e0 tx=0x7f24f010b530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:17.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.699+0000 7f250ae4f740 1 -- start start 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.699+0000 7f250ae4f740 1 Processor -- start 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x56410281d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x56410281dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x5641028296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f250ae4f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5641025da700 con 0x56410246c070 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f250ae4f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5641026fecf0 con 0x5641024431b0 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f250ae4f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564102820340 con 0x56410244b390 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f250ae4f740 10 client.0 _ll_register_callbacks cb 0x564102724470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f2502200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x5641028296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f2500e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x56410281dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f2500e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x56410281dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52618/0 (socket says 172.21.15.32:52618) 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f2500e00640 1 -- 172.21.15.32:0/3291683971 learned_addr learned my addr 172.21.15.32:0/3291683971 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:17.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.700+0000 7f2502200640 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x56410281d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.701+0000 7f2500e00640 1 -- 172.21.15.32:0/3291683971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 msgr2=0x56410281d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.701+0000 7f2500e00640 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x56410281d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.701+0000 7f2500e00640 1 -- 172.21.15.32:0/3291683971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 msgr2=0x5641028296c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.701+0000 7f2500e00640 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x5641028296c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.701+0000 7f2500e00640 1 -- 172.21.15.32:0/3291683971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56410273f8b0 con 0x56410244b390 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.701+0000 7f2500e00640 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x56410281dc50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f24e8012d20 tx=0x7f24e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67f8f145f16e4321 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.701+0000 7f24e3400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.701+0000 7f24e3400640 1 -- 172.21.15.32:0/3291683971 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8012110 con 0x56410244b390 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.702+0000 7f250ae4f740 1 -- 172.21.15.32:0/3291683971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f4004050 con 0x56410244b390 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.702+0000 7f24e3400640 1 -- 172.21.15.32:0/3291683971 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f24e8004050 con 0x56410244b390 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.702+0000 7f24e3400640 1 -- 172.21.15.32:0/3291683971 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f24e80122b0 con 0x56410244b390 2024-08-31T07:35:17.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.702+0000 7f250ae4f740 10 client.29719 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.702+0000 7f250ae4f740 1 -- 172.21.15.32:0/3291683971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x56410282c3e0 con 0x56410244b390 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.702+0000 7f24e3400640 1 -- 172.21.15.32:0/3291683971 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f4004050 con 0x56410244b390 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.702+0000 7f24e3400640 1 -- 172.21.15.32:0/3291683971 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f24e8010430 con 0x56410244b390 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.702+0000 7f250ae4f740 10 client.29719 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.702+0000 7f250ae4f740 10 client.29719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.703+0000 7f250ae4f740 20 client.29719 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.703+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56410242a9f0 0x56410282fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.703+0000 7f250ae4f740 4 client.29719 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.703+0000 7f250ae4f740 1 -- 172.21.15.32:0/3291683971 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56410278a700 con 0x56410242a9f0 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.703+0000 7f2502200640 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56410242a9f0 0x56410282fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.703+0000 7f2502200640 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56410242a9f0 0x56410282fa50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f178684e6a4492c server_cookie=58fe718beda9df0 in_seq=0 out_seq=0 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.703+0000 7f24e3400640 10 client.29719 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.704+0000 7f24e3400640 1 -- 172.21.15.32:0/3291683971 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56410278a700 con 0x56410242a9f0 2024-08-31T07:35:17.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.704+0000 7f24e3400640 10 client.29719 handle_command_reply: tid=0 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.704+0000 7f250ae4f740 1 client.29719 shutdown 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.704+0000 7f250ae4f740 1 -- 172.21.15.32:0/3291683971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 msgr2=0x56410281dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.704+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x56410281dc50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f24e8012d20 tx=0x7f24e800f790 comp rx=0 tx=0).stop 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.704+0000 7f250ae4f740 1 -- 172.21.15.32:0/3291683971 shutdown_connections 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.704+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x5641028296c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.704+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x56410281dc50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.704+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x56410242a9f0 0x56410282fa50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.705+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3291683971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x56410281d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.705+0000 7f24e3400640 0 client.29719 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.705+0000 7f250ae4f740 1 -- 172.21.15.32:0/3291683971 >> 172.21.15.32:0/3291683971 conn(0x564102460460 msgr2=0x564102814cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.705+0000 7f250ae4f740 1 -- 172.21.15.32:0/3291683971 shutdown_connections 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.705+0000 7f250ae4f740 1 -- 172.21.15.32:0/3291683971 wait complete. 2024-08-31T07:35:17.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.705+0000 7f250ae4f740 20 client.29719 trim_cache size 0 max 16384 2024-08-31T07:35:17.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.719+0000 7f250ae4f740 1 Processor -- start 2024-08-31T07:35:17.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.719+0000 7f250ae4f740 1 -- start start 2024-08-31T07:35:17.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.719+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.719+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410244b390 0x564102829b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.719+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410246c070 0x56410281a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.719+0000 7f250ae4f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc1b1480 con 0x56410244b390 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.719+0000 7f250ae4f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc1b31a0 con 0x5641024431b0 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f250ae4f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc1b1c10 con 0x56410246c070 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2500e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410244b390 0x564102829b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2500e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410244b390 0x564102829b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42440/0 (socket says 172.21.15.32:42440) 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2500e00640 1 -- 172.21.15.32:0/2539279487 learned_addr learned my addr 172.21.15.32:0/2539279487 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2502200640 1 --2- 172.21.15.32:0/2539279487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2502200640 1 --2- 172.21.15.32:0/2539279487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410246c070 0x56410281a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2500e00640 1 -- 172.21.15.32:0/2539279487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410246c070 msgr2=0x56410281a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2500e00640 1 --2- 172.21.15.32:0/2539279487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410246c070 0x56410281a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2500e00640 1 -- 172.21.15.32:0/2539279487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 msgr2=0x5641028295c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:17.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2500e00640 1 --2- 172.21.15.32:0/2539279487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028295c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2500e00640 1 -- 172.21.15.32:0/2539279487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56410282a040 con 0x56410244b390 2024-08-31T07:35:17.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.720+0000 7f2502200640 1 --2- 172.21.15.32:0/2539279487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028295c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:17.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.721+0000 7f2500e00640 1 --2- 172.21.15.32:0/2539279487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410244b390 0x564102829b00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f24e80108a0 tx=0x7f24e800fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46b43c6eab3e750 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.721+0000 7f24ee000640 1 -- 172.21.15.32:0/2539279487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8029070 con 0x56410244b390 2024-08-31T07:35:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.721+0000 7f24ee000640 1 -- 172.21.15.32:0/2539279487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e8010c50 con 0x56410244b390 2024-08-31T07:35:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.721+0000 7f24ee000640 1 -- 172.21.15.32:0/2539279487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8003830 con 0x56410244b390 2024-08-31T07:35:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.722+0000 7f250ae4f740 1 -- 172.21.15.32:0/2539279487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410244b390 msgr2=0x564102829b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.722+0000 7f250ae4f740 1 --2- 172.21.15.32:0/2539279487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410244b390 0x564102829b00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f24e80108a0 tx=0x7f24e800fe80 comp rx=0 tx=0).stop 2024-08-31T07:35:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.722+0000 7f250ae4f740 1 -- 172.21.15.32:0/2539279487 shutdown_connections 2024-08-31T07:35:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.722+0000 7f250ae4f740 1 --2- 172.21.15.32:0/2539279487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410246c070 0x56410281a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.722+0000 7f250ae4f740 1 --2- 172.21.15.32:0/2539279487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410244b390 0x564102829b00 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.722+0000 7f250ae4f740 1 --2- 172.21.15.32:0/2539279487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028295c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.722+0000 7f250ae4f740 1 -- 172.21.15.32:0/2539279487 >> 172.21.15.32:0/2539279487 conn(0x564102460460 msgr2=0x56410281bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.722+0000 7f250ae4f740 1 -- 172.21.15.32:0/2539279487 shutdown_connections 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.722+0000 7f250ae4f740 1 -- 172.21.15.32:0/2539279487 wait complete. 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.722+0000 7f250ae4f740 1 -- start start 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.723+0000 7f250ae4f740 1 Processor -- start 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.723+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.723+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x564102821f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.723+0000 7f250ae4f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x5641028993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.723+0000 7f250ae4f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc1b2ad0 con 0x56410246c070 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.723+0000 7f250ae4f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc1b2340 con 0x5641024431b0 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.723+0000 7f250ae4f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc1af480 con 0x56410244b390 2024-08-31T07:35:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.723+0000 7f250ae4f740 10 client.0 _ll_register_callbacks cb 0x7f24e8012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2500e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x564102821f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2502200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x5641028993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2500e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x564102821f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52626/0 (socket says 172.21.15.32:52626) 2024-08-31T07:35:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2500e00640 1 -- 172.21.15.32:0/3020785896 learned_addr learned my addr 172.21.15.32:0/3020785896 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2502200640 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2500e00640 1 -- 172.21.15.32:0/3020785896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 msgr2=0x5641028219e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2500e00640 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028219e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2500e00640 1 -- 172.21.15.32:0/3020785896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 msgr2=0x5641028993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2500e00640 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x5641028993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2500e00640 1 -- 172.21.15.32:0/3020785896 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564102899930 con 0x56410244b390 2024-08-31T07:35:17.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f2500e00640 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x564102821f20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f24e800f7d0 tx=0x7f24e801dba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=645461a4ae5d1635 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:17.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.724+0000 7f24e3400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:35:17.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.725+0000 7f24e3400640 1 -- 172.21.15.32:0/3020785896 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8029040 con 0x56410244b390 2024-08-31T07:35:17.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.725+0000 7f24e3400640 1 -- 172.21.15.32:0/3020785896 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f24e8018050 con 0x56410244b390 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.725+0000 7f250ae4f740 1 -- 172.21.15.32:0/3020785896 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e8010c50 con 0x56410244b390 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.725+0000 7f24e3400640 1 -- 172.21.15.32:0/3020785896 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f24e80050e0 con 0x56410244b390 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.725+0000 7f250ae4f740 10 client.29725 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.725+0000 7f250ae4f740 1 -- 172.21.15.32:0/3020785896 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x56410289aa00 con 0x56410244b390 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.725+0000 7f24e3400640 1 -- 172.21.15.32:0/3020785896 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e8010c50 con 0x56410244b390 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.726+0000 7f24e3400640 1 -- 172.21.15.32:0/3020785896 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f24e803e030 con 0x56410244b390 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.726+0000 7f250ae4f740 10 client.29725 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.726+0000 7f250ae4f740 10 client.29725 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.726+0000 7f250ae4f740 20 client.29725 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.726+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5641023ffda0 0x56410289d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:17.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.726+0000 7f250ae4f740 4 client.29725 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.726+0000 7f250ae4f740 1 -- 172.21.15.32:0/3020785896 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641027861a0 con 0x5641023ffda0 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.726+0000 7f2502200640 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5641023ffda0 0x56410289d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f2502200640 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5641023ffda0 0x56410289d190 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73868d94bf59ead1 server_cookie=7d5298bf7e0beea6 in_seq=0 out_seq=0 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f24e3400640 10 client.29725 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f24e3400640 1 -- 172.21.15.32:0/3020785896 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641027861a0 con 0x5641023ffda0 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f24e3400640 10 client.29725 handle_command_reply: tid=0 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 client.29725 shutdown 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 -- 172.21.15.32:0/3020785896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 msgr2=0x564102821f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x564102821f20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f24e800f7d0 tx=0x7f24e801dba0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 -- 172.21.15.32:0/3020785896 shutdown_connections 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56410246c070 0x5641028993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56410244b390 0x564102821f20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x5641023ffda0 0x56410289d190 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 --2- 172.21.15.32:0/3020785896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641024431b0 0x5641028219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 -- 172.21.15.32:0/3020785896 >> 172.21.15.32:0/3020785896 conn(0x564102460460 msgr2=0x564102812b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:17.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f24e3400640 0 client.29725 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:35:17.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 -- 172.21.15.32:0/3020785896 shutdown_connections 2024-08-31T07:35:17.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.727+0000 7f250ae4f740 1 -- 172.21.15.32:0/3020785896 wait complete. 2024-08-31T07:35:17.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f250ae4f740 20 client.29725 trim_cache size 0 max 16384 2024-08-31T07:35:17.732 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:35:17.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 -- 172.21.15.32:0/2787321395 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f24e40b4ca0 msgr2=0x7f24e40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:17.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f24e40b4ca0 0x7f24e40b71b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x5641028107e0 tx=0x7f24f010b530 comp rx=0 tx=0).stop 2024-08-31T07:35:17.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 -- 172.21.15.32:0/2787321395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24fc103730 msgr2=0x7f24fc19c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:17.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24fc103730 0x7f24fc19c2d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f24f0005ab0 tx=0x7f24f0003c10 comp rx=0 tx=0).stop 2024-08-31T07:35:17.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 -- 172.21.15.32:0/2787321395 shutdown_connections 2024-08-31T07:35:17.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f24e40b4ca0 0x7f24e40b71b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24fc1099c0 0x7f24fc1a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24fc109070 0x7f24fc19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 --2- 172.21.15.32:0/2787321395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24fc103730 0x7f24fc19c2d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:17.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 -- 172.21.15.32:0/2787321395 >> 172.21.15.32:0/2787321395 conn(0x7f24fc0ff180 msgr2=0x7f24fc108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:17.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.728+0000 7f2502c00640 1 -- 172.21.15.32:0/2787321395 shutdown_connections 2024-08-31T07:35:17.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:17.729+0000 7f2502c00640 1 -- 172.21.15.32:0/2787321395 wait complete. 2024-08-31T07:35:18.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.84 MB/sec execute 418 sec latency 14338.616 ms 2024-08-31T07:35:18.621 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:18 smithi032 ceph-mon[17252]: pgmap v1077: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 976 KiB/s wr, 135 op/s 2024-08-31T07:35:18.662 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:35:18.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:18 smithi084 ceph-mon[22387]: pgmap v1077: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 976 KiB/s wr, 135 op/s 2024-08-31T07:35:18.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:18 smithi073 ceph-mon[21995]: pgmap v1077: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 976 KiB/s wr, 135 op/s 2024-08-31T07:35:19.040 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:19.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.81 MB/sec execute 419 sec latency 15338.664 ms 2024-08-31T07:35:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.757+0000 7f2f11800640 1 -- 172.21.15.32:0/658525265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c075e20 msgr2=0x7f2f0c10f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.757+0000 7f2f11800640 1 --2- 172.21.15.32:0/658525265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c075e20 0x7f2f0c10f9a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009900 tx=0x7f2ef402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.757+0000 7f2f11800640 1 -- 172.21.15.32:0/658525265 shutdown_connections 2024-08-31T07:35:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.757+0000 7f2f11800640 1 --2- 172.21.15.32:0/658525265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c10fee0 0x7f2f0c112300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.757+0000 7f2f11800640 1 --2- 172.21.15.32:0/658525265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c075e20 0x7f2f0c10f9a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.757+0000 7f2f11800640 1 --2- 172.21.15.32:0/658525265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c0754d0 0x7f2f0c0758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.758+0000 7f2f11800640 1 -- 172.21.15.32:0/658525265 >> 172.21.15.32:0/658525265 conn(0x7f2f0c07b5f0 msgr2=0x7f2f0c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.758+0000 7f2f11800640 1 -- 172.21.15.32:0/658525265 shutdown_connections 2024-08-31T07:35:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.758+0000 7f2f11800640 1 -- 172.21.15.32:0/658525265 wait complete. 2024-08-31T07:35:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.759+0000 7f2f11800640 1 Processor -- start 2024-08-31T07:35:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.759+0000 7f2f11800640 1 -- start start 2024-08-31T07:35:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f11800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c0754d0 0x7f2f0c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f11800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c075e20 0x7f2f0c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f11800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c10fee0 0x7f2f0c1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f11800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c1159e0 con 0x7f2f0c075e20 2024-08-31T07:35:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f11800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c115860 con 0x7f2f0c10fee0 2024-08-31T07:35:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f11800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c115b60 con 0x7f2f0c0754d0 2024-08-31T07:35:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c075e20 0x7f2f0c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f0aa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c0754d0 0x7f2f0c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c075e20 0x7f2f0c19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42464/0 (socket says 172.21.15.32:42464) 2024-08-31T07:35:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f03e00640 1 -- 172.21.15.32:0/7651970 learned_addr learned my addr 172.21.15.32:0/7651970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.760+0000 7f2f0b400640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c10fee0 0x7f2f0c1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.761+0000 7f2f0aa00640 1 -- 172.21.15.32:0/7651970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c10fee0 msgr2=0x7f2f0c1a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.761+0000 7f2f0aa00640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c10fee0 0x7f2f0c1a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.761+0000 7f2f0aa00640 1 -- 172.21.15.32:0/7651970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c075e20 msgr2=0x7f2f0c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.761+0000 7f2f0aa00640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c075e20 0x7f2f0c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.761+0000 7f2f0aa00640 1 -- 172.21.15.32:0/7651970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c1a2400 con 0x7f2f0c0754d0 2024-08-31T07:35:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.761+0000 7f2f03e00640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c075e20 0x7f2f0c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.761+0000 7f2f0aa00640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c0754d0 0x7f2f0c19c490 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00e9f0 tx=0x7f2efc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.762+0000 7f2f03400640 1 -- 172.21.15.32:0/7651970 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00ccc0 con 0x7f2f0c0754d0 2024-08-31T07:35:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.762+0000 7f2f11800640 1 -- 172.21.15.32:0/7651970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0c1a26f0 con 0x7f2f0c0754d0 2024-08-31T07:35:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.762+0000 7f2f11800640 1 -- 172.21.15.32:0/7651970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0c1a8e50 con 0x7f2f0c0754d0 2024-08-31T07:35:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.762+0000 7f2f03400640 1 -- 172.21.15.32:0/7651970 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2efc0108e0 con 0x7f2f0c0754d0 2024-08-31T07:35:19.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.762+0000 7f2f03400640 1 -- 172.21.15.32:0/7651970 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc018750 con 0x7f2f0c0754d0 2024-08-31T07:35:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.765+0000 7f2f18499740 1 Processor -- start 2024-08-31T07:35:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.765+0000 7f2f18499740 1 -- start start 2024-08-31T07:35:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.765+0000 7f2f18499740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 0x557b71a27990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.765+0000 7f2f18499740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 0x557b71a27ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.765+0000 7f2f18499740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a28410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.765+0000 7f2f18499740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557b7199e7b0 con 0x557b71682070 2024-08-31T07:35:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.765+0000 7f2f18499740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557b7198dd40 con 0x557b716591b0 2024-08-31T07:35:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.765+0000 7f2f18499740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557b71991cf0 con 0x557b71661390 2024-08-31T07:35:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.766+0000 7f2f03400640 1 -- 172.21.15.32:0/7651970 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2efc0188f0 con 0x7f2f0c0754d0 2024-08-31T07:35:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.766+0000 7f2f0b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 0x557b71a27990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.766+0000 7f2f0b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 0x557b71a27990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42468/0 (socket says 172.21.15.32:42468) 2024-08-31T07:35:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.766+0000 7f2f0b400640 1 -- 172.21.15.32:0/3352972875 learned_addr learned my addr 172.21.15.32:0/3352972875 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.766+0000 7f2f03e00640 1 --2- 172.21.15.32:0/3352972875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a28410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.766+0000 7f2f0aa00640 1 --2- 172.21.15.32:0/3352972875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 0x557b71a27ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.766+0000 7f2f0b400640 1 -- 172.21.15.32:0/3352972875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 msgr2=0x557b71a28410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.766+0000 7f2f0b400640 1 --2- 172.21.15.32:0/3352972875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a28410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.766+0000 7f2f0b400640 1 -- 172.21.15.32:0/3352972875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 msgr2=0x557b71a27ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.766+0000 7f2f0b400640 1 --2- 172.21.15.32:0/3352972875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 0x557b71a27ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.767+0000 7f2f0b400640 1 -- 172.21.15.32:0/3352972875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b716ed650 con 0x557b71682070 2024-08-31T07:35:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.767+0000 7f2f0b400640 1 --2- 172.21.15.32:0/3352972875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 0x557b71a27990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009f50 tx=0x7f2f040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536afbf75bddd835 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.767+0000 7f2efbe00640 1 -- 172.21.15.32:0/3352972875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04004d20 con 0x557b71682070 2024-08-31T07:35:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.767+0000 7f2efbe00640 1 -- 172.21.15.32:0/3352972875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f04004050 con 0x557b71682070 2024-08-31T07:35:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.767+0000 7f2efbe00640 1 -- 172.21.15.32:0/3352972875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04005280 con 0x557b71682070 2024-08-31T07:35:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.768+0000 7f2f03400640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4d20 0x7f2ef00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.768+0000 7f2f03400640 1 -- 172.21.15.32:0/7651970 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f2efc0e5800 con 0x7f2f0c0754d0 2024-08-31T07:35:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.768+0000 7f2f0aa00640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4d20 0x7f2ef00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.768+0000 7f2f18499740 1 -- 172.21.15.32:0/3352972875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 msgr2=0x557b71a27990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.768+0000 7f2f18499740 1 --2- 172.21.15.32:0/3352972875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 0x557b71a27990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009f50 tx=0x7f2f040118d0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.769+0000 7f2f18499740 1 -- 172.21.15.32:0/3352972875 shutdown_connections 2024-08-31T07:35:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.769+0000 7f2f18499740 1 --2- 172.21.15.32:0/3352972875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a28410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.769+0000 7f2f18499740 1 --2- 172.21.15.32:0/3352972875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 0x557b71a27ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.769+0000 7f2f18499740 1 --2- 172.21.15.32:0/3352972875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 0x557b71a27990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.769+0000 7f2f18499740 1 -- 172.21.15.32:0/3352972875 >> 172.21.15.32:0/3352972875 conn(0x557b71676460 msgr2=0x557b71a25880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.769+0000 7f2f18499740 1 -- 172.21.15.32:0/3352972875 shutdown_connections 2024-08-31T07:35:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.769+0000 7f2f0aa00640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4d20 0x7f2ef00b7230 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x557b71a26610 tx=0x7f2efc0eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.769+0000 7f2f18499740 1 -- 172.21.15.32:0/3352972875 wait complete. 2024-08-31T07:35:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.769+0000 7f2f18499740 1 -- start start 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.770+0000 7f2f18499740 1 Processor -- start 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.770+0000 7f2f18499740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 0x557b71a335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.770+0000 7f2f18499740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a33b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.770+0000 7f2f18499740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 0x557b71a3f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.770+0000 7f2f18499740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557b717f0700 con 0x557b71682070 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.770+0000 7f2f18499740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557b71914cf0 con 0x557b716591b0 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.770+0000 7f2f18499740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557b71a36170 con 0x557b71661390 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.770+0000 7f2f18499740 10 client.0 _ll_register_callbacks cb 0x557b7193a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f0b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 0x557b71a335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f0b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 0x557b71a3f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a33b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a33b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52660/0 (socket says 172.21.15.32:52660) 2024-08-31T07:35:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f03e00640 1 -- 172.21.15.32:0/936081073 learned_addr learned my addr 172.21.15.32:0/936081073 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f03e00640 1 -- 172.21.15.32:0/936081073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 msgr2=0x557b71a335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f03e00640 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 0x557b71a335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f03e00640 1 -- 172.21.15.32:0/936081073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 msgr2=0x557b71a3f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f03e00640 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 0x557b71a3f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f03e00640 1 -- 172.21.15.32:0/936081073 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b719558b0 con 0x557b71661390 2024-08-31T07:35:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.771+0000 7f2f0b400640 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 0x557b71a335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.772+0000 7f2f03e00640 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a33b30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4006010 tx=0x7f2ef4031740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb8d12cb47741a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.772+0000 7f2ef8c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:35:19.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.772+0000 7f2ef8c00640 1 -- 172.21.15.32:0/936081073 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4035040 con 0x557b71661390 2024-08-31T07:35:19.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.772+0000 7f2f18499740 1 -- 172.21.15.32:0/936081073 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f04004050 con 0x557b71661390 2024-08-31T07:35:19.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.772+0000 7f2ef8c00640 1 -- 172.21.15.32:0/936081073 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f2ef4004450 con 0x557b71661390 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.772+0000 7f2ef8c00640 1 -- 172.21.15.32:0/936081073 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f2ef4005020 con 0x557b71661390 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.772+0000 7f2f18499740 10 client.29737 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.772+0000 7f2f18499740 1 -- 172.21.15.32:0/936081073 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x557b71a42290 con 0x557b71661390 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.772+0000 7f2ef8c00640 1 -- 172.21.15.32:0/936081073 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f04004050 con 0x557b71661390 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.773+0000 7f2ef8c00640 1 -- 172.21.15.32:0/936081073 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f2ef40035f0 con 0x557b71661390 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.773+0000 7f2f18499740 10 client.29737 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.773+0000 7f2f18499740 10 client.29737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.773+0000 7f2f18499740 20 client.29737 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.773+0000 7f2f18499740 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b716409f0 0x557b71a45750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.773+0000 7f2f18499740 4 client.29737 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.773+0000 7f2f18499740 1 -- 172.21.15.32:0/936081073 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b719a0700 con 0x557b716409f0 2024-08-31T07:35:19.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.773+0000 7f2f0b400640 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b716409f0 0x557b71a45750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.774+0000 7f2f0b400640 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b716409f0 0x557b71a45750 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4430a25c9982a8 server_cookie=c029ff6189d60c49 in_seq=0 out_seq=0 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.774+0000 7f2ef8c00640 10 client.29737 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2ef8c00640 1 -- 172.21.15.32:0/936081073 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b719a0700 con 0x557b716409f0 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2ef8c00640 10 client.29737 handle_command_reply: tid=0 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 client.29737 shutdown 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 -- 172.21.15.32:0/936081073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 msgr2=0x557b71a33b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a33b30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4006010 tx=0x7f2ef4031740 comp rx=0 tx=0).stop 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 -- 172.21.15.32:0/936081073 shutdown_connections 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b716409f0 0x557b71a45750 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b71682070 0x557b71a3f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a33b30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 --2- 172.21.15.32:0/936081073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b716591b0 0x557b71a335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2ef8c00640 0 client.29737 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 -- 172.21.15.32:0/936081073 >> 172.21.15.32:0/936081073 conn(0x557b71676460 msgr2=0x557b71a2ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 -- 172.21.15.32:0/936081073 shutdown_connections 2024-08-31T07:35:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 1 -- 172.21.15.32:0/936081073 wait complete. 2024-08-31T07:35:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.775+0000 7f2f18499740 20 client.29737 trim_cache size 0 max 16384 2024-08-31T07:35:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.789+0000 7f2f18499740 1 Processor -- start 2024-08-31T07:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f18499740 1 -- start start 2024-08-31T07:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f18499740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 0x557b71a2d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f18499740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a3f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f18499740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71682070 0x557b71a3f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f18499740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800e740 con 0x557b716591b0 2024-08-31T07:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f18499740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8010460 con 0x557b71682070 2024-08-31T07:35:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f18499740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800eed0 con 0x557b71661390 2024-08-31T07:35:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a3f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f0b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 0x557b71a2d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a3f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52664/0 (socket says 172.21.15.32:52664) 2024-08-31T07:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.790+0000 7f2f03e00640 1 -- 172.21.15.32:0/2119160082 learned_addr learned my addr 172.21.15.32:0/2119160082 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.791+0000 7f2f0b400640 1 --2- 172.21.15.32:0/2119160082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71682070 0x557b71a3f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.791+0000 7f2f03e00640 1 -- 172.21.15.32:0/2119160082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71682070 msgr2=0x557b71a3f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.791+0000 7f2f03e00640 1 --2- 172.21.15.32:0/2119160082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71682070 0x557b71a3f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.791+0000 7f2f03e00640 1 -- 172.21.15.32:0/2119160082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 msgr2=0x557b71a2d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.791+0000 7f2f03e00640 1 --2- 172.21.15.32:0/2119160082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 0x557b71a2d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.791+0000 7f2f03e00640 1 -- 172.21.15.32:0/2119160082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b71a29c80 con 0x557b71661390 2024-08-31T07:35:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.791+0000 7f2f0b400640 1 --2- 172.21.15.32:0/2119160082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71682070 0x557b71a3f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:35:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.791+0000 7f2f03e00640 1 --2- 172.21.15.32:0/2119160082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a3f3f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4000c00 tx=0x7f2ef40314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f54618bbc514572 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.792+0000 7f2efb400640 1 -- 172.21.15.32:0/2119160082 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4035030 con 0x557b71661390 2024-08-31T07:35:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.792+0000 7f2efb400640 1 -- 172.21.15.32:0/2119160082 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef40035f0 con 0x557b71661390 2024-08-31T07:35:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.792+0000 7f2efb400640 1 -- 172.21.15.32:0/2119160082 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4054040 con 0x557b71661390 2024-08-31T07:35:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.792+0000 7f2f18499740 1 -- 172.21.15.32:0/2119160082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 msgr2=0x557b71a3f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.792+0000 7f2f18499740 1 --2- 172.21.15.32:0/2119160082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a3f3f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4000c00 tx=0x7f2ef40314a0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.793+0000 7f2f18499740 1 -- 172.21.15.32:0/2119160082 shutdown_connections 2024-08-31T07:35:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.793+0000 7f2f18499740 1 --2- 172.21.15.32:0/2119160082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71682070 0x557b71a3f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.793+0000 7f2f18499740 1 --2- 172.21.15.32:0/2119160082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71661390 0x557b71a3f3f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.793+0000 7f2f18499740 1 --2- 172.21.15.32:0/2119160082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 0x557b71a2d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.793+0000 7f2f18499740 1 -- 172.21.15.32:0/2119160082 >> 172.21.15.32:0/2119160082 conn(0x557b71676460 msgr2=0x557b71a2caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.793+0000 7f2f18499740 1 -- 172.21.15.32:0/2119160082 shutdown_connections 2024-08-31T07:35:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.793+0000 7f2f18499740 1 -- 172.21.15.32:0/2119160082 wait complete. 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.793+0000 7f2f18499740 1 -- start start 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.793+0000 7f2f18499740 1 Processor -- start 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f18499740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 0x557b71a37860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f18499740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71661390 0x557b71a37e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f18499740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71682070 0x557b71aaf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f18499740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800fd90 con 0x557b716591b0 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f18499740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800f600 con 0x557b71661390 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f18499740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800c740 con 0x557b71682070 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f18499740 10 client.0 _ll_register_callbacks cb 0x7f2ef4005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f0b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 0x557b71a37860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f0b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 0x557b71a37860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42496/0 (socket says 172.21.15.32:42496) 2024-08-31T07:35:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f0b400640 1 -- 172.21.15.32:0/2118298954 learned_addr learned my addr 172.21.15.32:0/2118298954 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f03e00640 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71661390 0x557b71a37e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.794+0000 7f2f0b400640 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71682070 0x557b71aaf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.795+0000 7f2f03e00640 1 -- 172.21.15.32:0/2118298954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71682070 msgr2=0x557b71aaf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.795+0000 7f2f03e00640 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71682070 0x557b71aaf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.795+0000 7f2f03e00640 1 -- 172.21.15.32:0/2118298954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 msgr2=0x557b71a37860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.795+0000 7f2f03e00640 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 0x557b71a37860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.795+0000 7f2f03e00640 1 -- 172.21.15.32:0/2118298954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b71aaf6c0 con 0x557b71661390 2024-08-31T07:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.795+0000 7f2f03e00640 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71661390 0x557b71a37e00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40039c0 tx=0x7f2ef4031740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ae22d38c2903f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.795+0000 7f2ef8c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:35:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.796+0000 7f2ef8c00640 1 -- 172.21.15.32:0/2118298954 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4035030 con 0x557b71661390 2024-08-31T07:35:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.796+0000 7f2ef8c00640 1 -- 172.21.15.32:0/2118298954 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8485+0+0 (secure 0 0 0) 0x7f2ef40053c0 con 0x557b71661390 2024-08-31T07:35:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.796+0000 7f2f18499740 1 -- 172.21.15.32:0/2118298954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef40035f0 con 0x557b71661390 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.796+0000 7f2ef8c00640 1 -- 172.21.15.32:0/2118298954 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f2ef4054040 con 0x557b71661390 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.796+0000 7f2f18499740 10 client.29735 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.796+0000 7f2f18499740 1 -- 172.21.15.32:0/2118298954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x557b71ab07b0 con 0x557b71661390 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.796+0000 7f2ef8c00640 1 -- 172.21.15.32:0/2118298954 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef40035f0 con 0x557b71661390 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.797+0000 7f2ef8c00640 1 -- 172.21.15.32:0/2118298954 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f2ef40615f0 con 0x557b71661390 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.797+0000 7f2f18499740 10 client.29735 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.797+0000 7f2f18499740 10 client.29735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.797+0000 7f2f18499740 20 client.29735 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.797+0000 7f2f18499740 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b71615da0 0x557b71ab2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.797+0000 7f2f18499740 4 client.29735 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.797+0000 7f2f18499740 1 -- 172.21.15.32:0/2118298954 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b7199c1a0 con 0x557b71615da0 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.797+0000 7f2f0b400640 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b71615da0 0x557b71ab2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.797+0000 7f2f0b400640 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b71615da0 0x557b71ab2e50 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3581d08ce5672b2 server_cookie=eeab1af28c43a88c in_seq=0 out_seq=0 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.797+0000 7f2ef8c00640 10 client.29735 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2ef8c00640 1 -- 172.21.15.32:0/2118298954 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b7199c1a0 con 0x557b71615da0 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2ef8c00640 10 client.29735 handle_command_reply: tid=0 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 client.29735 shutdown 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 -- 172.21.15.32:0/2118298954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71661390 msgr2=0x557b71a37e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71661390 0x557b71a37e00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40039c0 tx=0x7f2ef4031740 comp rx=0 tx=0).stop 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 -- 172.21.15.32:0/2118298954 shutdown_connections 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b71615da0 0x557b71ab2e50 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b71682070 0x557b71aaf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b71661390 0x557b71a37e00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 --2- 172.21.15.32:0/2118298954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b716591b0 0x557b71a37860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2ef8c00640 0 client.29735 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 -- 172.21.15.32:0/2118298954 >> 172.21.15.32:0/2118298954 conn(0x557b71676460 msgr2=0x557b71a289c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 -- 172.21.15.32:0/2118298954 shutdown_connections 2024-08-31T07:35:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 1 -- 172.21.15.32:0/2118298954 wait complete. 2024-08-31T07:35:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.798+0000 7f2f18499740 20 client.29735 trim_cache size 0 max 16384 2024-08-31T07:35:19.803 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:35:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.799+0000 7f2f11800640 1 -- 172.21.15.32:0/7651970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4d20 msgr2=0x7f2ef00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.799+0000 7f2f11800640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4d20 0x7f2ef00b7230 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x557b71a26610 tx=0x7f2efc0eb530 comp rx=0 tx=0).stop 2024-08-31T07:35:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.799+0000 7f2f11800640 1 -- 172.21.15.32:0/7651970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c0754d0 msgr2=0x7f2f0c19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.799+0000 7f2f11800640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c0754d0 0x7f2f0c19c490 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00e9f0 tx=0x7f2efc00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.799+0000 7f2f11800640 1 -- 172.21.15.32:0/7651970 shutdown_connections 2024-08-31T07:35:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.799+0000 7f2f11800640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f2ef00b4d20 0x7f2ef00b7230 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.799+0000 7f2f11800640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f0c10fee0 0x7f2f0c1a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.799+0000 7f2f11800640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f0c075e20 0x7f2f0c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.799+0000 7f2f11800640 1 --2- 172.21.15.32:0/7651970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f0c0754d0 0x7f2f0c19c490 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.799+0000 7f2f11800640 1 -- 172.21.15.32:0/7651970 >> 172.21.15.32:0/7651970 conn(0x7f2f0c07b5f0 msgr2=0x7f2f0c1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.800+0000 7f2f11800640 1 -- 172.21.15.32:0/7651970 shutdown_connections 2024-08-31T07:35:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:19.800+0000 7f2f11800640 1 -- 172.21.15.32:0/7651970 wait complete. 2024-08-31T07:35:20.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.77 MB/sec execute 420 sec latency 16338.721 ms 2024-08-31T07:35:20.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:20 smithi032 ceph-mon[17252]: pgmap v1078: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 671 KiB/s wr, 117 op/s 2024-08-31T07:35:20.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:20 smithi073 ceph-mon[21995]: pgmap v1078: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 671 KiB/s wr, 117 op/s 2024-08-31T07:35:20.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:20 smithi084 ceph-mon[22387]: pgmap v1078: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 671 KiB/s wr, 117 op/s 2024-08-31T07:35:21.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.74 MB/sec execute 421 sec latency 17338.840 ms 2024-08-31T07:35:21.781 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:35:21.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:35:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:22.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.71 MB/sec execute 422 sec latency 18338.952 ms 2024-08-31T07:35:22.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:22 smithi032 ceph-mon[17252]: pgmap v1079: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 622 KiB/s wr, 69 op/s 2024-08-31T07:35:22.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:22 smithi032 ceph-mon[17252]: osdmap e114: 12 total, 12 up, 12 in 2024-08-31T07:35:22.794 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:22 smithi084 ceph-mon[22387]: pgmap v1079: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 622 KiB/s wr, 69 op/s 2024-08-31T07:35:22.794 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:22 smithi084 ceph-mon[22387]: osdmap e114: 12 total, 12 up, 12 in 2024-08-31T07:35:22.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.887+0000 7f2097e00640 1 -- 172.21.15.32:0/2061256467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f209810b290 msgr2=0x7f209810b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.887+0000 7f2097e00640 1 --2- 172.21.15.32:0/2061256467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f209810b290 0x7f209810b6a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2080009900 tx=0x7f208002a410 comp rx=0 tx=0).stop 2024-08-31T07:35:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.887+0000 7f2097e00640 1 -- 172.21.15.32:0/2061256467 shutdown_connections 2024-08-31T07:35:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.887+0000 7f2097e00640 1 --2- 172.21.15.32:0/2061256467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f209810bbe0 0x7f2098112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.887+0000 7f2097e00640 1 --2- 172.21.15.32:0/2061256467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f209810b290 0x7f209810b6a0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.887+0000 7f2097e00640 1 --2- 172.21.15.32:0/2061256467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2098105950 0x7f2098105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.887+0000 7f2097e00640 1 -- 172.21.15.32:0/2061256467 >> 172.21.15.32:0/2061256467 conn(0x7f2098101420 msgr2=0x7f2098103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.887+0000 7f2097e00640 1 -- 172.21.15.32:0/2061256467 shutdown_connections 2024-08-31T07:35:22.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.887+0000 7f2097e00640 1 -- 172.21.15.32:0/2061256467 wait complete. 2024-08-31T07:35:22.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.888+0000 7f2097e00640 1 Processor -- start 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.888+0000 7f2097e00640 1 -- start start 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f2097e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098105950 0x7f209819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f2097e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209810b290 0x7f209819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f2097e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f209810bbe0 0x7f20981a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f2097e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20981158a0 con 0x7f209810bbe0 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f2097e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2098115720 con 0x7f2098105950 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f2097e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2098115a20 con 0x7f209810b290 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f208fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209810b290 0x7f209819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f208fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209810b290 0x7f209819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52692/0 (socket says 172.21.15.32:52692) 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f208fe00640 1 -- 172.21.15.32:0/1159750796 learned_addr learned my addr 172.21.15.32:0/1159750796 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:22.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f2097400640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f209810bbe0 0x7f20981a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.889+0000 7f2096a00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098105950 0x7f209819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f208fe00640 1 -- 172.21.15.32:0/1159750796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098105950 msgr2=0x7f209819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f208fe00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098105950 0x7f209819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f208fe00640 1 -- 172.21.15.32:0/1159750796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f209810bbe0 msgr2=0x7f20981a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f208fe00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f209810bbe0 0x7f20981a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f208fe00640 1 -- 172.21.15.32:0/1159750796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20981a2220 con 0x7f209810b290 2024-08-31T07:35:22.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f208fe00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209810b290 0x7f209819c850 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f208002a930 tx=0x7f2080031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f208f400640 1 -- 172.21.15.32:0/1159750796 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208003f030 con 0x7f209810b290 2024-08-31T07:35:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f2097e00640 1 -- 172.21.15.32:0/1159750796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20981a24b0 con 0x7f209810b290 2024-08-31T07:35:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f208f400640 1 -- 172.21.15.32:0/1159750796 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2080004490 con 0x7f209810b290 2024-08-31T07:35:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f208f400640 1 -- 172.21.15.32:0/1159750796 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080035070 con 0x7f209810b290 2024-08-31T07:35:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.890+0000 7f2097e00640 1 -- 172.21.15.32:0/1159750796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20981a8c70 con 0x7f209810b290 2024-08-31T07:35:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.894+0000 7f20a445a740 1 Processor -- start 2024-08-31T07:35:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.894+0000 7f20a445a740 1 -- start start 2024-08-31T07:35:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.895+0000 7f20a445a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 0x564ddb19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.895+0000 7f20a445a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 0x564ddb19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.895+0000 7f20a445a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd8670 0x564ddb1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.895+0000 7f20a445a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564ddb115750 con 0x564ddadf9350 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.895+0000 7f20a445a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564ddb104ce0 con 0x564ddadd0490 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.895+0000 7f20a445a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564ddb108c90 con 0x564ddadd8670 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.895+0000 7f2097400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 0x564ddb19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.895+0000 7f2097400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 0x564ddb19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42514/0 (socket says 172.21.15.32:42514) 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.895+0000 7f2097400640 1 -- 172.21.15.32:0/1050353812 learned_addr learned my addr 172.21.15.32:0/1050353812 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.895+0000 7f2096a00640 1 --2- 172.21.15.32:0/1050353812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 0x564ddb19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.896+0000 7f2097400640 1 -- 172.21.15.32:0/1050353812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd8670 msgr2=0x564ddb1a0250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.896+0000 7f2097400640 1 --2- 172.21.15.32:0/1050353812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd8670 0x564ddb1a0250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.896+0000 7f2097400640 1 -- 172.21.15.32:0/1050353812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 msgr2=0x564ddb19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.896+0000 7f2097400640 1 --2- 172.21.15.32:0/1050353812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 0x564ddb19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.896+0000 7f2097400640 1 -- 172.21.15.32:0/1050353812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ddae64930 con 0x564ddadf9350 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.896+0000 7f2096a00640 1 --2- 172.21.15.32:0/1050353812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 0x564ddb19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.896+0000 7f208f400640 1 -- 172.21.15.32:0/1159750796 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2080042030 con 0x7f209810b290 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.896+0000 7f2097400640 1 --2- 172.21.15.32:0/1050353812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 0x564ddb19f7d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2090009fd0 tx=0x7f20900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4eac80db04068f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.897+0000 7f2087e00640 1 -- 172.21.15.32:0/1050353812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090004d20 con 0x564ddadf9350 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.897+0000 7f2087e00640 1 -- 172.21.15.32:0/1050353812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090004050 con 0x564ddadf9350 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.897+0000 7f2087e00640 1 -- 172.21.15.32:0/1050353812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090005220 con 0x564ddadf9350 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.897+0000 7f20a445a740 1 -- 172.21.15.32:0/1050353812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 msgr2=0x564ddb19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.897+0000 7f20a445a740 1 --2- 172.21.15.32:0/1050353812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 0x564ddb19f7d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2090009fd0 tx=0x7f20900118d0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f20a445a740 1 -- 172.21.15.32:0/1050353812 shutdown_connections 2024-08-31T07:35:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f20a445a740 1 --2- 172.21.15.32:0/1050353812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd8670 0x564ddb1a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f20a445a740 1 --2- 172.21.15.32:0/1050353812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 0x564ddb19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f20a445a740 1 --2- 172.21.15.32:0/1050353812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 0x564ddb19f7d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f20a445a740 1 -- 172.21.15.32:0/1050353812 >> 172.21.15.32:0/1050353812 conn(0x564ddaded740 msgr2=0x564ddb19d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f20a445a740 1 -- 172.21.15.32:0/1050353812 shutdown_connections 2024-08-31T07:35:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f20a445a740 1 -- 172.21.15.32:0/1050353812 wait complete. 2024-08-31T07:35:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f208f400640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f207c0b4c30 0x7f207c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f208f400640 1 -- 172.21.15.32:0/1159750796 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f20801057f0 con 0x7f209810b290 2024-08-31T07:35:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f2096a00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f207c0b4c30 0x7f207c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.898+0000 7f20a445a740 1 -- start start 2024-08-31T07:35:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.899+0000 7f20a445a740 1 Processor -- start 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.899+0000 7f2096a00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f207c0b4c30 0x7f207c0b7140 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x564ddb19e450 tx=0x7f2088005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.899+0000 7f20a445a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 0x564ddb1ab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.899+0000 7f20a445a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1ab940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.899+0000 7f20a445a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadf9350 0x564ddb1b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.899+0000 7f20a445a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564ddaf67890 con 0x564ddadd8670 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.899+0000 7f20a445a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564ddb08bcd0 con 0x564ddadd0490 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.899+0000 7f20a445a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564ddb1adfb0 con 0x564ddadf9350 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.899+0000 7f20a445a740 10 client.0 _ll_register_callbacks cb 0x564ddb0ffef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f208fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f2096a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 0x564ddb1ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f208fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1ab940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42524/0 (socket says 172.21.15.32:42524) 2024-08-31T07:35:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f208fe00640 1 -- 172.21.15.32:0/2054211664 learned_addr learned my addr 172.21.15.32:0/2054211664 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f2097400640 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadf9350 0x564ddb1b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f208fe00640 1 -- 172.21.15.32:0/2054211664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadf9350 msgr2=0x564ddb1b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f208fe00640 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadf9350 0x564ddb1b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f208fe00640 1 -- 172.21.15.32:0/2054211664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 msgr2=0x564ddb1ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f208fe00640 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 0x564ddb1ab400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f208fe00640 1 -- 172.21.15.32:0/2054211664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ddb0d51b0 con 0x564ddadd8670 2024-08-31T07:35:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f2096a00640 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 0x564ddb1ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f208fe00640 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1ab940 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f20800cbc80 tx=0x7f20800cb990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68fea0c4a618a1a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.900+0000 7f2084c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:35:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.901+0000 7f2084c00640 1 -- 172.21.15.32:0/2054211664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080049070 con 0x564ddadd8670 2024-08-31T07:35:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.901+0000 7f20a445a740 1 -- 172.21.15.32:0/2054211664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2090004050 con 0x564ddadd8670 2024-08-31T07:35:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.901+0000 7f2084c00640 1 -- 172.21.15.32:0/2054211664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f20800b71c0 con 0x564ddadd8670 2024-08-31T07:35:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.901+0000 7f2084c00640 1 -- 172.21.15.32:0/2054211664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f20800cbf80 con 0x564ddadd8670 2024-08-31T07:35:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.901+0000 7f20a445a740 10 client.19710 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.901+0000 7f20a445a740 1 -- 172.21.15.32:0/2054211664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x564ddb1ba000 con 0x564ddadd8670 2024-08-31T07:35:22.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.901+0000 7f2084c00640 1 -- 172.21.15.32:0/2054211664 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090004050 con 0x564ddadd8670 2024-08-31T07:35:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.901+0000 7f2084c00640 1 -- 172.21.15.32:0/2054211664 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f208003ba30 con 0x564ddadd8670 2024-08-31T07:35:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.901+0000 7f20a445a740 10 client.19710 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.901+0000 7f20a445a740 10 client.19710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:35:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.902+0000 7f20a445a740 20 client.19710 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.902+0000 7f20a445a740 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ddadb7cd0 0x564ddb1bd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.902+0000 7f20a445a740 4 client.19710 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.902+0000 7f20a445a740 1 -- 172.21.15.32:0/2054211664 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ddb1176a0 con 0x564ddadb7cd0 2024-08-31T07:35:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.902+0000 7f2096a00640 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ddadb7cd0 0x564ddb1bd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.903+0000 7f2096a00640 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ddadb7cd0 0x564ddb1bd450 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a6a23d1b30768b server_cookie=80c74b5a352d25a7 in_seq=0 out_seq=0 2024-08-31T07:35:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.903+0000 7f2084c00640 10 client.19710 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.903+0000 7f2084c00640 1 -- 172.21.15.32:0/2054211664 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ddb1176a0 con 0x564ddadb7cd0 2024-08-31T07:35:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.903+0000 7f2084c00640 10 client.19710 handle_command_reply: tid=0 2024-08-31T07:35:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.903+0000 7f20a445a740 1 client.19710 shutdown 2024-08-31T07:35:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 1 -- 172.21.15.32:0/2054211664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 msgr2=0x564ddb1ab940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1ab940 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f20800cbc80 tx=0x7f20800cb990 comp rx=0 tx=0).stop 2024-08-31T07:35:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 1 -- 172.21.15.32:0/2054211664 shutdown_connections 2024-08-31T07:35:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ddadb7cd0 0x564ddb1bd450 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadf9350 0x564ddb1b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1ab940 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 1 --2- 172.21.15.32:0/2054211664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd0490 0x564ddb1ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 1 -- 172.21.15.32:0/2054211664 >> 172.21.15.32:0/2054211664 conn(0x564ddaded740 msgr2=0x564ddb1a29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f2084c00640 0 client.19710 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:22.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 1 -- 172.21.15.32:0/2054211664 shutdown_connections 2024-08-31T07:35:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 1 -- 172.21.15.32:0/2054211664 wait complete. 2024-08-31T07:35:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.904+0000 7f20a445a740 20 client.19710 trim_cache size 0 max 16384 2024-08-31T07:35:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.924+0000 7f20a445a740 1 Processor -- start 2024-08-31T07:35:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.924+0000 7f20a445a740 1 -- start start 2024-08-31T07:35:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.924+0000 7f20a445a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 0x564ddb1a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.924+0000 7f20a445a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd8670 0x564ddb1a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.924+0000 7f20a445a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 0x564ddb1a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.924+0000 7f20a445a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b13c0 con 0x564ddadf9350 2024-08-31T07:35:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.924+0000 7f20a445a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b30e0 con 0x564ddadd8670 2024-08-31T07:35:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.924+0000 7f20a445a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b1b50 con 0x564ddadd0490 2024-08-31T07:35:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f208fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd8670 0x564ddb1a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f2096a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 0x564ddb1a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f208fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd8670 0x564ddb1a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50040/0 (socket says 172.21.15.32:50040) 2024-08-31T07:35:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f208fe00640 1 -- 172.21.15.32:0/1483553600 learned_addr learned my addr 172.21.15.32:0/1483553600 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f2097400640 1 --2- 172.21.15.32:0/1483553600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 0x564ddb1a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f2096a00640 1 -- 172.21.15.32:0/1483553600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd8670 msgr2=0x564ddb1a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f2096a00640 1 --2- 172.21.15.32:0/1483553600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd8670 0x564ddb1a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f2096a00640 1 -- 172.21.15.32:0/1483553600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 msgr2=0x564ddb1a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f2096a00640 1 --2- 172.21.15.32:0/1483553600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 0x564ddb1a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f2096a00640 1 -- 172.21.15.32:0/1483553600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ddb1a6a90 con 0x564ddadd0490 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f208fe00640 1 --2- 172.21.15.32:0/1483553600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd8670 0x564ddb1a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.925+0000 7f2096a00640 1 --2- 172.21.15.32:0/1483553600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 0x564ddb1a48b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x564ddb1bdb80 tx=0x7f208801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a8fb9fa79bb06c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.926+0000 7f2087400640 1 -- 172.21.15.32:0/1483553600 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088003e80 con 0x564ddadd0490 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.926+0000 7f2087400640 1 -- 172.21.15.32:0/1483553600 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208802a910 con 0x564ddadd0490 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.926+0000 7f2087400640 1 -- 172.21.15.32:0/1483553600 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208802b820 con 0x564ddadd0490 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.926+0000 7f20a445a740 1 -- 172.21.15.32:0/1483553600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 msgr2=0x564ddb1a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.926+0000 7f20a445a740 1 --2- 172.21.15.32:0/1483553600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 0x564ddb1a48b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x564ddb1bdb80 tx=0x7f208801a000 comp rx=0 tx=0).stop 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.926+0000 7f20a445a740 1 -- 172.21.15.32:0/1483553600 shutdown_connections 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.926+0000 7f20a445a740 1 --2- 172.21.15.32:0/1483553600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadf9350 0x564ddb1a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.927+0000 7f20a445a740 1 --2- 172.21.15.32:0/1483553600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadd8670 0x564ddb1a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.927+0000 7f20a445a740 1 --2- 172.21.15.32:0/1483553600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 0x564ddb1a48b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.927+0000 7f20a445a740 1 -- 172.21.15.32:0/1483553600 >> 172.21.15.32:0/1483553600 conn(0x564ddaded740 msgr2=0x564ddb1a9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.927+0000 7f20a445a740 1 -- 172.21.15.32:0/1483553600 shutdown_connections 2024-08-31T07:35:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.927+0000 7f20a445a740 1 -- 172.21.15.32:0/1483553600 wait complete. 2024-08-31T07:35:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.927+0000 7f20a445a740 1 -- start start 2024-08-31T07:35:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.927+0000 7f20a445a740 1 Processor -- start 2024-08-31T07:35:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f20a445a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 0x564ddb1af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f20a445a740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f20a445a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadf9350 0x564ddb226e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f20a445a740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b2a10 con 0x564ddadd8670 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f20a445a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b2280 con 0x564ddadf9350 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f20a445a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f20981af3c0 con 0x564ddadd0490 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f20a445a740 10 client.0 _ll_register_callbacks cb 0x7f2080005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f208fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f2096a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 0x564ddb1af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f208fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42542/0 (socket says 172.21.15.32:42542) 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f208fe00640 1 -- 172.21.15.32:0/3505475409 learned_addr learned my addr 172.21.15.32:0/3505475409 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.928+0000 7f2097400640 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadf9350 0x564ddb226e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.929+0000 7f2096a00640 1 -- 172.21.15.32:0/3505475409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadf9350 msgr2=0x564ddb226e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.929+0000 7f2096a00640 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadf9350 0x564ddb226e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.929+0000 7f2096a00640 1 -- 172.21.15.32:0/3505475409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 msgr2=0x564ddb1afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.929+0000 7f2096a00640 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.929+0000 7f2096a00640 1 -- 172.21.15.32:0/3505475409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ddb227380 con 0x564ddadd0490 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.929+0000 7f2096a00640 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 0x564ddb1af960 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f208800a010 tx=0x7f208802b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72a27a36c78336bb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.929+0000 7f2084c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.929+0000 7f2084c00640 1 -- 172.21.15.32:0/3505475409 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088040060 con 0x564ddadd0490 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.929+0000 7f2084c00640 1 -- 172.21.15.32:0/3505475409 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f208802a660 con 0x564ddadd0490 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.929+0000 7f20a445a740 1 -- 172.21.15.32:0/3505475409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f208802a910 con 0x564ddadd0490 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f2084c00640 1 -- 172.21.15.32:0/3505475409 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f208802aac0 con 0x564ddadd0490 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f20a445a740 10 client.29758 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f20a445a740 1 -- 172.21.15.32:0/3505475409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x564ddb228640 con 0x564ddadd0490 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f2084c00640 1 -- 172.21.15.32:0/3505475409 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208802a910 con 0x564ddadd0490 2024-08-31T07:35:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f2084c00640 1 -- 172.21.15.32:0/3505475409 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f208803a030 con 0x564ddadd0490 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f20a445a740 10 client.29758 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f20a445a740 10 client.29758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f20a445a740 20 client.29758 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f20a445a740 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ddad8d080 0x564ddb22adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f20a445a740 4 client.29758 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.930+0000 7f20a445a740 1 -- 172.21.15.32:0/3505475409 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564ddb113140 con 0x564ddad8d080 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.931+0000 7f208fe00640 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ddad8d080 0x564ddb22adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.931+0000 7f208fe00640 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ddad8d080 0x564ddb22adc0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efdeb4a9ccd6ff53 server_cookie=20b6779b53f388a8 in_seq=0 out_seq=0 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.931+0000 7f2084c00640 10 client.29758 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f2084c00640 1 -- 172.21.15.32:0/3505475409 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564ddb113140 con 0x564ddad8d080 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f2084c00640 10 client.29758 handle_command_reply: tid=0 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f20a445a740 1 client.29758 shutdown 2024-08-31T07:35:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f20a445a740 1 -- 172.21.15.32:0/3505475409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 msgr2=0x564ddb1af960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f20a445a740 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 0x564ddb1af960 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f208800a010 tx=0x7f208802b430 comp rx=0 tx=0).stop 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f20a445a740 1 -- 172.21.15.32:0/3505475409 shutdown_connections 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f20a445a740 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x564ddad8d080 0x564ddb22adc0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f20a445a740 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564ddadf9350 0x564ddb226e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f20a445a740 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564ddadd8670 0x564ddb1afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f20a445a740 1 --2- 172.21.15.32:0/3505475409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564ddadd0490 0x564ddb1af960 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f2084c00640 0 client.29758 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.933+0000 7f20a445a740 1 -- 172.21.15.32:0/3505475409 >> 172.21.15.32:0/3505475409 conn(0x564ddaded740 msgr2=0x564ddb1a0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.934+0000 7f20a445a740 1 -- 172.21.15.32:0/3505475409 shutdown_connections 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.934+0000 7f20a445a740 1 -- 172.21.15.32:0/3505475409 wait complete. 2024-08-31T07:35:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.934+0000 7f20a445a740 20 client.29758 trim_cache size 0 max 16384 2024-08-31T07:35:22.947 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:35:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:35:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "9e14a084-7459-442a-abc0-3ac74b83528a", 2024-08-31T07:35:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:35:22.947 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.934+0000 7f2097e00640 1 -- 172.21.15.32:0/1159750796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f207c0b4c30 msgr2=0x7f207c0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.934+0000 7f2097e00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f207c0b4c30 0x7f207c0b7140 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x564ddb19e450 tx=0x7f2088005f90 comp rx=0 tx=0).stop 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.934+0000 7f2097e00640 1 -- 172.21.15.32:0/1159750796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209810b290 msgr2=0x7f209819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.934+0000 7f2097e00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209810b290 0x7f209819c850 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f208002a930 tx=0x7f2080031a20 comp rx=0 tx=0).stop 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.934+0000 7f2097e00640 1 -- 172.21.15.32:0/1159750796 shutdown_connections 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.935+0000 7f2097e00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f207c0b4c30 0x7f207c0b7140 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.935+0000 7f2097e00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f209810bbe0 0x7f20981a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.935+0000 7f2097e00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209810b290 0x7f209819c850 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.935+0000 7f2097e00640 1 --2- 172.21.15.32:0/1159750796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2098105950 0x7f209819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.935+0000 7f2097e00640 1 -- 172.21.15.32:0/1159750796 >> 172.21.15.32:0/1159750796 conn(0x7f2098101420 msgr2=0x7f2098103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.935+0000 7f2097e00640 1 -- 172.21.15.32:0/1159750796 shutdown_connections 2024-08-31T07:35:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:22.935+0000 7f2097e00640 1 -- 172.21.15.32:0/1159750796 wait complete. 2024-08-31T07:35:22.949 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:22 smithi073 ceph-mon[21995]: pgmap v1079: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 622 KiB/s wr, 69 op/s 2024-08-31T07:35:22.949 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:22 smithi073 ceph-mon[21995]: osdmap e114: 12 total, 12 up, 12 in 2024-08-31T07:35:23.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.68 MB/sec execute 423 sec latency 19339.097 ms 2024-08-31T07:35:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:23 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:35:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:23 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:23 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:35:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:23 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:35:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:23 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:35:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:23 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:23 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:23 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:23 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:35:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:23 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:23 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:35:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:23 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:35:23.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:23 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:35:23.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:23 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:23 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:23 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:23 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:35:23.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:23 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:23 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:35:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:23 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:35:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:23 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:35:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:23 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:23 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:23.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:23 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:24.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:35:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:24.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.64 MB/sec execute 424 sec latency 20339.152 ms 2024-08-31T07:35:24.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:24 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:35:24.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:24 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:35:24.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:24 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:24.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:24 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:24.770 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:24 smithi032 ceph-mon[17252]: pgmap v1081: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 540 KiB/s wr, 51 op/s 2024-08-31T07:35:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:24 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:35:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:24 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:35:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:24 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:24 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:24.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:24 smithi073 ceph-mon[21995]: pgmap v1081: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 540 KiB/s wr, 51 op/s 2024-08-31T07:35:24.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:24 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:35:24.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:24 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:35:24.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:24 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:24.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:24 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:35:24.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:24 smithi084 ceph-mon[22387]: pgmap v1081: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 540 KiB/s wr, 51 op/s 2024-08-31T07:35:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.069+0000 7f2205000640 1 -- 172.21.15.32:0/831749216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220010bdb0 msgr2=0x7f22001125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.069+0000 7f2205000640 1 --2- 172.21.15.32:0/831749216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220010bdb0 0x7f22001125e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009ca0 tx=0x7f21f802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.069+0000 7f2205000640 1 -- 172.21.15.32:0/831749216 shutdown_connections 2024-08-31T07:35:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.069+0000 7f2205000640 1 --2- 172.21.15.32:0/831749216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220010bdb0 0x7f22001125e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.069+0000 7f2205000640 1 --2- 172.21.15.32:0/831749216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220010b460 0x7f220010b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.069+0000 7f2205000640 1 --2- 172.21.15.32:0/831749216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2200105b20 0x7f2200105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.069+0000 7f2205000640 1 -- 172.21.15.32:0/831749216 >> 172.21.15.32:0/831749216 conn(0x7f22001015b0 msgr2=0x7f2200103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.069+0000 7f2205000640 1 -- 172.21.15.32:0/831749216 shutdown_connections 2024-08-31T07:35:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.070+0000 7f2205000640 1 -- 172.21.15.32:0/831749216 wait complete. 2024-08-31T07:35:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.071+0000 7f2205000640 1 Processor -- start 2024-08-31T07:35:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.071+0000 7f2205000640 1 -- start start 2024-08-31T07:35:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.071+0000 7f2205000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2200105b20 0x7f220019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.071+0000 7f2205000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220010b460 0x7f220019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.071+0000 7f2205000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f220010bdb0 0x7f22001a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.071+0000 7f2205000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2200115a70 con 0x7f2200105b20 2024-08-31T07:35:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.071+0000 7f2205000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f22001158f0 con 0x7f220010bdb0 2024-08-31T07:35:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.071+0000 7f2205000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2200115bf0 con 0x7f220010b460 2024-08-31T07:35:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.071+0000 7f21fde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2200105b20 0x7f220019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2200105b20 0x7f220019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42556/0 (socket says 172.21.15.32:42556) 2024-08-31T07:35:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fde00640 1 -- 172.21.15.32:0/1637825180 learned_addr learned my addr 172.21.15.32:0/1637825180 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fd400640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220010b460 0x7f220019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fe800640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f220010bdb0 0x7f22001a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fde00640 1 -- 172.21.15.32:0/1637825180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220010b460 msgr2=0x7f220019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fde00640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220010b460 0x7f220019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fde00640 1 -- 172.21.15.32:0/1637825180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f220010bdb0 msgr2=0x7f22001a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fde00640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f220010bdb0 0x7f22001a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fde00640 1 -- 172.21.15.32:0/1637825180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22001a2410 con 0x7f2200105b20 2024-08-31T07:35:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fe800640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f220010bdb0 0x7f22001a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.072+0000 7f21fde00640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2200105b20 0x7f220019c4a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f21f000e9f0 tx=0x7f21f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.073+0000 7f21f6a00640 1 -- 172.21.15.32:0/1637825180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f000cd80 con 0x7f2200105b20 2024-08-31T07:35:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.073+0000 7f21f6a00640 1 -- 172.21.15.32:0/1637825180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f00108e0 con 0x7f2200105b20 2024-08-31T07:35:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.073+0000 7f2205000640 1 -- 172.21.15.32:0/1637825180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22001a2700 con 0x7f2200105b20 2024-08-31T07:35:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.073+0000 7f21f6a00640 1 -- 172.21.15.32:0/1637825180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0018980 con 0x7f2200105b20 2024-08-31T07:35:25.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.073+0000 7f2205000640 1 -- 172.21.15.32:0/1637825180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22001a8e60 con 0x7f2200105b20 2024-08-31T07:35:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.076+0000 7f220bc34740 1 Processor -- start 2024-08-31T07:35:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.076+0000 7f220bc34740 1 -- start start 2024-08-31T07:35:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f220bc34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e1a0a070 0x5633e1daf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f21f6a00640 1 -- 172.21.15.32:0/1637825180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f21f0010430 con 0x7f2200105b20 2024-08-31T07:35:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f220bc34740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 0x5633e1dafdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f220bc34740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f220bc34740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5633e1d26770 con 0x5633e19e9390 2024-08-31T07:35:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f220bc34740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5633e1d15d00 con 0x5633e19e11b0 2024-08-31T07:35:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f220bc34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5633e1d19cb0 con 0x5633e1a0a070 2024-08-31T07:35:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f21fd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f21fde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 0x5633e1dafdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f21fd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42570/0 (socket says 172.21.15.32:42570) 2024-08-31T07:35:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f21fd400640 1 -- 172.21.15.32:0/1772404556 learned_addr learned my addr 172.21.15.32:0/1772404556 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.077+0000 7f21fe800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e1a0a070 0x5633e1daf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21fd400640 1 -- 172.21.15.32:0/1772404556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e1a0a070 msgr2=0x5633e1daf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21fd400640 1 --2- 172.21.15.32:0/1772404556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e1a0a070 0x5633e1daf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21fd400640 1 -- 172.21.15.32:0/1772404556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 msgr2=0x5633e1dafdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21fd400640 1 --2- 172.21.15.32:0/1772404556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 0x5633e1dafdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21fd400640 1 -- 172.21.15.32:0/1772404556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e1a75650 con 0x5633e19e9390 2024-08-31T07:35:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21fde00640 1 --2- 172.21.15.32:0/1772404556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 0x5633e1dafdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21fd400640 1 --2- 172.21.15.32:0/1772404556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db02f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009ba0 tx=0x7f21ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb38e7d5bd185318 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21eb400640 1 -- 172.21.15.32:0/1772404556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec004230 con 0x5633e19e9390 2024-08-31T07:35:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21eb400640 1 -- 172.21.15.32:0/1772404556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21ec009e10 con 0x5633e19e9390 2024-08-31T07:35:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21eb400640 1 -- 172.21.15.32:0/1772404556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec004bd0 con 0x5633e19e9390 2024-08-31T07:35:25.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.078+0000 7f21f6a00640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f21e40b4ca0 0x7f21e40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.079+0000 7f21f6a00640 1 -- 172.21.15.32:0/1637825180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f21f00e5800 con 0x7f2200105b20 2024-08-31T07:35:25.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.079+0000 7f21fe800640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f21e40b4ca0 0x7f21e40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.079+0000 7f220bc34740 1 -- 172.21.15.32:0/1772404556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 msgr2=0x5633e1db02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:25.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.079+0000 7f220bc34740 1 --2- 172.21.15.32:0/1772404556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db02f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009ba0 tx=0x7f21ec011810 comp rx=0 tx=0).stop 2024-08-31T07:35:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.080+0000 7f220bc34740 1 -- 172.21.15.32:0/1772404556 shutdown_connections 2024-08-31T07:35:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.080+0000 7f220bc34740 1 --2- 172.21.15.32:0/1772404556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db02f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.080+0000 7f220bc34740 1 --2- 172.21.15.32:0/1772404556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 0x5633e1dafdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.080+0000 7f220bc34740 1 --2- 172.21.15.32:0/1772404556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e1a0a070 0x5633e1daf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.080+0000 7f220bc34740 1 -- 172.21.15.32:0/1772404556 >> 172.21.15.32:0/1772404556 conn(0x5633e19fe460 msgr2=0x5633e1dad760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.080+0000 7f220bc34740 1 -- 172.21.15.32:0/1772404556 shutdown_connections 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.080+0000 7f21fe800640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f21e40b4ca0 0x7f21e40b71b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f21f8006010 tx=0x7f21f8032ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.080+0000 7f220bc34740 1 -- 172.21.15.32:0/1772404556 wait complete. 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.080+0000 7f220bc34740 1 -- start start 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f220bc34740 1 Processor -- start 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f220bc34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dbb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f220bc34740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dbb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f220bc34740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 0x5633e1dc7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f220bc34740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5633e1b78700 con 0x5633e19e9390 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f220bc34740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5633e1c9ccb0 con 0x5633e1a0a070 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f220bc34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5633e1dbe050 con 0x5633e19e11b0 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f220bc34740 10 client.0 _ll_register_callbacks cb 0x5633e1cc2430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f21fd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dbb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f21fd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dbb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42584/0 (socket says 172.21.15.32:42584) 2024-08-31T07:35:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f21fd400640 1 -- 172.21.15.32:0/182781299 learned_addr learned my addr 172.21.15.32:0/182781299 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.081+0000 7f21fe800640 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 0x5633e1dc7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.082+0000 7f21fde00640 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dbb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.082+0000 7f21fde00640 1 -- 172.21.15.32:0/182781299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 msgr2=0x5633e1dc7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.082+0000 7f21fde00640 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 0x5633e1dc7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.082+0000 7f21fde00640 1 -- 172.21.15.32:0/182781299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 msgr2=0x5633e1dbb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.082+0000 7f21fde00640 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dbb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.082+0000 7f21fde00640 1 -- 172.21.15.32:0/182781299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e1cdd870 con 0x5633e19e11b0 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.082+0000 7f21fd400640 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dbb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.082+0000 7f21fde00640 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dbb370 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f21f00e2fa0 tx=0x7f21f000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d8797e24c0f88a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.082+0000 7f21dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.082+0000 7f21dfe00640 1 -- 172.21.15.32:0/182781299 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f00e8e60 con 0x5633e19e11b0 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f21dfe00640 1 -- 172.21.15.32:0/182781299 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f21f00ab250 con 0x5633e19e11b0 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f220bc34740 1 -- 172.21.15.32:0/182781299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ec009e10 con 0x5633e19e11b0 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f21dfe00640 1 -- 172.21.15.32:0/182781299 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f21f00ab760 con 0x5633e19e11b0 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f220bc34740 10 client.29770 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f220bc34740 1 -- 172.21.15.32:0/182781299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x5633e1dca010 con 0x5633e19e11b0 2024-08-31T07:35:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f21dfe00640 1 -- 172.21.15.32:0/182781299 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21ec009e10 con 0x5633e19e11b0 2024-08-31T07:35:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f21dfe00640 1 -- 172.21.15.32:0/182781299 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f21f00b03d0 con 0x5633e19e11b0 2024-08-31T07:35:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f220bc34740 10 client.29770 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f220bc34740 10 client.29770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:35:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f220bc34740 20 client.29770 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f220bc34740 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5633e19c89f0 0x5633e1dcd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f220bc34740 4 client.29770 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.083+0000 7f220bc34740 1 -- 172.21.15.32:0/182781299 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633e1d286c0 con 0x5633e19c89f0 2024-08-31T07:35:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.084+0000 7f21fd400640 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5633e19c89f0 0x5633e1dcd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.084+0000 7f21fd400640 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5633e19c89f0 0x5633e1dcd560 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ff3fad4161b12a server_cookie=ac43fdf7969fcdef in_seq=0 out_seq=0 2024-08-31T07:35:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.084+0000 7f21dfe00640 10 client.29770 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.085+0000 7f21dfe00640 1 -- 172.21.15.32:0/182781299 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5633e1d286c0 con 0x5633e19c89f0 2024-08-31T07:35:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.085+0000 7f21dfe00640 10 client.29770 handle_command_reply: tid=0 2024-08-31T07:35:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.085+0000 7f220bc34740 1 client.29770 shutdown 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.085+0000 7f220bc34740 1 -- 172.21.15.32:0/182781299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 msgr2=0x5633e1dbb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f220bc34740 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dbb370 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f21f00e2fa0 tx=0x7f21f000cf20 comp rx=0 tx=0).stop 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f220bc34740 1 -- 172.21.15.32:0/182781299 shutdown_connections 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f220bc34740 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5633e19c89f0 0x5633e1dcd560 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f220bc34740 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 0x5633e1dc7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f220bc34740 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dbb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f220bc34740 1 --2- 172.21.15.32:0/182781299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dbb370 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f220bc34740 1 -- 172.21.15.32:0/182781299 >> 172.21.15.32:0/182781299 conn(0x5633e19fe460 msgr2=0x5633e1db2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f21dfe00640 0 client.29770 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f220bc34740 1 -- 172.21.15.32:0/182781299 shutdown_connections 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f220bc34740 1 -- 172.21.15.32:0/182781299 wait complete. 2024-08-31T07:35:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.086+0000 7f220bc34740 20 client.29770 trim_cache size 0 max 16384 2024-08-31T07:35:25.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.104+0000 7f220bc34740 1 Processor -- start 2024-08-31T07:35:25.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f220bc34740 1 -- start start 2024-08-31T07:35:25.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f220bc34740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 0x5633e1dba060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f220bc34740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f220bc34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e1a0a070 0x5633e1dbcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f220bc34740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f22001b15e0 con 0x5633e19e9390 2024-08-31T07:35:25.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f220bc34740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f22001b3300 con 0x5633e19e11b0 2024-08-31T07:35:25.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f220bc34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f22001b1d70 con 0x5633e1a0a070 2024-08-31T07:35:25.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f21fd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f21fd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42590/0 (socket says 172.21.15.32:42590) 2024-08-31T07:35:25.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f21fd400640 1 -- 172.21.15.32:0/4269287424 learned_addr learned my addr 172.21.15.32:0/4269287424 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:25.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f21fde00640 1 --2- 172.21.15.32:0/4269287424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 0x5633e1dba060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.105+0000 7f21fe800640 1 --2- 172.21.15.32:0/4269287424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e1a0a070 0x5633e1dbcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.106+0000 7f21fde00640 1 -- 172.21.15.32:0/4269287424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e1a0a070 msgr2=0x5633e1dbcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.106+0000 7f21fde00640 1 --2- 172.21.15.32:0/4269287424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e1a0a070 0x5633e1dbcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.106+0000 7f21fde00640 1 -- 172.21.15.32:0/4269287424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 msgr2=0x5633e1db5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.106+0000 7f21fde00640 1 --2- 172.21.15.32:0/4269287424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.106+0000 7f21fde00640 1 -- 172.21.15.32:0/4269287424 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e1dbc210 con 0x5633e19e11b0 2024-08-31T07:35:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.106+0000 7f21fd400640 1 --2- 172.21.15.32:0/4269287424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.106+0000 7f21fde00640 1 --2- 172.21.15.32:0/4269287424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 0x5633e1dba060 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f21f000c6a0 tx=0x7f21f00e5b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23575078656b523c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.106+0000 7f21eaa00640 1 -- 172.21.15.32:0/4269287424 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f00abb00 con 0x5633e19e11b0 2024-08-31T07:35:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.106+0000 7f21eaa00640 1 -- 172.21.15.32:0/4269287424 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f00abff0 con 0x5633e19e11b0 2024-08-31T07:35:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.106+0000 7f21eaa00640 1 -- 172.21.15.32:0/4269287424 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f00a7960 con 0x5633e19e11b0 2024-08-31T07:35:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.107+0000 7f220bc34740 1 -- 172.21.15.32:0/4269287424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 msgr2=0x5633e1dba060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.107+0000 7f220bc34740 1 --2- 172.21.15.32:0/4269287424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 0x5633e1dba060 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f21f000c6a0 tx=0x7f21f00e5b90 comp rx=0 tx=0).stop 2024-08-31T07:35:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.107+0000 7f220bc34740 1 -- 172.21.15.32:0/4269287424 shutdown_connections 2024-08-31T07:35:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.107+0000 7f220bc34740 1 --2- 172.21.15.32:0/4269287424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e1a0a070 0x5633e1dbcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.107+0000 7f220bc34740 1 --2- 172.21.15.32:0/4269287424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1db5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.107+0000 7f220bc34740 1 --2- 172.21.15.32:0/4269287424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e19e11b0 0x5633e1dba060 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.107+0000 7f220bc34740 1 -- 172.21.15.32:0/4269287424 >> 172.21.15.32:0/4269287424 conn(0x5633e19fe460 msgr2=0x5633e1dbc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.107+0000 7f220bc34740 1 -- 172.21.15.32:0/4269287424 shutdown_connections 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.107+0000 7f220bc34740 1 -- 172.21.15.32:0/4269287424 wait complete. 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.108+0000 7f220bc34740 1 -- start start 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.108+0000 7f220bc34740 1 Processor -- start 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.108+0000 7f220bc34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dc32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.108+0000 7f220bc34740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dc37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.108+0000 7f220bc34740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 0x5633e1dbe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.108+0000 7f220bc34740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f22001b2c30 con 0x5633e19e9390 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.108+0000 7f220bc34740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f22001b24a0 con 0x5633e1a0a070 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.108+0000 7f220bc34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f22001af5e0 con 0x5633e19e11b0 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.108+0000 7f220bc34740 10 client.0 _ll_register_callbacks cb 0x7f21f00a7eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fde00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dc32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dc37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dc37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42600/0 (socket says 172.21.15.32:42600) 2024-08-31T07:35:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fd400640 1 -- 172.21.15.32:0/473828524 learned_addr learned my addr 172.21.15.32:0/473828524 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fde00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dc32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:52760/0 (socket says 172.21.15.32:52760) 2024-08-31T07:35:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fe800640 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 0x5633e1dbe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fd400640 1 -- 172.21.15.32:0/473828524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 msgr2=0x5633e1dc32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fd400640 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dc32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fd400640 1 -- 172.21.15.32:0/473828524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 msgr2=0x5633e1dbe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fd400640 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 0x5633e1dbe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fd400640 1 -- 172.21.15.32:0/473828524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e1dbe7a0 con 0x5633e19e9390 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fde00640 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dc32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.109+0000 7f21fe800640 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 0x5633e1dbe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.110+0000 7f21fd400640 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dc37f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0097b0 tx=0x7f21ec004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af3d3314f8e2094 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.110+0000 7f21dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.110+0000 7f21dfe00640 1 -- 172.21.15.32:0/473828524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec01e040 con 0x5633e19e9390 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.110+0000 7f220bc34740 1 -- 172.21.15.32:0/473828524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21f00abff0 con 0x5633e19e9390 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.110+0000 7f21dfe00640 1 -- 172.21.15.32:0/473828524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f21ec0037f0 con 0x5633e19e9390 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.110+0000 7f21dfe00640 1 -- 172.21.15.32:0/473828524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f21ec03a050 con 0x5633e19e9390 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.110+0000 7f220bc34740 10 client.19734 fetch_fsmap learned FSMap version 86 2024-08-31T07:35:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.110+0000 7f220bc34740 1 -- 172.21.15.32:0/473828524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x5633e1e385c0 con 0x5633e19e9390 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.111+0000 7f21dfe00640 1 -- 172.21.15.32:0/473828524 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f00abff0 con 0x5633e19e9390 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.111+0000 7f21dfe00640 1 -- 172.21.15.32:0/473828524 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f21ec018070 con 0x5633e19e9390 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.111+0000 7f220bc34740 10 client.19734 fetch_fsmap finished waiting for FSMap version 86 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.111+0000 7f220bc34740 10 client.19734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.111+0000 7f220bc34740 20 client.19734 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.111+0000 7f220bc34740 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5633e199dda0 0x5633e1e3aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.111+0000 7f220bc34740 4 client.19734 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.111+0000 7f220bc34740 1 -- 172.21.15.32:0/473828524 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5633e1d24160 con 0x5633e199dda0 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.111+0000 7f21fde00640 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5633e199dda0 0x5633e1e3aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.112+0000 7f21fde00640 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5633e199dda0 0x5633e1e3aaf0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb10010de362ac7 server_cookie=d539f47acadf07c3 in_seq=0 out_seq=0 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.112+0000 7f21dfe00640 10 client.19734 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.112+0000 7f21dfe00640 1 -- 172.21.15.32:0/473828524 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7386 (crc 0 0 0) 0x5633e1d24160 con 0x5633e199dda0 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.112+0000 7f21dfe00640 10 client.19734 handle_command_reply: tid=0 2024-08-31T07:35:25.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.112+0000 7f220bc34740 1 client.19734 shutdown 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.112+0000 7f220bc34740 1 -- 172.21.15.32:0/473828524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 msgr2=0x5633e1dc37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.112+0000 7f220bc34740 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dc37f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0097b0 tx=0x7f21ec004500 comp rx=0 tx=0).stop 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.113+0000 7f220bc34740 1 -- 172.21.15.32:0/473828524 shutdown_connections 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.113+0000 7f220bc34740 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5633e199dda0 0x5633e1e3aaf0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.113+0000 7f220bc34740 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633e1a0a070 0x5633e1dbe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.113+0000 7f220bc34740 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5633e19e9390 0x5633e1dc37f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.113+0000 7f220bc34740 1 --2- 172.21.15.32:0/473828524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5633e19e11b0 0x5633e1dc32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.113+0000 7f220bc34740 1 -- 172.21.15.32:0/473828524 >> 172.21.15.32:0/473828524 conn(0x5633e19fe460 msgr2=0x5633e1db08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.113+0000 7f21dfe00640 0 client.19734 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.113+0000 7f220bc34740 1 -- 172.21.15.32:0/473828524 shutdown_connections 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.113+0000 7f220bc34740 1 -- 172.21.15.32:0/473828524 wait complete. 2024-08-31T07:35:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.113+0000 7f220bc34740 20 client.19734 trim_cache size 0 max 16384 2024-08-31T07:35:25.117 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:35:25.117 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:35:25.117 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:35:25.117 INFO:teuthology.orchestra.run.smithi032.stdout: "9e14a084-7459-442a-abc0-3ac74b83528a": { 2024-08-31T07:35:25.117 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "9e14a084-7459-442a-abc0-3ac74b83528a", 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: "d83ecd52-a6c2-49e8-bdaa-039848b73de4": { 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:34:26.167734+0000", 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.118 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:32:41.165270+0000", 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.119 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.120 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:35:25.121 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:35:25.121 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.121 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.121 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.121 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.121 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.121 INFO:teuthology.orchestra.run.smithi032.stdout: "9e14a084-7459-442a-abc0-3ac74b83528a": { 2024-08-31T07:35:25.121 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:35:22.931369+0000", 2024-08-31T07:35:25.121 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.122 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.122 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.122 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.122 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.122 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:35:25.122 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.123 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.125 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:35:25.126 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.128 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.129 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:35:25.130 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:35:25.131 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:25.134 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:35:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 -- 172.21.15.32:0/1637825180 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f21e40b4ca0 msgr2=0x7f21e40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f21e40b4ca0 0x7f21e40b71b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f21f8006010 tx=0x7f21f8032ed0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 -- 172.21.15.32:0/1637825180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2200105b20 msgr2=0x7f220019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2200105b20 0x7f220019c4a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f21f000e9f0 tx=0x7f21f000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 -- 172.21.15.32:0/1637825180 shutdown_connections 2024-08-31T07:35:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f21e40b4ca0 0x7f21e40b71b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f220010bdb0 0x7f22001a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f220010b460 0x7f220019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 --2- 172.21.15.32:0/1637825180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2200105b20 0x7f220019c4a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 -- 172.21.15.32:0/1637825180 >> 172.21.15.32:0/1637825180 conn(0x7f22001015b0 msgr2=0x7f220010aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 -- 172.21.15.32:0/1637825180 shutdown_connections 2024-08-31T07:35:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:25.114+0000 7f2205000640 1 -- 172.21.15.32:0/1637825180 wait complete. 2024-08-31T07:35:25.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.61 MB/sec execute 425 sec latency 21339.195 ms 2024-08-31T07:35:26.045 INFO:tasks.cephfs.filesystem:scrub status for tag:9e14a084-7459-442a-abc0-3ac74b83528a - {'path': '/', 'tag': '9e14a084-7459-442a-abc0-3ac74b83528a', 'options': 'recursive,force'} 2024-08-31T07:35:26.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.58 MB/sec execute 426 sec latency 22339.296 ms 2024-08-31T07:35:26.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:26 smithi032 ceph-mon[17252]: pgmap v1082: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 313 KiB/s wr, 31 op/s 2024-08-31T07:35:26.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:26 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:26 smithi073 ceph-mon[21995]: pgmap v1082: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 313 KiB/s wr, 31 op/s 2024-08-31T07:35:26.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:26 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:26 smithi084 ceph-mon[22387]: pgmap v1082: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 313 KiB/s wr, 31 op/s 2024-08-31T07:35:26.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:26 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:27.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:35:27.423 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.55 MB/sec execute 427 sec latency 23339.360 ms 2024-08-31T07:35:27.747 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:28.424 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.52 MB/sec execute 428 sec latency 24339.416 ms 2024-08-31T07:35:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.538+0000 7f7142400640 1 -- 172.21.15.32:0/2547208407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c075550 msgr2=0x7f713c075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.538+0000 7f7142400640 1 --2- 172.21.15.32:0/2547208407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c075550 0x7f713c075960 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7130009900 tx=0x7f713002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:35:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.539+0000 7f7142400640 1 -- 172.21.15.32:0/2547208407 shutdown_connections 2024-08-31T07:35:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.539+0000 7f7142400640 1 --2- 172.21.15.32:0/2547208407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f713c10fcb0 0x7f713c1120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.539+0000 7f7142400640 1 --2- 172.21.15.32:0/2547208407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f713c075ea0 0x7f713c10f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.539+0000 7f7142400640 1 --2- 172.21.15.32:0/2547208407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c075550 0x7f713c075960 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.539+0000 7f7142400640 1 -- 172.21.15.32:0/2547208407 >> 172.21.15.32:0/2547208407 conn(0x7f713c0ff1e0 msgr2=0x7f713c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.539+0000 7f7142400640 1 -- 172.21.15.32:0/2547208407 shutdown_connections 2024-08-31T07:35:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.539+0000 7f7142400640 1 -- 172.21.15.32:0/2547208407 wait complete. 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.540+0000 7f7142400640 1 Processor -- start 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.540+0000 7f7142400640 1 -- start start 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.540+0000 7f7142400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f713c075550 0x7f713c19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.540+0000 7f7142400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c075ea0 0x7f713c19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.540+0000 7f7142400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f713c10fcb0 0x7f713c1a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.540+0000 7f7142400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f713c115860 con 0x7f713c075ea0 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.540+0000 7f7142400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f713c1156e0 con 0x7f713c075550 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.540+0000 7f7142400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f713c1159e0 con 0x7f713c10fcb0 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f713be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c075ea0 0x7f713c19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f713be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c075ea0 0x7f713c19c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43400/0 (socket says 172.21.15.32:43400) 2024-08-31T07:35:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f713be00640 1 -- 172.21.15.32:0/3822295573 learned_addr learned my addr 172.21.15.32:0/3822295573 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f7141000640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f713c075550 0x7f713c19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f7141a00640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f713c10fcb0 0x7f713c1a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f713be00640 1 -- 172.21.15.32:0/3822295573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f713c10fcb0 msgr2=0x7f713c1a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f713be00640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f713c10fcb0 0x7f713c1a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f713be00640 1 -- 172.21.15.32:0/3822295573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f713c075550 msgr2=0x7f713c19c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f713be00640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f713c075550 0x7f713c19c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f713be00640 1 -- 172.21.15.32:0/3822295573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713c1a21e0 con 0x7f713c075ea0 2024-08-31T07:35:28.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.541+0000 7f713be00640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c075ea0 0x7f713c19c810 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f712800e9c0 tx=0x7f712800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:28.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.542+0000 7f7139600640 1 -- 172.21.15.32:0/3822295573 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712800cde0 con 0x7f713c075ea0 2024-08-31T07:35:28.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.542+0000 7f7139600640 1 -- 172.21.15.32:0/3822295573 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71280108e0 con 0x7f713c075ea0 2024-08-31T07:35:28.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.542+0000 7f7139600640 1 -- 172.21.15.32:0/3822295573 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128018960 con 0x7f713c075ea0 2024-08-31T07:35:28.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.542+0000 7f7142400640 1 -- 172.21.15.32:0/3822295573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713c1a24d0 con 0x7f713c075ea0 2024-08-31T07:35:28.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.542+0000 7f7142400640 1 -- 172.21.15.32:0/3822295573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713c1a8c30 con 0x7f713c075ea0 2024-08-31T07:35:28.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.543+0000 7f7142400640 1 -- 172.21.15.32:0/3822295573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713c075960 con 0x7f713c075ea0 2024-08-31T07:35:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.545+0000 7f7139600640 1 -- 172.21.15.32:0/3822295573 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f712801f020 con 0x7f713c075ea0 2024-08-31T07:35:28.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.547+0000 7f7139600640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71240b4d20 0x7f71240b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:28.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.547+0000 7f7139600640 1 -- 172.21.15.32:0/3822295573 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f71280e5e20 con 0x7f713c075ea0 2024-08-31T07:35:28.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.547+0000 7f7141000640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71240b4d20 0x7f71240b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:28.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.548+0000 7f7141000640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71240b4d20 0x7f71240b7230 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7130031440 tx=0x7f7130009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:28.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.549+0000 7f7139600640 1 -- 172.21.15.32:0/3822295573 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7128026070 con 0x7f713c075ea0 2024-08-31T07:35:28.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.779+0000 7f7142400640 1 -- 172.21.15.32:0/3822295573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f713c1a8ee0 con 0x7f713c075ea0 2024-08-31T07:35:28.781 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:28 smithi032 ceph-mon[17252]: pgmap v1083: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 291 KiB/s wr, 24 op/s 2024-08-31T07:35:28.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.780+0000 7f7139600640 1 -- 172.21.15.32:0/3822295573 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+11971 (secure 0 0 0) 0x7f71280acc00 con 0x7f713c075ea0 2024-08-31T07:35:28.782 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:35:28.782 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"btime":"2024-08-31T07:35:25:510757+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":87,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:35:24.994280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:35:28.782 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 87 2024-08-31T07:35:28.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 -- 172.21.15.32:0/3822295573 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71240b4d20 msgr2=0x7f71240b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:28.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71240b4d20 0x7f71240b7230 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7130031440 tx=0x7f7130009560 comp rx=0 tx=0).stop 2024-08-31T07:35:28.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 -- 172.21.15.32:0/3822295573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c075ea0 msgr2=0x7f713c19c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:28.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c075ea0 0x7f713c19c810 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f712800e9c0 tx=0x7f712800ee80 comp rx=0 tx=0).stop 2024-08-31T07:35:28.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 -- 172.21.15.32:0/3822295573 shutdown_connections 2024-08-31T07:35:28.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71240b4d20 0x7f71240b7230 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:28.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f713c10fcb0 0x7f713c1a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:28.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c075ea0 0x7f713c19c810 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:28.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 --2- 172.21.15.32:0/3822295573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f713c075550 0x7f713c19c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:28.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 -- 172.21.15.32:0/3822295573 >> 172.21.15.32:0/3822295573 conn(0x7f713c0ff1e0 msgr2=0x7f713c109870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:28.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 -- 172.21.15.32:0/3822295573 shutdown_connections 2024-08-31T07:35:28.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:28.783+0000 7f7142400640 1 -- 172.21.15.32:0/3822295573 wait complete. 2024-08-31T07:35:28.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:28 smithi073 ceph-mon[21995]: pgmap v1083: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 291 KiB/s wr, 24 op/s 2024-08-31T07:35:28.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:28 smithi084 ceph-mon[22387]: pgmap v1083: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 291 KiB/s wr, 24 op/s 2024-08-31T07:35:29.424 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.49 MB/sec execute 429 sec latency 25339.898 ms 2024-08-31T07:35:29.730 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id fdae7e2f --release --await' 2024-08-31T07:35:29.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id fdae7e2f --release --await 2024-08-31T07:35:29.778 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:29 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3822295573' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:29.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:29 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3822295573' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:29.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:29 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3822295573' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:30.108 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:30.424 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157430 13.45 MB/sec execute 430 sec latency 26339.962 ms 2024-08-31T07:35:30.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.787+0000 7fbfbc600640 1 -- 172.21.15.32:0/2959722384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfb410b470 msgr2=0x7fbfb410b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:30.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.787+0000 7fbfbc600640 1 --2- 172.21.15.32:0/2959722384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfb410b470 0x7fbfb410b880 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009900 tx=0x7fbfa002a410 comp rx=0 tx=0).stop 2024-08-31T07:35:30.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.787+0000 7fbfbc600640 1 -- 172.21.15.32:0/2959722384 shutdown_connections 2024-08-31T07:35:30.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.787+0000 7fbfbc600640 1 --2- 172.21.15.32:0/2959722384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb410bdc0 0x7fbfb41125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.787+0000 7fbfbc600640 1 --2- 172.21.15.32:0/2959722384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfb410b470 0x7fbfb410b880 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.787+0000 7fbfbc600640 1 --2- 172.21.15.32:0/2959722384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb4105b30 0x7fbfb4105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.787+0000 7fbfbc600640 1 -- 172.21.15.32:0/2959722384 >> 172.21.15.32:0/2959722384 conn(0x7fbfb4101600 msgr2=0x7fbfb4103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:30.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.787+0000 7fbfbc600640 1 -- 172.21.15.32:0/2959722384 shutdown_connections 2024-08-31T07:35:30.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.787+0000 7fbfbc600640 1 -- 172.21.15.32:0/2959722384 wait complete. 2024-08-31T07:35:30.790 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:30 smithi032 ceph-mon[17252]: pgmap v1084: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 261 KiB/s wr, 21 op/s 2024-08-31T07:35:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.788+0000 7fbfbc600640 1 Processor -- start 2024-08-31T07:35:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.789+0000 7fbfbc600640 1 -- start start 2024-08-31T07:35:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.789+0000 7fbfbc600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfb4105b30 0x7fbfb419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.789+0000 7fbfbc600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb410b470 0x7fbfb419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.789+0000 7fbfbc600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb410bdc0 0x7fbfb41a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.789+0000 7fbfbc600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4115a80 con 0x7fbfb4105b30 2024-08-31T07:35:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.789+0000 7fbfbc600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4115900 con 0x7fbfb410b470 2024-08-31T07:35:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.789+0000 7fbfbc600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4115c00 con 0x7fbfb410bdc0 2024-08-31T07:35:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.789+0000 7fbfb9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfb4105b30 0x7fbfb419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.789+0000 7fbfb9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfb4105b30 0x7fbfb419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43412/0 (socket says 172.21.15.32:43412) 2024-08-31T07:35:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.789+0000 7fbfb9c00640 1 -- 172.21.15.32:0/555153120 learned_addr learned my addr 172.21.15.32:0/555153120 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.790+0000 7fbfb9200640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb410b470 0x7fbfb419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.790+0000 7fbfba600640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb410bdc0 0x7fbfb41a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.790+0000 7fbfb9c00640 1 -- 172.21.15.32:0/555153120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb410bdc0 msgr2=0x7fbfb41a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.790+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb410bdc0 0x7fbfb41a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.790+0000 7fbfb9c00640 1 -- 172.21.15.32:0/555153120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb410b470 msgr2=0x7fbfb419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.790+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb410b470 0x7fbfb419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.790+0000 7fbfb9c00640 1 -- 172.21.15.32:0/555153120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb41a2400 con 0x7fbfb4105b30 2024-08-31T07:35:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.790+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfb4105b30 0x7fbfb419c490 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e9f0 tx=0x7fbfa800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.790+0000 7fbfb2000640 1 -- 172.21.15.32:0/555153120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800cd80 con 0x7fbfb4105b30 2024-08-31T07:35:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.791+0000 7fbfb2000640 1 -- 172.21.15.32:0/555153120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa80108e0 con 0x7fbfb4105b30 2024-08-31T07:35:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.791+0000 7fbfbc600640 1 -- 172.21.15.32:0/555153120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb41a26f0 con 0x7fbfb4105b30 2024-08-31T07:35:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.791+0000 7fbfb2000640 1 -- 172.21.15.32:0/555153120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8018980 con 0x7fbfb4105b30 2024-08-31T07:35:30.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.791+0000 7fbfbc600640 1 -- 172.21.15.32:0/555153120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb41a8e50 con 0x7fbfb4105b30 2024-08-31T07:35:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.794+0000 7fbfc3423740 1 Processor -- start 2024-08-31T07:35:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.794+0000 7fbfc3423740 1 -- start start 2024-08-31T07:35:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.794+0000 7fbfb2000640 1 -- 172.21.15.32:0/555153120 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbfa8010430 con 0x7fbfb4105b30 2024-08-31T07:35:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.794+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7cf4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.794+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7cf5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.794+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 0x555ee7cf5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfc3423740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555ee7c6ba90 con 0x555ee794f520 2024-08-31T07:35:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfc3423740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555ee7c5b020 con 0x555ee7926660 2024-08-31T07:35:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfc3423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555ee7c5efd0 con 0x555ee792e840 2024-08-31T07:35:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfba600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7cf4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfba600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7cf4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43414/0 (socket says 172.21.15.32:43414) 2024-08-31T07:35:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfba600640 1 -- 172.21.15.32:0/1583422787 learned_addr learned my addr 172.21.15.32:0/1583422787 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:30.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/1583422787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7cf5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfb9200640 1 --2- 172.21.15.32:0/1583422787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 0x555ee7cf5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfb9200640 1 -- 172.21.15.32:0/1583422787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 msgr2=0x555ee7cf5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfb9200640 1 --2- 172.21.15.32:0/1583422787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7cf5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfb9200640 1 -- 172.21.15.32:0/1583422787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 msgr2=0x555ee7cf4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfb9200640 1 --2- 172.21.15.32:0/1583422787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7cf4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.795+0000 7fbfb9200640 1 -- 172.21.15.32:0/1583422787 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ee79bab00 con 0x555ee792e840 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.796+0000 7fbfb9200640 1 --2- 172.21.15.32:0/1583422787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 0x555ee7cf5970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbfa002a930 tx=0x7fbfa0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55fa5b6477243757 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.796+0000 7fbfa6a00640 1 -- 172.21.15.32:0/1583422787 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0040070 con 0x555ee792e840 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.796+0000 7fbfa6a00640 1 -- 172.21.15.32:0/1583422787 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa0004030 con 0x555ee792e840 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.796+0000 7fbfa6a00640 1 -- 172.21.15.32:0/1583422787 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0035040 con 0x555ee792e840 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.797+0000 7fbfb2000640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf9c0b4ca0 0x7fbf9c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.797+0000 7fbfb2000640 1 -- 172.21.15.32:0/555153120 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fbfa80e5860 con 0x7fbfb4105b30 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.797+0000 7fbfba600640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf9c0b4ca0 0x7fbf9c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.797+0000 7fbfc3423740 1 -- 172.21.15.32:0/1583422787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 msgr2=0x555ee7cf5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.797+0000 7fbfc3423740 1 --2- 172.21.15.32:0/1583422787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 0x555ee7cf5970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbfa002a930 tx=0x7fbfa0000bc0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.798+0000 7fbfc3423740 1 -- 172.21.15.32:0/1583422787 shutdown_connections 2024-08-31T07:35:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.798+0000 7fbfc3423740 1 --2- 172.21.15.32:0/1583422787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 0x555ee7cf5970 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.798+0000 7fbfc3423740 1 --2- 172.21.15.32:0/1583422787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7cf5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.798+0000 7fbfc3423740 1 --2- 172.21.15.32:0/1583422787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7cf4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.798+0000 7fbfc3423740 1 -- 172.21.15.32:0/1583422787 >> 172.21.15.32:0/1583422787 conn(0x555ee7943910 msgr2=0x555ee7cf2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.798+0000 7fbfc3423740 1 -- 172.21.15.32:0/1583422787 shutdown_connections 2024-08-31T07:35:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.798+0000 7fbfba600640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf9c0b4ca0 0x7fbf9c0b71b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbfac005f40 tx=0x7fbfac009600 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:30.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.798+0000 7fbfc3423740 1 -- 172.21.15.32:0/1583422787 wait complete. 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.798+0000 7fbfc3423740 1 -- start start 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.799+0000 7fbfc3423740 1 Processor -- start 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.799+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7d00310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.799+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 0x555ee7d00850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.799+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7d00d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.799+0000 7fbfc3423740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555ee7abdbb0 con 0x555ee794f520 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.799+0000 7fbfc3423740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555ee7be1f90 con 0x555ee7926660 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.799+0000 7fbfc3423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555ee7d03670 con 0x555ee792e840 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.799+0000 7fbfc3423740 10 client.0 _ll_register_callbacks cb 0x555ee7c07710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfba600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7d00d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfba600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7d00d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43426/0 (socket says 172.21.15.32:43426) 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfba600640 1 -- 172.21.15.32:0/1919762545 learned_addr learned my addr 172.21.15.32:0/1919762545 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfb9200640 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 0x555ee7d00850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7d00310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfba600640 1 -- 172.21.15.32:0/1919762545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 msgr2=0x555ee7d00850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfba600640 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 0x555ee7d00850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfba600640 1 -- 172.21.15.32:0/1919762545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 msgr2=0x555ee7d00310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfba600640 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7d00310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfba600640 1 -- 172.21.15.32:0/1919762545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ee7a31960 con 0x555ee794f520 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.800+0000 7fbfb9200640 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 0x555ee7d00850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.801+0000 7fbfba600640 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7d00d90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0042c0 tx=0x7fbfac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e66577c285b66eb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.801+0000 7fbf9b400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.801+0000 7fbf9b400640 1 -- 172.21.15.32:0/1919762545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac011de0 con 0x555ee794f520 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.801+0000 7fbf9b400640 1 -- 172.21.15.32:0/1919762545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fbfac020420 con 0x555ee794f520 2024-08-31T07:35:30.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.801+0000 7fbfc3423740 1 -- 172.21.15.32:0/1919762545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ee7bf9e30 con 0x555ee794f520 2024-08-31T07:35:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.801+0000 7fbf9b400640 1 -- 172.21.15.32:0/1919762545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fbfac020620 con 0x555ee794f520 2024-08-31T07:35:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.801+0000 7fbfc3423740 10 client.19752 fetch_fsmap learned FSMap version 87 2024-08-31T07:35:30.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.801+0000 7fbfc3423740 1 -- 172.21.15.32:0/1919762545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x555ee7d0f0f0 con 0x555ee794f520 2024-08-31T07:35:30.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.801+0000 7fbf9b400640 1 -- 172.21.15.32:0/1919762545 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555ee7bf9e30 con 0x555ee794f520 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.802+0000 7fbf9b400640 1 -- 172.21.15.32:0/1919762545 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) ==== 8390+0+0 (secure 0 0 0) 0x7fbfac00f040 con 0x555ee794f520 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.802+0000 7fbfc3423740 10 client.19752 fetch_fsmap finished waiting for FSMap version 87 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.802+0000 7fbfc3423740 10 client.19752 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.802+0000 7fbfc3423740 20 client.19752 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.802+0000 7fbfc3423740 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555ee790dea0 0x555ee7d12610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.802+0000 7fbfc3423740 4 client.19752 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.802+0000 7fbfc3423740 1 -- 172.21.15.32:0/1919762545 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ee7c6d9e0 con 0x555ee790dea0 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.802+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555ee790dea0 0x555ee7d12610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.803+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555ee790dea0 0x555ee7d12610 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0379a820453241 server_cookie=9703b0a78a792d5d in_seq=0 out_seq=0 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.803+0000 7fbf9b400640 10 client.19752 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbf9b400640 1 -- 172.21.15.32:0/1919762545 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555ee7c6d9e0 con 0x555ee790dea0 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbf9b400640 10 client.19752 handle_command_reply: tid=0 2024-08-31T07:35:30.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbfc3423740 1 client.19752 shutdown 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbfc3423740 1 -- 172.21.15.32:0/1919762545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 msgr2=0x555ee7d00d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbfc3423740 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7d00d90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0042c0 tx=0x7fbfac004a40 comp rx=0 tx=0).stop 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbfc3423740 1 -- 172.21.15.32:0/1919762545 shutdown_connections 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbfc3423740 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555ee790dea0 0x555ee7d12610 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbfc3423740 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee794f520 0x555ee7d00d90 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbfc3423740 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee792e840 0x555ee7d00850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbfc3423740 1 --2- 172.21.15.32:0/1919762545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7d00310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbf9b400640 0 client.19752 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbfc3423740 1 -- 172.21.15.32:0/1919762545 >> 172.21.15.32:0/1919762545 conn(0x555ee7943910 msgr2=0x555ee7cf80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.804+0000 7fbfc3423740 1 -- 172.21.15.32:0/1919762545 shutdown_connections 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.805+0000 7fbfc3423740 1 -- 172.21.15.32:0/1919762545 wait complete. 2024-08-31T07:35:30.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.805+0000 7fbfc3423740 20 client.19752 trim_cache size 0 max 16384 2024-08-31T07:35:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.823+0000 7fbfc3423740 1 Processor -- start 2024-08-31T07:35:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.823+0000 7fbfc3423740 1 -- start start 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.823+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee7926660 0x555ee7d0c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.823+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7cfb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.823+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee794f520 0x555ee7cfbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.823+0000 7fbfc3423740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41b1650 con 0x555ee792e840 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.823+0000 7fbfc3423740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41b32e0 con 0x555ee794f520 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.823+0000 7fbfc3423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41b1d50 con 0x555ee7926660 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfb9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7cfb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfb9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7cfb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43432/0 (socket says 172.21.15.32:43432) 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfb9200640 1 -- 172.21.15.32:0/3858230870 learned_addr learned my addr 172.21.15.32:0/3858230870 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/3858230870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee7926660 0x555ee7d0c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfba600640 1 --2- 172.21.15.32:0/3858230870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee794f520 0x555ee7cfbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfb9200640 1 -- 172.21.15.32:0/3858230870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee7926660 msgr2=0x555ee7d0c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfb9200640 1 --2- 172.21.15.32:0/3858230870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee7926660 0x555ee7d0c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfb9200640 1 -- 172.21.15.32:0/3858230870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee794f520 msgr2=0x555ee7cfbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfb9200640 1 --2- 172.21.15.32:0/3858230870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee794f520 0x555ee7cfbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfb9200640 1 -- 172.21.15.32:0/3858230870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ee7cf71e0 con 0x555ee792e840 2024-08-31T07:35:30.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.824+0000 7fbfb9200640 1 --2- 172.21.15.32:0/3858230870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7cfb5a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0005f70 tx=0x7fbfa00097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2300cbc40680d9dc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:30.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.825+0000 7fbfa6000640 1 -- 172.21.15.32:0/3858230870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0052070 con 0x555ee792e840 2024-08-31T07:35:30.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.825+0000 7fbfa6000640 1 -- 172.21.15.32:0/3858230870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa0004030 con 0x555ee792e840 2024-08-31T07:35:30.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.825+0000 7fbfa6000640 1 -- 172.21.15.32:0/3858230870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0035040 con 0x555ee792e840 2024-08-31T07:35:30.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 -- 172.21.15.32:0/3858230870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 msgr2=0x555ee7cfb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:30.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 --2- 172.21.15.32:0/3858230870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7cfb5a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0005f70 tx=0x7fbfa00097c0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 -- 172.21.15.32:0/3858230870 shutdown_connections 2024-08-31T07:35:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 --2- 172.21.15.32:0/3858230870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee794f520 0x555ee7cfbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 --2- 172.21.15.32:0/3858230870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7cfb5a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 --2- 172.21.15.32:0/3858230870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee7926660 0x555ee7d0c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 -- 172.21.15.32:0/3858230870 >> 172.21.15.32:0/3858230870 conn(0x555ee7943910 msgr2=0x555ee7cfae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 -- 172.21.15.32:0/3858230870 shutdown_connections 2024-08-31T07:35:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 -- 172.21.15.32:0/3858230870 wait complete. 2024-08-31T07:35:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 -- start start 2024-08-31T07:35:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.826+0000 7fbfc3423740 1 Processor -- start 2024-08-31T07:35:30.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7d070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7d07680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfc3423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee794f520 0x555ee7d7c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfc3423740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41b2c10 con 0x555ee792e840 2024-08-31T07:35:30.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfc3423740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41b2480 con 0x555ee7926660 2024-08-31T07:35:30.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfc3423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41af650 con 0x555ee794f520 2024-08-31T07:35:30.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfc3423740 10 client.0 _ll_register_callbacks cb 0x7fbfac003e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfb9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7d07680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfb9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7d070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfb9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7d070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:43506/0 (socket says 172.21.15.32:43506) 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfb9c00640 1 -- 172.21.15.32:0/3450473801 learned_addr learned my addr 172.21.15.32:0/3450473801 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.827+0000 7fbfba600640 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee794f520 0x555ee7d7c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbfb9200640 1 -- 172.21.15.32:0/3450473801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee794f520 msgr2=0x555ee7d7c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbfb9200640 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee794f520 0x555ee7d7c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbfb9200640 1 -- 172.21.15.32:0/3450473801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 msgr2=0x555ee7d070e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbfb9200640 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7d070e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbfb9200640 1 -- 172.21.15.32:0/3450473801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ee7d7c660 con 0x555ee792e840 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbfba600640 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee794f520 0x555ee7d7c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7d070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:30.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbfb9200640 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7d07680 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbfa002a450 tx=0x7fbfa00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef04583c507fbba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbf9b400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbf9b400640 1 -- 172.21.15.32:0/3450473801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0052070 con 0x555ee792e840 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbf9b400640 1 -- 172.21.15.32:0/3450473801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fbfa00316a0 con 0x555ee792e840 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.828+0000 7fbfc3423740 1 -- 172.21.15.32:0/3450473801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa0004030 con 0x555ee792e840 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbf9b400640 1 -- 172.21.15.32:0/3450473801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fbfa0035040 con 0x555ee792e840 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbfc3423740 10 client.19758 fetch_fsmap learned FSMap version 87 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbfc3423740 1 -- 172.21.15.32:0/3450473801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x555ee7d7d920 con 0x555ee792e840 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbf9b400640 1 -- 172.21.15.32:0/3450473801 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa0004030 con 0x555ee792e840 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbf9b400640 1 -- 172.21.15.32:0/3450473801 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) ==== 8390+0+0 (secure 0 0 0) 0x7fbfa004c070 con 0x555ee792e840 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbfc3423740 10 client.19758 fetch_fsmap finished waiting for FSMap version 87 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbfc3423740 10 client.19758 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbfc3423740 20 client.19758 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:30.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbfc3423740 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555ee78e3250 0x555ee7d7fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbfc3423740 4 client.19758 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fdae7e2f", "release": true, "await": true}] 2024-08-31T07:35:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.829+0000 7fbfc3423740 1 -- 172.21.15.32:0/3450473801 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fdae7e2f", "release": true, "await": true}) -- 0x555ee7c69480 con 0x555ee78e3250 2024-08-31T07:35:30.833 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:30 smithi073 ceph-mon[21995]: pgmap v1084: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 261 KiB/s wr, 21 op/s 2024-08-31T07:35:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.830+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555ee78e3250 0x555ee7d7fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.830+0000 7fbfb9c00640 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555ee78e3250 0x555ee7d7fba0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3bc2fb540d3771d server_cookie=a0599196af66caac in_seq=0 out_seq=0 2024-08-31T07:35:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.830+0000 7fbf9b400640 10 client.19758 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbf9b400640 1 -- 172.21.15.32:0/3450473801 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555ee7c69480 con 0x555ee78e3250 2024-08-31T07:35:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbf9b400640 10 client.19758 handle_command_reply: tid=0 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbfc3423740 1 client.19758 shutdown 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbfc3423740 1 -- 172.21.15.32:0/3450473801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 msgr2=0x555ee7d07680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbfc3423740 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7d07680 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbfa002a450 tx=0x7fbfa00058d0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbfc3423740 1 -- 172.21.15.32:0/3450473801 shutdown_connections 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbfc3423740 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x555ee78e3250 0x555ee7d7fba0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbfc3423740 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555ee794f520 0x555ee7d7c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbfc3423740 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555ee792e840 0x555ee7d07680 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbfc3423740 1 --2- 172.21.15.32:0/3450473801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ee7926660 0x555ee7d070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbf9b400640 0 client.19758 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.927+0000 7fbfc3423740 1 -- 172.21.15.32:0/3450473801 >> 172.21.15.32:0/3450473801 conn(0x555ee7943910 msgr2=0x555ee7cfb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.928+0000 7fbfc3423740 1 -- 172.21.15.32:0/3450473801 shutdown_connections 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.928+0000 7fbfc3423740 1 -- 172.21.15.32:0/3450473801 wait complete. 2024-08-31T07:35:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.928+0000 7fbfc3423740 20 client.19758 trim_cache size 0 max 16384 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 87, 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 72, 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "fdae7e2f": { 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 72, 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:35:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 150.0, 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:30.933 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:35:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.929+0000 7fbfbc600640 1 -- 172.21.15.32:0/555153120 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf9c0b4ca0 msgr2=0x7fbf9c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.929+0000 7fbfbc600640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf9c0b4ca0 0x7fbf9c0b71b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbfac005f40 tx=0x7fbfac009600 comp rx=0 tx=0).stop 2024-08-31T07:35:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.929+0000 7fbfbc600640 1 -- 172.21.15.32:0/555153120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfb4105b30 msgr2=0x7fbfb419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.929+0000 7fbfbc600640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfb4105b30 0x7fbfb419c490 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e9f0 tx=0x7fbfa800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.929+0000 7fbfbc600640 1 -- 172.21.15.32:0/555153120 shutdown_connections 2024-08-31T07:35:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.929+0000 7fbfbc600640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fbf9c0b4ca0 0x7fbf9c0b71b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.929+0000 7fbfbc600640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb410bdc0 0x7fbfb41a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.929+0000 7fbfbc600640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfb410b470 0x7fbfb419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.929+0000 7fbfbc600640 1 --2- 172.21.15.32:0/555153120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfb4105b30 0x7fbfb419c490 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.929+0000 7fbfbc600640 1 -- 172.21.15.32:0/555153120 >> 172.21.15.32:0/555153120 conn(0x7fbfb4101600 msgr2=0x7fbfb4103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.930+0000 7fbfbc600640 1 -- 172.21.15.32:0/555153120 shutdown_connections 2024-08-31T07:35:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:30.930+0000 7fbfbc600640 1 -- 172.21.15.32:0/555153120 wait complete. 2024-08-31T07:35:30.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:30 smithi084 ceph-mon[22387]: pgmap v1084: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 261 KiB/s wr, 21 op/s 2024-08-31T07:35:31.424 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157697 13.45 MB/sec execute 431 sec latency 26888.085 ms 2024-08-31T07:35:32.167 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fdae7e2f', total seconds elapsed: 34.8 2024-08-31T07:35:32.167 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-08-31T07:35:32.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:35:32.424 INFO:tasks.workunit.client.0.smithi032.stdout: 10 158486 13.51 MB/sec execute 432 sec latency 81.873 ms 2024-08-31T07:35:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:32.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:32 smithi032 ceph-mon[17252]: pgmap v1085: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 288 KiB/s wr, 22 op/s 2024-08-31T07:35:32.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:32 smithi073 ceph-mon[21995]: pgmap v1085: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 288 KiB/s wr, 22 op/s 2024-08-31T07:35:32.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:32 smithi084 ceph-mon[22387]: pgmap v1085: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 288 KiB/s wr, 22 op/s 2024-08-31T07:35:33.424 INFO:tasks.workunit.client.0.smithi032.stdout: 10 159273 13.61 MB/sec execute 433 sec latency 62.803 ms 2024-08-31T07:35:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.466+0000 7efff2600640 1 -- 172.21.15.32:0/3274209548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effec105960 msgr2=0x7effec105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.466+0000 7efff2600640 1 --2- 172.21.15.32:0/3274209548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effec105960 0x7effec105d70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7effdc009ca0 tx=0x7effdc02a8c0 comp rx=0 tx=0).stop 2024-08-31T07:35:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.466+0000 7efff2600640 1 -- 172.21.15.32:0/3274209548 shutdown_connections 2024-08-31T07:35:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.466+0000 7efff2600640 1 --2- 172.21.15.32:0/3274209548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effec10bbf0 0x7effec112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.466+0000 7efff2600640 1 --2- 172.21.15.32:0/3274209548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec10b2a0 0x7effec10b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.466+0000 7efff2600640 1 --2- 172.21.15.32:0/3274209548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effec105960 0x7effec105d70 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.466+0000 7efff2600640 1 -- 172.21.15.32:0/3274209548 >> 172.21.15.32:0/3274209548 conn(0x7effec1013d0 msgr2=0x7effec103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.466+0000 7efff2600640 1 -- 172.21.15.32:0/3274209548 shutdown_connections 2024-08-31T07:35:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.467+0000 7efff2600640 1 -- 172.21.15.32:0/3274209548 wait complete. 2024-08-31T07:35:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.468+0000 7efff2600640 1 Processor -- start 2024-08-31T07:35:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.469+0000 7efff2600640 1 -- start start 2024-08-31T07:35:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.469+0000 7efff2600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effec105960 0x7effec19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.469+0000 7efff2600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec10b2a0 0x7effec19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.470+0000 7efff2600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effec10bbf0 0x7effec1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.470+0000 7efff1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effec105960 0x7effec19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.470+0000 7efff1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effec105960 0x7effec19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:43526/0 (socket says 172.21.15.32:43526) 2024-08-31T07:35:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.470+0000 7efff1200640 1 -- 172.21.15.32:0/3066977783 learned_addr learned my addr 172.21.15.32:0/3066977783 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.470+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7effec115900 con 0x7effec10bbf0 2024-08-31T07:35:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.470+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7effec115780 con 0x7effec105960 2024-08-31T07:35:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.470+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7effec115a80 con 0x7effec10b2a0 2024-08-31T07:35:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.470+0000 7effe3e00640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec10b2a0 0x7effec19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.470+0000 7efff1c00640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effec10bbf0 0x7effec1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.471+0000 7efff1200640 1 -- 172.21.15.32:0/3066977783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec10b2a0 msgr2=0x7effec19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.471+0000 7efff1200640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec10b2a0 0x7effec19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.471+0000 7efff1200640 1 -- 172.21.15.32:0/3066977783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effec10bbf0 msgr2=0x7effec1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.471+0000 7efff1200640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effec10bbf0 0x7effec1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.471+0000 7efff1200640 1 -- 172.21.15.32:0/3066977783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effec1a2220 con 0x7effec105960 2024-08-31T07:35:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.472+0000 7efff1200640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effec105960 0x7effec19c2b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7effdc005e30 tx=0x7effdc031840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.473+0000 7effea000640 1 -- 172.21.15.32:0/3066977783 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc003b70 con 0x7effec105960 2024-08-31T07:35:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.473+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effec1a24b0 con 0x7effec105960 2024-08-31T07:35:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.473+0000 7effea000640 1 -- 172.21.15.32:0/3066977783 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effdc003d10 con 0x7effec105960 2024-08-31T07:35:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.473+0000 7effea000640 1 -- 172.21.15.32:0/3066977783 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc02fa50 con 0x7effec105960 2024-08-31T07:35:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.473+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effec1a8c70 con 0x7effec105960 2024-08-31T07:35:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.474+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effec105d70 con 0x7effec105960 2024-08-31T07:35:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.477+0000 7effea000640 1 -- 172.21.15.32:0/3066977783 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7effdc02fbf0 con 0x7effec105960 2024-08-31T07:35:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.479+0000 7effea000640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effd00b4d20 0x7effd00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.479+0000 7effea000640 1 -- 172.21.15.32:0/3066977783 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7effdc1067d0 con 0x7effec105960 2024-08-31T07:35:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.479+0000 7effe3e00640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effd00b4d20 0x7effd00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.481+0000 7effe3e00640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effd00b4d20 0x7effd00b7230 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7effec10ae00 tx=0x7effd4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.483+0000 7effea000640 1 -- 172.21.15.32:0/3066977783 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7effdc0cd5b0 con 0x7effec105960 2024-08-31T07:35:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.833+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effec19d3e0 con 0x7effec105960 2024-08-31T07:35:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.835+0000 7effea000640 1 -- 172.21.15.32:0/3066977783 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+11971 (secure 0 0 0) 0x7effec19d3e0 con 0x7effec105960 2024-08-31T07:35:33.838 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:35:33.838 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"btime":"2024-08-31T07:35:25:510757+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":87,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:35:24.994280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:35:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 87 2024-08-31T07:35:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effd00b4d20 msgr2=0x7effd00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effd00b4d20 0x7effd00b7230 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7effec10ae00 tx=0x7effd4005f90 comp rx=0 tx=0).stop 2024-08-31T07:35:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effec105960 msgr2=0x7effec19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effec105960 0x7effec19c2b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7effdc005e30 tx=0x7effdc031840 comp rx=0 tx=0).stop 2024-08-31T07:35:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 shutdown_connections 2024-08-31T07:35:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7effd00b4d20 0x7effd00b7230 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effec10bbf0 0x7effec1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec10b2a0 0x7effec19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 --2- 172.21.15.32:0/3066977783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effec105960 0x7effec19c2b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 >> 172.21.15.32:0/3066977783 conn(0x7effec1013d0 msgr2=0x7effec1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 shutdown_connections 2024-08-31T07:35:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:33.841+0000 7efff2600640 1 -- 172.21.15.32:0/3066977783 wait complete. 2024-08-31T07:35:34.424 INFO:tasks.workunit.client.0.smithi032.stdout: 10 159797 13.65 MB/sec execute 434 sec latency 111.337 ms 2024-08-31T07:35:34.814 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:34 smithi032 ceph-mon[17252]: pgmap v1086: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 419 KiB/s wr, 32 op/s 2024-08-31T07:35:34.814 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:34 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3066977783' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:34 smithi073 ceph-mon[21995]: pgmap v1086: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 419 KiB/s wr, 32 op/s 2024-08-31T07:35:34.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:34 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3066977783' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:34 smithi084 ceph-mon[22387]: pgmap v1086: 97 pgs: 97 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 419 KiB/s wr, 32 op/s 2024-08-31T07:35:34.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:34 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3066977783' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:35:35.424 INFO:tasks.workunit.client.0.smithi032.stdout: 10 160345 13.66 MB/sec execute 435 sec latency 75.769 ms 2024-08-31T07:35:35.624 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-08-31T07:35:35.624 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-08-31T07:35:36.306 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:36.424 INFO:tasks.workunit.client.0.smithi032.stdout: 10 160915 13.70 MB/sec execute 436 sec latency 105.117 ms 2024-08-31T07:35:36.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:36 smithi032 ceph-mon[17252]: pgmap v1087: 97 pgs: 97 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-08-31T07:35:36.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:36 smithi073 ceph-mon[21995]: pgmap v1087: 97 pgs: 97 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-08-31T07:35:36.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:36 smithi084 ceph-mon[22387]: pgmap v1087: 97 pgs: 97 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-08-31T07:35:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.359+0000 7fa53da00640 1 -- 172.21.15.32:0/1739087542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa53810bdf0 msgr2=0x7fa538112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.359+0000 7fa53da00640 1 --2- 172.21.15.32:0/1739087542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa53810bdf0 0x7fa538112620 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa524009900 tx=0x7fa52402a420 comp rx=0 tx=0).stop 2024-08-31T07:35:37.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.361+0000 7fa53da00640 1 -- 172.21.15.32:0/1739087542 shutdown_connections 2024-08-31T07:35:37.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.361+0000 7fa53da00640 1 --2- 172.21.15.32:0/1739087542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa53810bdf0 0x7fa538112620 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.361+0000 7fa53da00640 1 --2- 172.21.15.32:0/1739087542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53810b4a0 0x7fa53810b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.361+0000 7fa53da00640 1 --2- 172.21.15.32:0/1739087542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa538105b60 0x7fa538105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.361+0000 7fa53da00640 1 -- 172.21.15.32:0/1739087542 >> 172.21.15.32:0/1739087542 conn(0x7fa538101570 msgr2=0x7fa5381039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.361+0000 7fa53da00640 1 -- 172.21.15.32:0/1739087542 shutdown_connections 2024-08-31T07:35:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.361+0000 7fa53da00640 1 -- 172.21.15.32:0/1739087542 wait complete. 2024-08-31T07:35:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.363+0000 7fa53da00640 1 Processor -- start 2024-08-31T07:35:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.363+0000 7fa53da00640 1 -- start start 2024-08-31T07:35:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.364+0000 7fa53da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa538105b60 0x7fa53819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.364+0000 7fa53da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53810b4a0 0x7fa53819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.364+0000 7fa53da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa53810bdf0 0x7fa5381a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.364+0000 7fa53da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa538115af0 con 0x7fa53810b4a0 2024-08-31T07:35:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.364+0000 7fa53da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa538115970 con 0x7fa53810bdf0 2024-08-31T07:35:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.364+0000 7fa53da00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa538115c70 con 0x7fa538105b60 2024-08-31T07:35:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.365+0000 7fa536a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa538105b60 0x7fa53819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.365+0000 7fa536a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa538105b60 0x7fa53819c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33268/0 (socket says 172.21.15.32:33268) 2024-08-31T07:35:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.365+0000 7fa536a00640 1 -- 172.21.15.32:0/2023413125 learned_addr learned my addr 172.21.15.32:0/2023413125 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.365+0000 7fa537400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa53810bdf0 0x7fa5381a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.365+0000 7fa536000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53810b4a0 0x7fa53819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.367+0000 7fa536a00640 1 -- 172.21.15.32:0/2023413125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa53810bdf0 msgr2=0x7fa5381a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:37.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.367+0000 7fa536a00640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa53810bdf0 0x7fa5381a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.367+0000 7fa536a00640 1 -- 172.21.15.32:0/2023413125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53810b4a0 msgr2=0x7fa53819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:37.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.367+0000 7fa536a00640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53810b4a0 0x7fa53819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.367+0000 7fa536a00640 1 -- 172.21.15.32:0/2023413125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5381a2390 con 0x7fa538105b60 2024-08-31T07:35:37.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.368+0000 7fa536a00640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa538105b60 0x7fa53819c420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa53000eab0 tx=0x7fa53000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:37.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.368+0000 7fa52f400640 1 -- 172.21.15.32:0/2023413125 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53000cd80 con 0x7fa538105b60 2024-08-31T07:35:37.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.368+0000 7fa52f400640 1 -- 172.21.15.32:0/2023413125 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa530011920 con 0x7fa538105b60 2024-08-31T07:35:37.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.368+0000 7fa52f400640 1 -- 172.21.15.32:0/2023413125 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530010930 con 0x7fa538105b60 2024-08-31T07:35:37.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.369+0000 7fa53da00640 1 -- 172.21.15.32:0/2023413125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5381a2680 con 0x7fa538105b60 2024-08-31T07:35:37.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.369+0000 7fa53da00640 1 -- 172.21.15.32:0/2023413125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5380775a0 con 0x7fa538105b60 2024-08-31T07:35:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.374+0000 7fa54468f740 1 Processor -- start 2024-08-31T07:35:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.374+0000 7fa54468f740 1 -- start start 2024-08-31T07:35:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.374+0000 7fa52f400640 1 -- 172.21.15.32:0/2023413125 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa530011470 con 0x7fa538105b60 2024-08-31T07:35:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.374+0000 7fa54468f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0eb840 0x557b1d492110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.374+0000 7fa54468f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d492650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.375+0000 7fa54468f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 0x557b1d492b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.375+0000 7fa54468f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557b1d407530 con 0x557b1d0cab60 2024-08-31T07:35:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.375+0000 7fa54468f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557b1d3f6ac0 con 0x557b1d0eb840 2024-08-31T07:35:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.375+0000 7fa54468f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557b1d3faa70 con 0x557b1d0c2980 2024-08-31T07:35:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.375+0000 7fa536a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d492650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.375+0000 7fa536a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d492650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33276/0 (socket says 172.21.15.32:33276) 2024-08-31T07:35:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.375+0000 7fa536a00640 1 -- 172.21.15.32:0/1839767422 learned_addr learned my addr 172.21.15.32:0/1839767422 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.375+0000 7fa536000640 1 --2- 172.21.15.32:0/1839767422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 0x557b1d492b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.375+0000 7fa537400640 1 --2- 172.21.15.32:0/1839767422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0eb840 0x557b1d492110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.376+0000 7fa536a00640 1 -- 172.21.15.32:0/1839767422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0eb840 msgr2=0x557b1d492110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.376+0000 7fa536a00640 1 --2- 172.21.15.32:0/1839767422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0eb840 0x557b1d492110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.376+0000 7fa536a00640 1 -- 172.21.15.32:0/1839767422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 msgr2=0x557b1d492b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.376+0000 7fa536a00640 1 --2- 172.21.15.32:0/1839767422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 0x557b1d492b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.376+0000 7fa536a00640 1 -- 172.21.15.32:0/1839767422 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1d3a9880 con 0x557b1d0c2980 2024-08-31T07:35:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.377+0000 7fa536000640 1 --2- 172.21.15.32:0/1839767422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 0x557b1d492b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.377+0000 7fa536a00640 1 --2- 172.21.15.32:0/1839767422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d492650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa5300e30f0 tx=0x7fa5300e9c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d975e997b2a1bd7d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.377+0000 7fa52f400640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa51c0b4d50 0x7fa51c0b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.377+0000 7fa523e00640 1 -- 172.21.15.32:0/1839767422 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5300f9070 con 0x557b1d0c2980 2024-08-31T07:35:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.378+0000 7fa52f400640 1 -- 172.21.15.32:0/2023413125 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fa5300e59e0 con 0x7fa538105b60 2024-08-31T07:35:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.378+0000 7fa523e00640 1 -- 172.21.15.32:0/1839767422 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5300abc30 con 0x557b1d0c2980 2024-08-31T07:35:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.378+0000 7fa523e00640 1 -- 172.21.15.32:0/1839767422 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5300a65c0 con 0x557b1d0c2980 2024-08-31T07:35:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.378+0000 7fa537400640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa51c0b4d50 0x7fa51c0b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.379+0000 7fa54468f740 1 -- 172.21.15.32:0/1839767422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 msgr2=0x557b1d492650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.379+0000 7fa54468f740 1 --2- 172.21.15.32:0/1839767422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d492650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa5300e30f0 tx=0x7fa5300e9c40 comp rx=0 tx=0).stop 2024-08-31T07:35:37.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.379+0000 7fa54468f740 1 -- 172.21.15.32:0/1839767422 shutdown_connections 2024-08-31T07:35:37.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.379+0000 7fa54468f740 1 --2- 172.21.15.32:0/1839767422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 0x557b1d492b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.379+0000 7fa54468f740 1 --2- 172.21.15.32:0/1839767422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d492650 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.379+0000 7fa54468f740 1 --2- 172.21.15.32:0/1839767422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0eb840 0x557b1d492110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.379+0000 7fa54468f740 1 -- 172.21.15.32:0/1839767422 >> 172.21.15.32:0/1839767422 conn(0x557b1d0df990 msgr2=0x557b1d490000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:37.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.380+0000 7fa54468f740 1 -- 172.21.15.32:0/1839767422 shutdown_connections 2024-08-31T07:35:37.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.380+0000 7fa54468f740 1 -- 172.21.15.32:0/1839767422 wait complete. 2024-08-31T07:35:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.380+0000 7fa537400640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa51c0b4d50 0x7fa51c0b7260 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa524005ab0 tx=0x7fa52402a920 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.382+0000 7fa54468f740 1 -- start start 2024-08-31T07:35:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.382+0000 7fa54468f740 1 Processor -- start 2024-08-31T07:35:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.383+0000 7fa54468f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d49dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.383+0000 7fa54468f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 0x557b1d49e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.383+0000 7fa54468f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0eb840 0x557b1d4a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.383+0000 7fa536000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 0x557b1d49e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.384+0000 7fa536000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 0x557b1d49e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:58628/0 (socket says 172.21.15.32:58628) 2024-08-31T07:35:37.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.384+0000 7fa54468f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557b1d2597a0 con 0x557b1d0eb840 2024-08-31T07:35:37.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.384+0000 7fa54468f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557b1d37d7f0 con 0x557b1d0cab60 2024-08-31T07:35:37.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.384+0000 7fa54468f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557b1d4a08f0 con 0x557b1d0c2980 2024-08-31T07:35:37.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.384+0000 7fa54468f740 10 client.0 _ll_register_callbacks cb 0x557b1d3a2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:37.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.384+0000 7fa536a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d49dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.384+0000 7fa536a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d49dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33280/0 (socket says 172.21.15.32:33280) 2024-08-31T07:35:37.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.384+0000 7fa536a00640 1 -- 172.21.15.32:0/2904637252 learned_addr learned my addr 172.21.15.32:0/2904637252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:37.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.384+0000 7fa537400640 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0eb840 0x557b1d4a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.388+0000 7fa536a00640 1 -- 172.21.15.32:0/2904637252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 msgr2=0x557b1d49e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:37.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.388+0000 7fa536a00640 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 0x557b1d49e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.388+0000 7fa536a00640 1 -- 172.21.15.32:0/2904637252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0eb840 msgr2=0x557b1d4a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:37.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.388+0000 7fa536a00640 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0eb840 0x557b1d4a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.389+0000 7fa536a00640 1 -- 172.21.15.32:0/2904637252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1d165b90 con 0x557b1d0c2980 2024-08-31T07:35:37.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.389+0000 7fa536000640 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 0x557b1d49e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:37.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.389+0000 7fa537400640 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0eb840 0x557b1d4a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.389+0000 7fa536a00640 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d49dd00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa5300abe20 tx=0x7fa53000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6259a4f74d55c186 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.389+0000 7fa520c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:35:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.390+0000 7fa520c00640 1 -- 172.21.15.32:0/2904637252 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5300aba70 con 0x557b1d0c2980 2024-08-31T07:35:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.390+0000 7fa520c00640 1 -- 172.21.15.32:0/2904637252 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fa5300a83b0 con 0x557b1d0c2980 2024-08-31T07:35:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.390+0000 7fa54468f740 1 -- 172.21.15.32:0/2904637252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5300abc30 con 0x557b1d0c2980 2024-08-31T07:35:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.391+0000 7fa520c00640 1 -- 172.21.15.32:0/2904637252 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa5300ab890 con 0x557b1d0c2980 2024-08-31T07:35:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.393+0000 7fa54468f740 10 client.29818 fetch_fsmap learned FSMap version 88 2024-08-31T07:35:37.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.393+0000 7fa54468f740 1 -- 172.21.15.32:0/2904637252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) -- 0x557b1d4ac9b0 con 0x557b1d0c2980 2024-08-31T07:35:37.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.393+0000 7fa520c00640 1 -- 172.21.15.32:0/2904637252 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5300abc30 con 0x557b1d0c2980 2024-08-31T07:35:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.394+0000 7fa520c00640 1 -- 172.21.15.32:0/2904637252 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 88) ==== 8398+0+0 (secure 0 0 0) 0x7fa53000f040 con 0x557b1d0c2980 2024-08-31T07:35:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.394+0000 7fa54468f740 10 client.29818 fetch_fsmap finished waiting for FSMap version 88 2024-08-31T07:35:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.394+0000 7fa54468f740 10 client.29818 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:35:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.394+0000 7fa54468f740 20 client.29818 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.395+0000 7fa54468f740 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b1d0aa1c0 0x557b1d4afd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.395+0000 7fa54468f740 4 client.29818 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.395+0000 7fa54468f740 1 -- 172.21.15.32:0/2904637252 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b1d409480 con 0x557b1d0aa1c0 2024-08-31T07:35:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.395+0000 7fa536000640 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b1d0aa1c0 0x557b1d4afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.396+0000 7fa536000640 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b1d0aa1c0 0x557b1d4afd10 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b038d40904f0a78d server_cookie=72c0cbffd420b5f4 in_seq=0 out_seq=0 2024-08-31T07:35:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.396+0000 7fa520c00640 10 client.29818 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.397+0000 7fa520c00640 1 -- 172.21.15.32:0/2904637252 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b1d409480 con 0x557b1d0aa1c0 2024-08-31T07:35:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.397+0000 7fa520c00640 10 client.29818 handle_command_reply: tid=0 2024-08-31T07:35:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.397+0000 7fa54468f740 1 client.29818 shutdown 2024-08-31T07:35:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.398+0000 7fa54468f740 1 -- 172.21.15.32:0/2904637252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 msgr2=0x557b1d49dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.398+0000 7fa54468f740 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d49dd00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa5300abe20 tx=0x7fa53000cf20 comp rx=0 tx=0).stop 2024-08-31T07:35:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.399+0000 7fa54468f740 1 -- 172.21.15.32:0/2904637252 shutdown_connections 2024-08-31T07:35:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.399+0000 7fa54468f740 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b1d0aa1c0 0x557b1d4afd10 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.399+0000 7fa520c00640 0 client.29818 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.399+0000 7fa54468f740 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0eb840 0x557b1d4a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.399+0000 7fa54468f740 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 0x557b1d49e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.399+0000 7fa54468f740 1 --2- 172.21.15.32:0/2904637252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0c2980 0x557b1d49dd00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.399+0000 7fa54468f740 1 -- 172.21.15.32:0/2904637252 >> 172.21.15.32:0/2904637252 conn(0x557b1d0df990 msgr2=0x557b1d48e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.400+0000 7fa54468f740 1 -- 172.21.15.32:0/2904637252 shutdown_connections 2024-08-31T07:35:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.400+0000 7fa54468f740 1 -- 172.21.15.32:0/2904637252 wait complete. 2024-08-31T07:35:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.401+0000 7fa54468f740 20 client.29818 trim_cache size 0 max 16384 2024-08-31T07:35:37.425 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161653 13.75 MB/sec execute 437 sec latency 94.255 ms 2024-08-31T07:35:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.428+0000 7fa54468f740 1 Processor -- start 2024-08-31T07:35:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.428+0000 7fa54468f740 1 -- start start 2024-08-31T07:35:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.428+0000 7fa54468f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0c2980 0x557b1d48e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.428+0000 7fa54468f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 0x557b1d48ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.428+0000 7fa54468f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0eb840 0x557b1d497220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.428+0000 7fa54468f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381b5b20 con 0x557b1d0c2980 2024-08-31T07:35:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.428+0000 7fa54468f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381b7810 con 0x557b1d0cab60 2024-08-31T07:35:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.428+0000 7fa54468f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381b6220 con 0x557b1d0eb840 2024-08-31T07:35:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.429+0000 7fa536a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0c2980 0x557b1d48e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.429+0000 7fa536a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0c2980 0x557b1d48e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40804/0 (socket says 172.21.15.32:40804) 2024-08-31T07:35:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.429+0000 7fa536a00640 1 -- 172.21.15.32:0/741949981 learned_addr learned my addr 172.21.15.32:0/741949981 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.430+0000 7fa536a00640 1 -- 172.21.15.32:0/741949981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0eb840 msgr2=0x557b1d497220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:35:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.430+0000 7fa536a00640 1 --2- 172.21.15.32:0/741949981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0eb840 0x557b1d497220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.430+0000 7fa536a00640 1 -- 172.21.15.32:0/741949981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 msgr2=0x557b1d48ed20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:35:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.430+0000 7fa536a00640 1 --2- 172.21.15.32:0/741949981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 0x557b1d48ed20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.430+0000 7fa536a00640 1 -- 172.21.15.32:0/741949981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1d49e3c0 con 0x557b1d0c2980 2024-08-31T07:35:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.430+0000 7fa536a00640 1 --2- 172.21.15.32:0/741949981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0c2980 0x557b1d48e7e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa5300ac0b0 tx=0x7fa5300e9c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b22bf2b3fe727c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.431+0000 7fa523400640 1 -- 172.21.15.32:0/741949981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530097040 con 0x557b1d0c2980 2024-08-31T07:35:37.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.433+0000 7fa523400640 1 -- 172.21.15.32:0/741949981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5300e97c0 con 0x557b1d0c2980 2024-08-31T07:35:37.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.433+0000 7fa523400640 1 -- 172.21.15.32:0/741949981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5300ac820 con 0x557b1d0c2980 2024-08-31T07:35:37.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.434+0000 7fa54468f740 1 -- 172.21.15.32:0/741949981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0c2980 msgr2=0x557b1d48e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:37.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.434+0000 7fa54468f740 1 --2- 172.21.15.32:0/741949981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0c2980 0x557b1d48e7e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa5300ac0b0 tx=0x7fa5300e9c40 comp rx=0 tx=0).stop 2024-08-31T07:35:37.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.435+0000 7fa54468f740 1 -- 172.21.15.32:0/741949981 shutdown_connections 2024-08-31T07:35:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.435+0000 7fa54468f740 1 --2- 172.21.15.32:0/741949981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0eb840 0x557b1d497220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.435+0000 7fa54468f740 1 --2- 172.21.15.32:0/741949981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0cab60 0x557b1d48ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.435+0000 7fa54468f740 1 --2- 172.21.15.32:0/741949981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0c2980 0x557b1d48e7e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.435+0000 7fa54468f740 1 -- 172.21.15.32:0/741949981 >> 172.21.15.32:0/741949981 conn(0x557b1d0df990 msgr2=0x557b1d49c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:37.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.435+0000 7fa54468f740 1 -- 172.21.15.32:0/741949981 shutdown_connections 2024-08-31T07:35:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.435+0000 7fa54468f740 1 -- 172.21.15.32:0/741949981 wait complete. 2024-08-31T07:35:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.436+0000 7fa54468f740 1 -- start start 2024-08-31T07:35:37.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.436+0000 7fa54468f740 1 Processor -- start 2024-08-31T07:35:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.437+0000 7fa54468f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0c2980 0x557b1d4af070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.437+0000 7fa54468f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 0x557b1d4af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.437+0000 7fa54468f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0eb840 0x557b1d519480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.437+0000 7fa54468f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381b70e0 con 0x557b1d0cab60 2024-08-31T07:35:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.437+0000 7fa54468f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381b6950 con 0x557b1d0c2980 2024-08-31T07:35:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.437+0000 7fa54468f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381b3b20 con 0x557b1d0eb840 2024-08-31T07:35:37.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.437+0000 7fa54468f740 10 client.0 _ll_register_callbacks cb 0x7fa5300e9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.438+0000 7fa536a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0c2980 0x557b1d4af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.438+0000 7fa537400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0eb840 0x557b1d519480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.438+0000 7fa536000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 0x557b1d4af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.438+0000 7fa537400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0eb840 0x557b1d519480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33302/0 (socket says 172.21.15.32:33302) 2024-08-31T07:35:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.438+0000 7fa537400640 1 -- 172.21.15.32:0/4010703321 learned_addr learned my addr 172.21.15.32:0/4010703321 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.438+0000 7fa536a00640 1 -- 172.21.15.32:0/4010703321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0eb840 msgr2=0x557b1d519480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.438+0000 7fa536a00640 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0eb840 0x557b1d519480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.438+0000 7fa536a00640 1 -- 172.21.15.32:0/4010703321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 msgr2=0x557b1d4af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:37.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.438+0000 7fa536a00640 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 0x557b1d4af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.439+0000 7fa536a00640 1 -- 172.21.15.32:0/4010703321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1d4afe50 con 0x557b1d0c2980 2024-08-31T07:35:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.439+0000 7fa536000640 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 0x557b1d4af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:35:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.439+0000 7fa536a00640 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0c2980 0x557b1d4af070 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa5300ac0b0 tx=0x7fa5300a85e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ed867cc27cea68 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.439+0000 7fa520c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:35:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.439+0000 7fa520c00640 1 -- 172.21.15.32:0/4010703321 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530097040 con 0x557b1d0c2980 2024-08-31T07:35:37.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.439+0000 7fa520c00640 1 -- 172.21.15.32:0/4010703321 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fa5300b4fb0 con 0x557b1d0c2980 2024-08-31T07:35:37.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.440+0000 7fa54468f740 1 -- 172.21.15.32:0/4010703321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5300e97c0 con 0x557b1d0c2980 2024-08-31T07:35:37.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.440+0000 7fa520c00640 1 -- 172.21.15.32:0/4010703321 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa530011760 con 0x557b1d0c2980 2024-08-31T07:35:37.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.440+0000 7fa54468f740 10 client.29816 fetch_fsmap learned FSMap version 88 2024-08-31T07:35:37.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.440+0000 7fa54468f740 1 -- 172.21.15.32:0/4010703321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) -- 0x557b1d4afe50 con 0x557b1d0c2980 2024-08-31T07:35:37.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.441+0000 7fa520c00640 1 -- 172.21.15.32:0/4010703321 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5300e97c0 con 0x557b1d0c2980 2024-08-31T07:35:37.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.441+0000 7fa520c00640 1 -- 172.21.15.32:0/4010703321 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 88) ==== 8398+0+0 (secure 0 0 0) 0x7fa5300ac380 con 0x557b1d0c2980 2024-08-31T07:35:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.442+0000 7fa54468f740 10 client.29816 fetch_fsmap finished waiting for FSMap version 88 2024-08-31T07:35:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.442+0000 7fa54468f740 10 client.29816 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:35:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.442+0000 7fa54468f740 20 client.29816 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.442+0000 7fa54468f740 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b1d07f570 0x557b1d51cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:37.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.442+0000 7fa54468f740 4 client.29816 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-08-31T07:35:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.442+0000 7fa54468f740 1 -- 172.21.15.32:0/4010703321 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x557b1d404f20 con 0x557b1d07f570 2024-08-31T07:35:37.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.443+0000 7fa536000640 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b1d07f570 0x557b1d51cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.444+0000 7fa536000640 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b1d07f570 0x557b1d51cda0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c916f93f6232e99d server_cookie=8b5ae174c9a623a9 in_seq=0 out_seq=0 2024-08-31T07:35:37.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:37.444+0000 7fa520c00640 10 client.29816 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:37.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:37 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:37.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:37 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:37.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:37 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.053+0000 7fa520c00640 1 -- 172.21.15.32:0/4010703321 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557b1d404f20 con 0x557b1d07f570 2024-08-31T07:35:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.053+0000 7fa520c00640 10 client.29816 handle_command_reply: tid=0 2024-08-31T07:35:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.053+0000 7fa54468f740 1 client.29816 shutdown 2024-08-31T07:35:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.053+0000 7fa54468f740 1 -- 172.21.15.32:0/4010703321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0c2980 msgr2=0x557b1d4af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.053+0000 7fa54468f740 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0c2980 0x557b1d4af070 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa5300ac0b0 tx=0x7fa5300a85e0 comp rx=0 tx=0).stop 2024-08-31T07:35:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.054+0000 7fa54468f740 1 -- 172.21.15.32:0/4010703321 shutdown_connections 2024-08-31T07:35:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.054+0000 7fa54468f740 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557b1d07f570 0x557b1d51cda0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.054+0000 7fa54468f740 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557b1d0eb840 0x557b1d519480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.054+0000 7fa54468f740 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557b1d0cab60 0x557b1d4af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.054+0000 7fa520c00640 0 client.29816 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.054+0000 7fa54468f740 1 --2- 172.21.15.32:0/4010703321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557b1d0c2980 0x557b1d4af070 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.054+0000 7fa54468f740 1 -- 172.21.15.32:0/4010703321 >> 172.21.15.32:0/4010703321 conn(0x557b1d0df990 msgr2=0x557b1d48e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.054+0000 7fa54468f740 1 -- 172.21.15.32:0/4010703321 shutdown_connections 2024-08-31T07:35:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.054+0000 7fa54468f740 1 -- 172.21.15.32:0/4010703321 wait complete. 2024-08-31T07:35:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.054+0000 7fa54468f740 20 client.29816 trim_cache size 0 max 16384 2024-08-31T07:35:38.056 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:35:38.056 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 88, 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 74, 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "2e13427": { 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 74, 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 150.0, 2024-08-31T07:35:38.057 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "QUIESCED", 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:38.058 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:35:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 -- 172.21.15.32:0/2023413125 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa51c0b4d50 msgr2=0x7fa51c0b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa51c0b4d50 0x7fa51c0b7260 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa524005ab0 tx=0x7fa52402a920 comp rx=0 tx=0).stop 2024-08-31T07:35:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 -- 172.21.15.32:0/2023413125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa538105b60 msgr2=0x7fa53819c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa538105b60 0x7fa53819c420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa53000eab0 tx=0x7fa53000eeb0 comp rx=0 tx=0).stop 2024-08-31T07:35:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 -- 172.21.15.32:0/2023413125 shutdown_connections 2024-08-31T07:35:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa51c0b4d50 0x7fa51c0b7260 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa53810bdf0 0x7fa5381a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa53810b4a0 0x7fa53819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 --2- 172.21.15.32:0/2023413125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa538105b60 0x7fa53819c420 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 -- 172.21.15.32:0/2023413125 >> 172.21.15.32:0/2023413125 conn(0x7fa538101570 msgr2=0x7fa53810aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 -- 172.21.15.32:0/2023413125 shutdown_connections 2024-08-31T07:35:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:38.055+0000 7fa53da00640 1 -- 172.21.15.32:0/2023413125 wait complete. 2024-08-31T07:35:38.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.72 MB/sec execute 438 sec latency 995.587 ms 2024-08-31T07:35:38.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:38 smithi032 ceph-mon[17252]: pgmap v1088: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 267 op/s 2024-08-31T07:35:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:38 smithi073 ceph-mon[21995]: pgmap v1088: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 267 op/s 2024-08-31T07:35:38.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:38 smithi084 ceph-mon[22387]: pgmap v1088: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 267 op/s 2024-08-31T07:35:39.244 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2e13427', quiesce took 7.1 seconds. Will release after: 22.9 2024-08-31T07:35:39.425 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.69 MB/sec execute 439 sec latency 1995.746 ms 2024-08-31T07:35:40.425 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.66 MB/sec execute 440 sec latency 2995.885 ms 2024-08-31T07:35:40.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:40 smithi032 ceph-mon[17252]: pgmap v1089: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 334 op/s 2024-08-31T07:35:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:40 smithi073 ceph-mon[21995]: pgmap v1089: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 334 op/s 2024-08-31T07:35:40.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:40 smithi084 ceph-mon[22387]: pgmap v1089: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 334 op/s 2024-08-31T07:35:41.425 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.63 MB/sec execute 441 sec latency 3996.048 ms 2024-08-31T07:35:41.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:41 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:35:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:41 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:35:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:41 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:35:41.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:41 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:35:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:41 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:35:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:41 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:35:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:41 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:35:41.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:41 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:35:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:41 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:35:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:41 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:35:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:41 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:35:41.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:41 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:35:42.425 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.60 MB/sec execute 442 sec latency 4996.196 ms 2024-08-31T07:35:42.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:42 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:35:42.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:42 smithi032 ceph-mon[17252]: pgmap v1090: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.7 MiB/s wr, 369 op/s 2024-08-31T07:35:42.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:42 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:42 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:35:42.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:42 smithi073 ceph-mon[21995]: pgmap v1090: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.7 MiB/s wr, 369 op/s 2024-08-31T07:35:42.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:42 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:42 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:35:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:42 smithi084 ceph-mon[22387]: pgmap v1090: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.7 MiB/s wr, 369 op/s 2024-08-31T07:35:42.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:42 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:43.425 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.57 MB/sec execute 443 sec latency 5996.293 ms 2024-08-31T07:35:44.425 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.53 MB/sec execute 444 sec latency 6996.344 ms 2024-08-31T07:35:44.726 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:44 smithi032 ceph-mon[17252]: pgmap v1091: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.9 MiB/s wr, 388 op/s 2024-08-31T07:35:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:44 smithi073 ceph-mon[21995]: pgmap v1091: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.9 MiB/s wr, 388 op/s 2024-08-31T07:35:44.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:44 smithi084 ceph-mon[22387]: pgmap v1091: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.9 MiB/s wr, 388 op/s 2024-08-31T07:35:45.425 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.50 MB/sec execute 445 sec latency 7996.387 ms 2024-08-31T07:35:46.425 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.47 MB/sec execute 446 sec latency 8996.432 ms 2024-08-31T07:35:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:46 smithi073 ceph-mon[21995]: pgmap v1092: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.5 MiB/s wr, 410 op/s 2024-08-31T07:35:46.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:46 smithi084 ceph-mon[22387]: pgmap v1092: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.5 MiB/s wr, 410 op/s 2024-08-31T07:35:47.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:46 smithi032 ceph-mon[17252]: pgmap v1092: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.5 MiB/s wr, 410 op/s 2024-08-31T07:35:47.425 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.44 MB/sec execute 447 sec latency 9996.557 ms 2024-08-31T07:35:48.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.41 MB/sec execute 448 sec latency 10996.604 ms 2024-08-31T07:35:48.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:48 smithi032 ceph-mon[17252]: pgmap v1093: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.7 MiB/s wr, 374 op/s 2024-08-31T07:35:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:48 smithi073 ceph-mon[21995]: pgmap v1093: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.7 MiB/s wr, 374 op/s 2024-08-31T07:35:48.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:48 smithi084 ceph-mon[22387]: pgmap v1093: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.7 MiB/s wr, 374 op/s 2024-08-31T07:35:49.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.38 MB/sec execute 449 sec latency 11996.666 ms 2024-08-31T07:35:50.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.35 MB/sec execute 450 sec latency 12996.729 ms 2024-08-31T07:35:50.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:50 smithi084 ceph-mon[22387]: pgmap v1094: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 219 op/s 2024-08-31T07:35:51.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:50 smithi032 ceph-mon[17252]: pgmap v1094: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 219 op/s 2024-08-31T07:35:51.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:50 smithi073 ceph-mon[21995]: pgmap v1094: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 219 op/s 2024-08-31T07:35:51.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.32 MB/sec execute 451 sec latency 13996.847 ms 2024-08-31T07:35:51.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:51 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:52.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:51 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:52.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:51 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:35:52.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.30 MB/sec execute 452 sec latency 14996.949 ms 2024-08-31T07:35:52.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:52 smithi084 ceph-mon[22387]: pgmap v1095: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 170 op/s 2024-08-31T07:35:53.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:52 smithi032 ceph-mon[17252]: pgmap v1095: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 170 op/s 2024-08-31T07:35:53.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:52 smithi073 ceph-mon[21995]: pgmap v1095: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 170 op/s 2024-08-31T07:35:53.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.27 MB/sec execute 453 sec latency 15997.076 ms 2024-08-31T07:35:54.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.24 MB/sec execute 454 sec latency 16997.175 ms 2024-08-31T07:35:54.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:54 smithi084 ceph-mon[22387]: pgmap v1096: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 4.2 MiB/s wr, 147 op/s 2024-08-31T07:35:55.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:54 smithi032 ceph-mon[17252]: pgmap v1096: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 4.2 MiB/s wr, 147 op/s 2024-08-31T07:35:55.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:54 smithi073 ceph-mon[21995]: pgmap v1096: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 4.2 MiB/s wr, 147 op/s 2024-08-31T07:35:55.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.21 MB/sec execute 455 sec latency 17997.264 ms 2024-08-31T07:35:56.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:35:56.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.18 MB/sec execute 456 sec latency 18997.312 ms 2024-08-31T07:35:56.444 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:56.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:56 smithi032 ceph-mon[17252]: pgmap v1097: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 4.2 MiB/s wr, 139 op/s 2024-08-31T07:35:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:56 smithi084 ceph-mon[22387]: pgmap v1097: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 4.2 MiB/s wr, 139 op/s 2024-08-31T07:35:57.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.097+0000 7fd116e00640 1 -- 172.21.15.32:0/3479672642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1101099c0 msgr2=0x7fd1101101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:57.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.097+0000 7fd116e00640 1 --2- 172.21.15.32:0/3479672642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1101099c0 0x7fd1101101f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd108009ca0 tx=0x7fd10802a8b0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.097+0000 7fd116e00640 1 -- 172.21.15.32:0/3479672642 shutdown_connections 2024-08-31T07:35:57.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.097+0000 7fd116e00640 1 --2- 172.21.15.32:0/3479672642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1101099c0 0x7fd1101101f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.097+0000 7fd116e00640 1 --2- 172.21.15.32:0/3479672642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd110109070 0x7fd110109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.097+0000 7fd116e00640 1 --2- 172.21.15.32:0/3479672642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd110103730 0x7fd110103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.097+0000 7fd116e00640 1 -- 172.21.15.32:0/3479672642 >> 172.21.15.32:0/3479672642 conn(0x7fd1100ff180 msgr2=0x7fd1101015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:57.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.097+0000 7fd116e00640 1 -- 172.21.15.32:0/3479672642 shutdown_connections 2024-08-31T07:35:57.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.097+0000 7fd116e00640 1 -- 172.21.15.32:0/3479672642 wait complete. 2024-08-31T07:35:57.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.098+0000 7fd116e00640 1 Processor -- start 2024-08-31T07:35:57.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.099+0000 7fd116e00640 1 -- start start 2024-08-31T07:35:57.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.099+0000 7fd116e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd110103730 0x7fd11019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.099+0000 7fd116e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd110109070 0x7fd11019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.099+0000 7fd116e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1101099c0 0x7fd1101a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.099+0000 7fd116e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd110113690 con 0x7fd110109070 2024-08-31T07:35:57.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.099+0000 7fd116e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd110113510 con 0x7fd1101099c0 2024-08-31T07:35:57.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.099+0000 7fd116e00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd110113810 con 0x7fd110103730 2024-08-31T07:35:57.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.099+0000 7fd115000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd110109070 0x7fd11019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.100+0000 7fd115000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd110109070 0x7fd11019c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54332/0 (socket says 172.21.15.32:54332) 2024-08-31T07:35:57.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.100+0000 7fd115000640 1 -- 172.21.15.32:0/1507463427 learned_addr learned my addr 172.21.15.32:0/1507463427 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:57.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.100+0000 7fd115a00640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd110103730 0x7fd11019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.100+0000 7fd116400640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1101099c0 0x7fd1101a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.100+0000 7fd115000640 1 -- 172.21.15.32:0/1507463427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd110103730 msgr2=0x7fd11019c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.100+0000 7fd115000640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd110103730 0x7fd11019c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.100+0000 7fd115000640 1 -- 172.21.15.32:0/1507463427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1101099c0 msgr2=0x7fd1101a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.100+0000 7fd115000640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1101099c0 0x7fd1101a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.100+0000 7fd115000640 1 -- 172.21.15.32:0/1507463427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1101a2210 con 0x7fd110109070 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.100+0000 7fd115000640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd110109070 0x7fd11019c840 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00e9c0 tx=0x7fd0fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.101+0000 7fd10e000640 1 -- 172.21.15.32:0/1507463427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc00ccc0 con 0x7fd110109070 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.101+0000 7fd116e00640 1 -- 172.21.15.32:0/1507463427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1101a2500 con 0x7fd110109070 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.101+0000 7fd10e000640 1 -- 172.21.15.32:0/1507463427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0fc0054d0 con 0x7fd110109070 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.101+0000 7fd10e000640 1 -- 172.21.15.32:0/1507463427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc010430 con 0x7fd110109070 2024-08-31T07:35:57.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.101+0000 7fd116e00640 1 -- 172.21.15.32:0/1507463427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1101a8c60 con 0x7fd110109070 2024-08-31T07:35:57.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.104+0000 7fd11f1fa740 1 Processor -- start 2024-08-31T07:35:57.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.104+0000 7fd11f1fa740 1 -- start start 2024-08-31T07:35:57.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 0x56370b3e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 0x56370b3e5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 0x56370b3e55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56370b35b770 con 0x56370b01e390 2024-08-31T07:35:57.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56370b34ad00 con 0x56370b0161b0 2024-08-31T07:35:57.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56370b34ecb0 con 0x56370b03f070 2024-08-31T07:35:57.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd10e000640 1 -- 172.21.15.32:0/1507463427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd0fc005020 con 0x7fd110109070 2024-08-31T07:35:57.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd115a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 0x56370b3e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd115a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 0x56370b3e5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50210/0 (socket says 172.21.15.32:50210) 2024-08-31T07:35:57.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd115a00640 1 -- 172.21.15.32:0/780003970 learned_addr learned my addr 172.21.15.32:0/780003970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:57.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd116400640 1 --2- 172.21.15.32:0/780003970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 0x56370b3e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd115000640 1 --2- 172.21.15.32:0/780003970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 0x56370b3e55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd116400640 1 -- 172.21.15.32:0/780003970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 msgr2=0x56370b3e5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:57.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd116400640 1 --2- 172.21.15.32:0/780003970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 0x56370b3e5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd116400640 1 -- 172.21.15.32:0/780003970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 msgr2=0x56370b3e55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:57.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd116400640 1 --2- 172.21.15.32:0/780003970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 0x56370b3e55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.105+0000 7fd116400640 1 -- 172.21.15.32:0/780003970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56370b0aa650 con 0x56370b03f070 2024-08-31T07:35:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.106+0000 7fd115a00640 1 --2- 172.21.15.32:0/780003970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 0x56370b3e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.106+0000 7fd116400640 1 --2- 172.21.15.32:0/780003970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 0x56370b3e4b20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd108005e30 tx=0x7fd108009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ff197c23c81cb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.106+0000 7fd102a00640 1 -- 172.21.15.32:0/780003970 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1080305a0 con 0x56370b03f070 2024-08-31T07:35:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.106+0000 7fd102a00640 1 -- 172.21.15.32:0/780003970 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd108036030 con 0x56370b03f070 2024-08-31T07:35:57.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.106+0000 7fd102a00640 1 -- 172.21.15.32:0/780003970 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1080042a0 con 0x56370b03f070 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.107+0000 7fd11f1fa740 1 -- 172.21.15.32:0/780003970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 msgr2=0x56370b3e4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.107+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/780003970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 0x56370b3e4b20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd108005e30 tx=0x7fd108009800 comp rx=0 tx=0).stop 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.107+0000 7fd10e000640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0f80b4d20 0x7fd0f80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.107+0000 7fd11f1fa740 1 -- 172.21.15.32:0/780003970 shutdown_connections 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.107+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/780003970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 0x56370b3e55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.107+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/780003970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 0x56370b3e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.107+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/780003970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 0x56370b3e4b20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.107+0000 7fd11f1fa740 1 -- 172.21.15.32:0/780003970 >> 172.21.15.32:0/780003970 conn(0x56370b033460 msgr2=0x56370b3e2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.107+0000 7fd10e000640 1 -- 172.21.15.32:0/1507463427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fd0fc0e5800 con 0x7fd110109070 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.108+0000 7fd11f1fa740 1 -- 172.21.15.32:0/780003970 shutdown_connections 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.108+0000 7fd116400640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0f80b4d20 0x7fd0f80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.108+0000 7fd11f1fa740 1 -- 172.21.15.32:0/780003970 wait complete. 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.109+0000 7fd11f1fa740 1 -- start start 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.109+0000 7fd116400640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0f80b4d20 0x7fd0f80b7230 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd1080094e0 tx=0x7fd10802a8b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.109+0000 7fd11f1fa740 1 Processor -- start 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.109+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b0161b0 0x56370b3f0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.109+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b01e390 0x56370b3f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.109+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 0x56370b3fc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.109+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56370b1ad700 con 0x56370b0161b0 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.109+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56370b2d1cb0 con 0x56370b01e390 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.109+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56370b3f32a0 con 0x56370b03f070 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd11f1fa740 10 client.0 _ll_register_callbacks cb 0x56370b2f7430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd115000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b01e390 0x56370b3f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd115a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b0161b0 0x56370b3f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd115000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b01e390 0x56370b3f0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50220/0 (socket says 172.21.15.32:50220) 2024-08-31T07:35:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd115000640 1 -- 172.21.15.32:0/3240986212 learned_addr learned my addr 172.21.15.32:0/3240986212 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd116400640 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 0x56370b3fc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd115000640 1 -- 172.21.15.32:0/3240986212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 msgr2=0x56370b3fc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd115000640 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 0x56370b3fc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd115000640 1 -- 172.21.15.32:0/3240986212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b0161b0 msgr2=0x56370b3f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd115000640 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b0161b0 0x56370b3f0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.110+0000 7fd115000640 1 -- 172.21.15.32:0/3240986212 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56370b312870 con 0x56370b01e390 2024-08-31T07:35:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.111+0000 7fd115000640 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b01e390 0x56370b3f0b80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc004650 tx=0x7fd0fc0ab750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138b32b5612fdeaa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.111+0000 7fd0f7400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:35:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.111+0000 7fd0f7400640 1 -- 172.21.15.32:0/3240986212 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0e33d0 con 0x56370b01e390 2024-08-31T07:35:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.111+0000 7fd0f7400640 1 -- 172.21.15.32:0/3240986212 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fd0fc0b0360 con 0x56370b01e390 2024-08-31T07:35:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.111+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3240986212 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd108036030 con 0x56370b01e390 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.111+0000 7fd0f7400640 1 -- 172.21.15.32:0/3240986212 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc0d2020 con 0x56370b01e390 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.111+0000 7fd11f1fa740 10 client.29828 fetch_fsmap learned FSMap version 90 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.111+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3240986212 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x56370b3ff2a0 con 0x56370b01e390 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.111+0000 7fd0f7400640 1 -- 172.21.15.32:0/3240986212 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd108036030 con 0x56370b01e390 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.111+0000 7fd0f7400640 1 -- 172.21.15.32:0/3240986212 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7fd0fc0b4d70 con 0x56370b01e390 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.112+0000 7fd11f1fa740 10 client.29828 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.112+0000 7fd11f1fa740 10 client.29828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.112+0000 7fd11f1fa740 20 client.29828 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.112+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56370affd9f0 0x56370b4027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.112+0000 7fd11f1fa740 4 client.29828 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.112+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3240986212 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56370b35d6c0 con 0x56370affd9f0 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.112+0000 7fd115a00640 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56370affd9f0 0x56370b4027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.113+0000 7fd115a00640 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56370affd9f0 0x56370b4027c0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5af243a8433058 server_cookie=4599dfed5c124f7f in_seq=0 out_seq=0 2024-08-31T07:35:57.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.113+0000 7fd0f7400640 10 client.29828 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:57.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd0f7400640 1 -- 172.21.15.32:0/3240986212 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56370b35d6c0 con 0x56370affd9f0 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd0f7400640 10 client.29828 handle_command_reply: tid=0 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 client.29828 shutdown 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3240986212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b01e390 msgr2=0x56370b3f0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b01e390 0x56370b3f0b80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc004650 tx=0x7fd0fc0ab750 comp rx=0 tx=0).stop 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3240986212 shutdown_connections 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56370affd9f0 0x56370b4027c0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b03f070 0x56370b3fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b01e390 0x56370b3f0b80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/3240986212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b0161b0 0x56370b3f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd0f7400640 0 client.29828 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3240986212 >> 172.21.15.32:0/3240986212 conn(0x56370b033460 msgr2=0x56370b3e7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3240986212 shutdown_connections 2024-08-31T07:35:57.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.114+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3240986212 wait complete. 2024-08-31T07:35:57.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.115+0000 7fd11f1fa740 20 client.29828 trim_cache size 0 max 16384 2024-08-31T07:35:57.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.133+0000 7fd11f1fa740 1 Processor -- start 2024-08-31T07:35:57.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.133+0000 7fd11f1fa740 1 -- start start 2024-08-31T07:35:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b0161b0 0x56370b3e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 0x56370b3e8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b03f070 0x56370b3ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b1370 con 0x56370b01e390 2024-08-31T07:35:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b3030 con 0x56370b03f070 2024-08-31T07:35:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b1a40 con 0x56370b0161b0 2024-08-31T07:35:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd115000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 0x56370b3e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd115000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 0x56370b3e8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54350/0 (socket says 172.21.15.32:54350) 2024-08-31T07:35:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd115000640 1 -- 172.21.15.32:0/3138692485 learned_addr learned my addr 172.21.15.32:0/3138692485 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd116400640 1 --2- 172.21.15.32:0/3138692485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b03f070 0x56370b3ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd115a00640 1 --2- 172.21.15.32:0/3138692485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b0161b0 0x56370b3e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd115000640 1 -- 172.21.15.32:0/3138692485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b0161b0 msgr2=0x56370b3e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:57.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd115000640 1 --2- 172.21.15.32:0/3138692485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b0161b0 0x56370b3e2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd115000640 1 -- 172.21.15.32:0/3138692485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b03f070 msgr2=0x56370b3ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:57.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd115000640 1 --2- 172.21.15.32:0/3138692485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b03f070 0x56370b3ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.134+0000 7fd115000640 1 -- 172.21.15.32:0/3138692485 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56370b3eb0d0 con 0x56370b01e390 2024-08-31T07:35:57.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.135+0000 7fd115000640 1 --2- 172.21.15.32:0/3138692485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 0x56370b3e8330 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0abbe0 tx=0x7fd0fc0ab8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b96f444d8b92c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:57.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.135+0000 7fd102000640 1 -- 172.21.15.32:0/3138692485 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0c5030 con 0x56370b01e390 2024-08-31T07:35:57.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.135+0000 7fd102000640 1 -- 172.21.15.32:0/3138692485 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0fc005680 con 0x56370b01e390 2024-08-31T07:35:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.135+0000 7fd102000640 1 -- 172.21.15.32:0/3138692485 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0b5200 con 0x56370b01e390 2024-08-31T07:35:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.136+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3138692485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 msgr2=0x56370b3e8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.136+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/3138692485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 0x56370b3e8330 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0abbe0 tx=0x7fd0fc0ab8e0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.136+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3138692485 shutdown_connections 2024-08-31T07:35:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.136+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/3138692485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b03f070 0x56370b3ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.136+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/3138692485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b01e390 0x56370b3e8330 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.136+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/3138692485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b0161b0 0x56370b3e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.136+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3138692485 >> 172.21.15.32:0/3138692485 conn(0x56370b033460 msgr2=0x56370b3e7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.136+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3138692485 shutdown_connections 2024-08-31T07:35:57.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.136+0000 7fd11f1fa740 1 -- 172.21.15.32:0/3138692485 wait complete. 2024-08-31T07:35:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.137+0000 7fd11f1fa740 1 -- start start 2024-08-31T07:35:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.137+0000 7fd11f1fa740 1 Processor -- start 2024-08-31T07:35:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.137+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 0x56370b3f3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.137+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b01e390 0x56370b3f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.137+0000 7fd11f1fa740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b03f070 0x56370b46d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.137+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b28d0 con 0x56370b03f070 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.137+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101b2140 con 0x56370b0161b0 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.137+0000 7fd11f1fa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101af370 con 0x56370b01e390 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.137+0000 7fd11f1fa740 10 client.0 _ll_register_callbacks cb 0x7fd0fc0ab830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd115000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b01e390 0x56370b3f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd116400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b03f070 0x56370b46d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd115000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b01e390 0x56370b3f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36750/0 (socket says 172.21.15.32:36750) 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd115000640 1 -- 172.21.15.32:0/236904476 learned_addr learned my addr 172.21.15.32:0/236904476 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd115a00640 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 0x56370b3f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd115000640 1 -- 172.21.15.32:0/236904476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 msgr2=0x56370b3f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd115000640 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 0x56370b3f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd115000640 1 -- 172.21.15.32:0/236904476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b03f070 msgr2=0x56370b46d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd115000640 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b03f070 0x56370b46d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd115000640 1 -- 172.21.15.32:0/236904476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56370b3f46d0 con 0x56370b01e390 2024-08-31T07:35:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd115000640 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b01e390 0x56370b3f3e50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc004650 tx=0x7fd0fc0b0610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d64f680c953b481 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.138+0000 7fd0f7400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.139+0000 7fd0f7400640 1 -- 172.21.15.32:0/236904476 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0c5070 con 0x56370b01e390 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.139+0000 7fd0f7400640 1 -- 172.21.15.32:0/236904476 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fd0fc0932c0 con 0x56370b01e390 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.139+0000 7fd11f1fa740 1 -- 172.21.15.32:0/236904476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0fc005680 con 0x56370b01e390 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.139+0000 7fd0f7400640 1 -- 172.21.15.32:0/236904476 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc0ca030 con 0x56370b01e390 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.139+0000 7fd11f1fa740 10 client.29839 fetch_fsmap learned FSMap version 90 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.139+0000 7fd11f1fa740 1 -- 172.21.15.32:0/236904476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x56370b46e710 con 0x56370b01e390 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.139+0000 7fd0f7400640 1 -- 172.21.15.32:0/236904476 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0fc005680 con 0x56370b01e390 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.139+0000 7fd0f7400640 1 -- 172.21.15.32:0/236904476 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7fd0fc0a7850 con 0x56370b01e390 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.139+0000 7fd11f1fa740 10 client.29839 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.139+0000 7fd11f1fa740 10 client.29839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.140+0000 7fd11f1fa740 20 client.29839 populate_metadata read hostname 'smithi032' 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.140+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56370afd2da0 0x56370b470cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.140+0000 7fd11f1fa740 4 client.29839 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:35:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.140+0000 7fd11f1fa740 1 -- 172.21.15.32:0/236904476 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56370b359160 con 0x56370afd2da0 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.140+0000 7fd115a00640 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56370afd2da0 0x56370b470cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd115a00640 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56370afd2da0 0x56370b470cb0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848e48f8207a5333 server_cookie=4a53ad834dfb83b4 in_seq=0 out_seq=0 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd0f7400640 10 client.29839 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd0f7400640 1 -- 172.21.15.32:0/236904476 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7190 (crc 0 0 0) 0x56370b359160 con 0x56370afd2da0 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd0f7400640 10 client.29839 handle_command_reply: tid=0 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd11f1fa740 1 client.29839 shutdown 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd11f1fa740 1 -- 172.21.15.32:0/236904476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b01e390 msgr2=0x56370b3f3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b01e390 0x56370b3f3e50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc004650 tx=0x7fd0fc0b0610 comp rx=0 tx=0).stop 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd11f1fa740 1 -- 172.21.15.32:0/236904476 shutdown_connections 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x56370afd2da0 0x56370b470cb0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56370b03f070 0x56370b46d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56370b01e390 0x56370b3f3e50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd11f1fa740 1 --2- 172.21.15.32:0/236904476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56370b0161b0 0x56370b3f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd0f7400640 0 client.29839 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:35:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd11f1fa740 1 -- 172.21.15.32:0/236904476 >> 172.21.15.32:0/236904476 conn(0x56370b033460 msgr2=0x56370b3e2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.141+0000 7fd11f1fa740 1 -- 172.21.15.32:0/236904476 shutdown_connections 2024-08-31T07:35:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd11f1fa740 1 -- 172.21.15.32:0/236904476 wait complete. 2024-08-31T07:35:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd11f1fa740 20 client.29839 trim_cache size 0 max 16384 2024-08-31T07:35:57.146 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:35:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:35:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:35:57.146 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "d83ecd52-a6c2-49e8-bdaa-039848b73de4": { 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:34:26.167734+0000", 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:32:41.165270+0000", 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.147 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.148 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "9e14a084-7459-442a-abc0-3ac74b83528a": { 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:35:22.931369+0000", 2024-08-31T07:35:57.149 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:35:57.150 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:35:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:35:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:35:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.151 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.153 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.154 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:35:57.155 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:35:57.156 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:35:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:35:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:35:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.157 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:35:57.158 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:35:57.159 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.160 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:35:57.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd116e00640 1 -- 172.21.15.32:0/1507463427 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0f80b4d20 msgr2=0x7fd0f80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd116e00640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0f80b4d20 0x7fd0f80b7230 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd1080094e0 tx=0x7fd10802a8b0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd116e00640 1 -- 172.21.15.32:0/1507463427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd110109070 msgr2=0x7fd11019c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd116e00640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd110109070 0x7fd11019c840 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00e9c0 tx=0x7fd0fc00ee80 comp rx=0 tx=0).stop 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd116e00640 1 -- 172.21.15.32:0/1507463427 shutdown_connections 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd116e00640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd0f80b4d20 0x7fd0f80b7230 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd116e00640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1101099c0 0x7fd1101a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd116e00640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd110109070 0x7fd11019c840 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd116e00640 1 --2- 172.21.15.32:0/1507463427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd110103730 0x7fd11019c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.142+0000 7fd116e00640 1 -- 172.21.15.32:0/1507463427 >> 172.21.15.32:0/1507463427 conn(0x7fd1100ff180 msgr2=0x7fd1101086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.143+0000 7fd116e00640 1 -- 172.21.15.32:0/1507463427 shutdown_connections 2024-08-31T07:35:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:57.143+0000 7fd116e00640 1 -- 172.21.15.32:0/1507463427 wait complete. 2024-08-31T07:35:57.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:56 smithi073 ceph-mon[21995]: pgmap v1097: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 4.2 MiB/s wr, 139 op/s 2024-08-31T07:35:57.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.15 MB/sec execute 457 sec latency 19997.427 ms 2024-08-31T07:35:58.034 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:35:58.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:35:58.423 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:35:58.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.12 MB/sec execute 458 sec latency 20997.488 ms 2024-08-31T07:35:58.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:58 smithi032 ceph-mon[17252]: pgmap v1098: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.5 MiB/s wr, 109 op/s 2024-08-31T07:35:58.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:58 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:35:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:58 smithi084 ceph-mon[22387]: pgmap v1098: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.5 MiB/s wr, 109 op/s 2024-08-31T07:35:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:58 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:35:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.165+0000 7f5c24a00640 1 -- 172.21.15.32:0/3776160043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c20103750 msgr2=0x7f5c20103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.165+0000 7f5c24a00640 1 --2- 172.21.15.32:0/3776160043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c20103750 0x7f5c20103b60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009900 tx=0x7f5c1002a400 comp rx=0 tx=0).stop 2024-08-31T07:35:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.165+0000 7f5c24a00640 1 -- 172.21.15.32:0/3776160043 shutdown_connections 2024-08-31T07:35:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.165+0000 7f5c24a00640 1 --2- 172.21.15.32:0/3776160043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c201099e0 0x7f5c20110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.165+0000 7f5c24a00640 1 --2- 172.21.15.32:0/3776160043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c20109090 0x7f5c201094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.165+0000 7f5c24a00640 1 --2- 172.21.15.32:0/3776160043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c20103750 0x7f5c20103b60 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:59.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.165+0000 7f5c24a00640 1 -- 172.21.15.32:0/3776160043 >> 172.21.15.32:0/3776160043 conn(0x7f5c200ff1e0 msgr2=0x7f5c20101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:59.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.165+0000 7f5c24a00640 1 -- 172.21.15.32:0/3776160043 shutdown_connections 2024-08-31T07:35:59.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.165+0000 7f5c24a00640 1 -- 172.21.15.32:0/3776160043 wait complete. 2024-08-31T07:35:59.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.166+0000 7f5c24a00640 1 Processor -- start 2024-08-31T07:35:59.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.166+0000 7f5c24a00640 1 -- start start 2024-08-31T07:35:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c24a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c20103750 0x7f5c20076ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c24a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c20109090 0x7f5c200754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c24a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c201099e0 0x7f5c200759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c24a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20113690 con 0x7f5c20109090 2024-08-31T07:35:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c24a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20113510 con 0x7f5c20103750 2024-08-31T07:35:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c24a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20113810 con 0x7f5c201099e0 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c1de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c20109090 0x7f5c200754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c1e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c20103750 0x7f5c20076ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c1de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c20109090 0x7f5c200754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54378/0 (socket says 172.21.15.32:54378) 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c1de00640 1 -- 172.21.15.32:0/3997198021 learned_addr learned my addr 172.21.15.32:0/3997198021 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.167+0000 7f5c1e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c20103750 0x7f5c20076ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:50252/0 (socket says 172.21.15.32:50252) 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.168+0000 7f5c1f200640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c201099e0 0x7f5c200759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.168+0000 7f5c1de00640 1 -- 172.21.15.32:0/3997198021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c201099e0 msgr2=0x7f5c200759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.168+0000 7f5c1de00640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c201099e0 0x7f5c200759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.168+0000 7f5c1de00640 1 -- 172.21.15.32:0/3997198021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c20103750 msgr2=0x7f5c20076ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.168+0000 7f5c1de00640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c20103750 0x7f5c20076ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.168+0000 7f5c1de00640 1 -- 172.21.15.32:0/3997198021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c20076210 con 0x7f5c20109090 2024-08-31T07:35:59.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.168+0000 7f5c1e800640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c20103750 0x7f5c20076ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:35:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.168+0000 7f5c1de00640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c20109090 0x7f5c200754b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00e970 tx=0x7f5c0c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.169+0000 7f5c17400640 1 -- 172.21.15.32:0/3997198021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c00cd80 con 0x7f5c20109090 2024-08-31T07:35:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.169+0000 7f5c17400640 1 -- 172.21.15.32:0/3997198021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c0c0035f0 con 0x7f5c20109090 2024-08-31T07:35:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.169+0000 7f5c24a00640 1 -- 172.21.15.32:0/3997198021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c201aca80 con 0x7f5c20109090 2024-08-31T07:35:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.169+0000 7f5c17400640 1 -- 172.21.15.32:0/3997198021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c010690 con 0x7f5c20109090 2024-08-31T07:35:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.169+0000 7f5c24a00640 1 -- 172.21.15.32:0/3997198021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c201acfc0 con 0x7f5c20109090 2024-08-31T07:35:59.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.170+0000 7f5c24a00640 1 -- 172.21.15.32:0/3997198021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c20103b60 con 0x7f5c20109090 2024-08-31T07:35:59.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.174+0000 7f5c17400640 1 -- 172.21.15.32:0/3997198021 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5c0c003aa0 con 0x7f5c20109090 2024-08-31T07:35:59.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.175+0000 7f5c17400640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4ca0 0x7f5c040b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:35:59.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.175+0000 7f5c17400640 1 -- 172.21.15.32:0/3997198021 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f5c0c0e5850 con 0x7f5c20109090 2024-08-31T07:35:59.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.176+0000 7f5c1e800640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4ca0 0x7f5c040b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:35:59.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.177+0000 7f5c1e800640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4ca0 0x7f5c040b71b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5c10000c00 tx=0x7f5c10031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:35:59.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:58 smithi073 ceph-mon[21995]: pgmap v1098: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.5 MiB/s wr, 109 op/s 2024-08-31T07:35:59.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:58 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:35:59.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.179+0000 7f5c17400640 1 -- 172.21.15.32:0/3997198021 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5c0c0ac690 con 0x7f5c20109090 2024-08-31T07:35:59.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.416+0000 7f5c24a00640 1 -- 172.21.15.32:0/3997198021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c200767b0 con 0x7f5c20109090 2024-08-31T07:35:59.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.417+0000 7f5c17400640 1 -- 172.21.15.32:0/3997198021 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+11973 (secure 0 0 0) 0x7f5c0c0abf90 con 0x7f5c20109090 2024-08-31T07:35:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:35:59.419 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":90,"btime":"2024-08-31T07:35:50:675689+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":90,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:35:50.139610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:35:59.419 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 90 2024-08-31T07:35:59.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 -- 172.21.15.32:0/3997198021 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4ca0 msgr2=0x7f5c040b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4ca0 0x7f5c040b71b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5c10000c00 tx=0x7f5c10031400 comp rx=0 tx=0).stop 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 -- 172.21.15.32:0/3997198021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c20109090 msgr2=0x7f5c200754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c20109090 0x7f5c200754b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00e970 tx=0x7f5c0c00ee30 comp rx=0 tx=0).stop 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 -- 172.21.15.32:0/3997198021 shutdown_connections 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f5c040b4ca0 0x7f5c040b71b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c201099e0 0x7f5c200759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c20109090 0x7f5c200754b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 --2- 172.21.15.32:0/3997198021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c20103750 0x7f5c20076ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 -- 172.21.15.32:0/3997198021 >> 172.21.15.32:0/3997198021 conn(0x7f5c200ff1e0 msgr2=0x7f5c20101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 -- 172.21.15.32:0/3997198021 shutdown_connections 2024-08-31T07:35:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:35:59.420+0000 7f5c24a00640 1 -- 172.21.15.32:0/3997198021 wait complete. 2024-08-31T07:35:59.426 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.09 MB/sec execute 459 sec latency 21997.545 ms 2024-08-31T07:35:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:35:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:35:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:35:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:35:59 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3997198021' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:36:00.080 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:36:00.080 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:36:00.080 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:36:00.080 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:35:59 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3997198021' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:36:00.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:36:00.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:36:00.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:36:00.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:35:59 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3997198021' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:36:00.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:36:00.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.06 MB/sec execute 460 sec latency 22997.608 ms 2024-08-31T07:36:00.646 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:00.833 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:00 smithi032 ceph-mon[17252]: pgmap v1099: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-08-31T07:36:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:00 smithi084 ceph-mon[22387]: pgmap v1099: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-08-31T07:36:01.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:00 smithi073 ceph-mon[21995]: pgmap v1099: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-08-31T07:36:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.326+0000 7fa0dee00640 1 -- 172.21.15.32:0/1409054098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d8105950 msgr2=0x7fa0d8105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.326+0000 7fa0dee00640 1 --2- 172.21.15.32:0/1409054098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d8105950 0x7fa0d8105d60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009900 tx=0x7fa0cc02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.326+0000 7fa0dee00640 1 -- 172.21.15.32:0/1409054098 shutdown_connections 2024-08-31T07:36:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.326+0000 7fa0dee00640 1 --2- 172.21.15.32:0/1409054098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0d810bbe0 0x7fa0d8112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.326+0000 7fa0dee00640 1 --2- 172.21.15.32:0/1409054098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d810b290 0x7fa0d810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.326+0000 7fa0dee00640 1 --2- 172.21.15.32:0/1409054098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d8105950 0x7fa0d8105d60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.326+0000 7fa0dee00640 1 -- 172.21.15.32:0/1409054098 >> 172.21.15.32:0/1409054098 conn(0x7fa0d8101420 msgr2=0x7fa0d8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.326+0000 7fa0dee00640 1 -- 172.21.15.32:0/1409054098 shutdown_connections 2024-08-31T07:36:01.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.326+0000 7fa0dee00640 1 -- 172.21.15.32:0/1409054098 wait complete. 2024-08-31T07:36:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.327+0000 7fa0dee00640 1 Processor -- start 2024-08-31T07:36:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dee00640 1 -- start start 2024-08-31T07:36:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dee00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d8105950 0x7fa0d819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0d810b290 0x7fa0d819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d810bbe0 0x7fa0d81a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81158a0 con 0x7fa0d810bbe0 2024-08-31T07:36:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8115720 con 0x7fa0d810b290 2024-08-31T07:36:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dee00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8115a20 con 0x7fa0d8105950 2024-08-31T07:36:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dda00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d8105950 0x7fa0d819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0d810b290 0x7fa0d819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0de400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d810bbe0 0x7fa0d81a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dda00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d8105950 0x7fa0d819c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36774/0 (socket says 172.21.15.32:36774) 2024-08-31T07:36:01.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.328+0000 7fa0dda00640 1 -- 172.21.15.32:0/425870231 learned_addr learned my addr 172.21.15.32:0/425870231 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:01.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.329+0000 7fa0dd000640 1 -- 172.21.15.32:0/425870231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d8105950 msgr2=0x7fa0d819c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:01.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.329+0000 7fa0dd000640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d8105950 0x7fa0d819c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.329+0000 7fa0dd000640 1 -- 172.21.15.32:0/425870231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d810bbe0 msgr2=0x7fa0d81a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:01.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.329+0000 7fa0dd000640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d810bbe0 0x7fa0d81a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.329+0000 7fa0dd000640 1 -- 172.21.15.32:0/425870231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d81a2240 con 0x7fa0d810b290 2024-08-31T07:36:01.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.329+0000 7fa0dd000640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0d810b290 0x7fa0d819c870 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa0d806c660 tx=0x7fa0c400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:01.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.329+0000 7fa0d6000640 1 -- 172.21.15.32:0/425870231 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400cc40 con 0x7fa0d810b290 2024-08-31T07:36:01.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.329+0000 7fa0dee00640 1 -- 172.21.15.32:0/425870231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d81a2530 con 0x7fa0d810b290 2024-08-31T07:36:01.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.330+0000 7fa0d6000640 1 -- 172.21.15.32:0/425870231 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c400cde0 con 0x7fa0d810b290 2024-08-31T07:36:01.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.330+0000 7fa0d6000640 1 -- 172.21.15.32:0/425870231 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4018760 con 0x7fa0d810b290 2024-08-31T07:36:01.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.330+0000 7fa0dee00640 1 -- 172.21.15.32:0/425870231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d81a8c90 con 0x7fa0d810b290 2024-08-31T07:36:01.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.333+0000 7fa0e7096740 1 Processor -- start 2024-08-31T07:36:01.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.333+0000 7fa0e7096740 1 -- start start 2024-08-31T07:36:01.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.333+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 0x563618da9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.333+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618da9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.333+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636189e3390 0x563618daa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.333+0000 7fa0e7096740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563618d207b0 con 0x5636189db1b0 2024-08-31T07:36:01.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.333+0000 7fa0e7096740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563618d0fd40 con 0x563618a04070 2024-08-31T07:36:01.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.333+0000 7fa0e7096740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563618d13cf0 con 0x5636189e3390 2024-08-31T07:36:01.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.333+0000 7fa0d6000640 1 -- 172.21.15.32:0/425870231 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa0c4010a90 con 0x7fa0d810b290 2024-08-31T07:36:01.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0dda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618da9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0dda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618da9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54398/0 (socket says 172.21.15.32:54398) 2024-08-31T07:36:01.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0dda00640 1 -- 172.21.15.32:0/2984008299 learned_addr learned my addr 172.21.15.32:0/2984008299 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:01.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0de400640 1 --2- 172.21.15.32:0/2984008299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 0x563618da9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0dd000640 1 --2- 172.21.15.32:0/2984008299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636189e3390 0x563618daa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0dda00640 1 -- 172.21.15.32:0/2984008299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636189e3390 msgr2=0x563618daa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:01.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0dda00640 1 --2- 172.21.15.32:0/2984008299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636189e3390 0x563618daa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0dda00640 1 -- 172.21.15.32:0/2984008299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 msgr2=0x563618da9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:01.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0dda00640 1 --2- 172.21.15.32:0/2984008299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 0x563618da9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0dda00640 1 -- 172.21.15.32:0/2984008299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563618a6f650 con 0x5636189db1b0 2024-08-31T07:36:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.334+0000 7fa0de400640 1 --2- 172.21.15.32:0/2984008299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 0x563618da9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.335+0000 7fa0dda00640 1 --2- 172.21.15.32:0/2984008299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618da9fb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc000c00 tx=0x7fa0cc031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa22e7e2aac40355 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.335+0000 7fa0caa00640 1 -- 172.21.15.32:0/2984008299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc040070 con 0x5636189db1b0 2024-08-31T07:36:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.335+0000 7fa0caa00640 1 -- 172.21.15.32:0/2984008299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc004030 con 0x5636189db1b0 2024-08-31T07:36:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.335+0000 7fa0caa00640 1 -- 172.21.15.32:0/2984008299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc035040 con 0x5636189db1b0 2024-08-31T07:36:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.335+0000 7fa0d6000640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0c00b4da0 0x7fa0c00b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.336+0000 7fa0d6000640 1 -- 172.21.15.32:0/425870231 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fa0c40e5780 con 0x7fa0d810b290 2024-08-31T07:36:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.336+0000 7fa0de400640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0c00b4da0 0x7fa0c00b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.336+0000 7fa0e7096740 1 -- 172.21.15.32:0/2984008299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 msgr2=0x563618da9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.336+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2984008299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618da9fb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc000c00 tx=0x7fa0cc031b10 comp rx=0 tx=0).stop 2024-08-31T07:36:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.336+0000 7fa0e7096740 1 -- 172.21.15.32:0/2984008299 shutdown_connections 2024-08-31T07:36:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.336+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2984008299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636189e3390 0x563618daa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.336+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2984008299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618da9fb0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.336+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2984008299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 0x563618da9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.336+0000 7fa0e7096740 1 -- 172.21.15.32:0/2984008299 >> 172.21.15.32:0/2984008299 conn(0x5636189f8460 msgr2=0x563618da7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.336+0000 7fa0e7096740 1 -- 172.21.15.32:0/2984008299 shutdown_connections 2024-08-31T07:36:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.337+0000 7fa0e7096740 1 -- 172.21.15.32:0/2984008299 wait complete. 2024-08-31T07:36:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.337+0000 7fa0de400640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0c00b4da0 0x7fa0c00b72b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009fd0 tx=0x7fa0d0009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.337+0000 7fa0e7096740 1 -- start start 2024-08-31T07:36:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.337+0000 7fa0e7096740 1 Processor -- start 2024-08-31T07:36:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618db5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636189e3390 0x563618db5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618a04070 0x563618dc14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0e7096740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563618b72700 con 0x5636189db1b0 2024-08-31T07:36:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0e7096740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563618c96cf0 con 0x5636189e3390 2024-08-31T07:36:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0e7096740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563618db81f0 con 0x563618a04070 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0e7096740 10 client.0 _ll_register_callbacks cb 0x563618cbc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0dda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618db5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0dda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618db5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54412/0 (socket says 172.21.15.32:54412) 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0dd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636189e3390 0x563618db5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0dda00640 1 -- 172.21.15.32:0/3319030179 learned_addr learned my addr 172.21.15.32:0/3319030179 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.338+0000 7fa0de400640 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618a04070 0x563618dc14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0dda00640 1 -- 172.21.15.32:0/3319030179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618a04070 msgr2=0x563618dc14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0dda00640 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618a04070 0x563618dc14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0dda00640 1 -- 172.21.15.32:0/3319030179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636189e3390 msgr2=0x563618db5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0dda00640 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636189e3390 0x563618db5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0dda00640 1 -- 172.21.15.32:0/3319030179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563618cd78b0 con 0x5636189db1b0 2024-08-31T07:36:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0dd000640 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636189e3390 0x563618db5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0dda00640 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618db5660 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc004490 tx=0x7fa0cc02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f93286fce67af3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0bf400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0bf400640 1 -- 172.21.15.32:0/3319030179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc04c020 con 0x5636189db1b0 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0bf400640 1 -- 172.21.15.32:0/3319030179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fa0cc0314a0 con 0x5636189db1b0 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.339+0000 7fa0e7096740 1 -- 172.21.15.32:0/3319030179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0cc004030 con 0x5636189db1b0 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.340+0000 7fa0bf400640 1 -- 172.21.15.32:0/3319030179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa0cc057050 con 0x5636189db1b0 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.340+0000 7fa0e7096740 10 client.19815 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.340+0000 7fa0e7096740 1 -- 172.21.15.32:0/3319030179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x563618dc41d0 con 0x5636189db1b0 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.340+0000 7fa0bf400640 1 -- 172.21.15.32:0/3319030179 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc004030 con 0x5636189db1b0 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.340+0000 7fa0bf400640 1 -- 172.21.15.32:0/3319030179 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7fa0cc004d40 con 0x5636189db1b0 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.340+0000 7fa0e7096740 10 client.19815 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.340+0000 7fa0e7096740 10 client.19815 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:36:01.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.340+0000 7fa0e7096740 20 client.19815 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:01.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.340+0000 7fa0e7096740 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5636189c29f0 0x563618dc7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.341+0000 7fa0e7096740 4 client.19815 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:36:01.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.341+0000 7fa0e7096740 1 -- 172.21.15.32:0/3319030179 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563618d22700 con 0x5636189c29f0 2024-08-31T07:36:01.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.341+0000 7fa0dd000640 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5636189c29f0 0x563618dc7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.341+0000 7fa0dd000640 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5636189c29f0 0x563618dc7530 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a24780eb7fa67a39 server_cookie=33dbb3fa6a9bee67 in_seq=0 out_seq=0 2024-08-31T07:36:01.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.341+0000 7fa0bf400640 10 client.19815 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:36:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.342+0000 7fa0bf400640 1 -- 172.21.15.32:0/3319030179 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563618d22700 con 0x5636189c29f0 2024-08-31T07:36:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.342+0000 7fa0bf400640 10 client.19815 handle_command_reply: tid=0 2024-08-31T07:36:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.342+0000 7fa0e7096740 1 client.19815 shutdown 2024-08-31T07:36:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.342+0000 7fa0e7096740 1 -- 172.21.15.32:0/3319030179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 msgr2=0x563618db5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.342+0000 7fa0e7096740 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618db5660 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc004490 tx=0x7fa0cc02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.343+0000 7fa0e7096740 1 -- 172.21.15.32:0/3319030179 shutdown_connections 2024-08-31T07:36:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.343+0000 7fa0e7096740 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x5636189c29f0 0x563618dc7530 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.343+0000 7fa0e7096740 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618a04070 0x563618dc14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.343+0000 7fa0e7096740 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636189e3390 0x563618db5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.343+0000 7fa0e7096740 1 --2- 172.21.15.32:0/3319030179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189db1b0 0x563618db5660 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.343+0000 7fa0e7096740 1 -- 172.21.15.32:0/3319030179 >> 172.21.15.32:0/3319030179 conn(0x5636189f8460 msgr2=0x563618dacc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.343+0000 7fa0bf400640 0 client.19815 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:36:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.343+0000 7fa0e7096740 1 -- 172.21.15.32:0/3319030179 shutdown_connections 2024-08-31T07:36:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.343+0000 7fa0e7096740 1 -- 172.21.15.32:0/3319030179 wait complete. 2024-08-31T07:36:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.343+0000 7fa0e7096740 20 client.19815 trim_cache size 0 max 16384 2024-08-31T07:36:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.357+0000 7fa0e7096740 1 Processor -- start 2024-08-31T07:36:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.357+0000 7fa0e7096740 1 -- start start 2024-08-31T07:36:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.357+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636189e3390 0x563618dc1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.357+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563618a04070 0x563618daeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.357+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618997da0 0x563618daf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.357+0000 7fa0e7096740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81b1450 con 0x563618a04070 2024-08-31T07:36:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.357+0000 7fa0e7096740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81b3110 con 0x563618997da0 2024-08-31T07:36:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.357+0000 7fa0e7096740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81b1b20 con 0x5636189e3390 2024-08-31T07:36:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0dd000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563618a04070 0x563618daeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0de400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618997da0 0x563618daf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0dda00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636189e3390 0x563618dc1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0dd000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563618a04070 0x563618daeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54418/0 (socket says 172.21.15.32:54418) 2024-08-31T07:36:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0dd000640 1 -- 172.21.15.32:0/2672976919 learned_addr learned my addr 172.21.15.32:0/2672976919 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0dd000640 1 -- 172.21.15.32:0/2672976919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636189e3390 msgr2=0x563618dc1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0dd000640 1 --2- 172.21.15.32:0/2672976919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636189e3390 0x563618dc1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0dd000640 1 -- 172.21.15.32:0/2672976919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618997da0 msgr2=0x563618daf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0dd000640 1 --2- 172.21.15.32:0/2672976919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618997da0 0x563618daf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0dd000640 1 -- 172.21.15.32:0/2672976919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563618db6000 con 0x563618a04070 2024-08-31T07:36:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.358+0000 7fa0de400640 1 --2- 172.21.15.32:0/2672976919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618997da0 0x563618daf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.359+0000 7fa0dd000640 1 --2- 172.21.15.32:0/2672976919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563618a04070 0x563618daeb80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400c6a0 tx=0x7fa0c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ce0303e31b05ec server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.359+0000 7fa0ca000640 1 -- 172.21.15.32:0/2672976919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40aba20 con 0x563618a04070 2024-08-31T07:36:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.359+0000 7fa0ca000640 1 -- 172.21.15.32:0/2672976919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c40abf10 con 0x563618a04070 2024-08-31T07:36:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.359+0000 7fa0ca000640 1 -- 172.21.15.32:0/2672976919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40e5bb0 con 0x563618a04070 2024-08-31T07:36:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 -- 172.21.15.32:0/2672976919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563618a04070 msgr2=0x563618daeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2672976919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563618a04070 0x563618daeb80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400c6a0 tx=0x7fa0c400cf90 comp rx=0 tx=0).stop 2024-08-31T07:36:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 -- 172.21.15.32:0/2672976919 shutdown_connections 2024-08-31T07:36:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2672976919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618997da0 0x563618daf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2672976919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563618a04070 0x563618daeb80 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2672976919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5636189e3390 0x563618dc1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 -- 172.21.15.32:0/2672976919 >> 172.21.15.32:0/2672976919 conn(0x5636189f8460 msgr2=0x563618db3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 -- 172.21.15.32:0/2672976919 shutdown_connections 2024-08-31T07:36:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 -- 172.21.15.32:0/2672976919 wait complete. 2024-08-31T07:36:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 -- start start 2024-08-31T07:36:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.360+0000 7fa0e7096740 1 Processor -- start 2024-08-31T07:36:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618997da0 0x563618dc8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189e3390 0x563618dc9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0e7096740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 0x563618e30cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0e7096740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81b29b0 con 0x5636189e3390 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0e7096740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81b2220 con 0x563618a04070 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0e7096740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81af450 con 0x563618997da0 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0e7096740 10 client.0 _ll_register_callbacks cb 0x7fa0cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0dd000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189e3390 0x563618dc9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0dda00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618997da0 0x563618dc8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0dd000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189e3390 0x563618dc9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54426/0 (socket says 172.21.15.32:54426) 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0dd000640 1 -- 172.21.15.32:0/2315977243 learned_addr learned my addr 172.21.15.32:0/2315977243 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.361+0000 7fa0de400640 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 0x563618e30cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.362+0000 7fa0dda00640 1 -- 172.21.15.32:0/2315977243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 msgr2=0x563618e30cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.362+0000 7fa0dda00640 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 0x563618e30cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.362+0000 7fa0dda00640 1 -- 172.21.15.32:0/2315977243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189e3390 msgr2=0x563618dc9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.362+0000 7fa0dda00640 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189e3390 0x563618dc9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.362+0000 7fa0dda00640 1 -- 172.21.15.32:0/2315977243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563618e31240 con 0x563618997da0 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.362+0000 7fa0dd000640 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189e3390 0x563618dc9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.362+0000 7fa0dda00640 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618997da0 0x563618dc8d40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc045000 tx=0x7fa0cc0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3179ab318ffb9c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.362+0000 7fa0bf400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.363+0000 7fa0bf400640 1 -- 172.21.15.32:0/2315977243 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc04c040 con 0x563618997da0 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.363+0000 7fa0bf400640 1 -- 172.21.15.32:0/2315977243 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fa0cc0318b0 con 0x563618997da0 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.363+0000 7fa0e7096740 1 -- 172.21.15.32:0/2315977243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c40abf10 con 0x563618997da0 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.363+0000 7fa0bf400640 1 -- 172.21.15.32:0/2315977243 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa0cc040070 con 0x563618997da0 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.363+0000 7fa0e7096740 10 client.29863 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.363+0000 7fa0e7096740 1 -- 172.21.15.32:0/2315977243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x563618e3acd0 con 0x563618997da0 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.363+0000 7fa0bf400640 1 -- 172.21.15.32:0/2315977243 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c40abf10 con 0x563618997da0 2024-08-31T07:36:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.364+0000 7fa0bf400640 1 -- 172.21.15.32:0/2315977243 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7fa0cc004770 con 0x563618997da0 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.364+0000 7fa0e7096740 10 client.29863 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.364+0000 7fa0e7096740 10 client.29863 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.364+0000 7fa0e7096740 20 client.29863 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.364+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x563618e3b3f0 0x563618e3d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.364+0000 7fa0e7096740 4 client.29863 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.364+0000 7fa0e7096740 1 -- 172.21.15.32:0/2315977243 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563618d1e1a0 con 0x563618e3b3f0 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.364+0000 7fa0dd000640 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x563618e3b3f0 0x563618e3d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.364+0000 7fa0dd000640 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x563618e3b3f0 0x563618e3d810 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5712f50d5ee21e51 server_cookie=32203b4d7baa839e in_seq=0 out_seq=0 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.364+0000 7fa0bf400640 10 client.29863 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0bf400640 1 -- 172.21.15.32:0/2315977243 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563618d1e1a0 con 0x563618e3b3f0 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0bf400640 10 client.29863 handle_command_reply: tid=0 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 client.29863 shutdown 2024-08-31T07:36:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 -- 172.21.15.32:0/2315977243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618997da0 msgr2=0x563618dc8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618997da0 0x563618dc8d40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc045000 tx=0x7fa0cc0056d0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 -- 172.21.15.32:0/2315977243 shutdown_connections 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x563618e3b3f0 0x563618e3d810 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563618a04070 0x563618e30cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5636189e3390 0x563618dc9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 --2- 172.21.15.32:0/2315977243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563618997da0 0x563618dc8d40 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0bf400640 0 client.29863 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 -- 172.21.15.32:0/2315977243 >> 172.21.15.32:0/2315977243 conn(0x5636189f8460 msgr2=0x563618daeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 -- 172.21.15.32:0/2315977243 shutdown_connections 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 1 -- 172.21.15.32:0/2315977243 wait complete. 2024-08-31T07:36:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.365+0000 7fa0e7096740 20 client.29863 trim_cache size 0 max 16384 2024-08-31T07:36:01.372 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:36:01.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 -- 172.21.15.32:0/425870231 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0c00b4da0 msgr2=0x7fa0c00b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:01.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0c00b4da0 0x7fa0c00b72b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009fd0 tx=0x7fa0d0009500 comp rx=0 tx=0).stop 2024-08-31T07:36:01.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 -- 172.21.15.32:0/425870231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0d810b290 msgr2=0x7fa0d819c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:01.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0d810b290 0x7fa0d819c870 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa0d806c660 tx=0x7fa0c400ee50 comp rx=0 tx=0).stop 2024-08-31T07:36:01.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 -- 172.21.15.32:0/425870231 shutdown_connections 2024-08-31T07:36:01.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa0c00b4da0 0x7fa0c00b72b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0d810bbe0 0x7fa0d81a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0d810b290 0x7fa0d819c870 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 --2- 172.21.15.32:0/425870231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d8105950 0x7fa0d819c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:01.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 -- 172.21.15.32:0/425870231 >> 172.21.15.32:0/425870231 conn(0x7fa0d8101420 msgr2=0x7fa0d8103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:01.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.366+0000 7fa0dee00640 1 -- 172.21.15.32:0/425870231 shutdown_connections 2024-08-31T07:36:01.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:01.367+0000 7fa0dee00640 1 -- 172.21.15.32:0/425870231 wait complete. 2024-08-31T07:36:01.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.04 MB/sec execute 461 sec latency 23997.664 ms 2024-08-31T07:36:02.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:36:02.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:36:02.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 13.01 MB/sec execute 462 sec latency 24997.718 ms 2024-08-31T07:36:02.536 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:02 smithi084 ceph-mon[22387]: pgmap v1100: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1000 KiB/s wr, 80 op/s 2024-08-31T07:36:03.063 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:02 smithi032 ceph-mon[17252]: pgmap v1100: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1000 KiB/s wr, 80 op/s 2024-08-31T07:36:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:02 smithi073 ceph-mon[21995]: pgmap v1100: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1000 KiB/s wr, 80 op/s 2024-08-31T07:36:03.183 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.254+0000 7fc074200640 1 -- 172.21.15.32:0/3669247471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc06c103920 msgr2=0x7fc06c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.254+0000 7fc074200640 1 --2- 172.21.15.32:0/3669247471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc06c103920 0x7fc06c103d30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc060009900 tx=0x7fc06002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.254+0000 7fc074200640 1 -- 172.21.15.32:0/3669247471 shutdown_connections 2024-08-31T07:36:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.254+0000 7fc074200640 1 --2- 172.21.15.32:0/3669247471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c109bb0 0x7fc06c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.254+0000 7fc074200640 1 --2- 172.21.15.32:0/3669247471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc06c109260 0x7fc06c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.254+0000 7fc074200640 1 --2- 172.21.15.32:0/3669247471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc06c103920 0x7fc06c103d30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.254+0000 7fc074200640 1 -- 172.21.15.32:0/3669247471 >> 172.21.15.32:0/3669247471 conn(0x7fc06c0ff330 msgr2=0x7fc06c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.254+0000 7fc074200640 1 -- 172.21.15.32:0/3669247471 shutdown_connections 2024-08-31T07:36:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.254+0000 7fc074200640 1 -- 172.21.15.32:0/3669247471 wait complete. 2024-08-31T07:36:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.255+0000 7fc074200640 1 Processor -- start 2024-08-31T07:36:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc074200640 1 -- start start 2024-08-31T07:36:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc074200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc06c103920 0x7fc06c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc074200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c109260 0x7fc06c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc074200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc06c109bb0 0x7fc06c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc074200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c113870 con 0x7fc06c109bb0 2024-08-31T07:36:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc074200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1136f0 con 0x7fc06c103920 2024-08-31T07:36:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc074200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1139f0 con 0x7fc06c109260 2024-08-31T07:36:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc070e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c109260 0x7fc06c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc071800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc06c103920 0x7fc06c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc072200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc06c109bb0 0x7fc06c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc070e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c109260 0x7fc06c0759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:36824/0 (socket says 172.21.15.32:36824) 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.256+0000 7fc070e00640 1 -- 172.21.15.32:0/963319687 learned_addr learned my addr 172.21.15.32:0/963319687 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.257+0000 7fc070e00640 1 -- 172.21.15.32:0/963319687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc06c103920 msgr2=0x7fc06c0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.257+0000 7fc070e00640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc06c103920 0x7fc06c0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.257+0000 7fc070e00640 1 -- 172.21.15.32:0/963319687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc06c109bb0 msgr2=0x7fc06c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.257+0000 7fc070e00640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc06c109bb0 0x7fc06c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.257+0000 7fc070e00640 1 -- 172.21.15.32:0/963319687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c1a8540 con 0x7fc06c109260 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.257+0000 7fc071800640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc06c103920 0x7fc06c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.257+0000 7fc070e00640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c109260 0x7fc06c0759f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc05800e9f0 tx=0x7fc05800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.257+0000 7fc06a000640 1 -- 172.21.15.32:0/963319687 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05800ccc0 con 0x7fc06c109260 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.258+0000 7fc06a000640 1 -- 172.21.15.32:0/963319687 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0580108e0 con 0x7fc06c109260 2024-08-31T07:36:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.258+0000 7fc06a000640 1 -- 172.21.15.32:0/963319687 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058018860 con 0x7fc06c109260 2024-08-31T07:36:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.258+0000 7fc074200640 1 -- 172.21.15.32:0/963319687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06c1a8830 con 0x7fc06c109260 2024-08-31T07:36:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.258+0000 7fc074200640 1 -- 172.21.15.32:0/963319687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06c1a8d20 con 0x7fc06c109260 2024-08-31T07:36:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.259+0000 7fc074200640 1 -- 172.21.15.32:0/963319687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06c103d30 con 0x7fc06c109260 2024-08-31T07:36:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.262+0000 7fc06a000640 1 -- 172.21.15.32:0/963319687 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc058010430 con 0x7fc06c109260 2024-08-31T07:36:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.263+0000 7fc06a000640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0540b4d20 0x7fc0540b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.263+0000 7fc06a000640 1 -- 172.21.15.32:0/963319687 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fc0580e5c50 con 0x7fc06c109260 2024-08-31T07:36:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.263+0000 7fc071800640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0540b4d20 0x7fc0540b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.264+0000 7fc071800640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0540b4d20 0x7fc0540b7230 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc060000c00 tx=0x7fc060031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.265+0000 7fc06a000640 1 -- 172.21.15.32:0/963319687 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc0580ac3e0 con 0x7fc06c109260 2024-08-31T07:36:03.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 12.98 MB/sec execute 463 sec latency 25997.784 ms 2024-08-31T07:36:03.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.507+0000 7fc074200640 1 -- 172.21.15.32:0/963319687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc06c02e430 con 0x7fc06c109260 2024-08-31T07:36:03.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.509+0000 7fc06a000640 1 -- 172.21.15.32:0/963319687 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+11973 (secure 0 0 0) 0x7fc0580b0ba0 con 0x7fc06c109260 2024-08-31T07:36:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:36:03.511 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":90,"btime":"2024-08-31T07:35:50:675689+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":90,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:35:50.139610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:36:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 90 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.511+0000 7fc074200640 1 -- 172.21.15.32:0/963319687 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0540b4d20 msgr2=0x7fc0540b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.511+0000 7fc074200640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0540b4d20 0x7fc0540b7230 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc060000c00 tx=0x7fc060031400 comp rx=0 tx=0).stop 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.511+0000 7fc074200640 1 -- 172.21.15.32:0/963319687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c109260 msgr2=0x7fc06c0759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.511+0000 7fc074200640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c109260 0x7fc06c0759f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc05800e9f0 tx=0x7fc05800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.512+0000 7fc074200640 1 -- 172.21.15.32:0/963319687 shutdown_connections 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.512+0000 7fc074200640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fc0540b4d20 0x7fc0540b7230 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.512+0000 7fc074200640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc06c109bb0 0x7fc06c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.512+0000 7fc074200640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c109260 0x7fc06c0759f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.512+0000 7fc074200640 1 --2- 172.21.15.32:0/963319687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc06c103920 0x7fc06c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.512+0000 7fc074200640 1 -- 172.21.15.32:0/963319687 >> 172.21.15.32:0/963319687 conn(0x7fc06c0ff330 msgr2=0x7fc06c102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.512+0000 7fc074200640 1 -- 172.21.15.32:0/963319687 shutdown_connections 2024-08-31T07:36:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:03.512+0000 7fc074200640 1 -- 172.21.15.32:0/963319687 wait complete. 2024-08-31T07:36:03.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:03 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/963319687' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:36:03.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:03 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/963319687' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:36:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.056+0000 7fb9b7200640 1 -- 172.21.15.32:0/441175420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9b00754f0 msgr2=0x7fb9b0075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.056+0000 7fb9b7200640 1 --2- 172.21.15.32:0/441175420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9b00754f0 0x7fb9b0075900 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009900 tx=0x7fb9a402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.056+0000 7fb9b7200640 1 -- 172.21.15.32:0/441175420 shutdown_connections 2024-08-31T07:36:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.056+0000 7fb9b7200640 1 --2- 172.21.15.32:0/441175420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b010fd10 0x7fb9b0112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.056+0000 7fb9b7200640 1 --2- 172.21.15.32:0/441175420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b0075e40 0x7fb9b010f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.056+0000 7fb9b7200640 1 --2- 172.21.15.32:0/441175420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9b00754f0 0x7fb9b0075900 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.056+0000 7fb9b7200640 1 -- 172.21.15.32:0/441175420 >> 172.21.15.32:0/441175420 conn(0x7fb9b00ff1a0 msgr2=0x7fb9b01015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.056+0000 7fb9b7200640 1 -- 172.21.15.32:0/441175420 shutdown_connections 2024-08-31T07:36:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.056+0000 7fb9b7200640 1 -- 172.21.15.32:0/441175420 wait complete. 2024-08-31T07:36:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.057+0000 7fb9b7200640 1 Processor -- start 2024-08-31T07:36:04.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b7200640 1 -- start start 2024-08-31T07:36:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b00754f0 0x7fb9b019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9b0075e40 0x7fb9b019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b7200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b010fd10 0x7fb9b01a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0115880 con 0x7fb9b00754f0 2024-08-31T07:36:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0115700 con 0x7fb9b0075e40 2024-08-31T07:36:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b7200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0115a00 con 0x7fb9b010fd10 2024-08-31T07:36:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b00754f0 0x7fb9b019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b00754f0 0x7fb9b019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54450/0 (socket says 172.21.15.32:54450) 2024-08-31T07:36:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b5e00640 1 -- 172.21.15.32:0/1327195972 learned_addr learned my addr 172.21.15.32:0/1327195972 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.058+0000 7fb9b5400640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9b0075e40 0x7fb9b019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.059+0000 7fb9b6800640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b010fd10 0x7fb9b01a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.059+0000 7fb9b5e00640 1 -- 172.21.15.32:0/1327195972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b010fd10 msgr2=0x7fb9b01a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.059+0000 7fb9b5e00640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b010fd10 0x7fb9b01a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.059+0000 7fb9b5e00640 1 -- 172.21.15.32:0/1327195972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9b0075e40 msgr2=0x7fb9b019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.059+0000 7fb9b5e00640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9b0075e40 0x7fb9b019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.059+0000 7fb9b5e00640 1 -- 172.21.15.32:0/1327195972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b01a2200 con 0x7fb9b00754f0 2024-08-31T07:36:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.059+0000 7fb9b5400640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9b0075e40 0x7fb9b019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.059+0000 7fb9b5e00640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b00754f0 0x7fb9b019c290 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005940 tx=0x7fb9a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.060+0000 7fb9aaa00640 1 -- 172.21.15.32:0/1327195972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a403f030 con 0x7fb9b00754f0 2024-08-31T07:36:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.060+0000 7fb9b7200640 1 -- 172.21.15.32:0/1327195972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b01a2490 con 0x7fb9b00754f0 2024-08-31T07:36:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.060+0000 7fb9aaa00640 1 -- 172.21.15.32:0/1327195972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9a40044c0 con 0x7fb9b00754f0 2024-08-31T07:36:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.060+0000 7fb9b7200640 1 -- 172.21.15.32:0/1327195972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b01a8c50 con 0x7fb9b00754f0 2024-08-31T07:36:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.060+0000 7fb9aaa00640 1 -- 172.21.15.32:0/1327195972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4035070 con 0x7fb9b00754f0 2024-08-31T07:36:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.063+0000 7fb9bf4cb740 1 Processor -- start 2024-08-31T07:36:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.063+0000 7fb9bf4cb740 1 -- start start 2024-08-31T07:36:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9aaa00640 1 -- 172.21.15.32:0/1327195972 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb9a4042030 con 0x7fb9b00754f0 2024-08-31T07:36:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b925070 0x55556bccab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bccb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b904390 0x55556bccb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55556bc417b0 con 0x55556b8fc1b0 2024-08-31T07:36:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55556bc30d40 con 0x55556b904390 2024-08-31T07:36:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55556bc34cf0 con 0x55556b925070 2024-08-31T07:36:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9b5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bccb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9b5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bccb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54452/0 (socket says 172.21.15.32:54452) 2024-08-31T07:36:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9b5e00640 1 -- 172.21.15.32:0/2160665723 learned_addr learned my addr 172.21.15.32:0/2160665723 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9b6800640 1 --2- 172.21.15.32:0/2160665723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b925070 0x55556bccab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.064+0000 7fb9b5400640 1 --2- 172.21.15.32:0/2160665723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b904390 0x55556bccb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.065+0000 7fb9b6800640 1 -- 172.21.15.32:0/2160665723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b904390 msgr2=0x55556bccb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.065+0000 7fb9b6800640 1 --2- 172.21.15.32:0/2160665723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b904390 0x55556bccb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.065+0000 7fb9b6800640 1 -- 172.21.15.32:0/2160665723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 msgr2=0x55556bccb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.065+0000 7fb9b6800640 1 --2- 172.21.15.32:0/2160665723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bccb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.065+0000 7fb9b6800640 1 -- 172.21.15.32:0/2160665723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556b990650 con 0x55556b925070 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.065+0000 7fb9b5e00640 1 --2- 172.21.15.32:0/2160665723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bccb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.065+0000 7fb9b6800640 1 --2- 172.21.15.32:0/2160665723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b925070 0x55556bccab60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009f00 tx=0x7fb9ac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab69aec220fc8718 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.066+0000 7fb99f000640 1 -- 172.21.15.32:0/2160665723 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac004d20 con 0x55556b925070 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.066+0000 7fb9aaa00640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9980b4d20 0x7fb9980b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.066+0000 7fb99f000640 1 -- 172.21.15.32:0/2160665723 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9ac004030 con 0x55556b925070 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.066+0000 7fb99f000640 1 -- 172.21.15.32:0/2160665723 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac005280 con 0x55556b925070 2024-08-31T07:36:04.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.066+0000 7fb9aaa00640 1 -- 172.21.15.32:0/1327195972 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fb9a4105870 con 0x7fb9b00754f0 2024-08-31T07:36:04.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.066+0000 7fb9b5e00640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9980b4d20 0x7fb9980b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.067+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/2160665723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b925070 msgr2=0x55556bccab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.067+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/2160665723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b925070 0x55556bccab60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009f00 tx=0x7fb9ac011810 comp rx=0 tx=0).stop 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.067+0000 7fb9b5e00640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9980b4d20 0x7fb9980b7230 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40cbd00 tx=0x7fb9a410b530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.067+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/2160665723 shutdown_connections 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.067+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/2160665723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b904390 0x55556bccb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.067+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/2160665723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bccb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.067+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/2160665723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b925070 0x55556bccab60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.067+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/2160665723 >> 172.21.15.32:0/2160665723 conn(0x55556b919460 msgr2=0x55556bcc8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.067+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/2160665723 shutdown_connections 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.067+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/2160665723 wait complete. 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.068+0000 7fb9bf4cb740 1 -- start start 2024-08-31T07:36:04.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.068+0000 7fb9bf4cb740 1 Processor -- start 2024-08-31T07:36:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 0x55556bcd6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b925070 0x55556bce2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55556ba93700 con 0x55556b8fc1b0 2024-08-31T07:36:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55556bbb7cf0 con 0x55556b925070 2024-08-31T07:36:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55556bcd9340 con 0x55556b904390 2024-08-31T07:36:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9bf4cb740 10 client.0 _ll_register_callbacks cb 0x55556bbdd470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54458/0 (socket says 172.21.15.32:54458) 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b6800640 1 -- 172.21.15.32:0/439600435 learned_addr learned my addr 172.21.15.32:0/439600435 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b5400640 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 0x55556bcd6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b6800640 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b925070 0x55556bce2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b5400640 1 -- 172.21.15.32:0/439600435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b925070 msgr2=0x55556bce2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b5400640 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b925070 0x55556bce2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b5400640 1 -- 172.21.15.32:0/439600435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 msgr2=0x55556bcd6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b5400640 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b5400640 1 -- 172.21.15.32:0/439600435 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55556bbf88b0 con 0x55556b904390 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.069+0000 7fb9b6800640 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.070+0000 7fb9b5400640 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 0x55556bcd6c90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb9b006b6d0 tx=0x7fb9a000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe3aabb982ea53e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.070+0000 7fb993400640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:36:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.070+0000 7fb993400640 1 -- 172.21.15.32:0/439600435 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0012110 con 0x55556b904390 2024-08-31T07:36:04.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.070+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/439600435 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9ac004030 con 0x55556b904390 2024-08-31T07:36:04.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.070+0000 7fb993400640 1 -- 172.21.15.32:0/439600435 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fb9a0004050 con 0x55556b904390 2024-08-31T07:36:04.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb993400640 1 -- 172.21.15.32:0/439600435 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb9a00122b0 con 0x55556b904390 2024-08-31T07:36:04.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb9bf4cb740 10 client.29881 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:04.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/439600435 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55556bce5400 con 0x55556b904390 2024-08-31T07:36:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb993400640 1 -- 172.21.15.32:0/439600435 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9ac004030 con 0x55556b904390 2024-08-31T07:36:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb993400640 1 -- 172.21.15.32:0/439600435 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7fb9a0026480 con 0x55556b904390 2024-08-31T07:36:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb9bf4cb740 10 client.29881 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb9bf4cb740 10 client.29881 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:36:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb9bf4cb740 20 client.29881 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55556b8e39f0 0x55556bce8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb9bf4cb740 4 client.29881 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:36:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.071+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/439600435 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55556bc43700 con 0x55556b8e39f0 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.072+0000 7fb9b6800640 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55556b8e39f0 0x55556bce8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.072+0000 7fb9b6800640 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55556b8e39f0 0x55556bce8860 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=597c18535e532506 server_cookie=7b3b75c88c776c1b in_seq=0 out_seq=0 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.072+0000 7fb993400640 10 client.29881 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.073+0000 7fb993400640 1 -- 172.21.15.32:0/439600435 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55556bc43700 con 0x55556b8e39f0 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.073+0000 7fb993400640 10 client.29881 handle_command_reply: tid=0 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.073+0000 7fb9bf4cb740 1 client.29881 shutdown 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.073+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/439600435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 msgr2=0x55556bcd6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.073+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 0x55556bcd6c90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb9b006b6d0 tx=0x7fb9a000f790 comp rx=0 tx=0).stop 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.074+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/439600435 shutdown_connections 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.074+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55556b8e39f0 0x55556bce8860 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.074+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b925070 0x55556bce2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.074+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 0x55556bcd6c90 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.074+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/439600435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.074+0000 7fb993400640 0 client.29881 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:36:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.074+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/439600435 >> 172.21.15.32:0/439600435 conn(0x55556b919460 msgr2=0x55556bccdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.074+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/439600435 shutdown_connections 2024-08-31T07:36:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.074+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/439600435 wait complete. 2024-08-31T07:36:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.074+0000 7fb9bf4cb740 20 client.29881 trim_cache size 0 max 16384 2024-08-31T07:36:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.088+0000 7fb9bf4cb740 1 Processor -- start 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.088+0000 7fb9bf4cb740 1 -- start start 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.088+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.088+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b904390 0x55556bcd5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.088+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b925070 0x55556bccfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01b1420 con 0x55556b8fc1b0 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01b30e0 con 0x55556b904390 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01b1af0 con 0x55556b925070 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9b6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9b6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54462/0 (socket says 172.21.15.32:54462) 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9b6800640 1 -- 172.21.15.32:0/1223306418 learned_addr learned my addr 172.21.15.32:0/1223306418 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9b5400640 1 --2- 172.21.15.32:0/1223306418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b904390 0x55556bcd5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9b6800640 1 -- 172.21.15.32:0/1223306418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b925070 msgr2=0x55556bccfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9b6800640 1 --2- 172.21.15.32:0/1223306418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b925070 0x55556bccfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9b6800640 1 -- 172.21.15.32:0/1223306418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b904390 msgr2=0x55556bcd5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9b6800640 1 --2- 172.21.15.32:0/1223306418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b904390 0x55556bcd5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.089+0000 7fb9b6800640 1 -- 172.21.15.32:0/1223306418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556bcd7110 con 0x55556b8fc1b0 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.090+0000 7fb9b6800640 1 --2- 172.21.15.32:0/1223306418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd4dd0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac005c40 tx=0x7fb9ac011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603625ab84ef5102 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.090+0000 7fb99e600640 1 -- 172.21.15.32:0/1223306418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac004b60 con 0x55556b8fc1b0 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.090+0000 7fb99e600640 1 -- 172.21.15.32:0/1223306418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9ac02e030 con 0x55556b8fc1b0 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.090+0000 7fb99e600640 1 -- 172.21.15.32:0/1223306418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00f880 con 0x55556b8fc1b0 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.091+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/1223306418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 msgr2=0x55556bcd4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.091+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/1223306418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd4dd0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac005c40 tx=0x7fb9ac011dc0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.091+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/1223306418 shutdown_connections 2024-08-31T07:36:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.091+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/1223306418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b925070 0x55556bccfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.091+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/1223306418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b904390 0x55556bcd5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.091+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/1223306418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b8fc1b0 0x55556bcd4dd0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.091+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/1223306418 >> 172.21.15.32:0/1223306418 conn(0x55556b919460 msgr2=0x55556bcd1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.091+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/1223306418 shutdown_connections 2024-08-31T07:36:04.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.091+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/1223306418 wait complete. 2024-08-31T07:36:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.092+0000 7fb9bf4cb740 1 -- start start 2024-08-31T07:36:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.092+0000 7fb9bf4cb740 1 Processor -- start 2024-08-31T07:36:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.092+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b8fc1b0 0x55556bcda7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.092+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 0x55556bcdad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.092+0000 7fb9bf4cb740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b925070 0x55556bcdb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.092+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01b2980 con 0x55556b925070 2024-08-31T07:36:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.092+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01b21f0 con 0x55556b8fc1b0 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.092+0000 7fb9bf4cb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01af420 con 0x55556b904390 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.092+0000 7fb9bf4cb740 10 client.0 _ll_register_callbacks cb 0x7fb9a0012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b8fc1b0 0x55556bcda7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b925070 0x55556bcdb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b925070 0x55556bcdb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54464/0 (socket says 172.21.15.32:54464) 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b6800640 1 -- 172.21.15.32:0/726097259 learned_addr learned my addr 172.21.15.32:0/726097259 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b5400640 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 0x55556bcdad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b6800640 1 -- 172.21.15.32:0/726097259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 msgr2=0x55556bcdad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b6800640 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 0x55556bcdad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b6800640 1 -- 172.21.15.32:0/726097259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b8fc1b0 msgr2=0x55556bcda7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b6800640 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b8fc1b0 0x55556bcda7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b6800640 1 -- 172.21.15.32:0/726097259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55556bd52460 con 0x55556b925070 2024-08-31T07:36:04.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb9b6800640 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b925070 0x55556bcdb240 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x55556bcd1370 tx=0x7fb9ac004240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189bfa2c6e1b7142 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:04.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.093+0000 7fb993400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:36:04.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.094+0000 7fb993400640 1 -- 172.21.15.32:0/726097259 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac037070 con 0x55556b925070 2024-08-31T07:36:04.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.094+0000 7fb993400640 1 -- 172.21.15.32:0/726097259 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7fb9ac00f8f0 con 0x55556b925070 2024-08-31T07:36:04.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.094+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/726097259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9ac02e030 con 0x55556b925070 2024-08-31T07:36:04.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.094+0000 7fb993400640 1 -- 172.21.15.32:0/726097259 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb9ac01e040 con 0x55556b925070 2024-08-31T07:36:04.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.094+0000 7fb9bf4cb740 10 client.19845 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:04.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.094+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/726097259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55556bd536c0 con 0x55556b925070 2024-08-31T07:36:04.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.094+0000 7fb993400640 1 -- 172.21.15.32:0/726097259 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9ac02e030 con 0x55556b925070 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.095+0000 7fb993400640 1 -- 172.21.15.32:0/726097259 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7fb9ac024040 con 0x55556b925070 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.095+0000 7fb9bf4cb740 10 client.19845 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.095+0000 7fb9bf4cb740 10 client.19845 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.095+0000 7fb9bf4cb740 20 client.19845 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.095+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55556b8b8da0 0x55556bd55bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.095+0000 7fb9bf4cb740 4 client.19845 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.095+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/726097259 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55556bc3f1a0 con 0x55556b8b8da0 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.095+0000 7fb9b5400640 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55556b8b8da0 0x55556bd55bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9b5400640 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55556b8b8da0 0x55556bd55bd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b2bcd2ff4a9080e3 server_cookie=ee86219395f772b in_seq=0 out_seq=0 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb993400640 10 client.19845 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:36:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb993400640 1 -- 172.21.15.32:0/726097259 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55556bc3f1a0 con 0x55556b8b8da0 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb993400640 10 client.19845 handle_command_reply: tid=0 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 client.19845 shutdown 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/726097259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b925070 msgr2=0x55556bcdb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b925070 0x55556bcdb240 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x55556bcd1370 tx=0x7fb9ac004240 comp rx=0 tx=0).stop 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/726097259 shutdown_connections 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55556b8b8da0 0x55556bd55bd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55556b925070 0x55556bcdb240 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55556b904390 0x55556bcdad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 --2- 172.21.15.32:0/726097259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55556b8fc1b0 0x55556bcda7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/726097259 >> 172.21.15.32:0/726097259 conn(0x55556b919460 msgr2=0x55556bcd1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb993400640 0 client.19845 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/726097259 shutdown_connections 2024-08-31T07:36:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.096+0000 7fb9bf4cb740 1 -- 172.21.15.32:0/726097259 wait complete. 2024-08-31T07:36:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9bf4cb740 20 client.19845 trim_cache size 0 max 16384 2024-08-31T07:36:04.102 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:36:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9b7200640 1 -- 172.21.15.32:0/1327195972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9980b4d20 msgr2=0x7fb9980b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9b7200640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9980b4d20 0x7fb9980b7230 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40cbd00 tx=0x7fb9a410b530 comp rx=0 tx=0).stop 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9b7200640 1 -- 172.21.15.32:0/1327195972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b00754f0 msgr2=0x7fb9b019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9b7200640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b00754f0 0x7fb9b019c290 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005940 tx=0x7fb9a40040c0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9b7200640 1 -- 172.21.15.32:0/1327195972 shutdown_connections 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9b7200640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9980b4d20 0x7fb9980b7230 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9b7200640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9b010fd10 0x7fb9b01a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9b7200640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9b0075e40 0x7fb9b019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9b7200640 1 --2- 172.21.15.32:0/1327195972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b00754f0 0x7fb9b019c290 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.097+0000 7fb9b7200640 1 -- 172.21.15.32:0/1327195972 >> 172.21.15.32:0/1327195972 conn(0x7fb9b00ff1a0 msgr2=0x7fb9b01098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.098+0000 7fb9b7200640 1 -- 172.21.15.32:0/1327195972 shutdown_connections 2024-08-31T07:36:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:04.098+0000 7fb9b7200640 1 -- 172.21.15.32:0/1327195972 wait complete. 2024-08-31T07:36:04.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:03 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/963319687' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:36:04.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 12.95 MB/sec execute 464 sec latency 26997.877 ms 2024-08-31T07:36:04.557 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --set-id 2e13427 --release --await' 2024-08-31T07:36:04.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --set-id 2e13427 --release --await 2024-08-31T07:36:04.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:04 smithi032 ceph-mon[17252]: pgmap v1101: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 699 KiB/s wr, 60 op/s 2024-08-31T07:36:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:04 smithi084 ceph-mon[22387]: pgmap v1101: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 699 KiB/s wr, 60 op/s 2024-08-31T07:36:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:04 smithi073 ceph-mon[21995]: pgmap v1101: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 699 KiB/s wr, 60 op/s 2024-08-31T07:36:05.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:36:05.361 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:05.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161688 12.92 MB/sec execute 465 sec latency 27997.955 ms 2024-08-31T07:36:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.060+0000 7f1e6ec00640 1 -- 172.21.15.32:0/1330810483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e68104560 msgr2=0x7f1e68104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.060+0000 7f1e6ec00640 1 --2- 172.21.15.32:0/1330810483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e68104560 0x7f1e68104970 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009900 tx=0x7f1e5c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.060+0000 7f1e6ec00640 1 -- 172.21.15.32:0/1330810483 shutdown_connections 2024-08-31T07:36:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.060+0000 7f1e6ec00640 1 --2- 172.21.15.32:0/1330810483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6810d9c0 0x7f1e6810feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.060+0000 7f1e6ec00640 1 --2- 172.21.15.32:0/1330810483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e68104eb0 0x7f1e6810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.060+0000 7f1e6ec00640 1 --2- 172.21.15.32:0/1330810483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e68104560 0x7f1e68104970 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.060+0000 7f1e6ec00640 1 -- 172.21.15.32:0/1330810483 >> 172.21.15.32:0/1330810483 conn(0x7f1e680fe060 msgr2=0x7f1e681004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.060+0000 7f1e6ec00640 1 -- 172.21.15.32:0/1330810483 shutdown_connections 2024-08-31T07:36:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.060+0000 7f1e6ec00640 1 -- 172.21.15.32:0/1330810483 wait complete. 2024-08-31T07:36:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.062+0000 7f1e6ec00640 1 Processor -- start 2024-08-31T07:36:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.062+0000 7f1e6ec00640 1 -- start start 2024-08-31T07:36:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.062+0000 7f1e6ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e68104560 0x7f1e6819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.062+0000 7f1e6ec00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e68104eb0 0x7f1e6819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.062+0000 7f1e6ec00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6810d9c0 0x7f1e6819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.062+0000 7f1e6ec00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e68113650 con 0x7f1e6810d9c0 2024-08-31T07:36:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.062+0000 7f1e6ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e681134d0 con 0x7f1e68104560 2024-08-31T07:36:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.062+0000 7f1e6ec00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1e681137d0 con 0x7f1e68104eb0 2024-08-31T07:36:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.063+0000 7f1e6d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e68104560 0x7f1e6819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.063+0000 7f1e6d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e68104560 0x7f1e6819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55694/0 (socket says 172.21.15.32:55694) 2024-08-31T07:36:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.063+0000 7f1e6d800640 1 -- 172.21.15.32:0/4267683285 learned_addr learned my addr 172.21.15.32:0/4267683285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.063+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e68104eb0 0x7f1e6819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.063+0000 7f1e6e200640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6810d9c0 0x7f1e6819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.063+0000 7f1e6d800640 1 -- 172.21.15.32:0/4267683285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e68104eb0 msgr2=0x7f1e6819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.063+0000 7f1e6d800640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e68104eb0 0x7f1e6819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.063+0000 7f1e6d800640 1 -- 172.21.15.32:0/4267683285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6810d9c0 msgr2=0x7f1e6819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.063+0000 7f1e6d800640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6810d9c0 0x7f1e6819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.063+0000 7f1e6d800640 1 -- 172.21.15.32:0/4267683285 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e681a0030 con 0x7f1e68104560 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.064+0000 7f1e6d800640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e68104560 0x7f1e6819a0c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c005940 tx=0x7f1e5c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.064+0000 7f1e66000640 1 -- 172.21.15.32:0/4267683285 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c03f030 con 0x7f1e68104560 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.064+0000 7f1e66000640 1 -- 172.21.15.32:0/4267683285 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e5c0044c0 con 0x7f1e68104560 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.064+0000 7f1e66000640 1 -- 172.21.15.32:0/4267683285 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c035070 con 0x7f1e68104560 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.064+0000 7f1e6ec00640 1 -- 172.21.15.32:0/4267683285 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e681a02c0 con 0x7f1e68104560 2024-08-31T07:36:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.064+0000 7f1e6ec00640 1 -- 172.21.15.32:0/4267683285 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e68102180 con 0x7f1e68104560 2024-08-31T07:36:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.068+0000 7f1e76e58740 1 Processor -- start 2024-08-31T07:36:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.068+0000 7f1e76e58740 1 -- start start 2024-08-31T07:36:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.068+0000 7f1e66000640 1 -- 172.21.15.32:0/4267683285 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1e5c042030 con 0x7f1e68104560 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.068+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 0x557788680090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.068+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b1660 0x5577886805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.068+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 0x557788680b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.068+0000 7f1e76e58740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5577885f6a90 con 0x5577882b9840 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.068+0000 7f1e76e58740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5577885e6020 con 0x5577882b1660 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.069+0000 7f1e76e58740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5577885e9fd0 con 0x5577882da520 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.069+0000 7f1e6d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b1660 0x5577886805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.069+0000 7f1e6d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b1660 0x5577886805d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55698/0 (socket says 172.21.15.32:55698) 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.069+0000 7f1e6d800640 1 -- 172.21.15.32:0/147402818 learned_addr learned my addr 172.21.15.32:0/147402818 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.069+0000 7f1e6e200640 1 --2- 172.21.15.32:0/147402818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 0x557788680090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.069+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/147402818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 0x557788680b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.070+0000 7f1e6e200640 1 -- 172.21.15.32:0/147402818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b1660 msgr2=0x5577886805d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.070+0000 7f1e6e200640 1 --2- 172.21.15.32:0/147402818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b1660 0x5577886805d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.070+0000 7f1e6e200640 1 -- 172.21.15.32:0/147402818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 msgr2=0x557788680b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.070+0000 7f1e6e200640 1 --2- 172.21.15.32:0/147402818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 0x557788680b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.070+0000 7f1e6e200640 1 -- 172.21.15.32:0/147402818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557788345b00 con 0x5577882da520 2024-08-31T07:36:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.070+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/147402818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 0x557788680b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.070+0000 7f1e66000640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e500b4ca0 0x7f1e500b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.070+0000 7f1e6e200640 1 --2- 172.21.15.32:0/147402818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 0x557788680090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009f00 tx=0x7f1e60011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e80a331a4b43095 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.070+0000 7f1e66000640 1 -- 172.21.15.32:0/4267683285 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f1e5c1058e0 con 0x7f1e68104560 2024-08-31T07:36:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.071+0000 7f1e5aa00640 1 -- 172.21.15.32:0/147402818 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60004d20 con 0x5577882da520 2024-08-31T07:36:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.071+0000 7f1e5aa00640 1 -- 172.21.15.32:0/147402818 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e60004030 con 0x5577882da520 2024-08-31T07:36:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.071+0000 7f1e5aa00640 1 -- 172.21.15.32:0/147402818 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60005280 con 0x5577882da520 2024-08-31T07:36:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.071+0000 7f1e6d800640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e500b4ca0 0x7f1e500b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.072+0000 7f1e76e58740 1 -- 172.21.15.32:0/147402818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 msgr2=0x557788680090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.072+0000 7f1e76e58740 1 --2- 172.21.15.32:0/147402818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 0x557788680090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009f00 tx=0x7f1e60011810 comp rx=0 tx=0).stop 2024-08-31T07:36:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.072+0000 7f1e6d800640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e500b4ca0 0x7f1e500b71b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c006010 tx=0x7f1e5c10b530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.072+0000 7f1e76e58740 1 -- 172.21.15.32:0/147402818 shutdown_connections 2024-08-31T07:36:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.072+0000 7f1e76e58740 1 --2- 172.21.15.32:0/147402818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 0x557788680b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.072+0000 7f1e76e58740 1 --2- 172.21.15.32:0/147402818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b1660 0x5577886805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.072+0000 7f1e76e58740 1 --2- 172.21.15.32:0/147402818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 0x557788680090 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.072+0000 7f1e76e58740 1 -- 172.21.15.32:0/147402818 >> 172.21.15.32:0/147402818 conn(0x5577882ce910 msgr2=0x55778867df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.072+0000 7f1e76e58740 1 -- 172.21.15.32:0/147402818 shutdown_connections 2024-08-31T07:36:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.072+0000 7f1e76e58740 1 -- 172.21.15.32:0/147402818 wait complete. 2024-08-31T07:36:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.073+0000 7f1e76e58740 1 -- start start 2024-08-31T07:36:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.073+0000 7f1e76e58740 1 Processor -- start 2024-08-31T07:36:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.073+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x55778868b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.073+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 0x55778868b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.073+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882da520 0x55778868bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.073+0000 7f1e76e58740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557788448bb0 con 0x5577882b9840 2024-08-31T07:36:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.073+0000 7f1e76e58740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55778856cf90 con 0x5577882da520 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.073+0000 7f1e76e58740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55778868e870 con 0x5577882b1660 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.073+0000 7f1e76e58740 10 client.0 _ll_register_callbacks cb 0x557788592710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.074+0000 7f1e6e200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x55778868b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.074+0000 7f1e6e200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x55778868b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59746/0 (socket says 172.21.15.32:59746) 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.074+0000 7f1e6ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 0x55778868b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.074+0000 7f1e6e200640 1 -- 172.21.15.32:0/2644180776 learned_addr learned my addr 172.21.15.32:0/2644180776 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.074+0000 7f1e6e200640 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882da520 0x55778868bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.074+0000 7f1e6ce00640 1 -- 172.21.15.32:0/2644180776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 msgr2=0x55778868b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.074+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x55778868b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.074+0000 7f1e6ce00640 1 -- 172.21.15.32:0/2644180776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882da520 msgr2=0x55778868bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.074+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882da520 0x55778868bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.074+0000 7f1e6ce00640 1 -- 172.21.15.32:0/2644180776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577883bc960 con 0x5577882b9840 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.075+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 0x55778868b9a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1e6806a5a0 tx=0x7f1e54012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49dd35b58ca65f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.075+0000 7f1e4f400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.075+0000 7f1e4f400640 1 -- 172.21.15.32:0/2644180776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54004050 con 0x5577882b9840 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.075+0000 7f1e4f400640 1 -- 172.21.15.32:0/2644180776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f1e540041f0 con 0x5577882b9840 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.075+0000 7f1e76e58740 1 -- 172.21.15.32:0/2644180776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557788584e30 con 0x5577882b9840 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.075+0000 7f1e4f400640 1 -- 172.21.15.32:0/2644180776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1e540043f0 con 0x5577882b9840 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.076+0000 7f1e76e58740 10 client.19857 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.076+0000 7f1e76e58740 1 -- 172.21.15.32:0/2644180776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55778869a350 con 0x5577882b9840 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.076+0000 7f1e4f400640 1 -- 172.21.15.32:0/2644180776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557788584e30 con 0x5577882b9840 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.076+0000 7f1e4f400640 1 -- 172.21.15.32:0/2644180776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7f1e54003b20 con 0x5577882b9840 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.076+0000 7f1e76e58740 10 client.19857 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.076+0000 7f1e76e58740 10 client.19857 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:36:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.076+0000 7f1e76e58740 20 client.19857 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.076+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557788298ea0 0x55778869d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.076+0000 7f1e76e58740 4 client.19857 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:36:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.076+0000 7f1e76e58740 1 -- 172.21.15.32:0/2644180776 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577885f89e0 con 0x557788298ea0 2024-08-31T07:36:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.077+0000 7f1e6e200640 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557788298ea0 0x55778869d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.077+0000 7f1e6e200640 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557788298ea0 0x55778869d810 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe2ae5486bbe374e server_cookie=5bbfdbdeb2fa102c in_seq=0 out_seq=0 2024-08-31T07:36:06.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.077+0000 7f1e4f400640 10 client.19857 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:06.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.078+0000 7f1e4f400640 1 -- 172.21.15.32:0/2644180776 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577885f89e0 con 0x557788298ea0 2024-08-31T07:36:06.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.078+0000 7f1e4f400640 10 client.19857 handle_command_reply: tid=0 2024-08-31T07:36:06.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.078+0000 7f1e76e58740 1 client.19857 shutdown 2024-08-31T07:36:06.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.078+0000 7f1e76e58740 1 -- 172.21.15.32:0/2644180776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 msgr2=0x55778868b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:06.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.078+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 0x55778868b9a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1e6806a5a0 tx=0x7f1e54012b60 comp rx=0 tx=0).stop 2024-08-31T07:36:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.079+0000 7f1e76e58740 1 -- 172.21.15.32:0/2644180776 shutdown_connections 2024-08-31T07:36:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.079+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x557788298ea0 0x55778869d810 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.079+0000 7f1e4f400640 0 client.19857 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.079+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882da520 0x55778868bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.079+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b9840 0x55778868b9a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.079+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2644180776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x55778868b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.079+0000 7f1e76e58740 1 -- 172.21.15.32:0/2644180776 >> 172.21.15.32:0/2644180776 conn(0x5577882ce910 msgr2=0x557788683220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.079+0000 7f1e76e58740 1 -- 172.21.15.32:0/2644180776 shutdown_connections 2024-08-31T07:36:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.079+0000 7f1e76e58740 1 -- 172.21.15.32:0/2644180776 wait complete. 2024-08-31T07:36:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.079+0000 7f1e76e58740 20 client.19857 trim_cache size 0 max 16384 2024-08-31T07:36:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.098+0000 7f1e76e58740 1 Processor -- start 2024-08-31T07:36:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.098+0000 7f1e76e58740 1 -- start start 2024-08-31T07:36:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.098+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x557788697ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.098+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 0x55778867cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.098+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882da520 0x55778867d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.098+0000 7f1e76e58740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e681b1390 con 0x5577882da520 2024-08-31T07:36:06.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.098+0000 7f1e76e58740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e681b3050 con 0x5577882b9840 2024-08-31T07:36:06.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.098+0000 7f1e76e58740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1e681b1a60 con 0x5577882b1660 2024-08-31T07:36:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6e200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x557788697ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6e200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x557788697ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59748/0 (socket says 172.21.15.32:59748) 2024-08-31T07:36:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6e200640 1 -- 172.21.15.32:0/3260986530 learned_addr learned my addr 172.21.15.32:0/3260986530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6e200640 1 --2- 172.21.15.32:0/3260986530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882da520 0x55778867d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/3260986530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 0x55778867cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6ce00640 1 -- 172.21.15.32:0/3260986530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 msgr2=0x557788697ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/3260986530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x557788697ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6ce00640 1 -- 172.21.15.32:0/3260986530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882da520 msgr2=0x55778867d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/3260986530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882da520 0x55778867d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6ce00640 1 -- 172.21.15.32:0/3260986530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778867e170 con 0x5577882b9840 2024-08-31T07:36:06.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.099+0000 7f1e6e200640 1 --2- 172.21.15.32:0/3260986530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x557788697ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:06.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.100+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/3260986530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 0x55778867cc30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1e54026570 tx=0x7f1e54026800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc8bbc7481d6e5c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:06.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.100+0000 7f1e5a000640 1 -- 172.21.15.32:0/3260986530 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54004080 con 0x5577882b9840 2024-08-31T07:36:06.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.100+0000 7f1e5a000640 1 -- 172.21.15.32:0/3260986530 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e54004220 con 0x5577882b9840 2024-08-31T07:36:06.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.100+0000 7f1e5a000640 1 -- 172.21.15.32:0/3260986530 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54003650 con 0x5577882b9840 2024-08-31T07:36:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.101+0000 7f1e76e58740 1 -- 172.21.15.32:0/3260986530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 msgr2=0x55778867cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.101+0000 7f1e76e58740 1 --2- 172.21.15.32:0/3260986530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 0x55778867cc30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1e54026570 tx=0x7f1e54026800 comp rx=0 tx=0).stop 2024-08-31T07:36:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.101+0000 7f1e76e58740 1 -- 172.21.15.32:0/3260986530 shutdown_connections 2024-08-31T07:36:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.101+0000 7f1e76e58740 1 --2- 172.21.15.32:0/3260986530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882da520 0x55778867d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.101+0000 7f1e76e58740 1 --2- 172.21.15.32:0/3260986530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 0x55778867cc30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.101+0000 7f1e76e58740 1 --2- 172.21.15.32:0/3260986530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882b1660 0x557788697ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.101+0000 7f1e76e58740 1 -- 172.21.15.32:0/3260986530 >> 172.21.15.32:0/3260986530 conn(0x5577882ce910 msgr2=0x55778867c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.101+0000 7f1e76e58740 1 -- 172.21.15.32:0/3260986530 shutdown_connections 2024-08-31T07:36:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.101+0000 7f1e76e58740 1 -- 172.21.15.32:0/3260986530 wait complete. 2024-08-31T07:36:06.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.102+0000 7f1e76e58740 1 -- start start 2024-08-31T07:36:06.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.102+0000 7f1e76e58740 1 Processor -- start 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b1660 0x5577886923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 0x5577886929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e76e58740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 0x557788701220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e76e58740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e681b28f0 con 0x5577882b1660 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e76e58740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e681b2160 con 0x5577882b9840 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e76e58740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1e681af370 con 0x5577882da520 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e76e58740 10 client.0 _ll_register_callbacks cb 0x7f1e54009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e6ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 0x5577886929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e6e200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b1660 0x5577886923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e6ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 0x5577886929a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55712/0 (socket says 172.21.15.32:55712) 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e6ce00640 1 -- 172.21.15.32:0/2353296069 learned_addr learned my addr 172.21.15.32:0/2353296069 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:06.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e6e200640 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 0x557788701220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e6e200640 1 -- 172.21.15.32:0/2353296069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 msgr2=0x557788701220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e6e200640 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 0x557788701220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e6e200640 1 -- 172.21.15.32:0/2353296069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 msgr2=0x5577886929a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e6e200640 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 0x5577886929a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.103+0000 7f1e6e200640 1 -- 172.21.15.32:0/2353296069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557788701890 con 0x5577882b1660 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.104+0000 7f1e6e200640 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b1660 0x5577886923c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1e60004860 tx=0x7f1e6002b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c93a795119872b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.104+0000 7f1e4f400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.104+0000 7f1e4f400640 1 -- 172.21.15.32:0/2353296069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6001e040 con 0x5577882b1660 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.104+0000 7f1e76e58740 1 -- 172.21.15.32:0/2353296069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e54004220 con 0x5577882b1660 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.104+0000 7f1e4f400640 1 -- 172.21.15.32:0/2353296069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8441+0+0 (secure 0 0 0) 0x7f1e60003e20 con 0x5577882b1660 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.104+0000 7f1e4f400640 1 -- 172.21.15.32:0/2353296069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1e60004020 con 0x5577882b1660 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.104+0000 7f1e76e58740 10 client.19863 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:06.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.104+0000 7f1e76e58740 1 -- 172.21.15.32:0/2353296069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x557788708ba0 con 0x5577882b1660 2024-08-31T07:36:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.105+0000 7f1e4f400640 1 -- 172.21.15.32:0/2353296069 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e54004220 con 0x5577882b1660 2024-08-31T07:36:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.105+0000 7f1e4f400640 1 -- 172.21.15.32:0/2353296069 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7f1e60018070 con 0x5577882b1660 2024-08-31T07:36:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.105+0000 7f1e76e58740 10 client.19863 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.105+0000 7f1e76e58740 10 client.19863 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:36:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.105+0000 7f1e76e58740 20 client.19863 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.105+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55778826e250 0x55778870b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.105+0000 7f1e76e58740 4 client.19863 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2e13427", "release": true, "await": true}] 2024-08-31T07:36:06.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.105+0000 7f1e76e58740 1 -- 172.21.15.32:0/2353296069 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2e13427", "release": true, "await": true}) -- 0x5577885f4480 con 0x55778826e250 2024-08-31T07:36:06.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.105+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55778826e250 0x55778870b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:06.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.106+0000 7f1e6ce00640 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55778826e250 0x55778870b2a0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc5852b8e401e0a server_cookie=871ad3c9c83dd4 in_seq=0 out_seq=0 2024-08-31T07:36:06.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.106+0000 7f1e4f400640 10 client.19863 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:06.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.138+0000 7f1e4f400640 1 -- 172.21.15.32:0/2353296069 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5577885f4480 con 0x55778826e250 2024-08-31T07:36:06.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.138+0000 7f1e4f400640 10 client.19863 handle_command_reply: tid=0 2024-08-31T07:36:06.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.138+0000 7f1e76e58740 1 client.19863 shutdown 2024-08-31T07:36:06.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.139+0000 7f1e76e58740 1 -- 172.21.15.32:0/2353296069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b1660 msgr2=0x5577886923c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:06.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.139+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b1660 0x5577886923c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1e60004860 tx=0x7f1e6002b040 comp rx=0 tx=0).stop 2024-08-31T07:36:06.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.139+0000 7f1e76e58740 1 -- 172.21.15.32:0/2353296069 shutdown_connections 2024-08-31T07:36:06.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.139+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55778826e250 0x55778870b2a0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.139+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5577882da520 0x557788701220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.139+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577882b9840 0x5577886929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.139+0000 7f1e76e58740 1 --2- 172.21.15.32:0/2353296069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577882b1660 0x5577886923c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.139+0000 7f1e4f400640 0 client.19863 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:06.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.139+0000 7f1e76e58740 1 -- 172.21.15.32:0/2353296069 >> 172.21.15.32:0/2353296069 conn(0x5577882ce910 msgr2=0x55778867cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:06.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.140+0000 7f1e76e58740 1 -- 172.21.15.32:0/2353296069 shutdown_connections 2024-08-31T07:36:06.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.140+0000 7f1e76e58740 1 -- 172.21.15.32:0/2353296069 wait complete. 2024-08-31T07:36:06.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.140+0000 7f1e76e58740 20 client.19863 trim_cache size 0 max 16384 2024-08-31T07:36:06.142 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:36:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 90, 2024-08-31T07:36:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:36:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 76, 2024-08-31T07:36:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": { 2024-08-31T07:36:06.142 INFO:teuthology.orchestra.run.smithi032.stdout: "2e13427": { 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 76, 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "age_ref": 0.0, 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "timeout": 90.0, 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "expiration": 150.0, 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "members": { 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "file:/": { 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "excluded": false, 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "state": { 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RELEASED", 2024-08-31T07:36:06.143 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0 2024-08-31T07:36:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:36:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:36:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:36:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:36:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:36:06.144 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:36:06.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.142+0000 7f1e6ec00640 1 -- 172.21.15.32:0/4267683285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e500b4ca0 msgr2=0x7f1e500b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:06.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.142+0000 7f1e6ec00640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e500b4ca0 0x7f1e500b71b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c006010 tx=0x7f1e5c10b530 comp rx=0 tx=0).stop 2024-08-31T07:36:06.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.142+0000 7f1e6ec00640 1 -- 172.21.15.32:0/4267683285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e68104560 msgr2=0x7f1e6819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:06.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.142+0000 7f1e6ec00640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e68104560 0x7f1e6819a0c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c005940 tx=0x7f1e5c0040c0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.142+0000 7f1e6ec00640 1 -- 172.21.15.32:0/4267683285 shutdown_connections 2024-08-31T07:36:06.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.142+0000 7f1e6ec00640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f1e500b4ca0 0x7f1e500b71b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.142+0000 7f1e6ec00640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6810d9c0 0x7f1e6819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.142+0000 7f1e6ec00640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e68104eb0 0x7f1e6819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.142+0000 7f1e6ec00640 1 --2- 172.21.15.32:0/4267683285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e68104560 0x7f1e6819a0c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:06.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.142+0000 7f1e6ec00640 1 -- 172.21.15.32:0/4267683285 >> 172.21.15.32:0/4267683285 conn(0x7f1e680fe060 msgr2=0x7f1e68107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:06.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.143+0000 7f1e6ec00640 1 -- 172.21.15.32:0/4267683285 shutdown_connections 2024-08-31T07:36:06.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:06.143+0000 7f1e6ec00640 1 -- 172.21.15.32:0/4267683285 wait complete. 2024-08-31T07:36:06.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161871 12.92 MB/sec execute 466 sec latency 28696.833 ms 2024-08-31T07:36:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:06 smithi084 ceph-mon[22387]: pgmap v1102: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 344 KiB/s wr, 48 op/s 2024-08-31T07:36:07.052 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:06 smithi032 ceph-mon[17252]: pgmap v1102: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 344 KiB/s wr, 48 op/s 2024-08-31T07:36:07.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:06 smithi073 ceph-mon[21995]: pgmap v1102: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 344 KiB/s wr, 48 op/s 2024-08-31T07:36:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.257+0000 7f328b200640 1 -- 172.21.15.32:0/3205468828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32841021a0 msgr2=0x7f32841025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.257+0000 7f328b200640 1 --2- 172.21.15.32:0/3205468828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32841021a0 0x7f32841025b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3274009900 tx=0x7f327402a400 comp rx=0 tx=0).stop 2024-08-31T07:36:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.258+0000 7f328b200640 1 -- 172.21.15.32:0/3205468828 shutdown_connections 2024-08-31T07:36:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.258+0000 7f328b200640 1 --2- 172.21.15.32:0/3205468828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3284114540 0x7f3284116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.258+0000 7f328b200640 1 --2- 172.21.15.32:0/3205468828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3284102af0 0x7f3284114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.258+0000 7f328b200640 1 --2- 172.21.15.32:0/3205468828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32841021a0 0x7f32841025b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.258+0000 7f328b200640 1 -- 172.21.15.32:0/3205468828 >> 172.21.15.32:0/3205468828 conn(0x7f32840fe060 msgr2=0x7f32841004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:07.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.261+0000 7f328b200640 1 -- 172.21.15.32:0/3205468828 shutdown_connections 2024-08-31T07:36:07.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.261+0000 7f328b200640 1 -- 172.21.15.32:0/3205468828 wait complete. 2024-08-31T07:36:07.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.262+0000 7f328b200640 1 Processor -- start 2024-08-31T07:36:07.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.263+0000 7f328b200640 1 -- start start 2024-08-31T07:36:07.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.263+0000 7f328b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32841021a0 0x7f32841a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.263+0000 7f328b200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3284102af0 0x7f32841a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.264+0000 7f328b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3284114540 0x7f32841a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.264+0000 7f328b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3284119c90 con 0x7f3284114540 2024-08-31T07:36:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.264+0000 7f328b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3284119b10 con 0x7f32841021a0 2024-08-31T07:36:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.264+0000 7f328b200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3284119e10 con 0x7f3284102af0 2024-08-31T07:36:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.264+0000 7f3289400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3284102af0 0x7f32841a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.264+0000 7f3289400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3284102af0 0x7f32841a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59770/0 (socket says 172.21.15.32:59770) 2024-08-31T07:36:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.264+0000 7f3289400640 1 -- 172.21.15.32:0/1025188068 learned_addr learned my addr 172.21.15.32:0/1025188068 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.264+0000 7f3289e00640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32841021a0 0x7f32841a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.265+0000 7f328a800640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3284114540 0x7f32841a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.265+0000 7f3289e00640 1 -- 172.21.15.32:0/1025188068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3284102af0 msgr2=0x7f32841a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.265+0000 7f3289e00640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3284102af0 0x7f32841a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.265+0000 7f3289e00640 1 -- 172.21.15.32:0/1025188068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3284114540 msgr2=0x7f32841a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.265+0000 7f3289e00640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3284114540 0x7f32841a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.265+0000 7f3289e00640 1 -- 172.21.15.32:0/1025188068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32841a6530 con 0x7f32841021a0 2024-08-31T07:36:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.265+0000 7f3289e00640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32841021a0 0x7f32841a0680 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f327402a9f0 tx=0x7f3274003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.266+0000 7f327ea00640 1 -- 172.21.15.32:0/1025188068 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327403f070 con 0x7f32841021a0 2024-08-31T07:36:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.266+0000 7f328b200640 1 -- 172.21.15.32:0/1025188068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32841a67c0 con 0x7f32841021a0 2024-08-31T07:36:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.266+0000 7f327ea00640 1 -- 172.21.15.32:0/1025188068 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3274031b50 con 0x7f32841021a0 2024-08-31T07:36:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.266+0000 7f327ea00640 1 -- 172.21.15.32:0/1025188068 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274035030 con 0x7f32841021a0 2024-08-31T07:36:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.266+0000 7f328b200640 1 -- 172.21.15.32:0/1025188068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32841a6cc0 con 0x7f32841021a0 2024-08-31T07:36:07.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.271+0000 7f329349b740 1 Processor -- start 2024-08-31T07:36:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.272+0000 7f329349b740 1 -- start start 2024-08-31T07:36:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.272+0000 7f329349b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0e7070 0x56242c48cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.272+0000 7f329349b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0be1b0 0x56242c48d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.272+0000 7f329349b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c48d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.272+0000 7f329349b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56242c4037b0 con 0x56242c0be1b0 2024-08-31T07:36:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.272+0000 7f329349b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56242c3f2d40 con 0x56242c0c6390 2024-08-31T07:36:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.272+0000 7f329349b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56242c3f6cf0 con 0x56242c0e7070 2024-08-31T07:36:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.273+0000 7f3289400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c48d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.273+0000 7f3289400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c48d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55742/0 (socket says 172.21.15.32:55742) 2024-08-31T07:36:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.273+0000 7f3289400640 1 -- 172.21.15.32:0/2454245828 learned_addr learned my addr 172.21.15.32:0/2454245828 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:07.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.273+0000 7f328a800640 1 --2- 172.21.15.32:0/2454245828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0e7070 0x56242c48cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.274+0000 7f3289400640 1 -- 172.21.15.32:0/2454245828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0e7070 msgr2=0x56242c48cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:07.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.274+0000 7f3289400640 1 --2- 172.21.15.32:0/2454245828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0e7070 0x56242c48cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.274+0000 7f3289400640 1 -- 172.21.15.32:0/2454245828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0be1b0 msgr2=0x56242c48d0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:36:07.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.274+0000 7f3289400640 1 --2- 172.21.15.32:0/2454245828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0be1b0 0x56242c48d0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.274+0000 7f3289400640 1 -- 172.21.15.32:0/2454245828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242c152650 con 0x56242c0c6390 2024-08-31T07:36:07.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.275+0000 7f3289400640 1 --2- 172.21.15.32:0/2454245828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c48d610 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f327800acf0 tx=0x7f327800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d95bab2cbd34965 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:07.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.276+0000 7f3273000640 1 -- 172.21.15.32:0/2454245828 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327801d070 con 0x56242c0c6390 2024-08-31T07:36:07.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.276+0000 7f3273000640 1 -- 172.21.15.32:0/2454245828 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3278018af0 con 0x56242c0c6390 2024-08-31T07:36:07.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.276+0000 7f3273000640 1 -- 172.21.15.32:0/2454245828 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278010bc0 con 0x56242c0c6390 2024-08-31T07:36:07.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.277+0000 7f327ea00640 1 -- 172.21.15.32:0/1025188068 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3274042070 con 0x7f32841021a0 2024-08-31T07:36:07.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.277+0000 7f329349b740 1 -- 172.21.15.32:0/2454245828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 msgr2=0x56242c48d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:07.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.278+0000 7f329349b740 1 --2- 172.21.15.32:0/2454245828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c48d610 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f327800acf0 tx=0x7f327800f790 comp rx=0 tx=0).stop 2024-08-31T07:36:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.278+0000 7f329349b740 1 -- 172.21.15.32:0/2454245828 shutdown_connections 2024-08-31T07:36:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.278+0000 7f329349b740 1 --2- 172.21.15.32:0/2454245828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c48d610 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.278+0000 7f329349b740 1 --2- 172.21.15.32:0/2454245828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0be1b0 0x56242c48d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.278+0000 7f329349b740 1 --2- 172.21.15.32:0/2454245828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0e7070 0x56242c48cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.278+0000 7f329349b740 1 -- 172.21.15.32:0/2454245828 >> 172.21.15.32:0/2454245828 conn(0x56242c0db460 msgr2=0x56242c48aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:07.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.278+0000 7f329349b740 1 -- 172.21.15.32:0/2454245828 shutdown_connections 2024-08-31T07:36:07.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.278+0000 7f329349b740 1 -- 172.21.15.32:0/2454245828 wait complete. 2024-08-31T07:36:07.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.279+0000 7f329349b740 1 -- start start 2024-08-31T07:36:07.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.279+0000 7f329349b740 1 Processor -- start 2024-08-31T07:36:07.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.279+0000 7f327ea00640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f326c0b4ca0 0x7f326c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.280+0000 7f327ea00640 1 -- 172.21.15.32:0/1025188068 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8485+0+0 (secure 0 0 0) 0x7f3274105cb0 con 0x7f32841021a0 2024-08-31T07:36:07.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.280+0000 7f328a800640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f326c0b4ca0 0x7f326c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.282+0000 7f329349b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c4987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.282+0000 7f329349b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c498d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.282+0000 7f329349b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0e7070 0x56242c4a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.282+0000 7f329349b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56242c255700 con 0x56242c0e7070 2024-08-31T07:36:07.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.282+0000 7f329349b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56242c379cf0 con 0x56242c0c6390 2024-08-31T07:36:07.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.282+0000 7f329349b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56242c49b370 con 0x56242c0be1b0 2024-08-31T07:36:07.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.282+0000 7f329349b740 10 client.0 _ll_register_callbacks cb 0x56242c39f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:07.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.283+0000 7f3289400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c498d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.283+0000 7f3289400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c498d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55746/0 (socket says 172.21.15.32:55746) 2024-08-31T07:36:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.283+0000 7f3289400640 1 -- 172.21.15.32:0/2267104682 learned_addr learned my addr 172.21.15.32:0/2267104682 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.283+0000 7f328a800640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f326c0b4ca0 0x7f326c0b71b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3280005fd0 tx=0x7f3280009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.283+0000 7f328a800640 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0e7070 0x56242c4a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.283+0000 7f3289e00640 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c4987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.284+0000 7f3289400640 1 -- 172.21.15.32:0/2267104682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 msgr2=0x56242c4987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.284+0000 7f3289400640 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c4987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.284+0000 7f3289400640 1 -- 172.21.15.32:0/2267104682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0e7070 msgr2=0x56242c4a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.285+0000 7f3289400640 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0e7070 0x56242c4a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.285+0000 7f3289400640 1 -- 172.21.15.32:0/2267104682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242c3ba8b0 con 0x56242c0c6390 2024-08-31T07:36:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.285+0000 7f328a800640 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0e7070 0x56242c4a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.285+0000 7f3289400640 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c498d90 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f32780186a0 tx=0x7f32780186d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0380bf19e52ca41 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:07.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.285+0000 7f3267400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:36:07.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.286+0000 7f3267400640 1 -- 172.21.15.32:0/2267104682 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278027020 con 0x56242c0c6390 2024-08-31T07:36:07.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.287+0000 7f3267400640 1 -- 172.21.15.32:0/2267104682 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8485+0+0 (secure 0 0 0) 0x7f32780108c0 con 0x56242c0c6390 2024-08-31T07:36:07.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.287+0000 7f329349b740 1 -- 172.21.15.32:0/2267104682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3278018af0 con 0x56242c0c6390 2024-08-31T07:36:07.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.288+0000 7f3267400640 1 -- 172.21.15.32:0/2267104682 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3278027020 con 0x56242c0c6390 2024-08-31T07:36:07.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.288+0000 7f329349b740 10 client.29912 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:07.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.288+0000 7f329349b740 1 -- 172.21.15.32:0/2267104682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x56242c4a7400 con 0x56242c0c6390 2024-08-31T07:36:07.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.289+0000 7f3267400640 1 -- 172.21.15.32:0/2267104682 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3278018af0 con 0x56242c0c6390 2024-08-31T07:36:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.290+0000 7f3267400640 1 -- 172.21.15.32:0/2267104682 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7f32780046c0 con 0x56242c0c6390 2024-08-31T07:36:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.290+0000 7f329349b740 10 client.29912 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.290+0000 7f329349b740 10 client.29912 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:36:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.290+0000 7f329349b740 20 client.29912 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.290+0000 7f329349b740 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56242c0a59f0 0x56242c4aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.290+0000 7f329349b740 4 client.29912 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:36:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.290+0000 7f329349b740 1 -- 172.21.15.32:0/2267104682 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56242c405700 con 0x56242c0a59f0 2024-08-31T07:36:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.291+0000 7f3289e00640 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56242c0a59f0 0x56242c4aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.292+0000 7f3267400640 10 client.29912 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:36:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.293+0000 7f3289e00640 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56242c0a59f0 0x56242c4aa760 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32e5710c9f2cca9e server_cookie=6ee35bb465a799aa in_seq=0 out_seq=0 2024-08-31T07:36:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.294+0000 7f3267400640 1 -- 172.21.15.32:0/2267104682 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56242c405700 con 0x56242c0a59f0 2024-08-31T07:36:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.294+0000 7f3267400640 10 client.29912 handle_command_reply: tid=0 2024-08-31T07:36:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.294+0000 7f329349b740 1 client.29912 shutdown 2024-08-31T07:36:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.295+0000 7f329349b740 1 -- 172.21.15.32:0/2267104682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 msgr2=0x56242c498d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.295+0000 7f329349b740 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c498d90 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f32780186a0 tx=0x7f32780186d0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.295+0000 7f329349b740 1 -- 172.21.15.32:0/2267104682 shutdown_connections 2024-08-31T07:36:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.295+0000 7f329349b740 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56242c0a59f0 0x56242c4aa760 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.295+0000 7f329349b740 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0e7070 0x56242c4a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.295+0000 7f329349b740 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c498d90 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.295+0000 7f329349b740 1 --2- 172.21.15.32:0/2267104682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c4987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.295+0000 7f329349b740 1 -- 172.21.15.32:0/2267104682 >> 172.21.15.32:0/2267104682 conn(0x56242c0db460 msgr2=0x56242c4891a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.295+0000 7f3267400640 0 client.29912 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:36:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.296+0000 7f329349b740 1 -- 172.21.15.32:0/2267104682 shutdown_connections 2024-08-31T07:36:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.296+0000 7f329349b740 1 -- 172.21.15.32:0/2267104682 wait complete. 2024-08-31T07:36:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.296+0000 7f329349b740 20 client.29912 trim_cache size 0 max 16384 2024-08-31T07:36:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.325+0000 7f329349b740 1 Processor -- start 2024-08-31T07:36:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.325+0000 7f329349b740 1 -- start start 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.325+0000 7f329349b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c498e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.325+0000 7f329349b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0c6390 0x56242c4a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.325+0000 7f329349b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0e7070 0x56242c4891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.325+0000 7f329349b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32841b58d0 con 0x56242c0c6390 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.325+0000 7f329349b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32841b7590 con 0x56242c0e7070 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.325+0000 7f329349b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32841b5fa0 con 0x56242c0be1b0 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.326+0000 7f3289e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c498e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.326+0000 7f3289e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c498e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59806/0 (socket says 172.21.15.32:59806) 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.326+0000 7f3289e00640 1 -- 172.21.15.32:0/769654949 learned_addr learned my addr 172.21.15.32:0/769654949 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.326+0000 7f328a800640 1 --2- 172.21.15.32:0/769654949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0e7070 0x56242c4891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.326+0000 7f3289400640 1 --2- 172.21.15.32:0/769654949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0c6390 0x56242c4a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.326+0000 7f3289e00640 1 -- 172.21.15.32:0/769654949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0e7070 msgr2=0x56242c4891a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.326+0000 7f3289e00640 1 --2- 172.21.15.32:0/769654949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0e7070 0x56242c4891a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.326+0000 7f3289e00640 1 -- 172.21.15.32:0/769654949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0c6390 msgr2=0x56242c4a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.326+0000 7f3289e00640 1 --2- 172.21.15.32:0/769654949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0c6390 0x56242c4a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.326+0000 7f3289e00640 1 -- 172.21.15.32:0/769654949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242c48ee80 con 0x56242c0be1b0 2024-08-31T07:36:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.327+0000 7f328a800640 1 --2- 172.21.15.32:0/769654949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0e7070 0x56242c4891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:36:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.328+0000 7f3289e00640 1 --2- 172.21.15.32:0/769654949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c498e60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3274006010 tx=0x7f327403b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0fbbc0039fbbeda server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.329+0000 7f3272600640 1 -- 172.21.15.32:0/769654949 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274049030 con 0x56242c0be1b0 2024-08-31T07:36:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.329+0000 7f3272600640 1 -- 172.21.15.32:0/769654949 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32740cc360 con 0x56242c0be1b0 2024-08-31T07:36:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.329+0000 7f3272600640 1 -- 172.21.15.32:0/769654949 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32740c85f0 con 0x56242c0be1b0 2024-08-31T07:36:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.330+0000 7f329349b740 1 -- 172.21.15.32:0/769654949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 msgr2=0x56242c498e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.330+0000 7f329349b740 1 --2- 172.21.15.32:0/769654949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c498e60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3274006010 tx=0x7f327403b6b0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.331+0000 7f329349b740 1 -- 172.21.15.32:0/769654949 shutdown_connections 2024-08-31T07:36:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.331+0000 7f329349b740 1 --2- 172.21.15.32:0/769654949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0e7070 0x56242c4891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.331+0000 7f329349b740 1 --2- 172.21.15.32:0/769654949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0c6390 0x56242c4a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.331+0000 7f329349b740 1 --2- 172.21.15.32:0/769654949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c498e60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.331+0000 7f329349b740 1 -- 172.21.15.32:0/769654949 >> 172.21.15.32:0/769654949 conn(0x56242c0db460 msgr2=0x56242c496e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.331+0000 7f329349b740 1 -- 172.21.15.32:0/769654949 shutdown_connections 2024-08-31T07:36:07.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.331+0000 7f329349b740 1 -- 172.21.15.32:0/769654949 wait complete. 2024-08-31T07:36:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.332+0000 7f329349b740 1 -- start start 2024-08-31T07:36:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.333+0000 7f329349b740 1 Processor -- start 2024-08-31T07:36:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.334+0000 7f329349b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c4ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.334+0000 7f329349b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c4a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.335+0000 7f329349b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0e7070 0x56242c4aa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.335+0000 7f329349b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32841b6e30 con 0x56242c0e7070 2024-08-31T07:36:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.335+0000 7f3289400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c4a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.335+0000 7f329349b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32841b66a0 con 0x56242c0c6390 2024-08-31T07:36:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.335+0000 7f329349b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32841b3870 con 0x56242c0be1b0 2024-08-31T07:36:07.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.335+0000 7f329349b740 10 client.0 _ll_register_callbacks cb 0x7f327800a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:07.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.335+0000 7f3289e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c4ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.335+0000 7f3289400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c4a9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55772/0 (socket says 172.21.15.32:55772) 2024-08-31T07:36:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.335+0000 7f3289400640 1 -- 172.21.15.32:0/444392162 learned_addr learned my addr 172.21.15.32:0/444392162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.335+0000 7f3289e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c4ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59808/0 (socket says 172.21.15.32:59808) 2024-08-31T07:36:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.336+0000 7f3289400640 1 -- 172.21.15.32:0/444392162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 msgr2=0x56242c4ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.336+0000 7f3289400640 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c4ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.336+0000 7f3289400640 1 -- 172.21.15.32:0/444392162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0e7070 msgr2=0x56242c4aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.336+0000 7f3289400640 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0e7070 0x56242c4aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.336+0000 7f3289400640 1 -- 172.21.15.32:0/444392162 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242c4aaaa0 con 0x56242c0c6390 2024-08-31T07:36:07.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.336+0000 7f3289e00640 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c4ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:36:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.337+0000 7f3289400640 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c4a9d30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3278010890 tx=0x7f3278018a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee4f1935dcc7b231 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.337+0000 7f3267400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:36:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.337+0000 7f3267400640 1 -- 172.21.15.32:0/444392162 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278039020 con 0x56242c0c6390 2024-08-31T07:36:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.338+0000 7f3267400640 1 -- 172.21.15.32:0/444392162 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8485+0+0 (secure 0 0 0) 0x7f3278010cd0 con 0x56242c0c6390 2024-08-31T07:36:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.338+0000 7f329349b740 1 -- 172.21.15.32:0/444392162 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32740cc360 con 0x56242c0c6390 2024-08-31T07:36:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.338+0000 7f3267400640 1 -- 172.21.15.32:0/444392162 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3278044050 con 0x56242c0c6390 2024-08-31T07:36:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.338+0000 7f329349b740 10 client.29918 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.338+0000 7f329349b740 1 -- 172.21.15.32:0/444392162 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x56242c514eb0 con 0x56242c0c6390 2024-08-31T07:36:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.339+0000 7f3267400640 1 -- 172.21.15.32:0/444392162 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32740cc360 con 0x56242c0c6390 2024-08-31T07:36:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.339+0000 7f3267400640 1 -- 172.21.15.32:0/444392162 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7f3278005490 con 0x56242c0c6390 2024-08-31T07:36:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.339+0000 7f329349b740 10 client.29918 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.339+0000 7f329349b740 10 client.29918 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:36:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.339+0000 7f329349b740 20 client.29918 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.339+0000 7f329349b740 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56242c07ada0 0x56242c517580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.340+0000 7f329349b740 4 client.29918 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:36:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.340+0000 7f329349b740 1 -- 172.21.15.32:0/444392162 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56242c4011a0 con 0x56242c07ada0 2024-08-31T07:36:07.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.340+0000 7f3289e00640 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56242c07ada0 0x56242c517580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:07.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.342+0000 7f3289e00640 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56242c07ada0 0x56242c517580 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad316b57da39ce48 server_cookie=43f255b125e7a475 in_seq=0 out_seq=0 2024-08-31T07:36:07.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.342+0000 7f3267400640 10 client.29918 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:36:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.344+0000 7f3267400640 1 -- 172.21.15.32:0/444392162 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56242c4011a0 con 0x56242c07ada0 2024-08-31T07:36:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.344+0000 7f3267400640 10 client.29918 handle_command_reply: tid=0 2024-08-31T07:36:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.344+0000 7f329349b740 1 client.29918 shutdown 2024-08-31T07:36:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.344+0000 7f329349b740 1 -- 172.21.15.32:0/444392162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 msgr2=0x56242c4a9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.344+0000 7f329349b740 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c4a9d30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3278010890 tx=0x7f3278018a20 comp rx=0 tx=0).stop 2024-08-31T07:36:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.345+0000 7f329349b740 1 -- 172.21.15.32:0/444392162 shutdown_connections 2024-08-31T07:36:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.345+0000 7f329349b740 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x56242c07ada0 0x56242c517580 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.345+0000 7f329349b740 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56242c0e7070 0x56242c4aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.345+0000 7f329349b740 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56242c0c6390 0x56242c4a9d30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.345+0000 7f329349b740 1 --2- 172.21.15.32:0/444392162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56242c0be1b0 0x56242c4ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.345+0000 7f329349b740 1 -- 172.21.15.32:0/444392162 >> 172.21.15.32:0/444392162 conn(0x56242c0db460 msgr2=0x56242c491ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.345+0000 7f3267400640 0 client.29918 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:36:07.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.346+0000 7f329349b740 1 -- 172.21.15.32:0/444392162 shutdown_connections 2024-08-31T07:36:07.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.346+0000 7f329349b740 1 -- 172.21.15.32:0/444392162 wait complete. 2024-08-31T07:36:07.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.347+0000 7f329349b740 20 client.29918 trim_cache size 0 max 16384 2024-08-31T07:36:07.347 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:36:07.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.348+0000 7f328b200640 1 -- 172.21.15.32:0/1025188068 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f326c0b4ca0 msgr2=0x7f326c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.348+0000 7f328b200640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f326c0b4ca0 0x7f326c0b71b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3280005fd0 tx=0x7f3280009500 comp rx=0 tx=0).stop 2024-08-31T07:36:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.348+0000 7f328b200640 1 -- 172.21.15.32:0/1025188068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32841021a0 msgr2=0x7f32841a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.348+0000 7f328b200640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32841021a0 0x7f32841a0680 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f327402a9f0 tx=0x7f3274003c00 comp rx=0 tx=0).stop 2024-08-31T07:36:07.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.349+0000 7f328b200640 1 -- 172.21.15.32:0/1025188068 shutdown_connections 2024-08-31T07:36:07.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.349+0000 7f328b200640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f326c0b4ca0 0x7f326c0b71b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.349+0000 7f328b200640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3284114540 0x7f32841a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.349+0000 7f328b200640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3284102af0 0x7f32841a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.349+0000 7f328b200640 1 --2- 172.21.15.32:0/1025188068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32841021a0 0x7f32841a0680 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:07.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.349+0000 7f328b200640 1 -- 172.21.15.32:0/1025188068 >> 172.21.15.32:0/1025188068 conn(0x7f32840fe060 msgr2=0x7f3284100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:07.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.350+0000 7f328b200640 1 -- 172.21.15.32:0/1025188068 shutdown_connections 2024-08-31T07:36:07.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:07.350+0000 7f328b200640 1 -- 172.21.15.32:0/1025188068 wait complete. 2024-08-31T07:36:07.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 162532 12.98 MB/sec execute 467 sec latency 95.149 ms 2024-08-31T07:36:08.056 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:07 smithi032 ceph-mon[17252]: osdmap e115: 12 total, 12 up, 12 in 2024-08-31T07:36:08.108 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2e13427', total seconds elapsed: 35.9 2024-08-31T07:36:08.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:07 smithi073 ceph-mon[21995]: osdmap e115: 12 total, 12 up, 12 in 2024-08-31T07:36:08.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:07 smithi084 ceph-mon[22387]: osdmap e115: 12 total, 12 up, 12 in 2024-08-31T07:36:08.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 163223 13.05 MB/sec execute 468 sec latency 67.885 ms 2024-08-31T07:36:08.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:36:09.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:08 smithi032 ceph-mon[17252]: pgmap v1104: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 263 KiB/s wr, 46 op/s 2024-08-31T07:36:09.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:08 smithi073 ceph-mon[21995]: pgmap v1104: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 263 KiB/s wr, 46 op/s 2024-08-31T07:36:09.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:08 smithi084 ceph-mon[22387]: pgmap v1104: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 263 KiB/s wr, 46 op/s 2024-08-31T07:36:09.428 INFO:tasks.workunit.client.0.smithi032.stdout: 10 163728 13.07 MB/sec execute 469 sec latency 111.382 ms 2024-08-31T07:36:09.477 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:10.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.390+0000 7f9c51600640 1 -- 172.21.15.32:0/4219548046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c10f890 msgr2=0x7f9c4c10fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:10.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.390+0000 7f9c51600640 1 --2- 172.21.15.32:0/4219548046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c10f890 0x7f9c4c10fca0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009900 tx=0x7f9c3802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.391+0000 7f9c51600640 1 -- 172.21.15.32:0/4219548046 shutdown_connections 2024-08-31T07:36:10.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.391+0000 7f9c51600640 1 --2- 172.21.15.32:0/4219548046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4c1101e0 0x7f9c4c116a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.391+0000 7f9c51600640 1 --2- 172.21.15.32:0/4219548046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c10f890 0x7f9c4c10fca0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.391+0000 7f9c51600640 1 --2- 172.21.15.32:0/4219548046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c4c109f50 0x7f9c4c10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.391+0000 7f9c51600640 1 -- 172.21.15.32:0/4219548046 >> 172.21.15.32:0/4219548046 conn(0x7f9c4c077e80 msgr2=0x7f9c4c107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.391+0000 7f9c51600640 1 -- 172.21.15.32:0/4219548046 shutdown_connections 2024-08-31T07:36:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.391+0000 7f9c51600640 1 -- 172.21.15.32:0/4219548046 wait complete. 2024-08-31T07:36:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.392+0000 7f9c51600640 1 Processor -- start 2024-08-31T07:36:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.393+0000 7f9c51600640 1 -- start start 2024-08-31T07:36:10.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.393+0000 7f9c51600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c4c109f50 0x7f9c4c083aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.393+0000 7f9c51600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4c10f890 0x7f9c4c07cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.393+0000 7f9c51600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1101e0 0x7f9c4c07d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.393+0000 7f9c51600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c119ef0 con 0x7f9c4c109f50 2024-08-31T07:36:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.393+0000 7f9c51600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c119d70 con 0x7f9c4c1101e0 2024-08-31T07:36:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.393+0000 7f9c51600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c11a070 con 0x7f9c4c10f890 2024-08-31T07:36:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.394+0000 7f9c4a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4c10f890 0x7f9c4c07cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.394+0000 7f9c4a000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4c10f890 0x7f9c4c07cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59816/0 (socket says 172.21.15.32:59816) 2024-08-31T07:36:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.394+0000 7f9c4a000640 1 -- 172.21.15.32:0/2863022698 learned_addr learned my addr 172.21.15.32:0/2863022698 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.394+0000 7f9c4b400640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1101e0 0x7f9c4c07d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.395+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c4c109f50 0x7f9c4c083aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.395+0000 7f9c4a000640 1 -- 172.21.15.32:0/2863022698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1101e0 msgr2=0x7f9c4c07d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.395+0000 7f9c4a000640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1101e0 0x7f9c4c07d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.395+0000 7f9c4a000640 1 -- 172.21.15.32:0/2863022698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c4c109f50 msgr2=0x7f9c4c083aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:10.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.395+0000 7f9c4a000640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c4c109f50 0x7f9c4c083aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.395+0000 7f9c4a000640 1 -- 172.21.15.32:0/2863022698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c07daf0 con 0x7f9c4c10f890 2024-08-31T07:36:10.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.395+0000 7f9c4a000640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4c10f890 0x7f9c4c07cd70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9c38005ab0 tx=0x7f9c38003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:10.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.396+0000 7f9c43400640 1 -- 172.21.15.32:0/2863022698 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3803f070 con 0x7f9c4c10f890 2024-08-31T07:36:10.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.396+0000 7f9c43400640 1 -- 172.21.15.32:0/2863022698 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c38031d60 con 0x7f9c4c10f890 2024-08-31T07:36:10.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.396+0000 7f9c43400640 1 -- 172.21.15.32:0/2863022698 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38035030 con 0x7f9c4c10f890 2024-08-31T07:36:10.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.396+0000 7f9c51600640 1 -- 172.21.15.32:0/2863022698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c082710 con 0x7f9c4c10f890 2024-08-31T07:36:10.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.396+0000 7f9c51600640 1 -- 172.21.15.32:0/2863022698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c082c40 con 0x7f9c4c10f890 2024-08-31T07:36:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.404+0000 7f9c43400640 1 -- 172.21.15.32:0/2863022698 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9c38042070 con 0x7f9c4c10f890 2024-08-31T07:36:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.405+0000 7f9c5826c740 1 Processor -- start 2024-08-31T07:36:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.406+0000 7f9c5826c740 1 -- start start 2024-08-31T07:36:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.406+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 0x55e73d57d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.407+0000 7f9c43400640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c300b4d20 0x7f9c300b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.406+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1af1b0 0x55e73d57ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.407+0000 7f9c43400640 1 -- 172.21.15.32:0/2863022698 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8485+0+0 (secure 0 0 0) 0x7f9c38105d40 con 0x7f9c4c10f890 2024-08-31T07:36:10.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.407+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 0x55e73d57d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.407+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 0x55e73d57d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55800/0 (socket says 172.21.15.32:55800) 2024-08-31T07:36:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.408+0000 7f9c4a000640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c300b4d20 0x7f9c300b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.408+0000 7f9c4b400640 1 -- 172.21.15.32:0/2220531204 learned_addr learned my addr 172.21.15.32:0/2220531204 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.409+0000 7f9c5826c740 1 --2- 172.21.15.32:0/2220531204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1b7390 0x55e73d57e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.409+0000 7f9c5826c740 1 -- 172.21.15.32:0/2220531204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e73d4f47b0 con 0x55e73d1af1b0 2024-08-31T07:36:10.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.409+0000 7f9c4a000640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c300b4d20 0x7f9c300b7230 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9c380cc150 tx=0x7f9c380045e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.409+0000 7f9c5826c740 1 -- 172.21.15.32:0/2220531204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e73d4e3d40 con 0x55e73d1d8070 2024-08-31T07:36:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.409+0000 7f9c5826c740 1 -- 172.21.15.32:0/2220531204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e73d4e7cf0 con 0x55e73d1b7390 2024-08-31T07:36:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.410+0000 7f9c4b400640 1 --2- 172.21.15.32:0/2220531204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1b7390 0x55e73d57e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.410+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/2220531204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1af1b0 0x55e73d57ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.410+0000 7f9c4b400640 1 -- 172.21.15.32:0/2220531204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1b7390 msgr2=0x55e73d57e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.410+0000 7f9c4b400640 1 --2- 172.21.15.32:0/2220531204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1b7390 0x55e73d57e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.410+0000 7f9c4b400640 1 -- 172.21.15.32:0/2220531204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1af1b0 msgr2=0x55e73d57ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.410+0000 7f9c4b400640 1 --2- 172.21.15.32:0/2220531204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1af1b0 0x55e73d57ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.410+0000 7f9c4b400640 1 -- 172.21.15.32:0/2220531204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e73d243650 con 0x55e73d1d8070 2024-08-31T07:36:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.411+0000 7f9c4b400640 1 --2- 172.21.15.32:0/2220531204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 0x55e73d57d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9c44014700 tx=0x7f9c44009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0a0427b909313f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.411+0000 7f9c37400640 1 -- 172.21.15.32:0/2220531204 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44014d40 con 0x55e73d1d8070 2024-08-31T07:36:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.411+0000 7f9c37400640 1 -- 172.21.15.32:0/2220531204 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c44014110 con 0x55e73d1d8070 2024-08-31T07:36:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.411+0000 7f9c37400640 1 -- 172.21.15.32:0/2220531204 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44005280 con 0x55e73d1d8070 2024-08-31T07:36:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.413+0000 7f9c5826c740 1 -- 172.21.15.32:0/2220531204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 msgr2=0x55e73d57d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:10.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.413+0000 7f9c5826c740 1 --2- 172.21.15.32:0/2220531204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 0x55e73d57d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9c44014700 tx=0x7f9c44009b10 comp rx=0 tx=0).stop 2024-08-31T07:36:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.413+0000 7f9c5826c740 1 -- 172.21.15.32:0/2220531204 shutdown_connections 2024-08-31T07:36:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.413+0000 7f9c5826c740 1 --2- 172.21.15.32:0/2220531204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1b7390 0x55e73d57e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.413+0000 7f9c5826c740 1 --2- 172.21.15.32:0/2220531204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1af1b0 0x55e73d57ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.413+0000 7f9c5826c740 1 --2- 172.21.15.32:0/2220531204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 0x55e73d57d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.413+0000 7f9c5826c740 1 -- 172.21.15.32:0/2220531204 >> 172.21.15.32:0/2220531204 conn(0x55e73d1cc460 msgr2=0x55e73d57b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.413+0000 7f9c5826c740 1 -- 172.21.15.32:0/2220531204 shutdown_connections 2024-08-31T07:36:10.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.413+0000 7f9c5826c740 1 -- 172.21.15.32:0/2220531204 wait complete. 2024-08-31T07:36:10.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.414+0000 7f9c5826c740 1 -- start start 2024-08-31T07:36:10.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.414+0000 7f9c5826c740 1 Processor -- start 2024-08-31T07:36:10.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.415+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1af1b0 0x55e73d5896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.415+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 0x55e73d589c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.415+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1d8070 0x55e73d58f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.415+0000 7f9c5826c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e73d346700 con 0x55e73d1b7390 2024-08-31T07:36:10.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.415+0000 7f9c5826c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e73d46acf0 con 0x55e73d1af1b0 2024-08-31T07:36:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.415+0000 7f9c5826c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e73d58c170 con 0x55e73d1d8070 2024-08-31T07:36:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.415+0000 7f9c5826c740 10 client.0 _ll_register_callbacks cb 0x55e73d490470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.416+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1af1b0 0x55e73d5896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.416+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1d8070 0x55e73d58f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.416+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1d8070 0x55e73d58f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59834/0 (socket says 172.21.15.32:59834) 2024-08-31T07:36:10.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.416+0000 7f9c4b400640 1 -- 172.21.15.32:0/3287553761 learned_addr learned my addr 172.21.15.32:0/3287553761 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.417+0000 7f9c4b400640 1 -- 172.21.15.32:0/3287553761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1af1b0 msgr2=0x55e73d5896e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.417+0000 7f9c4b400640 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1af1b0 0x55e73d5896e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.417+0000 7f9c4b400640 1 -- 172.21.15.32:0/3287553761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 msgr2=0x55e73d589c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.417+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 0x55e73d589c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.417+0000 7f9c4b400640 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 0x55e73d589c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.417+0000 7f9c4b400640 1 -- 172.21.15.32:0/3287553761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e73d4ab8b0 con 0x55e73d1d8070 2024-08-31T07:36:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.418+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 0x55e73d589c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:36:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.418+0000 7f9c4b400640 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1d8070 0x55e73d58f430 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55e73d588d00 tx=0x7f9c440145a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f73b00d2a94aeb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.418+0000 7f9c2be00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:36:10.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.418+0000 7f9c2be00640 1 -- 172.21.15.32:0/3287553761 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44004c50 con 0x55e73d1d8070 2024-08-31T07:36:10.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.418+0000 7f9c2be00640 1 -- 172.21.15.32:0/3287553761 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8485+0+0 (secure 0 0 0) 0x7f9c44025030 con 0x55e73d1d8070 2024-08-31T07:36:10.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.418+0000 7f9c5826c740 1 -- 172.21.15.32:0/3287553761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c44014110 con 0x55e73d1d8070 2024-08-31T07:36:10.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.419+0000 7f9c2be00640 1 -- 172.21.15.32:0/3287553761 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f9c44004df0 con 0x55e73d1d8070 2024-08-31T07:36:10.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.419+0000 7f9c5826c740 10 client.29929 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:10.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.419+0000 7f9c5826c740 1 -- 172.21.15.32:0/3287553761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55e73d5982b0 con 0x55e73d1d8070 2024-08-31T07:36:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.419+0000 7f9c2be00640 1 -- 172.21.15.32:0/3287553761 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c44024020 con 0x55e73d1d8070 2024-08-31T07:36:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.420+0000 7f9c2be00640 1 -- 172.21.15.32:0/3287553761 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7f9c4400f040 con 0x55e73d1d8070 2024-08-31T07:36:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.420+0000 7f9c5826c740 10 client.29929 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.420+0000 7f9c5826c740 10 client.29929 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:36:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.420+0000 7f9c5826c740 20 client.29929 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.420+0000 7f9c5826c740 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55e73d1969f0 0x55e73d59b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.420+0000 7f9c5826c740 4 client.29929 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:36:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.420+0000 7f9c5826c740 1 -- 172.21.15.32:0/3287553761 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9c44014110 con 0x55e73d1969f0 2024-08-31T07:36:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.420+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55e73d1969f0 0x55e73d59b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.421+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55e73d1969f0 0x55e73d59b720 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c26cbd5ad8576394 server_cookie=4b059c538e9f3e30 in_seq=0 out_seq=0 2024-08-31T07:36:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.421+0000 7f9c2be00640 10 client.29929 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:36:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.426+0000 7f9c2be00640 1 -- 172.21.15.32:0/3287553761 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f9c44014110 con 0x55e73d1969f0 2024-08-31T07:36:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.426+0000 7f9c2be00640 10 client.29929 handle_command_reply: tid=0 2024-08-31T07:36:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.426+0000 7f9c5826c740 1 client.29929 shutdown 2024-08-31T07:36:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.426+0000 7f9c5826c740 1 -- 172.21.15.32:0/3287553761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1d8070 msgr2=0x55e73d58f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.426+0000 7f9c5826c740 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1d8070 0x55e73d58f430 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55e73d588d00 tx=0x7f9c440145a0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.428 INFO:tasks.workunit.client.0.smithi032.stdout: 10 164434 13.11 MB/sec execute 470 sec latency 65.261 ms 2024-08-31T07:36:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.427+0000 7f9c5826c740 1 -- 172.21.15.32:0/3287553761 shutdown_connections 2024-08-31T07:36:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.427+0000 7f9c5826c740 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1d8070 0x55e73d58f430 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.427+0000 7f9c5826c740 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 0x55e73d589c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.427+0000 7f9c5826c740 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55e73d1969f0 0x55e73d59b720 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.427+0000 7f9c5826c740 1 --2- 172.21.15.32:0/3287553761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1af1b0 0x55e73d5896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.427+0000 7f9c5826c740 1 -- 172.21.15.32:0/3287553761 >> 172.21.15.32:0/3287553761 conn(0x55e73d1cc460 msgr2=0x55e73d580b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.427+0000 7f9c2be00640 0 client.29929 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:36:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.428+0000 7f9c5826c740 1 -- 172.21.15.32:0/3287553761 shutdown_connections 2024-08-31T07:36:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.428+0000 7f9c5826c740 1 -- 172.21.15.32:0/3287553761 wait complete. 2024-08-31T07:36:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.428+0000 7f9c5826c740 20 client.29929 trim_cache size 0 max 16384 2024-08-31T07:36:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.454+0000 7f9c5826c740 1 Processor -- start 2024-08-31T07:36:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.454+0000 7f9c5826c740 1 -- start start 2024-08-31T07:36:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.454+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d57ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.454+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1b7390 0x55e73d588320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.454+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1d8070 0x55e73d58f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.454+0000 7f9c5826c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1be2c0 con 0x55e73d1d8070 2024-08-31T07:36:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.455+0000 7f9c5826c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1bff80 con 0x55e73d1b7390 2024-08-31T07:36:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.455+0000 7f9c5826c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1be990 con 0x55e73d1af1b0 2024-08-31T07:36:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.455+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d57ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.455+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d57ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59842/0 (socket says 172.21.15.32:59842) 2024-08-31T07:36:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.455+0000 7f9c4b400640 1 -- 172.21.15.32:0/2653052312 learned_addr learned my addr 172.21.15.32:0/2653052312 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.455+0000 7f9c4b400640 1 --2- 172.21.15.32:0/2653052312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1d8070 0x55e73d58f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.455+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/2653052312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1b7390 0x55e73d588320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.456+0000 7f9c4aa00640 1 -- 172.21.15.32:0/2653052312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 msgr2=0x55e73d57ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.456+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/2653052312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d57ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.456+0000 7f9c4aa00640 1 -- 172.21.15.32:0/2653052312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1d8070 msgr2=0x55e73d58f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.456+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/2653052312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1d8070 0x55e73d58f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.456+0000 7f9c4aa00640 1 -- 172.21.15.32:0/2653052312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e73d589270 con 0x55e73d1b7390 2024-08-31T07:36:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.456+0000 7f9c4b400640 1 --2- 172.21.15.32:0/2653052312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d57ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.456+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/2653052312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1b7390 0x55e73d588320 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x55e73d59bdd0 tx=0x7f9c3c005ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79f6b533000b3f17 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.457+0000 7f9c36a00640 1 -- 172.21.15.32:0/2653052312 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00eb80 con 0x55e73d1b7390 2024-08-31T07:36:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.457+0000 7f9c36a00640 1 -- 172.21.15.32:0/2653052312 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c3c01b070 con 0x55e73d1b7390 2024-08-31T07:36:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.457+0000 7f9c36a00640 1 -- 172.21.15.32:0/2653052312 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c018590 con 0x55e73d1b7390 2024-08-31T07:36:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.458+0000 7f9c5826c740 1 -- 172.21.15.32:0/2653052312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1b7390 msgr2=0x55e73d588320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.458+0000 7f9c5826c740 1 --2- 172.21.15.32:0/2653052312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1b7390 0x55e73d588320 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x55e73d59bdd0 tx=0x7f9c3c005ca0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.459+0000 7f9c5826c740 1 -- 172.21.15.32:0/2653052312 shutdown_connections 2024-08-31T07:36:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.459+0000 7f9c5826c740 1 --2- 172.21.15.32:0/2653052312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1d8070 0x55e73d58f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.459+0000 7f9c5826c740 1 --2- 172.21.15.32:0/2653052312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1b7390 0x55e73d588320 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.459+0000 7f9c5826c740 1 --2- 172.21.15.32:0/2653052312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d57ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.459+0000 7f9c5826c740 1 -- 172.21.15.32:0/2653052312 >> 172.21.15.32:0/2653052312 conn(0x55e73d1cc460 msgr2=0x55e73d587c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.459+0000 7f9c5826c740 1 -- 172.21.15.32:0/2653052312 shutdown_connections 2024-08-31T07:36:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.459+0000 7f9c5826c740 1 -- 172.21.15.32:0/2653052312 wait complete. 2024-08-31T07:36:10.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.460+0000 7f9c5826c740 1 -- start start 2024-08-31T07:36:10.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.461+0000 7f9c5826c740 1 Processor -- start 2024-08-31T07:36:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.462+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d5961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.462+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 0x55e73d5916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.462+0000 7f9c5826c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 0x55e73d591da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.462+0000 7f9c5826c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1bf820 con 0x55e73d1b7390 2024-08-31T07:36:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.462+0000 7f9c5826c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1bf090 con 0x55e73d1d8070 2024-08-31T07:36:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.462+0000 7f9c5826c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1bc2c0 con 0x55e73d1af1b0 2024-08-31T07:36:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.462+0000 7f9c5826c740 10 client.0 _ll_register_callbacks cb 0x7f9c44014420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:10.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.463+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d5961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.463+0000 7f9c4aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 0x55e73d5916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.463+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d5961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59854/0 (socket says 172.21.15.32:59854) 2024-08-31T07:36:10.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.463+0000 7f9c4b400640 1 -- 172.21.15.32:0/972777026 learned_addr learned my addr 172.21.15.32:0/972777026 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:10.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.463+0000 7f9c4b400640 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 0x55e73d591da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.463+0000 7f9c4b400640 1 -- 172.21.15.32:0/972777026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 msgr2=0x55e73d591da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.463+0000 7f9c4b400640 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 0x55e73d591da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.464+0000 7f9c4b400640 1 -- 172.21.15.32:0/972777026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 msgr2=0x55e73d5916b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.464+0000 7f9c4b400640 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 0x55e73d5916b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.464+0000 7f9c4b400640 1 -- 172.21.15.32:0/972777026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e73d605090 con 0x55e73d1af1b0 2024-08-31T07:36:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.464+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 0x55e73d5916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:36:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.464+0000 7f9c4b400640 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d5961b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009960 tx=0x7f9c440097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c65f28da097884d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.464+0000 7f9c2be00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:36:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.465+0000 7f9c2be00640 1 -- 172.21.15.32:0/972777026 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44014110 con 0x55e73d1af1b0 2024-08-31T07:36:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.465+0000 7f9c2be00640 1 -- 172.21.15.32:0/972777026 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8485+0+0 (secure 0 0 0) 0x7f9c4401b070 con 0x55e73d1af1b0 2024-08-31T07:36:10.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.465+0000 7f9c5826c740 1 -- 172.21.15.32:0/972777026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c3c01b070 con 0x55e73d1af1b0 2024-08-31T07:36:10.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.465+0000 7f9c2be00640 1 -- 172.21.15.32:0/972777026 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f9c4401f970 con 0x55e73d1af1b0 2024-08-31T07:36:10.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.465+0000 7f9c5826c740 10 client.29935 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:10.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.465+0000 7f9c5826c740 1 -- 172.21.15.32:0/972777026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55e73d606150 con 0x55e73d1af1b0 2024-08-31T07:36:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.466+0000 7f9c2be00640 1 -- 172.21.15.32:0/972777026 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c3c01b070 con 0x55e73d1af1b0 2024-08-31T07:36:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.466+0000 7f9c2be00640 1 -- 172.21.15.32:0/972777026 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7f9c44042020 con 0x55e73d1af1b0 2024-08-31T07:36:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.466+0000 7f9c5826c740 10 client.29935 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.466+0000 7f9c5826c740 10 client.29935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:36:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.466+0000 7f9c5826c740 20 client.29935 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.466+0000 7f9c5826c740 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55e73d16bda0 0x55e73d608910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.466+0000 7f9c5826c740 4 client.29935 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:36:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.466+0000 7f9c5826c740 1 -- 172.21.15.32:0/972777026 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e73d4f6700 con 0x55e73d16bda0 2024-08-31T07:36:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.467+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55e73d16bda0 0x55e73d608910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.468+0000 7f9c2be00640 10 client.29935 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:36:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.468+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55e73d16bda0 0x55e73d608910 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=594617a8ac5fb406 server_cookie=a5f020a7d821de51 in_seq=0 out_seq=0 2024-08-31T07:36:10.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.470+0000 7f9c2be00640 1 -- 172.21.15.32:0/972777026 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e73d4f6700 con 0x55e73d16bda0 2024-08-31T07:36:10.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.470+0000 7f9c2be00640 10 client.29935 handle_command_reply: tid=0 2024-08-31T07:36:10.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.471+0000 7f9c5826c740 1 client.29935 shutdown 2024-08-31T07:36:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.471+0000 7f9c5826c740 1 -- 172.21.15.32:0/972777026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 msgr2=0x55e73d5961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.471+0000 7f9c5826c740 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d5961b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009960 tx=0x7f9c440097b0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.471+0000 7f9c5826c740 1 -- 172.21.15.32:0/972777026 shutdown_connections 2024-08-31T07:36:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.471+0000 7f9c5826c740 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55e73d16bda0 0x55e73d608910 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.471+0000 7f9c5826c740 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e73d1d8070 0x55e73d591da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.471+0000 7f9c5826c740 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e73d1b7390 0x55e73d5916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.471+0000 7f9c5826c740 1 --2- 172.21.15.32:0/972777026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e73d1af1b0 0x55e73d5961b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.471+0000 7f9c5826c740 1 -- 172.21.15.32:0/972777026 >> 172.21.15.32:0/972777026 conn(0x55e73d1cc460 msgr2=0x55e73d57b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.473+0000 7f9c2be00640 0 client.29935 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:36:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.473+0000 7f9c5826c740 1 -- 172.21.15.32:0/972777026 shutdown_connections 2024-08-31T07:36:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.473+0000 7f9c5826c740 1 -- 172.21.15.32:0/972777026 wait complete. 2024-08-31T07:36:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.473+0000 7f9c5826c740 20 client.29935 trim_cache size 0 max 16384 2024-08-31T07:36:10.475 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:36:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.475+0000 7f9c51600640 1 -- 172.21.15.32:0/2863022698 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c300b4d20 msgr2=0x7f9c300b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.475+0000 7f9c51600640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c300b4d20 0x7f9c300b7230 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9c380cc150 tx=0x7f9c380045e0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.475+0000 7f9c51600640 1 -- 172.21.15.32:0/2863022698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4c10f890 msgr2=0x7f9c4c07cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.475+0000 7f9c51600640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4c10f890 0x7f9c4c07cd70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9c38005ab0 tx=0x7f9c38003cd0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.475+0000 7f9c51600640 1 -- 172.21.15.32:0/2863022698 shutdown_connections 2024-08-31T07:36:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.475+0000 7f9c51600640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9c300b4d20 0x7f9c300b7230 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.475+0000 7f9c51600640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1101e0 0x7f9c4c07d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.475+0000 7f9c51600640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c4c10f890 0x7f9c4c07cd70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.475+0000 7f9c51600640 1 --2- 172.21.15.32:0/2863022698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c4c109f50 0x7f9c4c083aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.475+0000 7f9c51600640 1 -- 172.21.15.32:0/2863022698 >> 172.21.15.32:0/2863022698 conn(0x7f9c4c077e80 msgr2=0x7f9c4c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.476+0000 7f9c51600640 1 -- 172.21.15.32:0/2863022698 shutdown_connections 2024-08-31T07:36:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:10.476+0000 7f9c51600640 1 -- 172.21.15.32:0/2863022698 wait complete. 2024-08-31T07:36:11.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:10 smithi032 ceph-mon[17252]: pgmap v1105: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-08-31T07:36:11.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:10 smithi073 ceph-mon[21995]: pgmap v1105: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-08-31T07:36:11.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:10 smithi084 ceph-mon[22387]: pgmap v1105: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-08-31T07:36:11.428 INFO:tasks.workunit.client.0.smithi032.stdout: 10 165017 13.15 MB/sec execute 471 sec latency 351.811 ms 2024-08-31T07:36:12.427 INFO:tasks.workunit.client.0.smithi032.stdout: 10 165772 13.21 MB/sec execute 472 sec latency 496.150 ms 2024-08-31T07:36:12.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:36:13.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:12 smithi032 ceph-mon[17252]: pgmap v1106: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-08-31T07:36:13.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:12 smithi073 ceph-mon[21995]: pgmap v1106: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-08-31T07:36:13.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:12 smithi084 ceph-mon[22387]: pgmap v1106: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-08-31T07:36:13.349 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:13.428 INFO:tasks.workunit.client.0.smithi032.stdout: 10 166462 13.27 MB/sec execute 473 sec latency 1496.204 ms 2024-08-31T07:36:14.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.120+0000 7f18a1800640 1 -- 172.21.15.32:0/3227207837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f189c10aab0 msgr2=0x7f189c118160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.120+0000 7f18a1800640 1 --2- 172.21.15.32:0/3227207837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f189c10aab0 0x7f189c118160 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f188c009900 tx=0x7f188c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.120+0000 7f18a1800640 1 -- 172.21.15.32:0/3227207837 shutdown_connections 2024-08-31T07:36:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.120+0000 7f18a1800640 1 --2- 172.21.15.32:0/3227207837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189c1186a0 0x7f189c11aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.120+0000 7f18a1800640 1 --2- 172.21.15.32:0/3227207837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f189c10aab0 0x7f189c118160 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.120+0000 7f18a1800640 1 --2- 172.21.15.32:0/3227207837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189c10a160 0x7f189c10a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.120+0000 7f18a1800640 1 -- 172.21.15.32:0/3227207837 >> 172.21.15.32:0/3227207837 conn(0x7f189c06ebf0 msgr2=0x7f189c06f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.120+0000 7f18a1800640 1 -- 172.21.15.32:0/3227207837 shutdown_connections 2024-08-31T07:36:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.121+0000 7f18a1800640 1 -- 172.21.15.32:0/3227207837 wait complete. 2024-08-31T07:36:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.122+0000 7f18a1800640 1 Processor -- start 2024-08-31T07:36:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.122+0000 7f18a1800640 1 -- start start 2024-08-31T07:36:14.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.122+0000 7f18a1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189c10a160 0x7f189c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.122+0000 7f18a1800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f189c10aab0 0x7f189c1a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.122+0000 7f18a1800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189c1186a0 0x7f189c1aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.122+0000 7f18a1800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f189c11e020 con 0x7f189c1186a0 2024-08-31T07:36:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.122+0000 7f18a1800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f189c11dea0 con 0x7f189c10a160 2024-08-31T07:36:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.122+0000 7f18a1800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f189c11e1a0 con 0x7f189c10aab0 2024-08-31T07:36:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f189be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189c10a160 0x7f189c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f189be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189c10a160 0x7f189c1a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55826/0 (socket says 172.21.15.32:55826) 2024-08-31T07:36:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f189be00640 1 -- 172.21.15.32:0/4199427112 learned_addr learned my addr 172.21.15.32:0/4199427112 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:14.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f189b400640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f189c10aab0 0x7f189c1a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f189be00640 1 -- 172.21.15.32:0/4199427112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f189c10aab0 msgr2=0x7f189c1a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f189be00640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f189c10aab0 0x7f189c1a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f189be00640 1 -- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189c1186a0 msgr2=0x7f189c1aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f18a0e00640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189c1186a0 0x7f189c1aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f189be00640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189c1186a0 0x7f189c1aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f189be00640 1 -- 172.21.15.32:0/4199427112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c1aaae0 con 0x7f189c10a160 2024-08-31T07:36:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.123+0000 7f189b400640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f189c10aab0 0x7f189c1a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:36:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.124+0000 7f18a0e00640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189c1186a0 0x7f189c1aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:36:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.124+0000 7f189be00640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189c10a160 0x7f189c1a4b70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f188400ea60 tx=0x7f188400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.124+0000 7f1898c00640 1 -- 172.21.15.32:0/4199427112 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188400cde0 con 0x7f189c10a160 2024-08-31T07:36:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.124+0000 7f1898c00640 1 -- 172.21.15.32:0/4199427112 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18840035f0 con 0x7f189c10a160 2024-08-31T07:36:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.124+0000 7f1898c00640 1 -- 172.21.15.32:0/4199427112 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884010690 con 0x7f189c10a160 2024-08-31T07:36:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.125+0000 7f18a1800640 1 -- 172.21.15.32:0/4199427112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1aadd0 con 0x7f189c10a160 2024-08-31T07:36:14.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.125+0000 7f18a1800640 1 -- 172.21.15.32:0/4199427112 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1b1530 con 0x7f189c10a160 2024-08-31T07:36:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.129+0000 7f18a99c7740 1 Processor -- start 2024-08-31T07:36:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.129+0000 7f18a99c7740 1 -- start start 2024-08-31T07:36:14.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.130+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853e8070 0x55a38578db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.130+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853bf1b0 0x55a38578e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.130+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853c7390 0x55a38578e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.130+0000 7f18a99c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a3857047b0 con 0x55a3853c7390 2024-08-31T07:36:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.130+0000 7f18a99c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a3856f3d40 con 0x55a3853e8070 2024-08-31T07:36:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.130+0000 7f18a99c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a3856f7cf0 con 0x55a3853bf1b0 2024-08-31T07:36:14.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.131+0000 7f18a0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853e8070 0x55a38578db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.131+0000 7f18a0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853e8070 0x55a38578db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55832/0 (socket says 172.21.15.32:55832) 2024-08-31T07:36:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.131+0000 7f18a0e00640 1 -- 172.21.15.32:0/116420369 learned_addr learned my addr 172.21.15.32:0/116420369 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.131+0000 7f189b400640 1 --2- 172.21.15.32:0/116420369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853c7390 0x55a38578e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.131+0000 7f18a0e00640 1 -- 172.21.15.32:0/116420369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853bf1b0 msgr2=0x55a38578e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:36:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.131+0000 7f18a0e00640 1 --2- 172.21.15.32:0/116420369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853bf1b0 0x55a38578e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.131+0000 7f18a0e00640 1 -- 172.21.15.32:0/116420369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853c7390 msgr2=0x55a38578e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.131+0000 7f18a0e00640 1 --2- 172.21.15.32:0/116420369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853c7390 0x55a38578e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.131+0000 7f18a0e00640 1 -- 172.21.15.32:0/116420369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a385453650 con 0x55a3853e8070 2024-08-31T07:36:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.132+0000 7f1898c00640 1 -- 172.21.15.32:0/4199427112 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1884010830 con 0x7f189c10a160 2024-08-31T07:36:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.132+0000 7f18a0e00640 1 --2- 172.21.15.32:0/116420369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853e8070 0x55a38578db60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1894009ee0 tx=0x7f189400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a718b586720b04f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.133+0000 7f1891600640 1 -- 172.21.15.32:0/116420369 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894011640 con 0x55a3853e8070 2024-08-31T07:36:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.133+0000 7f1891600640 1 -- 172.21.15.32:0/116420369 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1894011c90 con 0x55a3853e8070 2024-08-31T07:36:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.133+0000 7f1891600640 1 -- 172.21.15.32:0/116420369 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189401bd60 con 0x55a3853e8070 2024-08-31T07:36:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.134+0000 7f18a99c7740 1 -- 172.21.15.32:0/116420369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853e8070 msgr2=0x55a38578db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.134+0000 7f18a99c7740 1 --2- 172.21.15.32:0/116420369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853e8070 0x55a38578db60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1894009ee0 tx=0x7f189400f730 comp rx=0 tx=0).stop 2024-08-31T07:36:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.134+0000 7f1898c00640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18800b4d20 0x7f18800b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.135+0000 7f1898c00640 1 -- 172.21.15.32:0/4199427112 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8485+0+0 (secure 0 0 0) 0x7f18840e58f0 con 0x7f189c10a160 2024-08-31T07:36:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.135+0000 7f18a0e00640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18800b4d20 0x7f18800b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.135+0000 7f18a99c7740 1 -- 172.21.15.32:0/116420369 shutdown_connections 2024-08-31T07:36:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.135+0000 7f18a99c7740 1 --2- 172.21.15.32:0/116420369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853c7390 0x55a38578e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.135+0000 7f18a99c7740 1 --2- 172.21.15.32:0/116420369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853bf1b0 0x55a38578e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.135+0000 7f18a99c7740 1 --2- 172.21.15.32:0/116420369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853e8070 0x55a38578db60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.136+0000 7f18a99c7740 1 -- 172.21.15.32:0/116420369 >> 172.21.15.32:0/116420369 conn(0x55a3853dc460 msgr2=0x55a38578ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.137+0000 7f18a0e00640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18800b4d20 0x7f18800b7230 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1894000f80 tx=0x7f1894009930 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.138+0000 7f18a99c7740 1 -- 172.21.15.32:0/116420369 shutdown_connections 2024-08-31T07:36:14.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.138+0000 7f18a99c7740 1 -- 172.21.15.32:0/116420369 wait complete. 2024-08-31T07:36:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.140+0000 7f18a99c7740 1 -- start start 2024-08-31T07:36:14.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.140+0000 7f18a99c7740 1 Processor -- start 2024-08-31T07:36:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.141+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 0x55a385799790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.141+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385799d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.141+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a3857adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.141+0000 7f18a99c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a385556700 con 0x55a3853e8070 2024-08-31T07:36:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.141+0000 7f18a99c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a38567acf0 con 0x55a3853bf1b0 2024-08-31T07:36:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.141+0000 7f18a99c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a38579c340 con 0x55a3853c7390 2024-08-31T07:36:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.141+0000 7f18a99c7740 10 client.0 _ll_register_callbacks cb 0x55a3856a0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.142+0000 7f189b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385799d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.142+0000 7f189b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385799d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59898/0 (socket says 172.21.15.32:59898) 2024-08-31T07:36:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.142+0000 7f189b400640 1 -- 172.21.15.32:0/2126296487 learned_addr learned my addr 172.21.15.32:0/2126296487 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.143+0000 7f18a0e00640 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a3857adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.143+0000 7f189be00640 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 0x55a385799790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.144+0000 7f189b400640 1 -- 172.21.15.32:0/2126296487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 msgr2=0x55a385799790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.144+0000 7f189b400640 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 0x55a385799790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.144+0000 7f189b400640 1 -- 172.21.15.32:0/2126296487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 msgr2=0x55a3857adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.144+0000 7f189b400640 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a3857adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.145+0000 7f189b400640 1 -- 172.21.15.32:0/2126296487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3856bb8b0 con 0x55a3853c7390 2024-08-31T07:36:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.145+0000 7f189be00640 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 0x55a385799790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.145+0000 7f18a0e00640 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a3857adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.146+0000 7f189b400640 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385799d30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f188c031440 tx=0x7f188c031750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62e2a7981148e79d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:14.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.147+0000 7f188a000640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:36:14.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.147+0000 7f188a000640 1 -- 172.21.15.32:0/2126296487 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c035030 con 0x55a3853c7390 2024-08-31T07:36:14.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.147+0000 7f188a000640 1 -- 172.21.15.32:0/2126296487 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8485+0+0 (secure 0 0 0) 0x7f188c004250 con 0x55a3853c7390 2024-08-31T07:36:14.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.148+0000 7f18a99c7740 1 -- 172.21.15.32:0/2126296487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1894011c90 con 0x55a3853c7390 2024-08-31T07:36:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.149+0000 7f188a000640 1 -- 172.21.15.32:0/2126296487 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f188c004450 con 0x55a3853c7390 2024-08-31T07:36:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.149+0000 7f18a99c7740 10 client.29944 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.149+0000 7f18a99c7740 1 -- 172.21.15.32:0/2126296487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55a3857b0ce0 con 0x55a3853c7390 2024-08-31T07:36:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.149+0000 7f188a000640 1 -- 172.21.15.32:0/2126296487 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1894011c90 con 0x55a3853c7390 2024-08-31T07:36:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.149+0000 7f188a000640 1 -- 172.21.15.32:0/2126296487 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7f188c004c50 con 0x55a3853c7390 2024-08-31T07:36:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.150+0000 7f18a99c7740 10 client.29944 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.150+0000 7f18a99c7740 10 client.29944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:36:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.150+0000 7f18a99c7740 20 client.29944 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.150+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a3853a69f0 0x55a3857b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.150+0000 7f18a99c7740 4 client.29944 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:36:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.150+0000 7f18a99c7740 1 -- 172.21.15.32:0/2126296487 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a385706700 con 0x55a3853a69f0 2024-08-31T07:36:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.151+0000 7f189be00640 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a3853a69f0 0x55a3857b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.152+0000 7f188a000640 10 client.29944 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.153+0000 7f189be00640 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a3853a69f0 0x55a3857b4280 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a86bf65fee5d887 server_cookie=267e43b0bd238d4d in_seq=0 out_seq=0 2024-08-31T07:36:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.155+0000 7f188a000640 1 -- 172.21.15.32:0/2126296487 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a385706700 con 0x55a3853a69f0 2024-08-31T07:36:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.155+0000 7f188a000640 10 client.29944 handle_command_reply: tid=0 2024-08-31T07:36:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.155+0000 7f18a99c7740 1 client.29944 shutdown 2024-08-31T07:36:14.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.155+0000 7f18a99c7740 1 -- 172.21.15.32:0/2126296487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 msgr2=0x55a385799d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.155+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385799d30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f188c031440 tx=0x7f188c031750 comp rx=0 tx=0).stop 2024-08-31T07:36:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.156+0000 7f18a99c7740 1 -- 172.21.15.32:0/2126296487 shutdown_connections 2024-08-31T07:36:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.156+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a3853a69f0 0x55a3857b4280 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.156+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a3857adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.156+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385799d30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.156+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2126296487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 0x55a385799790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.156+0000 7f18a99c7740 1 -- 172.21.15.32:0/2126296487 >> 172.21.15.32:0/2126296487 conn(0x55a3853dc460 msgr2=0x55a385790cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.156+0000 7f188a000640 0 client.29944 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:14.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.157+0000 7f18a99c7740 1 -- 172.21.15.32:0/2126296487 shutdown_connections 2024-08-31T07:36:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.157+0000 7f18a99c7740 1 -- 172.21.15.32:0/2126296487 wait complete. 2024-08-31T07:36:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.157+0000 7f18a99c7740 20 client.29944 trim_cache size 0 max 16384 2024-08-31T07:36:14.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.183+0000 7f18a99c7740 1 Processor -- start 2024-08-31T07:36:14.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.183+0000 7f18a99c7740 1 -- start start 2024-08-31T07:36:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.184+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 0x55a385797e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.184+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385798350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.184+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a385792c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.184+0000 7f18a99c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b5a30 con 0x55a3853e8070 2024-08-31T07:36:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.184+0000 7f18a99c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b76f0 con 0x55a3853bf1b0 2024-08-31T07:36:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.184+0000 7f18a99c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b6100 con 0x55a3853c7390 2024-08-31T07:36:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.184+0000 7f189b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385798350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.184+0000 7f189b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385798350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59900/0 (socket says 172.21.15.32:59900) 2024-08-31T07:36:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.184+0000 7f189b400640 1 -- 172.21.15.32:0/1782720107 learned_addr learned my addr 172.21.15.32:0/1782720107 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.184+0000 7f189be00640 1 --2- 172.21.15.32:0/1782720107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 0x55a385797e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.185+0000 7f18a0e00640 1 --2- 172.21.15.32:0/1782720107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a385792c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.185+0000 7f189be00640 1 -- 172.21.15.32:0/1782720107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 msgr2=0x55a385798350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.185+0000 7f189be00640 1 --2- 172.21.15.32:0/1782720107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385798350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.185+0000 7f189be00640 1 -- 172.21.15.32:0/1782720107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 msgr2=0x55a385792c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.185+0000 7f189be00640 1 --2- 172.21.15.32:0/1782720107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a385792c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.185+0000 7f189be00640 1 -- 172.21.15.32:0/1782720107 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38579a850 con 0x55a3853bf1b0 2024-08-31T07:36:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.185+0000 7f189b400640 1 --2- 172.21.15.32:0/1782720107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385798350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:36:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.185+0000 7f189be00640 1 --2- 172.21.15.32:0/1782720107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 0x55a385797e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f18840e3000 tx=0x7f188400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490c74ab7d4ecd9a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:14.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.186+0000 7f1890c00640 1 -- 172.21.15.32:0/1782720107 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840b0810 con 0x55a3853bf1b0 2024-08-31T07:36:14.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.186+0000 7f1890c00640 1 -- 172.21.15.32:0/1782720107 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a385706700 con 0x55a3853bf1b0 2024-08-31T07:36:14.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.186+0000 7f1890c00640 1 -- 172.21.15.32:0/1782720107 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840b0420 con 0x55a3853bf1b0 2024-08-31T07:36:14.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.187+0000 7f18a99c7740 1 -- 172.21.15.32:0/1782720107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 msgr2=0x55a385797e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:14.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.187+0000 7f18a99c7740 1 --2- 172.21.15.32:0/1782720107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 0x55a385797e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f18840e3000 tx=0x7f188400ef90 comp rx=0 tx=0).stop 2024-08-31T07:36:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.188+0000 7f18a99c7740 1 -- 172.21.15.32:0/1782720107 shutdown_connections 2024-08-31T07:36:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.188+0000 7f18a99c7740 1 --2- 172.21.15.32:0/1782720107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a385792c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.188+0000 7f18a99c7740 1 --2- 172.21.15.32:0/1782720107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853c7390 0x55a385798350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.188+0000 7f18a99c7740 1 --2- 172.21.15.32:0/1782720107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853bf1b0 0x55a385797e10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.188+0000 7f18a99c7740 1 -- 172.21.15.32:0/1782720107 >> 172.21.15.32:0/1782720107 conn(0x55a3853dc460 msgr2=0x55a385799770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:14.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.189+0000 7f18a99c7740 1 -- 172.21.15.32:0/1782720107 shutdown_connections 2024-08-31T07:36:14.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.189+0000 7f18a99c7740 1 -- 172.21.15.32:0/1782720107 wait complete. 2024-08-31T07:36:14.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.189+0000 7f18a99c7740 1 -- start start 2024-08-31T07:36:14.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.189+0000 7f18a99c7740 1 Processor -- start 2024-08-31T07:36:14.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.190+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853bf1b0 0x55a3857aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.190+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853c7390 0x55a3857a5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.190+0000 7f18a99c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a3857a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.190+0000 7f18a99c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b6f90 con 0x55a3853e8070 2024-08-31T07:36:14.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.190+0000 7f18a99c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b6800 con 0x55a3853c7390 2024-08-31T07:36:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.190+0000 7f18a99c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b39f0 con 0x55a3853bf1b0 2024-08-31T07:36:14.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.190+0000 7f18a99c7740 10 client.0 _ll_register_callbacks cb 0x7f188c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.191+0000 7f189be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853bf1b0 0x55a3857aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.191+0000 7f189be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853bf1b0 0x55a3857aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:59904/0 (socket says 172.21.15.32:59904) 2024-08-31T07:36:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.191+0000 7f189be00640 1 -- 172.21.15.32:0/2742864663 learned_addr learned my addr 172.21.15.32:0/2742864663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.191+0000 7f189b400640 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853c7390 0x55a3857a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.191+0000 7f18a0e00640 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a3857a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.191+0000 7f189b400640 1 -- 172.21.15.32:0/2742864663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853bf1b0 msgr2=0x55a3857aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.191+0000 7f189b400640 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853bf1b0 0x55a3857aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.191+0000 7f189b400640 1 -- 172.21.15.32:0/2742864663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 msgr2=0x55a3857a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.191+0000 7f189b400640 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a3857a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.191+0000 7f189b400640 1 -- 172.21.15.32:0/2742864663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3857a62f0 con 0x55a3853c7390 2024-08-31T07:36:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.192+0000 7f189b400640 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853c7390 0x55a3857a5870 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55a385694c10 tx=0x7f188c0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffeae2512bb8f8b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.192+0000 7f188a000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:36:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.192+0000 7f188a000640 1 -- 172.21.15.32:0/2742864663 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c004450 con 0x55a3853c7390 2024-08-31T07:36:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.192+0000 7f188a000640 1 -- 172.21.15.32:0/2742864663 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8485+0+0 (secure 0 0 0) 0x7f188c031c00 con 0x55a3853c7390 2024-08-31T07:36:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.193+0000 7f18a99c7740 1 -- 172.21.15.32:0/2742864663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a385706700 con 0x55a3853c7390 2024-08-31T07:36:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.193+0000 7f188a000640 1 -- 172.21.15.32:0/2742864663 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f188c031e00 con 0x55a3853c7390 2024-08-31T07:36:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.193+0000 7f18a99c7740 10 client.29954 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.193+0000 7f18a99c7740 1 -- 172.21.15.32:0/2742864663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55a38581e840 con 0x55a3853c7390 2024-08-31T07:36:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.194+0000 7f188a000640 1 -- 172.21.15.32:0/2742864663 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a385706700 con 0x55a3853c7390 2024-08-31T07:36:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.194+0000 7f188a000640 1 -- 172.21.15.32:0/2742864663 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7f188c004900 con 0x55a3853c7390 2024-08-31T07:36:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.194+0000 7f18a99c7740 10 client.29954 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:14.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.194+0000 7f18a99c7740 10 client.29954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:36:14.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.194+0000 7f18a99c7740 20 client.29954 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:14.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.194+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a38537bda0 0x55a3858210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:14.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.194+0000 7f18a99c7740 4 client.29954 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:36:14.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.194+0000 7f18a99c7740 1 -- 172.21.15.32:0/2742864663 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3857021a0 con 0x55a38537bda0 2024-08-31T07:36:14.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.195+0000 7f189be00640 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a38537bda0 0x55a3858210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:14.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.195+0000 7f189be00640 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a38537bda0 0x55a3858210c0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7326897f8181bbb7 server_cookie=77b12ba2428e117 in_seq=0 out_seq=0 2024-08-31T07:36:14.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.196+0000 7f188a000640 10 client.29954 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.196+0000 7f188a000640 1 -- 172.21.15.32:0/2742864663 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3857021a0 con 0x55a38537bda0 2024-08-31T07:36:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.196+0000 7f188a000640 10 client.29954 handle_command_reply: tid=0 2024-08-31T07:36:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.196+0000 7f18a99c7740 1 client.29954 shutdown 2024-08-31T07:36:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.196+0000 7f18a99c7740 1 -- 172.21.15.32:0/2742864663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853c7390 msgr2=0x55a3857a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.196+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853c7390 0x55a3857a5870 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55a385694c10 tx=0x7f188c0045f0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.197+0000 7f18a99c7740 1 -- 172.21.15.32:0/2742864663 shutdown_connections 2024-08-31T07:36:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.197+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55a38537bda0 0x55a3858210c0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.197+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a3853e8070 0x55a3857a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.197+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3853c7390 0x55a3857a5870 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.197+0000 7f18a99c7740 1 --2- 172.21.15.32:0/2742864663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a3853bf1b0 0x55a3857aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.197+0000 7f188a000640 0 client.29954 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.197+0000 7f18a99c7740 1 -- 172.21.15.32:0/2742864663 >> 172.21.15.32:0/2742864663 conn(0x55a3853dc460 msgr2=0x55a385792c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.197+0000 7f18a99c7740 1 -- 172.21.15.32:0/2742864663 shutdown_connections 2024-08-31T07:36:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.197+0000 7f18a99c7740 1 -- 172.21.15.32:0/2742864663 wait complete. 2024-08-31T07:36:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.197+0000 7f18a99c7740 20 client.29954 trim_cache size 0 max 16384 2024-08-31T07:36:14.199 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:36:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.198+0000 7f18a1800640 1 -- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18800b4d20 msgr2=0x7f18800b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.198+0000 7f18a1800640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18800b4d20 0x7f18800b7230 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1894000f80 tx=0x7f1894009930 comp rx=0 tx=0).stop 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.198+0000 7f18a1800640 1 -- 172.21.15.32:0/4199427112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189c10a160 msgr2=0x7f189c1a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.198+0000 7f18a1800640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189c10a160 0x7f189c1a4b70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f188400ea60 tx=0x7f188400ef20 comp rx=0 tx=0).stop 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.198+0000 7f18a1800640 1 -- 172.21.15.32:0/4199427112 shutdown_connections 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.198+0000 7f18a1800640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f18800b4d20 0x7f18800b7230 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.198+0000 7f18a1800640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189c1186a0 0x7f189c1aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.198+0000 7f18a1800640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f189c10aab0 0x7f189c1a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.198+0000 7f18a1800640 1 --2- 172.21.15.32:0/4199427112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f189c10a160 0x7f189c1a4b70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.198+0000 7f18a1800640 1 -- 172.21.15.32:0/4199427112 >> 172.21.15.32:0/4199427112 conn(0x7f189c06ebf0 msgr2=0x7f189c118d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.199+0000 7f18a1800640 1 -- 172.21.15.32:0/4199427112 shutdown_connections 2024-08-31T07:36:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:14.199+0000 7f18a1800640 1 -- 172.21.15.32:0/4199427112 wait complete. 2024-08-31T07:36:14.428 INFO:tasks.workunit.client.0.smithi032.stdout: 10 166965 13.31 MB/sec execute 474 sec latency 2496.354 ms 2024-08-31T07:36:15.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:14 smithi032 ceph-mon[17252]: pgmap v1107: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 307 op/s 2024-08-31T07:36:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:14 smithi073 ceph-mon[21995]: pgmap v1107: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 307 op/s 2024-08-31T07:36:15.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:14 smithi084 ceph-mon[22387]: pgmap v1107: 97 pgs: 97 active+clean; 2.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 307 op/s 2024-08-31T07:36:15.428 INFO:tasks.workunit.client.0.smithi032.stdout: 10 167519 13.33 MB/sec execute 475 sec latency 3496.417 ms 2024-08-31T07:36:16.208 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T07:36:16.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T07:36:16.428 INFO:tasks.workunit.client.0.smithi032.stdout: 10 168071 13.39 MB/sec execute 476 sec latency 4496.469 ms 2024-08-31T07:36:16.691 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:17.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:16 smithi032 ceph-mon[17252]: pgmap v1108: 97 pgs: 97 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.2 MiB/s wr, 376 op/s 2024-08-31T07:36:17.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:16 smithi073 ceph-mon[21995]: pgmap v1108: 97 pgs: 97 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.2 MiB/s wr, 376 op/s 2024-08-31T07:36:17.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:16 smithi084 ceph-mon[22387]: pgmap v1108: 97 pgs: 97 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.2 MiB/s wr, 376 op/s 2024-08-31T07:36:17.428 INFO:tasks.workunit.client.0.smithi032.stdout: 10 168329 13.38 MB/sec execute 477 sec latency 5496.948 ms 2024-08-31T07:36:18.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:18 smithi032 ceph-mon[17252]: osdmap e116: 12 total, 12 up, 12 in 2024-08-31T07:36:18.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:18 smithi032 ceph-mon[17252]: pgmap v1110: 97 pgs: 97 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 9.6 MiB/s wr, 557 op/s 2024-08-31T07:36:18.428 INFO:tasks.workunit.client.0.smithi032.stdout: 10 168740 13.40 MB/sec execute 478 sec latency 6497.002 ms 2024-08-31T07:36:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:18 smithi073 ceph-mon[21995]: osdmap e116: 12 total, 12 up, 12 in 2024-08-31T07:36:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:18 smithi073 ceph-mon[21995]: pgmap v1110: 97 pgs: 97 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 9.6 MiB/s wr, 557 op/s 2024-08-31T07:36:18.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:18 smithi084 ceph-mon[22387]: osdmap e116: 12 total, 12 up, 12 in 2024-08-31T07:36:18.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:18 smithi084 ceph-mon[22387]: pgmap v1110: 97 pgs: 97 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 9.6 MiB/s wr, 557 op/s 2024-08-31T07:36:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.818+0000 7f11a2800640 1 -- 172.21.15.32:0/2106068062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f119c104560 msgr2=0x7f119c104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.818+0000 7f11a2800640 1 --2- 172.21.15.32:0/2106068062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f119c104560 0x7f119c104970 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1190009860 tx=0x7f119002a340 comp rx=0 tx=0).stop 2024-08-31T07:36:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.819+0000 7f11a2800640 1 -- 172.21.15.32:0/2106068062 shutdown_connections 2024-08-31T07:36:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.819+0000 7f11a2800640 1 --2- 172.21.15.32:0/2106068062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119c10d9c0 0x7f119c10feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.819+0000 7f11a2800640 1 --2- 172.21.15.32:0/2106068062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f119c104eb0 0x7f119c10d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.819+0000 7f11a2800640 1 --2- 172.21.15.32:0/2106068062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f119c104560 0x7f119c104970 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.819+0000 7f11a2800640 1 -- 172.21.15.32:0/2106068062 >> 172.21.15.32:0/2106068062 conn(0x7f119c0fe060 msgr2=0x7f119c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.819+0000 7f11a2800640 1 -- 172.21.15.32:0/2106068062 shutdown_connections 2024-08-31T07:36:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.819+0000 7f11a2800640 1 -- 172.21.15.32:0/2106068062 wait complete. 2024-08-31T07:36:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.820+0000 7f11a2800640 1 Processor -- start 2024-08-31T07:36:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.820+0000 7f11a2800640 1 -- start start 2024-08-31T07:36:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.821+0000 7f11a2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119c104560 0x7f119c1022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.821+0000 7f11a2800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f119c104eb0 0x7f119c1027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.821+0000 7f11a2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f119c10d9c0 0x7f119c102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.821+0000 7f11a2800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f119c113630 con 0x7f119c104560 2024-08-31T07:36:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.821+0000 7f11a2800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1134b0 con 0x7f119c10d9c0 2024-08-31T07:36:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.821+0000 7f11a2800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1137b0 con 0x7f119c104eb0 2024-08-31T07:36:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.821+0000 7f11a0a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f119c104eb0 0x7f119c1027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.821+0000 7f11a0a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f119c104eb0 0x7f119c1027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45212/0 (socket says 172.21.15.32:45212) 2024-08-31T07:36:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.821+0000 7f11a0a00640 1 -- 172.21.15.32:0/516514216 learned_addr learned my addr 172.21.15.32:0/516514216 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.821+0000 7f11a1400640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119c104560 0x7f119c1022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.822+0000 7f11a1e00640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f119c10d9c0 0x7f119c102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.822+0000 7f11a0a00640 1 -- 172.21.15.32:0/516514216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f119c10d9c0 msgr2=0x7f119c102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.822+0000 7f11a0a00640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f119c10d9c0 0x7f119c102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.822+0000 7f11a0a00640 1 -- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119c104560 msgr2=0x7f119c1022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.822+0000 7f11a0a00640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119c104560 0x7f119c1022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.822+0000 7f11a0a00640 1 -- 172.21.15.32:0/516514216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c1a8380 con 0x7f119c104eb0 2024-08-31T07:36:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.822+0000 7f11a1400640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119c104560 0x7f119c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:36:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.822+0000 7f11a1e00640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f119c10d9c0 0x7f119c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:36:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.822+0000 7f11a0a00640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f119c104eb0 0x7f119c1027e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f11880058c0 tx=0x7f118800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.823+0000 7f119a000640 1 -- 172.21.15.32:0/516514216 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118800cd30 con 0x7f119c104eb0 2024-08-31T07:36:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.823+0000 7f119a000640 1 -- 172.21.15.32:0/516514216 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11880035f0 con 0x7f119c104eb0 2024-08-31T07:36:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.823+0000 7f119a000640 1 -- 172.21.15.32:0/516514216 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11880044e0 con 0x7f119c104eb0 2024-08-31T07:36:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.823+0000 7f11a2800640 1 -- 172.21.15.32:0/516514216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119c1a8670 con 0x7f119c104eb0 2024-08-31T07:36:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.823+0000 7f11a2800640 1 -- 172.21.15.32:0/516514216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119c1a8b60 con 0x7f119c104eb0 2024-08-31T07:36:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.828+0000 7f11aaa30740 1 Processor -- start 2024-08-31T07:36:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.828+0000 7f119a000640 1 -- 172.21.15.32:0/516514216 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1188003aa0 con 0x7f119c104eb0 2024-08-31T07:36:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.828+0000 7f11aaa30740 1 -- start start 2024-08-31T07:36:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.829+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.829+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.829+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fdf670 0x55c3983a7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.829+0000 7f11aaa30740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c39831c750 con 0x55c397fdf670 2024-08-31T07:36:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.829+0000 7f11aaa30740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c39830bce0 con 0x55c397fd7490 2024-08-31T07:36:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.829+0000 7f11aaa30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c39830fc90 con 0x55c398000350 2024-08-31T07:36:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.830+0000 7f11a1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.830+0000 7f11a1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983a6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55192/0 (socket says 172.21.15.32:55192) 2024-08-31T07:36:18.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.830+0000 7f11a1400640 1 -- 172.21.15.32:0/3617158066 learned_addr learned my addr 172.21.15.32:0/3617158066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.830+0000 7f11a0a00640 1 --2- 172.21.15.32:0/3617158066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fdf670 0x55c3983a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.830+0000 7f11a1e00640 1 --2- 172.21.15.32:0/3617158066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.830+0000 7f11a1400640 1 -- 172.21.15.32:0/3617158066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 msgr2=0x55c3983a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.830+0000 7f11a1400640 1 --2- 172.21.15.32:0/3617158066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.830+0000 7f11a1400640 1 -- 172.21.15.32:0/3617158066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fdf670 msgr2=0x55c3983a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.830+0000 7f11a1400640 1 --2- 172.21.15.32:0/3617158066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fdf670 0x55c3983a7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.831+0000 7f11a1400640 1 -- 172.21.15.32:0/3617158066 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39806b930 con 0x55c397fd7490 2024-08-31T07:36:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.831+0000 7f11a1400640 1 --2- 172.21.15.32:0/3617158066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983a6d10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f11900303f0 tx=0x7f1190030b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c29eb631f91f9893 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.831+0000 7f119a000640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f11840b4ca0 0x7f11840b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.832+0000 7f119a000640 1 -- 172.21.15.32:0/516514216 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7f11880e5910 con 0x7f119c104eb0 2024-08-31T07:36:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.832+0000 7f118e600640 1 -- 172.21.15.32:0/3617158066 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190040020 con 0x55c397fd7490 2024-08-31T07:36:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.833+0000 7f118e600640 1 -- 172.21.15.32:0/3617158066 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1190038040 con 0x55c397fd7490 2024-08-31T07:36:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.833+0000 7f118e600640 1 -- 172.21.15.32:0/3617158066 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190037070 con 0x55c397fd7490 2024-08-31T07:36:18.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.833+0000 7f11a1e00640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f11840b4ca0 0x7f11840b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.835+0000 7f11aaa30740 1 -- 172.21.15.32:0/3617158066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 msgr2=0x55c3983a6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:18.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.835+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3617158066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983a6d10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f11900303f0 tx=0x7f1190030b50 comp rx=0 tx=0).stop 2024-08-31T07:36:18.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.835+0000 7f11a1e00640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f11840b4ca0 0x7f11840b71b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f11940099a0 tx=0x7f1194009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.835+0000 7f11aaa30740 1 -- 172.21.15.32:0/3617158066 shutdown_connections 2024-08-31T07:36:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.836+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3617158066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fdf670 0x55c3983a7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.836+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3617158066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983a6d10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.836+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3617158066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.836+0000 7f11aaa30740 1 -- 172.21.15.32:0/3617158066 >> 172.21.15.32:0/3617158066 conn(0x55c397ff4740 msgr2=0x55c3983a46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.836+0000 7f11aaa30740 1 -- 172.21.15.32:0/3617158066 shutdown_connections 2024-08-31T07:36:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.836+0000 7f11aaa30740 1 -- 172.21.15.32:0/3617158066 wait complete. 2024-08-31T07:36:18.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.837+0000 7f11aaa30740 1 -- start start 2024-08-31T07:36:18.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.837+0000 7f11aaa30740 1 Processor -- start 2024-08-31T07:36:18.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.838+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 0x55c3983b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.838+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 0x55c3983b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.838+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983be270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.838+0000 7f11aaa30740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c39816e890 con 0x55c397fd7490 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.838+0000 7f11aaa30740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c398292cd0 con 0x55c397fdf670 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.838+0000 7f11aaa30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c3983b4f50 con 0x55c398000350 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.838+0000 7f11aaa30740 10 client.0 _ll_register_callbacks cb 0x55c398306ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.839+0000 7f11a1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983be270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.839+0000 7f11a1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 0x55c3983b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.839+0000 7f11a1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983be270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45238/0 (socket says 172.21.15.32:45238) 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.839+0000 7f11a1e00640 1 -- 172.21.15.32:0/3237681972 learned_addr learned my addr 172.21.15.32:0/3237681972 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.839+0000 7f11a0a00640 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 0x55c3983b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.840+0000 7f11a1400640 1 -- 172.21.15.32:0/3237681972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 msgr2=0x55c3983be270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.840+0000 7f11a1400640 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983be270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.840+0000 7f11a1400640 1 -- 172.21.15.32:0/3237681972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 msgr2=0x55c3983b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.840+0000 7f11a1400640 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 0x55c3983b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.840+0000 7f11a1400640 1 -- 172.21.15.32:0/3237681972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3982dc1b0 con 0x55c397fd7490 2024-08-31T07:36:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.840+0000 7f11a1400640 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 0x55c3983b22f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f119002a860 tx=0x7f119002aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71054e4262cf50e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.840+0000 7f1182a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:36:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.841+0000 7f1182a00640 1 -- 172.21.15.32:0/3237681972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190049070 con 0x55c397fd7490 2024-08-31T07:36:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.841+0000 7f1182a00640 1 -- 172.21.15.32:0/3237681972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7f1190005050 con 0x55c397fd7490 2024-08-31T07:36:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.841+0000 7f11aaa30740 1 -- 172.21.15.32:0/3237681972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1190038040 con 0x55c397fd7490 2024-08-31T07:36:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.841+0000 7f1182a00640 1 -- 172.21.15.32:0/3237681972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1190055050 con 0x55c397fd7490 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.841+0000 7f11aaa30740 10 client.19893 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.841+0000 7f11aaa30740 1 -- 172.21.15.32:0/3237681972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55c3983c9700 con 0x55c397fd7490 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.842+0000 7f1182a00640 1 -- 172.21.15.32:0/3237681972 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1190038040 con 0x55c397fd7490 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.842+0000 7f1182a00640 1 -- 172.21.15.32:0/3237681972 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7f1190004cb0 con 0x55c397fd7490 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.842+0000 7f11aaa30740 10 client.19893 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.842+0000 7f11aaa30740 10 client.19893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.842+0000 7f11aaa30740 20 client.19893 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.842+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c397fbecd0 0x55c3983cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.842+0000 7f11aaa30740 4 client.19893 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.842+0000 7f11aaa30740 1 -- 172.21.15.32:0/3237681972 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c39831e6a0 con 0x55c397fbecd0 2024-08-31T07:36:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.843+0000 7f11a0a00640 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c397fbecd0 0x55c3983cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.844+0000 7f11a0a00640 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c397fbecd0 0x55c3983cca60 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=458954f4602cb874 server_cookie=2a919dfdd1f112bd in_seq=0 out_seq=0 2024-08-31T07:36:18.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.844+0000 7f1182a00640 10 client.19893 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:18.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f1182a00640 1 -- 172.21.15.32:0/3237681972 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c39831e6a0 con 0x55c397fbecd0 2024-08-31T07:36:18.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f1182a00640 10 client.19893 handle_command_reply: tid=0 2024-08-31T07:36:18.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f11aaa30740 1 client.19893 shutdown 2024-08-31T07:36:18.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f11aaa30740 1 -- 172.21.15.32:0/3237681972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 msgr2=0x55c3983b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:18.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 0x55c3983b22f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f119002a860 tx=0x7f119002aa80 comp rx=0 tx=0).stop 2024-08-31T07:36:18.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f11aaa30740 1 -- 172.21.15.32:0/3237681972 shutdown_connections 2024-08-31T07:36:18.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c397fbecd0 0x55c3983cca60 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 0x55c3983b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3237681972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 0x55c3983b22f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f11aaa30740 1 -- 172.21.15.32:0/3237681972 >> 172.21.15.32:0/3237681972 conn(0x55c397ff4740 msgr2=0x55c3983a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.846+0000 7f1182a00640 0 client.19893 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.847+0000 7f11aaa30740 1 -- 172.21.15.32:0/3237681972 shutdown_connections 2024-08-31T07:36:18.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.847+0000 7f11aaa30740 1 -- 172.21.15.32:0/3237681972 wait complete. 2024-08-31T07:36:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.848+0000 7f11aaa30740 20 client.19893 trim_cache size 0 max 16384 2024-08-31T07:36:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.878+0000 7f11aaa30740 1 Processor -- start 2024-08-31T07:36:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.878+0000 7f11aaa30740 1 -- start start 2024-08-31T07:36:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.878+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.878+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fdf670 0x55c3983be170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.878+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983be6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.878+0000 7f11aaa30740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1ad020 con 0x55c397fdf670 2024-08-31T07:36:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.878+0000 7f11aaa30740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1aece0 con 0x55c397fd7490 2024-08-31T07:36:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.878+0000 7f11aaa30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1ad6f0 con 0x55c398000350 2024-08-31T07:36:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.879+0000 7f11a1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.879+0000 7f11a1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55220/0 (socket says 172.21.15.32:55220) 2024-08-31T07:36:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.879+0000 7f11a1400640 1 -- 172.21.15.32:0/658100667 learned_addr learned my addr 172.21.15.32:0/658100667 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.879+0000 7f11a1e00640 1 --2- 172.21.15.32:0/658100667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983be6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.879+0000 7f11a1400640 1 -- 172.21.15.32:0/658100667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 msgr2=0x55c3983be6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.879+0000 7f11a1400640 1 --2- 172.21.15.32:0/658100667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983be6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.879+0000 7f11a1400640 1 -- 172.21.15.32:0/658100667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fdf670 msgr2=0x55c3983be170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.879+0000 7f11a1400640 1 --2- 172.21.15.32:0/658100667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fdf670 0x55c3983be170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.879+0000 7f11a1400640 1 -- 172.21.15.32:0/658100667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3983b1050 con 0x55c397fd7490 2024-08-31T07:36:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.880+0000 7f11a1400640 1 --2- 172.21.15.32:0/658100667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983b29e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1190000c00 tx=0x7f119002aaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27221fb346166e2e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.880+0000 7f118dc00640 1 -- 172.21.15.32:0/658100667 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190052020 con 0x55c397fd7490 2024-08-31T07:36:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.880+0000 7f118dc00640 1 -- 172.21.15.32:0/658100667 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11900044d0 con 0x55c397fd7490 2024-08-31T07:36:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.880+0000 7f118dc00640 1 -- 172.21.15.32:0/658100667 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190049040 con 0x55c397fd7490 2024-08-31T07:36:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.881+0000 7f11aaa30740 1 -- 172.21.15.32:0/658100667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 msgr2=0x55c3983b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.881+0000 7f11aaa30740 1 --2- 172.21.15.32:0/658100667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983b29e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1190000c00 tx=0x7f119002aaf0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.881+0000 7f11aaa30740 1 -- 172.21.15.32:0/658100667 shutdown_connections 2024-08-31T07:36:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.881+0000 7f11aaa30740 1 --2- 172.21.15.32:0/658100667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983be6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.881+0000 7f11aaa30740 1 --2- 172.21.15.32:0/658100667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fdf670 0x55c3983be170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.881+0000 7f11aaa30740 1 --2- 172.21.15.32:0/658100667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fd7490 0x55c3983b29e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.881+0000 7f11aaa30740 1 -- 172.21.15.32:0/658100667 >> 172.21.15.32:0/658100667 conn(0x55c397ff4740 msgr2=0x55c3983b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.882+0000 7f11aaa30740 1 -- 172.21.15.32:0/658100667 shutdown_connections 2024-08-31T07:36:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.882+0000 7f11aaa30740 1 -- 172.21.15.32:0/658100667 wait complete. 2024-08-31T07:36:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.882+0000 7f11aaa30740 1 -- start start 2024-08-31T07:36:18.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.883+0000 7f11aaa30740 1 Processor -- start 2024-08-31T07:36:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.883+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 0x55c3983ba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.883+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 0x55c3983b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.883+0000 7f11aaa30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983b7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.883+0000 7f11aaa30740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1ae580 con 0x55c397fd7490 2024-08-31T07:36:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.883+0000 7f11aaa30740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1addf0 con 0x55c397fdf670 2024-08-31T07:36:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.883+0000 7f11aaa30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1ab080 con 0x55c398000350 2024-08-31T07:36:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.883+0000 7f11aaa30740 10 client.0 _ll_register_callbacks cb 0x7f11940093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a1e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983b7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45260/0 (socket says 172.21.15.32:45260) 2024-08-31T07:36:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a1e00640 1 -- 172.21.15.32:0/3800393112 learned_addr learned my addr 172.21.15.32:0/3800393112 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a1400640 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 0x55c3983ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a0a00640 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 0x55c3983b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a1e00640 1 -- 172.21.15.32:0/3800393112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 msgr2=0x55c3983b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:18.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a1e00640 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 0x55c3983b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a1e00640 1 -- 172.21.15.32:0/3800393112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 msgr2=0x55c3983ba930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a1e00640 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 0x55c3983ba930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a1e00640 1 -- 172.21.15.32:0/3800393112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3983ccdd0 con 0x55c398000350 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a1400640 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 0x55c3983ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.884+0000 7f11a0a00640 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 0x55c3983b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.885+0000 7f11a1e00640 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983b7330 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1194009970 tx=0x7f1194004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d812d55619c0dc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.885+0000 7f1182a00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.885+0000 7f1182a00640 1 -- 172.21.15.32:0/3800393112 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119402e060 con 0x55c398000350 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.885+0000 7f1182a00640 1 -- 172.21.15.32:0/3800393112 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7f1194029680 con 0x55c398000350 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.885+0000 7f11aaa30740 1 -- 172.21.15.32:0/3800393112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11900044d0 con 0x55c398000350 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.885+0000 7f1182a00640 1 -- 172.21.15.32:0/3800393112 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1194004b20 con 0x55c398000350 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.885+0000 7f11aaa30740 10 client.29968 fetch_fsmap learned FSMap version 90 2024-08-31T07:36:18.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.885+0000 7f11aaa30740 1 -- 172.21.15.32:0/3800393112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55c398437fc0 con 0x55c398000350 2024-08-31T07:36:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.886+0000 7f1182a00640 1 -- 172.21.15.32:0/3800393112 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11900044d0 con 0x55c398000350 2024-08-31T07:36:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.886+0000 7f1182a00640 1 -- 172.21.15.32:0/3800393112 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 90) ==== 8394+0+0 (secure 0 0 0) 0x7f119401cce0 con 0x55c398000350 2024-08-31T07:36:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.886+0000 7f11aaa30740 10 client.29968 fetch_fsmap finished waiting for FSMap version 90 2024-08-31T07:36:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.886+0000 7f11aaa30740 10 client.29968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:36:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.886+0000 7f11aaa30740 20 client.29968 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.886+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c397f94080 0x55c39843a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:18.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.886+0000 7f11aaa30740 4 client.29968 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T07:36:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.886+0000 7f11aaa30740 1 -- 172.21.15.32:0/3800393112 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c39831a140 con 0x55c397f94080 2024-08-31T07:36:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.887+0000 7f11a1400640 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c397f94080 0x55c39843a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.888+0000 7f11a1400640 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c397f94080 0x55c39843a7f0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60778ef990baabd1 server_cookie=6cadb6e1cece3b0d in_seq=0 out_seq=0 2024-08-31T07:36:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.888+0000 7f1182a00640 10 client.29968 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:18.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.891+0000 7f1182a00640 1 -- 172.21.15.32:0/3800393112 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c39831a140 con 0x55c397f94080 2024-08-31T07:36:18.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.891+0000 7f1182a00640 10 client.29968 handle_command_reply: tid=0 2024-08-31T07:36:18.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.891+0000 7f11aaa30740 1 client.29968 shutdown 2024-08-31T07:36:18.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.891+0000 7f11aaa30740 1 -- 172.21.15.32:0/3800393112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 msgr2=0x55c3983b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:18.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.891+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983b7330 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1194009970 tx=0x7f1194004a40 comp rx=0 tx=0).stop 2024-08-31T07:36:18.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.892+0000 7f11aaa30740 1 -- 172.21.15.32:0/3800393112 shutdown_connections 2024-08-31T07:36:18.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.892+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55c397f94080 0x55c39843a7f0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.892+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c398000350 0x55c3983b7330 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.892+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c397fdf670 0x55c3983b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.892+0000 7f11aaa30740 1 --2- 172.21.15.32:0/3800393112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c397fd7490 0x55c3983ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.892+0000 7f11aaa30740 1 -- 172.21.15.32:0/3800393112 >> 172.21.15.32:0/3800393112 conn(0x55c397ff4740 msgr2=0x55c3983a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:18.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.892+0000 7f1182a00640 0 client.29968 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:18.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.892+0000 7f11aaa30740 1 -- 172.21.15.32:0/3800393112 shutdown_connections 2024-08-31T07:36:18.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.892+0000 7f11aaa30740 1 -- 172.21.15.32:0/3800393112 wait complete. 2024-08-31T07:36:18.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.893+0000 7f11aaa30740 20 client.29968 trim_cache size 0 max 16384 2024-08-31T07:36:18.895 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:36:18.895 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-08-31T07:36:18.895 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "7d087aa2-18ca-438a-86b9-b121fe13350b", 2024-08-31T07:36:18.895 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-08-31T07:36:18.895 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:36:18.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.894+0000 7f11a2800640 1 -- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f11840b4ca0 msgr2=0x7f11840b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:18.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.894+0000 7f11a2800640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f11840b4ca0 0x7f11840b71b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f11940099a0 tx=0x7f1194009500 comp rx=0 tx=0).stop 2024-08-31T07:36:18.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.894+0000 7f11a2800640 1 -- 172.21.15.32:0/516514216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f119c104eb0 msgr2=0x7f119c1027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:18.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.894+0000 7f11a2800640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f119c104eb0 0x7f119c1027e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f11880058c0 tx=0x7f118800ee50 comp rx=0 tx=0).stop 2024-08-31T07:36:18.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.894+0000 7f11a2800640 1 -- 172.21.15.32:0/516514216 shutdown_connections 2024-08-31T07:36:18.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.894+0000 7f11a2800640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f11840b4ca0 0x7f11840b71b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.894+0000 7f11a2800640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f119c10d9c0 0x7f119c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.894+0000 7f11a2800640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f119c104eb0 0x7f119c1027e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.894+0000 7f11a2800640 1 --2- 172.21.15.32:0/516514216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119c104560 0x7f119c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:18.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.894+0000 7f11a2800640 1 -- 172.21.15.32:0/516514216 >> 172.21.15.32:0/516514216 conn(0x7f119c0fe060 msgr2=0x7f119c107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:18.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.895+0000 7f11a2800640 1 -- 172.21.15.32:0/516514216 shutdown_connections 2024-08-31T07:36:18.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:18.895+0000 7f11a2800640 1 -- 172.21.15.32:0/516514216 wait complete. 2024-08-31T07:36:19.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 168902 13.38 MB/sec execute 479 sec latency 6760.124 ms 2024-08-31T07:36:19.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:19 smithi032 ceph-mon[17252]: scrub queued for path: / 2024-08-31T07:36:19.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:19 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:19.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:19 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:36:19.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:19 smithi073 ceph-mon[21995]: scrub queued for path: / 2024-08-31T07:36:19.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:19 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:19.681 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:19 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:36:19.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:19 smithi084 ceph-mon[22387]: scrub queued for path: / 2024-08-31T07:36:19.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:19 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:19.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:19 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:36:20.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:36:20.310 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:20 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:36:20.310 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:20 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:36:20.310 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:20 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:36:20.310 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:20 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:36:20.311 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:20 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:20.311 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:20 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:20.311 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:20 smithi084 ceph-mon[22387]: pgmap v1111: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 9.7 MiB/s wr, 552 op/s 2024-08-31T07:36:20.428 INFO:tasks.workunit.client.0.smithi032.stdout: 10 169332 13.38 MB/sec execute 480 sec latency 7607.975 ms 2024-08-31T07:36:20.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:20 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:36:20.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:20 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:36:20.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:20 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:36:20.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:20 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:36:20.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:20 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:20.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:20 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:20.609 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:20 smithi032 ceph-mon[17252]: pgmap v1111: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 9.7 MiB/s wr, 552 op/s 2024-08-31T07:36:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:20 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:36:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:20 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:36:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:20 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:36:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:20 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:36:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:20 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:20 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:20.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:20 smithi073 ceph-mon[21995]: pgmap v1111: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 9.7 MiB/s wr, 552 op/s 2024-08-31T07:36:20.771 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:21.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 169998 13.46 MB/sec execute 481 sec latency 578.666 ms 2024-08-31T07:36:21.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:21 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:21.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:21 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:36:21.482 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:21 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:21.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:21 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:21.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:21 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:36:21.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:21 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.654+0000 7f7344400640 1 -- 172.21.15.32:0/3626061572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f733c109250 msgr2=0x7f733c109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.654+0000 7f7344400640 1 --2- 172.21.15.32:0/3626061572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f733c109250 0x7f733c109660 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7328009900 tx=0x7f732802a410 comp rx=0 tx=0).stop 2024-08-31T07:36:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.654+0000 7f7344400640 1 -- 172.21.15.32:0/3626061572 shutdown_connections 2024-08-31T07:36:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.654+0000 7f7344400640 1 --2- 172.21.15.32:0/3626061572 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c109ba0 0x7f733c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.654+0000 7f7344400640 1 --2- 172.21.15.32:0/3626061572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f733c109250 0x7f733c109660 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.654+0000 7f7344400640 1 --2- 172.21.15.32:0/3626061572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f733c103910 0x7f733c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.654+0000 7f7344400640 1 -- 172.21.15.32:0/3626061572 >> 172.21.15.32:0/3626061572 conn(0x7f733c0ff380 msgr2=0x7f733c1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.655+0000 7f7344400640 1 -- 172.21.15.32:0/3626061572 shutdown_connections 2024-08-31T07:36:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.655+0000 7f7344400640 1 -- 172.21.15.32:0/3626061572 wait complete. 2024-08-31T07:36:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.656+0000 7f7344400640 1 Processor -- start 2024-08-31T07:36:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.656+0000 7f7344400640 1 -- start start 2024-08-31T07:36:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.657+0000 7f7344400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c103910 0x7f733c074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.657+0000 7f7344400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f733c109250 0x7f733c06ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.657+0000 7f7344400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f733c109ba0 0x7f733c06e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.657+0000 7f7344400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f733c1138b0 con 0x7f733c109250 2024-08-31T07:36:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.657+0000 7f7344400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f733c113730 con 0x7f733c109ba0 2024-08-31T07:36:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.657+0000 7f7344400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f733c113a30 con 0x7f733c103910 2024-08-31T07:36:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.657+0000 7f7342400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f733c109ba0 0x7f733c06e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.657+0000 7f7342400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f733c109ba0 0x7f733c06e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55258/0 (socket says 172.21.15.32:55258) 2024-08-31T07:36:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.657+0000 7f7342400640 1 -- 172.21.15.32:0/211377097 learned_addr learned my addr 172.21.15.32:0/211377097 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.657+0000 7f7341a00640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c103910 0x7f733c074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.658+0000 7f7341000640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f733c109250 0x7f733c06ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.659+0000 7f7342400640 1 -- 172.21.15.32:0/211377097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c103910 msgr2=0x7f733c074c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.659+0000 7f7342400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c103910 0x7f733c074c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.659+0000 7f7342400640 1 -- 172.21.15.32:0/211377097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f733c109250 msgr2=0x7f733c06ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.659+0000 7f7342400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f733c109250 0x7f733c06ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.659+0000 7f7342400640 1 -- 172.21.15.32:0/211377097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c06ec70 con 0x7f733c109ba0 2024-08-31T07:36:21.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.660+0000 7f7342400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f733c109ba0 0x7f733c06e450 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7334009fd0 tx=0x7f733400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:21.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.660+0000 7f733a000640 1 -- 172.21.15.32:0/211377097 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334018070 con 0x7f733c109ba0 2024-08-31T07:36:21.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.660+0000 7f733a000640 1 -- 172.21.15.32:0/211377097 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7334003e20 con 0x7f733c109ba0 2024-08-31T07:36:21.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.660+0000 7f733a000640 1 -- 172.21.15.32:0/211377097 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334004280 con 0x7f733c109ba0 2024-08-31T07:36:21.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.660+0000 7f7344400640 1 -- 172.21.15.32:0/211377097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733c073930 con 0x7f733c109ba0 2024-08-31T07:36:21.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.661+0000 7f7344400640 1 -- 172.21.15.32:0/211377097 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733c073df0 con 0x7f733c109ba0 2024-08-31T07:36:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.665+0000 7f733a000640 1 -- 172.21.15.32:0/211377097 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7334012880 con 0x7f733c109ba0 2024-08-31T07:36:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.666+0000 7f734b21f740 1 Processor -- start 2024-08-31T07:36:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.666+0000 7f734b21f740 1 -- start start 2024-08-31T07:36:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.667+0000 7f734b21f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e54070 0x55b94a1f9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.667+0000 7f734b21f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a1f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.667+0000 7f734b21f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a1fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.667+0000 7f734b21f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b94a170770 con 0x55b949e54070 2024-08-31T07:36:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.667+0000 7f734b21f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b94a15fd00 con 0x55b949e33390 2024-08-31T07:36:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.667+0000 7f734b21f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b94a163cb0 con 0x55b949e2b1b0 2024-08-31T07:36:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.667+0000 7f7341a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a1f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.667+0000 7f7341a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a1f9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45280/0 (socket says 172.21.15.32:45280) 2024-08-31T07:36:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.667+0000 7f7341a00640 1 -- 172.21.15.32:0/3954120989 learned_addr learned my addr 172.21.15.32:0/3954120989 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.668+0000 7f733a000640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f73240b4df0 0x7f73240b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.668+0000 7f7341000640 1 --2- 172.21.15.32:0/3954120989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a1fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.668+0000 7f7341a00640 1 -- 172.21.15.32:0/3954120989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 msgr2=0x55b94a1fa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.668+0000 7f7341a00640 1 --2- 172.21.15.32:0/3954120989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a1fa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.668+0000 7f7341a00640 1 -- 172.21.15.32:0/3954120989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e54070 msgr2=0x55b94a1f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.668+0000 7f7341a00640 1 --2- 172.21.15.32:0/3954120989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e54070 0x55b94a1f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.668+0000 7f7341a00640 1 -- 172.21.15.32:0/3954120989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b949ebf650 con 0x55b949e2b1b0 2024-08-31T07:36:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.668+0000 7f7342400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f73240b4df0 0x7f73240b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.669+0000 7f733a000640 1 -- 172.21.15.32:0/211377097 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7f73340e93d0 con 0x7f733c109ba0 2024-08-31T07:36:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.669+0000 7f7341a00640 1 --2- 172.21.15.32:0/3954120989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a1f9e90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7330009c80 tx=0x7f73300117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c835183f0ac49d10 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.670+0000 7f732ea00640 1 -- 172.21.15.32:0/3954120989 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330012450 con 0x55b949e2b1b0 2024-08-31T07:36:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.670+0000 7f732ea00640 1 -- 172.21.15.32:0/3954120989 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7330012aa0 con 0x55b949e2b1b0 2024-08-31T07:36:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.670+0000 7f732ea00640 1 -- 172.21.15.32:0/3954120989 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733000fb80 con 0x55b949e2b1b0 2024-08-31T07:36:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.671+0000 7f734b21f740 1 -- 172.21.15.32:0/3954120989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 msgr2=0x55b94a1f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.671+0000 7f734b21f740 1 --2- 172.21.15.32:0/3954120989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a1f9e90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7330009c80 tx=0x7f73300117b0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.672+0000 7f734b21f740 1 -- 172.21.15.32:0/3954120989 shutdown_connections 2024-08-31T07:36:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.672+0000 7f734b21f740 1 --2- 172.21.15.32:0/3954120989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a1fa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.672+0000 7f734b21f740 1 --2- 172.21.15.32:0/3954120989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a1f9e90 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.672+0000 7f734b21f740 1 --2- 172.21.15.32:0/3954120989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e54070 0x55b94a1f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.672+0000 7f734b21f740 1 -- 172.21.15.32:0/3954120989 >> 172.21.15.32:0/3954120989 conn(0x55b949e48460 msgr2=0x55b94a1f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.674+0000 7f734b21f740 1 -- 172.21.15.32:0/3954120989 shutdown_connections 2024-08-31T07:36:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.674+0000 7f734b21f740 1 -- 172.21.15.32:0/3954120989 wait complete. 2024-08-31T07:36:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.675+0000 7f7342400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f73240b4df0 0x7f73240b7300 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f733400b760 tx=0x7f7334012c20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.675+0000 7f734b21f740 1 -- start start 2024-08-31T07:36:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.676+0000 7f734b21f740 1 Processor -- start 2024-08-31T07:36:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.676+0000 7f734b21f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e2b1b0 0x55b94a205580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.676+0000 7f734b21f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a205b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.676+0000 7f734b21f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e54070 0x55b94a219c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.676+0000 7f734b21f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b949fc2700 con 0x55b949e2b1b0 2024-08-31T07:36:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.676+0000 7f734b21f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b94a0e6cb0 con 0x55b949e33390 2024-08-31T07:36:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.676+0000 7f734b21f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b94a210940 con 0x55b949e54070 2024-08-31T07:36:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.676+0000 7f734b21f740 10 client.0 _ll_register_callbacks cb 0x55b94a10c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.677+0000 7f7341a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a205b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.677+0000 7f7341a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a205b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55278/0 (socket says 172.21.15.32:55278) 2024-08-31T07:36:21.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:21 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:21.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:21 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:36:21.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:21 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:21.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:21 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:21.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:21 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:36:21.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:21 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.677+0000 7f7341a00640 1 -- 172.21.15.32:0/1197025869 learned_addr learned my addr 172.21.15.32:0/1197025869 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.678+0000 7f7341a00640 1 -- 172.21.15.32:0/1197025869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e54070 msgr2=0x55b94a219c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:36:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.678+0000 7f7341a00640 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e54070 0x55b94a219c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.678+0000 7f7341a00640 1 -- 172.21.15.32:0/1197025869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e2b1b0 msgr2=0x55b94a205580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.678+0000 7f7341a00640 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e2b1b0 0x55b94a205580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.678+0000 7f7341a00640 1 -- 172.21.15.32:0/1197025869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b94a127870 con 0x55b949e33390 2024-08-31T07:36:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.678+0000 7f7341a00640 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a205b20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7330000c00 tx=0x7f7330009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f0b43c25cefd7df server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.678+0000 7f7323400640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:36:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.679+0000 7f7323400640 1 -- 172.21.15.32:0/1197025869 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330029020 con 0x55b949e33390 2024-08-31T07:36:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.679+0000 7f734b21f740 1 -- 172.21.15.32:0/1197025869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7330012aa0 con 0x55b949e33390 2024-08-31T07:36:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.679+0000 7f7323400640 1 -- 172.21.15.32:0/1197025869 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7f7330012660 con 0x55b949e33390 2024-08-31T07:36:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.679+0000 7f7323400640 1 -- 172.21.15.32:0/1197025869 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f7330034050 con 0x55b949e33390 2024-08-31T07:36:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.680+0000 7f734b21f740 10 client.29981 fetch_fsmap learned FSMap version 91 2024-08-31T07:36:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.680+0000 7f734b21f740 1 -- 172.21.15.32:0/1197025869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) -- 0x55b94a21c9d0 con 0x55b949e33390 2024-08-31T07:36:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.680+0000 7f7323400640 1 -- 172.21.15.32:0/1197025869 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7330012aa0 con 0x55b949e33390 2024-08-31T07:36:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.680+0000 7f7323400640 1 -- 172.21.15.32:0/1197025869 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 91) ==== 8390+0+0 (secure 0 0 0) 0x7f73300035f0 con 0x55b949e33390 2024-08-31T07:36:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.681+0000 7f734b21f740 10 client.29981 fetch_fsmap finished waiting for FSMap version 91 2024-08-31T07:36:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.681+0000 7f734b21f740 10 client.29981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:36:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.681+0000 7f734b21f740 20 client.29981 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.681+0000 7f734b21f740 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b949e129f0 0x55b94a21fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.681+0000 7f734b21f740 4 client.29981 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:36:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.681+0000 7f734b21f740 1 -- 172.21.15.32:0/1197025869 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b94a1726c0 con 0x55b949e129f0 2024-08-31T07:36:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.681+0000 7f7341000640 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b949e129f0 0x55b94a21fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.682+0000 7f7341000640 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b949e129f0 0x55b94a21fdf0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9baac82863cd006d server_cookie=959749f5289624ce in_seq=0 out_seq=0 2024-08-31T07:36:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.682+0000 7f7323400640 10 client.29981 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f7323400640 1 -- 172.21.15.32:0/1197025869 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b94a1726c0 con 0x55b949e129f0 2024-08-31T07:36:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f7323400640 10 client.29981 handle_command_reply: tid=0 2024-08-31T07:36:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f734b21f740 1 client.29981 shutdown 2024-08-31T07:36:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f734b21f740 1 -- 172.21.15.32:0/1197025869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 msgr2=0x55b94a205b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f734b21f740 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a205b20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7330000c00 tx=0x7f7330009fa0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f734b21f740 1 -- 172.21.15.32:0/1197025869 shutdown_connections 2024-08-31T07:36:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f734b21f740 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b949e129f0 0x55b94a21fdf0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f734b21f740 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e54070 0x55b94a219c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f734b21f740 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a205b20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f734b21f740 1 --2- 172.21.15.32:0/1197025869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e2b1b0 0x55b94a205580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f7323400640 0 client.29981 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.683+0000 7f734b21f740 1 -- 172.21.15.32:0/1197025869 >> 172.21.15.32:0/1197025869 conn(0x55b949e48460 msgr2=0x55b94a1fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.684+0000 7f734b21f740 1 -- 172.21.15.32:0/1197025869 shutdown_connections 2024-08-31T07:36:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.684+0000 7f734b21f740 1 -- 172.21.15.32:0/1197025869 wait complete. 2024-08-31T07:36:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.684+0000 7f734b21f740 20 client.29981 trim_cache size 0 max 16384 2024-08-31T07:36:21.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.710+0000 7f734b21f740 1 Processor -- start 2024-08-31T07:36:21.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.710+0000 7f734b21f740 1 -- start start 2024-08-31T07:36:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.710+0000 7f734b21f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a205c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.710+0000 7f734b21f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a1fea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.710+0000 7f734b21f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e54070 0x55b94a1fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.710+0000 7f734b21f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f733c1b59e0 con 0x55b949e54070 2024-08-31T07:36:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.710+0000 7f734b21f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f733c1b7670 con 0x55b949e33390 2024-08-31T07:36:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.710+0000 7f734b21f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f733c1b60b0 con 0x55b949e2b1b0 2024-08-31T07:36:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a1fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a1fea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55282/0 (socket says 172.21.15.32:55282) 2024-08-31T07:36:21.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341a00640 1 -- 172.21.15.32:0/235957121 learned_addr learned my addr 172.21.15.32:0/235957121 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:21.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341000640 1 --2- 172.21.15.32:0/235957121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a205c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341000640 1 --2- 172.21.15.32:0/235957121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e54070 0x55b94a1fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341a00640 1 -- 172.21.15.32:0/235957121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 msgr2=0x55b94a205c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341a00640 1 --2- 172.21.15.32:0/235957121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a205c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341a00640 1 -- 172.21.15.32:0/235957121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e54070 msgr2=0x55b94a1fefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341a00640 1 --2- 172.21.15.32:0/235957121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e54070 0x55b94a1fefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341a00640 1 -- 172.21.15.32:0/235957121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94a2042e0 con 0x55b949e33390 2024-08-31T07:36:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.711+0000 7f7341000640 1 --2- 172.21.15.32:0/235957121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e54070 0x55b94a1fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:36:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.712+0000 7f7341a00640 1 --2- 172.21.15.32:0/235957121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a1fea60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f73300039c0 tx=0x7f7330009ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb2a6eca30f927bf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.712+0000 7f732e000640 1 -- 172.21.15.32:0/235957121 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733003d070 con 0x55b949e33390 2024-08-31T07:36:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.712+0000 7f732e000640 1 -- 172.21.15.32:0/235957121 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7330009890 con 0x55b949e33390 2024-08-31T07:36:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.712+0000 7f732e000640 1 -- 172.21.15.32:0/235957121 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330029040 con 0x55b949e33390 2024-08-31T07:36:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.713+0000 7f734b21f740 1 -- 172.21.15.32:0/235957121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 msgr2=0x55b94a1fea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.713+0000 7f734b21f740 1 --2- 172.21.15.32:0/235957121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a1fea60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f73300039c0 tx=0x7f7330009ad0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.713+0000 7f734b21f740 1 -- 172.21.15.32:0/235957121 shutdown_connections 2024-08-31T07:36:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.713+0000 7f734b21f740 1 --2- 172.21.15.32:0/235957121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e54070 0x55b94a1fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.713+0000 7f734b21f740 1 --2- 172.21.15.32:0/235957121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e33390 0x55b94a1fea60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.713+0000 7f734b21f740 1 --2- 172.21.15.32:0/235957121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a205c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.713+0000 7f734b21f740 1 -- 172.21.15.32:0/235957121 >> 172.21.15.32:0/235957121 conn(0x55b949e48460 msgr2=0x55b94a205580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:21.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.714+0000 7f734b21f740 1 -- 172.21.15.32:0/235957121 shutdown_connections 2024-08-31T07:36:21.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.714+0000 7f734b21f740 1 -- 172.21.15.32:0/235957121 wait complete. 2024-08-31T07:36:21.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.714+0000 7f734b21f740 1 -- start start 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.714+0000 7f734b21f740 1 Processor -- start 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f734b21f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a2123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f734b21f740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e33390 0x55b94a212980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f734b21f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e54070 0x55b94a2895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f734b21f740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f733c1b6f10 con 0x55b949e33390 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f734b21f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f733c1b6780 con 0x55b949e54070 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f734b21f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f733c1b3a00 con 0x55b949e2b1b0 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f734b21f740 10 client.0 _ll_register_callbacks cb 0x7f73300043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f7341a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e33390 0x55b94a212980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f7341000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a2123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f7341000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a2123e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:45304/0 (socket says 172.21.15.32:45304) 2024-08-31T07:36:21.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.715+0000 7f7341000640 1 -- 172.21.15.32:0/1892715250 learned_addr learned my addr 172.21.15.32:0/1892715250 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.716+0000 7f7341000640 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e54070 0x55b94a2895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.716+0000 7f7341a00640 1 -- 172.21.15.32:0/1892715250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 msgr2=0x55b94a2123e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.716+0000 7f7341a00640 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a2123e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.716+0000 7f7341a00640 1 -- 172.21.15.32:0/1892715250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e54070 msgr2=0x55b94a2895f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.716+0000 7f7341a00640 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e54070 0x55b94a2895f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.716+0000 7f7341a00640 1 -- 172.21.15.32:0/1892715250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b94a289c30 con 0x55b949e33390 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.716+0000 7f7341000640 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e54070 0x55b94a2895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.716+0000 7f7341a00640 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e33390 0x55b94a212980 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f733003b040 tx=0x7f7330003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6420c1b1b4998e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.716+0000 7f7323400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.717+0000 7f7323400640 1 -- 172.21.15.32:0/1892715250 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733003d070 con 0x55b949e33390 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.717+0000 7f734b21f740 1 -- 172.21.15.32:0/1892715250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7330009890 con 0x55b949e33390 2024-08-31T07:36:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.717+0000 7f7323400640 1 -- 172.21.15.32:0/1892715250 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7f7330012ca0 con 0x55b949e33390 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.717+0000 7f7323400640 1 -- 172.21.15.32:0/1892715250 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f7330012870 con 0x55b949e33390 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.717+0000 7f734b21f740 10 client.19908 fetch_fsmap learned FSMap version 91 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.717+0000 7f734b21f740 1 -- 172.21.15.32:0/1892715250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) -- 0x55b94a28aec0 con 0x55b949e33390 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.718+0000 7f7323400640 1 -- 172.21.15.32:0/1892715250 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7330009890 con 0x55b949e33390 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.718+0000 7f7323400640 1 -- 172.21.15.32:0/1892715250 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 91) ==== 8390+0+0 (secure 0 0 0) 0x7f7330004830 con 0x55b949e33390 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.718+0000 7f734b21f740 10 client.19908 fetch_fsmap finished waiting for FSMap version 91 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.718+0000 7f734b21f740 10 client.19908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.718+0000 7f734b21f740 20 client.19908 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.718+0000 7f734b21f740 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b949de7da0 0x55b94a28d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.718+0000 7f734b21f740 4 client.19908 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:36:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.718+0000 7f734b21f740 1 -- 172.21.15.32:0/1892715250 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b94a16e160 con 0x55b949de7da0 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.719+0000 7f7341000640 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b949de7da0 0x55b94a28d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.719+0000 7f7341000640 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b949de7da0 0x55b94a28d630 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54cf2e7688ab6f58 server_cookie=349c496aaebfb332 in_seq=0 out_seq=0 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.719+0000 7f7323400640 10 client.19908 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.720+0000 7f7323400640 1 -- 172.21.15.32:0/1892715250 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7670 (crc 0 0 0) 0x55b94a16e160 con 0x55b949de7da0 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.720+0000 7f7323400640 10 client.19908 handle_command_reply: tid=0 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.720+0000 7f734b21f740 1 client.19908 shutdown 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.720+0000 7f734b21f740 1 -- 172.21.15.32:0/1892715250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e33390 msgr2=0x55b94a212980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.720+0000 7f734b21f740 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e33390 0x55b94a212980 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f733003b040 tx=0x7f7330003820 comp rx=0 tx=0).stop 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.721+0000 7f734b21f740 1 -- 172.21.15.32:0/1892715250 shutdown_connections 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.721+0000 7f734b21f740 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b949de7da0 0x55b94a28d630 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.721+0000 7f734b21f740 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b949e54070 0x55b94a2895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.721+0000 7f734b21f740 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b949e33390 0x55b94a212980 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.721+0000 7f734b21f740 1 --2- 172.21.15.32:0/1892715250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b949e2b1b0 0x55b94a2123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.721+0000 7f734b21f740 1 -- 172.21.15.32:0/1892715250 >> 172.21.15.32:0/1892715250 conn(0x55b949e48460 msgr2=0x55b94a1fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.721+0000 7f7323400640 0 client.19908 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.721+0000 7f734b21f740 1 -- 172.21.15.32:0/1892715250 shutdown_connections 2024-08-31T07:36:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.721+0000 7f734b21f740 1 -- 172.21.15.32:0/1892715250 wait complete. 2024-08-31T07:36:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.722+0000 7f734b21f740 20 client.19908 trim_cache size 0 max 16384 2024-08-31T07:36:21.728 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "7d087aa2-18ca-438a-86b9-b121fe13350b": { 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "7d087aa2-18ca-438a-86b9-b121fe13350b", 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "d83ecd52-a6c2-49e8-bdaa-039848b73de4": { 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:34:26.167734+0000", 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:32:41.165270+0000", 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "7d087aa2-18ca-438a-86b9-b121fe13350b": { 2024-08-31T07:36:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:36:18.889025+0000", 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:36:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "9e14a084-7459-442a-abc0-3ac74b83528a": { 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:35:22.931369+0000", 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:36:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:36:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:36:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:36:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:36:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:36:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:36:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:36:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:36:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:36:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:36:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:36:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:36:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:36:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:36:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:36:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:36:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:36:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:36:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:36:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:36:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:36:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 -- 172.21.15.32:0/211377097 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f73240b4df0 msgr2=0x7f73240b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f73240b4df0 0x7f73240b7300 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f733400b760 tx=0x7f7334012c20 comp rx=0 tx=0).stop 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 -- 172.21.15.32:0/211377097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f733c109ba0 msgr2=0x7f733c06e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f733c109ba0 0x7f733c06e450 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7334009fd0 tx=0x7f733400b260 comp rx=0 tx=0).stop 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 -- 172.21.15.32:0/211377097 shutdown_connections 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f73240b4df0 0x7f73240b7300 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f733c109ba0 0x7f733c06e450 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f733c109250 0x7f733c06ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 --2- 172.21.15.32:0/211377097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c103910 0x7f733c074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 -- 172.21.15.32:0/211377097 >> 172.21.15.32:0/211377097 conn(0x7f733c0ff380 msgr2=0x7f733c1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 -- 172.21.15.32:0/211377097 shutdown_connections 2024-08-31T07:36:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:21.723+0000 7f7344400640 1 -- 172.21.15.32:0/211377097 wait complete. 2024-08-31T07:36:21.751 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:21 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:21.751 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:21 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:36:21.751 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:21 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:21.751 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:21 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:21.751 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:21 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:36:21.751 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:21 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:22.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 170418 13.48 MB/sec execute 482 sec latency 267.717 ms 2024-08-31T07:36:22.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:22 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:22.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:22 smithi032 ceph-mon[17252]: scrub summary: active paths [/] 2024-08-31T07:36:22.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:22 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:22.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:22 smithi032 ceph-mon[17252]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:22.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:22 smithi032 ceph-mon[17252]: pgmap v1112: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 9.6 MiB/s wr, 536 op/s 2024-08-31T07:36:22.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:22 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:22.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:22 smithi084 ceph-mon[22387]: scrub summary: active paths [/] 2024-08-31T07:36:22.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:22 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:22.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:22 smithi084 ceph-mon[22387]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:22.641 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:22 smithi084 ceph-mon[22387]: pgmap v1112: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 9.6 MiB/s wr, 536 op/s 2024-08-31T07:36:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:22 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:22 smithi073 ceph-mon[21995]: scrub summary: active paths [/] 2024-08-31T07:36:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:22 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:22 smithi073 ceph-mon[21995]: scrub summary: idle+waiting paths [/] 2024-08-31T07:36:22.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:22 smithi073 ceph-mon[21995]: pgmap v1112: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 9.6 MiB/s wr, 536 op/s 2024-08-31T07:36:23.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 170702 13.47 MB/sec execute 483 sec latency 360.111 ms 2024-08-31T07:36:24.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 171309 13.50 MB/sec execute 484 sec latency 90.361 ms 2024-08-31T07:36:24.435 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:24 smithi032 ceph-mon[17252]: pgmap v1113: 97 pgs: 97 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 8.9 MiB/s wr, 484 op/s 2024-08-31T07:36:24.438 INFO:tasks.cephfs.filesystem:scrub status for tag:7d087aa2-18ca-438a-86b9-b121fe13350b - {'path': '/', 'tag': '7d087aa2-18ca-438a-86b9-b121fe13350b', 'options': 'recursive,force'} 2024-08-31T07:36:24.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:24 smithi073 ceph-mon[21995]: pgmap v1113: 97 pgs: 97 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 8.9 MiB/s wr, 484 op/s 2024-08-31T07:36:24.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:24 smithi084 ceph-mon[22387]: pgmap v1113: 97 pgs: 97 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 8.9 MiB/s wr, 484 op/s 2024-08-31T07:36:25.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 172042 13.54 MB/sec execute 485 sec latency 97.357 ms 2024-08-31T07:36:26.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 172860 13.62 MB/sec execute 486 sec latency 118.818 ms 2024-08-31T07:36:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:26 smithi032 ceph-mon[17252]: pgmap v1114: 97 pgs: 97 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 9.8 MiB/s wr, 515 op/s 2024-08-31T07:36:26.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:26 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:26 smithi073 ceph-mon[21995]: pgmap v1114: 97 pgs: 97 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 9.8 MiB/s wr, 515 op/s 2024-08-31T07:36:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:26 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:26 smithi084 ceph-mon[22387]: pgmap v1114: 97 pgs: 97 active+clean; 2.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 9.8 MiB/s wr, 515 op/s 2024-08-31T07:36:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:26 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:27.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 173512 13.68 MB/sec execute 487 sec latency 131.690 ms 2024-08-31T07:36:28.429 INFO:tasks.workunit.client.0.smithi032.stdout: 10 174038 13.70 MB/sec execute 488 sec latency 76.233 ms 2024-08-31T07:36:28.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:28 smithi032 ceph-mon[17252]: pgmap v1115: 97 pgs: 97 active+clean; 2.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 11 MiB/s wr, 577 op/s 2024-08-31T07:36:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:28 smithi073 ceph-mon[21995]: pgmap v1115: 97 pgs: 97 active+clean; 2.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 11 MiB/s wr, 577 op/s 2024-08-31T07:36:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:28 smithi084 ceph-mon[22387]: pgmap v1115: 97 pgs: 97 active+clean; 2.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 11 MiB/s wr, 577 op/s 2024-08-31T07:36:29.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 174681 13.74 MB/sec execute 489 sec latency 136.355 ms 2024-08-31T07:36:30.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 175156 13.77 MB/sec execute 490 sec latency 461.197 ms 2024-08-31T07:36:30.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:30 smithi032 ceph-mon[17252]: pgmap v1116: 97 pgs: 97 active+clean; 2.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 9.3 MiB/s wr, 508 op/s 2024-08-31T07:36:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:30 smithi073 ceph-mon[21995]: pgmap v1116: 97 pgs: 97 active+clean; 2.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 9.3 MiB/s wr, 508 op/s 2024-08-31T07:36:30.714 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:30 smithi084 ceph-mon[22387]: pgmap v1116: 97 pgs: 97 active+clean; 2.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 9.3 MiB/s wr, 508 op/s 2024-08-31T07:36:31.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 175788 13.79 MB/sec execute 491 sec latency 251.615 ms 2024-08-31T07:36:31.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:31 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:31.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:31 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:31.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:31 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:32.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 175978 13.78 MB/sec execute 492 sec latency 832.849 ms 2024-08-31T07:36:32.483 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:32 smithi032 ceph-mon[17252]: pgmap v1117: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 10 MiB/s wr, 585 op/s 2024-08-31T07:36:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:32 smithi073 ceph-mon[21995]: pgmap v1117: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 10 MiB/s wr, 585 op/s 2024-08-31T07:36:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:32 smithi084 ceph-mon[22387]: pgmap v1117: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 10 MiB/s wr, 585 op/s 2024-08-31T07:36:33.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 176014 13.75 MB/sec execute 493 sec latency 1802.806 ms 2024-08-31T07:36:34.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 176312 13.77 MB/sec execute 494 sec latency 2467.738 ms 2024-08-31T07:36:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:34 smithi073 ceph-mon[21995]: pgmap v1118: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 9.9 MiB/s wr, 545 op/s 2024-08-31T07:36:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:34 smithi084 ceph-mon[22387]: pgmap v1118: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 9.9 MiB/s wr, 545 op/s 2024-08-31T07:36:34.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:34 smithi032 ceph-mon[17252]: pgmap v1118: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 9.9 MiB/s wr, 545 op/s 2024-08-31T07:36:35.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177061 13.84 MB/sec execute 495 sec latency 67.579 ms 2024-08-31T07:36:36.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177573 13.86 MB/sec execute 496 sec latency 164.664 ms 2024-08-31T07:36:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:36 smithi073 ceph-mon[21995]: pgmap v1119: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 12 MiB/s wr, 575 op/s 2024-08-31T07:36:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:36 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:36 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:36:36.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:36 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:36:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:36 smithi084 ceph-mon[22387]: pgmap v1119: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 12 MiB/s wr, 575 op/s 2024-08-31T07:36:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:36 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:36 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:36:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:36 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:36:36.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:36 smithi032 ceph-mon[17252]: pgmap v1119: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 12 MiB/s wr, 575 op/s 2024-08-31T07:36:36.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:36 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:36.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:36 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:36:36.771 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:36 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:36:37.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177755 13.85 MB/sec execute 497 sec latency 701.208 ms 2024-08-31T07:36:37.645 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:37 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:36:37.645 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:37 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:36:37.645 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:37 smithi032 ceph-mon[17252]: scrub summary: idle 2024-08-31T07:36:37.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:37 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:36:37.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:37 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:36:37.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:37 smithi073 ceph-mon[21995]: scrub summary: idle 2024-08-31T07:36:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:37 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:36:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:37 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:36:37.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:37 smithi084 ceph-mon[22387]: scrub summary: idle 2024-08-31T07:36:38.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177755 13.82 MB/sec execute 498 sec latency 1696.503 ms 2024-08-31T07:36:38.504 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:38 smithi032 ceph-mon[17252]: pgmap v1120: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 13 MiB/s wr, 600 op/s 2024-08-31T07:36:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:38 smithi084 ceph-mon[22387]: pgmap v1120: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 13 MiB/s wr, 600 op/s 2024-08-31T07:36:38.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:38 smithi073 ceph-mon[21995]: pgmap v1120: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 13 MiB/s wr, 600 op/s 2024-08-31T07:36:39.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.80 MB/sec execute 499 sec latency 2696.551 ms 2024-08-31T07:36:40.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.77 MB/sec execute 500 sec latency 3696.602 ms 2024-08-31T07:36:40.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:40 smithi032 ceph-mon[17252]: pgmap v1121: 97 pgs: 97 active+clean; 2.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 513 op/s 2024-08-31T07:36:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:40 smithi084 ceph-mon[22387]: pgmap v1121: 97 pgs: 97 active+clean; 2.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 513 op/s 2024-08-31T07:36:40.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:40 smithi073 ceph-mon[21995]: pgmap v1121: 97 pgs: 97 active+clean; 2.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 513 op/s 2024-08-31T07:36:41.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.74 MB/sec execute 501 sec latency 4696.645 ms 2024-08-31T07:36:42.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.71 MB/sec execute 502 sec latency 5696.688 ms 2024-08-31T07:36:42.499 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:42 smithi032 ceph-mon[17252]: pgmap v1122: 97 pgs: 97 active+clean; 2.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 596 op/s 2024-08-31T07:36:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:42 smithi084 ceph-mon[22387]: pgmap v1122: 97 pgs: 97 active+clean; 2.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 596 op/s 2024-08-31T07:36:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:42 smithi073 ceph-mon[21995]: pgmap v1122: 97 pgs: 97 active+clean; 2.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 596 op/s 2024-08-31T07:36:43.430 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.69 MB/sec execute 503 sec latency 6696.732 ms 2024-08-31T07:36:44.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.66 MB/sec execute 504 sec latency 7696.798 ms 2024-08-31T07:36:44.585 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:44 smithi032 ceph-mon[17252]: pgmap v1123: 97 pgs: 97 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 540 op/s 2024-08-31T07:36:44.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:44 smithi084 ceph-mon[22387]: pgmap v1123: 97 pgs: 97 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 540 op/s 2024-08-31T07:36:44.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:44 smithi073 ceph-mon[21995]: pgmap v1123: 97 pgs: 97 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 540 op/s 2024-08-31T07:36:45.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.63 MB/sec execute 505 sec latency 8696.862 ms 2024-08-31T07:36:46.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.61 MB/sec execute 506 sec latency 9696.920 ms 2024-08-31T07:36:46.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:46 smithi032 ceph-mon[17252]: pgmap v1124: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 537 op/s 2024-08-31T07:36:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:46 smithi084 ceph-mon[22387]: pgmap v1124: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 537 op/s 2024-08-31T07:36:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:46 smithi073 ceph-mon[21995]: pgmap v1124: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 537 op/s 2024-08-31T07:36:47.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.58 MB/sec execute 507 sec latency 10696.965 ms 2024-08-31T07:36:48.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.55 MB/sec execute 508 sec latency 11697.023 ms 2024-08-31T07:36:48.572 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:48 smithi032 ceph-mon[17252]: pgmap v1125: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 497 op/s 2024-08-31T07:36:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:48 smithi084 ceph-mon[22387]: pgmap v1125: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 497 op/s 2024-08-31T07:36:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:48 smithi073 ceph-mon[21995]: pgmap v1125: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 497 op/s 2024-08-31T07:36:49.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.53 MB/sec execute 509 sec latency 12697.084 ms 2024-08-31T07:36:50.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.50 MB/sec execute 510 sec latency 13697.157 ms 2024-08-31T07:36:50.533 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:50 smithi032 ceph-mon[17252]: pgmap v1126: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.2 MiB/s wr, 387 op/s 2024-08-31T07:36:50.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:50 smithi084 ceph-mon[22387]: pgmap v1126: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.2 MiB/s wr, 387 op/s 2024-08-31T07:36:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:50 smithi073 ceph-mon[21995]: pgmap v1126: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.2 MiB/s wr, 387 op/s 2024-08-31T07:36:51.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.47 MB/sec execute 511 sec latency 14697.199 ms 2024-08-31T07:36:52.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.45 MB/sec execute 512 sec latency 15697.263 ms 2024-08-31T07:36:52.529 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:52 smithi032 ceph-mon[17252]: pgmap v1127: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.8 MiB/s wr, 279 op/s 2024-08-31T07:36:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:52 smithi084 ceph-mon[22387]: pgmap v1127: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.8 MiB/s wr, 279 op/s 2024-08-31T07:36:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:52 smithi073 ceph-mon[21995]: pgmap v1127: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.8 MiB/s wr, 279 op/s 2024-08-31T07:36:53.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177756 13.42 MB/sec execute 513 sec latency 16697.328 ms 2024-08-31T07:36:54.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177851 13.40 MB/sec execute 514 sec latency 17689.367 ms 2024-08-31T07:36:54.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T07:36:54.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:54 smithi084 ceph-mon[22387]: pgmap v1128: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-08-31T07:36:54.744 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:54 smithi032 ceph-mon[17252]: pgmap v1128: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-08-31T07:36:54.870 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:54 smithi073 ceph-mon[21995]: pgmap v1128: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-08-31T07:36:55.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177917 13.38 MB/sec execute 515 sec latency 18689.441 ms 2024-08-31T07:36:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.743+0000 7ff2b7200640 1 -- 172.21.15.32:0/322006550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b010b490 msgr2=0x7ff2b010b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.743+0000 7ff2b7200640 1 --2- 172.21.15.32:0/322006550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b010b490 0x7ff2b010b8a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff298009900 tx=0x7ff29802a410 comp rx=0 tx=0).stop 2024-08-31T07:36:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.743+0000 7ff2b7200640 1 -- 172.21.15.32:0/322006550 shutdown_connections 2024-08-31T07:36:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.743+0000 7ff2b7200640 1 --2- 172.21.15.32:0/322006550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2b010bde0 0x7ff2b0112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.743+0000 7ff2b7200640 1 --2- 172.21.15.32:0/322006550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b010b490 0x7ff2b010b8a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.743+0000 7ff2b7200640 1 --2- 172.21.15.32:0/322006550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b0105b50 0x7ff2b0105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.743+0000 7ff2b7200640 1 -- 172.21.15.32:0/322006550 >> 172.21.15.32:0/322006550 conn(0x7ff2b0101580 msgr2=0x7ff2b01039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.743+0000 7ff2b7200640 1 -- 172.21.15.32:0/322006550 shutdown_connections 2024-08-31T07:36:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b7200640 1 -- 172.21.15.32:0/322006550 wait complete. 2024-08-31T07:36:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b7200640 1 Processor -- start 2024-08-31T07:36:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b7200640 1 -- start start 2024-08-31T07:36:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b0105b50 0x7ff2b019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b7200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b010b490 0x7ff2b019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2b010bde0 0x7ff2b01a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b0115af0 con 0x7ff2b010bde0 2024-08-31T07:36:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b0115970 con 0x7ff2b0105b50 2024-08-31T07:36:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b7200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b0115c70 con 0x7ff2b010b490 2024-08-31T07:36:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b0105b50 0x7ff2b019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b0105b50 0x7ff2b019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59250/0 (socket says 172.21.15.32:59250) 2024-08-31T07:36:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2afe00640 1 -- 172.21.15.32:0/919185601 learned_addr learned my addr 172.21.15.32:0/919185601 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2afe00640 1 -- 172.21.15.32:0/919185601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b010b490 msgr2=0x7ff2b019cae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:36:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2afe00640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b010b490 0x7ff2b019cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2afe00640 1 -- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2b010bde0 msgr2=0x7ff2b01a1d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T07:36:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b5200640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2b010bde0 0x7ff2b01a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2afe00640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2b010bde0 0x7ff2b01a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2afe00640 1 -- 172.21.15.32:0/919185601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b01a24b0 con 0x7ff2b0105b50 2024-08-31T07:36:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2b5200640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2b010bde0 0x7ff2b01a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:36:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.748+0000 7ff2afe00640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b0105b50 0x7ff2b019c540 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000d950 tx=0x7ff2a000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.753+0000 7ff2ad600640 1 -- 172.21.15.32:0/919185601 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0017030 con 0x7ff2b0105b50 2024-08-31T07:36:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.753+0000 7ff2ad600640 1 -- 172.21.15.32:0/919185601 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2a0004020 con 0x7ff2b0105b50 2024-08-31T07:36:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.753+0000 7ff2b7200640 1 -- 172.21.15.32:0/919185601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b01a27a0 con 0x7ff2b0105b50 2024-08-31T07:36:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.753+0000 7ff2ad600640 1 -- 172.21.15.32:0/919185601 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0003c70 con 0x7ff2b0105b50 2024-08-31T07:36:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.753+0000 7ff2b7200640 1 -- 172.21.15.32:0/919185601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b01a8f00 con 0x7ff2b0105b50 2024-08-31T07:36:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2bde53740 1 Processor -- start 2024-08-31T07:36:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2bde53740 1 -- start start 2024-08-31T07:36:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2ad600640 1 -- 172.21.15.32:0/919185601 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff2a001a070 con 0x7ff2b0105b50 2024-08-31T07:36:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 0x5596ed7b9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3eb1b0 0x5596ed7b9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 0x5596ed7ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2bde53740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5596ed730770 con 0x5596ed3eb1b0 2024-08-31T07:36:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2bde53740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5596ed71fd00 con 0x5596ed3f3390 2024-08-31T07:36:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2bde53740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5596ed723cb0 con 0x5596ed414070 2024-08-31T07:36:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2b5200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 0x5596ed7b9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2b5200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 0x5596ed7b9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33840/0 (socket says 172.21.15.32:33840) 2024-08-31T07:36:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2b5200640 1 -- 172.21.15.32:0/169791567 learned_addr learned my addr 172.21.15.32:0/169791567 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2a7e00640 1 --2- 172.21.15.32:0/169791567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 0x5596ed7ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2b5200640 1 -- 172.21.15.32:0/169791567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 msgr2=0x5596ed7ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2b5200640 1 --2- 172.21.15.32:0/169791567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 0x5596ed7ba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2afe00640 1 --2- 172.21.15.32:0/169791567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3eb1b0 0x5596ed7b9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2b5200640 1 -- 172.21.15.32:0/169791567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3eb1b0 msgr2=0x5596ed7b9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2b5200640 1 --2- 172.21.15.32:0/169791567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3eb1b0 0x5596ed7b9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2b5200640 1 -- 172.21.15.32:0/169791567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596ed47f650 con 0x5596ed414070 2024-08-31T07:36:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2afe00640 1 --2- 172.21.15.32:0/169791567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3eb1b0 0x5596ed7b9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:36:55.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.756+0000 7ff2ad600640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff2940b4da0 0x7ff2940b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.762+0000 7ff2ad600640 1 -- 172.21.15.32:0/919185601 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7ff2a00e4f90 con 0x7ff2b0105b50 2024-08-31T07:36:55.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.762+0000 7ff2afe00640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff2940b4da0 0x7ff2940b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.764+0000 7ff2afe00640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff2940b4da0 0x7ff2940b72b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5596ed7b85d0 tx=0x7ff2a0000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2b5200640 1 --2- 172.21.15.32:0/169791567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 0x5596ed7b9950 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009f10 tx=0x7ff2a800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca6268acdc062918 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2a5600640 1 -- 172.21.15.32:0/169791567 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8003b70 con 0x5596ed414070 2024-08-31T07:36:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2a5600640 1 -- 172.21.15.32:0/169791567 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2a8003d10 con 0x5596ed414070 2024-08-31T07:36:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2a5600640 1 -- 172.21.15.32:0/169791567 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80119a0 con 0x5596ed414070 2024-08-31T07:36:55.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2bde53740 1 -- 172.21.15.32:0/169791567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 msgr2=0x5596ed7b9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:55.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2bde53740 1 --2- 172.21.15.32:0/169791567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 0x5596ed7b9950 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009f10 tx=0x7ff2a800f730 comp rx=0 tx=0).stop 2024-08-31T07:36:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2bde53740 1 -- 172.21.15.32:0/169791567 shutdown_connections 2024-08-31T07:36:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2bde53740 1 --2- 172.21.15.32:0/169791567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 0x5596ed7ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2bde53740 1 --2- 172.21.15.32:0/169791567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3eb1b0 0x5596ed7b9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2bde53740 1 --2- 172.21.15.32:0/169791567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 0x5596ed7b9950 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2bde53740 1 -- 172.21.15.32:0/169791567 >> 172.21.15.32:0/169791567 conn(0x5596ed408460 msgr2=0x5596ed7b7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2bde53740 1 -- 172.21.15.32:0/169791567 shutdown_connections 2024-08-31T07:36:55.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2bde53740 1 -- 172.21.15.32:0/169791567 wait complete. 2024-08-31T07:36:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.767+0000 7ff2bde53740 1 -- start start 2024-08-31T07:36:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 Processor -- start 2024-08-31T07:36:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3f3390 0x5596ed7ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 0x5596ed7d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5596ed582700 con 0x5596ed3f3390 2024-08-31T07:36:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5596ed6a6cb0 con 0x5596ed3eb1b0 2024-08-31T07:36:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5596ed7c8130 con 0x5596ed414070 2024-08-31T07:36:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 10 client.0 _ll_register_callbacks cb 0x5596ed6cc430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2b5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2b5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59280/0 (socket says 172.21.15.32:59280) 2024-08-31T07:36:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2b5200640 1 -- 172.21.15.32:0/1860076001 learned_addr learned my addr 172.21.15.32:0/1860076001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2b5200640 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 0x5596ed7d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2a7e00640 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3f3390 0x5596ed7ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2b5200640 1 -- 172.21.15.32:0/1860076001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 msgr2=0x5596ed7d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2b5200640 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 0x5596ed7d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2b5200640 1 -- 172.21.15.32:0/1860076001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3f3390 msgr2=0x5596ed7ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2b5200640 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3f3390 0x5596ed7ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2b5200640 1 -- 172.21.15.32:0/1860076001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596ed6e7870 con 0x5596ed3eb1b0 2024-08-31T07:36:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2a7e00640 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3f3390 0x5596ed7ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:36:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2b5200640 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7ce410 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8003c20 tx=0x7ff2a8011580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8285bc7baa1dffa0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff29e000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:36:55.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff29e000640 1 -- 172.21.15.32:0/1860076001 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8011b30 con 0x5596ed3eb1b0 2024-08-31T07:36:55.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff29e000640 1 -- 172.21.15.32:0/1860076001 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7ff2a8011cd0 con 0x5596ed3eb1b0 2024-08-31T07:36:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 -- 172.21.15.32:0/1860076001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a8003d10 con 0x5596ed3eb1b0 2024-08-31T07:36:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff29e000640 1 -- 172.21.15.32:0/1860076001 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff2a8003ec0 con 0x5596ed3eb1b0 2024-08-31T07:36:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 10 client.29999 fetch_fsmap learned FSMap version 94 2024-08-31T07:36:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 -- 172.21.15.32:0/1860076001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=4+}) -- 0x5596ed7c0980 con 0x5596ed3eb1b0 2024-08-31T07:36:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff29e000640 1 -- 172.21.15.32:0/1860076001 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2a8003d10 con 0x5596ed3eb1b0 2024-08-31T07:36:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff29e000640 1 -- 172.21.15.32:0/1860076001 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 94) ==== 8410+0+0 (secure 0 0 0) 0x7ff2a8029020 con 0x5596ed3eb1b0 2024-08-31T07:36:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 10 client.29999 fetch_fsmap finished waiting for FSMap version 94 2024-08-31T07:36:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 10 client.29999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:36:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 20 client.29999 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:55.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596ed3d29f0 0x5596ed7c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 4 client.29999 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:36:55.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2bde53740 1 -- 172.21.15.32:0/1860076001 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596ed7326c0 con 0x5596ed3d29f0 2024-08-31T07:36:55.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.771+0000 7ff2a7e00640 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596ed3d29f0 0x5596ed7c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.776+0000 7ff2a7e00640 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596ed3d29f0 0x5596ed7c3d90 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3004e835643f3485 server_cookie=7dc38215e616a59b in_seq=0 out_seq=0 2024-08-31T07:36:55.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.776+0000 7ff29e000640 10 client.29999 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:55.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.778+0000 7ff29e000640 1 -- 172.21.15.32:0/1860076001 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596ed7326c0 con 0x5596ed3d29f0 2024-08-31T07:36:55.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.778+0000 7ff29e000640 10 client.29999 handle_command_reply: tid=0 2024-08-31T07:36:55.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.778+0000 7ff2bde53740 1 client.29999 shutdown 2024-08-31T07:36:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 1 -- 172.21.15.32:0/1860076001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 msgr2=0x5596ed7ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7ce410 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8003c20 tx=0x7ff2a8011580 comp rx=0 tx=0).stop 2024-08-31T07:36:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 1 -- 172.21.15.32:0/1860076001 shutdown_connections 2024-08-31T07:36:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596ed3d29f0 0x5596ed7c3d90 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed414070 0x5596ed7d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed3f3390 0x5596ed7ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 1 --2- 172.21.15.32:0/1860076001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7ce410 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff29e000640 0 client.29999 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 1 -- 172.21.15.32:0/1860076001 >> 172.21.15.32:0/1860076001 conn(0x5596ed408460 msgr2=0x5596ed7c7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 1 -- 172.21.15.32:0/1860076001 shutdown_connections 2024-08-31T07:36:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 1 -- 172.21.15.32:0/1860076001 wait complete. 2024-08-31T07:36:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.779+0000 7ff2bde53740 20 client.29999 trim_cache size 0 max 16384 2024-08-31T07:36:55.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.814+0000 7ff2bde53740 1 Processor -- start 2024-08-31T07:36:55.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.814+0000 7ff2bde53740 1 -- start start 2024-08-31T07:36:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3eb1b0 0x5596ed7bd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 0x5596ed7d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 0x5596ed7b7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c00e6c0 con 0x5596ed414070 2024-08-31T07:36:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c010440 con 0x5596ed3f3390 2024-08-31T07:36:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c00ee50 con 0x5596ed3eb1b0 2024-08-31T07:36:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 0x5596ed7d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 0x5596ed7d37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:59294/0 (socket says 172.21.15.32:59294) 2024-08-31T07:36:55.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 -- 172.21.15.32:0/1093545955 learned_addr learned my addr 172.21.15.32:0/1093545955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:55.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2b5200640 1 --2- 172.21.15.32:0/1093545955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3eb1b0 0x5596ed7bd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2b5200640 1 --2- 172.21.15.32:0/1093545955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 0x5596ed7b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 -- 172.21.15.32:0/1093545955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3eb1b0 msgr2=0x5596ed7bd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:55.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 --2- 172.21.15.32:0/1093545955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3eb1b0 0x5596ed7bd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 -- 172.21.15.32:0/1093545955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 msgr2=0x5596ed7b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 --2- 172.21.15.32:0/1093545955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 0x5596ed7b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 -- 172.21.15.32:0/1093545955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596ed7c8010 con 0x5596ed3f3390 2024-08-31T07:36:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2b5200640 1 --2- 172.21.15.32:0/1093545955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 0x5596ed7b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:36:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 --2- 172.21.15.32:0/1093545955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 0x5596ed7d37f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff298006010 tx=0x7ff298005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e35e78a7111010a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a4c00640 1 -- 172.21.15.32:0/1093545955 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298035040 con 0x5596ed3f3390 2024-08-31T07:36:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a4c00640 1 -- 172.21.15.32:0/1093545955 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff298004470 con 0x5596ed3f3390 2024-08-31T07:36:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a4c00640 1 -- 172.21.15.32:0/1093545955 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29803d690 con 0x5596ed3f3390 2024-08-31T07:36:55.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- 172.21.15.32:0/1093545955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 msgr2=0x5596ed7d37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 --2- 172.21.15.32:0/1093545955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 0x5596ed7d37f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff298006010 tx=0x7ff298005a20 comp rx=0 tx=0).stop 2024-08-31T07:36:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- 172.21.15.32:0/1093545955 shutdown_connections 2024-08-31T07:36:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 --2- 172.21.15.32:0/1093545955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 0x5596ed7b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 --2- 172.21.15.32:0/1093545955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3f3390 0x5596ed7d37f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 --2- 172.21.15.32:0/1093545955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3eb1b0 0x5596ed7bd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- 172.21.15.32:0/1093545955 >> 172.21.15.32:0/1093545955 conn(0x5596ed408460 msgr2=0x5596ed7b6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:55.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- 172.21.15.32:0/1093545955 shutdown_connections 2024-08-31T07:36:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- 172.21.15.32:0/1093545955 wait complete. 2024-08-31T07:36:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- start start 2024-08-31T07:36:55.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 Processor -- start 2024-08-31T07:36:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7da500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3f3390 0x5596ed7effb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 0x5596ed7f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c00fce0 con 0x5596ed414070 2024-08-31T07:36:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c00f550 con 0x5596ed3eb1b0 2024-08-31T07:36:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c00c6c0 con 0x5596ed3f3390 2024-08-31T07:36:55.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2bde53740 10 client.0 _ll_register_callbacks cb 0x7ff2a8009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:36:55.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3f3390 0x5596ed7effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3f3390 0x5596ed7effb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:33876/0 (socket says 172.21.15.32:33876) 2024-08-31T07:36:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2a7e00640 1 -- 172.21.15.32:0/802327919 learned_addr learned my addr 172.21.15.32:0/802327919 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2b5200640 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7da500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2b5200640 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 0x5596ed7f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2b5200640 1 -- 172.21.15.32:0/802327919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3f3390 msgr2=0x5596ed7effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2b5200640 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3f3390 0x5596ed7effb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2b5200640 1 -- 172.21.15.32:0/802327919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 msgr2=0x5596ed7f05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:36:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2b5200640 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 0x5596ed7f05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.815+0000 7ff2b5200640 1 -- 172.21.15.32:0/802327919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596ed7f0d40 con 0x5596ed3eb1b0 2024-08-31T07:36:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2b5200640 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7da500 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8003bf0 tx=0x7ff2a80114e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d874329407a13a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff29e000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:36:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff29e000640 1 -- 172.21.15.32:0/802327919 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8005bd0 con 0x5596ed3eb1b0 2024-08-31T07:36:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff29e000640 1 -- 172.21.15.32:0/802327919 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7ff2a802b630 con 0x5596ed3eb1b0 2024-08-31T07:36:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 -- 172.21.15.32:0/802327919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff298004470 con 0x5596ed3eb1b0 2024-08-31T07:36:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff29e000640 1 -- 172.21.15.32:0/802327919 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff2a8046050 con 0x5596ed3eb1b0 2024-08-31T07:36:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 10 client.30005 fetch_fsmap learned FSMap version 94 2024-08-31T07:36:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 -- 172.21.15.32:0/802327919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=4+}) -- 0x5596ed852d40 con 0x5596ed3eb1b0 2024-08-31T07:36:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff29e000640 1 -- 172.21.15.32:0/802327919 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff298004470 con 0x5596ed3eb1b0 2024-08-31T07:36:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff29e000640 1 -- 172.21.15.32:0/802327919 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 94) ==== 8410+0+0 (secure 0 0 0) 0x7ff2a8029040 con 0x5596ed3eb1b0 2024-08-31T07:36:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 10 client.30005 fetch_fsmap finished waiting for FSMap version 94 2024-08-31T07:36:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 10 client.30005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:36:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 20 client.30005 populate_metadata read hostname 'smithi032' 2024-08-31T07:36:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596ed3a7da0 0x5596ed855560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 4 client.30005 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T07:36:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 -- 172.21.15.32:0/802327919 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5596ed72e160 con 0x5596ed3a7da0 2024-08-31T07:36:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2a7e00640 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596ed3a7da0 0x5596ed855560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2a7e00640 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596ed3a7da0 0x5596ed855560 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9a29d78bf99f01e server_cookie=4ac55af09a99808 in_seq=0 out_seq=0 2024-08-31T07:36:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff29e000640 10 client.30005 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff29e000640 1 -- 172.21.15.32:0/802327919 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7474 (crc 0 0 0) 0x5596ed72e160 con 0x5596ed3a7da0 2024-08-31T07:36:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff29e000640 10 client.30005 handle_command_reply: tid=0 2024-08-31T07:36:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 client.30005 shutdown 2024-08-31T07:36:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 -- 172.21.15.32:0/802327919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 msgr2=0x5596ed7da500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7da500 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8003bf0 tx=0x7ff2a80114e0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 -- 172.21.15.32:0/802327919 shutdown_connections 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5596ed3a7da0 0x5596ed855560 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5596ed414070 0x5596ed7f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5596ed3f3390 0x5596ed7effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 --2- 172.21.15.32:0/802327919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596ed3eb1b0 0x5596ed7da500 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 -- 172.21.15.32:0/802327919 >> 172.21.15.32:0/802327919 conn(0x5596ed408460 msgr2=0x5596ed7b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff29e000640 0 client.30005 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 -- 172.21.15.32:0/802327919 shutdown_connections 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 1 -- 172.21.15.32:0/802327919 wait complete. 2024-08-31T07:36:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2bde53740 20 client.30005 trim_cache size 0 max 16384 2024-08-31T07:36:55.830 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:36:55.830 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-08-31T07:36:55.830 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {}, 2024-08-31T07:36:55.830 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_stats": { 2024-08-31T07:36:55.830 INFO:teuthology.orchestra.run.smithi032.stdout: "d83ecd52-a6c2-49e8-bdaa-039848b73de4": { 2024-08-31T07:36:55.830 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:34:26.167734+0000", 2024-08-31T07:36:55.830 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.830 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.830 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.830 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "5476ca92-3649-4ebb-868c-09cfdc2c594c": { 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:32:41.165270+0000", 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "7d087aa2-18ca-438a-86b9-b121fe13350b": { 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:36:18.889025+0000", 2024-08-31T07:36:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "d52d6274-7572-465a-9504-ef90242d910a": { 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:30:41.570175+0000", 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "05113edd-ca51-44d5-a01f-804676bde5ad": { 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:28:46.371996+0000", 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "e8e1d5a1-3d3b-41fd-a6d8-6f282a64c3b5": { 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:25:23.137958+0000", 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.833 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "9e14a084-7459-442a-abc0-3ac74b83528a": { 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:35:22.931369+0000", 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "ce8b7231-1fea-412d-a295-8119d9478cfa": { 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:26:17.986621+0000", 2024-08-31T07:36:55.834 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "96b7006d-efb1-4838-bb2b-531e42fd1874": { 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:24:32.308195+0000", 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "323cd93f-7bb8-4706-b7dc-0b247836a042": { 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:12:24.381228+0000", 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "2e220b7d-14bd-4c0f-9dab-7cb5e62ab378": { 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:13:17.014919+0000", 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.836 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "9fcca2ec-b3db-4784-8740-c1e275303481": { 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:14:07.467664+0000", 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "8efe0b72-f373-45ce-94be-5e685a09cf82": { 2024-08-31T07:36:55.837 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:00.661337+0000", 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "d4942a56-ab98-4c7e-922a-46e26db5d6ed": { 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:27:37.477681+0000", 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "7b4538c9-f63d-4e4f-a64a-4d047cc3b4a8": { 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:19:18.586903+0000", 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "cb664667-a37e-4c2a-b16d-2ae314b4a7c7": { 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:18:27.911161+0000", 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.839 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "fb535dd7-5c86-4ca9-bd4e-7017aaa318b3": { 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:31:35.174869+0000", 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "23179d43-8143-476f-8da3-09d7ff58e6e1": { 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:29:42.684617+0000", 2024-08-31T07:36:55.840 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "c2f847a7-820f-48ef-90f2-3d4a5f290866": { 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:23:40.051463+0000", 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.841 INFO:teuthology.orchestra.run.smithi032.stdout: "4ce84d20-d56a-4c0e-b303-5ada6c73ea15": { 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:01.869346+0000", 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "08644365-5854-418b-a4c9-1e76ad60211b": { 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:15:51.213071+0000", 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "fd7d31bf-f49a-41f6-8889-081d61bf9bb8": { 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:16:45.661706+0000", 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "47f0c443-ae9b-4a28-b8df-2ef348dd2274": { 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:17:36.280444+0000", 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.843 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "fa02e0e6-4fcf-48e2-9f34-5121c7f65c22": { 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:20:10.042628+0000", 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "7f58fa9a-908d-4f85-9621-856bff2ad0f2": { 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:21:54.470048+0000", 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.844 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: "a047226e-5dfa-4aa9-bb91-ffd989a87ee6": { 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: "start_time": "2024-08-31T07:22:47.134754+0000", 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_passed": 0, 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_failed": 0, 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_skipped": 0 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:36:55.845 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2b7200640 1 -- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff2940b4da0 msgr2=0x7ff2940b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2b7200640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff2940b4da0 0x7ff2940b72b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5596ed7b85d0 tx=0x7ff2a0000bc0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2b7200640 1 -- 172.21.15.32:0/919185601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b0105b50 msgr2=0x7ff2b019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2b7200640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b0105b50 0x7ff2b019c540 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000d950 tx=0x7ff2a000de10 comp rx=0 tx=0).stop 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2b7200640 1 -- 172.21.15.32:0/919185601 shutdown_connections 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2b7200640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff2940b4da0 0x7ff2940b72b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2b7200640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2b010bde0 0x7ff2b01a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2b7200640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b010b490 0x7ff2b019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.821+0000 7ff2b7200640 1 --2- 172.21.15.32:0/919185601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b0105b50 0x7ff2b019c540 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.826+0000 7ff2b7200640 1 -- 172.21.15.32:0/919185601 >> 172.21.15.32:0/919185601 conn(0x7ff2b0101580 msgr2=0x7ff2b010aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.826+0000 7ff2b7200640 1 -- 172.21.15.32:0/919185601 shutdown_connections 2024-08-31T07:36:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:55.826+0000 7ff2b7200640 1 -- 172.21.15.32:0/919185601 wait complete. 2024-08-31T07:36:56.431 INFO:tasks.workunit.client.0.smithi032.stdout: 10 178106 13.38 MB/sec execute 516 sec latency 19689.491 ms 2024-08-31T07:36:56.636 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:56 smithi032 ceph-mon[17252]: pgmap v1129: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 707 KiB/s wr, 89 op/s 2024-08-31T07:36:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:56 smithi073 ceph-mon[21995]: pgmap v1129: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 707 KiB/s wr, 89 op/s 2024-08-31T07:36:56.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:56 smithi084 ceph-mon[22387]: pgmap v1129: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 707 KiB/s wr, 89 op/s 2024-08-31T07:36:56.996 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T07:36:56.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:36:57.432 INFO:tasks.workunit.client.0.smithi032.stdout: 10 178303 13.38 MB/sec execute 517 sec latency 20162.538 ms 2024-08-31T07:36:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:36:58.432 INFO:tasks.workunit.client.0.smithi032.stdout: 10 178761 13.38 MB/sec execute 518 sec latency 19381.371 ms 2024-08-31T07:36:58.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.466+0000 7fb958200640 1 -- 172.21.15.32:0/297268153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb950075b70 msgr2=0x7fb950075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:58.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.466+0000 7fb958200640 1 --2- 172.21.15.32:0/297268153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb950075b70 0x7fb950075f80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009900 tx=0x7fb93c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:36:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.467+0000 7fb958200640 1 -- 172.21.15.32:0/297268153 shutdown_connections 2024-08-31T07:36:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.467+0000 7fb958200640 1 --2- 172.21.15.32:0/297268153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9501141f0 0x7fb9501166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.467+0000 7fb958200640 1 --2- 172.21.15.32:0/297268153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb950075b70 0x7fb950075f80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.467+0000 7fb958200640 1 --2- 172.21.15.32:0/297268153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb950077120 0x7fb950075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.467+0000 7fb958200640 1 -- 172.21.15.32:0/297268153 >> 172.21.15.32:0/297268153 conn(0x7fb9500ff3b0 msgr2=0x7fb950101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.467+0000 7fb958200640 1 -- 172.21.15.32:0/297268153 shutdown_connections 2024-08-31T07:36:58.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.467+0000 7fb958200640 1 -- 172.21.15.32:0/297268153 wait complete. 2024-08-31T07:36:58.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.471+0000 7fb958200640 1 Processor -- start 2024-08-31T07:36:58.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.471+0000 7fb958200640 1 -- start start 2024-08-31T07:36:58.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.472+0000 7fb958200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb950075b70 0x7fb9501a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:58.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.472+0000 7fb958200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb950077120 0x7fb9501a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:58.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.472+0000 7fb958200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9501141f0 0x7fb9501a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:58.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.472+0000 7fb958200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb950119ee0 con 0x7fb950075b70 2024-08-31T07:36:58.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.472+0000 7fb958200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb950119d60 con 0x7fb950077120 2024-08-31T07:36:58.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.472+0000 7fb958200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb95011a060 con 0x7fb9501141f0 2024-08-31T07:36:58.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.473+0000 7fb955800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb950075b70 0x7fb9501a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:58.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.473+0000 7fb956200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9501141f0 0x7fb9501a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:58.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.473+0000 7fb956200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9501141f0 0x7fb9501a60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50938/0 (socket says 172.21.15.32:50938) 2024-08-31T07:36:58.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.473+0000 7fb956200640 1 -- 172.21.15.32:0/31518177 learned_addr learned my addr 172.21.15.32:0/31518177 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:36:58.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.474+0000 7fb956200640 1 -- 172.21.15.32:0/31518177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb950077120 msgr2=0x7fb9501a0e30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:36:58.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.474+0000 7fb956200640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb950077120 0x7fb9501a0e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:58.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.474+0000 7fb956200640 1 -- 172.21.15.32:0/31518177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb950075b70 msgr2=0x7fb9501a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:58.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.474+0000 7fb956200640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb950075b70 0x7fb9501a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:58.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.474+0000 7fb956200640 1 -- 172.21.15.32:0/31518177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9501a6800 con 0x7fb9501141f0 2024-08-31T07:36:58.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.476+0000 7fb956200640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9501141f0 0x7fb9501a60d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb94800b970 tx=0x7fb94800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:58.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.477+0000 7fb94e000640 1 -- 172.21.15.32:0/31518177 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948019070 con 0x7fb9501141f0 2024-08-31T07:36:58.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.477+0000 7fb94e000640 1 -- 172.21.15.32:0/31518177 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb948014440 con 0x7fb9501141f0 2024-08-31T07:36:58.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.477+0000 7fb94e000640 1 -- 172.21.15.32:0/31518177 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9480075d0 con 0x7fb9501141f0 2024-08-31T07:36:58.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.477+0000 7fb958200640 1 -- 172.21.15.32:0/31518177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9501a6af0 con 0x7fb9501141f0 2024-08-31T07:36:58.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.477+0000 7fb958200640 1 -- 172.21.15.32:0/31518177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9501ad250 con 0x7fb9501141f0 2024-08-31T07:36:58.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.481+0000 7fb943400640 1 -- 172.21.15.32:0/31518177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb950075f80 con 0x7fb9501141f0 2024-08-31T07:36:58.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.484+0000 7fb94e000640 1 -- 172.21.15.32:0/31518177 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb94801c070 con 0x7fb9501141f0 2024-08-31T07:36:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.486+0000 7fb94e000640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9380b4e70 0x7fb9380b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:36:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.486+0000 7fb94e000640 1 -- 172.21.15.32:0/31518177 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7fb9480e48d0 con 0x7fb9501141f0 2024-08-31T07:36:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.488+0000 7fb955800640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9380b4e70 0x7fb9380b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:36:58.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.491+0000 7fb955800640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9380b4e70 0x7fb9380b7380 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb944009bd0 tx=0x7fb944005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:36:58.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.492+0000 7fb94e000640 1 -- 172.21.15.32:0/31518177 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb9480ab060 con 0x7fb9501141f0 2024-08-31T07:36:58.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:58 smithi032 ceph-mon[17252]: pgmap v1130: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 526 KiB/s wr, 38 op/s 2024-08-31T07:36:58.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.885+0000 7fb943400640 1 -- 172.21.15.32:0/31518177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb950063d60 con 0x7fb9501141f0 2024-08-31T07:36:58.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.886+0000 7fb94e000640 1 -- 172.21.15.32:0/31518177 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+11978 (secure 0 0 0) 0x7fb9480af820 con 0x7fb9501141f0 2024-08-31T07:36:58.887 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:36:58.887 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":95,"btime":"2024-08-31T07:36:58:498094+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":95,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:36:58.455142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24734,24694,24718,24692,24676]},"id":1}]} 2024-08-31T07:36:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 95 2024-08-31T07:36:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.890+0000 7fb943400640 1 -- 172.21.15.32:0/31518177 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9380b4e70 msgr2=0x7fb9380b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.890+0000 7fb943400640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9380b4e70 0x7fb9380b7380 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb944009bd0 tx=0x7fb944005f90 comp rx=0 tx=0).stop 2024-08-31T07:36:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.890+0000 7fb943400640 1 -- 172.21.15.32:0/31518177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9501141f0 msgr2=0x7fb9501a60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:36:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.890+0000 7fb943400640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9501141f0 0x7fb9501a60d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb94800b970 tx=0x7fb94800be30 comp rx=0 tx=0).stop 2024-08-31T07:36:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.890+0000 7fb943400640 1 -- 172.21.15.32:0/31518177 shutdown_connections 2024-08-31T07:36:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.890+0000 7fb943400640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb9380b4e70 0x7fb9380b7380 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.890+0000 7fb943400640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9501141f0 0x7fb9501a60d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.890+0000 7fb943400640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb950077120 0x7fb9501a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.890+0000 7fb943400640 1 --2- 172.21.15.32:0/31518177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb950075b70 0x7fb9501a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:36:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.890+0000 7fb943400640 1 -- 172.21.15.32:0/31518177 >> 172.21.15.32:0/31518177 conn(0x7fb9500ff3b0 msgr2=0x7fb9501100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:36:58.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.891+0000 7fb943400640 1 -- 172.21.15.32:0/31518177 shutdown_connections 2024-08-31T07:36:58.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:36:58.891+0000 7fb943400640 1 -- 172.21.15.32:0/31518177 wait complete. 2024-08-31T07:36:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:58 smithi073 ceph-mon[21995]: pgmap v1130: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 526 KiB/s wr, 38 op/s 2024-08-31T07:36:58.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:58 smithi084 ceph-mon[22387]: pgmap v1130: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 526 KiB/s wr, 38 op/s 2024-08-31T07:36:59.432 INFO:tasks.workunit.client.0.smithi032.stdout: 10 179509 13.45 MB/sec execute 519 sec latency 79.596 ms 2024-08-31T07:36:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:59 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:59 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/31518177' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:36:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:36:59 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:36:59.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-31T07:36:59.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:59 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:59.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:59 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/31518177' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:36:59.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:36:59 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:36:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:59 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:36:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:59 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/31518177' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:36:59.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:36:59 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:37:00.432 INFO:tasks.workunit.client.0.smithi032.stdout: 10 180000 13.46 MB/sec execute 520 sec latency 325.843 ms 2024-08-31T07:37:00.502 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:00.830 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:00 smithi073 ceph-mon[21995]: pgmap v1131: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-08-31T07:37:00.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:00 smithi032 ceph-mon[17252]: pgmap v1131: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-08-31T07:37:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:00 smithi084 ceph-mon[22387]: pgmap v1131: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-08-31T07:37:01.432 INFO:tasks.workunit.client.0.smithi032.stdout: 10 180270 13.45 MB/sec execute 521 sec latency 286.013 ms 2024-08-31T07:37:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.552+0000 7fa1b5000640 1 -- 172.21.15.32:0/3373756237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b0102350 msgr2=0x7fa1b0102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.552+0000 7fa1b5000640 1 --2- 172.21.15.32:0/3373756237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b0102350 0x7fa1b0102760 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009890 tx=0x7fa1a002a370 comp rx=0 tx=0).stop 2024-08-31T07:37:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.553+0000 7fa1b5000640 1 -- 172.21.15.32:0/3373756237 shutdown_connections 2024-08-31T07:37:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.553+0000 7fa1b5000640 1 --2- 172.21.15.32:0/3373756237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0107790 0x7fa1b0107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.553+0000 7fa1b5000640 1 --2- 172.21.15.32:0/3373756237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b0102ca0 0x7fa1b01071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.553+0000 7fa1b5000640 1 --2- 172.21.15.32:0/3373756237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b0102350 0x7fa1b0102760 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.553+0000 7fa1b5000640 1 -- 172.21.15.32:0/3373756237 >> 172.21.15.32:0/3373756237 conn(0x7fa1b00fe060 msgr2=0x7fa1b01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.553+0000 7fa1b5000640 1 -- 172.21.15.32:0/3373756237 shutdown_connections 2024-08-31T07:37:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.554+0000 7fa1b5000640 1 -- 172.21.15.32:0/3373756237 wait complete. 2024-08-31T07:37:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.555+0000 7fa1b5000640 1 Processor -- start 2024-08-31T07:37:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.556+0000 7fa1b5000640 1 -- start start 2024-08-31T07:37:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.557+0000 7fa1b5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b0102350 0x7fa1b01a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.557+0000 7fa1b5000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b0102ca0 0x7fa1b01a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.557+0000 7fa1b5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0107790 0x7fa1b01a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.557+0000 7fa1b5000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0119c30 con 0x7fa1b0102350 2024-08-31T07:37:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.557+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b0102ca0 0x7fa1b01a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.557+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b0102ca0 0x7fa1b01a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50948/0 (socket says 172.21.15.32:50948) 2024-08-31T07:37:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.557+0000 7fa1aea00640 1 -- 172.21.15.32:0/1933696801 learned_addr learned my addr 172.21.15.32:0/1933696801 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.557+0000 7fa1b5000640 1 -- 172.21.15.32:0/1933696801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0119ab0 con 0x7fa1b0107790 2024-08-31T07:37:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.557+0000 7fa1b5000640 1 -- 172.21.15.32:0/1933696801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0119db0 con 0x7fa1b0102ca0 2024-08-31T07:37:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.557+0000 7fa1af400640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b0102350 0x7fa1b01a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.558+0000 7fa1afe00640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0107790 0x7fa1b01a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.558+0000 7fa1aea00640 1 -- 172.21.15.32:0/1933696801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0107790 msgr2=0x7fa1b01a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.558+0000 7fa1aea00640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0107790 0x7fa1b01a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.558+0000 7fa1aea00640 1 -- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b0102350 msgr2=0x7fa1b01a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.558+0000 7fa1aea00640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b0102350 0x7fa1b01a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.558+0000 7fa1aea00640 1 -- 172.21.15.32:0/1933696801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b01a6570 con 0x7fa1b0102ca0 2024-08-31T07:37:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.558+0000 7fa1af400640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b0102350 0x7fa1b01a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.558+0000 7fa1afe00640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0107790 0x7fa1b01a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:37:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.559+0000 7fa1aea00640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b0102ca0 0x7fa1b01a0c60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0058c0 tx=0x7fa19c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.559+0000 7fa1a7e00640 1 -- 172.21.15.32:0/1933696801 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00cd30 con 0x7fa1b0102ca0 2024-08-31T07:37:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.559+0000 7fa1b5000640 1 -- 172.21.15.32:0/1933696801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b01a6860 con 0x7fa1b0102ca0 2024-08-31T07:37:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.559+0000 7fa1a7e00640 1 -- 172.21.15.32:0/1933696801 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c0035f0 con 0x7fa1b0102ca0 2024-08-31T07:37:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.559+0000 7fa1a7e00640 1 -- 172.21.15.32:0/1933696801 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0044e0 con 0x7fa1b0102ca0 2024-08-31T07:37:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.560+0000 7fa1b5000640 1 -- 172.21.15.32:0/1933696801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b01ad080 con 0x7fa1b0102ca0 2024-08-31T07:37:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.565+0000 7fa1a7e00640 1 -- 172.21.15.32:0/1933696801 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa19c004030 con 0x7fa1b0102ca0 2024-08-31T07:37:01.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.566+0000 7fa1bd3c7740 1 Processor -- start 2024-08-31T07:37:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.567+0000 7fa1bd3c7740 1 -- start start 2024-08-31T07:37:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.567+0000 7fa1a7e00640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1940b4dd0 0x7fa1940b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.568+0000 7fa1a7e00640 1 -- 172.21.15.32:0/1933696801 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7fa19c0e5810 con 0x7fa1b0102ca0 2024-08-31T07:37:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.568+0000 7fa1afe00640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1940b4dd0 0x7fa1940b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.567+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188c9070 0x55d918c6eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.568+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a01b0 0x55d918c6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.568+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a8390 0x55d918c6f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.568+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d918be57b0 con 0x55d9188c9070 2024-08-31T07:37:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.568+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d918bd4d40 con 0x55d9188a8390 2024-08-31T07:37:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.568+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d918bd8cf0 con 0x55d9188a01b0 2024-08-31T07:37:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.569+0000 7fa1afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a8390 0x55d918c6f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.569+0000 7fa1afe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a8390 0x55d918c6f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37602/0 (socket says 172.21.15.32:37602) 2024-08-31T07:37:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.569+0000 7fa1afe00640 1 -- 172.21.15.32:0/1421573997 learned_addr learned my addr 172.21.15.32:0/1421573997 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.569+0000 7fa1aea00640 1 --2- 172.21.15.32:0/1421573997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a01b0 0x55d918c6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.569+0000 7fa1af400640 1 --2- 172.21.15.32:0/1421573997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188c9070 0x55d918c6eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.570+0000 7fa1aea00640 1 -- 172.21.15.32:0/1421573997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a8390 msgr2=0x55d918c6f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.570+0000 7fa1aea00640 1 --2- 172.21.15.32:0/1421573997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a8390 0x55d918c6f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.570+0000 7fa1aea00640 1 -- 172.21.15.32:0/1421573997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188c9070 msgr2=0x55d918c6eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.570+0000 7fa1aea00640 1 --2- 172.21.15.32:0/1421573997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188c9070 0x55d918c6eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.570+0000 7fa1aea00640 1 -- 172.21.15.32:0/1421573997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d918934650 con 0x55d9188a01b0 2024-08-31T07:37:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.570+0000 7fa1af400640 1 --2- 172.21.15.32:0/1421573997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188c9070 0x55d918c6eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.571+0000 7fa1aea00640 1 --2- 172.21.15.32:0/1421573997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a01b0 0x55d918c6f0a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0e2f50 tx=0x7fa19c004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a041bf47f7e592bd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.571+0000 7fa1afe00640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1940b4dd0 0x7fa1940b72e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8007fd0 tx=0x7fa1a8012b30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.571+0000 7fa1a4c00640 1 -- 172.21.15.32:0/1421573997 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0aba00 con 0x55d9188a01b0 2024-08-31T07:37:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.571+0000 7fa1a4c00640 1 -- 172.21.15.32:0/1421573997 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c0ac5f0 con 0x55d9188a01b0 2024-08-31T07:37:01.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.571+0000 7fa1a4c00640 1 -- 172.21.15.32:0/1421573997 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0e31d0 con 0x55d9188a01b0 2024-08-31T07:37:01.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.573+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/1421573997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a01b0 msgr2=0x55d918c6f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:01.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.573+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/1421573997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a01b0 0x55d918c6f0a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0e2f50 tx=0x7fa19c004320 comp rx=0 tx=0).stop 2024-08-31T07:37:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.573+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/1421573997 shutdown_connections 2024-08-31T07:37:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.573+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/1421573997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a8390 0x55d918c6f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.573+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/1421573997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a01b0 0x55d918c6f0a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.573+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/1421573997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188c9070 0x55d918c6eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.573+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/1421573997 >> 172.21.15.32:0/1421573997 conn(0x55d9188bd460 msgr2=0x55d918c6ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.574+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/1421573997 shutdown_connections 2024-08-31T07:37:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.575+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/1421573997 wait complete. 2024-08-31T07:37:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.576+0000 7fa1bd3c7740 1 -- start start 2024-08-31T07:37:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.576+0000 7fa1bd3c7740 1 Processor -- start 2024-08-31T07:37:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.577+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188a01b0 0x55d918c7a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.577+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918c7ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.577+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188c9070 0x55d918c86630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.577+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d918a37700 con 0x55d9188a01b0 2024-08-31T07:37:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.577+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d918b5bcf0 con 0x55d9188c9070 2024-08-31T07:37:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.577+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d918c7d340 con 0x55d9188a8390 2024-08-31T07:37:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.577+0000 7fa1bd3c7740 10 client.0 _ll_register_callbacks cb 0x55d918b81470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.577+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918c7ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.577+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918c7ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50970/0 (socket says 172.21.15.32:50970) 2024-08-31T07:37:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.577+0000 7fa1aea00640 1 -- 172.21.15.32:0/2682721599 learned_addr learned my addr 172.21.15.32:0/2682721599 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.578+0000 7fa1aea00640 1 -- 172.21.15.32:0/2682721599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188c9070 msgr2=0x55d918c86630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:37:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.578+0000 7fa1aea00640 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188c9070 0x55d918c86630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.578+0000 7fa1aea00640 1 -- 172.21.15.32:0/2682721599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188a01b0 msgr2=0x55d918c7a7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:37:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.578+0000 7fa1aea00640 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188a01b0 0x55d918c7a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.578+0000 7fa1aea00640 1 -- 172.21.15.32:0/2682721599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d918b9c8b0 con 0x55d9188a8390 2024-08-31T07:37:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.578+0000 7fa1aea00640 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918c7ad60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0abe80 tx=0x7fa19c0e3480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56c3128ce70f823 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.578+0000 7fa199600640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:37:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.579+0000 7fa199600640 1 -- 172.21.15.32:0/2682721599 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c097070 con 0x55d9188a8390 2024-08-31T07:37:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.579+0000 7fa199600640 1 -- 172.21.15.32:0/2682721599 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7fa19c093320 con 0x55d9188a8390 2024-08-31T07:37:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.579+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2682721599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19c0ac5f0 con 0x55d9188a8390 2024-08-31T07:37:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.579+0000 7fa199600640 1 -- 172.21.15.32:0/2682721599 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fa19c0aba00 con 0x55d9188a8390 2024-08-31T07:37:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.579+0000 7fa1bd3c7740 10 client.30016 fetch_fsmap learned FSMap version 95 2024-08-31T07:37:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.579+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2682721599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x55d918c9a3f0 con 0x55d9188a8390 2024-08-31T07:37:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.580+0000 7fa199600640 1 -- 172.21.15.32:0/2682721599 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c0ac5f0 con 0x55d9188a8390 2024-08-31T07:37:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.580+0000 7fa199600640 1 -- 172.21.15.32:0/2682721599 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 95) ==== 8406+0+0 (secure 0 0 0) 0x7fa19c091ce0 con 0x55d9188a8390 2024-08-31T07:37:01.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.580+0000 7fa1bd3c7740 10 client.30016 fetch_fsmap finished waiting for FSMap version 95 2024-08-31T07:37:01.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.580+0000 7fa1bd3c7740 10 client.30016 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:37:01.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.580+0000 7fa1bd3c7740 20 client.30016 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:01.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.580+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d9188879f0 0x55d918c9d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.580+0000 7fa1bd3c7740 4 client.30016 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:37:01.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.580+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2682721599 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d918be7700 con 0x55d9188879f0 2024-08-31T07:37:01.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.581+0000 7fa1af400640 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d9188879f0 0x55d918c9d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.581+0000 7fa199600640 10 client.30016 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:37:01.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.583+0000 7fa1af400640 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d9188879f0 0x55d918c9d750 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7aff70bc4488d266 server_cookie=d7e53120b0688b2 in_seq=0 out_seq=0 2024-08-31T07:37:01.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.585+0000 7fa199600640 1 -- 172.21.15.32:0/2682721599 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d918be7700 con 0x55d9188879f0 2024-08-31T07:37:01.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.585+0000 7fa199600640 10 client.30016 handle_command_reply: tid=0 2024-08-31T07:37:01.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.585+0000 7fa1bd3c7740 1 client.30016 shutdown 2024-08-31T07:37:01.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.586+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2682721599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 msgr2=0x55d918c7ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:01.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.586+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918c7ad60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0abe80 tx=0x7fa19c0e3480 comp rx=0 tx=0).stop 2024-08-31T07:37:01.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.586+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2682721599 shutdown_connections 2024-08-31T07:37:01.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.586+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d9188879f0 0x55d918c9d750 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.586+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188c9070 0x55d918c86630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.586+0000 7fa199600640 0 client.30016 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:37:01.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.586+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918c7ad60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.587+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2682721599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188a01b0 0x55d918c7a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.587+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2682721599 >> 172.21.15.32:0/2682721599 conn(0x55d9188bd460 msgr2=0x55d918c71d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:01.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.587+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2682721599 shutdown_connections 2024-08-31T07:37:01.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.587+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2682721599 wait complete. 2024-08-31T07:37:01.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.587+0000 7fa1bd3c7740 20 client.30016 trim_cache size 0 max 16384 2024-08-31T07:37:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.618+0000 7fa1bd3c7740 1 Processor -- start 2024-08-31T07:37:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.620+0000 7fa1bd3c7740 1 -- start start 2024-08-31T07:37:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.620+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188879f0 0x55d918c73c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.620+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 0x55d918c741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.620+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188a8390 0x55d918c6c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.621+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b00750b0 con 0x55d9188a8390 2024-08-31T07:37:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.621+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01aed20 con 0x55d9188a01b0 2024-08-31T07:37:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.621+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01ad760 con 0x55d9188879f0 2024-08-31T07:37:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.621+0000 7fa1af400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188879f0 0x55d918c73c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.621+0000 7fa1af400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188879f0 0x55d918c73c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50986/0 (socket says 172.21.15.32:50986) 2024-08-31T07:37:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.621+0000 7fa1af400640 1 -- 172.21.15.32:0/3422004934 learned_addr learned my addr 172.21.15.32:0/3422004934 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:01.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.621+0000 7fa1afe00640 1 --2- 172.21.15.32:0/3422004934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188a8390 0x55d918c6c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.622+0000 7fa1af400640 1 -- 172.21.15.32:0/3422004934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 msgr2=0x55d918c741b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T07:37:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.622+0000 7fa1af400640 1 --2- 172.21.15.32:0/3422004934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 0x55d918c741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.622+0000 7fa1af400640 1 -- 172.21.15.32:0/3422004934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188a8390 msgr2=0x55d918c6c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.622+0000 7fa1af400640 1 --2- 172.21.15.32:0/3422004934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188a8390 0x55d918c6c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.622+0000 7fa1af400640 1 -- 172.21.15.32:0/3422004934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d918c719c0 con 0x55d9188879f0 2024-08-31T07:37:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.622+0000 7fa1afe00640 1 --2- 172.21.15.32:0/3422004934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188a8390 0x55d918c6c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:37:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.622+0000 7fa1af400640 1 --2- 172.21.15.32:0/3422004934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188879f0 0x55d918c73c70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00057c0 tx=0x7fa1a0009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80fb3212533dff0c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.623+0000 7fa19be00640 1 -- 172.21.15.32:0/3422004934 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0037040 con 0x55d9188879f0 2024-08-31T07:37:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.623+0000 7fa19be00640 1 -- 172.21.15.32:0/3422004934 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a0038040 con 0x55d9188879f0 2024-08-31T07:37:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.623+0000 7fa19be00640 1 -- 172.21.15.32:0/3422004934 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0032cc0 con 0x55d9188879f0 2024-08-31T07:37:01.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.624+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/3422004934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188879f0 msgr2=0x55d918c73c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:01.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.624+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/3422004934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188879f0 0x55d918c73c70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00057c0 tx=0x7fa1a0009760 comp rx=0 tx=0).stop 2024-08-31T07:37:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.627+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/3422004934 shutdown_connections 2024-08-31T07:37:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.627+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/3422004934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188a8390 0x55d918c6c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.627+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/3422004934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 0x55d918c741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.627+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/3422004934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188879f0 0x55d918c73c70 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.627+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/3422004934 >> 172.21.15.32:0/3422004934 conn(0x55d9188bd460 msgr2=0x55d918c78e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.627+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/3422004934 shutdown_connections 2024-08-31T07:37:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.627+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/3422004934 wait complete. 2024-08-31T07:37:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.628+0000 7fa1bd3c7740 1 -- start start 2024-08-31T07:37:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.628+0000 7fa1bd3c7740 1 Processor -- start 2024-08-31T07:37:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188879f0 0x55d918c7e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 0x55d918c7edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1bd3c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918d06ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188879f0 0x55d918c7e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 0x55d918c7edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1aea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 0x55d918c7edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37632/0 (socket says 172.21.15.32:37632) 2024-08-31T07:37:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188879f0 0x55d918c7e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49350/0 (socket says 172.21.15.32:49350) 2024-08-31T07:37:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1afe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918d06ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1afe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918d06ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50990/0 (socket says 172.21.15.32:50990) 2024-08-31T07:37:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01ae5c0 con 0x55d9188879f0 2024-08-31T07:37:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01ade60 con 0x55d9188a01b0 2024-08-31T07:37:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1bd3c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01071d0 con 0x55d9188a8390 2024-08-31T07:37:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1bd3c7740 10 client.0 _ll_register_callbacks cb 0x7fa19c0ab880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.629+0000 7fa1aea00640 1 -- 172.21.15.32:0/2998756628 learned_addr learned my addr 172.21.15.32:0/2998756628 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:01.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.633+0000 7fa1aea00640 1 -- 172.21.15.32:0/2998756628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 msgr2=0x55d918d06ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:01.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.633+0000 7fa1aea00640 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918d06ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.633+0000 7fa1aea00640 1 -- 172.21.15.32:0/2998756628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188879f0 msgr2=0x55d918c7e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:01.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.633+0000 7fa1aea00640 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188879f0 0x55d918c7e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.633+0000 7fa1aea00640 1 -- 172.21.15.32:0/2998756628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d918c7f660 con 0x55d9188a01b0 2024-08-31T07:37:01.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.633+0000 7fa1afe00640 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918d06ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.633+0000 7fa1aea00640 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 0x55d918c7edc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa19c005ab0 tx=0x7fa19c0b0560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a57db265a8a238c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.634+0000 7fa199600640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:37:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.634+0000 7fa199600640 1 -- 172.21.15.32:0/2998756628 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c097040 con 0x55d9188a01b0 2024-08-31T07:37:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.634+0000 7fa199600640 1 -- 172.21.15.32:0/2998756628 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7fa19c0971e0 con 0x55d9188a01b0 2024-08-31T07:37:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.634+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2998756628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a0038040 con 0x55d9188a01b0 2024-08-31T07:37:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.634+0000 7fa199600640 1 -- 172.21.15.32:0/2998756628 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fa19c0e33e0 con 0x55d9188a01b0 2024-08-31T07:37:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.634+0000 7fa1bd3c7740 10 client.30017 fetch_fsmap learned FSMap version 95 2024-08-31T07:37:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.634+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2998756628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x55d918d08480 con 0x55d9188a01b0 2024-08-31T07:37:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.635+0000 7fa199600640 1 -- 172.21.15.32:0/2998756628 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a0038040 con 0x55d9188a01b0 2024-08-31T07:37:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.635+0000 7fa199600640 1 -- 172.21.15.32:0/2998756628 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 95) ==== 8406+0+0 (secure 0 0 0) 0x7fa19c091ce0 con 0x55d9188a01b0 2024-08-31T07:37:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.635+0000 7fa1bd3c7740 10 client.30017 fetch_fsmap finished waiting for FSMap version 95 2024-08-31T07:37:01.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.635+0000 7fa1bd3c7740 10 client.30017 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-08-31T07:37:01.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.636+0000 7fa1bd3c7740 20 client.30017 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:01.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.636+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d9188c9070 0x55d918d0a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:01.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.636+0000 7fa1bd3c7740 4 client.30017 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:37:01.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.636+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2998756628 --> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d918be31a0 con 0x55d9188c9070 2024-08-31T07:37:01.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.638+0000 7fa1af400640 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d9188c9070 0x55d918d0a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:01.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.638+0000 7fa1af400640 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d9188c9070 0x55d918d0a8d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8fa44af817dcf292 server_cookie=ec5073c87a84d5fe in_seq=0 out_seq=0 2024-08-31T07:37:01.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.638+0000 7fa199600640 10 client.30017 ms_handle_connect on v2:172.21.15.84:6836/4208526312 2024-08-31T07:37:01.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.639+0000 7fa199600640 1 -- 172.21.15.32:0/2998756628 <== mds.4 v2:172.21.15.84:6836/4208526312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d918be31a0 con 0x55d9188c9070 2024-08-31T07:37:01.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.639+0000 7fa199600640 10 client.30017 handle_command_reply: tid=0 2024-08-31T07:37:01.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.639+0000 7fa1bd3c7740 1 client.30017 shutdown 2024-08-31T07:37:01.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.641+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2998756628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 msgr2=0x55d918c7edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:01.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.641+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 0x55d918c7edc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa19c005ab0 tx=0x7fa19c0b0560 comp rx=0 tx=0).stop 2024-08-31T07:37:01.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.641+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2998756628 shutdown_connections 2024-08-31T07:37:01.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.641+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.84:6836/4208526312,v1:172.21.15.84:6837/4208526312] conn(0x55d9188c9070 0x55d918d0a8d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.641+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d9188a8390 0x55d918d06ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.641+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9188a01b0 0x55d918c7edc0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.641+0000 7fa1bd3c7740 1 --2- 172.21.15.32:0/2998756628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9188879f0 0x55d918c7e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.641+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2998756628 >> 172.21.15.32:0/2998756628 conn(0x55d9188bd460 msgr2=0x55d918c73c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:01.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.641+0000 7fa199600640 0 client.30017 ms_handle_reset on v2:172.21.15.84:6836/4208526312 2024-08-31T07:37:01.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.642+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2998756628 shutdown_connections 2024-08-31T07:37:01.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.642+0000 7fa1bd3c7740 1 -- 172.21.15.32:0/2998756628 wait complete. 2024-08-31T07:37:01.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.642+0000 7fa1bd3c7740 20 client.30017 trim_cache size 0 max 16384 2024-08-31T07:37:01.645 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:37:01.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.643+0000 7fa1b5000640 1 -- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1940b4dd0 msgr2=0x7fa1940b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.643+0000 7fa1b5000640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1940b4dd0 0x7fa1940b72e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8007fd0 tx=0x7fa1a8012b30 comp rx=0 tx=0).stop 2024-08-31T07:37:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.643+0000 7fa1b5000640 1 -- 172.21.15.32:0/1933696801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b0102ca0 msgr2=0x7fa1b01a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.643+0000 7fa1b5000640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b0102ca0 0x7fa1b01a0c60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0058c0 tx=0x7fa19c00ee50 comp rx=0 tx=0).stop 2024-08-31T07:37:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.643+0000 7fa1b5000640 1 -- 172.21.15.32:0/1933696801 shutdown_connections 2024-08-31T07:37:01.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.643+0000 7fa1b5000640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fa1940b4dd0 0x7fa1940b72e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.643+0000 7fa1b5000640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0107790 0x7fa1b01a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.643+0000 7fa1b5000640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1b0102ca0 0x7fa1b01a0c60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.643+0000 7fa1b5000640 1 --2- 172.21.15.32:0/1933696801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1b0102350 0x7fa1b01a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:01.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.644+0000 7fa1b5000640 1 -- 172.21.15.32:0/1933696801 >> 172.21.15.32:0/1933696801 conn(0x7fa1b00fe060 msgr2=0x7fa1b01051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:01.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.644+0000 7fa1b5000640 1 -- 172.21.15.32:0/1933696801 shutdown_connections 2024-08-31T07:37:01.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:01.644+0000 7fa1b5000640 1 -- 172.21.15.32:0/1933696801 wait complete. 2024-08-31T07:37:01.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:37:01.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:37:01.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:37:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:37:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:37:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:37:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:37:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:37:01.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:37:02.432 INFO:tasks.workunit.client.0.smithi032.stdout: 10 180638 13.47 MB/sec execute 522 sec latency 281.539 ms 2024-08-31T07:37:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:02 smithi084 ceph-mon[22387]: pgmap v1132: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.2 MiB/s wr, 156 op/s 2024-08-31T07:37:03.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:02 smithi032 ceph-mon[17252]: pgmap v1132: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.2 MiB/s wr, 156 op/s 2024-08-31T07:37:03.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:02 smithi073 ceph-mon[21995]: pgmap v1132: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.2 MiB/s wr, 156 op/s 2024-08-31T07:37:03.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-31T07:37:03.432 INFO:tasks.workunit.client.0.smithi032.stdout: 10 181048 13.50 MB/sec execute 523 sec latency 156.055 ms 2024-08-31T07:37:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:04.432 INFO:tasks.workunit.client.0.smithi032.stdout: 10 181657 13.54 MB/sec execute 524 sec latency 129.216 ms 2024-08-31T07:37:04.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.765+0000 7f02cce00640 1 -- 172.21.15.32:0/3442411634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02c810bde0 msgr2=0x7f02c8112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.765+0000 7f02cce00640 1 --2- 172.21.15.32:0/3442411634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02c810bde0 0x7f02c8112610 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009ca0 tx=0x7f02c002a8c0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.766+0000 7f02cce00640 1 -- 172.21.15.32:0/3442411634 shutdown_connections 2024-08-31T07:37:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.766+0000 7f02cce00640 1 --2- 172.21.15.32:0/3442411634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02c810bde0 0x7f02c8112610 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.766+0000 7f02cce00640 1 --2- 172.21.15.32:0/3442411634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c810b490 0x7f02c810b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.766+0000 7f02cce00640 1 --2- 172.21.15.32:0/3442411634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c8105b50 0x7f02c8105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.766+0000 7f02cce00640 1 -- 172.21.15.32:0/3442411634 >> 172.21.15.32:0/3442411634 conn(0x7f02c8101580 msgr2=0x7f02c81039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.766+0000 7f02cce00640 1 -- 172.21.15.32:0/3442411634 shutdown_connections 2024-08-31T07:37:04.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.766+0000 7f02cce00640 1 -- 172.21.15.32:0/3442411634 wait complete. 2024-08-31T07:37:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.768+0000 7f02cce00640 1 Processor -- start 2024-08-31T07:37:04.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.769+0000 7f02cce00640 1 -- start start 2024-08-31T07:37:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.769+0000 7f02cce00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c8105b50 0x7f02c810fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.769+0000 7f02cce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c810b490 0x7f02c810ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.769+0000 7f02cce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02c810bde0 0x7f02c81104d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.769+0000 7f02cce00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8115b70 con 0x7f02c810b490 2024-08-31T07:37:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.769+0000 7f02cce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f02c81159f0 con 0x7f02c810bde0 2024-08-31T07:37:04.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.769+0000 7f02cce00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8115cf0 con 0x7f02c8105b50 2024-08-31T07:37:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.770+0000 7f02c6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02c810bde0 0x7f02c81104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.770+0000 7f02c6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02c810bde0 0x7f02c81104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37650/0 (socket says 172.21.15.32:37650) 2024-08-31T07:37:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.770+0000 7f02c6800640 1 -- 172.21.15.32:0/2749740238 learned_addr learned my addr 172.21.15.32:0/2749740238 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.770+0000 7f02c5400640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c810b490 0x7f02c810ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.771+0000 7f02c5e00640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c8105b50 0x7f02c810fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.771+0000 7f02c6800640 1 -- 172.21.15.32:0/2749740238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c8105b50 msgr2=0x7f02c810fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.771+0000 7f02c6800640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c8105b50 0x7f02c810fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.771+0000 7f02c6800640 1 -- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c810b490 msgr2=0x7f02c810ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.771+0000 7f02c6800640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c810b490 0x7f02c810ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.771+0000 7f02c6800640 1 -- 172.21.15.32:0/2749740238 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c81ac9e0 con 0x7f02c810bde0 2024-08-31T07:37:04.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.771+0000 7f02c5400640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c810b490 0x7f02c810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.772+0000 7f02c6800640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02c810bde0 0x7f02c81104d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009430 tx=0x7f02c0003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.772+0000 7f02bea00640 1 -- 172.21.15.32:0/2749740238 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0003d90 con 0x7f02c810bde0 2024-08-31T07:37:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.773+0000 7f02bea00640 1 -- 172.21.15.32:0/2749740238 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c00044f0 con 0x7f02c810bde0 2024-08-31T07:37:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.773+0000 7f02bea00640 1 -- 172.21.15.32:0/2749740238 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0030cd0 con 0x7f02c810bde0 2024-08-31T07:37:04.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.773+0000 7f02cce00640 1 -- 172.21.15.32:0/2749740238 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c81acc70 con 0x7f02c810bde0 2024-08-31T07:37:04.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.773+0000 7f02cce00640 1 -- 172.21.15.32:0/2749740238 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c81ad160 con 0x7f02c810bde0 2024-08-31T07:37:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.778+0000 7f02d3a8b740 1 Processor -- start 2024-08-31T07:37:04.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.780+0000 7f02bea00640 1 -- 172.21.15.32:0/2749740238 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f02c00046a0 con 0x7f02c810bde0 2024-08-31T07:37:04.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.782+0000 7f02d3a8b740 1 -- start start 2024-08-31T07:37:04.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.782+0000 7f02bea00640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f02ac0b4d20 0x7f02ac0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.782+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12ecf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.782+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12ecfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.782+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b09390 0x55fc12ed0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.782+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fc12e467b0 con 0x55fc12b09390 2024-08-31T07:37:04.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.782+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fc12e35d40 con 0x55fc12b011b0 2024-08-31T07:37:04.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.782+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fc12e39cf0 con 0x55fc12b2a070 2024-08-31T07:37:04.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.782+0000 7f02bea00640 1 -- 172.21.15.32:0/2749740238 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7f02c01067d0 con 0x7f02c810bde0 2024-08-31T07:37:04.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.783+0000 7f02c5400640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f02ac0b4d20 0x7f02ac0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.783+0000 7f02c5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12ecfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.783+0000 7f02c5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12ecfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37660/0 (socket says 172.21.15.32:37660) 2024-08-31T07:37:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.783+0000 7f02c5400640 1 -- 172.21.15.32:0/2113923722 learned_addr learned my addr 172.21.15.32:0/2113923722 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.784+0000 7f02c5400640 1 -- 172.21.15.32:0/2113923722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 msgr2=0x55fc12ecf990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:37:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.784+0000 7f02c5400640 1 --2- 172.21.15.32:0/2113923722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12ecf990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.784+0000 7f02c5400640 1 -- 172.21.15.32:0/2113923722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b09390 msgr2=0x55fc12ed0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.784+0000 7f02c5400640 1 --2- 172.21.15.32:0/2113923722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b09390 0x55fc12ed0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.784+0000 7f02c5400640 1 -- 172.21.15.32:0/2113923722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc12b95650 con 0x55fc12b011b0 2024-08-31T07:37:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.784+0000 7f02c5400640 1 --2- 172.21.15.32:0/2113923722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12ecfed0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f02b4013ad0 tx=0x7f02b4013f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9ca448435b1208 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.785+0000 7f02b3400640 1 -- 172.21.15.32:0/2113923722 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b400fb80 con 0x55fc12b011b0 2024-08-31T07:37:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.786+0000 7f02b3400640 1 -- 172.21.15.32:0/2113923722 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02b400fd20 con 0x55fc12b011b0 2024-08-31T07:37:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.786+0000 7f02b3400640 1 -- 172.21.15.32:0/2113923722 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b401f620 con 0x55fc12b011b0 2024-08-31T07:37:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.788+0000 7f02c5400640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f02ac0b4d20 0x7f02ac0b7230 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f02b4010fd0 tx=0x7f02b4010f20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.789+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2113923722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 msgr2=0x55fc12ecfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.789+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/2113923722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12ecfed0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f02b4013ad0 tx=0x7f02b4013f90 comp rx=0 tx=0).stop 2024-08-31T07:37:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.789+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2113923722 shutdown_connections 2024-08-31T07:37:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.789+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/2113923722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b09390 0x55fc12ed0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.789+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/2113923722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12ecfed0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.789+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/2113923722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12ecf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.790+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2113923722 >> 172.21.15.32:0/2113923722 conn(0x55fc12b1e460 msgr2=0x55fc12ecd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.790+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2113923722 shutdown_connections 2024-08-31T07:37:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.790+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2113923722 wait complete. 2024-08-31T07:37:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.791+0000 7f02d3a8b740 1 -- start start 2024-08-31T07:37:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.791+0000 7f02d3a8b740 1 Processor -- start 2024-08-31T07:37:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.793+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12edb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.793+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b09390 0x55fc12edba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.793+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12ee7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.793+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fc12c98700 con 0x55fc12b09390 2024-08-31T07:37:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.793+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fc12dbccf0 con 0x55fc12b011b0 2024-08-31T07:37:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.793+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fc12ede170 con 0x55fc12b2a070 2024-08-31T07:37:04.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.793+0000 7f02c6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12ee7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.793+0000 7f02c6800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12ee7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51028/0 (socket says 172.21.15.32:51028) 2024-08-31T07:37:04.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.793+0000 7f02c6800640 1 -- 172.21.15.32:0/3701089332 learned_addr learned my addr 172.21.15.32:0/3701089332 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:04.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.794+0000 7f02c5e00640 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12edb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.794+0000 7f02d3a8b740 10 client.0 _ll_register_callbacks cb 0x55fc12de2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:04.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.794+0000 7f02c6800640 1 -- 172.21.15.32:0/3701089332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 msgr2=0x55fc12edb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.794+0000 7f02c6800640 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12edb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.794+0000 7f02c6800640 1 -- 172.21.15.32:0/3701089332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b09390 msgr2=0x55fc12edba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.794+0000 7f02c6800640 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b09390 0x55fc12edba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.794+0000 7f02c6800640 1 -- 172.21.15.32:0/3701089332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc12dfd8b0 con 0x55fc12b2a070 2024-08-31T07:37:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.795+0000 7f02c6800640 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12ee7490 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f02c0103fd0 tx=0x7f02c00cc920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51a62f2fa464de9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.795+0000 7f02a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:37:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.795+0000 7f02a7e00640 1 -- 172.21.15.32:0/3701089332 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c00cd610 con 0x55fc12b2a070 2024-08-31T07:37:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.795+0000 7f02a7e00640 1 -- 172.21.15.32:0/3701089332 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7f02c0030e70 con 0x55fc12b2a070 2024-08-31T07:37:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.796+0000 7f02d3a8b740 1 -- 172.21.15.32:0/3701089332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02b400fd20 con 0x55fc12b2a070 2024-08-31T07:37:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.796+0000 7f02a7e00640 1 -- 172.21.15.32:0/3701089332 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f02c00ccf10 con 0x55fc12b2a070 2024-08-31T07:37:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.796+0000 7f02d3a8b740 10 client.30031 fetch_fsmap learned FSMap version 96 2024-08-31T07:37:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.796+0000 7f02d3a8b740 1 -- 172.21.15.32:0/3701089332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x55fc12eea1a0 con 0x55fc12b2a070 2024-08-31T07:37:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.797+0000 7f02a7e00640 1 -- 172.21.15.32:0/3701089332 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02b400fd20 con 0x55fc12b2a070 2024-08-31T07:37:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.798+0000 7f02a7e00640 1 -- 172.21.15.32:0/3701089332 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 96) ==== 8410+0+0 (secure 0 0 0) 0x7f02c0004b40 con 0x55fc12b2a070 2024-08-31T07:37:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.798+0000 7f02d3a8b740 10 client.30031 fetch_fsmap finished waiting for FSMap version 96 2024-08-31T07:37:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.798+0000 7f02d3a8b740 10 client.30031 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:37:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.798+0000 7f02d3a8b740 20 client.30031 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.798+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55fc12ae89f0 0x55fc12eed710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.798+0000 7f02d3a8b740 4 client.30031 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:37:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.798+0000 7f02d3a8b740 1 -- 172.21.15.32:0/3701089332 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc12e48700 con 0x55fc12ae89f0 2024-08-31T07:37:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.798+0000 7f02c5e00640 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55fc12ae89f0 0x55fc12eed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.799+0000 7f02c5e00640 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55fc12ae89f0 0x55fc12eed710 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d73a62b99768e361 server_cookie=62778dffa47b18 in_seq=0 out_seq=0 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.799+0000 7f02a7e00640 10 client.30031 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.800+0000 7f02a7e00640 1 -- 172.21.15.32:0/3701089332 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc12e48700 con 0x55fc12ae89f0 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.800+0000 7f02a7e00640 10 client.30031 handle_command_reply: tid=0 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.800+0000 7f02d3a8b740 1 client.30031 shutdown 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.800+0000 7f02d3a8b740 1 -- 172.21.15.32:0/3701089332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 msgr2=0x55fc12ee7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.800+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12ee7490 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f02c0103fd0 tx=0x7f02c00cc920 comp rx=0 tx=0).stop 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.800+0000 7f02d3a8b740 1 -- 172.21.15.32:0/3701089332 shutdown_connections 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.800+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55fc12ae89f0 0x55fc12eed710 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.800+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12ee7490 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.801+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b09390 0x55fc12edba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.801+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/3701089332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12edb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.801+0000 7f02d3a8b740 1 -- 172.21.15.32:0/3701089332 >> 172.21.15.32:0/3701089332 conn(0x55fc12b1e460 msgr2=0x55fc12ed2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:04.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.801+0000 7f02a7e00640 0 client.30031 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:37:04.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.801+0000 7f02d3a8b740 1 -- 172.21.15.32:0/3701089332 shutdown_connections 2024-08-31T07:37:04.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.801+0000 7f02d3a8b740 1 -- 172.21.15.32:0/3701089332 wait complete. 2024-08-31T07:37:04.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.801+0000 7f02d3a8b740 20 client.30031 trim_cache size 0 max 16384 2024-08-31T07:37:04.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.823+0000 7f02d3a8b740 1 Processor -- start 2024-08-31T07:37:04.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.823+0000 7f02d3a8b740 1 -- start start 2024-08-31T07:37:04.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.824+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b011b0 0x55fc12ed3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.824+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b09390 0x55fc12edcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.824+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b2a070 0x55fc12ecce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.824+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02c81b59a0 con 0x55fc12b011b0 2024-08-31T07:37:04.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.824+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f02c81b7720 con 0x55fc12b2a070 2024-08-31T07:37:04.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.824+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f02c81b6130 con 0x55fc12b09390 2024-08-31T07:37:04.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.824+0000 7f02c5400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b09390 0x55fc12edcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.825+0000 7f02c5400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b09390 0x55fc12edcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51040/0 (socket says 172.21.15.32:51040) 2024-08-31T07:37:04.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.825+0000 7f02c6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b2a070 0x55fc12ecce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.825+0000 7f02c6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b2a070 0x55fc12ecce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37680/0 (socket says 172.21.15.32:37680) 2024-08-31T07:37:04.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.825+0000 7f02c6800640 1 -- 172.21.15.32:0/1813293256 learned_addr learned my addr 172.21.15.32:0/1813293256 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:04.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.826+0000 7f02c5e00640 1 --2- 172.21.15.32:0/1813293256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b011b0 0x55fc12ed3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.826+0000 7f02c5400640 1 -- 172.21.15.32:0/1813293256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b2a070 msgr2=0x55fc12ecce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:04.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.826+0000 7f02c5400640 1 --2- 172.21.15.32:0/1813293256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b2a070 0x55fc12ecce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.826+0000 7f02c5400640 1 -- 172.21.15.32:0/1813293256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b011b0 msgr2=0x55fc12ed3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:04.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.826+0000 7f02c5400640 1 --2- 172.21.15.32:0/1813293256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b011b0 0x55fc12ed3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.826+0000 7f02c5400640 1 -- 172.21.15.32:0/1813293256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc12edc610 con 0x55fc12b09390 2024-08-31T07:37:04.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.827+0000 7f02c5e00640 1 --2- 172.21.15.32:0/1813293256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b011b0 0x55fc12ed3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:37:04.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.828+0000 7f02c5400640 1 --2- 172.21.15.32:0/1813293256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b09390 0x55fc12edcf80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x55fc12eddce0 tx=0x7f02b4024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c6220e0eb556cf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:04.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.829+0000 7f02b2a00640 1 -- 172.21.15.32:0/1813293256 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4009e60 con 0x55fc12b09390 2024-08-31T07:37:04.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.829+0000 7f02b2a00640 1 -- 172.21.15.32:0/1813293256 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02b40044f0 con 0x55fc12b09390 2024-08-31T07:37:04.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.829+0000 7f02b2a00640 1 -- 172.21.15.32:0/1813293256 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b400fb80 con 0x55fc12b09390 2024-08-31T07:37:04.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.831+0000 7f02d3a8b740 1 -- 172.21.15.32:0/1813293256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b09390 msgr2=0x55fc12edcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:04.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.831+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/1813293256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b09390 0x55fc12edcf80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x55fc12eddce0 tx=0x7f02b4024000 comp rx=0 tx=0).stop 2024-08-31T07:37:04.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.831+0000 7f02d3a8b740 1 -- 172.21.15.32:0/1813293256 shutdown_connections 2024-08-31T07:37:04.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.831+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/1813293256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b2a070 0x55fc12ecce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.831+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/1813293256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b09390 0x55fc12edcf80 secure :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0x55fc12eddce0 tx=0x7f02b4024000 comp rx=0 tx=0).stop 2024-08-31T07:37:04.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.831+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/1813293256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12b011b0 0x55fc12ed3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.831+0000 7f02d3a8b740 1 -- 172.21.15.32:0/1813293256 >> 172.21.15.32:0/1813293256 conn(0x55fc12b1e460 msgr2=0x55fc12edc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:04.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.833+0000 7f02d3a8b740 1 -- 172.21.15.32:0/1813293256 shutdown_connections 2024-08-31T07:37:04.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.833+0000 7f02d3a8b740 1 -- 172.21.15.32:0/1813293256 wait complete. 2024-08-31T07:37:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.834+0000 7f02d3a8b740 1 -- start start 2024-08-31T07:37:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.834+0000 7f02d3a8b740 1 Processor -- start 2024-08-31T07:37:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.835+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12edf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.835+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12edfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.835+0000 7f02d3a8b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12abdda0 0x55fc12f68120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.835+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02c81b6fc0 con 0x55fc12abdda0 2024-08-31T07:37:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.835+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f02c81b6830 con 0x55fc12b011b0 2024-08-31T07:37:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.835+0000 7f02d3a8b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f02c81b39a0 con 0x55fc12b2a070 2024-08-31T07:37:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.835+0000 7f02d3a8b740 10 client.0 _ll_register_callbacks cb 0x7f02c0104420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.836+0000 7f02c5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12edf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.836+0000 7f02c5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12edf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:37696/0 (socket says 172.21.15.32:37696) 2024-08-31T07:37:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.836+0000 7f02c5e00640 1 -- 172.21.15.32:0/2959034143 learned_addr learned my addr 172.21.15.32:0/2959034143 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.836+0000 7f02c5400640 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12edfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.837+0000 7f02c6800640 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12abdda0 0x55fc12f68120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.837+0000 7f02c5e00640 1 -- 172.21.15.32:0/2959034143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 msgr2=0x55fc12edfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.837+0000 7f02c5e00640 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12edfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.837+0000 7f02c5e00640 1 -- 172.21.15.32:0/2959034143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12abdda0 msgr2=0x55fc12f68120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.837+0000 7f02c5e00640 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12abdda0 0x55fc12f68120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.837+0000 7f02c5e00640 1 -- 172.21.15.32:0/2959034143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc12ee0470 con 0x55fc12b011b0 2024-08-31T07:37:04.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.837+0000 7f02c6800640 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12abdda0 0x55fc12f68120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:37:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.837+0000 7f02c5e00640 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12edf6b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x55fc12eedeb0 tx=0x7f02b8018820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feeb85163532d3a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.837+0000 7f02a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:37:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.838+0000 7f02a7e00640 1 -- 172.21.15.32:0/2959034143 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b800f040 con 0x55fc12b011b0 2024-08-31T07:37:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.838+0000 7f02a7e00640 1 -- 172.21.15.32:0/2959034143 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8441+0+0 (secure 0 0 0) 0x7f02b8005bc0 con 0x55fc12b011b0 2024-08-31T07:37:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.838+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2959034143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02b40044f0 con 0x55fc12b011b0 2024-08-31T07:37:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.838+0000 7f02a7e00640 1 -- 172.21.15.32:0/2959034143 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f02b802c050 con 0x55fc12b011b0 2024-08-31T07:37:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.838+0000 7f02d3a8b740 10 client.30032 fetch_fsmap learned FSMap version 96 2024-08-31T07:37:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.838+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2959034143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x55fc12f69670 con 0x55fc12b011b0 2024-08-31T07:37:04.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.839+0000 7f02a7e00640 1 -- 172.21.15.32:0/2959034143 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02b40044f0 con 0x55fc12b011b0 2024-08-31T07:37:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.839+0000 7f02a7e00640 1 -- 172.21.15.32:0/2959034143 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 96) ==== 8410+0+0 (secure 0 0 0) 0x7f02b801d620 con 0x55fc12b011b0 2024-08-31T07:37:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.839+0000 7f02d3a8b740 10 client.30032 fetch_fsmap finished waiting for FSMap version 96 2024-08-31T07:37:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.840+0000 7f02d3a8b740 10 client.30032 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-08-31T07:37:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.840+0000 7f02d3a8b740 20 client.30032 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.840+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55fc12f69e20 0x55fc12f6c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.840+0000 7f02d3a8b740 4 client.30032 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:37:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.840+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2959034143 --> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc12e441a0 con 0x55fc12f69e20 2024-08-31T07:37:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.841+0000 7f02c5400640 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55fc12f69e20 0x55fc12f6c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.842+0000 7f02c5400640 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55fc12f69e20 0x55fc12f6c240 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=396871e90a7efa6 server_cookie=22f2fcc07193f6a5 in_seq=0 out_seq=0 2024-08-31T07:37:04.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.842+0000 7f02a7e00640 10 client.30032 ms_handle_connect on v2:172.21.15.73:6836/2735170682 2024-08-31T07:37:04.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.843+0000 7f02a7e00640 1 -- 172.21.15.32:0/2959034143 <== mds.3 v2:172.21.15.73:6836/2735170682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc12e441a0 con 0x55fc12f69e20 2024-08-31T07:37:04.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.843+0000 7f02a7e00640 10 client.30032 handle_command_reply: tid=0 2024-08-31T07:37:04.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.843+0000 7f02d3a8b740 1 client.30032 shutdown 2024-08-31T07:37:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.844+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2959034143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 msgr2=0x55fc12edf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.844+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12edf6b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x55fc12eedeb0 tx=0x7f02b8018820 comp rx=0 tx=0).stop 2024-08-31T07:37:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.844+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2959034143 shutdown_connections 2024-08-31T07:37:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.844+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.73:6836/2735170682,v1:172.21.15.73:6837/2735170682] conn(0x55fc12f69e20 0x55fc12f6c240 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.844+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fc12abdda0 0x55fc12f68120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.844+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fc12b2a070 0x55fc12edfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.844+0000 7f02d3a8b740 1 --2- 172.21.15.32:0/2959034143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc12b011b0 0x55fc12edf6b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.844+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2959034143 >> 172.21.15.32:0/2959034143 conn(0x55fc12b1e460 msgr2=0x55fc12ed1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.844+0000 7f02a7e00640 0 client.30032 ms_handle_reset on v2:172.21.15.73:6836/2735170682 2024-08-31T07:37:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.845+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2959034143 shutdown_connections 2024-08-31T07:37:04.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.846+0000 7f02d3a8b740 1 -- 172.21.15.32:0/2959034143 wait complete. 2024-08-31T07:37:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.846+0000 7f02d3a8b740 20 client.30032 trim_cache size 0 max 16384 2024-08-31T07:37:04.847 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:37:04.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 -- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f02ac0b4d20 msgr2=0x7f02ac0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f02ac0b4d20 0x7f02ac0b7230 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f02b4010fd0 tx=0x7f02b4010f20 comp rx=0 tx=0).stop 2024-08-31T07:37:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 -- 172.21.15.32:0/2749740238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02c810bde0 msgr2=0x7f02c81104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02c810bde0 0x7f02c81104d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009430 tx=0x7f02c0003970 comp rx=0 tx=0).stop 2024-08-31T07:37:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 -- 172.21.15.32:0/2749740238 shutdown_connections 2024-08-31T07:37:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f02ac0b4d20 0x7f02ac0b7230 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02c810bde0 0x7f02c81104d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c810b490 0x7f02c810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 --2- 172.21.15.32:0/2749740238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02c8105b50 0x7f02c810fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 -- 172.21.15.32:0/2749740238 >> 172.21.15.32:0/2749740238 conn(0x7f02c8101580 msgr2=0x7f02c810aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 -- 172.21.15.32:0/2749740238 shutdown_connections 2024-08-31T07:37:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:04.847+0000 7f02cce00640 1 -- 172.21.15.32:0/2749740238 wait complete. 2024-08-31T07:37:04.957 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:04 smithi032 ceph-mon[17252]: pgmap v1133: 97 pgs: 97 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2024-08-31T07:37:04.957 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:04 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:37:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:04 smithi084 ceph-mon[22387]: pgmap v1133: 97 pgs: 97 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2024-08-31T07:37:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:04 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:37:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:04 smithi073 ceph-mon[21995]: pgmap v1133: 97 pgs: 97 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2024-08-31T07:37:05.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:04 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:37:05.432 INFO:tasks.workunit.client.0.smithi032.stdout: 10 182156 13.57 MB/sec execute 525 sec latency 92.047 ms 2024-08-31T07:37:05.933 INFO:tasks.workunit.client.0.smithi032.stdout:[214345] write failed on handle 10415 (Resource temporarily unavailable) 2024-08-31T07:37:05.933 INFO:tasks.workunit.client.0.smithi032.stdout:Child failed with status 1 2024-08-31T07:37:05.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T07:37:05.957 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-08-31T07:37:05.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-31T07:37:06.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T07:37:06.632 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:06 smithi084 ceph-mon[22387]: pgmap v1134: 97 pgs: 97 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.8 MiB/s wr, 423 op/s 2024-08-31T07:37:06.992 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:06 smithi032 ceph-mon[17252]: pgmap v1134: 97 pgs: 97 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.8 MiB/s wr, 423 op/s 2024-08-31T07:37:07.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:06 smithi073 ceph-mon[21995]: pgmap v1134: 97 pgs: 97 active+clean; 2.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.8 MiB/s wr, 423 op/s 2024-08-31T07:37:07.206 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi032 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d19e6118f7015f0047264006cb0c873528eb3ae6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-08-31T07:37:07.396 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b829706239c641bdb5409b5c2e7e6a41 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi032 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d19e6118f7015f0047264006cb0c873528eb3ae6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-08-31T07:37:07.398 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-08-31T07:37:07.410 INFO:tasks.quiescer:joining Quiescers 2024-08-31T07:37:07.410 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-08-31T07:37:07.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:37:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.751+0000 7f61c5800640 1 -- 172.21.15.32:0/2209272825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61c0103910 msgr2=0x7f61c0103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.751+0000 7f61c5800640 1 --2- 172.21.15.32:0/2209272825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61c0103910 0x7f61c0103d20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009900 tx=0x7f61b002a410 comp rx=0 tx=0).stop 2024-08-31T07:37:07.752 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:07 smithi032 ceph-mon[17252]: osdmap e117: 12 total, 12 up, 12 in 2024-08-31T07:37:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.751+0000 7f61c5800640 1 -- 172.21.15.32:0/2209272825 shutdown_connections 2024-08-31T07:37:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.751+0000 7f61c5800640 1 --2- 172.21.15.32:0/2209272825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c0109ba0 0x7f61c01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.751+0000 7f61c5800640 1 --2- 172.21.15.32:0/2209272825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c0109250 0x7f61c0109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.751+0000 7f61c5800640 1 --2- 172.21.15.32:0/2209272825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61c0103910 0x7f61c0103d20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.751+0000 7f61c5800640 1 -- 172.21.15.32:0/2209272825 >> 172.21.15.32:0/2209272825 conn(0x7f61c00ff380 msgr2=0x7f61c01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.751+0000 7f61c5800640 1 -- 172.21.15.32:0/2209272825 shutdown_connections 2024-08-31T07:37:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.751+0000 7f61c5800640 1 -- 172.21.15.32:0/2209272825 wait complete. 2024-08-31T07:37:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.753+0000 7f61c5800640 1 Processor -- start 2024-08-31T07:37:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.753+0000 7f61c5800640 1 -- start start 2024-08-31T07:37:07.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.753+0000 7f61c5800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61c0103910 0x7f61c019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.753+0000 7f61c5800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c0109250 0x7f61c019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.753+0000 7f61c5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c0109ba0 0x7f61c01a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.753+0000 7f61c5800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01138b0 con 0x7f61c0103910 2024-08-31T07:37:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.753+0000 7f61c5800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0113730 con 0x7f61c0109ba0 2024-08-31T07:37:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.753+0000 7f61c5800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0113a30 con 0x7f61c0109250 2024-08-31T07:37:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.754+0000 7f61bea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61c0103910 0x7f61c019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.754+0000 7f61bea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61c0103910 0x7f61c019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40322/0 (socket says 172.21.15.32:40322) 2024-08-31T07:37:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.754+0000 7f61bea00640 1 -- 172.21.15.32:0/895020570 learned_addr learned my addr 172.21.15.32:0/895020570 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.754+0000 7f61be000640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c0109250 0x7f61c019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.754+0000 7f61bea00640 1 -- 172.21.15.32:0/895020570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c0109250 msgr2=0x7f61c019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.754+0000 7f61bea00640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c0109250 0x7f61c019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.754+0000 7f61bea00640 1 -- 172.21.15.32:0/895020570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c0109ba0 msgr2=0x7f61c01a1c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T07:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.754+0000 7f61bea00640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c0109ba0 0x7f61c01a1c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.754+0000 7f61bea00640 1 -- 172.21.15.32:0/895020570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c01a23b0 con 0x7f61c0103910 2024-08-31T07:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.755+0000 7f61bea00640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61c0103910 0x7f61c019c440 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f61b0005ab0 tx=0x7f61b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.755+0000 7f61b7400640 1 -- 172.21.15.32:0/895020570 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b003f070 con 0x7f61c0103910 2024-08-31T07:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.755+0000 7f61b7400640 1 -- 172.21.15.32:0/895020570 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b0031b60 con 0x7f61c0103910 2024-08-31T07:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.755+0000 7f61c5800640 1 -- 172.21.15.32:0/895020570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c01a2640 con 0x7f61c0103910 2024-08-31T07:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.756+0000 7f61c5800640 1 -- 172.21.15.32:0/895020570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01a8e00 con 0x7f61c0103910 2024-08-31T07:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.756+0000 7f61b7400640 1 -- 172.21.15.32:0/895020570 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0035030 con 0x7f61c0103910 2024-08-31T07:37:07.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.760+0000 7f61cc484740 1 Processor -- start 2024-08-31T07:37:07.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.760+0000 7f61cc484740 1 -- start start 2024-08-31T07:37:07.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.760+0000 7f61b7400640 1 -- 172.21.15.32:0/895020570 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f61b0042070 con 0x7f61c0103910 2024-08-31T07:37:07.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.760+0000 7f61cc484740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eeb070 0x5644f82908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.760+0000 7f61cc484740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7ec21b0 0x5644f8290df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.760+0000 7f61cc484740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eca390 0x5644f8291330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.760+0000 7f61cc484740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5644f82077b0 con 0x5644f7eeb070 2024-08-31T07:37:07.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.760+0000 7f61cc484740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5644f81f6d40 con 0x5644f7ec21b0 2024-08-31T07:37:07.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.760+0000 7f61cc484740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5644f81facf0 con 0x5644f7eca390 2024-08-31T07:37:07.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.761+0000 7f61be000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eca390 0x5644f8291330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.761+0000 7f61bea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7ec21b0 0x5644f8290df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.761+0000 7f61be000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eca390 0x5644f8291330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37128/0 (socket says 172.21.15.32:37128) 2024-08-31T07:37:07.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.761+0000 7f61be000640 1 -- 172.21.15.32:0/549305843 learned_addr learned my addr 172.21.15.32:0/549305843 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:07.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.761+0000 7f61bf400640 1 --2- 172.21.15.32:0/549305843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eeb070 0x5644f82908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.761+0000 7f61be000640 1 -- 172.21.15.32:0/549305843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7ec21b0 msgr2=0x5644f8290df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:07.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.761+0000 7f61be000640 1 --2- 172.21.15.32:0/549305843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7ec21b0 0x5644f8290df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.761+0000 7f61be000640 1 -- 172.21.15.32:0/549305843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eeb070 msgr2=0x5644f82908b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:07.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.761+0000 7f61be000640 1 --2- 172.21.15.32:0/549305843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eeb070 0x5644f82908b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.761+0000 7f61be000640 1 -- 172.21.15.32:0/549305843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f7f56650 con 0x5644f7eca390 2024-08-31T07:37:07.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.762+0000 7f61be000640 1 --2- 172.21.15.32:0/549305843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eca390 0x5644f8291330 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009c20 tx=0x7f61ac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2b45f2b94da1d48 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:07.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.762+0000 7f61abe00640 1 -- 172.21.15.32:0/549305843 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac004230 con 0x5644f7eca390 2024-08-31T07:37:07.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.762+0000 7f61abe00640 1 -- 172.21.15.32:0/549305843 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61ac0043d0 con 0x5644f7eca390 2024-08-31T07:37:07.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.762+0000 7f61abe00640 1 -- 172.21.15.32:0/549305843 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac004d50 con 0x5644f7eca390 2024-08-31T07:37:07.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.762+0000 7f61b7400640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61a40b4d20 0x7f61a40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.763+0000 7f61b7400640 1 -- 172.21.15.32:0/895020570 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7f61b0105990 con 0x7f61c0103910 2024-08-31T07:37:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.763+0000 7f61bf400640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61a40b4d20 0x7f61a40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.763+0000 7f61cc484740 1 -- 172.21.15.32:0/549305843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eca390 msgr2=0x5644f8291330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.763+0000 7f61cc484740 1 --2- 172.21.15.32:0/549305843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eca390 0x5644f8291330 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009c20 tx=0x7f61ac011810 comp rx=0 tx=0).stop 2024-08-31T07:37:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.763+0000 7f61cc484740 1 -- 172.21.15.32:0/549305843 shutdown_connections 2024-08-31T07:37:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.763+0000 7f61cc484740 1 --2- 172.21.15.32:0/549305843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eca390 0x5644f8291330 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.763+0000 7f61cc484740 1 --2- 172.21.15.32:0/549305843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7ec21b0 0x5644f8290df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.763+0000 7f61cc484740 1 --2- 172.21.15.32:0/549305843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eeb070 0x5644f82908b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.763+0000 7f61cc484740 1 -- 172.21.15.32:0/549305843 >> 172.21.15.32:0/549305843 conn(0x5644f7edf460 msgr2=0x5644f828e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.763+0000 7f61cc484740 1 -- 172.21.15.32:0/549305843 shutdown_connections 2024-08-31T07:37:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.764+0000 7f61cc484740 1 -- 172.21.15.32:0/549305843 wait complete. 2024-08-31T07:37:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.764+0000 7f61bf400640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61a40b4d20 0x7f61a40b7230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f61b8005fd0 tx=0x7f61b8009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.764+0000 7f61cc484740 1 -- start start 2024-08-31T07:37:07.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.764+0000 7f61cc484740 1 Processor -- start 2024-08-31T07:37:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61cc484740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 0x5644f829c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61cc484740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eca390 0x5644f829c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61cc484740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f82a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61cc484740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5644f8059700 con 0x5644f7eca390 2024-08-31T07:37:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61cc484740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5644f817dcf0 con 0x5644f7eeb070 2024-08-31T07:37:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61cc484740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5644f829f090 con 0x5644f7ec21b0 2024-08-31T07:37:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61cc484740 10 client.0 _ll_register_callbacks cb 0x5644f81a3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61bf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f82a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61be000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eca390 0x5644f829c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61bf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f82a83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52596/0 (socket says 172.21.15.32:52596) 2024-08-31T07:37:07.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61bf400640 1 -- 172.21.15.32:0/3614604623 learned_addr learned my addr 172.21.15.32:0/3614604623 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.765+0000 7f61bea00640 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 0x5644f829c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.766+0000 7f61bea00640 1 -- 172.21.15.32:0/3614604623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 msgr2=0x5644f82a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:07.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.766+0000 7f61bea00640 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f82a83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.766+0000 7f61bea00640 1 -- 172.21.15.32:0/3614604623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eca390 msgr2=0x5644f829c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.766+0000 7f61bea00640 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eca390 0x5644f829c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.766+0000 7f61bea00640 1 -- 172.21.15.32:0/3614604623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644f81be8b0 con 0x5644f7ec21b0 2024-08-31T07:37:07.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.766+0000 7f61bea00640 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 0x5644f829c460 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f61b0103100 tx=0x7f61b0038000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd50f89bc8a290f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:07.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.766+0000 7f61a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-08-31T07:37:07.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.766+0000 7f61a8c00640 1 -- 172.21.15.32:0/3614604623 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b00cb9f0 con 0x5644f7ec21b0 2024-08-31T07:37:07.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.766+0000 7f61a8c00640 1 -- 172.21.15.32:0/3614604623 <== mon.1 v2:172.21.15.84:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7f61b00cc300 con 0x5644f7ec21b0 2024-08-31T07:37:07.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.766+0000 7f61cc484740 1 -- 172.21.15.32:0/3614604623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61ac0043d0 con 0x5644f7ec21b0 2024-08-31T07:37:07.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61a8c00640 1 -- 172.21.15.32:0/3614604623 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f61b00cb810 con 0x5644f7ec21b0 2024-08-31T07:37:07.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61cc484740 10 client.30046 fetch_fsmap learned FSMap version 96 2024-08-31T07:37:07.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61cc484740 1 -- 172.21.15.32:0/3614604623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x5644f82ab0b0 con 0x5644f7ec21b0 2024-08-31T07:37:07.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61a8c00640 1 -- 172.21.15.32:0/3614604623 <== mon.1 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61ac0043d0 con 0x5644f7ec21b0 2024-08-31T07:37:07.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61a8c00640 1 -- 172.21.15.32:0/3614604623 <== mon.1 v2:172.21.15.84:3300/0 5 ==== fsmap(e 96) ==== 8410+0+0 (secure 0 0 0) 0x7f61b00d4f30 con 0x5644f7ec21b0 2024-08-31T07:37:07.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61cc484740 10 client.30046 fetch_fsmap finished waiting for FSMap version 96 2024-08-31T07:37:07.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61cc484740 10 client.30046 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:37:07.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61cc484740 20 client.30046 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:07.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61cc484740 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5644f7ea99f0 0x5644f82ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61cc484740 4 client.30046 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:37:07.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.767+0000 7f61cc484740 1 -- 172.21.15.32:0/3614604623 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644f8209700 con 0x5644f7ea99f0 2024-08-31T07:37:07.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.768+0000 7f61be000640 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5644f7ea99f0 0x5644f82ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.768+0000 7f61be000640 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5644f7ea99f0 0x5644f82ae6a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69be8cf393370d99 server_cookie=13c89b9adf9da1c0 in_seq=0 out_seq=0 2024-08-31T07:37:07.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.768+0000 7f61a8c00640 10 client.30046 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:37:07.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.769+0000 7f61a8c00640 1 -- 172.21.15.32:0/3614604623 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644f8209700 con 0x5644f7ea99f0 2024-08-31T07:37:07.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.769+0000 7f61a8c00640 10 client.30046 handle_command_reply: tid=0 2024-08-31T07:37:07.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.769+0000 7f61cc484740 1 client.30046 shutdown 2024-08-31T07:37:07.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.769+0000 7f61cc484740 1 -- 172.21.15.32:0/3614604623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 msgr2=0x5644f829c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:07.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.769+0000 7f61cc484740 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 0x5644f829c460 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f61b0103100 tx=0x7f61b0038000 comp rx=0 tx=0).stop 2024-08-31T07:37:07.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.770+0000 7f61cc484740 1 -- 172.21.15.32:0/3614604623 shutdown_connections 2024-08-31T07:37:07.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.770+0000 7f61cc484740 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5644f7ea99f0 0x5644f82ae6a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.770+0000 7f61cc484740 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f82a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.770+0000 7f61cc484740 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eca390 0x5644f829c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.770+0000 7f61cc484740 1 --2- 172.21.15.32:0/3614604623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 0x5644f829c460 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.770+0000 7f61a8c00640 0 client.30046 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:37:07.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.770+0000 7f61cc484740 1 -- 172.21.15.32:0/3614604623 >> 172.21.15.32:0/3614604623 conn(0x5644f7edf460 msgr2=0x5644f8293a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.770+0000 7f61cc484740 1 -- 172.21.15.32:0/3614604623 shutdown_connections 2024-08-31T07:37:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.770+0000 7f61cc484740 1 -- 172.21.15.32:0/3614604623 wait complete. 2024-08-31T07:37:07.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.770+0000 7f61cc484740 20 client.30046 trim_cache size 0 max 16384 2024-08-31T07:37:07.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.790+0000 7f61cc484740 1 Processor -- start 2024-08-31T07:37:07.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.790+0000 7f61cc484740 1 -- start start 2024-08-31T07:37:07.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.790+0000 7f61cc484740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7ec21b0 0x5644f82959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.790+0000 7f61cc484740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eca390 0x5644f8295f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.791+0000 7f61cc484740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eeb070 0x5644f828e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.791+0000 7f61cc484740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61a000e6f0 con 0x5644f7ec21b0 2024-08-31T07:37:07.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.791+0000 7f61cc484740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f61a0010470 con 0x5644f7eca390 2024-08-31T07:37:07.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.791+0000 7f61cc484740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61a000ee80 con 0x5644f7eeb070 2024-08-31T07:37:07.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.791+0000 7f61be000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eca390 0x5644f8295f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.791+0000 7f61bf400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eeb070 0x5644f828e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.791+0000 7f61bf400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eeb070 0x5644f828e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37152/0 (socket says 172.21.15.32:37152) 2024-08-31T07:37:07.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.791+0000 7f61be000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eca390 0x5644f8295f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52598/0 (socket says 172.21.15.32:52598) 2024-08-31T07:37:07.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.791+0000 7f61bf400640 1 -- 172.21.15.32:0/3130844485 learned_addr learned my addr 172.21.15.32:0/3130844485 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:07.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.791+0000 7f61bf400640 1 -- 172.21.15.32:0/3130844485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eca390 msgr2=0x5644f8295f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:07.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.792+0000 7f61bf400640 1 --2- 172.21.15.32:0/3130844485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eca390 0x5644f8295f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.792+0000 7f61bf400640 1 -- 172.21.15.32:0/3130844485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7ec21b0 msgr2=0x5644f82959c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T07:37:07.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.792+0000 7f61bea00640 1 --2- 172.21.15.32:0/3130844485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7ec21b0 0x5644f82959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.792+0000 7f61bf400640 1 --2- 172.21.15.32:0/3130844485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7ec21b0 0x5644f82959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.792+0000 7f61bf400640 1 -- 172.21.15.32:0/3130844485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f82a8720 con 0x5644f7eeb070 2024-08-31T07:37:07.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.792+0000 7f61bea00640 1 --2- 172.21.15.32:0/3130844485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7ec21b0 0x5644f82959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:37:07.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.793+0000 7f61bf400640 1 --2- 172.21.15.32:0/3130844485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eeb070 0x5644f828e030 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f61b800bdd0 tx=0x7f61b801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8b84399cfde753f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:07.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.793+0000 7f61ab400640 1 -- 172.21.15.32:0/3130844485 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8032070 con 0x5644f7eeb070 2024-08-31T07:37:07.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.793+0000 7f61ab400640 1 -- 172.21.15.32:0/3130844485 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b8003e20 con 0x5644f7eeb070 2024-08-31T07:37:07.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.793+0000 7f61ab400640 1 -- 172.21.15.32:0/3130844485 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b80043c0 con 0x5644f7eeb070 2024-08-31T07:37:07.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.794+0000 7f61cc484740 1 -- 172.21.15.32:0/3130844485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eeb070 msgr2=0x5644f828e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:07.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.794+0000 7f61cc484740 1 --2- 172.21.15.32:0/3130844485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eeb070 0x5644f828e030 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f61b800bdd0 tx=0x7f61b801e000 comp rx=0 tx=0).stop 2024-08-31T07:37:07.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.794+0000 7f61cc484740 1 -- 172.21.15.32:0/3130844485 shutdown_connections 2024-08-31T07:37:07.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.794+0000 7f61cc484740 1 --2- 172.21.15.32:0/3130844485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7eeb070 0x5644f828e030 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.794+0000 7f61cc484740 1 --2- 172.21.15.32:0/3130844485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eca390 0x5644f8295f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.794+0000 7f61cc484740 1 --2- 172.21.15.32:0/3130844485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7ec21b0 0x5644f82959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.794+0000 7f61cc484740 1 -- 172.21.15.32:0/3130844485 >> 172.21.15.32:0/3130844485 conn(0x5644f7edf460 msgr2=0x5644f829aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:07.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.795+0000 7f61cc484740 1 -- 172.21.15.32:0/3130844485 shutdown_connections 2024-08-31T07:37:07.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.795+0000 7f61cc484740 1 -- 172.21.15.32:0/3130844485 wait complete. 2024-08-31T07:37:07.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.795+0000 7f61cc484740 1 -- start start 2024-08-31T07:37:07.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.796+0000 7f61cc484740 1 Processor -- start 2024-08-31T07:37:07.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.796+0000 7f61cc484740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 0x5644f82a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.796+0000 7f61cc484740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eca390 0x5644f82a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.796+0000 7f61cc484740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f829f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.796+0000 7f61cc484740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61a000fd10 con 0x5644f7eca390 2024-08-31T07:37:07.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.796+0000 7f61cc484740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f61a000f580 con 0x5644f7eeb070 2024-08-31T07:37:07.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.796+0000 7f61cc484740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61a000c6f0 con 0x5644f7ec21b0 2024-08-31T07:37:07.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.796+0000 7f61cc484740 10 client.0 _ll_register_callbacks cb 0x7f61b0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:07.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61bf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f829f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61bf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f829f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52612/0 (socket says 172.21.15.32:52612) 2024-08-31T07:37:07.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61bf400640 1 -- 172.21.15.32:0/3256229205 learned_addr learned my addr 172.21.15.32:0/3256229205 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:07.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61bea00640 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 0x5644f82a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61bf400640 1 -- 172.21.15.32:0/3256229205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 msgr2=0x5644f82a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:07.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61bf400640 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 0x5644f82a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61bf400640 1 -- 172.21.15.32:0/3256229205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eca390 msgr2=0x5644f82a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61bf400640 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eca390 0x5644f82a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61bf400640 1 -- 172.21.15.32:0/3256229205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644f829f840 con 0x5644f7eeb070 2024-08-31T07:37:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61bf400640 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f829f300 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f61b8003b80 tx=0x7f61b8005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a53f208191e460a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:37:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.797+0000 7f61a8c00640 1 -- 172.21.15.32:0/3256229205 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b803b060 con 0x5644f7eeb070 2024-08-31T07:37:07.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.798+0000 7f61a8c00640 1 -- 172.21.15.32:0/3256229205 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7f61b8022b30 con 0x5644f7eeb070 2024-08-31T07:37:07.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.798+0000 7f61cc484740 1 -- 172.21.15.32:0/3256229205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b8003e20 con 0x5644f7eeb070 2024-08-31T07:37:07.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.798+0000 7f61a8c00640 1 -- 172.21.15.32:0/3256229205 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f61b8032040 con 0x5644f7eeb070 2024-08-31T07:37:07.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.798+0000 7f61cc484740 10 client.30047 fetch_fsmap learned FSMap version 96 2024-08-31T07:37:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.798+0000 7f61cc484740 1 -- 172.21.15.32:0/3256229205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x5644f8319670 con 0x5644f7eeb070 2024-08-31T07:37:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.798+0000 7f61a8c00640 1 -- 172.21.15.32:0/3256229205 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b8003e20 con 0x5644f7eeb070 2024-08-31T07:37:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.798+0000 7f61a8c00640 1 -- 172.21.15.32:0/3256229205 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 96) ==== 8410+0+0 (secure 0 0 0) 0x7f61b8035070 con 0x5644f7eeb070 2024-08-31T07:37:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.799+0000 7f61cc484740 10 client.30047 fetch_fsmap finished waiting for FSMap version 96 2024-08-31T07:37:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.799+0000 7f61cc484740 10 client.30047 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T07:37:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.799+0000 7f61cc484740 20 client.30047 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.799+0000 7f61cc484740 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5644f7e7eda0 0x5644f831bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.799+0000 7f61cc484740 4 client.30047 mds_command: new command op to 24694 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:37:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.799+0000 7f61cc484740 1 -- 172.21.15.32:0/3256229205 --> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644f82051a0 con 0x5644f7e7eda0 2024-08-31T07:37:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.799+0000 7f61bea00640 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5644f7e7eda0 0x5644f831bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.800+0000 7f61bea00640 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5644f7e7eda0 0x5644f831bd60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=419618aac405764 server_cookie=a8b3b904bba38309 in_seq=0 out_seq=0 2024-08-31T07:37:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.800+0000 7f61a8c00640 10 client.30047 ms_handle_connect on v2:172.21.15.84:6838/2807364599 2024-08-31T07:37:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.800+0000 7f61a8c00640 1 -- 172.21.15.32:0/3256229205 <== mds.2 v2:172.21.15.84:6838/2807364599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644f82051a0 con 0x5644f7e7eda0 2024-08-31T07:37:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.800+0000 7f61a8c00640 10 client.30047 handle_command_reply: tid=0 2024-08-31T07:37:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.800+0000 7f61cc484740 1 client.30047 shutdown 2024-08-31T07:37:07.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 1 -- 172.21.15.32:0/3256229205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 msgr2=0x5644f829f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:07.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f829f300 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f61b8003b80 tx=0x7f61b8005ac0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 1 -- 172.21.15.32:0/3256229205 shutdown_connections 2024-08-31T07:37:07.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.84:6838/2807364599,v1:172.21.15.84:6839/2807364599] conn(0x5644f7e7eda0 0x5644f831bd60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644f7eeb070 0x5644f829f300 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5644f7eca390 0x5644f82a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 1 --2- 172.21.15.32:0/3256229205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5644f7ec21b0 0x5644f82a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61a8c00640 0 client.30047 ms_handle_reset on v2:172.21.15.84:6838/2807364599 2024-08-31T07:37:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 1 -- 172.21.15.32:0/3256229205 >> 172.21.15.32:0/3256229205 conn(0x5644f7edf460 msgr2=0x5644f82959c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 1 -- 172.21.15.32:0/3256229205 shutdown_connections 2024-08-31T07:37:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 1 -- 172.21.15.32:0/3256229205 wait complete. 2024-08-31T07:37:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.801+0000 7f61cc484740 20 client.30047 trim_cache size 0 max 16384 2024-08-31T07:37:07.818 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:37:07.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.802+0000 7f61c5800640 1 -- 172.21.15.32:0/895020570 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61a40b4d20 msgr2=0x7f61a40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.802+0000 7f61c5800640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61a40b4d20 0x7f61a40b7230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f61b8005fd0 tx=0x7f61b8009500 comp rx=0 tx=0).stop 2024-08-31T07:37:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.802+0000 7f61c5800640 1 -- 172.21.15.32:0/895020570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61c0103910 msgr2=0x7f61c019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.802+0000 7f61c5800640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61c0103910 0x7f61c019c440 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f61b0005ab0 tx=0x7f61b0003c10 comp rx=0 tx=0).stop 2024-08-31T07:37:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.802+0000 7f61c5800640 1 -- 172.21.15.32:0/895020570 shutdown_connections 2024-08-31T07:37:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.802+0000 7f61c5800640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f61a40b4d20 0x7f61a40b7230 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.802+0000 7f61c5800640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c0109ba0 0x7f61c01a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.802+0000 7f61c5800640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61c0109250 0x7f61c019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.802+0000 7f61c5800640 1 --2- 172.21.15.32:0/895020570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61c0103910 0x7f61c019c440 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.802+0000 7f61c5800640 1 -- 172.21.15.32:0/895020570 >> 172.21.15.32:0/895020570 conn(0x7f61c00ff380 msgr2=0x7f61c01088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:07.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.803+0000 7f61c5800640 1 -- 172.21.15.32:0/895020570 shutdown_connections 2024-08-31T07:37:07.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:07.803+0000 7f61c5800640 1 -- 172.21.15.32:0/895020570 wait complete. 2024-08-31T07:37:07.822 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:07.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:07 smithi084 ceph-mon[22387]: osdmap e117: 12 total, 12 up, 12 in 2024-08-31T07:37:08.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:07 smithi073 ceph-mon[21995]: osdmap e117: 12 total, 12 up, 12 in 2024-08-31T07:37:08.920 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:08 smithi032 ceph-mon[17252]: pgmap v1136: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.0 MiB/s wr, 667 op/s 2024-08-31T07:37:08.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:08 smithi084 ceph-mon[22387]: pgmap v1136: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.0 MiB/s wr, 667 op/s 2024-08-31T07:37:09.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:08 smithi073 ceph-mon[21995]: pgmap v1136: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.0 MiB/s wr, 667 op/s 2024-08-31T07:37:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 -- 172.21.15.32:0/3471560681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4988103750 msgr2=0x7f4988103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 --2- 172.21.15.32:0/3471560681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4988103750 0x7f4988103b60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4978009900 tx=0x7f497802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 -- 172.21.15.32:0/3471560681 shutdown_connections 2024-08-31T07:37:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 --2- 172.21.15.32:0/3471560681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49881099e0 0x7f4988110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 --2- 172.21.15.32:0/3471560681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4988109090 0x7f49881094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 --2- 172.21.15.32:0/3471560681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4988103750 0x7f4988103b60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 -- 172.21.15.32:0/3471560681 >> 172.21.15.32:0/3471560681 conn(0x7f49880ff1e0 msgr2=0x7f4988101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 -- 172.21.15.32:0/3471560681 shutdown_connections 2024-08-31T07:37:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 -- 172.21.15.32:0/3471560681 wait complete. 2024-08-31T07:37:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 Processor -- start 2024-08-31T07:37:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.231+0000 7f4987200640 1 -- start start 2024-08-31T07:37:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4987200640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4988103750 0x7f498819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4987200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4988109090 0x7f498819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4987200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49881099e0 0x7f49881a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4987200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49881136b0 con 0x7f4988109090 2024-08-31T07:37:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4987200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4988113530 con 0x7f49881099e0 2024-08-31T07:37:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4987200640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4988113830 con 0x7f4988103750 2024-08-31T07:37:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4985400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4988109090 0x7f498819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4986800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49881099e0 0x7f49881a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4985e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4988103750 0x7f498819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4986800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49881099e0 0x7f49881a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52636/0 (socket says 172.21.15.32:52636) 2024-08-31T07:37:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4986800640 1 -- 172.21.15.32:0/709873801 learned_addr learned my addr 172.21.15.32:0/709873801 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4985e00640 1 -- 172.21.15.32:0/709873801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49881099e0 msgr2=0x7f49881a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4985e00640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49881099e0 0x7f49881a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4985e00640 1 -- 172.21.15.32:0/709873801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4988109090 msgr2=0x7f498819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4985e00640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4988109090 0x7f498819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4985e00640 1 -- 172.21.15.32:0/709873801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49881a2230 con 0x7f4988103750 2024-08-31T07:37:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4986800640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49881099e0 0x7f49881a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4985e00640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4988103750 0x7f498819c2c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f497802a910 tx=0x7f4978031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f497ea00640 1 -- 172.21.15.32:0/709873801 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497803f030 con 0x7f4988103750 2024-08-31T07:37:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4987200640 1 -- 172.21.15.32:0/709873801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49881a24c0 con 0x7f4988103750 2024-08-31T07:37:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f497ea00640 1 -- 172.21.15.32:0/709873801 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4978004490 con 0x7f4988103750 2024-08-31T07:37:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f497ea00640 1 -- 172.21.15.32:0/709873801 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4978035070 con 0x7f4988103750 2024-08-31T07:37:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4987200640 1 -- 172.21.15.32:0/709873801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49881a8c80 con 0x7f4988103750 2024-08-31T07:37:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.236+0000 7f4987200640 1 -- 172.21.15.32:0/709873801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4988103b60 con 0x7f4988103750 2024-08-31T07:37:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.241+0000 7f497ea00640 1 -- 172.21.15.32:0/709873801 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4978042030 con 0x7f4988103750 2024-08-31T07:37:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.241+0000 7f497ea00640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f496c0b4ca0 0x7f496c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.241+0000 7f497ea00640 1 -- 172.21.15.32:0/709873801 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7f49781059e0 con 0x7f4988103750 2024-08-31T07:37:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.241+0000 7f4985400640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f496c0b4ca0 0x7f496c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:09.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.241+0000 7f4985400640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f496c0b4ca0 0x7f496c0b71b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f49881057e0 tx=0x7f4974005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.241+0000 7f497ea00640 1 -- 172.21.15.32:0/709873801 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f49780cc140 con 0x7f4988103750 2024-08-31T07:37:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.505+0000 7f4987200640 1 -- 172.21.15.32:0/709873801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49881a8f10 con 0x7f4988103750 2024-08-31T07:37:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.506+0000 7f497ea00640 1 -- 172.21.15.32:0/709873801 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+11980 (secure 0 0 0) 0x7f49780d0900 con 0x7f4988103750 2024-08-31T07:37:09.507 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:37:09.507 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":96,"btime":"2024-08-31T07:37:03:685617+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":96,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:37:03.457928+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:37:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 96 2024-08-31T07:37:09.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 -- 172.21.15.32:0/709873801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f496c0b4ca0 msgr2=0x7f496c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:09.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f496c0b4ca0 0x7f496c0b71b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f49881057e0 tx=0x7f4974005f90 comp rx=0 tx=0).stop 2024-08-31T07:37:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 -- 172.21.15.32:0/709873801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4988103750 msgr2=0x7f498819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4988103750 0x7f498819c2c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f497802a910 tx=0x7f4978031a20 comp rx=0 tx=0).stop 2024-08-31T07:37:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 -- 172.21.15.32:0/709873801 shutdown_connections 2024-08-31T07:37:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f496c0b4ca0 0x7f496c0b71b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49881099e0 0x7f49881a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4988109090 0x7f498819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 --2- 172.21.15.32:0/709873801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4988103750 0x7f498819c2c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 -- 172.21.15.32:0/709873801 >> 172.21.15.32:0/709873801 conn(0x7f49880ff1e0 msgr2=0x7f49881087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 -- 172.21.15.32:0/709873801 shutdown_connections 2024-08-31T07:37:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:09.508+0000 7f4987200640 1 -- 172.21.15.32:0/709873801 wait complete. 2024-08-31T07:37:09.768 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:09 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/709873801' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:37:09.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:09 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/709873801' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:37:10.024 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T07:37:10.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:09 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/709873801' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:37:10.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:10 smithi084 ceph-mon[22387]: pgmap v1137: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.4 MiB/s wr, 674 op/s 2024-08-31T07:37:11.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:10 smithi032 ceph-mon[17252]: pgmap v1137: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.4 MiB/s wr, 674 op/s 2024-08-31T07:37:11.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:10 smithi073 ceph-mon[21995]: pgmap v1137: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.4 MiB/s wr, 674 op/s 2024-08-31T07:37:11.421 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24734 quiesce db --cancel --all' 2024-08-31T07:37:11.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24734 quiesce db --cancel --all 2024-08-31T07:37:11.583 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:12.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:12 smithi084 ceph-mon[22387]: pgmap v1138: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.3 MiB/s wr, 612 op/s 2024-08-31T07:37:13.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:12 smithi032 ceph-mon[17252]: pgmap v1138: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.3 MiB/s wr, 612 op/s 2024-08-31T07:37:13.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:12 smithi073 ceph-mon[21995]: pgmap v1138: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.3 MiB/s wr, 612 op/s 2024-08-31T07:37:13.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.549+0000 7f7d6ae00640 1 -- 172.21.15.32:0/3018102661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d64103750 msgr2=0x7f7d64103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.549+0000 7f7d6ae00640 1 --2- 172.21.15.32:0/3018102661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d64103750 0x7f7d64103b60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009900 tx=0x7f7d5802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.549+0000 7f7d6ae00640 1 -- 172.21.15.32:0/3018102661 shutdown_connections 2024-08-31T07:37:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.549+0000 7f7d6ae00640 1 --2- 172.21.15.32:0/3018102661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d641099e0 0x7f7d64110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.549+0000 7f7d6ae00640 1 --2- 172.21.15.32:0/3018102661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64109090 0x7f7d641094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.549+0000 7f7d6ae00640 1 --2- 172.21.15.32:0/3018102661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d64103750 0x7f7d64103b60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.549+0000 7f7d6ae00640 1 -- 172.21.15.32:0/3018102661 >> 172.21.15.32:0/3018102661 conn(0x7f7d640ff1e0 msgr2=0x7f7d64101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.550+0000 7f7d6ae00640 1 -- 172.21.15.32:0/3018102661 shutdown_connections 2024-08-31T07:37:13.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.550+0000 7f7d6ae00640 1 -- 172.21.15.32:0/3018102661 wait complete. 2024-08-31T07:37:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.551+0000 7f7d6ae00640 1 Processor -- start 2024-08-31T07:37:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.551+0000 7f7d6ae00640 1 -- start start 2024-08-31T07:37:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.551+0000 7f7d6ae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d64103750 0x7f7d640754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.551+0000 7f7d6ae00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64109090 0x7f7d640759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.551+0000 7f7d6ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d641099e0 0x7f7d64075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.551+0000 7f7d6ae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64113690 con 0x7f7d641099e0 2024-08-31T07:37:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.551+0000 7f7d6ae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64113510 con 0x7f7d64103750 2024-08-31T07:37:13.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.551+0000 7f7d6ae00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64113810 con 0x7f7d64109090 2024-08-31T07:37:13.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d69a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d64103750 0x7f7d640754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d69a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d64103750 0x7f7d640754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52644/0 (socket says 172.21.15.32:52644) 2024-08-31T07:37:13.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d69a00640 1 -- 172.21.15.32:0/581825476 learned_addr learned my addr 172.21.15.32:0/581825476 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:13.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d69000640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64109090 0x7f7d640759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d6a400640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d641099e0 0x7f7d64075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d69a00640 1 -- 172.21.15.32:0/581825476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64109090 msgr2=0x7f7d640759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:13.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d69a00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64109090 0x7f7d640759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d69a00640 1 -- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d641099e0 msgr2=0x7f7d64075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:13.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d69a00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d641099e0 0x7f7d64075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d69a00640 1 -- 172.21.15.32:0/581825476 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d641a8380 con 0x7f7d64103750 2024-08-31T07:37:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d69000640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64109090 0x7f7d640759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.552+0000 7f7d6a400640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d641099e0 0x7f7d64075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.553+0000 7f7d69a00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d64103750 0x7f7d640754b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7d58005940 tx=0x7f7d580040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.553+0000 7f7d62000640 1 -- 172.21.15.32:0/581825476 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5803f030 con 0x7f7d64103750 2024-08-31T07:37:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.553+0000 7f7d6ae00640 1 -- 172.21.15.32:0/581825476 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d641a8610 con 0x7f7d64103750 2024-08-31T07:37:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.553+0000 7f7d62000640 1 -- 172.21.15.32:0/581825476 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d580044c0 con 0x7f7d64103750 2024-08-31T07:37:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.553+0000 7f7d62000640 1 -- 172.21.15.32:0/581825476 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58035070 con 0x7f7d64103750 2024-08-31T07:37:13.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.553+0000 7f7d6ae00640 1 -- 172.21.15.32:0/581825476 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d641a8b40 con 0x7f7d64103750 2024-08-31T07:37:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.556+0000 7f7d73044740 1 Processor -- start 2024-08-31T07:37:13.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d73044740 1 -- start start 2024-08-31T07:37:13.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d73044740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 0x55d25d1e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d73044740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 0x55d25d1e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d73044740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce22390 0x55d25d1e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d73044740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d25d15f7b0 con 0x55d25ce1a1b0 2024-08-31T07:37:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d73044740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d25d14ed40 con 0x55d25ce43070 2024-08-31T07:37:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d73044740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d25d152cf0 con 0x55d25ce22390 2024-08-31T07:37:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d62000640 1 -- 172.21.15.32:0/581825476 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7d58042030 con 0x7f7d64103750 2024-08-31T07:37:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d69000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce22390 0x55d25d1e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d6a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 0x55d25d1e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d69000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce22390 0x55d25d1e95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37232/0 (socket says 172.21.15.32:37232) 2024-08-31T07:37:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d69000640 1 -- 172.21.15.32:0/1675509491 learned_addr learned my addr 172.21.15.32:0/1675509491 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.557+0000 7f7d6a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 0x55d25d1e8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52656/0 (socket says 172.21.15.32:52656) 2024-08-31T07:37:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.558+0000 7f7d6a400640 1 -- 172.21.15.32:0/1675509491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce22390 msgr2=0x55d25d1e95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.558+0000 7f7d6a400640 1 --2- 172.21.15.32:0/1675509491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce22390 0x55d25d1e95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.558+0000 7f7d6a400640 1 -- 172.21.15.32:0/1675509491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 msgr2=0x55d25d1e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.558+0000 7f7d6a400640 1 --2- 172.21.15.32:0/1675509491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 0x55d25d1e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.558+0000 7f7d6a400640 1 -- 172.21.15.32:0/1675509491 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25ceae650 con 0x55d25ce43070 2024-08-31T07:37:13.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.558+0000 7f7d69000640 1 --2- 172.21.15.32:0/1675509491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce22390 0x55d25d1e95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:13.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.558+0000 7f7d6a400640 1 --2- 172.21.15.32:0/1675509491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 0x55d25d1e8b60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c005fd0 tx=0x7f7d5c011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c16f5587e3962a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.558+0000 7f7d56a00640 1 -- 172.21.15.32:0/1675509491 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c004d20 con 0x55d25ce43070 2024-08-31T07:37:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.558+0000 7f7d56a00640 1 -- 172.21.15.32:0/1675509491 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d5c00f9a0 con 0x55d25ce43070 2024-08-31T07:37:13.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.558+0000 7f7d56a00640 1 -- 172.21.15.32:0/1675509491 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c004380 con 0x55d25ce43070 2024-08-31T07:37:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.559+0000 7f7d62000640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d4c0b4da0 0x7f7d4c0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.559+0000 7f7d62000640 1 -- 172.21.15.32:0/581825476 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7f7d58105850 con 0x7f7d64103750 2024-08-31T07:37:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.559+0000 7f7d69a00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d4c0b4da0 0x7f7d4c0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.559+0000 7f7d73044740 1 -- 172.21.15.32:0/1675509491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 msgr2=0x55d25d1e8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.559+0000 7f7d73044740 1 --2- 172.21.15.32:0/1675509491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 0x55d25d1e8b60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c005fd0 tx=0x7f7d5c011bf0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.560+0000 7f7d73044740 1 -- 172.21.15.32:0/1675509491 shutdown_connections 2024-08-31T07:37:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.560+0000 7f7d73044740 1 --2- 172.21.15.32:0/1675509491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce22390 0x55d25d1e95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.560+0000 7f7d73044740 1 --2- 172.21.15.32:0/1675509491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 0x55d25d1e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.560+0000 7f7d73044740 1 --2- 172.21.15.32:0/1675509491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 0x55d25d1e8b60 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.560+0000 7f7d73044740 1 -- 172.21.15.32:0/1675509491 >> 172.21.15.32:0/1675509491 conn(0x55d25ce37460 msgr2=0x55d25d1e6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.560+0000 7f7d73044740 1 -- 172.21.15.32:0/1675509491 shutdown_connections 2024-08-31T07:37:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.560+0000 7f7d73044740 1 -- 172.21.15.32:0/1675509491 wait complete. 2024-08-31T07:37:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.560+0000 7f7d69a00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d4c0b4da0 0x7f7d4c0b72b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55d25d1e77e0 tx=0x7f7d58009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.561+0000 7f7d73044740 1 -- start start 2024-08-31T07:37:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.561+0000 7f7d73044740 1 Processor -- start 2024-08-31T07:37:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.561+0000 7f7d73044740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce1a1b0 0x55d25d1f4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.561+0000 7f7d73044740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d1f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.561+0000 7f7d73044740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 0x55d25d200690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.561+0000 7f7d73044740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d25cfb1700 con 0x55d25ce22390 2024-08-31T07:37:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.561+0000 7f7d73044740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d25d0d5cf0 con 0x55d25ce43070 2024-08-31T07:37:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.561+0000 7f7d73044740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d25d1f7340 con 0x55d25ce1a1b0 2024-08-31T07:37:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.561+0000 7f7d73044740 10 client.0 _ll_register_callbacks cb 0x55d25d0fb470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.562+0000 7f7d69000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d1f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.562+0000 7f7d6a400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce1a1b0 0x55d25d1f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.562+0000 7f7d69000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d1f4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40420/0 (socket says 172.21.15.32:40420) 2024-08-31T07:37:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.562+0000 7f7d69000640 1 -- 172.21.15.32:0/2111050892 learned_addr learned my addr 172.21.15.32:0/2111050892 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.562+0000 7f7d6a400640 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 0x55d25d200690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.562+0000 7f7d69000640 1 -- 172.21.15.32:0/2111050892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce1a1b0 msgr2=0x55d25d1f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.562+0000 7f7d69000640 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce1a1b0 0x55d25d1f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.562+0000 7f7d69000640 1 -- 172.21.15.32:0/2111050892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 msgr2=0x55d25d200690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.562+0000 7f7d69000640 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 0x55d25d200690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.562+0000 7f7d69000640 1 -- 172.21.15.32:0/2111050892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25d1168b0 con 0x55d25ce22390 2024-08-31T07:37:13.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.563+0000 7f7d69000640 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d1f4cd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7d50012d20 tx=0x7f7d5000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5cd74ba9043be5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:13.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.563+0000 7f7d4b400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:37:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.563+0000 7f7d4b400640 1 -- 172.21.15.32:0/2111050892 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d500120d0 con 0x55d25ce22390 2024-08-31T07:37:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.563+0000 7f7d4b400640 1 -- 172.21.15.32:0/2111050892 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7f7d50010430 con 0x55d25ce22390 2024-08-31T07:37:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.563+0000 7f7d73044740 1 -- 172.21.15.32:0/2111050892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d5c00f9a0 con 0x55d25ce22390 2024-08-31T07:37:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.563+0000 7f7d4b400640 1 -- 172.21.15.32:0/2111050892 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f7d50012270 con 0x55d25ce22390 2024-08-31T07:37:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.563+0000 7f7d73044740 10 client.19962 fetch_fsmap learned FSMap version 96 2024-08-31T07:37:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.563+0000 7f7d73044740 1 -- 172.21.15.32:0/2111050892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x55d25d2033c0 con 0x55d25ce22390 2024-08-31T07:37:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.564+0000 7f7d4b400640 1 -- 172.21.15.32:0/2111050892 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d5c00f9a0 con 0x55d25ce22390 2024-08-31T07:37:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.564+0000 7f7d4b400640 1 -- 172.21.15.32:0/2111050892 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 96) ==== 8410+0+0 (secure 0 0 0) 0x7f7d50023070 con 0x55d25ce22390 2024-08-31T07:37:13.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.564+0000 7f7d73044740 10 client.19962 fetch_fsmap finished waiting for FSMap version 96 2024-08-31T07:37:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.564+0000 7f7d73044740 10 client.19962 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:37:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.564+0000 7f7d73044740 20 client.19962 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.564+0000 7f7d73044740 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d25ce019f0 0x55d25d206a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.564+0000 7f7d73044740 4 client.19962 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:37:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.564+0000 7f7d73044740 1 -- 172.21.15.32:0/2111050892 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d25d161700 con 0x55d25ce019f0 2024-08-31T07:37:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.564+0000 7f7d6a400640 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d25ce019f0 0x55d25d206a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.565+0000 7f7d6a400640 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d25ce019f0 0x55d25d206a20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31c883feff825b13 server_cookie=24e2a7caa561c92b in_seq=0 out_seq=0 2024-08-31T07:37:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.565+0000 7f7d4b400640 10 client.19962 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:37:13.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d4b400640 1 -- 172.21.15.32:0/2111050892 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d25d161700 con 0x55d25ce019f0 2024-08-31T07:37:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d4b400640 10 client.19962 handle_command_reply: tid=0 2024-08-31T07:37:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 client.19962 shutdown 2024-08-31T07:37:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 -- 172.21.15.32:0/2111050892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 msgr2=0x55d25d1f4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d1f4cd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7d50012d20 tx=0x7f7d5000f790 comp rx=0 tx=0).stop 2024-08-31T07:37:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 -- 172.21.15.32:0/2111050892 shutdown_connections 2024-08-31T07:37:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d25ce019f0 0x55d25d206a20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce43070 0x55d25d200690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d1f4cd0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 --2- 172.21.15.32:0/2111050892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce1a1b0 0x55d25d1f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 -- 172.21.15.32:0/2111050892 >> 172.21.15.32:0/2111050892 conn(0x55d25ce37460 msgr2=0x55d25d1ebcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d4b400640 0 client.19962 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:37:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 -- 172.21.15.32:0/2111050892 shutdown_connections 2024-08-31T07:37:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 1 -- 172.21.15.32:0/2111050892 wait complete. 2024-08-31T07:37:13.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.566+0000 7f7d73044740 20 client.19962 trim_cache size 0 max 16384 2024-08-31T07:37:13.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.580+0000 7f7d73044740 1 Processor -- start 2024-08-31T07:37:13.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.580+0000 7f7d73044740 1 -- start start 2024-08-31T07:37:13.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d73044740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce1a1b0 0x55d25d1ee380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d73044740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d200590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d73044740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce43070 0x55d25d200ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d73044740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b1470 con 0x55d25ce22390 2024-08-31T07:37:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d73044740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b31f0 con 0x55d25ce1a1b0 2024-08-31T07:37:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d73044740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b1c00 con 0x55d25ce43070 2024-08-31T07:37:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d69000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d200590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d6a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce1a1b0 0x55d25d1ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d69000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d200590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40428/0 (socket says 172.21.15.32:40428) 2024-08-31T07:37:13.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d69000640 1 -- 172.21.15.32:0/232172515 learned_addr learned my addr 172.21.15.32:0/232172515 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.581+0000 7f7d6a400640 1 --2- 172.21.15.32:0/232172515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce43070 0x55d25d200ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.582+0000 7f7d69000640 1 -- 172.21.15.32:0/232172515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce43070 msgr2=0x55d25d200ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.582+0000 7f7d69000640 1 --2- 172.21.15.32:0/232172515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce43070 0x55d25d200ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.582+0000 7f7d69000640 1 -- 172.21.15.32:0/232172515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce1a1b0 msgr2=0x55d25d1ee380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.582+0000 7f7d69000640 1 --2- 172.21.15.32:0/232172515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce1a1b0 0x55d25d1ee380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.582+0000 7f7d69000640 1 -- 172.21.15.32:0/232172515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25d1eae50 con 0x55d25ce22390 2024-08-31T07:37:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.582+0000 7f7d6a400640 1 --2- 172.21.15.32:0/232172515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce1a1b0 0x55d25d1ee380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:13.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.582+0000 7f7d69000640 1 --2- 172.21.15.32:0/232172515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d200590 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000a010 tx=0x7f7d5000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e76a798a64608c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.582+0000 7f7d56000640 1 -- 172.21.15.32:0/232172515 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50029070 con 0x55d25ce22390 2024-08-31T07:37:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.582+0000 7f7d56000640 1 -- 172.21.15.32:0/232172515 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d50005580 con 0x55d25ce22390 2024-08-31T07:37:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.582+0000 7f7d56000640 1 -- 172.21.15.32:0/232172515 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50003920 con 0x55d25ce22390 2024-08-31T07:37:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.583+0000 7f7d73044740 1 -- 172.21.15.32:0/232172515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 msgr2=0x55d25d200590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.583+0000 7f7d73044740 1 --2- 172.21.15.32:0/232172515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d200590 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000a010 tx=0x7f7d5000fc90 comp rx=0 tx=0).stop 2024-08-31T07:37:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.583+0000 7f7d73044740 1 -- 172.21.15.32:0/232172515 shutdown_connections 2024-08-31T07:37:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.583+0000 7f7d73044740 1 --2- 172.21.15.32:0/232172515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce43070 0x55d25d200ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.583+0000 7f7d73044740 1 --2- 172.21.15.32:0/232172515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce22390 0x55d25d200590 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.583+0000 7f7d73044740 1 --2- 172.21.15.32:0/232172515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce1a1b0 0x55d25d1ee380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.583+0000 7f7d73044740 1 -- 172.21.15.32:0/232172515 >> 172.21.15.32:0/232172515 conn(0x55d25ce37460 msgr2=0x55d25d1edc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.583+0000 7f7d73044740 1 -- 172.21.15.32:0/232172515 shutdown_connections 2024-08-31T07:37:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.583+0000 7f7d73044740 1 -- 172.21.15.32:0/232172515 wait complete. 2024-08-31T07:37:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.584+0000 7f7d73044740 1 -- start start 2024-08-31T07:37:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.584+0000 7f7d73044740 1 Processor -- start 2024-08-31T07:37:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.584+0000 7f7d73044740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 0x55d25d1f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d73044740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce22390 0x55d25d1f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d73044740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce43070 0x55d25d26a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d73044740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b2a90 con 0x55d25ce1a1b0 2024-08-31T07:37:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d73044740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b2300 con 0x55d25ce22390 2024-08-31T07:37:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d73044740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641af470 con 0x55d25ce43070 2024-08-31T07:37:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d73044740 10 client.0 _ll_register_callbacks cb 0x7f7d50012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d69000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce22390 0x55d25d1f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d6a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 0x55d25d1f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d6a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 0x55d25d1f8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40442/0 (socket says 172.21.15.32:40442) 2024-08-31T07:37:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d69000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce22390 0x55d25d1f94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52672/0 (socket says 172.21.15.32:52672) 2024-08-31T07:37:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d6a400640 1 -- 172.21.15.32:0/3488974347 learned_addr learned my addr 172.21.15.32:0/3488974347 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d6a400640 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce43070 0x55d25d26a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d6a400640 1 -- 172.21.15.32:0/3488974347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce43070 msgr2=0x55d25d26a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d6a400640 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce43070 0x55d25d26a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d6a400640 1 -- 172.21.15.32:0/3488974347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce22390 msgr2=0x55d25d1f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d6a400640 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce22390 0x55d25d1f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.585+0000 7f7d6a400640 1 -- 172.21.15.32:0/3488974347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25d26aa20 con 0x55d25ce1a1b0 2024-08-31T07:37:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.586+0000 7f7d6a400640 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 0x55d25d1f8f00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c005bd0 tx=0x7f7d5c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda922cd3fb6bc2c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.586+0000 7f7d4b400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:37:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.586+0000 7f7d4b400640 1 -- 172.21.15.32:0/3488974347 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c01e040 con 0x55d25ce1a1b0 2024-08-31T07:37:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.586+0000 7f7d4b400640 1 -- 172.21.15.32:0/3488974347 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7f7d5c004c90 con 0x55d25ce1a1b0 2024-08-31T07:37:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.586+0000 7f7d73044740 1 -- 172.21.15.32:0/3488974347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d50005580 con 0x55d25ce1a1b0 2024-08-31T07:37:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.586+0000 7f7d4b400640 1 -- 172.21.15.32:0/3488974347 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f7d5c003a30 con 0x55d25ce1a1b0 2024-08-31T07:37:13.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.586+0000 7f7d73044740 10 client.19968 fetch_fsmap learned FSMap version 96 2024-08-31T07:37:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.586+0000 7f7d73044740 1 -- 172.21.15.32:0/3488974347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x55d25d271d40 con 0x55d25ce1a1b0 2024-08-31T07:37:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.587+0000 7f7d4b400640 1 -- 172.21.15.32:0/3488974347 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d50005580 con 0x55d25ce1a1b0 2024-08-31T07:37:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.587+0000 7f7d4b400640 1 -- 172.21.15.32:0/3488974347 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 96) ==== 8410+0+0 (secure 0 0 0) 0x7f7d5c03a020 con 0x55d25ce1a1b0 2024-08-31T07:37:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.587+0000 7f7d73044740 10 client.19968 fetch_fsmap finished waiting for FSMap version 96 2024-08-31T07:37:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.587+0000 7f7d73044740 10 client.19968 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-31T07:37:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.587+0000 7f7d73044740 20 client.19968 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:13.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.587+0000 7f7d73044740 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d25cdd6da0 0x55d25d274380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.587+0000 7f7d73044740 4 client.19968 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:37:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.587+0000 7f7d73044740 1 -- 172.21.15.32:0/3488974347 --> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d25d15d1a0 con 0x55d25cdd6da0 2024-08-31T07:37:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.587+0000 7f7d69000640 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d25cdd6da0 0x55d25d274380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.588+0000 7f7d69000640 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d25cdd6da0 0x55d25d274380 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3066a25b02c03440 server_cookie=6b3461304ae15e15 in_seq=0 out_seq=0 2024-08-31T07:37:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.588+0000 7f7d4b400640 10 client.19968 ms_handle_connect on v2:172.21.15.32:6840/2484505785 2024-08-31T07:37:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.588+0000 7f7d4b400640 1 -- 172.21.15.32:0/3488974347 <== mds.1 v2:172.21.15.32:6840/2484505785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d25d15d1a0 con 0x55d25cdd6da0 2024-08-31T07:37:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.588+0000 7f7d4b400640 10 client.19968 handle_command_reply: tid=0 2024-08-31T07:37:13.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.588+0000 7f7d73044740 1 client.19968 shutdown 2024-08-31T07:37:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 1 -- 172.21.15.32:0/3488974347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 msgr2=0x55d25d1f8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 0x55d25d1f8f00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c005bd0 tx=0x7f7d5c000f40 comp rx=0 tx=0).stop 2024-08-31T07:37:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 1 -- 172.21.15.32:0/3488974347 shutdown_connections 2024-08-31T07:37:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d25ce43070 0x55d25d26a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.32:6840/2484505785,v1:172.21.15.32:6841/2484505785] conn(0x55d25cdd6da0 0x55d25d274380 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d25ce22390 0x55d25d1f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 1 --2- 172.21.15.32:0/3488974347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d25ce1a1b0 0x55d25d1f8f00 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 1 -- 172.21.15.32:0/3488974347 >> 172.21.15.32:0/3488974347 conn(0x55d25ce37460 msgr2=0x55d25d1e9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:13.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d4b400640 0 client.19968 ms_handle_reset on v2:172.21.15.32:6840/2484505785 2024-08-31T07:37:13.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 1 -- 172.21.15.32:0/3488974347 shutdown_connections 2024-08-31T07:37:13.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 1 -- 172.21.15.32:0/3488974347 wait complete. 2024-08-31T07:37:13.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.589+0000 7f7d73044740 20 client.19968 trim_cache size 0 max 16384 2024-08-31T07:37:13.596 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:37:13.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.590+0000 7f7d6ae00640 1 -- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d4c0b4da0 msgr2=0x7f7d4c0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:13.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.590+0000 7f7d6ae00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d4c0b4da0 0x7f7d4c0b72b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55d25d1e77e0 tx=0x7f7d58009560 comp rx=0 tx=0).stop 2024-08-31T07:37:13.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.590+0000 7f7d6ae00640 1 -- 172.21.15.32:0/581825476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d64103750 msgr2=0x7f7d640754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:13.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.590+0000 7f7d6ae00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d64103750 0x7f7d640754b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7d58005940 tx=0x7f7d580040c0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.590+0000 7f7d6ae00640 1 -- 172.21.15.32:0/581825476 shutdown_connections 2024-08-31T07:37:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.590+0000 7f7d6ae00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d4c0b4da0 0x7f7d4c0b72b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.590+0000 7f7d6ae00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d641099e0 0x7f7d64075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.590+0000 7f7d6ae00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d64109090 0x7f7d640759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.590+0000 7f7d6ae00640 1 --2- 172.21.15.32:0/581825476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d64103750 0x7f7d640754b0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.590+0000 7f7d6ae00640 1 -- 172.21.15.32:0/581825476 >> 172.21.15.32:0/581825476 conn(0x7f7d640ff1e0 msgr2=0x7f7d64101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.591+0000 7f7d6ae00640 1 -- 172.21.15.32:0/581825476 shutdown_connections 2024-08-31T07:37:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:13.591+0000 7f7d6ae00640 1 -- 172.21.15.32:0/581825476 wait complete. 2024-08-31T07:37:13.926 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:14.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T07:37:14.795 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:14 smithi032 ceph-mon[17252]: pgmap v1139: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.2 MiB/s wr, 544 op/s 2024-08-31T07:37:14.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:14 smithi084 ceph-mon[22387]: pgmap v1139: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.2 MiB/s wr, 544 op/s 2024-08-31T07:37:15.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:14 smithi073 ceph-mon[21995]: pgmap v1139: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 8.2 MiB/s wr, 544 op/s 2024-08-31T07:37:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.291+0000 7fb467000640 1 -- 172.21.15.32:0/2194228165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb460109250 msgr2=0x7fb460109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.291+0000 7fb467000640 1 --2- 172.21.15.32:0/2194228165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb460109250 0x7fb460109660 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb448009900 tx=0x7fb44802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.291+0000 7fb467000640 1 -- 172.21.15.32:0/2194228165 shutdown_connections 2024-08-31T07:37:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.291+0000 7fb467000640 1 --2- 172.21.15.32:0/2194228165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb460109ba0 0x7fb4601103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.291+0000 7fb467000640 1 --2- 172.21.15.32:0/2194228165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb460109250 0x7fb460109660 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.291+0000 7fb467000640 1 --2- 172.21.15.32:0/2194228165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb460103910 0x7fb460103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.291+0000 7fb467000640 1 -- 172.21.15.32:0/2194228165 >> 172.21.15.32:0/2194228165 conn(0x7fb4600ff380 msgr2=0x7fb4601017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.291+0000 7fb467000640 1 -- 172.21.15.32:0/2194228165 shutdown_connections 2024-08-31T07:37:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.291+0000 7fb467000640 1 -- 172.21.15.32:0/2194228165 wait complete. 2024-08-31T07:37:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.292+0000 7fb467000640 1 Processor -- start 2024-08-31T07:37:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.292+0000 7fb467000640 1 -- start start 2024-08-31T07:37:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb467000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb460103910 0x7fb46019c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb467000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb460109250 0x7fb46019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb467000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb460109ba0 0x7fb4601a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb467000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb460113860 con 0x7fb460103910 2024-08-31T07:37:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb467000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601136e0 con 0x7fb460109250 2024-08-31T07:37:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb467000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601139e0 con 0x7fb460109ba0 2024-08-31T07:37:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb45fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb460103910 0x7fb46019c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb45fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb460103910 0x7fb46019c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40454/0 (socket says 172.21.15.32:40454) 2024-08-31T07:37:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb45fe00640 1 -- 172.21.15.32:0/2256166935 learned_addr learned my addr 172.21.15.32:0/2256166935 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb457400640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb460109250 0x7fb46019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.293+0000 7fb465000640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb460109ba0 0x7fb4601a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.294+0000 7fb45fe00640 1 -- 172.21.15.32:0/2256166935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb460109ba0 msgr2=0x7fb4601a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.294+0000 7fb45fe00640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb460109ba0 0x7fb4601a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.294+0000 7fb45fe00640 1 -- 172.21.15.32:0/2256166935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb460109250 msgr2=0x7fb46019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.294+0000 7fb45fe00640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb460109250 0x7fb46019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.294+0000 7fb45fe00640 1 -- 172.21.15.32:0/2256166935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4601a23e0 con 0x7fb460103910 2024-08-31T07:37:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.294+0000 7fb457400640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb460109250 0x7fb46019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.294+0000 7fb45fe00640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb460103910 0x7fb46019c470 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb45000ebc0 tx=0x7fb45000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.294+0000 7fb45d600640 1 -- 172.21.15.32:0/2256166935 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450010630 con 0x7fb460103910 2024-08-31T07:37:15.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.295+0000 7fb467000640 1 -- 172.21.15.32:0/2256166935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4601a26d0 con 0x7fb460103910 2024-08-31T07:37:15.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.295+0000 7fb45d600640 1 -- 172.21.15.32:0/2256166935 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb450010c80 con 0x7fb460103910 2024-08-31T07:37:15.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.295+0000 7fb467000640 1 -- 172.21.15.32:0/2256166935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4601a8e30 con 0x7fb460103910 2024-08-31T07:37:15.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.295+0000 7fb45d600640 1 -- 172.21.15.32:0/2256166935 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450026060 con 0x7fb460103910 2024-08-31T07:37:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.298+0000 7fb46dc21740 1 Processor -- start 2024-08-31T07:37:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.299+0000 7fb46dc21740 1 -- start start 2024-08-31T07:37:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.299+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d832d0 0x5641e412a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.299+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d5a410 0x5641e412a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.299+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d625f0 0x5641e412ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.299+0000 7fb46dc21740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5641e409f790 con 0x5641e3d5a410 2024-08-31T07:37:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.299+0000 7fb46dc21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5641e408ed20 con 0x5641e3d625f0 2024-08-31T07:37:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.299+0000 7fb46dc21740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5641e4092cd0 con 0x5641e3d832d0 2024-08-31T07:37:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.299+0000 7fb45d600640 1 -- 172.21.15.32:0/2256166935 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb4500107d0 con 0x7fb460103910 2024-08-31T07:37:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.299+0000 7fb465000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d832d0 0x5641e412a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb465000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d832d0 0x5641e412a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:37284/0 (socket says 172.21.15.32:37284) 2024-08-31T07:37:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb465000640 1 -- 172.21.15.32:0/1305553008 learned_addr learned my addr 172.21.15.32:0/1305553008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb457400640 1 --2- 172.21.15.32:0/1305553008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d625f0 0x5641e412ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb45fe00640 1 --2- 172.21.15.32:0/1305553008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d5a410 0x5641e412a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb465000640 1 -- 172.21.15.32:0/1305553008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d625f0 msgr2=0x5641e412ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb465000640 1 --2- 172.21.15.32:0/1305553008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d625f0 0x5641e412ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb465000640 1 -- 172.21.15.32:0/1305553008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d5a410 msgr2=0x5641e412a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb465000640 1 --2- 172.21.15.32:0/1305553008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d5a410 0x5641e412a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb465000640 1 -- 172.21.15.32:0/1305553008 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e3dee8b0 con 0x5641e3d832d0 2024-08-31T07:37:15.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb45fe00640 1 --2- 172.21.15.32:0/1305553008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d5a410 0x5641e412a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:37:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.300+0000 7fb465000640 1 --2- 172.21.15.32:0/1305553008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d832d0 0x5641e412a110 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb458009ca0 tx=0x7fb458011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c2d99b77ad8bab1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.301+0000 7fb455600640 1 -- 172.21.15.32:0/1305553008 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45801e070 con 0x5641e3d832d0 2024-08-31T07:37:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.301+0000 7fb455600640 1 -- 172.21.15.32:0/1305553008 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb458009e50 con 0x5641e3d832d0 2024-08-31T07:37:15.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.301+0000 7fb455600640 1 -- 172.21.15.32:0/1305553008 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45800ea10 con 0x5641e3d832d0 2024-08-31T07:37:15.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb46dc21740 1 -- 172.21.15.32:0/1305553008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d832d0 msgr2=0x5641e412a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:15.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb46dc21740 1 --2- 172.21.15.32:0/1305553008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d832d0 0x5641e412a110 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb458009ca0 tx=0x7fb458011810 comp rx=0 tx=0).stop 2024-08-31T07:37:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb45d600640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb4440b4ca0 0x7fb4440b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb46dc21740 1 -- 172.21.15.32:0/1305553008 shutdown_connections 2024-08-31T07:37:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb46dc21740 1 --2- 172.21.15.32:0/1305553008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d625f0 0x5641e412ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb46dc21740 1 --2- 172.21.15.32:0/1305553008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d5a410 0x5641e412a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb46dc21740 1 --2- 172.21.15.32:0/1305553008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d832d0 0x5641e412a110 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb46dc21740 1 -- 172.21.15.32:0/1305553008 >> 172.21.15.32:0/1305553008 conn(0x5641e3d776c0 msgr2=0x5641e4128000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb45d600640 1 -- 172.21.15.32:0/2256166935 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7fb4500e5b70 con 0x7fb460103910 2024-08-31T07:37:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb465000640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb4440b4ca0 0x7fb4440b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.302+0000 7fb46dc21740 1 -- 172.21.15.32:0/1305553008 shutdown_connections 2024-08-31T07:37:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.303+0000 7fb46dc21740 1 -- 172.21.15.32:0/1305553008 wait complete. 2024-08-31T07:37:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.303+0000 7fb465000640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb4440b4ca0 0x7fb4440b71b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb458009ab0 tx=0x7fb458005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.303+0000 7fb46dc21740 1 -- start start 2024-08-31T07:37:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.304+0000 7fb46dc21740 1 Processor -- start 2024-08-31T07:37:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.304+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e413e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.304+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e413ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.304+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e414a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.304+0000 7fb46dc21740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5641e3ef1810 con 0x5641e3d625f0 2024-08-31T07:37:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.304+0000 7fb46dc21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5641e402a2b0 con 0x5641e3d832d0 2024-08-31T07:37:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.304+0000 7fb46dc21740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5641e4141020 con 0x5641e3d5a410 2024-08-31T07:37:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.304+0000 7fb46dc21740 10 client.0 _ll_register_callbacks cb 0x5641e403b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.304+0000 7fb465000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb465000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e414a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52700/0 (socket says 172.21.15.32:52700) 2024-08-31T07:37:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb465000640 1 -- 172.21.15.32:0/3691315792 learned_addr learned my addr 172.21.15.32:0/3691315792 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb45fe00640 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e413e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb457400640 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e413ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb465000640 1 -- 172.21.15.32:0/3691315792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 msgr2=0x5641e413e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:15.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb465000640 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e413e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb465000640 1 -- 172.21.15.32:0/3691315792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 msgr2=0x5641e413ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:15.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb465000640 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e413ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb465000640 1 -- 172.21.15.32:0/3691315792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e405f1f0 con 0x5641e3d832d0 2024-08-31T07:37:15.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb45fe00640 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e413e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:15.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb465000640 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e414a350 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb45801b000 tx=0x7fb45800e5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110ff93d1b2895c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:15.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.305+0000 7fb44e000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:37:15.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.306+0000 7fb44e000640 1 -- 172.21.15.32:0/3691315792 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458034060 con 0x5641e3d832d0 2024-08-31T07:37:15.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.306+0000 7fb44e000640 1 -- 172.21.15.32:0/3691315792 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7fb4580046c0 con 0x5641e3d832d0 2024-08-31T07:37:15.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.306+0000 7fb46dc21740 1 -- 172.21.15.32:0/3691315792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb458009e50 con 0x5641e3d832d0 2024-08-31T07:37:15.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.306+0000 7fb44e000640 1 -- 172.21.15.32:0/3691315792 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7fb45801e040 con 0x5641e3d832d0 2024-08-31T07:37:15.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.306+0000 7fb46dc21740 10 client.30083 fetch_fsmap learned FSMap version 96 2024-08-31T07:37:15.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.306+0000 7fb46dc21740 1 -- 172.21.15.32:0/3691315792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x5641e414cff0 con 0x5641e3d832d0 2024-08-31T07:37:15.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.307+0000 7fb44e000640 1 -- 172.21.15.32:0/3691315792 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb458009e50 con 0x5641e3d832d0 2024-08-31T07:37:15.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.307+0000 7fb44e000640 1 -- 172.21.15.32:0/3691315792 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 96) ==== 8410+0+0 (secure 0 0 0) 0x7fb4580039f0 con 0x5641e3d832d0 2024-08-31T07:37:15.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.307+0000 7fb46dc21740 10 client.30083 fetch_fsmap finished waiting for FSMap version 96 2024-08-31T07:37:15.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.307+0000 7fb46dc21740 10 client.30083 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:37:15.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.307+0000 7fb46dc21740 20 client.30083 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:15.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.307+0000 7fb46dc21740 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5641e3d41c50 0x5641e41503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.307+0000 7fb46dc21740 4 client.30083 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:37:15.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.307+0000 7fb46dc21740 1 -- 172.21.15.32:0/3691315792 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641e40a16e0 con 0x5641e3d41c50 2024-08-31T07:37:15.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.308+0000 7fb45fe00640 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5641e3d41c50 0x5641e41503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.309+0000 7fb44e000640 10 client.30083 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:37:15.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.309+0000 7fb45fe00640 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5641e3d41c50 0x5641e41503b0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7beafc791149f6c server_cookie=13a300b8967cf4a0 in_seq=0 out_seq=0 2024-08-31T07:37:15.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.309+0000 7fb44e000640 1 -- 172.21.15.32:0/3691315792 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641e40a16e0 con 0x5641e3d41c50 2024-08-31T07:37:15.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.309+0000 7fb44e000640 10 client.30083 handle_command_reply: tid=0 2024-08-31T07:37:15.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.309+0000 7fb46dc21740 1 client.30083 shutdown 2024-08-31T07:37:15.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.310+0000 7fb46dc21740 1 -- 172.21.15.32:0/3691315792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 msgr2=0x5641e414a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:15.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.310+0000 7fb46dc21740 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e414a350 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb45801b000 tx=0x7fb45800e5f0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.310+0000 7fb46dc21740 1 -- 172.21.15.32:0/3691315792 shutdown_connections 2024-08-31T07:37:15.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.310+0000 7fb46dc21740 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5641e3d41c50 0x5641e41503b0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.310+0000 7fb46dc21740 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e414a350 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.310+0000 7fb46dc21740 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e413ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.310+0000 7fb46dc21740 1 --2- 172.21.15.32:0/3691315792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e413e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.310+0000 7fb44e000640 0 client.30083 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:37:15.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.310+0000 7fb46dc21740 1 -- 172.21.15.32:0/3691315792 >> 172.21.15.32:0/3691315792 conn(0x5641e3d776c0 msgr2=0x5641e41265a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:15.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.310+0000 7fb46dc21740 1 -- 172.21.15.32:0/3691315792 shutdown_connections 2024-08-31T07:37:15.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.311+0000 7fb46dc21740 1 -- 172.21.15.32:0/3691315792 wait complete. 2024-08-31T07:37:15.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.311+0000 7fb46dc21740 20 client.30083 trim_cache size 0 max 16384 2024-08-31T07:37:15.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.332+0000 7fb46dc21740 1 Processor -- start 2024-08-31T07:37:15.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.332+0000 7fb46dc21740 1 -- start start 2024-08-31T07:37:15.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.332+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e4122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.332+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e413e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.332+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e41265a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.332+0000 7fb46dc21740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601ad350 con 0x5641e3d625f0 2024-08-31T07:37:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.332+0000 7fb46dc21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601af0d0 con 0x5641e3d832d0 2024-08-31T07:37:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.332+0000 7fb46dc21740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601adae0 con 0x5641e3d5a410 2024-08-31T07:37:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.332+0000 7fb465000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e41265a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.332+0000 7fb457400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e413e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.333+0000 7fb465000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e41265a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:52712/0 (socket says 172.21.15.32:52712) 2024-08-31T07:37:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.333+0000 7fb465000640 1 -- 172.21.15.32:0/56083072 learned_addr learned my addr 172.21.15.32:0/56083072 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.333+0000 7fb45fe00640 1 --2- 172.21.15.32:0/56083072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e4122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.333+0000 7fb45fe00640 1 -- 172.21.15.32:0/56083072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 msgr2=0x5641e41265a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.333+0000 7fb45fe00640 1 --2- 172.21.15.32:0/56083072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e41265a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.333+0000 7fb45fe00640 1 -- 172.21.15.32:0/56083072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 msgr2=0x5641e413e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.333+0000 7fb45fe00640 1 --2- 172.21.15.32:0/56083072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e413e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.333+0000 7fb45fe00640 1 -- 172.21.15.32:0/56083072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e413f5a0 con 0x5641e3d5a410 2024-08-31T07:37:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.333+0000 7fb457400640 1 --2- 172.21.15.32:0/56083072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e413e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.333+0000 7fb465000640 1 --2- 172.21.15.32:0/56083072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e41265a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.334+0000 7fb45fe00640 1 --2- 172.21.15.32:0/56083072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e4122cc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb4500e32e0 tx=0x7fb45000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=342c541643d88ad2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.334+0000 7fb454c00640 1 -- 172.21.15.32:0/56083072 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450010ac0 con 0x5641e3d5a410 2024-08-31T07:37:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.334+0000 7fb454c00640 1 -- 172.21.15.32:0/56083072 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb450018420 con 0x5641e3d5a410 2024-08-31T07:37:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.334+0000 7fb454c00640 1 -- 172.21.15.32:0/56083072 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4500a85c0 con 0x5641e3d5a410 2024-08-31T07:37:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.335+0000 7fb46dc21740 1 -- 172.21.15.32:0/56083072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 msgr2=0x5641e4122cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.335+0000 7fb46dc21740 1 --2- 172.21.15.32:0/56083072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e4122cc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb4500e32e0 tx=0x7fb45000cf90 comp rx=0 tx=0).stop 2024-08-31T07:37:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.335+0000 7fb46dc21740 1 -- 172.21.15.32:0/56083072 shutdown_connections 2024-08-31T07:37:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.335+0000 7fb46dc21740 1 --2- 172.21.15.32:0/56083072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e41265a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.335+0000 7fb46dc21740 1 --2- 172.21.15.32:0/56083072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e413e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.335+0000 7fb46dc21740 1 --2- 172.21.15.32:0/56083072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e4122cc0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.335+0000 7fb46dc21740 1 -- 172.21.15.32:0/56083072 >> 172.21.15.32:0/56083072 conn(0x5641e3d776c0 msgr2=0x5641e4139530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.335+0000 7fb46dc21740 1 -- 172.21.15.32:0/56083072 shutdown_connections 2024-08-31T07:37:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.335+0000 7fb46dc21740 1 -- 172.21.15.32:0/56083072 wait complete. 2024-08-31T07:37:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.335+0000 7fb46dc21740 1 -- start start 2024-08-31T07:37:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.336+0000 7fb46dc21740 1 Processor -- start 2024-08-31T07:37:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.336+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e41462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.336+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e4146800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.336+0000 7fb46dc21740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e4141270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.336+0000 7fb46dc21740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601ae970 con 0x5641e3d625f0 2024-08-31T07:37:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.336+0000 7fb46dc21740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601ae1e0 con 0x5641e3d832d0 2024-08-31T07:37:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.336+0000 7fb46dc21740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601ab3d0 con 0x5641e3d5a410 2024-08-31T07:37:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.336+0000 7fb46dc21740 10 client.0 _ll_register_callbacks cb 0x7fb458009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb457400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e4146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb457400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e4146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40490/0 (socket says 172.21.15.32:40490) 2024-08-31T07:37:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb457400640 1 -- 172.21.15.32:0/4054018229 learned_addr learned my addr 172.21.15.32:0/4054018229 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb45fe00640 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e41462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb465000640 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e4141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb457400640 1 -- 172.21.15.32:0/4054018229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 msgr2=0x5641e41462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb457400640 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e41462c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb457400640 1 -- 172.21.15.32:0/4054018229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 msgr2=0x5641e4141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb457400640 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e4141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb457400640 1 -- 172.21.15.32:0/4054018229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e4146f40 con 0x5641e3d625f0 2024-08-31T07:37:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.337+0000 7fb45fe00640 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e41462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.338+0000 7fb457400640 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e4146800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb46006bae0 tx=0x7fb448031990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962f124bfffa9390 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.338+0000 7fb44e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:37:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.338+0000 7fb44e000640 1 -- 172.21.15.32:0/4054018229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448035040 con 0x5641e3d625f0 2024-08-31T07:37:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.338+0000 7fb44e000640 1 -- 172.21.15.32:0/4054018229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7fb448004310 con 0x5641e3d625f0 2024-08-31T07:37:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.338+0000 7fb46dc21740 1 -- 172.21.15.32:0/4054018229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641e409d180 con 0x5641e3d625f0 2024-08-31T07:37:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.338+0000 7fb44e000640 1 -- 172.21.15.32:0/4054018229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7fb448005030 con 0x5641e3d625f0 2024-08-31T07:37:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.338+0000 7fb46dc21740 10 client.19983 fetch_fsmap learned FSMap version 96 2024-08-31T07:37:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.338+0000 7fb46dc21740 1 -- 172.21.15.32:0/4054018229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x5641e41bb210 con 0x5641e3d625f0 2024-08-31T07:37:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.339+0000 7fb44e000640 1 -- 172.21.15.32:0/4054018229 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5641e409d180 con 0x5641e3d625f0 2024-08-31T07:37:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.339+0000 7fb44e000640 1 -- 172.21.15.32:0/4054018229 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 96) ==== 8410+0+0 (secure 0 0 0) 0x7fb44804d020 con 0x5641e3d625f0 2024-08-31T07:37:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.339+0000 7fb46dc21740 10 client.19983 fetch_fsmap finished waiting for FSMap version 96 2024-08-31T07:37:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.339+0000 7fb46dc21740 10 client.19983 resolve_mds: validated gid 24734 aka daemon mds.b 2024-08-31T07:37:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.339+0000 7fb46dc21740 20 client.19983 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.339+0000 7fb46dc21740 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5641e3d17000 0x5641e41bda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.339+0000 7fb46dc21740 4 client.19983 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-08-31T07:37:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.339+0000 7fb46dc21740 1 -- 172.21.15.32:0/4054018229 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5641e409a8c0 con 0x5641e3d17000 2024-08-31T07:37:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.339+0000 7fb45fe00640 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5641e3d17000 0x5641e41bda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.340+0000 7fb45fe00640 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5641e3d17000 0x5641e41bda50 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5460e6af7a0fda61 server_cookie=5cbd56e832357e6f in_seq=0 out_seq=0 2024-08-31T07:37:15.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.340+0000 7fb44e000640 10 client.19983 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:37:15.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.340+0000 7fb44e000640 1 -- 172.21.15.32:0/4054018229 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5641e409a8c0 con 0x5641e3d17000 2024-08-31T07:37:15.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.340+0000 7fb44e000640 10 client.19983 handle_command_reply: tid=0 2024-08-31T07:37:15.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.340+0000 7fb46dc21740 1 client.19983 shutdown 2024-08-31T07:37:15.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 1 -- 172.21.15.32:0/4054018229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 msgr2=0x5641e4146800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:15.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e4146800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb46006bae0 tx=0x7fb448031990 comp rx=0 tx=0).stop 2024-08-31T07:37:15.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 1 -- 172.21.15.32:0/4054018229 shutdown_connections 2024-08-31T07:37:15.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x5641e3d17000 0x5641e41bda50 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641e3d832d0 0x5641e4141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641e3d625f0 0x5641e4146800 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 1 --2- 172.21.15.32:0/4054018229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5641e3d5a410 0x5641e41462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 1 -- 172.21.15.32:0/4054018229 >> 172.21.15.32:0/4054018229 conn(0x5641e3d776c0 msgr2=0x5641e413e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb44e000640 0 client.19983 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:37:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 1 -- 172.21.15.32:0/4054018229 shutdown_connections 2024-08-31T07:37:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 1 -- 172.21.15.32:0/4054018229 wait complete. 2024-08-31T07:37:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.341+0000 7fb46dc21740 20 client.19983 trim_cache size 0 max 16384 2024-08-31T07:37:15.351 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:37:15.351 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 96, 2024-08-31T07:37:15.351 INFO:teuthology.orchestra.run.smithi032.stdout: "leader": 24734, 2024-08-31T07:37:15.351 INFO:teuthology.orchestra.run.smithi032.stdout: "set_version": 76, 2024-08-31T07:37:15.351 INFO:teuthology.orchestra.run.smithi032.stdout: "sets": {} 2024-08-31T07:37:15.351 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:37:15.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.342+0000 7fb467000640 1 -- 172.21.15.32:0/2256166935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb4440b4ca0 msgr2=0x7fb4440b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:15.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.342+0000 7fb467000640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb4440b4ca0 0x7fb4440b71b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb458009ab0 tx=0x7fb458005850 comp rx=0 tx=0).stop 2024-08-31T07:37:15.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.342+0000 7fb467000640 1 -- 172.21.15.32:0/2256166935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb460103910 msgr2=0x7fb46019c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:15.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.342+0000 7fb467000640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb460103910 0x7fb46019c470 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb45000ebc0 tx=0x7fb45000c6a0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.342+0000 7fb467000640 1 -- 172.21.15.32:0/2256166935 shutdown_connections 2024-08-31T07:37:15.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.342+0000 7fb467000640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb4440b4ca0 0x7fb4440b71b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.342+0000 7fb467000640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb460109ba0 0x7fb4601a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.342+0000 7fb467000640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb460109250 0x7fb46019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.342+0000 7fb467000640 1 --2- 172.21.15.32:0/2256166935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb460103910 0x7fb46019c470 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.342+0000 7fb467000640 1 -- 172.21.15.32:0/2256166935 >> 172.21.15.32:0/2256166935 conn(0x7fb4600ff380 msgr2=0x7fb460106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.343+0000 7fb467000640 1 -- 172.21.15.32:0/2256166935 shutdown_connections 2024-08-31T07:37:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:15.343+0000 7fb467000640 1 -- 172.21.15.32:0/2256166935 wait complete. 2024-08-31T07:37:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.297+0000 7f4b1be00640 1 -- 172.21.15.32:0/1918482491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1c109070 msgr2=0x7f4b1c109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.297+0000 7f4b1be00640 1 --2- 172.21.15.32:0/1918482491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1c109070 0x7f4b1c109480 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009900 tx=0x7f4b0402a410 comp rx=0 tx=0).stop 2024-08-31T07:37:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.297+0000 7f4b1be00640 1 -- 172.21.15.32:0/1918482491 shutdown_connections 2024-08-31T07:37:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.297+0000 7f4b1be00640 1 --2- 172.21.15.32:0/1918482491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c1099c0 0x7f4b1c1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.297+0000 7f4b1be00640 1 --2- 172.21.15.32:0/1918482491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1c109070 0x7f4b1c109480 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.297+0000 7f4b1be00640 1 --2- 172.21.15.32:0/1918482491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b1c103730 0x7f4b1c103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.297+0000 7f4b1be00640 1 -- 172.21.15.32:0/1918482491 >> 172.21.15.32:0/1918482491 conn(0x7f4b1c0ff180 msgr2=0x7f4b1c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.297+0000 7f4b1be00640 1 -- 172.21.15.32:0/1918482491 shutdown_connections 2024-08-31T07:37:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.297+0000 7f4b1be00640 1 -- 172.21.15.32:0/1918482491 wait complete. 2024-08-31T07:37:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.299+0000 7f4b1be00640 1 Processor -- start 2024-08-31T07:37:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.299+0000 7f4b1be00640 1 -- start start 2024-08-31T07:37:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.299+0000 7f4b1be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b1c103730 0x7f4b1c19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.299+0000 7f4b1be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1c109070 0x7f4b1c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.299+0000 7f4b1be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c1099c0 0x7f4b1c1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.299+0000 7f4b1be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c113650 con 0x7f4b1c109070 2024-08-31T07:37:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.299+0000 7f4b1be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1134d0 con 0x7f4b1c103730 2024-08-31T07:37:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.299+0000 7f4b1be00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1137d0 con 0x7f4b1c1099c0 2024-08-31T07:37:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1c109070 0x7f4b1c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b1aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b1c103730 0x7f4b1c19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b1b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c1099c0 0x7f4b1c1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b1aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b1c103730 0x7f4b1c19c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35800/0 (socket says 172.21.15.32:35800) 2024-08-31T07:37:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b1aa00640 1 -- 172.21.15.32:0/2537320036 learned_addr learned my addr 172.21.15.32:0/2537320036 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b1b400640 1 -- 172.21.15.32:0/2537320036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b1c103730 msgr2=0x7f4b1c19c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b1b400640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b1c103730 0x7f4b1c19c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b1b400640 1 -- 172.21.15.32:0/2537320036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1c109070 msgr2=0x7f4b1c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b1b400640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1c109070 0x7f4b1c19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b1b400640 1 -- 172.21.15.32:0/2537320036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c1a2210 con 0x7f4b1c1099c0 2024-08-31T07:37:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.300+0000 7f4b1b400640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c1099c0 0x7f4b1c1a1ae0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009fd0 tx=0x7f4b1400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.301+0000 7f4b13400640 1 -- 172.21.15.32:0/2537320036 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400ce20 con 0x7f4b1c1099c0 2024-08-31T07:37:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.301+0000 7f4b13400640 1 -- 172.21.15.32:0/2537320036 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b14005020 con 0x7f4b1c1099c0 2024-08-31T07:37:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.301+0000 7f4b1be00640 1 -- 172.21.15.32:0/2537320036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1c1a2500 con 0x7f4b1c1099c0 2024-08-31T07:37:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.301+0000 7f4b13400640 1 -- 172.21.15.32:0/2537320036 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14010690 con 0x7f4b1c1099c0 2024-08-31T07:37:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.301+0000 7f4b1be00640 1 -- 172.21.15.32:0/2537320036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1c1a8c60 con 0x7f4b1c1099c0 2024-08-31T07:37:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.304+0000 7f4b2845c740 1 Processor -- start 2024-08-31T07:37:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.305+0000 7f4b13400640 1 -- 172.21.15.32:0/2537320036 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4b140054d0 con 0x7f4b1c1099c0 2024-08-31T07:37:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.305+0000 7f4b2845c740 1 -- start start 2024-08-31T07:37:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.305+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e9a070 0x55b0e723fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.305+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 0x55b0e72400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.305+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e79390 0x55b0e72405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.305+0000 7f4b2845c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b0e71b67b0 con 0x55b0e6e711b0 2024-08-31T07:37:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.305+0000 7f4b2845c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b0e71a5d40 con 0x55b0e6e9a070 2024-08-31T07:37:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.305+0000 7f4b2845c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b0e71a9cf0 con 0x55b0e6e79390 2024-08-31T07:37:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b1aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 0x55b0e72400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e9a070 0x55b0e723fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e9a070 0x55b0e723fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35808/0 (socket says 172.21.15.32:35808) 2024-08-31T07:37:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b13e00640 1 -- 172.21.15.32:0/2927514338 learned_addr learned my addr 172.21.15.32:0/2927514338 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b13e00640 1 --2- 172.21.15.32:0/2927514338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e79390 0x55b0e72405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b1aa00640 1 -- 172.21.15.32:0/2927514338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e79390 msgr2=0x55b0e72405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/2927514338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e79390 0x55b0e72405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b1aa00640 1 -- 172.21.15.32:0/2927514338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e9a070 msgr2=0x55b0e723fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/2927514338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e9a070 0x55b0e723fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b1aa00640 1 -- 172.21.15.32:0/2927514338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e6f05650 con 0x55b0e6e711b0 2024-08-31T07:37:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.306+0000 7f4b13e00640 1 --2- 172.21.15.32:0/2927514338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e9a070 0x55b0e723fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.307+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/2927514338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 0x55b0e72400a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009d20 tx=0x7f4b0c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a5463836b900dd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.307+0000 7f4b13400640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b000b4d20 0x7f4b000b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.307+0000 7f4b13400640 1 -- 172.21.15.32:0/2537320036 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7f4b140e5ba0 con 0x7f4b1c1099c0 2024-08-31T07:37:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.307+0000 7f4b0be00640 1 -- 172.21.15.32:0/2927514338 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c005020 con 0x55b0e6e711b0 2024-08-31T07:37:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.307+0000 7f4b13e00640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b000b4d20 0x7f4b000b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.307+0000 7f4b0be00640 1 -- 172.21.15.32:0/2927514338 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b0c0047a0 con 0x55b0e6e711b0 2024-08-31T07:37:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.307+0000 7f4b0be00640 1 -- 172.21.15.32:0/2927514338 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c019670 con 0x55b0e6e711b0 2024-08-31T07:37:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.308+0000 7f4b2845c740 1 -- 172.21.15.32:0/2927514338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 msgr2=0x55b0e72400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.308+0000 7f4b2845c740 1 --2- 172.21.15.32:0/2927514338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 0x55b0e72400a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009d20 tx=0x7f4b0c011810 comp rx=0 tx=0).stop 2024-08-31T07:37:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.308+0000 7f4b13e00640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b000b4d20 0x7f4b000b7230 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4b040096c0 tx=0x7f4b04000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.308+0000 7f4b2845c740 1 -- 172.21.15.32:0/2927514338 shutdown_connections 2024-08-31T07:37:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.308+0000 7f4b2845c740 1 --2- 172.21.15.32:0/2927514338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e79390 0x55b0e72405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.308+0000 7f4b2845c740 1 --2- 172.21.15.32:0/2927514338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 0x55b0e72400a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.308+0000 7f4b2845c740 1 --2- 172.21.15.32:0/2927514338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e9a070 0x55b0e723fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.309+0000 7f4b2845c740 1 -- 172.21.15.32:0/2927514338 >> 172.21.15.32:0/2927514338 conn(0x55b0e6e8e460 msgr2=0x55b0e723da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.309+0000 7f4b2845c740 1 -- 172.21.15.32:0/2927514338 shutdown_connections 2024-08-31T07:37:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.309+0000 7f4b2845c740 1 -- 172.21.15.32:0/2927514338 wait complete. 2024-08-31T07:37:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.309+0000 7f4b2845c740 1 -- start start 2024-08-31T07:37:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.309+0000 7f4b2845c740 1 Processor -- start 2024-08-31T07:37:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 0x55b0e724b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 0x55b0e724bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 0x55b0e7257690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b2845c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b0e7008700 con 0x55b0e6e711b0 2024-08-31T07:37:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b2845c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b0e712ccf0 con 0x55b0e6e79390 2024-08-31T07:37:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b2845c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b0e724e340 con 0x55b0e6e9a070 2024-08-31T07:37:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b2845c740 10 client.0 _ll_register_callbacks cb 0x55b0e7152470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b1aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 0x55b0e724b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b1b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 0x55b0e7257690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 0x55b0e724bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b1b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 0x55b0e7257690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51554/0 (socket says 172.21.15.32:51554) 2024-08-31T07:37:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.310+0000 7f4b1b400640 1 -- 172.21.15.32:0/2804253194 learned_addr learned my addr 172.21.15.32:0/2804253194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.311+0000 7f4b1aa00640 1 -- 172.21.15.32:0/2804253194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 msgr2=0x55b0e7257690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.311+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 0x55b0e7257690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.311+0000 7f4b1aa00640 1 -- 172.21.15.32:0/2804253194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 msgr2=0x55b0e724bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.311+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 0x55b0e724bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.311+0000 7f4b1aa00640 1 -- 172.21.15.32:0/2804253194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e716d8b0 con 0x55b0e6e711b0 2024-08-31T07:37:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.311+0000 7f4b1b400640 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 0x55b0e7257690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.311+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 0x55b0e724b6f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c01b040 tx=0x7f4b0c005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3184d2484800988d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.311+0000 7f4b08c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-08-31T07:37:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.312+0000 7f4b08c00640 1 -- 172.21.15.32:0/2804253194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c028070 con 0x55b0e6e711b0 2024-08-31T07:37:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.312+0000 7f4b2845c740 1 -- 172.21.15.32:0/2804253194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b0c0047a0 con 0x55b0e6e711b0 2024-08-31T07:37:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.312+0000 7f4b08c00640 1 -- 172.21.15.32:0/2804253194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7f4b0c024440 con 0x55b0e6e711b0 2024-08-31T07:37:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.312+0000 7f4b08c00640 1 -- 172.21.15.32:0/2804253194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f4b0c034050 con 0x55b0e6e711b0 2024-08-31T07:37:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.312+0000 7f4b2845c740 10 client.19995 fetch_fsmap learned FSMap version 96 2024-08-31T07:37:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.312+0000 7f4b2845c740 1 -- 172.21.15.32:0/2804253194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x55b0e725a330 con 0x55b0e6e711b0 2024-08-31T07:37:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.312+0000 7f4b08c00640 1 -- 172.21.15.32:0/2804253194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b0c0047a0 con 0x55b0e6e711b0 2024-08-31T07:37:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.313+0000 7f4b08c00640 1 -- 172.21.15.32:0/2804253194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 96) ==== 8410+0+0 (secure 0 0 0) 0x7f4b0c024c90 con 0x55b0e6e711b0 2024-08-31T07:37:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.313+0000 7f4b2845c740 10 client.19995 fetch_fsmap finished waiting for FSMap version 96 2024-08-31T07:37:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.313+0000 7f4b2845c740 10 client.19995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:37:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.313+0000 7f4b2845c740 20 client.19995 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.313+0000 7f4b2845c740 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b0e6e589f0 0x55b0e725d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.313+0000 7f4b2845c740 4 client.19995 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T07:37:16.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.313+0000 7f4b2845c740 1 -- 172.21.15.32:0/2804253194 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0e71b8700 con 0x55b0e6e589f0 2024-08-31T07:37:16.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.313+0000 7f4b13e00640 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b0e6e589f0 0x55b0e725d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.314+0000 7f4b13e00640 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b0e6e589f0 0x55b0e725d750 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=164fdee152242a39 server_cookie=6a98cfe1828e3145 in_seq=0 out_seq=0 2024-08-31T07:37:16.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.314+0000 7f4b08c00640 10 client.19995 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:37:16.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b08c00640 1 -- 172.21.15.32:0/2804253194 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0e71b8700 con 0x55b0e6e589f0 2024-08-31T07:37:16.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b08c00640 10 client.19995 handle_command_reply: tid=0 2024-08-31T07:37:16.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 client.19995 shutdown 2024-08-31T07:37:16.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 -- 172.21.15.32:0/2804253194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 msgr2=0x55b0e724b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:16.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 0x55b0e724b6f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c01b040 tx=0x7f4b0c005210 comp rx=0 tx=0).stop 2024-08-31T07:37:16.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 -- 172.21.15.32:0/2804253194 shutdown_connections 2024-08-31T07:37:16.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b0e6e589f0 0x55b0e725d750 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 0x55b0e7257690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 0x55b0e724bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 --2- 172.21.15.32:0/2804253194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e711b0 0x55b0e724b6f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 -- 172.21.15.32:0/2804253194 >> 172.21.15.32:0/2804253194 conn(0x55b0e6e8e460 msgr2=0x55b0e7242cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:16.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b08c00640 0 client.19995 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:37:16.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 -- 172.21.15.32:0/2804253194 shutdown_connections 2024-08-31T07:37:16.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 1 -- 172.21.15.32:0/2804253194 wait complete. 2024-08-31T07:37:16.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.315+0000 7f4b2845c740 20 client.19995 trim_cache size 0 max 16384 2024-08-31T07:37:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.333+0000 7f4b2845c740 1 Processor -- start 2024-08-31T07:37:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.333+0000 7f4b2845c740 1 -- start start 2024-08-31T07:37:16.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.333+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e711b0 0x55b0e7243620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.333+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 0x55b0e72412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.334+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e9a070 0x55b0e723d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.334+0000 7f4b2845c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b14c0 con 0x55b0e6e9a070 2024-08-31T07:37:16.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.334+0000 7f4b2845c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b3240 con 0x55b0e6e79390 2024-08-31T07:37:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.334+0000 7f4b2845c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b1c50 con 0x55b0e6e711b0 2024-08-31T07:37:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.334+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 0x55b0e72412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.334+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 0x55b0e72412a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35820/0 (socket says 172.21.15.32:35820) 2024-08-31T07:37:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.334+0000 7f4b13e00640 1 -- 172.21.15.32:0/1888891758 learned_addr learned my addr 172.21.15.32:0/1888891758 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.334+0000 7f4b1b400640 1 --2- 172.21.15.32:0/1888891758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e9a070 0x55b0e723d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.334+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/1888891758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e711b0 0x55b0e7243620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.335+0000 7f4b13e00640 1 -- 172.21.15.32:0/1888891758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e711b0 msgr2=0x55b0e7243620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.335+0000 7f4b13e00640 1 --2- 172.21.15.32:0/1888891758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e711b0 0x55b0e7243620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.335+0000 7f4b13e00640 1 -- 172.21.15.32:0/1888891758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e9a070 msgr2=0x55b0e723d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.335+0000 7f4b13e00640 1 --2- 172.21.15.32:0/1888891758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e9a070 0x55b0e723d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.335+0000 7f4b13e00640 1 -- 172.21.15.32:0/1888891758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e6e58c20 con 0x55b0e6e79390 2024-08-31T07:37:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.335+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/1888891758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e711b0 0x55b0e7243620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.336+0000 7f4b13e00640 1 --2- 172.21.15.32:0/1888891758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 0x55b0e72412a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c06c5b0 tx=0x7f4b04009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8613f2b6f734593 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.337+0000 7f4b0b400640 1 -- 172.21.15.32:0/1888891758 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04035040 con 0x55b0e6e79390 2024-08-31T07:37:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.337+0000 7f4b0b400640 1 -- 172.21.15.32:0/1888891758 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b040054f0 con 0x55b0e6e79390 2024-08-31T07:37:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.337+0000 7f4b0b400640 1 -- 172.21.15.32:0/1888891758 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b040035f0 con 0x55b0e6e79390 2024-08-31T07:37:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.338+0000 7f4b2845c740 1 -- 172.21.15.32:0/1888891758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 msgr2=0x55b0e72412a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.338+0000 7f4b2845c740 1 --2- 172.21.15.32:0/1888891758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 0x55b0e72412a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c06c5b0 tx=0x7f4b04009260 comp rx=0 tx=0).stop 2024-08-31T07:37:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.339+0000 7f4b2845c740 1 -- 172.21.15.32:0/1888891758 shutdown_connections 2024-08-31T07:37:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.339+0000 7f4b2845c740 1 --2- 172.21.15.32:0/1888891758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e9a070 0x55b0e723d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.339+0000 7f4b2845c740 1 --2- 172.21.15.32:0/1888891758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e79390 0x55b0e72412a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.339+0000 7f4b2845c740 1 --2- 172.21.15.32:0/1888891758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e711b0 0x55b0e7243620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.339+0000 7f4b2845c740 1 -- 172.21.15.32:0/1888891758 >> 172.21.15.32:0/1888891758 conn(0x55b0e6e8e460 msgr2=0x55b0e7240b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:16.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.341+0000 7f4b2845c740 1 -- 172.21.15.32:0/1888891758 shutdown_connections 2024-08-31T07:37:16.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.341+0000 7f4b2845c740 1 -- 172.21.15.32:0/1888891758 wait complete. 2024-08-31T07:37:16.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.342+0000 7f4b2845c740 1 -- start start 2024-08-31T07:37:16.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.342+0000 7f4b2845c740 1 Processor -- start 2024-08-31T07:37:16.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.343+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e711b0 0x55b0e724edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.343+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e79390 0x55b0e724f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.343+0000 7f4b2845c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 0x55b0e72c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.343+0000 7f4b2845c740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b2ae0 con 0x55b0e6e79390 2024-08-31T07:37:16.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.343+0000 7f4b2845c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1b2350 con 0x55b0e6e711b0 2024-08-31T07:37:16.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.343+0000 7f4b2845c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1af4c0 con 0x55b0e6e9a070 2024-08-31T07:37:16.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.343+0000 7f4b2845c740 10 client.0 _ll_register_callbacks cb 0x7f4b0c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T07:37:16.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b1aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e711b0 0x55b0e724edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e79390 0x55b0e724f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b1aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e711b0 0x55b0e724edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35830/0 (socket says 172.21.15.32:35830) 2024-08-31T07:37:16.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b1aa00640 1 -- 172.21.15.32:0/179259465 learned_addr learned my addr 172.21.15.32:0/179259465 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:16.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b1b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 0x55b0e72c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b1aa00640 1 -- 172.21.15.32:0/179259465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 msgr2=0x55b0e72c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 0x55b0e72c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b1aa00640 1 -- 172.21.15.32:0/179259465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e79390 msgr2=0x55b0e724f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T07:37:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e79390 0x55b0e724f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b1aa00640 1 -- 172.21.15.32:0/179259465 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e72c6510 con 0x55b0e6e711b0 2024-08-31T07:37:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.344+0000 7f4b13e00640 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e79390 0x55b0e724f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.345+0000 7f4b1aa00640 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e711b0 0x55b0e724edc0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c022490 tx=0x7f4b0c022680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8715ff8b4ed9b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.345+0000 7f4b08c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-08-31T07:37:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.345+0000 7f4b08c00640 1 -- 172.21.15.32:0/179259465 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c03d060 con 0x55b0e6e711b0 2024-08-31T07:37:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.345+0000 7f4b08c00640 1 -- 172.21.15.32:0/179259465 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8485+0+0 (secure 0 0 0) 0x7f4b0c022d00 con 0x55b0e6e711b0 2024-08-31T07:37:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.345+0000 7f4b2845c740 1 -- 172.21.15.32:0/179259465 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b040054f0 con 0x55b0e6e711b0 2024-08-31T07:37:16.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.348+0000 7f4b08c00640 1 -- 172.21.15.32:0/179259465 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f4b0c028040 con 0x55b0e6e711b0 2024-08-31T07:37:16.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.348+0000 7f4b2845c740 10 client.30107 fetch_fsmap learned FSMap version 96 2024-08-31T07:37:16.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.348+0000 7f4b2845c740 1 -- 172.21.15.32:0/179259465 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x55b0e72c7760 con 0x55b0e6e711b0 2024-08-31T07:37:16.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.349+0000 7f4b08c00640 1 -- 172.21.15.32:0/179259465 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b040054f0 con 0x55b0e6e711b0 2024-08-31T07:37:16.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.349+0000 7f4b08c00640 1 -- 172.21.15.32:0/179259465 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 96) ==== 8410+0+0 (secure 0 0 0) 0x7f4b0c004250 con 0x55b0e6e711b0 2024-08-31T07:37:16.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.349+0000 7f4b2845c740 10 client.30107 fetch_fsmap finished waiting for FSMap version 96 2024-08-31T07:37:16.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.349+0000 7f4b2845c740 10 client.30107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T07:37:16.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.349+0000 7f4b2845c740 20 client.30107 populate_metadata read hostname 'smithi032' 2024-08-31T07:37:16.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.349+0000 7f4b2845c740 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b0e6e2dda0 0x55b0e72c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:16.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.349+0000 7f4b2845c740 4 client.30107 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T07:37:16.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.349+0000 7f4b2845c740 1 -- 172.21.15.32:0/179259465 --> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0e71b41a0 con 0x55b0e6e2dda0 2024-08-31T07:37:16.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.350+0000 7f4b13e00640 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b0e6e2dda0 0x55b0e72c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:16.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.351+0000 7f4b08c00640 10 client.30107 ms_handle_connect on v2:172.21.15.73:6838/794507464 2024-08-31T07:37:16.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.351+0000 7f4b13e00640 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b0e6e2dda0 0x55b0e72c9f80 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d0cfd5c8b90b7d server_cookie=73bcfa97a8ddb541 in_seq=0 out_seq=0 2024-08-31T07:37:16.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b08c00640 1 -- 172.21.15.32:0/179259465 <== mds.0 v2:172.21.15.73:6838/794507464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0e71b41a0 con 0x55b0e6e2dda0 2024-08-31T07:37:16.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b08c00640 10 client.30107 handle_command_reply: tid=0 2024-08-31T07:37:16.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b2845c740 1 client.30107 shutdown 2024-08-31T07:37:16.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b2845c740 1 -- 172.21.15.32:0/179259465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e711b0 msgr2=0x55b0e724edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:16.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b2845c740 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e711b0 0x55b0e724edc0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c022490 tx=0x7f4b0c022680 comp rx=0 tx=0).stop 2024-08-31T07:37:16.356 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-08-31T07:37:16.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b2845c740 1 -- 172.21.15.32:0/179259465 shutdown_connections 2024-08-31T07:37:16.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b2845c740 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.73:6838/794507464,v1:172.21.15.73:6839/794507464] conn(0x55b0e6e2dda0 0x55b0e72c9f80 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b2845c740 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0e6e9a070 0x55b0e72c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b08c00640 0 client.30107 ms_handle_reset on v2:172.21.15.73:6838/794507464 2024-08-31T07:37:16.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b2845c740 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0e6e79390 0x55b0e724f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b2845c740 1 --2- 172.21.15.32:0/179259465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0e6e711b0 0x55b0e724edc0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.352+0000 7f4b2845c740 1 -- 172.21.15.32:0/179259465 >> 172.21.15.32:0/179259465 conn(0x55b0e6e8e460 msgr2=0x55b0e7241550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:16.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.353+0000 7f4b2845c740 1 -- 172.21.15.32:0/179259465 shutdown_connections 2024-08-31T07:37:16.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.353+0000 7f4b2845c740 1 -- 172.21.15.32:0/179259465 wait complete. 2024-08-31T07:37:16.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.353+0000 7f4b2845c740 20 client.30107 trim_cache size 0 max 16384 2024-08-31T07:37:16.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.354+0000 7f4b1be00640 1 -- 172.21.15.32:0/2537320036 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b000b4d20 msgr2=0x7f4b000b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:16.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.354+0000 7f4b1be00640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b000b4d20 0x7f4b000b7230 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4b040096c0 tx=0x7f4b04000bc0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.354+0000 7f4b1be00640 1 -- 172.21.15.32:0/2537320036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c1099c0 msgr2=0x7f4b1c1a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:16.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.354+0000 7f4b1be00640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c1099c0 0x7f4b1c1a1ae0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009fd0 tx=0x7f4b1400ef10 comp rx=0 tx=0).stop 2024-08-31T07:37:16.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.354+0000 7f4b1be00640 1 -- 172.21.15.32:0/2537320036 shutdown_connections 2024-08-31T07:37:16.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.354+0000 7f4b1be00640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f4b000b4d20 0x7f4b000b7230 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.354+0000 7f4b1be00640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b1c1099c0 0x7f4b1c1a1ae0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.354+0000 7f4b1be00640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1c109070 0x7f4b1c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.354+0000 7f4b1be00640 1 --2- 172.21.15.32:0/2537320036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b1c103730 0x7f4b1c19c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:16.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.354+0000 7f4b1be00640 1 -- 172.21.15.32:0/2537320036 >> 172.21.15.32:0/2537320036 conn(0x7f4b1c0ff180 msgr2=0x7f4b1c10e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:16.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.357+0000 7f4b1be00640 1 -- 172.21.15.32:0/2537320036 shutdown_connections 2024-08-31T07:37:16.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:16.357+0000 7f4b1be00640 1 -- 172.21.15.32:0/2537320036 wait complete. 2024-08-31T07:37:16.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:16 smithi084 ceph-mon[22387]: pgmap v1140: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 331 op/s 2024-08-31T07:37:17.107 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:16 smithi032 ceph-mon[17252]: pgmap v1140: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 331 op/s 2024-08-31T07:37:17.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:16 smithi073 ceph-mon[21995]: pgmap v1140: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 331 op/s 2024-08-31T07:37:17.251 INFO:tasks.quiescer:done joining Quiescers 2024-08-31T07:37:17.251 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-08-31T07:37:17.264 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-08-31T07:37:17.264 INFO:teuthology.task.exec:Executing custom commands... 2024-08-31T07:37:17.264 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-08-31T07:37:17.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-08-31T07:37:17.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-08-31T07:37:17.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-31T07:37:17.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.612+0000 7fcffbe00640 1 -- 172.21.15.32:0/3553838179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff41091a0 msgr2=0x7fcff41095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:17.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.612+0000 7fcffbe00640 1 --2- 172.21.15.32:0/3553838179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff41091a0 0x7fcff41095b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009900 tx=0x7fcfe002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.612+0000 7fcffbe00640 1 -- 172.21.15.32:0/3553838179 shutdown_connections 2024-08-31T07:37:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.612+0000 7fcffbe00640 1 --2- 172.21.15.32:0/3553838179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcff4109af0 0x7fcff4110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.612+0000 7fcffbe00640 1 --2- 172.21.15.32:0/3553838179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff41091a0 0x7fcff41095b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.612+0000 7fcffbe00640 1 --2- 172.21.15.32:0/3553838179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcff4103870 0x7fcff4103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.612+0000 7fcffbe00640 1 -- 172.21.15.32:0/3553838179 >> 172.21.15.32:0/3553838179 conn(0x7fcff40ff270 msgr2=0x7fcff41016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.612+0000 7fcffbe00640 1 -- 172.21.15.32:0/3553838179 shutdown_connections 2024-08-31T07:37:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.612+0000 7fcffbe00640 1 -- 172.21.15.32:0/3553838179 wait complete. 2024-08-31T07:37:17.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.613+0000 7fcffbe00640 1 Processor -- start 2024-08-31T07:37:17.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.613+0000 7fcffbe00640 1 -- start start 2024-08-31T07:37:17.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcffbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcff4103870 0x7fcff40754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:17.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcffbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff41091a0 0x7fcff40759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:17.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcffbe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcff4109af0 0x7fcff4075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:17.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcffbe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4113780 con 0x7fcff41091a0 2024-08-31T07:37:17.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcffbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4113600 con 0x7fcff4103870 2024-08-31T07:37:17.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcffbe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4113900 con 0x7fcff4109af0 2024-08-31T07:37:17.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff41091a0 0x7fcff40759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:17.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcff4103870 0x7fcff40754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:17.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff41091a0 0x7fcff40759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58140/0 (socket says 172.21.15.32:58140) 2024-08-31T07:37:17.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcff4103870 0x7fcff40754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35850/0 (socket says 172.21.15.32:35850) 2024-08-31T07:37:17.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff8a00640 1 -- 172.21.15.32:0/2014823568 learned_addr learned my addr 172.21.15.32:0/2014823568 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:17.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff9e00640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcff4109af0 0x7fcff4075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:17.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff9400640 1 -- 172.21.15.32:0/2014823568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcff4109af0 msgr2=0x7fcff4075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:17.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff9400640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcff4109af0 0x7fcff4075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff9400640 1 -- 172.21.15.32:0/2014823568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff41091a0 msgr2=0x7fcff40759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff9400640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff41091a0 0x7fcff40759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff9400640 1 -- 172.21.15.32:0/2014823568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff41a8470 con 0x7fcff4103870 2024-08-31T07:37:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.614+0000 7fcff9e00640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcff4109af0 0x7fcff4075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.615+0000 7fcff9400640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcff4103870 0x7fcff40754b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e9f0 tx=0x7fcfe800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.615+0000 7fcff2000640 1 -- 172.21.15.32:0/2014823568 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800ccc0 con 0x7fcff4103870 2024-08-31T07:37:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.615+0000 7fcffbe00640 1 -- 172.21.15.32:0/2014823568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff41a8760 con 0x7fcff4103870 2024-08-31T07:37:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.615+0000 7fcff2000640 1 -- 172.21.15.32:0/2014823568 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfe80108e0 con 0x7fcff4103870 2024-08-31T07:37:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.615+0000 7fcff2000640 1 -- 172.21.15.32:0/2014823568 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8018880 con 0x7fcff4103870 2024-08-31T07:37:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.615+0000 7fcffbe00640 1 -- 172.21.15.32:0/2014823568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff41a8d00 con 0x7fcff4103870 2024-08-31T07:37:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.616+0000 7fcffbe00640 1 -- 172.21.15.32:0/2014823568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff4104ae0 con 0x7fcff4103870 2024-08-31T07:37:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.621+0000 7fcff2000640 1 -- 172.21.15.32:0/2014823568 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcfe8010430 con 0x7fcff4103870 2024-08-31T07:37:17.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.624+0000 7fcff2000640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfdc0b4cd0 0x7fcfdc0b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:17.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.624+0000 7fcff2000640 1 -- 172.21.15.32:0/2014823568 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7fcfe80e5c50 con 0x7fcff4103870 2024-08-31T07:37:17.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.624+0000 7fcff8a00640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfdc0b4cd0 0x7fcfdc0b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:17.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.625+0000 7fcff8a00640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfdc0b4cd0 0x7fcfdc0b71e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcff4108c40 tx=0x7fcfe0009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:17.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.627+0000 7fcff2000640 1 -- 172.21.15.32:0/2014823568 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcfe80ac440 con 0x7fcff4103870 2024-08-31T07:37:17.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.858+0000 7fcffbe00640 1 -- 172.21.15.32:0/2014823568 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fcff410b930 con 0x7fcfdc0b4cd0 2024-08-31T07:37:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.871+0000 7fcff2000640 1 -- 172.21.15.32:0/2014823568 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fcff410b930 con 0x7fcfdc0b4cd0 2024-08-31T07:37:17.872 INFO:teuthology.orchestra.run.smithi032.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-31T00:00:00", "created": "2024-08-31T07:12:23", "first": "2024-08-31T07:13:00", "last": "2024-08-31T07:37:00", "last_pruned": "2024-08-31T07:37:00", "created_count": 25, "pruned_count": 19, "active": true} 2024-08-31T07:37:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 -- 172.21.15.32:0/2014823568 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfdc0b4cd0 msgr2=0x7fcfdc0b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfdc0b4cd0 0x7fcfdc0b71e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcff4108c40 tx=0x7fcfe0009560 comp rx=0 tx=0).stop 2024-08-31T07:37:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 -- 172.21.15.32:0/2014823568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcff4103870 msgr2=0x7fcff40754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcff4103870 0x7fcff40754b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e9f0 tx=0x7fcfe800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 -- 172.21.15.32:0/2014823568 shutdown_connections 2024-08-31T07:37:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fcfdc0b4cd0 0x7fcfdc0b71e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcff4109af0 0x7fcff4075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff41091a0 0x7fcff40759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 --2- 172.21.15.32:0/2014823568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcff4103870 0x7fcff40754b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 -- 172.21.15.32:0/2014823568 >> 172.21.15.32:0/2014823568 conn(0x7fcff40ff270 msgr2=0x7fcff4101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 -- 172.21.15.32:0/2014823568 shutdown_connections 2024-08-31T07:37:17.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:17.874+0000 7fcffbe00640 1 -- 172.21.15.32:0/2014823568 wait complete. 2024-08-31T07:37:17.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-31T07:37:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.161+0000 7f71bfe00640 1 -- 172.21.15.32:0/4278266856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c01058e0 msgr2=0x7f71c0105cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.161+0000 7f71bfe00640 1 --2- 172.21.15.32:0/4278266856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c01058e0 0x7f71c0105cf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009900 tx=0x7f71b002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.161+0000 7f71bfe00640 1 -- 172.21.15.32:0/4278266856 shutdown_connections 2024-08-31T07:37:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.161+0000 7f71bfe00640 1 --2- 172.21.15.32:0/4278266856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c010bb60 0x7f71c0112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.161+0000 7f71bfe00640 1 --2- 172.21.15.32:0/4278266856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c010b210 0x7f71c010b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.161+0000 7f71bfe00640 1 --2- 172.21.15.32:0/4278266856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c01058e0 0x7f71c0105cf0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.161+0000 7f71bfe00640 1 -- 172.21.15.32:0/4278266856 >> 172.21.15.32:0/4278266856 conn(0x7f71c0101320 msgr2=0x7f71c0103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.161+0000 7f71bfe00640 1 -- 172.21.15.32:0/4278266856 shutdown_connections 2024-08-31T07:37:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.163+0000 7f71bfe00640 1 -- 172.21.15.32:0/4278266856 wait complete. 2024-08-31T07:37:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.164+0000 7f71bfe00640 1 Processor -- start 2024-08-31T07:37:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.165+0000 7f71bfe00640 1 -- start start 2024-08-31T07:37:18.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.165+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c01058e0 0x7f71c019c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:18.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.165+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c010b210 0x7f71c019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:18.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.165+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c010bb60 0x7f71c01a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:18.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.165+0000 7f71bfe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0115800 con 0x7f71c01058e0 2024-08-31T07:37:18.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.165+0000 7f71bfe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0115680 con 0x7f71c010bb60 2024-08-31T07:37:18.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.165+0000 7f71bfe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0115980 con 0x7f71c010b210 2024-08-31T07:37:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.165+0000 7f71be000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c010b210 0x7f71c019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.165+0000 7f71bea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c01058e0 0x7f71c019c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71be000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c010b210 0x7f71c019c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51618/0 (socket says 172.21.15.32:51618) 2024-08-31T07:37:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71be000640 1 -- 172.21.15.32:0/208194163 learned_addr learned my addr 172.21.15.32:0/208194163 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71bf400640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c010bb60 0x7f71c01a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71be000640 1 -- 172.21.15.32:0/208194163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c010bb60 msgr2=0x7f71c01a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71be000640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c010bb60 0x7f71c01a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71be000640 1 -- 172.21.15.32:0/208194163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c01058e0 msgr2=0x7f71c019c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71be000640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c01058e0 0x7f71c019c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71be000640 1 -- 172.21.15.32:0/208194163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c01a2150 con 0x7f71c010b210 2024-08-31T07:37:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71bf400640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c010bb60 0x7f71c01a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71bea00640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c01058e0 0x7f71c019c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.166+0000 7f71be000640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c010b210 0x7f71c019c7c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00e9f0 tx=0x7f71ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:18.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.167+0000 7f71c4a00640 1 -- 172.21.15.32:0/208194163 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00ccc0 con 0x7f71c010b210 2024-08-31T07:37:18.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.167+0000 7f71bfe00640 1 -- 172.21.15.32:0/208194163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c01a2440 con 0x7f71c010b210 2024-08-31T07:37:18.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.167+0000 7f71c4a00640 1 -- 172.21.15.32:0/208194163 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ac0108e0 con 0x7f71c010b210 2024-08-31T07:37:18.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.167+0000 7f71c4a00640 1 -- 172.21.15.32:0/208194163 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac018860 con 0x7f71c010b210 2024-08-31T07:37:18.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.167+0000 7f71bfe00640 1 -- 172.21.15.32:0/208194163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01a8bd0 con 0x7f71c010b210 2024-08-31T07:37:18.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.168+0000 7f71bfe00640 1 -- 172.21.15.32:0/208194163 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c0105cf0 con 0x7f71c010b210 2024-08-31T07:37:18.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.170+0000 7f71c4a00640 1 -- 172.21.15.32:0/208194163 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f71ac010430 con 0x7f71c010b210 2024-08-31T07:37:18.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.172+0000 7f71c4a00640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71a40b4c90 0x7f71a40b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:18.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.172+0000 7f71c4a00640 1 -- 172.21.15.32:0/208194163 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7f71ac0e5c80 con 0x7f71c010b210 2024-08-31T07:37:18.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.172+0000 7f71bea00640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71a40b4c90 0x7f71a40b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:18.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.173+0000 7f71bea00640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71a40b4c90 0x7f71a40b71a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f71b0031440 tx=0x7f71b0032ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:18.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.174+0000 7f71c4a00640 1 -- 172.21.15.32:0/208194163 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f71ac0ac410 con 0x7f71c010b210 2024-08-31T07:37:18.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:18 smithi032 ceph-mon[17252]: osdmap e118: 12 total, 12 up, 12 in 2024-08-31T07:37:18.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:18 smithi032 ceph-mon[17252]: pgmap v1142: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 164 op/s 2024-08-31T07:37:18.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:18 smithi032 ceph-mon[17252]: from='client.30113 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:37:18.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.397+0000 7f71bfe00640 1 -- 172.21.15.32:0/208194163 --> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f71c00635a0 con 0x7f71a40b4c90 2024-08-31T07:37:18.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.411+0000 7f71c4a00640 1 -- 172.21.15.32:0/208194163 <== mgr.14220 v2:172.21.15.32:6800/4041113440 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f71c00635a0 con 0x7f71a40b4c90 2024-08-31T07:37:18.412 INFO:teuthology.orchestra.run.smithi032.stdout:/ 1m 6m3h 2024-08-31T07:37:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 -- 172.21.15.32:0/208194163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71a40b4c90 msgr2=0x7f71a40b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71a40b4c90 0x7f71a40b71a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f71b0031440 tx=0x7f71b0032ed0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 -- 172.21.15.32:0/208194163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c010b210 msgr2=0x7f71c019c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c010b210 0x7f71c019c7c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00e9f0 tx=0x7f71ac00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 -- 172.21.15.32:0/208194163 shutdown_connections 2024-08-31T07:37:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f71a40b4c90 0x7f71a40b71a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c010bb60 0x7f71c01a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71c010b210 0x7f71c019c7c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 --2- 172.21.15.32:0/208194163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c01058e0 0x7f71c019c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 -- 172.21.15.32:0/208194163 >> 172.21.15.32:0/208194163 conn(0x7f71c0101320 msgr2=0x7f71c0103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 -- 172.21.15.32:0/208194163 shutdown_connections 2024-08-31T07:37:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.414+0000 7f71bfe00640 1 -- 172.21.15.32:0/208194163 wait complete. 2024-08-31T07:37:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:18 smithi073 ceph-mon[21995]: osdmap e118: 12 total, 12 up, 12 in 2024-08-31T07:37:18.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:18 smithi073 ceph-mon[21995]: pgmap v1142: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 164 op/s 2024-08-31T07:37:18.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:18 smithi073 ceph-mon[21995]: from='client.30113 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:37:18.438 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-08-31T07:37:18.452 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-08-31T07:37:18.452 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-08-31T07:37:18.456 INFO:tasks.fwd_scrub:done joining 2024-08-31T07:37:18.456 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-31T07:37:18.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-31T07:37:18.501 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:18 smithi084 ceph-mon[22387]: osdmap e118: 12 total, 12 up, 12 in 2024-08-31T07:37:18.501 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:18 smithi084 ceph-mon[22387]: pgmap v1142: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 164 op/s 2024-08-31T07:37:18.502 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:18 smithi084 ceph-mon[22387]: from='client.30113 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:37:18.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.726+0000 7f7d3fe00640 1 -- 172.21.15.32:0/2298924496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d40104510 msgr2=0x7f7d40104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:18.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.726+0000 7f7d3fe00640 1 --2- 172.21.15.32:0/2298924496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d40104510 0x7f7d40104920 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009900 tx=0x7f7d3002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.726+0000 7f7d3fe00640 1 -- 172.21.15.32:0/2298924496 shutdown_connections 2024-08-31T07:37:18.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.726+0000 7f7d3fe00640 1 --2- 172.21.15.32:0/2298924496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d4010d930 0x7f7d4010fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.726+0000 7f7d3fe00640 1 --2- 172.21.15.32:0/2298924496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d40104e60 0x7f7d4010d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.726+0000 7f7d3fe00640 1 --2- 172.21.15.32:0/2298924496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d40104510 0x7f7d40104920 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.726+0000 7f7d3fe00640 1 -- 172.21.15.32:0/2298924496 >> 172.21.15.32:0/2298924496 conn(0x7f7d400fdf80 msgr2=0x7f7d401003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:18.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.726+0000 7f7d3fe00640 1 -- 172.21.15.32:0/2298924496 shutdown_connections 2024-08-31T07:37:18.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.726+0000 7f7d3fe00640 1 -- 172.21.15.32:0/2298924496 wait complete. 2024-08-31T07:37:18.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.727+0000 7f7d3fe00640 1 Processor -- start 2024-08-31T07:37:18.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.727+0000 7f7d3fe00640 1 -- start start 2024-08-31T07:37:18.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3fe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d40104510 0x7f7d40102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:18.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d40104e60 0x7f7d40102780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:18.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d4010d930 0x7f7d40102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:18.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d40113580 con 0x7f7d4010d930 2024-08-31T07:37:18.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d40113400 con 0x7f7d40104e60 2024-08-31T07:37:18.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3fe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7d40113700 con 0x7f7d40104510 2024-08-31T07:37:18.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d40104510 0x7f7d40102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:18.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3ea00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d40104510 0x7f7d40102240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51636/0 (socket says 172.21.15.32:51636) 2024-08-31T07:37:18.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3ea00640 1 -- 172.21.15.32:0/632094956 learned_addr learned my addr 172.21.15.32:0/632094956 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:18.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d37e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d40104e60 0x7f7d40102780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:18.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3f400640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d4010d930 0x7f7d40102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:18.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.728+0000 7f7d3ea00640 1 -- 172.21.15.32:0/632094956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d40104e60 msgr2=0x7f7d40102780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:18.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d3ea00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d40104e60 0x7f7d40102780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d3ea00640 1 -- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d4010d930 msgr2=0x7f7d40102cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d3ea00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d4010d930 0x7f7d40102cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d3ea00640 1 -- 172.21.15.32:0/632094956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d401a8290 con 0x7f7d40104510 2024-08-31T07:37:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d3f400640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d4010d930 0x7f7d40102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:37:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d3ea00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d40104510 0x7f7d40102240 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7d30005940 tx=0x7f7d30004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d37400640 1 -- 172.21.15.32:0/632094956 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3003f030 con 0x7f7d40104510 2024-08-31T07:37:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d37400640 1 -- 172.21.15.32:0/632094956 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d30003aa0 con 0x7f7d40104510 2024-08-31T07:37:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d3fe00640 1 -- 172.21.15.32:0/632094956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d401a8520 con 0x7f7d40104510 2024-08-31T07:37:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d37400640 1 -- 172.21.15.32:0/632094956 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30035070 con 0x7f7d40104510 2024-08-31T07:37:18.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.729+0000 7f7d3fe00640 1 -- 172.21.15.32:0/632094956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d401a8a60 con 0x7f7d40104510 2024-08-31T07:37:18.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.730+0000 7f7d3fe00640 1 -- 172.21.15.32:0/632094956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d40104920 con 0x7f7d40104510 2024-08-31T07:37:18.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.733+0000 7f7d37400640 1 -- 172.21.15.32:0/632094956 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7d30042030 con 0x7f7d40104510 2024-08-31T07:37:18.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.735+0000 7f7d37400640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d240b4d50 0x7f7d240b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:18.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.735+0000 7f7d37400640 1 -- 172.21.15.32:0/632094956 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7f7d30105e60 con 0x7f7d40104510 2024-08-31T07:37:18.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.735+0000 7f7d37e00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d240b4d50 0x7f7d240b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:18.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.736+0000 7f7d37e00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d240b4d50 0x7f7d240b7260 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7d40100850 tx=0x7f7d28005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.736+0000 7f7d37400640 1 -- 172.21.15.32:0/632094956 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7d300ccc40 con 0x7f7d40104510 2024-08-31T07:37:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.932+0000 7f7d3fe00640 1 -- 172.21.15.32:0/632094956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d401a8e20 con 0x7f7d40104510 2024-08-31T07:37:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.934+0000 7f7d37400640 1 -- 172.21.15.32:0/632094956 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+23015 (secure 0 0 0) 0x7f7d300cc540 con 0x7f7d40104510 2024-08-31T07:37:18.935 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:37:18.935 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:37:17.098703+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:37:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.936+0000 7f7d3fe00640 1 -- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d240b4d50 msgr2=0x7f7d240b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.936+0000 7f7d3fe00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d240b4d50 0x7f7d240b7260 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7d40100850 tx=0x7f7d28005f90 comp rx=0 tx=0).stop 2024-08-31T07:37:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.936+0000 7f7d3fe00640 1 -- 172.21.15.32:0/632094956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d40104510 msgr2=0x7f7d40102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.936+0000 7f7d3fe00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d40104510 0x7f7d40102240 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7d30005940 tx=0x7f7d30004270 comp rx=0 tx=0).stop 2024-08-31T07:37:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.936+0000 7f7d3fe00640 1 -- 172.21.15.32:0/632094956 shutdown_connections 2024-08-31T07:37:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.936+0000 7f7d3fe00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f7d240b4d50 0x7f7d240b7260 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.936+0000 7f7d3fe00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d4010d930 0x7f7d40102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.936+0000 7f7d3fe00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d40104e60 0x7f7d40102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.936+0000 7f7d3fe00640 1 --2- 172.21.15.32:0/632094956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d40104510 0x7f7d40102240 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.936+0000 7f7d3fe00640 1 -- 172.21.15.32:0/632094956 >> 172.21.15.32:0/632094956 conn(0x7f7d400fdf80 msgr2=0x7f7d401074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.937+0000 7f7d3fe00640 1 -- 172.21.15.32:0/632094956 shutdown_connections 2024-08-31T07:37:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:18.937+0000 7f7d3fe00640 1 -- 172.21.15.32:0/632094956 wait complete. 2024-08-31T07:37:18.965 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-31T07:04:45.144128+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-31T07:09:43.795937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-31T07:09:46.609242+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 117, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-31T07:37:18.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-31T07:37:19.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.204+0000 7fd8ad000640 1 -- 172.21.15.32:0/2712960633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8077130 msgr2=0x7fd8a8075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.204+0000 7fd8ad000640 1 --2- 172.21.15.32:0/2712960633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8077130 0x7fd8a8075550 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd898009900 tx=0x7fd89802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.204+0000 7fd8ad000640 1 -- 172.21.15.32:0/2712960633 shutdown_connections 2024-08-31T07:37:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.204+0000 7fd8ad000640 1 --2- 172.21.15.32:0/2712960633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8113f80 0x7fd8a8116450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.204+0000 7fd8ad000640 1 --2- 172.21.15.32:0/2712960633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a8075b70 0x7fd8a8075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.204+0000 7fd8ad000640 1 --2- 172.21.15.32:0/2712960633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8077130 0x7fd8a8075550 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.204+0000 7fd8ad000640 1 -- 172.21.15.32:0/2712960633 >> 172.21.15.32:0/2712960633 conn(0x7fd8a80ff120 msgr2=0x7fd8a8101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.204+0000 7fd8ad000640 1 -- 172.21.15.32:0/2712960633 shutdown_connections 2024-08-31T07:37:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.204+0000 7fd8ad000640 1 -- 172.21.15.32:0/2712960633 wait complete. 2024-08-31T07:37:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.205+0000 7fd8ad000640 1 Processor -- start 2024-08-31T07:37:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.205+0000 7fd8ad000640 1 -- start start 2024-08-31T07:37:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd8ad000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8075b70 0x7fd8a81a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd8ad000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8077130 0x7fd8a81a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd8ad000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a8113f80 0x7fd8a81a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd8ad000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8119bc0 con 0x7fd8a8077130 2024-08-31T07:37:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd8ad000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8119a40 con 0x7fd8a8113f80 2024-08-31T07:37:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd8ad000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8119d40 con 0x7fd8a8075b70 2024-08-31T07:37:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd8a7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8075b70 0x7fd8a81a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd8a7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8075b70 0x7fd8a81a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51646/0 (socket says 172.21.15.32:51646) 2024-08-31T07:37:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd8a7400640 1 -- 172.21.15.32:0/1258093464 learned_addr learned my addr 172.21.15.32:0/1258093464 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd89fe00640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8077130 0x7fd8a81a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.206+0000 7fd8a7e00640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a8113f80 0x7fd8a81a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8a7400640 1 -- 172.21.15.32:0/1258093464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a8113f80 msgr2=0x7fd8a81a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8a7400640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a8113f80 0x7fd8a81a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8a7400640 1 -- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8077130 msgr2=0x7fd8a81a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8a7400640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8077130 0x7fd8a81a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8a7400640 1 -- 172.21.15.32:0/1258093464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a81a64b0 con 0x7fd8a8075b70 2024-08-31T07:37:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd89fe00640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8077130 0x7fd8a81a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8a7400640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8075b70 0x7fd8a81a0580 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd898005940 tx=0x7fd8980040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8a4c00640 1 -- 172.21.15.32:0/1258093464 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89803f030 con 0x7fd8a8075b70 2024-08-31T07:37:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8ad000640 1 -- 172.21.15.32:0/1258093464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a81a6740 con 0x7fd8a8075b70 2024-08-31T07:37:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8a4c00640 1 -- 172.21.15.32:0/1258093464 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8980044c0 con 0x7fd8a8075b70 2024-08-31T07:37:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8a4c00640 1 -- 172.21.15.32:0/1258093464 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898035070 con 0x7fd8a8075b70 2024-08-31T07:37:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.207+0000 7fd8ad000640 1 -- 172.21.15.32:0/1258093464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a81acf30 con 0x7fd8a8075b70 2024-08-31T07:37:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.208+0000 7fd8ad000640 1 -- 172.21.15.32:0/1258093464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a8076a60 con 0x7fd8a8075b70 2024-08-31T07:37:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.211+0000 7fd8a4c00640 1 -- 172.21.15.32:0/1258093464 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd898042030 con 0x7fd8a8075b70 2024-08-31T07:37:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.212+0000 7fd8a4c00640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4ca0 0x7fd88c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.212+0000 7fd8a4c00640 1 -- 172.21.15.32:0/1258093464 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7fd898105b10 con 0x7fd8a8075b70 2024-08-31T07:37:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.212+0000 7fd89fe00640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4ca0 0x7fd88c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.213+0000 7fd89fe00640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4ca0 0x7fd88c0b71b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8116000 tx=0x7fd890005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.214+0000 7fd8a4c00640 1 -- 172.21.15.32:0/1258093464 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd8980cc2a0 con 0x7fd8a8075b70 2024-08-31T07:37:19.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:19 smithi032 ceph-mon[17252]: from='client.30124 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:37:19.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:19 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/632094956' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:37:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.411+0000 7fd8ad000640 1 -- 172.21.15.32:0/1258093464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8a8112ce0 con 0x7fd8a8075b70 2024-08-31T07:37:19.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.412+0000 7fd8a4c00640 1 -- 172.21.15.32:0/1258093464 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7fd8980d0a60 con 0x7fd8a8075b70 2024-08-31T07:37:19.413 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-08-31T07:37:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.414+0000 7fd8ad000640 1 -- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4ca0 msgr2=0x7fd88c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.414+0000 7fd8ad000640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4ca0 0x7fd88c0b71b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8116000 tx=0x7fd890005f90 comp rx=0 tx=0).stop 2024-08-31T07:37:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.414+0000 7fd8ad000640 1 -- 172.21.15.32:0/1258093464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8075b70 msgr2=0x7fd8a81a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.414+0000 7fd8ad000640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8075b70 0x7fd8a81a0580 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd898005940 tx=0x7fd8980040c0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.415+0000 7fd8ad000640 1 -- 172.21.15.32:0/1258093464 shutdown_connections 2024-08-31T07:37:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.415+0000 7fd8ad000640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fd88c0b4ca0 0x7fd88c0b71b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.415+0000 7fd8ad000640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8a8113f80 0x7fd8a81a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.415+0000 7fd8ad000640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a8077130 0x7fd8a81a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.415+0000 7fd8ad000640 1 --2- 172.21.15.32:0/1258093464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8a8075b70 0x7fd8a81a0580 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.415+0000 7fd8ad000640 1 -- 172.21.15.32:0/1258093464 >> 172.21.15.32:0/1258093464 conn(0x7fd8a80ff120 msgr2=0x7fd8a810fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.415+0000 7fd8ad000640 1 -- 172.21.15.32:0/1258093464 shutdown_connections 2024-08-31T07:37:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.415+0000 7fd8ad000640 1 -- 172.21.15.32:0/1258093464 wait complete. 2024-08-31T07:37:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:19 smithi073 ceph-mon[21995]: from='client.30124 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:37:19.430 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:19 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/632094956' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:37:19.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-31T07:37:19.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:19 smithi084 ceph-mon[22387]: from='client.30124 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T07:37:19.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:19 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/632094956' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:37:19.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.667+0000 7f5083000640 1 -- 172.21.15.32:0/3454192061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c109180 msgr2=0x7f507c109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:19.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.667+0000 7f5083000640 1 --2- 172.21.15.32:0/3454192061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c109180 0x7f507c109590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5064009860 tx=0x7f506402a340 comp rx=0 tx=0).stop 2024-08-31T07:37:19.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.667+0000 7f5083000640 1 -- 172.21.15.32:0/3454192061 shutdown_connections 2024-08-31T07:37:19.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.667+0000 7f5083000640 1 --2- 172.21.15.32:0/3454192061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109ad0 0x7f507c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.667+0000 7f5083000640 1 --2- 172.21.15.32:0/3454192061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c109180 0x7f507c109590 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.667+0000 7f5083000640 1 --2- 172.21.15.32:0/3454192061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103850 0x7f507c103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.667+0000 7f5083000640 1 -- 172.21.15.32:0/3454192061 >> 172.21.15.32:0/3454192061 conn(0x7f507c0ff290 msgr2=0x7f507c1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:19.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.667+0000 7f5083000640 1 -- 172.21.15.32:0/3454192061 shutdown_connections 2024-08-31T07:37:19.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.667+0000 7f5083000640 1 -- 172.21.15.32:0/3454192061 wait complete. 2024-08-31T07:37:19.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.668+0000 7f5083000640 1 Processor -- start 2024-08-31T07:37:19.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.668+0000 7f5083000640 1 -- start start 2024-08-31T07:37:19.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f5083000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103850 0x7f507c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f5083000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c109180 0x7f507c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f5083000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109ad0 0x7f507c19f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f5083000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f507c113740 con 0x7f507c109ad0 2024-08-31T07:37:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f5083000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1135c0 con 0x7f507c109180 2024-08-31T07:37:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f5083000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1138c0 con 0x7f507c103850 2024-08-31T07:37:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f5081000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109ad0 0x7f507c19f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f5073e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c109180 0x7f507c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f5081000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109ad0 0x7f507c19f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58186/0 (socket says 172.21.15.32:58186) 2024-08-31T07:37:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f507be00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103850 0x7f507c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.669+0000 7f5081000640 1 -- 172.21.15.32:0/3648463391 learned_addr learned my addr 172.21.15.32:0/3648463391 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f507be00640 1 -- 172.21.15.32:0/3648463391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c109180 msgr2=0x7f507c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f507be00640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c109180 0x7f507c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f507be00640 1 -- 172.21.15.32:0/3648463391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109ad0 msgr2=0x7f507c19f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f507be00640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109ad0 0x7f507c19f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f507be00640 1 -- 172.21.15.32:0/3648463391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c1a00f0 con 0x7f507c103850 2024-08-31T07:37:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f5073e00640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c109180 0x7f507c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f5081000640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109ad0 0x7f507c19f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f507be00640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103850 0x7f507c19a1c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f506c00e9f0 tx=0x7f506c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f5079600640 1 -- 172.21.15.32:0/3648463391 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c00ccc0 con 0x7f507c103850 2024-08-31T07:37:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f5079600640 1 -- 172.21.15.32:0/3648463391 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f506c0108e0 con 0x7f507c103850 2024-08-31T07:37:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f5083000640 1 -- 172.21.15.32:0/3648463391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507c1a03e0 con 0x7f507c103850 2024-08-31T07:37:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f5079600640 1 -- 172.21.15.32:0/3648463391 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c018880 con 0x7f507c103850 2024-08-31T07:37:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.670+0000 7f5083000640 1 -- 172.21.15.32:0/3648463391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507c1a6b70 con 0x7f507c103850 2024-08-31T07:37:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.672+0000 7f5083000640 1 -- 172.21.15.32:0/3648463391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507c104ac0 con 0x7f507c103850 2024-08-31T07:37:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.674+0000 7f5079600640 1 -- 172.21.15.32:0/3648463391 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f506c010430 con 0x7f507c103850 2024-08-31T07:37:19.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.675+0000 7f5079600640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50600b4ca0 0x7f50600b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.676+0000 7f5079600640 1 -- 172.21.15.32:0/3648463391 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7f506c0e5ce0 con 0x7f507c103850 2024-08-31T07:37:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.676+0000 7f5073e00640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50600b4ca0 0x7f50600b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.677+0000 7f5073e00640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50600b4ca0 0x7f50600b71b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5064005940 tx=0x7f50640058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.677+0000 7f5079600640 1 -- 172.21.15.32:0/3648463391 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f506c0ac470 con 0x7f507c103850 2024-08-31T07:37:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.877+0000 7f5083000640 1 -- 172.21.15.32:0/3648463391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f507c02e430 con 0x7f507c103850 2024-08-31T07:37:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.878+0000 7f5079600640 1 -- 172.21.15.32:0/3648463391 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7f506c0b0c30 con 0x7f507c103850 2024-08-31T07:37:19.879 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-08-31T07:37:19.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.880+0000 7f5083000640 1 -- 172.21.15.32:0/3648463391 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50600b4ca0 msgr2=0x7f50600b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:19.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.880+0000 7f5083000640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50600b4ca0 0x7f50600b71b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5064005940 tx=0x7f50640058d0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.880+0000 7f5083000640 1 -- 172.21.15.32:0/3648463391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103850 msgr2=0x7f507c19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:19.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.880+0000 7f5083000640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103850 0x7f507c19a1c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f506c00e9f0 tx=0x7f506c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.880+0000 7f5083000640 1 -- 172.21.15.32:0/3648463391 shutdown_connections 2024-08-31T07:37:19.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.880+0000 7f5083000640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f50600b4ca0 0x7f50600b71b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.880+0000 7f5083000640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c109ad0 0x7f507c19f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.880+0000 7f5083000640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f507c109180 0x7f507c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.880+0000 7f5083000640 1 --2- 172.21.15.32:0/3648463391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f507c103850 0x7f507c19a1c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:19.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.880+0000 7f5083000640 1 -- 172.21.15.32:0/3648463391 >> 172.21.15.32:0/3648463391 conn(0x7f507c0ff290 msgr2=0x7f507c1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:19.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.881+0000 7f5083000640 1 -- 172.21.15.32:0/3648463391 shutdown_connections 2024-08-31T07:37:19.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:19.881+0000 7f5083000640 1 -- 172.21.15.32:0/3648463391 wait complete. 2024-08-31T07:37:19.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-31T07:37:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.144+0000 7fb33da00640 1 -- 172.21.15.32:0/3464893619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb338109950 msgr2=0x7fb338110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.144+0000 7fb33da00640 1 --2- 172.21.15.32:0/3464893619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb338109950 0x7fb338110180 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb330009ca0 tx=0x7fb33002a8b0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.144+0000 7fb33da00640 1 -- 172.21.15.32:0/3464893619 shutdown_connections 2024-08-31T07:37:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.144+0000 7fb33da00640 1 --2- 172.21.15.32:0/3464893619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb338109950 0x7fb338110180 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.144+0000 7fb33da00640 1 --2- 172.21.15.32:0/3464893619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb338109000 0x7fb338109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.144+0000 7fb33da00640 1 --2- 172.21.15.32:0/3464893619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3381036d0 0x7fb338103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.144+0000 7fb33da00640 1 -- 172.21.15.32:0/3464893619 >> 172.21.15.32:0/3464893619 conn(0x7fb3380ff130 msgr2=0x7fb338101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.144+0000 7fb33da00640 1 -- 172.21.15.32:0/3464893619 shutdown_connections 2024-08-31T07:37:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.144+0000 7fb33da00640 1 -- 172.21.15.32:0/3464893619 wait complete. 2024-08-31T07:37:20.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.145+0000 7fb33da00640 1 Processor -- start 2024-08-31T07:37:20.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.145+0000 7fb33da00640 1 -- start start 2024-08-31T07:37:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb33da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3381036d0 0x7fb3380754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb33da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb338109000 0x7fb3380759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb33da00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb338109950 0x7fb338075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb33da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb338113650 con 0x7fb3381036d0 2024-08-31T07:37:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb33da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381134d0 con 0x7fb338109000 2024-08-31T07:37:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb33da00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381137d0 con 0x7fb338109950 2024-08-31T07:37:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb337e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3381036d0 0x7fb3380754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb337400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb338109000 0x7fb3380759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb337400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb338109000 0x7fb3380759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35948/0 (socket says 172.21.15.32:35948) 2024-08-31T07:37:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb337e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3381036d0 0x7fb3380754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58194/0 (socket says 172.21.15.32:58194) 2024-08-31T07:37:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb337400640 1 -- 172.21.15.32:0/571927623 learned_addr learned my addr 172.21.15.32:0/571927623 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb33d000640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb338109950 0x7fb338075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb337e00640 1 -- 172.21.15.32:0/571927623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb338109950 msgr2=0x7fb338075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.146+0000 7fb337e00640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb338109950 0x7fb338075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.147+0000 7fb337e00640 1 -- 172.21.15.32:0/571927623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb338109000 msgr2=0x7fb3380759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.147+0000 7fb337e00640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb338109000 0x7fb3380759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.147+0000 7fb337e00640 1 -- 172.21.15.32:0/571927623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3381a8260 con 0x7fb3381036d0 2024-08-31T07:37:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.147+0000 7fb337e00640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3381036d0 0x7fb3380754b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb32800e9f0 tx=0x7fb32800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.147+0000 7fb334c00640 1 -- 172.21.15.32:0/571927623 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32800ccc0 con 0x7fb3381036d0 2024-08-31T07:37:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.147+0000 7fb334c00640 1 -- 172.21.15.32:0/571927623 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3280108e0 con 0x7fb3381036d0 2024-08-31T07:37:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.147+0000 7fb334c00640 1 -- 172.21.15.32:0/571927623 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328018880 con 0x7fb3381036d0 2024-08-31T07:37:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.147+0000 7fb33da00640 1 -- 172.21.15.32:0/571927623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3381a8550 con 0x7fb3381036d0 2024-08-31T07:37:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.147+0000 7fb33da00640 1 -- 172.21.15.32:0/571927623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3381a8a50 con 0x7fb3381036d0 2024-08-31T07:37:20.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.149+0000 7fb33da00640 1 -- 172.21.15.32:0/571927623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb338103ae0 con 0x7fb3381036d0 2024-08-31T07:37:20.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.151+0000 7fb334c00640 1 -- 172.21.15.32:0/571927623 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb328010430 con 0x7fb3381036d0 2024-08-31T07:37:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.152+0000 7fb334c00640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb31c0b4cd0 0x7fb31c0b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.153+0000 7fb334c00640 1 -- 172.21.15.32:0/571927623 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7fb3280e5cc0 con 0x7fb3381036d0 2024-08-31T07:37:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.153+0000 7fb337400640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb31c0b4cd0 0x7fb31c0b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.153+0000 7fb337400640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb31c0b4cd0 0x7fb31c0b71e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb3381029f0 tx=0x7fb324005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.154+0000 7fb334c00640 1 -- 172.21.15.32:0/571927623 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb3280ac450 con 0x7fb3381036d0 2024-08-31T07:37:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.350+0000 7fb33da00640 1 -- 172.21.15.32:0/571927623 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb33802e430 con 0x7fb3381036d0 2024-08-31T07:37:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.351+0000 7fb334c00640 1 -- 172.21.15.32:0/571927623 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7fb3280b0c10 con 0x7fb3381036d0 2024-08-31T07:37:20.352 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-08-31T07:37:20.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 -- 172.21.15.32:0/571927623 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb31c0b4cd0 msgr2=0x7fb31c0b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:20.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb31c0b4cd0 0x7fb31c0b71e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb3381029f0 tx=0x7fb324005f90 comp rx=0 tx=0).stop 2024-08-31T07:37:20.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 -- 172.21.15.32:0/571927623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3381036d0 msgr2=0x7fb3380754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:20.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3381036d0 0x7fb3380754b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb32800e9f0 tx=0x7fb32800eeb0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 -- 172.21.15.32:0/571927623 shutdown_connections 2024-08-31T07:37:20.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fb31c0b4cd0 0x7fb31c0b71e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb338109950 0x7fb338075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb338109000 0x7fb3380759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 --2- 172.21.15.32:0/571927623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3381036d0 0x7fb3380754b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 -- 172.21.15.32:0/571927623 >> 172.21.15.32:0/571927623 conn(0x7fb3380ff130 msgr2=0x7fb3381085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 -- 172.21.15.32:0/571927623 shutdown_connections 2024-08-31T07:37:20.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.353+0000 7fb33da00640 1 -- 172.21.15.32:0/571927623 wait complete. 2024-08-31T07:37:20.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:20 smithi032 ceph-mon[17252]: pgmap v1143: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 108 op/s 2024-08-31T07:37:20.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:20 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1258093464' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T07:37:20.359 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:20 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3648463391' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T07:37:20.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-31T07:37:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:20 smithi073 ceph-mon[21995]: pgmap v1143: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 108 op/s 2024-08-31T07:37:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:20 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1258093464' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T07:37:20.429 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:20 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3648463391' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T07:37:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:20 smithi084 ceph-mon[22387]: pgmap v1143: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 108 op/s 2024-08-31T07:37:20.490 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:20 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1258093464' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T07:37:20.491 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:20 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3648463391' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T07:37:20.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.602+0000 7f8260c00640 1 -- 172.21.15.32:0/3493151475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c075550 msgr2=0x7f825c075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.602+0000 7f8260c00640 1 --2- 172.21.15.32:0/3493151475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c075550 0x7f825c075960 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f824c009900 tx=0x7f824c02a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.602+0000 7f8260c00640 1 -- 172.21.15.32:0/3493151475 shutdown_connections 2024-08-31T07:37:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.602+0000 7f8260c00640 1 --2- 172.21.15.32:0/3493151475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f825c10fbe0 0x7f825c112040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.602+0000 7f8260c00640 1 --2- 172.21.15.32:0/3493151475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f825c075ea0 0x7f825c10f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.602+0000 7f8260c00640 1 --2- 172.21.15.32:0/3493151475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c075550 0x7f825c075960 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.602+0000 7f8260c00640 1 -- 172.21.15.32:0/3493151475 >> 172.21.15.32:0/3493151475 conn(0x7f825c0ff0d0 msgr2=0x7f825c101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:20.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.602+0000 7f8260c00640 1 -- 172.21.15.32:0/3493151475 shutdown_connections 2024-08-31T07:37:20.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.602+0000 7f8260c00640 1 -- 172.21.15.32:0/3493151475 wait complete. 2024-08-31T07:37:20.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.603+0000 7f8260c00640 1 Processor -- start 2024-08-31T07:37:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f8260c00640 1 -- start start 2024-08-31T07:37:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f8260c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f825c075550 0x7f825c19c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f8260c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c075ea0 0x7f825c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f8260c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f825c10fbe0 0x7f825c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f8260c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f825c1157d0 con 0x7f825c075ea0 2024-08-31T07:37:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f8260c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f825c115650 con 0x7f825c10fbe0 2024-08-31T07:37:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f8260c00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f825c115950 con 0x7f825c075550 2024-08-31T07:37:20.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f825aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c075ea0 0x7f825c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f825be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f825c10fbe0 0x7f825c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f825b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f825c075550 0x7f825c19c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f825be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f825c10fbe0 0x7f825c1a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:35966/0 (socket says 172.21.15.32:35966) 2024-08-31T07:37:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f825be00640 1 -- 172.21.15.32:0/3026557643 learned_addr learned my addr 172.21.15.32:0/3026557643 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.604+0000 7f825b400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f825c075550 0x7f825c19c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:51704/0 (socket says 172.21.15.32:51704) 2024-08-31T07:37:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f825be00640 1 -- 172.21.15.32:0/3026557643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f825c075550 msgr2=0x7f825c19c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f825be00640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f825c075550 0x7f825c19c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f825be00640 1 -- 172.21.15.32:0/3026557643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c075ea0 msgr2=0x7f825c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f825be00640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c075ea0 0x7f825c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f825be00640 1 -- 172.21.15.32:0/3026557643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c1a2100 con 0x7f825c10fbe0 2024-08-31T07:37:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f825be00640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f825c10fbe0 0x7f825c1a1a00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f825400eb10 tx=0x7f825400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f8253e00640 1 -- 172.21.15.32:0/3026557643 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254004b20 con 0x7f825c10fbe0 2024-08-31T07:37:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f8260c00640 1 -- 172.21.15.32:0/3026557643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c1a23f0 con 0x7f825c10fbe0 2024-08-31T07:37:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f8253e00640 1 -- 172.21.15.32:0/3026557643 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8254004cc0 con 0x7f825c10fbe0 2024-08-31T07:37:20.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f8253e00640 1 -- 172.21.15.32:0/3026557643 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254010670 con 0x7f825c10fbe0 2024-08-31T07:37:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.605+0000 7f8260c00640 1 -- 172.21.15.32:0/3026557643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c1a8b80 con 0x7f825c10fbe0 2024-08-31T07:37:20.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.606+0000 7f8260c00640 1 -- 172.21.15.32:0/3026557643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825c075960 con 0x7f825c10fbe0 2024-08-31T07:37:20.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.609+0000 7f8253e00640 1 -- 172.21.15.32:0/3026557643 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8254005000 con 0x7f825c10fbe0 2024-08-31T07:37:20.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.610+0000 7f8253e00640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f82400b4ca0 0x7f82400b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:20.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.611+0000 7f8253e00640 1 -- 172.21.15.32:0/3026557643 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7f82540e5cb0 con 0x7f825c10fbe0 2024-08-31T07:37:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.611+0000 7f825b400640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f82400b4ca0 0x7f82400b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.611+0000 7f825b400640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f82400b4ca0 0x7f82400b71b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f824c031440 tx=0x7f824c009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:20.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.612+0000 7f8253e00640 1 -- 172.21.15.32:0/3026557643 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8254056130 con 0x7f825c10fbe0 2024-08-31T07:37:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.864+0000 7f8260c00640 1 -- 172.21.15.32:0/3026557643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f825c19d2f0 con 0x7f825c10fbe0 2024-08-31T07:37:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.873+0000 7f8253e00640 1 -- 172.21.15.32:0/3026557643 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+826323 (secure 0 0 0) 0x7f825c19d2f0 con 0x7f825c10fbe0 2024-08-31T07:37:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:37:20.887 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:37:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 24, 2024-08-31T07:37:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "active_gid": 14220, 2024-08-31T07:37:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "active_name": "x", 2024-08-31T07:37:20.887 INFO:teuthology.orchestra.run.smithi032.stdout: "active_addrs": { 2024-08-31T07:37:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-08-31T07:37:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-08-31T07:37:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-08-31T07:37:20.888 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4041113440 2024-08-31T07:37:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-08-31T07:37:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-08-31T07:37:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4041113440 2024-08-31T07:37:20.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: "active_addr": "172.21.15.32:6801/4041113440", 2024-08-31T07:37:20.890 INFO:teuthology.orchestra.run.smithi032.stdout: "active_change": "2024-08-31T07:02:44.661936+0000", 2024-08-31T07:37:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "active_mgr_features": 4540701547738038271, 2024-08-31T07:37:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "available": true, 2024-08-31T07:37:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "standbys": [ 2024-08-31T07:37:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "gid": 24172, 2024-08-31T07:37:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "z", 2024-08-31T07:37:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_features": 4540701547738038271, 2024-08-31T07:37:20.891 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2024-08-31T07:37:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2024-08-31T07:37:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:20.892 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.893 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2024-08-31T07:37:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:20.894 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.895 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:20.896 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.897 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.898 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.900 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.901 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:20.902 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:20.903 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2024-08-31T07:37:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2024-08-31T07:37:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2024-08-31T07:37:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2024-08-31T07:37:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2024-08-31T07:37:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2024-08-31T07:37:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2024-08-31T07:37:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2024-08-31T07:37:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2024-08-31T07:37:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2024-08-31T07:37:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2024-08-31T07:37:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2024-08-31T07:37:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2024-08-31T07:37:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2024-08-31T07:37:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2024-08-31T07:37:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2024-08-31T07:37:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2024-08-31T07:37:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2024-08-31T07:37:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2024-08-31T07:37:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2024-08-31T07:37:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2024-08-31T07:37:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2024-08-31T07:37:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-31T07:37:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2024-08-31T07:37:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2024-08-31T07:37:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2024-08-31T07:37:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2024-08-31T07:37:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2024-08-31T07:37:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2024-08-31T07:37:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2024-08-31T07:37:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2024-08-31T07:37:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2024-08-31T07:37:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2024-08-31T07:37:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2024-08-31T07:37:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T07:37:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2024-08-31T07:37:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2024-08-31T07:37:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-08-31T07:37:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-08-31T07:37:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-31T07:37:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T07:37:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2024-08-31T07:37:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2024-08-31T07:37:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2024-08-31T07:37:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2024-08-31T07:37:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-31T07:37:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2024-08-31T07:37:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2024-08-31T07:37:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2024-08-31T07:37:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-31T07:37:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-31T07:37:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2024-08-31T07:37:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2024-08-31T07:37:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2024-08-31T07:37:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2024-08-31T07:37:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2024-08-31T07:37:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2024-08-31T07:37:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-31T07:37:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2024-08-31T07:37:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2024-08-31T07:37:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2024-08-31T07:37:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2024-08-31T07:37:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T07:37:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2024-08-31T07:37:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2024-08-31T07:37:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-08-31T07:37:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-08-31T07:37:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-31T07:37:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T07:37:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2024-08-31T07:37:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2024-08-31T07:37:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-31T07:37:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-08-31T07:37:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-08-31T07:37:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2024-08-31T07:37:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2024-08-31T07:37:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2024-08-31T07:37:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "read", 2024-08-31T07:37:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap", 2024-08-31T07:37:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap-read" 2024-08-31T07:37:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2024-08-31T07:37:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2024-08-31T07:37:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2024-08-31T07:37:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-31T07:37:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-31T07:37:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2024-08-31T07:37:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2024-08-31T07:37:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-31T07:37:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-31T07:37:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2024-08-31T07:37:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2024-08-31T07:37:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-31T07:37:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2024-08-31T07:37:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2024-08-31T07:37:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2024-08-31T07:37:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-08-31T07:37:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-31T07:37:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2024-08-31T07:37:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2024-08-31T07:37:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2024-08-31T07:37:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-31T07:37:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2024-08-31T07:37:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2024-08-31T07:37:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2024-08-31T07:37:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-31T07:37:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2024-08-31T07:37:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2024-08-31T07:37:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-31T07:37:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-31T07:37:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2024-08-31T07:37:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2024-08-31T07:37:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-31T07:37:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2024-08-31T07:37:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2024-08-31T07:37:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2024-08-31T07:37:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-31T07:37:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2024-08-31T07:37:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2024-08-31T07:37:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:20.982 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2024-08-31T07:37:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2024-08-31T07:37:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2024-08-31T07:37:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-31T07:37:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.983 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2024-08-31T07:37:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2024-08-31T07:37:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.984 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-31T07:37:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.985 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2024-08-31T07:37:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2024-08-31T07:37:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.986 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-31T07:37:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2024-08-31T07:37:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2024-08-31T07:37:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.987 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.988 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-31T07:37:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2024-08-31T07:37:20.989 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2024-08-31T07:37:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-31T07:37:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.990 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.991 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2024-08-31T07:37:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2024-08-31T07:37:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-31T07:37:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.992 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2024-08-31T07:37:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2024-08-31T07:37:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2024-08-31T07:37:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.993 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-31T07:37:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2024-08-31T07:37:20.994 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2024-08-31T07:37:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2024-08-31T07:37:20.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-31T07:37:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:20.996 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2024-08-31T07:37:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2024-08-31T07:37:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.997 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-31T07:37:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2024-08-31T07:37:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2024-08-31T07:37:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2024-08-31T07:37:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-31T07:37:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2024-08-31T07:37:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2024-08-31T07:37:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2024-08-31T07:37:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-31T07:37:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2024-08-31T07:37:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2024-08-31T07:37:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2024-08-31T07:37:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-31T07:37:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2024-08-31T07:37:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2024-08-31T07:37:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2024-08-31T07:37:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-31T07:37:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2024-08-31T07:37:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2024-08-31T07:37:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2024-08-31T07:37:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-31T07:37:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2024-08-31T07:37:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nginx": { 2024-08-31T07:37:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nginx", 2024-08-31T07:37:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-31T07:37:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nginx container image", 2024-08-31T07:37:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2024-08-31T07:37:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2024-08-31T07:37:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-31T07:37:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2024-08-31T07:37:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2024-08-31T07:37:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-08-31T07:37:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2024-08-31T07:37:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_oauth2_proxy": { 2024-08-31T07:37:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_oauth2_proxy", 2024-08-31T07:37:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-08-31T07:37:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "oauth2-proxy container image", 2024-08-31T07:37:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2024-08-31T07:37:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2024-08-31T07:37:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-31T07:37:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2024-08-31T07:37:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2024-08-31T07:37:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-31T07:37:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2024-08-31T07:37:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_samba": { 2024-08-31T07:37:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_samba", 2024-08-31T07:37:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-31T07:37:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Samba/SMB container image", 2024-08-31T07:37:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2024-08-31T07:37:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2024-08-31T07:37:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-31T07:37:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2024-08-31T07:37:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2024-08-31T07:37:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2024-08-31T07:37:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2024-08-31T07:37:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2024-08-31T07:37:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2024-08-31T07:37:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-31T07:37:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2024-08-31T07:37:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2024-08-31T07:37:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2024-08-31T07:37:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-31T07:37:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2024-08-31T07:37:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2024-08-31T07:37:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2024-08-31T07:37:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-31T07:37:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2024-08-31T07:37:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2024-08-31T07:37:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2024-08-31T07:37:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2024-08-31T07:37:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2024-08-31T07:37:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2024-08-31T07:37:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-31T07:37:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2024-08-31T07:37:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2024-08-31T07:37:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2024-08-31T07:37:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "grafana_dashboards_path": { 2024-08-31T07:37:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "grafana_dashboards_path", 2024-08-31T07:37:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-31T07:37:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-31T07:37:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2024-08-31T07:37:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2024-08-31T07:37:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2024-08-31T07:37:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "hw_monitoring": { 2024-08-31T07:37:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hw_monitoring", 2024-08-31T07:37:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-31T07:37:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2024-08-31T07:37:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2024-08-31T07:37:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-31T07:37:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2024-08-31T07:37:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2024-08-31T07:37:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-31T07:37:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-31T07:37:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-31T07:37:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-31T07:37:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-31T07:37:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-31T07:37:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-31T07:37:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2024-08-31T07:37:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-31T07:37:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2024-08-31T07:37:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2024-08-31T07:37:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2024-08-31T07:37:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2024-08-31T07:37:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2024-08-31T07:37:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-31T07:37:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2024-08-31T07:37:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2024-08-31T07:37:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2024-08-31T07:37:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-08-31T07:37:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-08-31T07:37:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2024-08-31T07:37:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2024-08-31T07:37:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2024-08-31T07:37:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2024-08-31T07:37:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "oob_default_addr": { 2024-08-31T07:37:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "oob_default_addr", 2024-08-31T07:37:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "169.254.1.1", 2024-08-31T07:37:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-31T07:37:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2024-08-31T07:37:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2024-08-31T07:37:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-31T07:37:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-31T07:37:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2024-08-31T07:37:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2024-08-31T07:37:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-31T07:37:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2024-08-31T07:37:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2024-08-31T07:37:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-31T07:37:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2024-08-31T07:37:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2024-08-31T07:37:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-31T07:37:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2024-08-31T07:37:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2024-08-31T07:37:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-31T07:37:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2024-08-31T07:37:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2024-08-31T07:37:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-31T07:37:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2024-08-31T07:37:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2024-08-31T07:37:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2024-08-31T07:37:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2024-08-31T07:37:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2024-08-31T07:37:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2024-08-31T07:37:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-31T07:37:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2024-08-31T07:37:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2024-08-31T07:37:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-31T07:37:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.096 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2024-08-31T07:37:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2024-08-31T07:37:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-31T07:37:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.098 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2024-08-31T07:37:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2024-08-31T07:37:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2024-08-31T07:37:21.100 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2024-08-31T07:37:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2024-08-31T07:37:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.101 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.102 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-31T07:37:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2024-08-31T07:37:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2024-08-31T07:37:21.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-31T07:37:21.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2024-08-31T07:37:21.105 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.106 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.107 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.108 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.109 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.111 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.113 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.114 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.115 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.116 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2024-08-31T07:37:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2024-08-31T07:37:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.117 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2024-08-31T07:37:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.118 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-31T07:37:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.119 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.120 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.121 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2024-08-31T07:37:21.122 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2024-08-31T07:37:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.123 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2024-08-31T07:37:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.124 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-31T07:37:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2024-08-31T07:37:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.127 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-31T07:37:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-31T07:37:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.128 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.129 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.130 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2024-08-31T07:37:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-31T07:37:21.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.132 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.133 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.134 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-31T07:37:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-31T07:37:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.135 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.136 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2024-08-31T07:37:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2024-08-31T07:37:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.138 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.139 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-31T07:37:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-31T07:37:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.140 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.141 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2024-08-31T07:37:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-31T07:37:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.142 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-31T07:37:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-31T07:37:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.144 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.145 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-31T07:37:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-31T07:37:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.147 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-31T07:37:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-31T07:37:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.148 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.149 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-31T07:37:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-31T07:37:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.150 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-31T07:37:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-31T07:37:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.152 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.153 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-31T07:37:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-31T07:37:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.154 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-31T07:37:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-31T07:37:21.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.156 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-31T07:37:21.157 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-31T07:37:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.158 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2024-08-31T07:37:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-31T07:37:21.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-08-31T07:37:21.160 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.161 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-31T07:37:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-31T07:37:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.162 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2024-08-31T07:37:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2024-08-31T07:37:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.164 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.165 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2024-08-31T07:37:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-31T07:37:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-08-31T07:37:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.166 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-31T07:37:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-31T07:37:21.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.168 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-31T07:37:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-31T07:37:21.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.170 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-31T07:37:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-31T07:37:21.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.172 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.173 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.174 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-31T07:37:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-31T07:37:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.175 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.176 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.177 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-31T07:37:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-31T07:37:21.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "[]", 2024-08-31T07:37:21.179 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.180 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: "MULTICLUSTER_CONFIG": { 2024-08-31T07:37:21.181 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-31T07:37:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.182 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "{}", 2024-08-31T07:37:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.183 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2024-08-31T07:37:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-31T07:37:21.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.186 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.187 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-31T07:37:21.188 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-31T07:37:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.189 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.190 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-31T07:37:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-31T07:37:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.191 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.192 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-31T07:37:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-31T07:37:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-31T07:37:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-31T07:37:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.195 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-31T07:37:21.196 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-31T07:37:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.197 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-31T07:37:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-31T07:37:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.198 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.200 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-31T07:37:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-31T07:37:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.201 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-31T07:37:21.202 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-31T07:37:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.203 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2024-08-31T07:37:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-31T07:37:21.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.205 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-31T07:37:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-31T07:37:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.206 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-31T07:37:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-31T07:37:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-31T07:37:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-31T07:37:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-31T07:37:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2024-08-31T07:37:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-31T07:37:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-31T07:37:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2024-08-31T07:37:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2024-08-31T07:37:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-31T07:37:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-31T07:37:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-31T07:37:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-08-31T07:37:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2024-08-31T07:37:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-31T07:37:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2024-08-31T07:37:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-31T07:37:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "UNSAFE_TLS_v1_2": { 2024-08-31T07:37:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-31T07:37:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-31T07:37:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-31T07:37:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-31T07:37:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-31T07:37:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-31T07:37:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-31T07:37:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2024-08-31T07:37:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2024-08-31T07:37:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2024-08-31T07:37:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2024-08-31T07:37:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2024-08-31T07:37:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2024-08-31T07:37:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2024-08-31T07:37:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2024-08-31T07:37:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2024-08-31T07:37:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2024-08-31T07:37:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-08-31T07:37:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-08-31T07:37:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2024-08-31T07:37:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2024-08-31T07:37:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2024-08-31T07:37:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2024-08-31T07:37:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2024-08-31T07:37:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-08-31T07:37:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-08-31T07:37:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-08-31T07:37:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-08-31T07:37:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-08-31T07:37:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2024-08-31T07:37:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-08-31T07:37:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-08-31T07:37:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2024-08-31T07:37:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2024-08-31T07:37:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2024-08-31T07:37:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-08-31T07:37:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-08-31T07:37:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2024-08-31T07:37:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2024-08-31T07:37:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-08-31T07:37:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-08-31T07:37:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-08-31T07:37:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-08-31T07:37:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-08-31T07:37:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2024-08-31T07:37:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2024-08-31T07:37:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2024-08-31T07:37:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2024-08-31T07:37:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2024-08-31T07:37:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2024-08-31T07:37:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2024-08-31T07:37:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2024-08-31T07:37:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2024-08-31T07:37:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-31T07:37:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2024-08-31T07:37:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2024-08-31T07:37:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2024-08-31T07:37:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-31T07:37:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2024-08-31T07:37:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2024-08-31T07:37:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2024-08-31T07:37:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2024-08-31T07:37:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2024-08-31T07:37:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2024-08-31T07:37:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-08-31T07:37:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-31T07:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2024-08-31T07:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2024-08-31T07:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-31T07:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2024-08-31T07:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2024-08-31T07:37:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2024-08-31T07:37:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2024-08-31T07:37:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-31T07:37:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2024-08-31T07:37:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2024-08-31T07:37:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2024-08-31T07:37:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-08-31T07:37:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2024-08-31T07:37:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2024-08-31T07:37:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2024-08-31T07:37:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2024-08-31T07:37:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2024-08-31T07:37:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2024-08-31T07:37:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2024-08-31T07:37:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2024-08-31T07:37:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2024-08-31T07:37:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-31T07:37:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2024-08-31T07:37:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2024-08-31T07:37:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2024-08-31T07:37:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-31T07:37:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2024-08-31T07:37:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2024-08-31T07:37:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2024-08-31T07:37:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2024-08-31T07:37:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2024-08-31T07:37:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-31T07:37:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2024-08-31T07:37:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2024-08-31T07:37:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.330 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2024-08-31T07:37:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2024-08-31T07:37:21.331 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2024-08-31T07:37:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2024-08-31T07:37:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.332 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2024-08-31T07:37:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.333 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.334 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-08-31T07:37:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-08-31T07:37:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2024-08-31T07:37:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T07:37:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.336 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2024-08-31T07:37:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2024-08-31T07:37:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:21.337 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2024-08-31T07:37:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-31T07:37:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.338 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2024-08-31T07:37:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2024-08-31T07:37:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.339 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2024-08-31T07:37:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2024-08-31T07:37:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2024-08-31T07:37:21.340 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2024-08-31T07:37:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.341 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T07:37:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2024-08-31T07:37:21.342 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.343 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.344 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.345 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.346 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.348 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.349 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.350 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.352 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2024-08-31T07:37:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.354 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.355 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.356 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.357 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.358 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.359 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.359 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.359 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.359 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.359 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.360 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.360 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.360 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.360 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.360 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.360 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.360 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.360 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.361 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.362 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.364 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2024-08-31T07:37:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.365 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2024-08-31T07:37:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2024-08-31T07:37:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2024-08-31T07:37:21.366 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-31T07:37:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.367 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2024-08-31T07:37:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2024-08-31T07:37:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2024-08-31T07:37:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.368 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-31T07:37:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.370 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.374 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.375 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.376 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.378 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.379 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2024-08-31T07:37:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.380 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2024-08-31T07:37:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2024-08-31T07:37:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2024-08-31T07:37:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.381 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2024-08-31T07:37:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-31T07:37:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.382 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.383 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.384 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.386 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.387 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.388 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.389 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2024-08-31T07:37:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2024-08-31T07:37:21.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2024-08-31T07:37:21.392 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-31T07:37:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2024-08-31T07:37:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2024-08-31T07:37:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.393 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2024-08-31T07:37:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2024-08-31T07:37:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2024-08-31T07:37:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2024-08-31T07:37:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2024-08-31T07:37:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2024-08-31T07:37:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2024-08-31T07:37:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2024-08-31T07:37:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2024-08-31T07:37:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2024-08-31T07:37:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2024-08-31T07:37:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2024-08-31T07:37:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-31T07:37:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-31T07:37:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2024-08-31T07:37:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.402 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.403 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.404 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.405 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.406 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.408 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.410 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.411 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.412 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2024-08-31T07:37:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.414 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.415 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2024-08-31T07:37:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-08-31T07:37:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2024-08-31T07:37:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2024-08-31T07:37:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-31T07:37:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2024-08-31T07:37:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-08-31T07:37:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2024-08-31T07:37:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2024-08-31T07:37:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2024-08-31T07:37:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2024-08-31T07:37:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2024-08-31T07:37:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2024-08-31T07:37:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2024-08-31T07:37:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2024-08-31T07:37:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2024-08-31T07:37:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-08-31T07:37:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2024-08-31T07:37:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2024-08-31T07:37:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-31T07:37:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2024-08-31T07:37:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2024-08-31T07:37:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2024-08-31T07:37:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-31T07:37:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-08-31T07:37:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-08-31T07:37:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2024-08-31T07:37:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2024-08-31T07:37:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2024-08-31T07:37:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2024-08-31T07:37:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2024-08-31T07:37:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2024-08-31T07:37:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2024-08-31T07:37:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2024-08-31T07:37:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2024-08-31T07:37:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2024-08-31T07:37:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.513 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-31T07:37:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-31T07:37:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.514 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.515 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.516 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.517 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.518 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.519 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.520 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.522 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.523 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2024-08-31T07:37:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2024-08-31T07:37:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.524 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-31T07:37:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-31T07:37:21.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2024-08-31T07:37:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.526 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2024-08-31T07:37:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2024-08-31T07:37:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.527 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2024-08-31T07:37:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.528 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-08-31T07:37:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-08-31T07:37:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-08-31T07:37:21.529 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-31T07:37:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-08-31T07:37:21.530 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-08-31T07:37:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2024-08-31T07:37:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.531 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-31T07:37:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.532 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2024-08-31T07:37:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2024-08-31T07:37:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.534 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2024-08-31T07:37:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.535 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-08-31T07:37:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-08-31T07:37:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2024-08-31T07:37:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.536 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2024-08-31T07:37:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2024-08-31T07:37:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.537 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-08-31T07:37:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-08-31T07:37:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-08-31T07:37:21.538 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-08-31T07:37:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-08-31T07:37:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2024-08-31T07:37:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.541 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.543 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.545 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.546 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.548 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2024-08-31T07:37:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2024-08-31T07:37:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.550 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2024-08-31T07:37:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2024-08-31T07:37:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.551 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.552 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.553 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2024-08-31T07:37:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2024-08-31T07:37:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.554 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2024-08-31T07:37:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.556 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2024-08-31T07:37:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2024-08-31T07:37:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.557 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-08-31T07:37:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-08-31T07:37:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.559 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.560 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.561 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.563 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.565 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.566 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.567 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.568 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "max_requests": { 2024-08-31T07:37:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_requests", 2024-08-31T07:37:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.569 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-08-31T07:37:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-08-31T07:37:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-08-31T07:37:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-08-31T07:37:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.571 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-08-31T07:37:21.572 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-08-31T07:37:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.573 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.574 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.575 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2024-08-31T07:37:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.576 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.577 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.578 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.579 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.580 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.582 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.584 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "secondary_zone_period_retry_limit": { 2024-08-31T07:37:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-31T07:37:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.586 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-31T07:37:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.587 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2024-08-31T07:37:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.590 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.591 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.593 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.594 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.595 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.596 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.598 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.599 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.600 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2024-08-31T07:37:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2024-08-31T07:37:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2024-08-31T07:37:21.601 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-31T07:37:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2024-08-31T07:37:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.603 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.604 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.605 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.606 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.607 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.608 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.609 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.610 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.611 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2024-08-31T07:37:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2024-08-31T07:37:21.612 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.613 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2024-08-31T07:37:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2024-08-31T07:37:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.614 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2024-08-31T07:37:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.615 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2024-08-31T07:37:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2024-08-31T07:37:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.616 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2024-08-31T07:37:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2024-08-31T07:37:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.618 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2024-08-31T07:37:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2024-08-31T07:37:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:21.619 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.620 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2024-08-31T07:37:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2024-08-31T07:37:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.621 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2024-08-31T07:37:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2024-08-31T07:37:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.623 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2024-08-31T07:37:21.624 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2024-08-31T07:37:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.625 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2024-08-31T07:37:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2024-08-31T07:37:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.626 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2024-08-31T07:37:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.628 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2024-08-31T07:37:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2024-08-31T07:37:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.630 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2024-08-31T07:37:21.631 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2024-08-31T07:37:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.632 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2024-08-31T07:37:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2024-08-31T07:37:21.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.634 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.635 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smb", 2024-08-31T07:37:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "internal_store_backend": { 2024-08-31T07:37:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "internal_store_backend", 2024-08-31T07:37:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-31T07:37:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.637 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.638 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.639 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.640 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.641 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.642 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.643 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.645 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.647 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "update_orchestration": { 2024-08-31T07:37:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "update_orchestration", 2024-08-31T07:37:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.648 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-31T07:37:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2024-08-31T07:37:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.650 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2024-08-31T07:37:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2024-08-31T07:37:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.651 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2024-08-31T07:37:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2024-08-31T07:37:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2024-08-31T07:37:21.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2024-08-31T07:37:21.653 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.654 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.655 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.657 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.658 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.659 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.660 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.661 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.662 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.663 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.664 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2024-08-31T07:37:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.665 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.666 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.667 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.669 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.670 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.671 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.673 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.675 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2024-08-31T07:37:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.676 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.677 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.678 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.680 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.681 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.682 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.683 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.684 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.685 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.689 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2024-08-31T07:37:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.690 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2024-08-31T07:37:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2024-08-31T07:37:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-31T07:37:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.691 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:21.692 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2024-08-31T07:37:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.693 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.694 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.695 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.696 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.697 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.698 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.699 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.700 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.701 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.702 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.703 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.704 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2024-08-31T07:37:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2024-08-31T07:37:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2024-08-31T07:37:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-31T07:37:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2024-08-31T07:37:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2024-08-31T07:37:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-31T07:37:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2024-08-31T07:37:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2024-08-31T07:37:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-31T07:37:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2024-08-31T07:37:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2024-08-31T07:37:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-31T07:37:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.712 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2024-08-31T07:37:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2024-08-31T07:37:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.713 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-31T07:37:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2024-08-31T07:37:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2024-08-31T07:37:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.715 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.716 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2024-08-31T07:37:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2024-08-31T07:37:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.717 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2024-08-31T07:37:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2024-08-31T07:37:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-31T07:37:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.719 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-08-31T07:37:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-08-31T07:37:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.720 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2024-08-31T07:37:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2024-08-31T07:37:21.722 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2024-08-31T07:37:21.723 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2024-08-31T07:37:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2024-08-31T07:37:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.724 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2024-08-31T07:37:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2024-08-31T07:37:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.725 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2024-08-31T07:37:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2024-08-31T07:37:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.727 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.728 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.729 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.732 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.733 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.734 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.735 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.736 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2024-08-31T07:37:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2024-08-31T07:37:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.738 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2024-08-31T07:37:21.739 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2024-08-31T07:37:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.740 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.742 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2024-08-31T07:37:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2024-08-31T07:37:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.743 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-31T07:37:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.744 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2024-08-31T07:37:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.745 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.746 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.747 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.749 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.750 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.751 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.752 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.753 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.754 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.755 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2024-08-31T07:37:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.758 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.759 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.760 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.763 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.765 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2024-08-31T07:37:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2024-08-31T07:37:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.766 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2024-08-31T07:37:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-31T07:37:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2024-08-31T07:37:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2024-08-31T07:37:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.768 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2024-08-31T07:37:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2024-08-31T07:37:21.769 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2024-08-31T07:37:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.770 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-31T07:37:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_no_wait": { 2024-08-31T07:37:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_no_wait", 2024-08-31T07:37:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.771 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-31T07:37:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.772 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.773 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.774 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2024-08-31T07:37:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2024-08-31T07:37:21.775 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2024-08-31T07:37:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2024-08-31T07:37:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.776 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2024-08-31T07:37:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2024-08-31T07:37:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.777 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.778 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.779 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.781 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.782 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.783 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.784 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.786 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.788 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.789 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.790 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2024-08-31T07:37:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2024-08-31T07:37:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.791 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2024-08-31T07:37:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2024-08-31T07:37:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.793 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2024-08-31T07:37:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2024-08-31T07:37:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2024-08-31T07:37:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-31T07:37:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.795 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "gid": 24179, 2024-08-31T07:37:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "y", 2024-08-31T07:37:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_features": 4540701547738038271, 2024-08-31T07:37:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2024-08-31T07:37:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.797 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2024-08-31T07:37:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.798 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2024-08-31T07:37:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.800 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.801 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.805 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.806 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.807 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.808 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2024-08-31T07:37:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2024-08-31T07:37:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.809 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2024-08-31T07:37:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2024-08-31T07:37:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2024-08-31T07:37:21.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2024-08-31T07:37:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2024-08-31T07:37:21.811 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2024-08-31T07:37:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2024-08-31T07:37:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.812 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2024-08-31T07:37:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.813 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2024-08-31T07:37:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2024-08-31T07:37:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2024-08-31T07:37:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2024-08-31T07:37:21.815 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2024-08-31T07:37:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2024-08-31T07:37:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.816 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2024-08-31T07:37:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2024-08-31T07:37:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2024-08-31T07:37:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.817 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2024-08-31T07:37:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2024-08-31T07:37:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2024-08-31T07:37:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.819 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-31T07:37:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2024-08-31T07:37:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2024-08-31T07:37:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.821 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2024-08-31T07:37:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2024-08-31T07:37:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.824 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2024-08-31T07:37:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2024-08-31T07:37:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.825 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2024-08-31T07:37:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.826 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2024-08-31T07:37:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2024-08-31T07:37:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2024-08-31T07:37:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2024-08-31T07:37:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T07:37:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2024-08-31T07:37:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2024-08-31T07:37:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-08-31T07:37:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-08-31T07:37:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-31T07:37:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T07:37:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2024-08-31T07:37:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2024-08-31T07:37:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2024-08-31T07:37:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2024-08-31T07:37:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-31T07:37:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2024-08-31T07:37:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2024-08-31T07:37:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2024-08-31T07:37:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-31T07:37:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-31T07:37:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2024-08-31T07:37:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2024-08-31T07:37:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2024-08-31T07:37:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2024-08-31T07:37:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2024-08-31T07:37:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2024-08-31T07:37:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-31T07:37:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2024-08-31T07:37:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2024-08-31T07:37:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2024-08-31T07:37:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2024-08-31T07:37:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T07:37:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2024-08-31T07:37:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2024-08-31T07:37:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-08-31T07:37:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-08-31T07:37:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-31T07:37:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T07:37:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2024-08-31T07:37:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2024-08-31T07:37:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-31T07:37:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-08-31T07:37:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-08-31T07:37:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2024-08-31T07:37:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2024-08-31T07:37:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2024-08-31T07:37:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "read", 2024-08-31T07:37:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap", 2024-08-31T07:37:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap-read" 2024-08-31T07:37:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2024-08-31T07:37:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2024-08-31T07:37:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2024-08-31T07:37:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-31T07:37:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-31T07:37:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2024-08-31T07:37:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2024-08-31T07:37:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-31T07:37:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-31T07:37:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2024-08-31T07:37:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2024-08-31T07:37:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-31T07:37:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2024-08-31T07:37:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2024-08-31T07:37:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2024-08-31T07:37:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-08-31T07:37:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-31T07:37:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2024-08-31T07:37:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2024-08-31T07:37:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2024-08-31T07:37:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-31T07:37:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2024-08-31T07:37:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2024-08-31T07:37:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2024-08-31T07:37:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-31T07:37:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2024-08-31T07:37:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2024-08-31T07:37:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-31T07:37:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-31T07:37:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2024-08-31T07:37:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2024-08-31T07:37:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-31T07:37:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2024-08-31T07:37:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2024-08-31T07:37:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2024-08-31T07:37:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-31T07:37:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2024-08-31T07:37:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2024-08-31T07:37:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2024-08-31T07:37:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2024-08-31T07:37:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2024-08-31T07:37:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-31T07:37:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2024-08-31T07:37:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2024-08-31T07:37:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-31T07:37:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2024-08-31T07:37:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2024-08-31T07:37:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-31T07:37:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2024-08-31T07:37:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2024-08-31T07:37:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-31T07:37:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2024-08-31T07:37:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2024-08-31T07:37:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-31T07:37:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2024-08-31T07:37:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2024-08-31T07:37:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-31T07:37:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2024-08-31T07:37:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2024-08-31T07:37:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2024-08-31T07:37:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-31T07:37:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2024-08-31T07:37:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2024-08-31T07:37:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2024-08-31T07:37:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-31T07:37:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2024-08-31T07:37:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2024-08-31T07:37:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-31T07:37:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2024-08-31T07:37:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2024-08-31T07:37:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2024-08-31T07:37:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-31T07:37:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2024-08-31T07:37:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2024-08-31T07:37:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2024-08-31T07:37:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-31T07:37:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2024-08-31T07:37:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2024-08-31T07:37:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2024-08-31T07:37:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-31T07:37:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2024-08-31T07:37:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2024-08-31T07:37:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2024-08-31T07:37:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-31T07:37:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2024-08-31T07:37:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2024-08-31T07:37:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2024-08-31T07:37:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-31T07:37:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2024-08-31T07:37:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nginx": { 2024-08-31T07:37:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nginx", 2024-08-31T07:37:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-31T07:37:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nginx container image", 2024-08-31T07:37:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2024-08-31T07:37:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2024-08-31T07:37:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-31T07:37:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2024-08-31T07:37:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2024-08-31T07:37:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-08-31T07:37:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2024-08-31T07:37:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_oauth2_proxy": { 2024-08-31T07:37:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_oauth2_proxy", 2024-08-31T07:37:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-08-31T07:37:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "oauth2-proxy container image", 2024-08-31T07:37:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2024-08-31T07:37:21.903 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2024-08-31T07:37:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-31T07:37:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.904 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2024-08-31T07:37:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2024-08-31T07:37:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.905 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-31T07:37:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2024-08-31T07:37:21.906 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_samba": { 2024-08-31T07:37:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_samba", 2024-08-31T07:37:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-31T07:37:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Samba/SMB container image", 2024-08-31T07:37:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.908 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2024-08-31T07:37:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2024-08-31T07:37:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-31T07:37:21.909 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2024-08-31T07:37:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.910 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2024-08-31T07:37:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2024-08-31T07:37:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.911 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2024-08-31T07:37:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2024-08-31T07:37:21.912 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2024-08-31T07:37:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.913 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-31T07:37:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2024-08-31T07:37:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2024-08-31T07:37:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.914 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2024-08-31T07:37:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-31T07:37:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2024-08-31T07:37:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2024-08-31T07:37:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.916 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2024-08-31T07:37:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-31T07:37:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2024-08-31T07:37:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2024-08-31T07:37:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2024-08-31T07:37:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.918 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2024-08-31T07:37:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.919 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2024-08-31T07:37:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2024-08-31T07:37:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.920 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-31T07:37:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2024-08-31T07:37:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2024-08-31T07:37:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.921 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2024-08-31T07:37:21.922 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "grafana_dashboards_path": { 2024-08-31T07:37:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "grafana_dashboards_path", 2024-08-31T07:37:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.923 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-31T07:37:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-31T07:37:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.924 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2024-08-31T07:37:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2024-08-31T07:37:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:21.925 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2024-08-31T07:37:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "hw_monitoring": { 2024-08-31T07:37:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hw_monitoring", 2024-08-31T07:37:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.927 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-31T07:37:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2024-08-31T07:37:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2024-08-31T07:37:21.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-31T07:37:21.929 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.930 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.931 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.932 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2024-08-31T07:37:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2024-08-31T07:37:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.933 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-31T07:37:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-31T07:37:21.935 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.937 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.939 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.940 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-31T07:37:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-31T07:37:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.941 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-31T07:37:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-31T07:37:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-31T07:37:21.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2024-08-31T07:37:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-31T07:37:21.943 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2024-08-31T07:37:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2024-08-31T07:37:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.944 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2024-08-31T07:37:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.945 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2024-08-31T07:37:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2024-08-31T07:37:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.946 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-31T07:37:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.947 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2024-08-31T07:37:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2024-08-31T07:37:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.948 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2024-08-31T07:37:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-08-31T07:37:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-08-31T07:37:21.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2024-08-31T07:37:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2024-08-31T07:37:21.950 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2024-08-31T07:37:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2024-08-31T07:37:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "oob_default_addr": { 2024-08-31T07:37:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "oob_default_addr", 2024-08-31T07:37:21.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "169.254.1.1", 2024-08-31T07:37:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-31T07:37:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2024-08-31T07:37:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2024-08-31T07:37:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-31T07:37:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-31T07:37:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.954 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2024-08-31T07:37:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2024-08-31T07:37:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.955 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-31T07:37:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2024-08-31T07:37:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2024-08-31T07:37:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.957 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-31T07:37:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.958 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2024-08-31T07:37:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2024-08-31T07:37:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.959 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-31T07:37:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2024-08-31T07:37:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2024-08-31T07:37:21.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-31T07:37:21.961 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2024-08-31T07:37:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2024-08-31T07:37:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.962 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-31T07:37:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.963 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2024-08-31T07:37:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2024-08-31T07:37:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2024-08-31T07:37:21.964 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2024-08-31T07:37:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.966 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2024-08-31T07:37:21.967 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2024-08-31T07:37:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.968 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-31T07:37:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2024-08-31T07:37:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2024-08-31T07:37:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.969 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-31T07:37:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2024-08-31T07:37:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2024-08-31T07:37:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-31T07:37:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2024-08-31T07:37:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2024-08-31T07:37:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2024-08-31T07:37:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2024-08-31T07:37:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2024-08-31T07:37:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-31T07:37:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2024-08-31T07:37:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2024-08-31T07:37:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-31T07:37:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2024-08-31T07:37:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.983 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:21.984 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:21.986 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2024-08-31T07:37:21.987 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2024-08-31T07:37:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2024-08-31T07:37:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.988 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-31T07:37:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.989 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.990 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2024-08-31T07:37:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2024-08-31T07:37:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2024-08-31T07:37:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-31T07:37:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.992 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2024-08-31T07:37:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:21.993 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-31T07:37:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-31T07:37:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.994 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2024-08-31T07:37:21.995 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-31T07:37:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.996 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-31T07:37:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-31T07:37:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.997 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:21.998 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2024-08-31T07:37:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2024-08-31T07:37:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:21.999 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-31T07:37:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-31T07:37:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.001 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2024-08-31T07:37:22.002 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-31T07:37:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.003 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-31T07:37:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-31T07:37:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.004 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-31T07:37:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-31T07:37:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.006 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-31T07:37:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-31T07:37:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.008 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-31T07:37:22.009 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-31T07:37:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.010 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-31T07:37:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-31T07:37:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.011 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-31T07:37:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-31T07:37:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.013 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-31T07:37:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-31T07:37:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.015 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-31T07:37:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-31T07:37:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.017 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2024-08-31T07:37:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-31T07:37:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.018 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-08-31T07:37:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.019 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-31T07:37:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-31T07:37:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.020 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2024-08-31T07:37:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2024-08-31T07:37:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.022 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2024-08-31T07:37:22.023 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-31T07:37:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-08-31T07:37:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.024 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-31T07:37:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-31T07:37:22.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.026 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-31T07:37:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-31T07:37:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.027 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.028 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-31T07:37:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-31T07:37:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.029 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-31T07:37:22.030 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-31T07:37:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.031 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-31T07:37:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-31T07:37:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.032 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "[]", 2024-08-31T07:37:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.033 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "MULTICLUSTER_CONFIG": { 2024-08-31T07:37:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-31T07:37:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "{}", 2024-08-31T07:37:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2024-08-31T07:37:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-31T07:37:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.036 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.037 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-31T07:37:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-31T07:37:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.038 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-31T07:37:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-31T07:37:22.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.040 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-31T07:37:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-31T07:37:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-31T07:37:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-31T07:37:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.043 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-31T07:37:22.044 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-31T07:37:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-31T07:37:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-31T07:37:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-31T07:37:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-31T07:37:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-31T07:37:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-31T07:37:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2024-08-31T07:37:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-31T07:37:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-31T07:37:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-31T07:37:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-31T07:37:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-31T07:37:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-31T07:37:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-31T07:37:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-31T07:37:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2024-08-31T07:37:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-31T07:37:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-31T07:37:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2024-08-31T07:37:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2024-08-31T07:37:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-31T07:37:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-31T07:37:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-31T07:37:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-08-31T07:37:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2024-08-31T07:37:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-31T07:37:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2024-08-31T07:37:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-31T07:37:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "UNSAFE_TLS_v1_2": { 2024-08-31T07:37:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-31T07:37:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-31T07:37:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-31T07:37:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-31T07:37:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-31T07:37:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-31T07:37:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-31T07:37:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2024-08-31T07:37:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2024-08-31T07:37:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2024-08-31T07:37:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2024-08-31T07:37:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2024-08-31T07:37:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2024-08-31T07:37:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2024-08-31T07:37:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2024-08-31T07:37:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2024-08-31T07:37:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2024-08-31T07:37:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-08-31T07:37:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-08-31T07:37:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.086 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.089 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.090 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.091 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2024-08-31T07:37:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2024-08-31T07:37:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.092 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2024-08-31T07:37:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.093 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2024-08-31T07:37:22.095 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2024-08-31T07:37:22.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.096 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-08-31T07:37:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-08-31T07:37:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.097 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-08-31T07:37:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.098 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-08-31T07:37:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-08-31T07:37:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.099 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2024-08-31T07:37:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.101 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-08-31T07:37:22.102 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-08-31T07:37:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.103 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2024-08-31T07:37:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2024-08-31T07:37:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.104 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2024-08-31T07:37:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.105 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-08-31T07:37:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-08-31T07:37:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.106 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2024-08-31T07:37:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2024-08-31T07:37:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.107 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-08-31T07:37:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-08-31T07:37:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.108 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-08-31T07:37:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-08-31T07:37:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-08-31T07:37:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.109 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2024-08-31T07:37:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2024-08-31T07:37:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2024-08-31T07:37:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2024-08-31T07:37:22.112 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2024-08-31T07:37:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.113 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2024-08-31T07:37:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.116 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.117 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.118 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.119 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.120 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.121 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.122 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2024-08-31T07:37:22.123 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2024-08-31T07:37:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2024-08-31T07:37:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.124 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-31T07:37:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2024-08-31T07:37:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2024-08-31T07:37:22.125 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2024-08-31T07:37:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-31T07:37:22.126 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2024-08-31T07:37:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2024-08-31T07:37:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2024-08-31T07:37:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2024-08-31T07:37:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.128 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2024-08-31T07:37:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2024-08-31T07:37:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-08-31T07:37:22.129 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-31T07:37:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.130 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2024-08-31T07:37:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2024-08-31T07:37:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.131 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-31T07:37:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.132 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2024-08-31T07:37:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.134 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2024-08-31T07:37:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2024-08-31T07:37:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2024-08-31T07:37:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.136 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-31T07:37:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2024-08-31T07:37:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.139 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.140 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.142 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.144 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.145 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.146 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2024-08-31T07:37:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2024-08-31T07:37:22.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-08-31T07:37:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.148 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2024-08-31T07:37:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2024-08-31T07:37:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2024-08-31T07:37:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.150 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:22.151 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.152 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.153 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2024-08-31T07:37:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2024-08-31T07:37:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2024-08-31T07:37:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2024-08-31T07:37:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2024-08-31T07:37:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2024-08-31T07:37:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-31T07:37:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2024-08-31T07:37:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2024-08-31T07:37:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.157 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2024-08-31T07:37:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-31T07:37:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2024-08-31T07:37:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2024-08-31T07:37:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.159 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2024-08-31T07:37:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:22.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2024-08-31T07:37:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2024-08-31T07:37:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-31T07:37:22.161 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.163 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.165 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.166 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.167 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.168 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.169 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.170 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2024-08-31T07:37:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2024-08-31T07:37:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.171 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2024-08-31T07:37:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.172 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2024-08-31T07:37:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2024-08-31T07:37:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2024-08-31T07:37:22.173 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2024-08-31T07:37:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.174 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.175 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-08-31T07:37:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-08-31T07:37:22.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2024-08-31T07:37:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T07:37:22.177 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2024-08-31T07:37:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2024-08-31T07:37:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.178 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2024-08-31T07:37:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-31T07:37:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.179 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2024-08-31T07:37:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2024-08-31T07:37:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.180 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2024-08-31T07:37:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.181 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2024-08-31T07:37:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2024-08-31T07:37:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2024-08-31T07:37:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.182 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T07:37:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2024-08-31T07:37:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.184 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.186 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.187 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.190 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.191 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.193 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2024-08-31T07:37:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.195 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.196 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.197 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.198 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.199 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.200 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.201 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.202 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.203 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.204 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.205 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2024-08-31T07:37:22.206 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2024-08-31T07:37:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2024-08-31T07:37:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2024-08-31T07:37:22.207 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-31T07:37:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2024-08-31T07:37:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2024-08-31T07:37:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2024-08-31T07:37:22.209 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-31T07:37:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.210 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.211 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.212 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.213 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.214 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.215 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.217 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.218 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.219 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.220 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2024-08-31T07:37:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2024-08-31T07:37:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2024-08-31T07:37:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.222 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2024-08-31T07:37:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2024-08-31T07:37:22.223 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-31T07:37:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.224 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.225 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.227 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.228 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.229 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.230 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.231 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2024-08-31T07:37:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2024-08-31T07:37:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.233 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2024-08-31T07:37:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-31T07:37:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.234 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2024-08-31T07:37:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2024-08-31T07:37:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2024-08-31T07:37:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.235 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2024-08-31T07:37:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2024-08-31T07:37:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2024-08-31T07:37:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.236 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2024-08-31T07:37:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2024-08-31T07:37:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.237 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2024-08-31T07:37:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2024-08-31T07:37:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.238 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2024-08-31T07:37:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.240 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2024-08-31T07:37:22.241 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2024-08-31T07:37:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2024-08-31T07:37:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.242 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-31T07:37:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-31T07:37:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2024-08-31T07:37:22.243 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.244 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.245 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.247 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.249 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.250 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.251 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.252 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.253 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2024-08-31T07:37:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.256 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.257 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.258 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.259 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.260 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.261 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.262 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2024-08-31T07:37:22.266 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.268 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.269 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.270 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.272 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.273 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.274 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.275 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.276 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-08-31T07:37:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2024-08-31T07:37:22.278 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2024-08-31T07:37:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.279 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-31T07:37:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.280 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.281 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.282 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.283 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.285 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.287 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.288 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2024-08-31T07:37:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-08-31T07:37:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2024-08-31T07:37:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2024-08-31T07:37:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2024-08-31T07:37:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.290 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2024-08-31T07:37:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.292 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2024-08-31T07:37:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.293 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.294 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.295 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.296 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.297 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.298 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.299 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.300 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.301 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.304 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.305 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.306 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2024-08-31T07:37:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.307 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.308 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.309 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.310 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.311 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.312 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.314 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.317 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2024-08-31T07:37:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.319 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.323 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.324 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.325 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.327 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.328 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.329 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2024-08-31T07:37:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2024-08-31T07:37:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:22.330 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-08-31T07:37:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2024-08-31T07:37:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2024-08-31T07:37:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-31T07:37:22.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2024-08-31T07:37:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.332 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2024-08-31T07:37:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2024-08-31T07:37:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.333 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-31T07:37:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-08-31T07:37:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-08-31T07:37:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.335 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.336 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.337 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.338 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.340 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.341 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.344 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2024-08-31T07:37:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2024-08-31T07:37:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.345 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2024-08-31T07:37:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2024-08-31T07:37:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2024-08-31T07:37:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.348 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.349 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.350 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2024-08-31T07:37:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2024-08-31T07:37:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2024-08-31T07:37:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.352 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2024-08-31T07:37:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2024-08-31T07:37:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-31T07:37:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-31T07:37:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.354 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.355 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.356 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.358 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.359 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.360 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.362 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2024-08-31T07:37:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2024-08-31T07:37:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.363 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-31T07:37:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-31T07:37:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.365 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2024-08-31T07:37:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2024-08-31T07:37:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2024-08-31T07:37:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2024-08-31T07:37:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-08-31T07:37:22.368 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-08-31T07:37:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-08-31T07:37:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.369 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-31T07:37:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-08-31T07:37:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-08-31T07:37:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.370 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2024-08-31T07:37:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-31T07:37:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.372 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.373 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2024-08-31T07:37:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2024-08-31T07:37:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2024-08-31T07:37:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.374 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-08-31T07:37:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-08-31T07:37:22.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2024-08-31T07:37:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2024-08-31T07:37:22.376 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2024-08-31T07:37:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-08-31T07:37:22.377 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-08-31T07:37:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-08-31T07:37:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-08-31T07:37:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-08-31T07:37:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.379 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2024-08-31T07:37:22.380 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.380 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.380 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.380 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.380 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.380 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.380 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.381 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.385 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.386 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.387 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.388 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2024-08-31T07:37:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2024-08-31T07:37:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.389 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2024-08-31T07:37:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2024-08-31T07:37:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.391 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.392 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.393 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2024-08-31T07:37:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2024-08-31T07:37:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.394 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.395 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2024-08-31T07:37:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.396 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2024-08-31T07:37:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2024-08-31T07:37:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.397 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.398 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-08-31T07:37:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-08-31T07:37:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.399 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.400 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.401 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.402 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.404 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.405 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.406 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.407 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.408 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "max_requests": { 2024-08-31T07:37:22.409 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_requests", 2024-08-31T07:37:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-08-31T07:37:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.410 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-08-31T07:37:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-08-31T07:37:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-08-31T07:37:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.411 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.412 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-08-31T07:37:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-08-31T07:37:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.413 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.414 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.415 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.416 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2024-08-31T07:37:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.417 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.418 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.419 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.420 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.421 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.422 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.423 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.424 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.425 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "secondary_zone_period_retry_limit": { 2024-08-31T07:37:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-31T07:37:22.426 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-31T07:37:22.427 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.429 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2024-08-31T07:37:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.430 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.431 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.432 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.433 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.434 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.435 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.436 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.437 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.438 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.440 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.441 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2024-08-31T07:37:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2024-08-31T07:37:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2024-08-31T07:37:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.442 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-31T07:37:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2024-08-31T07:37:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.444 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.445 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.446 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.447 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.448 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.449 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.451 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.452 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2024-08-31T07:37:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2024-08-31T07:37:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.453 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.454 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2024-08-31T07:37:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2024-08-31T07:37:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2024-08-31T07:37:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.455 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2024-08-31T07:37:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2024-08-31T07:37:22.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.457 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2024-08-31T07:37:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2024-08-31T07:37:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.458 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.459 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2024-08-31T07:37:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2024-08-31T07:37:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.460 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2024-08-31T07:37:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2024-08-31T07:37:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.462 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2024-08-31T07:37:22.463 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2024-08-31T07:37:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.464 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2024-08-31T07:37:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2024-08-31T07:37:22.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.466 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2024-08-31T07:37:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2024-08-31T07:37:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.467 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2024-08-31T07:37:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.469 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2024-08-31T07:37:22.470 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2024-08-31T07:37:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.471 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2024-08-31T07:37:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2024-08-31T07:37:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.472 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2024-08-31T07:37:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2024-08-31T07:37:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.474 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smb", 2024-08-31T07:37:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "internal_store_backend": { 2024-08-31T07:37:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "internal_store_backend", 2024-08-31T07:37:22.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-31T07:37:22.477 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.478 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.479 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.480 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.481 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.482 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.483 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.484 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.486 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.487 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "update_orchestration": { 2024-08-31T07:37:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "update_orchestration", 2024-08-31T07:37:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.489 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-31T07:37:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2024-08-31T07:37:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2024-08-31T07:37:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2024-08-31T07:37:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.491 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2024-08-31T07:37:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2024-08-31T07:37:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2024-08-31T07:37:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.493 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2024-08-31T07:37:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.494 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.496 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.497 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.499 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.500 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.503 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2024-08-31T07:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.510 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.511 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.513 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.514 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.515 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2024-08-31T07:37:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.516 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.517 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.518 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.519 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.520 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.521 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.522 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.523 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.524 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.525 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.526 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2024-08-31T07:37:22.527 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2024-08-31T07:37:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2024-08-31T07:37:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-31T07:37:22.528 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2024-08-31T07:37:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.530 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.531 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.532 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.534 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.536 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.537 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.538 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.539 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.540 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2024-08-31T07:37:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.542 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2024-08-31T07:37:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2024-08-31T07:37:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.543 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-31T07:37:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.544 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2024-08-31T07:37:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2024-08-31T07:37:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.545 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-31T07:37:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2024-08-31T07:37:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2024-08-31T07:37:22.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-31T07:37:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2024-08-31T07:37:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2024-08-31T07:37:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.548 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-31T07:37:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2024-08-31T07:37:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2024-08-31T07:37:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.550 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-31T07:37:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2024-08-31T07:37:22.551 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2024-08-31T07:37:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.552 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2024-08-31T07:37:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2024-08-31T07:37:22.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.554 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2024-08-31T07:37:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2024-08-31T07:37:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-31T07:37:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.556 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-08-31T07:37:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-08-31T07:37:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.557 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.558 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2024-08-31T07:37:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2024-08-31T07:37:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.559 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2024-08-31T07:37:22.560 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2024-08-31T07:37:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2024-08-31T07:37:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.561 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2024-08-31T07:37:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2024-08-31T07:37:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.563 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2024-08-31T07:37:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2024-08-31T07:37:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.564 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.565 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.566 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.567 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.568 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.570 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.573 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2024-08-31T07:37:22.574 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2024-08-31T07:37:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.575 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2024-08-31T07:37:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2024-08-31T07:37:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.576 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.577 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.578 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.579 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2024-08-31T07:37:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2024-08-31T07:37:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-31T07:37:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.580 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2024-08-31T07:37:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.583 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.584 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.585 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.587 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.588 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.589 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.590 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.592 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.593 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2024-08-31T07:37:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.594 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.595 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.597 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.599 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.601 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2024-08-31T07:37:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2024-08-31T07:37:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2024-08-31T07:37:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.603 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-31T07:37:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2024-08-31T07:37:22.604 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2024-08-31T07:37:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.605 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2024-08-31T07:37:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2024-08-31T07:37:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2024-08-31T07:37:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.606 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-31T07:37:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_no_wait": { 2024-08-31T07:37:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_no_wait", 2024-08-31T07:37:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.608 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-31T07:37:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.609 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.610 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2024-08-31T07:37:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2024-08-31T07:37:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2024-08-31T07:37:22.612 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2024-08-31T07:37:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.613 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2024-08-31T07:37:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2024-08-31T07:37:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.614 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.616 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.618 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.619 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.620 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.621 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.623 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.625 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.626 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.627 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2024-08-31T07:37:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2024-08-31T07:37:22.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.629 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2024-08-31T07:37:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2024-08-31T07:37:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.630 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2024-08-31T07:37:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.631 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2024-08-31T07:37:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2024-08-31T07:37:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.632 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-31T07:37:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.633 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "modules": [ 2024-08-31T07:37:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2024-08-31T07:37:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "dashboard", 2024-08-31T07:37:22.634 INFO:teuthology.orchestra.run.smithi032.stdout: "iostat", 2024-08-31T07:37:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "nfs", 2024-08-31T07:37:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "restful", 2024-08-31T07:37:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_schedule" 2024-08-31T07:37:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2024-08-31T07:37:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2024-08-31T07:37:22.635 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2024-08-31T07:37:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.637 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.638 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.639 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.641 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.642 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.643 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.645 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2024-08-31T07:37:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2024-08-31T07:37:22.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2024-08-31T07:37:22.647 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2024-08-31T07:37:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2024-08-31T07:37:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.648 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2024-08-31T07:37:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2024-08-31T07:37:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.649 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2024-08-31T07:37:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2024-08-31T07:37:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.650 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2024-08-31T07:37:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2024-08-31T07:37:22.651 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2024-08-31T07:37:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.652 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2024-08-31T07:37:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2024-08-31T07:37:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2024-08-31T07:37:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.653 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2024-08-31T07:37:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2024-08-31T07:37:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.654 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2024-08-31T07:37:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2024-08-31T07:37:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.655 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2024-08-31T07:37:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2024-08-31T07:37:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2024-08-31T07:37:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.657 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-31T07:37:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2024-08-31T07:37:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2024-08-31T07:37:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.658 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2024-08-31T07:37:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.659 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.660 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2024-08-31T07:37:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.662 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2024-08-31T07:37:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2024-08-31T07:37:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.663 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2024-08-31T07:37:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2024-08-31T07:37:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2024-08-31T07:37:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2024-08-31T07:37:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.665 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2024-08-31T07:37:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T07:37:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2024-08-31T07:37:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2024-08-31T07:37:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:22.666 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-08-31T07:37:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-08-31T07:37:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-31T07:37:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T07:37:22.667 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2024-08-31T07:37:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2024-08-31T07:37:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.668 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2024-08-31T07:37:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2024-08-31T07:37:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-31T07:37:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.669 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2024-08-31T07:37:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2024-08-31T07:37:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2024-08-31T07:37:22.670 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-31T07:37:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-31T07:37:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.671 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2024-08-31T07:37:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2024-08-31T07:37:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2024-08-31T07:37:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2024-08-31T07:37:22.672 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2024-08-31T07:37:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2024-08-31T07:37:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-31T07:37:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2024-08-31T07:37:22.673 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2024-08-31T07:37:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2024-08-31T07:37:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.674 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2024-08-31T07:37:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T07:37:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2024-08-31T07:37:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2024-08-31T07:37:22.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2024-08-31T07:37:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2024-08-31T07:37:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-31T07:37:22.676 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T07:37:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.677 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.678 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.679 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.680 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.682 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.683 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.684 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.685 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2024-08-31T07:37:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2024-08-31T07:37:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.686 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-31T07:37:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-08-31T07:37:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-08-31T07:37:22.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2024-08-31T07:37:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2024-08-31T07:37:22.688 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2024-08-31T07:37:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "read", 2024-08-31T07:37:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap", 2024-08-31T07:37:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap-read" 2024-08-31T07:37:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2024-08-31T07:37:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2024-08-31T07:37:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2024-08-31T07:37:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.690 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-31T07:37:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:22.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-31T07:37:22.692 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2024-08-31T07:37:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2024-08-31T07:37:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:22.695 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-31T07:37:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-31T07:37:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.696 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2024-08-31T07:37:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2024-08-31T07:37:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.697 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-31T07:37:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.698 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2024-08-31T07:37:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2024-08-31T07:37:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2024-08-31T07:37:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.699 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-08-31T07:37:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-31T07:37:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2024-08-31T07:37:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2024-08-31T07:37:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2024-08-31T07:37:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.701 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-31T07:37:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2024-08-31T07:37:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2024-08-31T07:37:22.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2024-08-31T07:37:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-31T07:37:22.703 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2024-08-31T07:37:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2024-08-31T07:37:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.704 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-31T07:37:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-31T07:37:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.705 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2024-08-31T07:37:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2024-08-31T07:37:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.706 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-31T07:37:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2024-08-31T07:37:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2024-08-31T07:37:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2024-08-31T07:37:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.708 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-31T07:37:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2024-08-31T07:37:22.709 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2024-08-31T07:37:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.710 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2024-08-31T07:37:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2024-08-31T07:37:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2024-08-31T07:37:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-31T07:37:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2024-08-31T07:37:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2024-08-31T07:37:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.712 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-31T07:37:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2024-08-31T07:37:22.713 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2024-08-31T07:37:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.714 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-31T07:37:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2024-08-31T07:37:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2024-08-31T07:37:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.715 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-31T07:37:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.716 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2024-08-31T07:37:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2024-08-31T07:37:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-31T07:37:22.717 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.718 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2024-08-31T07:37:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2024-08-31T07:37:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-31T07:37:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.719 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2024-08-31T07:37:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2024-08-31T07:37:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2024-08-31T07:37:22.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-31T07:37:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.721 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2024-08-31T07:37:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2024-08-31T07:37:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2024-08-31T07:37:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.722 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-31T07:37:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.723 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2024-08-31T07:37:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2024-08-31T07:37:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-31T07:37:22.724 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2024-08-31T07:37:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.725 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2024-08-31T07:37:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2024-08-31T07:37:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-31T07:37:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.726 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2024-08-31T07:37:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2024-08-31T07:37:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2024-08-31T07:37:22.727 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-31T07:37:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.728 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2024-08-31T07:37:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2024-08-31T07:37:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2024-08-31T07:37:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.729 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-31T07:37:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2024-08-31T07:37:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2024-08-31T07:37:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2024-08-31T07:37:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.731 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-31T07:37:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2024-08-31T07:37:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2024-08-31T07:37:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2024-08-31T07:37:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-31T07:37:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.733 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2024-08-31T07:37:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nginx": { 2024-08-31T07:37:22.734 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nginx", 2024-08-31T07:37:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-31T07:37:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.735 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nginx container image", 2024-08-31T07:37:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2024-08-31T07:37:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2024-08-31T07:37:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.736 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-31T07:37:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.737 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2024-08-31T07:37:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2024-08-31T07:37:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.738 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-08-31T07:37:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2024-08-31T07:37:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_oauth2_proxy": { 2024-08-31T07:37:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_oauth2_proxy", 2024-08-31T07:37:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-08-31T07:37:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.740 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "oauth2-proxy container image", 2024-08-31T07:37:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2024-08-31T07:37:22.741 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2024-08-31T07:37:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-31T07:37:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.742 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2024-08-31T07:37:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2024-08-31T07:37:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2024-08-31T07:37:22.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-31T07:37:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2024-08-31T07:37:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_samba": { 2024-08-31T07:37:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_samba", 2024-08-31T07:37:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.745 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-31T07:37:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Samba/SMB container image", 2024-08-31T07:37:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.746 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2024-08-31T07:37:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2024-08-31T07:37:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-31T07:37:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.747 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2024-08-31T07:37:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2024-08-31T07:37:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2024-08-31T07:37:22.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2024-08-31T07:37:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.749 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2024-08-31T07:37:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2024-08-31T07:37:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.750 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-31T07:37:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2024-08-31T07:37:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2024-08-31T07:37:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2024-08-31T07:37:22.752 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-31T07:37:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.753 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2024-08-31T07:37:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2024-08-31T07:37:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2024-08-31T07:37:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.754 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-31T07:37:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2024-08-31T07:37:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2024-08-31T07:37:22.755 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2024-08-31T07:37:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2024-08-31T07:37:22.756 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2024-08-31T07:37:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2024-08-31T07:37:22.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-31T07:37:22.758 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2024-08-31T07:37:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2024-08-31T07:37:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.759 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2024-08-31T07:37:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.760 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "grafana_dashboards_path": { 2024-08-31T07:37:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "grafana_dashboards_path", 2024-08-31T07:37:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.761 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-31T07:37:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-31T07:37:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2024-08-31T07:37:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2024-08-31T07:37:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.763 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2024-08-31T07:37:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "hw_monitoring": { 2024-08-31T07:37:22.764 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hw_monitoring", 2024-08-31T07:37:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-31T07:37:22.765 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2024-08-31T07:37:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2024-08-31T07:37:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.766 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-31T07:37:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.769 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2024-08-31T07:37:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2024-08-31T07:37:22.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-31T07:37:22.771 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.772 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-31T07:37:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.773 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.774 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.775 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.777 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-31T07:37:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-31T07:37:22.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-31T07:37:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-31T07:37:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-31T07:37:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.780 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2024-08-31T07:37:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-31T07:37:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.782 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2024-08-31T07:37:22.782 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2024-08-31T07:37:22.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.782 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.782 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.782 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:22.782 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.782 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.783 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.783 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2024-08-31T07:37:22.783 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.783 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.783 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2024-08-31T07:37:22.784 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2024-08-31T07:37:22.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.784 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.784 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.784 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:22.784 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.784 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.784 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.785 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-31T07:37:22.785 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.785 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.785 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2024-08-31T07:37:22.785 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2024-08-31T07:37:22.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.786 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.786 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.786 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.786 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.786 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.786 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.786 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2024-08-31T07:37:22.787 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.787 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.787 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.787 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2024-08-31T07:37:22.787 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2024-08-31T07:37:22.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.787 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.788 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.788 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2024-08-31T07:37:22.788 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.788 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.788 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2024-08-31T07:37:22.788 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2024-08-31T07:37:22.788 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.789 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2024-08-31T07:37:22.789 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.789 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.789 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.789 INFO:teuthology.orchestra.run.smithi032.stdout: "oob_default_addr": { 2024-08-31T07:37:22.789 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "oob_default_addr", 2024-08-31T07:37:22.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.790 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.790 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "169.254.1.1", 2024-08-31T07:37:22.790 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.790 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.790 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.790 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-31T07:37:22.790 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.791 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.791 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.791 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2024-08-31T07:37:22.791 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2024-08-31T07:37:22.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.791 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.791 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-31T07:37:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-31T07:37:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2024-08-31T07:37:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2024-08-31T07:37:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.793 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-31T07:37:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2024-08-31T07:37:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2024-08-31T07:37:22.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-31T07:37:22.795 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2024-08-31T07:37:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2024-08-31T07:37:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-31T07:37:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.797 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2024-08-31T07:37:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2024-08-31T07:37:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-31T07:37:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2024-08-31T07:37:22.799 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2024-08-31T07:37:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.800 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-31T07:37:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2024-08-31T07:37:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2024-08-31T07:37:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.801 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2024-08-31T07:37:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2024-08-31T07:37:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.802 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.803 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.804 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2024-08-31T07:37:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2024-08-31T07:37:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.805 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-31T07:37:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2024-08-31T07:37:22.806 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2024-08-31T07:37:22.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.807 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.807 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.807 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.807 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.807 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.807 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.807 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-31T07:37:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2024-08-31T07:37:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2024-08-31T07:37:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.808 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-31T07:37:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.809 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2024-08-31T07:37:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2024-08-31T07:37:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.810 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2024-08-31T07:37:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2024-08-31T07:37:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2024-08-31T07:37:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.812 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-31T07:37:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2024-08-31T07:37:22.813 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2024-08-31T07:37:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.814 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-31T07:37:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.815 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2024-08-31T07:37:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.816 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.817 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.819 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.821 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.822 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.823 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2024-08-31T07:37:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2024-08-31T07:37:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2024-08-31T07:37:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.825 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-31T07:37:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.827 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2024-08-31T07:37:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2024-08-31T07:37:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2024-08-31T07:37:22.828 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-31T07:37:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2024-08-31T07:37:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.830 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-31T07:37:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-31T07:37:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.831 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2024-08-31T07:37:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-31T07:37:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.832 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.833 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-31T07:37:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-31T07:37:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.834 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.835 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2024-08-31T07:37:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2024-08-31T07:37:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.836 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-31T07:37:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-31T07:37:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.838 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2024-08-31T07:37:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-31T07:37:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.839 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.840 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-31T07:37:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-31T07:37:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.842 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-31T07:37:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-31T07:37:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.843 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.844 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-31T07:37:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-31T07:37:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-31T07:37:22.846 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-31T07:37:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.847 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-31T07:37:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-31T07:37:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.848 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.849 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-31T07:37:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-31T07:37:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.850 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-31T07:37:22.851 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-31T07:37:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.852 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-31T07:37:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-31T07:37:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.854 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2024-08-31T07:37:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-31T07:37:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-08-31T07:37:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.855 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-31T07:37:22.856 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-31T07:37:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.857 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2024-08-31T07:37:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2024-08-31T07:37:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.858 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.859 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2024-08-31T07:37:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-31T07:37:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-08-31T07:37:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-31T07:37:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-31T07:37:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.862 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.863 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-31T07:37:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-31T07:37:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.864 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-31T07:37:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-31T07:37:22.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.866 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-31T07:37:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-31T07:37:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.867 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.868 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-31T07:37:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-31T07:37:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "[]", 2024-08-31T07:37:22.869 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.870 INFO:teuthology.orchestra.run.smithi032.stdout: "MULTICLUSTER_CONFIG": { 2024-08-31T07:37:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-31T07:37:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "{}", 2024-08-31T07:37:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.871 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2024-08-31T07:37:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-31T07:37:22.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.873 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-31T07:37:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-31T07:37:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.874 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.875 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-31T07:37:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-31T07:37:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.876 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-31T07:37:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-31T07:37:22.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.878 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-31T07:37:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-31T07:37:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.879 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-31T07:37:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-31T07:37:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.881 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-31T07:37:22.882 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-31T07:37:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.883 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-31T07:37:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-31T07:37:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.884 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.885 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-31T07:37:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-31T07:37:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.886 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2024-08-31T07:37:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-31T07:37:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.888 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.889 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-31T07:37:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-31T07:37:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-31T07:37:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.890 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-31T07:37:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-31T07:37:22.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.892 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.893 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.893 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.893 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-31T07:37:22.893 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-31T07:37:22.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.893 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.893 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.894 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2024-08-31T07:37:22.894 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.894 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.894 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.894 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.894 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.894 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.894 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.895 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-31T07:37:22.895 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-31T07:37:22.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.895 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.895 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.895 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2024-08-31T07:37:22.895 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.896 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.896 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.896 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.896 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.896 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.896 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.896 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2024-08-31T07:37:22.897 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-31T07:37:22.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.897 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.897 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.897 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.897 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.897 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.897 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.898 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.898 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.898 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.898 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.898 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-31T07:37:22.898 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-31T07:37:22.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.899 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.899 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.899 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2024-08-31T07:37:22.899 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.899 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.899 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.899 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.899 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2024-08-31T07:37:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-31T07:37:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.900 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.901 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.901 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.901 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.901 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.901 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.901 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.901 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.902 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2024-08-31T07:37:22.902 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-31T07:37:22.902 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.902 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.902 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.902 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.902 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: "UNSAFE_TLS_v1_2": { 2024-08-31T07:37:22.903 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-31T07:37:22.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.904 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.904 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.904 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.904 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.904 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.904 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.905 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.905 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.905 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-31T07:37:22.905 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-31T07:37:22.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.905 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.906 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.906 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.906 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.906 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.906 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.906 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.906 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.907 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.907 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-31T07:37:22.907 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-31T07:37:22.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.907 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.907 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:22.908 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.908 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.908 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.908 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.908 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.908 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.908 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-31T07:37:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-31T07:37:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.909 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.910 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.910 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.910 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.910 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.910 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2024-08-31T07:37:22.910 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2024-08-31T07:37:22.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.911 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.911 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.911 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.911 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.911 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.911 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.911 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.912 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.912 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.912 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2024-08-31T07:37:22.912 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2024-08-31T07:37:22.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.912 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2024-08-31T07:37:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2024-08-31T07:37:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.915 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.915 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.915 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.915 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2024-08-31T07:37:22.915 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.915 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.916 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2024-08-31T07:37:22.916 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2024-08-31T07:37:22.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.916 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.916 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2024-08-31T07:37:22.916 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.916 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-08-31T07:37:22.917 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-08-31T07:37:22.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.918 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.918 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.918 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.918 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.918 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.918 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.919 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.920 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.920 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.920 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.920 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.920 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.920 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.920 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.920 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.921 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.921 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.921 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.921 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.921 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.921 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.921 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.922 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.922 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.922 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.922 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.922 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.923 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.923 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.923 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.923 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.923 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.923 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.924 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.924 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.924 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.924 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.924 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.924 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.924 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.925 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.925 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.925 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.925 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.925 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.925 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.925 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.925 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.926 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.926 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.926 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.926 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.926 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.927 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.927 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.927 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.927 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.927 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.927 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.928 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.928 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2024-08-31T07:37:22.928 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2024-08-31T07:37:22.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.928 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.928 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.929 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.929 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.929 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.929 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.929 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2024-08-31T07:37:22.929 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.929 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.930 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2024-08-31T07:37:22.930 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2024-08-31T07:37:22.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.930 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.930 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.930 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.930 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.931 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-08-31T07:37:22.932 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-08-31T07:37:22.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.932 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.932 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-08-31T07:37:22.932 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.932 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.932 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-08-31T07:37:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-08-31T07:37:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.933 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.934 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.934 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2024-08-31T07:37:22.934 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.934 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.934 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.934 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.934 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.935 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.936 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.936 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.936 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.936 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.936 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.936 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.937 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-08-31T07:37:22.937 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-08-31T07:37:22.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.937 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.937 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.937 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.938 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.938 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.938 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.938 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.938 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.938 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2024-08-31T07:37:22.939 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2024-08-31T07:37:22.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.939 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.939 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2024-08-31T07:37:22.939 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.939 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.939 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.940 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.940 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.940 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.940 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.940 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-08-31T07:37:22.940 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-08-31T07:37:22.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.941 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.941 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2024-08-31T07:37:22.941 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.941 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.941 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.941 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.941 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2024-08-31T07:37:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-08-31T07:37:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-08-31T07:37:22.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.943 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.943 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-08-31T07:37:22.943 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-08-31T07:37:22.943 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-08-31T07:37:22.943 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.943 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2024-08-31T07:37:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2024-08-31T07:37:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.944 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.945 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.945 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.945 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.945 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.945 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.945 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.945 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.946 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.946 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2024-08-31T07:37:22.946 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.946 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2024-08-31T07:37:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2024-08-31T07:37:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.947 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.948 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.948 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2024-08-31T07:37:22.948 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.948 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.948 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.949 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.949 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.949 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.949 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.950 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.950 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.950 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.950 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.950 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.950 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.950 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.950 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.951 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.953 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.953 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.953 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.953 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.953 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.954 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.954 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.954 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.954 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.954 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.954 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.954 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.954 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.955 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.955 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.955 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.955 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.955 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.955 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.956 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.957 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.957 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.957 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.957 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2024-08-31T07:37:22.957 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2024-08-31T07:37:22.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.957 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.958 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2024-08-31T07:37:22.958 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.958 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.958 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.958 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-31T07:37:22.958 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.959 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.959 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2024-08-31T07:37:22.959 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2024-08-31T07:37:22.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.959 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.960 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2024-08-31T07:37:22.960 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.960 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.960 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.960 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-31T07:37:22.960 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2024-08-31T07:37:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2024-08-31T07:37:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2024-08-31T07:37:22.961 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.962 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.962 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.962 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2024-08-31T07:37:22.962 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.962 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.962 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2024-08-31T07:37:22.963 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2024-08-31T07:37:22.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.963 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.963 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-08-31T07:37:22.963 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.963 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.963 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.964 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-31T07:37:22.964 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.964 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.964 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2024-08-31T07:37:22.964 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2024-08-31T07:37:22.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.965 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.965 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.965 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:22.965 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.965 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.965 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.965 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-31T07:37:22.966 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.966 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.966 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.966 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:22.966 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:22.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.966 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.967 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:22.967 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.967 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.967 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.967 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2024-08-31T07:37:22.967 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.967 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.968 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.968 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.968 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.968 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.969 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.969 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.969 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.969 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.969 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.969 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.970 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2024-08-31T07:37:22.970 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2024-08-31T07:37:22.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.970 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.970 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2024-08-31T07:37:22.970 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.970 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.970 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.971 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-31T07:37:22.971 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.971 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.972 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2024-08-31T07:37:22.972 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:22.972 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:22.972 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.972 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.972 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.972 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.973 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.973 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.973 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.973 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.973 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.973 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.973 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.973 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.974 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.974 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.974 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.974 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.974 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.974 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.974 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.974 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.975 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.975 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.975 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.975 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.975 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.975 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.976 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.976 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.976 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.976 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.976 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:22.976 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:22.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.977 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.977 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:22.977 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.977 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.977 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.977 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.977 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.978 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.978 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.978 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.978 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.978 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.978 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.978 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.978 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.979 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.979 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:22.979 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:22.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.979 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.979 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.979 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.980 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.980 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.980 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.980 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.980 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.980 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.980 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.981 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2024-08-31T07:37:22.981 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2024-08-31T07:37:22.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.981 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.981 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2024-08-31T07:37:22.981 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.981 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.982 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.982 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.982 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.982 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.982 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2024-08-31T07:37:22.982 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2024-08-31T07:37:22.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.983 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.983 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.983 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2024-08-31T07:37:22.983 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.983 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.983 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.983 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.984 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.985 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.985 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2024-08-31T07:37:22.985 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.985 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.985 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.985 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.985 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.986 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.986 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:22.986 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:22.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.986 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:22.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.986 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:22.987 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.987 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.987 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.987 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.987 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.987 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:22.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:22.988 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2024-08-31T07:37:22.988 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2024-08-31T07:37:22.988 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2024-08-31T07:37:22.988 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:22.988 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2024-08-31T07:37:22.989 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2024-08-31T07:37:22.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:22.989 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.989 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.989 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2024-08-31T07:37:22.989 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.989 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.990 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.990 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-31T07:37:22.990 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.990 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.990 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.990 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2024-08-31T07:37:22.990 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2024-08-31T07:37:22.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.991 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.991 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2024-08-31T07:37:22.991 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.991 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.991 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.991 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-31T07:37:22.992 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.992 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.992 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2024-08-31T07:37:22.992 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2024-08-31T07:37:22.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.992 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.993 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.993 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.993 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.993 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.993 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2024-08-31T07:37:22.993 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.994 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.994 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:22.994 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:22.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:22.994 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:22.994 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2024-08-31T07:37:22.995 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2024-08-31T07:37:22.995 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.995 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.995 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-31T07:37:22.995 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.995 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.995 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:22.996 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:22.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:22.996 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.996 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:22.996 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.996 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.996 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:22.997 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:22.997 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:22.997 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:22.997 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:22.997 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:22.997 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:22.997 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:22.997 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.998 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.998 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:22.998 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:22.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:22.998 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:22.998 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:22.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:22.998 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:22.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:22.999 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:22.999 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:22.999 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:22.999 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:22.999 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:22.999 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:22.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.000 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.001 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.001 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.001 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.001 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.001 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.001 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.001 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.001 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.002 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.002 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.002 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.002 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.002 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.002 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.003 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.003 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.003 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.003 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.003 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.003 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.004 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.004 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.004 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.004 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2024-08-31T07:37:23.004 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2024-08-31T07:37:23.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.005 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.005 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.005 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.005 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.005 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.005 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2024-08-31T07:37:23.006 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.006 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.006 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.006 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2024-08-31T07:37:23.006 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2024-08-31T07:37:23.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.006 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.007 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2024-08-31T07:37:23.007 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.007 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.007 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2024-08-31T07:37:23.007 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.008 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.008 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.008 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.008 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.008 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.008 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.009 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.009 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.009 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.009 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.009 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.009 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.009 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.009 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2024-08-31T07:37:23.010 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2024-08-31T07:37:23.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.010 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.010 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2024-08-31T07:37:23.010 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.010 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.010 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.011 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T07:37:23.011 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.011 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.011 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.011 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2024-08-31T07:37:23.011 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2024-08-31T07:37:23.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.011 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.012 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:23.012 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:23.012 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2024-08-31T07:37:23.012 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.012 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-31T07:37:23.012 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.013 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.013 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2024-08-31T07:37:23.013 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2024-08-31T07:37:23.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.013 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.013 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.014 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.014 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.014 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.014 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2024-08-31T07:37:23.014 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.014 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.014 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.015 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2024-08-31T07:37:23.015 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2024-08-31T07:37:23.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.015 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.015 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2024-08-31T07:37:23.015 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.015 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.016 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.016 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T07:37:23.016 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.016 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.016 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.017 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2024-08-31T07:37:23.017 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.017 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.017 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.017 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.017 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.018 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.018 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.018 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.018 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.018 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.018 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.019 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.019 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.019 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.019 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.019 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.019 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.019 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.020 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.020 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.020 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.020 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.020 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.021 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.021 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.021 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.021 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.021 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.021 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.021 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.022 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.022 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.022 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.022 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.022 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.023 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.023 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.023 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.023 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.023 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.023 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.023 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.024 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.024 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.024 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.024 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.024 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.024 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.024 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.024 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.025 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.025 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.025 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.025 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.025 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.025 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.025 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.026 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.026 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.026 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.026 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.026 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.027 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.027 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.027 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.027 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.027 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.027 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.027 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.028 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.028 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2024-08-31T07:37:23.028 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.028 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.029 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.029 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.029 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.029 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.029 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.029 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.029 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.030 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.030 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.030 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.030 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.030 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.030 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.030 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.030 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.031 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.031 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.031 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.031 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.031 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.032 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.032 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.032 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.032 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.032 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.032 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.033 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.033 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.033 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.033 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.033 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.033 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.034 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.034 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.034 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.034 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.034 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.034 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.034 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.035 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.035 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.035 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.035 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.035 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.035 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.035 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.036 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.036 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.036 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.036 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.036 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.037 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.037 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.037 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.037 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.037 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.037 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.038 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.038 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.038 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.038 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.038 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.038 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.039 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.039 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.039 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.039 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.039 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2024-08-31T07:37:23.040 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.040 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.040 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.040 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2024-08-31T07:37:23.040 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2024-08-31T07:37:23.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.040 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.040 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.041 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2024-08-31T07:37:23.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.041 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.041 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.041 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-31T07:37:23.041 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.041 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.042 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2024-08-31T07:37:23.042 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2024-08-31T07:37:23.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.042 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.042 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.042 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:23.042 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2024-08-31T07:37:23.043 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.043 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.043 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-31T07:37:23.043 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.043 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.043 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.043 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.044 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.044 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.044 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.044 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.044 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.044 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.044 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.045 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.045 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.045 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.045 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.045 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.045 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.045 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.045 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.046 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.046 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.046 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.046 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.046 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.047 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.047 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.047 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.047 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.047 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.047 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.047 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.048 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.048 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.048 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.048 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.048 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.048 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.049 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.049 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.049 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.049 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.049 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.049 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.049 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.049 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.050 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.050 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.050 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.050 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.050 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.050 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.051 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.051 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.051 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.051 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.051 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.051 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.052 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.052 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.052 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.052 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.052 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.052 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.053 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.053 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.053 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.053 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.053 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.053 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.053 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.054 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.054 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2024-08-31T07:37:23.054 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.055 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.055 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.055 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2024-08-31T07:37:23.055 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2024-08-31T07:37:23.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.055 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.055 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2024-08-31T07:37:23.055 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.056 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.056 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.056 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2024-08-31T07:37:23.056 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-31T07:37:23.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.056 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.057 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.057 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.057 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.057 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.057 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.057 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.058 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.058 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.058 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.058 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.058 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.058 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.058 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.058 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.059 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.059 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.059 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.059 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.059 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.060 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.060 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.060 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.060 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.060 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.060 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.060 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.061 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.061 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.061 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.061 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.061 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.062 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.062 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.062 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.062 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.062 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.062 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.062 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.062 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.063 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.063 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.063 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.063 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.063 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.063 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.064 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.064 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.064 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.064 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.064 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.065 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.065 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.065 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.065 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.065 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2024-08-31T07:37:23.066 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2024-08-31T07:37:23.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.066 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.066 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.066 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.066 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.066 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.067 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2024-08-31T07:37:23.067 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-31T07:37:23.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.067 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.067 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2024-08-31T07:37:23.067 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2024-08-31T07:37:23.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.068 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.068 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.068 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2024-08-31T07:37:23.068 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.068 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.068 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.068 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2024-08-31T07:37:23.068 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.069 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.069 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2024-08-31T07:37:23.069 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2024-08-31T07:37:23.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.069 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.070 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2024-08-31T07:37:23.070 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.070 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.070 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.070 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2024-08-31T07:37:23.070 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.071 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.071 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2024-08-31T07:37:23.071 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2024-08-31T07:37:23.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.071 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.071 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.071 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.072 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.072 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.072 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2024-08-31T07:37:23.072 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.072 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.073 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.073 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.073 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.073 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.073 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.073 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.073 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.074 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.074 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.074 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.074 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.074 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.074 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2024-08-31T07:37:23.074 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2024-08-31T07:37:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2024-08-31T07:37:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-31T07:37:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-31T07:37:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2024-08-31T07:37:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.088 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.088 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2024-08-31T07:37:23.088 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.088 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.089 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.089 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.089 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.089 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.089 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.089 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.089 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.089 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.090 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.090 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.090 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.090 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.090 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.090 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.090 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.090 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.091 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.091 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.091 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.091 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.091 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.091 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.092 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.092 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.092 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.092 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.092 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.092 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.092 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.092 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.093 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.093 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.093 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.093 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.094 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.094 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.094 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.094 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.094 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.094 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.094 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.094 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.095 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.095 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.095 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.095 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.095 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.095 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.096 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.096 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.096 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.096 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.096 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.097 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.097 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.097 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.097 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.097 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.098 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.098 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.098 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.098 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.098 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.098 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.098 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.099 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.099 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.099 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.099 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2024-08-31T07:37:23.100 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.100 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.100 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.100 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.100 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.100 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.100 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.101 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.101 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.101 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.101 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.101 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.101 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.101 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.102 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.102 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.102 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.102 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.102 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.102 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.102 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.102 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.103 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.103 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.103 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.103 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.103 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.104 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.104 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.104 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.104 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.104 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.105 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.105 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.105 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.105 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.105 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.105 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.106 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.106 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.106 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.106 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.106 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.106 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.106 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.106 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.107 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.107 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.107 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.107 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.107 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.107 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.107 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.108 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.108 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.108 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.108 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.108 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.108 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.108 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.109 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.109 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.109 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.109 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.109 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.109 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.110 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.110 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.110 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.110 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.110 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.110 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.110 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.110 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.111 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-08-31T07:37:23.111 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.111 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.111 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.111 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2024-08-31T07:37:23.111 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2024-08-31T07:37:23.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.112 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.112 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.112 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.112 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.112 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-31T07:37:23.113 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.113 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.113 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.113 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.113 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.114 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.114 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.114 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.114 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.114 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.114 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.114 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.114 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.115 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.115 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.115 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.115 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.115 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.115 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.115 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.116 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.116 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.116 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.116 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.116 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.117 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.117 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.117 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.117 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.117 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.117 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.118 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.118 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.118 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.118 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.118 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.118 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.118 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.119 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.119 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.119 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.119 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.119 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.119 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.119 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.119 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.120 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.120 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.120 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.120 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.120 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.120 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.121 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.121 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.121 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.121 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.121 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.121 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.121 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.122 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2024-08-31T07:37:23.122 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2024-08-31T07:37:23.122 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.122 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.122 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.122 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.122 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.122 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.122 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.123 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2024-08-31T07:37:23.123 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2024-08-31T07:37:23.123 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2024-08-31T07:37:23.123 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.123 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2024-08-31T07:37:23.123 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.123 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.124 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.124 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.124 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.124 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.125 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.125 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.125 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.125 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.125 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.125 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.125 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.126 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2024-08-31T07:37:23.126 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.126 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.126 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.126 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.127 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.127 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.127 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.127 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.127 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.127 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.127 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.128 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.128 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.128 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.128 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.128 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.128 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.128 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.129 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.129 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.129 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.129 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.129 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.130 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.130 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.130 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.130 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.130 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.130 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.130 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.131 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.131 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.131 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.131 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.131 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.132 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.132 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.132 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.132 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.132 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.132 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.132 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.132 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.133 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.133 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.133 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.133 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.133 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.134 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.134 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.134 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.134 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.134 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.134 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.135 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.135 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.135 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.135 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.135 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.135 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.135 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.136 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.136 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.136 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.136 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.136 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.136 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.137 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.137 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.138 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2024-08-31T07:37:23.138 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.138 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.138 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.138 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.138 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.138 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.139 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.139 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.139 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.139 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.139 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.139 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.140 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.140 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.140 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.140 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.140 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.140 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.140 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.140 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.141 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.141 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.141 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.141 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.141 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.142 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.142 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.142 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.142 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.142 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.142 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.143 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.143 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.143 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.143 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.143 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.144 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.144 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.144 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.144 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.144 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.144 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.144 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.144 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.145 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.145 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.145 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.145 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.145 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.145 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.146 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.146 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.146 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.146 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.146 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.146 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.147 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.147 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.147 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.147 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.147 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.148 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.148 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.148 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.148 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.148 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.148 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.148 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.148 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.149 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2024-08-31T07:37:23.149 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.149 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.149 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.149 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.150 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.150 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.150 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.150 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.150 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.150 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.150 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.151 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.151 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.151 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.151 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.151 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.151 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.151 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.151 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.152 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.152 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.152 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.152 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.152 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.153 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.153 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.153 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.153 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.153 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.153 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.153 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.153 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.154 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.154 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.154 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.154 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.154 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.155 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.155 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.155 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.155 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.155 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.155 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.155 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.155 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.156 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.156 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.156 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.156 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.156 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.156 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.157 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.157 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.157 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.157 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.157 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.157 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.158 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.158 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.158 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.158 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.158 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:23.158 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:23.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:23.159 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.159 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.159 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:23.159 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.159 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.159 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.159 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.160 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.160 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.160 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.160 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.160 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.160 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.161 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.161 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.161 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.161 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.161 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.161 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.161 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.162 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.162 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2024-08-31T07:37:23.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2024-08-31T07:37:23.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:23.162 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.162 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2024-08-31T07:37:23.162 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2024-08-31T07:37:23.163 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.163 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.163 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2024-08-31T07:37:23.163 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-31T07:37:23.163 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.163 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.163 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.163 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.164 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2024-08-31T07:37:23.164 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.164 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.164 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.164 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2024-08-31T07:37:23.165 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2024-08-31T07:37:23.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.165 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.165 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.165 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.165 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.165 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.165 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.166 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-31T07:37:23.166 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.166 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.166 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.166 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-08-31T07:37:23.166 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-08-31T07:37:23.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.167 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.167 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.167 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:23.167 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.167 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.167 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.167 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.167 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.168 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.168 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.168 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.168 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.168 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.168 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.169 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.169 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.169 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.169 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.169 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.169 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.169 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.169 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.169 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.170 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.170 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.170 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.170 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.170 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.170 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.170 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.171 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.171 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.171 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.171 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.171 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.171 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.171 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.171 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.172 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.172 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.172 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.172 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.173 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.173 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.173 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.173 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.173 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.173 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.173 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.173 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.174 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.174 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.174 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.174 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.174 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.174 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.175 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.175 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.175 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.175 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.175 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.175 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.175 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.176 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.176 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.176 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.176 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.176 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2024-08-31T07:37:23.177 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2024-08-31T07:37:23.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.177 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.177 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.177 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2024-08-31T07:37:23.177 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.177 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.177 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.178 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2024-08-31T07:37:23.178 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.178 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.178 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.178 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2024-08-31T07:37:23.178 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2024-08-31T07:37:23.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:23.179 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.179 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:23.179 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.179 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.179 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.179 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2024-08-31T07:37:23.179 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.180 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.180 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.180 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.180 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.180 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.180 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.181 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.181 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.181 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.181 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.181 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.181 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.181 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.181 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.182 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2024-08-31T07:37:23.182 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.182 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.182 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.183 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2024-08-31T07:37:23.183 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2024-08-31T07:37:23.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.183 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.183 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:23.183 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.183 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.184 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.184 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.184 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.184 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.184 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2024-08-31T07:37:23.184 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2024-08-31T07:37:23.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.185 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.185 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:23.185 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.185 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.185 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.185 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-31T07:37:23.186 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-31T07:37:23.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.186 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.186 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.186 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.187 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.187 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.187 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.187 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.187 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.187 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.187 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.187 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.188 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.188 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.188 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.188 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.188 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.188 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.188 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.188 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.189 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.189 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.189 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.189 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.189 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.190 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.190 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.190 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.190 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.190 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.190 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2024-08-31T07:37:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2024-08-31T07:37:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-31T07:37:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-31T07:37:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2024-08-31T07:37:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2024-08-31T07:37:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2024-08-31T07:37:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2024-08-31T07:37:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-08-31T07:37:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-08-31T07:37:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2024-08-31T07:37:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-31T07:37:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-08-31T07:37:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-08-31T07:37:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.203 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.203 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2024-08-31T07:37:23.203 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.203 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.203 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.203 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-31T07:37:23.203 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.203 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.204 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.204 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.204 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.204 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.204 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.204 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.205 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.205 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.205 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.205 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.205 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.205 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.205 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.206 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2024-08-31T07:37:23.206 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2024-08-31T07:37:23.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.206 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.206 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.206 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2024-08-31T07:37:23.206 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.206 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.207 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.207 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.207 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.207 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.207 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2024-08-31T07:37:23.208 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2024-08-31T07:37:23.208 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.208 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.208 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.208 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2024-08-31T07:37:23.208 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.208 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.208 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.209 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2024-08-31T07:37:23.209 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2024-08-31T07:37:23.209 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.209 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.209 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.209 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.209 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.210 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2024-08-31T07:37:23.210 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2024-08-31T07:37:23.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.210 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.210 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-08-31T07:37:23.210 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2024-08-31T07:37:23.211 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2024-08-31T07:37:23.211 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.211 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.211 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.211 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.211 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.211 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.212 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2024-08-31T07:37:23.212 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.212 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.212 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.212 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.212 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.213 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.213 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.213 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.213 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.213 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.213 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.213 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.214 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.214 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.214 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.214 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.214 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.214 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.214 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.214 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.215 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.215 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.215 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.215 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.215 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.216 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.216 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.216 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.216 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.216 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.216 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.216 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.216 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.217 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.217 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.217 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.217 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.217 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.217 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.217 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.218 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.218 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.218 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.218 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.218 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.218 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.218 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.218 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.219 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.219 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.219 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.219 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.219 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.220 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.220 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.220 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.220 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.220 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.220 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.220 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.221 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.221 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.221 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2024-08-31T07:37:23.221 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2024-08-31T07:37:23.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.221 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.222 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2024-08-31T07:37:23.222 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.222 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.222 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.222 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.222 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.222 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.223 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.223 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2024-08-31T07:37:23.223 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2024-08-31T07:37:23.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.223 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.223 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.223 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.224 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.224 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.224 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.224 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.224 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.224 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.224 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.225 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.225 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.225 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.225 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.225 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.225 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.225 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.225 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.226 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.226 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.226 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.226 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.226 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.226 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2024-08-31T07:37:23.226 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2024-08-31T07:37:23.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.227 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.227 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.227 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.227 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.227 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.227 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.227 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.228 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.228 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2024-08-31T07:37:23.228 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.229 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.229 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.229 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2024-08-31T07:37:23.229 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2024-08-31T07:37:23.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.229 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.229 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:23.229 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.230 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.230 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.230 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.230 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.230 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.230 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.230 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2024-08-31T07:37:23.231 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2024-08-31T07:37:23.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.231 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.231 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.231 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.231 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.231 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.231 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.232 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.232 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.232 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.232 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.232 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.232 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.233 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.233 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.233 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.233 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.233 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.234 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.234 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.234 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.234 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.234 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.234 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.234 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.234 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.235 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.235 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.235 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.235 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.235 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.235 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.236 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.236 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.236 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.236 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.236 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.237 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.237 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.237 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.237 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.238 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.238 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.238 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.238 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.238 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.238 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.238 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.238 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.239 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.239 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.239 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.239 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.239 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.239 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.240 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.240 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.240 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.240 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.240 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.240 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.240 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.240 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.241 INFO:teuthology.orchestra.run.smithi032.stdout: "max_requests": { 2024-08-31T07:37:23.241 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_requests", 2024-08-31T07:37:23.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.241 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.241 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2024-08-31T07:37:23.241 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.241 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.242 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.242 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-08-31T07:37:23.242 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.242 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.242 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.242 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2024-08-31T07:37:23.243 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2024-08-31T07:37:23.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.243 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.243 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.243 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.243 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.243 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.244 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.244 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.244 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.244 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.244 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2024-08-31T07:37:23.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2024-08-31T07:37:23.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.245 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.245 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.245 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.245 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.245 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.245 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.245 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.245 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.246 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.246 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.246 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.246 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.246 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.246 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.247 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.247 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.247 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.247 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.247 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.247 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.247 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.247 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2024-08-31T07:37:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.248 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.249 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.249 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.249 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.249 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.249 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.249 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.249 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.249 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.250 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.250 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.250 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.250 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.250 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.250 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.250 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.250 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.250 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.251 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.251 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.251 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.251 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.251 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.251 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.252 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.252 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.252 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.252 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.252 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.252 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.252 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.252 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.253 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.253 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.253 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.253 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.254 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.254 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.254 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.254 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.254 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.254 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.254 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.254 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.255 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.255 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.255 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.255 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.255 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.255 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.255 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.256 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.257 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.257 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.257 INFO:teuthology.orchestra.run.smithi032.stdout: "secondary_zone_period_retry_limit": { 2024-08-31T07:37:23.257 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-31T07:37:23.257 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.257 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.257 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2024-08-31T07:37:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-31T07:37:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.258 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.259 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.259 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.259 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.259 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.260 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.260 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.260 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.260 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.260 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.261 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.261 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2024-08-31T07:37:23.261 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.261 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.261 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.261 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.261 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.262 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.262 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.262 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.262 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.262 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.262 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.263 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.263 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.263 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.263 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.263 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.263 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.263 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.263 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.264 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.264 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.264 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.264 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.264 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.264 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.265 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.265 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.265 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.265 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.265 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.265 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.266 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.266 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.266 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.266 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.266 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.267 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.267 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.267 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.267 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.267 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.267 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.267 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.267 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.268 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.268 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.268 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.268 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.268 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.268 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.268 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.269 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.269 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.269 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.269 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.270 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.270 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.270 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.270 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.270 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.270 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.270 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.271 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.271 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.271 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.271 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.271 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.271 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.271 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.271 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.272 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.272 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.272 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2024-08-31T07:37:23.272 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2024-08-31T07:37:23.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.272 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.273 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2024-08-31T07:37:23.273 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.273 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.273 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.273 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-31T07:37:23.273 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.273 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.274 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.274 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2024-08-31T07:37:23.274 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.274 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.275 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.275 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.275 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.275 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.275 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.275 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.275 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.276 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.276 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.276 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.276 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.276 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.276 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.276 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.276 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.281 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.281 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.281 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.281 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.281 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.281 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.281 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.282 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.282 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.282 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.282 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.282 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.282 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.282 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.283 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.283 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.283 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.283 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.283 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.283 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2024-08-31T07:37:23.283 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2024-08-31T07:37:23.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.284 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.284 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.284 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.284 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.284 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.284 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.284 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.285 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.285 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.285 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.285 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2024-08-31T07:37:23.285 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2024-08-31T07:37:23.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.285 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.286 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.286 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2024-08-31T07:37:23.286 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.286 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.286 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.286 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.286 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.286 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.286 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.287 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2024-08-31T07:37:23.287 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2024-08-31T07:37:23.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.287 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.287 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.287 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.287 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.288 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.288 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.288 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.288 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.288 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.288 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.288 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2024-08-31T07:37:23.289 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2024-08-31T07:37:23.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.289 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.289 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.289 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.289 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.289 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.290 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.290 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.290 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.290 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2024-08-31T07:37:23.290 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2024-08-31T07:37:23.290 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2024-08-31T07:37:23.290 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.291 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.291 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.291 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.291 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.291 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.291 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.291 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.292 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.292 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2024-08-31T07:37:23.292 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2024-08-31T07:37:23.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.292 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.292 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.292 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.293 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.293 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.293 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.293 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.293 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.293 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.294 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2024-08-31T07:37:23.294 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2024-08-31T07:37:23.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.294 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.294 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.294 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.295 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.295 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.295 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.295 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.295 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.295 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.295 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2024-08-31T07:37:23.296 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2024-08-31T07:37:23.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.296 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.296 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.296 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:23.296 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.296 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.296 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.296 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.297 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.297 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.297 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.297 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2024-08-31T07:37:23.297 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2024-08-31T07:37:23.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.297 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.298 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.298 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2024-08-31T07:37:23.298 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2024-08-31T07:37:23.298 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.298 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.298 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.299 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.299 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.299 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.299 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.299 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.299 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.300 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.300 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.300 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.300 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.300 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.300 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.301 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2024-08-31T07:37:23.301 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2024-08-31T07:37:23.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.301 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.301 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.301 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.301 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.301 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.302 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.302 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.302 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.302 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.302 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.302 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2024-08-31T07:37:23.302 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2024-08-31T07:37:23.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.303 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.303 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.303 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.303 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.303 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.303 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.303 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.304 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.304 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2024-08-31T07:37:23.304 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2024-08-31T07:37:23.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.304 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.305 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.305 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.305 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.305 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.305 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.305 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.305 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.306 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.306 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smb", 2024-08-31T07:37:23.306 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.306 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.307 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.307 INFO:teuthology.orchestra.run.smithi032.stdout: "internal_store_backend": { 2024-08-31T07:37:23.307 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "internal_store_backend", 2024-08-31T07:37:23.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.307 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.307 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.307 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.308 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.308 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.308 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-31T07:37:23.308 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.308 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.309 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.309 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.309 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.309 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.309 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.309 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.309 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.309 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.310 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.310 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.310 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.310 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.310 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.310 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.310 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.310 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.311 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.311 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.311 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.311 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.311 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.311 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.312 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.312 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.312 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.312 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.312 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.312 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.313 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.313 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.313 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.313 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.313 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.314 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.314 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.314 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.314 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.314 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.314 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.314 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.314 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.316 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.316 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.316 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.316 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.316 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.316 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.317 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.317 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.317 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.317 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.317 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.317 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.318 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.318 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.318 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.318 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.318 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.318 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.319 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.319 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.319 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.319 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.319 INFO:teuthology.orchestra.run.smithi032.stdout: "update_orchestration": { 2024-08-31T07:37:23.319 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "update_orchestration", 2024-08-31T07:37:23.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.320 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.320 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.320 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:23.320 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.320 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.320 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.320 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-31T07:37:23.321 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.321 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.321 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.321 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2024-08-31T07:37:23.322 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.322 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.322 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.322 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2024-08-31T07:37:23.322 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2024-08-31T07:37:23.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.322 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.323 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.323 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.323 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.323 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.323 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2024-08-31T07:37:23.323 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.323 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.324 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2024-08-31T07:37:23.324 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2024-08-31T07:37:23.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.324 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.324 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.324 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.324 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.324 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.325 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.325 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2024-08-31T07:37:23.325 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.325 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.328 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.328 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.328 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.328 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.329 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.329 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.329 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.329 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.329 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.329 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.329 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.329 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.330 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.330 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.330 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.330 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.330 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.330 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.330 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.330 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.331 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.331 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.331 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.331 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.331 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.332 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.332 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.332 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.332 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.332 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.332 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.333 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.333 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.333 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.333 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.333 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.333 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.334 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.334 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.334 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.334 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.334 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.334 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.334 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.334 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.335 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.335 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.335 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.335 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.335 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.336 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.336 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.336 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.336 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.336 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.336 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.337 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.337 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.337 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.337 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.337 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.337 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.338 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.338 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.338 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.338 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.338 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.338 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.338 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.339 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2024-08-31T07:37:23.339 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.339 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.339 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.339 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.340 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.340 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.340 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.340 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.340 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.340 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.340 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.340 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.341 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.341 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.341 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.341 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.341 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.341 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.341 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.341 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.342 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.342 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.342 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.342 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.342 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.343 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.343 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.343 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.343 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.343 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.343 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.343 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.344 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.344 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.344 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.344 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.344 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.344 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.345 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.345 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.345 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.345 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.345 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.345 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.345 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.345 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.346 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.346 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.346 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.346 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.346 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.346 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.347 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.347 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.347 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.347 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.347 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.348 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.348 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.348 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.348 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.348 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.348 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.349 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.349 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.349 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.349 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.349 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.350 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.350 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.350 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.351 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2024-08-31T07:37:23.351 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.351 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.351 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.351 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.351 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.351 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.352 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.352 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.352 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.352 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.352 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.352 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.352 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.352 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.353 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.353 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.353 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.353 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.353 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.353 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.353 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.354 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.354 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.354 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.354 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.354 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.354 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.354 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.354 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.355 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.355 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.355 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.355 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.355 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.355 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.356 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.356 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.356 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.356 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.356 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.356 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.356 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.357 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.357 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.357 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.357 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.357 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.357 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.357 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.357 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.358 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.358 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.358 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.358 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.358 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.358 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.359 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.359 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.359 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.359 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.359 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.359 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.360 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.360 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.360 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.360 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.360 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.361 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.361 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.361 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.361 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.361 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.362 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2024-08-31T07:37:23.362 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.362 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.362 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.362 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2024-08-31T07:37:23.362 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2024-08-31T07:37:23.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.363 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.363 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-31T07:37:23.363 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.363 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.363 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.363 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.364 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.364 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.364 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:23.364 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:23.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:23.364 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.365 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2024-08-31T07:37:23.365 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.365 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.365 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.365 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.365 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.365 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.366 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.366 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.366 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.366 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.366 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.367 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.367 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.367 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.367 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.367 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.367 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.367 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.367 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.368 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.368 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.368 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.368 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.368 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.368 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.368 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.369 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.369 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.369 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.369 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.369 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.370 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.370 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.370 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.370 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.370 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.370 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.371 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.371 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.371 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.371 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.371 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.371 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.371 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.371 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.372 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.372 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.372 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.372 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.372 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.372 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.372 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.373 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.373 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.373 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.373 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.373 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.374 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.374 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.374 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.374 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.374 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.374 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.374 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.374 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.375 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.375 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.375 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.375 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.375 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.375 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.375 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.376 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.376 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.376 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.377 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2024-08-31T07:37:23.377 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.377 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.377 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.377 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2024-08-31T07:37:23.377 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2024-08-31T07:37:23.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.377 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.378 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:23.378 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.378 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.378 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.378 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-31T07:37:23.378 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.379 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.379 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2024-08-31T07:37:23.379 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2024-08-31T07:37:23.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.379 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.379 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:23.379 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.380 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.380 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.380 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-31T07:37:23.380 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.380 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.380 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.381 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2024-08-31T07:37:23.381 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2024-08-31T07:37:23.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.381 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.381 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.381 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:23.381 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.381 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.381 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.382 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-31T07:37:23.382 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.382 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.382 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2024-08-31T07:37:23.382 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2024-08-31T07:37:23.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.383 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.383 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.383 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.383 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.383 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.383 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-31T07:37:23.383 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.384 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.384 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2024-08-31T07:37:23.384 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2024-08-31T07:37:23.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.384 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.385 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.385 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.385 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.385 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.385 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-31T07:37:23.385 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.386 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.386 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2024-08-31T07:37:23.386 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2024-08-31T07:37:23.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.386 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.386 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.386 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.386 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.387 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.387 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.387 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.387 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.387 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.387 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.387 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2024-08-31T07:37:23.388 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2024-08-31T07:37:23.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.388 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.388 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.388 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.388 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.388 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.388 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.389 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.389 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.389 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.389 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.389 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2024-08-31T07:37:23.389 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2024-08-31T07:37:23.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.390 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.390 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.390 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-31T07:37:23.390 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.390 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.390 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.390 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.390 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.391 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.391 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2024-08-31T07:37:23.391 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2024-08-31T07:37:23.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.391 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.391 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.392 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.392 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.392 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.392 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.392 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.392 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.392 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.392 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.393 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:23.393 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:23.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.393 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.393 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2024-08-31T07:37:23.393 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2024-08-31T07:37:23.394 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.394 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.394 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.394 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.394 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.394 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2024-08-31T07:37:23.395 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2024-08-31T07:37:23.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.395 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.395 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.395 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2024-08-31T07:37:23.395 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.395 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.395 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.395 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.396 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.396 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.396 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.396 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2024-08-31T07:37:23.396 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2024-08-31T07:37:23.396 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.397 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.397 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.397 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.397 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.397 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.397 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.397 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.398 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.398 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2024-08-31T07:37:23.398 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2024-08-31T07:37:23.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.398 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.398 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.399 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.399 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.399 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.399 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.399 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.399 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.399 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.400 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.400 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.400 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.400 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.400 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.400 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.400 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.401 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.401 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.401 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.401 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.401 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.401 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.401 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.401 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.402 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.402 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.402 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.402 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.402 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.402 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.403 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.403 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.403 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.403 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.403 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.403 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.404 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.404 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.404 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.404 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.404 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.405 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.405 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.405 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.405 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.405 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.405 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.405 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2024-08-31T07:37:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2024-08-31T07:37:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.410 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.410 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.410 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.410 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.410 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.410 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2024-08-31T07:37:23.410 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2024-08-31T07:37:23.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.411 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.411 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.411 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.411 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.411 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.411 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.411 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.412 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.412 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.412 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.412 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.412 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.412 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.412 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.413 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.413 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.413 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.413 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.413 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.413 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.413 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.413 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.414 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.414 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2024-08-31T07:37:23.414 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2024-08-31T07:37:23.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.414 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.414 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.414 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-31T07:37:23.415 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.415 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.415 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.415 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.415 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.415 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.415 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.416 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.416 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2024-08-31T07:37:23.416 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.416 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.416 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.416 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.416 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.417 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.417 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.417 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.417 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.417 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.417 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.417 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.418 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.418 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.418 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.418 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.418 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.418 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.418 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.418 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.418 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.419 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.419 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.419 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.419 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.420 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.420 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.420 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.420 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.420 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.420 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.420 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.421 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.422 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.423 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.423 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.423 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.423 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.423 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.423 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.424 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.424 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.424 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.424 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.424 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.424 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.425 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.425 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.425 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.425 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.425 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.426 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.426 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.426 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.426 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.426 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.426 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.426 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.426 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.426 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.427 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.427 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2024-08-31T07:37:23.427 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.427 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.428 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.428 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.428 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.428 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.428 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.428 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.429 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.429 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.429 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.429 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.429 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.429 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.429 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.430 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.430 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.430 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.430 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.430 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.430 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.430 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.431 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.431 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.431 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.431 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.431 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.431 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.431 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.431 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.431 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.432 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.432 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.432 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.432 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.432 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.432 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2024-08-31T07:37:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2024-08-31T07:37:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2024-08-31T07:37:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-31T07:37:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2024-08-31T07:37:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2024-08-31T07:37:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2024-08-31T07:37:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2024-08-31T07:37:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2024-08-31T07:37:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-31T07:37:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_no_wait": { 2024-08-31T07:37:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_no_wait", 2024-08-31T07:37:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2024-08-31T07:37:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-31T07:37:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2024-08-31T07:37:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2024-08-31T07:37:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2024-08-31T07:37:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2024-08-31T07:37:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2024-08-31T07:37:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2024-08-31T07:37:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2024-08-31T07:37:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.446 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2024-08-31T07:37:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.447 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2024-08-31T07:37:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2024-08-31T07:37:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.448 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2024-08-31T07:37:23.449 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2024-08-31T07:37:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2024-08-31T07:37:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2024-08-31T07:37:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.450 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2024-08-31T07:37:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2024-08-31T07:37:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.451 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.452 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.453 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2024-08-31T07:37:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2024-08-31T07:37:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.454 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.455 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2024-08-31T07:37:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2024-08-31T07:37:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2024-08-31T07:37:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.456 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2024-08-31T07:37:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2024-08-31T07:37:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2024-08-31T07:37:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2024-08-31T07:37:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2024-08-31T07:37:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2024-08-31T07:37:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2024-08-31T07:37:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.457 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2024-08-31T07:37:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2024-08-31T07:37:23.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2024-08-31T07:37:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2024-08-31T07:37:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.459 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "sqlite3_killpoint": { 2024-08-31T07:37:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sqlite3_killpoint", 2024-08-31T07:37:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "dev", 2024-08-31T07:37:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-08-31T07:37:23.460 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2024-08-31T07:37:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.461 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2024-08-31T07:37:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2024-08-31T07:37:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2024-08-31T07:37:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.462 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2024-08-31T07:37:23.463 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2024-08-31T07:37:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2024-08-31T07:37:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2024-08-31T07:37:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.464 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2024-08-31T07:37:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2024-08-31T07:37:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2024-08-31T07:37:23.465 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2024-08-31T07:37:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2024-08-31T07:37:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-31T07:37:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2024-08-31T07:37:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2024-08-31T07:37:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2024-08-31T07:37:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2024-08-31T07:37:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2024-08-31T07:37:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2024-08-31T07:37:23.466 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2024-08-31T07:37:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "services": { 2024-08-31T07:37:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "dashboard": "https://172.21.15.32:8443/" 2024-08-31T07:37:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.467 INFO:teuthology.orchestra.run.smithi032.stdout: "always_on_modules": { 2024-08-31T07:37:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "octopus": [ 2024-08-31T07:37:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2024-08-31T07:37:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-08-31T07:37:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2024-08-31T07:37:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2024-08-31T07:37:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2024-08-31T07:37:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2024-08-31T07:37:23.468 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2024-08-31T07:37:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2024-08-31T07:37:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2024-08-31T07:37:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2024-08-31T07:37:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "pacific": [ 2024-08-31T07:37:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2024-08-31T07:37:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-08-31T07:37:23.469 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2024-08-31T07:37:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2024-08-31T07:37:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2024-08-31T07:37:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2024-08-31T07:37:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2024-08-31T07:37:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2024-08-31T07:37:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2024-08-31T07:37:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2024-08-31T07:37:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.470 INFO:teuthology.orchestra.run.smithi032.stdout: "quincy": [ 2024-08-31T07:37:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2024-08-31T07:37:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-08-31T07:37:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2024-08-31T07:37:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2024-08-31T07:37:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2024-08-31T07:37:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2024-08-31T07:37:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2024-08-31T07:37:23.471 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2024-08-31T07:37:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2024-08-31T07:37:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2024-08-31T07:37:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "reef": [ 2024-08-31T07:37:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2024-08-31T07:37:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-08-31T07:37:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2024-08-31T07:37:23.472 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2024-08-31T07:37:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2024-08-31T07:37:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2024-08-31T07:37:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2024-08-31T07:37:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2024-08-31T07:37:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2024-08-31T07:37:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2024-08-31T07:37:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-08-31T07:37:23.473 INFO:teuthology.orchestra.run.smithi032.stdout: "squid": [ 2024-08-31T07:37:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2024-08-31T07:37:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-08-31T07:37:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2024-08-31T07:37:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2024-08-31T07:37:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2024-08-31T07:37:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2024-08-31T07:37:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2024-08-31T07:37:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2024-08-31T07:37:23.474 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2024-08-31T07:37:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2024-08-31T07:37:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "last_failure_osd_epoch": 5, 2024-08-31T07:37:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "active_clients": [ 2024-08-31T07:37:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2024-08-31T07:37:23.475 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-08-31T07:37:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-08-31T07:37:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-08-31T07:37:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 505518887 2024-08-31T07:37:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "libcephsqlite", 2024-08-31T07:37:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-08-31T07:37:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-08-31T07:37:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-08-31T07:37:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2201240747 2024-08-31T07:37:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.477 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2024-08-31T07:37:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-08-31T07:37:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-08-31T07:37:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-08-31T07:37:23.478 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1206032550 2024-08-31T07:37:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2024-08-31T07:37:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-08-31T07:37:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-08-31T07:37:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-08-31T07:37:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3021818831 2024-08-31T07:37:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2024-08-31T07:37:23.480 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-08-31T07:37:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-08-31T07:37:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-08-31T07:37:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2064789823 2024-08-31T07:37:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:23.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2024-08-31T07:37:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-08-31T07:37:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-08-31T07:37:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-08-31T07:37:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-08-31T07:37:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3733177002 2024-08-31T07:37:23.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:23.483 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-08-31T07:37:23.483 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:37:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 -- 172.21.15.32:0/3026557643 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f82400b4ca0 msgr2=0x7f82400b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f82400b4ca0 0x7f82400b71b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f824c031440 tx=0x7f824c009560 comp rx=0 tx=0).stop 2024-08-31T07:37:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 -- 172.21.15.32:0/3026557643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f825c10fbe0 msgr2=0x7f825c1a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f825c10fbe0 0x7f825c1a1a00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f825400eb10 tx=0x7f825400c2d0 comp rx=0 tx=0).stop 2024-08-31T07:37:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 -- 172.21.15.32:0/3026557643 shutdown_connections 2024-08-31T07:37:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f82400b4ca0 0x7f82400b71b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f825c10fbe0 0x7f825c1a1a00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f825c075ea0 0x7f825c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 --2- 172.21.15.32:0/3026557643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f825c075550 0x7f825c19c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 -- 172.21.15.32:0/3026557643 >> 172.21.15.32:0/3026557643 conn(0x7f825c0ff0d0 msgr2=0x7f825c1096f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 -- 172.21.15.32:0/3026557643 shutdown_connections 2024-08-31T07:37:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:20.881+0000 7f8260c00640 1 -- 172.21.15.32:0/3026557643 wait complete. 2024-08-31T07:37:23.493 DEBUG:tasks.check_counter:Getting stats from a 2024-08-31T07:37:23.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-31T07:37:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:21 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/571927623' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T07:37:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:21 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3026557643' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-31T07:37:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:22 smithi032 ceph-mon[17252]: pgmap v1144: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.1 MiB/s wr, 82 op/s 2024-08-31T07:37:23.495 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:21 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/571927623' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T07:37:23.495 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:21 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3026557643' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-31T07:37:23.495 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:22 smithi084 ceph-mon[22387]: pgmap v1144: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.1 MiB/s wr, 82 op/s 2024-08-31T07:37:23.496 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:21 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/571927623' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T07:37:23.496 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:21 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3026557643' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-31T07:37:23.496 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:22 smithi073 ceph-mon[21995]: pgmap v1144: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 2.1 MiB/s wr, 82 op/s 2024-08-31T07:37:23.885 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:24.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:24 smithi032 ceph-mon[17252]: pgmap v1145: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.5 MiB/s wr, 65 op/s 2024-08-31T07:37:24.644 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:37:24.645 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:24.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 97090, 2024-08-31T07:37:24.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 84746, 2024-08-31T07:37:24.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 37023510, 2024-08-31T07:37:24.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 134129160, 2024-08-31T07:37:24.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 58, 2024-08-31T07:37:24.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 10, 2024-08-31T07:37:24.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 11.888707550, 2024-08-31T07:37:24.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 5.648929436, 2024-08-31T07:37:24.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 6.327472046, 2024-08-31T07:37:24.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 1.047161114, 2024-08-31T07:37:24.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:24.647 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 84743, 2024-08-31T07:37:24.647 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 7.724688504, 2024-08-31T07:37:24.647 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000091154 2024-08-31T07:37:24.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:24.647 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 151554, 2024-08-31T07:37:24.648 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.042809757, 2024-08-31T07:37:24.648 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000282 2024-08-31T07:37:24.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 1440288, 2024-08-31T07:37:24.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 325056 2024-08-31T07:37:24.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.648 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:24.649 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 110362, 2024-08-31T07:37:24.649 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 107632, 2024-08-31T07:37:24.649 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 48992881, 2024-08-31T07:37:24.649 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 134457001, 2024-08-31T07:37:24.649 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 14, 2024-08-31T07:37:24.649 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 6, 2024-08-31T07:37:24.649 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 13.130630982, 2024-08-31T07:37:24.650 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 6.717065456, 2024-08-31T07:37:24.650 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 7.297374383, 2024-08-31T07:37:24.650 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.589909862, 2024-08-31T07:37:24.650 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:24.650 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 107633, 2024-08-31T07:37:24.650 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 14.674568188, 2024-08-31T07:37:24.650 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000136338 2024-08-31T07:37:24.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:24.651 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 189054, 2024-08-31T07:37:24.651 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.054845448, 2024-08-31T07:37:24.651 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000290 2024-08-31T07:37:24.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 4192, 2024-08-31T07:37:24.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 784544 2024-08-31T07:37:24.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.652 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:24.652 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 199103, 2024-08-31T07:37:24.652 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 283770, 2024-08-31T07:37:24.652 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 65588549, 2024-08-31T07:37:24.652 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 381078536, 2024-08-31T07:37:24.652 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 18, 2024-08-31T07:37:24.653 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 7, 2024-08-31T07:37:24.653 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 45.357644743, 2024-08-31T07:37:24.653 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 34.743242669, 2024-08-31T07:37:24.653 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 15.072940010, 2024-08-31T07:37:24.653 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 1.146664168, 2024-08-31T07:37:24.653 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:24.653 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 283768, 2024-08-31T07:37:24.653 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 39.444068094, 2024-08-31T07:37:24.654 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000139001 2024-08-31T07:37:24.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.654 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:24.654 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 264287, 2024-08-31T07:37:24.654 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.121262478, 2024-08-31T07:37:24.654 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000458 2024-08-31T07:37:24.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.655 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T07:37:24.655 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T07:37:24.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.655 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2024-08-31T07:37:24.655 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 1, 2024-08-31T07:37:24.655 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2024-08-31T07:37:24.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.655 INFO:teuthology.orchestra.run.smithi032.stdout: "finisher-MDSRank": { 2024-08-31T07:37:24.656 INFO:teuthology.orchestra.run.smithi032.stdout: "queue_len": 0, 2024-08-31T07:37:24.656 INFO:teuthology.orchestra.run.smithi032.stdout: "complete_latency": { 2024-08-31T07:37:24.656 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 41258, 2024-08-31T07:37:24.656 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 67.796162857, 2024-08-31T07:37:24.656 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.001643224 2024-08-31T07:37:24.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:24.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.657 INFO:teuthology.orchestra.run.smithi032.stdout: "finisher-PurgeQueue": { 2024-08-31T07:37:24.657 INFO:teuthology.orchestra.run.smithi032.stdout: "queue_len": 0, 2024-08-31T07:37:24.657 INFO:teuthology.orchestra.run.smithi032.stdout: "complete_latency": { 2024-08-31T07:37:24.657 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 26251, 2024-08-31T07:37:24.657 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 5.091621083, 2024-08-31T07:37:24.657 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000193959 2024-08-31T07:37:24.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:24.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.658 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-08-31T07:37:24.658 INFO:teuthology.orchestra.run.smithi032.stdout: "request": 101745, 2024-08-31T07:37:24.658 INFO:teuthology.orchestra.run.smithi032.stdout: "reply": 95646, 2024-08-31T07:37:24.658 INFO:teuthology.orchestra.run.smithi032.stdout: "reply_latency": { 2024-08-31T07:37:24.658 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 95646, 2024-08-31T07:37:24.658 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 738.259185557, 2024-08-31T07:37:24.659 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.007718662 2024-08-31T07:37:24.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.659 INFO:teuthology.orchestra.run.smithi032.stdout: "slow_reply": 0, 2024-08-31T07:37:24.659 INFO:teuthology.orchestra.run.smithi032.stdout: "forward": 6099, 2024-08-31T07:37:24.659 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_fetch_complete": 1, 2024-08-31T07:37:24.659 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_fetch_keys": 0, 2024-08-31T07:37:24.659 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_commit": 83, 2024-08-31T07:37:24.660 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_split": 8, 2024-08-31T07:37:24.660 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_merge": 4, 2024-08-31T07:37:24.660 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes": 1793, 2024-08-31T07:37:24.660 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_top": 396, 2024-08-31T07:37:24.660 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_bottom": 1320, 2024-08-31T07:37:24.660 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_pin_tail": 77, 2024-08-31T07:37:24.660 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_pinned": 1227, 2024-08-31T07:37:24.661 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_expired": 1308, 2024-08-31T07:37:24.661 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_with_caps": 251, 2024-08-31T07:37:24.661 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": 251, 2024-08-31T07:37:24.661 INFO:teuthology.orchestra.run.smithi032.stdout: "subtrees": 34, 2024-08-31T07:37:24.661 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse": 143801, 2024-08-31T07:37:24.661 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_hit": 88972, 2024-08-31T07:37:24.661 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_forward": 6076, 2024-08-31T07:37:24.661 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_discover": 62, 2024-08-31T07:37:24.662 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_dir_fetch": 0, 2024-08-31T07:37:24.662 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_remote_ino": 0, 2024-08-31T07:37:24.662 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_lock": 967, 2024-08-31T07:37:24.662 INFO:teuthology.orchestra.run.smithi032.stdout: "load_cent": 173707, 2024-08-31T07:37:24.662 INFO:teuthology.orchestra.run.smithi032.stdout: "q": 3, 2024-08-31T07:37:24.662 INFO:teuthology.orchestra.run.smithi032.stdout: "exported": 27, 2024-08-31T07:37:24.662 INFO:teuthology.orchestra.run.smithi032.stdout: "exported_inodes": 1009, 2024-08-31T07:37:24.663 INFO:teuthology.orchestra.run.smithi032.stdout: "imported": 18, 2024-08-31T07:37:24.663 INFO:teuthology.orchestra.run.smithi032.stdout: "imported_inodes": 969, 2024-08-31T07:37:24.663 INFO:teuthology.orchestra.run.smithi032.stdout: "openino_dir_fetch": 0, 2024-08-31T07:37:24.663 INFO:teuthology.orchestra.run.smithi032.stdout: "openino_backtrace_fetch": 1661, 2024-08-31T07:37:24.663 INFO:teuthology.orchestra.run.smithi032.stdout: "openino_peer_discover": 0, 2024-08-31T07:37:24.663 INFO:teuthology.orchestra.run.smithi032.stdout: "root_rfiles": 745, 2024-08-31T07:37:24.663 INFO:teuthology.orchestra.run.smithi032.stdout: "root_rbytes": 212490313, 2024-08-31T07:37:24.664 INFO:teuthology.orchestra.run.smithi032.stdout: "root_rsnaps": 6, 2024-08-31T07:37:24.664 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_backtrace_fetch": 1661, 2024-08-31T07:37:24.664 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_set_tag": 0, 2024-08-31T07:37:24.664 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T07:37:24.664 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_inotable_repaired": 0, 2024-08-31T07:37:24.664 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_dir_inodes": 180, 2024-08-31T07:37:24.665 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T07:37:24.665 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_dirfrag_rstats": 180, 2024-08-31T07:37:24.665 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_file_inodes": 1481, 2024-08-31T07:37:24.665 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_inode_file_caps": 9, 2024-08-31T07:37:24.665 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_cap_op_revoke": 12509, 2024-08-31T07:37:24.665 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_cap_op_grant": 66759, 2024-08-31T07:37:24.665 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T07:37:24.666 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_cap_op_flushsnap_ack": 6, 2024-08-31T07:37:24.666 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T07:37:24.666 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_client_caps": 41818, 2024-08-31T07:37:24.666 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_client_caps_dirty": 11703, 2024-08-31T07:37:24.666 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_client_cap_release": 2234, 2024-08-31T07:37:24.666 INFO:teuthology.orchestra.run.smithi032.stdout: "process_request_cap_release": 67551 2024-08-31T07:37:24.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.666 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache": { 2024-08-31T07:37:24.667 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_update": 21, 2024-08-31T07:37:24.667 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_update_receipt": 60, 2024-08-31T07:37:24.667 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_try_discover": 41, 2024-08-31T07:37:24.667 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_send_discover": 97, 2024-08-31T07:37:24.667 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_handle_discover": 71, 2024-08-31T07:37:24.667 INFO:teuthology.orchestra.run.smithi032.stdout: "num_strays": 94, 2024-08-31T07:37:24.668 INFO:teuthology.orchestra.run.smithi032.stdout: "num_strays_delayed": 0, 2024-08-31T07:37:24.668 INFO:teuthology.orchestra.run.smithi032.stdout: "num_strays_enqueuing": 0, 2024-08-31T07:37:24.668 INFO:teuthology.orchestra.run.smithi032.stdout: "strays_created": 14361, 2024-08-31T07:37:24.668 INFO:teuthology.orchestra.run.smithi032.stdout: "strays_enqueued": 14267, 2024-08-31T07:37:24.668 INFO:teuthology.orchestra.run.smithi032.stdout: "strays_reintegrated": 0, 2024-08-31T07:37:24.668 INFO:teuthology.orchestra.run.smithi032.stdout: "strays_migrated": 0, 2024-08-31T07:37:24.668 INFO:teuthology.orchestra.run.smithi032.stdout: "num_recovering_processing": 0, 2024-08-31T07:37:24.669 INFO:teuthology.orchestra.run.smithi032.stdout: "num_recovering_enqueued": 0, 2024-08-31T07:37:24.669 INFO:teuthology.orchestra.run.smithi032.stdout: "num_recovering_prioritized": 0, 2024-08-31T07:37:24.669 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_started": 0, 2024-08-31T07:37:24.669 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_completed": 0, 2024-08-31T07:37:24.669 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_quiesce_path": 19, 2024-08-31T07:37:24.670 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_quiesce_inode": 2002, 2024-08-31T07:37:24.670 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T07:37:24.670 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_exportdir": 45, 2024-08-31T07:37:24.670 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_flush": 0, 2024-08-31T07:37:24.670 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_fragmentdir": 14, 2024-08-31T07:37:24.670 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_fragstats": 0, 2024-08-31T07:37:24.670 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_inodestats": 0, 2024-08-31T07:37:24.670 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:37:24.671 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_succeeded": 0, 2024-08-31T07:37:24.671 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_write_failed": 0 2024-08-31T07:37:24.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.671 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log": { 2024-08-31T07:37:24.671 INFO:teuthology.orchestra.run.smithi032.stdout: "evlrg": 0, 2024-08-31T07:37:24.671 INFO:teuthology.orchestra.run.smithi032.stdout: "evadd": 76816, 2024-08-31T07:37:24.671 INFO:teuthology.orchestra.run.smithi032.stdout: "evex": 0, 2024-08-31T07:37:24.672 INFO:teuthology.orchestra.run.smithi032.stdout: "evtrm": 0, 2024-08-31T07:37:24.672 INFO:teuthology.orchestra.run.smithi032.stdout: "ev": 76816, 2024-08-31T07:37:24.672 INFO:teuthology.orchestra.run.smithi032.stdout: "evexg": 0, 2024-08-31T07:37:24.672 INFO:teuthology.orchestra.run.smithi032.stdout: "evexd": 0, 2024-08-31T07:37:24.672 INFO:teuthology.orchestra.run.smithi032.stdout: "segadd": 117, 2024-08-31T07:37:24.673 INFO:teuthology.orchestra.run.smithi032.stdout: "segex": 0, 2024-08-31T07:37:24.673 INFO:teuthology.orchestra.run.smithi032.stdout: "segtrm": 0, 2024-08-31T07:37:24.673 INFO:teuthology.orchestra.run.smithi032.stdout: "seg": 117, 2024-08-31T07:37:24.673 INFO:teuthology.orchestra.run.smithi032.stdout: "segmjr": 8, 2024-08-31T07:37:24.673 INFO:teuthology.orchestra.run.smithi032.stdout: "segexg": 0, 2024-08-31T07:37:24.673 INFO:teuthology.orchestra.run.smithi032.stdout: "segexd": 0, 2024-08-31T07:37:24.673 INFO:teuthology.orchestra.run.smithi032.stdout: "expos": 4194304, 2024-08-31T07:37:24.674 INFO:teuthology.orchestra.run.smithi032.stdout: "wrpos": 371959662, 2024-08-31T07:37:24.674 INFO:teuthology.orchestra.run.smithi032.stdout: "rdpos": 0, 2024-08-31T07:37:24.674 INFO:teuthology.orchestra.run.smithi032.stdout: "jlat": { 2024-08-31T07:37:24.674 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 22924, 2024-08-31T07:37:24.674 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 185.548440172, 2024-08-31T07:37:24.674 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.008094069 2024-08-31T07:37:24.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.675 INFO:teuthology.orchestra.run.smithi032.stdout: "replayed": 0 2024-08-31T07:37:24.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.675 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_mem": { 2024-08-31T07:37:24.675 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1491, 2024-08-31T07:37:24.675 INFO:teuthology.orchestra.run.smithi032.stdout: "ino+": 16702, 2024-08-31T07:37:24.675 INFO:teuthology.orchestra.run.smithi032.stdout: "ino-": 15211, 2024-08-31T07:37:24.675 INFO:teuthology.orchestra.run.smithi032.stdout: "dir": 201, 2024-08-31T07:37:24.676 INFO:teuthology.orchestra.run.smithi032.stdout: "dir+": 265, 2024-08-31T07:37:24.676 INFO:teuthology.orchestra.run.smithi032.stdout: "dir-": 64, 2024-08-31T07:37:24.676 INFO:teuthology.orchestra.run.smithi032.stdout: "dn": 1833, 2024-08-31T07:37:24.676 INFO:teuthology.orchestra.run.smithi032.stdout: "dn+": 31198, 2024-08-31T07:37:24.676 INFO:teuthology.orchestra.run.smithi032.stdout: "dn-": 29365, 2024-08-31T07:37:24.676 INFO:teuthology.orchestra.run.smithi032.stdout: "cap": 251, 2024-08-31T07:37:24.676 INFO:teuthology.orchestra.run.smithi032.stdout: "cap+": 16316, 2024-08-31T07:37:24.677 INFO:teuthology.orchestra.run.smithi032.stdout: "cap-": 16065, 2024-08-31T07:37:24.677 INFO:teuthology.orchestra.run.smithi032.stdout: "rss": 146268, 2024-08-31T07:37:24.677 INFO:teuthology.orchestra.run.smithi032.stdout: "heap": 357449728 2024-08-31T07:37:24.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.677 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_server": { 2024-08-31T07:37:24.677 INFO:teuthology.orchestra.run.smithi032.stdout: "dispatch_client_request": 140730, 2024-08-31T07:37:24.677 INFO:teuthology.orchestra.run.smithi032.stdout: "dispatch_server_request": 960, 2024-08-31T07:37:24.678 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_client_request": 101745, 2024-08-31T07:37:24.678 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_client_session": 1806, 2024-08-31T07:37:24.678 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_peer_request": 4446, 2024-08-31T07:37:24.678 INFO:teuthology.orchestra.run.smithi032.stdout: "req_create_latency": { 2024-08-31T07:37:24.678 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 14321, 2024-08-31T07:37:24.678 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 288.805274105, 2024-08-31T07:37:24.678 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.020166557 2024-08-31T07:37:24.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.679 INFO:teuthology.orchestra.run.smithi032.stdout: "req_getattr_latency": { 2024-08-31T07:37:24.679 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 1337, 2024-08-31T07:37:24.679 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 17.095147500, 2024-08-31T07:37:24.679 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.012786198 2024-08-31T07:37:24.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.679 INFO:teuthology.orchestra.run.smithi032.stdout: "req_getfilelock_latency": { 2024-08-31T07:37:24.679 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.680 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.680 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.680 INFO:teuthology.orchestra.run.smithi032.stdout: "req_link_latency": { 2024-08-31T07:37:24.680 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.680 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.680 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.681 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookup_latency": { 2024-08-31T07:37:24.681 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 22821, 2024-08-31T07:37:24.681 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 83.892606478, 2024-08-31T07:37:24.681 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.003676114 2024-08-31T07:37:24.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.681 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookuphash_latency": { 2024-08-31T07:37:24.682 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.682 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.682 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.682 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookupino_latency": { 2024-08-31T07:37:24.682 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.682 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.682 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.683 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookupname_latency": { 2024-08-31T07:37:24.683 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.683 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.683 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.684 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookupparent_latency": { 2024-08-31T07:37:24.684 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.684 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.685 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:24 smithi073 ceph-mon[21995]: pgmap v1145: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.5 MiB/s wr, 65 op/s 2024-08-31T07:37:24.685 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.685 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookupsnap_latency": { 2024-08-31T07:37:24.686 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.686 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.686 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.686 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lssnap_latency": { 2024-08-31T07:37:24.686 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.686 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.687 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.687 INFO:teuthology.orchestra.run.smithi032.stdout: "req_mkdir_latency": { 2024-08-31T07:37:24.687 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.687 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.687 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.688 INFO:teuthology.orchestra.run.smithi032.stdout: "req_mknod_latency": { 2024-08-31T07:37:24.688 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.688 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.688 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.688 INFO:teuthology.orchestra.run.smithi032.stdout: "req_mksnap_latency": { 2024-08-31T07:37:24.688 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.689 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.689 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.689 INFO:teuthology.orchestra.run.smithi032.stdout: "req_open_latency": { 2024-08-31T07:37:24.689 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 1, 2024-08-31T07:37:24.689 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000411423, 2024-08-31T07:37:24.689 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000411423 2024-08-31T07:37:24.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.690 INFO:teuthology.orchestra.run.smithi032.stdout: "req_readdir_latency": { 2024-08-31T07:37:24.690 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 39398, 2024-08-31T07:37:24.690 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 212.101444942, 2024-08-31T07:37:24.690 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.005383558 2024-08-31T07:37:24.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.690 INFO:teuthology.orchestra.run.smithi032.stdout: "req_rename_latency": { 2024-08-31T07:37:24.691 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 2985, 2024-08-31T07:37:24.691 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 10.272452011, 2024-08-31T07:37:24.691 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.003441357 2024-08-31T07:37:24.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.691 INFO:teuthology.orchestra.run.smithi032.stdout: "req_renamesnap_latency": { 2024-08-31T07:37:24.691 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.691 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.691 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.692 INFO:teuthology.orchestra.run.smithi032.stdout: "req_snapdiff_latency": { 2024-08-31T07:37:24.692 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.692 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.692 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.692 INFO:teuthology.orchestra.run.smithi032.stdout: "req_rmdir_latency": { 2024-08-31T07:37:24.693 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.693 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.693 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.693 INFO:teuthology.orchestra.run.smithi032.stdout: "req_rmsnap_latency": { 2024-08-31T07:37:24.693 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.693 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.694 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.694 INFO:teuthology.orchestra.run.smithi032.stdout: "req_rmxattr_latency": { 2024-08-31T07:37:24.694 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.694 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.694 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.695 INFO:teuthology.orchestra.run.smithi032.stdout: "req_setattr_latency": { 2024-08-31T07:37:24.695 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 74, 2024-08-31T07:37:24.695 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 1.055058909, 2024-08-31T07:37:24.695 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.014257552 2024-08-31T07:37:24.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.695 INFO:teuthology.orchestra.run.smithi032.stdout: "req_setdirlayout_latency": { 2024-08-31T07:37:24.695 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.696 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.696 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.696 INFO:teuthology.orchestra.run.smithi032.stdout: "req_setfilelock_latency": { 2024-08-31T07:37:24.696 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 348, 2024-08-31T07:37:24.696 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.423193878, 2024-08-31T07:37:24.696 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.001216074 2024-08-31T07:37:24.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.697 INFO:teuthology.orchestra.run.smithi032.stdout: "req_setlayout_latency": { 2024-08-31T07:37:24.697 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.697 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.697 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.697 INFO:teuthology.orchestra.run.smithi032.stdout: "req_setxattr_latency": { 2024-08-31T07:37:24.697 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.698 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.698 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.698 INFO:teuthology.orchestra.run.smithi032.stdout: "req_symlink_latency": { 2024-08-31T07:37:24.698 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.698 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.698 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.699 INFO:teuthology.orchestra.run.smithi032.stdout: "req_unlink_latency": { 2024-08-31T07:37:24.699 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 14361, 2024-08-31T07:37:24.699 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 124.613596311, 2024-08-31T07:37:24.699 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.008677222 2024-08-31T07:37:24.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.699 INFO:teuthology.orchestra.run.smithi032.stdout: "cap_revoke_eviction": 0, 2024-08-31T07:37:24.700 INFO:teuthology.orchestra.run.smithi032.stdout: "cap_acquisition_throttle": 0, 2024-08-31T07:37:24.700 INFO:teuthology.orchestra.run.smithi032.stdout: "req_getvxattr_latency": { 2024-08-31T07:37:24.700 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.700 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.700 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.700 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:24.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.700 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_sessions": { 2024-08-31T07:37:24.701 INFO:teuthology.orchestra.run.smithi032.stdout: "session_count": 2, 2024-08-31T07:37:24.701 INFO:teuthology.orchestra.run.smithi032.stdout: "session_add": 3, 2024-08-31T07:37:24.701 INFO:teuthology.orchestra.run.smithi032.stdout: "session_remove": 1, 2024-08-31T07:37:24.701 INFO:teuthology.orchestra.run.smithi032.stdout: "sessions_open": 2, 2024-08-31T07:37:24.701 INFO:teuthology.orchestra.run.smithi032.stdout: "sessions_stale": 0, 2024-08-31T07:37:24.701 INFO:teuthology.orchestra.run.smithi032.stdout: "total_load": 12563, 2024-08-31T07:37:24.702 INFO:teuthology.orchestra.run.smithi032.stdout: "average_load": 6281, 2024-08-31T07:37:24.702 INFO:teuthology.orchestra.run.smithi032.stdout: "avg_session_uptime": 1424, 2024-08-31T07:37:24.702 INFO:teuthology.orchestra.run.smithi032.stdout: "mdthresh_evicted": 0 2024-08-31T07:37:24.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.702 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2024-08-31T07:37:24.702 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:24.702 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:24.703 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:24.703 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:24.703 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:24.703 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:24.703 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:24.703 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:24.703 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:24.704 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:24.704 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:24.704 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:24.704 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:24.704 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:24.704 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:24.704 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:24.705 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:24.705 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:24.705 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:24.705 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:24.705 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:24.705 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:24.705 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:24.706 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:24.706 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:24.706 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:24.706 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:24.706 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:24.706 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:24.706 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:24.707 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:24.707 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2024-08-31T07:37:24.707 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:24.707 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:24.707 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:24.707 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:24.707 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 354118, 2024-08-31T07:37:24.708 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 788, 2024-08-31T07:37:24.708 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 88, 2024-08-31T07:37:24.708 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 1, 2024-08-31T07:37:24.708 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2024-08-31T07:37:24.708 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2024-08-31T07:37:24.708 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:24.708 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:24.709 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:24.709 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:24.709 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 7968, 2024-08-31T07:37:24.709 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 176, 2024-08-31T07:37:24.709 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:24.709 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:24.709 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:24.709 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2024-08-31T07:37:24.710 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 5648144, 2024-08-31T07:37:24.710 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 54034, 2024-08-31T07:37:24.710 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:24.710 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2024-08-31T07:37:24.710 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:24.710 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2024-08-31T07:37:24.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.711 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter": { 2024-08-31T07:37:24.711 INFO:teuthology.orchestra.run.smithi032.stdout: "op_active": 0, 2024-08-31T07:37:24.711 INFO:teuthology.orchestra.run.smithi032.stdout: "op_laggy": 0, 2024-08-31T07:37:24.711 INFO:teuthology.orchestra.run.smithi032.stdout: "op_send": 41792, 2024-08-31T07:37:24.711 INFO:teuthology.orchestra.run.smithi032.stdout: "op_send_bytes": 371691282, 2024-08-31T07:37:24.711 INFO:teuthology.orchestra.run.smithi032.stdout: "op_resend": 28, 2024-08-31T07:37:24.711 INFO:teuthology.orchestra.run.smithi032.stdout: "op_reply": 41792, 2024-08-31T07:37:24.712 INFO:teuthology.orchestra.run.smithi032.stdout: "op_latency": { 2024-08-31T07:37:24.712 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 41792, 2024-08-31T07:37:24.712 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 285.380429199, 2024-08-31T07:37:24.712 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.006828589 2024-08-31T07:37:24.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.712 INFO:teuthology.orchestra.run.smithi032.stdout: "op_inflight": 0, 2024-08-31T07:37:24.712 INFO:teuthology.orchestra.run.smithi032.stdout: "oplen_avg": { 2024-08-31T07:37:24.712 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 41792, 2024-08-31T07:37:24.713 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 42914 2024-08-31T07:37:24.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.713 INFO:teuthology.orchestra.run.smithi032.stdout: "op": 41792, 2024-08-31T07:37:24.713 INFO:teuthology.orchestra.run.smithi032.stdout: "op_r": 2432, 2024-08-31T07:37:24.713 INFO:teuthology.orchestra.run.smithi032.stdout: "op_w": 39360, 2024-08-31T07:37:24.713 INFO:teuthology.orchestra.run.smithi032.stdout: "op_rmw": 0, 2024-08-31T07:37:24.713 INFO:teuthology.orchestra.run.smithi032.stdout: "op_pg": 0, 2024-08-31T07:37:24.714 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_stat": 0, 2024-08-31T07:37:24.714 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_create": 10, 2024-08-31T07:37:24.714 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_read": 768, 2024-08-31T07:37:24.714 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_write": 23692, 2024-08-31T07:37:24.714 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_writefull": 479, 2024-08-31T07:37:24.714 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_writesame": 0, 2024-08-31T07:37:24.714 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_append": 0, 2024-08-31T07:37:24.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_zero": 0, 2024-08-31T07:37:24.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_truncate": 7, 2024-08-31T07:37:24.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_delete": 14392, 2024-08-31T07:37:24.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_mapext": 0, 2024-08-31T07:37:24.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_sparse_read": 0, 2024-08-31T07:37:24.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_clonerange": 0, 2024-08-31T07:37:24.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_getxattr": 1661, 2024-08-31T07:37:24.715 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_setxattr": 20, 2024-08-31T07:37:24.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_cmpxattr": 0, 2024-08-31T07:37:24.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_rmxattr": 0, 2024-08-31T07:37:24.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_resetxattrs": 0, 2024-08-31T07:37:24.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_call": 0, 2024-08-31T07:37:24.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_watch": 0, 2024-08-31T07:37:24.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_notify": 0, 2024-08-31T07:37:24.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T07:37:24.717 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_pgls": 0, 2024-08-31T07:37:24.717 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_pgls_filter": 0, 2024-08-31T07:37:24.717 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_other": 0, 2024-08-31T07:37:24.717 INFO:teuthology.orchestra.run.smithi032.stdout: "linger_active": 0, 2024-08-31T07:37:24.717 INFO:teuthology.orchestra.run.smithi032.stdout: "linger_send": 0, 2024-08-31T07:37:24.717 INFO:teuthology.orchestra.run.smithi032.stdout: "linger_resend": 0, 2024-08-31T07:37:24.717 INFO:teuthology.orchestra.run.smithi032.stdout: "linger_ping": 0, 2024-08-31T07:37:24.718 INFO:teuthology.orchestra.run.smithi032.stdout: "poolop_active": 0, 2024-08-31T07:37:24.718 INFO:teuthology.orchestra.run.smithi032.stdout: "poolop_send": 0, 2024-08-31T07:37:24.718 INFO:teuthology.orchestra.run.smithi032.stdout: "poolop_resend": 0, 2024-08-31T07:37:24.718 INFO:teuthology.orchestra.run.smithi032.stdout: "poolstat_active": 0, 2024-08-31T07:37:24.718 INFO:teuthology.orchestra.run.smithi032.stdout: "poolstat_send": 0, 2024-08-31T07:37:24.718 INFO:teuthology.orchestra.run.smithi032.stdout: "poolstat_resend": 0, 2024-08-31T07:37:24.718 INFO:teuthology.orchestra.run.smithi032.stdout: "statfs_active": 0, 2024-08-31T07:37:24.719 INFO:teuthology.orchestra.run.smithi032.stdout: "statfs_send": 0, 2024-08-31T07:37:24.719 INFO:teuthology.orchestra.run.smithi032.stdout: "statfs_resend": 0, 2024-08-31T07:37:24.719 INFO:teuthology.orchestra.run.smithi032.stdout: "command_active": 0, 2024-08-31T07:37:24.719 INFO:teuthology.orchestra.run.smithi032.stdout: "command_send": 0, 2024-08-31T07:37:24.719 INFO:teuthology.orchestra.run.smithi032.stdout: "command_resend": 0, 2024-08-31T07:37:24.719 INFO:teuthology.orchestra.run.smithi032.stdout: "map_epoch": 118, 2024-08-31T07:37:24.719 INFO:teuthology.orchestra.run.smithi032.stdout: "map_full": 0, 2024-08-31T07:37:24.720 INFO:teuthology.orchestra.run.smithi032.stdout: "map_inc": 40, 2024-08-31T07:37:24.720 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_sessions": 12, 2024-08-31T07:37:24.720 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_session_open": 12, 2024-08-31T07:37:24.720 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_session_close": 0, 2024-08-31T07:37:24.720 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_laggy": 0, 2024-08-31T07:37:24.720 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_wr": 1426, 2024-08-31T07:37:24.720 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_rd": 4, 2024-08-31T07:37:24.721 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_del": 455 2024-08-31T07:37:24.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.721 INFO:teuthology.orchestra.run.smithi032.stdout: "oft": { 2024-08-31T07:37:24.721 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_total_objs": 1, 2024-08-31T07:37:24.721 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_total_kv_pairs": 0, 2024-08-31T07:37:24.721 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_total_updates": 0, 2024-08-31T07:37:24.721 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_total_removes": 0 2024-08-31T07:37:24.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.722 INFO:teuthology.orchestra.run.smithi032.stdout: "purge_queue": { 2024-08-31T07:37:24.722 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executing_ops": 0, 2024-08-31T07:37:24.722 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executing_ops_high_water": 6, 2024-08-31T07:37:24.722 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executing": 0, 2024-08-31T07:37:24.722 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executing_high_water": 6, 2024-08-31T07:37:24.722 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executed_ops": 14267, 2024-08-31T07:37:24.723 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executed": 14267, 2024-08-31T07:37:24.723 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_item_in_journal": 0 2024-08-31T07:37:24.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.723 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:24.723 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:24.723 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2024-08-31T07:37:24.723 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:24.724 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 406551, 2024-08-31T07:37:24.724 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 117363203, 2024-08-31T07:37:24.724 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:24.724 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 406551, 2024-08-31T07:37:24.724 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-08-31T07:37:24.724 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-08-31T07:37:24.724 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 406551, 2024-08-31T07:37:24.725 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 117363203, 2024-08-31T07:37:24.725 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:24.725 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.725 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.725 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:24.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.726 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-objecter_bytes": { 2024-08-31T07:37:24.726 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:24.726 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2024-08-31T07:37:24.726 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:24.726 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 0, 2024-08-31T07:37:24.726 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 0, 2024-08-31T07:37:24.726 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:24.727 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:24.727 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 41792, 2024-08-31T07:37:24.727 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 373134842, 2024-08-31T07:37:24.727 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 41792, 2024-08-31T07:37:24.727 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 373134842, 2024-08-31T07:37:24.727 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:24.727 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.727 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.728 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.728 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:24.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.728 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-objecter_ops": { 2024-08-31T07:37:24.728 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:24.728 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 1024, 2024-08-31T07:37:24.728 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:24.729 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 0, 2024-08-31T07:37:24.729 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 0, 2024-08-31T07:37:24.729 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:24.729 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:24.729 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 41792, 2024-08-31T07:37:24.729 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 41792, 2024-08-31T07:37:24.729 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 41792, 2024-08-31T07:37:24.730 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 41792, 2024-08-31T07:37:24.730 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:24.730 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.730 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.730 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.730 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:24.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.731 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-write_buf_throttle": { 2024-08-31T07:37:24.731 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:24.731 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 3758096384, 2024-08-31T07:37:24.731 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:24.731 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 14267, 2024-08-31T07:37:24.731 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 1443639, 2024-08-31T07:37:24.731 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:24.731 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 14267, 2024-08-31T07:37:24.732 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-08-31T07:37:24.732 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-08-31T07:37:24.732 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 768, 2024-08-31T07:37:24.732 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 1443639, 2024-08-31T07:37:24.732 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:24.732 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.732 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.733 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:24.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.733 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-write_buf_throttle-0x55ab921a89c0": { 2024-08-31T07:37:24.733 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 18324, 2024-08-31T07:37:24.733 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 3758096384, 2024-08-31T07:37:24.734 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:24.734 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 76816, 2024-08-31T07:37:24.734 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 367765358, 2024-08-31T07:37:24.734 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:24.734 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 76816, 2024-08-31T07:37:24.734 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-08-31T07:37:24.734 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-08-31T07:37:24.734 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 22924, 2024-08-31T07:37:24.735 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 367747034, 2024-08-31T07:37:24.735 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:24.735 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:24.735 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:24.735 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:24.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:24.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:24.736 INFO:teuthology.orchestra.run.smithi032.stdout: "trackedop": { 2024-08-31T07:37:24.736 INFO:teuthology.orchestra.run.smithi032.stdout: "slow_ops_count": 2297 2024-08-31T07:37:24.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:24.736 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:37:24.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:24 smithi084 ceph-mon[22387]: pgmap v1145: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.5 MiB/s wr, 65 op/s 2024-08-31T07:37:25.754 INFO:tasks.check_counter:Daemon mds.a mds.exported=27 2024-08-31T07:37:25.754 INFO:tasks.check_counter:Daemon mds.a mds.imported=18 2024-08-31T07:37:25.754 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-08-31T07:37:25.755 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-08-31T07:37:25.755 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T07:37:25.755 DEBUG:tasks.check_counter:Getting stats from d 2024-08-31T07:37:25.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-31T07:37:26.134 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:26.525 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:26 smithi032 ceph-mon[17252]: pgmap v1146: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 44 op/s 2024-08-31T07:37:26.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:26 smithi073 ceph-mon[21995]: pgmap v1146: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 44 op/s 2024-08-31T07:37:26.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:26 smithi084 ceph-mon[22387]: pgmap v1146: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 44 op/s 2024-08-31T07:37:26.964 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:37:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 0, 2024-08-31T07:37:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 2, 2024-08-31T07:37:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 0, 2024-08-31T07:37:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 0, 2024-08-31T07:37:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2024-08-31T07:37:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2024-08-31T07:37:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.003526931, 2024-08-31T07:37:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000000000, 2024-08-31T07:37:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000798143, 2024-08-31T07:37:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T07:37:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T07:37:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T07:37:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 454, 2024-08-31T07:37:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 449, 2024-08-31T07:37:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 895008, 2024-08-31T07:37:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 278032, 2024-08-31T07:37:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2024-08-31T07:37:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2024-08-31T07:37:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.093491897, 2024-08-31T07:37:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.051698112, 2024-08-31T07:37:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.051413194, 2024-08-31T07:37:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T07:37:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 448, 2024-08-31T07:37:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.045689174, 2024-08-31T07:37:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000101984 2024-08-31T07:37:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 895008, 2024-08-31T07:37:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 278032 2024-08-31T07:37:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 6, 2024-08-31T07:37:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 345, 2024-08-31T07:37:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 3264, 2024-08-31T07:37:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 220944, 2024-08-31T07:37:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2024-08-31T07:37:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 2, 2024-08-31T07:37:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.046209935, 2024-08-31T07:37:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.036951363, 2024-08-31T07:37:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.005067548, 2024-08-31T07:37:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T07:37:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 344, 2024-08-31T07:37:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.029327559, 2024-08-31T07:37:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000085254 2024-08-31T07:37:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-08-31T07:37:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 220944 2024-08-31T07:37:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2024-08-31T07:37:26.975 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2024-08-31T07:37:26.975 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2024-08-31T07:37:26.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.975 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2024-08-31T07:37:26.975 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:26.975 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:26.975 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:26.975 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:26.976 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:26.976 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:26.976 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:26.976 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:26.976 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:26.976 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:26.976 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:26.977 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:26.977 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:26.977 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:26.977 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:26.977 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:26.977 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:26.977 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:26.978 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:26.978 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:26.978 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:26.978 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:26.978 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:26.978 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:26.978 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:26.978 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:26.979 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:26.979 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:26.979 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:26.979 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:26.979 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:26.979 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2024-08-31T07:37:26.979 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:26.979 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:26.980 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:26.980 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:26.980 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 150766, 2024-08-31T07:37:26.980 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 31, 2024-08-31T07:37:26.980 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2024-08-31T07:37:26.980 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2024-08-31T07:37:26.980 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2024-08-31T07:37:26.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2024-08-31T07:37:26.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:26.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:26.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:26.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:26.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2024-08-31T07:37:26.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2024-08-31T07:37:26.982 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:26.982 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:26.982 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:26.982 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2024-08-31T07:37:26.982 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2024-08-31T07:37:26.982 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2024-08-31T07:37:26.982 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:26.982 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2024-08-31T07:37:26.982 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:26.983 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2024-08-31T07:37:26.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:26.983 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:26.983 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:26.983 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2024-08-31T07:37:26.983 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:26.983 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 456, 2024-08-31T07:37:26.983 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 830791, 2024-08-31T07:37:26.984 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:26.984 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 456, 2024-08-31T07:37:26.984 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-08-31T07:37:26.984 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-08-31T07:37:26.984 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 456, 2024-08-31T07:37:26.984 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 830791, 2024-08-31T07:37:26.984 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:26.984 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:26.985 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:26.985 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:26.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:26.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:26.985 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:37:27.731 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-31T07:37:27.732 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-31T07:37:27.732 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-08-31T07:37:27.732 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-08-31T07:37:27.732 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-08-31T07:37:27.732 DEBUG:tasks.check_counter:Getting stats from g 2024-08-31T07:37:27.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-31T07:37:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:28.504 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:28 smithi032 ceph-mon[17252]: pgmap v1147: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 624 KiB/s wr, 42 op/s 2024-08-31T07:37:28.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:28 smithi073 ceph-mon[21995]: pgmap v1147: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 624 KiB/s wr, 42 op/s 2024-08-31T07:37:28.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:28 smithi084 ceph-mon[22387]: pgmap v1147: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 624 KiB/s wr, 42 op/s 2024-08-31T07:37:29.647 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:37:29.647 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:29.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 0, 2024-08-31T07:37:29.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 2, 2024-08-31T07:37:29.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 0, 2024-08-31T07:37:29.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 0, 2024-08-31T07:37:29.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2024-08-31T07:37:29.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2024-08-31T07:37:29.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.003819261, 2024-08-31T07:37:29.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000000000, 2024-08-31T07:37:29.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000493366, 2024-08-31T07:37:29.649 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T07:37:29.649 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:29.649 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:29.649 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:29.649 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:29.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.649 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:29.649 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:29.650 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:29.650 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:29.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.650 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T07:37:29.650 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T07:37:29.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.650 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:29.650 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 2, 2024-08-31T07:37:29.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 341, 2024-08-31T07:37:29.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 320, 2024-08-31T07:37:29.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 220144, 2024-08-31T07:37:29.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2024-08-31T07:37:29.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 2, 2024-08-31T07:37:29.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.040745508, 2024-08-31T07:37:29.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.034285605, 2024-08-31T07:37:29.651 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.002158196, 2024-08-31T07:37:29.652 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T07:37:29.652 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:29.652 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 339, 2024-08-31T07:37:29.652 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.023510570, 2024-08-31T07:37:29.652 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000069352 2024-08-31T07:37:29.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.652 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:29.652 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:29.653 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:29.653 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:29.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.653 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-31T07:37:29.653 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 220144 2024-08-31T07:37:29.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.653 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:29.653 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 455, 2024-08-31T07:37:29.654 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 448, 2024-08-31T07:37:29.654 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 897136, 2024-08-31T07:37:29.654 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 276560, 2024-08-31T07:37:29.654 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2024-08-31T07:37:29.654 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2024-08-31T07:37:29.654 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.091839136, 2024-08-31T07:37:29.654 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.049986612, 2024-08-31T07:37:29.654 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.052215055, 2024-08-31T07:37:29.654 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T07:37:29.655 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:29.655 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 448, 2024-08-31T07:37:29.655 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.037396465, 2024-08-31T07:37:29.655 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000083474 2024-08-31T07:37:29.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.655 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:29.656 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:29.656 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:29.656 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:29.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.656 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 897136, 2024-08-31T07:37:29.656 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 276560 2024-08-31T07:37:29.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.656 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2024-08-31T07:37:29.657 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2024-08-31T07:37:29.657 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2024-08-31T07:37:29.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.657 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2024-08-31T07:37:29.657 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:29.657 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:29.657 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:29.657 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:29.657 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:29.658 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:29.658 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:29.658 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:29.658 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:29.658 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:29.658 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:29.658 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:29.658 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:29.659 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:29.659 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:29.659 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:29.659 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:29.659 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:29.659 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:29.659 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:29.659 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:29.660 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:29.660 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:29.660 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:29.660 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:29.660 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:29.660 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:29.660 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:29.660 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:29.661 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:29.661 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:29.661 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2024-08-31T07:37:29.661 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:29.661 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:29.661 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:29.661 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:29.661 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 148704, 2024-08-31T07:37:29.662 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 29, 2024-08-31T07:37:29.662 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2024-08-31T07:37:29.662 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2024-08-31T07:37:29.662 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2024-08-31T07:37:29.662 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2024-08-31T07:37:29.662 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:29.662 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:29.662 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:29.662 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:29.663 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2024-08-31T07:37:29.663 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2024-08-31T07:37:29.663 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:29.663 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:29.663 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:29.663 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2024-08-31T07:37:29.663 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2024-08-31T07:37:29.663 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2024-08-31T07:37:29.663 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:29.664 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2024-08-31T07:37:29.664 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:29.664 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2024-08-31T07:37:29.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:29.664 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:29.664 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:29.664 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2024-08-31T07:37:29.665 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:29.665 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 453, 2024-08-31T07:37:29.665 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 830401, 2024-08-31T07:37:29.665 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:29.665 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 453, 2024-08-31T07:37:29.665 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-08-31T07:37:29.665 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-08-31T07:37:29.665 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 453, 2024-08-31T07:37:29.666 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 830401, 2024-08-31T07:37:29.666 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:29.666 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:29.666 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:29.666 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:29.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:29.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:29.666 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:37:30.662 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:30 smithi032 ceph-mon[17252]: pgmap v1148: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 553 KiB/s wr, 35 op/s 2024-08-31T07:37:30.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:30 smithi073 ceph-mon[21995]: pgmap v1148: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 553 KiB/s wr, 35 op/s 2024-08-31T07:37:30.699 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-08-31T07:37:30.699 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-08-31T07:37:30.700 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-08-31T07:37:30.700 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-08-31T07:37:30.700 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-08-31T07:37:30.700 DEBUG:tasks.check_counter:Getting stats from j 2024-08-31T07:37:30.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-31T07:37:30.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:30 smithi084 ceph-mon[22387]: pgmap v1148: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 553 KiB/s wr, 35 op/s 2024-08-31T07:37:31.086 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:32.091 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-08-31T07:37:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 1774, 2024-08-31T07:37:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 1774, 2024-08-31T07:37:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 148861427, 2024-08-31T07:37:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 563372, 2024-08-31T07:37:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 9, 2024-08-31T07:37:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 4, 2024-08-31T07:37:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.504333739, 2024-08-31T07:37:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.094800588, 2024-08-31T07:37:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.327589439, 2024-08-31T07:37:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.061008021, 2024-08-31T07:37:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 1773, 2024-08-31T07:37:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.112025367, 2024-08-31T07:37:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000063184 2024-08-31T07:37:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-31T07:37:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 448 2024-08-31T07:37:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 1518, 2024-08-31T07:37:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 1861, 2024-08-31T07:37:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 105258307, 2024-08-31T07:37:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 1174683, 2024-08-31T07:37:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 7, 2024-08-31T07:37:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 6, 2024-08-31T07:37:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.415531443, 2024-08-31T07:37:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.128091758, 2024-08-31T07:37:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.216006456, 2024-08-31T07:37:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.052534281, 2024-08-31T07:37:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 1859, 2024-08-31T07:37:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.107578679, 2024-08-31T07:37:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000057869 2024-08-31T07:37:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-31T07:37:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 692272 2024-08-31T07:37:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 3216, 2024-08-31T07:37:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 3142, 2024-08-31T07:37:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 123884025, 2024-08-31T07:37:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 1128934, 2024-08-31T07:37:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 5, 2024-08-31T07:37:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 4, 2024-08-31T07:37:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.682036237, 2024-08-31T07:37:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.233116458, 2024-08-31T07:37:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.350660399, 2024-08-31T07:37:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.088680216, 2024-08-31T07:37:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 3141, 2024-08-31T07:37:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.202112685, 2024-08-31T07:37:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000064346 2024-08-31T07:37:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 1404064, 2024-08-31T07:37:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 287824 2024-08-31T07:37:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2024-08-31T07:37:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 1, 2024-08-31T07:37:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2024-08-31T07:37:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "finisher-MDSRank": { 2024-08-31T07:37:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "queue_len": 0, 2024-08-31T07:37:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "complete_latency": { 2024-08-31T07:37:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 9818, 2024-08-31T07:37:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.302010345, 2024-08-31T07:37:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000030760 2024-08-31T07:37:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "finisher-PurgeQueue": { 2024-08-31T07:37:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "queue_len": 0, 2024-08-31T07:37:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "complete_latency": { 2024-08-31T07:37:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-08-31T07:37:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "request": 0, 2024-08-31T07:37:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "reply": 0, 2024-08-31T07:37:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "reply_latency": { 2024-08-31T07:37:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "slow_reply": 0, 2024-08-31T07:37:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "forward": 0, 2024-08-31T07:37:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_fetch_complete": 0, 2024-08-31T07:37:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_fetch_keys": 0, 2024-08-31T07:37:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_commit": 0, 2024-08-31T07:37:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_split": 0, 2024-08-31T07:37:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_merge": 0, 2024-08-31T07:37:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes": 16768, 2024-08-31T07:37:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_top": 47, 2024-08-31T07:37:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_bottom": 16721, 2024-08-31T07:37:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_pin_tail": 0, 2024-08-31T07:37:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_pinned": 16701, 2024-08-31T07:37:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_expired": 0, 2024-08-31T07:37:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "inodes_with_caps": 0, 2024-08-31T07:37:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": 0, 2024-08-31T07:37:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "subtrees": 13, 2024-08-31T07:37:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse": 0, 2024-08-31T07:37:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_hit": 0, 2024-08-31T07:37:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_forward": 0, 2024-08-31T07:37:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_discover": 0, 2024-08-31T07:37:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_dir_fetch": 0, 2024-08-31T07:37:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_remote_ino": 0, 2024-08-31T07:37:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "traverse_lock": 0, 2024-08-31T07:37:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "load_cent": 0, 2024-08-31T07:37:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "q": 0, 2024-08-31T07:37:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "exported": 0, 2024-08-31T07:37:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "exported_inodes": 0, 2024-08-31T07:37:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "imported": 0, 2024-08-31T07:37:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "imported_inodes": 0, 2024-08-31T07:37:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "openino_dir_fetch": 0, 2024-08-31T07:37:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "openino_backtrace_fetch": 0, 2024-08-31T07:37:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "openino_peer_discover": 0, 2024-08-31T07:37:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "root_rfiles": 0, 2024-08-31T07:37:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "root_rbytes": 0, 2024-08-31T07:37:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "root_rsnaps": 0, 2024-08-31T07:37:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_backtrace_fetch": 0, 2024-08-31T07:37:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_set_tag": 0, 2024-08-31T07:37:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T07:37:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_inotable_repaired": 0, 2024-08-31T07:37:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_dir_inodes": 0, 2024-08-31T07:37:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T07:37:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_dirfrag_rstats": 0, 2024-08-31T07:37:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_file_inodes": 0, 2024-08-31T07:37:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_inode_file_caps": 0, 2024-08-31T07:37:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_cap_op_revoke": 0, 2024-08-31T07:37:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_cap_op_grant": 0, 2024-08-31T07:37:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T07:37:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-31T07:37:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T07:37:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_client_caps": 0, 2024-08-31T07:37:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_client_caps_dirty": 0, 2024-08-31T07:37:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_client_cap_release": 0, 2024-08-31T07:37:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "process_request_cap_release": 0 2024-08-31T07:37:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache": { 2024-08-31T07:37:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_update": 0, 2024-08-31T07:37:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_update_receipt": 0, 2024-08-31T07:37:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_try_discover": 0, 2024-08-31T07:37:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_send_discover": 0, 2024-08-31T07:37:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "dir_handle_discover": 0, 2024-08-31T07:37:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "num_strays": 0, 2024-08-31T07:37:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "num_strays_delayed": 0, 2024-08-31T07:37:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "num_strays_enqueuing": 0, 2024-08-31T07:37:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "strays_created": 0, 2024-08-31T07:37:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "strays_enqueued": 0, 2024-08-31T07:37:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "strays_reintegrated": 0, 2024-08-31T07:37:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "strays_migrated": 0, 2024-08-31T07:37:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "num_recovering_processing": 0, 2024-08-31T07:37:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "num_recovering_enqueued": 0, 2024-08-31T07:37:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "num_recovering_prioritized": 0, 2024-08-31T07:37:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_started": 0, 2024-08-31T07:37:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_completed": 0, 2024-08-31T07:37:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_quiesce_path": 0, 2024-08-31T07:37:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_quiesce_inode": 0, 2024-08-31T07:37:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T07:37:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_exportdir": 0, 2024-08-31T07:37:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_flush": 0, 2024-08-31T07:37:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_fragmentdir": 0, 2024-08-31T07:37:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_fragstats": 0, 2024-08-31T07:37:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "ireq_inodestats": 0, 2024-08-31T07:37:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_started": 0, 2024-08-31T07:37:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_succeeded": 0, 2024-08-31T07:37:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "uninline_write_failed": 0 2024-08-31T07:37:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log": { 2024-08-31T07:37:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "evlrg": 0, 2024-08-31T07:37:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "evadd": 0, 2024-08-31T07:37:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "evex": 0, 2024-08-31T07:37:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "evtrm": 0, 2024-08-31T07:37:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "ev": 84057, 2024-08-31T07:37:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "evexg": 0, 2024-08-31T07:37:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "evexd": 0, 2024-08-31T07:37:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "segadd": 0, 2024-08-31T07:37:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "segex": 0, 2024-08-31T07:37:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "segtrm": 0, 2024-08-31T07:37:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "seg": 125, 2024-08-31T07:37:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "segmjr": 8, 2024-08-31T07:37:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "segexg": 0, 2024-08-31T07:37:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "segexd": 0, 2024-08-31T07:37:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "expos": 4194304, 2024-08-31T07:37:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "wrpos": 379062563, 2024-08-31T07:37:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "rdpos": 379053086, 2024-08-31T07:37:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "jlat": { 2024-08-31T07:37:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "replayed": 84057 2024-08-31T07:37:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_mem": { 2024-08-31T07:37:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1646, 2024-08-31T07:37:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "ino+": 17131, 2024-08-31T07:37:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "ino-": 15485, 2024-08-31T07:37:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "dir": 78, 2024-08-31T07:37:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "dir+": 126, 2024-08-31T07:37:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "dir-": 48, 2024-08-31T07:37:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "dn": 16780, 2024-08-31T07:37:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "dn+": 17299, 2024-08-31T07:37:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "dn-": 519, 2024-08-31T07:37:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cap": 0, 2024-08-31T07:37:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cap+": 0, 2024-08-31T07:37:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "cap-": 0, 2024-08-31T07:37:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "rss": 139416, 2024-08-31T07:37:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "heap": 352673792 2024-08-31T07:37:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_server": { 2024-08-31T07:37:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "dispatch_client_request": 0, 2024-08-31T07:37:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "dispatch_server_request": 0, 2024-08-31T07:37:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_client_request": 0, 2024-08-31T07:37:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_client_session": 0, 2024-08-31T07:37:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "handle_peer_request": 0, 2024-08-31T07:37:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "req_create_latency": { 2024-08-31T07:37:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "req_getattr_latency": { 2024-08-31T07:37:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "req_getfilelock_latency": { 2024-08-31T07:37:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "req_link_latency": { 2024-08-31T07:37:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookup_latency": { 2024-08-31T07:37:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookuphash_latency": { 2024-08-31T07:37:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookupino_latency": { 2024-08-31T07:37:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookupname_latency": { 2024-08-31T07:37:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookupparent_latency": { 2024-08-31T07:37:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lookupsnap_latency": { 2024-08-31T07:37:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "req_lssnap_latency": { 2024-08-31T07:37:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "req_mkdir_latency": { 2024-08-31T07:37:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "req_mknod_latency": { 2024-08-31T07:37:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "req_mksnap_latency": { 2024-08-31T07:37:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "req_open_latency": { 2024-08-31T07:37:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "req_readdir_latency": { 2024-08-31T07:37:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "req_rename_latency": { 2024-08-31T07:37:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "req_renamesnap_latency": { 2024-08-31T07:37:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "req_snapdiff_latency": { 2024-08-31T07:37:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "req_rmdir_latency": { 2024-08-31T07:37:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "req_rmsnap_latency": { 2024-08-31T07:37:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "req_rmxattr_latency": { 2024-08-31T07:37:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "req_setattr_latency": { 2024-08-31T07:37:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "req_setdirlayout_latency": { 2024-08-31T07:37:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "req_setfilelock_latency": { 2024-08-31T07:37:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "req_setlayout_latency": { 2024-08-31T07:37:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "req_setxattr_latency": { 2024-08-31T07:37:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "req_symlink_latency": { 2024-08-31T07:37:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "req_unlink_latency": { 2024-08-31T07:37:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "cap_revoke_eviction": 0, 2024-08-31T07:37:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "cap_acquisition_throttle": 0, 2024-08-31T07:37:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "req_getvxattr_latency": { 2024-08-31T07:37:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_sessions": { 2024-08-31T07:37:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "session_count": 1, 2024-08-31T07:37:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "session_add": 2, 2024-08-31T07:37:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "session_remove": 1, 2024-08-31T07:37:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "sessions_open": 1, 2024-08-31T07:37:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "sessions_stale": 0, 2024-08-31T07:37:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "total_load": 0, 2024-08-31T07:37:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "average_load": 0, 2024-08-31T07:37:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "avg_session_uptime": 2574, 2024-08-31T07:37:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mdthresh_evicted": 0 2024-08-31T07:37:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2024-08-31T07:37:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2024-08-31T07:37:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 312547, 2024-08-31T07:37:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 1962, 2024-08-31T07:37:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2024-08-31T07:37:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2024-08-31T07:37:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2024-08-31T07:37:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2024-08-31T07:37:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 7968, 2024-08-31T07:37:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 176, 2024-08-31T07:37:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2024-08-31T07:37:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 15643350, 2024-08-31T07:37:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 117617, 2024-08-31T07:37:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2024-08-31T07:37:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2024-08-31T07:37:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter": { 2024-08-31T07:37:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "op_active": 0, 2024-08-31T07:37:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "op_laggy": 0, 2024-08-31T07:37:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "op_send": 5932, 2024-08-31T07:37:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "op_send_bytes": 21, 2024-08-31T07:37:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "op_resend": 3, 2024-08-31T07:37:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "op_reply": 5932, 2024-08-31T07:37:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "op_latency": { 2024-08-31T07:37:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 5932, 2024-08-31T07:37:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 9.916805596, 2024-08-31T07:37:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.001671747 2024-08-31T07:37:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "op_inflight": 0, 2024-08-31T07:37:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "oplen_avg": { 2024-08-31T07:37:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 5932, 2024-08-31T07:37:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 5933 2024-08-31T07:37:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "op": 5932, 2024-08-31T07:37:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "op_r": 5932, 2024-08-31T07:37:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "op_w": 0, 2024-08-31T07:37:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "op_rmw": 0, 2024-08-31T07:37:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "op_pg": 0, 2024-08-31T07:37:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_stat": 3456, 2024-08-31T07:37:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_create": 0, 2024-08-31T07:37:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_read": 2474, 2024-08-31T07:37:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_write": 0, 2024-08-31T07:37:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_writefull": 0, 2024-08-31T07:37:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_writesame": 0, 2024-08-31T07:37:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_append": 0, 2024-08-31T07:37:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_zero": 0, 2024-08-31T07:37:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_truncate": 0, 2024-08-31T07:37:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_delete": 0, 2024-08-31T07:37:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_mapext": 0, 2024-08-31T07:37:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_sparse_read": 0, 2024-08-31T07:37:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_clonerange": 0, 2024-08-31T07:37:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_getxattr": 1, 2024-08-31T07:37:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_setxattr": 0, 2024-08-31T07:37:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_cmpxattr": 0, 2024-08-31T07:37:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_rmxattr": 0, 2024-08-31T07:37:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_resetxattrs": 0, 2024-08-31T07:37:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_call": 0, 2024-08-31T07:37:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_watch": 0, 2024-08-31T07:37:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_notify": 0, 2024-08-31T07:37:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T07:37:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_pgls": 0, 2024-08-31T07:37:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_pgls_filter": 0, 2024-08-31T07:37:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "osdop_other": 0, 2024-08-31T07:37:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "linger_active": 0, 2024-08-31T07:37:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "linger_send": 0, 2024-08-31T07:37:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "linger_resend": 0, 2024-08-31T07:37:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "linger_ping": 0, 2024-08-31T07:37:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "poolop_active": 0, 2024-08-31T07:37:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "poolop_send": 0, 2024-08-31T07:37:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "poolop_resend": 0, 2024-08-31T07:37:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "poolstat_active": 0, 2024-08-31T07:37:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "poolstat_send": 0, 2024-08-31T07:37:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "poolstat_resend": 0, 2024-08-31T07:37:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "statfs_active": 0, 2024-08-31T07:37:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "statfs_send": 0, 2024-08-31T07:37:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "statfs_resend": 0, 2024-08-31T07:37:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "command_active": 0, 2024-08-31T07:37:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "command_send": 0, 2024-08-31T07:37:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "command_resend": 0, 2024-08-31T07:37:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "map_epoch": 118, 2024-08-31T07:37:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "map_full": 0, 2024-08-31T07:37:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "map_inc": 40, 2024-08-31T07:37:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_sessions": 11, 2024-08-31T07:37:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_session_open": 12, 2024-08-31T07:37:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_session_close": 1, 2024-08-31T07:37:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_laggy": 0, 2024-08-31T07:37:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_wr": 0, 2024-08-31T07:37:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_rd": 2, 2024-08-31T07:37:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_del": 0 2024-08-31T07:37:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "oft": { 2024-08-31T07:37:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_total_objs": 0, 2024-08-31T07:37:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_total_kv_pairs": 0, 2024-08-31T07:37:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_total_updates": 0, 2024-08-31T07:37:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "omap_total_removes": 0 2024-08-31T07:37:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "purge_queue": { 2024-08-31T07:37:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executing_ops": 0, 2024-08-31T07:37:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executing_ops_high_water": 0, 2024-08-31T07:37:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executing": 0, 2024-08-31T07:37:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executing_high_water": 0, 2024-08-31T07:37:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executed_ops": 0, 2024-08-31T07:37:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_executed": 0, 2024-08-31T07:37:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "pq_item_in_journal": 0 2024-08-31T07:37:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2024-08-31T07:37:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 6504, 2024-08-31T07:37:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 377386553, 2024-08-31T07:37:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 6504, 2024-08-31T07:37:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-08-31T07:37:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-08-31T07:37:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 6504, 2024-08-31T07:37:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 377386553, 2024-08-31T07:37:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-objecter_bytes": { 2024-08-31T07:37:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2024-08-31T07:37:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 0, 2024-08-31T07:37:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 0, 2024-08-31T07:37:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 5932, 2024-08-31T07:37:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 374953199, 2024-08-31T07:37:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 5932, 2024-08-31T07:37:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 374953199, 2024-08-31T07:37:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-objecter_ops": { 2024-08-31T07:37:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 1024, 2024-08-31T07:37:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 0, 2024-08-31T07:37:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 0, 2024-08-31T07:37:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 5932, 2024-08-31T07:37:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 5932, 2024-08-31T07:37:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 5932, 2024-08-31T07:37:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 5932, 2024-08-31T07:37:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-write_buf_throttle": { 2024-08-31T07:37:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 3758096384, 2024-08-31T07:37:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 0, 2024-08-31T07:37:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 0, 2024-08-31T07:37:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-08-31T07:37:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-08-31T07:37:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 0, 2024-08-31T07:37:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 0, 2024-08-31T07:37:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-write_buf_throttle-0x557af5986750": { 2024-08-31T07:37:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2024-08-31T07:37:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 3758096384, 2024-08-31T07:37:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2024-08-31T07:37:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 0, 2024-08-31T07:37:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 0, 2024-08-31T07:37:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2024-08-31T07:37:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2024-08-31T07:37:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 0, 2024-08-31T07:37:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 0, 2024-08-31T07:37:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2024-08-31T07:37:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2024-08-31T07:37:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2024-08-31T07:37:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2024-08-31T07:37:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-08-31T07:37:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "trackedop": { 2024-08-31T07:37:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "slow_ops_count": 0 2024-08-31T07:37:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-08-31T07:37:32.183 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-08-31T07:37:32.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:32 smithi073 ceph-mon[21995]: pgmap v1149: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 632 KiB/s wr, 37 op/s 2024-08-31T07:37:32.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:32 smithi084 ceph-mon[22387]: pgmap v1149: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 632 KiB/s wr, 37 op/s 2024-08-31T07:37:32.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:32 smithi032 ceph-mon[17252]: pgmap v1149: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 632 KiB/s wr, 37 op/s 2024-08-31T07:37:33.004 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-08-31T07:37:33.004 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-08-31T07:37:33.004 INFO:tasks.check_counter:Daemon mds.j mds.root_rsnaps=0 2024-08-31T07:37:33.005 INFO:tasks.check_counter:Daemon mds.j mds_server.req_mksnap_latency.avgcount=0 2024-08-31T07:37:33.005 INFO:tasks.check_counter:Daemon mds.j mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T07:37:33.005 DEBUG:tasks.check_counter:Getting stats from b 2024-08-31T07:37:33.005 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-31T07:37:33.426 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:37:34.282 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-08-31T07:37:34.282 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:34.282 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 339021, 2024-08-31T07:37:34.283 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 434172, 2024-08-31T07:37:34.283 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 104226540, 2024-08-31T07:37:34.283 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 529079515, 2024-08-31T07:37:34.283 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 296, 2024-08-31T07:37:34.283 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-08-31T07:37:34.283 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 24.488502404, 2024-08-31T07:37:34.283 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 13.938566938, 2024-08-31T07:37:34.283 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 12.375845458, 2024-08-31T07:37:34.284 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 1.016530855, 2024-08-31T07:37:34.284 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:34.284 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 434171, 2024-08-31T07:37:34.284 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 16.147325759, 2024-08-31T07:37:34.284 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000037191 2024-08-31T07:37:34.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.284 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:34.284 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 441482, 2024-08-31T07:37:34.285 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.112685680, 2024-08-31T07:37:34.285 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000255 2024-08-31T07:37:34.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.285 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 63888, 2024-08-31T07:37:34.285 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 30192 2024-08-31T07:37:34.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.286 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:34.286 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 108448, 2024-08-31T07:37:34.286 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 114402, 2024-08-31T07:37:34.286 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 38394017, 2024-08-31T07:37:34.286 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 308155524, 2024-08-31T07:37:34.286 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 13, 2024-08-31T07:37:34.286 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 8, 2024-08-31T07:37:34.286 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 9.102504613, 2024-08-31T07:37:34.286 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 4.220650312, 2024-08-31T07:37:34.287 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 3.987251497, 2024-08-31T07:37:34.287 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 1.367813230, 2024-08-31T07:37:34.287 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:34.287 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 114397, 2024-08-31T07:37:34.287 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 4.318290596, 2024-08-31T07:37:34.287 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000037748 2024-08-31T07:37:34.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.287 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:34.288 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 164947, 2024-08-31T07:37:34.288 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.033242987, 2024-08-31T07:37:34.288 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000201 2024-08-31T07:37:34.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.288 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-31T07:37:34.288 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 707296 2024-08-31T07:37:34.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.288 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:34.289 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 184527, 2024-08-31T07:37:34.289 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 209258, 2024-08-31T07:37:34.289 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 68589812, 2024-08-31T07:37:34.289 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 257907428, 2024-08-31T07:37:34.289 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 16, 2024-08-31T07:37:34.289 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-08-31T07:37:34.289 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 14.978406690, 2024-08-31T07:37:34.289 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 7.905712502, 2024-08-31T07:37:34.289 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 7.783285207, 2024-08-31T07:37:34.290 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.657129697, 2024-08-31T07:37:34.290 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:34.290 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 209260, 2024-08-31T07:37:34.290 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 6.579672204, 2024-08-31T07:37:34.290 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000031442 2024-08-31T07:37:34.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.290 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:34.290 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 355789, 2024-08-31T07:37:34.290 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.065086705, 2024-08-31T07:37:34.291 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000182 2024-08-31T07:37:34.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.291 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2244272, 2024-08-31T07:37:34.291 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 598848 2024-08-31T07:37:34.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.291 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-08-31T07:37:34.291 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-08-31T07:37:34.291 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-08-31T07:37:34.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.292 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-08-31T07:37:34.292 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-08-31T07:37:34.292 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-08-31T07:37:34.292 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 81721, 2024-08-31T07:37:34.292 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 74.100867475, 2024-08-31T07:37:34.292 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000906754 2024-08-31T07:37:34.292 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:34.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.293 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-08-31T07:37:34.293 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-08-31T07:37:34.293 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-08-31T07:37:34.293 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 30808, 2024-08-31T07:37:34.293 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 3.564856285, 2024-08-31T07:37:34.293 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000115712 2024-08-31T07:37:34.293 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:34.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.294 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-08-31T07:37:34.294 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 152396, 2024-08-31T07:37:34.294 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 137043, 2024-08-31T07:37:34.294 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-08-31T07:37:34.294 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 137043, 2024-08-31T07:37:34.294 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1189.614508590, 2024-08-31T07:37:34.294 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.008680593 2024-08-31T07:37:34.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.295 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-08-31T07:37:34.295 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 15353, 2024-08-31T07:37:34.295 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 26, 2024-08-31T07:37:34.295 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 2, 2024-08-31T07:37:34.295 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 101, 2024-08-31T07:37:34.295 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 8, 2024-08-31T07:37:34.295 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 7, 2024-08-31T07:37:34.295 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 2651, 2024-08-31T07:37:34.296 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 548, 2024-08-31T07:37:34.296 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 1860, 2024-08-31T07:37:34.296 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 243, 2024-08-31T07:37:34.296 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 1869, 2024-08-31T07:37:34.296 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 3028, 2024-08-31T07:37:34.296 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 221, 2024-08-31T07:37:34.296 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 222, 2024-08-31T07:37:34.296 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 26, 2024-08-31T07:37:34.296 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 222886, 2024-08-31T07:37:34.297 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 149491, 2024-08-31T07:37:34.297 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 15324, 2024-08-31T07:37:34.297 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 163, 2024-08-31T07:37:34.297 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 2, 2024-08-31T07:37:34.297 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-08-31T07:37:34.297 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 1451, 2024-08-31T07:37:34.297 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 26210, 2024-08-31T07:37:34.297 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-08-31T07:37:34.298 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 37, 2024-08-31T07:37:34.298 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 2166, 2024-08-31T07:37:34.298 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 47, 2024-08-31T07:37:34.298 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 1587, 2024-08-31T07:37:34.298 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-08-31T07:37:34.298 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 2271, 2024-08-31T07:37:34.298 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-08-31T07:37:34.298 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 760, 2024-08-31T07:37:34.298 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 212060996, 2024-08-31T07:37:34.299 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 6, 2024-08-31T07:37:34.299 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 2271, 2024-08-31T07:37:34.299 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-08-31T07:37:34.299 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T07:37:34.299 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-08-31T07:37:34.299 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 677, 2024-08-31T07:37:34.299 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 26, 2024-08-31T07:37:34.299 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 651, 2024-08-31T07:37:34.300 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 1594, 2024-08-31T07:37:34.300 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 76, 2024-08-31T07:37:34.300 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 28748, 2024-08-31T07:37:34.300 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 73194, 2024-08-31T07:37:34.300 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T07:37:34.300 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-31T07:37:34.300 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T07:37:34.300 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 67457, 2024-08-31T07:37:34.300 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 13716, 2024-08-31T07:37:34.301 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 3189, 2024-08-31T07:37:34.301 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 91985 2024-08-31T07:37:34.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.301 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-08-31T07:37:34.301 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 13, 2024-08-31T07:37:34.301 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 39, 2024-08-31T07:37:34.301 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 26, 2024-08-31T07:37:34.301 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 176, 2024-08-31T07:37:34.301 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 253, 2024-08-31T07:37:34.302 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 57, 2024-08-31T07:37:34.302 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-08-31T07:37:34.302 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-08-31T07:37:34.302 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 16034, 2024-08-31T07:37:34.302 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 15978, 2024-08-31T07:37:34.302 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-08-31T07:37:34.302 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-08-31T07:37:34.302 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-08-31T07:37:34.303 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-08-31T07:37:34.303 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-08-31T07:37:34.303 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-08-31T07:37:34.303 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-08-31T07:37:34.303 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 19, 2024-08-31T07:37:34.303 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 2115, 2024-08-31T07:37:34.303 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 26, 2024-08-31T07:37:34.303 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 45, 2024-08-31T07:37:34.304 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-08-31T07:37:34.304 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 15, 2024-08-31T07:37:34.304 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-08-31T07:37:34.304 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0, 2024-08-31T07:37:34.304 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_started": 0, 2024-08-31T07:37:34.304 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_succeeded": 0, 2024-08-31T07:37:34.304 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_write_failed": 0 2024-08-31T07:37:34.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.305 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-08-31T07:37:34.305 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-08-31T07:37:34.305 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 99235, 2024-08-31T07:37:34.305 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 31242, 2024-08-31T07:37:34.305 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 24581, 2024-08-31T07:37:34.305 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 74654, 2024-08-31T07:37:34.306 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-08-31T07:37:34.306 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 6661, 2024-08-31T07:37:34.306 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 188, 2024-08-31T07:37:34.306 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 60, 2024-08-31T07:37:34.306 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 38, 2024-08-31T07:37:34.307 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 150, 2024-08-31T07:37:34.307 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 7, 2024-08-31T07:37:34.307 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-08-31T07:37:34.307 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 22, 2024-08-31T07:37:34.307 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 119189440, 2024-08-31T07:37:34.307 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 708677008, 2024-08-31T07:37:34.307 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 0, 2024-08-31T07:37:34.308 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-08-31T07:37:34.308 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 43028, 2024-08-31T07:37:34.308 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 247.006275755, 2024-08-31T07:37:34.308 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005740593 2024-08-31T07:37:34.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.308 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 0 2024-08-31T07:37:34.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.308 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-08-31T07:37:34.308 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 2278, 2024-08-31T07:37:34.309 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 20215, 2024-08-31T07:37:34.309 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 17937, 2024-08-31T07:37:34.309 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 209, 2024-08-31T07:37:34.309 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 358, 2024-08-31T07:37:34.309 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 149, 2024-08-31T07:37:34.309 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 2646, 2024-08-31T07:37:34.309 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 35659, 2024-08-31T07:37:34.309 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 33013, 2024-08-31T07:37:34.310 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 177, 2024-08-31T07:37:34.310 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 18707, 2024-08-31T07:37:34.310 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 18530, 2024-08-31T07:37:34.310 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 151848, 2024-08-31T07:37:34.310 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 368979968 2024-08-31T07:37:34.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.310 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-08-31T07:37:34.310 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 215137, 2024-08-31T07:37:34.311 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 15546, 2024-08-31T07:37:34.311 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 152396, 2024-08-31T07:37:34.311 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 2283, 2024-08-31T07:37:34.311 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 33169, 2024-08-31T07:37:34.311 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-08-31T07:37:34.311 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 16823, 2024-08-31T07:37:34.311 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 336.660441426, 2024-08-31T07:37:34.311 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.020011914 2024-08-31T07:37:34.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.312 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-08-31T07:37:34.312 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 20405, 2024-08-31T07:37:34.312 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 138.482947140, 2024-08-31T07:37:34.312 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.006786716 2024-08-31T07:37:34.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.312 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-08-31T07:37:34.312 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.312 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.312 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.313 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-08-31T07:37:34.313 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.313 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.313 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.313 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-08-31T07:37:34.313 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 25429, 2024-08-31T07:37:34.314 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 182.392565025, 2024-08-31T07:37:34.314 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.007172620 2024-08-31T07:37:34.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.314 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-08-31T07:37:34.314 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.314 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.314 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.314 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-08-31T07:37:34.315 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2, 2024-08-31T07:37:34.315 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000501149, 2024-08-31T07:37:34.315 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000250574 2024-08-31T07:37:34.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.315 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-08-31T07:37:34.315 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.315 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.315 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.316 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-08-31T07:37:34.316 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.316 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.317 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.317 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-08-31T07:37:34.317 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.317 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.317 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.317 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-08-31T07:37:34.318 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 25, 2024-08-31T07:37:34.318 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.005395858, 2024-08-31T07:37:34.318 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000215834 2024-08-31T07:37:34.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.318 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-08-31T07:37:34.318 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 119, 2024-08-31T07:37:34.318 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 16.941763920, 2024-08-31T07:37:34.318 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.142367764 2024-08-31T07:37:34.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.319 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-08-31T07:37:34.319 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.319 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.319 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.319 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-08-31T07:37:34.319 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 25, 2024-08-31T07:37:34.320 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 11.200480764, 2024-08-31T07:37:34.320 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.448019230 2024-08-31T07:37:34.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.320 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-08-31T07:37:34.320 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 14, 2024-08-31T07:37:34.320 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.028916820, 2024-08-31T07:37:34.320 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002065487 2024-08-31T07:37:34.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.321 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-08-31T07:37:34.321 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 54163, 2024-08-31T07:37:34.321 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 194.774800476, 2024-08-31T07:37:34.321 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.003596085 2024-08-31T07:37:34.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.321 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-08-31T07:37:34.321 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 3333, 2024-08-31T07:37:34.321 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 155.919502064, 2024-08-31T07:37:34.322 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.046780528 2024-08-31T07:37:34.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.322 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-08-31T07:37:34.322 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.322 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.322 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.322 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-08-31T07:37:34.322 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.323 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.323 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.323 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-08-31T07:37:34.323 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1, 2024-08-31T07:37:34.323 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.001379825, 2024-08-31T07:37:34.323 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001379825 2024-08-31T07:37:34.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.324 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-08-31T07:37:34.324 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 19, 2024-08-31T07:37:34.324 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 12.391082300, 2024-08-31T07:37:34.324 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.652162226 2024-08-31T07:37:34.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.324 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-08-31T07:37:34.324 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.325 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.325 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.325 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-08-31T07:37:34.325 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 148, 2024-08-31T07:37:34.325 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.301932753, 2024-08-31T07:37:34.325 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.008796842 2024-08-31T07:37:34.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.326 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-08-31T07:37:34.326 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.326 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.326 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.326 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-08-31T07:37:34.326 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 496, 2024-08-31T07:37:34.326 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.911635707, 2024-08-31T07:37:34.327 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001837975 2024-08-31T07:37:34.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.327 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-08-31T07:37:34.327 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.327 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.327 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.327 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-08-31T07:37:34.328 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 7, 2024-08-31T07:37:34.328 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.019882115, 2024-08-31T07:37:34.328 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002840302 2024-08-31T07:37:34.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.328 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-08-31T07:37:34.328 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.328 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.328 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.329 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-08-31T07:37:34.329 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 16034, 2024-08-31T07:37:34.329 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 138.581281248, 2024-08-31T07:37:34.329 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.008642963 2024-08-31T07:37:34.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.329 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-08-31T07:37:34.329 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-08-31T07:37:34.330 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-08-31T07:37:34.330 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.330 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.330 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.330 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:34.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.330 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-08-31T07:37:34.330 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-08-31T07:37:34.331 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 3, 2024-08-31T07:37:34.331 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 1, 2024-08-31T07:37:34.331 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-08-31T07:37:34.331 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-08-31T07:37:34.331 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 13659, 2024-08-31T07:37:34.331 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 6829, 2024-08-31T07:37:34.331 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 1494, 2024-08-31T07:37:34.331 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-08-31T07:37:34.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.332 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-08-31T07:37:34.332 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:34.332 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:34.332 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:34.332 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:34.332 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:34.332 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:34.333 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:34.333 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:34.333 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:34.333 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:34.333 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:34.333 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:34.333 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:34.333 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:34.334 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:34.334 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:34.334 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:34.334 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:34.334 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:34.334 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:34.334 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:34.334 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:34.335 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:34.335 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:34.335 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:34.335 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:34.335 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:34.335 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:34.335 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:34.335 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:34.335 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:34.336 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-08-31T07:37:34.336 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:34.336 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:34.336 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:34.336 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:34.336 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 1182178, 2024-08-31T07:37:34.336 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 1693, 2024-08-31T07:37:34.336 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 88, 2024-08-31T07:37:34.337 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 1, 2024-08-31T07:37:34.337 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-08-31T07:37:34.337 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-08-31T07:37:34.337 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:34.337 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:34.337 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:34.337 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:34.337 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 7968, 2024-08-31T07:37:34.338 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 176, 2024-08-31T07:37:34.338 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:34.338 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:34.338 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:34.338 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-08-31T07:37:34.338 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 8299319, 2024-08-31T07:37:34.338 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 78965, 2024-08-31T07:37:34.338 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:34.339 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-08-31T07:37:34.339 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:34.339 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-08-31T07:37:34.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.339 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-08-31T07:37:34.339 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-08-31T07:37:34.339 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-08-31T07:37:34.339 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 64942, 2024-08-31T07:37:34.340 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 708255325, 2024-08-31T07:37:34.340 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 37, 2024-08-31T07:37:34.340 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 64942, 2024-08-31T07:37:34.340 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-08-31T07:37:34.340 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 64942, 2024-08-31T07:37:34.340 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 373.421907329, 2024-08-31T07:37:34.340 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005750083 2024-08-31T07:37:34.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.341 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-08-31T07:37:34.341 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-08-31T07:37:34.341 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 64942, 2024-08-31T07:37:34.341 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 66186 2024-08-31T07:37:34.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.341 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 64942, 2024-08-31T07:37:34.341 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 3090, 2024-08-31T07:37:34.342 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 61852, 2024-08-31T07:37:34.342 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-08-31T07:37:34.342 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-08-31T07:37:34.342 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 2, 2024-08-31T07:37:34.342 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 47, 2024-08-31T07:37:34.342 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 733, 2024-08-31T07:37:34.342 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 43735, 2024-08-31T07:37:34.342 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 489, 2024-08-31T07:37:34.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-08-31T07:37:34.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-08-31T07:37:34.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 1, 2024-08-31T07:37:34.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 8, 2024-08-31T07:37:34.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 16715, 2024-08-31T07:37:34.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-08-31T07:37:34.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-08-31T07:37:34.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-08-31T07:37:34.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 2305, 2024-08-31T07:37:34.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 94, 2024-08-31T07:37:34.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-08-31T07:37:34.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-08-31T07:37:34.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-08-31T07:37:34.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-08-31T07:37:34.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-08-31T07:37:34.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-08-31T07:37:34.344 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T07:37:34.345 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-08-31T07:37:34.345 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-08-31T07:37:34.345 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-08-31T07:37:34.345 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-08-31T07:37:34.345 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-08-31T07:37:34.345 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-08-31T07:37:34.345 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-08-31T07:37:34.345 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-08-31T07:37:34.346 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-08-31T07:37:34.346 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-08-31T07:37:34.346 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-08-31T07:37:34.346 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-08-31T07:37:34.346 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-08-31T07:37:34.346 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-08-31T07:37:34.346 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-08-31T07:37:34.347 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-08-31T07:37:34.347 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-08-31T07:37:34.347 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-08-31T07:37:34.347 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-08-31T07:37:34.347 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 118, 2024-08-31T07:37:34.347 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-08-31T07:37:34.347 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 40, 2024-08-31T07:37:34.347 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 12, 2024-08-31T07:37:34.348 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-08-31T07:37:34.348 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-08-31T07:37:34.348 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-08-31T07:37:34.348 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 1537, 2024-08-31T07:37:34.348 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 88, 2024-08-31T07:37:34.348 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 432 2024-08-31T07:37:34.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.348 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-08-31T07:37:34.349 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 1, 2024-08-31T07:37:34.349 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-08-31T07:37:34.349 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-08-31T07:37:34.349 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-08-31T07:37:34.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.349 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-08-31T07:37:34.349 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-08-31T07:37:34.349 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 6, 2024-08-31T07:37:34.350 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-08-31T07:37:34.350 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 6, 2024-08-31T07:37:34.350 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 15978, 2024-08-31T07:37:34.350 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 15978, 2024-08-31T07:37:34.350 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-08-31T07:37:34.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.350 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:34.350 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:34.350 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-08-31T07:37:34.351 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:34.351 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 631992, 2024-08-31T07:37:34.351 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 157887609, 2024-08-31T07:37:34.351 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:34.351 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 631992, 2024-08-31T07:37:34.351 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:34.351 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:34.351 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 631992, 2024-08-31T07:37:34.352 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 157887609, 2024-08-31T07:37:34.352 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:34.352 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.352 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.352 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.352 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:34.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.352 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-08-31T07:37:34.353 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:34.353 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-08-31T07:37:34.353 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:34.353 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:34.353 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:34.353 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:34.353 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:34.353 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 64942, 2024-08-31T07:37:34.354 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 709871069, 2024-08-31T07:37:34.354 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 64942, 2024-08-31T07:37:34.354 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 709871069, 2024-08-31T07:37:34.354 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:34.354 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.354 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.354 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.354 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:34.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.355 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-08-31T07:37:34.355 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:34.355 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-08-31T07:37:34.355 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:34.355 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:34.355 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:34.355 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:34.356 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:34.356 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 64942, 2024-08-31T07:37:34.356 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 64942, 2024-08-31T07:37:34.356 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 64942, 2024-08-31T07:37:34.356 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 64942, 2024-08-31T07:37:34.356 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:34.356 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.356 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.357 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.357 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:34.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.357 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-08-31T07:37:34.357 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:34.357 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-08-31T07:37:34.357 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:34.358 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 15978, 2024-08-31T07:37:34.358 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 1617098, 2024-08-31T07:37:34.358 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:34.358 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 15978, 2024-08-31T07:37:34.358 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:34.358 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:34.358 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 707, 2024-08-31T07:37:34.358 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 1617098, 2024-08-31T07:37:34.359 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:34.359 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.359 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.359 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.359 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:34.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.359 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x5599443a4820": { 2024-08-31T07:37:34.359 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 750018, 2024-08-31T07:37:34.360 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-08-31T07:37:34.360 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:34.360 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 99235, 2024-08-31T07:37:34.360 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 704482704, 2024-08-31T07:37:34.360 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:34.360 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 99235, 2024-08-31T07:37:34.360 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:34.360 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:34.361 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 43028, 2024-08-31T07:37:34.361 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 703732686, 2024-08-31T07:37:34.361 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:34.361 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:34.361 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:34.361 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:34.361 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:34.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:34.362 INFO:teuthology.orchestra.run.smithi073.stdout: "trackedop": { 2024-08-31T07:37:34.362 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 2449 2024-08-31T07:37:34.362 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:34.362 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-08-31T07:37:34.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:34 smithi073 ceph-mon[21995]: pgmap v1150: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 650 KiB/s wr, 34 op/s 2024-08-31T07:37:34.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:34 smithi084 ceph-mon[22387]: pgmap v1150: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 650 KiB/s wr, 34 op/s 2024-08-31T07:37:34.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:34 smithi032 ceph-mon[17252]: pgmap v1150: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 650 KiB/s wr, 34 op/s 2024-08-31T07:37:35.773 INFO:tasks.check_counter:Daemon mds.b mds.exported=37 2024-08-31T07:37:35.774 INFO:tasks.check_counter:Daemon mds.b mds.imported=47 2024-08-31T07:37:35.774 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-08-31T07:37:35.774 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=25 2024-08-31T07:37:35.774 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=19 2024-08-31T07:37:35.774 DEBUG:tasks.check_counter:Getting stats from e 2024-08-31T07:37:35.774 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-31T07:37:36.025 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:37:36.633 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-08-31T07:37:36.633 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:36.633 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 2660, 2024-08-31T07:37:36.633 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2586, 2024-08-31T07:37:36.633 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 80956040, 2024-08-31T07:37:36.634 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 951804, 2024-08-31T07:37:36.634 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 7, 2024-08-31T07:37:36.634 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 3, 2024-08-31T07:37:36.634 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.363762725, 2024-08-31T07:37:36.634 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.158029159, 2024-08-31T07:37:36.634 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.142375611, 2024-08-31T07:37:36.634 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.056320491, 2024-08-31T07:37:36.634 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:36.635 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2585, 2024-08-31T07:37:36.635 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.322442742, 2024-08-31T07:37:36.635 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000124736 2024-08-31T07:37:36.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.635 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:36.635 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.635 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.635 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.636 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 1403792, 2024-08-31T07:37:36.636 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 287184 2024-08-31T07:37:36.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.636 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:36.636 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 960, 2024-08-31T07:37:36.636 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 1301, 2024-08-31T07:37:36.636 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 96721004, 2024-08-31T07:37:36.636 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 999752, 2024-08-31T07:37:36.637 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 7, 2024-08-31T07:37:36.637 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-08-31T07:37:36.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.171289955, 2024-08-31T07:37:36.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.053561732, 2024-08-31T07:37:36.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.083536078, 2024-08-31T07:37:36.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.025111106, 2024-08-31T07:37:36.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:36.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1300, 2024-08-31T07:37:36.640 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.048650918, 2024-08-31T07:37:36.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000037423 2024-08-31T07:37:36.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:36.641 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.641 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.641 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-08-31T07:37:36.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 696000 2024-08-31T07:37:36.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.642 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:36.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 2898, 2024-08-31T07:37:36.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2900, 2024-08-31T07:37:36.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 193693188, 2024-08-31T07:37:36.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 921571, 2024-08-31T07:37:36.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 7, 2024-08-31T07:37:36.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-08-31T07:37:36.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.361750606, 2024-08-31T07:37:36.643 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.089523504, 2024-08-31T07:37:36.643 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.185222788, 2024-08-31T07:37:36.643 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.070841234, 2024-08-31T07:37:36.643 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:36.643 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2898, 2024-08-31T07:37:36.643 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.087656322, 2024-08-31T07:37:36.643 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000030247 2024-08-31T07:37:36.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.643 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:36.644 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.644 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.644 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T07:37:36.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T07:37:36.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.644 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-08-31T07:37:36.645 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-08-31T07:37:36.645 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-08-31T07:37:36.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.645 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-08-31T07:37:36.645 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-08-31T07:37:36.645 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-08-31T07:37:36.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 9897, 2024-08-31T07:37:36.645 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.229640555, 2024-08-31T07:37:36.646 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000023203 2024-08-31T07:37:36.646 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:36.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.646 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-08-31T07:37:36.646 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-08-31T07:37:36.646 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-08-31T07:37:36.646 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.646 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.647 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.647 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:36.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-08-31T07:37:36.647 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 0, 2024-08-31T07:37:36.647 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 0, 2024-08-31T07:37:36.647 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-08-31T07:37:36.647 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.648 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.648 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.648 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-08-31T07:37:36.648 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-08-31T07:37:36.648 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-08-31T07:37:36.649 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-08-31T07:37:36.649 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 0, 2024-08-31T07:37:36.649 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-08-31T07:37:36.649 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-08-31T07:37:36.649 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 15792, 2024-08-31T07:37:36.649 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 2, 2024-08-31T07:37:36.649 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 15790, 2024-08-31T07:37:36.649 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-08-31T07:37:36.650 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 15790, 2024-08-31T07:37:36.650 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 0, 2024-08-31T07:37:36.650 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-08-31T07:37:36.650 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-08-31T07:37:36.650 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 18, 2024-08-31T07:37:36.650 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 0, 2024-08-31T07:37:36.650 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 0, 2024-08-31T07:37:36.650 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-08-31T07:37:36.650 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 0, 2024-08-31T07:37:36.651 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-08-31T07:37:36.651 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-08-31T07:37:36.651 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-08-31T07:37:36.651 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-08-31T07:37:36.651 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-08-31T07:37:36.651 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-08-31T07:37:36.651 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-08-31T07:37:36.651 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-08-31T07:37:36.652 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-08-31T07:37:36.652 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-08-31T07:37:36.652 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-08-31T07:37:36.652 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-08-31T07:37:36.652 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 0, 2024-08-31T07:37:36.652 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 0, 2024-08-31T07:37:36.652 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-08-31T07:37:36.653 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-08-31T07:37:36.653 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-08-31T07:37:36.653 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T07:37:36.653 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-08-31T07:37:36.653 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-08-31T07:37:36.653 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T07:37:36.653 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-08-31T07:37:36.653 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-08-31T07:37:36.654 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-08-31T07:37:36.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 0, 2024-08-31T07:37:36.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 0, 2024-08-31T07:37:36.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T07:37:36.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-31T07:37:36.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T07:37:36.654 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 0, 2024-08-31T07:37:36.654 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-08-31T07:37:36.655 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 0, 2024-08-31T07:37:36.655 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 0 2024-08-31T07:37:36.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-08-31T07:37:36.655 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-08-31T07:37:36.655 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-08-31T07:37:36.655 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-08-31T07:37:36.655 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 0, 2024-08-31T07:37:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-08-31T07:37:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-08-31T07:37:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-08-31T07:37:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-08-31T07:37:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-08-31T07:37:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-08-31T07:37:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-08-31T07:37:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-08-31T07:37:36.657 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-08-31T07:37:36.657 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-08-31T07:37:36.657 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-08-31T07:37:36.657 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-08-31T07:37:36.657 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-08-31T07:37:36.657 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 0, 2024-08-31T07:37:36.657 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 0, 2024-08-31T07:37:36.658 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T07:37:36.658 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-08-31T07:37:36.658 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-08-31T07:37:36.658 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-08-31T07:37:36.658 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-08-31T07:37:36.658 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0, 2024-08-31T07:37:36.658 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_started": 0, 2024-08-31T07:37:36.658 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_succeeded": 0, 2024-08-31T07:37:36.659 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_write_failed": 0 2024-08-31T07:37:36.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-08-31T07:37:36.659 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-08-31T07:37:36.659 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 0, 2024-08-31T07:37:36.659 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-08-31T07:37:36.659 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-08-31T07:37:36.659 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 76875, 2024-08-31T07:37:36.659 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-08-31T07:37:36.660 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-08-31T07:37:36.660 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 0, 2024-08-31T07:37:36.660 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-08-31T07:37:36.660 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-08-31T07:37:36.660 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 117, 2024-08-31T07:37:36.660 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 8, 2024-08-31T07:37:36.660 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-08-31T07:37:36.660 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-08-31T07:37:36.661 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 4194304, 2024-08-31T07:37:36.661 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 372485565, 2024-08-31T07:37:36.661 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 372469629, 2024-08-31T07:37:36.661 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-08-31T07:37:36.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.661 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.662 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 76875 2024-08-31T07:37:36.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-08-31T07:37:36.662 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1419, 2024-08-31T07:37:36.662 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 16546, 2024-08-31T07:37:36.662 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 15127, 2024-08-31T07:37:36.662 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 76, 2024-08-31T07:37:36.662 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 154, 2024-08-31T07:37:36.663 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 78, 2024-08-31T07:37:36.663 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 15792, 2024-08-31T07:37:36.663 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 16837, 2024-08-31T07:37:36.663 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 1045, 2024-08-31T07:37:36.663 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-08-31T07:37:36.663 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 0, 2024-08-31T07:37:36.663 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 0, 2024-08-31T07:37:36.664 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 140160, 2024-08-31T07:37:36.664 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 350113792 2024-08-31T07:37:36.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-08-31T07:37:36.664 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 0, 2024-08-31T07:37:36.664 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-08-31T07:37:36.664 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 0, 2024-08-31T07:37:36.664 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 0, 2024-08-31T07:37:36.665 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-08-31T07:37:36.665 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-08-31T07:37:36.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.665 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.665 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-08-31T07:37:36.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.666 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.666 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-08-31T07:37:36.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.666 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.666 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-08-31T07:37:36.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.667 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.667 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-08-31T07:37:36.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.667 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.668 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-08-31T07:37:36.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.668 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.668 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-08-31T07:37:36.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.669 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.669 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-08-31T07:37:36.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.669 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.670 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-08-31T07:37:36.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.670 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.670 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-08-31T07:37:36.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.670 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.671 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.671 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-08-31T07:37:36.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.672 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.672 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-08-31T07:37:36.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.673 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.673 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.673 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-08-31T07:37:36.673 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.673 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.673 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.674 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-08-31T07:37:36.674 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.674 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.674 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.674 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-08-31T07:37:36.674 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.674 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.674 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.675 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-08-31T07:37:36.675 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.675 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.675 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.675 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-08-31T07:37:36.675 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.676 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.676 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.676 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-08-31T07:37:36.676 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.676 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.676 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.677 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-08-31T07:37:36.677 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.677 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.677 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.677 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-08-31T07:37:36.677 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.677 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.678 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.678 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-08-31T07:37:36.678 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.678 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.678 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.678 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-08-31T07:37:36.679 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.679 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.679 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.679 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-08-31T07:37:36.679 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.679 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.679 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.680 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-08-31T07:37:36.680 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.680 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.680 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.680 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-08-31T07:37:36.680 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.680 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.681 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.681 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-08-31T07:37:36.681 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.681 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.681 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.681 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-08-31T07:37:36.682 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.682 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.682 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.682 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-08-31T07:37:36.682 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.682 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.682 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.683 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-08-31T07:37:36.683 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.683 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.683 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.683 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-08-31T07:37:36.683 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-08-31T07:37:36.684 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-08-31T07:37:36.684 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.684 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.684 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.684 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:36.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.684 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-08-31T07:37:36.685 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-08-31T07:37:36.685 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 3, 2024-08-31T07:37:36.685 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 1, 2024-08-31T07:37:36.685 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-08-31T07:37:36.685 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-08-31T07:37:36.685 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 0, 2024-08-31T07:37:36.685 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 0, 2024-08-31T07:37:36.685 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 2792, 2024-08-31T07:37:36.686 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-08-31T07:37:36.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.686 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-08-31T07:37:36.686 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:36.686 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:36.686 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:36.686 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:36.686 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:36.687 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:36.687 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:36.687 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:36.687 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:36.687 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:36.687 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:36.687 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:36.687 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:36.687 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:36.688 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:36.688 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:36.688 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:36.688 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:36.688 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:36.688 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:36.688 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:36.688 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:36.689 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:36.689 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:36.689 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:36.689 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:36.689 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:36.689 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:36.689 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:36.689 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:36.690 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:36.690 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-08-31T07:37:36.690 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:36.690 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:36.690 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:36.690 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:36.690 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 302610, 2024-08-31T07:37:36.690 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 1747, 2024-08-31T07:37:36.690 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-08-31T07:37:36.691 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-08-31T07:37:36.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-08-31T07:37:36.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-08-31T07:37:36.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:36.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:36.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:36.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:36.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 7968, 2024-08-31T07:37:36.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 176, 2024-08-31T07:37:36.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:36.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:36.692 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:36.692 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-08-31T07:37:36.692 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 14482147, 2024-08-31T07:37:36.692 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 110824, 2024-08-31T07:37:36.692 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:36.692 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-08-31T07:37:36.693 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:36.693 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-08-31T07:37:36.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.693 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-08-31T07:37:36.693 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-08-31T07:37:36.693 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-08-31T07:37:36.693 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 5943, 2024-08-31T07:37:36.693 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 21, 2024-08-31T07:37:36.694 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 3, 2024-08-31T07:37:36.694 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 5943, 2024-08-31T07:37:36.694 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-08-31T07:37:36.694 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5943, 2024-08-31T07:37:36.694 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 7.669813067, 2024-08-31T07:37:36.694 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001290562 2024-08-31T07:37:36.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.695 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-08-31T07:37:36.695 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-08-31T07:37:36.695 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5943, 2024-08-31T07:37:36.695 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 5944 2024-08-31T07:37:36.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.695 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 5943, 2024-08-31T07:37:36.695 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 5943, 2024-08-31T07:37:36.695 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 0, 2024-08-31T07:37:36.695 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-08-31T07:37:36.696 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-08-31T07:37:36.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 3470, 2024-08-31T07:37:36.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 0, 2024-08-31T07:37:36.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 2471, 2024-08-31T07:37:36.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 0, 2024-08-31T07:37:36.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 0, 2024-08-31T07:37:36.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-08-31T07:37:36.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-08-31T07:37:36.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-08-31T07:37:36.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-08-31T07:37:36.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 0, 2024-08-31T07:37:36.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-08-31T07:37:36.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-08-31T07:37:36.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-08-31T07:37:36.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 1, 2024-08-31T07:37:36.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 0, 2024-08-31T07:37:36.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-08-31T07:37:36.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-08-31T07:37:36.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-08-31T07:37:36.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-08-31T07:37:36.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-08-31T07:37:36.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-08-31T07:37:36.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T07:37:36.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-08-31T07:37:36.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-08-31T07:37:36.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-08-31T07:37:36.699 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-08-31T07:37:36.699 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-08-31T07:37:36.699 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-08-31T07:37:36.699 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-08-31T07:37:36.699 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-08-31T07:37:36.700 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-08-31T07:37:36.700 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-08-31T07:37:36.700 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-08-31T07:37:36.700 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-08-31T07:37:36.700 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-08-31T07:37:36.700 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-08-31T07:37:36.700 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-08-31T07:37:36.700 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-08-31T07:37:36.700 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-08-31T07:37:36.701 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-08-31T07:37:36.701 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-08-31T07:37:36.701 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 118, 2024-08-31T07:37:36.701 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-08-31T07:37:36.701 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 40, 2024-08-31T07:37:36.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 11, 2024-08-31T07:37:36.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-08-31T07:37:36.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 1, 2024-08-31T07:37:36.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-08-31T07:37:36.702 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 0, 2024-08-31T07:37:36.702 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 2, 2024-08-31T07:37:36.702 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 0 2024-08-31T07:37:36.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.702 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-08-31T07:37:36.702 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 0, 2024-08-31T07:37:36.702 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-08-31T07:37:36.703 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-08-31T07:37:36.703 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-08-31T07:37:36.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.703 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-08-31T07:37:36.703 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-08-31T07:37:36.703 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-08-31T07:37:36.703 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-08-31T07:37:36.703 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-08-31T07:37:36.703 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 0, 2024-08-31T07:37:36.704 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-08-31T07:37:36.704 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-08-31T07:37:36.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.704 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:36.704 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:36.704 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-08-31T07:37:36.704 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:36.704 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 6514, 2024-08-31T07:37:36.704 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 370752178, 2024-08-31T07:37:36.705 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:36.705 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 6514, 2024-08-31T07:37:36.705 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:36.705 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:36.705 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6514, 2024-08-31T07:37:36.705 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 370752178, 2024-08-31T07:37:36.705 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:36.705 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.706 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.706 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:36.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.706 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-08-31T07:37:36.706 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:36.706 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-08-31T07:37:36.706 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:36.707 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:36.707 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:36.707 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:36.707 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:36.707 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 5943, 2024-08-31T07:37:36.707 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 368316326, 2024-08-31T07:37:36.707 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 5943, 2024-08-31T07:37:36.707 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 368316326, 2024-08-31T07:37:36.707 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:36.708 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.708 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.708 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:36.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.708 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-08-31T07:37:36.708 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:36.708 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-08-31T07:37:36.708 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:36.709 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:36.709 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:36.709 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:36.709 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:36.709 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 5943, 2024-08-31T07:37:36.709 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 5943, 2024-08-31T07:37:36.709 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 5943, 2024-08-31T07:37:36.709 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 5943, 2024-08-31T07:37:36.710 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:36.710 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.710 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.710 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.710 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:36.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.710 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-08-31T07:37:36.710 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:36.711 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-08-31T07:37:36.711 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:36.711 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:36.711 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:36.711 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:36.711 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:36.711 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:36.711 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:36.712 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-08-31T07:37:36.712 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-08-31T07:37:36.712 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:36.712 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.712 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.712 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.712 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:36.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.713 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x561ed6d848f0": { 2024-08-31T07:37:36.713 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:36.713 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-08-31T07:37:36.713 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:36.713 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:36.713 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:36.713 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:36.714 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:36.714 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:36.714 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:36.714 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-08-31T07:37:36.714 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-08-31T07:37:36.714 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:36.714 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:36.714 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:36.715 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:36.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:36.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:36.715 INFO:teuthology.orchestra.run.smithi073.stdout: "trackedop": { 2024-08-31T07:37:36.715 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 0 2024-08-31T07:37:36.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:36.715 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-08-31T07:37:36.716 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:36 smithi073 ceph-mon[21995]: pgmap v1151: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.5 MiB/s wr, 52 op/s 2024-08-31T07:37:36.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:36 smithi084 ceph-mon[22387]: pgmap v1151: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.5 MiB/s wr, 52 op/s 2024-08-31T07:37:36.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:36 smithi032 ceph-mon[17252]: pgmap v1151: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.5 MiB/s wr, 52 op/s 2024-08-31T07:37:37.433 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-08-31T07:37:37.433 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-08-31T07:37:37.434 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-08-31T07:37:37.434 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-08-31T07:37:37.434 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T07:37:37.434 DEBUG:tasks.check_counter:Getting stats from h 2024-08-31T07:37:37.434 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-31T07:37:37.687 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:37:38.246 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-08-31T07:37:38.246 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 59430, 2024-08-31T07:37:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 61333, 2024-08-31T07:37:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 25087535, 2024-08-31T07:37:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 123038784, 2024-08-31T07:37:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 62, 2024-08-31T07:37:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-08-31T07:37:38.247 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 4.755469427, 2024-08-31T07:37:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 2.289966145, 2024-08-31T07:37:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 2.306134064, 2024-08-31T07:37:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.462393410, 2024-08-31T07:37:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 61332, 2024-08-31T07:37:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 2.893524727, 2024-08-31T07:37:38.248 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000047178 2024-08-31T07:37:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 93579, 2024-08-31T07:37:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.023027895, 2024-08-31T07:37:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000246 2024-08-31T07:37:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 1436944, 2024-08-31T07:37:38.249 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 347984 2024-08-31T07:37:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 200212, 2024-08-31T07:37:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 292686, 2024-08-31T07:37:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 64002113, 2024-08-31T07:37:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 339056569, 2024-08-31T07:37:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 17, 2024-08-31T07:37:38.250 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 7, 2024-08-31T07:37:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 15.614407104, 2024-08-31T07:37:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 9.449693705, 2024-08-31T07:37:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 7.304747775, 2024-08-31T07:37:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.506611385, 2024-08-31T07:37:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 292684, 2024-08-31T07:37:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 12.510754019, 2024-08-31T07:37:38.251 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000042744 2024-08-31T07:37:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 256437, 2024-08-31T07:37:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.071073968, 2024-08-31T07:37:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000277 2024-08-31T07:37:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-31T07:37:38.252 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 789072 2024-08-31T07:37:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 176771, 2024-08-31T07:37:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 163814, 2024-08-31T07:37:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 65769007, 2024-08-31T07:37:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 234573958, 2024-08-31T07:37:38.253 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 12, 2024-08-31T07:37:38.254 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 7, 2024-08-31T07:37:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 13.108231879, 2024-08-31T07:37:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 6.778900286, 2024-08-31T07:37:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 6.918471536, 2024-08-31T07:37:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.728393047, 2024-08-31T07:37:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 163813, 2024-08-31T07:37:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 6.130212263, 2024-08-31T07:37:38.255 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000037422 2024-08-31T07:37:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 295237, 2024-08-31T07:37:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.052625082, 2024-08-31T07:37:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000178 2024-08-31T07:37:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.256 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-31T07:37:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 448 2024-08-31T07:37:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-08-31T07:37:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-08-31T07:37:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-08-31T07:37:38.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-08-31T07:37:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-08-31T07:37:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-08-31T07:37:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 47879, 2024-08-31T07:37:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 55.273382055, 2024-08-31T07:37:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001154438 2024-08-31T07:37:38.258 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-08-31T07:37:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-08-31T07:37:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-08-31T07:37:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 29544, 2024-08-31T07:37:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 2.309186669, 2024-08-31T07:37:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000078160 2024-08-31T07:37:38.259 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-08-31T07:37:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 105034, 2024-08-31T07:37:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 97835, 2024-08-31T07:37:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-08-31T07:37:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 97835, 2024-08-31T07:37:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 657.145681738, 2024-08-31T07:37:38.260 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.006716877 2024-08-31T07:37:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-08-31T07:37:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 7199, 2024-08-31T07:37:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-08-31T07:37:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-08-31T07:37:38.261 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 56, 2024-08-31T07:37:38.262 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 5, 2024-08-31T07:37:38.262 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 2, 2024-08-31T07:37:38.262 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 1854, 2024-08-31T07:37:38.262 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 366, 2024-08-31T07:37:38.262 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 1472, 2024-08-31T07:37:38.262 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 16, 2024-08-31T07:37:38.262 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 1319, 2024-08-31T07:37:38.263 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 396, 2024-08-31T07:37:38.263 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 157, 2024-08-31T07:37:38.263 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 157, 2024-08-31T07:37:38.263 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 22, 2024-08-31T07:37:38.263 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 158669, 2024-08-31T07:37:38.264 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 95770, 2024-08-31T07:37:38.264 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 7186, 2024-08-31T07:37:38.264 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 59, 2024-08-31T07:37:38.264 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-08-31T07:37:38.264 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-08-31T07:37:38.264 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 1565, 2024-08-31T07:37:38.264 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 7566, 2024-08-31T07:37:38.264 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-08-31T07:37:38.265 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 6, 2024-08-31T07:37:38.265 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 297, 2024-08-31T07:37:38.265 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 9, 2024-08-31T07:37:38.265 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 495, 2024-08-31T07:37:38.265 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-08-31T07:37:38.265 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 1501, 2024-08-31T07:37:38.265 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-08-31T07:37:38.265 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 745, 2024-08-31T07:37:38.266 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 212490313, 2024-08-31T07:37:38.266 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 6, 2024-08-31T07:37:38.266 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 1501, 2024-08-31T07:37:38.266 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-08-31T07:37:38.266 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T07:37:38.266 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-08-31T07:37:38.266 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 180, 2024-08-31T07:37:38.266 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T07:37:38.267 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 180, 2024-08-31T07:37:38.267 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 1321, 2024-08-31T07:37:38.267 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 7, 2024-08-31T07:37:38.267 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 12518, 2024-08-31T07:37:38.267 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 71378, 2024-08-31T07:37:38.267 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T07:37:38.267 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 2, 2024-08-31T07:37:38.267 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T07:37:38.268 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 44758, 2024-08-31T07:37:38.268 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 14029, 2024-08-31T07:37:38.268 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 2931, 2024-08-31T07:37:38.268 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 70650 2024-08-31T07:37:38.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.268 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-08-31T07:37:38.268 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 10, 2024-08-31T07:37:38.269 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 67, 2024-08-31T07:37:38.269 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 45, 2024-08-31T07:37:38.269 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 102, 2024-08-31T07:37:38.269 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 38, 2024-08-31T07:37:38.269 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 14, 2024-08-31T07:37:38.269 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-08-31T07:37:38.269 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-08-31T07:37:38.269 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 15297, 2024-08-31T07:37:38.270 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 15283, 2024-08-31T07:37:38.270 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-08-31T07:37:38.270 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-08-31T07:37:38.270 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-08-31T07:37:38.270 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-08-31T07:37:38.270 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-08-31T07:37:38.270 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-08-31T07:37:38.270 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-08-31T07:37:38.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 19, 2024-08-31T07:37:38.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 1763, 2024-08-31T07:37:38.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T07:37:38.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 6, 2024-08-31T07:37:38.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-08-31T07:37:38.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 7, 2024-08-31T07:37:38.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-08-31T07:37:38.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0, 2024-08-31T07:37:38.272 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_started": 0, 2024-08-31T07:37:38.272 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_succeeded": 0, 2024-08-31T07:37:38.272 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_write_failed": 0 2024-08-31T07:37:38.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.273 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-08-31T07:37:38.273 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-08-31T07:37:38.273 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 84368, 2024-08-31T07:37:38.273 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 4758, 2024-08-31T07:37:38.273 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 1, 2024-08-31T07:37:38.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 84367, 2024-08-31T07:37:38.274 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-08-31T07:37:38.274 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 4757, 2024-08-31T07:37:38.274 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 134, 2024-08-31T07:37:38.274 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 6, 2024-08-31T07:37:38.274 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 1, 2024-08-31T07:37:38.274 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 133, 2024-08-31T07:37:38.275 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 7, 2024-08-31T07:37:38.275 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-08-31T07:37:38.275 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 5, 2024-08-31T07:37:38.275 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 4194352, 2024-08-31T07:37:38.275 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 409187139, 2024-08-31T07:37:38.275 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 0, 2024-08-31T07:37:38.275 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-08-31T07:37:38.275 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 25716, 2024-08-31T07:37:38.275 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 190.364707009, 2024-08-31T07:37:38.275 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.007402578 2024-08-31T07:37:38.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.276 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 0 2024-08-31T07:37:38.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.276 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-08-31T07:37:38.276 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1491, 2024-08-31T07:37:38.276 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 16964, 2024-08-31T07:37:38.276 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 15473, 2024-08-31T07:37:38.277 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 179, 2024-08-31T07:37:38.277 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 235, 2024-08-31T07:37:38.277 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 56, 2024-08-31T07:37:38.277 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 1854, 2024-08-31T07:37:38.277 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 31256, 2024-08-31T07:37:38.277 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 29402, 2024-08-31T07:37:38.277 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 152, 2024-08-31T07:37:38.277 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 16735, 2024-08-31T07:37:38.278 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 16583, 2024-08-31T07:37:38.278 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 146056, 2024-08-31T07:37:38.278 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 355348480 2024-08-31T07:37:38.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.278 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-08-31T07:37:38.278 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 152557, 2024-08-31T07:37:38.278 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 1540, 2024-08-31T07:37:38.278 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 105034, 2024-08-31T07:37:38.279 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 2215, 2024-08-31T07:37:38.279 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 7912, 2024-08-31T07:37:38.279 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-08-31T07:37:38.279 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 15260, 2024-08-31T07:37:38.279 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 172.370407398, 2024-08-31T07:37:38.279 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.011295570 2024-08-31T07:37:38.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.279 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-08-31T07:37:38.280 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1487, 2024-08-31T07:37:38.280 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 26.940250569, 2024-08-31T07:37:38.280 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.018117182 2024-08-31T07:37:38.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.280 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-08-31T07:37:38.280 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.280 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.280 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.281 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-08-31T07:37:38.281 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.281 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.281 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.286 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-08-31T07:37:38.286 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 23909, 2024-08-31T07:37:38.286 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 37.325490387, 2024-08-31T07:37:38.286 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001561148 2024-08-31T07:37:38.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.286 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-08-31T07:37:38.287 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.287 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.287 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.287 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-08-31T07:37:38.287 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.287 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.287 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.288 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-08-31T07:37:38.288 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.288 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.288 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.288 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-08-31T07:37:38.288 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.289 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.289 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.289 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-08-31T07:37:38.289 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.289 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.289 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.290 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-08-31T07:37:38.290 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.290 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.290 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.290 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-08-31T07:37:38.291 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.291 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.291 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.291 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-08-31T07:37:38.291 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.292 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.292 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.292 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-08-31T07:37:38.292 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.292 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.293 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.293 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-08-31T07:37:38.293 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.293 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.293 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.293 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-08-31T07:37:38.294 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 37559, 2024-08-31T07:37:38.294 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 197.669662537, 2024-08-31T07:37:38.294 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005262910 2024-08-31T07:37:38.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.294 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-08-31T07:37:38.294 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 3804, 2024-08-31T07:37:38.294 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 27.378723975, 2024-08-31T07:37:38.294 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.007197351 2024-08-31T07:37:38.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.295 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-08-31T07:37:38.295 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.295 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.295 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.295 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-08-31T07:37:38.295 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.295 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.296 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.296 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-08-31T07:37:38.296 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.296 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.296 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.296 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-08-31T07:37:38.297 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.297 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.297 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.297 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-08-31T07:37:38.297 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.297 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.298 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.298 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-08-31T07:37:38.298 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 99, 2024-08-31T07:37:38.298 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 136.763385298, 2024-08-31T07:37:38.298 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 1.381448336 2024-08-31T07:37:38.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.298 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-08-31T07:37:38.299 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.299 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.299 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.299 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-08-31T07:37:38.299 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 420, 2024-08-31T07:37:38.299 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.382121847, 2024-08-31T07:37:38.299 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000909813 2024-08-31T07:37:38.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.300 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-08-31T07:37:38.300 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.300 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.300 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.300 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-08-31T07:37:38.300 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.301 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.301 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.301 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-08-31T07:37:38.301 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.301 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.301 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.301 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-08-31T07:37:38.302 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 15297, 2024-08-31T07:37:38.302 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 58.315639727, 2024-08-31T07:37:38.302 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.003812227 2024-08-31T07:37:38.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.302 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-08-31T07:37:38.302 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-08-31T07:37:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-08-31T07:37:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.303 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-08-31T07:37:38.304 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-08-31T07:37:38.304 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 3, 2024-08-31T07:37:38.304 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 1, 2024-08-31T07:37:38.304 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-08-31T07:37:38.304 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-08-31T07:37:38.304 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 10548, 2024-08-31T07:37:38.304 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 5274, 2024-08-31T07:37:38.304 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 1438, 2024-08-31T07:37:38.305 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-08-31T07:37:38.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.305 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-08-31T07:37:38.305 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:38.305 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:38.305 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:38.305 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:38.305 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:38.305 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:38.306 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:38.306 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:38.306 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:38.306 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:38.306 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:38.306 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:38.306 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:38.306 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:38.307 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:38.307 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:38.307 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:38.307 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:38.307 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:38.307 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:38.307 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:38.307 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:38.308 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:38.308 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:38.308 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:38.308 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:38.308 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:38.308 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:38.308 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:38.308 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:38.309 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:38.309 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-08-31T07:37:38.309 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:38.309 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:38.309 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:38.309 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:38.309 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 2430734, 2024-08-31T07:37:38.310 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 1149, 2024-08-31T07:37:38.310 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 88, 2024-08-31T07:37:38.310 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 1, 2024-08-31T07:37:38.310 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-08-31T07:37:38.310 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-08-31T07:37:38.310 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:38.310 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:38.310 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:38.311 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:38.311 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 7968, 2024-08-31T07:37:38.311 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 176, 2024-08-31T07:37:38.311 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:38.311 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:38.311 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:38.311 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-08-31T07:37:38.311 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 5591466, 2024-08-31T07:37:38.312 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 56601, 2024-08-31T07:37:38.312 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:38.312 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-08-31T07:37:38.312 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:38.312 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-08-31T07:37:38.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.312 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-08-31T07:37:38.312 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-08-31T07:37:38.313 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-08-31T07:37:38.313 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 45478, 2024-08-31T07:37:38.313 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 408811987, 2024-08-31T07:37:38.313 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 33, 2024-08-31T07:37:38.313 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 45478, 2024-08-31T07:37:38.313 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-08-31T07:37:38.314 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 45478, 2024-08-31T07:37:38.314 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 295.657818610, 2024-08-31T07:37:38.314 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.006501117 2024-08-31T07:37:38.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.314 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-08-31T07:37:38.314 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-08-31T07:37:38.314 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 45478, 2024-08-31T07:37:38.314 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 46595 2024-08-31T07:37:38.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.315 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 45478, 2024-08-31T07:37:38.315 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 2279, 2024-08-31T07:37:38.315 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 43199, 2024-08-31T07:37:38.315 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-08-31T07:37:38.315 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-08-31T07:37:38.315 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 0, 2024-08-31T07:37:38.315 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 16, 2024-08-31T07:37:38.316 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 778, 2024-08-31T07:37:38.316 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 26494, 2024-08-31T07:37:38.316 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 492, 2024-08-31T07:37:38.316 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-08-31T07:37:38.316 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-08-31T07:37:38.317 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-08-31T07:37:38.317 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 5, 2024-08-31T07:37:38.317 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 15407, 2024-08-31T07:37:38.317 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-08-31T07:37:38.317 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-08-31T07:37:38.317 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-08-31T07:37:38.317 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 1501, 2024-08-31T07:37:38.317 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 32, 2024-08-31T07:37:38.318 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-08-31T07:37:38.318 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-08-31T07:37:38.318 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-08-31T07:37:38.318 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-08-31T07:37:38.319 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-08-31T07:37:38.319 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-08-31T07:37:38.319 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T07:37:38.319 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-08-31T07:37:38.319 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-08-31T07:37:38.319 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-08-31T07:37:38.319 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-08-31T07:37:38.319 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-08-31T07:37:38.319 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-08-31T07:37:38.320 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-08-31T07:37:38.320 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-08-31T07:37:38.320 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-08-31T07:37:38.320 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-08-31T07:37:38.320 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-08-31T07:37:38.320 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-08-31T07:37:38.321 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-08-31T07:37:38.321 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-08-31T07:37:38.321 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-08-31T07:37:38.321 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-08-31T07:37:38.321 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-08-31T07:37:38.321 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-08-31T07:37:38.321 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-08-31T07:37:38.322 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 118, 2024-08-31T07:37:38.322 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-08-31T07:37:38.322 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 40, 2024-08-31T07:37:38.322 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 12, 2024-08-31T07:37:38.322 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-08-31T07:37:38.322 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-08-31T07:37:38.322 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-08-31T07:37:38.322 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 1411, 2024-08-31T07:37:38.322 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 0, 2024-08-31T07:37:38.323 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 459 2024-08-31T07:37:38.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.323 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-08-31T07:37:38.323 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 1, 2024-08-31T07:37:38.323 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-08-31T07:37:38.324 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-08-31T07:37:38.324 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-08-31T07:37:38.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.324 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-08-31T07:37:38.324 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-08-31T07:37:38.324 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 6, 2024-08-31T07:37:38.324 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-08-31T07:37:38.324 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 6, 2024-08-31T07:37:38.325 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 15283, 2024-08-31T07:37:38.325 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 15283, 2024-08-31T07:37:38.325 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-08-31T07:37:38.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.325 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:38.325 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:38.325 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-08-31T07:37:38.325 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:38.326 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 436409, 2024-08-31T07:37:38.326 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 118044899, 2024-08-31T07:37:38.326 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:38.326 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 436409, 2024-08-31T07:37:38.326 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:38.326 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:38.326 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 436409, 2024-08-31T07:37:38.327 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 118044899, 2024-08-31T07:37:38.327 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:38.327 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.328 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.328 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.328 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:38.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.328 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-08-31T07:37:38.328 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:38.328 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-08-31T07:37:38.329 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:38.329 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:38.329 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:38.329 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:38.329 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:38.329 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 45478, 2024-08-31T07:37:38.329 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 410357794, 2024-08-31T07:37:38.329 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 45478, 2024-08-31T07:37:38.330 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 410357794, 2024-08-31T07:37:38.330 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:38.330 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.330 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.330 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.330 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:38.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.330 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-08-31T07:37:38.330 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:38.331 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-08-31T07:37:38.331 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:38.331 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:38.331 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:38.331 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:38.331 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:38.331 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 45478, 2024-08-31T07:37:38.331 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 45478, 2024-08-31T07:37:38.331 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 45478, 2024-08-31T07:37:38.332 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 45478, 2024-08-31T07:37:38.332 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:38.332 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.332 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.332 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.332 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:38.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.332 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-08-31T07:37:38.333 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:38.333 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-08-31T07:37:38.333 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:38.333 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 15283, 2024-08-31T07:37:38.333 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 1545807, 2024-08-31T07:37:38.333 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:38.333 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 15283, 2024-08-31T07:37:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 778, 2024-08-31T07:37:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 1545807, 2024-08-31T07:37:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x55d4f68e0820": { 2024-08-31T07:37:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 36539, 2024-08-31T07:37:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-08-31T07:37:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 84368, 2024-08-31T07:37:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 404992835, 2024-08-31T07:37:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 84368, 2024-08-31T07:37:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 25716, 2024-08-31T07:37:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 404956296, 2024-08-31T07:37:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "trackedop": { 2024-08-31T07:37:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 2060 2024-08-31T07:37:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:38.337 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-08-31T07:37:38.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:38 smithi073 ceph-mon[21995]: pgmap v1152: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 2.0 MiB/s wr, 69 op/s 2024-08-31T07:37:38.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:38 smithi084 ceph-mon[22387]: pgmap v1152: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 2.0 MiB/s wr, 69 op/s 2024-08-31T07:37:38.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:38 smithi032 ceph-mon[17252]: pgmap v1152: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 2.0 MiB/s wr, 69 op/s 2024-08-31T07:37:38.964 INFO:tasks.check_counter:Daemon mds.h mds.exported=6 2024-08-31T07:37:38.964 INFO:tasks.check_counter:Daemon mds.h mds.imported=9 2024-08-31T07:37:38.964 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-08-31T07:37:38.965 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-08-31T07:37:38.965 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T07:37:38.965 DEBUG:tasks.check_counter:Getting stats from k 2024-08-31T07:37:38.965 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-31T07:37:39.212 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T07:37:39.788 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-08-31T07:37:39.788 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:39.788 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 3158, 2024-08-31T07:37:39.788 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 3158, 2024-08-31T07:37:39.788 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 143574841, 2024-08-31T07:37:39.789 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 1003421, 2024-08-31T07:37:39.789 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 8, 2024-08-31T07:37:39.789 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-08-31T07:37:39.789 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.425467608, 2024-08-31T07:37:39.789 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.158599620, 2024-08-31T07:37:39.789 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.156045712, 2024-08-31T07:37:39.789 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.084420430, 2024-08-31T07:37:39.789 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:39.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 3157, 2024-08-31T07:37:39.790 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.122918349, 2024-08-31T07:37:39.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000038935 2024-08-31T07:37:39.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.790 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:39.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.790 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.791 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-31T07:37:39.791 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 448 2024-08-31T07:37:39.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.791 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:39.791 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 2398, 2024-08-31T07:37:39.791 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2324, 2024-08-31T07:37:39.791 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 144492686, 2024-08-31T07:37:39.791 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 869522, 2024-08-31T07:37:39.792 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 6, 2024-08-31T07:37:39.792 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-08-31T07:37:39.792 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.339479257, 2024-08-31T07:37:39.792 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.103051179, 2024-08-31T07:37:39.792 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.188668297, 2024-08-31T07:37:39.793 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.048928476, 2024-08-31T07:37:39.793 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:39.793 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2323, 2024-08-31T07:37:39.793 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.085249866, 2024-08-31T07:37:39.793 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000036698 2024-08-31T07:37:39.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.793 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:39.793 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.794 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.794 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.794 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 1404608, 2024-08-31T07:37:39.794 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 289104 2024-08-31T07:37:39.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.794 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:39.795 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 968, 2024-08-31T07:37:39.795 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 1314, 2024-08-31T07:37:39.795 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 120026133, 2024-08-31T07:37:39.795 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 1005234, 2024-08-31T07:37:39.795 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 6, 2024-08-31T07:37:39.796 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-08-31T07:37:39.796 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.178792073, 2024-08-31T07:37:39.796 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.054359453, 2024-08-31T07:37:39.796 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.089609719, 2024-08-31T07:37:39.796 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.025228882, 2024-08-31T07:37:39.796 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:39.796 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1312, 2024-08-31T07:37:39.797 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.048785020, 2024-08-31T07:37:39.797 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000037183 2024-08-31T07:37:39.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.797 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:39.797 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.797 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.797 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.798 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-31T07:37:39.798 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 697728 2024-08-31T07:37:39.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.798 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-08-31T07:37:39.798 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-08-31T07:37:39.798 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-08-31T07:37:39.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.798 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-08-31T07:37:39.799 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-08-31T07:37:39.799 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-08-31T07:37:39.799 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 9861, 2024-08-31T07:37:39.799 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.234144655, 2024-08-31T07:37:39.799 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000023744 2024-08-31T07:37:39.799 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:39.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.799 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-08-31T07:37:39.800 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-08-31T07:37:39.800 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-08-31T07:37:39.800 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.800 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.800 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.800 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:39.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.800 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-08-31T07:37:39.801 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 0, 2024-08-31T07:37:39.801 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 0, 2024-08-31T07:37:39.801 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-08-31T07:37:39.801 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.801 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.801 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.801 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-08-31T07:37:39.802 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-08-31T07:37:39.802 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-08-31T07:37:39.802 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-08-31T07:37:39.802 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 0, 2024-08-31T07:37:39.802 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-08-31T07:37:39.802 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-08-31T07:37:39.802 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 16810, 2024-08-31T07:37:39.802 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 16, 2024-08-31T07:37:39.803 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 16794, 2024-08-31T07:37:39.803 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-08-31T07:37:39.803 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 16788, 2024-08-31T07:37:39.803 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 0, 2024-08-31T07:37:39.803 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-08-31T07:37:39.803 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-08-31T07:37:39.803 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 14, 2024-08-31T07:37:39.803 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 0, 2024-08-31T07:37:39.804 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 0, 2024-08-31T07:37:39.804 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-08-31T07:37:39.804 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 0, 2024-08-31T07:37:39.804 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-08-31T07:37:39.804 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-08-31T07:37:39.804 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-08-31T07:37:39.804 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-08-31T07:37:39.804 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-08-31T07:37:39.805 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-08-31T07:37:39.805 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-08-31T07:37:39.805 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-08-31T07:37:39.805 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-08-31T07:37:39.805 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-08-31T07:37:39.805 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-08-31T07:37:39.805 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-08-31T07:37:39.805 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 0, 2024-08-31T07:37:39.805 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 0, 2024-08-31T07:37:39.806 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-08-31T07:37:39.806 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-08-31T07:37:39.806 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-08-31T07:37:39.806 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T07:37:39.806 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-08-31T07:37:39.806 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-08-31T07:37:39.807 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T07:37:39.807 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-08-31T07:37:39.807 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-08-31T07:37:39.807 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-08-31T07:37:39.807 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 0, 2024-08-31T07:37:39.807 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 0, 2024-08-31T07:37:39.807 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T07:37:39.807 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-31T07:37:39.807 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T07:37:39.808 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 0, 2024-08-31T07:37:39.808 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-08-31T07:37:39.808 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 0, 2024-08-31T07:37:39.808 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 0 2024-08-31T07:37:39.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.808 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-08-31T07:37:39.808 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-08-31T07:37:39.808 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-08-31T07:37:39.809 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-08-31T07:37:39.809 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 0, 2024-08-31T07:37:39.809 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-08-31T07:37:39.809 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-08-31T07:37:39.809 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-08-31T07:37:39.809 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-08-31T07:37:39.809 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-08-31T07:37:39.809 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-08-31T07:37:39.810 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-08-31T07:37:39.810 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-08-31T07:37:39.810 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-08-31T07:37:39.810 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-08-31T07:37:39.810 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-08-31T07:37:39.810 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-08-31T07:37:39.810 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-08-31T07:37:39.810 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 0, 2024-08-31T07:37:39.811 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 0, 2024-08-31T07:37:39.811 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T07:37:39.811 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-08-31T07:37:39.812 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-08-31T07:37:39.812 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-08-31T07:37:39.812 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-08-31T07:37:39.812 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0, 2024-08-31T07:37:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_started": 0, 2024-08-31T07:37:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_succeeded": 0, 2024-08-31T07:37:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "uninline_write_failed": 0 2024-08-31T07:37:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-08-31T07:37:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-08-31T07:37:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 0, 2024-08-31T07:37:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-08-31T07:37:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-08-31T07:37:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 84307, 2024-08-31T07:37:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-08-31T07:37:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-08-31T07:37:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 0, 2024-08-31T07:37:39.815 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-08-31T07:37:39.816 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-08-31T07:37:39.816 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 133, 2024-08-31T07:37:39.816 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 8, 2024-08-31T07:37:39.816 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-08-31T07:37:39.817 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-08-31T07:37:39.817 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 4194352, 2024-08-31T07:37:39.817 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 409150600, 2024-08-31T07:37:39.817 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 409137160, 2024-08-31T07:37:39.818 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-08-31T07:37:39.818 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.818 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.818 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.819 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 84307 2024-08-31T07:37:39.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.819 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-08-31T07:37:39.819 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1440, 2024-08-31T07:37:39.819 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 16898, 2024-08-31T07:37:39.819 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 15458, 2024-08-31T07:37:39.820 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 73, 2024-08-31T07:37:39.820 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 109, 2024-08-31T07:37:39.820 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 36, 2024-08-31T07:37:39.820 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 16810, 2024-08-31T07:37:39.820 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 17094, 2024-08-31T07:37:39.820 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 284, 2024-08-31T07:37:39.820 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-08-31T07:37:39.820 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 0, 2024-08-31T07:37:39.821 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 0, 2024-08-31T07:37:39.821 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 140436, 2024-08-31T07:37:39.821 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 352346112 2024-08-31T07:37:39.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.821 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-08-31T07:37:39.821 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 0, 2024-08-31T07:37:39.821 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-08-31T07:37:39.821 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 0, 2024-08-31T07:37:39.822 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 0, 2024-08-31T07:37:39.822 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-08-31T07:37:39.822 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-08-31T07:37:39.822 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.822 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.822 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.822 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-08-31T07:37:39.823 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.823 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.823 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.823 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-08-31T07:37:39.823 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.823 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.823 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.824 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-08-31T07:37:39.824 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.824 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.824 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.824 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-08-31T07:37:39.824 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.825 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.825 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.825 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-08-31T07:37:39.825 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.825 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.825 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.826 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-08-31T07:37:39.826 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.826 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.826 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.826 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-08-31T07:37:39.827 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.827 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.827 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.827 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-08-31T07:37:39.827 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.827 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.828 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.828 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-08-31T07:37:39.828 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.828 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.828 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.828 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-08-31T07:37:39.829 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.829 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.830 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.830 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-08-31T07:37:39.830 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.830 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.830 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.830 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-08-31T07:37:39.831 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.831 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.831 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.831 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-08-31T07:37:39.831 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.831 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.831 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.832 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-08-31T07:37:39.832 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.832 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.832 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.832 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-08-31T07:37:39.832 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.832 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.833 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.833 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-08-31T07:37:39.833 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.833 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.833 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.833 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-08-31T07:37:39.834 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.834 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.834 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.834 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-08-31T07:37:39.834 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.834 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.834 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.835 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-08-31T07:37:39.835 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.835 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.835 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.835 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-08-31T07:37:39.835 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.836 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.836 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.836 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-08-31T07:37:39.836 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.836 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.836 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.837 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-08-31T07:37:39.837 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.837 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.837 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.837 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-08-31T07:37:39.837 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.837 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.838 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.838 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-08-31T07:37:39.838 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.838 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.838 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.838 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-08-31T07:37:39.838 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.839 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.839 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.839 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-08-31T07:37:39.839 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.839 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.839 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.840 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-08-31T07:37:39.840 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.840 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.840 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.840 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-08-31T07:37:39.840 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.840 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.841 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.841 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-08-31T07:37:39.841 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-08-31T07:37:39.841 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-08-31T07:37:39.841 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.841 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.841 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.842 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:39.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.842 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-08-31T07:37:39.842 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-08-31T07:37:39.842 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 3, 2024-08-31T07:37:39.842 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 1, 2024-08-31T07:37:39.842 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-08-31T07:37:39.842 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-08-31T07:37:39.843 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 0, 2024-08-31T07:37:39.843 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 0, 2024-08-31T07:37:39.843 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 2796, 2024-08-31T07:37:39.843 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-08-31T07:37:39.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.843 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-08-31T07:37:39.844 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:39.844 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:39.844 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:39.844 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:39.844 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:39.844 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:39.844 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:39.845 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:39.845 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:39.845 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:39.845 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:39.845 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:39.845 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:39.845 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:39.845 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:39.846 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:39.846 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:39.846 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:39.846 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:39.846 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:39.846 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:39.846 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:39.846 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:39.847 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:39.847 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:39.847 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:39.847 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:39.847 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:39.847 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:39.847 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:39.847 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:39.848 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-08-31T07:37:39.848 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:39.848 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:39.848 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:39.848 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:39.848 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 304530, 2024-08-31T07:37:39.849 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 1781, 2024-08-31T07:37:39.849 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-08-31T07:37:39.849 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-08-31T07:37:39.849 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-08-31T07:37:39.850 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-08-31T07:37:39.850 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:39.850 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:39.850 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:39.851 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:39.851 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 7968, 2024-08-31T07:37:39.851 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 176, 2024-08-31T07:37:39.851 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:39.851 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:39.852 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:39.852 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-08-31T07:37:39.852 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 15239664, 2024-08-31T07:37:39.853 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 113383, 2024-08-31T07:37:39.853 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:39.853 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-08-31T07:37:39.853 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:39.854 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-08-31T07:37:39.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.854 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-08-31T07:37:39.854 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-08-31T07:37:39.854 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-08-31T07:37:39.855 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 5946, 2024-08-31T07:37:39.855 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 21, 2024-08-31T07:37:39.855 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 3, 2024-08-31T07:37:39.855 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 5946, 2024-08-31T07:37:39.855 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-08-31T07:37:39.855 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5946, 2024-08-31T07:37:39.855 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 7.825809297, 2024-08-31T07:37:39.855 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001316146 2024-08-31T07:37:39.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.856 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-08-31T07:37:39.856 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-08-31T07:37:39.856 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5946, 2024-08-31T07:37:39.856 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 5947 2024-08-31T07:37:39.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.856 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 5946, 2024-08-31T07:37:39.856 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 5946, 2024-08-31T07:37:39.856 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 0, 2024-08-31T07:37:39.857 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-08-31T07:37:39.857 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-08-31T07:37:39.857 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 3458, 2024-08-31T07:37:39.857 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 0, 2024-08-31T07:37:39.857 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 2486, 2024-08-31T07:37:39.857 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 0, 2024-08-31T07:37:39.857 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 0, 2024-08-31T07:37:39.857 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-08-31T07:37:39.857 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-08-31T07:37:39.858 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-08-31T07:37:39.858 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-08-31T07:37:39.858 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 0, 2024-08-31T07:37:39.858 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-08-31T07:37:39.858 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-08-31T07:37:39.858 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-08-31T07:37:39.858 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 1, 2024-08-31T07:37:39.858 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 0, 2024-08-31T07:37:39.858 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-08-31T07:37:39.859 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-08-31T07:37:39.859 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-08-31T07:37:39.859 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-08-31T07:37:39.859 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-08-31T07:37:39.859 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-08-31T07:37:39.859 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T07:37:39.859 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-08-31T07:37:39.859 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-08-31T07:37:39.860 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-08-31T07:37:39.860 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-08-31T07:37:39.860 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-08-31T07:37:39.860 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-08-31T07:37:39.860 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-08-31T07:37:39.860 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-08-31T07:37:39.860 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-08-31T07:37:39.860 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-08-31T07:37:39.861 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-08-31T07:37:39.861 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-08-31T07:37:39.861 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-08-31T07:37:39.861 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-08-31T07:37:39.861 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-08-31T07:37:39.861 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-08-31T07:37:39.861 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-08-31T07:37:39.861 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-08-31T07:37:39.862 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-08-31T07:37:39.862 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 118, 2024-08-31T07:37:39.862 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-08-31T07:37:39.862 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 40, 2024-08-31T07:37:39.862 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 11, 2024-08-31T07:37:39.862 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 11, 2024-08-31T07:37:39.862 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-08-31T07:37:39.862 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-08-31T07:37:39.862 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 0, 2024-08-31T07:37:39.863 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 2, 2024-08-31T07:37:39.863 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 0 2024-08-31T07:37:39.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.863 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-08-31T07:37:39.863 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 0, 2024-08-31T07:37:39.863 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-08-31T07:37:39.863 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-08-31T07:37:39.863 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-08-31T07:37:39.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.864 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-08-31T07:37:39.864 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-08-31T07:37:39.864 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-08-31T07:37:39.864 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-08-31T07:37:39.864 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-08-31T07:37:39.864 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 0, 2024-08-31T07:37:39.864 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-08-31T07:37:39.865 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-08-31T07:37:39.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.865 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:39.865 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:39.865 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-08-31T07:37:39.865 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:39.865 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 6520, 2024-08-31T07:37:39.865 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 407474910, 2024-08-31T07:37:39.865 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:39.866 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 6520, 2024-08-31T07:37:39.866 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:39.866 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:39.866 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6520, 2024-08-31T07:37:39.866 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 407474910, 2024-08-31T07:37:39.866 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:39.866 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.866 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.867 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.867 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:39.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.867 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-08-31T07:37:39.867 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:39.867 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-08-31T07:37:39.867 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:39.867 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:39.868 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:39.868 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:39.868 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:39.868 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 5946, 2024-08-31T07:37:39.868 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 405037488, 2024-08-31T07:37:39.868 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 5946, 2024-08-31T07:37:39.868 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 405037488, 2024-08-31T07:37:39.868 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:39.868 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.869 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.869 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:39.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.869 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-08-31T07:37:39.869 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:39.869 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-08-31T07:37:39.869 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:39.870 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:39.870 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:39.870 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:39.870 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:39.870 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 5946, 2024-08-31T07:37:39.870 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 5946, 2024-08-31T07:37:39.870 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 5946, 2024-08-31T07:37:39.870 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 5946, 2024-08-31T07:37:39.870 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:39.871 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.871 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.871 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.871 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:39.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.871 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-08-31T07:37:39.871 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:39.871 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-08-31T07:37:39.872 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:39.872 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:39.872 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:39.872 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:39.872 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:39.872 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:39.872 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:39.872 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-08-31T07:37:39.872 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-08-31T07:37:39.873 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:39.873 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.873 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.873 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.873 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:39.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.873 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x564ce0242c30": { 2024-08-31T07:37:39.873 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-08-31T07:37:39.874 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-08-31T07:37:39.874 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-08-31T07:37:39.874 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-08-31T07:37:39.874 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-08-31T07:37:39.874 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:39.874 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:39.874 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-08-31T07:37:39.874 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-08-31T07:37:39.874 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-08-31T07:37:39.875 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-08-31T07:37:39.875 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-08-31T07:37:39.875 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-08-31T07:37:39.875 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-08-31T07:37:39.875 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-08-31T07:37:39.875 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:39.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-08-31T07:37:39.875 INFO:teuthology.orchestra.run.smithi073.stdout: "trackedop": { 2024-08-31T07:37:39.876 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 0 2024-08-31T07:37:39.876 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-08-31T07:37:39.876 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-08-31T07:37:40.556 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:40 smithi073 ceph-mon[21995]: pgmap v1153: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-08-31T07:37:40.584 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-08-31T07:37:40.584 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-08-31T07:37:40.584 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-08-31T07:37:40.584 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-08-31T07:37:40.584 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T07:37:40.584 DEBUG:tasks.check_counter:Getting stats from c 2024-08-31T07:37:40.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-31T07:37:40.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:40 smithi084 ceph-mon[22387]: pgmap v1153: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-08-31T07:37:40.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:40 smithi032 ceph-mon[17252]: pgmap v1153: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-08-31T07:37:40.892 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:37:41.602 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-08-31T07:37:41.603 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:41.603 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 112646, 2024-08-31T07:37:41.603 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 112993, 2024-08-31T07:37:41.603 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 49773721, 2024-08-31T07:37:41.603 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 171174422, 2024-08-31T07:37:41.603 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 15, 2024-08-31T07:37:41.603 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 5, 2024-08-31T07:37:41.604 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 7.939864561, 2024-08-31T07:37:41.604 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 3.880327545, 2024-08-31T07:37:41.604 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 4.249183332, 2024-08-31T07:37:41.604 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.533048766, 2024-08-31T07:37:41.604 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:41.604 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 112992, 2024-08-31T07:37:41.604 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 6.344438762, 2024-08-31T07:37:41.605 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000056149 2024-08-31T07:37:41.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.605 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:41.605 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 186725, 2024-08-31T07:37:41.605 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.036977382, 2024-08-31T07:37:41.606 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000198 2024-08-31T07:37:41.606 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.606 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 3872, 2024-08-31T07:37:41.606 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 1184 2024-08-31T07:37:41.606 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.606 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:41.607 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 259746, 2024-08-31T07:37:41.607 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 344225, 2024-08-31T07:37:41.607 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 89306662, 2024-08-31T07:37:41.607 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 352558334, 2024-08-31T07:37:41.608 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 15, 2024-08-31T07:37:41.608 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 7, 2024-08-31T07:37:41.608 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 19.625221407, 2024-08-31T07:37:41.608 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 11.376974613, 2024-08-31T07:37:41.609 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 9.902594509, 2024-08-31T07:37:41.609 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.601981892, 2024-08-31T07:37:41.609 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:41.609 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 344224, 2024-08-31T07:37:41.610 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 15.775359913, 2024-08-31T07:37:41.610 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000045828 2024-08-31T07:37:41.610 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.610 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:41.610 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 362301, 2024-08-31T07:37:41.611 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.094658423, 2024-08-31T07:37:41.611 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000261 2024-08-31T07:37:41.611 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.611 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1423920, 2024-08-31T07:37:41.611 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 315392 2024-08-31T07:37:41.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.612 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:41.612 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 46039, 2024-08-31T07:37:41.612 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 55792, 2024-08-31T07:37:41.612 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 14222683, 2024-08-31T07:37:41.613 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 172378625, 2024-08-31T07:37:41.613 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 59, 2024-08-31T07:37:41.613 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 6, 2024-08-31T07:37:41.613 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 5.463481473, 2024-08-31T07:37:41.613 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 2.949026398, 2024-08-31T07:37:41.614 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 1.881837655, 2024-08-31T07:37:41.614 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.759500763, 2024-08-31T07:37:41.614 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:41.614 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 55790, 2024-08-31T07:37:41.614 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 2.839396142, 2024-08-31T07:37:41.615 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000050894 2024-08-31T07:37:41.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.615 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:41.615 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 69073, 2024-08-31T07:37:41.615 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.015835322, 2024-08-31T07:37:41.615 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000229 2024-08-31T07:37:41.616 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.616 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-31T07:37:41.616 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 699488 2024-08-31T07:37:41.616 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.617 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-08-31T07:37:41.617 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-08-31T07:37:41.617 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-08-31T07:37:41.617 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.617 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-08-31T07:37:41.617 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-08-31T07:37:41.618 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-08-31T07:37:41.618 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 44719, 2024-08-31T07:37:41.618 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 57.215287477, 2024-08-31T07:37:41.618 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001279440 2024-08-31T07:37:41.618 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:41.619 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.619 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-08-31T07:37:41.619 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-08-31T07:37:41.619 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-08-31T07:37:41.619 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 32422, 2024-08-31T07:37:41.619 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 2.837692903, 2024-08-31T07:37:41.620 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000087523 2024-08-31T07:37:41.620 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:41.620 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.620 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-08-31T07:37:41.620 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 110744, 2024-08-31T07:37:41.621 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 109832, 2024-08-31T07:37:41.621 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-08-31T07:37:41.621 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 109832, 2024-08-31T07:37:41.621 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 813.730104979, 2024-08-31T07:37:41.621 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.007408861 2024-08-31T07:37:41.621 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.621 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-08-31T07:37:41.621 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 912, 2024-08-31T07:37:41.621 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 2, 2024-08-31T07:37:41.622 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-08-31T07:37:41.622 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 56, 2024-08-31T07:37:41.622 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 5, 2024-08-31T07:37:41.622 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 1, 2024-08-31T07:37:41.622 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 1872, 2024-08-31T07:37:41.622 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 327, 2024-08-31T07:37:41.622 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 1502, 2024-08-31T07:37:41.622 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 43, 2024-08-31T07:37:41.623 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 1405, 2024-08-31T07:37:41.623 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 739, 2024-08-31T07:37:41.623 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 135, 2024-08-31T07:37:41.623 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 135, 2024-08-31T07:37:41.623 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 20, 2024-08-31T07:37:41.623 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 161941, 2024-08-31T07:37:41.623 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 96464, 2024-08-31T07:37:41.623 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 908, 2024-08-31T07:37:41.624 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 50, 2024-08-31T07:37:41.624 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 0, 2024-08-31T07:37:41.624 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-08-31T07:37:41.624 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 515, 2024-08-31T07:37:41.624 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 12484, 2024-08-31T07:37:41.624 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-08-31T07:37:41.624 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 8, 2024-08-31T07:37:41.624 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 484, 2024-08-31T07:37:41.625 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 11, 2024-08-31T07:37:41.625 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 722, 2024-08-31T07:37:41.625 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-08-31T07:37:41.625 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 1600, 2024-08-31T07:37:41.625 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-08-31T07:37:41.625 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 745, 2024-08-31T07:37:41.625 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 212490313, 2024-08-31T07:37:41.625 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 6, 2024-08-31T07:37:41.625 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 1600, 2024-08-31T07:37:41.626 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-08-31T07:37:41.626 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T07:37:41.626 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-08-31T07:37:41.626 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 180, 2024-08-31T07:37:41.626 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T07:37:41.626 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 180, 2024-08-31T07:37:41.626 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 1420, 2024-08-31T07:37:41.626 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 6, 2024-08-31T07:37:41.627 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 12971, 2024-08-31T07:37:41.627 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 73970, 2024-08-31T07:37:41.627 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T07:37:41.627 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 2, 2024-08-31T07:37:41.627 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T07:37:41.627 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 41200, 2024-08-31T07:37:41.627 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 11503, 2024-08-31T07:37:41.627 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 1968, 2024-08-31T07:37:41.628 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 81795 2024-08-31T07:37:41.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.628 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-08-31T07:37:41.628 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 34, 2024-08-31T07:37:41.628 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 75, 2024-08-31T07:37:41.628 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 55, 2024-08-31T07:37:41.629 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 86, 2024-08-31T07:37:41.629 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 54, 2024-08-31T07:37:41.629 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-08-31T07:37:41.629 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-08-31T07:37:41.629 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-08-31T07:37:41.629 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 16824, 2024-08-31T07:37:41.630 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 16824, 2024-08-31T07:37:41.630 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-08-31T07:37:41.630 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-08-31T07:37:41.630 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-08-31T07:37:41.630 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-08-31T07:37:41.630 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-08-31T07:37:41.630 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-08-31T07:37:41.631 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-08-31T07:37:41.631 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_path": 19, 2024-08-31T07:37:41.631 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_inode": 1828, 2024-08-31T07:37:41.631 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T07:37:41.631 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 13, 2024-08-31T07:37:41.631 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-08-31T07:37:41.631 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 8, 2024-08-31T07:37:41.632 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-08-31T07:37:41.632 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0, 2024-08-31T07:37:41.632 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_started": 0, 2024-08-31T07:37:41.632 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_succeeded": 0, 2024-08-31T07:37:41.632 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_write_failed": 0 2024-08-31T07:37:41.632 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.632 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-08-31T07:37:41.633 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-08-31T07:37:41.633 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 83536, 2024-08-31T07:37:41.633 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-08-31T07:37:41.633 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-08-31T07:37:41.633 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 83536, 2024-08-31T07:37:41.633 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-08-31T07:37:41.633 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-08-31T07:37:41.634 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 125, 2024-08-31T07:37:41.634 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-08-31T07:37:41.634 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-08-31T07:37:41.634 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 125, 2024-08-31T07:37:41.634 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 8, 2024-08-31T07:37:41.634 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-08-31T07:37:41.634 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-08-31T07:37:41.634 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 4194304, 2024-08-31T07:37:41.634 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 396920203, 2024-08-31T07:37:41.635 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 0, 2024-08-31T07:37:41.635 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-08-31T07:37:41.635 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 24176, 2024-08-31T07:37:41.635 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 213.722425932, 2024-08-31T07:37:41.635 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.008840272 2024-08-31T07:37:41.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.635 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 0 2024-08-31T07:37:41.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.636 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-08-31T07:37:41.636 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 1624, 2024-08-31T07:37:41.636 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 18903, 2024-08-31T07:37:41.636 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 17279, 2024-08-31T07:37:41.636 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 169, 2024-08-31T07:37:41.636 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 225, 2024-08-31T07:37:41.636 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 56, 2024-08-31T07:37:41.636 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 1832, 2024-08-31T07:37:41.637 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 34871, 2024-08-31T07:37:41.637 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 33039, 2024-08-31T07:37:41.637 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 135, 2024-08-31T07:37:41.637 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 18319, 2024-08-31T07:37:41.637 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 18184, 2024-08-31T07:37:41.637 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 148116, 2024-08-31T07:37:41.637 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 360587264 2024-08-31T07:37:41.638 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.638 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-08-31T07:37:41.638 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 160864, 2024-08-31T07:37:41.638 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 363, 2024-08-31T07:37:41.638 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 110744, 2024-08-31T07:37:41.638 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 1903, 2024-08-31T07:37:41.638 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 5495, 2024-08-31T07:37:41.638 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-08-31T07:37:41.639 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 16803, 2024-08-31T07:37:41.639 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 375.712615961, 2024-08-31T07:37:41.639 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.022359853 2024-08-31T07:37:41.639 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.639 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-08-31T07:37:41.639 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 834, 2024-08-31T07:37:41.639 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 12.130894012, 2024-08-31T07:37:41.639 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.014545436 2024-08-31T07:37:41.640 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.640 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-08-31T07:37:41.640 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.640 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.640 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.640 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.640 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-08-31T07:37:41.641 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.641 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.641 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.641 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-08-31T07:37:41.641 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 25035, 2024-08-31T07:37:41.641 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 40.480351362, 2024-08-31T07:37:41.642 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001616950 2024-08-31T07:37:41.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.642 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-08-31T07:37:41.642 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.642 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.642 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.643 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-08-31T07:37:41.643 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.643 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.643 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.643 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-08-31T07:37:41.643 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.643 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.644 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.644 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.644 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-08-31T07:37:41.644 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.644 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.645 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.645 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.645 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-08-31T07:37:41.645 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.645 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.646 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.646 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.646 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-08-31T07:37:41.646 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.646 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.646 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.646 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.647 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-08-31T07:37:41.647 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.647 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.647 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.647 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.647 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-08-31T07:37:41.647 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.648 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.648 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.648 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.648 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-08-31T07:37:41.648 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.648 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.648 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.648 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.649 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-08-31T07:37:41.649 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.649 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.649 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.649 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.649 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-08-31T07:37:41.650 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 47073, 2024-08-31T07:37:41.650 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 210.513965806, 2024-08-31T07:37:41.650 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.004472074 2024-08-31T07:37:41.650 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.650 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-08-31T07:37:41.650 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2756, 2024-08-31T07:37:41.650 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 41.010963284, 2024-08-31T07:37:41.650 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.014880610 2024-08-31T07:37:41.650 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.651 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-08-31T07:37:41.651 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.651 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.651 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.651 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.651 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-08-31T07:37:41.651 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.651 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.652 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.652 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.652 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-08-31T07:37:41.652 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.652 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.652 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.652 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.652 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-08-31T07:37:41.653 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.653 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.653 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.653 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.653 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-08-31T07:37:41.653 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.653 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.653 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.654 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.654 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-08-31T07:37:41.654 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 67, 2024-08-31T07:37:41.654 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.921641220, 2024-08-31T07:37:41.654 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.013755839 2024-08-31T07:37:41.654 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.654 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-08-31T07:37:41.655 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.655 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.655 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.655 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.655 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-08-31T07:37:41.655 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 440, 2024-08-31T07:37:41.655 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.438482237, 2024-08-31T07:37:41.655 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000996550 2024-08-31T07:37:41.656 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.656 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-08-31T07:37:41.656 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.656 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.656 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.656 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.656 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-08-31T07:37:41.656 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.657 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.657 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.657 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.657 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-08-31T07:37:41.657 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.657 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.657 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.657 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.658 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-08-31T07:37:41.658 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 16824, 2024-08-31T07:37:41.658 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 132.521191097, 2024-08-31T07:37:41.658 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.007876913 2024-08-31T07:37:41.658 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.658 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-08-31T07:37:41.659 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-08-31T07:37:41.659 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-08-31T07:37:41.659 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.659 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.659 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.659 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:41.659 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.659 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-08-31T07:37:41.659 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 1, 2024-08-31T07:37:41.660 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 2, 2024-08-31T07:37:41.660 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 1, 2024-08-31T07:37:41.660 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 1, 2024-08-31T07:37:41.660 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-08-31T07:37:41.660 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 13139, 2024-08-31T07:37:41.660 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 13139, 2024-08-31T07:37:41.660 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 2797, 2024-08-31T07:37:41.660 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-08-31T07:37:41.661 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.661 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-08-31T07:37:41.661 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:41.661 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:41.661 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:41.661 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:41.661 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:41.661 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:41.661 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:41.662 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:41.662 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:41.662 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:41.662 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:41.662 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:41.663 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:41.663 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:41.663 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:41.663 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:41.663 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:41.663 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:41.663 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:41.664 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:41.664 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:41.664 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:41.664 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:41.664 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:41.664 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:41.665 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:41.665 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:41.665 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:41.665 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:41.665 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:41.665 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:41.666 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-08-31T07:37:41.666 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:41.666 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:41.666 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:41.666 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:41.666 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 1749543, 2024-08-31T07:37:41.666 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 1172, 2024-08-31T07:37:41.666 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 88, 2024-08-31T07:37:41.666 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 1, 2024-08-31T07:37:41.667 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-08-31T07:37:41.667 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-08-31T07:37:41.667 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:41.667 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:41.667 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:41.667 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:41.667 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 7968, 2024-08-31T07:37:41.668 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 176, 2024-08-31T07:37:41.668 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:41.668 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:41.668 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:41.668 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-08-31T07:37:41.668 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 5997586, 2024-08-31T07:37:41.669 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 54315, 2024-08-31T07:37:41.669 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:41.669 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-08-31T07:37:41.669 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:41.670 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-08-31T07:37:41.670 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.670 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-08-31T07:37:41.670 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-08-31T07:37:41.671 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-08-31T07:37:41.671 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 45463, 2024-08-31T07:37:41.671 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 396776252, 2024-08-31T07:37:41.671 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 33, 2024-08-31T07:37:41.672 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 45463, 2024-08-31T07:37:41.672 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-08-31T07:37:41.672 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 45463, 2024-08-31T07:37:41.672 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 318.923297889, 2024-08-31T07:37:41.673 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.007015007 2024-08-31T07:37:41.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.674 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-08-31T07:37:41.674 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-08-31T07:37:41.674 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 45463, 2024-08-31T07:37:41.674 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 46527 2024-08-31T07:37:41.675 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.675 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 45463, 2024-08-31T07:37:41.675 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 2362, 2024-08-31T07:37:41.675 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 43101, 2024-08-31T07:37:41.676 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-08-31T07:37:41.676 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-08-31T07:37:41.676 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 0, 2024-08-31T07:37:41.676 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 10, 2024-08-31T07:37:41.677 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 759, 2024-08-31T07:37:41.677 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 24935, 2024-08-31T07:37:41.677 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 471, 2024-08-31T07:37:41.677 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-08-31T07:37:41.678 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-08-31T07:37:41.678 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-08-31T07:37:41.678 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 5, 2024-08-31T07:37:41.678 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 16937, 2024-08-31T07:37:41.679 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-08-31T07:37:41.679 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-08-31T07:37:41.679 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-08-31T07:37:41.680 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 1600, 2024-08-31T07:37:41.680 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 20, 2024-08-31T07:37:41.680 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-08-31T07:37:41.680 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-08-31T07:37:41.680 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-08-31T07:37:41.681 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-08-31T07:37:41.681 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-08-31T07:37:41.681 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-08-31T07:37:41.681 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T07:37:41.681 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-08-31T07:37:41.681 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-08-31T07:37:41.681 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-08-31T07:37:41.682 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-08-31T07:37:41.682 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-08-31T07:37:41.682 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-08-31T07:37:41.682 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-08-31T07:37:41.682 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-08-31T07:37:41.682 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-08-31T07:37:41.682 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-08-31T07:37:41.682 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-08-31T07:37:41.682 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-08-31T07:37:41.683 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-08-31T07:37:41.683 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-08-31T07:37:41.683 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-08-31T07:37:41.683 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-08-31T07:37:41.683 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-08-31T07:37:41.683 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-08-31T07:37:41.683 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-08-31T07:37:41.683 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 118, 2024-08-31T07:37:41.684 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-08-31T07:37:41.684 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 40, 2024-08-31T07:37:41.684 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 12, 2024-08-31T07:37:41.684 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 12, 2024-08-31T07:37:41.684 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 0, 2024-08-31T07:37:41.684 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-08-31T07:37:41.684 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 1317, 2024-08-31T07:37:41.684 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 5, 2024-08-31T07:37:41.685 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 468 2024-08-31T07:37:41.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.685 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-08-31T07:37:41.685 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 1, 2024-08-31T07:37:41.685 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-08-31T07:37:41.685 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-08-31T07:37:41.685 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-08-31T07:37:41.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.685 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-08-31T07:37:41.686 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-08-31T07:37:41.686 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 6, 2024-08-31T07:37:41.686 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-08-31T07:37:41.686 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 6, 2024-08-31T07:37:41.686 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 16824, 2024-08-31T07:37:41.686 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 16824, 2024-08-31T07:37:41.686 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-08-31T07:37:41.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.687 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:41.687 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:41.687 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-08-31T07:37:41.687 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:41.687 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 418427, 2024-08-31T07:37:41.687 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 118156222, 2024-08-31T07:37:41.687 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:41.687 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 418427, 2024-08-31T07:37:41.687 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:41.688 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:41.688 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 418427, 2024-08-31T07:37:41.688 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 118156222, 2024-08-31T07:37:41.688 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:41.688 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.688 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.688 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.689 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:41.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.689 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-08-31T07:37:41.689 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:41.689 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-08-31T07:37:41.689 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:41.689 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:41.689 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:41.690 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:41.690 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:41.690 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 45463, 2024-08-31T07:37:41.690 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 398478460, 2024-08-31T07:37:41.690 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 45463, 2024-08-31T07:37:41.690 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 398478460, 2024-08-31T07:37:41.690 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:41.690 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.690 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.691 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.691 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:41.691 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.691 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-08-31T07:37:41.691 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:41.691 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-08-31T07:37:41.691 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:41.692 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:41.692 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:41.692 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:41.692 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:41.692 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 45463, 2024-08-31T07:37:41.692 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 45463, 2024-08-31T07:37:41.692 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 45463, 2024-08-31T07:37:41.692 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 45463, 2024-08-31T07:37:41.692 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:41.693 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.693 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.693 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.693 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:41.693 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.693 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-08-31T07:37:41.693 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:41.694 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-08-31T07:37:41.694 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:41.694 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 16824, 2024-08-31T07:37:41.694 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 1702272, 2024-08-31T07:37:41.694 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:41.694 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 16824, 2024-08-31T07:37:41.695 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:41.695 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:41.695 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 759, 2024-08-31T07:37:41.695 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1702272, 2024-08-31T07:37:41.695 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:41.695 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.695 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.695 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.696 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:41.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.696 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x55715ecbc680": { 2024-08-31T07:37:41.696 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 23513, 2024-08-31T07:37:41.696 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-08-31T07:37:41.696 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:41.696 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 83536, 2024-08-31T07:37:41.696 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 392725899, 2024-08-31T07:37:41.697 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:41.697 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 83536, 2024-08-31T07:37:41.697 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:41.697 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:41.697 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 24176, 2024-08-31T07:37:41.697 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 392702386, 2024-08-31T07:37:41.697 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:41.697 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:41.698 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:41.698 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:41.698 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:41.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:41.698 INFO:teuthology.orchestra.run.smithi084.stdout: "trackedop": { 2024-08-31T07:37:41.698 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 2197 2024-08-31T07:37:41.698 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:41.698 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-08-31T07:37:42.566 INFO:tasks.check_counter:Daemon mds.c mds.exported=8 2024-08-31T07:37:42.567 INFO:tasks.check_counter:Daemon mds.c mds.imported=11 2024-08-31T07:37:42.567 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-08-31T07:37:42.567 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-08-31T07:37:42.567 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T07:37:42.567 DEBUG:tasks.check_counter:Getting stats from f 2024-08-31T07:37:42.567 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-31T07:37:42.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:42 smithi084 ceph-mon[22387]: pgmap v1154: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.6 MiB/s wr, 93 op/s 2024-08-31T07:37:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:37:42.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:42 smithi032 ceph-mon[17252]: pgmap v1154: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.6 MiB/s wr, 93 op/s 2024-08-31T07:37:42.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:42 smithi073 ceph-mon[21995]: pgmap v1154: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.6 MiB/s wr, 93 op/s 2024-08-31T07:37:43.379 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-08-31T07:37:43.380 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:43.380 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 3706, 2024-08-31T07:37:43.381 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 3706, 2024-08-31T07:37:43.381 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 307297028, 2024-08-31T07:37:43.381 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 1177694, 2024-08-31T07:37:43.381 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 8, 2024-08-31T07:37:43.382 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 4, 2024-08-31T07:37:43.382 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.590601482, 2024-08-31T07:37:43.382 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.166791366, 2024-08-31T07:37:43.382 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.279368294, 2024-08-31T07:37:43.382 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.113566395, 2024-08-31T07:37:43.382 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:43.382 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3705, 2024-08-31T07:37:43.382 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.153163402, 2024-08-31T07:37:43.383 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000041339 2024-08-31T07:37:43.383 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.383 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:43.383 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.383 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.383 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.383 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.383 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-31T07:37:43.383 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 448 2024-08-31T07:37:43.384 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.384 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:43.384 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 1729, 2024-08-31T07:37:43.384 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2073, 2024-08-31T07:37:43.384 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 227878367, 2024-08-31T07:37:43.384 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 1246962, 2024-08-31T07:37:43.384 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 7, 2024-08-31T07:37:43.384 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 6, 2024-08-31T07:37:43.385 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.305448721, 2024-08-31T07:37:43.385 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.089497090, 2024-08-31T07:37:43.385 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.158143149, 2024-08-31T07:37:43.385 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.043028976, 2024-08-31T07:37:43.385 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:43.385 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2071, 2024-08-31T07:37:43.385 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.086822363, 2024-08-31T07:37:43.385 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000041922 2024-08-31T07:37:43.386 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.386 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:43.386 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.386 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.386 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.386 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.386 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-31T07:37:43.386 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 697392 2024-08-31T07:37:43.387 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.387 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:43.387 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 1803, 2024-08-31T07:37:43.387 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 1729, 2024-08-31T07:37:43.387 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 173695950, 2024-08-31T07:37:43.387 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 679443, 2024-08-31T07:37:43.387 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 5, 2024-08-31T07:37:43.387 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 4, 2024-08-31T07:37:43.388 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.287381356, 2024-08-31T07:37:43.388 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.094636844, 2024-08-31T07:37:43.388 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.161182472, 2024-08-31T07:37:43.388 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.034207799, 2024-08-31T07:37:43.388 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:43.388 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1728, 2024-08-31T07:37:43.388 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.076043721, 2024-08-31T07:37:43.389 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000044006 2024-08-31T07:37:43.389 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.389 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:43.389 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.389 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.389 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.390 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1404224, 2024-08-31T07:37:43.390 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 287984 2024-08-31T07:37:43.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.390 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-08-31T07:37:43.390 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-08-31T07:37:43.390 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-08-31T07:37:43.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.391 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-08-31T07:37:43.391 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-08-31T07:37:43.391 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-08-31T07:37:43.391 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 10859, 2024-08-31T07:37:43.391 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.286714601, 2024-08-31T07:37:43.391 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000026403 2024-08-31T07:37:43.392 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:43.392 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.392 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-08-31T07:37:43.392 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-08-31T07:37:43.392 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-08-31T07:37:43.392 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.393 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.393 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.393 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:43.393 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.393 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-08-31T07:37:43.394 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 0, 2024-08-31T07:37:43.394 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 0, 2024-08-31T07:37:43.394 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-08-31T07:37:43.394 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.394 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.394 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.394 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.394 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-08-31T07:37:43.395 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 0, 2024-08-31T07:37:43.395 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 0, 2024-08-31T07:37:43.395 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-08-31T07:37:43.395 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 0, 2024-08-31T07:37:43.395 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 0, 2024-08-31T07:37:43.395 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-08-31T07:37:43.395 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 18589, 2024-08-31T07:37:43.395 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 2398, 2024-08-31T07:37:43.395 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 16191, 2024-08-31T07:37:43.396 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 0, 2024-08-31T07:37:43.396 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 15163, 2024-08-31T07:37:43.396 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 0, 2024-08-31T07:37:43.396 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 0, 2024-08-31T07:37:43.396 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 0, 2024-08-31T07:37:43.396 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 16, 2024-08-31T07:37:43.396 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 0, 2024-08-31T07:37:43.396 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 0, 2024-08-31T07:37:43.397 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 0, 2024-08-31T07:37:43.397 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 0, 2024-08-31T07:37:43.397 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 0, 2024-08-31T07:37:43.397 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-08-31T07:37:43.397 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 0, 2024-08-31T07:37:43.397 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-08-31T07:37:43.398 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-08-31T07:37:43.398 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 0, 2024-08-31T07:37:43.398 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 0, 2024-08-31T07:37:43.398 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 0, 2024-08-31T07:37:43.398 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 0, 2024-08-31T07:37:43.398 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-08-31T07:37:43.398 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 0, 2024-08-31T07:37:43.398 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-08-31T07:37:43.399 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 760, 2024-08-31T07:37:43.399 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 216740959, 2024-08-31T07:37:43.399 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 6, 2024-08-31T07:37:43.399 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 0, 2024-08-31T07:37:43.399 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-08-31T07:37:43.399 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T07:37:43.399 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-08-31T07:37:43.400 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 0, 2024-08-31T07:37:43.400 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T07:37:43.400 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 0, 2024-08-31T07:37:43.400 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 0, 2024-08-31T07:37:43.400 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 0, 2024-08-31T07:37:43.400 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 0, 2024-08-31T07:37:43.400 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 0, 2024-08-31T07:37:43.401 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T07:37:43.401 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-31T07:37:43.401 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T07:37:43.401 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 0, 2024-08-31T07:37:43.401 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 0, 2024-08-31T07:37:43.401 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 0, 2024-08-31T07:37:43.402 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 0 2024-08-31T07:37:43.402 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.402 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-08-31T07:37:43.402 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 0, 2024-08-31T07:37:43.402 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 0, 2024-08-31T07:37:43.403 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 0, 2024-08-31T07:37:43.403 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 0, 2024-08-31T07:37:43.403 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 0, 2024-08-31T07:37:43.403 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-08-31T07:37:43.403 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-08-31T07:37:43.403 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-08-31T07:37:43.403 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 0, 2024-08-31T07:37:43.404 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 0, 2024-08-31T07:37:43.404 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-08-31T07:37:43.404 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-08-31T07:37:43.404 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-08-31T07:37:43.404 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-08-31T07:37:43.404 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-08-31T07:37:43.405 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-08-31T07:37:43.405 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-08-31T07:37:43.405 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_path": 0, 2024-08-31T07:37:43.405 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_inode": 0, 2024-08-31T07:37:43.405 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T07:37:43.405 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 0, 2024-08-31T07:37:43.405 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-08-31T07:37:43.406 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 0, 2024-08-31T07:37:43.406 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-08-31T07:37:43.406 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0, 2024-08-31T07:37:43.406 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_started": 0, 2024-08-31T07:37:43.406 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_succeeded": 0, 2024-08-31T07:37:43.406 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_write_failed": 0 2024-08-31T07:37:43.406 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.406 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-08-31T07:37:43.407 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-08-31T07:37:43.407 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 0, 2024-08-31T07:37:43.407 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-08-31T07:37:43.407 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-08-31T07:37:43.407 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 99355, 2024-08-31T07:37:43.407 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-08-31T07:37:43.408 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-08-31T07:37:43.408 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 0, 2024-08-31T07:37:43.409 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-08-31T07:37:43.409 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-08-31T07:37:43.409 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 151, 2024-08-31T07:37:43.410 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 10, 2024-08-31T07:37:43.410 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-08-31T07:37:43.410 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-08-31T07:37:43.411 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 119189440, 2024-08-31T07:37:43.411 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 709589740, 2024-08-31T07:37:43.411 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 709571338, 2024-08-31T07:37:43.411 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-08-31T07:37:43.411 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.412 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.412 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.412 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 99355 2024-08-31T07:37:43.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.412 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-08-31T07:37:43.413 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 2377, 2024-08-31T07:37:43.413 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 19921, 2024-08-31T07:37:43.413 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 17544, 2024-08-31T07:37:43.413 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 105, 2024-08-31T07:37:43.414 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 240, 2024-08-31T07:37:43.414 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 135, 2024-08-31T07:37:43.414 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 18589, 2024-08-31T07:37:43.414 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 20622, 2024-08-31T07:37:43.414 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 2033, 2024-08-31T07:37:43.415 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 0, 2024-08-31T07:37:43.415 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 0, 2024-08-31T07:37:43.415 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 0, 2024-08-31T07:37:43.415 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 146360, 2024-08-31T07:37:43.415 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 361107456 2024-08-31T07:37:43.416 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.416 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-08-31T07:37:43.416 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 0, 2024-08-31T07:37:43.416 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 0, 2024-08-31T07:37:43.416 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 0, 2024-08-31T07:37:43.417 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 0, 2024-08-31T07:37:43.417 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 0, 2024-08-31T07:37:43.417 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-08-31T07:37:43.417 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.417 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.418 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.418 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-08-31T07:37:43.418 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.418 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.419 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.419 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-08-31T07:37:43.419 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.419 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.420 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.420 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.420 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-08-31T07:37:43.420 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.420 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.421 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.421 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-08-31T07:37:43.421 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.421 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.421 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.422 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-08-31T07:37:43.422 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.422 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.422 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.423 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.423 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-08-31T07:37:43.423 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.423 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.424 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.424 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.424 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-08-31T07:37:43.424 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.425 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.425 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.425 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.425 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-08-31T07:37:43.426 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.426 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.426 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.426 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.427 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-08-31T07:37:43.427 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.427 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.427 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.428 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.429 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-08-31T07:37:43.429 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.429 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.430 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.430 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-08-31T07:37:43.430 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.431 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.431 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.431 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-08-31T07:37:43.431 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.432 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.432 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.432 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.432 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-08-31T07:37:43.433 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.433 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.433 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.434 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-08-31T07:37:43.434 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.434 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.434 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.435 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.435 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-08-31T07:37:43.435 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.435 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.435 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.436 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.436 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-08-31T07:37:43.436 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.436 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.437 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.438 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.438 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-08-31T07:37:43.438 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.438 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.438 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.439 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.439 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-08-31T07:37:43.439 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.439 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.439 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.439 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.439 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-08-31T07:37:43.439 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.440 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.440 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.440 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.440 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-08-31T07:37:43.440 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.440 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.440 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.441 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.441 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-08-31T07:37:43.441 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.441 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.441 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.441 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.441 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-08-31T07:37:43.441 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.442 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.442 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.442 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.442 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-08-31T07:37:43.442 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.442 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.442 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.442 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.443 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-08-31T07:37:43.443 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.443 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.443 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.443 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-08-31T07:37:43.443 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.443 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.443 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.444 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.444 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-08-31T07:37:43.444 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.444 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.444 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.444 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.444 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-08-31T07:37:43.444 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.445 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.445 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.445 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-08-31T07:37:43.445 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.445 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.445 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.446 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-08-31T07:37:43.446 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-08-31T07:37:43.446 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-08-31T07:37:43.446 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.446 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.446 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.446 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:43.446 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.447 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-08-31T07:37:43.447 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 2, 2024-08-31T07:37:43.447 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 3, 2024-08-31T07:37:43.447 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 1, 2024-08-31T07:37:43.447 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 2, 2024-08-31T07:37:43.447 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-08-31T07:37:43.447 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 0, 2024-08-31T07:37:43.447 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 0, 2024-08-31T07:37:43.447 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 2799, 2024-08-31T07:37:43.448 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-08-31T07:37:43.448 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.448 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-08-31T07:37:43.448 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:43.448 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:43.448 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:43.448 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:43.448 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:43.449 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:43.449 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:43.449 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:43.449 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:43.449 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:43.449 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:43.449 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:43.449 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:43.450 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:43.450 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:43.450 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:43.450 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:43.450 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:43.450 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:43.450 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:43.450 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:43.450 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:43.451 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:43.451 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:43.451 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:43.451 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:43.451 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:43.451 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:43.451 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:43.451 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:43.452 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:43.452 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-08-31T07:37:43.452 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:43.452 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:43.452 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:43.452 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:43.452 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 344816, 2024-08-31T07:37:43.452 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 2889, 2024-08-31T07:37:43.453 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-08-31T07:37:43.453 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-08-31T07:37:43.453 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-08-31T07:37:43.453 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-08-31T07:37:43.453 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:43.453 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:43.453 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:43.453 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:43.453 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 7968, 2024-08-31T07:37:43.454 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 176, 2024-08-31T07:37:43.454 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:43.454 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:43.454 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:43.454 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-08-31T07:37:43.454 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 18879876, 2024-08-31T07:37:43.454 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 136361, 2024-08-31T07:37:43.454 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:43.455 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-08-31T07:37:43.455 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:43.455 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-08-31T07:37:43.455 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.455 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-08-31T07:37:43.455 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-08-31T07:37:43.455 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-08-31T07:37:43.455 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 6661, 2024-08-31T07:37:43.456 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 26, 2024-08-31T07:37:43.456 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 4, 2024-08-31T07:37:43.456 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 6661, 2024-08-31T07:37:43.456 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-08-31T07:37:43.456 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 6661, 2024-08-31T07:37:43.456 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 11.007801758, 2024-08-31T07:37:43.456 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001652574 2024-08-31T07:37:43.456 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.457 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-08-31T07:37:43.457 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-08-31T07:37:43.457 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 6661, 2024-08-31T07:37:43.457 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 6662 2024-08-31T07:37:43.457 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.457 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 6661, 2024-08-31T07:37:43.457 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 6661, 2024-08-31T07:37:43.457 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 0, 2024-08-31T07:37:43.458 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-08-31T07:37:43.458 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-08-31T07:37:43.458 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 3822, 2024-08-31T07:37:43.458 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 0, 2024-08-31T07:37:43.458 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 2836, 2024-08-31T07:37:43.458 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 0, 2024-08-31T07:37:43.458 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 0, 2024-08-31T07:37:43.458 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-08-31T07:37:43.458 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-08-31T07:37:43.459 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-08-31T07:37:43.459 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 0, 2024-08-31T07:37:43.459 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 0, 2024-08-31T07:37:43.459 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-08-31T07:37:43.459 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-08-31T07:37:43.459 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-08-31T07:37:43.459 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 2, 2024-08-31T07:37:43.459 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 0, 2024-08-31T07:37:43.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-08-31T07:37:43.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-08-31T07:37:43.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-08-31T07:37:43.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-08-31T07:37:43.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-08-31T07:37:43.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-08-31T07:37:43.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T07:37:43.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-08-31T07:37:43.461 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-08-31T07:37:43.461 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-08-31T07:37:43.461 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-08-31T07:37:43.461 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-08-31T07:37:43.461 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-08-31T07:37:43.461 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-08-31T07:37:43.461 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-08-31T07:37:43.461 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-08-31T07:37:43.462 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-08-31T07:37:43.462 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-08-31T07:37:43.462 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-08-31T07:37:43.462 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-08-31T07:37:43.462 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-08-31T07:37:43.462 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-08-31T07:37:43.462 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-08-31T07:37:43.462 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-08-31T07:37:43.462 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-08-31T07:37:43.463 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-08-31T07:37:43.463 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 118, 2024-08-31T07:37:43.463 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-08-31T07:37:43.463 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 40, 2024-08-31T07:37:43.463 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 11, 2024-08-31T07:37:43.463 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 11, 2024-08-31T07:37:43.463 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 0, 2024-08-31T07:37:43.463 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-08-31T07:37:43.464 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 0, 2024-08-31T07:37:43.464 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 2, 2024-08-31T07:37:43.464 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 0 2024-08-31T07:37:43.464 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.464 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-08-31T07:37:43.464 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 0, 2024-08-31T07:37:43.464 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-08-31T07:37:43.464 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-08-31T07:37:43.465 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-08-31T07:37:43.465 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.465 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-08-31T07:37:43.465 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-08-31T07:37:43.465 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 0, 2024-08-31T07:37:43.465 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-08-31T07:37:43.465 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 0, 2024-08-31T07:37:43.465 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 0, 2024-08-31T07:37:43.466 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 0, 2024-08-31T07:37:43.466 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-08-31T07:37:43.466 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.466 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:43.466 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:43.466 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-08-31T07:37:43.466 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:43.466 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 7234, 2024-08-31T07:37:43.467 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 708188393, 2024-08-31T07:37:43.467 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:43.467 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 7234, 2024-08-31T07:37:43.467 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:43.467 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:43.467 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 7234, 2024-08-31T07:37:43.467 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 708188393, 2024-08-31T07:37:43.467 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:43.468 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.468 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.468 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.468 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:43.468 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.468 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-08-31T07:37:43.468 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:43.468 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-08-31T07:37:43.469 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:43.469 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:43.469 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:43.469 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:43.469 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:43.469 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 6661, 2024-08-31T07:37:43.469 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 705628663, 2024-08-31T07:37:43.469 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 6661, 2024-08-31T07:37:43.470 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 705628663, 2024-08-31T07:37:43.470 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:43.470 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.470 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.470 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.470 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:43.470 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.470 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-08-31T07:37:43.470 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:43.471 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-08-31T07:37:43.471 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:43.471 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:43.471 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:43.471 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:43.471 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:43.471 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 6661, 2024-08-31T07:37:43.472 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 6661, 2024-08-31T07:37:43.472 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 6661, 2024-08-31T07:37:43.472 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 6661, 2024-08-31T07:37:43.472 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:43.472 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.472 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.472 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.472 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:43.472 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.473 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-08-31T07:37:43.473 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:43.473 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-08-31T07:37:43.473 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:43.473 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:43.473 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:43.473 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:43.473 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:43.474 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:43.474 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:43.474 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-08-31T07:37:43.474 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-08-31T07:37:43.474 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:43.474 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.474 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.474 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.475 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:43.475 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.475 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x55b66ae6c750": { 2024-08-31T07:37:43.475 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:43.475 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-08-31T07:37:43.475 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:43.475 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:43.475 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:43.475 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:43.476 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:43.476 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:43.476 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:43.476 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-08-31T07:37:43.476 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-08-31T07:37:43.476 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:43.476 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:43.476 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:43.477 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:43.477 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:43.477 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:43.477 INFO:teuthology.orchestra.run.smithi084.stdout: "trackedop": { 2024-08-31T07:37:43.477 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 0 2024-08-31T07:37:43.477 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:43.477 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-08-31T07:37:44.078 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-08-31T07:37:44.079 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-08-31T07:37:44.079 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-08-31T07:37:44.079 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-08-31T07:37:44.079 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T07:37:44.079 DEBUG:tasks.check_counter:Getting stats from i 2024-08-31T07:37:44.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-31T07:37:44.327 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:37:44.728 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:44 smithi084 ceph-mon[22387]: pgmap v1155: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 2.6 MiB/s wr, 97 op/s 2024-08-31T07:37:44.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:44 smithi032 ceph-mon[17252]: pgmap v1155: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 2.6 MiB/s wr, 97 op/s 2024-08-31T07:37:44.874 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-08-31T07:37:44.874 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:44.874 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 274713, 2024-08-31T07:37:44.874 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 332459, 2024-08-31T07:37:44.875 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 94215732, 2024-08-31T07:37:44.875 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 285681876, 2024-08-31T07:37:44.875 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 17, 2024-08-31T07:37:44.875 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 6, 2024-08-31T07:37:44.875 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 18.925687619, 2024-08-31T07:37:44.875 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 10.754114004, 2024-08-31T07:37:44.875 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 10.300382377, 2024-08-31T07:37:44.875 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.387732571, 2024-08-31T07:37:44.876 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:44.876 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 332457, 2024-08-31T07:37:44.876 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 14.208683105, 2024-08-31T07:37:44.876 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000042738 2024-08-31T07:37:44.876 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.876 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:44.876 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 372969, 2024-08-31T07:37:44.876 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.093470683, 2024-08-31T07:37:44.877 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000250 2024-08-31T07:37:44.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.877 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1438528, 2024-08-31T07:37:44.877 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 319040 2024-08-31T07:37:44.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.877 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:44.877 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 61218, 2024-08-31T07:37:44.877 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 54242, 2024-08-31T07:37:44.878 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 25463793, 2024-08-31T07:37:44.878 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 194844304, 2024-08-31T07:37:44.878 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 14, 2024-08-31T07:37:44.878 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 7, 2024-08-31T07:37:44.878 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 5.387284838, 2024-08-31T07:37:44.879 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 2.533559420, 2024-08-31T07:37:44.879 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 2.091038319, 2024-08-31T07:37:44.879 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 1.004154384, 2024-08-31T07:37:44.879 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:44.879 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 54240, 2024-08-31T07:37:44.879 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 2.723916069, 2024-08-31T07:37:44.879 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000050219 2024-08-31T07:37:44.880 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.880 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:44.880 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 70926, 2024-08-31T07:37:44.880 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.015458688, 2024-08-31T07:37:44.880 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000217 2024-08-31T07:37:44.880 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.880 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-31T07:37:44.880 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 699312 2024-08-31T07:37:44.881 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.881 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:44.881 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 95132, 2024-08-31T07:37:44.881 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 76447, 2024-08-31T07:37:44.881 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 46755595, 2024-08-31T07:37:44.881 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 145540074, 2024-08-31T07:37:44.882 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 58, 2024-08-31T07:37:44.882 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 5, 2024-08-31T07:37:44.882 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 7.200718651, 2024-08-31T07:37:44.882 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 3.475156836, 2024-08-31T07:37:44.882 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 3.652228423, 2024-08-31T07:37:44.882 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.803184875, 2024-08-31T07:37:44.882 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:44.882 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 76447, 2024-08-31T07:37:44.883 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 3.497911829, 2024-08-31T07:37:44.883 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000045756 2024-08-31T07:37:44.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.883 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:44.883 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 137455, 2024-08-31T07:37:44.883 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.027202089, 2024-08-31T07:37:44.883 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000197 2024-08-31T07:37:44.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.884 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 928, 2024-08-31T07:37:44.884 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 736 2024-08-31T07:37:44.884 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.884 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-08-31T07:37:44.884 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-08-31T07:37:44.884 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-08-31T07:37:44.884 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.884 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-08-31T07:37:44.885 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-08-31T07:37:44.885 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-08-31T07:37:44.885 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 49557, 2024-08-31T07:37:44.885 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 55.990783490, 2024-08-31T07:37:44.885 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001129825 2024-08-31T07:37:44.885 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:44.885 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.885 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-08-31T07:37:44.886 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-08-31T07:37:44.886 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-08-31T07:37:44.886 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 29207, 2024-08-31T07:37:44.886 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 3.168520188, 2024-08-31T07:37:44.886 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000108484 2024-08-31T07:37:44.886 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:44.886 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.886 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-08-31T07:37:44.886 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 104164, 2024-08-31T07:37:44.887 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 97847, 2024-08-31T07:37:44.887 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-08-31T07:37:44.887 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 97847, 2024-08-31T07:37:44.887 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 756.792126754, 2024-08-31T07:37:44.887 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.007734443 2024-08-31T07:37:44.887 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.887 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-08-31T07:37:44.887 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 6317, 2024-08-31T07:37:44.888 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 5, 2024-08-31T07:37:44.888 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-08-31T07:37:44.888 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 92, 2024-08-31T07:37:44.888 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 9, 2024-08-31T07:37:44.888 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 3, 2024-08-31T07:37:44.888 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 2128, 2024-08-31T07:37:44.888 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 565, 2024-08-31T07:37:44.888 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 1548, 2024-08-31T07:37:44.889 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 15, 2024-08-31T07:37:44.889 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 1320, 2024-08-31T07:37:44.889 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 820, 2024-08-31T07:37:44.889 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 112, 2024-08-31T07:37:44.889 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 112, 2024-08-31T07:37:44.889 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 23, 2024-08-31T07:37:44.889 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 164099, 2024-08-31T07:37:44.889 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 100270, 2024-08-31T07:37:44.890 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 6309, 2024-08-31T07:37:44.890 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 113, 2024-08-31T07:37:44.890 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 0, 2024-08-31T07:37:44.890 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-08-31T07:37:44.890 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 1867, 2024-08-31T07:37:44.890 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 13680, 2024-08-31T07:37:44.890 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-08-31T07:37:44.890 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 22, 2024-08-31T07:37:44.891 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 517, 2024-08-31T07:37:44.891 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 15, 2024-08-31T07:37:44.891 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 700, 2024-08-31T07:37:44.891 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-08-31T07:37:44.891 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 1655, 2024-08-31T07:37:44.891 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-08-31T07:37:44.891 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 745, 2024-08-31T07:37:44.891 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 212490313, 2024-08-31T07:37:44.892 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 6, 2024-08-31T07:37:44.892 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 1655, 2024-08-31T07:37:44.892 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-08-31T07:37:44.892 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T07:37:44.892 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-08-31T07:37:44.892 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 180, 2024-08-31T07:37:44.893 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T07:37:44.893 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 180, 2024-08-31T07:37:44.893 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 1475, 2024-08-31T07:37:44.893 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 23, 2024-08-31T07:37:44.893 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 12516, 2024-08-31T07:37:44.893 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 67393, 2024-08-31T07:37:44.893 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T07:37:44.893 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 2, 2024-08-31T07:37:44.894 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T07:37:44.894 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 49400, 2024-08-31T07:37:44.894 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 14664, 2024-08-31T07:37:44.894 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 2345, 2024-08-31T07:37:44.894 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 73654 2024-08-31T07:37:44.894 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.894 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-08-31T07:37:44.894 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 16, 2024-08-31T07:37:44.895 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 63, 2024-08-31T07:37:44.895 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 43, 2024-08-31T07:37:44.895 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 152, 2024-08-31T07:37:44.895 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 197, 2024-08-31T07:37:44.895 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-08-31T07:37:44.895 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-08-31T07:37:44.895 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-08-31T07:37:44.895 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 15044, 2024-08-31T07:37:44.896 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 15044, 2024-08-31T07:37:44.896 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-08-31T07:37:44.896 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-08-31T07:37:44.896 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-08-31T07:37:44.896 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-08-31T07:37:44.896 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-08-31T07:37:44.896 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-08-31T07:37:44.896 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-08-31T07:37:44.897 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_path": 19, 2024-08-31T07:37:44.897 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_inode": 1975, 2024-08-31T07:37:44.897 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T07:37:44.897 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 24, 2024-08-31T07:37:44.897 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-08-31T07:37:44.897 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 12, 2024-08-31T07:37:44.897 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-08-31T07:37:44.897 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0, 2024-08-31T07:37:44.898 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_started": 0, 2024-08-31T07:37:44.898 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_succeeded": 0, 2024-08-31T07:37:44.898 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_write_failed": 0 2024-08-31T07:37:44.898 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.898 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-08-31T07:37:44.898 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-08-31T07:37:44.898 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 84276, 2024-08-31T07:37:44.898 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-08-31T07:37:44.898 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-08-31T07:37:44.899 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 84276, 2024-08-31T07:37:44.899 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-08-31T07:37:44.899 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-08-31T07:37:44.899 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 125, 2024-08-31T07:37:44.899 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-08-31T07:37:44.899 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-08-31T07:37:44.899 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 125, 2024-08-31T07:37:44.899 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 8, 2024-08-31T07:37:44.899 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-08-31T07:37:44.900 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-08-31T07:37:44.900 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 4194304, 2024-08-31T07:37:44.900 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 380416176, 2024-08-31T07:37:44.900 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 0, 2024-08-31T07:37:44.900 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-08-31T07:37:44.900 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 26884, 2024-08-31T07:37:44.900 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 214.580608813, 2024-08-31T07:37:44.901 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.007981721 2024-08-31T07:37:44.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.901 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 0 2024-08-31T07:37:44.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.901 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-08-31T07:37:44.901 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 1663, 2024-08-31T07:37:44.901 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 17283, 2024-08-31T07:37:44.901 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 15620, 2024-08-31T07:37:44.902 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 182, 2024-08-31T07:37:44.902 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 257, 2024-08-31T07:37:44.902 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 75, 2024-08-31T07:37:44.902 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 2105, 2024-08-31T07:37:44.902 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 29056, 2024-08-31T07:37:44.902 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 26951, 2024-08-31T07:37:44.902 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 100, 2024-08-31T07:37:44.903 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 17051, 2024-08-31T07:37:44.903 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 16951, 2024-08-31T07:37:44.903 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 151988, 2024-08-31T07:37:44.903 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 359538688 2024-08-31T07:37:44.903 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.903 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-08-31T07:37:44.904 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 149291, 2024-08-31T07:37:44.904 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 2124, 2024-08-31T07:37:44.904 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 104164, 2024-08-31T07:37:44.904 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 2870, 2024-08-31T07:37:44.904 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 7279, 2024-08-31T07:37:44.904 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-08-31T07:37:44.904 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 15113, 2024-08-31T07:37:44.904 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 274.732672503, 2024-08-31T07:37:44.905 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.018178566 2024-08-31T07:37:44.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.905 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-08-31T07:37:44.905 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3927, 2024-08-31T07:37:44.905 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 42.626587393, 2024-08-31T07:37:44.905 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.010854745 2024-08-31T07:37:44.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.905 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-08-31T07:37:44.906 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.906 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.906 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.906 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.906 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-08-31T07:37:44.906 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.906 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.906 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.907 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.907 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-08-31T07:37:44.907 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 22831, 2024-08-31T07:37:44.907 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 54.631980559, 2024-08-31T07:37:44.907 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.002392886 2024-08-31T07:37:44.907 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.907 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-08-31T07:37:44.907 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.907 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.908 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.908 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.908 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-08-31T07:37:44.908 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.908 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.908 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.908 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.908 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-08-31T07:37:44.909 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.909 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.909 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.909 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.909 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-08-31T07:37:44.909 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.909 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.910 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.910 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.910 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-08-31T07:37:44.911 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.911 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.911 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.911 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.911 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-08-31T07:37:44.911 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.911 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.911 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.912 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.912 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-08-31T07:37:44.912 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.912 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.912 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.912 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.912 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-08-31T07:37:44.912 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.913 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.913 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.913 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.913 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-08-31T07:37:44.913 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.913 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.913 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.914 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-08-31T07:37:44.914 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5, 2024-08-31T07:37:44.914 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.002743626, 2024-08-31T07:37:44.914 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000548725 2024-08-31T07:37:44.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.914 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-08-31T07:37:44.914 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 36579, 2024-08-31T07:37:44.915 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 131.541476575, 2024-08-31T07:37:44.915 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.003596092 2024-08-31T07:37:44.915 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.915 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-08-31T07:37:44.915 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3455, 2024-08-31T07:37:44.915 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 69.082914549, 2024-08-31T07:37:44.915 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.019995054 2024-08-31T07:37:44.915 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.916 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-08-31T07:37:44.916 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.916 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.916 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.916 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.916 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-08-31T07:37:44.916 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.916 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.917 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.917 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.917 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-08-31T07:37:44.917 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.917 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.917 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.917 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.917 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-08-31T07:37:44.918 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.918 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.918 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.918 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.918 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-08-31T07:37:44.918 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.918 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.918 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.918 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.919 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-08-31T07:37:44.919 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 89, 2024-08-31T07:37:44.919 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 8.608396474, 2024-08-31T07:37:44.919 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.096723555 2024-08-31T07:37:44.919 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.919 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-08-31T07:37:44.919 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.919 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.920 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.920 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.920 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-08-31T07:37:44.920 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 804, 2024-08-31T07:37:44.920 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.634024726, 2024-08-31T07:37:44.920 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000788587 2024-08-31T07:37:44.920 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.920 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-08-31T07:37:44.921 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.921 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.921 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.921 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.921 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-08-31T07:37:44.921 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.921 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.921 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.922 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.922 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-08-31T07:37:44.922 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.922 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.922 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.922 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.923 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-08-31T07:37:44.923 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 15044, 2024-08-31T07:37:44.923 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 174.931330349, 2024-08-31T07:37:44.923 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.011627979 2024-08-31T07:37:44.923 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.923 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-08-31T07:37:44.923 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-08-31T07:37:44.923 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-08-31T07:37:44.924 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.924 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.924 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.924 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:44.924 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.924 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-08-31T07:37:44.924 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 1, 2024-08-31T07:37:44.924 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 2, 2024-08-31T07:37:44.925 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 1, 2024-08-31T07:37:44.925 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 1, 2024-08-31T07:37:44.925 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-08-31T07:37:44.925 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 12629, 2024-08-31T07:37:44.925 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 12629, 2024-08-31T07:37:44.925 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 1403, 2024-08-31T07:37:44.926 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-08-31T07:37:44.926 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.926 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-08-31T07:37:44.926 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:44.926 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:44.927 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:44.927 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:44.927 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:44.927 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:44.927 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:44.927 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:44.927 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:44.927 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:44.928 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:44.928 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:44.928 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:44.928 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:44.928 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:44.928 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:44.929 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:44.929 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:44.929 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:44.929 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:44.929 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:44.929 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:44.929 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:44.929 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:44.930 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:44.930 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:44.930 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:44.930 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:44.930 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:44.930 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:44.930 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:44.931 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-08-31T07:37:44.931 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:44.931 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:44.931 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:44.931 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:44.931 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 400909, 2024-08-31T07:37:44.931 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 801, 2024-08-31T07:37:44.931 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-08-31T07:37:44.931 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-08-31T07:37:44.932 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-08-31T07:37:44.932 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-08-31T07:37:44.932 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:44.932 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:44.932 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:44.932 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:44.932 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 7968, 2024-08-31T07:37:44.933 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 176, 2024-08-31T07:37:44.933 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:44.933 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:44.933 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:44.933 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-08-31T07:37:44.933 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 6138496, 2024-08-31T07:37:44.934 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 48916, 2024-08-31T07:37:44.934 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:44.934 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-08-31T07:37:44.934 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:44.934 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-08-31T07:37:44.934 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.934 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-08-31T07:37:44.934 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-08-31T07:37:44.934 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-08-31T07:37:44.935 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 46850, 2024-08-31T07:37:44.935 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 380459213, 2024-08-31T07:37:44.935 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 33, 2024-08-31T07:37:44.935 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 46850, 2024-08-31T07:37:44.935 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-08-31T07:37:44.935 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 46850, 2024-08-31T07:37:44.935 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 313.420388260, 2024-08-31T07:37:44.935 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.006689869 2024-08-31T07:37:44.935 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.936 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-08-31T07:37:44.936 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-08-31T07:37:44.936 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 46850, 2024-08-31T07:37:44.936 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 48117 2024-08-31T07:37:44.936 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.936 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 46850, 2024-08-31T07:37:44.936 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 2537, 2024-08-31T07:37:44.936 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 44313, 2024-08-31T07:37:44.937 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-08-31T07:37:44.937 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-08-31T07:37:44.937 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 0, 2024-08-31T07:37:44.937 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 10, 2024-08-31T07:37:44.937 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 861, 2024-08-31T07:37:44.937 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 27745, 2024-08-31T07:37:44.937 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 538, 2024-08-31T07:37:44.937 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-08-31T07:37:44.938 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-08-31T07:37:44.938 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-08-31T07:37:44.938 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 9, 2024-08-31T07:37:44.938 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 15169, 2024-08-31T07:37:44.938 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-08-31T07:37:44.938 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-08-31T07:37:44.938 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-08-31T07:37:44.938 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 1655, 2024-08-31T07:37:44.939 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 20, 2024-08-31T07:37:44.939 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-08-31T07:37:44.939 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-08-31T07:37:44.939 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-08-31T07:37:44.939 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-08-31T07:37:44.939 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-08-31T07:37:44.939 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-08-31T07:37:44.939 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T07:37:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-08-31T07:37:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-08-31T07:37:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-08-31T07:37:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-08-31T07:37:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-08-31T07:37:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-08-31T07:37:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-08-31T07:37:44.940 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-08-31T07:37:44.941 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-08-31T07:37:44.941 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-08-31T07:37:44.941 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-08-31T07:37:44.941 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-08-31T07:37:44.941 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-08-31T07:37:44.941 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-08-31T07:37:44.941 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-08-31T07:37:44.941 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-08-31T07:37:44.942 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-08-31T07:37:44.942 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-08-31T07:37:44.942 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-08-31T07:37:44.942 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 118, 2024-08-31T07:37:44.942 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-08-31T07:37:44.942 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 40, 2024-08-31T07:37:44.942 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 12, 2024-08-31T07:37:44.943 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 12, 2024-08-31T07:37:44.943 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 0, 2024-08-31T07:37:44.943 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-08-31T07:37:44.943 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 1526, 2024-08-31T07:37:44.943 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 26, 2024-08-31T07:37:44.943 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 558 2024-08-31T07:37:44.943 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.944 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-08-31T07:37:44.944 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 1, 2024-08-31T07:37:44.944 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-08-31T07:37:44.944 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-08-31T07:37:44.944 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-08-31T07:37:44.944 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.944 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-08-31T07:37:44.945 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-08-31T07:37:44.945 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 6, 2024-08-31T07:37:44.945 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-08-31T07:37:44.945 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 6, 2024-08-31T07:37:44.945 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 15044, 2024-08-31T07:37:44.945 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 15044, 2024-08-31T07:37:44.945 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-08-31T07:37:44.945 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.946 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:44.946 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:44.946 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-08-31T07:37:44.946 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:44.946 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 431059, 2024-08-31T07:37:44.946 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 130189518, 2024-08-31T07:37:44.946 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:44.946 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 431059, 2024-08-31T07:37:44.946 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:44.947 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:44.947 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 431059, 2024-08-31T07:37:44.947 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 130189518, 2024-08-31T07:37:44.947 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:44.947 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.947 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.947 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.947 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:44.947 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.948 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-08-31T07:37:44.948 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:44.948 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-08-31T07:37:44.948 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:44.948 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:44.948 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:44.948 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:44.948 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:44.948 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 46850, 2024-08-31T07:37:44.949 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 381981237, 2024-08-31T07:37:44.949 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 46850, 2024-08-31T07:37:44.949 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 381981237, 2024-08-31T07:37:44.949 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:44.949 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.949 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.949 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.950 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:44.950 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.950 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-08-31T07:37:44.950 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:44.950 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-08-31T07:37:44.950 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:44.950 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:44.950 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:44.950 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:44.951 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:44.951 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 46850, 2024-08-31T07:37:44.951 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 46850, 2024-08-31T07:37:44.951 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 46850, 2024-08-31T07:37:44.951 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 46850, 2024-08-31T07:37:44.951 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:44.951 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.951 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.952 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.952 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:44.952 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.952 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-08-31T07:37:44.952 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:44.952 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-08-31T07:37:44.952 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:44.952 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 15044, 2024-08-31T07:37:44.953 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 1522572, 2024-08-31T07:37:44.953 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:44.953 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 15044, 2024-08-31T07:37:44.953 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:44.953 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:44.953 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 861, 2024-08-31T07:37:44.953 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1522572, 2024-08-31T07:37:44.953 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:44.954 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.954 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.954 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.954 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:44.954 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.954 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x563aff214750": { 2024-08-31T07:37:44.954 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:44.954 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-08-31T07:37:44.955 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:44.955 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 84276, 2024-08-31T07:37:44.955 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 376221872, 2024-08-31T07:37:44.955 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:44.955 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 84276, 2024-08-31T07:37:44.955 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:44.955 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:44.955 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 26884, 2024-08-31T07:37:44.955 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 376221872, 2024-08-31T07:37:44.956 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:44.956 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:44.956 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:44.956 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:44.956 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:44.956 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:44.956 INFO:teuthology.orchestra.run.smithi084.stdout: "trackedop": { 2024-08-31T07:37:44.956 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 2372 2024-08-31T07:37:44.957 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:44.957 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-08-31T07:37:44.957 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:44 smithi073 ceph-mon[21995]: pgmap v1155: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 2.6 MiB/s wr, 97 op/s 2024-08-31T07:37:45.553 INFO:tasks.check_counter:Daemon mds.i mds.exported=22 2024-08-31T07:37:45.554 INFO:tasks.check_counter:Daemon mds.i mds.imported=15 2024-08-31T07:37:45.554 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-08-31T07:37:45.554 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-08-31T07:37:45.554 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T07:37:45.554 DEBUG:tasks.check_counter:Getting stats from l 2024-08-31T07:37:45.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-31T07:37:45.801 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T07:37:46.501 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-08-31T07:37:46.502 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T07:37:46.502 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2637, 2024-08-31T07:37:46.502 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2639, 2024-08-31T07:37:46.502 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 135055777, 2024-08-31T07:37:46.502 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 838567, 2024-08-31T07:37:46.503 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 9, 2024-08-31T07:37:46.503 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 4, 2024-08-31T07:37:46.503 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.399132966, 2024-08-31T07:37:46.503 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.131883578, 2024-08-31T07:37:46.503 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.152790477, 2024-08-31T07:37:46.503 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.089872172, 2024-08-31T07:37:46.503 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:46.503 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2637, 2024-08-31T07:37:46.504 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.111194379, 2024-08-31T07:37:46.504 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000042167 2024-08-31T07:37:46.504 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.504 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:46.504 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.504 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.504 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.504 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.505 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T07:37:46.505 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T07:37:46.505 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.505 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T07:37:46.505 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2965, 2024-08-31T07:37:46.505 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2891, 2024-08-31T07:37:46.505 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 130568045, 2024-08-31T07:37:46.505 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 1050783, 2024-08-31T07:37:46.506 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 6, 2024-08-31T07:37:46.506 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 5, 2024-08-31T07:37:46.506 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.377091340, 2024-08-31T07:37:46.506 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.139137428, 2024-08-31T07:37:46.506 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.176702511, 2024-08-31T07:37:46.506 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.060709800, 2024-08-31T07:37:46.506 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:46.506 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2890, 2024-08-31T07:37:46.507 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.124251980, 2024-08-31T07:37:46.507 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000042993 2024-08-31T07:37:46.507 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:46.507 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.507 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.507 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.507 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1405424, 2024-08-31T07:37:46.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 291024 2024-08-31T07:37:46.508 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.508 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T07:37:46.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 940, 2024-08-31T07:37:46.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 1286, 2024-08-31T07:37:46.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 130279186, 2024-08-31T07:37:46.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 1087279, 2024-08-31T07:37:46.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 7, 2024-08-31T07:37:46.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 5, 2024-08-31T07:37:46.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.190058421, 2024-08-31T07:37:46.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.058255554, 2024-08-31T07:37:46.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.096987620, 2024-08-31T07:37:46.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.025037338, 2024-08-31T07:37:46.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T07:37:46.510 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1285, 2024-08-31T07:37:46.510 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.053788606, 2024-08-31T07:37:46.510 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000041858 2024-08-31T07:37:46.510 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-08-31T07:37:46.510 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.511 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.511 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.511 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-08-31T07:37:46.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 790256 2024-08-31T07:37:46.511 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.511 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-08-31T07:37:46.511 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-08-31T07:37:46.511 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-08-31T07:37:46.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.512 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-08-31T07:37:46.512 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-08-31T07:37:46.512 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-08-31T07:37:46.512 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 9856, 2024-08-31T07:37:46.512 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.261727321, 2024-08-31T07:37:46.512 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000026555 2024-08-31T07:37:46.512 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:46.513 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.513 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-08-31T07:37:46.513 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-08-31T07:37:46.513 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-08-31T07:37:46.513 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.513 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.513 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.513 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:46.514 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.514 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-08-31T07:37:46.514 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 0, 2024-08-31T07:37:46.514 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 0, 2024-08-31T07:37:46.514 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-08-31T07:37:46.514 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.515 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.515 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.515 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.515 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-08-31T07:37:46.515 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 0, 2024-08-31T07:37:46.515 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 0, 2024-08-31T07:37:46.515 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-08-31T07:37:46.515 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 0, 2024-08-31T07:37:46.516 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 0, 2024-08-31T07:37:46.516 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-08-31T07:37:46.516 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 18572, 2024-08-31T07:37:46.516 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 43, 2024-08-31T07:37:46.516 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 18529, 2024-08-31T07:37:46.516 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 0, 2024-08-31T07:37:46.516 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 18511, 2024-08-31T07:37:46.516 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 0, 2024-08-31T07:37:46.517 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 0, 2024-08-31T07:37:46.517 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 0, 2024-08-31T07:37:46.517 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 9, 2024-08-31T07:37:46.517 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 0, 2024-08-31T07:37:46.517 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 0, 2024-08-31T07:37:46.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 0, 2024-08-31T07:37:46.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 0, 2024-08-31T07:37:46.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 0, 2024-08-31T07:37:46.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-08-31T07:37:46.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 0, 2024-08-31T07:37:46.518 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-08-31T07:37:46.518 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-08-31T07:37:46.519 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 0, 2024-08-31T07:37:46.519 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 0, 2024-08-31T07:37:46.519 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 0, 2024-08-31T07:37:46.519 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 0, 2024-08-31T07:37:46.519 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-08-31T07:37:46.519 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 0, 2024-08-31T07:37:46.519 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-08-31T07:37:46.520 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 0, 2024-08-31T07:37:46.520 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 0, 2024-08-31T07:37:46.520 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 0, 2024-08-31T07:37:46.520 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 0, 2024-08-31T07:37:46.520 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-08-31T07:37:46.520 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T07:37:46.520 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-08-31T07:37:46.521 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 0, 2024-08-31T07:37:46.521 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T07:37:46.521 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 0, 2024-08-31T07:37:46.521 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 0, 2024-08-31T07:37:46.521 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 0, 2024-08-31T07:37:46.521 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 0, 2024-08-31T07:37:46.521 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 0, 2024-08-31T07:37:46.522 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T07:37:46.522 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-31T07:37:46.522 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T07:37:46.522 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 0, 2024-08-31T07:37:46.522 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 0, 2024-08-31T07:37:46.522 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 0, 2024-08-31T07:37:46.522 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 0 2024-08-31T07:37:46.522 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.523 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-08-31T07:37:46.523 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 0, 2024-08-31T07:37:46.523 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 0, 2024-08-31T07:37:46.523 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 0, 2024-08-31T07:37:46.523 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 0, 2024-08-31T07:37:46.523 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 0, 2024-08-31T07:37:46.523 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-08-31T07:37:46.524 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-08-31T07:37:46.524 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-08-31T07:37:46.524 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 0, 2024-08-31T07:37:46.524 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 0, 2024-08-31T07:37:46.524 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-08-31T07:37:46.524 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-08-31T07:37:46.524 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-08-31T07:37:46.524 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-08-31T07:37:46.525 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-08-31T07:37:46.525 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-08-31T07:37:46.525 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-08-31T07:37:46.525 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_path": 0, 2024-08-31T07:37:46.525 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_inode": 0, 2024-08-31T07:37:46.525 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T07:37:46.525 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 0, 2024-08-31T07:37:46.525 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-08-31T07:37:46.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 0, 2024-08-31T07:37:46.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-08-31T07:37:46.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0, 2024-08-31T07:37:46.526 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_started": 0, 2024-08-31T07:37:46.526 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_succeeded": 0, 2024-08-31T07:37:46.526 INFO:teuthology.orchestra.run.smithi084.stdout: "uninline_write_failed": 0 2024-08-31T07:37:46.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.526 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-08-31T07:37:46.527 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-08-31T07:37:46.527 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 0, 2024-08-31T07:37:46.527 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-08-31T07:37:46.527 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-08-31T07:37:46.527 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 83539, 2024-08-31T07:37:46.527 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-08-31T07:37:46.527 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-08-31T07:37:46.527 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 0, 2024-08-31T07:37:46.528 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-08-31T07:37:46.528 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-08-31T07:37:46.528 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 125, 2024-08-31T07:37:46.528 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 8, 2024-08-31T07:37:46.528 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-08-31T07:37:46.528 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-08-31T07:37:46.528 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 4194304, 2024-08-31T07:37:46.528 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 396960117, 2024-08-31T07:37:46.529 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 396946677, 2024-08-31T07:37:46.529 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-08-31T07:37:46.529 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.529 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.529 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.529 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 83539 2024-08-31T07:37:46.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.530 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-08-31T07:37:46.530 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 1563, 2024-08-31T07:37:46.530 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 18757, 2024-08-31T07:37:46.530 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 17194, 2024-08-31T07:37:46.530 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 78, 2024-08-31T07:37:46.530 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 102, 2024-08-31T07:37:46.530 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 24, 2024-08-31T07:37:46.530 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 18572, 2024-08-31T07:37:46.530 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 19084, 2024-08-31T07:37:46.531 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 512, 2024-08-31T07:37:46.531 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 0, 2024-08-31T07:37:46.531 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 0, 2024-08-31T07:37:46.531 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 0, 2024-08-31T07:37:46.531 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 141528, 2024-08-31T07:37:46.531 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 354992128 2024-08-31T07:37:46.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.532 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-08-31T07:37:46.532 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 0, 2024-08-31T07:37:46.532 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 0, 2024-08-31T07:37:46.532 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 0, 2024-08-31T07:37:46.532 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 0, 2024-08-31T07:37:46.532 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 0, 2024-08-31T07:37:46.532 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-08-31T07:37:46.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.533 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.533 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-08-31T07:37:46.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.533 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.534 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-08-31T07:37:46.534 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.534 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.534 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.534 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-08-31T07:37:46.534 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.534 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.535 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-08-31T07:37:46.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.535 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.535 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-08-31T07:37:46.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.536 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.536 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-08-31T07:37:46.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.536 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.537 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-08-31T07:37:46.537 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.537 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.537 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.537 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-08-31T07:37:46.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.538 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.538 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-08-31T07:37:46.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.538 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.539 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.539 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-08-31T07:37:46.540 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.540 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.540 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.540 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-08-31T07:37:46.540 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.540 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.541 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.541 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-08-31T07:37:46.541 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.541 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.541 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.541 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-08-31T07:37:46.542 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.542 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.542 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.542 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-08-31T07:37:46.542 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.542 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.542 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.543 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.543 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-08-31T07:37:46.543 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.543 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.543 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.543 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.543 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-08-31T07:37:46.544 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.544 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.544 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.544 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-08-31T07:37:46.544 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.545 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.545 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.545 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.545 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-08-31T07:37:46.545 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.546 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.546 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.546 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-08-31T07:37:46.546 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.546 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.546 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.547 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-08-31T07:37:46.547 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.547 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.547 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.547 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.547 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-08-31T07:37:46.547 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.547 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.548 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.548 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-08-31T07:37:46.548 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.548 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.548 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.549 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-08-31T07:37:46.549 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.549 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.549 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.549 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-08-31T07:37:46.550 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.550 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.550 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.550 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-08-31T07:37:46.550 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.550 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.550 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.551 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-08-31T07:37:46.551 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.551 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.551 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.551 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-08-31T07:37:46.551 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.552 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.552 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.552 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-08-31T07:37:46.552 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.552 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.552 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.553 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-08-31T07:37:46.553 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-08-31T07:37:46.553 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-08-31T07:37:46.553 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.553 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.554 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.554 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:46.554 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.554 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-08-31T07:37:46.554 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 1, 2024-08-31T07:37:46.554 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 2, 2024-08-31T07:37:46.554 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 1, 2024-08-31T07:37:46.555 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 1, 2024-08-31T07:37:46.555 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-08-31T07:37:46.555 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 0, 2024-08-31T07:37:46.555 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 0, 2024-08-31T07:37:46.555 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 2803, 2024-08-31T07:37:46.556 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-08-31T07:37:46.556 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.556 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-08-31T07:37:46.556 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-08-31T07:37:46.556 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-08-31T07:37:46.556 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T07:37:46.557 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-08-31T07:37:46.557 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T07:37:46.557 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-08-31T07:37:46.557 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T07:37:46.557 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T07:37:46.557 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T07:37:46.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T07:37:46.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T07:37:46.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-08-31T07:37:46.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T07:37:46.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T07:37:46.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-08-31T07:37:46.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-08-31T07:37:46.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-08-31T07:37:46.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-08-31T07:37:46.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T07:37:46.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T07:37:46.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T07:37:46.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T07:37:46.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T07:37:46.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-08-31T07:37:46.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-08-31T07:37:46.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-08-31T07:37:46.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T07:37:46.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T07:37:46.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-08-31T07:37:46.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-08-31T07:37:46.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-08-31T07:37:46.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-08-31T07:37:46.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T07:37:46.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-08-31T07:37:46.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T07:37:46.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-08-31T07:37:46.561 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 309081, 2024-08-31T07:37:46.561 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 1922, 2024-08-31T07:37:46.561 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-08-31T07:37:46.562 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-08-31T07:37:46.562 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-08-31T07:37:46.562 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-08-31T07:37:46.562 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-08-31T07:37:46.562 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-08-31T07:37:46.562 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-08-31T07:37:46.562 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-08-31T07:37:46.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 7968, 2024-08-31T07:37:46.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 176, 2024-08-31T07:37:46.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T07:37:46.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-08-31T07:37:46.563 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-08-31T07:37:46.563 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-08-31T07:37:46.563 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 16716797, 2024-08-31T07:37:46.564 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 115240, 2024-08-31T07:37:46.564 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-08-31T07:37:46.564 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-08-31T07:37:46.564 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-08-31T07:37:46.564 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-08-31T07:37:46.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.564 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-08-31T07:37:46.564 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-08-31T07:37:46.565 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-08-31T07:37:46.565 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 5960, 2024-08-31T07:37:46.565 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 21, 2024-08-31T07:37:46.565 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 3, 2024-08-31T07:37:46.565 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 5960, 2024-08-31T07:37:46.565 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-08-31T07:37:46.565 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5960, 2024-08-31T07:37:46.565 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 10.286815258, 2024-08-31T07:37:46.566 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001725975 2024-08-31T07:37:46.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.566 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-08-31T07:37:46.566 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-08-31T07:37:46.566 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5960, 2024-08-31T07:37:46.566 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 5961 2024-08-31T07:37:46.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.566 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 5960, 2024-08-31T07:37:46.567 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 5960, 2024-08-31T07:37:46.567 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 0, 2024-08-31T07:37:46.567 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-08-31T07:37:46.567 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-08-31T07:37:46.567 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 3538, 2024-08-31T07:37:46.567 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 0, 2024-08-31T07:37:46.567 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 2420, 2024-08-31T07:37:46.567 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 0, 2024-08-31T07:37:46.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 0, 2024-08-31T07:37:46.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-08-31T07:37:46.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-08-31T07:37:46.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-08-31T07:37:46.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 0, 2024-08-31T07:37:46.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 0, 2024-08-31T07:37:46.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-08-31T07:37:46.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-08-31T07:37:46.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-08-31T07:37:46.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 1, 2024-08-31T07:37:46.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 0, 2024-08-31T07:37:46.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-08-31T07:37:46.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-08-31T07:37:46.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-08-31T07:37:46.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-08-31T07:37:46.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-08-31T07:37:46.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-08-31T07:37:46.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T07:37:46.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-08-31T07:37:46.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-08-31T07:37:46.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-08-31T07:37:46.570 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-08-31T07:37:46.570 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-08-31T07:37:46.571 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-08-31T07:37:46.571 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-08-31T07:37:46.571 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-08-31T07:37:46.571 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-08-31T07:37:46.571 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-08-31T07:37:46.571 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-08-31T07:37:46.571 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-08-31T07:37:46.571 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-08-31T07:37:46.572 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-08-31T07:37:46.572 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-08-31T07:37:46.572 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-08-31T07:37:46.572 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-08-31T07:37:46.572 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-08-31T07:37:46.572 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-08-31T07:37:46.572 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 118, 2024-08-31T07:37:46.572 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-08-31T07:37:46.572 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 40, 2024-08-31T07:37:46.573 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 11, 2024-08-31T07:37:46.573 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 12, 2024-08-31T07:37:46.573 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 1, 2024-08-31T07:37:46.573 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-08-31T07:37:46.573 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 0, 2024-08-31T07:37:46.573 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 2, 2024-08-31T07:37:46.573 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 0 2024-08-31T07:37:46.573 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.574 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-08-31T07:37:46.574 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 0, 2024-08-31T07:37:46.574 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-08-31T07:37:46.574 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-08-31T07:37:46.574 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-08-31T07:37:46.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.574 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-08-31T07:37:46.575 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-08-31T07:37:46.575 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 0, 2024-08-31T07:37:46.575 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-08-31T07:37:46.575 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 0, 2024-08-31T07:37:46.575 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 0, 2024-08-31T07:37:46.575 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 0, 2024-08-31T07:37:46.575 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-08-31T07:37:46.575 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.576 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T07:37:46.576 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:46.576 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-08-31T07:37:46.576 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:46.576 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 6538, 2024-08-31T07:37:46.576 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 395282439, 2024-08-31T07:37:46.576 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:46.576 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 6538, 2024-08-31T07:37:46.577 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:46.577 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:46.577 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 6538, 2024-08-31T07:37:46.577 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 395282439, 2024-08-31T07:37:46.577 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:46.577 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.577 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.577 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.578 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:46.578 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.578 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-08-31T07:37:46.578 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:46.578 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-08-31T07:37:46.578 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:46.578 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:46.579 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:46.579 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:46.579 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:46.579 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 5960, 2024-08-31T07:37:46.579 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 392841810, 2024-08-31T07:37:46.579 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 5960, 2024-08-31T07:37:46.579 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 392841810, 2024-08-31T07:37:46.579 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:46.580 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.580 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.580 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.580 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:46.580 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.580 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-08-31T07:37:46.580 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:46.580 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-08-31T07:37:46.581 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:46.581 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:46.581 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:46.581 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:46.581 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:46.581 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 5960, 2024-08-31T07:37:46.581 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 5960, 2024-08-31T07:37:46.581 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 5960, 2024-08-31T07:37:46.582 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 5960, 2024-08-31T07:37:46.582 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:46.582 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.582 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.582 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.582 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:46.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.582 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-08-31T07:37:46.583 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:46.583 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-08-31T07:37:46.583 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:46.583 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:46.583 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:46.583 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:46.583 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:46.584 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:46.584 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:46.584 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-08-31T07:37:46.584 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-08-31T07:37:46.584 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:46.584 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.584 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.584 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.585 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:46.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.585 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x55e11ce3c8f0": { 2024-08-31T07:37:46.585 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-08-31T07:37:46.585 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-08-31T07:37:46.585 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-08-31T07:37:46.585 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-08-31T07:37:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-08-31T07:37:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-08-31T07:37:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-08-31T07:37:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-08-31T07:37:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-08-31T07:37:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-08-31T07:37:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-08-31T07:37:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-08-31T07:37:46.586 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-08-31T07:37:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-08-31T07:37:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-08-31T07:37:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-08-31T07:37:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: "trackedop": { 2024-08-31T07:37:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 0 2024-08-31T07:37:46.587 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-08-31T07:37:46.588 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-08-31T07:37:46.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:46 smithi084 ceph-mon[22387]: pgmap v1156: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.6 MiB/s wr, 102 op/s 2024-08-31T07:37:46.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:46 smithi032 ceph-mon[17252]: pgmap v1156: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.6 MiB/s wr, 102 op/s 2024-08-31T07:37:46.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:46 smithi073 ceph-mon[21995]: pgmap v1156: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.6 MiB/s wr, 102 op/s 2024-08-31T07:37:47.131 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-08-31T07:37:47.131 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-08-31T07:37:47.131 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-08-31T07:37:47.131 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-08-31T07:37:47.132 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T07:37:47.132 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-31T07:37:47.143 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-31T07:37:47.143 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:37:47.143 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-31T07:37:47.184 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-08-31T07:37:47.185 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-31T07:37:47.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd blocklist ls 2024-08-31T07:37:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:48.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.341+0000 7fceabe00640 1 -- 172.21.15.32:0/547496198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceac075b70 msgr2=0x7fceac075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.341+0000 7fceabe00640 1 --2- 172.21.15.32:0/547496198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceac075b70 0x7fceac075f80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fce98009900 tx=0x7fce9802a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.341+0000 7fceabe00640 1 -- 172.21.15.32:0/547496198 shutdown_connections 2024-08-31T07:37:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.341+0000 7fceabe00640 1 --2- 172.21.15.32:0/547496198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceac114020 0x7fceac1164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.341+0000 7fceabe00640 1 --2- 172.21.15.32:0/547496198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceac075b70 0x7fceac075f80 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.341+0000 7fceabe00640 1 --2- 172.21.15.32:0/547496198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceac077120 0x7fceac075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.341+0000 7fceabe00640 1 -- 172.21.15.32:0/547496198 >> 172.21.15.32:0/547496198 conn(0x7fceac0ff1e0 msgr2=0x7fceac101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:48.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.341+0000 7fceabe00640 1 -- 172.21.15.32:0/547496198 shutdown_connections 2024-08-31T07:37:48.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.341+0000 7fceabe00640 1 -- 172.21.15.32:0/547496198 wait complete. 2024-08-31T07:37:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.342+0000 7fceabe00640 1 Processor -- start 2024-08-31T07:37:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceabe00640 1 -- start start 2024-08-31T07:37:48.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceabe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceac075b70 0x7fceac1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceabe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceac077120 0x7fceac1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceabe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceac114020 0x7fceac1a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceabe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fceac119cb0 con 0x7fceac114020 2024-08-31T07:37:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceabe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fceac119b30 con 0x7fceac077120 2024-08-31T07:37:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceabe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fceac119e30 con 0x7fceac075b70 2024-08-31T07:37:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceaaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceac075b70 0x7fceac1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceaaa00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceac075b70 0x7fceac1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41222/0 (socket says 172.21.15.32:41222) 2024-08-31T07:37:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceaaa00640 1 -- 172.21.15.32:0/3246718676 learned_addr learned my addr 172.21.15.32:0/3246718676 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.343+0000 7fceab400640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceac114020 0x7fceac1a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.344+0000 7fceaa000640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceac077120 0x7fceac1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.344+0000 7fceaaa00640 1 -- 172.21.15.32:0/3246718676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceac077120 msgr2=0x7fceac1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.344+0000 7fceaaa00640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceac077120 0x7fceac1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.344+0000 7fceaaa00640 1 -- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceac114020 msgr2=0x7fceac1a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.344+0000 7fceaaa00640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceac114020 0x7fceac1a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.344+0000 7fceaaa00640 1 -- 172.21.15.32:0/3246718676 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac1a65f0 con 0x7fceac075b70 2024-08-31T07:37:48.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.344+0000 7fceab400640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceac114020 0x7fceac1a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:37:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.344+0000 7fceaa000640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceac077120 0x7fceac1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T07:37:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.344+0000 7fceaaa00640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceac075b70 0x7fceac1a0680 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00e9f0 tx=0x7fce9c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.344+0000 7fceb0a00640 1 -- 172.21.15.32:0/3246718676 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c00cd80 con 0x7fceac075b70 2024-08-31T07:37:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.345+0000 7fceabe00640 1 -- 172.21.15.32:0/3246718676 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceac1a68e0 con 0x7fceac075b70 2024-08-31T07:37:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.345+0000 7fceb0a00640 1 -- 172.21.15.32:0/3246718676 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9c0054d0 con 0x7fceac075b70 2024-08-31T07:37:48.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.345+0000 7fceabe00640 1 -- 172.21.15.32:0/3246718676 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceac1ad040 con 0x7fceac075b70 2024-08-31T07:37:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.345+0000 7fceb0a00640 1 -- 172.21.15.32:0/3246718676 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c010690 con 0x7fceac075b70 2024-08-31T07:37:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.346+0000 7fceabe00640 1 -- 172.21.15.32:0/3246718676 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce8c005290 con 0x7fceac075b70 2024-08-31T07:37:48.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.349+0000 7fceb0a00640 1 -- 172.21.15.32:0/3246718676 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fce9c0046c0 con 0x7fceac075b70 2024-08-31T07:37:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.350+0000 7fceb0a00640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce900b4d20 0x7fce900b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.350+0000 7fceb0a00640 1 -- 172.21.15.32:0/3246718676 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7fce9c0e58e0 con 0x7fceac075b70 2024-08-31T07:37:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.350+0000 7fceaa000640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce900b4d20 0x7fce900b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.351+0000 7fceaa000640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce900b4d20 0x7fce900b7230 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fce98006010 tx=0x7fce98000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:48.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.352+0000 7fceb0a00640 1 -- 172.21.15.32:0/3246718676 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fce9c0ac6c0 con 0x7fceac075b70 2024-08-31T07:37:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.567+0000 7fceabe00640 1 -- 172.21.15.32:0/3246718676 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fce8c005580 con 0x7fceac075b70 2024-08-31T07:37:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.568+0000 7fceb0a00640 1 -- 172.21.15.32:0/3246718676 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v118) ==== 81+0+883 (secure 0 0 0) 0x7fce9c0abfc0 con 0x7fceac075b70 2024-08-31T07:37:48.570 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/973985469 2024-09-01T07:02:44.661783+0000 2024-08-31T07:37:48.570 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1291326212 2024-09-01T07:01:06.343703+0000 2024-08-31T07:37:48.570 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/717875748 2024-09-01T07:01:06.343703+0000 2024-08-31T07:37:48.570 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/4001745995 2024-09-01T07:02:44.661783+0000 2024-08-31T07:37:48.571 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/717875748 2024-09-01T07:01:06.343703+0000 2024-08-31T07:37:48.571 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2312402120 2024-09-01T07:01:06.343703+0000 2024-08-31T07:37:48.571 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2898888137 2024-09-01T07:01:06.343703+0000 2024-08-31T07:37:48.571 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/115361289 2024-09-01T07:01:35.585062+0000 2024-08-31T07:37:48.571 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2767703591 2024-09-01T07:02:44.661783+0000 2024-08-31T07:37:48.571 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3767202779 2024-09-01T07:01:35.585062+0000 2024-08-31T07:37:48.571 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/376966273 2024-09-01T07:02:44.661783+0000 2024-08-31T07:37:48.571 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/4009368030 2024-09-01T07:01:35.585062+0000 2024-08-31T07:37:48.572 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3157328338 2024-09-01T07:01:35.585062+0000 2024-08-31T07:37:48.572 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/4009368030 2024-09-01T07:01:35.585062+0000 2024-08-31T07:37:48.572 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/4001745995 2024-09-01T07:02:44.661783+0000 2024-08-31T07:37:48.573 INFO:teuthology.orchestra.run.smithi032.stderr:listed 15 entries 2024-08-31T07:37:48.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.570+0000 7fceabe00640 1 -- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce900b4d20 msgr2=0x7fce900b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:48.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.570+0000 7fceabe00640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce900b4d20 0x7fce900b7230 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fce98006010 tx=0x7fce98000bc0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.570+0000 7fceabe00640 1 -- 172.21.15.32:0/3246718676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceac075b70 msgr2=0x7fceac1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.570+0000 7fceabe00640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceac075b70 0x7fceac1a0680 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00e9f0 tx=0x7fce9c00edf0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.570+0000 7fceabe00640 1 -- 172.21.15.32:0/3246718676 shutdown_connections 2024-08-31T07:37:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.570+0000 7fceabe00640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fce900b4d20 0x7fce900b7230 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.570+0000 7fceabe00640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fceac114020 0x7fceac1a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.570+0000 7fceabe00640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fceac077120 0x7fceac1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.570+0000 7fceabe00640 1 --2- 172.21.15.32:0/3246718676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fceac075b70 0x7fceac1a0680 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.570+0000 7fceabe00640 1 -- 172.21.15.32:0/3246718676 >> 172.21.15.32:0/3246718676 conn(0x7fceac0ff1e0 msgr2=0x7fceac10ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.572+0000 7fceabe00640 1 -- 172.21.15.32:0/3246718676 shutdown_connections 2024-08-31T07:37:48.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:48.572+0000 7fceabe00640 1 -- 172.21.15.32:0/3246718676 wait complete. 2024-08-31T07:37:48.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:48 smithi084 ceph-mon[22387]: pgmap v1157: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.8 MiB/s wr, 86 op/s 2024-08-31T07:37:48.774 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:48 smithi032 ceph-mon[17252]: pgmap v1157: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.8 MiB/s wr, 86 op/s 2024-08-31T07:37:48.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:48 smithi073 ceph-mon[21995]: pgmap v1157: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.8 MiB/s wr, 86 op/s 2024-08-31T07:37:49.542 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-31T07:37:49.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:37:49.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-08-31T07:37:49.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:49 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3246718676' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-31T07:37:49.765 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:49 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3246718676' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-31T07:37:49.769 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-08-31T07:37:49.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:37:49.770 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-31T07:37:49.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:37:49.853 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-08-31T07:37:49.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:49 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3246718676' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-31T07:37:49.932 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-31T07:37:49.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:37:49.933 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-31T07:37:50.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T07:37:50.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-08-31T07:37:50.407 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-31T07:37:50.418 INFO:tasks.cephadm:Teardown begin 2024-08-31T07:37:50.419 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-31T07:37:50.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:37:50.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:50 smithi032 ceph-mon[17252]: pgmap v1158: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-08-31T07:37:50.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:50 smithi084 ceph-mon[22387]: pgmap v1158: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-08-31T07:37:50.806 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:50.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:50 smithi073 ceph-mon[21995]: pgmap v1158: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-08-31T07:37:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.528+0000 7f706a800640 1 -- 172.21.15.32:0/3228525737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f706410b290 msgr2=0x7f706410b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.528+0000 7f706a800640 1 --2- 172.21.15.32:0/3228525737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f706410b290 0x7f706410b6a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7050009900 tx=0x7f705002a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.528+0000 7f706a800640 1 -- 172.21.15.32:0/3228525737 shutdown_connections 2024-08-31T07:37:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.528+0000 7f706a800640 1 --2- 172.21.15.32:0/3228525737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f706410bbe0 0x7f7064112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.528+0000 7f706a800640 1 --2- 172.21.15.32:0/3228525737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f706410b290 0x7f706410b6a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.528+0000 7f706a800640 1 --2- 172.21.15.32:0/3228525737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7064105950 0x7f7064105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.528+0000 7f706a800640 1 -- 172.21.15.32:0/3228525737 >> 172.21.15.32:0/3228525737 conn(0x7f7064101420 msgr2=0x7f7064103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.528+0000 7f706a800640 1 -- 172.21.15.32:0/3228525737 shutdown_connections 2024-08-31T07:37:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.528+0000 7f706a800640 1 -- 172.21.15.32:0/3228525737 wait complete. 2024-08-31T07:37:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.529+0000 7f706a800640 1 Processor -- start 2024-08-31T07:37:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f706a800640 1 -- start start 2024-08-31T07:37:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f706a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7064105950 0x7f706419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f706a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706410b290 0x7f706419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f706a800640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f706410bbe0 0x7f70641a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f706a800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f70641158a0 con 0x7f7064105950 2024-08-31T07:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f706a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7064115720 con 0x7f706410b290 2024-08-31T07:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f706a800640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7064115a20 con 0x7f706410bbe0 2024-08-31T07:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f7069400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7064105950 0x7f706419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f7069400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7064105950 0x7f706419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60484/0 (socket says 172.21.15.32:60484) 2024-08-31T07:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f7069400640 1 -- 172.21.15.32:0/3195252157 learned_addr learned my addr 172.21.15.32:0/3195252157 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f7069e00640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f706410bbe0 0x7f70641a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.530+0000 7f7068a00640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706410b290 0x7f706419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.531+0000 7f7069400640 1 -- 172.21.15.32:0/3195252157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f706410bbe0 msgr2=0x7f70641a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.531+0000 7f7069400640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f706410bbe0 0x7f70641a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.531+0000 7f7069400640 1 -- 172.21.15.32:0/3195252157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706410b290 msgr2=0x7f706419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.531+0000 7f7069400640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706410b290 0x7f706419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.531+0000 7f7069400640 1 -- 172.21.15.32:0/3195252157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70641a2220 con 0x7f7064105950 2024-08-31T07:37:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.531+0000 7f7069400640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7064105950 0x7f706419c2b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f705800e9f0 tx=0x7f705800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:51.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.531+0000 7f7062000640 1 -- 172.21.15.32:0/3195252157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705800ccc0 con 0x7f7064105950 2024-08-31T07:37:51.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.531+0000 7f7062000640 1 -- 172.21.15.32:0/3195252157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70580108e0 con 0x7f7064105950 2024-08-31T07:37:51.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.531+0000 7f706a800640 1 -- 172.21.15.32:0/3195252157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70641a2510 con 0x7f7064105950 2024-08-31T07:37:51.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.532+0000 7f7062000640 1 -- 172.21.15.32:0/3195252157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058018880 con 0x7f7064105950 2024-08-31T07:37:51.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.532+0000 7f706a800640 1 -- 172.21.15.32:0/3195252157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70641a8c70 con 0x7f7064105950 2024-08-31T07:37:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.533+0000 7f706a800640 1 -- 172.21.15.32:0/3195252157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7064105d60 con 0x7f7064105950 2024-08-31T07:37:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.535+0000 7f7062000640 1 -- 172.21.15.32:0/3195252157 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7058010430 con 0x7f7064105950 2024-08-31T07:37:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.536+0000 7f7062000640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f704c0b4ca0 0x7f704c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.537+0000 7f7062000640 1 -- 172.21.15.32:0/3195252157 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7f70580e5de0 con 0x7f7064105950 2024-08-31T07:37:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.537+0000 7f7068a00640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f704c0b4ca0 0x7f704c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.538+0000 7f7068a00640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f704c0b4ca0 0x7f704c0b71b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7050009230 tx=0x7f7050031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:51.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.538+0000 7f7062000640 1 -- 172.21.15.32:0/3195252157 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f70580acbc0 con 0x7f7064105950 2024-08-31T07:37:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.772+0000 7f706a800640 1 -- 172.21.15.32:0/3195252157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f706419d3e0 con 0x7f7064105950 2024-08-31T07:37:51.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.773+0000 7f7062000640 1 -- 172.21.15.32:0/3195252157 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+11980 (secure 0 0 0) 0x7f706419d3e0 con 0x7f7064105950 2024-08-31T07:37:51.774 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:37:51.774 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":96,"btime":"2024-08-31T07:37:03:685617+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":96,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:37:03.457928+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:37:51.774 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 96 2024-08-31T07:37:51.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 -- 172.21.15.32:0/3195252157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f704c0b4ca0 msgr2=0x7f704c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:51.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f704c0b4ca0 0x7f704c0b71b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7050009230 tx=0x7f7050031400 comp rx=0 tx=0).stop 2024-08-31T07:37:51.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 -- 172.21.15.32:0/3195252157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7064105950 msgr2=0x7f706419c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:51.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7064105950 0x7f706419c2b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f705800e9f0 tx=0x7f705800edf0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 -- 172.21.15.32:0/3195252157 shutdown_connections 2024-08-31T07:37:51.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f704c0b4ca0 0x7f704c0b71b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f706410bbe0 0x7f70641a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706410b290 0x7f706419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 --2- 172.21.15.32:0/3195252157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7064105950 0x7f706419c2b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:51.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 -- 172.21.15.32:0/3195252157 >> 172.21.15.32:0/3195252157 conn(0x7f7064101420 msgr2=0x7f7064103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:51.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 -- 172.21.15.32:0/3195252157 shutdown_connections 2024-08-31T07:37:51.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:51.775+0000 7f706a800640 1 -- 172.21.15.32:0/3195252157 wait complete. 2024-08-31T07:37:52.682 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T07:37:52.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:52 smithi032 ceph-mon[17252]: pgmap v1159: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2024-08-31T07:37:52.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:52 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/3195252157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:37:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:52 smithi084 ceph-mon[22387]: pgmap v1159: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2024-08-31T07:37:52.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:52 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/3195252157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:37:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:52 smithi073 ceph-mon[21995]: pgmap v1159: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2024-08-31T07:37:52.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:52 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/3195252157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:37:53.064 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:53.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.788+0000 7f05a7400640 1 -- 172.21.15.32:0/2488285695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a0109240 msgr2=0x7f05a0109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:53.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.788+0000 7f05a7400640 1 --2- 172.21.15.32:0/2488285695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a0109240 0x7f05a0109650 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0588009860 tx=0x7f058802a340 comp rx=0 tx=0).stop 2024-08-31T07:37:53.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.788+0000 7f05a7400640 1 -- 172.21.15.32:0/2488285695 shutdown_connections 2024-08-31T07:37:53.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.788+0000 7f05a7400640 1 --2- 172.21.15.32:0/2488285695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a0109b90 0x7f05a01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:53.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.788+0000 7f05a7400640 1 --2- 172.21.15.32:0/2488285695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a0109240 0x7f05a0109650 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:53.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.788+0000 7f05a7400640 1 --2- 172.21.15.32:0/2488285695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a0103900 0x7f05a0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.788+0000 7f05a7400640 1 -- 172.21.15.32:0/2488285695 >> 172.21.15.32:0/2488285695 conn(0x7f05a007b5f0 msgr2=0x7f05a007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.788+0000 7f05a7400640 1 -- 172.21.15.32:0/2488285695 shutdown_connections 2024-08-31T07:37:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.788+0000 7f05a7400640 1 -- 172.21.15.32:0/2488285695 wait complete. 2024-08-31T07:37:53.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.789+0000 7f05a7400640 1 Processor -- start 2024-08-31T07:37:53.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f05a7400640 1 -- start start 2024-08-31T07:37:53.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f05a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a0103900 0x7f05a019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:53.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f05a7400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a0109240 0x7f05a019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f05a7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a0109b90 0x7f05a01a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f05a7400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05a0113850 con 0x7f05a0103900 2024-08-31T07:37:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f05a7400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05a01136d0 con 0x7f05a0109b90 2024-08-31T07:37:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f05a7400640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f05a01139d0 con 0x7f05a0109240 2024-08-31T07:37:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f0597e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a0109240 0x7f05a019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f05a4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a0103900 0x7f05a019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f0597e00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a0109240 0x7f05a019c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:41270/0 (socket says 172.21.15.32:41270) 2024-08-31T07:37:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.790+0000 7f0597e00640 1 -- 172.21.15.32:0/1562753838 learned_addr learned my addr 172.21.15.32:0/1562753838 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:53.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.791+0000 7f05a5400640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a0109b90 0x7f05a01a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:53.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.791+0000 7f0597e00640 1 -- 172.21.15.32:0/1562753838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a0109b90 msgr2=0x7f05a01a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:53.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.791+0000 7f0597e00640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a0109b90 0x7f05a01a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:53.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.791+0000 7f0597e00640 1 -- 172.21.15.32:0/1562753838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a0103900 msgr2=0x7f05a019c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:53.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.791+0000 7f0597e00640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a0103900 0x7f05a019c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:53.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.791+0000 7f0597e00640 1 -- 172.21.15.32:0/1562753838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a01a2390 con 0x7f05a0109240 2024-08-31T07:37:53.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.791+0000 7f0597e00640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a0109240 0x7f05a019c9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0588035040 tx=0x7f0588032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.791+0000 7f059e000640 1 -- 172.21.15.32:0/1562753838 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588040020 con 0x7f05a0109240 2024-08-31T07:37:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.792+0000 7f059e000640 1 -- 172.21.15.32:0/1562753838 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0588030d50 con 0x7f05a0109240 2024-08-31T07:37:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.792+0000 7f05a7400640 1 -- 172.21.15.32:0/1562753838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a01a2620 con 0x7f05a0109240 2024-08-31T07:37:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.792+0000 7f059e000640 1 -- 172.21.15.32:0/1562753838 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588037040 con 0x7f05a0109240 2024-08-31T07:37:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.792+0000 7f05a7400640 1 -- 172.21.15.32:0/1562753838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a01a8de0 con 0x7f05a0109240 2024-08-31T07:37:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.793+0000 7f05a7400640 1 -- 172.21.15.32:0/1562753838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a0103d10 con 0x7f05a0109240 2024-08-31T07:37:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.795+0000 7f059e000640 1 -- 172.21.15.32:0/1562753838 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0588043020 con 0x7f05a0109240 2024-08-31T07:37:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.797+0000 7f059e000640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4d20 0x7f05840b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.797+0000 7f059e000640 1 -- 172.21.15.32:0/1562753838 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7f0588104f50 con 0x7f05a0109240 2024-08-31T07:37:53.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.797+0000 7f05a4a00640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4d20 0x7f05840b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:53.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.797+0000 7f05a4a00640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4d20 0x7f05840b7230 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0590005fd0 tx=0x7f0590009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:53.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:53.798+0000 7f059e000640 1 -- 172.21.15.32:0/1562753838 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f058804a020 con 0x7f05a0109240 2024-08-31T07:37:54.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.030+0000 7f05a7400640 1 -- 172.21.15.32:0/1562753838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05a019d550 con 0x7f05a0109240 2024-08-31T07:37:54.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.031+0000 7f059e000640 1 -- 172.21.15.32:0/1562753838 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+11980 (secure 0 0 0) 0x7f05a019d550 con 0x7f05a0109240 2024-08-31T07:37:54.032 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:37:54.032 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":96,"btime":"2024-08-31T07:37:03:685617+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2918070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2918070851},{"type":"v1","addr":"172.21.15.32:6837","nonce":2918070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/1062605926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":1062605926},{"type":"v1","addr":"172.21.15.32:6839","nonce":1062605926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":96,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T07:09:48.797887+0000","modified":"2024-08-31T07:37:03.457928+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24734,"mds_1":24718,"mds_2":24694,"mds_3":24692,"mds_4":24676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24625":{"gid":24625,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6833/2138862695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2138862695},{"type":"v1","addr":"172.21.15.84:6833","nonce":2138862695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/2003936745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2003936745},{"type":"v1","addr":"172.21.15.73:6833","nonce":2003936745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"j","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6835/1250889023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1250889023},{"type":"v1","addr":"172.21.15.32:6835","nonce":1250889023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/1927490574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1927490574},{"type":"v1","addr":"172.21.15.73:6835","nonce":1927490574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24670":{"gid":24670,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/204703622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":204703622},{"type":"v1","addr":"172.21.15.84:6835","nonce":204703622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.84:6837/4208526312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":4208526312},{"type":"v1","addr":"172.21.15.84:6837","nonce":4208526312}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24692":{"gid":24692,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.73:6837/2735170682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2735170682},{"type":"v1","addr":"172.21.15.73:6837","nonce":2735170682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6839/2807364599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":2807364599},{"type":"v1","addr":"172.21.15.84:6839","nonce":2807364599}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.32:6841/2484505785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2484505785},{"type":"v1","addr":"172.21.15.32:6841","nonce":2484505785}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.73:6839/794507464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":794507464},{"type":"v1","addr":"172.21.15.73:6839","nonce":794507464}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24734,"qdb_cluster":[24676,24692,24718,24694,24734]},"id":1}]} 2024-08-31T07:37:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 96 2024-08-31T07:37:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.033+0000 7f05a7400640 1 -- 172.21.15.32:0/1562753838 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4d20 msgr2=0x7f05840b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.033+0000 7f05a7400640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4d20 0x7f05840b7230 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0590005fd0 tx=0x7f0590009260 comp rx=0 tx=0).stop 2024-08-31T07:37:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.033+0000 7f05a7400640 1 -- 172.21.15.32:0/1562753838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a0109240 msgr2=0x7f05a019c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.033+0000 7f05a7400640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a0109240 0x7f05a019c9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0588035040 tx=0x7f0588032d50 comp rx=0 tx=0).stop 2024-08-31T07:37:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.033+0000 7f05a7400640 1 -- 172.21.15.32:0/1562753838 shutdown_connections 2024-08-31T07:37:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.033+0000 7f05a7400640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f05840b4d20 0x7f05840b7230 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.033+0000 7f05a7400640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05a0109b90 0x7f05a01a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.033+0000 7f05a7400640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05a0109240 0x7f05a019c9c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.033+0000 7f05a7400640 1 --2- 172.21.15.32:0/1562753838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a0103900 0x7f05a019c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.033+0000 7f05a7400640 1 -- 172.21.15.32:0/1562753838 >> 172.21.15.32:0/1562753838 conn(0x7f05a007b5f0 msgr2=0x7f05a01070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:54.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.034+0000 7f05a7400640 1 -- 172.21.15.32:0/1562753838 shutdown_connections 2024-08-31T07:37:54.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:54.034+0000 7f05a7400640 1 -- 172.21.15.32:0/1562753838 wait complete. 2024-08-31T07:37:54.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:54 smithi084 ceph-mon[22387]: pgmap v1160: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 706 KiB/s wr, 40 op/s 2024-08-31T07:37:54.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:54 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1562753838' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:37:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:54 smithi032 ceph-mon[17252]: pgmap v1160: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 706 KiB/s wr, 40 op/s 2024-08-31T07:37:54.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:54 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1562753838' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:37:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:54 smithi073 ceph-mon[21995]: pgmap v1160: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 706 KiB/s wr, 40 op/s 2024-08-31T07:37:54.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:54 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1562753838' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T07:37:55.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T07:37:55.412 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.155+0000 7fda39600640 1 -- 172.21.15.32:0/1249599342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda34105950 msgr2=0x7fda34105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.155+0000 7fda39600640 1 --2- 172.21.15.32:0/1249599342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda34105950 0x7fda34105d60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fda24009900 tx=0x7fda2402a3f0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.155+0000 7fda39600640 1 -- 172.21.15.32:0/1249599342 shutdown_connections 2024-08-31T07:37:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.155+0000 7fda39600640 1 --2- 172.21.15.32:0/1249599342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3410bbe0 0x7fda34112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.155+0000 7fda39600640 1 --2- 172.21.15.32:0/1249599342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda3410b290 0x7fda3410b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.155+0000 7fda39600640 1 --2- 172.21.15.32:0/1249599342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda34105950 0x7fda34105d60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.155+0000 7fda39600640 1 -- 172.21.15.32:0/1249599342 >> 172.21.15.32:0/1249599342 conn(0x7fda34101420 msgr2=0x7fda34103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.155+0000 7fda39600640 1 -- 172.21.15.32:0/1249599342 shutdown_connections 2024-08-31T07:37:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.155+0000 7fda39600640 1 -- 172.21.15.32:0/1249599342 wait complete. 2024-08-31T07:37:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.156+0000 7fda39600640 1 Processor -- start 2024-08-31T07:37:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.156+0000 7fda39600640 1 -- start start 2024-08-31T07:37:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda39600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda34105950 0x7fda3419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda39600640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3410b290 0x7fda3419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda39600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda3410bbe0 0x7fda341a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda39600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda341158a0 con 0x7fda34105950 2024-08-31T07:37:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda39600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda34115720 con 0x7fda3410bbe0 2024-08-31T07:37:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda39600640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fda34115a20 con 0x7fda3410b290 2024-08-31T07:37:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda33e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda34105950 0x7fda3419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda33e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda34105950 0x7fda3419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44772/0 (socket says 172.21.15.32:44772) 2024-08-31T07:37:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda33e00640 1 -- 172.21.15.32:0/4113292066 learned_addr learned my addr 172.21.15.32:0/4113292066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda33400640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3410b290 0x7fda3419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.157+0000 7fda38c00640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda3410bbe0 0x7fda341a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.158+0000 7fda33400640 1 -- 172.21.15.32:0/4113292066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda3410bbe0 msgr2=0x7fda341a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.158+0000 7fda33400640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda3410bbe0 0x7fda341a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.158+0000 7fda33400640 1 -- 172.21.15.32:0/4113292066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda34105950 msgr2=0x7fda3419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.158+0000 7fda33400640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda34105950 0x7fda3419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.158+0000 7fda33400640 1 -- 172.21.15.32:0/4113292066 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda341a2220 con 0x7fda3410b290 2024-08-31T07:37:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.158+0000 7fda33e00640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda34105950 0x7fda3419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.158+0000 7fda33400640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3410b290 0x7fda3419c850 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fda3406c660 tx=0x7fda2000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.158+0000 7fda30c00640 1 -- 172.21.15.32:0/4113292066 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2000cd00 con 0x7fda3410b290 2024-08-31T07:37:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.158+0000 7fda39600640 1 -- 172.21.15.32:0/4113292066 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda341a2510 con 0x7fda3410b290 2024-08-31T07:37:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.158+0000 7fda30c00640 1 -- 172.21.15.32:0/4113292066 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda200035f0 con 0x7fda3410b290 2024-08-31T07:37:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.159+0000 7fda30c00640 1 -- 172.21.15.32:0/4113292066 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20010690 con 0x7fda3410b290 2024-08-31T07:37:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.159+0000 7fda39600640 1 -- 172.21.15.32:0/4113292066 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda341a8c70 con 0x7fda3410b290 2024-08-31T07:37:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.160+0000 7fda39600640 1 -- 172.21.15.32:0/4113292066 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda34105d60 con 0x7fda3410b290 2024-08-31T07:37:56.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.162+0000 7fda30c00640 1 -- 172.21.15.32:0/4113292066 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fda200037a0 con 0x7fda3410b290 2024-08-31T07:37:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.164+0000 7fda30c00640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda180b4d20 0x7fda180b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.164+0000 7fda30c00640 1 -- 172.21.15.32:0/4113292066 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7fda200e5900 con 0x7fda3410b290 2024-08-31T07:37:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.164+0000 7fda33e00640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda180b4d20 0x7fda180b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.165+0000 7fda33e00640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda180b4d20 0x7fda180b7230 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fda24031440 tx=0x7fda240058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.165+0000 7fda30c00640 1 -- 172.21.15.32:0/4113292066 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fda20056130 con 0x7fda3410b290 2024-08-31T07:37:56.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.366+0000 7fda39600640 1 -- 172.21.15.32:0/4113292066 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda3419d3e0 con 0x7fda3410b290 2024-08-31T07:37:56.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.369+0000 7fda30c00640 1 -- 172.21.15.32:0/4113292066 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+23015 (secure 0 0 0) 0x7fda200ac6e0 con 0x7fda3410b290 2024-08-31T07:37:56.370 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-08-31T07:37:56.370 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"fsid":"965fc21a-6766-11ef-bcd4-c7b262605968","created":"2024-08-31T07:00:40.646815+0000","modified":"2024-08-31T07:37:17.098703+0000","last_up_change":"2024-08-31T07:08:50.748048+0000","last_in_change":"2024-08-31T07:08:28.738716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T07:04:45.144128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T07:09:43.795937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T07:09:46.609242+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"91a7f222-d5e9-423d-ba87-fa1278455da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6803","nonce":1563132016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6805","nonce":1563132016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6809","nonce":1563132016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1563132016},{"type":"v1","addr":"172.21.15.32:6807","nonce":1563132016}]},"public_addr":"172.21.15.32:6803/1563132016","cluster_addr":"172.21.15.32:6805/1563132016","heartbeat_back_addr":"172.21.15.32:6809/1563132016","heartbeat_front_addr":"172.21.15.32:6807/1563132016","state":["exists","up"]},{"osd":1,"uuid":"0035b2ca-ad88-42e8-a46b-da03efcc7c04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6801","nonce":4226535850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6803","nonce":4226535850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6807","nonce":4226535850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4226535850},{"type":"v1","addr":"172.21.15.73:6805","nonce":4226535850}]},"public_addr":"172.21.15.73:6801/4226535850","cluster_addr":"172.21.15.73:6803/4226535850","heartbeat_back_addr":"172.21.15.73:6807/4226535850","heartbeat_front_addr":"172.21.15.73:6805/4226535850","state":["exists","up"]},{"osd":2,"uuid":"395e3fdd-1a48-43cc-afad-7af3af791e14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6801","nonce":904624859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6803","nonce":904624859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6807","nonce":904624859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":904624859},{"type":"v1","addr":"172.21.15.84:6805","nonce":904624859}]},"public_addr":"172.21.15.84:6801/904624859","cluster_addr":"172.21.15.84:6803/904624859","heartbeat_back_addr":"172.21.15.84:6807/904624859","heartbeat_front_addr":"172.21.15.84:6805/904624859","state":["exists","up"]},{"osd":3,"uuid":"bfaeb828-d33c-4791-95ec-79d1669b7316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6811","nonce":1969453562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6813","nonce":1969453562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6817","nonce":1969453562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1969453562},{"type":"v1","addr":"172.21.15.32:6815","nonce":1969453562}]},"public_addr":"172.21.15.32:6811/1969453562","cluster_addr":"172.21.15.32:6813/1969453562","heartbeat_back_addr":"172.21.15.32:6817/1969453562","heartbeat_front_addr":"172.21.15.32:6815/1969453562","state":["exists","up"]},{"osd":4,"uuid":"e286ae6e-6d64-452c-b26a-d2aebbfa8742","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6809","nonce":3774619077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6811","nonce":3774619077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6815","nonce":3774619077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3774619077},{"type":"v1","addr":"172.21.15.73:6813","nonce":3774619077}]},"public_addr":"172.21.15.73:6809/3774619077","cluster_addr":"172.21.15.73:6811/3774619077","heartbeat_back_addr":"172.21.15.73:6815/3774619077","heartbeat_front_addr":"172.21.15.73:6813/3774619077","state":["exists","up"]},{"osd":5,"uuid":"98d5b70c-95dd-4bbb-b715-9872c4a4102e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6809","nonce":3489144690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6811","nonce":3489144690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6815","nonce":3489144690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3489144690},{"type":"v1","addr":"172.21.15.84:6813","nonce":3489144690}]},"public_addr":"172.21.15.84:6809/3489144690","cluster_addr":"172.21.15.84:6811/3489144690","heartbeat_back_addr":"172.21.15.84:6815/3489144690","heartbeat_front_addr":"172.21.15.84:6813/3489144690","state":["exists","up"]},{"osd":6,"uuid":"a5e79d62-80ff-4e1b-b031-2ce622f81c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6819","nonce":1095521887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6821","nonce":1095521887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6825","nonce":1095521887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1095521887},{"type":"v1","addr":"172.21.15.32:6823","nonce":1095521887}]},"public_addr":"172.21.15.32:6819/1095521887","cluster_addr":"172.21.15.32:6821/1095521887","heartbeat_back_addr":"172.21.15.32:6825/1095521887","heartbeat_front_addr":"172.21.15.32:6823/1095521887","state":["exists","up"]},{"osd":7,"uuid":"afca1d86-d56c-498f-8086-0114cd88e0c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6817","nonce":2091730330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6819","nonce":2091730330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6823","nonce":2091730330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2091730330},{"type":"v1","addr":"172.21.15.73:6821","nonce":2091730330}]},"public_addr":"172.21.15.73:6817/2091730330","cluster_addr":"172.21.15.73:6819/2091730330","heartbeat_back_addr":"172.21.15.73:6823/2091730330","heartbeat_front_addr":"172.21.15.73:6821/2091730330","state":["exists","up"]},{"osd":8,"uuid":"c43e21ec-53ca-4ed2-a7dd-0a0ae220b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6817","nonce":2957423402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6819","nonce":2957423402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6823","nonce":2957423402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":2957423402},{"type":"v1","addr":"172.21.15.84:6821","nonce":2957423402}]},"public_addr":"172.21.15.84:6817/2957423402","cluster_addr":"172.21.15.84:6819/2957423402","heartbeat_back_addr":"172.21.15.84:6823/2957423402","heartbeat_front_addr":"172.21.15.84:6821/2957423402","state":["exists","up"]},{"osd":9,"uuid":"1d428503-9203-4ddd-b618-114e17fe9f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6827","nonce":2828083479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6829","nonce":2828083479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6833","nonce":2828083479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2828083479},{"type":"v1","addr":"172.21.15.32:6831","nonce":2828083479}]},"public_addr":"172.21.15.32:6827/2828083479","cluster_addr":"172.21.15.32:6829/2828083479","heartbeat_back_addr":"172.21.15.32:6833/2828083479","heartbeat_front_addr":"172.21.15.32:6831/2828083479","state":["exists","up"]},{"osd":10,"uuid":"3e92a61c-4465-44db-813e-d5b731ab2947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6825","nonce":2996715410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6827","nonce":2996715410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6831","nonce":2996715410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2996715410},{"type":"v1","addr":"172.21.15.73:6829","nonce":2996715410}]},"public_addr":"172.21.15.73:6825/2996715410","cluster_addr":"172.21.15.73:6827/2996715410","heartbeat_back_addr":"172.21.15.73:6831/2996715410","heartbeat_front_addr":"172.21.15.73:6829/2996715410","state":["exists","up"]},{"osd":11,"uuid":"2163952e-3884-434b-8d0b-d489ea25c1cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6825","nonce":3154011828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6827","nonce":3154011828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6831","nonce":3154011828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3154011828},{"type":"v1","addr":"172.21.15.84:6829","nonce":3154011828}]},"public_addr":"172.21.15.84:6825/3154011828","cluster_addr":"172.21.15.84:6827/3154011828","heartbeat_back_addr":"172.21.15.84:6831/3154011828","heartbeat_front_addr":"172.21.15.84:6829/3154011828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:03:51.799006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:17.463077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:04:41.908137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:12.062121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:05:38.850178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:03.943269+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:06:35.547949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:01.664127+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:27.166677+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:07:57.990513+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:24.666422+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T07:08:49.572404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/973985469":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/1291326212":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6801/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:6800/4001745995":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/717875748":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2312402120":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/2898888137":"2024-09-01T07:01:06.343703+0000","172.21.15.32:0/115361289":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/2767703591":"2024-09-01T07:02:44.661783+0000","172.21.15.32:0/3767202779":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/376966273":"2024-09-01T07:02:44.661783+0000","172.21.15.32:6800/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:0/3157328338":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4009368030":"2024-09-01T07:01:35.585062+0000","172.21.15.32:6801/4001745995":"2024-09-01T07:02:44.661783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T07:37:56.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.371+0000 7fda39600640 1 -- 172.21.15.32:0/4113292066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda180b4d20 msgr2=0x7fda180b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:56.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.371+0000 7fda39600640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda180b4d20 0x7fda180b7230 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fda24031440 tx=0x7fda240058d0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.371+0000 7fda39600640 1 -- 172.21.15.32:0/4113292066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3410b290 msgr2=0x7fda3419c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:56.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.371+0000 7fda39600640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3410b290 0x7fda3419c850 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fda3406c660 tx=0x7fda2000ee50 comp rx=0 tx=0).stop 2024-08-31T07:37:56.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.372+0000 7fda39600640 1 -- 172.21.15.32:0/4113292066 shutdown_connections 2024-08-31T07:37:56.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.372+0000 7fda39600640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7fda180b4d20 0x7fda180b7230 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.372+0000 7fda39600640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda3410bbe0 0x7fda341a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.372+0000 7fda39600640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda3410b290 0x7fda3419c850 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.372+0000 7fda39600640 1 --2- 172.21.15.32:0/4113292066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda34105950 0x7fda3419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:56.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.372+0000 7fda39600640 1 -- 172.21.15.32:0/4113292066 >> 172.21.15.32:0/4113292066 conn(0x7fda34101420 msgr2=0x7fda34103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:56.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.372+0000 7fda39600640 1 -- 172.21.15.32:0/4113292066 shutdown_connections 2024-08-31T07:37:56.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:56.372+0000 7fda39600640 1 -- 172.21.15.32:0/4113292066 wait complete. 2024-08-31T07:37:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:56 smithi084 ceph-mon[22387]: pgmap v1161: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 449 KiB/s wr, 32 op/s 2024-08-31T07:37:56.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:56 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/4113292066' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:37:56.784 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:56 smithi032 ceph-mon[17252]: pgmap v1161: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 449 KiB/s wr, 32 op/s 2024-08-31T07:37:56.784 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:56 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/4113292066' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:37:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:56 smithi073 ceph-mon[21995]: pgmap v1161: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 449 KiB/s wr, 32 op/s 2024-08-31T07:37:56.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:56 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/4113292066' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T07:37:57.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs fail cephfs 2024-08-31T07:37:57.598 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:57.739 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:57 smithi032 ceph-mon[17252]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:37:57.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:57 smithi084 ceph-mon[22387]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:37:57.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:57 smithi073 ceph-mon[21995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-31T07:37:58.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.302+0000 7f8eb3000640 1 -- 172.21.15.32:0/2068948738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eac109070 msgr2=0x7f8eac109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:58.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.302+0000 7f8eb3000640 1 --2- 172.21.15.32:0/2068948738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eac109070 0x7f8eac109480 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009900 tx=0x7f8e9802a410 comp rx=0 tx=0).stop 2024-08-31T07:37:58.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.302+0000 7f8eb3000640 1 -- 172.21.15.32:0/2068948738 shutdown_connections 2024-08-31T07:37:58.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.302+0000 7f8eb3000640 1 --2- 172.21.15.32:0/2068948738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eac1099c0 0x7f8eac1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:58.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.302+0000 7f8eb3000640 1 --2- 172.21.15.32:0/2068948738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eac109070 0x7f8eac109480 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:58.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.302+0000 7f8eb3000640 1 --2- 172.21.15.32:0/2068948738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eac103730 0x7f8eac103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:58.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.302+0000 7f8eb3000640 1 -- 172.21.15.32:0/2068948738 >> 172.21.15.32:0/2068948738 conn(0x7f8eac0ff180 msgr2=0x7f8eac1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:58.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.302+0000 7f8eb3000640 1 -- 172.21.15.32:0/2068948738 shutdown_connections 2024-08-31T07:37:58.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.302+0000 7f8eb3000640 1 -- 172.21.15.32:0/2068948738 wait complete. 2024-08-31T07:37:58.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.303+0000 7f8eb3000640 1 Processor -- start 2024-08-31T07:37:58.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.303+0000 7f8eb3000640 1 -- start start 2024-08-31T07:37:58.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb3000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eac103730 0x7f8eac19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:58.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb3000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eac109070 0x7f8eac19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:58.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb3000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eac1099c0 0x7f8eac19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:58.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb3000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac113630 con 0x7f8eac1099c0 2024-08-31T07:37:58.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb3000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac1134b0 con 0x7f8eac109070 2024-08-31T07:37:58.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb3000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac1137b0 con 0x7f8eac103730 2024-08-31T07:37:58.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb2600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eac1099c0 0x7f8eac19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:58.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb1c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eac103730 0x7f8eac19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:58.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb1c00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eac103730 0x7f8eac19a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.32:50664/0 (socket says 172.21.15.32:50664) 2024-08-31T07:37:58.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eac109070 0x7f8eac19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:58.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.304+0000 7f8eb1c00640 1 -- 172.21.15.32:0/580436365 learned_addr learned my addr 172.21.15.32:0/580436365 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:37:58.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eb2600640 1 -- 172.21.15.32:0/580436365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eac103730 msgr2=0x7f8eac19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:58.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eb2600640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eac103730 0x7f8eac19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:58.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eb2600640 1 -- 172.21.15.32:0/580436365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eac109070 msgr2=0x7f8eac19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:58.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eb2600640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eac109070 0x7f8eac19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:58.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eb2600640 1 -- 172.21.15.32:0/580436365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eac19ffd0 con 0x7f8eac1099c0 2024-08-31T07:37:58.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eb1c00640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eac103730 0x7f8eac19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:37:58.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eb2600640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eac1099c0 0x7f8eac19f8a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009fd0 tx=0x7f8ea400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:58.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eaa000640 1 -- 172.21.15.32:0/580436365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea400ce20 con 0x7f8eac1099c0 2024-08-31T07:37:58.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eaa000640 1 -- 172.21.15.32:0/580436365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea4005020 con 0x7f8eac1099c0 2024-08-31T07:37:58.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eb3000640 1 -- 172.21.15.32:0/580436365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eac1a02c0 con 0x7f8eac1099c0 2024-08-31T07:37:58.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.305+0000 7f8eaa000640 1 -- 172.21.15.32:0/580436365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4010690 con 0x7f8eac1099c0 2024-08-31T07:37:58.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.306+0000 7f8eb3000640 1 -- 172.21.15.32:0/580436365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eac1a6a20 con 0x7f8eac1099c0 2024-08-31T07:37:58.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.307+0000 7f8eb3000640 1 -- 172.21.15.32:0/580436365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eac103b40 con 0x7f8eac1099c0 2024-08-31T07:37:58.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.309+0000 7f8eaa000640 1 -- 172.21.15.32:0/580436365 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8ea40054d0 con 0x7f8eac1099c0 2024-08-31T07:37:58.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.310+0000 7f8eaa000640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e940b4ca0 0x7f8e940b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:37:58.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.311+0000 7f8eaa000640 1 -- 172.21.15.32:0/580436365 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8441+0+0 (secure 0 0 0) 0x7f8ea40e5c00 con 0x7f8eac1099c0 2024-08-31T07:37:58.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.311+0000 7f8eb1c00640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e940b4ca0 0x7f8e940b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:37:58.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.311+0000 7f8eb1c00640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e940b4ca0 0x7f8e940b71b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0011550 tx=0x7f8ea0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:37:58.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.312+0000 7f8eaa000640 1 -- 172.21.15.32:0/580436365 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8ea40ac9e0 con 0x7f8eac1099c0 2024-08-31T07:37:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.547+0000 7f8eb3000640 1 -- 172.21.15.32:0/580436365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8eac19b190 con 0x7f8eac1099c0 2024-08-31T07:37:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.550+0000 7f8eaa000640 1 -- 172.21.15.32:0/580436365 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) ==== 161+0+0 (secure 0 0 0) 0x7f8eac19b190 con 0x7f8eac1099c0 2024-08-31T07:37:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-31T07:37:58.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.554+0000 7f8eb3000640 1 -- 172.21.15.32:0/580436365 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e940b4ca0 msgr2=0x7f8e940b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:58.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.554+0000 7f8eb3000640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e940b4ca0 0x7f8e940b71b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0011550 tx=0x7f8ea0005f90 comp rx=0 tx=0).stop 2024-08-31T07:37:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.554+0000 7f8eb3000640 1 -- 172.21.15.32:0/580436365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eac1099c0 msgr2=0x7f8eac19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:37:58.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.554+0000 7f8eb3000640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eac1099c0 0x7f8eac19f8a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009fd0 tx=0x7f8ea400ef10 comp rx=0 tx=0).stop 2024-08-31T07:37:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.555+0000 7f8eb3000640 1 -- 172.21.15.32:0/580436365 shutdown_connections 2024-08-31T07:37:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.555+0000 7f8eb3000640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f8e940b4ca0 0x7f8e940b71b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.555+0000 7f8eb3000640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eac1099c0 0x7f8eac19f8a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.555+0000 7f8eb3000640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8eac109070 0x7f8eac19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:58.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.555+0000 7f8eb3000640 1 --2- 172.21.15.32:0/580436365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8eac103730 0x7f8eac19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:37:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.555+0000 7f8eb3000640 1 -- 172.21.15.32:0/580436365 >> 172.21.15.32:0/580436365 conn(0x7f8eac0ff180 msgr2=0x7f8eac1015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:37:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.555+0000 7f8eb3000640 1 -- 172.21.15.32:0/580436365 shutdown_connections 2024-08-31T07:37:58.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:58 smithi032 ceph-mon[17252]: pgmap v1162: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 258 KiB/s wr, 23 op/s 2024-08-31T07:37:58.557 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:58 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a[17248]: 2024-08-31T07:37:58.548+0000 7f6cd3400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-31T07:37:58.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:37:58.556+0000 7f8eb3000640 1 -- 172.21.15.32:0/580436365 wait complete. 2024-08-31T07:37:58.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-mon[22387]: pgmap v1162: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 258 KiB/s wr, 23 op/s 2024-08-31T07:37:58.858 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:37:58 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a[61919]: ignoring --setuser ceph since I am not root 2024-08-31T07:37:58.858 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:37:58 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a[61919]: ignoring --setgroup ceph since I am not root 2024-08-31T07:37:58.858 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:37:58 smithi032 ceph-mds[61941]: -- 172.21.15.32:0/4029941398 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559ccc52bc20 con 0x559cd06d1a80 2024-08-31T07:37:58.858 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:37:58 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a[61919]: starting mds.a at 2024-08-31T07:37:58.859 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:37:58 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j[60367]: ignoring --setuser ceph since I am not root 2024-08-31T07:37:58.859 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:37:58 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j[60367]: ignoring --setgroup ceph since I am not root 2024-08-31T07:37:58.859 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:37:58 smithi032 ceph-mds[60384]: -- 172.21.15.32:0/2069458489 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557aaa57f860 con 0x557aae725180 2024-08-31T07:37:58.859 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:37:58 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j[60367]: starting mds.j at 2024-08-31T07:37:58.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-mon[21995]: pgmap v1162: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 258 KiB/s wr, 23 op/s 2024-08-31T07:37:58.930 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k[52246]: ignoring --setuser ceph since I am not root 2024-08-31T07:37:58.930 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k[52246]: ignoring --setgroup ceph since I am not root 2024-08-31T07:37:58.930 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-mds[52250]: -- 172.21.15.73:0/2927190887 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557225391c20 con 0x557229537180 2024-08-31T07:37:58.930 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k[52246]: starting mds.k at 2024-08-31T07:37:58.931 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-h[52859]: ignoring --setuser ceph since I am not root 2024-08-31T07:37:58.931 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-h[52859]: ignoring --setgroup ceph since I am not root 2024-08-31T07:37:58.931 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-mds[52863]: -- 172.21.15.73:0/3155705035 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f9e013860 con 0x557fa21b9180 2024-08-31T07:37:58.931 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-h[52859]: starting mds.h at 2024-08-31T07:37:58.931 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b[53169]: ignoring --setuser ceph since I am not root 2024-08-31T07:37:58.932 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b[53169]: ignoring --setgroup ceph since I am not root 2024-08-31T07:37:58.932 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-mds[53173]: -- 172.21.15.73:0/2543870867 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fb1c8b9c20 con 0x55fb20a5fa80 2024-08-31T07:37:58.932 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b[53169]: starting mds.b at 2024-08-31T07:37:58.932 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e[52560]: ignoring --setuser ceph since I am not root 2024-08-31T07:37:58.932 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e[52560]: ignoring --setgroup ceph since I am not root 2024-08-31T07:37:58.932 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-mds[52564]: -- 172.21.15.73:0/834408099 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5617faaa1860 con 0x5617fec47180 2024-08-31T07:37:58.932 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:37:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e[52560]: starting mds.e at 2024-08-31T07:37:59.240 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f[53896]: ignoring --setuser ceph since I am not root 2024-08-31T07:37:59.241 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f[53896]: ignoring --setgroup ceph since I am not root 2024-08-31T07:37:59.241 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-mds[53900]: -- 172.21.15.84:0/332520648 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f0741d860 con 0x562f0b5c3600 2024-08-31T07:37:59.241 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f[53896]: starting mds.f at 2024-08-31T07:37:59.241 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i[54504]: ignoring --setuser ceph since I am not root 2024-08-31T07:37:59.242 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i[54504]: ignoring --setgroup ceph since I am not root 2024-08-31T07:37:59.242 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-mds[54508]: -- 172.21.15.84:0/726676120 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626b83af860 con 0x5626bc555180 2024-08-31T07:37:59.242 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i[54504]: starting mds.i at 2024-08-31T07:37:59.242 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c[54193]: ignoring --setuser ceph since I am not root 2024-08-31T07:37:59.242 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c[54193]: ignoring --setgroup ceph since I am not root 2024-08-31T07:37:59.242 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-mds[54197]: -- 172.21.15.84:0/540828543 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f607a2d860 con 0x55f60bbd3180 2024-08-31T07:37:59.243 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c[54193]: starting mds.c at 2024-08-31T07:37:59.243 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l[53595]: ignoring --setuser ceph since I am not root 2024-08-31T07:37:59.243 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l[53595]: ignoring --setgroup ceph since I am not root 2024-08-31T07:37:59.243 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-mds[53599]: -- 172.21.15.84:0/1961618995 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a0e72b5860 con 0x55a0eb45b180 2024-08-31T07:37:59.243 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:37:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l[53595]: starting mds.l at 2024-08-31T07:37:59.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-31T07:37:59.607 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:59 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/580436365' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-31T07:37:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:59 smithi032 ceph-mon[17252]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-31T07:37:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:59 smithi032 ceph-mon[17252]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-31T07:37:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:59 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/580436365' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-31T07:37:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:59 smithi032 ceph-mon[17252]: osdmap e119: 12 total, 12 up, 12 in 2024-08-31T07:37:59.608 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:37:59 smithi032 ceph-mon[17252]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-31T07:37:59.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:59 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/580436365' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-31T07:37:59.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:59 smithi084 ceph-mon[22387]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-31T07:37:59.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:59 smithi084 ceph-mon[22387]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-31T07:37:59.741 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:59 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/580436365' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-31T07:37:59.745 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:59 smithi084 ceph-mon[22387]: osdmap e119: 12 total, 12 up, 12 in 2024-08-31T07:37:59.745 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:37:59 smithi084 ceph-mon[22387]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-31T07:37:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:37:59.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:59 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/580436365' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-31T07:37:59.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:59 smithi073 ceph-mon[21995]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-31T07:37:59.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:59 smithi073 ceph-mon[21995]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-31T07:37:59.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:59 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/580436365' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-31T07:37:59.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:59 smithi073 ceph-mon[21995]: osdmap e119: 12 total, 12 up, 12 in 2024-08-31T07:37:59.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:37:59 smithi073 ceph-mon[21995]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-31T07:38:00.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.460+0000 7ff675a00640 1 -- 172.21.15.32:0/2466299945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff670109b90 msgr2=0x7ff6701103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:00.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.460+0000 7ff675a00640 1 --2- 172.21.15.32:0/2466299945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff670109b90 0x7ff6701103c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff668009900 tx=0x7ff66802a410 comp rx=0 tx=0).stop 2024-08-31T07:38:00.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.460+0000 7ff675a00640 1 -- 172.21.15.32:0/2466299945 shutdown_connections 2024-08-31T07:38:00.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.460+0000 7ff675a00640 1 --2- 172.21.15.32:0/2466299945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff670109b90 0x7ff6701103c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:00.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.460+0000 7ff675a00640 1 --2- 172.21.15.32:0/2466299945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff670109240 0x7ff670109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:00.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.460+0000 7ff675a00640 1 --2- 172.21.15.32:0/2466299945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff670103900 0x7ff670103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:00.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.460+0000 7ff675a00640 1 -- 172.21.15.32:0/2466299945 >> 172.21.15.32:0/2466299945 conn(0x7ff67007b5f0 msgr2=0x7ff67007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:38:00.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.460+0000 7ff675a00640 1 -- 172.21.15.32:0/2466299945 shutdown_connections 2024-08-31T07:38:00.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.460+0000 7ff675a00640 1 -- 172.21.15.32:0/2466299945 wait complete. 2024-08-31T07:38:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.461+0000 7ff675a00640 1 Processor -- start 2024-08-31T07:38:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.461+0000 7ff675a00640 1 -- start start 2024-08-31T07:38:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.462+0000 7ff675a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff670103900 0x7ff67019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.462+0000 7ff675a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff670109240 0x7ff67019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.462+0000 7ff675a00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff670109b90 0x7ff6701a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.462+0000 7ff675a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701138a0 con 0x7ff670109240 2024-08-31T07:38:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.462+0000 7ff675a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff670113720 con 0x7ff670103900 2024-08-31T07:38:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.462+0000 7ff675a00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff670113a20 con 0x7ff670109b90 2024-08-31T07:38:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.462+0000 7ff66e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff670109240 0x7ff67019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.462+0000 7ff66e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff670109240 0x7ff67019c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44858/0 (socket says 172.21.15.32:44858) 2024-08-31T07:38:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.462+0000 7ff66ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff670103900 0x7ff67019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.462+0000 7ff66e000640 1 -- 172.21.15.32:0/2801396205 learned_addr learned my addr 172.21.15.32:0/2801396205 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:38:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff66f400640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff670109b90 0x7ff6701a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff66e000640 1 -- 172.21.15.32:0/2801396205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff670109b90 msgr2=0x7ff6701a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff66e000640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff670109b90 0x7ff6701a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff66e000640 1 -- 172.21.15.32:0/2801396205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff670103900 msgr2=0x7ff67019c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff66e000640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff670103900 0x7ff67019c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff66e000640 1 -- 172.21.15.32:0/2801396205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6701a2300 con 0x7ff670109240 2024-08-31T07:38:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff66f400640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff670109b90 0x7ff6701a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T07:38:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff66ea00640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff670103900 0x7ff67019c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T07:38:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff66e000640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff670109240 0x7ff67019c930 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00e9f0 tx=0x7ff65c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:38:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff667400640 1 -- 172.21.15.32:0/2801396205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00ccc0 con 0x7ff670109240 2024-08-31T07:38:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff667400640 1 -- 172.21.15.32:0/2801396205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff65c0108e0 con 0x7ff670109240 2024-08-31T07:38:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff675a00640 1 -- 172.21.15.32:0/2801396205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6701a25f0 con 0x7ff670109240 2024-08-31T07:38:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff667400640 1 -- 172.21.15.32:0/2801396205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c018860 con 0x7ff670109240 2024-08-31T07:38:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.463+0000 7ff675a00640 1 -- 172.21.15.32:0/2801396205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6701a8d50 con 0x7ff670109240 2024-08-31T07:38:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.465+0000 7ff675a00640 1 -- 172.21.15.32:0/2801396205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff670103d10 con 0x7ff670109240 2024-08-31T07:38:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.470+0000 7ff667400640 1 -- 172.21.15.32:0/2801396205 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff65c010430 con 0x7ff670109240 2024-08-31T07:38:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.471+0000 7ff667400640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff6540b4d20 0x7ff6540b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.472+0000 7ff667400640 1 -- 172.21.15.32:0/2801396205 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8827+0+0 (secure 0 0 0) 0x7ff65c0e5eb0 con 0x7ff670109240 2024-08-31T07:38:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.472+0000 7ff66ea00640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff6540b4d20 0x7ff6540b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.473+0000 7ff66ea00640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff6540b4d20 0x7ff6540b7230 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff660005fd0 tx=0x7ff660009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:38:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.473+0000 7ff667400640 1 -- 172.21.15.32:0/2801396205 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff65c0acb10 con 0x7ff670109240 2024-08-31T07:38:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:00.712+0000 7ff675a00640 1 -- 172.21.15.32:0/2801396205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff6701a8fe0 con 0x7ff670109240 2024-08-31T07:38:00.831 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: pgmap v1164: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 257 KiB/s wr, 20 op/s 2024-08-31T07:38:00.831 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.73:6832/3557091371,v1:172.21.15.73:6834/3557091371] up:boot 2024-08-31T07:38:00.831 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.73:6833/3228997239,v1:172.21.15.73:6835/3228997239] up:boot 2024-08-31T07:38:00.831 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.73:6836/2047428626,v1:172.21.15.73:6838/2047428626] up:boot 2024-08-31T07:38:00.831 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.73:6837/2166301717,v1:172.21.15.73:6839/2166301717] up:boot 2024-08-31T07:38:00.832 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.32:6834/2675080673,v1:172.21.15.32:6835/2675080673] up:boot 2024-08-31T07:38:00.832 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.32:6840/2948940604,v1:172.21.15.32:6841/2948940604] up:boot 2024-08-31T07:38:00.832 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.84:6833/2112240714,v1:172.21.15.84:6835/2112240714] up:boot 2024-08-31T07:38:00.832 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.84:6832/373934538,v1:172.21.15.84:6834/373934538] up:boot 2024-08-31T07:38:00.832 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.84:6836/1481317008,v1:172.21.15.84:6838/1481317008] up:boot 2024-08-31T07:38:00.832 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: mds.? [v2:172.21.15.84:6837/2115844671,v1:172.21.15.84:6839/2115844671] up:boot 2024-08-31T07:38:00.832 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-31T07:38:00.832 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-31T07:38:00.833 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T07:38:00.833 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T07:38:00.833 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-31T07:38:00.833 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-31T07:38:00.833 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-31T07:38:00.833 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T07:38:00.833 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T07:38:00.834 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T07:38:00.834 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:00 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-31T07:38:00.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: pgmap v1164: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 257 KiB/s wr, 20 op/s 2024-08-31T07:38:00.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.73:6832/3557091371,v1:172.21.15.73:6834/3557091371] up:boot 2024-08-31T07:38:00.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.73:6833/3228997239,v1:172.21.15.73:6835/3228997239] up:boot 2024-08-31T07:38:00.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.73:6836/2047428626,v1:172.21.15.73:6838/2047428626] up:boot 2024-08-31T07:38:00.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.73:6837/2166301717,v1:172.21.15.73:6839/2166301717] up:boot 2024-08-31T07:38:00.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.32:6834/2675080673,v1:172.21.15.32:6835/2675080673] up:boot 2024-08-31T07:38:00.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.32:6840/2948940604,v1:172.21.15.32:6841/2948940604] up:boot 2024-08-31T07:38:00.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.84:6833/2112240714,v1:172.21.15.84:6835/2112240714] up:boot 2024-08-31T07:38:00.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.84:6832/373934538,v1:172.21.15.84:6834/373934538] up:boot 2024-08-31T07:38:00.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.84:6836/1481317008,v1:172.21.15.84:6838/1481317008] up:boot 2024-08-31T07:38:00.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: mds.? [v2:172.21.15.84:6837/2115844671,v1:172.21.15.84:6839/2115844671] up:boot 2024-08-31T07:38:00.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-31T07:38:00.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-31T07:38:00.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T07:38:00.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T07:38:00.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-31T07:38:00.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-31T07:38:00.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-31T07:38:00.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T07:38:00.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T07:38:00.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T07:38:00.860 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:00 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-31T07:38:00.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: pgmap v1164: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 257 KiB/s wr, 20 op/s 2024-08-31T07:38:00.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.73:6832/3557091371,v1:172.21.15.73:6834/3557091371] up:boot 2024-08-31T07:38:00.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.73:6833/3228997239,v1:172.21.15.73:6835/3228997239] up:boot 2024-08-31T07:38:00.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.73:6836/2047428626,v1:172.21.15.73:6838/2047428626] up:boot 2024-08-31T07:38:00.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.73:6837/2166301717,v1:172.21.15.73:6839/2166301717] up:boot 2024-08-31T07:38:00.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.32:6834/2675080673,v1:172.21.15.32:6835/2675080673] up:boot 2024-08-31T07:38:00.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.32:6840/2948940604,v1:172.21.15.32:6841/2948940604] up:boot 2024-08-31T07:38:00.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.84:6833/2112240714,v1:172.21.15.84:6835/2112240714] up:boot 2024-08-31T07:38:00.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.84:6832/373934538,v1:172.21.15.84:6834/373934538] up:boot 2024-08-31T07:38:00.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.84:6836/1481317008,v1:172.21.15.84:6838/1481317008] up:boot 2024-08-31T07:38:00.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: mds.? [v2:172.21.15.84:6837/2115844671,v1:172.21.15.84:6839/2115844671] up:boot 2024-08-31T07:38:00.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-31T07:38:00.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-31T07:38:00.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T07:38:00.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T07:38:00.992 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-31T07:38:00.993 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-31T07:38:00.993 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-31T07:38:00.993 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T07:38:00.993 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T07:38:00.993 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T07:38:00.993 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:00 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-31T07:38:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.564+0000 7ff667400640 1 -- 172.21.15.32:0/2801396205 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v100) ==== 104+0+0 (secure 0 0 0) 0x7ff65c0ac410 con 0x7ff670109240 2024-08-31T07:38:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.567+0000 7ff675a00640 1 -- 172.21.15.32:0/2801396205 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff6540b4d20 msgr2=0x7ff6540b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.567+0000 7ff675a00640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff6540b4d20 0x7ff6540b7230 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff660005fd0 tx=0x7ff660009500 comp rx=0 tx=0).stop 2024-08-31T07:38:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.567+0000 7ff675a00640 1 -- 172.21.15.32:0/2801396205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff670109240 msgr2=0x7ff67019c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.567+0000 7ff675a00640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff670109240 0x7ff67019c930 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00e9f0 tx=0x7ff65c00eeb0 comp rx=0 tx=0).stop 2024-08-31T07:38:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.567+0000 7ff675a00640 1 -- 172.21.15.32:0/2801396205 shutdown_connections 2024-08-31T07:38:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.567+0000 7ff675a00640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7ff6540b4d20 0x7ff6540b7230 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.567+0000 7ff675a00640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff670109b90 0x7ff6701a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.568+0000 7ff675a00640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff670109240 0x7ff67019c930 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.568+0000 7ff675a00640 1 --2- 172.21.15.32:0/2801396205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff670103900 0x7ff67019c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.568+0000 7ff675a00640 1 -- 172.21.15.32:0/2801396205 >> 172.21.15.32:0/2801396205 conn(0x7ff67007b5f0 msgr2=0x7ff670104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:38:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.568+0000 7ff675a00640 1 -- 172.21.15.32:0/2801396205 shutdown_connections 2024-08-31T07:38:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:01.568+0000 7ff675a00640 1 -- 172.21.15.32:0/2801396205 wait complete. 2024-08-31T07:38:01.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:01 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2801396205' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-31T07:38:01.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:01 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:38:01.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:01 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2801396205' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-31T07:38:02.009 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:01 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:38:02.010 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:01 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2801396205' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-31T07:38:02.010 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:01 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T07:38:02.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-31T07:38:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:02 smithi032 ceph-mon[17252]: pgmap v1165: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 214 KiB/s wr, 20 op/s 2024-08-31T07:38:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:02 smithi032 ceph-mon[17252]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-31T07:38:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:02 smithi032 ceph-mon[17252]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-31T07:38:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:02 smithi032 ceph-mon[17252]: Cluster is now healthy 2024-08-31T07:38:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:02 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/2801396205' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-31T07:38:02.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:02 smithi032 ceph-mon[17252]: fsmap 12 up:standby 2024-08-31T07:38:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:02 smithi073 ceph-mon[21995]: pgmap v1165: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 214 KiB/s wr, 20 op/s 2024-08-31T07:38:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:02 smithi073 ceph-mon[21995]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-31T07:38:02.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:02 smithi073 ceph-mon[21995]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-31T07:38:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:02 smithi073 ceph-mon[21995]: Cluster is now healthy 2024-08-31T07:38:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:02 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/2801396205' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-31T07:38:02.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:02 smithi073 ceph-mon[21995]: fsmap 12 up:standby 2024-08-31T07:38:02.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:02 smithi084 ceph-mon[22387]: pgmap v1165: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 214 KiB/s wr, 20 op/s 2024-08-31T07:38:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:02 smithi084 ceph-mon[22387]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-31T07:38:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:02 smithi084 ceph-mon[22387]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-31T07:38:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:02 smithi084 ceph-mon[22387]: Cluster is now healthy 2024-08-31T07:38:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:02 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/2801396205' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-31T07:38:02.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:02 smithi084 ceph-mon[22387]: fsmap 12 up:standby 2024-08-31T07:38:03.923 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:38:04.857 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:04 smithi032 ceph-mon[17252]: pgmap v1166: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 100 KiB/s wr, 15 op/s 2024-08-31T07:38:04.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:04 smithi073 ceph-mon[21995]: pgmap v1166: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 100 KiB/s wr, 15 op/s 2024-08-31T07:38:04.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:04 smithi084 ceph-mon[22387]: pgmap v1166: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 100 KiB/s wr, 15 op/s 2024-08-31T07:38:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.577+0000 7f9bccc00640 1 -- 172.21.15.32:0/1823182475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80754f0 msgr2=0x7f9bc8075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.577+0000 7f9bccc00640 1 --2- 172.21.15.32:0/1823182475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80754f0 0x7f9bc8075900 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009ca0 tx=0x7f9bb802a8c0 comp rx=0 tx=0).stop 2024-08-31T07:38:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.577+0000 7f9bccc00640 1 -- 172.21.15.32:0/1823182475 shutdown_connections 2024-08-31T07:38:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.577+0000 7f9bccc00640 1 --2- 172.21.15.32:0/1823182475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc810fcf0 0x7f9bc8112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.577+0000 7f9bccc00640 1 --2- 172.21.15.32:0/1823182475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bc8075e40 0x7f9bc810f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.577+0000 7f9bccc00640 1 --2- 172.21.15.32:0/1823182475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80754f0 0x7f9bc8075900 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.577+0000 7f9bccc00640 1 -- 172.21.15.32:0/1823182475 >> 172.21.15.32:0/1823182475 conn(0x7f9bc80ff180 msgr2=0x7f9bc81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:38:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.577+0000 7f9bccc00640 1 -- 172.21.15.32:0/1823182475 shutdown_connections 2024-08-31T07:38:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.577+0000 7f9bccc00640 1 -- 172.21.15.32:0/1823182475 wait complete. 2024-08-31T07:38:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bccc00640 1 Processor -- start 2024-08-31T07:38:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bccc00640 1 -- start start 2024-08-31T07:38:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bccc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80754f0 0x7f9bc819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bccc00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc8075e40 0x7f9bc819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bccc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bc810fcf0 0x7f9bc81a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bccc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc81158b0 con 0x7f9bc80754f0 2024-08-31T07:38:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bccc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8115730 con 0x7f9bc810fcf0 2024-08-31T07:38:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bccc00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8115a30 con 0x7f9bc8075e40 2024-08-31T07:38:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bc7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80754f0 0x7f9bc819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bc7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80754f0 0x7f9bc819c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44870/0 (socket says 172.21.15.32:44870) 2024-08-31T07:38:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.579+0000 7f9bc7400640 1 -- 172.21.15.32:0/1065237009 learned_addr learned my addr 172.21.15.32:0/1065237009 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:38:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.580+0000 7f9bc7e00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bc810fcf0 0x7f9bc81a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.580+0000 7f9bc6a00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc8075e40 0x7f9bc819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.580+0000 7f9bc7400640 1 -- 172.21.15.32:0/1065237009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc8075e40 msgr2=0x7f9bc819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.580+0000 7f9bc7400640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc8075e40 0x7f9bc819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.580+0000 7f9bc7400640 1 -- 172.21.15.32:0/1065237009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bc810fcf0 msgr2=0x7f9bc81a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.580+0000 7f9bc7400640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bc810fcf0 0x7f9bc81a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.580+0000 7f9bc7400640 1 -- 172.21.15.32:0/1065237009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc81a21f0 con 0x7f9bc80754f0 2024-08-31T07:38:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.580+0000 7f9bc7400640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80754f0 0x7f9bc819c280 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8005e30 tx=0x7f9bb8031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:38:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.581+0000 7f9bbfe00640 1 -- 172.21.15.32:0/1065237009 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8003b70 con 0x7f9bc80754f0 2024-08-31T07:38:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.581+0000 7f9bccc00640 1 -- 172.21.15.32:0/1065237009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc81a2480 con 0x7f9bc80754f0 2024-08-31T07:38:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.581+0000 7f9bbfe00640 1 -- 172.21.15.32:0/1065237009 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb8003d10 con 0x7f9bc80754f0 2024-08-31T07:38:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.581+0000 7f9bbfe00640 1 -- 172.21.15.32:0/1065237009 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb802fa30 con 0x7f9bc80754f0 2024-08-31T07:38:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.581+0000 7f9bccc00640 1 -- 172.21.15.32:0/1065237009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc81a8c40 con 0x7f9bc80754f0 2024-08-31T07:38:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.582+0000 7f9bccc00640 1 -- 172.21.15.32:0/1065237009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc8075900 con 0x7f9bc80754f0 2024-08-31T07:38:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.584+0000 7f9bbfe00640 1 -- 172.21.15.32:0/1065237009 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9bb802fbd0 con 0x7f9bc80754f0 2024-08-31T07:38:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.586+0000 7f9bbfe00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9bac0b4ca0 0x7f9bac0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.586+0000 7f9bbfe00640 1 -- 172.21.15.32:0/1065237009 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8827+0+0 (secure 0 0 0) 0x7f9bb8106980 con 0x7f9bc80754f0 2024-08-31T07:38:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.586+0000 7f9bc6a00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9bac0b4ca0 0x7f9bac0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.587+0000 7f9bc6a00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9bac0b4ca0 0x7f9bac0b71b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8112070 tx=0x7f9bb4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:38:05.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.588+0000 7f9bbfe00640 1 -- 172.21.15.32:0/1065237009 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9bb80cd670 con 0x7f9bc80754f0 2024-08-31T07:38:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:05.786+0000 7f9bccc00640 1 -- 172.21.15.32:0/1065237009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9bc81a8e70 con 0x7f9bc80754f0 2024-08-31T07:38:06.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.573+0000 7f9bbfe00640 1 -- 172.21.15.32:0/1065237009 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) ==== 181+0+0 (secure 0 0 0) 0x7f9bb80ccf70 con 0x7f9bc80754f0 2024-08-31T07:38:06.575 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-08-31T07:38:06.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.577+0000 7f9bccc00640 1 -- 172.21.15.32:0/1065237009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9bac0b4ca0 msgr2=0x7f9bac0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:06.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.577+0000 7f9bccc00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9bac0b4ca0 0x7f9bac0b71b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8112070 tx=0x7f9bb4005f90 comp rx=0 tx=0).stop 2024-08-31T07:38:06.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.577+0000 7f9bccc00640 1 -- 172.21.15.32:0/1065237009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80754f0 msgr2=0x7f9bc819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:06.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.577+0000 7f9bccc00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80754f0 0x7f9bc819c280 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8005e30 tx=0x7f9bb8031840 comp rx=0 tx=0).stop 2024-08-31T07:38:06.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.578+0000 7f9bccc00640 1 -- 172.21.15.32:0/1065237009 shutdown_connections 2024-08-31T07:38:06.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.578+0000 7f9bccc00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7f9bac0b4ca0 0x7f9bac0b71b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:06.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.578+0000 7f9bccc00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bc810fcf0 0x7f9bc81a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:06.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.578+0000 7f9bccc00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc8075e40 0x7f9bc819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:06.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.578+0000 7f9bccc00640 1 --2- 172.21.15.32:0/1065237009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bc80754f0 0x7f9bc819c280 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:06.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.578+0000 7f9bccc00640 1 -- 172.21.15.32:0/1065237009 >> 172.21.15.32:0/1065237009 conn(0x7f9bc80ff180 msgr2=0x7f9bc8109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:38:06.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.579+0000 7f9bccc00640 1 -- 172.21.15.32:0/1065237009 shutdown_connections 2024-08-31T07:38:06.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:06.579+0000 7f9bccc00640 1 -- 172.21.15.32:0/1065237009 wait complete. 2024-08-31T07:38:06.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:06 smithi032 ceph-mon[17252]: pgmap v1167: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-08-31T07:38:06.859 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:06 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1065237009' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T07:38:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:06 smithi073 ceph-mon[21995]: pgmap v1167: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-08-31T07:38:06.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:06 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1065237009' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T07:38:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:06 smithi084 ceph-mon[22387]: pgmap v1167: 97 pgs: 97 active+clean; 2.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-08-31T07:38:06.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:06 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1065237009' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T07:38:07.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-31T07:38:07.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:07 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1065237009' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T07:38:07.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:07 smithi032 ceph-mon[17252]: osdmap e120: 12 total, 12 up, 12 in 2024-08-31T07:38:07.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:07 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1065237009' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T07:38:07.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:07 smithi073 ceph-mon[21995]: osdmap e120: 12 total, 12 up, 12 in 2024-08-31T07:38:07.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:07 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1065237009' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T07:38:07.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:07 smithi084 ceph-mon[22387]: osdmap e120: 12 total, 12 up, 12 in 2024-08-31T07:38:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T07:38:08.696 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:08 smithi032 ceph-mon[17252]: pgmap v1169: 65 pgs: 65 active+clean; 331 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:38:08.696 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:38:08.697 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:38:08.697 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:38:08.697 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:38:08.697 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:08 smithi032 ceph-mon[17252]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:38:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:08 smithi073 ceph-mon[21995]: pgmap v1169: 65 pgs: 65 active+clean; 331 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:38:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:38:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:38:08.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:38:08.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:38:08.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:08 smithi073 ceph-mon[21995]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:38:08.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:08 smithi084 ceph-mon[22387]: pgmap v1169: 65 pgs: 65 active+clean; 331 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:38:08.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:38:08.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:38:08.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T07:38:08.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T07:38:08.991 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:08 smithi084 ceph-mon[22387]: from='mgr.14220 172.21.15.32:0/2471130534' entity='mgr.x' 2024-08-31T07:38:09.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.207+0000 7faaabe00640 1 -- 172.21.15.32:0/3068434657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaac075ea0 msgr2=0x7faaac10f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:09.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.207+0000 7faaabe00640 1 --2- 172.21.15.32:0/3068434657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaac075ea0 0x7faaac10f770 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7faa94009900 tx=0x7faa9402a410 comp rx=0 tx=0).stop 2024-08-31T07:38:09.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.207+0000 7faaabe00640 1 -- 172.21.15.32:0/3068434657 shutdown_connections 2024-08-31T07:38:09.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.207+0000 7faaabe00640 1 --2- 172.21.15.32:0/3068434657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaac10fcb0 0x7faaac1120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:09.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.207+0000 7faaabe00640 1 --2- 172.21.15.32:0/3068434657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaac075ea0 0x7faaac10f770 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:09.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.207+0000 7faaabe00640 1 --2- 172.21.15.32:0/3068434657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac075550 0x7faaac075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:09.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.207+0000 7faaabe00640 1 -- 172.21.15.32:0/3068434657 >> 172.21.15.32:0/3068434657 conn(0x7faaac0ff1e0 msgr2=0x7faaac101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:38:09.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.207+0000 7faaabe00640 1 -- 172.21.15.32:0/3068434657 shutdown_connections 2024-08-31T07:38:09.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.207+0000 7faaabe00640 1 -- 172.21.15.32:0/3068434657 wait complete. 2024-08-31T07:38:09.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.208+0000 7faaabe00640 1 Processor -- start 2024-08-31T07:38:09.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.208+0000 7faaabe00640 1 -- start start 2024-08-31T07:38:09.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaabe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaac075550 0x7faaac19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:09.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaabe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac075ea0 0x7faaac19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:09.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaabe00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaac10fcb0 0x7faaac1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:09.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaabe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faaac115860 con 0x7faaac075550 2024-08-31T07:38:09.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaabe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1156e0 con 0x7faaac075ea0 2024-08-31T07:38:09.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaabe00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1159e0 con 0x7faaac10fcb0 2024-08-31T07:38:09.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaaaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaac075550 0x7faaac19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:09.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaa2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac075ea0 0x7faaac19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:09.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaaaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaac075550 0x7faaac19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38046/0 (socket says 172.21.15.32:38046) 2024-08-31T07:38:09.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaaaa00640 1 -- 172.21.15.32:0/1417932704 learned_addr learned my addr 172.21.15.32:0/1417932704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-08-31T07:38:09.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaa2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac075ea0 0x7faaac19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.32:55256/0 (socket says 172.21.15.32:55256) 2024-08-31T07:38:09.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.209+0000 7faaab400640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaac10fcb0 0x7faaac1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:09.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.210+0000 7faaa2000640 1 -- 172.21.15.32:0/1417932704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaac10fcb0 msgr2=0x7faaac1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:09.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.210+0000 7faaa2000640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaac10fcb0 0x7faaac1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:09.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.210+0000 7faaa2000640 1 -- 172.21.15.32:0/1417932704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaac075550 msgr2=0x7faaac19c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:09.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.210+0000 7faaa2000640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaac075550 0x7faaac19c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:09.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.210+0000 7faaa2000640 1 -- 172.21.15.32:0/1417932704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaac1a2220 con 0x7faaac075ea0 2024-08-31T07:38:09.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.210+0000 7faaa2000640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac075ea0 0x7faaac19c850 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faa9402a930 tx=0x7faa94031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:38:09.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.210+0000 7faaa3e00640 1 -- 172.21.15.32:0/1417932704 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9403f070 con 0x7faaac075ea0 2024-08-31T07:38:09.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.210+0000 7faaa3e00640 1 -- 172.21.15.32:0/1417932704 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa94004490 con 0x7faaac075ea0 2024-08-31T07:38:09.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.210+0000 7faaabe00640 1 -- 172.21.15.32:0/1417932704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaac1a24b0 con 0x7faaac075ea0 2024-08-31T07:38:09.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.211+0000 7faaa3e00640 1 -- 172.21.15.32:0/1417932704 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94035070 con 0x7faaac075ea0 2024-08-31T07:38:09.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.211+0000 7faaabe00640 1 -- 172.21.15.32:0/1417932704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaac1a8c70 con 0x7faaac075ea0 2024-08-31T07:38:09.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.212+0000 7faaabe00640 1 -- 172.21.15.32:0/1417932704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaac076a60 con 0x7faaac075ea0 2024-08-31T07:38:09.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.214+0000 7faaa3e00640 1 -- 172.21.15.32:0/1417932704 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7faa9403b460 con 0x7faaac075ea0 2024-08-31T07:38:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.216+0000 7faaa3e00640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa900b4ca0 0x7faa900b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T07:38:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.216+0000 7faaa3e00640 1 -- 172.21.15.32:0/1417932704 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8331+0+0 (secure 0 0 0) 0x7faa94105c60 con 0x7faaac075ea0 2024-08-31T07:38:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.216+0000 7faaaaa00640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa900b4ca0 0x7faa900b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T07:38:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.217+0000 7faaaaa00640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa900b4ca0 0x7faa900b71b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009fd0 tx=0x7faa9c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T07:38:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.218+0000 7faaa3e00640 1 -- 172.21.15.32:0/1417932704 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faa940cc460 con 0x7faaac075ea0 2024-08-31T07:38:09.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.415+0000 7faaabe00640 1 -- 172.21.15.32:0/1417932704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7faaac1a8e40 con 0x7faaac075ea0 2024-08-31T07:38:09.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.593+0000 7faaa3e00640 1 -- 172.21.15.32:0/1417932704 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) ==== 169+0+0 (secure 0 0 0) 0x7faa940d0c20 con 0x7faaac075ea0 2024-08-31T07:38:09.594 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-08-31T07:38:09.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 -- 172.21.15.32:0/1417932704 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa900b4ca0 msgr2=0x7faa900b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:09.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa900b4ca0 0x7faa900b71b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009fd0 tx=0x7faa9c009500 comp rx=0 tx=0).stop 2024-08-31T07:38:09.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 -- 172.21.15.32:0/1417932704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac075ea0 msgr2=0x7faaac19c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T07:38:09.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac075ea0 0x7faaac19c850 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faa9402a930 tx=0x7faa94031a20 comp rx=0 tx=0).stop 2024-08-31T07:38:09.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 -- 172.21.15.32:0/1417932704 shutdown_connections 2024-08-31T07:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.32:6800/4041113440,v1:172.21.15.32:6801/4041113440] conn(0x7faa900b4ca0 0x7faa900b71b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaac10fcb0 0x7faaac1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac075ea0 0x7faaac19c850 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 --2- 172.21.15.32:0/1417932704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaac075550 0x7faaac19c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T07:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 -- 172.21.15.32:0/1417932704 >> 172.21.15.32:0/1417932704 conn(0x7faaac0ff1e0 msgr2=0x7faaac109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T07:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.598+0000 7faaabe00640 1 -- 172.21.15.32:0/1417932704 shutdown_connections 2024-08-31T07:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-08-31T07:38:09.599+0000 7faaabe00640 1 -- 172.21.15.32:0/1417932704 wait complete. 2024-08-31T07:38:09.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:09 smithi032 ceph-mon[17252]: from='client.? 172.21.15.32:0/1417932704' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T07:38:09.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:09 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T07:38:09.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:09 smithi073 ceph-mon[21995]: from='client.? 172.21.15.32:0/1417932704' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T07:38:09.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:09 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T07:38:09.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:09 smithi084 ceph-mon[22387]: from='client.? 172.21.15.32:0/1417932704' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T07:38:09.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:09 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T07:38:10.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T07:38:10.495 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T07:38:10.528 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T07:38:10.573 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-31T07:38:10.574 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-31T07:38:10.621 DEBUG:teuthology.orchestra.run.smithi073:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-31T07:38:10.640 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-31T07:38:10.659 INFO:tasks.cephadm:Stopping all daemons... 2024-08-31T07:38:10.660 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-31T07:38:10.660 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.a 2024-08-31T07:38:10.712 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:10 smithi032 ceph-mon[17252]: pgmap v1170: 65 pgs: 65 active+clean; 331 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:38:10.712 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:10 smithi032 ceph-mon[17252]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T07:38:10.712 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:10 smithi032 ceph-mon[17252]: osdmap e121: 12 total, 12 up, 12 in 2024-08-31T07:38:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:10 smithi073 ceph-mon[21995]: pgmap v1170: 65 pgs: 65 active+clean; 331 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:38:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:10 smithi073 ceph-mon[21995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T07:38:10.929 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:10 smithi073 ceph-mon[21995]: osdmap e121: 12 total, 12 up, 12 in 2024-08-31T07:38:10.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:10 smithi084 ceph-mon[22387]: pgmap v1170: 65 pgs: 65 active+clean; 331 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T07:38:10.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:10 smithi084 ceph-mon[22387]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T07:38:10.990 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:10 smithi084 ceph-mon[22387]: osdmap e121: 12 total, 12 up, 12 in 2024-08-31T07:38:11.071 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:10 smithi032 systemd[1]: Stopping Ceph mon.a for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:11.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:11 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a[17248]: 2024-08-31T07:38:11.069+0000 7f6cda200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:38:11.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:11 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a[17248]: 2024-08-31T07:38:11.069+0000 7f6cda200640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-31T07:38:11.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:11 smithi032 podman[126517]: 2024-08-31 07:38:11.1618071 +0000 UTC m=+0.214526055 container died b31472b21c6499fe68fb2fe01e124d4798873f526931f3f50a4fca1ba1a80989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-08-31T07:38:11.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:11 smithi032 podman[126517]: 2024-08-31 07:38:11.759245813 +0000 UTC m=+0.811964772 container cleanup b31472b21c6499fe68fb2fe01e124d4798873f526931f3f50a4fca1ba1a80989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-31T07:38:11.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Aug 31 07:38:11 smithi032 bash[126517]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-a 2024-08-31T07:38:12.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.a.service' 2024-08-31T07:38:12.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:38:12.323 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-31T07:38:12.323 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-31T07:38:12.324 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.b 2024-08-31T07:38:12.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:12 smithi073 systemd[1]: Stopping Ceph mon.b for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:13.108 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:12 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-b[21991]: 2024-08-31T07:38:12.684+0000 7fe557f20640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:38:13.109 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:12 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-b[21991]: 2024-08-31T07:38:12.684+0000 7fe557f20640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-31T07:38:13.109 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:12 smithi073 podman[61322]: 2024-08-31 07:38:12.889288888 +0000 UTC m=+0.322273618 container died 6548fef870e27d3b2b7b8a2a862ba7b533661c7ae2beff6312937c9965856047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-31T07:38:14.388 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:14 smithi073 podman[61322]: 2024-08-31 07:38:14.103285256 +0000 UTC m=+1.536269943 container cleanup 6548fef870e27d3b2b7b8a2a862ba7b533661c7ae2beff6312937c9965856047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1) 2024-08-31T07:38:14.388 INFO:journalctl@ceph.mon.b.smithi073.stdout:Aug 31 07:38:14 smithi073 bash[61322]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-b 2024-08-31T07:38:14.526 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.b.service' 2024-08-31T07:38:14.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:38:14.569 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-31T07:38:14.569 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-31T07:38:14.570 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.c 2024-08-31T07:38:14.920 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:14 smithi084 systemd[1]: Stopping Ceph mon.c for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:15.240 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:14 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-c[22383]: 2024-08-31T07:38:14.919+0000 7f81a38ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:38:15.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:14 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-c[22383]: 2024-08-31T07:38:14.919+0000 7f81a38ab640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-31T07:38:15.241 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:15 smithi084 podman[62871]: 2024-08-31 07:38:15.154819448 +0000 UTC m=+0.348749236 container died 6c50cf5198a6260a2c0d3d815cf802c6f618d294330ae70a89f8be5dff5a43bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-31T07:38:15.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:15 smithi084 podman[62871]: 2024-08-31 07:38:15.528641435 +0000 UTC m=+0.722571211 container cleanup 6c50cf5198a6260a2c0d3d815cf802c6f618d294330ae70a89f8be5dff5a43bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-c, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:38:15.740 INFO:journalctl@ceph.mon.c.smithi084.stdout:Aug 31 07:38:15 smithi084 bash[62871]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mon-c 2024-08-31T07:38:15.991 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mon.c.service' 2024-08-31T07:38:16.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:38:16.037 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-31T07:38:16.037 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-31T07:38:16.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.x 2024-08-31T07:38:16.358 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:38:16 smithi032 systemd[1]: Stopping Ceph mgr.x for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:16.858 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:38:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:38:16.526+0000 7f5fcd200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:38:16.858 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:38:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x[17477]: 2024-08-31T07:38:16.526+0000 7f5fcd200640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-08-31T07:38:16.858 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:38:16 smithi032 podman[126636]: 2024-08-31 07:38:16.619375758 +0000 UTC m=+0.291064579 container died 74ab0db0eca7851fa7969fecea00a799ca4bf05321c076ea4c006619337a6109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True) 2024-08-31T07:38:17.902 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:38:17 smithi032 podman[126636]: 2024-08-31 07:38:17.649803603 +0000 UTC m=+1.321492424 container cleanup 74ab0db0eca7851fa7969fecea00a799ca4bf05321c076ea4c006619337a6109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-08-31T07:38:17.902 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:38:17 smithi032 bash[126636]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x 2024-08-31T07:38:18.183 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Aug 31 07:38:17 smithi032 podman[126649]: 2024-08-31 07:38:17.901447904 +0000 UTC m=+1.268226504 container remove 74ab0db0eca7851fa7969fecea00a799ca4bf05321c076ea4c006619337a6109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:38:18.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.x.service' 2024-08-31T07:38:18.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:38:18.289 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-31T07:38:18.289 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-31T07:38:18.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.y 2024-08-31T07:38:18.645 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:38:18 smithi073 systemd[1]: Stopping Ceph mgr.y for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:18.929 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:38:18 smithi073 podman[61443]: 2024-08-31 07:38:18.645539421 +0000 UTC m=+0.154354343 container died 9bcb6b0713010539911125fe0aa0a1085240c678019c72a85220fde04965a586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:38:19.261 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:38:19 smithi073 podman[61443]: 2024-08-31 07:38:19.034426551 +0000 UTC m=+0.543241472 container cleanup 9bcb6b0713010539911125fe0aa0a1085240c678019c72a85220fde04965a586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-y, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:38:19.261 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:38:19 smithi073 bash[61443]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-y 2024-08-31T07:38:19.507 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.y.service' 2024-08-31T07:38:19.530 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:38:19 smithi073 podman[61456]: 2024-08-31 07:38:19.261148429 +0000 UTC m=+0.610960019 container remove 9bcb6b0713010539911125fe0aa0a1085240c678019c72a85220fde04965a586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2) 2024-08-31T07:38:19.530 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:38:19 smithi073 systemd[1]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-08-31T07:38:19.530 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:38:19 smithi073 systemd[1]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-08-31T07:38:19.530 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:38:19 smithi073 systemd[1]: Stopped Ceph mgr.y for 965fc21a-6766-11ef-bcd4-c7b262605968. 2024-08-31T07:38:19.530 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Aug 31 07:38:19 smithi073 systemd[1]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.y.service: Consumed 7.812s CPU time. 2024-08-31T07:38:19.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:38:19.542 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-31T07:38:19.543 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-31T07:38:19.543 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.z 2024-08-31T07:38:19.921 INFO:journalctl@ceph.mgr.z.smithi084.stdout:Aug 31 07:38:19 smithi084 systemd[1]: Stopping Ceph mgr.z for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:20.240 INFO:journalctl@ceph.mgr.z.smithi084.stdout:Aug 31 07:38:19 smithi084 podman[62992]: 2024-08-31 07:38:19.920968156 +0000 UTC m=+0.187007972 container died a92d031f0259810d4e04792b0fbadc5461a065fd4eb80047696ec4af90ae5130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-31T07:38:21.203 INFO:journalctl@ceph.mgr.z.smithi084.stdout:Aug 31 07:38:20 smithi084 podman[62992]: 2024-08-31 07:38:20.893001945 +0000 UTC m=+1.159041755 container cleanup a92d031f0259810d4e04792b0fbadc5461a065fd4eb80047696ec4af90ae5130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-31T07:38:21.204 INFO:journalctl@ceph.mgr.z.smithi084.stdout:Aug 31 07:38:20 smithi084 bash[62992]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mgr-z 2024-08-31T07:38:21.384 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mgr.z.service' 2024-08-31T07:38:21.419 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:38:21.419 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-31T07:38:21.419 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-31T07:38:21.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.0 2024-08-31T07:38:21.858 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:21 smithi032 systemd[1]: Stopping Ceph osd.0 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:22.358 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:21 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0[34910]: 2024-08-31T07:38:21.907+0000 7f00cbc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:38:22.358 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:21 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0[34910]: 2024-08-31T07:38:21.907+0000 7f00cbc00640 -1 osd.0 121 *** Got signal Terminated *** 2024-08-31T07:38:22.358 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:21 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0[34910]: 2024-08-31T07:38:21.907+0000 7f00cbc00640 -1 osd.0 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:38:27.858 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:27 smithi032 podman[126768]: 2024-08-31 07:38:27.486536109 +0000 UTC m=+5.768775901 container died 52e95a3065b3b17c745f2e071e110f86395537f461d62478362520e51e97312a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-31T07:38:28.434 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:28 smithi032 podman[126768]: 2024-08-31 07:38:28.139798631 +0000 UTC m=+6.422038422 container cleanup 52e95a3065b3b17c745f2e071e110f86395537f461d62478362520e51e97312a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-31T07:38:28.434 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:28 smithi032 bash[126768]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0 2024-08-31T07:38:28.858 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:28 smithi032 podman[126789]: 2024-08-31 07:38:28.433470333 +0000 UTC m=+0.934309014 container remove 52e95a3065b3b17c745f2e071e110f86395537f461d62478362520e51e97312a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-31T07:38:29.358 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:29 smithi032 podman[126873]: 2024-08-31 07:38:28.966255623 +0000 UTC m=+0.027267223 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:38:29.358 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:29 smithi032 podman[126873]: 2024-08-31 07:38:29.134644232 +0000 UTC m=+0.195655808 container create 69d5eb04b1adf3e1162f4294e7bd015e8a615d24c31c2cbe984cd687424c429b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:38:29.725 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:29 smithi032 podman[126873]: 2024-08-31 07:38:29.46301394 +0000 UTC m=+0.524025535 container init 69d5eb04b1adf3e1162f4294e7bd015e8a615d24c31c2cbe984cd687424c429b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:38:29.725 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:29 smithi032 podman[126873]: 2024-08-31 07:38:29.470435133 +0000 UTC m=+0.531446739 container start 69d5eb04b1adf3e1162f4294e7bd015e8a615d24c31c2cbe984cd687424c429b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-31T07:38:29.726 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:29 smithi032 podman[126873]: 2024-08-31 07:38:29.579665507 +0000 UTC m=+0.640677083 container attach 69d5eb04b1adf3e1162f4294e7bd015e8a615d24c31c2cbe984cd687424c429b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-31T07:38:29.726 INFO:journalctl@ceph.osd.0.smithi032.stdout:Aug 31 07:38:29 smithi032 podman[126873]: 2024-08-31 07:38:29.72478142 +0000 UTC m=+0.785793007 container died 69d5eb04b1adf3e1162f4294e7bd015e8a615d24c31c2cbe984cd687424c429b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1) 2024-08-31T07:38:30.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.0.service' 2024-08-31T07:38:30.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:38:30.643 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-31T07:38:30.643 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-31T07:38:30.643 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.1 2024-08-31T07:38:30.929 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:30 smithi073 systemd[1]: Stopping Ceph osd.1 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:31.429 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:31 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1[27600]: 2024-08-31T07:38:31.038+0000 7ff1f259a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:38:31.429 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:31 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1[27600]: 2024-08-31T07:38:31.038+0000 7ff1f259a640 -1 osd.1 121 *** Got signal Terminated *** 2024-08-31T07:38:31.429 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:31 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1[27600]: 2024-08-31T07:38:31.038+0000 7ff1f259a640 -1 osd.1 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:38:36.765 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:36 smithi073 podman[61567]: 2024-08-31 07:38:36.471997939 +0000 UTC m=+5.638394273 container died 90e5cb4190b8e99bf43f30670635289026f4cad0fd8a6094932341eb128c7e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-31T07:38:37.141 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:36 smithi073 podman[61567]: 2024-08-31 07:38:36.855829707 +0000 UTC m=+6.022226040 container cleanup 90e5cb4190b8e99bf43f30670635289026f4cad0fd8a6094932341eb128c7e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-31T07:38:37.142 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:36 smithi073 bash[61567]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1 2024-08-31T07:38:37.429 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:37 smithi073 podman[61593]: 2024-08-31 07:38:37.141291486 +0000 UTC m=+0.671423945 container remove 90e5cb4190b8e99bf43f30670635289026f4cad0fd8a6094932341eb128c7e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-31T07:38:37.704 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:37 smithi073 podman[61688]: 2024-08-31 07:38:37.351327109 +0000 UTC m=+0.015979926 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:38:37.705 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:37 smithi073 podman[61688]: 2024-08-31 07:38:37.467592291 +0000 UTC m=+0.132245102 container create 10476e776a77c749ebcc8de26b3d713b169fb5309a932239c8b7721fa55d0418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-31T07:38:37.705 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:37 smithi073 podman[61688]: 2024-08-31 07:38:37.703400613 +0000 UTC m=+0.368053434 container init 10476e776a77c749ebcc8de26b3d713b169fb5309a932239c8b7721fa55d0418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T07:38:37.997 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:37 smithi073 podman[61688]: 2024-08-31 07:38:37.706892929 +0000 UTC m=+0.371545745 container start 10476e776a77c749ebcc8de26b3d713b169fb5309a932239c8b7721fa55d0418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:38:37.998 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:37 smithi073 podman[61688]: 2024-08-31 07:38:37.828858085 +0000 UTC m=+0.493510911 container attach 10476e776a77c749ebcc8de26b3d713b169fb5309a932239c8b7721fa55d0418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-08-31T07:38:38.333 INFO:journalctl@ceph.osd.1.smithi073.stdout:Aug 31 07:38:37 smithi073 podman[61688]: 2024-08-31 07:38:37.997874623 +0000 UTC m=+0.662527434 container died 10476e776a77c749ebcc8de26b3d713b169fb5309a932239c8b7721fa55d0418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:38:38.673 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.1.service' 2024-08-31T07:38:38.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:38:38.715 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-31T07:38:38.715 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-31T07:38:38.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.2 2024-08-31T07:38:39.030 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:38 smithi084 systemd[1]: Stopping Ceph osd.2 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:39.490 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2[28669]: 2024-08-31T07:38:39.029+0000 7f2b773a2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:38:39.490 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2[28669]: 2024-08-31T07:38:39.029+0000 7f2b773a2640 -1 osd.2 121 *** Got signal Terminated *** 2024-08-31T07:38:39.490 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2[28669]: 2024-08-31T07:38:39.029+0000 7f2b773a2640 -1 osd.2 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:38:44.863 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:44 smithi084 podman[63130]: 2024-08-31 07:38:44.598980777 +0000 UTC m=+5.682850663 container died c8e82bab121e4c25fc0c76899fd45fcf1d47a54ee357a4408d9d510804885d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-31T07:38:45.224 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:44 smithi084 podman[63130]: 2024-08-31 07:38:44.947951281 +0000 UTC m=+6.031821152 container cleanup c8e82bab121e4c25fc0c76899fd45fcf1d47a54ee357a4408d9d510804885d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-08-31T07:38:45.224 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:44 smithi084 bash[63130]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2 2024-08-31T07:38:45.490 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:45 smithi084 podman[63147]: 2024-08-31 07:38:45.224279251 +0000 UTC m=+0.623525449 container remove c8e82bab121e4c25fc0c76899fd45fcf1d47a54ee357a4408d9d510804885d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1) 2024-08-31T07:38:45.794 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:45 smithi084 podman[63239]: 2024-08-31 07:38:45.435268924 +0000 UTC m=+0.015639279 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:38:45.794 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:45 smithi084 podman[63239]: 2024-08-31 07:38:45.550196399 +0000 UTC m=+0.130566751 container create 175c264cbfc6e92d7b72277249a15aa1c4b0362f3ccf101a2c66741d0de8ad02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD) 2024-08-31T07:38:45.794 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:45 smithi084 podman[63239]: 2024-08-31 07:38:45.794218269 +0000 UTC m=+0.374588618 container init 175c264cbfc6e92d7b72277249a15aa1c4b0362f3ccf101a2c66741d0de8ad02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T07:38:46.103 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:45 smithi084 podman[63239]: 2024-08-31 07:38:45.797795172 +0000 UTC m=+0.378165522 container start 175c264cbfc6e92d7b72277249a15aa1c4b0362f3ccf101a2c66741d0de8ad02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-08-31T07:38:46.103 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:45 smithi084 podman[63239]: 2024-08-31 07:38:45.919379622 +0000 UTC m=+0.499749972 container attach 175c264cbfc6e92d7b72277249a15aa1c4b0362f3ccf101a2c66741d0de8ad02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS) 2024-08-31T07:38:46.375 INFO:journalctl@ceph.osd.2.smithi084.stdout:Aug 31 07:38:46 smithi084 podman[63239]: 2024-08-31 07:38:46.103425434 +0000 UTC m=+0.683795786 container died 175c264cbfc6e92d7b72277249a15aa1c4b0362f3ccf101a2c66741d0de8ad02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:38:46.773 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.2.service' 2024-08-31T07:38:46.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:38:46.814 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-31T07:38:46.814 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-31T07:38:46.814 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.3 2024-08-31T07:38:47.300 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:46 smithi032 systemd[1]: Stopping Ceph osd.3 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:47.608 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:47 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3[41331]: 2024-08-31T07:38:47.298+0000 7f3aeda00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:38:47.608 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:47 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3[41331]: 2024-08-31T07:38:47.298+0000 7f3aeda00640 -1 osd.3 121 *** Got signal Terminated *** 2024-08-31T07:38:47.608 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:47 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3[41331]: 2024-08-31T07:38:47.298+0000 7f3aeda00640 -1 osd.3 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:38:49.064 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:48 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:48.802+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:49.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:49 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:49.069+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:49.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:49.063+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:49.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:49.159+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:50.086 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:49.755+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:50.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:50 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:50.071+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:50.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:50 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:50.086+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:50.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:50 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:50.196+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:50.833 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:50 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3[41331]: 2024-08-31T07:38:50.525+0000 7f3ae9400640 -1 osd.3 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:26.694373+0000 front 2024-08-31T07:38:26.694592+0000 (oldest deadline 2024-08-31T07:38:50.194692+0000) 2024-08-31T07:38:51.056 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:50 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:50.760+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:51.105 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:50 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:50.832+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:38:51.357 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:51 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:51.101+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:51.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:51 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:51.056+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:51.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:51 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:51.203+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:51.857 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:51 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3[41331]: 2024-08-31T07:38:51.496+0000 7f3ae9400640 -1 osd.3 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:26.694373+0000 front 2024-08-31T07:38:26.694592+0000 (oldest deadline 2024-08-31T07:38:50.194692+0000) 2024-08-31T07:38:51.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:38:51 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:38:51.625+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:38:51.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:51 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:51.718+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:52.358 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:51 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:51.875+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:38:52.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:52 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:52.081+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:52.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:52 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:52.217+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:52.491 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:52 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:52.058+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:52.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:38:52 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:38:52.588+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:38:52.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:52 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:52.712+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:53.358 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:52 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:52.864+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:38:53.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:53 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:53.072+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:53.358 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:53 smithi032 podman[126977]: 2024-08-31 07:38:53.083589378 +0000 UTC m=+5.973011604 container died b04ee35543bc85f5ad18b5d8f4c735490d8ec1d7816ddbe9fc857a57cae145d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:38:53.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:53.029+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:53.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:53.250+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:53.756 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:53 smithi032 podman[126977]: 2024-08-31 07:38:53.554332679 +0000 UTC m=+6.443754903 container cleanup b04ee35543bc85f5ad18b5d8f4c735490d8ec1d7816ddbe9fc857a57cae145d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.1) 2024-08-31T07:38:53.756 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:53 smithi032 bash[126977]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3 2024-08-31T07:38:53.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:38:53 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:38:53.570+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:38:54.017 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:53.755+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:54.107 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:53 smithi032 podman[127004]: 2024-08-31 07:38:53.755866107 +0000 UTC m=+0.662394349 container remove b04ee35543bc85f5ad18b5d8f4c735490d8ec1d7816ddbe9fc857a57cae145d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:38:54.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:53 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:53.818+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:38:54.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:54 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:54.015+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:54.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:54 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:54.222+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:54.491 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:54 smithi032 podman[127090]: 2024-08-31 07:38:54.228514085 +0000 UTC m=+0.027396571 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:38:54.492 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:54 smithi032 podman[127090]: 2024-08-31 07:38:54.356910192 +0000 UTC m=+0.155792677 container create 46c2f62e35729bf6af23a5f570b171d3b63b25bf5c6720cfdf8a17a422023548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-31T07:38:54.492 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:54 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:54.112+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:54.751 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:54 smithi032 podman[127090]: 2024-08-31 07:38:54.643690211 +0000 UTC m=+0.442572698 container init 46c2f62e35729bf6af23a5f570b171d3b63b25bf5c6720cfdf8a17a422023548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-31T07:38:54.752 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:54 smithi032 podman[127090]: 2024-08-31 07:38:54.650569779 +0000 UTC m=+0.449452265 container start 46c2f62e35729bf6af23a5f570b171d3b63b25bf5c6720cfdf8a17a422023548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-31T07:38:54.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:38:54 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:38:54.619+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:38:54.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:54 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:54.731+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:55.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:54 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:54.773+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:38:55.108 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:54 smithi032 podman[127090]: 2024-08-31 07:38:54.75148816 +0000 UTC m=+0.550370657 container attach 46c2f62e35729bf6af23a5f570b171d3b63b25bf5c6720cfdf8a17a422023548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-31T07:38:55.108 INFO:journalctl@ceph.osd.3.smithi032.stdout:Aug 31 07:38:54 smithi032 podman[127090]: 2024-08-31 07:38:54.920920008 +0000 UTC m=+0.719802495 container died 46c2f62e35729bf6af23a5f570b171d3b63b25bf5c6720cfdf8a17a422023548 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T07:38:55.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:55 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:55.013+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:55.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:55 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:55.181+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:55.607 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:55 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:55.149+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:55.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.3.service' 2024-08-31T07:38:55.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:38:55.744 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-31T07:38:55.744 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-31T07:38:55.744 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.4 2024-08-31T07:38:55.929 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:38:55 smithi073 systemd[1]: Stopping Ceph osd.4 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:38:55.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:38:55 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:38:55.661+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:38:55.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:55 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:55.695+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:56.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:55 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:55.752+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:38:56.429 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:38:56 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4[34058]: 2024-08-31T07:38:56.010+0000 7fcfb3d54640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:38:56.429 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:38:56 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4[34058]: 2024-08-31T07:38:56.010+0000 7fcfb3d54640 -1 osd.4 121 *** Got signal Terminated *** 2024-08-31T07:38:56.429 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:38:56 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4[34058]: 2024-08-31T07:38:56.010+0000 7fcfb3d54640 -1 osd.4 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:38:56.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:56 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:56.050+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:56.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:56 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:56.204+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:56.607 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:56 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:56.193+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:56.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:56 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:56.646+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:57.107 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:56 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:56.725+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:38:57.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:38:56 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:38:56.702+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:38:57.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:57.179+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:57.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:57.054+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:57.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:57 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:57.166+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:57.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:38:57 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:38:57.697+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:38:57.929 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:38:57 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4[34058]: 2024-08-31T07:38:57.519+0000 7fcfac337640 -1 osd.4 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.780747+0000 front 2024-08-31T07:38:34.780724+0000 (oldest deadline 2024-08-31T07:38:57.080879+0000) 2024-08-31T07:38:57.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:57.641+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:57.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:57.641+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:38:58.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:57 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:57.723+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:38:58.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:58.054+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:58.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:58.162+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:58.607 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:58 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:58.144+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:58.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:38:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:38:58.742+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:38:58.929 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:38:58 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4[34058]: 2024-08-31T07:38:58.501+0000 7fcfac337640 -1 osd.4 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.780747+0000 front 2024-08-31T07:38:34.780724+0000 (oldest deadline 2024-08-31T07:38:57.080879+0000) 2024-08-31T07:38:58.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:58.681+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:58.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:58 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:58.681+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:38:59.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:58 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:58.694+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:38:59.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:38:59 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:38:59.012+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:38:59.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:38:59 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:38:59.164+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:38:59.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:59 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:59.193+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:38:59.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:38:59 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:38:59.193+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:38:59.738 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:38:59 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4[34058]: 2024-08-31T07:38:59.457+0000 7fcfac337640 -1 osd.4 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.780747+0000 front 2024-08-31T07:38:34.780724+0000 (oldest deadline 2024-08-31T07:38:57.080879+0000) 2024-08-31T07:38:59.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:59 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:59.670+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:38:59.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:38:59 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:38:59.670+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:00.107 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:59 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:59.736+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:00.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:38:59 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:38:59.736+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:00.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:38:59 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:38:59.738+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:00.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:00 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:00.039+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:39:00.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:00 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:00.163+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:00.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:00.219+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:00.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:00.219+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:00.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:00 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:00.700+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:00.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:00 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:00.700+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:00.930 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:00 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4[34058]: 2024-08-31T07:39:00.478+0000 7fcfac337640 -1 osd.4 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.780747+0000 front 2024-08-31T07:38:34.780724+0000 (oldest deadline 2024-08-31T07:38:57.080879+0000) 2024-08-31T07:39:00.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:00 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:00.571+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:00.988 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:00.709+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:00.988 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:00.709+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:00.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:00 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:00.711+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:00.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:00 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:00.711+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:01.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:01 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:01.192+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:01.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:01 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:01.192+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:01.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:01 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:01.082+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:39:01.491 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:01 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:01.082+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.835842+0000 front 2024-08-31T07:38:34.836064+0000 (oldest deadline 2024-08-31T07:39:00.735934+0000) 2024-08-31T07:39:01.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:01 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:01.196+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:01.780 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:01 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:01.538+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:01.781 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:01 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:01.733+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:01.781 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:01 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:01.733+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:01.781 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:01 smithi073 podman[61789]: 2024-08-31 07:39:01.5294566 +0000 UTC m=+5.597710956 container died 7eecb0f6f05a98312c6a5f62dfdc999bbfbb6aad1f24396f2fed8ae3daa592db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:39:02.088 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:01 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:01.750+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:02.088 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:01 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:01.750+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:02.105 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:01 smithi073 podman[61789]: 2024-08-31 07:39:01.870003915 +0000 UTC m=+5.938258271 container cleanup 7eecb0f6f05a98312c6a5f62dfdc999bbfbb6aad1f24396f2fed8ae3daa592db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:39:02.105 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:01 smithi073 bash[61789]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4 2024-08-31T07:39:02.107 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:01 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:01.727+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:02.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:01 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:01.727+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:02.429 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:02 smithi073 podman[61803]: 2024-08-31 07:39:02.105063508 +0000 UTC m=+0.564582442 container remove 7eecb0f6f05a98312c6a5f62dfdc999bbfbb6aad1f24396f2fed8ae3daa592db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-31T07:39:02.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:02 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:02.224+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:02.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:02 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:02.087+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:39:02.491 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:02 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:02.087+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.835842+0000 front 2024-08-31T07:38:34.836064+0000 (oldest deadline 2024-08-31T07:39:00.735934+0000) 2024-08-31T07:39:02.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:02 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:02.196+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:02.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:02 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:02.196+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:02.767 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:02 smithi073 podman[61892]: 2024-08-31 07:39:02.412164032 +0000 UTC m=+0.015815589 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:39:02.767 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:02 smithi073 podman[61892]: 2024-08-31 07:39:02.530912791 +0000 UTC m=+0.134564338 container create a0255c88df3f6d714ad30d4c417c1b890d9c80909b8309debc7600249ab6a9d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-08-31T07:39:02.767 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:02 smithi073 podman[61892]: 2024-08-31 07:39:02.766883522 +0000 UTC m=+0.370535066 container init a0255c88df3f6d714ad30d4c417c1b890d9c80909b8309debc7600249ab6a9d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-08-31T07:39:02.768 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:02 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:02.531+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:02.768 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:02 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:02.706+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:02.768 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:02 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:02.706+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:02.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:02 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:02.721+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:02.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:02 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:02.721+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:03.107 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:02 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:02.718+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:03.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:02 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:02.718+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:03.179 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:02 smithi073 podman[61892]: 2024-08-31 07:39:02.770417194 +0000 UTC m=+0.374068747 container start a0255c88df3f6d714ad30d4c417c1b890d9c80909b8309debc7600249ab6a9d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-08-31T07:39:03.179 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:02 smithi073 podman[61892]: 2024-08-31 07:39:02.858637096 +0000 UTC m=+0.462288660 container attach a0255c88df3f6d714ad30d4c417c1b890d9c80909b8309debc7600249ab6a9d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-08-31T07:39:03.180 INFO:journalctl@ceph.osd.4.smithi073.stdout:Aug 31 07:39:02 smithi073 podman[61892]: 2024-08-31 07:39:02.953422917 +0000 UTC m=+0.557074473 container died a0255c88df3f6d714ad30d4c417c1b890d9c80909b8309debc7600249ab6a9d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T07:39:03.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:03 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:03.228+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:03.491 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:03 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:03.056+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:39:03.491 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:03 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:03.056+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.835842+0000 front 2024-08-31T07:38:34.836064+0000 (oldest deadline 2024-08-31T07:39:00.735934+0000) 2024-08-31T07:39:03.521 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:03 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:03.505+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:03.542 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.4.service' 2024-08-31T07:39:03.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:39:03.583 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-31T07:39:03.583 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-31T07:39:03.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.5 2024-08-31T07:39:03.607 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:03 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:03.221+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:03.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:03 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:03.221+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:03.884 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:03 smithi084 systemd[1]: Stopping Ceph osd.5 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:39:03.884 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:03 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:03.678+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:03.885 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:03 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:03.678+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:03.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:03 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:03.682+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:03.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:03 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:03.682+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:04.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:03 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:03.729+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:04.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:03 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:03.729+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:04.240 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:03 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:03.883+0000 7ff9c2299640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:39:04.241 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:03 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:03.883+0000 7ff9c2299640 -1 osd.5 121 *** Got signal Terminated *** 2024-08-31T07:39:04.241 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:03 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:03.883+0000 7ff9c2299640 -1 osd.5 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:39:04.241 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:04 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:04.021+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:39:04.241 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:04 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:04.021+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.835842+0000 front 2024-08-31T07:38:34.836064+0000 (oldest deadline 2024-08-31T07:39:00.735934+0000) 2024-08-31T07:39:04.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:04 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:04.220+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:04.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:04 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:04.220+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:04.668 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:04 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:04.242+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:04.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:04 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:04.491+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:04.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:04 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:04.637+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:04.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:04 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:04.637+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:04.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:04 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:04.666+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:04.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:04 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:04.666+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:05.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:04 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:04.767+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:05.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:04 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:04.767+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:05.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:05 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:05.275+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:05.491 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:05 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:05.056+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:39:05.491 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:05 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:05.056+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.835842+0000 front 2024-08-31T07:38:34.836064+0000 (oldest deadline 2024-08-31T07:39:00.735934+0000) 2024-08-31T07:39:05.607 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:05 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:05.181+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:05.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:05 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:05.181+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:05.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:05 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:05.511+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:05.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:05 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:05.600+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:05.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:05 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:05.600+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:05.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:05 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:05.660+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:05.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:05 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:05.660+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:05.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:05 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:05.660+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:06.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:05 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:05.760+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:06.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:05 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:05.760+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:06.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:05 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:05.760+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:06.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:06 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:06.323+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:06.490 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:06 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:06.080+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:39:06.491 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:06 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:06.080+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.835842+0000 front 2024-08-31T07:38:34.836064+0000 (oldest deadline 2024-08-31T07:39:00.735934+0000) 2024-08-31T07:39:06.607 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:06 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:06.193+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:06.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:06 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:06.193+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:06.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:06 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:06.503+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:06.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:06 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:06.503+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:06.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:06 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:06.561+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:06.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:06 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:06.561+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:06.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:06 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:06.617+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:06.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:06 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:06.617+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:06.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:06 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:06.617+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:07.107 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:06 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:06.804+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:07.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:06 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:06.804+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:07.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:06 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:06.804+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:07.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:07 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:07.293+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:07.491 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:07 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:07.111+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:39:07.491 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:07 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:07.111+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.835842+0000 front 2024-08-31T07:38:34.836064+0000 (oldest deadline 2024-08-31T07:39:00.735934+0000) 2024-08-31T07:39:07.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:07 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:07.154+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:07.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:07 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:07.154+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:07.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:07 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:07.551+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:07.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:07 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:07.551+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:07.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:07 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:07.551+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:07.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:07 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:07.551+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:07.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:07 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:07.552+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:07.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:07 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:07.599+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:07.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:07 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:07.599+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:07.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:07 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:07.599+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:08.107 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:07 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:07.762+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:08.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:07 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:07.762+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:08.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:07 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:07.762+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:08.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:08 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:08.318+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:08.498 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:08 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:08.081+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:25.534246+0000 front 2024-08-31T07:38:25.534275+0000 (oldest deadline 2024-08-31T07:38:49.034153+0000) 2024-08-31T07:39:08.498 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:08 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5[35150]: 2024-08-31T07:39:08.081+0000 7ff9be884640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.835842+0000 front 2024-08-31T07:38:34.836064+0000 (oldest deadline 2024-08-31T07:39:00.735934+0000) 2024-08-31T07:39:08.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:08.113+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:08.608 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:08.113+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:08.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:08 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:08.526+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:08.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:08 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:08.526+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:08.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:08 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:08.588+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:08.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:08 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:08.588+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:08.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:08 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:08.588+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:08.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:08 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:08.599+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:08.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:08 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:08.599+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:08.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:08 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:08.599+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:09.081 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:08.808+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:09.081 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:08.808+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:09.081 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:08 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:08.808+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:09.357 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:09 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:09.080+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:09.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:09 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:09.080+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:09.740 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:09 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:09.333+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:09.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:09 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:09.334+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:09.741 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:09 smithi084 podman[63352]: 2024-08-31 07:39:09.433404288 +0000 UTC m=+5.657862512 container died 2ef863cf2744fbf1072539f22f5383370939f9d7b143e76c6b284f8da0d20db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-31T07:39:09.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:09 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:09.576+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:09.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:09 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:09.576+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:09.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:09 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:09.576+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:09.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:09 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:09.531+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:09.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:09 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:09.531+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:09.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:09 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:09.567+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:09.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:09 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:09.567+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:09.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:09 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:09.567+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:10.050 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:09 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:09.770+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:10.050 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:09 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:09.770+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:10.050 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:09 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:09.770+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:10.110 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:09 smithi084 podman[63352]: 2024-08-31 07:39:09.841450757 +0000 UTC m=+6.065909020 container cleanup 2ef863cf2744fbf1072539f22f5383370939f9d7b143e76c6b284f8da0d20db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1) 2024-08-31T07:39:10.110 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:09 smithi084 bash[63352]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5 2024-08-31T07:39:10.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:10 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:10.049+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:10.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:10 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:10.049+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:10.445 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:10 smithi084 podman[63369]: 2024-08-31 07:39:10.109671693 +0000 UTC m=+0.677519030 container remove 2ef863cf2744fbf1072539f22f5383370939f9d7b143e76c6b284f8da0d20db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-08-31T07:39:10.446 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:10 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:10.288+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:10.446 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:10 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:10.288+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:10.729 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:10 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:10.540+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:10.730 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:10 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:10.540+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:10.730 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:10 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:10.540+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:10.730 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:10 smithi084 podman[63457]: 2024-08-31 07:39:10.347889512 +0000 UTC m=+0.015638058 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:39:10.730 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:10 smithi084 podman[63457]: 2024-08-31 07:39:10.477205604 +0000 UTC m=+0.144954175 container create b61a124045b2d8cc7a5a588ad809633dce16a2dd09a3d04c06a1aaa711304a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:39:10.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:10 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:10.503+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:10.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:10 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:10.503+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:10.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:10 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:10.553+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:10.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:10 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:10.553+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:10.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:10 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:10.553+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:10.990 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:10 smithi084 podman[63457]: 2024-08-31 07:39:10.729566066 +0000 UTC m=+0.397314592 container init b61a124045b2d8cc7a5a588ad809633dce16a2dd09a3d04c06a1aaa711304a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux ) 2024-08-31T07:39:10.990 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:10 smithi084 podman[63457]: 2024-08-31 07:39:10.733138658 +0000 UTC m=+0.400887190 container start b61a124045b2d8cc7a5a588ad809633dce16a2dd09a3d04c06a1aaa711304a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0) 2024-08-31T07:39:10.991 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:10 smithi084 podman[63457]: 2024-08-31 07:39:10.846338826 +0000 UTC m=+0.514087385 container attach b61a124045b2d8cc7a5a588ad809633dce16a2dd09a3d04c06a1aaa711304a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS) 2024-08-31T07:39:10.991 INFO:journalctl@ceph.osd.5.smithi084.stdout:Aug 31 07:39:10 smithi084 podman[63457]: 2024-08-31 07:39:10.92094624 +0000 UTC m=+0.588694772 container died b61a124045b2d8cc7a5a588ad809633dce16a2dd09a3d04c06a1aaa711304a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-31T07:39:11.042 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:10 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:10.776+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:11.043 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:10 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:10.776+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:11.043 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:10 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:10.776+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:11.357 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:11 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:11.042+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:11.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:11 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:11.042+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:11.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:11 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:11.042+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:11.483 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:11 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:11.254+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:11.484 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:11 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:11.254+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:11.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.5.service' 2024-08-31T07:39:11.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:39:11.547 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-31T07:39:11.547 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-31T07:39:11.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.6 2024-08-31T07:39:11.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:11 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:11.552+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:11.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:11 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:11.552+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:11.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:11 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:11.552+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:11.858 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:11 smithi032 systemd[1]: Stopping Ceph osd.6 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:39:11.858 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:11 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:11.803+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:11.858 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:11 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:11.803+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:11.859 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:11 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:11.803+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:11.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:11 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:11.510+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:11.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:11 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:11.510+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:11.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:11 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:11.556+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:11.931 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:11 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:11.556+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:11.931 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:11 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:11.556+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:12.358 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:12.020+0000 7f3ca4800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:39:12.358 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:12.020+0000 7f3ca4800640 -1 osd.6 121 *** Got signal Terminated *** 2024-08-31T07:39:12.358 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:12.020+0000 7f3ca4800640 -1 osd.6 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:39:12.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:12.008+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:12.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:12.009+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:12.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:12.009+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:12.604 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:12 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:12.281+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:12.604 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:12 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:12.281+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:12.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:12 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:12.520+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:12.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:12 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:12.520+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:12.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:12 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:12.591+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:12.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:12 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:12.591+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:12.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:12 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:12.591+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:12.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:12 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:12.603+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:12.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:12 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:12.603+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:12.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:12 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:12.603+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:13.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:12.797+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:13.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:12.797+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:13.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:12.797+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:13.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:12.973+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:13.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:12.973+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:13.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:12 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:12.973+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:13.646 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:13 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:13.322+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:13.646 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:13 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:13.322+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:13.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:13 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:13.554+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:13.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:13 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:13.554+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:13.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:13 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:13.628+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:13.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:13 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:13.628+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:13.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:13 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:13.628+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:13.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:13 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:13.645+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:13.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:13 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:13.645+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:13.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:13 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:13.645+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:14.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:13 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:13.801+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:14.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:13 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:13.801+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:14.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:13 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:13.801+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:14.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:14.012+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:14.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:14.012+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:14.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:14.012+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:14.639 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:14 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:14.295+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:14.639 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:14 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:14.295+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:14.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:14 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:14.545+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:14.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:14 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:14.545+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:14.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:14 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:14.674+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:14.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:14 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:14.674+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:14.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:14 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:14.674+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:14.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:14 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:14.638+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:14.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:14 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:14.638+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:14.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:14 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:14.638+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:15.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:14.841+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:15.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:14.841+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:15.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:14.841+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:15.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:14.973+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:15.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:14.973+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:15.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:14.973+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:15.633 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:15 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:15.269+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:15.633 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:15 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:15.269+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:15.634 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:15 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:15.269+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:15.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:15 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:15.536+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:15.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:15 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:15.536+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:15.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:15 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:15.705+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:15.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:15 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:15.705+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:15.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:15 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:15.705+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:15.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:15 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:15.632+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:15.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:15 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:15.632+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:15.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:15 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:15.632+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:16.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:15 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:15.792+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:16.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:15 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:15.792+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:16.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:15 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:15.792+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:16.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:16.006+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:16.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:16.006+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:16.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:16.006+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:16.660 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:16 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:16.256+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:16.660 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:16 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:16.256+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:16.661 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:16 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:16.256+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:16.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:16 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:16.533+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:16.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:16 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:16.533+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:16.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:16 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:16.533+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.288725+0000 front 2024-08-31T07:38:50.288622+0000 (oldest deadline 2024-08-31T07:39:15.588596+0000) 2024-08-31T07:39:16.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:16 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:16.700+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:16.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:16 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:16.700+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:16.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:16 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:16.700+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:16.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:16 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:16.659+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:16.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:16 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:16.659+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:16.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:16 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:16.659+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:16.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:16 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:16.659+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:17.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:16.751+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.576778+0000 front 2024-08-31T07:38:24.576990+0000 (oldest deadline 2024-08-31T07:38:50.477042+0000) 2024-08-31T07:39:17.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:16.751+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:35.678877+0000 front 2024-08-31T07:38:35.678815+0000 (oldest deadline 2024-08-31T07:38:59.178920+0000) 2024-08-31T07:39:17.108 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:16.751+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.880236+0000 front 2024-08-31T07:38:43.880528+0000 (oldest deadline 2024-08-31T07:39:05.580232+0000) 2024-08-31T07:39:17.109 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6[48240]: 2024-08-31T07:39:16.751+0000 7f3c9b200640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.882811+0000 front 2024-08-31T07:38:51.882660+0000 (oldest deadline 2024-08-31T07:39:15.982907+0000) 2024-08-31T07:39:17.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:16.984+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:17.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:16.984+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:17.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:16.984+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:17.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:16 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:16.984+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:17.690 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:17 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:17.263+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:17.690 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:17 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:17.263+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:17.690 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:17 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:17.263+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:17.858 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:17 smithi032 podman[127190]: 2024-08-31 07:39:17.54004651 +0000 UTC m=+5.711618873 container died 09f9fbbd1e9a033d117e192a4e5319176d873178e8619d593c7efbac4babc1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0) 2024-08-31T07:39:17.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:17 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:17.545+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:17.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:17 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:17.545+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:17.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:17 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:17.545+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.288725+0000 front 2024-08-31T07:38:50.288622+0000 (oldest deadline 2024-08-31T07:39:15.588596+0000) 2024-08-31T07:39:17.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:17 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:17.736+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:17.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:17 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:17.736+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:17.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:17 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:17.736+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:17.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:17 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:17.736+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:17.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:17 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:17.689+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:17.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:17 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:17.689+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:17.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:17 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:17.689+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:17.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:17 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:17.689+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:18.164 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:17 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:17.950+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:18.164 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:17 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:17.950+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:18.164 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:17 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:17.950+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:18.165 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:17 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:17.950+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:18.165 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:18 smithi032 podman[127190]: 2024-08-31 07:39:18.163404993 +0000 UTC m=+6.334977353 container cleanup 09f9fbbd1e9a033d117e192a4e5319176d873178e8619d593c7efbac4babc1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6, ceph=True, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-31T07:39:18.165 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:18 smithi032 bash[127190]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6 2024-08-31T07:39:18.608 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:18 smithi032 podman[127208]: 2024-08-31 07:39:18.378307142 +0000 UTC m=+0.827771809 container remove 09f9fbbd1e9a033d117e192a4e5319176d873178e8619d593c7efbac4babc1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-08-31T07:39:18.693 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:18 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:18.284+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:18.693 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:18 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:18.284+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:18.693 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:18 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:18.284+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:18.788 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:18 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:18.533+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:18.788 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:18 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:18.533+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:18.788 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:18 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:18.533+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.288725+0000 front 2024-08-31T07:38:50.288622+0000 (oldest deadline 2024-08-31T07:39:15.588596+0000) 2024-08-31T07:39:18.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:18 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:18.692+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:18.992 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:18 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:18.692+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:18.992 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:18 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:18.692+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:18.992 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:18 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:18.692+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:19.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:18 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:18.786+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:19.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:18 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:18.786+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:19.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:18 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:18.786+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:19.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:18 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:18.786+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:19.275 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:18 smithi032 podman[127292]: 2024-08-31 07:39:18.85514528 +0000 UTC m=+0.027784754 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:39:19.275 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:19 smithi032 podman[127292]: 2024-08-31 07:39:19.012686122 +0000 UTC m=+0.185325566 container create 9b49f04213d9df75d037365841176fae028f7d71ce0305e63913246c16d56782 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1) 2024-08-31T07:39:19.275 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:19 smithi032 podman[127292]: 2024-08-31 07:39:19.273968395 +0000 UTC m=+0.446607862 container init 9b49f04213d9df75d037365841176fae028f7d71ce0305e63913246c16d56782 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:39:19.275 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:18 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:18.974+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:19.276 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:18 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:18.974+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:19.276 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:18 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:18.974+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:19.276 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:18 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:18.974+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:19.537 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:19 smithi032 podman[127292]: 2024-08-31 07:39:19.281965361 +0000 UTC m=+0.454604806 container start 9b49f04213d9df75d037365841176fae028f7d71ce0305e63913246c16d56782 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-31T07:39:19.537 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:19 smithi032 podman[127292]: 2024-08-31 07:39:19.432128706 +0000 UTC m=+0.604768168 container attach 9b49f04213d9df75d037365841176fae028f7d71ce0305e63913246c16d56782 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T07:39:19.663 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:19 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:19.262+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:19.663 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:19 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:19.262+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:19.663 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:19 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:19.262+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:19.804 INFO:journalctl@ceph.osd.6.smithi032.stdout:Aug 31 07:39:19 smithi032 podman[127316]: 2024-08-31 07:39:19.582190892 +0000 UTC m=+0.030637804 container died 9b49f04213d9df75d037365841176fae028f7d71ce0305e63913246c16d56782 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-08-31T07:39:19.830 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:19 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:19.505+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:19.830 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:19 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:19.505+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:19.830 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:19 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:19.505+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.288725+0000 front 2024-08-31T07:38:50.288622+0000 (oldest deadline 2024-08-31T07:39:15.588596+0000) 2024-08-31T07:39:19.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:19 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:19.662+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:19.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:19 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:19.662+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:19.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:19 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:19.662+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:19.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:19 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:19.662+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:20.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:19 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:19.985+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:20.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:19 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:19.985+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:20.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:19 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:19.985+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:20.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:19 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:19.985+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:20.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:19 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:19.828+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:20.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:19 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:19.828+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:20.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:19 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:19.828+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:20.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:19 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:19.829+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:20.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.6.service' 2024-08-31T07:39:20.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:39:20.481 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-31T07:39:20.481 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-31T07:39:20.481 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.7 2024-08-31T07:39:20.618 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:20 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:20.248+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:20.618 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:20 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:20.248+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:20.619 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:20 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:20.248+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:20.790 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:20 smithi073 systemd[1]: Stopping Ceph osd.7 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:39:20.791 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:20.555+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:20.791 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:20.555+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:20.791 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:20.555+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.288725+0000 front 2024-08-31T07:38:50.288622+0000 (oldest deadline 2024-08-31T07:39:15.588596+0000) 2024-08-31T07:39:20.791 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:20.789+0000 7fb14e316640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:39:20.792 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:20.789+0000 7fb14e316640 -1 osd.7 121 *** Got signal Terminated *** 2024-08-31T07:39:20.792 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:20.789+0000 7fb14e316640 -1 osd.7 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:39:20.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:20 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:20.617+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:20.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:20 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:20.617+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:20.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:20 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:20.617+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:20.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:20 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:20.617+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:21.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:20.800+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:21.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:20.800+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:21.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:20.800+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:21.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:20.800+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:21.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:20 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:20.937+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:21.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:20 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:20.937+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:21.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:20 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:20.937+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:21.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:20 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:20.937+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:21.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:21 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:21.218+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:21.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:21 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:21.218+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:21.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:21 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:21.218+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:21.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:21 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:21.592+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:21.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:21 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:21.592+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:21.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:21 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:21.592+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.288725+0000 front 2024-08-31T07:38:50.288622+0000 (oldest deadline 2024-08-31T07:39:15.588596+0000) 2024-08-31T07:39:21.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:21 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:21.776+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:21.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:21 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:21.776+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:21.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:21 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:21.776+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:21.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:21 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:21.776+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:21.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:21 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:21.612+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:21.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:21 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:21.612+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:21.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:21 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:21.612+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:21.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:21 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:21.612+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:21.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:21 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:21.612+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:22.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:21 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:21.904+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:22.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:21 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:21.904+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:22.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:21 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:21.904+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:22.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:21 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:21.904+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:22.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:22 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:22.235+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:22.490 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:22 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:22.235+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:22.491 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:22 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:22.235+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:22.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:22 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:22.624+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:22.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:22 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:22.624+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:22.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:22 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:22.624+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.288725+0000 front 2024-08-31T07:38:50.288622+0000 (oldest deadline 2024-08-31T07:39:15.588596+0000) 2024-08-31T07:39:22.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:22 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:22.624+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.889635+0000 front 2024-08-31T07:39:00.889790+0000 (oldest deadline 2024-08-31T07:39:21.989635+0000) 2024-08-31T07:39:22.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:22 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:22.822+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:22.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:22 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:22.822+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:22.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:22 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:22.822+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:22.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:22 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:22.822+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:22.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:22 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:22.602+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:22.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:22 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:22.602+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:22.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:22 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:22.602+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:22.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:22 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:22.602+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:22.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:22 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:22.602+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:23.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:22 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:22.940+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:23.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:22 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:22.940+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:23.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:22 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:22.940+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:23.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:22 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:22.940+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:23.492 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:23 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:23.217+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:23.492 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:23 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:23.217+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:23.492 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:23 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:23.217+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:23.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:23.664+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:23.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:23.664+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:23.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:23.664+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.288725+0000 front 2024-08-31T07:38:50.288622+0000 (oldest deadline 2024-08-31T07:39:15.588596+0000) 2024-08-31T07:39:23.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:23.664+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.889635+0000 front 2024-08-31T07:39:00.889790+0000 (oldest deadline 2024-08-31T07:39:21.989635+0000) 2024-08-31T07:39:23.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:23.807+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:23.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:23.807+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:23.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:23.807+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:23.931 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:23.807+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:23.931 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:23.807+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:23.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:23 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:23.626+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:23.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:23 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:23.626+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:23.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:23 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:23.626+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:23.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:23 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:23.626+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:23.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:23 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:23.627+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:24.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:23 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:23.903+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:24.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:23 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:23.903+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:24.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:23 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:23.903+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:24.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:23 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:23.903+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:24.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:23 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:23.903+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:24.630 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:24 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:24.261+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:24.630 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:24 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:24.261+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:24.630 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:24 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:24.261+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:24.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:24 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:24.636+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:24.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:24 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:24.636+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:24.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:24 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:24.636+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.288725+0000 front 2024-08-31T07:38:50.288622+0000 (oldest deadline 2024-08-31T07:39:15.588596+0000) 2024-08-31T07:39:24.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:24 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:24.636+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.889635+0000 front 2024-08-31T07:39:00.889790+0000 (oldest deadline 2024-08-31T07:39:21.989635+0000) 2024-08-31T07:39:24.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:24 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:24.808+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:24.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:24 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:24.808+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:24.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:24 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:24.808+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:24.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:24 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:24.808+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:24.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:24 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:24.808+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:24.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:24 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:24.628+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:24.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:24 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:24.628+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:24.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:24 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:24.628+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:24.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:24 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:24.628+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:24.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:24 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:24.629+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:25.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:24 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:24.901+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:25.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:24 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:24.901+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:25.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:24 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:24.901+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:25.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:24 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:24.901+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:25.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:24 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:24.901+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:25.597 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:25 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:25.276+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:25.598 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:25 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:25.276+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:25.598 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:25 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:25.276+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:25.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:25.669+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.986461+0000 front 2024-08-31T07:38:34.986611+0000 (oldest deadline 2024-08-31T07:39:00.286661+0000) 2024-08-31T07:39:25.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:25.669+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.287366+0000 front 2024-08-31T07:38:40.287216+0000 (oldest deadline 2024-08-31T07:39:06.187367+0000) 2024-08-31T07:39:25.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:25.669+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.288725+0000 front 2024-08-31T07:38:50.288622+0000 (oldest deadline 2024-08-31T07:39:15.588596+0000) 2024-08-31T07:39:25.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7[40999]: 2024-08-31T07:39:25.669+0000 7fb14a901640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.889635+0000 front 2024-08-31T07:39:00.889790+0000 (oldest deadline 2024-08-31T07:39:21.989635+0000) 2024-08-31T07:39:25.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:25.779+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:25.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:25.779+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:25.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:25.779+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:25.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:25.779+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:25.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:25 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:25.779+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:25.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:25 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:25.597+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:25.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:25 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:25.597+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:25.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:25 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:25.597+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:25.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:25 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:25.597+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:25.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:25 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:25.597+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:26.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:25 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:25.895+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:26.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:25 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:25.895+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:26.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:25 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:25.895+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:26.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:25 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:25.895+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:26.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:25 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:25.895+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:26.598 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:26 smithi073 podman[61996]: 2024-08-31 07:39:26.290229367 +0000 UTC m=+5.617634698 container died 981c6be7c5011e85a2783a16e620b973e2ecfc3f5374f3fa9b57cb481611b705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-31T07:39:26.599 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:26 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:26.286+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:26.599 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:26 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:26.286+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:26.599 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:26 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:26.286+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:26.872 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:26 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:26.598+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:26.872 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:26 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:26.598+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:26.872 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:26 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:26.598+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:26.872 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:26 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:26.598+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:26.873 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:26 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:26.598+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:26.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:26 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:26.765+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:26.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:26 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:26.765+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:26.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:26 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:26.765+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:26.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:26 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:26.765+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:26.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:26 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:26.765+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:26.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:26 smithi073 podman[61996]: 2024-08-31 07:39:26.598312402 +0000 UTC m=+5.925717731 container cleanup 981c6be7c5011e85a2783a16e620b973e2ecfc3f5374f3fa9b57cb481611b705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-31T07:39:26.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:26 smithi073 bash[61996]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7 2024-08-31T07:39:26.930 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:26 smithi073 podman[62012]: 2024-08-31 07:39:26.766626618 +0000 UTC m=+0.475060755 container remove 981c6be7c5011e85a2783a16e620b973e2ecfc3f5374f3fa9b57cb481611b705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:39:27.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:26 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:26.926+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:27.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:26 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:26.926+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:27.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:26 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:26.926+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:27.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:26 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:26.926+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:27.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:26 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:26.926+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:27.577 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:27 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:27.289+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:27.578 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:27 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:27.289+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:27.578 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:27 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:27.289+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:27.578 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:27 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:27.289+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.846443+0000 front 2024-08-31T07:39:00.846469+0000 (oldest deadline 2024-08-31T07:39:26.746550+0000) 2024-08-31T07:39:27.579 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:27 smithi073 podman[62102]: 2024-08-31 07:39:27.111433811 +0000 UTC m=+0.015727902 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:39:27.579 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:27 smithi073 podman[62102]: 2024-08-31 07:39:27.225941934 +0000 UTC m=+0.130236023 container create 57ff33684407fe56d6b9618c8ffc875ea89484a903c89a7c4da7ae2bbc0a1a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-31T07:39:27.579 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:27 smithi073 podman[62102]: 2024-08-31 07:39:27.445505735 +0000 UTC m=+0.349799824 container init 57ff33684407fe56d6b9618c8ffc875ea89484a903c89a7c4da7ae2bbc0a1a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T07:39:27.579 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:27 smithi073 podman[62102]: 2024-08-31 07:39:27.448732756 +0000 UTC m=+0.353026845 container start 57ff33684407fe56d6b9618c8ffc875ea89484a903c89a7c4da7ae2bbc0a1a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-31T07:39:27.579 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:27 smithi073 podman[62102]: 2024-08-31 07:39:27.578747441 +0000 UTC m=+0.483041532 container attach 57ff33684407fe56d6b9618c8ffc875ea89484a903c89a7c4da7ae2bbc0a1a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1) 2024-08-31T07:39:27.829 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:27 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:27.576+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:27.829 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:27 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:27.576+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:27.829 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:27 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:27.576+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:27.829 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:27 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:27.576+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:27.829 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:27 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:27.576+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:27.929 INFO:journalctl@ceph.osd.7.smithi073.stdout:Aug 31 07:39:27 smithi073 podman[62102]: 2024-08-31 07:39:27.635031314 +0000 UTC m=+0.539325404 container died 57ff33684407fe56d6b9618c8ffc875ea89484a903c89a7c4da7ae2bbc0a1a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-31T07:39:27.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:27 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:27.725+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:27.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:27 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:27.725+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:27.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:27 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:27.725+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:27.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:27 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:27.725+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:27.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:27 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:27.725+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:28.233 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.7.service' 2024-08-31T07:39:28.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:39:28.271 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-31T07:39:28.272 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-31T07:39:28.272 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.8 2024-08-31T07:39:28.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:27 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:27.913+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:28.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:27 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:27.913+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:28.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:27 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:27.913+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:28.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:27 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:27.913+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:28.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:27 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:27.913+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:28.594 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:28.303+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:28.594 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:28.303+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:28.594 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:28.303+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:28.595 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:28.303+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.846443+0000 front 2024-08-31T07:39:00.846469+0000 (oldest deadline 2024-08-31T07:39:26.746550+0000) 2024-08-31T07:39:28.595 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:28 smithi084 systemd[1]: Stopping Ceph osd.8 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:39:28.595 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:28.593+0000 7fcb6123d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:39:28.595 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:28.593+0000 7fcb6123d640 -1 osd.8 121 *** Got signal Terminated *** 2024-08-31T07:39:28.595 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:28.593+0000 7fcb6123d640 -1 osd.8 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:39:28.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:28.598+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:28.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:28.598+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:28.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:28.598+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:28.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:28.598+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:28.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:28 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:28.598+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:29.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:28 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:28.756+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:29.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:28 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:28.756+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:29.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:28 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:28.756+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:29.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:28 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:28.756+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:29.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:28 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:28.756+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:29.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:28 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:28.906+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:29.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:28 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:28.906+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:29.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:28 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:28.906+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:29.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:28 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:28.906+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:29.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:28 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:28.906+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:29.616 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:29 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:29.336+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:29.616 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:29 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:29.336+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:29.616 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:29 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:29.336+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:29.616 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:29 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:29.336+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.846443+0000 front 2024-08-31T07:39:00.846469+0000 (oldest deadline 2024-08-31T07:39:26.746550+0000) 2024-08-31T07:39:29.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:29 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:29.615+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:29.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:29 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:29.615+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:29.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:29 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:29.615+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:29.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:29 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:29.615+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:29.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:29 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:29.615+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:30.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:29 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:29.719+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:30.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:29 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:29.719+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:30.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:29 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:29.719+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:30.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:29 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:29.719+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:30.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:29 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:29.719+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:30.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:29 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:29.898+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:30.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:29 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:29.898+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:30.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:29 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:29.898+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:30.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:29 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:29.898+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:30.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:29 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:29.898+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:30.740 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:30.380+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:30.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:30.380+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:30.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:30.380+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:30.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:30.380+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.846443+0000 front 2024-08-31T07:39:00.846469+0000 (oldest deadline 2024-08-31T07:39:26.746550+0000) 2024-08-31T07:39:30.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:30.380+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.746948+0000 front 2024-08-31T07:39:06.746993+0000 (oldest deadline 2024-08-31T07:39:30.247039+0000) 2024-08-31T07:39:30.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:30.579+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:30.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:30.579+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:30.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:30.579+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:30.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:30.579+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:30.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:30.579+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:30.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:30 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:30.579+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:31.081 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:30 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:30.715+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:31.081 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:30 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:30.715+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:31.081 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:30 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:30.715+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:31.081 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:30 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:30.715+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:31.082 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:30 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:30.715+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:31.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:30 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:30.894+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:31.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:30 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:30.894+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:31.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:30 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:30.894+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:31.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:30 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:30.894+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:31.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:30 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:30.894+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:31.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:30 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:30.894+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:31.740 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:31.403+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:31.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:31.403+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:31.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:31.403+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:31.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:31.403+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.846443+0000 front 2024-08-31T07:39:00.846469+0000 (oldest deadline 2024-08-31T07:39:26.746550+0000) 2024-08-31T07:39:31.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:31.403+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.746948+0000 front 2024-08-31T07:39:06.746993+0000 (oldest deadline 2024-08-31T07:39:30.247039+0000) 2024-08-31T07:39:31.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:31.595+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:31.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:31.595+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:31.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:31.595+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:31.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:31.595+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:31.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:31.595+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:31.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:31 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:31.595+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:32.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:31 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:31.723+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:32.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:31 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:31.723+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:32.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:31 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:31.723+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:32.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:31 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:31.723+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:32.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:31 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:31.723+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:32.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:31 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:31.884+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:32.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:31 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:31.884+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:32.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:31 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:31.884+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:32.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:31 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:31.884+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:32.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:31 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:31.884+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:32.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:31 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:31.884+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:32.740 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:32.409+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:32.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:32.409+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:32.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:32.409+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:32.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:32.409+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.846443+0000 front 2024-08-31T07:39:00.846469+0000 (oldest deadline 2024-08-31T07:39:26.746550+0000) 2024-08-31T07:39:32.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:32.409+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.746948+0000 front 2024-08-31T07:39:06.746993+0000 (oldest deadline 2024-08-31T07:39:30.247039+0000) 2024-08-31T07:39:32.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:32.584+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:32.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:32.584+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:32.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:32.584+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:32.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:32.584+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:32.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:32.584+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:32.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:32 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:32.584+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:33.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:32 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:32.744+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:33.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:32 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:32.744+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:33.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:32 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:32.744+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:33.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:32 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:32.744+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:33.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:32 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:32.744+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:33.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:32 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:32.861+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:33.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:32 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:32.861+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:33.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:32 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:32.861+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:33.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:32 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:32.861+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:33.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:32 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:32.861+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:33.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:32 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:32.861+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:33.740 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:33.366+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.139783+0000 front 2024-08-31T07:38:24.139637+0000 (oldest deadline 2024-08-31T07:38:48.140187+0000) 2024-08-31T07:39:33.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:33.366+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.442940+0000 front 2024-08-31T07:38:40.442904+0000 (oldest deadline 2024-08-31T07:39:09.243739+0000) 2024-08-31T07:39:33.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:33.366+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:50.845290+0000 front 2024-08-31T07:38:50.845169+0000 (oldest deadline 2024-08-31T07:39:14.945312+0000) 2024-08-31T07:39:33.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:33.366+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.846443+0000 front 2024-08-31T07:39:00.846469+0000 (oldest deadline 2024-08-31T07:39:26.746550+0000) 2024-08-31T07:39:33.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8[42029]: 2024-08-31T07:39:33.366+0000 7fcb5d828640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.746948+0000 front 2024-08-31T07:39:06.746993+0000 (oldest deadline 2024-08-31T07:39:30.247039+0000) 2024-08-31T07:39:33.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:33.585+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:33.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:33.585+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:33.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:33.585+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:33.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:33.585+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:33.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:33.585+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:33.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:33 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:33.585+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:34.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:33 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:33.710+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:34.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:33 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:33.710+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:34.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:33 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:33.710+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:34.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:33 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:33.710+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:34.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:33 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:33.710+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:34.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:33 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:33.710+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:34.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:33 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:33.874+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:34.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:33 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:33.874+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:34.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:33 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:33.874+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:34.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:33 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:33.874+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:34.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:33 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:33.874+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:34.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:33 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:33.874+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:34.427 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:34 smithi084 podman[63570]: 2024-08-31 07:39:34.137857893 +0000 UTC m=+5.680123392 container died 239da4646f7a912eaf115b2b108e40b29159ce59f3f632d513104cad21482668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-08-31T07:39:34.727 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:34 smithi084 podman[63570]: 2024-08-31 07:39:34.525718103 +0000 UTC m=+6.067983601 container cleanup 239da4646f7a912eaf115b2b108e40b29159ce59f3f632d513104cad21482668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8, ceph=True, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-31T07:39:34.727 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:34 smithi084 bash[63570]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8 2024-08-31T07:39:34.727 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:34 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:34.570+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:34.728 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:34 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:34.570+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:34.728 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:34 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:34.570+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:34.728 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:34 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:34.570+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:34.728 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:34 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:34.570+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:34.728 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:34 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:34.570+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:34.990 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:34 smithi084 podman[63588]: 2024-08-31 07:39:34.726969226 +0000 UTC m=+0.592290064 container remove 239da4646f7a912eaf115b2b108e40b29159ce59f3f632d513104cad21482668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8, io.buildah.version=1.37.1, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T07:39:35.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:34 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:34.834+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:35.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:34 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:34.834+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:35.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:34 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:34.834+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:35.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:34 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:34.834+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:35.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:34 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:34.834+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:35.109 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:34 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:34.834+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:35.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:34 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:34.732+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:35.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:34 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:34.732+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:35.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:34 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:34.732+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:35.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:34 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:34.732+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:35.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:34 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:34.732+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:35.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:34 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:34.732+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:35.441 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:35 smithi084 podman[63678]: 2024-08-31 07:39:35.040784194 +0000 UTC m=+0.015682640 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:39:35.441 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:35 smithi084 podman[63678]: 2024-08-31 07:39:35.177847284 +0000 UTC m=+0.152745728 container create 22e29abd3b4d7cef83af9beb9a9c6a61f4a61e83fe301f5877f1336ec5abef8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-08-31T07:39:35.441 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:35 smithi084 podman[63678]: 2024-08-31 07:39:35.441007283 +0000 UTC m=+0.415905726 container init 22e29abd3b4d7cef83af9beb9a9c6a61f4a61e83fe301f5877f1336ec5abef8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-31T07:39:35.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:35 smithi084 podman[63678]: 2024-08-31 07:39:35.44458177 +0000 UTC m=+0.419480218 container start 22e29abd3b4d7cef83af9beb9a9c6a61f4a61e83fe301f5877f1336ec5abef8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T07:39:35.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:35 smithi084 podman[63678]: 2024-08-31 07:39:35.530369378 +0000 UTC m=+0.505267836 container attach 22e29abd3b4d7cef83af9beb9a9c6a61f4a61e83fe301f5877f1336ec5abef8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-31T07:39:35.741 INFO:journalctl@ceph.osd.8.smithi084.stdout:Aug 31 07:39:35 smithi084 podman[63678]: 2024-08-31 07:39:35.630367659 +0000 UTC m=+0.605266113 container died 22e29abd3b4d7cef83af9beb9a9c6a61f4a61e83fe301f5877f1336ec5abef8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-31T07:39:35.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:35 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:35.575+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:35.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:35 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:35.575+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:35.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:35 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:35.575+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:35.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:35 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:35.575+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:35.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:35 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:35.575+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:35.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:35 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:35.575+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:36.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:35 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:35.849+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:36.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:35 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:35.849+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:36.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:35 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:35.849+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:36.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:35 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:35.849+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:36.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:35 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:35.849+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:36.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:35 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:35.849+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:36.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:35 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:35.771+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:36.181 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:35 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:35.771+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:36.181 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:35 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:35.771+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:36.181 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:35 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:35.771+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:36.181 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:35 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:35.771+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:36.181 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:35 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:35.771+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:36.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.8.service' 2024-08-31T07:39:36.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:39:36.376 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-31T07:39:36.376 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-31T07:39:36.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.9 2024-08-31T07:39:36.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:36 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:36.545+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:36.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:36 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:36.545+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:36.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:36 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:36.545+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:36.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:36 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:36.545+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:36.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:36 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:36.545+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:36.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:36 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:36.545+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:36.834 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:36 smithi032 systemd[1]: Stopping Ceph osd.9 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:39:36.834 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:36 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:36.833+0000 7fd2eea00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:39:36.834 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:36 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:36.833+0000 7fd2eea00640 -1 osd.9 121 *** Got signal Terminated *** 2024-08-31T07:39:36.835 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:36 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:36.833+0000 7fd2eea00640 -1 osd.9 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:39:37.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:36 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:36.890+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:37.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:36 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:36.890+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:37.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:36 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:36.890+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:37.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:36 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:36.890+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:37.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:36 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:36.890+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:37.108 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:36 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:36.890+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:37.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:36 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:36.727+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:37.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:36 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:36.727+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:37.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:36 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:36.727+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:37.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:36 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:36.727+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:37.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:36 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:36.727+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:37.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:36 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:36.727+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:37.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:37 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:37.504+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:37.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:37 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:37.504+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:37.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:37 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:37.504+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:37.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:37 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:37.504+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:37.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:37 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:37.504+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:37.991 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:37 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:37.504+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:38.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:37 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:37.691+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:38.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:37 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:37.691+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:38.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:37 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:37.691+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:38.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:37 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:37.691+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:38.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:37 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:37.691+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:38.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:37 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:37.691+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:38.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:37 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:37.691+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:38.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:37.870+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:38.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:37.870+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:38.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:37.870+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:38.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:37.870+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:38.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:37.870+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:38.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:37 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:37.870+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:38.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:38 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:38.488+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:38.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:38 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:38.488+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:38.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:38 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:38.488+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:38.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:38 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:38.489+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:38.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:38 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:38.489+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:38.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:38 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:38.489+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:39.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:38 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:38.702+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:39.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:38 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:38.702+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:39.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:38 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:38.702+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:39.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:38 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:38.702+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:39.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:38 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:38.702+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:39.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:38 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:38.702+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:39.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:38 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:38.702+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:39.357 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:38 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:38.919+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:39.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:38 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:38.919+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:39.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:38 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:38.919+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:39.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:38 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:38.919+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:39.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:38 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:38.919+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:39.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:38 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:38.919+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:39.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:38 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:38.919+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:14.673481+0000 front 2024-08-31T07:39:14.673546+0000 (oldest deadline 2024-08-31T07:39:38.773570+0000) 2024-08-31T07:39:39.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:39.459+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:39.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:39.459+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:39.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:39.459+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:39.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:39.459+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:39.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:39.459+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:39.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:39.459+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:39.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:39 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:39.678+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:39.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:39 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:39.678+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:39.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:39 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:39.678+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:39.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:39 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:39.678+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:39.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:39 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:39.678+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:39.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:39 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:39.678+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:39.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:39 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:39.678+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:40.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:39 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:39.946+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:40.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:39 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:39.946+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:40.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:39 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:39.946+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:40.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:39 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:39.946+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:40.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:39 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:39.946+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:40.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:39 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:39.946+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:40.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:39 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:39.946+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:14.673481+0000 front 2024-08-31T07:39:14.673546+0000 (oldest deadline 2024-08-31T07:39:38.773570+0000) 2024-08-31T07:39:40.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:40 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:40.474+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:40.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:40 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:40.474+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:40.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:40 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:40.474+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:40.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:40 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:40.474+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:40.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:40 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:40.474+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:40.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:40 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:40.474+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:41.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:40 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:40.697+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:41.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:40 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:40.697+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:41.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:40 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:40.697+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:41.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:40 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:40.697+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:41.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:40 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:40.697+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:41.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:40 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:40.697+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:41.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:40 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:40.697+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:41.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:40 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:40.951+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:24.758578+0000 front 2024-08-31T07:38:24.758745+0000 (oldest deadline 2024-08-31T07:38:48.258859+0000) 2024-08-31T07:39:41.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:40 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:40.951+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.160046+0000 front 2024-08-31T07:38:34.160237+0000 (oldest deadline 2024-08-31T07:38:58.860250+0000) 2024-08-31T07:39:41.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:40 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:40.951+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:42.263258+0000 front 2024-08-31T07:38:42.263126+0000 (oldest deadline 2024-08-31T07:39:10.464075+0000) 2024-08-31T07:39:41.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:40 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:40.951+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.165739+0000 front 2024-08-31T07:38:52.165779+0000 (oldest deadline 2024-08-31T07:39:16.865927+0000) 2024-08-31T07:39:41.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:40 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:40.951+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:39:00.268358+0000 front 2024-08-31T07:39:00.268397+0000 (oldest deadline 2024-08-31T07:39:23.168694+0000) 2024-08-31T07:39:41.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:40 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:40.951+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:07.671954+0000 front 2024-08-31T07:39:07.672057+0000 (oldest deadline 2024-08-31T07:39:29.972220+0000) 2024-08-31T07:39:41.359 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:40 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9[55243]: 2024-08-31T07:39:40.951+0000 7fd2e5400640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:14.673481+0000 front 2024-08-31T07:39:14.673546+0000 (oldest deadline 2024-08-31T07:39:38.773570+0000) 2024-08-31T07:39:41.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:41 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:41.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:41.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:41 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:41.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:41.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:41 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:41.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:41.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:41 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:41.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:41.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:41 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:41.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:41.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:41 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:41.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:42.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:41 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:41.681+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:42.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:41 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:41.681+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:42.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:41 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:41.681+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:42.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:41 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:41.681+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:42.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:41 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:41.681+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:42.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:41 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:41.681+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:42.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:41 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:41.681+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:42.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:42 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:42.431+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:42.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:42 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:42.431+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:42.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:42 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:42.431+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:42.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:42 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:42.431+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:42.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:42 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:42.431+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:42.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:42 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:42.431+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:42.807 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:42 smithi032 podman[127406]: 2024-08-31 07:39:42.446157095 +0000 UTC m=+5.777944685 container died 2aa17f86b69677901ff2a109a9453e1efce542b9e573302edb239600d23d77d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-31T07:39:43.066 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:43 smithi032 podman[127406]: 2024-08-31 07:39:43.047125673 +0000 UTC m=+6.378913262 container cleanup 2aa17f86b69677901ff2a109a9453e1efce542b9e573302edb239600d23d77d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:39:43.066 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:43 smithi032 bash[127406]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9 2024-08-31T07:39:43.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:42 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:42.684+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:43.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:42 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:42.684+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:43.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:42 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:42.684+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:43.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:42 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:42.684+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:43.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:42 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:42.684+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:43.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:42 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:42.684+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:43.180 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:42 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:42.684+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:43.607 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:43 smithi032 podman[127422]: 2024-08-31 07:39:43.240271695 +0000 UTC m=+0.786993661 container remove 2aa17f86b69677901ff2a109a9453e1efce542b9e573302edb239600d23d77d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-31T07:39:43.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:43 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:43.405+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:43.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:43 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:43.405+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:43.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:43 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:43.405+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:43.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:43 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:43.405+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:43.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:43 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:43.405+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:43.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:43 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:43.405+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:43.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:43 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:43.644+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:43.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:43 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:43.644+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:43.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:43 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:43.644+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:43.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:43 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:43.644+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:43.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:43 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:43.644+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:43.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:43 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:43.644+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:43.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:43 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:43.644+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:44.098 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:43 smithi032 podman[127505]: 2024-08-31 07:39:43.725474172 +0000 UTC m=+0.027688747 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:39:44.098 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:43 smithi032 podman[127505]: 2024-08-31 07:39:43.83278402 +0000 UTC m=+0.134998573 container create 529f50371762c879ecc47e4db30fd2834cdd189930e3cf1ac944d2e2bf02e920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:39:44.098 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:44 smithi032 podman[127505]: 2024-08-31 07:39:44.098049946 +0000 UTC m=+0.400264524 container init 529f50371762c879ecc47e4db30fd2834cdd189930e3cf1ac944d2e2bf02e920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:39:44.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:44 smithi032 podman[127505]: 2024-08-31 07:39:44.105078117 +0000 UTC m=+0.407292670 container start 529f50371762c879ecc47e4db30fd2834cdd189930e3cf1ac944d2e2bf02e920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-31T07:39:44.358 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:44 smithi032 podman[127505]: 2024-08-31 07:39:44.201740396 +0000 UTC m=+0.503954958 container attach 529f50371762c879ecc47e4db30fd2834cdd189930e3cf1ac944d2e2bf02e920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux ) 2024-08-31T07:39:44.705 INFO:journalctl@ceph.osd.9.smithi032.stdout:Aug 31 07:39:44 smithi032 podman[127505]: 2024-08-31 07:39:44.374180998 +0000 UTC m=+0.676395552 container died 529f50371762c879ecc47e4db30fd2834cdd189930e3cf1ac944d2e2bf02e920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD) 2024-08-31T07:39:44.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:44 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:44.424+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:44.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:44 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:44.424+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:44.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:44 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:44.424+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:44.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:44 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:44.424+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:44.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:44 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:44.424+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:44.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:44 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:44.424+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:44.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:44 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:44.650+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:44.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:44 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:44.650+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:44.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:44 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:44.650+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:44.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:44 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:44.650+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:44.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:44 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:44.650+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:44.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:44 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:44.650+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:44.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:44 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:44.650+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:45.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.9.service' 2024-08-31T07:39:45.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:39:45.141 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-31T07:39:45.142 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-31T07:39:45.142 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.10 2024-08-31T07:39:45.429 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 systemd[1]: Stopping Ceph osd.10 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:39:45.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:45 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:45.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:45.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:45 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:45.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:45.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:45 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:45.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:45.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:45 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:45.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:45.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:45 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:45.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:45.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:45 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:45.443+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:45.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:45.460+0000 7f199bf87640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:39:45.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:45.460+0000 7f199bf87640 -1 osd.10 121 *** Got signal Terminated *** 2024-08-31T07:39:45.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:45.460+0000 7f199bf87640 -1 osd.10 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:39:45.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:45.619+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:45.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:45.619+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:45.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:45.619+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:45.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:45.619+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:45.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:45.619+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:45.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:45.619+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:45.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:45 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:45.619+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:46.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:46 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:46.447+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:46.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:46 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:46.447+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:46.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:46 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:46.447+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:46.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:46 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:46.447+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:46.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:46 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:46.447+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:46.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:46 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:46.447+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:46.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:46 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:46.661+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:46.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:46 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:46.661+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:46.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:46 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:46.661+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:46.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:46 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:46.661+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:46.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:46 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:46.661+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:46.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:46 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:46.661+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:46.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:46 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:46.661+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:47.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:47 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:47.438+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:47.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:47 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:47.438+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:47.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:47 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:47.438+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:47.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:47 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:47.438+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:47.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:47 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:47.438+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:47.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:47 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:47.438+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:47.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:47 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:47.647+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:47.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:47 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:47.647+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:47.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:47 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:47.647+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:47.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:47 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:47.647+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:47.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:47 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:47.647+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:47.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:47 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:47.647+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:47.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:47 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:47.647+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:48.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:48 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:48.394+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:48.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:48 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:48.394+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:48.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:48 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:48.394+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:48.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:48 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:48.394+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:48.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:48 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:48.394+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:48.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:48 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:48.394+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:48.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:48 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:48.608+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:48.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:48 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:48.608+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:48.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:48 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:48.608+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:48.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:48 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:48.608+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:48.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:48 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:48.608+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:48.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:48 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:48.608+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:48.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:48 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:48.608+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:49.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:49.406+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:49.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:49.406+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:49.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:49.406+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:49.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:49.406+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:49.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:49.406+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:49.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:49 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:49.406+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:49.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:49 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:49.586+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:22.582978+0000 front 2024-08-31T07:38:22.583040+0000 (oldest deadline 2024-08-31T07:38:50.783434+0000) 2024-08-31T07:39:49.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:49 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:49.586+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:34.785014+0000 front 2024-08-31T07:38:34.784975+0000 (oldest deadline 2024-08-31T07:39:00.085180+0000) 2024-08-31T07:39:49.929 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:49 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:49.586+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:40.086024+0000 front 2024-08-31T07:38:40.085885+0000 (oldest deadline 2024-08-31T07:39:07.086537+0000) 2024-08-31T07:39:49.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:49 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:49.586+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:52.288177+0000 front 2024-08-31T07:38:52.288236+0000 (oldest deadline 2024-08-31T07:39:17.588150+0000) 2024-08-31T07:39:49.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:49 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:49.586+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:57.588519+0000 front 2024-08-31T07:38:57.588494+0000 (oldest deadline 2024-08-31T07:39:23.488708+0000) 2024-08-31T07:39:49.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:49 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:49.586+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:08.190036+0000 front 2024-08-31T07:39:08.189855+0000 (oldest deadline 2024-08-31T07:39:33.489745+0000) 2024-08-31T07:39:49.930 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:49 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10[47800]: 2024-08-31T07:39:49.586+0000 7f199456a640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-08-31T07:39:13.490545+0000 front 2024-08-31T07:39:13.490634+0000 (oldest deadline 2024-08-31T07:39:37.590357+0000) 2024-08-31T07:39:50.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:50 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:50.412+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:50.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:50 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:50.412+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:50.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:50 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:50.412+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:50.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:50 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:50.412+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:50.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:50 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:50.412+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:50.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:50 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:50.412+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:50.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:50 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:50.412+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-08-31T07:39:23.213170+0000 front 2024-08-31T07:39:23.213140+0000 (oldest deadline 2024-08-31T07:39:49.613370+0000) 2024-08-31T07:39:51.277 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:50 smithi073 podman[62218]: 2024-08-31 07:39:50.970297225 +0000 UTC m=+5.630025945 container died 62b1a507befa89e68706c6876cd71c74df5cf25e27f96b7b1e75f9dfc3e6c83b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1) 2024-08-31T07:39:51.679 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:51 smithi073 podman[62218]: 2024-08-31 07:39:51.27617149 +0000 UTC m=+5.935900210 container cleanup 62b1a507befa89e68706c6876cd71c74df5cf25e27f96b7b1e75f9dfc3e6c83b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=) 2024-08-31T07:39:51.679 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:51 smithi073 bash[62218]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10 2024-08-31T07:39:51.679 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:51 smithi073 podman[62233]: 2024-08-31 07:39:51.477554618 +0000 UTC m=+0.497700146 container remove 62b1a507befa89e68706c6876cd71c74df5cf25e27f96b7b1e75f9dfc3e6c83b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-31T07:39:51.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:51 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:51.411+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:51.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:51 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:51.411+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:51.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:51 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:51.411+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:51.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:51 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:51.411+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:51.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:51 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:51.411+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:51.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:51 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:51.411+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:51.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:51 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:51.411+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-08-31T07:39:23.213170+0000 front 2024-08-31T07:39:23.213140+0000 (oldest deadline 2024-08-31T07:39:49.613370+0000) 2024-08-31T07:39:52.064 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:51 smithi073 podman[62320]: 2024-08-31 07:39:51.761683015 +0000 UTC m=+0.093048678 container create dcf253fa7f680ba6ccafabd319c5665b3b510ada82ff75608be6bbec50a6d30c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-08-31T07:39:52.064 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:51 smithi073 podman[62320]: 2024-08-31 07:39:51.684736466 +0000 UTC m=+0.016102135 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:39:52.064 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:51 smithi073 podman[62320]: 2024-08-31 07:39:51.964269898 +0000 UTC m=+0.295635562 container init dcf253fa7f680ba6ccafabd319c5665b3b510ada82ff75608be6bbec50a6d30c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-31T07:39:52.065 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:51 smithi073 podman[62320]: 2024-08-31 07:39:51.967607699 +0000 UTC m=+0.298973367 container start dcf253fa7f680ba6ccafabd319c5665b3b510ada82ff75608be6bbec50a6d30c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-31T07:39:52.065 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:52 smithi073 podman[62320]: 2024-08-31 07:39:52.064233329 +0000 UTC m=+0.395599015 container attach dcf253fa7f680ba6ccafabd319c5665b3b510ada82ff75608be6bbec50a6d30c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-31T07:39:52.429 INFO:journalctl@ceph.osd.10.smithi073.stdout:Aug 31 07:39:52 smithi073 podman[62320]: 2024-08-31 07:39:52.155593097 +0000 UTC m=+0.486958764 container died dcf253fa7f680ba6ccafabd319c5665b3b510ada82ff75608be6bbec50a6d30c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T07:39:52.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.10.service' 2024-08-31T07:39:52.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:52 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:52.437+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:52.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:52 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:52.437+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:52.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:52 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:52.437+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:52.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:52 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:52.437+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:52.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:52 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:52.437+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:52.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:52 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:52.437+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:52.742 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:52 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:52.437+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-08-31T07:39:23.213170+0000 front 2024-08-31T07:39:23.213140+0000 (oldest deadline 2024-08-31T07:39:49.613370+0000) 2024-08-31T07:39:52.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:39:52.758 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-31T07:39:52.758 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-31T07:39:52.759 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.11 2024-08-31T07:39:53.077 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:52 smithi084 systemd[1]: Stopping Ceph osd.11 for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:39:53.421 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:53.075+0000 7f579abe1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:39:53.421 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:53.075+0000 7f579abe1640 -1 osd.11 121 *** Got signal Terminated *** 2024-08-31T07:39:53.421 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:53.075+0000 7f579abe1640 -1 osd.11 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T07:39:53.422 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:53.419+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:53.422 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:53.419+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:53.422 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:53.419+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:53.422 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:53.419+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:53.422 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:53.419+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:53.422 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:53.419+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:53.423 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:53 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:53.419+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-08-31T07:39:23.213170+0000 front 2024-08-31T07:39:23.213140+0000 (oldest deadline 2024-08-31T07:39:49.613370+0000) 2024-08-31T07:39:54.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:54 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:54.469+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:54.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:54 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:54.469+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:54.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:54 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:54.469+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:54.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:54 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:54.469+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:54.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:54 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:54.469+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:54.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:54 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:54.469+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:54.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:54 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:54.469+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-08-31T07:39:23.213170+0000 front 2024-08-31T07:39:23.213140+0000 (oldest deadline 2024-08-31T07:39:49.613370+0000) 2024-08-31T07:39:55.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:55 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:55.429+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:55.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:55 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:55.429+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:55.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:55 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:55.429+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:55.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:55 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:55.429+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:55.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:55 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:55.429+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:55.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:55 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:55.429+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:55.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:55 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:55.429+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-08-31T07:39:23.213170+0000 front 2024-08-31T07:39:23.213140+0000 (oldest deadline 2024-08-31T07:39:49.613370+0000) 2024-08-31T07:39:56.642 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:56 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:56.386+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:56.642 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:56 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:56.386+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:56.642 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:56 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:56.386+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:56.642 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:56 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:56.386+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:56.642 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:56 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:56.386+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:56.642 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:56 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:56.386+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:56.643 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:56 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:56.386+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-08-31T07:39:23.213170+0000 front 2024-08-31T07:39:23.213140+0000 (oldest deadline 2024-08-31T07:39:49.613370+0000) 2024-08-31T07:39:57.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:57.362+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-08-31T07:38:23.902278+0000 front 2024-08-31T07:38:23.902130+0000 (oldest deadline 2024-08-31T07:38:48.602291+0000) 2024-08-31T07:39:57.740 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:57.362+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-08-31T07:38:33.303803+0000 front 2024-08-31T07:38:33.303752+0000 (oldest deadline 2024-08-31T07:38:57.403777+0000) 2024-08-31T07:39:57.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:57.362+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6804 osd.2 since back 2024-08-31T07:38:43.205749+0000 front 2024-08-31T07:38:43.205791+0000 (oldest deadline 2024-08-31T07:39:04.905988+0000) 2024-08-31T07:39:57.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:57.362+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-08-31T07:38:51.208572+0000 front 2024-08-31T07:38:51.208507+0000 (oldest deadline 2024-08-31T07:39:16.309312+0000) 2024-08-31T07:39:57.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:57.362+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-08-31T07:38:56.309910+0000 front 2024-08-31T07:38:56.309791+0000 (oldest deadline 2024-08-31T07:39:21.609736+0000) 2024-08-31T07:39:57.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:57.362+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6812 osd.5 since back 2024-08-31T07:39:06.810958+0000 front 2024-08-31T07:39:06.810993+0000 (oldest deadline 2024-08-31T07:39:30.311033+0000) 2024-08-31T07:39:57.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:57.362+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-08-31T07:39:23.213170+0000 front 2024-08-31T07:39:23.213140+0000 (oldest deadline 2024-08-31T07:39:49.613370+0000) 2024-08-31T07:39:57.741 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:57 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11[48826]: 2024-08-31T07:39:57.362+0000 7f57971cc640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.84:6820 osd.8 since back 2024-08-31T07:39:31.314207+0000 front 2024-08-31T07:39:31.314241+0000 (oldest deadline 2024-08-31T07:39:57.214296+0000) 2024-08-31T07:39:58.736 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:58 smithi084 podman[63781]: 2024-08-31 07:39:58.478426613 +0000 UTC m=+5.513094174 container died 69cb6cbec855965d908ec470948a93f3bbf77ad9f95114766d299c0ee8f0aec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-08-31T07:39:58.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:58 smithi084 podman[63781]: 2024-08-31 07:39:58.83410702 +0000 UTC m=+5.868774572 container cleanup 69cb6cbec855965d908ec470948a93f3bbf77ad9f95114766d299c0ee8f0aec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:39:58.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:58 smithi084 bash[63781]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11 2024-08-31T07:39:59.409 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:59 smithi084 podman[63796]: 2024-08-31 07:39:59.101999254 +0000 UTC m=+0.622971925 container remove 69cb6cbec855965d908ec470948a93f3bbf77ad9f95114766d299c0ee8f0aec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-08-31T07:39:59.696 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:59 smithi084 podman[63884]: 2024-08-31 07:39:59.311367512 +0000 UTC m=+0.015657280 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T07:39:59.747 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:59 smithi084 podman[63884]: 2024-08-31 07:39:59.410967152 +0000 UTC m=+0.115256910 container create f22c401d197826bacf173270a8456b3a2d3af7a39b43532873878ad2e3d5d40c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-31T07:39:59.747 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:59 smithi084 podman[63884]: 2024-08-31 07:39:59.629768811 +0000 UTC m=+0.334058568 container init f22c401d197826bacf173270a8456b3a2d3af7a39b43532873878ad2e3d5d40c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-31T07:39:59.747 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:59 smithi084 podman[63884]: 2024-08-31 07:39:59.632911972 +0000 UTC m=+0.337201732 container start f22c401d197826bacf173270a8456b3a2d3af7a39b43532873878ad2e3d5d40c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.1, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:39:59.747 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:59 smithi084 podman[63884]: 2024-08-31 07:39:59.696396389 +0000 UTC m=+0.400686154 container attach f22c401d197826bacf173270a8456b3a2d3af7a39b43532873878ad2e3d5d40c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-08-31T07:39:59.990 INFO:journalctl@ceph.osd.11.smithi084.stdout:Aug 31 07:39:59 smithi084 podman[63884]: 2024-08-31 07:39:59.820278131 +0000 UTC m=+0.524567891 container died f22c401d197826bacf173270a8456b3a2d3af7a39b43532873878ad2e3d5d40c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-osd-11-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-31T07:40:00.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@osd.11.service' 2024-08-31T07:40:00.525 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:00.525 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-31T07:40:00.525 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-31T07:40:00.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.a 2024-08-31T07:40:00.858 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:40:00 smithi032 systemd[1]: Stopping Ceph mds.a for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:01.357 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:40:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a[61919]: 2024-08-31T07:40:00.976+0000 7f8609600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:01.358 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:40:00 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a[61919]: 2024-08-31T07:40:00.976+0000 7f8609600640 -1 mds.a *** got signal Terminated *** 2024-08-31T07:40:03.060 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:40:02 smithi032 podman[127608]: 2024-08-31 07:40:02.756565647 +0000 UTC m=+1.940564679 container died 080c3be7d34ff010755491a9f969acf132a074746824f9d5941b71cc10ad7bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-08-31T07:40:03.358 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:40:03 smithi032 podman[127608]: 2024-08-31 07:40:03.265952326 +0000 UTC m=+2.449951357 container cleanup 080c3be7d34ff010755491a9f969acf132a074746824f9d5941b71cc10ad7bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.license=GPLv2) 2024-08-31T07:40:03.358 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:40:03 smithi032 bash[127608]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a 2024-08-31T07:40:03.739 INFO:journalctl@ceph.mds.a.smithi032.stdout:Aug 31 07:40:03 smithi032 podman[127630]: 2024-08-31 07:40:03.492433574 +0000 UTC m=+0.729009414 container remove 080c3be7d34ff010755491a9f969acf132a074746824f9d5941b71cc10ad7bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-a, io.buildah.version=1.37.1, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-31T07:40:03.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.a.service' 2024-08-31T07:40:03.834 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:03.835 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-31T07:40:03.835 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-31T07:40:03.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.d 2024-08-31T07:40:04.108 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:40:03 smithi032 systemd[1]: Stopping Ceph mds.d for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:04.607 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:40:04 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-d[60747]: 2024-08-31T07:40:04.259+0000 7f2922400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:04.608 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:40:04 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-d[60747]: 2024-08-31T07:40:04.259+0000 7f2922400640 -1 mds.d *** got signal Terminated *** 2024-08-31T07:40:08.608 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:40:08 smithi032 podman[127741]: 2024-08-31 07:40:08.220360143 +0000 UTC m=+4.112994338 container died c677467ca775a1c944396c7a77cc5a8289a4d18a2399aa8053786152788fcd63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T07:40:08.917 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:40:08 smithi032 podman[127741]: 2024-08-31 07:40:08.897616775 +0000 UTC m=+4.790250970 container cleanup c677467ca775a1c944396c7a77cc5a8289a4d18a2399aa8053786152788fcd63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-31T07:40:08.917 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:40:08 smithi032 bash[127741]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-d 2024-08-31T07:40:09.358 INFO:journalctl@ceph.mds.d.smithi032.stdout:Aug 31 07:40:09 smithi032 podman[127753]: 2024-08-31 07:40:09.099412688 +0000 UTC m=+0.863412908 container remove c677467ca775a1c944396c7a77cc5a8289a4d18a2399aa8053786152788fcd63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.1) 2024-08-31T07:40:09.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.d.service' 2024-08-31T07:40:09.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:09.465 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-31T07:40:09.466 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-31T07:40:09.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.g 2024-08-31T07:40:09.855 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:40:09 smithi032 systemd[1]: Stopping Ceph mds.g for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:10.108 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:40:09 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-g[61523]: 2024-08-31T07:40:09.854+0000 7faa5ee00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:10.108 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:40:09 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-g[61523]: 2024-08-31T07:40:09.854+0000 7faa5ee00640 -1 mds.g *** got signal Terminated *** 2024-08-31T07:40:13.608 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:40:13 smithi032 podman[127863]: 2024-08-31 07:40:13.262518863 +0000 UTC m=+3.528880259 container died 972d2b5c098a89d64f3a3d2f2e782399fadf7b6cfb5b90bc3caee0f27f1e4c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-31T07:40:14.358 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:40:13 smithi032 podman[127863]: 2024-08-31 07:40:13.976622712 +0000 UTC m=+4.242984107 container cleanup 972d2b5c098a89d64f3a3d2f2e782399fadf7b6cfb5b90bc3caee0f27f1e4c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-g, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1) 2024-08-31T07:40:14.358 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:40:13 smithi032 bash[127863]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-g 2024-08-31T07:40:14.358 INFO:journalctl@ceph.mds.g.smithi032.stdout:Aug 31 07:40:14 smithi032 podman[127874]: 2024-08-31 07:40:14.153125613 +0000 UTC m=+0.882872193 container remove 972d2b5c098a89d64f3a3d2f2e782399fadf7b6cfb5b90bc3caee0f27f1e4c6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-31T07:40:14.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.g.service' 2024-08-31T07:40:14.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:14.502 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-31T07:40:14.503 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-31T07:40:14.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.j 2024-08-31T07:40:14.858 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:40:14 smithi032 systemd[1]: Stopping Ceph mds.j for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:15.358 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:40:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j[60367]: 2024-08-31T07:40:14.893+0000 7f40f0e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:15.358 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:40:14 smithi032 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j[60367]: 2024-08-31T07:40:14.893+0000 7f40f0e00640 -1 mds.j *** got signal Terminated *** 2024-08-31T07:40:19.057 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:40:18 smithi032 podman[127980]: 2024-08-31 07:40:18.742545842 +0000 UTC m=+3.967452736 container died 85284c2b24cc997496b80bc5f22607c5f0b6e288d6b206a408cce54c3bd0c57c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-31T07:40:19.357 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:40:19 smithi032 podman[127980]: 2024-08-31 07:40:19.230522104 +0000 UTC m=+4.455428997 container cleanup 85284c2b24cc997496b80bc5f22607c5f0b6e288d6b206a408cce54c3bd0c57c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-31T07:40:19.358 INFO:journalctl@ceph.mds.j.smithi032.stdout:Aug 31 07:40:19 smithi032 bash[127980]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-j 2024-08-31T07:40:19.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.j.service' 2024-08-31T07:40:19.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:19.746 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-31T07:40:19.746 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-31T07:40:19.746 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.b 2024-08-31T07:40:20.102 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:40:19 smithi073 systemd[1]: Stopping Ceph mds.b for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:20.429 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:40:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b[53169]: 2024-08-31T07:40:20.101+0000 7faea5c13640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:20.429 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:40:20 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b[53169]: 2024-08-31T07:40:20.101+0000 7faea5c13640 -1 mds.b *** got signal Terminated *** 2024-08-31T07:40:22.929 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:40:22 smithi073 podman[62435]: 2024-08-31 07:40:22.67434366 +0000 UTC m=+2.726997883 container died 44ccc555afa0678607b339df34128c3335005fc079c1b497c550483f953f8034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-08-31T07:40:23.320 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:40:23 smithi073 podman[62435]: 2024-08-31 07:40:23.043486402 +0000 UTC m=+3.096140618 container cleanup 44ccc555afa0678607b339df34128c3335005fc079c1b497c550483f953f8034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-31T07:40:23.320 INFO:journalctl@ceph.mds.b.smithi073.stdout:Aug 31 07:40:23 smithi073 bash[62435]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-b 2024-08-31T07:40:23.468 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.b.service' 2024-08-31T07:40:23.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:23.504 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-31T07:40:23.504 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-31T07:40:23.504 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.e 2024-08-31T07:40:23.929 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:40:23 smithi073 systemd[1]: Stopping Ceph mds.e for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:23.929 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:40:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e[52560]: 2024-08-31T07:40:23.805+0000 7f3d560ec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:23.929 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:40:23 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e[52560]: 2024-08-31T07:40:23.805+0000 7f3d560ec640 -1 mds.e *** got signal Terminated *** 2024-08-31T07:40:26.990 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:40:26 smithi073 podman[62556]: 2024-08-31 07:40:26.69233174 +0000 UTC m=+2.965312622 container died ca5c809c4cb553da515871d5033bbdf07dc13f28380b6d8b1130631723e7b909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-08-31T07:40:27.248 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:40:27 smithi073 podman[62556]: 2024-08-31 07:40:27.080711988 +0000 UTC m=+3.353692912 container cleanup ca5c809c4cb553da515871d5033bbdf07dc13f28380b6d8b1130631723e7b909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e, GIT_CLEAN=True, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-31T07:40:27.248 INFO:journalctl@ceph.mds.e.smithi073.stdout:Aug 31 07:40:27 smithi073 bash[62556]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-e 2024-08-31T07:40:27.463 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.e.service' 2024-08-31T07:40:27.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:27.496 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-31T07:40:27.496 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-31T07:40:27.496 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.h 2024-08-31T07:40:27.929 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:40:27 smithi073 systemd[1]: Stopping Ceph mds.h for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:27.929 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:40:27 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-h[52859]: 2024-08-31T07:40:27.784+0000 7fd4868d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:27.929 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:40:27 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-h[52859]: 2024-08-31T07:40:27.784+0000 7fd4868d1640 -1 mds.h *** got signal Terminated *** 2024-08-31T07:40:30.929 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:40:30 smithi073 podman[62677]: 2024-08-31 07:40:30.673834277 +0000 UTC m=+2.965534822 container died 6a87ec5165bcaa34e539fdfe1d8f6a845d71bdeef43348de35ccc79e3bac2968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-31T07:40:31.402 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:40:31 smithi073 podman[62677]: 2024-08-31 07:40:31.118167697 +0000 UTC m=+3.409868239 container cleanup 6a87ec5165bcaa34e539fdfe1d8f6a845d71bdeef43348de35ccc79e3bac2968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-08-31T07:40:31.403 INFO:journalctl@ceph.mds.h.smithi073.stdout:Aug 31 07:40:31 smithi073 bash[62677]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-h 2024-08-31T07:40:31.544 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.h.service' 2024-08-31T07:40:31.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:31.576 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-31T07:40:31.576 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-31T07:40:31.576 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.k 2024-08-31T07:40:31.929 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:40:31 smithi073 systemd[1]: Stopping Ceph mds.k for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:31.929 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:40:31 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k[52246]: 2024-08-31T07:40:31.887+0000 7f15da274640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:31.930 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:40:31 smithi073 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k[52246]: 2024-08-31T07:40:31.887+0000 7f15da274640 -1 mds.k *** got signal Terminated *** 2024-08-31T07:40:34.946 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:40:34 smithi073 podman[62801]: 2024-08-31 07:40:34.691593998 +0000 UTC m=+2.889663694 container died b6c895bcc623f3e873a9251933d47964942f257e4752acd5935bdc474de51de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-31T07:40:34.947 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:40:34 smithi073 podman[62801]: 2024-08-31 07:40:34.946547505 +0000 UTC m=+3.144617243 container cleanup b6c895bcc623f3e873a9251933d47964942f257e4752acd5935bdc474de51de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.build-date=20240731) 2024-08-31T07:40:35.222 INFO:journalctl@ceph.mds.k.smithi073.stdout:Aug 31 07:40:34 smithi073 bash[62801]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-k 2024-08-31T07:40:35.341 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.k.service' 2024-08-31T07:40:35.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:35.373 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-31T07:40:35.373 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-31T07:40:35.373 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.c 2024-08-31T07:40:35.680 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:40:35 smithi084 systemd[1]: Stopping Ceph mds.c for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:35.940 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:40:35 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c[54193]: 2024-08-31T07:40:35.679+0000 7fe1fbef9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:35.941 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:40:35 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c[54193]: 2024-08-31T07:40:35.679+0000 7fe1fbef9640 -1 mds.c *** got signal Terminated *** 2024-08-31T07:40:39.089 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:40:38 smithi084 podman[63988]: 2024-08-31 07:40:38.772841357 +0000 UTC m=+3.202263409 container died 4ee2e5b1123e71e4f81ecef74dc9a885d2e449179eaed5fd417f5f10e8b8b542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-08-31T07:40:39.396 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:40:39 smithi084 podman[63988]: 2024-08-31 07:40:39.170591085 +0000 UTC m=+3.600013133 container cleanup 4ee2e5b1123e71e4f81ecef74dc9a885d2e449179eaed5fd417f5f10e8b8b542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-31T07:40:39.397 INFO:journalctl@ceph.mds.c.smithi084.stdout:Aug 31 07:40:39 smithi084 bash[63988]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-c 2024-08-31T07:40:39.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.c.service' 2024-08-31T07:40:39.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:39.574 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-31T07:40:39.575 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-31T07:40:39.575 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.f 2024-08-31T07:40:39.908 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:40:39 smithi084 systemd[1]: Stopping Ceph mds.f for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:40.240 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:40:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f[53896]: 2024-08-31T07:40:39.907+0000 7f2ac7af7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:40.240 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:40:39 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f[53896]: 2024-08-31T07:40:39.907+0000 7f2ac7af7640 -1 mds.f *** got signal Terminated *** 2024-08-31T07:40:43.107 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:40:42 smithi084 podman[64121]: 2024-08-31 07:40:42.791186666 +0000 UTC m=+2.994333771 container died e0369cd25dcbb57d0f0194cada8d1bc984157d0730f172b3fe9f5d19412e2e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0) 2024-08-31T07:40:43.108 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:40:43 smithi084 podman[64121]: 2024-08-31 07:40:43.107301069 +0000 UTC m=+3.310448164 container cleanup e0369cd25dcbb57d0f0194cada8d1bc984157d0730f172b3fe9f5d19412e2e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-31T07:40:43.490 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:40:43 smithi084 bash[64121]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f 2024-08-31T07:40:43.490 INFO:journalctl@ceph.mds.f.smithi084.stdout:Aug 31 07:40:43 smithi084 podman[64133]: 2024-08-31 07:40:43.30015202 +0000 UTC m=+0.502181880 container remove e0369cd25dcbb57d0f0194cada8d1bc984157d0730f172b3fe9f5d19412e2e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T07:40:43.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.f.service' 2024-08-31T07:40:43.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:43.574 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-31T07:40:43.574 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-31T07:40:43.575 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.i 2024-08-31T07:40:43.936 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:40:43 smithi084 systemd[1]: Stopping Ceph mds.i for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:44.240 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:40:43 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i[54504]: 2024-08-31T07:40:43.935+0000 7f34bee11640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:44.240 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:40:43 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i[54504]: 2024-08-31T07:40:43.935+0000 7f34bee11640 -1 mds.i *** got signal Terminated *** 2024-08-31T07:40:47.036 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:40:46 smithi084 podman[64239]: 2024-08-31 07:40:46.772325603 +0000 UTC m=+2.979555188 container died 6e86158acbed2772ef857f643e384d3a76bb837367c66292d98d37d7f93c7fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-31T07:40:47.429 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:40:47 smithi084 podman[64239]: 2024-08-31 07:40:47.127183147 +0000 UTC m=+3.334412732 container cleanup 6e86158acbed2772ef857f643e384d3a76bb837367c66292d98d37d7f93c7fce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T07:40:47.430 INFO:journalctl@ceph.mds.i.smithi084.stdout:Aug 31 07:40:47 smithi084 bash[64239]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-i 2024-08-31T07:40:47.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.i.service' 2024-08-31T07:40:47.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:47.595 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-31T07:40:47.595 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-31T07:40:47.595 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.l 2024-08-31T07:40:47.740 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:40:47 smithi084 systemd[1]: Stopping Ceph mds.l for 965fc21a-6766-11ef-bcd4-c7b262605968... 2024-08-31T07:40:48.240 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:40:47 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l[53595]: 2024-08-31T07:40:47.905+0000 7fcfca069640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T07:40:48.241 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:40:47 smithi084 ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l[53595]: 2024-08-31T07:40:47.905+0000 7fcfca069640 -1 mds.l *** got signal Terminated *** 2024-08-31T07:40:51.064 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:40:50 smithi084 podman[64361]: 2024-08-31 07:40:50.792145711 +0000 UTC m=+2.978083643 container died 6508136a91b3c1a6091a7db1974af867a8f6f7aca29ec67411b64a23c3f94339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-31T07:40:51.064 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:40:51 smithi084 podman[64361]: 2024-08-31 07:40:51.063762001 +0000 UTC m=+3.249699933 container cleanup 6508136a91b3c1a6091a7db1974af867a8f6f7aca29ec67411b64a23c3f94339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, io.buildah.version=1.37.1) 2024-08-31T07:40:51.442 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:40:51 smithi084 bash[64361]: ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l 2024-08-31T07:40:51.443 INFO:journalctl@ceph.mds.l.smithi084.stdout:Aug 31 07:40:51 smithi084 podman[64374]: 2024-08-31 07:40:51.248029819 +0000 UTC m=+0.450483649 container remove 6508136a91b3c1a6091a7db1974af867a8f6f7aca29ec67411b64a23c3f94339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-965fc21a-6766-11ef-bcd4-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-31T07:40:51.458 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-965fc21a-6766-11ef-bcd4-c7b262605968@mds.l.service' 2024-08-31T07:40:51.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T07:40:51.491 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-31T07:40:51.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 --force --keep-logs 2024-08-31T07:40:51.776 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:40:55.715 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 --force --keep-logs 2024-08-31T07:40:55.916 INFO:teuthology.orchestra.run.smithi073.stdout:Deleting cluster with fsid: 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:40:58.473 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 --force --keep-logs 2024-08-31T07:40:58.675 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:41:01.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T07:41:01.213 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T07:41:01.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T07:41:01.274 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-31T07:41:01.276 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448/remote/smithi032/crash 2024-08-31T07:41:01.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/crash -- . 2024-08-31T07:41:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:tar: /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/crash: Cannot open: No such file or directory 2024-08-31T07:41:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:tar: Error is not recoverable: exiting now 2024-08-31T07:41:01.355 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448/remote/smithi073/crash 2024-08-31T07:41:01.356 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/crash -- . 2024-08-31T07:41:01.396 INFO:teuthology.orchestra.run.smithi073.stderr:tar: /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/crash: Cannot open: No such file or directory 2024-08-31T07:41:01.396 INFO:teuthology.orchestra.run.smithi073.stderr:tar: Error is not recoverable: exiting now 2024-08-31T07:41:01.398 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448/remote/smithi084/crash 2024-08-31T07:41:01.398 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/crash -- . 2024-08-31T07:41:01.434 INFO:teuthology.orchestra.run.smithi084.stderr:tar: /var/lib/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/crash: Cannot open: No such file or directory 2024-08-31T07:41:01.434 INFO:teuthology.orchestra.run.smithi084.stderr:tar: Error is not recoverable: exiting now 2024-08-31T07:41:01.435 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-31T07:41:01.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-31T07:41:01.518 INFO:tasks.cephadm:Compressing logs... 2024-08-31T07:41:01.518 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-31T07:41:01.523 DEBUG:teuthology.orchestra.run.smithi073:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-31T07:41:01.525 DEBUG:teuthology.orchestra.run.smithi084:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-31T07:41:01.558 INFO:teuthology.orchestra.run.smithi084.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-31T07:41:01.558 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-volume.log 2024-08-31T07:41:01.562 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.audit.log 2024-08-31T07:41:01.562 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.i.log 2024-08-31T07:41:01.562 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mon.c.log 2024-08-31T07:41:01.562 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.cephadm.log 2024-08-31T07:41:01.563 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.11.log 2024-08-31T07:41:01.563 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.2.log 2024-08-31T07:41:01.563 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.5.log 2024-08-31T07:41:01.563 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.log 2024-08-31T07:41:01.564 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.l.log 2024-08-31T07:41:01.564 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.8.log 2024-08-31T07:41:01.564 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mgr.z.log 2024-08-31T07:41:01.564 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.c.log 2024-08-31T07:41:01.564 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.f.log 2024-08-31T07:41:01.564 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-31T07:41:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-31T07:41:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.10.log 2024-08-31T07:41:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.1.log 2024-08-31T07:41:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.7.log 2024-08-31T07:41:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-volume.log 2024-08-31T07:41:01.566 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.b.log 2024-08-31T07:41:01.569 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.k.log 2024-08-31T07:41:01.569 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.4.log 2024-08-31T07:41:01.569 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.audit.log 2024-08-31T07:41:01.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.cephadm.log 2024-08-31T07:41:01.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mgr.y.log 2024-08-31T07:41:01.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.log 2024-08-31T07:41:01.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.h.log 2024-08-31T07:41:01.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mon.b.log 2024-08-31T07:41:01.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.e.log 2024-08-31T07:41:01.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-31T07:41:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.0.log 2024-08-31T07:41:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:: No such file or directory 2024-08-31T07:41:01.573 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.9.log 2024-08-31T07:41:01.573 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-volume.log 2024-08-31T07:41:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.audit.log 2024-08-31T07:41:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.a.log 2024-08-31T07:41:01.577 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.cephadm.log 2024-08-31T07:41:01.578 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mon.a.log 2024-08-31T07:41:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.g.log 2024-08-31T07:41:01.580 INFO:teuthology.orchestra.run.smithi032.stderr: 84.5% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.cephadm.log.gz 2024-08-31T07:41:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.6.log 2024-08-31T07:41:01.581 INFO:teuthology.orchestra.run.smithi032.stderr: 90.2% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.audit.log.gz 2024-08-31T07:41:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.log 2024-08-31T07:41:01.586 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.6.log: /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.d.log 2024-08-31T07:41:01.587 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mgr.x.log 2024-08-31T07:41:01.591 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.d.log: 90.8% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-volume.log.gz 2024-08-31T07:41:01.592 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.j.log 2024-08-31T07:41:01.593 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.3.log 2024-08-31T07:41:01.597 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.j.log: /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.3.log: /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mgr.x.log: 95.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.g.log.gz 2024-08-31T07:41:01.599 INFO:teuthology.orchestra.run.smithi032.stderr: 87.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.log.gz 2024-08-31T07:41:01.600 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-31T07:41:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cephadm.log: 95.0% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.d.log.gz 2024-08-31T07:41:01.617 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 82.0% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.cephadm.log.gz 2024-08-31T07:41:01.629 INFO:teuthology.orchestra.run.smithi032.stderr: 94.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-31T07:41:01.672 INFO:teuthology.orchestra.run.smithi073.stderr: 90.3% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.audit.log.gz 2024-08-31T07:41:01.701 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 82.0% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.cephadm.log.gz 2024-08-31T07:41:01.715 INFO:teuthology.orchestra.run.smithi084.stderr: 93.3% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mgr.z.log.gz 2024-08-31T07:41:01.761 INFO:teuthology.orchestra.run.smithi084.stderr: 90.3% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.audit.log.gz 2024-08-31T07:41:01.766 INFO:teuthology.orchestra.run.smithi084.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-31T07:41:01.790 INFO:teuthology.orchestra.run.smithi073.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-31T07:41:01.798 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.log.gz 2024-08-31T07:41:01.805 INFO:teuthology.orchestra.run.smithi073.stderr: 91.0% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-volume.log.gz 2024-08-31T07:41:01.810 INFO:teuthology.orchestra.run.smithi073.stderr: 93.2% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mgr.y.log.gz 2024-08-31T07:41:01.891 INFO:teuthology.orchestra.run.smithi084.stderr: 91.0% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-volume.log.gz 2024-08-31T07:41:01.903 INFO:teuthology.orchestra.run.smithi084.stderr: 87.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph.log.gz 2024-08-31T07:41:02.242 INFO:teuthology.orchestra.run.smithi032.stderr: 90.5% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mgr.x.log.gz 2024-08-31T07:41:03.901 INFO:teuthology.orchestra.run.smithi032.stderr: 90.0% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mon.a.log.gz 2024-08-31T07:41:04.464 INFO:teuthology.orchestra.run.smithi084.stderr: 91.4% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mon.c.log.gz 2024-08-31T07:41:08.045 INFO:teuthology.orchestra.run.smithi032.stderr: 95.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.j.log.gz 2024-08-31T07:41:19.500 INFO:teuthology.orchestra.run.smithi073.stderr: 92.9% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mon.b.log.gz 2024-08-31T07:41:26.012 INFO:teuthology.orchestra.run.smithi073.stderr: 95.2% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.k.log.gz 2024-08-31T07:41:26.923 INFO:teuthology.orchestra.run.smithi032.stderr: 91.0% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.9.log.gz 2024-08-31T07:41:28.881 INFO:teuthology.orchestra.run.smithi032.stderr: 91.5% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.0.log.gz 2024-08-31T07:41:31.364 INFO:teuthology.orchestra.run.smithi084.stderr: 95.2% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.l.log.gz 2024-08-31T07:41:33.013 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.3.log.gz 2024-08-31T07:41:35.519 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.6.log.gz 2024-08-31T07:42:16.233 INFO:teuthology.orchestra.run.smithi084.stderr: 95.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.f.log.gz 2024-08-31T07:42:31.917 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.5.log.gz 2024-08-31T07:42:32.449 INFO:teuthology.orchestra.run.smithi073.stderr: 95.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.e.log.gz 2024-08-31T07:42:34.491 INFO:teuthology.orchestra.run.smithi084.stderr: 90.9% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.8.log.gz 2024-08-31T07:42:47.084 INFO:teuthology.orchestra.run.smithi032.stderr: 92.9% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.a.log.gz 2024-08-31T07:42:47.087 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-08-31T07:42:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:real 1m45.536s 2024-08-31T07:42:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:user 3m36.417s 2024-08-31T07:42:47.088 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m16.477s 2024-08-31T07:42:47.757 INFO:teuthology.orchestra.run.smithi073.stderr: 91.4% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.1.log.gz 2024-08-31T07:42:48.388 INFO:teuthology.orchestra.run.smithi073.stderr: 91.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.7.log.gz 2024-08-31T07:43:28.492 INFO:teuthology.orchestra.run.smithi084.stderr: 91.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.11.log.gz 2024-08-31T07:43:40.642 INFO:teuthology.orchestra.run.smithi084.stderr: 91.2% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.2.log.gz 2024-08-31T07:43:44.748 INFO:teuthology.orchestra.run.smithi073.stderr: 91.2% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.4.log.gz 2024-08-31T07:44:14.831 INFO:teuthology.orchestra.run.smithi073.stderr: 91.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-osd.10.log.gz 2024-08-31T07:45:02.005 INFO:teuthology.orchestra.run.smithi084.stderr: 93.0% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.i.log.gz 2024-08-31T07:45:05.534 INFO:teuthology.orchestra.run.smithi084.stderr: 93.0% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.c.log.gz 2024-08-31T07:45:05.536 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-08-31T07:45:05.537 INFO:teuthology.orchestra.run.smithi084.stderr:real 4m3.996s 2024-08-31T07:45:05.537 INFO:teuthology.orchestra.run.smithi084.stderr:user 5m1.844s 2024-08-31T07:45:05.537 INFO:teuthology.orchestra.run.smithi084.stderr:sys 0m11.296s 2024-08-31T07:45:27.629 INFO:teuthology.orchestra.run.smithi073.stderr: 93.0% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.h.log.gz 2024-08-31T07:45:58.279 INFO:teuthology.orchestra.run.smithi073.stderr: 93.1% -- replaced with /var/log/ceph/965fc21a-6766-11ef-bcd4-c7b262605968/ceph-mds.b.log.gz 2024-08-31T07:45:58.281 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-08-31T07:45:58.282 INFO:teuthology.orchestra.run.smithi073.stderr:real 4m56.740s 2024-08-31T07:45:58.282 INFO:teuthology.orchestra.run.smithi073.stderr:user 5m19.093s 2024-08-31T07:45:58.282 INFO:teuthology.orchestra.run.smithi073.stderr:sys 0m12.432s 2024-08-31T07:45:58.283 INFO:tasks.cephadm:Archiving logs... 2024-08-31T07:45:58.283 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448/remote/smithi032/log 2024-08-31T07:45:58.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-31T07:46:30.398 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448/remote/smithi073/log 2024-08-31T07:46:30.400 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-31T07:47:21.653 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448/remote/smithi084/log 2024-08-31T07:47:21.655 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-31T07:48:12.086 INFO:tasks.cephadm:Removing cluster... 2024-08-31T07:48:12.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 --force 2024-08-31T07:48:12.374 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:48:13.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 --force 2024-08-31T07:48:13.790 INFO:teuthology.orchestra.run.smithi073.stdout:Deleting cluster with fsid: 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:48:14.439 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 965fc21a-6766-11ef-bcd4-c7b262605968 --force 2024-08-31T07:48:14.642 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: 965fc21a-6766-11ef-bcd4-c7b262605968 2024-08-31T07:48:15.262 INFO:tasks.cephadm:Removing cephadm ... 2024-08-31T07:48:15.262 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-31T07:48:15.300 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-31T07:48:15.320 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-31T07:48:15.335 INFO:tasks.cephadm:Teardown complete 2024-08-31T07:48:15.336 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-31T07:48:15.655 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-31T07:48:15.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-31T07:48:15.660 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-31T07:48:15.663 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-31T07:48:15.792 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-31T07:48:16.381 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-31T07:48:16.381 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T07:48:16.386 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T07:48:16.389 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T07:48:16.404 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-08-31T07:48:16.405 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-08-31T07:48:16.416 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-08-31T07:48:16.423 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T07:48:16.424 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-08-31T07:48:16.424 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 +128us[ +157us] +/- 45ms 2024-08-31T07:48:16.424 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -67us[ -38us] +/- 81ms 2024-08-31T07:48:16.424 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 -125us[ -125us] +/- 82ms 2024-08-31T07:48:16.424 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T07:48:16.426 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T07:48:16.426 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-08-31T07:48:16.426 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 +108us[ +97us] +/- 45ms 2024-08-31T07:48:16.426 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -96us[ -106us] +/- 81ms 2024-08-31T07:48:16.427 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -148us[ -158us] +/- 82ms 2024-08-31T07:48:16.427 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T07:48:16.433 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T07:48:16.433 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-08-31T07:48:16.433 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +108us[ +84us] +/- 45ms 2024-08-31T07:48:16.433 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -84us[ -84us] +/- 81ms 2024-08-31T07:48:16.433 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 -120us[ -120us] +/- 82ms 2024-08-31T07:48:16.433 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T07:48:16.434 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-31T07:48:16.674 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-31T07:48:16.675 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-31T07:48:16.793 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-31T07:48:16.928 DEBUG:teuthology.orchestra.run.smithi073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-31T07:48:17.004 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-31T07:48:17.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T07:48:17.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T07:48:17.157 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T07:48:17.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T07:48:17.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T07:48:17.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T07:48:17.219 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-31T07:48:17.230 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-31T07:48:17.254 INFO:teuthology.task.internal:Duration was 3183.494010 seconds 2024-08-31T07:48:17.254 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-31T07:48:17.287 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-31T07:48:17.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-31T07:48:17.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-31T07:48:17.294 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-31T07:48:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T07:48:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T07:48:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T07:48:17.797 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-31T07:48:17.797 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-08-31T07:48:17.798 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-31T07:48:17.848 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-08-31T07:48:17.848 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-31T07:48:17.894 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2024-08-31T07:48:17.895 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-31T07:48:17.934 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-31T07:48:17.935 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-31T07:48:17.939 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-31T07:48:17.942 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-31T07:48:18.005 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-31T07:48:18.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-31T07:48:18.009 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-31T07:48:18.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-31T07:48:20.394 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-31T07:48:20.406 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-31T07:48:20.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-31T07:48:20.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-31T07:48:20.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-31T07:48:20.540 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-31T07:48:20.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-31T07:48:20.556 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-31T07:48:20.559 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-31T07:48:20.585 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2024-08-31T07:48:20.592 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-08-31T07:48:20.624 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-08-31T07:48:20.653 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-31T07:48:20.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T07:48:20.687 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-31T07:48:20.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T07:48:20.704 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-31T07:48:20.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T07:48:20.720 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-31T07:48:20.732 INFO:teuthology.task.internal:Transferring archived files... 2024-08-31T07:48:20.734 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448/remote/smithi032 2024-08-31T07:48:20.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-31T07:48:20.855 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448/remote/smithi073 2024-08-31T07:48:20.856 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-31T07:48:20.893 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881448/remote/smithi084 2024-08-31T07:48:20.894 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-31T07:48:20.938 INFO:teuthology.task.internal:Removing archive directory... 2024-08-31T07:48:20.938 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-31T07:48:20.943 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-31T07:48:20.947 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-31T07:48:20.990 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-31T07:48:21.004 INFO:teuthology.task.internal:Not uploading archives. 2024-08-31T07:48:21.004 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-31T07:48:21.046 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-31T07:48:21.047 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-31T07:48:21.051 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-31T07:48:21.054 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-31T07:48:21.068 INFO:teuthology.orchestra.run.smithi073.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 31 07:48 /home/ubuntu/cephtest 2024-08-31T07:48:21.071 INFO:teuthology.orchestra.run.smithi084.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 31 07:48 /home/ubuntu/cephtest 2024-08-31T07:48:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: 393956 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 31 07:48 /home/ubuntu/cephtest 2024-08-31T07:48:21.085 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-31T07:48:21.098 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-31T07:48:21.161 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 3183.4940102100372 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi032 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d19e6118f7015f0047264006cb0c873528eb3ae6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b829706239c641bdb5409b5c2e7e6a41 status: fail success: false 2024-08-31T07:48:21.162 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-31T07:48:21.245 INFO:teuthology.run:FAIL